v1.1.0 Shell-Safe UX, Position-Independent Options, Pipelines & Batch

Highlights

• Global options work anywhere — scrapingbee google "query" --verbose --output-file out.json now works. No more "must appear before the subcommand" errors.
• No shell quoting needed — --extract-field organic_results.url (dot syntax replaces [] brackets), --duration short (replaces "<4"). Every option is shell-safe.
• Pipelines without jq — chain any search → batch in one line: scrapingbee google "query" --extract-field organic_results.url > urls.txt

New Commands

• export — merge batch/crawl output to CSV, NDJSON, or TXT
• schedule — run any command on a repeating interval (--every 1h) with automatic change detection (--auto-diff)

New Global Flags

--extract-field, --fields, --diff-dir, --resume, --no-progress, --chunk-size/--chunk-overlap (RAG output), --retries/--backoff

Batch & Crawl

• Concurrency control, progress counter, resume interrupted jobs
• Change detection via --diff-dir (MD5-based, unchanged files skipped)
• Enriched manifest.json with credits_used, latency_ms, content_md5, fetched_at
• Crawl from sitemap (--from-sitemap), crawl resume, crawl manifest

Pipelines (no jq, no sed)

Pattern	Command
SERP → scrape	google QUERY --extract-field organic_results.url > urls.txt → scrape --input-file urls.txt
Amazon search → details → CSV	amazon-search QUERY --extract-field products.asin > asins.txt → amazon-product --input-file asins.txt → export --format csv
YouTube search → metadata	youtube-search QUERY --extract-field results.link > vids.txt → youtube-metadata --input-file vids.txt
Monitor for changes	scrape --input-file urls.txt --diff-dir old_run/ --output-dir new_run/

AI Agent Support

Multi-tool agent compatibility — skill and pipeline agent definitions for Claude Code, Cursor, Amp, RooCode, Windsurf, Kiro, Gemini CLI, GitHub Copilot, Augment Code, OpenCode,
Amazon Q, and Factory AI.

Testing

• 343 unit tests (help-output tests for every command and parameter)
• 182 E2E tests (0 skipped)
• CI via GitHub Actions (Python 3.10–3.13)

v.1.0.1 SKILL.md Fix

What's Changed

• Update SKILL.md by @sahilsunny in #3

Full Changelog: v1.0.0...v1.0.1

v1.0.0 Initial Release

First stable release of the ScrapingBee CLI.