ScrapingBee CLI
Command-line client for the ScrapingBee API: scrape URLs (single or batch), crawl sites, check usage and credits, and use Google, Fast Search, Amazon, Walmart, YouTube, and ChatGPT from the terminal.
Requirements
Setup: Install (below), then authenticate (Configuration). You need a ScrapingBee API key before any command will work.
Installation
Recommended — install with uv (no virtual environment needed):
curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install scrapingbee-cli
Alternative — install with pip in a virtual environment:
pip install scrapingbee-cli
From source: clone the repo and run pip install -e . in the project root.
Configuration
You need a ScrapingBee API key:
scrapingbee auth – Validate and save the key to config (use --api-key KEY for non-interactive; --show to print config path).- Environment –
export SCRAPINGBEE_API_KEY=your_key
.env file – In the current directory or ~/.config/scrapingbee-cli/.env
Remove the stored key with scrapingbee logout. Get your API key from the ScrapingBee dashboard.
Usage
scrapingbee [command] [arguments] [options]
scrapingbee --help – List all commands.scrapingbee [command] --help – Options and parameters for that command.
Options are per-command. Each command has its own set of options — run scrapingbee [command] --help to see them. Common options across batch-capable commands include --output-file, --output-dir, --input-file, --input-column, --concurrency, --output-format, --overwrite, --retries, --backoff, --resume, --update-csv, --no-progress, --extract-field, --fields, --smart-extract, --deduplicate, --sample, --post-process, --on-complete, --scraping-config, and --verbose. For details, see the documentation.
Parameter values: Choice parameters accept both hyphens and underscores interchangeably (e.g. --sort-by price-low and --sort-by price_low both work).
Commands
| Command | Description |
|---|
usage | Check credits and max concurrency |
auth / logout | Save or remove API key |
docs | Print docs URL; --open to open in browser |
scrape [url] | Scrape a URL (HTML, JS, screenshot, extract) |
crawl | Crawl sites following links, with AI extraction and save-pattern filtering |
google / fast-search | Search SERP APIs |
amazon-product / amazon-search | Amazon product and search |
walmart-search / walmart-product | Walmart search and product |
youtube-search / youtube-metadata | YouTube search and video metadata |
chatgpt | ChatGPT API (--search true for web-enhanced responses) |
export | Merge batch/crawl output to ndjson, txt, or csv (with --flatten, --columns) |
schedule | Schedule commands via cron (--name, --list, --stop) |
tutorial | Interactive step-by-step guide to CLI features (--chapter N, --reset, --list, --output-dir) |
Batch mode: Commands that take a single input support --input-file (one line per input, or .csv with --input-column) and --output-dir. Use --output-format csv or --output-format ndjson to stream all results to a single file (or stdout) instead of individual files. Add --deduplicate to remove duplicate URLs, --sample N to test on a subset, or --post-process 'jq .title' to transform each result. Use --resume to skip already-completed items after interruption. Run bare scrapingbee --resume to discover incomplete batches in the current directory.
Parameters and options: Use space-separated values (e.g. --render-js false), not --option=value. For full parameter lists, response formats, and credit costs, see scrapingbee [command] --help and the ScrapingBee API documentation.
Key features
