ScraperAPI

Entry point for agents and developers new to ScraperAPI. Read first when integrating ScraperAPI into an agent or app, setting up the MCP server for Claude Code or other LLM tools, choosing the right ScraperAPI product, or obtaining an API key. Trigger on: "how do I use ScraperAPI", "set up ScraperAPI", "connect ScraperAPI to Claude", "which ScraperAPI product should I use", "get web data for my agent", "scrape a website in my app", "add web scraping to my project", "ScraperAPI getting started". After reading, route to the narrower skill that owns the selected path. Note: Transmits user-supplied queries, URLs, and content to ScraperAPI.

Product-usage reference for ScraperAPI's Async Jobs API — submit scraping jobs in the background and retrieve results via polling or webhook, including batch jobs up to 50,000 URLs. Consult when the user is scraping many URLs, needs non-blocking requests, or wants webhook delivery. Use when user asks: "how do I scrape 1000 URLs with ScraperAPI", "ScraperAPI async jobs", "batch scraping with ScraperAPI", "submit a scraping job and poll for results", "ScraperAPI webhook callback", "scrape URLs in the background", "ScraperAPI batchjobs endpoint". Covers single jobs, batch jobs (up to 50k URLs), webhook callbacks, all apiParams, async-exclusive parameters, binary response decoding, retention policy, and error handling. Note: Transmits user-supplied queries, URLs, and content to ScraperAPI.

Product-usage reference for the official ScraperAPI command-line tool (`sapi`, distributed as `scraperapi-cli`). Use this skill whenever the user wants to scrape, run async jobs, fetch structured data, manage crawls, check account credits, or drive DataPipeline projects from a terminal or shell script — anywhere a one-liner is more convenient than writing SDK code. Trigger on: "scrape this URL from the terminal", "use sapi to fetch X", "ScraperAPI CLI", "ScraperAPI from bash", "sapi scrape", "sapi cost", "sapi jobs", "sapi structured amazon", "pipe ScraperAPI into jq", "shell one-liner to scrape Y", "scrape from a Makefile / cron / CI", "check my ScraperAPI credits from the command line", "submit 10000 URLs as a batch from a file". Covers install, auth resolution order, every top-level command (`scrape`, `cost`, `jobs`, `structured`, `crawler`, `pipeline`, `account`, `config`, `init`), JSON / piping behaviour, pre-flight cost checks, and common shell recipes. Note: Transmits user-supplied queries, URLs, and content to ScraperAPI.

Product-usage reference for ScraperAPI's Crawler — crawl an entire site or section by following links automatically. Consult when the user needs to extract data from many pages of a site without knowing the URLs upfront. Use when user asks: "crawl an entire website with ScraperAPI", "scrape all pages on a domain", "follow links and scrape each page", "how do I use the ScraperAPI crawler API", "scrape a site map", "extract data from every product page on a site", "ScraperAPI crawler job API". Covers job creation, URL regex patterns, depth vs budget, per-page scraping parameters, status polling, webhooks, scheduling, and credit costs. Also invoke when the user is building a site-wide scraper and asks which ScraperAPI product to use. Note: Transmits user-supplied queries, URLs, and content to ScraperAPI.

Product-usage reference for ScraperAPI's DataPipeline — managed, scheduled scraping projects that run automatically and deliver results to a webhook or dashboard download. Consult when the user needs recurring scraping, has a large list of URLs/ASINs/queries to process, or wants to avoid building and maintaining their own scraping infrastructure. Use when user asks: "schedule recurring scraping with ScraperAPI", "ScraperAPI DataPipeline", "how do I run a scraping project on a schedule", "scrape 10000 ASINs automatically", "ScraperAPI managed scraping project", "set up a ScraperAPI pipeline", "deliver scraping results to a webhook automatically". Covers project types, input methods, scheduling, output delivery, the DataPipeline API, job management, and credit costs. Note: Transmits user-supplied queries, URLs, and content to ScraperAPI.