crw-search

crw-search — web search via SearXNG

When to use

• You have a question or topic, not a URL. Get candidate URLs first, then scrape the one that looks right.
• Step 1 in the crw ladder: most searches should end at step 2 (crw-scrape) — pick the best result URL and scrape it for full content.
• Token-heavy context? Pipe through a subprocess filter instead of dumping raw JSON. See crw-dynamic-search.
• SearXNG is self-hosted and free — no API key, no per-query billing, no usage cap. Queries never leave your infrastructure in embedded/local mode.

Quick start

CLI (binary on PATH):

crw search "rust async http client"                              # text output
crw search "site:docs.rs tokio" --json --fields title,url,snippet --limit 5
crw search "CVE-2024-1234" --category news --time-range week
crw search "climate policy 2025" --json -o .crw/results.json
crw search "rust crates" --language en --limit 20

MCP (inside an agent harness):

crw_search(query="rust async http client", limit=5, lang="en")
crw_search(query="latest CVE nginx", tbs="qdr:w", categories="news")
crw_search(query="openai pricing", scrapeOptions={"formats": ["markdown"]})

REST (drop-in for Firecrawl SDKs — just swap the base URL):

curl -X POST "$CRW_API_URL/v1/search" -H "Authorization: Bearer $CRW_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"query":"rust async http","limit":5,"lang":"en"}'

Options

Need	CLI flag	MCP / REST field
Result count	-l/--limit N (default 10)	limit (default 5)
JSON output	--json or --format json	— (always JSON)
Field projection	--fields title,url,snippet	—
Output to file	-o FILE	—
Filter by category	--category news|images|videos|general|…	categories
Language	--language en	lang
Time filter	--time-range day|week|month|year	tbs: qdr:h|qdr:d|qdr:w|qdr:m|qdr:y
Safe search	--safesearch 0|1|2	—
Custom SearXNG	--searxng-url URL / $CRW_SEARXNG_URL	—
Group by source	—	sources: ["web","news","images"]
Scrape results inline	— (use crw scrape separately)	scrapeOptions: {formats:["markdown"]}

--fields available values: title, url, description, snippet, position, score, category. snippet is an alias for description.

A note on result scores

SearXNG is a meta-search aggregator — it merges results from multiple engines (Google, Bing, DuckDuckGo, etc.) and the score field reflects internal engine weighting, not a universal relevance measure. Do not rely on score for ranking or filtering. Use position (1-based rank) or result order instead — position 1 is the most relevant result SearXNG surfaced.

Tips

• No results / 403? SearXNG needs JSON output enabled in its config. Run crw setup --local to spin up a pre-configured sidecar automatically. Public instances (searx.be, priv.au) usually block JSON with 403/429.
• --fields saves context. --json --fields title,url,snippet --limit 5 is one call; piping to jq is two. Prefer the flag.
• Inline scraping via MCP. Pass scrapeOptions: {formats: ["markdown"]} to get page content alongside search results in one round-trip. There is no --scrape CLI flag — use the MCP/REST path for this.
• Time-sensitive queries. Use --time-range week (CLI) or tbs: "qdr:w" (MCP/REST) for news, CVEs, releases, or any freshness-sensitive topic.
• After search, scrape the winner. crw search "…" returns candidates; crw scrape "<url>" gets the full content. Don't try to read content from search snippets alone.
• Write large result sets to .crw/. Never stream a 20-result JSON blob to stdout into context. Use -o .crw/results.json then jq/grep.

See also

• crw-scrape — scrape the URL you found
• crw-dynamic-search — filter output in a subprocess to save context (use this on token-heavy tasks)
• crw — hub skill with the full workflow ladder