paper-search

paper-search

Search arxiv's q-fin.* categories and download PDFs into a shared ./papers/<arxiv-id>/ artifact tree.

When to use

• User asks for recent research in a q-fin subcategory ("cross-sectional momentum papers from last 6 months in q-fin.PM")
• User provides one or more arxiv ids and wants the PDFs + metadata fetched locally
• User wants to build a reading queue filtered by date, keyword, or category
• User says: search arxiv, find papers, download paper , pull this paper

Runtime

Requires Python ≥ 3.9. No third-party dependencies — uses the standard library only.

Artifact tree

Everything lands at ./papers/<arxiv-id>/ relative to the current working directory:

papers/
└── 2403.12345/
    ├── paper.pdf
    └── meta.json

meta.json schema:

{
  "arxiv_id": "2403.12345",
  "title": "...",
  "authors": ["Last, First", "..."],
  "categories": ["q-fin.PM", "q-fin.TR"],
  "published": "2024-03-18",
  "pdf_url": "http://arxiv.org/pdf/2403.12345.pdf",
  "abstract": "..."
}

Scope

• q-fin.CP (computational finance), q-fin.EC (economics), q-fin.GN (general), q-fin.MF (mathematical finance), q-fin.PM (portfolio management), q-fin.PR (pricing), q-fin.RM (risk management), q-fin.ST (statistical finance), q-fin.TR (trading)

Behavior rules

• Dedup: if papers/<arxiv-id>/paper.pdf already exists, skip download and reuse existing meta.json.
• Throttle: respect arxiv's rate guidance — 3 seconds between API requests. The bundled utils enforce this.
• No guessing: if the arxiv API returns no results, surface that to the user; do not fabricate ids or metadata.

How to use the bundled scripts

Run scripts from this skill's directory.

Search by keyword / category / date range

python scripts/search_arxiv.py --query "cross-sectional momentum" \
    --category q-fin.PM --since 2024-10-01 --max 10

Prints a JSON list of match records (arxiv_id, title, authors, categories, published, pdf_url, abstract). Does not download PDFs.

Flags:

• --query "<text>" — full-text keyword (optional)
• --category q-fin.PM — repeatable; defaults to all q-fin.*
• --since YYYY-MM-DD / --until YYYY-MM-DD — date filters on published
• --max N — cap results (default 20)

Download one or more papers by id

python scripts/download_paper.py 2403.12345 2401.98765

For each id:

1. Creates ./papers/<arxiv-id>/ (CWD-relative).
2. Writes meta.json.
3. Downloads paper.pdf unless already present.
4. Throttles 3s between arxiv requests.

Prints a short per-paper status line.

Search-then-download in one go

Pipe ids from search into download:

python scripts/search_arxiv.py --query "..." --max 5 --ids-only | \
    xargs python scripts/download_paper.py

Non-goals

• Does not parse the PDF — that's paper-extract.
• Does not handle paywalled or non-arxiv sources.
• Does not silently widen scope outside q-fin unless the user passes a non-q-fin category explicitly.