lit-digest — deep-dive on a single paper

lit-digest — deep-dive on a single paper

Purpose

Given one specific paper, produce a markdown digest the user can actually act on: TLDR, contributions, method, evidence, limitations, what-it-builds-on, how-it-was-received, what-to-read-next.

This skill is for single-paper deep-dives. If the user wants a survey of a topic or "what's been done on X", that's lit-scan.

When to invoke

Triggers:

• User pastes a paper URL or arXiv ID and asks for a summary / deep read / critique
• "What does <paper> build on?"
• "What's been said about <paper>?"
• "Read <paper> and tell me <question>"
• User runs /research-helper:lit-digest <id-or-url>

Do NOT invoke for:

• Topic queries ("what's been done on X") — that's lit-scan
• Questions about a research area broadly — also lit-scan
• "Summarize this PDF I have on disk" — there's no script for local PDFs in v0.2; just read it directly

Configuration

API keys

lit-digest reads two optional environment variables:

• OPENALEX_API_KEY — enables authenticated access to the OpenAlex API. Used as a fallback metadata + open-access PDF source.
• SEMANTIC_SCHOLAR_API_KEY — strongly recommended; reduces 429 throttling on the primary metadata source.

If a .env file is present in the current working directory (or any parent directory), unset keys are loaded from it before the script runs. Existing environment variables always win — .env is only a fallback.

Format:

OPENALEX_API_KEY=your-key-here
SEMANTIC_SCHOLAR_API_KEY=another-key
# comments and KEY="quoted values" supported

No shell interpolation. The export prefix is not supported.

Workflow

1. Confirm the identifier

The script accepts arXiv (URL/ID), DOI (URL/bare), Semantic Scholar (URL/40-char ID), ACL Anthology URLs, and OpenReview URLs. If the user gives free-text (a paper title rather than an ID), the script will run an S2 search and take the top hit — surface the resolved title to the user and ask them to confirm before going further.

arXiv version suffixes are preserved through the lookup, so 2401.12345v2 fetches that specific version rather than the latest. Omit the suffix to get the latest.

2. Decide on --pdf

Default behavior is metadata + abstract + S2 TLDR + refs + citations + recommendations. That's enough for most digests.

Pull the PDF (--pdf) when:

• The abstract is short or vague
• The user is going to deeply engage with the paper (write about it, replicate it, review it)
• The user explicitly asks for the full text

PDF extraction needs pypdf. If it's not installed, the script returns a clear message and the digest is written from abstract alone.

3. Call the script

python skills/lit-digest/scripts/digest.py \
  --paper "<id-or-url>" \
  --output research/digests/<slug>-<YYYY-MM-DD>.json

Useful knobs:

• --refs N, --citations N: caps (default 50, 30). Don't lower casually — the digest is better with more context.
• --no-recommendations: skip the S2 recommendations call (sometimes noisy).
• --no-openalex: never use OpenAlex even if OPENALEX_API_KEY is set.
• --openalex-key <key>: override OPENALEX_API_KEY env var (mostly for testing).
• --pdf, --pdf-pages N: PDF extraction.

4. Read the JSON and write the digest

Output goes to research/digests/<arxiv-id-or-slug>-<YYYY-MM-DD>.md. Structure:

# Digest: <title>

**Authors:** ...
**Venue, year:** ...
**Links:** [arXiv](...) · [S2](...) · [DOI](...)

## TLDR

Three sentences. Start from S2's `tldr` if present; rewrite for clarity.

## Key contributions

Bulleted. What does the paper *claim* is new?

## Method

One paragraph. Call out any non-standard choices.

## Evidence

What experiments, headline numbers, what's the strongest result.

## Limitations

The paper's stated limitations, plus your read of any unstated ones.

## Builds on

5-8 references, one sentence each. Bias toward `is_influential = true`.

## Reception

Top citing papers. Flag critiques and improvements explicitly.

## Read next

Top recommendations + the most interesting citations. Skim the recommendations
list for the few that look genuinely related (S2's recommendations sometimes
include adjacent-but-not-relevant work).

## Open questions

Prompts for the user to think about.

If pdf.available == false, work from abstract + TLDR alone and flag the gap.

If paper.source == "openalex", the digest can't reference S2's refs/citations/recommendations (those endpoints aren't reachable through OpenAlex). Say so explicitly and offer to do a lit-scan of the topic for context.

If OpenAlex resolved the paper, the JSON also carries a topics field (top-3 OpenAlex topics with full domain → field → subfield → topic taxonomy). Empty/absent for S2- or arXiv-resolved papers.

5. Tell the user where it lives

Don't dump the full digest in the chat. Say: "Digest at <path>. Top contribution: X. Three things to read next: A, B, C."

Failure modes

Couldn't resolve identifier — exit 1 with "could not resolve <id>". Ask the user to double-check the link/ID.

S2 rate-limited (HTTP 429) — surface a one-time stderr suggestion to set SEMANTIC_SCHOLAR_API_KEY. If the user already has one and is still throttled, suggest waiting or batching.

SEMANTIC_SCHOLAR_API_KEY unset — calls share the global anonymous pool with everyone on the internet. Works for casual use; flaky under load. Surface to the user once, don't keep nagging.

OpenAlex over budget (402) — silently skip; surface a stderr note. Don't bail the whole digest.

pdf.available == false with reason:

• "pypdf not installed" → tell the user pip install pypdf if they want full-text extraction.
• "no PDF URL available" → can't be fixed; work from abstract.
• "extraction yielded < 200 chars" → PDF was image-only; OCR is out of scope.

OpenReview title-lookup returned wrong paper — verify the resolved title matches what the user expected before writing the digest. If wrong, ask the user for the arXiv ID instead.

Empty refs/citations — flag "very new or very niche; no citation-graph signal yet" in the digest.

Setup notes

Optional env vars:

• SEMANTIC_SCHOLAR_API_KEY — strongly recommended; get one at https://www.semanticscholar.org/product/api
• OPENALEX_API_KEY — optional; enables PDF resolution for non-arXiv papers and metadata recovery for S2 404s

Optional pip:

• pypdf — needed for --pdf extraction

The skill works without any of these. Coverage degrades gracefully.

Defaults summary

Knob	Default	When to change
--refs	50	Lower (10-20) for very famous papers with thousands of refs
--citations	30	Lower (10) if you only care about the most influential responses
--pdf	off	Turn on when the abstract isn't enough or you're deeply reading
--no-recommendations	off	Turn on if recommendations look noisy
--no-openalex	off	Turn on if you don't want the fallback for any reason
Output path	research/digests/<slug>-<date>.md	Override only if user asks