paper-replicate

paper-replicate

Independent replication of a quant paper in three explicit phases: A) plan, B) backtest, C) benchmark.

When to use

• papers/<arxiv-id>/metrics.json and note.md exist (from paper-extract) and the user wants to reproduce the strategy end-to-end.
• The user supplies their own metrics.json stub and wants to kick off replication without searching/extracting first.

Runtime

Requires Python ≥ 3.9 and the plugin's root requirements.txt installed (pandas, numpy, matplotlib, yfinance).

Phase A — Plan (STOP for user confirmation)

1. Read papers/<arxiv-id>/note.md and metrics.json.
2. Copy references/plan_template.md into papers/<arxiv-id>/plan.md and fill in:
   • Data source: ticker list, start/end, frequency, vendor. Default vendor: yfinance (daily). State this explicitly even if using the default.
   • Signal logic: pseudocode, one line per step, each line annotated with the equation label from metrics.json.core_formulas (e.g. # from Eq. 3).
   • Portfolio construction: weighting scheme, normalization, rebalance cadence.
   • Assumptions being filled in: every implicit choice the paper left unstated (signal lag, dividend handling, transaction cost flag, universe filters).
3. Halt. Surface plan.md to the user and ask for approval before writing any backtest code. Do not proceed without an explicit "go" — the source principle is: "a plan without confirmation is just a guess".

Use an existing template from templates/ if the paper clearly matches one:

• templates/tsmom.py — time-series momentum (Moskowitz/Ooi/Pedersen style)
• templates/csmom.py — cross-sectional momentum (Jegadeesh/Titman style)
• templates/risk_parity.py — inverse-volatility weighting
• templates/trend_vol.py — trend filter × inverse-vol sizing

Phase B — Backtest (after user approves plan.md)

Write three small files into papers/<arxiv-id>/code/ (NOT into this skill's directory):

• signal.py — pure signal generation from price frame.
• portfolio.py — positions / weights.
• backtest.py — P&L, equity curve, drawdown.

Use the bundled scripts to fetch data and run the backtest:

# fetch + cache daily bars under papers/<id>/data/
python scripts/fetch_data.py <arxiv-id> --tickers "SPY,IEF,GLD,DBC" --start 2005-01-01

# run backtest (expects papers/<id>/code/{signal,portfolio,backtest}.py)
python scripts/run_backtest.py <arxiv-id>

# build nav.png + drawdown.png
python scripts/plot_nav.py <arxiv-id>

run_backtest.py imports the user-written backtest.py by path and expects it to expose a run(price_frame) -> dict returning:

{
  "equity": pd.Series,       # daily NAV, indexed by date
  "returns": pd.Series,      # daily returns used to compute stats
  "weights": pd.DataFrame,   # optional, for diagnostics
  "trades": pd.DataFrame,    # optional; written to trades.csv if present
}

It computes Sharpe (annualized from daily using √252), max drawdown, annual return (CAGR), volatility (annualized), Calmar, win rate — writes metrics.json (replicated) into papers/<id>/.

Dependencies: pandas, numpy, matplotlib, yfinance.

Phase C — Benchmark

python scripts/compare_metrics.py <arxiv-id>

Joins papers/<id>/metrics.json from paper-extract (paper-reported) against the replicated metrics.json produced in Phase B. Produces benchmark.md with:

• Side-by-side table: paper value | replicated value | diff | probable cause.
• Verdict: replicated | partially replicated | not replicated.
• Confidence label: low | medium | high.

compare_metrics.py normalizes units (percent vs decimal) before comparing.

Hard rules (from the source post)

• Do NOT tune parameters to make the replication match. That is overfitting, not replication. Record deltas honestly; let the benchmark show a gap.
• Do NOT silently add transaction costs, slippage, or borrow fees unless the paper specifies them. If you add them, note the assumption in plan.md and benchmark.md.
• Do NOT skip Phase A confirmation. Plan confirmation is the safety interlock for everything downstream.
• Treat "too clean" replications with suspicion — flag them in benchmark.md rather than celebrating.

Standalone entry points

You can enter at any sub-phase:

• Skip Phase A if plan.md already exists and user has approved — jump straight to Phase B.
• Skip Phase B if metrics.json (replicated) already exists from a prior run — jump to Phase C.