retaggen-powerup

RetAgGen PowerUP

RetAgGen PowerUP is an agentic RAG evaluation workflow for industrial maintenance documentation QA. It is designed for portfolio demos where retrieval quality, citation validity, safety-aware refusal behavior, and agent traces must be visible and testable.

Core Principles

1. Make Retrieval Assumptions Explicit

• Name which benchmark query a user query maps to.
• Keep expected evidence IDs stable unless the corpus changes.
• Surface retrieval misses as a metric or failure label, not as a successful answer.

2. Prefer Deterministic Evaluation

• Use deterministic BM25, dense hashing, RRF, reranking, and metrics for the demo baseline.
• Avoid hidden model calls in tests.
• Keep expected behavior testable through npm run verify.

3. Protect Safety-Critical Context

• Unsafe operator requests should be refused when evidence says the action is unsafe.
• Distinguish a correct safety refusal from an evidence failure.
• Do not optimize answers for production throughput when the retrieved safety note contradicts the request.

4. Keep The Demo Auditable

• The corpus must stay synthetic.
• Evidence cards must be citeable by visible citation IDs.
• The workbench should show answer, evidence, metrics, verdict, failure label, and trace together.

Implementation Checklist

1. Add or change corpus material in data/corpus.
2. Update chunking or benchmark IDs if headings changed.
3. Add focused tests in tests.
4. Run npm run verify.
5. Update README or LinkedIn draft if the public demo story changed.

Useful Commands

npm run verify
npm run dev:api
npm run dev