From agami
Opens the trust-layer review dashboard for the active profile's semantic model. Lists every entry needing review (Rule 1 metrics + named filters, plus Rule 2 entries below the confidence threshold) as cards with the source-signal block that produced each entry. The user replies in chat with structured commands (approve / reject / edit / threshold / done) to mark entries reviewed. Each approval writes back to the canonical YAML files in <artifacts_dir>/<profile>/ and runs the validator before promotion.
How this skill is triggered — by the user, by Claude, or both
Slash command
/agami:agami-review [threshold N.NN | done]When to use
Use when the user says 'open the review dashboard', 'review my model', 'show me what needs review', '/agami-review', 'walk through the review queue', or after agami-connect's Phase 7 summary box prompts to open the dashboard. Also use when the user replies to a previously-rendered dashboard with one of the chat back-channel commands (approve N / reject N / edit N / threshold X / approve all below X / done).
[threshold N.NN | done]The summary Claude sees in its skill listing — used to decide when to auto-load this skill
You are running the trust-layer review surface. Goal: take the items in the user's semantic model that don't meet the trust threshold (or are Rule 1 entries needing sign-off) and let the user walk through them — see *why* each was inferred, and approve / reject / edit each. Every approval is a structured edit to a YAML file under `<artifacts_dir>/<profile>/`, gated by the validator.
You are running the trust-layer review surface. Goal: take the items in the user's semantic model that don't meet the trust threshold (or are Rule 1 entries needing sign-off) and let the user walk through them — see why each was inferred, and approve / reject / edit each. Every approval is a structured edit to a YAML file under <artifacts_dir>/<profile>/, gated by the validator.
This skill orchestrates:
render_review.py.done.Trust-layer spec lives in shared/agami-osi-extensions.md (the canonical contract). Confidence formulas live in scripts/compute_confidence.py.
#N (matching the dashboard). Don't paste the full card text.Run the same plan-mode + credentials checks as agami-query-database:
ExitPlanMode. Refusal text: "I can't apply review edits in plan mode — switch to Auto or Edit Automatically mode (Shift+Tab to cycle) and re-invoke. (You can still inspect a previously-rendered dashboard at ~/.agami/review/<profile>/<ts>.html.)"<profile> and <artifacts_dir> per the standard chain (AGAMI_PROFILE env → ~/.agami/.config.active_profile → default; AGAMI_ARTIFACTS_DIR env → ~/.agami/.config.artifacts_dir → ~/agami-artifacts).<artifacts_dir>/<profile>/index.yaml doesn't exist, invoke agami-connect and stop.python3 -c 'import yaml, jsonschema'. If not, surface the install hint and stop — we won't write YAML edits without the validator gate.Resolve the active threshold:
$ARGUMENTS starts with threshold, parse the number and use it for this session, AND persist it to <artifacts_dir>/<profile>/agami.config.yaml under review.threshold.<artifacts_dir>/<profile>/agami.config.yaml → review.threshold. Default 0.7.The dashboard now has four tabs: For Review · Approved Automatically · Manually Approved · Rejected. So this phase loads EVERY entity (not just review-needing) and tags each with a tab field. The template filters by tab.
Scope filter — AGAMI_REVIEW_SCOPE=rule_1_only (added 2026-05-12 for the hybrid review order in /agami-connect Phase 4). When this env var is set, the dashboard shows ONLY Rule 1 + stale items (metrics, named filters, drift) in the For Review tab. Everything else (low-confidence joins / field descriptions — Rule 2) is excluded from the items array entirely. The Auto / Manual / Rejected tabs are unaffected — read-only inspection is always allowed. This scope is used during the post-introspect upfront review gate, where surfacing the long tail would defeat the "fast time to value" goal. Without the env var (the default), the dashboard shows everything as before.
Load every yaml under <artifacts_dir>/<profile>/ — index.yaml, every <schema>/_schema.yaml, every <schema>/<table>.yaml. Walk the structures.
For each entity (dataset, field, relationship, metric, named_filter), parse its agami extension payload (one entry per custom_extensions[] whose vendor_name=COMMON and JSON has an agami top-level key — see shared/agami-osi-extensions.md).
Skip fields with empty descriptions entirely. A field whose description is empty ("" or whitespace) has nothing for the curator to look at — do not include it in any tab. This rule applies regardless of review_state and origin:
review_state: not_applicable + origin: no_description on these fields (the canonical shape).review_state: approved + origin: introspect_heuristic. Still skip them — the empty description is the trigger, not the trust block.The field stays in the YAML (with its name and agami.type); the dashboard just doesn't surface a card. If a curator wants to add a description for such a field, they edit the YAML directly (or, in a future build, via an "Add description" action on the table-level card — not in scope here).
For the remaining entries, classify:
needs_review(entry) =
(entity_type ∈ {metric, named_filter} AND review_state != approved) # Rule 1
OR
(review_state == unreviewed AND confidence < threshold) # Rule 2
OR
(review_state == stale) # drift
tab(entry) =
"rejected" if review_state == "rejected"
"review" if needs_review(entry)
"auto" if review_state == "approved" AND (signed_off_by == "agami_introspect_v1"
OR signed_off_role == "system")
"manual" otherwise (review_state == "approved" with a human signer)
Set item.tab on each item. The template:
unreject N.No per-tab item cap. Earlier versions of this SKILL capped the For Review tab at 50 items because the old flat layout became a wall past that. The new dashboard groups by entity type with collapsible sections — 237 items split across 4 groups (Metrics / Named Filters / Joins / Field Descriptions) is navigable. The user expands the group they want to work on; the others stay collapsed.
(If a user reports the page is sluggish on a model with, say, 5000+ unreviewed field descriptions, then introduce a per-group cap with "Show more" pagination. Don't optimize speculatively.)
Number items 1, 2, 3… globally across all tabs, in a stable order — Rule 1 first (metrics, then named_filters), then Rule 2 by ascending confidence, then approved entries grouped by entity type, then rejected. The numbering corresponds to chat commands, so it must stay stable across re-renders.
Walk the same yamls and count, per category:
auto_approved.datasets — datasets with review_state: approved (typically all, since the dataset itself is mechanical)auto_approved.fields — fields with review_state: approvedauto_approved.fk_relationships — relationships with review_state: approved AND origin: fkauto_approved.field_descriptions_from_comments — fields with review_state: approved AND origin: column_commentneeds_review.inferred_relationships — relationships with review_state: unreviewed AND origin: introspect_heuristicneeds_review.low_confidence_field_descriptions — fields with review_state: unreviewed AND confidence < thresholdneeds_review.metric_proposals — metrics with review_state: unreviewedneeds_review.named_filter_proposals — named_filters with review_state: unreviewedneeds_review.stale — any entry with review_state: staleThese feed the summary card via --summary-file.
For each item, build the item object per shared/review-dashboard-template.html → ITEMS_JSON. Specifically:
signals — translate the entry's agami.signal_breakdown into a list of {ok: bool, text: string}. The text should be human-readable — not just fk_declared: false but ✗ No FK declared in DB metadata. See examples below.inferred — the SQL fragment / definition / mapping the system proposed:
<from>.<from_col> = <to>.<to_col>expression.dialects[0].expressionextra_lines — for metrics, include Definition (from agami.definition_prose) and Assumptions (from agami.assumptions). For field descriptions, include Choices (formatted from agami.choice_field).reply_hint — for Rule 1 items: approve N by [email protected] role=cfo. For Rule 2 items: approve N.rule_1 — boolean. true if entity_type ∈ {metric, named_filter}. The dashboard uses this to split the For Review tab into a primary "must-do-to-ship" section (Rule 1 + drift/stale) and a secondary "Optional" collapsed section (everything else). Pillar D, added 2026-05-12. Don't forget to set this — missing rule_1: true on a metric pushes it into the optional collapsed section and the user might miss the sign-off.Signal-text translation table (used to render the ✓/✗ list):
| signal_breakdown key | When true (✓ text) | When false (✗ text) |
|---|---|---|
fk_declared | FK declared in DB metadata | No FK declared in DB metadata |
pk_overlap | Both endpoints are primary keys | No primary key on the source side |
unique_index_match | Target column has a unique index | Target column has no unique index |
column_type_match | Column types match exactly | Column types do not match |
column_name_similarity | (number — show only if ≥ 0.7: Column-name similarity: <X>) | (omit) |
plural_pattern_match | Plural-of-table-name pattern matches | (omit) |
dba_column_comment | DBA-authored column comment present | No DBA column comment |
well_known_measure_pattern | Column name matches a known measure pattern | (omit) |
numeric_type | Source column is numeric | Source column is not numeric |
aggregate_friendly_distribution | Distribution looks aggregate-friendly | (omit) |
business_term_match | Column name matches a known business term | (omit) |
enum_like_distribution | Distinct values look enum-like | (omit) |
synonym_match | Synonym matches an already-approved entry | (omit) |
llm_inferred | LLM proposed this with no DB-side signal | (omit) |
Skip a signal entirely if its truthful rendering would be empty. Negative signals are interesting only when their absence is meaningful — column_type_match=False is a red flag; plural_pattern_match=False is just noise.
Build /tmp/agami-review-items-<ts>.json and /tmp/agami-review-summary-<ts>.json. Then:
ts=$(date +%Y%m%d-%H%M%S)
# Per-profile subdir so multi-profile users can tell renders apart and
# clean up per-profile (dev/reset-yamls.sh --clean-renders scopes to the
# named profile).
mkdir -p ~/.agami/review/"$profile"
python3 "$AGAMI_PLUGIN_ROOT/scripts/render_review.py" \
--title "Review queue · $profile · threshold $threshold" \
--threshold "$threshold" \
--model-version "$model_version" \
--items-file "/tmp/agami-review-items-$ts.json" \
--summary-file "/tmp/agami-review-summary-$ts.json" \
--out "$HOME/.agami/review/$profile/$ts.html"
rm -f "/tmp/agami-review-items-$ts.json" "/tmp/agami-review-summary-$ts.json"
Surface in chat (single block, no padding):
Review queue rendered — <N> items at threshold <X>.
~/.agami/review/<profile>/<ts>.html
The dashboard has 4 tabs (For Review · Approved Automatically · Manually
Approved · Rejected) and click-to-approve buttons on each card. Click
the actions you want, hit "Generate feedback for Claude" at the bottom,
then paste the result back here.
You can also type commands directly:
approve N (or `approve 1, 3, 7`)
approve all below 0.95
reject N
edit N
unreject N
threshold 0.5
done
Auto-open the file on EVERY render, not just the first. Earlier versions of this SKILL tried "first render only" to avoid browser-tab sprawl, but in practice that produced worse UX — users kept looking at the old stale tab. Every re-render targets a NEW timestamped file (Phase 5), so opening every time gets the user the fresh state without ambiguity. Tab sprawl is the lesser cost; tell the user once in the closing chat that the previous tab can be closed if they want.
Use the same multi-command fallback chain as agami-query-database Phase 4e.vi:
out="$HOME/.agami/review/$profile/<ts>.html"
( command -v open >/dev/null 2>&1 && open "$out" ) || \
( command -v xdg-open >/dev/null 2>&1 && xdg-open "$out" ) || \
( command -v start >/dev/null 2>&1 && start "$out" ) || \
( command -v cmd >/dev/null 2>&1 && cmd /c start "" "$out" ) || \
echo "agami: couldn't auto-open the dashboard — open manually: $out"
Treat as best-effort — never block on the open call. If it fails, the printed path is the contract.
End the turn here. Wait for the user.
The user replies with one or more commands. Commands can come from the dashboard's "Generate feedback for Claude" button (newline-separated block) or be typed directly. Same grammar either way.
approve <num-list> [by <email>] [role=<role>]
reject <num-list>
unreject <num-list>
edit <num>
edit <num> <kind>>>>\n<new text>\n<<<
approve all below <number>
threshold <number>
done
edit N <kind>>>>\n...\n<<< is the inline-edit form the dashboard generates when the user fills in the per-card Edit textarea. <kind> ∈ {description, definition_prose}:
description — write the new prose to the entry's description field (or to dataset.description for a dataset entry).definition_prose — write the new prose to the entry's agami.definition_prose field. Used for metrics and named filters. After the write, set agami.review_state to whatever the user batched it with (or, if no approve was queued for this item, leave it as-is — the edit alone does not re-trigger sign-off).Parser: see a line matching edit N <kind>>>> (case-sensitive, exact closing token <<< on its own line); read every line between as the new value. Apply via the same Edit-tool flow as Phase 4c/4d, then re-run the validator. If validator fails, revert and surface the error.
The bare edit N form (no >>>...<<< block) is the chat-side form — Claude reads the current entry, surfaces it as a fenced code block, accepts the new content conversationally, and writes back. Use this for entity types without an inline form (joins, dataset trust blocks).
Where:
<num-list> = 1 | 1, 3, 5 | 1-5 (range)<email> = anything matching \S+@\S+\.\S+<role> ∈ {cfo, cto, data_lead, engineer, analyst, other} (no system — that's auto-only)<number> = float in [0, 1]Multiple commands on one line are allowed, comma-separated. Newline-separated blocks (as the dashboard generates) are also fine:
approve 1, 3, 7 by [email protected] role=cfo
approve 2, 4
reject 5
unreject 12
done
unreject N flips a rejected entry back to unreviewed — clears signed_off_by / signed_off_at / signed_off_role. The item appears on the For Review tab on the next re-render. Used when the curator wants a second look at something they previously rejected.
Per command:
approve N — N must be a valid item number in the current queue. For Rule 1 items (metric / named_filter), the sign-off email + role are required. Source them in this order, stop at the first hit:
approve N by <email> role=<role> — use these verbatim.~/.agami/.config's reviewer_email and reviewer_role fields, if present.git config, environment, or credentials — that path produces silent inconsistency). Use this exact prompt:
To sign off the Rule 1 items in this batch, I need your email and role. Reply like:
[email protected] / data_leadValid roles:
cfo,cto,data_lead,engineer,analyst,other.I'll save these to
~/.agami/.configso I don't ask again.
Parse the user's response: split on /, trim each half. Validate the email against \S+@\S+\.\S+ and the role against the enum above. If either fails, re-prompt with the same format.
Persist on success — merge into ~/.agami/.config (preserve any existing fields like tier, host, tool_paths):
python3 - <<PY
import json, pathlib
p = pathlib.Path.home() / ".agami" / ".config"
cfg = json.loads(p.read_text()) if p.exists() else {"schema_version": 1}
cfg["reviewer_email"] = "<email>"
cfg["reviewer_role"] = "<role>"
p.write_text(json.dumps(cfg, indent=2))
PY
chmod 600 ~/.agami/.config
If the user later passes a different email/role in an explicit by/role= clause, use those for that command only — don't overwrite the stored defaults unless they say "remember this" or edit .config directly.
Refusal text only fires when sources 1, 2, and the asked response all fail: "Item #N is a metric and requires sign-off and I couldn't get an email + role. Reply: approve N by <email> role=<role> or update ~/.agami/.config."
reject N — same numbering check.
unreject N — N must be an item with review_state: rejected. If not, surface: "Item #N isn't currently rejected — no-op."
edit N — open YAML for review (see Phase 4d).
approve all below X — bulk-approve every Rule 2 item where confidence < X AND it's not Rule 1. Skip Rule 1 items. Surface what would be skipped: "Skipping 8 Rule 1 items (metrics + named filters) — those need explicit sign-off."
threshold X — change the threshold for this render. Persist to agami.config.yaml. Re-render.
done — close the session. Surface: "Closed. Run /agami-review anytime to reopen." and stop.
Multiple commands often touch the same YAML file. Group commands by yaml_path so we read each file once, apply all relevant edits, validate, then move to the next file. Don't read+write per command — too many round-trips and harder to revert atomically.
For each (yaml_path, list_of_edits) group:
Use the Read tool on <artifacts_dir>/<profile>/<yaml_path>. Parse mentally — locate the entity by its qualified ID (e.g., relationships.orders_to_customers ↔ the relationship array entry whose name == "orders_to_customers").
Each entity has a custom_extensions[] array. Find the entry with vendor_name: COMMON AND a data: JSON string whose top-level key is agami. That's the trust block.
For approve (Rule 2 item):
review_state: approved, signed_off_by: "<email>" (the user's email from credentials or default to a placeholder string the user provides), signed_off_at: "<UTC ISO>", signed_off_role: "<role>" (if provided; else omit).For approve (Rule 1 item — metric / named_filter):
Same as above, but signed_off_role is REQUIRED. If missing in the command, surface a refusal as in Phase 3a.
CHECK definition_prose BEFORE attempting any YAML write. Read the entry's current agami.definition_prose. If it's missing, empty, or whitespace-only, refuse the approve upfront with this exact message — don't write to the YAML, don't let the validator catch it after the fact (validator rollback is a worse UX than upfront refusal):
Item #N is a
<metric | named_filter>and Rule 1 requires a non-emptydefinition_prosebefore it can be approved. Replyedit Nand I'll walk you through adding the definition, then re-issue the approve.
This check happens for every Rule 1 item in the batch before the first Edit tool call. If multiple Rule 1 items in the batch fail this check, list all of them in one message: "Items #3, #7, #12 are metrics with empty definition_prose. Reply edit 3, edit 7, edit 12 to fill them in (one per turn or batch them in the dashboard), then approve. Other items in this batch will proceed." Then continue applying the non-blocked items.
For reject:
review_state: rejected. Per Hard Rule #10, set signed_off_by: null, signed_off_at: null, signed_off_role: null (rejected entries don't carry sign-off attribution; the rejecter is recorded in the curation log instead).For unreject:
review_state: unreviewed. Set signed_off_by: null, signed_off_at: null, signed_off_role: null. The entry's confidence and signal_breakdown are preserved — only the review state flips. Item reappears on the For Review tab on the next re-render. Append a unreject event to curation_log.jsonl with the curator's identity so the audit trail captures it.For edit:
description, expression, definition_prose, or choice_field.Use the Edit tool with old_string = the original JSON-string line for that entity, new_string = the updated JSON-string line. Preserve indentation (the JSON sits inside a YAML data: '...' value — the YAML quoting is what you preserve; the JSON inside the quotes is what you change).
For convenience, build the new JSON string with json.dumps(...) ordering keys consistently — agami outermost, then the existing keys in the order they appeared (so diffs stay clean).
After all edits in a single (yaml_path) group are applied, run:
python3 "$AGAMI_PLUGIN_ROOT/scripts/validate_semantic_model.py" --directory "$artifacts_dir/$profile"
git checkout <yaml_path> (the <artifacts_dir>/<profile>/.git repo is initialized by agami-connect Phase 3e). Surface the validator errors verbatim. Tell the user which command caused the failure (best-effort — usually the most recent one). Do NOT continue applying further edits.For every applied edit (approve / reject / edit), append one line to <artifacts_dir>/<profile>/curation_log.jsonl:
jq -nc \
--arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--arg actor "$user_email_or_placeholder" \
--arg action "approve" \
--arg entity_type "relationship" \
--arg entity_qname "orders_to_customers" \
--arg from_state "unreviewed" \
--arg to_state "approved" \
--argjson confidence 0.62 \
'{ts:$ts, actor:$actor, action:$action, entity_type:$entity_type,
entity_qname:$entity_qname, from_state:$from_state, to_state:$to_state,
confidence:$confidence}' \
>> "$artifacts_dir/$profile/curation_log.jsonl"
The curation log is the audit trail — append-only, chmod 600. It captures rejecter identity (which we don't keep on the entity itself, per Hard Rule #10).
After all edits in this turn are applied and validated, commit to the repo at <artifacts_dir>/<profile>/.git:
( cd "$artifacts_dir/$profile" \
&& git add -A \
&& git -c user.name="${USER_NAME:-curator}" \
-c user.email="${USER_EMAIL:-curator@local}" \
commit -q -m "review: <N> entries approved/rejected by ${USER_EMAIL:-curator}" \
) || true
Best-effort — never block on git failure. The YAML files + curation log are the source of truth.
After a batch of edits applies cleanly, always re-render the dashboard with a new timestamped filename at ~/.agami/review/<profile>/<new-ts>.html. Recompute Phase 1 from scratch (re-walk the YAMLs, re-classify into tabs, re-count the summary). The numbering shifts as approved/rejected items leave the For Review tab — that's expected.
Delete the previous timestamped file from the same profile dir before writing the new one (rm -f "$HOME/.agami/review/$profile/$prev_ts.html"). Earlier versions kept old files around so the user would "notice the refresh," but real testing showed the stale files accumulate, confuse the user about which tab is current, and clutter the directory. The auto-open of the new file is the refresh signal; the previous file is dead. Track $prev_ts in session state across re-renders so you always know which file to delete. If the user already had the old file open in a browser tab, the new auto-open opens a fresh tab — they can close the stale one (we mention that in the chat ack).
Surface BOTH the ack AND the new file path in one chat block so the user can't miss it:
✓ Applied: approved 2 (#1, #3), rejected 1 (#7). 38 items remain.
Re-rendered: ~/.agami/review/<profile>/<new-ts>.html
(Open the new file — the previous one is stale.)
The <new-ts> is a fresh date +%Y%m%d-%H%M%S timestamp. Surfacing it inline (not just internally) is mandatory: the alternative — writing to a new path without telling the user — is what produced the "page summary says 237 but chat says 187" confusion in real testing. The user's expectation is "if you say the dashboard re-rendered, give me the URL to open."
Best-effort auto-open on every re-render — same multi-command fallback chain as Phase 2 above (open → xdg-open → start → cmd /c start → echo the path). The new file gets a new tab in the user's browser; the previous tab now points at stale content and can be closed.
Then end the turn. The user replies with the next batch of commands against the NEW dashboard's numbering.
If the queue is empty after edits, surface:
✓ Review queue is empty at threshold <X>. The model is fully reviewed.
Run `/agami-review threshold 0.5` to inspect borderline entries, or `done` to close.
If the user types done, close. Don't re-render.
git checkout (assumes Phase 3e of agami-connect ran and the dir is a git repo). If .git is absent, take a backup copy of the file before edit and restore on failure.by + role from a human in the chat command. Auto-approve is reserved for the introspect step (Phase 2c of agami-connect), and even there only joins / fields / descriptions can auto-approve.stale entry remains stale until the user explicitly re-reviews it with approve N or reject N. The skill surfaces stale entries first because they're the most surprising — a previously-trusted entry that's no longer trustable.threshold X, the new value is written to <artifacts_dir>/<profile>/agami.config.yaml. Don't keep it session-only — the next time the user runs /agami-review, they get their preferred threshold.| Symptom | Action |
|---|---|
index.yaml missing | Invoke agami-connect. Stop. |
agami.config.yaml missing | Use defaults (threshold 0.7). Don't error. |
| Validator missing | Refuse to run. Tell the user pip install pyyaml jsonschema. |
| User replies with a number that's out of range | Surface: "Item #N doesn't exist — current queue has 1..K. Re-render to see numbering." |
Approve on a Rule 1 item without by/role | Refuse with the exact required form. Don't apply. |
| Validator fails after an edit | Revert via git checkout. Surface errors. Stop applying further edits in this turn. |
.git missing in profile dir | Surface: "This profile predates the trust-layer launch — initialize with cd <artifacts_dir>/<profile> && git init && git add -A && git commit -m 'baseline', then re-run." (Newer agami-connects do this automatically — Phase 3e.) |
| The user typed prose, not a command | Try to interpret intent. If unclear, surface: "I expected approve N / reject N / edit N / threshold X / done. What did you want to do?" |
npx claudepluginhub agamiai/litebi --plugin agamiSearches MemPalace before answering questions about past work, people, projects, or prior decisions. Returns verbatim stored content instead of guessing from model memory.
Guides Payload CMS config (payload.config.ts), collections, fields, hooks, access control, APIs. Debugs validation errors, security, relationships, queries, transactions, hook behavior.
Implements vector databases with Pinecone, Weaviate, Qdrant, Milvus, pgvector for semantic search, RAG, recommendations, and similarity systems. Optimizes embeddings, indexing, and hybrid search.