agami-model

agami model

You are running the model-explorer surface. Goal: let the user see every dataset and field in the live semantic model, search for things by name or description, and remove tables or columns they don't want the runtime to use — without re-introspecting.

This skill orchestrates:

1. Render — invoke render_model_explorer.py to walk every YAML and write a self-contained HTML artifact at ~/.agami/model/<profile>/<ts>.html. The Python script does the YAML reading — no LLM tokens spent on the walk.
2. Open + wait — auto-open the file, end the turn, wait for the user to come back with exclude / include commands from the dashboard's "Generate feedback for Claude" button.
3. Apply — for each batch of commands, invoke apply_model_exclusions.py with a JSON actions file. The script edits the YAMLs, runs the validator, reverts via git checkout on failure, appends to curation_log.jsonl on success, and commits to the profile's git repo.
4. Re-render — render to a new timestamped file and re-open. Wait for the next batch.

Trust-spine semantics: "exclude" flips agami.review_state to rejected. The runtime model loader in agami-query-database Phase 1c skips rejected datasets and fields entirely — they never appear in prompts, never get joined to, never get aggregated. "Include" flips back to unreviewed; the user can re-approve via /agami-review if they want a sign-off badge.

Conversation style

• Tight loops. The dashboard is the surface; the chat is just the input channel.
• Don't restate the dashboard in chat. A successful apply gets a one-line ack with the count and the new file path.
• Qualified names everywhere. Tables are <schema>.<table>. Columns are <schema>.<table>.<column>. No bare names — they're ambiguous across schemas.

---

Phase 0: Preflight

Same shape as agami-review:

• Plan-mode check — this skill writes YAMLs. If plan mode is active, refuse: "I can't apply model edits in plan mode — switch to Auto or Edit Automatically mode (Shift+Tab to cycle) and re-invoke. (You can still inspect a previously-rendered model-explorer artifact at ~/.agami/model/<profile>/<ts>.html.)" Do NOT write a plan file. Do NOT call ExitPlanMode.
• Resolve <profile> and <artifacts_dir> via the standard chain (AGAMI_PROFILE → ~/.agami/.config.active_profile → default; AGAMI_ARTIFACTS_DIR → .config.artifacts_dir → ~/agami-artifacts).
• If <artifacts_dir>/<profile>/index.yaml doesn't exist, invoke agami-connect and stop — there's no model to explore yet.
• Verify Python + PyYAML are importable (the renderer + applier both depend on PyYAML): python3 -c 'import yaml'. If not, surface the install hint and stop.

Resolve the curator's identity for curation_log.jsonl + git commits:

1. ~/.agami/.config.reviewer_email — read once-and-persist; see agami-review/SKILL.md → Phase 3a.
2. If absent, ask once: "What's your email? I'll save it to ~/.agami/.config.reviewer_email so future review + model actions don't re-ask." Validate the shape, persist. Do NOT infer from git config / env / credentials — that path produces silent inconsistency.

---

Phase 1: Render the explorer

ts=$(date -u +%Y%m%d-%H%M%S)
# Per-profile subdir so multi-profile users can tell renders apart.
mkdir -p ~/.agami/model/"$profile"
out="$HOME/.agami/model/$profile/$ts.html"
python3 "$AGAMI_PLUGIN_ROOT/scripts/render_model_explorer.py" \
  --profile "$profile" \
  --artifacts-dir "$artifacts_dir" \
  --out "$out"

$AGAMI_PLUGIN_ROOT resolves the same way as in agami-review (the plugin dir under ~/.claude/plugins/cache/<marketplace>/agami/). If the env var isn't set, fall back to the conventional install paths described in shared/invocation-conventions.md.

Surface in chat (single block, no padding):

Model explorer rendered — <N> schemas · <M> tables · <K> fields.
~/.agami/model/<profile>/<ts>.html

The dashboard has live search, filter chips (All / Active / Excluded /
Unreviewed / Queued), and per-table + per-column Exclude / Include
buttons. Click your way through, hit "Generate feedback for Claude" at
the bottom, paste back here.

You can also type commands directly:
  exclude tables:  <schema>.<table>, <schema>.<table>
  include tables:  <schema>.<table>
  exclude columns: <schema>.<table>.<column>, <schema>.<table>.<column>
  include columns: <schema>.<table>.<column>
  done

Auto-open with the same multi-command fallback chain as agami-review:

( command -v open    >/dev/null 2>&1 && open "$out" ) || \
( command -v xdg-open >/dev/null 2>&1 && xdg-open "$out" ) || \
( command -v start    >/dev/null 2>&1 && start "$out" ) || \
( command -v cmd      >/dev/null 2>&1 && cmd /c start "" "$out" ) || \
echo "agami: couldn't auto-open the explorer — open manually: $out"

End the turn. Wait for the user.

---

Phase 2: Parse the chat back-channel

The user replies with a block like:

exclude tables: BUREAU_DATA.NOHIT_DATA, BUREAU_DATA.UNSCRUBBED_DATA
exclude columns: BUREAU_DATA.GENERAL_INFO.PAN, BUREAU_DATA.GENERAL_INFO.AADHAAR
include tables: BUREAU_DATA.LOAN_REPAYMENT
done

Grammar:

exclude tables:  <qname-list>
include tables:  <qname-list>
exclude columns: <qname-list>
include columns: <qname-list>
done

Where <qname-list> is comma-separated, whitespace-tolerant:

• Table qname: <schema>.<table> (e.g., BUREAU_DATA.NOHIT_DATA)
• Column qname: <schema>.<table>.<column> (e.g., BUREAU_DATA.GENERAL_INFO.PAN)

Tolerate trailing commas, mixed-case schema/table/column names (the YAMLs typically preserve the DB's casing — pass them through verbatim).

Reject malformed targets in chat, not by silently dropping them. If a user types exclude tables: NOHIT_DATA without the schema prefix, surface: "Tables need a schema prefix — BUREAU_DATA.NOHIT_DATA not just NOHIT_DATA. Did you mean that?" and stop.

---

Phase 3: Apply via apply_model_exclusions.py

Build the actions JSON:

cat > /tmp/agami-model-actions-$ts.json <<EOF
{
  "exclude_tables":  ["BUREAU_DATA.NOHIT_DATA", "BUREAU_DATA.UNSCRUBBED_DATA"],
  "include_tables":  ["BUREAU_DATA.LOAN_REPAYMENT"],
  "exclude_columns": ["BUREAU_DATA.GENERAL_INFO.PAN", "BUREAU_DATA.GENERAL_INFO.AADHAAR"],
  "include_columns": []
}
EOF

python3 "$AGAMI_PLUGIN_ROOT/scripts/apply_model_exclusions.py" \
  --profile "$profile" \
  --artifacts-dir "$artifacts_dir" \
  --actor "$reviewer_email" \
  --actions-file "/tmp/agami-model-actions-$ts.json"

The script's stdout is a JSON document with applied, skipped, errors, validator_ok, validator_output. Parse it.

Success case (validator_ok: true, no errors):

• Surface a one-line ack: "✓ Applied: tables excluded, columns excluded, re-included. Re-rendering…"
• If skipped[] is non-empty, list each on its own line: "⚠ Skipped: BUREAU_DATA.X.Y (column not found on dataset)" — the apply script returns these for targets it couldn't resolve.
• Continue to Phase 4 (re-render).

Failure case (validator_ok: false):

• The applier has already reverted all changes via git checkout. Surface: "⚠ Validator rejected the batch — all changes reverted. No YAML files were modified."
• Show the validator output verbatim (the contract for Phase 4 → 4d in agami-review applies here too).
• Stop. Do not re-render. The user can adjust and resubmit.

---

Phase 4: Re-render

Every successful apply produces a new explorer file at a new timestamp.

Delete the previous timestamped file from this profile's dir before writing the new one (rm -f "$HOME/.agami/model/$profile/$prev_ts.html"). Track $prev_ts across re-renders. The auto-open of the new file is the refresh signal; old files just accumulate and confuse the user about which tab is current.

new_ts=$(date -u +%Y%m%d-%H%M%S)
new_out="$HOME/.agami/model/$profile/$new_ts.html"
python3 "$AGAMI_PLUGIN_ROOT/scripts/render_model_explorer.py" \
  --profile "$profile" --artifacts-dir "$artifacts_dir" --out "$new_out"

Auto-open the new path. Surface the new file path in the chat ack:

✓ Applied: 2 tables excluded, 2 columns excluded. Re-rendered.
~/.agami/model/<profile>/<new-ts>.html
(Previous tab is stale and can be closed.)

End the turn. The user comes back with the next batch.

---

Phase 5: Closing

When the user types done AND no further commands are in the message, surface:

✓ Model exploration session ended. Final state:
  <X> tables excluded · <Y> columns excluded across the profile.

The runtime semantic model now skips these entries — they will not
appear in any prompt or join. To restore them later, run /agami-model
and queue `include tables: ...` / `include columns: ...`.

Then end the turn. The skill is one-shot per invocation — re-enter via the slash command or natural-language phrase to start another session.

---

Edge cases

Case	Behavior
User types exclude tables: schema.table for a table that's already excluded	The applier still runs (idempotent flip to rejected with the curator's name on the curation log). No-op for the YAML, but the audit trail records the re-confirm.
User types include tables: schema.table for a table that's already unreviewed	Same — idempotent flip; entry stays unreviewed; curator name recorded.
User types exclude columns: schema.table.col and the column doesn't exist	The applier returns it in skipped[] with reason "field not found on dataset". Surface the skip; continue with the rest.
User types exclude tables: schema.table.col (3 segments — looks like a column)	The grammar is positional; agami treats the exclude tables: prefix as authoritative. The applier will fail to find a table with that 3-segment name and return skipped with reason "table yaml not found". Surface clearly.
Validator fails on a partially-applied batch	The applier reverts ALL YAML changes via git checkout -- . before returning. applied counts are zeroed in the response (defense against partial-apply confusion). The user sees validator_output verbatim and can fix + retry.
User has the profile dir but no .git/ (legacy from before Phase 3e)	The applier skips the git commit + revert steps silently. Curation log still gets appended. The user loses revert-on-validator-fail safety — surface a one-liner suggesting git init in the profile dir.
The model has 5000+ fields	The renderer handles it (JSON manifest is ~hundreds of KB at the high end). Client-side search is instant up to the millions of DOM nodes mark. If it gets sluggish, file an issue.

---

What the runtime does with rejected entries

The trust spine has always read agami.review_state. The model loader in plugins/agami/skills/agami-query-database/SKILL.md → Phase 1c filters entries with review_state: rejected out of datasets_by_name, datasets_by_qname, fields_by_qname, and relationships_by_endpoints. Rejected entries:

• Never appear in the schema context the SQL generator sees (Phase 2b).
• Are not joinable — the join-path picker skips relationships whose endpoints reference a rejected dataset.
• Don't trigger the strict gate — a rejected metric or named filter is excluded from runtime; queries that would have referenced them get a "metric not found" path-not-an-error.
• Stay in the YAML for audit. The user can include them later without re-introspect.

This is the same mechanism /agami-review's Reject button uses on individual entries. /agami-model is the bulk-and-search interface for the same operation, with tables-as-units instead of metric-by-metric.

---

Examples

Excluding PII columns

You: open the model explorer

[skill renders ~/.agami/model/finbud/20260512-101500.html and opens it]

You (in dashboard): search "pan" → toggle Exclude on every PAN/AADHAAR
                    field → click Generate feedback → paste back in chat:

exclude columns: BUREAU_DATA.GENERAL_INFO.PAN,
                 BUREAU_DATA.GENERAL_INFO.AADHAAR,
                 BUREAU_DATA.NOHIT_DATA.PAN
done

[skill applies, validator passes, commits, re-renders]

agami: ✓ Applied: 3 columns excluded. Re-rendered.
       ~/.agami/model/finbud/20260512-101830.html

Removing staging tables

You: I never want agami to use the staging tables

[skill checks ORGANIZATION.md / table names for `_stg` suffix; if
 unsure, asks: "I see GENERAL_INFO_SCORE_STG — anything else? Click
 Exclude on the table cards in the dashboard."]

[user excludes via dashboard, pastes back]
exclude tables: BUREAU_DATA.GENERAL_INFO_SCORE_STG
done

agami: ✓ Applied: 1 table excluded.