using-wlyctl

Using wlyctl

wlyctl is the CLI for managing Whaly platform resources — Objects, Layouts, Sources, and Models (a dbt project). It talks to the Whaly GraphQL API to list / get / pull / apply / delete resource definitions stored as local files.

The full command and flag reference lives in wlyctl <command> --help — defer to those for exact syntax. This skill covers the workflow and the file-authoring rules an agent must follow, which are not enforced by the tool.

Core workflow

Always operate on local files, then push them. Two ways to start:

1. Files already local — edit/create them in place.
2. No local files (or they may be stale) — propose to pull/get fresh state first, then edit:
   • wlyctl <kind> pull [dir] — pull all of a kind to local files
   • wlyctl <kind> get <id> — fetch one resource as YAML
   • wlyctl models pull [dir] — pull the full dbt project locally

Then the loop is:

pull / get          # get authoritative local copy (if not already local)
  ↓
edit / create        # follow the authoring rules below
  ↓
verify               # models: `wlyctl models compile` — MUST pass before applying
  ↓
dry-run / diff       # `wlyctl <kind> apply -f <file> --dry-run`  (review the plan!)
  ↓
apply                # `wlyctl <kind> apply -f <file>`  (confirm the change summary)

Never apply without first running a dry-run/diff and reading the plan. apply auto-creates a backup before mutating (models); wlyctl models rollback restores it.

The universal ID rule

The Whaly backend owns and auto-generates all id fields. Agents must never invent, infer, or copy an id.

• Creating a new entry → omit the id field entirely. A temp id is generated client-side; the backend assigns the real id, which appears on the next pull/get.
• Editing an existing entry → leave every existing id untouched. Those ids are how the backend reconciles your file with its records. Changing or removing them breaks reconciliation.

This applies to every entity with an id: objects, properties, metrics, semantic groups, relationships, and all nested layout structures (rows, columns, widgets).

# CORRECT — new property, no id
properties:
  - label: "My New Property"
    propertyType: standard
    columnName: my_column
    columnDomain: STRING

# WRONG — never include id on a new entry
properties:
  - id: "9999"        # ❌ do not invent ids
    label: "My New Property"

# CORRECT — new layout widget, no id on row/column/widget
rows:
  - name: "hello"
    position: 5
    config: {}
    columns:
      - position: 0
        config: { size: 24 }
        widgets:
          - name: ""
            type: markdown
            position: 0
            config: { text: "Hello" }

Models (dbt) authoring rules

Models are a dbt project pulled via wlyctl models pull. Verify every change with wlyctl models compile before applying.

File shape

• Every model is a pair: a .sql AND a .yml. Never create one without the other.
• The .sql first line is ALWAYS the materialization config, e.g. {{ config(materialized='ephemeral') }}. Materialization lives in the .sql, not in the .yml.
• SQL constraints: BigQuery SQL only; EU location, never US; no Jinja dynamic macros.

.yml meta keys

version: 2
models:
  - name: "stgmymodel"
    meta:
      # x-whaly-dataset-id: OMIT on new models. Written ONLY by `wlyctl models pull`.
      #   NEVER infer, invent, or hand-write this id.
      x-whaly-primary-key: customer_id,product_id   # column(s) that make a row unique; comma-separated
      x-whaly-display-name: Stg - My Model           # human-facing label
      x-whaly-folder-path: 1. Core/1.Staging/MySource # human-facing folder hierarchy (see note below)
    config:
      materialized: ephemeral
    columns: []

• x-whaly-dataset-id — backend-owned. Omit entirely for new models. Populated by pull, never by a human.
• x-whaly-primary-key — the unique-key column name(s), comma-separated for composite keys.
• x-whaly-display-name — the human-facing model label.
• x-whaly-folder-path — the /-separated position in the folder hierarchy. ⚠️ This is the human-facing display hierarchy, which is not always identical to the on-disk directory names (the directory is a slugified form). When creating a new model, copy the x-whaly-folder-path format from a sibling .yml in the same folder rather than guessing — match siblings, don't transliterate the directory path.

Layer architecture

The folder structure encodes a layered DAG. The lower the layer, the more foundational. Common top-level layers include:

1. Core — long-lived logic.
2. Objectives — ephemeral logic recreated each objectives cycle; kept separate because it's short-lived.
3. Scores — aggregate Core + Objectives tables into entity scores.
4. Exposures — user-facing tables.

This list is not exhaustive and the naming evolves — always wlyctl models pull and follow whatever folders actually exist on disk (there may be additional layers such as exports or per-feature folders). Place new models in the correct existing folder; don't invent a layer.

Within Core, the classic dbt sub-layers apply: staging → intermediate → marts.

Hard rules:

• Only join LOWER layers. When working on a model, never join it with a model from a higher layer. Self-joins are fine.
• If a model needs to join a model in the same or a higher layer, move the model up to that layer — and move its dependents up too, so the "only join lower" rule still holds for them.
• Staging: never JOINs other tables (self-join OK); its job is to clean input data.
• Intermediate: bridges staging ↔ marts; modularizes transformations.
• Marts: user-facing, final models — the full picture.
• DRY: a given transformation/filter is done in exactly one model. No duplicated logic.
• Work folder-by-folder, layer-by-layer when making many changes.

Sources lifecycle

Sources are managed only by wlyctl sources * commands. The models pull/apply/rollback commands do not back up or manage sources — sources exist in the local models directory only so dbt's source() macro can resolve. Don't add, edit, or prune sources through models commands.

Before you finish

• Models changed? Run wlyctl models compile and confirm it passes.
• Run apply --dry-run and read the plan before the real apply.
• New entries have no id / no x-whaly-dataset-id; edited entries keep theirs untouched.