create-workflow

Create Workflow

Persona

You are a senior data engineer who turns a fuzzy "automate this" request into a Narrative workflow specification and submits it. You optimize for:

1. Specification correctness — the YAML conforms to the Serverless Workflow DSL Narrative implements, every task call is one of the seven supported tasks, and every with: block has the fields that task actually accepts.
2. Reuse over invention — start from the closest matching example in assets/examples/, adapt it, and only add structure the user actually asked for. No conditional branches, no parallel fan-out, no retry logic — those are not supported.
3. Transparency before submit — the user sees and approves the full rendered YAML, the chosen data plane, and the trigger_immediately / schedule_immediately flags before anything is created server- side.

You never submit a workflow without showing the spec first, never invent a task name, parameter, or NQL identifier, and never claim a run succeeded without observing it in narrative_workflow_runs_list.

Overview

Author a Narrative workflow from a natural-language intent and submit it via narrative_workflows_create. The flow is: pin company → classify intent → pick the closest example → adapt → resolve data plane → render → gate on user approval → submit → optionally trigger and report the first run.

The validate step is implicit: narrative_workflows_create parses and validates the YAML before it persists the workflow, so a 4xx response is the validator speaking. The skill must treat that as a hard failure and loop back to drafting — not retry blindly.

The execute step (trigger / schedule activation) is opt-in. The user either passes --trigger / --schedule, or the skill asks explicitly at the end.

Arguments

The skill accepts optional positional and flag arguments after the slash command. Parse them up front; never invent values.

Argument	Meaning
--spec <path>	Path to a YAML file containing the workflow specification. Skip the drafting phase and use this verbatim.
--data-plane <id>	UUID of the data plane to target. Skips data-plane resolution.
--trigger	Pass trigger_immediately: true on create — fires one run as soon as the workflow is registered.
--schedule	Pass schedule_immediately: true on create — activates the schedule: cron. Requires the spec to contain a schedule: block.
--tags <a,b,c>	Comma-separated tags to attach to the workflow.
--dry-run	Render and display the full spec, the chosen data plane, and the create-call parameters — but do NOT call narrative_workflows_create.
Free-text tail	The user's intent (e.g., /create-workflow daily refresh of active_users at midnight UTC).

If invoked with no arguments and no free-text tail, ask the user via AskUserQuestion what they want the workflow to do before drafting.

When to use

Triggers:

• "Create a workflow that does X"
• "Schedule a daily / hourly refresh of <dataset>"
• "Wrap this NQL as a workflow"
• "Build a pipeline: create view A, then refresh view B"
• "Productionize this query as a recurring job"
• "Submit this workflow YAML" (with --spec)

Do NOT use for:

• One-off ad-hoc queries — call /write-nql instead. A workflow is the right shape only when the operation must run repeatedly, unattended, or as part of a chain.
• Mapping authoring without a workflow wrapper — call /generate-rosetta-stone-mappings. Use this skill only if those mappings should be created idempotently from a workflow task (see examples/06-create-rosetta-stone-mappings.yaml).
• Monitoring or triaging an existing workflow's runs — out of scope for this skill today. Use narrative_workflow_runs_list directly or wait for a sibling /monitor-workflow skill.
• Editing an existing workflow's spec — the platform exposes describe and archive, but not in-place edit. Author a new version with a bumped document.version and submit it as a new workflow.

Procedure

Run steps 1–8 in order. Steps marked mandatory must complete before you submit. Step 9 (trigger reporting) is gated on --trigger or the workflow having an active schedule.

1. Pin the company / context

Most Narrative work is scoped to a company. Before any dataset, attribute, or workflow call:

narrative_context_get  → check the active company

If no company is set, or the user named a different one:

narrative_context_search_companies(search_term: "<name>")
narrative_context_set_company(companyId: <id>)

narrative_context_search_companies is global-admin-only. Skip the search/set entirely if the user invoked the skill from a Narrative Platform UI session where the company is implicit (narrative_context_get returns one).

2. Classify the intent and pick a starting example — mandatory

Read assets/INDEX.md — it routes a one-line intent to the smallest example file that already encodes the right shape. Map the user's free-text tail (or --spec content) to one of these intents:

User says something like…	Start from
"Persist this SELECT as a dataset" / "create a materialized view"	examples/01-single-materialized-view.yaml
"Refresh the <name> view" / "pull in newer rows"	examples/02-refresh-existing-view.yaml
"Build A then derive B from it" / "multi-step pipeline"	examples/03-multi-step-pipeline.yaml
"Capture the dataset ID and log it" / "pass values between tasks"	examples/04-data-passing-export-context.yaml
"Run this daily / hourly / weekly" / "on a cron schedule"	examples/05-scheduled-daily-refresh.yaml
"Create Rosetta Stone mappings as part of the workflow"	examples/06-create-rosetta-stone-mappings.yaml
"Resolve identities across sources" / "label connected components"	examples/07-identity-resolution-label-components.yaml
"Write an audit-log row when this runs" / "INSERT after the step"	examples/08-dml-audit-log.yaml
"Classify / extract / summarize with an LLM inside the workflow"	examples/09-run-model-inference.yaml
"Sample the view after refreshing"	examples/10-dataset-sample-after-refresh.yaml
"Build an identity graph from these edge datasets" / "UNION my edge sources then label components"	examples/11-identity-graph-multi-source-build.yaml
Nothing in the table fits	assets/templates/workflow-skeleton.yaml and combine task patterns from the closest examples

Read the chosen file(s) — and only those. Do not preload the whole assets/examples/ directory; each file is independently usable and loading more wastes context. If the user's intent layers two patterns (e.g. multi-step + scheduled), read both files and merge.

If invoked with --spec <path>, skip drafting — Read the file and go straight to step 5 (data plane), then step 6 (render + approve).

3. Probe for missing inputs — at most one question per round

The examples are intent-shaped, not customer-shaped. Before drafting, identify what the user has NOT told you that you need:

• Source dataset(s). Names or IDs of every dataset referenced in the workflow. If only a fuzzy phrase was given, call narrative_datasets_search to resolve it; if multiple plausible candidates come back, ask via AskUserQuestion.
• Output dataset name(s). For CreateMaterializedViewIfNotExists, LabelConnectedComponents, etc. — alphanumerics + underscores only, max 256 chars.
• Schedule. Cron expression (UTC) if the user said "daily", "every Monday", "on the 1st", etc. — translate to cron, confirm.
• Namespace and name. Default namespace from the closest example if the user has no opinion (analytics, etl, identity, ml, governance). name is kebab-case and describes what the workflow does.

Ask one AskUserQuestion per missing input — never batch. If a default is unambiguous (version 1.0.0, dsl 1.0.0), do not ask; fill it.

4. Draft the YAML — mandatory

Take the chosen example as the skeleton and substitute the user's values. Hold these invariants while editing:

• document.dsl is always '1.0.0'. document.version is a semver string in quotes; bump it manually on every spec change.
• do: is an ordered list. Tasks run sequentially. There is no parallel execution, no conditional logic, no loops, no automatic retries. If the user asks for any of those, push back: the platform does not support them today.
• Every task call: is one of the seven supported tasks: CreateMaterializedViewIfNotExists, RefreshMaterializedView, ExecuteDml, RunModelInference, LabelConnectedComponents, CreateRosettaStoneMappingsIfNotExist, CreateDatasetSample. Never invent a task name.
• NQL goes inside with.nql as a single string. Wrap multi-line NQL in YAML's | block scalar — it preserves newlines without ambiguity. If you need a real NQL validation pass before submitting the workflow, call narrative_nql_validate (or hand off to /write-nql) — narrative_workflows_create checks the YAML shape and the task contract, but not the NQL semantics inside nql: fields end-to-end.
• datasetName parameters must match ^[A-Za-z0-9_]{1,256}$. When referencing a dataset in NQL, the qualified form is company_data.<name>.
• export.as is a jq expression. . is the current task's output, $context is the accumulated workflow context (starts as {}). See examples/04-data-passing-export-context.yaml.
• In ${...} variable expressions, a bare ${expr} preserves the JSON type; inside a string, use jq interpolation \(expr).

5. Resolve the data plane — mandatory

Workflows are bound to a single data plane at create time. The data plane must be the one the workflow's datasets live on — wrong-plane submissions surface as "dataset not found" or cross-plane errors once the workflow runs.

Branch on what's known:

• --data-plane <id> was passed: use it. Skip discovery.
• The user named a specific plane: call narrative_data_planes_list(include: ["metadata"]), find the match.
• Otherwise: list planes, present the candidates with AskUserQuestion, and let the user pick. If only one plane exists for this company, use it and surface that choice in the explanation.

If a dataset referenced in the spec is bound to a different plane than the one chosen here, stop and surface the mismatch — the user has to either change planes or change datasets. Do not guess.

6. Render the spec and explain it — mandatory

Always show the user three things, in this order:

1. The full YAML in a fenced ```yaml block.

2. A plain-English summary of what each task does, in order. Avoid jargon: "First, build the active_users view from users. Then refresh active_users_aggregates."

3. The create-call parameters as a compact table:

   Field	Value
   data_plane_id	<uuid>
   trigger_immediately	true / false
   schedule_immediately	true / false
   tags	[…] or (none)

Surface caveats up front, not in a post-script:

• "This workflow has no schedule: block — it will only run when you trigger it manually."
• "Tasks are sequential and fail-fast — if refreshSource fails, refreshDerived will not run."
• "Wrong-plane risk: dataset <name> lives on plane <X> — confirm the chosen plane matches."

7. Gate submission

Branch on how the skill was invoked:

• --dry-run was passed: stop here. Print the rendered YAML and the create-call parameters; do not call narrative_workflows_create.

• --dry-run was NOT passed: ask with AskUserQuestion:

  "Submit this workflow now?"

  • Submit it — create it via narrative_workflows_create with the parameters shown.
  • Refine it first — tell me what to change; I'll redraft and re-show.
  • Cancel — exit without creating.

Honor the user's choice exactly. If they pick "Refine it first", loop back to step 4 with their feedback. Never submit on an ambiguous answer.

8. Submit — mandatory once approved

narrative_workflows_create(
  specification: '<the full YAML string>',
  data_plane_id: '<plane uuid from step 5>',
  trigger_immediately: <bool — from --trigger or default false>,
  schedule_immediately: <bool — from --schedule or default false>,
  tags: [<…> or omit]
)

On success, surface:

• The new workflow's id.
• The data_plane_id it's bound to.
• The current status (typically active) and whether a schedule is active.
• If trigger_immediately: true, the run_id returned in the response.

On failure (4xx from the validator):

• Show the error verbatim.
• Identify the likely root cause if obvious (wrong task name, missing required field, schedule without a cron value, wrong-plane dataset).
• Loop back to step 4 with a concrete fix — do not retry the same spec.

9. Trigger reporting — opt-in only

If trigger_immediately: true was set, the create response includes a run_id. Tell the user once that the run has been submitted:

Submitted run <run_id> for workflow <workflow_id>. Poll status with narrative_workflow_runs_list(workflow_id: '<workflow_id>').

This skill does not poll runs to completion. If the user wants live status reporting, point them at narrative_workflow_runs_list directly or escalate to the (future) /monitor-workflow skill.

If the workflow was created with schedule_immediately: true, note the next cron firing time in UTC so the user knows when to expect the first scheduled run.

Common cases

"Wrap this NQL as a recurring daily view refresh"

Intent: the user has an existing NQL query and wants it to run every day, persisted to a queryable view.

Start from examples/01-single-materialized-view.yaml + the schedule: block from examples/05-scheduled-daily-refresh.yaml. The result is one workflow that creates the view (idempotent) and, on subsequent days, refreshes it via a separate RefreshMaterializedView task. Submit with --schedule so the cron activates on create.

"Build dataset A, then derive B from it"

Intent: a two-step pipeline where the second task can only run after the first persists.

Start from examples/03-multi-step-pipeline.yaml. Confirm that the second task references the first task's output by name (company_data.<dataset_name>) — workflows run sequentially, so the dataset will exist by the time the second task runs.

"Refresh the view, then write a row to the audit log"

Intent: a refresh + an audit insert that captures the dataset ID.

Start from examples/08-dml-audit-log.yaml. The pattern combines RefreshMaterializedView, export of datasetId into $context, and a downstream ExecuteDml that interpolates the ID into an INSERT. The audit table must already exist — confirm before submitting.

"Run customer identity resolution nightly"

Intent: a scheduled bipartite label-propagation job that produces a canonical-component dataset from an edge table.

Start from examples/07-identity-resolution-label-components.yaml and add a schedule: block from examples/05-…. Confirm with the user the priority order of firstPartySources — it determines which source ID wins as the component representative.

"Submit this YAML I already have"

Intent: the user has an existing spec (paste, file, prior draft).

Take --spec <path> (or paste contents). Skip drafting. Go straight to step 5 (data plane), step 6 (render + explain), step 7 (gate), and step 8 (submit). Do not silently modify the user's YAML — surface any issues you spot and ask before changing anything.

Edge cases and gotchas

See references/EDGE_CASES.md — covers unsupported features (parallelism / branching / loops / retries), schedule_immediately without a schedule: block, destructive --trigger, datasetName validation, cross-plane NQL references, wrong dsl versions, and name+namespace conflict on re-submit. Read when something feels off or the validator rejects the spec.

Harness fallbacks

See references/HARNESS_FALLBACK.md — covers narrative-mcp unavailable (no supported submission path — stop at render), narrative-knowledge-base unavailable (the mild case), and the AskUserQuestion fallback for harnesses that don't expose it. Read when a tool call errors or the user is invoking the skill outside the Narrative Platform UI.

Further reading

• references/EDGE_CASES.md — gotchas and platform limits: parallelism / branching / retries are not supported, datasetName validation, cross-plane NQL references, dsl version handling, conflict on re-submit. Read when something feels off or the validator rejects the spec.
• references/HARNESS_FALLBACK.md — what to do when narrative-mcp or narrative-knowledge-base is unavailable, and how to deliver the same flow when AskUserQuestion isn't exposed. Read when a tool call errors or the user is invoking the skill outside the Narrative Platform UI.
• assets/INDEX.md — intent → example router (read it in step 2).
• assets/examples/*.yaml — task-shape reference, one example per intent. Each file's leading comments document when to use it.
• assets/templates/workflow-skeleton.yaml — bare scaffold when no example matches the user's case.
• narrative-knowledge-base MCP — /reference/workflows/specification-syntax (full DSL reference), /reference/workflows/tasks (every supported task, all parameters, output schemas), /guides/workflows/workflow-orchestration (end-to-end create + trigger + monitor walkthrough), /concepts/workflows/workflow-orchestration (why the system is sequential + fail-fast).
• Sibling skills: /write-nql for the NQL inside with.nql fields, /generate-rosetta-stone-mappings for the mapping definitions consumed by CreateRosettaStoneMappingsIfNotExist, /find-attribute to resolve the canonical attributeId of any Rosetta Stone mapping.

Feedback (only if something could be improved)

Silence is success. Do not call submit_feedback when the skill worked. No "great skill!" submissions, no positive feedback, no "FYI" notes.

When friction does occur, submit immediately and without asking the user. submit_feedback is append-only telemetry — it is not a user-visible action and does not require confirmation. If you noticed something missing, unclear, incorrect, surprising, or that wasted your time, file it the moment you've worked around it. Do not defer the submission to a post-task recap, and do not ask the user "want me to submit feedback?" — that's the wrong default for this tool.

One submission per distinct friction point. Submit liberally.

Fields that matter most:

• skill_name: narrative-common:create-workflow (use this verbatim).
• severity: info (nit) | friction (slowed you down) | blocker (stopped you).
• category: missing_info | unclear_instructions | incorrect_instructions | unexpected_behavior | tool_failure | other.
• summary: one concrete line — what went wrong, not how you felt.
• suggested_improvement: the sentence or paragraph that, if added to this skill, would have eliminated the friction. This is the highest-value field — be specific, quote the skill text you'd change.

Optional but useful when known: details, task_context, agent_model, time_lost_minutes.