cli-output-conventions

cli-output-conventions

Overview

Per clig:

"Send output to stdout. The primary output for your command should go to stdout. Anything that is machine readable should also go to stdout."

"Send messaging to stderr. Log messages, errors, and so on should all be sent to stderr."

Tests can only assert on output that is stable, separated, and documented. This skill defines those conventions; sister skills (bats-testing, tui-snapshot-tester) test against them.

When to use

• Authoring or reviewing a CLI's output (any new flag, command, or error message).
• Writing tests for a CLI: this skill tells you WHAT to assert on; bats / pytest etc. tell you HOW.
• A CLI's tests are flaky / brittle - usually the underlying output isn't deterministic. Fix the output, not the assertion.

Convention 1 - Exit codes

Per clig:

"Return zero exit code on success, non-zero on failure."

"Map non-zero codes to important failure modes for script integration."

Exit code | Meaning
----------+----------------------------
0         | Success
1         | General error (catch-all)
2         | Misuse (bad flag, bad usage)
3         | Resource not found
4         | Permission denied
5         | Network / external failure
... documented per CLI

Test pattern:

@test "exit 0 on success" {
  run mycli list
  [ "$status" -eq 0 ]
}

@test "exit 2 on bad flag" {
  run mycli --nonexistent
  [ "$status" -eq 2 ]
}

Don't test [ "$status" -ne 0 ] - assert the specific code. Otherwise refactors silently change the contract.

Convention 2 - stdout vs stderr

Per clig: "This separation ensures piped commands receive only data, not messages."

# Good: data on stdout, message on stderr
$ mycli list 2>/dev/null
alice
bob

$ mycli list >/dev/null
fetched 2 users in 0.3s

Test pattern (bats):

@test "list emits names on stdout, status on stderr" {
  run --separate-stderr mycli list
  [ "$status" -eq 0 ]
  [ "$output" = $'alice\nbob' ]
  [[ "$stderr" == *"fetched 2 users"* ]]
}

run --separate-stderr (Bats 1.5+) splits the streams; without it, $output mixes both.

Convention 3 - Machine-readable mode

Per clig:

"Display output as formatted JSON if --json is passed."

"If human-readable output breaks machine-readable output, use --plain to display output in plain, tabular text format for integration with tools like grep or awk."

"Encourage your users to use --plain or --json in scripts to keep output stable."

$ mycli list --json | jq '.[] | .name'
"alice"
"bob"

$ mycli list --plain
alice  active   2026-04-15
bob    active   2026-04-20

Test pattern: assert against --json, never against the human-formatted default. Human output is allowed to evolve; JSON is the contract.

@test "list --json contract" {
  run mycli list --json
  assert_success
  echo "$output" | jq -e '.[0] | has("name") and has("status")'
}

Convention 4 - Determinism

Stable output for tests requires:

• No timestamps in default output (only with --verbose or human-mode banner).
• Stable ordering - sort lists by a canonical key, not insertion order.
• No random IDs in output unless documented.
• Locale-independent number formatting in --json / --plain (no thousand separators; ISO 8601 dates).

# Bad: randomized order, locale-dependent date
$ mycli list
bob   2026/04/20
alice 2026/04/15

# Good: sorted, ISO 8601
$ mycli list --plain
alice 2026-04-15
bob   2026-04-20

Test pattern: golden-file comparison with sorted output.

Convention 5 - Color & TTY

Per clig:

"Disable color if your program is not in a terminal or the user requested it."

Disable color when:

• stdout (or stderr) is not a TTY.
• NO_COLOR env var is set (per https://no-color.org/).
• TERM=dumb.
• --no-color flag passed.

"If stdout is not an interactive terminal, don't display any animations."

Progress bars / spinners only on TTY - they pollute CI logs and break wc -l assertions.

Test pattern:

@test "no ANSI codes when piped" {
  run bash -c 'mycli list | cat'
  [ "$status" -eq 0 ]
  refute_output --regexp $'\\x1b\\['
}

@test "NO_COLOR honored" {
  NO_COLOR=1 run mycli list
  refute_output --regexp $'\\x1b\\['
}

Convention 6 - Quiet & verbose

Per clig:

"you can provide a -q option to suppress all non-essential output."

"By default, don't output information that's only understandable by the creators of the software... only in verbose mode."

Mode      | Flag         | Use
----------+--------------+--------------------
Quiet     | -q / --quiet | Scripts; only data + errors
Default   | (none)       | Interactive; status messages on stderr
Verbose   | -v           | Debug info on stderr
Debug     | -vv / --debug| Internal traces on stderr

Test pattern:

@test "-q suppresses status messages" {
  run --separate-stderr mycli -q list
  [ "$status" -eq 0 ]
  [ -n "$output" ]    # data still on stdout
  [ -z "$stderr" ]    # no status messages
}

Convention 7 - --help and --version

Per clig:

"Show full help when -h and --help are passed... you should be able to add -h to the end of anything and it should show help."

@test "--help exits 0 and mentions Usage" {
  run mycli --help
  [ "$status" -eq 0 ]
  [[ "$output" == *"Usage:"* ]]
}

@test "subcommand --help exits 0" {
  run mycli list --help
  [ "$status" -eq 0 ]
  [[ "$output" == *"list"* ]]
}

@test "--version emits machine-parseable version" {
  run mycli --version
  [ "$status" -eq 0 ]
  [[ "$output" =~ ^[0-9]+\.[0-9]+\.[0-9]+ ]]
}

--version should be parseable: mycli 1.2.3 or mycli version 1.2.3 - never Welcome to mycli! Version 1.2.3.

Convention 8 - Document the contract

Each CLI repo should have a CONVENTIONS.md (or section in README):

## Output contract

- Exit codes: 0 ok, 2 bad usage, 3 not found, 4 perm denied, 5 network.
- `stdout` = primary data; `stderr` = status / errors / progress.
- `--json` is the stable machine contract; default human output may evolve.
- `NO_COLOR` and TTY detection respected.
- All dates in `--json` are ISO 8601 UTC.

Tests assert against the contract. If the contract changes, both the document and the tests update in the same PR.

Anti-patterns

Anti-pattern	Why it fails	Fix
Tests assert status -ne 0	Refactors silently change error code; scripts break.	Assert specific code (Convention 1).
Errors on stdout	Pipes carry error text into downstream parsers.	stderr (Convention 2).
Tests assert against default human output	Human output evolves; tests churn.	Test against --json (Convention 3).
Timestamps in default output	Tests fail on every run.	Verbose-mode only (Convention 4).
ANSI codes leak when piped	wc -l, grep, etc. break.	TTY check + NO_COLOR (Convention 5).
Progress bars in CI	CI logs flooded; tests can't assert frame counts.	TTY check (Convention 5).
Help output requires network / config	--help should always work.	Help is local + static.
--version mixed with marketing	Tooling can't parse.	Pure version string (Convention 7).

Limitations

• Legacy CLIs. Adopting this skill against an existing CLI requires a deprecation cycle for output changes.
• Verbose / debug modes are inherently non-deterministic. Don't snapshot them; assert presence of key markers.
• Cross-shell quoting. Bats runs in Bash; PowerShell tests via Pester need their own assertions.
• Color contract is OS-dependent. Windows terminal emulators have different ANSI support; test on the target platforms.

References

• clig - Command Line Interface Guidelines: exit codes, stdout / stderr, --json / --plain, NO_COLOR, -q / --verbose, --help / --version.
• https://no-color.org/ - NO_COLOR informal standard.
• ISO 8601 - date / time format for --json output.
• bats-testing - Bash-based assertion runner that consumes this contract.
• tui-snapshot-tester - layout-level snapshots; this skill covers text-level contract.