bats-testing

bats-testing

Overview

Per bats-readthedocs:

"Bats (Bash Automated Testing System) is a TAP-compliant testing framework for Bash 3.2 or above."

Per bats-github:

"Bats is a TAP-compliant testing framework for Bash 3.2 or above. It provides a simple way to verify that the UNIX programs you write behave as expected."

Bats works "with any Unix program," not just Bash - it captures stdout / stderr / exit code of any executable.

When to use

• Testing a shell script (deployment scripts, CI scripts, installers).
• Testing a CLI binary's interface (commands, flags, exit codes, output) at the integration layer.
• Validating that wrapper scripts (e.g., kubectl plugins, git aliases) behave as expected.
• Cross-platform smoke tests for a CLI's install scripts.

Step 1 - Install

Per bgh:

# npm
npm install -g bats

# Homebrew
brew install bats-core

# Git submodule (vendored, deterministic for CI)
git submodule add https://github.com/bats-core/bats-core.git test/bats
git submodule add https://github.com/bats-core/bats-support.git test/test_helper/bats-support
git submodule add https://github.com/bats-core/bats-assert.git test/test_helper/bats-assert

The git-submodule approach pins the Bats version per repo - recommended for CI determinism.

Step 2 - First test file

Per bgh (verbatim example):

#!/usr/bin/env bats

@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}

@test "addition using dc" {
  result="$(echo 2 2+p | dc)"
  [ "$result" -eq 4 ]
}

Save as test/math.bats; run bats test/math.bats.

Step 3 - run for exit-code + output capture

The run helper invokes a command and captures $status, $output, and $lines[]:

@test "myscript --help exits 0 and mentions usage" {
  run ./myscript --help
  [ "$status" -eq 0 ]
  [[ "$output" == *"Usage:"* ]]
}

@test "myscript with no args exits 1" {
  run ./myscript
  [ "$status" -eq 1 ]
  [ "$output" = "error: missing required argument" ]
}

@test "myscript output line count" {
  run ./myscript list
  [ "$status" -eq 0 ]
  [ "${#lines[@]}" -eq 3 ]
  [ "${lines[0]}" = "alice" ]
}

Without run, a non-zero exit code aborts the test before assertions execute.

Step 4 - setup / teardown

Per bats: "setup and teardown: Pre- and post-test hooks."

setup() {
  TEST_DIR="$(mktemp -d)"
  cp fixtures/sample.txt "$TEST_DIR/"
}

teardown() {
  rm -rf "$TEST_DIR"
}

@test "myscript processes the sample file" {
  run ./myscript "$TEST_DIR/sample.txt"
  [ "$status" -eq 0 ]
}

setup_file / teardown_file run once per file (vs once per test). Use for expensive setup (Docker container start, database init).

Step 5 - load shared helpers

Per bats: "load: Share common code."

# test/test_helper.bash
make_temp_repo() {
  local d
  d="$(mktemp -d)"
  (cd "$d" && git init -q)
  echo "$d"
}

# test/integration.bats
load test_helper

@test "git status in fresh repo is clean" {
  local d
  d="$(make_temp_repo)"
  cd "$d"
  run git status --porcelain
  [ "$status" -eq 0 ]
  [ -z "$output" ]
}

Step 6 - bats-assert / bats-support

Better assertions than raw [ ]:

load test_helper/bats-support/load
load test_helper/bats-assert/load

@test "myscript produces expected JSON" {
  run ./myscript --json
  assert_success
  assert_output --partial '"version":'
  refute_output --partial 'ERROR'
}

@test "myscript fails on bad input" {
  run ./myscript --bad-flag
  assert_failure 2
  assert_line --index 0 "Unknown flag: --bad-flag"
}

assert_success / assert_failure / assert_output / assert_line give clear diff-style failure messages (vs [ ]'s silent fail).

Step 7 - skip

Per bats: "skip: Easily skip tests."

@test "macOS-only behavior" {
  if [[ "$(uname -s)" != "Darwin" ]]; then
    skip "macOS-only test"
  fi
  run ./myscript --use-keychain
  [ "$status" -eq 0 ]
}

@test "needs docker daemon" {
  if ! command -v docker >/dev/null; then
    skip "docker not installed"
  fi
  if ! docker info >/dev/null 2>&1; then
    skip "docker daemon not running"
  fi
  run ./deploy-script --dry-run
  [ "$status" -eq 0 ]
}

skip reports the reason in test output (vs commenting out, which hides the fact that coverage dropped).

Step 8 - Parallel execution

Per bats: "Parallel Execution."

# Install GNU parallel first (apt install parallel / brew install parallel)
bats --jobs 4 test/

Tests must be independent - share no global filesystem state, no shared ports. Use setup/teardown with mktemp -d to isolate.

Step 9 - Output formats

# Default (pretty)
bats test/

# TAP (CI-friendly, parseable)
bats --tap test/

# JUnit XML (for CI dashboards via TAP-to-JUnit converters)
bats --formatter junit test/ > junit.xml

# Verbose (show stdout/stderr of every command)
bats --verbose-run test/

Step 10 - CI integration

jobs:
  bats:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
        with: { submodules: recursive }
      - run: |
          sudo apt-get install -y parallel
      - run: bats --jobs 4 --formatter junit test/ > junit.xml
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: bats-junit, path: junit.xml }

For Docker-based CI, per bats: "Running Bats in Docker" using the official bats/bats:latest image:

- run: docker run --rm -v "$PWD:/code" bats/bats:latest test/

Anti-patterns

Anti-pattern	Why it fails	Fix
Calling commands directly without run	Non-zero exit aborts the test before any assertion runs.	Wrap with run; check $status.
Tests sharing /tmp / fixed ports	Parallel execution breaks intermittently.	Per-test mktemp -d in setup (Step 4).
Using [ ] for everything	Silent failures; no diff message.	bats-assert (Step 6).
Commenting out skipped tests	Hides coverage loss; no signal.	skip with reason (Step 7).
Asserting full multiline output	Brittle - any whitespace change breaks tests.	Use assert_output --partial or assert_line --index N.
One mega .bats file	Slow; hard to grep failures.	Per-feature files in test/.

Limitations

• Bash-only. Tests are written in Bash; PowerShell scripts need Pester instead.
• Slow vs in-language tests. Each run forks a process - expensive vs library-level tests in the language.
• Output capture quirks. run strips trailing newlines; some binary output is not faithfully captured.
• Windows requires WSL or Git Bash. Per bats: "Installing Bats from source via Git Bash."

References

• bats - overview, install methods, test structure, @test / run / setup / teardown / load / skip, parallel execution, Docker.
• bgh - example test file, install one-liners.
• tui-snapshot-tester - TUI snapshot tests (visual layer; bats covers exit code + text output).
• cli-output-conventions - what to assert on (stable formats, exit codes, stderr).