newman-runner

Newman Runner

This skill provides tools to run Newman (Postman CLI) tests and analyze the results for API testing and validation.

Instructions

Running Newman Tests

1. Verify Newman Installation

   • Check if Newman is installed: which newman
   • If not installed, install with: npm install -g newman

2. Run Collection

   • Use script: scripts/run-newman.sh <collection.json>
   • Or manually: newman run collection.json --reporters cli,json --reporter-json-export output.json

3. Analyze Results

   • Parse JSON output with: scripts/analyze-newman-results.py output.json
   • Extract: Pass/fail status, response times, error messages, assertions

4. Run in CI/CD

   • Use scripts/run-newman-ci.sh <collection.json> for CI-optimized execution
   • Produces JUnit XML for CI reporting + JSON for analysis
   • Uses GitHub Actions annotation format for inline error reporting

Available Scripts

• scripts/run-newman.sh - Run Newman with standard options
• scripts/run-newman-ci.sh - CI-optimized runner with JUnit output
• scripts/analyze-newman-results.py - Parse Newman JSON output
• scripts/validate-collection.sh - Validate Postman collection structure

Advanced Patterns

For OpenAPI-to-Newman pipeline, auth injection, and collection management, see the api-contract-testing skill which builds on this runner.

Examples

Example 1: Run API Tests

# Run Newman tests on Postman collection
./scripts/run-newman.sh my-api-tests.json

# Analyze results
./scripts/analyze-newman-results.py newman-results.json

Example 2: Validate Collection

# Check collection is valid before running
./scripts/validate-collection.sh my-collection.json

Example 3: Run in CI

# CI-optimized run with JUnit output
./scripts/run-newman-ci.sh my-api-tests.json

# Results in:
# test-results/api-contract/newman-results.json  (JSON)
# test-results/api-contract/newman-junit.xml     (JUnit XML)

Requirements

• Newman installed globally: npm install -g newman
• Valid Postman collection JSON file
• Python 3.7+ for analysis scripts

Success Criteria

• Newman tests run successfully
• Results parsed and analyzed
• Pass/fail status clearly reported
• Error details extracted for failures