ai-training/.cursor/skills/test-spec/templates/run-tests-script.md

Test Runner Script Structure

Reference for generating scripts/run-tests.sh and scripts/run-performance-tests.sh.

When to generate a local run-tests.sh

A local shell script is needed only for hardware-dependent projects that require real hardware (GPU, CoreML, TPU, sensors, etc.) to exercise the actual code paths. If the Hardware-Dependency Assessment (Phase 4 prerequisite) determined local or both execution, generate this script.

For all other projects, use Docker (docker-compose.test.yml / Dockerfile.test). Docker is the default — it provides reproducibility, isolation, and CI parity. Do not generate a local run-tests.sh when Docker is sufficient.

scripts/run-tests.sh (local / hardware-dependent only)

#!/usr/bin/env bash
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
UNIT_ONLY=false
RESULTS_DIR="$PROJECT_ROOT/test-results"

for arg in "$@"; do
  case $arg in
    --unit-only) UNIT_ONLY=true ;;
  esac
done

cleanup() {
  # tear down services started by this script
}
trap cleanup EXIT

mkdir -p "$RESULTS_DIR"

# --- Install Dependencies ---
# MANDATORY: install all project + test dependencies before building or running.
# A fresh clone or CI runner may have nothing installed.
# Python:  pip install -q -r requirements.txt -r e2e/requirements.txt
# .NET:    dotnet restore
# Rust:    cargo fetch
# Node:    npm ci

# --- Build (if needed) ---
# [e.g. Cython: python setup.py build_ext --inplace]

# --- Unit Tests ---
# [detect runner: pytest / dotnet test / cargo test / npm test]
# [run and capture exit code]

# --- Blackbox Tests (skip if --unit-only) ---
# if ! $UNIT_ONLY; then
#   [start mock services]
#   [start system under test]
#   [wait for health checks]
#   [run blackbox test suite]
# fi

# --- Summary ---
# [print passed / failed / skipped counts]
# [exit 0 if all passed, exit 1 otherwise]

scripts/run-performance-tests.sh

#!/usr/bin/env bash
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
RESULTS_DIR="$PROJECT_ROOT/test-results"

cleanup() {
  # tear down test environment if started
}
trap cleanup EXIT

mkdir -p "$RESULTS_DIR"

# --- Install Dependencies ---
# [same as above — always install first]

# --- Start System Under Test ---
# [docker compose up -d or start local server]
# [wait for health checks]

# --- Run Performance Scenarios ---
# [detect tool: k6 / locust / artillery / wrk / built-in]
# [run each scenario from performance-tests.md]
# [capture metrics: latency P50/P95/P99, throughput, error rate]

# --- Compare Against Thresholds ---
# [read thresholds from test spec or CLI args]
# [print per-scenario pass/fail]

# --- Summary ---
# [exit 0 if all thresholds met, exit 1 otherwise]

Key Requirements

• Docker is the default: only generate a local run-tests.sh for hardware-dependent projects. Otherwise use docker-compose.test.yml.
• Always install dependencies first: the script must install all project and test dependencies before building or running tests. A fresh clone or CI runner may have nothing installed. Missing a single dependency causes collection errors that abort the entire test run.
• Both scripts must be idempotent (safe to run multiple times)
• Both scripts must work in CI (no interactive prompts, no GUI)
• Use trap cleanup EXIT to ensure teardown even on failure
• Exit codes: 0 = all pass, 1 = failures detected
• Write results to test-results/ directory (add to .gitignore if not already present)
• The actual commands depend on the detected tech stack — fill them in during Phase 4 of the test-spec skill