missions/_docs/02_document/tests/environment.md

Test Environment

Status: produced by autodev /test-spec Phase 2 (2026-05-14). Naming: post-rename target — missions service, Azaion.Missions.* namespace, /vehicles + /missions + /missions/{id}/waypoints routes. Until B5–B8 land, the missions service image is built from the existing Azaion.Flights.csproj source — tests will be RED until the rename converges. This is the autodev-aligned path: Step 8 (Refactor) closes the gap. Hardware Assessment section is filled by phases/hardware-assessment.md between Phase 3 and Phase 4.

Overview

System under test: the missions .NET 10 REST service exposed on http://missions:8080 inside the test network. Public surface = the HTTP endpoints documented in _docs/00_problem/input_data/data_parameters.md § 7.

Consumer app purpose: a standalone xUnit test project (tests/Azaion.Missions.E2E.Tests.csproj) that exercises the running service through HTTP only. No Azaion.Missions.* types are referenced; the consumer never opens a DataConnection to the system-under-test's runtime DB except via a side-channel postgres-test connection used to (a) seed fixtures and (b) assert DB side-effects (cascade row counts, default-vehicle invariants).

The side-channel DB access is allowed because the AC catalogue (AC-1.2, AC-1.4, AC-3.1, AC-3.3, AC-10.2) explicitly defines DB state as the verifiable observable. It is NEVER used to mutate state under-test that the API would normally own — only to (1) seed fixtures and (2) assert.

Docker Environment

Services

Service	Image / Build	Purpose	Ports (host:container)
missions	build context ./ (Dockerfile); image tag azaion/missions:test	System under test	5002:8080
postgres-test	postgres:16-alpine	Owned PostgreSQL for test isolation. Started fresh per test class via Testcontainers OR via docker compose down -v && docker compose up -d between scenarios that mutate startup-sensitive state (AC-6.5 legacy drop, AC-6.6 idempotency)	5433:5432
jwks-mock	build context tests/Azaion.Missions.JwksMock/; image tag azaion/jwks-mock:test	In-process stand-in for the admin service's JWKS endpoint. Holds a fixed ECDSA P-256 keypair (in-memory), serves the public key as JWKS at https://jwks-mock:8443/.well-known/jwks.json (HTTPS-only, self-signed CA mounted into missions and e2e-consumer), and signs tokens for the consumer via POST /sign. Supports POST /rotate-key for NFT-RES-07 (JWKS rotation). The mock's Cache-Control: max-age is set to 60s in tests (vs admin's 3600s) so rotation completes within the 15-min CI gate.	— (internal only)
e2e-consumer	build context tests/Azaion.Missions.E2E.Tests/; runs dotnet test	xUnit test runner; produces report.csv. Fetches signed test tokens from jwks-mock instead of minting locally — the private key never leaves jwks-mock, eliminating the class of bugs where the consumer signs with a key that doesn't match the published JWKS.	—
pg-side (optional)	reused postgres-test connection on a side port	Side-channel DB connection for fixture seeding + post-call assertions	shares postgres-test

The only external services not running are the sibling backend services (annotations, detection, autopilot, flight-gate, Watchtower, suite reverse proxy) — their tables (media, annotations, detection, map_objects) are seeded directly by the side-channel for cascade tests; the services themselves are out of scope for service-level e2e.

The jwks-mock service replaces the pre-2026-05-14 "consumer mints HS256 tokens with a shared secret" pattern. The current code path (per Auth/JwtExtensions.cs) is ECDSA-SHA256 + JWKS — there is no shared secret to mint with anymore. See test-data.md § External Dependency Mocks for the mock's contract.

Networks

Network	Services	Purpose
e2e-net	missions, postgres-test, jwks-mock, e2e-consumer	Isolated bridge network; no host network access

Volumes

Volume	Mounted to	Purpose
pg-test-data	postgres-test:/var/lib/postgresql/data	Ephemeral; recreated per scenario class (docker compose down -v between class boundaries when the test asserts startup behavior)
e2e-results	e2e-consumer:/app/results and host ./e2e-results/	Output of report.csv

docker-compose structure

The canonical compose file is docker-compose.test.yml at the repo root. Below is the abbreviated structural outline — see the actual file for the full healthchecks, depends_on, and volume mounts.

services:
  postgres-test:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: azaion
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres-test
    # healthcheck: pg_isready

  jwks-mock:
    build: { context: tests/Azaion.Missions.JwksMock }
    environment:
      JWT_ISSUER: https://admin-test.azaion.local
      JWT_AUDIENCE: azaion-edge
      OLD_KEY_GRACE_SECONDS: 5
    # healthcheck: GET /.well-known/jwks.json over HTTPS (--no-check-certificate)

  missions:
    build: { context: . }
    environment:
      DATABASE_URL: postgresql://postgres:postgres-test@postgres-test:5432/azaion
      JWT_ISSUER: https://admin-test.azaion.local
      JWT_AUDIENCE: azaion-edge
      JWT_JWKS_URL: https://jwks-mock:8443/.well-known/jwks.json
      ASPNETCORE_ENVIRONMENT: Test    # NOT Production -- CORS falls back to permissive with a warning log line
    volumes:
      - ./tests/jwks-mock-ca.crt:/usr/local/share/ca-certificates/jwks-mock-ca.crt:ro
    depends_on:
      postgres-test:  { condition: service_healthy }
      jwks-mock:      { condition: service_healthy }

  e2e-consumer:
    build: { context: tests/Azaion.Missions.E2E.Tests }
    environment:
      MISSIONS_BASE_URL: http://missions:8080
      DB_SIDE_CHANNEL: Host=postgres-test;Port=5432;Database=azaion;Username=postgres;Password=postgres-test
      JWKS_MOCK_SIGN_URL: https://jwks-mock:8443/sign
      JWT_ISSUER: https://admin-test.azaion.local
      JWT_AUDIENCE: azaion-edge
    volumes:
      - ./test-results:/app/results
      - ./tests/jwks-mock-ca.crt:/usr/local/share/ca-certificates/jwks-mock-ca.crt:ro
    depends_on:
      missions:    { condition: service_healthy }
      jwks-mock:   { condition: service_healthy }

Production-gate (E9) variant: for the CORS production-gate lock test (E9), the test runner spawns missions with ASPNETCORE_ENVIRONMENT=Production and an empty CorsConfig:AllowedOrigins and asserts startup THROWS InvalidOperationException. This variant runs OUTSIDE the main compose stack via docker run to avoid disturbing the rest of the suite.

Consumer Application

Tech stack: xUnit 2.x + plain HttpClient against the dockerized missions service. Bogus 35.x for synthetic data. JWT acquisition via HTTPS POST jwks-mock:8443/sign (no in-process JWT library on the consumer side — the consumer treats tokens as opaque bearer strings). PostgreSQL side-channel via Npgsql (NOT linq2db — keep the consumer free of system-under-test runtime libs).

Entry point: dotnet test tests/Azaion.Missions.E2E.Tests/Azaion.Missions.E2E.Tests.csproj --logger "trx;LogFileName=results.trx" followed by a small post-processor that converts trx → report.csv.

Communication with system under test

Interface	Protocol	Endpoint	Authentication
Vehicle API	HTTP/1.1 JSON	http://missions:8080/vehicles[?name=&isDefault=] and /vehicles/{id}[/setDefault]	Authorization: Bearer <ECDSA-SHA256, iss=$JWT_ISSUER, aud=$JWT_AUDIENCE, permissions=FL>
Mission API	HTTP/1.1 JSON	http://missions:8080/missions[?name=&fromDate=&toDate=&page=&pageSize=]	same
Waypoint API	HTTP/1.1 JSON	http://missions:8080/missions/{id}/waypoints[/{wpId}]	same
Health	HTTP/1.1 JSON	http://missions:8080/health	anonymous
DB side-channel (assertions only)	TCP/Postgres wire	postgres-test:5432	postgres:postgres-test
JWKS mock sign endpoint	HTTPS/1.1 JSON	https://jwks-mock:8443/sign body { "iss":..., "aud":..., "exp":..., "permissions":..., ... } returns signed JWT	none (test-network internal only)
JWKS mock JWKS endpoint	HTTPS/1.1 JSON	https://jwks-mock:8443/.well-known/jwks.json	none (consumed by missions itself, not by the test consumer)
JWKS mock rotate-key endpoint	HTTPS/1.1 JSON	POST https://jwks-mock:8443/rotate-key body {} returns { "newKid": "..." } and starts the OldKeyGraceSeconds window	none

What the consumer does NOT have access to

• No using Azaion.Missions.*; — the consumer is a separate csproj with no project reference to the system under test.
• No AppDataConnection instantiation; the side-channel uses raw Npgsql NpgsqlCommand only.
• No file-system overlap; e2e-consumer is a separate container.
• No JWT signing key in the consumer process — the consumer requests signed tokens from jwks-mock via HTTPS POST /sign. The ECDSA private key never leaves the mock container. This guarantees the test setup cannot drift away from "consumer-signed token matches missions-cached JWKS public key".

CI/CD Integration

When to run: on every push to dev (Woodpecker pipeline .woodpecker/test-arm.yml and .woodpecker/test-amd.yml after the existing build-arm.yml job). Currently the repo has only build-arm.yml (per O4); the test runner pipeline is a follow-up artifact produced by Step 6 (Implement Tests) — see scripts/run-tests.sh (Phase 4).

Pipeline stage: post-build, pre-push (the test runner pulls the just-built azaion/missions:test tag).

Gate behavior: blocking on dev branch. Per O4, today's pipeline has no test stage; this gate is added by Step 6 implementation.

Timeout: max 15 minutes total wall-clock. Cascade-delete fixtures and the bootstrap-failure scenarios (AC-6.6, AC-6.7) dominate.

Reporting

Format: CSV Columns: TestId, TestName, Category, Traces, ExecutionTimeMs, Result, ErrorMessage Output path: ./e2e-results/report.csv

Categories: BLACKBOX, PERF, RES, SEC, RES_LIM. Traces is a comma-separated list of AC and restriction IDs from traceability-matrix.md.

Hardware Assessment

Filled by autodev /test-spec Hardware Assessment phase (2026-05-14).

Decision: Docker execution

The project is NOT hardware-dependent. Test execution is fully containerised; no local-mode runner is needed.

Hardware dependencies found: NONE

Documentation scan (restrictions.md, solution.md, architecture.md, components/*/description.md):

• H1–H6 cover edge-device deployment shape (Jetson Orin / OrangePI / operator-PC, multi-arch ARM64+AMD64, vertical scale only) but none of those concerns require hardware code paths inside the test runner — the suite-level CI matrix builds for both arches separately (per O4 + H2).
• No GPU / model inference / sensor / camera / GPIO / V4L2 mention in any docs.
• solution.md § 2.1 explicitly classifies every component as standard ASP.NET Core + linq2db over Postgres — no hardware adapter.

Code scan (*.csproj, Dockerfile, all .cs source):

• Azaion.Flights.csproj (post-B5: Azaion.Missions.csproj) packages: linq2db 6.2.0, Npgsql 10.0.2, Microsoft.AspNetCore.Authentication.JwtBearer 10.0.5, Swashbuckle.AspNetCore 10.1.5. None is hardware-specific.
• Dockerfile: stock mcr.microsoft.com/dotnet/sdk:10.0 build stage + mcr.microsoft.com/dotnet/aspnet:10.0 runtime stage. Multi-arch via --platform=$BUILDPLATFORM + dotnet publish --os linux --arch $arch. No runtime: nvidia, no GPU device mounts.
• No RuntimeInformation.IsOSPlatform, no coreml / cuda / gpio / v4l2 / opencl / vulkan / tpu / fpga references in any production source file (verified via grep — matches were only in skill templates and docs, not in Controllers/, Services/, Database/, Auth/, Middleware/, Program.cs).

Multi-arch matters for the production deploy (per H2), but it does NOT affect the test runner: tests exercise the API black-box, identical on both architectures. The suite CI matrix (.woodpecker/build-arm.yml + the future .woodpecker/build-amd.yml) tests each arch in its own container.

Execution instructions — Docker mode (the only mode)

Prerequisites on the test host:

• Docker Engine ≥ 24.0 with Docker Compose v2 plugin.
• 2.5 GB free RAM (1 GB for missions, 256 MB for postgres-test, 256 MB for jwks-mock, 512 MB for e2e-consumer, 512 MB headroom).
• 2 CPU cores recommended for the test wall-clock to fit under the 15-minute CI gate.
• Free TCP ports 5002 (host → missions:8080) and 5433 (host → postgres-test:5432) — used only when running outside compose. The jwks-mock HTTPS port (8443 inside the container) is NOT mapped to the host; only missions and e2e-consumer reach it via the internal e2e-net network.

Run command (from repo root):

./scripts/run-tests.sh

The runner script (Phase 4) wires up docker compose up, waits for missions health, executes dotnet test inside e2e-consumer, collects report.csv, and tears down the compose stack. See scripts/run-tests.sh for the canonical command sequence.

Performance tests:

./scripts/run-performance-tests.sh

Same compose stack, but with the perf seed (1000 missions for NFT-PERF-04, 100 minimal missions for NFT-PERF-01) and the [Trait("Category","Perf")] filter. See scripts/run-performance-tests.sh (Phase 4).

Resource ceiling

Total RAM: ≤ 2.5 GB for missions + postgres-test + jwks-mock + e2e-consumer together. Test wall-clock budget: ≤ 15 minutes (CI gate). Storage: ephemeral (the pg-test-data tmpfs is recreated per scenario class via docker compose down -v for the bootstrap-sensitive scenarios — NFT-RES-03, NFT-RES-04, NFT-RES-05, NFT-RES-06, NFT-RES-07).