ui/_docs/02_document/tests/environment.md

Test Environment

Overview

System under test: the Azaion UI single-page application — a React 19 + Vite 6 static bundle served by nginx:alpine (port 80) that talks to the parent suite microservices through the same nginx instance (reverse-proxied /api/<service>/ routes per nginx.conf). The SPA's observable surface is everything reachable from a browser: outgoing HTTP requests (URL, method, headers, body), incoming responses (rendered DOM, console, errors), outgoing EventSource streams, browser storage (localStorage / sessionStorage / document.cookie), and the built artifact (dist/).

Consumer app purpose: the test runners are the consumer. They drive a real browser (or jsdom) at the SPA's public surface, capture every outbound request/event, and assert against _docs/00_problem/input_data/expected_results/results_report.md (95 rows).

Black-box discipline: the consumer MUST NOT import from src/ (except for the typed enum shapes that ARE part of the wire contract per P9), MUST NOT bypass the React tree to call internal hooks, and MUST NOT inspect React component state directly. Assertions are made on the rendered DOM, ARIA roles, outgoing network activity, EventSource state machine, console output, and built artifacts.

Test Execution Profiles

Two profiles share the artifact directory but address different black-box levels. Runner selection is deferred to the Decompose Tests step (autodev Step 5) — this document specifies the environment requirements, not the runner choice.

Profile	Scope	Black-box level	Backing services	Browser
fast	Unit + component + static checks	DOM + network requests issued by a mounted component or by a code-level helper, captured at the fetch / EventSource boundary	Stubbed (request interception layer, e.g. MSW or equivalent). No real services.	jsdom or headless Chromium (component renderer).
e2e	Browser smoke + cross-service flows	Real browser → real nginx (UI image) → real suite docker-compose stack	Full suite docker-compose stack (admin/, flights/, annotations/, detect/, gps-denied-desktop/, gps-denied-onboard/, autopilot/, resource/, loader/).	Headless Chromium + Firefox latest 2 versions per AC-18.
static	Source / config / bundle checks	The repo + the dist/ artifact	None (no runtime).	None (CLI).

Tests in blackbox-tests.md, performance-tests.md, etc. tag themselves with Profile: fast | e2e | static to make runner routing unambiguous.

Docker Environment

The Azaion UI image carries no DB. The "Docker environment" is the test-time choreography of UI + suite services + stubs.

Services (e2e profile)

Service	Image / Build	Purpose	Ports
azaion-ui	Built from this repo (Dockerfile, ARM64 per H1 / S5) — final stage nginx:alpine	The SPA under test	80
admin	Suite admin/ image (auth + users + classes write + GPS settings)	Auth + RBAC; cookie issuer per E3	per suite compose
flights	Suite flights/ image	Flight CRUD + waypoints + aircraft + live-GPS SSE	per suite compose
annotations	Suite annotations/ image	Media + annotations + dataset + class read + settings + status SSE	per suite compose
detect	Suite detect/ image	Sync image detect (and future async video detect F7)	per suite compose
gps-denied-desktop, gps-denied-onboard, autopilot, resource, loader	Suite microservice images	Auxiliary services hit by the SPA (only loader/ and resource/ are hit on production paths today; gps-denied-* is target-only F12)	per suite compose
owm-stub	Tiny HTTP server returning canned OpenWeatherMap responses	Replace direct OWM HTTPS (E10) so tests are deterministic and rate-limit-free	8081
tile-stub	Tiny HTTP server serving GET /tiles/{z}/{x}/{y} → 256x256 JPEG with Content-Type: image/jpeg, Cache-Control, and ETag headers (mirrors the satellite-provider contract at _docs/02_document/contracts/satellite-provider/tiles.md)	Replace the suite's satellite-provider tile endpoint in the e2e profile (since cycle 2 / AZ-498). The stub does NOT enforce cookie auth — the same-origin cookie path is exercised once the cross-workspace satellite-provider cookie-auth ticket lands and tile traffic flows through the real service.	8082
test-db	Suite-managed (Postgres per suite default)	Backs admin/, flights/, annotations/	Internal

Networks

Network	Services	Purpose
azaion-test-net	all of the above	Isolated test network; no internet egress (owm-stub + tile-stub replace the only external hops — OWM HTTPS, and since cycle 2 / AZ-498 the suite's own satellite-provider /tiles/{z}/{x}/{y} endpoint stands in for the previously-used external OSM/Esri tile servers).

Volumes

Volume	Mounted to	Purpose
test-db-data	test-db:/var/lib/postgresql/data	Suite DB persistence — wiped between e2e runs (see Data Isolation below).
seed-fixtures	admin:/seed, flights:/seed, annotations:/seed (read-only)	Bootstrap data loaded at service start (users, flights, classes, sample media). See test-data.md.
test-output	playwright-runner:/output	Where the consumer writes CSV reports, screenshots, traces.

docker-compose structure (outline)

services:
  azaion-ui:
    build: .
    ports: ["80:80"]
    depends_on: [admin, flights, annotations, detect]
    environment:
      AZAION_REVISION: ${CI_COMMIT_SHA:-test}

  admin:
    image: azaion/admin:test
    depends_on: [test-db]

  flights:
    image: azaion/flights:test
    depends_on: [test-db]

  annotations:
    image: azaion/annotations:test
    depends_on: [test-db]

  detect:
    image: azaion/detect:test

  test-db:
    image: postgres:16-alpine

  owm-stub:
    build: ./testing/stubs/owm
  tile-stub:
    build: ./testing/stubs/tile

  playwright-runner:
    build: ./testing/runner
    depends_on: [azaion-ui]
    environment:
      BASE_URL: http://azaion-ui:80
      OWM_BASE_URL: http://owm-stub:8081
      VITE_SATELLITE_TILE_URL: "http://tile-stub:8082/tiles/{z}/{x}/{y}"

The compose file is part of the test-spec output; its concrete shape lands when the Decompose Tests step picks the runner (Step 5).

Consumer Application

fast profile

Tech stack (target — chosen at Step 5): a component-testing harness in TypeScript (Vitest or Jest + React Testing Library) plus a request-interception layer (MSW or equivalent) and jsdom (or headless Chromium component renderer).

Entry point: npm run test:fast (or bun test) — runs all *.test.ts(x) files under the test root.

Communication with system under test

Interface	Protocol	Endpoint / Topic	Authentication
Mounted React component under test	direct mount via the testing library	n/a — observe the DOM + outbound requests captured by MSW	Stubbed bearer / cookie in test helpers
Outgoing fetch (under test)	HTTP via MSW handlers	mock /api/<service>/... per test	per handler
Outgoing EventSource (under test)	SSE via MSW or EventSourcePolyfill test double	mock /api/<service>/... per test	bearer in query string (ADR-008)
Static check	bun run script + filesystem regex (e.g. via ripgrep)	n/a	n/a

e2e profile

Tech stack (target — chosen at Step 5): Playwright (Chromium + Firefox per AC-18) driving the deployed azaion-ui nginx; assertion library is the runner's built-in expectations + a small request-interception adapter that logs every outbound request for assertion.

Entry point: bun run test:e2e — runs all *.e2e.ts files under the test root against the live compose stack.

Communication with system under test

Interface	Protocol	Endpoint / Topic	Authentication
Browser navigation	HTTPS	${BASE_URL}/login, /flights, /annotations, /dataset, /admin, /settings	login via the public /login flow
Suite REST	HTTPS via SPA's nginx proxy	/api/admin/*, /api/flights/*, /api/annotations/*, /api/detect/*, /api/loader/*, /api/resource/*, /api/gps-denied-{desktop,onboard}/*, /api/autopilot/*	bearer in Authorization header + cookie (HttpOnly)
Suite SSE	HTTPS	/api/flights/<id>/live-gps, /api/annotations/annotations/events, /api/detect/stream/<jobId> (F7 target)	bearer in ?token= per ADR-008
Bundle / image inspection	filesystem / docker inspect	n/a	n/a
OpenWeatherMap	HTTPS via owm-stub	per stub	none
Satellite tiles	HTTPS via tile-stub (replacing the suite's own satellite-provider /tiles/{z}/{x}/{y} endpoint in the e2e profile)	per stub at /tiles/{z}/{x}/{y}	none in stub; production uses an HttpOnly same-origin cookie set by admin/ (see crossOrigin="use-credentials" on every <TileLayer> per cycle 2 / AZ-498)

What the consumer does NOT have access to

• No direct DB access to test-db. Suite DB queries are forbidden from the test runner; the consumer asserts only through the suite's REST + SSE.
• No internal src/ imports beyond the typed enum shapes that ARE part of the wire contract (AnnotationStatus, MediaStatus, Affiliation, CombatReadiness, MediaType, AnnotationSource per data_parameters.md §1) — these are the spec the test asserts against per P9.
• No React component state read via hooks or test-only escape hatches; only the DOM + outbound network surface is observable.
• No shared memory or filesystem with the SPA.

CI/CD Integration

When to run:

• fast profile: on every commit (planned addition to .woodpecker/build-arm.yml; currently absent — O14).
• e2e profile: on PR merge to dev / stage and pre-release on main. Long-running; not gating regular commits.
• static profile: on every commit (lints + bundle / config checks run as part of the build).

Pipeline stage:

• fast + static: between bun install and bun run build.
• e2e: after bun run build, against the just-built image, in a separate compose job.

Gate behavior:

• fast + static: block merge on failure.
• e2e: block merge on failure for dev / stage. On main, manual approval is allowed for known-quarantined tests (e.g., the Phase B target tests for AC-11 / AC-24 / AC-40 that assert "when implemented").

Timeout: fast ≤ 5 min suite total. e2e ≤ 30 min suite total. Individual tests timeout per the Max execution time field in each scenario.

Reporting

Format: CSV (and JUnit XML for CI consumption when the runner produces it).

Columns: Test ID, Test Name, Profile, Execution Time (ms), Result (PASS|FAIL|SKIP|QUARANTINE), Error Message, Traces to AC, Traces to results_report.md row.

Output path: ./test-output/report.csv (mounted from the playwright-runner / vitest-runner container). For static checks, ./test-output/static-report.csv. Suite-level rollup written to ./test-output/summary.csv.

Test Execution

Decision: Docker (preferred) for the e2e and static profiles; local Bun for the fast profile and as an option for e2e on developer machines that already have Playwright + the suite stack running. The project is not hardware-dependent — see "Hardware dependencies found" below.

Hardware dependencies found

Indicator	Found at	Verdict
GPU / CUDA imports	none in src/ or mission-planner/	absent
CoreML / MPS imports	none	absent
Camera / sensor / GPIO / V4L2	none	absent — the SPA reads <video> elements rendered from a server-supplied URL
OS-specific drivers / kernel modules	none	absent
Platform-gated source branches	none	absent
Spec-level constraint	_docs/00_problem/restrictions.md H3 ("No GPU expectation in UI image") + H4 ("Chromium / Firefox latest 2")	confirms platform-neutral browser surface

Conclusion: classify as Not hardware-dependent. Docker headless Chromium reproduces the real production runtime; no real-hardware execution path is required.

Execution instructions

Docker mode (preferred; CI default)

1. Prerequisites: Docker Engine 24+ with the azaion-test-net network reachable, ARM64 or amd64 host (the UI image is ARM64-only per H1 — CI runners on ARM64; multi-arch builds optional for local dev).
2. Build: docker buildx build --platform linux/arm64 -t azaion-ui:test . (or --platform linux/amd64 on amd64 dev machines).
3. Compose up: docker compose -f e2e/docker-compose.suite-e2e.yml up -d — brings up azaion-ui, admin, flights, annotations, detect, the auxiliary services, owm-stub, tile-stub, test-db, and the playwright-runner.
4. Run tests: docker compose -f e2e/docker-compose.suite-e2e.yml run --rm playwright-runner — the runner image entrypoint is bun run test:e2e. Reports land in ./test-output/.
5. Tear down: docker compose -f e2e/docker-compose.suite-e2e.yml down -v (volumes wiped between runs).
6. Required environment: BASE_URL=http://azaion-ui:80, OWM_BASE_URL=http://owm-stub:8081, VITE_SATELLITE_TILE_URL=http://tile-stub:8082/tiles/{z}/{x}/{y} (since cycle 2 / AZ-498 — was TILE_BASE_URL=http://tile-stub:8082), CI_COMMIT_SHA=<sha> (stamped into AZAION_REVISION).

Local mode (for fast profile + developer-machine e2e runs)

1. Prerequisites: Bun 1.3.11 (matches S4), Chromium + Firefox latest two stable lines installed via Playwright (bunx playwright install --with-deps chromium firefox).
2. fast profile: bun install && bun run test:fast (alias for bun test, runs Vitest under jsdom plus MSW handlers).
3. e2e profile (local): bring up the suite stack via the parent suite's compose file (../docker-compose.yml), point BASE_URL=http://localhost:80, then bun run test:e2e.
4. Required environment: same as Docker mode plus OWM_API_KEY=test-key (passed through the OWM stub).

CI runner mapping

Profile	Runner type	Mode	Gate
static	ARM64 build runner	local (no browser)	block merge on failure
fast	ARM64 build runner	local Bun (jsdom + MSW)	block merge on failure
e2e	ARM64 e2e runner with Docker	Docker compose stack	block merge on failure for dev/stage; manual approval allowed for quarantined tests on main (per CI/CD Integration above)

The decision is consumed by Phase 4 to choose between scripts/run-tests.sh (local Bun for fast + static) and e2e/docker-compose.suite-e2e.yml (Docker for e2e).