ui/_docs/02_document/tests/environment.md

# Test Environment

## Overview

**System under test**: the Azaion UI single-page application — a React 19 + Vite 6 static bundle served by `nginx:alpine` (port 80) that talks to the parent suite microservices through the same nginx instance (reverse-proxied `/api/<service>/` routes per `nginx.conf`). The SPA's observable surface is everything reachable from a browser: outgoing HTTP requests (URL, method, headers, body), incoming responses (rendered DOM, console, errors), outgoing EventSource streams, browser storage (`localStorage` / `sessionStorage` / `document.cookie`), and the built artifact (`dist/`).

**Consumer app purpose**: the test runners are the consumer. They drive a real browser (or jsdom) at the SPA's public surface, capture every outbound request/event, and assert against `_docs/00_problem/input_data/expected_results/results_report.md` (95 rows).

**Black-box discipline**: the consumer MUST NOT import from `src/` (except for the typed enum shapes that ARE part of the wire contract per `P9`), MUST NOT bypass the React tree to call internal hooks, and MUST NOT inspect React component state directly. Assertions are made on the rendered DOM, ARIA roles, outgoing network activity, EventSource state machine, console output, and built artifacts.

## Test Execution Profiles

Two profiles share the artifact directory but address different black-box levels. Runner selection is deferred to the Decompose Tests step (`autodev` Step 5) — this document specifies the **environment requirements**, not the runner choice.

| Profile | Scope | Black-box level | Backing services | Browser |
|---------|-------|----------------|------------------|---------|
| `fast` | Unit + component + static checks | DOM + network requests issued by a mounted component or by a code-level helper, captured at the `fetch` / `EventSource` boundary | Stubbed (request interception layer, e.g. MSW or equivalent). No real services. | jsdom or headless Chromium (component renderer). |
| `e2e` | Browser smoke + cross-service flows | Real browser → real nginx (UI image) → real suite docker-compose stack | Full suite docker-compose stack (`admin/`, `flights/`, `annotations/`, `detect/`, `gps-denied-desktop/`, `gps-denied-onboard/`, `autopilot/`, `resource/`, `loader/`). | Headless Chromium + Firefox latest 2 versions per AC-18. |
| `static` | Source / config / bundle checks | The repo + the `dist/` artifact | None (no runtime). | None (CLI). |

Tests in `blackbox-tests.md`, `performance-tests.md`, etc. tag themselves with `Profile: fast | e2e | static` to make runner routing unambiguous.

## Docker Environment

The Azaion UI image carries no DB. The "Docker environment" is the test-time choreography of UI + suite services + stubs.

### Services (e2e profile)

| Service | Image / Build | Purpose | Ports |
|---------|--------------|---------|-------|
| `azaion-ui` | Built from this repo (`Dockerfile`, ARM64 per H1 / S5) — final stage `nginx:alpine` | The SPA under test | `80` |
| `admin` | Suite `admin/` image (auth + users + classes write + GPS settings) | Auth + RBAC; cookie issuer per E3 | per suite compose |
| `flights` | Suite `flights/` image | Flight CRUD + waypoints + aircraft + live-GPS SSE | per suite compose |
| `annotations` | Suite `annotations/` image | Media + annotations + dataset + class read + settings + status SSE | per suite compose |
| `detect` | Suite `detect/` image | Sync image detect (and future async video detect F7) | per suite compose |
| `gps-denied-desktop`, `gps-denied-onboard`, `autopilot`, `resource`, `loader` | Suite microservice images | Auxiliary services hit by the SPA (only `loader/` and `resource/` are hit on production paths today; `gps-denied-*` is target-only F12) | per suite compose |
| `owm-stub` | Tiny HTTP server returning canned OpenWeatherMap responses | Replace direct OWM HTTPS (E10) so tests are deterministic and rate-limit-free | `8081` |
| `tile-stub` | Tiny HTTP server returning a 256x256 PNG | Replace OSM tile servers | `8082` |
| `test-db` | Suite-managed (Postgres per suite default) | Backs `admin/`, `flights/`, `annotations/` | Internal |

### Networks

| Network | Services | Purpose |
|---------|----------|---------|
| `azaion-test-net` | all of the above | Isolated test network; no internet egress (OWM + tile stubs replace the only external hops). |

### Volumes

| Volume | Mounted to | Purpose |
|--------|-----------|---------|
| `test-db-data` | `test-db:/var/lib/postgresql/data` | Suite DB persistence — wiped between e2e runs (see Data Isolation below). |
| `seed-fixtures` | `admin:/seed`, `flights:/seed`, `annotations:/seed` (read-only) | Bootstrap data loaded at service start (users, flights, classes, sample media). See `test-data.md`. |
| `test-output` | `playwright-runner:/output` | Where the consumer writes CSV reports, screenshots, traces. |

### docker-compose structure (outline)

```yaml
services:
  azaion-ui:
    build: .
    ports: ["80:80"]
    depends_on: [admin, flights, annotations, detect]
    environment:
      AZAION_REVISION: ${CI_COMMIT_SHA:-test}

  admin:
    image: azaion/admin:test
    depends_on: [test-db]

  flights:
    image: azaion/flights:test
    depends_on: [test-db]

  annotations:
    image: azaion/annotations:test
    depends_on: [test-db]

  detect:
    image: azaion/detect:test

  test-db:
    image: postgres:16-alpine

  owm-stub:
    build: ./testing/stubs/owm
  tile-stub:
    build: ./testing/stubs/tile

  playwright-runner:
    build: ./testing/runner
    depends_on: [azaion-ui]
    environment:
      BASE_URL: http://azaion-ui:80
      OWM_BASE_URL: http://owm-stub:8081
      TILE_BASE_URL: http://tile-stub:8082
```

The compose file is part of the test-spec output; its concrete shape lands when the Decompose Tests step picks the runner (Step 5).

## Consumer Application

### `fast` profile

**Tech stack** (target — chosen at Step 5): a component-testing harness in TypeScript (Vitest or Jest + React Testing Library) plus a request-interception layer (MSW or equivalent) and jsdom (or headless Chromium component renderer).

**Entry point**: `npm run test:fast` (or `bun test`) — runs all `*.test.ts(x)` files under the test root.

#### Communication with system under test

| Interface | Protocol | Endpoint / Topic | Authentication |
|-----------|----------|-----------------|----------------|
| Mounted React component under test | direct mount via the testing library | n/a — observe the DOM + outbound requests captured by MSW | Stubbed bearer / cookie in test helpers |
| Outgoing `fetch` (under test) | HTTP via MSW handlers | mock `/api/<service>/...` per test | per handler |
| Outgoing `EventSource` (under test) | SSE via MSW or `EventSourcePolyfill` test double | mock `/api/<service>/...` per test | bearer in query string (ADR-008) |
| Static check | `bun run` script + filesystem regex (e.g. via `ripgrep`) | n/a | n/a |

### `e2e` profile

**Tech stack** (target — chosen at Step 5): Playwright (Chromium + Firefox per AC-18) driving the deployed `azaion-ui` nginx; assertion library is the runner's built-in expectations + a small request-interception adapter that logs every outbound request for assertion.

**Entry point**: `bun run test:e2e` — runs all `*.e2e.ts` files under the test root against the live compose stack.

#### Communication with system under test

| Interface | Protocol | Endpoint / Topic | Authentication |
|-----------|----------|-----------------|----------------|
| Browser navigation | HTTPS | `${BASE_URL}/login`, `/flights`, `/annotations`, `/dataset`, `/admin`, `/settings` | login via the public `/login` flow |
| Suite REST | HTTPS via SPA's nginx proxy | `/api/admin/*`, `/api/flights/*`, `/api/annotations/*`, `/api/detect/*`, `/api/loader/*`, `/api/resource/*`, `/api/gps-denied-{desktop,onboard}/*`, `/api/autopilot/*` | bearer in `Authorization` header + cookie (HttpOnly) |
| Suite SSE | HTTPS | `/api/flights/<id>/live-gps`, `/api/annotations/annotations/events`, `/api/detect/stream/<jobId>` (F7 target) | bearer in `?token=` per ADR-008 |
| Bundle / image inspection | filesystem / `docker inspect` | n/a | n/a |
| OpenWeatherMap | HTTPS via `owm-stub` | per stub | none |
| OSM tiles | HTTPS via `tile-stub` | per stub | none |

### What the consumer does NOT have access to

- No direct DB access to `test-db`. Suite DB queries are forbidden from the test runner; the consumer asserts only through the suite's REST + SSE.
- No internal `src/` imports beyond the typed enum shapes that ARE part of the wire contract (`AnnotationStatus`, `MediaStatus`, `Affiliation`, `CombatReadiness`, `MediaType`, `AnnotationSource` per `data_parameters.md` §1) — these are the spec the test asserts against per `P9`.
- No React component state read via hooks or test-only escape hatches; only the DOM + outbound network surface is observable.
- No shared memory or filesystem with the SPA.

## CI/CD Integration

**When to run**:

- `fast` profile: on every commit (planned addition to `.woodpecker/build-arm.yml`; currently absent — O14).
- `e2e` profile: on PR merge to `dev` / `stage` and pre-release on `main`. Long-running; not gating regular commits.
- `static` profile: on every commit (lints + bundle / config checks run as part of the build).

**Pipeline stage**:

- `fast` + `static`: between `bun install` and `bun run build`.
- `e2e`: after `bun run build`, against the just-built image, in a separate compose job.

**Gate behavior**:

- `fast` + `static`: block merge on failure.
- `e2e`: block merge on failure for `dev` / `stage`. On `main`, manual approval is allowed for known-quarantined tests (e.g., the Phase B target tests for AC-11 / AC-24 / AC-40 that assert "when implemented").

**Timeout**: `fast` ≤ 5 min suite total. `e2e` ≤ 30 min suite total. Individual tests timeout per the `Max execution time` field in each scenario.

## Reporting

**Format**: CSV (and JUnit XML for CI consumption when the runner produces it).

**Columns**: `Test ID, Test Name, Profile, Execution Time (ms), Result (PASS|FAIL|SKIP|QUARANTINE), Error Message, Traces to AC, Traces to results_report.md row`.

**Output path**: `./test-output/report.csv` (mounted from the `playwright-runner` / `vitest-runner` container). For `static` checks, `./test-output/static-report.csv`. Suite-level rollup written to `./test-output/summary.csv`.

## Test Execution

**Decision**: **Docker (preferred)** for the `e2e` and `static` profiles; **local Bun** for the `fast` profile and as an option for `e2e` on developer machines that already have Playwright + the suite stack running. The project is **not hardware-dependent** — see "Hardware dependencies found" below.

### Hardware dependencies found

| Indicator | Found at | Verdict |
|-----------|----------|---------|
| GPU / CUDA imports | none in `src/` or `mission-planner/` | absent |
| CoreML / MPS imports | none | absent |
| Camera / sensor / GPIO / V4L2 | none | absent — the SPA reads `<video>` elements rendered from a server-supplied URL |
| OS-specific drivers / kernel modules | none | absent |
| Platform-gated source branches | none | absent |
| Spec-level constraint | `_docs/00_problem/restrictions.md` H3 ("No GPU expectation in UI image") + H4 ("Chromium / Firefox latest 2") | confirms platform-neutral browser surface |

Conclusion: classify as **Not hardware-dependent**. Docker headless Chromium reproduces the real production runtime; no real-hardware execution path is required.

### Execution instructions

#### Docker mode (preferred; CI default)

1. **Prerequisites**: Docker Engine 24+ with the `azaion-test-net` network reachable, ARM64 or amd64 host (the UI image is ARM64-only per H1 — CI runners on ARM64; multi-arch builds optional for local dev).
2. **Build**: `docker buildx build --platform linux/arm64 -t azaion-ui:test .` (or `--platform linux/amd64` on amd64 dev machines).
3. **Compose up**: `docker compose -f e2e/docker-compose.suite-e2e.yml up -d` — brings up `azaion-ui`, `admin`, `flights`, `annotations`, `detect`, the auxiliary services, `owm-stub`, `tile-stub`, `test-db`, and the `playwright-runner`.
4. **Run tests**: `docker compose -f e2e/docker-compose.suite-e2e.yml run --rm playwright-runner` — the runner image entrypoint is `bun run test:e2e`. Reports land in `./test-output/`.
5. **Tear down**: `docker compose -f e2e/docker-compose.suite-e2e.yml down -v` (volumes wiped between runs).
6. **Required environment**: `BASE_URL=http://azaion-ui:80`, `OWM_BASE_URL=http://owm-stub:8081`, `TILE_BASE_URL=http://tile-stub:8082`, `CI_COMMIT_SHA=<sha>` (stamped into `AZAION_REVISION`).

#### Local mode (for `fast` profile + developer-machine `e2e` runs)

1. **Prerequisites**: Bun 1.3.11 (matches S4), Chromium + Firefox latest two stable lines installed via Playwright (`bunx playwright install --with-deps chromium firefox`).
2. **`fast` profile**: `bun install && bun run test:fast` (alias for `bun test`, runs Vitest under jsdom plus MSW handlers).
3. **`e2e` profile (local)**: bring up the suite stack via the parent suite's compose file (`../docker-compose.yml`), point `BASE_URL=http://localhost:80`, then `bun run test:e2e`.
4. **Required environment**: same as Docker mode plus `OWM_API_KEY=test-key` (passed through the OWM stub).

#### CI runner mapping

| Profile | Runner type | Mode | Gate |
|---------|------------|------|------|
| `static` | ARM64 build runner | local (no browser) | block merge on failure |
| `fast` | ARM64 build runner | local Bun (jsdom + MSW) | block merge on failure |
| `e2e` | ARM64 e2e runner with Docker | Docker compose stack | block merge on failure for `dev`/`stage`; manual approval allowed for quarantined tests on `main` (per CI/CD Integration above) |

The decision is consumed by Phase 4 to choose between `scripts/run-tests.sh` (local Bun for `fast` + `static`) and `e2e/docker-compose.suite-e2e.yml` (Docker for `e2e`).