Captures the full output of autodev existing-code Phase A through Step 4 (Code Testability Revision) for the Azaion UI workspace: - Step 1 Document: _docs/02_document/ (FINAL_report, architecture, glossary, components/, modules/, diagrams/, system-flows, module-layout) plus _docs/00_problem/ + _docs/01_solution/ + _docs/legacy/ + _docs/how_to_test + README. - Step 2 Architecture Baseline: architecture_compliance_baseline.md. - Step 3 Test Spec: _docs/02_document/tests/ (environment, test-data, blackbox/performance/resilience/security/ resource-limit tests, traceability-matrix), enum_spec_snapshot, expected_results/results_report.md (98 rows), plus the run-tests.sh + run-performance-tests.sh runners. - Step 4 Code Testability Revision: 01-testability-refactoring/ run dir (list-of-changes C01-C07, deferred_to_refactor, analysis/research_findings + refactoring_roadmap) and the 7 child task specs AZ-448..AZ-454 under _docs/02_tasks/todo/ plus _dependencies_table.md. - _docs/_autodev_state.md pins the cursor at Step 4 / refactor Phase 4 entry so /autodev resumes cleanly. Epic AZ-447 (UI testability gates) tracks the 7 child tasks that will land in subsequent commits. Co-authored-by: Cursor <cursoragent@cursor.com>
14 KiB
Test Data Management
The Azaion UI is a thin client over a typed REST + SSE contract — it carries no database of its own (P3 / P4 / data_parameters.md). "Test data" therefore means three things:
- Suite-side seed fixtures loaded into the
e2eprofile's docker-compose stack (users, flights, aircraft, classes, sample media). - Stub responses mounted on the
fastprofile's request-interception layer (canned/api/<service>/*payloads, canned OpenWeatherMap and OSM tile responses). - Source-level fixtures committed in this repo: i18n bundles, enum spec snapshots,
nginx.conf,vite.config.ts,package.json, and thedist/build output.
There are NO image / video / .tlog input files under _docs/00_problem/input_data/ for this project — the cell data_parameters.md and results_report.md already make this explicit. Every observable assertion lives in _docs/00_problem/input_data/expected_results/results_report.md (95 rows) which the tests reference verbatim.
Seed Data Sets
| Data Set | Description | Used by Tests | How Loaded | Cleanup |
|---|---|---|---|---|
seed_users |
4 users: op_alice (Operator), op_bob (Operator, no SETTINGS permission), admin_carol (Admin), integrator_dave (System Integrator). All with known passwords for test login. |
All e2e tests that authenticate |
admin/ service init script reads seed-fixtures/users.json at container start |
docker compose down -v wipes test-db-data between e2e suite runs |
seed_aircraft |
3 aircraft with one marked isDefault: true |
E2E tests touching /flights, /admin aircraft tab |
flights/ service init script |
as above |
seed_flights |
5 flights spanning the 4 users; some with waypoints, one with LiveGpsEvent simulator wired |
E2E tests touching /flights and the live-GPS stream |
flights/ service init script |
as above |
seed_classes |
The contract's [0..N-1, 20..20+N-1, 40..40+N-1] ordering (per AC-37 / data_model.md:158) — N≥9 so number-key hotkeys 1..9 are all hot. |
<DetectionClasses> component + <AdminPage> class CRUD tests |
annotations/ (read path) + admin/ (write path) init scripts |
as above |
seed_media |
6 media items (3 images, 3 videos) attached to seed_flights, with mediaStatus values exercising the full enum after AC-04 fix lands (None, New, AiProcessing, AiProcessed, ManualCreated, Confirmed, Error) |
E2E tests touching /annotations and /dataset |
annotations/ init script |
as above |
seed_annotations |
Annotations for seed_media, including: some with Source: AI, some Manual; one with isSplit: true and a valid splitTile "3 0.5 0.5 0.2 0.2" (AC-39); one with malformed splitTile ("garbage", AC-39 sad path); status values spanning the full AnnotationStatus enum after AC-04 fix (None=0, Created=10, Edited=20, Validated=30, Deleted=40) |
E2E tests for <AnnotationsPage>, <CanvasEditor>, <DatasetPage> |
annotations/ init script |
as above |
seed_user_settings |
Known selectedFlightId and panelWidths for op_alice so the rehydration tests assert against a deterministic state |
AC-21, AC-06 tests | annotations/ init script |
as above |
enum_spec_snapshot |
A committed JSON file _docs/00_problem/input_data/enum_spec_snapshot.json that pins the contract values for AnnotationStatus, MediaStatus, Affiliation, CombatReadiness, MediaType, AnnotationSource, WaypointSource, WaypointObjective. Populated from ../_docs/00_database_schema.md (authoritative) with the UI's current drift documented in ui_drift_summary. CombatReadiness + MediaType are flagged verification_pending: true because the schema does not pin numeric values; Step 4 .NET-service inspection finalizes them. |
Static checks for AC-04 / AC-29 (Group 2 rows 14-19 of results_report.md) |
Read directly from the repo at test time. | n/a (committed at Phase 3). |
bundle_artifact |
The dist/ produced by vite build. |
AC-11, AC-31, AC-33 (Group 7 rows 40-44) | Built by the CI step before tests run. | Ephemeral per CI run. |
Data Isolation Strategy
fastprofile: each test creates its own MSW handlers and tears them down in the test runner'safterEach. No process-wide state. React tree is unmounted between tests. Test files MAY share an enum spec snapshot (read-only) since it is a contract pin.e2eprofile: eache2erun gets a fresh suite docker-compose stack (docker compose down -vthenup -d) before the suite executes. Inside a run, tests group into isolation buckets by data set; a bucket runs sequentially against a known seed state, then the seeds are reset between buckets viaadmin/service'sPOST /test-only/reset(a test-only endpoint, gated behind a non-production build flag). Buckets across different machines run on independent compose stacks.- Per-user isolation: tests that mutate user-scoped state (e.g.,
selectedFlightId, panel widths) use distinct seed users so concurrent tests within a bucket cannot race on the sameUserSettingsrow. - No cross-test order dependencies — any test must be re-runnable in isolation by booting the bucket's seed snapshot.
Input Data Mapping
The UI has no consumer-side input files (images, videos, .tlogs). Every test "input" is a trigger classified per test-spec/SKILL.md "Behavioral shape" — a user action, a request the SPA initiates, an SSE event, or a static check against the repo / dist/. The Input column of each test in blackbox-tests.md / etc. references a results_report.md row (which already defines the trigger and the quantifiable observable).
| Input Reference | Source Location | Description | Covers Scenarios |
|---|---|---|---|
results_report.md rows 1-13 |
_docs/00_problem/input_data/expected_results/results_report.md |
Auth & token-handling triggers | Group 1 tests (FT-P-01..FT-P-13 / FT-N-* per blackbox-tests.md) |
results_report.md rows 14-21 |
as above | Wire-contract / enum compliance triggers | Group 2 tests |
results_report.md rows 22-31 |
as above | Annotations endpoint / payload / SSE / overlay-window triggers | Group 3 tests |
results_report.md rows 32-35 |
as above | Flight selection persistence + live-GPS SSE triggers | Group 4 tests |
results_report.md rows 36-37 |
as above | Dataset bulk-validate triggers | Group 5 tests |
results_report.md rows 38-39 |
as above | Upload size cap triggers | Group 6 tests |
results_report.md rows 40-44 |
as above | Build / bundle / routing triggers | Group 7 tests |
results_report.md rows 45-48 |
as above | i18n triggers | Group 8 tests |
results_report.md rows 49-51 |
as above | Destructive-UX triggers | Group 9 tests |
results_report.md rows 52-59 |
as above | A11y triggers | Group 10 tests |
results_report.md rows 60-62 |
as above | Browser-support + responsive triggers | Group 11 tests |
results_report.md row 63 |
as above | OWM secrets check | Group 12 test |
results_report.md rows 64-65 |
as above | User-settings persistence triggers | Group 13 tests |
results_report.md rows 66-69 |
as above | Form hygiene triggers | Group 14 tests |
results_report.md rows 70-72 |
as above | CI / image / labels triggers | Group 15 tests |
results_report.md rows 73-84 |
as above | Canvas + DetectionClasses + PhotoMode triggers | Group 16 tests |
results_report.md rows 85-90 |
as above | Tile splitting + tile-zoom triggers | Group 17 tests |
results_report.md rows 91-95 |
as above | Anti-criteria triggers | Group 18 tests |
Expected Results Mapping
Every test in blackbox-tests.md, performance-tests.md, resilience-tests.md, security-tests.md, and resource-limit-tests.md carries an Expected result source: results_report.md row <N> line in its body. The comparison method, tolerance, and reference file (if any) are inherited from that row — tests do not re-state them. The traceability-matrix.md aggregates: AC → results_report row(s) → test ID(s).
| Test Scenario ID | Input Data | Expected Result | Comparison Method | Tolerance | Expected Result Source |
|---|---|---|---|---|---|
(see blackbox-tests.md etc.) |
(row reference) | (row's Expected Result column) | (row's Comparison column) | (row's Tolerance column) | results_report.md row N |
No reference files (e.g. JSON / CSV) are required at this stage — every observable in results_report.md is small enough to fit inline. If a downstream test needs a multi-row expected payload (e.g. the suite's full class-distribution response shape), the file will be added in _docs/00_problem/input_data/expected_results/ per the naming convention <input_name>_expected.<ext> and the row updated to reference it. The Phase 3 Data Validation Gate will surface this if it happens.
External Dependency Mocks
| External Service | Mock/Stub | How Provided | Behavior |
|---|---|---|---|
| OpenWeatherMap (E10) | owm-stub HTTP service (e2e) / MSW handler (fast) |
Docker service owm-stub:8081 (e2e) / per-test MSW handler (fast) |
Returns canned /data/2.5/onecall JSON with deterministic wind/precip values. Tests that exercise wind compute (flightPlanUtils.ts once moved into src/) assert against these canned values. |
| OSM tile servers | tile-stub HTTP service (e2e) / never hit in fast |
Docker service tile-stub:8082 returning a fixed 256x256 PNG |
Replaces a.tile.openstreetmap.org etc.; map tile loads do not depend on internet. |
Suite microservices (admin/, flights/, annotations/, detect/, ...) |
Real services (e2e) / MSW handlers (fast) | docker-compose (e2e) / per-test handler (fast) | Production-shape responses per data_parameters.md §1. Errors injected per resilience tests (5xx, 413, network drop). |
| GPS-Denied services (target) | gps-denied-desktop + gps-denied-onboard real services (e2e); no fast coverage today |
docker-compose | F12 Test Mode is target-only — tests for it are quarantined until the feature lands (see Open Items in results_report.md). |
| LiveGPS simulator | Embedded in flights/ service test mode |
docker-compose | Emits deterministic LiveGpsEvent payloads at 1 Hz on /api/flights/<id>/live-gps so AC-08 timing assertions are stable. |
| Annotation-status events generator | Embedded in annotations/ service test mode |
docker-compose | Allows tests to trigger AnnotationStatusEvent SSE deliveries on demand. |
All mocks are deterministic: same input always yields the same output. Non-determinism (timestamps, IDs assigned by the suite) is bounded to suite-managed fields that tests do not compare directly — tests use field presence + shape, not value equality, for those fields.
Data Validation Rules
| Data Type | Validation | Invalid Examples | Expected System Behavior |
|---|---|---|---|
| Bearer (Authorization header) | Non-empty string starting Bearer |
empty header, missing header | 01_api-transport adds it automatically for authenticated requests; absence triggers a refresh attempt via 401-retry. |
| Refresh cookie (response Set-Cookie) | Secure; HttpOnly; SameSite=Strict per AC-03 |
Missing HttpOnly |
Test rejects the response (regression — server contract violation). Documented in the suite, asserted by the UI test for defence-in-depth. |
| Annotation save body | {Source, WaypointId, videoTime, mediaId, detections, status} keys all present (AC-05 / row 23) |
missing Source, contains time instead of videoTime |
Test FAILS — finding #32 fix regression. |
| Waypoint POST body | {Geopoint:{Lat,Lon,MGRS}, Source, Objective, OrderNum, Height} shape per data_parameters.md (Step 4 fix candidate) |
UI's current {name, latitude, longitude, order} |
Test FAILS once the Step 4 fix lands; pre-fix the test is quarantined (documents the contract drift). |
| Enum numeric value on the wire | Member of the spec value set (AC-04 / rows 14-19) | status: 1 for Edited (UI today) instead of 20 (spec) |
Test FAILS — exactly the regression that motivated Step 4 AC-04. |
splitTile YOLO label |
5 space-separated numeric tokens (AC-39 / row 86) | "garbage" (row 87) |
Parser surfaces a user-visible error; does NOT silently swallow. Test asserts the error path. |
| Numeric form input | Non-empty, parseable, in range (AC-26 / rows 66-67) | empty string, non-numeric, out-of-range | Validation error shown; no PUT fires. |
| File upload | ≤ 500 MB (E9 / AC-10 / row 39) | 501 MB | HTTP 413; UI surfaces a user-visible i18n error; no alert(). |
| i18n key set | keys(en.json) == keys(ua.json) (AC-12 / row 45) |
UA missing a key present in EN | Static check FAILS. |
Open Items For Phase 3 Validation
- Enum spec numeric values (AC-04): resolved at Phase 3. Snapshot committed at
_docs/00_problem/input_data/enum_spec_snapshot.jsonper../_docs/00_database_schema.md(authoritative).AnnotationStatus,MediaStatus,Affiliation,AnnotationSourcehave pinned numerics from the schema.CombatReadiness+MediaTypecarry averification_pending: trueflag because the schema doesn't pin numerics on those — Step 4 .NET-service inspection lifts the flag (or reorders if the inferred sequential mapping is wrong). UI drift (5 enums) is pinned in the snapshot'sui_drift_summaryfor Step 4 to consume. - Phase B / target ACs (AC-11 bundle gate, AC-18 browser-list, AC-24 SSE refresh, AC-25 async-video path, AC-40 tile-zoom UX): the rows are written but the consumer-side behavior does not exist today. Phase 3 will surface these to the user for: (a) downgrade to documentary, (b) quarantine, or (c) accept as gating the day the feature lands.
- Waypoint POST shape (
data_parameters.mdfinding #20): the contract pin is the suite spec, but the UI's current shape is wrong. Tests assert against the contract. Pre-fix the test is quarantined; the Step 4 fix lands together with un-quarantining. - AC-37 backend ordering: the class-hotkey contract depends on the
annotations/service returning classes in[0..N-1, 20..20+N-1, 40..40+N-1]. If the seed reveals a different shape, AC-37 row 79 will fail; the fix may need to land server-side or the UI may need a client-side resort. Phase 3 will surface this gap. - No
Reference Filerows are needed today — everyReference Filecell inresults_report.mdisN/A. If Phase 2 reveals a need (e.g. for a complex SSE payload sequence), the reference file lands in_docs/00_problem/input_data/expected_results/and the row is updated.