mirror of
https://github.com/azaion/ui.git
synced 2026-06-21 12:01:11 +00:00
[AZ-447] autodev Steps 1-4 baseline: docs, tests, refactor specs
Captures the full output of autodev existing-code Phase A through Step 4 (Code Testability Revision) for the Azaion UI workspace: - Step 1 Document: _docs/02_document/ (FINAL_report, architecture, glossary, components/, modules/, diagrams/, system-flows, module-layout) plus _docs/00_problem/ + _docs/01_solution/ + _docs/legacy/ + _docs/how_to_test + README. - Step 2 Architecture Baseline: architecture_compliance_baseline.md. - Step 3 Test Spec: _docs/02_document/tests/ (environment, test-data, blackbox/performance/resilience/security/ resource-limit tests, traceability-matrix), enum_spec_snapshot, expected_results/results_report.md (98 rows), plus the run-tests.sh + run-performance-tests.sh runners. - Step 4 Code Testability Revision: 01-testability-refactoring/ run dir (list-of-changes C01-C07, deferred_to_refactor, analysis/research_findings + refactoring_roadmap) and the 7 child task specs AZ-448..AZ-454 under _docs/02_tasks/todo/ plus _dependencies_table.md. - _docs/_autodev_state.md pins the cursor at Step 4 / refactor Phase 4 entry so /autodev resumes cleanly. Epic AZ-447 (UI testability gates) tracks the 7 child tasks that will land in subsequent commits. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,332 @@
|
||||
# Expected Results — Azaion UI
|
||||
|
||||
> Maps every behavioral test trigger (REST request, SSE event, user action,
|
||||
> build/config artifact, static check) to a **quantifiable** expected result.
|
||||
> Sourced from `_docs/00_problem/acceptance_criteria.md` (34 ACs + 5
|
||||
> anti-criteria) and cross-checked against `restrictions.md`,
|
||||
> `data_parameters.md`, and `_docs/02_document/architecture.md`.
|
||||
>
|
||||
> **Project shape note.** The Azaion UI is a thin SPA over a typed REST + SSE
|
||||
> contract; it carries no database and consumes no sample-image / sample-video
|
||||
> input files of its own. "Input" here therefore means a trigger condition
|
||||
> (HTTP request, SSE message, click, build invocation, code search) and the
|
||||
> "expected result" is the observable behavior the test asserts. Almost every
|
||||
> row uses the **behavioral shape** defined in `test-spec/SKILL.md` ("trigger
|
||||
> + observable + quantifiable pass/fail"); a few rows that exchange concrete
|
||||
> JSON bodies use the **input/output shape**.
|
||||
>
|
||||
> No reference files are required at this stage — every observable in the
|
||||
> table below is small enough to inline. If `Phase 2` of `/test-spec` finds a
|
||||
> case that needs a multi-row expected payload (e.g. full
|
||||
> `traceability-matrix` output), a JSON / CSV file will be added in this
|
||||
> directory and referenced from the corresponding row.
|
||||
|
||||
**Status**: agent-drafted (autodev Step 3 — Test Spec, Phase 1 prereq)
|
||||
**Date**: 2026-05-10
|
||||
|
||||
---
|
||||
|
||||
## Result Format Legend
|
||||
|
||||
| Result Type | When to Use | Example |
|
||||
|-------------|-------------|---------|
|
||||
| Exact value | Output must match precisely | `status_code: 200`, `count: 0` |
|
||||
| Tolerance range | Numeric output with acceptable variance | `position ± 50ms`, `width ± 1px` |
|
||||
| Threshold | Output must exceed or stay below a limit | `latency ≤ 500ms`, `count == 0` |
|
||||
| Pattern match | Output must match a string/regex pattern | URL regex, header presence |
|
||||
| Set/count | Output must contain specific items or counts | `keys(en) == keys(ua)` |
|
||||
|
||||
## Comparison Methods
|
||||
|
||||
| Method | Description | Tolerance Syntax |
|
||||
|--------|-------------|------------------|
|
||||
| `exact` | Actual == Expected | N/A |
|
||||
| `numeric_tolerance` | abs(actual - expected) ≤ tolerance | `± <value>` |
|
||||
| `range` | min ≤ actual ≤ max | `[min, max]` |
|
||||
| `threshold_min` | actual ≥ threshold | `≥ <value>` |
|
||||
| `threshold_max` | actual ≤ threshold | `≤ <value>` |
|
||||
| `regex` | actual matches regex pattern | regex string |
|
||||
| `substring` | actual contains substring | substring |
|
||||
| `set_equals` | sets of items match exactly | set notation |
|
||||
| `set_contains` | actual set ⊇ expected | subset notation |
|
||||
| `present` / `absent` | observable exists / does not exist | N/A |
|
||||
|
||||
---
|
||||
|
||||
## Input → Expected Result Mapping
|
||||
|
||||
### Group 1 — Authentication & Token Handling (AC-01 / AC-02 / AC-03 / AC-22 / AC-23 / AC-24)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 01 | Any authenticated `fetch` issued via `apiClient` | Outbound API call that requires the bearer | RequestInit MUST include `credentials: 'include'` | exact | N/A | N/A |
|
||||
| 02 | Bootstrap refresh call on app mount (`AuthContext` init) | The refresh-on-load flow before any user interaction | RequestInit MUST include `credentials: 'include'`; cookie sent | exact | N/A | N/A |
|
||||
| 03 | First 401 from `/api/admin/...` while a session is active | Bearer expired mid-session | Sequence: `POST /api/admin/auth/refresh` (cookie-bound) → original request retried with new bearer → final response 200 | exact (sequence) | N/A | N/A |
|
||||
| 04 | Code-search across `src/` for `localStorage.|sessionStorage.` references that touch the bearer | Static check on token-storage policy | `match_count == 0` | exact | N/A | N/A |
|
||||
| 05 | Code-search across `src/` for `document.cookie` reads that target the refresh token | Static check on cookie-readability policy | `match_count == 0` (cookie is HttpOnly server-side; UI must not even attempt) | exact | N/A | N/A |
|
||||
| 06 | Programmatic read of `document.cookie` after login | Browser-level visibility test of the refresh cookie | Returned string MUST NOT contain the refresh-token value | absent | N/A | N/A |
|
||||
| 07 | Refresh-cookie response header from `/api/admin/auth/login` (test fixture) | Set-Cookie attribute audit | Header value matches regex `Secure;.*HttpOnly;.*SameSite=Strict` (order tolerant, case insensitive) | regex | case-insensitive, attribute-order-tolerant | N/A |
|
||||
| 08 | Authenticated user with `role != admin` navigating to `/admin` | RBAC route gate (admin) | Final URL is `/flights`; `<AdminPage>` MUST NOT mount | exact (URL), absent (component) | N/A | N/A |
|
||||
| 09 | Unauthenticated user navigating to `/admin` | RBAC route gate (login) | Final URL is `/login` | exact | N/A | N/A |
|
||||
| 10 | Authenticated user without settings permission navigating to `/settings` | RBAC route gate (settings) | Final URL is `/flights`; `<SettingsPage>` MUST NOT mount | exact, absent | N/A | N/A |
|
||||
| 11 | Auth refresh occurring while the user is on `/flights` (mid-session) | Refresh transparency | `<ProtectedRoute>` MUST NOT unmount its children during refresh; render-counter delta ≤ 1 across refresh | numeric_tolerance | `≤ 1` re-render | N/A |
|
||||
| 12 | Single `/api/admin/auth/refresh` invocation | Refresh round-trip count | Exactly 1 outbound network call observed during one refresh cycle | exact | N/A | N/A |
|
||||
| 13 | Bearer rotation while two SSE streams are open | SSE refresh-rotation handling | Both EventSource instances MUST close, reconnect with new token in query string, and resume within 5 s | exact (close+open count), threshold_max | `≤ 5000ms` | N/A |
|
||||
|
||||
### Group 2 — Wire Contract & Enum Compliance (AC-04 / AC-29)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 14 | Read `AnnotationStatus` enum from `src/types/index.ts` | Numeric values must match suite spec | `{None:0, Created:10, Edited:20, Validated:30, Deleted:40}` | exact (key+value map) | N/A | N/A |
|
||||
| 15 | Read `MediaStatus` enum | Numeric values must match suite spec | Members `{None, New, AiProcessing, AiProcessed, ManualCreated, Confirmed, Error}` present; numeric values match the spec map | set_contains, exact (per member) | N/A | N/A |
|
||||
| 16 | Read `Affiliation` enum | Numeric values must match suite spec | Members `{None, Unknown, Friendly, Hostile}` present; numeric values match the spec map | set_contains, exact | N/A | N/A |
|
||||
| 17 | Read `CombatReadiness` enum | Numeric values must match suite spec | Members `{Unknown, NotReady, Ready}` present; numeric values match the spec map | set_contains, exact | N/A | N/A |
|
||||
| 18 | Outbound payload of `POST /api/annotations/annotations` containing `status` field | Wire-format check | `body.status` is a number from the `AnnotationStatus` value set | set_contains | N/A | N/A |
|
||||
| 19 | Outbound payload containing a `Detection` array | Per-detection wire check | Every `detection.affiliation ∈ Affiliation values` and `detection.combatReadiness ∈ CombatReadiness values` | set_contains (per element) | N/A | N/A |
|
||||
| 20 | Code-search `src/` for `mediaType\s*[!=]==?\s*[0-9]` | Magic-literal hygiene for `MediaType` | `match_count == 0` | exact | N/A | N/A |
|
||||
| 21 | Code-search `src/` for `mediaType\s*[!=]==?\s*['"]` | Magic-string hygiene for `MediaType` | `match_count == 0` | exact | N/A | N/A |
|
||||
|
||||
### Group 3 — Annotations endpoints, payload, SSE, overlay window (AC-05 / AC-09 / AC-25 / AC-28)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 22 | Save action in `<AnnotationsPage>` | Annotation save endpoint | Exactly one `POST` to URL matching `^/api/annotations/annotations$` | exact (count + URL) | N/A | N/A |
|
||||
| 23 | Annotation save body | Required fields | Body is JSON containing keys `{Source, WaypointId, videoTime, mediaId, detections, status}` (no `time` key) | set_contains, absent (`time`) | N/A | N/A |
|
||||
| 24 | Mount of `06_annotations` page | Status-events SSE subscribe | Exactly one EventSource opened to URL matching `^/api/annotations/annotations/events(\?|$)` | exact (count + URL regex) | N/A | N/A |
|
||||
| 25 | Unmount of `06_annotations` page | Status-events SSE unsubscribe | EventSource readyState transitions to `CLOSED` (2) within 1 s | exact (state), threshold_max | `≤ 1000ms` | N/A |
|
||||
| 26 | Sync image detect trigger (`<AnnotationsSidebar>` Detect on a `MediaType.Image`) | Detect endpoint correctness | Exactly one `POST` to URL matching `^/api/detect/[0-9]+$` | exact, regex | N/A | N/A |
|
||||
| 27 | Async video detect trigger (Phase B; behind feature flag if pre-shipped) | Async detect endpoint | Exactly one `POST` to URL matching `^/api/detect/video/[0-9]+$`; response `{jobId: <int>}`; subsequent EventSource opened to `^/api/detect/stream/[0-9]+(\?|$)` | exact, regex (3 assertions) | N/A | N/A |
|
||||
| 28 | Long-video async detect | Header policy per `_docs/10_auth.md` | Outgoing request includes `X-Refresh-Token` header (non-empty) | present (header) | N/A | N/A |
|
||||
| 29 | Annotation overlay membership at `currentTime = T` | Asymmetric time window | Annotation with `videoTime` in `[T-50ms, T+150ms]` is rendered; outside this window NOT rendered | range, absent | exact bounds | N/A |
|
||||
| 30 | Overlay membership at `currentTime = T` with annotation at `T - 60ms` | Lower-bound exclusion | Annotation NOT rendered | absent | N/A | N/A |
|
||||
| 31 | Overlay membership at `currentTime = T` with annotation at `T + 160ms` | Upper-bound exclusion | Annotation NOT rendered | absent | N/A | N/A |
|
||||
|
||||
### Group 4 — Flight selection persistence + Live-GPS SSE (AC-06 / AC-08)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 32 | `FlightContext.selectFlight(flightId)` call | Selected-flight persistence path | Exactly one `PUT /api/annotations/settings/user` with body containing `{selectedFlightId: flightId}`; NO call to `/api/flights/select` (deprecated path must not exist) | exact (count + URL + body subset), absent | N/A | N/A |
|
||||
| 33 | Reload after a flight was selected | Selected-flight rehydration | On boot, `userSettings.selectedFlightId` is read and the flight is reselected without a user click | exact (selection state matches stored id) | N/A | N/A |
|
||||
| 34 | A flight is selected | Live-GPS SSE open | Exactly one EventSource to URL matching `^/api/flights/[0-9]+/live-gps(\?|$)`; readyState reaches `OPEN` (1) within 5 s | exact (count + URL regex), threshold_max | `≤ 5000ms` | N/A |
|
||||
| 35 | Flight is deselected | Live-GPS SSE close | All EventSources matching `^/api/flights/[0-9]+/live-gps` reach readyState `CLOSED` (2) within 1 s | exact (count → 0), threshold_max | `≤ 1000ms` | N/A |
|
||||
|
||||
### Group 5 — Dataset bulk-validate (AC-07)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 36 | Select N dataset items → click `Validate` | Bulk-validate endpoint + body | Exactly one `POST /api/annotations/dataset/bulk-status` with body `{ids: <length N int array>, targetStatus: 30}` (`AnnotationStatus.Validated`) | exact (URL + body) | N/A | N/A |
|
||||
| 37 | Successful 200 response from bulk-validate | UI status reflection | Each selected item's row status changes to `Validated` within 2 s of response | exact (per-row state), threshold_max | `≤ 2000ms` | N/A |
|
||||
|
||||
### Group 6 — Upload size cap (AC-10)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 38 | Read `client_max_body_size` from `nginx.conf` | nginx upload cap | Value equals `500M` | exact | N/A | N/A |
|
||||
| 39 | UI upload of a synthetic 501 MB file | 413 surfacing | API call resolves with HTTP 413; UI presents a user-visible error containing the i18n key for "file too large" (or its rendered string) — NO silent failure, NO `alert()` | exact (status), substring (rendered error), absent (`alert()`) | N/A | N/A |
|
||||
|
||||
### Group 7 — Build, bundle, and routing (AC-11 / AC-31 / AC-33 / AC-34)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 40 | `vite build` output `dist/` | Initial JS bundle budget (target — not yet enforced in CI) | Sum of gzipped initial-route JS chunks ≤ 2 MB | threshold_max | `≤ 2 MB gzipped` | N/A |
|
||||
| 41 | `vite build` output `dist/` | mission-planner exclusion | No file under `dist/` originates from `mission-planner/**`; static-import scan from `src/main.tsx` does not reach into `mission-planner/` | absent (file origin), absent (graph edge) | N/A | N/A |
|
||||
| 42 | `docker inspect azaion/ui:<tag>` | Production runtime image | Final image is based on `nginx:alpine`; no `node` binary present in the image filesystem | exact (base image), absent (binary) | N/A | N/A |
|
||||
| 43 | Read `nginx.conf` route blocks | Service routing | Exactly the 9 `location` blocks present: `/api/admin/`, `/api/flights/`, `/api/annotations/`, `/api/detect/`, `/api/loader/`, `/api/gps-denied-desktop/`, `/api/gps-denied-onboard/`, `/api/autopilot/`, `/api/resource/` | set_equals | N/A | N/A |
|
||||
| 44 | Each of the 9 `location` blocks | Prefix stripping | Each block rewrites/strips its `/api/<service>/` prefix before forwarding upstream (verified via `proxy_pass`/`rewrite` directive shape) | regex per block | N/A | N/A |
|
||||
|
||||
### Group 8 — Internationalization (AC-12 / AC-13)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 45 | Compare `src/i18n/en.json` vs `src/i18n/ua.json` | Key parity | `keys(en) == keys(ua)` (deep, sorted) | set_equals | N/A | N/A |
|
||||
| 46 | Lint sweep over `src/**/*.tsx` | No raw user-visible strings | Every JSX text node and every `aria-label` / `placeholder` / `title` string resolves through `t(...)` (or is a proper-noun acronym in the allow-list) | exact (lint findings == 0) | acronym allow-list | N/A |
|
||||
| 47 | First boot in a clean profile | i18n detector | `i18next.language` resolves from cookie or `Accept-Language`; not the literal `'en'` from a hardcoded init | exact (detector path used), absent (hardcoded `lng:'en'`) | N/A | N/A |
|
||||
| 48 | Toggle language in `<Header>` then reload | i18n persistence | After reload, `i18next.language` equals the previously selected language | exact | N/A | N/A |
|
||||
|
||||
### Group 9 — Destructive-action UX (AC-14 / AC-30)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 49 | Click `Delete class` in `<AdminPage>` | Class-delete confirmation | `<ConfirmDialog>` is rendered before any HTTP request fires; on Cancel NO `DELETE` request is made; on Confirm exactly one `DELETE` to URL matching `^/api/admin/classes/[0-9]+$` | exact (sequence), exact (count + URL regex) | N/A | N/A |
|
||||
| 50 | Code-search `src/` and `mission-planner/src/` for `\balert\(` | `alert()` ban | `match_count == 0` | exact | N/A | N/A |
|
||||
| 51 | Each destructive action surfaced in `_docs/ui_design/` (delete, validate-with-overwrite, irreversible bulk) | Confirm-before-fire policy | A `<ConfirmDialog>` opens before the destructive request fires (no direct submit path) | present (dialog), exact (sequence) | N/A | N/A |
|
||||
|
||||
### Group 10 — Accessibility (AC-15 / AC-16 / AC-17)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 52 | Render `<ConfirmDialog>` open | a11y attributes | Root element has `role="dialog"` AND `aria-modal="true"` | exact | N/A | N/A |
|
||||
| 53 | Open `<ConfirmDialog>` and Tab through | Focus trap | Focus stays inside the dialog (first ↔ last loop); never reaches an element outside | exact (focus stays in tree) | N/A | N/A |
|
||||
| 54 | Press `Escape` while `<ConfirmDialog>` is open | Cancel-on-Escape | Dialog unmounts; cancel callback invoked exactly once; no destructive HTTP request fires | exact (count) | N/A | N/A |
|
||||
| 55 | Render `<Header>` flight dropdown closed | Combobox a11y attrs | Trigger has `role="combobox"`, `aria-expanded="false"`, `aria-haspopup="listbox"` | exact | N/A | N/A |
|
||||
| 56 | Open the flight dropdown | Combobox a11y on open | `aria-expanded` switches to `"true"`; outside-click handler is now attached (was NOT attached while closed) | exact | N/A | N/A |
|
||||
| 57 | Press `Escape` while flight dropdown is open | Close-on-Escape | Dropdown closes; `aria-expanded` returns to `"false"`; outside-click handler is detached | exact | N/A | N/A |
|
||||
| 58 | `<ProtectedRoute>` shows the loading state | a11y on spinner | Spinner element has `role="status"` and an accessible label (non-empty `aria-label` or visually-hidden text) | exact, present (label) | N/A | N/A |
|
||||
| 59 | `<ProtectedRoute>` loading exceeds the timeout (10 s simulated) | Timeout fallback | A user-visible fallback (retry CTA or error message) is rendered; the indeterminate spinner is unmounted | present (fallback), absent (spinner) | timeout configurable | N/A |
|
||||
|
||||
### Group 11 — Browser support & responsive layout (AC-18 / AC-19)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 60 | Smoke render of `/flights`, `/annotations`, `/dataset` in headless Chromium and Firefox (latest 2 versions) | Browser support floor | No console error logged; main page region has rendered with expected landmark roles | exact (errors == 0), present (landmarks) | N/A | N/A |
|
||||
| 61 | Headless render at viewport width 480 px | Mobile bottom-nav variant | `<Header>` bottom-nav variant renders; top-bar variant is NOT in DOM | present, absent | N/A | N/A |
|
||||
| 62 | Headless render at viewport width 1024 px | Desktop variant | Top-bar `<Header>` renders; bottom-nav variant NOT in DOM | present, absent | N/A | N/A |
|
||||
|
||||
### Group 12 — Secrets (AC-20)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 63 | Code-search `src/` and `mission-planner/src/` for the literal OWM key value (and for any `appid=` / `api_key=` in source URLs) | Secrets-in-source check | `match_count == 0` for the literal key; the key is read only via `import.meta.env.VITE_OPENWEATHERMAP_API_KEY` (or proxied through the suite) | exact, exact (single read site) | N/A | N/A |
|
||||
|
||||
### Group 13 — User-settings persistence (AC-21)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 64 | Drag a resizable panel divider via `useResizablePanel` and release | Persist on resize-end | Within 1 s of release, exactly one `PUT /api/annotations/settings/user` fires with body containing `panelWidths: { ... }` reflecting the new sizes | exact (count + URL), substring (key), threshold_max (1 s) | debounce-aware | N/A |
|
||||
| 65 | Reload after a panel resize | Width rehydration | Restored panel widths equal the pre-reload widths within 1 px | numeric_tolerance | `± 1px` | N/A |
|
||||
|
||||
### Group 14 — Form hygiene (AC-26 / AC-27)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 66 | Submit a `09_settings` numeric field with empty value | No silent zero | Form does NOT save `0`; submit button disabled OR explicit validation error rendered; NO `PUT` request fires | absent (request), present (validation surface) | N/A | N/A |
|
||||
| 67 | Submit `09_settings` numeric field with non-numeric input | Reject non-numeric | Validation error rendered; NO `PUT` fires | absent (request), present (error) | N/A | N/A |
|
||||
| 68 | `09_settings` save action where the upstream PUT returns HTTP 500 | Error surfacing | A toast / inline error renders within 2 s; `saving` flag returns to `false` (i.e. button is re-enabled); NO route navigation occurs | present (error), exact (state), absent (navigation), threshold_max (2 s) | N/A | N/A |
|
||||
| 69 | `09_settings` save action where the PUT throws (network failure) | Error path via `try/finally` | `saving` flag returns to `false`; user-visible error rendered | exact (state), present (error) | N/A | N/A |
|
||||
|
||||
### Group 15 — CI / image / labels (AC-32)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 70 | Image push step in `.woodpecker/build-arm.yml` for branch `main` | Tag scheme | Pushed tag matches `^main-arm$` | regex | N/A | N/A |
|
||||
| 71 | Image push step | OCI labels | Pushed image carries non-empty labels `org.opencontainers.image.revision`, `org.opencontainers.image.created`, `org.opencontainers.image.source` | present (each), exact (count == 3) | N/A | N/A |
|
||||
| 72 | `org.opencontainers.image.revision` label value | Revision plumbing | Equals `$CI_COMMIT_SHA` from the pipeline run | exact | N/A | N/A |
|
||||
|
||||
### Group 16 — Manual annotation interactions on `CanvasEditor` + `DetectionClasses` (AC-35 / AC-36 / AC-37 / AC-38)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 73 | Synthetic `mousedown(x1,y1) → mousemove(x2,y2) → mouseup` over `CanvasEditor` with `selectedClassNum = C` and `photoMode = P` | Manual bbox draw (AC-35) | Exactly one new local detection appended with `classNum == C + P`, `x == min(x1,x2)/W`, `y == min(y1,y2)/H`, `w == |x2-x1|/W`, `h == |y2-y1|/H` (W,H = canvas pixel size) | numeric_tolerance (per coord), exact (count + classNum) | `± 1px / canvas px` | N/A |
|
||||
| 74 | `mousedown` on resize handle `h ∈ {NW, N, NE, W, E, SW, S, SE}` over a selected bbox, drag by `(dx, dy)`, mouseup | 8-handle resize (AC-36a) | Only the edges adjacent to `h` move; opposite edges unchanged; resulting bbox dimensions clamped to a minimum normalised size > 0 (no negative or zero w/h) | exact (per-edge invariance), threshold_min (w,h > 0) | N/A | N/A |
|
||||
| 75 | `Ctrl+click` on a bbox that is not currently selected | Canvas multi-select (AC-36b) | The bbox is added to the selection set; previous selection is preserved; clicking the same bbox a second time with Ctrl removes it from the set | exact (selection set delta), idempotent (toggle) | N/A | N/A |
|
||||
| 76 | `Ctrl+wheel` over the canvas at cursor position `(cx, cy)` | Canvas zoom (AC-36c) | Viewport zoom level changes; the world point at `(cx, cy)` before zoom maps back to `(cx, cy)` after zoom (zoom-around-cursor invariant) | numeric_tolerance | `± 1 viewport px` | N/A |
|
||||
| 77 | `Ctrl+drag` starting on empty canvas (no bbox under the pointer) | Canvas pan (AC-36d) | Viewport origin translates by exactly `(-dx, -dy)`; no bbox is created or modified | numeric_tolerance, absent (state delta) | `± 1 viewport px` | N/A |
|
||||
| 78 | Mount of `<DetectionClasses>` with a successful `GET /api/annotations/classes` response of N classes (mode-ordered) | Class list load (AC-37 / load path) | All N entries are rendered; the active-mode filter is applied; no fallback list is shown | exact (count rendered), absent (fallback) | N/A | N/A |
|
||||
| 79 | `keydown` on `window` with key `'1'..'9'` while `photoMode = P` and `classes` are mode-ordered per the contract | Class hotkey 1–9 (AC-37 / hotkey path) | `onSelect` fires exactly once with `class.id == classes[(key-1) + P].id`; the visible label index `i+1` on the rendered list element matches `key` | exact | N/A | N/A |
|
||||
| 80 | Click on a class entry in the strip | Class click path (AC-37 / click path) | `onSelect` fires once with that entry's `class.id` | exact (count + value) | N/A | N/A |
|
||||
| 81 | `GET /api/annotations/classes` returning empty or 5xx | Fallback list (AC-37 / fallback) | `FALLBACK_CLASS_NAMES` × 3 PhotoMode offsets is rendered (IDs in the contiguous `[0..N-1, 20..20+N-1, 40..40+N-1]` shape) | exact (set of IDs) | N/A | N/A |
|
||||
| 82 | Click the `Winter` PhotoMode button while `photoMode = 0` | PhotoMode switch — mode + filter (AC-38 / mode set + filter) | Outgoing `onPhotoModeChange(20)` fires once; rendered class list is filtered to entries whose `photoMode == 20` | exact (call + filter) | N/A | N/A |
|
||||
| 83 | PhotoMode switch where the previously-selected `classNum` is NOT in the new filtered set | PhotoMode auto-select (AC-38 / auto-select) | `onSelect` fires once with `modeClasses[0].id` (first class of the new mode) | exact (count + value) | N/A | N/A |
|
||||
| 84 | Save annotation after drawing bbox with `selectedClassNum = C` and `photoMode = P` | yoloId on wire (AC-38 / wire) | Outgoing `POST /api/annotations/annotations` body has `detections[i].classNum == C + P` for the newly drawn detection | exact | N/A | N/A |
|
||||
|
||||
### Group 17 — Tile splitting (AC-39 / AC-40)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 85 | Click `Split tile` action on a dataset item | Split endpoint contract (AC-39 / endpoint) | Exactly one `POST` to URL matching `^/api/annotations/dataset/[0-9]+/split$`; success response is HTTP 200 JSON | exact (count + URL regex + status) | N/A | N/A |
|
||||
| 86 | `AnnotationListItem` with `isSplit: true, splitTile: "3 0.5 0.5 0.2 0.2"` | YOLO label parse — valid (AC-39 / parser happy) | Parser yields `{ classNum: 3, cx: 0.5, cy: 0.5, w: 0.2, h: 0.2 }` without throwing | exact | N/A | N/A |
|
||||
| 87 | `AnnotationListItem` with `isSplit: true, splitTile: "garbage"` | YOLO label parse — malformed (AC-39 / parser sad) | User-visible error surfaced (toast or inline); NO silent swallow; NO render with NaN values | present (error), absent (NaN render) | N/A | N/A |
|
||||
| 88 | `DatasetItem` response containing `isSplit: true` from `GET /api/annotations/dataset` | DatasetItem.isSplit honored (AC-39 / dataset list) | UI reads `item.isSplit` without crash; downstream rendering uses the boolean (rendering policy is per item type, but presence is required) | present (field read) | N/A | N/A |
|
||||
| 89 | Double-click a `splitTile`-bearing annotation in `<AnnotationsSidebar>` | Tile auto-zoom viewport (AC-40 / viewport) | `CanvasEditor` viewport rect equals the tile rect encoded by `splitTile` (per AC-39 parse) within ±1 px per edge | numeric_tolerance | `± 1px per edge` | N/A |
|
||||
| 90 | While tile zoom is active | Tile-zoom indicator (AC-40 / indicator) | A visible tile-zoom indicator (icon or badge) is present in the canvas chrome; clearing the tile zoom removes it | present, absent (after clear) | N/A | N/A |
|
||||
|
||||
### Group 18 — Anti-criteria (AC-N1..AC-N5) — negative behavioral assertions
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 91 | Two browsers editing the same annotation simultaneously | No collaborative-edit semantics (AC-N1) | UI MUST NOT reconcile concurrent edits; last-write-wins on the server is the expected behavior (no merge UI, no presence indicators) | absent (merge UI) | N/A | N/A |
|
||||
| 92 | Static dependency scan of `package.json` and `mission-planner/package.json` | No in-browser ML (AC-N2) | No package matching the pattern `(onnxruntime|tensorflow|tflite|coreml|tfjs|@tensorflow/.*|@huggingface/.*|transformers\.js)` is declared | absent | N/A | N/A |
|
||||
| 93 | App boot in a network-disabled environment (offline) | No offline mode (AC-N3) | App enters an error / login-failed state; does NOT serve cached app data; no service worker registered | present (error), absent (sw) | N/A | N/A |
|
||||
| 94 | Static dependency scan | No response-signature library (AC-N4) | No package matching `(jsrsasign|tweetnacl|@noble/.*|jose)` is imported on the request-validation path | absent | N/A | N/A |
|
||||
| 95 | Code-search across `src/` and `mission-planner/` for symbols/components named `SoundDetections|DroneMaintenance` | Dropped legacy features (AC-N5) | `match_count == 0` | exact | N/A | N/A |
|
||||
|
||||
### Group 19 — Phase-3-added (Data Validation Gate)
|
||||
|
||||
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|
||||
|----|-------|-------------------|-----------------|------------|-----------|---------------|
|
||||
| 96 | Click `Download` in `<AnnotationsPage>` on a canvas tainted by a cross-origin video frame (CORS-less video source) | Tainted-canvas fallback (NFT-RES-09; derived from `modules/src__features__annotations.md` finding on `handleDownload`) | A user-visible error (toast or inline message) is rendered; NO silent swallow; NO `alert()` invoked; no fabricated blob is offered for download | present (error), absent (silent swallow), absent (`alert()` invocation) | N/A | N/A |
|
||||
| 97 | Server force-closes an open `EventSource` (live-GPS or annotation-status) without rotation | SSE server disconnect indicator (NFT-RES-10; derived from AC-08 + AC-09 + AC-24) | UI either renders a connection-lost indicator (badge/icon) OR invokes a reconnect attempt within 10 s. Stale event data is NOT re-rendered as live; the most recent live timestamp is frozen + flagged stale | present (indicator OR reconnect), absent (stale data rendered as live), exact (reconnect_attempts ≤ 1 in the 10 s window) | dt ≤ 10 000 ms | N/A |
|
||||
| 98 | Warm-cache navigation to `/flights` on the post-login route in headless Chromium on the edge profile (2 vCPU / 4 GB RAM) | First Contentful Paint baseline (NFT-PERF-10; derived from AC-11 target + H2 edge deploy) | `performance.getEntriesByName('first-contentful-paint')[0].startTime` reported by the browser is below the threshold | threshold_max | `≤ 3 000 ms` | N/A |
|
||||
|
||||
---
|
||||
|
||||
## Coverage Summary
|
||||
|
||||
| AC ID | Mapped row(s) | Coverage |
|
||||
|--------|---------------|----------|
|
||||
| AC-01 | 01, 02, 03 | full (default + bootstrap + 401 retry) |
|
||||
| AC-02 | 04 | full |
|
||||
| AC-03 | 05, 06, 07 | full (JS-readable check, fixture cookie regex) |
|
||||
| AC-04 | 14, 15, 16, 17, 18, 19 | full (per enum + payload contract) |
|
||||
| AC-05 | 22, 23 | full (URL + required body fields) |
|
||||
| AC-06 | 32, 33 | full (write path + boot rehydration) |
|
||||
| AC-07 | 36, 37 | full (request + UI reflection) |
|
||||
| AC-08 | 34, 35 | full (open + close) |
|
||||
| AC-09 | 24, 25 | full (subscribe + unsubscribe) |
|
||||
| AC-10 | 38, 39 | full (server cap + UI surfacing) |
|
||||
| AC-11 | 40 | target (no CI gate today; row documents the threshold) |
|
||||
| AC-12 | 45, 46 | full |
|
||||
| AC-13 | 47, 48 | full |
|
||||
| AC-14 | 49, 50, 51 | full (dialog presence + alert ban + general policy) |
|
||||
| AC-15 | 52, 53, 54 | full |
|
||||
| AC-16 | 55, 56, 57 | full |
|
||||
| AC-17 | 58, 59 | full |
|
||||
| AC-18 | 60 | manual smoke today (test row exists; gate is target) |
|
||||
| AC-19 | 61, 62 | full (two breakpoint smokes) |
|
||||
| AC-20 | 63 | full |
|
||||
| AC-21 | 64, 65 | full |
|
||||
| AC-22 | 08, 09, 10 | full (admin + login + settings) |
|
||||
| AC-23 | 11, 12 | full |
|
||||
| AC-24 | 13 | target (Phase B / Step 8 hardening) |
|
||||
| AC-25 | 26, 27, 28 | full (sync image + async video + token header) |
|
||||
| AC-26 | 66, 67 | full |
|
||||
| AC-27 | 68, 69 | full |
|
||||
| AC-28 | 29, 30, 31 | full (in-window + below + above) |
|
||||
| AC-29 | 20, 21 | full |
|
||||
| AC-30 | 49 | overlapped with AC-14 row 49 |
|
||||
| AC-31 | 41 | full |
|
||||
| AC-32 | 70, 71, 72 | full |
|
||||
| AC-33 | 42 | full |
|
||||
| AC-34 | 43, 44 | full (route set + prefix strip) |
|
||||
| AC-35 | 73 | full (one-shot draw assertion) |
|
||||
| AC-36 | 74, 75, 76, 77 | full (resize + multi-select + zoom + pan) |
|
||||
| AC-37 | 78, 79, 80, 81 | full (load + hotkey + click + fallback) |
|
||||
| AC-38 | 82, 83, 84 | full (mode set + filter + auto-select + wire yoloId) |
|
||||
| AC-39 | 85, 86, 87, 88 | full (endpoint + parser happy + parser sad + DatasetItem flag) |
|
||||
| AC-40 | 89, 90 | target (UX missing today — finding #24; rows assert when implementation lands) |
|
||||
| AC-N1 | 91 | full |
|
||||
| AC-N2 | 92 | full |
|
||||
| AC-N3 | 93 | full |
|
||||
| AC-N4 | 94 | full |
|
||||
| AC-N5 | 95 | full |
|
||||
| (NFT-RES-09 anchor) | 96 | added Phase 3 — tainted-canvas fallback observable |
|
||||
| (NFT-RES-10 anchor) | 97 | added Phase 3 — SSE server-disconnect observable |
|
||||
| (NFT-PERF-10 anchor) | 98 | added Phase 3 — FCP baseline threshold |
|
||||
|
||||
Every AC + anti-criterion has at least one row. Every row is quantifiable.
|
||||
|
||||
## Open Items For Phase 1 Validation
|
||||
|
||||
- **AC-04 enum maps**: rows 14–17 reference "the spec map" for `MediaStatus`,
|
||||
`Affiliation`, `CombatReadiness`. The exact numeric values must be harvested
|
||||
from the suite spec at Phase 1 time and inlined; they are deliberately left
|
||||
symbolic here because the UI today drifts (per `restrictions.md` O7 + the
|
||||
data-parameters drift notes) and we want the test to assert against the
|
||||
spec, not against the current `src/types/index.ts`.
|
||||
- **AC-11 / AC-18 / AC-24 / AC-25 (async) / AC-40** are flagged "Phase B /
|
||||
target" or "Step 4 fix" in `acceptance_criteria.md`. The rows above produce
|
||||
verifiable assertions when those features ship; until then `/test-spec`
|
||||
Phase 3 may downgrade them from blocking to documentary. AC-40 in particular
|
||||
has zero consumer of `splitTile` in the UI today (finding #24 in
|
||||
`modules/src__features__annotations.md`); its rows are written so the test
|
||||
exists the day the tile-zoom UX ships.
|
||||
- **AC-37 backend ordering**: the class-hotkey contract depends on the
|
||||
`annotations/` service returning classes in the contiguous
|
||||
`[0..N-1, 20..20+N-1, 40..40+N-1]` shape. This was flagged Step 4
|
||||
verification in `data_model.md:158`. If the backend ordering does not match,
|
||||
AC-37 row 79 will fail at integration time and we may need a server-side
|
||||
fix or a client-side resort.
|
||||
- **Reference files**: none required at this stage. If `/test-spec` Phase 2
|
||||
produces a per-route JSON expectation that doesn't fit a single row,
|
||||
reference files will be added in this folder following the
|
||||
`<input_name>_expected.<ext>` convention.
|
||||
Reference in New Issue
Block a user