diff --git a/_docs/03_implementation/test_run_report.md b/_docs/03_implementation/test_run_report.md new file mode 100644 index 0000000..9bfffbe --- /dev/null +++ b/_docs/03_implementation/test_run_report.md @@ -0,0 +1,73 @@ +# Test Run Report (Step 7) + +**Date**: 2026-05-11 +**Mode**: functional +**Runner**: `scripts/run-tests.sh` (default profiles: static + fast; e2e env-blocked, see below) +**Verdict**: PASS_WITH_DOCUMENTED_GATE + +## Profile Outcomes + +| Profile | Status | Counts | Wall-clock | Report file | +|---------|--------|--------|------------|-------------| +| static | PASS | 29 / 29 | ~13 s | `test-output/static-report.csv` | +| fast | PASS | 26 files / 163 PASS / 13 SKIP / 0 FAIL | ~14.6 s | `test-output/fast-report.xml` | +| e2e | env-blocked (deferred) | n/a | n/a | n/a — see "Environment block" below | + +## System-Under-Test Reality Gate + +PASS: +- `_docs/00_problem/input_data/expected_results/results_report.md` exists; `_docs/02_document/tests/traceability-matrix.md` maps every AC to a test scenario and a results-report row. +- Stubs are confined to external systems: suite services (admin / flights / annotations / detect / loader / resource) are stubbed via MSW (fast) or are the unbuilt SUT for e2e. Internal UI modules (``, ``, ``, ``, `AuthContext`, `FlightProvider`, `
`, etc.) render as production. +- No internal product module is faked, monkeypatched, or replaced with a deterministic fallback — verified by Phase 7 of every per-batch code review and the three cumulative reviews. +- CSV report inspected (`test-output/summary.csv`) — 29 / 29 static rows PASS; fast profile reported as one rolled-up PASS row pointing at the JUnit XML. + +## Skipped Tests — All 13 Accepted as Legitimate + +User-approved (option A) per test-run skill section 5. All 13 are quarantine markers for absent production features, paired with control PASS tests that pin current behaviour: + +| # | Test | Quarantine reason | Drift backlog | +|---|------|-------------------|---------------| +| 1 | `tests/annotations_endpoint.test.tsx` AI-suggestion-accept save | No "accept AI suggestion" button wired to a save POST in production yet | F-CUM-3 #18 | +| 2 | `tests/annotations_endpoint.test.tsx` bulk-edit save | No bulk-edit save path in production yet | F-CUM-3 #18 | +| 3 | `tests/destructive_ux.test.tsx` per-surface enumeration | Static gate `STC-SEC8` covers it; per-surface tests defer to Phase B feature work | structural placeholder | +| 4 | `tests/sse_lifecycle.test.tsx` annotation-status SSE | Production has not wired `` to an annotation-status SSE yet | F-CUM-1 #5 | +| 5 | `tests/i18n.test.tsx` detector path on first boot | i18n detector pending Step 4 (testability refactor scoped it out) | F-CUM-1 #6 | +| 6 | `tests/i18n.test.tsx` persistence across reload | i18n persistence pending Step 4 | F-CUM-1 #7 | +| 7 | `tests/wire_contract.test.ts` CombatReadiness | `enum_spec_snapshot.verification_pending=true` (Step 4 .NET inspection pending) | spec-side gate | +| 8 | `tests/wire_contract.test.ts` MediaType | `enum_spec_snapshot.verification_pending=true` (Step 4 .NET inspection pending) | spec-side gate | +| 9 | `src/auth/ProtectedRoute.test.tsx` 10s loading timeout fallback | No timeout fallback in production | F-CUM-1 #9 | +| 10 | `src/auth/ProtectedRoute.test.tsx` Operator → /admin redirect | No RBAC permission gating in production | F-CUM-1 #9 | +| 11 | `src/auth/ProtectedRoute.test.tsx` integrator → /settings redirect | Same RBAC gap | F-CUM-1 #9 | +| 12 | `src/components/Header.test.tsx` FT-N-09 Escape close + handler detach | No document-level keydown listener in production | F-CUM-1 #10 | +| 13 | `src/components/ConfirmDialog.test.tsx` focus trap | No focus trap in production | F-CUM-3 #12 | + +All 13 satisfy the test-run skill's "feature-flag-gated test whose feature is intentionally disabled in this environment" pattern (broadened: feature-not-yet-built). None are flaky-test quarantines, missing-fixture, missing-credential, or service-not-running. They're tracked in F-CUM-3 (cumulative 04–06) and F-CUM-5 (cumulative 07–08) as Phase B / Step 9 work. + +## Environment Block — e2e Profile (User-Approved Defer) + +User-approved (option A): treat static + fast as the Step 7 per-commit gate; defer e2e to the dev/stage merge lane / CI runner that has registry access. + +**Block**: the UI's `e2e/docker-compose.suite-e2e.yml` references service images +`azaion/{admin,flights,annotations,detect,loader,resource}:test` that: + +- are not available locally (`docker image ls` shows zero `azaion/*` images), +- are not buildable from sibling-repo source today (e.g. `/Users/obezdienie001/dev/azaion/suite/annotations/` has no `Dockerfile`), +- are normally pulled from the project's CI registry by the suite-level harness `/Users/obezdienie001/dev/azaion/suite/e2e/run-local.sh` via `docker compose pull --ignore-pull-failures` — that path needs registry auth not configured in this workspace. + +This matches the test-run skill's "Stubs are allowed only for external systems outside the product boundary" — every blocked image is an external-service from the UI's perspective and is the canonical SUT, not a faked internal module. The block is legitimate per skill section 0 #2. + +**Coverage gap**: the e2e tests committed in batches 4–8 (10 fast/e2e companions + the 5 batch-7/8 perf and prod-image probes) cannot run in this Step 7 invocation. They WILL run on the CI / merge lane that has registry access. The contracts they assert (NFT-PERF-10 FCP, NFT-RES-LIM-05 memory soak, NFT-RES-LIM-08 RAM soak, NFT-RES-LIM-10 prefix-strip runtime, AZ-471/473/478/480 e2e companions) are also covered by: + +- the equivalent fast-profile assertions inside `tests/**` (PASS today), +- and the new commit-time static gates (`STC-PERF01`, `STC-RES02`, `STC-RES03`, `STC-RES09`, `STC-RES10`) — all PASS. + +So no AC is uncovered — the e2e companions are defence-in-depth on real-browser timing, not the only assertion path for any single AC. + +## Outcome + +Step 7 **passes** the gate with the documented env-block above. Auto-chain to Step 8 (Refactor — optional, user choice). + +## Open Items + +- F-CUM-5 production-drift backlog (23 entries; see `cumulative_review_batches_07-08_cycle1_report.md`) — Phase B / Step 9 work. +- F-CUM-4 long-running-soak `@long-running` Playwright config tag — recommended fold-in to the same merge-lane configuration that adds registry access for e2e. diff --git a/_docs/_autodev_state.md b/_docs/_autodev_state.md index 9601f8f..26a46b7 100644 --- a/_docs/_autodev_state.md +++ b/_docs/_autodev_state.md @@ -2,8 +2,8 @@ ## Current Step flow: existing-code -step: 7 -name: Run Tests +step: 8 +name: Refactor status: not_started sub_step: phase: 0