--- phase: 02-acceptance-criteria-test-taxonomy-observability-spine plan: "04" subsystem: traceability tags: [ac-traceability, pytest-markers, bidirectional-coverage] dependency_graph: requires: [02-01, 02-02, 02-03] provides: [scripts/gen_ac_traceability.py, .planning/AC-TRACEABILITY.md, @pytest.mark.ac decorations] affects: [02-05-CI-gate, AC-06-satisfaction] tech_stack: added: [] patterns: [argparse-cli, subprocess-pytest-collect, ac-marker-plugin, class-level-pytestmark] key_files: created: - scripts/gen_ac_traceability.py - .planning/AC-TRACEABILITY.md modified: - tests/test_acceptance.py - tests/test_accuracy.py - tests/test_processor_pipe.py - tests/test_gps_input_encoding.py - tests/test_sitl_integration.py - tests/test_mavlink.py - tests/test_schemas.py decisions: - "AC-TRACEABILITY.md committed to .planning/ with git add -f (path is gitignored but other .planning files are tracked)" - "test_gps_input_encoding.py marked at module/pytestmark level (all 13 tests -> AC-4.3)" - "TestGPSPoint and TestWaypoint marked at class level -> AC-6.3 (propagates to methods)" - "Orphan ACs get pending-phase-3 annotation in Plan 02-05 (documented decision carried forward)" metrics: duration: "~15 minutes" completed: "2026-05-11" tasks_completed: 4 files_created: 2 files_modified: 7 --- # Phase 02 Plan 04: AC Traceability Script + Test Decorations Summary One-liner: `scripts/gen_ac_traceability.py` + 38 `@pytest.mark.ac(...)` decorations cover 14 of 35 non-deferred ACs in the initial matrix, with `--check` deterministic for CI gating. ## Tasks Completed | Task | Status | Commit | |------|--------|--------| | 1: Implement gen_ac_traceability.py | Done | 4bf6f67 | | 2: Decorate 7 test files | Done | 6a1cd51 | | 3: Regenerate and commit AC-TRACEABILITY.md | Done | 419e9c5 | | 4: Regression gate | Done | (verification only) | ## Task 1: Script Implementation The script matches RESEARCH.md §2.3 exactly with one minor adaptation: the `--strict` alias is added as `--check, --strict` on the same argument (both names accepted by argparse via nargs). **Functions implemented:** - `collect_acs_from_doc()` — regex parses `- **AC-X.Y**` headers + `deferred-hardware` token within each block - `collect_acs_from_tests()` — subprocess `pytest --collect-only -q --ac-dump=` reads the JSON from conftest Plan 02-02 hook - `render_md()` — produces header stats, `| AC ID | Test count | Tests | Status |` table, backward-orphan section - `main()` — argparse + `--check` exit-1 on any non-deferred orphan or unknown AC ID **Deviations vs RESEARCH.md §2.3:** None material. The plan's verbatim source was used as written. ## Task 2: Decoration Counts Per File | File | Decorator lines added | AC IDs tagged | Notes | |------|-----------------------|---------------|-------| | test_acceptance.py | 8 | AC-1.1, AC-2.1a, AC-3.3, AC-3.4, AC-4.1, AC-4.4, AC-1.4 | per-function decorators | | test_accuracy.py | 11 | AC-1.1, AC-1.2, AC-1.3, AC-2.1a, AC-2.2, AC-4.1 | per-function decorators | | test_processor_pipe.py | 2 | AC-4.4, AC-1.4 | per-function decorators | | test_gps_input_encoding.py | 1 | AC-4.3 | module-level pytestmark (13 tests inherit) | | test_sitl_integration.py | 9 | AC-4.3, AC-4.4, AC-NEW-2, AC-5.2, AC-3.4 | per-function decorators | | test_mavlink.py | 5 | AC-4.3, AC-5.2, AC-3.4 | per-function decorators | | test_schemas.py | 2 | AC-6.3 | class-level decorators (propagate to methods) | | **Total** | **38** | **14 unique ACs** | | Total tests collected under `-m ac`: **45** (from 298 collected). ## AC Coverage Table | AC ID | Covered? | Reason if orphan | |-------|----------|-----------------| | AC-1.1 | Yes | test_acceptance (2), test_accuracy (3) | | AC-1.2 | Yes | test_accuracy (2) | | AC-1.3 | Yes | test_accuracy (1) | | AC-1.4 | Yes | test_acceptance (1), test_processor_pipe (1) | | AC-2.1a | Yes | test_acceptance (1), test_accuracy (1) | | AC-2.1b | No | No test explicitly checks cross-domain registration separately from VO | | AC-2.2 | Yes | test_accuracy (1) | | AC-3.1 | No | No outlier-tolerance test in Phase 2 test files | | AC-3.2 | No | No sharp-turn test in Phase 2 test files | | AC-3.3 | Yes | test_acceptance (1) — factor graph convergence | | AC-3.4 | Yes | test_acceptance (1), test_mavlink (1), test_sitl (1) | | AC-3.5 | No | No visual-blackout mode-switch test in Phase 2 | | AC-4.1 | Yes | test_acceptance (1), test_accuracy (2) | | AC-4.2 | No | No memory-bound test exists in Phase 2 | | AC-4.3 | Yes | test_gps_input_encoding (12), test_mavlink (3), test_sitl (5) | | AC-4.4 | Yes | test_acceptance (3), test_processor_pipe (1), test_sitl (1) | | AC-4.5 | No | No refinement/correction round-trip test in Phase 2 | | AC-5.1 | No | Phase 3 target — FC EKF init test not yet written | | AC-5.2 | Yes | test_mavlink (1), test_sitl (1) | | AC-5.3 | No | Phase 3 target — mid-flight reboot recovery | | AC-6.1 | No | No GCS downsample rate test in Phase 2 | | AC-6.2 | No | No operator reloc command test in Phase 2 | | AC-6.3 | Yes | test_schemas TestGPSPoint (4), TestWaypoint (2) | | AC-7.1 | No | Phase 3+ — AI camera localization accuracy | | AC-7.2 | No | Phase 3+ — trig computation | | AC-8.1 | No | Phase 3/4 — satellite imagery integration | | AC-8.2 | No | Phase 3/4 — tile freshness | | AC-8.3 | No | Phase 3/4 — pre-flight loading | | AC-8.4 | No | deferred-stage3 per AC doc | | AC-8.5 | No | Phase 4 — storage policy | | AC-8.6 | No | Phase 4/VPR — conditional VPR | | AC-NEW-1 | DEFERRED (hardware) | cold-boot bench requires Jetson | | AC-NEW-2 | Yes | test_sitl (1) — spoofing-promotion via GPS_INPUT accepted | | AC-NEW-3 | No (partial deferred) | integration-test scope not yet written; deferred-hardware for real NVMe | | AC-NEW-4 | No | Phase 3 — Monte Carlo false-position budget | | AC-NEW-5 | DEFERRED (hardware) | hot/cold soak chamber | | AC-NEW-6 | No | Phase 3 — freshness enforcement | | AC-NEW-7 | DEFERRED (hardware) | multi-flight voting | | AC-NEW-8 | No | Phase 3 — visual blackout + GPS spoofing budget | **Summary:** 14 ACs covered / 35 non-deferred / 4 deferred = 40% initial coverage. Expected — Phase 3+ plans will add the bulk of missing tests. ## Outstanding Orphan List for Plan 02-05 The following 21 non-deferred ACs have zero tests after Plan 02-04. Per the plan-recorded decision, Plan 02-05 should annotate these with `status: pending-phase-3` in the AC doc and widen the `--check` orphan detector to exempt `pending-phase-3` ACs: ``` AC-2.1b AC-3.1 AC-3.2 AC-3.5 AC-4.2 AC-4.5 AC-5.1 AC-5.3 AC-6.1 AC-6.2 AC-7.1 AC-7.2 AC-8.1 AC-8.2 AC-8.3 AC-8.4 AC-8.5 AC-8.6 AC-NEW-4 AC-NEW-6 AC-NEW-8 ``` (21 non-deferred orphans; `AC-NEW-3` has partial deferred status and is not counted.) ## --check Determinism Confirmation Two consecutive `python scripts/gen_ac_traceability.py --check` runs produce identical stderr output and identical exit code (rc=1 in both runs). `diff /tmp/c1.log /tmp/c2.log` is empty. ## Final Regression Count ``` 216 passed, 8 skipped in 13.44s ``` Baseline: `passed=216`. Current: 216 passed. Parity maintained. ## Git-Diff Hygiene Check ``` git diff tests/test_acceptance.py tests/test_accuracy.py \ tests/test_processor_pipe.py tests/test_gps_input_encoding.py \ tests/test_sitl_integration.py tests/test_mavlink.py tests/test_schemas.py \ | grep -E '^[-+]' \ | grep -vE '^(\+\+\+|---|@@|[-+]@pytest\.mark\.ac\(|[-+]pytestmark = \[pytest\.mark\.|[-+]\s*$)' ``` Output: empty. Only `@pytest.mark.ac(...)` decorators and one `pytestmark =` list extension were added. No logic edits. ## Deviations from Plan None — plan executed exactly as written. The `.planning/AC-TRACEABILITY.md` was committed with `git add -f` because the `.gitignore` excludes `.planning/` globally, but other `.planning/` files (STATE.md, ROADMAP.md, REQUIREMENTS.md) are already tracked via prior force-adds — this is consistent with project convention. ## Known Stubs None. The script and matrix are fully wired: script produces real data from pytest collection, matrix reflects actual tagged tests. ## Threat Flags None. Script reads local filesystem and invokes pytest as subprocess — no new network endpoints, auth paths, or trust-boundary surface introduced. ## Self-Check: PASSED - `scripts/gen_ac_traceability.py` exists: FOUND - `.planning/AC-TRACEABILITY.md` exists with 39 ACs: FOUND - Commits 4bf6f67, 6a1cd51, 419e9c5: verified via `git log --oneline`