mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 07:01:14 +00:00
docs(02-04): complete plan summary — 14 ACs covered, 216 passed, --check deterministic
This commit is contained in:
Yuzviak
+180
@@ -0,0 +1,180 @@
|
||||
---
|
||||
phase: 02-acceptance-criteria-test-taxonomy-observability-spine
|
||||
plan: "04"
|
||||
subsystem: traceability
|
||||
tags: [ac-traceability, pytest-markers, bidirectional-coverage]
|
||||
dependency_graph:
|
||||
requires: [02-01, 02-02, 02-03]
|
||||
provides: [scripts/gen_ac_traceability.py, .planning/AC-TRACEABILITY.md, @pytest.mark.ac decorations]
|
||||
affects: [02-05-CI-gate, AC-06-satisfaction]
|
||||
tech_stack:
|
||||
added: []
|
||||
patterns: [argparse-cli, subprocess-pytest-collect, ac-marker-plugin, class-level-pytestmark]
|
||||
key_files:
|
||||
created:
|
||||
- scripts/gen_ac_traceability.py
|
||||
- .planning/AC-TRACEABILITY.md
|
||||
modified:
|
||||
- tests/test_acceptance.py
|
||||
- tests/test_accuracy.py
|
||||
- tests/test_processor_pipe.py
|
||||
- tests/test_gps_input_encoding.py
|
||||
- tests/test_sitl_integration.py
|
||||
- tests/test_mavlink.py
|
||||
- tests/test_schemas.py
|
||||
decisions:
|
||||
- "AC-TRACEABILITY.md committed to .planning/ with git add -f (path is gitignored but other .planning files are tracked)"
|
||||
- "test_gps_input_encoding.py marked at module/pytestmark level (all 13 tests -> AC-4.3)"
|
||||
- "TestGPSPoint and TestWaypoint marked at class level -> AC-6.3 (propagates to methods)"
|
||||
- "Orphan ACs get pending-phase-3 annotation in Plan 02-05 (documented decision carried forward)"
|
||||
metrics:
|
||||
duration: "~15 minutes"
|
||||
completed: "2026-05-11"
|
||||
tasks_completed: 4
|
||||
files_created: 2
|
||||
files_modified: 7
|
||||
---
|
||||
|
||||
# Phase 02 Plan 04: AC Traceability Script + Test Decorations Summary
|
||||
|
||||
One-liner: `scripts/gen_ac_traceability.py` + 38 `@pytest.mark.ac(...)` decorations cover 14 of 35 non-deferred ACs in the initial matrix, with `--check` deterministic for CI gating.
|
||||
|
||||
## Tasks Completed
|
||||
|
||||
| Task | Status | Commit |
|
||||
|------|--------|--------|
|
||||
| 1: Implement gen_ac_traceability.py | Done | 4bf6f67 |
|
||||
| 2: Decorate 7 test files | Done | 6a1cd51 |
|
||||
| 3: Regenerate and commit AC-TRACEABILITY.md | Done | 419e9c5 |
|
||||
| 4: Regression gate | Done | (verification only) |
|
||||
|
||||
## Task 1: Script Implementation
|
||||
|
||||
The script matches RESEARCH.md §2.3 exactly with one minor adaptation: the `--strict` alias is added as `--check, --strict` on the same argument (both names accepted by argparse via nargs).
|
||||
|
||||
**Functions implemented:**
|
||||
- `collect_acs_from_doc()` — regex parses `- **AC-X.Y**` headers + `deferred-hardware` token within each block
|
||||
- `collect_acs_from_tests()` — subprocess `pytest --collect-only -q --ac-dump=<tmp>` reads the JSON from conftest Plan 02-02 hook
|
||||
- `render_md()` — produces header stats, `| AC ID | Test count | Tests | Status |` table, backward-orphan section
|
||||
- `main()` — argparse + `--check` exit-1 on any non-deferred orphan or unknown AC ID
|
||||
|
||||
**Deviations vs RESEARCH.md §2.3:** None material. The plan's verbatim source was used as written.
|
||||
|
||||
## Task 2: Decoration Counts Per File
|
||||
|
||||
| File | Decorator lines added | AC IDs tagged | Notes |
|
||||
|------|-----------------------|---------------|-------|
|
||||
| test_acceptance.py | 8 | AC-1.1, AC-2.1a, AC-3.3, AC-3.4, AC-4.1, AC-4.4, AC-1.4 | per-function decorators |
|
||||
| test_accuracy.py | 11 | AC-1.1, AC-1.2, AC-1.3, AC-2.1a, AC-2.2, AC-4.1 | per-function decorators |
|
||||
| test_processor_pipe.py | 2 | AC-4.4, AC-1.4 | per-function decorators |
|
||||
| test_gps_input_encoding.py | 1 | AC-4.3 | module-level pytestmark (13 tests inherit) |
|
||||
| test_sitl_integration.py | 9 | AC-4.3, AC-4.4, AC-NEW-2, AC-5.2, AC-3.4 | per-function decorators |
|
||||
| test_mavlink.py | 5 | AC-4.3, AC-5.2, AC-3.4 | per-function decorators |
|
||||
| test_schemas.py | 2 | AC-6.3 | class-level decorators (propagate to methods) |
|
||||
| **Total** | **38** | **14 unique ACs** | |
|
||||
|
||||
Total tests collected under `-m ac`: **45** (from 298 collected).
|
||||
|
||||
## AC Coverage Table
|
||||
|
||||
| AC ID | Covered? | Reason if orphan |
|
||||
|-------|----------|-----------------|
|
||||
| AC-1.1 | Yes | test_acceptance (2), test_accuracy (3) |
|
||||
| AC-1.2 | Yes | test_accuracy (2) |
|
||||
| AC-1.3 | Yes | test_accuracy (1) |
|
||||
| AC-1.4 | Yes | test_acceptance (1), test_processor_pipe (1) |
|
||||
| AC-2.1a | Yes | test_acceptance (1), test_accuracy (1) |
|
||||
| AC-2.1b | No | No test explicitly checks cross-domain registration separately from VO |
|
||||
| AC-2.2 | Yes | test_accuracy (1) |
|
||||
| AC-3.1 | No | No outlier-tolerance test in Phase 2 test files |
|
||||
| AC-3.2 | No | No sharp-turn test in Phase 2 test files |
|
||||
| AC-3.3 | Yes | test_acceptance (1) — factor graph convergence |
|
||||
| AC-3.4 | Yes | test_acceptance (1), test_mavlink (1), test_sitl (1) |
|
||||
| AC-3.5 | No | No visual-blackout mode-switch test in Phase 2 |
|
||||
| AC-4.1 | Yes | test_acceptance (1), test_accuracy (2) |
|
||||
| AC-4.2 | No | No memory-bound test exists in Phase 2 |
|
||||
| AC-4.3 | Yes | test_gps_input_encoding (12), test_mavlink (3), test_sitl (5) |
|
||||
| AC-4.4 | Yes | test_acceptance (3), test_processor_pipe (1), test_sitl (1) |
|
||||
| AC-4.5 | No | No refinement/correction round-trip test in Phase 2 |
|
||||
| AC-5.1 | No | Phase 3 target — FC EKF init test not yet written |
|
||||
| AC-5.2 | Yes | test_mavlink (1), test_sitl (1) |
|
||||
| AC-5.3 | No | Phase 3 target — mid-flight reboot recovery |
|
||||
| AC-6.1 | No | No GCS downsample rate test in Phase 2 |
|
||||
| AC-6.2 | No | No operator reloc command test in Phase 2 |
|
||||
| AC-6.3 | Yes | test_schemas TestGPSPoint (4), TestWaypoint (2) |
|
||||
| AC-7.1 | No | Phase 3+ — AI camera localization accuracy |
|
||||
| AC-7.2 | No | Phase 3+ — trig computation |
|
||||
| AC-8.1 | No | Phase 3/4 — satellite imagery integration |
|
||||
| AC-8.2 | No | Phase 3/4 — tile freshness |
|
||||
| AC-8.3 | No | Phase 3/4 — pre-flight loading |
|
||||
| AC-8.4 | No | deferred-stage3 per AC doc |
|
||||
| AC-8.5 | No | Phase 4 — storage policy |
|
||||
| AC-8.6 | No | Phase 4/VPR — conditional VPR |
|
||||
| AC-NEW-1 | DEFERRED (hardware) | cold-boot bench requires Jetson |
|
||||
| AC-NEW-2 | Yes | test_sitl (1) — spoofing-promotion via GPS_INPUT accepted |
|
||||
| AC-NEW-3 | No (partial deferred) | integration-test scope not yet written; deferred-hardware for real NVMe |
|
||||
| AC-NEW-4 | No | Phase 3 — Monte Carlo false-position budget |
|
||||
| AC-NEW-5 | DEFERRED (hardware) | hot/cold soak chamber |
|
||||
| AC-NEW-6 | No | Phase 3 — freshness enforcement |
|
||||
| AC-NEW-7 | DEFERRED (hardware) | multi-flight voting |
|
||||
| AC-NEW-8 | No | Phase 3 — visual blackout + GPS spoofing budget |
|
||||
|
||||
**Summary:** 14 ACs covered / 35 non-deferred / 4 deferred = 40% initial coverage. Expected — Phase 3+ plans will add the bulk of missing tests.
|
||||
|
||||
## Outstanding Orphan List for Plan 02-05
|
||||
|
||||
The following 21 non-deferred ACs have zero tests after Plan 02-04. Per the plan-recorded decision, Plan 02-05 should annotate these with `status: pending-phase-3` in the AC doc and widen the `--check` orphan detector to exempt `pending-phase-3` ACs:
|
||||
|
||||
```
|
||||
AC-2.1b AC-3.1 AC-3.2 AC-3.5 AC-4.2
|
||||
AC-4.5 AC-5.1 AC-5.3 AC-6.1 AC-6.2
|
||||
AC-7.1 AC-7.2 AC-8.1 AC-8.2 AC-8.3
|
||||
AC-8.4 AC-8.5 AC-8.6 AC-NEW-4 AC-NEW-6
|
||||
AC-NEW-8
|
||||
```
|
||||
|
||||
(21 non-deferred orphans; `AC-NEW-3` has partial deferred status and is not counted.)
|
||||
|
||||
## --check Determinism Confirmation
|
||||
|
||||
Two consecutive `python scripts/gen_ac_traceability.py --check` runs produce identical stderr output and identical exit code (rc=1 in both runs). `diff /tmp/c1.log /tmp/c2.log` is empty.
|
||||
|
||||
## Final Regression Count
|
||||
|
||||
```
|
||||
216 passed, 8 skipped in 13.44s
|
||||
```
|
||||
|
||||
Baseline: `passed=216`. Current: 216 passed. Parity maintained.
|
||||
|
||||
## Git-Diff Hygiene Check
|
||||
|
||||
```
|
||||
git diff tests/test_acceptance.py tests/test_accuracy.py \
|
||||
tests/test_processor_pipe.py tests/test_gps_input_encoding.py \
|
||||
tests/test_sitl_integration.py tests/test_mavlink.py tests/test_schemas.py \
|
||||
| grep -E '^[-+]' \
|
||||
| grep -vE '^(\+\+\+|---|@@|[-+]@pytest\.mark\.ac\(|[-+]pytestmark = \[pytest\.mark\.|[-+]\s*$)'
|
||||
```
|
||||
|
||||
Output: empty. Only `@pytest.mark.ac(...)` decorators and one `pytestmark =` list extension were added. No logic edits.
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None — plan executed exactly as written.
|
||||
|
||||
The `.planning/AC-TRACEABILITY.md` was committed with `git add -f` because the `.gitignore` excludes `.planning/` globally, but other `.planning/` files (STATE.md, ROADMAP.md, REQUIREMENTS.md) are already tracked via prior force-adds — this is consistent with project convention.
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None. The script and matrix are fully wired: script produces real data from pytest collection, matrix reflects actual tagged tests.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
None. Script reads local filesystem and invokes pytest as subprocess — no new network endpoints, auth paths, or trust-boundary surface introduced.
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- `scripts/gen_ac_traceability.py` exists: FOUND
|
||||
- `.planning/AC-TRACEABILITY.md` exists with 39 ACs: FOUND
|
||||
- Commits 4bf6f67, 6a1cd51, 419e9c5: verified via `git log --oneline`
|
||||
Reference in New Issue
Block a user