gps-denied-onboard/_docs/02_document/tests/traceability-matrix.md

Traceability Matrix

This matrix is the canonical view of test coverage for the planning context. It traces every numbered AC and every restriction to the test scenario IDs that exercise it.

Coverage discipline: an AC counts as Covered when at least one test scenario has a quantifiable pass/fail criterion that exercises it. PARTIAL rows are exercised but with reduced confidence — the row's "Mitigation" column points to the action item (Plan-phase decision or D-PROJ gate) that, when resolved, lifts the row to Covered. NOT COVERED rows are deliberately deferred (out-of-scope for data acquisition per Phase 1 gate, or covered at a later workflow stage); each has a stated mitigation.

Acceptance Criteria Coverage

AC ID	Acceptance Criterion (one-line)	Test IDs	Coverage
AC-1.1	Frame-center GPS within 50 m for ≥80% of normal-flight photos	FT-P-01, FT-P-21 (orchestrator-level supplementary)	Covered
AC-1.2	Frame-center GPS within 20 m for ≥50% of normal-flight photos	FT-P-01, FT-P-21 (orchestrator-level supplementary)	Covered
AC-1.3	Cumulative drift between satellite-anchored fixes <100 m visual / <50 m IMU-fused	FT-P-02	Covered
AC-1.4	Estimate reports 95% covariance + source label	FT-P-03	Covered
AC-2.1a	Frame-to-frame registration ≥95% on normal segments	FT-P-04	Covered
AC-2.1b	Satellite-anchor registration meets AC-1.1/1.2/2.2/8.2/8.6	FT-P-05, FT-P-19	Covered
AC-2.2	MRE <1 px frame-to-frame, <2.5 px cross-domain	FT-P-05, FT-P-06	Covered
AC-3.1	Tolerate up to 350 m outliers, tilt ±20°	FT-N-01	Covered
AC-3.2	Tolerate sharp turns; recovery via satellite re-loc	FT-P-07, FT-N-02	Covered
AC-3.3	Handle ≥3 disconnected segments via satellite re-loc	FT-P-08	Covered
AC-3.4	On ≥3 frames + ≥2 s outage, request operator re-loc; FC dead-reckons	FT-N-03	Covered
AC-3.5	Visual blackout + spoofed GPS failsafe	FT-N-04	Covered
AC-4.1	E2E latency <400 ms p95	NFT-PERF-01 (Tier-2), NFT-RES-05 (bootstrap precondition)	Covered
AC-4.2	Memory <8 GB on Jetson	NFT-LIM-01 (Tier-2)	Covered
AC-4.3	FC output contract: GPS_INPUT (AP) + MSP2_SENSOR_GPS (iNav) with honest covariance	FT-P-03, FT-P-09-AP, FT-P-09-iNav	Covered
AC-4.4	Estimates streamed frame-by-frame	NFT-PERF-02	Covered
AC-4.5 (revised)	Internal smoothing improves past-keyframe estimates (NOT FC retroactive correction per Mode B Fact #107)	FT-P-10	Covered
AC-5.1	Init from FC EKF's last valid GPS + IMU-extrapolated	FT-P-11	Covered
AC-5.2	On >3 s without estimate, FC IMU-only fallback; SUT logs	NFT-RES-01	Covered
AC-5.3	On reboot, re-init from FC IMU-extrapolated pose	NFT-RES-02	Covered
AC-6.1	GCS stream at 1-2 Hz	FT-P-12	Covered
AC-6.2	GCS may send commands via standard MAVLink	FT-P-13	Covered
AC-6.3	WGS84 output	FT-P-14	Covered
AC-7.1	AI-camera object localization, level-flight accuracy	—	NOT COVERED — out of scope for current data acquisition (no AI-camera fixture; AC-7.x scoped to a different sensor). Mitigation: defer to a follow-up cycle with AI-camera fixture; flag in _docs/_process_leftovers/ as 2026-05-09_ai-camera-fixture-deferred.md
AC-7.2	AI-camera object coordinates from gimbal/zoom/altitude	—	NOT COVERED — same as AC-7.1
AC-8.1	Imagery via Suite Sat Service offline cache, ≥0.5 m/px	FT-P-15, FT-P-16, NFT-SEC-02	Covered
AC-8.2	Tile freshness <6 mo (active-conflict) / <12 mo (rear)	FT-N-05	Covered
AC-8.3	Imagery pre-loaded onto companion before flight	FT-P-15, FT-P-16, FT-P-21 (route-driven via real satellite-provider)	Covered
AC-8.4	Mid-flight tile generation with quality metadata	FT-P-17	Covered
AC-8.5	No raw nav/AI-cam frame retention except thumbnail log	FT-P-18	Covered
AC-8.6	Satellite relocalization scale-ratio + scene-change	FT-P-19 (scale FULL; scene-change PARTIAL)	PARTIAL — scene-change subset reduced confidence (only 2/60 stills have paired sat refs; no labeled change-pair dataset). Independent of the AC-NEW-4 / AC-NEW-7 multi-flight gap (those rows were resolved by AC-text relaxation 2026-05-09; AC-8.6 scene-change still requires a labeled change-pair dataset that synthetic perturbations cannot substitute for). Mitigation: deferred to a follow-up cycle when labeled change-pair data becomes available; surfaced in the Step 4 risk register
AC-NEW-1	Cold-start TTFF <30 s p95	NFT-PERF-03 (Tier-2), NFT-RES-05 (bootstrap precondition)	Covered
AC-NEW-2	Spoofing-promotion latency <3 s p95	NFT-PERF-04	Covered
AC-NEW-3	FDR ≤64 GB / flight, no silent drops	NFT-LIM-02	Covered
AC-NEW-4	False-position safety: P(>500 m)<0.1%, P(>1 km)<0.01%	NFT-RES-03	Covered — AC text relaxed 2026-05-09 to Monte-Carlo-over-current-data with stated 95% CI (Plan Phase 2a.0 outcome). Multi-flight statistical headroom is residual risk in the Step 4 risk register; D-PROJ-3 reopens validation when additional multi-flight data becomes available
AC-NEW-5	Operating envelope -20 °C to +50 °C, 25 W TDP, 8 h, no throttle	NFT-LIM-04 (workstation baseline only)	PARTIAL — workstation thermal-day baseline only. Mitigation: chamber-attached Jetson runner + DO-160G shaker rig — out of scope for data-acquisition per Phase 1 gate; tracked as a release-tag-blocking gate
AC-NEW-6	System rejects/downgrades stale tiles	FT-N-05, FT-N-06	Covered
AC-NEW-7	Cache poisoning: P(misalign>30 m)<1%, P(>100 m)<0.1%	NFT-SEC-01	Covered (onboard-side) — AC text relaxed 2026-05-09 to Monte-Carlo-over-current-data with stated 95% CI for the onboard contribution. Cross-suite voting-layer contract verification (D-PROJ-2) is a parent-suite design task tracked outside this Plan cycle; multi-flight statistical headroom remains residual risk (D-PROJ-3)
AC-NEW-8	Visual blackout + spoof degraded-mode escalation	FT-N-04, NFT-RES-04	Covered

Restrictions Coverage

Restriction ID	Restriction (one-line)	Test IDs	Coverage
RESTRICT-UAV-1	Fixed-wing UAV, nav-camera fixed downward	FT-N-01 (tilt envelope)	Covered (envelope assertion)
RESTRICT-UAV-2	Mission profile: 8 h flights, 60 km/h, ≤400 km² area	NFT-LIM-01, NFT-LIM-02 (8 h replay)	Covered
RESTRICT-UAV-3	Sharp turns may share <5% overlap	FT-P-07, FT-N-02	Covered
RESTRICT-UAV-4	No raw-photo storage; tile cache + FDR only	FT-P-18, NFT-LIM-03	Covered
RESTRICT-CAM-1	Nav camera ADTi 20MP 20L V1 nadir-fixed	FT-N-01 (tilt envelope), test fixture validation	Covered
RESTRICT-CAM-2	AI camera: gimbal+zoom only; level-flight scope	—	NOT COVERED — paired with AC-7.x deferral
RESTRICT-SAT-1	Onboard cache offline-only; no in-flight Service calls	FT-P-16, NFT-SEC-02, NFT-SEC-05	Covered
RESTRICT-SAT-2	Cache budget 10 GB across operational area	NFT-LIM-03	Covered
RESTRICT-SAT-3	Tile freshness per AC-8.2 / AC-NEW-6	FT-N-05, FT-N-06	Covered
RESTRICT-SAT-4	No Sentinel-2 / sub-0.5 m/px imagery	FT-P-15 (resolution floor)	Covered
RESTRICT-HW-1	Jetson Orin Nano Super, 8 GB shared LPDDR5, 25 W	NFT-LIM-01, NFT-LIM-04, NFT-LIM-05	Covered
RESTRICT-HW-2	Cooling 25 W continuous, 8 h, upper temp envelope	NFT-LIM-04, deferred chamber test	PARTIAL — chamber portion deferred; same as AC-NEW-5
RESTRICT-FC-1	ArduPilot Plane + iNav supported; PX4 out of scope	FT-P-09-AP, FT-P-09-iNav, parameterized matrix	Covered
RESTRICT-FC-2	iNav has no inbound MAVLink ext-positioning; MSP2 only	FT-P-09-iNav	Covered
RESTRICT-FC-3	Output contract: WGS84 GPS via per-FC interface	FT-P-09-AP, FT-P-09-iNav, FT-P-14	Covered
RESTRICT-COMM-1	MAVLink for GCS link (QGroundControl)	FT-P-12, FT-P-13	Covered
RESTRICT-COMM-2	iNav has no MAVLink signing; accepted residual risk	NFT-SEC-03 (asymmetry note)	Covered (documented asymmetry)
RESTRICT-FAIL-1	>3 s no estimate → FC IMU-only fallback	NFT-RES-01	Covered
RESTRICT-FAIL-2	False-position safety budget (AC-NEW-4)	NFT-RES-03	Covered (via AC-NEW-4 relaxation 2026-05-09); multi-flight statistical headroom is residual risk in Step 4
RESTRICT-FAIL-3	Cold-start TTFF (AC-NEW-1), spoofing-promotion (AC-NEW-2)	NFT-PERF-03, NFT-PERF-04	Covered

Coverage Summary

Revised 2026-05-09 (Plan Phase 2a.0 outcomes): three rows moved PARTIAL → Covered (AC-NEW-4, AC-NEW-7, RESTRICT-FAIL-2) following AC-text relaxation per Q3=B. Restriction row count corrected from 19 to 20 (pre-existing arithmetic error).

Revised 2026-05-19 (Greenfield Step 12 cycle-update — autodev): NFT-RES-05 appended to resilience-tests.md capturing the composition-root bootstrap contract introduced by AZ-591 / AZ-618 / AZ-687 (replay-mode minimal config, AirborneBootstrapError operator-error contract, Tier-2 replay.compose_root.ready + replay.input.frame_emitted log-boundary gate). NFT-RES-05 is added to AC-NEW-1 and AC-4.1 as bootstrap-precondition coverage; no coverage counts move because the scenario is supplementary, not promoting any PARTIAL row.

Revised 2026-05-24 (Existing-code cycle-3 Step 12 cycle-update — autodev): FT-P-21 appended to blackbox-tests.md capturing the Epic AZ-835 orchestrator-level end-to-end pipeline (AZ-836 RouteSpec extractor + AZ-838 SatelliteProviderRouteClient + AZ-839 route-driven operator_pre_flight_setup + AZ-840 orchestrator test). FT-P-21 is supplementary route-driven coverage on AC-1.1, AC-1.2 (orchestrator-level pipeline accuracy) and AC-8.3 (pre-loaded cache realised via the production C11→satellite-provider path rather than the bbox-seeded FT-P-15/FT-P-16 fixture). No coverage counts move — FT-P-21 supplements already-Covered rows. Currently blocked on Jetson by AZ-848 (eskf_out_of_order regression introduced by AZ-776's missing Jetson-verification gate — pre-existing, surfaced cycle-3 Step 11; tracked locally at _docs/02_tasks/todo/AZ-848_jetson_eskf_out_of_order_regression.md). Cycle-3 internal changes (C11 contract adaptation per AZ-777 Phase 1; RouteSpec relocation per AZ-845; module-layout refresh AZ-846; AZ-270 lint widening AZ-847; C12 cold-start unit-NFR threshold relax AZ-844) are implementation-only and produce no new black-box scenarios.

Category	Total Items	Covered	PARTIAL	Not Covered	Coverage % (Covered + PARTIAL counted half)
Acceptance Criteria	39	35	2	2	92.3%
Restrictions	20	18	1	1	92.5%
Total	59	53	3	3	92.4%

Coverage clears the 75% gate with margin under both the inclusive reading (PARTIAL = covered) and the strict reading (PARTIAL not counted) — strict coverage is (53 / 59) = 89.8%. The remaining PARTIAL / Not Covered items are: AC-8.6 scene-change subset (needs labeled change-pair dataset, deferred), AC-NEW-5 hot-soak chamber (physical hardware, deferred), AC-7.1 / AC-7.2 (no AI-camera fixture, deferred), RESTRICT-CAM-2 (paired with AC-7.x), RESTRICT-HW-2 chamber portion (paired with AC-NEW-5).

Uncovered Items Analysis

Revised 2026-05-09 (Plan Phase 2a.0): AC-NEW-4 and AC-NEW-7 rows removed from this section after AC-text relaxation (Q3=B) flipped them to Covered with residual risk tracked in the Step 4 risk register.

Item	Reason Not Covered	Risk	Mitigation
AC-7.1	No AI-camera fixture in input_data/; AC scoped to a different sensor than the nav camera; level-flight assumption + bank/pitch <5° is independent of the nav-cam pipeline	Object-localization accuracy untested; AI consumers may receive wrong coordinates if not flight-tested	Deferred to a follow-up Plan cycle scoped to AI-camera integration; recorded in _docs/_process_leftovers/2026-05-09_ai-camera-fixture-deferred.md (will be created in Phase 3 if confirmed).
AC-7.2	Same as AC-7.1	Same	Same
AC-8.6 (scene-change subset)	Only 2/60 stills paired with _gmaps.png; no labeled change-pair dataset bundled in input_data/. Independent of the AC-NEW-4 / AC-NEW-7 multi-flight gap (those were resolved by AC-text relaxation; AC-8.6 still needs labeled change-pair data)	Stale-tile match in active-conflict sectors may yield false satellite_anchored; AC-NEW-6 partially compensates but scene-change recall is unmeasured	Deferred to a follow-up cycle when labeled change-pair data becomes available (Maxar Open Data Ukraine + AerialVL change-pair subset). Scale-ratio half of AC-8.6 IS covered.
AC-NEW-5	Workstation thermal-day baseline only. AC-NEW-5 hot-soak (25 W @ +50 °C, 8 h, no throttle) requires a thermal chamber — physical hardware, not data	Without chamber test, AC-4.1 latency budget at +50 °C is not validated; D-CROSS-LATENCY-1 hybrid auto-degrade unproven under real thermal stress	Chamber-attached Jetson runner gated as release-tag-blocker. NOT counted as data-acquisition deferral; counted as physical hardware deferral.
RESTRICT-CAM-2	Paired with AC-7.x — no AI-camera fixture	Same as AC-7.x	Same as AC-7.x
RESTRICT-HW-2 (chamber portion)	Paired with AC-NEW-5 — physical chamber required	Same as AC-NEW-5	Same as AC-NEW-5

New findings forwarded into Plan (Steps 2 + 3 inputs)

These insights from Phase 2 augment the F1-F5 carried over from Phase 1; together they feed forward into Solution Analysis (Step 2) and Component Decomposition (Step 3):

1. F6 — Two-tier execution profile is a first-class architectural concern. The split between Tier-1 (workstation Docker) and Tier-2 (Jetson hardware) means several AC have validation locations that must appear in the deployment plan and in the CI matrix design. Add a "Tier-2 hardware-runner availability" entry to the project's risk register (Step 4).
2. F7 — mock-suite-sat-service is an e2e-test fixture for the not-yet-shipped D-PROJ-2 POST contract. It is not a first-class component (ADR-007 reversed 2026-05-09); the architectural counterparty for both download and upload is the real satellite-provider. The contract sketch is the source of truth and lives in _docs/_process_leftovers/2026-05-09_satellite-provider-design-tasks.md; the fixture mirrors it for NFT-SEC-01 / FT-P-17 / IT runs and is retired when the real endpoint ships. Component decomposition (Step 3) treats the Service-publish contract as a C11 TileUploader ↔ satellite-provider boundary (not buried inside C8).
3. F8 — VioStrategy parameterization in CI requires both a production binary AND a research binary. D-C1-1-SUB-A locked the BUILD_VINS_MONO=ON/OFF split; the test plan must produce both binaries on every PR for the comparative-study report (IT-12 in solution.md). Add to deployment plan (Step 2) and to epic/work-item planning (Step 6).
4. F9 — D-PROJ-3 (fixture acquisition) is now a named deliverable with a clear gate: must resolve before greenfield Step 5 re-runs the full test-spec with architecture context. Promote to risk register and to the architecture's open-items list.
5. F10 — Defense-in-depth security layer (NFT-SEC-05 DNS blackholing, OPENCV ASan build, SBOM signing-passkey verification) implies CI/build infrastructure features (multi-stage build for ASan instrumentation, SBOM generator, lockfile linter). Add to deployment plan (Step 2).