mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 04:01:13 +00:00
[AZ-416] [AZ-417] [AZ-419] Test batch 72: FT-P-09 AP/iNav + FT-P-11 cold start
- AZ-416 (FT-P-09-AP): fills mavproxy_tlog_reader.iter_messages with pymavlink body (AZ-406 surface kept); adds ap_contract_evaluator covering AC-1 (signing handshake <=5s), AC-2 (GPS_INPUT >=4.5 Hz), AC-3 (EK3_SRC1_POSXY=3), AC-4 (GPS_RAW_INT health >=80%); scenario forces fc_adapter=ardupilot. - AZ-417 (FT-P-09-iNav): msp_frame_observer covering AC-2 (MSP rate) and AC-3 (fix_type/provider/numSat); scenario forces fc_adapter=inav. - AZ-419 (FT-P-11): cold_start_evaluator covering AC-1 (operator manifest origin), AC-2 (FC EKF fallback), AC-3 (no-origin abort), AC-4 (bounded-delta conflict, ADR-010 Principle #11 amended); scenario parametrized on origin_source plus dedicated no-origin abort scenario. - All scenarios skip-gated on upstream frame_source_replay / imu_replay / fdr_reader / sitl_observer extensions. - +67 unit tests; full e2e unit suite: 460 passed. - K=3 cumulative review fired: PASS for batches 70-72. See _docs/03_implementation/batch_72_report.md, _docs/03_implementation/reviews/batch_72_review.md, _docs/03_implementation/cumulative_review_batches_70-72_cycle1_report.md. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,142 @@
|
||||
# Batch 72 Report — Test Implementation (cycle 1, batch 6 of test phase)
|
||||
|
||||
**Batch**: 72
|
||||
**Date**: 2026-05-16
|
||||
**Context**: Test implementation (greenfield Step 10 — Implement Tests)
|
||||
**Tasks**: AZ-416 (5pt), AZ-417 (3pt), AZ-419 (3pt) — 11 cp / 3 tasks
|
||||
**Cycle**: 1
|
||||
**Verdict**: COMPLETE — PASS (self-reviewed + K=3 cumulative reviewed; see
|
||||
`reviews/batch_72_review.md` and `cumulative_review_batches_70-72_cycle1_report.md`)
|
||||
|
||||
## Summary
|
||||
|
||||
FC contract conformance + cold-start init — the three remaining
|
||||
scenarios that consume mavproxy / signing / cold-boot fixtures already
|
||||
built in batches 67-68. Same pattern as prior batches:
|
||||
|
||||
* Pure-logic helper under `e2e/runner/helpers/` (everything the
|
||||
scenario can express without docker-bound SITL access).
|
||||
* Scenario file(s) under `e2e/tests/positive/`, parameterized across
|
||||
conftest fixtures, skip-gated on upstream replay / SITL observer
|
||||
/ FDR helpers (auto-activates when AZ-441 + AZ-407 leftovers land).
|
||||
* Helper-driven unit test file under `e2e/_unit_tests/helpers/`.
|
||||
|
||||
### AZ-416 — FT-P-09-AP ArduPilot signing + GPS_INPUT contract (5pt)
|
||||
|
||||
* **`runner/helpers/mavproxy_tlog_reader.py`** — AZ-416 fills in the
|
||||
pymavlink-backed `iter_messages` body that AZ-406 reserved. Uses
|
||||
`mavutil.mavlink_connection(str(tlog_path))` with `recv_match` to
|
||||
iterate frames; exposes `TlogMessage(timestamp_us, msg_type, signed,
|
||||
fields)`. The `signed` flag uses `msg.get_signed()` with a
|
||||
defensive `AttributeError` fallback. The function is FAIL-FAST on
|
||||
missing files (raises FileNotFoundError); pymavlink's BAD_DATA
|
||||
frames are skipped silently per the standard idiom.
|
||||
* **`runner/helpers/ap_contract_evaluator.py`** — four analysers:
|
||||
- `observe_signing_handshake` (AC-1): first signed frame within
|
||||
`HANDSHAKE_BUDGET_S = 5.0` s AND no `BAD_SIGNATURE` STATUSTEXT
|
||||
within that window.
|
||||
- `compute_gps_input_rate` (AC-2): GPS_INPUT cadence ≥4.5 Hz
|
||||
(constant `GPS_INPUT_MIN_RATE_HZ`).
|
||||
- `validate_ek3_src1_posxy` (AC-3): the AP EKF source-set parameter
|
||||
must equal `EK3_SRC1_POSXY_REQUIRED = 3` (GPS).
|
||||
- `evaluate_gps_raw_int_health` (AC-4): GPS_RAW_INT
|
||||
`fix_type ≥ 3 AND eph ≤ 200` for ≥80 % of the window.
|
||||
- `collect_messages_to_list` — explicit single-pass-iterator
|
||||
materialisation so multiple analysers can share the tlog.
|
||||
* **`tests/positive/test_ft_p_09_ap_signing.py`** — scenario forces
|
||||
`fc_adapter=ardupilot` (skips other adapters), parameterised per
|
||||
`vio_strategy`. Records `signing_handshake_s`,
|
||||
`gps_input_rate_hz`, `ek3_src1_posxy`, `gps_raw_int_healthy_fraction`
|
||||
NFR metrics with AC IDs.
|
||||
* **22 unit tests** in `test_ap_contract_evaluator.py` + **6** in
|
||||
`test_mavproxy_tlog_reader.py`.
|
||||
|
||||
### AZ-417 — FT-P-09-iNav MSP2_SENSOR_GPS contract (3pt)
|
||||
|
||||
* **`runner/helpers/msp_frame_observer.py`** — pure logic for AC-2
|
||||
(`compute_rate_hz` with `MSP2_SENSOR_GPS_FUNCTION_ID = 0x1F03` +
|
||||
`MIN_OBSERVED_RATE_HZ = 4.5`) and AC-3 (`evaluate_inav_gps_state`
|
||||
with `MIN_FIX_TYPE = 3` and `REQUIRED_PROVIDER = "MSP"`).
|
||||
* **`tests/positive/test_ft_p_09_inav.py`** — scenario forces
|
||||
`fc_adapter=inav` (skips other adapters), parameterised per
|
||||
`vio_strategy`. Probes TCP handshake via
|
||||
`sitl_observer.observe_inav_tcp_handshake` (gated), captures MSP
|
||||
frames via `collect_inav_msp_frames` (gated), queries iNav GPS
|
||||
state via `query_inav_gps_state` (gated).
|
||||
* **14 unit tests** in `test_msp_frame_observer.py`.
|
||||
|
||||
### AZ-419 — FT-P-11 cold-start init (3pt)
|
||||
|
||||
* **`runner/helpers/cold_start_evaluator.py`** — covers ADR-010's
|
||||
primary + secondary + bounded-delta paths plus AC-3 no-origin
|
||||
abort:
|
||||
- `write_manifest` / `read_manifest` — test-fixture builder for the
|
||||
C10 Manifest's `flight.takeoff_origin` (the test fabricates one
|
||||
instead of fetching from C12 because the SUT consumes a Manifest
|
||||
file path, not a service URL).
|
||||
- `read_cold_boot_fixture` — parse the AZ-408 fixture JSON into a
|
||||
typed `ColdBootSnapshot` (converts `lat_e7 / lon_e7 / alt_mm` →
|
||||
decimal degrees + meters).
|
||||
- `evaluate_first_estimate` (AC-1/2/4): distance vs expected origin
|
||||
+ source_label rule for bounded-delta + FDR record audit.
|
||||
- `evaluate_no_origin_path` (AC-3): SUT must produce NO outbound
|
||||
estimate AND FDR must record `c5.cold_start_origin.unavailable`.
|
||||
- Constants for accuracy budget (50 m), bounded-delta trigger
|
||||
(200 m), forbidden first-label (`satellite_anchored`), and the
|
||||
three FDR record types.
|
||||
* **`tests/positive/test_ft_p_11_cold_start_init.py`** — two scenario
|
||||
functions:
|
||||
- `test_ft_p_11_cold_start_origin_variants` — parametrized on
|
||||
`origin_source ∈ {operator_manifest, fc_ekf,
|
||||
bounded_delta_conflict}`; one fixture / one assertion path per
|
||||
variant.
|
||||
- `test_ft_p_11_cold_start_no_origin_aborts` — AC-3 dedicated
|
||||
scenario.
|
||||
Both rely on `sitl_observer.prepare_sitl_cold_boot` +
|
||||
`prepare_sitl_no_gps` (gated until AZ-407 leftovers land).
|
||||
* **19 unit tests** in `test_cold_start_evaluator.py`.
|
||||
|
||||
## Tests
|
||||
|
||||
* **Full e2e unit suite**: 460 passed in 134.35 s (was 393 at end of
|
||||
batch 71 → +67 net new tests this batch).
|
||||
* **Pre-existing**: macOS-only `/e2e-results` plugin issue in
|
||||
scenario invocation outside Docker. Unit suite unaffected.
|
||||
|
||||
## Files Touched
|
||||
|
||||
**New helpers:**
|
||||
* `e2e/runner/helpers/msp_frame_observer.py`
|
||||
* `e2e/runner/helpers/ap_contract_evaluator.py`
|
||||
* `e2e/runner/helpers/cold_start_evaluator.py`
|
||||
|
||||
**Modified helper:**
|
||||
* `e2e/runner/helpers/mavproxy_tlog_reader.py` — AZ-416 fills the
|
||||
pymavlink-backed `iter_messages` body that AZ-406 reserved
|
||||
(NotImplementedError → real iterator). Surface unchanged.
|
||||
|
||||
**New unit tests:**
|
||||
* `e2e/_unit_tests/helpers/test_mavproxy_tlog_reader.py` (6 tests)
|
||||
* `e2e/_unit_tests/helpers/test_ap_contract_evaluator.py` (22 tests)
|
||||
* `e2e/_unit_tests/helpers/test_msp_frame_observer.py` (14 tests)
|
||||
* `e2e/_unit_tests/helpers/test_cold_start_evaluator.py` (19 tests)
|
||||
|
||||
**New scenarios:**
|
||||
* `e2e/tests/positive/test_ft_p_09_ap_signing.py`
|
||||
* `e2e/tests/positive/test_ft_p_09_inav.py`
|
||||
* `e2e/tests/positive/test_ft_p_11_cold_start_init.py`
|
||||
|
||||
**Updated:**
|
||||
* `e2e/_unit_tests/test_directory_layout.py` — added 6 new paths.
|
||||
|
||||
**Archived:**
|
||||
* `_docs/02_tasks/todo/AZ-416_*.md` → `done/`
|
||||
* `_docs/02_tasks/todo/AZ-417_*.md` → `done/`
|
||||
* `_docs/02_tasks/todo/AZ-419_*.md` → `done/`
|
||||
|
||||
## Cumulative Review Trigger
|
||||
|
||||
K=3 FIRED at end of batch 72 (last cumulative covered batches 67-69;
|
||||
since then 70 + 71 + 72 = 3 batches). Report written:
|
||||
`_docs/03_implementation/cumulative_review_batches_70-72_cycle1_report.md`.
|
||||
Verdict: PASS. Next cumulative trigger: end of batch 75.
|
||||
Reference in New Issue
Block a user