[AZ-416] [AZ-417] [AZ-419] Test batch 72: FT-P-09 AP/iNav + FT-P-11 cold start

- AZ-416 (FT-P-09-AP): fills mavproxy_tlog_reader.iter_messages with
  pymavlink body (AZ-406 surface kept); adds ap_contract_evaluator
  covering AC-1 (signing handshake <=5s), AC-2 (GPS_INPUT >=4.5 Hz),
  AC-3 (EK3_SRC1_POSXY=3), AC-4 (GPS_RAW_INT health >=80%); scenario
  forces fc_adapter=ardupilot.
- AZ-417 (FT-P-09-iNav): msp_frame_observer covering AC-2 (MSP rate)
  and AC-3 (fix_type/provider/numSat); scenario forces
  fc_adapter=inav.
- AZ-419 (FT-P-11): cold_start_evaluator covering AC-1 (operator
  manifest origin), AC-2 (FC EKF fallback), AC-3 (no-origin abort),
  AC-4 (bounded-delta conflict, ADR-010 Principle #11 amended);
  scenario parametrized on origin_source plus dedicated no-origin
  abort scenario.
- All scenarios skip-gated on upstream frame_source_replay /
  imu_replay / fdr_reader / sitl_observer extensions.
- +67 unit tests; full e2e unit suite: 460 passed.
- K=3 cumulative review fired: PASS for batches 70-72.

See _docs/03_implementation/batch_72_report.md,
_docs/03_implementation/reviews/batch_72_review.md,
_docs/03_implementation/cumulative_review_batches_70-72_cycle1_report.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-17 07:49:17 +03:00
parent c6e6cba237
commit a644debdb7
19 changed files with 3041 additions and 9 deletions
@@ -0,0 +1,74 @@
# FT-P-09-AP — ArduPilot Plane GPS_INPUT contract conformance + MAVLink 2.0 signing
**Task**: AZ-416_ft_p_09_ap_signing
**Name**: AP `GPS_INPUT` contract + MAVLink 2.0 signing handshake (AC-4.3 / D-C8-9 / AC-NEW-2 precondition)
**Description**: Implement FT-P-09-AP — start SUT with `ardupilot` adapter; signing handshake completes within 5 s; SUT emits signed `GPS_INPUT` at the configured rate during a 60 s Derkachi segment; AP `EK3_SRC1_POSXY=3` (GPS source); AP `GPS_RAW_INT` shows healthy fix.
**Complexity**: 5 points
**Dependencies**: AZ-406, AZ-407 (mavlink-passkey), AZ-408 (no — straight Derkachi 60 s segment)
**Component**: Blackbox Tests / Positive / FC contract / Security (epic AZ-262)
**Tracker**: AZ-416
**Epic**: AZ-262 (E-BBT)
## Problem
The ArduPilot wired channel is the highest-risk FC contract: signing handshake (D-C8-9 / Mode B Fact #109 mitigation), per-flight key handling, AP-side EKF source-set acceptance — all must be validated as one cohesive scenario before any production deployment.
## Outcome
- pytest scenario at `e2e/tests/positive/test_ft_p_09_ap_signing.py`.
- Forces `fc_adapter=ardupilot`; loads `mavlink-test-passkey.txt` as the Docker secret.
- Asserts: signing handshake completes within 5 s; signed-channel established; SUT emits signed `GPS_INPUT` at the configured rate during 60 s of Derkachi replay; `EK3_SRC1_POSXY` reads `3`; `GPS_RAW_INT` shows fix_type ≥ 3 with HDOP within nominal.
## Scope
### Included
- AP-only test method.
- Signing-handshake observation via mavproxy listener (handshake messages are public MAVLink).
- Param-read of `EK3_SRC1_POSXY` via mavproxy.
- `GPS_RAW_INT` parsing for fix-type + HDOP.
### Excluded
- iNav variant — owned by FT-P-09-iNav (AZ-417).
- Unsigned-message rejection (negative-path) — owned by NFT-SEC-03 (AZ-438).
- Spoofing-promotion latency — owned by NFT-PERF-04 (AZ-431).
- Source-set switch on spoof recovery — owned by FT-N-04 / NFT-RES-04.
## Acceptance Criteria
**AC-1: signing handshake completes**
Given SUT started with `ardupilot` + the test passkey
Then within ≤5 s, the signed channel is established (observable via `MAVLink2 SETUP_SIGNING` exchange OR by the absence of `BAD_SIGNATURE` STATUSTEXT during the handshake window).
**AC-2: signed `GPS_INPUT` flow**
Given the signed channel is up
When 60 s of Derkachi replays
Then `GPS_INPUT` messages reach AP SITL at the configured rate (≥4.5 Hz observed for a 5 Hz target — AC-4.3).
**AC-3: AP EKF source-set acceptance**
Given the SUT-emitted signed `GPS_INPUT` flow
Then `EK3_SRC1_POSXY` (read via mavproxy parameter request) returns `3` (GPS source).
**AC-4: AP-side GPS health**
Given the same flow
Then `GPS_RAW_INT.fix_type ≥ 3` AND `GPS_RAW_INT.eph` (HDOP × 100) within nominal (≤200 — i.e. HDOP ≤ 2.0) for ≥80 % of the 60 s window.
**AC-5: vio_strategy parameterization**
Given the conftest's `vio_strategy` parameterization
Then the scenario runs once per VIO strategy; `fc_adapter` is fixed to `ardupilot`.
## System Under Test Boundary
End-to-end through public boundaries with a real (SITL-simulated) FC.
- **Allowed**: AP SITL parameter reads via mavproxy, MAVLink message capture in `.tlog`.
- **Forbidden**: importing the SUT's signing key state, monkeypatching the handshake; if signing fails for any reason the scenario fails — it does NOT bypass to an unsigned channel.
## Constraints
- The test passkey is the fixture-level passkey; the production passkey path is `/run/secrets/mavlink_passkey` per `environment.md` and is never used in tests.
- Signing-handshake observation does NOT require parsing the encrypted contents — only the handshake message types and the rejection-vs-acceptance signal at AP.
## Document Dependencies
- `_docs/02_document/tests/blackbox-tests.md` § FT-P-09-AP
- `_docs/02_document/tests/test-data.md` § FC contract & startup (FT-P-09-AP row)
@@ -0,0 +1,67 @@
# FT-P-09-iNav — iNav MSP2_SENSOR_GPS contract conformance
**Task**: AZ-417_ft_p_09_inav
**Name**: iNav `MSP2_SENSOR_GPS` contract + provider state (AC-4.3)
**Description**: Implement FT-P-09-iNav — start SUT with `inav` adapter; TCP connection to `inav-sitl:5760` established; SUT emits `MSP2_SENSOR_GPS` (ID 0x1F03) frames at 5 Hz during 60 s of Derkachi replay; iNav GPS state shows `gpsSol.fixType ≥ 3`, `gpsSol.numSat` reflects emitted value, `provider=MSP`.
**Complexity**: 3 points
**Dependencies**: AZ-406, AZ-407
**Component**: Blackbox Tests / Positive / FC contract (epic AZ-262)
**Tracker**: AZ-417
**Epic**: AZ-262 (E-BBT)
## Problem
iNav uses MSP2 over TCP rather than MAVLink — a separate protocol pathway with no signing (per Mode B Fact #109 accepted residual risk). The protocol-conformance leg is small but distinct from the AP path; it must be validated independently.
## Outcome
- pytest scenario at `e2e/tests/positive/test_ft_p_09_inav.py`.
- Forces `fc_adapter=inav`; SUT connects to `inav-sitl:5760` via TCP; emits `MSP2_SENSOR_GPS` (ID 0x1F03) frames during 60 s of Derkachi replay.
- Reads iNav GPS state via MSP query; asserts `gpsSol.fixType ≥ 3`, `gpsSol.numSat` matches the emitted value, `provider=MSP`.
## Scope
### Included
- iNav-only test method.
- TCP connection establishment check.
- MSP2 frame ID + rate observation (5 Hz target).
- iNav GPS state read via MSP query (`msp_gps_toy` Rust binary called via subprocess per `environment.md`).
### Excluded
- AP variant — owned by FT-P-09-AP.
- Signing handshake — N/A for iNav.
- Source-set switch — N/A for iNav (no equivalent to AP `EK3_SRC1_POSXY`).
## Acceptance Criteria
**AC-1: TCP connection to iNav SITL**
Given SUT started with `fc_adapter=inav`
Then the SUT establishes a TCP connection to `inav-sitl:5760` within ≤5 s.
**AC-2: MSP2_SENSOR_GPS frame flow**
When 60 s of Derkachi replays
Then `MSP2_SENSOR_GPS` (ID 0x1F03) frames reach iNav at the configured rate (≥4.5 Hz observed for a 5 Hz target).
**AC-3: iNav GPS state**
Given the frame flow
Then a follow-up MSP query reports `gpsSol.fixType ≥ 3`, `gpsSol.numSat` reflects the emitted value, `provider=MSP` (no fallback to internal GPS).
**AC-4: vio_strategy parameterization**
Given the conftest's `vio_strategy` parameterization
Then the scenario runs once per VIO strategy; `fc_adapter` is fixed to `inav`.
## System Under Test Boundary
End-to-end through public boundaries with a real (SITL-simulated) FC.
- **Allowed**: TCP connection observation, MSP query.
- **Forbidden**: importing SUT internal state; if the TCP connection fails, the test fails (no fallback to UDP or MAVLink).
## Constraints
- The MSP2 protocol implementation in the runner uses the same `msp_gps_toy` library that the consumer-side ground tooling uses; the runner does not embed iNav internals.
## Document Dependencies
- `_docs/02_document/tests/blackbox-tests.md` § FT-P-09-iNav
- `_docs/02_document/tests/test-data.md` § FC contract & startup (FT-P-09-iNav row)
@@ -0,0 +1,81 @@
# FT-P-11 — Cold-start initialization from operator origin (primary) OR FC EKF (secondary)
**Task**: AZ-419_ft_p_11_cold_start_init
**Name**: Cold-start initialization — operator-origin-from-Manifest primary; FC EKF GPS secondary (ADR-010 + AC-5.1)
**Description**: Implement FT-P-11 — exercise both cold-start paths defined by ADR-010. **Primary path (AZ-490)**: pre-bake a `takeoff_origin` into the C10 Manifest, start SUT cold, push first nav-camera frame, assert the first outbound estimate's lat/lon falls within ±50 m of the operator origin even when the SITL FC EKF reports NO valid GPS. **Secondary path (legacy AC-5.1)**: clear the Manifest's `takeoff_origin`, load a `cold-boot-fixture` snapshot into SITL, start SUT cold, push first nav-camera frame, assert the first outbound estimate's lat/lon falls within ±50 m of the FC EKF snapshot. The two paths share a single test module parameterised on `(origin_source ∈ {operator_manifest, fc_ekf})`. The test also exercises the bounded-delta gate (Principle #11 amended): set Manifest origin to A and SITL FC EKF to a position B with `|A B| > 200 m`; assert the operator origin wins and the FC GPS is logged as suspect.
**Complexity**: 3 points
**Dependencies**: AZ-406, AZ-407 (cold-boot-fixture), AZ-323 / AZ-325 (Manifest with takeoff_origin), AZ-490 (set_takeoff_origin), AZ-489 (FlightsApiClient — used by the test fixture builder to fabricate Manifests with a known origin)
**Component**: Blackbox Tests / Positive / Startup (epic AZ-262)
**Tracker**: AZ-419
**Epic**: AZ-262 (E-BBT)
## Problem
Cold-start initialization is a critical path. The original assumption that the FC EKF's last valid GPS fix is always available at takeoff (AC-5.1) does not hold under realistic EW conditions — a UAV can be jammed at the launch site before takeoff, leaving the FC EKF with no valid GPS. ADR-010 introduces the operator-planned mission as the **primary** cold-start trust anchor: the operator authors the route in the Mission Planner UI, C12 fetches the `Flight`, derives `takeoff_origin` from `waypoints[0]`, and bakes it into the C10 Manifest. The airborne C5's `set_takeoff_origin` (AZ-490) consumes it before any sensor sample. The FC EKF GPS becomes the **secondary** path used only when the Manifest carries no origin (back-compat). Both paths must be measured end-to-end, and the bounded-delta gate (the third clause of the spoof-promotion gate, Principle #11) must be exercised so an inconsistent FC GPS at takeoff does not silently override the operator origin.
## Outcome
- pytest scenario at `e2e/tests/positive/test_ft_p_11_cold_start_init.py`.
- Parameterised on `origin_source ∈ {operator_manifest, fc_ekf, bounded_delta_conflict}`:
- **`operator_manifest`** (primary path, AZ-490): the test fixture builder writes a Manifest with `flight.takeoff_origin = A` (a known `LatLonAlt`); SITL starts with NO valid GPS (`GPS_TYPE = 0` or simulated denial); SUT cold-starts; the test asserts the first outbound estimate's lat/lon is within ±50 m of `A`.
- **`fc_ekf`** (secondary path, legacy AC-5.1): Manifest has no `flight.takeoff_origin`; `cold-boot-fixture` JSON pose loaded into SITL (parameter-load path); SUT cold-starts; the test asserts the first outbound estimate's lat/lon is within ±50 m of the FC-EKF snapshot pose.
- **`bounded_delta_conflict`** (ADR-010 Principle #11 amended): Manifest carries `takeoff_origin = A`; SITL FC EKF reports `B` with `vincenty(A, B) > 200 m`; the test asserts the first outbound estimate falls within ±50 m of `A` (operator origin wins), the source label on the first estimate is NOT `SATELLITE_ANCHORED` (no immediate spoof-promotion), and the FDR carries a `c5.gps_bounded_delta.reject` record naming both A and B.
- Starts SUT (cold — no prior FDR, no in-memory state). Pushes a single first nav-camera frame. Reads the first outbound estimate; computes Vincenty distance to the expected origin.
## Scope
### Included
- Cold-boot SITL parameter load (secondary path).
- Test fixture builder that produces a C10 Manifest with a known `flight.takeoff_origin` (primary path); reuses AZ-323's canonical JSON serialization.
- SUT cold start (`docker compose up gps-denied-onboard` from clean state; OR `systemctl start` on Tier-2).
- First-frame push and first-emission read.
- Distance comparison.
- FDR record assertions for the bounded-delta conflict scenario.
### Excluded
- Cold-start TTFF latency — owned by NFT-PERF-03 (AZ-430).
- Companion mid-flight reboot — owned by NFT-RES-02 (AZ-433).
- Mid-flight bounded-delta gate (only the takeoff slice is covered here; mid-flight is part of AZ-385 follow-up).
## Acceptance Criteria
**AC-1: Primary path (operator origin) — SUT cold-starts even when FC EKF has no GPS (ADR-010)**
Given a C10 Manifest with `flight.takeoff_origin = LatLonAlt(50.0, 36.2, 200.0)` AND SITL configured with no valid GPS
When SUT cold-starts and the first nav-camera frame is pushed
Then the SUT emits its first outbound message within ≤30 s; `vincenty(estimate.position, manifest.takeoff_origin) ≤ 50 m`; the FDR carries a `c5.cold_start_origin.set` record with `source = "manifest"`
**AC-2: Secondary path (FC EKF) — Manifest has no origin (back-compat)**
Given a C10 Manifest with no `flight.takeoff_origin` AND `cold-boot-fixture` JSON loaded into SITL
When SUT cold-starts
Then the first outbound estimate is within ±50 m of the FC EKF snapshot; the FDR carries a `c5.cold_start_origin.set` record with `source = "fc_ekf"`
**AC-3: No origin available — SUT refuses takeoff**
Given a C10 Manifest with no `flight.takeoff_origin` AND SITL with no valid GPS
When SUT cold-starts
Then NO outbound `EmittedExternalPosition` is produced within the AC-NEW-1 30 s budget; the SUT logs `c5.cold_start_origin.unavailable` to FDR + GCS STATUSTEXT; the test asserts the FT-P-11 takeoff-abort policy fires
**AC-4: Bounded-delta conflict — operator origin wins (ADR-010 Principle #11 amended)**
Given Manifest `takeoff_origin = A` AND SITL FC EKF reports `B` with `vincenty(A, B) > 200 m`
When SUT cold-starts and the first nav-camera frame is pushed
Then the first outbound estimate is within ±50 m of `A`; the source label is NOT `SATELLITE_ANCHORED` (no immediate spoof-promotion); the FDR carries a `c5.gps_bounded_delta.reject` record naming both A and B and the computed distance
**AC-5: parameterization across FC adapters + VIO strategies**
Given conftest parameterization on `(fc_adapter, vio_strategy, origin_source)`
Then each combination listed in the test matrix runs the appropriate ACs (AC-1 / AC-2 / AC-3 / AC-4 per `origin_source`)
## System Under Test Boundary
End-to-end through public boundaries.
- **Allowed**: SITL parameter load (a public SITL feature), SITL state read, outbound message read.
- **Forbidden**: pre-seeding SUT internal C5 state to bypass cold-start.
## Constraints
- The cold-boot-fixture JSON is loaded via the SITL's standard parameter-load path (`param load <file>` in mavproxy or equivalent for iNav); it is NOT loaded into the SUT.
- "Cold" means no prior FDR, no in-memory state — `fdr-output` volume is freshly created before this test.
## Document Dependencies
- `_docs/02_document/tests/blackbox-tests.md` § FT-P-11
- `_docs/02_document/tests/test-data.md` § FC contract & startup (FT-P-11 row)