[AZ-595] Batch 75: sitl_observer FDR-replay + scenario probe cleanup

Implement all 11 `sitl_observer` public surfaces as an offline
FDR-replay strategy (reads JSON fixtures under `${E2E_SITL_REPLAY_DIR}`
instead of live pymavlink/yamspy). Replace 12 per-scenario
`_harness_helpers_implemented` probes with one shared session-scoped
`sitl_replay_ready` fixture in `e2e/tests/conftest.py`.

Net: -636 LoC of duplicated scenario gating, +17 LoC shared fixture,
+38 new unit tests (596 total, up from 558). Includes K=3 cumulative
review for batches 73-75 (PASS).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-17 09:00:55 +03:00
parent 1d260f7e41
commit 43fdef1aac
23 changed files with 1485 additions and 639 deletions
+120
View File
@@ -0,0 +1,120 @@
# Batch 75 Report — sitl_observer FDR-replay + probe cleanup (cycle 1, batch 9 of test phase)
**Batch**: 75
**Date**: 2026-05-17
**Context**: Test implementation (greenfield Step 10 — Implement Tests)
**Tasks**: AZ-595 (3 cp) — 1 task (sitl_observer FDR-replay strategy + scenario probe cleanup)
**Cycle**: 1
**Verdict**: COMPLETE — PASS (self-reviewed; see `reviews/batch_75_review.md`,
K=3 cumulative `reviews/cumulative_73_75_review.md`)
## Summary
Closes the second half of the harness-stubs planning gap surfaced in
batch 74. Where batch 74 landed the three lowest-risk core helpers
(`fdr_reader`, `frame_source_replay`, `imu_replay`), batch 75 fills
the largest remaining stub: every `sitl_observer` surface that the
batch-71/72/73 scenarios reference. Strategy: **offline FDR-replay**,
not live pymavlink/yamspy plumbing. Each observer surface reads a
deterministic JSON fixture under `${E2E_SITL_REPLAY_DIR}` instead of
connecting to a real SITL container.
The same batch also collapses the per-scenario
`_harness_helpers_implemented` probe pattern (12 copies across
scenario files) into one shared session-scoped `sitl_replay_ready`
fixture in `e2e/tests/conftest.py`. Net: -636 LoC of duplicated
scenario gating, +17 LoC of shared fixture.
### AZ-595 — sitl_observer FDR-replay + probe cleanup (3 cp)
* **`runner/helpers/sitl_observer.py`** — 11 surfaces implemented:
* `replay_dir()` / `replay_dir_available()` — env-var-rooted
fixture resolver; the single reader of `E2E_SITL_REPLAY_DIR`.
* `get_observer(fc_kind, host)` — frozen-dataclass
`_FdrReplayObserver` reading `gps_state.json` once; exposes
`read_gps_state()` + `read_parameter(name)`.
* `read_ekf_divergence_events()`, `read_gps_health_samples()`,
`read_consistency_check_events()``_load_optional_json_list`
pattern (fixture absent → `[]`; malformed → `ValueError`).
* `capture_ap_tlog(host, duration_s)` — returns a `Path` from the
fixture; tlog binary is staged by the fixture builder.
* `read_ap_parameter(host, name)` — loads `ap_parameters.json`.
* `observe_inav_tcp_handshake(host, port, timeout_s)` — returns a
`TcpHandshakeReport` from `inav_tcp_handshake.json`.
* `collect_inav_msp_frames(host, port, window_s)` — returns a
`MspFrameCapture` (`frames: list[MspFrameSample]` +
`expected_num_sat`) from `inav_msp_frames.json`.
* `query_inav_gps_state(host)` — returns an `InavGpsState` from
`inav_gps_state.json`.
* `prepare_sitl_cold_boot(host, fixture_path)` /
`prepare_sitl_no_gps(host)` — no-ops in replay mode (the
fixture builder bakes the prepared state into the JSONs); the
`prepare_sitl_cold_boot` body still raises `RuntimeError` on
`fixture_path=None` so callers can't accidentally pass empty.
* **`_load_optional_json_list` + `_load_required_json`** — the two
fixture-loader helpers. Any present-but-malformed JSON still
raises `ValueError` with the file path; only genuinely missing
optional fixtures fall back to `[]`.
* **Public dataclasses added** (no consumer required edits — all
field names match what the batch-72/73 evaluators already
reference): `FcGpsState`, `EkfDivergenceEvent`, `GpsHealthSample`,
`ConsistencyCheckEvent`, `TcpHandshakeReport`, `MspFrameSample`,
`MspFrameCapture`, `InavGpsState`.
* **`e2e/tests/conftest.py`** — added the session-scoped
`sitl_replay_ready: bool` fixture (returns
`sitl_observer.replay_dir_available()`).
* **Scenarios refactored** — 12 scenarios stripped of their local
`_harness_helpers_implemented` fixture (+ `_NullSink` /
`_NullImuEmitter` helper classes) and rewired to consume
`sitl_replay_ready`:
* Positive: FT-P-01, FT-P-02, FT-P-03/14, FT-P-04, FT-P-05,
FT-P-07, FT-P-08, FT-P-09-AP, FT-P-09-iNav, FT-P-10, FT-P-11.
* Negative: FT-N-01, FT-N-02, FT-N-03, FT-N-04.
* **Stale docstrings updated** — FT-P-01, FT-P-02, FT-P-04 module
docstrings used to claim "skip is keyed off `NotImplementedError`
from the helper imports". They now point at the
`sitl_replay_ready` fixture and the `E2E_SITL_REPLAY_DIR` env
var. The FT-P-02 docstring also no longer claims that
`imu_replay` raises `NotImplementedError` (batch 74 landed it).
## Out of scope (deferred)
* **Live SITL parameter loading** — `prepare_sitl_cold_boot` /
`prepare_sitl_no_gps` only no-op in replay mode. A future
live-mode observer ticket will own the pymavlink param-set path
for hardware-in-the-loop runs.
* **`fc_proxy_runtime` driver** — FT-N-04 still depends on a
runtime fc-proxy driver to inject spoofed GPS. The blackout-spoof
scenario therefore continues to skip via `sitl_replay_ready`
AND a future fc-proxy-runtime gate.
* **Fixture builder** — the JSON fixtures themselves
(`gps_state.json`, `ekf_divergence_events.json`, …) are produced
by a SITL runner that does not yet exist. Until it lands, every
scenario keeps skipping cleanly via `sitl_replay_ready` — the
unit tests cover all branches today by writing tmp_path JSONs.
## Test Results
* New unit tests: **38** (sitl_observer end-to-end — replay_dir
resolution, every `read_*` / `capture_*` / `observe_*` /
`collect_*` / `query_*` parse path, `get_observer` factory,
`prepare_sitl_*` no-op semantics, error branches for every
optional + required loader).
* Full `e2e/_unit_tests` suite: **596 passed in 123 s** (previous
cumulative: 558 → +38 net).
* No new linter errors (`ReadLints` clean on `sitl_observer.py`,
`test_sitl_observer.py`, `conftest.py`, and all 12 refactored
scenario files).
* The pre-existing `/e2e-results/evidence` collection-time
teardown warning persists when scenarios are collected outside
docker; not caused by this batch.
## State
* Spec moved: `_docs/02_tasks/todo/AZ-595_sitl_observer_fdr_replay.md`
`_docs/02_tasks/done/`.
* `_docs/_autodev_state.md` advanced to `last_completed_batch: 75`.
* K=3 cumulative review for batches 73-75 written at
`_docs/03_implementation/reviews/cumulative_73_75_review.md`
(Verdict: PASS). `last_cumulative_review` advances to
`batches_73-75`.