mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 09:41:13 +00:00
[AZ-595] Batch 75: sitl_observer FDR-replay + scenario probe cleanup
Implement all 11 `sitl_observer` public surfaces as an offline
FDR-replay strategy (reads JSON fixtures under `${E2E_SITL_REPLAY_DIR}`
instead of live pymavlink/yamspy). Replace 12 per-scenario
`_harness_helpers_implemented` probes with one shared session-scoped
`sitl_replay_ready` fixture in `e2e/tests/conftest.py`.
Net: -636 LoC of duplicated scenario gating, +17 LoC shared fixture,
+38 new unit tests (596 total, up from 558). Includes K=3 cumulative
review for batches 73-75 (PASS).
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,120 @@
|
||||
# Batch 75 Report — sitl_observer FDR-replay + probe cleanup (cycle 1, batch 9 of test phase)
|
||||
|
||||
**Batch**: 75
|
||||
**Date**: 2026-05-17
|
||||
**Context**: Test implementation (greenfield Step 10 — Implement Tests)
|
||||
**Tasks**: AZ-595 (3 cp) — 1 task (sitl_observer FDR-replay strategy + scenario probe cleanup)
|
||||
**Cycle**: 1
|
||||
**Verdict**: COMPLETE — PASS (self-reviewed; see `reviews/batch_75_review.md`,
|
||||
K=3 cumulative `reviews/cumulative_73_75_review.md`)
|
||||
|
||||
## Summary
|
||||
|
||||
Closes the second half of the harness-stubs planning gap surfaced in
|
||||
batch 74. Where batch 74 landed the three lowest-risk core helpers
|
||||
(`fdr_reader`, `frame_source_replay`, `imu_replay`), batch 75 fills
|
||||
the largest remaining stub: every `sitl_observer` surface that the
|
||||
batch-71/72/73 scenarios reference. Strategy: **offline FDR-replay**,
|
||||
not live pymavlink/yamspy plumbing. Each observer surface reads a
|
||||
deterministic JSON fixture under `${E2E_SITL_REPLAY_DIR}` instead of
|
||||
connecting to a real SITL container.
|
||||
|
||||
The same batch also collapses the per-scenario
|
||||
`_harness_helpers_implemented` probe pattern (12 copies across
|
||||
scenario files) into one shared session-scoped `sitl_replay_ready`
|
||||
fixture in `e2e/tests/conftest.py`. Net: -636 LoC of duplicated
|
||||
scenario gating, +17 LoC of shared fixture.
|
||||
|
||||
### AZ-595 — sitl_observer FDR-replay + probe cleanup (3 cp)
|
||||
|
||||
* **`runner/helpers/sitl_observer.py`** — 11 surfaces implemented:
|
||||
* `replay_dir()` / `replay_dir_available()` — env-var-rooted
|
||||
fixture resolver; the single reader of `E2E_SITL_REPLAY_DIR`.
|
||||
* `get_observer(fc_kind, host)` — frozen-dataclass
|
||||
`_FdrReplayObserver` reading `gps_state.json` once; exposes
|
||||
`read_gps_state()` + `read_parameter(name)`.
|
||||
* `read_ekf_divergence_events()`, `read_gps_health_samples()`,
|
||||
`read_consistency_check_events()` — `_load_optional_json_list`
|
||||
pattern (fixture absent → `[]`; malformed → `ValueError`).
|
||||
* `capture_ap_tlog(host, duration_s)` — returns a `Path` from the
|
||||
fixture; tlog binary is staged by the fixture builder.
|
||||
* `read_ap_parameter(host, name)` — loads `ap_parameters.json`.
|
||||
* `observe_inav_tcp_handshake(host, port, timeout_s)` — returns a
|
||||
`TcpHandshakeReport` from `inav_tcp_handshake.json`.
|
||||
* `collect_inav_msp_frames(host, port, window_s)` — returns a
|
||||
`MspFrameCapture` (`frames: list[MspFrameSample]` +
|
||||
`expected_num_sat`) from `inav_msp_frames.json`.
|
||||
* `query_inav_gps_state(host)` — returns an `InavGpsState` from
|
||||
`inav_gps_state.json`.
|
||||
* `prepare_sitl_cold_boot(host, fixture_path)` /
|
||||
`prepare_sitl_no_gps(host)` — no-ops in replay mode (the
|
||||
fixture builder bakes the prepared state into the JSONs); the
|
||||
`prepare_sitl_cold_boot` body still raises `RuntimeError` on
|
||||
`fixture_path=None` so callers can't accidentally pass empty.
|
||||
* **`_load_optional_json_list` + `_load_required_json`** — the two
|
||||
fixture-loader helpers. Any present-but-malformed JSON still
|
||||
raises `ValueError` with the file path; only genuinely missing
|
||||
optional fixtures fall back to `[]`.
|
||||
* **Public dataclasses added** (no consumer required edits — all
|
||||
field names match what the batch-72/73 evaluators already
|
||||
reference): `FcGpsState`, `EkfDivergenceEvent`, `GpsHealthSample`,
|
||||
`ConsistencyCheckEvent`, `TcpHandshakeReport`, `MspFrameSample`,
|
||||
`MspFrameCapture`, `InavGpsState`.
|
||||
* **`e2e/tests/conftest.py`** — added the session-scoped
|
||||
`sitl_replay_ready: bool` fixture (returns
|
||||
`sitl_observer.replay_dir_available()`).
|
||||
* **Scenarios refactored** — 12 scenarios stripped of their local
|
||||
`_harness_helpers_implemented` fixture (+ `_NullSink` /
|
||||
`_NullImuEmitter` helper classes) and rewired to consume
|
||||
`sitl_replay_ready`:
|
||||
* Positive: FT-P-01, FT-P-02, FT-P-03/14, FT-P-04, FT-P-05,
|
||||
FT-P-07, FT-P-08, FT-P-09-AP, FT-P-09-iNav, FT-P-10, FT-P-11.
|
||||
* Negative: FT-N-01, FT-N-02, FT-N-03, FT-N-04.
|
||||
* **Stale docstrings updated** — FT-P-01, FT-P-02, FT-P-04 module
|
||||
docstrings used to claim "skip is keyed off `NotImplementedError`
|
||||
from the helper imports". They now point at the
|
||||
`sitl_replay_ready` fixture and the `E2E_SITL_REPLAY_DIR` env
|
||||
var. The FT-P-02 docstring also no longer claims that
|
||||
`imu_replay` raises `NotImplementedError` (batch 74 landed it).
|
||||
|
||||
## Out of scope (deferred)
|
||||
|
||||
* **Live SITL parameter loading** — `prepare_sitl_cold_boot` /
|
||||
`prepare_sitl_no_gps` only no-op in replay mode. A future
|
||||
live-mode observer ticket will own the pymavlink param-set path
|
||||
for hardware-in-the-loop runs.
|
||||
* **`fc_proxy_runtime` driver** — FT-N-04 still depends on a
|
||||
runtime fc-proxy driver to inject spoofed GPS. The blackout-spoof
|
||||
scenario therefore continues to skip via `sitl_replay_ready`
|
||||
AND a future fc-proxy-runtime gate.
|
||||
* **Fixture builder** — the JSON fixtures themselves
|
||||
(`gps_state.json`, `ekf_divergence_events.json`, …) are produced
|
||||
by a SITL runner that does not yet exist. Until it lands, every
|
||||
scenario keeps skipping cleanly via `sitl_replay_ready` — the
|
||||
unit tests cover all branches today by writing tmp_path JSONs.
|
||||
|
||||
## Test Results
|
||||
|
||||
* New unit tests: **38** (sitl_observer end-to-end — replay_dir
|
||||
resolution, every `read_*` / `capture_*` / `observe_*` /
|
||||
`collect_*` / `query_*` parse path, `get_observer` factory,
|
||||
`prepare_sitl_*` no-op semantics, error branches for every
|
||||
optional + required loader).
|
||||
* Full `e2e/_unit_tests` suite: **596 passed in 123 s** (previous
|
||||
cumulative: 558 → +38 net).
|
||||
* No new linter errors (`ReadLints` clean on `sitl_observer.py`,
|
||||
`test_sitl_observer.py`, `conftest.py`, and all 12 refactored
|
||||
scenario files).
|
||||
* The pre-existing `/e2e-results/evidence` collection-time
|
||||
teardown warning persists when scenarios are collected outside
|
||||
docker; not caused by this batch.
|
||||
|
||||
## State
|
||||
|
||||
* Spec moved: `_docs/02_tasks/todo/AZ-595_sitl_observer_fdr_replay.md`
|
||||
→ `_docs/02_tasks/done/`.
|
||||
* `_docs/_autodev_state.md` advanced to `last_completed_batch: 75`.
|
||||
* K=3 cumulative review for batches 73-75 written at
|
||||
`_docs/03_implementation/reviews/cumulative_73_75_review.md`
|
||||
(Verdict: PASS). `last_cumulative_review` advances to
|
||||
`batches_73-75`.
|
||||
Reference in New Issue
Block a user