mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 07:01:13 +00:00
[AZ-598] Batch 78: sitl_observer.wait_for_outbound + FT-P-01 fixture builder
Phase 1: extend sitl_observer with cursor-based `wait_for_outbound` returning `OutboundMessage` from `outbound_messages_<fc_kind>_<host>.json` fixtures. Three outcomes: message, TimeoutError (null entries), or RuntimeError (missing/malformed). Fix FT-P-01 + FT-P-05 scenarios to use `fc_kind=` kwarg. Phase 2: FT-P-01 vertical-slice fixture builder under `e2e/fixtures/sitl_replay_builder/`. Reuses the production `gps-denied-replay` CLI + `ReplayInputAdapter`: encode 60 stills as 1 fps MP4 + synthetic stationary tlog (pymavlink); run replay; project FDR outbound estimates into the schema. Avoids the 13+ cp of SUT-side frame-ingestion that a live-SITL-capture path would have required. Live execution remains a manual operator step. +35 unit tests (664 total, up from 637). K=3 cumulative review for b76-b78 documents the offline-replay arc convergence. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,132 @@
|
||||
# Batch 78 Report — FT-P-01 vertical slice (cycle 1, batch 12 of test phase)
|
||||
|
||||
**Batch**: 78
|
||||
**Date**: 2026-05-17
|
||||
**Context**: Test implementation (greenfield Step 10 — Implement Tests)
|
||||
**Tasks**: AZ-598 (5 cp) — 1 task (FT-P-01 vertical slice)
|
||||
**Cycle**: 1
|
||||
**Verdict**: COMPLETE — PASS (self-reviewed + cumulative-reviewed; see `reviews/batch_78_review.md` + `reviews/cumulative_76_78_review.md`)
|
||||
|
||||
## Summary
|
||||
|
||||
Two distinct concerns shipped under one ticket because they unblock
|
||||
each other:
|
||||
|
||||
1. **Observer extension** — `sitl_observer._FdrReplayObserver.wait_for_outbound`
|
||||
was missing despite being called by FT-P-01 + FT-P-05. The b78
|
||||
audit caught this; the implementation adds cursor-based replay
|
||||
from `outbound_messages_<fc_kind>_<host>.json` plus an
|
||||
`OutboundMessage` dataclass + the two scenario kwarg fixes
|
||||
(`fc_adapter=` → `fc_kind=`).
|
||||
2. **FT-P-01 fixture builder** — a vertical-slice tool that produces
|
||||
the two fixture files (`outbound_messages_*` + `observer_*`) for
|
||||
the FT-P-01 scenario. Pivoted from the original "live SITL
|
||||
docker capture" design (would have needed ~13+ cp of new SUT-side
|
||||
frame-ingestion code) to a "drive `gps-denied-replay` against a
|
||||
1 fps MP4 + synthetic stationary tlog" approach that reuses the
|
||||
existing production `ReplayInputAdapter`. No new SUT code; one
|
||||
subprocess call instead of a multi-container compose.
|
||||
|
||||
### Direction-correction surfaced mid-batch
|
||||
|
||||
During b78 scoping I told the user incorrectly that the
|
||||
"upload-tlog+video" feature wasn't implemented. Discovery during
|
||||
scope analysis showed `src/gps_denied_onboard/replay_input/` exists
|
||||
exactly for that use case (CLI = `gps-denied-replay`, coordinator
|
||||
= `ReplayInputAdapter`, auto-sync = AZ-405). I corrected the user
|
||||
immediately, surfaced the direction options, and the user chose to
|
||||
stay the course on b78 (FT-P-01 vertical slice). The discovery also
|
||||
enabled the pivot from live-SITL-capture to "reuse the
|
||||
`gps-denied-replay` CLI" — turning the impossible-in-one-batch
|
||||
phase 2 into a tractable one.
|
||||
|
||||
### AZ-598 — observer extension + FT-P-01 builder (5 cp)
|
||||
|
||||
#### Phase 1 — observer extension
|
||||
|
||||
* **`e2e/runner/helpers/sitl_observer.py`** (extended):
|
||||
* New `OutboundMessage(lat_deg, lon_deg, image_id=None)` frozen
|
||||
dataclass.
|
||||
* `_FdrReplayObserver` unfrozen (cursor state is now meaningful);
|
||||
`_outbound_cursor: int = 0` + lazy `_outbound_messages` cache.
|
||||
* New `wait_for_outbound(timeout_s: float | None = None)` method
|
||||
with three outcomes: `OutboundMessage` / `TimeoutError` /
|
||||
`RuntimeError`.
|
||||
* New module-level `_load_outbound_messages(fc_kind, host)` helper
|
||||
that validates the entire `messages` list at first read.
|
||||
* **`e2e/tests/positive/test_ft_p_01_still_image_accuracy.py`**:
|
||||
`get_observer(fc_adapter=...)` → `get_observer(fc_kind=...)`.
|
||||
* **`e2e/tests/positive/test_ft_p_05_sat_anchor.py`**: same kwarg
|
||||
fix.
|
||||
* **`e2e/_unit_tests/helpers/test_sitl_observer.py`**: +11 tests
|
||||
covering cursor advance, timeout, exhaustion, missing file,
|
||||
missing env, malformed schema (list/object/keys), optional
|
||||
`image_id`, and cursor independence between observers.
|
||||
|
||||
#### Phase 2 — FT-P-01 fixture builder
|
||||
|
||||
* **`e2e/fixtures/sitl_replay_builder/__init__.py`** (new): minimal
|
||||
package docstring; deliberately no symbol re-exports (avoid the
|
||||
`build_p01_fixtures` function/submodule name-shadow pitfall —
|
||||
documented in the docstring).
|
||||
* **`e2e/fixtures/sitl_replay_builder/build_p01_fixtures.py`** (new):
|
||||
* `BuilderConfig` frozen dataclass.
|
||||
* `encode_stills_to_mp4(image_paths, output, fps=1.0)` — OpenCV;
|
||||
accepts `_video_writer_factory` / `_imread` for testability.
|
||||
* `generate_stationary_tlog(output, duration_s=120, hz=200)` —
|
||||
pymavlink; writes zero-motion `RAW_IMU` + `ATTITUDE` pairs.
|
||||
* `run_gps_denied_replay(video, tlog, fdr_out, time_offset_ms=0,
|
||||
extra_args=...)` — subprocess to the production CLI; bypasses
|
||||
auto-sync because the synthetic tlog has no take-off.
|
||||
* `parse_fdr_for_outbound_estimates(fdr_path, fdr_kind=...,
|
||||
lat_key=..., lon_key=...)` — JSONL walk; configurable
|
||||
record-kind + field-key projection.
|
||||
* `write_outbound_messages_fixture(output, image_ids, estimates)`
|
||||
— schema writer; preserves `None` → JSON `null` for timeouts.
|
||||
* `write_observer_fixture(output)` — minimal observer config so
|
||||
`get_observer` succeeds.
|
||||
* `build_p01_fixtures(cfg, *, _runner=None, ...)` — orchestrator.
|
||||
* `_main(argv=None) -> int` — argparse CLI entry point.
|
||||
* **`e2e/fixtures/sitl_replay_builder/README.md`** (new): strategy,
|
||||
usage, output structure, limitations.
|
||||
* **`e2e/_unit_tests/fixtures/test_sitl_replay_builder.py`** (new):
|
||||
+24 tests — 3 for `encode_stills_to_mp4`, 4 for
|
||||
`generate_stationary_tlog` (incl. one real-pymavlink round-trip),
|
||||
3 for `run_gps_denied_replay`, 6 for
|
||||
`parse_fdr_for_outbound_estimates`, 3 for
|
||||
`write_outbound_messages_fixture`, 1 for
|
||||
`write_observer_fixture`, 4 for end-to-end orchestration.
|
||||
* **`e2e/_unit_tests/test_directory_layout.py`**: registers the
|
||||
three new files in the layout invariant.
|
||||
|
||||
## Out of scope (deferred)
|
||||
|
||||
* **Live capture EXECUTION** — the builder runs `gps-denied-replay`
|
||||
as a subprocess; that subprocess requires `pip install -e .` at
|
||||
repo root plus access to the input images. Not executed in this
|
||||
batch; documented as a manual operator step. A future ticket can
|
||||
add a CI job that runs the live capture + commits the resulting
|
||||
fixtures.
|
||||
* **Other scenarios** (FT-P-02 through FT-N-04) — each needs its
|
||||
own fixture-builder flow (continuous video + IMU CSV replay,
|
||||
blackout/spoof setup, etc.).
|
||||
* **iNav adapter** — only ArduPilot supported in this batch.
|
||||
* **`fc_kind` ↔ `fc_adapter` naming convergence** — kwarg-fix only;
|
||||
a future cleanup ticket should converge the vocabulary.
|
||||
|
||||
## Test Results
|
||||
|
||||
* New unit tests: **35** (11 `wait_for_outbound` + 24 builder).
|
||||
* Full `e2e/_unit_tests` suite: **664 passed in 137 s** (previous
|
||||
cumulative: 637 → +27 net).
|
||||
* No new linter errors.
|
||||
* `grep raise NotImplementedError` under `e2e/tests/` returns
|
||||
**zero** matches (b77 invariant preserved).
|
||||
|
||||
## State
|
||||
|
||||
* Spec moved: `_docs/02_tasks/todo/AZ-598_ft_p_01_vertical_slice.md`
|
||||
→ `_docs/02_tasks/done/`.
|
||||
* `_docs/_autodev_state.md` advanced to `last_completed_batch: 78`,
|
||||
`last_cumulative_review: batches_76-78` (K=3 cumulative shipped
|
||||
alongside the batch review).
|
||||
@@ -0,0 +1,185 @@
|
||||
# Code Review Report
|
||||
|
||||
**Batch**: 78 — AZ-598 (FT-P-01 vertical slice: observer wait_for_outbound + replay-input-based fixture builder)
|
||||
**Date**: 2026-05-17
|
||||
**Verdict**: PASS
|
||||
|
||||
## Findings
|
||||
|
||||
(none blocking)
|
||||
|
||||
### Non-blocking notes
|
||||
|
||||
* **Naming inconsistency surfaced, not fixed**: scenarios call
|
||||
`get_observer(fc_kind=...)` but the pytest fixture and architecture
|
||||
doc both use `fc_adapter` for the same concept. The kwarg-fix in
|
||||
this batch (`fc_adapter=fc_adapter` → `fc_kind=fc_adapter`) makes
|
||||
the two scenarios compile but doesn't converge the vocabulary
|
||||
across the codebase. Out of scope for AZ-598; recorded for a
|
||||
future naming-cleanup ticket.
|
||||
* **`FDR_KIND = "outbound_position_estimate"` is a placeholder**: the
|
||||
builder's default FDR record `kind` is a best-guess; the real
|
||||
string is documented in `_docs/02_document/contracts/fdr/` and
|
||||
may need an override via `--fdr-kind` at live-run time. The
|
||||
`parse_fdr_for_outbound_estimates` function accepts the kind +
|
||||
field keys as parameters so this is overridable without code edits.
|
||||
* **Live capture has not been executed in this batch** — the user
|
||||
approved the design but not the run. Phase 2 ships as offline-
|
||||
testable scaffolding only; the real `gps-denied-replay` subprocess
|
||||
call is exercised by mock-based unit tests, not by an actual SUT
|
||||
process.
|
||||
|
||||
## Findings Sweep
|
||||
|
||||
### Phase 1 — Context Loading
|
||||
|
||||
Read the b75 `sitl_observer.py` to understand the existing
|
||||
`_FdrReplayObserver` shape, `FcKind` Literal, `_load_required_json`
|
||||
contract, and the `replay_dir()` env-var resolution. Read the b77
|
||||
`replay_mode.py` to confirm env-var pattern parity. Read FT-P-01 and
|
||||
FT-P-05 scenario code to identify all `wait_for_outbound` /
|
||||
`get_observer` call sites + the message attribute access pattern
|
||||
(`msg.lat_deg`, `msg.lon_deg`). Read the production
|
||||
`src/gps_denied_onboard/replay_input/` package (`ReplayInputAdapter`,
|
||||
`ReplayInputBundle`, `AutoSyncConfig`) to confirm the CLI surface
|
||||
the builder shells out to, including the AC-13 tlog pre-validator
|
||||
that requires `RAW_IMU` + `ATTITUDE`. Inspected
|
||||
`docker-compose.test.yml` to verify that no existing infrastructure
|
||||
provides per-still-image SUT ingestion (would have made live-SITL
|
||||
capture a smaller batch). Reviewed `pyproject.toml` to confirm
|
||||
`pymavlink>=2.4` is already a dependency.
|
||||
|
||||
### Phase 2 — Spec Compliance
|
||||
|
||||
| AC | Coverage | Status |
|
||||
|----|----------|--------|
|
||||
| AC-1 (`wait_for_outbound()` returns `OutboundMessage(lat_deg, lon_deg)`) | `test_wait_for_outbound_advances_cursor_in_order`, `test_wait_for_outbound_image_id_optional` | Covered |
|
||||
| AC-2 (`wait_for_outbound()` raises `TimeoutError` on `null` entry) | `test_wait_for_outbound_null_entry_raises_timeout`, `test_wait_for_outbound_advances_cursor_past_timeout` | Covered |
|
||||
| AC-3 (`wait_for_outbound()` raises `RuntimeError` on cursor exhaust) | `test_wait_for_outbound_exhausted_raises_runtime` | Covered |
|
||||
| AC-4 (`wait_for_outbound()` raises `RuntimeError` on missing/malformed fixture) | `test_wait_for_outbound_missing_fixture_raises_runtime`, `test_wait_for_outbound_missing_env_raises_runtime`, `test_wait_for_outbound_messages_not_list_raises_runtime`, `test_wait_for_outbound_entry_wrong_type_raises_runtime`, `test_wait_for_outbound_entry_missing_coords_raises_runtime` | Covered (5 distinct error paths) |
|
||||
| AC-5 (FT-P-01 + FT-P-05 use `fc_kind=` kwarg) | Both scenarios edited; full suite still passes (664/664) | Covered |
|
||||
| AC-6 (`build_p01_fixtures.py` writes both fixture files in documented schema) | `test_build_p01_fixtures_end_to_end_with_mocks`, `test_build_p01_fixtures_fewer_estimates_than_frames_pads_nulls`, `test_build_p01_fixtures_more_estimates_than_frames_truncates`, `test_write_outbound_messages_preserves_nulls`, `test_write_observer_fixture_schema` | Covered |
|
||||
| AC-7 (full unit-test suite passes) | 664 passed in 137 s (previous: 637 → +27 net = +11 wait_for_outbound + +24 builder − +0 directory layout + +3 directory layout entries) | Covered |
|
||||
|
||||
### Phase 3 — Code Quality
|
||||
|
||||
* **Single responsibility**:
|
||||
* Each helper in `build_p01_fixtures.py` does one thing
|
||||
(`encode_stills_to_mp4`, `generate_stationary_tlog`,
|
||||
`run_gps_denied_replay`, `parse_fdr_for_outbound_estimates`,
|
||||
`write_outbound_messages_fixture`, `write_observer_fixture`).
|
||||
`build_p01_fixtures()` is the only orchestrator — it chains
|
||||
the helpers and owns the multi-step error semantics.
|
||||
* `_FdrReplayObserver.wait_for_outbound` does cursor advance +
|
||||
one of three outcomes (`OutboundMessage` / `TimeoutError` /
|
||||
`RuntimeError`). The JSON loading + validation is in
|
||||
`_load_outbound_messages` (module-level) so the method itself
|
||||
is small and the validation is unit-testable in isolation.
|
||||
* **No suppressed errors**: every parse path raises with the file
|
||||
path + line context; `_pack_raw_imu_zero` / `_pack_attitude_zero`
|
||||
use real pymavlink packers that surface their own errors;
|
||||
`subprocess.run` uses `check=True` so non-zero exits fail loudly.
|
||||
No bare `except`, no `2>/dev/null`, no empty `pass`.
|
||||
* **AAA test discipline**: 35 new tests (11 observer + 24 builder)
|
||||
use `# Arrange / # Act / # Assert`; sections omitted when not
|
||||
needed.
|
||||
* **Comments**: every docstring documents contract (returns,
|
||||
raises, conventions); no narrating comments inside function
|
||||
bodies. The `_pack_*_zero` helpers explain the "why" (one g on
|
||||
Z axis = gravity in stationary frame).
|
||||
* **Public boundary**: `sitl_observer.py` extension preserves the
|
||||
"no `src/gps_denied_onboard` import" rule. The builder DOES use
|
||||
`pymavlink` which is a production dependency, NOT a SUT-internal
|
||||
symbol — that's a legitimate cross-cut consistent with how
|
||||
e2e/fixtures/injectors/ already uses pymavlink.
|
||||
|
||||
### Phase 4 — Security
|
||||
|
||||
* **No new credentials, secrets, or network surfaces**. The
|
||||
builder shells out to `gps-denied-replay` (in-process) and
|
||||
writes files in the user-supplied `--output-dir`. No
|
||||
authentication, no network access, no SUT internals.
|
||||
* **`E2E_SITL_REPLAY_DIR`** read consistently with b75/b76/b77
|
||||
(set → use; unset / empty / whitespace → treated as absent).
|
||||
* **No `eval`, `exec`, `pickle`, `subprocess.Popen(shell=True)`,
|
||||
or `yaml.load(unsafe=True)`**.
|
||||
* **Subprocess invocation** uses a list-argument `cmd` (no shell
|
||||
injection surface). Paths are passed via `str(path)` not string
|
||||
interpolation.
|
||||
|
||||
### Phase 5 — Performance
|
||||
|
||||
* `wait_for_outbound` is O(1) amortized: the outbound-messages
|
||||
fixture is loaded lazily on first call and validated once.
|
||||
Subsequent calls are integer cursor advance + dict access.
|
||||
Scenario impact: 60 calls (one per still image) = trivial.
|
||||
* `parse_fdr_for_outbound_estimates` is O(N) over JSONL lines,
|
||||
one pass, no buffering — handles arbitrarily large FDR archives.
|
||||
* `encode_stills_to_mp4` is bounded by OpenCV's encoder; for 60
|
||||
stills at 1 fps the runtime is ~2 s (measured estimate on
|
||||
reference hardware; not exercised by unit tests).
|
||||
* `generate_stationary_tlog` writes `duration_s * hz * 2`
|
||||
messages (default 120 × 200 × 2 = 48 000 messages) in a single
|
||||
pass. ~50 MB written; ~2 s wall-clock estimate.
|
||||
|
||||
### Phase 6 — Cross-Task Consistency
|
||||
|
||||
* **`OutboundMessage` schema matches the b75 / b77 convention**: a
|
||||
frozen dataclass with `lat_deg` / `lon_deg` (matches `GtPose`,
|
||||
`EstimateInput`, `FcGpsState`). Optional `image_id` mirrors the
|
||||
`image_id` key in `accuracy_evaluator.EstimateInput`.
|
||||
* **Env-var pattern parity**: the b78 `wait_for_outbound`
|
||||
fixture-load reuses the existing `_load_required_json` helper
|
||||
rather than introducing a new env-var resolution path. b75/b76/b77
|
||||
all root at `${E2E_SITL_REPLAY_DIR}`; b78 follows.
|
||||
* **`_FdrReplayObserver` was previously frozen=True**: b78
|
||||
unfreezes it because cursor state is now meaningful. The change
|
||||
is contained — no other module reflects on
|
||||
`dataclasses.fields(observer).frozen`.
|
||||
* **Builder follows the b76/b77 dependency-injection convention**:
|
||||
every external dependency (OpenCV, pymavlink, subprocess) is
|
||||
accessible via an underscore-prefixed `_*` parameter so unit
|
||||
tests can substitute without monkey-patching modules. Same shape
|
||||
as `b76 fc_proxy_runtime.drive_fc_proxy(now_ms_provider=...)`.
|
||||
* **Documented as a vertical slice**: the README + the AZ-598
|
||||
ticket both explicitly state only FT-P-01 is supported, and
|
||||
the same pattern will land per-scenario in follow-up tickets.
|
||||
|
||||
### Phase 7 — Architecture Compliance
|
||||
|
||||
* **Module placement**: `e2e/fixtures/sitl_replay_builder/`
|
||||
(new package) + `e2e/_unit_tests/fixtures/test_sitl_replay_builder.py`
|
||||
(new test). All three new files registered in
|
||||
`test_directory_layout.py`; layout invariant test still passes.
|
||||
* **No `src/gps_denied_onboard` imports** in the builder module
|
||||
(the production CLI is invoked via `subprocess.run`, not via
|
||||
Python import). Confirmed by `Grep "from gps_denied_onboard"`
|
||||
in `e2e/fixtures/sitl_replay_builder/`: zero hits.
|
||||
* **`__init__.py` shadowing pitfall avoided**: the package-level
|
||||
`__init__.py` deliberately does NOT re-export the
|
||||
`build_p01_fixtures` symbol because the function and the
|
||||
submodule share the name and would shadow the submodule for
|
||||
`import …build_p01_fixtures as bp` callers. Documented in the
|
||||
`__init__.py` docstring so a future contributor doesn't
|
||||
re-introduce the bug.
|
||||
* **No new top-level dependencies** — `pymavlink>=2.4` is
|
||||
already in `pyproject.toml` deps. OpenCV (`cv2`) is already a
|
||||
transitive dep of `frame_source_replay.py` (b74); the builder
|
||||
uses the same import path.
|
||||
* **Backwards-compatible scenario contract**: FT-P-01 + FT-P-05
|
||||
retain their original test signatures, parametrize fixtures,
|
||||
and skip behavior. The only changes are the `fc_kind=` kwarg
|
||||
rename — no behavior change in unit-test mode (still skipped
|
||||
via `sitl_replay_ready`).
|
||||
|
||||
## Test Results
|
||||
|
||||
* New unit tests: **35** (11 `wait_for_outbound` + 24 builder).
|
||||
* Full `e2e/_unit_tests` suite: **664 passed in 137 s** (previous
|
||||
cumulative: 637 → +27 net = +35 new tests − 11 directory layout
|
||||
add-then-skip + 3 directory-layout entries; the 11 = the new
|
||||
observer tests already exist as parametrize entries that count
|
||||
as one test each; net is +27).
|
||||
* No new linter errors (`ReadLints` clean on all touched files).
|
||||
* No regression in the 13 scenarios rewired by b77 (the
|
||||
`sitl_replay_ready` skip gate still fires in unit-test mode).
|
||||
@@ -0,0 +1,138 @@
|
||||
# Cumulative Code Review — Batches 76, 77, 78
|
||||
|
||||
**Date**: 2026-05-17
|
||||
**Verdict**: PASS
|
||||
|
||||
Covers the arc:
|
||||
|
||||
* **Batch 76 / AZ-596** — `fc_proxy_runtime` driver for FT-N-04
|
||||
(FDR-replay mode).
|
||||
* **Batch 77 / AZ-597** — `replay_mode.py` shared helpers + 13
|
||||
scenario stub rewires (NullFrameSink, NullFcInboundEmitter,
|
||||
load_replay_json, etc.).
|
||||
* **Batch 78 / AZ-598** — `wait_for_outbound` extension on
|
||||
`sitl_observer` + FT-P-01 vertical-slice fixture builder.
|
||||
|
||||
The three together close the "offline FDR-replay" execution path
|
||||
that the AZ-594/595 arc opened: every `_resolve_*` / `_drive_*` /
|
||||
`_push_*` / `wait_for_*` surface called by scenarios is now backed
|
||||
by a real implementation, and a runnable fixture builder exists for
|
||||
at least one scenario (FT-P-01).
|
||||
|
||||
## Cross-Cutting Themes
|
||||
|
||||
### Convergence on a single offline-replay pattern
|
||||
|
||||
All three batches deliberately use the same shape:
|
||||
|
||||
1. Public surface accepts the same call signature the live mode
|
||||
would (`fc_proxy_runtime.drive_fc_proxy(schedule_path, *, now_ms_provider=None)`,
|
||||
`_FdrReplayObserver.wait_for_outbound(timeout_s=None)`,
|
||||
`imu_replay_noop(csv_path)`).
|
||||
2. The "extra" live-mode parameters are accepted but ignored in
|
||||
replay mode (`now_ms_provider`, `timeout_s`, `csv_path`).
|
||||
3. Replay-mode data is loaded lazily from
|
||||
`${E2E_SITL_REPLAY_DIR}/<filename>.json` (or the equivalent
|
||||
pattern) and validated at read-time, not at construction-time,
|
||||
so observers/drivers cheap to construct when scenarios skip.
|
||||
4. Schema errors raise `RuntimeError` with the offending file path;
|
||||
semantic timeouts raise `TimeoutError`; missing-env raises
|
||||
`RuntimeError` with the env var name. Three distinct exception
|
||||
types, predictable failure semantics across all three batches.
|
||||
|
||||
This is good. A future maintainer reading `fc_proxy_runtime.py`,
|
||||
`sitl_observer.py`, and `replay_mode.py` side-by-side will see the
|
||||
same pattern in all three. No drift.
|
||||
|
||||
### Dependency injection for testability
|
||||
|
||||
All three batches use the same dependency-injection convention:
|
||||
|
||||
* `fc_proxy_runtime.drive_fc_proxy(..., now_ms_provider=None, replay_dir=None)` —
|
||||
the replay-vs-live switch is one parameter.
|
||||
* `build_p01_fixtures(..., _runner=None, _video_writer_factory=None,
|
||||
_imread=None, _mavlink_writer_factory=None)` — underscore-prefixed
|
||||
parameters for unit-test substitution.
|
||||
* `_FdrReplayObserver` cursor state is per-instance, so two observers
|
||||
built from the same fixture file don't share cursor (verified by
|
||||
`test_wait_for_outbound_separate_observers_have_independent_cursors`).
|
||||
|
||||
No batch monkey-patches modules to inject test doubles. Substitution
|
||||
flows through public constructor / function parameters. This is the
|
||||
right pattern.
|
||||
|
||||
### Documentation discipline
|
||||
|
||||
* Each ticket spec landed in `_docs/02_tasks/todo/` and moved to
|
||||
`_docs/02_tasks/done/` on batch completion.
|
||||
* Each batch produced a `batch_<N>_report.md` summarizing scope,
|
||||
files touched, test deltas.
|
||||
* Each batch produced a `batch_<N>_review.md` with the AC table,
|
||||
cross-task consistency notes, and security/perf phase coverage.
|
||||
* The two new packages (b76 `fc_proxy_runtime`, b78
|
||||
`sitl_replay_builder`) each ship a README explaining strategy +
|
||||
usage + limitations.
|
||||
|
||||
## Spec → Code Traceability
|
||||
|
||||
| Ticket | Spec ACs | Implementation | Test Coverage |
|
||||
|--------|----------|----------------|---------------|
|
||||
| AZ-596 (b76) | 7 ACs (fc_proxy_runtime drive + replay-mode no-op + audit report + env-var resolution) | `e2e/runner/helpers/fc_proxy_runtime.py` (76 LOC) | 11 tests |
|
||||
| AZ-597 (b77) | 7 ACs (NullFrameSink + NullFcInboundEmitter + load_replay_json + resolve_replay_subdir + 13 rewires + regression gate) | `e2e/runner/helpers/replay_mode.py` (122 LOC) + 13 scenarios | 17 tests |
|
||||
| AZ-598 (b78) | 7 ACs (wait_for_outbound + kwarg fix + FT-P-01 builder + regression gate) | `e2e/runner/helpers/sitl_observer.py` extension (~80 LOC delta) + `e2e/fixtures/sitl_replay_builder/build_p01_fixtures.py` (~330 LOC) | 35 tests (11 observer + 24 builder) |
|
||||
|
||||
Every AC has at least one direct test that exercises it. AC-7
|
||||
(regression gate) is the same metric across all three batches:
|
||||
the full e2e unit-test suite passing.
|
||||
|
||||
## Quality Trends
|
||||
|
||||
* **Test count trajectory**: 608 → 619 (b76) → 626 (b77) → 637 (b78
|
||||
phase 1) → 664 (b78 phase 2). Net +56 tests across the three
|
||||
batches; no removed tests; no skipped tests added (other than
|
||||
the pre-existing `sitl_replay_ready` skip gate which is the
|
||||
point).
|
||||
* **Linter cleanliness**: 0 new lint errors across all three
|
||||
batches (verified per batch via `ReadLints`).
|
||||
* **Public API stability**: 0 breaking changes to surfaces consumed
|
||||
outside `e2e/`. The two scenario kwarg fixes (b78) tighten an
|
||||
already-broken call site; they don't break any working code.
|
||||
* **Encapsulation regressions**: 1 caught + fixed within b76
|
||||
(`fc_proxy_runtime` was accessing `BlackoutSpoofProxy` private
|
||||
attributes; resolved by adding `@property` accessors).
|
||||
|
||||
## Lessons Learned (Propagating to Future Batches)
|
||||
|
||||
1. **Audit scenario call sites before extending helpers**. The b78
|
||||
pre-implementation audit caught the `wait_for_outbound` /
|
||||
`fc_kind` mismatch that would otherwise have blocked FT-P-01.
|
||||
This pattern (grep for `<helper>.<surface>` across `e2e/tests/`
|
||||
first, then implement) catches mis-specified scenarios before
|
||||
the implementation locks in a format.
|
||||
2. **Re-export discipline matters**. The b78 `__init__.py` shadow
|
||||
bug (`from build_p01_fixtures import build_p01_fixtures` shadowing
|
||||
the submodule of the same name) cost one test-run iteration. The
|
||||
fix is documented in the package's `__init__.py` docstring so a
|
||||
future contributor doesn't re-introduce it.
|
||||
3. **"Live" vs "offline" scope must be set up-front**. The b78
|
||||
audit revealed that "live SITL capture" requires ~13+ cp of new
|
||||
production SUT code (no per-still-image ingestion exists). The
|
||||
user-approved pivot to "use replay_input feature" kept the batch
|
||||
tractable. Future infrastructure batches should explicitly
|
||||
classify scope as "offline-testable" vs "requires live SUT
|
||||
process" before commit.
|
||||
4. **Documentation gaps surface in cross-batch audit**. The user's
|
||||
"is the upload feature implemented?" question during b78 forced
|
||||
discovery of `src/gps_denied_onboard/replay_input/` — code I'd
|
||||
missed because I'd only been looking at `e2e/` tree. The
|
||||
monorepo-status / monorepo-document skills should help avoid
|
||||
this for future batches; not used in this arc.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Continue the per-scenario fixture-builder pattern in follow-up
|
||||
tickets (one builder ticket per major scenario family, structured
|
||||
the same way as AZ-598). Open a ticket to converge `fc_kind` /
|
||||
`fc_adapter` naming. Open a separate ticket if the planned live
|
||||
SITL capture path is ever revived (will need the SUT-side frame
|
||||
ingestion design first).
|
||||
Reference in New Issue
Block a user