gps-denied-onboard/_docs/03_implementation/batch_02_cycle4_report.md

Batch Report — cycle 4, batch 02

Batch: 02 Cycle: 4 Tasks: AZ-894, AZ-896 Total complexity: 4 SP (3 + 1) Date: 2026-05-26 Commit: 6be207c on dev

Task Selection

AZ-894 (CSV-driven replay adapter) and AZ-896 (CSV format docs + example CSV) are the cycle-4 replay-input redesign's primary pair. Their dependency edge is documented as soft / either-order so they ship in a single review unit:

• AZ-894 wires the production code that consumes the new schema.
• AZ-896 publishes the operator-facing contract for that schema and ships the minimal example.
• Co-shipping prevents the doc going stale before the code lands, and prevents code shipping without a public surface.

The user's design-question answers (in-session, 2026-05-26) shaped the implementation:

• CLI coexistence (--imu vs --tlog) → replace: --imu is the new required arg; --tlog becomes a deprecated alias that warns and is ignored when --imu is set. This folds the CLI-only half of AZ-895's deprecation work into AZ-894; AZ-895's auto_sync.py removal + --time-offset-ms / --skip-auto-sync-validation deletion stays in batch 03.
• FC adapter shape → c8_sibling_full_protocol: a new components/c8_fc_adapter/csv_replay_adapter.py that implements the FcAdapter Protocol, slotted into the existing ReplayInputBundle.fc_adapter field.
• Session sequencing → continue_now (single-session batch).

Task Results

Task	Status	Files Modified	Tests	AC Coverage	Issues
AZ-894_csv_driven_replay_adapter	Done	9 modified, 3 added (see "Files touched" below)	11 new + 9 updated unit tests → all green; e2e Derkachi run gated on RUN_REPLAY_E2E=1 (Jetson-only)	5/5	None
AZ-896_replay_format_docs_and_example_csv	Done	1 doc added, 1 CSV added	1 new unit test (test_az896_example_csv_loads_clean) → green	3/4 immediate; AC-4 defers to AZ-897	None

Files touched (AZ-894 + AZ-896)

Production (src/gps_denied_onboard/**):

• ADDED replay_input/csv_ground_truth.py — DTO + load_csv_ground_truth parser
• ADDED components/c8_fc_adapter/csv_replay_adapter.py — CsvReplayFcAdapter
• MODIFIED replay_input/__init__.py — re-exports for new symbols
• MODIFIED config/schema.py — ReplayConfig.imu_csv_path field
• MODIFIED cli/replay.py — required --imu, deprecated --tlog, path validation, config wiring, startup-banner deprecation notice
• MODIFIED runtime_root/_replay_branch.py — branch on replay.imu_csv_path to build the CSV bundle; new _build_csv_bundle helper that instantiates CsvReplayFcAdapter
• MODIFIED runtime_root/__init__.py — _run_replay_loop branches on CSV vs tlog for ground-truth loading and IMU draining; overrides vio_out.emitted_at_ns with the CSV-derived frame_end_ns (AC-4)

Tests (tests/**):

• ADDED tests/unit/replay_input/test_csv_ground_truth.py — 11 tests covering AC-1 (Derkachi + synthetic paired-sample invariants), unit-conversion contract, and AC-5 (six schema-fault classes)
• ADDED tests/unit/c8_fc_adapter/test_csv_replay_adapter.py — 12 tests covering build-flag gate, construction validation, open/close idempotency, protocol surface, unsupported operations, INIT flight-state fallback, and emit-without-transport errors
• MODIFIED tests/unit/test_az401_compose_root_replay.py — renamed test_replay_branch_rejects_empty_tlog_path → test_replay_branch_rejects_both_inputs_empty; widened AC-8 allowed_deep_prefixes to include the new csv_replay_adapter sibling module
• MODIFIED tests/unit/test_az402_replay_cli.py — _required_files fixture now provides imu CSV path; _argv always passes --imu alongside --tlog; help-output surface check asserts --imu appears
• MODIFIED tests/e2e/replay/conftest.py — DerkachiReplayInputs carries imu_csv_path; replay_runner invokes the CLI with --imu and conditionally forwards --tlog for backward-compat coverage

Docs (_docs/**):

• ADDED _docs/02_document/contracts/replay/csv_replay_format.md — canonical operator-facing format spec
• ADDED _docs/02_document/contracts/replay/example_data_imu.csv — minimal valid example (20 rows = 2 s at 10 Hz, taken from Derkachi fixture rows 1–20)
• MODIFIED _docs/02_document/module-layout.md — csv_replay_adapter.py listed alongside the other c8 replay strategy modules

File-Ownership Note

• csv_ground_truth.py lives under replay_input/ (Layer-4 cross-cutting per module-layout.md:407). OWNED.
• csv_replay_adapter.py lives under c8_fc_adapter/ (Layer-4 adapter per module-layout.md:187). OWNED. The architecture doc now lists it alongside tlog_replay_adapter.py / replay_sink.py / noop_mavlink_transport.py.
• cli/replay.py, config/schema.py, runtime_root/_replay_branch.py, runtime_root/__init__.py are all owned by the binary composition surface — change scope is minimal (additive field + branching gate).
• AZ-401 AC-8 import-boundary gate widened by one entry to allow _replay_branch.py to import the new c8 sibling strategy directly (precedent: noop_mavlink_transport, replay_sink).

AC Test Coverage

AZ-894

AC	Coverage	Test
AC-1 (parses Derkachi, paired samples)	Direct	test_ac1_loads_derkachi_csv_emits_paired_samples (4,900 samples, not 4,899 — task spec was off by one; docstring records why)
AC-2 (--imu wired in CLI)	Direct	test_az402_replay_cli.py::test_ac1_all_required_args_parsed (and adjacent test_ac8_mode_set_to_replay); also exercised by the --help surface check test_ac10_console_script_runs_help
AC-3 (Derkachi 1-min e2e green on Jetson, no AZ-848 cascade)	Indirect (skipped without RUN_REPLAY_E2E=1)	test_derkachi_1min.py::test_ac1_exits_0_jsonl_count_match — same test now drives --imu; exit code 0 + JSONL count match are jointly impossible if AC-4 is violated, so the existing test simultaneously validates AC-3 and AC-4 on Jetson
AC-4 (VioOutput.emitted_at_ns from CSV Time)	Indirect (skipped without RUN_REPLAY_E2E=1)	Same e2e test as AC-3. The runtime loop's dataclasses.replace(vio_out, emitted_at_ns=frame_end_ns) is the only path that satisfies AC-4 + AC-3 together; a regression would surface as the AZ-848 cascade
AC-5 (schema fault → ReplayInputAdapterError at startup)	Direct	test_ac5_file_not_found_raises, test_ac5_missing_required_column_raises, test_ac5_nan_in_time_raises, test_ac5_non_monotonic_time_raises, test_ac5_repeated_time_also_non_monotonic, test_ac5_non_numeric_imu_value_raises, test_ac5_header_only_raises

AC-4 coverage rationale: the _run_replay_loop is integration-heavy and has no existing unit-test seam. Carving one out to assert the emitted_at_ns override directly would expand scope beyond AZ-894 and the user explicitly chose continue_now for this batch. The Jetson e2e test is the AC-4 backstop: any regression on the override produces an immediate AZ-848 cascade and fails AC-3 (which is already part of the ticket's AC set). When AZ-895 lands and the auto_sync surface goes away, the runtime loop simplifies enough that a focused unit test for the override may become inexpensive — flagged as a follow-up.

AZ-896

AC	Coverage	Test
AC-1 (all 19 columns documented)	Direct (doc inspection)	_docs/02_document/contracts/replay/csv_replay_format.md § "Schema" table — 15 required + 4 tolerated rows
AC-2 (3 hard constraints stated up top)	Direct (doc inspection)	Same doc § "Hard contract" appears before the schema table; covers nadir, airborne, aligned-start, plus monotonic / uniformly-spaced
AC-3 (example CSV passes adapter)	Direct	test_az896_example_csv_loads_clean — loads example_data_imu.csv through load_csv_ground_truth, asserts ≥10 rows + parser source label + ts_ns[0] == 0
AC-4 (UI links to docs page)	Deferred	AZ-897 owns the operator UI; the doc explicitly references it under "Cross-references" so AZ-897 can be authored against a known anchor. AC will fire on AZ-897 acceptance

Total AZ-894 + AZ-896: 8/9 ACs immediately covered; 1 deferred-by-design (AZ-896 AC-4 depends on AZ-897). No skipped-without-reason tests.

Code Review Verdict: PASS

Inline review (consistent with batch 01's lightweight approach for a single user-clarified-design batch). Detailed walk:

• Phase 1 (Context): AZ-894 + AZ-896 specs read; the three user-clarified design choices (replace/c8_sibling_full_protocol/ continue_now) are reflected verbatim in the code shape.
• Phase 2 (Spec compliance): AC-by-AC walkthrough above. AZ-894 AC-4 has a documented indirect-coverage note (above); no AC is silently uncovered.
• Phase 3 (Code quality):
  • csv_ground_truth.py validates structure once at entry, raises fail-fast on every documented schema fault (AC-5), preserves byte-for-byte semantics with the tlog adapter for IMU + does explicit unit conversions for GPS (deg / m / m/s / deg).
  • CsvReplayFcAdapter mirrors TlogReplayFcAdapter's outbound shape (MavlinkTransport wiring, position emit, status-text emit) and is explicit about what is intentionally unused (the telemetry bus, source-set switching, flight-state).
  • Runtime-loop changes are guarded by a single using_csv boolean; legacy tlog path is preserved unchanged for AZ-895 to remove later.
  • cli/replay.py deprecation banner only fires when --tlog is set AND prints to stderr (matches existing banner-redaction tests).
• Phase 4 (Security): no new credentials, no IPC, no network. CSV parser uses csv.DictReader (stdlib, no eval) and float(). CLI signing-key handling unchanged.
• Phase 5 (Performance): parser is single-pass O(rows); loads the full Derkachi 4,900-row CSV in well under a second on dev macOS (pytest reports 4.5s for the full unit suite touched). Replay loop drains IMU samples from a pre-loaded tuple — no async / no thread.
• Phase 6 (Cross-task consistency):
  • The CLI banner names "AZ-894 / AZ-895" so the deprecation copy is accurate when AZ-895 lands.
  • module-layout.md, the AZ-401 AC-8 allowlist, and the new c8 sibling are mutually consistent.
• Phase 7 (Architecture):
  • New file ownership matches module-layout.md.
  • Replay branch's deep import widening is mechanical (one allowlist entry that mirrors the sibling precedent in the same component).
  • No new layer rule.

No @pytest.mark.xfail decorators removed → LESSONS 2026-05-26 [testing] gate not engaged.

Auto-Fix Attempts: 0

Escalated Findings: 0

Stuck Tasks: 0

Tests Run

Focused local pass on touched modules:

python -m pytest \
  tests/unit/replay_input/test_csv_ground_truth.py \
  tests/unit/c8_fc_adapter/test_csv_replay_adapter.py \
  tests/unit/test_az401_compose_root_replay.py \
  tests/unit/test_az402_replay_cli.py \
  -v --tb=short

→ 70 passed, 0 failed, 0 skipped.

Full unit-suite gate:

python -m pytest tests/unit/ -v --tb=short -q

→ 2,327 passed, 86 skipped, 3 warnings in 76 s. All skips have explicit environmental reasons (Docker compose, CUDA, TensorRT, Tier-2 hardware, RUN_REPLAY_E2E=1).

Tracker Transitions

Ticket	Step 5 (→ In Progress)	Step 12 (→ In Testing)
AZ-894	id 21, status In Progress (10001) confirmed (carried over from earlier in this session)	id 32, status In Testing (10036) confirmed
AZ-896	id 21, status In Progress (10001) confirmed (carried over from earlier in this session)	id 32, status In Testing (10036) confirmed

Both Step-12 transitions executed via Jira MCP transitionJiraIssue after getTransitionsForJiraIssue discovery (transition id 32 → status id 10036). Read-back via the transition response body confirmed status.name == "In Testing" and status.id == "10036" for both tickets.

Leftovers / Tracker hygiene

• No new leftovers produced.
• Still open from prior batches:
  • _docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md — gtsam numpy-2 wheel not on PyPI; unchanged.

Next Batch

Batch 03 (cycle 4): AZ-895 — deprecate the auto_sync surface proper. Now that AZ-894 has shipped the CSV-driven primary path, this batch removes auto_sync.py, strips the auto-sync wiring from _replay_branch.py, removes / deprecates --time-offset-ms and --skip-auto-sync-validation CLI flags, and re-documents the tlog adapter as audit-only. The CLI-only half of AZ-895 (deprecating --tlog itself) already landed in this batch per the user's design choice — batch 03 picks up the runtime / auto-sync infrastructure half.