[AZ-399] [AZ-400] C8 TlogReplayFcAdapter + ReplaySink + JsonlReplaySink

Opens E-DEMO-REPLAY (AZ-265): the two C8 strategies that let the
upcoming compose_replay (AZ-401) and gps-denied-replay CLI (AZ-402)
run the production C1-C5 pipeline against a recorded (.tlog, video)
pair without touching live FC I/O.

AZ-400 lands the contract ReplaySink Protocol (emit + close per
replay_protocol.md v1.0.0) and JsonlReplaySink: orjson-serialised
JSONL, fsync-on-close, build-flag gated (BUILD_REPLAY_SINK_JSONL),
double-close idempotent, FDR mirror on open/close. The drifted
AZ-390 stub in interface.py is removed; the canonical Protocol now
lives in replay_sink.py per module-layout.md and is re-exported via
__init__.py. AZ-390 conformance test widened.

AZ-399 lands TlogReplayFcAdapter: full FcAdapter Protocol surface,
build-flag gated (BUILD_TLOG_REPLAY_ADAPTER), pymavlink stream-parse
with bounded pre-scan + fail-fast on missing required messages
(R-DEMO-3), dedicated decode thread feeding the existing AZ-391
SubscriptionBus. Outbound surface raises FcEmitError per Invariant 5;
request_source_set_switch raises SourceSetSwitchNotSupportedError.
Pacing honours Invariant 6 via Clock.sleep_until_ns. time_offset_ms
shifts every emitted received_at per Invariant 8. Non-monotonic
timestamps raise FcOpenError.

Test coverage: 188 c8_fc_adapter tests pass; 1 skipped (AZ-399 AC-1
500 MB tlog RSS bound, deferred to AZ-404 e2e behind RUN_REPLAY_E2E).
Code review: PASS_WITH_WARNINGS — 1 Medium (mapping logic duplicates
AZ-391 live decoder; intentional today, four behavioural deltas
documented), 2 Low.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-14 05:33:20 +03:00
parent 4eac24f37a
commit fa3742d582
13 changed files with 2688 additions and 23 deletions
@@ -0,0 +1,83 @@
# Batch 59 — Cycle 1 Report
**Date**: 2026-05-14
**Tasks**: AZ-399 (C8 `TlogReplayFcAdapter`), AZ-400 (C8 `ReplaySink` Protocol + `JsonlReplaySink`)
**Verdict**: COMPLETE — PASS_WITH_WARNINGS
## Summary
Opened the E-DEMO-REPLAY epic (AZ-265) by landing the two C8 strategies that let the upcoming `compose_replay` (AZ-401) and `gps-denied-replay` CLI (AZ-402) consume a recorded `(.tlog, video)` pair without touching live FC I/O.
`JsonlReplaySink` (AZ-400) implements the contract `ReplaySink` Protocol from `_docs/02_document/contracts/replay/replay_protocol.md` v1.0.0: `emit(EstimatorOutput)` writes exactly one orjson-serialised JSONL line, `close()` `fsync`s, idempotent close fires `replay.sink.double_close` DEBUG. The on-wire shape is computed by an explicit `_to_jsonable` helper instead of `dataclasses.asdict``asdict` cannot meet AC-4 (numpy 6×6 → flat 36-float list, not a nested 2-D shape) or AC-5 (enum → name string, not value or repr) within the orjson default options.
The AZ-390 `interface.py` previously carried a `ReplaySink` stub with a single-method `write(...)` shape that had drifted from the contract. `interface.py` now owns only `FcAdapter` + `GcsAdapter` per the corrected `module-layout.md` line; the canonical Protocol lives in `replay_sink.py` and is re-exported through `__init__.py`. The AZ-390 conformance test was widened to assert both `emit` + `close` and to reject partial implementations.
`TlogReplayFcAdapter` (AZ-399) is the replay-mode `FcAdapter` strategy. Construction validates `BUILD_TLOG_REPLAY_ADAPTER` and the dialect (`ARDUPILOT_PLANE` or `INAV` only — `GCS_QGC` is rejected). `open(...)` runs a bounded pre-scan (`_PRESCAN_MAX_MESSAGES = 6000`) that asserts every required message group (RAW_IMU OR SCALED_IMU2; ATTITUDE; GPS_RAW_INT OR GPS2_RAW; HEARTBEAT) is represented, then starts a dedicated decode thread that streams the rest of the tlog through pymavlink's `recv_match(blocking=False)` — never materialising the file (R-DEMO-2). Frames are wrapped in `FcTelemetryFrame(received_at=msg._timestamp_ns + time_offset_ns, signed=False)` and dispatched via the existing AZ-391 `SubscriptionBus` so live and replay consumers see identical fan-out shape (Invariant 1).
Outbound surface is hard-failed per Invariant 5 (`emit_external_position` and `emit_status_text` raise `FcEmitError("replay adapter does not emit to FC")`); `request_source_set_switch` raises `SourceSetSwitchNotSupportedError`. Pacing honours Invariant 6: `pace=REALTIME` calls `Clock.sleep_until_ns(received_at)` between frames, `pace=ASAP` skips the call. Non-monotonic timestamps inside the dispatched stream raise `FcOpenError` (mirrors the AZ-398 TlogDerivedClock policy).
The decode-side path duplicates the AP mapping logic from `_inbound_mavlink.PymavlinkInboundDecoder` deliberately — replay differs in four observable ways (timestamp source, signed flag, non-monotonic policy, no STATUSTEXT spoof promotion) and a shared mapper would have to expose seams for all four. Captured as F1 Medium/Maintainability in the batch review.
## Files added / modified
### Added (4)
- `src/gps_denied_onboard/components/c8_fc_adapter/replay_sink.py``ReplaySink` Protocol, `JsonlReplaySink`, `_to_jsonable` helper, module-level `create(...)` factory, `_BUILD_FLAG = "BUILD_REPLAY_SINK_JSONL"` gating, `replay.sink.opened/closed/emit_progress/double_close` log + FDR mirror.
- `src/gps_denied_onboard/components/c8_fc_adapter/tlog_replay_adapter.py``TlogReplayFcAdapter`, `ReplayPace` enum, `REQUIRED_MESSAGE_TYPES`, fail-fast pre-scan, dedicated decode thread, build-flag gating, FDR mirror on open + missing-messages.
- `tests/unit/c8_fc_adapter/test_az400_replay_sink.py` — 21 tests covering AC-1..AC-10 plus schema fidelity, write-side OS error, JSON validity, double-close DEBUG, factory entrypoint.
- `tests/unit/c8_fc_adapter/test_az399_tlog_replay_adapter.py` — 22 tests covering AC-2..AC-10 (AC-1 deferred to AZ-404 e2e via `@pytest.mark.skip` with prerequisite reason), constructor validation, double-open guard, INAV dialect parity, multi-subscriber fan-out, current_flight_state init/update, warm-start hint cache, non-monotonic guard, INFO log + FDR mirror, required-message catalog sanity.
### Modified (4)
- `src/gps_denied_onboard/components/c8_fc_adapter/__init__.py` — re-exports `ReplaySink` from `replay_sink.py` and updates `__all__`.
- `src/gps_denied_onboard/components/c8_fc_adapter/interface.py` — removed the drifted `ReplaySink` stub; module docstring clarifies that the Protocol now lives in `replay_sink.py`.
- `tests/unit/c8_fc_adapter/test_az390_adapter_protocol.py``test_ac1_replay_sink_protocol_conformance` widened to the contract shape; new `test_ac1_replay_sink_rejects_partial_surface` guards against future stub drift.
- `_docs/02_document/module-layout.md``c8_fc_adapter` Public API line corrected: `ReplaySink` lives in `replay_sink.py`, not `interface.py`.
## Task Results
| Task | Status | Files Modified | Focused tests | AC Coverage | Issues |
|--------|--------|------------------------------------------|---------------|--------------------------------------------|--------|
| AZ-400 | Done | 1 added (`replay_sink.py`) / 3 modified | 21/21 pass | 10/10 covered | None |
| AZ-399 | Done | 1 added (`tlog_replay_adapter.py`) / 0 modified | 22/22 pass + 1 skipped (AC-1 with prerequisite) | 10/10 covered (AC-1 skipped per skill rule) | None |
## AC Test Coverage: 20/20 covered (1 AC skipped with prerequisite reason; counts as Covered per skill rule)
- AZ-400 AC-1..AC-10 — all directly asserted.
- AZ-399 AC-2..AC-10 — all directly asserted (AC-4 + AC-8 + AC-10 also have ancillary edge-case tests).
- AZ-399 AC-1 (500 MB tlog RSS bound) — `@pytest.mark.skip` with explicit prerequisite reason; deferred to AZ-404 e2e gated behind `RUN_REPLAY_E2E=1`.
## Code Review Verdict: PASS_WITH_WARNINGS
See `_docs/03_implementation/reviews/batch_59_review.md`. Three findings recorded — Medium ×1, Low ×2 — none blocking:
1. **F1 Medium / Maintainability**`_handle_imu/_attitude/_gps/_heartbeat` + `_map_fix_type` + `_map_mav_state` duplicate the AZ-391 live decoder. Intentional today (four behavioural deltas — timestamp source, signed flag, non-monotonic policy, STATUSTEXT absence). Revisit during AZ-405 or a future refactor if a third caller appears.
2. **F2 Low / Maintainability**`_PRESCAN_MAX_MESSAGES = 6000` is module-level. Future-proofing note; thread through the constructor when a real fixture pushes the budget.
3. **F3 Low / Maintainability**`# noqa: SIM115` on the unbuffered `open(...)` carries a justification comment; acceptable.
No Critical / High / Architecture findings. Auto-fix not required.
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Tests Run
- Focused suite (`tests/unit/c8_fc_adapter/`): **188 passed, 1 skipped** (the AZ-399 AC-1 prerequisite gate).
- Full repo suite: deferred to Step 16 (Final Test Run) per the implement skill's "exactly once at end of implementation phase" cadence.
## Next Batch
The replay track is half-wired:
-`Clock` Protocol (AZ-398, batch 57)
-`FrameSource` + `VideoFileFrameSource` (AZ-398, batch 57)
-`TlogReplayFcAdapter` (this batch)
-`ReplaySink` + `JsonlReplaySink` (this batch)
-`compose_replay(config) -> ReplayRoot` (AZ-401)
-`gps-denied-replay` CLI (AZ-402)
-`gps-denied-replay-cli` Dockerfile + CI matrix + SBOM diff (AZ-403)
- ⏳ E2E replay fixture test (AZ-404)
- ⏳ Auto-sync IMU take-off detection (AZ-405)
Next eligible batch (dependencies satisfied): AZ-401 + AZ-389 (C5 orthorectifier, independent track) — to be selected by the next `/autodev` batch loop.