mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 05:51:13 +00:00
[AZ-399] [AZ-400] C8 TlogReplayFcAdapter + ReplaySink + JsonlReplaySink
Opens E-DEMO-REPLAY (AZ-265): the two C8 strategies that let the upcoming compose_replay (AZ-401) and gps-denied-replay CLI (AZ-402) run the production C1-C5 pipeline against a recorded (.tlog, video) pair without touching live FC I/O. AZ-400 lands the contract ReplaySink Protocol (emit + close per replay_protocol.md v1.0.0) and JsonlReplaySink: orjson-serialised JSONL, fsync-on-close, build-flag gated (BUILD_REPLAY_SINK_JSONL), double-close idempotent, FDR mirror on open/close. The drifted AZ-390 stub in interface.py is removed; the canonical Protocol now lives in replay_sink.py per module-layout.md and is re-exported via __init__.py. AZ-390 conformance test widened. AZ-399 lands TlogReplayFcAdapter: full FcAdapter Protocol surface, build-flag gated (BUILD_TLOG_REPLAY_ADAPTER), pymavlink stream-parse with bounded pre-scan + fail-fast on missing required messages (R-DEMO-3), dedicated decode thread feeding the existing AZ-391 SubscriptionBus. Outbound surface raises FcEmitError per Invariant 5; request_source_set_switch raises SourceSetSwitchNotSupportedError. Pacing honours Invariant 6 via Clock.sleep_until_ns. time_offset_ms shifts every emitted received_at per Invariant 8. Non-monotonic timestamps raise FcOpenError. Test coverage: 188 c8_fc_adapter tests pass; 1 skipped (AZ-399 AC-1 500 MB tlog RSS bound, deferred to AZ-404 e2e behind RUN_REPLAY_E2E). Code review: PASS_WITH_WARNINGS — 1 Medium (mapping logic duplicates AZ-391 live decoder; intentional today, four behavioural deltas documented), 2 Low. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,83 @@
|
||||
# Batch 59 — Cycle 1 Report
|
||||
|
||||
**Date**: 2026-05-14
|
||||
**Tasks**: AZ-399 (C8 `TlogReplayFcAdapter`), AZ-400 (C8 `ReplaySink` Protocol + `JsonlReplaySink`)
|
||||
**Verdict**: COMPLETE — PASS_WITH_WARNINGS
|
||||
|
||||
## Summary
|
||||
|
||||
Opened the E-DEMO-REPLAY epic (AZ-265) by landing the two C8 strategies that let the upcoming `compose_replay` (AZ-401) and `gps-denied-replay` CLI (AZ-402) consume a recorded `(.tlog, video)` pair without touching live FC I/O.
|
||||
|
||||
`JsonlReplaySink` (AZ-400) implements the contract `ReplaySink` Protocol from `_docs/02_document/contracts/replay/replay_protocol.md` v1.0.0: `emit(EstimatorOutput)` writes exactly one orjson-serialised JSONL line, `close()` `fsync`s, idempotent close fires `replay.sink.double_close` DEBUG. The on-wire shape is computed by an explicit `_to_jsonable` helper instead of `dataclasses.asdict` — `asdict` cannot meet AC-4 (numpy 6×6 → flat 36-float list, not a nested 2-D shape) or AC-5 (enum → name string, not value or repr) within the orjson default options.
|
||||
|
||||
The AZ-390 `interface.py` previously carried a `ReplaySink` stub with a single-method `write(...)` shape that had drifted from the contract. `interface.py` now owns only `FcAdapter` + `GcsAdapter` per the corrected `module-layout.md` line; the canonical Protocol lives in `replay_sink.py` and is re-exported through `__init__.py`. The AZ-390 conformance test was widened to assert both `emit` + `close` and to reject partial implementations.
|
||||
|
||||
`TlogReplayFcAdapter` (AZ-399) is the replay-mode `FcAdapter` strategy. Construction validates `BUILD_TLOG_REPLAY_ADAPTER` and the dialect (`ARDUPILOT_PLANE` or `INAV` only — `GCS_QGC` is rejected). `open(...)` runs a bounded pre-scan (`_PRESCAN_MAX_MESSAGES = 6000`) that asserts every required message group (RAW_IMU OR SCALED_IMU2; ATTITUDE; GPS_RAW_INT OR GPS2_RAW; HEARTBEAT) is represented, then starts a dedicated decode thread that streams the rest of the tlog through pymavlink's `recv_match(blocking=False)` — never materialising the file (R-DEMO-2). Frames are wrapped in `FcTelemetryFrame(received_at=msg._timestamp_ns + time_offset_ns, signed=False)` and dispatched via the existing AZ-391 `SubscriptionBus` so live and replay consumers see identical fan-out shape (Invariant 1).
|
||||
|
||||
Outbound surface is hard-failed per Invariant 5 (`emit_external_position` and `emit_status_text` raise `FcEmitError("replay adapter does not emit to FC")`); `request_source_set_switch` raises `SourceSetSwitchNotSupportedError`. Pacing honours Invariant 6: `pace=REALTIME` calls `Clock.sleep_until_ns(received_at)` between frames, `pace=ASAP` skips the call. Non-monotonic timestamps inside the dispatched stream raise `FcOpenError` (mirrors the AZ-398 TlogDerivedClock policy).
|
||||
|
||||
The decode-side path duplicates the AP mapping logic from `_inbound_mavlink.PymavlinkInboundDecoder` deliberately — replay differs in four observable ways (timestamp source, signed flag, non-monotonic policy, no STATUSTEXT spoof promotion) and a shared mapper would have to expose seams for all four. Captured as F1 Medium/Maintainability in the batch review.
|
||||
|
||||
## Files added / modified
|
||||
|
||||
### Added (4)
|
||||
|
||||
- `src/gps_denied_onboard/components/c8_fc_adapter/replay_sink.py` — `ReplaySink` Protocol, `JsonlReplaySink`, `_to_jsonable` helper, module-level `create(...)` factory, `_BUILD_FLAG = "BUILD_REPLAY_SINK_JSONL"` gating, `replay.sink.opened/closed/emit_progress/double_close` log + FDR mirror.
|
||||
- `src/gps_denied_onboard/components/c8_fc_adapter/tlog_replay_adapter.py` — `TlogReplayFcAdapter`, `ReplayPace` enum, `REQUIRED_MESSAGE_TYPES`, fail-fast pre-scan, dedicated decode thread, build-flag gating, FDR mirror on open + missing-messages.
|
||||
- `tests/unit/c8_fc_adapter/test_az400_replay_sink.py` — 21 tests covering AC-1..AC-10 plus schema fidelity, write-side OS error, JSON validity, double-close DEBUG, factory entrypoint.
|
||||
- `tests/unit/c8_fc_adapter/test_az399_tlog_replay_adapter.py` — 22 tests covering AC-2..AC-10 (AC-1 deferred to AZ-404 e2e via `@pytest.mark.skip` with prerequisite reason), constructor validation, double-open guard, INAV dialect parity, multi-subscriber fan-out, current_flight_state init/update, warm-start hint cache, non-monotonic guard, INFO log + FDR mirror, required-message catalog sanity.
|
||||
|
||||
### Modified (4)
|
||||
|
||||
- `src/gps_denied_onboard/components/c8_fc_adapter/__init__.py` — re-exports `ReplaySink` from `replay_sink.py` and updates `__all__`.
|
||||
- `src/gps_denied_onboard/components/c8_fc_adapter/interface.py` — removed the drifted `ReplaySink` stub; module docstring clarifies that the Protocol now lives in `replay_sink.py`.
|
||||
- `tests/unit/c8_fc_adapter/test_az390_adapter_protocol.py` — `test_ac1_replay_sink_protocol_conformance` widened to the contract shape; new `test_ac1_replay_sink_rejects_partial_surface` guards against future stub drift.
|
||||
- `_docs/02_document/module-layout.md` — `c8_fc_adapter` Public API line corrected: `ReplaySink` lives in `replay_sink.py`, not `interface.py`.
|
||||
|
||||
## Task Results
|
||||
|
||||
| Task | Status | Files Modified | Focused tests | AC Coverage | Issues |
|
||||
|--------|--------|------------------------------------------|---------------|--------------------------------------------|--------|
|
||||
| AZ-400 | Done | 1 added (`replay_sink.py`) / 3 modified | 21/21 pass | 10/10 covered | None |
|
||||
| AZ-399 | Done | 1 added (`tlog_replay_adapter.py`) / 0 modified | 22/22 pass + 1 skipped (AC-1 with prerequisite) | 10/10 covered (AC-1 skipped per skill rule) | None |
|
||||
|
||||
## AC Test Coverage: 20/20 covered (1 AC skipped with prerequisite reason; counts as Covered per skill rule)
|
||||
|
||||
- AZ-400 AC-1..AC-10 — all directly asserted.
|
||||
- AZ-399 AC-2..AC-10 — all directly asserted (AC-4 + AC-8 + AC-10 also have ancillary edge-case tests).
|
||||
- AZ-399 AC-1 (500 MB tlog RSS bound) — `@pytest.mark.skip` with explicit prerequisite reason; deferred to AZ-404 e2e gated behind `RUN_REPLAY_E2E=1`.
|
||||
|
||||
## Code Review Verdict: PASS_WITH_WARNINGS
|
||||
|
||||
See `_docs/03_implementation/reviews/batch_59_review.md`. Three findings recorded — Medium ×1, Low ×2 — none blocking:
|
||||
|
||||
1. **F1 Medium / Maintainability** — `_handle_imu/_attitude/_gps/_heartbeat` + `_map_fix_type` + `_map_mav_state` duplicate the AZ-391 live decoder. Intentional today (four behavioural deltas — timestamp source, signed flag, non-monotonic policy, STATUSTEXT absence). Revisit during AZ-405 or a future refactor if a third caller appears.
|
||||
2. **F2 Low / Maintainability** — `_PRESCAN_MAX_MESSAGES = 6000` is module-level. Future-proofing note; thread through the constructor when a real fixture pushes the budget.
|
||||
3. **F3 Low / Maintainability** — `# noqa: SIM115` on the unbuffered `open(...)` carries a justification comment; acceptable.
|
||||
|
||||
No Critical / High / Architecture findings. Auto-fix not required.
|
||||
|
||||
## Auto-Fix Attempts: 0
|
||||
|
||||
## Stuck Agents: None
|
||||
|
||||
## Tests Run
|
||||
|
||||
- Focused suite (`tests/unit/c8_fc_adapter/`): **188 passed, 1 skipped** (the AZ-399 AC-1 prerequisite gate).
|
||||
- Full repo suite: deferred to Step 16 (Final Test Run) per the implement skill's "exactly once at end of implementation phase" cadence.
|
||||
|
||||
## Next Batch
|
||||
|
||||
The replay track is half-wired:
|
||||
|
||||
- ✅ `Clock` Protocol (AZ-398, batch 57)
|
||||
- ✅ `FrameSource` + `VideoFileFrameSource` (AZ-398, batch 57)
|
||||
- ✅ `TlogReplayFcAdapter` (this batch)
|
||||
- ✅ `ReplaySink` + `JsonlReplaySink` (this batch)
|
||||
- ⏳ `compose_replay(config) -> ReplayRoot` (AZ-401)
|
||||
- ⏳ `gps-denied-replay` CLI (AZ-402)
|
||||
- ⏳ `gps-denied-replay-cli` Dockerfile + CI matrix + SBOM diff (AZ-403)
|
||||
- ⏳ E2E replay fixture test (AZ-404)
|
||||
- ⏳ Auto-sync IMU take-off detection (AZ-405)
|
||||
|
||||
Next eligible batch (dependencies satisfied): AZ-401 + AZ-389 (C5 orthorectifier, independent track) — to be selected by the next `/autodev` batch loop.
|
||||
Reference in New Issue
Block a user