Files
gps-denied-onboard/_docs/02_tasks/todo/AZ-399_replay_tlog_adapter.md
T
Oleksandr Bezdieniezhnykh 880eabcb3f Decompose Step 6 snapshot: 140 task specs + contract docs
Closes out greenfield Step 6 (Decompose) for all 14 components
(C1-C13 + cross-cutting helpers/replay). Covers tasks AZ-266..AZ-446
plus the _dependencies_table.md and component contract documents.

State file updated to greenfield Step 7 (Implement), not_started.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 00:39:48 +03:00

8.9 KiB
Raw Blame History

Replay — TlogReplayFcAdapter (pymavlink stream parser → inbound DTOs)

Task: AZ-399_replay_tlog_adapter Name: TlogReplayFcAdapter — replay-only FcAdapter strategy parsing pymavlink .tlog Description: Implement TlogReplayFcAdapter (gated BUILD_TLOG_REPLAY_ADAPTER) at src/gps_denied_onboard/components/c8_fc_adapter/tlog_replay_adapter.py. The class implements the full FcAdapter Protocol from AZ-390. STREAM-PARSE the pymavlink .tlog (R-DEMO-2; never materialise; multi-GB tlogs); map AP/iNav message types → FcTelemetryFrame (RAW_IMU/SCALED_IMU2 → IMU_SAMPLE; ATTITUDE → ATTITUDE; GPS_RAW_INT/GPS2_RAW → GPS_HEALTH; HEARTBEAT.system_status → MAV_STATE / FlightStateSignal). subscribe_telemetry is the primary surface — fan out to all subscribers at the configured pace: REALTIME → use Clock.sleep_until_ns(target_ns) between frames; ASAP → no-op pace. time_offset_ms shifts every emitted timestamp at construction (Invariant 8). target_fc_dialect chooses pymavlink dialect at parse time. Fail fast at startup (R-DEMO-3): if any required message type is absent (RAW_IMU + ATTITUDE + GPS_RAW_INT/GPS2_RAW + HEARTBEAT), raise FcOpenError("tlog missing required messages: <list>") with the components that need them. emit_external_position and emit_status_text raise FcEmitError("replay adapter does not emit to FC") (Invariant 5). request_source_set_switch raises SourceSetSwitchNotSupportedError. current_flight_state returns the latest FlightStateSignal from the parsed stream. WgsConverter (AZ-279) constructor-injected for tlog GPS → local-tangent-plane. Complexity: 5 points Dependencies: AZ-398 (Clock Protocol), AZ-390 (FcAdapter Protocol from E-C8); AZ-391 (DTO surface; FcTelemetryFrame); AZ-279 (WgsConverter); AZ-273 (FDR); AZ-263, AZ-269, AZ-266, AZ-272 Component: c8_fc_adapter (epic AZ-265 / E-DEMO-REPLAY) — strategy lives in c8_fc_adapter/tlog_replay_adapter.py per module-layout.md Tracker: AZ-399 Epic: AZ-265 (E-DEMO-REPLAY)

Document Dependencies

  • _docs/02_document/contracts/replay/replay_protocol.mdTlogReplayFcAdapter concrete shape; Invariants 5, 6, 8.
  • _docs/02_document/contracts/c8_fc_adapter/fc_adapter_protocol.mdFcAdapter Protocol surface this strategy implements.
  • _docs/02_document/components/10_c8_fc_adapter/description.md — § 1 (live AP/iNav adapter shape that the replay strategy mirrors).
  • _docs/02_document/architecture.md — D-C8-3 (pymavlink bundled), R-DEMO-2 (stream-parse).

Problem

Without this task, the replay binary has no FC inbound — there's no IMU/attitude/GPS-health/MAV_STATE feeding C1 + C5; the live PymavlinkArdupilotAdapter cannot be used because there's no FC, only a .tlog file. The TlogReplayFcAdapter is the strategy that lets C1C5 run unchanged.

Outcome

  • src/gps_denied_onboard/components/c8_fc_adapter/tlog_replay_adapter.pyTlogReplayFcAdapter class implementing FcAdapter.
  • Constructor: __init__(self, tlog_path, target_fc_dialect, clock, wgs_converter, time_offset_ms=0, pace=ReplayPace.ASAP, fdr_client).
  • open(...) — open + validate the tlog (fail-fast on missing message types); start the dedicated decode-thread (mirrors live AP adapter's decode-thread semantics per the contract notes).
  • subscribe_telemetry(callback) — register against the multi-subscriber bus (re-uses the bus from AZ-391 inbound subscription).
  • emit_external_position / emit_status_text — raise FcEmitError("replay adapter does not emit to FC") per Invariant 5.
  • request_source_set_switchSourceSetSwitchNotSupportedError.
  • current_flight_state — return latest FlightStateSignal from the parsed stream.
  • close() — stop the decode thread; close the tlog file.
  • INFO log on open(...): kind="c8.tlog_replay.opened" with {tlog_path, target_fc_dialect, time_offset_ms, pace, message_counts: {RAW_IMU: N, ATTITUDE: M, ...}}.
  • ERROR log + raise on missing message types: kind="c8.tlog_replay.missing_messages" with the list of missing types.
  • DEBUG log every 1000 frames: kind="c8.tlog_replay.frame_progress".

Scope

Included

  • TlogReplayFcAdapter class.
  • pymavlink stream parser (no materialisation).
  • AP + iNav dialect support.
  • Multi-subscriber fan-out (re-uses AZ-391's bus implementation).
  • Fail-fast on missing message types (R-DEMO-3).
  • time_offset_ms shift.
  • Pace honoured via injected Clock.
  • Build-flag gating.
  • Unit tests: tlog open + dialect detection, fail-fast missing messages, time_offset_ms applied, pace=REALTIME calls Clock.sleep_until_ns, pace=ASAP no-op, subscribers receive frames in tlog order, emit_external_position raises, source_set_switch unsupported, build-flag gating.

Excluded

  • FrameSource / Clock — owned by AZ-398.
  • ReplaySink — owned by AZ-400.
  • compose_replay — owned by AZ-401.
  • CLI — owned by AZ-402.
  • Auto-sync IMU take-off detection — owned by AZ-405 (this task accepts time_offset_ms as a constructor input; the auto-sync TASK computes it).
  • E2E replay fixture test — owned by AZ-404.

Acceptance Criteria

AC-1: Tlog stream-parse memory bound — open a 500 MB synthetic tlog; subscribe; assert peak RSS during subscribe_telemetry consumption stays within 100 MB above baseline (no materialisation per R-DEMO-2).

AC-2: AP dialect frame mapping — synthetic AP tlog with RAW_IMU + ATTITUDE + GPS_RAW_INT + HEARTBEAT; subscribe; assert four FcTelemetryFrame kinds (IMU_SAMPLE, ATTITUDE, GPS_HEALTH, MAV_STATE) emitted in tlog order with correct payload fields.

AC-3: iNav dialect frame mapping — synthetic iNav tlog (uses AP MAVLink dialect for telemetry per RESTRICT-COMM-2 secondary channel); same frame mapping.

AC-4: Fail-fast missing messages — tlog WITHOUT any RAW_IMU; open(...)FcOpenError("tlog missing required messages: ['RAW_IMU']; consumed by: [C1 VIO, C5 StateEstimator]"). ERROR log + FDR record.

AC-5: time_offset_ms shift — open with time_offset_ms=5000; assert every emitted received_at is shifted by 5e9 ns relative to the raw tlog timestamp; verify with first + last + sample mid-stream frames.

AC-6: Pace REALTIME calls Clock.sleep_until_ns — open with pace=ReplayPace.REALTIME + a wall-clock-faking Clock; subscribe; assert Clock.sleep_until_ns called between every emitted frame with target_ns = received_at.

AC-7: Pace ASAP no-op — open with pace=ReplayPace.ASAP; assert Clock.sleep_until_ns NEVER called between frames; throughput proxy test: 1000 frames consumed in < 1 s on Tier-1 hardware.

AC-8: emit_external_position raises — call emit_external_position(EstimatorOutput(...))FcEmitError("replay adapter does not emit to FC") per Invariant 5.

AC-9: source_set_switch unsupportedrequest_source_set_switch()SourceSetSwitchNotSupportedError.

AC-10: Build-flag gatingBUILD_TLOG_REPLAY_ADAPTER=OFF → constructing the class raises FcAdapterConfigError("BUILD_TLOG_REPLAY_ADAPTER is OFF...").

Non-Functional Requirements

  • Throughput proxy: 1000 frames consumed in < 1 s on Tier-1 hardware (supports the ≥ 5× real-time epic NFT).
  • Memory bound: peak RSS stays within 100 MB above baseline for tlogs up to 5 GB.
  • subscribe_telemetry callback dispatch p99 ≤ 1 ms (parallel to live AP adapter).

Constraints

  • pymavlink bundled unmodified per D-C8-3.
  • Stream-parse only — never materialise (R-DEMO-2).
  • time_offset_ms set ONCE at construction (Invariant 8); no live re-tuning.
  • The decode thread runs on the SAME thread-binding semantics as the live AP adapter (mirrors live behaviour for C1 + C5 consumers; per the contract notes).

Risks & Mitigation

  • R-DEMO-2 (multi-GB tlogs)Mitigation: stream-parse; AC-1 enforces 100 MB bound.
  • R-DEMO-3 (missing message types)Mitigation: fail-fast in open(...); AC-4 enforces; ERROR log lists the missing types AND the components that need them.
  • Risk: pymavlink dialect auto-detection wrong on a tlogMitigation: target_fc_dialect is an explicit constructor input — operator (or CLI) MUST pass the correct value; CLI defaults to ARDUPILOT_PLANE per the most-common case.
  • Risk: tlog timestamps non-monotonic (rare)Mitigation: assert monotonic on read; non-monotonic frames raise FcOpenError (parallel to FrameSource Invariant 3).

Runtime Completeness

  • Named capability: replay-only FcAdapter strategy parsing pymavlink .tlog.
  • Production code: real pymavlink stream-parser, real multi-subscriber fan-out, real Clock-paced subscription.
  • Allowed external stubs: test fakes only.
  • Unacceptable substitutes: a fake-IMU generator masquerading as a tlog adapter (defeats AC-2/AC-3 message-fidelity).

Contract

Implements _docs/02_document/contracts/replay/replay_protocol.mdTlogReplayFcAdapter concrete shape; _docs/02_document/contracts/c8_fc_adapter/fc_adapter_protocol.mdFcAdapter Protocol surface.