[AZ-594] Implement core-three harness stubs (fdr_reader, frame_source_replay, imu_replay)

Replaces the NotImplementedError stubs AZ-406 reserved on three runner-
side helpers; these were stranded from any tracker ticket since
AZ-407/408 never came back to fill them. Concrete bodies:

* fdr_reader.iter_records: JSONL parser + wire-envelope validator;
  recursive *.jsonl walk; projects {schema_version, ts, producer_id,
  kind, payload} to runner-side FdrRecord with record_type/monotonic_ms
  renames; yields oldest-first.
* frame_source_replay.replay_video: OpenCV VideoCapture decode + JPEG
  re-encode; auto-detects file vs directory; injectable sleep_fn for
  unit-test pacing.
* imu_replay.ImuReplayer.replay: csv.DictReader parse; degrees->radians
  attitude conversion; tolerates scientific notation; same sleep_fn
  injection pattern.

Adds 34 unit tests (14 + 10 + 10). Full e2e unit suite: 558 passed (+31).
Existing scenario _harness_helpers_implemented probes still return False
because they also depend on sitl_observer / fc_proxy_runtime stubs that
remain pending; scenario probe cleanup is out of AZ-594 scope.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-17 08:42:12 +03:00
parent 2d6d44af5d
commit 1d260f7e41
10 changed files with 1196 additions and 76 deletions
+90 -12
View File
@@ -1,22 +1,32 @@
"""Post-run filesystem read of the FDR archive.
The FDR archive is a line-delimited JSON record stream per AZ-272 / AZ-273.
Each line is an `FdrRecord` envelope (producer_id, type, monotonic_ms,
payload). The runner image must NEVER import the SUT's FdrRecord schema
directly — it parses the JSON bytes and validates against a duplicate
record-type allowlist baked into this module.
Each line is an FDR envelope on the wire schema
``{schema_version, ts, producer_id, kind, payload, extra?}``. This module
parses the JSON bytes and validates the wire envelope structurally — the
runner image NEVER imports the SUT's FdrRecord schema directly so a
breaking SUT change surfaces as a parse failure here (visible drift)
rather than silently following along.
Public surface only; concrete parser + assertion helpers are owned by
AZ-441 (NFT-LIM-02 — FDR size budget) and the resilience scenario tasks
that need to crawl the archive (AZ-432, AZ-433, AZ-435).
The runner-side `FdrRecord` dataclass renames `kind` → `record_type` and
projects `ts` (ISO 8601 wall-clock) onto an integer `monotonic_ms` field
for downstream evaluators that work in milliseconds. Within one flight,
ISO 8601 ms-since-epoch is monotonic at the millisecond resolution the
evaluators care about (NFR-RES NTP drift is excluded by AC-7 of the FDR
contract: the on-board clock is monotonic over the lifetime of one
flight session).
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Iterator
_WIRE_REQUIRED_KEYS = ("schema_version", "ts", "producer_id", "kind", "payload")
@dataclass(frozen=True)
class FdrRecord:
@@ -33,15 +43,83 @@ class FdrRecord:
payload: dict[str, object]
def _ts_to_monotonic_ms(ts: str) -> int:
"""Project ISO 8601 ``ts`` onto an int millisecond value.
Accepts trailing ``Z`` (UTC) which ``datetime.fromisoformat`` did not
accept until 3.11; we normalise to ``+00:00`` first.
"""
normalised = ts[:-1] + "+00:00" if ts.endswith("Z") else ts
dt = datetime.fromisoformat(normalised)
return int(dt.timestamp() * 1000)
def _parse_envelope(line_bytes: bytes, source: Path, line_no: int) -> FdrRecord:
"""Decode one JSONL line into a typed envelope.
Wire-side keys are validated structurally; downstream payload keys are
NOT validated here (the consuming evaluator owns its own payload contract).
"""
decoded = json.loads(line_bytes)
if not isinstance(decoded, dict):
raise ValueError(
f"FDR line is not a JSON object: {source}:{line_no}: type={type(decoded).__name__}"
)
missing = [k for k in _WIRE_REQUIRED_KEYS if k not in decoded]
if missing:
raise ValueError(
f"FDR wire envelope missing required keys {missing} at {source}:{line_no}"
)
ts = decoded["ts"]
if not isinstance(ts, str) or not ts:
raise ValueError(f"FDR envelope `ts` must be a non-empty ISO 8601 string at {source}:{line_no}")
producer_id = decoded["producer_id"]
if not isinstance(producer_id, str) or not producer_id:
raise ValueError(
f"FDR envelope `producer_id` must be a non-empty string at {source}:{line_no}"
)
kind = decoded["kind"]
if not isinstance(kind, str) or not kind:
raise ValueError(f"FDR envelope `kind` must be a non-empty string at {source}:{line_no}")
payload = decoded["payload"]
if not isinstance(payload, dict):
raise ValueError(f"FDR envelope `payload` must be an object at {source}:{line_no}")
return FdrRecord(
producer_id=producer_id,
monotonic_ms=_ts_to_monotonic_ms(ts),
record_type=kind,
payload=payload,
)
def iter_records(fdr_archive_root: Path) -> Iterator[FdrRecord]:
"""Iterate every FDR record in the archive root (ordered by monotonic_ms).
Raises NotImplementedError until AZ-441 supplies the orjson-backed parser.
Walks every ``*.jsonl`` file under ``fdr_archive_root`` (recursive),
parses each line as a wire envelope, and yields the runner-side
``FdrRecord`` projection. Records are emitted oldest-first across the
union of all files.
Raises ``FileNotFoundError`` if the archive root does not exist.
Raises ``ValueError`` (with a file + line pointer) on malformed JSON,
a wrong-shape envelope, or an unparseable ``ts``.
"""
raise NotImplementedError(
"fdr_reader.iter_records is owned by AZ-441 — AZ-406 supplies only "
"the public surface."
)
if not fdr_archive_root.exists():
raise FileNotFoundError(
f"FDR archive root not found: {fdr_archive_root}"
)
records: list[FdrRecord] = []
for jsonl_path in sorted(fdr_archive_root.rglob("*.jsonl")):
if not jsonl_path.is_file():
continue
with jsonl_path.open("rb") as fh:
for line_no, raw in enumerate(fh, start=1):
stripped = raw.strip()
if not stripped:
continue
records.append(_parse_envelope(stripped, jsonl_path, line_no))
records.sort(key=lambda r: r.monotonic_ms)
yield from records
def archive_size_bytes(fdr_archive_root: Path) -> int: