[AZ-398] Replay: FrameSource + Clock Protocols + Clock injection

Ship the two Layer-1 cross-cutting Protocols replay mode needs to leave
production C1-C5 components mode-agnostic (Invariant 1) and replay-
deterministic (Invariant 2). Live + replay binaries see the same
interfaces; only the strategy differs.

* Clock Protocol (monotonic_ns / time_ns / sleep_until_ns) +
  WallClock (live + REALTIME replay) + TlogDerivedClock (ASAP replay;
  advance-on-call; non-monotonic source → ClockOrderingError).
* FrameSource Protocol (next_frame -> NavCameraFrame | None / close)
  + LiveCameraFrameSource (cv2.VideoCapture device index) +
  VideoFileFrameSource (cv2.VideoCapture file).
* Build-flag gating: BUILD_VIDEO_FILE_FRAME_SOURCE,
  BUILD_LIVE_CAMERA_FRAME_SOURCE (constructor-time check; Tier-0 OFF
  refuses construction with FrameSourceConfigError).
* Composition-root factories: build_clock + build_frame_source.
* Injected Clock across every component that previously called
  time.monotonic_ns() / time.sleep() directly: c5_state (estimator,
  ESKF, fallback watcher, source-label SM, isam2 handle), c8_fc_adapter
  (inbound MAVLink + MSP2, AP outbound, iNav outbound, QGC GCS),
  c13_fdr writer, c12_operator_tooling httpx flights client. All
  constructors default to WallClock() so existing call sites keep
  live-binary behaviour without a wiring change.
* AC-4 CI guard (tests/_meta/test_no_direct_time_in_components.py)
  AST-scans components/**/*.py for direct time.monotonic_ns /
  time.time_ns / time.sleep references and fails loudly with file:line.
* Conformance + factory tests: tests/unit/clock + tests/unit/frame_source.
* Test fixture updates: FallbackWatcher / SourceLabelStateMachine
  clock_ns is now required (removed time.monotonic_ns default);
  test_az388 patches estimator._clock instead of a module-level time;
  test_az393 ardupilot adapter uses a _FixedClock test double.

Excluded per the task spec: TlogReplayFcAdapter (AZ-399), ReplaySink
(AZ-400), compose_replay (AZ-401), CLI (AZ-402), Docker/CI (AZ-403),
E2E fixture (AZ-404), IMU auto-sync (AZ-405).

Co-authored-by: Cursor <cursoragent@cursor.com>

src/gps_denied_onboard/frame_source/__init__.py

@@ -1,9 +1,21 @@
-"""FrameSource interface + concrete implementations.
+"""``FrameSource`` cross-cutting interface — public surface (AZ-398 v1.0.0).
 
-The interface is bootstrap-stubbed here. `LiveCameraFrameSource` and
-`VideoFileFrameSource` are owned by AZ-398.
+Per AC-9, this module re-exports the Protocol and the error family
+ONLY. Concrete strategies (``LiveCameraFrameSource``,
+``VideoFileFrameSource``) live in their own modules and are imported
+LAZILY by ``runtime_root.frame_source_factory.build_frame_source``;
+this keeps the lazy-import boundary explicit and lets Tier-0 builds
+omit the OpenCV runtime entirely.
 """
 
+from gps_denied_onboard.frame_source.errors import (
+    FrameSourceConfigError,
+    FrameSourceError,
+)
 from gps_denied_onboard.frame_source.interface import FrameSource
 
-__all__ = ["FrameSource"]
+__all__ = [
+    "FrameSource",
+    "FrameSourceConfigError",
+    "FrameSourceError",
+]

src/gps_denied_onboard/frame_source/errors.py

@@ -0,0 +1,48 @@
+"""``FrameSource`` error taxonomy (AZ-398 v1.0.0).
+
+Per the replay contract
+(``_docs/02_document/contracts/replay/replay_protocol.md``), every
+transient I/O failure on the camera path MUST surface as
+:class:`FrameSourceError` (Invariant 4 — replay must be deterministic,
+silent ``None`` drops are forbidden).
+
+The two-class hierarchy mirrors the C6/C7/C1 component taxonomies:
+
+- :class:`FrameSourceError` — operational failures during streaming
+  (decode error, device disconnect, out-of-order frame).
+- :class:`FrameSourceConfigError` — composition-time failures (build
+  flag OFF, missing dependency, invalid config).
+"""
+
+from __future__ import annotations
+
+
+class FrameSourceError(RuntimeError):
+    """Transient or fatal failure during frame ingestion.
+
+    Examples:
+
+    - A corrupt H.264 keyframe in the replay video file.
+    - An ordering violation: ``next_frame()`` returned a frame whose
+      ``monotonic_ns`` is < the previous frame's (Invariant 3).
+    - A USB camera disconnect mid-flight (live source).
+
+    The error message MUST identify the frame index or timestamp where
+    the failure occurred so the operator can correlate against the
+    upstream recording.
+    """
+
+
+class FrameSourceConfigError(RuntimeError):
+    """Composition-time configuration failure for a frame source.
+
+    Examples:
+
+    - ``BUILD_VIDEO_FILE_FRAME_SOURCE=OFF`` and the binary tried to
+      construct :class:`VideoFileFrameSource`.
+    - The configured video path does not exist or is not readable.
+    - OpenCV is not importable (Tier-0 / docker-minimal build).
+    """
+
+
+__all__ = ["FrameSourceError", "FrameSourceConfigError"]

src/gps_denied_onboard/frame_source/interface.py

@@ -1,18 +1,62 @@
-"""`FrameSource` Protocol.
+"""``FrameSource`` Protocol — public Layer 1 cross-cutting interface (AZ-398 v1.0.0).
 
-Owned by AZ-398 (E-DEMO-REPLAY) for the formalisation; bootstrap ships the
-interface stub so C1 can be constructor-injected against it.
+Frozen per ``_docs/02_document/contracts/replay/replay_protocol.md``.
+
+Two strategies implement this Protocol:
+
+- :class:`LiveCameraFrameSource` — the formalised live camera ingest
+  path (gated ``BUILD_LIVE_CAMERA_FRAME_SOURCE``).
+- :class:`VideoFileFrameSource` — the replay-only file decoder (gated
+  ``BUILD_VIDEO_FILE_FRAME_SOURCE``).
+
+Consumers (C1 :class:`VioStrategy`) accept a :class:`FrameSource` via
+constructor injection so production code stays mode-agnostic
+(Invariant 1).
 """
 
 from __future__ import annotations
 
-from collections.abc import Iterator
-from typing import Protocol
+from typing import TYPE_CHECKING, Protocol, runtime_checkable
 
-from gps_denied_onboard._types.nav import NavCameraFrame
+if TYPE_CHECKING:
+    from gps_denied_onboard._types.nav import NavCameraFrame
 
 
+@runtime_checkable
 class FrameSource(Protocol):
-    """A source of `NavCameraFrame` instances."""
+    """A pluggable camera-frame producer.
 
-    def frames(self) -> Iterator[NavCameraFrame]: ...
+    The Protocol exposes two methods and one ordering invariant:
+
+    - :meth:`next_frame` returns the next :class:`NavCameraFrame` (with
+      ``metadata["monotonic_ns"]`` set by the strategy from its
+      injected :class:`Clock`) or ``None`` ONLY when the stream is
+      permanently exhausted (Invariant 4).
+    - Consecutive ``next_frame()`` returns MUST have non-decreasing
+      ``metadata["monotonic_ns"]`` (Invariant 3); out-of-order frames
+      raise :class:`FrameSourceError`.
+    - :meth:`close` releases the underlying capture handle and is
+      idempotent (AC-10).
+    """
+
+    def next_frame(self) -> "NavCameraFrame | None":
+        """Return the next frame, ``None`` on end-of-stream.
+
+        Transient I/O failures (decode error, disconnect) MUST raise
+        :class:`FrameSourceError` — never return ``None`` silently
+        (Invariant 4). After ``None`` has been returned once, every
+        subsequent call MUST also return ``None`` (idempotent EOS).
+        """
+        ...
+
+    def close(self) -> None:
+        """Release the underlying capture handle.
+
+        Idempotent: a second call is a no-op (AC-10); the strategy
+        SHOULD log a DEBUG line on the second call so a debug trace
+        can prove no double-free occurred.
+        """
+        ...
+
+
+__all__ = ["FrameSource"]

src/gps_denied_onboard/frame_source/live_camera.py

@@ -0,0 +1,161 @@
+"""``LiveCameraFrameSource`` — live nav-camera ingest (AZ-398).
+
+Wraps :class:`cv2.VideoCapture` against an integer device index (the
+USB / CSI camera bound at boot by the airborne / research / operator
+binaries). The strategy is intentionally minimal: each
+:meth:`next_frame` call performs one blocking ``capture.read()`` and
+returns the freshest frame; no dedicated decode thread, no ring
+buffer. C1 (the only consumer) drives the loop at its target
+rate, and a blocking read is the simplest way to apply backpressure.
+
+Gated by ``BUILD_LIVE_CAMERA_FRAME_SOURCE`` (Invariant 9). The flag is
+``ON`` for live / research / operator / replay binaries and ``OFF``
+only for unit tests that need to construct a substitute without
+touching a real camera.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from datetime import datetime, timezone
+from typing import TYPE_CHECKING, Any
+
+from gps_denied_onboard.frame_source.errors import (
+    FrameSourceConfigError,
+    FrameSourceError,
+)
+
+if TYPE_CHECKING:
+    from gps_denied_onboard._types.nav import NavCameraFrame
+    from gps_denied_onboard.clock import Clock
+
+
+_BUILD_FLAG = "BUILD_LIVE_CAMERA_FRAME_SOURCE"
+
+_logger = logging.getLogger(__name__)
+
+
+def _build_flag_on() -> bool:
+    raw = os.environ.get(_BUILD_FLAG, "")
+    return raw.strip().lower() in {"on", "1", "true", "yes"}
+
+
+class LiveCameraFrameSource:
+    """Live :class:`FrameSource` strategy backed by ``cv2.VideoCapture``.
+
+    Constructor parameters:
+
+    - ``device_index`` — integer index passed to ``cv2.VideoCapture``;
+      typically ``0`` for the first attached camera.
+    - ``camera_calibration_id`` — string identifier baked into every
+      emitted frame (matches the intrinsics file shipped with the
+      binary).
+    - ``clock`` — injected :class:`Clock`; supplies the per-frame
+      ``monotonic_ns`` ordering key and the wall-clock timestamp.
+    """
+
+    __slots__ = (
+        "_device_index",
+        "_camera_calibration_id",
+        "_clock",
+        "_capture",
+        "_frame_counter",
+        "_last_monotonic_ns",
+        "_closed",
+    )
+
+    def __init__(
+        self,
+        *,
+        device_index: int,
+        camera_calibration_id: str,
+        clock: "Clock",
+    ) -> None:
+        if not _build_flag_on():
+            raise FrameSourceConfigError(
+                f"{_BUILD_FLAG} is OFF in this binary; "
+                "LiveCameraFrameSource is unavailable."
+            )
+        try:
+            import cv2 as _cv2
+        except ImportError as exc:
+            raise FrameSourceConfigError(
+                "LiveCameraFrameSource requires opencv-python; not "
+                "importable in this binary."
+            ) from exc
+        capture = _cv2.VideoCapture(device_index)
+        if not capture.isOpened():
+            capture.release()
+            raise FrameSourceConfigError(
+                f"LiveCameraFrameSource: cv2.VideoCapture could not open "
+                f"device index {device_index}"
+            )
+        self._device_index = device_index
+        self._camera_calibration_id = camera_calibration_id
+        self._clock = clock
+        self._capture = capture
+        self._frame_counter = 0
+        self._last_monotonic_ns = -1
+        self._closed = False
+
+    def next_frame(self) -> "NavCameraFrame | None":
+        from gps_denied_onboard._types.nav import NavCameraFrame
+
+        if self._closed:
+            return None
+        ok, image = self._capture.read()
+        if not ok or image is None:
+            # Live camera: a failed read is a transient error (USB
+            # glitch, driver hiccup). Invariant 4 requires raising,
+            # not returning None — the only legitimate None is EOS,
+            # and a live camera never EOSes.
+            raise FrameSourceError(
+                f"LiveCameraFrameSource: cv2.VideoCapture.read failed at "
+                f"frame {self._frame_counter} (device "
+                f"{self._device_index})"
+            )
+        monotonic_ns = self._clock.monotonic_ns()
+        if monotonic_ns < self._last_monotonic_ns:
+            raise FrameSourceError(
+                f"LiveCameraFrameSource: clock went backwards at frame "
+                f"{self._frame_counter}: {monotonic_ns} ns followed "
+                f"{self._last_monotonic_ns} ns (Invariant 3)"
+            )
+        timestamp = datetime.fromtimestamp(
+            self._clock.time_ns() / 1e9, tz=timezone.utc
+        )
+        metadata: dict[str, Any] = {
+            "monotonic_ns": monotonic_ns,
+            "source": "live_camera",
+            "device_index": self._device_index,
+        }
+        frame = NavCameraFrame(
+            frame_id=self._frame_counter,
+            timestamp=timestamp,
+            image=image,
+            camera_calibration_id=self._camera_calibration_id,
+            metadata=metadata,
+        )
+        self._frame_counter += 1
+        self._last_monotonic_ns = monotonic_ns
+        return frame
+
+    def close(self) -> None:
+        if self._closed:
+            _logger.debug(
+                "LiveCameraFrameSource(device=%s) close called twice; no-op",
+                self._device_index,
+            )
+            return
+        self._closed = True
+        try:
+            self._capture.release()
+        except Exception:  # pragma: no cover — defensive.
+            _logger.exception(
+                "LiveCameraFrameSource(device=%s) cv2.release() raised",
+                self._device_index,
+            )
+
+
+__all__ = ["LiveCameraFrameSource"]

src/gps_denied_onboard/frame_source/video_file.py

@@ -0,0 +1,199 @@
+"""``VideoFileFrameSource`` — replay-only file decoder (AZ-398).
+
+Streams an MP4 / MKV / AVI file frame-by-frame via OpenCV's
+:class:`cv2.VideoCapture`. Each emitted :class:`NavCameraFrame`
+carries:
+
+- ``frame_id`` — a strictly-increasing counter starting at 0.
+- ``timestamp`` — UTC wall-clock at decode time (from the injected
+  :class:`Clock`); the file's own pts is NOT used for this field
+  because replay deterministically remaps it.
+- ``image`` — the decoded BGR ``numpy.ndarray`` (OpenCV native order).
+- ``metadata["monotonic_ns"]`` — the injected :class:`Clock`'s
+  ``monotonic_ns()`` at decode time. This is the value AC-2 asserts
+  non-decreasing.
+- ``metadata["source_pts_ns"]`` — the file's per-frame PTS in ns (the
+  ``CAP_PROP_POS_MSEC`` reading × 1e6) for downstream determinism.
+
+Gated by ``BUILD_VIDEO_FILE_FRAME_SOURCE`` (Invariant 9). The check is
+performed at constructor entry — a Tier-0 build that imports this
+module by accident still raises ``FrameSourceConfigError`` cleanly
+without attempting an OpenCV import.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+from gps_denied_onboard.frame_source.errors import (
+    FrameSourceConfigError,
+    FrameSourceError,
+)
+
+if TYPE_CHECKING:
+    from gps_denied_onboard._types.nav import NavCameraFrame
+    from gps_denied_onboard.clock import Clock
+
+
+_BUILD_FLAG = "BUILD_VIDEO_FILE_FRAME_SOURCE"
+
+_logger = logging.getLogger(__name__)
+
+
+def _build_flag_on() -> bool:
+    """``ON`` / ``1`` / ``true`` / ``yes`` (case-insensitive) → ``True``."""
+    raw = os.environ.get(_BUILD_FLAG, "")
+    return raw.strip().lower() in {"on", "1", "true", "yes"}
+
+
+class VideoFileFrameSource:
+    """Replay :class:`FrameSource` strategy backed by ``cv2.VideoCapture``.
+
+    Stream-decodes a video file; per-frame decode is amortised by
+    OpenCV's internal buffer. The strategy preserves the file's frame
+    order — there is no seek, no random-access path; this keeps
+    replay deterministic (Invariant 10).
+
+    Constructor parameters:
+
+    - ``path`` — filesystem path to an MP4/MKV/AVI (existence checked
+      at construction).
+    - ``camera_calibration_id`` — string identifier propagated into
+      every emitted :class:`NavCameraFrame` so downstream consumers
+      (C1, C3, C4) load the correct intrinsics for the recording.
+    - ``clock`` — injected :class:`Clock`; the strategy reads
+      ``clock.monotonic_ns()`` per emitted frame for the
+      ``metadata["monotonic_ns"]`` ordering field.
+    """
+
+    __slots__ = (
+        "_path",
+        "_camera_calibration_id",
+        "_clock",
+        "_capture",
+        "_frame_counter",
+        "_last_monotonic_ns",
+        "_closed",
+        "_eos_returned",
+    )
+
+    def __init__(
+        self,
+        *,
+        path: Path | str,
+        camera_calibration_id: str,
+        clock: "Clock",
+    ) -> None:
+        if not _build_flag_on():
+            raise FrameSourceConfigError(
+                f"{_BUILD_FLAG} is OFF in this binary; "
+                "VideoFileFrameSource is unavailable. Rebuild with the "
+                "flag set to ON in the replay binary's Dockerfile."
+            )
+        resolved = Path(path)
+        if not resolved.is_file():
+            raise FrameSourceConfigError(
+                f"VideoFileFrameSource: path {resolved!s} does not exist "
+                "or is not a regular file."
+            )
+        try:
+            import cv2 as _cv2
+        except ImportError as exc:
+            raise FrameSourceConfigError(
+                "VideoFileFrameSource requires opencv-python; not "
+                "importable in this binary."
+            ) from exc
+        capture = _cv2.VideoCapture(str(resolved))
+        if not capture.isOpened():
+            capture.release()
+            raise FrameSourceConfigError(
+                f"VideoFileFrameSource: cv2.VideoCapture could not open "
+                f"{resolved!s} (unsupported codec or corrupt header)."
+            )
+        self._path = resolved
+        self._camera_calibration_id = camera_calibration_id
+        self._clock = clock
+        self._capture = capture
+        self._frame_counter = 0
+        self._last_monotonic_ns = -1
+        self._closed = False
+        self._eos_returned = False
+
+    def next_frame(self) -> "NavCameraFrame | None":
+        from gps_denied_onboard._types.nav import NavCameraFrame
+
+        if self._closed or self._eos_returned:
+            return None
+        try:
+            import cv2 as _cv2
+        except ImportError as exc:  # pragma: no cover — established at __init__.
+            raise FrameSourceError(
+                "VideoFileFrameSource: opencv-python disappeared between "
+                "construction and next_frame()"
+            ) from exc
+        ok, image = self._capture.read()
+        if not ok:
+            self._eos_returned = True
+            return None
+        if image is None:
+            # OpenCV's read() returning ok=True with image=None signals a
+            # decoder-internal failure for the current frame; treat as a
+            # transient error per Invariant 4 rather than silently
+            # advancing.
+            raise FrameSourceError(
+                f"VideoFileFrameSource: video decode failed at frame "
+                f"{self._frame_counter} (cv2.VideoCapture.read returned "
+                "ok=True with image=None)"
+            )
+        monotonic_ns = self._clock.monotonic_ns()
+        if monotonic_ns < self._last_monotonic_ns:
+            raise FrameSourceError(
+                f"VideoFileFrameSource: clock went backwards at frame "
+                f"{self._frame_counter}: {monotonic_ns} ns followed "
+                f"{self._last_monotonic_ns} ns (Invariant 3)"
+            )
+        pos_msec = float(self._capture.get(_cv2.CAP_PROP_POS_MSEC))
+        source_pts_ns = int(pos_msec * 1_000_000)
+        timestamp = datetime.fromtimestamp(
+            self._clock.time_ns() / 1e9, tz=timezone.utc
+        )
+        metadata: dict[str, Any] = {
+            "monotonic_ns": monotonic_ns,
+            "source_pts_ns": source_pts_ns,
+            "source": "video_file",
+        }
+        frame = NavCameraFrame(
+            frame_id=self._frame_counter,
+            timestamp=timestamp,
+            image=image,
+            camera_calibration_id=self._camera_calibration_id,
+            metadata=metadata,
+        )
+        self._frame_counter += 1
+        self._last_monotonic_ns = monotonic_ns
+        return frame
+
+    def close(self) -> None:
+        if self._closed:
+            _logger.debug(
+                "VideoFileFrameSource(%s) close called twice; no-op",
+                self._path,
+            )
+            return
+        self._closed = True
+        try:
+            self._capture.release()
+        except Exception:  # pragma: no cover — defensive.
+            # cv2.VideoCapture.release should never raise; if it does on
+            # an exotic backend, we still want to flag the source as
+            # closed so a second close() stays a no-op.
+            _logger.exception(
+                "VideoFileFrameSource(%s) cv2.release() raised", self._path
+            )
+
+
+__all__ = ["VideoFileFrameSource"]