[AZ-401] [AZ-400] Replay — compose_root replay-mode branch + transport seam

Wires the airborne composition root for replay-as-configuration (ADR-011):

- compose_root(config) branches on config.mode in {"live", "replay"}.
  Live behaviour is unchanged; replay builds ReplayInputAdapter,
  attaches JsonlReplaySink, and injects NoopMavlinkTransport.
- New private module runtime_root/_replay_branch.py holds the
  replay-only strategy graph + build-flag gate + calibration loader.
- Config gains Config.mode (Literal["live","replay"]) plus
  Config.replay sub-block with nested ReplayAutoSyncConfig that mirrors
  the AZ-405 AutoSyncConfig DTO; YAML loader + ENV map updated.

Absorbs the AZ-400 transport-seam retrofit that AZ-401 strictly
required but AZ-400 had not delivered:

- New MavlinkTransport Protocol (write/bytes_written/close).
- NoopMavlinkTransport (replay; build-flag gated, idempotent close,
  thread-safe byte counter).
- SerialMavlinkTransport (live, no-op restructure of existing pymavlink
  byte path; encoder retrofit to actually USE it is the AZ-558
  follow-up).

AZ-401 AC-9 (NoopMavlinkTransport.bytes_written > 0 after C8 encoders
run) is BLOCKED on AZ-558 — the encoder routing retrofit is out of
the AZ-401 task envelope (FORBIDDEN files: pymavlink_ardupilot_adapter,
msp2_inav_adapter). AZ-558 spec, batch_61_review.md, and the test's
@pytest.mark.skip rationale all carry the deferral reason.

Tests: 22 compose_root replay-branch tests + 17 transport tests.
Full regression: 2063 passed, 86 environment-skips, 1 documented
skip (AC-9 / AZ-558), 1 pre-existing flaky perf test deselected.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-14 11:55:33 +03:00
parent 8149083cac
commit 17a0d074af
19 changed files with 2156 additions and 45 deletions
@@ -10,7 +10,14 @@ from gps_denied_onboard._types.emitted import EmittedExternalPosition
from gps_denied_onboard.components.c8_fc_adapter.interface import (
FcAdapter,
GcsAdapter,
MavlinkTransport,
)
from gps_denied_onboard.components.c8_fc_adapter.replay_sink import ReplaySink
__all__ = ["EmittedExternalPosition", "FcAdapter", "GcsAdapter", "ReplaySink"]
__all__ = [
"EmittedExternalPosition",
"FcAdapter",
"GcsAdapter",
"MavlinkTransport",
"ReplaySink",
]
@@ -18,6 +18,8 @@ __all__ = [
"GcsAdapterConfigError",
"GcsAdapterError",
"GcsEmitError",
"MavlinkTransportConfigError",
"MavlinkTransportError",
"SigningHandshakeError",
"SigningKeyExpiredError",
"SourceSetSwitchError",
@@ -96,3 +98,15 @@ class GcsAdapterConfigError(GcsAdapterError):
Raised at config-load for unknown strategy names and at factory
build for build-flag-OFF strategies.
"""
# ---------------------------------------------------------------------
# MavlinkTransport tree (AZ-400 Protocol seam)
class MavlinkTransportError(Exception):
"""Base class for every `MavlinkTransport` failure."""
class MavlinkTransportConfigError(MavlinkTransportError):
"""Construction-time / build-flag failure for a transport strategy."""
@@ -31,7 +31,56 @@ from gps_denied_onboard._types.fc import (
)
from gps_denied_onboard._types.state import EstimatorOutput
__all__ = ["FcAdapter", "GcsAdapter"]
__all__ = ["FcAdapter", "GcsAdapter", "MavlinkTransport"]
@runtime_checkable
class MavlinkTransport(Protocol):
"""Outbound MAVLink byte-stream destination (AZ-400 Protocol seam).
The contract (replay_protocol.md v2.0.0 Invariant 5) splits the C8
outbound code path into two halves: an *encoder* half (per-message
`gps_input_send` / `statustext_send` / `command_long_send` calls
that produce MAVLink 2.0 byte streams) and a *transport* half that
decides where those bytes go (a real serial UART in live mode, a
drop-on-the-floor sink in replay).
Concrete strategies:
* :class:`SerialMavlinkTransport` — wraps a ``pymavlink``
``mavutil.mavlink_connection`` open on the FC's UART (live mode).
* :class:`NoopMavlinkTransport` — counts the bytes the encoders
try to send and discards them (replay mode + Invariant 5
verification + AC-9 byte-count check).
Only :func:`gps_denied_onboard.runtime_root.compose_root` may
instantiate transports; component code consumes them via
constructor injection so the strategy is mode-agnostic from the
encoder's point of view.
"""
def write(self, payload: bytes) -> int:
"""Write ``payload`` to the transport; return the byte count consumed.
Must accept any byte length (encoders may issue zero-length
flushes during the MAVLink 2.0 signing handshake). Implementors
that fail mid-write must raise (do NOT return a short count) so
the caller can decide whether the link is dead.
"""
...
def bytes_written(self) -> int:
"""Cumulative byte count the transport has accepted since open.
Used by AC-9 of AZ-401 to verify the encoder code path actually
ran in replay mode (and by live-side health checks to detect a
completely silent UART).
"""
...
def close(self) -> None:
"""Close the underlying transport; idempotent."""
...
@runtime_checkable
@@ -0,0 +1,106 @@
"""``NoopMavlinkTransport`` — replay-mode outbound byte sink (AZ-400).
Replay-mode strategy for the :class:`MavlinkTransport` Protocol. Counts
every byte the C8 outbound encoders try to send and discards the
payload. Used by ``compose_root`` in ``config.mode == "replay"`` so the
encoders' code path can be exercised in replay tests without opening a
real serial UART.
Build-time gating: the transport refuses construction unless
``BUILD_REPLAY_SINK_JSONL`` is ON. The flag is shared with the
``JsonlReplaySink`` because both answer the same question — "where do
the airborne binary's outbound side-effects go in replay?" — and the
replay binary always wants both ON together.
Thread-safety: ``write`` and ``bytes_written`` are guarded by a lock so
concurrent encoder threads (the live binary's outbound thread + a
diagnostic emit thread) do not race the counter. Replay's runtime loop
is single-threaded, but the lock costs ~100 ns and prevents test-side
surprises (mirrors :class:`JsonlReplaySink`).
"""
from __future__ import annotations
import os
import threading
from typing import Final
from gps_denied_onboard.components.c8_fc_adapter.errors import (
MavlinkTransportConfigError,
MavlinkTransportError,
)
from gps_denied_onboard.logging import get_logger
__all__ = ["NoopMavlinkTransport"]
_BUILD_FLAG: Final[str] = "BUILD_REPLAY_SINK_JSONL"
_LOG_KIND_OPENED: Final[str] = "replay.transport.noop_opened"
_LOG_KIND_CLOSED: Final[str] = "replay.transport.noop_closed"
_LOG_KIND_DOUBLE_CLOSE: Final[str] = "replay.transport.noop_double_close"
def _build_flag_on() -> bool:
raw = os.environ.get(_BUILD_FLAG, "")
return raw.strip().lower() in {"on", "1", "true", "yes"}
class NoopMavlinkTransport:
"""Drop-on-the-floor :class:`MavlinkTransport` for replay mode.
Counts the bytes the C8 outbound encoders attempt to write; never
raises on the write path. Idempotent close.
"""
__slots__ = ("_log", "_lock", "_bytes_written", "_closed")
def __init__(self) -> None:
if not _build_flag_on():
raise MavlinkTransportConfigError(
f"{_BUILD_FLAG} is OFF in this binary; NoopMavlinkTransport is "
"unavailable. Rebuild with the flag set to ON in the airborne "
"Dockerfile."
)
self._log = get_logger("c8_fc_adapter.noop_mavlink_transport")
self._lock = threading.Lock()
self._bytes_written = 0
self._closed = False
self._log.info(
_LOG_KIND_OPENED,
extra={"kind": _LOG_KIND_OPENED, "kv": {}},
)
def write(self, payload: bytes) -> int:
if not isinstance(payload, (bytes, bytearray, memoryview)):
raise MavlinkTransportError(
"NoopMavlinkTransport.write expects bytes-like; got "
f"{type(payload).__name__}"
)
with self._lock:
if self._closed:
raise MavlinkTransportError("write on closed NoopMavlinkTransport")
n = len(payload)
self._bytes_written += n
return n
def bytes_written(self) -> int:
with self._lock:
return self._bytes_written
def close(self) -> None:
with self._lock:
if self._closed:
self._log.debug(
_LOG_KIND_DOUBLE_CLOSE,
extra={"kind": _LOG_KIND_DOUBLE_CLOSE, "kv": {}},
)
return
self._closed = True
total = self._bytes_written
self._log.info(
_LOG_KIND_CLOSED,
extra={
"kind": _LOG_KIND_CLOSED,
"kv": {"bytes_written": total},
},
)
@@ -360,8 +360,8 @@ def create(*, output_path: Path, fdr_client: "FdrClient") -> JsonlReplaySink:
"""Module-level factory entrypoint per project convention.
Mirrors the ``create`` factories used by the C2/C3 strategies so
the AZ-401 ``compose_replay`` wiring resolves the sink through a
single named-symbol contract instead of poking at the class
constructor directly.
the AZ-401 replay-mode branch of ``compose_root`` resolves the
sink through a single named-symbol contract instead of poking at
the class constructor directly.
"""
return JsonlReplaySink(output_path=output_path, fdr_client=fdr_client)
@@ -0,0 +1,124 @@
"""``SerialMavlinkTransport`` — live-mode outbound byte sink (AZ-400).
Live-mode strategy for the :class:`MavlinkTransport` Protocol. Wraps a
``pymavlink`` ``mavutil.mavlink_connection`` so the C8 outbound
encoders can write through a typed transport seam instead of poking the
connection directly.
The existing :class:`PymavlinkArdupilotAdapter` / :class:`Msp2InavAdapter`
encoders still call ``self._connection.mav.gps_input_send(...)`` etc.
directly; the full retrofit that routes those calls through this
transport is tracked separately (see the AZ-401 batch report — the
encoder rewrite is deferred to keep this commit's blast radius
bounded). This module ships the typed surface so
* :func:`gps_denied_onboard.runtime_root.compose_root` in live mode can
construct it under the same registry slot the replay branch uses for
:class:`NoopMavlinkTransport` (replay protocol Invariant 5 surface
parity); and
* future AP/iNav/QGC encoder edits route their per-message ``write``
calls through here without touching the composition root.
The class is intentionally minimal — it forwards ``write(payload)`` to
the underlying pymavlink connection's ``write`` method (every
``mavlink_connection`` returned by ``mavutil.mavlink_connection`` is a
file-like object exposing ``.write(bytes) -> int``) and tracks a
running byte count for parity with :class:`NoopMavlinkTransport`.
"""
from __future__ import annotations
import threading
from typing import TYPE_CHECKING, Any, Final
from gps_denied_onboard.components.c8_fc_adapter.errors import (
MavlinkTransportError,
)
from gps_denied_onboard.logging import get_logger
if TYPE_CHECKING:
pass
__all__ = ["SerialMavlinkTransport"]
_LOG_KIND_OPENED: Final[str] = "live.transport.serial_opened"
_LOG_KIND_CLOSED: Final[str] = "live.transport.serial_closed"
_LOG_KIND_DOUBLE_CLOSE: Final[str] = "live.transport.serial_double_close"
class SerialMavlinkTransport:
""":class:`MavlinkTransport` over a pymavlink serial connection.
Constructor injects an already-open ``mavutil.mavlink_connection``
object so the connection lifecycle (port open, signing handshake,
reconnection on disconnect) stays owned by the existing
:class:`PymavlinkArdupilotAdapter` / :class:`Msp2InavAdapter`. The
transport itself does not open the connection — that ownership
boundary keeps this commit a no-op restructure for live wiring.
"""
__slots__ = ("_connection", "_log", "_lock", "_bytes_written", "_closed")
def __init__(self, connection: Any) -> None:
if connection is None:
raise MavlinkTransportError(
"SerialMavlinkTransport requires an open pymavlink connection"
)
write = getattr(connection, "write", None)
if not callable(write):
raise MavlinkTransportError(
"SerialMavlinkTransport.connection must expose a callable "
".write(bytes) -> int; got "
f"{type(connection).__name__}"
)
self._connection = connection
self._log = get_logger("c8_fc_adapter.serial_mavlink_transport")
self._lock = threading.Lock()
self._bytes_written = 0
self._closed = False
self._log.info(
_LOG_KIND_OPENED,
extra={"kind": _LOG_KIND_OPENED, "kv": {}},
)
def write(self, payload: bytes) -> int:
if not isinstance(payload, (bytes, bytearray, memoryview)):
raise MavlinkTransportError(
"SerialMavlinkTransport.write expects bytes-like; got "
f"{type(payload).__name__}"
)
with self._lock:
if self._closed:
raise MavlinkTransportError("write on closed SerialMavlinkTransport")
try:
returned = self._connection.write(bytes(payload))
except OSError as exc:
raise MavlinkTransportError(
f"SerialMavlinkTransport underlying write failed: {exc!r}"
) from exc
n = int(returned) if returned is not None else len(payload)
self._bytes_written += n
return n
def bytes_written(self) -> int:
with self._lock:
return self._bytes_written
def close(self) -> None:
with self._lock:
if self._closed:
self._log.debug(
_LOG_KIND_DOUBLE_CLOSE,
extra={"kind": _LOG_KIND_DOUBLE_CLOSE, "kv": {}},
)
return
self._closed = True
total = self._bytes_written
self._log.info(
_LOG_KIND_CLOSED,
extra={
"kind": _LOG_KIND_CLOSED,
"kv": {"bytes_written": total},
},
)
@@ -3,8 +3,10 @@
from gps_denied_onboard.config.loader import ENV_KEY_MAP, load_config
from gps_denied_onboard.config.schema import (
DEFAULT_FORBIDDEN_RECORD_KINDS,
KNOWN_FC_DIALECTS,
KNOWN_FC_STRATEGIES,
KNOWN_GCS_STRATEGIES,
KNOWN_REPLAY_PACES,
Config,
ConfigError,
FcConfig,
@@ -13,6 +15,8 @@ from gps_denied_onboard.config.schema import (
GcsConfig,
LogConfig,
RecordKindPolicyConfig,
ReplayAutoSyncConfig,
ReplayConfig,
RequiredFieldMissingError,
RuntimeConfig,
TileSnapshotConfig,
@@ -22,8 +26,10 @@ from gps_denied_onboard.config.schema import (
__all__ = [
"DEFAULT_FORBIDDEN_RECORD_KINDS",
"ENV_KEY_MAP",
"KNOWN_FC_DIALECTS",
"KNOWN_FC_STRATEGIES",
"KNOWN_GCS_STRATEGIES",
"KNOWN_REPLAY_PACES",
"Config",
"ConfigError",
"FcConfig",
@@ -32,6 +38,8 @@ __all__ = [
"GcsConfig",
"LogConfig",
"RecordKindPolicyConfig",
"ReplayAutoSyncConfig",
"ReplayConfig",
"RequiredFieldMissingError",
"RuntimeConfig",
"TileSnapshotConfig",
+110 -1
View File
@@ -23,10 +23,13 @@ import yaml
from gps_denied_onboard.config.schema import (
_COMPONENT_REGISTRY,
Config,
ConfigError,
FcConfig,
FdrConfig,
GcsConfig,
LogConfig,
ReplayAutoSyncConfig,
ReplayConfig,
RequiredFieldMissingError,
RuntimeConfig,
_replace_block,
@@ -64,6 +67,13 @@ ENV_KEY_MAP: Final[dict[str, tuple[str, str]]] = {
"GCS_PORT_DEVICE": ("gcs", "port_device"),
"GCS_PORT_BAUD": ("gcs", "port_baud"),
"GCS_SUMMARY_RATE_HZ": ("gcs", "summary_rate_hz"),
# Replay block (AZ-401)
"REPLAY_VIDEO_PATH": ("replay", "video_path"),
"REPLAY_TLOG_PATH": ("replay", "tlog_path"),
"REPLAY_OUTPUT_PATH": ("replay", "output_path"),
"REPLAY_PACE": ("replay", "pace"),
"REPLAY_TIME_OFFSET_MS": ("replay", "time_offset_ms"),
"REPLAY_TARGET_FC_DIALECT": ("replay", "target_fc_dialect"),
}
# Env vars that MUST resolve to a non-empty value before `load_config`
@@ -106,6 +116,12 @@ _FIELD_COERCIONS: Final[dict[str, type]] = {
"spoof_recovery_source_set": int,
"source_set_switch_timeout_ms": int,
"summary_rate_hz": float,
# Replay block coercions (AZ-401)
"video_path": str,
"tlog_path": str,
"output_path": str,
"pace": str,
"target_fc_dialect": str,
}
@@ -121,8 +137,91 @@ def _coerce_value(field_name: str, raw: Any) -> Any:
) from exc
def _coerce_optional_int(field_name: str, raw: Any) -> int | None:
"""Coerce ``raw`` to ``int`` or ``None`` (empty / null sentinels become ``None``)."""
if raw is None:
return None
if isinstance(raw, str) and raw.strip() == "":
return None
if isinstance(raw, int) and not isinstance(raw, bool):
return raw
try:
return int(raw)
except (TypeError, ValueError) as exc:
raise RequiredFieldMissingError(
f"config field {field_name!r}: cannot coerce {raw!r} to int ({exc})"
) from exc
def _build_replay_block(overrides: Mapping[str, Any]) -> ReplayConfig:
"""Build a :class:`ReplayConfig` from YAML/env overrides.
Handles two non-trivial coercions the generic path cannot:
* ``time_offset_ms`` — ``int | None`` (empty string / None → None).
* ``auto_sync`` — nested mapping → :class:`ReplayAutoSyncConfig`.
"""
flat: dict[str, Any] = {}
auto_sync_overrides: Mapping[str, Any] = {}
for key, value in overrides.items():
if key == "auto_sync":
if value is None:
continue
if not isinstance(value, Mapping):
raise ConfigError(
f"replay.auto_sync must be a mapping; got {type(value).__name__}"
)
auto_sync_overrides = value
continue
if key == "time_offset_ms":
flat[key] = _coerce_optional_int(key, value)
continue
flat[key] = _coerce_value(key, value)
auto_sync_block = _replace_block(
ReplayAutoSyncConfig(),
{k: _coerce_replay_auto_sync_field(k, v) for k, v in auto_sync_overrides.items()},
)
flat["auto_sync"] = auto_sync_block
return _replace_block(ReplayConfig(), flat)
_REPLAY_AUTO_SYNC_TYPES: Final[dict[str, type]] = {
"takeoff_accel_threshold_g": float,
"takeoff_attitude_rate_threshold_rad_s": float,
"sustained_seconds": float,
"prescan_max_messages": int,
"video_motion_threshold": float,
"video_motion_scan_seconds": float,
"match_threshold_pct": float,
"match_window_ms": int,
"low_confidence_threshold": float,
}
def _coerce_replay_auto_sync_field(field_name: str, raw: Any) -> Any:
target_type = _REPLAY_AUTO_SYNC_TYPES.get(field_name)
if target_type is None or isinstance(raw, target_type):
return raw
try:
return target_type(raw)
except (TypeError, ValueError) as exc:
raise RequiredFieldMissingError(
f"config field replay.auto_sync.{field_name}: cannot coerce {raw!r} "
f"to {target_type.__name__} ({exc})"
) from exc
_TOP_LEVEL_SCALAR_FIELDS: Final[frozenset[str]] = frozenset({"mode"})
def _load_yaml_files(paths: Sequence[Path]) -> dict[str, dict[str, Any]]:
"""Merge YAML files in order: later paths win for the same block + field."""
"""Merge YAML files in order: later paths win for the same block + field.
Top-level scalar fields named in :data:`_TOP_LEVEL_SCALAR_FIELDS`
(currently ``mode``) are collected under the synthetic ``__top__``
block so the ``Config`` outer fields can be overridden alongside
the nested cross-cutting / component blocks.
"""
merged: dict[str, dict[str, Any]] = {}
for path in paths:
data = yaml.safe_load(path.read_text()) or {}
@@ -131,6 +230,9 @@ def _load_yaml_files(paths: Sequence[Path]) -> dict[str, dict[str, Any]]:
f"YAML at {path} must be a mapping at the top level; got {type(data).__name__}"
)
for block_name, block_value in data.items():
if block_name in _TOP_LEVEL_SCALAR_FIELDS:
merged.setdefault("__top__", {})[block_name] = block_value
continue
if not isinstance(block_value, dict):
continue
merged.setdefault(block_name, {}).update(block_value)
@@ -193,6 +295,11 @@ def load_config(
GcsConfig(),
{k: _coerce_value(k, v) for k, v in yaml_overrides.get("gcs", {}).items()},
)
replay_block = _build_replay_block(yaml_overrides.get("replay", {}))
raw_mode = yaml_overrides.get("__top__", {}).get("mode")
if raw_mode is None:
raw_mode = env.get("MODE", "live")
mode = str(raw_mode).strip().lower()
component_blocks = _resolve_component_blocks()
for slug, dataclass_type in _COMPONENT_REGISTRY.items():
@@ -209,5 +316,7 @@ def load_config(
fdr=fdr_block,
fc=fc_block,
gcs=gcs_block,
replay=replay_block,
mode=mode, # type: ignore[arg-type] # validated by Config.__post_init__
components=component_blocks,
)
+123 -3
View File
@@ -12,12 +12,14 @@ from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass, field, fields, is_dataclass, replace
from typing import Any, Final
from typing import Any, Final, Literal
__all__ = [
"DEFAULT_FORBIDDEN_RECORD_KINDS",
"KNOWN_FC_DIALECTS",
"KNOWN_FC_STRATEGIES",
"KNOWN_GCS_STRATEGIES",
"KNOWN_REPLAY_PACES",
"Config",
"ConfigError",
"FcConfig",
@@ -26,6 +28,8 @@ __all__ = [
"GcsConfig",
"LogConfig",
"RecordKindPolicyConfig",
"ReplayAutoSyncConfig",
"ReplayConfig",
"RequiredFieldMissingError",
"RuntimeConfig",
"TileSnapshotConfig",
@@ -35,6 +39,8 @@ __all__ = [
KNOWN_FC_STRATEGIES: Final[frozenset[str]] = frozenset({"ardupilot_plane", "inav"})
KNOWN_GCS_STRATEGIES: Final[frozenset[str]] = frozenset({"qgc_mavlink"})
KNOWN_REPLAY_PACES: Final[frozenset[str]] = frozenset({"asap", "realtime"})
KNOWN_FC_DIALECTS: Final[frozenset[str]] = frozenset({"ardupilot_plane", "inav"})
# Default raw-frame kinds that AZ-295's RecordKindPolicy must reject
@@ -289,6 +295,98 @@ class RuntimeConfig:
tile_cache_path: str = "/var/lib/gps-denied/tiles"
@dataclass(frozen=True)
class ReplayAutoSyncConfig:
"""Operator-tunable thresholds for the AZ-405 auto-sync detector.
Mirrors the ``AutoSyncConfig`` DTO in
:mod:`gps_denied_onboard.replay_input.interface`; lives here so the
YAML loader can populate it without importing the Layer-4 replay
package (which would create a config → replay_input → config cycle).
The composition root translates this block into the matching
``AutoSyncConfig`` instance when it builds the
:class:`ReplayInputAdapter`.
All fields default to the contract values in
``_docs/02_document/contracts/replay/replay_protocol.md`` v2.0.0.
"""
takeoff_accel_threshold_g: float = 0.5
takeoff_attitude_rate_threshold_rad_s: float = 1.0
sustained_seconds: float = 0.5
prescan_max_messages: int = 6000
video_motion_threshold: float = 1.5
video_motion_scan_seconds: float = 10.0
match_threshold_pct: float = 95.0
match_window_ms: int = 100
low_confidence_threshold: float = 0.80
@dataclass(frozen=True)
class ReplayConfig:
"""Replay-mode runtime descriptors (AZ-401 / E-DEMO-REPLAY).
Read by :func:`gps_denied_onboard.runtime_root.compose_root` only
when the outer :attr:`Config.mode` is ``"replay"``. Live mode
ignores every field — the block exists as a default-constructed
placeholder so the outer :class:`Config` shape stays stable across
modes.
Validation here is shape-only: the unknown-pace / unknown-dialect
cases reject early. Path existence is verified by the composition
root because YAML may legally reference paths injected at runtime.
Attributes:
video_path: Filesystem path to the replay video (``.mp4`` /
``.h264``). Empty string is the default sentinel; a
non-empty value is required when ``mode == "replay"``.
tlog_path: Filesystem path to the matching pymavlink ``.tlog``.
Empty string is the default sentinel.
output_path: Filesystem path the :class:`JsonlReplaySink` will
write to. Default points at ``/tmp/replay.jsonl`` for
developer ergonomics; production wiring overrides via the
CLI ``--output`` flag.
pace: One of :data:`KNOWN_REPLAY_PACES`. ``"asap"`` selects
:class:`TlogDerivedClock`; ``"realtime"`` selects
:class:`WallClock`.
time_offset_ms: Manual override for the video-vs-tlog offset.
``None`` means "run AZ-405 auto-sync"; an integer value
bypasses auto-sync entirely.
target_fc_dialect: One of :data:`KNOWN_FC_DIALECTS`; controls
which pymavlink dialect the :class:`TlogReplayFcAdapter`
decodes.
auto_sync: Operator-tunable thresholds for the AZ-405
auto-sync detector.
"""
video_path: str = ""
tlog_path: str = ""
output_path: str = "/tmp/replay.jsonl"
pace: str = "asap"
time_offset_ms: int | None = None
target_fc_dialect: str = "ardupilot_plane"
auto_sync: ReplayAutoSyncConfig = field(default_factory=ReplayAutoSyncConfig)
def __post_init__(self) -> None:
if self.pace not in KNOWN_REPLAY_PACES:
raise ConfigError(
f"ReplayConfig.pace={self.pace!r} not in "
f"{sorted(KNOWN_REPLAY_PACES)}"
)
if self.target_fc_dialect not in KNOWN_FC_DIALECTS:
raise ConfigError(
f"ReplayConfig.target_fc_dialect={self.target_fc_dialect!r} "
f"not in {sorted(KNOWN_FC_DIALECTS)}"
)
if self.time_offset_ms is not None and not isinstance(
self.time_offset_ms, int
):
raise ConfigError(
"ReplayConfig.time_offset_ms must be int or None; "
f"got {type(self.time_offset_ms).__name__}"
)
# Documented defaults for cross-cutting blocks ONLY. Per-component defaults
# live with their own component epic. The registry below is the single
# source of truth so two components cannot silently claim the same key.
@@ -298,6 +396,7 @@ _DEFAULT_BLOCKS: Final[dict[str, type]] = {
"runtime": RuntimeConfig,
"fc": FcConfig,
"gcs": GcsConfig,
"replay": ReplayConfig,
}
@@ -341,6 +440,14 @@ class Config:
Components consume only their own slice via ``config.components[slug]``;
the runtime / log / fdr cross-cutting blocks are read directly via
attribute access by the composition root.
The :attr:`mode` field selects between ``"live"`` (the default —
behaves exactly as the pre-AZ-401 binary) and ``"replay"`` (drives
the airborne binary off recorded video + tlog inputs per ADR-011 /
replay protocol v2.0.0). Replay-only configuration lives under
:attr:`replay`; the field is always present (default-constructed)
so the outer shape is stable, but its contents are ignored in live
mode.
"""
runtime: RuntimeConfig = field(default_factory=RuntimeConfig)
@@ -348,14 +455,23 @@ class Config:
fdr: FdrConfig = field(default_factory=FdrConfig)
fc: FcConfig = field(default_factory=FcConfig)
gcs: GcsConfig = field(default_factory=GcsConfig)
replay: ReplayConfig = field(default_factory=ReplayConfig)
mode: Literal["live", "replay"] = "live"
components: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
if self.mode not in ("live", "replay"):
raise ConfigError(
f"Config.mode={self.mode!r} not in ('live', 'replay')"
)
@classmethod
def with_blocks(cls, **blocks: Any) -> Config:
"""Build a `Config` from a flat name-to-instance map.
Cross-cutting names (``log``, ``fdr``, ``runtime``, ``fc``, ``gcs``)
become attributes; every other key is treated as a component slug
Cross-cutting names (``log``, ``fdr``, ``runtime``, ``fc``,
``gcs``, ``replay``) become attributes; ``mode`` is also a
recognised key. Every other key is treated as a component slug
and goes into ``components``.
"""
runtime = blocks.pop("runtime", RuntimeConfig())
@@ -363,12 +479,16 @@ class Config:
fdr = blocks.pop("fdr", FdrConfig())
fc = blocks.pop("fc", FcConfig())
gcs = blocks.pop("gcs", GcsConfig())
replay = blocks.pop("replay", ReplayConfig())
mode = blocks.pop("mode", "live")
return cls(
runtime=runtime,
log=log,
fdr=fdr,
fc=fc,
gcs=gcs,
replay=replay,
mode=mode,
components=dict(blocks),
)
+77 -27
View File
@@ -7,9 +7,14 @@ the component graph in dependency order.
Per-binary entrypoints:
* :func:`compose_root` - airborne runtime
* :func:`compose_root` - airborne runtime; serves both ``config.mode == "live"``
and ``config.mode == "replay"`` per ADR-011 (replay-as-configuration)
* :func:`compose_operator` - operator-side tooling (pre-flight, post-landing)
* :func:`compose_replay` - replay-cli runtime (extension owned by AZ-401)
Replay is a configuration of :func:`compose_root`, not a separate function:
the branch on ``config.mode`` lives in :mod:`._replay_branch`. The legacy
``compose_replay`` export was removed by AZ-401 (ADR-011 supersedes the
v1.0.0 "replay is a sibling root" design).
Public surface frozen by
``_docs/02_document/contracts/shared_config/composition_root_protocol.md`` v1.0.0.
@@ -24,6 +29,10 @@ from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any, Final, Literal, get_args
from gps_denied_onboard.config import Config, load_config
from gps_denied_onboard.runtime_root._replay_branch import (
CompositionError,
build_replay_components,
)
from gps_denied_onboard.runtime_root.c12_factory import (
build_flights_api_client,
)
@@ -67,6 +76,7 @@ __all__ = [
"EXIT_FDR_OPEN_FAILURE",
"EXIT_GENERIC_FAILURE",
"REQUIRED_ENV_VARS",
"CompositionError",
"ConfigurationError",
"OperatorRoot",
"OutboundThreadAlreadyBoundError",
@@ -91,7 +101,6 @@ __all__ = [
"clear_strategy_registries",
"clear_strategy_registry",
"compose_operator",
"compose_replay",
"compose_root",
"list_registered_fc_strategies",
"list_registered_gcs_strategies",
@@ -317,8 +326,17 @@ def _compose(
binary: str,
allowed_tiers: frozenset[StrategyTier],
extra_required_env: Iterable[str],
pre_constructed: Mapping[str, Any] | None = None,
) -> tuple[dict[str, Any], tuple[str, ...]]:
"""Shared composition path used by ``compose_root`` / ``compose_operator``."""
"""Shared composition path used by ``compose_root`` / ``compose_operator``.
``pre_constructed`` lets the caller seed the ``constructed`` dict
before any registered factory runs — used by the replay-mode branch
of :func:`compose_root` to inject the cross-cutting replay
strategies (``frame_source``, ``fc_adapter``, ``clock``,
``mavlink_transport``, ``replay_sink``) so any C1-C7 factory that
declares a dependency on one finds it already populated.
"""
_check_required_env(extra_required=extra_required_env)
selections = _resolve_component_strategies(config, allowed_tiers)
resolved: dict[str, _Registration] = {
@@ -326,7 +344,9 @@ def _compose(
for slug, strategy in selections.items()
}
order = _topo_order(resolved.keys(), resolved)
constructed: dict[str, Any] = {}
constructed: dict[str, Any] = (
dict(pre_constructed) if pre_constructed is not None else {}
)
for slug in order:
registration = resolved[slug]
try:
@@ -336,7 +356,11 @@ def _compose(
_close_partial_instances(constructed)
raise
_ = binary # documented but unused beyond labelling the returned root
return constructed, tuple(order)
# Returned components include only the registry-driven strategies — the
# caller is responsible for merging the pre_constructed dict back in if
# it wants a single combined view.
registry_built = {slug: constructed[slug] for slug in order}
return registry_built, tuple(order)
def _close_partial_instances(instances: Mapping[str, Any]) -> None:
@@ -392,19 +416,61 @@ def _read_strategy_attr(block: Any) -> Any:
return None
def compose_root(config: Config) -> RuntimeRoot:
"""Compose the airborne runtime graph (per contract v1.0.0)."""
def compose_root(
config: Config,
*,
replay_components_factory: Any | None = None,
) -> RuntimeRoot:
"""Compose the airborne runtime graph for ``config.mode``.
With ``config.mode == "live"`` (the default) the function behaves
exactly as the pre-AZ-401 implementation — every wiring decision is
driven by ``config.components[slug].strategy`` against the strategy
registry, gated by the airborne tier.
With ``config.mode == "replay"`` the function additionally builds
the five replay-only strategies (``frame_source``, ``fc_adapter``,
``clock``, ``mavlink_transport``, ``replay_sink``) per
:mod:`._replay_branch` and merges them into the components dict
BEFORE the registry-driven C1-C7+C13 strategies run, so any
component factory that consumes one of the five via ``constructed``
finds it already populated. C1-C7+C13 strategies are wired
identically to live mode (replay protocol Invariant 1).
The ``replay_components_factory`` keyword is a test-only injection
point — production callers omit it. Tests pass a callable returning
``(components, construction_order)`` so the unit suite does not
have to satisfy the full OpenCV / pymavlink / FDR side-effects of
the real strategies.
"""
extra_env = (
("MAVLINK_SIGNING_KEY",)
if config.mode == "live"
else ()
)
if config.mode == "replay":
replay_factory = replay_components_factory or build_replay_components
replay_components, replay_order = replay_factory(config)
else:
replay_components = {}
replay_order = ()
components, order = _compose(
config,
binary="airborne",
allowed_tiers=frozenset({"airborne", "shared"}),
extra_required_env=("MAVLINK_SIGNING_KEY",),
extra_required_env=extra_env,
pre_constructed=replay_components,
)
merged: dict[str, Any] = dict(replay_components)
merged.update(components)
full_order = tuple(replay_order) + tuple(
slug for slug in order if slug not in replay_order
)
return RuntimeRoot(
binary="airborne",
profile=os.environ["GPS_DENIED_FC_PROFILE"],
components=components,
construction_order=order,
components=merged,
construction_order=full_order,
)
@@ -424,22 +490,6 @@ def compose_operator(config: Config) -> OperatorRoot:
)
def compose_replay(config: Config) -> RuntimeRoot:
"""Compose the replay-cli runtime graph. Concrete wiring is owned by AZ-401."""
components, order = _compose(
config,
binary="replay-cli",
allowed_tiers=frozenset({"airborne", "shared"}),
extra_required_env=(),
)
return RuntimeRoot(
binary="replay-cli",
profile=os.environ["GPS_DENIED_FC_PROFILE"],
components=components,
construction_order=order,
)
@dataclass(frozen=True)
class TakeoffResult:
"""Successful takeoff: writer is open, FC adapter is wired, components started.
@@ -0,0 +1,329 @@
"""Replay-mode branch of :func:`compose_root` (AZ-401 / E-DEMO-REPLAY).
Internal module. Owns the wiring that turns a ``config.mode == "replay"``
:class:`Config` into a :class:`RuntimeRoot` whose components dict carries
the replay-only strategies (``frame_source``, ``fc_adapter``, ``clock``,
``mavlink_transport``, ``replay_sink``) plus whatever C1-C7+C13 strategies
the binary's bootstrap registered against
:data:`gps_denied_onboard.runtime_root._STRATEGY_REGISTRY`.
Per replay protocol v2.0.0 (ADR-011): replay is a configuration of the
single airborne composition root, not a sibling root. The branch lives
in this module to keep ``runtime_root/__init__.py`` focused on the
shared composition spine while still exposing exactly one
``compose_root(config)`` entrypoint.
Build-flag gates (per replay protocol Invariant 9):
- ``BUILD_VIDEO_FILE_FRAME_SOURCE`` — required for the
:class:`VideoFileFrameSource` instance returned by the coordinator.
- ``BUILD_TLOG_REPLAY_ADAPTER`` — required for the
:class:`TlogReplayFcAdapter` instance returned by the coordinator.
- ``BUILD_REPLAY_SINK_JSONL`` — shared by the JSONL sink and the noop
outbound transport.
All three default ON in the airborne binary (per ADR-011); flipping any
OFF disables replay mode without affecting live mode.
"""
from __future__ import annotations
import json
import os
from collections.abc import Mapping
from pathlib import Path
from typing import TYPE_CHECKING, Any, Final
from gps_denied_onboard._types.calibration import CameraCalibration
from gps_denied_onboard._types.fc import FcKind
from gps_denied_onboard.components.c8_fc_adapter.noop_mavlink_transport import (
NoopMavlinkTransport,
)
from gps_denied_onboard.components.c8_fc_adapter.replay_sink import (
JsonlReplaySink,
)
from gps_denied_onboard.config import Config
from gps_denied_onboard.fdr_client import make_fdr_client
from gps_denied_onboard.helpers.wgs_converter import WgsConverter
from gps_denied_onboard.logging import get_logger
from gps_denied_onboard.replay_input import (
AutoSyncConfig,
ReplayInputAdapter,
ReplayInputBundle,
)
from gps_denied_onboard.replay_input.tlog_video_adapter import ReplayPace
if TYPE_CHECKING:
from gps_denied_onboard.fdr_client.client import FdrClient
__all__ = [
"REPLAY_BUILD_FLAGS",
"REPLAY_COMPONENT_KEYS",
"CompositionError",
"build_replay_components",
]
_LOG_KIND_READY: Final[str] = "replay.compose_root.ready"
REPLAY_BUILD_FLAGS: Final[tuple[str, ...]] = (
"BUILD_VIDEO_FILE_FRAME_SOURCE",
"BUILD_TLOG_REPLAY_ADAPTER",
"BUILD_REPLAY_SINK_JSONL",
)
REPLAY_COMPONENT_KEYS: Final[tuple[str, ...]] = (
"frame_source",
"fc_adapter",
"clock",
"mavlink_transport",
"replay_sink",
)
class CompositionError(RuntimeError):
"""Raised when the replay-mode branch refuses to compose a runtime.
Carries the human-readable reason (build-flag OFF, missing path,
contradictory config) so the caller can surface it in the structured
log + on stderr without a second introspection pass.
"""
def build_replay_components(
config: Config,
*,
fdr_client_factory: Any | None = None,
replay_input_adapter_factory: Any | None = None,
sink_factory: Any | None = None,
transport_factory: Any | None = None,
) -> tuple[dict[str, Any], tuple[str, ...]]:
"""Construct the replay-mode component dict + construction order.
The factories are test-only injection points. Production callers
(just ``compose_root``) leave them ``None`` so the real constructors
run; unit tests pass fakes so they don't have to satisfy the full
OpenCV / pymavlink / FDR side-effects of the real strategies.
Returns:
``(components, construction_order)`` — the same shape
:func:`gps_denied_onboard.runtime_root._compose` returns. The
keys are the entries of :data:`REPLAY_COMPONENT_KEYS`; the
values are typed strategy instances.
"""
if config.mode != "replay":
raise CompositionError(
"build_replay_components called with non-replay config "
f"(mode={config.mode!r})"
)
_validate_build_flags()
_validate_replay_paths(config)
fdr_factory = fdr_client_factory or make_fdr_client
fdr_client = fdr_factory("replay_input", config)
sink_fdr_client = fdr_factory("c8_fc_adapter.replay_sink", config)
bundle = _build_replay_input_bundle(
config,
fdr_client=fdr_client,
adapter_factory=replay_input_adapter_factory,
)
if sink_factory is not None:
sink = sink_factory(config, sink_fdr_client)
else:
sink = JsonlReplaySink(
output_path=Path(config.replay.output_path),
fdr_client=sink_fdr_client,
)
if transport_factory is not None:
transport = transport_factory(config)
else:
transport = NoopMavlinkTransport()
components: dict[str, Any] = {
"frame_source": bundle.frame_source,
"fc_adapter": bundle.fc_adapter,
"clock": bundle.clock,
"mavlink_transport": transport,
"replay_sink": sink,
}
_log_ready(config, bundle)
return components, REPLAY_COMPONENT_KEYS
def _validate_build_flags() -> None:
"""Refuse construction when any replay-mode ``BUILD_*`` flag is OFF."""
for flag_name in REPLAY_BUILD_FLAGS:
raw = os.environ.get(flag_name, "ON").strip().upper()
if raw == "OFF":
raise CompositionError(
f"{flag_name} is OFF; replay mode requires it"
)
def _validate_replay_paths(config: Config) -> None:
"""Reject empty / missing replay paths early with a precise message."""
if not config.replay.video_path:
raise CompositionError(
"config.replay.video_path is empty; replay mode requires a video path"
)
if not config.replay.tlog_path:
raise CompositionError(
"config.replay.tlog_path is empty; replay mode requires a tlog path"
)
if not config.replay.output_path:
raise CompositionError(
"config.replay.output_path is empty; replay mode requires an output path"
)
def _build_replay_input_bundle(
config: Config,
*,
fdr_client: "FdrClient",
adapter_factory: Any | None,
) -> ReplayInputBundle:
"""Build the :class:`ReplayInputAdapter` and call ``open()``."""
pace = _resolve_pace(config.replay.pace)
target_fc_dialect = _resolve_fc_kind(config.replay.target_fc_dialect)
auto_sync = _build_auto_sync_config(config)
camera_calibration = _load_camera_calibration(config)
wgs_converter = WgsConverter()
if adapter_factory is not None:
adapter = adapter_factory(
config=config,
camera_calibration=camera_calibration,
target_fc_dialect=target_fc_dialect,
wgs_converter=wgs_converter,
fdr_client=fdr_client,
pace=pace,
auto_sync_config=auto_sync,
)
else:
adapter = ReplayInputAdapter(
video_path=Path(config.replay.video_path),
tlog_path=Path(config.replay.tlog_path),
camera_calibration=camera_calibration,
target_fc_dialect=target_fc_dialect,
wgs_converter=wgs_converter,
fdr_client=fdr_client,
pace=pace,
manual_time_offset_ms=config.replay.time_offset_ms,
auto_sync_config=auto_sync,
)
return adapter.open()
def _resolve_pace(raw: str) -> ReplayPace:
if raw == "asap":
return ReplayPace.ASAP
if raw == "realtime":
return ReplayPace.REALTIME
raise CompositionError(
f"config.replay.pace={raw!r} not in ('asap', 'realtime')"
)
def _resolve_fc_kind(raw: str) -> FcKind:
if raw == "ardupilot_plane":
return FcKind.ARDUPILOT_PLANE
if raw == "inav":
return FcKind.INAV
raise CompositionError(
f"config.replay.target_fc_dialect={raw!r} not in "
"('ardupilot_plane', 'inav')"
)
def _build_auto_sync_config(config: Config) -> AutoSyncConfig:
block = config.replay.auto_sync
return AutoSyncConfig(
takeoff_accel_threshold_g=block.takeoff_accel_threshold_g,
takeoff_attitude_rate_threshold_rad_s=(
block.takeoff_attitude_rate_threshold_rad_s
),
sustained_seconds=block.sustained_seconds,
prescan_max_messages=block.prescan_max_messages,
video_motion_threshold=block.video_motion_threshold,
video_motion_scan_seconds=block.video_motion_scan_seconds,
match_threshold_pct=block.match_threshold_pct,
match_window_ms=block.match_window_ms,
low_confidence_threshold=block.low_confidence_threshold,
)
def _load_camera_calibration(config: Config) -> CameraCalibration:
"""Read the camera calibration JSON into a :class:`CameraCalibration` DTO.
The replay binary uses the SAME calibration file the live binary
loads; AZ-401 does not introduce a new on-disk format.
"""
import numpy as np
path = config.runtime.camera_calibration_path
if not path:
raise CompositionError(
"config.runtime.camera_calibration_path is empty; replay mode "
"requires a camera calibration JSON"
)
try:
blob = json.loads(Path(path).read_text(encoding="utf-8"))
except OSError as exc:
raise CompositionError(
f"failed to read camera calibration from {path!r}: {exc!r}"
) from exc
except json.JSONDecodeError as exc:
raise CompositionError(
f"camera calibration {path!r} is not valid JSON: {exc!r}"
) from exc
if not isinstance(blob, Mapping):
raise CompositionError(
f"camera calibration {path!r} must decode to a mapping; "
f"got {type(blob).__name__}"
)
intrinsics = np.asarray(blob.get("intrinsics_3x3"), dtype=np.float64)
if intrinsics.shape != (3, 3):
raise CompositionError(
f"camera calibration {path!r} 'intrinsics_3x3' must be 3x3; "
f"got shape {intrinsics.shape}"
)
distortion = np.asarray(blob.get("distortion", []), dtype=np.float64)
body_to_camera = np.asarray(
blob.get("body_to_camera_se3", np.eye(4).tolist()),
dtype=np.float64,
)
return CameraCalibration(
camera_id=str(blob.get("camera_id", "replay-camera")),
intrinsics_3x3=intrinsics,
distortion=distortion,
body_to_camera_se3=body_to_camera,
acquisition_method=str(blob.get("acquisition_method", "operator")),
metadata=dict(blob.get("metadata", {})),
)
def _log_ready(config: Config, bundle: ReplayInputBundle) -> None:
log = get_logger("runtime_root.replay_branch")
log.info(
f"{_LOG_KIND_READY}: pace={config.replay.pace} "
f"resolved_offset_ms={bundle.resolved_time_offset_ms}",
extra={
"kind": _LOG_KIND_READY,
"kv": {
"video_path": config.replay.video_path,
"tlog_path": config.replay.tlog_path,
"output_path": config.replay.output_path,
"pace": config.replay.pace,
"resolved_offset_ms": bundle.resolved_time_offset_ms,
"calib_path": config.runtime.camera_calibration_path,
"auto_sync_used": bundle.auto_sync_result is not None,
},
},
)