Files
Oleksandr Bezdieniezhnykh c3639a5d1c [AZ-624] [AZ-618] Phase F: wire build_pre_constructed into main()
Wire register_airborne_strategies + build_pre_constructed +
compose_root(config, pre_constructed=...) into runtime_root.main(). The
existing exception block now catches AirborneBootstrapError distinctly
before the broader (ConfigurationError, StrategyNotLinkedError,
RuntimeError) clause so the operator-facing "airborne_bootstrap:"
prefix carried by every bootstrap error reaches stderr cleanly with
EXIT_GENERIC_FAILURE rather than getting absorbed into a generic
backtrace.

This closes the AZ-618 umbrella: AZ-619..AZ-623 + AZ-625 had built
each pre_constructed key; this batch lands the integration that the
production main() actually invokes them. Both the live
gps-denied-onboard and replay gps-denied-replay binaries dispatch
through this main() per ADR-011, so both reach takeoff with
pre_constructed populated end-to-end.

Tests: tests/unit/runtime_root/test_az618_pre_constructed.py adds 6
tests covering AC-618-1..AC-618-4 + AZ-624 local handler-ordering
regression guard. The strategy factories are stubbed at the
airborne_bootstrap module boundary so the test exercises the
integration seam without standing up gtsam / FAISS / TensorRT /
PyTorch / OpenCV at unit-test scope.

AC-618-5 (Jetson tier-2 e2e) is BLOCKED on operator-supplied hardware
evidence: scripts/run-tests-jetson.sh
tests/e2e/replay/test_derkachi_1min.py must run on Jetson Orin Nano
(JetPack 6.2.2+b24) and the terminal log path + JetPack version + run
timestamp captured per _docs/02_document/tests/tier2-jetson-testing.md.

Quality gates: ruff format clean, ruff lint clean, 6/6 new umbrella
tests pass, 261/261 runtime_root + c5_state regression suite passes,
25/25 test_az401_compose_root_replay regression passes, full Tier-1
unit suite 2150/2151 passes (1 unrelated pre-existing failure:
c12_operator_orchestrator subprocess cold-start NFR fails on Mac dev
host's Python startup ~700 ms; not regressed by AZ-624). Code review
verdict PASS (1 Low finding; full report in
_docs/03_implementation/reviews/batch_96_review.md).

Archives AZ-624 task spec + AZ-618 umbrella reference to done/.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 10:28:43 +03:00

484 lines
17 KiB
Python

"""AZ-624 / AZ-618 — umbrella ACs for ``runtime_root.main()`` + ``build_pre_constructed`` wiring.
Verifies the full AZ-618 acceptance suite (AC-1..AC-4) at the
integration seam, plus AZ-624's local ACs:
* AC-618-1: ``build_pre_constructed(config)`` returns a dict containing
every key in :data:`AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS`
flattened (no key maps to ``None``).
* AC-618-2: ``compose_root(config, pre_constructed=...)`` reaches
takeoff and returns a :class:`RuntimeRoot` whose ``components`` dict
contains all 7 strategy-selecting slots (c1_vio, c2_vpr, c2_5_rerank,
c3_matcher, c3_5_adhop, c4_pose, c5_state) without raising
:class:`AirborneBootstrapError`.
* AC-618-3: a ``BUILD_*`` flag mismatch surfaces an
:class:`AirborneBootstrapError` whose message names both the
missing infrastructure key and the gating ``BUILD_*`` flag plus the
consuming component slug.
* AC-618-4: ``runtime_root.main()`` returns ``0`` (success) when all
infra deps resolve; returns :data:`EXIT_GENERIC_FAILURE` (``1``) and
stderr contains the ``airborne_bootstrap:`` prefix when a single
required infra dep is forcibly unavailable.
AC-618-5 (Jetson tier-2 e2e replay run) is verified out-of-band on
real hardware via ``scripts/run-tests-jetson.sh
tests/e2e/replay/test_derkachi_1min.py``; this unit-test file owns
only AC-1..AC-4.
The phase-specific tests (``test_az619..test_az625``) already cover
each builder's contract in isolation. This file owns the integrated
contract: every phase builds something usable + the assembly composes.
The heavy-builder stubs are NOT autouse: tests that need to exercise
the real builder path (AC-618-3 against the C7 factory's flag-OFF
branch) opt out by simply not calling :func:`_stub_heavy_builders`.
"""
from __future__ import annotations
from collections.abc import Iterator
from typing import Any
from unittest.mock import MagicMock
import pytest
from gps_denied_onboard.config import Config
from gps_denied_onboard.fdr_client import client as fdr_client_module
from gps_denied_onboard.runtime_root import (
EXIT_GENERIC_FAILURE,
RuntimeRoot,
airborne_bootstrap,
clear_strategy_registry,
compose_root,
main,
)
from gps_denied_onboard.runtime_root.airborne_bootstrap import (
AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS,
AirborneBootstrapError,
build_pre_constructed,
clear_imu_preintegrator_cache,
register_airborne_strategies,
)
# ----------------------------------------------------------------------
# Shared fixtures
@pytest.fixture(autouse=True)
def _isolated_caches() -> Iterator[None]:
# Arrange: every test starts with empty FdrClient + ImuPreintegrator
# caches + an empty strategy registry so the AZ-624 assertions are
# exercised against fresh state rather than carry-over from prior
# runs in the same pytest process.
fdr_client_module._reset_for_tests()
clear_imu_preintegrator_cache()
clear_strategy_registry()
yield
fdr_client_module._reset_for_tests()
clear_imu_preintegrator_cache()
clear_strategy_registry()
def _stub_heavy_builders(monkeypatch: pytest.MonkeyPatch) -> None:
"""Replace each heavy AZ-619..AZ-625 builder with an opaque sentinel.
Called explicitly from each test that needs the integrated
``build_pre_constructed`` to succeed without standing up FAISS,
TensorRT, PyTorch, OpenCV, or gtsam at unit-test scope. The
AC-618-3 path deliberately skips this so the real C7 builder's
``BUILD_*`` flag-OFF branch fires.
"""
monkeypatch.setattr(
airborne_bootstrap,
"_build_c6_descriptor_index",
lambda _config: MagicMock(name="DescriptorIndex"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c6_tile_store",
lambda _config: MagicMock(name="TileStore"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c7_inference",
lambda _config: MagicMock(name="InferenceRuntime"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c3_lightglue_runtime",
lambda _config, *, inference_runtime: MagicMock(name="LightGlueRuntime"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c3_feature_extractor",
lambda _config: MagicMock(name="FeatureExtractor"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c282_ransac_filter",
lambda _config: MagicMock(name="RansacFilter"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c5_imu_preintegrator",
lambda _config: MagicMock(name="ImuPreintegrator"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c5_se3_utils",
lambda _config: MagicMock(name="Se3Utils"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c5_wgs_converter",
lambda _config: MagicMock(name="WgsConverter"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c5_state_estimator_pair",
lambda *_args, **_kwargs: (
MagicMock(name="StateEstimator"),
MagicMock(name="ISam2GraphHandle"),
),
)
def _airborne_env(monkeypatch: pytest.MonkeyPatch) -> None:
"""Set the env vars compose_root requires for the airborne binary.
Mirrors ``tests/unit/test_az401_compose_root_replay.py``'s
``_airborne_live_env`` fixture.
"""
for name, value in (
("GPS_DENIED_FC_PROFILE", "ardupilot_plane"),
("GPS_DENIED_TIER", "1"),
("DB_URL", "postgresql+psycopg://gps_denied:dev@db:5432/gps_denied"),
("CAMERA_CALIBRATION_PATH", "/etc/gps-denied/calib.yml"),
("LOG_LEVEL", "INFO"),
("LOG_SINK", "console"),
("INFERENCE_BACKEND", "pytorch_fp16"),
("FDR_PATH", "/var/lib/gps-denied/fdr"),
("TILE_CACHE_PATH", "/var/lib/gps-denied/tiles"),
("MAVLINK_SIGNING_KEY", "ZZZZZZZZ"),
):
monkeypatch.setenv(name, value)
def _all_seven_components_config() -> Config:
"""Return a :class:`Config` whose ``components`` selects every default strategy.
Each block uses the ``mapping``-with-``"strategy"``-key shape that
:func:`runtime_root._resolve_component_strategies` accepts as a
fallback for raw YAML — keeps the test free of seven imports of
per-component config block dataclasses.
"""
return Config.with_blocks(
c1_vio={"strategy": "klt_ransac"},
c2_vpr={"strategy": "ultra_vpr"},
c2_5_rerank={"strategy": "inlier_count"},
c3_matcher={"strategy": "disk_lightglue"},
c3_5_adhop={"strategy": "adhop"},
c4_pose={"strategy": "opencv_gtsam"},
c5_state={"strategy": "gtsam_isam2"},
)
def _stub_strategy_factories(monkeypatch: pytest.MonkeyPatch) -> None:
"""Stub each per-component factory the airborne wrappers delegate to.
The wrappers in ``airborne_bootstrap._c{N}_*_wrapper`` import each
factory by name from its sibling ``runtime_root.*_factory`` module.
Replacing those names in ``airborne_bootstrap``'s own namespace
makes the wrappers hand back deterministic sentinels instead of
constructing real OpenCV / gtsam / TensorRT objects.
"""
monkeypatch.setattr(
airborne_bootstrap,
"build_vio_strategy",
lambda _config, **_kwargs: MagicMock(name="VioStrategy"),
)
monkeypatch.setattr(
airborne_bootstrap,
"build_vpr_strategy",
lambda _config, **_kwargs: MagicMock(name="VprStrategy"),
)
monkeypatch.setattr(
airborne_bootstrap,
"build_rerank_strategy",
lambda _config, **_kwargs: MagicMock(name="RerankStrategy"),
)
monkeypatch.setattr(
airborne_bootstrap,
"build_matcher_strategy",
lambda _config, **_kwargs: MagicMock(name="MatcherStrategy"),
)
monkeypatch.setattr(
airborne_bootstrap,
"build_refiner_strategy",
lambda _config, **_kwargs: MagicMock(name="RefinerStrategy"),
)
monkeypatch.setattr(
airborne_bootstrap,
"build_pose_estimator",
lambda _config, **_kwargs: MagicMock(name="PoseEstimator"),
)
# AZ-625 short-circuits the C5 wrapper on _c5_prebuilt_estimator,
# so build_state_estimator is never reached on the success path.
# The _stub_heavy_builders stub already seeds a MagicMock estimator
# under the look-aside key.
# ----------------------------------------------------------------------
# AC-618-1
def test_ac_618_1_build_pre_constructed_populates_every_required_key(
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Arrange
_airborne_env(monkeypatch)
_stub_heavy_builders(monkeypatch)
config = Config()
expected_keys: set[str] = set().union(*AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS.values())
# Act
pre_constructed = build_pre_constructed(config)
# Assert: every documented public key is present and non-None.
missing = expected_keys - pre_constructed.keys()
assert not missing, (
f"build_pre_constructed must populate every key in "
f"AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS; missing: {sorted(missing)}"
)
for key in expected_keys:
assert pre_constructed[key] is not None, (
f"pre_constructed[{key!r}] is None; the AC-618-1 contract "
f"requires every required key to map to a real instance"
)
# ----------------------------------------------------------------------
# AC-618-2
def test_ac_618_2_compose_root_reaches_takeoff_with_all_seven_slots(
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Arrange
_airborne_env(monkeypatch)
_stub_heavy_builders(monkeypatch)
_stub_strategy_factories(monkeypatch)
register_airborne_strategies()
config = _all_seven_components_config()
pre_constructed = build_pre_constructed(config)
# Act
runtime = compose_root(config, pre_constructed=pre_constructed)
# Assert: every strategy-selecting slot produced a component (via
# the stubbed factory). compose_root returns
# registry-built-only components in `RuntimeRoot.components` for
# live mode (replay components would be merged in by replay-branch).
assert isinstance(runtime, RuntimeRoot)
expected_slots: set[str] = {
"c1_vio",
"c2_vpr",
"c2_5_rerank",
"c3_matcher",
"c3_5_adhop",
"c4_pose",
"c5_state",
}
missing_slots = expected_slots - runtime.components.keys()
assert not missing_slots, (
f"compose_root must populate every registered airborne slot; "
f"missing: {sorted(missing_slots)}"
)
# ----------------------------------------------------------------------
# AC-618-3
def test_ac_618_3_build_flag_off_raises_named_error_with_consuming_component(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""When the C7 inference factory's gating flags are OFF and a
consumer is configured, ``build_pre_constructed`` must raise
:class:`AirborneBootstrapError` whose message names BOTH the
missing infrastructure key (``c7_inference``) AND the airborne
BUILD_* flags (``BUILD_TENSORRT_RUNTIME``,
``BUILD_PYTORCH_FP16_RUNTIME``) AND the consuming component slug
(``c2_vpr``).
Stubs only the upstream builders the C7 factory does not depend
on — leaves the real ``_build_c7_inference`` in place so the
production C7 ``RuntimeNotAvailableError`` path fires. Stubs the
underlying :func:`build_inference_runtime` factory to simulate the
flag-OFF condition without depending on env-var defaults.
"""
# Arrange
_airborne_env(monkeypatch)
# Stub upstream builders the C6 / FdrClient layer needs so we
# reach the C7 builder in normal sequence; do NOT stub the C7
# builder itself — that's the path under test.
monkeypatch.setattr(
airborne_bootstrap,
"_build_c6_descriptor_index",
lambda _config: MagicMock(name="DescriptorIndex"),
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c6_tile_store",
lambda _config: MagicMock(name="TileStore"),
)
# Force the C7 factory's "no airborne runtime buildable" branch via
# the same RuntimeNotAvailableError shape it raises in production
# when both flags resolve to OFF.
from gps_denied_onboard.runtime_root.errors import RuntimeNotAvailableError
def _raise_no_airborne_c7_runtime(_config: Any) -> Any:
raise RuntimeNotAvailableError(
"no airborne C7 inference runtime is buildable; both "
"BUILD_TENSORRT_RUNTIME and BUILD_PYTORCH_FP16_RUNTIME are OFF"
)
monkeypatch.setattr(
airborne_bootstrap,
"build_inference_runtime",
_raise_no_airborne_c7_runtime,
)
config = Config.with_blocks(c2_vpr={"strategy": "net_vlad"})
# Act + Assert
with pytest.raises(AirborneBootstrapError) as excinfo:
build_pre_constructed(config)
message = str(excinfo.value)
assert "c7_inference" in message
# Both airborne C7 flags must be named in the operator-facing error
# (per :data:`C7_AIRBORNE_BUILD_FLAGS` — the operator sees both the
# production-default and the Tier-0 fallback options).
assert "BUILD_TENSORRT_RUNTIME" in message
assert "BUILD_PYTORCH_FP16_RUNTIME" in message
# The consuming component is named so the operator knows which
# config block to revisit.
assert "c2_vpr" in message
# ----------------------------------------------------------------------
# AC-618-4
def test_ac_618_4_main_returns_zero_on_success(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
# Arrange
_airborne_env(monkeypatch)
_stub_heavy_builders(monkeypatch)
_stub_strategy_factories(monkeypatch)
config = _all_seven_components_config()
# Act: main() drives register_airborne_strategies +
# build_pre_constructed + compose_root in sequence. The stubs
# cover the heavy seams.
exit_code = main(config)
# Assert
assert exit_code == 0
# The success path writes nothing to stderr (operator contract:
# stderr is ONLY the failure surface for runtime_root).
captured = capsys.readouterr()
assert "airborne_bootstrap:" not in captured.err
def test_ac_618_4_main_returns_generic_failure_with_airborne_bootstrap_prefix(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Force one infra builder to raise :class:`AirborneBootstrapError` and
assert main() surfaces it cleanly:
* exit code is :data:`EXIT_GENERIC_FAILURE` (the AC-618-4 contract).
* stderr contains the ``airborne_bootstrap:`` prefix the bootstrap
error message itself emits — confirms main() now catches
:class:`AirborneBootstrapError` distinctly (rather than letting
the broader :class:`RuntimeError` clause hide the prefix in a
generic backtrace).
"""
# Arrange
_airborne_env(monkeypatch)
_stub_heavy_builders(monkeypatch)
_stub_strategy_factories(monkeypatch)
def _raise_named_bootstrap_error(_config: Any) -> Any:
raise AirborneBootstrapError(
"airborne_bootstrap: cannot construct "
"pre_constructed['c6_descriptor_index'] because "
"BUILD_FAISS_INDEX is OFF (forced via test). "
"Consuming component: c2_vpr."
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c6_descriptor_index",
_raise_named_bootstrap_error,
)
config = _all_seven_components_config()
# Act
exit_code = main(config)
# Assert
assert exit_code == EXIT_GENERIC_FAILURE
captured = capsys.readouterr()
assert "airborne_bootstrap:" in captured.err, (
"main() must surface AirborneBootstrapError messages with the "
"documented prefix to stderr; got: " + repr(captured.err)
)
assert "c6_descriptor_index" in captured.err
assert "c2_vpr" in captured.err
def test_ac_624_local_main_distinct_handler_does_not_double_print(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Regression guard: after AZ-624 added a dedicated
:class:`AirborneBootstrapError` handler before the broader
``(ConfigurationError, StrategyNotLinkedError, RuntimeError)``
clause, the same exception must NOT also fall through to that
second handler (which would emit a duplicate stderr line).
Verifies the catch ordering is a strict short-circuit.
"""
# Arrange
_airborne_env(monkeypatch)
_stub_heavy_builders(monkeypatch)
_stub_strategy_factories(monkeypatch)
def _raise_bootstrap_error(_config: Any) -> Any:
raise AirborneBootstrapError(
"airborne_bootstrap: cannot construct "
"pre_constructed['c5_imu_preintegrator'] because forced via test. "
"Consuming component: c5_state."
)
monkeypatch.setattr(
airborne_bootstrap,
"_build_c5_imu_preintegrator",
_raise_bootstrap_error,
)
config = _all_seven_components_config()
# Act
exit_code = main(config)
captured = capsys.readouterr()
# Assert
assert exit_code == EXIT_GENERIC_FAILURE
# Exactly ONE "runtime_root: airborne_bootstrap:" line — the
# dedicated handler short-circuits before the RuntimeError clause.
occurrences = captured.err.count("runtime_root: airborne_bootstrap:")
assert occurrences == 1, (
f"Expected exactly one runtime_root: airborne_bootstrap: line in "
f"stderr, got {occurrences}. Full stderr: {captured.err!r}"
)