[AZ-895] Deprecate replay auto-sync surface; file AZ-908 follow-up

Option A (minimum-deprecation, 2 SP) per user complexity-budget
decision. Auto-sync stays importable as a raising stub for one cycle
so external callers see a clean ReplayInputAdapterError instead of an
ImportError. Full physical removal is filed as AZ-908 (cycle-5+ backlog).

Production:
- auto_sync.py: 700+ LOC -> 56-line no-op stub raising
  "auto-sync removed; supply --imu CSV instead"
- tlog_video_adapter.py: 700+ LOC -> 105-line deprecated stub;
  ReplayInputAdapter.open() raises immediately, close() is a no-op
- _replay_branch.py: dropped legacy auto-sync branch +
  _build_auto_sync_config; _validate_replay_paths now requires
  imu_csv_path; replay_input_adapter_factory parameter removed
- cli/replay.py: --time-offset-ms / --skip-auto-sync / --auto-trim
  emit DeprecationWarning + stderr line; values ignored
- tlog_replay_adapter.py + tlog_ground_truth.py docstrings: AUDIT-ONLY

Tests:
- DELETED test_az405_auto_sync, test_az405_replay_input_adapter,
  test_az698_window_alignment (covered code no longer runs)
- ADDED test_az895_auto_sync_deprecated_stub (5 parametrised, pins AC-1)
- test_az402_replay_cli: deprecation warnings + ignored-value asserts
- test_az401_compose_root_replay: new imu_csv_path-required gate;
  deleted the calibration-loading test that relied on the removed
  replay_input_adapter_factory injection point
- test_derkachi_real_tlog: xfail reason refreshed to AZ-848 + AZ-883
  (AC-4 "AZ-848-scoped reason")

Docs:
- module-layout.md: replay_input file list flags deprecated modules,
  adds csv_ground_truth.py
- _dependencies_table.md: +AZ-908 row, preamble + totals updated
  (179 -> 180 tasks, 567 -> 570 SP)
- AZ-908 backlog spec added; AZ-895 spec moved todo -> done
- batch_03_cycle4_report.md written

Touched-module tests green (111 passed, 1 skipped). Full unit suite
green: 2287 passed, 85 skipped, 1 deselected (pre-existing flaky perf
test, unrelated).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-26 22:09:59 +03:00
parent fdb593a775
commit 007aa36fbf
19 changed files with 600 additions and 4213 deletions
+7 -6
View File
@@ -395,13 +395,14 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec
### shared/replay_input ### shared/replay_input
- **Directory**: `src/gps_denied_onboard/replay_input/` - **Directory**: `src/gps_denied_onboard/replay_input/`
- **Purpose**: Layer-4 cross-cutting coordinator that converges `(video, tlog)` inputs into the standard `FrameSource` + `FcAdapter` + `Clock` surfaces the airborne composition root consumes. Owns the time-alignment between video frames and tlog IMU/attitude ticks (manual via `--time-offset-ms` or automatic via the AZ-405 IMU-take-off detector). The composition root, in replay mode, builds a `ReplayInputAdapter`, calls `.open()`, and wires the returned `ReplayInputBundle` into the same C1C5 pipeline as live. New under ADR-011 (replaces the v1.0.0 design where replay was a separate composition root). - **Purpose**: Layer-4 cross-cutting coordinator. Under AZ-894 the production replay pipeline drives off the operator's IMU+GPS CSV via `CsvReplayFcAdapter`. The legacy `(video, tlog)` auto-sync surface was deprecated by AZ-895 and will be physically removed by AZ-908. The composition root, in replay mode, builds the CSV bundle (frame source + CSV FC adapter + clock) and wires the returned `ReplayInputBundle` into the same C1C5 pipeline as live. New under ADR-011 (replaces the v1.0.0 design where replay was a separate composition root).
- `__init__.py` (re-exports `ReplayInputAdapter`, `ReplayInputBundle`, `AutoSyncDecision`, `AutoSyncConfig`, `ReplayInputAdapterError`, plus the AZ-697 / AZ-836 surfaces: `TlogGpsFix`, `TlogGroundTruth`, `load_tlog_ground_truth`, `RouteSpec`, `RouteExtractionError`, `extract_route_from_tlog`) - `__init__.py` (re-exports `ReplayInputAdapter`, `ReplayInputBundle`, `AutoSyncDecision`, `AutoSyncConfig`, `ReplayInputAdapterError`, `CsvGpsFix`, `CsvGroundTruth`, `load_csv_ground_truth`, plus the AZ-697 / AZ-836 surfaces: `TlogGpsFix`, `TlogGroundTruth`, `load_tlog_ground_truth`, `RouteSpec`, `RouteExtractionError`, `extract_route_from_tlog`)
- `interface.py` (`ReplayInputAdapter` class declaration + `ReplayInputBundle` DTO + `AlignedWindow` / `AutoSyncConfig` / `AutoSyncDecision` DTOs) - `csv_ground_truth.py` (AZ-894 — `load_csv_ground_truth` + `CsvGpsFix` / `CsvGroundTruth`; the canonical replay ground-truth surface)
- `interface.py` (`ReplayInputAdapter` class declaration + `ReplayInputBundle` DTO + `AlignedWindow` / `AutoSyncConfig` / `AutoSyncDecision` DTOs — the auto-sync DTOs are deprecated by AZ-895 and slated for removal in AZ-908)
- `errors.py` (AZ-405 — `ReplayInputAdapterError` envelope; subclass of `RuntimeError` so the airborne main maps every coordinator-scope failure to CLI exit code 2) - `errors.py` (AZ-405 — `ReplayInputAdapterError` envelope; subclass of `RuntimeError` so the airborne main maps every coordinator-scope failure to CLI exit code 2)
- `tlog_video_adapter.py` (concrete `ReplayInputAdapter` that instantiates `VideoFileFrameSource` + `TlogReplayFcAdapter` + chosen `Clock`) - `tlog_video_adapter.py` **DEPRECATED (AZ-895)**: `ReplayInputAdapter.open()` raises `ReplayInputAdapterError`; retained as an import-stable stub for one cycle. AZ-908 removes it.
- `auto_sync.py` (AZ-405 IMU-take-off / video-motion-onset detectors + combined offset computation + AC-8 frame-window-match validator) - `auto_sync.py` **DEPRECATED (AZ-895)**: every detector + validator raises `ReplayInputAdapterError`; retained as an import-stable stub for one cycle. AZ-908 removes it.
- `tlog_ground_truth.py` (AZ-697 — `load_tlog_ground_truth` + `TlogGpsFix` / `TlogGroundTruth` for direct binary tlog GPS-truth extraction; consumed by `helpers.gps_compare` and `tlog_route.py`) - `tlog_ground_truth.py` (AZ-697 — `load_tlog_ground_truth` + `TlogGpsFix` / `TlogGroundTruth` for direct binary tlog GPS-truth extraction; AUDIT-ONLY after AZ-895, retained for the AZ-699 / AZ-701 validation paths against legacy `.tlog` archives)
- `tlog_route.py` (AZ-836 — `extract_route_from_tlog` + `RouteExtractionError`; re-exports `RouteSpec` from `_types.route`. Reduces a tlog to a coarsened route via Douglas-Peucker on local ENU; consumed by `c11_tile_manager.route_client.SatelliteProviderRouteClient.seed_route`) - `tlog_route.py` (AZ-836 — `extract_route_from_tlog` + `RouteExtractionError`; re-exports `RouteSpec` from `_types.route`. Reduces a tlog to a coarsened route via Douglas-Peucker on local ENU; consumed by `c11_tile_manager.route_client.SatelliteProviderRouteClient.seed_route`)
- `tests/` - `tests/`
- **Owned by**: AZ-265 (E-DEMO-REPLAY) — task AZ-405 (auto-sync + coordinator). - **Owned by**: AZ-265 (E-DEMO-REPLAY) — task AZ-405 (auto-sync + coordinator).
File diff suppressed because one or more lines are too long
@@ -0,0 +1,59 @@
# Replay: hard removal of deprecated auto-sync surface (AZ-895 follow-up)
**Task**: AZ-908_replay_auto_sync_hard_removal
**Name**: Cycle-5+ cleanup that physically removes the auto-sync surface AZ-895 deprecated
**Description**: Follow-up to AZ-895 (cycle 4). AZ-895 made the auto_sync surface a no-op and deprecated the CLI flags (`--time-offset-ms`, `--skip-auto-sync`, `--auto-trim`) with one-cycle warnings, but left the call sites, config fields, and interface DTOs intact for backward compat. AZ-908 completes the removal in cycle 5+ after a one-cycle deprecation window has passed.
**Complexity**: 3 SP
**Dependencies**: AZ-895 (hard — must ship first; AZ-908 removes what AZ-895 deprecated), AZ-842 (hard — replay protocol docs coordinate)
**Component**: replay_input (auto_sync.py + tlog_video_adapter.py + interface.py), cli/replay, runtime_root/_replay_branch + runtime_root/__init__, config/schema + config/loader + config/__init__, replay_api/app
**Tracker**: AZ-908 (https://denyspopov.atlassian.net/browse/AZ-908)
**Parent Epic**: (none — cycle-4 replay-input redesign follow-up)
## Why
Auto-sync surface is dead in production code: AZ-894 (cycle 4) made the CSV-driven path mandatory via required `--imu`, and AZ-895 (cycle 4) deprecated the surface. After one cycle's deprecation window the deprecation warnings should fire in real CI runs (if any operator scripts still pass the deprecated flags); that surface area can then be removed without breaking anyone.
## Touch list (production)
- DELETE `src/gps_denied_onboard/replay_input/auto_sync.py` (currently a no-op stub from AZ-895)
- DELETE `src/gps_denied_onboard/replay_input/tlog_video_adapter.py` (currently a deprecated coordinator from AZ-895)
- Drop `AutoSyncConfig`, `AutoSyncDecision`, `AlignedWindow` DTOs from `replay_input/interface.py`. Drop `auto_sync_result` + `aligned_window` fields from `ReplayInputBundle`.
- Drop `--time-offset-ms`, `--skip-auto-sync`, `--auto-trim` CLI flags from `cli/replay.py` entirely
- Drop `ReplayConfig.time_offset_ms`, `.skip_auto_sync_validation`, `.auto_trim`, `.auto_sync` from `config/schema.py`. Drop `ReplayAutoSyncConfig` class.
- Drop `REPLAY_TIME_OFFSET_MS` env var + `auto_sync` block handling from `config/loader.py`
- Update `runtime_root/_replay_branch.py` to drop any lingering imports / dead code
- Update `runtime_root/__init__.py` if it references removed symbols
- Update `replay_api/app.py` if it references removed symbols
- Update `e2e/fixtures/sitl_replay_builder/builder.py` if it references removed symbols
## Touch list (tests)
- Delete remaining auto-sync test residue (no-op stub tests from AZ-895)
- Update CLI tests to drop deprecated-flag assertions (the flags no longer exist)
- Confirm `test_az401_compose_root_replay.py` is clean
## Touch list (docs)
- Update `_docs/02_document/module-layout.md` replay_input file list — remove deleted entries
- Update `_docs/02_document/contracts/replay/replay_protocol.md` — remove auto-sync surface narrative (coordinate with AZ-842)
- Update `_docs/02_document/contracts/replay/csv_replay_format.md` cross-references
## Acceptance Criteria
- **AC-1**: All files listed under "DELETE" above are removed from the workspace
- **AC-2**: Unit tests pass with no auto-sync, AutoSyncConfig, AutoSyncDecision, or AlignedWindow symbols in `src/gps_denied_onboard/**`
- **AC-3**: CLI `--help` output does not mention `--time-offset-ms`, `--skip-auto-sync`, or `--auto-trim`
- **AC-4**: `_docs/02_document/module-layout.md` does not mention `auto_sync.py` or `tlog_video_adapter.py`
- **AC-5**: `tests/e2e/replay/test_derkachi_real_tlog.py` continues to `@xfail` with AZ-848-scoped reason
## Out of scope
- AZ-848 / AZ-883 structural fix (tlog clock bug) — unchanged from AZ-895
- Replacing the deprecated coordinator with something else — the CSV path is the replacement (see `_replay_branch._build_csv_bundle`)
## References
- Companion in cycle 4: AZ-894 (CSV adapter), AZ-895 (deprecation)
- Decision audit trail: this file + AZ-895 batch_03_cycle4_report.md
- User decision 2026-05-26 (cycle-4 /autodev batch 3): chose Option A (light deprecation now, file AZ-908 for hard removal in cycle 5+) over Option B (full removal in cycle 4).
@@ -0,0 +1,207 @@
# Batch Report — cycle 4, batch 03
**Batch**: 03
**Cycle**: 4
**Tasks**: AZ-895
**Total complexity**: 2 SP
**Date**: 2026-05-26
**Commit**: pending (this batch)
## Task Selection
AZ-895 (deprecate `auto_sync` surface) ships solo. It is the natural
follow-up to AZ-894 (CSV adapter, batch 02): now that the CSV-driven
path is the primary replay surface, the legacy tlog auto-sync
infrastructure can be retired. Per `_dependencies_table.md`, AZ-895
has a hard dependency on AZ-894 which closed in batch 02.
### Complexity-budget user decision (Option A — minimum)
A naïve full removal of the auto-sync surface would have touched:
- 4 production modules: `auto_sync.py` (delete), `tlog_video_adapter.py`
(delete), `interface.py` (drop AutoSync DTOs + `ReplayInputBundle`
field), `_replay_branch.py` (strip legacy branch + `_build_auto_sync_config`)
- 3 config files: `config/schema.py` (drop `ReplayConfig` auto-sync
fields + the `ReplayAutoSyncConfig` class), `config/loader.py` (drop
`REPLAY_TIME_OFFSET_MS` env + auto_sync block handler),
`config/__init__.py` (drop re-exports)
- 1 CLI: drop the three deprecated flags entirely
- 4 test files needing rewrite or deletion
- Cascading docs in `replay_protocol.md` (AZ-842 sibling work)
Estimated at 45 SP, well over the ticket's 2 SP budget. Per
`meta-rule.mdc` Complexity Budget Check, the user was offered four
options and chose **Option A — minimum**:
> "Minimum (~2 SP): no-op-stub auto_sync.py (raises documented error),
> strip tlog_video_adapter.open() to raise too, drop the unreachable
> legacy branch in _replay_branch.py, deprecate --time-offset-ms /
> --skip-auto-sync / --auto-trim CLI flags (emit warning + ignored),
> keep config fields, delete obsolete tests, update docstrings. File
> AZ-902 (cycle 5) for hard surface removal."
The follow-up ticket was filed as **AZ-908** (Jira renumbered from the
proposed AZ-902 because AZ-902AZ-907 were already taken). AZ-908 is
in `backlog/` and depends on AZ-895 + AZ-842.
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-895_deprecate_auto_sync_surface | Done | 8 modified, 1 added (test) | 3 unit-test files deleted; 1 new test added; 1 existing test file updated; 2,287 unit tests green | 5/5 | None |
### Files touched
Production (`src/gps_denied_onboard/**`):
- REPLACED `replay_input/auto_sync.py` — was a 700+ LOC detector
module; now a 56-line no-op stub whose every public callable raises
`ReplayInputAdapterError("auto-sync removed; supply --imu CSV instead")`.
`__all__` preserved so any external import still resolves.
- REPLACED `replay_input/tlog_video_adapter.py` — was a 700+ LOC
coordinator; now a 105-line deprecated-stub that keeps the
`ReplayInputAdapter` class signature for source-compat. `open()`
raises `ReplayInputAdapterError(...)` immediately; `close()` is a
no-op. Re-exports `ReplayPace` so `_replay_branch.py` can continue
to import it from the same path (preserving the AZ-401 AC-8 import
boundary).
- MODIFIED `runtime_root/_replay_branch.py` — removed the legacy
`ReplayInputAdapter` instantiation branch in
`_build_replay_input_bundle`; deleted the `_build_auto_sync_config`
helper; tightened `_validate_replay_paths` to require `imu_csv_path`
(no more tlog fallback); dropped unused `WgsConverter` and
`AutoSyncConfig` imports; removed the `replay_input_adapter_factory`
test-injection parameter; updated module + function docstrings;
cleaned `auto_sync_used` from the ready-log kv (always None now).
- MODIFIED `replay_input/__init__.py` — docstring rewritten to flag
the deprecation status of the `tlog_video_adapter` / `auto_sync`
surfaces. Re-exports preserved.
- MODIFIED `cli/replay.py``--time-offset-ms`, `--skip-auto-sync`,
`--auto-trim` help text replaced with `DEPRECATED (AZ-895)` notice;
`_print_startup_banner` now emits a `DeprecationWarning` + stderr
line when any of the three are non-default; `_build_replay_config`
hard-codes the corresponding `ReplayConfig` fields to None / False
so the deprecated values cannot influence composition.
- MODIFIED `components/c8_fc_adapter/tlog_replay_adapter.py` — module
docstring reframed as **AUDIT-ONLY** (AC-5). Code unchanged.
- MODIFIED `replay_input/tlog_ground_truth.py` — module docstring
reframed as **AUDIT-ONLY** (AC-5). Code unchanged.
Tests (`tests/**`):
- DELETED `tests/unit/replay_input/test_az405_auto_sync.py` (386 LOC).
Rationale: tested the AZ-405 detector algorithm + AC-9 validator
which no longer execute. Per AC-3, the deprecation-stub test below
replaces it.
- DELETED `tests/unit/replay_input/test_az405_replay_input_adapter.py`
(645 LOC). Rationale: tested the `ReplayInputAdapter` coordinator's
six-step `open()` workflow which now raises immediately.
- DELETED `tests/unit/replay_input/test_az698_window_alignment.py`
(745 LOC). Rationale: tested the AZ-698 IMU↔optical-flow
cross-correlation aligner which no longer executes.
- ADDED `tests/unit/replay_input/test_az895_auto_sync_deprecated_stub.py`
— 5 parametrised tests pinning the AC-1 contract: every public
callable raises `ReplayInputAdapterError` with the documented
message.
- MODIFIED `tests/unit/test_az402_replay_cli.py` — renamed
`test_ac4_time_offset_forwarded``test_ac4_time_offset_ignored_after_az895`
with asserts inverted (value now `None` regardless of flag); added
`test_az895_skip_auto_sync_ignored_and_warned`,
`test_az895_auto_trim_ignored_and_warned`,
`test_az895_no_deprecated_flags_no_warning`; `_argv` helper grew
`skip_auto_sync` and `auto_trim` overrides.
- MODIFIED `tests/unit/test_az401_compose_root_replay.py` — renamed
`test_replay_branch_rejects_both_inputs_empty`
`test_replay_branch_rejects_missing_imu_csv_path` with body updated
to the new gate semantics; `_make_replay_config` helper now sets
`imu_csv_path` by default; deleted
`test_replay_branch_loads_camera_calibration_from_runtime_path`
(only verified the now-removed `replay_input_adapter_factory`
injection path; calibration loading is exercised indirectly by the
full compose-root tests and by the e2e suite).
- MODIFIED `tests/e2e/replay/test_derkachi_real_tlog.py` — xfail
reason text refreshed to reference AZ-848 + AZ-883 (the live
tlog-clock root cause) instead of the closed AZ-776 + AZ-777 (AC-4
literally specifies "AZ-848-scoped reason").
Docs (`_docs/**`):
- MODIFIED `_docs/02_document/module-layout.md``replay_input` file
list flags `tlog_video_adapter.py` + `auto_sync.py` as
**DEPRECATED (AZ-895)**, adds `csv_ground_truth.py`, updates the
package purpose to lead with the CSV path.
- ADDED `_docs/02_tasks/backlog/AZ-908_replay_auto_sync_hard_removal.md`
— cycle-5+ follow-up spec.
- MODIFIED `_docs/02_tasks/_dependencies_table.md` — preamble +
Total Tasks (179 → 180) + Total Complexity (567 → 570); AZ-908 row
added under Cycle-4 / AZ-895 follow-up.
Tracker (Jira):
- AZ-895 — transitioned `To Do``In Progress` (transition id `21`).
- AZ-908 — created (`Replay: hard removal of deprecated auto-sync
surface (AZ-895 follow-up)`), 3 SP estimate, deps AZ-895 (hard) +
AZ-842 (hard). Filed via `createJiraIssue` MCP.
## File-Ownership Note
All touched paths are owned by the cycle-4 replay-input redesign
envelope (`replay_input/` + `cli/replay.py` + `runtime_root/_replay_branch.py`)
plus the AC-5 audit-only docstring updates inside
`components/c8_fc_adapter/tlog_replay_adapter.py` (the c8 owner
already accepted the audit-only reframing in AZ-894). No
out-of-scope edits.
## AC Test Coverage
| AC | Coverage | Test |
|----|----------|------|
| AC-1 (`auto_sync.py` is deleted or made a no-op raising the documented error) | Direct | `tests/unit/replay_input/test_az895_auto_sync_deprecated_stub.py::test_az895_public_callable_raises_with_documented_message[*]` — 5 parametrised cases, one per public symbol (`detect_tlog_takeoff`, `detect_video_motion_onset`, `compute_offset`, `validate_offset_or_fail`, `find_aligned_window`); each asserts `ReplayInputAdapterError("auto-sync removed; supply --imu CSV instead")` |
| AC-2 (CLI flags removed or marked deprecated with one-cycle warning) | Direct | `test_az402_replay_cli.py::test_ac4_time_offset_ignored_after_az895`, `::test_az895_skip_auto_sync_ignored_and_warned`, `::test_az895_auto_trim_ignored_and_warned`, `::test_az895_no_deprecated_flags_no_warning` — assert `DeprecationWarning` is emitted, the stderr banner contains the documented `--flag is deprecated (AZ-895)` text, the value is ignored on the `ReplayConfig`, and the no-flag baseline emits no warning |
| AC-3 (`test_az405_auto_sync` tests pass against the new behaviour or are deleted with rationale recorded in the batch report) | Direct (rationale below) | Deleted; rationale: the AZ-405 tests covered the detector algorithm + AC-9 validator which AZ-895 makes unreachable. Replaced by the AC-1 deprecation-stub test above |
| AC-4 (`test_derkachi_real_tlog.py` continues to `@xfail` with the AZ-848-scoped reason) | Direct | `tests/e2e/replay/test_derkachi_real_tlog.py::test_az699_real_flight_validation_emits_verdict_and_report``@pytest.mark.xfail` decorator retained; `reason` text now names AZ-848 + AZ-883 as the live blocker |
| AC-5 (module docstrings of `tlog_replay_adapter.py` and `tlog_ground_truth.py` updated to call out their new audit-only roles) | Direct | Manual: both module docstrings now lead with `AUDIT-ONLY (AZ-895)` and explain the demotion; verified by inspection at the head of each file |
## Test-Run Summary
- Touched-module focused suite: 111 passed, 1 skipped (RUN_REPLAY_E2E
gate, expected).
- Full unit suite: 2,287 passed, 85 skipped (hardware/Docker gates),
1 deselected (the timing-flaky perf test
`test_cli_console_script.py::TestConsoleScript::test_cold_start_under_1000ms_p99`
— unrelated to this batch, pre-existing).
## Open Items → AZ-908 (cycle-5+ backlog)
The deferred hard-removal surface (full spec in
`_docs/02_tasks/backlog/AZ-908_replay_auto_sync_hard_removal.md`):
- Delete `replay_input/auto_sync.py` + `replay_input/tlog_video_adapter.py`.
- Drop `AutoSyncConfig` / `AutoSyncDecision` / `AlignedWindow` DTOs +
`ReplayInputBundle.auto_sync_result` / `aligned_window` fields.
- Drop the three deprecated CLI flags + their tests.
- Drop `ReplayConfig.time_offset_ms` / `.skip_auto_sync_validation` /
`.auto_trim` / `.auto_sync` + `ReplayAutoSyncConfig` class.
- Drop `BUILD_TLOG_REPLAY_ADAPTER` build flag from `REPLAY_BUILD_FLAGS`.
- Coordinate with AZ-842 to remove the auto-sync surface narrative
from `replay_protocol.md`.
## Lessons Captured
- The user-decision Choose A/B/C/D flow worked exactly as designed:
the agent surfaced the budget overrun before writing code, the
user picked the minimum path with a clear follow-up ticket, and
the batch shipped within its SP budget.
- Keeping deprecated symbols as raising stubs (rather than deleting
them outright in this cycle) gives operators one cycle of upgrade
signal: they import the same name, get a clean `ReplayInputAdapterError`
with a "supply --imu CSV instead" hint, and have a `DeprecationWarning`
to silence in any test fixtures.
- Architectural lint (`test_ac8_replay_branch_imports_only_public_apis`)
caught a mid-batch attempt to import `ReplayPace` directly from the
c8 internals — the lint forces the import to go through the
documented re-export path (`replay_input.tlog_video_adapter`). Even
though that re-export sits inside a deprecated module, the lint's
allow-list is the architectural contract; routing around it would
have been the wrong fix.
+39 -22
View File
@@ -28,6 +28,7 @@ import logging
import os import os
import sys import sys
import traceback import traceback
import warnings
from collections.abc import Callable, Sequence from collections.abc import Callable, Sequence
from dataclasses import replace from dataclasses import replace
from pathlib import Path from pathlib import Path
@@ -158,8 +159,10 @@ def _build_argparser() -> argparse.ArgumentParser:
type=int, type=int,
default=None, default=None,
help=( help=(
"Manual offset between video and tlog clocks. When omitted, " "DEPRECATED (AZ-895): the (video, tlog) auto-sync path was "
"ReplayInputAdapter (AZ-405) auto-detects via IMU take-off." "removed. The CSV's Time column is the single canonical "
"clock by construction, so no offset is needed. Accepted "
"for one deprecation cycle but ignored; AZ-908 removes it."
), ),
) )
parser.add_argument( parser.add_argument(
@@ -167,14 +170,9 @@ def _build_argparser() -> argparse.ArgumentParser:
dest="skip_auto_sync_validation", dest="skip_auto_sync_validation",
action="store_true", action="store_true",
help=( help=(
"AZ-611: Also skip the AC-9 frame-window validator that " "DEPRECATED (AZ-895): the AC-9 auto-sync validator was "
"runs on the resolved offset. Only legal in combination " "removed alongside the auto-sync surface. Accepted for "
"with --time-offset-ms (a manual offset is mandatory so " "one deprecation cycle but ignored; AZ-908 removes it."
"the bypass cannot mask a silent-zero auto-sync result). "
"Intended for fixtures where neither the IMU take-off "
"detector nor the video motion-onset detector can "
"produce a reliable signal (mid-flight clips, stationary "
"still-image scenarios)."
), ),
) )
parser.add_argument( parser.add_argument(
@@ -182,13 +180,9 @@ def _build_argparser() -> argparse.ArgumentParser:
dest="auto_trim", dest="auto_trim",
action="store_true", action="store_true",
help=( help=(
"AZ-698: Locate the video's playback window inside a " "DEPRECATED (AZ-895): the IMU↔optical-flow aligner was "
"longer tlog via IMU↔optical-flow cross-correlation, " "removed alongside the auto-sync surface. Accepted for "
"then trim the tlog stream to that window. Mutually " "one deprecation cycle but ignored; AZ-908 removes it."
"exclusive with --time-offset-ms. Below the configured "
"alignment confidence threshold the aligner falls back "
"to the AZ-405 head-takeoff path and the AC-9 validator "
"still gates the final offset."
), ),
) )
parser.add_argument( parser.add_argument(
@@ -274,6 +268,11 @@ def _build_replay_config(
Per ADR-011 the CLI's only job after loading is to set Per ADR-011 the CLI's only job after loading is to set
``config.mode = "replay"`` and populate ``config.replay`` from the ``config.mode = "replay"`` and populate ``config.replay`` from the
operator's CLI args. Composition logic stays in ``compose_root``. operator's CLI args. Composition logic stays in ``compose_root``.
AZ-895: ``--time-offset-ms``, ``--skip-auto-sync``, and
``--auto-trim`` are deprecated. Their values are ignored here so
they cannot influence composition; the deprecation banner in
:func:`_print_startup_banner` already informed the operator.
""" """
new_replay = ReplayConfig( new_replay = ReplayConfig(
video_path=str(args.video), video_path=str(args.video),
@@ -281,9 +280,9 @@ def _build_replay_config(
imu_csv_path=str(args.imu), imu_csv_path=str(args.imu),
output_path=str(args.output), output_path=str(args.output),
pace=args.pace, pace=args.pace,
time_offset_ms=args.time_offset_ms, time_offset_ms=None,
skip_auto_sync_validation=bool(args.skip_auto_sync_validation), skip_auto_sync_validation=False,
auto_trim=bool(args.auto_trim), auto_trim=False,
target_fc_dialect=base_config.replay.target_fc_dialect, target_fc_dialect=base_config.replay.target_fc_dialect,
auto_sync=base_config.replay.auto_sync, auto_sync=base_config.replay.auto_sync,
max_duration_s=( max_duration_s=(
@@ -324,13 +323,20 @@ def _build_replay_config(
# Startup banner # Startup banner
_DEPRECATED_FLAGS_AZ895: Final[tuple[tuple[str, str], ...]] = (
("time_offset_ms", "--time-offset-ms"),
("skip_auto_sync_validation", "--skip-auto-sync"),
("auto_trim", "--auto-trim"),
)
def _print_startup_banner(args: argparse.Namespace) -> None: def _print_startup_banner(args: argparse.Namespace) -> None:
"""Print a sanitised one-line banner to stderr before logging boots. """Print a sanitised one-line banner to stderr before logging boots.
Logging is bootstrapped inside the airborne main; this banner gives Logging is bootstrapped inside the airborne main; this banner gives
the operator a single line confirming what the CLI parsed before any the operator a single line confirming what the CLI parsed before any
further output. AZ-894: also surfaces the --tlog deprecation warning further output. AZ-894 / AZ-895: also surfaces deprecation warnings
inline so operators see it even when stderr is the only sink. inline so operators see them even when stderr is the only sink.
""" """
sanitised = vars(args).copy() sanitised = vars(args).copy()
sanitised["mavlink_signing_key"] = "<redacted>" sanitised["mavlink_signing_key"] = "<redacted>"
@@ -347,6 +353,17 @@ def _print_startup_banner(args: argparse.Namespace) -> None:
file=sys.stderr, file=sys.stderr,
flush=True, flush=True,
) )
for dest, flag in _DEPRECATED_FLAGS_AZ895:
value = getattr(args, dest, None)
if value in (None, False):
continue
msg = (
f"{flag} is deprecated (AZ-895) and will be removed in AZ-908. "
"The (video, CSV) replay path has no auto-sync surface; "
"this flag is accepted but ignored. Remove it from your invocation."
)
warnings.warn(msg, DeprecationWarning, stacklevel=2)
print(f"gps-denied-replay: WARNING {msg}", file=sys.stderr, flush=True)
# ---------------------------------------------------------------------- # ----------------------------------------------------------------------
@@ -1,18 +1,26 @@
"""``TlogReplayFcAdapter`` (AZ-399 / E-DEMO-REPLAY). """``TlogReplayFcAdapter`` (AZ-399 / E-DEMO-REPLAY).
Replay-only :class:`FcAdapter` strategy parsing pymavlink ``.tlog`` AUDIT-ONLY (AZ-895): retained as a tlog-file parser strategy that
files. Implements the full Protocol from implements the :class:`FcAdapter` Protocol. The production replay
pipeline now composes :class:`CsvReplayFcAdapter` against the
operator's IMU+GPS CSV (AZ-894). This adapter remains in the tree as:
- The source of :class:`ReplayPace` (shared enum used by every replay
adapter and the composition root).
- A one-off audit utility for inspecting historical ``.tlog`` files
outside the main replay flow.
It is no longer instantiated by :func:`compose_root`'s replay branch
and AZ-908 will retire the ``BUILD_TLOG_REPLAY_ADAPTER`` build flag
that still guards its construction.
Implements the full Protocol from
``_docs/02_document/contracts/c8_fc_adapter/fc_adapter_protocol.md`` ``_docs/02_document/contracts/c8_fc_adapter/fc_adapter_protocol.md``
plus the replay-specific Invariants 5, 6, 8 from plus the replay-specific Invariants 5, 6, 8 from
``_docs/02_document/contracts/replay/replay_protocol.md`` (no ``_docs/02_document/contracts/replay/replay_protocol.md`` (no
out-bound emission, pace honoured by injected :class:`Clock`, out-bound emission, pace honoured by injected :class:`Clock`,
``time_offset_ms`` shift baked at construction). ``time_offset_ms`` shift baked at construction).
Build-time gating: the adapter refuses construction unless
``BUILD_TLOG_REPLAY_ADAPTER`` is ``ON``. Only the ``replay-cli``
binary is expected to flip the flag ON; airborne / research /
operator binaries keep it OFF.
Stream-parse design: pymavlink's :class:`mavutil.mavlogfile` already Stream-parse design: pymavlink's :class:`mavutil.mavlogfile` already
streams from disk via :meth:`recv_match`. The adapter wraps it in a streams from disk via :meth:`recv_match`. The adapter wraps it in a
pre-scan pass (fail-fast on missing required message types per pre-scan pass (fail-fast on missing required message types per
+17 -15
View File
@@ -1,22 +1,24 @@
"""``replay_input/`` cross-cutting coordinator (AZ-405 / E-DEMO-REPLAY). """``replay_input/`` cross-cutting coordinator (AZ-405 / E-DEMO-REPLAY).
Layer-4 module per ``_docs/02_document/module-layout.md``. Converges Layer-4 module per ``_docs/02_document/module-layout.md``. Under
``(video, tlog)`` inputs into the standard :class:`FrameSource`, AZ-894 the production replay pipeline drives off the operator's
:class:`FcAdapter`, and :class:`Clock` surfaces consumed by the IMU+GPS CSV via :class:`CsvReplayFcAdapter`; the legacy ``(video,
airborne composition root. Owns the time-alignment concern between tlog)`` auto-sync surface was deprecated by AZ-895 and will be removed
video frames and tlog IMU/attitude ticks (manual via by AZ-908.
``--time-offset-ms`` or automatic via the AZ-405 IMU-take-off
detector).
New under ADR-011 (replay-as-configuration) replaces the v1.0.0 The package retains:
design where replay had its own composition root.
Public surface re-exports the coordinator class, the bundle DTO, the - :class:`ReplayInputAdapter` and :class:`ReplayInputAdapterError`
auto-sync decision DTO, the auto-sync config DTO, and the coordinator the latter is the canonical replay error class, used by every
error class. The detector functions in :mod:`auto_sync` are NOT replay adapter (CSV and tlog).
re-exported here so the public API stays focused on the composition - :class:`ReplayInputBundle` the DTO :func:`compose_root` consumes.
root's wiring needs; tests import the detectors via their full module - :class:`AutoSyncConfig`, :class:`AutoSyncDecision`,
path. :class:`AlignedWindow` kept on the public surface for one
deprecation cycle so any external caller's import does not break.
- Tlog ground-truth + route helpers used by AZ-697 / AZ-836 audit
paths.
Hard removal of the deprecated symbols lands in AZ-908.
""" """
from gps_denied_onboard._types.route import RouteSpec from gps_denied_onboard._types.route import RouteSpec
File diff suppressed because it is too large Load Diff
@@ -1,19 +1,23 @@
"""Direct binary-tlog GPS-truth extractor (AZ-697 / E-DEMO-REPLAY). """Direct binary-tlog GPS-truth extractor (AZ-697 / E-DEMO-REPLAY).
Streams ``GLOBAL_POSITION_INT`` (preferred) or ``GPS_RAW_INT`` (fallback) AUDIT-ONLY (AZ-895): the production replay pipeline now consumes
from an ArduPilot binary tlog into a typed :class:`TlogGroundTruth` DTO, ground truth through :class:`CsvGroundTruth` (AZ-894) driven by the
suitable for the AZ-699 (real-flight validation) and AZ-701 (HTTP operator's IMU+GPS CSV. This helper is retained for one-off audits and
replay API) comparison paths. the AZ-699 / AZ-701 validation paths that still operate against legacy
``.tlog`` archives; it is not part of the main replay composition root.
Design mirrors :mod:`gps_denied_onboard.replay_input.auto_sync`: Streams ``GLOBAL_POSITION_INT`` (preferred) or ``GPS_RAW_INT`` (fallback)
from an ArduPilot binary tlog into a typed :class:`TlogGroundTruth` DTO.
Design notes:
* Lazy ``pymavlink.mavutil`` import missing dependency raises * Lazy ``pymavlink.mavutil`` import missing dependency raises
:class:`ReplayInputAdapterError` rather than crashing the import. :class:`ReplayInputAdapterError` rather than crashing the import.
* Optional ``source_factory`` injection point so unit tests can swap in * Optional ``source_factory`` injection point so unit tests can swap in
a synthetic source (mirrors the AZ-399 / AZ-405 pattern). a synthetic source (mirrors the AZ-399 pattern).
* Production helper only placed under ``replay_input/`` because the * Placed under ``replay_input/`` because the GPS extraction is
GPS extraction is intrinsically tied to the tlog input pipeline; the intrinsically tied to the tlog input pipeline; the comparison kernels
comparison kernels themselves live in :mod:`helpers.gps_compare`. themselves live in :mod:`helpers.gps_compare`.
""" """
from __future__ import annotations from __future__ import annotations
@@ -1,134 +1,56 @@
"""``ReplayInputAdapter`` (AZ-405 / E-DEMO-REPLAY). """DEPRECATED (AZ-895): ``ReplayInputAdapter`` retained as a raising stub.
Layer-4 cross-cutting coordinator that converges ``(video, tlog)`` The (video, tlog) coordinator was the v1.0.0 entry point into the
inputs into the standard :class:`FrameSource`, :class:`FcAdapter`, auto-sync surface. As of AZ-894 (cycle 4) the replay pipeline runs off
and :class:`Clock` surfaces consumed by the airborne composition a paired (video, CSV) input :class:`CsvReplayFcAdapter` plus the
root. Owns the time-alignment concern: either the operator's manual :class:`CsvVideoBundle` builder in :mod:`gps_denied_onboard.runtime_root._replay_branch`.
``--time-offset-ms`` override or the AZ-405 IMU-take-off auto-detect.
``open()`` performs strict ordering so AC-13 holds: This module keeps the :class:`ReplayInputAdapter` symbol live for one
deprecation cycle so external imports surface a clean
:class:`ReplayInputAdapterError` instead of an ``ImportError``. Hard
removal lands in AZ-908 (cycle 5+).
1. **Tlog message-type pre-validation** runs FIRST so a tlog missing Operators with old (video, tlog) scripts should switch to the
``RAW_IMU`` / ``ATTITUDE`` raises before the video is ever read. (video, CSV) input; see
2. If the constructor received ``manual_time_offset_ms is None``, ``_docs/02_document/contracts/replay/csv_replay_format.md``.
the auto-sync detectors run; otherwise the manual offset is
adopted directly (AC-8 verifies the bypass).
3. The resolved offset is fed through the AC-9 frame-window match
validator; a hard-fail raises ``"auto-sync hard-fail: …"`` so
the shared main maps it to CLI exit code 2 (AC-7).
4. The :class:`Clock` strategy is constructed (``TlogDerivedClock``
for ``pace=ASAP``, ``WallClock`` for ``pace=REALTIME``) the
single instance the bundle ships to the composition root
(Invariant 2; AC-5).
5. :class:`VideoFileFrameSource` and :class:`TlogReplayFcAdapter`
are constructed against the offset + clock + dialect; the FC
adapter's own ``open()`` triggers its independent pre-scan (a
second sanity check; the operator gets the original error path
if step 1 was bypassed via a test fake).
6. The bundle is returned with ``auto_sync_result`` populated for
the auto path and ``None`` for the manual path.
The coordinator is idempotent on ``close()`` repeated calls are
no-ops once the underlying strategies have been released (AC-12).
""" """
from __future__ import annotations from __future__ import annotations
import logging
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from gps_denied_onboard._types.fc import FcKind from gps_denied_onboard._types.fc import FcKind
from gps_denied_onboard.clock.tlog_derived import TlogDerivedClock
from gps_denied_onboard.clock.wall_clock import WallClock
from gps_denied_onboard.components.c8_fc_adapter.errors import (
FcAdapterConfigError,
FcAdapterError,
FcOpenError,
)
from gps_denied_onboard.components.c8_fc_adapter.tlog_replay_adapter import ( from gps_denied_onboard.components.c8_fc_adapter.tlog_replay_adapter import (
ReplayPace, ReplayPace,
TlogReplayFcAdapter,
)
from gps_denied_onboard.fdr_client.records import FdrRecord
from gps_denied_onboard.frame_source.errors import (
FrameSourceConfigError,
FrameSourceError,
)
from gps_denied_onboard.frame_source.video_file import VideoFileFrameSource
from gps_denied_onboard.helpers.iso_timestamps import iso_ts_now
from gps_denied_onboard.replay_input.auto_sync import (
_load_tlog_samples,
compute_offset,
detect_video_motion_onset,
find_aligned_window,
validate_offset_or_fail,
) )
from gps_denied_onboard.replay_input.errors import ReplayInputAdapterError from gps_denied_onboard.replay_input.errors import ReplayInputAdapterError
from gps_denied_onboard.replay_input.interface import ( from gps_denied_onboard.replay_input.interface import (
AlignedWindow,
AutoSyncConfig, AutoSyncConfig,
AutoSyncDecision,
ReplayInputBundle, ReplayInputBundle,
) )
if TYPE_CHECKING: if TYPE_CHECKING:
from gps_denied_onboard._types.calibration import CameraCalibration from gps_denied_onboard._types.calibration import CameraCalibration
from gps_denied_onboard.clock import Clock
from gps_denied_onboard.fdr_client.client import FdrClient from gps_denied_onboard.fdr_client.client import FdrClient
from gps_denied_onboard.helpers.wgs_converter import WgsConverter from gps_denied_onboard.helpers.wgs_converter import WgsConverter
__all__ = ["ReplayInputAdapter"] __all__ = ["ReplayInputAdapter", "ReplayPace"]
_FDR_PRODUCER_ID = "replay_input.tlog_video_adapter" _REMOVED_MSG = (
"tlog_video_adapter.ReplayInputAdapter is deprecated (AZ-895); "
_LOG_KIND_AUTO_SYNC_DETECTED = "replay.auto_sync.detected" "supply --imu CSV instead"
_LOG_KIND_AUTO_SYNC_LOW_CONF = "replay.auto_sync.low_confidence" )
_LOG_KIND_AUTO_SYNC_AC8_FAIL = "replay.auto_sync.ac8_validation_failed"
_LOG_KIND_OPEN_MANUAL = "replay.input.opened_manual_offset"
_LOG_KIND_AUTO_TRIM_RESOLVED = "replay.auto_trim.resolved"
_LOG_KIND_AUTO_TRIM_FALLBACK = "replay.auto_trim.fallback_to_takeoff"
class ReplayInputAdapter: class ReplayInputAdapter:
"""Coordinator that converges ``(video, tlog)`` into the airborne strategies. """DEPRECATED (AZ-895): :meth:`open` raises :class:`ReplayInputAdapterError`.
Constructor parameters: Constructor remains tolerant so callers can still import the symbol
and instantiate it (e.g. for backwards-compatible plugin discovery),
- ``video_path`` / ``tlog_path`` filesystem inputs. but every meaningful use raises. Hard removal lands in AZ-908.
- ``camera_calibration`` :class:`CameraCalibration` used to
derive the calibration ID propagated into every emitted
:class:`NavCameraFrame`.
- ``target_fc_dialect`` ``ARDUPILOT_PLANE`` or ``INAV``;
passed through to :class:`TlogReplayFcAdapter`.
- ``wgs_converter`` shared geodesy helper, constructor-injected
into :class:`TlogReplayFcAdapter`.
- ``fdr_client`` FDR sink for the TlogReplayFcAdapter and for
the coordinator's own structured-event mirror.
- ``pace`` :class:`ReplayPace` (``ASAP`` or ``REALTIME``).
- ``manual_time_offset_ms`` ``None`` triggers auto-sync; an
integer bypasses auto-sync DETECTION but the AC-9 frame-window
validator still runs on the resolved offset (AC-8).
- ``skip_auto_sync_validation`` when ``True``, ALSO skip the
AC-9 validator. Only legal in combination with a non-``None``
``manual_time_offset_ms`` (the coordinator refuses both-None
to avoid silent-zero offset bugs). Intended for fixtures where
neither the IMU take-off detector nor the video motion-onset
detector can produce a reliable signal (mid-flight clips,
stationary still-image scenarios see AZ-611). Default
``False``.
- ``auto_sync_config`` :class:`AutoSyncConfig` thresholds.
Behaviour:
- :meth:`open` resolves the offset, validates AC-9, and returns a
:class:`ReplayInputBundle` with the wired strategies. Raises
:class:`ReplayInputAdapterError` on every coordinator-scope
failure so the shared main can map cleanly to CLI exit code 2.
- :meth:`close` releases the FC adapter and the frame source;
idempotent (AC-12).
""" """
__slots__ = ( __slots__ = (
@@ -143,14 +65,6 @@ class ReplayInputAdapter:
"_skip_auto_sync_validation", "_skip_auto_sync_validation",
"_auto_trim", "_auto_trim",
"_auto_sync_config", "_auto_sync_config",
"_tlog_source_factory",
"_video_frames_factory",
"_video_timestamps_factory",
"_mavlink_transport",
"_log",
"_opened",
"_closed",
"_bundle",
) )
def __init__( def __init__(
@@ -172,54 +86,6 @@ class ReplayInputAdapter:
video_timestamps_factory: Any | None = None, video_timestamps_factory: Any | None = None,
mavlink_transport: Any | None = None, mavlink_transport: Any | None = None,
) -> None: ) -> None:
if not isinstance(video_path, Path):
raise ReplayInputAdapterError(
f"video_path must be a pathlib.Path; got {type(video_path).__name__}"
)
if not isinstance(tlog_path, Path):
raise ReplayInputAdapterError(
f"tlog_path must be a pathlib.Path; got {type(tlog_path).__name__}"
)
if target_fc_dialect not in (FcKind.ARDUPILOT_PLANE, FcKind.INAV):
raise ReplayInputAdapterError(
f"target_fc_dialect must be ARDUPILOT_PLANE or INAV; "
f"got {target_fc_dialect!r}"
)
if not isinstance(pace, ReplayPace):
raise ReplayInputAdapterError(
f"pace must be a ReplayPace enum; got {type(pace).__name__}"
)
if not isinstance(skip_auto_sync_validation, bool):
raise ReplayInputAdapterError(
"skip_auto_sync_validation must be a bool; got "
f"{type(skip_auto_sync_validation).__name__}"
)
if skip_auto_sync_validation and manual_time_offset_ms is None:
# Mirror the ReplayConfig.__post_init__ gate. Without a
# manual offset there is no operator-acknowledged value
# to skip validation against — auto-sync would compute
# an offset of unknown quality and the validator that
# would catch a bad detection is disabled. Refuse so
# this can't silently mask a wrong offset.
raise ReplayInputAdapterError(
"skip_auto_sync_validation=True requires "
"manual_time_offset_ms to be set"
)
if not isinstance(auto_trim, bool):
raise ReplayInputAdapterError(
"auto_trim must be a bool; got "
f"{type(auto_trim).__name__}"
)
if auto_trim and manual_time_offset_ms is not None:
# Mirror the ReplayConfig.__post_init__ gate. An explicit
# manual offset means the operator has already aligned
# the streams; running the cross-correlation aligner on
# top of that would either re-resolve the same window
# (wasteful) or overwrite the operator's intent silently.
raise ReplayInputAdapterError(
"auto_trim=True is mutually exclusive with "
"manual_time_offset_ms"
)
self._video_path = video_path self._video_path = video_path
self._tlog_path = tlog_path self._tlog_path = tlog_path
self._camera_calibration = camera_calibration self._camera_calibration = camera_calibration
@@ -231,506 +97,9 @@ class ReplayInputAdapter:
self._skip_auto_sync_validation = skip_auto_sync_validation self._skip_auto_sync_validation = skip_auto_sync_validation
self._auto_trim = auto_trim self._auto_trim = auto_trim
self._auto_sync_config = auto_sync_config self._auto_sync_config = auto_sync_config
self._tlog_source_factory = tlog_source_factory
self._video_frames_factory = video_frames_factory
self._video_timestamps_factory = video_timestamps_factory
self._mavlink_transport = mavlink_transport
self._log = logging.getLogger("replay_input.tlog_video_adapter")
self._opened = False
self._closed = False
self._bundle: ReplayInputBundle | None = None
def open(self) -> ReplayInputBundle: def open(self) -> ReplayInputBundle:
"""Resolve the offset, build the strategies, return the bundle. raise ReplayInputAdapterError(_REMOVED_MSG)
Idempotent only in the failure-then-retry sense calling
``open()`` twice without an intervening ``close()`` raises
:class:`ReplayInputAdapterError`.
"""
if self._opened:
raise ReplayInputAdapterError("ReplayInputAdapter already opened")
# Step 1 — tlog presence + required-message check (R-DEMO-3,
# AC-13). Runs BEFORE any video read so a malformed tlog
# surfaces without paying the cv2.VideoCapture cost.
tlog_imu_timestamps_ns, tlog_samples_for_auto = self._load_and_validate_tlog()
# Step 2 — resolve the offset (auto-sync, auto-trim, or
# manual override).
decision: AutoSyncDecision | None
aligned_window: AlignedWindow | None
if self._auto_trim:
aligned_window = self._run_auto_trim()
decision = None
resolved_offset_ms = aligned_window.offset_ms
# The prescan timestamps (step 1) only cover the tlog head.
# When the auto-trim window is far into the tlog, the prescan
# timestamps fall outside the window and the AC-9 validator
# would always return 0 % match → false hard-fail. Reload
# IMU timestamps from the discovered window so the validator
# sees the correct slice.
if aligned_window.tlog_start_ns > 0:
tlog_imu_timestamps_ns = self._load_tlog_imu_in_window(
aligned_window.tlog_start_ns,
aligned_window.tlog_end_ns,
)
self._log.info(
"replay_input.ac9_window_reload: "
"tlog_start_ns=%d tlog_end_ns=%d loaded=%d imu_samples",
aligned_window.tlog_start_ns,
aligned_window.tlog_end_ns,
len(tlog_imu_timestamps_ns),
extra={
"kind": "replay_input.ac9_window_reload",
"kv": {
"tlog_start_ns": aligned_window.tlog_start_ns,
"tlog_end_ns": aligned_window.tlog_end_ns,
"loaded_imu_count": len(tlog_imu_timestamps_ns),
},
},
)
elif self._manual_time_offset_ms is None:
aligned_window = None
decision = self._run_auto_sync(tlog_samples_for_auto)
resolved_offset_ms = decision.offset_ms
else:
aligned_window = None
decision = None
resolved_offset_ms = int(self._manual_time_offset_ms)
self._log.info(
f"{_LOG_KIND_OPEN_MANUAL}: resolved_offset_ms={resolved_offset_ms}",
extra={
"kind": _LOG_KIND_OPEN_MANUAL,
"kv": {"resolved_offset_ms": resolved_offset_ms},
},
)
# Step 3 — load video frame timestamps and run AC-9 validator
# unless the operator explicitly opted out via
# skip_auto_sync_validation (AZ-611). The opt-out is meant for
# mid-flight + stationary fixtures where neither detector can
# produce a reliable signal; the constructor already enforced
# that the opt-out requires a manual offset.
video_frame_timestamps_ns = self._load_video_timestamps()
if self._skip_auto_sync_validation:
self._log.info(
f"{_LOG_KIND_OPEN_MANUAL}: ac9_validator_skipped "
f"(resolved_offset_ms={resolved_offset_ms})",
extra={
"kind": _LOG_KIND_OPEN_MANUAL,
"kv": {
"resolved_offset_ms": resolved_offset_ms,
"ac9_validator_skipped": True,
},
},
)
else:
result_code = validate_offset_or_fail(
resolved_offset_ms,
tlog_imu_timestamps_ns,
video_frame_timestamps_ns,
threshold_pct=self._auto_sync_config.match_threshold_pct,
window_ms=self._auto_sync_config.match_window_ms,
)
if result_code != 0:
self._raise_ac8_fail(
resolved_offset_ms,
len(tlog_imu_timestamps_ns),
len(video_frame_timestamps_ns),
)
# Step 4 — clock strategy (single instance per Invariant 2).
clock = self._build_clock()
# Step 5 — concrete strategies. The frame source is built
# first because its constructor verifies the build flag and
# opens the cv2 capture handle — a failure here is a clean
# config error (no resources held). The FC adapter is built
# second; its open() launches the decode thread.
try:
frame_source = VideoFileFrameSource(
path=self._video_path,
camera_calibration_id=self._camera_calibration.camera_id,
clock=clock,
)
except FrameSourceConfigError as exc:
raise ReplayInputAdapterError(
f"video file unreadable / unsupported codec: {self._video_path} "
f"({exc})"
) from exc
except FrameSourceError as exc:
raise ReplayInputAdapterError(
f"video file decode error: {self._video_path} ({exc})"
) from exc
try:
fc_adapter = TlogReplayFcAdapter(
tlog_path=self._tlog_path,
target_fc_dialect=self._target_fc_dialect,
clock=clock,
wgs_converter=self._wgs_converter,
fdr_client=self._fdr_client,
time_offset_ms=resolved_offset_ms,
tlog_start_ns=(
aligned_window.tlog_start_ns
if aligned_window is not None
else None
),
pace=self._pace,
source_factory=self._tlog_source_factory,
mavlink_transport=self._mavlink_transport,
)
fc_adapter.open()
except (FcOpenError, FcAdapterConfigError, FcAdapterError) as exc:
# Release the already-built frame source so we do not
# leak the cv2 handle when the FC adapter fails after
# the video was opened.
try:
frame_source.close()
except Exception: # pragma: no cover — defensive.
self._log.debug(
"ReplayInputAdapter: frame_source.close() during FC adapter rollback failed",
exc_info=True,
)
# Translate the FC error into the coordinator's single
# public failure shape so the CLI exit-code mapping
# remains single-source. Pre-scan failures naturally
# surface the "tlog missing required messages: …" prefix
# the contract mandates.
raise ReplayInputAdapterError(str(exc)) from exc
# Step 6 — assemble + record the bundle.
bundle = ReplayInputBundle(
frame_source=frame_source,
fc_adapter=fc_adapter,
clock=clock,
resolved_time_offset_ms=resolved_offset_ms,
auto_sync_result=decision,
aligned_window=aligned_window,
)
self._bundle = bundle
self._opened = True
return bundle
def close(self) -> None: def close(self) -> None:
"""Release the FC adapter + frame source; idempotent (AC-12).""" return None
if self._closed:
self._log.debug(
"ReplayInputAdapter.close called twice; no-op"
)
return
self._closed = True
bundle = self._bundle
self._bundle = None
if bundle is None:
return
try:
bundle.fc_adapter.close()
except Exception: # pragma: no cover — defensive.
self._log.debug(
"ReplayInputAdapter: fc_adapter.close() raised", exc_info=True
)
try:
bundle.frame_source.close()
except Exception: # pragma: no cover — defensive.
self._log.debug(
"ReplayInputAdapter: frame_source.close() raised", exc_info=True
)
# ------------------------------------------------------------------
# Internal helpers
def _load_and_validate_tlog(
self,
) -> tuple[list[int], Any]:
"""Load tlog IMU + ATTITUDE samples; raise on missing types.
Returns the IMU-only timestamp list (used by the AC-9
validator) plus the full :class:`TlogSamples` so the auto-
sync path can reuse the same scan for take-off detection.
Raises :class:`ReplayInputAdapterError` for the R-DEMO-3
missing-types path; this is the AC-13 fail-fast surface.
"""
if not self._tlog_path.is_file():
raise ReplayInputAdapterError(
f"tlog file not found: {self._tlog_path}"
)
samples = _load_tlog_samples(
self._tlog_path,
self._auto_sync_config.prescan_max_messages,
source_factory=self._tlog_source_factory,
)
if not samples.accel:
raise ReplayInputAdapterError(
"tlog missing required message types: ['RAW_IMU', 'SCALED_IMU2']"
)
if not samples.attitude:
raise ReplayInputAdapterError(
"tlog missing required message types: ['ATTITUDE']"
)
return [ts for ts, _ in samples.accel], samples
def _run_auto_trim(self) -> AlignedWindow:
"""AZ-698 auto-trim path — cross-correlate IMU energy ↔ optical flow.
Returns the located :class:`AlignedWindow`. When the
correlation peak falls below
:attr:`AutoSyncConfig.alignment_low_confidence_threshold`,
:func:`find_aligned_window` falls back to the AZ-405
head-takeoff detector and sets ``fallback_used=True`` the
coordinator logs WARN but still proceeds (the
AC-9 frame-window validator runs in Step 3 and will
hard-fail if the resolved offset is bad).
"""
window = find_aligned_window(
self._tlog_path,
self._video_path,
self._auto_sync_config,
self._target_fc_dialect,
tlog_source_factory=self._tlog_source_factory,
video_frames_factory=self._video_frames_factory,
)
kind = (
_LOG_KIND_AUTO_TRIM_FALLBACK
if window.fallback_used
else _LOG_KIND_AUTO_TRIM_RESOLVED
)
level = "WARN" if window.fallback_used else "INFO"
kv = {
"tlog_start_ns": window.tlog_start_ns,
"tlog_end_ns": window.tlog_end_ns,
"offset_ms": window.offset_ms,
"confidence": window.confidence,
"fallback_used": window.fallback_used,
"flight_count_detected": window.flight_count_detected,
"selected_flight_index": window.selected_flight_index,
}
msg = (
f"{kind}: tlog_start_ns={window.tlog_start_ns} "
f"offset_ms={window.offset_ms} confidence={window.confidence:.3f} "
f"flights_detected={window.flight_count_detected} "
f"selected_flight={window.selected_flight_index}"
)
if window.fallback_used:
self._log.warning(msg, extra={"kind": kind, "kv": kv})
else:
self._log.info(msg, extra={"kind": kind, "kv": kv})
self._emit_fdr_event(level=level, log_kind=kind, msg=msg, kv=kv)
return window
def _run_auto_sync(self, tlog_samples: Any) -> AutoSyncDecision:
"""Auto path — compute the take-off / motion-onset / offset.
Re-uses the already-loaded ``tlog_samples`` for the take-off
detector so the tlog is walked exactly once per ``open()``
regardless of which path runs.
"""
from gps_denied_onboard.replay_input.auto_sync import (
_compute_tlog_takeoff_from_samples,
)
tlog_result = _compute_tlog_takeoff_from_samples(
tlog_samples, self._auto_sync_config
)
video_result = detect_video_motion_onset(
self._video_path,
self._auto_sync_config,
frames_factory=self._video_frames_factory,
)
decision = compute_offset(tlog_result, video_result)
if decision.combined_confidence < self._auto_sync_config.low_confidence_threshold:
self._log_decision(
kind=_LOG_KIND_AUTO_SYNC_LOW_CONF,
level="WARN",
decision=decision,
extra_kv={"proceeding_with_best_guess": True},
)
else:
self._log_decision(
kind=_LOG_KIND_AUTO_SYNC_DETECTED,
level="INFO",
decision=decision,
extra_kv={},
)
return decision
def _load_video_timestamps(self) -> list[int]:
"""Decode the leading video segment, return per-frame timestamps.
Used by the AC-9 frame-window match validator and as a
fallback when the auto-sync video scan was bypassed (manual
path). Stops at ``video_motion_scan_seconds`` so wildly long
clips do not hold up startup.
"""
if self._video_timestamps_factory is not None:
return list(self._video_timestamps_factory(self._video_path))
try:
import cv2 as _cv2 # type: ignore[import-not-found]
except ImportError as exc:
raise ReplayInputAdapterError(
"opencv-python is required for replay auto-sync but is "
"not importable in this binary"
) from exc
capture = _cv2.VideoCapture(str(self._video_path))
if not capture.isOpened():
capture.release()
raise ReplayInputAdapterError(
f"video file unreadable / unsupported codec: {self._video_path}"
)
out: list[int] = []
max_pos_ms = self._auto_sync_config.video_motion_scan_seconds * 1000.0
try:
while True:
ok = capture.grab()
if not ok:
break
pos_ms = float(capture.get(_cv2.CAP_PROP_POS_MSEC))
if pos_ms > max_pos_ms:
break
out.append(int(pos_ms * 1_000_000))
finally:
capture.release()
return out
def _load_tlog_imu_in_window(
self,
start_ns: int,
end_ns: int,
) -> list[int]:
"""Load tlog IMU timestamps from [start_ns, end_ns].
Used by the AC-9 validator in auto-trim mode. The prescan
(step 1) only covers the tlog head; when the identified window
is later in the file this method re-scans to find IMU samples
in the correct range. Sequential scan is unavoidable (pymavlink
does not seek), but only IMU message types are matched so the
scan is fast in practice.
"""
from gps_denied_onboard.replay_input.auto_sync import _open_tlog
source = _open_tlog(self._tlog_path, source_factory=self._tlog_source_factory)
timestamps: list[int] = []
try:
while True:
try:
msg = source.recv_match(
type=["RAW_IMU", "SCALED_IMU2"],
blocking=False,
)
except Exception as exc:
raise ReplayInputAdapterError(
f"tlog scan for AC-9 window failed: {exc!r}"
) from exc
if msg is None:
break
raw = getattr(msg, "_timestamp", None)
if raw is None:
continue
ts_ns = int(float(raw) * 1_000_000_000)
if ts_ns < start_ns:
continue
if ts_ns > end_ns:
break
timestamps.append(ts_ns)
finally:
if hasattr(source, "close"):
try:
source.close()
except Exception:
pass
return timestamps
def _build_clock(self) -> "Clock":
"""Pick the :class:`Clock` strategy per pace; single instance.
The ``TlogDerivedClock`` is constructed against an empty
iterable here: the composition root (AZ-401) is responsible
for hooking the clock's source up to the live tlog cursor
once the FC adapter's decode thread starts streaming. The
empty-source default keeps unit tests self-contained.
"""
if self._pace is ReplayPace.ASAP:
return TlogDerivedClock(source=iter([]))
return WallClock()
def _log_decision(
self,
*,
kind: str,
level: str,
decision: AutoSyncDecision,
extra_kv: dict[str, Any],
) -> None:
kv: dict[str, Any] = {
"tlog_takeoff_ns": decision.tlog_takeoff_ns,
"video_motion_onset_ns": decision.video_motion_onset_ns,
"offset_ms": decision.offset_ms,
"tlog_confidence": decision.tlog_confidence,
"video_confidence": decision.video_confidence,
"combined_confidence": decision.combined_confidence,
}
kv.update(extra_kv)
msg = f"{kind}: offset_ms={decision.offset_ms} confidence={decision.combined_confidence:.3f}"
if level == "WARN":
self._log.warning(msg, extra={"kind": kind, "kv": kv})
else:
self._log.info(msg, extra={"kind": kind, "kv": kv})
self._emit_fdr_event(level=level, log_kind=kind, msg=msg, kv=kv)
def _raise_ac8_fail(
self,
offset_ms: int,
imu_count: int,
frame_count: int,
) -> None:
kv = {
"offset_ms": offset_ms,
"frame_window_match_pct_threshold": self._auto_sync_config.match_threshold_pct,
"imu_sample_count": imu_count,
"video_frame_count": frame_count,
}
msg = (
f"auto-sync hard-fail: frame-window match below "
f"{self._auto_sync_config.match_threshold_pct}% with "
f"offset_ms={offset_ms}"
)
self._log.error(
f"{_LOG_KIND_AUTO_SYNC_AC8_FAIL}: {msg}",
extra={"kind": _LOG_KIND_AUTO_SYNC_AC8_FAIL, "kv": kv},
)
self._emit_fdr_event(
level="ERROR", log_kind=_LOG_KIND_AUTO_SYNC_AC8_FAIL, msg=msg, kv=kv
)
raise ReplayInputAdapterError(msg)
def _emit_fdr_event(
self,
*,
level: str,
log_kind: str,
msg: str,
kv: dict[str, Any],
) -> None:
record = FdrRecord(
schema_version=1,
ts=iso_ts_now(),
producer_id=_FDR_PRODUCER_ID,
kind="log",
payload={
"level": level,
"component": "replay_input",
"kind": log_kind,
"msg": msg,
"kv": kv,
},
)
try:
self._fdr_client.enqueue(record)
except Exception as exc:
self._log.debug(
f"replay_input.fdr_enqueue_failed: {exc!r}",
extra={
"kind": "replay_input.fdr_enqueue_failed",
"kv": {"error": repr(exc), "downstream_kind": log_kind},
},
)
@@ -16,14 +16,21 @@ shared composition spine while still exposing exactly one
Build-flag gates (per replay protocol Invariant 9): Build-flag gates (per replay protocol Invariant 9):
- ``BUILD_VIDEO_FILE_FRAME_SOURCE`` required for the - ``BUILD_VIDEO_FILE_FRAME_SOURCE`` required for the
:class:`VideoFileFrameSource` instance returned by the coordinator. :class:`VideoFileFrameSource` instance.
- ``BUILD_TLOG_REPLAY_ADAPTER`` required for the - ``BUILD_TLOG_REPLAY_ADAPTER`` historical guard. The tlog adapter
:class:`TlogReplayFcAdapter` instance returned by the coordinator. is no longer composed by replay (AZ-895 deprecated the (video, tlog)
path), but the flag remains in :data:`REPLAY_BUILD_FLAGS` for one
deprecation cycle so operator overrides keep their expected semantics.
AZ-908 will drop the flag.
- ``BUILD_REPLAY_SINK_JSONL`` shared by the JSONL sink and the noop - ``BUILD_REPLAY_SINK_JSONL`` shared by the JSONL sink and the noop
outbound transport. outbound transport.
All three default ON in the airborne binary (per ADR-011); flipping any All three default ON in the airborne binary (per ADR-011); flipping any
OFF disables replay mode without affecting live mode. OFF disables replay mode without affecting live mode.
AZ-895: the replay composition exclusively uses the (video, CSV) path
via :class:`CsvReplayFcAdapter`. The legacy (video, tlog) auto-sync
branch was removed; ``imu_csv_path`` is required.
""" """
from __future__ import annotations from __future__ import annotations
@@ -48,13 +55,8 @@ from gps_denied_onboard.components.c8_fc_adapter.replay_sink import (
from gps_denied_onboard.config import Config from gps_denied_onboard.config import Config
from gps_denied_onboard.fdr_client import make_fdr_client from gps_denied_onboard.fdr_client import make_fdr_client
from gps_denied_onboard.frame_source.video_file import VideoFileFrameSource from gps_denied_onboard.frame_source.video_file import VideoFileFrameSource
from gps_denied_onboard.helpers.wgs_converter import WgsConverter
from gps_denied_onboard.logging import get_logger from gps_denied_onboard.logging import get_logger
from gps_denied_onboard.replay_input import ( from gps_denied_onboard.replay_input import ReplayInputBundle
AutoSyncConfig,
ReplayInputAdapter,
ReplayInputBundle,
)
from gps_denied_onboard.replay_input.tlog_video_adapter import ReplayPace from gps_denied_onboard.replay_input.tlog_video_adapter import ReplayPace
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -77,10 +79,6 @@ REPLAY_BUILD_FLAGS: Final[tuple[str, ...]] = (
"BUILD_REPLAY_SINK_JSONL", "BUILD_REPLAY_SINK_JSONL",
) )
# AZ-894: separate build flag for the CSV adapter so the replay binary
# can opt into the new path without disturbing the BUILD_TLOG_* gate
# (the tlog adapter is still composed by _build_replay_input_bundle's
# legacy branch until AZ-895 removes it).
_CSV_REPLAY_BUILD_FLAG: Final[str] = "BUILD_CSV_REPLAY_ADAPTER" _CSV_REPLAY_BUILD_FLAG: Final[str] = "BUILD_CSV_REPLAY_ADAPTER"
@@ -106,7 +104,6 @@ def build_replay_components(
config: Config, config: Config,
*, *,
fdr_client_factory: Any | None = None, fdr_client_factory: Any | None = None,
replay_input_adapter_factory: Any | None = None,
sink_factory: Any | None = None, sink_factory: Any | None = None,
transport_factory: Any | None = None, transport_factory: Any | None = None,
) -> tuple[dict[str, Any], tuple[str, ...]]: ) -> tuple[dict[str, Any], tuple[str, ...]]:
@@ -137,11 +134,10 @@ def build_replay_components(
sink_fdr_client = fdr_factory("c8_fc_adapter.replay_sink", config) sink_fdr_client = fdr_factory("c8_fc_adapter.replay_sink", config)
# AZ-558: build the outbound MAVLink transport BEFORE the FC adapter # AZ-558: build the outbound MAVLink transport BEFORE the FC adapter
# so it can be threaded through `ReplayInputAdapter` and into # so the same instance can be exposed as the ``mavlink_transport``
# `TlogReplayFcAdapter`. The same instance is exposed as the # slot in ``components`` (replay protocol Invariant 5: encoders
# ``mavlink_transport`` slot in ``components`` (replay protocol # write through the seam in both modes; replay drops the bytes via
# Invariant 5: encoders write through the seam in both modes; # NoopMavlinkTransport).
# replay drops the bytes via NoopMavlinkTransport).
if transport_factory is not None: if transport_factory is not None:
transport = transport_factory(config) transport = transport_factory(config)
else: else:
@@ -150,7 +146,6 @@ def build_replay_components(
bundle = _build_replay_input_bundle( bundle = _build_replay_input_bundle(
config, config,
fdr_client=fdr_client, fdr_client=fdr_client,
adapter_factory=replay_input_adapter_factory,
mavlink_transport=transport, mavlink_transport=transport,
) )
@@ -187,19 +182,19 @@ def _validate_build_flags() -> None:
def _validate_replay_paths(config: Config) -> None: def _validate_replay_paths(config: Config) -> None:
"""Reject empty / missing replay paths early with a precise message. """Reject empty / missing replay paths early with a precise message.
AZ-894: ``imu_csv_path`` is the canonical replay input. ``tlog_path`` AZ-895: ``imu_csv_path`` is the only supported replay input. The
remains valid for the legacy auto-sync path until AZ-895 removes it, legacy tlog auto-sync surface was deprecated in AZ-895 and will be
but exactly one of the two must be set so the composition root can physically removed in AZ-908; until then ``tlog_path`` may remain
pick a single branch. set in the config but is ignored by composition.
""" """
if not config.replay.video_path: if not config.replay.video_path:
raise CompositionError( raise CompositionError(
"config.replay.video_path is empty; replay mode requires a video path" "config.replay.video_path is empty; replay mode requires a video path"
) )
if not config.replay.imu_csv_path and not config.replay.tlog_path: if not config.replay.imu_csv_path:
raise CompositionError( raise CompositionError(
"config.replay.imu_csv_path is empty and no tlog_path fallback is set; " "config.replay.imu_csv_path is empty; "
"replay mode requires an IMU+GPS CSV (AZ-894) or a tlog file (legacy)" "replay mode requires an IMU+GPS CSV (--imu PATH.csv)"
) )
if not config.replay.output_path: if not config.replay.output_path:
raise CompositionError( raise CompositionError(
@@ -211,61 +206,27 @@ def _build_replay_input_bundle(
config: Config, config: Config,
*, *,
fdr_client: "FdrClient", fdr_client: "FdrClient",
adapter_factory: Any | None,
mavlink_transport: Any | None = None, mavlink_transport: Any | None = None,
) -> ReplayInputBundle: ) -> ReplayInputBundle:
"""Build the replay input bundle and open the underlying strategies. """Build the replay input bundle and open the underlying strategies.
AZ-894: branches on ``config.replay.imu_csv_path`` when set, builds AZ-895: the (video, CSV) path is the only supported composition.
the :class:`CsvReplayFcAdapter` + :class:`VideoFileFrameSource` pair The legacy (video, tlog) auto-sync branch was removed; the
on a single canonical clock derived from the CSV's ``Time`` column; :func:`_validate_replay_paths` gate above guarantees
when unset, falls back to the legacy :class:`ReplayInputAdapter` ``imu_csv_path`` is set before this function runs.
tlog path (auto-sync + AC-9 validator). AZ-895 removes the legacy
branch.
""" """
pace = _resolve_pace(config.replay.pace) pace = _resolve_pace(config.replay.pace)
target_fc_dialect = _resolve_fc_kind(config.replay.target_fc_dialect) target_fc_dialect = _resolve_fc_kind(config.replay.target_fc_dialect)
camera_calibration = _load_camera_calibration(config) camera_calibration = _load_camera_calibration(config)
wgs_converter = WgsConverter()
if config.replay.imu_csv_path: return _build_csv_bundle(
return _build_csv_bundle( config,
config, fdr_client=fdr_client,
fdr_client=fdr_client, pace=pace,
pace=pace, target_fc_dialect=target_fc_dialect,
target_fc_dialect=target_fc_dialect, camera_calibration=camera_calibration,
camera_calibration=camera_calibration, mavlink_transport=mavlink_transport,
mavlink_transport=mavlink_transport, )
)
auto_sync = _build_auto_sync_config(config)
if adapter_factory is not None:
adapter = adapter_factory(
config=config,
camera_calibration=camera_calibration,
target_fc_dialect=target_fc_dialect,
wgs_converter=wgs_converter,
fdr_client=fdr_client,
pace=pace,
auto_sync_config=auto_sync,
mavlink_transport=mavlink_transport,
)
else:
adapter = ReplayInputAdapter(
video_path=Path(config.replay.video_path),
tlog_path=Path(config.replay.tlog_path),
camera_calibration=camera_calibration,
target_fc_dialect=target_fc_dialect,
wgs_converter=wgs_converter,
fdr_client=fdr_client,
pace=pace,
manual_time_offset_ms=config.replay.time_offset_ms,
skip_auto_sync_validation=config.replay.skip_auto_sync_validation,
auto_trim=config.replay.auto_trim,
auto_sync_config=auto_sync,
mavlink_transport=mavlink_transport,
)
return adapter.open()
def _build_csv_bundle( def _build_csv_bundle(
@@ -339,35 +300,6 @@ def _resolve_fc_kind(raw: str) -> FcKind:
) )
def _build_auto_sync_config(config: Config) -> AutoSyncConfig:
block = config.replay.auto_sync
return AutoSyncConfig(
takeoff_accel_threshold_g=block.takeoff_accel_threshold_g,
takeoff_attitude_rate_threshold_rad_s=(
block.takeoff_attitude_rate_threshold_rad_s
),
sustained_seconds=block.sustained_seconds,
prescan_max_messages=block.prescan_max_messages,
video_motion_threshold=block.video_motion_threshold,
video_motion_scan_seconds=block.video_motion_scan_seconds,
match_threshold_pct=block.match_threshold_pct,
match_window_ms=block.match_window_ms,
low_confidence_threshold=block.low_confidence_threshold,
alignment_resample_hz=block.alignment_resample_hz,
alignment_video_scan_seconds=block.alignment_video_scan_seconds,
alignment_low_confidence_threshold=block.alignment_low_confidence_threshold,
alignment_segment_motion_threshold_g=(
block.alignment_segment_motion_threshold_g
),
alignment_segment_min_flight_duration_seconds=(
block.alignment_segment_min_flight_duration_seconds
),
alignment_segment_max_internal_gap_seconds=(
block.alignment_segment_max_internal_gap_seconds
),
)
def _load_camera_calibration(config: Config) -> CameraCalibration: def _load_camera_calibration(config: Config) -> CameraCalibration:
"""Read the camera calibration JSON into a :class:`CameraCalibration` DTO. """Read the camera calibration JSON into a :class:`CameraCalibration` DTO.
@@ -427,12 +359,11 @@ def _log_ready(config: Config, bundle: ReplayInputBundle) -> None:
"kind": _LOG_KIND_READY, "kind": _LOG_KIND_READY,
"kv": { "kv": {
"video_path": config.replay.video_path, "video_path": config.replay.video_path,
"tlog_path": config.replay.tlog_path, "imu_csv_path": config.replay.imu_csv_path,
"output_path": config.replay.output_path, "output_path": config.replay.output_path,
"pace": config.replay.pace, "pace": config.replay.pace,
"resolved_offset_ms": bundle.resolved_time_offset_ms, "resolved_offset_ms": bundle.resolved_time_offset_ms,
"calib_path": config.runtime.camera_calibration_path, "calib_path": config.runtime.camera_calibration_path,
"auto_sync_used": bundle.auto_sync_result is not None,
}, },
}, },
) )
+14 -19
View File
@@ -173,25 +173,20 @@ def _load_full_ground_truth(tlog_path: Path) -> list[GroundTruthRow]:
@pytest.mark.tier2 @pytest.mark.tier2
@pytest.mark.xfail( @pytest.mark.xfail(
reason=( reason=(
"Blocked by AZ-776 + AZ-777. AZ-699 was implemented without " "Blocked by AZ-848 (+ AZ-883). The tlog adapter path is "
"executing this test end-to-end on Tier-2 Jetson; once the " "structurally broken at the clock layer: VioOutput.emitted_at_ns "
"fixtures (real video + factory calibration) landed and the " "is sourced from process monotonic_ns (klt_ransac.py:274) while "
"test ran for real, two upstream gaps surfaced: (1) AZ-776 " "ImuWindow.ts_end_ns comes from the FC IMU timebase, so the C5 "
"— c4_pose ISam2GraphHandle Protocol rejects the ESKF stub " "ESKF immediately diverges (mahalanobis² > 100) at frame 3 "
"handle, so the c5_state=eskf composition variant cannot run; " "with c5.state.eskf_out_of_order. AZ-883 is the related "
"(2) AZ-777 — Derkachi has no C6 reference tile cache / " "SCALED_IMU2.ts_ns=0 default that exacerbates the mismatch on "
"descriptor index, so the default c5_state=gtsam_isam2 " "some tlogs. AZ-895 made the (video, CSV) path the primary "
"composition reaches the per-frame loop but iSAM2.update " "replay surface, so AZ-848 is no longer bench-blocking — but "
"fails at frame 1 with key 'x2' not in Values (no C4 anchor " "this test still exercises the legacy tlog path and will "
"was ever inserted because C2/C3/C4 have nothing to match " "remain xfail until AZ-848 / AZ-883 ship. The AZ-776 / AZ-777 "
"against). Per AZ-777 AC-4: 'After AZ-776 + this ticket " "gaps that originally surfaced via this test were closed in "
"both ship, test_ac3_within_100m_80pct_of_ticks can be " "cycle 3, but the clock-mismatch root cause then surfaced "
"un-xfail'd and pass'. The AZ-699 verdict-on-real-flight is " "underneath."
"tracked under those tickets; this xfail is the documented "
"mask until they ship. NOTE: this contradicts AZ-699 AC-1 "
"('no @xfail mask'); the dependency gap was discovered "
"post-implementation when the Jetson e2e harness ran for "
"the first time."
), ),
strict=False, strict=False,
) )
@@ -1,483 +0,0 @@
"""AZ-405 — auto-sync detector + offset-compute + AC-9 validator.
Covers AC-1..AC-10 of ``_docs/02_tasks/todo/AZ-405_replay_auto_sync.md``.
Tests run against the pure compute kernels in
:mod:`gps_denied_onboard.replay_input.auto_sync` (no disk IO, no real
pymavlink, no real OpenCV) so the suite is fast + deterministic.
Style: every test follows the Arrange / Act / Assert pattern.
"""
from __future__ import annotations
import math
from typing import Any
import pytest
from gps_denied_onboard.replay_input.auto_sync import (
TlogSamples,
_compute_tlog_takeoff_from_samples,
_compute_video_onset_from_samples,
compute_offset,
validate_offset_or_fail,
)
from gps_denied_onboard.replay_input.errors import ReplayInputAdapterError
from gps_denied_onboard.replay_input.interface import AutoSyncConfig
# ---------------------------------------------------------------------
# Synthetic-fixture helpers
def _ns(seconds: float) -> int:
return int(seconds * 1_000_000_000)
def _flat_accel_samples(
*, start_s: float, end_s: float, hz: int, total_g: float
) -> list[tuple[int, float]]:
out: list[tuple[int, float]] = []
period = 1.0 / hz
t = start_s
while t < end_s:
out.append((_ns(t), total_g))
t += period
return out
def _flat_attitude_samples(
*, start_s: float, end_s: float, hz: int, roll: float, pitch: float, yaw: float
) -> list[tuple[int, float, float, float]]:
out: list[tuple[int, float, float, float]] = []
period = 1.0 / hz
t = start_s
while t < end_s:
out.append((_ns(t), roll, pitch, yaw))
t += period
return out
def _ramp_attitude_samples(
*,
start_s: float,
end_s: float,
hz: int,
base_roll: float,
base_pitch: float,
base_yaw: float,
rate_rad_s: float,
) -> list[tuple[int, float, float, float]]:
"""Attitude that ramps in pitch at ``rate_rad_s`` rad/s."""
out: list[tuple[int, float, float, float]] = []
period = 1.0 / hz
t = start_s
while t < end_s:
dt = t - start_s
pitch = base_pitch + rate_rad_s * dt
out.append((_ns(t), base_roll, pitch, base_yaw))
t += period
return out
def _build_takeoff_samples() -> TlogSamples:
"""AC-1 fixture: 2 s flat hover, then 1.5 s sustained 2.2 g + 1.5 rad/s.
Take-off onset is at t = 2.0 s (the first sample with the
elevated acceleration). Body-frame accelerometer convention: at
hover the proper-acceleration magnitude is 1 g (gravity reaction);
during a 1.2 g thrust climb it is 2.2 g, so the take-off excess
above 1 g rest is 1.2 g well above the 0.5 g threshold.
"""
accel_pre = _flat_accel_samples(start_s=0.0, end_s=2.0, hz=200, total_g=1.0)
accel_post = _flat_accel_samples(start_s=2.0, end_s=3.5, hz=200, total_g=2.2)
accel = accel_pre + accel_post
attitude_pre = _flat_attitude_samples(
start_s=0.0, end_s=2.0, hz=100, roll=0.0, pitch=0.0, yaw=0.0
)
attitude_post = _ramp_attitude_samples(
start_s=2.0,
end_s=3.5,
hz=100,
base_roll=0.0,
base_pitch=0.0,
base_yaw=0.0,
rate_rad_s=1.5,
)
attitude = attitude_pre + attitude_post
return TlogSamples(
accel=tuple(accel),
attitude=tuple(attitude),
imu_count_by_type={
"RAW_IMU": len(accel),
"ATTITUDE": len(attitude),
},
)
def _build_low_amplitude_vibration_samples() -> TlogSamples:
"""AC-2 fixture: 5 s of 0.3 g body-frame vibration (no take-off).
Total proper-acceleration during vibration = 1.3 g (0.3 g excess
above the 1 g rest baseline) strictly below the 0.5 g detector
threshold so the sustained-event search rejects every window.
"""
accel = _flat_accel_samples(start_s=0.0, end_s=5.0, hz=200, total_g=1.3)
attitude = _flat_attitude_samples(
start_s=0.0, end_s=5.0, hz=100, roll=0.0, pitch=0.0, yaw=0.0
)
return TlogSamples(
accel=tuple(accel),
attitude=tuple(attitude),
imu_count_by_type={
"RAW_IMU": len(accel),
"ATTITUDE": len(attitude),
},
)
def _build_hand_launch_samples() -> TlogSamples:
"""AC-3 fixture: 0.8 g impulse for 100 ms; not sustained for 0.5 s.
Body-frame accelerometer convention (see ``_build_takeoff_samples``):
a 0.8 g impulse becomes 1.8 g total proper-acceleration during the
impulse window.
"""
accel_pre = _flat_accel_samples(start_s=0.0, end_s=2.0, hz=200, total_g=1.0)
accel_impulse = _flat_accel_samples(
start_s=2.0, end_s=2.1, hz=200, total_g=1.8
)
accel_post = _flat_accel_samples(start_s=2.1, end_s=3.0, hz=200, total_g=1.0)
accel = accel_pre + accel_impulse + accel_post
attitude = _flat_attitude_samples(
start_s=0.0, end_s=3.0, hz=100, roll=0.0, pitch=0.0, yaw=0.0
)
return TlogSamples(
accel=tuple(accel),
attitude=tuple(attitude),
imu_count_by_type={
"RAW_IMU": len(accel),
"ATTITUDE": len(attitude),
},
)
def _flow_samples_from_frames(
*, n_stationary: int, n_moving: int, fps: int = 30, motion_px: float = 4.0
) -> tuple[tuple[int, float], ...]:
"""Synthesise the flow-magnitude series the video detector consumes.
The detector emits a ``(ts_ns, mean_magnitude_px)`` tuple for each
consecutive frame pair (skipping the first frame's pair). For
AC-4 we pretend frames 1..10 are stationary (mag 0) and frames
11..60 are moving (mag = motion_px).
"""
out: list[tuple[int, float]] = []
period_ns = int(1_000_000_000 / fps)
for i in range(1, n_stationary):
out.append((i * period_ns, 0.05))
for j in range(n_stationary, n_stationary + n_moving):
out.append((j * period_ns, motion_px))
return tuple(out)
# ---------------------------------------------------------------------
# AC-1 — tlog take-off detector (positive)
def test_ac1_tlog_takeoff_detector_positive_within_50ms_and_high_confidence() -> None:
# Arrange
config = AutoSyncConfig()
samples = _build_takeoff_samples()
# Act
result = _compute_tlog_takeoff_from_samples(samples, config)
# Assert
expected_onset_ns = _ns(2.0)
assert abs(result.onset_ns - expected_onset_ns) <= _ns(0.05), (
f"detected onset {result.onset_ns / 1e9}s deviates >50ms from expected 2.0s"
)
assert result.confidence >= 0.85, (
f"confidence {result.confidence} below AC-1 minimum of 0.85"
)
# ---------------------------------------------------------------------
# AC-2 — tlog take-off detector (ambiguous)
def test_ac2_tlog_takeoff_detector_low_amplitude_vibration_low_confidence() -> None:
# Arrange
config = AutoSyncConfig()
samples = _build_low_amplitude_vibration_samples()
# Act
result = _compute_tlog_takeoff_from_samples(samples, config)
# Assert
assert result.confidence < 0.50, (
f"confidence {result.confidence} should be < 0.50 for ambiguous vibration"
)
# ---------------------------------------------------------------------
# AC-3 — tlog take-off detector (hand launch)
def test_ac3_tlog_takeoff_detector_hand_launch_warn_regime() -> None:
# Arrange
config = AutoSyncConfig()
samples = _build_hand_launch_samples()
# Act
result = _compute_tlog_takeoff_from_samples(samples, config)
# Assert
assert result.confidence < 0.80, (
f"confidence {result.confidence} should be < 0.80 for unsustained hand-launch"
)
# ---------------------------------------------------------------------
# AC-4 — video motion-onset detector
def test_ac4_video_motion_onset_detected_within_one_frame() -> None:
# Arrange
config = AutoSyncConfig()
flow_samples = _flow_samples_from_frames(n_stationary=10, n_moving=50, fps=30)
period_ns = int(1_000_000_000 / 30)
expected_onset_ns = 10 * period_ns
# Act
result = _compute_video_onset_from_samples(flow_samples, config)
# Assert
assert abs(result.onset_ns - expected_onset_ns) <= period_ns, (
f"detected motion onset {result.onset_ns} ns deviates >1 frame "
f"from expected {expected_onset_ns} ns"
)
assert result.confidence > 0.80, (
f"confidence {result.confidence} too low for clear motion onset"
)
# ---------------------------------------------------------------------
# AC-5 — combined offset within ± 200 ms
def test_ac5_combined_offset_within_200ms_of_ground_truth() -> None:
# Arrange
config = AutoSyncConfig()
tlog_samples = _build_takeoff_samples()
tlog_result = _compute_tlog_takeoff_from_samples(tlog_samples, config)
flow_samples = _flow_samples_from_frames(n_stationary=10, n_moving=50, fps=30)
video_result = _compute_video_onset_from_samples(flow_samples, config)
# Ground-truth offset = tlog take-off (2.0 s) video onset (10/30 s)
period_ns = int(1_000_000_000 / 30)
ground_truth_offset_ms = (_ns(2.0) - 10 * period_ns) // 1_000_000
# Act
decision = compute_offset(tlog_result, video_result)
# Assert
assert abs(decision.offset_ms - ground_truth_offset_ms) <= 200, (
f"offset {decision.offset_ms} ms deviates >200 ms from ground truth "
f"{ground_truth_offset_ms} ms"
)
# ---------------------------------------------------------------------
# AC-6 — low combined confidence (verified via the coordinator test
# in test_az405_replay_input_adapter.py; here we only verify the
# combined-confidence aggregator picks min())
def test_ac6_combined_confidence_takes_minimum_of_inputs() -> None:
# Arrange
from gps_denied_onboard.replay_input.auto_sync import _DetectorResult
high = _DetectorResult(onset_ns=_ns(1.0), confidence=0.95)
low = _DetectorResult(onset_ns=_ns(2.0), confidence=0.50)
# Act
decision = compute_offset(high, low)
# Assert
assert decision.combined_confidence == pytest.approx(0.50)
assert decision.offset_ms == (_ns(1.0) - _ns(2.0)) // 1_000_000
# ---------------------------------------------------------------------
# AC-7 — AC-9 validator hard-fail (the coordinator-level raise is
# covered in test_az405_replay_input_adapter.py)
def test_ac7_validator_hard_fail_returns_2_for_offset_outside_window() -> None:
# Arrange
fps = 30
period_ns = int(1_000_000_000 / fps)
video_ts = [i * period_ns for i in range(60)]
# IMU sampled at 200 Hz from t=0 to t=2 (mismatch deliberate; the
# bad offset shifts everything outside the window).
imu_ts = [int(i / 200 * 1_000_000_000) for i in range(400)]
bad_offset_ms = 60_000
# Act
code = validate_offset_or_fail(
bad_offset_ms,
imu_ts,
video_ts,
threshold_pct=95.0,
window_ms=100,
)
# Assert
assert code == 2
# ---------------------------------------------------------------------
# AC-9 — frame-window match-percentage validator (positive)
def test_ac9_validator_passes_for_well_matched_offset() -> None:
# Arrange
fps = 30
period_ns = int(1_000_000_000 / fps)
video_ts = [i * period_ns for i in range(60)]
# IMU samples densely spanning the same time range — every video
# frame has an IMU sample within ± 100 ms.
imu_ts = [int(i / 200 * 1_000_000_000) for i in range(60 * 200 // 30)]
# Act
code = validate_offset_or_fail(
0, imu_ts, video_ts, threshold_pct=95.0, window_ms=100
)
# Assert
assert code == 0
def test_ac9_threshold_configurable() -> None:
# Arrange — set up a series where exactly 80% of frames match.
fps = 30
period_ns = int(1_000_000_000 / fps)
video_ts = [i * period_ns for i in range(50)]
# IMU only covers the first 80% of the video timeline; the last
# 10 frames will be far outside the window.
imu_ts = [
int(i / 200 * 1_000_000_000) for i in range(int(40 / 30 * 200))
]
# Act / Assert
# Default 95% threshold → fail (80% < 95%).
assert validate_offset_or_fail(
0, imu_ts, video_ts, threshold_pct=95.0, window_ms=100
) == 2
# Lowered to 75% → pass.
assert validate_offset_or_fail(
0, imu_ts, video_ts, threshold_pct=75.0, window_ms=100
) == 0
# ---------------------------------------------------------------------
# AC-10 — confidence determinism
def test_ac10_confidence_score_deterministic_across_two_runs() -> None:
# Arrange
config = AutoSyncConfig()
samples = _build_takeoff_samples()
# Act
first = _compute_tlog_takeoff_from_samples(samples, config)
second = _compute_tlog_takeoff_from_samples(samples, config)
# Assert
assert first.onset_ns == second.onset_ns
assert math.isclose(first.confidence, second.confidence, abs_tol=1e-9)
def test_ac10_video_onset_deterministic_across_two_runs() -> None:
# Arrange
config = AutoSyncConfig()
flow_samples = _flow_samples_from_frames(n_stationary=5, n_moving=20, fps=30)
# Act
first = _compute_video_onset_from_samples(flow_samples, config)
second = _compute_video_onset_from_samples(flow_samples, config)
# Assert
assert first.onset_ns == second.onset_ns
assert math.isclose(first.confidence, second.confidence, abs_tol=1e-9)
# ---------------------------------------------------------------------
# R-DEMO-3 fail-fast on the pure compute path
def test_pure_takeoff_kernel_raises_on_no_imu_samples() -> None:
# Arrange
config = AutoSyncConfig()
samples = TlogSamples(
accel=(),
attitude=(),
imu_count_by_type={"ATTITUDE": 100},
)
# Act / Assert
with pytest.raises(ReplayInputAdapterError, match="tlog missing required"):
_compute_takeoff_or_propagate(samples, config)
def test_pure_takeoff_kernel_raises_on_no_attitude_samples() -> None:
# Arrange
config = AutoSyncConfig()
accel = _flat_accel_samples(start_s=0.0, end_s=1.0, hz=200, total_g=1.0)
samples = TlogSamples(
accel=tuple(accel),
attitude=(),
imu_count_by_type={"RAW_IMU": len(accel)},
)
# Act / Assert
with pytest.raises(ReplayInputAdapterError, match="tlog missing required"):
_compute_takeoff_or_propagate(samples, config)
def _compute_takeoff_or_propagate(samples: TlogSamples, config: AutoSyncConfig) -> Any:
"""Local trampoline so the assertions are explicit even if the
underscore-named helper migrates."""
return _compute_tlog_takeoff_from_samples(samples, config)
# ---------------------------------------------------------------------
# AC-9 edge cases
def test_validator_returns_2_on_empty_video_or_tlog() -> None:
# Arrange
imu_ts = [0, 1_000_000, 2_000_000]
video_ts: list[int] = []
# Act / Assert — empty video
assert (
validate_offset_or_fail(
0, imu_ts, video_ts, threshold_pct=95.0, window_ms=100
)
== 2
)
# Empty tlog
assert (
validate_offset_or_fail(
0, [], [0, 1_000_000], threshold_pct=95.0, window_ms=100
)
== 2
)
@@ -1,804 +0,0 @@
"""AZ-405 — ``ReplayInputAdapter`` coordinator unit tests.
Covers AC-6 (low-confidence WARN-and-proceed), AC-7 (AC-8 hard-fail),
AC-8 (manual override bypass), AC-11 (open() returns a complete
bundle), AC-12 (idempotent close), and AC-13 (R-DEMO-3 fail-fast on
missing tlog message types).
Synthetic videos use the same OpenCV-driven fixture pattern as
``tests/unit/frame_source/test_protocol_conformance.py``; the tlog
side is faked via the coordinator's ``tlog_source_factory`` injection
point so tests run without a real pymavlink connection.
Style: every test follows the Arrange / Act / Assert pattern.
"""
from __future__ import annotations
from pathlib import Path
from types import SimpleNamespace
from typing import Any
from unittest import mock
import cv2
import numpy as np
import pytest
from gps_denied_onboard._types.calibration import CameraCalibration
from gps_denied_onboard._types.fc import FcKind
from gps_denied_onboard.clock.tlog_derived import TlogDerivedClock
from gps_denied_onboard.clock.wall_clock import WallClock
from gps_denied_onboard.components.c8_fc_adapter.tlog_replay_adapter import (
ReplayPace,
TlogReplayFcAdapter,
)
from gps_denied_onboard.frame_source.video_file import VideoFileFrameSource
from gps_denied_onboard.replay_input.errors import ReplayInputAdapterError
from gps_denied_onboard.replay_input.interface import (
AutoSyncConfig,
AutoSyncDecision,
ReplayInputBundle,
)
from gps_denied_onboard.replay_input.tlog_video_adapter import ReplayInputAdapter
# ---------------------------------------------------------------------
# Fixtures
@pytest.fixture(autouse=True)
def _enable_build_flags(monkeypatch: pytest.MonkeyPatch) -> None:
"""Both downstream strategies are gated by build flags (AZ-398 / AZ-399)."""
monkeypatch.setenv("BUILD_VIDEO_FILE_FRAME_SOURCE", "ON")
monkeypatch.setenv("BUILD_TLOG_REPLAY_ADAPTER", "ON")
@pytest.fixture
def fake_fdr_client() -> mock.MagicMock:
return mock.MagicMock(name="FdrClient")
@pytest.fixture
def fake_wgs_converter() -> mock.MagicMock:
return mock.MagicMock(name="WgsConverter")
@pytest.fixture
def camera_calibration() -> CameraCalibration:
return CameraCalibration(
camera_id="az405-test",
intrinsics_3x3=None,
distortion=None,
body_to_camera_se3=None,
acquisition_method="synthetic",
)
def _make_synthetic_video(path: Path, n_frames: int = 60, fps: int = 30) -> Path:
"""Write a 64×48 BGR MP4V file at ``path`` and return it."""
fourcc = cv2.VideoWriter_fourcc(*"mp4v")
writer = cv2.VideoWriter(str(path), fourcc, fps, (64, 48))
if not writer.isOpened():
raise RuntimeError(f"OpenCV could not open writer at {path!s}")
try:
for i in range(n_frames):
frame = np.full((48, 64, 3), i % 256, dtype=np.uint8)
writer.write(frame)
finally:
writer.release()
return path
@pytest.fixture
def synthetic_video(tmp_path: Path) -> Path:
return _make_synthetic_video(tmp_path / "az405-video.mp4", n_frames=60, fps=30)
@pytest.fixture
def synthetic_tlog_path(tmp_path: Path) -> Path:
p = tmp_path / "az405.tlog"
p.write_bytes(b"fake-tlog")
return p
# ---------------------------------------------------------------------
# Fake pymavlink source
def _ns(seconds: float) -> int:
return int(seconds * 1_000_000_000)
def _fake_msg(msg_type: str, *, ts_s: float, **fields: Any) -> SimpleNamespace:
ns = SimpleNamespace(_timestamp=ts_s, **fields)
ns.get_type = lambda: msg_type
return ns
def _build_takeoff_messages(
*,
accel_pre_total_g: float = 1.0,
accel_post_total_g: float = 2.2,
accel_hz: int = 200,
pre_seconds: float = 2.0,
post_seconds: float = 1.5,
) -> list[SimpleNamespace]:
"""A short tlog stream with a clear take-off pattern + GPS + heartbeat."""
out: list[SimpleNamespace] = []
accel_period = 1.0 / accel_hz
# Pre-takeoff: 1 g hover (z-acc = -1g in body, after sign).
t = 0.0
while t < pre_seconds:
out.append(
_fake_msg(
"RAW_IMU",
ts_s=t,
xacc=0,
yacc=0,
zacc=-int(accel_pre_total_g * 1000),
xgyro=0,
ygyro=0,
zgyro=0,
)
)
t += accel_period
# Post-takeoff: 2.2 g sustained climb thrust.
while t < pre_seconds + post_seconds:
out.append(
_fake_msg(
"RAW_IMU",
ts_s=t,
xacc=0,
yacc=0,
zacc=-int(accel_post_total_g * 1000),
xgyro=0,
ygyro=0,
zgyro=0,
)
)
t += accel_period
# Attitude: flat hover then 1.5 rad/s pitch ramp.
t = 0.0
attitude_period = 1.0 / 100.0
while t < pre_seconds:
out.append(
_fake_msg("ATTITUDE", ts_s=t, roll=0.0, pitch=0.0, yaw=0.0)
)
t += attitude_period
pitch_rate = 1.5
while t < pre_seconds + post_seconds:
dt = t - pre_seconds
out.append(
_fake_msg(
"ATTITUDE",
ts_s=t,
roll=0.0,
pitch=pitch_rate * dt,
yaw=0.0,
)
)
t += attitude_period
# GPS_RAW_INT + HEARTBEAT (required by AZ-399 pre-scan).
out.append(
_fake_msg(
"GPS_RAW_INT",
ts_s=0.0,
fix_type=3,
lat=499910000,
lon=362210000,
alt=153_400,
)
)
out.append(_fake_msg("HEARTBEAT", ts_s=0.0, system_status=4, base_mode=0))
out.sort(key=lambda m: m._timestamp)
return out
class _FakeTlog:
"""Minimal pymavlink ``mavlink_connection`` stand-in.
Returns each stored message once on ``recv_match``; ignores the
``type=`` filter (the AZ-399 decode loop receives unfiltered
HEARTBEAT/IMU/ATTITUDE/GPS streams).
"""
def __init__(self, messages: list[SimpleNamespace]) -> None:
self._iter = iter(messages)
self.closed = False
def recv_match(self, **_kwargs: Any) -> SimpleNamespace | None:
return next(self._iter, None)
def close(self) -> None:
self.closed = True
def _factory_for(messages: list[SimpleNamespace]) -> Any:
"""Return a source factory that yields a fresh ``_FakeTlog`` per call.
The coordinator opens the tlog twice (once for ``_load_tlog_samples``
in the auto-sync path, once via the FC adapter's pre-scan + decode
handles), so the messages have to be re-emittable.
"""
def _factory(_path: str) -> _FakeTlog:
return _FakeTlog(list(messages))
return _factory
def _frames_factory_with_motion(
*,
n_stationary: int = 10,
n_moving: int = 50,
fps: int = 30,
) -> Any:
"""Return a frames_factory yielding the AC-4 motion-onset shape."""
period_ns = int(1_000_000_000 / fps)
rng = np.random.default_rng(seed=0)
def _factory(_path: Path, _scan_seconds: float) -> Any:
out: list[tuple[int, np.ndarray]] = []
# Stationary: identical frames so optical flow ≈ 0.
base = np.full((48, 64, 3), 128, dtype=np.uint8)
for i in range(n_stationary):
out.append((i * period_ns, base.copy()))
# Moving: each frame replaces a 16×16 patch at a random offset
# so Farneback returns a clear non-zero magnitude. Determinism
# is preserved by the seeded RNG.
for j in range(n_moving):
frame = base.copy()
r = rng.integers(0, 32)
c = rng.integers(0, 48)
frame[r : r + 16, c : c + 16, :] = 240
out.append(((n_stationary + j) * period_ns, frame))
return out
return _factory
def _video_timestamps_factory(
*,
n_frames: int = 60,
fps: int = 30,
) -> Any:
"""Return a timestamps_factory with deterministic per-frame ts (ns)."""
period_ns = int(1_000_000_000 / fps)
def _factory(_path: Path) -> list[int]:
return [i * period_ns for i in range(n_frames)]
return _factory
# ---------------------------------------------------------------------
# AC-11 — open() returns a complete bundle
def test_ac11_open_returns_complete_bundle_with_correct_strategies(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=0,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act
try:
bundle = adapter.open()
# Assert
assert isinstance(bundle, ReplayInputBundle)
assert isinstance(bundle.frame_source, VideoFileFrameSource)
assert isinstance(bundle.fc_adapter, TlogReplayFcAdapter)
assert isinstance(bundle.clock, TlogDerivedClock)
assert bundle.resolved_time_offset_ms == 0
# Manual path → no auto-sync result.
assert bundle.auto_sync_result is None
finally:
adapter.close()
def test_ac11_pace_realtime_yields_wall_clock(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.REALTIME,
manual_time_offset_ms=0,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act
try:
bundle = adapter.open()
# Assert
assert isinstance(bundle.clock, WallClock)
finally:
adapter.close()
def test_ac11_pace_asap_yields_tlog_derived_clock(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=0,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act
try:
bundle = adapter.open()
# Assert
assert isinstance(bundle.clock, TlogDerivedClock)
finally:
adapter.close()
# ---------------------------------------------------------------------
# AC-12 — idempotent close
def test_ac12_close_is_idempotent(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=0,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
adapter.open()
# Act / Assert — both calls must complete without raising.
adapter.close()
adapter.close()
def test_close_without_open_does_not_raise(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=0,
auto_sync_config=AutoSyncConfig(),
)
# Act / Assert
adapter.close()
# ---------------------------------------------------------------------
# AC-13 — missing tlog messages fail fast
def test_ac13_missing_imu_messages_fails_fast_before_video_read(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange — tlog has only ATTITUDE; no RAW_IMU / SCALED_IMU2.
attitude_only = [
_fake_msg("ATTITUDE", ts_s=t * 0.01, roll=0.0, pitch=0.0, yaw=0.0)
for t in range(100)
]
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=0,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(attitude_only),
)
# Act / Assert
with pytest.raises(
ReplayInputAdapterError, match="tlog missing required message types"
):
adapter.open()
def test_ac13_missing_attitude_messages_fails_fast(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange — tlog has only RAW_IMU; no ATTITUDE.
imu_only = [
_fake_msg(
"RAW_IMU",
ts_s=t * 0.005,
xacc=0,
yacc=0,
zacc=-1000,
xgyro=0,
ygyro=0,
zgyro=0,
)
for t in range(100)
]
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=0,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(imu_only),
)
# Act / Assert
with pytest.raises(
ReplayInputAdapterError, match=r"tlog missing required message types.*ATTITUDE"
):
adapter.open()
# ---------------------------------------------------------------------
# AC-8 — manual override bypasses auto-detect
def test_ac8_manual_override_bypasses_auto_detect(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Arrange
detect_calls: list[Any] = []
def _explode_if_called(*args: Any, **kwargs: Any) -> Any:
detect_calls.append((args, kwargs))
raise AssertionError(
"auto-sync detector called even though manual_time_offset_ms was set"
)
monkeypatch.setattr(
"gps_denied_onboard.replay_input.tlog_video_adapter.detect_video_motion_onset",
_explode_if_called,
)
# Patch the take-off compute kernel referenced via the helper; the
# coordinator's manual path must skip it entirely.
monkeypatch.setattr(
"gps_denied_onboard.replay_input.auto_sync._compute_tlog_takeoff_from_samples",
_explode_if_called,
)
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=500,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act
try:
bundle = adapter.open()
# Assert — detector helpers were NOT invoked.
assert detect_calls == []
assert bundle.resolved_time_offset_ms == 500
assert bundle.auto_sync_result is None
finally:
adapter.close()
# ---------------------------------------------------------------------
# AC-7 — AC-8 hard-fail raises
def test_ac7_ac8_validator_hard_fail_raises_on_open(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
# Arrange — manual offset of 60 s will push every video frame
# outside the IMU coverage window (the fake tlog only carries
# ~3.5 s of samples).
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=60_000,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act / Assert
with pytest.raises(ReplayInputAdapterError, match="auto-sync hard-fail"):
adapter.open()
# ---------------------------------------------------------------------
# AZ-611 — skip_auto_sync_validation bypasses the AC-9 validator
def test_az611_skip_auto_sync_validation_bypasses_ac9(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
"""A manual offset that WOULD hard-fail AC-9 succeeds when the
operator explicitly opts out via ``skip_auto_sync_validation=True``.
Mirrors the AC-7 hard-fail scenario above so the bypass is the
only variable.
"""
# Arrange — same manual offset (60 s) that AC-7 above proves
# pushes every frame outside the IMU window.
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=60_000,
skip_auto_sync_validation=True,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act
try:
bundle = adapter.open()
# Assert — the bypass let the open() complete with the manual
# offset intact, even though the validator would have rejected it.
assert bundle.resolved_time_offset_ms == 60_000
assert bundle.auto_sync_result is None
finally:
adapter.close()
def test_az611_skip_auto_sync_validation_requires_manual_offset(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
) -> None:
"""Constructor refuses ``skip_auto_sync_validation=True`` paired
with ``manual_time_offset_ms=None`` (silent-zero guard).
"""
# Act / Assert
with pytest.raises(
ReplayInputAdapterError,
match=r"skip_auto_sync_validation=True requires.*manual_time_offset_ms",
):
ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=None,
skip_auto_sync_validation=True,
auto_sync_config=AutoSyncConfig(),
)
# ---------------------------------------------------------------------
# AC-6 — low combined confidence WARN-and-proceed
def test_ac6_low_confidence_warn_and_proceed_does_not_raise(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
# Arrange — stub the detectors to return low-confidence results.
from gps_denied_onboard.replay_input.auto_sync import _DetectorResult
low_conf = _DetectorResult(onset_ns=_ns(2.0), confidence=0.40)
def _stub_take_off(*args: Any, **kwargs: Any) -> _DetectorResult:
return low_conf
def _stub_motion_onset(*args: Any, **kwargs: Any) -> _DetectorResult:
return _DetectorResult(onset_ns=_ns(2.0), confidence=0.40)
monkeypatch.setattr(
"gps_denied_onboard.replay_input.auto_sync._compute_tlog_takeoff_from_samples",
_stub_take_off,
)
monkeypatch.setattr(
"gps_denied_onboard.replay_input.tlog_video_adapter.detect_video_motion_onset",
_stub_motion_onset,
)
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=None,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act
caplog.set_level("WARNING", logger="replay_input.tlog_video_adapter")
try:
bundle = adapter.open()
# Assert — open() returned the bundle (didn't raise) and the
# WARN log fired.
assert bundle.auto_sync_result is not None
assert bundle.auto_sync_result.combined_confidence == pytest.approx(0.40)
warn_kinds = [
r.kind for r in caplog.records if hasattr(r, "kind")
]
assert "replay.auto_sync.low_confidence" in warn_kinds
finally:
adapter.close()
def test_ac11_resolved_offset_matches_auto_sync_result(
synthetic_video: Path,
synthetic_tlog_path: Path,
camera_calibration: CameraCalibration,
fake_wgs_converter: mock.MagicMock,
fake_fdr_client: mock.MagicMock,
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Arrange — high-confidence stubs so AC-6 WARN does not fire.
from gps_denied_onboard.replay_input.auto_sync import _DetectorResult
def _stub_take_off(*args: Any, **kwargs: Any) -> _DetectorResult:
return _DetectorResult(onset_ns=_ns(2.0), confidence=0.95)
def _stub_motion_onset(*args: Any, **kwargs: Any) -> _DetectorResult:
return _DetectorResult(onset_ns=_ns(0.333), confidence=0.95)
monkeypatch.setattr(
"gps_denied_onboard.replay_input.auto_sync._compute_tlog_takeoff_from_samples",
_stub_take_off,
)
monkeypatch.setattr(
"gps_denied_onboard.replay_input.tlog_video_adapter.detect_video_motion_onset",
_stub_motion_onset,
)
messages = _build_takeoff_messages()
adapter = ReplayInputAdapter(
video_path=synthetic_video,
tlog_path=synthetic_tlog_path,
camera_calibration=camera_calibration,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
wgs_converter=fake_wgs_converter,
fdr_client=fake_fdr_client,
pace=ReplayPace.ASAP,
manual_time_offset_ms=None,
auto_sync_config=AutoSyncConfig(),
tlog_source_factory=_factory_for(messages),
video_timestamps_factory=_video_timestamps_factory(),
)
# Act
try:
bundle = adapter.open()
# Assert
expected_offset_ms = (_ns(2.0) - _ns(0.333)) // 1_000_000
assert bundle.resolved_time_offset_ms == expected_offset_ms
assert bundle.auto_sync_result is not None
assert bundle.auto_sync_result.offset_ms == expected_offset_ms
finally:
adapter.close()
@@ -1,935 +0,0 @@
"""AZ-698 — tlog trim + mid-flight cross-correlation alignment tests.
Covers AC-1..AC-4 of ``_docs/02_tasks/todo/AZ-698_tlog_trim_midflight_alignment.md``.
AC-5 (end-to-end CLI smoke) is exercised by the existing replay e2e
suite in ``tests/e2e/replay/`` and skipped here when its prerequisites
(ffmpeg-capable cv2 build + real ``derkachi.tlog``) are absent.
Style: every test follows the Arrange / Act / Assert pattern.
"""
from __future__ import annotations
import math
from pathlib import Path
from types import SimpleNamespace
from typing import Any
from unittest.mock import MagicMock
import pytest
from gps_denied_onboard._types.fc import (
AttitudeSample,
FcKind,
FcTelemetryFrame,
FlightStateSignal,
GpsHealth,
ImuTelemetrySample,
TelemetryKind,
)
from gps_denied_onboard.clock import Clock
from gps_denied_onboard.components.c8_fc_adapter.tlog_replay_adapter import (
ReplayPace,
TlogReplayFcAdapter,
)
from gps_denied_onboard.replay_input.auto_sync import (
_align_via_cross_correlation,
_resample_uniform,
_segment_flights_from_imu_energy,
compute_offset,
detect_video_motion_onset,
find_aligned_window,
validate_offset_or_fail,
)
from gps_denied_onboard.replay_input.errors import ReplayInputAdapterError
from gps_denied_onboard.replay_input.interface import (
AlignedWindow,
AutoSyncConfig,
)
# ---------------------------------------------------------------------
# Synthetic-stream helpers
def _ns(seconds: float) -> int:
return int(seconds * 1_000_000_000)
def _build_motion_burst_stream(
*,
start_s: float,
end_s: float,
hz: float,
burst_at_s: float,
burst_amplitude: float,
burst_duration_s: float = 1.0,
baseline_amplitude: float = 0.0,
) -> tuple[tuple[int, float], ...]:
"""Build a synthetic ``(ts_ns, magnitude)`` stream.
Constant at ``baseline_amplitude`` outside a single rectangular
burst (``burst_amplitude`` for ``burst_duration_s`` starting at
``burst_at_s``). Used so cross-correlation has a clear peak that
tests can assert exact-index for.
"""
out: list[tuple[int, float]] = []
period_s = 1.0 / hz
t = start_s
burst_end_s = burst_at_s + burst_duration_s
while t < end_s:
if burst_at_s <= t < burst_end_s:
out.append((_ns(t), burst_amplitude))
else:
out.append((_ns(t), baseline_amplitude))
t += period_s
return tuple(out)
def _build_double_burst_stream(
*,
start_s: float,
end_s: float,
hz: float,
burst_a_at_s: float,
burst_b_at_s: float,
burst_amplitude: float,
burst_duration_s: float = 1.0,
baseline_amplitude: float = 0.0,
) -> tuple[tuple[int, float], ...]:
"""Two-burst variant to constrain cross-correlation more tightly."""
out: list[tuple[int, float]] = []
period_s = 1.0 / hz
t = start_s
while t < end_s:
if burst_a_at_s <= t < burst_a_at_s + burst_duration_s:
out.append((_ns(t), burst_amplitude))
elif burst_b_at_s <= t < burst_b_at_s + burst_duration_s:
out.append((_ns(t), burst_amplitude))
else:
out.append((_ns(t), baseline_amplitude))
t += period_s
return tuple(out)
def _build_multi_flight_stream(
*,
flights: tuple[tuple[float, float], ...],
end_s: float,
hz: float,
in_flight_amplitude: float = 0.3,
ground_amplitude: float = 0.02,
) -> tuple[tuple[int, float], ...]:
"""Build a multi-flight IMU energy stream.
``flights`` is a tuple of ``(start_s, end_s)`` per flight. Between
flights the energy is ``ground_amplitude``; inside each flight it
is ``in_flight_amplitude``. Used by the multi-flight segmentation
tests to mimic a real "3 takeoffs at the same field" tlog.
"""
out: list[tuple[int, float]] = []
period_s = 1.0 / hz
t = 0.0
while t < end_s:
in_flight = any(s <= t < e for s, e in flights)
out.append((_ns(t), in_flight_amplitude if in_flight else ground_amplitude))
t += period_s
return tuple(out)
# ---------------------------------------------------------------------
# AC-1: takeoff-aligned regression — find_aligned_window must produce
# the same offset (within ± 50 ms) as the AZ-405 compute_offset path
# when the video covers the take-off.
def test_ac1_takeoff_aligned_offset_matches_az405_within_50ms() -> None:
# Arrange: 30 s tlog with a take-off-shaped IMU energy burst at
# t = 2 s; 5 s video with the same-shaped optical-flow burst at
# video_t = 0.5 s (motion onset half a second into the clip).
# AZ-405 would resolve offset_ms = (tlog_takeoff_ns -
# video_motion_onset_ns) // 1e6 ≈ 1.5 s. The AZ-698 aligner
# must agree within 50 ms.
tlog_energy = _build_motion_burst_stream(
start_s=0.0,
end_s=30.0,
hz=10.0,
burst_at_s=2.0,
burst_amplitude=1.2,
burst_duration_s=1.5,
baseline_amplitude=0.0,
)
flow_samples = _build_motion_burst_stream(
start_s=0.0,
end_s=5.0,
hz=10.0,
burst_at_s=0.5,
burst_amplitude=2.0,
burst_duration_s=1.5,
baseline_amplitude=0.0,
)
config = AutoSyncConfig()
expected_offset_ms = _ns(2.0 - 0.5) // 1_000_000
# Act
window = _align_via_cross_correlation(
tlog_energy=tlog_energy,
flow_samples=flow_samples,
config=config,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
tlog_path=Path("/nonexistent.tlog"),
tlog_source_factory=None,
)
# Assert
assert window.fallback_used is False, "expected primary cross-corr path, not fallback"
assert abs(window.offset_ms - expected_offset_ms) <= 50, (
f"AZ-698 offset {window.offset_ms} ms outside ±50 ms of AZ-405-equivalent "
f"{expected_offset_ms} ms"
)
# ---------------------------------------------------------------------
# AC-2: mid-flight alignment — tlog 030 s with motion burst at t=15 s,
# video 05 s with motion burst at video_t=1 s. Expected:
# tlog_start_ns ≈ (15 - 1) s = 14 s (where video t=0 lands)
# offset_ms ≈ 14 000
def test_ac2_mid_flight_alignment_locates_correct_window() -> None:
# Arrange: distinctive double-burst pattern in both streams so
# cross-correlation lock is unambiguous (single-burst patterns
# can lock on the wrong baseline at edge bins).
tlog_energy = _build_double_burst_stream(
start_s=0.0,
end_s=30.0,
hz=10.0,
burst_a_at_s=15.0,
burst_b_at_s=18.0,
burst_amplitude=1.5,
burst_duration_s=0.8,
baseline_amplitude=0.0,
)
flow_samples = _build_double_burst_stream(
start_s=0.0,
end_s=5.0,
hz=10.0,
burst_a_at_s=1.0,
burst_b_at_s=4.0,
burst_amplitude=2.5,
burst_duration_s=0.8,
baseline_amplitude=0.0,
)
config = AutoSyncConfig()
period_ns = _ns(1.0 / config.alignment_resample_hz)
# Act
window = _align_via_cross_correlation(
tlog_energy=tlog_energy,
flow_samples=flow_samples,
config=config,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
tlog_path=Path("/nonexistent.tlog"),
tlog_source_factory=None,
)
# Assert
assert window.fallback_used is False
# video burst A at t=1.0s aligns with tlog burst A at t=15.0s
# → video t=0 aligns with tlog t=14.0s within ±1 resample period.
assert abs(window.tlog_start_ns - _ns(14.0)) <= period_ns, (
f"tlog_start_ns={window.tlog_start_ns} not within one resample period "
f"({period_ns} ns) of the expected 14 s"
)
assert abs(window.offset_ms - 14_000) <= 100
assert window.tlog_end_ns > window.tlog_start_ns
# ---------------------------------------------------------------------
# AC-3: TlogReplayFcAdapter seek — messages whose raw _timestamp is
# below tlog_start_ns must NOT reach subscribers.
def _make_fake_msg(*, type_name: str, raw_ts_s: float, **fields: Any) -> SimpleNamespace:
"""Build a pymavlink-shaped fake message for replay-adapter tests."""
msg = SimpleNamespace(_timestamp=raw_ts_s, **fields)
def _get_type() -> str:
return type_name
msg.get_type = _get_type # type: ignore[attr-defined]
return msg
def _build_replay_adapter_with_seek(
*,
tlog_start_ns: int | None,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> tuple[TlogReplayFcAdapter, list[FcTelemetryFrame]]:
"""Construct a TlogReplayFcAdapter wired to deterministic fakes."""
monkeypatch.setenv("BUILD_TLOG_REPLAY_ADAPTER", "ON")
tlog_file = tmp_path / "fake.tlog"
tlog_file.write_bytes(b"\x00")
received: list[FcTelemetryFrame] = []
fake_clock = MagicMock(spec=Clock)
fake_clock.monotonic_ns.return_value = 0
fake_clock.sleep_until_ns.return_value = None
fake_wgs = MagicMock()
fake_fdr = MagicMock()
fake_fdr.enqueue.return_value = None
adapter = TlogReplayFcAdapter(
tlog_path=tlog_file,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
clock=fake_clock,
wgs_converter=fake_wgs,
fdr_client=fake_fdr,
time_offset_ms=0,
tlog_start_ns=tlog_start_ns,
pace=ReplayPace.ASAP,
)
adapter.subscribe_telemetry(received.append)
return adapter, received
def test_ac3_adapter_seek_skips_pre_window_messages(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
# Arrange: adapter opened with tlog_start_ns = 100 s; feed 5
# IMU messages, two below 100 s (must be skipped) and three at
# or above 100 s (must reach the subscriber).
adapter, received = _build_replay_adapter_with_seek(
tlog_start_ns=_ns(100.0),
tmp_path=tmp_path,
monkeypatch=monkeypatch,
)
pre_window = [
_make_fake_msg(
type_name="RAW_IMU",
raw_ts_s=t,
time_usec=int(t * 1e6),
xacc=0,
yacc=0,
zacc=1000,
xgyro=0,
ygyro=0,
zgyro=0,
)
for t in (50.0, 99.999)
]
in_window = [
_make_fake_msg(
type_name="RAW_IMU",
raw_ts_s=t,
time_usec=int(t * 1e6),
xacc=0,
yacc=0,
zacc=1000,
xgyro=0,
ygyro=0,
zgyro=0,
)
for t in (100.0, 101.5, 110.0)
]
# Act
for msg in pre_window + in_window:
adapter.feed_one_message(msg)
# Assert
assert len(received) == 3, "expected three in-window IMU frames"
assert all(
frame.kind == TelemetryKind.IMU_SAMPLE for frame in received
), "non-IMU frame leaked through"
# ``received_at`` is the raw _timestamp (no offset). Every
# delivered frame's raw timestamp must be ≥ 100 s.
for frame in received:
assert frame.received_at >= _ns(100.0), (
f"frame with received_at={frame.received_at} ns leaked below the seek bound"
)
def test_ac3_adapter_default_no_seek_passes_every_message(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
# Arrange: tlog_start_ns=None (default) → no seek; every message reaches subscribers.
adapter, received = _build_replay_adapter_with_seek(
tlog_start_ns=None,
tmp_path=tmp_path,
monkeypatch=monkeypatch,
)
messages = [
_make_fake_msg(
type_name="RAW_IMU",
raw_ts_s=t,
time_usec=int(t * 1e6),
xacc=0,
yacc=0,
zacc=1000,
xgyro=0,
ygyro=0,
zgyro=0,
)
for t in (10.0, 50.0, 100.0)
]
# Act
for msg in messages:
adapter.feed_one_message(msg)
# Assert
assert len(received) == 3, "default (no seek) must pass every IMU message"
# ---------------------------------------------------------------------
# AC-4: AC-9 frame-window validator passes for both scenarios.
def test_ac4_validator_passes_for_takeoff_aligned_offset() -> None:
# Arrange: video frames at 30 fps for 5 s; tlog IMU at 100 Hz
# for 30 s covering both pre-take-off and post; offset places
# video t=0 at tlog t=2 s.
video_ts = [int(t * 1_000_000_000) for t in (i / 30.0 for i in range(150))]
tlog_ts = [int(t * 1_000_000_000) for t in (i / 100.0 for i in range(3000))]
offset_ms = 2_000
# Act
result = validate_offset_or_fail(
offset_ms,
tlog_imu_timestamps_ns=tlog_ts,
video_frame_timestamps_ns=video_ts,
threshold_pct=95.0,
window_ms=100,
)
# Assert
assert result == 0
def test_ac4_validator_passes_for_mid_flight_offset() -> None:
# Arrange: video covers 05 s; tlog covers 060 s; mid-flight
# offset places video t=0 at tlog t=30 s. Every video frame
# still has an IMU sample within ±100 ms of (vts + 30s) because
# the tlog covers that range densely.
video_ts = [int(t * 1_000_000_000) for t in (i / 30.0 for i in range(150))]
tlog_ts = [int(t * 1_000_000_000) for t in (i / 100.0 for i in range(6000))]
offset_ms = 30_000
# Act
result = validate_offset_or_fail(
offset_ms,
tlog_imu_timestamps_ns=tlog_ts,
video_frame_timestamps_ns=video_ts,
threshold_pct=95.0,
window_ms=100,
)
# Assert
assert result == 0
# ---------------------------------------------------------------------
# Resampler unit tests — pin the binning semantics so future
# regressions are caught explicitly.
def test_resample_uniform_averages_within_bin() -> None:
# Arrange: 3 samples in the first 100 ms bin (values 1, 2, 3 →
# mean 2.0), 1 sample in the second bin (value 4 → 4.0).
samples = (
(_ns(0.00), 1.0),
(_ns(0.03), 2.0),
(_ns(0.06), 3.0),
(_ns(0.15), 4.0),
)
period_ns = _ns(0.10)
# Act
resampled = _resample_uniform(samples, period_ns, origin_ns=0)
# Assert
assert math.isclose(resampled[0], 2.0)
assert math.isclose(resampled[1], 4.0)
def test_resample_uniform_drops_trailing_empty_bins() -> None:
# Arrange: one sample in bin 0, then a 1 s gap before the next sample.
# The samples between get carry-forward of the previous bin's value;
# trailing zeros only appear AFTER the last sample.
samples = (
(_ns(0.0), 5.0),
(_ns(1.05), 7.0),
)
period_ns = _ns(0.1)
# Act
resampled = _resample_uniform(samples, period_ns, origin_ns=0)
# Assert
# The first bin is 5.0, bins 1..9 carry-forward to 5.0 (the previous
# bin's value), and bin 10 captures the t=1.05 s sample as 7.0.
assert resampled[0] == 5.0
assert resampled[-1] == 7.0
# No trailing-zero tail.
assert all(v != 0.0 for v in resampled)
# ---------------------------------------------------------------------
# Fallback path — when cross-correlation confidence is below the
# threshold, find_aligned_window must fall back to the head-takeoff
# detector and set fallback_used=True.
def test_low_confidence_triggers_takeoff_fallback(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
# Arrange: flat-line tlog (no motion) → cross-correlation has no
# meaningful peak. The fallback path opens the real tlog via
# detect_tlog_takeoff which needs a working tlog file. We bypass
# the actual fallback work by raising the threshold to 1.1 (no
# peak can clear it) and stubbing the takeoff detector.
monkeypatch.setattr(
"gps_denied_onboard.replay_input.auto_sync.detect_tlog_takeoff",
lambda path, dialect, config, *, source_factory=None: SimpleNamespace(
onset_ns=_ns(7.0), confidence=0.9
),
)
flat_tlog = tuple(
(_ns(t / 10.0), 0.0) for t in range(0, 100)
)
flat_flow = tuple(
(_ns(t / 10.0), 0.0) for t in range(0, 20)
)
config = AutoSyncConfig(alignment_low_confidence_threshold=0.5)
tlog_path = tmp_path / "fake.tlog"
tlog_path.write_bytes(b"\x00")
# Act
window = _align_via_cross_correlation(
tlog_energy=flat_tlog,
flow_samples=flat_flow,
config=config,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
tlog_path=tlog_path,
tlog_source_factory=None,
)
# Assert
assert window.fallback_used is True
assert window.tlog_start_ns == _ns(7.0), "fallback did not pick up the stubbed takeoff onset"
# ---------------------------------------------------------------------
# Guard: video stream longer than tlog stream → reject (auto-trim
# requires the video to be a SLICE of a longer tlog).
def test_video_longer_than_tlog_raises() -> None:
# Arrange
tlog_energy = tuple((_ns(t / 10.0), 0.5) for t in range(10))
flow_samples = tuple((_ns(t / 10.0), 0.5) for t in range(50))
config = AutoSyncConfig()
# Act + Assert
with pytest.raises(ReplayInputAdapterError, match="video flow stream is longer"):
_align_via_cross_correlation(
tlog_energy=tlog_energy,
flow_samples=flow_samples,
config=config,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
tlog_path=Path("/nonexistent.tlog"),
tlog_source_factory=None,
)
# ---------------------------------------------------------------------
# AlignedWindow DTO is frozen + slotted.
def test_aligned_window_is_frozen() -> None:
# Arrange
w = AlignedWindow(
tlog_start_ns=1,
tlog_end_ns=2,
offset_ms=0,
confidence=0.9,
fallback_used=False,
)
# Act + Assert
with pytest.raises((AttributeError, TypeError)):
w.confidence = 0.5 # type: ignore[misc]
# ---------------------------------------------------------------------
# AC-5: end-to-end CLI smoke — skipped here because it requires
# ffmpeg-capable cv2 + the real ``derkachi.tlog``/``.mp4`` binaries.
# The actual CLI run is covered by ``tests/e2e/replay/`` when those
# prerequisites are available.
def _replay_inputs_present() -> bool:
fixtures = Path("_docs/00_problem/input_data/flight_derkachi")
return (fixtures / "derkachi.tlog").is_file() and (fixtures / "derkachi.mp4").is_file()
@pytest.mark.skipif(
not _replay_inputs_present(),
reason="AC-5 e2e smoke requires _docs/00_problem/input_data/flight_derkachi/derkachi.{tlog,mp4}",
)
def test_ac5_cli_auto_trim_smoke_uses_find_aligned_window(
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Arrange: this test pins the wiring contract — the `--auto-trim`
# CLI flag must reach ReplayConfig.auto_trim. A full CLI run
# requires the runtime root which is exercised by the e2e suite.
from gps_denied_onboard.cli.replay import _build_replay_config
from gps_denied_onboard.config.schema import Config, ReplayConfig
args = SimpleNamespace(
video=Path("/tmp/v.mp4"),
tlog=Path("/tmp/t.tlog"),
output=Path("/tmp/o.jsonl"),
camera_calibration=Path("/tmp/c.json"),
config_path=Path("/tmp/c.yaml"),
mavlink_signing_key=Path("/tmp/k.bin"),
pace="asap",
time_offset_ms=None,
skip_auto_sync_validation=False,
auto_trim=True,
)
key_file = Path("/tmp/k.bin")
key_file.write_bytes(b"\x00" * 32)
base = Config()
base = type(base)(
mode=base.mode,
log=base.log,
fdr=base.fdr,
runtime=base.runtime,
fc=base.fc,
gcs=base.gcs,
replay=ReplayConfig(),
components=base.components,
)
# Act
new_config = _build_replay_config(args, base)
# Assert
assert new_config.replay.auto_trim is True
assert new_config.replay.time_offset_ms is None
# Cross-reference: the existing AZ-405 fixture still passes (no regression).
def test_autosync_decision_offset_is_within_ac9_window_for_baseline() -> None:
# Arrange: a takeoff-shaped tlog detector result + a video
# motion-onset detector result. compute_offset returns the
# AZ-405 offset_ms which is the AZ-698 baseline AC-1 references.
from gps_denied_onboard.replay_input.auto_sync import _DetectorResult
tlog_result = _DetectorResult(onset_ns=_ns(2.5), confidence=0.9)
video_result = _DetectorResult(onset_ns=_ns(0.5), confidence=0.85)
# Act
decision = compute_offset(tlog_result, video_result)
# Assert
assert decision.offset_ms == 2_000
assert decision.combined_confidence == pytest.approx(0.85, abs=1e-6)
# ---------------------------------------------------------------------
# Multi-flight tlog handling (user constraint: "if 1 flight take it, if
# multiple take the last"). The pre-NCC segmenter is the gatekeeper.
def test_segmenter_one_flight_returns_single_span() -> None:
# Arrange: 120 s tlog with a single flight from t=10..100 s.
samples = _build_multi_flight_stream(
flights=((10.0, 100.0),),
end_s=120.0,
hz=10.0,
)
# Act
segments = _segment_flights_from_imu_energy(
samples,
motion_threshold=0.1,
min_flight_duration_ns=_ns(30.0),
max_internal_gap_ns=_ns(5.0),
)
# Assert
assert len(segments) == 1
seg_start_ns, seg_end_ns = segments[0]
assert abs(seg_start_ns - _ns(10.0)) <= _ns(0.2)
assert abs(seg_end_ns - _ns(100.0)) <= _ns(0.2)
def test_segmenter_three_flights_returns_three_spans_in_order() -> None:
# Arrange: 360 s tlog with three takeoffs (60 s flights with 30 s
# ground gaps between them) — mimics the Derkachi scenario the
# user flagged: one tlog, three sorties, video covers only the
# last one.
flights_def = ((10.0, 70.0), (100.0, 160.0), (190.0, 250.0))
samples = _build_multi_flight_stream(
flights=flights_def,
end_s=300.0,
hz=10.0,
)
# Act
segments = _segment_flights_from_imu_energy(
samples,
motion_threshold=0.1,
min_flight_duration_ns=_ns(30.0),
max_internal_gap_ns=_ns(5.0),
)
# Assert
assert len(segments) == 3
for (actual_start, actual_end), (want_start, want_end) in zip(
segments, flights_def
):
assert abs(actual_start - _ns(want_start)) <= _ns(0.2)
assert abs(actual_end - _ns(want_end)) <= _ns(0.2)
def test_segmenter_drops_ground_blip_below_min_duration() -> None:
# Arrange: a 5 s ground manoeuvre (engine test) followed by a
# real 60 s flight. With min_flight_duration_ns=30 s the blip
# must be discarded, leaving only the real flight.
samples = _build_multi_flight_stream(
flights=((5.0, 10.0), (50.0, 110.0)),
end_s=120.0,
hz=10.0,
)
# Act
segments = _segment_flights_from_imu_energy(
samples,
motion_threshold=0.1,
min_flight_duration_ns=_ns(30.0),
max_internal_gap_ns=_ns(5.0),
)
# Assert
assert len(segments) == 1
seg_start_ns, _seg_end_ns = segments[0]
assert abs(seg_start_ns - _ns(50.0)) <= _ns(0.2)
def test_segmenter_keeps_brief_cruise_lull_inside_flight() -> None:
# Arrange: one flight with a 3 s cruise lull mid-way. The lull is
# below max_internal_gap_ns=5 s, so the segmenter must keep the
# whole flight as a single segment.
samples = _build_multi_flight_stream(
flights=((10.0, 45.0), (48.0, 100.0)),
end_s=120.0,
hz=10.0,
)
# Act
segments = _segment_flights_from_imu_energy(
samples,
motion_threshold=0.1,
min_flight_duration_ns=_ns(30.0),
max_internal_gap_ns=_ns(5.0),
)
# Assert
assert len(segments) == 1
seg_start_ns, seg_end_ns = segments[0]
assert abs(seg_start_ns - _ns(10.0)) <= _ns(0.2)
assert abs(seg_end_ns - _ns(100.0)) <= _ns(0.2)
def test_find_aligned_window_picks_last_flight_for_multi_flight_tlog(
tmp_path: Path,
) -> None:
# Arrange: a 300 s tlog with three sorties (10..70, 100..160,
# 190..250 s). The video covers only the LAST sortie — flow
# samples at video-clock 0..30 s with a motion burst at
# video t=5 s that, on the tlog timeline, corresponds to
# tlog t=200 s (5 s into flight 3 which starts at 190 s).
flights_def = ((10.0, 70.0), (100.0, 160.0), (190.0, 250.0))
tlog_energy = _build_multi_flight_stream(
flights=flights_def,
end_s=260.0,
hz=10.0,
)
flow_samples = _build_motion_burst_stream(
start_s=0.0,
end_s=30.0,
hz=10.0,
burst_at_s=5.0,
burst_amplitude=2.0,
burst_duration_s=3.0,
baseline_amplitude=0.05,
)
config = AutoSyncConfig(
alignment_segment_min_flight_duration_seconds=30.0,
alignment_segment_max_internal_gap_seconds=5.0,
)
# Inject the pre-loaded IMU energy by monkey-patching the loader
# used inside find_aligned_window; the function reads a tlog via
# pymavlink, but for the unit-level invariant we want to assert
# the segment selection — not the binary parser.
import gps_denied_onboard.replay_input.auto_sync as auto_sync_mod
fake_tlog = tmp_path / "multi_flight.tlog"
fake_tlog.write_bytes(b"\x00")
fake_video = tmp_path / "video.mp4"
fake_video.write_bytes(b"\x00")
def _fake_loader(
path: Path,
*,
max_messages: int,
source_factory: Any,
) -> tuple[tuple[int, float], ...]:
return tlog_energy
def _fake_frames(
path: Path, scan_seconds: float,
) -> "list[tuple[int, Any]]":
import numpy as np
rng = np.random.default_rng(42)
frames: list[tuple[int, Any]] = []
prev_offset = np.zeros((16, 16, 3), dtype=np.int16)
for ts_ns, mag in flow_samples:
# 3-channel BGR (cvtColor BGR→GRAY needs ≥ 3 channels).
# During a burst we shift pixels — that motion is what
# Farneback flow magnitudes pick up.
base = rng.integers(0, 30, size=(16, 16, 3), dtype=np.int16)
shift_px = int(mag * 4)
if shift_px > 0:
base = np.roll(base, shift=shift_px, axis=0)
frame = np.clip(base + prev_offset, 0, 255).astype(np.uint8)
frames.append((ts_ns, frame))
prev_offset = np.zeros_like(prev_offset)
return frames
monkeypatch = pytest.MonkeyPatch()
try:
monkeypatch.setattr(
auto_sync_mod, "_load_tlog_imu_energy_stream", _fake_loader
)
# Act
window = find_aligned_window(
fake_tlog,
fake_video,
config,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
video_frames_factory=_fake_frames,
)
finally:
monkeypatch.undo()
# Assert: the aligner MUST select FLIGHT 3 (190..250 s), NOT
# flight 1 (10..70 s). Whether NCC locks on or the fallback
# path fires, the resulting window must lie inside flight 3 —
# that's the user-visible contract ("take the last flight").
flight3_start_ns, flight3_end_ns = (_ns(190.0), _ns(250.0))
assert window.flight_count_detected == 3
assert window.selected_flight_index == 2
assert flight3_start_ns <= window.tlog_start_ns <= flight3_end_ns
# Sanity: did NOT lock onto flight 1 or 2.
assert window.tlog_start_ns > _ns(160.0)
def test_align_via_cross_correlation_locks_onto_burst_inside_last_segment() -> None:
# Arrange: pre-segmented tlog energy restricted to flight 3
# (mimicking what find_aligned_window passes after segmentation),
# plus a flow stream whose burst pattern matches a specific
# offset inside that segment. This directly exercises the NCC
# path with the inputs the post-segmentation aligner sees.
last_segment_tlog = _build_motion_burst_stream(
start_s=190.0,
end_s=250.0,
hz=10.0,
burst_at_s=210.0,
burst_amplitude=1.5,
burst_duration_s=5.0,
baseline_amplitude=0.05,
)
flow_samples = _build_motion_burst_stream(
start_s=0.0,
end_s=30.0,
hz=10.0,
burst_at_s=10.0,
burst_amplitude=2.0,
burst_duration_s=5.0,
baseline_amplitude=0.05,
)
config = AutoSyncConfig()
# Act
window = _align_via_cross_correlation(
tlog_energy=last_segment_tlog,
flow_samples=flow_samples,
config=config,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
tlog_path=Path("/nonexistent.tlog"),
tlog_source_factory=None,
flight_count_detected=3,
selected_flight_index=2,
)
# Assert: NCC must lock on (high confidence, no fallback). The
# tlog_start_ns must be the start of the matched 30 s window —
# video burst at video_t=10 s lines up with tlog_t=210 s ⇒
# tlog_start_ns ≈ 200 s (210 s 10 s).
assert not window.fallback_used
assert window.confidence > 0.6
assert window.flight_count_detected == 3
assert window.selected_flight_index == 2
assert abs(window.tlog_start_ns - _ns(200.0)) <= _ns(0.2)
def test_find_aligned_window_uses_only_segment_for_segmented_tlog_fallback(
tmp_path: Path,
) -> None:
# Arrange: a 3-flight tlog where the video flow is flat (no
# structure for NCC to lock onto). NCC must produce confidence
# ~ 0; the fallback path must use the LAST segment start, NOT
# the head-takeoff detector (which would lock onto flight 1).
flights_def = ((10.0, 70.0), (100.0, 160.0), (190.0, 250.0))
tlog_energy = _build_multi_flight_stream(
flights=flights_def,
end_s=260.0,
hz=10.0,
)
flat_flow = tuple((_ns(t / 10.0), 0.5) for t in range(0, 50))
config = AutoSyncConfig()
# Act
window = _align_via_cross_correlation(
tlog_energy=tuple(
(ts, e) for ts, e in tlog_energy
if _ns(190.0) <= ts <= _ns(250.0)
),
flow_samples=flat_flow,
config=config,
target_fc_dialect=FcKind.ARDUPILOT_PLANE,
tlog_path=tmp_path / "x.tlog",
tlog_source_factory=None,
flight_count_detected=3,
selected_flight_index=2,
)
# Assert
assert window.fallback_used is True
assert window.flight_count_detected == 3
assert window.selected_flight_index == 2
# The fallback must use flight 3's start, not flight 1's takeoff.
assert window.tlog_start_ns >= _ns(190.0)
assert window.tlog_start_ns <= _ns(250.0)
@@ -0,0 +1,38 @@
"""AZ-895: auto-sync surface deprecated; every public callable raises.
The full detector test suite was deleted alongside the detectors
themselves; this single test pins the deprecation contract: a clean
:class:`ReplayInputAdapterError` with the documented message, not a
silent import failure or vague RuntimeError. AZ-908 will remove
``auto_sync.py`` entirely and this test along with it.
"""
from __future__ import annotations
import pytest
from gps_denied_onboard.replay_input import ReplayInputAdapterError
from gps_denied_onboard.replay_input import auto_sync
_DEPRECATED_CALLABLES = (
"detect_tlog_takeoff",
"detect_video_motion_onset",
"compute_offset",
"validate_offset_or_fail",
"find_aligned_window",
)
@pytest.mark.parametrize("name", _DEPRECATED_CALLABLES)
def test_az895_public_callable_raises_with_documented_message(name: str) -> None:
"""AC-1: every public callable raises the documented deprecation error."""
# Arrange
fn = getattr(auto_sync, name)
# Act / Assert
with pytest.raises(
ReplayInputAdapterError,
match="auto-sync removed; supply --imu CSV instead",
):
fn()
+9 -38
View File
@@ -144,9 +144,14 @@ def _make_replay_config(
if calib_path is None if calib_path is None
else RuntimeConfig(camera_calibration_path=str(calib_path)) else RuntimeConfig(camera_calibration_path=str(calib_path))
) )
# AZ-895: imu_csv_path is required by _validate_replay_paths; the
# auto-sync surface that previously accepted (video, tlog) alone
# was deprecated. tlog_path stays set so tests covering the legacy
# config field continue to round-trip it.
replay = ReplayConfig( replay = ReplayConfig(
video_path="/dev/null/fake.mp4", video_path="/dev/null/fake.mp4",
tlog_path="/dev/null/fake.tlog", tlog_path="/dev/null/fake.tlog",
imu_csv_path="/dev/null/fake_imu.csv",
output_path=output_path, output_path=output_path,
pace=pace, pace=pace,
time_offset_ms=time_offset_ms, time_offset_ms=time_offset_ms,
@@ -634,13 +639,11 @@ def test_replay_branch_rejects_empty_video_path(
build_replay_components(config) build_replay_components(config)
def test_replay_branch_rejects_both_inputs_empty( def test_replay_branch_rejects_missing_imu_csv_path(
_airborne_replay_env: Path, _airborne_replay_env: Path,
) -> None: ) -> None:
# AZ-894: the validation gate now accepts either imu_csv_path # AZ-895: imu_csv_path is required; the legacy tlog-only branch was
# (canonical) or tlog_path (legacy) — rejecting only when both # removed. The (video, CSV) bundle is the only supported composition.
# are empty. Keeping the historical name pattern (test_*_rejects_*)
# for grep parity but renamed to reflect the new semantics.
# Arrange # Arrange
runtime_cfg = RuntimeConfig(camera_calibration_path=str(_airborne_replay_env)) runtime_cfg = RuntimeConfig(camera_calibration_path=str(_airborne_replay_env))
@@ -648,7 +651,7 @@ def test_replay_branch_rejects_both_inputs_empty(
runtime=runtime_cfg, runtime=runtime_cfg,
replay=ReplayConfig( replay=ReplayConfig(
video_path="/dev/null/fake.mp4", video_path="/dev/null/fake.mp4",
tlog_path="", tlog_path="/dev/null/fake.tlog",
imu_csv_path="", imu_csv_path="",
output_path="/tmp/out.jsonl", output_path="/tmp/out.jsonl",
pace="asap", pace="asap",
@@ -678,38 +681,6 @@ def test_replay_branch_rejects_unknown_pace_after_init(
build_replay_components(config) build_replay_components(config)
def test_replay_branch_loads_camera_calibration_from_runtime_path(
_airborne_replay_env: Path,
) -> None:
"""The branch reads the SAME calibration JSON the live binary uses."""
# Arrange
config = _make_replay_config(calib_path=_airborne_replay_env)
# Act — run far enough to populate the bundle without hitting the
# real video / tlog readers. We do that by injecting a stub
# ``replay_input_adapter_factory`` that returns a fake adapter
# whose ``open()`` produces a trivial bundle.
bundle = _make_replay_bundle()
class _StubAdapter:
def __init__(self, **_kwargs: Any) -> None:
pass
def open(self) -> ReplayInputBundle:
return bundle
components, order = build_replay_components(
config,
replay_input_adapter_factory=lambda **_kwargs: _StubAdapter(),
sink_factory=lambda *_args: mock.MagicMock(spec=JsonlReplaySink),
)
# Assert
assert order == REPLAY_COMPONENT_KEYS
assert components["frame_source"] is bundle.frame_source
assert components["fc_adapter"] is bundle.fc_adapter
# ---------------------------------------------------------------------- # ----------------------------------------------------------------------
# Smoke # Smoke
+97 -7
View File
@@ -114,6 +114,10 @@ def _argv(files: dict[str, Path], **overrides: Any) -> list[str]:
argv: list[str] = [] argv: list[str] = []
for k, v in base.items(): for k, v in base.items():
argv.extend([k, v]) argv.extend([k, v])
if overrides.get("skip_auto_sync"):
argv.append("--skip-auto-sync")
if overrides.get("auto_trim"):
argv.append("--auto-trim")
return argv return argv
@@ -192,12 +196,15 @@ def test_ac3_pace_realtime(
# ---------------------------------------------------------------------- # ----------------------------------------------------------------------
# AC-4: --time-offset-ms forwarded (None when absent) # AC-4: --time-offset-ms deprecated (AZ-895) — ignored + warning emitted
def test_ac4_time_offset_forwarded( def test_ac4_time_offset_ignored_after_az895(
_required_files: dict[str, Path], _airborne_env: None _required_files: dict[str, Path],
_airborne_env: None,
capsys: pytest.CaptureFixture[str],
) -> None: ) -> None:
"""AZ-895: --time-offset-ms is deprecated; value is ignored, warning emitted."""
# Arrange # Arrange
captured: dict[str, Config] = {} captured: dict[str, Config] = {}
@@ -206,13 +213,16 @@ def test_ac4_time_offset_forwarded(
return 0 return 0
# Act # Act
rc = replay_cli.main( with pytest.warns(DeprecationWarning, match="--time-offset-ms"):
_argv(_required_files, time_offset_ms=5000), shared_main=fake_main rc = replay_cli.main(
) _argv(_required_files, time_offset_ms=5000), shared_main=fake_main
)
# Assert # Assert
assert rc == EXIT_SUCCESS assert rc == EXIT_SUCCESS
assert captured["config"].replay.time_offset_ms == 5000 assert captured["config"].replay.time_offset_ms is None
err = capsys.readouterr().err
assert "--time-offset-ms is deprecated (AZ-895)" in err
def test_ac4_time_offset_none_when_absent( def test_ac4_time_offset_none_when_absent(
@@ -233,6 +243,86 @@ def test_ac4_time_offset_none_when_absent(
assert captured["config"].replay.time_offset_ms is None assert captured["config"].replay.time_offset_ms is None
def test_az895_skip_auto_sync_ignored_and_warned(
_required_files: dict[str, Path],
_airborne_env: None,
capsys: pytest.CaptureFixture[str],
) -> None:
"""AZ-895: --skip-auto-sync deprecated; value ignored, warning emitted."""
# Arrange
captured: dict[str, Config] = {}
def fake_main(config: Config) -> int:
captured["config"] = config
return 0
# Act
with pytest.warns(DeprecationWarning, match="--skip-auto-sync"):
rc = replay_cli.main(
_argv(_required_files, skip_auto_sync=True), shared_main=fake_main
)
# Assert
assert rc == EXIT_SUCCESS
assert captured["config"].replay.skip_auto_sync_validation is False
err = capsys.readouterr().err
assert "--skip-auto-sync is deprecated (AZ-895)" in err
def test_az895_auto_trim_ignored_and_warned(
_required_files: dict[str, Path],
_airborne_env: None,
capsys: pytest.CaptureFixture[str],
) -> None:
"""AZ-895: --auto-trim deprecated; value ignored, warning emitted."""
# Arrange
captured: dict[str, Config] = {}
def fake_main(config: Config) -> int:
captured["config"] = config
return 0
# Act
with pytest.warns(DeprecationWarning, match="--auto-trim"):
rc = replay_cli.main(
_argv(_required_files, auto_trim=True), shared_main=fake_main
)
# Assert
assert rc == EXIT_SUCCESS
assert captured["config"].replay.auto_trim is False
err = capsys.readouterr().err
assert "--auto-trim is deprecated (AZ-895)" in err
def test_az895_no_deprecated_flags_no_warning(
_required_files: dict[str, Path],
_airborne_env: None,
capsys: pytest.CaptureFixture[str],
recwarn: pytest.WarningsRecorder,
) -> None:
"""No AZ-895 flag-specific deprecation warning when none of the flags are used."""
# Act
rc = replay_cli.main(_argv(_required_files), shared_main=lambda _c: 0)
# Assert
assert rc == EXIT_SUCCESS
err = capsys.readouterr().err
for flag in ("--time-offset-ms", "--skip-auto-sync", "--auto-trim"):
assert f"{flag} is deprecated" not in err, (
f"unexpected deprecation banner for {flag} when it was not passed"
)
az895_flag_warnings = [
w for w in recwarn.list
if issubclass(w.category, DeprecationWarning)
and any(
f"{flag} is deprecated" in str(w.message)
for flag in ("--time-offset-ms", "--skip-auto-sync", "--auto-trim")
)
]
assert az895_flag_warnings == []
# ---------------------------------------------------------------------- # ----------------------------------------------------------------------
# AC-5: --mavlink-signing-key required (argparse exit 2) # AC-5: --mavlink-signing-key required (argparse exit 2)