- Modified the autodev state to reflect the current testing phase and details of the new `jetson-e2e` tests.
- Enhanced the "How to Test" documentation to provide clearer instructions on the demo replay validation process, including video and tlog alignment steps.
- Updated architectural documentation to include the new demo replay operator flow and its dependencies.
- Documented the removal of deprecated auto-sync features and clarified the operator-facing UI for replay validation.
- Added new entries in the dependencies table for upcoming tasks related to the demo replay flow.
These changes improve clarity and usability for operators and developers working with the demo replay system.
Unit suite: 2303P/0F/86S after relaxing C12 cold-start NFR (commit
05f1143). Cycle-3-scope PASS.
Jetson e2e: 48P/4F/3S/1XF/1XP. Four test_derkachi_1min.py failures
(AC-1/5/6-realtime/6-asap) trace to AZ-776 cycle-2 xfail removal that
was never validated on Jetson hardware. Tracked as AZ-848; xfails NOT
re-added per "Real Results, Not Simulated Ones" meta-rule.
Step 11 cycle-3 outcome recorded in run_tests_step11_report.md.
Advances autodev state pointer: step 11 -> 12 (Test-Spec Sync).
Co-authored-by: Cursor <cursoragent@cursor.com>
Cycle-3 addendum captures the layered Jetson rerun progression:
synth time-base fix (AZ-614) drops offset_ms from 1.7e12 to -4334;
AZ-611 skip-auto-sync then crosses the AC-9 validator; AZ-602
build-flag completeness opens VideoFileFrameSource and
TlogReplayFcAdapter; composition root logs
'replay.compose_root.ready: auto_sync_used=false', then crashes
inside runtime_root.airborne_bootstrap because production main()
never builds c13_fdr / c6_* / c7_inference / c3_lightglue_runtime /
c3_feature_extractor / c2_82_ransac_filter into pre_constructed.
The bootstrap gap is filed as AZ-618 (Story under AZ-602). It
affects both live and replay binaries -- every prior Reality-Gate
run died at auto-sync before the composition graph was walked, so
the gap was hidden. The 38 compose_root unit tests pass only via
the replay_components_factory stub kwarg, which bypasses the
bootstrap entirely.
Autodev sub_step advances to phase 8
'az614-az611-landed-bootstrap-gap-discovered' pending the user's
decision on whether to start AZ-618 immediately or close out
Step 11 with the current Reality-Gate signal.
Co-authored-by: Cursor <cursoragent@cursor.com>
Records the first Jetson Tier-2 run results in the step-11 report:
17 pass / 5 fail / 1 skip / 1 xfail (24 total, 10m09s) — identical to
Colima because all 5 failures hit AZ-614 (tlog time-base mismatch)
BEFORE reaching the GPU. So the infrastructure is proven (image
builds, GPU exposed inside container, SUT subprocess runs to the
auto-sync stage) but the heavy ACs haven't yet exercised ALIKED /
DISK LightGlue. Fixing AZ-614 is the gating prerequisite to actually
drive the GPU stages.
Also captures lessons learned that are now in the setup doc:
* Only dustynv/l4t-pytorch:r36.4.0 is a usable Jetson PyTorch base
on Docker Hub for R36 / JetPack 6 (l4t-base deprecated, official
l4t-pytorch has no R36 tags).
* The dustynv image bakes a maintainer-LAN-only pip mirror into
/etc/pip.conf — must be wiped + --index-url pinned to pypi.org.
* pip 24.2 (image default) rejects gtsam-4.3a0 pre-release; pip 26.x
accepts the same wheel for `gtsam<5.0,>=4.2` because there are no
stable aarch64 builds. Upgrade pip in the build, don't relax pin.
* nvidia-container-runtime mounts nvidia-smi from host, so the GPU
smoke test needs only ubuntu:22.04 (80 MB), not l4t-jetpack (5 GB).
Autodev state advances to phase 7 / jetson-harness-online.
Co-authored-by: Cursor <cursoragent@cursor.com>
Multi-stage Ubuntu 22.04 e2e-runner image installs gps-denied-onboard
(editable) into /opt/venv so the AZ-404 replay tests can subprocess
gps-denied-replay against the Derkachi fixture. Image layout mirrors
the host repo (/opt/pyproject.toml + /opt/src + /opt/tests bind mount)
so Path(__file__).parents[3] resolves to /opt and AC-4's AST scan
finds the components dir.
Entrypoint now runs `pytest /opt/tests/e2e/` instead of the empty
`scenarios/` dir. The bootstrap harness collects 24 tests vs. 0 before.
Compose: e2e-runner env mirrors the companion service (FullSystemConfig
requirements) plus RUN_REPLAY_E2E=1, BUILD_REPLAY_SINK_JSONL=ON;
bind-mounts the Derkachi fixture dir; adds writable fdr-data /
tile-data volumes the SUT requires.
Reality Gate signal is now real: 17 pass / 5 fail / 1 skip / 1 xfail.
The 5 heavy-AC failures share root cause AZ-614 (tlog synth time-base
mismatch, surfaced by the now-functional harness).
Also archives the replayed leftover entries (csv_reporter -> AZ-601,
harness rehab -> AZ-602 epic + 11 child stories).
Co-authored-by: Cursor <cursoragent@cursor.com>
Attempted Path-3 (Full SITL with community images) for the SUT Reality
Gate. Discovered sitl_observer is offline-fixture replay, not a live
SITL client -- compose-file SITL services in environment.md are
aspirational. The real Path-3 needs the fixture builders + SUT CLI
end-to-end, which surfaced 5 additional integration drifts (H-10..H-14)
on top of the prior 9.
Fixes:
- tests/fixtures/calibration/adti26.json: body_to_camera_se3 was a
{rotation_xyzw, translation_xyz_m} dict; runtime_root/_replay_branch.py
loader strictly expects a 4x4 SE3. Identity quaternion + zero
translation = identity 4x4, semantically equivalent.
New files:
- tests/fixtures/replay_config_minimal.yaml: minimal replay-mode config
for harness reproduction (mode=replay, ardupilot_plane defaults).
- .gitignore: e2e/fixtures/sitl_replay/ (generated by build_p0X_fixtures).
Documentation:
- Step 11 report: appended Path-3 attempt section.
- Leftover doc: H-10..H-14 ticket payloads added.
- Autodev state: reflects Path-3 outcome.
Step 11 stays blocked; H-13 (auto-sync AC-8 hard-fails on stationary
fixtures) requires a SUT design decision and cannot be unilaterally
fixed mid-session.
Co-authored-by: Cursor <cursoragent@cursor.com>
Local Tier-1 pytest suite: 3343 pass / 88 skip / 0 fail across 12 chunks.
Docker harness SUT Reality Gate UNMET — both Tier-1 docker harnesses
(scripts/run-tests.sh and e2e/docker/run-tier1.sh) have pre-existing
drift that prevents them from running end-to-end. Findings:
H-1..H-3 (fixed in 6ce3158): dockerfile rename, fdr-output tmpfs cap,
e2e-results bind dir + gitignore.
H-4..H-6 (deferred): three SITL/MAVLink Docker Hub images don't exist
(ardupilot/mavproxy, ardupilot/ardupilot-sitl,
inavflight/inav-sitl). environment.md spec was
written against aspirational image names.
H-7..H-8 (deferred): tests/e2e/Dockerfile entrypoint points at empty
scenarios dir + doesn't install the SUT package.
H-9 (deferred): tile-cache-fixture seeder missing (relates to AZ-595).
Plus a regression caught and fixed mid-run: pytest-csv autoload
conflicts with our custom --csv flag (commit eb6dc17). Also surfaced a
false-positive batch-89 test-result report; proposed preventive
meta-rule pending user approval.
Step 11 marked status=blocked pending harness rehabilitation tickets
(payloads recorded in _docs/_process_leftovers/). Full outcome report:
_docs/03_implementation/run_tests_step11_report.md.
Co-authored-by: Cursor <cursoragent@cursor.com>
Oleksandr Bezdieniezhnykh