[AZ-842] Batch 04 cycle 4: AZ-835 docs + cycle-4 redesign narrative

Closes AZ-835 Epic C6 (docs) and folds the cycle-4 replay-input
redesign narrative (AZ-894 CSV adapter / AZ-895 auto-sync deprecation
/ AZ-896 format spec / AZ-897 UI follow-up) into the three
authoritative documents.

Modified:
- _docs/02_document/contracts/replay/replay_protocol.md: extend
  Invariant 12 with sub-invariants 12.c (route-driven supersedes
  bbox; ~100x tile efficiency + did-fly-vs-might-fly honesty) and
  12.d (fixture failure-handling: validation/terminal re-raise;
  transient -> C11 backoff x3). Add Invariant 14 with sub-
  invariants 14.a-14.d covering the single canonical clock model,
  the CSV-driven path, the tlog adapter's audit-only role, the
  auto-sync deprecation, and the AZ-897 UI follow-up pointer.
- _docs/02_document/architecture.md: add the AZ-777 Phase 3+
  superseded-by-Epic-AZ-835 supersession block + new "Replay input
  redesign (cycle 4)" sub-section with the cycle-4 ticket table.
- tests/e2e/replay/README.md: top section restructured for two
  distinct entry points (AZ-265/AZ-404 vs. AZ-835/AZ-840); add
  full AZ-835 orchestrator-test section (env vars, skip gates,
  expected runtime, verdict report path); add Imagery (c) Google
  attribution + dev-only caveat; add Epic AZ-835 ticket map.

Spec deviation: AC-1b says "new Invariant 13" but Invariant 13 is
already taken (C4<->C5 pairing, AZ-776 / ADR-012), and is referenced
by number in architecture.md, c4_pose description.md, and ADR-012
prose. Cycle-4 content shipped as Invariant 14 to preserve those
cross-references; renumbering would have cascaded to 3 files outside
AZ-842's ownership envelope. Documented in batch report.

Out-of-scope hygiene gap (NOT fixed in this batch):
BUILD_CSV_REPLAY_ADAPTER flag is not yet enumerated in
_docs/02_document/module-layout.md's Build-Time Exclusion Map.
Inherited from cycle-4 AZ-894. Suggested as a cycle-5+ hygiene PBI.

AZ-835 epic file stays in todo/ until AZ-841 (backlog) is resolved.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-29 11:13:33 +03:00
parent e4409df228
commit 42b1db6ace
6 changed files with 296 additions and 8 deletions
+111 -5
View File
@@ -1,20 +1,104 @@
# E2E replay tests (AZ-404)
# E2E replay tests (AZ-404 + AZ-835 + cycle-4)
End-to-end regression suite that runs the `gps-denied-replay`
console-script (AZ-402) against the Derkachi 60 s clip and asserts
the AZ-265 epic acceptance criteria.
End-to-end regression suite for the `gps-denied-replay` console-script
(AZ-402). Two distinct entry points live here:
| Entry point | Source | Coverage |
|-------------|--------|----------|
| **AZ-265 / AZ-404** — 60 s Derkachi clip with synthetic tlog | `test_derkachi_1min.py` | Original AC-1..AC-10 of the replay epic; runs on Tier-1 + Tier-2 |
| **AZ-835 / AZ-840** — full `(tlog, video, calibration)` orchestrator | `test_az835_e2e_real_flight.py` | Tier-2 only; closes the real-flight validation loop end-to-end (extract → seed → FAISS → run → verdict) |
The cycle-4 replay-input redesign (AZ-894 / AZ-895 / AZ-896) replaces
the tlog auto-sync surface with a CSV-driven path; the AZ-265 suite is
the regression net that catches drift in the legacy path during the
deprecation window. See `replay_protocol.md` Invariants 12-14 for the
authoritative contract.
## How to run
### AZ-404 Derkachi 60 s suite (Tier-1 + Tier-2)
```bash
# In a fresh venv with the package installed:
RUN_REPLAY_E2E=1 pytest tests/e2e/replay/ -v
RUN_REPLAY_E2E=1 pytest tests/e2e/replay/test_derkachi_1min.py -v
```
Without `RUN_REPLAY_E2E=1` the heavy tests skip cleanly. The two
unconditional tests (AC-4a mode-agnosticism scan + AC-7 skip-gate
self-check + the helpers in `test_helpers.py`) still run.
### AZ-835 orchestrator test — full `(tlog, video, calibration)` loop (Tier-2 only)
Closes Epic AZ-835's narrative: given a real-flight `.tlog` + the
matching nadir video + camera calibration, the orchestrator runs the
7-step pipeline end-to-end and writes a verdict report.
**Required inputs** (already in-repo for the Derkachi reference fixture):
- `.tlog` — pymavlink binary log from a real flight. Reference fixture:
`_docs/00_problem/input_data/flight_derkachi/data_imu.csv` (the canonical
CSV that `_tlog_synth.py` reconstructs the tlog from) plus the synthesised
tlog the conftest emits at session start.
- Nadir video — `_docs/00_problem/input_data/flight_derkachi/*.mp4` (large
asset; not always checked in to the workstation clone — pull from the
Jetson e2e harness or git LFS if absent).
- Calibration — `tests/fixtures/calibration/adti26.json` (factory-sheet
approximation for the Topotek KHP20S30; real intrinsics still TBD).
**Tier-2 invocation** (Jetson):
```bash
ssh jetson-e2e
cd /workspace/gps-denied-onboard
export RUN_REPLAY_E2E=1
export GPS_DENIED_OPERATOR_CONFIG_PATH=/workspace/configs/operator_replay.yaml
pytest tests/e2e/replay/test_az835_e2e_real_flight.py -v --tb=short -m tier2
```
The bundled local-development entry point is `scripts/run-tests-jetson.sh`,
which handles the SSH alias + rsync + remote pytest invocation. See
`_docs/02_document/tests/tier2-jetson-testing.md` for the harness contract.
**Skip gates (in evaluation order)**:
1. `@pytest.mark.tier2` — the per-suite Tier-2 plugin gates this off on dev
macOS / Tier-1 Docker (matches the AZ-839 / AZ-699 contract).
2. `RUN_REPLAY_E2E` not in `{1, true, yes, on}`.
3. `gps-denied-replay` console-script not on `PATH`.
4. Real Derkachi video missing or placeholder-sized.
5. `operator_pre_flight_setup` fixture itself skipped — the downstream
consumer inherits the SKIP automatically (pytest's fixture-skip
propagation).
**Expected runtime on Tier-2 Jetson AGX Orin** (cold cache): ≤ 8 min
end-to-end (≤ 5 min C3 fixture cold-start budget + ≤ 3 min for the
replay + verdict compute). Warm-cache reinvocations within the same
compose session: ≤ 60 s.
**Verdict report location**: `_docs/06_metrics/real_flight_validation_<YYYY-MM-DD>.md`.
The report is emitted ALWAYS, regardless of PASS / FAIL on the AZ-696
AC-3 threshold (≥ 80 % of emissions within 100 m of tlog ground truth).
The success criterion at the fixture level is "honest report exists with
distribution data", not "PASS". The PASS / FAIL line of the report itself
is the operator-facing answer to "did this flight clip localise within
the threshold".
### Imagery source license attribution (dev/research use only)
The Jetson e2e harness's `satellite-provider` instance downloads tiles
from the **Google Maps satellite layer** (`mt0..mt3.google.com/vt/lyrs=s`),
governed by Google Maps Platform Terms of Service. Every tile served by
the harness carries the **"Imagery © Google"** attribution string.
**This is dev/research use only.** Production deployment of the
gps-denied-onboard companion against a Google-Maps-sourced
`satellite-provider` requires either a Google Maps Platform licensing
review or migration to a true CC-BY satellite source on the parent-suite
side (parent-suite ticket TBD; see `_docs/02_document/architecture.md`
§ `satellite-provider` integration). The onboard-side seed scripts
(`tests/fixtures/derkachi_c6/seed_region.py`, `seed_route.py`) propagate
the attribution into the test fixture's metadata; do not remove it.
## Fixture state
| Artifact | Status | Source |
@@ -97,3 +181,25 @@ tests/e2e/replay/
* **AZ-558** — closes AC-4b (route C8 encoders through `MavlinkTransport`).
* **D-PROJ-2 mock-suite-sat-service** — unblocks AC-8 (operator
workflow rehearsal).
## Epic AZ-835 ticket map
The Tier-2 orchestrator path shipped under Epic
[AZ-835](https://denyspopov.atlassian.net/browse/AZ-835). Sub-tickets:
| Ticket | Role |
|--------|------|
| [AZ-836](https://denyspopov.atlassian.net/browse/AZ-836) | `TlogRouteExtractor` — active-segment trim + Douglas-Peucker coarsen tlog GPS to ≤ N waypoints |
| [AZ-838](https://denyspopov.atlassian.net/browse/AZ-838) | `SatelliteProviderRouteClient` + `seed_route.py` CLI — POST RouteSpec to satellite-provider, poll `mapsReady` |
| [AZ-839](https://denyspopov.atlassian.net/browse/AZ-839) | C3 `operator_pre_flight_setup` real fixture — wires C1+C2+C11+C10 against the seeded catalog |
| [AZ-840](https://denyspopov.atlassian.net/browse/AZ-840) | C4 E2E orchestrator test — drives the full 7-step pipeline from `(tlog, video, calibration)` |
| [AZ-842](https://denyspopov.atlassian.net/browse/AZ-842) | C6 Docs — `replay_protocol.md` Invariants 12-14 + `architecture.md` + this README (cycle-4 rescope) |
The cycle-4 replay-input redesign tickets ride alongside the Epic:
| Ticket | Role |
|--------|------|
| [AZ-894](https://denyspopov.atlassian.net/browse/AZ-894) | `CsvReplayInputAdapter` — new CSV-driven primary path on the single canonical clock |
| [AZ-895](https://denyspopov.atlassian.net/browse/AZ-895) | Auto-sync surface deprecation — tlog adapter reduced to audit-only role |
| [AZ-896](https://denyspopov.atlassian.net/browse/AZ-896) | CSV format spec (`csv_replay_format.md`) + example `data_imu.csv` |
| [AZ-897](https://denyspopov.atlassian.net/browse/AZ-897) | Operator-facing UI (React + Tailwind paired-upload form) — cycle 5+ |