# End-to-end real-flight validation pipeline (Epic)

**Task**: AZ-835_e2e_real_flight_validation_epic
**Name**: End-to-end real-flight validation: raw (tlog, video) → route-driven satellite seeding → gps-denied verdict
**Description**: Drive the full gps-denied-onboard validation pipeline from raw operator inputs to a verdict. Given a `.tlog` binary + a flight video, the system automatically extracts the flight cut, syncs frames to IMU, builds the satellite imagery the descriptor stack needs (route-driven, not bbox-driven), runs the airborne pipeline, and reports the horizontal-error distribution against the tlog's own GPS ground truth. Supersedes AZ-777 Phase 3+ design.
**Complexity**: Epic — ~17 SP decomposed into 6 child tasks of ≤ 5 SP each (see decomposition table below)
**Dependencies**: AZ-777 Phase 1 (landed cycle 3 batch 105 — C11 contract adaptation + e2e-runner wiring); AZ-405 (tlog↔video auto-sync adapter); AZ-699 (verdict report writer); AZ-809 SOFT (Route API validation — landing AZ-809 before C2 lets the client consume RFC 7807 validator responses cleanly)
**Component**: cross-cutting — replay_input + new TlogRouteExtractor + new SatelliteProviderRouteClient + e2e fixtures + tests/e2e/replay
**Tracker**: AZ-835 (https://denyspopov.atlassian.net/browse/AZ-835)
**Originating directive**: user (2026-05-22) after AZ-777 Phase 2 deliverables landed — "In the end it should be full e2e flow. You give it a tlog + video, and the system does everything else."

Jira AZ-835 is the authoritative spec; this file mirrors the in-workspace-only sections that gps-denied-onboard implementers will need.

## Goal

A single pytest test takes only `(tlog, video, calibration)` as input and runs the full 7-step pipeline end-to-end on the Jetson harness, producing an honest PASS/FAIL verdict against the AZ-696 AC-3 threshold (≥ 80 % of emissions within 100 m).

## The 7-step pipeline

| # | Step | Existing? | Component / new code |
|---|------|-----------|----------------------|
| 1 | Extract active flight cut + sync with video | **Mostly existing** (AZ-405 `tlog_video_adapter.py`) | small extension for take-off/landing boundary detection if needed |
| 2 | On-fly frame + IMU extraction | **Existing** | `VideoFileFrameSource` + `TlogReplayFcAdapter` (no change) |
| 3 | Auto-create route from tlog GPS, coarsen to ≤ 10 pts | **New** | `TlogRouteExtractor` (Douglas-Peucker on `GLOBAL_POSITION_INT` rows) → `RouteSpec` |
| 4 | POST route to satellite-provider, get tiles | **New consumer** | `SatelliteProviderRouteClient` (POST `/api/satellite/route`, poll `mapsReady`) |
| 5 | Calc FAISS index from tiles | **Mostly existing** | C10 `DescriptorBatcher` runs; new fixture wires C11 → C10 trigger |
| 6 | Run gps-denied from all the info | **Existing** | `gps-denied-replay` console-script + airborne composition root |
| 7 | Get GPS fixes, check against tlog GPS | **Existing** | `helpers/accuracy_report.py` + `helpers/gps_compare.py` |

## Decomposition (6 child tasks)

| # | Title | Est | Depends |
|---|-------|-----|---------|
| C1 | `TlogRouteExtractor` — extract active segment + coarsen to N waypoints | 3 | — |
| C2 | `SatelliteProviderRouteClient` + `route_seed.py` CLI | 3 | AZ-809 (soft) |
| C3 | New `operator_pre_flight_setup` fixture (C1 + C2 + C11 + C10) — replaces placeholder, supersedes AZ-777 Phase 3 | 5 | C1, C2, AZ-777 Phase 1 |
| C4 | E2E test ingesting raw `(tlog, video)` and running steps 1-7 — extends/replaces AZ-699 verdict test | 3 | C3 |
| C5 | Un-xfail AZ-777 AC-4 + AC-5 tests | 1 | C4 |
| C6 | Docs: `replay_protocol.md` Invariant 12 + AZ-777 amendment + new-test README | 2 | C5 |

**Total ~17 SP**.

## Why route-driven seeding (not bbox)

- **Efficiency**: AZ-777 spec bbox = ~11400 tiles z15-z18 (~140 MB, 48% over budget). 10-point coarsened route with `regionSizeMeters=500` per point = ~50-100 unique tiles (~1.5 MB) for the same VPR descriptor lock area. **~100× reduction**.
- **Honesty**: bbox pre-commits to where the operator *might* fly. Route pre-commits to where they *did* fly. For real-flight validation, the latter is the right primitive.
- **Probe-confirmed**: Route API works end-to-end in ~15s for a 2-point route per 2026-05-22 black-box probe. Uses `lat`/`lon` already (no AZ-812 rename needed).

## Coordination with prior work

- **AZ-777** — Phase 1 + Phase 2 reused; Phase 3+ design **superseded** by this Epic when C3 lands.
- **AZ-699** — verdict-report-writing path preserved; C4 extends or wraps it.
- **AZ-405** — tlog↔video auto-sync adapter reused as-is for step 1.
- **AZ-702** — camera factory-sheet calibration unchanged.
- **AZ-696** — ≥ 80 % within 100 m threshold gate unchanged.
- **AZ-808** — Region-endpoint validation; not on this Epic's critical path (Route used, not Region).
- **AZ-809** — Route-endpoint validation; soft prereq for C2.
- **AZ-812** — Region rename to lat/lon; not on this Epic's critical path.

## Acceptance criteria (Epic-level)

**AC-1**: New pytest test gated by `RUN_REPLAY_E2E=1` + `@pytest.mark.tier2` takes only `(tlog, video, calibration)` and runs the full 7-step pipeline on Jetson.

**AC-2**: Step 1 auto-detects active flight cut from raw tlog (take-off → landing) without operator intervention.

**AC-3**: Step 3 produces ≤ 10 waypoints that materially follow the tlog GPS trajectory (DP tolerance documented in config).

**AC-4**: Step 4 succeeds against real satellite-provider on Jetson docker network, downloads route tiles from Google Maps, `mapsReady=true` within runtime budget.

**AC-5**: Step 5 builds FAISS HNSW index over route-seeded C6 cache; sidecar triple-consistency holds (AZ-306).

**AC-6**: Step 7 emits AZ-699 verdict report at `_docs/06_metrics/real_flight_validation_<YYYY-MM-DD>.md` with honest horizontal-error distribution — PASS or FAIL on AZ-696 AC-3 threshold, no xfail mask.

**AC-7**: End-to-end run ≤ 15 min on Tier-2 Jetson for the Derkachi clip (soft target for first delivery; hard NFR after first measurement).

**AC-8**: Docs: `replay_protocol.md` Invariant 12 sub-section + AZ-777 marked Phase 3+ superseded + new-test README.

## Out of scope

- Satellite-provider imagery-source migration to CC-BY (parent-suite ticket, TBD).
- FAISS / NetVLAD backbone replacement.
- Real-time tlog ingestion (this Epic operates on finished `.tlog` files).
- Multi-flight aggregate validation.
- ZERO modifications to `../satellite-provider/` (Route API consumed as-is).
- CI gating (test stays behind `RUN_REPLAY_E2E=1`).

## References

- Jira AZ-835: https://denyspopov.atlassian.net/browse/AZ-835
- Supersedes AZ-777 Phase 3+ design (AZ-777 Phase 1 + Phase 2 reused)
- Probe foundation: 2026-05-22 black-box probe of Route API confirmed end-to-end viability
- Related: AZ-405, AZ-696, AZ-699, AZ-702, AZ-777, AZ-808, AZ-809, AZ-812
