mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 11:41:12 +00:00
Update demo replay validation and testing documentation
ci/woodpecker/push/02-build-push Pipeline failed
ci/woodpecker/push/02-build-push Pipeline failed
- Modified the autodev state to reflect the current testing phase and details of the new `jetson-e2e` tests. - Enhanced the "How to Test" documentation to provide clearer instructions on the demo replay validation process, including video and tlog alignment steps. - Updated architectural documentation to include the new demo replay operator flow and its dependencies. - Documented the removal of deprecated auto-sync features and clarified the operator-facing UI for replay validation. - Added new entries in the dependencies table for upcoming tasks related to the demo replay flow. These changes improve clarity and usability for operators and developers working with the demo replay system.
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -203,6 +203,17 @@ are all declared and documented below under **Cycle Check**.
|
||||
| AZ-951 | OKVIS2 v2 upstream patch: expose 6×6 pose covariance accessor (+ ADR for pin deviation) | 3 | AZ-332; AZ-592 | AZ-254 |
|
||||
| AZ-952 | OKVIS2 v2 upstream patch: expose tracking-stats accessor (counts + parallax + MRE) | 3 | AZ-332; AZ-592; AZ-951 (SOFT) | AZ-254 |
|
||||
| AZ-959 | replay_api: extend POST /replay to accept (video, csv) multipart for AZ-897 UI | 3 | AZ-701; AZ-894; AZ-896 | (none) |
|
||||
| AZ-969 | Demo replay operator flow (Epic) — F11 tlog+video align → cache seed → verdict | 21 (epic) | AZ-894; AZ-836; AZ-838; AZ-701; AZ-959 | AZ-897 |
|
||||
| AZ-970 | Tlog/video timeline preview API (AZ-969 C1) | 3 | AZ-697; AZ-836 | AZ-897; AZ-971 |
|
||||
| AZ-971 | Alignment library restore + refine (AZ-969 C2) | 5 | AZ-405 (historical) | AZ-972; AZ-973 |
|
||||
| AZ-972 | Aligned CSV export from tlog + offset (AZ-969 C3) | 3 | AZ-896; AZ-697; AZ-971; AZ-836 | AZ-973 |
|
||||
| AZ-973 | replay_api demo orchestration endpoints (AZ-969 C4) | 5 | AZ-970; AZ-971; AZ-972; AZ-974 (soft); AZ-960; AZ-701 | AZ-897 |
|
||||
| AZ-974 | C12 seed-cache-from-tlog production CLI (AZ-969 C5) | 3 | AZ-836; AZ-838; AZ-839; AZ-326 | AZ-973 (soft) |
|
||||
| AZ-975 | System design docs F11 + Invariant 15 (AZ-969 C6) | 2 | AZ-969 | (none) |
|
||||
| AZ-976 | gRPC streaming tile provision epic (ADR-013) | Epic ~13 | AZ-838; AZ-316; ADR-004 | AZ-977; AZ-978; AZ-979 |
|
||||
| AZ-977 | satellite-provider TileProvision gRPC service (AZ-976 C1) | 5 | AZ-976 | AZ-978 |
|
||||
| AZ-978 | C11 GrpcTileProvisionClient + C12 wiring (AZ-976 C2) | 5 | AZ-977; AZ-836; AZ-838; AZ-974 (soft) | AZ-979 |
|
||||
| AZ-979 | gRPC tile provision Jetson e2e + benchmark (AZ-976 C3) | 3 | AZ-977; AZ-978 | (none) |
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Replay: hard removal of deprecated auto-sync surface (AZ-895 follow-up)
|
||||
|
||||
> **BLOCKED by Epic AZ-969 (2026-06-19).** AZ-971 restores alignment kernels as operator-driven refine behind `replay_input/alignment.py`. Do not delete alignment logic until AZ-969 ships. AZ-908 scope shrinks to: remove deprecated CLI flags and `auto_sync.py` stub re-exports only — **not** the new alignment module.
|
||||
|
||||
**Task**: AZ-908_replay_auto_sync_hard_removal
|
||||
**Name**: Cycle-5+ cleanup that physically removes the auto-sync surface AZ-895 deprecated
|
||||
**Description**: Follow-up to AZ-895 (cycle 4). AZ-895 made the auto_sync surface a no-op and deprecated the CLI flags (`--time-offset-ms`, `--skip-auto-sync`, `--auto-trim`) with one-cycle warnings, but left the call sites, config fields, and interface DTOs intact for backward compat. AZ-908 completes the removal in cycle 5+ after a one-cycle deprecation window has passed.
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
# Operator replay sync UI (relocated)
|
||||
|
||||
**Task**: AZ-897_operator_replay_sync_ui
|
||||
**Tracker**: AZ-897
|
||||
**Repo**: `../ui` (Azaion suite front-end)
|
||||
|
||||
Authoritative spec: `ui/_docs/02_tasks/todo/AZ-897_operator_replay_sync_ui.md` (sibling repo at `../ui` relative to monorepo root).
|
||||
|
||||
Parent epic (backend): [AZ-969_demo_replay_operator_flow_epic.md](./AZ-969_demo_replay_operator_flow_epic.md)
|
||||
|
||||
Implement in the UI workspace. Backend blockers: AZ-970, AZ-973.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Demo replay operator flow (Epic)
|
||||
|
||||
**Task**: AZ-969_demo_replay_operator_flow_epic
|
||||
**Name**: Demo replay operator flow — tlog + video alignment → cache seed → airborne replay verdict
|
||||
**Description**: Promote the demo replay path from an e2e-test harness concern to a first-class operator workflow (F11). Given raw `(video, tlog, calibration)`, the system lets the operator align timelines in the suite UI, exports a canonical aligned CSV, seeds the satellite corridor cache from the tlog, runs the airborne replay pipeline, and returns a map + accuracy verdict. Supersedes the cycle-4 `(video, CSV)` upload-only shortcut as the **default** demo entry; CSV upload remains an advanced bypass.
|
||||
**Complexity**: Epic — ~21 SP across 6 backend children + AZ-897 UI (5 SP in `../ui`)
|
||||
**Dependencies**: AZ-894 (CSV adapter — done), AZ-836 (route extractor — done), AZ-838 (route client — done), AZ-701 (replay_api — done), AZ-959 (CSV API path — done)
|
||||
**Component**: cross-cutting — `replay_input`, `replay_api`, `c12_operator_orchestrator`, `c11_tile_manager`
|
||||
**Tracker**: AZ-969 (https://denyspopov.atlassian.net/browse/AZ-969)
|
||||
**Originating directive**: user (2026-06-19) — demo flow must accept tlog + video with manual alignment UI; not test-only.
|
||||
|
||||
## Goal
|
||||
|
||||
An operator with no Python install completes the full GPS-denied validation demo from the suite UI: upload → align → run → read verdict. The same code path powers Tier-2 e2e (`test_az835_e2e_real_flight`) without a separate test-only fixture graph.
|
||||
|
||||
## Pipeline (7 steps — production, not test-only)
|
||||
|
||||
| # | Step | Owner | New? |
|
||||
|---|------|-------|------|
|
||||
| 1 | Preview timelines (video metadata + tlog IMU2 activity) | AZ-970 `replay_api` | **New** |
|
||||
| 2 | Operator coarse-align + backend refine offset | AZ-897 UI + AZ-971 | **New** |
|
||||
| 3 | Export aligned CSV (`Time` col = video frame 0) | AZ-972 | **New** |
|
||||
| 4 | Extract route + seed corridor tiles + FAISS | AZ-974 (promotes AZ-836/838 from e2e fixture) | **Wire production** |
|
||||
| 5 | Run `gps-denied-replay` on `(video, aligned_csv)` | existing CLI + AZ-973 orchestration | existing |
|
||||
| 6 | Render map + verdict report | AZ-960 path | done |
|
||||
| 7 | Display in UI | AZ-897 | **New** |
|
||||
|
||||
## Decomposition
|
||||
|
||||
| # | Ticket | Est | Repo | Depends |
|
||||
|---|--------|-----|------|---------|
|
||||
| C1 | AZ-970 — tlog/video preview API | 3 | onboard | — |
|
||||
| C2 | AZ-971 — alignment library restore + refine | 5 | onboard | AZ-970 (soft) |
|
||||
| C3 | AZ-972 — aligned CSV export | 3 | onboard | AZ-971 |
|
||||
| C4 | AZ-973 — replay_api demo orchestration endpoints | 5 | onboard | AZ-972, AZ-974 (soft) |
|
||||
| C5 | AZ-974 — C12 `seed-cache-from-tlog` production CLI | 3 | onboard | AZ-836, AZ-838 |
|
||||
| C6 | AZ-975 — system design docs (F11, protocol, architecture) | 2 | onboard | C1–C5 specs |
|
||||
| UI | AZ-897 — dual-timeline sync UI | 5 | `../ui` | AZ-970, AZ-973 |
|
||||
|
||||
**Total ~21 SP backend + 5 SP UI.**
|
||||
|
||||
## Architectural decisions
|
||||
|
||||
1. **Single canonical clock preserved** — alignment happens **before** replay; exported CSV's `Time` column is authoritative (Invariant 14.a unchanged). Tlog runtime parsing is not reintroduced into `compose_root`.
|
||||
2. **Alignment is operator-visible** — auto-sync (AZ-405) is restored as a **refinement kernel** behind explicit operator consent, not a silent default.
|
||||
3. **Route seeding leaves test fixtures** — `extract_route_from_tlog` becomes a C12/replay_api production step, not only `operator_pre_flight_setup`.
|
||||
4. **AZ-908 deferred** — hard removal of alignment stubs blocked until AZ-971 lands; stub module renamed, not deleted.
|
||||
|
||||
## Acceptance criteria (Epic-level)
|
||||
|
||||
- **AC-1**: F11 documented in `system-flows.md` with sequence diagram; `architecture.md` lists demo flow alongside F1–F10.
|
||||
- **AC-2**: `POST /replay/demo` runs steps 3–6 without manual CLI on docker-compose dev stack.
|
||||
- **AC-3**: AZ-897 UI completes Derkachi demo end-to-end against local `replay_api`.
|
||||
- **AC-4**: `tests/e2e/replay/test_az835_e2e_real_flight.py` refactored to call production orchestration API/helpers — no parallel test-only graph.
|
||||
- **AC-5**: Advanced `(video, csv)` upload still works (AZ-959 regression green).
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Replacing live FC adapter with tlog at runtime (F3 stays live MAVLink).
|
||||
- OKVIS2 / AZ-943 chain.
|
||||
- Removing CSV bypass path (AZ-908 remains backlog after this epic).
|
||||
|
||||
## Coordination
|
||||
|
||||
- **AZ-897** spec: `../ui/_docs/02_tasks/todo/AZ-897_operator_replay_sync_ui.md`
|
||||
- **AZ-908** backlog: amend — do not execute until AZ-969 ships
|
||||
@@ -0,0 +1,79 @@
|
||||
# Tlog/video timeline preview API
|
||||
|
||||
**Task**: AZ-970_tlog_timeline_preview_api
|
||||
**Name**: `replay_api` preview endpoint — video metadata + tlog IMU2 activity timeline for AZ-897 UI
|
||||
**Description**: First backend building block of Epic AZ-969. Exposes `POST /replay/preview` accepting `(video, tlog)` multipart and returning JSON the dual-bar UI needs: video duration/fps/frame count, tlog duration, active-flight segment bounds, and per-bin IMU2 activity energy for heatmap rendering. Pure read-only — no alignment, no replay.
|
||||
**Complexity**: 3 SP
|
||||
**Dependencies**: AZ-697 (`load_tlog_ground_truth` — done), AZ-836 (`_detect_active_segment` semantics — reuse via shared trim helper or import)
|
||||
**Blocks**: AZ-897 (UI), AZ-971 (soft — refine can ship without preview in isolation but UI cannot)
|
||||
**Component**: `replay_api` + new `replay_input/tlog_timeline.py`
|
||||
**Tracker**: AZ-970
|
||||
**Parent Epic**: AZ-969
|
||||
|
||||
## Public surface
|
||||
|
||||
```python
|
||||
# replay_input/tlog_timeline.py
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Imu2ActivityBin:
|
||||
t_ms: int # bin start, FC-boot-relative ms
|
||||
energy: float # 0..1 normalized IMU2 magnitude
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class TlogTimelinePreview:
|
||||
duration_ms: int
|
||||
active_segment: tuple[int, int] # (start_idx, end_idx) into GPS rows
|
||||
active_start_ms: int
|
||||
active_end_ms: int
|
||||
imu2_activity: tuple[Imu2ActivityBin, ...]
|
||||
has_scaled_imu2: bool
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class VideoTimelinePreview:
|
||||
duration_ms: int
|
||||
frame_count: int
|
||||
fps: float
|
||||
|
||||
def build_tlog_timeline_preview(tlog: Path, *, bin_width_ms: int = 100) -> TlogTimelinePreview: ...
|
||||
def build_video_timeline_preview(video: Path) -> VideoTimelinePreview: ...
|
||||
```
|
||||
|
||||
## HTTP
|
||||
|
||||
`POST /replay/preview` — multipart `video` + `tlog` (both required).
|
||||
|
||||
Response 200:
|
||||
```json
|
||||
{
|
||||
"video": { "duration_ms": 490000, "frame_count": 14700, "fps": 30.0 },
|
||||
"tlog": {
|
||||
"duration_ms": 520000,
|
||||
"active_segment": [120, 4980],
|
||||
"active_start_ms": 12000,
|
||||
"active_end_ms": 498000,
|
||||
"imu2_activity": [{ "t_ms": 0, "energy": 0.02 }, ...],
|
||||
"has_scaled_imu2": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Errors: 400 missing file; 422 tlog missing SCALED_IMU2/RAW_IMU; 422 unreadable video.
|
||||
|
||||
## Implementation notes
|
||||
|
||||
- IMU2 energy: RMS of `(xacc,yacc,zacc)` from SCALED_IMU2 messages, binned, min-max normalized over full tlog.
|
||||
- Reuse active-segment thresholds from `extract_route_from_tlog` defaults for consistency.
|
||||
- Video probe via OpenCV `cv2.VideoCapture` — lazy-import gated like existing replay paths.
|
||||
- Optional: persist upload to temp job dir (same storage as AZ-701) and return `preview_id` for subsequent refine/demo calls.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: Derkachi tlog returns ≥ 1 activity peak in active segment; pre-takeoff bins < 0.15 normalized energy.
|
||||
- **AC-2**: Derkachi video returns fps within 0.5 of ffprobe ground truth.
|
||||
- **AC-3**: Unit tests for binning + normalization without disk video (synthetic IMU samples).
|
||||
- **AC-4**: Integration test in `test_az701_replay_api.py` for happy path + missing IMU types.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Thumbnail strip generation (UI may request later; optional `GET /replay/preview/{id}/frames` follow-up).
|
||||
- Alignment refine (AZ-971).
|
||||
@@ -0,0 +1,59 @@
|
||||
# Alignment library restore + refine offset
|
||||
|
||||
**Task**: AZ-971_alignment_library_restore_refine
|
||||
**Name**: Restore `replay_input` alignment kernels (AZ-405) as operator-driven refine behind explicit offset
|
||||
**Description**: Second building block of Epic AZ-969. AZ-895 replaced `auto_sync.py` with raising stubs. Restore the pure compute kernels from pre-AZ-895 history (`_compute_tlog_takeoff_from_samples`, `_compute_video_onset_from_samples`, `validate_offset_or_fail`, `find_aligned_window` from AZ-698) into a new module `replay_input/alignment.py`. Public API: `refine_video_offset(tlog, video, manual_offset_ms) -> AlignmentResult` — takes the operator's coarse bar offset and returns refined offset + confidence + frame-window match %. No silent auto-run at upload.
|
||||
**Complexity**: 5 SP
|
||||
**Dependencies**: AZ-405 (historical implementation — restore from git), AZ-698 (`find_aligned_window` — optional cross-correlation pass)
|
||||
**Blocks**: AZ-972, AZ-973
|
||||
**Component**: `replay_input/alignment.py`
|
||||
**Tracker**: AZ-971
|
||||
**Parent Epic**: AZ-969
|
||||
|
||||
## Public surface
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AlignmentResult:
|
||||
manual_offset_ms: int
|
||||
refined_offset_ms: int
|
||||
confidence: float # 0..1
|
||||
frame_window_match_pct: float # AC-8 metric
|
||||
hard_fail: bool
|
||||
|
||||
def refine_video_offset(
|
||||
tlog: Path,
|
||||
video: Path,
|
||||
manual_offset_ms: int,
|
||||
*,
|
||||
target_fc_dialect: str = "ardupilot_plane",
|
||||
match_threshold_pct: float = 95.0,
|
||||
) -> AlignmentResult: ...
|
||||
```
|
||||
|
||||
Semantics: `refined_offset_ms` = best offset after cross-correlating IMU energy (from manual anchor ± 2 s window) with video optical-flow onset. If `frame_window_match_pct < match_threshold_pct`, set `hard_fail=True` but still return best offset (UI decides whether to proceed).
|
||||
|
||||
## Scope
|
||||
|
||||
1. New `replay_input/alignment.py` with restored kernels (not re-exported from deprecated `auto_sync.py`).
|
||||
2. `auto_sync.py` stubs updated to delegate to `alignment` with deprecation warning OR left as-is until AZ-908 post-AZ-969.
|
||||
3. Unit tests ported from AZ-405 / AZ-698 test matrix (synthetic fixtures).
|
||||
4. `POST /replay/align/refine` handler stub in AZ-973 may call this module — implement library here first.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: Derkachi fixture with known ground-truth offset: `refine_video_offset` within ± 200 ms of truth when manual offset within ± 2 s.
|
||||
- **AC-2**: Deliberately wrong manual offset (± 30 s) → `hard_fail=True`, `frame_window_match_pct < 50`.
|
||||
- **AC-3**: Deterministic: same inputs → same `refined_offset_ms` within 1 ms.
|
||||
- **AC-4**: Missing SCALED_IMU2 → `ReplayInputAdapterError` at entry, not deep in OpenCV.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Automatic alignment without manual seed (operator must drag bar first).
|
||||
- Re-enabling `TlogReplayFcAdapter` in `compose_root`.
|
||||
- AZ-908 hard removal.
|
||||
|
||||
## Notes
|
||||
|
||||
- Restore source from commit before AZ-895 stub landing; do not resurrect `ReplayInputAdapter.open()` tlog path.
|
||||
- Keep OpenCV lazy-import discipline from batch 60.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Aligned CSV export from tlog + video offset
|
||||
|
||||
**Task**: AZ-972_aligned_csv_export
|
||||
**Name**: Export AZ-896 canonical CSV from tlog trimmed and aligned to video frame 0
|
||||
**Description**: Third building block of Epic AZ-969. Given `(tlog, video_offset_ms, optional active_segment)`, stream-parse the tlog and write a CSV matching `csv_replay_format.md`: `Time` column starts at 0.0 s at the video frame that aligns to the chosen tlog instant; only rows inside the active flight segment are exported; IMU + GLOBAL_POSITION_INT columns populated at 10 Hz (resample if needed).
|
||||
**Complexity**: 3 SP
|
||||
**Dependencies**: AZ-896 (format spec — done), AZ-697 (`load_tlog_ground_truth` / IMU parse), AZ-971 (refined offset input), AZ-836 (active segment detection — reuse)
|
||||
**Blocks**: AZ-973
|
||||
**Component**: `replay_input/tlog_to_csv.py` + CLI `gps-denied-tlog-to-csv`
|
||||
**Tracker**: AZ-972
|
||||
**Parent Epic**: AZ-969
|
||||
|
||||
## Public surface
|
||||
|
||||
```python
|
||||
def export_aligned_csv(
|
||||
tlog: Path,
|
||||
output_csv: Path,
|
||||
*,
|
||||
video_offset_ms: int,
|
||||
active_segment: tuple[int, int] | None = None,
|
||||
min_takeoff_speed_m_s: float = 2.0,
|
||||
min_takeoff_altitude_agl_m: float = 5.0,
|
||||
) -> Path: ...
|
||||
```
|
||||
|
||||
CLI: `gps-denied-tlog-to-csv --tlog PATH --output PATH --video-offset-ms N [--active-segment START,END]`
|
||||
|
||||
## Alignment math
|
||||
|
||||
Let `tlog_anchor_ms` be the FC-boot-relative instant matching video `t=0` after applying `video_offset_ms` (positive = video starts before tlog anchor). For each exported row at tlog time `t_fc_ms`:
|
||||
|
||||
`Time = (t_fc_ms - tlog_anchor_ms) / 1000.0`
|
||||
|
||||
Only rows with `Time >= 0` and within active segment are emitted. First row MUST have `Time == 0` within one IMU sample period (Invariant 14.a / AZ-896).
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: Round-trip: export Derkachi with known offset → `load_csv_ground_truth` → 10 Hz monotonic `Time`.
|
||||
- **AC-2**: `gps-denied-replay --video derkachi.mp4 --imu exported.csv` starts without `ReplayInputAdapterError`.
|
||||
- **AC-3**: Row count matches active segment duration × 10 Hz ± 1 row.
|
||||
- **AC-4**: Unit test: schema header exact match to `example_data_imu.csv`.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- PX4 / non-ArduPilot dialects.
|
||||
- Magnetometer columns (optional in AZ-896).
|
||||
@@ -0,0 +1,47 @@
|
||||
# replay_api demo orchestration endpoints
|
||||
|
||||
**Task**: AZ-973_replay_api_demo_orchestration
|
||||
**Name**: `replay_api` align/refine/export/demo endpoints — production F11 orchestrator
|
||||
**Description**: Fourth building block of Epic AZ-969. Extends `replay_api` with the operator demo job lifecycle: refine offset, export aligned CSV, run full pipeline (export → route seed → subprocess replay → map render → verdict). Replaces the ad-hoc wiring in `tests/e2e/replay/conftest.py` and `_operator_pre_flight.py` as the canonical orchestration surface for demo runs.
|
||||
**Complexity**: 5 SP
|
||||
**Dependencies**: AZ-970, AZ-971, AZ-972, AZ-974 (soft — demo can use pre-seeded cache env override), AZ-960 (map — done), AZ-701 (job storage — done)
|
||||
**Blocks**: AZ-897 (UI)
|
||||
**Component**: `replay_api`
|
||||
**Tracker**: AZ-973
|
||||
**Parent Epic**: AZ-969
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|--------|------|---------|
|
||||
| POST | `/replay/preview` | AZ-970 (may land in same or prior batch) |
|
||||
| POST | `/replay/align/refine` | Body/json: `{ job_id, video_offset_ms }` → `AlignmentResult` |
|
||||
| POST | `/replay/align/export` | Returns aligned CSV bytes or `{ csv_path }` in job dir |
|
||||
| POST | `/replay/demo` | multipart: `video`, `tlog`, `calibration`, `video_offset_ms` → starts async job |
|
||||
| GET | `/jobs/{id}` | Extend status with `phase`: `queued`, `aligning`, `exporting_csv`, `seeding_cache`, `replaying`, `rendering_map`, `complete`, `failed` |
|
||||
|
||||
## Demo job pipeline (in-process or subprocess chain)
|
||||
|
||||
1. Validate uploads; persist to job dir.
|
||||
2. `refine_video_offset` (AZ-971) — log refined offset; fail job if `hard_fail` and `REPLAY_API_STRICT_ALIGN=1`.
|
||||
3. `export_aligned_csv` (AZ-972) → `{job}/work/data_imu.csv`.
|
||||
4. `extract_route_from_tlog` + `SatelliteProviderRouteClient.seed_route` + tile download + FAISS build (delegate to shared helper extracted from `tests/e2e/replay/_operator_pre_flight.py` — **move to** `src/gps_denied_onboard/operator_replay/cache_seed.py` or `replay_api/orchestrator.py`).
|
||||
5. Shell `gps-denied-replay --video ... --imu ... --output ...` with populated `GPS_DENIED_OPERATOR_CONFIG_PATH` / cache mount.
|
||||
6. `_maybe_render_map` + verdict report (AZ-960 / AZ-699 paths).
|
||||
|
||||
## Refactor requirement
|
||||
|
||||
Extract `populate_c6_from_route` from test module into production package importable by both `replay_api` and C12. E2e fixture becomes thin wrapper calling production orchestrator. Satisfies Epic AC-4.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: `POST /replay/demo` on Derkachi fixtures (docker-compose) reaches `phase=complete` with map URL + verdict markdown path in response.
|
||||
- **AC-2**: `GET /jobs/{id}` exposes phase transitions in order.
|
||||
- **AC-3**: Unit tests mock satellite-provider; no network in unit tier.
|
||||
- **AC-4**: `test_az835_e2e_real_flight` refactored to call production orchestrator helper (same code path as API).
|
||||
- **AC-5**: AZ-959 `(video, csv)` bypass unchanged.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- WebSocket progress streaming (poll-only for v1).
|
||||
- Authentication changes beyond AZ-701 bearer token.
|
||||
@@ -0,0 +1,45 @@
|
||||
# C12 production CLI — seed cache from tlog route
|
||||
|
||||
**Task**: AZ-974_c12_seed_cache_from_tlog
|
||||
**Name**: C12 `seed-cache-from-tlog` — production binding for route-driven cache build (AZ-836 + AZ-838)
|
||||
**Description**: Fifth building block of Epic AZ-969. Promotes `extract_route_from_tlog` + `SatelliteProviderRouteClient.seed_route` + C11 tile download + C10 FAISS build from the e2e-only `operator_pre_flight_setup` fixture into the C12 operator CLI. Operators and `replay_api` demo jobs invoke the same production module — not test `conftest.py`.
|
||||
**Complexity**: 3 SP
|
||||
**Dependencies**: AZ-836, AZ-838, AZ-839 (fixture reference impl), AZ-326 (C12 CLI — done)
|
||||
**Blocks**: AZ-973 (soft — demo can seed inline via shared module landed here)
|
||||
**Component**: `c12_operator_orchestrator` + extracted `operator_replay/cache_seed.py`
|
||||
**Tracker**: AZ-974
|
||||
**Parent Epic**: AZ-969
|
||||
|
||||
## CLI
|
||||
|
||||
```
|
||||
gps-denied-operator seed-cache-from-tlog \
|
||||
--tlog PATH \
|
||||
--cache-root PATH \
|
||||
[--max-waypoints 10] \
|
||||
[--region-size-meters 500]
|
||||
```
|
||||
|
||||
Exit 0 on `PopulatedC6Cache` written; exit 2 on `RouteValidationError` / `RouteExtractionError`; exit 1 on transient exhaustion.
|
||||
|
||||
## Shared module
|
||||
|
||||
Move core of `tests/e2e/replay/_operator_pre_flight.py::populate_c6_from_route` to:
|
||||
|
||||
`src/gps_denied_onboard/operator_replay/cache_seed.py`
|
||||
|
||||
Public: `populate_c6_from_route(route_spec, *, cache_root, config) -> PopulatedC6Cache`
|
||||
|
||||
Imported by: C12 CLI, `replay_api` orchestrator (AZ-973), thinned e2e fixture.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: CLI succeeds against mock/real satellite-provider in docker-compose test stack.
|
||||
- **AC-2**: Output matches `PopulatedC6Cache` shape from AZ-839.
|
||||
- **AC-3**: `system-flows.md` F11 Phase 1 references this CLI — not "deferred to future cycle".
|
||||
- **AC-4**: E2e fixture imports production module; no duplicate logic in `tests/`.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Bbox-driven F1 Phase 1 (unchanged).
|
||||
- Companion NVM push (separate C12 bring-up).
|
||||
@@ -0,0 +1,30 @@
|
||||
# System design — F11 demo replay operator flow docs
|
||||
|
||||
**Task**: AZ-975_demo_replay_system_design_docs
|
||||
**Name**: Document F11 demo replay operator flow in system-flows, architecture, replay_protocol
|
||||
**Description**: Sixth building block of Epic AZ-969. Capture the demo replay path as a first-class system flow (F11), update architecture and replay protocol invariants, amend F1 route-driven variant to reference production C12/replay_api bindings, and cross-link AZ-897 UI spec.
|
||||
**Complexity**: 2 SP
|
||||
**Dependencies**: AZ-969 epic spec (this lands with or immediately after child specs)
|
||||
**Blocks**: (none)
|
||||
**Component**: `_docs/02_document/`
|
||||
**Tracker**: AZ-975
|
||||
**Parent Epic**: AZ-969
|
||||
|
||||
## Modified files
|
||||
|
||||
1. `_docs/02_document/system-flows.md` — add F11 to inventory + full section (sequence, flowchart, data flow).
|
||||
2. `_docs/02_document/architecture.md` — replace cycle-4 AZ-897 row; add § "Demo replay operator flow (cycle 5 — AZ-969)".
|
||||
3. `_docs/02_document/contracts/replay/replay_protocol.md` — add **Invariant 15** (operator demo path); note AZ-908 deferred.
|
||||
4. `_docs/how_to_test.md` — align with tlog+video UI flow (user-facing intent).
|
||||
5. `_docs/02_tasks/_dependencies_table.md` — register AZ-969 children.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: F11 appears in flow inventory; depends on F1 route variant + replay mode.
|
||||
- **AC-2**: Invariant 15 documents: raw upload → align → export CSV → single clock replay.
|
||||
- **AC-3**: No doc claims route seeding is "test-only" or "deferred" without pointing at AZ-974.
|
||||
- **AC-4**: `../ui` AZ-897 spec cross-linked.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Jira bulk sync (process leftover).
|
||||
@@ -0,0 +1,54 @@
|
||||
# gRPC streaming tile provision (Epic)
|
||||
|
||||
**Task**: AZ-976_grpc_tile_provision_epic
|
||||
**Name**: gRPC streaming tile provision — route + local index in, batched tiles out
|
||||
**Description**: Replace operator-side REST pre-flight tile transfer (`route poll` + `inventory` + per-tile GET) with a single gRPC server-streaming RPC. satellite-provider streams cached tiles immediately while fetching missing tiles from external imagery; gps-denied sends a local tile index so SP skips tiles the client already has at equal-or-better quality and equal-or-newer capture time. Documented in ADR-013 and `tile_provision.proto`.
|
||||
**Complexity**: Epic — ~13 SP across 3 children (split repos)
|
||||
**Dependencies**: AZ-838 (route client — done), AZ-316 (tile downloader — done), ADR-004 (operator-only boundary)
|
||||
**Component**: cross-cutting — `c11_tile_manager`, `c12_operator_orchestrator`, satellite-provider (sibling repo)
|
||||
**Tracker**: pending
|
||||
**Originating directive**: user (2026-06-19) — speed up pre-flight cache fill; gRPC streaming with client-side dedup index.
|
||||
|
||||
## Goal
|
||||
|
||||
Minimize wall-clock from route submit → C6 cache complete on the operator workstation. Time-to-first-tile and total bytes on the wire both improve vs REST.
|
||||
|
||||
## Pipeline
|
||||
|
||||
| Step | Owner | Mechanism |
|
||||
|------|-------|-----------|
|
||||
| 1 | C12 | Build `Route` + collect `local_tiles` from C6 (route bbox intersection) |
|
||||
| 2 | C11 | `DeliverRouteTiles` gRPC call |
|
||||
| 3 | satellite-provider | Skip dedup → stream `CACHED` batches → fetch externals → stream `FRESHLY_FETCHED` batches |
|
||||
| 4 | C11 | Write batches to C6 (existing gates) |
|
||||
| 5 | Operator | Stage C6 volume to Jetson (USB/rsync) — unchanged |
|
||||
|
||||
## Decomposition
|
||||
|
||||
| # | Ticket | Est | Repo | Depends |
|
||||
|---|--------|-----|------|---------|
|
||||
| C1 | AZ-977 — satellite-provider `RouteTileDelivery` gRPC service | 5 | `../satellite-provider` | — |
|
||||
| C2 | AZ-978 — C11 `RouteTileDeliveryClient` + C12 integration | 5 | onboard | AZ-977 |
|
||||
| C3 | AZ-979 — Jetson e2e smoke + ADR/doc sync | 3 | onboard + SP | AZ-978 |
|
||||
|
||||
**Total ~13 SP.**
|
||||
|
||||
## Acceptance criteria (Epic-level)
|
||||
|
||||
- **AC-1**: ADR-013 accepted in `architecture.md`; `tile_provision.proto` + `tile_provision_grpc.md` published.
|
||||
- **AC-2**: Derkachi corridor provision completes over gRPC with fewer round-trips than REST baseline (measured in AZ-979 report).
|
||||
- **AC-3**: Client local index suppresses re-transfer when C6 already holds equal-or-better tile (unit test on skip rule).
|
||||
- **AC-4**: Airborne image build excludes gRPC provision stubs (ADR-004 regression test unchanged).
|
||||
- **AC-5**: REST `route_client` + `HttpTileDownloader` remain as fallback until AZ-979 marks gRPC primary.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- In-flight tile download on the UAV (RESTRICT-SAT-1)
|
||||
- Implementing REST `POST /api/satellite/tiles/inventory` (superseded by this epic)
|
||||
- Browser/Web UI transport (operator CLI / C12 first)
|
||||
|
||||
## References
|
||||
|
||||
- ADR-013 — `_docs/02_document/architecture.md`
|
||||
- Proto — `_docs/02_document/contracts/c11_tilemanager/tile_provision.proto`
|
||||
- Contract — `_docs/02_document/contracts/c11_tilemanager/tile_provision_grpc.md`
|
||||
@@ -0,0 +1,23 @@
|
||||
# satellite-provider TileProvision gRPC service
|
||||
|
||||
**Task**: AZ-977_sp_tile_provision_grpc_service
|
||||
**Epic**: AZ-976
|
||||
**Name**: Implement `RouteTileDelivery.DeliverRouteTiles` in satellite-provider
|
||||
**Description**: Add gRPC host implementing `satellite.v1.RouteTileDelivery` from `tile_provision.proto`. Emit `RouteManifest` first, stream `TileBatch` (cached tiles before external fetch), optional `ProgressUpdate`, then `DeliveryComplete` or `DeliveryError`. JWT via gRPC metadata.
|
||||
**Complexity**: 5 SP
|
||||
**Dependencies**: AZ-976 (proto contract)
|
||||
**Component**: satellite-provider (sibling repo)
|
||||
**Tracker**: pending
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: `DeliverRouteTiles` stream matches `tile_provision_grpc.md` event sequence.
|
||||
- **AC-2**: Skip rule omits tiles when client snapshot is equal-or-better resolution and equal-or-newer `captured_at`.
|
||||
- **AC-3**: `phase=CACHED` batches emit before external fetch completes for on-disk hits.
|
||||
- **AC-4**: gRPC + existing REST coexist behind feature flag until AZ-979 flips default.
|
||||
- **AC-5**: OpenAPI/gRPC reflection or grpcurl smoke documented in satellite-provider README.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- gps-denied Python client (AZ-978)
|
||||
- Post-landing ingest (D-PROJ-2)
|
||||
@@ -0,0 +1,22 @@
|
||||
# C11 RouteTileDeliveryClient
|
||||
|
||||
**Task**: AZ-978_c11_grpc_tile_provision_client
|
||||
**Name**: Python gRPC consumer for RouteTileDelivery + C12 wiring
|
||||
**Description**: Implement `RouteTileDeliveryClient` in `c11_tile_manager` using `grpcio` + stubs from `tile_provision.proto`. Map internal `RouteSpec` → `satellite.v1.RouteSpec`; build `client_tiles` from C6; consume `RouteTileEvent` oneof (manifest, batch, progress, complete, error). Wire from C12 seed path behind `c11.tile_provision.transport: grpc|rest`.
|
||||
**Complexity**: 5 SP
|
||||
**Dependencies**: AZ-977, AZ-974 (soft), AZ-836, AZ-838
|
||||
**Component**: c11_tile_manager, c12_operator_orchestrator
|
||||
**Tracker**: pending
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: Unit tests with fake server cover manifest-first ordering and `batch_seq` resume per `route_id`.
|
||||
- **AC-2**: `local_tiles` populated from C6 metadata query intersecting route corridor.
|
||||
- **AC-3**: RESTRICT-SAT-4 / freshness / budget gates unchanged — reject bad tiles even if SP sent them.
|
||||
- **AC-4**: Generated stubs not imported by airborne/runtime_root build (BUILD flag or package split).
|
||||
- **AC-5**: Config default `rest` until AZ-979 benchmark promotes `grpc`.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- satellite-provider server (AZ-977)
|
||||
- Jetson benchmark report (AZ-979)
|
||||
@@ -0,0 +1,21 @@
|
||||
# gRPC tile provision e2e + benchmark
|
||||
|
||||
**Task**: AZ-979_grpc_tile_provision_e2e_benchmark
|
||||
**Epic**: AZ-976
|
||||
**Name**: Jetson e2e smoke and REST vs gRPC benchmark for tile provision
|
||||
**Description**: Add Tier-2 smoke test calling `RouteTileDeliveryClient` against real satellite-provider on Jetson harness. Benchmark wall-clock and bytes transferred vs REST path on Derkachi corridor. Update `architecture.md` integration table to mark gRPC primary. Document resume behaviour after disconnect.
|
||||
**Complexity**: 3 SP
|
||||
**Dependencies**: AZ-978, AZ-977
|
||||
**Component**: tests/e2e, docs
|
||||
**Tracker**: pending
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- **AC-1**: `tests/e2e/satellite_provider/test_grpc_provision.py` passes on Jetson with `JETSON_SSH_ALIAS=jetson`.
|
||||
- **AC-2**: Benchmark report in `_docs/06_metrics/` with REST vs gRPC timings and byte counts.
|
||||
- **AC-3**: `docker-compose.test.jetson.yml` exposes gRPC port for satellite-provider.
|
||||
- **AC-4**: `c11.tile_provision.transport` default flipped to `grpc` after green benchmark.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Deprecating REST route_client in same ticket (follow-up after soak)
|
||||
Reference in New Issue
Block a user