azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-22 07:01:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-839] [AZ-835] operator_pre_flight_setup real fixture (E-AZ-835 C3)

Browse Source 
Replace the placeholder operator_pre_flight_setup pytest fixture (the
mkdir stub at tests/e2e/replay/conftest.py:293-310) with a real driver
that wires C1 (AZ-836 RouteSpec) + C2 (AZ-838 SatelliteProviderRoute
Client) + C11 (AZ-316 HttpTileDownloader) + C10 (AZ-322 Descriptor
Batcher) end-to-end and yields a typed PopulatedC6Cache. AZ-306 FAISS
sidecar triple-consistency is verified post-rebuild via a caller-
supplied descriptor_index_factory; partial sidecars are cleaned up on
failure (AC-7) while pre-existing warm-cache files are preserved.
Algorithm lives in tests/e2e/replay/_operator_pre_flight.py with
pure dependency injection so the AC-8 unit suite (11 tests covering
happy / transient-retry / terminal-failure / validation-error /
tamper-detection / cleanup-on-failure) runs against stubs and the
AC-9 Tier-2 integration test runs the same algorithm against the
real Jetson harness. The conftest fixture skip-gates on RUN_REPLAY
_E2E + SATELLITE_PROVIDER_URL/API_KEY + BUILD_FAISS_INDEX +
GPS_DENIED_OPERATOR_CONFIG_PATH and wires deps through the existing
runtime_root factories. Supersedes AZ-777 Phase 3.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-23 15:08:34 +03:00
parent 0ed1a5d988
commit bfcac2cb9f
7 changed files with 1544 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_tasks/todo/AZ-839_operator_pre_flight_setup_real_fixture.md
-85
View File
@@ -1,85 +0,0 @@
# operator_pre_flight_setup real fixture (AZ-835 C3)
**Task**: AZ-839_operator_pre_flight_setup_real_fixture
**Name**: operator_pre_flight_setup fixture: wire C1+C2+C11+C10 into real fixture, supersede AZ-777 Phase 3 (AZ-835 C3)
**Description**: Third building block of Epic AZ-835. Replace the placeholder `operator_pre_flight_setup` fixture (currently a `mkdir` stub at `tests/e2e/replay/conftest.py` lines 293-310) with a real driver that wires C1 (AZ-836) + C2 (AZ-838) + C11 (AZ-777 Phase 1) + C10 to populate C6 from a tlog-derived route. Supersedes AZ-777 Phase 3 (the bbox-seeded placeholder-replacement design) per the 2026-05-22 user directive — route-driven seeding is ~100x more tile-efficient and pre-commits to where the operator did fly per the tlog.
**Complexity**: 5 SP
**Dependencies**: AZ-836 (C1, RouteSpec + extractor — In Testing); AZ-838 (C2, SatelliteProviderRouteClient + seed_route.py CLI — In Testing); AZ-777 Phase 1 (e2e-runner ↔ satellite-provider wire + C11 contract adaptation — done, batch 104); AZ-322 (C10 DescriptorBatcher — done); AZ-316+AZ-777 Phase 1 (C11 HttpTileDownloader.download_for_bbox — done); AZ-306 (FAISS sidecar triple-consistency — done); AZ-835 (parent Epic)
**Component**: `tests/e2e/replay/conftest.py` (`operator_pre_flight_setup` fixture rewrite + new `PopulatedC6Cache` dataclass)
**Tracker**: AZ-839 (https://denyspopov.atlassian.net/browse/AZ-839)
**Parent Epic**: AZ-835
Jira AZ-839 is the authoritative spec; this file is the in-workspace mirror.
## Public surface
```python
from dataclasses import dataclass
from pathlib import Path
from gps_denied_onboard.replay_input.tlog_route import RouteSpec # AZ-836
@dataclass(frozen=True, slots=True)
class PopulatedC6Cache:
cache_root: Path # named-volume mount inside the e2e-runner container
tile_store_path: Path # postgres + filesystem store root
faiss_index_path: Path # .index file
faiss_sidecar_sha256_path: Path # .sha256 file
faiss_sidecar_meta_path: Path # .meta.json file
route_spec: RouteSpec # provenance — which tlog/route produced this cache
tile_count: int # how many tiles ended up in C6
elapsed_seconds: float # wall time, for the AC-1/AC-2 perf budget
```
The fixture remains a pytest fixture at `tests/e2e/replay/conftest.py::operator_pre_flight_setup`, same `session` scope as today. Input contract unchanged (same args the placeholder takes) plus a new dependency on `RouteSpec` — either fixture-injected or extracted from the test's tlog parameter via `extract_route_from_tlog`.
## Behaviour
1. Read the route spec (fixture-injected or extracted from test tlog via `extract_route_from_tlog`).
2. Instantiate `SatelliteProviderRouteClient` from env (`SATELLITE_PROVIDER_URL`, `SATELLITE_PROVIDER_API_KEY`, `SATELLITE_PROVIDER_TLS_INSECURE`).
3. Call `seed_route(route_spec)`. On `RouteValidationError` / `RouteTerminalFailureError` → re-raise with original cause. On `RouteTransientError` → retry up to 3 attempts using C11's `_DEFAULT_BACKOFF_SCHEDULE_S = (1, 2, 4, 8)`.
4. Enumerate tile coverage locally (mirror `route_client._enumerate_route_tile_coords` from AZ-838); call C11 `HttpTileDownloader.download_for_bbox` to pull every tile into C6.
5. Invoke C10 `DescriptorBatcher` against the populated C6 to build the FAISS HNSW index using the NetVLAD backbone (per `c2_vpr/config.py:67` default).
6. Verify sidecar coherence (`.index` + `.sha256` + `.meta.json` triple-consistency per AZ-306). Mismatch → `IndexUnavailableError`.
7. Yield `PopulatedC6Cache(...)`. On any failure path, clean up partial cache state (no half-built FAISS index left behind).
**Mount strategy**: write into a named docker volume that survives across pytest sessions. Cold first invocation populates; subsequent invocations within the same compose session reuse (warm cache). Same pattern AZ-777 Phase 3 originally specced; only the cache **source** changes (route, not bbox).
## Acceptance criteria
| # | Criterion |
|---|-----------|
| AC-1 | Cold first invocation on the Derkachi tlog completes in ≤ 5 min on Tier-2 Jetson (includes satellite-provider Google Maps round-trips). |
| AC-2 | Warm invocation within the same compose session completes in ≤ 30 s (named-volume reuse). |
| AC-3 | Yielded `PopulatedC6Cache` has all paths populated; `tile_count > 0`; FAISS sidecar triple-consistency passes (AZ-306). |
| AC-4 | `RouteValidationError` / `RouteTerminalFailureError` from `seed_route` is re-raised with original cause; no silent swallow. |
| AC-5 | `RouteTransientError` is retried up to 3 attempts using C11's existing backoff schedule; final attempt's exception is propagated. |
| AC-6 | Tamper test — corrupt one of the three sidecar files between fixture runs; next invocation raises `IndexUnavailableError`. |
| AC-7 | On any failure path inside the fixture, partial state is cleaned up (no half-built FAISS index, no orphaned postgres rows). |
| AC-8 | Unit tests (stubbed `SatelliteProviderRouteClient` + stubbed C11 + stubbed C10) cover: happy path, transient-retry, terminal-failure, validation-error, tamper-detection, cleanup-on-failure. |
| AC-9 | Integration test gated by `RUN_REPLAY_E2E=1` + `@pytest.mark.tier2` against the Jetson harness produces a real `PopulatedC6Cache` from the Derkachi tlog. |
## Out of scope
- Driving the airborne replay pipeline against the populated cache (AZ-840 / C4).
- Un-xfailing the existing AZ-777 AC-4 / AC-5 tests (AZ-841 / C5).
- Updating `replay_protocol.md` (AZ-842 / C6).
- Switching the C2 default backbone away from NetVLAD.
- Multi-tlog aggregate caches (one route per fixture invocation).
## Risks
**Risk 1 — Docker named-volume lifecycle across pytest sessions.** First invocation may leave half-populated volume on crash; the cleanup-on-failure path in step 7 must be robust. Mitigation: AC-7 covers explicitly + a `try/finally` around the four wiring steps.
**Risk 2 — Cold-start budget (AC-1, 5 min) tight on first Jetson run.** Google Maps round-trips for ~50-100 tiles may exceed budget on slow networks. Mitigation: instrument elapsed_seconds on every step and surface in the verdict report; if AC-1 fails, file a perf-tuning ticket rather than skipping the AC.
## References
- Parent Epic: AZ-835 — https://denyspopov.atlassian.net/browse/AZ-835
- Existing placeholder: `tests/e2e/replay/conftest.py` lines 293-310
- C1: AZ-836 (`extract_route_from_tlog`) — https://denyspopov.atlassian.net/browse/AZ-836
- C2: AZ-838 (`SatelliteProviderRouteClient`) — https://denyspopov.atlassian.net/browse/AZ-838
- AZ-777 (Phase 3+ superseded): `_docs/02_tasks/done/AZ-777_derkachi_c6_reference_fixture.md`
- C10 DescriptorBatcher: `src/gps_denied_onboard/components/c10_provisioning/descriptor_batcher.py`
- C11 HttpTileDownloader: `src/gps_denied_onboard/components/c11_tile_manager/tile_downloader.py`
- AZ-306 FAISS sidecar triple-consistency reference