mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 07:41:12 +00:00
Update autodev state, architecture documentation, and glossary terms
Transitioned the autodev state to phase 21, reflecting the completion of Step 5 and the drafting of Step 6 epics. Revised the architecture documentation to clarify the roles of the Tile Manager and its components, ensuring accurate representation of the system's operational flow. Updated glossary entries for Flight State and Operator to incorporate recent changes and enhance clarity on component interactions and responsibilities.
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,138 @@
|
||||
# C2 — Visual Place Recognition
|
||||
|
||||
## 1. High-Level Overview
|
||||
|
||||
**Purpose**: given the current `NavCameraFrame`, retrieve the top-K=10 candidate satellite tiles from the pre-cached corpus by descriptor similarity. C2 owns the *retrieval* step; C2.5 narrows K=10 → N=3 via inlier-based re-rank.
|
||||
|
||||
**Architectural Pattern**: Strategy — `VprStrategy` interface; concrete implementations (UltraVPR primary, MegaLoc secondary, MixVPR / SelaVPR / EigenPlaces / NetVLAD / SALAD additional candidates) selected at startup by config (ADR-001); build-time gated per-implementation by `BUILD_*` flags (ADR-002); composition-root wired (ADR-009).
|
||||
|
||||
**Upstream dependencies**:
|
||||
- Camera ingest thread → `NavCameraFrame` (parallel fan-out with C1; same frame, distinct queue depth).
|
||||
- C7 InferenceRuntime → backbone forward pass (TRT/ONNX/PyTorch per active runtime).
|
||||
- C6 DescriptorIndex → FAISS HNSW lookup over pre-cached tile descriptors.
|
||||
- Camera calibration artifact — for backbone input preprocessing (resize/crop/normalise).
|
||||
|
||||
**Downstream consumers**:
|
||||
- C2.5 ReRanker (consumes `VprResult`).
|
||||
|
||||
## 2. Internal Interfaces
|
||||
|
||||
### Interface: `VprStrategy`
|
||||
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
| `embed_query` | `NavCameraFrame, CameraCalibration` | `VprQuery` | No | `VprBackboneError` |
|
||||
| `retrieve_topk` | `VprQuery, k: int` | `VprResult` | No | `IndexUnavailableError`, `VprBackboneError` |
|
||||
| `descriptor_dim` | `()` | `int` | No | — |
|
||||
|
||||
**Input DTOs**:
|
||||
```
|
||||
NavCameraFrame: see C1 spec — same DTO
|
||||
|
||||
VprQuery:
|
||||
frame_id: uuid (required)
|
||||
embedding: ndarray[D, dtype=float16|float32] (required) — D depends on backbone
|
||||
produced_at: monotonic_ns
|
||||
```
|
||||
|
||||
**Output DTOs**:
|
||||
```
|
||||
VprResult:
|
||||
frame_id: uuid
|
||||
candidates: list[VprCandidate] (length = k, ranked by descriptor distance ascending)
|
||||
retrieved_at: monotonic_ns
|
||||
backbone_label: string — for FDR provenance
|
||||
|
||||
VprCandidate:
|
||||
tile_id: composite (zoomLevel, lat, lon)
|
||||
descriptor_distance: float — backbone-specific metric (cosine for L2-normalised embeddings)
|
||||
descriptor_dim: int
|
||||
```
|
||||
|
||||
## 3. External API Specification
|
||||
|
||||
Not applicable — internal-only component.
|
||||
|
||||
## 4. Data Access Patterns
|
||||
|
||||
### Queries
|
||||
|
||||
| Query | Frequency | Hot Path | Index Needed |
|
||||
|-------|-----------|----------|--------------|
|
||||
| FAISS HNSW top-K=10 search | 3 Hz (per nav frame) | Yes | Yes — pre-built HNSW (C6) |
|
||||
|
||||
### Caching Strategy
|
||||
|
||||
| Data | Cache Type | TTL | Invalidation |
|
||||
|------|-----------|-----|-------------|
|
||||
| Backbone weights | TRT engine on disk + GPU resident | flight lifetime | Manifest content-hash gate (D-C10-3) at takeoff |
|
||||
| FAISS HNSW index | mmap (C6 owns the file) | flight lifetime | Same as above |
|
||||
|
||||
### Storage Estimates
|
||||
|
||||
C2 itself stores no persistent data; it consumes C6's descriptor index. Sizing belongs in C6.
|
||||
|
||||
### Data Management
|
||||
|
||||
C2 is read-only against C6 during F3/F4/F6. Pre-flight, F1 triggers C10 (after C11 `TileDownloader` has populated C6) to call `embed_query` on every staged tile to populate the descriptor matrix consumed by C6.
|
||||
|
||||
## 5. Implementation Details
|
||||
|
||||
**Algorithmic Complexity**: HNSW search is `O(log N)` in corpus size for k=10; backbone forward pass is `O(1)` per frame (GPU-bound).
|
||||
|
||||
**State Management**: stateless per-frame; the only persistent state is the loaded backbone weights and the FAISS index pointer (held by C6 and passed in via constructor).
|
||||
|
||||
**Key Dependencies**:
|
||||
|
||||
| Library | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| FAISS (Python + C++) | upstream HEAD pinned per Plan-phase | HNSW retrieval; consumed via C6 |
|
||||
| TensorRT | 10.3 (JetPack 6.2 pin) | Primary inference backend; consumed via C7 |
|
||||
| ONNX Runtime + TRT EP | matches C7 | Fallback backend |
|
||||
| PyTorch | matches simple-baseline track | FP16 baseline (NetVLAD / MixVPR mandatory) |
|
||||
| UltraVPR (research code drop) | upstream HEAD pinned per Plan-phase | Documentary Lead PRIMARY backbone |
|
||||
| MegaLoc, MixVPR, SelaVPR, EigenPlaces, NetVLAD | upstream HEAD pinned per Plan-phase | Secondary + mandatory simple-baselines |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
- `VprBackboneError`: backbone forward pass failed (CUDA OOM, TRT engine deserialize mismatch). C2 emits no `VprResult`; C5 falls back to VIO-only with provenance label `visual_propagated` (AC-1.4).
|
||||
- `IndexUnavailableError`: FAISS index handle invalid (e.g., post-F8 reboot before warm-up). Same fallback as above; F8 recovery flow re-mmaps the index.
|
||||
|
||||
## 6. Extensions and Helpers
|
||||
|
||||
| Helper | Purpose | Used By |
|
||||
|--------|---------|---------|
|
||||
| `BackbonePreprocessor` | resize / crop / normalise per backbone's input contract | C2 only — keep inside the component, not a shared helper |
|
||||
| `DescriptorNormaliser` | L2-normalise descriptors so cosine similarity aligns with Euclidean | C2 (query side), C10 (corpus side at cache artifact build) |
|
||||
|
||||
## 7. Caveats & Edge Cases
|
||||
|
||||
**Known limitations**:
|
||||
- VPR is sensitive to scene change between cache build and flight time — AC-NEW-6 freshness gating is the project-level mitigation, not a C2 concern.
|
||||
- Backbone choice is constrained by ADR-002: only the linked-in implementations are selectable at runtime.
|
||||
|
||||
**Potential race conditions**:
|
||||
- Concurrent `embed_query` calls on a single strategy instance can race on the GPU stream. Bind one strategy instance to one ingest thread — composition root enforces.
|
||||
|
||||
**Performance bottlenecks**:
|
||||
- Backbone forward pass is the dominant cost (~30–80 ms on Jetson per backbone). FAISS HNSW search is sub-millisecond for 100k-tile corpora.
|
||||
- D-CROSS-LATENCY-1 hybrid does not change C2 behaviour — C2's budget is fixed; the auto-degrade happens at C4.
|
||||
|
||||
## 8. Dependency Graph
|
||||
|
||||
**Must be implemented after**: C6 (descriptor index), C7 (inference runtime), C10 (descriptor population at cache artifact build).
|
||||
|
||||
**Can be implemented in parallel with**: C1, C8 — independent paths.
|
||||
|
||||
**Blocks**: C2.5 (no candidates without `VprResult`), F3 / F6.
|
||||
|
||||
## 9. Logging Strategy
|
||||
|
||||
| Log Level | When | Example |
|
||||
|-----------|------|---------|
|
||||
| ERROR | `VprBackboneError` or `IndexUnavailableError` | `VPR backbone OOM: backbone=ultravpr, frame=12345` |
|
||||
| WARN | top-1 distance exceeds drift threshold (potential false-positive retrieval) | `VPR top-1 distance 0.42 above warn threshold 0.30; backbone=ultravpr` |
|
||||
| INFO | Strategy ready; backbone loaded | `VPR ready: backbone=ultravpr, dim=512, corpus_size=87654` |
|
||||
| DEBUG | Per-frame top-K distances | `VPR frame=12345 top10_distances=[0.12, 0.14, ...]` |
|
||||
|
||||
**Log format**: structured JSON.
|
||||
**Log storage**: stdout / journald / FDR via C13 (ERROR + WARN only).
|
||||
@@ -0,0 +1,144 @@
|
||||
# Test Specification — C2 Visual Place Recognition
|
||||
|
||||
Component-scoped. Suite-level coverage is in `_docs/02_document/tests/*.md`; canonical traceability is `tests/traceability-matrix.md`.
|
||||
|
||||
## Acceptance Criteria Traceability
|
||||
|
||||
| AC ID | Acceptance Criterion (one-line) | Test IDs | Coverage |
|
||||
|-------|---------------------------------|----------|----------|
|
||||
| AC-2.1b | Satellite-anchor registration meets AC-1.1/1.2/2.2/8.2/8.6 | FT-P-05, FT-P-19, **C2-IT-01** | Covered |
|
||||
| AC-2.2 (cross-domain portion) | MRE <2.5 px cross-domain | FT-P-06, **C2-IT-02** (recall floor only — MRE owned by C3) | Covered |
|
||||
| AC-4.1 | E2E latency <400 ms p95 | NFT-PERF-01, **C2-PT-01** | Covered |
|
||||
| AC-NEW-7 | Cache poisoning safety budget | NFT-SEC-01 (onboard-side), **C2-IT-03** | Covered (relaxed) |
|
||||
| AC-8.6 (scale-ratio portion) | Scale-ratio satellite relocalization | FT-P-19, **C2-IT-04** | Covered |
|
||||
|
||||
---
|
||||
|
||||
## Component-Internal Tests
|
||||
|
||||
### C2-IT-01: top-K=10 recall at p=10 on Derkachi
|
||||
|
||||
**Summary**: the primary backbone (UltraVPR) achieves recall@10 ≥ 0.95 on the Derkachi normal segment against the pre-built corpus.
|
||||
|
||||
**Traces to**: AC-2.1b
|
||||
|
||||
**Description**: for each query frame in `flight_derkachi/normal_segment_60_stills/`, embed via UltraVPR; query the FAISS HNSW index; assert that the ground-truth tile (per recorded GPS + sector classification) is within the top-10 candidates ≥95% of frames.
|
||||
|
||||
**Input data**: `flight_derkachi/normal_segment_60_stills/` + the Derkachi corpus FAISS index built by C10 (consumed read-only here).
|
||||
|
||||
**Expected result**: recall@10 ≥ 0.95 for UltraVPR; recall@10 ≥ 0.85 for the mandatory simple-baseline NetVLAD (engine rule check).
|
||||
|
||||
**Max execution time**: 90 s.
|
||||
|
||||
---
|
||||
|
||||
### C2-IT-02: VprResult schema invariants
|
||||
|
||||
**Summary**: every `VprResult` carries `len(candidates) == k`, monotonically-non-decreasing `descriptor_distance`, and a non-empty `backbone_label`.
|
||||
|
||||
**Traces to**: AC-2.2 (downstream-coverage prerequisite)
|
||||
|
||||
**Description**: 100 frames through `retrieve_topk(k=10)`; assert (a) length, (b) sorted-ascending distance, (c) `backbone_label` non-empty.
|
||||
|
||||
**Input data**: `tests/fixtures/synthetic_vpr/diverse_100f/`.
|
||||
|
||||
**Expected result**: 100/100 results pass all invariants.
|
||||
|
||||
**Max execution time**: 10 s.
|
||||
|
||||
---
|
||||
|
||||
### C2-IT-03: cache-poisoning seed rejection at retrieval
|
||||
|
||||
**Summary**: when the corpus contains a poisoned tile injected during NFT-SEC-01 setup, the top-1 distance to that tile is bounded so downstream RANSAC + voting can reject it within the AC-NEW-7 relaxed budget.
|
||||
|
||||
**Traces to**: AC-NEW-7 (component-level partition)
|
||||
|
||||
**Description**: load the NFT-SEC-01 setup corpus (3-flight cumulative dataset with deflated covariance × 1.5–3); query each Derkachi frame; record the top-1 candidate for each. Pass if the poisoned tile is top-1 in fewer than the relaxed-CI threshold (the relaxation per AC-text 2026-05-09).
|
||||
|
||||
**Input data**: NFT-SEC-01 setup corpus.
|
||||
|
||||
**Expected result**: poisoned-tile top-1 rate within AC-NEW-7 relaxed CI.
|
||||
|
||||
**Max execution time**: 5 min.
|
||||
|
||||
---
|
||||
|
||||
### C2-IT-04: scale-ratio invariance on satellite re-loc
|
||||
|
||||
**Summary**: when the nav-camera scale changes (e.g., altitude shift), VPR top-K still surfaces the correct tile.
|
||||
|
||||
**Traces to**: AC-8.6 (scale-ratio half)
|
||||
|
||||
**Description**: synthetically scale the Derkachi normal-segment frames by ±20% (mimicking altitude variation); assert recall@10 stays ≥ 0.85 across the scale sweep.
|
||||
|
||||
**Input data**: `flight_derkachi/scaled_+20%/` and `flight_derkachi/scaled_-20%/` (generated via deterministic resize).
|
||||
|
||||
**Expected result**: recall@10 ≥ 0.85 at both scale extremes for UltraVPR.
|
||||
|
||||
**Max execution time**: 90 s.
|
||||
|
||||
---
|
||||
|
||||
## Performance Tests
|
||||
|
||||
### C2-PT-01: backbone forward + HNSW lookup budget on Tier-2
|
||||
|
||||
**Traces to**: AC-4.1
|
||||
|
||||
**Load scenario**: 3 Hz frame rate, 10 min replay; corpus size 87 654 tiles (Derkachi area at 0.5 m/px).
|
||||
|
||||
**Expected results**:
|
||||
|
||||
| Metric | Target | Failure Threshold |
|
||||
|--------|--------|-------------------|
|
||||
| `embed_query` p95 | ≤ 60 ms (UltraVPR / FP16) | 100 ms |
|
||||
| `retrieve_topk(k=10)` p95 | ≤ 2 ms (HNSW) | 10 ms |
|
||||
| Combined p95 | ≤ 65 ms | 110 ms |
|
||||
|
||||
**Resource limits**:
|
||||
- GPU memory: ≤ 600 MB resident for backbone weights.
|
||||
- System memory: ≤ 200 MB for the mmap'd FAISS index handle.
|
||||
|
||||
---
|
||||
|
||||
## Security Tests
|
||||
|
||||
### C2-ST-01: index-handle invalidation safety
|
||||
|
||||
**Summary**: after C10 rebuilds the FAISS index (post-takeoff is FORBIDDEN, but unit-level safety check), the previous handle held by C2 must not silently return stale results.
|
||||
|
||||
**Traces to**: defensive — no AC trace; backstops a code-injection / config-drift mode that AC-NEW-7 already covers at the suite level.
|
||||
|
||||
**Test procedure**:
|
||||
1. Build a tiny 100-tile FAISS index; mmap it through C2.
|
||||
2. Replace the underlying file (simulating an out-of-band rebuild, which is FORBIDDEN at flight time per D-C10-3 but defended-in-depth here).
|
||||
3. Call `retrieve_topk` and assert C2 raises `IndexUnavailableError` rather than returning stale candidates.
|
||||
|
||||
**Pass criteria**: `IndexUnavailableError` raised; no candidates returned.
|
||||
**Fail criteria**: any candidate returned.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Tests
|
||||
|
||||
C2 contributes to AC-2.1b / AC-8.6 via the suite-level FT scenarios. No additional C2-only acceptance tests.
|
||||
|
||||
---
|
||||
|
||||
## Test Data Management
|
||||
|
||||
| Data Set | Description | Source | Size |
|
||||
|----------|-------------|--------|------|
|
||||
| `synthetic_vpr/diverse_100f/` | 100 diverse synthetic frames for invariant checks | generated, deterministic | ~50 MB |
|
||||
| `flight_derkachi/normal_segment_60_stills/` | shared with C1 | curated | shared |
|
||||
| `flight_derkachi/scaled_±20%/` | scale-ratio sweep | generated from above | ~40 MB |
|
||||
| Derkachi FAISS corpus | C10's output, consumed read-only | C10 build artifact | ~200 MB |
|
||||
|
||||
**Setup procedure**:
|
||||
1. C10 must have built the Derkachi FAISS index (this is a Step-5-test-side prereq; in the test runner, the corpus is staged from `tests/fixtures/cache_artifacts/`).
|
||||
2. Synthetic + scaled fixtures generated by deterministic scripts.
|
||||
|
||||
**Teardown**: corpus is read-only; nothing to clean up.
|
||||
|
||||
**Data isolation**: each test gets its own `tests/tmp/c2/<test-id>/`.
|
||||
Reference in New Issue
Block a user