mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 11:31:13 +00:00
Update autodev state, architecture documentation, and glossary terms
Transitioned the autodev state to phase 21, reflecting the completion of Step 5 and the drafting of Step 6 epics. Revised the architecture documentation to clarify the roles of the Tile Manager and its components, ensuring accurate representation of the system's operational flow. Updated glossary entries for Flight State and Operator to incorporate recent changes and enhance clarity on component interactions and responsibilities.
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,127 @@
|
||||
# C3 — Cross-domain Matcher
|
||||
|
||||
## 1. High-Level Overview
|
||||
|
||||
**Purpose**: produce 2D-3D correspondences between the current `NavCameraFrame` and the top-N=3 satellite tiles from C2.5, with RANSAC-filtered inliers and reprojection residual statistics. C3 is the **cross-domain** step (nav-camera ↔ satellite-imagery domain gap) and is the dominant compute cost in F3.
|
||||
|
||||
**Architectural Pattern**: Strategy — `CrossDomainMatcher` interface, with concrete implementations DISK+LightGlue (D-C3-1 = (a) primary), ALIKED+LightGlue (secondary), XFeat (alternate). Selection at startup by config (ADR-001); build-time gating by `BUILD_*` flags (ADR-002); composition-root wired (ADR-009).
|
||||
|
||||
**Upstream dependencies**:
|
||||
- C2.5 → `RerankResult` (top-N=3 candidates).
|
||||
- C7 InferenceRuntime → backbone forward pass.
|
||||
- Camera calibration artifact — for nav-frame preprocessing.
|
||||
- C6 TileStore — for tile pixels (handle carried in `RerankCandidate`).
|
||||
|
||||
**Downstream consumers**:
|
||||
- C3.5 ConditionalRefiner (consumes `MatchResult`; passthrough or AdHoP refinement).
|
||||
|
||||
## 2. Internal Interfaces
|
||||
|
||||
### Interface: `CrossDomainMatcher`
|
||||
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
| `match` | `NavCameraFrame, RerankResult, CameraCalibration` | `MatchResult` | No | `MatcherBackboneError`, `InsufficientInliersError` |
|
||||
| `health_snapshot` | `()` | `MatcherHealth` | No | — |
|
||||
|
||||
**Input DTOs**:
|
||||
```
|
||||
NavCameraFrame: see C1
|
||||
RerankResult: see C2.5
|
||||
CameraCalibration: see C5
|
||||
```
|
||||
|
||||
**Output DTOs**:
|
||||
```
|
||||
MatchResult:
|
||||
frame_id: uuid
|
||||
per_candidate: list[CandidateMatchSet] (length up to N=3, drop on failure)
|
||||
best_candidate_idx: int — argmax(inlier_count) among per_candidate
|
||||
reprojection_residual_px: float — best candidate's median residual
|
||||
matched_at: monotonic_ns
|
||||
matcher_label: string — for FDR provenance
|
||||
|
||||
CandidateMatchSet:
|
||||
tile_id: composite (zoomLevel, lat, lon)
|
||||
inlier_count: int
|
||||
inlier_correspondences: ndarray[I, 4, dtype=float32] — (px_query, py_query, px_tile, py_tile)
|
||||
ransac_outlier_count: int
|
||||
per_candidate_residual_px: float
|
||||
|
||||
MatcherHealth:
|
||||
consecutive_low_inlier: int
|
||||
mean_inliers_60s: float
|
||||
```
|
||||
|
||||
## 3. External API Specification
|
||||
|
||||
Not applicable.
|
||||
|
||||
## 4. Data Access Patterns
|
||||
|
||||
| Query | Frequency | Hot Path | Index Needed |
|
||||
|-------|-----------|----------|--------------|
|
||||
| Tile pixel access (3 tiles per frame) | 3 Hz × 3 = 9 Hz | Yes | C6 mmap |
|
||||
|
||||
No additional caching beyond C6.
|
||||
|
||||
## 5. Implementation Details
|
||||
|
||||
**Algorithmic Complexity**: `O(N · F · F)` for the matching pass per backbone (N=3 candidates, F features per image), plus RANSAC `O(I · trials)` on inliers. Dominant cost is the backbone forward pass.
|
||||
|
||||
**State Management**: stateless per-frame. Holds the shared LightGlue / DISK runtime handle.
|
||||
|
||||
**Key Dependencies**:
|
||||
|
||||
| Library | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| DISK (Python) | upstream HEAD pinned per Plan-phase | Primary feature extractor (D-C3-1 = (a)) |
|
||||
| LightGlue (Python) | upstream HEAD pinned per Plan-phase | Primary matcher; replaces SuperPoint+SuperGlue (Magic Leap noncommercial) |
|
||||
| ALIKED (Python) | upstream HEAD pinned per Plan-phase | Secondary feature extractor |
|
||||
| XFeat (Python) | upstream HEAD pinned per Plan-phase | Alternate (lightweight) feature+matcher |
|
||||
| OpenCV | ≥ 4.12.0 | RANSAC + reprojection residual computation |
|
||||
| TensorRT | matches C7 | Backbone engine compilation/runtime |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
- `MatcherBackboneError`: backbone forward pass failed on a candidate. Candidate dropped from `per_candidate`; if all N=3 candidates fail, emit `InsufficientInliersError`.
|
||||
- `InsufficientInliersError`: RANSAC inlier count below configurable threshold on every candidate. C5 falls back to VIO-only with provenance label `visual_propagated`. F6 satellite re-localization may trigger if the condition persists.
|
||||
- Catastrophic backbone failure (engine unloadable): treated as `MatcherBackboneError` for every frame until F8 reboot recovery.
|
||||
|
||||
## 6. Extensions and Helpers
|
||||
|
||||
| Helper | Purpose | Used By |
|
||||
|--------|---------|---------|
|
||||
| `LightGlueRuntime` | shared LightGlue inference handle | C2.5, C3 |
|
||||
| `RansacFilter` | RANSAC + reprojection residual computation thin wrapper | C3, C3.5, C4 |
|
||||
|
||||
## 7. Caveats & Edge Cases
|
||||
|
||||
**Known limitations**:
|
||||
- The cross-domain gap (nav-camera vs satellite tile) is the hardest step in the pipeline. Backbone choice depends on the deployment camera's spectral and resolution characteristics; the current default (DISK+LightGlue) is locked per Mode B Fact #110 / D-C3-1 = (a) pending IT-12 verdict.
|
||||
- D-C2-12 (DINOv2-feature-based matcher) is a carryforward research item that may displace DISK in a future cycle.
|
||||
|
||||
**Potential race conditions**:
|
||||
- Shared LightGlue runtime with C2.5; serial access from one ingest thread.
|
||||
|
||||
**Performance bottlenecks**:
|
||||
- C3 dominates the F3 latency budget. The D-CROSS-LATENCY-1 hybrid does NOT change C3 (K=N=3 stays); it changes C4 covariance recovery.
|
||||
|
||||
## 8. Dependency Graph
|
||||
|
||||
**Must be implemented after**: C2.5 (input), C7 (inference runtime).
|
||||
|
||||
**Can be implemented in parallel with**: C1, C6 — independent paths.
|
||||
|
||||
**Blocks**: C3.5 / C4 / C5, F3 / F6.
|
||||
|
||||
## 9. Logging Strategy
|
||||
|
||||
| Log Level | When | Example |
|
||||
|-----------|------|---------|
|
||||
| ERROR | `InsufficientInliersError`; all N=3 candidates failed | `C3 zero-inlier on all candidates; frame=12345; backbone=disk_lightglue` |
|
||||
| WARN | reprojection residual above threshold (will trigger AdHoP at C3.5) | `C3 residual=4.2px > threshold=2.5px; frame=12345; will refine` |
|
||||
| INFO | Strategy ready | `C3 ready: matcher=disk_lightglue` |
|
||||
| DEBUG | per-candidate inlier + residual | `C3 frame=12345 candidates=[(412,1.1px),(287,1.4px),(198,2.0px)]` |
|
||||
|
||||
**Log format**: structured JSON.
|
||||
**Log storage**: stdout / journald / FDR via C13 (ERROR + WARN only).
|
||||
Reference in New Issue
Block a user