mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 07:01:14 +00:00
[autodev] Step 13 partial: c1/c2/c2_5/c3 cycle-1 doc sync
Item 2 (C1) + item 3 batch 1 of ~5 (C2 VPR, C2.5 Rerank, C3 Matcher) of the cycle-1 component-description reconciliation called out in ripple_log_cycle1.md. For each touched description.md: - Add a "Cycle-1 operational reality" paragraph in section 1 that names the _STRATEGY_REGISTRY + register_airborne_strategies() runtime gate (AZ-591), the pre_constructed dict path through compose_root (AZ-618 umbrella), the per-component AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS row, and any cycle-1 strategy-default vs documented-primary disambiguation (net_vlad as the C2 default; xfeat parked from the C3 airborne registry). - Relax the OpenCV row in section 5 Key Dependencies to the D-CROSS-CVE-1 cycle-1 pin (>=4.11.0.86,<4.12) wherever the component imports cv2 (C2 preprocessors, C2.5 ORB placeholder, C3 RANSAC + reprojection). - Add a "Cycle-1 Tier-2 follow-up dependencies" subsection in section 7 only for components with a strategy module that is built but parked from the airborne registry (C3 xfeat). Refresh ripple_log_cycle1.md follow-up ordering with per-batch progress + extracted batch pattern so the next batch session has a self-contained recipe. Bump _autodev_state.md sub_step.detail to reflect batch 1 completion (10 components + 8 helpers + tests/ remain). Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -4,7 +4,9 @@
|
||||
|
||||
**Purpose**: produce a per-frame relative pose SE(3) + 6×6 covariance + IMU bias estimate + feature-quality summary from the nav-camera frame and the FC IMU/attitude window, fusing visual and inertial cues without any external (satellite) reference.
|
||||
|
||||
**Architectural Pattern**: Strategy — `VioStrategy` interface with three concrete implementations (Okvis2 production-default, VinsMono research-only, KltRansac mandatory simple-baseline), constructor-injected at the composition root (ADR-009), build-time gated by per-implementation CMake `BUILD_*` flags (ADR-002), runtime selection by config at startup (ADR-001), not hot-swappable mid-flight.
|
||||
**Architectural Pattern**: Strategy — `VioStrategy` interface with three concrete implementations (Okvis2 nominal production-default, VinsMono research-only, KltRansac mandatory simple-baseline), constructor-injected at the composition root (ADR-009), build-time gated by per-implementation CMake `BUILD_*` flags (ADR-002), runtime selection by config at startup (ADR-001), not hot-swappable mid-flight.
|
||||
|
||||
**Cycle-1 operational reality**: the airborne binary ships with **`KltRansac` as the production-default selection** while the OKVIS2 + VINS-Mono native wirings are parked as Tier-2 follow-ups (`AZ-592` for AZ-332 OKVIS2; `AZ-593` for AZ-333 VINS-Mono — see `FINAL_report.md` § Cycle 1 Implementation Status). Both higher-fidelity strategies have their Python facade + pybind11 binding skeleton + `_STRATEGY_REGISTRY` registration in place; the first `process_frame` call into the OKVIS2/VINS-Mono native side raises until the upstream wiring (`okvis::ThreadedSlam` / VINS-Mono ROS-strip) lands. ADR-001 / ADR-002 remain correct — the seam exists, the build-flag gating works — only the operational default-selection shifts. Runtime selection of `okvis2` or `vins_mono` via config currently raises `StrategyNotAvailableError` from `runtime_root/vio_factory.py` until their `BUILD_*` flag is ON.
|
||||
|
||||
**Upstream dependencies**:
|
||||
- Camera ingest thread → `NavCameraFrame` (3 Hz nominal, drop-oldest queue).
|
||||
@@ -85,7 +87,7 @@ No database access, no cache layer beyond the in-process keyframe window.
|
||||
|---------|---------|---------|
|
||||
| OKVIS2 (C++) | upstream HEAD pinned per Plan-phase | Production-default tightly-coupled VIO; BSD-3-Clause |
|
||||
| VINS-Mono (C++) | upstream HEAD pinned per Plan-phase | Research-only loosely-coupled VIO for IT-12 comparative study; behind `BUILD_VINS_MONO` |
|
||||
| OpenCV | ≥ 4.12.0 (CVE-2025-53644 mitigation) | KLT pyramidal optical flow + RANSAC for the simple-baseline strategy |
|
||||
| OpenCV | `>=4.11.0.86,<4.12` (cycle-1 relaxed pin; D-CROSS-CVE-1 deferred — see `_docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md`) | KLT pyramidal optical flow + RANSAC for the KltRansac strategy (cycle-1 production-default) |
|
||||
| Eigen | matches OKVIS2 / GTSAM pin | Lie-algebra math for SE(3) + 6×6 covariance |
|
||||
| pybind11 | matches OKVIS2 / VINS-Mono build | Python bindings for the C++ strategies |
|
||||
|
||||
@@ -112,7 +114,11 @@ No database access, no cache layer beyond the in-process keyframe window.
|
||||
- The camera ingest thread is the sole producer; C5 is the sole consumer. Concurrent calls to `process_frame` on a single strategy instance are forbidden — enforce in the composition root by binding one strategy instance to the camera ingest thread.
|
||||
|
||||
**Performance bottlenecks**:
|
||||
- Okvis2 sliding-window optimisation can spike to 80–120 ms on a thermally-throttled Jetson; D-CROSS-LATENCY-1 hybrid auto-degrades C4 covariance recovery (not C1) to free budget.
|
||||
- Okvis2 sliding-window optimisation can spike to 80–120 ms on a thermally-throttled Jetson; D-CROSS-LATENCY-1 hybrid auto-degrades C4 covariance recovery (not C1) to free budget. (Behaviour is documented for when AZ-592 wires the OKVIS2 native side; in cycle-1, KltRansac is the active backend and its per-frame cost is `O(F)` only.)
|
||||
|
||||
**Cycle-1 Tier-2 follow-up dependencies**:
|
||||
- AZ-592 (parked from AZ-332): wires `okvis::ThreadedSlam` into `_native/okvis2_binding.cpp` and lands the OKVIS2 CI matrix (Ceres + vendored submodules) + Tier-2 Jetson validation against Derkachi-class fixtures. Until this lands, requesting `config.vio.strategy="okvis2"` raises `StrategyNotAvailableError` regardless of `BUILD_OKVIS2`.
|
||||
- AZ-593 (parked from AZ-333): finalises the de-ROSified VINS-Mono upstream pin (HKUST + in-tree ROS-strip vs. community fork) and wires the `vins_estimator::Estimator` into `_native/vins_mono_binding.cpp`. Until this lands, requesting `config.vio.strategy="vins_mono"` raises `StrategyNotAvailableError` regardless of `BUILD_VINS_MONO`.
|
||||
|
||||
## 8. Dependency Graph
|
||||
|
||||
|
||||
@@ -6,6 +6,8 @@
|
||||
|
||||
**Architectural Pattern**: Strategy — `VprStrategy` interface; concrete implementations (UltraVPR primary, MegaLoc secondary, MixVPR / SelaVPR / EigenPlaces / NetVLAD / SALAD additional candidates) selected at startup by config (ADR-001); build-time gated per-implementation by `BUILD_*` flags (ADR-002); composition-root wired (ADR-009).
|
||||
|
||||
**Cycle-1 operational reality**: the airborne binary wires C2 through the `_STRATEGY_REGISTRY` + `register_airborne_strategies()` runtime gate (AZ-591) on top of the build-flag matrix, and constructor injection flows through the `pre_constructed` dict passed to `compose_root(config, pre_constructed=...)` (AZ-618 umbrella → AZ-620 c6 storage phase + AZ-623 c7 inference phase). All seven backbones (`ultra_vpr`, `net_vlad`, `mega_loc`, `mix_vpr`, `sela_vpr`, `eigen_places`, `salad`) have wired strategy modules + `_preprocessor_*` siblings + `_faiss_bridge`; their `BUILD_VPR_<variant>` env flags default OFF (tests/CI must opt in per strategy — see `runtime_root/vpr_factory.py::_is_build_flag_on`). The cycle-1 `C2VprConfig.strategy` default is `net_vlad` (the mandatory simple-baseline per Plan-phase D-C2-1) — `ultra_vpr` remains the Documentary Lead's PRIMARY backbone but additionally requires a pre-compiled `.trt` engine produced by C10's engine compiler (AZ-321). The `c2_vpr` slot lists `("c6_descriptor_index", "c7_inference")` in `AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS`; missing keys raise `AirborneBootstrapError` at composition time, not at first frame.
|
||||
|
||||
**Upstream dependencies**:
|
||||
- Camera ingest thread → `NavCameraFrame` (parallel fan-out with C1; same frame, distinct queue depth).
|
||||
- C7 InferenceRuntime → backbone forward pass (TRT/ONNX/PyTorch per active runtime).
|
||||
@@ -92,6 +94,7 @@ C2 is read-only against C6 during F3/F4/F6. Pre-flight, F1 triggers C10 (after C
|
||||
| PyTorch | matches simple-baseline track | FP16 baseline (NetVLAD / MixVPR mandatory) |
|
||||
| UltraVPR (research code drop) | upstream HEAD pinned per Plan-phase | Documentary Lead PRIMARY backbone |
|
||||
| MegaLoc, MixVPR, SelaVPR, EigenPlaces, NetVLAD | upstream HEAD pinned per Plan-phase | Secondary + mandatory simple-baselines |
|
||||
| OpenCV (`cv2`) | `>=4.11.0.86,<4.12` (cycle-1 relaxed pin; D-CROSS-CVE-1 deferred — see `_docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md`) | Image decode + colour-space conversions in the per-strategy `_preprocessor_*.py` modules |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
- `VprBackboneError`: backbone forward pass failed (CUDA OOM, TRT engine deserialize mismatch). C2 emits no `VprResult`; C5 falls back to VIO-only with provenance label `visual_propagated` (AC-1.4).
|
||||
|
||||
@@ -6,6 +6,8 @@
|
||||
|
||||
**Architectural Pattern**: Strategy (single concrete implementation today: `InlierCountReRanker`, AZ-343). Future re-rank algorithms can be added as additional `ReRankStrategy` implementations behind the same interface.
|
||||
|
||||
**Cycle-1 operational reality**: the airborne binary registers `c2_5_rerank` into `_STRATEGY_REGISTRY` via `register_airborne_strategies()` (AZ-591) with a single strategy slot (`inlier_count`); the composition root passes infrastructure deps through the `pre_constructed` dict (AZ-618 → AZ-620 c6 phase + AZ-621 c3 helpers phase). The `c2_5_rerank` slot lists `("c6_tile_store", "c3_lightglue_runtime", "c3_feature_extractor", "clock")` in `AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS`; `c13_fdr` is optional. Missing keys raise `AirborneBootstrapError` at composition time, naming the consumer and missing key. The `FeatureExtractor` placeholder remains `OpenCvOrbExtractor` in cycle-1; the TRT-backed DISK/ALIKED swap is parked as a Tier-2 follow-up (carried as the `FeatureExtractor` row in § 6 below).
|
||||
|
||||
**Upstream dependencies**:
|
||||
- C2 → `VprResult` (top-K=10 candidates).
|
||||
- Shared `LightGlueRuntime` helper (used in single-pair mode for inlier counting; the same matcher object is shared with C3 — owned by the helper, not by C3, so neither component depends on the other at build time).
|
||||
@@ -68,6 +70,7 @@ No caching layer beyond C6's mmap. The same tile may be fetched repeatedly acros
|
||||
|---------|---------|---------|
|
||||
| LightGlue (Python) | upstream HEAD pinned per Plan-phase | Single-pair matching for inlier count |
|
||||
| TensorRT | matches C7 | LightGlue inference engine reuse |
|
||||
| OpenCV (`cv2`) | `>=4.11.0.86,<4.12` (cycle-1 relaxed pin; D-CROSS-CVE-1 deferred — see `_docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md`) | `OpenCvOrbExtractor` placeholder feature extraction (will be swapped to a TRT-backed DISK/ALIKED extractor in a follow-up) |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
- `RerankBackboneError`: LightGlue forward pass failed on one or more candidates. The candidate is dropped from the rerank set; if fewer than N=3 candidates survive, C2.5 returns whatever it has and C3 proceeds with reduced N.
|
||||
|
||||
@@ -6,6 +6,8 @@
|
||||
|
||||
**Architectural Pattern**: Strategy — `CrossDomainMatcher` interface, with concrete implementations DISK+LightGlue (D-C3-1 = (a) primary), ALIKED+LightGlue (secondary), XFeat (alternate). Selection at startup by config (ADR-001); build-time gating by `BUILD_*` flags (ADR-002); composition-root wired (ADR-009).
|
||||
|
||||
**Cycle-1 operational reality**: the airborne binary wires C3 through `_STRATEGY_REGISTRY` + `register_airborne_strategies()` (AZ-591) on top of the `BUILD_MATCHER_*` build-flag matrix (`disk_lightglue` / `aliked_lightglue` / `xfeat` — see `runtime_root/airborne_bootstrap.py::C3_MATCHER_BUILD_FLAGS`). The `c3_matcher` airborne slot registers **only `disk_lightglue` + `aliked_lightglue`** — `xfeat` has its build-flag wired but is parked as a Tier-2 follow-up (no airborne registry slot, no airborne `_C3_MATCHER_STRATEGIES` entry). Constructor injection flows through the `pre_constructed` dict passed to `compose_root(config, pre_constructed=...)` (AZ-618 umbrella → AZ-621 c3 helpers phase + AZ-622 LightGlue runtime builder + AZ-623 c7 inference phase). The `c3_matcher` slot lists `("c3_lightglue_runtime", "c282_ransac_filter", "c7_inference")` in `AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS`; `clock` and `c13_fdr` are optional. Missing required keys raise `AirborneBootstrapError` at composition time, naming the consumer and missing key.
|
||||
|
||||
**Upstream dependencies**:
|
||||
- C2.5 → `RerankResult` (top-N=3 candidates).
|
||||
- C7 InferenceRuntime → backbone forward pass.
|
||||
@@ -79,7 +81,7 @@ No additional caching beyond C6.
|
||||
| LightGlue (Python) | upstream HEAD pinned per Plan-phase | Primary matcher; replaces SuperPoint+SuperGlue (Magic Leap noncommercial) |
|
||||
| ALIKED (Python) | upstream HEAD pinned per Plan-phase | Secondary feature extractor |
|
||||
| XFeat (Python) | upstream HEAD pinned per Plan-phase | Alternate (lightweight) feature+matcher |
|
||||
| OpenCV | ≥ 4.12.0 | RANSAC + reprojection residual computation |
|
||||
| OpenCV | `>=4.11.0.86,<4.12` (cycle-1 relaxed pin; D-CROSS-CVE-1 deferred — see `_docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md`) | RANSAC + reprojection residual computation (consumed via the shared `RansacFilter` helper) |
|
||||
| TensorRT | matches C7 | Backbone engine compilation/runtime |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
@@ -100,6 +102,9 @@ No additional caching beyond C6.
|
||||
- The cross-domain gap (nav-camera vs satellite tile) is the hardest step in the pipeline. Backbone choice depends on the deployment camera's spectral and resolution characteristics; the current default (DISK+LightGlue) is locked per Mode B Fact #110 / D-C3-1 = (a) pending IT-12 verdict.
|
||||
- D-C2-12 (DINOv2-feature-based matcher) is a carryforward research item that may displace DISK in a future cycle.
|
||||
|
||||
**Cycle-1 Tier-2 follow-up dependencies**:
|
||||
- XFeat — build-flag (`BUILD_MATCHER_XFEAT`) + `matcher_factory._STRATEGY_TO_BUILD_FLAG` + concrete `c3_matcher/xfeat.py` module are all in place, but `c3_matcher`'s `_C3_MATCHER_STRATEGIES` tuple in `runtime_root/airborne_bootstrap.py` registers only `disk_lightglue` + `aliked_lightglue`. Selecting `xfeat` via airborne config currently raises `StrategyNotLinkedError` from the `_STRATEGY_REGISTRY` lookup. Tier-2 follow-up: extend the airborne registration tuple + airborne Jetson validation against Derkachi-class fixtures.
|
||||
|
||||
**Potential race conditions**:
|
||||
- Shared LightGlue runtime with C2.5; serial access from one ingest thread.
|
||||
|
||||
|
||||
@@ -36,9 +36,12 @@ The Task workflow's three update levels each have a different effective scope th
|
||||
|
||||
## Follow-up session — recommended ordering
|
||||
|
||||
1. **Architecture-level updates (continue Task Step 3)** — `architecture.md` ADR-009 sibling notes for `_STRATEGY_REGISTRY` + `pre_constructed`; OpenCV mentions in § Technology stack / § Risks; KltRansac-as-production-default note in § C1. `module-layout.md` § "Composition Root" extension for `airborne_bootstrap` + `build_pre_constructed`. `system-flows.md` only if any flow's wiring narrative drifted (likely F2 takeoff + F8 reboot — the only flows where `compose_root` is the visible actor).
|
||||
2. **C1 component refresh** — `components/01_c1_vio/description.md` should explicitly mark KltRansac as the cycle-1 production-default; reflect the Tier-2 status of OKVIS2 + VINS-Mono.
|
||||
1. **Architecture-level updates (continue Task Step 3)** — `architecture.md` ADR-009 sibling notes for `_STRATEGY_REGISTRY` + `pre_constructed`; OpenCV mentions in § Technology stack / § Risks; KltRansac-as-production-default note in § C1. `module-layout.md` § "Composition Root" extension for `airborne_bootstrap` + `build_pre_constructed`. `system-flows.md` only if any flow's wiring narrative drifted (likely F2 takeoff + F8 reboot — the only flows where `compose_root` is the visible actor). **DONE** — commits `4f122b6` (FINAL_report + glossary) + `1ca9a59` (arch + module-layout).
|
||||
2. **C1 component refresh** — `components/01_c1_vio/description.md` should explicitly mark KltRansac as the cycle-1 production-default; reflect the Tier-2 status of OKVIS2 + VINS-Mono. **DONE** — uncommitted in working tree at handoff time; rolled into the cycle-1 doc-sync batch with C2/C2.5/C3.
|
||||
3. **Other 13 components + 8 helpers** — task-by-task reconciliation, batched ~3 components per session.
|
||||
- **Batch 1 (this session, 2026-05-19)**: C2 (VPR), C2.5 (Rerank), C3 (Matcher) — cycle-1 reality paragraphs + OpenCV pin relaxation (where applicable) + C3 xfeat Tier-2 follow-up note. Source of truth crossed: `runtime_root/airborne_bootstrap.py` (`_C*_STRATEGIES`, `AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS`, `C3_MATCHER_BUILD_FLAGS`), `runtime_root/vpr_factory.py`, per-component `config.py`.
|
||||
- **Remaining (10 components)**: C3.5 (AdHoP), C4 (Pose), C5 (StateEstimator), C6 (TileCache), C7 (Inference), C8 (FC adapter), C10 (Provisioning), C11 (TileManager), C12 (OperatorOrchestrator), C13 (FDR).
|
||||
- **Helpers (8 files)**: `imu_preintegrator`, `se3_utils`, `lightglue_runtime`, `wgs_converter`, `sha256_sidecar`, `engine_filename_schema`, `ransac_filter`, `descriptor_normaliser`.
|
||||
4. **`tests/*.md`** — pick up cycle-1 deltas; Step 12 (Test-Spec Sync) already touched `traceability-matrix.md` and `resilience-tests.md` in the uncommitted working tree; the remaining test-doc surfaces (`blackbox-tests.md`, `performance-tests.md`, `resource-limit-tests.md`, `security-tests.md`, `tier2-jetson-testing.md`, `environment.md`, `test-data.md`) should be checked against the ~36 done Blackbox Tests task specs.
|
||||
|
||||
The component-level pass (item 2 + 3) is the bulk of the work. Each component-batch session should re-read this ripple log + the relevant component description.md + the task specs in `_docs/02_tasks/done/AZ-*_<component>*.md` + the actual source in `src/gps_denied_onboard/components/<c>/`.
|
||||
The component-level pass (item 2 + 3) is the bulk of the work. Each component-batch session should re-read this ripple log + the relevant component description.md + the task specs in `_docs/02_tasks/done/AZ-*_<component>*.md` + the actual source in `src/gps_denied_onboard/components/<c>/`. Per-batch session pattern proven this cycle: read `runtime_root/airborne_bootstrap.py` `_C*_STRATEGIES` + `AIRBORNE_REQUIRED_PRE_CONSTRUCTED_KEYS` rows for each slug, cross-check against component `config.py` defaults, then add (a) a "Cycle-1 operational reality" paragraph in § 1 of `description.md`, (b) an OpenCV pin row in § 5 if the component imports `cv2`, (c) a "Cycle-1 Tier-2 follow-up dependencies" subsection in § 7 only when a strategy exists in code but is parked from the airborne registry.
|
||||
|
||||
@@ -8,7 +8,7 @@ status: in_progress
|
||||
sub_step:
|
||||
phase: 6
|
||||
name: component-doc-updates
|
||||
detail: "item-1 done (architecture.md + module-layout.md; system-flows.md no-op); items 2-4 deferred (C1 description, 13 components, 8 helpers, tests/)"
|
||||
detail: "items 1-2 done; item 3 batch 1/~5 done (c2/c2_5/c3); 10 components + 8 helpers + tests/ remain"
|
||||
retry_count: 0
|
||||
cycle: 1
|
||||
tracker: jira
|
||||
|
||||
Reference in New Issue
Block a user