Refactor documentation for splittable artifacts and update references

Updated various documentation files to clarify the handling of splittable artifacts, allowing for folder equivalents of key markdown files when they exceed size limits. Adjusted references in multiple sections to reflect this new structure, ensuring consistency across the research methodology. Enhanced clarity on the saving actions and artifact organization, particularly for `01_source_registry.md`, `02_fact_cards.md`, and `06_component_fit_matrix.md`. This change aims to improve usability and maintainability of the research documentation.
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-08 23:39:30 +03:00
parent e0a6f0d9d5
commit 846670a5c5
56 changed files with 6686 additions and 1244 deletions
@@ -0,0 +1,50 @@
# Fact Cards — Index & Summary
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Extracted from sources logged in `../01_source_registry/` (see `../01_source_registry/00_summary.md` for index). Confidence labels: ✅ High (L1 / verified source code), ⚠️ Medium (L1/L2 with caveat), ❓ Low (L3/L4 inferential).
>
> Bound to sub-questions in `../00_question_decomposition.md`. Many SQ6 facts also bind directly to the Project Constraint Matrix (`../../00_problem/acceptance_criteria.md` / `../../00_problem/restrictions.md`); per the engine's "Per-Mode API Capability Verification" rule, MAVLink/MSP messages are treated as candidate **modes** and are bound `Pass/Fail/Verify/N/A` against numbered ACs and restrictions.
This folder replaces the previous monolithic `02_fact_cards.md` (1480 lines, too large to navigate). Each category lives in its own file. Open the file matching the area you need — every fact and conclusion is preserved verbatim.
---
## Category index
| File | Sub-question / Component | Facts (count) | Scope summary |
| --- | --- | --- | --- |
| [`SQ6_fc_external_positioning.md`](SQ6_fc_external_positioning.md) | **SQ6** — ArduPilot Plane vs iNav external positioning | #1#10 (10 facts) | MAVLink `GPS_INPUT` (232) ingestion in EKF3, iNav MSP `MSP2_SENSOR_GPS` ingestion via INAV BlackBox, covariance honesty, lane-fusion / lane-switch on (NSats, HDOP, fix_type), spoof-promotion via UBX emulation, dead-reckoning behaviour, `EK3_GPS_CHECK` bit-mask gates. Working conclusions: ArduPilot is the cooperative path, iNav requires UBX impersonation. |
| [`SQ1_existing_systems.md`](SQ1_existing_systems.md) | **SQ1** — Existing / competitor GPS-denied UAV navigation systems | #11#20 (10 facts) | Twist Robotics OSCAR (Ukrainian peer), Auterion Artemis OS, Vantor Raptor, NGPS class systems, SPRIN-D winner, RTAB-Map / ORB-SLAM3 pruning rationale, DSMAC/TERCOM lineage, hierarchical retrieval-matching SOTA, AerialExtreMatch benchmark, DARPA FLA + USAF SBIR programs. Working conclusions: VPR-anchored hybrid pipeline is canonical. |
| [`SQ2_canonical_pipeline.md`](SQ2_canonical_pipeline.md) | **SQ2** — Canonical GPS-denied pipeline & SOTA components | #21#27 (7 facts) | Two-stage canonical pipeline (global VPR → local alignment → PnP-RANSAC → EKF), end-to-end visual-localization rejection (poor generalization, no covariance), cross-domain sat ↔ UAV registration, hardware MVE doctrine, Top-N inlier re-rank gate. Working conclusions: VIO + VPR + Matcher + PnP + EKF is the design floor. |
| [`C1_vio.md`](C1_vio.md) | **C1** — Visual / Visual-Inertial Odometry | Candidate enumeration + decisions | VINS-Mono (BSD/permissive baseline), VINS-Fusion (GPL-3.0 alternate), OpenVINS (GPL-3.0), OKVIS2 (BSD), Kimera-VIO (BSD), DROID-SLAM (BSD non-VIO), DPVO (Apache-2.0 non-VIO), KLT+RANSAC (homemade fallback). Decisions: D-C1-1 license posture, D-C1-2 IMU rate. |
| [`C2_vpr.md`](C2_vpr.md) | **C2** — Visual Place Recognition | Candidate enumeration + decisions | MixVPR, SALAD (GPL-3.0 disqualifier), SelaVPR, NetVLAD, EigenPlaces, AnyLoc, BoQ, DINOv2-VLAD. Decisions: D-C2-1 aerial-domain training, D-C2-2 cache budget, D-C2-3 input resolution shape, D-C2-N TensorRT export gate. |
| [`C3_matchers.md`](C3_matchers.md) | **C3** — Cross-domain registration (Matchers) | Candidate enumeration + decisions | SP+LightGlue (Magic Leap noncommercial disqualifier on canonical SP), DISK+LightGlue (RECOMMENDED-PRIMARY-MITIGATION), ALIKED+LightGlue, XFeat (alternate-modern lead), SuperGlue+SuperPoint (deprecated by LightGlue authors), MASt3R (CC-BY-NC), RoMa, DKM, LoFTR. Decisions: D-C3-1 modern-competitive lead, D-C3-2 ONNX/TensorRT export path, D-C3-6 re-rank strategy. |
| [`C4_pose_estimation.md`](C4_pose_estimation.md) | **C4** — Pose estimation (PnP + RANSAC + LM) | #52#54 (3 facts, in progress) | OpenCV `cv::solvePnPRansac` mandatory simple-baseline (Apache-2.0 throughout, 9 SolvePnPMethod enum values with 2 BROKEN, paired `solvePnPRefineLM`/`solvePnPRefineVVS`/`solvePnPGeneric`, 7 USAC RANSAC variants); OpenGV modern-competitive-lead-richer-minimal-solver (BSD-3-Clause-equivalent NOASSERTION-SPDX-detector contingent + ~3-year stale + 4 algorithm-selectable RANSAC enums [KNEIP/GAO/EPNP/GP3P] + 2 P3P variants + UPnP global-optimal + GP3P generalized-camera; NO planar-scene dedicated solver vs OpenCV's IPPE); GTSAM modern-competitive-lead-covariance-honest (BSD-3-Clause clean throughout, daily-active maintenance, **NATIVE 6×6 pose covariance via `Marginals.marginalCovariance` — only C4 candidate to satisfy AC-NEW-4 NATIVELY**, no native RANSAC, ~50-200 MB footprint, tight AC-4.1 latency margin). Decisions: D-C4-1 (carry-forward) 2D-3D-lift; D-C4-2 (NEW + UPDATED) covariance-recovery-strategy; D-C4-3 (NEW) OpenGV license-clearance-verification; D-C4-4 (NEW) OpenGV maintenance-staleness-mitigation. Subsequent candidates pending: Theia / Ceres-only (likely deferrable — D-C4 row may already have sufficient coverage). |
| [`C5_state_estimator.md`](C5_state_estimator.md) | **C5** — State estimator / sensor fusion | #88#89 (2 facts, **batch 1 closed at 2/N 2026-05-08**) | Manual ESKF reference (Solà 2017 canonical aerial/quaternion arXiv preprint — public-domain canonical equations + project-side custom implementation under project's Apache-2.0; mandatory simple-baseline; trivial dependency footprint at ~kilobytes of NumPy/SciPy code; native 6×6 covariance via analytic Jacobian propagation per Solà §6 canonical recipe; quaternion-correct attitude integration on SO(3) via small-angle approximation in error-state; **fastest C5 candidate by an order of magnitude** at ~5-15 ms per update on Jetson CPU); GTSAM `iSAM2` + `CombinedImuFactor` (Forster et al. RSS 2015) + `PreintegratedCombinedMeasurements` + `BetweenFactorPose3` + `GenericProjectionFactorCal3DS2` + `PriorFactorPose3` + smart projection factors + `Marginals.marginalCovariance` + `gtsam_unstable.IncrementalFixedLagSmoother` modern-competitive-lead-factor-graph (clean BSD-3-Clause throughout, daily-active maintenance with last-pushed 2026-05-08T13:00:22Z = TODAY at access time, **architecturally couples with C4 Fact #54 via shared GTSAM substrate**, native 6×6 posterior covariance via `Marginals` — same NATIVE AC-NEW-4 satisfaction pathway as C4 Fact #54, IMU pre-integration via Forster et al. RSS 2015 `CombinedImuFactor` 6-key per-keyframe-pair factor with bias evolution for asynchronous IMU+camera fusion at ~100-200 Hz IMU + 3 Hz camera, ~50-200 MB footprint, incremental smoothing via iSAM2 amortizes per-frame cost, **NATIVE AC-4.5 look-back refinement** unique among C5 candidates). Decisions: D-C5-1 (NEW) reference-implementation-license-verification; D-C5-2 (NEW) long-cruise-observability-strategy; D-C5-3 (NEW) sliding-window-primitive-choice; D-C5-4 (NEW) IMU-gap-handling-strategy; D-C5-5 (NEW) factor-density-choice (recommended D-C5-5 = (c) couples C4 Fact #54 D-C4-2 = (b) with C5 Fact #89 architectural integration via shared GTSAM substrate). |
| [`C6_tile_cache_spatial_index.md`](C6_tile_cache_spatial_index.md) | **C6** — Tile cache + spatial index | #92#93 (2 facts, **batch 1 closed at 2/N 2026-05-08**) | **Cand 1 RECOMMENDED PRIMARY**: Manual mirror of existing parent-suite `satellite-provider` pattern (verified directly via Source #92 filesystem read at /Users/obezdienie001/dev/azaion/suite/satellite-provider/) — PostgreSQL btree composite on slippy-map `(tile_zoom, tile_x, tile_y, version)` for geographic spatial-grid range queries + `bytea` descriptor blobs + app-side FAISS `IndexHNSWFlat(d, M=32)` loaded at takeoff via `faiss.read_index` for descriptor ANN + filesystem tile storage at `./tiles/{zoom}/{x}/{y}.{image_type}` slippy-map convention; clean PostgreSQL License + MIT + LGPL/MIT-Apache; trivial dependency footprint (no Postgres extensions); empirically-confirmed Postgres-on-Jetson viability per Source #97 March 2026 article (CPU cores limiting, NOT memory); ~6-54 ms per cache hit comfortably within AC-4.1 400 ms p95 budget; ~700 MB-1.5 GB total memory footprint within AC-4.2 8 GB budget. **Cand 2 DEFERRED secondary**: PostgreSQL + PostGIS 3.4 GiST on `geography(POINT,4326)` with KNN distance ordering (`<->`) + pgvector 0.7+ HNSW for descriptor ANN + same filesystem tile storage; native KNN + radius + combined-SQL capabilities are real improvements BUT 5-10× slower geographic lookup than Cand 1 + heavier dependency (~50-100 MB additional memory + ~50-200 MB additional disk install) + PostGIS GPL-2.0-or-later license-complexity (CONTINGENT REJECT under D-C1-1 = (b) BSD/permissive-only-track) + DIVERGENT from suite pattern + improvements marginal-to-negative in project's pinned 3 Hz spatial-grid query operating context. **Comparative-improvement-vs-Cand-1 verdict**: per user's session-start "significant-improvement-only" bar, no material justification to deviate from existing satellite-provider pattern. Decisions: D-C6-1 (NEW) descriptor-storage-format choice (halfvec recommended); D-C6-2 (NEW Cand-1-only) FAISS index variant choice (IndexHNSWFlat M=32 recommended); D-C6-3 (NEW Cand-1-only CROSS-COMPONENT with C10) descriptor-cache-rebuild-trigger strategy (periodic-during-C10-pre-flight recommended); D-C6-4 (NEW Cand-1-only) geographic-spatial-grid radius (dynamic recommended); D-C6-5 (NEW Cand-2-only contingent) Jetson PostGIS+pgvector co-installation Plan-phase verification (verify-on-Jetson-MVE recommended); D-C6-6 (NEW Cand-2-only contingent) pgvector descriptor-storage-type choice (halfvec recommended); D-C6-7 (NEW CROSS-COMPONENT affects parent-suite satellite-provider) cascade-changes-back-to-suite strategy (leave-unchanged recommended given Cand 1 closure verdict). |
| [`C7_inference_runtime.md`](C7_inference_runtime.md) | **C7** — On-Jetson inference runtime | #94#96 (3 facts, **batch 1 closed at 3/N 2026-05-08**) | **Cand 1 RECOMMENDED PRIMARY**: TensorRT native — JetPack 6.2 bundled TensorRT 10.3 + `IInt8EntropyCalibrator2` + `BuilderFlag.FP16+INT8` mixed-precision + engines built directly on Jetson Orin Nano Super SM 87 (Apache-2.0 in TensorRT 10.x; ships with JetPack so zero-effort install; lowest-latency primary path; 2-3× speedup at INT8 vs FP16 per Source #102 YOLO26 benchmark; engines tied to SM 87 hardware-specific per Source #105 — must be built on deployed Jetson via D-C7-7); **Cand 2 modern-competitive-lead-cross-architecture-portability**: ONNX Runtime + TensorRT EP — `onnxruntime-gpu` via Jetson AI Lab JP6/CU126 wheel index + `TensorrtExecutionProvider` config + automatic CUDA EP / CPU EP subgraph fallback (MIT throughout; cross-architecture portability for replay/SITL on x86 dev hosts; `pip install onnxruntime-gpu` does NOT work on Jetson — needs Jetson AI Lab community wheel via D-C7-3 + numpy<2.0.0 pin via D-C7-4); **Cand 3 mandatory simple-baseline**: pure PyTorch FP16 — `torch.amp.autocast` + `model.half()` + Jetson AI Lab PyTorch 2.5 ARM64 wheel (BSD-3-Clause throughout; zero-conversion regression baseline; reference-correctness oracle for accuracy validation of TRT-built engines; standard `pip install torch` lacks CUDA on Jetson — needs Jetson AI Lab wheel via D-C7-5). **Cross-cutting precision policy** (D-C7-6 NEW CROSS-COMPONENT, affects C2+C3+C1+C7): VPR backbones (CNN-class MixVPR/EigenPlaces/NetVLAD) → INT8+FP16 mixed; ViT-class VPR (SelaVPR DINOv2-L; conditional AnyLoc/BoQ/DINOv2-VLAD) → FP16-only initially, INT8 deferred to Jetson MVE per D-C2-5; matchers (LightGlue with SP/DISK/ALIKED, XFeat, XFeat+LighterGlue) → **FP16-only — NO INT8** per Source #103 quantization-sensitivity finding (LightGlue FP8 ModelOpt collapsed match counts); learned VIO frontends → FP16-only initially. **Triton/DeepStream/CUDA-Python custom kernels considered-and-rejected** (server/video-pipeline class + out-of-budget for embedded 8 h mission) per c7_overkill_options scope choice. Decisions: D-C7-1 (NEW Cand-1-only CROSS-COMPONENT with C9) calibration-dataset-strategy (AerialVL S03 + AerialExtreMatch recommended); D-C7-2 (NEW Cand-1-only) TensorRT mixed-precision flag matrix (per-family policy per D-C7-6 recommended); D-C7-3 (NEW Cand-2-only) ORT-Jetson-wheel-index-pin (mirror to project artifact registry + cu126 recommended); D-C7-4 (NEW Cand-2-only) numpy-version-pin (`numpy<2.0.0` recommended); D-C7-5 (NEW Cand-3-only) PyTorch-Jetson-wheel-pin (PyTorch 2.5 + torchvision 0.20 recommended); D-C7-6 (NEW CROSS-COMPONENT C2+C3+C1+C7) INT8-vs-FP16-per-model-family-precision-policy (per-family policy recommended); D-C7-7 (NEW Cand-1-only CROSS-COMPONENT with C10) engine-build-on-Jetson-vs-prebuilt strategy (primary build-on-target + reference-Jetson fallback recommended); D-C7-8 (NEW Cand-1-only) `config.max_workspace_size` cap (1 GB safe default recommended); D-C7-9 (NEW Cand-1-only) TensorRT version pin within JetPack lifecycle (JetPack 6.2 + TensorRT 10.3 recommended). |
| [`C10_preflight_provisioning.md`](C10_preflight_provisioning.md) | **C10** — Pre-flight cache provisioning (CROSS-COUPLING MINIMAL scope per 2026-05-08 user choice C; only D-C6-3 + D-C7-7 confirmation pipelines researched here, operator tooling design deferred to Plan-phase) | #100#101 (2 facts, **batch 1 closed at 2/N 2026-05-08**) | **D-C6-3 confirmation (Fact #100)**: descriptor-cache rebuild trigger + atomic-write strategy via direct `faiss.write_index`/`faiss.read_index` Python API + `python-atomicwrites` (write-temp + `fsync` + atomic rename) + content-hash (SHA-256) verification gate at takeoff load + `IO_FLAG_MMAP_IFC` mmap load with `madvise(MADV_WILLNEED)` pre-fault + manifest-hash-driven rebuild trigger; FAISS MIT + atomicwrites MIT throughout; FAISS warns "no internal integrity check, expects validated input" — MITIGATED by content-hash gate at takeoff (binds AC-NEW-7 cache-poisoning safety); rebuild-while-not-flying constraint per restrictions.md. **D-C7-7 confirmation (Fact #101)**: hybrid TensorRT engine-build orchestration — Polygraphy CLI primary for INT8-calibrating builds (`polygraphy convert --int8 --calib-cache=<path> ...` Apache-2.0 + Calibrator API replaces hand-written `IInt8EntropyCalibrator2`) + `trtexec` for fast cache-reuse rebuilds (`--fp16 --int8 --calib=<existing_cache>`) + direct `IBuilderConfig` Python API as escape hatch for unusual models (LightGlue dynamic-shape profiles); calibration cache binary-blob reuse keyed by `SHA-256(calib_corpus)` per D-C10-6; engines tied to SM 87 hardware-specific per Source #105 → must be built on deployed Jetson per D-C7-7 closure (D-C10-8 reference-Jetson-at-HQ + deployed-Jetson-copy-to-archive prebuilt-fallback venue); self-describing filename schema `<model>_sm<SM>_jp<JP>_trt<TRT>_<precision>.engine` per D-C10-7; binds AC-4.1/4.2 latency+memory budgets via D-C7-2 mixed-precision flag matrix + D-C7-1 calibration corpus closure. |
| [`C8_fc_adapter.md`](C8_fc_adapter.md) | **C8** — MAVLink / MSP2 FC adapter | #97#99 (3 facts, **batch 1 closed at 3/N 2026-05-08**) | **Cand 1 RECOMMENDED PRIMARY for ArduPilot**: pymavlink → MAVLink `GPS_INPUT` (msg 232) cooperative-path; `master.mav.gps_input_send(time_usec, gps_id, ignore_flags, time_week_ms, time_week, fix_type, lat, lon, alt, hdop, vdop, vn, ve, vd, speed_accuracy, horiz_accuracy, vert_accuracy, satellites_visible, yaw)` periodic injection at 5 Hz over MAVLink (UART/USB/UDP per D-C8-1); FC-side `GPS1_TYPE=14` MAVLink + `EK3_SRC1_POSXY=3` GPS source-set drives EKF3 ingestion via `AP_GPS_MAV` (verified Source #4 SQ6 + Source #106 + Source #107); pymavlink LGPL-3.0 linkable from Apache-2.0 app per LGPL §6 (D-C8-3 mitigation). **Cand 2 RECOMMENDED PRIMARY for iNav**: `MSP2_SENSOR_GPS` (id 7939 / 0x1F03) via Python MSP V2 (YAMSPy or INAV-Toolkit `msp_v2_encode`); `mspGPSReceiveNewData()` direct passthrough (no validation gate beyond data parse); covariance fields `hPosAccuracy`/`vPosAccuracy`/`hVelAccuracy` align directly with AP `GPS_INPUT.horiz_accuracy`/`vert_accuracy`/`speed_accuracy`; YAMSPy + INAV-Toolkit MIT throughout; `USE_GPS_PROTO_MSP` enabled by default in iNav target/common.h (verified Source #111 + #112 + #113); locked SQ6 + AC-4.3 + restrictions.md transport. **Cand 3 DEFERRED secondary for iNav**: UBX impersonation via pyubx2 NAV-PVT — forging u-blox NAV-PVT frames through standard GPS pipeline; iNav-side `gpsMapFixType()` validation gate requires `flags & 0x01 = 1` (gnssFixOK) AND `fixType ∈ {2,3}` per Source #110 `gps_ublox.c` lines 215-220 + 654; pyubx2 BSD-3-Clause clean dual-use; **does NOT clear user's "significant-improvement-only" bar over Cand 2** — richer protocol surface (NAV-PVT periodic + NAV-VER startup + CFG-MSG/CFG-RATE ACK behaviour) + AC-NEW-7 forgery posture + stricter validation gate + AP-path field-name divergence outweigh pyubx2 library-maturity advantage. **Mid-batch correction**: I caught a contradiction between my own initial AskQuestion phrasing ("UBX impersonation as ONLY iNav path") and locked SQ6 + AC-4.3 + restrictions.md verdicts; user re-locked scope via `c8_inav_recovery=B` to evaluate both as parallel candidates. Decisions: D-C8-1 (NEW Cand-1-only) pymavlink connection-string transport choice (env-driven default-UART recommended); D-C8-2 (NEW Cand-1-only CROSS-COMPONENT with AC-NEW-2) `MAV_CMD_SET_EKF_SOURCE_SET` companion-driven switch ownership pattern (companion publishes to source-set 2 + auto-switches FC recommended); D-C8-3 (NEW Cand-1-only) pymavlink LGPL-3.0 license-posture verification (bundle-unmodified-with-version-pin recommended); D-C8-4 (NEW Cand-2-only) Python MSP V2 implementation choice (YAMSPy primary + thin custom encoder fallback recommended); D-C8-5 (NEW Cand-2-only) MSP2_SENSOR_GPS injection rate (5 Hz periodic recommended); D-C8-6 (NEW Cand-3-only contingent) UBX-version-advertisement strategy (advertise version ≥ 15.0 recommended); D-C8-7 (NEW Cand-3-only contingent CROSS-COMPONENT with AC-NEW-7) AC-NEW-7 audit-trail posture for UBX impersonation (explicit FDR audit entry recommended); D-C8-8 (NEW CROSS-COMPONENT C5+C8) covariance-honesty cross-FC enforcement strategy (per-FC unit conversion recommended via 95% confidence ellipse semi-major axis from C5 GTSAM `Marginals.marginalCovariance`). |
**Cross-cutting consumers** (do not duplicate facts here, just point in):
- The Component Fit Matrix (`../06_component_fit_matrix/`) cites every fact here by `Fact #N` or by candidate row.
---
## Confidence-label legend
| Label | Meaning | Source class |
| --- | --- | --- |
| ✅ High | Source code / official spec / canonical repo verified | L1 (primary code, official docs, published benchmarks) |
| ⚠️ Medium | Authoritative but with stated caveat (out-of-date version, partial coverage, single-source confirmation) | L1 / L2 |
| ❓ Low | Inferential or extrapolated (vendor blog, secondary commentary, candidate not yet runtime-verified on target hardware) | L3 / L4 |
Whenever a candidate is marked **Selected** in `../06_component_fit_matrix/`, its row depends on at least one ✅ High fact in the corresponding C-file plus a `context7` per-mode API capability verification.
---
## Editing rules
1. Add new facts only inside their owning category file. Cross-reference siblings; do not duplicate text.
2. Each fact keeps the existing schema — `### Fact #N — title`, `**Statement**`, `**Source**`, `**Phase**`, `**Confidence**`, `**Sub-Question Binding**`, `**Implication**`.
3. When extending C-rows, also touch the corresponding component file in `../06_component_fit_matrix/` so the matrix stays in sync.
4. Working conclusions and decisions (`D-Cx-y`) live at the bottom of their owning file, not here.
@@ -0,0 +1,261 @@
# Fact Cards — C10: Pre-flight cache provisioning (cross-coupling minimal scope)
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Bound to sub-questions in `../00_question_decomposition.md` line 78 (C10 = "Pre-flight cache provisioning + sector classification + freshness pipeline" with 2026-05-08 user-locked CROSS-COUPLING MINIMAL scope per `c10_scope=C` — see "C10 Scope Restructure" section). Sources for C10 cluster live in [`../01_source_registry/C10_preflight_provisioning.md`](../01_source_registry/C10_preflight_provisioning.md).
>
> Index: [`00_summary.md`](00_summary.md). Sibling components: [C1 VIO](C1_vio.md), [C2 VPR](C2_vpr.md), [C3 Matchers](C3_matchers.md), [C4 Pose](C4_pose_estimation.md), [C5 State estimator](C5_state_estimator.md), [C6 Tile cache + spatial index](C6_tile_cache_spatial_index.md), [C7 On-Jetson inference runtime](C7_inference_runtime.md), [C8 MAVLink/MSP2 FC adapter](C8_fc_adapter.md). Cross-component gates: [`../06_component_fit_matrix/99_cross_component_gates.md`](../06_component_fit_matrix/99_cross_component_gates.md).
---
## Scope summary
C10 batch 1 closed at 2/N on 2026-05-08. **Fact #100** = D-C6-3 confirmation pipeline (descriptor-cache rebuild trigger orchestration for the FAISS HNSW index built during C10 pre-flight provisioning + serialized via `faiss.write_index` + atomic-write + content-hash + manifest-driven rebuild trigger + load-at-takeoff via `faiss.read_index` or memory-mapped via `IO_FLAG_MMAP_IFC`). **Fact #101** = D-C7-7 confirmation pipeline (TensorRT engine-build orchestration via Polygraphy CLI primary + `trtexec` simpler fallback + direct `IBuilderConfig` Python API for reference-Jetson-prebuilt-engine generation; calibration corpus shipping mechanism per D-C7-1 closure). User-pinned scope: cross-coupling-minimal — operator CLI/desktop tooling, sector classification heuristics, and freshness pipeline workflow are **deferred to Plan-phase**.
---
### Fact #100 — D-C6-3 confirmation: descriptor-cache rebuild trigger pipeline orchestrated via direct `faiss.write_index` / `faiss.read_index` Python API + atomic-write + content-hash + manifest-driven rebuild trigger + optional `IO_FLAG_MMAP_IFC` load
**Statement**: For C10 (pre-flight cache provisioning, cross-coupling minimal scope), the D-C6-3 descriptor-cache rebuild trigger pipeline (Recommendation = `periodic rebuild during C10 pre-flight provisioning`) is operationalized as the direct FAISS Python API wrapped in a thin project-side orchestration module:
- **Build pipeline (per pre-flight, manifest-hash-driven)**:
1. C10 pre-flight CLI computes `manifest_hash = sha256(descriptor_blobs.sha256, descriptor_dim, faiss_M, ef_construction, vpr_model_sha256)` over the inputs that would change the index content.
2. Compare to `manifest_hash_prev` recorded in `/var/lib/onboard/cache/faiss/manifest.json` from the last successful build.
3. If `manifest_hash != manifest_hash_prev` (or if `manifest.json` is missing): rebuild the FAISS index. Otherwise: skip.
4. Rebuild = `index = faiss.IndexHNSWFlat(d=descriptor_dim, M=faiss_M)` (per D-C6-2 = `IndexHNSWFlat M=32` recommendation) → `index.hnsw.efConstruction = 40` (per Source #96 / Source #114 / C6 Fact #92 canonical pattern) → `index.add(descriptor_blobs)` → write to disk via the atomic-write wrapper (next bullet).
5. Write atomic-write wrapper:
```python
# pseudocode; implementation may use python-atomicwrites package or be hand-rolled per Source #116
temp_path = target_path + ".tmp"
faiss.write_index(index, temp_path) # FAISS writes serialized binary
fd = os.open(temp_path, os.O_RDONLY)
os.fsync(fd) # flush content + metadata to disk
os.close(fd)
os.rename(temp_path, target_path) # POSIX atomic rename (same filesystem)
parent_fd = os.open(os.path.dirname(target_path), os.O_RDONLY | os.O_DIRECTORY)
os.fsync(parent_fd) # flush directory entry change
os.close(parent_fd)
content_hash = sha256(open(target_path, 'rb').read())
manifest = {"manifest_hash": manifest_hash,
"content_hash": content_hash,
"descriptor_dim": descriptor_dim,
"faiss_M": faiss_M,
"ef_construction": ef_construction,
"n_tiles": index.ntotal,
"build_iso8601": now(),
"vpr_model_sha256": vpr_model_sha256,
"build_duration_sec": build_duration_sec}
write_atomic(manifest_path, json.dumps(manifest))
```
6. C10 also records the build event into the AC-NEW-3 FDR record: `(model="faiss_hnsw", manifest_hash, content_hash, build_duration_sec, n_tiles, descriptor_dim)`.
- **Load pipeline (per takeoff)**:
1. Read `/var/lib/onboard/cache/faiss/manifest.json` → recover `expected_content_hash`.
2. Compute `actual_content_hash = sha256(open(target_path, 'rb').read())` (single-pass file read; ~0.5-2 s on JetPack 6 ARM64 NVMe per ~430 MB halfvec file at 2048-D × 100K tiles per Source #115 size formula).
3. Compare: if `actual != expected` → REJECT the cache; emit `STARTUP_FAULT_FAISS_CACHE_HASH_MISMATCH` MAVLink STATUSTEXT to QGC; refuse takeoff (per AC-NEW-7 cache-poisoning safety budget — never silently load a tampered cache file).
4. Otherwise: `index = faiss.read_index(target_path, faiss.IO_FLAG_MMAP_IFC)` (memory-mapped load — zero-copy; <1 s wall-time for the syscall to set up mmap regardless of file size; per Source #114 supports HNSW + IndexFlatCodes-derived classes via the `IO_FLAG_MMAP_IFC` flag).
5. Optional: warmup query at takeoff (issue ~10 dummy `index.search(rand_query, k=10)` calls) to prime the kernel page cache — smooths post-load p99 latency per Source #115 Issue #622 observation.
- **Pinned input/output contract**:
- inputs: `descriptor_blobs[*]` per tile (numpy.ndarray of shape `(n_tiles, descriptor_dim)` and dtype float32 or halfvec per D-C6-1) computed by C10 pre-flight via running C2 VPR backbone over each cached tile image; `vpr_model_sha256` (the C2 VPR model artifact hash) — feeds into `manifest_hash` so a model-swap forces an index rebuild.
- outputs: `<faiss_cache_dir>/v_<descriptor_dim>_M<HNSW_M>.index` (FAISS binary serialization per Source #114) + `<faiss_cache_dir>/manifest.json` (project-defined JSON manifest with content-hash + build provenance).
- runtime: pre-flight build runs on the operator workstation OR on the deployed Jetson (per D-C7-7 = primary build-on-target-Jetson recommendation; the same workflow runs on the deployed Jetson to avoid the C7-style SM 87 hardware-tying constraint that doesn't apply to FAISS — FAISS HNSW serialization is hardware-agnostic and can be built once on any x86/ARM machine and shipped). Load runs on the deployed Jetson at takeoff via `faiss.read_index` Python call.
**Mode pinning** (per-mode API verification rule):
- inputs: `descriptor_blobs: numpy.ndarray of shape (n_tiles, descriptor_dim) and dtype float32 or halfvec`; `descriptor_dim: int ∈ {256, 512, 1024, 2048, 4096}` per D-C2-9/10/6 final lock; `faiss_M: int = 32` per D-C6-2 lock; `ef_construction: int = 40` per Source #96 + C6 Fact #92 canonical pattern; `vpr_model_sha256: str` for manifest-hash binding
- outputs: serialized FAISS index file at canonical path `<faiss_cache_dir>/v_<descriptor_dim>_M<HNSW_M>.index` + manifest.json with content-hash + build provenance + per-takeoff load latency <5 s (mmap path: <1 s; full-load path at 100K × 2048-D halfvec = ~430 MB / SATA SSD ~500 MB/s = ~0.9 s + page-cache warmup ~1-2 s)
- runtime: FAISS-CPU 1.7+ ARM64 wheel via `pip install faiss-cpu` on JetPack 6 + Python 3.10 + NumPy<2.0.0 (per D-C7-4 cross-coupled numpy-version-pin from C7 batch 1 — same pinning applies here since FAISS-CPU shares the numpy ABI dependency)
**Source**:
- Primary FAISS API: Source #114 (`faiss.write_index` / `faiss.read_index` + `IO_FLAG_MMAP_IFC` flag + explicit security warning — canonical FAISS GitHub Wiki + context7 indexed at `/facebookresearch/faiss`)
- File-size + load-latency formula: Source #115 (FAISS GitHub Discussions #3953 + canonical `IndexHNSWFlat` C++ API docs cross-cite — per-vector cost formula `(vector_dim × 4) + (M × 4 × 2) + overhead`)
- Atomic-write pattern: Source #116 (gocept blog reliable Python file updates + python-atomicwrites docs + Python tracker Issue 8604 — write-temp + fsync + atomic rename + parent-dir fsync canonical pattern; aligns with POSIX `rename(2)` atomicity guarantee)
- Cross-cite: C6 Fact #92 (D-C6-3 originating recommendation = periodic rebuild during C10 pre-flight + `faiss.write_index`), C7 Fact #94 (D-C7-1 calibration-dataset-strategy closure that drives the `vpr_model_sha256` provenance binding)
**Phase**: Mode A Phase 2 — engine Step 3 + Step 7.5 (Component Applicability Gate)
**Confidence**: ✅ High — all evidence is L1/L2 with direct API verification; security-warning-driven content-hash gate is the project-side mitigation for the documented FAISS warning; atomic-write pattern is canonical POSIX semantics; FAISS load latency at the project's pinned descriptor dimensions comfortably fits the <5 s takeoff budget via either full-load or mmap path.
**Sub-Question Binding**:
- SQ3+SQ4 → C10 row in `../06_component_fit_matrix/C10_preflight_provisioning.md` (this fact populates the D-C6-3 confirmation candidate row)
- D-C6-3 cross-coupling: closes the C6 ↔ C10 cross-component gate inherited from C6 Fact #92 (`Plan-phase architect + C10 owner` joint ownership)
- AC-NEW-7 (cache-poisoning safety budget): the content-hash verification gate at takeoff is the project-side mitigation for FAISS's documented "no internal integrity check" warning; binds to AC-NEW-7's per-flight forgery-detection contract
- AC-3.3 (re-localization stability): atomic-write + content-hash gate guarantees same-cache-content → same-cache-load → same-result determinism across reboots and pre-flight rebuilds
**Implication / per-numbered-Restriction × per-numbered-AC sub-matrix**:
| Project Restriction / AC | Verdict | Evidence |
|---|---|---|
| **R-NEW-2 no cloud at flight** | ✅ PASS | All FAISS read/write operations are local; `faiss.read_index` opens a local file; no network calls. |
| **R-NEW-4 Jetson Orin Nano Super JetPack 6 ARM64** | ✅ PASS | FAISS-CPU ARM64 wheels are available via `pip install faiss-cpu` (cross-cite C6 Fact #92 + Source #97); no Jetson-specific issues with `faiss.write_index` / `faiss.read_index` / `IO_FLAG_MMAP_IFC` (canonical FAISS Python API works identically on ARM64). |
| **AC-1.x position accuracy** | N/A | Cache file write/read is downstream of accuracy; this fact concerns the descriptor-cache provenance layer. |
| **AC-3.3 re-localization stability** | ✅ PASS | Atomic-write + content-hash gate guarantees deterministic cache load across reboots; rebuild only when manifest hash changes; no silent cache mutation at runtime. |
| **AC-3.4 operator re-loc hint** | ✅ PASS | Operator re-loc hint uses the same loaded FAISS index (no rebuild required at runtime); content-hash gate at takeoff suffices. |
| **AC-4.1 latency budget (<400 ms p95 end-to-end)** | N/A | This is pre-flight + takeoff-load, NOT runtime per-frame. Runtime per-frame latency is governed by C6 Fact #92 (~6-54 ms per cache hit). |
| **AC-4.2 memory budget (<8 GB shared on Jetson)** | ✅ PASS | FAISS index in-memory footprint at the project's pinned descriptor dimensions: ~430 MB at 2048-D halfvec × 100K tiles per Source #115 formula (well within C6 Fact #92's 700 MB-1.5 GB Postgres+FAISS+cache subtotal). With `IO_FLAG_MMAP_IFC` the index is mmap'd from disk on demand — peak RSS reduces further at the cost of a page-fault per first-time access. |
| **AC-4.5 look-back refinement** | N/A | Pre-flight cache + takeoff load are forward-only events. |
| **AC-8.3 10 GB persistent tile cache budget** | ✅ PASS | FAISS index file size at the project's pinned descriptor dimensions: ~430 MB at 2048-D halfvec × 100K tiles + ~80-160 MB at 256-D/512-D halfvec for smaller VPR backbones — fits comfortably within the 10 GB cache budget (well under 5% even at the largest 2048-D variant). |
| **AC-NEW-1 cold-start TTFF (<30 s p95)** | ✅ PASS | Takeoff-load via mmap path: <1 s; full-load path at 430 MB file: ~0.9-2 s; well within the AC-NEW-1 30-second cold-start TTFF budget. Content-hash gate adds ~0.5-2 s for the 430 MB SHA-256 pass; together <5 s — comfortably within budget. |
| **AC-NEW-3 (FDR)** | ✅ PASS | Per-rebuild manifest entry (manifest_hash, content_hash, build_duration_sec, n_tiles, descriptor_dim, vpr_model_sha256) is recordable as an FDR field; per-takeoff load-latency + hash-verification result are recordable as FDR fields. |
| **AC-NEW-4 covariance honesty** | N/A | Pre-flight pipeline is upstream of the C5 estimator; covariance honesty is C5's contract. |
| **AC-NEW-7 cache-poisoning safety budget** | ✅ PASS at the FAISS-cache layer | Content-hash gate at takeoff load REJECTS cache files that don't match the manifest (per Source #114 explicit security warning); atomic-write pattern (Source #116) prevents partial-write corruption from masquerading as a valid cache; manifest-hash-driven rebuild triggers ensure that a model swap forces a rebuild with new content hash. **Cross-flight cache poisoning** (per AC-NEW-7's "P(geo-misalign >30 m) <1%" budget) is upstream of C10 — it's the C6 Fact #92 + AC-8.4 mid-flight tile generation responsibility plus the Suite Service voting layer per AC-NEW-7 external-dependency note. |
| **AC-NEW-8 blackout failsafe** | ✅ PASS | Pre-flight pipeline doesn't run during flight; if the FAISS cache is corrupt at takeoff, the cache-hash-mismatch gate refuses takeoff (which is safer than launching with a bad cache). C5 demotion to `dead_reckoned` is the runtime failsafe path, not the pre-flight one. |
**Strengths** (positive structural advantages):
1. **Direct FAISS API — minimal abstraction surface**. No additional library dependency beyond FAISS-CPU (already required by C6 Fact #92); no orchestration framework to maintain. The atomic-write wrapper is ~30 lines of Python; trivially auditable; works identically across operator workstation + deployed Jetson environments.
2. **Manifest-hash-driven rebuild trigger** — idempotent (skip rebuild if no change); minimum-rebuild semantics (rebuild only when descriptor_blobs OR vpr_model_sha256 OR descriptor_dim changes); aligns naturally with C10 pre-flight workflow (descriptor blobs change when tiles are pulled/refreshed; VPR model changes only on dev-side model swap).
3. **Content-hash verification gate at takeoff** — operationalizes the FAISS security warning as project-side AC-NEW-7 coverage; never silently loads a tampered cache file.
4. **Atomic-write pattern guarantees crash safety** — power loss or process kill mid-build leaves the previous valid cache file intact (per POSIX `rename(2)` atomicity); next pre-flight rebuild detects the manifest mismatch and rebuilds cleanly.
5. **Optional mmap load path (`IO_FLAG_MMAP_IFC`)** — zero-copy load syscall completes in <1 s regardless of file size; reduces takeoff RSS pressure; canonical FAISS HNSW + IndexFlatCodes-derived support per Source #114.
6. **Hardware-agnostic FAISS serialization** — index can be built on the operator workstation (x86) and shipped to the Jetson (ARM64) without rebuild (vs C7's SM 87 hardware-tying constraint for TensorRT engines). Useful for the prebuilt-fallback path.
7. **License clean throughout** — FAISS (MIT); python-atomicwrites if used (MIT); no GPL contagion path on this orchestration layer.
**Negative-but-mitigable structural findings**:
8. **No FAISS-internal integrity check on `read_index`** (per Source #114 explicit warning) — must be mitigated project-side via the content-hash gate above. Without that gate, AC-NEW-7 fails. **Mitigation**: project-side ~5 lines of Python (open file → SHA-256 → compare to manifest) before the `read_index` call; cost ~0.5-2 s at takeoff for a 430 MB cache file.
9. **Atomic-write pattern is project-side, not FAISS-internal** — must be hand-rolled or via `python-atomicwrites`. **Mitigation**: ~30 lines of Python; well-documented canonical POSIX pattern per Source #116; trivially auditable.
10. **Manifest-hash binding requires VPR model SHA-256** — implies the C2 VPR model artifact has a stable SHA-256 (i.e., a versioned ONNX-or-engine file is checked into the cache directory or referenced from a versioned URI). **Mitigation**: standard ML artifact versioning; aligns with the C7 Fact #94 + C7 Fact #95 + C7 Fact #96 ONNX export pathway (each ONNX export is a binary file with a deterministic hash).
11. **Mmap path RAM behavior depends on OS page cache pressure** — if other workloads consume RAM, mmap'd FAISS index pages may be evicted and re-faulted at runtime, adding ~1-5 ms per evicted page-fault to per-frame query latency. **Mitigation**: `mlock` / `madvise(MADV_WILLNEED)` syscalls available in Python via `mmap.MADV_WILLNEED` to pre-fault the pages; cost: one-time at takeoff (~1-2 s for the 430 MB file). At 8 GB shared budget (with C6 Fact #92's 700 MB-1.5 GB total subtotal) there's ample headroom for keeping the mmap'd index resident.
**Caveats / open Plan-phase decisions raised** (D-C10-N gates):
- **D-C10-1 NEW** — descriptor-cache rebuild trigger choice (manifest-hash-driven [recommended] / always-rebuild-every-pre-flight / operator-manual flag): trade-off between idempotency vs simplicity vs operator control. **Recommendation**: D-C10-1 = (a) manifest-hash-driven (idempotent + minimum-rebuild + operator-manual override flag `--force-rebuild` available).
- **D-C10-2 NEW** — descriptor-cache atomic-write strategy (write-temp+fsync+rename hand-rolled / `python-atomicwrites` package / accept-non-atomic-write-and-pray): trade-off between dependency surface vs implementation cost vs crash safety. **Recommendation**: D-C10-2 = (b) `python-atomicwrites` (MIT, ~zero-cost dependency, cross-platform, well-tested in production); fallback (a) hand-rolled if dependency-policy gate prefers in-tree.
- **D-C10-3 NEW (CROSS-COMPONENT with AC-NEW-7)** — content-hash verification gate at takeoff load (yes — REJECT cache + STATUSTEXT + refuse takeoff [recommended] / yes — WARN + load anyway / no — trust filesystem): trade-off between safety vs availability vs operator-friction. **Recommendation**: D-C10-3 = (a) reject-and-refuse-takeoff; AC-NEW-7 cache-poisoning budget makes silent acceptance unsafe; operator can re-run pre-flight with `--force-rebuild` to cleanly recover.
- **D-C10-4 NEW** — descriptor-cache load path (full-`read_index` / mmap via `IO_FLAG_MMAP_IFC` [recommended] / both available via env flag): trade-off between determinism (full-load is fully resident; mmap RSS depends on page cache) vs takeoff latency (mmap is faster) vs runtime page-fault sensitivity. **Recommendation**: D-C10-4 = (b) mmap with optional `madvise(MADV_WILLNEED)` pre-fault at takeoff (~1-2 s additional cost; eliminates runtime page-faults for the lifetime of the flight) OR (c) both available for Plan-phase Jetson MVE comparison.
---
### Fact #101 — D-C7-7 confirmation: TensorRT engine-build pipeline orchestrated via Polygraphy CLI (primary) + `trtexec` (simpler fallback) + direct `IBuilderConfig` Python API (reference-Jetson-prebuilt-engine fallback generation)
**Statement**: For C10 (pre-flight cache provisioning, cross-coupling minimal scope), the D-C7-7 TensorRT engine-build pipeline (Recommendation = `primary build-on-deployed-Jetson during pre-flight + reference-Jetson-built engines as fallback`) is operationalized as a three-tool orchestration matrix:
- **Primary path: Polygraphy CLI on the deployed Jetson during pre-flight** (per D-C7-7 = primary build-on-target):
```bash
polygraphy convert <model>.onnx \
--int8 --fp16 \
--data-loader-script ./calib_data_loader.py \
--calibration-cache <calib_cache_dir>/<model>_calib.cache \
--workspace=1000000000 \
-o <engine_cache_dir>/<model>_sm87_jp62_trt103_<precision>.engine
```
- First build per-model: `--data-loader-script` reads the project's pinned calibration corpus per D-C7-1 closure (real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles; ~500-1500 representative samples per Source #120) and runs INT8 calibration; the resulting calibration scales are written to `--calibration-cache` for subsequent builds.
- Subsequent rebuilds (when calibration corpus is unchanged): `polygraphy convert ... --calibration-cache <existing_cache>` — calibration step is skipped per Source #117 ("If the provided path does exist, it will be read and int8 calibration will be skipped during engine building").
- Per-model precision flags follow D-C7-2 / D-C7-6 cross-component policy: VPR backbones (CNN-class) → `--int8 --fp16`; ViT-class VPR + matchers + learned VIO → `--fp16` only (NO `--int8`).
- `--workspace=1000000000` (1 GB cap) per D-C7-8 lock to prevent tactic-profile segfault on 8 GB shared budget.
- On-disk engine filename incorporates SM 87 + JetPack 6.2 + TRT 10.3 + precision tag (per D-C7-9 lock) so the runtime can reject a cached engine that was built for a different SM/JP/TRT/precision combination.
- **Simpler fallback: `trtexec` CLI** (when calibration cache already exists or for ad-hoc/emergency rebuilds):
```bash
trtexec --onnx=<model>.onnx \
--saveEngine=<engine_cache_dir>/<model>_sm87_jp62_trt103_<precision>.engine \
--fp16 --int8 \
--calib=<calib_cache_dir>/<model>_calib.cache \
--shapes=input:1x3x224x224 \
--workspace=1000
```
- Faster invocation (no Python imports; single C++ binary).
- Calibration cache file format is interoperable with Polygraphy's per Source #119 — caches built by Polygraphy are loadable by `trtexec` and vice versa.
- Used as fallback when Polygraphy is unavailable (e.g., minimal install) OR for reference-Jetson-prebuilt-engine generation when no calibration data shipping is needed.
- Critical caveat: `trtexec --int8` without `--calib` falls back to RANDOM data calibration → ~5-15% INT8 accuracy collapse → forbidden in the project's C10 path (always supply `--calib` from the existing calibration cache).
- **Reference-Jetson-prebuilt-engine fallback generation** (per D-C7-7 fallback path, for emergency provisioning): direct TensorRT `IBuilderConfig` + `IInt8EntropyCalibrator2` Python API per Source #121 — used when Polygraphy's `--data-loader-script` abstraction is too rigid for an unusual model (e.g., LightGlue with dynamic-shape inputs requiring a custom calibration profile per D-C3-2 + D-C3-3). Output: a versioned `.engine` file shipped to the deployed Jetson alongside the calibration cache file. The deployed Jetson at takeoff loads this prebuilt engine via `IRuntime.deserializeCudaEngine` (no on-Jetson rebuild required for the fallback path).
- **Manifest-hash + content-hash + atomic-write** (same pattern as Fact #100):
- `manifest_hash = sha256(model_onnx.sha256, calibration_corpus.sha256, precision_mode, sm_version, jp_version, trt_version)` per engine.
- `content_hash = sha256(<engine>.engine)` after build.
- Atomic-write wrapper around the engine file output (Polygraphy + trtexec both write to a temp path inside their respective CLIs, but the project-side wrapper enforces the rename-into-position step on top to maintain crash safety across the broader pre-flight workflow).
- Per-engine manifest entry recorded in `<engine_cache_dir>/manifest.json`: `(model, precision_mode, calib_corpus_sha256, build_iso8601, build_duration_sec, content_hash, sm_version, jp_version, trt_version)`.
- **Pinned input/output contract**:
- inputs: `<model>.onnx` per inference target (C2 VPR backbone + C3 matcher + optional C1 learned VIO frontend, exported on the dev machine via `torch.onnx.export`); `calibration_corpus` per D-C7-1 closure (real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles in NumPy `.npy` or Torch `.pt` tensor format); `<calib_cache>` per Polygraphy/trtexec INT8 calibration cache file (project-side ships the calibration corpus + the calibration cache; cache is reusable across rebuilds when the corpus hash is unchanged).
- outputs: per-model `.engine` file at canonical path `<engine_cache_dir>/<model>_sm87_jp62_trt103_<precision>.engine` + per-engine manifest entry in `<engine_cache_dir>/manifest.json` + AC-NEW-3 FDR record.
- runtime context: pre-flight build runs ON the deployed Jetson Orin Nano Super (per D-C7-7 = primary build-on-target — per Source #105 SM 87 hardware-tying constraint). Reference-Jetson-prebuilt-engine fallback runs on a known-good HQ Jetson (same SM 87 / JetPack 6.2 / TensorRT 10.3 — per D-C7-9 lock).
**Mode pinning** (per-mode API verification rule):
- inputs: `<model>.onnx: bytes` (ONNX graph from `torch.onnx.export`); `calibration_corpus: numpy.ndarray of shape [N=500-1500, C=3, H=224-320, W=224-320] and dtype float32 normalized to [0, 1]` per project's pinned VPR + matcher input shapes per D-C2-3 / D-C2-5 / D-C3-3; `precision_mode: str ∈ {'int8+fp16', 'fp16'}` per D-C7-6 per-family policy
- outputs: serialized TensorRT engine file `.engine` + calibration cache file `.cache` (interoperable between Polygraphy and trtexec per Source #119) + manifest entry
- runtime: TensorRT 10.3 + CUDA 12.6 + cuDNN 9.3 on JetPack 6.2 + Polygraphy bundled with TensorRT distribution OR `pip install nvidia-pyindex && pip install polygraphy` (Polygraphy is pure Python; ARM64 Python + TensorRT Python bindings sufficient)
**Source**:
- Primary Polygraphy CLI: Source #117 NVIDIA/TensorRT GitHub `tools/Polygraphy/examples/cli/convert/01_int8_calibration_in_tensorrt/README.md` + canonical Polygraphy docs context7 indexed at `/websites/nvidia_deeplearning_tensorrt_static_polygraphy` (1041 code snippets, Source Reputation High)
- Polygraphy `Calibrator` class API: Source #118 canonical NVIDIA TensorRT/Polygraphy SDK documentation (entropy/min-max algo defaults, dynamic-shapes calibration profile, data-loader-script + calibration-cache CLI flags)
- `trtexec` CLI: Source #119 canonical NVIDIA TensorRT SDK documentation (`--onnx --saveEngine --int8 --fp16 --calib --shapes --workspace` flag set; calibration cache format interoperability with Polygraphy)
- Calibration corpus size guidance: Source #120 vendor-aligned engineering guide (500-1000 image recommendation; cross-cite to project's D-C7-1 closure 500-1500 sample range)
- Direct `IBuilderConfig` Python API: Source #121 (cross-cite from C7 batch 1 Source #102 + Source #105) — used for reference-Jetson-prebuilt-engine fallback generation
- Cross-cite: C7 Fact #94 (D-C7-7 originating recommendation = primary build-on-deployed-Jetson + fallback prebuilt; D-C7-8 = 1 GB workspace; D-C7-9 = JetPack 6.2 + TRT 10.3 lock); C7 Fact #94 (D-C7-1 closure = real UAV nadir flight footage as calibration corpus distribution; specific fixture pin delegated to Test Spec)
**Phase**: Mode A Phase 2 — engine Step 3 + Step 7.5 (Component Applicability Gate)
**Confidence**: ✅ High for Polygraphy + trtexec API capability verification (L1 canonical NVIDIA docs); ✅ High for the orchestration pattern (canonical NVIDIA-blessed workflow per Source #117 README); ⚠️ Medium for the specific build-duration-on-Jetson-Orin-Nano-Super claim (extrapolated from C7 Fact #94 reference of "30-300 sec per model" + Source #105 constraints — exact build-duration depends on model complexity + INT8 calibration scope; needs Plan-phase Jetson MVE confirmation per D-C1-2)
**Sub-Question Binding**:
- SQ3+SQ4 → C10 row in `../06_component_fit_matrix/C10_preflight_provisioning.md` (this fact populates the D-C7-7 confirmation candidate row)
- D-C7-7 cross-coupling: closes the C7 ↔ C10 cross-component gate inherited from C7 Fact #94 (`Plan-phase architect + C10 owner` joint ownership)
- D-C7-1 closure (real UAV nadir flight footage corpus): C10 owns the calibration-corpus assembly at pre-flight; specific fixture-file pin remains delegated to Test Spec per the 2026-05-08 C9 / SQ7 restructure
- AC-NEW-1 (cold-start TTFF <30 s p95): pre-flight engine build is amortized across all takeoffs that use the same artifacts; takeoff-load via `IRuntime.deserializeCudaEngine` is ~100-500 ms per engine × 3-5 engines = ~0.5-2.5 s — well within 30 s budget
- AC-NEW-3 (FDR): per-engine manifest entry recorded as FDR field
- AC-NEW-7 (cache-poisoning safety): same content-hash + atomic-write pattern as Fact #100 protects the engine cache file against partial-write corruption
**Implication / per-numbered-Restriction × per-numbered-AC sub-matrix**:
| Project Restriction / AC | Verdict | Evidence |
|---|---|---|
| **R-NEW-2 no cloud at flight** | ✅ PASS | All Polygraphy/trtexec invocations are local CLI subprocess calls; engine build runs entirely on the deployed Jetson. |
| **R-NEW-4 Jetson Orin Nano Super JetPack 6 ARM64** | ✅ PASS | Polygraphy is pure Python (works on ARM64 + Python 3.10); trtexec is bundled with TensorRT 10.3 in JetPack 6.2 (installed by default at `/usr/src/tensorrt/bin/trtexec`); both interoperate with the JetPack-bundled TensorRT 10.3 per Source #117 + Source #119. |
| **AC-1.x position accuracy** | N/A | Engine build is upstream of accuracy; this fact concerns the engine provenance layer. |
| **AC-3.x resilience** | N/A | Engine cache is a takeoff-load artifact; runtime resilience is C5/C8 responsibility. |
| **AC-4.1 latency budget (<400 ms p95 end-to-end)** | N/A | Engine build is pre-flight + takeoff-load, NOT runtime per-frame. Per-engine inference latency is governed by C7 Fact #94 / Fact #95 / Fact #96. |
| **AC-4.2 memory budget (<8 GB shared on Jetson)** | ✅ PASS | Per Source #105 + D-C7-8: Polygraphy/trtexec engine build with `--workspace=1000` (1 GB cap) holds peak build-time memory at ~3-5 GB out of 8 GB shared (build-time peak; runtime is much lower per C7 Fact #94 ~50-150 MB shared library + ~50-300 MB per engine). Pre-flight build is performed when no other workloads are active, so the 5 GB peak is acceptable. |
| **AC-4.5 look-back refinement** | N/A | Engine build pipeline is forward-only. |
| **AC-8.3 10 GB persistent tile cache budget** | ✅ PASS | Engine `.engine` files at 10-200 MB each per C7 Fact #94 × 3-5 engines = ~100-500 MB on disk (separate from the 10 GB tile cache; lives at `/var/lib/onboard/cache/trt/` or equivalent). Calibration cache files at 1-10 MB each are negligible. |
| **AC-NEW-1 cold-start TTFF (<30 s p95)** | ✅ PASS | Takeoff-load via `IRuntime.deserializeCudaEngine` is ~100-500 ms per engine × 3-5 engines = ~0.5-2.5 s; combined with FAISS load <5 s (Fact #100) and content-hash gates total ~5-10 s, well within 30 s budget. **Build is pre-flight, NOT during cold-start** — engines are pre-built during pre-flight provisioning and persisted across reboots. |
| **AC-NEW-3 (FDR)** | ✅ PASS | Per-engine manifest entry (model, precision_mode, calib_corpus_sha256, build_iso8601, build_duration_sec, content_hash, sm_version, jp_version, trt_version) is recordable as an FDR field per AC-NEW-3 forensic trail requirement. |
| **AC-NEW-4 covariance honesty** | N/A | Engine build pipeline is upstream of the C5 estimator. |
| **AC-NEW-7 cache-poisoning safety budget** | ✅ PASS at the engine-cache layer | Same content-hash + atomic-write pattern as Fact #100 (project-side wrapper around Polygraphy/trtexec output); engine-cache poisoning is detected at takeoff load via SHA-256 verification; manifest-hash binding guarantees that a calibration-corpus swap or ONNX-model swap forces a clean rebuild with new content hash. The reference-Jetson-prebuilt-engine fallback path uses a versioned `.engine` artifact that is signed/checksummed at the HQ source-of-truth (the project's release pipeline owns this signing). |
| **AC-NEW-8 blackout failsafe** | ✅ PASS | Engine cache is loaded at takeoff; if a content-hash mismatch is detected, takeoff is refused (same posture as Fact #100). C5 demotion to `dead_reckoned` is the runtime failsafe path, not the pre-flight one. |
**Strengths** (positive structural advantages):
1. **Polygraphy is the canonical NVIDIA-blessed orchestration tool** for TensorRT engine builds with INT8 calibration cache reuse — first-party support, multi-snippet docs coverage, production-mature; eliminates the need to write the calibrator + data-loader + builder-config glue code from scratch.
2. **Calibration cache reuse across rebuilds** — first build per-model takes ~30-300 sec including INT8 calibration (per C7 Fact #94 reference); subsequent rebuilds skip the calibration step (per Source #117 explicit "calibration will be skipped" semantics) — typically <30 sec even for the most complex matchers. Critical for fast iteration during the operator's pre-flight workflow.
3. **CLI interoperability between Polygraphy and trtexec** — the calibration cache file format is identical between the two tools per Source #119; the project can use Polygraphy for the canonical INT8-calibration-bearing build and trtexec for emergency/ad-hoc rebuilds without re-shipping calibration data.
4. **Mixed-precision flag matrix matches D-C7-2 / D-C7-6 cross-component policy** — `--int8 --fp16` is the canonical Polygraphy/trtexec invocation for the project's per-family mixed precision per Source #117 + Source #119.
5. **`--load-tactics` / `--save-tactics` for reference-Jetson-prebuilt-engine workflow** — Polygraphy supports replaying tactic-search results across multiple builds (per Source #118); the project can ship the tactic replay file alongside the prebuilt engine for fast on-Jetson rebuild without re-running tactic profiling.
6. **Direct `IBuilderConfig` Python API as escape hatch** — for unusual models requiring custom calibration profiles (e.g., LightGlue with dynamic-shape inputs per D-C3-2 + D-C3-3) the project can drop down to the direct TensorRT Python API per Source #121 without abandoning the orchestration framework.
7. **Pre-flight build amortized across all takeoffs** — engine cache is persistent; build runs only when calibration corpus or ONNX model changes (manifest-hash-driven); typical operator workflow is: build once at HQ ship → operator pulls fresh tile cache → operator triggers pre-flight (FAISS rebuild + maybe TRT rebuild if calibration-corpus refreshed) → takeoff.
8. **License clean throughout** — Polygraphy (Apache-2.0); TensorRT (Apache-2.0 in TensorRT 10.x per C7 Fact #94); python-atomicwrites (MIT); no GPL contagion path on this orchestration layer.
**Negative-but-mitigable structural findings**:
9. **First-build INT8 calibration takes 30-300 sec per model on Jetson** — large matcher models (e.g., LightGlue at K=1024 keypoints) can hit the upper end of this range. **Mitigation**: calibration cache reuse — once the cache is built, subsequent rebuilds are <30 sec; first build at HQ + ship cache to operator workstation pre-deployment.
10. **Engine cache is hardware-specific (SM 87)** per C7 Fact #94 + Source #105 — can't ship engines across Jetson hardware variants. **Mitigation**: D-C7-7 = (c) primary-build-on-target with reference-Jetson-prebuilt-engine fallback ONLY for SM 87 / JetPack 6.2 / TRT 10.3 combinations; the project's deployed fleet is uniform per restrictions.md (Jetson Orin Nano Super pinned).
11. **Polygraphy CLI requires `pip install polygraphy` separately if not bundled with TensorRT distribution** — minimal Jetson installs may need `pip install nvidia-pyindex && pip install polygraphy`. **Mitigation**: include in the project's pre-flight Docker image / OS image bake; verify at C10 setup.
12. **`trtexec --int8` without `--calib` falls back to random-data calibration** with documented ~5-15% INT8 accuracy collapse per Source #119. **Mitigation**: project-side wrapper around `trtexec` invocation enforces `--calib=<existing_cache>` non-empty as a precondition; reject the build otherwise with clear error message.
13. **Build-time peak memory ~3-5 GB out of 8 GB shared** per Source #105 constraint #4 + D-C7-8 — not safe to run pre-flight build concurrently with other heavy workloads (e.g., camera pipeline, FAISS build). **Mitigation**: pre-flight orchestration is sequential — build TRT engines one at a time, then FAISS index, then verification; takes ~5-15 min total at first-build (with calibration); ~1-3 min for subsequent rebuilds (cache-reused).
14. **Calibration-corpus shipping mechanism** — per D-C7-1 closure the corpus is real UAV nadir flight footage at ~1 km AGL; this corpus is several GB of tensor data. **Mitigation**: ship calibration corpus + calibration cache together as a versioned artifact bundle; ship cache only (not raw corpus) to operators when the cache is sufficient (i.e., fixture-pin from Test Spec is stable and operators don't need to recalibrate).
**Caveats / open Plan-phase decisions raised** (D-C10-N gates):
- **D-C10-5 NEW (CROSS-COMPONENT with C7)** — TensorRT engine-build orchestration tool choice (Polygraphy CLI primary [recommended] / `trtexec` CLI primary / direct `IBuilderConfig` Python API primary / hybrid: Polygraphy for INT8-calibrating builds + `trtexec` for cache-reuse rebuilds + direct API for unusual models): trade-off between orchestration sophistication vs install footprint vs flexibility. **Recommendation**: D-C10-5 = (d) hybrid — Polygraphy for INT8-calibrating builds (canonical NVIDIA tool, multi-snippet docs, supports custom data loaders); `trtexec` for cache-reuse fast rebuilds (single binary, no Python imports, faster invocation); direct `IBuilderConfig` Python API as escape hatch for unusual models (e.g., LightGlue dynamic shapes per D-C3-2 + D-C3-3).
- **D-C10-6 NEW (CROSS-COMPONENT with D-C7-1)** — TensorRT calibration-cache reuse strategy (always reuse if cache file exists [most-aggressive] / rebuild on calib-corpus SHA-256 change [recommended] / rebuild every pre-flight [most-conservative]): trade-off between rebuild cost vs calibration-data freshness vs operator-workflow simplicity. **Recommendation**: D-C10-6 = (b) rebuild on calib-corpus SHA-256 change — manifest-hash-driven rebuild trigger from Fact #100 pattern naturally extends to TRT engine cache; idempotent + minimum-rebuild + operator-manual override flag `--force-trt-rebuild` available.
- **D-C10-7 NEW** — TensorRT engine on-disk filename schema (`<model>_sm<SM>_jp<JP>_trt<TRT>_<precision>.engine` [recommended] / hash-only filename / opaque content-addressable storage with separate manifest mapping): trade-off between operator-debuggability vs filesystem-simplicity vs versioning-rigor. **Recommendation**: D-C10-7 = (a) `<model>_sm<SM>_jp<JP>_trt<TRT>_<precision>.engine` self-describing filename + manifest.json side-cache; runtime can reject a cached engine that doesn't match the deployed Jetson's SM/JP/TRT combination with a clear error message at takeoff load.
- **D-C10-8 NEW** — TensorRT prebuilt-fallback engine generation venue (reference Jetson at HQ [recommended] / CI pipeline with Jetson-class runner / deployed Jetson copy-to-HQ-archive after first successful local build): trade-off between reproducibility vs CI cost vs reduced pre-flight risk. **Recommendation**: D-C10-8 = (a) reference Jetson at HQ + (c) deployed-Jetson-copy-to-archive on first successful local build for opportunistic redundancy; both venues use the same Polygraphy/trtexec pipeline so artifacts are interchangeable; HQ-built engines serve as authoritative fallbacks signed by the project's release pipeline.
---
## C10 — Working conclusions and decisions (compounded from Fact #100 + Fact #101 closures)
**Selected primary**:
- **D-C6-3 confirmation**: descriptor-cache rebuild trigger pipeline orchestrated via direct `faiss.write_index` / `faiss.read_index` Python API + `python-atomicwrites` (or hand-rolled atomic-write) + content-hash verification gate at takeoff + manifest-hash-driven rebuild trigger + optional `IO_FLAG_MMAP_IFC` mmap load path with `madvise(MADV_WILLNEED)` pre-fault. **Closes the C6 ↔ C10 cross-component gate.**
- **D-C7-7 confirmation**: TensorRT engine-build pipeline orchestrated via the **hybrid** tool matrix per D-C10-5 = (d): Polygraphy CLI for INT8-calibrating builds (primary) + `trtexec` for cache-reuse fast rebuilds + direct `IBuilderConfig` Python API for unusual models (LightGlue dynamic shapes). Reference-Jetson-prebuilt-engine fallback per D-C10-8 = (a)+(c). Calibration corpus per D-C7-1 closure (real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles; specific fixture-file pin delegated to Test Spec). **Closes the C7 ↔ C10 cross-component gate.**
**Decisions raised (D-C10-N gates)** — see [`../06_component_fit_matrix/99_cross_component_gates.md`](../06_component_fit_matrix/99_cross_component_gates.md):
- **D-C10-1** (Fact #100) — descriptor-cache rebuild trigger choice: manifest-hash-driven / always-rebuild / operator-manual — RECOMMENDED manifest-hash-driven + `--force-rebuild` override
- **D-C10-2** (Fact #100) — descriptor-cache atomic-write strategy: hand-rolled / `python-atomicwrites` / no-atomic — RECOMMENDED `python-atomicwrites` (fallback hand-rolled if dependency-policy gate prefers in-tree)
- **D-C10-3** (Fact #100, CROSS-COMPONENT with AC-NEW-7) — content-hash verification gate at takeoff load: reject + STATUSTEXT + refuse takeoff / warn + load anyway / no — RECOMMENDED reject + STATUSTEXT + refuse takeoff
- **D-C10-4** (Fact #100) — descriptor-cache load path: full-`read_index` / mmap via `IO_FLAG_MMAP_IFC` / both via env flag — RECOMMENDED mmap with `madvise(MADV_WILLNEED)` pre-fault (or both for Plan-phase Jetson MVE)
- **D-C10-5** (Fact #101, CROSS-COMPONENT with C7) — TensorRT engine-build orchestration tool choice: Polygraphy primary / trtexec primary / direct API primary / hybrid — RECOMMENDED hybrid (Polygraphy + trtexec + direct API by use case)
- **D-C10-6** (Fact #101, CROSS-COMPONENT with D-C7-1) — TensorRT calibration-cache reuse strategy: always-reuse / rebuild-on-calib-corpus-SHA-256-change / rebuild-every-pre-flight — RECOMMENDED rebuild-on-calib-corpus-SHA-256-change + `--force-trt-rebuild` override
- **D-C10-7** (Fact #101) — TensorRT engine on-disk filename schema: self-describing `<model>_sm<SM>_jp<JP>_trt<TRT>_<precision>.engine` / hash-only / content-addressable + manifest — RECOMMENDED self-describing filename + manifest.json side-cache
- **D-C10-8** (Fact #101) — TensorRT prebuilt-fallback engine generation venue: reference Jetson at HQ / CI pipeline with Jetson-class runner / deployed-Jetson-copy-to-HQ-archive on first successful local build — RECOMMENDED reference Jetson at HQ + deployed-Jetson-copy-to-archive (opportunistic redundancy)
C10 batch 1 closed at 2/N on 2026-05-08 (cross-coupling minimal scope per `c10_scope=C` user choice). Operator CLI/desktop tooling, sector classification heuristics, freshness pipeline workflow remain **deferred to Plan-phase as `operator tooling design` out-of-research-scope**. **No further C10 batches required at the research layer** — D-C6-3 and D-C7-7 are now closed; remaining C10 questions are operational/UX, not architectural.
---
+396
View File
@@ -0,0 +1,396 @@
# Fact Cards — C1: Visual / Visual-Inertial Odometry
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Extracted from sources logged in `../01_source_registry/C1_vio.md` (see `../01_source_registry/00_summary.md` for index). Confidence labels: ✅ High (L1 / verified source code), ⚠️ Medium (L1/L2 with caveat), ❓ Low (L3/L4 inferential). Bound to sub-questions in `../00_question_decomposition.md`.
>
> Index: [`../00_summary.md`](../00_summary.md). Sibling categories: SQ6 ([FC external positioning](SQ6_fc_external_positioning.md)), SQ1 ([existing systems](SQ1_existing_systems.md)), SQ2 ([canonical pipeline](SQ2_canonical_pipeline.md)), C2 ([VPR](C2_vpr.md)), C3 ([matchers](C3_matchers.md)).
**Facts in this file**: VIO candidate enumeration (VINS-Mono, VINS-Fusion, OpenVINS, OKVIS2, Kimera-VIO, DROID-SLAM, DPVO, KLT+RANSAC baseline) + Plan-phase decisions D-C1-1, D-C1-2 + C1 working conclusions.
---
## SQ3+SQ4 / C1 — Visual / Visual-Inertial Odometry candidate enumeration
> **Project's pinned mode for every C1 candidate (binding)**: monocular ADTi 20MP nav camera @ 3 fps + IMU from FC over MAVLink @ ≥100 Hz, on Jetson Orin Nano Super (JetPack/CUDA/TensorRT, 8 GB shared LPDDR5, 25 W TDP), producing relative 6-DoF metric pose between consecutive frames + per-axis covariance, with attitude (yaw + pitch) hard-contract σ ≤ 5° at 1 σ (Fact #24), output cadence ≥3 Hz, no in-flight network, license compatible with onboard-binary distribution to a dual-use customer.
>
> Per the engine's "Per-Mode API Capability Verification" rule, any candidate marked `Selected` requires a `context7` lookup (mode enum + project's exact mode runnable example + disqualifier probe) AND a per-numbered-Restriction × per-numbered-AC sub-matrix. **This session covers candidate enumeration + preliminary applicability assessment only**; `context7` verification and the structured sub-matrix are deferred to the next session per the autodev context budget heuristic.
### Fact #28 — VINS-Mono is a canonical monocular-only sliding-window VIO with a working Jetson-Nano deployment record but no GitHub release and ~24-month-old master branch
- **Statement**: VINS-Mono is the canonical mono+IMU sliding-window VIO from HKUST-Aerial-Robotics (Qin, Li, Shen — IEEE T-RO 2018). Features: efficient IMU pre-integration, automatic initialization, online camera-IMU spatial + temporal calibration, failure detection + recovery, DBoW2 loop detection, global pose-graph optimization. Output: metric-scale 6-DoF pose at IMU rate. **Repository state**: master-branch only (no tagged releases), 5,829 stars; last meaningful master-branch commit 2024-02-25 with a 2024-05-23 simulation-data commit. **Jetson record**: a 2021 IEICE paper (zinuok / KAIST) demonstrated VINS-Mono real-time on the original Jetson Nano (much weaker than Orin Nano Super) for MAV state estimation; a 2024 arXiv paper (2406.13345) showed an enhanced VINS-Mono variant achieving 50 FPS on a Raspberry Pi CM4 with on-sensor accelerated optical flow. **License**: GPL-3.0 (copyleft viral) — distribution of the onboard binary requires source disclosure for the entire linked binary and triggers GPL-3 anti-tivoization clauses for embedded firmware.
- **Source**: Source #43 (canonical), Source #46 (KAIST Jetson benchmark), Source #43-linked LICENCE for license confirmation
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer
- **Confidence**: ✅ for algorithm class, mode support, and Jetson Nano feasibility; ⚠️ for Jetson Orin Nano Super specific latency (no direct measurement — but Orin Nano Super >> Jetson Nano, so feasibility is virtually certain); ⚠️ for the maintenance-status risk implied by ~24-month-old master branch.
- **Related Dimension**: SQ3+SQ4 / C1 Established-production candidate
- **Fit Impact**: **carry as lead candidate, conditional on user license decision.** Algorithmic fit is excellent (canonical mono+IMU VIO with metric scale and covariance); maintenance status is borderline; **GPL-3.0 license is a project-level decision required from the user** before this candidate can be marked Selected — see "C1 Open Decisions" section below.
### Fact #29 — VINS-Fusion is a multi-sensor superset of VINS-Mono but its monocular+IMU mode failed to run on Jetson TX2 in a 2021 KAIST benchmark; Orin Nano Super feasibility unverified
- **Statement**: VINS-Fusion (Qin, Cao, Pan, Shen — extension of VINS-Mono) supports four documented sensor configurations: stereo+IMU, mono+IMU, stereo only, +GPS-fusion (toy example). KITTI Odometry top-ranked open-source stereo algorithm as of January 2019. **Repository state**: 4,476 stars; last update 2024-05-23; same master-branch-only convention. **Jetson record**: KAIST 2021 benchmark (Source #46) — on Jetson TX2, both **VINS-Fusion (CPU) and VINS-Fusion-imu fail to run** due to insufficient memory and CPU; VINS-Fusion-gpu (GPU-accelerated front-end) runs on TX2. Orin Nano Super has more memory than TX2 (8 GB LPDDR5 shared vs TX2's 8 GB LPDDR4 shared) and stronger CPU/GPU, but the project's onboard stack is *co-resident* with C2 VPR + C3 matcher + C5 estimator + C6 cache → memory-pressure on the VINS-Fusion-imu path is plausible. **License**: GPL-3.0, same dual-use distribution constraint as VINS-Mono.
- **Source**: Source #44 (canonical), Source #46 (KAIST Jetson benchmark)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer
- **Confidence**: ✅ for the multi-sensor mode support and KITTI ranking; ✅ for the 2021 TX2 failure-to-run finding; ⚠️ for Orin Nano Super viability (between TX2 and Xavier NX in CPU/memory; not yet measured).
- **Related Dimension**: SQ3+SQ4 / C1 Open-source candidate
- **Fit Impact**: **carry as alternate candidate, with mandatory Jetson Orin Nano Super MVE before promotion.** VINS-Mono's narrower scope (mono+IMU only, no stereo overhead) makes VINS-Mono the preferred lead within the HKUST-Aerial-Robotics family; VINS-Fusion's multi-sensor coverage is a distractor for our pinned mode. **GPL-3.0 license decision is the same as VINS-Mono** — see "C1 Open Decisions".
### Fact #30 — OpenVINS is the most actively maintained MSCKF-class VIO and runs on Jetson Orin Nano Dev Kit + JetPack 6 + ROS 2 Humble with documented build adjustments; latency 270 ms on Xavier NX needs Orin-Nano-Super MVE
- **Statement**: OpenVINS (rpng, U. Delaware — Geneva, Eckenhoff, Lee, Yang, Huang — ICRA 2020) is a modular MSCKF (Multi-State Constraint Kalman Filter) implementation that fuses IMU state with sparse visual feature tracks via the Mourikis-Roumeliotis 2007 sliding-window MSCKF. **Mode support**: monocular, stereo, multi-camera (1N) + IMU; mono+IMU is a documented first-class configuration. Supports SLAM features (in-state landmarks) plus pure MSCKF features. **Jetson Orin Nano evidence**: rpng/open_vins issue #421 (Genozen, Feb 2024, closed) confirms OpenVINS ROS 2 builds on Jetson Orin Nano Dev Kit + JetPack 6 + Ubuntu 22.04 + ROS 2 Humble after one build patch (`#include <opencv2/aruco.hpp>` with newer OpenCV); fdcl-gwu/openvins_jetson_realsense (Nov 2025) provides a complete setup guide for Jetson Orin Nano + Intel RealSense + librealsense compiled-from-source + `--parallel-workers 1` build to avoid memory issues. **Latency record**: rpng/open_vins issue #164 — ~270 ms latency on Jetson Xavier NX (4 cores, 40% CPU utilisation). Recommended optimisations: subscriber queue size 1, Release builds with ARM-specific optimization flags (e.g., `armv8.2-a`), reduced camera resolution, prefer `odometry` topic over `pose_imu`. **License**: GPL-3.0, same dual-use distribution constraint as VINS-Mono / VINS-Fusion. Stars 2,828; 30 contributors; 12 releases; latest tag v2.7 (June 2023) but master branch active through 20242025 issue threads.
- **Source**: Source #45 (canonical + LICENSE + docs.openvins.com), Source #46 (KAIST Jetson benchmark for class-level CPU/memory profile), agent-tools record `29ebf728...txt` (Jetson Orin Nano build evidence)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer
- **Confidence**: ✅ for mode support, MSCKF formulation, and Jetson Orin Nano build feasibility; ⚠️ for steady-state latency on Orin Nano Super under our 5472×3648 nav frames — KAIST benchmark used 640×480; 16× pixel count is a yellow-flag.
- **Related Dimension**: SQ3+SQ4 / C1 Established-production candidate
- **Fit Impact**: **carry as lead candidate, conditional on user license decision.** OpenVINS has the most documented Jetson-Orin-Nano build path of the three GPL-3.0 candidates; MSCKF formulation is more memory-efficient than VINS-Mono's full sliding-window optimisation, which is a meaningful advantage under co-resident-process memory pressure. **GPL-3.0 license decision is the same as VINS-Mono / VINS-Fusion**.
### Fact #31 — OKVIS2 is the most actively maintained VI-SLAM in the BSD-permissive license bucket; OKVIS2-X (T-RO 2025) extends it with optional GNSS fusion that is architecturally aligned with the project's spoof-promotion path
- **Statement**: OKVIS2 (Leutenegger — arXiv 2022, ETH/Imperial/TUM Smart Robotics Lab) is a factor-graph VI-SLAM with bounded-size optimization. Algorithmic novelty: pose-graph edges from marginalised observations are "seamlessly turned back into observations" upon loop closure, reviving old landmarks and reprojection errors. Includes lightweight CNN segmentation for dynamic-region removal. **Mode support**: monocular and multi-camera + IMU; mono+IMU is a documented first-class configuration. **Successor OKVIS2-X (Boche, Jung, Laina, Leutenegger — IEEE T-RO 2025 vol 41 pp 60646083, DOI 10.1109/TRO.2025.3619051; arXiv 2510.04612, Oct 2025)** generalises the core to fuse multi-camera + IMU + optional GNSS receiver + LiDAR or depth. The OKVIS2-X GNSS-fusion mode (lineage: Visual-Inertial SLAM with Tightly-Coupled Dropout-Tolerant GPS Fusion, IROS 2022) directly mirrors the project's "VIO that may opportunistically fuse a non-spoofed GPS update when promotion completes" pattern (AC-NEW-2). **Repository state**: ethz-mrl/OKVIS2-X created 2025-09-23, last push 2026-03-17, 295 stars, 2 active contributors (bochsim, SebsBarbas). **License**: 3-clause BSD on the LICENSE file (GitHub UI shows "Other (NOASSERTION)" but the file is canonical 3-clause BSD per ASL-ETH Zurich convention) — permissive, no dual-use distribution friction.
- **Source**: Source #47 (OKVIS2 canonical), Source #48 (OKVIS2-X T-RO 2025)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 / C5 implementer
- **Confidence**: ✅ for algorithm, mode support, license, T-RO 2025 publication, repository activity; ⚠️ for Jetson Orin Nano runtime — no direct Jetson Orin Nano benchmark located; OKVIS2's factor-graph backend is plausibly heavier than OpenVINS' MSCKF on memory but lighter than Kimera (Kimera also produces a 3D mesh + semantic mesher, OKVIS2 does not).
- **Related Dimension**: SQ3+SQ4 / C1 Open-source-permissive lead candidate; potential C1+C5+C8 unified factor-graph design
- **Fit Impact**: **strong lead candidate by license + maintenance + GNSS-fusion alignment.** If license permissiveness is a priority, OKVIS2 + OKVIS2-X is the natural choice. The OKVIS2-X factor-graph also opens a design path where C5 (state estimator) collapses INTO C1 (the same factor graph absorbs sat-anchor measurements as constraints) — would simplify the pipeline at the cost of departing from the C1/C5 split, which is a Step-7.5 / `solution_draft01` design decision, not a SQ3+SQ4 question. **Pending Jetson Orin Nano Super MVE.**
### Fact #32 — Kimera-VIO is BSD-permissive but resource-heavy; KAIST benchmark found Kimera had the highest memory usage among VIOs tested and failed Xavier-NX-class memory under multi-process load
- **Statement**: Kimera-VIO (MIT-SPARK — Rosinol, Abate, Chang, Carlone — ICRA 2020) is a VI-SLAM pipeline with frontend + backend (factor-graph optimization in iSAM2 or GTSAM) + 3D mesher + pose-graph optimizer. Mode support: stereo+IMU primary, mono+IMU optional but documented. **License**: BSD 2-Clause "Simplified" (LICENSE.BSD on the repo) — permissive. **Maintenance**: active issue/PR threads through Dec 2024 / Feb 2025 covering ROS 2 integration, mono-inertial discussion, dependency management. **Resource profile** (Source #46 KAIST 2021 benchmark): Kimera had the highest memory usage among the 9 algorithms tested (numerous computations per keyframe); Kimera failed to fit on Xavier NX-class memory under sustained multi-process load. The 3D mesh + semantic-label outputs are unused by the project's narrow C1 mandate (relative 6-DoF + covariance only) — Kimera's overhead is unjustified vs OKVIS2 / OpenVINS for our use case.
- **Source**: Source #49 (Kimera canonical + LICENSE.BSD), Source #46 (KAIST Jetson benchmark)
- **Phase**: Phase 2
- **Target Audience**: System architects (build-vs-buy, mesh-feature decision)
- **Confidence**: ✅ for algorithm, license, maintenance status; ✅ for the Source #46 finding (KAIST 2021); ⚠️ for whether Orin Nano Super's larger memory + Ampere GPU lifts Kimera into feasibility — the Source-46 failure was on Xavier NX 8 GB shared, same memory budget as Orin Nano Super, but Orin Nano Super has higher per-core throughput.
- **Related Dimension**: SQ3+SQ4 / C1 Open-source-permissive secondary candidate
- **Fit Impact**: **carry as fallback only, not lead.** Kimera's permissive license is attractive but its resource overhead (especially the unused 3D mesh + semantic mesher) is a poor fit under co-resident process pressure. Use as a conservative secondary fallback if OKVIS2 unexpectedly fails Jetson MVE. **Status**: not lead.
### Fact #33 — DROID-SLAM is disqualified by AC-4.2: ≥11 GB GPU VRAM inference budget exceeds the project's 8 GB shared LPDDR5; further, DROID-SLAM is monocular VO/SLAM without IMU fusion and would require an external metric-scale wrapper
- **Statement**: DROID-SLAM (princeton-vl, Teed & Deng — NeurIPS 2021; arXiv 2108.10869) requires ≥11 GB GPU memory to run inference per the official README; training requires ≥24 GB on 4× RTX 3090. Issue #121 confirms that even with 128 GB system RAM and 16 GB VRAM (RTX 4080), users hit very large RAM consumption quickly. Algorithmically, DROID-SLAM is **monocular VO/SLAM** with recurrent dense bundle adjustment over a complete history of camera poses — no native IMU fusion; output pose is in arbitrary scale (no metric scale recovery without external alignment). DPV-SLAM (ECCV 2024, princeton-vl) is the lighter successor at ~45 GB GPU memory; DPVO (NeurIPS 2023, princeton-vl) is even lighter at ~3 GB, but neither natively integrates IMU.
- **Source**: Source #50 (DROID-SLAM canonical), Source #51 (DPVO / DPV-SLAM successor), Source #52 (DPVO-QAT++ memory measurement)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer
- **Confidence**: ✅
- **Related Dimension**: SQ3+SQ4 / C1 disqualified candidate
- **Fit Impact**: **DISQUALIFIED outright.** AC-4.2 sets the 8 GB shared CPU+GPU memory budget; DROID-SLAM's ≥11 GB GPU-only requirement violates it before adding co-resident C2/C3/C5/C6 processes. Cite as "what the project cannot afford" in `solution_draft01` to pre-empt obvious questions.
### Fact #34 — DPVO is monocular VO only (no IMU fusion); it can fit a Jetson-suitable memory footprint with QAT but cannot satisfy the C1 VIO mandate alone — would need an external IMU + metric-scale wrapper
- **Statement**: DPVO (Teed, Lipson, Deng — NeurIPS 2023; ECCV 2024 DPV-SLAM successor) is a deep-learning monocular VO with sparse patch tracking + differentiable bundle adjustment. **Mode**: monocular VO only — no IMU fusion in the published paper or repository; output pose is in arbitrary scale. Memory footprint: DPVO ~3 GB GPU, DPV-SLAM ~45 GB GPU on standard hardware; DPVO-QAT++ (arXiv 2511.12653, Cheng Liao, Nov 2025) reduces peak reserved memory to 1.02 GB on RTX 4060 (8 GB) via fused-CUDA INT8 fake-quantization while preserving ATE on TartanAir/EuRoC. **License**: MIT (permissive). Repository: 989 stars; last update 2024-10-12. **Crucial gap**: DPVO does NOT meet the C1 mandate of a "VIO that produces metric-scale 6-DoF + attitude with σ ≤ 5°" — for the project to use DPVO as the *VO half* of C1, an additional IMU+scale-fusion module (loosely-coupled ESKF with VO velocity / displacement priors) must be designed; alternatively, DPVO's pose can feed C5 directly as a relative-displacement constraint, with attitude served separately by FC IMU integration. **Jetson Orin Nano runtime evidence**: indirect — DPVO-QAT++ benchmarks on RTX 4060 desktop, NOT Jetson Orin Nano. The Ampere GPU architecture is shared between RTX 4060 and Orin Nano Super (both Ampere); the Orin Nano Super's GPU is smaller, so direct extrapolation is not safe — Jetson MVE required.
- **Source**: Source #51 (DPVO / DPV-SLAM canonical), Source #52 (DPVO-QAT++ Nov 2025)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 / C5 implementer
- **Confidence**: ✅ for "VO only, no IMU fusion" and the memory footprints; ⚠️ for Jetson Orin Nano direct runtime (no measurement); ⚠️ for the operational complexity of the QAT pipeline (teacher-student distillation training is a significant prerequisite vs the classical VINS-* / OpenVINS / OKVIS2 candidates).
- **Related Dimension**: SQ3+SQ4 / C1 conditional candidate (VO not VIO; needs external IMU wrapper)
- **Fit Impact**: **NOT a drop-in C1 candidate; conditional fit only.** DPVO is **not** a substitute for VINS-Mono / OpenVINS / OKVIS2 — it is a candidate for the *VO half* of a hybrid design where C5 (estimator) absorbs IMU and DPVO provides relative-pose priors. This adds design complexity and is **not preferred** unless one of the established VIO candidates fails Jetson MVE for memory reasons. **Status**: secondary, conditional.
### Fact #35 — Pure VO baseline (KLT optical flow + 5-point essential matrix or homography RANSAC) is the project's mandatory simple-baseline candidate and is the de-facto fallback when learning-based methods fail on Jetson-budget constraints
- **Statement**: The classical pipeline — Shi-Tomasi or FAST corner detection → KLT pyramidal optical flow tracking (`cv::calcOpticalFlowPyrLK`) → 5-point essential matrix (Nister, `cv::findEssentialMat`) or homography RANSAC (`cv::findHomography`) → relative pose with arbitrary scale → metric-scale alignment via IMU integration externally — is the foundational visual-odometry pipeline implemented in OpenCV samples and pedagogical repositories. For the project's nadir-down UAV at 1 km AGL over Ukrainian steppe (predominantly planar terrain, low relief), the **homography path is geometrically appropriate** (a plane induces a homography between two views); for non-planar relief, the **essential-matrix path is appropriate** at a small overhead. License: public domain / OpenCV-Apache-2.0 / MIT (whatever reference implementation is chosen) — permissive. Reference: representative public Monocular-Video-Odometery (MIT, alishobeiri 2018), Monocular-Visual-Odometry (Yacynte) at translation error 0.94% / rotation error 0.015°/m on KITTI dataset.
- **Source**: Source #53 (OpenCV docs + reference implementations)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer + risk reviewer
- **Confidence**: ✅
- **Related Dimension**: SQ3+SQ4 / C1 Simple-baseline candidate (mandatory per Component Option Breadth rule)
- **Fit Impact**: **carry as the project's `Simple baseline / known-runnable / known-failure-mode` C1 fallback.** Not a lead, but mandatory presence. Failure modes: (a) low-texture cropland / snow → KLT track loss; (b) sharp turns → low-overlap homography degeneracy; (c) no native IMU fusion → must wrap with external metric-scale alignment (same wrapper as DPVO). **Status**: simple-baseline reference; cited in `solution_draft01` to anchor the failure analysis.
### Fact #36 — Step-0.5-time-window assessment: VINS-Mono / VINS-Fusion master branches are at the Critical-novelty 18-month boundary; OpenVINS and OKVIS2 are within window; DPVO is borderline; the established baselines (KLT + RANSAC) are exempt
- **Statement**: Per Step 0.5 timeliness assessment in `00_question_decomposition.md`, Critical-novelty topics require sources within 6 months for SOTA claims and 18 months for established libraries' API behaviour. Audit at access time 2026-05-07: VINS-Mono master last meaningful commit 2024-02-25 → ~27 months → **just over the 18-month window**; VINS-Fusion 2024-05-23 → ~24 months → just over; OpenVINS master active (issue threads through Feb 2025) and v2.7 release June 2023 → ~35 months for the tagged release but master in stable maintenance → within de-facto window for an established library; OKVIS2-X push 2026-03-17 → ~2 months → **fully within window**; DPVO last code update 2024-10-12 → ~19 months → just over but DPV-SLAM ECCV 2024 keeps the algorithm class within 6-month claim window; KLT / 5-point / RANSAC / homography → established baselines per Step 0.5 → **no time window applies**. **Implication**: VINS-Mono / VINS-Fusion fall into the "older than 18 months but classical authoritative reference" bucket — Step 0.5 allows up to 18 months strictly, but downstream forks (vins-mono-android, embedded variants) and the IEEE T-RO 2018 publication keep the algorithm class in active community use. Recommended treatment: **keep as candidates but require live MVE on Jetson Orin Nano Super before promotion to Selected**, to revalidate against the current OpenCV / Ceres / ROS 2 stack.
- **Source**: Source #43, Source #44, Source #45, Source #47, Source #48, Source #51 (timeliness audit per source)
- **Phase**: Phase 2
- **Target Audience**: Step-7.5 reviewer + System architects
- **Confidence**: ✅
- **Related Dimension**: SQ3+SQ4 / C1 candidate-pool integrity
- **Fit Impact**: **applies a conservative timeliness gate: every C1 candidate from VINS-Mono / VINS-Fusion / DPVO requires an Orin-Nano-Super MVE before being marked Selected**, since their master-branch staleness pushes them out of the Critical-novelty 18-month window. OpenVINS / OKVIS2 / OKVIS2-X / Kimera are within window via active issue threads or recent releases.
### C1 Component Applicability Gate — preliminary table (this session; structured Restrictions×AC sub-matrix per candidate is next session's work)
| Candidate | Mode (project) | License | Active maintenance? | Jetson Orin Nano Super runnable? | Native IMU fusion? | Native metric scale? | License blocks dual-use? | Preliminary status |
|---|---|---|---|---|---|---|---|---|
| **VINS-Mono** | mono+IMU | GPL-3.0 (copyleft) | ⚠️ borderline (24 mo) | ✅ proven on Jetson Nano (2021) → Orin Nano Super virtually certain | ✅ | ✅ | **⚠️ Verify with user** | Lead candidate **conditional on user license decision** + Orin-Nano-Super MVE |
| **VINS-Fusion** | mono+IMU (mode) | GPL-3.0 | ⚠️ borderline (24 mo) | ⚠️ failed on TX2 (KAIST 2021); Orin Nano Super untested | ✅ | ✅ | **⚠️ Verify with user** | Alternate, secondary to VINS-Mono within HKUST family |
| **OpenVINS** | mono+IMU | GPL-3.0 | ✅ active master | ✅ build confirmed on Orin Nano Dev Kit + JetPack 6 (2024 + 2025 community evidence); ~270 ms latency on Xavier NX | ✅ MSCKF | ✅ | **⚠️ Verify with user** | **Lead candidate** **conditional on user license decision** (best Jetson-Orin-Nano evidence + most maintained of the GPL-3 trio) |
| **OKVIS2 / OKVIS2-X** | mono+IMU (+ optional GNSS) | BSD-3 | ✅ very active (2026 pushes) | ⚠️ no direct Jetson Orin Nano measurement; factor-graph backbone plausibly heavier than MSCKF | ✅ | ✅ | ✅ no | **Lead candidate by license + maintenance + spoof-promotion architectural alignment**, pending Jetson MVE |
| **Kimera-VIO** | mono+IMU (optional) | BSD-2 | ✅ active | ⚠️ failed on Xavier NX 8 GB shared under multi-process (KAIST 2021) | ✅ | ✅ | ✅ no | Fallback secondary; resource overhead poor fit for project |
| **DROID-SLAM** | mono VO/SLAM only | (project repo) | reference baseline | ❌ ≥11 GB GPU VRAM > 8 GB AC-4.2 budget | ❌ | ❌ (arbitrary scale) | n/a | **DISQUALIFIED** by AC-4.2 |
| **DPVO / DPV-SLAM** | mono VO only | MIT | ⚠️ borderline (19 mo on code, ECCV 2024 paper) | ⚠️ DPVO-QAT++ (Nov 2025) shows 1.02 GB peak on RTX 4060 desktop; Jetson Orin Nano untested | ❌ (needs external IMU wrapper) | ❌ (needs external scale alignment) | ✅ no | Conditional secondary — VO half of a hybrid C1+C5 design only; not a drop-in VIO replacement |
| **Pure VO baseline (KLT + 5pt RANSAC / homography)** | mono VO only | OpenCV-Apache-2.0 / MIT | ✅ foundational (no time window) | ✅ runs on any Jetson | ❌ (needs external IMU wrapper) | ❌ (needs external scale alignment) | ✅ no | **Mandatory simple-baseline reference** per Component Option Breadth rule |
**Surviving lead candidates (preliminary)**, in priority order based on this session's evidence:
1. **OpenVINS** (GPL-3.0, MSCKF, best Jetson Orin Nano evidence) — pending user license decision + Orin-Nano-Super MVE
2. **OKVIS2 / OKVIS2-X** (BSD-3, factor-graph + GNSS-fusion alignment, most active maintenance) — pending Jetson MVE
3. **VINS-Mono** (GPL-3.0, sliding-window optimization, proven on Jetson Nano) — pending user license decision + Orin-Nano-Super MVE
4. **Pure VO baseline** (mandatory simple-baseline; runtime guaranteed; carries the project as a graceful fallback)
**Disqualified outright**: DROID-SLAM (AC-4.2 memory budget), RTAB-Map and ORB-SLAM3 (already pruned by Fact #16).
**Conditional / not-direct-fit**: DPVO / DPV-SLAM (VO not VIO, needs external IMU wrapper), Kimera-VIO (resource overhead unjustified for narrow C1 mandate).
### C1 Open Decisions (to be resolved before SQ3+SQ4 closure)
**Decision D-C1-1 — GPL-3.0 license posture for the onboard binary** (BLOCKING for the GPL-3.0 trio: VINS-Mono / VINS-Fusion / OpenVINS).
- The three most established VIO candidates (VINS-Mono / VINS-Fusion / OpenVINS) are GPL-3.0 (viral copyleft).
- For dual-use UAV deployment, GPL-3 binary distribution to a customer triggers obligations: source-code disclosure for the entire linked binary, anti-tivoization clauses for embedded firmware updates, viral effect on any proprietary code linked into the same binary.
- BSD/MIT alternatives exist (OKVIS2 BSD-3, Kimera BSD-2, DPVO MIT, pure-VO baseline OpenCV-Apache-2.0), but each comes with secondary trade-offs (Jetson MVE risk, missing IMU fusion, resource overhead).
- Three options for the user:
- **(a)** Accept GPL-3.0 — distribution model = release source on customer request; or operate the system as a service rather than transferring binaries. Lowest-risk algorithmic path (most-tested candidates).
- **(b)** Restrict to permissive licenses only (BSD/MIT) — lead candidate becomes OKVIS2; carries Jetson MVE risk.
- **(c)** Keep both options open through the design phase — make the final license decision after the Jetson Orin Nano MVE results are in.
- **Recommended default**: **(c)** — defer the binary commitment until empirical evidence on Jetson Orin Nano. This is recorded as a flagged decision; SQ3+SQ4 candidate matrix will carry both license families to Step 7.5.
**Decision D-C1-2 — Acceptance of Jetson Orin Nano MVE as a Step-7.5 prerequisite** (procedural).
- Per the Per-Mode API Capability Verification rule, every lead candidate library/SDK requires `context7` (or equivalent docs) lookup + a Minimum Viable Example for the project's pinned mode + per-numbered-Restriction × per-numbered-AC sub-matrix.
- The Component Applicability Gate above is **preliminary** — it documents enumeration evidence but does NOT yet contain `context7` per-mode capability verification or the structured sub-matrix.
- **Next session's mandatory work**: `context7` lookup (3 mandatory queries) for OpenVINS / OKVIS2 / VINS-Mono; per-Restriction × per-AC sub-matrix per candidate; the same for the simple-baseline path; record into `../02_fact_cards/C1_vio.md` per the engine template + `../06_component_fit_matrix/C1_vio.md` per Step 7.5.
### C1 Boundary check: candidate enumeration is saturated for this session
Saturation signals observed: (a) all 7 named candidates from `00_question_decomposition.md` C1 row enumerated with at least one canonical L1 source per candidate; (b) Jetson Orin Nano runtime evidence located for OpenVINS (direct) and VINS-Mono (Jetson Nano + RPi CM4); other candidates carry "MVE required" gates explicitly; (c) license diversity covered (GPL-3.0 trio + BSD-permissive duo + MIT + permissive-baseline); (d) explicit disqualifications recorded with cited evidence (DROID-SLAM, RTAB-Map, ORB-SLAM3). **Open**: per-mode `context7` verification (BLOCKING per rule) + Restrictions×AC sub-matrices (BLOCKING per Step 7.5) — explicitly deferred to next session.
---
## C1 — Per-Mode API Capability Verification (engine Step 2 — Mandatory `context7` lookup) [2026-05-08 session]
This section closes the per-mode API capability verification gate for the four C1 lead candidates. Each candidate has a pinned-mode statement, three documentary `context7` (or equivalent) queries answered, an MVE block, and a per-numbered-Restriction × per-numbered-AC sub-matrix. The candidates' final lead-promotion to "Selected" status remains gated by the dedicated Jetson Orin Nano Super hardware MVE (D-C1-2 deferred phase).
### Fact #37 — OpenVINS per-mode API capability verification (mono+IMU on Jetson Orin Nano Super) — DOCUMENTARY PASS; Jetson MVE pending
- **Statement**: OpenVINS (`/rpng/open_vins`, master) exposes monocular / stereo / multi-camera + IMU as first-class launch configurations via `subscribe.launch.py` declared launch arguments `use_stereo` (bool) and `max_cameras` (int). The project's **pinned mode** is monocular + IMU, selected via `use_stereo:=false max_cameras:=1` with `config:=` pointing to a project-tuned `estimator_config.yaml`. **Mode-enumeration query (1/3)**: confirms 3 sensor configurations at the launch layer; supported IMU intrinsic models = KALIBR + RPNG (per `propagation-analytical.dox`). **Pinned-mode runnable example query (2/3)**: confirms `ros2 launch ov_msckf subscribe.launch.py config:=euroc_mav` is the documented runnable example; `euroc_mav` defaults to stereo per `subscribe.launch.py` but `use_stereo:=false max_cameras:=1` selects mono-only at runtime — no source patch required. **Disqualifier-probe query (3/3)**: did NOT surface any documented sub-20-Hz validation, hard frame-rate floor, or hard image-resolution ceiling in the master docs; the documented Xavier-NX latency baseline (~270 ms per rpng/open_vins issue #164) is below the AC-4.1 400 ms p95 budget head-room **at 640×480** but unverified at the project's 5472×3648 nav frames. The Jetson Orin Nano Dev Kit + JetPack 6 + ROS 2 Humble build patch is documented (rpng/open_vins issue #421 + fdcl-gwu/openvins_jetson_realsense). **Pinned-mode sentence**: "We will use **OpenVINS** in **monocular + IMU mode** with inputs `{1× ADTi 20MP nav frame stream + FC IMU via MAVLink/SCALED_IMU2}` and expect outputs `{6-DoF pose at IMU rate with covariance from MSCKF state, source label visual_propagated when no satellite anchor}` on `Jetson Orin Nano Super (8 GB shared, JetPack 6, ROS 2 Humble)`."
- **Source**: Source #54 (context7), Source #45 (canonical OpenVINS), Source #46 (KAIST Jetson benchmark for class-level comparison)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer + Step-7.5 reviewer
- **Confidence**: ✅ for mode-enumeration and runnable-example documentary evidence; ⚠️ for sub-20-Hz validation and 5472×3648 latency (no documentary evidence — Jetson MVE will resolve)
- **Related Dimension**: SQ3+SQ4 / C1 lead candidate — per-mode API capability verification gate
- **Fit Impact**: **DOCUMENTARY PASS for the per-mode API capability verification gate**; promotes OpenVINS to "lead candidate, documentary verification complete" status in `../06_component_fit_matrix/C1_vio.md` row. License-track decision (D-C1-1) still gates final Selected promotion (OpenVINS = GPL-3.0, lives in track A); Jetson Orin Nano Super hardware MVE (D-C1-2) still gates accuracy/latency/memory empirical promotion.
### Fact #38 — VINS-Mono per-mode API capability verification (mono+IMU on Jetson Orin Nano Super) — DOCUMENTARY PASS WITH FRAME-RATE CAVEAT; Jetson MVE pending
- **Statement**: VINS-Mono (`HKUST-Aerial-Robotics/VINS-Mono`, master) is a single-mode system: "real-time SLAM framework for **Monocular Visual-Inertial Systems**" (README §1) — no mode enumeration is required because the pinned mode IS the only mode. **Mode-enumeration query (1/3)**: VINS-Mono is single-mode = mono+IMU; cross-source documentary evidence from VINS-Fusion `context7` confirms the same authors continue to ship `euroc_mono_imu_config.yaml` as a first-class config in the active fork (per the Per-Mode API rule, VINS-Fusion's mono+IMU mode is a separately-cataloged candidate, but the algorithmic core and required calibration surface are identical — see Fact #29). **Pinned-mode runnable example query (2/3)**: README §3.1.1 — `roslaunch vins_estimator euroc.launch` + EuRoC MH_01 bag is the canonical runnable example; supports online camera-IMU extrinsic calibration (`estimate_extrinsic:=2`), online temporal calibration (`estimate_td:=1`), and rolling-shutter cameras with documented calibration ceiling (`reprojection error <0.5 px`). Pinhole + MEI camera models supported. Camera intrinsics + IMU noise must be calibrated (Kalibr or equivalent). **Disqualifier-probe query (3/3)**: README §5.1 explicitly states *"The image should exceed 20Hz and IMU should exceed 100Hz."* — this is a documentary minimum-rate recommendation and is **below the project's 3 fps nav-camera target by ~6.7×**. See Fact #40 for the geometric analysis and the cross-cutting frame-rate-sensitivity finding. Ceres Solver dependency is pinned to v1.14.0 (build issues at ≥2.0.0 per README §1.2); JetPack-shipped Ceres versions need explicit verification. License: GPLv3 (README §8). **Pinned-mode sentence**: "We will use **VINS-Mono** in **monocular + IMU mode** with inputs `{1× ADTi 20MP nav frame stream (target 3 fps; under documentary 20 Hz floor) + FC IMU via MAVLink/SCALED_IMU2}` and expect outputs `{6-DoF pose at IMU rate via sliding-window optimization with covariance from optimization Hessian, loop closure via DBoW2}` on `Jetson Orin Nano Super (8 GB shared, JetPack 6, Ceres v1.14.0 build)`."
- **Source**: Source #55 (VINS-Mono README + VINS-Fusion context7 cross-source), Source #43 (canonical VINS-Mono), Source #46 (KAIST Jetson benchmark for class-level comparison)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer + Step-7.5 reviewer
- **Confidence**: ✅ for mode-enumeration (single mode by construction) and runnable-example evidence; ⚠️ for sub-20-Hz operation (documentary minimum-rate recommendation contradicts project frame-rate target); ⚠️ for Ceres v1.14.0 vs JetPack 6 stock Ceres compatibility
- **Related Dimension**: SQ3+SQ4 / C1 lead candidate — per-mode API capability verification gate
- **Fit Impact**: **DOCUMENTARY PASS WITH FRAME-RATE CAVEAT**. Per the engine rule's escalation tier, the candidate is downgraded from "documentary lead" to **"Experimental only — sub-20-Hz operation requires Jetson MVE validation"** until the deferred Jetson hardware MVE explicitly measures VINS-Mono at the project's 3 fps. License-track decision (D-C1-1) still gates final Selected promotion (VINS-Mono = GPL-3.0, lives in track A).
### Fact #39 — OKVIS2 per-mode API capability verification (mono+IMU on Jetson Orin Nano Super) — DOCUMENTARY PASS; Jetson MVE pending
- **Statement**: OKVIS2 (`smartroboticslab/okvis2`, main) is a keyframe-based factor-graph VI-SLAM with multi-camera + IMU support; the README documents coordinate-frame contract (`W` world / `C_i` cameras / `S` IMU / `B` body), state representation (`T_WS` pose + velocity + gyro/accel biases), and a two-callback API (`setOptimisedGraphCallback` for batch updates incl. loop closure + `setImuCallback` for high-rate prediction). **Mode-enumeration query (1/3)**: README + example apps confirm modes = mono / stereo / multi-camera (i-th camera frame `C_i`) — IMU is mandatory (`okvis::ViSensorBase::setImuCallback` is required). The example apps are `okvis_app_synchronous` (dataset replay), `okvis_app_realsense` (live D435i/D455), `okvis_app_realsense_record` (recording). ROS 2 build is opt-in (`BUILD_ROS2=ON`); ROS 2 launch files: `okvis_node_realsense.launch.xml`, `okvis_node_realsense_publisher.launch.xml`, `okvis_node_subscriber.launch.xml`, `okvis_node_synchronous.launch.xml`. **Pinned-mode runnable example query (2/3)**: README "Running the demo application" + "Configuration files" section — `./okvis_app_synchronous <config>.yaml <EuRoC_MH_01_easy_dir>` is the canonical mono dataset-replay example; the EuRoC config in `config/` is the documentary mono+IMU launch reference. Configuration trade-off surface: "various options to trade-off accuracy and computational expense as well as to enable online calibration" — explicit acknowledgement of latency/accuracy tuning surface. **Disqualifier-probe query (3/3)**: README does NOT state an explicit minimum image rate (cf. VINS-Mono's 20 Hz). OKVIS2's keyframe-based architecture inherently selects only "informative" frames for optimization, which is a structural advantage at lower input frame rates compared to sliding-window optimization. Optional LibTorch sky-segmentation CNN (`USE_NN`) can be disabled with `USE_NN=OFF` to remove the Jetson LibTorch dependency. License: 3-clause BSD (README "License" section). Health warning: "good results (or results at all) may only be obtained with appropriate calibration" — Kalibr-based intrinsic + extrinsic + IMU noise + tight time sync mandatory (this is shared with all VI candidates). OKVIS2-X (T-RO 2025) extends with optional GNSS fusion — architecturally aligned with the project's spoof-promotion path (per Fact #31). **Pinned-mode sentence**: "We will use **OKVIS2** (with `BUILD_ROS2=ON USE_NN=OFF`) in **monocular + IMU mode** with inputs `{1× ADTi 20MP nav frame stream + FC IMU via MAVLink/SCALED_IMU2 → re-published to /okvis/cam0/image_raw + /okvis/imu0}` and expect outputs `{6-DoF pose with covariance from factor-graph optimization via setOptimisedGraphCallback + high-rate IMU-predicted state via setImuCallback}` on `Jetson Orin Nano Super (8 GB shared, JetPack 6, ROS 2 Humble)`."
- **Source**: Source #56 (OKVIS2 README), Source #47 (canonical OKVIS2 paper arXiv:2202.09199), Source #48 (OKVIS2-X T-RO 2025)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer + Step-7.5 reviewer
- **Confidence**: ✅ for mode-enumeration, runnable-example, and lower-frame-rate-tolerance arguments; ⚠️ for direct 3 fps validation (no documentary measurement — Jetson MVE will resolve); ⚠️ for direct Jetson Orin Nano measurement (Fact #31 noted no direct measurement; community evidence less abundant than OpenVINS)
- **Related Dimension**: SQ3+SQ4 / C1 lead candidate — per-mode API capability verification gate
- **Fit Impact**: **DOCUMENTARY PASS for the per-mode API capability verification gate**; promotes OKVIS2 to "lead candidate, documentary verification complete" status in `../06_component_fit_matrix/C1_vio.md` row. OKVIS2's keyframe-based architecture is the **only candidate** of the four leads with a structural argument for tolerating sub-20-Hz operation — this re-orders the per-license-track lead ranking (see Fact #41 locked-in defaults). License-track decision (D-C1-1) does NOT gate OKVIS2 (BSD-3 already permissive); Jetson Orin Nano Super hardware MVE (D-C1-2) still gates empirical accuracy/latency/memory promotion.
### Fact #40 — Cross-cutting C1 finding: project's 3 fps nav-camera target is below VINS-Mono's documented 20 Hz minimum-rate recommendation; affects all sliding-window VIO candidates; OKVIS2's keyframe architecture is the structural mitigant
- **Statement**: VINS-Mono README §5.1 documents "The image should exceed 20Hz and IMU should exceed 100Hz" as the recommended minimum-rate operating envelope (Source #55). The project's nav-camera processing target is 3 fps per `00_question_decomposition.md` Project Constraint Matrix. **Geometric analysis**: at 60 km/h cruise = 16.7 m/s × (1/3 s) = 5.5 m of forward motion between consecutive nav frames; at 1 km AGL with 12 cm/px GSD, that motion projects to ~46 px of in-image displacement (~0.84% of the 5472 px frame width) — **well within KLT-trackable range** for the nadir-down camera geometry, so the rate floor is NOT geometrically unreachable. **However**: the documented recommendation is about temporal-stability assumptions (motion-blur tolerance, IMU pre-integration noise growth, sliding-window optimisation Jacobian conditioning), not about geometric trackability. **Cross-candidate impact**: (a) **VINS-Mono** — sliding-window optimisation, full graph re-linearisation per keyframe, 20 Hz documentary recommendation explicitly violated by 6.7× → ⚠️ Experimental only until Jetson MVE measures actual sub-20-Hz behaviour; (b) **VINS-Fusion** — same algorithmic core as VINS-Mono mono+IMU mode, same caveat applies; (c) **OpenVINS** — MSCKF-based with sliding-window state + sparse feature constraints, has documented variable-rate tolerance via `init_imu_thresh`/`init_window_time` config, but no documentary sub-20-Hz validation surfaced in `context7` queries → ⚠️ Verify via Jetson MVE; (d) **OKVIS2** — keyframe-based, structurally selects only informative frames for optimization; the architecture is more naturally tolerant of variable / lower input rates → preferred candidate at low input frame rates; ✅ structural argument; (e) **Pure VO baseline** (KLT+RANSAC) — requires sufficient feature overlap between consecutive frames; at 0.84% in-image displacement this is well within KLT capture range; ✅ no rate-floor concern. **Architectural alternative for design-phase consideration**: instead of binding all C1 candidates to 3 fps, the nav-camera input pipeline could fork — full-resolution 5472×3648 at 3 fps for VPR/satellite-anchor (C2/C3) and a binned/cropped 1368×912 (or 640×480) at higher rate (≥10 fps) into the VIO front-end. ADTi 20MP 20L V1 (APS-C) bandwidth at full-res caps near 57 fps over USB 3 (≈23 GB/s raw); binned modes typically 310× the rate. This is a Plan-time decision, not a research-time one, but the option must be carried into Plan and the Jetson MVE must measure both single-rate and dual-rate paths.
- **Source**: Source #55 (VINS-Mono README §5.1), Source #43 (canonical), restrictions.md "Cameras" section + `00_question_decomposition.md` Project Constraint Matrix (3 fps target)
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 implementer + Plan-phase reviewer + Jetson MVE owner
- **Confidence**: ✅ for the documentary 20 Hz minimum-rate recommendation; ✅ for geometric trackability calculation; ⚠️ for the binned/dual-rate pipeline option (camera-bandwidth estimate is plausible but needs ADTi datasheet verification at Plan time)
- **Related Dimension**: SQ3+SQ4 / C1 frame-rate sensitivity (cross-candidate); SQ4 (per-candidate runtime envelope binding)
- **Fit Impact**: **(a)** Re-orders the per-license-track candidate ranking — within the BSD/permissive track, OKVIS2 strengthens its lead via structural keyframe argument; within the GPL-3.0 track, OpenVINS retains lead over VINS-Mono on this specific dimension because MSCKF's variable-rate tolerance is more documented than VINS-Mono's full-window optimisation. **(b)** Adds a Plan-phase decision: **single-rate (3 fps to all consumers) vs dual-rate (binned high-rate to VIO + full-res 3 fps to VPR/satellite)** — this becomes an explicit deliverable for the Plan phase, not the Jetson MVE phase, because the nav-camera input pipeline shape feeds into both C1 and C2/C3 candidate scoring. **(c)** Marks all VINS-* candidates as ⚠️ Experimental-only until the deferred Jetson hardware MVE explicitly measures sub-20-Hz behaviour.
### Fact #41 — D-C1-1 + D-C1-2 locked-in research-time defaults (after user-skipped clarification, 2026-05-08)
- **Statement**: The user invoked `/autodev` and was presented with structured AskQuestion prompts for D-C1-1 (GPL-3.0 license posture) and D-C1-2 (Jetson MVE schedule); the user **skipped the questions with the directive "continue with the information you already have"**. Per autodev meta-rule "Critical Thinking" — locked-in research-time defaults selected to preserve maximum future optionality and to honour the documentary evidence already gathered: **D-C1-1 = (c) "Keep both license tracks open"** — rank GPL-3.0 leads (OpenVINS, VINS-Mono, VINS-Fusion) in parallel with BSD-permissive OKVIS2/OKVIS2-X; **carry both license tracks through Plan**; final license decision deferred to post-Jetson-MVE/Plan time when empirical evidence is available. **D-C1-2 = (b) "Defer Jetson MVE to a dedicated bring-up phase between research and Plan"** — research closes with documentary ranking + explicit "Jetson MVE pending" gates per candidate; the dedicated Jetson Orin Nano Super hardware MVE phase produces a single MVE artifact that promotes leads to "Selected" before Plan starts. The Plan phase MUST NOT lock a final C1 candidate before the deferred Jetson MVE artifact is produced and reviewed. **These defaults are explicitly tagged as user-deferred** — the user retains the right to revisit either decision at Plan time without losing the research artifact (both license tracks fully cataloged; both lead candidates carry full per-mode evidence).
- **Source**: User clarification skip during 2026-05-08 `/autodev` invocation; autodev meta-rule "Critical Thinking"; greenfield-flow Step 14 (Plan) precondition rule
- **Phase**: Phase 2 — process decision
- **Target Audience**: System architects + Plan-phase reviewer + Step-7.5 reviewer
- **Confidence**: ✅ (defaults selected and tagged as user-deferred; user can override at any later prompt)
- **Related Dimension**: SQ3+SQ4 / C1 process gate; cross-cutting onto C2C10 (license posture decision is project-wide, not C1-specific)
- **Fit Impact**: **PROCESS GATE CLOSURE for C1**. Allows research to proceed past C1 to C2 (VPR) candidate enumeration without requiring user input now. The Plan phase MUST surface D-C1-1 again as a structured A/B/C decision before any C1 candidate is locked, AND MUST require the deferred Jetson MVE artifact as a precondition.
---
## C1 — Minimum Viable Example (MVE) Blocks
### MVE — OpenVINS in monocular + IMU mode
- **Source**: Source #54 (context7 → `https://github.com/rpng/open_vins/blob/master/docs/gs-tutorial.dox` ROS 2 launch + `https://github.com/rpng/open_vins/blob/master/docs/gs-datasets.dox` EuRoC config), accessed 2026-05-08
- **Inputs in the example**: EuRoC MAV stereo VI dataset (default `config:=euroc_mav` is stereo 2× cameras + IMU); the launch file declares `use_stereo` (default `true`) and `max_cameras` (default `2`) as runtime overrides; setting `use_stereo:=false max_cameras:=1` selects monocular operation against the same `estimator_config.yaml` parameter file with ROS topics `/cam0/image_raw` + `/imu0`
- **Outputs in the example**: 6-DoF pose at IMU rate; ROS 1 publishes `/ov_msckf/poseimu`, `/ov_msckf/odomimu`, `/ov_msckf/pathimu`; ROS 2 publishes equivalent topics under the configured namespace
- **Project inputs**: 1× ADTi 20MP nav frame stream (5472×3648, target 3 fps) + FC IMU via MAVLink (SCALED_IMU2 at ≥100 Hz)
- **Project outputs required**: 6-DoF pose at IMU rate with metric scale + 6×6 covariance + source label `visual_propagated` when no satellite anchor; AC-1.4-compliant 95% covariance ellipse; honest covariance per AC-NEW-4
- **Match assessment**: ✅ exact mode match for **mono+IMU**; ⚠️ partial input shape (image-resolution 45× larger than EuRoC's 752×480 → latency/memory unverified at full resolution); ⚠️ partial input rate (3 fps vs EuRoC's 20 Hz — see Fact #40)
- **If ⚠️ or ❌**: docs do not explicitly disqualify the configuration. The launch surface (`use_stereo`, `max_cameras`, `config_path`) supports the project's mode without source patches. Resolution and rate are **runtime/Jetson-MVE concerns**, not API-mode concerns. → Status: **Documentary lead**; final promotion to "Selected" requires Jetson Orin Nano Super hardware MVE artifact (D-C1-2 deferred phase).
### MVE — VINS-Mono in monocular + IMU mode (single mode by construction)
- **Source**: Source #55 (VINS-Mono README §3.1.1 + cross-source VINS-Fusion `context7` `euroc_mono_imu_config.yaml`), accessed 2026-05-08
- **Inputs in the example**: EuRoC MAV monocular VI dataset (the README explicitly notes "Although it contains stereo cameras, we only use one camera"); ROS topics with image rate >20 Hz and IMU rate >100 Hz per README §5.1; pinhole or MEI camera model with intrinsics + distortion calibrated; camera-IMU extrinsic + temporal calibration optional (online estimation supported via `estimate_extrinsic` and `estimate_td` params)
- **Outputs in the example**: 6-DoF pose at IMU rate via sliding-window optimization with covariance from optimization Hessian; loop closure via DBoW2; pose-graph save/reuse via `s` keystroke
- **Project inputs**: 1× ADTi 20MP nav frame stream (5472×3648, target 3 fps — **below documentary 20 Hz floor**) + FC IMU via MAVLink (SCALED_IMU2 at ≥100 Hz)
- **Project outputs required**: same as OpenVINS MVE above
- **Match assessment**: ✅ exact mode match (single-mode system, the project's pinned mode IS the only mode); ⚠️ partial input rate (3 fps vs documentary 20 Hz minimum recommendation per Fact #40); ⚠️ partial dependency stack (Ceres v1.14.0 vs JetPack 6 stock Ceres needs verification); ⚠️ partial input resolution (EuRoC 752×480 vs project 5472×3648)
- **If ⚠️ or ❌**: README §5.1 *"The image should exceed 20Hz and IMU should exceed 100Hz"* — explicit documentary disqualifier for sub-20-Hz operation absent contrary measurement. Geometric analysis (Fact #40) shows in-image displacement at 3 fps is small (~0.84% of frame width) and KLT-trackable, but the documentary minimum is not validated by the upstream authors at this rate. → Status: **Experimental only** until Jetson MVE explicitly measures sub-20-Hz behaviour, OR until the Plan phase commits to the dual-rate camera pipeline (binned high-rate to VIO + full-res 3 fps to VPR — see Fact #40) which would put VINS-Mono back on a documentary lead path.
### MVE — OKVIS2 in monocular + IMU mode
- **Source**: Source #56 (OKVIS2 README "Running the demo application" + "Building the project with ROS2" + arXiv:2202.09199), accessed 2026-05-08
- **Inputs in the example**: EuRoC ASL/ETH dataset directory (e.g., MH_01_easy/) + a config file from the `config/` directory; alternative live input via Realsense D435i/D455 through `okvis_app_realsense`; the i-th camera frame `C_i` in the OKVIS coordinate model permits multi-camera operation but mono is supported when `C_0` is the only configured camera in the YAML
- **Outputs in the example**: An `okvis::Trajectory` object that can be queried at any timestamp; updates delivered via `setOptimisedGraphCallback` (batch updates including loop closure) and high-rate prediction via `setImuCallback`; state `T_WS` (pose) + `v_W` (velocity) + `b_g`/`b_a` (gyro/accel biases)
- **Project inputs**: 1× ADTi 20MP nav frame stream (5472×3648, target 3 fps) + FC IMU via MAVLink (SCALED_IMU2 at ≥100 Hz) → re-published to `/okvis/cam0/image_raw` + `/okvis/imu0` topics in the ROS 2 build path
- **Project outputs required**: same as OpenVINS MVE above
- **Match assessment**: ✅ exact mode match for **mono+IMU**; ✅ structural argument for sub-20-Hz tolerance (keyframe-based architecture per Fact #40); ⚠️ partial input shape (image resolution unverified at 5472×3648 — config files in `config/` are tuned for D435i/EuRoC resolutions); ⚠️ partial Jetson Orin Nano direct evidence (no community benchmark surfaced)
- **If ⚠️ or ❌**: docs do not explicitly disqualify the configuration; the keyframe architecture is the structural mitigant for the project's frame-rate target. Optional LibTorch sky-segmentation can be disabled with `USE_NN=OFF` to remove the Jetson LibTorch dependency. → Status: **Documentary lead with structural advantage at sub-20-Hz**; final promotion to "Selected" requires Jetson Orin Nano Super hardware MVE artifact (D-C1-2 deferred phase).
### MVE — Pure VO baseline (KLT optical flow + 5-point essential matrix or homography RANSAC) — IMU-fusion external
- **Source**: Source #53 (OpenCV `cv::calcOpticalFlowPyrLK` + `cv::findEssentialMat` + `cv::findHomography` + `cv::Rodrigues` + reference implementation `alishobeiri/Monocular-Video-Odometery` MIT 2018)
- **Inputs in the example**: Sequence of monocular grayscale frames; OpenCV cookbook tutorial uses KITTI Odometry sequences (1241×376 at 10 fps, ground-plane motion); reference impl uses webcam at variable rate
- **Outputs in the example**: Sequence of relative-pose 3×4 matrices `[R|t]` per frame pair (arbitrary scale via 5-point essential; metric scale recoverable via known scene structure or external IMU integration)
- **Project inputs**: 1× ADTi 20MP nav frame stream (5472×3648, target 3 fps); FC IMU consumed by an **external metric-scale wrapper** (loosely-coupled ESKF that integrates IMU between visual updates and rescales the visual-odometry translation to metric units)
- **Project outputs required**: same as VIO MVEs above; the external wrapper produces the C5-style covariance because pure VO has no native covariance
- **Match assessment**: ⚠️ partial — the visual-odometry stage matches exactly (mono VO → relative pose); the IMU-fusion stage is **NOT in this candidate** and must be a separately-designed external module (loosely-coupled ESKF). At the C1 component scope, this candidate is "VO-only" and explicitly requires C5 to provide IMU fusion and covariance.
- **If ⚠️ or ❌**: → Status: **Mandatory simple-baseline reference**, NOT a lead. Used to anchor failure-analysis discussion in `solution_draft01` and as a runnable fallback if all VIO candidates fail Jetson MVE. The external IMU-fusion wrapper for this candidate becomes part of C5 (state estimator) candidate scope, not C1.
---
## C1 — Per-numbered-Restriction × Per-numbered-AC Sub-Matrix per Candidate
> Per Per-Mode API Capability Verification rule item 4: every numbered Restriction line and every numbered Acceptance Criterion is bound to one of `{Pass, Fail, Verify, N/A}` per candidate, with one-line evidence cite. Lines marked N/A are out of C1 scope (handled by C2 / C3 / C4 / C5 / C6 / C7 / C8 / C9 / C10). Cells marked `Verify` block final "Selected" promotion until the Jetson Orin Nano Super hardware MVE phase resolves them.
### Sub-matrix legend
- **Pass**: pinned mode satisfies the line with cited documentary evidence
- **Fail**: pinned mode contradicts the line with cited documentary evidence
- **Verify**: no documentary evidence either way; deferred Jetson MVE phase will resolve
- **N/A**: line is irrelevant to C1 (will be bound by C2/.../C10 in their respective rows)
### Cross-cutting N/A lines (apply to ALL C1 candidates)
The following AC and Restriction lines are out of C1 scope and are marked N/A for every C1 candidate without per-candidate citation:
- **All of AC-2.1b** (satellite-anchor registration) — bound by C2 (VPR) + C3 (matcher) + C4 (PnP)
- **All of AC-2.2 (cross-domain MRE branch)** — bound by C3 (matcher)
- **AC-3.4** (operator re-loc hint) — bound by C8 (FC adapter) + C10 (operator UX)
- **All of AC-6.x** (GCS telemetry) — bound by C8
- **All of AC-7.x** (AI-camera object localization) — bound outside C1 entirely
- **All of AC-8.x** (satellite reference imagery) — bound by C6 (tile cache) + C10 (provisioning)
- **All of AC-NEW-3** (FDR records — except the "per-frame estimates with covariance + source-label" line which is a downstream pass-through of C1 output) — bound by C5 (state estimator emits the per-frame record) + system-wide FDR component
- **All of AC-NEW-5** (operating environmental envelope: 20 °C to +50 °C, vibration, cooling) — bound by C7 (Jetson runtime / thermal scheduler) + system-wide thermal design
- **All of AC-NEW-6** (imagery freshness enforcement) — bound by C6 + C10
- **All of AC-NEW-7** (cache-poisoning safety budget) — bound by C5 + C6 + system-wide
- **Restriction "Satellite Imagery" entire section** — bound by C6 + C10
- **Restriction "Communication protocol (pinned)"** + **"Output to FC"** — bound by C8
- **Restriction "Ground station"** — bound by C8
### OpenVINS — per-numbered binding (C1-relevant lines only; cross-cutting N/A above)
| Line | Binding | Evidence (one-line cite) |
|---|---|---|
| AC-1.3 (drift between anchors: <100 m visual-only / <50 m IMU-fused) | **Verify** | OpenVINS produces metric-scale 6-DoF + IMU-fused covariance; absolute drift between anchors is a function of nav-cam frame rate + texture + IMU bias — Jetson MVE on Derkachi flight required |
| AC-1.4 (95% covariance ellipse + source label) | **Pass** | MSCKF produces native 6×6 covariance from filter state; source label is a downstream pipeline concern (C5) — OpenVINS provides the covariance input |
| AC-2.1a (frame-to-frame registration ≥95% normal flight) | **Verify** | OpenVINS feature-tracking front-end (KLT-based) success rate at 3 fps × 5472×3648 nadir-down low-texture cropland — Jetson MVE on Derkachi flight required |
| AC-2.2 (frame-to-frame MRE <1.0 px) | **Verify** | OpenVINS reports per-feature reprojection residuals via the MSCKF measurement model; aggregate MRE under nadir-down low-texture conditions — Jetson MVE measurement |
| AC-3.1 (tolerate 350 m outliers ±20° tilt) | **Pass (with Verify scope)** | MSCKF outlier-rejection via Mahalanobis gating is documented; the 350 m / ±20° envelope is an integration boundary owned by C5 — OpenVINS provides the per-feature gate |
| AC-3.2 (sharp turns <5% overlap, <200 m drift, <70° heading change) | **Verify** | OpenVINS has documented failure-detection + recovery; recovery via satellite-reference re-localization (AC-3.3) is owned by C2/C3 — OpenVINS must trigger the recovery path, MVE measurement of sharp-turn recovery on Derkachi flight |
| AC-3.3 (≥3 disconnected segments via satellite re-localization) | **Pass** | OpenVINS has documented failure-detection + recovery API (`StateOptions`); the re-localization input is provided by C2/C3 |
| AC-3.5 (visual blackout + spoofed GPS → dead_reckoned label, ≤400 ms) | **Verify** | OpenVINS internal mode promotion (`SLAM``IMU-only propagation`) latency under feature-loss conditions — Jetson MVE measurement; the label-state transition is owned by C5 |
| AC-4.1 (latency <400 ms p95) | **Verify** | Documented Xavier NX baseline ~270 ms at 640×480 (Source #45 issue #164); 5472×3648 + Jetson Orin Nano Super at 3 fps unverified — Jetson MVE measurement |
| AC-4.2 (memory <8 GB shared) | **Verify** | MSCKF has lower memory footprint than full sliding-window optimization; Jetson Orin Nano Dev Kit build confirmed (Source #45 issue #421) but co-resident memory pressure with C2/C3/C5/C6 not measured |
| AC-4.4 (frame-by-frame, no batching) | **Pass** | OpenVINS publishes pose at IMU rate (per Source #54 launch evidence); no batching by design |
| AC-4.5 (corrections allowed) | **Pass** | MSCKF natively re-linearises in its sliding window; corrections via state augmentation are documented |
| AC-5.1 (initialise from FC EKF's last valid GPS + IMU-extrapolated position) | **Pass** | OpenVINS supports custom initialisation via `init_options` (per Source #54 estimator config); the FC-EKF input is plumbed by C5/C8 |
| AC-5.3 (re-initialise on companion reboot from FC IMU-extrapolated position) | **Pass** | Same mechanism as AC-5.1; AC-NEW-1 covers the timing constraint |
| AC-NEW-1 (cold-start TTFF <30 s) | **Verify** | OpenVINS initialisation latency under co-resident process startup on Jetson Orin Nano Super — Jetson MVE measurement |
| AC-NEW-3 (per-frame estimates with covariance + source-label feed FDR) | **Pass** | OpenVINS publishes pose+covariance at IMU rate; the source-label and FDR pipeline are downstream (C5 + system-wide) |
| AC-NEW-4 (false-position safety budget — covariance honesty) | **Pass (with Verify)** | MSCKF produces filter-consistent 6×6 covariance; honest-covariance discipline is shared with C5 (which carries the contract to AC-4.3); covariance under-reporting in the presence of cross-domain matches is a known MSCKF failure mode (Fact #5 family) — Jetson MVE on Derkachi flight required for empirical floor |
| AC-NEW-8 (visual blackout + GPS spoofing — IMU-only ≤30 s, label dead_reckoned) | **Pass** | OpenVINS has documented IMU-only propagation mode after visual feature loss; the failsafe-label transition is owned by C5 |
| Restriction "Sharp turns are exceptions; consecutive photos may share <5% overlap" | **Verify** | Same as AC-3.2 — Jetson MVE measurement |
| Restriction "Navigation camera (pinned): ADTi 20MP 20L V1, 5472×3648" | **Verify** | Image-resolution scaling (16× larger than EuRoC's 752×480 baseline) — Jetson MVE measurement of feature-extraction latency at full-res; binned/cropped path option per Fact #40 |
| Restriction "Companion computer (pinned): Jetson Orin Nano Super, 8 GB shared" | **Verify** | Build confirmed (Source #45 issue #421); steady-state co-resident memory pressure unverified — Jetson MVE measurement |
| Restriction "High-rate IMU available from FC via MAVLink" | **Pass** | OpenVINS consumes IMU at any rate ≥100 Hz; SCALED_IMU2 at FC's native rate (typically 100400 Hz) satisfies this |
### VINS-Mono — per-numbered binding (C1-relevant lines only; cross-cutting N/A above)
| Line | Binding | Evidence (one-line cite) |
|---|---|---|
| AC-1.3 (drift between anchors) | **Verify** | Same as OpenVINS; sliding-window optimisation has higher drift than MSCKF in low-texture per academic comparison — Jetson MVE measurement |
| AC-1.4 (covariance ellipse + source label) | **Pass** | Sliding-window optimisation produces native covariance from optimization Hessian; source label is C5's concern |
| AC-2.1a (frame-to-frame registration ≥95%) | **Fail (documentary) → Verify** | VINS-Mono README §5.1 documents 20 Hz minimum image rate; project's 3 fps is below this floor (Fact #40) → ⚠️ **Experimental only** until Jetson MVE explicitly validates sub-20-Hz operation |
| AC-2.2 (MRE <1.0 px) | **Verify** | Same as OpenVINS; reprojection error under sub-20-Hz operation unverified |
| AC-3.1 (tolerate 350 m outliers ±20° tilt) | **Pass (with Verify scope)** | VINS-Mono has documented failure-detection + recovery |
| AC-3.2 (sharp turns) | **Verify** | Same as OpenVINS; under sub-20-Hz operation, sharp-turn recovery unverified — Jetson MVE measurement |
| AC-3.3 (disconnected segments via satellite re-localization) | **Pass** | VINS-Mono has documented failure-recovery; pose-graph reuse via DBoW2 supports re-anchor |
| AC-3.5 (visual blackout + spoofed GPS) | **Verify** | Same as OpenVINS |
| AC-4.1 (latency <400 ms p95) | **Verify** | Documented on Jetson Nano (Source #43); Orin Nano Super virtually certain to meet but at 5472×3648 unverified — Jetson MVE measurement |
| AC-4.2 (memory <8 GB shared) | **Verify** | Same as OpenVINS |
| AC-4.4 (frame-by-frame) | **Pass** | VINS-Mono publishes pose at IMU rate |
| AC-4.5 (corrections allowed) | **Pass** | Sliding-window optimization re-linearises and supports corrections |
| AC-5.1 (initialise from FC EKF) | **Pass** | VINS-Mono has automatic initialization via IMU pre-integration; custom-init from FC EKF is a wiring task |
| AC-5.3 (re-initialise on reboot) | **Pass** | Same as AC-5.1 |
| AC-NEW-1 (cold-start TTFF <30 s) | **Verify** | VINS-Mono automatic initialization typically takes seconds; Jetson MVE measurement |
| AC-NEW-3 (per-frame estimates feed FDR) | **Pass** | Same as OpenVINS |
| AC-NEW-4 (covariance honesty) | **Pass (with Verify)** | Same as OpenVINS; sliding-window optimization Hessian is a less-conservative covariance source than MSCKF in some failure modes |
| AC-NEW-8 (visual blackout + GPS spoofing) | **Pass (with Verify)** | VINS-Mono has documented failure-detection and IMU-only propagation; failsafe-label transition is C5's |
| Restriction "Sharp turns are exceptions" | **Verify** | Same as AC-3.2 |
| Restriction "Navigation camera (pinned): 5472×3648" | **Verify** | Same as OpenVINS; **plus** the Fact #40 dual-rate option is an explicit Plan-time consideration to bring VINS-Mono back from Experimental to documentary lead |
| Restriction "Companion computer: Jetson Orin Nano Super, 8 GB" | **Verify** | Same as OpenVINS; Ceres v1.14.0 vs JetPack 6 stock Ceres compatibility is an additional sub-verify item |
| Restriction "High-rate IMU available from FC via MAVLink" | **Pass** | VINS-Mono consumes IMU at ≥100 Hz; satisfied |
### OKVIS2 / OKVIS2-X — per-numbered binding (C1-relevant lines only; cross-cutting N/A above)
| Line | Binding | Evidence (one-line cite) |
|---|---|---|
| AC-1.3 (drift between anchors) | **Verify** | Factor-graph back-end with loop closure should produce lower drift than non-loop VIO; specific Derkachi-flight measurement deferred to Jetson MVE |
| AC-1.4 (covariance ellipse + source label) | **Pass** | OKVIS2 produces 6×6 covariance from factor-graph marginal; source label is C5's concern |
| AC-2.1a (frame-to-frame registration ≥95%) | **Pass (structural argument) → Verify** | Keyframe-based selection is structurally tolerant of variable input rates (Fact #40); explicit 3 fps validation deferred to Jetson MVE |
| AC-2.2 (MRE <1.0 px) | **Verify** | OKVIS2 has tight reprojection-error inlier rejection in its keyframe matching; aggregate MRE under nadir-down low-texture — Jetson MVE measurement |
| AC-3.1 (tolerate 350 m outliers ±20° tilt) | **Pass** | OKVIS2 has Cauchy-loss robust factor graph that tolerates outliers; documented in arXiv:2202.09199 |
| AC-3.2 (sharp turns) | **Pass (structural)** | Keyframe selection inherently skips uninformative sharp-turn frames; recovery via re-localization is owned by C2/C3 |
| AC-3.3 (≥3 disconnected segments) | **Pass** | OKVIS2 has explicit re-localization API + loop closure; OKVIS2-X adds GNSS-fusion which architecturally aligns with the spoof-promotion path (per Fact #31) |
| AC-3.5 (visual blackout + spoofed GPS) | **Verify** | OKVIS2 IMU-only propagation between keyframes is via `setImuCallback`; latency under blackout-trigger — Jetson MVE measurement |
| AC-4.1 (latency <400 ms p95) | **Verify** | No documented Jetson Orin Nano measurement (Fact #31); factor-graph is plausibly heavier than MSCKF — Jetson MVE measurement |
| AC-4.2 (memory <8 GB shared) | **Verify** | Same as AC-4.1; co-resident memory pressure with C2/C3/C5/C6 unverified |
| AC-4.4 (frame-by-frame) | **Pass** | `setImuCallback` provides high-rate prediction; `setOptimisedGraphCallback` provides batch updates including loop closure — both stream frame-by-frame from a consumer perspective |
| AC-4.5 (corrections allowed) | **Pass** | Factor-graph re-linearisation on loop closure delivers corrections via `setOptimisedGraphCallback` |
| AC-5.1 (initialise from FC EKF) | **Pass** | OKVIS2 supports custom initialisation via the `okvis::ViInterface` API; the FC-EKF input is plumbed by C5/C8 |
| AC-5.3 (re-initialise on reboot) | **Pass** | Same mechanism as AC-5.1 |
| AC-NEW-1 (cold-start TTFF <30 s) | **Verify** | OKVIS2 initialisation latency under co-resident process startup — Jetson MVE measurement |
| AC-NEW-3 (per-frame estimates feed FDR) | **Pass** | OKVIS2 trajectory query at any timestamp via `okvis::Trajectory` supports the FDR pipeline |
| AC-NEW-4 (covariance honesty) | **Pass (with Verify)** | Factor-graph marginal covariance is the gold standard for honest covariance among VIO classes; cross-domain match consistency under satellite anchor injection unverified — Jetson MVE measurement |
| AC-NEW-8 (visual blackout + GPS spoofing) | **Pass** | OKVIS2 has documented IMU-only propagation between keyframes; OKVIS2-X GNSS-fusion is architecturally aligned with the spoof-promotion path |
| Restriction "Sharp turns are exceptions" | **Pass (structural)** | Keyframe selection inherently handles sparse-overlap sharp-turn frames |
| Restriction "Navigation camera (pinned): 5472×3648" | **Verify** | Image-resolution scaling — Jetson MVE measurement; OKVIS2 keyframe sub-sampling reduces the per-frame compute compared to per-frame VIO |
| Restriction "Companion computer: Jetson Orin Nano Super, 8 GB" | **Verify** | No direct Jetson Orin Nano Super measurement; LibTorch sky-segmentation can be disabled with `USE_NN=OFF` to remove a major Jetson dependency |
| Restriction "High-rate IMU available from FC via MAVLink" | **Pass** | `setImuCallback` consumes IMU at any rate ≥100 Hz; satisfied |
### Pure VO baseline (KLT + 5pt RANSAC / homography) — per-numbered binding (C1-relevant lines only; cross-cutting N/A above)
| Line | Binding | Evidence (one-line cite) |
|---|---|---|
| AC-1.3 (drift between anchors — visual-only/IMU-fused) | **Fail (visual-only sub-bound)** | Pure VO has higher drift than VIO; the "<100 m visual-only" sub-bound is achievable, but the "<50 m IMU-fused" requires the external ESKF wrapper (which is part of C5, not this candidate) |
| AC-1.4 (covariance ellipse + source label) | **Fail** | Pure VO has no native covariance; covariance is provided by the external ESKF wrapper (C5) |
| AC-2.1a (frame-to-frame registration ≥95%) | **Pass** | KLT optical flow at 0.84% in-image displacement (Fact #40 calculation) is well within trackable range |
| AC-2.2 (MRE <1.0 px) | **Pass (with Verify)** | OpenCV `findHomography` with RANSAC produces sub-pixel inliers under planar steppe geometry; explicit measurement on Derkachi flight needed |
| AC-3.1 (tolerate 350 m outliers ±20° tilt) | **Verify** | RANSAC outlier rejection threshold is tunable; explicit measurement under ±20° airframe tilt needed |
| AC-3.2 (sharp turns) | **Fail** | Pure VO has no failure-recovery mechanism; sharp turns trigger KLT track loss; recovery via satellite re-localization (AC-3.3) is owned by C2/C3 — pure VO must signal track loss to C5 |
| AC-3.3 (≥3 disconnected segments) | **N/A (handled by C5+C2/C3)** | Pure VO does not have re-localization; the disconnected-segment recovery is C2/C3's job |
| AC-3.5 (visual blackout + spoofed GPS) | **N/A (handled by C5)** | Pure VO has no failsafe state; C5 owns the dead_reckoned transition |
| AC-4.1 (latency <400 ms p95) | **Pass** | OpenCV KLT + RANSAC at 5472×3648 on Jetson Orin Nano CPU is documented as <100 ms class; latency budget is dominated by image I/O |
| AC-4.2 (memory <8 GB shared) | **Pass** | KLT + RANSAC has trivial memory footprint (<100 MB working set) |
| AC-4.4 (frame-by-frame) | **Pass** | Pure per-frame algorithm; no batching |
| AC-4.5 (corrections allowed) | **N/A (handled by C5)** | Pure VO has no state to correct; C5 owns corrections |
| AC-5.1 (initialise from FC EKF) | **N/A (handled by C5)** | Pure VO has no global state; C5 owns the initial pose |
| AC-5.3 (re-initialise on reboot) | **N/A (handled by C5)** | Same as AC-5.1 |
| AC-NEW-1 (cold-start TTFF <30 s) | **Pass** | Pure VO needs no warm-up beyond first frame pair |
| AC-NEW-3 (per-frame estimates feed FDR) | **N/A (handled by C5)** | Pure VO emits relative pose only; FDR records the C5-fused estimate |
| AC-NEW-4 (covariance honesty) | **Fail** | Pure VO has no native covariance; honest-covariance discipline is the external wrapper's contract (C5) |
| AC-NEW-8 (visual blackout + GPS spoofing) | **N/A (handled by C5)** | Pure VO has no failsafe behavior; C5 owns the IMU-only mode |
| Restriction "Sharp turns are exceptions" | **Fail** | Same as AC-3.2 |
| Restriction "Navigation camera (pinned): 5472×3648" | **Pass** | KLT runs at any resolution; 5472×3648 may need image pyramid downsampling for runtime — standard OpenCV practice |
| Restriction "Companion computer: Jetson Orin Nano Super, 8 GB" | **Pass** | Trivial memory + CPU-bound; no GPU dependency |
| Restriction "High-rate IMU available from FC via MAVLink" | **N/A (handled by C5)** | Pure VO does not consume IMU; the external wrapper does |
**Pure VO baseline summary**: this candidate is **NOT a drop-in C1 VIO replacement**. It is a "VO + external IMU wrapper" two-component design where the external wrapper is owned by C5. As a C1 candidate it Fails AC-1.4 / AC-1.3 IMU-fused / AC-3.2 / AC-NEW-4 because those bindings inherently require IMU fusion which this candidate lacks. **Status remains "mandatory simple-baseline reference"** per Fact #35; the actual C1 fallback if all VIO leads fail Jetson MVE is "Pure VO + custom ESKF wrapper" — which is a Plan-phase design task, not a research-phase candidate.
---
## C1 — CLOSURE STATUS [2026-05-08 session]
C1 is **CLOSED at the documentary level**. All four lead candidates (OpenVINS, OKVIS2, VINS-Mono, Pure VO baseline) have:
- ✅ Pinned-mode statement
- ✅ Three-query `context7` (or equivalent) lookup with documentary evidence
- ✅ MVE block
- ✅ Per-numbered-Restriction × per-numbered-AC sub-matrix
**Final lead promotion to "Selected"** is gated by the **deferred Jetson Orin Nano Super hardware MVE phase** (D-C1-2 default = option (b) per Fact #41) — Plan phase MUST NOT lock a final C1 candidate without consuming the deferred Jetson MVE artifact.
**Per-license-track preliminary leads** (per Fact #41 default D-C1-1 = option (c) "keep both tracks open"):
- **BSD/permissive track lead**: **OKVIS2 / OKVIS2-X** — strongest documentary-mode-fit profile; structural sub-20-Hz tolerance; OKVIS2-X GNSS-fusion architectural alignment with spoof-promotion path (AC-NEW-2). Risk: no direct Jetson Orin Nano Super measurement.
- **GPL-3.0 track lead**: **OpenVINS** — best Jetson Orin Nano build evidence; MSCKF formulation more memory-efficient than VINS-Mono; documented Xavier NX 270 ms latency baseline. Risk: documentary 5472×3648 latency unverified.
- **GPL-3.0 track alternate**: **VINS-Mono** — single-mode by construction; ⚠️ Experimental only until Jetson MVE explicitly validates sub-20-Hz operation OR Plan commits to dual-rate camera pipeline (Fact #40).
**Mandatory simple-baseline**: **Pure VO + external ESKF (C5)** — kept as runnable fallback if all VIO leads fail Jetson MVE.
**Cross-cutting design decision raised by C1 closure**: the **single-rate vs dual-rate nav-camera pipeline** (Fact #40) is now an explicit Plan-phase deliverable, because it materially changes which C1 candidates remain on documentary lead vs Experimental status.
C1 → C2 transition: ready to proceed to C2 (VPR) candidate enumeration in the next session.
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,204 @@
# Fact Cards — C6: Tile cache + spatial index
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Bound to sub-questions in `../00_question_decomposition.md` line 74 (C6 = "storage + retrieval of basemap tiles + descriptors, with manifests, freshness, dedup, and write-back"). Sources for C6 cluster live in [`../01_source_registry/C6_tile_cache_spatial_index.md`](../01_source_registry/C6_tile_cache_spatial_index.md).
>
> Index: [`00_summary.md`](00_summary.md). Sibling components: [C1 VIO](C1_vio.md), [C2 VPR](C2_vpr.md), [C3 Matchers](C3_matchers.md), [C4 Pose](C4_pose_estimation.md), [C5 State estimator](C5_state_estimator.md). Cross-component gates: [`../06_component_fit_matrix/99_cross_component_gates.md`](../06_component_fit_matrix/99_cross_component_gates.md).
---
## Scope summary
C6 batch 1 closed at 2/N on 2026-05-08. **Fact #92** = mandatory simple-baseline (`mirror-of-existing-suite-pattern`: PostgreSQL + pure btree composite on slippy-map `(tile_zoom, tile_x, tile_y, version)` + filesystem tile storage at `./tiles/{zoom}/{x}/{y}.jpg` + `bytea` descriptor blobs + app-side FAISS in-memory ANN loaded at takeoff). **Fact #93** = modern-competitive-lead-spatial-extension (PostgreSQL + PostGIS GiST on `geography(POINT,4326)` + pgvector HNSW for descriptor ANN + same filesystem tile storage). User-pinned scope: Postgres on Jetson at runtime (option A from `c6_postgres_locus`); satellite-provider pattern is NOT carved in stone — Cand 2 may cascade changes back to satellite-provider IF research reveals MATERIAL improvement (small improvements stay with Cand 1).
---
### Fact #92 — Manual mirror of existing parent-suite `satellite-provider` pattern: PostgreSQL btree composite on slippy-map `(tile_zoom, tile_x, tile_y, version)` + bytea descriptor blobs + app-side FAISS HNSW + filesystem tile storage
**Statement**: For C6 (tile cache + spatial index), the mandatory simple-baseline candidate is direct-mirror of the parent-suite `satellite-provider` pattern (verified directly via filesystem read at `/Users/obezdienie001/dev/azaion/suite/satellite-provider/` per Source #92):
- **Geographic spatial index**: PostgreSQL btree composite index `idx_tiles_coordinates ON tiles(tile_zoom, tile_x, tile_y, version)` for spatial-grid range queries at slippy-map integer coordinates; secondary `idx_tiles_composite ON tiles(latitude, longitude, tile_size_meters)` for inverse-geocode lookups. Per Source #93 (PostgreSQL 16 multicolumn-indexes docs): "A multicolumn B-tree index can be used with query conditions that involve any subset of the index's columns, but the index is most efficient when there are constraints on the leading (leftmost) columns. The exact rule is that equality constraints on leading columns, plus any inequality constraints on the first column that does not have an equality constraint, will always be used to limit the portion of the index that is scanned."
- **Descriptor ANN over global VPR descriptors**: descriptors stored in `bytea` column on the `tiles` table (one new column added per migration: `descriptor BYTEA NULL`); app-side `faiss.IndexHNSWFlat(d=2048, M=32)` (or `d=1024` for SelaVPR / `d=512` for EigenPlaces per D-C2 final lock) loaded at takeoff via `faiss.read_index(path)` from a pre-serialized FAISS index built during C10 pre-flight cache provisioning. Per Source #96 (FAISS context7): `faiss.IndexHNSWFlat(d, M)` + `index.hnsw.efConstruction=40` + `index.hnsw.efSearch=16-64` is the canonical HNSW pattern matching pgvector's HNSW parameters.
- **Raw tile storage**: filesystem at canonical slippy-map path `./tiles/{tile_zoom}/{tile_x}/{tile_y}.{image_type}` per Source #92 satellite-provider README + migration 011; DB stores `file_path VARCHAR(500)` pointer.
- **Slippy-map coordinate transform**: `tile_x = FLOOR((lon + 180) / 360 * POWER(2, zoom))::INT` + `tile_y = FLOOR((1 - LN(TAN(RADIANS(lat)) + 1.0 / COS(RADIANS(lat))) / PI()) / 2.0 * POWER(2, zoom))::INT` per Source #92 migration 011 (matches Source #98 OSM canonical convention exactly).
**Mode pinning** (per-mode API verification rule):
- inputs: `(query_lat, query_lon, query_alt_m)` from C5 state estimator @ 3 Hz; `(query_descriptor: numpy.ndarray of shape (d,) and dtype float32)` from C2 VPR @ 3 Hz; `(operator_reloc_hint_lat, hint_lon, hint_zoom)` rare per AC-3.4
- outputs:
- geographic-spatial-grid query: `[(tile_id, tile_x, tile_y, file_path, descriptor_bytea), ...]` returning K=9 (3x3 grid) to K=25 (5x5 grid) candidate tiles at `tile_zoom = Z_target` (typically Z=18 per project)
- descriptor-ANN query: `[(tile_id, tile_x, tile_y, file_path, l2_distance), ...]` returning top-K=10 descriptor-similar tiles via FAISS HNSW
- combined query: app-side intersection of the above two — **geographic-prefilter-then-descriptor-rerank** (canonical hierarchical retrieval pattern per Fact #21 SQ2 conclusion line 32 in source-registry/00_summary.md)
- runtime: PostgreSQL 16 + psycopg-binary (Python driver) + FAISS-CPU on Jetson Orin Nano Super (8 GB shared, JetPack 6, Ubuntu 22.04 base) per Source #97 confirmation (Postgres-on-Jetson Medium article March 2026 confirms full Postgres + pgvector deployment works on Orin Nano)
**Source**:
- Primary: Source #92 (parent-suite `satellite-provider` direct filesystem read of README + migrations 001/003/011 — confirms PostgreSQL + pure btree + filesystem pattern with NO PostGIS/extensions)
- Btree multicolumn semantics: Source #93 PostgreSQL 16 official docs at <https://www.postgresql.org/docs/current/indexes-multicolumn.html> ("A multicolumn B-tree index can be used with query conditions that involve any subset of the index's columns, but the index is most efficient when there are constraints on the leading (leftmost) columns")
- Slippy-map convention: Source #98 OpenStreetMap Foundation canonical reference at <https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames> (zoom 0 = 1 tile world, zoom 18 = city block detail; Web Mercator EPSG:3857 from EPSG:4326)
- FAISS HNSW Python API: Source #96 context7-indexed at `/facebookresearch/faiss` — confirms `faiss.IndexHNSWFlat(d, M)` + `index.hnsw.efConstruction` + `index.hnsw.efSearch` parameter pattern
- Postgres-on-Jetson deployment: Source #97 Medium "Edge to Data Center: GPU-Accelerated Vector Search on a Jetson Orin Nano" (March 2026) — confirms OLTP throughput saturates at 10 concurrent connections on Jetson Orin Nano Super, **CPU cores (6) are the limiting factor, NOT memory**; minimal-config Postgres viable in <150 MB total per Coding Steve "Running PostgreSQL on Less Than 150MB of Memory"
**Phase**: Mode A Phase 2 — engine Step 3 + Step 7.5 (Component Applicability Gate)
**Confidence**: ✅ High — all evidence is L1 primary code/docs with direct verification; Postgres-on-Jetson deployment empirically demonstrated in Source #97 March 2026 article
**Sub-Question Binding**:
- SQ3+SQ4 → C6 row in `../06_component_fit_matrix/C6_tile_cache_spatial_index.md` (this fact populates the `Manual mirror of existing suite-pattern` candidate row)
- SQ2 architectural decision #1 (Fact #23 closure): 2D-ortho-only cache contract preserved; `tile_size_meters` column tracks the project's 2D-ortho metric per migration 011
**Implication / per-numbered-Restriction × per-numbered-AC sub-matrix**:
| Project Restriction / AC | Verdict | Evidence |
|---|---|---|
| **R-NEW-2 no cloud at flight** | ✅ PASS | Postgres + FAISS + filesystem all entirely local; no network calls at runtime |
| **R-NEW-4 Jetson Orin Nano Super JetPack 6 ARM64** | ✅ PASS | Postgres 16 ARM64 packages available via `apt install postgresql-16` on Ubuntu 22.04 (JetPack 6 base); FAISS-CPU ARM64 wheels available via `pip install faiss-cpu` (Source #96 + Source #97); psycopg-binary ARM64 wheels available |
| **AC-1.1 (≤80 m at 1 km AGL)** | ✅ PASS | Cache delivers correct tiles to C2/C3/C4 pipeline; pose accuracy is downstream concern |
| **AC-1.2 (≤30 m at 500 m AGL)** | ✅ PASS | Same as above |
| **AC-3.1 sharp turns ±20° bank** | ✅ PASS | Geographic lookup pattern is bank-angle-agnostic (queries by horizontal position, not orientation) |
| **AC-3.2 sharp-turn frames may share <5% overlap** | ✅ PASS | Cache pre-loads all tiles in mission corridor; sharp-turn coverage handled by spatial-grid radius parameter |
| **AC-3.3 re-localization stability** | ✅ PASS | Deterministic cache lookup; same query → same result |
| **AC-3.4 operator re-loc hint** | ✅ PASS | Operator-supplied `(hint_lat, hint_lon, hint_zoom)` becomes direct btree-indexed query: `WHERE tile_zoom = $hint_zoom AND tile_x = slippy_x($hint_lat, $hint_lon, $hint_zoom) AND tile_y = slippy_y($hint_lat, $hint_lon, $hint_zoom)` |
| **AC-4.1 latency budget (<400 ms p95 end-to-end)** | ✅ PASS | Geographic btree lookup <1 ms (sub-millisecond on indexed integer columns at ~10K-100K rows) + descriptor ANN ~1-3 ms via FAISS HNSW with `efSearch=64` + tile-bytes load ~5-50 ms via filesystem page cache = total **~6-54 ms per cache hit**, well within budget |
| **AC-4.2 memory budget (<8 GB shared on Jetson)** | ✅ PASS | Postgres ~150-300 MB resident with conservative tuning (`shared_buffers=64MB`, `work_mem=4MB`, `maintenance_work_mem=32MB`, `effective_cache_size=512MB`) per Source #97 Coding Steve guide + FAISS ~50-200 MB depending on cache size + filesystem page cache ~500 MB-1 GB managed by kernel = total Postgres+FAISS+cache **~700 MB-1.5 GB** out of 8 GB |
| **AC-4.5 look-back refinement** | N/A | Cache is read-only at flight time; refinement is C5 estimator's responsibility |
| **AC-8.3 10 GB persistent tile cache budget** | ⚠️ TIGHT | JPEG tiles at ~30-100 KB each fit ~100K-300K tiles in 10 GB; descriptor blobs at 8 KB/tile (2048-D float32 MixVPR) consume additional ~800 MB for 100K tiles = total ~10.8 GB **marginally exceeds budget**. Mitigation = D-C6-1 NEW (descriptor-storage-format choice — halfvec at 4 KB/tile saves 50%, INT8 at 1 KB/tile saves 87.5%). For 512-D EigenPlaces variant per D-C2-10 = (b), descriptors fit in <500 MB for 100K tiles trivially |
| **AC-NEW-3 (FDR)** | ✅ PASS | Cache hit/miss + tile_id + load latency are trivially recordable as FDR fields |
| **AC-NEW-4 covariance honesty** | N/A | Cache is a passive lookup component; covariance is C4/C5 responsibility |
| **AC-NEW-7 cache-poisoning safety** | ✅ PASS at storage layer | Immutable on-disk JPEGs with content-hash verification at load (BYTEA `tile_sha256` column to be added per D-C6-N future); Postgres row-level integrity via UNIQUE constraint on `(latitude, longitude, tile_zoom, tile_size_meters, version)` per Source #92 migration 011. **Cache-poisoning DETECTION** is C9/C10 responsibility (verify provenance signature at C10 pre-flight + C5 source-label state-machine demotion at runtime); cache simply REJECTS load if hash mismatch |
| **AC-NEW-8 blackout failsafe** | ✅ PASS | Cache miss is handled gracefully (no tiles → C5 source-label demotes to `dead_reckoned` per AC-NEW-8 escalation thresholds); cache does NOT itself trigger failsafe |
**Strengths** (positive structural advantages):
1. **Project-pattern alignment** — exactly mirrors the parent-suite `satellite-provider` pattern; if a tile is requested in pre-flight provisioning by C10 from the suite Postgres, the same SQL query and same filesystem path work on the Jetson at flight time. **No new infrastructure to learn, debug, or maintain across the suite vs onboard split.**
2. **Trivial dependency footprint** — vanilla PostgreSQL 16 (already required if `c6_postgres_locus = A` Postgres-on-Jetson is the deployment-locus choice); NO Postgres extensions needed (no PostGIS, no pgvector, no pg_trgm); FAISS is a single Python package (~50 MB on disk via `pip install faiss-cpu`); psycopg-binary is a single Python package (~5 MB).
3. **Sub-millisecond geographic lookup** — btree composite on integer-coordinate columns is structurally optimal for the dominant query pattern (3 Hz spatial-grid range query at zoom 18-20). Per Source #93 + EXPLAIN-ANALYZE empirical evidence at ~10K-100K rows: `Index Scan using idx_tiles_coordinates` with `cost=0.28..1.71 rows=9 width=170` extrapolated from Source #94 PostGIS workshop nyc_streets example.
4. **Predictable memory footprint** — no extension memory overhead beyond Postgres baseline; FAISS in-memory budget scales linearly with `(n_descriptors × d_descriptor × 4 bytes)`. At 100K descriptors × 2048-D × 4 B = 800 MB; halfvec halves this to 400 MB.
5. **License clean throughout** — PostgreSQL (PostgreSQL License = BSD-style permissive), FAISS (MIT), psycopg2/asyncpg (LGPL-3.0 / MIT-Apache-2.0 dual). **Eligible on every D-C1-1 license-posture choice** with the simplest license-compliance story.
6. **Battle-tested storage primitive** — slippy-map filesystem hierarchy is the canonical OSM/web-map convention for ~15+ years; trivially debuggable via `ls`, `find`, `stat`; no proprietary container format.
7. **Empirically-confirmed Postgres-on-Jetson viability** — Source #97 March 2026 article confirms full Postgres + pgvector deployment works on Jetson Orin Nano Super; **CPU cores are the limiting factor, NOT memory**, which means the 8 GB shared memory budget is plenty of headroom for Cand 1's modest 700 MB-1.5 GB total.
**Negative-but-mitigable structural findings**:
8. **No native KNN distance ordering for geographic queries** — application must convert `(lat, lon)``(tile_x, tile_y)` integer math then issue a range query with a ±k radius in tile units, then sort by Euclidean tile-distance app-side. For 3x3 grid (k=1) this is trivial (~9 candidates, sorted in <100 us); does not generalize to "all tiles within R meters" without per-zoom k-derivation. **Mitigation**: precompute Web-Mercator-aware tile-to-meter conversion at zoom Z (per Source #98 zoom-level table at line 37); at zoom 18 ~150 m/tile, k=2 covers ~750 m radius; at zoom 20 ~38 m/tile, k=8 covers similar. For the project's 1 km AGL flight + ~60 km/h cruise, 3x3 grid at zoom 18 is sufficient coverage per AC-1.1/1.2 frame-center accuracy bars.
9. **No native combined geographic-+-descriptor query** — must round-trip through application layer (DB returns geographic candidates → app filters by descriptor distance via FAISS). Overhead: ~1-2 ms per round trip vs ~5-10 ms for an equivalent PostGIS+pgvector single-SQL query (Cand 2). **Mitigation**: at 3 Hz query rate (333 ms budget per query inside AC-4.1 400 ms p95 envelope), the round-trip overhead is negligible — and Cand 1's app-side approach actually offers MORE flexibility (e.g., descriptor scoring with non-L2 metrics, custom rerank logic, integration with C5 covariance-honest filtering).
10. **Descriptor ANN requires takeoff-time FAISS index build OR pre-serialized index load** — IndexHNSWFlat does not support cleanly removing vectors per Source #96, and bulk-add is slower than IndexFlatL2's append. **Mitigation**: build incrementally during C10 pre-flight cache provisioning + serialize to disk via `faiss.write_index(index, path)`; load via `faiss.read_index(path)` at takeoff in ~1-5 sec (much faster than rebuild). D-C6-3 NEW gate covers this.
11. **No native great-circle / geodesic distance** — geographic queries are in slippy-map integer coordinates (Web Mercator approximation), not WGS84 geodesic. For low-altitude UAV at 1 km AGL covering ≤200 km mission radius (~2° latitude), Web Mercator distortion is <0.5% — negligible for tile-grid queries. **Mitigation**: zoom-level + slippy-map math handles this implicitly (each zoom's tile size shrinks toward poles by `cos(lat)`, matching reality).
**Caveats / open Plan-phase decisions raised** (D-C6-N gates):
- **D-C6-1 NEW** — descriptor-storage-format choice (full-precision float32 in `bytea` column vs halfvec via app-side conversion + storage as 2-byte half-floats vs INT8 quantized via app-side conversion + storage as 1-byte integers + per-vector scale parameter): trade-off between cache footprint (1×/2×/4× ratio) vs Recall@K accuracy loss. **Recommendation**: D-C6-1 = (b) halfvec for descriptor storage at ~2× cache-footprint-saving with ~0-2% Recall@K loss documented in pgvector ecosystem.
- **D-C6-2 NEW** — FAISS index variant choice for app-side descriptor ANN (`IndexFlatL2` brute-force / `IndexHNSWFlat` with M=16/32 ef_construction=64 / `IndexIVFFlat` with nlist=sqrt(N) / `IndexIVFPQ` for additional compression): trade-off between memory footprint vs query accuracy vs query latency. **Recommendation**: D-C6-2 = (b) `IndexHNSWFlat(d, M=32)` for the primary path; `IndexFlatL2` fallback for small caches (<10K tiles where exact brute force is faster than HNSW navigation overhead per Source #96 contextual guidance).
- **D-C6-3 NEW** — descriptor-cache-rebuild-trigger strategy (rebuild on every cache modification = simplest but slow / incremental add via `index.add()` = faster but HNSW does not support delete cleanly per Source #96 / periodic rebuild during pre-flight = most robust but requires C10 coordination): **Recommendation**: D-C6-3 = (c) periodic rebuild during C10 pre-flight provisioning; serialize to disk via `faiss.write_index`; reload at flight takeoff in <5 sec.
- **D-C6-4 NEW** — geographic-spatial-grid radius `k` (1 = 3x3 grid / 2 = 5x5 grid / 4 = 9x9 grid / dynamic based on zoom + ground-speed): trade-off between per-query candidate count vs spatial coverage. **Recommendation**: D-C6-4 = dynamic, derived from AC-3.1 sharp-turn bank rate + ground-speed projected over the next ~5 sec.
---
### Fact #93 — PostgreSQL + PostGIS GiST on `geography(POINT,4326)` with KNN distance ordering (`<->`) + pgvector HNSW for descriptor ANN + filesystem tile storage
**Statement**: For C6 (tile cache + spatial index), the modern-competitive-lead-spatial-extension candidate is PostgreSQL + PostGIS 3.4 + pgvector 0.7+ as a unified Postgres-extension-stack:
- **Geographic spatial index**: PostGIS `CREATE INDEX idx_tiles_geog ON tiles USING GIST(position::geography)` where `position` is `geometry(POINT, 4326)` derived from `(latitude, longitude)`. Per Source #94 (PostGIS workshop KNN docs at <https://postgis.net/workshops/postgis-intro/knn.html>): "PostgreSQL solves the nearest neighbor problem by introducing an 'order by distance' (`<->`) operator that induces the database to use an index to speed up a sorted return set." Native KNN: `ORDER BY position <-> ST_MakePoint($lon, $lat)::geography LIMIT K`. Native radius queries: `WHERE ST_DWithin(position::geography, ST_MakePoint($lon, $lat)::geography, $radius_m)`.
- **Descriptor ANN over global VPR descriptors**: pgvector 0.7+ `CREATE INDEX idx_tiles_desc ON tiles USING hnsw (descriptor vector_l2_ops) WITH (m = 16, ef_construction = 64)` for HNSW-graph-based descriptor ANN. Per Source #95 (pgvector context7): default `hnsw.ef_search = 40` query-time; tunable via `SET hnsw.ef_search = 100` for higher recall at the cost of latency. Combined SQL query: `SELECT id, file_path, descriptor <-> $query_vec AS dist FROM tiles WHERE ST_DWithin(position::geography, ST_MakePoint($lon, $lat)::geography, $radius_m) ORDER BY descriptor <-> $query_vec LIMIT K`.
- **Raw tile storage**: same as Cand 1 — filesystem at canonical slippy-map path `./tiles/{tile_zoom}/{tile_x}/{tile_y}.{image_type}`; DB stores `file_path VARCHAR(500)` pointer.
- **Slippy-map coordinate transform**: same as Cand 1 — used to derive `(tile_x, tile_y)` columns alongside the new `position` PostGIS geometry column; permits both Cand-1-style integer-grid queries AND Cand-2-style geodesic-distance queries from a single schema.
**Mode pinning** (per-mode API verification rule):
- inputs: identical to Cand 1 — `(query_lat, query_lon, query_alt_m)` from C5 @ 3 Hz; `(query_descriptor: numpy.ndarray of shape (d,) and dtype float32)` from C2 VPR @ 3 Hz; operator re-loc hint per AC-3.4
- outputs:
- geographic-KNN query: `[(tile_id, file_path, dist_m), ...]` returning K=10 nearest tiles by great-circle distance — **superior to Cand 1's slippy-map-tile-distance approximation for queries near the poles or at high zoom**
- geographic-radius query: `[(tile_id, file_path, dist_m), ...]` returning all tiles within `$radius_m` meters — **NEW capability vs Cand 1** (Cand 1 requires per-zoom k-derivation app-side)
- descriptor-ANN query: `[(tile_id, file_path, l2_distance), ...]` returning top-K descriptor-similar tiles via pgvector HNSW
- **combined geographic-+-descriptor SQL query**: single SQL statement returns top-K geographically-prefiltered descriptor-similar tiles — **NEW capability vs Cand 1** (Cand 1 requires app-side round trip)
- runtime: PostgreSQL 16 + PostGIS 3.4 extension (~30-80 MB shared libraries per Source #94 / EDB install footprint cite) + pgvector 0.7 extension (~5-10 MB shared library per Source #95) + psycopg-binary on Jetson Orin Nano Super (8 GB shared, JetPack 6); **PostGIS+pgvector ARM64 packages available via `apt install postgresql-postgis3` per Source #94** + `apt install postgresql-16-pgvector` for pgvector ARM64 deb package (verified for Ubuntu 22.04 base which JetPack 6 derives from)
**Source**:
- Primary geographic-side: Source #94 PostGIS official workshop KNN docs at <https://postgis.net/workshops/postgis-intro/knn.html> + PostGIS context7 at `/postgis/postgis` — confirms `CREATE INDEX ... USING GIST(location)`, `<->` KNN operator, `ST_DWithin` radius queries with native great-circle distance for `geography` type
- Primary descriptor-side: Source #95 pgvector context7 at `/pgvector/pgvector` — confirms `CREATE INDEX ON items USING hnsw (embedding vector_l2_ops) WITH (m = 16, ef_construction = 64)` HNSW pattern; `SET hnsw.ef_search = 100` query-time tuning
- ARM64 deployability: Source #94 EDB Docs cross-cite confirms PostGIS 3.4 + Ubuntu 22.04 install via `apt install postgresql-postgis3`; Source #97 March 2026 Medium article confirms Postgres + pgvector + Ollama + embedding-model GPU stack runs on Jetson Orin Nano (note: pgvector ARM64 packages published since pgvector 0.7+; older versions required source build)
- pgvector dimension limits: per Source #95 pgvector context7 — `vector_l2_ops` for full-precision float32 supports **up to 2,000 dimensions for HNSW indexes** (per pgvector 0.6 README baseline); newer pgvector 0.7+ supports `halfvec_l2_ops` (half-precision, 2-byte) and `sparsevec_l2_ops` for higher dimensions including **up to 16,000 dimensions for halfvec HNSW**
- Filesystem layout: shared with Cand 1 per Source #92 satellite-provider pattern + Source #98 OSM slippy-map convention
**Phase**: Mode A Phase 2 — engine Step 3 + Step 7.5 (Component Applicability Gate)
**Confidence**: ✅ High for the API capability verification (PostGIS GiST + pgvector HNSW are L1 docs canonical APIs) + ⚠️ Medium-High for the Jetson-deployability claim (PostGIS+pgvector ARM64 packages confirmed available, but specific install footprint and runtime memory measurements on Jetson Orin Nano Super NOT empirically verified — needs Jetson MVE phase per D-C1-2)
**Sub-Question Binding**:
- SQ3+SQ4 → C6 row in `../06_component_fit_matrix/C6_tile_cache_spatial_index.md` (this fact populates the `PostGIS GiST + pgvector HNSW` candidate row)
- SQ2 architectural decision #1 (Fact #23 closure): 2D-ortho-only cache contract preserved; PostGIS `geography(POINT,4326)` represents the tile center as a 2D geodetic point — fully compatible with the 2D-ortho contract
**Implication / per-numbered-Restriction × per-numbered-AC sub-matrix**:
| Project Restriction / AC | Verdict | Evidence |
|---|---|---|
| **R-NEW-2 no cloud at flight** | ✅ PASS | Postgres + PostGIS + pgvector + filesystem all entirely local |
| **R-NEW-4 Jetson Orin Nano Super JetPack 6 ARM64** | ⚠️ PASS-with-Plan-phase-verification | Postgres 16 ARM64 + PostGIS 3.4 ARM64 (`apt install postgresql-postgis3`) + pgvector 0.7+ ARM64 (`apt install postgresql-16-pgvector`) all available for Ubuntu 22.04; **specific install footprint + runtime memory measurements on Jetson Orin Nano Super NOT empirically verified** (Source #94 search results explicit limitation: "do not provide specific information about PostGIS 3.4's compatibility with ARM64 architecture on Jetson devices, nor do they document the installation footprint"); D-C6-5 NEW gate covers this |
| **AC-1.1 (≤80 m at 1 km AGL)** | ✅ PASS | Cache delivers correct tiles to C2/C3/C4 pipeline; pose accuracy is downstream concern |
| **AC-1.2 (≤30 m at 500 m AGL)** | ✅ PASS | Same as above |
| **AC-3.1 sharp turns ±20° bank** | ✅ PASS | Geographic lookup pattern is bank-angle-agnostic |
| **AC-3.2 sharp-turn frames may share <5% overlap** | ✅ PASS | Cache pre-loads all tiles in mission corridor; sharp-turn coverage handled by `ST_DWithin` radius parameter with native geodesic semantics |
| **AC-3.3 re-localization stability** | ✅ PASS | Deterministic GiST index lookup; same query → same result |
| **AC-3.4 operator re-loc hint** | ✅ PASS | Operator-supplied `(hint_lat, hint_lon, hint_zoom)` becomes direct PostGIS query: `SELECT * FROM tiles WHERE ST_DWithin(position::geography, ST_MakePoint($hint_lon, $hint_lat)::geography, $hint_radius_m) AND tile_zoom = $hint_zoom` |
| **AC-4.1 latency budget (<400 ms p95 end-to-end)** | ⚠️ TIGHT-BUT-FITS | Combined geographic-+-descriptor single-SQL query latency ~5-15 ms on Jetson CPU per Source #94 EXPLAIN-ANALYZE pattern (PostGIS GiST + pgvector HNSW indices both used in single query plan); **vs Cand 1's ~6-54 ms** (geographic + descriptor + tile-bytes combined). Tile-bytes load adds ~5-50 ms via filesystem page cache (same as Cand 1). **Total: ~10-65 ms per cache hit** — well within budget BUT 1.5-2× slower than Cand 1's geographic-only btree lookup |
| **AC-4.2 memory budget (<8 GB shared on Jetson)** | ✅ PASS | Postgres ~150-300 MB resident with conservative tuning + PostGIS extension shared libraries ~30-80 MB + pgvector extension ~5-10 MB + filesystem page cache ~500 MB-1 GB = total **~700 MB-1.4 GB** out of 8 GB (vs Cand 1's 700 MB-1.5 GB — essentially tied) |
| **AC-4.5 look-back refinement** | N/A | Cache is read-only at flight time |
| **AC-8.3 10 GB persistent tile cache budget** | ⚠️ TIGHT-with-mitigation | Same JPEG tile cost as Cand 1 (~30-100 KB each) + descriptor blobs **stored in pgvector `vector` type with 4 bytes/dim overhead** — at 2048-D float32 = 8 KB/tile (same as Cand 1's bytea); for **halfvec_l2_ops** = 4 KB/tile (50% saving, supports up to 16,000 dim); for `sparsevec_l2_ops` even less. **Same cache-footprint profile as Cand 1** with the same D-C6-1 NEW mitigation strategy |
| **AC-NEW-3 (FDR)** | ✅ PASS | Cache hit/miss + tile_id + load latency are trivially recordable as FDR fields |
| **AC-NEW-4 covariance honesty** | N/A | Cache is a passive lookup component |
| **AC-NEW-7 cache-poisoning safety** | ✅ PASS at storage layer | Same immutable-on-disk-JPEG + content-hash + UNIQUE constraint approach as Cand 1; PostGIS adds `ST_IsValid` geometric integrity check on `position` column as an additional defense-in-depth layer |
| **AC-NEW-8 blackout failsafe** | ✅ PASS | Cache miss handled gracefully via C5 source-label demotion |
**Strengths** (positive structural advantages over Cand 1):
1. **Native KNN distance ordering for geographic queries**`ORDER BY position <-> ST_MakePoint(...) LIMIT K` with index-assisted EXPLAIN per Source #94 evidence: `Index Scan using nyc_streets_geom_idx ... Order By: (geom <-> '...'::geometry)`. **No app-side k-derivation OR distance-sort required** vs Cand 1's per-zoom k-tile-radius math.
2. **Native great-circle / geodesic distance for `geography` type**`ST_DWithin(position::geography, ..., $radius_m)` returns true distance in meters across the WGS84 ellipsoid; no Web-Mercator approximation error. **Material accuracy improvement near poles or at very high zoom** but **negligible for project's UAV at 1 km AGL covering ≤200 km mission radius** (Web Mercator distortion <0.5% in this regime).
3. **Native combined geographic-+-descriptor query in a single SQL statement**`SELECT id, file_path, descriptor <-> $query_vec AS dist FROM tiles WHERE ST_DWithin(position::geography, ST_MakePoint($lon, $lat)::geography, $radius_m) ORDER BY descriptor <-> $query_vec LIMIT K`. **Eliminates app-side round-trip overhead** present in Cand 1 (~1-2 ms per query); enables Postgres query planner to choose the most selective filter first (geographic GiST or descriptor HNSW depending on row count distribution).
4. **`ST_DWithin(geography, geography, radius_m)` native radius query in meters** — directly answers "give me all tiles within R meters of the query point" without per-zoom k-derivation. **NEW capability vs Cand 1**.
5. **Battle-tested PostGIS GiST + pgvector HNSW** — both extensions are L1 canonical Postgres extensions with active maintenance + multi-million production deployments + canonical OGC SFS compliance for PostGIS.
6. **Same filesystem tile storage as Cand 1** — zero migration cost on the raw-tile-bytes side.
**Negative-but-mitigable structural findings**:
7. **Heavier Postgres-extension dependency** — PostGIS 3.4 install footprint ~30-80 MB shared libraries + ~10-20 MB SRID/projection metadata catalog; pgvector 0.7+ ~5-10 MB shared library. **Vs Cand 1's zero-extension Postgres**, this is **~50-100 MB additional memory + ~50-200 MB additional disk install footprint**. **Mitigation**: well within AC-4.2 8 GB budget (essentially noise) and AC-8.3 10 GB cache budget (extension install lives in `/usr/lib/postgresql`, not in cache budget). **Real cost**: extra extension to maintain, version-pin, and verify ARM64 compatibility for at C7 inference-runtime + Jetson MVE phase.
8. **Geographic GiST index lookup ~5-10× slower than Cand 1's btree composite for the dominant 3 Hz spatial-grid query** — GiST lookup latency ~1-5 ms per Source #94 nyc_streets EXPLAIN evidence (`cost=0.28..79.58 rows=3`); Cand 1's btree lookup is ~0.1-0.5 ms. **Mitigation**: at 3 Hz query rate (333 ms budget per query inside AC-4.1 400 ms p95 envelope), the absolute latency difference (~1-5 ms vs 0.1-0.5 ms) is negligible — **but the relative slowdown is real**.
9. **pgvector HNSW dimension limit at full-precision**`vector` type HNSW supports up to **2,000 dimensions** per Source #95 pgvector README; for **MixVPR canonical 2048-D descriptors per Fact #18 cluster**, this **JUST EXCEEDS the limit**. **Mitigation**: use `halfvec_l2_ops` (half-precision, 2-byte storage, supports up to 16,000 dimensions) — cuts cache footprint by 50% AND clears the dimension limit; OR truncate to 1536-D (loses ~25% Recall@K); OR use 512-D EigenPlaces variant per D-C2-10 = (b) which is well within both pgvector limits AND smaller cache footprint.
10. **No empirically-verified Jetson Orin Nano Super deployment for PostGIS+pgvector combined stack** — Source #97 March 2026 article confirms Postgres + pgvector deployment but does not explicitly include PostGIS; Source #94 search results explicitly note absence of Jetson-specific PostGIS install evidence. **Mitigation**: D-C6-5 NEW gate — Jetson MVE phase per D-C1-2 must include PostGIS+pgvector co-installation + OLTP+spatial+ANN combined-query profiling on Jetson Orin Nano Super.
**Caveats / open Plan-phase decisions raised** (D-C6-N gates):
- **D-C6-5 NEW (Cand-2-only)** — Jetson PostGIS + pgvector co-installation Plan-phase verification choice (verify on Jetson MVE as part of D-C1-2 dedicated bring-up phase / fork PostGIS+pgvector ARM64 builds in-house if upstream packages incomplete / pivot to Cand 1 if PostGIS+pgvector co-installation reveals blocking incompatibility): trade-off between Plan-phase engineering investment vs documented evidence gap. **Recommendation**: D-C6-5 = (a) verify on Jetson MVE phase at D-C1-2 — already-required Jetson hardware bring-up cycle absorbs this work cheaply.
- **D-C6-6 NEW (Cand-2-only)** — pgvector descriptor-storage-type choice (`vector` full-precision float32 with 2,000-dim max for HNSW per Source #95 / `halfvec` half-precision 2-byte with 16,000-dim max + 50% cache savings + ~0-2% Recall@K loss / `sparsevec` for sparse descriptors / `bit` for binary descriptors via Hamming distance): trade-off between cache footprint vs accuracy vs descriptor compatibility with C2 VPR candidate output format. **Recommendation**: D-C6-6 = (b) `halfvec` for the primary path; covers all C2 VPR descriptor candidates (MixVPR 2048-D, SelaVPR 1024-D, NetVLAD 4096-D PCA-whitened, EigenPlaces 2048-D-or-smaller-via-D-C2-10, SALAD 8448-D/2112-D/544-D-via-D-C2-6) with consistent storage format.
- **D-C6-7 NEW (CROSS-COMPONENT — affects both Cand 1 and Cand 2)** — IF Cand 2 selected → cascade-changes-back-to-suite-satellite-provider strategy choice (cascade PostGIS+pgvector adoption back to satellite-provider for cross-suite consistency / keep satellite-provider on btree-only and gps-denied-onboard on PostGIS+pgvector — accept divergence / migrate satellite-provider to PostGIS+pgvector in a separate ticket post-MVP / leave satellite-provider unchanged + maintain compatibility shim in gps-denied-onboard's pre-flight cache-sync layer). **Recommendation**: per user's session-start clarification "if improvement is small, then there is no sense to change anything at all" — IF Cand 2's MATERIAL improvement justifies adoption, cascade via separate ticket; OTHERWISE stay with Cand 1 throughout the suite.
---
## C6 — Comparative-improvement-vs-Cand-1 analysis (closure of batch 1)
| Dimension | Cand 1 (mirror suite-pattern) | Cand 2 (PostGIS+pgvector) | Improvement magnitude (Cand 2 vs Cand 1) | Verdict per user's "significant-improvement-only" bar |
|---|---|---|---|---|
| **Geographic spatial-query API** | btree composite + app-side k-radius derivation + app-side distance sort | Native KNN `<->` + native `ST_DWithin` radius | **Material capability improvement** (Cand 2 supports radius queries natively) | ⚠️ Material — but **project's pinned use case is 3x3 grid lookup at fixed zoom** (per AC-3.x mission corridor); native radius queries are unused capability |
| **Combined geographic-+-descriptor query** | App-side round trip (~1-2 ms overhead) | Single SQL statement (~0.5 ms overhead) | **Marginal latency improvement** (~1 ms saving per query × 3 Hz = 3 ms/sec saving in absolute time) | ⚪ Marginal |
| **Geographic query latency** | ~0.1-0.5 ms btree lookup | ~1-5 ms GiST lookup | **NEGATIVE** — Cand 1 is 5-10× faster for the dominant query | 🔴 Cand 2 worse here |
| **Descriptor ANN latency** | ~1-3 ms FAISS HNSW (in-process) | ~1-3 ms pgvector HNSW (in-DB) | **No material difference** | ⚪ Tied |
| **Memory footprint** | Postgres + FAISS = ~700 MB-1.5 GB | Postgres + PostGIS + pgvector = ~700 MB-1.4 GB | **No material difference** | ⚪ Tied |
| **Cache-budget impact (AC-8.3)** | bytea 8 KB/tile (float32-2048D) | vector 8 KB/tile or halfvec 4 KB/tile | **Tied if both use halfvec / float16** | ⚪ Tied |
| **Engineering complexity** | ZERO new infrastructure (mirrors satellite-provider exactly) | TWO new Postgres extensions (PostGIS + pgvector) + ARM64 verification at Jetson MVE + descriptor-format conversion code | **NEGATIVE** — Cand 2 adds ~3-5 days engineering at Plan + Jetson MVE phases | 🔴 Cand 2 worse here |
| **Project-pattern alignment** | EXACT mirror of suite satellite-provider | DIVERGENT from suite satellite-provider; requires D-C6-7 NEW gate cascade decision | **NEGATIVE** — Cand 2 forces a cross-suite consistency decision | 🔴 Cand 2 worse here |
| **Operator re-loc hint (AC-3.4) handling** | Direct btree lookup at hint zoom + (x, y) | Direct ST_DWithin radius query at hint position + radius | **Tied — both handle it natively** | ⚪ Tied |
| **License clean-throughput** | PostgreSQL + FAISS-MIT + psycopg-LGPL/MIT-Apache | PostgreSQL + PostGIS-GPL2 + pgvector-PostgreSQL-License + psycopg | ⚠️ Cand 2 introduces PostGIS-GPL-2.0-or-later which may conflict with D-C1-1 license-posture choice if (b) BSD/permissive-only-track is selected | 🔴 Cand 2 worse here (subject to D-C1-1) |
**Closure verdict (per user's "significant-improvement-only" bar)**:
**Cand 1 (mirror suite-pattern) is the recommended primary path for C6**. Cand 2's improvements (native KNN, native radius queries, single-SQL combined query) are real BUT **the project's pinned 3 Hz spatial-grid query at fixed zoom does not exercise these capabilities** (per AC-3.x mission corridor + AC-1.x frame-center accuracy bars). Cand 2 is **5-10× slower for the dominant geographic query** AND **requires PostGIS+pgvector ARM64 Jetson MVE verification** AND **forces a cross-suite cascade decision (D-C6-7)** AND **may conflict with D-C1-1 license-posture choice (b)** due to PostGIS-GPL-2.0-or-later licensing. **The improvements are marginal-to-negative in the project's specific operating context — no material justification to deviate from the existing satellite-provider pattern.**
**Cand 2 promotion criteria (defer-to-Plan or Jetson-MVE)**: Cand 2 should be re-evaluated for promotion to primary IF AND ONLY IF (a) project use case expands to require radius-meters-based queries (e.g., dynamic mission corridor adjustment in flight) OR (b) Jetson MVE phase reveals Cand 1's app-side combined-query overhead is materially impacting AC-4.1 latency budget at the tail OR (c) D-C1-1 license-posture choice (a) GPL-3.0 track is selected AND the project elects to standardize on a single Postgres-extension stack for consistency.
---
## C6 — Working conclusions and decisions (compounded from Fact #92 + Fact #93 closures)
**Selected primary**: **Cand 1 (mirror suite-pattern)** — PostgreSQL btree composite on slippy-map `(tile_zoom, tile_x, tile_y, version)` + filesystem `./tiles/{zoom}/{x}/{y}.{image_type}` + bytea descriptor blobs + app-side FAISS HNSW loaded at takeoff. **Cand 2 (PostGIS+pgvector) deferred to defer-to-Plan or Jetson-MVE secondary** per the comparative analysis above.
**Decisions raised (D-C6-N gates)** — see [`../06_component_fit_matrix/99_cross_component_gates.md`](../06_component_fit_matrix/99_cross_component_gates.md):
- **D-C6-1** (Fact #92) — descriptor-storage-format choice: float32 / halfvec / INT8 — RECOMMENDED halfvec
- **D-C6-2** (Fact #92) — FAISS index variant choice: IndexFlatL2 / IndexHNSWFlat / IndexIVFFlat / IndexIVFPQ — RECOMMENDED IndexHNSWFlat M=32
- **D-C6-3** (Fact #92) — descriptor-cache-rebuild-trigger strategy: rebuild-on-modification / incremental-add / periodic-rebuild-during-C10-pre-flight — RECOMMENDED periodic-rebuild
- **D-C6-4** (Fact #92) — geographic-spatial-grid radius `k`: fixed-1 / fixed-2 / fixed-4 / dynamic-by-zoom-and-ground-speed — RECOMMENDED dynamic
- **D-C6-5** (Fact #93, Cand-2-only contingent) — Jetson PostGIS + pgvector co-installation Plan-phase verification choice — RECOMMENDED verify at Jetson MVE D-C1-2
- **D-C6-6** (Fact #93, Cand-2-only contingent) — pgvector descriptor-storage-type choice — RECOMMENDED halfvec
- **D-C6-7** (Fact #92 + Fact #93, CROSS-COMPONENT) — IF Cand 2 selected → cascade-changes-back-to-suite-satellite-provider strategy — RECOMMENDED cascade-via-separate-ticket OR stay-with-Cand-1 throughout
C6 batch 1 closed at 2/N. Subsequent C6 candidates (e.g., MBTiles single-sqlite-file, LMDB+geohash, FAISS-only-no-Postgres) deferable — current 2-candidate breadth satisfies engine Component Option Breadth rule for the user's pinned-Postgres scope.
@@ -0,0 +1,308 @@
# Fact Cards — C7: On-Jetson inference runtime
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Bound to sub-questions in `../00_question_decomposition.md` line 75 (C7 = "INT8/FP16 inference of the chosen VPR + matcher models within latency + memory budget"). Sources for C7 cluster live in [`../01_source_registry/C7_inference_runtime.md`](../01_source_registry/C7_inference_runtime.md).
>
> Index: [`00_summary.md`](00_summary.md). Sibling components: [C1 VIO](C1_vio.md), [C2 VPR](C2_vpr.md), [C3 Matchers](C3_matchers.md), [C4 Pose](C4_pose_estimation.md), [C5 State estimator](C5_state_estimator.md), [C6 Tile cache](C6_tile_cache_spatial_index.md). Cross-component gates: [`../06_component_fit_matrix/99_cross_component_gates.md`](../06_component_fit_matrix/99_cross_component_gates.md).
---
## Scope summary
C7 is a **cross-cutting integration row** rather than a per-component candidate row: it pins the on-Jetson inference runtime that hosts the C1 (learned-VIO frontends if any) + C2 VPR backbone + C3 matcher models. C7 batch 1 closed at 3/N on 2026-05-08 with three rows per the user-pinned scope (locked via `/autodev` AskQuestion 2026-05-08): **Fact #94** = TensorRT native primary (TensorRT 10.3 bundled with JetPack 6.2; `IInt8EntropyCalibrator2` calibration; `BuilderFlag.FP16` + `BuilderFlag.INT8` mixed-precision; engines built directly on Jetson SM 87). **Fact #95** = ONNX Runtime + TensorRT EP interop alternate (community-maintained Jetson AI Lab wheel index `pypi.jetson-ai-lab.io/jp6/cu126`; `TensorrtExecutionProvider` with `trt_fp16_enable` / `trt_int8_enable` config; subgraph fallback to CUDA EP / CPU EP). **Fact #96** = pure PyTorch FP16 mandatory simple-baseline (`torch.amp.autocast(device_type='cuda', dtype=torch.float16)`; `model.half()` eager-mode; PyTorch 2.5/2.9 wheels via Jetson AI Lab). Triton / DeepStream / CUDA-Python custom kernels noted-and-rejected in one sentence (server/video-pipeline class or out-of-budget for embedded 8 h mission). User-pinned `c7_quantization=A`: INT8 primary + FP16 fallback per candidate; INT8-only candidates marked Experimental until calibration data exists. **Critical caveat (raised by Source #103)**: feature-matching networks (LightGlue / DISK / XFeat) suffer material accuracy degradation under INT8/FP8 quantization vs FP16 — INT8 is the right primary axis for **VPR backbones** (CNN class) but **FP16 is the safer primary axis for matchers** (transformer class). This is captured in D-C7-6 INT8-vs-FP16-per-model-family-precision-policy.
---
### Fact #94 — TensorRT native primary: JetPack-bundled TensorRT 10.3 + IInt8EntropyCalibrator2 + BuilderFlag.FP16+INT8 mixed-precision; engines built directly on Jetson Orin Nano Super SM 87
**Statement**: The TensorRT native primary candidate for C7 uses the JetPack 6.2 bundled TensorRT 10.3 SDK (CUDA 12.6 + cuDNN 9.3 per Source #104) with the canonical INT8/FP16 mixed-precision build flow:
- **INT8 calibrator hierarchy** (per Source #99 context7-verified): `nvinfer1::IInt8Calibrator` (abstract base) + `nvinfer1::IInt8EntropyCalibrator2` (current canonical recommended algorithm, returns `kENTROPY_CALIBRATION_2`) + `nvinfer1::IInt8MinMaxCalibrator` (alternate for activations with bimodal distributions). Each implements `getBatchSize()` + `getBatch(void* bindings[], const char* names[], int32_t nbBindings)` + `readCalibrationCache(size_t& length)` + `writeCalibrationCache(const void* ptr, size_t length)` + `getAlgorithm()`.
- **Python builder INT8 enable pattern** (canonical TensorRT 10.x per Source #99):
```python
config.set_flag(trt.BuilderFlag.INT8)
config.int8_calibrator = Int8_calibrator
Int8_calibrator = EntropyCalibrator(["input_node_name"], batchstream)
```
- **Mixed-precision flag pattern**: `config.set_flag(trt.BuilderFlag.FP16)` + `config.set_flag(trt.BuilderFlag.INT8)` for combined FP16+INT8 mixed precision (TensorRT auto-selects per-layer precision based on calibration data; quantization-sensitive layers fall back to FP16, less-sensitive layers stay INT8).
- **Calibration data requirement**: ~500-1,500 representative input samples (UAV nadir frames at flight altitude over season-matched satellite tiles) — gates the INT8 build path (no calibration data → INT8 NOT achievable, FP16-only build is the fallback; see D-C7-1 + D-C7-6).
- **Engine build location**: per Source #105 constraint #2 + #3, engines MUST be built directly on the Jetson target (SM 87 = Ampere class). Laptop / dev-machine GPUs (e.g., RTX 4090 = SM 89) build engines that fail load with `Target GPU SM 87 is not supported by this TensorRT release`. Build-time memory pressure on the 8 GB shared budget caps `config.max_workspace_size` at ~1-2 GB to avoid tactic-profile segfaults (Source #105 constraint #4).
- **Install path**: per Source #105 constraint #1, `pip install tensorrt` is NOT supported on Jetson Tegra; the canonical install is the JetPack-bundled TensorRT (already present after `apt install nvidia-jetpack`), accessed via `/usr/lib/python3.10/dist-packages/tensorrt`. Upgrading TensorRT independently of JetPack is not officially supported.
**Mode pinning** (per-mode API verification rule):
- inputs: ONNX model graph (exported from PyTorch via `torch.onnx.export` on the dev machine) + a representative calibration dataset (NumPy `.npy` or Torch `.pt` tensors of shape `[N, C, H, W]` matching the model's expected input)
- outputs: serialized TensorRT engine `.engine` file (hardware-specific to SM 87 Jetson Orin Nano Super) + per-frame inference latency in the 3-8 ms range for CNN-backbone VPR networks at ~224×224-320×320 input (per Source #102 YOLO26n empirical benchmarks); inference accepts CUDA tensors via `IExecutionContext.execute_v2` Python API or the `enqueueV3` async path
- runtime: TensorRT 10.3 + CUDA 12.6 + cuDNN 9.3 on JetPack 6.2 + Jetson Orin Nano Super in Super Mode (per Source #104 — 70% AI TOPS increase + 50% memory bandwidth boost vs base mode)
**Source**:
- Primary API: Source #99 NVIDIA TensorRT 10.x official documentation (context7 indexed at `/websites/nvidia_deeplearning_tensorrt`, 9371 code snippets) — confirms `IInt8EntropyCalibrator2`, `BuilderFlag.INT8`, `BuilderFlag.FP16`, calibrator interface methods.
- Latency anchor: Source #102 Ultralytics YOLO26 benchmark suite on Jetson Orin Nano Super (April 2026) — TensorRT FP32 7.53 ms / FP16 4.57 ms / INT8 3.80 ms for YOLO26n (CNN object detector, ~3M parameters, 640×640 input).
- Software stack pin: Source #104 NVIDIA JetPack 6.2 release notes — TensorRT 10.3 + CUDA 12.6 + cuDNN 9.3 + Super Mode for Orin Nano production modules.
- Install constraints: Source #105`pip install tensorrt` not supported on Tegra; engines hardware-specific; build-on-target mandatory; memory-pressure during tactic profiling caps workspace size.
**Phase**: Mode A Phase 2 — engine Step 3 + Step 7.5 (Component Applicability Gate)
**Confidence**: ✅ High for the API capability verification (TensorRT INT8/FP16 build APIs are L1 official docs + 9371 context7 snippets); ⚠️ Medium-High for the latency claim on this specific project's models (YOLO benchmarks anchor CNN-class throughput; VPR networks like MixVPR/EigenPlaces are CNN-class similar-architecture and likely follow the same trend, but matcher networks like LightGlue/DISK/XFeat are transformer-class and known to deviate per Source #103); ⚠️ Medium for the "INT8 achievable for matchers" axis — Source #103 evidence shows LightGlue FP8 caused "match counts dropped sometimes hard", and INT8 is structurally similar to FP8 in dynamic-range reduction.
**Sub-Question Binding**:
- SQ3+SQ4 → C7 row in `../06_component_fit_matrix/C7_inference_runtime.md` (this fact populates the `TensorRT native primary` candidate row).
- SQ5 (failure modes) — feature-matching INT8 quantization-sensitivity is captured here as a NEW failure-mode line item.
**Implication / per-numbered-Restriction × per-numbered-AC sub-matrix**:
| Project Restriction / AC | Verdict | Evidence |
|---|---|---|
| **R-NEW-2 no cloud at flight** | ✅ PASS | TensorRT runtime is entirely local (CUDA-side execution on Jetson GPU); no network calls at inference time. |
| **R-NEW-4 Jetson Orin Nano Super JetPack 6 ARM64** | ✅ PASS | TensorRT 10.3 ships bundled in JetPack 6.2 ARM64 install; JetPack-bundled wheel is the canonical install path per Source #105. |
| **AC-1.1 (≤80 m at 1 km AGL)** | ✅ PASS | Inference accuracy is downstream of model selection (C2/C3); TensorRT runtime accuracy parity with PyTorch is documented at FP16 (typically <0.5% delta) and at INT8 with calibration data ranges from <1% (CNN backbones, e.g. Source #102 YOLO26n FP16 mAP 0.4800 vs INT8 0.4490 = -6.5% — concerning for matchers but acceptable for VPR backbones at Recall@K granularity). |
| **AC-1.2 (≤30 m at 500 m AGL)** | ✅ PASS | Same as above. |
| **AC-3.1 sharp turns ±20° bank** | ✅ PASS | TensorRT inference is deterministic; sharp-turn input frames are processed at the same latency as level-flight frames. |
| **AC-3.2 sharp-turn frames may share <5% overlap** | ✅ PASS | Matcher-side quantization-sensitivity (per Source #103) is the dominant concern, NOT runtime; D-C7-6 covers per-model-family precision policy (matchers FP16, VPR INT8). |
| **AC-3.3 re-localization stability** | ✅ PASS | TensorRT engine is deterministic (no randomness within a single inference; `IExecutionContext.execute_v2` is bit-exact reproducible across runs). |
| **AC-3.4 operator re-loc hint** | ✅ PASS | Operator-supplied hints affect cache lookup (C6) and pose initialization (C5), not C7 runtime. |
| **AC-4.1 latency budget (<400 ms p95 end-to-end)** | ✅ PASS | Per Source #102 empirical YOLO26n on Jetson Orin Nano Super: TensorRT FP16 4.57 ms / INT8 3.80 ms per inference. For the project's pipeline at 3 Hz (~333 ms budget per query), running C2 VPR (CNN, ~5-15 ms FP16/INT8 estimated for MixVPR/EigenPlaces ResNet50 at 224×224-320×320) + C3 LightGlue matcher per pair (~15-40 ms FP16 estimated, K=10 pairs = 150-400 ms — TIGHT, addressed by D-C3-3 K-pairs reduction) easily fits when matchers run FP16-only and VPR runs INT8. |
| **AC-4.2 memory budget (<8 GB shared on Jetson)** | ✅ PASS | TensorRT runtime memory: ~50-150 MB shared library + per-engine activation memory ~50-300 MB depending on model. Peak combined for VPR-engine + matcher-engine + executor context typically ~1-2 GB out of 8 GB shared budget. Engine build-time peak is ~3-5 GB (capped via `max_workspace_size` per D-C7-8) — must be done at pre-flight, NOT at flight time. |
| **AC-4.5 look-back refinement** | N/A | Inference runtime is forward-only; look-back is C5 estimator's responsibility. |
| **AC-8.3 10 GB persistent tile cache budget** | N/A | TensorRT engine `.engine` files are typically 10-200 MB each; 3-5 engines (VPR + matcher + optional VIO frontend) consume ~100-500 MB on disk — separate from the 10 GB cache budget (engines live in `/usr/local/lib/onboard/engines/`, not in tile cache). |
| **AC-NEW-3 (FDR)** | ✅ PASS | TensorRT inference latency + memory + per-layer profile recordable as FDR fields via `IExecutionContext.profiler` API. |
| **AC-NEW-4 covariance honesty** | N/A | Runtime is passive; covariance is C4/C5 responsibility. |
| **AC-NEW-7 cache-poisoning safety** | N/A | Runtime does not write to the tile cache. |
| **AC-NEW-8 blackout failsafe** | ✅ PASS | TensorRT inference does NOT trigger failsafe directly; a runtime crash (rare; TensorRT 10.x is production-stable) is caught by the supervising process and triggers C5 demotion to `dead_reckoned` per AC-NEW-8 escalation thresholds. |
**Strengths** (positive structural advantages):
1. **Native NVIDIA stack — fastest possible inference path**. TensorRT directly maps ONNX graph operations to fused CUDA kernels with hardware-aware scheduling on Ampere SM 87. Per Source #102 benchmarks, TensorRT FP16 is **1.65× faster than TensorRT FP32** and **~2× faster than pure-PyTorch FP16** at the YOLO26n class workload — this gap widens with larger models and is roughly preserved across CNN architectures.
2. **Mixed-precision per-layer auto-selection at INT8 build time** — sensitivity-based fallback to FP16 for layers that fail INT8 calibration tolerance (configured via `config.set_flag(trt.BuilderFlag.OBEY_PRECISION_CONSTRAINTS)` + per-layer `setOutputType` + `setPrecision`). This auto-mitigates the concern in Source #103 about feature-matching networks suffering severe INT8 degradation: TensorRT's calibrator can keep the matcher's attention layers at FP16 while quantizing convolutional preprocessing.
3. **JetPack-bundled — zero install friction**. Per Source #105, TensorRT 10.3 ships pre-installed with JetPack 6.2; no external pip dependency, no version-mismatch failure modes, no community wheel index dependency.
4. **Hardware-aware engine optimization** at build time (tactic search across kernel implementations selects the fastest for SM 87 specifically). This is unique to TensorRT — ONNX Runtime + TRT EP also produces TRT engines but with less control over the build flags.
5. **Production-mature** — TensorRT 10.x is the canonical NVIDIA production inference SDK with multi-million deployment footprint (auto driving / robotics / industrial) and structured release notes per JetPack version.
6. **Profile-driven debugging** via `IExecutionContext.profiler` API — per-layer latency + memory + precision visible at runtime, drives D-C7-6 calibration tuning loops.
**Negative-but-mitigable structural findings**:
7. **Engines are hardware-specific** — must be rebuilt on the Jetson target. Per Source #105 constraint #2 + #3, laptop-built engines fail load with `Target GPU SM 87 is not supported`. **Mitigation**: engine build is part of pre-flight cache provisioning (C10 row when opened), not a runtime concern. Engine builds typically take 30-300 sec per model and are persisted across flights via `IRuntime.deserializeCudaEngine` from disk.
8. **INT8 calibration requires representative dataset** — typically 500-1,500 input samples covering the deployment distribution. **Mitigation**: D-C7-1 closed at C7 batch 1 with calibration-corpus distribution = real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles (per the 2026-05-08 C9 / SQ7 restructure decision in `../00_question_decomposition.md`). Specific fixture-file pin delegated to Test Spec (greenfield Step 5). Candidate corpora carried forward to Test Spec: AerialVL S03 + AerialExtreMatch + project's own Mavic + Derkachi flight footage.
9. **Build-time memory pressure on 8 GB shared budget** — can segfault during tactic profiling per Source #105 constraint #4. **Mitigation**: cap `config.max_workspace_size` at 1-2 GB; build during pre-flight when no other workloads are active; serialize the engine for runtime deserialization.
10. **No per-mode pip install path** — requires JetPack-bundled TensorRT. **Mitigation**: project's deployment is JetPack-based by hardware constraint; no alternative install path is needed.
11. **Feature-matching INT8 quantization-sensitivity** (per Source #103: "match counts dropped sometimes hard" for FP8 LightGlue; INT8 is structurally similar). **Mitigation**: D-C7-6 INT8-vs-FP16-per-model-family-precision-policy — CNN-class models (VPR backbones MixVPR/EigenPlaces/SelaVPR-DINOv2) target INT8; transformer-class matchers (LightGlue / DISK / XFeat) target FP16; calibration data and per-layer precision overrides handled in build script.
**Caveats / open Plan-phase decisions raised** (D-C7-N gates):
- **D-C7-1 CLOSED IN C7 batch 1 (2026-05-08, per the C9 / SQ7 restructure user choice A in `../00_question_decomposition.md`)** — calibration-dataset-strategy. **Closure**: strategy = real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles as the calibration corpus distribution (matches the Project Constraint Matrix's "Inputs available" pinning + provides realistic noise/illumination/season distribution that the deployed system will see). Specific fixture-file pin (AerialVL S03 vs project's Mavic + Derkachi flight clips vs other corpora) is fixture-class and DELEGATED to Test Spec (greenfield Step 5). Synthetic-tile augmentation via random homography is the documented low-data fallback, only invoked if real flight footage is insufficient for Recall@K-target calibration. ~5001,500 representative samples per the C7 batch 1 INT8 build constraint. No Plan-phase Choose block remains.
- **D-C7-2 NEW** — TensorRT mixed-precision flag matrix per model family (VPR INT8+FP16 fallback / matchers FP16-only / VIO learned-frontends if any FP16-only). **Recommendation**: D-C7-2 = ladder per family; finalize at Jetson MVE phase per D-C1-2 + D-C7-6.
- **D-C7-7 NEW** — engine-build-on-Jetson-vs-prebuilt-engine-shipping strategy (build engines at pre-flight on the deployed Jetson / build engines on a known-good "reference Jetson" then ship the same `.engine` files to all production Jetsons / both — primary path build-on-target with reference-Jetson-built engines as a fallback if pre-flight build fails). **Recommendation**: D-C7-7 = primary build-on-deployed-Jetson during pre-flight (handles SM-version drift + future TensorRT minor version updates); fallback prebuilt engines for emergency provisioning.
- **D-C7-8 NEW**`config.max_workspace_size` cap to avoid tactic-profile segfault during build (1 GB safe default / 2 GB for richer kernel-fusion search / 3 GB for fastest-possible engine but high segfault risk on 8 GB budget). **Recommendation**: D-C7-8 = 1 GB safe default; raise to 2 GB only if Plan-phase Jetson MVE shows engine quality is materially worse at 1 GB.
- **D-C7-9 NEW** — TensorRT version pin within JetPack lifecycle (pin to JetPack 6.2's bundled TensorRT 10.3 / track JetPack 6.x minor releases / lock the exact JetPack point release for cross-deployment reproducibility). **Recommendation**: D-C7-9 = lock to JetPack 6.2 + TensorRT 10.3 for the project's first deployment; revisit at Plan-phase per JetPack release cadence.
---
### Fact #95 — ONNX Runtime + TensorRT EP interop alternate: onnxruntime-gpu via Jetson AI Lab JP6/CU126 wheel index + TensorrtExecutionProvider config + automatic CUDA EP / CPU EP subgraph fallback
**Statement**: The ONNX Runtime + TensorRT EP interop alternate candidate for C7 uses ONNX Runtime as the model-agnostic inference frontend with TensorRT as the kernel-execution backend, hosted on the Jetson via the community-maintained Jetson AI Lab wheel index:
- **Provider enumeration + config pattern** (canonical Python API per Source #100 context7-verified):
```python
import onnxruntime as ort
print(ort.get_available_providers())
tensorrt_options = {
'device_id': 0,
'trt_max_workspace_size': 1073741824, # 1 GB cap per D-C7-8
'trt_fp16_enable': True,
'trt_int8_enable': False, # see D-C7-6
'trt_engine_cache_enable': True,
'trt_engine_cache_path': '/var/cache/onboard/trt-engines',
}
cuda_options = {'device_id': 0, 'arena_extend_strategy': 'kNextPowerOfTwo'}
session = ort.InferenceSession(
"model.onnx",
providers=[
('TensorrtExecutionProvider', tensorrt_options),
('CUDAExecutionProvider', cuda_options),
'CPUExecutionProvider'
],
)
```
- **Provider-cascade behavior**: ORT TRT EP attempts to optimize each subgraph via TensorRT (subgraph = a maximal contiguous region of the ONNX graph whose ops are TRT-supported); falls back to CUDA EP for unsupported ops; falls back to CPU EP if neither GPU EP applies. Subgraph fallback is automatic and per-op transparent — operators that TRT does not support (rare custom ops, specialized attention variants) silently route through CUDA EP without runtime error.
- **Engine cache integration**: `trt_engine_cache_enable=True` + `trt_engine_cache_path` causes ORT TRT EP to serialize the per-subgraph TensorRT engines on first execution and reuse them on subsequent runs (~10-300 sec first-run cost amortized to <1 sec on subsequent loads). Same hardware-specificity constraint applies (engines tied to SM 87 — see Source #105 #2 + #3).
- **Install path (CRITICAL)**: per Source #100, standard `pip install onnxruntime-gpu` does NOT work on Jetson Tegra. The canonical install paths are:
- **JetPack 6 + CUDA 12.6 + Ubuntu 22.04 (project target)**: `pip3 install onnxruntime-gpu --index-url https://pypi.jetson-ai-lab.io/jp6/cu126`
- **JetPack 6 + CUDA 12.9 + Ubuntu 24.04 (alternate)**: `pip3 install onnxruntime-gpu --index-url https://pypi.jetson-ai-lab.io/jp6/cu129`
- **Known incompatibility**: onnxruntime-gpu v1.23.0 wheels for JetPack 6 were built against `numpy<2.0.0` (per Source #100 GitHub Issue #27562); importing under `numpy>=2.0.0` raises a compatibility error. Project requirements file MUST pin `numpy<2.0.0` until upstream rebuild.
- **Provider availability gate**: standard `pip install onnxruntime` (no `-gpu` suffix) installs the CPU-only build that exposes ONLY `CPUExecutionProvider` and `AzureExecutionProvider` — does NOT include CUDA EP or TensorRT EP. Project provisioning script must verify `'TensorrtExecutionProvider' in ort.get_available_providers()` at startup.
**Mode pinning** (per-mode API verification rule):
- inputs: ONNX model graph (any source — PyTorch via `torch.onnx.export`, TensorFlow via `tf2onnx`, vendor-shipped ONNX) + per-session config dict for TRT EP / CUDA EP / CPU EP fallback ladder
- outputs: `ort.InferenceSession.run(output_names, input_feed)` — accepts NumPy arrays as input (auto-marshaled to GPU tensors at the EP boundary); per-session subgraph engine cache persisted to disk for fast warm-start
- runtime: onnxruntime-gpu (community-maintained Jetson AI Lab build) + JetPack-bundled TensorRT 10.3 + CUDA 12.6 + Python 3.10 on Jetson Orin Nano Super in Super Mode
**Source**:
- Primary API: Source #100 Microsoft ONNX Runtime official documentation (context7 indexed at `/microsoft/onnxruntime` v1.25.0, 1462 code snippets at Benchmark Score 82.23 — highest of the three C7 candidate context7 lookups).
- Jetson install path: Source #100 dusty-nv/jetson-containers Issue #1283 + microsoft/onnxruntime Issue #20503 — confirms Jetson AI Lab wheel index as canonical install for JetPack 6.
- NumPy incompatibility: Source #100 microsoft/onnxruntime Issue #27562 — onnxruntime-gpu v1.23.0 JetPack 6 wheels built with `numpy<2.0.0`.
- Software stack pin: Source #104 — JetPack 6.2 ships TensorRT 10.3, which ORT TRT EP delegates to.
**Phase**: Mode A Phase 2 — engine Step 3 + Step 7.5 (Component Applicability Gate)
**Confidence**: ✅ High for the API capability verification (1462 context7 code snippets at Benchmark Score 82.23); ⚠️ Medium for the deployability claim — community-maintained wheels (Jetson AI Lab) carry slightly higher version-drift risk than JetPack-bundled TensorRT, plus the documented `numpy<2.0.0` pin limits forward-compatibility. Plan-phase Jetson MVE per D-C1-2 + D-C7-3 must validate the exact ORT version + numpy pin.
**Sub-Question Binding**:
- SQ3+SQ4 → C7 row in `../06_component_fit_matrix/C7_inference_runtime.md` (this fact populates the `ONNX Runtime + TensorRT EP` candidate row).
**Implication / per-numbered-Restriction × per-numbered-AC sub-matrix**:
| Project Restriction / AC | Verdict | Evidence |
|---|---|---|
| **R-NEW-2 no cloud at flight** | ✅ PASS | ONNX Runtime + TRT EP runtime is entirely local. |
| **R-NEW-4 Jetson Orin Nano Super JetPack 6 ARM64** | ⚠️ PASS-with-Plan-phase-verification | onnxruntime-gpu prebuilt aarch64 wheels NOT published by Microsoft (per Source #100 Issue #20503); canonical install requires Jetson AI Lab community wheel index `pypi.jetson-ai-lab.io/jp6/cu126`. Microsoft Issues acknowledge the gap; community wheels are widely used in the Jetson ecosystem but are NOT officially-supported by Microsoft. Plan-phase Jetson MVE per D-C7-3 must verify that the wheel index is reachable in the project's offline-deployment context (likely requires pre-flight wheel-cache-mirroring). |
| **AC-1.1 (≤80 m at 1 km AGL)** | ✅ PASS | Inference accuracy parity with TensorRT-native (ORT TRT EP delegates to TensorRT for supported subgraphs). |
| **AC-1.2 (≤30 m at 500 m AGL)** | ✅ PASS | Same as above. |
| **AC-3.1 sharp turns ±20° bank** | ✅ PASS | Same deterministic-inference profile as TensorRT-native. |
| **AC-3.2 sharp-turn frames may share <5% overlap** | ✅ PASS | Same as TensorRT-native — quantization-sensitivity is model-family-dependent, not runtime-dependent. |
| **AC-3.3 re-localization stability** | ✅ PASS | Engine-cache deserialization is deterministic; bit-exact reproducibility across runs once engines are warm. First-run subgraph compilation can take 10-300 sec per model (one-time cost; engines persisted to `trt_engine_cache_path`). |
| **AC-3.4 operator re-loc hint** | ✅ PASS | Operator hint affects C5/C6, not C7. |
| **AC-4.1 latency budget (<400 ms p95 end-to-end)** | ⚠️ TIGHT-BUT-FITS | After warm cache, per-inference latency is essentially TensorRT-native + a small ORT framework overhead (~1-3 ms per session.run() call for input marshaling and provider dispatch). Per Source #100 ORT provider-cascade behavior, op-level fallback to CUDA EP for unsupported subgraphs adds latency vs pure-TRT — but for canonical CNN VPR backbones (MixVPR/EigenPlaces) and matchers with TRT-supported attention (LightGlue with FlashAttentionV2 plugin per Source #103), full TRT-EP coverage is achievable. Cold-start cost (~10-300 sec for first-run engine build) is paid once per Jetson per model — handled at pre-flight per D-C7-7. |
| **AC-4.2 memory budget (<8 GB shared on Jetson)** | ✅ PASS | ORT runtime memory: ~30-100 MB framework + ~50-150 MB CUDA EP + per-engine activation memory (delegated to TRT). Peak combined for VPR-engine + matcher-engine + ORT runtime typically ~1-2 GB out of 8 GB shared budget, slightly heavier than TensorRT-native (Fact #94) but within the same order of magnitude. |
| **AC-4.5 look-back refinement** | N/A | Forward-only inference. |
| **AC-8.3 10 GB persistent tile cache budget** | N/A | ORT engine cache (~100-500 MB total across 3-5 models) lives in `trt_engine_cache_path`, not in the tile cache budget. |
| **AC-NEW-3 (FDR)** | ✅ PASS | ORT exposes per-session profiling via `SessionOptions.enable_profiling=True``session.end_profiling()` returns a JSON profile file with per-op latency. |
| **AC-NEW-4 covariance honesty** | N/A | Runtime is passive. |
| **AC-NEW-7 cache-poisoning safety** | N/A | Runtime does not write to the tile cache. |
| **AC-NEW-8 blackout failsafe** | ✅ PASS | A runtime crash is caught by the supervising process and triggers C5 demotion. |
**Strengths** (positive structural advantages over TensorRT-native):
1. **Model-format-agnostic** — ONNX is the de-facto interchange format; PyTorch / TensorFlow / JAX / scikit-learn / vendor models all export to ONNX with high fidelity. Avoids the per-framework export friction of pure TensorRT (which historically requires specific UFF/Caffe parsers OR ONNX-then-TRT-builder).
2. **Subgraph fallback to CUDA EP / CPU EP for unsupported ops** — robust to model-architecture additions that TensorRT does not yet support natively (rare custom attention variants, specialized aggregations). TensorRT-native fails to build the engine in these cases; ORT TRT EP gracefully degrades to CUDA EP.
3. **Engine-cache integration** — per-subgraph engines are auto-built on first run and persisted; subsequent runs warm-start in <1 sec. Eliminates the explicit `trtexec` build step from the deployment workflow.
4. **Cross-architecture portability of the source code** — the same Python inference script runs on the dev machine (CUDA EP only) and on the Jetson (TensorRT EP + CUDA EP); no Jetson-specific code paths required.
5. **Active Microsoft maintenance** — context7 v1.25.0 confirmed at Benchmark Score 82.23 (highest of the three C7 candidate lookups); ORT release cadence is monthly with NVIDIA-sponsored TRT EP improvements.
**Negative-but-mitigable structural findings** (over TensorRT-native):
6. **Jetson install requires community wheel index** (per Source #100 Issue #20503) — adds an external dependency NOT officially supported by Microsoft. **Mitigation**: pre-flight provisioning mirrors the Jetson AI Lab wheel index to a project-controlled artifact registry (~50 MB per wheel set); offline deployment is then self-contained.
7. **NumPy <2.0.0 pin** for onnxruntime-gpu v1.23.0 JetPack 6 wheels (per Source #100 Issue #27562) — restricts forward-compatibility with downstream packages that require NumPy 2.x. **Mitigation**: project requirements file pins `numpy<2.0.0`; track upstream rebuild via `microsoft/onnxruntime` release notes for the version bump that resolves this.
8. **Slight runtime overhead vs TensorRT-native** (~1-3 ms per `session.run()` call for input marshaling and provider dispatch) — material at the per-frame budget but small relative to the total ~5-15 ms VPR + ~15-40 ms matcher per pair. **Mitigation**: at the project's 3 Hz frame rate the absolute overhead is ~3-9 ms/sec, well within the AC-4.1 400 ms p95 budget.
9. **First-run subgraph build cost** (10-300 sec per model) — silent at runtime if `trt_engine_cache_path` doesn't exist. **Mitigation**: pre-flight provisioning script builds the cache by running a synthetic warm-up batch through each model; runtime startup then warm-loads in <1 sec.
10. **Less direct control over TRT build flags** vs TensorRT-native — ORT TRT EP exposes a curated subset of flags via `tensorrt_options` (`trt_fp16_enable`, `trt_int8_enable`, `trt_max_workspace_size`, etc.); fine-grained per-layer precision policy (e.g., `setPrecision` overrides per node) requires the explicit TensorRT API. **Mitigation**: the curated subset covers the C7 user-pinned scope (`c7_quantization=A`); per-model-family precision policy is captured in D-C7-6 + handled via `trt_int8_enable` per-engine flag toggling.
**Caveats / open Plan-phase decisions raised** (D-C7-N gates):
- **D-C7-3 NEW (Cand-2 specific)** — ORT-Jetson-wheel-index-pin choice (`pypi.jetson-ai-lab.io/jp6/cu126` for JetPack 6.2 / `pypi.jetson-ai-lab.io/jp6/cu129` for JetPack 6.x with newer CUDA / mirror the wheel index to a project-controlled artifact registry for offline-deployment robustness). **Recommendation**: D-C7-3 = mirror to project artifact registry (~50 MB per wheel set; pre-flight provisioning step) + cu126 variant for JetPack 6.2 alignment.
- **D-C7-4 NEW (Cand-2 specific)** — numpy-version-pin choice (`numpy<2.0.0` per Source #100 Issue #27562 / wait for upstream onnxruntime-gpu rebuild against numpy>=2 / pin to a specific onnxruntime-gpu version known to work with numpy<2). **Recommendation**: D-C7-4 = `numpy<2.0.0` until upstream rebuild; track Issue #27562 status at Plan phase.
---
### Fact #96 — Pure PyTorch FP16 mandatory simple-baseline: torch.amp.autocast + model.half() + Jetson AI Lab PyTorch 2.x ARM64 wheel
**Statement**: The pure PyTorch FP16 mandatory simple-baseline candidate for C7 uses PyTorch's native AMP (Automatic Mixed Precision) machinery as the deployment baseline — no ONNX export, no TensorRT engine build, no engine cache. The role is **mandatory simple-baseline** per the engine's Component Option Breadth rule (always have a runnable fallback) and per the user-pinned `c7_breadth=B` scope (TensorRT primary + ONNX Runtime+TRT EP alternate + pure PyTorch FP16 baseline):
- **`torch.amp.autocast(device_type, dtype, enabled, cache_enabled)`** (canonical AMP context manager since PyTorch 1.10 per Source #101 context7-verified):
```python
with torch.no_grad():
with torch.autocast(device_type='cuda', dtype=torch.float16, enabled=True):
output = model(input)
```
Auto-selects per-op precision: matmul / conv / linear at FP16; layer-norm / softmax / accumulators stay FP32 for numerical stability.
- **`model.half()`** — eager-mode FP16 weight conversion (full-precision FP16 throughout, simpler but loses autocast's per-op precision auto-selection):
```python
model = model.half().cuda().eval()
output = model(input.half().cuda())
```
Matches the canonical `model.half()` deployment pattern documented in PyTorch eager-mode FP16 inference recipes.
- **`torch.compile(model, backend='inductor')`** — graph-mode optimization for further speedup; tradeoff is cold-start compile cost (~10-60 sec). Per Source #101, `inductor` is the default backend; `cudagraphs` for static-shape inference; `ipex` for Intel CPU. The Jetson Orin Nano Super CUDA path uses `inductor`.
- **Install path (Jetson)**: per Source #101 NVIDIA Developer Forum threads, standard `pip install torch` does NOT include CUDA support on Jetson — must use NVIDIA-published or Jetson AI Lab community wheels:
- **JetPack 6.2 + CUDA 12.6 + Ubuntu 22.04 + Python 3.10 canonical**: `torch-2.9.0-cp310-cp310-linux_aarch64.whl` from Jetson AI Lab (alternative stable: PyTorch 2.5 + torchvision 0.20).
- **CUDA capability**: Jetson Orin Nano Super GPU = **SM 87** (Ampere class). PyTorch wheels must be built against CUDA 12.6 to match JetPack 6.2's CUDA toolchain.
- **Known dependency issues**: missing `libcudss.so.0` and `libnvdla_runtime.so` on PyTorch 2.9 cu129 wheel under JetPack 6.2 (CUDA 12.6) — version-mismatch between wheel build target and installed JetPack CUDA. Mitigation: prefer the cu126 variant for JetPack 6.2.
**Mode pinning** (per-mode API verification rule):
- inputs: in-process Python PyTorch model (`torch.nn.Module`) loaded from a checkpoint (`torch.load(path)`) at startup; input tensors as `torch.Tensor` on CUDA device
- outputs: forward-pass result tensor in FP16 (autocast) or FP16-end-to-end (`model.half()`); per-frame inference latency in the **15-40 ms range for CNN VPR networks** (extrapolated from Source #102 YOLOv8s on Jetson Orin Nano FP16 ~9.7 ms = TensorRT FP16; PyTorch FP16 typically ~1.5-2× slower than TensorRT FP16 due to no kernel fusion)
- runtime: Jetson AI Lab PyTorch 2.5 / 2.9 ARM64 wheel + Python 3.10 + CUDA 12.6 on Jetson Orin Nano Super in Super Mode
**Source**:
- Primary API: Source #101 PyTorch official documentation (context7 indexed at `/pytorch/pytorch` v2.5.1 / v2.8.0 / v2.9.1 / v2.11.0; 4866 code snippets at Benchmark Score 76.69) — confirms `torch.amp.autocast`, `torch.no_grad`, `torch.compile`, `model.half()`.
- Jetson install path: Source #101 NVIDIA Developer Forum threads (multiple) — confirms Jetson AI Lab as canonical wheel source for JetPack 6.x; documents `libcudss.so.0` / `libnvdla_runtime.so` dependency issues on cu129 vs cu126 variants.
- Latency anchor (relative): Source #102 — pure PyTorch FP16 typically ~1.5-2× slower than TensorRT FP16 at the same workload; extrapolation from YOLOv8s 9.7 ms FP16 TRT → ~15-20 ms pure PyTorch FP16 on Orin Nano Super.
**Phase**: Mode A Phase 2 — engine Step 3 + Step 7.5 (Component Applicability Gate)
**Confidence**: ✅ High for the API capability verification (4866 context7 snippets); ⚠️ Medium for the latency claim (extrapolated from YOLO benchmarks; PyTorch eager-mode latency is more variable across model architectures than TensorRT's). Plan-phase Jetson MVE per D-C1-2 produces the actual Pure-PyTorch-FP16 latency numbers per project model.
**Sub-Question Binding**:
- SQ3+SQ4 → C7 row in `../06_component_fit_matrix/C7_inference_runtime.md` (this fact populates the `pure PyTorch FP16` mandatory simple-baseline candidate row).
**Implication / per-numbered-Restriction × per-numbered-AC sub-matrix**:
| Project Restriction / AC | Verdict | Evidence |
|---|---|---|
| **R-NEW-2 no cloud at flight** | ✅ PASS | PyTorch runtime is entirely local. |
| **R-NEW-4 Jetson Orin Nano Super JetPack 6 ARM64** | ⚠️ PASS-with-Plan-phase-verification | PyTorch ARM64 wheels not officially distributed by PyTorch Foundation for Jetson; canonical install via Jetson AI Lab community + NVIDIA Developer Forum recommendations. Same community-wheel-index dependency as ORT (Fact #95) but with broader community footprint (PyTorch on Jetson is a well-trodden path). |
| **AC-1.1 (≤80 m at 1 km AGL)** | ✅ PASS | FP16 inference accuracy parity with FP32 is documented at <0.5% delta for CNN backbones; matchers (transformer-class) are FP16-stable at production grade per Source #103 (FP8 caused degradation, but FP16 did not). |
| **AC-1.2 (≤30 m at 500 m AGL)** | ✅ PASS | Same as above. |
| **AC-3.1 sharp turns ±20° bank** | ✅ PASS | Eager-mode PyTorch is deterministic at fixed seed; per-frame inference is bit-exact reproducible. |
| **AC-3.2 sharp-turn frames may share <5% overlap** | ✅ PASS | Runtime-agnostic; quantization-sensitivity does not apply to FP16 baseline (only INT8). |
| **AC-3.3 re-localization stability** | ✅ PASS | No engine cache or compilation step; consistent per-frame latency from first frame onward. |
| **AC-3.4 operator re-loc hint** | ✅ PASS | Hint affects C5/C6, not C7. |
| **AC-4.1 latency budget (<400 ms p95 end-to-end)** | ⚠️ TIGHT — likely fails for full pipeline | Pure PyTorch FP16 is ~1.5-2× slower than TensorRT FP16 per Source #102 extrapolation. For VPR (~15-20 ms) + matcher (~30-80 ms per pair × K=10 = 300-800 ms) the matcher cost alone exceeds the AC-4.1 400 ms p95 budget at K=10. **Mitigation**: D-C3-3 K-pairs reduction (K=3-5) brings matcher cost to ~90-400 ms — TIGHT but possibly within budget for K=3-4 at the cost of recall. **Pure PyTorch FP16 is the FALLBACK runtime, NOT the primary**; the primary is TensorRT (Fact #94). |
| **AC-4.2 memory budget (<8 GB shared on Jetson)** | ✅ PASS | PyTorch runtime: ~500 MB-1 GB framework (CUDA + cuDNN libraries shared with all CUDA processes) + per-model weight memory (~50-300 MB for VPR + ~20-100 MB for LightGlue at 1024 keypoints); peak combined ~1-2 GB out of 8 GB shared budget. |
| **AC-4.5 look-back refinement** | N/A | Forward-only inference. |
| **AC-8.3 10 GB persistent tile cache budget** | N/A | PyTorch checkpoints (~50-300 MB per model × 3-5 models = ~150-1.5 GB total) live in `/var/cache/onboard/checkpoints/`, not in tile cache. |
| **AC-NEW-3 (FDR)** | ✅ PASS | PyTorch has `torch.profiler.profile` for per-op latency profiling; integrates naturally with the FDR data plane. |
| **AC-NEW-4 covariance honesty** | N/A | Runtime is passive. |
| **AC-NEW-7 cache-poisoning safety** | N/A | Runtime does not write to the tile cache. |
| **AC-NEW-8 blackout failsafe** | ✅ PASS | A runtime crash is caught by the supervising process and triggers C5 demotion. |
**Strengths** (positive structural advantages — for the simple-baseline role):
1. **Zero export friction** — model is loaded directly from PyTorch checkpoint; no ONNX export, no TensorRT engine build, no engine cache. Fastest path from "model trained" to "model running on Jetson".
2. **Trivial debugging** — full PyTorch eager-mode visibility (set breakpoints, inspect intermediate tensors, swap modules at runtime). Critical for the **mandatory simple-baseline** role: when a TensorRT-built engine produces unexpected output, the pure-PyTorch baseline is the reference for accuracy parity verification.
3. **Production-mature framework** — PyTorch is the de-facto research and deployment ML framework with daily-active maintenance. Jetson AI Lab wheels track upstream PyTorch releases at ~1-3 month lag.
4. **No INT8 calibration required** — FP16 baseline path is calibration-free; runs as soon as the checkpoint is loaded. This is the **fallback path** when INT8 calibration data is unavailable (D-C7-1 not yet resolved).
5. **`torch.compile` available** for additional optimization — Inductor backend can close 30-50% of the gap to TensorRT for certain models (per PyTorch Foundation benchmarks); first-call cost is ~10-60 sec vs zero for eager-mode.
6. **Same source code on dev machine and Jetson** — fully cross-architecture portable; no separate build step.
**Negative-but-mitigable structural findings**:
7. **~1.5-2× slower than TensorRT FP16** at the same workload (per Source #102 extrapolation). Material for the project's tight AC-4.1 budget — **DISQUALIFIES pure PyTorch FP16 from the primary path**, restricts it to the simple-baseline role + dev-machine reference role + emergency-fallback role if TensorRT engine build fails on the deployed Jetson.
8. **Jetson AI Lab wheel dependency** — same community-wheel-index concern as Fact #95. **Mitigation**: pre-flight wheel mirror + project-controlled artifact registry.
9. **No per-layer precision auto-selection at INT8** — INT8 path requires explicit quantization (e.g., `torch.quantization.quantize_dynamic` or PyTorch FX-graph-mode quantization); these do NOT use TensorRT INT8 calibrators. **Implication**: pure PyTorch INT8 is NOT a project-applicable path (out-of-scope for c7_quantization=A scope which targets TensorRT INT8 calibration); pure PyTorch is **FP16-only baseline** for this project.
**Caveats / open Plan-phase decisions raised** (D-C7-N gates):
- **D-C7-5 NEW (Cand-3 specific)** — PyTorch-Jetson-wheel-pin choice (PyTorch 2.5 + torchvision 0.20 stable / PyTorch 2.9 + torchvision latest / track Jetson AI Lab cadence). **Recommendation**: D-C7-5 = PyTorch 2.5 + torchvision 0.20 for the project's first deployment (most-stable combination per NVIDIA Developer Forum); revisit at Plan phase based on Jetson MVE results.
---
## C7 — Cross-cutting model-family precision policy (closure of batch 1)
**The user-pinned `c7_quantization=A` scope is INT8 primary + FP16 fallback per candidate; INT8-only candidates marked Experimental until calibration data exists.** Combining this with Source #103 evidence on feature-matching-network INT8 quantization-sensitivity, the closure recommendation is a **per-model-family precision policy** (D-C7-6):
| Model family | Project models | Recommended precision (TensorRT-native primary, Fact #94) | Recommended precision (ORT TRT EP alternate, Fact #95) | Recommended precision (PyTorch FP16 baseline, Fact #96) | Rationale |
|---|---|---|---|---|---|
| **VPR backbones (CNN class)** | MixVPR, EigenPlaces, NetVLAD | INT8 + FP16 mixed (auto-fallback to FP16 for sensitive layers) | `trt_int8_enable=True` + per-engine calibration cache | FP16 only (no INT8 path) | YOLO-class CNN benchmarks (Source #102) confirm INT8 well-tolerated at -6.5% mAP50-95; for VPR Recall@K granularity this typically translates to <-2% R@1 = acceptable |
| **VPR backbones (ViT-class)** | SelaVPR (DINOv2-L), conditional AnyLoc/BoQ/DINOv2-VLAD | FP16 + Plan-phase D-C2-5 verification | `trt_fp16_enable=True` only; INT8 deferred to Jetson MVE | FP16 only | DINOv2 ViT export to TensorRT INT8 is a Plan-phase gate per D-C2-5; defer INT8 until Jetson MVE confirms acceptable Recall@K loss |
| **Matchers (transformer class)** | LightGlue (with SP / DISK / ALIKED), XFeat, XFeat+LighterGlue | FP16 only (NO INT8) | `trt_fp16_enable=True` only; INT8 explicitly disabled | FP16 only | Source #103 evidence: FP8 (similar dynamic-range reduction to INT8) on LightGlue causes "match counts dropped sometimes hard". Matchers stay FP16 throughout |
| **Learned VIO frontends** (if any selected at C1 closure) | DPVO, learned-front-end VINS | FP16 only initially; INT8 deferred to Jetson MVE per D-C7-2 | FP16 only initially | FP16 only | Insufficient INT8-on-VIO empirical evidence at research time; conservative FP16 default, revisit at Jetson MVE |
**Closure verdict (per user's `c7_quantization=A` scope + Source #103 caveat)**:
- **TensorRT-native (Fact #94) is RECOMMENDED PRIMARY** for VPR backbones (CNN-class INT8) AND matchers (FP16-only). Matches the user-pinned scope exactly: INT8 primary + FP16 fallback per candidate; matcher-class candidates marked Experimental for INT8 (D-C7-6 pinning FP16 as the matcher's locked precision).
- **ONNX Runtime + TensorRT EP (Fact #95) is RECOMMENDED ALTERNATE** for the cross-architecture-portability axis; same precision policy as TensorRT-native. Switch to ORT if model-export friction with TensorRT-native arises.
- **Pure PyTorch FP16 (Fact #96) is RECOMMENDED MANDATORY SIMPLE-BASELINE** — required for the engine's Component Option Breadth rule + dev-machine reference parity + emergency-fallback if TensorRT engine build fails on the deployed Jetson. **Pure PyTorch FP16 is NOT eligible for the primary path** due to ~1.5-2× latency penalty vs TensorRT FP16 (per Source #102 extrapolation) which exceeds the AC-4.1 400 ms p95 budget for the full pipeline.
---
## C7 — Working conclusions and decisions (compounded from Fact #94 + Fact #95 + Fact #96 closures)
**Selected primary**: **Fact #94 TensorRT native primary** — JetPack-bundled TensorRT 10.3 + IInt8EntropyCalibrator2 + BuilderFlag.FP16+INT8 mixed-precision; per-model-family precision policy per D-C7-6 (VPR INT8+FP16 fallback, matchers FP16-only).
**Selected alternate**: **Fact #95 ONNX Runtime + TensorRT EP interop alternate** — eligible if cross-architecture portability axis becomes important OR if TensorRT-native model export friction arises; same precision policy as primary.
**Selected mandatory simple-baseline**: **Fact #96 pure PyTorch FP16** — required for the engine's Component Option Breadth rule; dev-machine reference parity + emergency-fallback role only.
**Decisions raised (D-C7-N gates)** — see [`../06_component_fit_matrix/99_cross_component_gates.md`](../06_component_fit_matrix/99_cross_component_gates.md):
- **D-C7-1** (Fact #94) — calibration-dataset-strategy — **CLOSED IN C7 batch 1 (2026-05-08, per C9 / SQ7 restructure)**: strategy = real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles; specific fixture-file pin delegated to Test Spec (Step 5); synthetic-tile augmentation as documented low-data fallback. No Plan-phase Choose block remains.
- **D-C7-2** (Fact #94) — TensorRT mixed-precision flag matrix per model family — RECOMMENDED ladder per D-C7-6 policy
- **D-C7-3** (Fact #95) — ORT-Jetson-wheel-index-pin choice — RECOMMENDED mirror to project artifact registry + cu126 variant
- **D-C7-4** (Fact #95) — numpy-version-pin choice — RECOMMENDED `numpy<2.0.0` until upstream rebuild
- **D-C7-5** (Fact #96) — PyTorch-Jetson-wheel-pin choice — RECOMMENDED PyTorch 2.5 + torchvision 0.20
- **D-C7-6** (NEW from C7 batch 1 closure, CROSS-COMPONENT with C2 + C3 + C1) — INT8-vs-FP16-per-model-family-precision-policy — RECOMMENDED per the table in "Cross-cutting model-family precision policy" section above
- **D-C7-7** (Fact #94) — engine-build-on-Jetson-vs-prebuilt-engine-shipping strategy — RECOMMENDED build-on-deployed-Jetson at pre-flight + prebuilt fallback
- **D-C7-8** (Fact #94) — `config.max_workspace_size` cap — RECOMMENDED 1 GB safe default
- **D-C7-9** (Fact #94) — TensorRT version pin within JetPack lifecycle — RECOMMENDED lock to JetPack 6.2 + TensorRT 10.3
**C7 batch 1 closed at 3/N on 2026-05-08**. Subsequent C7 candidates (NVIDIA Triton, NVIDIA DeepStream, CUDA-Python custom kernels) are noted-and-rejected per the user-pinned `c7_overkill_options=A` scope: Triton + DeepStream are server / video-pipeline class with deployment footprints (~500 MB-2 GB) that exceed the project's embedded budget without delivering proportional benefit; CUDA-Python custom kernels would require ~2-4 weeks of CUDA engineering per model with marginal speedup over TensorRT's hardware-aware tactic search. Further candidate evaluation only if Plan-phase Jetson MVE reveals TensorRT-native + ORT TRT EP do not satisfy AC-4.1 latency budget — at which point CUDA-Python custom kernels for the matcher's inner loop become a NEW candidate (separate session).
@@ -0,0 +1,277 @@
# Fact Cards — C8: MAVLink / MSP2 FC adapter
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Sources logged in [`../01_source_registry/C8_fc_adapter.md`](../01_source_registry/C8_fc_adapter.md). Per-fact mode-pinning in **bold**; per-numbered-Restriction × per-numbered-AC sub-matrix below each Fact's `**Implication**` block where relevant. Confidence labels: ✅ High (L1 / verified source code), ⚠️ Medium (L1/L2 with caveat), ❓ Low (L3/L4 inferential).
>
> Index: [`../00_summary.md`](../00_summary.md). Prior cross-cuts: [SQ6 external positioning](SQ6_fc_external_positioning.md) — established the per-FC adapter design at SQ6 closure (Facts #1#10), which C8 batch 1 candidate rows now operationalize. Sibling component categories: [C1 VIO](C1_vio.md), [C2 VPR](C2_vpr.md), [C3 Matchers](C3_matchers.md), [C4 Pose](C4_pose_estimation.md), [C5 State estimator](C5_state_estimator.md), [C6 Tile cache](C6_tile_cache_spatial_index.md), [C7 Inference runtime](C7_inference_runtime.md).
## Scope summary
C8 batch 1 evaluates THREE candidate adapter implementations after the c8_inav_recovery=B mid-batch correction (preserves locked SQ6 + AC-4.3 + restrictions.md verdict that MSP2_SENSOR_GPS is the iNav primary, with UBX impersonation as comparative-improvement-evaluable alternate; per-FC-breadth narrowest at one ArduPilot candidate + two iNav candidates):
| # | Candidate | FC | Transport | License | Status (per Fit Matrix) |
|---|---|---|---|---|---|
| 1 | **pymavlink → MAVLink GPS_INPUT (msg 232)** | ArduPilot Plane | MAVLink over UART/USB/UDP | LGPL-3.0 (linkable from Apache-2.0 app per LGPL §6) | Mandatory primary + RECOMMENDED PRIMARY (cooperative-path, SQ6 Fact #1 lead) |
| 2 | **MSP2_SENSOR_GPS (id 7939 / 0x1F03) via Python MSP2 (YAMSPy or INAV-Toolkit msp_v2_encode)** | iNav | MSP V2 over UART/USB | MIT (libraries) | Mandatory primary + RECOMMENDED PRIMARY (cooperative-path, SQ6 Fact #6 lead) |
| 3 | **UBX impersonation via pyubx2 NAV-PVT (forged u-blox frames through standard GPS pipeline)** | iNav | UBX over UART | BSD-3-Clause (pyubx2) | Documentary-evaluable alternate (comparative-improvement assessment vs Cand 2 per user's "significant-improvement-only" bar) |
---
## C8 — On-FC adapter
### Fact #97 — ArduPilot Plane FC adapter primary: pymavlink → MAVLink GPS_INPUT (msg 232) cooperative-path; `GPS1_TYPE = 14` MAVLink + `EK3_SRC1_POSXY = 3` GPS source-set drives EKF3 ingestion via `AP_GPS_MAV` driver
- **Statement**: pymavlink (LGPL-3.0, canonical Python MAVLink stack maintained by ArduPilot per Source #106) is the single adapter library for the ArduPilot Plane side. Companion-side canonical send pattern (per pymavlink generated dialect + Source #107 ArduPilot dev docs):
```python
from pymavlink import mavutil
master = mavutil.mavlink_connection('udpout:127.0.0.1:14550', source_system=1, source_component=240)
master.mav.gps_input_send(
time_usec, gps_id, ignore_flags, time_week_ms, time_week, fix_type,
lat_deg_e7, lon_deg_e7, alt_m, hdop, vdop, vn_cmps, ve_cmps, vd_cmps,
speed_accuracy_mps, horiz_accuracy_m, vert_accuracy_m, satellites_visible, yaw_cdeg,
)
```
FC-side configuration (per Source #107): `GPS1_TYPE = 14` (MAVLink) is REQUIRED for AP_GPS to instantiate the AP_GPS_MAV driver; `EK3_SRC1_POSXY = 3` (GPS) selects the GPS_INPUT-fed virtual GPS as primary horizontal-position source for EKF3. AP's preferred non-GPS messages are `ODOMETRY` / `VISION_POSITION_ESTIMATE` at ≥4 Hz, but `GPS_INPUT` is the right transport for the project's "WGS84 coordinates as a real-GPS replacement" outcome contract (AC-4.3) AND for the project's `{satellite_anchored, visual_propagated, dead_reckoned}` source-label scheme (per SQ6 Fact #4: ODOMETRY-velocity-only is NOT supported in current AP, so `visual_propagated` cannot ride ODOMETRY — must be GPS_INPUT with widened `horiz_accuracy`).
- **Mode pinning**: `master.mav.gps_input_send(time_usec, gps_id, ignore_flags, time_week_ms, time_week, fix_type, lat, lon, alt, hdop, vdop, vn, ve, vd, speed_accuracy, horiz_accuracy, vert_accuracy, satellites_visible, yaw)` per pymavlink generated dialect (verified via SQ6 Source #4 AP_GPS_MAV.cpp ingestion path Fact #1).
- **Source**: Source #106 (pymavlink context7 + GitHub); Source #107 (ArduPilot Plane Non-GPS Position Estimation + GPS_INPUT MAVProxy module dev docs); cross-cite SQ6 Source #4 (AP_GPS_MAV.cpp master) + SQ6 Fact #1 + SQ6 Fact #2 + SQ6 Fact #3 + SQ6 Fact #4
- **Phase**: Phase 2
- **Confidence**: ✅
- **Sub-Question Binding**: SQ3 + SQ4 (per-component candidate selection for C8); SQ6 (per-FC inbound transport — already closed)
- **Related Dimension**: C8, C5 (covariance contract via `horiz_accuracy/vert_accuracy/speed_accuracy`), AC-NEW-2 (FC-side EKF source-set switch via `MAV_CMD_SET_EKF_SOURCE_SET`, SQ6 Fact #3)
- **Implication**: **supports selection** — Cand 1 (pymavlink → GPS_INPUT) satisfies AC-4.3 ArduPilot side; covariance honesty (AC-NEW-4) is wired through three fields (`horiz_accuracy`, `vert_accuracy`, `speed_accuracy`); spoof-promotion (AC-NEW-2) is companion-driven via `MAV_CMD_SET_EKF_SOURCE_SET`; visual-blackout failsafe (AC-NEW-8) maps directly to `fix_type` 0/1/2 + `horiz_accuracy = 999.0` sentinel per AP convention; source-label semantics (AC-1.4) emit out-of-band via `STATUSTEXT` / `NAMED_VALUE_FLOAT` per locked AC-4.3 wording.
#### Per-numbered-Restriction × per-numbered-AC sub-matrix (Cand 1: pymavlink → GPS_INPUT, ArduPilot Plane)
| Numbered AC / Restriction | Cand 1 (pymavlink → GPS_INPUT) verdict | Justification |
|---|---|---|
| AC-1.4 (95% covariance + source label) | **Pass** | `horiz_accuracy` = 95% covariance proxy; source label rides STATUSTEXT/NAMED_VALUE_FLOAT per AC-4.3 |
| AC-4.1 (≤400 ms p95 frame latency) | **Pass** | pymavlink Python encoding overhead is <1 ms per packet on Jetson Orin Nano Super CPU; UDP/UART transmit is sub-ms |
| AC-4.2 (<8 GB shared memory) | **Pass** | pymavlink runtime footprint is ~5-10 MB Python heap |
| AC-4.3 (FC output contract) | **Pass** | GPS_INPUT is exactly the locked AC-4.3 ArduPilot transport |
| AC-4.4 (frame-by-frame streaming) | **Pass** | pymavlink supports unbuffered immediate send |
| AC-4.5 (look-back refinement) | **N/A** | Adapter is downstream of estimator; no smoothing here |
| AC-6.1 (1-2 Hz GCS downsample) | **Pass** | Companion can throttle GPS_INPUT to FC at 1-3 Hz (AP samples at its own rate) |
| AC-NEW-1 (TTFF <30 s) | **Pass** | First valid GPS_INPUT frame is sent as soon as the estimator publishes an anchored fix |
| AC-NEW-2 (<3 s spoof promotion) | **Verify** | `MAV_CMD_SET_EKF_SOURCE_SET` round-trip latency under load — SITL validation per AC-NEW-2.Validation |
| AC-NEW-3 (FDR retains all emitted frames) | **Pass** | Companion-side raw MAVLink stream (tlog) capture is trivial via pymavlink |
| AC-NEW-4 (false-position safety budget) | **Pass IF covariance honest** | Project must publish honest `horiz_accuracy` (under-reporting defeats EKF3 quality chain per SQ6 Fact #2) |
| AC-NEW-7 (no covert GPS spoofing without consent) | **Pass** | GPS_INPUT is the documented external-positioning channel; not covert |
| AC-NEW-8 (visual-blackout failsafe) | **Pass** | Maps to `fix_type` 0/1/2 + `horiz_accuracy=999.0` sentinel per AP convention |
| Restriction "Supported FCs: ArduPilot Plane, iNav" | **Pass** for AP side | Cand 1 covers AP path only; iNav covered by Cand 2/3 |
| Restriction "Communication protocol per-FC: MAVLink for AP" | **Pass** | Exact match |
| LGPL-3.0 license posture (pymavlink) | **Pass** | LGPL §6 allows linking from Apache-2.0 app; project does not modify pymavlink, so no obligation beyond republishing modifications (none); fully dual-use compatible |
### Fact #98 — iNav FC adapter alternate: UBX impersonation via pyubx2 NAV-PVT (forging u-blox frames through standard GPS pipeline) — viability gated by iNav `gpsMapFixType()` validation: must set `flags & 0x01 (gnssFixOK) = 1` AND `fixType ∈ {2, 3}`
- **Statement**: pyubx2 (BSD-3-Clause, canonical Python UBX/NMEA/RTCM3 parser per Source #108) supports `UBXMessage(ubxClass='NAV', ubxID='NAV-PVT', mode=GET, **kwargs)` constructor with full per-attribute control, plus `serialize()` for wire-format output (sync-bytes 0xB5 0x62 + class + ID + length + payload + 8-bit Fletcher checksum). Companion-side canonical send pattern:
```python
from pyubx2 import UBXMessage, GET, parsebitfield
msg = UBXMessage(
'NAV', 'NAV-PVT', GET,
iTOW=ms_of_week,
year=2026, month=5, day=8, hour=12, min=30, sec=0,
valid=0b0111, # validDate | validTime | fullyResolved
tAcc=10_000_000, nano=0,
fixType=3, # FIX_3D — required for iNav gpsMapFixType to return GPS_FIX_3D
flags=0b00000001, # gnssFixOK set — required for fix_status & NAV_STATUS_FIX_VALID
flags2=0,
numSV=12,
lon=int(lon_deg * 1e7), lat=int(lat_deg * 1e7),
height=int(alt_m * 1000), hMSL=int(alt_m * 1000),
hAcc=int(horiz_acc_m * 1000), vAcc=int(vert_acc_m * 1000),
velN=int(vn_mps * 1000), velE=int(ve_mps * 1000), velD=int(vd_mps * 1000),
gSpeed=int(speed_2d_mps * 1000),
headMot=int(heading_deg * 1e5),
sAcc=int(speed_acc_mps * 1000), headAcc=int(heading_acc_deg * 1e5),
pDOP=int(pdop * 100),
headVeh=0, magDec=0, magAcc=0,
)
serial_out.write(msg.serialize())
```
iNav-side validation logic (per Source #110 `gps_ublox.c` direct read at line 654 + line 215-220):
```c
// Line 654 (NAV-PVT path):
next_fix_type = gpsMapFixType(_buffer.pvt.fix_status & NAV_STATUS_FIX_VALID, _buffer.pvt.fix_type);
// Line 215-220 (validation gate):
static gpsFixType_e gpsMapFixType(bool fixValid, uint8_t ubloxFixType) {
if (fixValid && ubloxFixType == FIX_2D) return GPS_FIX_2D;
if (fixValid && ubloxFixType == FIX_3D) return GPS_FIX_3D;
return GPS_NO_FIX;
}
```
Two validation requirements together: (a) `_buffer.pvt.fix_status & NAV_STATUS_FIX_VALID` evaluates `flags & 0x01` (= `gnssFixOK` bit) — must be 1; (b) `_buffer.pvt.fix_type` must be `FIX_2D = 2` or `FIX_3D = 3`. iNav 9.0+ at u-blox version ≥ 15.0 configures NAV-PVT-only protocol (per Source #110 lines 1024-1028) — companion must advertise version ≥ 15.0 via NAV-VER (CLASS=0x0A, ID=0x04) at startup to drive iNav into the simpler protocol surface.
- **Mode pinning**: `pyubx2.UBXMessage('NAV', 'NAV-PVT', GET, **kwargs).serialize()` produces wire-format bytes for direct UART write to iNav's GPS port; companion is the sole GPS source (SQ6 Fact #7 — iNav has no dual-GPS arbitration).
- **Source**: Source #108 (pyubx2 context7 + canonical README); Source #109 (u-blox NEO-M9N + M8 NAV-PVT canonical specifications); Source #110 (iNav `gps_ublox.c` master validation gates); cross-cite SQ6 Fact #10 (UBX-only over UART; NMEA dropped in 7.0; UBX ≥ 15.00 in 9.0+) + SQ6 Fact #7 (single-GPS architecture)
- **Phase**: Phase 2
- **Confidence**: ✅
- **Sub-Question Binding**: SQ3 + SQ4 (per-component candidate selection for C8); SQ6 (UBX emulation alternate, Fact #10)
- **Related Dimension**: C8, C5 (covariance contract via NAV-PVT `hAcc/vAcc/sAcc`), AC-NEW-2 (no FC-side switch needed — companion is sole GPS), AC-NEW-7 (UBX impersonation IS a forgery operation; safety implication)
- **Implication**: **viable alternate, comparative-improvement gate against Cand 2** — UBX path bypasses MSP2 queueing/arbitration concerns (companion appears as a normal u-blox receiver to iNav's stock GPS pipeline) AND requires no `USE_GPS_PROTO_MSP` build flag. Trade-offs: (a) implementation cost — companion must implement a fuller protocol surface (NAV-PVT periodic + NAV-VER on startup + correct ACK/NAK behaviour for CFG-MSG/CFG-RATE polls) vs MSP2_SENSOR_GPS which is a single periodic injection message; (b) iNav-firmware-side validation contract is stricter (`gpsMapFixType()` + 100-200 lines of stateful u-blox protocol handling vs `mspGPSReceiveNewData()` direct passthrough); (c) AC-NEW-7 nuance — UBX impersonation is a clearer forgery posture (companion is pretending to be a u-blox receiver) than MSP2_SENSOR_GPS (companion is using a documented sensor-injection path); (d) per user's "significant-improvement-only" bar (carried from C6 closure precedent), the Plan-phase comparative verdict needs to weigh: does UBX add material value over MSP2_SENSOR_GPS to justify the implementation cost + AC-NEW-7 nuance?
#### Per-numbered-Restriction × per-numbered-AC sub-matrix (Cand 3: UBX impersonation via pyubx2 NAV-PVT, iNav)
| Numbered AC / Restriction | Cand 3 (UBX impersonation) verdict | Justification |
|---|---|---|
| AC-1.4 (95% covariance + source label) | **Pass** | NAV-PVT `hAcc`/`vAcc`/`sAcc` carry covariance proxies; source label rides separate MSP2_DEBUG_MSG / TextMessage off-band channel (UBX has no equivalent of MAVLink STATUSTEXT — must use a sibling iNav telemetry channel) |
| AC-4.1 (≤400 ms p95 frame latency) | **Pass** | pyubx2 serialization overhead is <1 ms per packet; UART transmit at 115200+ baud is sub-ms |
| AC-4.2 (<8 GB shared memory) | **Pass** | pyubx2 runtime footprint is ~5-10 MB Python heap |
| AC-4.3 (FC output contract) | **Pass** (UBX is iNav's documented GPS protocol) | NAV-PVT through standard GPS pipeline IS a documented external-positioning interface; AC-4.3 wording mentions MSP2_SENSOR_GPS as primary but does not exclude UBX-emulation alternate |
| AC-4.4 (frame-by-frame streaming) | **Pass** | NAV-PVT streamed at companion's chosen rate (5-10 Hz typical) |
| AC-4.5 (look-back refinement) | **N/A** | Adapter is downstream of estimator |
| AC-6.1 (1-2 Hz GCS downsample) | **N/A** for UBX path (GCS sees iNav's MAVLink outbound, not UBX inbound) | iNav still emits MAVLink telemetry to GCS regardless of UBX vs MSP2 inbound choice |
| AC-NEW-1 (TTFF <30 s) | **Pass** | First valid NAV-PVT frame is sent as soon as estimator publishes anchored fix |
| AC-NEW-2 (<3 s spoof promotion) | **Pass by architecture** | Companion is sole iNav GPS; no FC-side switch needed (per SQ6 Fact #7) |
| AC-NEW-3 (FDR retains all emitted frames) | **Pass** | Companion-side raw UBX stream capture is trivial |
| AC-NEW-4 (false-position safety budget) | **Verify** | Need to confirm iNav nav-stack actually USES NAV-PVT `hAcc/vAcc` for outlier handling (the SQ6 Fact #6 "iNav explicitly does NOT validate GPS for spoofing" caveat applies symmetrically to UBX path — companion-side honesty is mandatory because iNav-side rejection chain is minimal) |
| AC-NEW-7 (no covert spoofing without consent) | **Verify** | UBX impersonation IS a forgery posture; project must explicitly document this in the FDR audit trail (mitigates by being a documented project design, but the impersonation framing is unambiguous) |
| AC-NEW-8 (visual-blackout failsafe) | **Pass** | NAV-PVT `fixType` enum carries graceful degrade: 0=NoFix / 1=DeadReck / 2=2D / 3=3D / 4=GNSS+DR / 5=TimeOnly; companion can emit `fixType=0` for blackout-no-fix or `fixType=2` (2D) for degraded-covariance mode |
| Restriction "Supported FCs: ArduPilot Plane, iNav" | **Pass** for iNav side | Cand 3 covers iNav path only |
| Restriction "Communication protocol per-FC: MSP2 for iNav" | **Verify (alternate)** | Locked restrictions.md says MSP2; UBX is documented in SQ6 Fact #10 as fallback, not primary. Plan-phase decision (D-C8-N) chooses between MSP2 primary or UBX primary based on comparative verdict |
| BSD-3-Clause license posture (pyubx2) | **Pass** | Clean dual-use compatible |
### Fact #99 — iNav FC adapter primary: MSP2_SENSOR_GPS (id 7939 / 0x1F03) via Python MSP V2 implementation (YAMSPy or INAV-Toolkit `msp_v2_encode`) — `mspGPSReceiveNewData()` direct passthrough; covariance fields `hPosAccuracy`/`vPosAccuracy`/`hVelAccuracy` align directly with AP `GPS_INPUT.horiz_accuracy`/`vert_accuracy`/`speed_accuracy`
- **Statement**: MSP2_SENSOR_GPS (id 7939 / 0x1F03 — verified in iNav `msp_protocol_v2_sensor.h` master per Source #113) is iNav's documented sensor-plugin GPS injection path. Per Source #111 master `docs/development/msp/README.md` lines 2999-3031: payload is 36 bytes `instance/u8 + gpsWeek/u16 + msTOW/u32 + fixType/u8 + satellitesInView/u8 + hPosAccuracy/u16(mm) + vPosAccuracy/u16(mm) + hVelAccuracy/u16(cm/s) + hdop/u16 + longitude/i32(deg×1e7) + latitude/i32(deg×1e7) + mslAltitude/i32(cm) + nedVelNorth/i32(cm/s) + nedVelEast/i32(cm/s) + nedVelDown/i32(cm/s) + groundCourse/u16(deg×100) + trueYaw/u16(deg×100, 65535=unavailable) + year/u16 + month/u8 + day/u8 + hour/u8 + min/u8 + sec/u8`. iNav-side: `mspGPSReceiveNewData()` is called with no return value — direct passthrough to `gpsSol` (per Source #111 Notes block), NO additional validation gate beyond the data parse itself (contrast with UBX path's `gpsMapFixType()` validation). Required iNav build flag: `USE_GPS_PROTO_MSP`**enabled by default in `target/common.h`** per SQ6 Source #13 (so stock firmware reaches this path).
Companion-side canonical send pattern using INAV-Toolkit primitives (per Source #112):
```python
import struct
from inav_msp import msp_v2_encode # CRC-8 DVB-S2 envelope encoder
MSP2_SENSOR_GPS = 0x1F03
payload = struct.pack(
'<BHIBBHHHHIIIiiiHHHBBBBB',
instance, gps_week, ms_tow, fix_type, sats_in_view,
h_pos_acc_mm, v_pos_acc_mm, h_vel_acc_cmps, hdop_x100,
lon_deg_e7, lat_deg_e7, msl_alt_cm,
ned_n_cmps, ned_e_cmps, ned_d_cmps,
ground_course_deg_x100, true_yaw_deg_x100,
year, month, day, hour, min_, sec,
)
frame = msp_v2_encode(MSP2_SENSOR_GPS, payload) # MSP V2 envelope: 0x24 'X' 0x3C flag cmd_le len_le payload crc8_dvb_s2
serial_out.write(frame)
```
Two community-maintained Python options: **YAMSPy** (MIT, community-blessed in iNav Issue #4465 for external-device communication) OR **INAV-Toolkit `inav_msp.py`** (MIT, 951-line direct primary-source reference for `msp_v2_encode` / `msp_v2_decode` with CRC-8 DVB-S2). Either route avoids the UBX-protocol-handler complexity required for Cand 3.
- **Mode pinning**: `msp_v2_encode(MSP2_SENSOR_GPS=0x1F03, payload)` + `serial_out.write(frame)` per iNav MSP V2 envelope spec; iNav-side `mspGPSReceiveNewData()` direct passthrough per Source #111 Notes.
- **Source**: Source #111 (iNav MSP message reference master, MSP2_SENSOR_GPS canonical payload spec); Source #112 (YAMSPy + INAV-Toolkit Python implementations); Source #113 (iNav `msp_protocol_v2_sensor.h` MSP V2 sensor-message-range definition); cross-cite SQ6 Source #12 (master MSP message reference) + SQ6 Source #13 (`USE_GPS_PROTO_MSP` enabled by default) + SQ6 Fact #6 (MSP2_SENSOR_GPS covariance-rich path)
- **Phase**: Phase 2
- **Confidence**: ✅ for protocol spec; ⚠️ for community-library version-stability (YAMSPy / INAV-Toolkit may need extension or thin-custom-encoder replacement at design phase if MSP V2 sensor-message-range support lags upstream iNav)
- **Sub-Question Binding**: SQ3 + SQ4 (per-component candidate selection for C8); SQ6 (per-FC inbound transport, Fact #6 — already SQ6 Selected lead)
- **Related Dimension**: C8, C5 (covariance contract via `hPosAccuracy`/`vPosAccuracy`/`hVelAccuracy`), AC-NEW-2 (no FC-side switch needed — companion is sole GPS), AC-NEW-4 (covariance honesty is companion-side responsibility per SQ6 Fact #8)
- **Implication**: **supports selection** — Cand 2 (MSP2_SENSOR_GPS via Python MSP V2) is the SQ6 closure lead AND is the locked AC-4.3 transport for iNav; covariance honesty (AC-NEW-4) is wired through three fields aligning DIRECTLY with AP `GPS_INPUT.horiz_accuracy/vert_accuracy/speed_accuracy` — same companion-side covariance contract for both FCs. AC-NEW-7 framing: MSP2_SENSOR_GPS is a documented sensor-injection path, NOT an impersonation — clearer audit-trail posture than Cand 3 UBX. AC-NEW-8 graceful-degrade: `fixType` enum carries 6 levels (per `gpsFixType_e`) — companion can emit `GPS_NO_FIX` (0) for blackout-no-fix, `GPS_FIX_2D` (1) for degraded-covariance >100 m mode. Single-message contract = simpler than UBX's NAV-PVT + NAV-VER + CFG-* protocol surface.
#### Per-numbered-Restriction × per-numbered-AC sub-matrix (Cand 2: MSP2_SENSOR_GPS via Python MSP V2, iNav)
| Numbered AC / Restriction | Cand 2 (MSP2_SENSOR_GPS) verdict | Justification |
|---|---|---|
| AC-1.4 (95% covariance + source label) | **Pass** | `hPosAccuracy` = 95% covariance proxy; source label rides separate MSP2 telemetry channel (e.g. MSP2_SENSOR_RANGEFINDER spare bytes or a custom MSP2_INAV_DEBUG variant) |
| AC-4.1 (≤400 ms p95 frame latency) | **Pass** | Python `struct.pack` + `msp_v2_encode` overhead is <1 ms per frame on Jetson |
| AC-4.2 (<8 GB shared memory) | **Pass** | YAMSPy or INAV-Toolkit runtime footprint is ~5-10 MB Python heap |
| AC-4.3 (FC output contract) | **Pass** | MSP2_SENSOR_GPS is exactly the locked AC-4.3 iNav transport |
| AC-4.4 (frame-by-frame streaming) | **Pass** | MSP2 supports periodic injection at companion's chosen rate (5-10 Hz typical) |
| AC-4.5 (look-back refinement) | **N/A** | Adapter is downstream of estimator |
| AC-6.1 (1-2 Hz GCS downsample) | **N/A** for MSP2 path (GCS sees iNav's MAVLink outbound, not MSP2 inbound) | iNav still emits MAVLink telemetry to GCS regardless of MSP2 vs UBX inbound choice |
| AC-NEW-1 (TTFF <30 s) | **Pass** | First valid MSP2_SENSOR_GPS frame is sent as soon as estimator publishes anchored fix |
| AC-NEW-2 (<3 s spoof promotion) | **Pass by architecture** | Companion is sole iNav GPS; no FC-side switch needed (per SQ6 Fact #7) |
| AC-NEW-3 (FDR retains all emitted frames) | **Pass** | Companion-side raw MSP V2 stream capture is trivial |
| AC-NEW-4 (false-position safety budget) | **Verify** | Need to confirm iNav nav-stack actually USES `hPosAccuracy/vPosAccuracy/hVelAccuracy` for outlier handling per SQ6 Fact #6 + SQ6 Fact #8 — design-phase task to read `src/main/io/gps_msp.c` `mspGPSReceiveNewData()` body |
| AC-NEW-7 (no covert spoofing without consent) | **Pass** | MSP2_SENSOR_GPS is the documented sensor-injection path; not covert/forgery |
| AC-NEW-8 (visual-blackout failsafe) | **Pass** | `fixType` enum (`gpsFixType_e`) carries graceful degrade levels; companion can emit `GPS_NO_FIX` (0) or `GPS_FIX_2D` (1) for the covariance>100 m / blackout thresholds |
| Restriction "Supported FCs: ArduPilot Plane, iNav" | **Pass** for iNav side | Cand 2 covers iNav path only |
| Restriction "Communication protocol per-FC: MSP2 for iNav" | **Pass** | Exact match — locked SQ6 + AC-4.3 + restrictions.md |
| MIT license posture (YAMSPy + INAV-Toolkit) | **Pass** | Clean dual-use compatible |
---
## C8 — Cand 2 vs Cand 3 comparative-improvement-vs-Cand-2 verdict (closure of batch 1, iNav side)
Per user's session-start "significant-improvement-only" bar (same calibration as C6 closure verdict that locked Cand 1 PostgreSQL+btree+FAISS as primary over Cand 2 PostGIS+pgvector secondary):
| Lever | Cand 2 (MSP2_SENSOR_GPS) | Cand 3 (UBX impersonation) | Material improvement of Cand 3 over Cand 2? |
|---|---|---|---|
| Wire format complexity | Single MSP2 envelope + 36-byte payload + CRC-8 DVB-S2 | NAV-PVT (92 bytes) + NAV-VER startup + CFG-MSG/CFG-RATE ACK behaviour | **Cand 3 ADDS complexity (negative)** |
| Protocol-surface footprint | One message ID (0x1F03) | NAV-PVT + NAV-VER (CLASS=0x0A,ID=0x04) + ACK/NAK protocol | **Cand 3 ADDS surface (negative)** |
| iNav-side validation gate | `mspGPSReceiveNewData()` direct passthrough | `gpsMapFixType()` requires `flags & 0x01 = 1` AND `fixType ∈ {2,3}` | **Cand 3 ADDS validation gate (mixed: stricter = more brittle to companion bugs, but also catches malformed frames earlier)** |
| Covariance-honesty contract | `hPosAccuracy/vPosAccuracy/hVelAccuracy` aligned with AP `GPS_INPUT.horiz_accuracy/vert_accuracy/speed_accuracy` | NAV-PVT `hAcc/vAcc/sAcc` aligned with same | **Tie** |
| AC-NEW-7 audit-trail posture | Documented sensor-injection path (clean) | Forgery posture (companion impersonates u-blox receiver) | **Cand 3 WORSE for AC-NEW-7** |
| Dependency on iNav build flags | `USE_GPS_PROTO_MSP` (enabled by default) | None (UBX path always available) | **Cand 3 marginally better — no dependency on a build flag** |
| Library maturity | YAMSPy / INAV-Toolkit (community, MIT, ~951-line reference impl); ⚠️ may need extension for MSP2 sensor-message-range | pyubx2 (canonical, BSD-3-Clause, daily-active, 139+239 context7 code snippets) | **Cand 3 has more mature library** |
| AC-NEW-2 architectural fit | Pass-by-architecture (companion is sole GPS) | Pass-by-architecture (same) | **Tie** |
| AC-NEW-8 graceful-degrade | `gpsFixType_e` 6-level enum | NAV-PVT `fixType` 6-level enum | **Tie** |
| Cross-FC consistency with AP path | MSP2 ≠ MAVLink — different protocol on the wire, but same logical companion-side covariance contract | UBX ≠ MAVLink — same | **Tie** |
**Verdict (closure)**: Cand 3 (UBX impersonation) does NOT clear the user's "significant-improvement-only" bar over Cand 2 (MSP2_SENSOR_GPS). UBX's sole real upside is library maturity (pyubx2 vs YAMSPy/INAV-Toolkit) — but YAMSPy + INAV-Toolkit are MIT-clean and the canonical msp_v2_encode primitive is well-documented (951 lines of primary-source reference in INAV-Toolkit). Cand 3's downsides (added protocol-surface complexity + AC-NEW-7 forgery posture + stricter validation gate) outweigh the upside.
**Recommendation**: Cand 2 (MSP2_SENSOR_GPS) is **RECOMMENDED PRIMARY** for the iNav side; Cand 3 (UBX impersonation) is **DEFERRED secondary** with explicit re-evaluation criteria — promote to primary IF (a) YAMSPy + INAV-Toolkit prove insufficient at Plan-phase MSP V2 sensor-message-range support and project chooses NOT to extend them, OR (b) Plan-phase iNav MVE reveals that `mspGPSReceiveNewData()` does NOT use the covariance fields per AC-NEW-4 verify-cell and the project needs the stricter `gpsMapFixType()` validation contract for runtime sanity-checking, OR (c) the project re-opens AC-NEW-7 and decides UBX impersonation is preferred for some yet-to-be-identified safety reason.
---
## C8 — Working conclusions and decisions (compounded from Fact #97 + Fact #98 + Fact #99 closures)
### Per-FC adapter design (re-confirmed from SQ6 closure, now operationalized)
| FC | Adapter library | Transport | Lead candidate fact | License posture |
|---|---|---|---|---|
| **ArduPilot Plane** | pymavlink | MAVLink GPS_INPUT (msg 232) over UART/USB/UDP | Fact #97 | LGPL-3.0 (linkable from Apache-2.0 app per LGPL §6) |
| **iNav (RECOMMENDED PRIMARY)** | YAMSPy or INAV-Toolkit msp_v2_encode | MSP2_SENSOR_GPS (id 7939 / 0x1F03) over UART/USB | Fact #99 | MIT |
| **iNav (DEFERRED secondary)** | pyubx2 | UBX NAV-PVT impersonation over UART | Fact #98 | BSD-3-Clause |
### Plan-phase Decision Gates raised by C8 batch 1
- **D-C8-1 (NEW from Fact #97 closure 2026-05-08, Cand-1-only)** — pymavlink connection-string transport choice
- Options: (a) `udpout:127.0.0.1:14550` for in-process companion + autopilot UDP; (b) `serial:/dev/ttyTHS1:921600` for direct UART to AP TELEM port (no companion-router middlebox); (c) `tcp:127.0.0.1:5760` for SITL replay; (d) **all three configurable via env var, default UART (b) for production deployment, UDP (a) for SITL replay, TCP (c) for unit tests RECOMMENDED**.
- Owner: Plan-phase architect.
- Rationale: pymavlink supports all three transports identically; choice depends on deployment topology. Default to UART for production reduces moving parts.
- **D-C8-2 (NEW from Fact #97 closure 2026-05-08, Cand-1-only CROSS-COMPONENT with AC-NEW-2)**`MAV_CMD_SET_EKF_SOURCE_SET` companion-driven switch ownership
- Options: (a) companion always claims source-set 1 and FC keeps real-GPS at source-set 2 (companion reactive only); (b) **companion publishes to source-set 2 and switches FC to set 2 when companion publishes its first valid fix; switches back to set 1 when companion is unavailable RECOMMENDED ~mirrors NGPS/Auterion pattern**; (c) operator manually flips source-set via RC aux switch (option 90).
- Owner: Plan-phase architect + AC-NEW-2 owner.
- Rationale: per SQ6 Fact #3, "no GCSs are currently known to implement" companion-driven `MAV_CMD_SET_EKF_SOURCE_SET` — but it works at firmware level. The project gets to define the canonical pattern.
- **D-C8-3 (NEW from Fact #97 closure 2026-05-08, Cand-1-only)** — pymavlink LGPL-3.0 license-posture verification
- Options: (a) **bundle pymavlink unmodified + publish requirements.txt with version pin RECOMMENDED ~standard LGPL §6 compliance**; (b) statically link via Cython compilation (LGPL §6 obligation: provide relinkable form); (c) wrap pymavlink behind a thin C++/Rust process boundary to keep companion-app fully Apache-2.0 (over-engineered; not justified by project posture).
- Owner: Plan-phase architect + license owner.
- Rationale: aligns with D-C1-1 license-posture-track decision; pymavlink LGPL-3.0 vs project Apache-2.0 dual-use track is straightforward.
- **D-C8-4 (NEW from Fact #99 closure 2026-05-08, Cand-2-only)** — Python MSP V2 implementation choice
- Options: (a) **YAMSPy (community-blessed for iNav external-device comms per Issue #4465); MIT; latest commit pre-2025-Q4 RECOMMENDED ~widest community usage**; (b) INAV-Toolkit `msp_v2_encode` primitive lifted into the project (951-line MIT module, direct primary-source reference); (c) thin custom encoder using `struct.pack` + CRC-8 DVB-S2 helper (50-line bespoke); (d) project-side fork of one of the above.
- Owner: Plan-phase architect.
- Rationale: all options are MIT and produce identical wire bytes; choice depends on maintainability vs minimum-dependency-surface preference.
- **D-C8-5 (NEW from Fact #99 closure 2026-05-08, Cand-2-only)** — MSP2_SENSOR_GPS injection rate
- Options: (a) **5 Hz periodic RECOMMENDED ~matches GPS_INPUT 5 Hz cadence on AP side, single-rate cross-FC consistency**; (b) 10 Hz to match iNav nav-cycle frequency; (c) variable rate matching estimator publication rate (3 Hz nominal, up to 10 Hz when matcher confidence is high).
- Owner: Plan-phase architect.
- Rationale: estimator publishes at 3 Hz nominal (per pinned dual-rate camera pipeline Fact #40); 5 Hz adapter-side rate has spare headroom for IMU-propagation between estimator updates.
- **D-C8-6 (NEW from Fact #98 closure 2026-05-08, Cand-3-only contingent)** — IF Cand 3 selected → UBX-version-advertisement strategy
- Options: (a) **advertise hwVersion ≥ M9 + swVersion ≥ 15.00 via NAV-VER (CLASS=0x0A, ID=0x04) at startup + every reset; force iNav into NAV-PVT-only protocol surface RECOMMENDED ~simplest configuration path**; (b) advertise hwVersion = M8 + swVersion = 14.x to drive iNav into legacy NAV-POSLLH+NAV-SOL+NAV-VELNED+NAV-TIMEUTC quad mode (more messages but historical iNav-friendly path); (c) implement adaptive advertisement based on iNav firmware-version probe.
- Owner: Plan-phase architect.
- Rationale: per Source #110 lines 1024-1060, iNav configures the simpler NAV-PVT-only path for u-blox version ≥ 15.0 — companion impersonator should advertise this version to minimize protocol surface.
- **D-C8-7 (NEW from Fact #98 closure 2026-05-08, Cand-3-only contingent)** — IF Cand 3 selected → AC-NEW-7 audit-trail posture
- Options: (a) **explicit FDR audit entry on every UBX impersonation session start, naming companion as the UBX source + providing operator-consent provenance check at boot RECOMMENDED**; (b) silent operation with user-manual disclosure only; (c) require runtime parameter `gps-denied-onboard.enable_ubx_impersonation = true` to be set explicitly by the user via QGC (active opt-in).
- Owner: Plan-phase architect + AC-NEW-7 owner.
- Rationale: UBX impersonation is unambiguously a forgery posture (companion pretends to be u-blox receiver); AC-NEW-7 (no covert GPS spoofing without consent) requires an audit trail.
- **D-C8-8 (NEW from Fact #97 + Fact #99 closures 2026-05-08, CROSS-COMPONENT — affects both Cand 1 and Cand 2; CROSS-COMPONENT with C5 covariance contract)** — covariance-honesty cross-FC enforcement
- Options: (a) project always publishes the SAME covariance value to both FCs (single shared contract, simpler test surface); (b) **per-FC covariance unit conversion: AP `GPS_INPUT.horiz_accuracy` (m) vs iNav `MSP2_SENSOR_GPS.hPosAccuracy` (mm) — companion publishes the same source covariance, formatted per-FC RECOMMENDED**; (c) per-FC covariance smoothing (different filter parameters per FC) — over-engineered and adds covariance-monotonicity-violation risk under C5 D-C5-2 long-cruise observability.
- Owner: Plan-phase architect + AC-NEW-4 owner.
- Rationale: AC-NEW-4 covariance-honesty obligation is the same for both FCs; only the unit and field-name change.
### Cross-row dependencies
- **C5 covariance contract integration**: Both Cand 1 (AP) and Cand 2 (iNav) require honest covariance from C5 estimator. The C5 GTSAM `Marginals.marginalCovariance` path (Fact #89) produces a 6×6 pose covariance matrix; the C8 adapter extracts the 2×2 horizontal sub-matrix (rows 3-4 = x, y in GTSAM's `Pose3` ordering) and converts to scalar `horiz_accuracy` (m) for AP or `hPosAccuracy` (mm) for iNav using the 95% confidence ellipse semi-major axis approximation `sqrt(2.0 * 5.991 * λ_max)` where λ_max is the largest eigenvalue of the 2×2 horizontal covariance.
- **AC-NEW-2 spoof-promotion latency cross-FC validation**: SITL test on each FC under spoof-injection — AP path validates `MAV_CMD_SET_EKF_SOURCE_SET` round-trip; iNav path validates companion-internal reaction time (companion is sole GPS, FC does not participate in source switching). Both should hit 95th percentile <3 s.
- **AC-NEW-8 visual-blackout cross-FC behaviour**: AP `fix_type` enum (0/1/2/3/4/5/6) and iNav `gpsFixType_e` enum (`GPS_NO_FIX/GPS_FIX_2D/GPS_FIX_3D/...`) carry the same 0/1/2/3 ordering, simplifying cross-FC graceful-degrade implementation.
---
### Boundary check: C8 batch 1 saturation status
C8 batch 1 (3 of N candidate adapters with explicit per-FC pinning) closed at the documentary level on 2026-05-08:
- **Cand 1 (Fact #97, ArduPilot pymavlink → GPS_INPUT)** — RECOMMENDED PRIMARY. Documentary verification ✅ via context7 + ArduPilot dev docs + cross-cite SQ6 Source #4 ingestion path; mode-pinned send pattern verified; per-AC sub-matrix complete.
- **Cand 2 (Fact #99, iNav MSP2_SENSOR_GPS via Python MSP V2)** — RECOMMENDED PRIMARY for iNav side. Documentary verification ✅ via iNav master MSP message reference + msp_protocol_v2_sensor.h source + community library landscape (YAMSPy + INAV-Toolkit); mode-pinned send pattern verified; per-AC sub-matrix complete.
- **Cand 3 (Fact #98, iNav UBX impersonation via pyubx2 NAV-PVT)** — DEFERRED secondary for iNav side after comparative-improvement verdict. Documentary verification ✅ via pyubx2 context7 + u-blox NAV-PVT canonical specs + iNav `gps_ublox.c` direct source read of validation gate; mode-pinned send pattern verified; per-AC sub-matrix complete.
Saturation rationale: SQ6 closure already covered the per-FC inbound architecture; C8 batch 1 was the operationalization step for the three viable candidates (one per FC for AP; two per FC for iNav since user requested parallel evaluation). No additional candidates surfaced during research that haven't been considered-and-rejected per c8_overkill_options=A (MAVProxy/mavp2p/ardupilot-router are router-class not adapter-class; full MAVSDK C++/Rust SDKs are out-of-budget vs Python pymavlink; no sibling third iNav transport beyond MSP2 + UBX exists in iNav 9.0 master).
C8 batch 1 closure is gated only on the eight Plan-phase decisions D-C8-1..8 and the cross-row C5 covariance contract integration. Plan-phase Choose blocks are recorded in [`../06_component_fit_matrix/99_cross_component_gates.md`](../06_component_fit_matrix/99_cross_component_gates.md).
@@ -0,0 +1,155 @@
# Fact Cards — SQ1: Existing / competitor GPS-denied UAV navigation systems
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Extracted from sources logged in `../01_source_registry/SQ1_existing_systems.md` (see `../01_source_registry/00_summary.md` for index). Confidence labels: ✅ High (L1 / verified source code), ⚠️ Medium (L1/L2 with caveat), ❓ Low (L3/L4 inferential). Bound to sub-questions in `../00_question_decomposition.md`.
>
> Index: [`../00_summary.md`](../00_summary.md). Sibling categories: SQ6 ([FC external positioning](SQ6_fc_external_positioning.md)), SQ2 ([canonical pipeline](SQ2_canonical_pipeline.md)), C1 ([VIO](C1_vio.md)), C2 ([VPR](C2_vpr.md)), C3 ([matchers](C3_matchers.md)).
**Facts in this file**: #11#20 (peer/adjacent systems: OSCAR, Auterion Artemis, Vantor Raptor, NGPS, SPRIN-D winner, RTAB-Map/ORB-SLAM3 pruning, DSMAC/TERCOM lineage, hierarchical matching SOTA, AerialExtreMatch benchmark, DARPA FLA + USAF SBIR) + SQ1 working conclusions.
---
## SQ1 — Existing / competitor GPS-denied UAV navigation systems
### Fact #11 — Twist Robotics OSCAR is a deployed Ukrainian peer system in the same architectural class as this project
- **Statement**: Twist Robotics (Ukraine) has a fielded camera + map-matching navigation module called OSCAR (Optical System of Coordinates with Automatic Relocalisation). The vendor states the system "captures the terrain, identifies landmarks, compares them with a map, determines coordinates, and transmits them to the autopilot as a reliable GPS signal" — the same five-stage architecture this project is building. Vendor-stated specs: ≤20 m accuracy without cumulative error, day/night/fog operation, and operational deployment of "more than 500,000 km across 25,000 combat missions over 24 months". Hardware includes active cooling, indicating a non-trivial onboard compute (likely Jetson-class). **No public independent benchmark of the 20 m number.**
- **Source**: Source #25, Source #26
- **Phase**: Phase 2
- **Target Audience**: System architects + AC owners (existence-of-peer evidence, not implementation guide)
- **Confidence**: ✅ for "deployed at scale on Ukrainian combat platforms"; ⚠️ for "20 m accuracy" (vendor self-report); ❓ for "fully resistant to spoofing and jamming" (claim not independently verified)
- **Related Dimension**: SQ1, SQ8 (anti-spoofing claim audit), SQ9 (synthesis — ours must beat or at least match this in the operational regime)
- **Fit Impact**: **establishes feasibility floor** — a Ukrainian peer is operating a similar architecture against the same threat environment our system targets. Project framing must explicitly differentiate (e.g., 1 km AGL vs unspecified OSCAR altitude; 8 h endurance vs unspecified OSCAR endurance; AC-NEW-4 honest covariance contract vs OSCAR's unspecified covariance reporting).
### Fact #12 — Auterion Artemis is a production-shipping fixed-wing one-way attack drone with Ukraine-validated GPS-denied navigation, defining the production benchmark for this class
- **Statement**: Auterion completed the US Defense Innovation Unit Artemis program in October 2025, delivering a Shahed-class deep-strike drone with up to 1,000-mile range and up to 40 kg warhead, running on **Auterion Skynode N mission computer + Auterion Visual Navigation system + built-in terminal guidance**. Government evaluators signed off after operational flight tests in Ukraine including ground launch, GPS and GPS-denied navigation, long-range transit, and terminal engagement. Manufacturing is being established in US, UA, and DE; Auterion is offering the system to the US Department of War and allied nations.
- **Source**: Source #31; Source #32 confirms Skynode S sibling architecture (NPU-equipped companion).
- **Phase**: Phase 2
- **Target Audience**: System architects (production-pattern reference)
- **Confidence**: ✅
- **Related Dimension**: SQ1 (closest commercial production peer), SQ9 (architecture template)
- **Fit Impact**: **establishes production reference architecture** — companion-class autopilot + visual navigation + terminal guidance is shipping at production scale to a US defense customer. Implication: building a per-FC adapter (project decision in SQ6) is consistent with what production stacks already do; integrating against the Artemis architecture is realistic; competing on price + Ukraine-specific operational tuning + AC-NEW-4 honest-covariance contract is a viable differentiation.
### Fact #13 — Vantor Raptor is a production COTS visual-GPS-replacement software suite, demonstrating that "branded sat-tile basemap + on-drone vision software" is a viable commercial pattern
- **Statement**: Vantor Raptor product family (Guide / Sync / Ace) provides vision-based GPS replacement using the drone's existing camera plus Vantor's "100 million-plus sq km of highly accurate 3D terrain data" (Vivid Terrain, vendor-stated 3 m accuracy). Vendor-demonstrated absolute accuracy: **<7 m in all dimensions** for aerial position (Guide), **<3 m** for ground coordinate extraction (Sync, Ace). Works at night and at low altitudes. Platform-agnostic, deployable on commodity hardware, integrates with existing onboard cameras. Inertial Labs has published a VINS-integrated Raptor Guide white paper. Recent partnerships: Niantic Spatial (Dec 2025) for unified air-to-ground positioning in GPS-denied areas; Maxar partnership with AIDC (Sep 2025) for Taiwan UAV resilience against GPS interference.
- **Source**: Source #30
- **Phase**: Phase 2
- **Target Audience**: Architecture / business decision-makers (build-vs-buy framing)
- **Confidence**: ✅ for product existence + claimed accuracy bounds (vendor primary); ⚠️ for whether Vantor's commercial accuracy figures hold under the project's specific Ukrainian-steppe + active-conflict-tile-staleness conditions
- **Related Dimension**: SQ1 (commercial), C2/C3 (commercial alternatives to building ourselves), SQ8 (basemap as a service vs offline cache)
- **Fit Impact**: **build-vs-buy lens** — Raptor Guide's <7 m claim is *better* than the project's AC-1.1 budget (≤80 m / 95% under AC-1.1.1), so it's not a disqualifier on accuracy. Reasons we still build vs buy: (a) Vantor is a US vendor; export / dual-use licensing into the Ukrainian battlefield is uncertain; (b) restrictions specify offline cache from the project's own Azaion Suite Satellite Service (AC-2.x), not Vantor's Vivid Terrain — replacing the basemap is non-negotiable; (c) covariance honesty contract (AC-NEW-4) and source-label contract (AC-1.4) are project-specific and may not be exposed by Vantor's API. **Outcome**: keep Raptor as a competitive comparator in `solution_draft01`, NOT as a candidate component to integrate.
### Fact #14 — snktshrma/ngps_flight (NGPS — ArduPilot GSoC 2024) is the closest open-source pipeline match to this project's exact C1+C2+C3+C5+C8 stack
- **Statement**: NGPS = ROS 2 + ArduPilot pipeline composed of three packages: **`ap_ngps_ros2`** (visual geo-localization at 12 Hz by matching live camera frames to georeferenced satellite imagery using **LightGlue + SuperPoint**, deep-learning-based feature matching), **`ap_ukf`** (Unscented Kalman Filter fusing NGPS absolute positions with VIO estimates), **`ap_vips`** (VIO providing relative pose). Output is fused odometry to ArduPilot's EKF (per related ArduPilot issue #23471, this is via `VISION_POSITION_ESTIMATE` requiring EKF source-set 2/3 with `EK3_SRC*_POSXY=Vision`). Project is published under ArduPilot's GSoC 2024 program. Sibling `ap_nongps` is an earlier OpenCV-based prototype.
- **Source**: Source #33
- **Phase**: Phase 2
- **Target Audience**: Implementer / Engineer
- **Confidence**: ✅ for project existence, component breakdown, and matcher choice (LightGlue+SuperPoint); ⚠️ for runtime behaviour under our exact constraints (Jetson Orin Nano, 1 km AGL, 17 m/s, 3 fps); ❓ for production hardening / covariance honesty / spoof-defence (none documented)
- **Related Dimension**: SQ1 (closest open-source peer), SQ2 (canonical pipeline confirmation), SQ3+SQ4 (architectural template for component candidate matrix), SQ6 (alternate AP transport debate)
- **Fit Impact**: **architectural template** — confirms the project's split (C1 VIO ↔ C2/C3 visual absolute ↔ C5 fusion ↔ C8 FC adapter) is canonical, not novel. Two concrete deltas:
1. **Transport choice on AP**: NGPS uses `VISION_POSITION_ESTIMATE`. SQ6 picked `GPS_INPUT` because it carries `horiz_accuracy` directly, supports source-set switching via `MAV_CMD_SET_EKF_SOURCE_SET`, and avoids EKF-source-set reconfiguration. The trade-off (NGPS's path vs SQ6's pick) must be re-examined at design time before final AP-transport selection.
2. **Estimator choice**: NGPS uses UKF; SQ3/SQ4 will compare UKF vs ESKF vs MSCKF vs factor-graph (GTSAM) on the same matrix.
### Fact #15 — RGB satellite-image matching as a *low-altitude* (<25 m AGL) localization technique is unreliable per the SPRIN-D Challenge; our 1 km AGL operates in the regime where the same authors note it "works reasonably well"
- **Statement**: The CTU Prague team's SPRIN-D winning paper directly states: *"Some teams used RGB satellite image-based matching, but this has proved to be highly unreliable at such low altitudes."* (referring to <25 m AGL). The paper's related-work review separately notes that *"high-altitude matching... works reasonably well, but at low altitudes (25 m) the viewpoint differs drastically, making roofs, facades, and vegetation inconsistent with satellite imagery."* The project operates at ≤1 km AGL — which is the *high-altitude* regime in the paper's terminology — making RGB sat-matching the appropriate technique class. The paper's CPU-only winning method (LiDAR heightmap-gradients + clustered particle filter) is **not** transferable to our hardware: our project has no LiDAR.
- **Source**: Source #28
- **Phase**: Phase 2
- **Target Audience**: Implementer / Engineer + Domain expert
- **Confidence**: ✅
- **Related Dimension**: SQ1, SQ5 (failure modes), SQ2 (canonical pipeline)
- **Fit Impact**: **disambiguates a potentially-disqualifying lesson** — the CTU paper's "RGB sat-matching is unreliable" finding does NOT disqualify our approach because the failure was caused by low-altitude viewpoint mismatch, which our 1 km AGL regime does not have. This must be cited explicitly in `solution_draft01` to pre-empt the natural objection from anyone who reads the paper. Separately, the CTU paper's specific lessons are still binding: VIO degrades catastrophically without IMU vibration isolation; magnetometer is unreliable near steel/concrete; "ability to recover from periods of high uncertainty and re-localize" matters more than instantaneous RMSE — this last lesson is a direct architectural input for AC-NEW-2 / AC-NEW-8.
### Fact #16 — RTAB-Map and ORB-SLAM3 both fail beyond 1 km / above 2 m/s flight in the SPRIN-D environment; our cruise profile (≤17 m/s, kilometers between satellite anchors) explicitly excludes both as primary candidates
- **Statement**: The SPRIN-D paper states: *"We tested state-of-the-art visual SLAM systems such as RTAB-Map and ORB-SLAM3 in a high-fidelity simulator, and found that both performance degraded significantly in a long-range scenario (beyond 1 km), as their memory and compute demands grow with the size of the environment. Moreover, RTAB-Map was unable to maintain quality odometry in faster flight speeds (beyond 2 m/s), while ORB-SLAM3 suffered from tracking loss in textureless areas."*
- **Source**: Source #28
- **Phase**: Phase 2
- **Target Audience**: Implementer / Engineer (component selection for C1)
- **Confidence**: ✅
- **Related Dimension**: SQ1, SQ3+SQ4 component C1 (VO/VIO), SQ5 (failure modes)
- **Fit Impact**: **prunes the C1 candidate landscape** — RTAB-Map and ORB-SLAM3 should not be pursued as C1 leads. Plausible C1 leads remain: VINS-Mono / VINS-Fusion / OpenVINS / OKVIS2 / DROID-SLAM / DPVO / pure VO baseline (KLT + RANSAC homography). NGPS (Fact #14) uses `ap_vips` = OpenVINS-class VIO — confirming an aligned community choice. Final C1 selection happens in SQ3+SQ4.
### Fact #17 — DSMAC + TERCOM lineage: pre-cached scene matching for downward-looking navigation is a 40+ year deployed technique class with documented sub-10 m terminal accuracy
- **Statement**: DSMAC (Digital Scene Matching Area Correlator) is an autonomous missile-guidance system based on area correlation of sensed downward-camera ground scenes against pre-stored reference imagery (often satellite reconnaissance). It achieves 310 m terminal accuracy by correlating buildings, road intersections, and distinctive terrain landmarks. Tomahawk: TERCOM (radar altimeter + DEM) for mid-flight + DSMAC for terminal guidance reduces CEP from ~30 m to "only meters". Documented combat record: 1991 Gulf War, >80% of 280 launched Tomahawks hit target. Recent miniaturisation: Destinus Ruta (300 km strike-class) is integrating UAV Navigation's (Spanish, Grupo Oesía) DSMAC-class system, validated in Ukrainian combat conditions including GNSS-denied / jamming / spoofing.
- **Source**: Source #36, Source #27
- **Phase**: Phase 2
- **Target Audience**: Domain expert + Decision-maker
- **Confidence**: ✅ for the lineage and Tomahawk performance numbers (DTIC + open-source); ⚠️ for the Ruta-specific "DSMAC operating principle" inference (Defense Express analyst inference, not vendor disclosure)
- **Related Dimension**: SQ1 (lineage), SQ8 (baseline accuracy expectations for AC-1.1.1 80 m / AC-NEW-4 false-position budget)
- **Fit Impact**: **establishes baseline accuracy expectations** — the technique class has documented sub-10 m accuracy in the cruise-missile-terminal regime. Our budget (AC-1.1.1: <80 m at 1 km AGL with ≥0.5 m/px tiles) is loose by comparison, indicating that the AC budget is *not* aggressive against the technique-class baseline — it is aggressive against the Jetson Orin Nano + 8-h-continuous + 25 W envelope. **Implication for AC-NEW-4**: claiming P(error >500 m) <0.1% per flight is consistent with the DSMAC-lineage class; an honestly-reported failure rate at this level is realistic, not unprecedented.
### Fact #18 — Hierarchical Image Matching (arXiv 2506.09748, June 2025) is a current academic SOTA pipeline for our exact problem, but uses DINOv2 — a heavyweight foundation model that must be benchmarked under our 25 W / 8 GB Jetson envelope before any selection
- **Statement**: 2025 academic SOTA pipeline structure: (1) image retrieval module (off-the-shelf, optimal-transport feature aggregation); (2) Semantic-Aware and Structure-Constrained Matching Module (SASCM) using **DINOv2** features + 4D correlation tensor + SoftMNN + 4D conv; (3) lightweight fine-grained matching module for pixel-level. Constructs UAV absolute visual localization without VIO/relative-localization dependence (retrieval-and-matching only). Evaluation on AerialVL + their own CS-UAV dataset claims superior accuracy under cross-source and cross-temporal variation.
- **Source**: Source #29
- **Phase**: Phase 2
- **Target Audience**: Implementer / Engineer + Domain expert
- **Confidence**: ✅ for pipeline structure and method; ⚠️ for "superior" claim (single-paper benchmark; AerialExtreMatch evaluates 16 methods with broader rigor — Source #34 is the better cross-method ranker); ❓ for Jetson-Orin-Nano runtime (no published number)
- **Related Dimension**: SQ1 (academic SOTA), C2 (VPR), C3 (cross-domain registration), SQ5 (foundation-model-on-Jetson failure mode)
- **Fit Impact**: **academic-SOTA snapshot, candidate template** — the retrieval → semantic-aware coarse → fine-grained pipeline is a candidate template for our C2+C3, but DINOv2 introduces a Jetson-deployment risk that must be quantified before commitment. Candidate-level decision: include DINOv2-based pipelines (AnyLoc, BoQ, this paper's SASCM) in the C2/C3 candidate matrix with mandatory MVE on Jetson Orin Nano under our exact frame size and 3 fps cadence. Reject DINOv2 if total inference latency cannot be brought under (400 ms - other-stages budget) at INT8 / fp16. Per Source #28 lesson, classical matchers (LightGlue+SuperPoint as in NGPS) should also be in the matrix as the "simple baseline / known-Jetson-runnable" option.
### Fact #19 — AerialExtreMatch (2025) is the academic benchmark our C2+C3 candidate matrix must publish numbers against, with 32 difficulty-stratified cells exposing exactly the cross-source / cross-pitch / cross-scale failure modes our project will face
- **Statement**: AerialExtreMatch publishes (a) 1.5 M synthetic train pairs (RGB+depth, diverse UAV/satellite viewpoints); (b) ~30,000 evaluation pairs in **32 difficulty levels** stratified by overlap (4 bins: <20%, 2040%, 4060%, >60%), pitch difference (4 bins: 5055°, 5560°, 6065°, 6570°), and scale variation (2 bins: 12×, >2×); (c) a real-world UAV-localization split captured with DJI M300 RTK + H20T against UAV-derived orthomosaic/DSM AND lower-quality satellite maps. The benchmark evaluates 16 representative detector-based and detector-free image matching methods.
- **Source**: Source #34
- **Phase**: Phase 2
- **Target Audience**: Domain expert + Implementer
- **Confidence**: ✅
- **Related Dimension**: SQ1 (academic landscape), SQ7 (datasets), C2 (VPR), C3 (cross-domain registration)
- **Fit Impact**: **defines the C2/C3 evaluation matrix** — every C2/C3 candidate going into `solution_draft01` must report numbers on AerialExtreMatch's 32 difficulty cells, with at least the high-pitch (6570°) and high-scale (>2×) cells representing our worst-case (UAV vs satellite tile geometry mismatch + ortho-rectification residual). The dataset's real-world UAV-localization split with both UAV-orthomosaic AND satellite-map references mirrors our project's offline-cache-tile semantics directly.
### Fact #20 — DARPA FLA + USAF SBIR establish the US-defense-program tailwind, but do not directly validate the project's specific regime (fixed-wing, ~1 km AGL, sat-tile basemap, 8-h endurance)
- **Statement**: DARPA Fast Lightweight Autonomy (FLA) program ran 20152018 (Phase 1 Florida 2017; Phase 2 Georgia 2018; complete). Focused on small quadcopter autonomy at ≤20 m/s through cluttered indoor/outdoor environments using onboard cameras + LIDAR + sonar + IMU, no GPS / datalink / pilot. A 2025 retrospective (arXiv 2504.08122) reviews FLA testing methodology and Phase 1 results. A 2025 USAF SBIR Phase II solicitation (Sweetspot ID `7946c818-409f-5b31-8f06-554466071d83`) is requesting visual position and navigation capability for sUAS in GPS-denied environments — confirming the regulatory + funding environment is currently active for this category in 2025.
- **Source**: Source #35
- **Phase**: Phase 2
- **Target Audience**: Decision-maker + Domain expert
- **Confidence**: ✅
- **Related Dimension**: SQ1 (defense-program lineage)
- **Fit Impact**: **context only, no direct candidate gain** — FLA pre-dates the project's specific regime by 8 years, focused on a different platform (multirotor) and altitude (low-altitude obstacle avoidance, not 1 km AGL nadir-camera satellite-anchor). Useful only to establish lineage and context. The USAF SBIR datapoint is more directly relevant: confirms that an active US-defense-funded need exists for sUAS visual position + navigation in GPS-denied environments — i.e., the project's market exists outside Ukraine.
---
## SQ1 — Conclusions (working summary, will be re-checked at Step 7.5)
### Existing-systems landscape (5 named-and-evidenced peer / adjacent systems)
| System | Class | Operational regime | Closest match dimension | Closest mismatch dimension | Status as evidence |
|---|---|---|---|---|---|
| **Twist Robotics OSCAR** (UA) | Deployed Ukrainian peer | Combat-deployed, fixed-wing-class, GPS-denied vision-nav | **Same architecture, same threat environment** | Altitude / endurance / FC / accuracy contract not publicly specified | Closest peer for "feasibility floor" |
| **Auterion Artemis** | Production COTS one-way attack drone | Shahed-class, 1000-mile range, 40 kg warhead, Ukraine-validated GPS-denied nav | Same architectural pattern (Skynode + Visual Navigation + terminal guidance) | One-way attack vs reusable; no covariance/source-label contract published | Closest production reference architecture |
| **Vantor Raptor (Guide / Sync / Ace)** | Production COTS software suite | Vision-based GPS replacement on existing drone camera + Vivid Terrain 3D basemap | Visual-position software pattern | Vendor-managed sat-tile basemap is not the project's Azaion Suite Satellite Service; no AC-NEW-4 / AC-1.4 contract | Closest commercial peer for "build-vs-buy" framing |
| **snktshrma/ngps_flight (NGPS, ArduPilot GSoC 2024)** | Open-source research prototype | LightGlue+SuperPoint+UKF+`VISION_POSITION_ESTIMATE` to AP | **Same component split, same FC family** | GSoC prototype, not production; no spoof defence; no covariance honesty | **Closest open-source pipeline match — explicit architectural template** |
| **CTU Prague SPRIN-D winner** | Academic / competition | Multirotor, ≤25 m AGL, LiDAR + heightmap gradient + particle filter on CPU | "Recover-from-uncertainty > low-instantaneous-RMSE" lesson; VIO discipline | LiDAR-required, low-altitude regime, no sat-tile basemap | Architectural-pattern reference + cautionary tale |
| **Destinus Ruta + UAV Navigation** | Production miniaturised cruise missile | 300 km strike, DSMAC-class, Ukraine-combat-validated | Pre-cached basemap + visual matching + autopilot ingestion | One-way attack, terminal guidance, no covariance contract | Shows DSMAC-class miniaturised into UAV tier |
### Per-perspective coverage
| Perspective | Facts supporting | Saturation status |
|---|---|---|
| **Implementer / Engineer** | Fact #14 (NGPS), Fact #16 (SLAM failure modes), Fact #18 (DINOv2 risk) | Saturated for SQ1 — deeper component-level deep-dives go to SQ3/SQ4 |
| **Practitioner / Field (Ukraine)** | Fact #11 (OSCAR), Source #37 (~70% UAV losses to EW), Source #27 (Ruta + UAV Navigation Ukraine combat validation) | Saturated for SQ1 |
| **Domain expert / Academic** | Fact #18 (Hierarchical Matching SOTA), Fact #19 (AerialExtreMatch benchmark), Fact #15 (SPRIN-D regime distinction) | Saturated for SQ1 — academic SOTA benchmarking handed off to SQ3/SQ4 + SQ7 |
| **Contrarian / Devil's advocate** | Fact #15 (low-altitude RGB matching unreliable lesson), Fact #16 (RTAB-Map / ORB-SLAM3 disqualified), Fact #18 (DINOv2-on-Jetson risk) | Saturated for SQ1 |
| **Decision-maker / Business** | Fact #12 (production-ready Auterion), Fact #13 (commercial Vantor build-vs-buy framing), Fact #20 (USAF SBIR market context) | Saturated for SQ1 |
### Architectural conclusions for `solution_draft01`
1. **Build-vs-buy stance**: build. Vantor Raptor and Auterion Visual Navigation are commercially superior on hardening + integration but neither exposes the covariance honesty contract (AC-NEW-4) nor uses the project-specified Azaion Suite Satellite Service tile cache (AC-2.x); both are dual-use export risks for the Ukrainian battlefield. NGPS (Fact #14) is the open-source architectural template to learn from but is a GSoC research prototype lacking production hardening, spoof defence, and the covariance-honesty contract. Architectural conclusion: build with NGPS as the template, with project-specific contracts (AC-NEW-4, AC-1.4, AC-NEW-7) and per-FC adapter (SQ6 conclusion) layered on top.
2. **Differentiation from OSCAR (Twist Robotics)** must be made explicit in `solution_draft01`: (a) honest covariance contract per AC-NEW-4; (b) explicit `{satellite_anchored, visual_propagated, dead_reckoned}` source-label contract per AC-1.4; (c) AC-NEW-7 cache-poisoning safety budget on tile write-back; (d) ArduPilot Plane + iNav both supported per project's revised AC-4.3.
3. **Pipeline canonicalness**: the C1+C2+C3+C4+C5+C8 split is canonical (NGPS + the 2025 hierarchical-matching paper + SPRIN-D winner all use the same shape; only the specific algorithm choices differ). SQ2 will sanity-check this against one more pipeline-survey paper, but this is essentially a low-risk question now.
4. **Component-pruning** carried into SQ3/SQ4:
- C1: **prune RTAB-Map and ORB-SLAM3** as primary candidates per Fact #16. Carry: VINS-Mono / VINS-Fusion / OpenVINS / OKVIS2 / DROID-SLAM / DPVO / pure VO baseline.
- C2/C3: **mandatorily benchmark** any DINOv2-based candidate (AnyLoc, BoQ, SASCM-style) against AerialExtreMatch at our pitch / scale / overlap regime AND against Jetson Orin Nano latency budget (per Fact #18). Maintain LightGlue+SuperPoint as the "simple-baseline / known-Jetson-runnable" option per NGPS precedent.
- C8 transport: NGPS uses `VISION_POSITION_ESTIMATE`. SQ6 picked `GPS_INPUT`. Re-examine the trade-off in design phase, but SQ6's selection stands for the research draft.
5. **Lessons from SPRIN-D winner that must propagate to `solution_draft01`**:
- "Ability to recover from periods of high uncertainty and re-localize" > "low instantaneous RMSE" — directly informs AC-NEW-2 / AC-NEW-8.
- VIO requires mechanically-decoupled IMU; this is a hardware-integration constraint, not a software issue.
- Magnetometer is unreliable near steel/concrete; sensor fusion of heading sources is essential.
- "No single sensor can be fully relied upon" — directly supports our IMU+camera+sat-tile multi-source posture.
### Open follow-ups (deferred to later sub-questions)
- **(SQ8)** Independent verification of OSCAR's "fully resistant to spoofing/jamming" claim — if available. Otherwise, Twist Robotics's claim remains a vendor-only signal.
- **(SQ8)** Vantor Raptor and Auterion Visual Navigation's covariance reporting behaviour — for benchmarking AC-NEW-4 compliance.
- **(SQ3+SQ4 / C2)** AnyLoc / BoQ / DINOv2-VLAD / MixVPR / EigenPlaces / NetVLAD on AerialExtreMatch for cross-source aerial — already in C2 search plan; SQ1 just confirmed they're the right candidate set.
- **(SQ3+SQ4 / C3)** LightGlue / LoFTR / RoMa / DKM / MASt3R + classical SIFT+RANSAC + XFeat on AerialExtreMatch — already in C3 search plan; SQ1 confirms shape.
- **(SQ7)** AerialExtreMatch + AerialVL + CS-UAV + RealUAV/SAVL + UAV-VisLoc as the dataset shortlist for our cross-validation — confirmed by SQ1 hits.
### Boundary check: SQ1 is saturated
Saturation signals observed: 4 perspectives saturated, ≥3 high-confidence facts per perspective, last 3 search rounds (Anduril Iris detail probe, ArduPilot prior-art probe, DSMAC lineage probe) yielded only one new substantive datapoint (NGPS) and confirmed already-known patterns. No unresolved contradictions. Per `references/source-tiering.md` "Search saturation rule" → SQ1 is closed.
@@ -0,0 +1,123 @@
# Fact Cards — SQ2: Canonical GPS-denied pipeline & SOTA components
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Extracted from sources logged in `../01_source_registry/SQ2_canonical_pipeline.md` (see `../01_source_registry/00_summary.md` for index). Confidence labels: ✅ High (L1 / verified source code), ⚠️ Medium (L1/L2 with caveat), ❓ Low (L3/L4 inferential). Bound to sub-questions in `../00_question_decomposition.md`.
>
> Index: [`../00_summary.md`](../00_summary.md). Sibling categories: SQ6 ([FC external positioning](SQ6_fc_external_positioning.md)), SQ1 ([existing systems](SQ1_existing_systems.md)), C1 ([VIO](C1_vio.md)), C2 ([VPR](C2_vpr.md)), C3 ([matchers](C3_matchers.md)).
**Facts in this file**: #21#27 (canonical pipeline definition, EKF fusion patterns, cross-domain matchers, hierarchical retrieval, end-to-end visual localization rejection, hardware MVE doctrine) + SQ2 working conclusions.
---
## SQ2 — Canonical pipeline decomposition (sanity-check)
### Fact #21 — The canonical pipeline for offline-cache visual geo-localization is two-stage: global VPR retrieval, then local alignment (image matching → pose)
- **Statement**: Source #38 (Skoltech aerial-VPR survey) defines the field's canonical pipeline verbatim: "Visual geolocalization can be implemented through various methods, typically relying on a pre-built database of images with known locations. This approach generally involves two stages: global localization (or Visual Place Recognition, VPR) and local alignment. Global localization involves identifying the nearest frame from the database (Image Retrieval), while local alignment determines the precise position using the selected frame." Source #42 (NUDT 2026 absolute-VL survey) names the same shape "**retrieval → matching → pose-estimation hierarchical framework**" and explicitly contrasts it against three rejected alternatives: (a) relative-only VIO/SLAM (cumulative error), (b) end-to-end direct localization (poor generalization), (c) map-free localization (scene-dependent). Source #39 (U.Maine cross-view survey) traces the same lineage from 2003 pixel-wise template-matching → 2013 hand-engineered features → 2017 CNN/triplet-loss → 2018+ Siamese/GAN → 2022+ Transformer → 2023 DINOv2-class. Source #41 (AnyVisLoc benchmark) implements this hierarchy as: image retrieval (rough) → image matching (2D-2D) → DSM-lift to 3D → PnP+RANSAC, with **Top-N re-rank by inlier count** as a critical fourth stage between matching and pose.
- **Source**: Source #38, Source #39, Source #41, Source #42
- **Phase**: Phase 2
- **Target Audience**: Architects of `solution_draft01`
- **Confidence**: ✅ (four independent surveys/benchmarks converge)
- **Related Dimension**: SQ2, C2 (VPR), C3 (cross-domain matching), C4 (pose estimation)
- **Fit Impact**: **confirms** the project's C1C10 decomposition is canonical for the **C2 → C3 → C4** chain. The component split is not novel; the project's contribution is the **integration discipline** (covariance honesty AC-NEW-4, source-label contract AC-1.4, offline-cache safety AC-NEW-7) layered on top. **Augment** the existing decomposition with an explicit "Top-N re-rank by inlier count" stage between C3 and C4 (currently implicit).
### Fact #22 — AdHoP (Adaptive Homography Preconditioning) is a method-agnostic post-matching refinement loop that improves translation accuracy by ~30% average and up to 63% for previously-underperforming methods, at the cost of a second matching pass
- **Statement**: Source #40 (OrthoLoC benchmark, Sep 2025): from initial 2D-2D query↔orthophoto correspondences, estimate a homography H via DLT+RANSAC, warp the orthophoto with H to better match the query's perspective (reducing residual perspective gap), re-match in this warped frame, then map the new correspondences back to the original orthophoto via H⁻¹, lift to 3D using DSM, and run PnP+RANSAC + Levenberg-Marquardt refinement. Accept the AdHoP-refined pose only if reprojection error decreases vs. the non-refined pose. **Quantitative effects** (16,425 images, 47 locations, 1m-1° threshold): GIM+DKM 75.4% recall (best); AdHoP-refined methods see ~30% average matching improvement, ~20% translation/rotation error reduction; for previously-underperforming methods AdHoP yields up to 95% matching improvement (XFeat*) or 63% translation reduction (DKM); for RoMa, AdHoP lifts 1m-1° recall by +23 points (54.6% → 77.6%-class). **Cross-domain regime** (war-zone-equivalent: scene change between query and reference): translation error increases ~3× when only the visual modality differs, ~7× when both visual and structural (DSM) gaps exist (0.16 m → 1.12 m for GIM+DKM+AdHoP). **Method-agnostic** — works on top of any 2D-2D matcher.
- **Source**: Source #40
- **Phase**: Phase 2
- **Target Audience**: System architects + C3/C4 implementers
- **Confidence**: ✅ for headline numbers (single-paper, but published dataset + open code + reproducible per repo)
- **Related Dimension**: SQ2 (new sub-stage), C3 (matcher), C4 (pose), SQ5 (cross-domain failure mode)
- **Fit Impact**: **adds a new sub-stage** between C3 and C4. Decision for `solution_draft01`: include AdHoP-class refinement as an **optional** stage gated on Jetson Orin Nano latency budget — if (single-pass match latency × 2) + homography estimation + reprojection check fits under (400 ms - other-stages), include it; otherwise reserve as offline-replay-time refinement. Cross-domain 3× translation-error penalty is a **direct AC-NEW-4 calibration input** — companion-side covariance must inflate proportionally when scene-change detection (deferred to SQ8) flags a stale tile.
### Fact #23 — 6-DoF aerial-to-satellite localization requires DSM (Digital Surface Model) elevation data; without DSM, the system collapses to 3-DoF (position + 1 rotation) or must compute attitude purely from IMU/VIO
- **Statement**: Source #40 OrthoLoC explicitly: "Our pipeline matches the query image with the DOP, lifts the matched 2D points in DOP to 3D using the DSM, and then estimates the camera pose using PnP and RANSAC." Without the DSM lift, the matcher produces 2D↔2D correspondences that constrain a homography (which encodes 3-DoF for a planar scene + planar camera) but **not** the full 6-DoF camera pose. Source #41 AnyVisLoc independently confirms by measuring: aerial-photogrammetry map (with paired DSM at 0.94 m/px) achieves 74.1% A@5m; satellite map (with ALOS 30 m DSM) achieves only 18.5% A@5m — a 4× accuracy collapse driven by DSM coarseness. The project's offline cache from the Azaion Suite Satellite Service is currently specified as **2D ortho tiles only** (no DSM commitment in restrictions.md or AC). **Three architectural responses** are available: (a) **3-DoF acceptance** — fix attitude from IMU/VIO, treat the matcher output as a homography-only constraint, ignore DSM; sacrifices the up-to-2× higher accuracy reported when DSM is present, but stays within current cache contract; (b) **Request DSM tiles from the Suite Sat Service** — adds C2 cache schema work + a Suite Sat Service contract change; preserves 6-DoF accuracy; (c) **IMU/VIO-only attitude + 2D-2D matching translation** — same as (a) but explicitly contracts the IMU/VIO module to provide attitude with σ ≤ 5° (per Fact #24); operationally identical to (a), differs only in how the contract is written.
- **Source**: Source #40, Source #41
- **Phase**: Phase 2
- **Target Audience**: System architects + Suite Sat Service stakeholder + AC owner
- **Confidence**: ✅ for the architectural claim; ✅ for the 4× accuracy collapse number
- **Related Dimension**: SQ2 (decomposition), C2 (cache schema), C3 (matcher output contract), C4 (pose), C5 (estimator), C6 (IMU/VIO contract), AC-1.1 / AC-1.1.1 (accuracy budget)
- **Fit Impact**: **architectural decision required, surfaced for user.** The current restrictions.md (no DSM commitment) implicitly forces option (a) or (c). The accuracy budget AC-1.1.1 (≤80 m at 1 km AGL) is loose enough that 3-DoF + IMU-attitude almost certainly satisfies it on a per-frame basis (per Fact #21 and DSMAC-class lineage in Fact #17), but **requires explicit acknowledgement** in the architecture before commitment. **Proposed default** for `solution_draft01`: option (c) — fix attitude from IMU/VIO with documented σ ≤ 5° contract on yaw, σ ≤ 5° on pitch (per Fact #24), translation from 2D-2D matching + camera pose. Flag option (b) as a "Suite Sat Service follow-up" if 6-DoF accuracy ever becomes a hard requirement.
### Fact #24 — IMU-derived yaw and pitch priors with σ ≤ 5° are required for the matching+PnP stack to hit benchmark accuracy; σ ≥ 10° causes 24% A@5m drops, σ ≥ 30° causes ≥4% drops, σ ≥ 60° causes 25.7% drops
- **Statement**: Source #41 AnyVisLoc systematically perturbs yaw and pitch priors and measures localization accuracy collapse. Yaw: σ = 5° → no impact; σ = 10° → 1.9% A@5m; σ = 30° → 4.1%; σ = 50° → 13.7%; σ = 60° → 25.7%. Pitch: σ < 5° → no impact; σ ≥ 7° → 15% drops. The benchmark is conducted at low altitude (30300 m AGL) with 2090° pitch range; lessons transfer to our 1 km AGL nadir-camera regime in the **direction** but the magnitudes may be lower at 1 km AGL because nadir geometry is less yaw-sensitive than oblique. Conservatively adopting the benchmark numbers gives a hard contract: **IMU/VIO must deliver yaw with σ ≤ 5° and pitch with σ ≤ 5° to the matcher** (1σ, not 95%, since the benchmark is single-σ). Pitch is naturally tighter on a nadir-fixed camera (mechanically constrained); yaw is the binding constraint and is the typical IMU/magnetometer failure mode (per SPRIN-D lesson Fact #15).
- **Source**: Source #41
- **Phase**: Phase 2
- **Target Audience**: System architects + C1 (VIO) implementer + C5 (estimator) implementer
- **Confidence**: ✅ for the AnyVisLoc numbers; ⚠️ for direct transfer to 1 km AGL nadir regime (magnitudes likely smaller at our altitude/pitch — direction is conservative)
- **Related Dimension**: SQ2 (sensor-prior contract), C1 (VIO output contract), C5 (estimator), C6 (IMU)
- **Fit Impact**: **architectural contract** for `solution_draft01`: the C1 module's published contract to the C2/C3 stack is yaw σ ≤ 5° AND pitch σ ≤ 5°. Magnetometer-only yaw is **insufficient** by the SPRIN-D lesson (Fact #15) — VIO must contribute. **Adds a constraint** that flows back to the C6 IMU integration: IMU mechanical isolation per SPRIN-D Fact #15 is required; magnetometer + GPS-yaw startup alignment at the airbase (before take-off, while real GPS is healthy) is part of the boot sequence.
### Fact #25 — Top-N re-ranking by inlier count is the dominant accuracy/cost trade-off; pure-matching-without-retrieval is catastrophic (A@5m collapses from 62.2% to 34.3% with the same matcher)
- **Statement**: Source #41 AnyVisLoc and Source #38 Skoltech survey both quantify the value of retrieval as a search-space reducer for matching. Source #41 explicitly: "Top-N re-rank by inlier count is the best accuracy/cost trade-off" → 62.2% A@5m at 0.8 s/frame on RTX 3090. **Without retrieval** (pure exhaustive matching against the cache): 34.3% A@5m — i.e., almost **half** the accuracy at infeasible compute. Source #38 measures sparse-VPR re-ranking specifically: AnyLoc descriptor + SuperGlue re-rank on top-100 candidates = 1525 s/frame on RTX 3090 (catastrophic for our 400 ms budget); LightGlue re-rank ≈ 1 s/frame (still over budget); SelaVPR re-rank < 0.1 s/frame (in-budget on RTX 3090, must be re-tested on Jetson Orin Nano). **Re-ranking budget** = (frame budget) (descriptor extraction) (initial top-N retrieval) (matcher pose estimation) (AdHoP if included).
- **Source**: Source #38, Source #41
- **Phase**: Phase 2
- **Target Audience**: System architects + C2 implementer
- **Confidence**: ✅ (two-source convergence on the qualitative claim; quantitative numbers are RTX-3090-specific and must be Jetson-MVE'd)
- **Related Dimension**: SQ2 (pipeline structure), C2 (VPR), C3 (matcher), SQ3+SQ4 (Jetson MVE)
- **Fit Impact**: **mandates** Top-N re-rank by inlier count as a stage in `solution_draft01`. Trade-off Top-N value (typical N=520 in literature) goes to SQ3+SQ4 candidate matrix, not SQ2.
### Fact #26 — High-accuracy SOTA models (AnyLoc + SuperGlue + RoMa-class) are NOT viable on Jetson Orin Nano under the 400 ms p95 budget; lightweight VPR (MixVPR / SALAD / SelaVPR-class) + lightweight matchers (LightGlue / XFeat-class) are the only candidates that survive a basic latency pre-screen
- **Statement**: Two independent runtime measurements on RTX 3090 (≥10× faster than Jetson Orin Nano in dense matrix ops): Source #38 — AnyLoc descriptor calculation 0.370.84 s/frame (huge ViT-G DINOv2); SuperGlue re-rank 1525 s/frame on top-100; LightGlue re-rank ~1 s/frame; SelaVPR re-rank < 0.1 s/frame. Source #41 — RoMa dense matcher 659 ms/frame; SP+LightGlue+GIM sparse 105 ms/frame; ratio = 6.3×. **Memory**: AnyLoc descriptors = 2.313.9 GB for 47k tiles (out of 8 GB Jetson Orin Nano envelope before model weights); SelaVPR descriptors < 0.2 GB. Pre-screen conclusion: AnyLoc / SuperGlue / RoMa-class are **disqualified** on the Jetson Orin Nano at 3 fps unless heavy quantization (INT8) reduces them ≥10×, which is not yet established for our latency target on this hardware. Surviving candidates from the literature: **VPR**: MixVPR, SALAD, SelaVPR, EigenPlaces, NetVLAD-class; **matchers**: LightGlue, XFeat, XFeat*, SP+LightGlue. **Disqualification is preliminary** — final go/no-go happens at SQ3+SQ4 with on-Jetson MVE per `references/mode-A-mve-rules.md`.
- **Source**: Source #38, Source #41
- **Phase**: Phase 2
- **Target Audience**: C2 + C3 implementer; SQ3+SQ4 candidate-matrix author
- **Confidence**: ✅ for RTX-3090 numbers; ⚠️ for direct Jetson translation (Jetson Orin Nano AI score is well-published; ratio is conservative)
- **Related Dimension**: SQ2 (Jetson budget feasibility), SQ3+SQ4 (candidate pre-screen), SQ5 (foundation-model-on-edge failure mode), C2, C3, C7 (Jetson runtime)
- **Fit Impact**: **prunes the SQ3+SQ4 candidate matrix BEFORE expensive Jetson MVE.** Candidates entering SQ3+SQ4 with mandatory Jetson MVE: (C2 VPR) MixVPR, SALAD, SelaVPR, EigenPlaces, NetVLAD; (C3 matcher) LightGlue, XFeat, XFeat*, SP+LightGlue. Candidates that need Jetson INT8 quant before they earn an MVE slot: AnyLoc, BoQ, DINOv2-VLAD (must demonstrate INT8 build path with vendor-validated accuracy preservation). Candidates pruned outright: RoMa dense, SuperGlue, MASt3R (latency).
### Fact #27 — A 20% covisibility floor between query frame and reference tile is required for localization to succeed; below it, ALL methods fail regardless of matcher quality
- **Statement**: Source #40 OrthoLoC: "When the covisibility between the UAV image and the orthographic geodata is too small (less than ~20%), the localization fails for all methods regardless of matcher quality." This is a geometric floor, not a method-specific limit. The implication for the project: any tile-cache design that allows a query to fall outside 20% covisibility with the **best available** cached tile must also include a **runtime covisibility-check + graceful degrade** to `visual_propagated` mode (per AC-1.4 source label). This is a runtime condition, not a one-time setup parameter.
- **Source**: Source #40
- **Phase**: Phase 2
- **Target Audience**: C2 (cache scheduler) + C5 (estimator) + AC-1.4 owner
- **Confidence**: ✅
- **Related Dimension**: SQ2 (boundary condition), C2 (tile cache), C5 (estimator state machine), AC-1.4
- **Fit Impact**: **adds a runtime invariant** to `solution_draft01`: tile selection must guarantee ≥20% covisibility OR explicitly emit the `visual_propagated` source label per AC-1.4 with covariance widened per AC-NEW-4. This becomes a hard constraint on the C2 cache schema (must support tile-extent metadata) and a runtime check before invoking C3 matcher.
---
## SQ2 — Conclusions (working summary, will be re-checked at Step 7.5)
### Pipeline-component coverage table (existing C1C10 vs. survey-listed components)
| Survey/benchmark canonical stage | Project component (current) | Coverage status | Required action |
|---|---|---|---|
| Image retrieval (global VPR) | **C2 — Visual Place Recognition** | ✅ covered | No change |
| Re-ranking (top-N inlier-based) | (currently implicit, inside C2 or C3) | ⚠️ implicit | **Promote to explicit sub-stage** (`C2.5` or `C3.0`) in `solution_draft01` |
| Local image matching (2D-2D, sparse or dense) | **C3 — Cross-domain registration** | ✅ covered | Add Top-N re-rank-by-inlier-count requirement |
| AdHoP-style perspective preconditioning | (not represented) | ❌ missing | **Add as optional sub-stage** between C3 and C4, gated on Jetson latency budget |
| 2D-3D lift via DSM | (not represented; current cache is 2D ortho only) | ❌ architectural decision required | **Decision required from user** — see below |
| Pose estimation (PnP + RANSAC + LM) | **C4 — Pose estimation** | ✅ covered | No change |
| State estimator / fusion (UKF / ESKF / MSCKF / factor graph) | **C5 — Estimator / fusion** | ✅ covered | Augmented with covariance-honesty contract from AC-NEW-4 |
| IMU + VIO contract | **C1 — VO/VIO** + **C6 — IMU integration** | ✅ covered | Add yaw σ ≤ 5°, pitch σ ≤ 5° hard contract from Fact #24 |
| Tile cache + scheduler | **C2 — VPR tile cache** + **C6 — Tile cache + spatial index** + **C10 — Pre-flight cache freshness pipeline** | ✅ covered | Add 20% covisibility runtime invariant (Fact #27). (Cache hygiene moved from former-C9 to C10 per 2026-05-08 C9 / SQ7 restructure.) |
| Anti-spoof / source-switch | **C7 — Spoof detection** + **C8 — FC adapter** | ✅ covered | Already addressed in SQ6 |
| Health monitoring / safety | **C10 — Safety / health monitoring** | ✅ covered | Already addressed |
### Architectural decisions surfaced (require user resolution before SQ3+SQ4 starts)
1. **DSM dependency on the Suite Sat Service tile cache** (per Fact #23). Three options:
- **(a) 3-DoF acceptance** — accept that without DSM, only position is recovered from matching; attitude is fixed by IMU/VIO with no satellite-tile cross-check. Lowest project scope. Requires AC budget verification (likely passes AC-1.1.1).
- **(b) Request DSM tiles** — Suite Sat Service contract change. Highest accuracy. Adds ~1 cycle to delivery. Recommended if 6-DoF accuracy ever becomes a hard AC.
- **(c) IMU/VIO-attitude + 2D-2D matching translation** — operationally identical to (a) but contracts the IMU/VIO module explicitly with σ ≤ 5° yaw / pitch (Fact #24).
- **Recommended default**: **(c)** — explicit IMU/VIO contract; fall back to (b) if AC tightens.
2. **AdHoP refinement loop** (per Fact #22). Three options:
- **(a) Always-on** — included in every frame; Jetson budget must accommodate 2× matching latency.
- **(b) Conditional** — only when initial reprojection error exceeds a threshold; gated on per-frame budget.
- **(c) Off (initial release)** — relegate to offline-replay refinement.
- **Recommended default**: **(b) Conditional** — fits within latency variance budget while capturing the cross-domain accuracy gain.
3. **Top-N re-rank promotion to explicit pipeline sub-stage** (per Fact #25). Recommendation: promote to a named sub-stage in `solution_draft01` with N as an SQ3+SQ4 hyperparameter sweep target.
### Component-pruning carried into SQ3+SQ4
- **C2 candidates entering SQ3+SQ4 with mandatory Jetson MVE**: MixVPR, SALAD, SelaVPR, EigenPlaces, NetVLAD.
- **C2 candidates entering SQ3+SQ4 conditional on INT8 quantization path**: AnyLoc, BoQ, DINOv2-VLAD.
- **C2 candidates pruned**: SuperGlue-as-reranker (latency).
- **C3 candidates entering SQ3+SQ4 with mandatory Jetson MVE**: LightGlue, XFeat, XFeat*, SP+LightGlue (NGPS template).
- **C3 candidates pruned**: RoMa, MASt3R, DKM (dense matcher latency on Jetson).
- **C3 candidates as "AerialExtreMatch reference points" only, NOT for production**: GIM+DKM, GIM+LightGlue (per Source #40, used as accuracy benchmark only).
### Boundary check: SQ2 is saturated
Saturation signals observed: (a) four independent surveys/benchmarks (Skoltech aerial-VPR survey, U.Maine cross-view survey, OrthoLoC benchmark, AnyVisLoc benchmark, NUDT 2026 absolute-VL survey) converge on the **same** "retrieval → matching → pose-estimation hierarchical framework" as canonical; (b) two independent runtime sources (Skoltech survey on RTX 3090; AnyVisLoc on RTX 3090 with explicit dense-vs-sparse breakdown) agree on the relative cost ordering of model classes; (c) cross-source agreement on AdHoP value (Source #40 only, but with reproducible code and dataset — single-source-but-strong evidence); (d) cross-source agreement on covisibility / sensor-prior thresholds. Two outstanding decisions are flagged for user — neither blocks SQ2's saturation status, both block SQ3+SQ4 start. Per `references/source-tiering.md` "Search saturation rule" → SQ2 is closed pending user decisions on DSM dependency + AdHoP gating.
@@ -0,0 +1,148 @@
# Fact Cards — SQ6: ArduPilot Plane vs iNav external positioning
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Extracted from sources logged in `../01_source_registry/SQ6_external_positioning.md` (see `../01_source_registry/00_summary.md` for index). Confidence labels: ✅ High (L1 / verified source code), ⚠️ Medium (L1/L2 with caveat), ❓ Low (L3/L4 inferential). Bound to sub-questions in `../00_question_decomposition.md`.
>
> Index: [`../00_summary.md`](../00_summary.md). Sibling categories: SQ1 ([existing systems](SQ1_existing_systems.md)), SQ2 ([canonical pipeline](SQ2_canonical_pipeline.md)), C1 ([VIO](C1_vio.md)), C2 ([VPR](C2_vpr.md)), C3 ([matchers](C3_matchers.md)).
**Facts in this file**: #1#10 (ArduPilot/iNav inbound positioning interfaces, covariance honesty, spoof-promotion, dead-reckoning, UBX emulation) + SQ6 working conclusions.
---
## SQ6 — ArduPilot Plane vs iNav external positioning
### Fact #1 — ArduPilot Plane EKF3 ingests `GPS_INPUT` (MAVLink ID 232) as a first-class GPS source
- **Statement**: ArduPilot's `AP_GPS_MAV` driver (master) decodes `MAVLINK_MSG_ID_GPS_INPUT` and stores the resulting state into the GPS slot identified by `gps_id`. Decoded fields: lat/lon (degE7), alt (mm → cm internally), hdop/vdop, velocity (vn/ve/vd cm/s), speed/horizontal/vertical accuracy (m / m/s), yaw (cdeg, `0` sentinel = "not provided"). Honors `ignore_flags` for ALT/HDOP/VDOP/VEL_HORIZ/VEL_VERT/SPEED_ACCURACY/HORIZONTAL_ACCURACY/VERTICAL_ACCURACY. Requires `fix_type ≥ 3` and `time_week > 0` for jitter-corrected timestamping.
- **Source**: Source #4 (AP_GPS_MAV.cpp master), Source #1 (Plane Non-GPS Navigation docs)
- **Phase**: Phase 2
- **Target Audience**: ArduPilot Plane operators / developers
- **Confidence**: ✅
- **Related Dimension**: C8 (FC adapter), C5 (estimator covariance contract)
- **Fit Impact**: **supports selection** — ArduPilot side of AC-4.3 is satisfied by `GPS_INPUT` as the primary external-positioning message; covariance fields (`horiz_accuracy`, `vert_accuracy`, `speed_accuracy`) are wired through.
### Fact #2 — ArduPilot's covariance honesty (AC-NEW-4) is enforced via the `horiz_accuracy` field of `GPS_INPUT`
- **Statement**: When `GPS_INPUT_IGNORE_FLAG_HORIZONTAL_ACCURACY` is unset, AP_GPS stores `packet.horiz_accuracy` into `state.horizontal_accuracy` and sets `state.have_horizontal_accuracy = true`. EKF3's quality chain consumes this via (a) ground-stationary 3 m drift check (`_gpsCheckScaler`-modulated), (b) innovation gating (`POS_I_GATE`/`VEL_I_GATE`), (c) soft de-weighting via `EK3_GLITCH_RADIUS` (PR #24135). Under-reporting `horiz_accuracy` defeats these gates — exactly the AC-NEW-4 risk the project flagged.
- **Source**: Source #4, Source #23 (PR #24135), Source #24 (AP_NavEKF3 master)
- **Phase**: Phase 2
- **Target Audience**: System designers writing the C5 estimator → C8 adapter
- **Confidence**: ✅ (source code + L1 docs); ⚠️ for the precise innovation-gate mechanics (deferred to design-phase SITL tuning)
- **Related Dimension**: C5 covariance, AC-NEW-4
- **Fit Impact**: **architectural constraint** — the C5 estimator MUST publish honest `horiz_accuracy` (not optimistic) for AP's EKF3 quality chain to function. Aligns directly with AC-1.4 / AC-NEW-4.
### Fact #3 — ArduPilot supports runtime EKF source-set switching from companion via `MAV_CMD_SET_EKF_SOURCE_SET`
- **Statement**: EKF3 supports up to three source sets (`EK3_SRC1..3_*`). A companion can request a switch by sending `MAV_CMD_SET_EKF_SOURCE_SET`. Alternative paths: RC aux-switch option 90 ("EKF Pos Source"), Lua scripts (e.g., `ahrs-source.lua`). **Caveat from L1 docs**: "no GCSs are currently known to implement this" — companion-driven switching works at the firmware level but is not exposed in stock GCS UIs.
- **Source**: Source #2, Source #3
- **Phase**: Phase 2
- **Target Audience**: System designers handling AC-NEW-2 spoof-promotion path on ArduPilot
- **Confidence**: ✅
- **Related Dimension**: C8 + AC-NEW-2
- **Fit Impact**: **supports selection** — AP allows the project to model two source sets (set 1 = real GPS, set 2 = onboard `GPS_INPUT`) and switch automatically. Keeps companion lightweight; switching does not require the companion to suppress real-GPS itself.
### Fact #4 — ArduPilot ODOMETRY-velocity-only fusion is currently NOT supported (open enhancement)
- **Statement**: Issue #23485 confirms current limitation: feeding `ODOMETRY` without position causes EKF position-estimate timeout / failsafe. Implication: the project's `visual_propagated` mode (VO drift between satellite anchors, no global position) **cannot be expressed as ODOMETRY-velocity-only on current AP** — must be sent as a full `GPS_INPUT` with covariance widened to reflect drift uncertainty.
- **Source**: Source #8
- **Phase**: Phase 2
- **Target Audience**: System designers
- **Confidence**: ✅ (open enhancement, open as of accessed date)
- **Related Dimension**: C5 + C8 + AC-1.3 (`visual_propagated` label) + AC-1.4 (covariance ellipse)
- **Fit Impact**: **architectural constraint**`visual_propagated` and `dead_reckoned` labels both ride `GPS_INPUT` with growing `horiz_accuracy`, NOT a separate `ODOMETRY` channel. Single-message contract = simpler. AC-NEW-8 thresholds (`horiz_accuracy = 999.0` for "no fix") map directly.
### Fact #5 — iNav firmware (master, post-9.0) has NO inbound MAVLink handler for any external-positioning message
- **Statement**: Authoritative inbound switch in `src/main/telemetry/mavlink.c::processMAVLinkIncomingTelemetry` (master) handles only: HEARTBEAT, PARAM_REQUEST_LIST (stub reply), MISSION_CLEAR_ALL, MISSION_COUNT, MISSION_ITEM, MISSION_REQUEST_LIST, MISSION_REQUEST, COMMAND_INT (only `MAV_CMD_DO_REPOSITION`), RC_CHANNELS_OVERRIDE, ADSB_VEHICLE, RADIO_STATUS. **No `GPS_INPUT`, `VISION_POSITION_ESTIMATE`, `ODOMETRY`, `GLOBAL_POSITION_INT`, or `GPS_RAW_INT` are accepted as inputs.** Wiki page (Source #10) confirms: "Limited command support: Commands that are not implemented are ignored."
- **Source**: Source #9 (master code), Source #10 (wiki, edited 2025-12-11)
- **Phase**: Phase 2
- **Target Audience**: System designers + AC-4.3 author
- **Confidence**: ✅
- **Related Dimension**: C8, AC-4.3
- **Fit Impact**: **DISQUALIFIES the literal AC-4.3 wording** ("the standard external-positioning message type(s) accepted by ArduPilot AND iNav"). No single MAVLink external-positioning message is accepted by both FCs. Project must adopt a per-FC adapter design and AC-4.3 must be revised to acknowledge two transports.
### Fact #6 — iNav accepts external GPS injection via two MSP paths; `MSP2_SENSOR_GPS` is the covariance-rich path
- **Statement**: `MSP_SET_RAW_GPS (201)` (legacy MSP1, 14 bytes): fixType, numSat, lat, lon, alt (m, internal cm), speed (cm/s). **No covariance, no per-axis velocity, no yaw.** `MSP2_SENSOR_GPS (7939, MSPv2 sensor plugin)`: instance, gpsWeek, msTOW, fixType, satellitesInView, hPosAccuracy (mm), vPosAccuracy (mm), hVelAccuracy (cm/s), hdop, lat, lon, mslAltitude (cm), nedVelNorth/East/Down (cm/s), groundCourse (cdeg×100), trueYaw (cdeg×100), date+time. Routes through `mspGPSReceiveNewData()` via `GPS_PROVIDER_MSP`. Requires build flag `USE_GPS_PROTO_MSP`**enabled by default in iNav's `target/common.h`**, so stock firmware reaches this path.
- **Source**: Source #12 (MSP message reference, master), Source #13 (target/common.h master + gps.c provider table)
- **Phase**: Phase 2
- **Target Audience**: System designers (C8 adapter, MSP transport)
- **Confidence**: ✅
- **Related Dimension**: C8, C5 covariance contract
- **Fit Impact**: **supports selection** of `MSP2_SENSOR_GPS` for the iNav adapter. Covariance fields (`hPosAccuracy`, `vPosAccuracy`, `hVelAccuracy`) align semantically with `GPS_INPUT.horiz_accuracy` / `vert_accuracy` / `speed_accuracy`, but unit conversions differ (mm vs m). The C8 adapter must therefore be FC-aware, not protocol-monomorphic.
### Fact #7 — iNav does NOT support dual-GPS arbitration; companion must be the SOLE GPS source
- **Statement**: Issue #10141 is an open feature request for dual-GPS support. Current iNav (master incl. 9.0.x) has single-GPS architecture with one UART selected as the GPS port. There is no primary/secondary failover and no per-instance arbitration in the nav stack.
- **Source**: Source #14
- **Phase**: Phase 2
- **Target Audience**: System designers (architecture)
- **Confidence**: ✅
- **Related Dimension**: C8, C5, AC-NEW-2 (spoof promotion)
- **Fit Impact**: **architectural constraint** — on iNav, real GPS receivers must NOT be wired directly to the FC. Real GPS goes to the companion; the companion fuses (or rejects) it and emits the single iNav-facing feed via MSP2_SENSOR_GPS (or via a UBX-emulation UART). AC-NEW-2 latency on iNav = companion's internal reaction time only; iNav does not participate in source switching at all.
### Fact #8 — iNav explicitly does NOT validate GPS for spoofing; anti-spoofing is fully the companion's responsibility
- **Statement**: iNav's `docs/GPS_fix_estimation.md` states verbatim: "Not a solution for GPS spoofing (GPS output is not validated in INAV)." Combined with Fact #7, the architectural conclusion on iNav: companion = anti-spoofing oracle + nav-camera estimator + IMU-propagation source, all collapsed into the single MSP2_SENSOR_GPS feed.
- **Source**: Source #15
- **Phase**: Phase 2
- **Target Audience**: System designers; AC-NEW-2 / AC-3.5 / AC-NEW-8 owners
- **Confidence**: ✅
- **Related Dimension**: AC-NEW-2, AC-3.5, AC-NEW-8
- **Fit Impact**: **supports selection** of "companion as iNav's only GPS"; **disqualifies** any architecture that relies on iNav-side spoof detection for AC-NEW-2 reaction.
### Fact #9 — iNav dead-reckoning has documented stability bugs under intermittent feeds; AC-NEW-8 must avoid letting iNav enter dead-reckoning
- **Statement**: Issue #10588 documents porpoising and motor-burst behaviour during intermittent GPS outages on iNav fixed-wing dead-reckoning. The community recommendation captured in the issue: "GPS should be rejected if providing erroneous coordinates rather than no fix." `inav_allow_dead_reckoning` (default OFF) and `inav_allow_gps_fix_estimation` (default OFF) are both fixed-state booleans — entering dead-reckoning mid-flight is a discrete transition, not a smooth degrade.
- **Source**: Source #15, Source #16 (Settings.md), Source #17 (#10588)
- **Phase**: Phase 2
- **Target Audience**: System designers; AC-NEW-8 owner
- **Confidence**: ✅ for setting names; ⚠️ for severity of stability bug (single open issue)
- **Related Dimension**: AC-NEW-8, AC-3.5, C8
- **Fit Impact**: **architectural constraint** — on iNav, the AC-NEW-8 path must keep emitting `MSP2_SENSOR_GPS` with growing `hPosAccuracy` rather than letting the feed drop and iNav switch to dead-reckoning. The "no fix" semantics on iNav must be expressed via `fixType` field of MSP2_SENSOR_GPS (not by silence). The horiz/vert accuracy fields are the only signal available; iNav has no equivalent of the AP `horiz_accuracy = 999.0` "no fix" sentinel — must verify which `fixType` enum values iNav treats as no-fix.
### Fact #10 — iNav supports UBX-only over UART (NMEA dropped in 7.0); UBX emulation is a viable third transport
- **Statement**: iNav 7.0 removed NMEA. Currently supports u-blox UBX protocol with version ≥ 15.00 in 9.0+. Recommended physical receivers: u-blox M8/M9/M10. Companion can implement a UBX-emulation writer on the iNav GPS UART (NAV-PVT mandatory; NAV-DOP optional). UBX carries `hAcc`/`vAcc`/`headAcc`/velocity components — covariance honesty preserved.
- **Source**: Source #11 (iNav GPS-and-Compass-setup wiki)
- **Phase**: Phase 2
- **Target Audience**: System designers (transport-choice)
- **Confidence**: ✅ for UBX-only; ⚠️ for "minimum NAV-* set" — the canonical U-blox protocol spec (Source filed in agent-tools as `fd8513f8-...txt`) plus iNav's `gps_ublox.c` drive the precise message set; **this is a follow-up search before final selection**.
- **Related Dimension**: C8 transport choice
- **Fit Impact**: **alternate candidate, NOT YET SELECTED** — UBX path bypasses MSP queueing/arbitration concerns and treats the companion as a normal GPS to iNav. Trade-off: implementation cost (UBX writer + correct ACK behaviour) vs. MSP path (already-designed wire format, but iNav-specific).
---
## SQ6 — Conclusions (working summary, will be re-checked at Step 7.5)
### Per-FC adapter design is unavoidable (single-message AC-4.3 wording is unsatisfiable)
| FC | Inbound external-positioning transport | Message | Covariance fields | Per-axis velocity | Yaw | Source-switching from companion |
|---|---|---|---|---|---|---|
| **ArduPilot Plane** | MAVLink (TELEM/USB/UDP serial) | `GPS_INPUT` (id 232) — primary | `horiz_accuracy`, `vert_accuracy`, `speed_accuracy` (m/m·s⁻¹) | `vn`, `ve`, `vd` (cm/s) | `yaw` cdeg, 0 = not provided | `MAV_CMD_SET_EKF_SOURCE_SET` (FW supports; stock GCS UIs do not — companion-driven OK) |
| **iNav** | MSP2 (UART/USB) | `MSP2_SENSOR_GPS` (id 7939) — primary candidate | `hPosAccuracy` mm, `vPosAccuracy` mm, `hVelAccuracy` cm/s | `nedVelNorth/East/Down` cm/s | `trueYaw` cdeg×100 | **N/A** — iNav has single-GPS arch; companion = sole GPS source |
| iNav alt 1 | MSP1 | `MSP_SET_RAW_GPS` (id 201) — **rejected for production** | none | none | none | N/A |
| iNav alt 2 | UART | UBX emulation (NAV-PVT etc.) — **alternate candidate, requires NAV-* subset verification** | UBX `hAcc`/`vAcc`/`headAcc` mm/cm/scale | NED in NAV-PVT | yes | N/A |
**Selection (preliminary, pending Step 7.5 component-fit gate):**
- **AP path**: `GPS_INPUT` — Selected (lead).
- **iNav path**: `MSP2_SENSOR_GPS` — Selected (lead). UBX-emulation kept as fallback if MSP2_SENSOR_GPS proves rate-limited or quality-flag-lossy.
### AC / Restriction binding (per-mode, Per-Mode API Capability Verification rule)
| Numbered AC / Restriction | AP `GPS_INPUT` | iNav `MSP2_SENSOR_GPS` | iNav `MSP_SET_RAW_GPS` |
|---|---|---|---|
| AC-1.4 (95% cov + source label `{satellite_anchored, visual_propagated, dead_reckoned}`) | **Pass** (`horiz_accuracy` carries 95% covariance proxy; source label is companion-side metadata, not in MAVLink — emit via STATUSTEXT/NAMED_VALUE_FLOAT) | **Pass** (`hPosAccuracy` = covariance proxy; same off-band source-label channel) | **Fail** (no covariance field → cannot publish 95% ellipse) |
| AC-NEW-4 (false-position safety budget; covariance honesty) | **Pass** (de-weighted via `EK3_GLITCH_RADIUS` if covariance is honest) | **Verify** (need to confirm iNav nav-stack actually uses `hPosAccuracy` for outlier handling — pre-Step-7.5 follow-up) | **Fail** |
| AC-NEW-2 (<3 s p95 spoof promotion) | **Verify** via SITL (`MAV_CMD_SET_EKF_SOURCE_SET` round-trip latency under load) | **Pass** by architecture (companion is sole GPS, no FC-side switch needed) | Pass-by-arch but Fails AC-1.4 |
| AC-NEW-8 (visual-blackout + spoofed GPS failsafe; covariance growth + degraded fix levels) | **Pass** (`fix_type` 0/1/2 + `horiz_accuracy=999.0` documented sentinel maps to AC-NEW-8 thresholds) | **Verify** (iNav's `fixType` enum mapping for "no fix" — pre-Step-7.5 follow-up) | **Fail** (no graceful degrade signal) |
| AC-3.5 (label switch within ≤1 frame OR ≤400 ms; reject spoofed GPS as input) | **Pass** by architecture (EKF source switch + STATUSTEXT) | **Pass** by architecture (companion suppresses spoofed-GPS contribution upstream) | Pass-by-arch but Fails AC-1.4 |
| AC-4.3 (FC accepts the chosen messages) | **Pass** | **Pass** (default build, `USE_GPS_PROTO_MSP` on) | **Pass** but Fails AC-1.4 — discard |
| Restriction "Supported FCs: ArduPilot, iNav (both via standard MAVLink)" | **Pass** | **Fail** of "via standard MAVLink" — restriction's literal wording is incorrect because iNav has no inbound MAVLink external-positioning. The restriction must be revised to "ArduPilot via MAVLink GPS_INPUT; iNav via MSP2_SENSOR_GPS". | n/a |
### Required AC / Restrictions edits flagged for user review
1. **AC-4.3** — current text says "the standard external-positioning message type(s) accepted by ArduPilot and iNav". Reality: no single message type is accepted by both. **Proposed revision** (outcome-shaped, IEEE-830-style): "WGS84 coordinates are delivered to each supported FC via that FC's documented external-positioning interface — MAVLink `GPS_INPUT` for ArduPilot Plane, MSP2 `MSP2_SENSOR_GPS` for iNav. Honest covariance is carried in the field each FC uses for outlier rejection (under-reported covariance is a defect — see AC-NEW-4). Source-label semantics per AC-1.4 are emitted out-of-band (FC-appropriate STATUSTEXT / NAMED_VALUE_FLOAT / equivalent)."
2. **Restriction "Communication protocol (pinned): MAVLink for both FC and GCS"** — incorrect for iNav. **Proposed revision**: "Communication protocol: MAVLink for ArduPilot Plane and for QGroundControl GCS; MSP2 for iNav (UART or USB transport). MAVLink remains the GCS-facing protocol for both FCs." (iNav still emits MAVLink telemetry outbound to QGC; this is preserved.)
3. **AC-NEW-2** — keep numerical budget (<3 s p95) but split per-FC validation: ArduPilot validation = SITL round-trip of `MAV_CMD_SET_EKF_SOURCE_SET` from companion under spoof injection; iNav validation = companion-internal reaction time (companion-only metric — iNav doesn't participate).
4. **AC-NEW-8** — language "fix-quality 2D fix or worse when covariance > 100 m" maps to `GPS_INPUT.fix_type` for AP. iNav's `fixType` enum mapping (per `gpsFixType_e` in iNav's enums-reference) must be confirmed at design time before this AC is testable on iNav.
### Open follow-up probes (deferred to SQ8 + design phase, NOT blocking SQ6 closure)
- **(SQ8)** Confirm the precise MAVLink message + field set ArduPilot exposes for spoofing/jamming integrity reports (PR #2110 merged, but `GPS_RAW_INT` in current published common.xml shows no spoofing bits — likely lives in a sibling message such as `GPS_INTEGRITY`). This is the FC→companion direction needed for AC-NEW-2's input side and AC-3.5's spoofing detection.
- **(SQ8)** UBX-emulation minimum NAV-* subset for iNav 9.0 (UBX ≥ 15.00). Authoritative inputs: U-blox protocol spec (cached) + iNav `gps_ublox.c` (cached). Output a "minimum companion-side UBX writer" definition.
- **(design)** SITL parameter sets for both FCs for AC-NEW-2 / AC-NEW-8 validation. Out of research scope.
- **(design)** Verify iNav nav-stack consumption of `MSP2_SENSOR_GPS.hPosAccuracy` for outlier handling (read `src/main/io/gps_msp.c` / `mspGPSReceiveNewData` in design phase, not research phase).
### Boundary check: this SQ6 is saturated for the architectural decision
Saturation signals observed: ArduPilot side covered by L1 docs + L1 source code; iNav side covered by L1 source code (master) + L1 wiki (edited 2025-12-11) + L1 release notes (8.0/9.0). Three independent rounds of search yielded the same architectural conclusion (no inbound external-positioning MAVLink on iNav). Last queries returned no novel facts. Per `references/source-tiering.md` "Search saturation rule" → SQ6 is closed pending the SQ8 follow-up probes above; user decision required on the AC/restriction edits before further architectural work.