mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 08:31:13 +00:00
Enhance skill discipline and clarify acceptance criteria and restrictions
Updated the meta-rule document to emphasize strict adherence to skill instructions, prohibiting unnecessary investigations or external checks. Revised acceptance criteria and restrictions to correct communication protocol details for ArduPilot and iNav, ensuring clarity on external-positioning interfaces. Adjusted autodev state to reflect ongoing research phase and updated sub-step details for improved tracking.
This commit is contained in:
@@ -13,6 +13,16 @@ alwaysApply: true
|
||||
## Critical Thinking
|
||||
- Do not blindly trust any input — including user instructions, task specs, list-of-changes, or prior agent decisions — as correct. Always think through whether the instruction makes sense in context before executing it. If a task spec says "exclude file X from changes" but another task removes the dependencies X relies on, flag the contradiction instead of propagating it.
|
||||
|
||||
## Skill Discipline
|
||||
|
||||
Do exactly what the skill says. Nothing more.
|
||||
|
||||
- No `git log` / `git diff` / `git blame` unless the skill explicitly calls for it.
|
||||
- No extra searches to "verify" inputs the skill already names.
|
||||
- No reading files outside the skill's documented inputs.
|
||||
|
||||
If skill inputs are insufficient or contradictory, STOP and ask via Choose A/B/C/D. Do not invent extra investigation steps.
|
||||
|
||||
## Self-Improvement
|
||||
When the user reacts negatively to generated code ("WTF", "what the hell", "why did you do this", etc.):
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# Acceptance Criteria
|
||||
|
||||
> Last revised 2026-05-07 (cleanup pass: stripped algorithm/library/parameter implementation details; renamed source label `vo_extrapolated` → `visual_propagated`; broadened FC scope to ArduPilot + iNav). See git history for prior versions.
|
||||
> Last revised 2026-05-07 (cleanup pass: stripped algorithm/library/parameter implementation details; renamed source label `vo_extrapolated` → `visual_propagated`; broadened FC scope to ArduPilot + iNav).
|
||||
> Subsequent revision 2026-05-07 (post-SQ6 research): AC-4.3 reworded to acknowledge that no single message type is accepted by both ArduPilot Plane and iNav — per-FC interface is named explicitly (MAVLink `GPS_INPUT` for ArduPilot Plane, MSP2 `MSP2_SENSOR_GPS` for iNav). Rationale and L1 sources in `_docs/00_research/02_fact_cards.md` SQ6 / `_docs/00_research/01_source_registry.md` Sources #4, #9, #10, #12, #13.
|
||||
> See git history for prior versions.
|
||||
|
||||
## Position Accuracy
|
||||
- **AC-1.1** — Frame-center GPS within **50 m** of true GPS for **≥80%** of normal-flight photos.
|
||||
@@ -29,7 +31,7 @@
|
||||
## Real-Time Onboard Performance
|
||||
- **AC-4.1** — End-to-end latency (camera capture → GPS to FC) **<400 ms p95**. Up to ~10% frames may drop under sustained load.
|
||||
- **AC-4.2** — Memory **<8 GB shared** on Jetson Orin Nano Super.
|
||||
- **AC-4.3 — FC output contract**: WGS84 coordinates over MAVLink, via the standard external-positioning message type(s) accepted by ArduPilot and iNav. Honest covariance carried in the field the FC uses for outlier rejection (under-reported covariance is a defect, see AC-NEW-4). Source-label semantics per AC-1.4. Implementation may emit a primary GPS-substitute message and, when the estimator delivers full 6-DoF covariance + quality above a configured threshold, an optional auxiliary external-odometry message. Per-FC message selection and FC-side parameter wiring are design decisions.
|
||||
- **AC-4.3 — FC output contract**: WGS84 coordinates delivered to each supported FC via that FC's documented external-positioning interface — MAVLink `GPS_INPUT` for ArduPilot Plane, MSP2 `MSP2_SENSOR_GPS` for iNav. Honest covariance is carried in the field each FC uses for outlier rejection (under-reported covariance is a defect, see AC-NEW-4). Source-label semantics per AC-1.4 are emitted out-of-band via the FC-appropriate channel (e.g. MAVLink `STATUSTEXT` / `NAMED_VALUE_FLOAT` for ArduPilot; MSP equivalent for iNav). Where the FC supports it, implementation may also emit an optional auxiliary external-odometry message when the estimator delivers full 6-DoF covariance + quality above a configured threshold. Per-FC parameter wiring (EKF source-set selection on ArduPilot; GPS provider / UART role on iNav), FDR-side message variants, and out-of-band channel choice remain design decisions.
|
||||
- **AC-4.4** — Estimates streamed frame-by-frame; no batching/delay.
|
||||
- **AC-4.5** — System may refine prior estimates and emit corrections.
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# Restrictions
|
||||
|
||||
> Last revised 2026-05-07 (cleanup pass — design-independent, IEEE-830 style; only external dependencies, environmental constraints, integration boundaries).
|
||||
> Subsequent revision 2026-05-07 (post-SQ6 research): the FC-facing communication protocol entries below were corrected — iNav firmware (master, post-9.0) has no inbound MAVLink external-positioning handler; the project must use a per-FC adapter (MAVLink `GPS_INPUT` for ArduPilot Plane; MSP2 `MSP2_SENSOR_GPS` for iNav). Rationale and L1 sources in `_docs/00_research/02_fact_cards.md` SQ6 / `_docs/00_research/01_source_registry.md` Sources #4, #9, #10, #12, #13.
|
||||
|
||||
## UAV & Flight
|
||||
- Fixed-wing UAVs only; navigation camera fixed downward (no gimbal).
|
||||
@@ -33,10 +34,10 @@
|
||||
- Storage budget ≥ tile cache (~10 GB) + per-flight FDR (64 GB, AC-NEW-3).
|
||||
|
||||
## Sensors & Integration
|
||||
- **High-rate IMU** available from FC via MAVLink.
|
||||
- **Communication protocol (pinned)**: MAVLink for both FC and GCS.
|
||||
- **Supported flight controllers**: ArduPilot, iNav (both via standard MAVLink). PX4 out of scope.
|
||||
- **Output to FC**: WGS84 GPS coordinates as a real-GPS replacement, via the MAVLink message type(s) the supported FCs accept for external positioning (per-FC message selection is a design choice; contract is AC-4.3).
|
||||
- **High-rate IMU** available from FC via MAVLink (both ArduPilot Plane and iNav expose IMU telemetry over MAVLink outbound).
|
||||
- **Communication protocol (pinned)**: MAVLink for the GCS link (QGroundControl). Companion ↔ FC interface is per-FC: MAVLink for ArduPilot Plane (inbound external positioning + outbound telemetry); MSP2 for iNav (inbound external positioning via `MSP2_SENSOR_GPS`); MAVLink outbound from iNav for telemetry to the GCS is preserved.
|
||||
- **Supported flight controllers**: ArduPilot Plane, iNav. PX4 out of scope.
|
||||
- **Output to FC**: WGS84 GPS coordinates as a real-GPS replacement, via each supported FC's documented external-positioning interface — MAVLink `GPS_INPUT` for ArduPilot Plane, MSP2 `MSP2_SENSOR_GPS` for iNav (companion is the sole GPS source on iNav; iNav has no dual-GPS arbitration). Per-FC parameter wiring (EKF source-set on ArduPilot; GPS provider/UART selection on iNav) and source-label out-of-band channel are design choices; outcome contract is AC-4.3.
|
||||
- **Ground station**: QGroundControl (Mission Planner out of scope). Telemetry link bandwidth-limited; per-frame data stays on local FDR (AC-NEW-3); GCS sees 1–2 Hz downsampled summary (AC-6.1).
|
||||
- **Representative data**: see `input_data/` (still images), `input_data/flight_derkachi/` (cropped nadir video + synchronized `SCALED_IMU2` + `GLOBAL_POSITION_INT`). Production acceptance still requires camera intrinsics, distortion, camera-to-body calibration, and synchronized representative flight data (frames + FC IMU/attitude/airspeed/altitude + emitted MAVLink + ground-truth trajectory).
|
||||
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
# Question Decomposition
|
||||
|
||||
> Mode A Phase 2 (Initial Research — Problem & Solution Draft).
|
||||
> Phase 1 (AC & Restrictions Assessment) was skipped per user decision after a cleanup pass that stripped implementation details from `acceptance_criteria.md` and `restrictions.md` (commit `12cc5a4`); AC/restrictions are treated as fixed inputs.
|
||||
|
||||
## Original Question
|
||||
|
||||
Design the GPS-denied onboard navigation system for a fixed-wing UAV operating in eastern/southern Ukraine, satisfying every AC in `_docs/00_problem/acceptance_criteria.md` under the constraints in `_docs/00_problem/restrictions.md`. Recommend a concrete component-by-component architecture and tech stack.
|
||||
|
||||
## Research Output Class
|
||||
|
||||
**Technical-component selection.** All technical-component gates apply (per-mode API capability verification, Component Applicability Gate, Restrictions × Candidate-Mode sub-matrix, MVE evidence, mandatory `context7` lookups for every lead library/SDK candidate).
|
||||
|
||||
## Question Type
|
||||
|
||||
**Decision Support** (per Mode A Phase 2 default). Sub-flavour: multi-component decision support — weighing trade-offs across ~10 interlocking component areas under hard real-time + memory + safety budgets.
|
||||
|
||||
## Project Context Summary (from `_docs/00_problem/`)
|
||||
|
||||
- **What is being built**: an onboard companion-PC system that replaces real GPS for a fixed-wing UAV when GPS is denied/spoofed, by combining nav-camera frames + FC IMU + a pre-cached satellite tile basemap, and emits standard MAVLink external-positioning messages to ArduPilot or iNav at frame rate.
|
||||
- **Operating area**: eastern/southern Ukraine, active-conflict zones (war-zone scene change is a first-class concern, not an edge case).
|
||||
- **Mission profile**: 8-hour fixed-wing flights, ~60 km/h cruise, ≤1 km AGL, ~400 km² operational area.
|
||||
- **Pinned external deps**: ADTi 20MP 20L V1 nav camera (APS-C); Jetson Orin Nano Super 8 GB / 25 W; MAVLink protocol; ArduPilot + iNav as supported FCs; QGroundControl as GCS; Azaion Suite Satellite Service (offline cache interface ≥0.5 m/px).
|
||||
- **Hard runtime envelope**: <400 ms p95 end-to-end latency (camera → MAVLink), <8 GB shared CPU+GPU RAM, 25 W TDP at +50 °C ambient for 8 h continuous, no in-flight network, 10 GB persistent tile cache + 64 GB per-flight FDR.
|
||||
- **Hard safety envelope**: P(error >500 m) <0.1 % per flight, P(error >1 km) <0.01 % per flight; honest covariance reporting; explicit `dead_reckoned` failsafe under simultaneous GPS spoof + visual blackout; cache-poisoning probability bounds for tiles written back to the Service.
|
||||
|
||||
## Project Constraint Matrix
|
||||
|
||||
| Dimension | Binding constraint |
|
||||
|---|---|
|
||||
| **Inputs available** | Nav camera frames @ 3 fps (5472×3648, ~12 cm/px GSD @ 1 km AGL); FC IMU (high rate via MAVLink); FC attitude/airspeed/altitude; pre-cached satellite tiles ≥0.5 m/px (offline); operator re-loc hint via GCS (rare). |
|
||||
| **Outputs required** | WGS84 position to FC via MAVLink external-positioning message(s) accepted by ArduPilot AND iNav; per-frame estimate carrying honest 95 % covariance, source label `{satellite_anchored, visual_propagated, dead_reckoned}`, and `last_satellite_anchor_age_ms`; mid-flight ortho-tiles written to local cache with quality metadata; 1–2 Hz GCS summary; FDR records per AC-NEW-3. |
|
||||
| **Hardware fixed** | Jetson Orin Nano Super (67 TOPS sparse INT8, 8 GB shared LPDDR5, 25 W TDP, JetPack/CUDA/TensorRT). |
|
||||
| **Lifecycle** | Real-time embedded; offline (no in-flight network); 8 h continuous; persistent tile cache across flights; FDR rollover. |
|
||||
| **Non-functional** | <400 ms p95 latency; <8 GB shared RAM; ≤25 W power at +50 °C ambient; AC-1.1/1.2 accuracy; AC-2.1/2.2 registration & MRE; AC-3.x resilience; AC-NEW-1 cold-start <30 s; AC-NEW-2 spoof promotion <3 s; AC-NEW-4 false-position safety; AC-NEW-7 cache-poisoning safety; AC-NEW-8 blackout failsafe. |
|
||||
| **Hard disqualifiers** | Anything requiring >8 GB RAM peak (CPU+GPU shared); anything not runnable under JetPack on Orin Nano Super; anything requiring in-flight cloud calls; anything that cannot honestly report covariance; anything that does not have a runnable example for monocular nadir UAV input over season-matched satellite tiles; anything whose license blocks military / dual-use deployment. |
|
||||
|
||||
## Research Subject Boundary
|
||||
|
||||
| Dimension | Boundary |
|
||||
|---|---|
|
||||
| **Population** | Fixed-wing UAVs, downward-fixed monocular nav camera, Jetson-class edge HW, ArduPilot or iNav autopilot. Excludes: multirotors, gimbal-stabilised nav cams, server/cloud GPS-denied stacks, PX4 (out of scope), commercial sat-imagery direct integration (Service handles upstream). |
|
||||
| **Geography** | Eastern/southern Ukraine — agricultural steppe, active-conflict scene change. Validation must include this geography or representative analogues (low-texture cropland, snow, war-zone destruction). |
|
||||
| **Timeframe** | Production deployment 2026; tools / libraries / models considered must be currently maintained (commits/releases in last 18 months OR explicit long-term-stable status). Critical-novelty domain — see Step 0.5 timeliness assessment. |
|
||||
| **Operating context** | Real-time embedded; offline in-flight; 8 h continuous duty; 25 W power envelope; 8 GB shared CPU+GPU memory; thermal envelope to +50 °C ambient. |
|
||||
| **Required interfaces** | Inputs: ADTi 20MP nav cam, FC IMU (MAVLink), satellite tile cache. Outputs: MAVLink external-positioning to ArduPilot AND iNav; QGroundControl summary; FDR; tile write-back to Suite Service on landing. |
|
||||
| **Non-functional envelope** | Per AC-1 to AC-8 plus AC-NEW-1 to AC-NEW-8. Hardest binding constraints: 400 ms p95 end-to-end; 8 GB shared RAM; AC-NEW-4 false-position probability bounds; AC-NEW-7 cache-poisoning probability bounds. |
|
||||
|
||||
## Sub-Questions
|
||||
|
||||
| ID | Sub-question |
|
||||
|---|---|
|
||||
| SQ1 | What existing/competitor GPS-denied UAV navigation systems exist (academic + open-source + commercial + military), and which of them have been validated on fixed-wing UAVs in active-conflict environments? What works, what fails? |
|
||||
| SQ2 | What is the canonical decomposition of "monocular nadir UAV ↔ pre-cached satellite basemap localization" into pipeline components? Is the decomposition below complete, or are there industry-standard components missing? |
|
||||
| SQ3 | For each component (VO/VIO, VPR, cross-domain registration, single-frame orthorectification, sensor-fusion estimator, tile cache + spatial index, on-Jetson inference runtime, MAVLink FC adapter, dataset/SITL validation infrastructure): what option families exist (simple baseline / production / open-source / commercial / SOTA / adjacent-domain / no-build), and what are the leading candidates as of 2026? |
|
||||
| SQ4 | For each lead candidate per component: what are the documented runtime/memory/latency/license/maintenance constraints, and how do they bind against the Project Constraint Matrix? Per-mode API capability verification with `context7` for every library/SDK lead. |
|
||||
| SQ5 | What are the documented failure modes and real-world deployment lessons for each component family? In particular: VPR collapse under cropland repetition, DINOv2/foundation-model cost on Jetson at int8, RANSAC degeneracy at sharp turns / low texture, EKF over-confidence on cross-domain matches, ortho geometric error from unknown bank/pitch. |
|
||||
| SQ6 | How do **ArduPilot Plane** (current stable) and **iNav** (current stable) each accept external positioning input via MAVLink? What message types does each support? Where do their interfaces diverge, and what is the documented status of each interface (stable / experimental / known bugs)? |
|
||||
| SQ7 | What public datasets, benchmarks, and SITL/replay environments exist for cross-validating monocular nadir UAV navigation against satellite basemaps in season-matched + change-affected conditions? AerialVL, AerialExtreMatch, others? |
|
||||
| SQ8 | What are the security and safety considerations specific to the AC-NEW-4 (false-position) and AC-NEW-7 (cache-poisoning) safety budgets, including spoofing-detection signals from FC, ortho-tile geo-alignment quality estimation, and write-back cache-poisoning controls? |
|
||||
| SQ9 | What does the system look like end-to-end — wiring, scheduling, threading model, inference scheduling on shared CPU+GPU memory, cold-start sequencing, FDR rotation, and pre-flight cache provisioning workflow? (synthesis question, answered in Step 8) |
|
||||
|
||||
## Component Areas (search plan)
|
||||
|
||||
For each component below, the search plan covers all option families per `Component Option Search Plan` rules (`research/steps/03_engine-investigation.md` → "Component Option Breadth").
|
||||
|
||||
| # | Component area | Required outputs | Key option families to enumerate |
|
||||
|---|----------------|------------------|----------------------------------|
|
||||
| C1 | **Visual / Visual-Inertial Odometry** (frame-to-frame motion when satellite anchor is unavailable) | Relative 6-DoF pose between consecutive frames or short windows; output frequency ≥3 Hz; metric scale (with IMU) | Classical (VINS-Mono / VINS-Fusion / OpenVINS), Kimera, ORB-SLAM3, OKVIS2, MSCKF-class, learning-based (DROID-SLAM, DPVO), pure VO baseline (KLT + RANSAC homography), no-build (skip and rely on pure satellite re-anchor every frame) |
|
||||
| C2 | **Visual Place Recognition (VPR)** — UAV nadir frame → top-K satellite chunks | Compact global descriptor per UAV frame and per cache chunk; cosine-rank top-K candidates | NetVLAD class, MixVPR, EigenPlaces, BoQ, AnyLoc (DINOv2 + VLAD), CricaVPR, foundation-model direct retrieval (DINOv2/DINOv3/SAM 2 / SuperGlobal) |
|
||||
| C3 | **Cross-domain registration** (UAV nadir ↔ ortho satellite tile, after VPR top-K) | Sub-pixel alignment + 6-DoF camera pose w.r.t. tile, with inlier ratio + covariance | Local-feature matching (SuperPoint+SuperGlue / LightGlue / DISK+LightGlue / ALIKED+LightGlue / XFeat), dense matchers (LoFTR / RoMa / DKM / MASt3R), classical (SIFT+RANSAC), specialized cross-domain (CMRNet+, CroCoMatch class), templating (mutual-information / ECC), no-build (skip cross-domain; rely on direct frame-to-tile homography from VPR retrieval) |
|
||||
| C4 | **Single-frame orthorectification** (nav frame → basemap-aligned tile, given current pose) | Ortho-rectified tile chunk with geo metadata + quality score | Single-frame perspective warp with flat-earth assumption; OpenCV homography; bundled-DEM-aware (rare for flat steppe — likely overkill); GDAL warp utilities; custom GPU shader on Jetson |
|
||||
| C5 | **State estimator / sensor fusion** (VO + IMU + sat anchors → fused estimate with covariance) | WGS84 position + 3D velocity + attitude + 6×6 covariance, frame-rate output, honest covariance, source label | EKF (manual), ESKF (manual or via library), MSCKF, factor-graph (GTSAM, iSAM2), particle filter, learned (out-of-scope for safety budget). Supporting: Mahalanobis outlier gates |
|
||||
| C6 | **Tile cache + spatial index** (storage + retrieval of basemap tiles + descriptors, with manifests, freshness, dedup, and write-back) | mmap-friendly storage; ANN over global descriptors; spatial query for geographic prior; manifest schema per AC | Storage: GeoTIFF + COG, MBTiles, custom flat layout. ANN: FAISS (IVF/PQ/HNSW), hnswlib, ScaNN, brute-force (small index). Spatial: R-tree / KD-tree / GeoPandas / SQLite+SpatiaLite. Manifest: SQLite, JSON-per-tile, Parquet sidecar |
|
||||
| C7 | **On-Jetson inference runtime** | INT8/FP16 inference of the chosen VPR + matcher models within latency + memory budget | TensorRT (native), Torch-TensorRT, ONNX Runtime + TRT EP, NVIDIA Triton (probably overkill), pure PyTorch fp16, NVIDIA DeepStream (for video), CUDA-Python custom kernels |
|
||||
| C8 | **MAVLink FC adapter** (per-FC external-positioning emission + spoofing-signal subscription, for ArduPilot AND iNav) | MAVLink frames consumed by ArduPilot Plane and iNav as external position; spoofing signals consumed from each FC | Libraries: `pymavlink` (per-message), MAVSDK (high-level), ArduPilot/iNav SITL for verification. Per-FC choice of message: `GPS_INPUT` vs `ODOMETRY` vs `VISION_POSITION_ESTIMATE` vs `GLOBAL_POSITION_INT` (documented capability per FC must be verified, not assumed) |
|
||||
| C9 | **Datasets + SITL / replay** | Reproducible validation against AC-1/2/3/4/NEW-4/NEW-7/NEW-8 budgets; fixtures for AerialVL S03, AerialExtreMatch, own Mavic flights, Derkachi flight footage | AerialVL (VISTA / NTU), AerialExtreMatch, VPR-Bench, MahalNotchVPR / Mid-Air UAV; SITL: ArduPilot Plane SITL, iNav SITL/HITL, Gazebo, Webots; replay: PX4-Avionics-Replay-style or custom |
|
||||
| C10 | **Pre-flight cache provisioning + sector classification + freshness pipeline** | Tooling (operator-side) to pull tiles from Suite Sat Service for an operational area, classify active-conflict vs stable rear, age-stamp, populate descriptor index | Likely a custom CLI/desktop tool — research existing UAV mission-prep tools (QGC plan files, MAVProxy, ArduPilot Mission Planner equivalents on the operator side) |
|
||||
|
||||
## Perspectives Chosen (≥3 mandatory)
|
||||
|
||||
1. **Implementer / Engineer** — Will the chosen stack actually compile, link, and run on the pinned Jetson within the latency + memory budget? Pitfalls of MAVLink GPS injection on each FC. Sub-pixel registration on UAV-nadir × ortho satellite. Inference-scheduler contention on shared CPU+GPU memory.
|
||||
2. **Practitioner / Field** — What do UAV teams actually report from GPS-denied missions in real war-zone deployments? (Ukraine context if findable; otherwise analogous high-stakes deployments.) Real-world VPR collapse on agricultural cropland / snow / season change. Real-world FDR usefulness for post-mission forensics.
|
||||
3. **Domain expert / Academic** — Recent (2024–2026) VPR + cross-domain matching benchmarks and their relative ranks under cross-season / cross-domain / cross-altitude conditions. Foundation-model-based VPR (AnyLoc, BoQ, MASt3R) — academic claims vs reproducibility. Recent factor-graph vs ESKF comparisons.
|
||||
4. **Contrarian / Devil's advocate** — Why might foundation-model VPR fail on the Jetson budget? Where does cross-domain matching degrade silently? When does ortho-tile write-back amplify bad poses? When does honest covariance turn into "system never trusts itself" (over-cautious failure)?
|
||||
|
||||
## Search Query Variants per Sub-Question
|
||||
|
||||
(Detailed query lists are appended below per sub-question; these will be executed in Step 2 and saved to `01_source_registry.md`. The shape is shown here so the search plan is auditable; the full execution log will populate downstream files.)
|
||||
|
||||
**SQ1** (existing systems / competitors): "GPS-denied UAV navigation 2025", "visual GPS denied fixed wing UAV", "satellite map matching UAV localization 2024 2025", "Ukraine UAV GPS spoofing countermeasures", "ARL ANT Project visual navigation", "vision-based GPS replacement UAV production", "UAV GPS spoofing real-world deployment 2025".
|
||||
|
||||
**SQ2** (canonical pipeline): "visual aerial localization pipeline survey", "UAV satellite map matching architecture", "monocular UAV global localization pipeline 2024 2025".
|
||||
|
||||
**SQ3 / SQ4** (per-component candidates + binding): per-component query templates (5+ variants each) — see Step 2 plan in `01_source_registry.md` once initialised. Each lead library/SDK candidate triggers the mandatory `context7` per-mode capability verification per `research/steps/03_engine-investigation.md`.
|
||||
|
||||
**SQ5** (failure modes): "VPR cropland failure", "DINOv2 Jetson Orin Nano latency", "SuperGlue LightGlue Jetson Orin", "ESKF cross-domain over-confidence", "RANSAC homography low-texture failure UAV", "ortho photo geometric error airframe tilt".
|
||||
|
||||
**SQ6** (ArduPilot vs iNav external positioning): "ArduPilot Plane GPS_INPUT external", "ArduPilot ODOMETRY EKF3 source switching", "iNav external positioning MAVLink GPS_INPUT", "iNav MAVLink GPS substitute", "iNav GPS denied flight 2025", "ArduPilot vs iNav external nav comparison".
|
||||
|
||||
**SQ7** (datasets): "AerialVL dataset", "AerialExtreMatch", "VPR-Bench cross-season aerial", "Mid-Air UAV dataset", "Mavic Mavik UAV public flight dataset", "satellite-aerial cross-view localization benchmark".
|
||||
|
||||
**SQ8** (safety): "MAVLink GPS_RAW_INT spoofing detection", "EKF lane switch ArduPilot", "covariance under-reporting risk EKF", "geo-misalign detection ortho tile".
|
||||
|
||||
## Completeness Audit
|
||||
|
||||
Probes (per `references/comparison-frameworks.md` → Decomposition Completeness Probes — applied here without re-reading the full file; will reconcile during Step 2):
|
||||
|
||||
| Probe | Coverage |
|
||||
|---|---|
|
||||
| Functional decomposition complete? | C1–C10 cover all data flows from camera in to MAVLink out + back. ✓ |
|
||||
| Non-functional dimensions covered? | Latency, memory, accuracy, safety, freshness, security all in Project Constraint Matrix. ✓ |
|
||||
| Failure-mode dimension covered? | SQ5 explicitly. ✓ |
|
||||
| Cost / TCO dimension? | Hardware is pinned (Jetson Orin Nano Super); Service-side cost is out of scope; SW cost = mostly open-source candidates. Will revisit during Phase 3 (tech stack consolidation) if commercial options emerge. ✓ |
|
||||
| Maintenance / community-health dimension? | SQ4 binds it per candidate. ✓ |
|
||||
| Adjacent-domain dimension? | Robot SLAM, AGV warehouse navigation, aerial photogrammetry will be searched as analogues. ✓ |
|
||||
| Validation / dataset coverage? | SQ7 + C9. ✓ |
|
||||
| Integration / boundary coverage? | SQ6 (FC adapters) + C8 + C10 (pre-flight provisioning). ✓ |
|
||||
| Operational/human-factors? | Pre-flight cache provisioning (C10) and operator re-loc hint (AC-3.4) covered. Mission-planning UX is out of scope. ✓ |
|
||||
| Security / threat model? | SQ8. Will deepen in Phase 4 (Security Deep Dive) if invoked. ✓ |
|
||||
|
||||
No major gap detected at decomposition time. If domain-discovery searches in Step 2 surface a missed dimension, a "gap-fill" entry will be appended here.
|
||||
|
||||
## Notes on Output-Class Mode-Verification
|
||||
|
||||
Because this is **Technical-component selection**, every lead library/SDK candidate triggers:
|
||||
- Pinned mode/configuration sentence in `02_fact_cards.md`.
|
||||
- `context7` lookup with the three mandatory queries (mode enumeration; project's exact mode runnable example; disqualifier probe).
|
||||
- MVE block per candidate.
|
||||
- Per-numbered-Restriction and per-numbered-AC binding (`Pass` / `Fail` / `Verify` / `N/A`).
|
||||
- Two modes of one library = two distinct candidates.
|
||||
|
||||
## Step 0.5 — Novelty Sensitivity Assessment
|
||||
|
||||
**Classification: Critical sensitivity.**
|
||||
|
||||
Justification:
|
||||
- Foundation-model VPR is moving fast: DINOv2 (Apr 2023), AnyLoc (Aug 2023), BoQ (CVPR 2024), MASt3R (May 2024), MASt3R-SfM / new VPR-leader candidates 2025; rankings on cross-season aerial benchmarks have shifted multiple times since 2023.
|
||||
- ArduPilot Plane / iNav external-positioning interfaces have moved: ArduPilot EKF3 source-switching parameters and known double-fusion bugs between `GPS_INPUT` and `ODOMETRY` were a moving target through 2024–2025; iNav GPS-denied support has matured separately.
|
||||
- TensorRT / JetPack stacks on Jetson Orin Nano Super have version-dependent INT8 quantisation behaviour and runtime tooling differences worth verifying against current releases.
|
||||
- Public aerial-localization datasets (AerialVL, AerialExtreMatch, etc.) have had multiple revisions and added splits.
|
||||
|
||||
Source-time-window rules for this run:
|
||||
- **Lead-candidate selection / SOTA claims**: prioritise sources from **last 6 months**; allow up to **18 months** if no newer source covers the same claim and the older source is the official authority.
|
||||
- **Established baselines / classical algorithms** (KLT, RANSAC, EKF, ORB, SIFT, GTSAM): no time window — canonical references are fine.
|
||||
- **Library/SDK API behaviour**: must be verified against the **currently shipped version** at the time of search (`context7` mandatory per lead candidate; release notes / changelog cross-checked).
|
||||
- **Cross-validation**: every Critical-sensitivity claim that drives a candidate selection must have **≥2 independent sources** or one official + one runnable MVE; single-source SOTA claims must be downgraded to `Experimental only` at Step 7.5 unless cross-validated.
|
||||
|
||||
## SQ2 Closure — Pipeline-component coverage table (Mode A Phase 2, Step 3 result)
|
||||
|
||||
The C1–C10 decomposition was sanity-checked against five independent surveys/benchmarks (Skoltech aerial-VPR survey, U.Maine cross-view survey, OrthoLoC benchmark, AnyVisLoc benchmark, NUDT 2026 absolute-VL survey — all logged in `01_source_registry.md` as Sources #38–#42). The canonical hierarchical framework `retrieval → matching → pose-estimation` is unanimously confirmed; project's split is **canonical, not novel**. Two augmentations are required.
|
||||
|
||||
| Survey/benchmark canonical stage | Project component | Coverage status | Required action |
|
||||
|---|---|---|---|
|
||||
| Image retrieval (global VPR) | **C2 — VPR** | ✅ covered | None |
|
||||
| Re-ranking (top-N inlier-based) | (implicit, inside C2/C3) | ⚠️ implicit | Promote to explicit sub-stage in `solution_draft01` |
|
||||
| Local image matching (2D-2D, sparse or dense) | **C3 — Cross-domain registration** | ✅ covered | Add Top-N inlier re-rank requirement |
|
||||
| AdHoP-style perspective preconditioning | (not represented) | ❌ missing | Add as optional sub-stage between C3 and C4, gated on Jetson latency budget |
|
||||
| 2D-3D lift via DSM | (not represented; current cache is 2D ortho only) | ❌ architectural decision required | **Decision required from user** — see "Open architectural decisions" below |
|
||||
| Pose estimation (PnP + RANSAC + LM) | **C4 — Pose estimation** | ✅ covered | None |
|
||||
| State estimator / fusion | **C5 — Estimator / fusion** | ✅ covered | Augmented with covariance-honesty contract (already from AC-NEW-4) |
|
||||
| IMU + VIO contract | **C1 (VIO)** + **C6 (Tile cache)** ⁂ | ✅ covered | Add yaw σ ≤ 5°, pitch σ ≤ 5° hard contract (Fact #24) |
|
||||
| Tile cache + scheduler | **C6 (Tile cache + spatial index)** | ✅ covered | Add 20% covisibility runtime invariant (Fact #27) |
|
||||
| On-Jetson runtime | **C7 — On-Jetson inference runtime** | ✅ covered | Pre-screen prunes non-viable candidates (Fact #26) |
|
||||
| Anti-spoof / FC adapter | **C8 — MAVLink FC adapter** | ✅ covered | Already addressed by SQ6 |
|
||||
| Datasets / SITL / replay | **C9 — Datasets + SITL / replay** | ✅ covered | None |
|
||||
| Pre-flight cache provisioning | **C10 — Pre-flight cache + sector classification** | ✅ covered | None |
|
||||
|
||||
⁂ The "IMU integration" concern lives in C1 (VIO) and partially flows from FC IMU; there is no separately numbered IMU component in the original C1–C10 split. SQ2 confirms this was correct — IMU is best owned by C1 (VIO) which already produces the yaw/pitch attitude. The σ ≤ 5° contract belongs on C1's output interface.
|
||||
|
||||
### SQ2 — Architectural decisions (resolved by user, 2026-05-07)
|
||||
|
||||
| # | Decision | Choice | Implication for SQ3+SQ4 |
|
||||
|---|---|---|---|
|
||||
| 1 | DSM dependency on Suite Sat Service tile cache (Fact #23) | **(a) 3-DoF acceptance** — fix attitude from IMU/VIO, ignore DSM; current 2D-ortho cache contract preserved. | C6 (Tile cache) candidate matrix excludes DSM-dependent storage formats. C3 (matcher) candidates evaluated on 2D-2D output (homography) only. Yaw/pitch σ ≤ 5° (Fact #24) is **noted as an empirical requirement on C1's output but NOT bound as a hard interface contract** — emerges as an output of C1 candidate selection in SQ3+SQ4. AC-1.1.1 (≤80 m at 1 km AGL) likely satisfied per DSMAC-class lineage in Fact #17; if AC ever tightens, revisit option (b). |
|
||||
| 2 | AdHoP refinement loop (Fact #22) | **(b) Conditional** — only invoked when initial reprojection error exceeds a threshold. | C3 (matcher) latency budget = base (single-pass) + AdHoP-conditional overhead (worst-case 2× when triggered). Per-frame Jetson MVE must measure both modes. The reprojection-error threshold becomes a SQ3+SQ4 hyperparameter. |
|
||||
| 3 | Top-N re-rank promotion (Fact #25) | **(a) Promote** to an explicit named sub-stage between C2 and C3. | SQ3+SQ4 will hyperparameter-sweep N ∈ {5, 10, 15, 20}; C2 candidates evaluated jointly with re-rank cost. Top-N re-rank by inlier-count is now a hard pipeline component, not implicit. |
|
||||
|
||||
### SQ2 — Component-pruning carried into SQ3+SQ4 (Jetson-pre-screen result)
|
||||
|
||||
Per Fact #26 (RTX-3090-measured runtime → conservative Jetson-Orin-Nano translation):
|
||||
|
||||
- **C2 candidates entering SQ3+SQ4 with mandatory Jetson MVE**: MixVPR, SALAD, SelaVPR, EigenPlaces, NetVLAD.
|
||||
- **C2 candidates entering SQ3+SQ4 conditional on INT8 quantization path**: AnyLoc, BoQ, DINOv2-VLAD.
|
||||
- **C2 candidates pruned outright**: SuperGlue-as-reranker (latency).
|
||||
- **C3 candidates entering SQ3+SQ4 with mandatory Jetson MVE**: LightGlue, XFeat, XFeat*, SP+LightGlue (NGPS template confirmed).
|
||||
- **C3 candidates pruned outright**: RoMa, MASt3R, DKM (dense-matcher latency on Jetson).
|
||||
- **C3 candidates as "AerialExtreMatch reference points" only**: GIM+DKM, GIM+LightGlue (per Source #40 — accuracy benchmark, not for production deployment).
|
||||
|
||||
## Next Step
|
||||
|
||||
SQ1 ✓ → SQ2 ✓ (with three architectural decisions resolved) → **SQ3+SQ4 per component (C1→C10)** → SQ5 interleaved → SQ7 → SQ8 → SQ9 synthesis at engine Step 8.
|
||||
|
||||
Pipeline shape entering SQ3+SQ4: `C1 (VIO) → C2 (VPR) → Top-N re-rank by inlier count → C3 (matcher) → AdHoP-conditional refinement → C4 (PnP+RANSAC+LM) → C5 (estimator) → C8 (FC adapter)` with C6 (cache, 2D ortho) + C7 (Jetson runtime) + C9 (datasets) + C10 (provisioning) cross-cutting.
|
||||
|
||||
First C1 (VIO) candidate batch: VINS-Mono / VINS-Fusion / OpenVINS / OKVIS2 / DROID-SLAM / DPVO / pure-VO baseline (RTAB-Map and ORB-SLAM3 already pruned by Fact #16). Per-mode `context7` capability verification mandatory for every lead library/SDK candidate.
|
||||
@@ -0,0 +1,523 @@
|
||||
# Source Registry
|
||||
|
||||
> Mode A Phase 2 — engine Step 2 (Source Tiering & Exhaustive Web Investigation).
|
||||
> Critical-novelty sensitivity per Step 0.5 in `00_question_decomposition.md`. Time windows applied:
|
||||
> - **Lead-candidate / SOTA claims**: prefer sources within last 6 months; up to 18 months if older is the official authority.
|
||||
> - **Library/SDK API behaviour**: must reflect the currently shipped version at search time (`context7` mandatory per lead candidate).
|
||||
> - **Established baselines** (KLT, RANSAC, EKF, ORB, SIFT, GTSAM): no time window.
|
||||
>
|
||||
> Investigation order saved in `00_question_decomposition.md` → "Next Step": SQ6 → SQ1 → SQ2 → SQ3+SQ4 per component (C1→C10) → SQ5 interleaved → SQ7 → SQ8 → SQ9 synthesis at engine Step 8.
|
||||
|
||||
## Investigation Status
|
||||
|
||||
| Sub-question | Status | Notes |
|
||||
|---|---|---|
|
||||
| SQ6 — ArduPilot vs iNav external positioning | **Saturated for protocol-level architectural decision** (further detail deferred to SQ8 for spoofing-side fields and to design phase for SITL parameter tuning) | Major finding: iNav has no inbound external-positioning MAVLink handler; AC-4.3 wording must be revised. See `02_fact_cards.md` "SQ6 Conclusions". |
|
||||
| SQ1 — Existing GPS-denied UAV systems | **Saturated.** 13 sources logged across academic / open-source / commercial / defense-program / Ukraine-practitioner. Closest peer system: Twist Robotics OSCAR (deployed in Ukraine). Closest open-source pipeline-match: snktshrma/ngps_flight (NGPS, ArduPilot GSoC 2024 — LightGlue+SuperPoint+UKF+VISION_POSITION_ESTIMATE). Closest deployed commercial: Auterion Artemis (Skynode N + Visual Navigation, Ukraine-tested, 1000-mile range). | See `02_fact_cards.md` SQ1 cluster + working summary. |
|
||||
| SQ2 — Canonical pipeline decomposition | **Saturated.** 5 surveys/benchmarks logged (Skoltech aerial VPR, U.Maine cross-view, OrthoLoC 2.5D geodata, AnyVisLoc low-altitude multi-view, NUDT 2026 sciopen survey). All converge on **`retrieval → matching → pose-estimation`** hierarchical framework with VIO/IMU as auxiliary. Two new architectural facts added to C1–C10: (a) **AdHoP-style perspective-refinement loop** between matching and PnP (+63% translation accuracy, method-agnostic), (b) **DSM 2.5D dependency** for full 6-DoF on aerial-to-satellite (must be resolved with the Suite Sat Service or accepted as a 3-DoF degraded mode). Practitioner runtime evidence: AnyLoc on RTX 3090 = 0.63s/descriptor, SuperGlue re-rank = 17–25s; on Jetson Orin Nano these are non-viable for our 400 ms p95 budget — must restrict to lightweight VPR (e.g., MixVPR / SALAD class) + LightGlue/XFeat-class matchers. See `02_fact_cards.md` "SQ2 Conclusions". |
|
||||
| SQ3+SQ4 — Per-component candidates (C1–C10) | Not started | |
|
||||
| SQ5 — Failure modes / deployment lessons | Not started (interleaved with SQ3/SQ4) | |
|
||||
| SQ7 — Datasets, SITL, replay environments | Not started | |
|
||||
| SQ8 — Safety considerations (AC-NEW-4 / AC-NEW-7) | Not started | Carries the AP_GPS spoofing-signal probe deferred from SQ6. |
|
||||
| SQ9 — End-to-end synthesis | Step 8 of engine (deferred) | |
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
### Source #1
|
||||
- **Title**: Non-GPS Navigation — Plane documentation
|
||||
- **Link**: https://ardupilot.org/plane/docs/common-non-gps-navigation-landing-page.html
|
||||
- **Tier**: L1
|
||||
- **Publication Date**: live docs (current ArduPilot stable, accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: ArduPilot 4.7+ (persistent origin storage); applies to current Plane stable
|
||||
- **Target Audience**: ArduPilot Plane operators / developers
|
||||
- **Research Boundary Match**: Full match (fixed-wing, ArduPilot Plane is in scope)
|
||||
- **Summary**: Lists supported non-GPS navigation systems for Plane. Notes that boards <1MB flash still support `GPS_INPUT` even when they cannot run other non-GPS messages. Notes that Plane (non-VTOL) is generally not applicable for low-altitude non-GPS — but `GPS_INPUT` as an external GPS replacement is not constrained by that note.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #2
|
||||
- **Title**: GPS / Non-GPS Transitions — Plane documentation
|
||||
- **Link**: https://ardupilot.org/plane/docs/common-non-gps-to-gps.html
|
||||
- **Tier**: L1
|
||||
- **Publication Date**: live docs (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: EKF3 (default since AP 4.0+)
|
||||
- **Target Audience**: ArduPilot operators using mixed GPS / non-GPS sources
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Documents the EKF3 source-set mechanism (`EK3_SRC1..3_POSXY/VELXY/POSZ/VELZ/YAW`), three source sets, RC aux switch (option 90 "EKF Pos Source"), `MAV_CMD_SET_EKF_SOURCE_SET`, Lua-script driven switching. Explicitly named messages for non-GPS path: ExternalNav (option 6). GPS_INPUT is treated as a GPS source (set 1).
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #3
|
||||
- **Title**: EKF Source Selection and Switching — Plane documentation
|
||||
- **Link**: https://ardupilot.org/plane/docs/common-ekf-sources.html
|
||||
- **Tier**: L1
|
||||
- **Publication Date**: live docs (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: EKF3 stable
|
||||
- **Target Audience**: ArduPilot operators / developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Authoritative parameter reference for `EK3_SRCx_*` (POSXY/VELXY/POSZ/VELZ/YAW). Important caveat: "Ground stations or companion computers may set the source by sending a `MAV_CMD_SET_EKF_SOURCE_SET` mavlink command **but no GCSs are currently known to implement this**." Source-set switching from companion is supported by AP, not by stock GCS UI. Mentions ExternalNAV/OpticalFlow transition options via `EK3_SRC_OPTIONS` bit 1.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #4
|
||||
- **Title**: ArduPilot AP_GPS_MAV.cpp (master)
|
||||
- **Link**: https://raw.githubusercontent.com/ArduPilot/ardupilot/master/libraries/AP_GPS/AP_GPS_MAV.cpp
|
||||
- **Tier**: L1 (source code)
|
||||
- **Publication Date**: master HEAD (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: master branch
|
||||
- **Target Audience**: ArduPilot developers, integrators of external GPS via MAVLink
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Authoritative implementation of `MAVLINK_MSG_ID_GPS_INPUT` ingestion into AP_GPS state. Decodes lat/lon/alt, hdop/vdop, velocity (vn/ve/vd), speed/horizontal/vertical accuracy, yaw. Honors `gps_id` (multi-GPS instance), `ignore_flags` bitmask (ALT, HDOP, VDOP, VEL_HORIZ, VEL_VERT, SPEED_ACCURACY, HORIZONTAL_ACCURACY, VERTICAL_ACCURACY). Requires `fix_type ≥ 3` and `time_week > 0` for jitter-corrected timestamping. Yaw uses `0` as "not provided" sentinel. Only `GPS_INPUT` is handled by this driver — `VISION_POSITION_ESTIMATE` / `ODOMETRY` go via the external-nav driver, not AP_GPS_MAV.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #5
|
||||
- **Title**: ArduPilot PR #28750 — AP_NavEKF3: added two more EK3_OPTION bits (GPS-denied testing)
|
||||
- **Link**: https://github.com/ArduPilot/ardupilot/pull/28750
|
||||
- **Tier**: L2 (development PR, ArduPilot core team)
|
||||
- **Publication Date**: 2024 (accessed via search 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: master / pending stable branch propagation
|
||||
- **Target Audience**: ArduPilot developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Adds new `EK3_OPTION` bits to allow easier GPS-denied testing of EKF3, including an aux-switch / MAVLink command path to disable GPS use. Confirms ongoing 2024-2025 work on GPS-denied robustness.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #6
|
||||
- **Title**: ArduPilot Issue #15859 — EKF3: improve source switching (GPS<->NonGPS)
|
||||
- **Link**: https://github.com/ArduPilot/ardupilot/issues/15859
|
||||
- **Tier**: L4 (issue tracker — open enhancement list)
|
||||
- **Publication Date**: ongoing (long-running issue, accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid (still open per dev docs reference)
|
||||
- **Target Audience**: ArduPilot developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Authoritative list of planned improvements for source-switching. Linked from the L1 GPS-Non-GPS Transitions page. Indicates current source switching has known rough edges acknowledged by the core team.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #7
|
||||
- **Title**: ArduPilot Issue #27193 — EK3 Source Switching wrong frame for GUIDED commands SOLVED
|
||||
- **Link**: https://github.com/ArduPilot/ardupilot/issues/27193
|
||||
- **Tier**: L4 (issue tracker, resolved)
|
||||
- **Publication Date**: 2024 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Reference only (resolved as user-config)
|
||||
- **Target Audience**: ArduPilot operators using GPS↔Vision source switching
|
||||
- **Research Boundary Match**: Partial overlap (Copter context but the bug was in shared SET_POSITION_TARGET_GLOBAL_INT path)
|
||||
- **Summary**: Documented frame-interpretation issue when companion switches source set 1 (GPS) → set 3 (VISION_POSITION_ESTIMATES) and back. Resolved as configuration not code, but illustrates the kind of edge case to validate in SITL for AC-NEW-2 promotion.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #8
|
||||
- **Title**: ArduPilot Issue #23485 — AP_NavEKF3: support fusing only External Nav Velocities (without position)
|
||||
- **Link**: https://github.com/ArduPilot/ardupilot/issues/23485
|
||||
- **Tier**: L4 (open enhancement)
|
||||
- **Publication Date**: ongoing (open as of accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: ArduPilot developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Confirms current limitation: ODOMETRY without position causes position-estimate timeout / failsafe. Implies the project's `visual_propagated` path (VO without satellite anchor) cannot be expressed as ODOMETRY-velocity-only on current AP — must be sent as full GPS_INPUT with widened covariance.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #9
|
||||
- **Title**: iNavFlight/inav — telemetry/mavlink.c (master, processMAVLinkIncomingTelemetry)
|
||||
- **Link**: https://github.com/iNavFlight/inav/blob/master/src/main/telemetry/mavlink.c
|
||||
- **Tier**: L1 (source code, authoritative)
|
||||
- **Publication Date**: master HEAD (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav master (post-9.0)
|
||||
- **Target Audience**: iNav developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Authoritative inbound MAVLink switch (lines ~1334–1390). Handles only: HEARTBEAT, PARAM_REQUEST_LIST (stub), MISSION_CLEAR_ALL, MISSION_COUNT, MISSION_ITEM, MISSION_REQUEST_LIST, MISSION_REQUEST, COMMAND_INT (only `MAV_CMD_DO_REPOSITION`), RC_CHANNELS_OVERRIDE, ADSB_VEHICLE, RADIO_STATUS. **No `GPS_INPUT`, no `VISION_POSITION_ESTIMATE`, no `ODOMETRY`, no `GLOBAL_POSITION_INT`, no `GPS_RAW_INT`** are accepted as inputs. Wiki page (Source #10) confirms.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #10
|
||||
- **Title**: iNav Wiki — MAVLink (frogmane edited 2025-12-11)
|
||||
- **Link**: https://github.com/iNavFlight/inav/wiki/Mavlink
|
||||
- **Tier**: L1 (project wiki)
|
||||
- **Publication Date**: 2025-12-11
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav 8.0 / 9.0 era
|
||||
- **Target Audience**: iNav users / integrators
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Authoritative inbound/outbound MAVLink message lists. "Limited command support: Commands that are not implemented are ignored." Explicitly enumerates the supported incoming list (matches Source #9). Confirms iNav MAVLink is "intended primarily for simple telemetry and operation" and "not 100% compatible".
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #11
|
||||
- **Title**: iNav Wiki — GPS and Compass setup
|
||||
- **Link**: https://github.com/iNavFlight/inav/wiki/GPS-and-Compass-setup
|
||||
- **Tier**: L1
|
||||
- **Publication Date**: live wiki (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav 7.0+ (UBX-only); 9.0 requires UBX protocol ≥15.00
|
||||
- **Target Audience**: iNav operators
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: From iNav 7.0 NMEA was removed; only UBX is supported. Recommends u-blox M8/M9/M10 with protocol ≥15.00. Sets up the constraint for any UBX-emulation path the companion would take.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #12
|
||||
- **Title**: iNavFlight/inav docs/development/msp/README.md (MSP message reference)
|
||||
- **Link**: https://github.com/iNavFlight/inav/blob/master/docs/development/msp/README.md
|
||||
- **Tier**: L1 (project docs)
|
||||
- **Publication Date**: live (master, accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav master
|
||||
- **Target Audience**: iNav developers / integrators
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Authoritative spec for `MSP_SET_RAW_GPS (201)` and `MSP2_SENSOR_GPS (7939)`. `MSP_SET_RAW_GPS` is 14-byte, lossy (no covariance, no per-axis velocity, altitude in meters with cm internal mismatch — bug fixed in 5.0.0 per issue #8336). `MSP2_SENSOR_GPS` is the newer plugin-style message with `hPosAccuracy`/`vPosAccuracy`/`hVelAccuracy` (mm and cm/s), `hdop`, NED velocity components, `trueYaw`, GPS week + time-of-week, fix type, satellite count. Requires `USE_GPS_PROTO_MSP` build flag and routes through `mspGPSReceiveNewData()` (the GPS_PROVIDER_MSP driver path).
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #13
|
||||
- **Title**: iNavFlight/inav src/main/io/gps.c + src/main/target/common.h (master)
|
||||
- **Link**: https://github.com/iNavFlight/inav/blob/master/src/main/target/common.h
|
||||
- **Tier**: L1 (source code)
|
||||
- **Publication Date**: master (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: master
|
||||
- **Target Audience**: iNav developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: `USE_GPS_PROTO_MSP` is enabled by default in the common target configuration; on default builds the MSP GPS provider (`GPS_PROVIDER_MSP`) is registered with `gpsRestartMSP` / `gpsHandleMSP`. Confirms the MSP2_SENSOR_GPS path is reachable on stock iNav firmware without custom builds.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #14
|
||||
- **Title**: iNav Issue #10141 — dual GPS support
|
||||
- **Link**: https://github.com/iNavFlight/inav/issues/10141
|
||||
- **Tier**: L4 (open feature request)
|
||||
- **Publication Date**: ongoing (open as of accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: iNav users
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Confirms iNav does **not** support dual-GPS / primary-secondary failover. Open enhancement; no implementation in 8.0 / 9.0. Architectural implication: companion must be the sole GPS source for iNav (not a backup to a real GPS connected directly to FC).
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #15
|
||||
- **Title**: iNav docs/GPS_fix_estimation.md (master)
|
||||
- **Link**: https://github.com/iNavFlight/inav/blob/master/docs/GPS_fix_estimation.md
|
||||
- **Tier**: L1
|
||||
- **Publication Date**: live (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav 8.0+
|
||||
- **Target Audience**: iNav fixed-wing operators
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: iNav's internal dead-reckoning ("GPS fix estimation") for fixed-wing. Uses gyro/accel/baro/(mag/pitot). RTH-only intent. **Explicitly states: "Not a solution for GPS spoofing (GPS output is not validated in INAV)"** — iNav has no internal anti-spoofing, so anti-spoofing is fully the companion's responsibility. Two settings: `inav_allow_gps_fix_estimation` (RTH-with-no-GPS) and `inav_allow_dead_reckoning` (short-outage tolerance) — both default OFF. `failsafe_gps_fix_estimation_delay` controls mission-vs-RTH tradeoff (default 7 s).
|
||||
- **Related Sub-question**: SQ6 (dead-reckoning fallback) + SQ8 (anti-spoofing implication)
|
||||
|
||||
### Source #16
|
||||
- **Title**: iNav docs/Settings.md (master)
|
||||
- **Link**: https://github.com/iNavFlight/inav/blob/master/docs/Settings.md
|
||||
- **Tier**: L1
|
||||
- **Publication Date**: master (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav master
|
||||
- **Target Audience**: iNav operators
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Authoritative parameter list. Confirms `inav_allow_dead_reckoning` (line 2081, default OFF) ≠ `inav_allow_gps_fix_estimation` (line 2091, default OFF). The two settings address different scenarios. `failsafe_gps_fix_estimation_delay` (line 1041, default 7 s) governs mission-abort timing.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #17
|
||||
- **Title**: iNav Issue #10588 — Weird behaviour in DeadReckoning mode while GPS outage is not constant
|
||||
- **Link**: https://github.com/iNavFlight/inav/issues/10588
|
||||
- **Tier**: L4 (open issue, 2025)
|
||||
- **Publication Date**: 2025
|
||||
- **Timeliness Status**: Currently valid (open)
|
||||
- **Target Audience**: iNav operators
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Documented stability bug: intermittent GPS outages cause porpoising and motor bursts in dead-reckoning. Cited recommendation: "GPS should be rejected if providing erroneous coordinates rather than no fix." Risk for AC-NEW-8 (visual blackout + spoofed GPS) on iNav: do NOT rely on iNav's dead-reckoning for the spoof-active failsafe path; companion must actively suppress its own MSP feed and accept that iNav may misbehave during the gap. Better: continue feeding companion-IMU-propagated position with growing covariance via MSP2_SENSOR_GPS so iNav never enters its dead-reckoning state.
|
||||
- **Related Sub-question**: SQ6 + AC-NEW-8 design implication
|
||||
|
||||
### Source #18
|
||||
- **Title**: iNav Release 8.0.0 (highlights, Dec 2024)
|
||||
- **Link**: https://github.com/iNavFlight/inav/releases/tag/8.0.0
|
||||
- **Tier**: L1 (project release notes)
|
||||
- **Publication Date**: late 2024 / early 2025
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav 8.0
|
||||
- **Target Audience**: iNav users
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Introduces fixed-wing GPS fix estimation (dead reckoning RTH-only) — the milestone for #8347. No new external-positioning inbound MAVLink in 8.0. Confirms iNav's 2024–2025 trajectory has not added a `GPS_INPUT`-equivalent inbound interface.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #19
|
||||
- **Title**: iNav Release 9.0.0 / 9.0.1 + 9.0.0 Release Notes wiki
|
||||
- **Link**: https://github.com/iNavFlight/inav/wiki/9.0.0-Release-Notes
|
||||
- **Tier**: L1
|
||||
- **Publication Date**: 2025-2026
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: iNav 9.0.x
|
||||
- **Target Audience**: iNav users
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: New in 9.0: pitot APA/TPA, position estimator improvements, MSP_REBOOT DFU, GCS NAV via `COMMAND_INT` `MAV_CMD_DO_REPOSITION`. **No** new external-positioning inbound MAVLink. UBX <15.00 dropped. Confirms iNav 9.x continues the same external-positioning architecture as 8.x.
|
||||
- **Related Sub-question**: SQ6
|
||||
|
||||
### Source #20
|
||||
- **Title**: MAVLink common message set — GPS_RAW_INT (24)
|
||||
- **Link**: https://mavlink.io/en/messages/common.html
|
||||
- **Tier**: L1 (MAVLink spec, live)
|
||||
- **Publication Date**: live (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: MAVLink common, current
|
||||
- **Target Audience**: MAVLink integrators
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Current published `GPS_RAW_INT` extension fields: `alt_ellipsoid`, `h_acc` (mm), `v_acc` (mm), `vel_acc` (mm/s), `hdg_acc` (degE5), `yaw` (cdeg). **No spoofing/jamming/integrity bitfield is present in `GPS_RAW_INT` at the time of access**, despite PR #2110 having been merged for spoofing/integrity reporting. Spoofing/integrity may live in a separate message (`GPS_INTEGRITY` or similar — to be verified in SQ8). For now, spoof-detection signals available to companion from FC are limited at the message-shape level; FC-side textual signals (`STATUSTEXT`) and `NAMED_VALUE_INT` are the documented practical path.
|
||||
- **Related Sub-question**: SQ6 + SQ8
|
||||
|
||||
### Source #21
|
||||
- **Title**: MAVLink PR #2110 — gps: add status and integrity information
|
||||
- **Link**: https://github.com/mavlink/mavlink/pull/2110
|
||||
- **Tier**: L2 (protocol PR with cross-project sign-off)
|
||||
- **Publication Date**: merged (accessed via search 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: MAVLink common
|
||||
- **Target Audience**: MAVLink integrators across PX4 / ArduPilot / QGC / Mission Planner
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Adds GNSS status / integrity reporting (jamming/spoofing/error) at the protocol level. Cross-project sign-off across PX4, ArduPilot, QGC, Mission Planner. Field-level breakdown to be cross-checked in SQ8 against the dialect XML — current `common.html` does not show those fields inside `GPS_RAW_INT` itself, suggesting they live in a sibling message (likely `GPS_INTEGRITY` or `GPS_STATUS_EXT`).
|
||||
- **Related Sub-question**: SQ6 → defer to SQ8 for the precise message name and field set ArduPilot uses to expose spoofing.
|
||||
|
||||
### Source #22
|
||||
- **Title**: AirDroper — GNSS Spoofing Filter (companion device, MAVLink2 NAMED_VALUE_INT pattern)
|
||||
- **Link**: https://gps.airdroper.org/
|
||||
- **Tier**: L3 (vendor product page; design pattern reference, not protocol authority)
|
||||
- **Publication Date**: live (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: ArduPilot integrators considering anti-spoofing
|
||||
- **Research Boundary Match**: Reference only (vendor's specific algorithm not relevant; the integration pattern is)
|
||||
- **Summary**: Establishes a precedent that "companion-runs-spoofing-detection → publishes confidence to GCS as MAVLink2 `NAMED_VALUE_INT`, logged to dataflash" is a real-world integration pattern with ArduPilot, not novel to this project. Useful for SQ8.
|
||||
- **Related Sub-question**: SQ8 (referenced from SQ6)
|
||||
|
||||
### Source #23
|
||||
- **Title**: ArduPilot PR #24135 — Add option to make EKF3 more robust to bad IMU and lagged GPS data
|
||||
- **Link**: https://github.com/ArduPilot/ardupilot/pull/24135
|
||||
- **Tier**: L2 (development PR)
|
||||
- **Publication Date**: 2023-2024 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: master / propagated to stable
|
||||
- **Target Audience**: ArduPilot developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Introduces `EK3_GLITCH_RADIUS` parameter — soft outlier rejection: instead of dropping a GPS measurement that fails innovation gating, the EKF inflates innovation variance to the minimum that just passes, effectively de-weighting the measurement. Implication for AC-NEW-4 (false-position safety): the project's covariance honesty contract on `GPS_INPUT.horiz_accuracy` is the ONLY way for AP's EKF to detect and de-weight a bad estimate; under-reporting collapses this safety net.
|
||||
- **Related Sub-question**: SQ6 + AC-NEW-4 design implication
|
||||
|
||||
### Source #24
|
||||
- **Title**: ArduPilot AP_NavEKF3 — VehicleStatus.cpp + AP_NavEKF3.cpp (master)
|
||||
- **Link**: https://github.com/ArduPilot/ardupilot/blob/master/libraries/AP_NavEKF3/AP_NavEKF3_VehicleStatus.cpp ; https://github.com/ArduPilot/ardupilot/blob/master/libraries/AP_NavEKF3/AP_NavEKF3.cpp
|
||||
- **Tier**: L1 (source code)
|
||||
- **Publication Date**: master HEAD (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: master
|
||||
- **Target Audience**: ArduPilot EKF3 developers
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: EKF3 quality control: (a) ground-stationary GPS drift check ≤ 3 m (gated by `_gpsCheckScaler`); (b) innovation gating per `POS_I_GATE` / `VEL_I_GATE`; (c) soft de-weighting via `EK3_GLITCH_RADIUS` (Source #23). Confirms AP's covariance-driven quality path actually exists; companion-supplied `horiz_accuracy` flows into this chain.
|
||||
- **Related Sub-question**: SQ6 (full file analysis deferred to design phase)
|
||||
|
||||
---
|
||||
|
||||
## SQ1 — Existing / competitor GPS-denied UAV navigation systems
|
||||
|
||||
### Source #25
|
||||
- **Title**: Twist Robotics develops OSCAR — a GPS-independent visual navigation system for drones resistant to electronic warfare equipment
|
||||
- **Link**: https://www.pravda.com.ua/eng/news/2026/01/28/8018266/
|
||||
- **Tier**: L2 (national newspaper of record reporting on a Technology Forces of Ukraine release; primary press is the Technology Forces of Ukraine FB post)
|
||||
- **Publication Date**: 2026-01-28 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid (within 6-month critical-novelty window)
|
||||
- **Target Audience**: Ukraine-deployment practitioners; UAV companion-system designers
|
||||
- **Research Boundary Match**: **Full match** — Ukrainian fixed-wing-class UAV, GPS-denied, vision-based, deployed in active conflict
|
||||
- **Summary**: Twist Robotics (UA) deployed OSCAR ("Optical System of Coordinates with Automatic Relocalisation") — camera + landmark-matching + map → autopilot ingests as a "reliable GPS signal". Vendor claims: 20 m accuracy without cumulative error, day/night/fog operation, 500,000 km logged across 25,000 combat missions over 24 months development, AI-augmented + Obrii proprietary simulator for training. Note: hardware photo shows active cooling on the module — implies non-trivial compute (probably Jetson-class). **No public independent benchmark.** Closest deployed peer system to this project.
|
||||
- **Related Sub-question**: SQ1 (closest peer); also informs SQ8 (anti-spoofing claims), SQ9 (synthesis)
|
||||
|
||||
### Source #26
|
||||
- **Title**: Ukraine Gives Drones Vision-Based Navigation to Push Past Heavy Jamming — The Defense Post
|
||||
- **Link**: https://thedefensepost.com/2026/01/29/ukraine-drones-vision-navigation/
|
||||
- **Tier**: L2 (defense-trade publication; corroborates Source #25 with a second-party byline)
|
||||
- **Publication Date**: 2026-01-29 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: Defense-policy / procurement readership
|
||||
- **Research Boundary Match**: Full match
|
||||
- **Summary**: Confirms OSCAR is operational, terrain-imagery-against-mapped-landmarks pattern, autopilot-ingestion. Adds "live imagery" framing. No new technical detail beyond Source #25.
|
||||
- **Related Sub-question**: SQ1
|
||||
|
||||
### Source #27
|
||||
- **Title**: Ukraine's Ruta Missile Drone Will Get an EW-Immune Navigation System — Defense Express
|
||||
- **Link**: https://en.defence-ua.com/weapon_and_tech/ukraines_ruta_missile_drone_will_get_an_ew_immune_navigation_system-14541.html
|
||||
- **Tier**: L2 (defense-trade publication, Ukraine-domestic)
|
||||
- **Publication Date**: 2025-05-17 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid (within 18-month authority window)
|
||||
- **Target Audience**: Defense-procurement / industry analysts
|
||||
- **Research Boundary Match**: Partial — operational profile (cruise-missile-class, terminal guidance) differs from our 8-h fixed-wing surveillance/strike profile; technique class is closely related (DSMAC pattern)
|
||||
- **Summary**: Destinus Ruta (Ukrainian-Swiss origin; ~300 km strike range, miniature cruise missile) will integrate a navigation system from UAV Navigation (Spanish, Grupo Oesía). Defense Express infers DSMAC-style operating principle: "takes images of surface mid-flight, identifies location through comparison with reference". Vendor announcement notes validation in Ukrainian combat conditions including GNSS-denied / jamming / spoofing. Establishes that the cruise-missile-tier vision-nav pattern is now being miniaturised for ~300 km strike drones.
|
||||
- **Related Sub-question**: SQ1 (commercial/military landscape)
|
||||
|
||||
### Source #28
|
||||
- **Title**: Kilometer-Scale GNSS-Denied UAV Navigation via Heightmap Gradients: A Winning System from the SPRIN-D Challenge
|
||||
- **Link**: https://arxiv.org/abs/2510.01348
|
||||
- **Tier**: L1 (peer-style preprint, full system description, real flight data, competition results)
|
||||
- **Publication Date**: October 2025 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: arXiv v1 (2510.01348v1)
|
||||
- **Target Audience**: GNSS-denied UAV system designers (academic + practitioner)
|
||||
- **Research Boundary Match**: **Partial — different regime.** Multirotor (≤25 kg), <25 m AGL, LiDAR-equipped, no satellite-tile basemap; 9 km waypoint mission. Our project is fixed-wing, ~1 km AGL, no LiDAR, monocular + sat-tile basemap. **Architectural pattern transfers; specific algorithm does NOT** (heightmap gradients require LiDAR).
|
||||
- **Summary**: CTU Prague team won SPRIN-D Funke Fully Autonomous Flight Challenge with: VIO (OpenVINS) + LiDAR-derived local heightmap + gradient template matching against open-data DEM + clustered K-means particle filter, all on Intel NUC i7 16 GB CPU-only (no GPU). Achieved RMSE <11 m over kilometer-scale flights vs ≤53 m for raw odometry. Critical observations explicitly stated:
|
||||
- **RTAB-Map and ORB-SLAM3 both fail** beyond 1 km / above 2 m/s flight (compute/memory) and ORB-SLAM3 loses tracking in textureless areas — directly applicable to our 17 m/s cruise over agricultural steppe.
|
||||
- **"Some teams used RGB satellite image-based matching, but this has proved to be highly unreliable at such low altitudes."** This is a low-altitude (<25 m AGL) finding; our 1 km AGL operates in the high-altitude regime where the same paper notes RGB sat-matching "works reasonably well" (refs [5][6]).
|
||||
- Lesson: "ability to recover from periods of high uncertainty and re-localize is more critical than maintaining consistently low instantaneous RMSE." Direct architectural input for AC-NEW-2 / AC-NEW-8.
|
||||
- Lesson: IMU-from-airframe vibration isolation is mission-critical for VIO usability.
|
||||
- Lesson: magnetometer is unreliable near steel-reinforced structures; sensor-fusion is essential for heading robustness.
|
||||
- **Related Sub-question**: SQ1 + SQ5 (failure modes for VIO/SLAM at speed) + SQ2 (canonical pipeline)
|
||||
|
||||
### Source #29
|
||||
- **Title**: Hierarchical Image Matching for UAV Absolute Visual Localization via Semantic and Structural Constraints
|
||||
- **Link**: https://arxiv.org/abs/2506.09748 (PDF: https://arxiv.org/pdf/2506.09748)
|
||||
- **Tier**: L1 (peer-submitted preprint, IEEE-bound, with public CS-UAV dataset)
|
||||
- **Publication Date**: June 2025 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid (within 6-month critical-novelty window for SOTA claims)
|
||||
- **Version Info**: arXiv v1 (2506.09748v1)
|
||||
- **Target Audience**: Academic SOTA researchers + UAV-localization implementers
|
||||
- **Research Boundary Match**: **Full match** — exact same problem (UAV absolute visual localization in GNSS-denied conditions, downward-facing camera, satellite reference)
|
||||
- **Summary**: 2025 SOTA pipeline: (1) image retrieval module (off-the-shelf, optimal-transport feature aggregation), (2) Semantic-Aware and Structure-Constrained Matching Module using **DINOv2** features + 4D correlation tensor + SoftMNN + 4D conv, (3) lightweight fine-grained module for pixel-level. Constructs UAV absolute visual-loc pipeline **without VIO/relative-loc dependence** (retrieval-and-matching only). Evaluation on AerialVL + their own CS-UAV. **Direct relevance**: this is a candidate template for our C2 (VPR) + C3 (cross-domain registration) components, but DINOv2 is a heavyweight foundation model — must be benchmarked under our 25 W / 8 GB Jetson Orin Nano envelope before selection (handed off to SQ3/SQ4 + SQ5 for that component).
|
||||
- **Related Sub-question**: SQ1 (academic SOTA), SQ3+SQ4 (C2/C3 candidates), SQ5 (Jetson-on-Foundation-Model failure mode)
|
||||
|
||||
### Source #30
|
||||
- **Title**: Raptor — GPS-Denied UAV Navigation & Coordinate Extraction (Vantor product page; Guide / Sync / Ace suite)
|
||||
- **Link**: https://www.vantor.com/product/mission-solutions/raptor/
|
||||
- **Tier**: L2 (vendor product spec; primary for the product itself, not for independent benchmark numbers)
|
||||
- **Publication Date**: live (accessed 2026-05-07; references Mar 2026 + Dec 2025 + Sep 2025 partner blog posts indicating active product line)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: Defense / commercial / industrial UAV integrators
|
||||
- **Research Boundary Match**: **Full match** — vision-based aerial position software using existing camera + 3D terrain data, deployable on commodity hardware
|
||||
- **Summary**: Vantor Raptor product family: **Guide** (on-drone vision-based positioning, demonstrated <7 m absolute accuracy in all dimensions, day/night/low-altitude, runs on commodity HW); **Sync** (georegisters live drone video against 3D terrain in real time, <3 m coordinate extraction); **Ace** (laptop-side coordinate extraction at <3 m). Backbone: Vantor's "100 million-plus sq km of highly accurate 3D terrain data, regularly updated" (Vivid Terrain, 3 m accuracy). Inertial Labs partnership (VINS-integrated Raptor Guide). Use cases include joint multi-domain ops, large-scale autonomous delivery, search-and-rescue. **This is the closest production-grade commercial peer to the project's architecture (sat-basemap-as-service + on-drone vision).**
|
||||
- **Related Sub-question**: SQ1 (commercial), SQ3+SQ4 (commercial alternatives to building C2/C3 ourselves), SQ8 (basemap as a service vs offline cache)
|
||||
|
||||
### Source #31
|
||||
- **Title**: Auterion successfully completes Artemis program to deliver long-range deep strike drone (press release)
|
||||
- **Link**: https://auterion.com/auterion-successfully-completes-artemis-program-to-deliver-long-range-deep-strike-drone/
|
||||
- **Tier**: L1 (official vendor press release)
|
||||
- **Publication Date**: 2025-10-15 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: Defense-procurement; UAV-integration architects
|
||||
- **Research Boundary Match**: **Full match** — fixed-wing-class one-way attack drone with Ukraine-validated GPS-denied navigation; the system architecture is directly comparable
|
||||
- **Summary**: Auterion Artemis (DIU project, completed Oct 2025) = Shahed-style design developed in Ukraine; up to 1,000-mile range; up to 40 kg warhead; runs on Auterion Skynode N mission computer + Auterion Visual Navigation system + built-in terminal guidance. Government evaluators signed off after operational flight tests in Ukraine including ground launch, GPS and GPS-denied navigation, long-range transit, and terminal engagement. **Establishes that the integration pattern (companion-class autopilot + visual navigation + terminal guidance) is shipping at production scale to a US defense customer.** Open architecture, manufacturing in US/UA/DE.
|
||||
- **Related Sub-question**: SQ1
|
||||
|
||||
### Source #32
|
||||
- **Title**: Bring AI and computer vision to small autonomous systems — Auterion Skynode S product page
|
||||
- **Link**: https://auterion.com/product/skynode-s
|
||||
- **Tier**: L2 (vendor product spec)
|
||||
- **Publication Date**: live (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: Small-UAS integrators
|
||||
- **Research Boundary Match**: Full match (companion-class autopilot with NPU)
|
||||
- **Summary**: Auterion Skynode S = compact mission computer with **dedicated Neural Processing Unit** for AI / computer-vision applications on small UAS systems. Architecturally the same niche our Jetson Orin Nano Super sits in (companion compute + autopilot integration), but with Auterion's PX4 fork pre-integrated. Hardware/runtime envelope is comparable; the product establishes that this is a product category, not a one-off integration.
|
||||
- **Related Sub-question**: SQ1, SQ7 (alternate companion HW for adjacent context)
|
||||
|
||||
### Source #33
|
||||
- **Title**: snktshrma/ngps_flight — Next-Generation Positioning System for ArduPilot (GSoC 2024)
|
||||
- **Link**: https://github.com/snktshrma/ngps_flight (sibling: https://github.com/snktshrma/ap_nongps)
|
||||
- **Tier**: L1 (open-source code repository, published GSoC project under ArduPilot organisation)
|
||||
- **Publication Date**: GSoC 2024 timeframe (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Version Info**: GSoC 2024 prototype (research-grade, not production firmware)
|
||||
- **Target Audience**: ArduPilot integrators building visual-positioning companion stacks
|
||||
- **Research Boundary Match**: **Full match — closest open-source peer to our exact pipeline.** ArduPilot, downward-facing camera, satellite-image reference, deep-learning matching, fused with VIO, fed back to autopilot.
|
||||
- **Summary**: NGPS = ROS 2 + ArduPilot pipeline composed of three packages: **`ap_ngps_ros2`** (visual geo-localization at 1–2 Hz by matching live camera frames to georeferenced satellite imagery using **LightGlue + SuperPoint**); **`ap_ukf`** (Unscented Kalman Filter fusing NGPS absolute positions with VIO estimates); **`ap_vips`** (VIO providing relative pose). Output is fused odometry to ArduPilot's EKF via `VISION_POSITION_ESTIMATE` (per the related issue #23471 framing). **This is the architectural template** the project should explicitly compare against — same component split as our C1+C2+C3+C5+C8 stack.
|
||||
- Caveats: (a) GSoC prototype, not production-hardened; (b) uses `VISION_POSITION_ESTIMATE` which on AP requires EKF source set 2/3 with EK3_SRC*_POSXY=Vision; our SQ6 conclusion picked `GPS_INPUT` as primary AP path because it carries `horiz_accuracy` directly and supports source-set switching via `MAV_CMD_SET_EKF_SOURCE_SET` — must compare the trade-off in design phase; (c) no documented spoofing-defence integration; (d) no documented covariance-honesty contract.
|
||||
- **Related Sub-question**: SQ1 (closest open-source peer), SQ2 (canonical-pipeline confirmation), SQ3+SQ4 (architectural template for component selection), SQ6 (alternate AP transport: `VISION_POSITION_ESTIMATE` vs `GPS_INPUT`)
|
||||
|
||||
### Source #34
|
||||
- **Title**: AerialExtreMatch — A Benchmark for Extreme-View Image Matching and Localization (project page + GitHub + Hugging Face dataset)
|
||||
- **Link**: https://xecades.github.io/AerialExtreMatch/ ; https://github.com/Xecades/AerialExtreMatch ; https://huggingface.co/datasets/Xecades/AerialExtreMatch-Localization
|
||||
- **Tier**: L1 (peer-reviewed benchmark with public dataset, code, model checkpoints; OpenReview submission)
|
||||
- **Publication Date**: 2025 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: Academic + practitioner image-matching evaluators
|
||||
- **Research Boundary Match**: **Full match** for cross-source UAV-satellite image matching evaluation
|
||||
- **Summary**: 2025 benchmark with: 1.5 M synthetic train pairs (RGB+depth, diverse UAV/satellite viewpoints); ~30,000 evaluation pairs in 32 difficulty levels stratified by overlap (4 bins: <20/20-40/40-60/>60%), pitch difference (4 bins: 50–55, 55–60, 60–65, 65–70°), and scale (2 bins: 1-2×, >2×); a real-world UAV-localization split captured with DJI M300 RTK + H20T against UAV-derived orthomosaic/DSM AND lower-quality satellite maps. Evaluates 16 representative detector-based + detector-free image matching methods. **This is the academic benchmark our C2+C3 candidate selection must publish numbers against.**
|
||||
- **Related Sub-question**: SQ1 (academic landscape), SQ7 (datasets)
|
||||
|
||||
### Source #35
|
||||
- **Title**: DARPA Fast Lightweight Autonomy (FLA) program page + Test-and-Evaluation review (arXiv 2504.08122)
|
||||
- **Link**: https://www.darpa.mil/research/programs/fast-lightweight-autonomy ; https://arxiv.org/abs/2504.08122
|
||||
- **Tier**: L1 (DARPA program page + 2025 academic review of program results)
|
||||
- **Publication Date**: program 2015–2018 (concluded); review 2025-04 (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Foundational reference; review is current (within 18-month authority window)
|
||||
- **Target Audience**: Defense-program historians + indoor-low-altitude GPS-denied autonomy researchers
|
||||
- **Research Boundary Match**: **Partial — different regime.** FLA = small quadcopters at ≤20 m/s in cluttered indoor/outdoor with onboard sensing only, no satellite-tile basemap. Our project is fixed-wing, ~17 m/s, 1 km AGL, with sat-tile basemap.
|
||||
- **Summary**: Foundational US-defense lineage for GPS-denied autonomy (2015–2018, complete). Set the template for "small UAV + onboard sensors + onboard compute → autonomous obstacle-avoidance + navigation without datalink/GPS". Phase 1 in Florida 2017; Phase 2 in Georgia 2018. The 2025 retrospective (arXiv 2504.08122) reviews FLA's testing methodology and Phase 1 results. Companion 2025 USAF SBIR Phase II solicitation (Sweetspot ID `7946c818-409f-5b31-8f06-554466071d83`) is requesting visual-position-and-navigation capability for sUAS in GPS-denied environments — the regulatory tailwind is now active.
|
||||
- **Related Sub-question**: SQ1 (defense-program lineage)
|
||||
|
||||
### Source #36
|
||||
- **Title**: DSMAC / TERCOM lineage — DTIC ADA315439 (Scene Matching Missile Guidance Technologies) + Wikipedia / SPIE references
|
||||
- **Link**: https://apps.dtic.mil/sti/tr/pdf/ADA315439.pdf ; https://en.wikipedia.org/wiki/DSMAC ; https://www.spiedigitallibrary.org/conference-proceedings-of-spie/0238/1/Terrain-Contour-Matching-TERCOM-A-Cruise-Missile-Guidance-Aid/10.1117/12.959127.short
|
||||
- **Tier**: L1 (DTIC unclassified technical report) + L2 (encyclopedia/SPIE proceedings)
|
||||
- **Publication Date**: DTIC: 1996; SPIE: 1980; Wikipedia: live
|
||||
- **Timeliness Status**: Foundational baseline (no time window per Step 0.5 — established classical algorithms)
|
||||
- **Target Audience**: Cruise-missile-class designers; analogues for downward-vision navigation
|
||||
- **Research Boundary Match**: **Partial — different regime** (cruise missile, terminal guidance). Architectural pattern (pre-cached scene reference + downward camera + correlation matching) is the direct ancestor of our C3 pipeline.
|
||||
- **Summary**: DSMAC = electro-optical camera correlated against pre-stored reference scenes (often from satellite reconnaissance), achieving 3–10 m terminal accuracy. Tomahawk: TERCOM (radar altimeter + DEM) for mid-flight; DSMAC for terminal. CEP without DSMAC: ~30 m; with DSMAC: "only meters". Gulf War 1991: >80% of 280 launched Tomahawks hit target. **Establishes that downward-vision-against-pre-stored-imagery is a 40+ year-old well-characterised technique class with documented accuracy bounds; our project's claim of <500 m / 99.9% reliability is achievable in the same technique class.**
|
||||
- **Related Sub-question**: SQ1 (lineage), SQ8 (baseline accuracy expectations)
|
||||
|
||||
### Source #37
|
||||
- **Title**: Electronic Warfare in Ukraine: The Invisible Battle — Ukraine War Analytics
|
||||
- **Link**: https://ukraine-war-analytics.com/analysis/electronic-warfare-ukraine.html
|
||||
- **Tier**: L3 (analytical aggregator; primary-source numbers cite vendor / OSINT reports)
|
||||
- **Publication Date**: live (accessed 2026-05-07)
|
||||
- **Timeliness Status**: Currently valid (operational-context reference)
|
||||
- **Target Audience**: Ukraine-deployment practitioners
|
||||
- **Research Boundary Match**: Full match (operational geography, threat environment)
|
||||
- **Summary**: Operational-context anchor: Russian EW systems including Pole-21 GPS jammers (25+ km range) plus spoofing capabilities have driven ~70% of small-tactical-UAV losses to EW across the conflict. Twist Robotics' OSCAR cites the same approximate number (~75% of small tactical UAV losses to EW at the front per Source #25). **Confirms the demand-side number is consistent across two independent reporting chains.**
|
||||
- **Related Sub-question**: SQ1 (Ukraine practitioner perspective)
|
||||
|
||||
---
|
||||
|
||||
## SQ2 — Canonical pipeline decomposition
|
||||
|
||||
### Source #38
|
||||
- **Title**: Visual Place Recognition for Aerial Imagery: A Survey (Moskalenko, Kornilova, Ferrer — Skoltech)
|
||||
- **Link**: https://arxiv.org/abs/2406.00885 (v2)
|
||||
- **Tier**: L1 (peer-reviewed survey, accepted in Robotics and Autonomous Systems; companion benchmark code: https://github.com/prime-slam/aero-vloc)
|
||||
- **Publication Date**: arXiv 2024-06; v2 update through 2024
|
||||
- **Timeliness Status**: Currently valid (within 18-month authority window for established surveys; specific candidate latency numbers will need cross-validation against newer Jetson-class hardware reports)
|
||||
- **Target Audience**: Aerial-VPR practitioners + UAV navigation system architects
|
||||
- **Research Boundary Match**: **Full match** for the offline-cache visual geo-localization decomposition (aerial-nadir UAV vs. satellite tile basemap)
|
||||
- **Summary**: Authoritative two-stage pipeline definition (verbatim): "Visual geolocalization can be implemented through various methods, typically relying on a pre-built database of images with known locations. This approach generally involves two stages: **global localization (or Visual Place Recognition, VPR) and local alignment**. Global localization involves identifying the nearest frame from the database (Image Retrieval), while local alignment determines the precise position using the selected frame." Re-ranking is treated as an integral sub-stage of VPR for aerial data because of agricultural/urban grid repetition. Local alignment = SuperPoint/keypoint detector → LightGlue/SuperGlue/SelaVPR matcher → cv2.findHomography → cv2.perspectiveTransform → Web-Mercator coordinate conversion. **Practitioner-critical runtime numbers (RTX 3090, NOT Jetson)**: AnyLoc descriptor calculation = 0.37–0.84 s/frame (huge ViT-G DINOv2); MixVPR / SALAD = 0.05–0.20 s; SelaVPR = 0.04 s; SuperGlue re-rank = 15–25 s on top-100 candidates; LightGlue re-rank = ~1 s; SelaVPR re-rank = <0.1 s. Memory: AnyLoc descriptors = 2.3–13.9 GB for 4–7k tiles; SelaVPR = <0.2 GB. Final commentary: "While our methodology alone may not provide comprehensive robustness, it can be effectively augmented with additional sensors, such as inertial measurement units (IMUs). This integration enhances its utility for Visual Inertial Odometry (VIO) and Simultaneous Localization and Mapping (SLAM) systems, particularly for periodic location refinement and loop closure tasks. Additionally, our methodology could serve as a dependable emergency localization fallback in the event of an unexpected GNSS signal loss." → **Validates the project's IMU/VIO + sat-anchor architecture as the canonical extension of the survey's two-stage core.**
|
||||
- **Related Sub-question**: SQ2 (canonical decomposition), SQ3+SQ4 (C2/C3 candidate latency budgets), SQ5 (foundation-model-on-Jetson failure mode)
|
||||
|
||||
### Source #39
|
||||
- **Title**: Cross-View Geo-Localization: A Survey (Durgam, Paheding, Dhiman, Devabhaktuni — U. Maine / Fairfield / ISU)
|
||||
- **Link**: https://arxiv.org/abs/2406.09722 (v1)
|
||||
- **Tier**: L1 (peer-style preprint, journal-bound — Expert Systems with Applications)
|
||||
- **Publication Date**: arXiv 2024-06
|
||||
- **Timeliness Status**: Currently valid (≤18 months for survey-of-deep-learning architectures)
|
||||
- **Target Audience**: Cross-view (ground↔aerial) geo-localization researchers; partial overlap with our aerial↔satellite pipeline
|
||||
- **Research Boundary Match**: **Partial — different cross-view setup** (the survey focuses on ground panorama → aerial overhead; ours is aerial nadir → satellite ortho). The pipeline-shape lessons transfer; the polar-transform / Siamese-network / GAN-based view-synthesis lessons do NOT directly apply because our two views are both top-down.
|
||||
- **Summary**: Confirms the canonical pipeline decomposition (feature extraction → cross-view matching → similarity-driven retrieval) is the dominant pattern across 2015–2024 SOTA. Establishes the historical lineage: pixel-wise (Sheikh 2003) → feature-based (Lin 2013) → CNN/triplet-loss (Tian 2017) → Siamese+GAN (Hu 2018) → polar-transform (Shi 2019) → CosPlace/EigenPlaces (2022–2023) → DINOv2-class (AnyLoc 2023) → Transformer-only (TransGeo 2022, MGTL 2022) → multi-method fusion (2023+). Backbone comparison table establishes that ViT/DINOv2 is the current SOTA backbone; ResNet-class is the established production baseline; SIFT/SURF/PHOW remain the handcrafted baseline. **Confirms our component-area split (C2 VPR + C3 cross-domain matching) is canonical and matches the survey's two-axis organization (backbone × matching strategy).**
|
||||
- **Related Sub-question**: SQ2 (decomposition lineage), SQ3+SQ4 (C2 candidate landscape)
|
||||
|
||||
### Source #40
|
||||
- **Title**: OrthoLoC: UAV 6-DoF Localization and Calibration Using Orthographic Geodata (Dhaouadi, Marin, Meier, Kaiser, Cremers — DeepScenario / TU Munich / MCML)
|
||||
- **Link**: https://arxiv.org/abs/2509.18350 ; project page https://deepscenario.github.io/OrthoLoC
|
||||
- **Tier**: L1 (peer-style preprint with public dataset, code, model checkpoints; 16,425 UAV images Germany+US, full 6-DoF ground truth)
|
||||
- **Publication Date**: arXiv 2025-09 (within 6-month critical-novelty window)
|
||||
- **Timeliness Status**: Currently valid (within 6-month critical-novelty window for SOTA aerial-localization claims)
|
||||
- **Target Audience**: UAV-localization implementers + system architects building on Digital Orthophotos (DOP) + Digital Surface Models (DSM)
|
||||
- **Research Boundary Match**: **Full match — direct paradigm match** to our project: "lightweight orthographic representations" instead of 3D meshes; "increasingly accessible through free releases by governmental authorities"; "no internet connection or GNSS/GPS support" — exactly the project's constraint envelope.
|
||||
- **Summary**: **Most directly applicable SQ2 source.** Defines the 6-DoF localization pipeline using 2.5D geodata: (1) match query UAV image against DOP (orthophoto raster) using state-of-the-art matchers; (2) lift each 2D match in the DOP to 3D using the corresponding DSM elevation; (3) PnP+RANSAC (RANSAC-EPnP, 5-pixel inlier threshold) → initial pose; (4) Levenberg-Marquardt joint refinement of intrinsics + extrinsics; (5) **AdHoP refinement**: estimate homography from initial 2D-2D correspondences via DLT+RANSAC, warp the DOP to better match the query's perspective, re-match, map back via H⁻¹, lift to 3D, refine pose; accept refinement only if reprojection error decreases. **Quantitative results** on 16.4k images, 47 locations: best matcher = GIM+DKM achieves 75.4% recall at 1m-1° threshold (sparse SP+SG = 64.4%, sparse SP+LG = 64.2%, MASt3R = 63.5%, RoMa+AdHoP = 54.6%, XFeat*+AdHoP = 59.8%; LoFTR / eLoFTR / XoFTR all <23% recall). AdHoP yields ~30% average matching improvement, ~20% translation/rotation error reduction; for previously-underperforming methods (XFeat* → 95% matching improvement; DKM → 63% translation reduction; RoMa → 1m-1° recall +23%). **Performance factors** explicitly characterized: (a) **cross-domain DOPs (visual gap only) cause ~3× translation error increase** even on best method; (b) **cross-domain DOPs+DSMs (visual + structural gap) cause ~7× translation error increase** (0.16 m → 1.12 m for GIM+DKM+AdHoP) — **this is exactly the war-zone scene-change scenario AC-3.x covers**; (c) **20% covisibility floor** between query and reference; below it localization fails; (d) **Calibration is fundamentally ambiguous** between focal length and translation → camera intrinsics MUST be calibrated upstream, not jointly optimized in flight. (e) Resolution: scaling images to 30% of original (~300 px) still works; geodata at 13 m/pixel is the floor, with degradation below.
|
||||
- **Related Sub-question**: SQ2 (canonical pipeline + AdHoP refinement loop), SQ3+SQ4 (C3 matcher candidate ranks), SQ5 (war-zone scene-change failure mode), SQ8 (covisibility safety gate)
|
||||
|
||||
### Source #41
|
||||
- **Title**: Exploring the best way for UAV visual localization under Low-altitude Multi-view Observation Condition: a Benchmark — AnyVisLoc (Ye, Teng, Chen, Li, Liu, Yu, Tan — NUDT / Macao Polytechnic)
|
||||
- **Link**: https://arxiv.org/abs/2503.10692 ; benchmark code https://github.com/UAV-AVL/Benchmark
|
||||
- **Tier**: L1 (peer-style preprint with public 18,000-image dataset across 15 Chinese cities, multi-pitch / multi-altitude / multi-scene, with both aerial-photogrammetry AND satellite reference maps)
|
||||
- **Publication Date**: arXiv 2025-03 (within 6-month critical-novelty window)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: Aerial AVL practitioners; UAV-system designers facing pitch/altitude/yaw uncertainty
|
||||
- **Research Boundary Match**: **Partial — different altitude regime** (the benchmark covers 30–300 m AGL, ours is ~1 km AGL); pitch range is 20–90° (ours is mostly nadir, ~80–90°). Lessons on the **pipeline structure, retrieval-vs-matching trade-offs, sensor-prior noise tolerance, and aerial-vs-satellite reference-map gap** transfer directly.
|
||||
- **Summary**: Independently confirms the SAME pipeline as Source #40: image retrieval (rough position) → image matching (2D-2D) → DSM-lift to 3D → PnP+RANSAC. Best baseline = CAMP (retrieval) + RoMa (dense matcher) + Top-N re-rank → 74.1% A@5m on aerial photogrammetry map, 18.5% A@5m on satellite map (ALOS 30m DSM). **Critical AC-quantitative findings**: (a) **Aerial map vs satellite map**: 4× accuracy gap at A@5m (74.1% vs 18.5%) — driven by satellite-DSM coarseness (ALOS 30m vs aerial 0.94m) and modality difference. **Direct relevance**: project's offline cache is satellite tiles ≥0.5 m/px without DSM; this places us between the two data points (better than ALOS 30m, worse than aerial photogrammetry) — exact accuracy must be re-established once tile resolution is pinned. (b) **Yaw prior noise**: σ ≤ 5° → no impact; σ = 10° → 1.9% A@5m drop; σ = 30° → 4.1% drop; σ = 50° → 13.7% drop; σ = 60° → 25.7% drop. **Implication for project's C1+C5+IMU**: companion-side yaw estimate must hold σ < 10°. (c) **Pitch prior noise**: σ < 5° → no impact; σ ≥ 7° causes ~1–5% drops. (d) **Pitch angle**: smaller pitch (more oblique) → lower accuracy; nadir is best. Project's nadir-fixed camera at 1 km AGL is consistent with the benchmark's most-favourable regime. (e) **Sparse vs dense matchers**: SP+LightGlue+GIM+k2s = 75.4% A@10m at 105 ms/frame; RoMa = 81.3% A@10m at 659 ms/frame. **Implication for project's C7 Jetson runtime**: dense matchers ~6× more accurate but ~6× slower → SP+LightGlue-class is the production sweet spot under our 400 ms budget. (f) **Re-ranking strategy**: Top-N re-rank by inlier count = best accuracy/cost trade-off (62.2% A@5m at 0.8 s/frame on RTX 3090). Match-without-retrieval = catastrophic (34.3% A@5m, search-space too large).
|
||||
- **Related Sub-question**: SQ2 (pipeline + sensor-prior tolerance), SQ3+SQ4 (C2 retrieval-vs-matcher trade-offs, C5 IMU prior contract), SQ5 (war-zone reference-map staleness failure mode), SQ7 (aerial-vs-satellite reference benchmarks)
|
||||
|
||||
### Source #42
|
||||
- **Title**: Survey on absolute visual localization techniques for low-altitude unmanned aerial vehicles (Ye, Chen, Teng, Li, Yang, Song, Yu — NUDT, College of Aerospace Science)
|
||||
- **Link**: https://www.sciopen.com/article/10.11887/j.issn.1001-2486.25120033 ; DOI 10.11887/j.issn.1001-2486.25120033
|
||||
- **Tier**: L1 (peer-reviewed Chinese journal — Journal of National University of Defense Technology, vol 48 issue 2, 2026; same lab as Source #41 with overlapping authorship — confirmed cross-validation, not duplicative)
|
||||
- **Publication Date**: 2026-04-01 (within 6-month critical-novelty window)
|
||||
- **Timeliness Status**: Currently valid
|
||||
- **Target Audience**: UAV-system architects + Chinese-defense-research community
|
||||
- **Research Boundary Match**: **Full match** (low-altitude UAV AVL is the survey's exact subject)
|
||||
- **Summary**: Survey-level confirmation of the canonical "**retrieval-matching-pose estimation**" hierarchical framework. Verbatim claim: "the hierarchical framework balances search efficiency, positioning accuracy, and scene generalization, becoming a robust technical path for low-altitude long-endurance absolute localization." Compares the framework against alternatives that are explicitly rejected: (a) relative visual localization (cumulative errors — VIO/SLAM only); (b) end-to-end direct localization (poor generalization); (c) map-free localization (scene-dependent). Sub-component evolution per stage: (a) retrieval = template-matching (SAD/SSD/NCC) → BoW/VLAD → deep-learning (annular/dense feature segmentation, contrastive InfoNCE, self-supervised); (b) matching = SIFT/SURF/ORB → SuperPoint+LightGlue/RoMa (sparse / semi-dense / dense); (c) pose estimation = PnP variants + RANSAC + IMU prior fusion. **Identifies four open challenges** that align with project risks: (i) cross-domain generalization (war-zone scene change); (ii) real-time inference on edge platforms (Jetson); (iii) robustness to complex environments (cropland, snow, low texture); (iv) high-quality datasets (the same gap our project's AC-NEW-7 / cache provisioning works around). **Lightweight-model-design-for-edge-deployment is named as a primary future-research direction** — directly validates project's Jetson Orin Nano constraint as a recognized field-level challenge, not a project-specific oddity.
|
||||
- **Related Sub-question**: SQ2 (framework canonicalness), SQ3+SQ4 (per-component evolution), SQ5 (named open challenges align with project risks)
|
||||
@@ -0,0 +1,410 @@
|
||||
# Fact Cards
|
||||
|
||||
> Mode A Phase 2 — engine Step 3 (Fact Extraction & Evidence Cards). Extracted from sources logged in `01_source_registry.md`. Confidence labels: ✅ High (L1 / verified source code), ⚠️ Medium (L1/L2 with caveat), ❓ Low (L3/L4 inferential).
|
||||
>
|
||||
> Bound to sub-questions in `00_question_decomposition.md`. Many SQ6 facts also bind directly to the Project Constraint Matrix (`acceptance_criteria.md` / `restrictions.md`); per the engine's "Per-Mode API Capability Verification" rule, MAVLink/MSP messages are treated as candidate **modes** and are bound `Pass/Fail/Verify/N/A` against numbered ACs and restrictions.
|
||||
|
||||
---
|
||||
|
||||
## SQ6 — ArduPilot Plane vs iNav external positioning
|
||||
|
||||
### Fact #1 — ArduPilot Plane EKF3 ingests `GPS_INPUT` (MAVLink ID 232) as a first-class GPS source
|
||||
- **Statement**: ArduPilot's `AP_GPS_MAV` driver (master) decodes `MAVLINK_MSG_ID_GPS_INPUT` and stores the resulting state into the GPS slot identified by `gps_id`. Decoded fields: lat/lon (degE7), alt (mm → cm internally), hdop/vdop, velocity (vn/ve/vd cm/s), speed/horizontal/vertical accuracy (m / m/s), yaw (cdeg, `0` sentinel = "not provided"). Honors `ignore_flags` for ALT/HDOP/VDOP/VEL_HORIZ/VEL_VERT/SPEED_ACCURACY/HORIZONTAL_ACCURACY/VERTICAL_ACCURACY. Requires `fix_type ≥ 3` and `time_week > 0` for jitter-corrected timestamping.
|
||||
- **Source**: Source #4 (AP_GPS_MAV.cpp master), Source #1 (Plane Non-GPS Navigation docs)
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: ArduPilot Plane operators / developers
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: C8 (FC adapter), C5 (estimator covariance contract)
|
||||
- **Fit Impact**: **supports selection** — ArduPilot side of AC-4.3 is satisfied by `GPS_INPUT` as the primary external-positioning message; covariance fields (`horiz_accuracy`, `vert_accuracy`, `speed_accuracy`) are wired through.
|
||||
|
||||
### Fact #2 — ArduPilot's covariance honesty (AC-NEW-4) is enforced via the `horiz_accuracy` field of `GPS_INPUT`
|
||||
- **Statement**: When `GPS_INPUT_IGNORE_FLAG_HORIZONTAL_ACCURACY` is unset, AP_GPS stores `packet.horiz_accuracy` into `state.horizontal_accuracy` and sets `state.have_horizontal_accuracy = true`. EKF3's quality chain consumes this via (a) ground-stationary 3 m drift check (`_gpsCheckScaler`-modulated), (b) innovation gating (`POS_I_GATE`/`VEL_I_GATE`), (c) soft de-weighting via `EK3_GLITCH_RADIUS` (PR #24135). Under-reporting `horiz_accuracy` defeats these gates — exactly the AC-NEW-4 risk the project flagged.
|
||||
- **Source**: Source #4, Source #23 (PR #24135), Source #24 (AP_NavEKF3 master)
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers writing the C5 estimator → C8 adapter
|
||||
- **Confidence**: ✅ (source code + L1 docs); ⚠️ for the precise innovation-gate mechanics (deferred to design-phase SITL tuning)
|
||||
- **Related Dimension**: C5 covariance, AC-NEW-4
|
||||
- **Fit Impact**: **architectural constraint** — the C5 estimator MUST publish honest `horiz_accuracy` (not optimistic) for AP's EKF3 quality chain to function. Aligns directly with AC-1.4 / AC-NEW-4.
|
||||
|
||||
### Fact #3 — ArduPilot supports runtime EKF source-set switching from companion via `MAV_CMD_SET_EKF_SOURCE_SET`
|
||||
- **Statement**: EKF3 supports up to three source sets (`EK3_SRC1..3_*`). A companion can request a switch by sending `MAV_CMD_SET_EKF_SOURCE_SET`. Alternative paths: RC aux-switch option 90 ("EKF Pos Source"), Lua scripts (e.g., `ahrs-source.lua`). **Caveat from L1 docs**: "no GCSs are currently known to implement this" — companion-driven switching works at the firmware level but is not exposed in stock GCS UIs.
|
||||
- **Source**: Source #2, Source #3
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers handling AC-NEW-2 spoof-promotion path on ArduPilot
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: C8 + AC-NEW-2
|
||||
- **Fit Impact**: **supports selection** — AP allows the project to model two source sets (set 1 = real GPS, set 2 = onboard `GPS_INPUT`) and switch automatically. Keeps companion lightweight; switching does not require the companion to suppress real-GPS itself.
|
||||
|
||||
### Fact #4 — ArduPilot ODOMETRY-velocity-only fusion is currently NOT supported (open enhancement)
|
||||
- **Statement**: Issue #23485 confirms current limitation: feeding `ODOMETRY` without position causes EKF position-estimate timeout / failsafe. Implication: the project's `visual_propagated` mode (VO drift between satellite anchors, no global position) **cannot be expressed as ODOMETRY-velocity-only on current AP** — must be sent as a full `GPS_INPUT` with covariance widened to reflect drift uncertainty.
|
||||
- **Source**: Source #8
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers
|
||||
- **Confidence**: ✅ (open enhancement, open as of accessed date)
|
||||
- **Related Dimension**: C5 + C8 + AC-1.3 (`visual_propagated` label) + AC-1.4 (covariance ellipse)
|
||||
- **Fit Impact**: **architectural constraint** — `visual_propagated` and `dead_reckoned` labels both ride `GPS_INPUT` with growing `horiz_accuracy`, NOT a separate `ODOMETRY` channel. Single-message contract = simpler. AC-NEW-8 thresholds (`horiz_accuracy = 999.0` for "no fix") map directly.
|
||||
|
||||
### Fact #5 — iNav firmware (master, post-9.0) has NO inbound MAVLink handler for any external-positioning message
|
||||
- **Statement**: Authoritative inbound switch in `src/main/telemetry/mavlink.c::processMAVLinkIncomingTelemetry` (master) handles only: HEARTBEAT, PARAM_REQUEST_LIST (stub reply), MISSION_CLEAR_ALL, MISSION_COUNT, MISSION_ITEM, MISSION_REQUEST_LIST, MISSION_REQUEST, COMMAND_INT (only `MAV_CMD_DO_REPOSITION`), RC_CHANNELS_OVERRIDE, ADSB_VEHICLE, RADIO_STATUS. **No `GPS_INPUT`, `VISION_POSITION_ESTIMATE`, `ODOMETRY`, `GLOBAL_POSITION_INT`, or `GPS_RAW_INT` are accepted as inputs.** Wiki page (Source #10) confirms: "Limited command support: Commands that are not implemented are ignored."
|
||||
- **Source**: Source #9 (master code), Source #10 (wiki, edited 2025-12-11)
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers + AC-4.3 author
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: C8, AC-4.3
|
||||
- **Fit Impact**: **DISQUALIFIES the literal AC-4.3 wording** ("the standard external-positioning message type(s) accepted by ArduPilot AND iNav"). No single MAVLink external-positioning message is accepted by both FCs. Project must adopt a per-FC adapter design and AC-4.3 must be revised to acknowledge two transports.
|
||||
|
||||
### Fact #6 — iNav accepts external GPS injection via two MSP paths; `MSP2_SENSOR_GPS` is the covariance-rich path
|
||||
- **Statement**: `MSP_SET_RAW_GPS (201)` (legacy MSP1, 14 bytes): fixType, numSat, lat, lon, alt (m, internal cm), speed (cm/s). **No covariance, no per-axis velocity, no yaw.** `MSP2_SENSOR_GPS (7939, MSPv2 sensor plugin)`: instance, gpsWeek, msTOW, fixType, satellitesInView, hPosAccuracy (mm), vPosAccuracy (mm), hVelAccuracy (cm/s), hdop, lat, lon, mslAltitude (cm), nedVelNorth/East/Down (cm/s), groundCourse (cdeg×100), trueYaw (cdeg×100), date+time. Routes through `mspGPSReceiveNewData()` via `GPS_PROVIDER_MSP`. Requires build flag `USE_GPS_PROTO_MSP` — **enabled by default in iNav's `target/common.h`**, so stock firmware reaches this path.
|
||||
- **Source**: Source #12 (MSP message reference, master), Source #13 (target/common.h master + gps.c provider table)
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers (C8 adapter, MSP transport)
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: C8, C5 covariance contract
|
||||
- **Fit Impact**: **supports selection** of `MSP2_SENSOR_GPS` for the iNav adapter. Covariance fields (`hPosAccuracy`, `vPosAccuracy`, `hVelAccuracy`) align semantically with `GPS_INPUT.horiz_accuracy` / `vert_accuracy` / `speed_accuracy`, but unit conversions differ (mm vs m). The C8 adapter must therefore be FC-aware, not protocol-monomorphic.
|
||||
|
||||
### Fact #7 — iNav does NOT support dual-GPS arbitration; companion must be the SOLE GPS source
|
||||
- **Statement**: Issue #10141 is an open feature request for dual-GPS support. Current iNav (master incl. 9.0.x) has single-GPS architecture with one UART selected as the GPS port. There is no primary/secondary failover and no per-instance arbitration in the nav stack.
|
||||
- **Source**: Source #14
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers (architecture)
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: C8, C5, AC-NEW-2 (spoof promotion)
|
||||
- **Fit Impact**: **architectural constraint** — on iNav, real GPS receivers must NOT be wired directly to the FC. Real GPS goes to the companion; the companion fuses (or rejects) it and emits the single iNav-facing feed via MSP2_SENSOR_GPS (or via a UBX-emulation UART). AC-NEW-2 latency on iNav = companion's internal reaction time only; iNav does not participate in source switching at all.
|
||||
|
||||
### Fact #8 — iNav explicitly does NOT validate GPS for spoofing; anti-spoofing is fully the companion's responsibility
|
||||
- **Statement**: iNav's `docs/GPS_fix_estimation.md` states verbatim: "Not a solution for GPS spoofing (GPS output is not validated in INAV)." Combined with Fact #7, the architectural conclusion on iNav: companion = anti-spoofing oracle + nav-camera estimator + IMU-propagation source, all collapsed into the single MSP2_SENSOR_GPS feed.
|
||||
- **Source**: Source #15
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers; AC-NEW-2 / AC-3.5 / AC-NEW-8 owners
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: AC-NEW-2, AC-3.5, AC-NEW-8
|
||||
- **Fit Impact**: **supports selection** of "companion as iNav's only GPS"; **disqualifies** any architecture that relies on iNav-side spoof detection for AC-NEW-2 reaction.
|
||||
|
||||
### Fact #9 — iNav dead-reckoning has documented stability bugs under intermittent feeds; AC-NEW-8 must avoid letting iNav enter dead-reckoning
|
||||
- **Statement**: Issue #10588 documents porpoising and motor-burst behaviour during intermittent GPS outages on iNav fixed-wing dead-reckoning. The community recommendation captured in the issue: "GPS should be rejected if providing erroneous coordinates rather than no fix." `inav_allow_dead_reckoning` (default OFF) and `inav_allow_gps_fix_estimation` (default OFF) are both fixed-state booleans — entering dead-reckoning mid-flight is a discrete transition, not a smooth degrade.
|
||||
- **Source**: Source #15, Source #16 (Settings.md), Source #17 (#10588)
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers; AC-NEW-8 owner
|
||||
- **Confidence**: ✅ for setting names; ⚠️ for severity of stability bug (single open issue)
|
||||
- **Related Dimension**: AC-NEW-8, AC-3.5, C8
|
||||
- **Fit Impact**: **architectural constraint** — on iNav, the AC-NEW-8 path must keep emitting `MSP2_SENSOR_GPS` with growing `hPosAccuracy` rather than letting the feed drop and iNav switch to dead-reckoning. The "no fix" semantics on iNav must be expressed via `fixType` field of MSP2_SENSOR_GPS (not by silence). The horiz/vert accuracy fields are the only signal available; iNav has no equivalent of the AP `horiz_accuracy = 999.0` "no fix" sentinel — must verify which `fixType` enum values iNav treats as no-fix.
|
||||
|
||||
### Fact #10 — iNav supports UBX-only over UART (NMEA dropped in 7.0); UBX emulation is a viable third transport
|
||||
- **Statement**: iNav 7.0 removed NMEA. Currently supports u-blox UBX protocol with version ≥ 15.00 in 9.0+. Recommended physical receivers: u-blox M8/M9/M10. Companion can implement a UBX-emulation writer on the iNav GPS UART (NAV-PVT mandatory; NAV-DOP optional). UBX carries `hAcc`/`vAcc`/`headAcc`/velocity components — covariance honesty preserved.
|
||||
- **Source**: Source #11 (iNav GPS-and-Compass-setup wiki)
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System designers (transport-choice)
|
||||
- **Confidence**: ✅ for UBX-only; ⚠️ for "minimum NAV-* set" — the canonical U-blox protocol spec (Source filed in agent-tools as `fd8513f8-...txt`) plus iNav's `gps_ublox.c` drive the precise message set; **this is a follow-up search before final selection**.
|
||||
- **Related Dimension**: C8 transport choice
|
||||
- **Fit Impact**: **alternate candidate, NOT YET SELECTED** — UBX path bypasses MSP queueing/arbitration concerns and treats the companion as a normal GPS to iNav. Trade-off: implementation cost (UBX writer + correct ACK behaviour) vs. MSP path (already-designed wire format, but iNav-specific).
|
||||
|
||||
---
|
||||
|
||||
## SQ6 — Conclusions (working summary, will be re-checked at Step 7.5)
|
||||
|
||||
### Per-FC adapter design is unavoidable (single-message AC-4.3 wording is unsatisfiable)
|
||||
|
||||
| FC | Inbound external-positioning transport | Message | Covariance fields | Per-axis velocity | Yaw | Source-switching from companion |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **ArduPilot Plane** | MAVLink (TELEM/USB/UDP serial) | `GPS_INPUT` (id 232) — primary | `horiz_accuracy`, `vert_accuracy`, `speed_accuracy` (m/m·s⁻¹) | `vn`, `ve`, `vd` (cm/s) | `yaw` cdeg, 0 = not provided | `MAV_CMD_SET_EKF_SOURCE_SET` (FW supports; stock GCS UIs do not — companion-driven OK) |
|
||||
| **iNav** | MSP2 (UART/USB) | `MSP2_SENSOR_GPS` (id 7939) — primary candidate | `hPosAccuracy` mm, `vPosAccuracy` mm, `hVelAccuracy` cm/s | `nedVelNorth/East/Down` cm/s | `trueYaw` cdeg×100 | **N/A** — iNav has single-GPS arch; companion = sole GPS source |
|
||||
| iNav alt 1 | MSP1 | `MSP_SET_RAW_GPS` (id 201) — **rejected for production** | none | none | none | N/A |
|
||||
| iNav alt 2 | UART | UBX emulation (NAV-PVT etc.) — **alternate candidate, requires NAV-* subset verification** | UBX `hAcc`/`vAcc`/`headAcc` mm/cm/scale | NED in NAV-PVT | yes | N/A |
|
||||
|
||||
**Selection (preliminary, pending Step 7.5 component-fit gate):**
|
||||
- **AP path**: `GPS_INPUT` — Selected (lead).
|
||||
- **iNav path**: `MSP2_SENSOR_GPS` — Selected (lead). UBX-emulation kept as fallback if MSP2_SENSOR_GPS proves rate-limited or quality-flag-lossy.
|
||||
|
||||
### AC / Restriction binding (per-mode, Per-Mode API Capability Verification rule)
|
||||
|
||||
| Numbered AC / Restriction | AP `GPS_INPUT` | iNav `MSP2_SENSOR_GPS` | iNav `MSP_SET_RAW_GPS` |
|
||||
|---|---|---|---|
|
||||
| AC-1.4 (95% cov + source label `{satellite_anchored, visual_propagated, dead_reckoned}`) | **Pass** (`horiz_accuracy` carries 95% covariance proxy; source label is companion-side metadata, not in MAVLink — emit via STATUSTEXT/NAMED_VALUE_FLOAT) | **Pass** (`hPosAccuracy` = covariance proxy; same off-band source-label channel) | **Fail** (no covariance field → cannot publish 95% ellipse) |
|
||||
| AC-NEW-4 (false-position safety budget; covariance honesty) | **Pass** (de-weighted via `EK3_GLITCH_RADIUS` if covariance is honest) | **Verify** (need to confirm iNav nav-stack actually uses `hPosAccuracy` for outlier handling — pre-Step-7.5 follow-up) | **Fail** |
|
||||
| AC-NEW-2 (<3 s p95 spoof promotion) | **Verify** via SITL (`MAV_CMD_SET_EKF_SOURCE_SET` round-trip latency under load) | **Pass** by architecture (companion is sole GPS, no FC-side switch needed) | Pass-by-arch but Fails AC-1.4 |
|
||||
| AC-NEW-8 (visual-blackout + spoofed GPS failsafe; covariance growth + degraded fix levels) | **Pass** (`fix_type` 0/1/2 + `horiz_accuracy=999.0` documented sentinel maps to AC-NEW-8 thresholds) | **Verify** (iNav's `fixType` enum mapping for "no fix" — pre-Step-7.5 follow-up) | **Fail** (no graceful degrade signal) |
|
||||
| AC-3.5 (label switch within ≤1 frame OR ≤400 ms; reject spoofed GPS as input) | **Pass** by architecture (EKF source switch + STATUSTEXT) | **Pass** by architecture (companion suppresses spoofed-GPS contribution upstream) | Pass-by-arch but Fails AC-1.4 |
|
||||
| AC-4.3 (FC accepts the chosen messages) | **Pass** | **Pass** (default build, `USE_GPS_PROTO_MSP` on) | **Pass** but Fails AC-1.4 — discard |
|
||||
| Restriction "Supported FCs: ArduPilot, iNav (both via standard MAVLink)" | **Pass** | **Fail** of "via standard MAVLink" — restriction's literal wording is incorrect because iNav has no inbound MAVLink external-positioning. The restriction must be revised to "ArduPilot via MAVLink GPS_INPUT; iNav via MSP2_SENSOR_GPS". | n/a |
|
||||
|
||||
### Required AC / Restrictions edits flagged for user review
|
||||
|
||||
1. **AC-4.3** — current text says "the standard external-positioning message type(s) accepted by ArduPilot and iNav". Reality: no single message type is accepted by both. **Proposed revision** (outcome-shaped, IEEE-830-style): "WGS84 coordinates are delivered to each supported FC via that FC's documented external-positioning interface — MAVLink `GPS_INPUT` for ArduPilot Plane, MSP2 `MSP2_SENSOR_GPS` for iNav. Honest covariance is carried in the field each FC uses for outlier rejection (under-reported covariance is a defect — see AC-NEW-4). Source-label semantics per AC-1.4 are emitted out-of-band (FC-appropriate STATUSTEXT / NAMED_VALUE_FLOAT / equivalent)."
|
||||
2. **Restriction "Communication protocol (pinned): MAVLink for both FC and GCS"** — incorrect for iNav. **Proposed revision**: "Communication protocol: MAVLink for ArduPilot Plane and for QGroundControl GCS; MSP2 for iNav (UART or USB transport). MAVLink remains the GCS-facing protocol for both FCs." (iNav still emits MAVLink telemetry outbound to QGC; this is preserved.)
|
||||
3. **AC-NEW-2** — keep numerical budget (<3 s p95) but split per-FC validation: ArduPilot validation = SITL round-trip of `MAV_CMD_SET_EKF_SOURCE_SET` from companion under spoof injection; iNav validation = companion-internal reaction time (companion-only metric — iNav doesn't participate).
|
||||
4. **AC-NEW-8** — language "fix-quality 2D fix or worse when covariance > 100 m" maps to `GPS_INPUT.fix_type` for AP. iNav's `fixType` enum mapping (per `gpsFixType_e` in iNav's enums-reference) must be confirmed at design time before this AC is testable on iNav.
|
||||
|
||||
### Open follow-up probes (deferred to SQ8 + design phase, NOT blocking SQ6 closure)
|
||||
|
||||
- **(SQ8)** Confirm the precise MAVLink message + field set ArduPilot exposes for spoofing/jamming integrity reports (PR #2110 merged, but `GPS_RAW_INT` in current published common.xml shows no spoofing bits — likely lives in a sibling message such as `GPS_INTEGRITY`). This is the FC→companion direction needed for AC-NEW-2's input side and AC-3.5's spoofing detection.
|
||||
- **(SQ8)** UBX-emulation minimum NAV-* subset for iNav 9.0 (UBX ≥ 15.00). Authoritative inputs: U-blox protocol spec (cached) + iNav `gps_ublox.c` (cached). Output a "minimum companion-side UBX writer" definition.
|
||||
- **(design)** SITL parameter sets for both FCs for AC-NEW-2 / AC-NEW-8 validation. Out of research scope.
|
||||
- **(design)** Verify iNav nav-stack consumption of `MSP2_SENSOR_GPS.hPosAccuracy` for outlier handling (read `src/main/io/gps_msp.c` / `mspGPSReceiveNewData` in design phase, not research phase).
|
||||
|
||||
### Boundary check: this SQ6 is saturated for the architectural decision
|
||||
|
||||
Saturation signals observed: ArduPilot side covered by L1 docs + L1 source code; iNav side covered by L1 source code (master) + L1 wiki (edited 2025-12-11) + L1 release notes (8.0/9.0). Three independent rounds of search yielded the same architectural conclusion (no inbound external-positioning MAVLink on iNav). Last queries returned no novel facts. Per `references/source-tiering.md` "Search saturation rule" → SQ6 is closed pending the SQ8 follow-up probes above; user decision required on the AC/restriction edits before further architectural work.
|
||||
|
||||
---
|
||||
|
||||
## SQ1 — Existing / competitor GPS-denied UAV navigation systems
|
||||
|
||||
### Fact #11 — Twist Robotics OSCAR is a deployed Ukrainian peer system in the same architectural class as this project
|
||||
- **Statement**: Twist Robotics (Ukraine) has a fielded camera + map-matching navigation module called OSCAR (Optical System of Coordinates with Automatic Relocalisation). The vendor states the system "captures the terrain, identifies landmarks, compares them with a map, determines coordinates, and transmits them to the autopilot as a reliable GPS signal" — the same five-stage architecture this project is building. Vendor-stated specs: ≤20 m accuracy without cumulative error, day/night/fog operation, and operational deployment of "more than 500,000 km across 25,000 combat missions over 24 months". Hardware includes active cooling, indicating a non-trivial onboard compute (likely Jetson-class). **No public independent benchmark of the 20 m number.**
|
||||
- **Source**: Source #25, Source #26
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System architects + AC owners (existence-of-peer evidence, not implementation guide)
|
||||
- **Confidence**: ✅ for "deployed at scale on Ukrainian combat platforms"; ⚠️ for "20 m accuracy" (vendor self-report); ❓ for "fully resistant to spoofing and jamming" (claim not independently verified)
|
||||
- **Related Dimension**: SQ1, SQ8 (anti-spoofing claim audit), SQ9 (synthesis — ours must beat or at least match this in the operational regime)
|
||||
- **Fit Impact**: **establishes feasibility floor** — a Ukrainian peer is operating a similar architecture against the same threat environment our system targets. Project framing must explicitly differentiate (e.g., 1 km AGL vs unspecified OSCAR altitude; 8 h endurance vs unspecified OSCAR endurance; AC-NEW-4 honest covariance contract vs OSCAR's unspecified covariance reporting).
|
||||
|
||||
### Fact #12 — Auterion Artemis is a production-shipping fixed-wing one-way attack drone with Ukraine-validated GPS-denied navigation, defining the production benchmark for this class
|
||||
- **Statement**: Auterion completed the US Defense Innovation Unit Artemis program in October 2025, delivering a Shahed-class deep-strike drone with up to 1,000-mile range and up to 40 kg warhead, running on **Auterion Skynode N mission computer + Auterion Visual Navigation system + built-in terminal guidance**. Government evaluators signed off after operational flight tests in Ukraine including ground launch, GPS and GPS-denied navigation, long-range transit, and terminal engagement. Manufacturing is being established in US, UA, and DE; Auterion is offering the system to the US Department of War and allied nations.
|
||||
- **Source**: Source #31; Source #32 confirms Skynode S sibling architecture (NPU-equipped companion).
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System architects (production-pattern reference)
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: SQ1 (closest commercial production peer), SQ9 (architecture template)
|
||||
- **Fit Impact**: **establishes production reference architecture** — companion-class autopilot + visual navigation + terminal guidance is shipping at production scale to a US defense customer. Implication: building a per-FC adapter (project decision in SQ6) is consistent with what production stacks already do; integrating against the Artemis architecture is realistic; competing on price + Ukraine-specific operational tuning + AC-NEW-4 honest-covariance contract is a viable differentiation.
|
||||
|
||||
### Fact #13 — Vantor Raptor is a production COTS visual-GPS-replacement software suite, demonstrating that "branded sat-tile basemap + on-drone vision software" is a viable commercial pattern
|
||||
- **Statement**: Vantor Raptor product family (Guide / Sync / Ace) provides vision-based GPS replacement using the drone's existing camera plus Vantor's "100 million-plus sq km of highly accurate 3D terrain data" (Vivid Terrain, vendor-stated 3 m accuracy). Vendor-demonstrated absolute accuracy: **<7 m in all dimensions** for aerial position (Guide), **<3 m** for ground coordinate extraction (Sync, Ace). Works at night and at low altitudes. Platform-agnostic, deployable on commodity hardware, integrates with existing onboard cameras. Inertial Labs has published a VINS-integrated Raptor Guide white paper. Recent partnerships: Niantic Spatial (Dec 2025) for unified air-to-ground positioning in GPS-denied areas; Maxar partnership with AIDC (Sep 2025) for Taiwan UAV resilience against GPS interference.
|
||||
- **Source**: Source #30
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Architecture / business decision-makers (build-vs-buy framing)
|
||||
- **Confidence**: ✅ for product existence + claimed accuracy bounds (vendor primary); ⚠️ for whether Vantor's commercial accuracy figures hold under the project's specific Ukrainian-steppe + active-conflict-tile-staleness conditions
|
||||
- **Related Dimension**: SQ1 (commercial), C2/C3 (commercial alternatives to building ourselves), SQ8 (basemap as a service vs offline cache)
|
||||
- **Fit Impact**: **build-vs-buy lens** — Raptor Guide's <7 m claim is *better* than the project's AC-1.1 budget (≤80 m / 95% under AC-1.1.1), so it's not a disqualifier on accuracy. Reasons we still build vs buy: (a) Vantor is a US vendor; export / dual-use licensing into the Ukrainian battlefield is uncertain; (b) restrictions specify offline cache from the project's own Azaion Suite Satellite Service (AC-2.x), not Vantor's Vivid Terrain — replacing the basemap is non-negotiable; (c) covariance honesty contract (AC-NEW-4) and source-label contract (AC-1.4) are project-specific and may not be exposed by Vantor's API. **Outcome**: keep Raptor as a competitive comparator in `solution_draft01`, NOT as a candidate component to integrate.
|
||||
|
||||
### Fact #14 — snktshrma/ngps_flight (NGPS — ArduPilot GSoC 2024) is the closest open-source pipeline match to this project's exact C1+C2+C3+C5+C8 stack
|
||||
- **Statement**: NGPS = ROS 2 + ArduPilot pipeline composed of three packages: **`ap_ngps_ros2`** (visual geo-localization at 1–2 Hz by matching live camera frames to georeferenced satellite imagery using **LightGlue + SuperPoint**, deep-learning-based feature matching), **`ap_ukf`** (Unscented Kalman Filter fusing NGPS absolute positions with VIO estimates), **`ap_vips`** (VIO providing relative pose). Output is fused odometry to ArduPilot's EKF (per related ArduPilot issue #23471, this is via `VISION_POSITION_ESTIMATE` requiring EKF source-set 2/3 with `EK3_SRC*_POSXY=Vision`). Project is published under ArduPilot's GSoC 2024 program. Sibling `ap_nongps` is an earlier OpenCV-based prototype.
|
||||
- **Source**: Source #33
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Implementer / Engineer
|
||||
- **Confidence**: ✅ for project existence, component breakdown, and matcher choice (LightGlue+SuperPoint); ⚠️ for runtime behaviour under our exact constraints (Jetson Orin Nano, 1 km AGL, 17 m/s, 3 fps); ❓ for production hardening / covariance honesty / spoof-defence (none documented)
|
||||
- **Related Dimension**: SQ1 (closest open-source peer), SQ2 (canonical pipeline confirmation), SQ3+SQ4 (architectural template for component candidate matrix), SQ6 (alternate AP transport debate)
|
||||
- **Fit Impact**: **architectural template** — confirms the project's split (C1 VIO ↔ C2/C3 visual absolute ↔ C5 fusion ↔ C8 FC adapter) is canonical, not novel. Two concrete deltas:
|
||||
1. **Transport choice on AP**: NGPS uses `VISION_POSITION_ESTIMATE`. SQ6 picked `GPS_INPUT` because it carries `horiz_accuracy` directly, supports source-set switching via `MAV_CMD_SET_EKF_SOURCE_SET`, and avoids EKF-source-set reconfiguration. The trade-off (NGPS's path vs SQ6's pick) must be re-examined at design time before final AP-transport selection.
|
||||
2. **Estimator choice**: NGPS uses UKF; SQ3/SQ4 will compare UKF vs ESKF vs MSCKF vs factor-graph (GTSAM) on the same matrix.
|
||||
|
||||
### Fact #15 — RGB satellite-image matching as a *low-altitude* (<25 m AGL) localization technique is unreliable per the SPRIN-D Challenge; our 1 km AGL operates in the regime where the same authors note it "works reasonably well"
|
||||
- **Statement**: The CTU Prague team's SPRIN-D winning paper directly states: *"Some teams used RGB satellite image-based matching, but this has proved to be highly unreliable at such low altitudes."* (referring to <25 m AGL). The paper's related-work review separately notes that *"high-altitude matching... works reasonably well, but at low altitudes (25 m) the viewpoint differs drastically, making roofs, facades, and vegetation inconsistent with satellite imagery."* The project operates at ≤1 km AGL — which is the *high-altitude* regime in the paper's terminology — making RGB sat-matching the appropriate technique class. The paper's CPU-only winning method (LiDAR heightmap-gradients + clustered particle filter) is **not** transferable to our hardware: our project has no LiDAR.
|
||||
- **Source**: Source #28
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Implementer / Engineer + Domain expert
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: SQ1, SQ5 (failure modes), SQ2 (canonical pipeline)
|
||||
- **Fit Impact**: **disambiguates a potentially-disqualifying lesson** — the CTU paper's "RGB sat-matching is unreliable" finding does NOT disqualify our approach because the failure was caused by low-altitude viewpoint mismatch, which our 1 km AGL regime does not have. This must be cited explicitly in `solution_draft01` to pre-empt the natural objection from anyone who reads the paper. Separately, the CTU paper's specific lessons are still binding: VIO degrades catastrophically without IMU vibration isolation; magnetometer is unreliable near steel/concrete; "ability to recover from periods of high uncertainty and re-localize" matters more than instantaneous RMSE — this last lesson is a direct architectural input for AC-NEW-2 / AC-NEW-8.
|
||||
|
||||
### Fact #16 — RTAB-Map and ORB-SLAM3 both fail beyond 1 km / above 2 m/s flight in the SPRIN-D environment; our cruise profile (≤17 m/s, kilometers between satellite anchors) explicitly excludes both as primary candidates
|
||||
- **Statement**: The SPRIN-D paper states: *"We tested state-of-the-art visual SLAM systems such as RTAB-Map and ORB-SLAM3 in a high-fidelity simulator, and found that both performance degraded significantly in a long-range scenario (beyond 1 km), as their memory and compute demands grow with the size of the environment. Moreover, RTAB-Map was unable to maintain quality odometry in faster flight speeds (beyond 2 m/s), while ORB-SLAM3 suffered from tracking loss in textureless areas."*
|
||||
- **Source**: Source #28
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Implementer / Engineer (component selection for C1)
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: SQ1, SQ3+SQ4 component C1 (VO/VIO), SQ5 (failure modes)
|
||||
- **Fit Impact**: **prunes the C1 candidate landscape** — RTAB-Map and ORB-SLAM3 should not be pursued as C1 leads. Plausible C1 leads remain: VINS-Mono / VINS-Fusion / OpenVINS / OKVIS2 / DROID-SLAM / DPVO / pure VO baseline (KLT + RANSAC homography). NGPS (Fact #14) uses `ap_vips` = OpenVINS-class VIO — confirming an aligned community choice. Final C1 selection happens in SQ3+SQ4.
|
||||
|
||||
### Fact #17 — DSMAC + TERCOM lineage: pre-cached scene matching for downward-looking navigation is a 40+ year deployed technique class with documented sub-10 m terminal accuracy
|
||||
- **Statement**: DSMAC (Digital Scene Matching Area Correlator) is an autonomous missile-guidance system based on area correlation of sensed downward-camera ground scenes against pre-stored reference imagery (often satellite reconnaissance). It achieves 3–10 m terminal accuracy by correlating buildings, road intersections, and distinctive terrain landmarks. Tomahawk: TERCOM (radar altimeter + DEM) for mid-flight + DSMAC for terminal guidance reduces CEP from ~30 m to "only meters". Documented combat record: 1991 Gulf War, >80% of 280 launched Tomahawks hit target. Recent miniaturisation: Destinus Ruta (300 km strike-class) is integrating UAV Navigation's (Spanish, Grupo Oesía) DSMAC-class system, validated in Ukrainian combat conditions including GNSS-denied / jamming / spoofing.
|
||||
- **Source**: Source #36, Source #27
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Domain expert + Decision-maker
|
||||
- **Confidence**: ✅ for the lineage and Tomahawk performance numbers (DTIC + open-source); ⚠️ for the Ruta-specific "DSMAC operating principle" inference (Defense Express analyst inference, not vendor disclosure)
|
||||
- **Related Dimension**: SQ1 (lineage), SQ8 (baseline accuracy expectations for AC-1.1.1 80 m / AC-NEW-4 false-position budget)
|
||||
- **Fit Impact**: **establishes baseline accuracy expectations** — the technique class has documented sub-10 m accuracy in the cruise-missile-terminal regime. Our budget (AC-1.1.1: <80 m at 1 km AGL with ≥0.5 m/px tiles) is loose by comparison, indicating that the AC budget is *not* aggressive against the technique-class baseline — it is aggressive against the Jetson Orin Nano + 8-h-continuous + 25 W envelope. **Implication for AC-NEW-4**: claiming P(error >500 m) <0.1% per flight is consistent with the DSMAC-lineage class; an honestly-reported failure rate at this level is realistic, not unprecedented.
|
||||
|
||||
### Fact #18 — Hierarchical Image Matching (arXiv 2506.09748, June 2025) is a current academic SOTA pipeline for our exact problem, but uses DINOv2 — a heavyweight foundation model that must be benchmarked under our 25 W / 8 GB Jetson envelope before any selection
|
||||
- **Statement**: 2025 academic SOTA pipeline structure: (1) image retrieval module (off-the-shelf, optimal-transport feature aggregation); (2) Semantic-Aware and Structure-Constrained Matching Module (SASCM) using **DINOv2** features + 4D correlation tensor + SoftMNN + 4D conv; (3) lightweight fine-grained matching module for pixel-level. Constructs UAV absolute visual localization without VIO/relative-localization dependence (retrieval-and-matching only). Evaluation on AerialVL + their own CS-UAV dataset claims superior accuracy under cross-source and cross-temporal variation.
|
||||
- **Source**: Source #29
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Implementer / Engineer + Domain expert
|
||||
- **Confidence**: ✅ for pipeline structure and method; ⚠️ for "superior" claim (single-paper benchmark; AerialExtreMatch evaluates 16 methods with broader rigor — Source #34 is the better cross-method ranker); ❓ for Jetson-Orin-Nano runtime (no published number)
|
||||
- **Related Dimension**: SQ1 (academic SOTA), C2 (VPR), C3 (cross-domain registration), SQ5 (foundation-model-on-Jetson failure mode)
|
||||
- **Fit Impact**: **academic-SOTA snapshot, candidate template** — the retrieval → semantic-aware coarse → fine-grained pipeline is a candidate template for our C2+C3, but DINOv2 introduces a Jetson-deployment risk that must be quantified before commitment. Candidate-level decision: include DINOv2-based pipelines (AnyLoc, BoQ, this paper's SASCM) in the C2/C3 candidate matrix with mandatory MVE on Jetson Orin Nano under our exact frame size and 3 fps cadence. Reject DINOv2 if total inference latency cannot be brought under (400 ms - other-stages budget) at INT8 / fp16. Per Source #28 lesson, classical matchers (LightGlue+SuperPoint as in NGPS) should also be in the matrix as the "simple baseline / known-Jetson-runnable" option.
|
||||
|
||||
### Fact #19 — AerialExtreMatch (2025) is the academic benchmark our C2+C3 candidate matrix must publish numbers against, with 32 difficulty-stratified cells exposing exactly the cross-source / cross-pitch / cross-scale failure modes our project will face
|
||||
- **Statement**: AerialExtreMatch publishes (a) 1.5 M synthetic train pairs (RGB+depth, diverse UAV/satellite viewpoints); (b) ~30,000 evaluation pairs in **32 difficulty levels** stratified by overlap (4 bins: <20%, 20–40%, 40–60%, >60%), pitch difference (4 bins: 50–55°, 55–60°, 60–65°, 65–70°), and scale variation (2 bins: 1–2×, >2×); (c) a real-world UAV-localization split captured with DJI M300 RTK + H20T against UAV-derived orthomosaic/DSM AND lower-quality satellite maps. The benchmark evaluates 16 representative detector-based and detector-free image matching methods.
|
||||
- **Source**: Source #34
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Domain expert + Implementer
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: SQ1 (academic landscape), SQ7 (datasets), C2 (VPR), C3 (cross-domain registration)
|
||||
- **Fit Impact**: **defines the C2/C3 evaluation matrix** — every C2/C3 candidate going into `solution_draft01` must report numbers on AerialExtreMatch's 32 difficulty cells, with at least the high-pitch (65–70°) and high-scale (>2×) cells representing our worst-case (UAV vs satellite tile geometry mismatch + ortho-rectification residual). The dataset's real-world UAV-localization split with both UAV-orthomosaic AND satellite-map references mirrors our project's offline-cache-tile semantics directly.
|
||||
|
||||
### Fact #20 — DARPA FLA + USAF SBIR establish the US-defense-program tailwind, but do not directly validate the project's specific regime (fixed-wing, ~1 km AGL, sat-tile basemap, 8-h endurance)
|
||||
- **Statement**: DARPA Fast Lightweight Autonomy (FLA) program ran 2015–2018 (Phase 1 Florida 2017; Phase 2 Georgia 2018; complete). Focused on small quadcopter autonomy at ≤20 m/s through cluttered indoor/outdoor environments using onboard cameras + LIDAR + sonar + IMU, no GPS / datalink / pilot. A 2025 retrospective (arXiv 2504.08122) reviews FLA testing methodology and Phase 1 results. A 2025 USAF SBIR Phase II solicitation (Sweetspot ID `7946c818-409f-5b31-8f06-554466071d83`) is requesting visual position and navigation capability for sUAS in GPS-denied environments — confirming the regulatory + funding environment is currently active for this category in 2025.
|
||||
- **Source**: Source #35
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Decision-maker + Domain expert
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: SQ1 (defense-program lineage)
|
||||
- **Fit Impact**: **context only, no direct candidate gain** — FLA pre-dates the project's specific regime by 8 years, focused on a different platform (multirotor) and altitude (low-altitude obstacle avoidance, not 1 km AGL nadir-camera satellite-anchor). Useful only to establish lineage and context. The USAF SBIR datapoint is more directly relevant: confirms that an active US-defense-funded need exists for sUAS visual position + navigation in GPS-denied environments — i.e., the project's market exists outside Ukraine.
|
||||
|
||||
---
|
||||
|
||||
## SQ1 — Conclusions (working summary, will be re-checked at Step 7.5)
|
||||
|
||||
### Existing-systems landscape (5 named-and-evidenced peer / adjacent systems)
|
||||
|
||||
| System | Class | Operational regime | Closest match dimension | Closest mismatch dimension | Status as evidence |
|
||||
|---|---|---|---|---|---|
|
||||
| **Twist Robotics OSCAR** (UA) | Deployed Ukrainian peer | Combat-deployed, fixed-wing-class, GPS-denied vision-nav | **Same architecture, same threat environment** | Altitude / endurance / FC / accuracy contract not publicly specified | Closest peer for "feasibility floor" |
|
||||
| **Auterion Artemis** | Production COTS one-way attack drone | Shahed-class, 1000-mile range, 40 kg warhead, Ukraine-validated GPS-denied nav | Same architectural pattern (Skynode + Visual Navigation + terminal guidance) | One-way attack vs reusable; no covariance/source-label contract published | Closest production reference architecture |
|
||||
| **Vantor Raptor (Guide / Sync / Ace)** | Production COTS software suite | Vision-based GPS replacement on existing drone camera + Vivid Terrain 3D basemap | Visual-position software pattern | Vendor-managed sat-tile basemap is not the project's Azaion Suite Satellite Service; no AC-NEW-4 / AC-1.4 contract | Closest commercial peer for "build-vs-buy" framing |
|
||||
| **snktshrma/ngps_flight (NGPS, ArduPilot GSoC 2024)** | Open-source research prototype | LightGlue+SuperPoint+UKF+`VISION_POSITION_ESTIMATE` to AP | **Same component split, same FC family** | GSoC prototype, not production; no spoof defence; no covariance honesty | **Closest open-source pipeline match — explicit architectural template** |
|
||||
| **CTU Prague SPRIN-D winner** | Academic / competition | Multirotor, ≤25 m AGL, LiDAR + heightmap gradient + particle filter on CPU | "Recover-from-uncertainty > low-instantaneous-RMSE" lesson; VIO discipline | LiDAR-required, low-altitude regime, no sat-tile basemap | Architectural-pattern reference + cautionary tale |
|
||||
| **Destinus Ruta + UAV Navigation** | Production miniaturised cruise missile | 300 km strike, DSMAC-class, Ukraine-combat-validated | Pre-cached basemap + visual matching + autopilot ingestion | One-way attack, terminal guidance, no covariance contract | Shows DSMAC-class miniaturised into UAV tier |
|
||||
|
||||
### Per-perspective coverage
|
||||
|
||||
| Perspective | Facts supporting | Saturation status |
|
||||
|---|---|---|
|
||||
| **Implementer / Engineer** | Fact #14 (NGPS), Fact #16 (SLAM failure modes), Fact #18 (DINOv2 risk) | Saturated for SQ1 — deeper component-level deep-dives go to SQ3/SQ4 |
|
||||
| **Practitioner / Field (Ukraine)** | Fact #11 (OSCAR), Source #37 (~70% UAV losses to EW), Source #27 (Ruta + UAV Navigation Ukraine combat validation) | Saturated for SQ1 |
|
||||
| **Domain expert / Academic** | Fact #18 (Hierarchical Matching SOTA), Fact #19 (AerialExtreMatch benchmark), Fact #15 (SPRIN-D regime distinction) | Saturated for SQ1 — academic SOTA benchmarking handed off to SQ3/SQ4 + SQ7 |
|
||||
| **Contrarian / Devil's advocate** | Fact #15 (low-altitude RGB matching unreliable lesson), Fact #16 (RTAB-Map / ORB-SLAM3 disqualified), Fact #18 (DINOv2-on-Jetson risk) | Saturated for SQ1 |
|
||||
| **Decision-maker / Business** | Fact #12 (production-ready Auterion), Fact #13 (commercial Vantor build-vs-buy framing), Fact #20 (USAF SBIR market context) | Saturated for SQ1 |
|
||||
|
||||
### Architectural conclusions for `solution_draft01`
|
||||
|
||||
1. **Build-vs-buy stance**: build. Vantor Raptor and Auterion Visual Navigation are commercially superior on hardening + integration but neither exposes the covariance honesty contract (AC-NEW-4) nor uses the project-specified Azaion Suite Satellite Service tile cache (AC-2.x); both are dual-use export risks for the Ukrainian battlefield. NGPS (Fact #14) is the open-source architectural template to learn from but is a GSoC research prototype lacking production hardening, spoof defence, and the covariance-honesty contract. Architectural conclusion: build with NGPS as the template, with project-specific contracts (AC-NEW-4, AC-1.4, AC-NEW-7) and per-FC adapter (SQ6 conclusion) layered on top.
|
||||
2. **Differentiation from OSCAR (Twist Robotics)** must be made explicit in `solution_draft01`: (a) honest covariance contract per AC-NEW-4; (b) explicit `{satellite_anchored, visual_propagated, dead_reckoned}` source-label contract per AC-1.4; (c) AC-NEW-7 cache-poisoning safety budget on tile write-back; (d) ArduPilot Plane + iNav both supported per project's revised AC-4.3.
|
||||
3. **Pipeline canonicalness**: the C1+C2+C3+C4+C5+C8 split is canonical (NGPS + the 2025 hierarchical-matching paper + SPRIN-D winner all use the same shape; only the specific algorithm choices differ). SQ2 will sanity-check this against one more pipeline-survey paper, but this is essentially a low-risk question now.
|
||||
4. **Component-pruning** carried into SQ3/SQ4:
|
||||
- C1: **prune RTAB-Map and ORB-SLAM3** as primary candidates per Fact #16. Carry: VINS-Mono / VINS-Fusion / OpenVINS / OKVIS2 / DROID-SLAM / DPVO / pure VO baseline.
|
||||
- C2/C3: **mandatorily benchmark** any DINOv2-based candidate (AnyLoc, BoQ, SASCM-style) against AerialExtreMatch at our pitch / scale / overlap regime AND against Jetson Orin Nano latency budget (per Fact #18). Maintain LightGlue+SuperPoint as the "simple-baseline / known-Jetson-runnable" option per NGPS precedent.
|
||||
- C8 transport: NGPS uses `VISION_POSITION_ESTIMATE`. SQ6 picked `GPS_INPUT`. Re-examine the trade-off in design phase, but SQ6's selection stands for the research draft.
|
||||
5. **Lessons from SPRIN-D winner that must propagate to `solution_draft01`**:
|
||||
- "Ability to recover from periods of high uncertainty and re-localize" > "low instantaneous RMSE" — directly informs AC-NEW-2 / AC-NEW-8.
|
||||
- VIO requires mechanically-decoupled IMU; this is a hardware-integration constraint, not a software issue.
|
||||
- Magnetometer is unreliable near steel/concrete; sensor fusion of heading sources is essential.
|
||||
- "No single sensor can be fully relied upon" — directly supports our IMU+camera+sat-tile multi-source posture.
|
||||
|
||||
### Open follow-ups (deferred to later sub-questions)
|
||||
|
||||
- **(SQ8)** Independent verification of OSCAR's "fully resistant to spoofing/jamming" claim — if available. Otherwise, Twist Robotics's claim remains a vendor-only signal.
|
||||
- **(SQ8)** Vantor Raptor and Auterion Visual Navigation's covariance reporting behaviour — for benchmarking AC-NEW-4 compliance.
|
||||
- **(SQ3+SQ4 / C2)** AnyLoc / BoQ / DINOv2-VLAD / MixVPR / EigenPlaces / NetVLAD on AerialExtreMatch for cross-source aerial — already in C2 search plan; SQ1 just confirmed they're the right candidate set.
|
||||
- **(SQ3+SQ4 / C3)** LightGlue / LoFTR / RoMa / DKM / MASt3R + classical SIFT+RANSAC + XFeat on AerialExtreMatch — already in C3 search plan; SQ1 confirms shape.
|
||||
- **(SQ7)** AerialExtreMatch + AerialVL + CS-UAV + RealUAV/SAVL + UAV-VisLoc as the dataset shortlist for our cross-validation — confirmed by SQ1 hits.
|
||||
|
||||
### Boundary check: SQ1 is saturated
|
||||
|
||||
Saturation signals observed: 4 perspectives saturated, ≥3 high-confidence facts per perspective, last 3 search rounds (Anduril Iris detail probe, ArduPilot prior-art probe, DSMAC lineage probe) yielded only one new substantive datapoint (NGPS) and confirmed already-known patterns. No unresolved contradictions. Per `references/source-tiering.md` "Search saturation rule" → SQ1 is closed.
|
||||
|
||||
---
|
||||
|
||||
## SQ2 — Canonical pipeline decomposition (sanity-check)
|
||||
|
||||
### Fact #21 — The canonical pipeline for offline-cache visual geo-localization is two-stage: global VPR retrieval, then local alignment (image matching → pose)
|
||||
- **Statement**: Source #38 (Skoltech aerial-VPR survey) defines the field's canonical pipeline verbatim: "Visual geolocalization can be implemented through various methods, typically relying on a pre-built database of images with known locations. This approach generally involves two stages: global localization (or Visual Place Recognition, VPR) and local alignment. Global localization involves identifying the nearest frame from the database (Image Retrieval), while local alignment determines the precise position using the selected frame." Source #42 (NUDT 2026 absolute-VL survey) names the same shape "**retrieval → matching → pose-estimation hierarchical framework**" and explicitly contrasts it against three rejected alternatives: (a) relative-only VIO/SLAM (cumulative error), (b) end-to-end direct localization (poor generalization), (c) map-free localization (scene-dependent). Source #39 (U.Maine cross-view survey) traces the same lineage from 2003 pixel-wise template-matching → 2013 hand-engineered features → 2017 CNN/triplet-loss → 2018+ Siamese/GAN → 2022+ Transformer → 2023 DINOv2-class. Source #41 (AnyVisLoc benchmark) implements this hierarchy as: image retrieval (rough) → image matching (2D-2D) → DSM-lift to 3D → PnP+RANSAC, with **Top-N re-rank by inlier count** as a critical fourth stage between matching and pose.
|
||||
- **Source**: Source #38, Source #39, Source #41, Source #42
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: Architects of `solution_draft01`
|
||||
- **Confidence**: ✅ (four independent surveys/benchmarks converge)
|
||||
- **Related Dimension**: SQ2, C2 (VPR), C3 (cross-domain matching), C4 (pose estimation)
|
||||
- **Fit Impact**: **confirms** the project's C1–C10 decomposition is canonical for the **C2 → C3 → C4** chain. The component split is not novel; the project's contribution is the **integration discipline** (covariance honesty AC-NEW-4, source-label contract AC-1.4, offline-cache safety AC-NEW-7) layered on top. **Augment** the existing decomposition with an explicit "Top-N re-rank by inlier count" stage between C3 and C4 (currently implicit).
|
||||
|
||||
### Fact #22 — AdHoP (Adaptive Homography Preconditioning) is a method-agnostic post-matching refinement loop that improves translation accuracy by ~30% average and up to 63% for previously-underperforming methods, at the cost of a second matching pass
|
||||
- **Statement**: Source #40 (OrthoLoC benchmark, Sep 2025): from initial 2D-2D query↔orthophoto correspondences, estimate a homography H via DLT+RANSAC, warp the orthophoto with H to better match the query's perspective (reducing residual perspective gap), re-match in this warped frame, then map the new correspondences back to the original orthophoto via H⁻¹, lift to 3D using DSM, and run PnP+RANSAC + Levenberg-Marquardt refinement. Accept the AdHoP-refined pose only if reprojection error decreases vs. the non-refined pose. **Quantitative effects** (16,425 images, 47 locations, 1m-1° threshold): GIM+DKM 75.4% recall (best); AdHoP-refined methods see ~30% average matching improvement, ~20% translation/rotation error reduction; for previously-underperforming methods AdHoP yields up to 95% matching improvement (XFeat*) or 63% translation reduction (DKM); for RoMa, AdHoP lifts 1m-1° recall by +23 points (54.6% → 77.6%-class). **Cross-domain regime** (war-zone-equivalent: scene change between query and reference): translation error increases ~3× when only the visual modality differs, ~7× when both visual and structural (DSM) gaps exist (0.16 m → 1.12 m for GIM+DKM+AdHoP). **Method-agnostic** — works on top of any 2D-2D matcher.
|
||||
- **Source**: Source #40
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System architects + C3/C4 implementers
|
||||
- **Confidence**: ✅ for headline numbers (single-paper, but published dataset + open code + reproducible per repo)
|
||||
- **Related Dimension**: SQ2 (new sub-stage), C3 (matcher), C4 (pose), SQ5 (cross-domain failure mode)
|
||||
- **Fit Impact**: **adds a new sub-stage** between C3 and C4. Decision for `solution_draft01`: include AdHoP-class refinement as an **optional** stage gated on Jetson Orin Nano latency budget — if (single-pass match latency × 2) + homography estimation + reprojection check fits under (400 ms - other-stages), include it; otherwise reserve as offline-replay-time refinement. Cross-domain 3× translation-error penalty is a **direct AC-NEW-4 calibration input** — companion-side covariance must inflate proportionally when scene-change detection (deferred to SQ8) flags a stale tile.
|
||||
|
||||
### Fact #23 — 6-DoF aerial-to-satellite localization requires DSM (Digital Surface Model) elevation data; without DSM, the system collapses to 3-DoF (position + 1 rotation) or must compute attitude purely from IMU/VIO
|
||||
- **Statement**: Source #40 OrthoLoC explicitly: "Our pipeline matches the query image with the DOP, lifts the matched 2D points in DOP to 3D using the DSM, and then estimates the camera pose using PnP and RANSAC." Without the DSM lift, the matcher produces 2D↔2D correspondences that constrain a homography (which encodes 3-DoF for a planar scene + planar camera) but **not** the full 6-DoF camera pose. Source #41 AnyVisLoc independently confirms by measuring: aerial-photogrammetry map (with paired DSM at 0.94 m/px) achieves 74.1% A@5m; satellite map (with ALOS 30 m DSM) achieves only 18.5% A@5m — a 4× accuracy collapse driven by DSM coarseness. The project's offline cache from the Azaion Suite Satellite Service is currently specified as **2D ortho tiles only** (no DSM commitment in restrictions.md or AC). **Three architectural responses** are available: (a) **3-DoF acceptance** — fix attitude from IMU/VIO, treat the matcher output as a homography-only constraint, ignore DSM; sacrifices the up-to-2× higher accuracy reported when DSM is present, but stays within current cache contract; (b) **Request DSM tiles from the Suite Sat Service** — adds C2 cache schema work + a Suite Sat Service contract change; preserves 6-DoF accuracy; (c) **IMU/VIO-only attitude + 2D-2D matching translation** — same as (a) but explicitly contracts the IMU/VIO module to provide attitude with σ ≤ 5° (per Fact #24); operationally identical to (a), differs only in how the contract is written.
|
||||
- **Source**: Source #40, Source #41
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System architects + Suite Sat Service stakeholder + AC owner
|
||||
- **Confidence**: ✅ for the architectural claim; ✅ for the 4× accuracy collapse number
|
||||
- **Related Dimension**: SQ2 (decomposition), C2 (cache schema), C3 (matcher output contract), C4 (pose), C5 (estimator), C6 (IMU/VIO contract), AC-1.1 / AC-1.1.1 (accuracy budget)
|
||||
- **Fit Impact**: **architectural decision required, surfaced for user.** The current restrictions.md (no DSM commitment) implicitly forces option (a) or (c). The accuracy budget AC-1.1.1 (≤80 m at 1 km AGL) is loose enough that 3-DoF + IMU-attitude almost certainly satisfies it on a per-frame basis (per Fact #21 and DSMAC-class lineage in Fact #17), but **requires explicit acknowledgement** in the architecture before commitment. **Proposed default** for `solution_draft01`: option (c) — fix attitude from IMU/VIO with documented σ ≤ 5° contract on yaw, σ ≤ 5° on pitch (per Fact #24), translation from 2D-2D matching + camera pose. Flag option (b) as a "Suite Sat Service follow-up" if 6-DoF accuracy ever becomes a hard requirement.
|
||||
|
||||
### Fact #24 — IMU-derived yaw and pitch priors with σ ≤ 5° are required for the matching+PnP stack to hit benchmark accuracy; σ ≥ 10° causes 2–4% A@5m drops, σ ≥ 30° causes ≥4% drops, σ ≥ 60° causes 25.7% drops
|
||||
- **Statement**: Source #41 AnyVisLoc systematically perturbs yaw and pitch priors and measures localization accuracy collapse. Yaw: σ = 5° → no impact; σ = 10° → −1.9% A@5m; σ = 30° → −4.1%; σ = 50° → −13.7%; σ = 60° → −25.7%. Pitch: σ < 5° → no impact; σ ≥ 7° → 1–5% drops. The benchmark is conducted at low altitude (30–300 m AGL) with 20–90° pitch range; lessons transfer to our 1 km AGL nadir-camera regime in the **direction** but the magnitudes may be lower at 1 km AGL because nadir geometry is less yaw-sensitive than oblique. Conservatively adopting the benchmark numbers gives a hard contract: **IMU/VIO must deliver yaw with σ ≤ 5° and pitch with σ ≤ 5° to the matcher** (1σ, not 95%, since the benchmark is single-σ). Pitch is naturally tighter on a nadir-fixed camera (mechanically constrained); yaw is the binding constraint and is the typical IMU/magnetometer failure mode (per SPRIN-D lesson Fact #15).
|
||||
- **Source**: Source #41
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System architects + C1 (VIO) implementer + C5 (estimator) implementer
|
||||
- **Confidence**: ✅ for the AnyVisLoc numbers; ⚠️ for direct transfer to 1 km AGL nadir regime (magnitudes likely smaller at our altitude/pitch — direction is conservative)
|
||||
- **Related Dimension**: SQ2 (sensor-prior contract), C1 (VIO output contract), C5 (estimator), C6 (IMU)
|
||||
- **Fit Impact**: **architectural contract** for `solution_draft01`: the C1 module's published contract to the C2/C3 stack is yaw σ ≤ 5° AND pitch σ ≤ 5°. Magnetometer-only yaw is **insufficient** by the SPRIN-D lesson (Fact #15) — VIO must contribute. **Adds a constraint** that flows back to the C6 IMU integration: IMU mechanical isolation per SPRIN-D Fact #15 is required; magnetometer + GPS-yaw startup alignment at the airbase (before take-off, while real GPS is healthy) is part of the boot sequence.
|
||||
|
||||
### Fact #25 — Top-N re-ranking by inlier count is the dominant accuracy/cost trade-off; pure-matching-without-retrieval is catastrophic (A@5m collapses from 62.2% to 34.3% with the same matcher)
|
||||
- **Statement**: Source #41 AnyVisLoc and Source #38 Skoltech survey both quantify the value of retrieval as a search-space reducer for matching. Source #41 explicitly: "Top-N re-rank by inlier count is the best accuracy/cost trade-off" → 62.2% A@5m at 0.8 s/frame on RTX 3090. **Without retrieval** (pure exhaustive matching against the cache): 34.3% A@5m — i.e., almost **half** the accuracy at infeasible compute. Source #38 measures sparse-VPR re-ranking specifically: AnyLoc descriptor + SuperGlue re-rank on top-100 candidates = 15–25 s/frame on RTX 3090 (catastrophic for our 400 ms budget); LightGlue re-rank ≈ 1 s/frame (still over budget); SelaVPR re-rank < 0.1 s/frame (in-budget on RTX 3090, must be re-tested on Jetson Orin Nano). **Re-ranking budget** = (frame budget) − (descriptor extraction) − (initial top-N retrieval) − (matcher pose estimation) − (AdHoP if included).
|
||||
- **Source**: Source #38, Source #41
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: System architects + C2 implementer
|
||||
- **Confidence**: ✅ (two-source convergence on the qualitative claim; quantitative numbers are RTX-3090-specific and must be Jetson-MVE'd)
|
||||
- **Related Dimension**: SQ2 (pipeline structure), C2 (VPR), C3 (matcher), SQ3+SQ4 (Jetson MVE)
|
||||
- **Fit Impact**: **mandates** Top-N re-rank by inlier count as a stage in `solution_draft01`. Trade-off Top-N value (typical N=5–20 in literature) goes to SQ3+SQ4 candidate matrix, not SQ2.
|
||||
|
||||
### Fact #26 — High-accuracy SOTA models (AnyLoc + SuperGlue + RoMa-class) are NOT viable on Jetson Orin Nano under the 400 ms p95 budget; lightweight VPR (MixVPR / SALAD / SelaVPR-class) + lightweight matchers (LightGlue / XFeat-class) are the only candidates that survive a basic latency pre-screen
|
||||
- **Statement**: Two independent runtime measurements on RTX 3090 (≥10× faster than Jetson Orin Nano in dense matrix ops): Source #38 — AnyLoc descriptor calculation 0.37–0.84 s/frame (huge ViT-G DINOv2); SuperGlue re-rank 15–25 s/frame on top-100; LightGlue re-rank ~1 s/frame; SelaVPR re-rank < 0.1 s/frame. Source #41 — RoMa dense matcher 659 ms/frame; SP+LightGlue+GIM sparse 105 ms/frame; ratio = 6.3×. **Memory**: AnyLoc descriptors = 2.3–13.9 GB for 4–7k tiles (out of 8 GB Jetson Orin Nano envelope before model weights); SelaVPR descriptors < 0.2 GB. Pre-screen conclusion: AnyLoc / SuperGlue / RoMa-class are **disqualified** on the Jetson Orin Nano at 3 fps unless heavy quantization (INT8) reduces them ≥10×, which is not yet established for our latency target on this hardware. Surviving candidates from the literature: **VPR**: MixVPR, SALAD, SelaVPR, EigenPlaces, NetVLAD-class; **matchers**: LightGlue, XFeat, XFeat*, SP+LightGlue. **Disqualification is preliminary** — final go/no-go happens at SQ3+SQ4 with on-Jetson MVE per `references/mode-A-mve-rules.md`.
|
||||
- **Source**: Source #38, Source #41
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: C2 + C3 implementer; SQ3+SQ4 candidate-matrix author
|
||||
- **Confidence**: ✅ for RTX-3090 numbers; ⚠️ for direct Jetson translation (Jetson Orin Nano AI score is well-published; ratio is conservative)
|
||||
- **Related Dimension**: SQ2 (Jetson budget feasibility), SQ3+SQ4 (candidate pre-screen), SQ5 (foundation-model-on-edge failure mode), C2, C3, C7 (Jetson runtime)
|
||||
- **Fit Impact**: **prunes the SQ3+SQ4 candidate matrix BEFORE expensive Jetson MVE.** Candidates entering SQ3+SQ4 with mandatory Jetson MVE: (C2 VPR) MixVPR, SALAD, SelaVPR, EigenPlaces, NetVLAD; (C3 matcher) LightGlue, XFeat, XFeat*, SP+LightGlue. Candidates that need Jetson INT8 quant before they earn an MVE slot: AnyLoc, BoQ, DINOv2-VLAD (must demonstrate INT8 build path with vendor-validated accuracy preservation). Candidates pruned outright: RoMa dense, SuperGlue, MASt3R (latency).
|
||||
|
||||
### Fact #27 — A 20% covisibility floor between query frame and reference tile is required for localization to succeed; below it, ALL methods fail regardless of matcher quality
|
||||
- **Statement**: Source #40 OrthoLoC: "When the covisibility between the UAV image and the orthographic geodata is too small (less than ~20%), the localization fails for all methods regardless of matcher quality." This is a geometric floor, not a method-specific limit. The implication for the project: any tile-cache design that allows a query to fall outside 20% covisibility with the **best available** cached tile must also include a **runtime covisibility-check + graceful degrade** to `visual_propagated` mode (per AC-1.4 source label). This is a runtime condition, not a one-time setup parameter.
|
||||
- **Source**: Source #40
|
||||
- **Phase**: Phase 2
|
||||
- **Target Audience**: C2 (cache scheduler) + C5 (estimator) + AC-1.4 owner
|
||||
- **Confidence**: ✅
|
||||
- **Related Dimension**: SQ2 (boundary condition), C2 (tile cache), C5 (estimator state machine), AC-1.4
|
||||
- **Fit Impact**: **adds a runtime invariant** to `solution_draft01`: tile selection must guarantee ≥20% covisibility OR explicitly emit the `visual_propagated` source label per AC-1.4 with covariance widened per AC-NEW-4. This becomes a hard constraint on the C2 cache schema (must support tile-extent metadata) and a runtime check before invoking C3 matcher.
|
||||
|
||||
---
|
||||
|
||||
## SQ2 — Conclusions (working summary, will be re-checked at Step 7.5)
|
||||
|
||||
### Pipeline-component coverage table (existing C1–C10 vs. survey-listed components)
|
||||
|
||||
| Survey/benchmark canonical stage | Project component (current) | Coverage status | Required action |
|
||||
|---|---|---|---|
|
||||
| Image retrieval (global VPR) | **C2 — Visual Place Recognition** | ✅ covered | No change |
|
||||
| Re-ranking (top-N inlier-based) | (currently implicit, inside C2 or C3) | ⚠️ implicit | **Promote to explicit sub-stage** (`C2.5` or `C3.0`) in `solution_draft01` |
|
||||
| Local image matching (2D-2D, sparse or dense) | **C3 — Cross-domain registration** | ✅ covered | Add Top-N re-rank-by-inlier-count requirement |
|
||||
| AdHoP-style perspective preconditioning | (not represented) | ❌ missing | **Add as optional sub-stage** between C3 and C4, gated on Jetson latency budget |
|
||||
| 2D-3D lift via DSM | (not represented; current cache is 2D ortho only) | ❌ architectural decision required | **Decision required from user** — see below |
|
||||
| Pose estimation (PnP + RANSAC + LM) | **C4 — Pose estimation** | ✅ covered | No change |
|
||||
| State estimator / fusion (UKF / ESKF / MSCKF / factor graph) | **C5 — Estimator / fusion** | ✅ covered | Augmented with covariance-honesty contract from AC-NEW-4 |
|
||||
| IMU + VIO contract | **C1 — VO/VIO** + **C6 — IMU integration** | ✅ covered | Add yaw σ ≤ 5°, pitch σ ≤ 5° hard contract from Fact #24 |
|
||||
| Tile cache + scheduler | **C2 — VPR tile cache** + **C9 — Cache hygiene** | ✅ covered | Add 20% covisibility runtime invariant (Fact #27) |
|
||||
| Anti-spoof / source-switch | **C7 — Spoof detection** + **C8 — FC adapter** | ✅ covered | Already addressed in SQ6 |
|
||||
| Health monitoring / safety | **C10 — Safety / health monitoring** | ✅ covered | Already addressed |
|
||||
|
||||
### Architectural decisions surfaced (require user resolution before SQ3+SQ4 starts)
|
||||
|
||||
1. **DSM dependency on the Suite Sat Service tile cache** (per Fact #23). Three options:
|
||||
- **(a) 3-DoF acceptance** — accept that without DSM, only position is recovered from matching; attitude is fixed by IMU/VIO with no satellite-tile cross-check. Lowest project scope. Requires AC budget verification (likely passes AC-1.1.1).
|
||||
- **(b) Request DSM tiles** — Suite Sat Service contract change. Highest accuracy. Adds ~1 cycle to delivery. Recommended if 6-DoF accuracy ever becomes a hard AC.
|
||||
- **(c) IMU/VIO-attitude + 2D-2D matching translation** — operationally identical to (a) but contracts the IMU/VIO module explicitly with σ ≤ 5° yaw / pitch (Fact #24).
|
||||
- **Recommended default**: **(c)** — explicit IMU/VIO contract; fall back to (b) if AC tightens.
|
||||
|
||||
2. **AdHoP refinement loop** (per Fact #22). Three options:
|
||||
- **(a) Always-on** — included in every frame; Jetson budget must accommodate 2× matching latency.
|
||||
- **(b) Conditional** — only when initial reprojection error exceeds a threshold; gated on per-frame budget.
|
||||
- **(c) Off (initial release)** — relegate to offline-replay refinement.
|
||||
- **Recommended default**: **(b) Conditional** — fits within latency variance budget while capturing the cross-domain accuracy gain.
|
||||
|
||||
3. **Top-N re-rank promotion to explicit pipeline sub-stage** (per Fact #25). Recommendation: promote to a named sub-stage in `solution_draft01` with N as an SQ3+SQ4 hyperparameter sweep target.
|
||||
|
||||
### Component-pruning carried into SQ3+SQ4
|
||||
|
||||
- **C2 candidates entering SQ3+SQ4 with mandatory Jetson MVE**: MixVPR, SALAD, SelaVPR, EigenPlaces, NetVLAD.
|
||||
- **C2 candidates entering SQ3+SQ4 conditional on INT8 quantization path**: AnyLoc, BoQ, DINOv2-VLAD.
|
||||
- **C2 candidates pruned**: SuperGlue-as-reranker (latency).
|
||||
- **C3 candidates entering SQ3+SQ4 with mandatory Jetson MVE**: LightGlue, XFeat, XFeat*, SP+LightGlue (NGPS template).
|
||||
- **C3 candidates pruned**: RoMa, MASt3R, DKM (dense matcher latency on Jetson).
|
||||
- **C3 candidates as "AerialExtreMatch reference points" only, NOT for production**: GIM+DKM, GIM+LightGlue (per Source #40, used as accuracy benchmark only).
|
||||
|
||||
### Boundary check: SQ2 is saturated
|
||||
|
||||
Saturation signals observed: (a) four independent surveys/benchmarks (Skoltech aerial-VPR survey, U.Maine cross-view survey, OrthoLoC benchmark, AnyVisLoc benchmark, NUDT 2026 absolute-VL survey) converge on the **same** "retrieval → matching → pose-estimation hierarchical framework" as canonical; (b) two independent runtime sources (Skoltech survey on RTX 3090; AnyVisLoc on RTX 3090 with explicit dense-vs-sparse breakdown) agree on the relative cost ordering of model classes; (c) cross-source agreement on AdHoP value (Source #40 only, but with reproducible code and dataset — single-source-but-strong evidence); (d) cross-source agreement on covisibility / sensor-prior thresholds. Two outstanding decisions are flagged for user — neither blocks SQ2's saturation status, both block SQ3+SQ4 start. Per `references/source-tiering.md` "Search saturation rule" → SQ2 is closed pending user decisions on DSM dependency + AdHoP gating.
|
||||
@@ -6,8 +6,8 @@ step: 2
|
||||
name: Research
|
||||
status: in_progress
|
||||
sub_step:
|
||||
phase: 1
|
||||
name: ac-restrictions-assessment
|
||||
detail: "Mode A — reading problem context, preparing AC/restrictions assessment"
|
||||
phase: 10
|
||||
name: sq3-sq4-c1-vio-candidates
|
||||
detail: "Mode A Phase 2 engine Step 4 — SQ3+SQ4 per-component candidate matrix, starting with C1 (VO/VIO). SQ1 ✓, SQ2 ✓ closed with three architectural decisions resolved by user (recorded in 00_question_decomposition.md): (1) DSM dependency = (a) 3-DoF acceptance (no DSM in cache; attitude from IMU/VIO); (2) AdHoP = (b) conditional (only when initial reprojection error exceeds threshold); (3) Top-N re-rank = (a) promoted to explicit pipeline sub-stage between C2 and C3. Pipeline shape entering SQ3+SQ4: C1 (VIO) → C2 (VPR) → Top-N re-rank → C3 (matcher) → AdHoP-conditional → C4 (PnP+RANSAC+LM) → C5 (estimator) → C8 (FC adapter), with C6 (cache 2D-ortho) + C7 (Jetson runtime) + C9 (datasets) + C10 (provisioning) cross-cutting. C1 candidates entering SQ3+SQ4 (RTAB-Map and ORB-SLAM3 already pruned by Fact #16): VINS-Mono / VINS-Fusion / OpenVINS / OKVIS2 / DROID-SLAM / DPVO / pure-VO baseline (KLT + RANSAC homography). Per-mode context7 capability verification mandatory for every lead library/SDK candidate. Next conversation: re-enter Step 2 / sub_step phase 10 name 'sq3-sq4-c1-vio-candidates', execute Per-Component Search Plan (`research/steps/03_engine-investigation.md` Step 4) starting with C1: enumerate 5+ search variants per candidate, log to 01_source_registry.md + 02_fact_cards.md, then Component Applicability Gate (must pass: monocular VIO, ≥3 fps output, Jetson Orin Nano runnable, license OK for dual-use)."
|
||||
retry_count: 0
|
||||
cycle: 1
|
||||
|
||||
Reference in New Issue
Block a user