gps-denied-onboard/_docs/00_research/05_validation_log.md

Validation Log

Validation Scenario

An 8-hour fixed-wing mission enters GPS-denied/spoofed mode after takeoff. The onboard system starts from last trusted FC state, processes 3 fps nadir frames, emits GPS_INPUT, handles normal flight, sharp turns, short visual blackouts, stale/changed tiles, and post-flight tile write-back.

Expected Based On Conclusions

• Normal segment: VO/IMU propagates every processed frame; satellite anchors refresh state conditionally before covariance grows too large.
• Sharp turn / <5% overlap: VO is expected to fail; relocalization uses FAISS top-K VPR chunks followed by LightGlue/RANSAC.
• Visual blackout + spoofing: estimator switches to dead_reckoned, covariance grows monotonically, spoofed GPS is ignored, GPS_INPUT degrades honestly.
• Stale tile: anchor is rejected or down-confidence weighted and cannot emit satellite_anchored.
• Cache write-back: onboard generated tile is written only when parent-pose covariance passes AC-NEW-7 gates and carries metadata for Satellite Service voting.

Actual Validation Plan

Validation Target	Test Method	Pass Evidence
VO/VIO propagation	EuRoC and synthetic nadir replay; then representative flight data	Drift vs anchor-age bins; AC-1.3 pass/fail.
Satellite anchor	AerialVL/VPAir-style benchmark plus project sample imagery with satellite cache	AC-1.1/1.2 accuracy, AC-2.2 MRE, georeference recall.
Runtime	Jetson Orin Nano Super profiling under 25 W, hot-soak included	<400 ms p95, <8 GB memory, no thermal throttle.
VPR retrieval	Offline descriptor build and FAISS query benchmark	Top-K recall, query latency, index size within cache budget.
MAVLink output	ArduPilot Plane SITL with GPS1_TYPE=14	Valid GPS_INPUT, fix-type/accuracy degradation, QGC status.
Spoofing promotion	Plane SITL false GPS injection	Promotion <3 s and spoofed GPS rejected during blackout.
FDR	8-hour synthetic load	<=64 GB, rollover logged, no silent payload loss.
Cache poisoning	Monte Carlo with over-confident wrong anchors	AC-NEW-7 probabilities below budget; metadata contract emitted.
OpenVINS reference comparison	Replay the same synchronized camera+IMU segments through OpenVINS and the project-owned estimator	OpenVINS establishes a VIO baseline; production estimator must match/beat drift where applicable while preserving source labels and GPS_INPUT behavior.
BASALT production VIO candidate	Replay synchronized camera+IMU segments through BASALT, OpenVINS, and Kimera-VIO	BASALT selected if drift, completion rate, latency, and wrapper-calibrated covariance meet project gates.
DINOv2-VLAD fidelity	Compare PyTorch, ONNX, and TensorRT descriptor distances and FAISS rankings	Optimized engines accepted only if rank/top-K behavior stays within tolerance.
ALIKED/LightGlue runtime	Jetson benchmark across K candidates and project image sizes	Candidate accepted for runtime only if relocalization trigger path fits AC-4.1 with bounded frame drops.

Counterexamples And Risks

• Large DINOv2 variants or many local-match candidates may violate the Jetson latency/memory envelope.
• Agricultural fields can be visually repetitive; VPR confidence must not be treated as sufficient without geometric verification.
• Public datasets do not fully match Ukrainian fixed-wing operational conditions; final evidence requires representative data.
• GPL VIO/SLAM libraries are not production dependencies unless licensing is explicitly accepted.
• OpenVINS may outperform the first custom estimator prototype on pure VIO drift; that would trigger estimator improvement, not automatic GPL production adoption.
• BASALT covariance/confidence is less directly exposed than OpenVINS EKF covariance; the project wrapper must calibrate uncertainty before mapping it to GPS_INPUT.horiz_accuracy.

Review Checklist

• Draft conclusions are consistent with fact cards.
• No important dimensions missed: architecture, VO, VPR, local matching, cache, estimator, MAVLink, FDR, validation covered.
• No selected component relies only on field-adjacent fit.
• Mismatches are recorded as rejected/reference/needs-decision rather than hidden.
• Step 7.5 Component Applicability Gate applies and is saved in 06_component_fit_matrix.md.

Conclusions Requiring Revision

Round 3 applies the user decision to select BASALT as the production VIO candidate. The selected implementation is BASALT VIO plus a project-owned safety/anchor wrapper; OpenVINS remains the covariance/reference baseline, Kimera-VIO remains a backup candidate, and custom OpenCV-only VIO is no longer the primary path. Runtime gates and Plane SITL gates are implementation validation gates, not API capability blockers.