gps-denied-onboard/_docs/00_research/05_validation_log.md

Validation Log

Validation Scenario

Mode B validates the revised architecture against four weak-point scenarios:

1. Package a representative 400 km² cache slice at 0.3-0.5 m/px with COG exchange files, onboard SQLite/MBTiles-like tiles, manifests, overviews, VPR descriptors, and sidecars.
2. Build a VPR index for 600-800 m chunks with 40-50% overlap, using raw AnyLoc/DINOv2-VLAD descriptors and at least one compressed descriptor variant.
3. Benchmark local verification on the provided 60-frame sequence plus AerialVL/UAV-VisLoc-style public data using ALIKED + LightGlue, OpenCV SIFT/AKAZE, and DeDoDe by default.
4. Replay a GPS-denied flight in ArduPilot SITL with synthetic IMU and camera timestamps, then repeat with real FC/camera logs once available.
5. Stress the scheduler with synthetic heavy VPR/local-matcher events and verify stale frames are dropped rather than queued.

Expected Based on Conclusions

• Cache package stays under 10 GB only if representative compression, overviews, descriptor compression, and metadata are measured together.
• VPR remains event-triggered and can load/query within the 8 GB memory envelope after descriptor compression.
• Official Magic Leap SuperPoint weights are absent from product builds unless commercial licensing is obtained.
• ALIKED + LightGlue, OpenCV SIFT/AKAZE, and DeDoDe are benchmarked as distinct local-matcher candidates instead of hiding behind a generic "license-cleared extractor" label.
• The camera queue remains bounded; frame drops are counted; emitted GPS_INPUT messages reference fresh estimator timestamps.
• Local matcher acceptance is based on measured geodetic error, inlier quality, covariance consistency, and false-positive rejection, not only pixel MRE.
• Public datasets are used for early retrieval/matching proof; final fusion and MAVLink acceptance uses SITL or real FC IMU/camera timing traces.
• v1 emits GPS_INPUT only; no ODOMETRY message appears on the wire.

Actual Validation Results

Not executed in research phase. This log defines the validation plan for implementation and test decomposition.

Counterexamples

• A technically accurate SuperPoint benchmark is not product-usable if it depends on noncommercial weights.
• A VPR demo can fit latency while still violating memory/cache budgets if uncompressed 49,152-dimensional descriptors are used at all chunk scales.
• A 400 km² imagery package can fit compressed raster storage but fail once descriptors, indexes, generated tiles, and metadata are counted.
• Public datasets can pass retrieval metrics but still miss FC timing, IMU covariance, thermal, and MAVLink source behavior.
• A FIFO image queue can meet throughput on average while still violating the p95 freshness requirement under bursty VPR/matcher load.

Review Checklist

• Draft conclusions consistent with fact cards.
• No important Mode B weak-point dimensions missed for v1 architecture.
• No over-extrapolation from urban-only cross-view datasets.
• Selected components checked against the Project Constraint Matrix.
• Mismatches recorded as disqualifiers or gates.
• Noncommercial SuperPoint weights rejected for product v1 unless licensed.
• Concrete local-matcher candidates named for planning.
• Real-time scheduler/drop policy added as an architectural component.
• Cache-packing benchmark still required with Suite Satellite Service sample imagery.
• VPR descriptor compression benchmark still required on Jetson Orin Nano Super.
• Real IMU/FC logs still required for final validation.

Conclusions Requiring Revision

• Replace direct SuperPoint dependency language with named license-cleared matcher candidates; official Magic Leap weights are rejected for product v1 unless separately licensed.
• Replace generic "license-cleared extractor" with named candidates: ALIKED + LightGlue, OpenCV SIFT/AKAZE, and DeDoDe fallback.
• Add a bounded latest-frame scheduler and scheduler stress tests to the plan.
• Add an explicit VPR descriptor compression/index-size gate before implementation freeze.
• Add an explicit cache-packing benchmark before accepting the 10 GB persistent-cache budget.
• If CPU FAISS/HNSW retrieval exceeds latency, benchmark source-built GPU FAISS or a smaller descriptor/index design.
• If the selected lens fails 10-20 cm/px at <=1 km AGL, recalibrate footprint, VPR chunk size, and matcher scale assumptions.