mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 21:21:13 +00:00
Oleksandr Bezdieniezhnykh
52 lines
4.7 KiB
Markdown
52 lines
4.7 KiB
Markdown
# Validation Log
|
|
|
|
## Validation Scenario
|
|
|
|
An 8-hour fixed-wing mission enters GPS-denied/spoofed mode after takeoff. The onboard system starts from last trusted FC state, processes 3 fps nadir frames, emits `GPS_INPUT`, handles normal flight, sharp turns, short visual blackouts, stale/changed tiles, and post-flight tile write-back.
|
|
|
|
## Expected Based On Conclusions
|
|
|
|
- **Normal segment**: VO/IMU propagates every processed frame; satellite anchors refresh state conditionally before covariance grows too large.
|
|
- **Sharp turn / <5% overlap**: VO is expected to fail; relocalization uses FAISS top-K VPR chunks followed by LightGlue/RANSAC.
|
|
- **Visual blackout + spoofing**: estimator switches to `dead_reckoned`, covariance grows monotonically, spoofed GPS is ignored, `GPS_INPUT` degrades honestly.
|
|
- **Stale tile**: anchor is rejected or down-confidence weighted and cannot emit `satellite_anchored`.
|
|
- **Cache write-back**: onboard generated tile is written only when parent-pose covariance passes AC-NEW-7 gates and carries metadata for Satellite Service voting.
|
|
|
|
## Actual Validation Plan
|
|
|
|
| Validation Target | Test Method | Pass Evidence |
|
|
|-------------------|-------------|---------------|
|
|
| VO/VIO propagation | EuRoC and synthetic nadir replay; then representative flight data | Drift vs anchor-age bins; AC-1.3 pass/fail. |
|
|
| Satellite anchor | AerialVL/VPAir-style benchmark plus project sample imagery with satellite cache | AC-1.1/1.2 accuracy, AC-2.2 MRE, georeference recall. |
|
|
| Runtime | Jetson Orin Nano Super profiling under 25 W, hot-soak included | <400 ms p95, <8 GB memory, no thermal throttle. |
|
|
| VPR retrieval | Offline descriptor build and FAISS query benchmark | Top-K recall, query latency, index size within cache budget. |
|
|
| MAVLink output | ArduPilot Plane SITL with `GPS1_TYPE=14` | Valid `GPS_INPUT`, fix-type/accuracy degradation, QGC status. |
|
|
| Spoofing promotion | Plane SITL false GPS injection | Promotion <3 s and spoofed GPS rejected during blackout. |
|
|
| FDR | 8-hour synthetic load | <=64 GB, rollover logged, no silent payload loss. |
|
|
| Cache poisoning | Monte Carlo with over-confident wrong anchors | AC-NEW-7 probabilities below budget; metadata contract emitted. |
|
|
| OpenVINS reference comparison | Replay the same synchronized camera+IMU segments through OpenVINS and the project-owned estimator | OpenVINS establishes a VIO baseline; production estimator must match/beat drift where applicable while preserving source labels and GPS_INPUT behavior. |
|
|
| BASALT production VIO candidate | Replay synchronized camera+IMU segments through BASALT, OpenVINS, and Kimera-VIO | BASALT selected if drift, completion rate, latency, and wrapper-calibrated covariance meet project gates. |
|
|
| DINOv2-VLAD fidelity | Compare PyTorch, ONNX, and TensorRT descriptor distances and FAISS rankings | Optimized engines accepted only if rank/top-K behavior stays within tolerance. |
|
|
| ALIKED/LightGlue runtime | Jetson benchmark across K candidates and project image sizes | Candidate accepted for runtime only if relocalization trigger path fits AC-4.1 with bounded frame drops. |
|
|
|
|
## Counterexamples And Risks
|
|
|
|
- Large DINOv2 variants or many local-match candidates may violate the Jetson latency/memory envelope.
|
|
- Agricultural fields can be visually repetitive; VPR confidence must not be treated as sufficient without geometric verification.
|
|
- Public datasets do not fully match Ukrainian fixed-wing operational conditions; final evidence requires representative data.
|
|
- GPL VIO/SLAM libraries are not production dependencies unless licensing is explicitly accepted.
|
|
- OpenVINS may outperform the first custom estimator prototype on pure VIO drift; that would trigger estimator improvement, not automatic GPL production adoption.
|
|
- BASALT covariance/confidence is less directly exposed than OpenVINS EKF covariance; the project wrapper must calibrate uncertainty before mapping it to `GPS_INPUT.horiz_accuracy`.
|
|
|
|
## Review Checklist
|
|
|
|
- [x] Draft conclusions are consistent with fact cards.
|
|
- [x] No important dimensions missed: architecture, VO, VPR, local matching, cache, estimator, MAVLink, FDR, validation covered.
|
|
- [x] No selected component relies only on field-adjacent fit.
|
|
- [x] Mismatches are recorded as rejected/reference/needs-decision rather than hidden.
|
|
- [x] Step 7.5 Component Applicability Gate applies and is saved in `06_component_fit_matrix.md`.
|
|
|
|
## Conclusions Requiring Revision
|
|
|
|
Round 3 applies the user decision to select BASALT as the production VIO candidate. The selected implementation is BASALT VIO plus a project-owned safety/anchor wrapper; OpenVINS remains the covariance/reference baseline, Kimera-VIO remains a backup candidate, and custom OpenCV-only VIO is no longer the primary path. Runtime gates and Plane SITL gates are implementation validation gates, not API capability blockers.
|