azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-21 10:21:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

update GPS-denied onboard research docs

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-04-29 17:03:57 +03:00
parent 8fcdbd4cf6
commit af5eb13ecb
22 changed files with 1938 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/01_solution/security_analysis.md
+44
View File
@@ -0,0 +1,44 @@
# Security Analysis
## Threat Model
| Asset | Threat Actors | Attack Vectors | Impact |
|-------|---------------|----------------|--------|
| Flight-controller position input | GPS spoofer, compromised companion process, malicious ground operator | False `GPS_INPUT`, EKF source misconfiguration, replayed MAVLink packets | Aircraft navigates to wrong location or leaves route/geofence |
| Satellite cache | Compromised cache sync source, stale imagery, physical access attacker | Tile replacement, stale metadata, manifest tampering | False satellite anchors or cache poisoning |
| Onboard tile write-back | Bad EKF state, compromised companion, service ingestion bug | Misaligned generated tiles promoted into shared basemap | Cross-flight error propagation |
| Local API | Unauthorized network client, operator laptop malware | Object localization abuse, health/session tampering, denial of service | Data leakage or service interruption |
| FDR logs | Physical capture, insider misuse | Extraction of route, imagery thumbnails, telemetry | Operational intelligence exposure |
| Model/runtime artifacts | Supply-chain attacker, license-incompatible artifact source | Modified TensorRT engines, malicious Python packages, poisoned descriptors, noncommercial model weights in product build | Silent false outputs, code execution, or product license violation |
## Per-Component Security Requirements and Controls
| Component | Risk Level | Controls |
|-----------|------------|----------|
| Frame ingest and calibration | Medium | Store signed calibration profiles; reject unexpected resolution/intrinsics; log camera timestamp drift; never persist raw frames except allowed failure thumbnails. |
| Satellite cache | High | Signed manifests, checksums per package, capture-date metadata, source identity, freshness gates, immutable trusted service-source tiles, local cache verification at startup. |
| VPR and local matching | High | Treat retrieval as untrusted candidate only; require geometric verification, inlier thresholds, freshness checks, covariance consistency, and ESKF innovation gates. |
| ESKF/state estimator | High | Conservative covariance floors, Mahalanobis gates, source-label transitions, false-position event logging, fail-closed to degraded fix_type when uncertainty is high. |
| MAVLink output | High | Pin ArduPilot parameters; emit GPS_INPUT from one process; validate rate and sequence; no v1 ODOMETRY; send `fix_type=0` or degraded accuracy when estimator is invalid. |
| Local API | Medium | Bind to localhost by default; require JWT/API key for network exposure; validate pixel bounds and request schema; rate-limit commands. |
| FDR | Medium/high | Segment files with checksums, rollover logs, no raw frame archive, encrypt or protect storage when mission secrecy requires it. |
| Tile write-back | High | Only write candidate tiles when parent pose covariance passes strict threshold; sidecar stores parent pose, covariance, source ancestry, and quality score; Suite Service requires multi-flight voting before trusted promotion. |
| Dependency/runtime | Medium | Pin package versions, build TensorRT engines at install time, verify model checksums, run dependency vulnerability scanning in CI, and block noncommercial model weights such as official Magic Leap SuperPoint unless separately licensed. |
## Security Controls Summary
1. **Trust boundary**: The onboard system trusts only signed Satellite Service cache packages and live FC telemetry from the configured MAVLink link.
2. **No direct provider calls**: Commercial provider credentials never live on the aircraft; the onboard system consumes only prebuilt cache artifacts.
3. **Fail closed**: Match failures, stale tiles, bad covariance, or state-estimator inconsistency downgrade the source label and `GPS_INPUT` accuracy/fix state.
4. **No dual-source v1 fusion**: `ODOMETRY` is intentionally disabled in v1 to avoid EKF source ambiguity.
5. **Cache poisoning defense**: Generated tiles remain candidate/soft trust until covariance gates and Satellite Service voting promote them.
6. **Local-first API**: The API is not part of the hot path and is local-only unless explicitly configured with authentication.
7. **Forensics without raw-frame hoarding**: FDR captures enough to replay decisions while respecting the no-raw-photo restriction.
## Open Security Work
- Define the cache manifest schema and signing mechanism.
- Pin ArduPilot version and parameter set in deployment docs.
- Decide whether onboard FDR encryption is mandatory for the operating environment.
- Select and scan final model weights and TensorRT engine build pipeline; confirm ALIKED/SIFT/DeDoDe artifact licenses before product packaging.
- Add CI checks for dependency vulnerabilities and generated OpenAPI schema drift.
_docs/01_solution/solution.md
+198
View File
@@ -0,0 +1,198 @@
# Solution Draft
## Assessment Findings
| Old Component Solution | Weak Point (functional/security/performance) | New Solution |
|------------------------|----------------------------------------------|--------------|
| "License-cleared extractor" in the local matcher | Too abstract for planning; tasks need concrete candidates and legal baselines. | Use ALIKED + LightGlue as the first learned-feature candidate, OpenCV SIFT/AKAZE as the legal baseline, and DeDoDe as an experimental fallback. |
| Image pipeline without an explicit scheduler | A FIFO frame queue can violate <400 ms p95 latency even when individual stages are fast, because frames arrive every ~333 ms at 3 Hz. | Add a bounded latest-frame scheduler: camera queue size 1, explicit frame-drop accounting, deadline-aware VPR/matching, and timestamp-correct `GPS_INPUT`. |
| SuperPoint + LightGlue-style local matching | Official Magic Leap SuperPoint pretrained weights are noncommercial research-only. | Reject official SuperPoint weights for product v1 unless a commercial license is obtained. |
| AnyLoc/DINOv2-VLAD VPR chunks | Raw 49,152-dimensional descriptors can consume too much RAM/cache once multi-scale chunks, overlap, indexes, and metadata are included. | Keep event-triggered VPR, but add a mandatory descriptor compression/index-size gate before implementation freeze. |
| 10 GB persistent satellite cache | 400 km² at 0.3-0.5 m/px plus overviews, manifests, VPR descriptors, and generated tiles is not proven by zoom-level math. | Keep the 10 GB target, but require a representative cache-packing benchmark using Suite Satellite Service sample imagery. |
| Public datasets for validation | AerialVL/UAV-VisLoc are useful but do not prove FC IMU timing, covariance calibration, thermal behavior, or MAVLink source behavior. | Use public datasets for early VPR/matcher tests, then require ArduPilot SITL IMU traces and real FC/camera timing captures before final acceptance. |
| Visual blackout during GPS spoofing | Clouds/whiteout can remove all visual signal exactly when real GPS cannot be trusted. | Add a Visual Blackout / IMU-Only Degraded Mode that rejects spoofed GPS, propagates solely from trusted prior state + FC IMU, grows covariance, and fails closed when uncertainty/duration exceeds the safety budget. |
| `GPS_INPUT` + `ODOMETRY` hybrid | Richer `ODOMETRY` semantics are attractive, but source-fusion behavior is version-sensitive. | v1 emits `GPS_INPUT` only. `ODOMETRY` remains a v1.1 item gated by exact ArduPilot release and SITL proof. |
## Product Solution Description
Build an onboard GPS-denied localization service for fixed-wing UAVs. The service estimates the WGS84 coordinate of each navigation-camera frame center, localizes AI-camera detections on flat terrain, and emits ArduPilot-compatible `GPS_INPUT` messages with calibrated confidence.
```text
Nav camera + FC IMU/attitude/altitude
-> bounded latest-frame scheduler + timestamp sync
-> calibration + frame normalization
-> visual-health / blackout classifier
-> planar VO/IMU relative motion
-> conditional compressed-descriptor VPR over preloaded satellite chunks
-> ALIKED/LightGlue or SIFT/AKAZE local geometric verification
-> ESKF state + covariance + source label + IMU-only degraded mode
-> pymavlink GPS_INPUT + local API + FDR
```
The architecture separates fast steady-state tracking from heavier relocalization. Normal frames use VO/IMU prediction and local map priors. VPR runs only on cold start, sharp turns, disconnected segments, VO failure, covariance growth, or operator-assisted relocalization. The scheduler owns frame freshness: if processing pressure rises, it drops stale frames instead of letting a FIFO backlog delay flight-controller output. If the camera is fully occluded while GPS is spoofed or denied, the estimator switches to `{dead_reckoned}` and propagates solely from the last trusted state plus flight-controller IMU/attitude/airspeed/altitude until a trusted visual/satellite anchor recovers or the fail threshold is reached.
## Architecture
### Component: Real-Time Frame Scheduler
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Bounded latest-frame scheduler | Python/C++ worker loop, monotonic timestamps, metrics counters | Makes latency/drop behavior explicit and prevents stale FIFO backlog | Requires careful timestamp ownership across camera, IMU, VO, and MAVLink output | Camera queue size 1, drop accounting, deadline-aware VPR/matching, IMU propagation between image fixes | Logs every drop and stale-frame rejection to FDR | Supports <400 ms p95 and <=10% frame drops without batching | Selected |
**Exact-fit evidence**:
- Project constraints checked: 3 Hz camera input, AC-4.1 latency/drop budget, AC-4.4 no batching, GPS_INPUT timestamp correctness.
- Evidence: Fact #27.
- Disqualifiers: unbounded FIFO image queues are rejected.
### Component: Frame Ingest, Calibration, and Time Sync
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Python/C++ ingest with OpenCV/GStreamer, camera calibration files, and MAVLink timestamp alignment | OpenCV, GStreamer, NumPy, calibration YAML | Simple, debuggable, works with USB/MIPI/GigE once the camera module is pinned | Driver and hardware timestamp behavior are module-specific | Locked nav camera/lens, checkerboard calibration, FC clock sync, altitude/attitude stream | Reject unexpected dimensions/intrinsics; signed calibration profiles; no raw-frame persistence | 3 Hz full-res ingest; hot path may downsample/ROI | Selected |
**Exact-fit evidence**:
- Project constraints checked: fixed downward nav camera, no raw photo storage, high-res frames, FC IMU/attitude, 400 ms p95.
- Evidence: Facts #4, #9, #20.
- Disqualifiers: final v1 camera/lens and hardware timestamp behavior must be pinned before calibration tasks.
### Component: Relative Motion Estimation
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Custom planar VO/IMU module | OpenCV, Eigen/SciPy, optional C++ hot path | Matches nadir fixed camera, flat-terrain assumption, and FC attitude/altitude | Needs calibration, rolling-shutter assessment, and covariance model | Camera intrinsics, altitude, FC attitude/IMU, frame timestamps | Reject low-inlier/high-innovation updates | Must stay within steady-state deadline after downsampling/ROI | Selected |
| NVIDIA cuVSLAM | Isaac ROS/cuVSLAM | Strong Jetson ecosystem and IMU fallback | Official docs emphasize stereo-visual-inertial assumptions; IMU-only fallback is short-duration | Stereo or documented exact monocular path | ROS 2 surface area | Good Jetson acceleration, wrong v1 input fit | Rejected for v1 |
| ORB-SLAM3 / VINS-Fusion | Research SLAM/VIO stacks | Mono-IMU capability | GPL-family licensing and product integration risk | Legal approval, ROS/C++ integration | Larger dependency surface | Benchmark/offline only | Experimental only |
**Exact-fit evidence**:
- Project constraints checked: single downward nav camera, Jetson runtime, flat terrain, VO drift AC.
- Evidence: Facts #7, #8, #20.
- Disqualifiers: stereo-required or GPL-family stacks are not product dependencies for v1.
### Component: Visual Blackout and IMU-Only Degraded Mode
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Visual-health classifier + ESKF IMU-only propagation mode | OpenCV image statistics, exposure/texture checks, FC IMU/attitude/airspeed/altitude, ESKF covariance floors | Handles clouds/whiteout/no-texture frames without trusting spoofed GPS or stale visual fixes | IMU-only position error grows quickly and cannot meet normal accuracy ACs for long gaps | Last trusted visual/satellite anchor, synchronized IMU, GPS health/spoofing signal, covariance thresholds | Spoofed GPS is rejected as an estimator input; every degraded estimate is logged and labeled `{dead_reckoned}` | Transition to blackout mode ≤400 ms; continue IMU-only up to 30 s or until covariance fail threshold | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-1.4, AC-3.5, AC-5.2, AC-NEW-2, AC-NEW-4, AC-NEW-8.
- Evidence: visual blackout/spoofing degraded-mode requirement in `acceptance_criteria.md`.
- Disqualifiers: IMU-only mode cannot be treated as normal navigation; it is degraded/failsafe behavior with honest covariance growth.
### Component: Satellite Cache and Preprocessing
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Suite Satellite Service exchange via COG/GeoTIFF, onboard SQLite/MBTiles-like package with manifests and descriptor sidecars | GDAL/Rasterio, SQLite, local manifest schema | Clear offline service boundary, explicit pixel-size/freshness metadata, fast local lookup | The 10 GB budget is unproven until representative imagery, overviews, descriptors, and sidecars are packed together | 0.5 m/px minimum, 0.3 m/px ideal, capture date, source, CRS, tile matrix, compression profile | Signed manifests, checksums, immutable service-source tiles, stale-tile rejection | Cache-packing benchmark must include descriptors and generated-tile sidecars | Selected with storage gate |
**Exact-fit evidence**:
- Project constraints checked: offline-only cache, 10 GB cap, freshness gates, mid-flight tile write-back, no direct provider calls.
- Evidence: Facts #13, #16, #21.
- Disqualifiers: zoom level alone cannot define physical resolution or storage cost.
### Component: Visual Place Recognition
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| AnyLoc/DINOv2-VLAD-style descriptors over 600-800 m chunks | PyTorch/TensorRT export path, FAISS CPU/HNSW-flat baseline, PCA/quantization | Strong cross-domain retrieval family; offline gallery descriptors; event-triggered online cost | Raw 49,152-dimensional descriptors can violate memory/cache budgets | Precomputed compressed descriptors, top-K dynamic sizing, covariance-aware search window | Retrieval is candidate generation only; never trusted without local verification | VPR invoked only on relocalization triggers; descriptor compression/index-size gate required | Selected with compression gate |
| FAISS GPU/cuVS | FAISS source build or cuVS | Potential lower query latency | ARM64 GPU deployment must be proven; not assumed | Jetson source build and benchmark | Same candidate-only trust model | Optimization path only | Experimental only |
**Exact-fit evidence**:
- Project constraints checked: event-triggered VPR, active-conflict change robustness, Jetson memory/latency, 10 GB cache cap.
- Evidence: Facts #5, #6, #10, #15, #19.
- Disqualifiers: uncompressed descriptors and per-frame VPR are rejected.
### Component: Local Satellite/UAV Geometric Verification
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| ALIKED + LightGlue + RANSAC | LightGlue, ALIKED, OpenCV, ONNX/TensorRT path | Concrete learned-feature candidate without the official SuperPoint license blocker | Jetson speed and sparse-steppe accuracy must be measured | Candidate chunk/tile, camera intrinsics, attitude, altitude, freshness metadata | Strict inlier, reprojection, freshness, Mahalanobis, and covariance gates | Inline matcher target <=200 ms/pair | Selected candidate |
| OpenCV SIFT/AKAZE + classical matching | OpenCV | Commercial-safe legal baseline and regression target | May be weaker on cross-domain imagery and sparse fields | Same geometric verification inputs | Same verification gates | CPU/GPU baseline before learned extractor optimization | Selected baseline |
| DeDoDe | DeDoDe, ONNX/TensorRT ports | MIT-licensed learned-feature fallback | Model size, DINOv2-related variants, and Jetson runtime need validation | Model artifact approval and benchmark | Same verification gates | Fallback if ALIKED/SIFT miss robustness targets | Experimental only |
| Official Magic Leap SuperPoint pretrained weights | SuperPoint | Technically strong local features | Noncommercial research license blocks product use by default | Separate commercial license | License noncompliance risk | Not product path | Rejected for v1 |
**Exact-fit evidence**:
- Project constraints checked: product licensing, cross-view false-match risk, sparse terrain, <400 ms p95.
- Evidence: Facts #10, #17, #18, #24, #25, #26.
- Disqualifiers: official SuperPoint weights are not selected unless licensing changes.
### Component: State Estimator and Confidence
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Error-state Kalman filter in local NED/ENU | NumPy/SciPy prototype, C++ Eigen if profiling requires | Owns covariance, source labels, anchor gating, and output smoothing | Requires calibration, Monte Carlo validation, and conservative covariance floors | IMU propagation, VO deltas, satellite-anchor measurements, innovation gates | Reject overconfident anchors; log every gate decision | Bounded CPU path; hot path may move to C++ only if measured | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-1.4, AC-NEW-4, AC-NEW-7, GPS_INPUT accuracy fields.
- Evidence: Facts #1, #2, #9, #10.
- Disqualifiers: direct matcher-to-GPS output is rejected.
### Component: Flight Controller and Ground Station Interface
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| v1 `GPS_INPUT` emitter | pymavlink | Matches GPS-replacement framing and ArduPilot MAVLink GPS input path | Less expressive than full external-nav `ODOMETRY`; fields must be honest raw-GPS-sensor values | ArduPilot params, SITL tests, WGS84 conversion, h_acc/v_acc fields | Validate rate, sequence, fix_type, and fail-closed behavior | 5-10 Hz output; freshest estimator timestamp only | Selected |
| `ODOMETRY` auxiliary | pymavlink | Better covariance/yaw semantics | EKF source-fusion and source-switching risk by ArduPilot version | Version-pinned SITL and source-switch tests | Avoid double-fusion | v1.1 only | Deferred |
**Exact-fit evidence**:
- Project constraints checked: ArduPilot-only, QGC, v1 GPS_INPUT-only scope.
- Evidence: Facts #1, #2, #3.
- Disqualifiers: v1 emits no `ODOMETRY`.
### Component: Local API, Object Localization, and FDR
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| FastAPI local service + segmented FDR writer | FastAPI, Pydantic, SQLite/Parquet/JSONL segments | OpenAPI docs, health/session/object endpoints, replayable FDR | Must stay outside hot frame path | Local-first API, object pixel validation, rollover schema, no raw frame retention | Bind localhost by default; JWT/API key for network exposure; segment checksums | 1-2 Hz GCS summary; high-rate data local only | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-6, AC-7, AC-NEW-3, OpenAPI documentation, no raw frame storage.
- Evidence: Fact #14 and Context7 FastAPI docs.
- Disqualifiers: API cannot block GPS_INPUT emission.
## Testing Strategy
### Integration / Functional Tests
- Process the 60-frame sample sequence and assert AC-1.1 / AC-1.2 aggregate thresholds and no frame over the allowed maximum error.
- Simulate heavy VPR/local-matching frames and assert stale frames are dropped, not queued, with <=10% drops under the defined sustained-load scenario.
- Simulate sharp turns, disconnected segments, and 350 m outliers; assert VPR/local verification recovers or the estimator downgrades confidence without false anchors.
- Simulate full visual blackout during GPS spoofing for 5 s, 15 s, and 35 s; assert switch to `{dead_reckoned}` ≤400 ms, spoofed GPS ignored, covariance grows monotonically, QGC receives `VISUAL_BLACKOUT_IMU_ONLY`, and `VISUAL_BLACKOUT_FAILSAFE` triggers at the configured duration/covariance threshold.
- Run ArduPilot SITL with v1 parameters and assert accepted `GPS_INPUT` messages at 5-10 Hz, no `ODOMETRY` emission, correct fix_type transitions, and QGC downsampled telemetry.
- Inject stale tiles, mismatched manifests, corrupted descriptors, and cache-poisoning candidates; assert rejection or confidence downgrade.
- Validate object localization with level-flight AI-camera geometry and out-of-frame input errors.
### Non-Functional Tests
- Jetson benchmark: capture-to-`GPS_INPUT` p95 <400 ms with scheduler, VO, VPR triggers, local matcher, ESKF, API, and FDR active.
- IMU-only propagation benchmark: verify blackout-mode CPU cost is negligible, covariance thresholds are deterministic, and `GPS_INPUT.horiz_accuracy` never under-reports the 95% covariance semi-major axis.
- Local matcher bake-off: ALIKED + LightGlue, SIFT/AKAZE, and DeDoDe on sample/public datasets, measuring accuracy, false positives, latency, memory, and license status.
- VPR memory/index benchmark: prove compressed descriptors, FAISS index, TensorRT engines, and runtime buffers stay below 8 GB.
- Cache-packing benchmark: package representative 400 km² imagery with overviews, manifests, descriptors, indexes, and generated-tile sidecars under the 10 GB persistent-cache budget.
- Thermal soak: 25 W workload for 8 hours at the upper environmental envelope without throttling.
- Monte Carlo false-position and cache-poisoning validation over public datasets plus SITL/real FC traces.
- License and dependency scan: fail CI if noncommercial SuperPoint weights or unapproved model artifacts enter product builds.
## References
- ArduPilot MAVProxy GPSInput documentation: `https://ardupilot.org/mavproxy/docs/modules/GPSInput.html`
- MAVLink `GPS_INPUT` message spec: `https://mavlink.io/en/messages/common.html#GPS_INPUT`
- NVIDIA Isaac ROS cuVSLAM docs: `https://nvidia-isaac-ros.github.io/concepts/visual_slam/cuvslam/index.html`
- Magic Leap SuperPoint pretrained network license: `https://github.com/magicleap/SuperPointPretrainedNetwork/blob/master/LICENSE`
- LightGlue repository: `https://github.com/cvg/LightGlue`
- DeDoDe repository: `https://github.com/Parskatt/DeDoDe`
- OpenCV SIFT source: `https://github.com/opencv/opencv/blob/4.x/modules/features2d/src/sift.dispatch.cpp`
- AnyLoc/DINO repository: `https://github.com/AnyLoc/DINO`
- GDAL COG driver and OGC COG standard: `https://gdal.org/en/stable/drivers/raster/cog.html`, `http://www.opengis.net/doc/is/COG/1.0`
- AerialVL dataset: `https://github.com/hmf21/AerialVL`
- UAV-VisLoc paper: `https://arxiv.org/html/2405.11936v1`
## Related Artifacts
- AC assessment: `_docs/00_research/00_ac_assessment.md`
- Question decomposition: `_docs/00_research/00_question_decomposition.md`
- Source registry: `_docs/00_research/01_source_registry.md`
- Fact cards: `_docs/00_research/02_fact_cards.md`
- Component fit matrix: `_docs/00_research/06_component_fit_matrix.md`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md`
- Security analysis: `_docs/01_solution/security_analysis.md`
_docs/01_solution/solution_draft01.md
+153
View File
@@ -0,0 +1,153 @@
# Solution Draft
## Product Solution Description
Build an onboard GPS-denied localization service for fixed-wing UAVs. The service estimates the UAV navigation-camera frame center in WGS84, localizes AI-camera detections on flat terrain, and emits ArduPilot-compatible `GPS_INPUT` messages with calibrated confidence.
High-level flow:
```text
Nav camera + FC IMU/attitude/altitude
-> frame ingest + timestamp sync + calibration
-> planar VO/IMU relative motion
-> conditional VPR over preloaded satellite chunks
-> local satellite/UAV geometric verification
-> ESKF state + covariance + source label
-> pymavlink GPS_INPUT + local API + FDR
```
The architecture deliberately separates fast steady-state tracking from heavier relocalization. Normal frames use VO/IMU prediction and local map priors; VPR runs only on cold start, sharp turns, disconnected segments, VO failure, or covariance growth.
## Existing/Competitor Solutions Analysis
| Solution Class | What It Provides | Why It Is Not Enough Alone | Role In This Draft |
|----------------|------------------|----------------------------|--------------------|
| Pure visual odometry / SLAM | Relative motion from camera frames | Drifts over long fixed-wing flights and fails at disconnected segments or low-overlap turns | Used only as relative motion input |
| Stereo VIO stacks such as cuVSLAM | Strong Jetson-optimized stereo visual-inertial odometry | v1 has one fixed downward navigation camera, not a stereo rig | Rejected for v1, reconsider if hardware changes |
| ORB-SLAM3 / VINS-Fusion | Mono-inertial research baselines | GPL-family license and generic SLAM assumptions make direct product use risky | Experimental/offline benchmark only |
| Direct UAV-to-satellite retrieval | Absolute place recognition | Top-1 retrieval is vulnerable to similar fields, stale tiles, and appearance changes | Used as top-K candidate generation only |
| Cross-view local matching | Geometric proof against satellite reference | Too expensive and false-positive-prone if run everywhere blindly | Run after VPR/prior narrowing with strict gates |
## Architecture
### Component: Frame Ingest, Calibration, and Time Sync
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| Python/C++ ingest with OpenCV/GStreamer, camera calibration files, MAVLink timestamp alignment | OpenCV, GStreamer, NumPy, calibration YAML | Simple, debuggable, works with USB/MIPI/GigE once driver selected | Camera driver and hardware timestamp details are module-specific | Locked nav camera/lens, checkerboard calibration, FC time sync | Validate input dimensions and timestamps; no raw frame persistence | Low/medium | Selected |
**Exact-fit evidence**:
- Project constraints checked: fixed camera, no raw photo storage, 3 Hz frame rate, high-res frames, FC IMU.
- Evidence: `_docs/00_research/06_component_fit_matrix.md`.
- Disqualifiers: none, but final camera driver must be chosen during implementation planning.
### Component: Relative Motion Estimation
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| Custom planar VO/IMU module | OpenCV, Eigen/SciPy, optional TensorRT local features | Matches nadir fixed camera and flat-terrain assumption; avoids stereo/GPL dependency | Requires careful calibration, attitude compensation, and covariance model | Camera intrinsics, altitude, FC attitude/IMU, frame timestamps | Reject low-inlier or high-innovation updates | Medium | Selected |
| cuVSLAM | Isaac ROS/cuVSLAM | Strong Jetson acceleration and IMU fallback | Stereo-visual-inertial design mismatches single nav camera | Stereo camera rig | ROS 2 surface area | Medium | Rejected for v1 |
| ORB-SLAM3 / VINS-Fusion | Research SLAM/VIO stacks | Mono-IMU capability | GPL-family licensing and product integration risk | Legal approval, ROS/C++ integration | Larger attack/dependency surface | Medium/high | Experimental only |
**Exact-fit evidence**:
- Project constraints checked: single downward nav camera, Jetson runtime, flat-terrain assumption, VO drift AC.
- Evidence: Facts #7-#9.
- Disqualifiers: cuVSLAM requires stereo; ORB-SLAM3/VINS-Fusion licensing.
### Component: Satellite Cache and Preprocessing
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| Suite Satellite Service exchange via COG/GeoTIFF, onboard SQLite/MBTiles-like package with manifests and descriptor sidecars | GDAL/Rasterio, SQLite, local manifest schema | Clear service boundary, offline lookup, explicit pixel-size/freshness metadata | Storage estimate depends on final provider compression | 0.5 m/px min, 0.3 m/px ideal, capture date, source, CRS, tile matrix | Reject stale/unsigned manifests; immutable trusted service-source tiles | Medium | Selected |
**Exact-fit evidence**:
- Project constraints checked: offline-only, 10 GB cache cap, freshness gates, mid-flight tile write-back.
- Evidence: Facts #11-#13, #16.
- Disqualifiers: zoom level alone cannot define physical resolution.
### Component: Visual Place Recognition
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| AnyLoc/DINOv2-VLAD-style descriptors over 600-800 m VPR chunks | PyTorch/TensorRT, FAISS CPU/HNSW-flat baseline | Good cross-domain retrieval candidate; offline gallery descriptors; conditional online cost | Must benchmark on steppe/agricultural imagery; CPU index may be enough, GPU FAISS not assumed | Precomputed descriptors, top-K dynamic sizing, covariance-aware search window | Never trust retrieval without local verification | Medium | Selected |
**Exact-fit evidence**:
- Project constraints checked: event-triggered VPR, active-conflict change robustness, Jetson memory/latency.
- Evidence: Facts #5, #6, #10, #15.
- Disqualifiers: per-frame VPR is rejected.
### Component: Local Satellite/UAV Geometric Verification
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| SuperPoint/LightGlue-style local matching + RANSAC homography + geodesic projection | TensorRT/OpenCV, fallback SIFT/AKAZE | Produces inlier count, reprojection error, and covariance evidence for `satellite_anchored` fixes | SuperPoint weights need license review; Jetson speed must be measured | Candidate tile/chunk, camera intrinsics, attitude, altitude, freshness metadata | Strict inlier, Mahalanobis, freshness, and covariance gates | Medium/high | Selected with gates |
**Exact-fit evidence**:
- Project constraints checked: cross-view false-match risk, sparse terrain, <400 ms p95.
- Evidence: Fact #10, component fit matrix.
- Disqualifiers: no single match can bypass ESKF gates.
### Component: State Estimator and Confidence
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| Error-state Kalman filter in local NED/ENU | NumPy/SciPy or C++ Eigen core | Owns covariance, source labels, anchor gating, and output smoothing | Requires calibration and Monte Carlo validation | IMU propagation, VO deltas, satellite-anchor measurements, innovation gates | Reject overconfident anchors; log every gate decision | Medium | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-1.4, AC-NEW-4, AC-NEW-7, GPS_INPUT accuracy fields.
- Evidence: Facts #1, #2, #9, #10.
- Disqualifiers: direct matcher-to-GPS output is rejected.
### Component: Flight Controller and Ground Station Interface
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| v1 `GPS_INPUT` emitter | pymavlink | Matches GPS-replacement framing and ArduPilot `GPS1_TYPE=14` | Less expressive than full external-nav ODOMETRY | ArduPilot params, SITL tests, WGS84 conversion, h_acc/v_acc fields | Validate outbound rates and fail closed on bad state | Low | Selected |
| ODOMETRY auxiliary | pymavlink | Better covariance/yaw semantics | EKF source-fusion risk by ArduPilot version | Version-pinned SITL and source-switch tests | Avoid double-fusion | Medium | Deferred |
**Exact-fit evidence**:
- Project constraints checked: ArduPilot-only, QGC, v1 GPS_INPUT-only scope.
- Evidence: Facts #1-#3.
- Disqualifiers: ODOMETRY disabled for v1.
### Component: Local API, Object Localization, and FDR
| Solution | Tools | Advantages | Limitations | Requirements | Security | Cost | Fit |
|----------|-------|------------|-------------|--------------|----------|------|-----|
| FastAPI local service + FDR writer | FastAPI, Pydantic, SQLite/Parquet/log segments | OpenAPI docs, local health/session/object endpoints, replayable FDR | Must stay outside hot frame path | Localhost or authenticated LAN, rollover, schema versioning | JWT/API key for non-local access; no raw frame retention | Low/medium | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-6, AC-7, AC-NEW-3, OpenAPI documentation.
- Evidence: Fact #14.
- Disqualifiers: API cannot block GPS_INPUT emission.
## Testing Strategy
### Integration / Functional Tests
- Process the 60-frame sample sequence and assert AC-1.1 / AC-1.2 aggregate thresholds.
- Verify no frame exceeds the maximum allowed error in `position_accuracy.csv`.
- Simulate frames 32-43 as a sharp-turn/disconnected-segment scenario and assert relocalization.
- Inject stale tiles and assert no stale match emits `satellite_anchored`.
- Run ArduPilot SITL with `GPS1_TYPE=14` and assert `GPS_INPUT` messages are accepted at configured rate.
- Reboot the companion process mid-replay and assert first valid output within AC-NEW-1 budget.
- Call object-localization API with level-flight inputs and invalid pixel coordinates.
### Non-Functional Tests
- Jetson benchmark: p95 capture-to-`GPS_INPUT` latency <400 ms with VPR triggers and <=10% frame drops.
- Memory profile: peak below 8 GB with descriptors, TensorRT engines, cache index, API, and FDR active.
- Thermal soak: 25 W workload for 8 hours at upper environmental envelope without throttling.
- Monte Carlo false-position: verify AC-NEW-4 and AC-NEW-7 probability budgets over synthetic and real replay sets.
- Cache storage: validate final provider format stays within 10 GB persistent cache and 64 GB FDR cap.
- Security: verify manifest signing/checksums, stale-tile rejection, and local API authentication.
## References
See `_docs/00_research/01_source_registry.md` and `_docs/00_research/02_fact_cards.md`.
## Related Artifacts
- AC assessment: `_docs/00_research/00_ac_assessment.md`
- Question decomposition: `_docs/00_research/00_question_decomposition.md`
- Component fit matrix: `_docs/00_research/06_component_fit_matrix.md`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md` (generated after this draft)
- Security analysis: `_docs/01_solution/security_analysis.md` (generated after this draft)
_docs/01_solution/solution_draft02.md
+164
View File
@@ -0,0 +1,164 @@
# Solution Draft
## Assessment Findings
| Old Component Solution | Weak Point (functional/security/performance) | New Solution |
|------------------------|----------------------------------------------|--------------|
| SuperPoint + LightGlue-style local matching | Official Magic Leap SuperPoint pretrained weights are noncommercial research-only, so they are not a selectable product dependency by default. | Use a local-verification abstraction: LightGlue only with a license-cleared extractor, plus SIFT/AKAZE/classical matching as the legal v1 baseline. |
| AnyLoc/DINOv2-VLAD VPR chunks | Raw 49,152-dimensional descriptors can consume too much RAM/cache once multi-scale chunks, overlap, indexes, and metadata are included. | Keep event-triggered VPR, but add a mandatory descriptor compression/index-size gate before implementation freeze. |
| 10 GB persistent satellite cache | 400 km² at 0.3-0.5 m/px plus overviews, manifests, VPR descriptors, and generated tiles is not proven by zoom-level math. | Keep the 10 GB target, but require a representative cache-packing benchmark using Suite Satellite Service sample imagery. |
| Public datasets for validation | AerialVL/UAV-VisLoc are useful but do not prove FC IMU timing, covariance calibration, thermal behavior, or MAVLink source behavior. | Use public datasets for early VPR/matcher tests, then require ArduPilot SITL IMU traces and real FC/camera timing captures before final acceptance. |
| cuVSLAM rejection rationale | The first draft treated the stereo mismatch as enough; official docs also show IMU-only degraded tracking is short-duration. | Keep cuVSLAM rejected for v1 product use, but retain it as a benchmark/reference if future hardware adds stereo. |
| `GPS_INPUT` + `ODOMETRY` hybrid | Richer `ODOMETRY` semantics are attractive, but source-fusion behavior is version-sensitive. | v1 emits `GPS_INPUT` only. `ODOMETRY` remains a v1.1 item gated by exact ArduPilot release and SITL proof. |
## Product Solution Description
Build an onboard GPS-denied localization service for fixed-wing UAVs. The service estimates the WGS84 coordinate of each navigation-camera frame center, localizes AI-camera detections on flat terrain, and emits ArduPilot-compatible `GPS_INPUT` messages with calibrated confidence.
```text
Nav camera + FC IMU/attitude/altitude
-> frame ingest + timestamp sync + calibration
-> planar VO/IMU relative motion
-> conditional compressed-descriptor VPR over preloaded satellite chunks
-> license-cleared local geometric verification
-> ESKF state + covariance + source label
-> pymavlink GPS_INPUT + local API + FDR
```
The architecture separates fast steady-state tracking from heavier relocalization. Normal frames use VO/IMU prediction and local map priors. VPR runs only on cold start, sharp turns, disconnected segments, VO failure, covariance growth, or operator-assisted relocalization.
## Architecture
### Component: Frame Ingest, Calibration, and Time Sync
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Python/C++ ingest with OpenCV/GStreamer, camera calibration files, and MAVLink timestamp alignment | OpenCV, GStreamer, NumPy, calibration YAML | Simple, debuggable, works with USB/MIPI/GigE once the camera module is pinned | Driver and hardware timestamp behavior are module-specific | Locked nav camera/lens, checkerboard calibration, FC clock sync, altitude/attitude stream | Reject unexpected dimensions/intrinsics; signed calibration profiles; no raw-frame persistence | 3 Hz full-res ingest; processing may downsample/ROI before hot path | Selected |
**Exact-fit evidence**:
- Project constraints checked: fixed downward nav camera, no raw photo storage, high-res frames, FC IMU/attitude, 400 ms p95.
- Evidence: Facts #4, #9, #20.
- Disqualifiers: final v1 camera/lens and hardware timestamp behavior must be pinned before calibration tasks.
### Component: Relative Motion Estimation
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Custom planar VO/IMU module | OpenCV, Eigen/SciPy, optional C++ hot path | Matches nadir fixed camera, flat-terrain assumption, and FC attitude/altitude | Needs careful calibration, rolling-shutter assessment, and covariance model | Camera intrinsics, altitude, FC attitude/IMU, frame timestamps | Reject low-inlier/high-innovation updates | Must stay within steady-state budget after downsampling/ROI | Selected |
| NVIDIA cuVSLAM | Isaac ROS/cuVSLAM | Strong Jetson ecosystem and IMU fallback | Official docs emphasize stereo-visual-inertial assumptions; IMU-only fallback is short-duration | Stereo or documented exact monocular path | ROS 2 surface area | Good Jetson acceleration, wrong v1 input fit | Rejected for v1 |
| ORB-SLAM3 / VINS-Fusion | Research SLAM/VIO stacks | Mono-IMU capability | GPL-family licensing and product integration risk | Legal approval, ROS/C++ integration | Larger dependency surface | Benchmark/offline only | Experimental only |
**Exact-fit evidence**:
- Project constraints checked: single downward nav camera, Jetson runtime, flat terrain, VO drift AC.
- Evidence: Facts #7, #8, #20.
- Disqualifiers: stereo-required or GPL-family stacks are not product dependencies for v1.
### Component: Satellite Cache and Preprocessing
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Suite Satellite Service exchange via COG/GeoTIFF, onboard SQLite/MBTiles-like package with manifests and descriptor sidecars | GDAL/Rasterio, SQLite, local manifest schema | Clear offline service boundary, explicit pixel-size/freshness metadata, fast local lookup | The 10 GB budget is unproven until representative imagery, overviews, descriptors, and sidecars are packed together | 0.5 m/px minimum, 0.3 m/px ideal, capture date, source, CRS, tile matrix, compression profile | Signed manifests, checksums, immutable service-source tiles, stale-tile rejection | Cache-packing benchmark must include descriptors and generated-tile sidecars | Selected with storage gate |
**Exact-fit evidence**:
- Project constraints checked: offline-only cache, 10 GB cap, freshness gates, mid-flight tile write-back, no direct provider calls.
- Evidence: Facts #13, #16, #21.
- Disqualifiers: zoom level alone cannot define physical resolution or storage cost.
### Component: Visual Place Recognition
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| AnyLoc/DINOv2-VLAD-style descriptors over 600-800 m chunks | PyTorch/TensorRT export path, FAISS CPU/HNSW-flat baseline, PCA/quantization | Strong cross-domain retrieval family; offline gallery descriptors; event-triggered online cost | Raw 49,152-dimensional descriptors can violate memory/cache budgets | Precomputed compressed descriptors, top-K dynamic sizing, covariance-aware search window | Retrieval is candidate generation only; never trusted without local verification | VPR invoked only on relocalization triggers; descriptor compression/index-size gate required | Selected with compression gate |
| FAISS GPU/cuVS | FAISS source build or cuVS | Potential lower query latency | ARM64 GPU deployment must be proven; not assumed | Jetson source build and benchmark | Same candidate-only trust model | Optimization path only | Experimental only |
**Exact-fit evidence**:
- Project constraints checked: event-triggered VPR, active-conflict change robustness, Jetson memory/latency, 10 GB cache cap.
- Evidence: Facts #5, #6, #10, #15, #19.
- Disqualifiers: uncompressed descriptors and per-frame VPR are rejected.
### Component: Local Satellite/UAV Geometric Verification
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| License-cleared feature extractor + LightGlue/classical matching + RANSAC homography/geodesic projection | LightGlue where licensed, SIFT/AKAZE fallback, OpenCV, TensorRT if applicable | Keeps geometric proof stage without depending on noncommercial weights | Exact extractor accuracy and Jetson speed must be measured on steppe/agricultural imagery | Candidate chunk/tile, camera intrinsics, attitude, altitude, freshness metadata | Strict inlier, reprojection, freshness, Mahalanobis, and covariance gates | Inline matcher target <=200 ms/pair; fallback relocalization can use longer budget | Selected |
| Official Magic Leap SuperPoint pretrained weights | SuperPoint | Technically strong local features | Noncommercial research license blocks product use by default | Separate commercial license | License noncompliance risk | Not product path | Rejected for v1 |
**Exact-fit evidence**:
- Project constraints checked: product licensing, cross-view false-match risk, sparse terrain, <400 ms p95.
- Evidence: Facts #10, #17, #18.
- Disqualifiers: official SuperPoint weights are not selected unless licensing changes.
### Component: State Estimator and Confidence
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Error-state Kalman filter in local NED/ENU | NumPy/SciPy prototype, C++ Eigen if profiling requires | Owns covariance, source labels, anchor gating, and output smoothing | Requires calibration, Monte Carlo validation, and conservative covariance floors | IMU propagation, VO deltas, satellite-anchor measurements, innovation gates | Reject overconfident anchors; log every gate decision | Bounded CPU path; hot path may move to C++ only if measured | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-1.4, AC-NEW-4, AC-NEW-7, GPS_INPUT accuracy fields.
- Evidence: Facts #1, #2, #9, #10.
- Disqualifiers: direct matcher-to-GPS output is rejected.
### Component: Flight Controller and Ground Station Interface
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| v1 `GPS_INPUT` emitter | pymavlink | Matches GPS-replacement framing and ArduPilot MAVLink GPS input path | Less expressive than full external-nav `ODOMETRY`; fields must be honest raw-GPS-sensor values | ArduPilot params, SITL tests, WGS84 conversion, h_acc/v_acc fields | Validate rate, sequence, fix_type, and fail-closed behavior | 5-10 Hz output; never batch frame-center outputs | Selected |
| `ODOMETRY` auxiliary | pymavlink | Better covariance/yaw semantics | EKF source-fusion and source-switching risk by ArduPilot version | Version-pinned SITL and source-switch tests | Avoid double-fusion | v1.1 only | Deferred |
**Exact-fit evidence**:
- Project constraints checked: ArduPilot-only, QGC, v1 GPS_INPUT-only scope.
- Evidence: Facts #1, #2, #3.
- Disqualifiers: v1 emits no `ODOMETRY`.
### Component: Local API, Object Localization, and FDR
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| FastAPI local service + segmented FDR writer | FastAPI, Pydantic, SQLite/Parquet/JSONL segments | OpenAPI docs, health/session/object endpoints, replayable FDR | Must stay outside hot frame path | Local-first API, object pixel validation, rollover schema, no raw frame retention | Bind localhost by default; JWT/API key for network exposure; segment checksums | 1-2 Hz GCS summary; high-rate data local only | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-6, AC-7, AC-NEW-3, OpenAPI documentation, no raw frame storage.
- Evidence: Fact #14 and Context7 FastAPI docs.
- Disqualifiers: API cannot block GPS_INPUT emission.
## Testing Strategy
### Integration / Functional Tests
- Process the 60-frame sample sequence and assert AC-1.1 / AC-1.2 aggregate thresholds and no frame over the allowed maximum error.
- Simulate sharp turns, disconnected segments, and 350 m outliers; assert VPR/local verification recovers or the estimator downgrades confidence without false anchors.
- Run ArduPilot SITL with v1 parameters and assert accepted `GPS_INPUT` messages at 5-10 Hz, no `ODOMETRY` emission, correct fix_type transitions, and QGC downsampled telemetry.
- Inject stale tiles, mismatched manifests, corrupted descriptors, and cache-poisoning candidates; assert rejection or confidence downgrade.
- Validate object localization with level-flight AI-camera geometry and out-of-frame input errors.
### Non-Functional Tests
- Jetson benchmark: capture-to-`GPS_INPUT` p95 <400 ms with VO, VPR triggers, local matcher, ESKF, API, and FDR active.
- VPR memory/index benchmark: prove compressed descriptors, FAISS index, TensorRT engines, and runtime buffers stay below 8 GB.
- Cache-packing benchmark: package representative 400 km² imagery with overviews, manifests, descriptors, indexes, and generated-tile sidecars under the 10 GB persistent-cache budget.
- Thermal soak: 25 W workload for 8 hours at the upper environmental envelope without throttling.
- Monte Carlo false-position and cache-poisoning validation over public datasets plus SITL/real FC traces.
- License and dependency scan: fail CI if noncommercial SuperPoint weights or unapproved model artifacts enter product builds.
## References
- ArduPilot MAVProxy GPSInput documentation: `https://ardupilot.org/mavproxy/docs/modules/GPSInput.html`
- MAVLink `GPS_INPUT` message spec: `https://mavlink.io/en/messages/common.html#GPS_INPUT`
- NVIDIA Isaac ROS cuVSLAM docs: `https://nvidia-isaac-ros.github.io/concepts/visual_slam/cuvslam/index.html`
- Magic Leap SuperPoint pretrained network license: `https://github.com/magicleap/SuperPointPretrainedNetwork/blob/master/LICENSE`
- LightGlue license and extractor-license issue: `https://github.com/cvg/LightGlue/blob/main/LICENSE`, `https://github.com/cvg/LightGlue/issues/38`
- AnyLoc/DINO repository: `https://github.com/AnyLoc/DINO`
- GDAL COG driver and OGC COG standard: `https://gdal.org/en/stable/drivers/raster/cog.html`, `http://www.opengis.net/doc/is/COG/1.0`
- AerialVL dataset: `https://github.com/hmf21/AerialVL`
- UAV-VisLoc paper: `https://arxiv.org/html/2405.11936v1`
## Related Artifacts
- AC assessment: `_docs/00_research/00_ac_assessment.md`
- Question decomposition: `_docs/00_research/00_question_decomposition.md`
- Source registry: `_docs/00_research/01_source_registry.md`
- Fact cards: `_docs/00_research/02_fact_cards.md`
- Component fit matrix: `_docs/00_research/06_component_fit_matrix.md`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md`
- Security analysis: `_docs/01_solution/security_analysis.md`
_docs/01_solution/solution_draft03.md
+183
View File
@@ -0,0 +1,183 @@
# Solution Draft
## Assessment Findings
| Old Component Solution | Weak Point (functional/security/performance) | New Solution |
|------------------------|----------------------------------------------|--------------|
| "License-cleared extractor" in the local matcher | Too abstract for planning; tasks need concrete candidates and legal baselines. | Use ALIKED + LightGlue as the first learned-feature candidate, OpenCV SIFT/AKAZE as the legal baseline, and DeDoDe as an experimental fallback. |
| Image pipeline without an explicit scheduler | A FIFO frame queue can violate <400 ms p95 latency even when individual stages are fast, because frames arrive every ~333 ms at 3 Hz. | Add a bounded latest-frame scheduler: camera queue size 1, explicit frame-drop accounting, deadline-aware VPR/matching, and timestamp-correct `GPS_INPUT`. |
| SuperPoint + LightGlue-style local matching | Official Magic Leap SuperPoint pretrained weights are noncommercial research-only. | Reject official SuperPoint weights for product v1 unless a commercial license is obtained. |
| AnyLoc/DINOv2-VLAD VPR chunks | Raw 49,152-dimensional descriptors can consume too much RAM/cache once multi-scale chunks, overlap, indexes, and metadata are included. | Keep event-triggered VPR, but add a mandatory descriptor compression/index-size gate before implementation freeze. |
| 10 GB persistent satellite cache | 400 km² at 0.3-0.5 m/px plus overviews, manifests, VPR descriptors, and generated tiles is not proven by zoom-level math. | Keep the 10 GB target, but require a representative cache-packing benchmark using Suite Satellite Service sample imagery. |
| Public datasets for validation | AerialVL/UAV-VisLoc are useful but do not prove FC IMU timing, covariance calibration, thermal behavior, or MAVLink source behavior. | Use public datasets for early VPR/matcher tests, then require ArduPilot SITL IMU traces and real FC/camera timing captures before final acceptance. |
| `GPS_INPUT` + `ODOMETRY` hybrid | Richer `ODOMETRY` semantics are attractive, but source-fusion behavior is version-sensitive. | v1 emits `GPS_INPUT` only. `ODOMETRY` remains a v1.1 item gated by exact ArduPilot release and SITL proof. |
## Product Solution Description
Build an onboard GPS-denied localization service for fixed-wing UAVs. The service estimates the WGS84 coordinate of each navigation-camera frame center, localizes AI-camera detections on flat terrain, and emits ArduPilot-compatible `GPS_INPUT` messages with calibrated confidence.
```text
Nav camera + FC IMU/attitude/altitude
-> bounded latest-frame scheduler + timestamp sync
-> calibration + frame normalization
-> planar VO/IMU relative motion
-> conditional compressed-descriptor VPR over preloaded satellite chunks
-> ALIKED/LightGlue or SIFT/AKAZE local geometric verification
-> ESKF state + covariance + source label
-> pymavlink GPS_INPUT + local API + FDR
```
The architecture separates fast steady-state tracking from heavier relocalization. Normal frames use VO/IMU prediction and local map priors. VPR runs only on cold start, sharp turns, disconnected segments, VO failure, covariance growth, or operator-assisted relocalization. The scheduler owns frame freshness: if processing pressure rises, it drops stale frames instead of letting a FIFO backlog delay flight-controller output.
## Architecture
### Component: Real-Time Frame Scheduler
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Bounded latest-frame scheduler | Python/C++ worker loop, monotonic timestamps, metrics counters | Makes latency/drop behavior explicit and prevents stale FIFO backlog | Requires careful timestamp ownership across camera, IMU, VO, and MAVLink output | Camera queue size 1, drop accounting, deadline-aware VPR/matching, IMU propagation between image fixes | Logs every drop and stale-frame rejection to FDR | Supports <400 ms p95 and <=10% frame drops without batching | Selected |
**Exact-fit evidence**:
- Project constraints checked: 3 Hz camera input, AC-4.1 latency/drop budget, AC-4.4 no batching, GPS_INPUT timestamp correctness.
- Evidence: Fact #27.
- Disqualifiers: unbounded FIFO image queues are rejected.
### Component: Frame Ingest, Calibration, and Time Sync
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Python/C++ ingest with OpenCV/GStreamer, camera calibration files, and MAVLink timestamp alignment | OpenCV, GStreamer, NumPy, calibration YAML | Simple, debuggable, works with USB/MIPI/GigE once the camera module is pinned | Driver and hardware timestamp behavior are module-specific | Locked nav camera/lens, checkerboard calibration, FC clock sync, altitude/attitude stream | Reject unexpected dimensions/intrinsics; signed calibration profiles; no raw-frame persistence | 3 Hz full-res ingest; hot path may downsample/ROI | Selected |
**Exact-fit evidence**:
- Project constraints checked: fixed downward nav camera, no raw photo storage, high-res frames, FC IMU/attitude, 400 ms p95.
- Evidence: Facts #4, #9, #20.
- Disqualifiers: final v1 camera/lens and hardware timestamp behavior must be pinned before calibration tasks.
### Component: Relative Motion Estimation
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Custom planar VO/IMU module | OpenCV, Eigen/SciPy, optional C++ hot path | Matches nadir fixed camera, flat-terrain assumption, and FC attitude/altitude | Needs calibration, rolling-shutter assessment, and covariance model | Camera intrinsics, altitude, FC attitude/IMU, frame timestamps | Reject low-inlier/high-innovation updates | Must stay within steady-state deadline after downsampling/ROI | Selected |
| NVIDIA cuVSLAM | Isaac ROS/cuVSLAM | Strong Jetson ecosystem and IMU fallback | Official docs emphasize stereo-visual-inertial assumptions; IMU-only fallback is short-duration | Stereo or documented exact monocular path | ROS 2 surface area | Good Jetson acceleration, wrong v1 input fit | Rejected for v1 |
| ORB-SLAM3 / VINS-Fusion | Research SLAM/VIO stacks | Mono-IMU capability | GPL-family licensing and product integration risk | Legal approval, ROS/C++ integration | Larger dependency surface | Benchmark/offline only | Experimental only |
**Exact-fit evidence**:
- Project constraints checked: single downward nav camera, Jetson runtime, flat terrain, VO drift AC.
- Evidence: Facts #7, #8, #20.
- Disqualifiers: stereo-required or GPL-family stacks are not product dependencies for v1.
### Component: Satellite Cache and Preprocessing
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Suite Satellite Service exchange via COG/GeoTIFF, onboard SQLite/MBTiles-like package with manifests and descriptor sidecars | GDAL/Rasterio, SQLite, local manifest schema | Clear offline service boundary, explicit pixel-size/freshness metadata, fast local lookup | The 10 GB budget is unproven until representative imagery, overviews, descriptors, and sidecars are packed together | 0.5 m/px minimum, 0.3 m/px ideal, capture date, source, CRS, tile matrix, compression profile | Signed manifests, checksums, immutable service-source tiles, stale-tile rejection | Cache-packing benchmark must include descriptors and generated-tile sidecars | Selected with storage gate |
**Exact-fit evidence**:
- Project constraints checked: offline-only cache, 10 GB cap, freshness gates, mid-flight tile write-back, no direct provider calls.
- Evidence: Facts #13, #16, #21.
- Disqualifiers: zoom level alone cannot define physical resolution or storage cost.
### Component: Visual Place Recognition
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| AnyLoc/DINOv2-VLAD-style descriptors over 600-800 m chunks | PyTorch/TensorRT export path, FAISS CPU/HNSW-flat baseline, PCA/quantization | Strong cross-domain retrieval family; offline gallery descriptors; event-triggered online cost | Raw 49,152-dimensional descriptors can violate memory/cache budgets | Precomputed compressed descriptors, top-K dynamic sizing, covariance-aware search window | Retrieval is candidate generation only; never trusted without local verification | VPR invoked only on relocalization triggers; descriptor compression/index-size gate required | Selected with compression gate |
| FAISS GPU/cuVS | FAISS source build or cuVS | Potential lower query latency | ARM64 GPU deployment must be proven; not assumed | Jetson source build and benchmark | Same candidate-only trust model | Optimization path only | Experimental only |
**Exact-fit evidence**:
- Project constraints checked: event-triggered VPR, active-conflict change robustness, Jetson memory/latency, 10 GB cache cap.
- Evidence: Facts #5, #6, #10, #15, #19.
- Disqualifiers: uncompressed descriptors and per-frame VPR are rejected.
### Component: Local Satellite/UAV Geometric Verification
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| ALIKED + LightGlue + RANSAC | LightGlue, ALIKED, OpenCV, ONNX/TensorRT path | Concrete learned-feature candidate without the official SuperPoint license blocker | Jetson speed and sparse-steppe accuracy must be measured | Candidate chunk/tile, camera intrinsics, attitude, altitude, freshness metadata | Strict inlier, reprojection, freshness, Mahalanobis, and covariance gates | Inline matcher target <=200 ms/pair | Selected candidate |
| OpenCV SIFT/AKAZE + classical matching | OpenCV | Commercial-safe legal baseline and regression target | May be weaker on cross-domain imagery and sparse fields | Same geometric verification inputs | Same verification gates | CPU/GPU baseline before learned extractor optimization | Selected baseline |
| DeDoDe | DeDoDe, ONNX/TensorRT ports | MIT-licensed learned-feature fallback | Model size, DINOv2-related variants, and Jetson runtime need validation | Model artifact approval and benchmark | Same verification gates | Fallback if ALIKED/SIFT miss robustness targets | Experimental only |
| Official Magic Leap SuperPoint pretrained weights | SuperPoint | Technically strong local features | Noncommercial research license blocks product use by default | Separate commercial license | License noncompliance risk | Not product path | Rejected for v1 |
**Exact-fit evidence**:
- Project constraints checked: product licensing, cross-view false-match risk, sparse terrain, <400 ms p95.
- Evidence: Facts #10, #17, #18, #24, #25, #26.
- Disqualifiers: official SuperPoint weights are not selected unless licensing changes.
### Component: State Estimator and Confidence
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| Error-state Kalman filter in local NED/ENU | NumPy/SciPy prototype, C++ Eigen if profiling requires | Owns covariance, source labels, anchor gating, and output smoothing | Requires calibration, Monte Carlo validation, and conservative covariance floors | IMU propagation, VO deltas, satellite-anchor measurements, innovation gates | Reject overconfident anchors; log every gate decision | Bounded CPU path; hot path may move to C++ only if measured | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-1.4, AC-NEW-4, AC-NEW-7, GPS_INPUT accuracy fields.
- Evidence: Facts #1, #2, #9, #10.
- Disqualifiers: direct matcher-to-GPS output is rejected.
### Component: Flight Controller and Ground Station Interface
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| v1 `GPS_INPUT` emitter | pymavlink | Matches GPS-replacement framing and ArduPilot MAVLink GPS input path | Less expressive than full external-nav `ODOMETRY`; fields must be honest raw-GPS-sensor values | ArduPilot params, SITL tests, WGS84 conversion, h_acc/v_acc fields | Validate rate, sequence, fix_type, and fail-closed behavior | 5-10 Hz output; freshest estimator timestamp only | Selected |
| `ODOMETRY` auxiliary | pymavlink | Better covariance/yaw semantics | EKF source-fusion and source-switching risk by ArduPilot version | Version-pinned SITL and source-switch tests | Avoid double-fusion | v1.1 only | Deferred |
**Exact-fit evidence**:
- Project constraints checked: ArduPilot-only, QGC, v1 GPS_INPUT-only scope.
- Evidence: Facts #1, #2, #3.
- Disqualifiers: v1 emits no `ODOMETRY`.
### Component: Local API, Object Localization, and FDR
| Solution | Tools | Advantages | Limitations | Requirements | Security | Performance | Fit |
|----------|-------|------------|-------------|--------------|----------|-------------|-----|
| FastAPI local service + segmented FDR writer | FastAPI, Pydantic, SQLite/Parquet/JSONL segments | OpenAPI docs, health/session/object endpoints, replayable FDR | Must stay outside hot frame path | Local-first API, object pixel validation, rollover schema, no raw frame retention | Bind localhost by default; JWT/API key for network exposure; segment checksums | 1-2 Hz GCS summary; high-rate data local only | Selected |
**Exact-fit evidence**:
- Project constraints checked: AC-6, AC-7, AC-NEW-3, OpenAPI documentation, no raw frame storage.
- Evidence: Fact #14 and Context7 FastAPI docs.
- Disqualifiers: API cannot block GPS_INPUT emission.
## Testing Strategy
### Integration / Functional Tests
- Process the 60-frame sample sequence and assert AC-1.1 / AC-1.2 aggregate thresholds and no frame over the allowed maximum error.
- Simulate heavy VPR/local-matching frames and assert stale frames are dropped, not queued, with <=10% drops under the defined sustained-load scenario.
- Simulate sharp turns, disconnected segments, and 350 m outliers; assert VPR/local verification recovers or the estimator downgrades confidence without false anchors.
- Run ArduPilot SITL with v1 parameters and assert accepted `GPS_INPUT` messages at 5-10 Hz, no `ODOMETRY` emission, correct fix_type transitions, and QGC downsampled telemetry.
- Inject stale tiles, mismatched manifests, corrupted descriptors, and cache-poisoning candidates; assert rejection or confidence downgrade.
- Validate object localization with level-flight AI-camera geometry and out-of-frame input errors.
### Non-Functional Tests
- Jetson benchmark: capture-to-`GPS_INPUT` p95 <400 ms with scheduler, VO, VPR triggers, local matcher, ESKF, API, and FDR active.
- Local matcher bake-off: ALIKED + LightGlue, SIFT/AKAZE, and DeDoDe on sample/public datasets, measuring accuracy, false positives, latency, memory, and license status.
- VPR memory/index benchmark: prove compressed descriptors, FAISS index, TensorRT engines, and runtime buffers stay below 8 GB.
- Cache-packing benchmark: package representative 400 km² imagery with overviews, manifests, descriptors, indexes, and generated-tile sidecars under the 10 GB persistent-cache budget.
- Thermal soak: 25 W workload for 8 hours at the upper environmental envelope without throttling.
- Monte Carlo false-position and cache-poisoning validation over public datasets plus SITL/real FC traces.
- License and dependency scan: fail CI if noncommercial SuperPoint weights or unapproved model artifacts enter product builds.
## References
- ArduPilot MAVProxy GPSInput documentation: `https://ardupilot.org/mavproxy/docs/modules/GPSInput.html`
- MAVLink `GPS_INPUT` message spec: `https://mavlink.io/en/messages/common.html#GPS_INPUT`
- NVIDIA Isaac ROS cuVSLAM docs: `https://nvidia-isaac-ros.github.io/concepts/visual_slam/cuvslam/index.html`
- Magic Leap SuperPoint pretrained network license: `https://github.com/magicleap/SuperPointPretrainedNetwork/blob/master/LICENSE`
- LightGlue repository: `https://github.com/cvg/LightGlue`
- DeDoDe repository: `https://github.com/Parskatt/DeDoDe`
- OpenCV SIFT source: `https://github.com/opencv/opencv/blob/4.x/modules/features2d/src/sift.dispatch.cpp`
- AnyLoc/DINO repository: `https://github.com/AnyLoc/DINO`
- GDAL COG driver and OGC COG standard: `https://gdal.org/en/stable/drivers/raster/cog.html`, `http://www.opengis.net/doc/is/COG/1.0`
- AerialVL dataset: `https://github.com/hmf21/AerialVL`
- UAV-VisLoc paper: `https://arxiv.org/html/2405.11936v1`
## Related Artifacts
- AC assessment: `_docs/00_research/00_ac_assessment.md`
- Question decomposition: `_docs/00_research/00_question_decomposition.md`
- Source registry: `_docs/00_research/01_source_registry.md`
- Fact cards: `_docs/00_research/02_fact_cards.md`
- Component fit matrix: `_docs/00_research/06_component_fit_matrix.md`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md`
- Security analysis: `_docs/01_solution/security_analysis.md`
_docs/01_solution/tech_stack.md
+98
View File
@@ -0,0 +1,98 @@
# Tech Stack Evaluation
## Requirements Analysis
| Requirement | Implication |
|-------------|-------------|
| Jetson Orin Nano Super, 8 GB, 25 W | Prefer Python orchestration with C++/TensorRT hot paths; benchmark memory early. |
| Fixed nadir monocular nav camera | Avoid stereo-only VO dependencies; use custom planar VO/IMU. |
| ArduPilot v1 GPS substitute | Use `pymavlink` `GPS_INPUT`; defer `ODOMETRY`. |
| Offline satellite cache | Use local SQLite/MBTiles-like packages with explicit manifests and precomputed descriptors. |
| OpenAPI docs | Use FastAPI for local control/health/object API. |
| No raw frame persistence | FDR stores metadata, estimates, MAVLink, IMU, health, generated tiles, and failure thumbnails only. |
## Technology Evaluation
### Language and Runtime
| Option | Fitness | Maturity | Security | Team/Cost | Scalability | Score | Decision |
|--------|---------|----------|----------|-----------|-------------|-------|----------|
| Python 3.11+ orchestration | High | High | Medium | High | Medium | 4/5 | Selected |
| C++ hot-path modules | High | High | Medium | Medium | High | 4/5 | Selected for VO/matching kernels as needed |
| .NET backend | Low | High | High | Medium | High | 2/5 | Rejected for onboard CV hot path |
| Rust hot-path modules | Medium | Medium | High | Medium | High | 3/5 | Deferred unless memory safety becomes critical |
### Vision and Inference
| Option | Fitness | Maturity | Security | Team/Cost | Scalability | Score | Decision |
|--------|---------|----------|----------|-----------|-------------|-------|----------|
| OpenCV | High | High | Medium | High | Medium | 5/5 | Selected |
| TensorRT | High | High | Medium | Medium | High | 5/5 | Selected for deployed models |
| PyTorch runtime | Medium | High | Medium | High | Medium | 3/5 | Dev/training only, not hot v1 runtime |
| cuVSLAM | Low for v1 | High | Medium | Medium | High | 2/5 | Rejected for v1 sensor mismatch |
| ORB-SLAM3 / VINS-Fusion | Medium | Medium | Low licensing fit | Medium | Medium | 2/5 | Experimental only |
### Matching and VPR
| Option | Fitness | Maturity | Security | Team/Cost | Scalability | Score | Decision |
|--------|---------|----------|----------|-----------|-------------|-------|----------|
| AnyLoc/DINOv2-VLAD style descriptors | High | Medium | Medium | Medium | High | 4/5 | Selected with benchmark |
| FAISS CPU/HNSW-flat | High | High | Medium | High | Medium | 4/5 | Selected v1 baseline |
| FAISS GPU/cuVS | Medium | Medium | Medium | Low on Jetson | High | 3/5 | Optimization only |
| ALIKED + LightGlue | High | Medium | Medium | Medium | High | 4/5 | Selected learned-feature candidate |
| OpenCV SIFT/AKAZE | Medium | High | High | High | Medium | 3/5 | Selected legal baseline |
| DeDoDe | Medium | Medium | Medium | Medium | Medium | 3/5 | Experimental fallback |
| Official Magic Leap SuperPoint weights | High | Medium | Low for product | Medium | High | 1/5 | Rejected unless commercial license is obtained |
### State, API, and Storage
| Option | Fitness | Maturity | Security | Team/Cost | Scalability | Score | Decision |
|--------|---------|----------|----------|-----------|-------------|-------|----------|
| ESKF with NumPy/SciPy prototype, C++ Eigen if needed | High | High | Medium | Medium | High | 5/5 | Selected |
| FastAPI + Pydantic | High | High | Medium | High | Medium | 5/5 | Selected |
| SQLite / MBTiles-like local cache | High | High | Medium | High | Medium | 4/5 | Selected |
| COG/GeoTIFF + GDAL/Rasterio exchange | High | High | Medium | Medium | High | 4/5 | Selected |
| PostgreSQL/PostGIS onboard | Low | High | Medium | Low | High | 2/5 | Rejected for embedded v1; too heavy |
| Parquet/JSONL FDR segments | High | High | Medium | High | Medium | 4/5 | Selected |
### MAVLink and Ground Station
| Option | Fitness | Maturity | Security | Team/Cost | Scalability | Score | Decision |
|--------|---------|----------|----------|-----------|-------------|-------|----------|
| pymavlink `GPS_INPUT` | High | High | Medium | High | Medium | 5/5 | Selected |
| MAVSDK telemetry plumbing | Medium | High | Medium | High | Medium | 4/5 | Selected for non-GPS telemetry where useful |
| Mission Planner support | Low | High | Medium | Medium | Medium | 2/5 | Out of scope |
| QGroundControl | High | High | Medium | High | Medium | 5/5 | Selected |
## Tech Stack Summary
- **Primary language**: Python 3.11+ for orchestration, API, data pipelines, tests, and integration.
- **Hot path**: C++/CUDA/TensorRT modules only where profiling proves Python/OpenCV is insufficient.
- **Computer vision**: OpenCV, TensorRT, DINOv2/AnyLoc-style descriptors, ALIKED + LightGlue candidate, SIFT/AKAZE legal baseline, DeDoDe experimental fallback.
- **State estimation**: ESKF in local NED/ENU, prototyped in Python and migrated to C++ Eigen only if latency requires.
- **Autopilot**: pymavlink `GPS_INPUT` v1; MAVSDK only for general telemetry if it does not interfere with GPS emission.
- **API**: FastAPI with generated OpenAPI at `/openapi.json`, local-first deployment.
- **Storage**: SQLite/MBTiles-like onboard cache, manifest sidecars, GDAL/Rasterio for COG/GeoTIFF exchange, segmented FDR logs.
- **Testing**: pytest, SITL ArduPilot, replay harness, Jetson benchmark harness, thermal/memory profiling.
## Risk Assessment
| Risk | Impact | Mitigation |
|------|--------|------------|
| Noncommercial SuperPoint weights enter product build | High | Reject official Magic Leap weights by default; CI/license scan model artifacts; use ALIKED + LightGlue or SIFT/AKAZE unless a commercial grant exists. |
| VPR model too slow or memory-heavy | High | Conditional VPR only; benchmark DINOv2-S/tiny variants; CPU FAISS baseline; precompute gallery descriptors. |
| Cross-view false positives in sparse fields | High | Top-K retrieval, local geometry, freshness gates, ESKF innovation gating, covariance calibration. |
| ArduPilot EKF source behavior changes by version | High | Pin ArduPilot version and SITL tests; v1 GPS_INPUT only. |
| Cache format exceeds 10 GB | Medium | Validate provider compression and descriptor index size before implementation freeze. |
| Python hot path misses 400 ms p95 | High | Profile first; move only measured bottlenecks to C++/TensorRT. |
## Learning / Validation Requirements
| Area | Required Proof |
|------|----------------|
| Jetson runtime | Benchmark VO, VPR trigger, local matching, ESKF, API, and FDR together under sustained load. |
| Scheduler latency | Prove bounded latest-frame queue, frame-drop accounting, and timestamp-correct GPS_INPUT under heavy VPR/local-matcher load. |
| ArduPilot integration | SITL proves `GPS_INPUT` acceptance, failsafe behavior, and no ODOMETRY emission in v1. |
| Dataset realism | Add real FC IMU logs or approved SITL IMU generated from the trajectory. |
| Cache ingestion | Validate COG/GeoTIFF import, local package generation, manifests, and stale-tile rejection. |
| Safety | Monte Carlo validates false-position and cache-poisoning budgets before production release. |