gps-denied-onboard/.planning/PROJECT.md

GPS-Denied Onboard Navigation System

What This Is

Real-time GPS-independent position estimation system for a fixed-wing UAV operating in GPS-denied/spoofed environments (flat terrain, Ukraine). Runs onboard a Jetson Orin Nano Super (8GB shared, 67 TOPS). Fuses visual odometry (cuVSLAM), satellite image matching (TensorRT FP16), and IMU via an ESKF to output MAVLink GPS_INPUT to an ArduPilot flight controller at 5-10Hz, while also streaming position and confidence over SSE to a ground station.

Core Value

The flight controller must receive valid MAVLink GPS_INPUT at 5-10Hz with position accuracy ≤50m for 80% of frames — without this, the UAV cannot navigate in GPS-denied airspace.

Requirements

Validated

• ✓ FastAPI service scaffold with SSE streaming — existing
• ✓ FlightProcessor orchestrator with NORMAL/LOST/RECOVERY state machine — existing
• ✓ CoordinateTransformer (GPS↔ENU, pixel→camera→body→NED→WGS84) — existing
• ✓ SatelliteDataManager (tile fetch, diskcache, GeoHash lookup) — existing
• ✓ ImageInputPipeline (frame queue, batch validation, storage) — existing
• ✓ SQLAlchemy async DB layer (flights, waypoints, frames, results) — existing
• ✓ Pydantic schema contracts for all inter-component data — existing
• ✓ ABC interfaces for all core components (VO, GPR, metric, graph) — existing

Active

• ESKF implementation (15-state error-state Kalman filter: IMU prediction + VO update + satellite update + covariance propagation)
• MAVLink GPS_INPUT output (pymavlink, UART/UDP, 5-10Hz loop, ESKF state→GPS_INPUT field mapping)
• Real VO implementation (cuVSLAM on Jetson / OpenCV ORB stub on dev for CI)
• Real TensorRT inference (SuperPoint+LightGlue for VO, XFeat for satellite matching — FP16 on Jetson)
• Satellite feature matching pipeline (tile selection by ESKF uncertainty, RANSAC homography, WGS84 extraction)
• GlobalPlaceRecognition implementation (AnyLoc/DINOv2 candidate retrieval, FAISS index, tile scoring)
• FactorGraph implementation (pose graph with VO edges + satellite anchor nodes, optimization loop)
• FailureRecoveryCoordinator (tracking loss detection, re-init protocol, operator re-localization hint)
• End-to-end pipeline wiring (processor.process_frame → VO → ESKF → satellite → GPS_INPUT)
• Docker SITL test harness (ArduPilot SITL, camera replay, tile server mock, CI integration)
• Confidence scoring and GPS_INPUT fix_type mapping (HIGH/MEDIUM/LOW → fix_type 3/2/0)
• Object GPS localization endpoint (POST /objects/locate with gimbal angle projection)

Out of Scope

• TensorRT engine building tooling — engines are pre-built offline, system only loads them
• Google Maps tile download tooling — tiles pre-cached before flight, not streamed live
• Full ArduPilot integration testing on hardware — Jetson hardware validation is post-v1
• Mobile/web ground station UI — SSE stream is consumed by external systems
• Multi-UAV coordination — single UAV instance only

Context

Hardware target: Jetson Orin Nano Super (8GB LPDDR5 shared, JetPack 6.2.2, CUDA 12.6, TRT 10.3.0). All development happens on x86 Linux; cuVSLAM and TRT are Jetson-only — dev machine uses OpenCV ORB stub and MockInferenceEngine.

Camera: ADTI 20L V1 (5456×3632, APS-C, 16mm lens, nadir fixed, 0.7fps). AI detection camera: Viewpro A40 Pro (separate).

Flight controller: ArduPilot via MAVLink UART. System sends GPS_INPUT; receives IMU (200Hz) and GLOBAL_POSITION_INT (1Hz) from FC.

Key latency budget: <400ms end-to-end per frame (camera @ 0.7fps = 1430ms window).

Existing scaffold: ~2800 lines of Python code exist as a well-structured scaffold. All modules are present with ABC interfaces and schemas, but critical algorithmic kernels (ESKF, real VO, TRT inference, MAVLink) are missing or mocked.

Test data: 60 UAV frames (AD000001-AD000060.jpg), coordinates.csv with ground-truth GPS, expected_results/position_accuracy.csv. 43 documented test scenarios across 7 categories.

Constraints

• Performance: <400ms/frame end-to-end, <8GB RAM+VRAM — non-negotiable for real-time flight
• Hardware: cuVSLAM v15.0.0 (aarch64-only wheel) — stub interface required for CI
• Platform: JetPack 6.2.2, Python 3.10+, TensorRT 10.3.0, CUDA 12.6
• Navigation accuracy: 80% frames ≤50m, 60% frames ≤20m, max drift 100m between satellite corrections
• Resilience: Handle sharp turns (disconnected VO segments), 3+ consecutive satellite match failures

Key Decisions

Decision	Rationale	Outcome
ESKF over EKF/UKF	15-state error-state formulation avoids quaternion singularities, standard for INS	— Pending
XFeat over LiteSAM for satellite matching	LiteSAM may exceed 400ms budget on Jetson; XFeat is faster	— Pending (benchmark required)
OpenCV ORB stub for dev/CI	cuVSLAM is aarch64-only; CI must run on x86	— Pending
AnyLoc/DINOv2 for GPR	Validated on UAV-VisLoc benchmark (17.86m RMSE)	— Pending
diskcache + GeoHash for tiles	O(1) tile lookup, no DB overhead, LRU eviction	✓ Good
AsyncSQLAlchemy + aiosqlite	Non-blocking DB for async FastAPI service	✓ Good

Evolution

This document evolves at phase transitions and milestone boundaries.

After each phase transition (via /gsd:transition):

1. Requirements invalidated? → Move to Out of Scope with reason
2. Requirements validated? → Move to Validated with phase reference
3. New requirements emerged? → Add to Active
4. Decisions to log? → Add to Key Decisions
5. "What This Is" still accurate? → Update if drifted

After each milestone (via /gsd:complete-milestone):

1. Full review of all sections
2. Core Value check — still the right priority?
3. Audit Out of Scope — reasons still valid?
4. Update Context with current state

---

Last updated: 2026-04-01 after initialization