mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-04-22 21:56:38 +00:00

Files

T

Yuzviak 78dcf7b4e7 fix: post-audit — runtime bugs, functional gaps, docs, hardening

Phase A — Runtime bugs:
  - SSE: add push_event() method to SSEEventStreamer (was missing, masked by mocks)
  - MAVLink: satellites_visible=10 (was 0, triggers ArduPilot failsafe)
  - MAVLink: horiz_accuracy=sqrt(P[0,0]+P[1,1]) per spec (was sqrt(avg))
  - MAVLink: MEDIUM confidence → fix_type=3 per solution.md (was 2)

Phase B — Functional gaps:
  - handle_user_fix() injects operator GPS into ESKF with noise=500m
  - app.py uses create_vo_backend() factory (was hardcoded SequentialVO)
  - ESKF: Mahalanobis gating on satellite updates (rejects outliers >5σ)
  - ESKF: public accessors (position, quaternion, covariance, last_timestamp)
  - Processor: no more private ESKF field access

Phase C — Documentation:
  - README: correct API endpoints, CLI command, 40+ env vars documented
  - Dockerfile: ENV prefixes match pydantic-settings (DB_, SATELLITE_, MAVLINK_)
  - tech_stack.md marked ARCHIVED (contradicts solution.md)

Phase D — Hardening:
  - JWT auth middleware (AUTH_ENABLED=false default, verify_token on /flights)
  - TLS config env vars (AUTH_SSL_CERTFILE, AUTH_SSL_KEYFILE)
  - SHA-256 tile manifest verification in SatelliteDataManager
  - AuthConfig, ESKFSettings, MAVLinkConfig, SatelliteConfig in config.py

Also: conftest.py shared fixtures, download_tiles.py, convert_to_trt.py scripts,
config wiring into app.py lifespan, config-driven ESKF, calculate_precise_angle fix.

Tests: 196 passed / 8 skipped. Ruff clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-02 18:27:35 +03:00

13 KiB

Raw Blame History

⚠️ ARCHIVED — Дивіться solution.md для актуальної специфікації

Увага: Цей документ містить застарілі дані (3fps замість 0.7fps, LiteSAM 480px замість 1280px). Актуальна специфікація: _docs/01_solution/solution.md

Tech Stack Evaluation (ARCHIVED)

Requirements Summary

Functional

GPS-denied visual navigation for fixed-wing UAV
Frame-center GPS estimation via VO + satellite matching + IMU fusion
Object-center GPS via geometric projection
Real-time streaming via REST API + SSE
Disconnected route segment handling
User-input fallback for unresolvable frames

Non-Functional

<400ms per-frame processing (camera @ ~3fps)
<50m accuracy for 80% of frames, <20m for 60%
<8GB total memory (CPU+GPU shared pool)
Up to 3000 frames per flight session
Image Registration Rate >95% (normal segments)

Hardware Constraints

Jetson Orin Nano Super (8GB LPDDR5, 1024 CUDA cores, 67 TOPS INT8)
JetPack 6.2.2: CUDA 12.6.10, TensorRT 10.3.0, cuDNN 9.3
ARM64 (aarch64) architecture
No internet connectivity during flight

Technology Evaluation

Platform & OS

Option	Version	Score (1-5)	Notes
JetPack 6.2.2 (L4T)	Ubuntu 22.04 based	5	Only supported OS for Orin Nano Super. Includes CUDA 12.6, TensorRT 10.3, cuDNN 9.3

Selected: JetPack 6.2.2 — no alternative.

Primary Language

Option	Fitness	Maturity	Perf on Jetson	Ecosystem	Score
Python 3.10+	5	5	4	5	4.8
C++	5	5	5	3	4.5
Rust	3	3	5	2	3.3

Selected: Python 3.10+ as primary language.

cuVSLAM provides Python bindings (PyCuVSLAM v15.0.0)
TensorRT has Python API
FastAPI is Python-native
OpenCV has full Python+CUDA bindings
Performance-critical paths offloaded to CUDA via cuVSLAM/TensorRT — Python is glue code only
C++ for custom ESKF if NumPy proves too slow (unlikely for 16-state EKF at 100Hz)

Visual Odometry

Option	Version	FPS on Orin Nano	Memory	License	Score
cuVSLAM (PyCuVSLAM)	v15.0.0 (Mar 2026)	116fps @ 720p	~200-300MB	Free (NVIDIA, closed-source)	5
XFeat frame-to-frame	TensorRT engine	~30-50ms/frame	~50MB	MIT	3.5
ORB-SLAM3	v1.0	~30fps	~300MB	GPLv3	2.5

Selected: PyCuVSLAM v15.0.0

116fps on Orin Nano 8G at 720p (verified via Intermodalics benchmark)
Mono + IMU mode natively supported
Auto IMU fallback on tracking loss
Pre-built aarch64 wheel: pip install -e bin/aarch64
Loop closure built-in

Risk: Closed-source; nadir-only camera not explicitly tested. Fallback: XFeat frame-to-frame matching.

Satellite Image Matching (Benchmark-Driven Selection)

Day-one benchmark decides between two candidates:

Option	Params	Accuracy (UAV-VisLoc)	Est. Time on Orin Nano	License	Score
LiteSAM (opt)	6.31M	RMSE@30 = 17.86m	~300-500ms @ 480px (estimated)	Open-source	4 (if fast enough)
XFeat semi-dense	~5M	Not benchmarked on UAV-VisLoc	~50-100ms	MIT	4 (if LiteSAM too slow)

Decision rule:

Export LiteSAM (opt) to TensorRT FP16 on Orin Nano Super
Benchmark at 480px, 640px, 800px
If ≤400ms at 480px → LiteSAM
If >400ms → abandon LiteSAM, XFeat is primary

Requirement	LiteSAM (opt)	XFeat semi-dense
PyTorch → ONNX → TensorRT export	Required	Required
TensorRT FP16 engine	6.31M params, ~25MB engine	~5M params, ~20MB engine
Input preprocessing	Resize to 480px, normalize	Resize to 640px, normalize
Matching pipeline	End-to-end (detect + match + refine)	Detect → KNN match → geometric verify
Cross-view robustness	Designed for satellite-aerial gap	General-purpose, less robust

Sensor Fusion

Option	Complexity	Accuracy	Compute @ 100Hz	Score
ESKF (custom)	Low	Good	<1ms/step	5
Hybrid ESKF/UKF	Medium	49% better	~2-3ms/step	3.5
GTSAM Factor Graph	High	Best	~10-50ms/step	2

Selected: Custom ESKF in Python (NumPy/SciPy)

16-state vector, well within NumPy capability
FilterPy (v1.4.5, MIT) as reference/fallback, but custom implementation preferred for tighter control
If 100Hz IMU prediction step proves slow in Python: rewrite as Cython or C extension (~1 day effort)

Image Preprocessing

Option	Tool	Time on Orin Nano	Notes	Score
OpenCV CUDA resize	cv2.cuda.resize	~2-3ms (pre-allocated)	Must build OpenCV with CUDA from source. Pre-allocate GPU mats to avoid allocation overhead	4
NVIDIA VPI resize	VPI 3.2	~1-2ms	Part of JetPack, potentially faster	4
CPU resize (OpenCV)	cv2.resize	~5-10ms	No GPU needed, simpler	3

Selected: OpenCV CUDA (pre-allocated GPU memory) or VPI 3.2 (whichever is faster in benchmark). Both available in JetPack 6.2.

Must build OpenCV from source with CUDA_ARCH_BIN=8.7 for Orin Nano Ampere architecture
Alternative: VPI 3.2 is pre-installed in JetPack 6.2, no build step needed

API & Streaming Framework

Option	Version	Async Support	SSE Support	Score
FastAPI + sse-starlette	FastAPI 0.115+, sse-starlette 3.3.2	Native async/await	EventSourceResponse with auto-disconnect	5
Flask + flask-sse	Flask 3.x	Limited	Redis dependency	2
Raw aiohttp	aiohttp 3.x	Full	Manual SSE implementation	3

Selected: FastAPI + sse-starlette v3.3.2

sse-starlette: 108M downloads/month, BSD-3 license, production-stable
Auto-generated OpenAPI docs
Native async for non-blocking VO + satellite pipeline
Uvicorn as ASGI server

Satellite Tile Storage & Indexing

Option	Complexity	Lookup Speed	Score
GeoHash-indexed directory	Low	O(1) hash lookup	5
SQLite + spatial index	Medium	O(log n)	4
PostGIS	High	O(log n)	2 (overkill)

Selected: GeoHash-indexed directory structure

Pre-flight: download tiles, store as {geohash}/{zoom}_{x}_{y}.jpg + {geohash}/{zoom}_{x}_{y}_resized.jpg
Runtime: compute geohash from ESKF position → direct directory lookup
Metadata in JSON sidecar files
No database dependency on the Jetson during flight

Satellite Tile Provider

Provider	Max Zoom	GSD	Pricing	Eastern Ukraine Coverage	Score
Google Maps Tile API	18-19	~0.3-0.5 m/px	100K tiles free/month, then $0.48/1K	Partial (conflict zone gaps)	4
Bing Maps	18-19	~0.3-0.5 m/px	125K free/year (basic)	Similar	3.5
Mapbox Satellite	18-19	~0.5 m/px	200K free/month	Similar	3.5

Selected: Google Maps Tile API (per restrictions.md). 100K free tiles/month covers ~25km² at zoom 19. For larger operational areas, costs are manageable at $0.48/1K tiles.

Output Format

Format	Standard	Tooling	Score
GeoJSON	RFC 7946	Universal GIS support	5
CSV (lat, lon, confidence)	De facto	Simple, lightweight	4

Selected: GeoJSON as primary, CSV as export option. Per AC: WGS84 coordinates.

Tech Stack Summary

┌────────────────────────────────────────────────────┐
│  HARDWARE: Jetson Orin Nano Super 8GB               │
│  OS: JetPack 6.2.2 (L4T / Ubuntu 22.04)            │
│  CUDA 12.6.10 / TensorRT 10.3.0 / cuDNN 9.3       │
├────────────────────────────────────────────────────┤
│  LANGUAGE: Python 3.10+                             │
│  FRAMEWORK: FastAPI + sse-starlette 3.3.2           │
│  SERVER: Uvicorn (ASGI)                             │
├────────────────────────────────────────────────────┤
│  VISUAL ODOMETRY: PyCuVSLAM v15.0.0                │
│  SATELLITE MATCH: LiteSAM(opt) or XFeat (benchmark) │
│  SENSOR FUSION: Custom ESKF (NumPy/SciPy)           │
│  PREPROCESSING: OpenCV CUDA or VPI 3.2              │
│  INFERENCE: TensorRT 10.3.0 (FP16)                  │
├────────────────────────────────────────────────────┤
│  TILE PROVIDER: Google Maps Tile API                 │
│  TILE STORAGE: GeoHash-indexed directory             │
│  OUTPUT: GeoJSON (WGS84) via SSE stream              │
└────────────────────────────────────────────────────┘

Dependency List

Python Packages (pip)

Package	Version	Purpose
pycuvslam	v15.0.0 (aarch64 wheel)	Visual odometry
fastapi	>=0.115	REST API framework
sse-starlette	>=3.3.2	SSE streaming
uvicorn	>=0.30	ASGI server
numpy	>=1.26	ESKF math, array ops
scipy	>=1.12	Rotation matrices, spatial transforms
opencv-python (CUDA build)	>=4.8	Image preprocessing (must build from source with CUDA)
torch (aarch64)	>=2.3 (JetPack-compatible)	LiteSAM model loading (if selected)
tensorrt	10.3.0 (JetPack bundled)	Inference engine
pycuda	>=2024.1	CUDA stream management
geojson	>=3.1	GeoJSON output formatting
pygeohash	>=1.2	GeoHash tile indexing

System Dependencies (JetPack 6.2.2)

Component	Version	Notes
CUDA Toolkit	12.6.10	Pre-installed
TensorRT	10.3.0	Pre-installed
cuDNN	9.3	Pre-installed
VPI	3.2	Pre-installed, alternative to OpenCV CUDA for resize
cuVSLAM runtime	Bundled with PyCuVSLAM wheel

Offline Preprocessing Tools (developer machine, not Jetson)

Tool	Purpose
Python 3.10+	Tile download script
Google Maps Tile API key	Satellite tile access
torch + LiteSAM weights	Feature pre-extraction (if LiteSAM selected)
trtexec (TensorRT)	Model export to TensorRT engine

Risk Assessment

Technology	Risk	Likelihood	Impact	Mitigation
cuVSLAM	Closed-source, nadir camera untested	Medium	High	XFeat frame-to-frame as open-source fallback
LiteSAM	May exceed 400ms on Orin Nano Super	High	High	Abandon for XFeat — day-one benchmark is go/no-go
OpenCV CUDA build	Build complexity on Jetson, CUDA arch compatibility	Medium	Low	VPI 3.2 as drop-in alternative (pre-installed)
Google Maps Tile API	Conflict zone coverage gaps, EEA restrictions	Medium	Medium	Test tile availability for operational area pre-flight; alternative providers (Bing, Mapbox)
Custom ESKF	Implementation bugs, tuning effort	Low	Medium	FilterPy v1.4.5 as reference; well-understood algorithm
Python GIL	Concurrent VO + satellite matching contention	Low	Low	CUDA operations release GIL; use asyncio + threading for I/O

Learning Requirements

Technology	Team Expertise Needed	Ramp-up Time
PyCuVSLAM	SLAM concepts, Python API, camera calibration	2-3 days
TensorRT model export	ONNX export, trtexec, FP16 optimization	2-3 days
LiteSAM architecture	Transformer-based matching (if selected)	1-2 days
XFeat	Feature detection/matching concepts	1 day
ESKF	Kalman filtering, quaternion math, multi-rate fusion	3-5 days
FastAPI + SSE	Async Python, ASGI, SSE protocol	1 day
GeoHash spatial indexing	Geospatial concepts	0.5 days
Jetson deployment	JetPack, power modes, thermal management	2-3 days

Development Environment

Environment	Purpose	Setup
Developer machine (x86_64, GPU)	Development, unit testing, model export	Docker with CUDA + TensorRT
Jetson Orin Nano Super	Integration testing, benchmarking, deployment	JetPack 6.2.2 flashed, SSH access

Code should be developed and unit-tested on x86_64, then deployed to Jetson for integration/performance testing. cuVSLAM and TensorRT engines are aarch64-only — mock these in x86_64 tests.

13 KiB Raw Blame History