gps-denied-onboard/_docs/00_research/00_ac_assessment.md

Acceptance Criteria Assessment

Scope

• Mode: Mode A Phase 1 — acceptance criteria and restrictions assessment.
• Problem boundary: fixed-wing UAV, downward fixed navigation camera, high-rate IMU over MAVLink, GPS may be denied or spoofed, offline satellite tile cache, eastern/southern Ukraine operating region, onboard Jetson Orin Nano Super.
• Novelty sensitivity: High for onboard AI/VPR, Jetson/TensorRT, and ArduPilot EKF integration; Medium for photogrammetry/GSD and slippy-map math; Low for high-level VO/INS principles.
• Input data observed: 60 nadir-like agricultural/steppe frames with sparse landmarks, ground-truth frame-center coordinates, two Google Maps reference screenshots, no real IMU trace.

Acceptance Criteria

Criterion	Our Values	Researched Values	Cost/Timeline Impact	Status
AC-1.1 / AC-1.2 frame-center accuracy	80% <50 m, 50% <20 m	Recent UAV GNSS-denied visual localization work reports roughly 15-20 m mean localization error in favorable datasets, while fixed-wing satellite-aided VO demonstrates drift reduction over >17 km rather than a universal meter-level guarantee. The thresholds are plausible if satellite anchors succeed regularly, but they require dataset-specific validation.	Keep, but budget a full evaluation harness and Monte Carlo replay early.	Keep
Expected-results threshold for 20 m accuracy	AC says 50%; expected_results/results_report.md says 60%	This is an internal contradiction, not a research issue. Tests will enforce the stricter 60% if left as-is.	Low effort to correct, high risk if unnoticed because it changes pass/fail.	Needs decision
AC-1.3 VO drift between anchors	<100 m VO-only, <50 m IMU-fused	Visual-inertial odometry drift below ~1% of distance is realistic in some outdoor tests; at 60 km/h and 3 Hz, anchor cadence and turn handling dominate. The criterion is feasible if satellite relocalization prevents long VO-only intervals.	Keep, but require explicit anchor-age limits in tests.	Keep
AC-1.4 confidence score	95% covariance ellipse + source label	MAVLink GPS_INPUT supports horizontal/vertical accuracy fields; producing calibrated covariance is feasible but must be statistically validated against false-position budgets.	Medium effort; requires calibration and reliability testing, not just API plumbing.	Keep
AC-2.1 registration rate	>95% normal segments	Plausible only under the definition already scoped: nadir, daylight, season-matched, >=40% overlap. Sparse agricultural frames and active-conflict scene changes make this too optimistic outside that scope.	Keep the scoped definition; add separate degraded-condition metrics later.	Keep
AC-2.2 reprojection error	<1.0 px VO, <2.5 px UAV-satellite	VO MRE <1 px is plausible after calibration. Cross-domain UAV-satellite <2.5 px is aggressive because appearance, season, scale, and orthorectification errors dominate.	Keep as a stretch or lab-gated target; do not make it the only production acceptance signal.	Modify
AC-3.1 outliers / AC-3.2 sharp turns / AC-3.3 disconnected segments	350 m outliers; <5% overlap turns; >=3 disconnected segments	These are operationally important and realistic as failure modes. They cannot be solved by VO alone; they require VPR/global retrieval + EKF gating.	High implementation risk; keep because they drive the right architecture.	Keep
AC-3.4 relocalization request	>=3 failed frames and >=2 s	At 3 fps, this means the frame-count trigger can fire around 1 s but the time trigger waits 2 s. The combined trigger is coherent.	Low implementation cost; important for operator workflow.	Keep
AC-4.1 latency	<400 ms p95 capture-to-FC output, <=10% frame drop	Jetson Orin Nano Super has official 67 TOPS sparse / 8 GB / 25 W mode. DINOv2-base TensorRT examples can run at high FPS, but DINOv2-S GitHub issue data shows ~22-23 ms GPU compute and limited INT8 gain; SuperPoint/LightGlue data on Orin Nano is thin. End-to-end 400 ms is plausible only with conditional VPR and heavy offline preprocessing.	High risk; needs prototype benchmark before implementation decomposition.	Keep with benchmark gate
AC-4.2 memory	<8 GB shared LPDDR5	Feasible only if model set is small, descriptors are tiled/indexed, TensorRT engines are prebuilt, and raw frames are not retained.	Medium risk; memory profiling must be an acceptance test.	Keep
AC-4.3 MAVLink output	v1 GPS_INPUT only; ODOMETRY disabled	ArduPilot/MAVProxy documents GPS1_TYPE=14 for MAVLink GPS input. MAVLink says GPS_INPUT is raw sensor input, not global position estimate, so covariance/accuracy fields matter. ArduPilot issue #30076 shows external-nav/GPS fusion risk was real and is now marked closed; v1 GPS_INPUT-only remains conservative, but ODOMETRY should be re-evaluated against the exact ArduPilot release before v1.1.	Keep v1 GPS_INPUT-only; add version-pinned SITL gate before enabling ODOMETRY.	Keep
AC-5 startup/failsafe	TTFF <30 s; fail if no estimate >3 s	Cold-start <30 s is plausible only if TensorRT engines and tile indexes are loaded without first-run compilation. >3 s failover matches the 3 fps / sharp-turn recovery intent.	Medium effort; requires boot profiling and watchdog tests.	Keep
AC-6 QGC telemetry	1-2 Hz downsampled summary, commands via MAVLink	Feasible and appropriately scoped; keeps high-rate data local.	Low/medium depending on command dialect choice.	Keep
AC-7 object localization	Frame-center-equivalent in level flight; publish bank/pitch bound outside level flight	This is realistic because AI-camera bank/pitch is unavailable. The bound altitude * sin(angle) correctly exposes the limitation instead of hiding it.	Keep; API must always return accuracy bound.	Keep
AC-8.1 imagery resolution	>=0.5 m/px, ideal 0.3 m/px	Commercial 30 cm imagery is realistic through providers such as Airbus Pléiades Neo and Vantor/Maxar-class constellations, but upstream availability/freshness in active-conflict areas remains a service risk.	Keep; dependency belongs to Suite Satellite Service SLA.	Keep
AC-8.2 / AC-NEW-6 freshness	<6 months active sectors, <12 months stable sectors	Operationally justified. It will reduce false positives but increases mission planning/sourcing burden.	Medium/high cost delegated to Satellite Service and cache metadata.	Keep
AC-8.3 preprocessing	Offline cache with descriptors	Correct architectural choice for onboard latency and no in-flight network dependency.	Medium offline compute/storage cost.	Keep
AC-8.4 / AC-8.5 tile write-back and no raw frame storage	Persist generated tiles, not raw frames	Good storage control and privacy/security posture. Requires tile quality gates to prevent poisoning.	Medium/high because write-back quality scoring is non-trivial.	Keep
AC-8.6 VPR chunks	600-800 m chunks, 40-50% overlap, conditional VPR	Fits the latency evidence: online VPR should be event-triggered, not per-frame. Multi-scale chunks are appropriate for active-conflict scene changes.	Keep; index size and load time need measurement.	Keep
AC-NEW-3 FDR	<=64 GB / flight	Feasible because raw frames are excluded. Must include rollover logging so no payload class disappears silently.	Low/medium.	Keep
AC-NEW-4 false-position budget	P(error >500 m) <0.1%, P(error >1 km) <0.01% per flight	This is the right safety metric, but it cannot be proven from unit tests alone. It requires calibrated covariance, outlier rejection, and Monte Carlo over representative datasets.	High validation cost; keep as safety gate.	Keep
AC-NEW-5 environmental envelope	-20 C to +50 C, 25 W for 8 h, no throttling	DO-160G-style environmental testing is a recognized airborne-equipment pattern; applying it to a small UAV is conservative. The thermal target is realistic only with designed cooling, not a bare dev kit.	Hardware integration and chamber testing required.	Keep
AC-NEW-7 cache-poisoning budget	Tile misalignment >30 m <1%, >100 m <0.1%	Correct risk to control, but the thresholds depend on covariance calibration and multi-flight voting. Single-flight promotion in active sectors should remain exceptional and heavily gated.	High validation cost; keep with explicit service-side voting tests.	Keep

Restrictions Assessment

Restriction	Our Values	Researched Values	Cost/Timeline Impact	Status
UAV type and camera pose	Fixed-wing, fixed downward navigation camera	This fits visual odometry and orthorectification assumptions better than gimbaled imagery. Lack of gimbal stabilization means bank/pitch compensation must come from FC attitude.	Keep; camera/FC time sync is critical.	Keep
Operational area	Eastern/southern Ukraine, steppe/agricultural terrain	Sample images confirm low-texture fields, tree lines, roads, sparse structures, and seasonal/appearance risk. This is a hard test for cross-view matching.	Keep; research and tests must avoid urban-only datasets.	Keep
Camera spec inconsistency	Problem text says ~6200x4100; restrictions say 20 MP 5472x3648; input data says 26 MP 6252x4168 at 400 m	The repo distinguishes target camera from example data, but the solution needs one authoritative v1 navigation camera/lens for GSD, intrinsics, and latency.	Needs confirmation or the plan will produce conflicting calibration tasks.	Needs decision
GSD target	10-20 cm/px at <=1 km AGL	Formula GSD = sensor width * altitude / (focal length * image width) supports tuning GSD by lens choice. With 23.5 mm sensor width, 6252 px, 25 mm lens: ~6 cm/px at 400 m and ~15 cm/px at 1 km.	Keep; lens selection must be locked before calibration tasks.	Keep
Tile zoom statement	"slippy-XYZ z=20 (~30 cm/px, 512x512)"	OSM zoom table gives z20 = 0.149 m/px at equator for 256 px tiles, about 0.10 m/px at latitude 48; 512 px tile conventions can shift effective zoom. The stated "z=20 ~30 cm/px" is inconsistent unless this uses a custom provider convention.	Needs correction before storage and cache-index tasks.	Needs decision
Satellite provider boundary	Onboard consumes Azaion Suite Satellite Service cache only	Correct. Commercial provider contracts, tasking, licensing, and freshness are outside this build but must become Satellite Service SLAs.	Keep.	Keep
Hardware	Jetson Orin Nano Super, 8 GB, 25 W	Official specs support the constraint, but thermal and memory limits are tight for multi-model CV.	Keep; prototype benchmark is mandatory.	Keep
IMU source	High-rate IMU from FC via MAVLink	Feasible. Test gap remains because current input data has no real IMU trace. Synthetic/SITL IMU is acceptable for early tests but not final validation.	Add real or representative flight IMU capture plan.	Modify
Autopilot	ArduPilot only, QGroundControl only	Good scope control. ArduPilot GPS_INPUT path is documented; EKF source/fusion behavior must be version-pinned.	Keep.	Keep
Storage	~10 GB persistent tile cache plus 64 GB FDR	Feasible if tile zoom/resolution is corrected and raw frames remain excluded.	Keep after zoom correction.	Keep
No raw photo storage	Tiles only; failure thumbnails <=0.1 Hz	Strong restriction for storage and security; compatible with FDR cap.	Keep.	Keep

Key Findings

1. Most existing ACs are directionally sound and should remain, especially the safety budgets, covariance reporting, freshness gates, GPS_INPUT-only v1 scope, and no-raw-frame storage policy.
2. Three items should be resolved before planning proceeds: AC-1.2's 50% vs expected-results 60% contradiction, the authoritative v1 navigation camera/lens, and the tile zoom/resolution convention.
3. The 400 ms p95 latency target is plausible only if VPR is conditional and reference descriptors are precomputed; it should be treated as a benchmark gate before implementation tasks are finalized.
4. Cross-domain reprojection error <2.5 px is an aggressive lab target. Production acceptance should rely on position error, covariance calibration, registration success, and false-position rejection, not pixel error alone.
5. The public evidence supports satellite-aided visual localization as feasible, but the project's operating region is harder than many benchmark datasets because farmland/steppe imagery has sparse stable features and active-conflict areas change quickly.

Recommended Adjustments Before Phase 2

Decision	Recommendation	Rationale
AC-1.2 threshold mismatch	Align expected_results/results_report.md with AC-1.2 at 50%, unless the user intentionally wants a 60% stretch target.	Avoid hidden test/spec disagreement.
Navigation camera	Confirm ADTi 20MP 20L V1 + selected lens as v1 target; keep 26 MP images as sample/test data only.	Calibration, GSD, FOV, latency, and orthorectification depend on intrinsics.
Tile zoom	Replace "z=20 (~30 cm/px, 512x512)" with an explicit cache convention: provider pixel size, tile matrix, tile dimension, CRS, and latitude-adjusted resolution.	Prevent wrong storage estimates and matcher scale assumptions.
IMU validation data	Add a requirement to obtain real FC MAVLink IMU logs or approved SITL-generated IMU traces before final acceptance.	Current sample data cannot validate fusion, covariance, or latency-to-FC behavior.
Cross-domain MRE	Mark <2.5 px as lab/stretched diagnostic; keep position/covariance as production gate.	Pixel error does not fully capture orthorectification, map freshness, and geodetic error.

Sources

• ArduPilot MAVProxy GPSInput documentation, accessed 2026-04-29: https://ardupilot.org/mavproxy/docs/modules/GPSInput.html
• MAVLink common message spec, GPS_INPUT, accessed 2026-04-29: https://mavlink.io/en/messages/common.html#GPS_INPUT
• ArduPilot issue #30076, ExternalNav + GPS fusion behavior, accessed 2026-04-29: https://github.com/ArduPilot/ardupilot/issues/30076
• NVIDIA Jetson Orin Nano Super technical blog, published 2024-12-17, accessed 2026-04-29: https://developer.nvidia.com/blog/nvidia-jetson-orin-nano-developer-kit-gets-a-super-boost
• NVIDIA JetPack 6.2 Super Mode benchmarks, accessed 2026-04-29: https://developer.nvidia.com/blog/nvidia-jetpack-6-2-brings-super-mode-to-nvidia-jetson-orin-nano-and-jetson-orin-nx-modules/
• NVIDIA TensorRT issue #4348, DINOv2-S Jetson Orin TensorRT measurements, accessed 2026-04-29: https://github.com/NVIDIA/TensorRT/issues/4348
• MDPI Applied Sciences 2024, "Visual Odometry in GPS-Denied Zones for Fixed-Wing Unmanned Aerial Vehicle with Reduced Accumulative Error Based on Satellite Imagery", accessed 2026-04-29: https://www.mdpi.com/2076-3417/14/16/7420
• MDPI Sensors 2024, "A Cross-View Geo-Localization Algorithm Using UAV Image and Satellite Image", accessed 2026-04-29: https://www.mdpi.com/1424-8220/24/12/3719
• Airbus Pléiades Neo official imagery page, accessed 2026-04-29: https://www.airbus.com/en/pleiades-neo-satellite-imagery
• Vantor/Maxar WorldView Legion page, accessed 2026-04-29: https://www.maxar.com/worldview-legion
• OpenStreetMap Wiki zoom levels, accessed 2026-04-29: https://wiki.openstreetmap.org/wiki/Zoom_levels
• FAA AC 21-16G / RTCA DO-160G reference, accessed 2026-04-29: https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_21-16G.pdf