gps-denied-onboard/_docs/00_problem/acceptance_criteria.md

Acceptance Criteria

Last revised 2026-05-07 (cleanup pass: stripped algorithm/library/parameter implementation details; renamed source label vo_extrapolated → visual_propagated; broadened FC scope to ArduPilot + iNav). See git history for prior versions.

Position Accuracy

• AC-1.1 — Frame-center GPS within 50 m of true GPS for ≥80% of normal-flight photos.
• AC-1.2 — Frame-center GPS within 20 m of true GPS for ≥50% of normal-flight photos.
• AC-1.3 — Cumulative drift between two consecutive satellite-anchored fixes: <100 m visual-only / <50 m with IMU fused. Measured as ‖propagated centre − next anchor centre‖ at anchor fix. Every estimate carries last_satellite_anchor_age_ms; validation binned by anchor age. The solution must define the max anchor age beyond which estimates degrade to visual_propagated / dead_reckoned with monotonically growing covariance.
• AC-1.4 — Each estimate reports: 95% covariance ellipse semi-major axis (m) AND a label {satellite_anchored, visual_propagated, dead_reckoned}.

Image Processing Quality

• AC-2.1a — Frame-to-frame registration: succeeds for >95% of normal flight segments (defined: nadir ±10° bank/pitch, ≥40% prior-frame overlap, daytime, usable texture, no full visual blackout).
• AC-2.1b — Satellite-anchor registration: measured separately from AC-2.1a; must satisfy AC-1.1/1.2 accuracy, AC-2.2 cross-domain MRE, AC-8.2 freshness, AC-8.6 retrieval behaviour.
• AC-2.2 — Mean Reprojection Error: <1.0 px frame-to-frame; <2.5 px satellite-anchored cross-domain.

Resilience & Edge Cases

• AC-3.1 — Tolerate up to 350 m outliers between two consecutive photos (airframe tilt up to ±20°).
• AC-3.2 — Tolerate sharp turns: <5% overlap, <200 m drift, <70° heading change. Sharp-turn frames may fail frame-to-frame registration; recovery via satellite-reference re-localization.
• AC-3.3 — Handle ≥3 disconnected segments per flight via satellite-reference re-localization. Core capability, not degraded mode.
• AC-3.4 — On ≥3 consecutive frames AND ≥2 s without a position, request operator re-loc via telemetry; continue dead-reckoned propagation; FC uses last known + IMU extrapolation.
• AC-3.5 — Visual blackout + spoofed GPS (clouds/occlusion/whiteout while FC reports GPS denial/spoof):
  • Switch label to {dead_reckoned} within ≤1 processed frame OR ≤400 ms.
  • Reject spoofed GPS as estimator input.
  • Propagate from last trusted state + FC IMU/attitude/airspeed/altitude until visual or satellite anchoring recovers.
  • Covariance grows monotonically.
  • horiz_accuracy field of the GPS message to the FC must not under-report the 95% covariance semi-major axis.
  • VISUAL_BLACKOUT_IMU_ONLY STATUSTEXT to QGroundControl at 1–2 Hz.

Real-Time Onboard Performance

• AC-4.1 — End-to-end latency (camera capture → GPS to FC) <400 ms p95. Up to ~10% frames may drop under sustained load.
• AC-4.2 — Memory <8 GB shared on Jetson Orin Nano Super.
• AC-4.3 — FC output contract: WGS84 coordinates over MAVLink, via the standard external-positioning message type(s) accepted by ArduPilot and iNav. Honest covariance carried in the field the FC uses for outlier rejection (under-reported covariance is a defect, see AC-NEW-4). Source-label semantics per AC-1.4. Implementation may emit a primary GPS-substitute message and, when the estimator delivers full 6-DoF covariance + quality above a configured threshold, an optional auxiliary external-odometry message. Per-FC message selection and FC-side parameter wiring are design decisions.
• AC-4.4 — Estimates streamed frame-by-frame; no batching/delay.
• AC-4.5 — System may refine prior estimates and emit corrections.

Startup & Failsafe

• AC-5.1 — Initialise from FC EKF's last valid GPS + IMU-extrapolated position at GPS denial.
• AC-5.2 — On >3 s without estimate, FC falls back to IMU-only dead reckoning; system logs failure. Verify in production param sets of each supported FC (ArduPilot Plane SITL + iNav SITL or equivalent).
• AC-5.3 — On companion reboot mid-flight, re-initialise from FC's current IMU-extrapolated position. Cold-start TTFF in AC-NEW-1.

Ground Station & Telemetry

• AC-6.1 — Position estimates + confidence stream to QGroundControl over MAVLink at 1–2 Hz downsampled (high-rate stays on local FDR).
• AC-6.2 — GCS may send commands (e.g., operator re-loc hint) via standard MAVLink (STATUSTEXT, NAMED_VALUE_FLOAT) or a custom dialect.
• AC-6.3 — Output coordinates in WGS84.

Object Localization (AI Camera)

• AC-7.1 — AI systems may request GPS for AI-camera-detected objects. Accuracy consistent with frame-center accuracy in level flight (bank/pitch <5°). In maneuvering flight, error bounded by altitude × |sin(unknown_bank_or_pitch)| and that bound is published alongside the estimate.
• AC-7.2 — Object coordinates computed trigonometrically from current UAV position, AI-camera gimbal angle, zoom, and altitude. Flat-terrain assumption.

Satellite Reference Imagery

• AC-8.1 — Imagery via Azaion Suite Satellite Service (offline cache interface; no direct commercial-provider calls). Cache-interface resolution ≥0.5 m/px, ideally 0.3 m/px.
• AC-8.2 — Tile freshness: <6 mo (active-conflict sectors), <12 mo (stable rear). Older → reject or downgrade (AC-NEW-6).
• AC-8.3 — Imagery pre-loaded onto companion before flight; offline preprocessing time not time-critical. Pre-extracted descriptors/indices count against the cache budget unless explicitly carved out.
• AC-8.4 — Mid-flight tile generation: continuously orthorectify nav-camera frames into basemap-projected tiles, deduplicated (latest/highest-quality wins). Upload to Service on landing. Each uploaded tile carries quality metadata sufficient for the Service's ingest pipeline (AC-NEW-7).
• AC-8.5 — No raw nav-camera or AI-camera frames retained in normal operation; tiles are the only persistent imagery. Forensic exception: ≤0.1 Hz thumbnail log of frames that failed tile generation, within FDR budget (AC-NEW-3).
• AC-8.6 — Satellite-anchor relocalization robustness:
  • Scale-ratio: any UAV-frame ground footprint at the deployment altitude band must be retrievable from the cache regardless of internal tiling/indexing.
  • Scene change in active-conflict sectors: cratering / building destruction / road realignment must not collapse retrieval recall, measured against a labelled change-pair dataset over season-matched tiles. No satellite_anchored label on stale-tile match (per AC-NEW-6).
  • Compute & latency: relocalization must remain inside AC-4.1 latency + AC-4.2 memory budgets under both steady-state and re-loc-trigger workloads.

Additional AC

AC-NEW-1 — Cold-start TTFF

Statement. From companion boot, first valid external-position MAVLink frame <30 s p95, given an IMU-extrapolated initial position from FC EKF. Why. Mid-flight reboot is realistic on 8 h missions; FC dead-reckons during the gap, ~500 m drift max at 60 km/h. Validation. Cold-boot 50× with simulated FC pose; measure boot → first frame; pass = 95th percentile <30 s.

AC-NEW-2 — Spoofing-promotion latency

Statement. When FC signals GPS denial/spoof, promote onboard estimate to FC's primary position source within <3 s p95. Why. Without this, FC may follow a spoofed source while a valid onboard estimate sits idle; 3 s rides out one-frame anomalies but blocks malicious heading changes. Validation. SITL on each supported FC (ArduPilot Plane + iNav, production param sets): inject false GPS, measure spoof onset → promotion; pass = 95th percentile <3 s on both.

AC-NEW-3 — Flight Data Recorder

Statement. Per flight, retain to NVM: per-frame estimates with covariance + source-label; FC IMU traces (full rate); all emitted external-position MAVLink frames; raw MAVLink stream (tlog); system health (CPU/GPU/temp/throttle); mid-flight tiles (AC-8.4); ≤0.1 Hz thumbnail log of failed tile-gen frames. No raw nav-cam/AI-cam frames (AC-8.5). Cap 64 GB / flight; oldest segment dropped first on rollover. Why. Tiles + telemetry + IMU reproduce the mission, feed next mission's cache (AC-8.4), explain false-position events (AC-NEW-4). Raw frames are large + redundant once tiles exist. Validation. 8 h synthetic load (3 Hz nav frames replayed); assert FDR ≤64 GB; no payload class silently dropped without a logged rollover.

AC-NEW-4 — False-position safety budget

Statement. Per flight: P(error >500 m) <0.1 %, P(error >1 km) <0.01 %. Why. A single 1-km-off frame can fly the UAV outside the geofence; covariance carried in the MAVLink message is the FC's only defense. Validation. Monte Carlo over a public aerial-localization dataset (e.g. AerialVL S03) + own recorded flights; report error CDF; pass = both probabilities below budget across ≥100 flights.

AC-NEW-5 — Operational environmental envelope

Statement. Operating temp −20 °C to +50 °C; vibration/shock per RTCA DO-160G low-altitude UAV-class. Cooling sustains 25 W at the upper temp for the full 8-hour duty cycle without throttling. Why. Without this, all latency/accuracy AC are conditional on a benign thermal day; +35 °C bay temps cause Jetson to throttle to 15 W, collapsing the 400 ms latency budget. Validation. Hot-soak: 25 W @ +50 °C for 8 h, no throttle. Cold-soak: −20 °C cold-start within AC-NEW-1.

AC-NEW-6 — Imagery freshness enforcement

Statement. System rejects (or downgrades) any tile whose capture date violates AC-8.2. Mid-flight tiles (AC-8.4) not yet uploaded are timestamped current and treated as fresh. Why. Stale tiles are the dominant cross-view-matching failure mode in active-conflict sectors; a confident match on a stale tile is worse than no match. Validation. Inject synthetic-age tiles; verify rejection/decay matches spec; verify stale-tile match never produces satellite_anchored.

AC-NEW-7 — Cache-poisoning safety budget

Statement. Per flight, across all onboard tiles written (AC-8.4): P(geo-misalign >30 m) <1 %, P(>100 m) <0.1 %. Why. Onboard tiles feed back into the Service basemap (AC-8.4). A bad onboard pose with optimistic covariance writes a misaligned tile that becomes the next flight's anchor — cross-flight error compounding that AC-NEW-4 doesn't capture. External-dependency note. The Suite Satellite Service is expected to operate a multi-flight ingest-side voting layer that gates onboard-tile promotion to "trusted basemap" until multiple independent flights agree on geo-alignment. Voting algorithm is the Service's concern; onboard's job (AC-8.4) is to publish per-tile quality metadata sufficient for that layer. End-to-end AC-NEW-7 evidence depends on this Service contract. Validation. Multi-flight Monte Carlo replay over public datasets (e.g. AerialVL, AerialExtreMatch) + own flights, with synthetic over-confidence injection (deflate covariance ×1.5–3): assert both probabilities below budget across ≥100 flights. Independently exercise the Service-side voting contract.

AC-NEW-8 — Visual blackout + GPS spoofing degraded mode

Statement. When the navigation camera is fully unusable AND FC reports GPS denial/spoof:

• continue emitting external-position MAVLink frames from IMU-only propagation for ≤30 s after the last trusted anchor (or until covariance trips fail threshold);
• label every estimate {dead_reckoned}; degrade MAVLink fix-quality to "2D fix or worse" when 95% covariance semi-major axis >100 m;
• escalate to "no fix" (horiz_accuracy=999.0) + VISUAL_BLACKOUT_FAILSAFE STATUSTEXT when 95% covariance >500 m OR blackout >30 s without a trusted re-anchor;
• never promote spoofed real-GPS back into the estimator unless FC GPS health stable + non-spoofed for ≥10 s AND a visual/satellite consistency check has succeeded. Why. During cloud/whiteout + spoofing, no honest correction is available; only safe behaviour is IMU-only dead reckoning with rapidly-growing uncertainty, never pretending stale visual or spoofed GPS remains valid. Validation. SITL/replay on each FC: inject 5 s / 15 s / 35 s blackouts while spoofing GPS; assert mode transition ≤400 ms, spoofed GPS ignored, covariance grows monotonically, MAVLink fields degrade at thresholds, recovery only via trusted anchor or 10-s GPS-health + visual-consistency gate.