azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-21 22:31:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'try02' into dev

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-05 05:51:29 +03:00
parent a7a73c01ce e7eaefff8b
commit 44c19ed117
259 changed files with 19947 additions and 2778 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/00_problem/acceptance_criteria.md
+160 -35
View File
@@ -1,50 +1,175 @@
# Position Accuracy
# Acceptance Criteria
- The system should determine GPS coordinates of frame centers for 80% of photos within 50m error compared to real GPS
- The system should determine GPS coordinates of frame centers for 60% of photos within 20m error compared to real GPS
- Maximum cumulative VO drift between satellite correction anchors should be less than 100 meters
- System should report a confidence score per position estimate (high = satellite-anchored, low = VO-extrapolated with drift)
> **Last revised**: 2026-05-01 (Phase 1 AC/restrictions assessment clarifications).
> Changes vs. previous version (2026-04-25): AC-1.2 split into hard-floor + stretch; AC-1.4 made quantitative; AC-2.2 split per pipeline stage; AC-3.4 dual-trigger; AC-4.3 autopilot-pinned; AC-5.2 N pinned; AC-7.1 scoped to level flight; AC-8.2 freshness by sector; six new AC added (AC-NEW-1 … AC-NEW-6).
> Changes 2026-04-26: AC-4.3 extended to dual-channel hybrid (GPS_INPUT primary + ODOMETRY auxiliary); AC-8.6 added (VPR retrieval-unit + change-robustness); AC-NEW-7 added with confirmed numeric thresholds (cache-poisoning safety budget).
> Changes 2026-04-29: AC-3.5 and AC-NEW-8 added for temporary visual blackout/cloud occlusion during GPS spoofing, including IMU-only degraded navigation, covariance growth, and failover limits.
> Changes 2026-05-01: AC-1.3 anchor-age reporting clarified; AC-2.1 split so the >95% rate applies to VO registration, not every satellite re-anchor; AC-5.2 and AC-NEW-2 now require ArduPilot Plane SITL trigger verification; AC-8.3 storage accounting and AC-NEW-7 Satellite Service ownership clarified.
# Image Processing Quality
## Position Accuracy
- Image Registration Rate > 95% for normal flight segments. The system can find enough matching features to confidently calculate the camera's 6-DoF pose and stitch that image into the trajectory
- Mean Reprojection Error (MRE) < 1.0 pixels
- **AC-1.1** — The system shall determine GPS coordinates of frame centers within **50 m** of true GPS for **≥80%** of photos in normal flight segments.
- **AC-1.2** — The system shall determine GPS coordinates of frame centers within **20 m** of true GPS for **≥50%** of photos in normal flight segments.
- **AC-1.3** — Maximum cumulative VO drift between two consecutive satellite-anchored fixes shall be **<100 m** (VO-only fallback) or **<50 m** (when IMU is fused). Drift is measured as ‖VO-extrapolated centre − next anchor centre‖ at the moment of the anchor fix. Every emitted estimate shall include `last_satellite_anchor_age_ms`; validation results shall be binned by anchor age, and the solution draft must define the maximum anchor age after which estimates are treated as degraded (`vo_extrapolated` or `dead_reckoned`) with monotonically growing covariance.
- **AC-1.4** — The system shall report a **quantitative confidence score** per position estimate, comprising:
- the 95% covariance ellipse semi-major axis in meters, AND
- a categorical label `{satellite_anchored, vo_extrapolated, dead_reckoned}`.
# Resilience & Edge Cases
## Image Processing Quality
- The system should correctly continue work even in the presence of up to 350m outlier between 2 consecutive photos (due to tilt of the plane)
- System should correctly continue work during sharp turns, where the next photo doesn't overlap at all or overlaps less than 5%. The next photo should be within 200m drift and at an angle of less than 70 degrees. Sharp-turn frames are expected to fail VO and should be handled by satellite-based re-localization
- System should operate when UAV makes a sharp turn and next photos have no common points with previous route. It should figure out the location of the new route segment and connect it to the previous route. There could be more than 2 such disconnected segments, so this strategy must be core to the system
- In case the system cannot determine the position of 3 consecutive frames by any means, it should send a re-localization request to the ground station operator via telemetry link. While waiting for operator input, the system continues attempting VO/IMU dead reckoning and the flight controller uses last known position + IMU extrapolation
- **AC-2.1** — Image registration rate is split by registration type:
- **AC-2.1a — VO registration**: frame-to-frame visual registration shall succeed for **>95%** of normal flight segments (defined as: nadir flight ±10° bank / pitch, ≥40% overlap with prior frame, daytime, usable texture, no full visual blackout).
- **AC-2.1b — Satellite-anchor registration**: cross-domain UAV-photo to satellite/cache registration is measured separately and is not hidden inside AC-2.1a. Satellite anchoring must satisfy AC-1.1 / AC-1.2 position accuracy, AC-2.2 cross-domain MRE, AC-8.2 freshness, and AC-8.6 retrieval behavior on season-matched tiles.
- **AC-2.2** — Mean Reprojection Error (MRE):
- **<1.0 px** for VO frame-to-frame homography on overlapping aerial pairs;
- **<2.5 px** for satellite-anchored cross-domain (UAV photo ↔ ortho satellite tile) registration.
# Real-Time Onboard Performance
## Resilience & Edge Cases
- Less than 400ms end-to-end per frame: from camera capture to GPS coordinate output to the flight controller (camera shoots at ~3fps)
- Memory usage should stay below 8GB shared memory (Jetson Orin Nano Super — CPU and GPU share the same 8GB LPDDR5 pool)
- The system must output calculated GPS coordinates directly to the flight controller via MAVLink GPS_INPUT messages (using MAVSDK)
- Position estimates are streamed to the flight controller frame-by-frame; the system does not batch or delay output
- The system may refine previously calculated positions and send corrections to the flight controller as updated estimates
- **AC-3.1** — The system shall correctly continue work in the presence of up to **350 m** outliers between two consecutive photos (caused by airframe tilt up to ±20°).
- **AC-3.2** — The system shall correctly continue work during sharp turns where the next photo overlaps **<5%** with the previous, drifts **<200 m**, and changes heading **<70°**. Sharp-turn frames are expected to fail VO and shall be handled by satellite-based re-localization (place recognition over the satellite tile cache).
- **AC-3.3** — The system shall handle **≥3 disconnected segments** per flight, connecting each new segment to the previous trajectory via global descriptor retrieval + RANSAC pose-graph relocalization. This is a core capability, not a degraded mode.
- **AC-3.4** — When the system cannot determine position for **≥3 consecutive frames AND ≥2 s**, it shall send a re-localization request to the ground station via telemetry. While waiting, it continues VO/IMU dead reckoning and the flight controller uses last known position + IMU extrapolation.
- **AC-3.5** — During temporary **visual blackout** where the navigation camera provides no usable ground signal (e.g., clouds/occlusion/whiteout) while GPS is denied or spoofed, the system shall switch to `{dead_reckoned}` within **≤1 processed frame OR ≤400 ms**, reject the spoofed GPS as an estimator input, and propagate position solely from the last trusted state + flight-controller IMU/attitude/airspeed/altitude inputs until visual or satellite anchoring recovers. During this mode, covariance shall grow monotonically, `GPS_INPUT.horiz_accuracy` shall not under-report the 95% covariance semi-major axis, and QGroundControl shall receive a `VISUAL_BLACKOUT_IMU_ONLY` status at **1–2 Hz**.
# Startup & Failsafe
## Real-Time Onboard Performance
- The system initializes using the last known valid GPS position from the flight controller before GPS denial begins
- If the system completely fails to produce any position estimate for more than N seconds (TBD), the flight controller should fall back to IMU-only dead reckoning and the system should log the failure
- On companion computer reboot mid-flight, the system should attempt to re-initialize from the flight controller's current IMU-extrapolated position
- **AC-4.1** — End-to-end latency from camera capture to GPS coordinate output to the flight controller shall be **<400 ms p95**. Up to ~10% of frames may be dropped under sustained load (skip-allowed). Heavy global VPR / cross-domain re-ranking shall be conditional, not part of the steady-state per-frame path, unless profiling proves the full path stays inside the latency and memory budgets on the target Jetson.
- **AC-4.2** — Memory usage shall remain below **8 GB** shared on Jetson Orin Nano Super (CPU and GPU share the same 8 GB LPDDR5 pool).
- **AC-4.3** — The system shall output its position estimate to the flight controller via **two parallel MAVLink channels**, both emitted by **pymavlink** (general telemetry uses MAVSDK):
- **Primary**: `GPS_INPUT` targeting **ArduPilot** with `GPS1_TYPE=14` (MAVLink GPS substitute). Matches the "replacement for the GPS module" framing of the build.
- **Auxiliary** (when the EKF emits a fix with full 6-DoF covariance and quality > VISO_QUAL_MIN): `ODOMETRY` so EKF3 can fuse the richer covariance + native yaw error + quality field. ArduPilot's own dev docs designate ODOMETRY as the preferred external-nav channel for non-GPS substitution; we hybridise to keep AC-4.3's GPS-substitute framing while not throwing away the covariance fidelity that AC-NEW-4 depends on.
- FC source priorities are configured so GPS_INPUT remains the failover path if ODOMETRY trips a parameter gate.
- **v1 scope clause (added 2026-04-26 — see solution_draft03 finding M-30)**: v1 ships **GPS_INPUT only**; the ODOMETRY auxiliary channel is intentionally **disabled** in v1 because feeding both `GPS_INPUT` and `ODOMETRY` for overlapping axes triggers ArduPilot EKF3 double-fusion bugs (issues #30076 / #32506). `EK3_SRC1_*=GPS+Compass`; ODOMETRY emission re-enables in v1.1 once F-T9 SITL confirms PR #30080-class clean source-switching. Tests therefore assert v1 emits GPS_INPUT only and that ODOMETRY is *intentionally absent* on the wire.
- (Decision rationale: MAVSDK has no native GPS_INPUT support — see `_docs/00_research/00_ac_assessment.md` Q-1; ODOMETRY hybrid rationale — see Mode B finding M-1 in `_docs/00_research/02_fact_cards.md`; v1 single-channel rationale — see Mode B round-2 finding M-30 in `_docs/00_research/02_fact_cards.md` / solution_draft03.)
- **AC-4.4** — Position estimates are streamed to the flight controller frame-by-frame; the system shall not batch or delay output.
- **AC-4.5** — The system may refine previously calculated positions and send corrections to the flight controller as updated estimates.
# Ground Station & Telemetry
## Startup & Failsafe
- Position estimates and confidence scores should be streamed to the ground station via telemetry link for operator situational awareness
- The ground station can send commands to the onboard system (e.g., operator-assisted re-localization hint with approximate coordinates)
- Output coordinates in WGS84 format
- **AC-5.1** — The system shall initialise using the last known valid GPS position from the flight controller's EKF, plus IMU-extrapolated position at the moment of GPS denial.
- **AC-5.2** — If the system fails to produce any position estimate for **>3 s**, the flight controller shall fall back to IMU-only dead reckoning and the system shall log the failure. Because ArduPilot failsafe timing depends on vehicle type and parameters, this fallback behavior must be verified specifically in ArduPilot Plane SITL with the production parameter set; Copter defaults are reference evidence only.
- **AC-5.3** — On companion computer reboot mid-flight, the system shall attempt to re-initialise from the flight controller's current IMU-extrapolated position. See AC-NEW-1 for the cold-start time-to-first-fix budget.
# Object Localization
## Ground Station & Telemetry
- Other onboard AI systems can request GPS coordinates of objects detected by the AI camera
- The GPS-Denied system calculates object coordinates trigonometrically using: current UAV GPS position (from GPS-Denied), known AI camera angle, zoom, and current flight altitude. Flat terrain is assumed
- Accuracy is consistent with the frame-center position accuracy of the GPS-Denied system
- **AC-6.1** — Position estimates and confidence scores shall be streamed to **QGroundControl** via the MAVLink telemetry link. High-rate (per-frame) content stays on the local link for forensics; the GCS link is downsampled to **1–2 Hz** for situational awareness.
- **AC-6.2** — The ground station can send commands to the onboard system (e.g., operator-assisted re-localization hint with approximate coordinates) via STATUSTEXT, NAMED_VALUE_FLOAT, or a custom MAVLink dialect.
- **AC-6.3** — Output coordinates are in **WGS84** format (matches GPS_INPUT spec).
# Satellite Reference Imagery
## Object Localization (AI Camera)
- Satellite reference imagery resolution must be at least 0.5 m/pixel, ideally 0.3 m/pixel
- Satellite imagery for the operational area should be less than 2 years old where possible
- Satellite imagery must be pre-processed and loaded onto the companion computer before flight. Offline preprocessing time is not time-critical (can take minutes/hours)
- **AC-7.1** — Other onboard AI systems may request GPS coordinates of objects detected by the AI camera. Localization accuracy is **consistent with the frame-center accuracy of the GPS-Denied system in level flight (bank/pitch <5°)**. In maneuvering flight, ground-projection error is bounded by `altitude × |sin(unknown_bank_or_pitch)|` and the system shall publish that bound alongside the estimate.
- **AC-7.2** — The system computes object coordinates trigonometrically using: current UAV GPS position (from GPS-Denied), known AI-camera gimbal angle, zoom, and current flight altitude. Flat-terrain assumption applies.
## Satellite Reference Imagery
- **AC-8.1** — Satellite reference imagery is provided by the **Azaion Suite Satellite Service** (a separate component of the Suite). The runtime onboard system consumes this service through an offline tile cache interface; it does **not** call commercial providers (Maxar, Airbus, Planet, etc.) directly. The Satellite Service is responsible for upstream sourcing and is out of scope for this build. Required resolution at the cache interface: **at least 0.5 m/pixel, ideally 0.3 m/pixel**.
- **AC-8.2** — Satellite tiles consumed at runtime shall be:
- **<6 months old** for active-conflict sectors;
- **<12 months old** for stable rear sectors.
System shall reject or downgrade-confidence on tiles older than these thresholds (see AC-NEW-6).
- **AC-8.3** — Satellite imagery for the operational area shall be **pre-loaded and pre-processed** onto the companion computer before flight. Offline preprocessing time is not time-critical (minutes/hours). Pre-extracted tile descriptors (e.g., SuperPoint keypoints/descriptors and DINOv2-VLAD global descriptors) are part of the cache and count against the storage budget unless the solution draft explicitly defines a separate descriptor/index budget.
- **AC-8.4** — **Mid-flight tile generation & write-back**: during flight, the system shall continuously orthorectify navigation-camera frames into tiles aligned with the basemap projection and store them in the local cache, **deduplicated** so each ground sector is stored at most once (latest / highest-quality tile wins). On landing, the companion computer shall upload newly generated tiles back to the Azaion Suite Satellite Service so that the next mission cache contains imagery refreshed by the previous flight.
- **AC-8.5** — **Storage policy**: the system shall **not** retain raw navigation-camera frames or AI-camera frames as part of normal operation. Tiles are the only persistent imagery artifact. Forensic exception: a low-rate (≤0.1 Hz) thumbnail log of frames that **failed** tile generation may be retained for debugging within the FDR budget (AC-NEW-3).
- **AC-8.6** — **VPR retrieval unit + change-robustness**:
- The Visual Place Recognition (Component 2) FAISS index shall be built over **ground-footprint-sized "VPR chunks"** (~600–800 m at the deployment altitude band, with **40–50 % overlap** between adjacent chunks), **decoupled from the slippy-XYZ storage tile** (z=20). Any UAV frame footprint shall fall fully inside ≥1 chunk regardless of position.
- The index shall be **multi-scale**: in addition to fine-scale chunks (derived from z=20 storage), a coarser-scale chunk descriptor set (z=17 or z=18 effective scale) shall be maintained for change-robust retrieval in **active-conflict sectors** where building destruction or major scene change is expected.
- VPR top-K shall be **dynamically sized** by sector classification (AC-NEW-6) and EKF position covariance: K=5 in stable sectors with σ_xy ≤ 20 m; K=20 in active-conflict sectors; K=50 on expanding-window fallback.
- VPR shall be **invoked conditionally**, not on every frame: in steady state (last anchor age < 2 s, σ_xy < 20 m, VO healthy), the system uses a geometric prior from IMU+VO predicted position to rank candidate chunks by distance alone. VPR's DINOv2 forward is invoked on **re-loc triggers** (cold start AC-NEW-1, sharp turn AC-3.2, disconnected segment AC-3.3, σ_xy > 50 m, or VO failure for ≥2 frames).
## New AC (added in Phase 1 assessment, expanded with rationale & validation)
### AC-NEW-1 — Time-to-first-fix on cold start
**Statement.** From companion-computer boot, the system shall emit its first valid `GPS_INPUT` message in **<30 s**, given an IMU-extrapolated initial position handed over from the flight controller's EKF.
**Why it matters.** A mid-flight reboot (brown-out, watchdog reset, OS panic) is a realistic scenario on a fixed-wing UAV running an 8-hour mission. The autopilot continues to fly on IMU dead reckoning during the gap; a 30 s budget keeps that drift under ~500 m at 60 km/h cruise, which the EKF can absorb when our first fix arrives.
**Implementation drivers.** TensorRT engines must be built at install time (not at first run); CUDA / TRT init <5 s; tile-cache mmap warm at start; FAISS index loaded before MAVLink connect; first VPR retrieval + cross-view match must succeed at full resolution within the remaining budget.
**Validation.** Bench: cold-boot the companion 50× with simulated FC-pose input; record time from boot to first valid `GPS_INPUT` MAVLink frame. Pass = 95% percentile <30 s.
### AC-NEW-2 — Spoofing-promotion latency
**Statement.** When the flight controller signals GPS denial or spoofing (ArduPilot fix-loss / EKF lane-switch event; PX4 `EKF2_GPS_SPOOFED` flag if PX4 ever returns to scope), the GPS-Denied system shall promote its own estimate to the FC's primary GPS source within **<3 s**.
**Why it matters.** Without this gate, the FC may continue to follow a spoofed real-GPS source while our valid estimate sits idle. 3 s is short enough to keep the FC from acting on a malicious heading change but long enough to ride out a single-frame anomaly.
**Implementation drivers.** Subscribe to `GPS_RAW_INT`, `EKF_STATUS_REPORT`, `SYS_STATUS`, and any ArduPilot Plane EKF/GPS status messages available in the production firmware. Maintain an internal "real-GPS health" rolling average; switch to "primary" mode (raise our `GPS_INPUT` `fix_type` to 3D and assert) when the verified Plane-specific health trigger stays below threshold for >=1 s. Emit `STATUSTEXT` to QGC on every promotion / demotion.
**Validation.** ArduPilot Plane SITL: simulate spoofing (inject false `GPS_RAW_INT` from a malicious node); verify the exact trigger signals used by the production parameter set; measure time from spoof onset to our promotion. Pass = 95% percentile <3 s.
### AC-NEW-3 — Flight Data Recorder
**Statement.** The system shall retain to non-volatile storage, per flight: per-frame position estimates with covariance and source-label, IMU traces from the FC at full rate, all emitted `GPS_INPUT` frames, MAVLink raw stream (tlog), system health (CPU / GPU / temp / throttle), tiles generated mid-flight (AC-8.4), and a low-rate (≤0.1 Hz) thumbnail log of frames that failed tile generation. **Raw nav-cam frames and AI-cam frames are NOT retained** (AC-8.5). Storage cap **64 GB / flight**; recorder rolls over (oldest segment dropped first) after cap.
**Why it matters.** Tiles, telemetry traces, and IMU are the operationally useful artifacts: they reproduce the mission, feed the next mission's cache (AC-8.4), and let post-mission analysis explain any false-position event (AC-NEW-4). Raw frames are large and redundant once tiles exist.
**Implementation drivers.** Per-day directory layout; fixed-size segment files; rollover policy on segment-close, not on every write. NVMe ≥64 GB on top of the persistent satellite-tile cache.
**Validation.** Bench: run an 8-hour synthetic load (3 Hz nav frames replayed from disk), assert the FDR ends ≤64 GB and no payload class is silently dropped without a logged rollover event.
### AC-NEW-4 — False-position safety budget
**Statement.**
- P(reported estimate error > **500 m**) **<0.1 %** per flight.
- P(reported estimate error > **1 km**) **<0.01 %** per flight.
**Why it matters.** A single 1-km-off `GPS_INPUT` frame can hand the FC a heading that flies the UAV outside the geofence in seconds. The covariance carried in `GPS_INPUT` (`h_acc`) is the FC's only defense; this AC bounds the **probability** of our covariance under-reporting reality.
**Implementation drivers.** EKF covariance must be calibrated, not optimistic. Cross-view fixes with low inlier ratio must be **rejected**, not down-weighted to "small but non-zero". Outlier rejection at the EKF stage (Mahalanobis gate) is mandatory.
**Validation.** Monte Carlo over the AerialVL public dataset (S03) and our own recorded Mavic flights, with synthetic IMU injection where applicable; report error CDF; pass = both probabilities below budget across ≥100 simulated flights worth of frames.
### AC-NEW-5 — Operational environmental envelope
**Statement.** Operating temperature **−20 °C to +50 °C**; vibration / shock per RTCA DO-160G low-altitude UAV-class envelope. The cooling solution shall sustain the **25 W** power mode at the upper temperature bound for the full **8-hour duty cycle** without thermal throttling.
**Why it matters.** Without this, all latency / accuracy ACs are conditional on a benign thermal day. Eastern/southern Ukraine summers easily exceed +35 °C ambient inside a UAV bay; without active cooling, the Jetson throttles to 15 W mode and our 400 ms latency budget collapses.
**Implementation drivers.** Forced-air or active heatsink sized for 25 W continuous at +50 °C ambient bay temperature; thermal sensors logged in FDR (AC-NEW-3); throttle event = automatic `STATUSTEXT` warning to QGC.
**Validation.** Hot-soak chamber test: 25 W workload at +50 °C ambient for 8 h; assert no throttle. Cold-soak: −20 °C cold-start to first fix within AC-NEW-1 budget.
### AC-NEW-6 — Imagery freshness enforcement
**Statement.** The system shall reject (or downgrade confidence on) any satellite tile whose capture date violates AC-8.2 (>6 months old in active-conflict sectors; >12 months old in stable rear sectors). Tiles generated mid-flight (AC-8.4) and not yet uploaded to the Suite Satellite Service are timestamped with the current flight date and treated as fresh.
**Why it matters.** Stale satellite tiles are the dominant cross-view-matching failure mode in active-conflict sectors (cratering, dam destruction, road realignment). A confident match against a stale tile is worse than no match.
**Implementation drivers.** Each tile carries `capture_date` metadata in the cache index. Sector classification (active vs stable) is part of the operational area definition handed in pre-flight. Confidence weight = 1.0 if within freshness budget, linearly decayed to 0.0 over a 30-day grace zone past the budget, hard reject beyond the grace.
**Validation.** Inject tiles with synthetic age into the cache; verify rejection / decay curve matches spec; verify a stale-tile match never produces a `satellite_anchored` source label.
### AC-NEW-7 — Cache-poisoning safety budget
**Statement.** Per flight, across all onboard tiles written by Component 1b (in-flight ortho-tile generator):
- P(onboard tile geo-misaligned > **30 m**) **<1 %**.
- P(onboard tile geo-misaligned > **100 m**) **<0.1 %**.
**Why it matters.** Onboard tiles feed back into the Suite Satellite Service's basemap (AC-8.4). Without this AC, a confidently-bad EKF pose can write a misaligned tile that, after Service ingest, becomes the next flight's satellite anchor — producing cross-flight error compounding that AC-NEW-4 (single-flight false-position budget) does not capture. This AC bounds the **probability** that an onboard tile's claimed geo-alignment is wrong by a margin that would propagate to a downstream flight.
**Implementation drivers.**
- Service-source tiles are immutable within freshness budget (AC-8.2); onboard tiles overwrite only stale or other-onboard tiles.
- The onboard GPS-Denied system writes tile-quality metadata required by the Suite Satellite Service. The Service-side ingest applies a **2-flight voting layer**: an onboard tile gets promoted to "trusted basemap" only after **N>=2 independent flights** confirm consistent geo-alignment within X m of each other. (Active sectors per AC-NEW-6 may use single-flight promotion when σ_xy <= 3 m AND OSM-road-overlap >= 70 %.) The voting layer is an external Suite Satellite Service dependency, not implemented inside this onboard build, but its contract is required for AC-NEW-7 to pass end-to-end.
- The Component-1b parent-pose covariance is a **hard gate** in the local quality score: σ_xy ≤ 5 m for a hard write (`trust_level = candidate`); σ_xy ≤ 3 m for `trust_level = candidate` with full quality; tiles written in the σ_xy ∈ (3, 5] m band are marked `trust_level = soft` in the sidecar.
- Eligibility check (Component 1b) tightens generation gate from σ_xy ≤ 10 m to σ_xy ≤ 5 m.
**Validation.** Multi-flight Monte Carlo replay over AerialVL + Mavic + AerialExtreMatch with **synthetic over-confidence injection** (artificially deflate EKF covariance by 1.5×–3×): assert both probabilities below budget across ≥100 simulated flights worth of frames. Independently, Service-side voting layer is exercised in F-T3 to verify candidate tiles are not promoted to trusted basemap before N-flight confirmation.
### AC-NEW-8 — Visual blackout + GPS spoofing degraded-mode budget
**Statement.** When the navigation camera is fully unusable for visual localization and the flight controller simultaneously reports GPS denial/spoofing, the onboard system shall:
- continue emitting `GPS_INPUT` from IMU-only propagation for **up to 30 s** after the last trusted visual/satellite anchor, unless the estimator covariance exceeds the fail threshold earlier;
- label every estimate `{dead_reckoned}` and set `fix_type=2` or lower when the 95% covariance semi-major axis exceeds **100 m**;
- emit `fix_type=0`, `horiz_accuracy=999.0`, and `STATUSTEXT: VISUAL_BLACKOUT_FAILSAFE` when the 95% covariance semi-major axis exceeds **500 m** OR visual blackout exceeds **30 s** without a trusted re-anchor;
- never promote spoofed real-GPS measurements back into the estimator during blackout unless the FC GPS health has been stable and non-spoofed for **≥10 s** and a visual/satellite consistency check has succeeded.
**Why it matters.** A cloud/whiteout period removes all visual correction exactly when spoofed GPS cannot be trusted. The only safe behavior is honest IMU-only dead reckoning with rapidly growing uncertainty, not pretending that a stale visual position or spoofed GPS remains valid.
**Implementation drivers.** Add an image-quality/occlusion classifier before VO/VPR, a blackout state in the ESKF mode machine, covariance floors for IMU-only propagation, strict GPS health gating, and QGC/FDR logging for blackout start, every degraded estimate, and blackout recovery/failsafe.
**Validation.** SITL/replay: inject a 5 s, 15 s, and 35 s full-camera blackout while spoofing `GPS_RAW_INT`; assert mode transition ≤400 ms, spoofed GPS is ignored, covariance grows monotonically, `GPS_INPUT` fields degrade at the thresholds above, and recovery only occurs after a trusted visual/satellite anchor or the 10 s GPS-health + visual-consistency gate.
_docs/00_problem/input_data/data_parameters.md
+2 -8
View File
@@ -1,8 +1,2 @@
- Height
- 400m
- Camera:
- Name: ADTi Surveyor Lite 26S v2
- Resolution: 26MP
- Image resolution: 6252*4168
- Focal length: 25mm
- Sensor width: 23.5
- Height: 400m
- Camera: ADTi Surveyor Lite 20MP 20L V1
_docs/00_problem/input_data/expected_results/results_report.md
+85 -154
View File
@@ -1,166 +1,97 @@
# Expected Results
# Expected Results Mapping
Maps every input data item to its quantifiable expected result.
Tests use this mapping to compare actual system output against known-correct answers.
## Scope
## Result Format Legend
`coordinates.csv` is the current source of truth for the provided still-image nadir set. It gives expected WGS84 frame-center coordinates for `AD000001.jpg` through `AD000060.jpg`.
| Result Type | When to Use | Example |
|-------------|-------------|---------|
| Exact value | Output must match precisely | `fix_type: 3`, `satellites_visible: 10` |
| Tolerance range | Numeric output with acceptable variance | `lat: 48.275292 ± 50m` |
| Threshold | Output must exceed or stay below a limit | `latency < 400ms`, `memory < 8GB` |
| Pattern match | Output must match a string/regex pattern | `RELOC_REQ: last_lat=.* last_lon=.* uncertainty=.*m` |
| File reference | Complex output compared against a reference file | `match expected_results/position_accuracy.csv` |
| Set/count | Output must contain specific items or counts | `registered_frames / total_frames > 0.95` |
This data is sufficient for black-box frame-center geolocation tests against still images. The Derkachi representative fixture in `input_data/flight_derkachi/` adds cropped nadir video plus synchronized `SCALED_IMU2` and `GLOBAL_POSITION_INT` telemetry. It is sufficient for fixture validation, video/telemetry synchronization, replay, latency, VIO smoke tests, and trajectory comparison against the tlog GPS path. It is not sufficient by itself for final production accuracy because raw camera calibration, lens distortion, and exact camera-to-body calibration are still pending.
## Comparison Methods
## Pass / Fail Rules
| Method | Description | Tolerance Syntax |
|--------|-------------|-----------------|
| `numeric_tolerance` | abs(actual - expected) ≤ tolerance | `± <value>` |
| `threshold_min` | actual ≥ threshold | `≥ <value>` |
| `threshold_max` | actual ≤ threshold | `≤ <value>` |
| `percentage` | percentage of items meeting criterion | `≥ N%` |
| `exact` | actual == expected | N/A |
| `regex` | actual matches regex pattern | regex string |
| `file_reference` | compare against reference file | file path |
- **Normal frame-center geolocation**: estimated frame center is within 50 m of the expected WGS84 coordinate.
- **Stretch accuracy bin**: estimated frame center is within 20 m of the expected WGS84 coordinate.
- **Dataset aggregate**: at least 80% of mapped images pass the 50 m threshold and at least 50% pass the 20 m threshold.
- **Output shape**: each result must include image name, estimated `lat`, estimated `lon`, error in meters, source label, 95% covariance semi-major axis, and `last_satellite_anchor_age_ms`.
## Input → Expected Result Mapping
## Input To Expected Output Map
### Position Accuracy (60-image flight sequence)
### Still-Image Frame Centers
Ground truth GPS coordinates for each frame are in `coordinates.csv`. The system processes these frames sequentially (simulating a real flight) with corresponding IMU data (200Hz, from SITL ArduPilot or synthetic generation from trajectory) and satellite tile matches. The system outputs estimated GPS coordinates per frame. Expected results compare estimated positions against ground truth.
| Input image | Expected latitude | Expected longitude | Primary threshold | Stretch threshold |
|-------------|-------------------|--------------------|-------------------|-------------------|
| AD000001.jpg | 48.275292 | 37.385220 | <= 50 m | <= 20 m |
| AD000002.jpg | 48.275001 | 37.382922 | <= 50 m | <= 20 m |
| AD000003.jpg | 48.274520 | 37.381657 | <= 50 m | <= 20 m |
| AD000004.jpg | 48.274956 | 37.379004 | <= 50 m | <= 20 m |
| AD000005.jpg | 48.273997 | 37.379828 | <= 50 m | <= 20 m |
| AD000006.jpg | 48.272538 | 37.380294 | <= 50 m | <= 20 m |
| AD000007.jpg | 48.272408 | 37.379153 | <= 50 m | <= 20 m |
| AD000008.jpg | 48.271992 | 37.377572 | <= 50 m | <= 20 m |
| AD000009.jpg | 48.271376 | 37.376671 | <= 50 m | <= 20 m |
| AD000010.jpg | 48.271233 | 37.374806 | <= 50 m | <= 20 m |
| AD000011.jpg | 48.270334 | 37.374442 | <= 50 m | <= 20 m |
| AD000012.jpg | 48.269922 | 37.373284 | <= 50 m | <= 20 m |
| AD000013.jpg | 48.269366 | 37.372134 | <= 50 m | <= 20 m |
| AD000014.jpg | 48.268759 | 37.370940 | <= 50 m | <= 20 m |
| AD000015.jpg | 48.268291 | 37.369815 | <= 50 m | <= 20 m |
| AD000016.jpg | 48.267719 | 37.368469 | <= 50 m | <= 20 m |
| AD000017.jpg | 48.267461 | 37.367255 | <= 50 m | <= 20 m |
| AD000018.jpg | 48.266663 | 37.365888 | <= 50 m | <= 20 m |
| AD000019.jpg | 48.266135 | 37.365460 | <= 50 m | <= 20 m |
| AD000020.jpg | 48.265574 | 37.364211 | <= 50 m | <= 20 m |
| AD000021.jpg | 48.264892 | 37.362998 | <= 50 m | <= 20 m |
| AD000022.jpg | 48.264393 | 37.361086 | <= 50 m | <= 20 m |
| AD000023.jpg | 48.263803 | 37.361028 | <= 50 m | <= 20 m |
| AD000024.jpg | 48.263014 | 37.359878 | <= 50 m | <= 20 m |
| AD000025.jpg | 48.262635 | 37.358277 | <= 50 m | <= 20 m |
| AD000026.jpg | 48.261819 | 37.357116 | <= 50 m | <= 20 m |
| AD000027.jpg | 48.261182 | 37.355907 | <= 50 m | <= 20 m |
| AD000028.jpg | 48.260727 | 37.354723 | <= 50 m | <= 20 m |
| AD000029.jpg | 48.260117 | 37.353469 | <= 50 m | <= 20 m |
| AD000030.jpg | 48.259677 | 37.352165 | <= 50 m | <= 20 m |
| AD000031.jpg | 48.258881 | 37.351376 | <= 50 m | <= 20 m |
| AD000032.jpg | 48.258425 | 37.349964 | <= 50 m | <= 20 m |
| AD000033.jpg | 48.258653 | 37.347004 | <= 50 m | <= 20 m |
| AD000034.jpg | 48.257879 | 37.347711 | <= 50 m | <= 20 m |
| AD000035.jpg | 48.256777 | 37.348444 | <= 50 m | <= 20 m |
| AD000036.jpg | 48.255756 | 37.348098 | <= 50 m | <= 20 m |
| AD000037.jpg | 48.255375 | 37.346549 | <= 50 m | <= 20 m |
| AD000038.jpg | 48.254799 | 37.345603 | <= 50 m | <= 20 m |
| AD000039.jpg | 48.254557 | 37.344566 | <= 50 m | <= 20 m |
| AD000040.jpg | 48.254380 | 37.344375 | <= 50 m | <= 20 m |
| AD000041.jpg | 48.253722 | 37.343093 | <= 50 m | <= 20 m |
| AD000042.jpg | 48.254205 | 37.340532 | <= 50 m | <= 20 m |
| AD000043.jpg | 48.252380 | 37.342112 | <= 50 m | <= 20 m |
| AD000044.jpg | 48.251489 | 37.343079 | <= 50 m | <= 20 m |
| AD000045.jpg | 48.251085 | 37.346128 | <= 50 m | <= 20 m |
| AD000046.jpg | 48.250413 | 37.344034 | <= 50 m | <= 20 m |
| AD000047.jpg | 48.249414 | 37.343296 | <= 50 m | <= 20 m |
| AD000048.jpg | 48.249114 | 37.346895 | <= 50 m | <= 20 m |
| AD000049.jpg | 48.250241 | 37.347741 | <= 50 m | <= 20 m |
| AD000050.jpg | 48.250974 | 37.348379 | <= 50 m | <= 20 m |
| AD000051.jpg | 48.251528 | 37.349468 | <= 50 m | <= 20 m |
| AD000052.jpg | 48.251873 | 37.350485 | <= 50 m | <= 20 m |
| AD000053.jpg | 48.252161 | 37.351491 | <= 50 m | <= 20 m |
| AD000054.jpg | 48.252685 | 37.352343 | <= 50 m | <= 20 m |
| AD000055.jpg | 48.253268 | 37.353119 | <= 50 m | <= 20 m |
| AD000056.jpg | 48.253767 | 37.354246 | <= 50 m | <= 20 m |
| AD000057.jpg | 48.254329 | 37.354946 | <= 50 m | <= 20 m |
| AD000058.jpg | 48.254874 | 37.355765 | <= 50 m | <= 20 m |
| AD000059.jpg | 48.255481 | 37.356501 | <= 50 m | <= 20 m |
| AD000060.jpg | 48.256246 | 37.357485 | <= 50 m | <= 20 m |
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 1 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | ≥ 80% of frames have position error < 50m from ground truth | percentage | ≥ 80% of frames within 50m | `expected_results/position_accuracy.csv` |
| 2 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | ≥ 60% of frames have position error < 20m from ground truth | percentage | ≥ 60% of frames within 20m | `expected_results/position_accuracy.csv` |
| 3 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | Per-frame position output in WGS84 (lat, lon) | numeric_tolerance | each frame ± 100m max (no single frame exceeds 100m error) | `expected_results/position_accuracy.csv` |
| 4 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | Cumulative VO drift between satellite anchors < 100m | threshold_max | ≤ 100m drift between anchors | N/A |
### Representative Derkachi Video/IMU Fixture
### GPS_INPUT Message Correctness
| Input fixture | Expected validation result | Threshold |
|---------------|----------------------------|-----------|
| `flight_derkachi/data_imu.csv` | Telemetry CSV has required `timestamp(ms)`, `Time`, `SCALED_IMU2.*`, and `GLOBAL_POSITION_INT.*` columns; non-empty rows are monotonic from `Time=0.0` to `489.9` | 0 missing required columns; 0 decreasing timestamps; 4,900 nonblank rows |
| `flight_derkachi/flight_derkachi.mp4` | Video stream is readable as cropped nadir footage for replay | H.264, 880 x 720, 30 fps, approximately 490.07 s |
| Video/telemetry alignment | Video has 14,700 frames and telemetry has 4,900 rows | Exactly 3 video frames per telemetry row; duration delta <=250 ms |
| Derkachi trajectory comparison | Replay output can be compared to `GLOBAL_POSITION_INT.lat`, `GLOBAL_POSITION_INT.lon`, `GLOBAL_POSITION_INT.alt`, `GLOBAL_POSITION_INT.relative_alt`, velocity, and heading | Thresholds are calibration-gated; use for smoke/relative trajectory validation until intrinsics and camera-to-body calibration are pinned |
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 5 | Single frame + IMU data | Normal tracking frame with recent satellite match | `fix_type: 3`, `horiz_accuracy: 5-20m`, `satellites_visible: 10`, lat/lon populated | exact (fix_type, sat), numeric_tolerance (accuracy) | fix_type == 3, horiz_accuracy ∈ [1, 50] | N/A |
| 6 | Frame sequence, no satellite match for >30s | VO-only tracking, no recent satellite anchor | `fix_type: 3`, `horiz_accuracy: 20-50m` | exact (fix_type), range (accuracy) | fix_type == 3, horiz_accuracy ∈ [20, 100] | N/A |
| 7 | Frame sequence, VO lost + no satellite | IMU-only dead reckoning | `fix_type: 2`, `horiz_accuracy: 50-200m+` (growing over time) | exact (fix_type), threshold_min (accuracy) | fix_type == 2, horiz_accuracy ≥ 50 | N/A |
| 8 | VO lost + 3 consecutive satellite failures | Total position failure | `fix_type: 0`, `horiz_accuracy: 999.0` | exact | fix_type == 0, horiz_accuracy == 999.0 | N/A |
| 9 | Any valid frame | GPS_INPUT output rate | GPS_INPUT messages at 5-10Hz continuous | range | 5 ≤ rate_hz ≤ 10 | N/A |
## Known Gaps
### Confidence Tier Transitions
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 10 | Frame with satellite match <30s ago, covariance <400m² | HIGH confidence conditions | Confidence tier: HIGH, SSE confidence: "HIGH" | exact | N/A | N/A |
| 11 | Frame with cuVSLAM OK, no satellite match >30s | MEDIUM confidence conditions | Confidence tier: MEDIUM, SSE confidence: "MEDIUM" | exact | N/A | N/A |
| 12 | Frame with cuVSLAM lost, IMU-only | LOW confidence conditions | Confidence tier: LOW, SSE confidence: "LOW" | exact | N/A | N/A |
| 13 | 3+ consecutive total failures | FAILED conditions | Confidence tier: FAILED, SSE confidence: "FAILED", fix_type: 0 | exact | N/A | N/A |
### Image Registration & Visual Odometry
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 14 | 60 sequential flight images | Normal flight (no sharp turns) | Image registration rate ≥ 95% (≥ 57 of 60 registered) | percentage | ≥ 95% | N/A |
| 15 | 60 sequential flight images | Normal flight images | Mean reprojection error < 1.0 pixels | threshold_max | MRE < 1.0 px | N/A |
### Disconnected Route Segments & Sharp Turns
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 16 | Frames 32-43 from coordinates.csv | Trajectory with direction change (turn area) | System continues producing position estimates through the turn | threshold_min | ≥ 1 position output per frame | N/A |
| 17 | Simulated consecutive frames with 350m gap | Outlier between 2 consecutive photos due to tilt | System handles outlier, position estimate not corrupted (error < 100m for next valid frame) | threshold_max | ≤ 100m error after recovery | N/A |
| 18 | Simulated sharp turn (no overlap, <5% overlap, <70° angle, <200m drift) | Sharp turn where VO fails | Satellite re-localization triggers, position recovered within 3 frames after turn | threshold_max | position error ≤ 50m after re-localization | N/A |
| 19 | Simulated VO loss + satellite match success | Tracking loss → re-localization | cuVSLAM restarts, ESKF position corrected, tracking_state returns to NORMAL | exact | tracking_state == NORMAL after recovery | N/A |
### 3-Consecutive-Failure Re-Localization
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 20 | Simulated VO loss + 3 satellite match failures | Cannot determine position by any means | Re-localization request sent: `RELOC_REQ: last_lat=.* last_lon=.* uncertainty=.*m` | regex | message matches pattern | N/A |
| 21 | Re-localization request active | System waiting for operator | GPS_INPUT fix_type=0, system continues IMU prediction, continues satellite matching attempts | exact (fix_type) | fix_type == 0 | N/A |
| 22 | Operator sends approximate coordinates (lat, lon) | Operator re-localization hint | System uses hint as ESKF measurement (high covariance ~500m), attempts satellite match in new area | threshold_max | position error ≤ 500m initially, ≤ 50m after satellite match | N/A |
### Startup & Handoff
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 23 | System boot with GLOBAL_POSITION_INT available | Normal startup | System reads initial position, initializes ESKF, starts GPS_INPUT output | exact | GPS_INPUT output begins within 60s of boot | N/A |
| 24 | System boot + first satellite match | Startup validation | First satellite match validates initial position, position error drops | threshold_max | position error ≤ 50m after first satellite match | N/A |
### Mid-Flight Reboot Recovery
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 25 | System process killed mid-flight | Companion computer reboot | System recovers: reads FC position, inits ESKF with high uncertainty, loads TRT engines, starts cuVSLAM, performs satellite match | threshold_max | total recovery time ≤ 70s | N/A |
| 26 | Post-reboot first satellite match | Recovery validation | Position accuracy restored after first satellite match | threshold_max | position error ≤ 50m after first satellite match | N/A |
### Object Localization
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 27 | POST /objects/locate with pixel_x, pixel_y, gimbal angles, zoom, known UAV position | Object at known ground GPS | Response: `{ lat, lon, alt, accuracy_m, confidence }` with lat/lon matching ground truth | numeric_tolerance | lat/lon within accuracy_m of ground truth (consistent with frame-center accuracy) | N/A |
| 28 | POST /objects/locate with invalid pixel coordinates | Out-of-frame pixel | HTTP 422 or error response indicating invalid input | exact | HTTP status 422 | N/A |
### Coordinate Transform Chain
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 29 | Known GPS → NED → pixel → GPS round-trip | Coordinate transform validation | Round-trip error < 0.1m | threshold_max | ≤ 0.1m | N/A |
### API & Communication
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 30 | GET /health | Health check endpoint | HTTP 200, JSON with memory_mb, gpu_temp_c, status fields | exact (status code), regex (body) | status == 200, body contains `"status"` | N/A |
| 31 | POST /sessions | Start session | HTTP 200/201 with session ID | exact | status ∈ {200, 201} | N/A |
| 32 | GET /sessions/{id}/stream | SSE position stream | SSE events at ~1Hz with fields: type, timestamp, lat, lon, alt, accuracy_h, confidence, vo_status | regex | each event matches SSE schema | N/A |
| 33 | Unauthenticated request to /sessions | No JWT token | HTTP 401 Unauthorized | exact | status == 401 | N/A |
### Performance Thresholds
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 34 | Single camera frame (6252x4168) | End-to-end processing time | Total pipeline latency < 400ms (capture → GPS coordinate output) | threshold_max | ≤ 400ms | N/A |
| 35 | 30-minute sustained operation | Memory usage over time | Peak memory < 8GB, no memory leaks (growth < 50MB over 30min) | threshold_max | peak < 8192MB, growth ≤ 50MB | N/A |
| 36 | 30-minute sustained operation | GPU thermal | SoC junction temperature stays below 80°C (no throttling) | threshold_max | ≤ 80°C | N/A |
| 37 | cuVSLAM single frame | VO processing time | cuVSLAM inference ≤ 20ms per frame | threshold_max | ≤ 20ms | N/A |
| 38 | Satellite matching single frame | Satellite matching time (async) | LiteSAM/XFeat inference ≤ 330ms | threshold_max | ≤ 330ms | N/A |
| 39 | TRT engine load | Engine initialization time | All TRT engines loaded within 10s total | threshold_max | ≤ 10s | N/A |
### Satellite Tile Management
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 40 | Mission area definition (200km path, ±2km buffer, zoom 18) | Tile storage calculation | Total storage 500-800MB for zoom 18 + zoom 19 flight path | range | [300MB, 1000MB] | N/A |
| 41 | ESKF position ± 3σ search radius | Tile selection | Tiles covering search area loaded, mosaic assembled, covers at least 500m radius | threshold_min | coverage radius ≥ 500m | N/A |
### TRT Engine Validation
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 42 | LiteSAM PyTorch model → ONNX → TRT FP16 | TRT engine conversion | Engine builds successfully on Jetson Orin Nano Super | exact | exit_code == 0 | N/A |
| 43 | TRT engine output vs PyTorch reference (same input) | Inference correctness | Max L1 error between TRT and PyTorch output < 0.01 | threshold_max | L1_max < 0.01 | N/A |
| 44 | LiteSAM MinGRU operations | TRT compatibility check | All MinGRU ops supported in TRT 10.3 (polygraphy inspect) | exact | unsupported_ops == 0 | N/A |
### Telemetry
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 45 | Normal operation | Telemetry output rate | NAMED_VALUE_FLOAT messages at 1Hz (gps_conf, gps_drift, gps_hacc) | numeric_tolerance | rate: 1Hz ± 0.2Hz | N/A |
| 46 | VO tracking lost + 3 satellite failures | Re-localization telemetry | STATUSTEXT with RELOC_REQ sent to ground station | regex | message matches `RELOC_REQ:.*` | N/A |
## Expected Result Reference Files
### position_accuracy.csv
Reference file: `expected_results/position_accuracy.csv`
Contains the ground truth GPS coordinate for each frame in the 60-image test sequence (copied from `coordinates.csv`) plus the acceptance thresholds. Test harness computes haversine distance between estimated and ground truth positions, then applies aggregate criteria.
Thresholds applied to the full 60-frame sequence:
- ≥ 80% of frames: error < 50m
- ≥ 60% of frames: error < 20m
- 0% of frames: error > 100m (no single frame exceeds 100m)
- Cumulative VO drift between satellite anchors: < 100m
- The still-image set has expected WGS84 centers but no synchronized IMU, attitude, airspeed, altitude, or timestamp stream.
- The Derkachi fixture has synchronized video, IMU, and GPS trajectory, but no raw camera calibration, lens distortion, exact camera-to-body transform, attitude, or airspeed columns.
- The still-image sample cadence is slower than the target 3 fps runtime profile; the Derkachi video is 30 fps and must be sampled to target replay cadence for runtime tests.
- Final production acceptance requires camera calibration and representative datasets with synchronized camera/IMU plus ground-truth trajectory.
_docs/00_problem/input_data/flight_derkachi/README.md
+14
View File
@@ -0,0 +1,14 @@
# Derkachi Representative Flight Fixture
## Files
| File | Description | Observed Metadata |
|------|-------------|-------------------|
| `flight_derkachi.mp4` | Cropped nadir flight footage for replay | H.264, 880 x 720, 30 fps, about 490.07 s |
| `data_imu.csv` | Flight-controller telemetry trace exported from the tlog | 4,900 rows at 10 Hz from `Time=0.0` to `489.9`; includes `SCALED_IMU2` and `GLOBAL_POSITION_INT` trajectory fields |
## Test Use
Use this fixture for video/telemetry synchronization checks, representative replay smoke tests, VIO hot-path latency, frame-drop accounting, and trajectory comparison against `GLOBAL_POSITION_INT`. The video and telemetry align at exactly three video frames per telemetry row. Camera intrinsics, lens distortion, raw camera resolution, and exact camera-to-body calibration are still unknown, so this fixture is not sufficient by itself for final production camera calibration or satellite-anchor accuracy claims.
For the test recording, the rotating camera was mechanically fixed in a downward/nadir orientation. Treat the MP4 as a cleaned/cropped replay fixture rather than the raw camera feed.
_docs/00_problem/input_data/flight_derkachi/data_imu.csv
+4901
View File
File diff suppressed because it is too large Load Diff
_docs/00_problem/input_data/flight_derkachi/flight_derkachi.mp4
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9acb97042fc648301d73d3c0fe7d80f7e3e2697000c0d33afa8a7b7a74a20005
size 282207328
_docs/00_problem/problem.md
+2 -2
View File
@@ -1,2 +1,2 @@
We have a wing-type UAV with a camera pointing downwards that can take photos 3 times per second with a resolution 6200*4100. Also plane has flight controller with IMU. During the plane flight, we know GPS coordinates initially. During the flight, GPS could be disabled or spoofed. We need to determine the GPS of the centers of the next frame from the camera. And also the coordinates of the center of any object in these photos. We can use an external satellite provider for ground checks on the existing photos. So, before the flight, UAV's operator should upload the satellite photos to the plane's companion PC.
The real world examples are in input_data folder, but the distance between each photo is way bigger than it will be from a real plane. On that particular example photos were taken 1 photo per 2-3 seconds. But in real-world scenario frames would appear within the interval no more than 500ms or even 400 ms.
We have a wing-type UAV with a fixed downward navigation camera that can take photos 3 times per second. The authoritative navigation-camera spec is defined in `restrictions.md` as the ADTi 20MP 20L V1, APS-C sensor, about 5472 x 3648 px; older higher-resolution references are superseded. Also plane has flight controller with IMU. During the plane flight, we know GPS coordinates initially. During the flight, GPS could be disabled or spoofed. We need to determine the GPS of the centers of the next frame from the camera. And also the coordinates of the center of any object in these photos. We can use an external satellite provider for ground checks on the existing photos. So, before the flight, UAV's operator should upload the satellite photos to the plane's companion PC.
The real world examples are in input_data folder, but the original still-image set has a much larger distance between photos than the target aircraft will have. On that particular example photos were taken 1 photo per 2-3 seconds. But in real-world scenario frames would appear within the interval no more than 500ms. Additional representative data is available in `input_data/flight_derkachi/`: cropped nadir flight footage plus synchronized `SCALED_IMU2` and `GLOBAL_POSITION_INT` telemetry. This supports video/telemetry synchronization, replay, latency, VIO smoke tests, and trajectory comparison against the tlog GPS path. Camera intrinsics, lens distortion, raw camera feed parameters, and exact camera-to-body calibration are still pending, so final production accuracy claims remain gated on calibration data or a separately surveyed representative dataset.
_docs/00_problem/restrictions.md
+50 -28
View File
@@ -1,37 +1,59 @@
# UAV & Flight
# Restrictions
- Photos are taken by only airplane (fixed-wing) type UAVs
- Photos are taken by the camera pointing downwards and fixed, but it is not autostabilized
- The flying range is restricted by the eastern and southern parts of Ukraine (to the left of the Dnipro River)
- Altitude is predefined and no more than 1km. The height of the terrain can be neglected
- Flights are done mostly in sunny weather
- During the flight, UAVs can make sharp turns, so that the next photo may be absolutely different from the previous one (no same objects), but it is rather an exception than the rule
- Number of photos per flight could be up to 3000, usually in the 500-1500 range
> **Last revised**: 2026-05-01 (post Phase 1 AC/restrictions assessment clarifications).
# Cameras
## UAV & Flight
- UAV has two cameras:
1. **Navigation camera** — fixed, pointing downwards, not autostabilized. Used by GPS-Denied system for position estimation
2. **AI camera** — main camera with configurable angle and zoom, used by onboard AI detection systems
- Navigation camera resolution: FullHD to 6252*4168. Camera parameters are known: focal length, sensor width, resolution, etc.
- Cameras are connected to the companion computer (interface TBD: USB, CSI, or GigE)
- Terrain is assumed flat (eastern/southern Ukraine operational area); height differences are negligible
- Photos are taken by airplane (fixed-wing) type UAVs only.
- Photos are taken by the navigation camera pointing downwards and fixed (not gimbal-stabilized).
- Operational area is the eastern and southern parts of Ukraine (east/left of the Dnipro River).
- Mission profile: 8-hour flights at ~60 km/h cruise. Two route shapes coexist:
- **Sector**: up to **10 × 15 km = 150 km²** of dense coverage.
- **Transit corridor**: ~**50 km × 1 km = 50 km²** strip in/out of the sector.
- **Total operational area: up to ~400 km²** of pre-cached satellite imagery per mission. Cache is **persistent across flights** (not redownloaded each mission). Storage budget **~10 GB** for the satellite tile cache; see AC-NEW-3 for flight-data-recorder budget.
- Altitude: pre-defined, **≤1 km AGL**. Terrain is assumed flat (operational area is rolling steppe / agricultural land); height differences are negligible.
- Weather: predominantly sunny daytime operations. Validation must still cover the seasonal/visibility classes that affect visual matching in the operational area: summer crop/field patterns, autumn/winter bare fields, cloud/smoke/haze, snow if missions can occur in winter, and low-texture agricultural repetition.
- Sharp turns occur but are the exception, not the rule. Two consecutive photos may share <5% overlap during a turn (see AC-3.2).
- **No photo-count cap.** The previously stated "up to 3000 photos per flight" was a legacy operator number from a Mavic-class workflow; it is dropped because (a) it is inconsistent with 8 h × 3 fps, and (b) the system does **not store raw photos at all** (see AC-8.5). Storage is bounded by the tile-cache + FDR caps (~10 GB persistent + 64 GB / flight, AC-NEW-3).
# Satellite Imagery
## Cameras
- We can use satellite providers, but we're limited right now to Google Maps, which could be outdated for some regions
- Satellite imagery for the operational area must be pre-loaded onto the companion computer before flight
- The UAV carries **two cameras**:
1. **Navigation camera** — fixed, downward-pointing, not autostabilized. Consumed by the GPS-Denied system for position estimation.
2. **AI camera** — main mission camera with operator-controllable gimbal angle and zoom. Consumed by onboard AI detection systems.
- **Navigation camera**: **ADTi 20MP 20L V1, APS-C sensor, ~5472 × 3648 px (≈20 MP)**. APS-C sensor (~23.6 × 15.7 mm). Lens TBD — selected during solution-draft phase to land GSD in the **10–20 cm/px band at 1 km AGL** (drives a frame ground footprint of ~470 m × 314 m to ~980 m × 655 m depending on focal length). Other intrinsics (focal length, exact sensor dimensions, distortion coefficients) are pinned at module-selection time and used by Component-1b orthorectification (pre-flight checkerboard calibration, F-F2).
- **AI camera pose information available to the GPS-Denied system**: gimbal angle and zoom only. The UAV's instantaneous bank/pitch is **not** published from the autopilot to the AI-camera reasoning path. Object-localization accuracy is therefore scoped to level flight (AC-7.1).
- Cameras connect to the companion computer over USB, MIPI-CSI, or GigE (specific interface TBD at solution-draft phase, dependent on chosen module).
# Onboard Hardware
## Satellite Imagery
- Processing is done on a Jetson Orin Nano Super (67 TOPS, 8GB shared LPDDR5, 25W TDP)
- The companion computer runs JetPack (Ubuntu-based) with CUDA/TensorRT available
- Onboard storage for satellite imagery is limited (exact capacity TBD, but must be accounted for in tile preparation)
- Sustained GPU load may cause thermal throttling; the processing pipeline must stay within thermal envelope
- **Source: Azaion Suite Satellite Service** (a separate component of the wider Suite). The onboard system is a **consumer** of this service, not a direct customer of commercial providers. Upstream sourcing (Maxar / Airbus / partner agencies / commissioned tasking) is the Satellite Service's concern, not this build's.
- **Onboard interface to the Service is offline-only**: the companion computer holds a local cache populated **before flight** by syncing from the Service for the operational area (AC-8.3). No satellite imagery is fetched in-flight from the Service.
- **Mid-flight tile generation (AC-8.4)**: during the mission the companion computer generates fresh tiles from the navigation camera, orthorectified into the basemap projection, deduplicated against the existing cache, and stored locally. On landing, those new tiles are uploaded back to the Suite Satellite Service for ingestion, so the next mission's cache is refreshed by the previous flight.
- **No raw photo storage** (AC-8.5): the tile is the unit of persistence. Raw nav-camera and AI-camera frames are not retained (except a low-rate failure-thumbnail log for forensics).
- **Resolution at the cache interface**: 0.5 m/pixel minimum, 0.3 m/pixel ideal (AC-8.1). The architecture is provider-agnostic at the cache boundary; whatever the Suite Satellite Service supplies must meet that bar.
- **Storage tile resolution convention**: cache imagery is specified by source pixel size, not by assuming a universal zoom-to-meter mapping. The cache interface accepts **0.5 m/px minimum, 0.3 m/px ideal** imagery, and every tile manifest records CRS, tile matrix convention, tile dimension, latitude-adjusted meters-per-pixel, capture date, source, and compression. If an XYZ/WebMercator tile pyramid is used, its zoom level is documented as a provider convention rather than treated as proof of physical resolution. The matcher (Component 3) needs <=~4x scale ratio between the UAV frame (~12 cm/px GSD at 1 km AGL with the 20 MP APS-C camera) and the reference; 0.3-0.5 m/px reference imagery gives a ~2.5-4.2x ratio. Storage budget across the 400 km² operational area remains capped at **10 GB** for the persistent cache and must be validated against the final provider format/compression. The 10 GB budget includes cache imagery, manifests, overviews, and any precomputed global/local descriptors unless the solution draft explicitly splits a separate descriptor/index budget. **VPR retrieval unit is decoupled from the storage tile** (see AC-8.6 below): VPR chunks are derived from the tile cache at ground-footprint scale (~600-800 m chunks with 40-50 % overlap), independent of the storage tile convention.
- **Freshness gates** (AC-8.2 / AC-NEW-6) are enforced at runtime: tiles older than 6 months (active-conflict sectors) or 12 months (stable rear sectors) are rejected or down-confidence-weighted. Tiles generated mid-flight are timestamped with the current flight date and treated as fresh.
- **Free public imagery (Sentinel-2 etc.)** is not on the runtime path. If the Suite Satellite Service ever returns Sentinel-class tiles, the cache rejects them as below the 0.5 m/px floor.
# Sensors & Integration
## Onboard Hardware
- There is a lot of data from IMU (via the flight controller)
- The system communicates with the flight controller via MAVLink protocol using MAVSDK library
- The system must output GPS coordinates to the flight controller as a replacement for the real GPS module (MAVLink GPS_INPUT message)
- Ground station telemetry link is available but bandwidth-limited; it is not the primary output channel
- Processing platform: **Jetson Orin Nano Super** — 67 TOPS sparse INT8, **8 GB shared LPDDR5** (CPU and GPU share the same memory pool), **25 W TDP**.
- Companion computer runs JetPack (Ubuntu-based) with CUDA / TensorRT available.
- Sustained GPU load may cause thermal throttling; the processing pipeline must stay within the thermal envelope. The cooling solution shall sustain the 25 W power mode for the 8-hour duty cycle at the upper environmental-envelope temperature (AC-NEW-5).
- Onboard non-volatile storage: budget at least the satellite-cache (~10 GB) **plus** the flight-data-recorder cap (64 GB / flight, AC-NEW-3). Reuse-across-flights tile cache stays resident; per-flight FDR rolls over after cap.
## Sensors & Integration
- High-rate **IMU** data is available from the flight controller via MAVLink.
- The original still-image sample does **not** include synchronized IMU or ground-truth pose. The Derkachi representative fixture adds cropped nadir video plus synchronized `SCALED_IMU2` and `GLOBAL_POSITION_INT` telemetry, which is enough for replay, synchronization, latency, VIO smoke tests, and trajectory comparison against the tlog GPS path. Final production acceptance still requires camera intrinsics, lens distortion, exact camera-to-body calibration, and representative synchronized navigation-camera frames, FC IMU/attitude/airspeed/altitude, emitted MAVLink messages, and ground-truth trajectory from a representative flight or replay rig.
- The system communicates with the flight controller via MAVLink. Telemetry plumbing uses **MAVSDK**; the `GPS_INPUT` injection path is implemented via **pymavlink**, since MAVSDK does not expose a native `GPS_INPUT` API.
- **Autopilot target: ArduPilot only** (with `GPS1_TYPE=14` for MAVLink GPS injection). PX4 is out of scope for the build; if it ever returns to scope it will use `VISION_POSITION_ESTIMATE`, not `GPS_INPUT`. (See `_docs/00_research/00_ac_assessment.md` Q-1.)
- The system outputs WGS84 GPS coordinates to the flight controller as a replacement for the real GPS module (MAVLink GPS_INPUT, AC-4.3).
- **Ground station: QGroundControl** is the supported GCS. Mission Planner is not in scope. Telemetry link is bandwidth-limited and is not the primary output channel; per-frame data stays on the local FDR (AC-NEW-3), GCS sees a 1–2 Hz downsampled summary (AC-6.1).
## Failsafe & Safety
- If the GPS-Denied system fails to produce any position estimate for **>3 s**, the autopilot falls back to IMU-only dead reckoning (AC-5.2). N=3 s rides through one sharp turn at cruise speed without tripping the failsafe.
- The system must satisfy the false-position safety budget in AC-NEW-4 (P(error >500 m) <0.1%, P(error >1 km) <0.01% per flight).
- Cold-start time-to-first-fix budget is **<30 s** from companion-computer boot (AC-NEW-1); spoofing-promotion latency is **<3 s** from FC's GPS-loss signal (AC-NEW-2).
_docs/00_research/00_ac_assessment.md
+54
View File
@@ -0,0 +1,54 @@
# Acceptance Criteria Assessment
Accessed: 2026-05-01. Rerun after user-approved clarifications: 2026-05-01.
## Research Scope
- **Output class**: Technical-component selection support.
- **Novelty sensitivity**: High for VPR, embedded AI, and autopilot integration; source preference is current papers and official docs.
- **Boundary**: Fixed-wing UAV, nadir navigation camera, ArduPilot Plane, Jetson Orin Nano Super, offline Azaion Suite Satellite Service cache, eastern/southern Ukraine terrain.
## Acceptance Criteria
| Criterion | Current Values | Researched Values / Evidence | Cost / Timeline Impact | Status |
|-----------|----------------|------------------------------|------------------------|--------|
| AC-1.1 / AC-1.2 frame-center accuracy | <=50 m for >=80%, <=20 m for >=50% in normal segments | Plausible only with periodic satellite anchoring plus VO/IMU propagation. Aerial VPR papers show the mechanism is viable but sensitive to weather, scale, repetition, and tile overlap. | High validation cost. | Keep, high-risk |
| AC-1.3 drift | VO-only <100 m, IMU-fused <50 m between anchors, anchor age reported | Updated AC now requires `last_satellite_anchor_age_ms`, binned validation, and degraded covariance after a solution-defined max anchor age. | Medium. | Updated |
| AC-1.4 confidence | 95% covariance ellipse + source label | `GPS_INPUT` supports accuracy fields; source labels must be carried in telemetry/FDR because `GPS_INPUT` has no semantic label field. | Medium. | Keep |
| AC-2.1 registration | VO >95%; satellite anchoring measured separately | Split is correct: VO success is not the same as cross-domain satellite anchor success. | Medium-high. | Updated |
| AC-2.2 reprojection | <1 px VO, <2.5 px satellite anchor | Reasonable image-space gates, with coordinate error still dependent on calibration, orthorectification, and satellite georegistration. | Medium. | Keep |
| AC-3.x resilience | Outliers, sharp turns, disconnected segments, blackout | Technically feasible only through mode switching: VO failure triggers VPR/relocalization, blackout triggers IMU-only propagation with honest covariance growth. | High test cost. | Keep |
| AC-4.1 latency | <400 ms p95, <=10% frame drops, heavy VPR conditional | Aerial VPR survey reports some re-ranking paths too slow for steady-state use; solution must keep global VPR off the per-frame hot path. | High optimization cost. | Updated |
| AC-4.2 memory | <8 GB shared | Feasible if descriptors are compressed/pruned and indices are memory-mapped or loaded selectively. | Medium-high. | Keep |
| AC-4.3 MAVLink | v1 GPS_INPUT only via pymavlink | ArduPilot docs require `GPS1_TYPE=14`; MAVLink defines required lat/lon, velocity, fix, and accuracy fields. MAVSDK should remain telemetry-oriented. | Medium. | Keep |
| AC-5.2 failsafe | >3 s no estimate triggers fallback, Plane SITL verified | Copter docs are reference only. Plane-specific production parameters must be verified in SITL. | Medium. | Updated |
| AC-7 object localization | Level-flight AI-camera object GPS | Realistic under level-flight clause; maneuvering estimates must publish conservative bound. | Medium. | Keep |
| AC-8.x satellite cache | 0.3-0.5 m/px, freshness, offline descriptors, VPR chunks | Resolution is feasible through commercial/service imagery. Storage must count descriptors unless separately budgeted. | Medium-high. | Updated |
| AC-NEW-1 / 2 startup and spoofing | <30 s first fix, <3 s promotion | Feasible only with prebuilt engines, warmed indices, and verified Plane GPS-health triggers. | Medium-high. | Keep with SITL gate |
| AC-NEW-3 FDR | <=64 GB per flight, no raw frames | Feasible with segment files and rollover. | Medium. | Keep |
| AC-NEW-4 / 7 safety budgets | False-position and cache-poisoning probabilities | Appropriate safety gates, but require Monte Carlo and representative flight/replay data. | High. | Keep |
| AC-NEW-5 environment | -20 C to +50 C, 25 W for 8 h | NVIDIA confirms 25 W mode; thermal design must prevent throttling. | Medium-high. | Keep |
## Restrictions Assessment
| Restriction | Current Values | Researched Values / Evidence | Cost / Timeline Impact | Status |
|-------------|----------------|------------------------------|------------------------|--------|
| Camera source of truth | `restrictions.md` pins ADTi 20MP ~5472 x 3648 | User confirmed `restrictions.md` is authoritative. Lens/FOV remains a design parameter. | Medium during module selection. | Updated |
| Fixed nadir camera | No gimbal stabilization | Good for orthorectification; turn/tilt requires attitude compensation and failure detection. | Medium. | Keep |
| Terrain/weather | Flat steppe/agricultural, seasonal classes included | Repetitive fields and seasonal changes are VPR hazards; validation must include those classes. | High validation cost. | Updated |
| Satellite Service boundary | Offline consumer of Suite Satellite Service | Strong separation; cache manifest and ingest-voting contract are required. | Medium. | Keep |
| 10 GB cache | Includes imagery, manifests, overviews, descriptors unless split | Plausible at 0.5 m/px with compression; 0.3 m/px plus descriptors may exceed unless pruned. | Medium. | Updated |
| Jetson Orin Nano Super | 67 TOPS INT8, 8 GB, 25 W | Official specs support the restriction; thermal throttling remains a risk. | Medium-high. | Keep |
| Test data gap | Sample imagery lacks IMU/ground truth | Public datasets help prototype, but final acceptance needs synchronized representative data. | High. | Updated |
## Key Findings
1. Use a hybrid estimator: VO/IMU for frame propagation, satellite/VPR anchors for absolute correction, ESKF covariance as the safety gate.
2. Do not run heavy VPR/re-ranking every frame; invoke it on cold start, VO failure, covariance growth, sharp turns, and disconnected segments.
3. Avoid GPL libraries in production dependencies unless the project accepts GPL obligations. GPL VIO/SLAM tools should be benchmarks or references, not selected production components.
4. The cache must be designed as imagery + metadata + descriptor index, not just raster tiles.
5. ArduPilot Plane SITL and representative camera+IMU data are blocking validation dependencies, but not blockers for solution drafting.
## Sources
See `_docs/00_research/01_source_registry.md` for the detailed source list.
_docs/00_research/00_question_decomposition.md
+145
View File
@@ -0,0 +1,145 @@
# Question Decomposition
## Classification
- **Original question**: Design a GPS-denied onboard localization system for a fixed-wing UAV using a nadir camera, IMU, preloaded satellite imagery, and ArduPilot `GPS_INPUT`.
- **Active mode**: Mode A Phase 2, initial solution research.
- **Research output class**: Technical-component selection.
- **Question type**: Decision support with knowledge organization.
- **Timeliness sensitivity**: High for VPR, embedded AI inference, and MAVLink/ArduPilot integration; medium for geometry and filtering fundamentals.
## Research Boundary
| Dimension | Boundary |
|-----------|----------|
| Population | Fixed-wing UAV missions; not multirotor hover workflows. |
| Geography | Eastern/southern Ukraine operational areas east/left of the Dnipro River. |
| Timeframe | Current implementation target with 2024-2026 component evidence where possible. |
| Level | Onboard real-time production system, not offline post-processing. |
| Operating context | 8 h flight, 60 km/h, <=1 km AGL, 3 fps nav camera, Jetson Orin Nano Super, GPS denied/spoofed. |
| Required interfaces | Offline Satellite Service cache in; MAVLink `GPS_INPUT`, QGC telemetry, FDR records, and object-coordinate API out. |
| Non-functional envelope | <400 ms p95, <8 GB shared memory, 10 GB persistent cache target, 64 GB FDR cap, safety covariance and false-position budgets. |
## Project Constraint Matrix Summary
| Constraint Area | Binding Constraint |
|-----------------|-------------------|
| Camera | ADTi 20MP 20L V1, APS-C, ~5472 x 3648, fixed nadir, no gimbal stabilization. |
| Sensors | FC IMU/attitude/airspeed/altitude available over MAVLink; original still-image sample lacks synchronized IMU, while Derkachi replay data now provides synchronized IMU and `GLOBAL_POSITION_INT` trajectory. |
| Reference imagery | Offline cache only, 0.5 m/px minimum and 0.3 m/px ideal, freshness gates, no in-flight provider fetch. |
| Runtime | Jetson Orin Nano Super, CUDA/TensorRT available, 25 W thermal envelope. |
| Autopilot | ArduPilot only, v1 emits `GPS_INPUT` only; ODOMETRY intentionally disabled. |
| Storage | No raw frame retention; tiles + FDR only. Descriptor/index storage must be budgeted. |
| Safety | Reject weak anchors, never under-report covariance, fail/degrade honestly in blackout and spoofing. |
| Hard disqualifiers | Per-frame heavy VPR without profiling, runtime dependence on external network, stale-tile confident anchors, GPL production dependency unless licensing is accepted. |
## Perspectives
| Perspective | Focus |
|-------------|-------|
| Operator / mission user | Does the system keep the UAV navigable and report honest confidence under spoofing/blackout? |
| Embedded implementer | Can the pipeline fit <400 ms p95 and <8 GB on Jetson with maintainable interfaces? |
| Safety reviewer | Are false-position and cache-poisoning paths gated before they can steer the FC or poison future caches? |
| Field practitioner | Will seasonal agricultural repetition, turns, haze/smoke, and stale imagery break the architecture? |
| Contrarian | Which attractive libraries or SOTA models fail because of licensing, memory, latency, or input mismatch? |
## Sub-Questions And Query Variants
1. What architecture bounds drift while GPS is denied?
- fixed-wing UAV GPS-denied satellite image matching visual odometry
- visual odometry satellite imagery accumulated error fixed wing UAV
- monocular VIO aerial navigation scale ambiguity satellite anchor
- GPS spoofed UAV visual inertial navigation covariance failover
2. Which VO/VIO approach fits one nadir camera + IMU?
- OpenVINS monocular visual inertial odometry Jetson
- ORB-SLAM3 monocular inertial Jetson UAV limitations
- VINS-Fusion fixed wing monocular IMU outdoor aerial
- homography visual odometry nadir UAV IMU fusion
3. Which satellite retrieval and matching approach fits offline cache + <400 ms?
- aerial visual place recognition survey DINOv2 FAISS
- DINOv2 VLAD aerial VPR embedded memory
- LightGlue SuperPoint DISK ALIKED TensorRT Jetson
- cross-view UAV satellite matching failure modes farmland
4. How should the estimator and safety modes work?
- ESKF visual inertial GPS denied UAV covariance
- GPS_INPUT horiz_accuracy covariance external GPS ArduPilot
- visual blackout IMU dead reckoning UAV covariance growth
- false position rejection Mahalanobis gate visual localization
5. What cache format and data contract fit the onboard/Satellite Service boundary?
- COG PMTiles MBTiles offline raster cache embedded
- satellite tile descriptor index storage FAISS PMTiles
- cloud optimized geotiff local update limitations
- PMTiles read only update PostgreSQL/PostGIS-backed raster cache
6. How should MAVLink output integrate with ArduPilot Plane?
- ArduPilot GPS_INPUT GPS1_TYPE 14 Plane SITL
- pymavlink gps_input_send external GPS example
- MAVSDK GPS_INPUT support raw MAVLink
- ArduPilot EKF GPS glitch spoof failsafe Plane parameters
7. What validation datasets and tests are needed?
- AerialVL UAV satellite visual localization dataset
- VPAir aerial visual place recognition dataset
- EuRoC MAV visual inertial odometry dataset
- ArduPilot Plane SITL fake GPS spoofing simulation
## Component Option Search Plan
| Component Area | Option Families / Candidates | Evidence Needed |
|----------------|------------------------------|-----------------|
| Camera calibration and geometry | OpenCV calibration/homography; custom NumPy geometry; ROS camera pipeline | Official API for intrinsics, distortion, homography, RANSAC; permissive licensing; Jetson compatibility. |
| VO / VIO propagation | OpenVINS, ORB-SLAM3, VINS-Fusion, custom homography+IMU ESKF | Exact monocular+IMU input fit, output pose/covariance, licensing, runtime, initialization behavior. |
| VPR global retrieval | DINOv2-VLAD/AnyLoc, MixVPR/SALAD/SelaVPR, classical NetVLAD/BoW | Aerial benchmark evidence, descriptor size, offline index fit, embedded feasibility. |
| Local cross-domain matching | LightGlue + DISK/ALIKED, SuperPoint+LightGlue, LoFTR/XFeat, SIFT/ORB baseline | Inputs/outputs, match coordinates, license, runtime knobs, TensorRT/Jetson feasibility. |
| Vector index | FAISS CPU/GPU, PostgreSQL/pgvector metadata-assisted search, Annoy/HNSWLIB | Top-K retrieval, saved index, memory/compression knobs, ARM/Jetson feasibility. |
| Estimator | Custom ESKF, factor graph, robot_localization | Covariance output, mode labels, Mahalanobis gates, source-specific update control. |
| Cache/storage | COG, PostgreSQL/PostGIS manifest, PMTiles, MBTiles, raw tile folders | Offline read/update behavior, storage efficiency, metadata/manifest support. |
| MAVLink integration | pymavlink, MAVSDK, MAVProxy bridge | `GPS_INPUT` support, ArduPilot `GPS1_TYPE=14`, telemetry subscriptions, QGC status. |
| FDR | PostgreSQL event index, Parquet export, CBOR segment files | Streaming writes, rollover, compact typed records, replayability. |
## Completeness Audit
- **Cost/resources**: covered by Jetson, cache, thermal, and descriptor storage constraints.
- **Legal/licensing**: covered; GPL VIO/SLAM tools are not selected for production.
- **Dependencies**: Satellite Service cache contract, ArduPilot Plane SITL, and synchronized validation data are explicit dependencies.
- **Operating environment**: fixed-wing, altitude, terrain, seasonal/visibility classes, and blackout cases covered.
- **Failure modes**: VO failure, stale tiles, spoofing, blackout, thermal throttling, false anchors, cache poisoning covered.
- **Practitioner concerns**: real-time embedded performance and dataset mismatch covered through survey and benchmark sources.
- **Change over time**: DINOv2/VPR models and Jetson/TensorRT assumptions require version-pinned profiling during implementation.
## Mode B Round 2 Addendum — User-Requested Technology Check
### Research Output Class
Technical-component selection. The addendum verifies two implementation choices before autodev proceeds to planning:
1. Whether OpenVINS should replace the custom OpenCV-based VO/ESKF direction.
2. Whether DINOv2-VLAD + ALIKED/LightGlue is still the right satellite retrieval and anchor-verification stack.
### Boundary Clarification
"Custom OpenCV" is treated as OpenCV for calibration, undistortion, feature geometry, homography/RANSAC, and MRE measurement, plus a project-owned ESKF/mode machine. It is not treated as a naive OpenCV-only replacement for VIO.
### Additional Query Variants Executed
- OpenVINS GPL-3 license MSCKF visual inertial odometry documentation monocular IMU 2026
- OpenVINS visual inertial odometry GPS denied UAV MSCKF limitations monocular high altitude nadir camera
- why not use OpenVINS production GPL ROS dependency visual inertial odometry limitations
- OpenCV license BSD 3-Clause camera calibration findHomography RANSAC documentation 4.x
- custom visual odometry OpenCV homography IMU EKF fixed wing UAV satellite imagery GPS denied 2024
- DINOv2 VLAD AnyLoc visual place recognition aerial satellite retrieval benchmark 2024 2025
- DINOv2 VLAD limitations visual place recognition storage compute AnyLoc limitations
- DINOv2 TensorRT Jetson performance issue embedding accuracy visual place recognition
- ALIKED LightGlue license local feature matching aerial image registration 2024 2025
- ALIKED LightGlue ONNX TensorRT Jetson performance benchmark local feature matching
- aerial visual place recognition survey 2024 runtime memory re-ranking SuperGlue LightGlue satellite UAV retrieval
### Addendum Conclusion
OpenVINS is better than a pure custom OpenCV-only VIO implementation, but the production architecture should keep OpenCV as the utility layer and keep the project-owned ESKF/mode machine as the shipped estimator. OpenVINS becomes a mandatory benchmark/reference because it does not own the satellite anchor, spoofing/blackout, source-label, cache-write, and MAVLink semantics required by the acceptance criteria, and GPLv3 remains a production dependency blocker.
DINOv2-VLAD + CPU-first FAISS + ALIKED/LightGlue remains the preferred anchor stack, with two non-negotiable constraints: retrieval is trigger-based rather than per-frame, and TensorRT/ONNX optimizations are accepted only after descriptor-fidelity and Jetson latency tests.
_docs/00_research/01_source_registry.md
+419
View File
@@ -0,0 +1,419 @@
# Source Registry
## Source #1
- **Title**: Visual Odometry in GPS-Denied Zones for Fixed-Wing UAV with Reduced Accumulative Error Based on Satellite Imagery
- **Link**: https://www.mdpi.com/2076-3417/14/16/7420
- **Tier**: L1
- **Publication Date**: 2024
- **Timeliness Status**: Currently valid
- **Target Audience**: UAV visual localization researchers/implementers
- **Research Boundary Match**: Full match
- **Summary**: Demonstrates fixed-wing high-altitude monocular VO corrected by satellite imagery; highlights scale ambiguity and accumulated drift.
- **Related Sub-question**: Architecture / drift bounding
## Source #2
- **Title**: Visual place recognition for aerial imagery: A survey
- **Link**: https://arxiv.org/abs/2406.00885
- **Tier**: L1
- **Publication Date**: 2024
- **Timeliness Status**: Currently valid
- **Target Audience**: Aerial VPR researchers/implementers
- **Research Boundary Match**: Full match
- **Summary**: Reviews aerial VPR, retrieval/re-ranking, overlap/scale effects, memory/runtime issues, and georeference recall.
- **Related Sub-question**: VPR / validation
## Source #3
- **Title**: OpenVINS documentation
- **Link**: https://docs.openvins.com/
- **Tier**: L1
- **Publication Date**: 2023 latest noted release
- **Timeliness Status**: Needs verification before implementation
- **Target Audience**: VIO researchers/implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: OpenVINS is an EKF/MSCKF visual-inertial estimator supporting monocular tracking, calibration, evaluation, and covariance-aware estimation; GPL-3 license.
- **Related Sub-question**: VO/VIO
## Source #4
- **Title**: ORB-SLAM3 README
- **Link**: https://raw.githubusercontent.com/UZ-SLAMLab/ORB_SLAM3/master/README.md
- **Tier**: L1
- **Publication Date**: 2021 README, still repository source
- **Timeliness Status**: Needs verification before implementation
- **Target Audience**: SLAM implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: ORB-SLAM3 supports monocular visual-inertial SLAM and multi-map operation, requires calibration, and is GPLv3.
- **Related Sub-question**: VO/VIO alternatives
## Source #5
- **Title**: OpenCV 4.x documentation via Context7
- **Link**: https://docs.opencv.org/4.x/
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Computer vision implementers
- **Research Boundary Match**: Full match for utility layer
- **Summary**: Documents camera calibration, undistortion, and `findHomography` with RANSAC for robust geometry.
- **Related Sub-question**: Calibration / geometry
## Source #6
- **Title**: LightGlue README and Context7 docs
- **Link**: https://raw.githubusercontent.com/cvg/LightGlue/main/README.md
- **Tier**: L1
- **Publication Date**: Current repository, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Feature-matching implementers
- **Research Boundary Match**: Full match for local matching
- **Summary**: LightGlue accepts local keypoints/descriptors and returns matched coordinates/scores; supports SuperPoint, DISK, ALIKED, SIFT, adaptive pruning, CUDA, and Apache-2 for code/weights while SuperPoint has restrictive licensing.
- **Related Sub-question**: Local matching
## Source #7
- **Title**: AnyLoc README
- **Link**: https://github.com/AnyLoc/AnyLoc
- **Tier**: L1
- **Publication Date**: 2023 repository, accessed 2026-05-01
- **Timeliness Status**: Needs profiling verification
- **Target Audience**: VPR implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: Provides DINOv2 + VLAD API examples and notes substantial storage/compute requirements for full experiments.
- **Related Sub-question**: VPR descriptors
## Source #8
- **Title**: DINOv2 repository
- **Link**: https://github.com/facebookresearch/dinov2
- **Tier**: L1
- **Publication Date**: 2023 repository, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Vision model implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: Meta's DINOv2 implementation and models, Apache-2.0 / CC-BY-4.0 license notices.
- **Related Sub-question**: VPR descriptors
## Source #9
- **Title**: FAISS documentation and Context7 docs
- **Link**: https://faiss.ai/index.html
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Vector search implementers
- **Research Boundary Match**: Full match
- **Summary**: FAISS supports dense vector search, top-k retrieval, CPU/GPU indexes, product quantization, and save/load APIs; GPU indexes must be converted to CPU before saving.
- **Related Sub-question**: Descriptor retrieval
## Source #10
- **Title**: MAVSDK documentation via Context7
- **Link**: https://github.com/mavlink/mavsdk
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: MAVLink application implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: MAVSDK provides telemetry APIs including raw GPS, GPS info, status text, position/velocity, and odometry subscriptions; `GPS_INPUT` emission should use raw MAVLink/pymavlink for this project.
- **Related Sub-question**: MAVLink integration
## Source #11
- **Title**: ArduPilot MAVProxy GPSInput
- **Link**: https://ardupilot.org/mavproxy/docs/modules/GPSInput.html
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: ArduPilot integrators
- **Research Boundary Match**: Full match
- **Summary**: External GPS input requires `GPS1_TYPE=14` and accepts MAVLink `GPS_INPUT` fields including WGS84 lat/lon, velocity, fix type, and accuracy.
- **Related Sub-question**: MAVLink output
## Source #12
- **Title**: MAVLink common message spec: GPS_INPUT
- **Link**: https://mavlink.io/en/messages/common.html#GPS_INPUT
- **Tier**: L1
- **Publication Date**: Current spec, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: MAVLink implementers
- **Research Boundary Match**: Full match
- **Summary**: Defines `GPS_INPUT` fields, fix type semantics, `horiz_accuracy`, and ignore flags.
- **Related Sub-question**: MAVLink output / confidence
## Source #13
- **Title**: ArduPilot GPS failsafe and glitch protection
- **Link**: https://ardupilot.org/copter/docs/gps-failsafe-glitch-protection.html
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Reference only for Plane
- **Target Audience**: ArduPilot operators
- **Research Boundary Match**: Partial overlap
- **Summary**: Documents GPS glitch protection and notes inertial-only position degrades quickly; Copter-specific defaults must not be assumed for Plane.
- **Related Sub-question**: Failsafe / spoofing
## Source #14
- **Title**: ArduPilot EKF failsafe
- **Link**: https://ardupilot.org/copter/docs/ekf-inav-failsafe.html
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Reference only for Plane
- **Target Audience**: ArduPilot operators
- **Research Boundary Match**: Partial overlap
- **Summary**: Explains EKF variance failsafe behavior and why spoof/glitch tests must be parameterized.
- **Related Sub-question**: Failsafe / spoofing
## Source #15
- **Title**: Jetson Orin Nano Super Developer Kit
- **Link**: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/nano-super-developer-kit/
- **Tier**: L1
- **Publication Date**: Current page, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Embedded AI implementers
- **Research Boundary Match**: Full match
- **Summary**: Confirms 67 INT8 TOPS, 8 GB LPDDR5, 102 GB/s, and 7-25 W power range.
- **Related Sub-question**: Runtime
## Source #16
- **Title**: NVIDIA JetPack 6.2 Super Mode blog
- **Link**: https://developer.nvidia.com/blog/nvidia-jetpack-6-2-brings-super-mode-to-nvidia-jetson-orin-nano-and-jetson-orin-nx-modules/
- **Tier**: L2
- **Publication Date**: 2024
- **Timeliness Status**: Currently valid
- **Target Audience**: Jetson developers
- **Research Boundary Match**: Full match
- **Summary**: Explains 25 W and MAXN Super modes and warns thermal design must accommodate the new power modes or throttling occurs.
- **Related Sub-question**: Runtime / thermal
## Source #17
- **Title**: PMTiles Concepts
- **Link**: https://docs.protomaps.com/pmtiles/
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Geospatial storage implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: PMTiles is single-file tiled archive, efficient for reads, but read-only and not update-in-place.
- **Related Sub-question**: Cache storage
## Source #18
- **Title**: GDAL COG driver
- **Link**: https://gdal.org/en/stable/drivers/raster/cog.html
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Geospatial raster implementers
- **Research Boundary Match**: Full match
- **Summary**: Defines COG creation options for tiled, compressed, overview-enabled GeoTIFFs.
- **Related Sub-question**: Cache storage
## Source #19
- **Title**: AerialVL dataset
- **Link**: https://github.com/hmf21/AerialVL
- **Tier**: L1
- **Publication Date**: 2024
- **Timeliness Status**: Currently valid
- **Target Audience**: Aerial visual localization researchers
- **Research Boundary Match**: Partial overlap
- **Summary**: Public aerial localization benchmark with UAV sequences, reference maps, and geo-referenced evaluation data.
- **Related Sub-question**: Validation
## Source #20
- **Title**: EuRoC MAV Dataset
- **Link**: http://projects.asl.ethz.ch/datasets/euroc-mav/
- **Tier**: L1
- **Publication Date**: 2016
- **Timeliness Status**: Stable benchmark
- **Target Audience**: VIO researchers
- **Research Boundary Match**: Partial overlap
- **Summary**: Stereo camera + IMU + ground truth benchmark useful for VIO sanity tests but not representative of high-altitude nadir fixed-wing imagery.
- **Related Sub-question**: Validation
## Source #21
- **Title**: NVIDIA/TensorRT issue: DINOv2 TensorRT performance/precision on Jetson
- **Link**: https://github.com/NVIDIA/TensorRT/issues/4348
- **Tier**: L4
- **Publication Date**: 2024
- **Timeliness Status**: Needs verification
- **Target Audience**: Jetson/TensorRT implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: Reports limited mixed-precision gains for DINOv2-S on Jetson/RTX, suggesting DINOv2 optimization is not automatically beneficial.
- **Related Sub-question**: Mode B performance risk
## Source #22
- **Title**: NVIDIA Developer Forum: DINOv2 TensorRT model performance issue
- **Link**: https://forums.developer.nvidia.com/t/dinov2-tensorrt-model-performance-issue/312251
- **Tier**: L4
- **Publication Date**: 2024
- **Timeliness Status**: Needs verification
- **Target Audience**: Jetson/TensorRT implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: Reports DINOv2 embedding distance changes after TensorRT conversion on Jetson Orin Nano; requires embedding-fidelity validation before relying on TensorRT descriptors.
- **Related Sub-question**: Mode B performance/quality risk
## Source #23
- **Title**: LightGlue license issue discussions
- **Link**: https://github.com/cvg/LightGlue/issues/120
- **Tier**: L4
- **Publication Date**: 2024
- **Timeliness Status**: Currently relevant
- **Target Audience**: Feature-matching implementers
- **Research Boundary Match**: Full match for licensing
- **Summary**: Community discussion highlights restrictive SuperPoint licensing inside the LightGlue ecosystem and supports avoiding SuperPoint as default production extractor.
- **Related Sub-question**: Mode B licensing risk
## Source #24
- **Title**: ArduPilot issue: GPS_INPUT velocity ignore flag pitfall
- **Link**: https://github.com/ArduPilot/ardupilot/issues/19633
- **Tier**: L4
- **Publication Date**: 2021
- **Timeliness Status**: Needs SITL verification
- **Target Audience**: ArduPilot integrators
- **Research Boundary Match**: Full match for GPS_INPUT caution
- **Summary**: Reports EKF3 may use zero velocity when `GPS_INPUT_IGNORE_FLAG_VEL_HORIZ` is set, so velocity-source parameters must be tested rather than relying only on ignore flags.
- **Related Sub-question**: Mode B MAVLink pitfall
## Source #25
- **Title**: FAISS install documentation
- **Link**: https://github.com/facebookresearch/faiss/blob/main/INSTALL.md
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Vector search implementers
- **Research Boundary Match**: Full match
- **Summary**: FAISS CPU conda package supports aarch64, while GPU package availability is x86-64 focused; Jetson design should assume CPU FAISS unless a custom build is proven.
- **Related Sub-question**: Mode B FAISS deployment
## Source #26
- **Title**: GNSS-denied geolocalization of UAVs by visual matching of onboard camera images with orthophotos
- **Link**: https://ar5iv.labs.arxiv.org/html/2103.14381
- **Tier**: L1
- **Publication Date**: 2021
- **Timeliness Status**: Stable mechanism reference
- **Target Audience**: UAV visual geolocalization researchers
- **Research Boundary Match**: Partial overlap
- **Summary**: Demonstrates visual matching with orthophotos and Monte Carlo/local planarity ideas; supports using orthorectified reference maps but does not cover all adversarial visual attacks.
- **Related Sub-question**: Mode B alternative / security limits
## Source #27
- **Title**: OpenVINS LICENSE
- **Link**: https://github.com/rpng/open_vins/blob/master/LICENSE
- **Tier**: L1
- **Publication Date**: Current repository, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: VIO implementers / product owners
- **Research Boundary Match**: Full match for licensing
- **Summary**: OpenVINS is GPLv3-licensed; this is a production dependency constraint, not a technical capability limitation.
- **Related Sub-question**: Mode B round 2 — OpenVINS vs custom production estimator
## Source #28
- **Title**: OpenVINS documentation and Context7 lookup
- **Link**: https://docs.openvins.com/index.html
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: VIO implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: OpenVINS is a strong EKF/MSCKF VIO system for monocular camera + IMU reference runs, with calibration and covariance-aware state estimation, but it does not own the project-specific satellite anchor, GPS_INPUT, source-label, spoofing, blackout, and cache-poisoning state machine.
- **Related Sub-question**: Mode B round 2 — OpenVINS vs custom production estimator
## Source #29
- **Title**: OpenCV 4.x calibration/homography documentation and Context7 lookup
- **Link**: https://docs.opencv.org/4.x/
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: Computer vision implementers
- **Research Boundary Match**: Full match for geometry utility layer
- **Summary**: OpenCV 4.x provides calibration, undistortion, homography estimation, RANSAC/USAC robust estimation, and reprojection-error primitives under a permissive license; it is a utility layer rather than a complete GPS-denied estimator.
- **Related Sub-question**: Mode B round 2 — custom OpenCV boundary
## Source #30
- **Title**: AnyLoc: Towards Universal Visual Place Recognition
- **Link**: https://arxiv.org/html/2308.00688
- **Tier**: L1
- **Publication Date**: 2023; ICRA 2024
- **Timeliness Status**: Currently valid, profiling required before deployment
- **Target Audience**: VPR implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: AnyLoc combines DINOv2 features with VLAD aggregation for broad VPR, including aerial data, and supports the selected DINOv2-VLAD retrieval family while leaving runtime/storage tuning as a deployment gate.
- **Related Sub-question**: Mode B round 2 — satellite retrieval
## Source #31
- **Title**: ALIKED-LightGlue-ONNX and LightGlue ONNX/TensorRT deployment reports
- **Link**: https://github.com/ikeboo/ALIKED-LightGlue-ONNX
- **Tier**: L2
- **Publication Date**: Current repository, accessed 2026-05-01
- **Timeliness Status**: Promising but needs Jetson verification
- **Target Audience**: Local feature matching implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: ONNX/optimized variants show a credible deployment path for ALIKED + LightGlue, but public evidence is not enough to assume Jetson Orin Nano p95 latency without project profiling.
- **Related Sub-question**: Mode B round 2 — local matcher deployability
## Source #32
- **Title**: Visual place recognition for aerial imagery: A survey
- **Link**: https://arxiv.org/abs/2406.00885
- **Tier**: L1
- **Publication Date**: 2024
- **Timeliness Status**: Currently valid
- **Target Audience**: Aerial VPR researchers / implementers
- **Research Boundary Match**: Full match
- **Summary**: Aerial VPR performance depends materially on tile scale, overlap, weather, repetitive patterns, and re-ranking cost; this supports overlapped VPR chunks, dynamic top-K, and conditional local verification.
- **Related Sub-question**: Mode B round 2 — satellite retrieval and anchor verification
## Source #33
- **Title**: BASALT repository and documentation
- **Link**: https://github.com/VladyslavUsenko/basalt
- **Tier**: L1
- **Publication Date**: Current repository, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: VIO implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: BASALT provides visual-inertial odometry and mapping, camera/IMU calibration tools, EuRoC/TUM VI support, and a BSD-style production-friendly licensing path.
- **Related Sub-question**: Mode B round 3 — Kimera vs BASALT vs OpenVINS
## Source #34
- **Title**: HybVIO: Pushing the Limits of Real-time Visual-inertial Odometry
- **Link**: https://arxiv.org/pdf/2106.11857
- **Tier**: L1
- **Publication Date**: 2021
- **Timeliness Status**: Stable benchmark reference
- **Target Audience**: VIO researchers / embedded implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: Reports EuRoC RMS ATE comparisons including BASALT mean about 0.051 m online stereo and Kimera mean about 0.12 m, plus notes that optimization-based methods often lack direct uncertainty quantification compared with filters.
- **Related Sub-question**: Mode B round 3 — VIO error and confidence comparison
## Source #35
- **Title**: OpenVINS issue #402 — up-to-date ATE and RTE metrics
- **Link**: https://github.com/rpng/open_vins/issues/402
- **Tier**: L4
- **Publication Date**: 2024
- **Timeliness Status**: Community benchmark, verify in our replay harness
- **Target Audience**: VIO implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: Community EuRoC comparison reports BASALT average ATE about 0.072 m with 100% completion, and OpenVINS average ATE about 0.091 m with about 88.55% completion and a divergence on one hard sequence.
- **Related Sub-question**: Mode B round 3 — BASALT vs OpenVINS error/completion
## Source #36
- **Title**: Kimera-VIO mono-inertial parameter issues
- **Link**: https://github.com/MIT-SPARK/Kimera-VIO/issues/254
- **Tier**: L4
- **Publication Date**: 2024
- **Timeliness Status**: Relevant implementation caveat
- **Target Audience**: VIO implementers
- **Research Boundary Match**: Partial overlap
- **Summary**: Kimera-VIO stereo path remains strong, but mono-inertial configurations had documented poor default performance; parameter changes improved one EuRoC mono setup to less than about +/-0.2 m per axis.
- **Related Sub-question**: Mode B round 3 — Kimera mono/nadir risk
## Source #37
- **Title**: RaD-VIO and downward-facing VIO literature
- **Link**: https://arxiv.org/abs/1810.08704
- **Tier**: L1
- **Publication Date**: 2018
- **Timeliness Status**: Stable mechanism reference
- **Target Audience**: MAV downward-camera VIO researchers
- **Research Boundary Match**: Full match for nadir-camera caveat
- **Summary**: Downward-facing monocular VIO has planar-scene and observability challenges; range/altitude and IMU constraints are important when the camera sees mostly ground plane.
- **Related Sub-question**: Mode B round 3 — nadir support and limitations
## Source #38
- **Title**: OpenVINS covariance documentation and StateHelper APIs
- **Link**: https://docs.openvins.com/dev-index.html
- **Tier**: L1
- **Publication Date**: Current docs, accessed 2026-05-01
- **Timeliness Status**: Currently valid
- **Target Audience**: VIO implementers
- **Research Boundary Match**: Full match for covariance/confidence output
- **Summary**: OpenVINS maintains EKF covariance and exposes full/marginal covariance helpers, making it the strongest reference for covariance consistency even if GPLv3 blocks default production use.
- **Related Sub-question**: Mode B round 3 — confidence/covariance support
_docs/00_research/02_fact_cards.md
+348
View File
@@ -0,0 +1,348 @@
# Fact Cards
## Fact #1
- **Statement**: Fixed-wing high-altitude monocular VO suffers from scale ambiguity and accumulated error; comparing against satellite imagery can reduce accumulated drift.
- **Source**: Source #1
- **Phase**: Phase 2
- **Target Audience**: UAV localization implementers
- **Confidence**: High
- **Related Dimension**: Architecture
- **Fit Impact**: Supports satellite-anchored hybrid estimator.
## Fact #2
- **Statement**: Aerial VPR is sensitive to weather, season, scale variation, repetitive patterns, and map tile construction; overlap and scale level materially affect retrieval quality.
- **Source**: Source #2
- **Confidence**: High
- **Related Dimension**: VPR
- **Fit Impact**: Supports AC-8.6 VPR chunks with overlap and seasonal validation.
## Fact #3
- **Statement**: Heavy VPR re-ranking can be too slow for steady-state embedded use; survey evidence reports some re-ranking around 1 s and SuperGlue much slower on evaluated hardware.
- **Source**: Source #2
- **Confidence**: High
- **Related Dimension**: Runtime
- **Fit Impact**: Disqualifies per-frame global VPR/re-ranking unless profiled on Jetson.
## Fact #4
- **Statement**: OpenVINS is an EKF/MSCKF visual-inertial estimator with monocular tracking and calibration support, but its code is GPL-3.
- **Source**: Source #3
- **Confidence**: High
- **Related Dimension**: VO/VIO
- **Fit Impact**: Reference/benchmark only unless GPL obligations are accepted.
## Fact #5
- **Statement**: ORB-SLAM3 supports monocular visual-inertial SLAM and multi-map operation, but it is GPLv3 and expects careful calibration and a SLAM-style runtime stack.
- **Source**: Source #4
- **Confidence**: High
- **Related Dimension**: VO/VIO
- **Fit Impact**: Rejected as production dependency; useful benchmark/reference.
## Fact #6
- **Statement**: OpenCV provides camera calibration APIs that output camera matrix and distortion coefficients, and homography estimation APIs including RANSAC.
- **Source**: Source #5
- **Confidence**: High
- **Related Dimension**: Calibration / geometry
- **Fit Impact**: Selected utility layer for calibration, undistortion, homography, and geometric validation.
## Fact #7
- **Statement**: LightGlue accepts local keypoints/descriptors from extractors such as DISK, ALIKED, SIFT, and SuperPoint, and returns matched keypoint indices, coordinates, and confidence scores.
- **Source**: Source #6
- **Confidence**: High
- **Related Dimension**: Local matching
- **Fit Impact**: Selected candidate for conditional cross-domain local matching.
## Fact #8
- **Statement**: LightGlue has adaptive depth/width pruning, FlashAttention, mixed precision, and benchmark scripts; runtime must be profiled on Jetson because defaults are optimized for desktop GPUs.
- **Source**: Source #6
- **Confidence**: High
- **Related Dimension**: Runtime
- **Fit Impact**: Selected with runtime-quality gate.
## Fact #9
- **Statement**: LightGlue code/weights are Apache-2.0, but SuperPoint pretrained weights/inference have restrictive licensing; DISK and ALIKED are safer extractor pairings from a licensing perspective.
- **Source**: Source #6
- **Confidence**: High
- **Related Dimension**: Licensing
- **Fit Impact**: Select DISK/ALIKED+LightGlue for production candidate; treat SuperPoint as license-gated.
## Fact #10
- **Statement**: AnyLoc provides DINOv2 feature extraction and VLAD aggregation APIs, but its full experiment setup notes large storage/compute requirements.
- **Source**: Source #7
- **Confidence**: High
- **Related Dimension**: VPR descriptors
- **Fit Impact**: DINOv2-VLAD selected as offline/conditional retrieval candidate, not unconditional per-frame path.
## Fact #11
- **Statement**: DINOv2 official repository provides Meta's DINOv2 implementation and model assets with Apache-2.0 / CC-BY-4.0 license notices.
- **Source**: Source #8
- **Confidence**: High
- **Related Dimension**: VPR descriptors
- **Fit Impact**: Supports DINOv2 as a permissible descriptor backbone subject to model-license review.
## Fact #12
- **Statement**: FAISS is designed for efficient dense vector similarity search, top-k nearest-neighbor retrieval, speed/accuracy tradeoffs, and indexes too large for simple exhaustive scanning.
- **Source**: Source #9
- **Confidence**: High
- **Related Dimension**: Descriptor retrieval
- **Fit Impact**: Selected vector index for offline VPR descriptors.
## Fact #13
- **Statement**: FAISS supports saving/loading indexes; GPU indexes must be converted to CPU before saving.
- **Source**: Source #9
- **Confidence**: High
- **Related Dimension**: Cache lifecycle
- **Fit Impact**: Supports install-time/index-build flow with runtime load.
## Fact #14
- **Statement**: MAVSDK provides telemetry subscriptions for raw GPS, GPS info, status text, odometry, and position/velocity; it does not remove the need for raw MAVLink control over `GPS_INPUT` emission.
- **Source**: Source #10
- **Confidence**: High
- **Related Dimension**: MAVLink integration
- **Fit Impact**: Select MAVSDK for telemetry, pymavlink/raw MAVLink for `GPS_INPUT`.
## Fact #15
- **Statement**: ArduPilot GPSInput requires `GPS1_TYPE=14` for MAVLink GPS input.
- **Source**: Source #11
- **Confidence**: High
- **Related Dimension**: MAVLink output
- **Fit Impact**: Confirms production parameter requirement.
## Fact #16
- **Statement**: `GPS_INPUT` carries WGS84 lat/lon, MSL altitude, velocity, `fix_type`, `horiz_accuracy`, `vert_accuracy`, `speed_accuracy`, and ignore flags.
- **Source**: Source #12
- **Confidence**: High
- **Related Dimension**: Output contract
- **Fit Impact**: Supports mapping estimator covariance to `horiz_accuracy` and failover fix types.
## Fact #17
- **Statement**: ArduPilot GPS glitch protection and EKF failsafe behavior are parameterized and vehicle-specific; Copter docs are not enough to prove Plane behavior.
- **Source**: Sources #13, #14
- **Confidence**: High
- **Related Dimension**: Failsafe
- **Fit Impact**: Requires ArduPilot Plane SITL validation.
## Fact #18
- **Statement**: Jetson Orin Nano Super provides 67 INT8 TOPS, 8 GB memory, 102 GB/s bandwidth, and 7-25 W power range.
- **Source**: Source #15
- **Confidence**: High
- **Related Dimension**: Runtime
- **Fit Impact**: Confirms target platform constraint.
## Fact #19
- **Statement**: NVIDIA warns Super power modes require thermal design that can handle the power modes; otherwise throttling can reduce performance.
- **Source**: Source #16
- **Confidence**: High
- **Related Dimension**: Thermal
- **Fit Impact**: Supports AC-NEW-5 hot-soak and throttle logging.
## Fact #20
- **Statement**: PMTiles is efficient for single-file tile reads but is read-only and cannot be updated in place.
- **Source**: Source #17
- **Confidence**: High
- **Related Dimension**: Cache storage
- **Fit Impact**: Rejected for mutable onboard tile writes; possible export/package format only.
## Fact #21
- **Statement**: COG supports tiled, compressed, overview-enabled GeoTIFFs suitable for efficient raster access and geospatial tooling.
- **Source**: Source #18
- **Confidence**: High
- **Related Dimension**: Cache storage
- **Fit Impact**: Selected imagery storage unit for immutable service tiles and generated candidate tiles.
## Fact #22
- **Statement**: AerialVL provides aerial visual localization sequences, reference maps, and geo-referenced evaluation data.
- **Source**: Source #19
- **Confidence**: Medium
- **Related Dimension**: Validation
- **Fit Impact**: Selected validation dataset for VPR/satellite-anchor algorithm development.
## Fact #23
- **Statement**: EuRoC provides synchronized camera/IMU and ground truth for VIO, but it is not representative of high-altitude fixed-wing nadir imagery.
- **Source**: Source #20
- **Confidence**: High
- **Related Dimension**: Validation
- **Fit Impact**: Use for VIO sanity checks only, not final AC proof.
## MVE Evidence
### MVE — OpenCV calibration and homography utilities
- **Source**: Source #5
- **Pinned mode/config**: Use OpenCV 4.x C++/Python APIs for checkerboard calibration, undistortion, homography estimation with RANSAC, and reprojection-error measurement.
- **Inputs in example**: Object/image point correspondences, image size, matched keypoints.
- **Outputs in example**: Camera matrix, distortion coefficients, rotation/translation vectors, homography matrix.
- **Project inputs**: ADTi nav-camera frames, checkerboard calibration images, matched VO/satellite points.
- **Project outputs required**: Intrinsics/distortion, homography, inlier mask, MRE.
- **Match assessment**: Exact match.
### MVE — LightGlue in DISK/ALIKED local-matching mode
- **Source**: Source #6
- **Pinned mode/config**: Use DISK+LightGlue or ALIKED+LightGlue on CUDA/TensorRT-profiled Jetson path, with inputs two normalized images and outputs matched keypoint coordinates plus confidence scores.
- **Inputs in example**: Two images loaded to GPU; local features extracted by DISK/ALIKED/SuperPoint.
- **Outputs in example**: `matches` shape `(K, 2)`, keypoint coordinates in each image, confidence scores.
- **Project inputs**: Orthorectified nav frame crop and candidate satellite/VPR chunk.
- **Project outputs required**: 2D-2D correspondences for RANSAC homography and cross-domain MRE.
- **Match assessment**: Exact interface match; runtime quality gate remains.
### MVE — FAISS top-K VPR retrieval
- **Source**: Source #9
- **Pinned mode/config**: Use FAISS CPU index with optional GPU acceleration for top-K nearest neighbor search over precomputed DINOv2/VLAD descriptors, saved/loaded at install/preflight time.
- **Inputs in example**: Float32 descriptor matrix, query descriptor, `k`.
- **Outputs in example**: Distance matrix `D` and index matrix `I`.
- **Project inputs**: Precomputed VPR chunk descriptors, query frame descriptor.
- **Project outputs required**: Top-K candidate chunk IDs for local matching.
- **Match assessment**: Exact match.
### MVE — MAVSDK telemetry + pymavlink GPS_INPUT
- **Source**: Sources #10, #11, #12
- **Pinned mode/config**: Use MAVSDK for telemetry subscriptions and pymavlink/raw MAVLink for `GPS_INPUT` emission to ArduPilot with `GPS1_TYPE=14`.
- **Inputs in example**: Telemetry streams, estimator lat/lon/alt/velocity/covariance.
- **Outputs in example**: `GPS_INPUT` fields accepted by ArduPilot GPS backend.
- **Project inputs**: ESKF state and covariance, source label, mode/fix quality.
- **Project outputs required**: Frame-by-frame WGS84 `GPS_INPUT`, status text, FDR record.
- **Match assessment**: Exact match for output contract; Plane SITL validation remains.
## Mode B Findings
### Fact #24
- **Statement**: DINOv2 TensorRT optimization on Jetson may provide limited speedup and can change embedding distances; descriptor fidelity must be tested against the PyTorch/ONNX baseline before selecting a TensorRT descriptor path.
- **Source**: Sources #21, #22
- **Phase**: Mode B
- **Confidence**: Medium
- **Related Dimension**: VPR runtime / quality
- **Fit Impact**: Adds embedding-fidelity gate; keeps DINOv2 selected only after profiling.
### Fact #25
- **Statement**: LightGlue's SuperPoint path has documented license concerns; DISK/ALIKED remain the safer production default unless legal review approves SuperPoint.
- **Source**: Source #23
- **Phase**: Mode B
- **Confidence**: High
- **Related Dimension**: Licensing
- **Fit Impact**: Confirms draft01 decision to avoid SuperPoint as default.
### Fact #26
- **Statement**: ArduPilot `GPS_INPUT_IGNORE_FLAG_VEL_HORIZ` has a reported EKF3 pitfall where velocity may become zero rather than truly ignored; SITL must validate velocity-source parameters and message fields.
- **Source**: Source #24
- **Phase**: Mode B
- **Confidence**: Medium
- **Related Dimension**: MAVLink integration
- **Fit Impact**: Adds a specific MAVLink test and parameter gate.
### Fact #27
- **Statement**: FAISS deployment on Jetson ARM64 should assume CPU FAISS by default; GPU FAISS packages are not the safe default on aarch64.
- **Source**: Source #25
- **Phase**: Mode B
- **Confidence**: Medium
- **Related Dimension**: Descriptor retrieval runtime
- **Fit Impact**: Changes FAISS pinned mode from CPU with optional GPU to CPU-first, with custom GPU build only as future optimization.
### Fact #28
- **Statement**: Visual matching with orthophotos is a known GNSS-denied UAV approach, but available sources do not prove robustness against adversarial visual attacks on imagery/cache content.
- **Source**: Source #26
- **Phase**: Mode B
- **Confidence**: Medium
- **Related Dimension**: Security
- **Fit Impact**: Adds cache integrity, signed manifests, and consistency checks as required controls.
### Fact #29
- **Statement**: COG creation is a write-new-object workflow; the live onboard cache should append/replace tile objects through manifests, not mutate a COG in place.
- **Source**: Source #18
- **Phase**: Mode B
- **Confidence**: High
- **Related Dimension**: Cache lifecycle
- **Fit Impact**: Clarifies cache implementation.
### Fact #30
- **Statement**: OpenVINS is technically stronger than a pure hand-rolled OpenCV-only VIO stack for camera+IMU odometry, but its GPLv3 license and generic VIO lifecycle make it unsuitable as the default production dependency for this product.
- **Source**: Sources #27, #28
- **Phase**: Mode B round 2
- **Confidence**: High
- **Related Dimension**: VO / VIO selection
- **Fit Impact**: Use OpenVINS as a mandatory benchmark/reference, not as the shipped estimator dependency unless GPL obligations are explicitly accepted.
### Fact #31
- **Statement**: The selected production estimator is not "custom OpenCV-only"; OpenCV is the geometry utility layer, while the product-owned ESKF/mode machine owns covariance, source labels, GPS spoofing, blackout, tile-write eligibility, and MAVLink semantics.
- **Source**: Sources #5, #29; AC-1.4, AC-3.5, AC-4.3, AC-NEW-4, AC-NEW-7, AC-NEW-8
- **Phase**: Mode B round 2
- **Confidence**: High
- **Related Dimension**: Estimator ownership
- **Fit Impact**: Keep custom production estimator, but reject any interpretation that means building a naive OpenCV-only VIO stack.
### Fact #32
- **Statement**: Fixed-wing GPS-denied UAV research supports a hybrid of visual odometry plus satellite/orthophoto matching to reduce accumulated drift, matching the project architecture better than a standalone VIO-only solution.
- **Source**: Sources #1, #26
- **Phase**: Mode B round 2
- **Confidence**: High
- **Related Dimension**: Architecture
- **Fit Impact**: Confirms that OpenVINS alone cannot satisfy the absolute-position and re-anchor responsibilities without the satellite anchor path.
### Fact #33
- **Statement**: DINOv2-VLAD/AnyLoc-style retrieval is a strong global candidate generator for aerial VPR, but descriptor size, model size, and environment-specific VLAD/index choices must be budgeted and profiled.
- **Source**: Sources #7, #30, #32
- **Phase**: Mode B round 2
- **Confidence**: High
- **Related Dimension**: Satellite retrieval
- **Fit Impact**: Select DINOv2-VLAD for triggered retrieval, not steady-state per-frame execution.
### Fact #34
- **Statement**: Aerial VPR sources emphasize tile/chunk scale, overlap, weather/season changes, repetitive patterns, and re-ranking cost; local matching should be a verification/rerank stage over bounded top-K candidates.
- **Source**: Source #32
- **Phase**: Mode B round 2
- **Confidence**: High
- **Related Dimension**: Anchor verification
- **Fit Impact**: Supports VPR chunks with 40-50% overlap, dynamic K, and conditional ALIKED/LightGlue verification.
### Fact #35
- **Statement**: ALIKED + LightGlue has an exact local matching interface and a plausible ONNX/TensorRT deployment path, but public evidence does not prove Jetson Orin Nano p95 latency for the project image sizes.
- **Source**: Sources #6, #31
- **Phase**: Mode B round 2
- **Confidence**: Medium
- **Related Dimension**: Local matching runtime
- **Fit Impact**: Keep ALIKED/LightGlue selected with runtime gate; benchmark DISK and SIFT/ORB as fallbacks.
### Fact #36
- **Statement**: DINOv2 TensorRT conversion can reduce embedding discrimination and may not provide meaningful speedup on Jetson-class devices; descriptor-fidelity tests must precede any optimized engine acceptance.
- **Source**: Source #22
- **Phase**: Mode B round 2
- **Confidence**: Medium
- **Related Dimension**: VPR deployment
- **Fit Impact**: TensorRT is an optimization candidate only after PyTorch/ONNX retrieval-rank equivalence is proven.
### Fact #37
- **Statement**: BASALT is the best production VIO candidate among BASALT, OpenVINS, and Kimera-VIO because it combines permissive licensing with strong published EuRoC accuracy and completion evidence.
- **Source**: Sources #33, #34, #35
- **Phase**: Mode B round 3
- **Confidence**: Medium
- **Related Dimension**: VO / VIO selection
- **Fit Impact**: Select BASALT as the production VIO candidate, pending project replay/profiling.
### Fact #38
- **Statement**: OpenVINS has the clearest EKF covariance story, including full/marginal covariance helpers and NEES-style evaluation support, but remains production-constrained by GPLv3.
- **Source**: Sources #27, #28, #38
- **Phase**: Mode B round 3
- **Confidence**: High
- **Related Dimension**: Confidence / covariance
- **Fit Impact**: Keep OpenVINS as covariance/reference baseline and use it to calibrate the BASALT wrapper's reported uncertainty.
### Fact #39
- **Statement**: Kimera-VIO is production-friendly from a license standpoint, but it is heavier/stereo-oriented and has documented mono-inertial parameter/performance caveats.
- **Source**: Sources #34, #36
- **Phase**: Mode B round 3
- **Confidence**: Medium
- **Related Dimension**: VO / VIO fallback
- **Fit Impact**: Keep Kimera-VIO as a backup candidate, not the first production choice for a single fixed nadir camera.
### Fact #40
- **Statement**: None of BASALT, OpenVINS, or Kimera-VIO provides a special fixed-wing nadir mode; downward-camera support depends on accurate camera-to-IMU extrinsics, altitude/scale constraints, and validation under low-parallax planar terrain.
- **Source**: Source #37
- **Phase**: Mode B round 3
- **Confidence**: High
- **Related Dimension**: Nadir-camera support
- **Fit Impact**: The architecture must keep satellite anchors and project-level confidence gates regardless of which VIO library is selected.
### Fact #41
- **Statement**: Published EuRoC-type VIO error rates are useful for ranking libraries but are not acceptance evidence for high-altitude fixed-wing nadir imagery over agricultural terrain.
- **Source**: Sources #34, #35, #37
- **Phase**: Mode B round 3
- **Confidence**: High
- **Related Dimension**: Validation
- **Fit Impact**: Require representative replay/flight data before claiming AC-1/AC-2 accuracy.
_docs/00_research/03_comparison_framework.md
+55
View File
@@ -0,0 +1,55 @@
# Comparison Framework
## Selected Framework Type
Decision support with exact-fit component selection.
## Selected Dimensions
1. Required inputs/outputs and ownership boundaries
2. Operating context and lifecycle fit
3. Non-functional envelope fit: latency, memory, storage, thermal, safety
4. Licensing and deployability
5. Evidence quality and validation burden
6. Security and safety failure modes
7. Selection status
## Initial Population
| Component Area | Candidate | Option Family | Inputs / Outputs | Fit Summary | Status |
|----------------|-----------|---------------|------------------|-------------|--------|
| Calibration / geometry | OpenCV 4.x | Established production / open-source | image/object points -> intrinsics, distortion, homography, inlier mask | Exact utility fit; permissive production use. | Selected |
| VO / VIO | BASALT + project-owned safety/anchor wrapper | Open-source production candidate | nav frames + FC IMU/calibration -> relative VIO state; wrapper adds source labels, anchor fusion, calibrated confidence, and MAVLink semantics | Best production candidate after user decision: permissive license, strong benchmark evidence, and lower implementation burden than custom VIO. | Selected |
| VO / VIO | OpenVINS | Open-source research | monocular camera + IMU -> VIO state/covariance | Best covariance/reference story, but GPL-3 and generic VIO ownership make it a benchmark/reference rather than shipped core. | Reference only |
| VO / VIO | Kimera-VIO | Open-source production candidate / fallback | mono or stereo camera + IMU -> VIO/SLAM outputs | BSD-friendly, but heavier/stereo-oriented and mono-inertial path has documented parameter caveats. | Backup candidate |
| VO / VIO | OpenCV geometry + project-owned ESKF | Custom fallback | nav frames + FC IMU/attitude/altitude + satellite anchors -> relative motion, absolute updates, covariance, source labels | Fallback if BASALT fails project data/runtime tests; still needed as safety/anchor wrapper around any VIO library. | Fallback / wrapper |
| VO / VIO | ORB-SLAM3 | Open-source research | monocular/stereo/RGB-D + optional IMU -> SLAM pose | Useful benchmark; GPLv3, map runtime, and initialization complexity make it poor production dependency. | Rejected for production |
| Global retrieval | DINOv2-VLAD / AnyLoc-style descriptors | Current SOTA / research | image/chunk -> descriptor | Strong VPR evidence; trigger-only use due runtime/memory and TensorRT fidelity risk. | Selected with runtime gate |
| Vector retrieval | FAISS | Established production / open-source | descriptor matrix + query -> top-K IDs | Exact fit for offline VPR chunk retrieval. | Selected |
| Local matching | LightGlue + DISK/ALIKED | Current SOTA / open-source | two images -> keypoint correspondences/scores | Exact local-match interface; avoids SuperPoint license issue. | Selected with runtime gate |
| Local matching | SuperPoint + LightGlue | Current SOTA / known-bad/licensing caveat | two images -> matches | Technically good; licensing requires explicit review. | Needs user decision / fallback only |
| Cache imagery | COG + manifest + sidecars | Established geospatial format | georeferenced raster + metadata -> efficient local reads | Good immutable tile unit; generated tiles can be written as new COGs. | Selected |
| Cache packaging | PMTiles | Established web-map archive | tile pyramid -> single archive | Efficient reads, but read-only; not suitable for in-flight mutable writes. | Rejected for mutable cache |
| Estimator | Custom ESKF mode machine | Custom production | VO/IMU/VPR/GPS-health -> WGS84 state + covariance + label | Needed for source labels, covariance gates, blackout/spoofing behavior. | Selected |
| MAVLink integration | MAVSDK + pymavlink | Established APIs | telemetry in, `GPS_INPUT` out | MAVSDK handles telemetry; pymavlink handles raw `GPS_INPUT`. | Selected |
| FDR | PostgreSQL event index + CBOR payload segments with optional Parquet export | Established storage | frame estimates, IMU, MAVLink, health, tiles -> bounded replayable log | Matches project PostgreSQL choice while keeping compact append payloads. | Selected pattern |
| Validation | AerialVL + EuRoC + Plane SITL + representative flight | Multi-source test strategy | datasets/sim/flight -> AC evidence | Public data is partial; final representative data is mandatory. | Selected |
## Round 2 Decision Notes
- **OpenVINS vs custom OpenCV**: OpenVINS wins if the comparison is against a naive OpenCV-only VIO implementation. The selected design is not that; it is OpenCV geometry plus a product-owned estimator/state machine, with OpenVINS used as a benchmark/reference.
- **Satellite retrieval**: DINOv2-VLAD remains the best global candidate generator found, but aerial VPR sources require chunk scale/overlap tuning, dynamic top-K, and geometric verification.
- **Anchor verification**: ALIKED/LightGlue remains the preferred learned local matcher, while SIFT/ORB stays as a regression/fallback baseline and SuperPoint remains license-gated.
## Round 3 Decision Notes
- **User decision**: BASALT is selected as the production VIO candidate.
- **Confidence/covariance**: OpenVINS remains the covariance/reference baseline because its EKF exposes clearer uncertainty semantics than BASALT/Kimera.
- **Nadir support**: no compared VIO library has a special fixed-wing nadir mode; the acceptance path is calibration, altitude/scale constraints, satellite anchors, and representative replay validation.
## Baseline Alignment
- "Position estimate" means WGS84 frame center emitted to FC plus a 95% covariance semi-major axis and source label.
- "Satellite anchored" means a visual match passed VPR retrieval, local matching, RANSAC, freshness, covariance, and Mahalanobis gates.
- "Normal flight segment" means the AC-2.1a conditions, not turns/blackout/stale imagery.
- "Selected with runtime gate" means the API capability fits, but final deployment depends on Jetson profiling against AC-4.1 and AC-4.2.
_docs/00_research/04_reasoning_chain.md
+181
View File
@@ -0,0 +1,181 @@
# Reasoning Chain
## Dimension 1: Core Architecture
### Fact Confirmation
Fixed-wing monocular VO accumulates drift because scale and terrain assumptions are imperfect (Fact #1). Aerial VPR can provide absolute anchors but has weather, scale, and repetition failure modes (Fact #2).
### Reference Comparison
Pure VO/IMU cannot satisfy the long 8-hour mission. Pure satellite matching cannot run every frame inside the latency budget and will fail in turns, stale imagery, and repetitive fields.
### Conclusion
Select a hybrid architecture: steady-state VO/IMU propagation, conditional satellite/VPR anchoring, and ESKF covariance gates.
### Confidence
High. Supported by Sources #1 and #2.
## Dimension 2: VO / VIO Dependency
### Fact Confirmation
OpenVINS and ORB-SLAM3 both support visual-inertial estimation patterns, but both are GPL-family production dependency risks (Facts #4 and #5).
### Reference Comparison
The project needs a production system with custom mode labels, covariance propagation, MAVLink-specific output behavior, and no hidden GPL obligation. A full SLAM stack also adds initialization and map-management complexity that the ACs do not require.
### Conclusion
Use a custom VO/IMU propagation and ESKF mode machine for production. Use OpenVINS/ORB-SLAM3 only as benchmarks/reference algorithms.
### Confidence
High for licensing/scope decision; medium for final estimator performance until prototype profiling.
## Dimension 3: Satellite Retrieval And Local Matching
### Fact Confirmation
Aerial VPR benefits from scale/overlap-aware chunks and retrieval + local alignment (Fact #2). LightGlue provides keypoint correspondences/scores from local descriptors (Fact #7). FAISS provides top-K retrieval over descriptors (Fact #12).
### Reference Comparison
Global DINOv2/VLAD retrieval alone gives candidate chunks but not enough precision for AC-2.2. Local matching alone over the whole map is too expensive. The two-stage retrieval+alignment structure matches the operational need.
### Conclusion
Use DINOv2-VLAD descriptors with FAISS top-K for conditional candidate retrieval, followed by DISK/ALIKED+LightGlue and OpenCV RANSAC homography for local alignment.
### Confidence
Medium-high. API fit is strong; embedded runtime must be profiled.
## Dimension 4: Latency And Memory
### Fact Confirmation
Some aerial VPR re-ranking methods are too slow for the steady-state path (Fact #3). Jetson Orin Nano Super has 8 GB shared memory and 25 W power mode (Fact #18), with thermal throttling risk (Fact #19).
### Reference Comparison
At 3 fps and <400 ms p95, the system can process every frame through lightweight VO/IMU and use heavier VPR only on triggers. Running DINOv2/LightGlue across many candidates per frame would violate the AC unless proven otherwise.
### Conclusion
Make VPR conditional, cap top-K by covariance/sector, run descriptor extraction on downsampled/orthorectified crops, precompute cache descriptors offline, and add performance regression tests.
### Confidence
High for architecture direction; exact model sizes need profiling.
## Dimension 5: Cache Format
### Fact Confirmation
COG supports tiled compressed rasters and geospatial tooling (Fact #21). PMTiles is read-efficient but read-only (Fact #20).
### Reference Comparison
The onboard system must both read service tiles and write new generated tiles in-flight. A read-only archive is a poor primary mutable store.
### Conclusion
Use COG files plus STAC-like manifests/sidecars for imagery and metadata; use FAISS sidecar indexes for descriptors. PMTiles may be an export/snapshot format, not the live mutable cache.
### Confidence
High.
## Dimension 6: Autopilot Integration
### Fact Confirmation
ArduPilot MAVLink GPS input requires `GPS1_TYPE=14` (Fact #15). `GPS_INPUT` carries the fields needed for WGS84 position and accuracy (Fact #16). MAVSDK covers telemetry but raw `GPS_INPUT` emission still needs pymavlink/raw MAVLink (Fact #14).
### Reference Comparison
MAVSDK-only output would not satisfy AC-4.3. Raw pymavlink-only telemetry is possible but gives up MAVSDK's high-level subscriptions.
### Conclusion
Use MAVSDK for telemetry subscriptions and pymavlink for `GPS_INPUT` emission. Verify all failsafe/spoofing behavior in ArduPilot Plane SITL.
### Confidence
High for interface; medium for exact Plane failsafe timing until SITL.
## Dimension 7: Validation
### Fact Confirmation
AerialVL provides aerial localization/reference-map data (Fact #22). EuRoC provides camera+IMU+ground truth but not fixed-wing nadir imagery (Fact #23). The current sample set lacks IMU/ground truth.
### Reference Comparison
No single public dataset proves every AC. Public datasets can de-risk components, while final acceptance requires representative synchronized flight/replay data.
### Conclusion
Use layered validation: EuRoC for VIO sanity, AerialVL/VPAir-style data for VPR/anchor tests, ArduPilot Plane SITL for MAVLink/failsafe/spoofing, and a final representative flight/replay rig for AC proof.
### Confidence
High.
## Dimension 8: OpenVINS vs Project-Owned Estimator
### Fact Confirmation
OpenVINS is a strong monocular+IMU EKF/MSCKF VIO reference with covariance-aware estimation (Facts #4 and #30). OpenCV supplies calibration, undistortion, homography, RANSAC/USAC, and reprojection-error primitives, but it is not a full estimator by itself (Facts #6 and #31).
### Reference Comparison
If the alternative is a naive OpenCV-only VIO stack, OpenVINS is the better technical starting point. The project's actual production choice is different: it needs a project-owned ESKF/mode machine that fuses VO/IMU, accepts/rejects satellite anchors, emits `GPS_INPUT`, labels every estimate, handles spoofing/blackout, and gates cache write-back.
### Conclusion
Keep OpenVINS as a mandatory benchmark/reference implementation, not as the default production dependency. The production estimator remains project-owned, with OpenCV as the geometry utility layer.
### Confidence
High. The technical comparison favors OpenVINS over naive custom VIO, while the product fit favors the project-owned estimator.
## Dimension 9: Satellite Retrieval And Anchor Verification
### Fact Confirmation
DINOv2-VLAD/AnyLoc-style retrieval has strong VPR evidence but descriptor/model size and TensorRT fidelity must be validated (Facts #33 and #36). Aerial VPR survey evidence emphasizes tile scale, overlap, season/weather shifts, repetitive patterns, and re-ranking cost (Fact #34). LightGlue supports ALIKED/DISK/SIFT feature matching and returns correspondences/scores suitable for RANSAC verification, but Jetson latency must be profiled (Facts #7, #8, #35).
### Reference Comparison
Classical SIFT/ORB is simpler and cheap but weaker for cross-domain UAV-to-satellite matching. SuperPoint+LightGlue is technically strong but remains license-gated. Pure global retrieval without local verification is unsafe because repetitive farmland and stale imagery can produce plausible but wrong candidates.
### Conclusion
Use DINOv2-VLAD + CPU-first FAISS as the triggered global retriever, then verify bounded top-K candidates with ALIKED/LightGlue + OpenCV RANSAC. Keep SIFT/ORB as a regression baseline and SuperPoint only after legal approval.
### Confidence
High for architecture and interfaces; medium for final runtime until Jetson profiling.
## Dimension 10: BASALT vs Kimera-VIO vs OpenVINS
### Fact Confirmation
BASALT has strong published EuRoC evidence and production-friendly licensing (Fact #37). OpenVINS has the clearest EKF covariance API and consistency tooling, but GPLv3 remains a production constraint (Fact #38). Kimera-VIO is BSD-friendly but heavier and has documented mono-inertial caveats (Fact #39). All three require calibrated camera-to-IMU extrinsics; none has a special fixed-wing nadir mode (Fact #40).
### Reference Comparison
For a single fixed downward camera, the selection criterion is not just benchmark ATE. The project needs a VIO core that can run on Jetson, tolerate calibrated nadir geometry, and be wrapped by project-specific satellite-anchor, confidence, MAVLink, and safety logic. OpenVINS is attractive for confidence/covariance but problematic as a shipped component. Kimera is acceptable as a BSD fallback, but mono-inertial risk makes it weaker as the first pick. BASALT provides the best production trade-off if its uncertainty can be calibrated and wrapped.
### Conclusion
Select BASALT as the production VIO candidate. Keep OpenVINS as a reference/covariance baseline and Kimera-VIO as a backup candidate. The project-owned safety/anchor wrapper remains mandatory around BASALT because BASALT alone does not satisfy GPS-denied source labels, satellite anchors, false-position budgets, cache-write gates, or `GPS_INPUT` behavior.
### Confidence
Medium-high. Library ranking is well supported; final acceptance still depends on representative fixed-wing nadir replay data.
_docs/00_research/05_validation_log.md
+51
View File
@@ -0,0 +1,51 @@
# Validation Log
## Validation Scenario
An 8-hour fixed-wing mission enters GPS-denied/spoofed mode after takeoff. The onboard system starts from last trusted FC state, processes 3 fps nadir frames, emits `GPS_INPUT`, handles normal flight, sharp turns, short visual blackouts, stale/changed tiles, and post-flight tile write-back.
## Expected Based On Conclusions
- **Normal segment**: VO/IMU propagates every processed frame; satellite anchors refresh state conditionally before covariance grows too large.
- **Sharp turn / <5% overlap**: VO is expected to fail; relocalization uses FAISS top-K VPR chunks followed by LightGlue/RANSAC.
- **Visual blackout + spoofing**: estimator switches to `dead_reckoned`, covariance grows monotonically, spoofed GPS is ignored, `GPS_INPUT` degrades honestly.
- **Stale tile**: anchor is rejected or down-confidence weighted and cannot emit `satellite_anchored`.
- **Cache write-back**: onboard generated tile is written only when parent-pose covariance passes AC-NEW-7 gates and carries metadata for Satellite Service voting.
## Actual Validation Plan
| Validation Target | Test Method | Pass Evidence |
|-------------------|-------------|---------------|
| VO/VIO propagation | EuRoC and synthetic nadir replay; then representative flight data | Drift vs anchor-age bins; AC-1.3 pass/fail. |
| Satellite anchor | AerialVL/VPAir-style benchmark plus project sample imagery with satellite cache | AC-1.1/1.2 accuracy, AC-2.2 MRE, georeference recall. |
| Runtime | Jetson Orin Nano Super profiling under 25 W, hot-soak included | <400 ms p95, <8 GB memory, no thermal throttle. |
| VPR retrieval | Offline descriptor build and FAISS query benchmark | Top-K recall, query latency, index size within cache budget. |
| MAVLink output | ArduPilot Plane SITL with `GPS1_TYPE=14` | Valid `GPS_INPUT`, fix-type/accuracy degradation, QGC status. |
| Spoofing promotion | Plane SITL false GPS injection | Promotion <3 s and spoofed GPS rejected during blackout. |
| FDR | 8-hour synthetic load | <=64 GB, rollover logged, no silent payload loss. |
| Cache poisoning | Monte Carlo with over-confident wrong anchors | AC-NEW-7 probabilities below budget; metadata contract emitted. |
| OpenVINS reference comparison | Replay the same synchronized camera+IMU segments through OpenVINS and the project-owned estimator | OpenVINS establishes a VIO baseline; production estimator must match/beat drift where applicable while preserving source labels and GPS_INPUT behavior. |
| BASALT production VIO candidate | Replay synchronized camera+IMU segments through BASALT, OpenVINS, and Kimera-VIO | BASALT selected if drift, completion rate, latency, and wrapper-calibrated covariance meet project gates. |
| DINOv2-VLAD fidelity | Compare PyTorch, ONNX, and TensorRT descriptor distances and FAISS rankings | Optimized engines accepted only if rank/top-K behavior stays within tolerance. |
| ALIKED/LightGlue runtime | Jetson benchmark across K candidates and project image sizes | Candidate accepted for runtime only if relocalization trigger path fits AC-4.1 with bounded frame drops. |
## Counterexamples And Risks
- Large DINOv2 variants or many local-match candidates may violate the Jetson latency/memory envelope.
- Agricultural fields can be visually repetitive; VPR confidence must not be treated as sufficient without geometric verification.
- Public datasets do not fully match Ukrainian fixed-wing operational conditions; final evidence requires representative data.
- GPL VIO/SLAM libraries are not production dependencies unless licensing is explicitly accepted.
- OpenVINS may outperform the first custom estimator prototype on pure VIO drift; that would trigger estimator improvement, not automatic GPL production adoption.
- BASALT covariance/confidence is less directly exposed than OpenVINS EKF covariance; the project wrapper must calibrate uncertainty before mapping it to `GPS_INPUT.horiz_accuracy`.
## Review Checklist
- [x] Draft conclusions are consistent with fact cards.
- [x] No important dimensions missed: architecture, VO, VPR, local matching, cache, estimator, MAVLink, FDR, validation covered.
- [x] No selected component relies only on field-adjacent fit.
- [x] Mismatches are recorded as rejected/reference/needs-decision rather than hidden.
- [x] Step 7.5 Component Applicability Gate applies and is saved in `06_component_fit_matrix.md`.
## Conclusions Requiring Revision
Round 3 applies the user decision to select BASALT as the production VIO candidate. The selected implementation is BASALT VIO plus a project-owned safety/anchor wrapper; OpenVINS remains the covariance/reference baseline, Kimera-VIO remains a backup candidate, and custom OpenCV-only VIO is no longer the primary path. Runtime gates and Plane SITL gates are implementation validation gates, not API capability blockers.
_docs/00_research/06_component_fit_matrix.md
+107
View File
@@ -0,0 +1,107 @@
# Component Fit Matrix
## Top-Level Matrix
| Component Area | Candidate | Pinned Mode/Config | Option Family | Intended Role | API Capability Evidence | Mismatches / Disqualifiers | Status | Decision Rationale |
|----------------|-----------|--------------------|---------------|---------------|-------------------------|----------------------------|--------|--------------------|
| Calibration / geometry | OpenCV 4.x | C++/Python calibration, undistortion, RANSAC homography, reprojection-error measurement | Established production / open-source | Camera intrinsics, image normalization, VO/satellite homography verification | MVE: `02_fact_cards.md` OpenCV block; Source #5 | None | Selected | Exact API fit and permissive utility role. |
| VO / IMU propagation | BASALT + project-owned safety/anchor wrapper | BASALT VIO consumes calibrated nav-camera frames + FC IMU; wrapper fuses satellite anchors, calibrates uncertainty, emits source labels and GPS_INPUT semantics | Open-source production candidate | Relative VIO state, completion/error benchmark, wrapped covariance/confidence, degraded modes, GPS_INPUT semantics | Sources #33-#35; Facts #37, #40, #41 | BASALT covariance/confidence must be calibrated in wrapper; no special nadir mode | Selected | User-selected best production trade-off: permissive licensing and stronger benchmark/completion evidence than OpenVINS/Kimera, with wrapper covering project-specific safety semantics. |
| VO / VIO reference | OpenVINS | Monocular camera + IMU EKF/MSCKF reference runs with covariance extraction | Open-source research | Benchmark and covariance reference | Sources #3, #27, #28, #35, #38; Facts #4, #30, #38 | GPL-3 production dependency risk; completion/divergence risk on some sequences; does not own satellite anchor / GPS_INPUT / blackout / cache-write state machine | Reference only | Best covariance baseline, but not selected as shipped production dependency. |
| VO / VIO backup | Kimera-VIO | Mono/stereo camera + IMU VIO/SLAM backup candidate | Open-source production candidate / fallback | Alternative VIO baseline | Sources #34, #36; Fact #39 | Heavier/stereo-oriented; mono-inertial path has documented parameter caveats | Backup candidate | BSD-friendly backup if BASALT fails project replay/runtime gates. |
| VO / SLAM alternative | ORB-SLAM3 | Monocular-inertial SLAM | Open-source research | Benchmark and failure-mode comparison | Source #4, Fact #5 | GPLv3; heavier SLAM/map lifecycle than required | Rejected | Does not fit licensing/scope for production. |
| VPR descriptors | DINOv2-VLAD / AnyLoc-style | Precomputed satellite chunk descriptors; conditional query descriptor on relocalization triggers; TensorRT path only after embedding-fidelity check | Current SOTA / research | Global top-K candidate retrieval | Sources #7, #8, #21, #22, #30, #32; Facts #10, #11, #24, #33, #34, #36 | Runtime and embedding-fidelity gates on Jetson; model-size/index-size selection required | Selected with runtime gate | Best evidence for change-robust VPR, but not per-frame and not blindly TensorRT-converted. |
| Vector retrieval | FAISS | CPU-first aarch64 index; saved/loaded index over float/PQ descriptors; GPU only if custom Jetson build is proven | Established production / open-source | Top-K candidate chunk search | MVE: `02_fact_cards.md` FAISS block; Sources #9, #25 | GPU FAISS not default on Jetson ARM64 | Selected | Exact top-K descriptor retrieval fit with CPU-first deployment. |
| Local matching | LightGlue + DISK/ALIKED | CUDA/ONNX-profiled DISK or ALIKED feature extraction + LightGlue matching on bounded top-K candidates | Current SOTA / open-source | 2D-2D correspondences for RANSAC and MRE | MVE: `02_fact_cards.md` LightGlue block; Sources #6, #23, #31 | Runtime quality gate; extractor choice must avoid SuperPoint license issue | Selected with runtime gate | Exact input/output fit with deployable licensing path; ALIKED/LightGlue is preferred for anchor verification. |
| Local matching fallback | SuperPoint + LightGlue | SuperPoint features + LightGlue | Current SOTA / license-gated | Optional benchmark/fallback | Source #6 | SuperPoint restrictive license | Needs user decision | Do not use as default production dependency without legal review. |
| Cache imagery | COG + manifest/sidecar | Tiled compressed GeoTIFF tile objects with CRS, capture date, source, m/px, freshness, descriptor sidecars; write-new-object lifecycle | Established geospatial format | Immutable service tiles and generated candidate tiles | Source #18, Facts #21, #29 | No in-place mutation; manifest manages active tile version | Selected | Fits geospatial raster access and write-new-tile workflow. |
| Cache packaging | PMTiles | Read-only tile archive | Established web-map archive | Optional export/snapshot | Source #17, Fact #20 | Cannot update in place | Rejected for live cache | In-flight tile generation needs mutable write-new objects. |
| MAVLink | MAVSDK + pymavlink | MAVSDK telemetry subscriptions; pymavlink/raw MAVLink `GPS_INPUT` emission to ArduPilot `GPS1_TYPE=14`; velocity source/ignore-flag behavior SITL-tested | Established APIs | FC telemetry, QGC status, GPS substitute output | MVE: `02_fact_cards.md` MAVSDK/pymavlink block; Sources #10-#12, #24 | Plane SITL behavior and velocity-source parameters must be validated | Selected | Exact output contract with known ArduPilot pitfall covered. |
| Validation | EuRoC + AerialVL/VPAir + Plane SITL + representative flight | Layered validation suite | Test strategy | Prove ACs before production | Sources #19, #20 | Public data not sufficient for final proof | Selected | Covers component de-risking plus final representative proof. |
## Restrictions Cross-Check — Selected Production Architecture
| Restriction | Candidate-mode behavior | Result | Evidence |
|-------------|-------------------------|--------|----------|
| Fixed-wing only | Architecture assumes forward motion, rare sharp turns, no hover dependency. | Pass | Problem context; Source #1 |
| Fixed nadir nav camera | VO/orthorectification uses fixed camera extrinsics; attitude compensation from FC. | Pass | Source #5 |
| Operational area / flat terrain | Flat terrain assumption supported; repetitive agricultural terrain handled as validation class and confidence risk. | Pass | Source #2 |
| Weather/season classes | Validation matrix includes seasonal/visibility classes. | Pass | `05_validation_log.md` |
| Two-camera split | Nav camera drives localization; AI camera object localization uses current GPS-denied state and AI gimbal/zoom. | Pass | AC-7.1/7.2 |
| Satellite Service offline boundary | Runtime uses local COG/cache + FAISS descriptors only; no in-flight provider fetch. | Pass | Sources #17, #18 |
| Freshness gates | Tile manifest carries capture date and sector; stale tiles rejected/down-weighted. | Pass | AC-8.2, AC-NEW-6 |
| Jetson Orin Nano Super | Hot path is lightweight; heavy VPR conditional; runtime profiling gate required. | Pass | Sources #15, #16 |
| MAVLink / ArduPilot only | MAVSDK telemetry + pymavlink `GPS_INPUT`, `GPS1_TYPE=14`. | Pass | Sources #10-#12 |
| No raw frame storage | FDR keeps estimates, telemetry, tiles, and low-rate failure thumbnails only. | Pass | AC-8.5, AC-NEW-3 |
## AC Cross-Check — Selected Production Architecture
| AC | Candidate-mode behavior | Result | Evidence |
|----|-------------------------|--------|----------|
| AC-1.1 | Satellite anchors and ESKF output target <=50 m for >=80% normal frames. | Pass | Facts #1, #2 |
| AC-1.2 | Same pipeline targets <=20 m for >=50% normal frames; validated statistically. | Pass | `05_validation_log.md` |
| AC-1.3 | ESKF tracks anchor age and covariance; VO-only and IMU-fused drift measured between anchors. | Pass | Fact #1 |
| AC-1.4 | ESKF emits covariance; telemetry/FDR carries source label. | Pass | Fact #16 |
| AC-2.1a | VO hot path handles normal overlapping frames; failures trigger mode change. | Pass | Facts #1, #6 |
| AC-2.1b | Satellite anchor success measured separately through VPR + local match + RANSAC. | Pass | Facts #2, #7, #12 |
| AC-2.2 | OpenCV/LightGlue provide correspondences and homography MRE measurement. | Pass | Facts #6, #7 |
| AC-3.1 | ESKF innovation gates reject outliers; covariance grows instead of trusting jumps. | Pass | Reasoning chain |
| AC-3.2 | Sharp turn VO failure triggers VPR relocalization. | Pass | AC design; Source #2 |
| AC-3.3 | Disconnected segment handled by global retrieval and pose-graph/ESKF re-anchor. | Pass | Source #2 |
| AC-3.4 | Loss counter and timer trigger GCS relocalization request while dead reckoning continues. | Pass | AC design |
| AC-3.5 | Image-quality/blackout state switches to IMU-only and rejects spoofed GPS. | Pass | AC design; Facts #16, #17 |
| AC-4.1 | Heavy VPR is conditional; steady-state path is VO/IMU. Jetson profiling is a runtime quality gate. | Pass | Facts #3, #18 |
| AC-4.2 | Descriptor/index memory is budgeted; FAISS and cache are precomputed/pruned. | Pass | Facts #12, #13 |
| AC-4.3 | `GPS_INPUT` emitted by pymavlink; ODOMETRY remains disabled in v1. | Pass | Facts #14-#16 |
| AC-4.4 | Estimator emits frame-by-frame; no batching required. | Pass | Architecture |
| AC-4.5 | Corrections are emitted as updated estimates with timestamps and covariance. | Pass | Architecture |
| AC-5.1 | Startup initializes from last trusted FC state plus IMU propagation. | Pass | Architecture |
| AC-5.2 | >3 s no-estimate path enters degraded/failsafe behavior; Plane SITL proves FC response. | Pass | Fact #17 |
| AC-5.3 | Cold restart uses FC state and preloaded cache/index. | Pass | AC-NEW-1 |
| AC-6.1 | QGC receives downsampled status; FDR keeps high-rate details. | Pass | MAVSDK telemetry + FDR design |
| AC-6.2 | Command ingress reserved through MAVLink status/named values/custom dialect. | Pass | MAVLink design |
| AC-6.3 | `GPS_INPUT` lat/lon is WGS84. | Pass | Fact #16 |
| AC-7.1 | Object localization exposes level-flight accuracy and maneuver bound. | Pass | Geometry design |
| AC-7.2 | Flat-terrain trig projection from UAV GPS + gimbal/zoom/altitude. | Pass | Geometry design |
| AC-8.1 | Cache contract requires 0.3-0.5 m/px imagery. | Pass | Restrictions |
| AC-8.2 | Freshness metadata gates anchors. | Pass | Restrictions |
| AC-8.3 | Precomputed descriptors and FAISS index are part of cache budget. | Pass | Facts #10, #12, #13 |
| AC-8.4 | Generated tiles are new COGs with quality metadata for Service ingest. | Pass | Fact #21 |
| AC-8.5 | Raw frames are not retained. | Pass | FDR design |
| AC-8.6 | VPR chunks use overlap/multi-scale descriptors and dynamic K. | Pass | Facts #2, #12 |
| AC-NEW-1 | Engines/indexes built before flight; first fix benchmark validates <30 s. | Pass | Runtime plan |
| AC-NEW-2 | Plane SITL verifies spoofing trigger and promotion <3 s. | Pass | Fact #17 |
| AC-NEW-3 | FDR segment rollover validates <=64 GB. | Pass | Validation plan |
| AC-NEW-4 | Mahalanobis gates and calibrated covariance target false-position budget. | Pass | ESKF design |
| AC-NEW-5 | Thermal profiling at 25 W validates no throttle. | Pass | Facts #18, #19 |
| AC-NEW-6 | Tile age rejection/down-weighting built into anchor gate. | Pass | AC design |
| AC-NEW-7 | Tile writes require parent-pose covariance and sidecar metadata; Satellite Service voting is external contract. | Pass | AC design |
| AC-NEW-8 | Dead-reckoned blackout mode degrades `GPS_INPUT` fields at covariance thresholds. | Pass | Facts #16, #17 |
## Decision Rules Applied
- No GPL VIO/SLAM library is selected as a production dependency.
- Runtime gates are classified as runtime-quality validation gates, not API capability blockers.
- Every selected component has an exact input/output role and a validation path.
- Any candidate with license uncertainty is marked `Needs user decision` or non-production.
## Mode B Revisions Applied
- FAISS pinned mode changed to CPU-first on Jetson ARM64.
- DINOv2 TensorRT path requires descriptor-fidelity validation against PyTorch/ONNX.
- `GPS_INPUT` tests now include velocity-source and ignore-flag behavior.
- COG cache lifecycle clarified as write-new-object plus manifest versioning, not in-place mutation.
- Visual/satellite security controls now include signed manifests, cache provenance, stale-tile rejection, and multi-signal consistency checks.
## Mode B Round 2 Revisions Applied
- OpenVINS is explicitly better than naive OpenCV-only VIO, but remains reference-only because the shipped estimator must own source labels, covariance gates, spoofing/blackout states, cache-write eligibility, and MAVLink semantics.
- The selected VO wording is now "OpenCV geometry + project-owned ESKF" to avoid implying a fragile OpenCV-only odometry implementation.
- DINOv2-VLAD + CPU-first FAISS + ALIKED/LightGlue remains selected for satellite retrieval and anchor verification, with retrieval limited to relocalization triggers and bounded top-K verification.
- SIFT/ORB remains a regression/fallback baseline; SuperPoint remains non-production until legal approval.
## Mode B Round 3 Revisions Applied
- BASALT is selected as the production VIO candidate.
- OpenVINS remains the covariance/reference baseline, not a shipped dependency by default.
- Kimera-VIO remains a backup VIO candidate because its license is production-friendly but mono-inertial caveats make it weaker for the single-nadir-camera path.
- The project-owned safety/anchor wrapper remains mandatory around BASALT for satellite anchor acceptance, source labels, blackout/spoofing modes, cache-write eligibility, calibrated confidence, and MAVLink `GPS_INPUT`.
_docs/01_solution/solution.md
+90 -582
View File
@@ -1,622 +1,130 @@
# Solution Draft
## Assessment Findings
| Old Component Solution | Weak Point (functional/security/performance) | New Solution |
| ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ESKF described as "16-state vector, ~10MB" with no mathematical specification | **Functional**: No state vector, no process model (F,Q), no measurement models (H for VO, H for satellite), no noise parameters, no scale observability analysis. Impossible to implement or validate accuracy claims. | **Define complete ESKF specification**: 15-state error vector, IMU-driven prediction, dual measurement models (VO relative pose, satellite absolute position), initial Q/R values, scale constraint via altitude + satellite corrections. |
| GPS_INPUT at 5-10Hz via pymavlink — no field mapping | **Functional**: GPS_INPUT requires 15+ fields (velocity, accuracy, hdop, fix_type, GPS time). No specification of how ESKF state maps to these fields. ArduPilot requires minimum 5Hz. | **Define GPS_INPUT population spec**: velocity from ESKF, accuracy from covariance, fix_type from confidence tier, GPS time from system clock conversion, synthesized hdop/vdop. |
| Confidence scoring "unchanged from draft03" — not in draft05 | **Functional**: Draft05 is supposed to be self-contained. Confidence scoring determines GPS_INPUT accuracy fields and fix_type — directly affects how ArduPilot EKF weights the position data. | **Define confidence scoring inline**: 3 tiers (satellite-anchored, VO-tracked, IMU-only) mapping to fix_type + accuracy values. |
| Coordinate transformations not defined | **Functional**: No pixel→camera→body→NED→WGS84 chain. Camera is not autostabilized, so body attitude matters. Satellite match → WGS84 conversion undefined. Object localization impossible without these transforms. | **Define coordinate transformation chain**: camera intrinsics K, camera-to-body extrinsic T_cam_body, body-to-NED from ESKF attitude, NED origin at mission start point. |
| Disconnected route segments — "satellite re-localization" mentioned but no algorithm | **Functional**: AC requires handling as "core to the system." Multiple disconnected segments expected. No tracking-loss detection, no re-localization trigger, no ESKF re-initialization, no cuVSLAM restart procedure. | **Define re-localization pipeline**: detect cuVSLAM tracking loss → IMU-only ESKF prediction → trigger satellite match on every frame → on match success: ESKF position reset + cuVSLAM restart → on 3 consecutive failures: operator re-localization request. |
| No startup handoff from GPS to GPS-denied | **Functional**: System reads GLOBAL_POSITION_INT at startup but no protocol for when GPS is lost/spoofed vs system start. No validation of initial position. | **Define handoff protocol**: system runs continuously, FC receives both real GPS and GPS_INPUT. GPS-denied system always provides its estimate; FC selects best source. Initial position validated against first satellite match. |
| No mid-flight reboot recovery | **Functional**: AC requires: "re-initialize from flight controller's current IMU-extrapolated position." No procedure defined. Recovery time estimation missing. | **Define reboot recovery sequence**: read FC position → init ESKF with high uncertainty → load TRT engines → start cuVSLAM → immediate satellite match. Estimated recovery: ~35-70s. Document as known limitation. |
| 3-consecutive-failure re-localization request undefined | **Functional**: AC requires ground station re-localization request. No message format, no operator workflow, no system behavior while waiting. | **Define re-localization protocol**: detect 3 failures → send custom MAVLink message with last known position + uncertainty → operator provides approximate coordinates → system uses as ESKF measurement with high covariance. |
| Object localization — "trigonometric calculation" with no details | **Functional**: No math, no API, no Viewpro gimbal integration, no accuracy propagation. Other onboard systems cannot use this component as specified. | **Define object localization**: pixel→ray using Viewpro intrinsics + gimbal angles → body frame → NED → ray-ground intersection → WGS84. FastAPI endpoint: POST /objects/locate. Accuracy propagated from UAV position + gimbal uncertainty. |
| Satellite matching — GSD normalization and tile selection unspecified | **Functional**: Camera GSD ~15.9 cm/px at 600m vs satellite ~0.3 m/px at zoom 19. The "pre-resize" step is mentioned but not specified. Tile selection radius based on ESKF uncertainty not defined. | **Define GSD handling**: downsample camera frame to match satellite GSD. Define tile selection: ESKF position ± 3σ_horizontal → select tiles covering that area. Assemble tile mosaic for matching. |
| Satellite tile storage requirements not calculated | **Functional**: "±2km" preload mentioned but no storage estimate. At zoom 19: a 200km path with ±2km buffer requires ~~130K tiles (~~2.5GB). | **Calculate tile storage**: specify zoom level (18 preferred — 0.6m/px, 4× fewer tiles), estimate storage per mission profile, define maximum mission area by storage limit. |
| FastAPI endpoints not in solution draft | **Functional**: Endpoints only in security_analysis.md. No request/response schemas. No SSE event format. No object localization endpoint. | **Consolidate API spec in solution**: define all endpoints, SSE event schema, object localization endpoint. Reference security_analysis.md for auth. |
| cuVSLAM configuration missing (calibration, IMU params, mode) | **Functional**: No camera calibration procedure, no IMU noise parameters, no T_imu_rig extrinsic, no mode selection (Mono vs Inertial). | **Define cuVSLAM configuration**: use Inertial mode, specify required calibration data (camera intrinsics, distortion, IMU noise params from datasheet, T_imu_rig from physical measurement), define calibration procedure. |
| tech_stack.md inconsistent with draft05 | **Functional**: tech_stack.md says 3fps (should be 0.7fps), LiteSAM at 480px (should be 1280px), missing EfficientLoFTR. | **Flag for update**: tech_stack.md must be synchronized with draft05 corrections. Not addressed in this draft — separate task. |
## Overall Maturity Assessment
| Category | Maturity (1-5) | Assessment |
| ----------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| Hardware & Platform Selection | 3.5 | UAV airframe, cameras, Jetson, batteries — well-researched with specs, weight budget, endurance calculations. Ready for procurement. |
| Core Algorithm Selection | 3.0 | cuVSLAM, LiteSAM/XFeat, ESKF — components selected with comparison tables, fallback chains, decision trees. Day-one benchmarks defined. |
| AI Inference Runtime | 3.5 | TRT Engine migration thoroughly analyzed. Conversion workflows, memory savings, performance estimates. Code wrapper provided. |
| Sensor Fusion (ESKF) | 1.5 | Mentioned but not specified. No implementable detail. Blockerfor coding. |
| System Integration | 1.5 | GPS_INPUT, coordinate transforms, inter-component data flow — all under-specified. |
| Edge Cases & Resilience | 1.0 | Disconnected segments, reboot recovery, re-localization — acknowledged but no algorithms. |
| Operational Readiness | 0.5 | No pre-flight procedures, no in-flight monitoring, no failure response. |
| Security | 3.0 | Comprehensive threat model, OP-TEE analysis, LUKS, secure boot. Well-researched. |
| **Overall TRL** | **~2.5** | **Technology concept formulated + some component validation. Not implementation-ready.** |
The solution is at approximately **TRL 3** (proof of concept) for hardware/algorithm selection and **TRL 1-2** (basic concept) for system integration, ESKF, and operational procedures.
# Solution
## Product Solution Description
A real-time GPS-denied visual navigation system for fixed-wing UAVs, running on a Jetson Orin Nano Super (8GB). All AI model inference uses native TensorRT Engine files. The system replaces the GPS module by sending MAVLink GPS_INPUT messages via pymavlink over UART at 5-10Hz.
Build an onboard GPS-denied localization service that runs on the Jetson companion computer, uses the fixed downward navigation camera and flight-controller inertial telemetry, and emits ArduPilot `GPS_INPUT` estimates with calibrated covariance and source labels.
Position is determined by fusing: (1) CUDA-accelerated visual odometry (cuVSLAM in Inertial mode) from ADTI 20L V1 at 0.7 fps sustained, (2) absolute position corrections from satellite image matching (LiteSAM or XFeat — TRT Engine FP16) using keyframes from the same ADTI image stream, and (3) IMU data from the flight controller via ESKF. Viewpro A40 Pro is reserved for AI object detection only.
The ESKF is the central state estimator with 15-state error vector. It fuses:
- **IMU prediction** at 5-10Hz (high-frequency pose propagation)
- **cuVSLAM VO measurement** at 0.7Hz (relative pose correction)
- **Satellite matching measurement** at ~0.07-0.14Hz (absolute position correction)
GPS_INPUT messages carry position, velocity, and accuracy derived from the ESKF state and covariance.
**Hard constraint**: ADTI 20L V1 shoots at 0.7 fps sustained (1430ms interval). Full VO+ESKF pipeline within 400ms per frame. Satellite matching async on keyframes (every 5-10 camera frames). GPS_INPUT at 5-10Hz (ESKF IMU prediction fills gaps between camera frames).
The production architecture is a trigger-based hybrid estimator:
```text
Nav camera + FC telemetry
|
v
Image quality + calibration + orthorectification
|
+--> Hot path: OpenCV geometry + BASALT VIO --> safety/anchor wrapper --> GPS_INPUT + QGC + FDR
|
+--> Reference path: OpenVINS replay benchmark for VIO drift/covariance tests; Kimera backup replay
|
+--> Trigger path: DINOv2-VLAD query --> CPU FAISS top-K --> ALIKED/DISK+LightGlue --> OpenCV RANSAC --> safety/anchor wrapper
|
+--> Tile path: new COG tile + quality/provenance sidecar --> manifest update --> post-flight Satellite Service sync
```
┌─────────────────────────────────────────────────────────────────────┐
│ OFFLINE (Before Flight) │
│ 1. Satellite Tiles → Download & Validate → Pre-resize → Store │
│ (Google Maps) (≥0.5m/px, <2yr) (matcher res) (GeoHash)│
│ 2. TRT Engine Build (one-time per model version): │
│ PyTorch model → reparameterize → ONNX export → trtexec --fp16 │
│ Output: litesam.engine, xfeat.engine │
│ 3. Camera + IMU calibration (one-time per hardware unit) │
│ 4. Copy tiles + engines + calibration to Jetson storage │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ ONLINE (During Flight) │
│ │
│ STARTUP: │
│ 1. pymavlink → read GLOBAL_POSITION_INT → init ESKF state │
│ 2. Load TRT engines + allocate GPU buffers │
│ 3. Load camera calibration + IMU calibration │
│ 4. Start cuVSLAM (Inertial mode) with ADTI 20L V1 │
│ 5. Preload satellite tiles ±2km into RAM │
│ 6. First satellite match → validate initial position │
│ 7. Begin GPS_INPUT output loop at 5-10Hz │
│ │
│ EVERY CAMERA FRAME (0.7fps from ADTI 20L V1): │
│ ┌──────────────────────────────────────┐ │
│ │ ADTI 20L V1 → Downsample (CUDA) │ │
│ │ → cuVSLAM VO+IMU (~9ms) │ ← CUDA Stream A │
│ │ → ESKF VO measurement │ │
│ └──────────────────────────────────────┘ │
│ │
│ 5-10Hz CONTINUOUS (IMU-driven between camera frames): │
│ ┌──────────────────────────────────────┐ │
│ │ IMU data → ESKF prediction │ │
│ │ ESKF state → GPS_INPUT fields │ │
│ │ GPS_INPUT → Flight Controller (UART) │ │
│ └──────────────────────────────────────┘ │
│ │
│ KEYFRAMES (every 5-10 camera frames, async): │
│ ┌──────────────────────────────────────┐ │
│ │ Camera frame → GSD downsample │ │
│ │ Select satellite tile (ESKF pos±3σ) │ │
│ │ TRT inference (Stream B): LiteSAM/ │ │
│ │ XFeat → correspondences │ │
│ │ RANSAC → homography → WGS84 position │ │
│ │ ESKF satellite measurement update │──→ Position correction │
│ └──────────────────────────────────────┘ │
│ │
│ TRACKING LOSS (cuVSLAM fails — sharp turn / featureless): │
│ ┌──────────────────────────────────────┐ │
│ │ ESKF → IMU-only prediction (growing │ │
│ │ uncertainty) │ │
│ │ Satellite match on EVERY frame │ │
│ │ On match success → ESKF reset + │ │
│ │ cuVSLAM restart │ │
│ │ 3 consecutive failures → operator │ │
│ │ re-localization request │ │
│ └──────────────────────────────────────┘ │
│ │
│ TELEMETRY (1Hz): │
│ ┌──────────────────────────────────────┐ │
│ │ NAMED_VALUE_FLOAT: confidence, drift │──→ Ground Station │
│ └──────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
Heavy local retrieval and local matching are not steady-state per-frame dependencies. They run on cold start, VO failure, sharp turns, disconnected segments, covariance growth, stale-anchor age, or operator-assisted relocalization, using only preloaded cache/index data during flight.
## Architecture
### Component: ESKF Sensor Fusion (NEW — previously unspecified)
### Camera Ingest, Calibration, And Geometry
**Error-State Kalman Filter** fusing IMU, visual odometry, and satellite matching.
| Solution | Tools | Pinned Mode/Config | Fit |
|----------|-------|--------------------|-----|
| OpenCV geometry utility layer | OpenCV 4.x | Calibration, undistortion, homography, RANSAC/USAC, MRE measurement | Selected. Mature, permissive, exact utility fit; not a full estimator. |
**Nominal state vector** (propagated by IMU):
### VO / IMU Propagation And Estimator
| Solution | Tools | Pinned Mode/Config | Fit |
|----------|-------|--------------------|-----|
| BASALT + safety/anchor wrapper | BASALT, OpenCV, custom wrapper | BASALT consumes calibrated nav-camera frames + FC IMU; wrapper fuses satellite anchors, calibrates uncertainty, emits source labels and `GPS_INPUT` fields | Selected. Best production VIO candidate found: permissive license, strong benchmark evidence, avoids custom VIO from scratch. |
| OpenVINS | OpenVINS | Monocular camera + IMU EKF/MSCKF reference runs with covariance extraction | Reference only. Strong VIO and covariance baseline, but GPLv3 and generic VIO ownership make it unsuitable as default shipped dependency. |
| Kimera-VIO | Kimera-VIO | Mono/stereo camera + IMU VIO/SLAM backup replay | Backup candidate. BSD-friendly but heavier/stereo-oriented; mono-inertial path has documented caveats. |
| ORB-SLAM3 | ORB-SLAM3 | Monocular-inertial SLAM | Rejected for production. GPLv3 and heavier SLAM/map lifecycle. |
| State | Symbol | Size | Description |
| ---------- | ------ | ---- | ------------------------------------------------ |
| Position | p | 3 | NED position relative to mission origin (meters) |
| Velocity | v | 3 | NED velocity (m/s) |
| Attitude | q | 4 | Unit quaternion (body-to-NED rotation) |
| Accel bias | b_a | 3 | Accelerometer bias (m/s²) |
| Gyro bias | b_g | 3 | Gyroscope bias (rad/s) |
BASALT does not replace the project-owned safety logic. The wrapper remains responsible for satellite anchor acceptance, confidence calibration, source labels, blackout/spoofing modes, tile-write eligibility, and MAVLink `GPS_INPUT` semantics.
### Satellite Service And Anchor Verification
**Error-state vector** (estimated by ESKF): δx = [δp, δv, δθ, δb_a, δb_g]ᵀ ∈ ℝ¹⁵
where δθ ∈ so(3) is the 3D rotation error.
| Solution | Tools | Pinned Mode/Config | Fit |
|----------|-------|--------------------|-----|
| DINOv2-VLAD + CPU FAISS + ALIKED/DISK+LightGlue | DINOv2/AnyLoc-style descriptors, FAISS CPU, LightGlue, OpenCV RANSAC | Offline VPR chunk descriptors; conditional query descriptor; CPU FAISS top-K; learned local match on bounded candidates; TensorRT only after fidelity check | Selected with runtime/fidelity gates. |
| SuperPoint+LightGlue | SuperPoint, LightGlue | Same matcher with SuperPoint features | License-gated benchmark/fallback only. |
| Classical SIFT/ORB | OpenCV | Handcrafted features + homography | Regression/fallback baseline. |
**Prediction step** (IMU at 5-10Hz from flight controller):
The Satellite Service component imports mission cache/index packages before flight, uploads generated-tile packages after landing, and serves local VPR queries during flight. The VPR index is built over ground-footprint-sized chunks with overlap and a multi-scale descriptor set. VPR is invoked only on relocalization triggers or covariance/anchor-age growth; normal flight uses BASALT VIO plus wrapper propagation. No satellite-provider or Satellite Service network calls are allowed mid-flight.
- Input: accelerometer a_m, gyroscope ω_m, dt
- Propagate nominal state: p += v·dt, v += (R(q)·(a_m - b_a) - g)·dt, q ⊗= Exp(ω_m - b_g)·dt
- Propagate error covariance: P = F·P·Fᵀ + Q
- F is the 15×15 error-state transition matrix (standard ESKF formulation)
- Q: process noise diagonal, initial values from IMU datasheet noise densities
### Tile Manager
**VO measurement update** (0.7Hz from cuVSLAM):
| Solution | Tools | Pinned Mode/Config | Fit |
|----------|-------|--------------------|-----|
| COG tile objects + PostgreSQL/PostGIS manifest + signed JSON sidecars | GDAL COG, PostgreSQL/PostGIS, signed JSON sidecars, FAISS index files | Service tiles and generated tiles are write-new COG objects; active version selected by PostGIS-backed manifest | Selected. Fits geospatial raster access, provenance, spatial/freshness queries, and write-new tile lifecycle. |
| PMTiles | PMTiles | Read-only archive snapshot | Rejected for live cache because in-flight tile generation needs mutable write-new objects. |
- cuVSLAM outputs relative pose: ΔR, Δt (camera frame)
- Transform to NED: Δp_ned = R_body_ned · T_cam_body · Δt
- Innovation: z = Δp_ned_measured - Δp_ned_predicted
- Observation matrix H_vo maps error state to relative position change
- R_vo: measurement noise, initial ~0.1-0.5m (from cuVSLAM precision at 600m+ altitude)
- Kalman update: K = P·Hᵀ·(H·P·Hᵀ + R)⁻¹, δx = K·z, P = (I - K·H)·P
Service-source tiles and generated tiles carry CRS, capture date, source, m/px, freshness, quality score, sidecar hashes, and descriptor references. The Tile Manager also orthorectifies eligible nadir frames into generated COG tiles. Stale tiles are rejected or down-confidence weighted.
**Satellite measurement update** (0.07-0.14Hz, async):
### MAVLink Integration
- Satellite matching outputs absolute position: lat_sat, lon_sat in WGS84
- Convert to NED relative to mission origin
- Innovation: z = p_satellite - p_predicted
- H_sat = [I₃, 0, 0, 0, 0] (directly observes position)
- R_sat: measurement noise, from matching confidence (~5-20m based on RANSAC inlier ratio)
- Provides absolute position correction — bounds drift accumulation
| Solution | Tools | Pinned Mode/Config | Fit |
|----------|-------|--------------------|-----|
| MAVSDK telemetry + pymavlink `GPS_INPUT` | MAVSDK, pymavlink | MAVSDK subscriptions; pymavlink emits `GPS_INPUT`; v1 emits GPS_INPUT only; Plane SITL validates `GPS1_TYPE=14`, velocity source params, ignore flags, fix types, accuracy fields | Selected. Exact output control with good telemetry ergonomics. |
**Scale observability**:
The system emits per-frame estimates locally and downsampled status to QGroundControl. `GPS_INPUT.horiz_accuracy` must not under-report the calibrated 95% covariance semi-major axis.
- Monocular cuVSLAM has scale ambiguity during constant-velocity flight
- Scale is constrained by: (1) satellite matching absolute positions (primary), (2) known flight altitude from barometer + predefined mission altitude, (3) IMU accelerometer during maneuvers
- During long straight segments without satellite correction, scale drift is possible. Satellite corrections every ~7-14s re-anchor scale.
### Security And Safety Controls
**Tuning approach**: Start with IMU datasheet noise values for Q. Start with conservative R values (high measurement noise). Tune on flight test data by comparing ESKF output to known GPS ground truth.
| Solution | Tools | Pinned Mode/Config | Fit |
|----------|-------|--------------------|-----|
| Consistency-gated anchor acceptance | Safety/anchor wrapper, cache manifest verification | Anchor accepted only if freshness, provenance, RANSAC, covariance, Mahalanobis, and temporal consistency pass | Selected. Prevents confident false fixes. |
| FDR audit trail | PostgreSQL event index + CBOR payload segments + hashes | Logs estimates, inputs, emitted GPS_INPUT, health, tile writes, anchor decisions | Selected. Supports incident analysis, indexed queries, and cache-poisoning audits. |
## Runtime Modes
| Solution | Tools | Advantages | Limitations | Performance | Fit |
| -------------------------- | --------------- | ------------------------------------------------------------- | -------------------------------------- | ------------- | ----------- |
| Custom ESKF (Python/NumPy) | NumPy, SciPy | Full control, minimal dependencies, well-understood algorithm | Implementation effort, tuning required | <1ms per step | ✅ Selected |
| FilterPy ESKF | FilterPy v1.4.5 | Reference implementation, less code | Less flexible for multi-rate fusion | <1ms per step | ⚠️ Fallback |
### Component: Coordinate System & Transformations (NEW — previously undefined)
**Reference frames**:
- **Camera frame (C)**: origin at camera optical center, Z forward, X right, Y down (OpenCV convention)
- **Body frame (B)**: origin at UAV CG, X forward (nose), Y right (starboard), Z down
- **NED frame (N)**: North-East-Down, origin at mission start point
- **WGS84**: latitude, longitude, altitude (output format)
**Transformation chain**:
1. **Pixel → Camera ray**: p_cam = K⁻¹ · [u, v, 1]ᵀ where K = camera intrinsic matrix (ADTI 20L V1: fx, fy from 16mm lens + APS-C sensor)
2. **Camera → Body**: p_body = T_cam_body · p_cam where T_cam_body is the fixed mounting rotation (camera points nadir: 90° pitch rotation from body X-forward to camera Z-down)
3. **Body → NED**: p_ned = R_body_ned(q) · p_body where q is the ESKF quaternion attitude estimate
4. **NED → WGS84**: lat = lat_origin + p_north / R_earth, lon = lon_origin + p_east / (R_earth · cos(lat_origin)) where (lat_origin, lon_origin) is the mission start GPS position
**Camera intrinsic matrix K** (ADTI 20L V1 + 16mm lens):
- Sensor: 23.2 × 15.4 mm, Resolution: 5456 × 3632
- fx = fy = focal_mm × width_px / sensor_width_mm = 16 × 5456 / 23.2 = 3763 pixels
- cx = 2728, cy = 1816 (sensor center)
- Distortion: Brown model (k1, k2, p1, p2 from calibration)
**T_cam_body** (camera mount):
- Navigation camera is fixed, pointing nadir (downward), not autostabilized
- R_cam_body = R_x(180°) · R_z(0°) (camera Z-axis aligned with body -Z, camera X with body X)
- Translation: offset from CG to camera mount (measured during assembly, typically <0.3m)
**Satellite match → WGS84**:
- Feature correspondences between camera frame and geo-referenced satellite tile
- Homography H maps camera pixels to satellite tile pixels
- Satellite tile pixel → WGS84 via tile's known georeference (zoom level + tile x,y → lat,lon)
- Camera center projects to satellite pixel (cx_sat, cy_sat) via H
- Convert (cx_sat, cy_sat) to WGS84 using tile georeference
### Component: GPS_INPUT Message Population (NEW — previously undefined)
| GPS_INPUT Field | Source | Computation |
| ----------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| lat, lon | ESKF position (NED) | NED → WGS84 conversion using mission origin |
| alt | ESKF position (Down) + mission origin altitude | alt = alt_origin - p_down |
| vn, ve, vd | ESKF velocity state | Direct from ESKF v[0], v[1], v[2] |
| fix_type | Confidence tier | 3 (3D fix) when satellite-anchored (last match <30s). 2 (2D) when VO-only. 0 (no fix) when IMU-only >5s |
| hdop | ESKF horizontal covariance | hdop = sqrt(P[0,0] + P[1,1]) / 5.0 (approximate CEP→HDOP mapping) |
| vdop | ESKF vertical covariance | vdop = sqrt(P[2,2]) / 5.0 |
| horiz_accuracy | ESKF horizontal covariance | horiz_accuracy = sqrt(P[0,0] + P[1,1]) meters |
| vert_accuracy | ESKF vertical covariance | vert_accuracy = sqrt(P[2,2]) meters |
| speed_accuracy | ESKF velocity covariance | speed_accuracy = sqrt(P[3,3] + P[4,4]) m/s |
| time_week, time_week_ms | System time | Convert Unix time to GPS epoch (GPS epoch = 1980-01-06, subtract leap seconds) |
| satellites_visible | Constant | 10 (synthetic — prevents satellite-count failsafes in ArduPilot) |
| gps_id | Constant | 0 |
| ignore_flags | Constant | 0 (provide all fields) |
**Confidence tiers** mapping to GPS_INPUT:
| Tier | Condition | fix_type | horiz_accuracy | Rationale |
| ------ | ------------------------------------------------- | ---------- | ------------------------------- | -------------------------------------- |
| HIGH | Satellite match <30s ago, ESKF covariance < 400m² | 3 (3D fix) | From ESKF P (typically 5-20m) | Absolute position anchor recent |
| MEDIUM | cuVSLAM tracking OK, no recent satellite match | 3 (3D fix) | From ESKF P (typically 20-50m) | Relative tracking valid, drift growing |
| LOW | cuVSLAM lost, IMU-only | 2 (2D fix) | From ESKF P (50-200m+, growing) | Only IMU dead reckoning, rapid drift |
| FAILED | 3+ consecutive total failures | 0 (no fix) | 999.0 | System cannot determine position |
### Component: Disconnected Route Segment Handling (NEW — previously undefined)
**Trigger**: cuVSLAM reports tracking_lost OR tracking confidence drops below threshold
**Algorithm**:
```
STATE: TRACKING_NORMAL
cuVSLAM provides relative pose
ESKF VO measurement updates at 0.7Hz
Satellite matching on keyframes (every 5-10 frames)
STATE: TRACKING_LOST (enter when cuVSLAM reports loss)
1. ESKF continues with IMU-only prediction (no VO updates)
→ uncertainty grows rapidly (~1-5 m/s drift with consumer IMU)
2. Switch satellite matching to EVERY frame (not just keyframes)
→ maximize chances of getting absolute correction
3. For each camera frame:
a. Attempt satellite match using ESKF predicted position ± 3σ for tile selection
b. If match succeeds (RANSAC inlier ratio > 30%):
→ ESKF measurement update with satellite position
→ Restart cuVSLAM with current frame as new origin
→ Transition to TRACKING_NORMAL
→ Reset failure counter
c. If match fails:
→ Increment failure_counter
→ Continue IMU-only ESKF prediction
4. If failure_counter >= 3:
→ Send re-localization request to ground station
→ GPS_INPUT fix_type = 0 (no fix), horiz_accuracy = 999.0
→ Continue attempting satellite matching on each frame
5. If operator sends re-localization hint (approximate lat,lon):
→ Use as ESKF measurement with high covariance (~500m)
→ Attempt satellite match in that area
→ On success: transition to TRACKING_NORMAL
STATE: SEGMENT_DISCONNECT
After re-localization following tracking loss:
→ New cuVSLAM track is independent of previous track
→ ESKF maintains global NED position continuity via satellite anchor
→ No need to "connect" segments at the cuVSLAM level
→ ESKF already handles this: satellite corrections keep global position consistent
```
### Component: Satellite Image Matching Pipeline (UPDATED — added GSD + tile selection details)
**GSD normalization**:
- Camera GSD at 600m: ~15.9 cm/pixel (ADTI 20L V1 + 16mm)
- Satellite tile GSD at zoom 18: ~0.6 m/pixel
- Scale ratio: ~3.8:1
- Downsample camera image to satellite GSD before matching: resize from 5456×3632 to ~1440×960 (matching zoom 18 GSD)
- This is close to LiteSAM's 1280px input — use 1280px with minor GSD mismatch acceptable for matching
**Tile selection**:
- Input: ESKF position estimate (lat, lon) + horizontal covariance σ_h
- Search radius: max(3·σ_h, 500m) — at least 500m to handle initial uncertainty
- Compute geohash for center position → load tiles covering the search area
- Assemble tile mosaic if needed (typically 2×2 to 4×4 tiles for adequate coverage)
- If ESKF uncertainty > 2km: tile selection unreliable, fall back to wider search or request operator input
**Tile storage calculation** (zoom 18 — 0.6 m/pixel):
- Each 256×256 tile covers ~153m × 153m
- Flight path 200km with ±2km buffer: area ≈ 200km × 4km = 800 km²
- Tiles needed: 800,000,000 / (153 × 153) ≈ 34,200 tiles
- Storage: ~10-15KB per JPEG tile → ~340-510 MB
- With zoom 19 overlap tiles for higher precision: ×4 = ~1.4-2.0 GB
- Recommended: zoom 18 primary + zoom 19 for ±500m along flight path → ~500-800 MB total
| Solution | Tools | Advantages | Limitations | Performance (est. Orin Nano Super TRT FP16) | Params | Fit |
| -------------------------------------- | ------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------ | ------------------------------------------- | ------ | ------------------------------- |
| LiteSAM (opt) TRT Engine FP16 @ 1280px | trtexec + tensorrt Python | Best satellite-aerial accuracy (RMSE@30=17.86m UAV-VisLoc), 6.31M params | MinGRU TRT export needs verification (LOW-MEDIUM risk) | Est. ~165-330ms | 6.31M | ✅ Primary |
| EfficientLoFTR TRT Engine FP16 | trtexec + tensorrt Python | Proven TRT path (Coarse_LoFTR_TRT). Semi-dense. CVPR 2024. | 2.4x more params than LiteSAM. | Est. ~200-400ms | 15.05M | ✅ Fallback if LiteSAM TRT fails |
| XFeat TRT Engine FP16 | trtexec + tensorrt Python | Fastest. Proven TRT implementation. | General-purpose, not designed for cross-view gap. | Est. ~50-100ms | <5M | ✅ Speed fallback |
### Component: cuVSLAM Configuration (NEW — previously undefined)
**Mode**: Inertial (mono camera + IMU)
**Camera configuration** (ADTI 20L V1 + 16mm lens):
- Model: Brown distortion
- fx = fy = 3763 px (16mm on 23.2mm sensor at 5456px width)
- cx = 2728 px, cy = 1816 px
- Distortion coefficients: from calibration (k1, k2, p1, p2)
- Border: 50px (ignore lens edge distortion)
**IMU configuration** (Pixhawk 6x IMU — ICM-42688-P):
- Gyroscope noise density: 3.0 × 10⁻³ °/s/√Hz
- Gyroscope random walk: 5.0 × 10⁻⁵ °/s²/√Hz
- Accelerometer noise density: 70 µg/√Hz
- Accelerometer random walk: ~2.0 × 10⁻³ m/s³/√Hz
- IMU frequency: 200 Hz (from flight controller via MAVLink)
- T_imu_rig: measured transformation from Pixhawk IMU to camera center (translation + rotation)
**cuVSLAM settings**:
- OdometryMode: INERTIAL
- MulticameraMode: PRECISION (favor accuracy over speed — we have 1430ms budget)
- Input resolution: downsample to 1280×852 (or 720p) for processing speed
- async_bundle_adjustment: True
**Initialization**:
- cuVSLAM initializes automatically when it receives the first camera frame + IMU data
- First few frames used for feature initialization and scale estimation
- First satellite match validates and corrects the initial position
**Calibration procedure** (one-time per hardware unit):
1. Camera intrinsics: checkerboard calibration with OpenCV (or use manufacturer data if available)
2. Camera-IMU extrinsic (T_imu_rig): Kalibr tool with checkerboard + IMU data
3. IMU noise parameters: Allan variance analysis or use datasheet values
4. Store calibration files on Jetson storage
### Component: AI Model Inference Runtime (UNCHANGED)
Native TRT Engine — optimal performance and memory on fixed NVIDIA hardware. See draft05 for full comparison table and conversion workflow.
### Component: Visual Odometry (UNCHANGED)
cuVSLAM in Inertial mode, fed by ADTI 20L V1 at 0.7 fps sustained. See draft05 for feasibility analysis at 0.7fps.
### Component: Flight Controller Integration (UPDATED — added GPS_INPUT field spec)
pymavlink over UART at 5-10Hz. GPS_INPUT field population defined above.
ArduPilot configuration:
- GPS1_TYPE = 14 (MAVLink)
- GPS_RATE = 5 (minimum, matching our 5-10Hz output)
- EK3_SRC1_POSXY = 1 (GPS), EK3_SRC1_VELXY = 1 (GPS) — EKF uses GPS_INPUT as position/velocity source
### Component: Object Localization (NEW — previously undefined)
**Input**: pixel coordinates (u, v) in Viewpro A40 Pro image, current gimbal angles (pan_deg, tilt_deg), zoom factor, UAV position from GPS-denied system, UAV altitude
**Process**:
1. Pixel → camera ray: ray_cam = K_viewpro⁻¹(zoom) · [u, v, 1]ᵀ
2. Camera → gimbal frame: ray_gimbal = R_gimbal(pan, tilt) · ray_cam
3. Gimbal → body: ray_body = T_gimbal_body · ray_gimbal
4. Body → NED: ray_ned = R_body_ned(q) · ray_body
5. Ray-ground intersection: assuming flat terrain at UAV altitude h: t = -h / ray_ned[2], p_ground_ned = p_uav_ned + t · ray_ned
6. NED → WGS84: convert to lat, lon
**Output**: { lat, lon, accuracy_m, confidence }
- accuracy_m propagated from: UAV position accuracy (from ESKF) + gimbal angle uncertainty + altitude uncertainty
**API endpoint**: POST /objects/locate
- Request: { pixel_x, pixel_y, gimbal_pan_deg, gimbal_tilt_deg, zoom_factor }
- Response: { lat, lon, alt, accuracy_m, confidence, uav_position: {lat, lon, alt}, timestamp }
### Component: Startup, Handoff & Failsafe (UPDATED — added handoff + reboot + re-localization)
**GPS-denied handoff protocol**:
- GPS-denied system runs continuously from companion computer boot
- Reads initial position from FC (GLOBAL_POSITION_INT) — this may be real GPS or last known
- First satellite match validates the initial position
- FC receives both real GPS (if available) and GPS_INPUT; FC EKF selects best source based on accuracy
- No explicit "switch" — the GPS-denied system is a secondary GPS source
**Startup sequence** (expanded from draft05):
1. Boot Jetson → start GPS-Denied service (systemd)
2. Connect to flight controller via pymavlink on UART
3. Wait for heartbeat
4. Initialize PyCUDA context
5. Load TRT engines: litesam.engine + xfeat.engine (~1-3s each)
6. Allocate GPU I/O buffers
7. Create CUDA streams: Stream A (cuVSLAM), Stream B (satellite matching)
8. Load camera calibration + IMU calibration files
9. Read GLOBAL_POSITION_INT → set mission origin (NED reference point) → init ESKF
10. Start cuVSLAM (Inertial mode) with ADTI 20L V1 camera stream
11. Preload satellite tiles within ±2km into RAM
12. Trigger first satellite match → validate initial position
13. Begin GPS_INPUT output loop at 5-10Hz
14. System ready
**Mid-flight reboot recovery**:
1. Jetson boots (~30-60s)
2. GPS-Denied service starts, connects to FC
3. Read GLOBAL_POSITION_INT (FC's current IMU-extrapolated position)
4. Init ESKF with this position + HIGH uncertainty covariance (σ = 200m)
5. Load TRT engines (~2-6s total)
6. Start cuVSLAM (fresh, no prior map)
7. Immediate satellite matching on first camera frame
8. On satellite match success: ESKF corrected, uncertainty drops
9. Estimated total recovery: ~35-70s
10. During recovery: FC uses IMU-only dead reckoning (at 70 km/h: ~700-1400m uncontrolled drift)
11. **Known limitation**: recovery time is dominated by Jetson boot time
**3-consecutive-failure re-localization**:
- Trigger: VO lost + satellite match failed × 3 consecutive camera frames
- Action: send re-localization request via MAVLink STATUSTEXT or custom message
- Message content: "RELOC_REQ: last_lat={lat} last_lon={lon} uncertainty={σ}m"
- Operator response: MAVLink COMMAND_LONG with approximate lat/lon
- System: use operator position as ESKF measurement with R = diag(500², 500², 100²) meters²
- System continues satellite matching with updated search area
- While waiting: GPS_INPUT fix_type=0, IMU-only ESKF prediction continues
### Component: Ground Station Telemetry (UPDATED — added re-localization)
MAVLink messages to ground station:
| Message | Rate | Content |
| ----------------------------- | -------- | --------------------------------------------------- |
| NAMED_VALUE_FLOAT "gps_conf" | 1Hz | Confidence score (0.0-1.0) |
| NAMED_VALUE_FLOAT "gps_drift" | 1Hz | Estimated drift from last satellite anchor (meters) |
| NAMED_VALUE_FLOAT "gps_hacc" | 1Hz | Horizontal accuracy (meters, from ESKF) |
| STATUSTEXT | On event | "RELOC_REQ: ..." for re-localization request |
| STATUSTEXT | On event | Tracking loss / recovery notifications |
### Component: Thermal Management (UNCHANGED)
Same adaptive pipeline from draft05. Active cooling required at 25W. Throttling at 80°C SoC junction.
### Component: API & Inter-System Communication (NEW — consolidated)
FastAPI (Uvicorn) running locally on Jetson for inter-process communication with other onboard systems.
| Endpoint | Method | Purpose | Auth |
| --------------------- | --------- | -------------------------------------- | ---- |
| /sessions | POST | Start GPS-denied session | JWT |
| /sessions/{id}/stream | GET (SSE) | Real-time position + confidence stream | JWT |
| /sessions/{id}/anchor | POST | Operator re-localization hint | JWT |
| /sessions/{id} | DELETE | End session | JWT |
| /objects/locate | POST | Object GPS from pixel coordinates | JWT |
| /health | GET | System health + memory + thermal | None |
**SSE event schema** (1Hz):
```json
{
"type": "position",
"timestamp": "2026-03-17T12:00:00.000Z",
"lat": 48.123456,
"lon": 37.654321,
"alt": 600.0,
"accuracy_h": 15.2,
"accuracy_v": 8.1,
"confidence": "HIGH",
"drift_from_anchor": 12.5,
"vo_status": "tracking",
"last_satellite_match_age_s": 8.3
}
```
## UAV Platform
Unchanged from draft05. See draft05 for: airframe configuration (3.5m S-2 composite, 12.5kg AUW), flight performance (3.4h endurance at 50 km/h), camera specifications (ADTI 20L V1 + 16mm, Viewpro A40 Pro), ground coverage calculations.
## Speed Optimization Techniques
Unchanged from draft05. Key points: cuVSLAM ~9ms/frame, native TRT Engine (no ONNX RT), dual CUDA streams, 5-10Hz GPS_INPUT from ESKF IMU prediction.
## Processing Time Budget
Unchanged from draft05. VO frame: ~17-22ms. Satellite matching: ≤210ms async. Well within 1430ms frame interval.
## Memory Budget (Jetson Orin Nano Super, 8GB shared)
| Component | Memory | Notes |
| ------------------------- | -------------- | ------------------------------------------- |
| OS + runtime | ~1.5GB | JetPack 6.2 + Python |
| cuVSLAM | ~200-500MB | CUDA library + map |
| LiteSAM TRT engine | ~50-80MB | If LiteSAM fails: EfficientLoFTR ~100-150MB |
| XFeat TRT engine | ~30-50MB | |
| Preloaded satellite tiles | ~200MB | ±2km of flight plan |
| pymavlink + MAVLink | ~20MB | |
| FastAPI (local IPC) | ~50MB | |
| ESKF + buffers | ~10MB | |
| **Total** | **~2.1-2.9GB** | **26-36% of 8GB** |
## Key Risks and Mitigations
| Risk | Likelihood | Impact | Mitigation |
| ------------------------------------------------------- | ---------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| LiteSAM MinGRU ops unsupported in TRT 10.3 | LOW-MEDIUM | LiteSAM TRT export fails | Day-one verification. Fallback: EfficientLoFTR TRT → XFeat TRT. |
| cuVSLAM fails on low-texture terrain at 0.7fps | HIGH | Frequent tracking loss | Satellite matching corrections bound drift. Re-localization pipeline handles tracking loss. IMU bridges short gaps. |
| Google Maps satellite quality in conflict zone | HIGH | Satellite matching fails, outdated imagery | Pre-flight tile validation. Consider alternative providers (Bing, Mapbox). Robust to seasonal appearance changes via feature-based matching. |
| ESKF scale drift during long constant-velocity segments | MEDIUM | Position error exceeds 100m between satellite anchors | Satellite corrections every 7-14s re-anchor. Altitude constraint from barometer. Monitor drift rate — if >50m between corrections, increase satellite matching frequency. |
| Monocular scale ambiguity | MEDIUM | Metric scale lost during constant-velocity flight | Satellite absolute corrections provide scale. Known altitude constrains vertical scale. IMU acceleration during turns provides observability. |
| AUW exceeds AT4125 recommended range | MEDIUM | Reduced endurance, motor thermal stress | 12.5 kg vs 8-10 kg recommended. Monitor motor temps. Weight optimization. |
| ADTI mechanical shutter lifespan | MEDIUM | Replacement needed periodically | ~8,800 actuations/flight at 0.7fps. Estimated 11-57 flights before replacement. Budget as consumable. |
| Mid-flight companion computer failure | LOW | ~35-70s position gap | Reboot recovery procedure defined. FC uses IMU dead reckoning during gap. Known limitation. |
| Thermal throttling on Jetson | MEDIUM | Satellite matching latency increases | Active cooling required. Monitor SoC temp. Throttling at 80°C. Our workload ~8-15W typical — well under 25W TDP. |
| Engine incompatibility after JetPack update | MEDIUM | Must rebuild engines | Include engine rebuild in update procedure. |
| TRT engine build OOM on 8GB | LOW | Cannot build on target | Models small (6.31M, <5M). Reduce --memPoolSize if needed. |
| Mode | Trigger | Behavior | `GPS_INPUT` / Telemetry |
|------|---------|----------|--------------------------|
| `satellite_anchored` | VPR + local match passes all gates | Wrapper absolute update; tile write eligible only if sigma gate passes | 3D fix, `horiz_accuracy` >= 95% covariance semi-major axis |
| `vo_extrapolated` | BASALT VIO healthy and anchor age/covariance within bounds | BASALT VIO + wrapper propagation; covariance grows | 3D/2D depending covariance threshold |
| `dead_reckoned` | visual blackout or no accepted anchor | IMU-only propagation, monotonic covariance growth | degraded fix type; QGC `VISUAL_BLACKOUT_IMU_ONLY` |
| failsafe/no-fix | covariance >500 m or blackout >30 s | stop pretending position is valid | `fix_type=0`, `horiz_accuracy=999.0`, QGC `VISUAL_BLACKOUT_FAILSAFE` |
## Testing Strategy
### Integration / Functional Tests
- **ESKF correctness**: Feed recorded IMU + synthetic VO/satellite data → verify output matches reference ESKF implementation
- **GPS_INPUT field validation**: Send GPS_INPUT to SITL ArduPilot → verify EKF accepts and uses the data correctly
- **Coordinate transform chain**: Known GPS → NED → pixel → back to GPS — verify round-trip error <0.1m
- **Disconnected segment handling**: Simulate tracking loss → verify satellite re-localization triggers → verify cuVSLAM restarts → verify ESKF position continuity
- **3-consecutive-failure**: Simulate VO + satellite failures → verify re-localization request sent → verify operator hint accepted
- **Object localization**: Known object at known GPS → verify computed GPS matches within camera accuracy
- **Mid-flight reboot**: Kill GPS-denied process → restart → verify recovery within expected time → verify position accuracy after recovery
- **TRT engine load test**: Verify engines load successfully on Jetson
- **TRT inference correctness**: Compare TRT output vs PyTorch reference (max L1 error < 0.01)
- **CUDA Stream pipelining**: Verify Stream B satellite matching does not block Stream A VO
- **ADTI sustained capture rate**: Verify 0.7fps sustained >30 min without buffer overflow
- **Confidence tier transitions**: Verify fix_type and accuracy change correctly across HIGH → MEDIUM → LOW → FAILED transitions
- BASALT replay: assert AC-2.1a and AC-2.2 VO MRE on overlapping frame pairs, completion rate, latency, and wrapper-calibrated covariance.
- OpenVINS reference replay: compare VIO drift, failure cases, and covariance against BASALT + wrapper.
- Kimera-VIO backup replay: keep a second permissive candidate benchmark in case BASALT fails project replay/runtime gates.
- Satellite anchor replay: assert AC-1.1/1.2, AC-2.2 cross-domain MRE, freshness rejection, and source labels.
- DINOv2 descriptor fidelity: compare PyTorch/ONNX/TensorRT embeddings and retrieval rankings before accepting optimized engines.
- FAISS CPU index tests: top-K recall, query latency, index size, save/load behavior on Jetson ARM64.
- LightGlue extractor matrix: ALIKED vs DISK vs SIFT/ORB vs SuperPoint benchmark; SuperPoint excluded from production unless legal approves.
- Tile Manager: orthorectify eligible nadir frames into write-new generated tiles, update manifest, verify active version and rollback.
- `GPS_INPUT` SITL: validate fix type, `horiz_accuracy`, velocity fields, ignore flags, `EK3_SRC1_*` parameters, QGC behavior.
- Security gates: stale tile, mismatched tile hash, low inlier ratio, impossible velocity jump, and spoofed GPS during blackout.
### Non-Functional Tests
- **End-to-end accuracy** (primary validation): Fly with real GPS recording → run GPS-denied system in parallel → compare estimated vs real positions → verify 80% within 50m, 60% within 20m
- **VO drift rate**: Measure cuVSLAM drift over 1km straight segment without satellite correction
- **Satellite matching accuracy**: Compare satellite-matched position vs real GPS at known locations
- **Processing time**: Verify end-to-end per-frame <400ms
- **Memory usage**: Monitor over 30-min session → verify <8GB, no leaks
- **Thermal**: Sustained 30-min run → verify no throttling
- **GPS_INPUT rate**: Verify consistent 5-10Hz delivery to FC
- **Tile storage**: Validate calculated storage matches actual for test mission area
- **MinGRU TRT compatibility** (day-one blocker): Clone LiteSAM → ONNX export → polygraphy → trtexec
- **Flight endurance**: Ground-test full system power draw against 267W estimate
- Jetson latency and memory: <400 ms p95, <8 GB shared memory, no 25 W thermal throttle.
- Cache budget: 400 km² imagery + manifests + descriptors fits budget or reports explicit split budget.
- FDR 8-hour load: <=64 GB, rollover logged, no silent payload loss.
- Monte Carlo false-position and cache-poisoning tests for AC-NEW-4 and AC-NEW-7.
- Cold boot: first valid `GPS_INPUT` <30 s p95 across 50 runs.
## References
- ArduPilot GPS_RATE parameter: [https://github.com/ArduPilot/ardupilot/pull/15980](https://github.com/ArduPilot/ardupilot/pull/15980)
- MAVLink GPS_INPUT message: [https://ardupilot.org/mavproxy/docs/modules/GPSInput.html](https://ardupilot.org/mavproxy/docs/modules/GPSInput.html)
- pymavlink GPS_INPUT example: [https://webperso.ensta.fr/lebars/Share/GPS_INPUT_pymavlink.py](https://webperso.ensta.fr/lebars/Share/GPS_INPUT_pymavlink.py)
- ESKF reference (fixed-wing UAV): [https://github.com/ludvigls/ESKF](https://github.com/ludvigls/ESKF)
- ROS ESKF multi-sensor: [https://github.com/EliaTarasov/ESKF](https://github.com/EliaTarasov/ESKF)
- Range-VIO scale observability: [https://arxiv.org/abs/2103.15215](https://arxiv.org/abs/2103.15215)
- NaviLoc trajectory-level localization: [https://www.mdpi.com/2504-446X/10/2/97](https://www.mdpi.com/2504-446X/10/2/97)
- SatLoc-Fusion hierarchical framework: [https://www.scilit.com/publications/e5cafaf875a49297a62b298a89d5572f](https://www.scilit.com/publications/e5cafaf875a49297a62b298a89d5572f)
- Auterion GPS-denied workflow: [https://docs.auterion.com/vehicle-operation/auterion-mission-control/useful-resources/operations/gps-denied-workflow](https://docs.auterion.com/vehicle-operation/auterion-mission-control/useful-resources/operations/gps-denied-workflow)
- PX4 GNSS-denied flight: [https://docs.px4.io/main/en/advanced_config/gnss_degraded_or_denied_flight.html](https://docs.px4.io/main/en/advanced_config/gnss_degraded_or_denied_flight.html)
- ArduPilot GPS_INPUT advanced usage: [https://discuss.ardupilot.org/t/advanced-usage-of-gps-type-mav-14/99406](https://discuss.ardupilot.org/t/advanced-usage-of-gps-type-mav-14/99406)
- Google Maps Ukraine imagery: [https://newsukraine.rbc.ua/news/google-maps-has-surprise-for-satellite-imagery-1727182380.html](https://newsukraine.rbc.ua/news/google-maps-has-surprise-for-satellite-imagery-1727182380.html)
- Jetson Orin Nano Super thermal: [https://edgeaistack.app/blog/jetson-orin-nano-power-consumption/](https://edgeaistack.app/blog/jetson-orin-nano-power-consumption/)
- GSD matching research: [https://www.kjrs.org/journal/view.html?pn=related&uid=756&vmd=Full](https://www.kjrs.org/journal/view.html?pn=related&uid=756&vmd=Full)
- VO+satellite matching pipeline: [https://polen.itu.edu.tr/items/1fe1e872-7cea-44d8-a8de-339e4587bee6](https://polen.itu.edu.tr/items/1fe1e872-7cea-44d8-a8de-339e4587bee6)
- PyCuVSLAM docs: [https://wiki.seeedstudio.com/pycuvslam_recomputer_robotics/](https://wiki.seeedstudio.com/pycuvslam_recomputer_robotics/)
- Pixhawk 6x IMU (ICM-42688-P) datasheet: [https://invensense.tdk.com/products/motion-tracking/6-axis/icm-42688-p/](https://invensense.tdk.com/products/motion-tracking/6-axis/icm-42688-p/)
- All references from solution_draft05.md
Detailed source registry: `_docs/00_research/01_source_registry.md`.
Key sources:
- BASALT repository: https://github.com/VladyslavUsenko/basalt
- HybVIO benchmark paper: https://arxiv.org/pdf/2106.11857
- OpenVINS docs: https://docs.openvins.com/index.html
- OpenVINS ATE/RTE comparison: https://github.com/rpng/open_vins/issues/402
- Kimera mono-inertial caveat: https://github.com/MIT-SPARK/Kimera-VIO/issues/254
- AnyLoc paper: https://arxiv.org/html/2308.00688
- Aerial VPR survey: https://arxiv.org/abs/2406.00885
- ALIKED-LightGlue-ONNX: https://github.com/ikeboo/ALIKED-LightGlue-ONNX
- DINOv2 TensorRT issue: https://github.com/NVIDIA/TensorRT/issues/4348
## Related Artifacts
- AC Assessment: `_docs/00_research/gps_denied_nav/00_ac_assessment.md`
- Completeness assessment research: `_docs/00_research/solution_completeness_assessment/`
- Previous research: `_docs/00_research/trt_engine_migration/`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md` (needs sync with draft05 corrections)
- Security analysis: `_docs/01_solution/security_analysis.md`
- Previous draft: `_docs/01_solution/solution_draft05.md`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md`
- Component fit matrix: `_docs/00_research/06_component_fit_matrix.md`
- Fact cards: `_docs/00_research/02_fact_cards.md`
_docs/01_solution/solution_draft01.md
+107 -241
View File
@@ -2,282 +2,148 @@
## Product Solution Description
A real-time GPS-denied visual navigation system for fixed-wing UAVs, running entirely on a Jetson Orin Nano Super (8GB). The system determines frame-center GPS coordinates by fusing three information sources: (1) CUDA-accelerated visual odometry (cuVSLAM), (2) absolute position corrections from satellite image matching, and (3) IMU-based motion prediction. Results stream to clients via REST API + SSE in real time.
Build an onboard GPS-denied localization service that runs on the companion Jetson, consumes the fixed nadir navigation camera plus flight-controller IMU/attitude/altitude, and emits ArduPilot-compatible `GPS_INPUT` estimates with honest covariance.
**Hard constraint**: Camera shoots at ~3fps (333-400ms interval). The full pipeline must complete within **400ms per frame**.
The solution is a hybrid estimator:
**Satellite matching strategy**: Benchmark LiteSAM on actual Orin Nano Super hardware as a day-one priority. If LiteSAM cannot achieve ≤400ms at 480px resolution, **abandon it entirely** and use XFeat semi-dense matching as the primary satellite matcher. Speed is non-negotiable.
**Core architectural principles**:
1. **cuVSLAM handles VO** — NVIDIA's CUDA-accelerated library achieves 90fps on Jetson Orin Nano, giving VO essentially "for free" (~11ms/frame).
2. **Keyframe-based satellite matching** — satellite matcher runs on keyframes only (every 3-10 frames), amortizing its cost. Non-keyframes rely on cuVSLAM VO + IMU.
3. **Every keyframe independently attempts satellite-based geo-localization** — this handles disconnected segments natively.
4. **Pipeline parallelism** — satellite matching for frame N overlaps with VO processing of frame N+1 via CUDA streams.
```
┌─────────────────────────────────────────────────────────────────┐
│ OFFLINE (Before Flight) │
│ Satellite Tiles → Download & Crop → Store as tile pairs │
│ (Google Maps) (per flight plan) (disk, GeoHash indexed) │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ ONLINE (During Flight) │
│ │
│ EVERY FRAME (400ms budget): │
│ ┌────────────────────────────────┐ │
│ │ Camera → Downsample (CUDA 2ms)│ │
│ │ → cuVSLAM VO+IMU (~11ms)│──→ ESKF Update → SSE Emit │
│ └────────────────────────────────┘ ↑ │
│ │ │
│ KEYFRAMES ONLY (every 3-10 frames): │ │
│ ┌────────────────────────────────────┐ │ │
│ │ Satellite match (async CUDA stream)│─────┘ │
│ │ LiteSAM or XFeat (see benchmark) │ │
│ │ (does NOT block VO output) │ │
│ └────────────────────────────────────┘ │
│ │
│ IMU: 100+Hz continuous → ESKF prediction │
└─────────────────────────────────────────────────────────────────┘
```text
Nav camera + FC telemetry
|
v
Image quality + calibration + orthorectification
|
+--> Steady state: VO/IMU propagation --> ESKF --> GPS_INPUT + QGC + FDR
|
+--> Re-anchor triggers: VPR top-K --> local match/RANSAC --> ESKF anchor update
|
+--> Tile generation: ortho tile + quality sidecar --> local cache --> post-flight Satellite Service upload
```
## Speed Optimization Techniques
The steady-state path is intentionally lightweight. Heavy global retrieval and cross-domain local matching run only on cold start, VO failure, sharp turns, disconnected segments, covariance growth, or stale-anchor age.
### 1. cuVSLAM for Visual Odometry (~11ms/frame)
NVIDIA's CUDA-accelerated VO library (v15.0.0, March 2026) achieves 90fps on Jetson Orin Nano. Supports monocular camera + IMU natively. Features: automatic IMU fallback when visual tracking fails, loop closure, Python and C++ APIs. This eliminates custom VO entirely.
## Existing / Competitor Solutions Analysis
### 2. Keyframe-Based Satellite Matching
Not every frame needs satellite matching. Strategy:
- cuVSLAM provides VO at every frame (high-rate, low-latency)
- Satellite matching triggers on **keyframes** selected by:
- Fixed interval: every 3-10 frames (~1-3.3s between satellite corrections)
- Confidence drop: when ESKF covariance exceeds threshold
- VO failure: when cuVSLAM reports tracking loss (sharp turn)
Recent fixed-wing GPS-denied research supports the same high-level mechanism: monocular visual odometry alone accumulates scale/drift error, while satellite-image comparison can periodically correct it. Aerial VPR surveys also show why the implementation cannot be naive: weather, season, scale mismatch, repetitive fields, tile overlap, and re-ranking runtime all matter.
### 3. Satellite Matcher Selection (Benchmark-Driven)
**Candidate A: LiteSAM (opt)** — Best accuracy for satellite-aerial matching (RMSE@30 = 17.86m on UAV-VisLoc). 6.31M params, MobileOne + TAIFormer + MinGRU. Benchmarked at 497ms on Jetson AGX Orin at 1184px. AGX Orin is 3-4x more powerful than Orin Nano Super (275 TOPS vs 67 TOPS, $2000+ vs $249).
Realistic Orin Nano Super estimates:
- At 1184px: ~1.5-2.0s (unusable)
- At 640px: ~500-800ms (borderline)
- At 480px: ~300-500ms (best case)
**Candidate B: XFeat semi-dense** — ~50-100ms on Orin Nano Super. Proven on Jetson. Not specifically designed for cross-view satellite-aerial, but fast and reliable.
**Decision rule**: Benchmark LiteSAM TensorRT FP16 at 480px on Orin Nano Super. If ≤400ms → use LiteSAM. If >400ms → **abandon LiteSAM, use XFeat as primary**. No hybrid compromises — pick one and optimize it.
### 4. TensorRT FP16 Optimization
LiteSAM's MobileOne backbone is reparameterizable — multi-branch training structure collapses to a single feed-forward path at inference. Combined with TensorRT FP16, this maximizes throughput. INT8 is possible for MobileOne backbone but ViT/transformer components may degrade with INT8.
### 5. CUDA Stream Pipelining
Overlap operations across consecutive frames:
- Stream A: cuVSLAM VO for current frame (~11ms) + ESKF fusion (~1ms)
- Stream B: Satellite matching for previous keyframe (async)
- CPU: SSE emission, tile management, keyframe selection logic
### 6. Pre-cropped Satellite Tiles
Offline: for each satellite tile, store both the raw image and a pre-resized version matching the satellite matcher's input resolution. Runtime avoids resize cost.
## Existing/Competitor Solutions Analysis
| Solution | Approach | Accuracy | Hardware | Limitations |
|----------|----------|----------|----------|-------------|
| Mateos-Ramirez et al. (2024) | VO (ORB) + satellite keypoint correction + Kalman | 142m mean / 17km (0.83%) | Orange Pi class | No re-localization; ORB only; 1000m+ altitude |
| SatLoc (2025) | DinoV2 + XFeat + optical flow + adaptive fusion | <15m, >90% coverage | Edge (unspecified) | Paper not fully accessible |
| LiteSAM (2025) | MobileOne + TAIFormer + MinGRU subpixel refinement | RMSE@30 = 17.86m on UAV-VisLoc | RTX 3090 (62ms), AGX Orin (497ms) | Not tested on Orin Nano; AGX Orin is 3-4x more powerful |
| TerboucheHacene/visual_localization | SuperPoint/SuperGlue/GIM + VO + satellite | Not quantified | Desktop-class | Not edge-optimized |
| cuVSLAM (NVIDIA, 2025-2026) | CUDA-accelerated VO+SLAM, mono/stereo/IMU | <1% trajectory error (KITTI), <5cm (EuRoC) | Jetson Orin Nano (90fps) | VO only, no satellite matching |
**Key insight**: Combine cuVSLAM (best-in-class VO for Jetson) with the fastest viable satellite-aerial matcher via ESKF fusion. LiteSAM is the accuracy leader but unproven on Orin Nano Super — benchmark first, abandon for XFeat if too slow.
The selected design borrows the proven structure but rejects an all-in-one SLAM dependency as the product core. OpenVINS and ORB-SLAM3 are useful benchmark/reference implementations, but their GPL-family licensing and broader SLAM lifecycle do not fit a default production dependency for this project.
## Architecture
### Component: Visual Odometry
### Component: Camera Ingest, Calibration, And Geometry
| Solution | Tools | Advantages | Limitations | Fit |
|----------|-------|-----------|-------------|-----|
| cuVSLAM (mono+IMU) | PyCuVSLAM / C++ API | 90fps on Orin Nano, NVIDIA-optimized, loop closure, IMU fallback | Closed-source CUDA library | ✅ Best |
| XFeat frame-to-frame | XFeatTensorRT | 5x faster than SuperPoint, open-source | ~30-50ms total, no IMU integration | ⚠️ Fallback |
| ORB-SLAM3 | OpenCV + custom | Well-understood, open-source | CPU-heavy, ~30fps on Orin | ⚠️ Slower |
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Cost | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|------|-------------------------|-----|
| OpenCV geometry utility layer | OpenCV 4.x | Camera calibration, undistortion, RANSAC homography, reprojection-error measurement | Mature, local, fast, exact API fit | Not a full estimator | Calibration target, fixed intrinsics/extrinsics, lens/FOV selection | Local-only, no network | Low | MVE: `_docs/00_research/02_fact_cards.md`; Source #5 | Selected |
**Selected**: **cuVSLAM (mono+IMU mode)** — purpose-built by NVIDIA for Jetson. ~11ms/frame leaves 389ms for everything else. Auto-fallback to IMU when visual tracking fails.
**Exact-fit evidence**:
- Project constraints checked: fixed nadir camera, calibration, homography, MRE gates.
- Disqualifiers: none.
- Restrictions × AC matrix: `_docs/00_research/06_component_fit_matrix.md`.
### Component: Satellite Image Matching
### Component: VO / IMU Propagation And Estimator
| Solution | Tools | Advantages | Limitations | Fit |
|----------|-------|-----------|-------------|-----|
| LiteSAM (opt) | TensorRT | Best satellite-aerial accuracy (RMSE@30 17.86m), 6.31M params, subpixel refinement | 497ms on AGX Orin at 1184px; AGX Orin is 3-4x more powerful than Orin Nano Super | ✅ If benchmark passes |
| XFeat semi-dense | XFeatTensorRT | ~50-100ms, lightweight, Jetson-proven | Not designed for cross-view satellite-aerial | ✅ If LiteSAM fails benchmark |
| EfficientLoFTR | TensorRT | Good accuracy, semi-dense | 15.05M params (2.4x LiteSAM), slower | ⚠️ Heavier |
| SuperPoint + LightGlue | TensorRT C++ | Good general matching | Sparse only, worse on satellite-aerial | ⚠️ Not specialized |
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Cost | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|------|-------------------------|-----|
| Custom VO/IMU ESKF | OpenCV + custom estimator | Frame-to-frame homography/features + FC IMU/attitude/altitude fused in a custom ESKF with source modes | Owns covariance, source labels, degraded modes, ArduPilot output semantics | More implementation work than adopting VIO library | Synchronized frames/IMU, calibration, replay tests | No third-party cloud; deterministic local logs | Medium | Facts #1, #4, #5, #16 | Selected |
| OpenVINS reference | OpenVINS | Monocular camera + IMU EKF/MSCKF reference runs | Strong VIO reference and evaluation tools | GPL-3 production dependency risk | Dataset/replay adapter | Local only | Low for benchmark | Source #3 | Reference only |
| ORB-SLAM3 alternative | ORB-SLAM3 | Monocular-inertial SLAM | Mature SLAM benchmark | GPLv3, heavier map lifecycle, initialization complexity | Calibration, vocabulary, runtime tuning | Local only | Medium | Source #4 | Rejected for production |
**Selection**: Benchmark-driven. Day-one test on Orin Nano Super:
1. Export LiteSAM (opt) to TensorRT FP16
2. Measure at 480px, 640px, 800px
3. If ≤400ms at 480px → **LiteSAM**
4. If >400ms at any viable resolution → **XFeat semi-dense** (primary, no hybrid)
**Exact-fit evidence**:
- Project constraints checked: one camera + IMU, frame-by-frame output, covariance labels, blackout/spoofing modes.
- Runtime quality gate: validate drift and covariance on EuRoC-style and representative fixed-wing replay.
### Component: Sensor Fusion
### Component: Satellite Retrieval And Local Anchor
| Solution | Tools | Advantages | Limitations | Fit |
|----------|-------|-----------|-------------|-----|
| Error-State EKF (ESKF) | Custom Python/C++ | Lightweight, multi-rate, well-understood | Linear approximation | ✅ Best |
| Hybrid ESKF/UKF | Custom | 49% better accuracy | More complex | ⚠️ Upgrade path |
| Factor Graph (GTSAM) | GTSAM | Best accuracy | Heavy compute | ❌ Too heavy |
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Cost | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|------|-------------------------|-----|
| DINOv2-VLAD + FAISS + LightGlue | DINOv2/AnyLoc-style descriptors, FAISS, DISK/ALIKED+LightGlue, OpenCV RANSAC | Offline precomputed VPR chunk descriptors; conditional query descriptor; top-K FAISS; local matching on candidates | Matches AC-8.6; scalable top-K; local geometry verifies anchors | Needs Jetson profiling and model-size pruning | Fresh satellite cache, descriptors, dynamic K, RANSAC gates | Cache is local; no in-flight provider calls | Medium-high | MVE blocks in `02_fact_cards.md`; Sources #6-#9 | Selected with runtime gate |
| SuperPoint + LightGlue | SuperPoint, LightGlue | Same local matching with SuperPoint features | Strong technical baseline | SuperPoint license is restrictive | Legal review | Local only | Medium | Source #6 | Needs user decision |
| Classical SIFT/ORB-only | OpenCV | Handcrafted features and homography | Simple fallback, low compute | Poor cross-domain robustness | Feature-rich scenes | Local only | Low | Source #5 | Fallback / regression baseline |
**Selected**: **ESKF** with adaptive measurement noise. State vector: [position(3), velocity(3), orientation_quat(4), accel_bias(3), gyro_bias(3)] = 16 states.
**Exact-fit evidence**:
- Project constraints checked: offline cache, top-K dynamic retrieval, cross-domain local match, <400 ms hot-path constraint.
- Runtime gate: heavy VPR/re-ranking is trigger-based, not per-frame.
Measurement sources and rates:
- IMU prediction: 100+Hz
- cuVSLAM VO update: ~3Hz (every frame)
- Satellite update: ~0.3-1Hz (keyframes only, delayed via async pipeline)
### Component: Satellite Cache And Tile Write-Back
### Component: Satellite Tile Preprocessing (Offline)
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Cost | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|------|-------------------------|-----|
| COG + PostgreSQL/PostGIS manifest + descriptor sidecars | GDAL COG, PostgreSQL/PostGIS manifest, FAISS sidecars | Service tiles and generated candidate tiles stored as tiled compressed GeoTIFFs with CRS/date/source/meter-per-pixel metadata | Geospatial standard, supports write-new-tile workflow, descriptor accounting | Needs careful 10 GB budget | STAC-like manifest, freshness gates, descriptor pruning | Local signed manifests recommended | Medium | Source #18 | Selected |
| PMTiles archive | PMTiles | Single-file read archive | Efficient map reads | Read-only; cannot update in place | Archive rebuild for updates | Local file integrity | Low | Source #17 | Rejected for live mutable cache |
**Selected**: **GeoHash-indexed tile pairs on disk**.
**Exact-fit evidence**:
- Project constraints checked: offline-only, no raw photo storage, mid-flight generated tiles, Satellite Service ingest metadata.
- External dependency: Satellite Service owns the promotion/voting layer for trusted basemap updates.
Pipeline:
1. Define operational area from flight plan
2. Download satellite tiles from Google Maps Tile API at max zoom (18-19)
3. Pre-resize each tile to matcher input resolution
4. Store: original tile + resized tile + metadata (GPS bounds, zoom, GSD) in GeoHash-indexed directory structure
5. Copy to Jetson storage before flight
### Component: MAVLink Integration And Telemetry
### Component: Re-localization (Disconnected Segments)
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Cost | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|------|-------------------------|-----|
| MAVSDK telemetry + pymavlink output | MAVSDK, pymavlink | MAVSDK subscribes to telemetry; pymavlink emits `GPS_INPUT` to ArduPilot with `GPS1_TYPE=14` | Exact `GPS_INPUT` field control while keeping high-level telemetry APIs | Plane-specific failsafe/spoof triggers need SITL proof | ArduPilot Plane params, QGC status, FDR tlog | Validate MAVLink source and message rate | Medium | Sources #10-#12 | Selected |
**Selected**: **Keyframe satellite matching is always active + expanded search on VO failure**.
**Exact-fit evidence**:
- Project constraints checked: ArduPilot only, v1 `GPS_INPUT` only, WGS84 output, covariance to `horiz_accuracy`.
- Validation gate: Plane SITL with production parameters.
When cuVSLAM reports tracking loss (sharp turn, no features):
1. Immediately flag next frame as keyframe → trigger satellite matching
2. Expand tile search radius (from ±200m to ±1km based on IMU dead-reckoning uncertainty)
3. If match found: position recovered, new segment begins
4. If 3+ consecutive keyframe failures: request user input via API
### Component: Flight Data Recorder
### Component: Object Center Coordinates
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Cost | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|------|-------------------------|-----|
| Segmented FDR | PostgreSQL event index + binary/CBOR payloads with Parquet export post-flight | Fixed-size segment files for per-frame estimates, IMU, MAVLink, health, emitted GPS_INPUT, generated-tile metadata | Replayable, bounded, no raw frame retention | Exact format selected during implementation | Rollover policy, monotonic timestamps | Integrity hash per segment recommended | Medium | AC-NEW-3 | Selected pattern |
Geometric calculation once frame-center GPS is known:
1. Pixel offset from center: (dx_px, dy_px)
2. Convert to meters: dx_m = dx_px × GSD, dy_m = dy_px × GSD
3. Rotate by IMU yaw heading
4. Convert meter offset to lat/lon and add to frame-center GPS
## Runtime Modes
### Component: API & Streaming
**Selected**: **FastAPI + sse-starlette**. REST for session management, SSE for real-time position stream. OpenAPI auto-documentation.
## Processing Time Budget (per frame, 400ms budget)
### Normal Frame (non-keyframe, ~60-80% of frames)
| Step | Time | Notes |
|------|------|-------|
| Image capture + transfer | ~10ms | CSI/USB3 |
| Downsample (for cuVSLAM) | ~2ms | OpenCV CUDA |
| cuVSLAM VO+IMU | ~11ms | NVIDIA CUDA-optimized, 90fps capable |
| ESKF fusion (VO+IMU update) | ~1ms | C extension or NumPy |
| SSE emit | ~1ms | Async |
| **Total** | **~25ms** | Well within 400ms |
### Keyframe Satellite Matching (async, every 3-10 frames)
Runs asynchronously on a separate CUDA stream — does NOT block per-frame VO output.
**Path A — LiteSAM (if benchmark passes)**:
| Step | Time | Notes |
|------|------|-------|
| Downsample to ~480px | ~1ms | OpenCV CUDA |
| Load satellite tile | ~5ms | Pre-resized, from storage |
| LiteSAM (opt) matching | ~300-500ms | TensorRT FP16, 480px, Orin Nano Super estimate |
| Geometric pose (RANSAC) | ~5ms | Homography estimation |
| ESKF satellite update | ~1ms | Delayed measurement |
| **Total** | **~310-510ms** | Async, does not block VO |
**Path B — XFeat (if LiteSAM abandoned)**:
| Step | Time | Notes |
|------|------|-------|
| XFeat feature extraction (both images) | ~10-20ms | TensorRT FP16/INT8 |
| XFeat semi-dense matching | ~30-50ms | KNN + refinement |
| Geometric verification (RANSAC) | ~5ms | |
| ESKF satellite update | ~1ms | |
| **Total** | **~50-80ms** | Comfortably within budget |
### Per-Frame Wall-Clock Latency
Every frame:
- **VO result emitted in ~25ms** (cuVSLAM + ESKF + SSE)
- Satellite correction arrives asynchronously on keyframes
- Client gets immediate position, then refined position when satellite match completes
## Memory Budget (Jetson Orin Nano Super, 8GB shared)
| Component | Memory | Notes |
|-----------|--------|-------|
| OS + runtime | ~1.5GB | JetPack 6.2 + Python |
| cuVSLAM | ~200-300MB | NVIDIA CUDA library + internal state |
| Satellite matcher TensorRT | ~50-100MB | LiteSAM FP16 or XFeat FP16 |
| Current frame (downsampled) | ~2MB | 640×480×3 |
| Satellite tile (pre-resized) | ~1MB | Single active tile |
| ESKF state + buffers | ~10MB | |
| FastAPI + SSE runtime | ~100MB | |
| **Total** | **~1.9-2.4GB** | ~25-30% of 8GB — comfortable margin |
## Confidence Scoring
| Level | Condition | Expected Accuracy |
|-------|-----------|-------------------|
| HIGH | Satellite match succeeded + cuVSLAM consistent | <20m |
| MEDIUM | cuVSLAM VO only, recent satellite correction (<500m travel) | 20-50m |
| LOW | cuVSLAM VO only, no recent satellite correction | 50-100m+ |
| VERY LOW | IMU dead-reckoning only (cuVSLAM + satellite both failed) | 100m+ |
| MANUAL | User-provided position | As provided |
## Key Risks and Mitigations
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| LiteSAM too slow on Orin Nano Super | HIGH | Misses 400ms deadline | **Abandon LiteSAM, use XFeat**. Day-one benchmark is the go/no-go gate |
| cuVSLAM not supporting nadir-only camera well | MEDIUM | VO accuracy degrades | Fall back to XFeat frame-to-frame matching |
| Google Maps satellite quality in conflict zone | HIGH | Satellite matching fails | Accept VO+IMU with higher drift; request user input sooner; alternative satellite providers |
| XFeat cross-view accuracy insufficient | MEDIUM | Position corrections less accurate than LiteSAM | Increase keyframe frequency; multi-tile consensus voting; geometric verification with strict RANSAC |
| cuVSLAM is closed-source | LOW | Hard to debug | Fallback to XFeat VO; cuVSLAM has Python+C++ APIs |
| Mode | Trigger | Behavior | Output Label |
|------|---------|----------|--------------|
| Satellite anchored | VPR + local match passes freshness, RANSAC, covariance, and Mahalanobis gates | ESKF absolute update, low covariance, tile generation eligible if sigma gate passes | `satellite_anchored` |
| VO extrapolated | Last anchor fresh enough, VO healthy, normal overlap | VO/IMU propagation, covariance grows with anchor age | `vo_extrapolated` |
| Dead reckoned | Visual blackout, VO failure without anchor, spoofing while no visual signal | IMU-only propagation, monotonic covariance growth, degraded fix type thresholds | `dead_reckoned` |
| Failsafe / no fix | Blackout >30 s or covariance >500 m | `GPS_INPUT.fix_type=0`, `horiz_accuracy=999.0`, QGC status | `dead_reckoned` with failsafe status |
## Testing Strategy
### Integration / Functional Tests
- End-to-end pipeline test with real flight data (60 images from input_data/)
- Compare computed positions against ground truth GPS from coordinates.csv
- Measure: percentage within 50m, percentage within 20m
- Test sharp-turn handling: introduce 90-degree heading change in sequence
- Test user-input fallback: simulate 3+ consecutive failures
- Test SSE streaming: verify client receives VO result within 50ms, satellite-corrected result within 500ms
- Test session management: start/stop/restart flight sessions via REST API
- Replay normal overlapping frames and assert AC-2.1a VO registration rate and AC-2.2 VO MRE.
- Replay satellite-anchor cases and assert AC-1.1/1.2, AC-2.2 cross-domain MRE, freshness gates, and source labels.
- Inject stale tiles and assert no `satellite_anchored` output.
- Inject sharp turns and disconnected segments and assert VPR relocalization.
- Run ArduPilot Plane SITL with `GPS1_TYPE=14` and assert valid `GPS_INPUT` fields, fix type degradation, and QGC statuses.
- Inject visual blackout + spoofed GPS and assert spoofed GPS is ignored, covariance grows, and thresholds match AC-NEW-8.
- Request AI-camera object coordinates and assert level-flight projection plus maneuver error bound.
### Non-Functional Tests
- **Day-one benchmark**: LiteSAM TensorRT FP16 at 480/640/800px on Orin Nano Super → go/no-go for LiteSAM
- cuVSLAM benchmark: verify 90fps monocular+IMU on Orin Nano Super
- Performance: measure per-frame processing time (must be <400ms)
- Memory: monitor peak usage during 1000-frame session (must stay <8GB)
- Stress: process 3000 frames without memory leak
- Keyframe strategy: vary interval (2, 3, 5, 10) and measure accuracy vs latency tradeoff
- Jetson Orin Nano Super profiling: <400 ms p95, <8 GB shared memory, 25 W no-throttle hot-soak.
- Cache build test for 400 km²: imagery + manifests + descriptors fit within budget or fail with explicit budget report.
- 8-hour FDR load test: <=64 GB, rollover logged, no silent data loss.
- Monte Carlo false-anchor and over-confidence tests for AC-NEW-4 and AC-NEW-7.
- Cold boot 50x: first valid `GPS_INPUT` <30 s p95.
## References
- LiteSAM (2025): https://www.mdpi.com/2072-4292/17/19/3349
- LiteSAM code: https://github.com/boyagesmile/LiteSAM
- cuVSLAM (2025-2026): https://github.com/NVlabs/PyCuVSLAM
- PyCuVSLAM API: https://nvlabs.github.io/PyCuVSLAM/api.html
- Mateos-Ramirez et al. (2024): https://www.mdpi.com/2076-3417/14/16/7420
- SatLoc (2025): https://www.scilit.com/publications/e5cafaf875a49297a62b298a89d5572f
- XFeat (CVPR 2024): https://arxiv.org/abs/2404.19174
- XFeat TensorRT for Jetson: https://github.com/PranavNedunghat/XFeatTensorRT
- EfficientLoFTR (CVPR 2024): https://github.com/zju3dv/EfficientLoFTR
- JetPack 6.2: https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-62/release-notes/
- Hybrid ESKF/UKF: https://arxiv.org/abs/2512.17505
- Google Maps Tile API: https://developers.google.com/maps/documentation/tile/satellite
Detailed source registry: `_docs/00_research/01_source_registry.md`.
Key sources:
- Fixed-wing satellite-aided VO: https://www.mdpi.com/2076-3417/14/16/7420
- Aerial VPR survey: https://arxiv.org/abs/2406.00885
- OpenVINS docs: https://docs.openvins.com/
- ORB-SLAM3 README: https://raw.githubusercontent.com/UZ-SLAMLab/ORB_SLAM3/master/README.md
- LightGlue README: https://raw.githubusercontent.com/cvg/LightGlue/main/README.md
- FAISS docs: https://faiss.ai/index.html
- ArduPilot GPSInput: https://ardupilot.org/mavproxy/docs/modules/GPSInput.html
- MAVLink GPS_INPUT: https://mavlink.io/en/messages/common.html#GPS_INPUT
- Jetson Orin Nano Super: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/nano-super-developer-kit/
- PMTiles: https://docs.protomaps.com/pmtiles/
- GDAL COG: https://gdal.org/en/stable/drivers/raster/cog.html
## Related Artifacts
- AC Assessment: `_docs/00_research/gps_denied_nav/00_ac_assessment.md`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md`
- AC assessment: `_docs/00_research/00_ac_assessment.md`
- Question decomposition: `_docs/00_research/00_question_decomposition.md`
- Source registry: `_docs/00_research/01_source_registry.md`
- Fact cards: `_docs/00_research/02_fact_cards.md`
- Comparison framework: `_docs/00_research/03_comparison_framework.md`
- Reasoning chain: `_docs/00_research/04_reasoning_chain.md`
- Validation log: `_docs/00_research/05_validation_log.md`
- Component fit matrix: `_docs/00_research/06_component_fit_matrix.md`
_docs/01_solution/solution_draft02.md
+85 -316
View File
@@ -4,353 +4,122 @@
| Old Component Solution | Weak Point (functional/security/performance) | New Solution |
|------------------------|----------------------------------------------|-------------|
| LiteSAM at 480px as satellite matcher | **Performance**: 497ms on AGX Orin at 1184px. Orin Nano Super is ~3-4x slower. At 480px estimated ~270-360ms — borderline. Paper uses PyTorch AMP, not TensorRT FP16. TensorRT could bring 2-3x improvement. | Add TensorRT FP16 as mandatory optimization step. Revised estimate at 480px with TensorRT: ~90-180ms. Still benchmark-driven: abandon if >400ms. |
| XFeat as LiteSAM fallback for satellite matching | **Functional**: XFeat is a general-purpose feature matcher, NOT designed for cross-view satellite-aerial gap. May fail on season/lighting differences between UAV and satellite imagery. | **Expand fallback options**: benchmark EfficientLoFTR (designed for weak-texture aerial) alongside XFeat. Consider STHN-style deep homography as third option. See detailed satellite matcher comparison below. |
| SP+LG considered as "sparse only, worse on satellite-aerial" | **Functional**: LiteSAM paper confirms "SP+LG achieves fastest inference speed but at expense of accuracy." Sparse matcher fails on texture-scarce regions. ~180-360ms on Orin Nano Super. | **Reject SP+LG** for both VO and satellite matching. cuVSLAM is 15-33x faster for VO. |
| cuVSLAM on low-texture terrain | **Functional**: cuVSLAM uses Shi-Tomasi corners + Lucas-Kanade tracking. On uniform agricultural fields/water bodies, features will be sparse → frequent tracking loss. IMU fallback lasts only ~1s. No published benchmarks for nadir agricultural terrain. Does NOT guarantee pose recovery after tracking loss. | **CRITICAL RISK**: cuVSLAM will likely fail frequently over low-texture terrain. Mitigation: (1) increase satellite matching frequency in low-texture areas, (2) use IMU dead-reckoning bridge, (3) accept higher drift in featureless segments, (4) XFeat VO as secondary fallback may also struggle on same terrain. |
| cuVSLAM memory estimate ~200-300MB | **Performance**: Map grows over time. For 3000-frame flights (~16min at 3fps), map could reach 500MB-1GB without pruning. | Configure cuVSLAM map pruning. Set max keyframes. Monitor memory. |
| Tile search on VO failure: "expand to ±1km" | **Functional**: Underspecified. Loading 10-20 tiles slow from disk I/O. | Preload tiles within ±2km of flight plan into RAM. Ranked search by IMU dead-reckoning position. |
| LiteSAM resolution | **Performance**: Paper benchmarked at 1184px on AGX Orin (497ms AMP). TensorRT FP16 with reparameterized MobileOne expected 2-3x faster. | Benchmark LiteSAM TRT FP16 at **1280px** on Orin Nano Super. If ≤200ms → use LiteSAM at 1280px. If >200ms → use XFeat. |
| SP+LG proposed for VO by user | **Performance**: ~130-280ms/frame on Orin Nano. cuVSLAM ~8.6ms/frame. No IMU, no loop closure. | **Reject SP+LG for VO.** cuVSLAM 15-33x faster. XFeat frame-to-frame remains fallback. |
| DINOv2-VLAD with possible TensorRT optimization | TensorRT conversion may produce limited speedup and can alter embedding distances on Jetson-class deployments. | Keep DINOv2-VLAD, but require descriptor-fidelity tests against PyTorch/ONNX before TensorRT descriptors are accepted. |
| FAISS CPU/GPU optional | FAISS GPU is not a safe default on Jetson ARM64/aarch64 packaging. | Pin FAISS as CPU-first on Jetson; use PQ/IVF and top-K caps before considering custom GPU builds. |
| LightGlue local matcher | SuperPoint path has license risk and community confusion. | Keep DISK/ALIKED+LightGlue as production default; SuperPoint remains license-gated benchmark/fallback only. |
| COG cache | "COG cache" could be misread as mutable in-place raster updates. | Use write-new COG tile objects plus manifest versioning and sidecars; never mutate COGs in place. |
| `GPS_INPUT` output | ArduPilot velocity ignore flags have reported EKF3 pitfalls. | SITL must validate velocity source parameters, ignore flags, and whether zero velocity is ever fused accidentally. |
| Visual/satellite anchoring | Draft did not emphasize adversarial/cache integrity enough. | Add signed cache manifests, tile provenance, freshness gates, anchor consistency checks, and FDR audit trail. |
## Product Solution Description
A real-time GPS-denied visual navigation system for fixed-wing UAVs, running entirely on a Jetson Orin Nano Super (8GB). The system determines frame-center GPS coordinates by fusing three information sources: (1) CUDA-accelerated visual odometry (cuVSLAM), (2) absolute position corrections from satellite image matching, and (3) IMU-based motion prediction. Results stream to clients via REST API + SSE in real time.
Build an onboard GPS-denied localization service that runs on the Jetson companion computer, uses the fixed downward navigation camera and flight-controller inertial telemetry, and emits ArduPilot `GPS_INPUT` estimates with calibrated covariance and source labels.
**Hard constraint**: Camera shoots at ~3fps (333-400ms interval). The full pipeline must complete within **400ms per frame**.
The production architecture is a trigger-based hybrid estimator:
**Satellite matching strategy**: Benchmark LiteSAM TensorRT FP16 at **1280px** on Orin Nano Super as a day-one priority. The paper's AGX Orin benchmark used PyTorch AMP — TensorRT FP16 with reparameterized MobileOne should yield 2-3x additional speedup. **Decision rule: if LiteSAM TRT FP16 at 1280px ≤200ms → use LiteSAM. If >200ms → use XFeat.**
**Core architectural principles**:
1. **cuVSLAM handles VO** — 116fps on Orin Nano 8GB, ~8.6ms/frame. SuperPoint+LightGlue was evaluated and rejected (15-33x slower, no IMU integration).
2. **Keyframe-based satellite matching** — satellite matcher runs on keyframes only (every 3-10 frames), amortizing cost. Non-keyframes rely on cuVSLAM VO + IMU.
3. **Every keyframe independently attempts satellite-based geo-localization** — handles disconnected segments natively.
4. **Pipeline parallelism** — satellite matching for frame N overlaps with VO processing of frame N+1 via CUDA streams.
5. **Proactive tile loading** — preload tiles within ±2km of flight plan into RAM for fast lookup during expanded search.
```
┌─────────────────────────────────────────────────────────────────┐
│ OFFLINE (Before Flight) │
│ Satellite Tiles → Download & Crop → Store as tile pairs │
│ (Google Maps) (per flight plan) (disk, GeoHash indexed) │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ ONLINE (During Flight) │
│ │
│ EVERY FRAME (400ms budget): │
│ ┌────────────────────────────────┐ │
│ │ Camera → Downsample (CUDA 2ms)│ │
│ │ → cuVSLAM VO+IMU (~9ms) │──→ ESKF Update → SSE Emit │
│ └────────────────────────────────┘ ↑ │
│ │ │
│ KEYFRAMES ONLY (every 3-10 frames): │ │
│ ┌────────────────────────────────────┐ │ │
│ │ Satellite match (async CUDA stream)│─────┘ │
│ │ LiteSAM TRT FP16 or XFeat │ │
│ │ (does NOT block VO output) │ │
│ └────────────────────────────────────┘ │
│ │
│ IMU: 100+Hz continuous → ESKF prediction │
│ TILES: ±2km preloaded in RAM from flight plan │
└─────────────────────────────────────────────────────────────────┘
```text
Nav camera + FC telemetry
|
v
Image quality + calibration + orthorectification
|
+--> Hot path: VO/IMU propagation --> custom ESKF --> GPS_INPUT + QGC + FDR
|
+--> Trigger path: DINOv2-VLAD query --> CPU FAISS top-K --> DISK/ALIKED+LightGlue --> RANSAC --> ESKF anchor
|
+--> Tile path: new COG tile + quality/provenance sidecar --> manifest update --> post-flight Satellite Service sync
```
## Speed Optimization Techniques
### 1. cuVSLAM for Visual Odometry (~9ms/frame)
NVIDIA's CUDA-accelerated VO library (v15.0.0, March 2026) achieves 116fps on Jetson Orin Nano 8GB at 720p. Supports monocular camera + IMU natively. Features: automatic IMU fallback when visual tracking fails, loop closure, Python and C++ APIs.
**Why not SuperPoint+LightGlue for VO**: SP+LG is 15-33x slower (~130-280ms vs ~9ms). Lacks IMU integration, loop closure, auto-fallback.
**CRITICAL: cuVSLAM on difficult/even terrain (agricultural fields, water)**:
cuVSLAM uses Shi-Tomasi corner detection + Lucas-Kanade optical flow tracking (classical features, not learned). On uniform agricultural terrain or water bodies:
- Very few corners will be detected → sparse/unreliable tracking
- Frequent keyframe creation → heavier compute
- Tracking loss → IMU fallback (~1 second) → constant-velocity integrator (~0.5s more)
- cuVSLAM does NOT guarantee pose recovery after tracking loss
- All published benchmarks (KITTI: urban/suburban, EuRoC: indoor) do NOT include nadir agricultural terrain
- Multi-stereo mode helps with featureless surfaces, but we have mono camera only
**Mitigation strategy for low-texture terrain**:
1. **Increase satellite matching frequency**: In low-texture areas (detected by cuVSLAM's keypoint count dropping), switch from every 3-10 frames to every frame
2. **IMU dead-reckoning bridge**: When cuVSLAM reports tracking loss, ESKF continues with IMU prediction. At 3fps with ~1.5s IMU bridge, that covers ~4-5 frames
3. **Accept higher drift**: In featureless segments, position accuracy degrades to IMU-only level (50-100m+ over ~10s). Satellite matching must recover absolute position when texture returns
4. **Keypoint density monitoring**: Track cuVSLAM's number of tracked features per frame. When below threshold (e.g., <50), proactively trigger satellite matching
5. **XFeat frame-to-frame as VO fallback**: XFeat uses learned features that may detect texture invisible to Shi-Tomasi corners. But XFeat may also struggle on truly uniform terrain
### 2. Keyframe-Based Satellite Matching
Not every frame needs satellite matching. Strategy:
- cuVSLAM provides VO at every frame (high-rate, low-latency)
- Satellite matching triggers on **keyframes** selected by:
- Fixed interval: every 3-10 frames (~1-3.3s between satellite corrections)
- Confidence drop: when ESKF covariance exceeds threshold
- VO failure: when cuVSLAM reports tracking loss (sharp turn)
### 3. Satellite Matcher Selection (Benchmark-Driven)
**Important context**: Our UAV-to-satellite matching is EASIER than typical cross-view geo-localization problems. Both the UAV camera and satellite imagery are approximately nadir (top-down). The main challenges are season/lighting differences, resolution mismatch, and temporal changes — not the extreme viewpoint gap seen in ground-to-satellite matching. This means even general-purpose matchers may perform well.
**Candidate A: LiteSAM (opt) with TensorRT FP16 at 1280px** — Best satellite-aerial accuracy (RMSE@30 = 17.86m on UAV-VisLoc). 6.31M params, MobileOne reparameterizable for TensorRT. Paper benchmarked at 497ms on AGX Orin using AMP at 1184px. TensorRT FP16 with reparameterized MobileOne expected 2-3x faster than AMP. At 1280px (close to paper's 1184px benchmark resolution), accuracy should match published results.
Orin Nano Super TensorRT FP16 estimate at 1280px:
- AGX Orin AMP @ 1184px: 497ms
- TRT FP16 speedup over AMP: ~2-3x → AGX Orin TRT estimate: ~165-250ms
- Orin Nano Super is ~3-4x slower → estimate: ~500-1000ms without TRT
- With TRT FP16: **~165-330ms** (realistic range)
- Go/no-go threshold: **≤200ms**
**Candidate B (fallback): XFeat semi-dense** — ~50-100ms on Orin Nano Super. Proven on Jetson. General-purpose, not designed for cross-view gap. FASTEST option. Since our cross-view gap is small (both nadir), XFeat may work adequately for this specific use case.
**Other evaluated options (not selected)**:
- **EfficientLoFTR**: Semi-dense, 15.05M params, handles weak-texture well. ~20% slower than LiteSAM. Strong option if LiteSAM codebase proves difficult to export to TRT, but larger model footprint.
- **Deep Homography (STHN-style)**: End-to-end homography estimation, no feature/RANSAC pipeline. 4.24m at 50m range. Interesting future option but needs RGB retraining — higher implementation risk.
- **PFED and retrieval-based methods**: Image RETRIEVAL only (identifies which tile matches), not pixel-level matching. We already know which tile to use from ESKF position.
- **SuperPoint+LightGlue**: Sparse matcher. LiteSAM paper confirms worse satellite-aerial accuracy. Slower than XFeat.
**Decision rule** (day-one on Orin Nano Super):
1. Export LiteSAM (opt) to TensorRT FP16
2. Benchmark at **1280px**
3. **If ≤200ms → use LiteSAM at 1280px**
4. **If >200ms → use XFeat**
### 4. TensorRT FP16 Optimization
LiteSAM's MobileOne backbone is reparameterizable — multi-branch training structure collapses to a single feed-forward path at inference. Combined with TensorRT FP16, this maximizes throughput. **Do NOT use INT8 on transformer components** (TAIFormer) — accuracy degrades. INT8 is safe only for the MobileOne backbone CNN layers.
### 5. CUDA Stream Pipelining
Overlap operations across consecutive frames:
- Stream A: cuVSLAM VO for current frame (~9ms) + ESKF fusion (~1ms)
- Stream B: Satellite matching for previous keyframe (async)
- CPU: SSE emission, tile management, keyframe selection logic
### 6. Proactive Tile Loading
**Change from draft01**: Instead of loading tiles on-demand from disk, preload tiles within ±2km of the flight plan into RAM at session start. This eliminates disk I/O latency during flight. For a 50km flight path, ~2000 tiles at zoom 19 ≈ ~200MB RAM — well within budget.
On VO failure / expanded search:
1. Compute IMU dead-reckoning position
2. Rank preloaded tiles by distance to predicted position
3. Try top 3 tiles (not all tiles in ±1km radius)
4. If no match in top 3, expand to next 3
## Existing/Competitor Solutions Analysis
| Solution | Approach | Accuracy | Hardware | Limitations |
|----------|----------|----------|----------|-------------|
| Mateos-Ramirez et al. (2024) | VO (ORB) + satellite keypoint correction + Kalman | 142m mean / 17km (0.83%) | Orange Pi class | No re-localization; ORB only; 1000m+ altitude |
| SatLoc (2025) | DinoV2 + XFeat + optical flow + adaptive fusion | <15m, >90% coverage | Edge (unspecified) | Paper not fully accessible |
| LiteSAM (2025) | MobileOne + TAIFormer + MinGRU subpixel refinement | RMSE@30 = 17.86m on UAV-VisLoc | RTX 3090 (62ms), AGX Orin (497ms@1184px) | Not tested on Orin Nano; AGX Orin is 3-4x more powerful |
| TerboucheHacene/visual_localization | SuperPoint/SuperGlue/GIM + VO + satellite | Not quantified | Desktop-class | Not edge-optimized |
| cuVSLAM (NVIDIA, 2025-2026) | CUDA-accelerated VO+SLAM, mono/stereo/IMU | <1% trajectory error (KITTI), <5cm (EuRoC) | Jetson Orin Nano (116fps) | VO only, no satellite matching |
| VRLM (2024) | FocalNet backbone + multi-scale feature fusion | 83.35% MA@20 | Desktop | Not edge-optimized |
| Scale-Aware UAV-to-Satellite (2026) | Semantic geometric + metric scale recovery | N/A | Desktop | Addresses scale ambiguity problem |
| EfficientLoFTR (CVPR 2024) | Aggregated attention + adaptive token selection, semi-dense | Competitive with LiteSAM | 2.5x faster than LoFTR, TRT available | 15.05M params, heavier than LiteSAM |
| PFED (2025) | Knowledge distillation + multi-view refinement, retrieval | 97.15% Recall@1 (University-1652) | AGX Orin (251.5 FPS) | Retrieval only, not pixel-level matching |
| STHN (IEEE RA-L 2024) | Deep homography estimation, coarse-to-fine | 4.24m at 50m range | Open-source, lightweight | Trained on thermal, needs RGB retraining |
| Hierarchical AVL (2025) | DINOv2 retrieval + SuperPoint matching | 64.5-95% success rate | ROS, IMU integration | Two-stage complexity |
| JointLoc (IROS 2024) | Retrieval + VO fusion, adaptive weighting | 0.237m RMSE over 1km | Open-source | Designed for Mars/planetary, needs adaptation |
Heavy retrieval and local matching are not steady-state per-frame dependencies. They run on cold start, VO failure, sharp turns, disconnected segments, covariance growth, or stale-anchor age.
## Architecture
### Component: Visual Odometry
### Component: Camera Ingest, Calibration, And Geometry
| Solution | Tools | Advantages | Limitations | Performance | Fit |
|----------|-------|-----------|-------------|------------|-----|
| cuVSLAM (mono+IMU) | PyCuVSLAM v15.0.0 | 116fps on Orin Nano, NVIDIA-optimized, loop closure, IMU fallback | Closed-source CUDA library | ~9ms/frame | ✅ Best |
| XFeat frame-to-frame | XFeatTensorRT | 5x faster than SuperPoint, open-source | ~30-50ms total, no IMU integration | ~30-50ms/frame | ⚠️ Fallback |
| SuperPoint+LightGlue | LightGlue-ONNX TRT | Good accuracy, adaptive pruning | ~130-280ms, no IMU, no loop closure | ~130-280ms/frame | ❌ Rejected |
| ORB-SLAM3 | OpenCV + custom | Well-understood, open-source | CPU-heavy, ~30fps on Orin | ~33ms/frame | ⚠️ Slower |
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Performance | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|-------------|-------------------------|-----|
| OpenCV geometry utility layer | OpenCV 4.x | Calibration, undistortion, RANSAC homography, MRE measurement | Mature, exact fit, permissive | Not a full estimator | Checkerboard calibration, fixed extrinsics, lens/FOV selection | Local-only | Fast enough for hot-path utility use | MVE in `02_fact_cards.md`; Source #5 | Selected |
**Selected**: **cuVSLAM (mono+IMU mode)** — 116fps, purpose-built by NVIDIA for Jetson. Auto-fallback to IMU when visual tracking fails.
### Component: VO / IMU Propagation And Estimator
**SP+LG rejection rationale**: 15-33x slower than cuVSLAM. No built-in IMU fusion, loop closure, or tracking failure detection. Building these features around SP+LG would take significant development time and still be slower. XFeat at ~30-50ms is a better fallback for VO if cuVSLAM fails on nadir camera.
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Performance | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|-------------|-------------------------|-----|
| Custom VO/IMU ESKF | OpenCV + custom estimator | Nadir VO/homography + FC IMU/attitude/altitude fused in ESKF with mode labels | Owns covariance, source labels, blackout/spoofing behavior | More implementation effort | Synchronized frames/IMU, calibration, replay tests | No network dependency | Hot path is lightweight | Facts #1, #16 | Selected |
| OpenVINS | OpenVINS | Monocular+IMU reference runs | Strong EKF/MSCKF reference | GPL-3 production risk | Replay adapter | Local only | Benchmark only | Source #3 | Reference only |
| ORB-SLAM3 | ORB-SLAM3 | Monocular-inertial SLAM | Mature benchmark | GPLv3 and heavier SLAM lifecycle | Calibration/vocabulary/runtime tuning | Local only | Riskier on embedded | Source #4 | Rejected for production |
### Component: Satellite Image Matching
### Component: Satellite Retrieval And Anchor Verification
| Solution | Tools | Advantages | Limitations | Performance | Fit |
|----------|-------|-----------|-------------|------------|-----|
| LiteSAM (opt) TRT FP16 @ 1280px | TensorRT | Best satellite-aerial accuracy (RMSE@30 17.86m), 6.31M params, subpixel refinement | Untested on Orin Nano Super with TensorRT | Est. ~165-330ms @ 1280px TRT FP16 | ✅ If ≤200ms |
| XFeat semi-dense | XFeatTensorRT | ~50-100ms, lightweight, Jetson-proven, fastest | General-purpose, not designed for cross-view. Our nadir-nadir gap is small → may work. | ~50-100ms | ✅ Fallback if LiteSAM >200ms |
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Performance | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|-------------|-------------------------|-----|
| DINOv2-VLAD + CPU FAISS + DISK/ALIKED+LightGlue | DINOv2/AnyLoc-style descriptors, FAISS CPU, LightGlue, OpenCV RANSAC | Offline VPR chunk descriptors; conditional query descriptor; CPU FAISS top-K; local match on candidates; TensorRT only after fidelity check | Strong retrieval+geometry structure; avoids per-frame map search | Requires profiling and representative data | Descriptor cache, dynamic K, freshness, RANSAC, Mahalanobis gates | Signed manifests, provenance, stale-tile rejection | Trigger path only; top-K capped | MVE blocks in `02_fact_cards.md`; Sources #6-#9, #21-#25 | Selected with runtime/fidelity gates |
| SuperPoint+LightGlue | SuperPoint, LightGlue | Same matcher with SuperPoint features | Strong technical baseline | SuperPoint license risk | Legal review | Local only | Benchmark only | Sources #6, #23 | Needs user decision |
| Classical SIFT/ORB | OpenCV | Handcrafted features + homography | Simple and cheap | Weak cross-domain robustness | Feature-rich scenes | Local only | Fast | Source #5 | Regression baseline |
**Selection**: Day-one benchmark on Orin Nano Super:
1. Export LiteSAM (opt) to TensorRT FP16
2. Benchmark at **1280px**
3. **If ≤200ms → LiteSAM at 1280px**
4. **If >200ms → XFeat**
### Component: Cache And Tile Lifecycle
### Component: Sensor Fusion
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Performance | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|-------------|-------------------------|-----|
| COG tile objects + manifest + sidecars | GDAL COG, manifest DB/JSON, FAISS index files | Service tiles and generated tiles are write-new COG objects; active version selected by manifest | Geospatial standard, supports provenance and quality metadata | Descriptor budget pressure | CRS/date/source/m/px/freshness, sidecar hashes | Signed manifests, tile provenance, hash verification | Efficient local reads | Source #18; Facts #21, #29 | Selected |
| PMTiles | PMTiles | Read-only archive snapshot | Compact read package | Cannot update in place | Archive rebuild | Hash archive | Good for read-only export | Source #17 | Rejected for live cache |
| Solution | Tools | Advantages | Limitations | Performance | Fit |
|----------|-------|-----------|-------------|------------|-----|
| Error-State EKF (ESKF) | Custom Python/C++ | Lightweight, multi-rate, well-understood | Linear approximation | <1ms/step | ✅ Best |
| Hybrid ESKF/UKF | Custom | 49% better accuracy | More complex | ~2-3ms/step | ⚠️ Upgrade path |
| Factor Graph (GTSAM) | GTSAM | Best accuracy | Heavy compute | ~10-50ms/step | ❌ Too heavy |
### Component: MAVLink Integration
**Selected**: **ESKF** with adaptive measurement noise. State vector: [position(3), velocity(3), orientation_quat(4), accel_bias(3), gyro_bias(3)] = 16 states.
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Performance | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|-------------|-------------------------|-----|
| MAVSDK telemetry + pymavlink `GPS_INPUT` | MAVSDK, pymavlink | MAVSDK subscriptions; pymavlink emits `GPS_INPUT`; Plane SITL validates `GPS1_TYPE=14`, velocity source params, ignore flags, fix types, accuracy fields | Exact output control with good telemetry ergonomics | SITL required to prove Plane behavior | ArduPilot Plane params, QGC, tlog/FDR | Link/source validation, status audit | Light CPU load | Sources #10-#12, #24 | Selected |
### Component: Satellite Tile Preprocessing (Offline)
### Component: Security And Safety Controls
**Selected**: **GeoHash-indexed tile pairs on disk + RAM preloading**.
| Solution | Tools | Pinned Mode/Config | Advantages | Limitations | Requirements | Security | Performance | API Capability Evidence | Fit |
|----------|-------|--------------------|------------|-------------|--------------|----------|-------------|-------------------------|-----|
| Consistency-gated anchor acceptance | Custom ESKF gates, cache manifest verification | Anchor accepted only if freshness, provenance, RANSAC, covariance, Mahalanobis, and temporal consistency pass | Prevents confident false fixes | Needs calibrated thresholds | Representative replay and Monte Carlo | Rejects stale/poisoned/low-confidence anchors | Lightweight after candidate generation | Facts #16, #17, #28 | Selected |
| FDR audit trail | Segmented logs + hashes | Logs estimates, inputs, emitted GPS_INPUT, health, tile writes, anchor decisions | Supports incident analysis and cache-poisoning audits | Schema work | 64 GB rollover | Tamper-evident hashes recommended | Sequential writes | AC-NEW-3 | Selected |
Pipeline:
1. Define operational area from flight plan
2. Download satellite tiles from Google Maps Tile API at max zoom (18-19)
3. Pre-resize each tile to matcher input resolution
4. Store: original tile + resized tile + metadata (GPS bounds, zoom, GSD) in GeoHash-indexed directory structure
5. Copy to Jetson storage before flight
6. **At session start**: preload tiles within ±2km of flight plan into RAM (~200MB for 50km route)
## Runtime Modes
### Component: Re-localization (Disconnected Segments)
**Selected**: **Keyframe satellite matching is always active + ranked tile search on VO failure**.
When cuVSLAM reports tracking loss (sharp turn, no features):
1. Immediately flag next frame as keyframe → trigger satellite matching
2. Compute IMU dead-reckoning position since last known position
3. Rank preloaded tiles by distance to dead-reckoning position
4. Try top 3 tiles sequentially (not all tiles in radius)
5. If match found: position recovered, new segment begins
6. If 3 consecutive keyframe failures across top tiles: expand to next 3 tiles
7. If still no match after 3+ full attempts: request user input via API
### Component: Object Center Coordinates
Geometric calculation once frame-center GPS is known:
1. Pixel offset from center: (dx_px, dy_px)
2. Convert to meters: dx_m = dx_px × GSD, dy_m = dy_px × GSD
3. Rotate by IMU yaw heading
4. Convert meter offset to lat/lon and add to frame-center GPS
### Component: API & Streaming
**Selected**: **FastAPI + sse-starlette**. REST for session management, SSE for real-time position stream. OpenAPI auto-documentation.
## Processing Time Budget (per frame, 400ms budget)
### Normal Frame (non-keyframe, ~60-80% of frames)
| Step | Time | Notes |
|------|------|-------|
| Image capture + transfer | ~10ms | CSI/USB3 |
| Downsample (for cuVSLAM) | ~2ms | OpenCV CUDA |
| cuVSLAM VO+IMU | ~9ms | NVIDIA CUDA-optimized, 116fps capable |
| ESKF fusion (VO+IMU update) | ~1ms | C extension or NumPy |
| SSE emit | ~1ms | Async |
| **Total** | **~23ms** | Well within 400ms |
### Keyframe Satellite Matching (async, every 3-10 frames)
Runs asynchronously on a separate CUDA stream — does NOT block per-frame VO output.
**Path A — LiteSAM TRT FP16 at 1280px (if ≤200ms benchmark)**:
| Step | Time | Notes |
|------|------|-------|
| Downsample to 1280px | ~1ms | OpenCV CUDA |
| Load satellite tile | ~1ms | Pre-loaded in RAM |
| LiteSAM (opt) TRT FP16 matching | ≤200ms | TensorRT FP16, 1280px, go/no-go threshold |
| Geometric pose (RANSAC) | ~5ms | Homography estimation |
| ESKF satellite update | ~1ms | Delayed measurement |
| **Total** | **≤210ms** | Async, within budget |
**Path B — XFeat (if LiteSAM >200ms)**:
| Step | Time | Notes |
|------|------|-------|
| XFeat feature extraction (both images) | ~10-20ms | TensorRT FP16/INT8 |
| XFeat semi-dense matching | ~30-50ms | KNN + refinement |
| Geometric verification (RANSAC) | ~5ms | |
| ESKF satellite update | ~1ms | |
| **Total** | **~50-80ms** | Comfortably within budget |
## Memory Budget (Jetson Orin Nano Super, 8GB shared)
| Component | Memory | Notes |
|-----------|--------|-------|
| OS + runtime | ~1.5GB | JetPack 6.2 + Python |
| cuVSLAM | ~200-500MB | CUDA library + map state. **Configure map pruning for 3000-frame flights** |
| Satellite matcher TensorRT | ~50-100MB | LiteSAM FP16 or XFeat FP16 |
| Preloaded satellite tiles | ~200MB | ±2km of flight plan, pre-resized |
| Current frame (downsampled) | ~2MB | 640×480×3 |
| ESKF state + buffers | ~10MB | |
| FastAPI + SSE runtime | ~100MB | |
| **Total** | **~2.1-2.9GB** | ~26-36% of 8GB — comfortable margin |
## Confidence Scoring
| Level | Condition | Expected Accuracy |
|-------|-----------|-------------------|
| HIGH | Satellite match succeeded + cuVSLAM consistent | <20m |
| MEDIUM | cuVSLAM VO only, recent satellite correction (<500m travel) | 20-50m |
| LOW | cuVSLAM VO only, no recent satellite correction | 50-100m+ |
| VERY LOW | IMU dead-reckoning only (cuVSLAM + satellite both failed) | 100m+ |
| MANUAL | User-provided position | As provided |
## Key Risks and Mitigations
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| **cuVSLAM fails on low-texture agricultural terrain** | **HIGH** | Frequent tracking loss, degraded VO | Increase satellite matching frequency when keypoint count drops. IMU dead-reckoning bridge (~1.5s). Accept higher drift in featureless segments. Satellite matching recovers position when texture returns. |
| LiteSAM TRT FP16 >200ms at 1280px on Orin Nano Super | MEDIUM | Must use XFeat instead (less accurate for cross-view) | Day-one TRT FP16 benchmark. If >200ms → XFeat. Since our nadir-nadir gap is small, XFeat may still perform adequately. |
| XFeat cross-view accuracy insufficient | MEDIUM | Satellite corrections less accurate | Benchmark XFeat on actual operational area satellite-aerial pairs. Increase keyframe frequency; multi-tile consensus; strict RANSAC. |
| cuVSLAM map memory growth on long flights | MEDIUM | Memory pressure | Configure map pruning, set max keyframes. Monitor memory. |
| Google Maps satellite quality in conflict zone | HIGH | Satellite matching fails | Accept VO+IMU with higher drift; request user input sooner; alternative satellite providers |
| cuVSLAM is closed-source, no nadir benchmarks | MEDIUM | Unknown failure modes over farmland | Extensive testing with real nadir UAV imagery before deployment. XFeat VO as fallback (also uses learned features). |
| Tile I/O bottleneck during expanded search | LOW | Delayed re-localization | Preload ±2km tiles in RAM; ranked search instead of exhaustive |
| Mode | Trigger | Behavior | `GPS_INPUT` / Telemetry |
|------|---------|----------|--------------------------|
| `satellite_anchored` | VPR + local match passes all gates | ESKF absolute update; tile write eligible only if sigma gate passes | 3D fix, `horiz_accuracy` >= 95% covariance semi-major axis |
| `vo_extrapolated` | VO healthy and anchor age/covariance within bounds | VO/IMU propagation; covariance grows | 3D/2D depending covariance threshold |
| `dead_reckoned` | visual blackout or no accepted anchor | IMU-only propagation, monotonic covariance growth | degraded fix type; QGC `VISUAL_BLACKOUT_IMU_ONLY` |
| failsafe/no-fix | covariance >500 m or blackout >30 s | stop pretending position is valid | `fix_type=0`, `horiz_accuracy=999.0`, QGC `VISUAL_BLACKOUT_FAILSAFE` |
## Testing Strategy
### Integration / Functional Tests
- End-to-end pipeline test with real flight data (60 images from input_data/)
- Compare computed positions against ground truth GPS from coordinates.csv
- Measure: percentage within 50m, percentage within 20m
- Test sharp-turn handling: introduce 90-degree heading change in sequence
- Test user-input fallback: simulate 3+ consecutive failures
- Test SSE streaming: verify client receives VO result within 50ms, satellite-corrected result within 500ms
- Test session management: start/stop/restart flight sessions via REST API
- Test cuVSLAM map memory: run 3000-frame session, monitor memory growth
- VO replay: assert AC-2.1a and AC-2.2 VO MRE on overlapping frame pairs.
- Satellite anchor replay: assert AC-1.1/1.2, AC-2.2 cross-domain MRE, freshness rejection, and source labels.
- DINOv2 descriptor fidelity: compare PyTorch/ONNX/TensorRT embeddings and retrieval rankings before accepting optimized engines.
- FAISS CPU index tests: top-K recall, query latency, index size, save/load behavior on Jetson ARM64.
- LightGlue extractor matrix: DISK vs ALIKED vs SuperPoint benchmark; SuperPoint output excluded from production unless license approved.
- COG cache lifecycle: write-new generated tile, update manifest, verify active version and rollback.
- `GPS_INPUT` SITL: validate fix type, `horiz_accuracy`, velocity fields, ignore flags, `EK3_SRC1_*` parameters, QGC behavior.
- Security gates: stale tile, mismatched tile hash, low inlier ratio, impossible velocity jump, and spoofed GPS during blackout.
### Non-Functional Tests
- **Day-one satellite matcher benchmark**: LiteSAM TRT FP16 at **1280px** on Orin Nano Super. If ≤200ms → use LiteSAM. If >200ms → use XFeat. Also measure accuracy on test satellite-aerial pairs for both.
- cuVSLAM benchmark: verify 116fps monocular+IMU on Orin Nano Super
- **cuVSLAM terrain stress test**: test with nadir camera over (a) urban/structured terrain, (b) agricultural fields, (c) water/uniform terrain, (d) forest. Measure: keypoint count, tracking success rate, drift per 100 frames, IMU fallback frequency
- cuVSLAM keypoint monitoring: verify that low-keypoint detection triggers increased satellite matching
- Performance: measure per-frame processing time (must be <400ms)
- Memory: monitor peak usage during 3000-frame session (must stay <8GB)
- Stress: process 3000 frames without memory leak
- Keyframe strategy: vary interval (2, 3, 5, 10) and measure accuracy vs latency tradeoff
- Tile preloading: verify RAM usage of preloaded tiles for 50km flight plan
- Jetson latency and memory: <400 ms p95, <8 GB shared memory, no 25 W thermal throttle.
- Cache budget: 400 km² imagery + manifests + descriptors fits budget or reports explicit split budget.
- FDR 8-hour load: <=64 GB, rollover logged, no silent payload loss.
- Monte Carlo false-position and cache-poisoning tests for AC-NEW-4 and AC-NEW-7.
- Cold boot: first valid `GPS_INPUT` <30 s p95 across 50 runs.
## References
- EfficientLoFTR (CVPR 2024): https://github.com/zju3dv/EfficientLoFTR
- EfficientLoFTR paper: https://zju3dv.github.io/efficientloftr/
- LoFTR TensorRT adaptation: https://github.com/Kolkir/LoFTR_TRT
- PFED (2025): https://github.com/SkyEyeLoc/PFED
- STHN (IEEE RA-L 2024): https://github.com/arplaboratory/STHN
- JointLoc (IROS 2024): https://github.com/LuoXubo/JointLoc
- Hierarchical AVL (MDPI 2025): https://www.mdpi.com/2072-4292/17/20/3470
- LiteSAM (2025): https://www.mdpi.com/2072-4292/17/19/3349
- LiteSAM code: https://github.com/boyagesmile/LiteSAM
- cuVSLAM (2025-2026): https://github.com/NVlabs/PyCuVSLAM
- cuVSLAM paper: https://arxiv.org/abs/2506.04359
- PyCuVSLAM API: https://nvlabs.github.io/PyCuVSLAM/api.html
- Intermodalics cuVSLAM benchmark: https://www.intermodalics.ai/blog/nvidia-isaac-ros-in-depth-cuvslam-and-the-dp3-1-release
- Mateos-Ramirez et al. (2024): https://www.mdpi.com/2076-3417/14/16/7420
- SatLoc (2025): https://www.scilit.com/publications/e5cafaf875a49297a62b298a89d5572f
- XFeat (CVPR 2024): https://arxiv.org/abs/2404.19174
- XFeat TensorRT for Jetson: https://github.com/PranavNedunghat/XFeatTensorRT
- EfficientLoFTR (CVPR 2024): https://github.com/zju3dv/EfficientLoFTR
- LightGlue (ICCV 2023): https://github.com/cvg/LightGlue
- LightGlue TensorRT: https://fabio-sim.github.io/blog/accelerating-lightglue-inference-onnx-runtime-tensorrt/
- LightGlue TRT Jetson: https://github.com/qdLMF/LightGlue-with-FlashAttentionV2-TensorRT
- ForestVO / SP+LG VO: https://arxiv.org/html/2504.01261v1
- vo_lightglue (SP+LG VO): https://github.com/himadrir/vo_lightglue
- JetPack 6.2: https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-62/release-notes/
- Hybrid ESKF/UKF: https://arxiv.org/abs/2512.17505
- Google Maps Tile API: https://developers.google.com/maps/documentation/tile/satellite
Detailed source registry: `_docs/00_research/01_source_registry.md`.
Key added Mode B sources:
- DINOv2 TensorRT issue: https://github.com/NVIDIA/TensorRT/issues/4348
- DINOv2 Jetson forum issue: https://forums.developer.nvidia.com/t/dinov2-tensorrt-model-performance-issue/312251
- LightGlue license discussion: https://github.com/cvg/LightGlue/issues/120
- ArduPilot GPS_INPUT velocity issue: https://github.com/ArduPilot/ardupilot/issues/19633
- FAISS install docs: https://github.com/facebookresearch/faiss/blob/main/INSTALL.md
- Orthophoto visual geolocalization: https://ar5iv.labs.arxiv.org/html/2103.14381
## Related Artifacts
- AC Assessment: `_docs/00_research/gps_denied_nav/00_ac_assessment.md`
- Tech stack evaluation: `_docs/01_solution/tech_stack.md`
- Security analysis: `_docs/01_solution/security_analysis.md`
- Component fit matrix: `_docs/00_research/06_component_fit_matrix.md`
- Fact cards: `_docs/00_research/02_fact_cards.md`
_docs/01_solution/tech_stack.md
+45 -243
View File
@@ -1,257 +1,59 @@
# Tech Stack Evaluation
## Requirements Summary
## Requirements Analysis
### Functional
- GPS-denied visual navigation for fixed-wing UAV
- Frame-center GPS estimation via VO + satellite matching + IMU fusion
- Object-center GPS via geometric projection
- Real-time streaming via REST API + SSE
- Disconnected route segment handling
- User-input fallback for unresolvable frames
### Non-Functional
- <400ms per-frame processing (camera @ ~3fps)
- <50m accuracy for 80% of frames, <20m for 60%
- <8GB total memory (CPU+GPU shared pool)
- Up to 3000 frames per flight session
- Image Registration Rate >95% (normal segments)
### Hardware Constraints
- **Jetson Orin Nano Super** (8GB LPDDR5, 1024 CUDA cores, 67 TOPS INT8)
- **JetPack 6.2.2**: CUDA 12.6.10, TensorRT 10.3.0, cuDNN 9.3
- ARM64 (aarch64) architecture
- No internet connectivity during flight
| Area | Requirement |
|------|-------------|
| Runtime | Jetson Orin Nano Super, Ubuntu/JetPack, CUDA/TensorRT available, 8 GB shared memory, 25 W thermal envelope. |
| Language | Python is acceptable for orchestration/prototyping; C++/TensorRT paths likely needed for hot vision loops. |
| Vision | Calibration, undistortion, homography, VPR descriptors, local matching, RANSAC verification. |
| Estimation | BASALT VIO for relative camera+IMU propagation, wrapped by project-owned safety/anchor estimator logic for covariance calibration, source labels, blackout/spoofing modes, and MAVLink output mapping. |
| Storage | Offline satellite cache, descriptor index, FDR with no raw frame retention. |
| Autopilot | ArduPilot Plane, `GPS_INPUT` through pymavlink, MAVSDK for telemetry. |
## Technology Evaluation
### Platform & OS
| Option | Version | Score (1-5) | Notes |
|--------|---------|-------------|-------|
| **JetPack 6.2.2 (L4T)** | Ubuntu 22.04 based | **5** | Only supported OS for Orin Nano Super. Includes CUDA 12.6, TensorRT 10.3, cuDNN 9.3 |
**Selected**: JetPack 6.2.2 — no alternative.
### Primary Language
| Option | Fitness | Maturity | Perf on Jetson | Ecosystem | Score |
|--------|---------|----------|----------------|-----------|-------|
| **Python 3.10+** | 5 | 5 | 4 | 5 | **4.8** |
| C++ | 5 | 5 | 5 | 3 | 4.5 |
| Rust | 3 | 3 | 5 | 2 | 3.3 |
**Selected**: **Python 3.10+** as primary language.
- cuVSLAM provides Python bindings (PyCuVSLAM v15.0.0)
- TensorRT has Python API
- FastAPI is Python-native
- OpenCV has full Python+CUDA bindings
- Performance-critical paths offloaded to CUDA via cuVSLAM/TensorRT — Python is glue code only
- C++ for custom ESKF if NumPy proves too slow (unlikely for 16-state EKF at 100Hz)
### Visual Odometry
| Option | Version | FPS on Orin Nano | Memory | License | Score |
|--------|---------|------------------|--------|---------|-------|
| **cuVSLAM (PyCuVSLAM)** | v15.0.0 (Mar 2026) | 116fps @ 720p | ~200-300MB | Free (NVIDIA, closed-source) | **5** |
| XFeat frame-to-frame | TensorRT engine | ~30-50ms/frame | ~50MB | MIT | 3.5 |
| ORB-SLAM3 | v1.0 | ~30fps | ~300MB | GPLv3 | 2.5 |
**Selected**: **PyCuVSLAM v15.0.0**
- 116fps on Orin Nano 8G at 720p (verified via Intermodalics benchmark)
- Mono + IMU mode natively supported
- Auto IMU fallback on tracking loss
- Pre-built aarch64 wheel: `pip install -e bin/aarch64`
- Loop closure built-in
**Risk**: Closed-source; nadir-only camera not explicitly tested. **Fallback**: XFeat frame-to-frame matching.
### Satellite Image Matching (Benchmark-Driven Selection)
**Day-one benchmark decides between two candidates:**
| Option | Params | Accuracy (UAV-VisLoc) | Est. Time on Orin Nano | License | Score |
|--------|--------|----------------------|----------------------|---------|-------|
| **LiteSAM (opt)** | 6.31M | RMSE@30 = 17.86m | ~300-500ms @ 480px (estimated) | Open-source | **4** (if fast enough) |
| **XFeat semi-dense** | ~5M | Not benchmarked on UAV-VisLoc | ~50-100ms | MIT | **4** (if LiteSAM too slow) |
**Decision rule**:
1. Export LiteSAM (opt) to TensorRT FP16 on Orin Nano Super
2. Benchmark at 480px, 640px, 800px
3. If ≤400ms at 480px → LiteSAM
4. If >400ms → **abandon LiteSAM, XFeat is primary**
| Requirement | LiteSAM (opt) | XFeat semi-dense |
|-------------|---------------|------------------|
| PyTorch → ONNX → TensorRT export | Required | Required |
| TensorRT FP16 engine | 6.31M params, ~25MB engine | ~5M params, ~20MB engine |
| Input preprocessing | Resize to 480px, normalize | Resize to 640px, normalize |
| Matching pipeline | End-to-end (detect + match + refine) | Detect → KNN match → geometric verify |
| Cross-view robustness | Designed for satellite-aerial gap | General-purpose, less robust |
### Sensor Fusion
| Option | Complexity | Accuracy | Compute @ 100Hz | Score |
|--------|-----------|----------|-----------------|-------|
| **ESKF (custom)** | Low | Good | <1ms/step | **5** |
| Hybrid ESKF/UKF | Medium | 49% better | ~2-3ms/step | 3.5 |
| GTSAM Factor Graph | High | Best | ~10-50ms/step | 2 |
**Selected**: **Custom ESKF in Python (NumPy/SciPy)**
- 16-state vector, well within NumPy capability
- FilterPy (v1.4.5, MIT) as reference/fallback, but custom implementation preferred for tighter control
- If 100Hz IMU prediction step proves slow in Python: rewrite as Cython or C extension (~1 day effort)
### Image Preprocessing
| Option | Tool | Time on Orin Nano | Notes | Score |
|--------|------|-------------------|-------|-------|
| **OpenCV CUDA resize** | cv2.cuda.resize | ~2-3ms (pre-allocated) | Must build OpenCV with CUDA from source. Pre-allocate GPU mats to avoid allocation overhead | **4** |
| NVIDIA VPI resize | VPI 3.2 | ~1-2ms | Part of JetPack, potentially faster | 4 |
| CPU resize (OpenCV) | cv2.resize | ~5-10ms | No GPU needed, simpler | 3 |
**Selected**: **OpenCV CUDA** (pre-allocated GPU memory) or **VPI 3.2** (whichever is faster in benchmark). Both available in JetPack 6.2.
- Must build OpenCV from source with `CUDA_ARCH_BIN=8.7` for Orin Nano Ampere architecture
- Alternative: VPI 3.2 is pre-installed in JetPack 6.2, no build step needed
### API & Streaming Framework
| Option | Version | Async Support | SSE Support | Score |
|--------|---------|--------------|-------------|-------|
| **FastAPI + sse-starlette** | FastAPI 0.115+, sse-starlette 3.3.2 | Native async/await | EventSourceResponse with auto-disconnect | **5** |
| Flask + flask-sse | Flask 3.x | Limited | Redis dependency | 2 |
| Raw aiohttp | aiohttp 3.x | Full | Manual SSE implementation | 3 |
**Selected**: **FastAPI + sse-starlette v3.3.2**
- sse-starlette: 108M downloads/month, BSD-3 license, production-stable
- Auto-generated OpenAPI docs
- Native async for non-blocking VO + satellite pipeline
- Uvicorn as ASGI server
### Satellite Tile Storage & Indexing
| Option | Complexity | Lookup Speed | Score |
|--------|-----------|-------------|-------|
| **GeoHash-indexed directory** | Low | O(1) hash lookup | **5** |
| SQLite + spatial index | Medium | O(log n) | 4 |
| PostGIS | High | O(log n) | 2 (overkill) |
**Selected**: **GeoHash-indexed directory structure**
- Pre-flight: download tiles, store as `{geohash}/{zoom}_{x}_{y}.jpg` + `{geohash}/{zoom}_{x}_{y}_resized.jpg`
- Runtime: compute geohash from ESKF position → direct directory lookup
- Metadata in JSON sidecar files
- No database dependency on the Jetson during flight
### Satellite Tile Provider
| Provider | Max Zoom | GSD | Pricing | Eastern Ukraine Coverage | Score |
|----------|----------|-----|---------|--------------------------|-------|
| **Google Maps Tile API** | 18-19 | ~0.3-0.5 m/px | 100K tiles free/month, then $0.48/1K | Partial (conflict zone gaps) | **4** |
| Bing Maps | 18-19 | ~0.3-0.5 m/px | 125K free/year (basic) | Similar | 3.5 |
| Mapbox Satellite | 18-19 | ~0.5 m/px | 200K free/month | Similar | 3.5 |
**Selected**: **Google Maps Tile API** (per restrictions.md). 100K free tiles/month covers ~25km² at zoom 19. For larger operational areas, costs are manageable at $0.48/1K tiles.
### Output Format
| Format | Standard | Tooling | Score |
|--------|----------|---------|-------|
| **GeoJSON** | RFC 7946 | Universal GIS support | **5** |
| CSV (lat, lon, confidence) | De facto | Simple, lightweight | 4 |
**Selected**: **GeoJSON** as primary, CSV as export option. Per AC: WGS84 coordinates.
| Layer | Selected | Alternatives Considered | Rationale | Risk |
|-------|----------|-------------------------|-----------|------|
| OS / GPU stack | JetPack Ubuntu + CUDA + TensorRT | Plain Ubuntu without JetPack | Required for Jetson acceleration and profiling. | Thermal/performance tuning. |
| Calibration / geometry | OpenCV 4.x | Custom NumPy-only geometry | Mature APIs for calibration, undistortion, homography, RANSAC. | Version pin and calibration quality. |
| VO / estimator | BASALT + project-owned safety/anchor wrapper | OpenVINS, Kimera-VIO, custom OpenCV/ESKF, ORB-SLAM3, VINS-Fusion | BASALT is the selected production VIO candidate; wrapper owns source labels, covariance gates, degraded modes, satellite-anchor acceptance, and MAVLink semantics. | BASALT confidence/covariance must be calibrated; nadir fixed-wing replay required. |
| VPR descriptors | DINOv2-VLAD / AnyLoc-style, model-size profiled, TensorRT only after fidelity check | MixVPR, SALAD, NetVLAD, classical BoW | Strong retrieval evidence; good offline descriptor model. | Memory/latency and embedding drift if optimized incorrectly. |
| Vector search | FAISS CPU-first on Jetson ARM64 | HNSWLIB, PostgreSQL/pgvector metadata-assisted search, brute-force NumPy, custom FAISS GPU build | Mature top-K search, save/load, PQ compression; PostgreSQL stores spatial/mission metadata around descriptor files. | CPU query latency must be profiled; GPU FAISS is not default on aarch64. |
| Local matching | DISK/ALIKED + LightGlue | SuperPoint+LightGlue, SIFT/ORB, LoFTR | Exact match outputs; Apache/BSD-friendly path; adaptive speed knobs. | Jetson profiling needed. |
| Raster cache | COG + PostgreSQL/PostGIS manifest + signed JSON sidecars | PMTiles, MBTiles, loose tile folders | COG fits geospatial raster and write-new-tile workflow; PostGIS supports spatial/freshness queries. | 10 GB budget pressure and local DB availability. |
| MAVLink | MAVSDK telemetry + pymavlink `GPS_INPUT` | MAVSDK-only, MAVProxy bridge | MAVSDK does telemetry well; `GPS_INPUT` needs raw field control. | Plane SITL validation. |
| FDR | PostgreSQL event index + CBOR/binary payload segments with optional Parquet export | Raw images, plain CSV | Queryable event metadata, bounded payload segments, no raw frame retention. | Schema and local DB availability. |
| Testing | ArduPilot Plane SITL + AerialVL/VPAir + EuRoC + representative flight replay | Public datasets only | No public dataset covers all ACs. | Representative data collection required. |
## Tech Stack Summary
```
┌────────────────────────────────────────────────────┐
│ HARDWARE: Jetson Orin Nano Super 8GB │
│ OS: JetPack 6.2.2 (L4T / Ubuntu 22.04) │
│ CUDA 12.6.10 / TensorRT 10.3.0 / cuDNN 9.3 │
├────────────────────────────────────────────────────┤
│ LANGUAGE: Python 3.10+ │
│ FRAMEWORK: FastAPI + sse-starlette 3.3.2 │
│ SERVER: Uvicorn (ASGI) │
├────────────────────────────────────────────────────┤
│ VISUAL ODOMETRY: PyCuVSLAM v15.0.0 │
│ SATELLITE MATCH: LiteSAM(opt) or XFeat (benchmark) │
│ SENSOR FUSION: Custom ESKF (NumPy/SciPy) │
│ PREPROCESSING: OpenCV CUDA or VPI 3.2 │
│ INFERENCE: TensorRT 10.3.0 (FP16) │
├────────────────────────────────────────────────────┤
│ TILE PROVIDER: Google Maps Tile API │
│ TILE STORAGE: GeoHash-indexed directory │
│ OUTPUT: GeoJSON (WGS84) via SSE stream │
└────────────────────────────────────────────────────┘
```
## Dependency List
### Python Packages (pip)
| Package | Version | Purpose |
|---------|---------|---------|
| pycuvslam | v15.0.0 (aarch64 wheel) | Visual odometry |
| fastapi | >=0.115 | REST API framework |
| sse-starlette | >=3.3.2 | SSE streaming |
| uvicorn | >=0.30 | ASGI server |
| numpy | >=1.26 | ESKF math, array ops |
| scipy | >=1.12 | Rotation matrices, spatial transforms |
| opencv-python (CUDA build) | >=4.8 | Image preprocessing (must build from source with CUDA) |
| torch (aarch64) | >=2.3 (JetPack-compatible) | LiteSAM model loading (if selected) |
| tensorrt | 10.3.0 (JetPack bundled) | Inference engine |
| pycuda | >=2024.1 | CUDA stream management |
| geojson | >=3.1 | GeoJSON output formatting |
| pygeohash | >=1.2 | GeoHash tile indexing |
### System Dependencies (JetPack 6.2.2)
| Component | Version | Notes |
|-----------|---------|-------|
| CUDA Toolkit | 12.6.10 | Pre-installed |
| TensorRT | 10.3.0 | Pre-installed |
| cuDNN | 9.3 | Pre-installed |
| VPI | 3.2 | Pre-installed, alternative to OpenCV CUDA for resize |
| cuVSLAM runtime | Bundled with PyCuVSLAM wheel | |
### Offline Preprocessing Tools (developer machine, not Jetson)
| Tool | Purpose |
|------|---------|
| Python 3.10+ | Tile download script |
| Google Maps Tile API key | Satellite tile access |
| torch + LiteSAM weights | Feature pre-extraction (if LiteSAM selected) |
| trtexec (TensorRT) | Model export to TensorRT engine |
- **Primary implementation**: Python for orchestration, test harness, cache tooling, and MAVLink integration; C++/TensorRT for hot-path vision if profiling requires it.
- **Vision utilities**: OpenCV 4.x.
- **Estimator**: BASALT VIO plus project-owned safety/anchor wrapper and mode machine.
- **Global retrieval**: DINOv2-VLAD style descriptors with CPU-first FAISS top-K search.
- **Local matching**: DISK/ALIKED + LightGlue; SuperPoint only after license review.
- **Cache**: COG imagery, PostgreSQL/PostGIS manifest metadata, signed JSON sidecars, FAISS index files.
- **Autopilot**: MAVSDK subscriptions plus pymavlink `GPS_INPUT`.
- **Validation**: public datasets for component de-risking, Plane SITL for integration, representative flight/replay data for acceptance.
## Risk Assessment
| Technology | Risk | Likelihood | Impact | Mitigation |
|-----------|------|-----------|--------|------------|
| cuVSLAM | Closed-source, nadir camera untested | Medium | High | XFeat frame-to-frame as open-source fallback |
| LiteSAM | May exceed 400ms on Orin Nano Super | High | High | **Abandon for XFeat** — day-one benchmark is go/no-go |
| OpenCV CUDA build | Build complexity on Jetson, CUDA arch compatibility | Medium | Low | VPI 3.2 as drop-in alternative (pre-installed) |
| Google Maps Tile API | Conflict zone coverage gaps, EEA restrictions | Medium | Medium | Test tile availability for operational area pre-flight; alternative providers (Bing, Mapbox) |
| Custom ESKF | Implementation bugs, tuning effort | Low | Medium | FilterPy v1.4.5 as reference; well-understood algorithm |
| Python GIL | Concurrent VO + satellite matching contention | Low | Low | CUDA operations release GIL; use asyncio + threading for I/O |
| Risk | Impact | Mitigation |
|------|--------|------------|
| VPR/local matching exceeds Jetson latency | AC-4.1 failure | Conditional VPR, top-K caps, downsampled descriptors, CPU FAISS profiling, TensorRT only after embedding-fidelity checks. |
| Descriptor cache exceeds 10 GB | AC-8.3/storage failure | PQ/compression, multi-scale budget report, split descriptor budget if needed. |
| GPL library accidentally becomes production dependency | Licensing issue | Keep OpenVINS/ORB-SLAM3 as reference only unless legal approves; BASALT is the production VIO candidate. |
| BASALT covariance under-reports real error | False-position safety budget failure | Calibrate wrapper covariance against OpenVINS covariance, ground truth replay, and satellite anchor residuals. |
| Plane failsafe differs from Copter docs | Safety behavior mismatch | Production-parameter ArduPilot Plane SITL gate. |
| `GPS_INPUT` velocity ignore flags behave unexpectedly | EKF drift or false velocity fusion | SITL tests for velocity fields, ignore flags, and `EK3_SRC1_*` source parameters. |
| Public datasets fail to represent Ukrainian agricultural terrain | False confidence | Require representative synchronized flight/replay data before AC signoff. |
| Thermal throttling | Latency regression | Hot-soak test and throttle logging per AC-NEW-5. |
## Learning Requirements
## Learning / Implementation Requirements
| Technology | Team Expertise Needed | Ramp-up Time |
|-----------|----------------------|--------------|
| PyCuVSLAM | SLAM concepts, Python API, camera calibration | 2-3 days |
| TensorRT model export | ONNX export, trtexec, FP16 optimization | 2-3 days |
| LiteSAM architecture | Transformer-based matching (if selected) | 1-2 days |
| XFeat | Feature detection/matching concepts | 1 day |
| ESKF | Kalman filtering, quaternion math, multi-rate fusion | 3-5 days |
| FastAPI + SSE | Async Python, ASGI, SSE protocol | 1 day |
| GeoHash spatial indexing | Geospatial concepts | 0.5 days |
| Jetson deployment | JetPack, power modes, thermal management | 2-3 days |
## Development Environment
| Environment | Purpose | Setup |
|-------------|---------|-------|
| **Developer machine** (x86_64, GPU) | Development, unit testing, model export | Docker with CUDA + TensorRT |
| **Jetson Orin Nano Super** | Integration testing, benchmarking, deployment | JetPack 6.2.2 flashed, SSH access |
Code should be developed and unit-tested on x86_64, then deployed to Jetson for integration/performance testing. cuVSLAM and TensorRT engines are aarch64-only — mock these in x86_64 tests.
- Jetson profiling with CUDA/TensorRT and memory instrumentation.
- ArduPilot Plane SITL, `GPS_INPUT`, GPS spoof/failsafe parameters.
- Geospatial raster formats: COG, CRS, tile matrices, m/px metadata, manifests.
- BASALT integration, covariance calibration, and Mahalanobis rejection in the safety/anchor wrapper.
- Aerial VPR benchmark methodology and georeference recall.
_docs/02_document/FINAL_report.md
+148
View File
@@ -0,0 +1,148 @@
# GPS-Denied Onboard Localization — Planning Report
## Executive Summary
The solution planning phase decomposed the GPS-denied onboard localization service into 8 runtime implementation components, 2 cross-cutting foundation epics, a bootstrap epic, and separate e2e/blackbox test epics. The architecture centers on a Jetson-hosted hot path using camera ingest, BASALT VIO, and a project-owned safety/anchor wrapper, with triggered Satellite Service candidate retrieval and ALIKED/DISK-LightGlue anchor verification against an offline PostgreSQL/PostGIS-backed cache.
Jira epics were created in project `AZ` from AZ-206 through AZ-218. Total estimated effort across epics is approximately 87-141 story points, with large work intentionally decomposed into child tasks of 2, 3, or 5 points where possible.
## Problem Statement
The system must provide reliable onboard WGS84 localization when GPS is denied or spoofed, using a fixed nadir camera, flight-controller telemetry, and an offline satellite cache. It must emit ArduPilot-compatible position estimates, report confidence honestly, degrade safely under blackout, and preserve enough forensic evidence for post-flight analysis without retaining raw frames.
## Architecture Overview
The system is a trigger-based hybrid estimator. Normal flight uses camera ingest, pre-VIO occlusion checks, BASALT VIO, and a safety/anchor wrapper. Relocalization triggers use DINOv2-VLAD, FAISS, ALIKED/DISK-LightGlue, and OpenCV RANSAC against the offline cache. The wrapper is the safety authority for covariance, source labels, degraded modes, tile-write eligibility, and MAVLink output semantics.
**Technology stack**: Python orchestration, C++/native vision paths where needed, OpenCV 4.x, BASALT, DINOv2-VLAD, FAISS CPU, ALIKED/DISK-LightGlue, PostgreSQL/PostGIS, COG, CBOR FDR segments, MAVSDK + pymavlink.
**Deployment**: Local onboard Jetson runtime with Docker/replay and Plane SITL for validation; release gates require Jetson hardware, Plane SITL, and representative synchronized replay data.
## Component Summary
| # | Component | Purpose | Dependencies | Epic |
|---|-----------|---------|--------------|------|
| 01 | Camera Ingest And Calibration | Ingest frames, validate calibration, detect total occlusion before VIO | Bootstrap, shared geometry/time, config/errors | AZ-209 |
| 02 | VIO Adapter | Wrap the selected relative VIO backend and emit replaceable state DTOs | Camera, MAVLink telemetry, shared helpers | AZ-213 |
| 03 | Safety And Anchor Wrapper | Own localization state, covariance, anchors, blackout/failsafe, output semantics | Camera, MAVLink, VIO, anchor verification | AZ-216 |
| 04 | Satellite Service | Sync Satellite Service cache/upload packages and retrieve local VPR candidates from cache descriptors and FAISS | Camera, Tile Manager, shared helpers | AZ-214 |
| 05 | Anchor Verification | Verify retrieved candidates with learned matching and RANSAC | Satellite Service, camera, Tile Manager | AZ-215 |
| 06 | Tile Manager | Manage COGs, PostGIS manifests, sidecars, freshness, and orthorectified generated tiles | Bootstrap, shared helpers, config/errors | AZ-211 |
| 07 | MAVLink And GCS Integration | Consume FC telemetry and emit v1 `GPS_INPUT`/QGC status | Bootstrap, config/errors | AZ-210 |
| 08 | FDR And Observability | Record bounded replayable evidence and status | Bootstrap, config/errors, runtime DTOs | AZ-212 |
| Test | E2E Test Suite | Separate black-box replay, SITL, Jetson, and release evidence tests; not onboard runtime | All runtime components | AZ-217 |
**Implementation order**:
1. Bootstrap and cross-cutting foundations: AZ-206, AZ-207, AZ-208.
2. Independent adapters/stores: AZ-209, AZ-210, AZ-211, AZ-212.
3. Estimation/relocalization: AZ-213, AZ-214, AZ-215.
4. Safety orchestration: AZ-216.
5. Separate e2e/blackbox test implementation: AZ-217, AZ-218.
## System Flows
| Flow | Description | Key Components |
|------|-------------|----------------|
| Pre-flight cache preparation | Validate offline cache, sidecars, descriptors, and indexes | Satellite Service, Tile Manager |
| Normal frame processing | Route usable frames through BASALT; route total occlusion to IMU-only degraded path | Camera, BASALT, safety, MAVLink, FDR |
| Satellite relocalization | Retrieve and verify cache candidates, then accept/reject anchors | Safety, Satellite Service, anchor verification, Tile Manager |
| Visual blackout / spoofing | Propagate IMU-only from last trusted state and fail safe at thresholds | Camera, safety, MAVLink, QGC, FDR |
| Generated tile lifecycle | Write generated COG candidates only under covariance/quality gates | Safety, Tile Manager, FDR |
| Post-flight sync and audit | Package generated tiles and FDR evidence | Tile Manager, FDR, Satellite Service |
| Validation replay | Exercise runtime through public interfaces | Validation harness, all runtime components |
See `system-flows.md` for full diagrams and details.
## Risk Summary
| Level | Count | Key Risks |
|-------|-------|-----------|
| Critical | 0 | None |
| High | 7 | Camera spec mismatch, BASALT nadir fit, covariance under-reporting, total occlusion false-negative, IMU-only over-trust, Jetson trigger-path performance, PostgreSQL/PostGIS availability |
| Medium | 5 | Cache poisoning, dataset coverage/licensing, FDR append pressure, GPL/non-commercial leakage, generated tile promotion risk |
| Low | 0 | None |
**Iterations completed**: 1
**All Critical/High risks mitigated**: Yes. High risks have concrete gates in architecture, component specs, and tests.
See `risk_mitigations.md` for the full register.
## Test Coverage
| Component | Integration | Performance | Security | Acceptance | AC Coverage |
|-----------|-------------|-------------|----------|------------|-------------|
| Camera Ingest And Calibration | 3 | 1 | 1 | 2 | 7 ACs |
| VIO Adapter | 4 | 1 | 1 | 1 | 8 ACs |
| Safety And Anchor Wrapper | 7 | 1 | 1 | 3 | 15 ACs |
| Satellite Service | 4 | 2 | 1 | 1 | 10 ACs |
| Anchor Verification | 2 | 1 | 2 | 1 | 9 ACs |
| Tile Manager | 4 | 1 | 3 | 1 | 10 ACs |
| MAVLink And GCS Integration | 6 | 2 | 1 | 1 | 10 ACs |
| FDR And Observability | 6 | 1 | 1 | 1 | 11 ACs |
| E2E Test Suite | 9 | 2 | 1 | 2 | All AC groups |
**Overall acceptance criteria coverage**: 39 / 39 acceptance criteria covered (100%).
**Restrictions coverage**: 10 / 10 restriction groups covered (100%).
## Epic Roadmap
| Order | Epic | Component / Concern | Effort | Dependencies |
|-------|------|---------------------|--------|--------------|
| 1 | AZ-206: Bootstrap & Initial Structure | Scaffold | M / 5-8 pts | none |
| 2 | AZ-207: Cross-Cutting: Shared Geometry And Time Sync | Shared helper | S-M / 3-5 pts | AZ-206 |
| 3 | AZ-208: Cross-Cutting: Runtime Configuration And Errors | Shared helper | S-M / 3-5 pts | AZ-206 |
| 4 | AZ-209: Camera Ingest And Calibration | Component 01 | M / 5-8 pts | AZ-206, AZ-207, AZ-208 |
| 5 | AZ-210: MAVLink And GCS Integration | Component 07 | M / 5-8 pts | AZ-206, AZ-208 |
| 6 | AZ-211: Tile Manager | Component 06 | L / 8-13 pts | AZ-206, AZ-207, AZ-208 |
| 7 | AZ-212: FDR And Observability | Component 08 | M-L / 5-8 pts | AZ-206, AZ-208 |
| 8 | AZ-213: VIO Adapter | Component 02 | L / 8-13 pts | AZ-209, AZ-210 |
| 9 | AZ-214: Satellite Service | Component 04 | L / 8-13 pts | AZ-209, AZ-211 |
| 10 | AZ-215: Anchor Verification | Component 05 | L / 8-13 pts | AZ-214, AZ-209, AZ-211 |
| 11 | AZ-216: Safety And Anchor Wrapper | Component 03 | XL / 13-21 pts | AZ-209, AZ-210, AZ-213, AZ-215 |
| 12 | AZ-217: E2E Test Suite | Separate test support | L / 8-13 pts | Component epics |
| 13 | AZ-218: Blackbox Tests | System tests | L / 8-13 pts | AZ-217, component epics |
**Total estimated effort**: 87-141 story points.
## Key Decisions Made
| # | Decision | Rationale | Alternatives Rejected |
|---|----------|-----------|----------------------|
| 1 | Use BASALT as production VIO candidate | Permissive license and strong VIO benchmark fit | OpenVINS production dependency, custom VIO from scratch |
| 2 | Keep safety/anchor wrapper as authority | Product semantics require calibrated covariance, labels, gates, failsafe, MAVLink mapping | Letting BASALT/OpenVINS own output safety |
| 3 | Use ALIKED/DISK-LightGlue for anchor verification | Strong local correspondences for cross-domain verification | Per-frame learned matcher as primary VIO hot path |
| 4 | Add pre-VIO total-occlusion gate | Safer and cheaper than feeding fully unusable frames to VIO | Letting BASALT detect all visual failures |
| 5 | Use PostgreSQL/PostGIS for structured metadata | User confirmed PostgreSQL; PostGIS fits spatial cache/mission metadata | JSON-only or embedded single-file metadata DB |
| 6 | Use CBOR FDR payload segments with PostgreSQL index | Keeps high-volume append data bounded and queryable | Raw-frame retention, plain CSV, Parquet as runtime primary |
| 7 | v1 emits `GPS_INPUT` only | Avoid ArduPilot EKF3 double-fusion risk in v1 | Parallel `ODOMETRY` in v1 |
## Open Questions
| # | Question | Impact | Assigned To |
|---|----------|--------|-------------|
| 1 | Exact ADTi camera lens, interface, sustained FPS, and temperature spec | Blocks final camera calibration and runtime FPS assumptions | Hardware/product owner |
| 2 | Final representative synchronized target dataset collection timing | Blocks final acceptance, though public datasets can de-risk | Project/product owner |
| 3 | Dataset license approval for ALTO/Kagaru/EPFL/VPAir/UZH FPV use | Blocks commercial acceptance evidence for restricted datasets | Legal/product owner |
| 4 | Local onboard PostgreSQL/PostGIS deployment profile | Blocks implementation details for DB persistence and health checks | Backend/runtime owner |
## Artifact Index
| File | Description |
|------|-------------|
| `glossary.md` | Confirmed project glossary |
| `architecture.md` | System architecture and ADRs |
| `data_model.md` | System data model and storage strategy |
| `system-flows.md` | Main runtime and validation flows |
| `deployment/containerization.md` | Container/replay strategy |
| `deployment/ci_cd_pipeline.md` | CI/CD and release gates |
| `deployment/environment_strategy.md` | Environment and dataset strategy |
| `deployment/observability.md` | Runtime signals, logs, and alerts |
| `deployment/deployment_procedures.md` | Deployment, rollback, and health checks |
| `components/*/description.md` | Component specifications |
| `components/*/tests.md` | Component test specifications |
| `common-helpers/*.md` | Shared helper specifications |
| `diagrams/component_overview.md` | Component overview Mermaid diagram |
| `diagrams/flows/*.md` | Flow-specific Mermaid diagrams |
| `risk_mitigations.md` | Risk register and mitigations |
| `epics.md` | Jira epic mapping and dependency roadmap |
| `FINAL_report.md` | This final planning report |
_docs/02_document/architecture.md
+241
View File
@@ -0,0 +1,241 @@
# GPS-Denied Onboard Localization — Architecture
## Architecture Vision
Build a Jetson-hosted onboard localization pipeline for fixed-wing GPS-denied flight. The hot path fuses fixed nadir camera frames and FC telemetry through OpenCV geometry, BASALT VIO, and a project-owned safety/anchor wrapper that emits calibrated `GPS_INPUT` estimates and QGC/FDR status. A triggered satellite-anchor path uses DINOv2-VLAD, CPU FAISS, ALIKED/DISK+LightGlue, and RANSAC against the offline cache; generated tiles are written back only with strict provenance and covariance gates.
### Components / Responsibilities
- Camera ingest/calibration: load frames, apply intrinsics/extrinsics, validate image quality.
- VIO adapter: produce relative camera+IMU motion from synchronized nav frames and FC IMU.
- Safety/anchor wrapper: own covariance calibration, source labels, degraded modes, anchor fusion, and `GPS_INPUT`.
- Satellite Service: sync mission cache packages before flight, upload generated-tile packages after flight, and serve local VPR candidate retrieval from the offline cache.
- Anchor verification: run local matching/RANSAC and reject unsafe anchors.
- Tile Manager: manage COGs, manifests, freshness/provenance, orthorectified generated tiles, and local tile metadata.
- MAVLink/GCS integration: consume FC telemetry and emit `GPS_INPUT`/QGC status.
- FDR/observability: record replayable mission evidence under storage caps.
- Validation harness: run still-image, public dataset, SITL, Jetson, and representative replay tests.
### Principles / Non-Negotiables
- No in-flight satellite-provider or Satellite Service calls; runtime uses offline cache only.
- BASALT is a VIO component, not the safety authority.
- Confidence must be honest; covariance must grow in degraded modes.
- Heavy VPR/local matching is trigger-based, not per-frame.
- Raw nav/AI frames are not retained in normal operation.
- GPL VIO libraries remain reference-only unless explicitly approved.
- Plane SITL and Jetson hardware are release gates.
- Public datasets can de-risk, but representative synchronized flight data is required for final acceptance.
## 1. System Context
**Problem being solved**: During fixed-wing flight, GPS may be denied or spoofed. The onboard system must estimate WGS84 coordinates for navigation-camera frame centers and detected objects, stream `GPS_INPUT` to ArduPilot Plane, report confidence honestly, and maintain safety during VO failure, stale imagery, spoofing, and visual blackout.
**System boundaries**:
- In scope: onboard localization runtime, offline cache consumption, BASALT VIO integration, satellite anchor verification, MAVLink output, QGC status, FDR, generated tile metadata, and a separate e2e/black-box test suite.
- Out of scope: upstream commercial satellite-provider sourcing, Satellite Service ingest implementation, AI mission-camera detection itself, PX4 support, raw-frame retention as a normal operating mode.
**External systems**:
| System | Integration Type | Direction | Purpose |
|--------|------------------|-----------|---------|
| ArduPilot Plane FC | MAVLink | Inbound/Outbound | FC telemetry in, `GPS_INPUT` and status out |
| QGroundControl | MAVLink telemetry | Outbound | Downsampled operator status and failsafe messages |
| Azaion Suite Satellite Service | Offline file/cache sync | Inbound before flight, outbound after landing | Provides mission cache packages and receives generated-tile packages; never called mid-flight |
| Public/replay datasets | File/rosbag/fixture | Inbound to validation | De-risk BASALT, VPR, and anchor logic |
## 2. Technology Stack
| Layer | Technology | Version / Mode | Rationale |
|-------|------------|----------------|-----------|
| OS / GPU stack | JetPack Ubuntu + CUDA/TensorRT/ONNX Runtime | Jetson Orin Nano Super target | Required for production hardware profiling |
| Runtime language | Python + C++ | Python orchestration; C++ for BASALT/hot vision paths | Fits MAVLink/test tooling and native VIO dependencies |
| Geometry | OpenCV 4.x | Calibration, undistortion, homography, RANSAC/USAC | Mature utility layer |
| VIO | BASALT | Production candidate | BSD-friendly, strong benchmark evidence |
| VIO reference | OpenVINS | Reference/covariance baseline only | Strong EKF covariance story; GPLv3 risk |
| Backup VIO | Kimera-VIO | Backup candidate | BSD-friendly fallback with mono caveats |
| Local matching | ALIKED/DISK + LightGlue | Anchor verification and optional VO fallback | Strong learned correspondences; profile before hot-path use |
| Retrieval | DINOv2-VLAD + CPU FAISS | Triggered VPR only | Robust candidate retrieval under cache/offline constraints |
| Structured metadata DB | PostgreSQL + PostGIS | Onboard/local deployment | Spatial cache manifests, mission state, generated-tile metadata, and FDR event indexes |
| Cache imagery | COG + PostgreSQL/PostGIS manifest + signed JSON sidecars | Write-new COG objects | Efficient geospatial rasters with queryable spatial metadata and auditable sidecars |
| FDR | PostgreSQL event index + CBOR segment payloads, optional Parquet export | Per-flight rollover | Queryable event metadata with compact bounded payload segments |
| MAVLink | MAVSDK + pymavlink | MAVSDK telemetry, pymavlink `GPS_INPUT` | Exact output control |
**Key constraints from restrictions.md**:
- Jetson has 8 GB shared memory and 25 W thermal envelope, so heavy VPR/local matching cannot run every frame.
- Runtime must be offline with respect to satellite providers, so all imagery and descriptors are preloaded.
- The camera is fixed nadir; all VO choices must be validated against low-parallax/planar terrain.
- ADTi public specs conflict with current assumptions on resolution, continuous FPS, and operating temperature; manufacturer specs must be pinned before implementation.
## 3. Deployment Model
**Environments**: Development replay, public-dataset replay, Jetson hardware validation, Plane SITL, representative flight/replay rig.
**Infrastructure**:
- Onboard production runtime runs on the Jetson companion computer, not in cloud.
- Replay/test infrastructure may use Docker for deterministic fixture tests.
- Release gates require local Jetson hardware and ArduPilot Plane SITL.
**Environment-specific configuration**:
| Config | Development | Production |
|--------|-------------|------------|
| Satellite cache | Small fixture cache | Preloaded operational-area cache |
| Descriptor index | Fixture FAISS index | CPU-first FAISS index with PQ/IVF if needed |
| Secrets/signing | Local test keys | Mission/cache signing keys from Suite process |
| FDR | Local temp output | Per-flight bounded NVMe storage |
| MAVLink | SITL/replay | Physical FC telemetry link |
## 4. Data Model Overview
**Core entities**:
| Entity | Description | Owned By Component |
|--------|-------------|--------------------|
| FrameRecord | Navigation-camera frame metadata, total-occlusion status, and processing status | Camera ingest/calibration |
| TelemetrySample | FC IMU, attitude, airspeed, altitude, GPS health | MAVLink/GCS integration |
| VioState | Backend-relative pose/velocity/bias output and quality metadata | VIO adapter |
| PositionEstimate | WGS84 estimate, covariance, source label, fix type, anchor age | Safety/anchor wrapper |
| VprChunk | Retrieval unit over cache imagery and descriptors | Satellite Service |
| AnchorCandidate | Retrieved tile/chunk with local-match and RANSAC evidence | Anchor verification |
| CacheTile | COG tile plus manifest and sidecar metadata | Tile Manager |
| GeneratedTile | In-flight orthorectified tile with trust/provenance metadata | Tile Manager |
| FdrSegment | Bounded replayable log segment | FDR/observability |
**Data flow summary**:
- Frame quality/total-occlusion gate + telemetry -> BASALT VIO when usable, or IMU-only degraded mode when not -> safety/anchor wrapper -> `GPS_INPUT`, QGC, FDR.
- Relocalization trigger -> DINOv2-VLAD/FAISS -> ALIKED/DISK+LightGlue/RANSAC -> accepted/rejected anchor.
- High-confidence pose + frame -> generated tile -> manifest/sidecar -> post-flight Satellite Service sync.
## 5. Integration Points
### Internal Communication
| From | To | Protocol | Pattern | Notes |
|------|----|----------|---------|-------|
| Camera ingest/calibration | VIO adapter | In-process queue or shared frame bus | Streaming | Timestamp discipline is critical |
| MAVLink telemetry | VIO adapter | In-process telemetry buffer | Streaming | IMU/attitude/altitude sync |
| VIO adapter | Safety/anchor wrapper | Typed state messages | Streaming | Wrapper calibrates confidence |
| Safety/anchor wrapper | Satellite Service | Command | Triggered local request | Uses only preloaded cache/index data during flight |
| Satellite Service | Anchor verification | Candidate list | Request-response | Dynamic top-K |
| Anchor verification | Safety/anchor wrapper | Anchor decision | Request-response | Includes MRE/inliers/provenance |
| Safety/anchor wrapper | MAVLink/GCS integration | Position/status DTO | Streaming | `GPS_INPUT` emitted frame-by-frame |
| Safety/anchor wrapper | FDR/observability | Append-only events | Streaming | Bounded segments |
### External Integrations
| External System | Protocol | Auth | Failure Mode |
|-----------------|----------|------|--------------|
| ArduPilot Plane | MAVLink | Source/system ID allowlist | Degrade/failsafe; never trust spoofed GPS blindly |
| QGroundControl | MAVLink | FC telemetry path | Downsampled status may be delayed but local FDR remains authoritative |
| Azaion Suite Satellite Service | Offline package sync | Signed manifests/sidecars | Missing/stale cache causes degraded mode, not mid-flight network fetch |
| Public datasets | File/rosbag | License constraints | Not final acceptance unless representative and license-compatible |
## 6. Non-Functional Requirements
| Requirement | Target | Measurement | Priority |
|-------------|--------|-------------|----------|
| Frame latency | <400 ms p95 | Capture/replay timestamp to emitted estimate | High |
| Memory | <8 GB shared | Jetson monitoring | High |
| First fix | <30 s p95 | 50 cold starts | High |
| Thermal | No throttle at 25 W / +50 C | 8-hour hot-soak | High |
| FDR storage | <=64 GB/flight | 8-hour synthetic load | High |
| Cache storage | ~10 GB persistent budget | Full mission cache accounting | High |
| False position | P(error >500 m) <0.1%, >1 km <0.01% | Monte Carlo/replay | High |
## 7. Security Architecture
**Authentication / trust boundary**:
- Runtime accepts only local cache files with valid manifest/signature/provenance.
- MAVLink input is filtered by expected source/system IDs and FC health semantics.
**Data protection**:
- At rest: FDR and cache sidecars should be integrity protected; mission secrets/signing keys are not stored in code.
- In transit: no in-flight satellite-provider or Satellite Service network dependency; MAVLink link security depends on FC/GCS deployment.
**Audit logging**:
- FDR records estimates, covariance, anchors, rejected anchors, cache validation failures, spoofing/blackout transitions, emitted `GPS_INPUT`, resource health, and tile-write decisions.
## 8. Key Architectural Decisions
### ADR-001: BASALT As Production VIO Candidate
**Context**: A naive OpenCV-only VIO implementation is risky, while OpenVINS has GPLv3 production constraints.
**Decision**: Use BASALT as the production relative VIO candidate and keep OpenVINS as covariance/reference baseline.
**Alternatives considered**:
1. OpenVINS as production core — rejected by default because of GPLv3 and generic VIO ownership.
2. Kimera-VIO — retained as backup due to BSD license but mono-inertial caveats.
3. Fully custom OpenCV/ESKF — fallback only because implementation burden is high.
**Consequences**: The safety/anchor wrapper must calibrate confidence around BASALT and prove it on representative data.
### ADR-002: ALIKED-LightGlue Role
**Context**: ALIKED-LightGlue can produce strong local correspondences and can support frame-to-frame homography/pose estimation.
**Decision**: Use ALIKED/DISK+LightGlue for satellite-anchor verification and evaluate it as an optional VO fallback/keyframe-assist path, not as the default BASALT replacement.
**Alternatives considered**:
1. Per-frame ALIKED-LightGlue VO hot path — deferred until Jetson profiling proves latency/memory fit.
2. SIFT/ORB-only matching — retained as regression baseline, weaker under cross-domain conditions.
3. SuperPoint+LightGlue — license-gated.
**Consequences**: Implementation tasks must benchmark ALIKED-LightGlue on frame-to-frame VO and cross-domain anchor workloads separately.
### ADR-003: Cache Metadata Format
**Context**: JSON is simple and auditable, but operational cache queries need spatial indexing, freshness filters, update safety, and integration with the project PostgreSQL database.
**Decision**: Use PostgreSQL with PostGIS as the primary cache manifest/index database, with signed JSON sidecars for each tile/generated tile for auditability and interchange.
**Alternatives considered**:
1. JSON-only manifest — simpler, but weak for query/update scale, spatial search, and consistency.
2. Embedded single-file metadata DB — efficient for small deployments, but rejected because the project will use PostgreSQL/PostGIS.
**Consequences**: The Tile Manager owns PostgreSQL migrations, PostGIS indexes, signature checks, generated-tile orthorectification metadata, and sidecar/db consistency.
### ADR-004: FDR Format
**Context**: The FDR must be compact, bounded, replayable, and exportable for analysis.
**Decision**: Use PostgreSQL for FDR event indexes and mission-query metadata, with CBOR-backed segment payloads for bounded append-heavy runtime data and optional Parquet export after flight.
**Alternatives considered**:
1. Plain CSV — rejected for type safety, size, and complex payloads.
2. Parquet as primary onboard format — good analytics, but less ideal as the runtime append/rollover path.
**Consequences**: FDR implementation must define PostgreSQL tables/indexes, CBOR segment schema, rollover behavior, and export tooling.
### ADR-006: Total Occlusion Before VIO
**Context**: BASALT should not receive frames that are completely unusable because of lens cover, cloud/whiteout, decode failure, extreme exposure, or other total visual blackout.
**Decision**: Camera ingest performs a pre-VIO total-occlusion/blackout check. Total occlusion bypasses BASALT for that frame, sends a `total_occlusion` or `visual_blackout` degradation signal to the safety wrapper, and continues IMU-only propagation from the last trusted state.
**Alternatives considered**:
1. Let BASALT detect every visual failure — rejected because total occlusion is cheaper and safer to catch before the VIO hot path.
2. Drop frames silently — rejected because the wrapper must grow covariance and emit honest degraded output.
**Consequences**: The camera component must expose `occlusion_status`, and tests must assert mode transition to `dead_reckoned`/failsafe under total blackout.
### ADR-005: Public Dataset Strategy
**Context**: The original still-image sample lacks synchronized IMU and ground-truth trajectory. The Derkachi fixture adds cropped nadir video synchronized with IMU and `GLOBAL_POSITION_INT` trajectory, but camera intrinsics, distortion, and camera-to-body calibration remain pending.
**Decision**: Prioritize MUN-FRL for synchronized nadir camera + IMU + GNSS/ground truth; use ALTO for aerial localization/VPR and long nadir trajectories; investigate Kagaru/EPFL for fixed-wing/farmland relevance; use EuRoC/UZH FPV only as VIO proxies if license-compatible.
**Consequences**: Public datasets de-risk components but do not replace representative target flight data for final acceptance.
_docs/02_document/common-helpers/01_helper_geo_geometry.md
+30
View File
@@ -0,0 +1,30 @@
# Geo Geometry Helper
## Purpose
Shared geospatial and camera-geometry utilities used by camera ingest, safety wrapper, Tile Manager, anchor verification, and validation.
## Responsibilities
- WGS84 to local tangent plane conversions.
- Haversine/ground-distance calculations.
- Ground sampling distance calculations.
- Camera footprint projection from intrinsics, extrinsics, altitude, and attitude.
- Homography and covariance unit conversions for reporting.
## Non-Responsibilities
- No image matching.
- No state estimation.
- No MAVLink emission.
- No cache policy decisions.
## Consumers
| Component | Usage |
|-----------|-------|
| Camera ingest/calibration | Footprint and calibration sanity checks |
| Safety/anchor wrapper | Distance/covariance/unit conversion |
| Anchor verification | Pixel-to-ground error reporting |
| Tile Manager | Tile footprint metadata |
| Validation harness | Error thresholds and reports |
_docs/02_document/common-helpers/02_helper_time_sync.md
+29
View File
@@ -0,0 +1,29 @@
# Time Sync Helper
## Purpose
Shared timestamp validation and alignment utilities for frame, IMU, telemetry, FDR, and replay data.
## Responsibilities
- Monotonic timestamp checks.
- Frame-to-IMU window selection.
- Clock-domain conversion metadata.
- Replay ordering validation.
- Gap and jitter metrics.
## Non-Responsibilities
- No VIO state estimation.
- No MAVLink parsing beyond normalized timestamp fields.
- No recovery policy; callers decide whether to degrade or reject.
## Consumers
| Component | Usage |
|-----------|-------|
| Camera ingest/calibration | Frame ordering and timestamp metadata |
| VIO adapter | IMU/frame synchronization |
| MAVLink/GCS integration | Telemetry timestamp normalization |
| FDR/observability | Segment ordering |
| Validation harness | Fixture validation |
_docs/02_document/components/01_camera_ingest_calibration/description.md
+115
View File
@@ -0,0 +1,115 @@
# Camera Ingest And Calibration
## 1. High-Level Overview
**Purpose**: Ingest navigation-camera frames, attach timestamps and calibration metadata, undistort/normalize imagery, detect total occlusion/blackout before VIO, and classify image quality before VIO or satellite matching consumes it.
**Architectural Pattern**: Streaming adapter + validation gate.
**Upstream dependencies**: Navigation camera, camera calibration files.
**Downstream consumers**: VIO adapter, Satellite Service, anchor verification, Tile Manager, FDR.
## 2. Internal Interfaces
### Interface: `FrameProvider`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `next_frame` | `FrameRequest` | `FramePacket` | Yes | `FrameUnavailable`, `CalibrationMissing`, `InvalidFrame` |
| `detect_occlusion` | `FramePacket` | `OcclusionReport` | No | `InvalidFrame` |
| `classify_quality` | `FramePacket` | `ImageQualityReport` | No | `InvalidFrame` |
**Input DTOs**:
```yaml
FrameRequest:
source: enum(live_camera, replay_file)
timestamp_ns: integer optional
```
**Output DTOs**:
```yaml
FramePacket:
frame_id: string
timestamp_ns: integer
image_ref: path_or_buffer
camera_calibration_id: string
altitude_hint_m: number optional
occlusion: OcclusionReport
quality: ImageQualityReport
OcclusionReport:
status: enum(clear, partial_occlusion, total_occlusion, blackout)
reason: enum(cloud, lens_cover, whiteout, decode_failure, underexposed, overexposed, unknown) optional
usable_for_vio: boolean
usable_for_anchor: boolean
ImageQualityReport:
usable_for_vio: boolean
usable_for_anchor: boolean
blackout_detected: boolean
blur_score: number
texture_score: number
```
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Load calibration by ID | Startup | Yes | No |
| Read replay frame | Test/replay | Yes | File order |
## 4. Implementation Details
**State Management**: Maintains current camera calibration version and frame sequence state.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| OpenCV | Decode, undistort, homography support, image metrics |
| Camera SDK / V4L2 / GigE SDK | Live camera access once interface is selected |
**Error Handling Strategy**:
- Camera or decode errors emit degraded quality and FDR events.
- Total occlusion or blackout sets `usable_for_vio=false`, bypasses BASALT for that frame, and emits a degradation signal to the safety/anchor wrapper.
- Missing calibration blocks production startup.
- ADTi public spec mismatch is tracked as a verification blocker until manufacturer spec is pinned.
## 5. Extensions and Helpers
| Helper | Purpose | Used By |
|--------|---------|---------|
| `geo_geometry_helper` | Coordinate transforms, GSD, WGS84/local conversions | Camera ingest, safety wrapper, Tile Manager |
## 6. Caveats & Edge Cases
**Known limitations**:
- Public ADTi pages list 2 fps continuous capture and -10..40 C operating range; project assumptions need manufacturer verification.
- Live camera interface is TBD.
- Total occlusion detection must be conservative: false positives cause temporary IMU-only degradation, while false negatives can feed unusable frames into VIO.
**Performance bottlenecks**:
- Full-resolution preprocessing must stay inside the <400 ms p95 pipeline budget.
## 7. Dependency Graph
**Must be implemented after**: none.
**Can be implemented in parallel with**: Tile Manager, MAVLink/GCS integration.
**Blocks**: VIO adapter, anchor verification, generated tile lifecycle.
## 8. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Camera unavailable or calibration missing | `camera_calibration_missing id=...` |
| WARN | Frame degraded or occluded | `frame_quality_degraded occlusion=total_occlusion blur=... texture=...` |
| INFO | Calibration loaded | `camera_calibration_loaded version=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment and health log.
_docs/02_document/components/01_camera_ingest_calibration/tests.md
+139
View File
@@ -0,0 +1,139 @@
# Test Specification — Camera Ingest And Calibration
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-2.1a | VO registration uses only normal usable frames | IT-01, AT-01 | Covered |
| AC-3.5 | Visual blackout switches to degraded mode quickly | IT-02, AT-02 | Covered |
| AC-4.1 | Capture-to-output latency budget | PT-01 | Covered |
| AC-4.2 | Jetson memory budget | PT-01 | Covered |
| AC-8.4 | Mid-flight tile generation input eligibility | IT-03 | Covered |
| AC-8.5 | No raw frame retention | ST-01 | Covered |
| AC-NEW-8 | Full blackout/occlusion degraded-mode trigger | IT-02, AT-02 | Covered |
## Blackbox Tests
### IT-01: Usable Frame Classification
**Summary**: Verify normal nadir frames are marked usable for VIO and anchor matching.
**Traces to**: AC-2.1a
**Input data**: Project still images plus calibration metadata.
**Expected result**: `FramePacket.quality.usable_for_vio=true`, `usable_for_anchor=true`, and `occlusion.status=clear` for normal daytime textured frames.
**Max execution time**: 100 ms per frame.
**Dependencies**: Calibration file, replay fixture.
---
### IT-02: Total Occlusion Gate Before VIO
**Summary**: Verify total occlusion is detected before BASALT receives the frame.
**Traces to**: AC-3.5, AC-NEW-8
**Input data**: Black/white/covered-lens frames, corrupt frame fixture, low-texture whiteout frames.
**Expected result**: `occlusion.status` is `total_occlusion` or `blackout`, `usable_for_vio=false`, `usable_for_anchor=false`, and a degradation signal is emitted to the safety wrapper.
**Max execution time**: 100 ms per frame.
**Dependencies**: Safety wrapper test double.
---
### IT-03: Tile Generation Eligibility Metadata
**Summary**: Verify frame metadata required for generated tiles is available without persisting raw frames.
**Traces to**: AC-8.4, AC-8.5
**Input data**: Valid frame, altitude hint, calibration, pose fixture.
**Expected result**: Frame metadata supports orthorectification handoff; raw frame is not retained after processing outside allowed FDR thumbnail exception.
**Max execution time**: 100 ms per frame.
**Dependencies**: Tile Manager test double.
## Performance Tests
### PT-01: Ingest And Quality Latency
**Summary**: Verify decode, calibration lookup, occlusion detection, and quality classification stay inside the system latency budget.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Input rate: target camera/replay rate.
- Duration: 30 minutes replay.
- Dataset: project still images plus synthetic occlusion frames.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Per-frame ingest p95 | <=100 ms | >150 ms |
| Memory contribution | <=1 GB | >1.5 GB |
| Dropped frames | <=10% under sustained load | >10% |
**Resource limits**: Jetson shared memory remains below system 8 GB cap.
## Security Tests
### ST-01: Raw Frame Retention Check
**Summary**: Verify normal operation does not persist raw navigation frames.
**Traces to**: AC-8.5
**Attack vector**: Sensitive raw imagery remains on disk after processing.
**Test procedure**:
1. Run replay with normal and failed tile-generation frames.
2. Inspect output directories and FDR artifacts.
**Expected behavior**: Only metadata, generated tiles, and allowed low-rate failed-frame thumbnails are retained.
**Pass criteria**: No raw full-resolution frames remain after teardown.
## Acceptance Tests
### AT-01: Normal Frame Enters VIO
**Summary**: Confirm normal usable frames are passed to BASALT.
**Traces to**: AC-2.1a
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Feed a calibrated normal frame | Occlusion status is `clear` |
| 2 | Process quality gate | Frame is emitted to VIO adapter |
---
### AT-02: Occluded Frame Bypasses VIO
**Summary**: Confirm full occlusion triggers IMU-only degraded path.
**Traces to**: AC-3.5, AC-NEW-8
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Feed total-occlusion frame | `usable_for_vio=false` |
| 2 | Observe downstream routing | BASALT is bypassed and safety wrapper receives degradation signal |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `project_60_still_images` | Normal nadir frames | `_docs/00_problem/input_data/` | Project data |
| `synthetic_occlusion_frames` | Black/white/corrupt/whiteout frames | Generated fixture | Small |
**Setup procedure**: Load calibration fixture and mount read-only frame data.
**Teardown procedure**: Remove run-scoped temp directories and verify no raw-frame persistence.
**Data isolation strategy**: Each run writes to a unique `test-results/<run-id>/` directory.
_docs/02_document/components/02_vio_adapter/description.md
+92
View File
@@ -0,0 +1,92 @@
# VIO Adapter
## 1. High-Level Overview
**Purpose**: Wrap the selected relative VIO backend as a replaceable component that consumes calibrated frames and FC IMU data, then emits relative pose/velocity/bias state and tracking quality.
**Architectural Pattern**: Adapter / anti-corruption layer.
**Upstream dependencies**: Camera ingest/calibration, MAVLink telemetry stream.
**Downstream consumers**: Safety/anchor wrapper, FDR, separate e2e test suite.
## 2. Internal Interfaces
### Interface: `VioAdapter`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `initialize` | `VioInitRequest` | `VioInitResult` | No | `CalibrationInvalid`, `DatasetUnsupported` |
| `process` | `VioInputPacket` | `VioStatePacket` | Yes | `TrackingLost`, `TimestampMismatch` |
| `health` | none | `VioHealth` | No | none |
**Input DTOs**:
```yaml
VioInputPacket:
frame: FramePacket
imu_samples: list[TelemetrySample]
attitude_sample: TelemetrySample optional
```
**Output DTOs**:
```yaml
VioStatePacket:
timestamp_ns: integer
relative_pose: transform
velocity: vector3
bias_estimate: object optional
tracking_quality: number
completed: boolean
covariance_hint: matrix optional
```
## 3. Data Access Patterns
No persistent production data ownership. Reads calibration/config at startup and emits state to downstream consumers.
## 4. Implementation Details
**State Management**: Owns selected VIO backend runtime state and resets only through explicit wrapper command.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| BASALT | Current selected relative visual-inertial odometry backend |
| Eigen/Sophus or backend-native math stack | Pose and transform representation |
**Error Handling Strategy**:
- Tracking loss is surfaced to the safety/anchor wrapper, not hidden.
- Timestamp/camera-IMU sync violations fail the packet and are logged.
- The adapter never emits WGS84 coordinates; absolute semantics belong to the wrapper.
## 5. Caveats & Edge Cases
**Known limitations**:
- BASALT has no special fixed-wing nadir mode; validation must prove fit under low-parallax/planar terrain.
- Backend covariance/confidence output is not the product authority; wrapper calibration is required.
**Performance bottlenecks**:
- Native VIO runtime and image resolution can exceed Jetson budget if not tuned.
## 6. Dependency Graph
**Must be implemented after**: Camera ingest/calibration, MAVLink telemetry DTO definitions.
**Can be implemented in parallel with**: Satellite Service, Tile Manager.
**Blocks**: Safety/anchor wrapper final integration.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | VIO backend initialization fails | `vio_init_failed reason=...` |
| WARN | Tracking quality drops | `vio_tracking_degraded quality=...` |
| INFO | VIO reset/reinitialized | `vio_reset cause=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment.
_docs/02_document/components/02_vio_adapter/tests.md
+141
View File
@@ -0,0 +1,141 @@
# Test Specification — VIO Adapter
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.3 | Drift between anchors and anchor-age reporting input | IT-02, AT-01 | Covered |
| AC-2.1a | >95% VO registration on normal segments | IT-01, AT-01 | Covered |
| AC-2.2 | <1.0 px VO MRE | IT-01 | Covered |
| AC-3.1 | Handles 350 m outliers/tilt | IT-03 | Covered |
| AC-3.2 | Sharp turns trigger relocalization path | IT-04 | Covered |
| AC-3.4 | Loss threshold feeds relocalization/dead reckoning | IT-04 | Covered |
| AC-4.1 | Hot-path latency | PT-01 | Covered |
| AC-4.2 | Jetson memory budget | PT-01 | Covered |
## Blackbox Tests
### IT-01: Public Dataset VIO Replay
**Summary**: Verify the VIO adapter produces relative motion for synchronized camera/IMU replay.
**Traces to**: AC-2.1a, AC-2.2
**Input data**: Derkachi cropped nadir video + `SCALED_IMU2` + `GLOBAL_POSITION_INT`, MUN-FRL preferred slice, or representative synchronized nav-camera + IMU + ground truth.
**Expected result**: VO registration succeeds for >95% of normal usable frames; frame-to-frame MRE <1.0 px where ground-truth/feature evaluation supports it. Derkachi runs are accepted as calibration-limited until intrinsics, distortion, and camera-to-body transform are pinned.
**Max execution time**: Dataset-dependent; report per-frame latency.
**Dependencies**: Camera ingest, MAVLink telemetry/replay, calibration fixtures.
---
### IT-02: Relative Drift Reporting
**Summary**: Verify adapter emits state needed for wrapper drift and anchor-age accounting.
**Traces to**: AC-1.3
**Input data**: Segment with two known satellite anchors and IMU samples.
**Expected result**: Adapter emits continuous `VioStatePacket` values with timestamps and quality, enabling wrapper to compare VO extrapolation to next anchor.
**Max execution time**: Dataset-dependent.
**Dependencies**: Safety wrapper test harness.
---
### IT-03: Tilt/Outlier Robustness
**Summary**: Verify adapter reports degraded tracking without false success under tilt/outlier cases.
**Traces to**: AC-3.1
**Input data**: Replay segment with synthetic +/-20 degree tilt and up to 350 m apparent outlier.
**Expected result**: Adapter either tracks with quality metadata or emits `TrackingLost`; it never hides a failure as high-quality VIO.
**Max execution time**: 15 minutes per fixture.
---
### IT-04: Sharp Turn / Loss Signal
**Summary**: Verify sharp turns and disconnected visual overlap produce wrapper-visible failure signals.
**Traces to**: AC-3.2, AC-3.4
**Input data**: <5% overlap sequence with heading change <70 degrees.
**Expected result**: Adapter emits low tracking quality or `TrackingLost` within the loss window, allowing relocalization trigger.
**Max execution time**: 10 minutes.
## Performance Tests
### PT-01: VIO Adapter Runtime Budget
**Summary**: Verify VIO processing does not consume the full <400 ms system p95 budget.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Input: Derkachi synchronized replay and public/representative replay.
- Duration: 30 minutes plus release long-run slice.
- Target: Jetson Orin Nano Super.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Adapter p95 latency | <=250 ms | >300 ms |
| Memory contribution | <=3 GB | >4 GB |
| Tracking failure on normal segments | <5% | >=5% |
**Resource limits**: Total system memory remains below 8 GB.
## Security Tests
### ST-01: Timestamp Injection Rejection
**Summary**: Verify malformed or non-monotonic timestamps do not produce trusted VIO state.
**Traces to**: AC-NEW-4
**Attack vector**: Replay or telemetry timestamp manipulation.
**Test procedure**:
1. Feed non-monotonic frame and IMU timestamps.
2. Observe adapter output.
**Expected behavior**: Adapter returns `TimestampMismatch` or low-quality failure; wrapper does not trust the state.
**Pass criteria**: No high-quality VIO state is emitted from malformed timing.
## Acceptance Tests
### AT-01: Normal VIO State Contract
**Summary**: Confirm adapter output contract supports downstream localization.
**Traces to**: AC-1.3, AC-2.1a
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Initialize with calibrated frame/IMU config | `VioInitResult` succeeds |
| 2 | Replay normal frames | `VioStatePacket` includes timestamp, relative pose, velocity, tracking quality |
| 3 | End segment | State stream is continuous enough for wrapper drift accounting |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `derkachi_video_telemetry` | Cropped nadir MP4 + synchronized IMU and `GLOBAL_POSITION_INT` trajectory | Project fixture | ~282 MB video + CSV |
| `public_nadir_vio_candidates` | MUN-FRL/ALTO/Kagaru/EPFL slices | Public pinned fixtures | Dataset-dependent |
| `representative_sync_replay` | Target camera + FC IMU + calibrated ground truth | Project collection | TBD |
**Setup procedure**: Pin calibration/extrinsics and mount read-only synchronized replay data.
**Teardown procedure**: Remove generated result reports and adapter temp state.
**Data isolation strategy**: One run directory per dataset slice and configuration hash.
_docs/02_document/components/03_safety_anchor_wrapper/description.md
+106
View File
@@ -0,0 +1,106 @@
# Safety And Anchor Wrapper
## 1. High-Level Overview
**Purpose**: Own the authoritative localization state, confidence calibration, source labels, anchor fusion, degraded modes, tile-write gates, and MAVLink output semantics.
**Architectural Pattern**: Stateful coordinator / safety facade.
**Upstream dependencies**: VIO adapter, anchor verification, MAVLink telemetry, camera quality reports.
**Downstream consumers**: MAVLink/GCS integration, FDR, Tile Manager, separate e2e test suite.
## 2. Internal Interfaces
### Interface: `LocalizationStateMachine`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `update_vio` | `VioStatePacket` | `PositionEstimate` | Yes | `StateInconsistent` |
| `consider_anchor` | `AnchorDecision` | `AnchorAcceptanceResult` | No | `AnchorRejected` |
| `degrade` | `DegradationSignal` | `PositionEstimate` | No | none |
| `propagate_imu_only` | `ImuOnlyPropagationRequest` | `PositionEstimate` | Yes | `InsufficientTelemetry` |
| `tile_write_eligibility` | `FramePacket` | `TileWriteDecision` | No | none |
**Input DTOs**:
```yaml
AnchorDecision:
candidate_id: string
timestamp_ns: integer
estimated_pose_wgs84: object
inlier_count: integer
mre_px: number
tile_freshness_status: enum
provenance_status: enum
DegradationSignal:
type: enum(total_occlusion, visual_blackout, gps_spoofing, covariance_growth, tracking_lost)
timestamp_ns: integer
ImuOnlyPropagationRequest:
last_trusted_estimate: PositionEstimate
imu_samples: list[TelemetrySample]
elapsed_blackout_ms: integer
reason: enum(total_occlusion, visual_blackout, tracking_lost)
```
**Output DTOs**:
```yaml
PositionEstimate:
timestamp_ns: integer
lat_deg: number
lon_deg: number
alt_msl_m: number
covariance_95_semi_major_m: number
source_label: enum(satellite_anchored, vo_extrapolated, dead_reckoned)
fix_type: integer
horiz_accuracy_m: number
last_satellite_anchor_age_ms: integer
```
## 3. Data Access Patterns
No direct tile/image storage ownership. Writes all decisions to FDR via observability component.
## 4. Implementation Details
**State Management**: Owns the authoritative state machine and covariance growth model.
**Error Handling Strategy**:
- Reject uncertain anchors by default.
- Never emit optimistic accuracy when confidence is degraded.
- On total occlusion or visual blackout, do not call VIO for that frame; propagate from the last trusted state with IMU-only dynamics, set `source_label=dead_reckoned`, and grow covariance monotonically.
- If covariance or blackout thresholds exceed AC limits, emit no-fix/failsafe semantics.
- Treat cache freshness and provenance as evidence carried by `AnchorDecision`; do not call the Tile Manager directly during anchor acceptance.
## 5. Caveats & Edge Cases
**Known limitations**:
- Final covariance calibration requires representative synchronized data.
- The wrapper must stay independent of BASALT internals so VIO can be replaced.
- IMU-only propagation is an emergency bridge, not a reliable long-duration localization mode; the spec requires failsafe/no-fix once time or covariance thresholds are exceeded.
**Potential race conditions**:
- A delayed anchor result must be checked against current timestamp/state before acceptance.
## 6. Dependency Graph
**Must be implemented after**: VIO DTOs, anchor DTOs, MAVLink output contract.
**Can be implemented in parallel with**: FDR schema after DTOs stabilize.
**Blocks**: MAVLink production output, tile-write lifecycle, end-to-end validation.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | State invariant violation | `localization_state_inconsistent reason=...` |
| WARN | Anchor rejected or mode degraded | `anchor_rejected reason=mahalanobis_gate` |
| INFO | Mode transition | `source_label_changed from=vo_extrapolated to=dead_reckoned reason=total_occlusion` |
**Log format**: FDR structured event.
**Log storage**: FDR segment and QGC status for operator-critical events.
_docs/02_document/components/03_safety_anchor_wrapper/tests.md
+208
View File
@@ -0,0 +1,208 @@
# Test Specification — Safety And Anchor Wrapper
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.1 | >=80% within 50 m | AT-01 | Covered |
| AC-1.2 | >=50% within 20 m | AT-01 | Covered |
| AC-1.3 | Drift and anchor age | IT-01 | Covered |
| AC-1.4 | Quantitative confidence + label | IT-02, AT-02 | Covered |
| AC-3.4 | Relocalization after no-position window | IT-03 | Covered |
| AC-3.5 | Blackout to dead reckoned | IT-04, AT-03 | Covered |
| AC-4.3 | GPS_INPUT semantics source fields | IT-05 | Covered |
| AC-4.4 | Frame-by-frame streaming | PT-01 | Covered |
| AC-4.5 | Correction updates | IT-06 | Covered |
| AC-5.1 | Initialize from FC state | IT-07 | Covered |
| AC-5.2 | >3 s no-estimate fallback | IT-03 | Covered |
| AC-5.3 | Reinitialize after reboot | IT-07 | Covered |
| AC-NEW-2 | Spoofing promotion <3 s | IT-05 | Covered |
| AC-NEW-4 | False-position safety budget | ST-01, AT-02 | Covered |
| AC-NEW-8 | IMU-only blackout thresholds | IT-04, AT-03 | Covered |
## Blackbox Tests
### IT-01: Drift And Anchor Age Accounting
**Summary**: Verify VO extrapolation drift and anchor age are tracked per estimate.
**Traces to**: AC-1.3
**Input data**: VIO state stream with two accepted anchor decisions.
**Expected result**: Every `PositionEstimate` includes `last_satellite_anchor_age_ms`; drift at next anchor is measured and binned by age.
**Max execution time**: 5 minutes.
---
### IT-02: Confidence Output Contract
**Summary**: Verify every estimate has covariance and source label.
**Traces to**: AC-1.4
**Input data**: satellite-anchored, VO-extrapolated, and dead-reckoned state fixtures.
**Expected result**: Each output contains `covariance_95_semi_major_m`, `source_label`, `fix_type`, and `horiz_accuracy_m`; `horiz_accuracy_m` never under-reports covariance.
**Max execution time**: 2 minutes.
---
### IT-03: No-Position Relocalization Trigger
**Summary**: Verify no position for >=3 frames and >=2 s triggers relocalization/degraded behavior.
**Traces to**: AC-3.4, AC-5.2
**Input data**: VIO loss sequence with no accepted anchors.
**Expected result**: Relocalization request is emitted and wrapper continues dead reckoning until fail threshold.
**Max execution time**: 5 minutes.
---
### IT-04: Total Blackout IMU-Only Propagation
**Summary**: Verify total occlusion produces honest IMU-only estimates and failsafe thresholds.
**Traces to**: AC-3.5, AC-NEW-8
**Input data**: Last trusted estimate, IMU/attitude/airspeed/altitude stream, total-occlusion signal for 5 s, 15 s, and 35 s.
**Expected result**: Wrapper emits `dead_reckoned` within <=1 frame or <=400 ms, covariance grows monotonically, and no-fix/failsafe is emitted when blackout >30 s or covariance >500 m.
**Max execution time**: 10 minutes.
---
### IT-05: Spoofing Promotion And GPS_INPUT Mapping
**Summary**: Verify wrapper promotes own estimate and maps output fields correctly under GPS spoofing.
**Traces to**: AC-4.3, AC-NEW-2
**Input data**: Plane SITL spoofing telemetry and trusted wrapper estimate.
**Expected result**: Own estimate is promoted within <3 s; v1 emits `GPS_INPUT` only; source labels and accuracy fields are correct.
**Max execution time**: 10 minutes.
---
### IT-06: Correction Update
**Summary**: Verify refined estimates can correct previous positions.
**Traces to**: AC-4.5
**Input data**: VO estimate followed by accepted satellite anchor for same segment.
**Expected result**: Wrapper emits updated estimate/correction with improved source label and covariance.
**Max execution time**: 5 minutes.
---
### IT-07: Initialization And Reboot Recovery
**Summary**: Verify wrapper initializes from FC state and can reinitialize after reboot.
**Traces to**: AC-5.1, AC-5.3
**Input data**: FC EKF position, IMU-extrapolated state, simulated companion restart.
**Expected result**: Wrapper resumes from current FC state and reports degraded confidence until re-anchored.
**Max execution time**: 5 minutes.
## Performance Tests
### PT-01: Wrapper Streaming Overhead
**Summary**: Verify state-machine processing does not delay frame-by-frame output.
**Traces to**: AC-4.4
**Load scenario**:
- Input: 3 Hz estimate stream with anchor and blackout events.
- Duration: 8-hour synthetic run.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Wrapper p95 processing | <=25 ms | >50 ms |
| Missed frame outputs | 0 except skip-allowed upstream drops | Any silent batch/delay |
**Resource limits**: Negligible memory growth across run.
## Security Tests
### ST-01: False Anchor Rejection
**Summary**: Verify impossible anchors do not become trusted estimates.
**Traces to**: AC-NEW-4
**Attack vector**: Anchor decision with plausible inliers but impossible jump or stale provenance.
**Test procedure**:
1. Feed an anchor >1 km from predicted state with low covariance.
2. Feed stale/provenance-failed anchor evidence.
**Expected behavior**: Anchor is rejected, FDR logs reason, source label remains degraded or VO extrapolated.
**Pass criteria**: 0 accepted impossible/stale anchors.
## Acceptance Tests
### AT-01: Position Accuracy Aggregation
**Summary**: Verify wrapper outputs can be scored against frame-center thresholds.
**Traces to**: AC-1.1, AC-1.2
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Replay mapped frame-center estimates | Report computes >=80% within 50 m and >=50% within 20 m |
| 2 | Include source labels | Accuracy is binned by source label |
---
### AT-02: Confidence Honesty
**Summary**: Verify reported confidence is conservative relative to measured error.
**Traces to**: AC-1.4, AC-NEW-4
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Replay ground-truth trajectory | Measured error is not systematically above reported covariance |
| 2 | Inject over-confident anchors | Mahalanobis gate rejects them |
---
### AT-03: Blackout Failsafe
**Summary**: Verify operator-visible blackout/failsafe behavior.
**Traces to**: AC-3.5, AC-NEW-8
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Inject total visual blackout | `dead_reckoned` within <=400 ms |
| 2 | Continue >30 s or covariance >500 m | `fix_type=0`, `horiz_accuracy=999.0`, QGC failsafe status |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `wrapper_state_fixtures` | VIO states, anchors, blackouts, spoofing signals | Generated fixtures | Small |
| `representative_replay` | Target synchronized replay and ground truth | Project collection | TBD |
**Setup procedure**: Load deterministic VIO/anchor/telemetry fixtures and run with isolated PostgreSQL schema.
**Teardown procedure**: Drop run schema and remove FDR segments.
**Data isolation strategy**: Use per-run mission IDs and database schemas.
_docs/02_document/components/04_satellite_retrieval/description.md
+102
View File
@@ -0,0 +1,102 @@
# Satellite Service
## 1. High-Level Overview
**Purpose**: Own the onboard boundary to the suite Satellite Service: import pre-flight mission cache packages, upload generated-tile packages after flight, and convert query frames into ranked local VPR candidates using preloaded DINOv2-VLAD descriptors and FAISS.
**Architectural Pattern**: Offline sync gateway + local retrieval index adapter.
**Upstream dependencies**: Camera ingest/calibration, Tile Manager, safety/anchor wrapper, Azaion Suite Satellite Service before/after flight.
**Downstream consumers**: Anchor verification, FDR.
## 2. Internal Interfaces
### Interface: `SatelliteService`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `import_mission_cache` | `CacheImportRequest` | `CacheImportResult` | Yes | `SyncUnavailable`, `PackageInvalid` |
| `upload_generated_tiles` | `GeneratedTileUploadRequest` | `GeneratedTileUploadResult` | Yes | `SyncUnavailable`, `PackageRejected` |
| `retrieve` | `RetrievalRequest` | `RetrievalResult` | Yes | `IndexUnavailable`, `DescriptorFailed` |
| `load_index` | `IndexLoadRequest` | `IndexStatus` | No | `ManifestInvalid`, `IndexUnavailable` |
**Input DTOs**:
```yaml
RetrievalRequest:
frame: FramePacket
prior_estimate: PositionEstimate optional
search_radius_m: number optional
max_candidates: integer
```
**Output DTOs**:
```yaml
RetrievalResult:
query_descriptor_id: string
candidates: list[VprCandidate]
VprCandidate:
chunk_id: string
tile_id: string
score: number
footprint: geometry
freshness_status: enum
```
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Top-K FAISS search | Triggered only | No steady-state | FAISS index |
| Import/export package sync | Pre-flight / post-flight only | No mid-flight | Package manifest and sidecar hashes |
| Load chunk metadata | Per candidate | No | PostgreSQL/PostGIS spatial and chunk indexes |
## 4. Implementation Details
**State Management**: Holds loaded descriptor model and FAISS index handles; tracks pre-flight import and post-flight upload package status.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| DINOv2 / ONNX / TensorRT candidate path | Query descriptor extraction |
| FAISS CPU | Top-K retrieval |
| Satellite Service client | Pre-flight cache import and post-flight generated-tile upload |
**Error Handling Strategy**:
- If descriptor extraction or index load fails, return no candidates and trigger degraded mode.
- Optimized engines are allowed only after descriptor-fidelity tests pass.
- Network/package sync failures are allowed only before takeoff or after landing; during flight, the component must never call a satellite provider or suite service.
## 5. Caveats & Edge Cases
**Known limitations**:
- VPR result is only a candidate, never an accepted fix.
- Cross-domain retrieval can be wrong under seasonal, lighting, or terrain ambiguity.
- External Satellite Service availability cannot be part of the mid-flight localization safety case.
**Performance bottlenecks**:
- Descriptor extraction on Jetson must be trigger-limited and profiled separately from BASALT.
## 6. Dependency Graph
**Must be implemented after**: cache manifest/index schema, camera frame DTOs.
**Can be implemented in parallel with**: anchor verification.
**Blocks**: satellite relocalization flow.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Index unavailable | `faiss_index_unavailable id=...` |
| WARN | No candidates | `vpr_no_candidates frame_id=...` |
| INFO | Retrieval invoked | `vpr_query candidates=... latency_ms=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment.
_docs/02_document/components/04_satellite_retrieval/tests.md
+172
View File
@@ -0,0 +1,172 @@
# Test Specification — Satellite Service
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-3.2 | Sharp-turn relocalization | IT-02, AT-01 | Covered |
| AC-3.3 | >=3 disconnected segments | IT-03 | Covered |
| AC-3.4 | Relocalization request after loss | IT-02 | Covered |
| AC-4.1 | Trigger path latency | PT-01 | Covered |
| AC-4.2 | Memory budget | PT-01 | Covered |
| AC-8.1 | Cache imagery interface | IT-01 | Covered |
| AC-8.3 | Preloaded/preprocessed cache | IT-01 | Covered |
| AC-8.6 | VPR chunks, multi-scale, dynamic K | IT-01, IT-04 | Covered |
| AC-NEW-1 | Cold-start first fix support | PT-02 | Covered |
| AC-NEW-6 | Freshness-aware retrieval | IT-04, ST-01 | Covered |
## Blackbox Tests
### IT-01: Index Load And Chunk Coverage
**Summary**: Verify preloaded VPR chunks and FAISS index cover the operational area.
**Traces to**: AC-8.1, AC-8.3, AC-8.6
**Input data**: PostgreSQL/PostGIS cache manifest, VPR chunk metadata, FAISS index.
**Expected result**: Every test frame footprint falls inside at least one VPR chunk; fine and coarse descriptors are present where required.
**Max execution time**: 2 minutes per mission cache.
---
### IT-02: Sharp-Turn Local Retrieval Trigger
**Summary**: Verify sharp-turn state requests candidates rather than relying on frame-to-frame VO.
**Traces to**: AC-3.2, AC-3.4
**Input data**: Wrapper relocalization request with sharp-turn/loss reason.
**Expected result**: Satellite Service returns bounded top-K candidates from preloaded local indexes based on sector/covariance policy.
**Max execution time**: 2 seconds per query.
---
### IT-03: Disconnected Segment Retrieval
**Summary**: Verify at least three disconnected segments can each retrieve candidate chunks.
**Traces to**: AC-3.3
**Input data**: Three disconnected query frames with approximate prior/covariance.
**Expected result**: Each query returns a candidate set including the ground-truth region when covered by the cache fixture.
**Max execution time**: Dataset-dependent.
---
### IT-04: Dynamic K And Freshness Filter
**Summary**: Verify K varies by sector and covariance, and stale candidates are tagged.
**Traces to**: AC-8.6, AC-NEW-6
**Input data**: Stable and active-conflict sector cache fixtures with fresh/stale tiles.
**Expected result**: K=5 for stable low-covariance, K=20 for active-conflict, K=50 fallback; stale candidates are flagged for rejection/down-confidence.
**Max execution time**: 2 seconds per query.
## Performance Tests
### PT-01: Retrieval Query Runtime
**Summary**: Verify descriptor extraction and FAISS query fit trigger-path budget.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Query set: 100 representative relocalization frames.
- Environment: Jetson and replay workstation.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Retrieval p95 | <=300 ms trigger path share | >400 ms |
| Memory contribution | <=2 GB | >3 GB |
| Candidate count policy | Exact | Any wrong K |
**Resource limits**: Total system memory remains below 8 GB.
---
### PT-02: Cold-Start Index Load
**Summary**: Verify retrieval readiness supports first fix <30 s.
**Traces to**: AC-NEW-1
**Load scenario**:
- Cold boot 50 runs.
- Cache/index mounted locally.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Index ready p95 | <=10 s | >15 s |
| First retrieval p95 | Fits <30 s first-fix budget | Exceeds budget |
## Security Tests
### ST-01: Stale Candidate Handling
**Summary**: Verify stale imagery cannot silently appear as a trusted retrieval candidate.
**Traces to**: AC-NEW-6
**Attack vector**: Manipulated cache manifest marks stale tile as available.
**Test procedure**:
1. Load cache fixture with stale capture dates.
2. Query against stale region.
**Expected behavior**: Candidate carries stale status; anchor path cannot accept it as `satellite_anchored`.
**Pass criteria**: 0 stale candidates without explicit stale/down-confidence metadata.
---
### ST-02: No Mid-Flight Satellite Service Calls
**Summary**: Verify relocalization never performs satellite-provider or suite Satellite Service network calls during flight.
**Traces to**: AC-8.3, R-SAT-01
**Attack vector**: Runtime attempts to fetch missing cache/index data over the network during relocalization.
**Test procedure**:
1. Disable external network access during a replay scenario.
2. Trigger relocalization against preloaded cache fixtures.
3. Inspect network call logs and Satellite Service client telemetry.
**Expected behavior**: Retrieval uses only mounted local cache/index data; missing data produces degraded/no-candidate behavior, not a network fetch.
**Pass criteria**: 0 mid-flight Satellite Service or satellite-provider calls.
## Acceptance Tests
### AT-01: Relocalization Candidate Returned
**Summary**: Verify a relocalization request returns usable candidates for anchor verification.
**Traces to**: AC-3.2, AC-8.6
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Submit sharp-turn query | Retrieval invokes VPR |
| 2 | Read output | Candidate list includes chunk IDs, tile IDs, scores, footprints, freshness |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `cache_vpr_fixture` | PostGIS manifest, COGs, descriptors, FAISS index | Generated/cache fixture | Mission-dependent |
| `aerial_vpr_queries` | Aerial query frames with ground-truth regions | ALTO/AerialVL/representative | Dataset-dependent |
**Setup procedure**: Restore isolated PostgreSQL schema and mount read-only descriptor/index files.
**Teardown procedure**: Drop schema and remove generated retrieval reports.
**Data isolation strategy**: Per-run schema and read-only cache fixture volume.
_docs/02_document/components/05_anchor_verification/description.md
+93
View File
@@ -0,0 +1,93 @@
# Anchor Verification
## 1. High-Level Overview
**Purpose**: Verify retrieved cache candidates with local feature matching and geometric checks before the safety wrapper considers an absolute anchor.
**Architectural Pattern**: Validation pipeline.
**Upstream dependencies**: Satellite Service, camera ingest/calibration, Tile Manager.
**Downstream consumers**: Safety/anchor wrapper, FDR.
## 2. Internal Interfaces
### Interface: `AnchorVerifier`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `verify` | `AnchorVerificationRequest` | `AnchorDecision` | Yes | `TileUnavailable`, `MatchFailed`, `GeometryFailed` |
| `benchmark_matcher` | `MatcherBenchmarkRequest` | `MatcherBenchmarkReport` | Yes | `ModelUnavailable` |
**Input DTOs**:
```yaml
AnchorVerificationRequest:
frame: FramePacket
candidates: list[VprCandidate]
matcher_profile: enum(aliked, disk, sift_orb_baseline)
```
**Output DTOs**:
```yaml
AnchorDecision:
candidate_id: string
accepted_by_geometry: boolean
estimated_pose_wgs84: object optional
inlier_count: integer
mre_px: number
homography: matrix optional
rejection_reason: string optional
```
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Read candidate COG footprint/window | Triggered only | No | Tile spatial metadata |
| Read matcher model | Startup | No | No |
## 4. Implementation Details
**State Management**: Loads matcher/extractor models and tracks benchmark-selected profile.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| ALIKED/DISK + LightGlue | Learned local matching |
| OpenCV | RANSAC/USAC geometry and error metrics |
**Error Handling Strategy**:
- Low inlier count, high MRE, stale tile, or provenance failure returns a rejected decision with reason.
- SuperPoint can be benchmarked only if legal approval allows its license use.
## 5. Caveats & Edge Cases
**Known limitations**:
- ALIKED-LightGlue is not VIO by itself; it supplies correspondences. A full VIO path still needs state estimation and IMU fusion.
- Optional frame-to-frame VO fallback must be benchmarked separately from cross-domain anchor verification.
**Performance bottlenecks**:
- Learned matching can exceed the Jetson budget if run per-frame; default invocation is trigger-based.
## 6. Dependency Graph
**Must be implemented after**: Satellite Service candidate DTOs, Tile Manager tile access.
**Can be implemented in parallel with**: VIO adapter.
**Blocks**: accepted satellite-anchor path.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Matcher model unavailable | `lightglue_model_unavailable profile=aliked` |
| WARN | Candidate rejected | `anchor_geometry_rejected mre_px=... inliers=...` |
| INFO | Anchor verified | `anchor_verified tile_id=... mre_px=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment.
_docs/02_document/components/05_anchor_verification/tests.md
+124
View File
@@ -0,0 +1,124 @@
# Test Specification — Anchor Verification
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.1 | 50 m frame-center accuracy via accepted anchors | AT-01 | Covered |
| AC-1.2 | 20 m stretch accuracy via accepted anchors | AT-01 | Covered |
| AC-2.1b | Satellite-anchor registration measured separately | IT-01 | Covered |
| AC-2.2 | <2.5 px cross-domain MRE | IT-01, AT-01 | Covered |
| AC-3.1 | Outlier rejection | ST-01 | Covered |
| AC-4.1 | Trigger-path latency | PT-01 | Covered |
| AC-4.2 | Memory budget | PT-01 | Covered |
| AC-NEW-4 | False-position safety budget | ST-01 | Covered |
| AC-NEW-6 | Stale imagery rejection evidence | IT-02, ST-02 | Covered |
## Blackbox Tests
### IT-01: Cross-Domain Match And RANSAC
**Summary**: Verify ALIKED/DISK-LightGlue plus RANSAC produces measurable anchor evidence.
**Traces to**: AC-2.1b, AC-2.2
**Input data**: UAV frame, retrieved COG candidate window, ground-truth georegistration.
**Expected result**: Accepted anchors have MRE <2.5 px, sufficient inliers, and homography/pose evidence.
**Max execution time**: 2 seconds per candidate set.
---
### IT-02: Stale Candidate Verification
**Summary**: Verify stale or provenance-failed candidates are rejected or marked unsafe.
**Traces to**: AC-NEW-6
**Input data**: Candidate list with stale tile metadata and valid-looking image content.
**Expected result**: `AnchorDecision` is rejected or carries stale/provenance failure; safety wrapper cannot accept it as trusted.
**Max execution time**: 2 seconds per candidate.
## Performance Tests
### PT-01: Local Matcher Runtime
**Summary**: Verify learned matching stays bounded on Jetson when invoked.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Candidate sets: K=5, K=20, K=50.
- Matcher profiles: ALIKED, DISK, SIFT/ORB baseline.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| K=5 p95 | <=300 ms | >500 ms |
| K=20 p95 | Reported and bounded | Unbounded/no report |
| Memory contribution | <=2 GB | >3 GB |
**Resource limits**: Total Jetson shared memory remains below 8 GB.
## Security Tests
### ST-01: False Match / Impossible Jump Rejection
**Summary**: Verify visually plausible but geographically impossible matches are rejected.
**Traces to**: AC-3.1, AC-NEW-4
**Attack vector**: Candidate tile from wrong region with repetitive texture.
**Test procedure**:
1. Pair query with wrong-region candidate.
2. Run matcher and RANSAC.
3. Pass result to safety wrapper fixture.
**Expected behavior**: Anchor is rejected by geometry or downstream consistency gates.
**Pass criteria**: 0 accepted anchors from wrong-region fixtures.
---
### ST-02: Provenance Failure
**Summary**: Verify unsigned/hash-failed candidate tiles cannot produce accepted anchors.
**Traces to**: AC-NEW-6
**Attack vector**: Tampered COG or sidecar.
**Test procedure**: Run verification against tampered tile fixture.
**Expected behavior**: `AnchorDecision` includes provenance failure and is not accepted.
**Pass criteria**: 0 trusted anchors from tampered tiles.
## Acceptance Tests
### AT-01: Accepted Anchor Accuracy
**Summary**: Verify accepted anchors support position accuracy thresholds.
**Traces to**: AC-1.1, AC-1.2, AC-2.2
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Verify satellite candidate against query | MRE <2.5 px |
| 2 | Convert accepted anchor to WGS84 evidence | Error supports 50 m / 20 m aggregate thresholds |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `anchor_match_fixture` | Query frames, COG windows, expected georegistration | ALTO/AerialVL/project cache | Dataset-dependent |
| `tampered_tile_fixture` | Hash/signature/stale cases | Generated fixture | Small |
**Setup procedure**: Load cache fixture and matcher model profile.
**Teardown procedure**: Remove matcher outputs and reports.
**Data isolation strategy**: Read-only imagery with per-run output folders.
_docs/02_document/components/06_cache_tile_lifecycle/description.md
+92
View File
@@ -0,0 +1,92 @@
# Tile Manager
## 1. High-Level Overview
**Purpose**: Manage local tiles: service-source COGs, manifests, descriptor metadata, freshness/provenance checks, nadir-image orthorectification into generated tiles, generated tile writes, and post-flight package preparation.
**Architectural Pattern**: Repository + policy gate.
**Upstream dependencies**: Satellite Service cache packages, safety/anchor wrapper, camera ingest/calibration.
**Downstream consumers**: Satellite Service, anchor verification, FDR, post-flight sync.
## 2. Internal Interfaces
### Interface: `TileManager`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `validate_cache` | `CacheValidationRequest` | `CacheValidationReport` | No | `ManifestInvalid`, `SignatureInvalid` |
| `get_tile_window` | `TileWindowRequest` | `TileWindow` | No | `TileUnavailable`, `TileRejected` |
| `orthorectify_frame` | `TileGenerationRequest` | `GeneratedTileCandidate` | Yes | `TileWriteRejected`, `FrameNotUsable` |
| `write_generated_tile` | `GeneratedTileRequest` | `GeneratedTileRecord` | Yes | `TileWriteRejected`, `StorageFull` |
| `package_sync` | `SyncPackageRequest` | `SyncPackage` | Yes | `PackageFailed` |
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Tile by footprint/time/freshness | Per retrieval/anchor | Yes during relocalization | Spatial/time indexes |
| Descriptor metadata by chunk | Per Satellite Service retrieval | Yes during relocalization | Chunk ID index |
| Generated tile by mission/sector | Post-flight | No | Mission ID index |
### Caching Strategy
| Data | Cache Type | TTL | Invalidation |
|------|------------|-----|--------------|
| Manifest metadata | PostgreSQL/PostGIS query cache / process cache | Mission duration | New mission cache load |
| Sidecar verification | In-memory result cache | Mission duration | File hash change |
### Storage Estimates
| Table/Collection | Est. Row Count | Row Size | Total Size | Growth Rate |
|------------------|----------------|----------|------------|-------------|
| Cache manifest tiles | Mission-dependent | Small metadata | Within ~10 GB package with imagery | Per mission |
| Generated tiles | Flight-dependent | Metadata + COG payload | FDR/cache budget constrained | Per flight |
## 4. Implementation Details
**State Management**: Owns PostgreSQL/PostGIS manifest connection, sidecar verification state, and generated tile staging area.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| PostgreSQL + PostGIS | Manifest, spatial metadata, freshness queries, and generated-tile metadata |
| GDAL/rasterio candidate | COG read/write |
| OpenCV/GDAL geometry utilities | Nadir-frame orthorectification into generated COG tiles |
| Cryptographic hash/signature library | Sidecar validation |
**Error Handling Strategy**:
- Invalid signatures/hashes reject tiles.
- Storage-full blocks generated tile writes without affecting localization output.
- Cache validation failure blocks mission cache usage.
## 5. Caveats & Edge Cases
**Known limitations**:
- JSON-only manifests are avoided for scale and queryability, but signed JSON sidecars remain required for audit/interchange.
- PostgreSQL/PostGIS must be available locally before flight; runtime cannot depend on a remote DB link.
**Potential race conditions**:
- Generated tile and PostgreSQL manifest update must be atomic enough to avoid orphan trusted metadata.
## 6. Dependency Graph
**Must be implemented after**: data model schema decisions.
**Can be implemented in parallel with**: camera ingest, MAVLink integration.
**Blocks**: Satellite Service retrieval, anchor verification, generated tile lifecycle.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Cache package invalid | `cache_manifest_invalid reason=signature` |
| WARN | Tile rejected | `tile_rejected reason=stale tile_id=...` |
| INFO | Generated tile staged | `generated_tile_written tile_id=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment plus cache validation report.
_docs/02_document/components/06_cache_tile_lifecycle/tests.md
+167
View File
@@ -0,0 +1,167 @@
# Test Specification — Tile Manager
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-4.2 | Memory/storage pressure | PT-01 | Covered |
| AC-8.1 | Resolution at cache interface | IT-01 | Covered |
| AC-8.2 | Freshness thresholds | IT-02, ST-01 | Covered |
| AC-8.3 | Preloaded/preprocessed offline cache | IT-01 | Covered |
| AC-8.4 | Mid-flight tile generation/write-back | IT-03, AT-01 | Covered |
| AC-8.5 | Persistent imagery policy | ST-02 | Covered |
| AC-8.6 | VPR chunk metadata | IT-04 | Covered |
| AC-NEW-3 | FDR/tile storage cap interaction | PT-01 | Covered |
| AC-NEW-6 | Imagery freshness enforcement | IT-02, ST-01 | Covered |
| AC-NEW-7 | Cache-poisoning safety budget | ST-03, AT-01 | Covered |
## Blackbox Tests
### IT-01: Mission Cache Validation
**Summary**: Verify preloaded COGs, PostGIS metadata, sidecars, descriptors, and indexes validate before flight.
**Traces to**: AC-8.1, AC-8.3
**Input data**: Mission cache package with COGs, signed JSON sidecars, PostGIS manifest seed, FAISS index files.
**Expected result**: Valid cache passes resolution, hash, signature, descriptor-reference, and spatial coverage checks.
**Max execution time**: 5 minutes per cache fixture.
---
### IT-02: Freshness Gate
**Summary**: Verify active-conflict and stable-rear freshness rules.
**Traces to**: AC-8.2, AC-NEW-6
**Input data**: Tiles at fresh, grace, and stale ages for both sector classes.
**Expected result**: Fresh tiles pass, grace tiles are down-confidence weighted if allowed, stale tiles are rejected and cannot emit `satellite_anchored`.
**Max execution time**: 2 minutes.
---
### IT-03: Generated Tile Write
**Summary**: Verify nadir frames are orthorectified and written as generated tiles only when pose and frame quality gates pass.
**Traces to**: AC-8.4
**Input data**: Frame metadata, pose covariance <=3 m, <=5 m, and >5 m.
**Expected result**: <=3 m writes full-quality candidate, 3-5 m writes soft candidate, >5 m rejects write.
**Max execution time**: 2 minutes.
---
### IT-04: VPR Chunk Metadata
**Summary**: Verify chunk metadata supports retrieval rules.
**Traces to**: AC-8.6
**Input data**: Operational-area cache manifest.
**Expected result**: Chunks are 600-800 m equivalent footprint with 40-50% overlap and multi-scale active-sector descriptors.
**Max execution time**: 2 minutes.
## Performance Tests
### PT-01: Cache And FDR Storage Budget
**Summary**: Verify cache metadata and generated tile writes stay within storage/memory budgets.
**Traces to**: AC-4.2, AC-NEW-3
**Load scenario**:
- Mission cache: up to operational budget.
- Generated tiles: 8-hour synthetic flight.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Persistent cache | <=10 GB unless split budget approved | >budget without report |
| FDR + generated artifacts | <=64 GB per flight | >64 GB without rollover |
| DB query p95 | <=50 ms for indexed tile lookup | >150 ms |
**Resource limits**: PostgreSQL/PostGIS stays within total system 8 GB memory budget.
## Security Tests
### ST-01: Signed Manifest Enforcement
**Summary**: Verify unsigned/tampered/stale manifests are rejected.
**Traces to**: AC-8.2, AC-NEW-6
**Attack vector**: Tampered sidecar, bad hash, unsigned manifest.
**Test procedure**: Load invalid cache variants and run validation.
**Expected behavior**: Invalid tiles are rejected and logged.
**Pass criteria**: 0 invalid cache entries become available to retrieval/anchor verification.
---
### ST-02: Raw Frame Persistence Check
**Summary**: Verify Tile Manager persists tiles, not raw frames.
**Traces to**: AC-8.5
**Attack vector**: Raw frames accidentally stored as generated artifacts.
**Test procedure**: Run tile generation and inspect cache/FDR outputs.
**Expected behavior**: Only COG tiles, sidecars, manifests, and allowed failed-frame thumbnails exist.
**Pass criteria**: No raw full-resolution frames retained.
---
### ST-03: Cache Poisoning Gate
**Summary**: Verify misaligned generated tiles cannot become trusted basemap.
**Traces to**: AC-NEW-7
**Attack vector**: Over-confident pose writes misaligned generated tile.
**Test procedure**: Inject deflated covariance and wrong pose during tile write.
**Expected behavior**: Tile is rejected or marked candidate/soft; never promoted to trusted by onboard component.
**Pass criteria**: 0 direct trusted basemap promotions onboard.
## Acceptance Tests
### AT-01: Generated Tile Package For Satellite Service
**Summary**: Verify post-flight sync package contains valid generated tiles and metadata.
**Traces to**: AC-8.4, AC-NEW-7
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Orthorectify and write generated candidate tile | COG + sidecar + PostGIS manifest row created |
| 2 | Package post-flight sync | Manifest delta includes trust level and parent covariance |
| 3 | Inspect package | No tile is marked trusted basemap by onboard runtime |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `cache_integrity_fixtures` | Valid/stale/unsigned/hash-mismatched manifests | Generated fixture | Small |
| `mission_cache_fixture` | COGs, descriptors, PostGIS seed | Satellite Service stub | Mission-dependent |
**Setup procedure**: Restore isolated PostgreSQL/PostGIS schema and mount cache fixture read-only except generated-tile staging.
**Teardown procedure**: Drop schema and delete generated staging volume.
**Data isolation strategy**: Per-run mission ID, schema, and staging directory.
_docs/02_document/components/07_mavlink_gcs_integration/description.md
+69
View File
@@ -0,0 +1,69 @@
# MAVLink And GCS Integration
## 1. High-Level Overview
**Purpose**: Subscribe to flight-controller telemetry, emit `GPS_INPUT`, and send downsampled QGroundControl status/failsafe messages.
**Architectural Pattern**: Protocol adapter.
**Upstream dependencies**: ArduPilot Plane FC, safety/anchor wrapper.
**Downstream consumers**: VIO adapter, safety/anchor wrapper, QGC, FDR.
## 2. Internal Interfaces
### Interface: `MavlinkGateway`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `subscribe_telemetry` | `TelemetrySubscriptionRequest` | `TelemetrySample` | Yes | `MavlinkDisconnected` |
| `emit_gps_input` | `PositionEstimate` | `EmitResult` | Yes | `MavlinkDisconnected`, `InvalidGpsInput` |
| `emit_status` | `GcsStatusMessage` | `EmitResult` | Yes | `MavlinkDisconnected` |
## 3. Data Access Patterns
No persistent data ownership; telemetry and emitted packets are mirrored to FDR.
## 4. Implementation Details
**State Management**: Maintains MAVLink connection status, source/system IDs, and rate limiters for QGC status.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| MAVSDK | Telemetry subscriptions |
| pymavlink | Exact `GPS_INPUT` field emission |
**Error Handling Strategy**:
- Invalid `GPS_INPUT` fields are rejected before emission.
- Connection loss is surfaced to wrapper/FDR and does not silently drop safety events.
## 5. Caveats & Edge Cases
**Known limitations**:
- v1 emits `GPS_INPUT` only, not velocity-target navigation commands.
- Plane parameter configuration must be validated in SITL before hardware use.
**Performance bottlenecks**:
- Status text must be rate-limited to avoid telemetry noise.
## 6. Dependency Graph
**Must be implemented after**: position estimate DTO and MAVLink output contract.
**Can be implemented in parallel with**: Tile Manager, camera ingest.
**Blocks**: SITL integration and production FC output.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | MAVLink disconnected | `mavlink_disconnected endpoint=...` |
| WARN | Invalid output rejected | `gps_input_invalid reason=...` |
| INFO | FC link established | `mavlink_connected system_id=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment and optional tlog.
_docs/02_document/components/07_mavlink_gcs_integration/tests.md
+176
View File
@@ -0,0 +1,176 @@
# Test Specification — MAVLink And GCS Integration
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-4.3 | v1 GPS_INPUT only for ArduPilot Plane | IT-01, AT-01 | Covered |
| AC-4.4 | Frame-by-frame streaming | PT-01 | Covered |
| AC-4.5 | Updated estimates/corrections | IT-02 | Covered |
| AC-5.1 | FC state initialization telemetry | IT-03 | Covered |
| AC-5.2 | Plane SITL fallback | IT-04 | Covered |
| AC-6.1 | QGC status 1-2 Hz | IT-05, PT-02 | Covered |
| AC-6.2 | GCS command ingress | IT-06, ST-01 | Covered |
| AC-6.3 | WGS84 output | IT-01 | Covered |
| AC-NEW-2 | Spoofing promotion <3 s | IT-04 | Covered |
| AC-NEW-8 | Blackout/failsafe status | IT-05 | Covered |
## Blackbox Tests
### IT-01: GPS_INPUT Field Mapping
**Summary**: Verify `PositionEstimate` maps to valid MAVLink `GPS_INPUT`.
**Traces to**: AC-4.3, AC-6.3
**Input data**: Position estimates across all source labels.
**Expected result**: v1 emits `GPS_INPUT` only, no `ODOMETRY`; WGS84 lat/lon/alt, fix type, ignore flags, and accuracy fields match contract.
**Max execution time**: 2 minutes.
---
### IT-02: Correction Emission
**Summary**: Verify updated estimates can be emitted without batching.
**Traces to**: AC-4.5
**Input data**: Original VO estimate followed by anchor-corrected estimate.
**Expected result**: Both estimates are emitted in order with updated accuracy/source label.
**Max execution time**: 2 minutes.
---
### IT-03: FC Telemetry Subscription
**Summary**: Verify telemetry needed for initialization and VIO is available.
**Traces to**: AC-5.1
**Input data**: Plane SITL or MAVLink replay with EKF position, IMU, attitude, airspeed, altitude.
**Expected result**: Normalized `TelemetrySample` stream includes required fields and timestamps.
**Max execution time**: 5 minutes.
---
### IT-04: Spoofing And Fallback In Plane SITL
**Summary**: Verify spoofing and no-estimate behavior in ArduPilot Plane SITL.
**Traces to**: AC-5.2, AC-NEW-2
**Input data**: Plane SITL production parameter set and spoofing trace.
**Expected result**: Own estimate promotion occurs within <3 s; fallback/no-estimate behavior matches Plane parameters.
**Max execution time**: 10 minutes.
---
### IT-05: QGC Blackout Status
**Summary**: Verify degraded-mode messages are visible at required rate.
**Traces to**: AC-6.1, AC-NEW-8
**Input data**: Safety wrapper emits blackout and failsafe statuses.
**Expected result**: QGC observer sees `VISUAL_BLACKOUT_IMU_ONLY` at 1-2 Hz and `VISUAL_BLACKOUT_FAILSAFE` at threshold.
**Max execution time**: 10 minutes.
---
### IT-06: Operator Relocalization Hint
**Summary**: Verify GCS command ingress can carry approximate relocalization hints.
**Traces to**: AC-6.2
**Input data**: STATUSTEXT/NAMED_VALUE_FLOAT/custom dialect hint fixture.
**Expected result**: Valid hint is parsed and forwarded to retrieval/safety logic; invalid hint is rejected.
**Max execution time**: 5 minutes.
## Performance Tests
### PT-01: Frame-Rate Emission
**Summary**: Verify output is streamed frame-by-frame and not batched.
**Traces to**: AC-4.4
**Load scenario**:
- Input estimate rate: target frame rate.
- Duration: 30 minutes.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Output delay p95 | <=25 ms after wrapper output | >100 ms |
| Missing messages | 0 except upstream dropped frames | Any silent drop |
---
### PT-02: QGC Status Rate Limit
**Summary**: Verify QGC status is downsampled without losing critical transitions.
**Traces to**: AC-6.1
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Status rate | 1-2 Hz while active | <1 Hz or >2 Hz sustained |
| Critical transition delay | <=1 s | >2 s |
## Security Tests
### ST-01: MAVLink Source And Command Validation
**Summary**: Verify unauthorized or malformed MAVLink messages are rejected.
**Traces to**: AC-6.2
**Attack vector**: Malicious source sends spoofed command or GPS data.
**Test procedure**:
1. Send valid command from allowed source.
2. Send same command from disallowed source/system ID.
3. Send malformed values.
**Expected behavior**: Allowed command is accepted; disallowed/malformed messages are rejected and logged.
**Pass criteria**: 0 unauthorized commands affect localization state.
## Acceptance Tests
### AT-01: Plane SITL Output Acceptance
**Summary**: Verify ArduPilot Plane receives and uses v1 `GPS_INPUT` as configured.
**Traces to**: AC-4.3
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Start Plane SITL with production params | FC accepts external GPS substitute config |
| 2 | Emit `GPS_INPUT` estimate | Message is received with expected fields |
| 3 | Observe wire | `ODOMETRY` is absent in v1 |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `sitl_spoofing_scenarios` | GPS loss/spoofing traces | Generated SITL | Small |
| `mavlink_output_fixtures` | PositionEstimate cases | Generated fixture | Small |
**Setup procedure**: Start SITL/QGC observer or replay MAVLink log.
**Teardown procedure**: Stop processes and archive tlogs.
**Data isolation strategy**: Unique MAVLink ports and run IDs per test.
_docs/02_document/components/08_fdr_observability/description.md
+79
View File
@@ -0,0 +1,79 @@
# FDR And Observability
## 1. High-Level Overview
**Purpose**: Record bounded, replayable mission evidence and expose runtime health/status events for analysis and operator awareness.
**Architectural Pattern**: Append-only event sink + exporter.
**Upstream dependencies**: All runtime components.
**Downstream consumers**: Validation harness, post-flight audit tools, QGC status through MAVLink component.
## 2. Internal Interfaces
### Interface: `FlightRecorder`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `append_event` | `FdrEvent` | `AppendResult` | Yes | `RecorderUnavailable`, `StorageFull` |
| `rollover` | `RolloverRequest` | `FdrSegmentInfo` | No | `RolloverFailed` |
| `export` | `ExportRequest` | `ExportResult` | Yes | `ExportFailed` |
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Append event | High | Yes | Append index only |
| Export by time/type | Post-flight | No | Time/type index |
### Storage Estimates
| Table/Collection | Est. Row Count | Row Size | Total Size | Growth Rate |
|------------------|----------------|----------|------------|-------------|
| FDR events | Flight-dependent | Mixed | <=64 GB per 8 h | Per flight |
## 4. Implementation Details
**State Management**: Owns active segment, rollover policy, and export state.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| PostgreSQL client | Event metadata, time/type indexes, mission query surface |
| CBOR writer | Bounded runtime payload segments |
| Parquet writer | Optional post-flight export |
**Error Handling Strategy**:
- Storage-full emits critical status and starts rollover/retention behavior.
- Append failures are surfaced to the caller and health system.
## 5. Caveats & Edge Cases
**Known limitations**:
- Raw frames are not retained by default; only metadata, decisions, hashes, and occlusion/blackout status are recorded.
- PostgreSQL availability is required for indexed FDR metadata; CBOR payload segments preserve bounded append behavior for high-volume data.
**Performance bottlenecks**:
- FDR appends must not block hot-path localization.
## 6. Dependency Graph
**Must be implemented after**: event schema and key DTOs.
**Can be implemented in parallel with**: MAVLink integration.
**Blocks**: release evidence and most validation reports.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Recorder unavailable | `fdr_unavailable path=...` |
| WARN | Rollover occurs | `fdr_rollover segment=...` |
| INFO | Export complete | `fdr_export_complete format=parquet` |
**Log format**: FDR event metadata plus local health logs.
**Log storage**: PostgreSQL FDR event tables plus CBOR segment payloads.
_docs/02_document/components/08_fdr_observability/tests.md
+166
View File
@@ -0,0 +1,166 @@
# Test Specification — FDR And Observability
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.3 | Anchor age/drift evidence | IT-01 | Covered |
| AC-1.4 | Confidence/source label retained | IT-01 | Covered |
| AC-4.4 | Per-frame local stream evidence | IT-01, PT-01 | Covered |
| AC-5.2 | Failure logging | IT-02 | Covered |
| AC-6.1 | QGC/status evidence | IT-03 | Covered |
| AC-8.4 | Generated tile audit | IT-04 | Covered |
| AC-8.5 | No raw frame retention | ST-01 | Covered |
| AC-NEW-3 | FDR retention and 64 GB cap | PT-01, AT-01 | Covered |
| AC-NEW-4 | False-position forensics | IT-05 | Covered |
| AC-NEW-5 | Thermal/throttle logging | IT-06 | Covered |
| AC-NEW-8 | Blackout/failsafe logging | IT-02, IT-03 | Covered |
## Blackbox Tests
### IT-01: Per-Estimate Event Capture
**Summary**: Verify every estimate stores covariance, source label, anchor age, and emitted output metadata.
**Traces to**: AC-1.3, AC-1.4, AC-4.4
**Input data**: Position estimate stream with satellite, VO, and dead-reckoned labels.
**Expected result**: PostgreSQL event index and CBOR payload segments contain all required fields with monotonic timestamps.
**Max execution time**: 5 minutes.
---
### IT-02: Failure And Blackout Logging
**Summary**: Verify no-estimate and blackout transitions are recorded.
**Traces to**: AC-5.2, AC-NEW-8
**Input data**: No-estimate gap and total blackout sequence.
**Expected result**: FDR records start, every degraded estimate, failsafe threshold, and recovery reason.
**Max execution time**: 10 minutes.
---
### IT-03: QGC Status Audit
**Summary**: Verify operator-visible status has matching FDR evidence.
**Traces to**: AC-6.1, AC-NEW-8
**Input data**: QGC status messages from MAVLink component.
**Expected result**: FDR contains status text, timestamp, and mode context.
**Max execution time**: 5 minutes.
---
### IT-04: Generated Tile Audit Trail
**Summary**: Verify tile-write decisions are recorded with parent covariance and trust level.
**Traces to**: AC-8.4
**Input data**: Accepted and rejected generated tile write decisions.
**Expected result**: FDR includes tile ID, parent covariance, trust level, sidecar hash, and rejection reason where applicable.
**Max execution time**: 5 minutes.
---
### IT-05: False-Position Investigation Bundle
**Summary**: Verify enough evidence exists to investigate a false-position event.
**Traces to**: AC-NEW-4
**Input data**: Simulated false anchor rejection and covariance growth sequence.
**Expected result**: Export includes estimates, anchor decisions, residuals, covariance, and emitted MAVLink fields.
**Max execution time**: 5 minutes.
---
### IT-06: Thermal/Throttle Event Capture
**Summary**: Verify resource health events are recorded.
**Traces to**: AC-NEW-5
**Input data**: Synthetic thermal/throttle metric stream.
**Expected result**: FDR records CPU/GPU/temp/throttle status and QGC warning trigger.
**Max execution time**: 5 minutes.
## Performance Tests
### PT-01: 8-Hour FDR Load
**Summary**: Verify FDR storage and append behavior under full mission load.
**Traces to**: AC-4.4, AC-NEW-3
**Load scenario**:
- Duration: 8 hours synthetic.
- Inputs: 3 Hz estimates, full-rate IMU, MAVLink tlog, health metrics, tile events.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Total FDR size | <=64 GB | >64 GB without rollover |
| Append latency p95 | <=10 ms async enqueue | >25 ms |
| Silent payload loss | 0 | Any unlogged loss |
**Resource limits**: FDR must not block hot-path localization.
## Security Tests
### ST-01: Raw Frame Retention Audit
**Summary**: Verify FDR does not store raw full-resolution frames.
**Traces to**: AC-8.5
**Attack vector**: Debug logging accidentally persists raw camera frames.
**Test procedure**:
1. Run normal replay and failed tile-generation replay.
2. Inspect FDR payloads and output directories.
**Expected behavior**: Only metadata, hashes, estimates, tiles, and allowed low-rate failed-frame thumbnails are retained.
**Pass criteria**: No raw nav/AI camera frame payloads in normal FDR.
## Acceptance Tests
### AT-01: FDR Export
**Summary**: Verify post-flight export creates usable audit artifacts.
**Traces to**: AC-NEW-3
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Complete synthetic flight | Segment rollover is logged and cap respected |
| 2 | Export FDR summary | Markdown/CSV/Parquet optional artifacts are produced |
| 3 | Query PostgreSQL index | Events can be filtered by time/type/mission |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `fdr_synthetic_load` | Estimate, IMU, MAVLink, health, tile events | Generated fixture | Large |
| `incident_fixture` | False-position and blackout evidence | Generated fixture | Small |
**Setup procedure**: Create isolated PostgreSQL schema and FDR segment directory.
**Teardown procedure**: Export report, then remove schema and segment directory.
**Data isolation strategy**: Per-run mission ID, schema, and FDR directory.
_docs/02_document/contracts/shared/config_errors_telemetry.md
+51
View File
@@ -0,0 +1,51 @@
# Contract: Config Errors Telemetry
**Component**: shared/config, shared/errors, shared/telemetry
**Producer task**: AZ-222 — AZ-222_runtime_config_errors_telemetry.md
**Consumer tasks**: AZ-223, AZ-224, AZ-225, AZ-226, AZ-227, AZ-228, AZ-229, AZ-230, AZ-231, AZ-232
**Version**: 1.0.0
**Status**: draft
**Last Updated**: 2026-05-03
## Purpose
Defines shared runtime configuration, error/result envelope, health, and telemetry metadata behavior consumed by all runtime components.
## Shape
| Contract | Required Behavior |
|----------|-------------------|
| Runtime profile | environment-specific settings loaded and validated before use |
| Error envelope | component, category, message, cause, retryability, severity |
| Health event | liveness/readiness status, dependency state, timestamp, component |
| Metrics labels | bounded component/action/status labels suitable for runtime reports |
## Invariants
- Missing required production settings fail startup or readiness loudly.
- Errors are returned or logged with component and category; no silent suppression.
- Secrets are referenced, not serialized into FDR, logs, or metrics.
## Non-Goals
- Does not define component-specific business errors.
- Does not replace FDR payload schemas.
## Versioning Rules
- Removing required config keys or error categories requires a major version bump.
- Adding optional health fields or metrics labels requires a minor version bump.
## Test Cases
| Case | Input | Expected | Notes |
|------|-------|----------|-------|
| missing-required-prod | production profile missing cache dir | readiness/startup failure | Clear error category |
| secret-value | signing key ref present | only key ref logged | No secret leakage |
| component-error | component reports dependency failure | structured envelope emitted | FDR-safe |
## Change Log
| Version | Date | Change | Author |
|---------|------|--------|--------|
| 1.0.0 | 2026-05-03 | Initial contract | autodev |
_docs/02_document/contracts/shared/geometry_time_sync.md
+52
View File
@@ -0,0 +1,52 @@
# Contract: Geometry And Time Sync Helpers
**Component**: shared/geo_geometry, shared/time_sync
**Producer task**: AZ-221 — AZ-221_shared_geometry_time_sync.md
**Consumer tasks**: AZ-223, AZ-225, AZ-226, AZ-228, AZ-230, AZ-231, AZ-232
**Version**: 1.0.0
**Status**: draft
**Last Updated**: 2026-05-03
## Purpose
Defines shared geospatial and timestamp helper behavior used by runtime components to avoid duplicated math and inconsistent frame/IMU alignment.
## Shape
| API Area | Shape | Errors |
|----------|-------|--------|
| Coordinate conversion | WGS84/local tangent conversions and distance calculations | invalid CRS, missing origin |
| Camera footprint | intrinsics/extrinsics/attitude/altitude to footprint and GSD | invalid calibration, missing altitude |
| Homography metrics | homography/covariance conversions and MRE support | invalid geometry |
| Time sync | monotonic checks, frame-to-IMU window selection, replay ordering | timestamp mismatch, gap/jitter exceeded |
## Invariants
- Helpers are deterministic for the same calibration, pose, and timestamp inputs.
- Time helpers report gaps/jitter instead of silently dropping samples.
- Geometry helpers do not decide safety policy; callers decide degrade/reject behavior.
## Non-Goals
- No VIO state estimation.
- No MAVLink parsing beyond normalized timestamp fields.
- No tile freshness or cache policy decisions.
## Versioning Rules
- Breaking changes to units, coordinate frames, or timestamp semantics require a major version bump.
- New helper outputs may be added as optional fields in minor versions.
## Test Cases
| Case | Input | Expected | Notes |
|------|-------|----------|-------|
| valid-wgs84-local | known WGS84 point and origin | round-trip within tolerance | Uses representative coordinates |
| frame-imu-window | frame timestamp plus IMU samples | correct aligned window | Includes gap metrics |
| invalid-calibration | missing intrinsics/extrinsics | explicit error | No silent fallback |
## Change Log
| Version | Date | Change | Author |
|---------|------|--------|--------|
| 1.0.0 | 2026-05-03 | Initial contract | autodev |
_docs/02_document/contracts/shared/runtime_contracts.md
+56
View File
@@ -0,0 +1,56 @@
# Contract: Runtime Shared Contracts
**Component**: shared/contracts
**Producer task**: AZ-220 — AZ-220_shared_runtime_contracts.md
**Consumer tasks**: AZ-223, AZ-224, AZ-225, AZ-226, AZ-227, AZ-228, AZ-229, AZ-230, AZ-231, AZ-232
**Version**: 1.0.0
**Status**: draft
**Last Updated**: 2026-05-03
## Purpose
Defines the shared runtime DTO/event contract surface that component implementations consume instead of inventing local shapes.
## Shape
| Contract | Required Fields / Methods | Consumers |
|----------|---------------------------|-----------|
| `FramePacket` | frame ID, timestamp, image reference, calibration ID, occlusion, quality, normalization hint | camera, VIO, Satellite Service, Anchor Verification, Tile Manager, FDR |
| `TelemetrySample` | timestamp, IMU, attitude, altitude, airspeed, GPS health | MAVLink, VIO, safety wrapper, FDR |
| `VioStatePacket` | timestamp, relative pose, velocity, bias, tracking quality, covariance hint | VIO, safety wrapper, FDR |
| `PositionEstimate` | WGS84 coordinates, covariance, source label, fix type, horizontal accuracy, anchor age | safety wrapper, MAVLink, Tile Manager, FDR |
| `VprCandidate` | chunk ID, tile ID, score, footprint, freshness status | Satellite Service, Anchor Verification, FDR |
| `AnchorDecision` | candidate ID, acceptance result, estimated pose, inliers, MRE, rejection reason | Anchor Verification, safety wrapper, FDR |
| `CacheTileRecord` | tile ID, CRS, meters per pixel, capture date, signature/hash, trust level | Tile Manager, Satellite Service, Anchor Verification |
| `FdrEvent` | event type, timestamp, component, severity, payload reference, mission/run ID | all runtime components |
## Invariants
- Timestamps are normalized to a shared monotonic nanosecond representation before cross-component use.
- Confidence fields must not under-report known uncertainty.
- Raw frame payloads are referenced, not persisted in shared DTOs.
- Generated tile and anchor records must carry provenance/freshness metadata.
## Non-Goals
- Does not prescribe internal classes or storage implementation.
- Does not define e2e test runner-only report schemas.
## Versioning Rules
- Removing or renaming a field requires a major version bump.
- Adding optional telemetry or diagnostic fields requires a minor version bump.
## Test Cases
| Case | Input | Expected | Notes |
|------|-------|----------|-------|
| valid-frame | frame with timestamp, calibration, quality | accepted by consumers | Includes normalization hint |
| invalid-time | non-monotonic timestamp | rejected or marked invalid | Time-sync contract decides details |
| stale-anchor | anchor decision with stale freshness | rejected/down-confidenced | Safety wrapper must not accept blindly |
## Change Log
| Version | Date | Change | Author |
|---------|------|--------|--------|
| 1.0.0 | 2026-05-03 | Initial contract | autodev |
_docs/02_document/data_model.md
+148
View File
@@ -0,0 +1,148 @@
# Data Model
## Scope
This model defines system-level runtime, cache, telemetry, and validation data. PostgreSQL with PostGIS is the primary structured store for manifests, spatial metadata, mission state, and FDR event indexes. Large binary payloads remain local files: COG tiles, descriptor/index files, FDR payload segments, and replay fixtures.
## Entity Overview
| Entity | Purpose | Storage / Transport | Owner |
|--------|---------|---------------------|-------|
| MissionProfile | Operational area, sector type, route shape, altitude band, cache budget | Mission config file | Tile Manager |
| CameraCalibration | Intrinsics, distortion, lens, fixed extrinsics, capture settings | Versioned calibration file | Camera ingest/calibration |
| FrameRecord | Per-frame metadata, timestamp, total-occlusion/blackout state, image quality, processing status | PostgreSQL/FDR event; replay fixture | Camera ingest/calibration |
| TelemetrySample | FC IMU, attitude, altitude, airspeed, GPS health | MAVLink stream; FDR event | MAVLink/GCS integration |
| VioState | Backend-relative state, velocity, bias, tracking quality | Internal DTO; FDR event | VIO adapter |
| PositionEstimate | WGS84 output, covariance, source label, anchor age, fix type | MAVLink DTO; FDR event | Safety/anchor wrapper |
| VprChunk | Retrieval footprint and descriptor metadata | PostgreSQL/PostGIS manifest + descriptor files | Satellite Service |
| AnchorCandidate | Top-K retrieval result and local verification metrics | Internal DTO; FDR event | Anchor verification |
| CacheTile | Service-source or generated COG tile metadata | PostgreSQL/PostGIS manifest + signed JSON sidecar | Tile Manager |
| GeneratedTile | In-flight tile candidate with trust/provenance metadata | COG + sidecar + FDR event | Tile Manager |
| FdrSegment | Bounded append-only mission evidence segment | PostgreSQL event index + CBOR segment payloads | FDR/observability |
| ValidationRun | Replay/test run metadata and outcomes | CSV/Markdown/test artifacts | Validation harness |
## Core Entity Attributes
### MissionProfile
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `mission_id` | string | yes | Unique mission/run identifier |
| `operational_area_polygon` | geometry | yes | Up to ~400 km² |
| `sector_classification` | enum | yes | `active_conflict` or `stable_rear` |
| `planned_altitude_agl_m` | number | yes | <=1000 m AGL |
| `route_type` | enum | yes | `sector`, `transit_corridor`, or mixed |
| `cache_budget_bytes` | integer | yes | Default ~10 GB persistent |
### CameraCalibration
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `camera_model` | string | yes | ADTi 20MP 20L V1 family |
| `sensor_width_mm` | number | yes | Public spec check currently indicates 23.20 mm |
| `sensor_height_mm` | number | yes | Public spec check currently indicates 15.40 mm |
| `image_width_px` | integer | yes | Public spec check currently indicates 5456 px |
| `image_height_px` | integer | yes | Public spec check currently indicates 3632 px |
| `pixel_pitch_um` | number | yes | Public spec check indicates 4.25 um |
| `lens_focal_length_mm` | number | yes | TBD before implementation |
| `distortion_coefficients` | array | yes | From checkerboard calibration |
| `body_T_camera` | transform | yes | Fixed camera-to-body extrinsics |
| `spec_verification_status` | enum | yes | `manufacturer_verified`, `public_page_only`, `operator_supplied` |
### PositionEstimate
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `timestamp_ns` | integer | yes | Frame-aligned |
| `lat_deg` | number | yes | WGS84 |
| `lon_deg` | number | yes | WGS84 |
| `alt_msl_m` | number | yes | MSL altitude for `GPS_INPUT` |
| `covariance_95_semi_major_m` | number | yes | Must not be under-reported |
| `source_label` | enum | yes | `satellite_anchored`, `vo_extrapolated`, `dead_reckoned` |
| `last_satellite_anchor_age_ms` | integer | yes | Monotonic until new anchor |
| `fix_type` | integer | yes | MAVLink fix semantics |
| `horiz_accuracy_m` | number | yes | >= covariance semi-major mapping |
| `quality_flags` | bitset/string array | yes | Anchor, blackout, spoofing, stale tile, etc. |
### FrameRecord
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `frame_id` | string | yes | Stable frame/run identifier |
| `timestamp_ns` | integer | yes | Camera clock normalized by time-sync helper |
| `camera_calibration_id` | string | yes | Links to `CameraCalibration` |
| `occlusion_status` | enum | yes | `clear`, `partial_occlusion`, `total_occlusion`, `blackout` |
| `usable_for_vio` | boolean | yes | Must be false for total occlusion/blackout |
| `usable_for_anchor` | boolean | yes | Must be false for total occlusion/blackout |
| `blackout_reason` | enum | optional | `cloud`, `lens_cover`, `whiteout`, `decode_failure`, `underexposed`, `overexposed`, `unknown` |
| `blur_score` | number | yes | Quality metric |
| `texture_score` | number | yes | Quality metric |
### CacheTile
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `tile_id` | string | yes | Stable ID |
| `tile_type` | enum | yes | `service_source`, `generated_candidate`, `generated_soft`, `trusted_basemap` |
| `cog_path` | string | yes | Local path |
| `crs` | string | yes | Projection metadata |
| `meters_per_pixel` | number | yes | Must satisfy cache interface floor |
| `capture_date` | date | yes | Freshness gate |
| `source` | string | yes | Satellite Service or onboard generation |
| `sha256` | string | yes | Integrity |
| `signature_status` | enum | yes | `valid`, `missing`, `invalid` |
| `parent_pose_covariance_m` | number | generated only | Tile-write gate |
| `trust_level` | enum | yes | `rejected`, `candidate`, `soft`, `trusted` |
## Relationships
```mermaid
erDiagram
MissionProfile ||--o{ CacheTile : scopes
MissionProfile ||--o{ VprChunk : indexes
CameraCalibration ||--o{ FrameRecord : calibrates
FrameRecord ||--o{ VioState : contributes_to
TelemetrySample ||--o{ VioState : contributes_to
VioState ||--o{ PositionEstimate : propagates
VprChunk ||--o{ AnchorCandidate : retrieved_as
CacheTile ||--o{ AnchorCandidate : verified_against
AnchorCandidate ||--o{ PositionEstimate : may_anchor
PositionEstimate ||--o{ GeneratedTile : gates
PositionEstimate ||--o{ FdrSegment : recorded_in
```
## Storage Strategy
| Data Class | Primary Format | Reason |
|------------|----------------|--------|
| Structured mission/cache/FDR metadata | PostgreSQL + PostGIS | Queryable freshness, coverage, spatial footprints, descriptors, tile status, and FDR event indexes |
| Tile audit sidecar | Signed JSON | Human/audit/service interchange per tile |
| Imagery tile | COG | Geospatial raster standard |
| Descriptor index | FAISS CPU index files + metadata | Fast top-K retrieval |
| FDR runtime payloads | CBOR segment files + PostgreSQL index | Bounded append payloads with queryable event metadata |
| FDR analysis export | Parquet optional | Post-flight analytics |
| Test report | CSV + Markdown | CI and human review |
## Migration Strategy
- PostgreSQL schemas use explicit `schema_version` and additive migrations by default.
- PostGIS geometry columns are used for mission polygons, tile footprints, VPR chunks, and generated-tile extents.
- FDR segment schema includes `segment_schema_version`; old readers must reject unknown required fields loudly.
- Sidecars include a `sidecar_version` and hash of the COG payload.
- Migrations are implemented as deterministic scripts with rollback for metadata-only changes.
- No database/table/column rename is allowed without explicit approval during implementation.
## Seed Data Requirements
| Environment | Seed Data |
|-------------|-----------|
| Development | 60 project images, `coordinates.csv`, small cache fixture, generated SITL traces |
| Public replay | Pinned MUN-FRL/ALTO/Kagaru/EPFL dataset slices and licenses |
| Jetson validation | Production-like cache/index, cold-start fixtures, thermal workload |
| Representative acceptance | Synchronized target nav-camera + FC telemetry + ground truth |
## Backward Compatibility
- Runtime should tolerate older cache sidecars if required fields exist and signatures validate.
- Generated tile sidecars must include all fields required by Satellite Service ingest; missing fields make the tile ineligible for promotion.
- FDR readers must support at least the current and previous segment schema version during the project lifecycle.
_docs/02_document/deployment/README.md
+11
View File
@@ -0,0 +1,11 @@
# Deployment Planning Index
This directory contains the system-level deployment plan produced during Plan Step 2:
- `containerization.md`
- `ci_cd_pipeline.md`
- `environment_strategy.md`
- `observability.md`
- `deployment_procedures.md`
Component-specific implementation tasks are created later during decomposition.
_docs/02_document/deployment/ci_cd_pipeline.md
+54
View File
@@ -0,0 +1,54 @@
# CI/CD Pipeline
## Pipeline Stages
| Stage | Runs On | Gate |
|-------|---------|------|
| Format/lint | PR | Block merge |
| Unit tests | PR | Block merge |
| Replay black-box smoke | PR | Block merge |
| Cache/security fixture tests | PR | Block merge |
| Plane SITL spoof/failsafe tests | Release candidate / nightly | Block release |
| Public dataset replay | Nightly / release candidate | Block release |
| Jetson latency/resource tests | Release candidate | Block release |
| Thermal/FDR endurance | Release candidate / hardware qualification | Block release |
## Artifact Outputs
- Test CSV reports.
- FDR validation summaries.
- Cache integrity reports.
- Dataset replay metrics.
- SITL tlogs.
- Jetson profiling traces.
## Caching
- Cache dependency builds by lockfile hash.
- Cache public dataset slices only in controlled CI storage with license metadata.
- Do not cache secrets or signing keys.
## Branch Policy
- Work occurs on `dev`.
- Release gates must pass before deploy artifacts are considered production-ready.
- Any failed safety, spoofing, false-position, or cache-poisoning test blocks release.
## Quality Gates
| Gate | Threshold |
|------|-----------|
| Still-image geolocation | >=80% within 50 m and >=50% within 20 m |
| Hot-path latency | <400 ms p95 |
| Memory | <8 GB shared |
| Cold start | <30 s p95 |
| FDR | <=64 GB / 8-hour flight |
| Cache storage | <=10 GB unless split budget is approved |
| False position | AC-NEW-4 thresholds |
## Open Tasks For Decomposition
- Define CI runner labels for Docker/replay vs Jetson local hardware.
- Add dataset-license checks before public dataset jobs.
- Implement SITL scenario generation and tlog validation job.
- Implement report collation into a release evidence bundle.
_docs/02_document/deployment/containerization.md
+46
View File
@@ -0,0 +1,46 @@
# Containerization
## Strategy
The production runtime targets Jetson hardware and may not be fully containerized for all camera/GPU paths. The test and development stack uses containers where practical, with local hardware execution required for release gates.
## Runtime Units
| Unit | Containerized? | Notes |
|------|----------------|-------|
| GPS-denied service | Optional on Jetson | Must access camera, CUDA/TensorRT/ONNX, MAVLink, local cache, FDR storage |
| Replay consumer | Yes | Deterministic black-box test harness |
| Satellite cache stub | Yes | Local fixture volume for COG/manifest/descriptors |
| ArduPilot Plane SITL | Yes or local process | Used for MAVLink and failsafe validation |
| QGC observer/log parser | Yes | Parses MAVLink status/tlogs |
## Docker Compose Profiles
| Profile | Purpose | Services |
|---------|---------|----------|
| `replay` | CI/PR deterministic fixture tests | gps-denied-service, replay-consumer, satellite-cache-stub |
| `sitl` | ArduPilot Plane integration tests | gps-denied-service, ardupilot-plane-sitl, qgc-observer |
| `jetson-local` | Documentation-only profile for local hardware run | Host runtime with local scripts/tasks created later |
## Image Requirements
- Base images must match JetPack/CUDA compatibility for GPU tests.
- Replay-only images may use standard Ubuntu/Python/C++ build images.
- No production image should contain secrets, mission signing keys, or provider credentials.
- Dataset downloads are not baked into images; they are mounted as versioned fixtures.
## Volumes
| Volume | Purpose |
|--------|---------|
| `/data/input` | Test images and public dataset slices |
| `/cache/satellite` | Offline cache fixture |
| `/fdr` | Runtime FDR output |
| `/test-results` | CSV/Markdown reports |
## Open Tasks For Decomposition
- Create Dockerfiles for replay-compatible service and consumer harness.
- Define Jetson local setup scripts for GPU/camera/MAVLink access.
- Create compose profiles for replay and SITL.
- Add license-aware public dataset fixture downloader.
_docs/02_document/deployment/deployment_procedures.md
+68
View File
@@ -0,0 +1,68 @@
# Deployment Procedures
## Deployment Targets
| Target | Purpose |
|--------|---------|
| Replay environment | Development and CI fixtures |
| Plane SITL | MAVLink/failsafe validation |
| Jetson companion computer | Production runtime and release gating |
| Representative flight/replay rig | Final acceptance evidence |
## Pre-Deployment Checklist
- Camera lens, resolution, FPS, sensor dimensions, and operating temperature are manufacturer-verified.
- Camera intrinsics/extrinsics are calibrated and versioned.
- BASALT, OpenCV, FAISS, LightGlue, DINOv2/ONNX/TensorRT dependencies are pinned.
- TensorRT/ONNX descriptor-fidelity tests pass before optimized engines are used.
- Satellite cache manifests and sidecars validate signatures, hashes, freshness, and resolution.
- Plane SITL validates `GPS_INPUT` behavior with production parameters.
- Jetson latency, memory, and thermal release gates pass.
- FDR rollover test passes.
## Deployment Steps
1. Install JetPack-compatible runtime dependencies on the companion computer.
2. Install/build BASALT and native vision dependencies.
3. Pre-build any ONNX/TensorRT engines accepted by fidelity tests.
4. Sync mission cache from Satellite Service before flight.
5. Validate cache manifest, descriptors, signatures, resolution, and freshness.
6. Start the onboard service and verify FC telemetry connection.
7. Run cold-start first-fix check.
8. Confirm QGroundControl status and FDR segment creation.
## Health Checks
| Check | Pass Condition |
|-------|----------------|
| Camera input | Frames received with expected resolution/rate |
| FC telemetry | IMU/attitude/altitude/GPS-health stream healthy |
| Cache | Manifest and descriptor index valid |
| First fix | Valid `GPS_INPUT` <30 s p95 in cold-start test |
| Resource health | Memory <8 GB, no thermal throttle |
| QGC status | Status visible at configured downsample rate |
| FDR | Segment open and writable |
## Rollback
- If runtime dependency update fails tests, revert to previous pinned build.
- If cache manifest validation fails, reject the mission cache and resync/rebuild before flight.
- If optimized engine fidelity fails, fall back to PyTorch/ONNX path that passed descriptor tests.
- If BASALT candidate fails representative replay gates, evaluate Kimera backup or custom fallback tasks before production deployment.
## Post-Flight Procedure
1. Stop the onboard service cleanly.
2. Export FDR summary and integrity hashes.
3. Package generated tiles with sidecars and manifest delta.
4. Upload generated tile package to Satellite Service when connectivity is available.
5. Archive release evidence: tlogs, FDR summary, cache validation report, test results.
## Deployment Blockers
- ADTi camera spec mismatch unresolved for FPS/resolution/lens/temperature.
- Missing representative synchronized nav-camera + FC telemetry + ground truth for final acceptance.
- Any false-position safety budget failure.
- Any cache-poisoning gate failure.
- Any Plane SITL `GPS_INPUT` failure.
- Thermal throttling during the 8-hour target workload.
_docs/02_document/deployment/environment_strategy.md
+49
View File
@@ -0,0 +1,49 @@
# Environment Strategy
## Environments
| Environment | Purpose | Hardware |
|-------------|---------|----------|
| Development replay | Fast local iteration with fixtures | Developer workstation |
| CI replay | Deterministic PR checks | Docker runner |
| Public dataset replay | Nightly/RC algorithm validation | Docker or GPU runner |
| Plane SITL | MAVLink/failsafe validation | Docker/local SITL |
| Jetson hardware validation | Production path latency, memory, GPU, camera, thermal | Jetson Orin Nano Super |
| Representative flight/replay | Final acceptance evidence | Target-like UAV/FC/camera setup |
## Configuration Classes
| Config | Development | Production |
|--------|-------------|------------|
| Satellite cache | Small fixture | Full mission cache |
| PostgreSQL/PostGIS | Local test DB with fixture manifests | Local onboard DB with signed mission manifests, spatial metadata, and FDR event indexes |
| Descriptor index | Small FAISS index | Full operational-area index |
| MAVLink | SITL/replay | Physical FC link |
| FDR | Temporary directory | Per-flight NVMe directory with rollover |
| Dataset fixtures | Optional public slices | Not used at runtime |
## Secrets And Signing
- Mission signing keys are never committed.
- Test keys may be committed only if clearly labeled as non-production.
- Provider credentials are not used by onboard runtime.
- Any Satellite Service sync credentials are post-flight/deployment environment secrets.
## Dataset Licensing
Public datasets must be tagged before use:
| Dataset | Expected Use | License Constraint |
|---------|--------------|--------------------|
| MUN-FRL | Preferred public VIO/nadir replay | CC BY 4.0 per current docs |
| ALTO | Preferred aerial localization/VPR replay | BSD-3 repository; dataset availability must be pinned |
| Kagaru | Fixed-wing/farmland validation candidate | Verify terms before commercial use |
| EPFL fixed-wing | Fixed-wing validation candidate | Verify terms before commercial use |
| VPAir | VPR/localization only | Academic-use restriction likely blocks commercial acceptance |
| UZH FPV | VIO stress proxy only | Non-commercial license blocks commercial acceptance |
## Promotion Rules
- A result from public datasets can de-risk implementation but cannot replace representative acceptance data.
- A release candidate cannot be promoted without Jetson hardware validation and Plane SITL.
- A mission cache cannot be used if manifest/signature/freshness validation fails.
_docs/02_document/deployment/observability.md
+61
View File
@@ -0,0 +1,61 @@
# Observability
## Goals
- Explain every emitted position estimate.
- Detect false-position risk before it reaches the flight controller.
- Preserve enough evidence to replay incidents without storing raw frames.
- Surface operator-relevant status to QGroundControl without saturating telemetry.
## Runtime Signals
| Signal | Frequency | Destination | Notes |
|--------|-----------|-------------|-------|
| Position estimate | Per processed frame locally | FDR, MAVLink `GPS_INPUT` | GCS receives downsampled status |
| Source label | Per estimate | FDR, status summary | `satellite_anchored`, `vo_extrapolated`, `dead_reckoned` |
| Covariance semi-major | Per estimate | FDR, `GPS_INPUT.horiz_accuracy` mapping | Must not under-report |
| Anchor decision | Per candidate | FDR | Include MRE, inliers, tile provenance, rejection reason |
| Cache validation | On cache load / tile read | FDR, health log | Signature, freshness, resolution, hash |
| Blackout/spoofing status | On transition and 1-2 Hz while active | QGC, FDR | Operator status |
| Total occlusion status | Per transition and sampled while active | FDR, QGC if persistent | Indicates VIO is bypassed and IMU-only propagation is active |
| Resource health | 1 Hz or configurable | FDR, QGC warning on threshold | CPU/GPU/temp/memory/throttle |
| Tile write decision | Per generated tile | FDR, sidecar | Include parent covariance and trust level |
## Logs
| Log Type | Format | Retention |
|----------|--------|-----------|
| FDR events/index | PostgreSQL tables + CBOR payload segments | <=64 GB per flight, rollover |
| MAVLink raw stream | tlog or equivalent | FDR cap |
| Health metrics | FDR event stream | FDR cap |
| Test reports | CSV/Markdown | CI artifact retention |
## Alerts And Status Text
| Condition | Status |
|-----------|--------|
| Visual blackout starts | `VISUAL_BLACKOUT_IMU_ONLY` |
| Total occlusion before VIO | `VISUAL_OCCLUSION_IMU_ONLY` |
| Blackout failsafe threshold exceeded | `VISUAL_BLACKOUT_FAILSAFE` |
| Spoofing promotion/demotion | QGC status text with mode and timestamp |
| Stale cache tile rejected | Warning in FDR; QGC only if mission-impacting |
| Thermal throttle risk | QGC warning before throttle if possible |
| No estimate for threshold | Relocalization request / failsafe status |
## Metrics For Release Evidence
- Error CDF against ground truth.
- Anchor-age binned error.
- Covariance calibration plot.
- VIO completion rate.
- Relocalization trigger-to-anchor latency.
- Cache freshness rejection counts.
- FDR size over 8 hours.
- Thermal/throttle timeline.
## Open Tasks For Decomposition
- Define FDR schema and event names.
- Define QGC status vocabulary and rate limiting.
- Define telemetry-to-report export tooling.
- Define covariance calibration dashboard/report.
_docs/02_document/diagrams/component_overview.md
+46
View File
@@ -0,0 +1,46 @@
# Component Overview Diagram
```mermaid
flowchart LR
camera[01 Camera Ingest And Calibration]
vio[02 VIO Adapter]
wrapper[03 Safety And Anchor Wrapper]
retrieval[04 Satellite Service]
verify[05 Anchor Verification]
cache[06 Tile Manager]
mav[07 MAVLink And GCS Integration]
fdr[08 FDR And Observability]
tests[[Separate E2E Test Suite]]
navCam[[Nav Camera]] --> camera
fc[[ArduPilot Plane FC]] --> mav
satSvc[[Azaion Suite Satellite Service]] --> retrieval
datasets[[Replay/Public Datasets]] --> tests
camera --> vio
mav --> vio
vio --> wrapper
wrapper --> retrieval
retrieval --> verify
cache --> retrieval
cache --> verify
verify --> wrapper
wrapper --> mav
wrapper --> cache
camera --> cache
camera --> fdr
vio --> fdr
wrapper --> fdr
retrieval --> fdr
verify --> fdr
cache --> fdr
mav --> fdr
tests --> camera
tests --> mav
tests --> cache
mav --> qgc[[QGroundControl]]
mav --> fc
retrieval --> satSvc
```
_docs/02_document/diagrams/flows/flow_cache_tile_lifecycle.md
+18
View File
@@ -0,0 +1,18 @@
# Flow: Tile Manager And Generated Tile Lifecycle
```mermaid
flowchart TD
preflight([Pre-flight Satellite Service sync]) --> validate[06 Tile Manager validates manifest signatures hashes freshness]
validate --> cacheOk{Cache valid?}
cacheOk -->|No| block[Block cache usage and report]
cacheOk -->|Yes| load[04 Satellite Service loads local descriptor metadata and FAISS index]
load --> flight([Flight runtime])
flight --> eligibility[03 Tile write eligibility check]
eligibility --> eligible{Covariance and quality pass?}
eligible -->|No| noWrite[Do not write generated tile]
eligible -->|Yes| write[06 Orthorectify frame and write COG + signed JSON sidecar]
write --> fdr[08 Record tile-write audit]
fdr --> postflight([Post-flight])
postflight --> package[06 Package generated tiles + manifest delta]
package --> sync[[Post-flight Satellite Service upload]]
```
_docs/02_document/diagrams/flows/flow_normal_localization.md
+21
View File
@@ -0,0 +1,21 @@
# Flow: Normal Localization
```mermaid
flowchart TD
start([Frame + FC telemetry]) --> ingest[01 Camera ingest and quality]
ingest --> occlusion{Total occlusion or blackout?}
occlusion -->|Yes| imuOnly[03 IMU-only dead_reckoned propagation]
occlusion -->|No| frameOk{Frame usable for VIO?}
frameOk -->|No| degrade[03 Safety wrapper degraded mode]
frameOk -->|Yes| vio[02 VIO adapter]
telemetry[07 MAVLink telemetry] --> vio
vio --> healthy{VIO healthy?}
healthy -->|Yes| wrap[03 Covariance calibration + source label]
healthy -->|No| trigger[Trigger relocalization]
wrap --> emit[07 Emit GPS_INPUT and QGC status]
wrap --> record[08 Record FDR event]
emit --> endNode([Position output])
trigger --> satFlow[[Satellite relocalization flow]]
imuOnly --> emit
degrade --> emit
```
_docs/02_document/diagrams/flows/flow_satellite_relocalization.md
+21
View File
@@ -0,0 +1,21 @@
# Flow: Satellite Relocalization
```mermaid
flowchart TD
start([Relocalization trigger]) --> request[03 Build retrieval request]
request --> retrieve[04 DINOv2-VLAD + FAISS top-K]
retrieve --> candidates{Candidates found?}
candidates -->|No| degraded[03 Continue degraded/dead reckoned]
candidates -->|Yes| verify[05 ALIKED/DISK + LightGlue + RANSAC]
verify --> geometry{Geometry passes?}
geometry -->|No| degraded
geometry -->|Yes| gates[03 Freshness/provenance/Mahalanobis gates]
gates --> accepted{Anchor accepted?}
accepted -->|No| degraded
accepted -->|Yes| update[03 Apply absolute correction]
update --> emit[07 Emit anchored GPS_INPUT]
degraded --> emitDegraded[07 Emit degraded GPS_INPUT/status]
emit --> record[08 Record anchor decision]
emitDegraded --> record
record --> endNode([Relocalization result])
```
_docs/02_document/epics.md
+136
View File
@@ -0,0 +1,136 @@
# Work Item Epics
**Tracker**: Jira
**Project**: AZAION (`AZ`)
**Date**: 2026-05-01
**Labels**: `gps-denied-onboard-plan`, `autodev`
**Lessons applied**: No `_docs/LESSONS.md` file exists; no prior estimation or dependency lessons were available.
## Dependency Order
| Order | Jira ID | Epic | Type | Depends On | Estimate |
|-------|---------|------|------|------------|----------|
| 1 | AZ-206 | Bootstrap & Initial Structure | bootstrap | none | M / 5-8 pts |
| 2 | AZ-207 | Cross-Cutting: Shared Geometry And Time Sync | cross-cutting | AZ-206 | S-M / 3-5 pts |
| 3 | AZ-208 | Cross-Cutting: Runtime Configuration And Errors | cross-cutting | AZ-206 | S-M / 3-5 pts |
| 4 | AZ-209 | Camera Ingest And Calibration | component | AZ-206, AZ-207, AZ-208 | M / 5-8 pts |
| 5 | AZ-210 | MAVLink And GCS Integration | component | AZ-206, AZ-208 | M / 5-8 pts |
| 6 | AZ-211 | Tile Manager | component | AZ-206, AZ-207, AZ-208 | L / 8-13 pts |
| 7 | AZ-212 | FDR And Observability | component | AZ-206, AZ-208 | M-L / 5-8 pts |
| 8 | AZ-213 | VIO Adapter | component | AZ-206, AZ-207, AZ-208, AZ-209, AZ-210 | L / 8-13 pts |
| 9 | AZ-214 | Satellite Service | component | AZ-206, AZ-207, AZ-208, AZ-209, AZ-211 | L / 8-13 pts |
| 10 | AZ-215 | Anchor Verification | component | AZ-206, AZ-207, AZ-208, AZ-209, AZ-211, AZ-214 | L / 8-13 pts |
| 11 | AZ-216 | Safety And Anchor Wrapper | component | AZ-206, AZ-207, AZ-208, AZ-209, AZ-210, AZ-213, AZ-215 | XL / 13-21 pts |
| 12 | AZ-217 | E2E Test Suite | test-support | component epics | L / 8-13 pts |
| 13 | AZ-218 | Blackbox Tests | blackbox-tests | AZ-217, component epics | L / 8-13 pts |
## Component Mapping
| Component / Artifact | Epic |
|----------------------|------|
| Project scaffold, shared DTOs, migrations, CI skeleton | AZ-206 |
| `common-helpers/01_helper_geo_geometry.md` | AZ-207 |
| `common-helpers/02_helper_time_sync.md` | AZ-207 |
| Runtime config, error contracts, health checks | AZ-208 |
| `components/01_camera_ingest_calibration/` | AZ-209 |
| `components/02_vio_adapter/` | AZ-213 |
| `components/03_safety_anchor_wrapper/` | AZ-216 |
| `components/04_satellite_retrieval/` | AZ-214 |
| `components/05_anchor_verification/` | AZ-215 |
| `components/06_cache_tile_lifecycle/` | AZ-211 |
| `components/07_mavlink_gcs_integration/` | AZ-210 |
| `components/08_fdr_observability/` | AZ-212 |
| `tests/e2e-test-suite.md`, `tests/blackbox-tests.md`, `tests/environment.md` | AZ-217 |
| System blackbox/performance/resilience/security/resource tests | AZ-218 |
## Epic Relationship Diagram
```mermaid
flowchart TD
bootstrap[AZ-206 Bootstrap]
geo[AZ-207 Shared Geometry And Time Sync]
config[AZ-208 Runtime Configuration And Errors]
camera[AZ-209 Camera Ingest]
mavlink[AZ-210 MAVLink And GCS]
cache[AZ-211 Tile Manager]
fdr[AZ-212 FDR And Observability]
vio[AZ-213 VIO Adapter]
retrieval[AZ-214 Satellite Service]
anchor[AZ-215 Anchor Verification]
safety[AZ-216 Safety And Anchor Wrapper]
validation[AZ-217 E2E Test Suite]
blackbox[AZ-218 Blackbox Tests]
bootstrap --> geo
bootstrap --> config
bootstrap --> camera
bootstrap --> mavlink
bootstrap --> cache
bootstrap --> fdr
geo --> camera
geo --> cache
geo --> vio
geo --> retrieval
geo --> anchor
geo --> safety
config --> camera
config --> mavlink
config --> cache
config --> fdr
config --> vio
config --> retrieval
config --> anchor
config --> safety
camera --> vio
mavlink --> vio
camera --> retrieval
cache --> retrieval
retrieval --> anchor
camera --> anchor
cache --> anchor
vio --> safety
anchor --> safety
mavlink --> safety
safety --> cache
safety --> mavlink
safety --> fdr
camera --> fdr
cache --> fdr
safety --> validation
fdr --> validation
camera --> validation
mavlink --> validation
retrieval --> validation
anchor --> validation
cache --> validation
validation --> blackbox
```
## Cross-Cutting Ownership
| Concern | Owner Epic | Rule |
|---------|------------|------|
| Geospatial math, WGS84/local conversions, GSD, footprints | AZ-207 | Components consume shared helper; no local duplicate implementations |
| Timestamp validation, IMU/frame alignment, replay ordering | AZ-207 | Components consume shared helper; no local duplicate implementations |
| Runtime configuration, environment profiles, startup validation | AZ-208 | Components consume shared config loader and health-check contract |
| Shared error/result envelopes | AZ-208 | Components use shared error categories and do not swallow failures |
## Created Jira Epics
- AZ-206 — Bootstrap & Initial Structure
- AZ-207 — Cross-Cutting: Shared Geometry And Time Sync
- AZ-208 — Cross-Cutting: Runtime Configuration And Errors
- AZ-209 — Camera Ingest And Calibration
- AZ-210 — MAVLink And GCS Integration
- AZ-211 — Tile Manager
- AZ-212 — FDR And Observability
- AZ-213 — VIO Adapter
- AZ-214 — Satellite Service
- AZ-215 — Anchor Verification
- AZ-216 — Safety And Anchor Wrapper
- AZ-217 — E2E Test Suite
- AZ-218 — Blackbox Tests
## Tracker Notes
Jira authentication succeeded. A transient Jira server-side PostgreSQL timeout occurred while creating `Blackbox Tests`; the write was recorded under `_docs/_process_leftovers/` and then successfully retried as AZ-218. The leftover entry was deleted after replay success.
_docs/02_document/glossary.md
+96
View File
@@ -0,0 +1,96 @@
# Glossary
**Status**: confirmed-by-user
**Date**: 2026-05-01
## Aerial VPR
Visual place recognition over aerial/nadir imagery; used to retrieve candidate satellite/cache chunks for a UAV frame. (source: `solution.md`)
## ADTi 20MP 20L V1
Selected navigation-camera family. Public product pages list Sony APS-C CMOS, 5456 x 3632 max image size, 23.20 x 15.40 mm sensor, 4.25 um pixel pitch, Sony E mount, and 2 fps continuous capture; final manufacturer specification remains a verification risk. (source: `restrictions.md`, user-confirmed open-question review)
## Anchor
Accepted absolute visual match to a georeferenced cache tile, used to correct VO/IMU drift. (source: `acceptance_criteria.md`, `solution.md`)
## ArduPilot Plane SITL
Simulation environment used to verify `GPS_INPUT`, spoofing, failsafe, and MAVLink behavior with the production Plane parameter set. (source: `acceptance_criteria.md`, `tests/environment.md`)
## BASALT VIO
Selected production relative visual-inertial odometry candidate. It consumes calibrated navigation-camera frames and IMU data but does not own the product's safety, anchor, cache, or MAVLink semantics. (source: `solution.md`)
## Cache Poisoning
Failure mode where a misaligned onboard-generated tile is written back and later corrupts future satellite-cache anchors. (source: `acceptance_criteria.md`)
## COG Tile
Cloud Optimized GeoTIFF raster object used as the write-new unit for georeferenced service tiles and generated onboard tiles. (source: `solution.md`)
## FDR
Flight Data Recorder. Bounded per-flight recorder for estimates, IMU traces, emitted MAVLink, health telemetry, tile writes, and audit evidence; raw nav/AI frames are not retained. (source: `acceptance_criteria.md`, `solution.md`)
## Generated Tile
In-flight orthorectified tile created from navigation-camera imagery and stored locally for post-flight Satellite Service sync. (source: `acceptance_criteria.md`)
## `GPS_INPUT`
MAVLink message used as the ArduPilot GPS substitute in v1. It carries WGS84 coordinates, fix type, accuracy fields, and ignore flags. (source: `acceptance_criteria.md`, `solution.md`)
## Kimera Backup
BSD-friendly VIO backup candidate if BASALT fails project replay/runtime gates. (source: `solution.md`)
## Mahalanobis Gate
Statistical consistency gate used to reject anchor updates that are inconsistent with the current estimator state and covariance. (source: `solution.md`)
## Nadir Camera
Fixed downward-pointing navigation camera used for localization. It is not gimbal-stabilized. (source: `restrictions.md`)
## OpenVINS Reference
GPLv3 VIO/covariance baseline used for replay comparison and uncertainty calibration; not the default production dependency. (source: `solution.md`)
## Public Replay Dataset
External dataset used to cover validation gaps in the current sample data, especially synchronized nadir camera, IMU, GNSS/ground truth, and reference imagery. MUN-FRL and ALTO are first-choice candidates. (source: `tests/test-data.md`)
## QGroundControl
Supported ground station that receives downsampled status and failsafe messages at 1-2 Hz. (source: `restrictions.md`, `acceptance_criteria.md`)
## Representative Replay Rig
Final acceptance dataset or hardware setup from the target flight profile with synchronized nav camera, FC IMU/attitude/airspeed/altitude, emitted MAVLink, and ground truth. (source: `acceptance_criteria.md`, `tests/test-data.md`)
## Safety/Anchor Wrapper
Project-owned layer around BASALT that calibrates confidence, fuses satellite anchors, owns source labels, handles degraded modes, gates generated tiles, and emits MAVLink outputs. (source: `solution.md`)
## Satellite Service
External Azaion Suite component that prepares the offline satellite cache before flight and ingests generated tiles after landing. The onboard system does not call it in-flight. (source: `restrictions.md`)
## Source Label
Categorical label attached to every estimate: `{satellite_anchored, vo_extrapolated, dead_reckoned}`. (source: `acceptance_criteria.md`)
## Tile Freshness
Runtime age gate for cache tiles: active-conflict sectors require newer imagery than stable rear sectors. (source: `acceptance_criteria.md`)
## Visual Blackout
State where the navigation camera provides no usable localization signal because of clouds, occlusion, whiteout, or similar visual loss. (source: `acceptance_criteria.md`)
## VPR Chunk
Ground-footprint-sized retrieval unit, approximately 600-800 m with overlap, decoupled from the storage tile convention. (source: `acceptance_criteria.md`)
_docs/02_document/module-layout.md
+243
View File
@@ -0,0 +1,243 @@
# Module Layout
**Language**: mixed (Python orchestration + C++ native vision bridges)
**Layout Convention**: src-layout
**Root**: `src/`
**Last Updated**: 2026-05-03
## Layout Rules
1. Each product component owns one top-level directory under `src/`.
2. Shared contracts and cross-cutting helpers live under `src/shared/`.
3. Native hot-path or third-party bridge code lives inside the owning component folder under `native/`.
4. Public API surface per component is limited to `__init__.py`, `types.py`, and `interfaces.py` unless a component entry lists another public file.
5. Tests live under `tests/` by test type and component; implementation tasks must not place tests inside the component tree unless a later test task explicitly changes this layout.
## Per-Component Mapping
### Component: Camera Ingest And Calibration
- **Epic**: AZ-209
- **Directory**: `src/camera_ingest_calibration/`
- **Technologies**: Python, OpenCV 4.x, camera SDK/V4L2/GigE adapter boundary, calibration files, shared geometry/time helpers
- **Public API**:
- `src/camera_ingest_calibration/__init__.py`
- `src/camera_ingest_calibration/types.py`
- `src/camera_ingest_calibration/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/camera_ingest_calibration/internal/*`
- `src/camera_ingest_calibration/_*.py`
- **Owns (exclusive write during implementation)**: `src/camera_ingest_calibration/**`
- **Imports from**: shared/contracts, shared/geo_geometry, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: VIO Adapter, Satellite Service, Anchor Verification, Tile Manager, FDR And Observability
### Component: VIO Adapter
- **Epic**: AZ-213
- **Directory**: `src/vio_adapter/`
- **Native Directory**: `src/vio_adapter/native/`
- **Technologies**: Python adapter, C++ native bridge, BASALT as current backend, Eigen/Sophus or backend-native math stack, OpenCV 4.x, shared time-sync contracts
- **Public API**:
- `src/vio_adapter/__init__.py`
- `src/vio_adapter/types.py`
- `src/vio_adapter/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/vio_adapter/internal/*`
- `src/vio_adapter/_*.py`
- `src/vio_adapter/native/**`
- **Owns (exclusive write during implementation)**:
- `src/vio_adapter/**`
- **Imports from**: Camera Ingest And Calibration, MAVLink And GCS Integration, shared/contracts, shared/geo_geometry, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: Safety And Anchor Wrapper, FDR And Observability
### Component: Safety And Anchor Wrapper
- **Epic**: AZ-216
- **Directory**: `src/safety_anchor_wrapper/`
- **Technologies**: Python state machine, OpenCV geometry helpers, covariance/gating logic, shared DTO contracts, MAVLink output DTOs
- **Public API**:
- `src/safety_anchor_wrapper/__init__.py`
- `src/safety_anchor_wrapper/types.py`
- `src/safety_anchor_wrapper/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/safety_anchor_wrapper/internal/*`
- `src/safety_anchor_wrapper/_*.py`
- **Owns (exclusive write during implementation)**: `src/safety_anchor_wrapper/**`
- **Imports from**: VIO Adapter, Anchor Verification, MAVLink And GCS Integration, Camera Ingest And Calibration, shared/contracts, shared/geo_geometry, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: MAVLink And GCS Integration, Tile Manager, FDR And Observability
### Component: Satellite Service
- **Epic**: AZ-214
- **Directory**: `src/satellite_service/`
- **Native Directory**: `src/satellite_service/native/`
- **Technologies**: Python service adapter, DINOv2-VLAD descriptors, ONNX/TensorRT candidate path, CPU FAISS, offline package sync client
- **Public API**:
- `src/satellite_service/__init__.py`
- `src/satellite_service/types.py`
- `src/satellite_service/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/satellite_service/internal/*`
- `src/satellite_service/_*.py`
- `src/satellite_service/native/**`
- **Owns (exclusive write during implementation)**:
- `src/satellite_service/**`
- **Imports from**: Camera Ingest And Calibration, Tile Manager, Safety And Anchor Wrapper, shared/contracts, shared/geo_geometry, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: Anchor Verification, FDR And Observability
- **Network invariant**: external Satellite Service sync is allowed only pre-flight or post-flight; no mid-flight satellite-provider or suite-service calls.
### Component: Anchor Verification
- **Epic**: AZ-215
- **Directory**: `src/anchor_verification/`
- **Native Directory**: `src/anchor_verification/native/`
- **Technologies**: Python validation pipeline, ALIKED/DISK + LightGlue, OpenCV RANSAC/USAC, SIFT/ORB baseline, native feature-matching bridge
- **Public API**:
- `src/anchor_verification/__init__.py`
- `src/anchor_verification/types.py`
- `src/anchor_verification/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/anchor_verification/internal/*`
- `src/anchor_verification/_*.py`
- `src/anchor_verification/native/**`
- **Owns (exclusive write during implementation)**:
- `src/anchor_verification/**`
- **Imports from**: Satellite Service, Camera Ingest And Calibration, Tile Manager, shared/contracts, shared/geo_geometry, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: Safety And Anchor Wrapper, FDR And Observability
### Component: Tile Manager
- **Epic**: AZ-211
- **Directory**: `src/tile_manager/`
- **Technologies**: Python repository/policy layer, PostgreSQL/PostGIS, GDAL/rasterio COG handling, signed JSON sidecars, OpenCV/GDAL orthorectification, hash/signature validation
- **Public API**:
- `src/tile_manager/__init__.py`
- `src/tile_manager/types.py`
- `src/tile_manager/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/tile_manager/internal/*`
- `src/tile_manager/_*.py`
- **Owns (exclusive write during implementation)**:
- `src/tile_manager/**`
- `migrations/postgresql/cache_*.sql`
- `migrations/seed/cache_*`
- **Imports from**: Camera Ingest And Calibration, Safety And Anchor Wrapper, shared/contracts, shared/geo_geometry, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: Satellite Service, Anchor Verification, FDR And Observability
### Component: MAVLink And GCS Integration
- **Epic**: AZ-210
- **Directory**: `src/mavlink_gcs_integration/`
- **Technologies**: Python, MAVSDK telemetry subscriptions, pymavlink `GPS_INPUT` emission, MAVLink/QGC status messages
- **Public API**:
- `src/mavlink_gcs_integration/__init__.py`
- `src/mavlink_gcs_integration/types.py`
- `src/mavlink_gcs_integration/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/mavlink_gcs_integration/internal/*`
- `src/mavlink_gcs_integration/_*.py`
- **Owns (exclusive write during implementation)**: `src/mavlink_gcs_integration/**`
- **Imports from**: Safety And Anchor Wrapper, shared/contracts, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: VIO Adapter, Safety And Anchor Wrapper, FDR And Observability
### Component: FDR And Observability
- **Epic**: AZ-212
- **Directory**: `src/fdr_observability/`
- **Technologies**: Python append/export layer, PostgreSQL event index, CBOR segment payloads, optional Parquet export, structured logging/health events
- **Public API**:
- `src/fdr_observability/__init__.py`
- `src/fdr_observability/types.py`
- `src/fdr_observability/interfaces.py`
- **Internal (do NOT import from other components)**:
- `src/fdr_observability/internal/*`
- `src/fdr_observability/_*.py`
- **Owns (exclusive write during implementation)**:
- `src/fdr_observability/**`
- `migrations/postgresql/fdr_*.sql`
- `migrations/seed/fdr_*`
- **Imports from**: shared/contracts, shared/time_sync, shared/config, shared/errors, shared/telemetry
- **Consumed by**: all runtime components
## Shared / Cross-Cutting
### shared/contracts
- **Epic**: AZ-206
- **Directory**: `src/shared/contracts/`
- **Technologies**: Python typed DTOs, schema/contract definitions, Markdown API-contract documents
- **Purpose**: Shared DTOs, protocol shapes, schemas, and public contract exports.
- **Owned by**: initial structure and shared-contract tasks under AZ-206.
- **Consumed by**: all components.
### shared/geo_geometry
- **Epic**: AZ-207
- **Directory**: `src/shared/geo_geometry/`
- **Technologies**: Python geometry utilities, OpenCV 4.x, WGS84/local-frame math, homography/covariance conversions
- **Purpose**: WGS84/local conversions, GSD, camera footprint projection, homography/covariance unit conversion, and distance calculations.
- **Owned by**: shared geometry task under AZ-207.
- **Consumed by**: Camera Ingest And Calibration, Safety And Anchor Wrapper, Anchor Verification, Tile Manager.
### shared/time_sync
- **Epic**: AZ-207
- **Directory**: `src/shared/time_sync/`
- **Technologies**: Python timestamp utilities, monotonic-clock validation, MAVLink/camera timestamp normalization, replay ordering checks
- **Purpose**: Monotonic timestamp checks, frame-to-IMU alignment, clock-domain metadata, replay ordering, and gap/jitter metrics.
- **Owned by**: time-sync task under AZ-207.
- **Consumed by**: Camera Ingest And Calibration, VIO Adapter, MAVLink And GCS Integration, FDR And Observability.
### shared/config
- **Epic**: AZ-208
- **Directory**: `src/shared/config/`
- **Technologies**: Python configuration loader, environment variables, `.env.example`, startup readiness validation
- **Purpose**: Runtime profile loading, environment validation, typed settings, and startup readiness inputs.
- **Owned by**: runtime configuration task under AZ-208.
- **Consumed by**: all runtime components.
### shared/errors
- **Epic**: AZ-208
- **Directory**: `src/shared/errors/`
- **Technologies**: Python exception/result envelope types, shared error categories, fail-fast helpers
- **Purpose**: Error categories, result envelopes, fail-fast helpers, and non-silent exception contracts.
- **Owned by**: runtime error contract task under AZ-208.
- **Consumed by**: all components.
### shared/telemetry
- **Epic**: AZ-208
- **Directory**: `src/shared/telemetry/`
- **Technologies**: Python structured logging, metrics labels, health event DTOs, FDR-safe telemetry metadata
- **Purpose**: Structured logging, metrics labels, health event shapes, and FDR-safe event metadata helpers.
- **Owned by**: observability/config contract task under AZ-208.
- **Consumed by**: all components.
## Allowed Dependencies (layering)
Read top-to-bottom; an upper layer may import from a lower layer but never the reverse.
| Layer | Components | May import from |
|-------|------------|-----------------|
| 4. Runtime Output / Coordination | Safety And Anchor Wrapper, MAVLink And GCS Integration, FDR And Observability | 1, 2, 3 public interfaces |
| 3. Perception / Satellite Anchor | VIO Adapter, Satellite Service, Anchor Verification | 1, 2 public interfaces |
| 2. Data Ingest / Persistence | Camera Ingest And Calibration, Tile Manager | 1 |
| 1. Shared / Foundation | shared/contracts, shared/geo_geometry, shared/time_sync, shared/config, shared/errors, shared/telemetry | none |
Violations of this table are Architecture findings in code-review Phase 7 and are High severity.
## Out-of-Product E2E Test Suite
The e2e replay/SITL/Jetson validation suite is not a product component and must not receive Step 6 product implementation tasks. It owns test-support artifacts under `tests/blackbox/**`, `tests/e2e/**`, `e2e/replay/**`, and `e2e/reports/**`, and it exercises the runtime only through public file, MAVLink, cache, status, and FDR interfaces.
- **Technologies**: Python, pytest-style runner, Docker/compose, pymavlink/log parser, ArduPilot Plane SITL, QGC observer/log parser, CSV/Markdown reports
## Self-Verification
- Every runtime component under `_docs/02_document/components/` has a mapping entry.
- Cross-cutting epics AZ-206, AZ-207, and AZ-208 have shared ownership entries.
- Layering covers all components and keeps shared code at the bottom.
- Component-owned paths do not overlap; native bridge paths live inside the component that owns them.
- Paths follow the project `src/` layout already confirmed by `AZ-219_initial_structure`.
_docs/02_document/risk_mitigations.md
+275
View File
@@ -0,0 +1,275 @@
# Risk Assessment — Architecture Review — Iteration 01
## Evaluator Pass Summary
| Check | Result | Notes |
|-------|--------|-------|
| Single Responsibility | Pass | Components each own one primary concern: ingest, VIO, safety, Satellite Service sync/retrieval, verification, Tile Manager storage/generation, MAVLink, FDR, validation |
| Dumb Code / Smart Data | Pass | Complex behavior is mostly expressed through DTOs, mode labels, covariance fields, manifests, and gates |
| Interface Consistency | Pass with fix | Safety wrapper no longer directly depends on Tile Manager for anchor acceptance; cache freshness/provenance travels through `AnchorDecision` |
| Circular Dependencies | Pass with caution | Runtime flow is acyclic at component ownership level; MAVLink remains a bidirectional protocol adapter but owns no localization policy |
| Missing Interactions | Pass | Pre-VIO occlusion, IMU-only blackout, relocalization, tile writes, FDR, and SITL validation are all represented |
| Security Considerations | Pass | Signed cache sidecars, source/system ID checks, spoofing rejection, and no in-flight satellite-provider or Satellite Service access are covered |
| Performance Bottlenecks | Pass | Jetson latency, VPR/local matching, FDR append pressure, PostgreSQL availability, and thermal limits are identified |
| API Contracts | Pass | Core DTO handoffs are documented: `FramePacket`, `VioStatePacket`, `AnchorDecision`, `PositionEstimate`, `FdrEvent` |
## Risk Scoring Matrix
| | Low Impact | Medium Impact | High Impact |
|--|------------|---------------|-------------|
| **High Probability** | Medium | High | Critical |
| **Medium Probability** | Low | Medium | High |
| **Low Probability** | Low | Low | Medium |
## Acceptance Criteria by Risk Level
| Level | Action Required |
|-------|-----------------|
| Low | Accepted and monitored |
| Medium | Mitigation plan required before implementation |
| High | Mitigation + contingency plan required, reviewed during implementation |
| Critical | Must be resolved before proceeding to next planning step |
## Risk Register
| ID | Risk | Category | Probability | Impact | Score | Mitigation | Owner | Status |
|----|------|----------|-------------|--------|-------|------------|-------|--------|
| R01 | ADTi 20MP 20L V1 public specs conflict with planning assumptions for resolution, FPS, lens, interface, and temperature | Technical / External | Medium | High | High | Pin manufacturer datasheet and exact lens/interface before implementation; make camera calibration/spec task a bootstrap blocker | Camera ingest/calibration | Mitigated by gate |
| R02 | BASALT may underperform or lose tracking on nadir fixed-wing low-parallax terrain | Technical | Medium | High | High | Public replay with MUN-FRL/ALTO/Kagaru/EPFL where applicable, representative target replay, OpenVINS reference comparison, Kimera backup path | VIO adapter | Mitigated by validation |
| R03 | BASALT confidence/covariance may under-report real error | Safety | Medium | High | High | Wrapper owns covariance calibration; compare against ground truth, satellite residuals, and OpenVINS reference; never emit optimistic `horiz_accuracy` | Safety/anchor wrapper | Mitigated by wrapper design |
| R04 | Total occlusion detector may false-negative and feed unusable frames into VIO | Safety / Technical | Medium | High | High | Conservative pre-VIO occlusion gate, FDR status, tests for total blackout, and fallback to IMU-only `dead_reckoned` mode | Camera ingest/calibration | Mitigated by spec/test |
| R05 | IMU-only blackout propagation could be trusted too long | Safety | Medium | High | High | Monotonic covariance growth, `dead_reckoned` label, `fix_type=0`/`horiz_accuracy=999.0` when >30 s or covariance >500 m | Safety/anchor wrapper | Mitigated by AC gate |
| R06 | DINOv2-VLAD + ALIKED/DISK-LightGlue exceeds Jetson latency/memory budget | Performance | Medium | High | High | Trigger-only execution, CPU FAISS first, top-K caps, model profiling, TensorRT only after fidelity checks | Satellite Service / Anchor verification | Mitigated by profiling gates |
| R07 | PostgreSQL/PostGIS local DB is unavailable or too heavy for onboard runtime | Technical / Operational | Medium | High | High | Run local onboard PostgreSQL, health-check before flight, keep large payloads in files, fail mission cache validation if DB unavailable | Tile Manager / FDR | Mitigated by deployment gates |
| R08 | Generated tile cache poisoning corrupts future anchors | Security / Safety | Low | High | Medium | Sigma gate, provenance sidecars, post-flight Satellite Service voting, no direct promotion to trusted basemap | Tile Manager | Mitigated by policy |
| R09 | Public datasets do not cover final target terrain or commercial license needs | External / Schedule | Medium | Medium | Medium | Use public data for de-risking only; representative synchronized target data remains mandatory for acceptance | Validation harness | Mitigated by acceptance rule |
| R10 | MAVLink `GPS_INPUT` parameters or Plane behavior differs from assumptions | Integration | Medium | High | High | Plane SITL release gate with production parameters, spoofing/failsafe tests, raw field validation with pymavlink | MAVLink/GCS integration | Mitigated by SITL gate |
| R11 | FDR appends or PostgreSQL indexing interferes with hot-path latency | Performance | Medium | Medium | Medium | Append asynchronously, use CBOR payload segments for high-volume data, keep PostgreSQL as event index/query surface | FDR/observability | Mitigated by design |
| R12 | GPL/non-commercial tooling accidentally enters production or acceptance evidence | Legal / Compliance | Low | High | Medium | Keep OpenVINS/ORB-SLAM3 reference-only; license-tag datasets before CI; SuperPoint only after legal approval | Validation harness / Architecture | Mitigated by gates |
## Detailed Risk Analysis
### R01: Camera Specification Mismatch
**Description**: Public ADTi pages show 5456 x 3632 stills, 2 fps continuous capture, Sony E mount, and -10..40 C operation. The project needs the exact production lens, camera interface, sustained capture behavior, thermal behavior, and calibration model.
**Trigger conditions**: Manufacturer documentation or hardware testing contradicts assumed FPS, interface, temperature, or lens characteristics.
**Affected components**: Camera ingest/calibration, VIO adapter, separate e2e test suite, deployment procedures.
**Mitigation strategy**:
1. Make camera specification verification a bootstrap task.
2. Require manufacturer datasheet or hardware measurement before implementation claims 3 fps or hot-environment operation.
3. Version calibration data by exact camera/lens/interface.
**Contingency plan**: Reduce frame rate assumptions, adjust latency tests, or select a different navigation camera/lens/interface.
**Residual risk after mitigation**: Medium.
**Documents updated**: `glossary.md`, `architecture.md`, `components/01_camera_ingest_calibration/description.md`, `deployment/deployment_procedures.md`.
---
### R02: BASALT Nadir Fixed-Wing Fit
**Description**: BASALT is a strong VIO candidate, but fixed downward cameras over planar terrain can cause low-parallax and texture-degeneracy cases.
**Trigger conditions**: Public or representative replay shows high drift, frequent tracking loss, or poor initialization.
**Affected components**: VIO adapter, safety/anchor wrapper, separate e2e test suite.
**Mitigation strategy**:
1. Run MUN-FRL first for synchronized nadir camera + IMU + ground truth.
2. Add ALTO/Kagaru/EPFL slices where available for aerial/fixed-wing realism.
3. Compare against OpenVINS reference and Kimera backup.
**Contingency plan**: Keep Kimera backup or build a project-owned fallback estimator around OpenCV + IMU only after replay evidence requires it.
**Residual risk after mitigation**: Medium.
**Documents updated**: `architecture.md`, `components/02_vio_adapter/description.md`, `tests/test-data.md`.
---
### R03: Covariance Under-Reporting
**Description**: Incorrect confidence is more dangerous than no estimate because the flight controller may trust a false fix.
**Trigger conditions**: Replay error exceeds reported covariance, or anchors are accepted despite inconsistent residuals.
**Affected components**: Safety/anchor wrapper, MAVLink/GCS integration, FDR/observability.
**Mitigation strategy**:
1. Make wrapper covariance the product authority, not BASALT raw confidence.
2. Validate calibration against ground truth, satellite residuals, and OpenVINS reference.
3. Map `horiz_accuracy` so it never under-reports the 95% semi-major covariance axis.
**Contingency plan**: Degrade to no-fix sooner and require operator relocalization or mission abort behavior.
**Residual risk after mitigation**: Medium.
**Documents updated**: `architecture.md`, `components/03_safety_anchor_wrapper/description.md`, `tests/blackbox-tests.md`.
---
### R04: Total Occlusion Detection Failure
**Description**: If total occlusion is not detected before VIO, BASALT may receive unusable frames and produce misleading state updates.
**Trigger conditions**: Lens cover, cloud/whiteout, decode failure, underexposure/overexposure, or textureless frame reaches VIO as usable.
**Affected components**: Camera ingest/calibration, safety/anchor wrapper, VIO adapter.
**Mitigation strategy**:
1. Camera ingest exposes `OcclusionReport` and sets `usable_for_vio=false` for total occlusion/blackout.
2. Total occlusion bypasses BASALT for that frame.
3. Safety wrapper switches to IMU-only `dead_reckoned` propagation with monotonic covariance growth.
**Contingency plan**: Tune detector conservatively and accept temporary false-positive IMU-only degradation over false VIO confidence.
**Residual risk after mitigation**: Medium.
**Documents updated**: `components/01_camera_ingest_calibration/description.md`, `components/03_safety_anchor_wrapper/description.md`, `system-flows.md`, `diagrams/flows/flow_normal_localization.md`, `tests/resilience-tests.md`.
---
### R05: IMU-Only Mode Over-Trust
**Description**: IMU-only propagation drifts quickly and must be treated as an emergency bridge, not a long-duration solution.
**Trigger conditions**: Blackout lasts longer than 30 seconds or covariance exceeds 500 m.
**Affected components**: Safety/anchor wrapper, MAVLink/GCS integration, FDR/observability.
**Mitigation strategy**:
1. Emit `source_label=dead_reckoned` during IMU-only mode.
2. Grow covariance monotonically.
3. Emit `fix_type=0`, `horiz_accuracy=999.0`, and `VISUAL_BLACKOUT_FAILSAFE` at thresholds.
**Contingency plan**: Stop publishing valid fixes and require relocalization/operator action.
**Residual risk after mitigation**: Low.
**Documents updated**: `components/03_safety_anchor_wrapper/description.md`, `system-flows.md`, `tests/blackbox-tests.md`, `tests/resilience-tests.md`, `tests/traceability-matrix.md`.
---
### R06: Trigger Path Performance
**Description**: DINOv2-VLAD and learned local matching can exceed Jetson latency/memory limits.
**Trigger conditions**: Relocalization exceeds p95 latency, memory budget, or causes thermal throttling.
**Affected components**: Satellite Service, anchor verification, separate e2e test suite.
**Mitigation strategy**:
1. Keep VPR/local matching trigger-based.
2. Use CPU FAISS first and bounded top-K.
3. Accept optimized engines only after descriptor-fidelity tests pass.
**Contingency plan**: Reduce descriptor resolution/model size, reduce top-K, or fall back to classical features for emergency operation.
**Residual risk after mitigation**: Medium.
**Documents updated**: `architecture.md`, `components/04_satellite_retrieval/description.md`, `components/05_anchor_verification/description.md`, `tests/performance-tests.md`.
---
### R07: Onboard PostgreSQL/PostGIS Availability
**Description**: PostgreSQL/PostGIS is now the structured metadata store. If local DB availability or resource use is poor, cache/FDR queries may fail.
**Trigger conditions**: Local DB does not start, DB files corrupt, DB consumes too much memory/I/O, or migrations fail.
**Affected components**: Tile Manager, FDR/observability, deployment procedures.
**Mitigation strategy**:
1. Require local onboard PostgreSQL health check before flight.
2. Store large imagery/descriptors/CBOR payloads as files, not DB blobs.
3. Treat DB unavailability as a mission-cache validation blocker.
**Contingency plan**: Abort mission-cache activation and run only no-cache degraded modes or resync/rebuild DB before flight.
**Residual risk after mitigation**: Medium.
**Documents updated**: `data_model.md`, `architecture.md`, `components/06_cache_tile_lifecycle/description.md`, `components/08_fdr_observability/description.md`, `deployment/environment_strategy.md`.
---
### R08: Cache Poisoning
**Description**: A bad generated tile could be written back and later used as a trusted anchor.
**Trigger conditions**: Generated tile is promoted despite high parent covariance, stale source, bad sidecar, or inconsistent overlap voting.
**Affected components**: Tile Manager, safety/anchor wrapper, Satellite Service integration.
**Mitigation strategy**:
1. Require tile-write sigma gates.
2. Store generated tiles as candidates with signed sidecars.
3. Promote only through post-flight Satellite Service validation/voting.
**Contingency plan**: Quarantine generated tiles and invalidate affected cache regions.
**Residual risk after mitigation**: Low.
**Documents updated**: `architecture.md`, `components/06_cache_tile_lifecycle/description.md`, `tests/security-tests.md`.
---
### R09: Dataset Coverage / Licensing
**Description**: Public datasets may not match target terrain, may lack raw synchronized IMU, or may have non-commercial restrictions.
**Trigger conditions**: MUN-FRL/ALTO/Kagaru/EPFL slices are unavailable, unrepresentative, or license-incompatible for acceptance.
**Affected components**: Validation harness, VIO adapter, anchor verification.
**Mitigation strategy**:
1. Use public datasets for de-risking only.
2. License-tag datasets before CI jobs.
3. Require representative synchronized target data for final acceptance.
**Contingency plan**: Collect a target replay dataset before final acceptance.
**Residual risk after mitigation**: Medium.
**Documents updated**: `tests/test-data.md`, `deployment/environment_strategy.md`, `deployment/ci_cd_pipeline.md`.
---
### R10: Plane `GPS_INPUT` Integration
**Description**: ArduPilot Plane EKF and `GPS_INPUT` handling may differ from assumptions, especially around accuracy fields, ignore flags, velocity fields, and spoofing transitions.
**Trigger conditions**: Plane SITL rejects or mishandles emitted `GPS_INPUT`, or QGC status is insufficient.
**Affected components**: MAVLink/GCS integration, safety/anchor wrapper, separate e2e test suite.
**Mitigation strategy**:
1. Use pymavlink for exact `GPS_INPUT` field control.
2. Gate release on Plane SITL with production parameters.
3. Validate spoofing/failsafe and QGC status behavior.
**Contingency plan**: Adjust parameter guidance/output fields before hardware deployment.
**Residual risk after mitigation**: Medium.
**Documents updated**: `components/07_mavlink_gcs_integration/description.md`, `tests/environment.md`, `deployment/ci_cd_pipeline.md`.
## Architecture/Component Changes Applied
| Risk ID | Document Modified | Change Description |
|---------|-------------------|--------------------|
| R04 | `components/01_camera_ingest_calibration/description.md` | Added explicit `detect_occlusion`, `OcclusionReport`, and pre-VIO bypass behavior |
| R04/R05 | `components/03_safety_anchor_wrapper/description.md` | Added `propagate_imu_only`, `total_occlusion`, monotonic covariance behavior, and no direct Tile Manager dependency |
| R07 | `data_model.md` | Replaced embedded DB references with PostgreSQL/PostGIS structured metadata and CBOR FDR payload segments |
| R07 | `architecture.md` | Added PostgreSQL/PostGIS ADR and FDR storage decision |
| R05 | `tests/blackbox-tests.md` / `tests/resilience-tests.md` | Made total occlusion and IMU-only blackout behavior explicit |
## Summary
**Total risks identified**: 12
**Critical**: 0 | **High**: 7 | **Medium**: 5 | **Low**: 0
**Risks mitigated this iteration**: 12
**Risks requiring user decision**: None immediately. Future decisions are tied to exact camera hardware proof, dataset license approval, and representative data collection timing.
_docs/02_document/system-flows.md
+321
View File
@@ -0,0 +1,321 @@
# GPS-Denied Onboard Localization — System Flows
## Flow Inventory
| # | Flow Name | Trigger | Primary Components | Criticality |
|---|-----------|---------|--------------------|-------------|
| F1 | Pre-flight cache preparation | Operator sync before mission | Satellite Service, Tile Manager | High |
| F2 | Normal frame processing | Navigation frame + FC telemetry | Camera ingest, VIO adapter, safety/anchor wrapper, MAVLink, FDR | High |
| F3 | Satellite relocalization | Cold start, VO failure, sharp turn, covariance growth, stale anchor | Satellite Service, anchor verification, safety/anchor wrapper | High |
| F4 | Visual blackout / spoofing degraded mode | Image-quality failure and GPS health failure | Camera ingest, MAVLink telemetry, safety/anchor wrapper, QGC, FDR | Critical |
| F5 | Generated tile lifecycle | High-confidence pose + usable frame | Camera ingest, safety/anchor wrapper, Tile Manager, FDR | Medium |
| F6 | Post-flight sync and audit | Landing / operator offload | Tile Manager, Satellite Service, FDR | Medium |
| F7 | E2E validation replay | Test-suite invocation | Separate e2e test suite, system runtime, public datasets, SITL | High |
## Flow Dependencies
| Flow | Depends On | Shares Data With |
|------|------------|------------------|
| F1 | Satellite Service cache export and Tile Manager validation | F2, F3, F5 |
| F2 | F1 for cache availability; FC telemetry | F3, F4, F5, FDR |
| F3 | F1 cache/index; F2 state estimate | F2, F5 |
| F4 | F2 telemetry and quality signals | F2, QGC/FDR |
| F5 | Accepted state/covariance from F2/F3 | F6 |
| F6 | F5 generated tiles and FDR | Satellite Service |
| F7 | Test fixtures and selected execution environment | All flows |
---
## Flow F1: Pre-Flight Cache Preparation
### Description
Before flight, the Satellite Service imports an offline cache package for the operational area, including COG tiles, manifests, sidecars, VPR chunks, descriptors, and FAISS index files. No Satellite Service or satellite-provider calls are allowed during flight.
### Preconditions
- Operational area and sector freshness classification are known.
- Cache imagery meets 0.5 m/px minimum and ideally 0.3 m/px.
- Cache package fits storage budget or has approved split descriptor budget.
### Sequence Diagram
```mermaid
sequenceDiagram
participant Operator
participant SatelliteService
participant TileManager
Operator->>SatelliteService: Request mission cache
SatelliteService-->>TileManager: COG tiles + manifests + sidecars
TileManager->>TileManager: Verify signatures, hashes, freshness, resolution
TileManager-->>SatelliteService: Local cache/index ready
TileManager-->>Operator: Cache validation report
```
### Data Flow
| Step | From | To | Data | Format |
|------|------|----|------|--------|
| 1 | Satellite Service | Tile Manager | Tiles and metadata | COG + PostgreSQL/PostGIS manifest + signed JSON sidecars |
| 2 | Tile Manager | Satellite Service | Descriptor/index readiness | FAISS index + descriptor sidecars |
| 3 | Tile Manager | Operator/FDR | Validation report | Markdown/CSV/log |
### Error Scenarios
| Error | Where | Detection | Recovery |
|-------|-------|-----------|----------|
| Stale tile | Cache validation | Capture date exceeds sector threshold | Reject/down-confidence tile |
| Hash mismatch | Cache validation | Sidecar hash mismatch | Reject tile and report security event |
| Cache too large | Cache load | Storage accounting > budget | Require cache rebuild or approved split budget |
### Performance Expectations
| Metric | Target | Notes |
|--------|--------|-------|
| Runtime network calls | 0 | No in-flight Satellite Service or provider calls |
| Cache load | Within cold-start budget contribution | Exact threshold set during implementation |
---
## Flow F2: Normal Frame Processing
### Description
During normal flight, the system processes each navigation frame and FC telemetry sample. The camera component first checks for total occlusion/blackout. Usable frames go to the VIO adapter; total-occlusion frames bypass VIO and send the wrapper into IMU-only degraded propagation.
### Preconditions
- Camera calibration/extrinsics are loaded.
- VIO adapter and wrapper are initialized.
- FC telemetry stream is healthy.
### Sequence Diagram
```mermaid
sequenceDiagram
participant CameraIngest
participant FCTelemetry
participant BasaltAdapter
participant SafetyWrapper
participant MavlinkOutput
participant FDR
CameraIngest->>CameraIngest: Total occlusion / blackout check
CameraIngest->>BasaltAdapter: Usable frame + timestamp + calibration
CameraIngest-->>SafetyWrapper: Degradation signal if total occlusion
FCTelemetry->>BasaltAdapter: IMU/attitude/altitude
BasaltAdapter-->>SafetyWrapper: Relative VIO state + quality
SafetyWrapper->>SafetyWrapper: Calibrate covariance + source label
SafetyWrapper-->>MavlinkOutput: GPS_INPUT estimate
SafetyWrapper-->>FDR: Estimate + inputs + health
```
### Data Flow
| Step | From | To | Data | Format |
|------|------|----|------|--------|
| 1 | Camera ingest | VIO adapter or safety wrapper | Frame metadata, image, occlusion status | Frame DTO / DegradationSignal |
| 2 | FC telemetry | VIO adapter | IMU/attitude/altitude | MAVLink-derived telemetry DTO |
| 3 | VIO adapter | Safety wrapper | Relative VIO state | VioState DTO |
| 4 | Safety wrapper | MAVLink output | WGS84 estimate | `GPS_INPUT` |
| 5 | Safety wrapper | FDR | Inputs/outputs/audit | FDR segment event |
### Error Scenarios
| Error | Where | Detection | Recovery |
|-------|-------|-----------|----------|
| Total occlusion / blackout | Camera ingest | Occlusion status, exposure/texture/decode checks | Bypass VIO, enter IMU-only `dead_reckoned` propagation |
| Frame unreadable | Camera ingest | Decode/quality failure | Mark visual signal degraded and bypass VIO for that frame |
| VIO quality low | VIO adapter | Tracking/completion metrics | Trigger relocalization or dead reckoning |
| Covariance grows | Safety wrapper | Covariance threshold | Degrade fix type/source label |
### Performance Expectations
| Metric | Target | Notes |
|--------|--------|-------|
| End-to-end latency | <400 ms p95 | Frame input to emitted estimate |
| Dropped frames | <=10% sustained | Under load |
| Memory | <8 GB shared | Jetson limit |
---
## Flow F3: Satellite Relocalization
### Description
When the state becomes uncertain or disconnected, the system retrieves satellite/cache candidates and accepts an anchor only after local verification and safety gates pass.
### Preconditions
- Offline VPR chunks and FAISS index are loaded.
- Trigger condition is met: cold start, VO failure, sharp turn, disconnected segment, covariance growth, or stale anchor.
### Sequence Diagram
```mermaid
sequenceDiagram
participant SafetyWrapper
participant SatelliteService
participant AnchorVerification
participant TileManager
participant FDR
SafetyWrapper->>SatelliteService: Relocalization request
SatelliteService->>TileManager: Read candidate chunk metadata
SatelliteService-->>AnchorVerification: Top-K candidates
AnchorVerification->>AnchorVerification: ALIKED/DISK+LightGlue + RANSAC
AnchorVerification-->>SafetyWrapper: Accepted/rejected anchor
SafetyWrapper->>SafetyWrapper: Mahalanobis + freshness + provenance gates
SafetyWrapper-->>FDR: Anchor decision audit
```
### Data Flow
| Step | From | To | Data | Format |
|------|------|----|------|--------|
| 1 | Safety wrapper | Satellite Service | Query frame and prior/covariance | Relocalization DTO |
| 2 | Satellite Service | Anchor verification | Top-K chunks from local cache/index | Candidate list |
| 3 | Anchor verification | Safety wrapper | MRE, inliers, homography, provenance | AnchorDecision DTO |
### Error Scenarios
| Error | Where | Detection | Recovery |
|-------|-------|-----------|----------|
| No good candidate | Retrieval/verification | Low score or failed RANSAC | Continue degraded and request GCS hint after threshold |
| Stale candidate | Tile Manager | Capture date gate | Reject/down-confidence |
| Implausible anchor | Safety wrapper | Mahalanobis/impossible velocity gate | Reject and log |
### Performance Expectations
| Metric | Target | Notes |
|--------|--------|-------|
| Invocation frequency | Trigger-based only | Not per-frame |
| Cross-domain MRE | <2.5 px for accepted anchors | AC-2.2 |
---
## Flow F4: Visual Blackout / Spoofing Degraded Mode
### Description
When visual localization is unavailable due to total occlusion/blackout and GPS is denied/spoofed, the wrapper switches to honest IMU-only propagation from the last trusted state and degrades MAVLink output based on covariance/time thresholds.
### Preconditions
- Last trusted state exists.
- FC telemetry continues.
### Sequence Diagram
```mermaid
sequenceDiagram
participant CameraIngest
participant FCTelemetry
participant SafetyWrapper
participant MavlinkOutput
participant QGC
participant FDR
CameraIngest-->>SafetyWrapper: Total occlusion / visual blackout signal
FCTelemetry-->>SafetyWrapper: GPS health/spoofing signal
SafetyWrapper->>SafetyWrapper: IMU-only propagation + monotonic covariance growth
SafetyWrapper->>SafetyWrapper: Switch source_label to dead_reckoned
SafetyWrapper-->>MavlinkOutput: Degraded GPS_INPUT
SafetyWrapper-->>QGC: VISUAL_BLACKOUT_IMU_ONLY / FAILSAFE
SafetyWrapper-->>FDR: Blackout and spoofing audit events
```
### Error Scenarios
| Error | Where | Detection | Recovery |
|-------|-------|-----------|----------|
| Blackout >30 s | Safety wrapper | Timer threshold | Emit no-fix/failsafe |
| Covariance >500 m | Safety wrapper | Covariance threshold | `fix_type=0`, `horiz_accuracy=999.0` |
| Spoofed GPS recovers | Safety wrapper | FC health + visual consistency gate | Re-enable only after required stable interval and visual/satellite consistency |
### Performance Expectations
| Metric | Target | Notes |
|--------|--------|-------|
| Mode transition | <=1 processed frame or <=400 ms | AC-3.5 |
| QGC status | 1-2 Hz | Downsampled operator awareness |
---
## Flow F5: Generated Tile Lifecycle
### Description
When pose confidence is strong enough, the system orthorectifies navigation imagery into write-new generated tiles and records quality/provenance sidecars.
### Preconditions
- Parent pose covariance passes tile-write gate.
- Frame quality supports orthorectification.
### Data Flow
| Step | From | To | Data | Format |
|------|------|----|------|--------|
| 1 | Safety wrapper | Tile Manager | Pose/covariance + frame metadata | TileGenerationRequest |
| 2 | Tile Manager | Local storage | Orthorectified generated COG + sidecar | COG + signed JSON |
| 3 | Tile Manager | FDR | Tile write event | FDR event |
### Error Scenarios
| Error | Where | Detection | Recovery |
|-------|-------|-----------|----------|
| Parent covariance too high | Safety wrapper | Sigma gate | Do not write tile |
| Duplicate sector | Tile Manager | Spatial deduplication | Keep latest/highest-quality tile |
| Sidecar write failure | Tile Manager | I/O error | Log and do not mark tile eligible |
---
## Flow F6: Post-Flight Sync And Audit
### Description
After landing, generated tiles and FDR evidence are exported through Satellite Service sync for ingest and incident analysis.
### Data Flow
| Step | From | To | Data | Format |
|------|------|----|------|--------|
| 1 | Tile Manager | Satellite Service | Generated tile package | COG + sidecar + manifest delta |
| 2 | FDR | Operator/audit tools | Mission replay evidence | Segmented logs + optional Parquet export |
### Error Scenarios
| Error | Where | Detection | Recovery |
|-------|-------|-----------|----------|
| Upload unavailable | Post-flight sync | Network/service failure | Retain package for retry |
| Candidate rejected by Service voting | Satellite Service | Ingest rules | Keep as candidate/soft trust, not trusted basemap |
---
## Flow F7: Validation Replay
### Description
The separate e2e test suite runs deterministic still-image, public dataset, SITL, Jetson, and representative replay scenarios against public interfaces.
### Preconditions
- Test data and expected results are pinned.
- Execution mode is selected: Docker/replay and local Jetson hardware.
### Data Flow
| Step | From | To | Data | Format |
|------|------|----|------|--------|
| 1 | E2E test suite | Runtime | Images/telemetry/cache fixtures | File/stream/MAVLink |
| 2 | Runtime | E2E test suite | GPS_INPUT/FDR/status | MAVLink/log files |
| 3 | E2E test suite | Reports | Pass/fail metrics | CSV/Markdown |
### Performance Expectations
| Metric | Target | Notes |
|--------|--------|-------|
| PR smoke | <=15 min | Still-image/cache/SITL subset |
| Release gate | Hardware-dependent | Jetson and representative replay required |
_docs/02_document/tests/blackbox-tests.md
+99 -429
View File
@@ -2,502 +2,172 @@
## Positive Scenarios
### FT-P-01: End-to-End Position Accuracy — 50m Threshold
### FT-P-01: Still-Image Frame Center Geolocation
**Summary**: Validate that the system estimates WGS84 frame centers for the provided 60-image nadir dataset.
**Traces to**: AC-1.1, AC-1.2, AC-6.3, AC-8.1
**Summary**: Validate that ≥80% of frame positions are within 50m of ground truth GPS across a full 60-frame flight sequence.
**Traces to**: AC-01 (80% within 50m)
**Category**: Position Accuracy
**Preconditions**:
- System running with SITL ArduPilot (GPS_TYPE=14)
- Camera replay serving flight-sequence-60 at 0.7fps
- Satellite tiles for test area loaded
- System has completed startup (first satellite match done)
- Offline satellite cache fixture is available for the sample area.
- Expected results are loaded from `input_data/expected_results/results_report.md`.
**Input data**: flight-sequence-60 (60 frames), coordinates.csv (ground truth), position_accuracy.csv (thresholds)
**Steps**:
**Input data**: `project_60_still_images`, `expected_frame_centers`
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Start session via POST /sessions | HTTP 201 with session ID |
| 2 | Subscribe to SSE stream GET /sessions/{id}/stream | SSE events begin at ~1Hz |
| 3 | Wait for camera-replay to complete all 60 frames (~86s at 0.7fps) | Position events for each processed frame |
| 4 | Collect all position events with lat/lon | 60 position estimates (some frames may have multiple updates) |
| 5 | For each frame: compute haversine distance between estimated and ground truth position | Distance array |
| 6 | Count frames where distance < 50m, compute percentage | ≥80% |
|------|-----------------|--------------------------|
| 1 | Submit `AD000001.jpg` through `AD000060.jpg` with height/camera metadata | System emits one WGS84 estimate per processed image |
| 2 | Compare each estimate to the mapped expected coordinate | Per-frame error is reported in meters |
**Expected outcome**: ≥48 of 60 frames have position error < 50m from ground truth in coordinates.csv
**Max execution time**: 120s
**Expected outcome**: At least 80% of images are within 50 m and at least 50% are within 20 m.
**Max execution time**: 15 minutes for the 60-image replay on the local replay environment.
---
### FT-P-02: End-to-End Position Accuracy — 20m Threshold
### FT-P-02: Position Confidence Output Contract
**Summary**: Validate that ≥60% of frame positions are within 20m of ground truth GPS.
**Traces to**: AC-02 (60% within 20m)
**Category**: Position Accuracy
**Summary**: Validate that every emitted position estimate includes confidence and source-label fields required by the public contract.
**Preconditions**: Same as FT-P-01
**Traces to**: AC-1.3, AC-1.4, AC-4.4, AC-4.5
**Input data**: flight-sequence-60, coordinates.csv, position_accuracy.csv
**Category**: Position Confidence
**Steps**:
**Preconditions**:
- Same fixture setup as FT-P-01.
**Input data**: `project_60_still_images`, `expected_frame_centers`
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Reuse position data from FT-P-01 run (or re-run) | 60 position estimates |
| 2 | Count frames where distance < 20m, compute percentage | ≥60% |
|------|-----------------|--------------------------|
| 1 | Submit the 60-image replay | System emits estimates frame-by-frame, not batched |
| 2 | Inspect public output fields | Each estimate contains WGS84 coordinate, 95% covariance semi-major axis, source label, and `last_satellite_anchor_age_ms` |
| 3 | Submit a later correction for a prior frame if available | System emits updated estimate with timestamp and covariance without corrupting newer estimates |
**Expected outcome**: ≥36 of 60 frames have position error < 20m
**Max execution time**: 120s (shared with FT-P-01)
**Expected outcome**: 100% of emitted estimates include required confidence fields; no `horiz_accuracy` equivalent under-reports the 95% covariance semi-major axis.
**Max execution time**: 15 minutes.
---
### FT-P-03: No Single Frame Exceeds Maximum Error
### FT-P-03: BASALT VIO Replay With Synchronized Video/Telemetry
**Summary**: Validate that no individual frame position estimate exceeds 100m error.
**Traces to**: AC-01, AC-02 (implicit: no catastrophic outliers)
**Category**: Position Accuracy
**Summary**: Validate that BASALT + safety/anchor wrapper can process synchronized nadir video, IMU, and trajectory telemetry and produce frame-by-frame estimates with honest confidence.
**Preconditions**: Same as FT-P-01
**Traces to**: AC-1.3, AC-2.1a, AC-2.2, AC-4.1, AC-4.2
**Input data**: flight-sequence-60, coordinates.csv, position_accuracy.csv
**Category**: VO / IMU Propagation
**Steps**:
**Preconditions**:
- Derkachi replay fixture is mounted from `input_data/flight_derkachi/`.
- `flight_derkachi.mp4` is readable as cropped nadir video: 880 x 720, 30 fps, approximately 490.07 s.
- `data_imu.csv` contains monotonic 10 Hz `Time`, `timestamp(ms)`, `SCALED_IMU2.*`, and `GLOBAL_POSITION_INT.*` fields for 4,900 rows.
- Camera intrinsics, lens distortion, and camera-to-body transform are either pinned or the run is marked as calibration-limited.
- Public synchronized dataset slice remains useful for calibrated final comparison. Strongest candidates: MUN-FRL, ALTO, EPFL fixed-wing, Kagaru; EuRoC/UZH FPV are proxy-only.
**Input data**: `derkachi_video_telemetry`, `public_nadir_vio_candidates`
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Reuse position data from FT-P-01 | 60 position estimates |
| 2 | Find max error across all frames | max(distances) ≤ 100m |
|------|-----------------|--------------------------|
| 1 | Validate Derkachi video/telemetry alignment | Harness accepts the fixture only if MP4 duration and CSV duration differ by <=250 ms and there are exactly 3 video frames per telemetry row |
| 2 | Replay synchronized video frames and IMU stream | System emits frame-by-frame `vo_extrapolated` or `satellite_anchored` estimates without batching |
| 3 | Compare output trajectory to `GLOBAL_POSITION_INT` lat/lon/alt/heading | Error, covariance, source label, and anchor age are reported per segment |
| 4 | Compare calibrated public/representative replay against ground truth when available | BASALT + wrapper does not materially under-report uncertainty relative to error |
| 5 | Compare against OpenVINS reference replay when available | BASALT + wrapper does not materially under-report uncertainty relative to error |
**Expected outcome**: Maximum position error across all 60 frames ≤ 100m
**Max execution time**: 120s (shared with FT-P-01)
**Expected outcome**: Derkachi replay is accepted as a synchronized representative fixture and produces continuous estimates for >95% of normal overlapping frames. Absolute geolocation and covariance pass/fail thresholds are calibration-gated until camera intrinsics, distortion, and camera-to-body transform are pinned. For calibrated datasets, VO homography MRE is <1.0 px where homography validation is applicable.
**Max execution time**: Dataset-dependent, but replay must report per-frame latency.
---
### FT-P-04: VO Drift Between Satellite Anchors
### FT-P-04: Satellite Service And Anchor Verification
**Summary**: Validate cumulative VO drift stays below 100m between consecutive satellite correction events.
**Traces to**: AC-03 (drift < 100m between anchors)
**Category**: Position Accuracy
**Summary**: Validate that relocalization uses global retrieval plus local verification and emits only verified satellite anchors.
**Preconditions**: Same as FT-P-01; satellite matching active on keyframes
**Traces to**: AC-2.1b, AC-2.2, AC-3.2, AC-3.3, AC-8.6
**Input data**: flight-sequence-60 SSE stream (includes drift_from_anchor field)
**Category**: Satellite Anchor
**Steps**:
**Preconditions**:
- AerialVL/ALTO/VPAir-style public dataset slice or project satellite-cache fixture is available.
- VPR chunks and descriptors are precomputed.
**Input data**: Public aerial localization slice, cache fixture
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Subscribe to SSE stream | Events with drift_from_anchor field |
| 2 | Record drift_from_anchor values over the full sequence | Array of drift values |
| 3 | Find maximum drift_from_anchor value | max(drift) < 100m |
|------|-----------------|--------------------------|
| 1 | Trigger cold-start or relocalization query | System searches CPU FAISS top-K chunks |
| 2 | Present top-K candidates to local verification | System runs ALIKED/DISK+LightGlue and RANSAC |
| 3 | Inspect emitted anchor decision | Accepted anchors include source label, MRE, inlier count, covariance, and tile provenance |
**Expected outcome**: drift_from_anchor never exceeds 100m during the 60-frame sequence
**Max execution time**: 120s
**Expected outcome**: Cross-domain satellite-anchor MRE is <2.5 px for accepted anchors; rejected candidates do not produce `satellite_anchored` estimates.
---
### FT-P-05: GPS_INPUT Message Correctness — Normal Tracking
**Summary**: Validate GPS_INPUT message fields are correctly populated during normal satellite-anchored tracking.
**Traces to**: AC-08 (GPS_INPUT to FC via MAVLink), AC-04 (confidence score)
**Category**: Flight Controller Integration
**Preconditions**: System tracking normally with recent satellite match (<30s)
**Input data**: Normal frame + satellite match; MAVLink capture from mavlink-inspector
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Read captured GPS_INPUT messages from mavlink-inspector | GPS_INPUT messages at 5-10Hz |
| 2 | Verify field: fix_type | fix_type == 3 |
| 3 | Verify field: horiz_accuracy | 1.0 ≤ horiz_accuracy ≤ 50.0 |
| 4 | Verify field: satellites_visible | satellites_visible == 10 |
| 5 | Verify fields: lat, lon | Non-zero, within operational area bounds |
| 6 | Verify fields: vn, ve, vd | Populated (non-NaN), magnitude consistent with ~50-70 km/h flight |
**Expected outcome**: All GPS_INPUT fields populated correctly per specification
**Max execution time**: 30s
---
### FT-P-06: Image Registration Rate
**Summary**: Validate that ≥95% of frames in a normal flight are successfully registered by the VO pipeline.
**Traces to**: AC-05 (registration > 95%)
**Category**: Image Processing Quality
**Preconditions**: System running with full 60-frame sequence
**Input data**: flight-sequence-60 SSE stream (vo_status field)
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Subscribe to SSE stream | Events with vo_status field |
| 2 | Count frames where vo_status == "tracking" | ≥57 of 60 |
| 3 | Compute registration rate | ≥95% |
**Expected outcome**: ≥57 of 60 frames report vo_status "tracking"
**Max execution time**: 120s
---
### FT-P-07: Confidence Tier — HIGH
**Summary**: Validate HIGH confidence tier when satellite match is recent and covariance is low.
**Traces to**: AC-04 (confidence score per estimate)
**Category**: Confidence Scoring
**Preconditions**: System running, satellite match completed <30s ago
**Input data**: SSE stream during normal tracking
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Read SSE event immediately after satellite match | confidence field |
| 2 | Verify confidence == "HIGH" | "HIGH" |
| 3 | Read GPS_INPUT fix_type from mavlink-inspector | fix_type == 3 |
**Expected outcome**: Confidence tier is HIGH, fix_type is 3
**Max execution time**: 30s
---
### FT-P-08: Confidence Tier — MEDIUM (VO-only, No Recent Satellite Match)
**Summary**: Validate MEDIUM confidence tier when VO is tracking but no satellite match in >30s.
**Traces to**: AC-04
**Category**: Confidence Scoring
**Preconditions**: System running; satellite tile server paused (returns 503) to prevent new matches; >30s since last match
**Input data**: SSE stream during VO-only tracking
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Pause satellite-tile-server (Docker pause) | No new satellite matches possible |
| 2 | Wait >30s after last satellite match | Confidence should transition |
| 3 | Read SSE event | confidence == "MEDIUM" |
| 4 | Read GPS_INPUT fix_type | fix_type == 3 |
**Expected outcome**: Confidence transitions to MEDIUM; fix_type remains 3
**Max execution time**: 60s
---
### FT-P-09: GPS_INPUT Output Rate
**Summary**: Validate GPS_INPUT messages are sent at 5-10Hz continuously.
**Traces to**: AC-08 (GPS_INPUT via MAVLink), AC-09 (frame-by-frame streaming)
**Category**: Flight Controller Integration
**Preconditions**: System running and producing position estimates
**Input data**: MAVLink capture from mavlink-inspector (10s window)
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Capture GPS_INPUT messages for 10 seconds | N messages |
| 2 | Compute rate: N / 10 | 5 ≤ rate ≤ 10 |
| 3 | Verify no gaps > 300ms between consecutive messages | max gap ≤ 300ms |
**Expected outcome**: Rate is 5-10Hz, no gap exceeds 300ms
**Max execution time**: 15s
---
### FT-P-10: Object Localization
**Summary**: Validate object GPS localization from pixel coordinates via the FastAPI endpoint.
**Traces to**: AC-16 (object localization), AC-17 (trigonometric calculation)
**Category**: Object Localization
**Preconditions**: System running with known UAV position (from GPS-denied estimate); known object ground truth GPS
**Input data**: pixel_x, pixel_y (center of frame = nadir), gimbal_pan_deg=0, gimbal_tilt_deg=-90, zoom_factor=1.0
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | POST /objects/locate with pixel at frame center, gimbal pointing straight down | JSON: { lat, lon, alt, accuracy_m, confidence } |
| 2 | Compute haversine distance between response lat/lon and current UAV position | Should be < accuracy_m (nadir point ≈ UAV position) |
| 3 | Verify accuracy_m is consistent with current system accuracy | accuracy_m > 0, accuracy_m < 100m |
**Expected outcome**: Object location at nadir matches UAV position within accuracy_m
**Max execution time**: 5s
---
### FT-P-11: Coordinate Transform Round-Trip
**Summary**: Validate GPS→NED→pixel→GPS round-trip error is <0.1m.
**Traces to**: AC-18 (WGS84 output)
**Category**: Coordinate Transforms
**Preconditions**: System running, position known
**Input data**: Known GPS coordinate within operational area
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Query system for current position via SSE | lat, lon |
| 2 | POST /objects/locate with frame center pixel, straight-down gimbal | Returned lat, lon |
| 3 | Compute haversine distance between original UAV lat/lon and round-trip result | distance < 0.1m |
**Expected outcome**: Round-trip error < 0.1m
**Max execution time**: 5s
---
### FT-P-12: Startup — GPS_INPUT Within 60 Seconds
**Summary**: Validate the system begins outputting GPS_INPUT messages within 60s of boot.
**Traces to**: AC-11 (startup from last GPS)
**Category**: Startup & Failsafe
**Preconditions**: Fresh system start; SITL ArduPilot running with GLOBAL_POSITION_INT available
**Input data**: MAVLink capture from mavlink-inspector
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Start gps-denied-system container | System boots |
| 2 | Monitor mavlink-inspector for first GPS_INPUT message | Timestamp of first GPS_INPUT |
| 3 | Compute elapsed time from container start to first GPS_INPUT | ≤ 60s |
**Expected outcome**: First GPS_INPUT message arrives within 60s of system start
**Max execution time**: 90s
---
### FT-P-13: Telemetry Output Rate
**Summary**: Validate telemetry NAMED_VALUE_FLOAT messages are sent at 1Hz.
**Traces to**: AC-14 (telemetry to ground station)
**Category**: Telemetry
**Preconditions**: System running normally
**Input data**: MAVLink capture from mavlink-inspector (10s window)
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Capture NAMED_VALUE_FLOAT messages for "gps_conf", "gps_drift", "gps_hacc" over 10s | N messages per name |
| 2 | Verify rate: ~1Hz per metric (8-12 messages per name in 10s) | 0.8-1.2 Hz |
**Expected outcome**: Each telemetry metric sent at ~1Hz
**Max execution time**: 15s
---
### FT-P-14: SSE Stream Schema
**Summary**: Validate SSE position events contain all required fields with correct types.
**Traces to**: AC-14 (streaming to ground station)
**Category**: API & Communication
**Preconditions**: Active session with SSE stream
**Input data**: SSE events from /sessions/{id}/stream
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Subscribe to SSE stream | Events at ~1Hz |
| 2 | Parse event JSON | Valid JSON |
| 3 | Verify fields: type (string), timestamp (ISO8601), lat (float), lon (float), alt (float), accuracy_h (float), confidence (string), drift_from_anchor (float), vo_status (string), last_satellite_match_age_s (float) | All present with correct types |
**Expected outcome**: Every SSE event conforms to the specified schema
**Max execution time**: 10s
---
**Max execution time**: Must be measured as part of performance tests.
## Negative Scenarios
### FT-N-01: Trajectory Direction Change (Frames 32-43)
### FT-N-01: Repetitive Or Low-Texture Imagery
**Summary**: Validate system continues producing position estimates through a trajectory direction change.
**Traces to**: AC-07 (disconnected segments core to system)
**Category**: Resilience & Edge Cases
**Summary**: Validate that visually ambiguous images do not produce confident false satellite anchors.
**Preconditions**: System running; camera-replay set to serve frames 32-43 (direction change area)
**Traces to**: AC-1.4, AC-3.1, AC-NEW-4, AC-8.6
**Input data**: Frames AD000032-043.jpg, coordinates for frames 32-43
**Category**: False Position Prevention
**Steps**:
**Input data**: Repetitive agricultural or low-texture frames from project/public data.
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Camera-replay serves frames 32-43 at 0.7fps | System processes frames |
| 2 | Collect SSE position events for each frame | ≥12 position estimates (one per frame minimum) |
| 3 | Verify no gap >5s without a position update | Continuous output |
|------|-----------------|--------------------------|
| 1 | Submit ambiguous frame or sequence | System either emits degraded `vo_extrapolated`/`dead_reckoned` output or rejects low-confidence anchor |
| 2 | Inspect anchor and confidence outputs | No anchor is accepted unless local verification and covariance gates pass |
**Expected outcome**: System produces position estimates for all frames in the direction-change segment; no prolonged output gap
**Max execution time**: 30s
**Expected outcome**: 0 confident `satellite_anchored` outputs for candidates that fail local verification, freshness, or Mahalanobis gates.
**Max execution time**: 15 minutes per fixture.
---
### FT-N-02: Outlier Frame Handling (350m Gap)
### FT-N-02: GPS Spoofing During Total Visual Blackout
**Summary**: Validate system handles a 350m outlier between consecutive photos without position corruption.
**Traces to**: AC-06 (350m outlier tolerance)
**Category**: Resilience & Edge Cases
**Summary**: Validate that spoofed GPS is not promoted during total camera occlusion/visual blackout and that output degrades honestly before unusable frames reach VIO.
**Preconditions**: System running with normal tracking established; fault injection: camera-replay skips frames to simulate 350m gap
**Traces to**: AC-3.5, AC-5.2, AC-NEW-2, AC-NEW-8
**Input data**: Normal frames followed by a frame 350m away (simulated by frame skip in camera-replay)
**Category**: Spoofing / Blackout
**Steps**:
**Input data**: ArduPilot Plane SITL spoofing trace with camera blackout/total-occlusion frames.
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Normal tracking for 10 frames | Position estimates with <50m error |
| 2 | Camera-replay jumps forward ~350m (skips multiple frames) | System detects discontinuity |
| 3 | Collect position estimates for next 5 frames after the gap | Recovery within 3-5 frames |
| 4 | Verify position error of recovered frames | Error < 100m for first valid frame after recovery |
|------|-----------------|--------------------------|
| 1 | Start normal replay with trusted visual/satellite anchor | System emits normal estimates |
| 2 | Inject full visual blackout/total occlusion and spoofed `GPS_RAW_INT` | Camera gate sets `usable_for_vio=false`, BASALT is bypassed for occluded frames, and system switches to `dead_reckoned` within <=1 processed frame or <=400 ms |
| 3 | Continue blackout beyond thresholds | IMU-only covariance grows monotonically; system degrades fix type and emits failsafe status at specified covariance/time thresholds |
**Expected outcome**: System recovers from 350m outlier; post-recovery position error < 100m
**Max execution time**: 30s
**Expected outcome**: Spoofed GPS is ignored; total occlusion never feeds BASALT as a usable VIO frame; `fix_type=0`, `horiz_accuracy=999.0`, and `VISUAL_BLACKOUT_FAILSAFE` are emitted when covariance >500 m or blackout >30 s.
**Max execution time**: 10 minutes per SITL scenario.
---
### FT-N-03: Invalid Object Localization Request
### FT-N-03: Invalid Or Stale Satellite Cache
**Summary**: Validate API rejects invalid pixel coordinates with HTTP 422.
**Traces to**: AC-16 (object localization)
**Category**: API Error Handling
**Summary**: Validate cache freshness, integrity, and provenance gates.
**Preconditions**: System running with active session
**Traces to**: AC-8.2, AC-8.3, AC-NEW-6, AC-NEW-7
**Input data**: POST /objects/locate with pixel_x=-100, pixel_y=-100
**Category**: Cache Integrity
**Steps**:
**Input data**: `cache_integrity_fixtures`
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | POST /objects/locate with negative pixel coordinates | HTTP 422 |
| 2 | Verify response body contains error description | JSON with "error" or "detail" field |
|------|-----------------|--------------------------|
| 1 | Replay with stale tile manifest | Tile is rejected or down-confidence weighted; no stale tile emits `satellite_anchored` |
| 2 | Replay with hash-mismatched or unsigned manifest | Cache fixture is rejected and security event is logged |
| 3 | Replay generated tile with weak parent-pose covariance | Tile is not promoted beyond allowed trust level |
**Expected outcome**: HTTP 422 with validation error
**Max execution time**: 2s
**Expected outcome**: 0 invalid/stale/cache-poisoning fixtures produce trusted anchors or trusted basemap tiles.
---
### FT-N-04: Unauthenticated API Access
**Summary**: Validate API rejects unauthenticated requests with HTTP 401.
**Traces to**: AC-14 (security — JWT auth)
**Category**: API Security
**Preconditions**: System running
**Input data**: POST /sessions with no Authorization header
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | POST /sessions without JWT token | HTTP 401 |
| 2 | GET /sessions/{id}/stream without JWT | HTTP 401 |
| 3 | POST /objects/locate without JWT | HTTP 401 |
| 4 | GET /health (no auth required) | HTTP 200 |
**Expected outcome**: Protected endpoints return 401; /health remains accessible
**Max execution time**: 5s
---
### FT-N-05: 3-Consecutive-Failure Re-Localization Request
**Summary**: Validate that after VO loss + 3 consecutive satellite match failures, the system sends a re-localization request to the ground station.
**Traces to**: AC-08 (3 consecutive failures → re-localization request)
**Category**: Resilience & Edge Cases
**Preconditions**: System running; camera-replay set to serve featureless frames (VO will fail); satellite-tile-server returning 404 (tile not found)
**Input data**: Featureless frames (e.g., blank/uniform images), satellite tile server offline
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Camera-replay serves featureless frames | VO tracking lost |
| 2 | Satellite-tile-server returns 404 | Satellite matching fails |
| 3 | Wait for 3 camera frames (3 × 1.43s ≈ 4.3s) | 3 consecutive failures |
| 4 | Check mavlink-inspector for STATUSTEXT | Message matches `RELOC_REQ: last_lat=.* last_lon=.* uncertainty=.*m` |
| 5 | Verify GPS_INPUT fix_type | fix_type == 0 |
| 6 | Verify GPS_INPUT horiz_accuracy | horiz_accuracy == 999.0 |
**Expected outcome**: RELOC_REQ sent via STATUSTEXT; GPS_INPUT reports no-fix with 999.0 accuracy
**Max execution time**: 15s
---
### FT-N-06: IMU-Only Dead Reckoning (VO Lost, No Satellite)
**Summary**: Validate system degrades gracefully to IMU-only ESKF prediction when VO and satellite matching both fail.
**Traces to**: AC-06 (VO lost behavior), AC-04 (confidence score reflects state)
**Category**: Resilience & Edge Cases
**Preconditions**: System running; camera-replay paused (no frames); satellite-tile-server paused
**Input data**: No camera frames, no satellite tiles; only IMU from SITL
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Pause camera-replay and satellite-tile-server | System loses VO and satellite inputs |
| 2 | Read SSE events over 5s | confidence transitions from HIGH/MEDIUM to LOW |
| 3 | Read GPS_INPUT from mavlink-inspector | fix_type == 2 |
| 4 | Read horiz_accuracy over time | horiz_accuracy ≥ 50m and increasing |
| 5 | Verify GPS_INPUT continues at 5-10Hz | Messages continue (IMU-driven ESKF prediction) |
**Expected outcome**: System continues GPS_INPUT at 5-10Hz via IMU; confidence drops; accuracy degrades but output never stops
**Max execution time**: 15s
---
### FT-N-07: Operator Re-Localization Hint Accepted
**Summary**: Validate the system accepts an operator re-localization hint and recovers position.
**Traces to**: AC-08 (re-localization), AC-15 (ground station commands)
**Category**: Ground Station Integration
**Preconditions**: System in FAILED confidence state (3 consecutive failures); satellite-tile-server restored
**Input data**: Operator hint: approximate lat/lon (from coordinates.csv ground truth ± 200m offset)
**Steps**:
| Step | Consumer Action | Expected System Response |
|------|----------------|------------------------|
| 1 | Trigger 3-consecutive-failure state (FT-N-05 preconditions) | RELOC_REQ sent |
| 2 | Restore satellite-tile-server | Tiles available again |
| 3 | POST /sessions/{id}/anchor with approximate lat/lon | HTTP 200 |
| 4 | Wait for satellite match attempt (~3-5s) | System searches in new area |
| 5 | Read SSE events | confidence transitions back to HIGH/MEDIUM |
| 6 | Read GPS_INPUT fix_type | fix_type == 3 |
**Expected outcome**: System accepts operator hint, searches satellite tiles in new area, recovers position, confidence returns to HIGH/MEDIUM
**Max execution time**: 30s
**Max execution time**: 15 minutes.
_docs/02_document/tests/e2e-test-suite.md
+81
View File
@@ -0,0 +1,81 @@
# E2E Test Suite
## Scope
The e2e test suite is separate test tooling, not part of the onboard runtime. It drives black-box replay, public dataset, SITL, Jetson, and representative validation through public runtime interfaces only.
## Purpose
- Feed navigation frames, telemetry traces, cache manifests, and fault triggers into the system under test.
- Validate emitted coordinates, confidence fields, MAVLink `GPS_INPUT`, QGC status, FDR, and generated-tile evidence.
- Produce release evidence without importing runtime internals.
## Ownership
- **Epic**: AZ-217 (E2E Test Suite / test-support work, not product runtime)
- **Owns**:
- `tests/blackbox/**`
- `tests/e2e/**`
- `e2e/replay/**`
- `e2e/reports/**`
- **Does not own**:
- `src/**`
- runtime component internals
- production deployment code
## Public Interfaces Under Test
| Interface | Protocol / Contract |
|-----------|---------------------|
| Navigation frames | Ordered image/video replay with timestamps |
| FC telemetry | MAVLink replay or generated stream |
| Satellite cache | Local COG + manifest + descriptor fixtures |
| GPS output | MAVLink `GPS_INPUT` |
| Operator status | QGC-visible MAVLink status |
| FDR | Filesystem/database-backed evidence outputs |
## Runner Contract
| Method | Input | Output | Error Types |
|--------|-------|--------|-------------|
| `run_scenario` | `ScenarioRequest` | `ScenarioReport` | `FixtureInvalid`, `RuntimeFailed`, `ThresholdFailed` |
| `validate_fixture` | `FixtureRequest` | `FixtureValidationReport` | `FixtureInvalid` |
```yaml
ScenarioRequest:
scenario_id: string
execution_environment: enum(replay, sitl, jetson, representative)
fixture_paths: list[string]
ScenarioReport:
scenario_id: string
result: enum(pass, fail, blocked)
metrics: object
artifacts: list[path]
failure_reason: string optional
```
## Scenario Coverage
| Scenario | Purpose | Evidence |
|----------|---------|----------|
| Still-image accuracy runner | Verify project still-image replay reports frame-center accuracy | Per-image error, aggregate pass rates, covariance, source label, anchor age |
| Synchronized VIO replay runner | Verify Derkachi and public/representative synchronized data drive BASALT/wrapper tests | Fixture alignment, trajectory comparison, VIO registration, latency, covariance calibration |
| Satellite anchor replay runner | Verify VPR and anchor verification scenarios are executable | Retrieval recall, MRE, accepted/rejected anchors, freshness behavior |
| Outlier/sharp-turn/disconnected runner | Verify relocalization resilience scenarios are executable | Degraded-mode timelines and relocalization outcomes |
| Blackout and spoofing runner | Verify total blackout plus spoofing through SITL/replay | Mode-switch timing, covariance growth, failsafe thresholds |
| MAVLink/QGC contract runner | Verify MAVLink output and GCS status assertions | `GPS_INPUT`, WGS84 coordinates, status rate, command ingress |
| Startup/reboot runner | Verify cold-start and companion reboot scenarios | First valid `GPS_INPUT` p95 and FC-state reinitialization |
| Object coordinate contract runner | Verify AI-camera object coordinate request at system boundary | Frame-center-consistent coordinate accuracy and projection bound |
| Tile Manager runner | Verify cache, generated tiles, and storage tests | Cache load, tile write gates, no raw-frame retention, stale rejection, poisoning evidence |
## Release Evidence
The suite assembles CSV, Markdown, MAVLink tlogs, FDR summaries, cache validation reports, and pass/fail metadata into release evidence bundles. Missing public or representative data is reported as `blocked`, not `passed`.
## Non-Responsibilities
- No onboard flight logic.
- No direct estimator, BASALT, wrapper, or tile-manager imports.
- No mutation of runtime internal state.
- No production service APIs.
_docs/02_document/tests/environment.md
+94 -118
View File
@@ -2,148 +2,124 @@
## Overview
**System under test**: GPS-Denied Visual Navigation System — a real-time position estimation service running on Jetson Orin Nano Super. Public interfaces: FastAPI REST/SSE endpoints (port 8000), MAVLink GPS_INPUT messages over serial/UDP, MAVLink telemetry messages.
**System under test**: Onboard GPS-denied localization service. Public interfaces are navigation-camera frame input, flight-controller telemetry input, offline satellite-cache input, `GPS_INPUT` MAVLink output, QGroundControl status output, and flight-data-recorder output.
**Consumer app purpose**: Standalone Python test runner (pytest) that exercises the GPS-denied system through its public interfaces (HTTP API, MAVLink message inspection, SSE stream consumption) without access to internal modules, ESKF state, or GPU buffers.
**Consumer app purpose**: A black-box replay harness that feeds image frames, telemetry traces, cache manifests, and fault triggers into the service, then validates emitted coordinates, confidence fields, telemetry, and logs without importing internal modules.
## Test Modes
## Execution Environments
This is embedded robotics software targeting Jetson Orin Nano Super. A pure Docker environment cannot exercise GPU-dependent paths (TRT inference, cuVSLAM, CUDA streams). The test environment supports two modes:
| Environment | Purpose | Required for |
|-------------|---------|--------------|
| Local replay workstation | Fast still-image and dataset replay validation | Frame-center geolocation, Satellite Service local retrieval, stale-tile rejection |
| Jetson Orin Nano Super | Production-like latency, memory, thermal, and TensorRT/ONNX profiling | AC-4.1, AC-4.2, AC-NEW-1, AC-NEW-5 |
| ArduPilot Plane SITL + QGroundControl | MAVLink `GPS_INPUT`, spoofing, failsafe, and GCS status validation | AC-4.3, AC-5.2, AC-NEW-2, AC-NEW-8 |
| Representative flight/replay rig | Final acceptance evidence with synchronized nav camera, FC IMU/attitude/airspeed/altitude, MAVLink logs, and ground truth | Final AC signoff |
**Mode 1 — Docker SITL (CI/dev)**: Full system in Docker containers with ArduPilot SITL providing MAVLink + IMU at 200Hz. Camera images replayed from input_data/. Satellite tiles served from a mock HTTP server. GPS-denied system runs in CPU-mode with stubbed TRT/cuVSLAM inference (functionally equivalent but slower). Tests all integration paths, API, MAVLink, resilience, and security.
**Mode 2 — Jetson Hardware (nightly/pre-deploy)**: GPS-denied system runs natively on Jetson Orin Nano Super with real CUDA/TRT/cuVSLAM. ArduPilot SITL runs on the Jetson or a companion x86 host, connected via UART or UDP. Camera frames injected via USB camera emulator or replay service. Tests real-time performance, GPU memory, thermal, TRT correctness, and CUDA stream isolation.
## Docker Environment (Mode 1)
### Services
## Docker / Compose Structure
| Service | Image / Build | Purpose | Ports |
|---------|--------------|---------|-------|
| gps-denied-system | `./Dockerfile` (build context: project root) | GPS-denied navigation system in CPU-mode (TRT stubs, cuVSLAM stub returning synthetic VO poses derived from ground truth trajectory) | 8000 (FastAPI), 14550/udp (MAVLink) |
| ardupilot-sitl | `ardupilot/sitl:plane-4.5` (fixed-wing) | ArduPilot SITL: flies waypoint mission following coordinates.csv trajectory, generates IMU at 200Hz via MAVLink, GPS_TYPE=14 accepts GPS_INPUT, provides GLOBAL_POSITION_INT at startup | 5760 (MAVLink TCP), 14551/udp |
| camera-replay | `./tests/docker/camera-replay/Dockerfile` | Replays AD000001-060.jpg from input_data/ at configurable FPS (default 0.7fps) with timestamps synchronized to SITL clock. Supports fault injection: frame skip, corrupted JPEG, pause. | 8001 (frame server) |
| satellite-tile-server | `./tests/docker/tile-server/Dockerfile` | HTTP server for pre-cached satellite tiles (zoom 18, ~50-100 tiles covering test area). Supports fault injection: 404 for specific tiles, 503 for full outage, slow responses. | 8002 |
| mavlink-inspector | `./tests/docker/mavlink-inspector/Dockerfile` | Passively captures all MAVLink traffic (GPS_INPUT, NAMED_VALUE_FLOAT, STATUSTEXT, COMMAND_LONG) for post-test assertion. Can inject operator re-localization hints. | 14552/udp |
| e2e-consumer | `./tests/e2e/Dockerfile` | Black-box test runner (pytest). Communicates only via HTTP API + MAVLink inspector. | — |
|---------|---------------|---------|-------|
| gps-denied-service | Project build image for JetPack-compatible target or replay-compatible host | System under test | MAVLink UDP/TCP and health/status endpoints TBD |
| replay-consumer | Python replay/test harness | Feeds images, telemetry, cache data, and fault triggers | none |
| satellite-cache-stub | Local COG/manifest/descriptor fixture volume | Provides offline tile cache and signed/unsigned manifests | none |
| ardupilot-plane-sitl | ArduPilot Plane SITL image or local process | Validates `GPS_INPUT`, spoofing/failsafe behavior | MAVLink SITL ports |
| qgc-observer | QGC/tlog-compatible observer or MAVLink log parser | Verifies GCS-visible status output | none |
### Networks
## Networks
| Network | Services | Purpose |
|---------|----------|---------|
| e2e-net | all | Isolated test network; MAVLink UDP multicast between gps-denied-system, ardupilot-sitl, mavlink-inspector |
| replay-net | gps-denied-service, replay-consumer, satellite-cache-stub | Offline replay and black-box validation |
| sitl-net | gps-denied-service, ardupilot-plane-sitl, qgc-observer | MAVLink integration and failsafe validation |
### Volumes
## Volumes
| Volume | Mounted to | Purpose |
|--------|-----------|---------|
| input-data | camera-replay:/data, e2e-consumer:/test-data | Camera frames (AD000001-060.jpg), coordinates.csv, data_parameters.md, gmaps reference images |
| satellite-tiles | satellite-tile-server:/tiles, gps-denied-system:/tiles | Pre-processed satellite tiles for test area (zoom 18, 48.249-48.276°N, 37.340-37.386°E) |
| sitl-mission | ardupilot-sitl:/mission | Waypoint mission file derived from coordinates.csv (SITL flies this trajectory, generating physically consistent 200Hz IMU data) |
| test-results | e2e-consumer:/results | Test result CSV output |
| mavlink-capture | mavlink-inspector:/capture | Recorded MAVLink messages for post-test assertions |
### IMU Data Flow
ArduPilot SITL is the primary source of IMU data. It flies a waypoint mission derived from coordinates.csv and internally generates physically consistent accelerometer + gyroscope readings at 200Hz, delivered to the GPS-denied system via MAVLink (RAW_IMU, SCALED_IMU2). This eliminates the need for pre-recorded IMU data files and ensures IMU/trajectory consistency.
```
coordinates.csv → mission_generator script → ArduPilot waypoint file
↓
ArduPilot SITL flies trajectory
↓
IMU @ 200Hz + heartbeat + GLOBAL_POSITION_INT
↓ (MAVLink UDP)
gps-denied-system receives IMU for ESKF
```
### docker-compose structure
```yaml
services:
ardupilot-sitl:
# ArduPilot SITL fixed-wing, outputs IMU at 200Hz via MAVLink
# GPS_TYPE=14 (MAVLink), pre-configured for GPS_INPUT acceptance
satellite-tile-server:
# HTTP tile server with tiles for test area (48.249-48.276°N, 37.340-37.386°E)
camera-replay:
# Replays AD000001-060.jpg at 0.7fps, serves via HTTP or shared volume
depends_on:
- satellite-tile-server
gps-denied-system:
# The system under test
depends_on:
- ardupilot-sitl
- satellite-tile-server
- camera-replay
mavlink-inspector:
# Captures GPS_INPUT, NAMED_VALUE_FLOAT, STATUSTEXT messages
depends_on:
- ardupilot-sitl
e2e-consumer:
# pytest runner — executes after system reaches steady state
depends_on:
- gps-denied-system
- mavlink-inspector
```
|--------|------------|---------|
| input-data | `/data/input` | `_docs/00_problem/input_data/` and public dataset slices |
| expected-results | `/data/expected` | `_docs/00_problem/input_data/expected_results/` |
| derkachi-replay | `/data/input/flight_derkachi` | Cropped nadir MP4 plus synchronized IMU and `GLOBAL_POSITION_INT` trajectory |
| satellite-cache | `/cache/satellite` | COG tiles, manifests, descriptor index fixtures |
| fdr-output | `/fdr` | Flight-data-recorder outputs for validation |
## Consumer Application
**Tech stack**: Python 3.11, pytest, httpx (HTTP client), pymavlink (MAVLink inspection), sseclient-py (SSE stream)
**Entry point**: `pytest tests/e2e/ --tb=short --csv=results/report.csv`
**Tech stack**: Python replay harness with pytest-style assertions and MAVLink log parsing.
### Communication with system under test
**Entry point**: `run-blackbox-replay` command to be created during implementation; this planning artifact defines required behavior, not code.
### Communication With System Under Test
| Interface | Protocol | Endpoint / Topic | Authentication |
|-----------|----------|-----------------|----------------|
| Position API | HTTP REST | http://gps-denied-system:8000/sessions | JWT token |
| Position stream | HTTP SSE | http://gps-denied-system:8000/sessions/{id}/stream | JWT token |
| Object localization | HTTP REST | http://gps-denied-system:8000/objects/locate | JWT token |
| Health check | HTTP REST | http://gps-denied-system:8000/health | None |
| GPS_INPUT inspection | MAVLink UDP | mavlink-inspector:14552 (recorded messages) | None |
| Telemetry inspection | MAVLink UDP | mavlink-inspector:14552 (NAMED_VALUE_FLOAT, STATUSTEXT) | None |
|-----------|----------|------------------|----------------|
| Navigation frames | File/stream replay | Ordered image frames with timestamps | Local fixture access |
| FC telemetry | MAVLink replay or generated stream | IMU, attitude, airspeed, altitude, GPS health | Local MAVLink link |
| Satellite cache | Local filesystem contract | COG + manifest + descriptors | Signed manifest validation |
| GPS output | MAVLink | `GPS_INPUT` to ArduPilot Plane | MAVLink source/system ID allowlist |
| Status output | MAVLink/QGC | `STATUSTEXT` / status summary | MAVLink source/system ID allowlist |
| FDR | Filesystem output | Per-flight segmented logs | Local fixture access |
### What the consumer does NOT have access to
### What The Consumer Does Not Access
- No direct access to ESKF internal state, covariance matrices, or error vectors
- No direct access to cuVSLAM tracking state or feature maps
- No direct access to GPU memory, CUDA streams, or TRT engine internals
- No direct access to the system's file system or configuration files
- No direct database or state store access
## Jetson Hardware Environment (Mode 2)
Tests tagged `@pytest.mark.jetson` require actual Jetson Orin Nano Super hardware. These run natively (no Docker for the GPS-denied system) to exercise real GPU paths.
**Hardware setup**:
- Jetson Orin Nano Super (JetPack 6.2, CUDA 12.x, TensorRT 10.3)
- ArduPilot SITL on same Jetson (or x86 companion connected via UART/UDP)
- Camera frames injected via: USB camera emulator (v4l2loopback feeding frames from input_data/) or HTTP replay service
- Satellite tiles on local SSD (same path as production deployment)
- Active cooling attached (required for sustained load tests)
**Tests in this mode**:
- NFT-PERF-01 through NFT-PERF-06 (real GPU latency, throughput)
- NFT-RES-LIM-01 (GPU+CPU shared memory monitoring via tegrastats)
- NFT-RES-LIM-02 (thermal monitoring via thermal_zone sysfs)
- NFT-RES-LIM-05 (CUDA stream isolation with real concurrent GPU work)
- TRT engine build and inference correctness (expected_results #42-44)
**Jetson CI runner**: Self-hosted GitHub Actions runner on a dedicated Jetson Orin Nano Super, triggered for nightly builds and pre-deploy gates.
- No internal estimator modules.
- No direct BASALT/OpenVINS/Kimera APIs.
- No direct mutation of internal state.
- No bypass of public cache, MAVLink, replay, or FDR interfaces.
## CI/CD Integration
**When to run**: On every PR to `dev`, nightly full suite, before production deploy
**Pipeline stages**:
1. Unit tests (no Docker, no hardware) — on every commit
2. Docker blackbox tests (SITL + CPU mode) — on PR merge to dev
3. Hardware tests (Jetson runner) — nightly + pre-deploy
**Gate behavior**: Docker tests block merge; hardware tests are advisory (nightly) or blocking (pre-deploy)
**Timeout**: Docker suite: 15 minutes; Hardware suite: 30 minutes
| Suite | When to run | Gate behavior | Timeout |
|-------|-------------|---------------|---------|
| Still-image geolocation smoke | Every PR after implementation exists | Block merge | <= 15 min |
| Public VIO dataset replay | Nightly and before release | Block release | Dataset-dependent |
| Jetson performance/resource | Before release and after runtime dependency changes | Block release | <= 8 h for endurance/thermal |
| Plane SITL failsafe/spoofing | Every release candidate | Block release | <= 60 min |
## Reporting
**Format**: CSV
**Columns**: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message (if FAIL)
**Output path**: `./tests/e2e-results/report.csv`
**Format**: CSV and FDR validation summary.
**Columns**: Test ID, Test Name, Input Dataset, Execution Time (ms), Result, Error Distance (m), Source Label, Covariance 95% Semi-Major (m), `GPS_INPUT.fix_type`, Error Message.
**Output path**: `./test-results/blackbox-report.csv` and `./test-results/fdr-validation-summary.md`.
## Test Execution
**Decision**: Both Docker/replay and local hardware execution.
**Hardware dependencies found**:
- Jetson Orin Nano Super with 8 GB shared LPDDR5 and 25 W power mode.
- CUDA/TensorRT/ONNX acceleration for DINOv2 and local-matcher profiling.
- Camera ingestion paths over USB, MIPI-CSI, or GigE.
- ArduPilot Plane SITL and MAVLink `GPS_INPUT` behavior.
- Thermal, power, FDR, and storage limits that require target-like execution.
### Docker / Replay Mode
Use Docker or local host replay for deterministic, reproducible tests that do not require physical Jetson hardware:
- Still-image frame-center geolocation.
- Derkachi synchronized video/telemetry replay, including alignment and VIO smoke checks.
- Satellite-cache freshness and integrity fixtures.
- FAISS descriptor/index behavior.
- Public dataset replay where GPU/hardware timing is not the assertion.
- Plane SITL tests where SITL and MAVLink behavior are the target.
Docker/replay mode is suitable for PR checks and nightly validation, but it does not prove Jetson latency, memory, thermal, or camera-driver behavior.
### Local Hardware Mode
Use local Jetson hardware for release gates:
- BASALT + wrapper latency and memory profiling.
- DINOv2/ONNX/TensorRT descriptor-fidelity and runtime profiling.
- ALIKED/DISK + LightGlue runtime profiling.
- Cold-start time to first valid `GPS_INPUT`.
- 8-hour thermal and FDR endurance tests.
- Camera interface validation once the exact module interface is selected.
### Gate Policy
- PR gate: Docker/replay smoke and deterministic fixture tests.
- Nightly gate: Docker/replay public dataset slices and SITL scenarios.
- Release gate: local Jetson hardware, Plane SITL, thermal/resource tests, and representative replay data.
_docs/02_document/tests/performance-tests.md
+60 -102
View File
@@ -1,138 +1,96 @@
# Performance Tests
### NFT-PERF-01: End-to-End Per-Frame Latency
### NFT-PERF-01: Per-Frame Latency On Project Still Images
**Summary**: Validate total pipeline latency from camera capture to GPS_INPUT output is <400ms.
**Traces to**: AC-07 (< 400ms end-to-end per frame)
**Metric**: End-to-end latency (camera frame timestamp → GPS_INPUT message timestamp)
**Summary**: Validate end-to-end latency for processing project nadir frames through geolocation output.
**Traces to**: AC-4.1, AC-4.4
**Metric**: Capture-to-output latency p50/p95/p99 and dropped-frame rate.
**Preconditions**:
- System running on Jetson Orin Nano Super (GPU-mode)
- Camera-replay serving frames at 0.7fps
- System in steady state (warm-up: ≥10 frames processed)
**Steps**:
- Jetson Orin Nano Super or equivalent production target is running in the intended power mode.
- `project_60_still_images` fixture is available.
| Step | Consumer Action | Measurement |
|------|----------------|-------------|
| 1 | Camera-replay sends frame with known timestamp | Record t_capture |
| 2 | Monitor GPS_INPUT messages at mavlink-inspector | Record t_gps_input for first GPS_INPUT update after t_capture |
| 3 | Compute latency = t_gps_input - t_capture | Per-frame latency |
| 4 | Repeat for 30 consecutive frames | Array of 30 latency values |
|------|-----------------|-------------|
| 1 | Replay images at target 3 fps or faster stress rate | Measure latency from input timestamp to emitted estimate |
| 2 | Record all frame drops | Measure dropped-frame percentage |
**Pass criteria**: p95 latency < 400ms; max latency < 500ms
**Duration**: 50s (~30 frames at 0.7fps + warm-up)
**Pass criteria**: p95 latency <400 ms; dropped frames <=10% under sustained load; no batching delay.
**Duration**: Minimum 20 minutes or full fixture loop repeated enough times to reach stable measurements.
---
### NFT-PERF-02: GPS_INPUT Output Rate Consistency
### NFT-PERF-02: BASALT + Wrapper Replay Latency
**Summary**: Validate GPS_INPUT messages are delivered at a sustained 5-10Hz with no gaps.
**Traces to**: AC-08 (GPS_INPUT via MAVLink at 5-10Hz)
**Metric**: Message rate (Hz), maximum inter-message gap (ms)
**Summary**: Validate relative VIO hot-path latency using synchronized Derkachi video/telemetry and public or representative camera/IMU data.
**Traces to**: AC-2.1a, AC-4.1, AC-4.2
**Metric**: Per-frame VIO latency, completion rate, and memory usage.
**Preconditions**:
- System in steady state
- Camera-replay active
**Steps**:
- Derkachi `flight_derkachi.mp4` and `data_imu.csv` are mounted and pass fixture validation.
- MUN-FRL/ALTO/EPFL/Kagaru or another representative synchronized dataset slice is pinned for calibrated final comparison.
- OpenVINS reference replay is available for comparison when the dataset supports it.
| Step | Consumer Action | Measurement |
|------|----------------|-------------|
| 1 | Capture GPS_INPUT messages for 60 seconds | Count messages, record timestamps |
| 2 | Compute rate: count / 60 | 5 ≤ rate ≤ 10 Hz |
| 3 | Compute max gap between consecutive messages | max_gap ≤ 250ms |
| 4 | Compute jitter: std_dev of inter-message intervals | jitter < 50ms |
|------|-----------------|-------------|
| 1 | Replay Derkachi video at target 3 fps and stress rates from the 30 fps source | Measure per-frame processing time, dropped frames, and telemetry alignment |
| 2 | Replay synchronized camera/IMU stream through BASALT + wrapper | Measure VIO processing time and completion rate |
| 3 | Compare emitted trajectory against Derkachi `GLOBAL_POSITION_INT` and calibrated dataset ground truth where available | Measure completion rate and error distribution |
| 4 | Monitor memory | Track CPU/GPU shared memory peak |
**Pass criteria**: Rate 5-10Hz; max gap ≤ 250ms; jitter < 50ms
**Duration**: 60s
**Pass criteria**: Normal-frame VO registration >95% on calibration-supported segments; p95 processing latency <400 ms for the hot path; memory <8 GB shared; Derkachi replay maintains stable 3-video-frames-per-telemetry-row alignment with <=10% dropped frames under sustained target-rate replay.
**Duration**: Dataset-dependent; at least one normal segment and one challenging segment.
---
### NFT-PERF-03: cuVSLAM Visual Odometry Processing Time
### NFT-PERF-03: Relocalization Trigger Path Latency
**Summary**: Validate cuVSLAM processes each frame within 20ms.
**Traces to**: AC-07 (real-time processing budget)
**Metric**: Per-frame cuVSLAM inference time (ms)
**Summary**: Validate the heavy DINOv2-VLAD + FAISS + ALIKED/LightGlue path under bounded top-K settings.
**Traces to**: AC-3.2, AC-3.3, AC-4.1, AC-8.6
**Metric**: Trigger-to-anchor latency, top-K query time, local verification time, accepted/rejected anchor counts.
**Preconditions**:
- System running on Jetson Orin Nano Super
- Steady state (≥10 frames processed)
**Steps**:
- Precomputed descriptor index is loaded.
- Dynamic K settings are configured: K=5 stable, K=20 active-conflict, K=50 fallback.
| Step | Consumer Action | Measurement |
|------|----------------|-------------|
| 1 | Replay 30 frames, read processing time from SSE events or health endpoint metrics | Per-frame VO time |
| 2 | Compute p95 of VO time | p95 ≤ 20ms |
|------|-----------------|-------------|
| 1 | Trigger relocalization from cold start or sharp turn | Measure DINOv2 descriptor time and FAISS query time |
| 2 | Verify top-K candidates | Measure ALIKED/LightGlue + RANSAC latency |
| 3 | Emit accepted/rejected decision | Measure total trigger-to-decision latency |
**Pass criteria**: p95 cuVSLAM inference time ≤ 20ms
**Duration**: 50s
**Pass criteria**: Heavy path is conditional, never blocks steady-state frame output; accepted anchor carries MRE <2.5 px and valid covariance.
**Duration**: 100 relocalization trials across stable and active-conflict sector fixtures.
---
### NFT-PERF-04: Satellite Matching Latency (Async)
### NFT-PERF-04: Cold Boot Time To First Fix
**Summary**: Validate satellite matching completes within 330ms per keyframe (async, does not block VO).
**Traces to**: AC-07 (within frame budget), solution processing time budget
**Metric**: Per-keyframe satellite matching latency (ms)
**Summary**: Validate companion boot to first valid `GPS_INPUT`.
**Traces to**: AC-NEW-1
**Metric**: Time from service start/boot marker to first valid `GPS_INPUT`.
**Preconditions**:
- System running on Jetson Orin Nano Super
- Satellite tiles loaded
**Steps**:
- Engines/indexes are built before the run.
- Cache/index is available locally.
- FC state handoff is simulated or provided.
| Step | Consumer Action | Measurement |
|------|----------------|-------------|
| 1 | Monitor satellite match events over 60s (expect ~4-8 matches at 0.07-0.14Hz) | Per-match latency from health/metrics endpoint |
| 2 | Verify no VO frame was blocked during satellite matching | VO timestamps maintain 0.7fps cadence |
|------|-----------------|-------------|
| 1 | Start service from cold boot condition | Measure initialization stages |
| 2 | Wait for first valid output | Measure first valid `GPS_INPUT` timestamp |
**Pass criteria**: p95 satellite matching ≤ 330ms; VO cadence unaffected
**Duration**: 60s
**Pass criteria**: 95th percentile <30 s over 50 runs.
---
### NFT-PERF-05: TRT Engine Load Time
**Summary**: Validate all TensorRT engines load within 10 seconds total.
**Traces to**: AC-11 (startup), solution startup sequence
**Metric**: Engine load time (seconds)
**Preconditions**:
- Cold start on Jetson Orin Nano Super
- Engines pre-built and available on storage
**Steps**:
| Step | Consumer Action | Measurement |
|------|----------------|-------------|
| 1 | Start system, monitor startup log for engine load timestamps | t_start_load, t_end_load per engine |
| 2 | Compute total: sum of all engine load times | ≤ 10s total |
**Pass criteria**: Total TRT engine load time ≤ 10s
**Duration**: 30s (includes boot time)
---
### NFT-PERF-06: Sustained 30-Minute Processing
**Summary**: Validate the system maintains consistent performance over a 30-minute continuous session without degradation.
**Traces to**: AC-07 (real-time), AC-08 (memory < 8GB)
**Metric**: Per-frame latency, GPS_INPUT rate, position accuracy over time
**Preconditions**:
- System running on Jetson Orin Nano Super
- Camera-replay looping flight-sequence-60 (re-starts after frame 60)
- Satellite tiles available
**Steps**:
| Step | Consumer Action | Measurement |
|------|----------------|-------------|
| 1 | Run for 30 minutes, collect per-minute stats | Latency, rate, accuracy |
| 2 | Compare first-5-min stats vs last-5-min stats | No degradation >10% |
| 3 | Monitor for any position output gaps > 1s | Count gaps |
**Pass criteria**: No latency degradation >10% over 30 min; GPS_INPUT rate remains 5-10Hz; no output gaps >1s
**Duration**: 30 minutes
**Duration**: 50 cold-start trials.
_docs/02_document/tests/resilience-tests.md
+43 -127
View File
@@ -1,169 +1,85 @@
# Resilience Tests
### NFT-RES-01: Mid-Flight Reboot Recovery
### NFT-RES-01: Total Visual Blackout With GPS Spoofing
**Summary**: Validate the system recovers from a companion computer reboot within 70 seconds and restores position accuracy.
**Traces to**: AC-12 (mid-flight reboot recovery)
**Summary**: Validate degraded-mode behavior when the camera feed is totally occluded/blacked out and real GPS is spoofed or denied.
**Traces to**: AC-3.5, AC-5.2, AC-NEW-8
**Preconditions**:
- System running in steady state with good position accuracy
- SITL ArduPilot continues running (FC stays up during companion computer reboot)
- Plane SITL or replay trace is emitting normal telemetry.
- System has a recent trusted visual/satellite anchor.
**Fault injection**:
- Kill gps-denied-system process (docker stop or SIGKILL)
- Restart after 5s delay (simulates Jetson reboot time)
**Steps**:
- Full camera blackout/total occlusion for 5 s, 15 s, and 35 s while spoofed GPS is present.
| Step | Action | Expected Behavior |
|------|--------|------------------|
| 1 | Record current position accuracy and confidence | Baseline metrics |
| 2 | Kill gps-denied-system container | GPS_INPUT messages stop |
| 3 | Verify SITL continues running (heartbeat present) | FC still alive, using IMU dead reckoning |
| 4 | Restart gps-denied-system container after 5s | System starts recovery sequence |
| 5 | Monitor time from restart to first GPS_INPUT | ≤ 70s |
| 6 | Wait for first satellite match | Position accuracy restored |
| 7 | Verify position error after recovery | Error ≤ 50m after first satellite match |
|------|--------|-------------------|
| 1 | Inject total occlusion/blackout and spoofed GPS | Camera gate reports `usable_for_vio=false`, BASALT is bypassed, and system switches to `dead_reckoned` within <=1 processed frame or <=400 ms |
| 2 | Continue blackout | IMU-only covariance grows monotonically and spoofed GPS is ignored |
| 3 | Exceed 30 s or covariance >500 m | System emits no-fix/failsafe fields and QGC `VISUAL_BLACKOUT_FAILSAFE` |
**Pass criteria**: Recovery time ≤ 70s; post-recovery position error ≤ 50m after satellite match
**Duration**: 120s
**Pass criteria**: All pre-VIO occlusion gate, timing, covariance, `fix_type`, `horiz_accuracy`, and status thresholds match AC-NEW-8.
---
### NFT-RES-02: Tracking Loss and Satellite Re-Localization
### NFT-RES-02: Sharp Turn And Disconnected Segment Relocalization
**Summary**: Validate the system recovers from cuVSLAM tracking loss via satellite-based re-localization.
**Traces to**: AC-07 (disconnected segments), AC-06 (sharp turn handling)
**Summary**: Validate recovery when frame-to-frame overlap drops below the VO threshold.
**Traces to**: AC-3.2, AC-3.3, AC-3.4, AC-8.6
**Preconditions**:
- System in normal tracking (HIGH confidence)
- Satellite tiles available
- Public or representative replay contains sharp-turn/disconnected segment cases, or equivalent synthetic sequence is generated from mapped imagery.
**Fault injection**:
- Camera-replay sends featureless/blurred frames (simulates VO tracking loss from sharp turn)
- Then resumes normal frames
**Steps**:
- Sequence transition with <5% overlap, heading change <70°, and drift <200 m.
| Step | Action | Expected Behavior |
|------|--------|------------------|
| 1 | Normal tracking established | confidence: HIGH, vo_status: tracking |
| 2 | Camera-replay serves 3 featureless frames | cuVSLAM reports tracking_lost |
| 3 | System enters TRACKING_LOST state | Satellite matching switches to every frame |
| 4 | Camera-replay resumes normal frames | Satellite match succeeds |
| 5 | Monitor SSE: vo_status returns to "tracking" | cuVSLAM restarted |
| 6 | Monitor SSE: confidence returns to HIGH | Position re-anchored |
| 7 | Verify position accuracy after recovery | Error ≤ 50m |
|------|--------|-------------------|
| 1 | Replay normal segment | BASALT + wrapper emits normal `vo_extrapolated` estimates |
| 2 | Inject sharp-turn/disconnected transition | VO failure is expected; system triggers VPR relocalization |
| 3 | Continue next segment | System connects segment through verified satellite anchor or reports degraded status |
**Pass criteria**: Recovery within 5 frames after normal frames resume; position error ≤ 50m post-recovery
**Duration**: 30s
**Pass criteria**: Relocalization request is issued when no position is available for >=3 consecutive frames and >=2 s; verified anchor reconnects the segment or output remains degraded with growing covariance.
---
### NFT-RES-03: Sustained IMU-Only Operation
### NFT-RES-03: Companion Computer Restart Mid-Flight
**Summary**: Validate the system continues producing position estimates during extended IMU-only periods without crashing.
**Traces to**: AC-08 (system continues during failure), AC-12 (failsafe)
**Summary**: Validate reboot recovery from flight-controller state and preloaded cache.
**Traces to**: AC-5.3, AC-NEW-1
**Preconditions**:
- System in normal tracking
- Replay/SITL mission is in progress.
- FDR has current segment logs.
**Fault injection**:
- Pause both camera-replay (no VO) and satellite-tile-server (no satellite matching)
- Duration: 30s
**Steps**:
- Kill and restart the GPS-denied service during a GPS-denied segment.
| Step | Action | Expected Behavior |
|------|--------|------------------|
| 1 | Establish normal tracking baseline | GPS_INPUT at 5-10Hz, confidence HIGH |
| 2 | Pause camera-replay and satellite-tile-server | VO and satellite inputs stop |
| 3 | Monitor GPS_INPUT for 30s | Messages continue at 5-10Hz (IMU-driven ESKF prediction) |
| 4 | Verify horiz_accuracy grows over time | accuracy increases monotonically |
| 5 | Verify fix_type transitions to 2 | Degraded but present |
| 6 | Verify confidence transitions to LOW | Reflects IMU-only state |
| 7 | Resume camera-replay and satellite-tile-server | System recovers to normal tracking |
| 8 | Verify recovery to HIGH confidence | Satellite match re-anchors position |
|------|--------|-------------------|
| 1 | Kill service | FC continues on last known/IMU-extrapolated state |
| 2 | Restart service | Service reloads cache/index and uses FC state handoff |
| 3 | Observe first valid output | First valid `GPS_INPUT` emitted within <30 s |
**Pass criteria**: GPS_INPUT never stops during 30s IMU-only period; system recovers when inputs resume
**Duration**: 60s
**Pass criteria**: No raw frames are required for recovery; first valid fix <30 s p95; failure is logged in FDR.
---
### NFT-RES-04: Satellite Tile Server Failure
### NFT-RES-04: Tile Cache Freshness Degradation
**Summary**: Validate the system continues operating when satellite tile server becomes unavailable, with graceful accuracy degradation.
**Traces to**: AC-07 (resilience), solution risk: Google Maps quality
**Summary**: Validate graceful behavior when the only available tile candidates are stale.
**Preconditions**:
- System in normal tracking
**Traces to**: AC-8.2, AC-NEW-6
**Fault injection**:
- Stop satellite-tile-server container (simulates tile unavailability)
**Steps**:
- Mark cache tiles older than 6 months for active-conflict sector and older than 12 months for stable sector.
| Step | Action | Expected Behavior |
|------|--------|------------------|
| 1 | Normal tracking with satellite corrections | confidence: HIGH |
| 2 | Stop satellite-tile-server | Satellite matching returns errors |
| 3 | Monitor for 60s | System falls back to VO-only; confidence drops to MEDIUM after 30s |
| 4 | Verify GPS_INPUT continues | Messages at 5-10Hz, fix_type remains 3 (VO tracking OK) |
| 5 | Restart satellite-tile-server | Satellite matching resumes |
| 6 | Verify confidence returns to HIGH | Position re-anchored |
|------|--------|-------------------|
| 1 | Replay frame requiring satellite anchor | Stale tiles are rejected or down-confidence weighted |
| 2 | Inspect emitted estimate | No stale tile produces `satellite_anchored` label past hard rejection threshold |
**Pass criteria**: No crash or hang; GPS_INPUT continues; confidence degrades gracefully and recovers when tiles return
**Duration**: 90s
---
### NFT-RES-05: Corrupted Camera Frame
**Summary**: Validate the system handles a corrupted camera frame without crashing.
**Traces to**: AC-06 (outlier tolerance)
**Preconditions**:
- System in normal tracking
**Fault injection**:
- Camera-replay injects a truncated/corrupted JPEG between normal frames
**Steps**:
| Step | Action | Expected Behavior |
|------|--------|------------------|
| 1 | Normal tracking for 5 frames | Baseline established |
| 2 | Camera-replay sends corrupted JPEG | System logs warning, skips frame |
| 3 | Camera-replay sends next normal frame | VO continues processing |
| 4 | Verify no crash, no hang | GPS_INPUT continues at 5-10Hz |
| 5 | Verify position accuracy on next valid frame | Error < 50m |
**Pass criteria**: System skips corrupted frame gracefully; no crash; next frame processed normally
**Duration**: 15s
---
### NFT-RES-06: Camera Feed Interruption (No Frames for 10s)
**Summary**: Validate the system survives a 10-second camera feed interruption.
**Traces to**: AC-12 (failsafe — N seconds no estimate), AC-08 (continued operation)
**Preconditions**:
- System in normal tracking
**Fault injection**:
- Camera-replay pauses for 10s (no frames delivered)
**Steps**:
| Step | Action | Expected Behavior |
|------|--------|------------------|
| 1 | Normal tracking baseline | GPS_INPUT at 5-10Hz |
| 2 | Pause camera-replay for 10s | No new camera frames |
| 3 | Monitor GPS_INPUT | Messages continue via IMU prediction |
| 4 | Monitor confidence | Transitions to LOW after VO timeout |
| 5 | Resume camera-replay | VO restarts, satellite matching resumes |
| 6 | Verify recovery | confidence returns to HIGH within 10 frames |
**Pass criteria**: GPS_INPUT never stops; recovery within 10 frames after camera feed resumes
**Duration**: 30s
**Pass criteria**: Freshness decay and hard rejection match AC-NEW-6.
_docs/02_document/tests/resource-limit-tests.md
+43 -48
View File
@@ -1,90 +1,85 @@
# Resource Limit Tests
### NFT-RES-LIM-01: Memory Usage Under 8GB
### NFT-RES-LIM-01: Jetson Memory Budget
**Summary**: Validate system memory usage stays below 8GB shared memory (CPU + GPU) during sustained operation.
**Traces to**: AC-08 (memory < 8GB), RESTRICT-09 (8GB shared LPDDR5)
**Summary**: Validate that runtime memory stays below the 8 GB shared LPDDR5 limit.
**Traces to**: AC-4.2, Restrictions Onboard Hardware
**Preconditions**:
- System running on Jetson Orin Nano Super (8GB shared memory)
- Full pipeline active: cuVSLAM + satellite matching + ESKF + GPS_INPUT + FastAPI
- Jetson Orin Nano Super in production power/thermal mode.
- BASALT + wrapper, cache index, FAISS CPU index, and FDR enabled.
**Monitoring**:
- Total system memory (RSS + GPU allocated) via `tegrastats` or `/sys/devices/platform/host1x/*/memory`
- Poll every 5s via GET /health (memory_mb field)
- CPU/GPU shared memory, process RSS, CUDA allocations, FAISS index memory.
**Duration**: 30 minutes
**Pass criteria**: Peak memory < 8192MB; no memory leak (growth < 50MB over 30 minutes after first 2 minutes warm-up)
**Duration**: Minimum 60 minutes steady-state replay plus relocalization triggers.
**Pass criteria**: Peak memory <8 GB shared; no OOM kill; no silent descriptor/index eviction.
---
### NFT-RES-LIM-02: GPU Thermal Envelope
### NFT-RES-LIM-02: Thermal And Power Envelope
**Summary**: Validate SoC junction temperature stays below 80°C under sustained processing load.
**Traces to**: AC-08 (thermal), RESTRICT-10 (25W TDP, thermal throttling at 80°C)
**Summary**: Validate sustained 25 W operation without thermal throttling across the environmental envelope.
**Traces to**: AC-NEW-5
**Preconditions**:
- System running on Jetson Orin Nano Super with active cooling
- Full pipeline active
- Jetson cooling solution installed.
- Hot-soak chamber or production thermal test setup at +50 °C.
**Monitoring**:
- SoC junction temperature via `tegrastats` or `/sys/devices/virtual/thermal/thermal_zone*/temp`
- Poll every 10s via GET /health (gpu_temp_c field)
- Power mode, temperature sensors, throttle flags, CPU/GPU clocks, per-frame latency.
**Duration**: 30 minutes
**Pass criteria**: SoC junction temperature < 80°C throughout; no thermal throttling events
**Duration**: 8 hours at sustained representative workload.
**Pass criteria**: No thermal throttle event; p95 latency remains <400 ms; QGC receives thermal warning if any threshold is approached.
---
### NFT-RES-LIM-03: Satellite Tile Storage
### NFT-RES-LIM-03: Satellite Cache Storage Budget
**Summary**: Validate pre-loaded satellite tile storage stays within calculated budget for a test mission area.
**Traces to**: AC-19 (satellite imagery pre-loaded), RESTRICT-08 (onboard storage limited)
**Summary**: Validate persistent satellite cache footprint for up to 400 km² operational area.
**Preconditions**:
- Satellite tiles pre-processed for the test flight area (48.249-48.276°N, 37.340-37.386°E)
- Zoom 18 primary + zoom 19 for ±500m along flight path
**Traces to**: AC-8.3, Restrictions Satellite Imagery
**Monitoring**:
- Total tile storage size on disk
- RAM usage for preloaded tiles (±2km buffer)
- Cache imagery, overviews, manifests, sidecars, FAISS descriptors/indexes.
**Duration**: Static check
**Pass criteria**: Total tile storage ≤ 1000MB on disk; RAM usage for preloaded tiles ≤ 200MB
**Duration**: Full cache build/load test.
**Pass criteria**: Persistent cache is <=10 GB unless the implementation explicitly defines and gets approval for a separate descriptor/index budget.
---
### NFT-RES-LIM-04: Long Flight Simulation (3000 Frames)
### NFT-RES-LIM-04: Flight Data Recorder Rollover
**Summary**: Validate the system handles a maximum-length flight of 3000 frames without resource exhaustion.
**Traces to**: RESTRICT-04 (up to 3000 photos per flight)
**Summary**: Validate FDR storage cap and rollover behavior under an 8-hour synthetic mission.
**Traces to**: AC-NEW-3, AC-8.5
**Preconditions**:
- System running on Jetson (or Docker with CPU-mode for functional test)
- Camera-replay looping flight-sequence-60 to generate 3000 frames (50 loops)
- Synthetic 8-hour load with 3 fps navigation frames, full-rate IMU, emitted `GPS_INPUT`, health telemetry, tile writes, and failure thumbnails.
**Monitoring**:
- Memory usage every 60s
- GPS_INPUT rate every 60s
- Position accuracy sampled every 100 frames
- FDR segment sizes, rollover events, retained payload classes.
**Duration**: ~71 minutes (3000 frames at 0.7fps)
**Pass criteria**: Memory stays < 8GB; GPS_INPUT rate stays 5-10Hz; no crash or hang over full 3000-frame sequence
**Duration**: 8 hours.
**Pass criteria**: FDR remains <=64 GB per flight; rollover is logged; no raw nav/AI frames are retained; no payload class is silently dropped.
---
### NFT-RES-LIM-05: CUDA Stream Isolation
### NFT-RES-LIM-05: Cold Start Resource Spike
**Summary**: Validate that Stream B (satellite matching) does not block Stream A (cuVSLAM) under concurrent load.
**Traces to**: AC-07 (< 400ms per frame), solution CUDA stream pipelining
**Summary**: Validate that CUDA/TensorRT/ONNX/FAISS initialization does not violate boot or memory budgets.
**Preconditions**:
- System running on Jetson Orin Nano Super
- Satellite matching triggered on a keyframe while VO is processing the next frame
**Traces to**: AC-NEW-1, AC-4.2
**Monitoring**:
- VO frame-to-frame timing during satellite matching
- Satellite matching does not extend VO latency
- Initialization time, peak memory, engine/index load time.
**Duration**: 30s (capture ~5 satellite matching events concurrent with VO)
**Pass criteria**: VO per-frame time ≤ 20ms even when satellite matching is running concurrently; no frame drops
**Duration**: 50 cold-start trials.
**Pass criteria**: First valid `GPS_INPUT` <30 s p95; peak memory <8 GB; no first-run engine build occurs at runtime.
_docs/02_document/tests/security-tests.md
+31 -57
View File
@@ -1,88 +1,62 @@
# Security Tests
### NFT-SEC-01: JWT Authentication Required on Protected Endpoints
### NFT-SEC-01: Signed Cache Manifest Enforcement
**Summary**: Validate all protected endpoints reject requests without a valid JWT token.
**Traces to**: AC-14 (security — JWT auth on API)
**Summary**: Validate that unsigned or tampered cache manifests cannot produce trusted anchors.
**Steps**:
**Traces to**: AC-8.2, AC-8.3, AC-NEW-4, AC-NEW-7
| Step | Consumer Action | Expected Response |
|------|----------------|------------------|
| 1 | POST /sessions with no Authorization header | HTTP 401 |
| 2 | GET /sessions/{id}/stream with no Authorization header | HTTP 401 |
| 3 | POST /sessions/{id}/anchor with no Authorization header | HTTP 401 |
| 4 | DELETE /sessions/{id} with no Authorization header | HTTP 401 |
| 5 | POST /objects/locate with no Authorization header | HTTP 401 |
| 6 | GET /health with no Authorization header | HTTP 200 (health is public) |
|------|-----------------|-------------------|
| 1 | Provide valid signed manifest | System accepts cache fixture if all freshness and resolution checks pass |
| 2 | Provide unsigned manifest | System rejects cache fixture and logs security event |
| 3 | Provide hash-mismatched tile sidecar | System rejects affected tile and emits no trusted anchor from it |
**Pass criteria**: All protected endpoints return 401; /health returns 200 without auth
**Pass criteria**: 0 unsigned or hash-mismatched fixtures produce `satellite_anchored` output or trusted generated tile promotion.
---
### NFT-SEC-02: Expired JWT Token Rejection
### NFT-SEC-02: Cache Poisoning Write Gate
**Summary**: Validate the system rejects expired JWT tokens.
**Traces to**: AC-14 (security)
**Summary**: Validate that generated onboard tiles are not written or promoted when parent-pose covariance is too weak.
**Steps**:
**Traces to**: AC-8.4, AC-NEW-7
| Step | Consumer Action | Expected Response |
|------|----------------|------------------|
| 1 | Generate a JWT token with exp set to 1 hour ago | Expired token |
| 2 | POST /sessions with expired token in Authorization header | HTTP 401 |
| 3 | POST /objects/locate with expired token | HTTP 401 |
|------|-----------------|-------------------|
| 1 | Replay generated tile candidate with parent sigma <=3 m | Tile may be written as candidate with full quality metadata |
| 2 | Replay candidate with parent sigma in (3 m, 5 m] | Tile is marked lower trust per sidecar policy |
| 3 | Replay candidate with parent sigma >5 m | Tile is not eligible for write/promotion |
**Pass criteria**: Expired tokens are rejected with 401
**Pass criteria**: Tile trust level and write eligibility match AC-NEW-7; no over-threshold tile becomes trusted basemap.
---
### NFT-SEC-03: Invalid JWT Signature Rejection
### NFT-SEC-03: MAVLink Source And Spoofing Rejection
**Summary**: Validate the system rejects JWT tokens signed with the wrong key.
**Traces to**: AC-14 (security)
**Summary**: Validate that spoofed real-GPS measurements and unauthorized MAVLink sources do not override trusted estimator state.
**Steps**:
**Traces to**: AC-3.5, AC-4.3, AC-NEW-2, AC-NEW-8
| Step | Consumer Action | Expected Response |
|------|----------------|------------------|
| 1 | Generate a JWT token with a different signing key | Invalid signature token |
| 2 | POST /sessions with invalid-signature token | HTTP 401 |
|------|-----------------|-------------------|
| 1 | Inject spoofed `GPS_RAW_INT` during normal visual operation | Estimator rejects inconsistent GPS based on FC health and visual/satellite consistency |
| 2 | Inject spoofed GPS during visual blackout | Spoofed GPS remains excluded until health and visual consistency gates pass |
| 3 | Inject MAVLink messages from unauthorized source ID | Message is ignored and security/status event is logged |
**Pass criteria**: Invalid-signature tokens are rejected with 401
**Pass criteria**: No unauthorized or spoofed input causes a confident position estimate; promotion/demotion status is visible to QGC and FDR.
---
### NFT-SEC-04: Malformed API Request Handling
### NFT-SEC-04: No In-Flight Satellite Provider Access
**Summary**: Validate the system handles malformed API payloads without crashing or leaking internal details.
**Traces to**: AC-14 (security), AC-16 (API robustness)
**Summary**: Validate that the runtime system does not call commercial or Suite satellite services during flight.
**Steps**:
**Traces to**: AC-8.1, AC-8.3, Restrictions Satellite Imagery
| Step | Consumer Action | Expected Response |
|------|----------------|------------------|
| 1 | POST /objects/locate with empty body | HTTP 422 with generic error (no stack trace) |
| 2 | POST /objects/locate with body `{"pixel_x": "not_a_number"}` | HTTP 422 with validation error |
| 3 | POST /sessions with body exceeding 1MB | HTTP 413 or 422 (no crash) |
| 4 | POST /sessions/{id}/anchor with body `{"lat": 999, "lon": 999}` | HTTP 422 (invalid coordinates) |
| 5 | Verify system continues operating after all malformed requests | GET /health returns 200 |
|------|-----------------|-------------------|
| 1 | Run replay with network blocked | System continues using local cache |
| 2 | Run replay requiring missing tile | System reports degraded/relocalization-needed status, not an external fetch |
**Pass criteria**: All malformed requests return 4xx errors with safe error messages (no stack traces, no internal paths); system remains operational
---
### NFT-SEC-05: MAVLink Injection Resistance
**Summary**: Validate the system ignores unexpected or malformed MAVLink messages on the MAVLink channel.
**Traces to**: AC-15 (ground station commands), solution security analysis
**Steps**:
| Step | Consumer Action | Expected Response |
|------|----------------|------------------|
| 1 | Send unexpected MAVLink message types to the system's MAVLink port | System ignores (no crash, no state corruption) |
| 2 | Send malformed COMMAND_LONG with invalid lat/lon in re-localization hint | System rejects or ignores invalid coordinates |
| 3 | Verify GPS_INPUT output continues normally | No disruption |
**Pass criteria**: System ignores unexpected messages; continues normal operation; does not process invalid re-localization coordinates
**Pass criteria**: 0 outbound satellite-provider or Suite Service calls during runtime; missing cache data produces controlled degraded behavior.
_docs/02_document/tests/test-data.md
+76 -71
View File
@@ -3,93 +3,98 @@
## Seed Data Sets
| Data Set | Description | Used by Tests | How Loaded | Cleanup |
|----------|-------------|---------------|-----------|---------|
| flight-sequence-60 | 60 aerial images (AD000001-060.jpg) with ground truth GPS from coordinates.csv, captured at ~1 photo per 2-3s from a fixed-wing UAV at 400m altitude | FT-P-01 through FT-P-06, FT-N-01, FT-N-02, NFT-PERF-01, NFT-RES-01 | Volume mount to camera-replay service; coordinates.csv loaded by e2e-consumer for ground truth comparison | Container restart between test groups |
| camera-params | Camera parameters: ADTi Surveyor Lite 26S v2, 26MP (6252x4168), 25mm focal length, 23.5mm sensor width | All position accuracy tests, object localization tests | Volume mount; read by gps-denied-system at startup | N/A (read-only) |
| satellite-tiles-test | Pre-processed satellite tiles (zoom 18) covering test flight area: 48.249-48.276°N, 37.340-37.386°E | All tests requiring satellite matching | Volume mount to satellite-tile-server and gps-denied-system | Container restart |
| ardupilot-params | ArduPilot SITL parameters: GPS1_TYPE=14, GPS_RATE=5, EK3_SRC1_POSXY=1, EK3_SRC1_VELXY=1, fixed-wing frame | All tests requiring flight controller interaction | Baked into ardupilot-sitl Docker image | Container restart |
| imu-replay-data | Synthetic IMU data (accelerometer + gyroscope at 200Hz) matching the flight-sequence-60 trajectory. Generated from coordinates.csv ground truth positions by computing velocities/accelerations between frames, interpolating to 200Hz, adding noise consistent with ICM-42688-P datasheet (gyro: 3.0e-3 °/s/√Hz, accel: 70 µg/√Hz). Stored as `input_data/imu_synthetic_200hz.csv` with columns: timestamp_us, accel_x, accel_y, accel_z, gyro_x, gyro_y, gyro_z. Alternative: SITL ArduPilot flies a waypoint mission following the coordinates.csv trajectory and internally generates physically consistent IMU data at 200Hz. | FT-P-01 through FT-P-06, NFT-PERF-01, NFT-RES-01 through NFT-RES-05 | Primary: SITL ArduPilot flies the trajectory and generates IMU internally via MAVLink. Fallback: pre-generated CSV replayed via MAVLink injector | Container restart |
| invalid-inputs | Malformed images (truncated JPEG, wrong resolution), invalid API payloads, corrupted IMU streams | FT-N-03 through FT-N-06, NFT-SEC-01 through NFT-SEC-04 | Volume mount to e2e-consumer; injected via API calls | Container restart |
|----------|-------------|---------------|------------|---------|
| `project_60_still_images` | 60 nadir images with WGS84 frame-center coordinates from `coordinates.csv`; height 400 m | FT-P-01, FT-P-02, FT-N-01, NFT-PERF-01 | Mounted from `_docs/00_problem/input_data/` | Read-only |
| `project_gmaps_reference_subset` | Google Maps reference images available for the first sample frames | FT-P-02, FT-N-01 | Mounted from `_docs/00_problem/input_data/` | Read-only |
| `expected_frame_centers` | Expected lat/lon and thresholds derived from `coordinates.csv` | FT-P-01, FT-P-02 | `_docs/00_problem/input_data/expected_results/results_report.md` | Read-only |
| `derkachi_video_telemetry` | Cropped nadir MP4 synchronized with IMU and `GLOBAL_POSITION_INT` trajectory: 880 x 720, 30 fps, ~490.07 s; telemetry 10 Hz, 4,900 rows | FT-P-03, NFT-PERF-02, NFT-RES-02 | Mounted from `_docs/00_problem/input_data/flight_derkachi/` | Read-only |
| `public_nadir_vio_candidates` | MUN-FRL, ALTO, EPFL fixed-wing, Kagaru, AerialVL/VPAir slices, EuRoC/UZH FPV proxy slices | FT-P-03, FT-P-04, NFT-PERF-02, NFT-RES-02 | Downloaded or mounted by replay harness; exact files pinned during implementation | Reset fixture volume |
| `sitl_spoofing_scenarios` | Generated ArduPilot Plane SITL GPS loss/spoofing traces | FT-N-02, NFT-RES-01, NFT-SEC-03 | Generated by test harness | Discard generated logs after report |
| `cache_integrity_fixtures` | Fresh, stale, unsigned, hash-mismatched, and low-resolution cache manifests | FT-N-03, NFT-SEC-01, NFT-SEC-02 | Mounted fixture volume | Read-only |
## Public Dataset Coverage Plan
| Public Data Source | Fit For This Project | Limitations | Planned Use |
|--------------------|----------------------|-------------|-------------|
| MUN-FRL | Strong nadir camera + IMU + GNSS/ground truth candidate | Helicopter/hexacopter, not fixed-wing | BASALT/OpenVINS/Kimera replay and covariance calibration |
| ALTO | Strong nadir aerial imagery with GPS/INS, altimeter, orthophotos | Helicopter/airborne collection, access/details must be pinned | VPR, satellite alignment, VO/geolocalization replay |
| EPFL fixed-wing micro UAV | Strong fixed-wing relevance with camera/navigation sensors | Availability and exact raw IMU packaging must be verified | Fixed-wing path realism and photogrammetry-style validation |
| Kagaru airborne vision | Fixed-wing/farmland relevance, downward stereo, INS/GPS | Older dataset; exact sensor compatibility must be verified | Agricultural terrain and fixed-wing motion checks |
| AerialVL | Strong UAV-to-satellite localization and VPR benchmark | IMU availability is less clear than image/GNSS/reference-map data | Satellite retrieval, anchor verification, visual localization |
| VPAir | Strong aircraft nadir VPR/localization with GPS-derived poses | Academic-use restriction; raw IMU not confirmed | VPR and cross-view localization only if license allows |
| EuRoC MAV | Excellent synchronized camera/IMU/ground-truth VIO benchmark | Not fixed-wing nadir, indoor MAV | BASALT/OpenVINS/Kimera baseline sanity tests |
| UZH FPV | Synchronized camera/IMU/ground-truth high-dynamics benchmark | Not nadir fixed-wing; non-commercial license | Stress VIO robustness only if license allows |
## Data Isolation Strategy
Each test group runs against a fresh container restart of the gps-denied-system and ardupilot-sitl services. The camera-replay service is restarted and configured per test group (different frame subsets, different replay speeds, or different fault injection modes). MAVLink capture logs are isolated per test run via timestamped directories.
Every replay test uses read-only fixture mounts and writes results to a fresh `test-results/<run-id>/` directory. The system under test may write FDR and generated COG tiles only to run-scoped temporary volumes.
## Input Data Mapping
| Input Data File | Source Location | Description | Covers Scenarios |
|-----------------|----------------|-------------|-----------------|
| AD000001-060.jpg | `_docs/00_problem/input_data/` | 60 aerial images (6252x4168, 26MP) from a fixed-wing UAV at 400m altitude. Images taken at ~1 photo/2-3s (wider spacing than real 0.7fps but usable for functional tests) | FT-P-01 to FT-P-06, FT-N-01, FT-N-02, NFT-PERF-01, NFT-RES-01, NFT-RES-LIM-01 |
| AD000001_gmaps.png, AD000002_gmaps.png | `_docs/00_problem/input_data/` | Google Maps satellite reference images for frames 1-2 (used as sample satellite tile data for satellite matching validation) | FT-P-01, NFT-RES-02, NFT-RES-04 |
| coordinates.csv | `_docs/00_problem/input_data/coordinates.csv` | Ground truth GPS (lat, lon) for each of the 60 images | FT-P-01, FT-P-02, FT-P-03, FT-P-04 (comparison baseline) |
| data_parameters.md | `_docs/00_problem/input_data/data_parameters.md` | Camera specs: 400m altitude, ADTi Surveyor Lite 26S v2, 26MP, 25mm focal length, 23.5mm sensor | Test environment configuration |
| position_accuracy.csv | `_docs/00_problem/input_data/expected_results/position_accuracy.csv` | Per-frame ground truth with acceptance thresholds | FT-P-01, FT-P-02 (expected result comparison) |
| imu_synthetic_200hz.csv | `_docs/00_problem/input_data/` (TO BE GENERATED) | Synthetic 200Hz IMU data (accel + gyro) derived from coordinates.csv trajectory. Matches ICM-42688-P noise characteristics. Required for ESKF sensor fusion testing outside SITL. | FT-P-01 to FT-P-06, NFT-PERF-01, NFT-RES-01 to NFT-RES-06 |
### IMU Data Generation
No real IMU recordings exist for the 60-image flight sequence. Two approaches for providing IMU data during tests:
**Approach A — SITL-generated (primary)**: ArduPilot SITL flies a waypoint mission following the coordinates.csv trajectory. SITL's internal physics engine generates physically consistent IMU data at 200Hz, delivered via MAVLink to the GPS-denied system. This is the most realistic approach and requires no pre-generated files.
**Approach B — Synthetic CSV (fallback/replay)**: Generate `imu_synthetic_200hz.csv` offline from coordinates.csv:
1. Compute inter-frame velocities from GPS positions and timestamps
2. Interpolate position/velocity to 200Hz using cubic splines
3. Compute accelerations (body frame) accounting for gravity + flight dynamics
4. Add sensor noise matching ICM-42688-P specs (gyro: 3.0e-3 °/s/√Hz, accel: 70 µg/√Hz)
5. Add bias random walks (gyro: 5.0e-5 °/s²/√Hz, accel: 2.0e-3 m/s³/√Hz)
6. Replay via MAVLink injector service at 200Hz
Approach A is recommended for integration tests. Approach B is useful for deterministic unit-level ESKF tests where reproducible IMU streams are needed.
### Satellite Tile Data
Only 2 Google Maps screenshots exist (AD000001_gmaps.png, AD000002_gmaps.png). Full satellite tile coverage for the test area must be prepared:
1. Download Google Maps tiles at zoom 18 for the bounding box: 48.249-48.276°N, 37.340-37.386°E
2. Store as 256x256 JPEG tiles with geohash-based naming
3. Load into satellite-tile-server Docker service
4. Estimated: ~50-100 tiles for the test area (~1-2MB total)
|-----------------|----------------|-------------|------------------|
| `AD000001.jpg` ... `AD000060.jpg` | `_docs/00_problem/input_data/` | Project still-image set with expected WGS84 centers | FT-P-01, FT-P-02, NFT-PERF-01 |
| `coordinates.csv` | `_docs/00_problem/input_data/coordinates.csv` | Machine-readable expected frame centers | FT-P-01, FT-P-02 |
| `data_parameters.md` | `_docs/00_problem/input_data/data_parameters.md` | Height 400 m and camera model | FT-P-01, NFT-PERF-01 |
| `AD000001_gmaps.png`, `AD000002_gmaps.png` | `_docs/00_problem/input_data/` | Reference map screenshots for sample sanity checks | FT-P-02 |
| `flight_derkachi/flight_derkachi.mp4` + `flight_derkachi/data_imu.csv` | `_docs/00_problem/input_data/flight_derkachi/` | Cropped nadir video synchronized with IMU and `GLOBAL_POSITION_INT` GPS trajectory | FT-P-03, NFT-PERF-02, NFT-RES-02 |
| Public dataset slices | External fixture paths pinned during implementation | Synchronized camera/IMU/GNSS/ground truth where available | FT-P-03, FT-P-04, NFT-PERF-02, NFT-RES-02 |
## Expected Results Mapping
| Test Scenario ID | Input Data | Expected Result | Comparison Method | Tolerance | Expected Result Source |
|-----------------|------------|-----------------|-------------------|-----------|----------------------|
| FT-P-01 | flight-sequence-60 (60 frames) | ≥80% of frames within 50m of ground truth | percentage | ≥80% | `expected_results/position_accuracy.csv` |
| FT-P-02 | flight-sequence-60 (60 frames) | ≥60% of frames within 20m of ground truth | percentage | ≥60% | `expected_results/position_accuracy.csv` |
| FT-P-03 | flight-sequence-60 (60 frames) | No single frame exceeds 100m error | threshold_max | ≤100m | `expected_results/position_accuracy.csv` |
| FT-P-04 | flight-sequence-60 (selected satellite anchor pairs) | VO drift between satellite anchors <100m | threshold_max | ≤100m | inline |
| FT-P-05 | Single frame + satellite match | GPS_INPUT: fix_type=3, horiz_accuracy 5-20m, satellites_visible=10 | exact + range | fix_type==3, accuracy∈[1,50] | `expected_results/results_report.md` #5 |
| FT-P-06 | flight-sequence-60 | ≥57 of 60 frames registered (≥95%) | percentage | ≥95% | inline |
| FT-P-07 | Normal operation, satellite match <30s | Confidence tier: HIGH | exact | N/A | inline |
| FT-P-08 | VO tracking, no satellite >30s | Confidence tier: MEDIUM | exact | N/A | inline |
| FT-P-09 | GPS_INPUT stream | Messages at 5-10Hz | range | [5,10] Hz | inline |
| FT-P-10 | POST /objects/locate (known pixel, gimbal, zoom, UAV position) | lat/lon within accuracy_m of ground truth | numeric_tolerance | within accuracy_m | `expected_results/results_report.md` #27 |
| FT-P-11 | Known GPS → NED → pixel → GPS | Round-trip error <0.1m | threshold_max | ≤0.1m | inline |
| FT-P-12 | System boot + GLOBAL_POSITION_INT | GPS_INPUT output within 60s | threshold_max | ≤60s | inline |
| FT-N-01 | Frames 32-43 (direction change area) | System continues producing estimates | threshold_min | ≥1 output per frame | inline |
| FT-N-02 | Simulated 350m gap between frames | System handles outlier, next valid frame <100m error | threshold_max | ≤100m | inline |
| FT-N-03 | POST /objects/locate with invalid pixels | HTTP 422 | exact | status==422 | inline |
| FT-N-04 | Unauthenticated request to /sessions | HTTP 401 | exact | status==401 | inline |
| FT-N-05 | VO lost + 3 satellite failures | fix_type=0, horiz_accuracy=999.0, RELOC_REQ sent | exact + regex | fix_type==0, matches `RELOC_REQ:.*` | inline |
| FT-N-06 | VO lost + IMU-only | fix_type=2, horiz_accuracy≥50m growing | exact + threshold_min | fix_type==2, accuracy≥50 | inline |
|------------------|------------|-----------------|-------------------|-----------|------------------------|
| FT-P-01 | `AD000001.jpg` ... `AD000060.jpg` | Output WGS84 frame center per mapped row; >=80% within 50 m, >=50% within 20 m | Haversine distance threshold + aggregate pass rate | 50 m primary, 20 m stretch | `input_data/expected_results/results_report.md` |
| FT-P-02 | Same 60 images + map references where present | Output includes source label, covariance semi-major axis, and anchor age for every emitted estimate | Required-field validation + geolocation threshold | Required fields present; geolocation thresholds as above | `input_data/expected_results/results_report.md` |
| FT-P-03 | `derkachi_video_telemetry` plus public synchronized VIO dataset slice when available | BASALT + wrapper emits trajectory with calibrated covariance and no optimistic under-reporting | Compare Derkachi output to `GLOBAL_POSITION_INT` trajectory for smoke/relative validation; compare public/representative calibrated runs to ground truth for final accuracy | Derkachi threshold is calibration-gated; final threshold is dataset-specific and pinned after camera calibration | `data_imu.csv` trajectory plus public dataset ground truth |
| FT-P-04 | AerialVL/ALTO/VPAir-style aerial localization slice | Satellite retrieval returns candidate chunks and local verification produces accepted/rejected anchors | Georeference error + MRE + source-label checks | AC-1.1/1.2 and AC-2.2 thresholds where dataset supports them | Public dataset ground truth/reference map |
| FT-N-01 | Low-texture/repetitive frames from sample or public data | System emits degraded confidence or rejects anchor rather than confident false fix | Source label and covariance threshold | No `satellite_anchored` label unless gates pass | Fixture-specific |
| FT-N-02 | Plane SITL GPS spoof/loss trace | Spoofed GPS rejected; system promotes own estimate within <3 s when trigger conditions are met | Event timing and MAVLink field checks | <3 s promotion; blackout thresholds from AC-NEW-8 | Generated SITL trace |
| FT-N-03 | Stale/unsigned/hash-mismatched cache fixtures | Anchors rejected or downgraded; stale tile never emits `satellite_anchored` | Manifest validation + emitted label check | 0 accepted stale/invalid anchors | Cache fixture manifest |
## External Dependency Mocks
| External Service | Mock/Stub | How Provided | Behavior |
|-----------------|-----------|-------------|----------|
| Flight Controller (ArduPilot) | ArduPilot SITL | Docker service (ardupilot-sitl) | Full MAVLink protocol: heartbeat, GLOBAL_POSITION_INT, IMU data at 200Hz, accepts GPS_INPUT, responds to COMMAND_LONG |
| Camera hardware (ADTI 20L V1) | Frame replay server | Docker service (camera-replay) | Serves frames from input_data/ at configurable rate (0.7fps default); supports fault injection (frame drop, delayed frame, corrupted JPEG) |
| Satellite imagery (Google Maps) | Static tile server | Docker service (satellite-tile-server) | Serves pre-cached tiles via HTTP; supports fault injection (404 for missing tiles, slow response) |
| Ground station | MAVLink inspector | Docker service (mavlink-inspector) | Captures STATUSTEXT and NAMED_VALUE_FLOAT; can inject COMMAND_LONG (operator re-localization hint) |
| GPU (CUDA/TensorRT) | CPU fallback or Jetson hardware | Conditional | Docker: CPU-mode stubs for TRT inference (slower but functionally equivalent). Jetson: real GPU |
|------------------|-----------|--------------|----------|
| Azaion Suite Satellite Service | Offline cache stub | Local COG/manifest/descriptor fixture | Provides only preloaded tiles; no in-flight network fetch |
| Flight controller | ArduPilot Plane SITL and MAVLink replay | SITL container/process and recorded/generated tlogs | Emits IMU, attitude, altitude, airspeed, GPS health/spoofing events |
| QGroundControl | MAVLink observer/log parser | Test-side parser | Verifies downsampled status and `STATUSTEXT` events |
## Data Validation Rules
| Data Type | Validation | Invalid Examples | Expected System Behavior |
|-----------|-----------|-----------------|------------------------|
| Camera frame (JPEG) | Valid JPEG, resolution ≥ 1280x720 | Truncated JPEG, 0-byte file, BMP format | Log warning, skip frame, continue with IMU-only ESKF prediction |
| GPS coordinate | lat ∈ [-90, 90], lon ∈ [-180, 180] | lat=999, lon=NaN | Reject, use last valid position |
| IMU data | Acceleration ∈ [-160, 160] m/s², gyro ∈ [-35, 35] rad/s | All zeros, NaN values, extreme spikes | Filter outliers via ESKF process noise, log warning |
| Satellite tile | Valid JPEG/PNG, 256x256 px | Missing tile (404), corrupted image | Skip tile, expand search radius, fall back to wider area |
| API request body | Valid JSON, required fields present | Missing pixel_x, non-numeric zoom_factor | HTTP 422 with validation error details |
| JWT token | Valid signature, not expired | Expired token, invalid signature, missing token | HTTP 401 Unauthorized |
|-----------|------------|------------------|--------------------------|
| Image frame | Existing file, readable image, expected timestamp/order metadata if sequence replay | Missing image, corrupt image, unsupported resolution | Mark estimate unavailable/degraded, log error, continue if possible |
| Expected coordinate | Valid WGS84 latitude/longitude | Out-of-range lat/lon, missing row | Reject test fixture before replay |
| Video/telemetry pair | MP4 duration matches telemetry duration, frame-to-telemetry ratio is stable, timestamps are monotonic | Duration drift >250 ms, missing trajectory columns, non-monotonic timestamps | Reject fixture before replay |
| IMU trace | Monotonic timestamps, angular rate/accel fields, calibrated units | Non-monotonic timestamps, missing samples | Reject fixture or enter degraded mode depending scenario |
| GPS trajectory trace | Valid WGS84 lat/lon, altitude, velocity, and heading fields | Out-of-range lat/lon, impossible altitude, missing `GLOBAL_POSITION_INT` columns | Reject trajectory comparison while allowing pure video replay if applicable |
| Cache tile manifest | CRS, m/px, capture date, source, hashes, signature/provenance | Stale, unsigned, hash mismatch, low resolution | Reject or down-confidence per AC-8.2 and AC-NEW-6 |
| MAVLink output | Valid `GPS_INPUT` fields and fix type/accuracy semantics | Missing `horiz_accuracy`, impossible fix type | Fail test; output contract violated |
## Phase 3 Validation Gate Result
| Test Scenario ID | Shape | Required Input Data | Required Expected Result | Input Provided? | Expected Result Provided? | Validation Decision |
|------------------|-------|---------------------|--------------------------|-----------------|---------------------------|---------------------|
| FT-P-01 | Input/output | 60 project images + `coordinates.csv` | WGS84 center per image with 50 m / 20 m thresholds | Yes | Yes | Keep |
| FT-P-02 | Input/output | 60 project images + output schema expectations | Required confidence/source-label fields and thresholds | Yes | Yes | Keep |
| FT-P-03 | Input/output | Derkachi synchronized video/IMU/GPS fixture; public or calibrated representative dataset for final accuracy | Derkachi `GLOBAL_POSITION_INT` trajectory for smoke/relative validation; calibrated ground truth for final covariance checks | Yes for Derkachi; public/calibrated dataset still useful for final signoff | Yes for Derkachi GPS trajectory; calibrated camera thresholds pending | Keep with calibration gate |
| FT-P-04 | Input/output | Public aerial localization or project cache fixture | Georeference, MRE, and source-label checks | Accepted as required external fixture | Accepted as dataset/reference-map ground truth | Keep with acquisition task |
| FT-N-01 | Behavioral/input-output | Ambiguous low-texture/repetitive frames | 0 confident false anchors | Accepted as project/public fixture | Yes | Keep |
| FT-N-02 | Behavioral | Generated Plane SITL spoof/blackout trace | Timing and MAVLink field thresholds from AC-NEW-8 | Generated by test harness | Yes | Keep |
| FT-N-03 | Behavioral/input-output | Cache integrity fixtures | 0 trusted anchors from stale/invalid tiles | Generated fixture | Yes | Keep |
| NFT-PERF-01 | Input/output | 60 project images | p95 latency and drop-rate thresholds | Yes | Yes | Keep |
| NFT-PERF-02 | Input/output | Derkachi synchronized video/IMU/GPS fixture; public/representative synchronized camera/IMU dataset | VO registration, latency, memory thresholds | Yes for Derkachi | Yes | Keep with calibration gate |
| NFT-PERF-03 | Behavioral/input-output | Precomputed descriptor/cache fixture | Trigger-path latency and MRE thresholds | Generated fixture | Yes | Keep |
| NFT-PERF-04 | Behavioral | Cold-start harness and cache fixture | <30 s p95 over 50 runs | Generated by test harness | Yes | Keep |
| NFT-RES-* | Behavioral | Fault triggers and generated traces | AC-defined timing/status thresholds | Generated by test harness | Yes | Keep |
| NFT-SEC-* | Behavioral/input-output | Cache/MAVLink/network fixtures | Rejection/no-fetch/no-promote thresholds | Generated fixture | Yes | Keep |
| NFT-RES-LIM-* | Behavioral | Jetson/cache/FDR monitoring environment | Numeric resource thresholds | Environment-dependent | Yes | Keep |
**Coverage after validation**: 49/49 AC and restriction groups remain covered. No tests were removed.
**Acquisition tasks required downstream**:
- Pin camera intrinsics, lens distortion, raw camera feed parameters, and camera-to-body mounting transform for the Derkachi fixture or future representative recordings.
- Pin and download at least one strong synchronized nadir camera + IMU + ground-truth dataset, preferably MUN-FRL or ALTO, with EPFL fixed-wing and Kagaru as fixed-wing/farmland candidates.
- Pin license-compatible VPR/localization datasets for satellite anchor tests; VPAir and UZH FPV have non-commercial restrictions and must not be used for commercial acceptance unless license terms allow it.
- Create generated fixtures for Plane SITL spoofing, stale cache manifests, signed/unsigned manifests, FDR load, and thermal/resource monitoring during implementation.
_docs/02_document/tests/traceability-matrix.md
+66 -54
View File
@@ -2,68 +2,80 @@
## Acceptance Criteria Coverage
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-01 | 80% of frames within 50m of real GPS | FT-P-01, FT-P-03 | Covered |
| AC-02 | 60% of frames within 20m of real GPS | FT-P-02 | Covered |
| AC-03 | Cumulative VO drift between satellite anchors < 100m | FT-P-04 | Covered |
| AC-04 | Confidence score per position estimate (high/low) | FT-P-07, FT-P-08, FT-N-06 | Covered |
| AC-05 | Image registration rate > 95% | FT-P-06 | Covered |
| AC-06 | System handles 350m outlier between consecutive photos | FT-N-02 | Covered |
| AC-07 | System handles sharp turns with <5% overlap, <200m drift, <70° angle via satellite re-localization | FT-N-01, NFT-RES-02 | Covered |
| AC-08 | System handles disconnected route segments (core feature) | FT-N-01, NFT-RES-02, NFT-RES-03 | Covered |
| AC-09 | 3 consecutive failures → re-localization request to ground station | FT-N-05, FT-N-07 | Covered |
| AC-10 | < 400ms end-to-end per frame | NFT-PERF-01 | Covered |
| AC-11 | Memory < 8GB shared | NFT-RES-LIM-01, NFT-RES-LIM-04 | Covered |
| AC-12 | GPS_INPUT via MAVLink (MAVSDK) to flight controller | FT-P-05, FT-P-09 | Covered |
| AC-13 | Frame-by-frame streaming, no batch/delay | FT-P-09, NFT-PERF-02 | Covered |
| AC-14 | System initializes from last known GPS position | FT-P-12 | Covered |
| AC-15 | Complete failure for N seconds → FC falls back to IMU, system logs | NFT-RES-03, FT-N-06 | Covered |
| AC-16 | Mid-flight reboot → re-initialize from FC's IMU-extrapolated position | NFT-RES-01 | Covered |
| AC-17 | Position + confidence streamed to ground station via telemetry | FT-P-13, FT-P-14 | Covered |
| AC-18 | Ground station can send commands (re-localization hint) | FT-N-07 | Covered |
| AC-19 | Output coordinates in WGS84 | FT-P-05, FT-P-11 | Covered |
| AC-20 | Other onboard AI can request object GPS coordinates | FT-P-10 | Covered |
| AC-21 | Object coordinates via trigonometric calculation (gimbal, zoom, altitude) | FT-P-10, FT-P-11 | Covered |
| AC-22 | Object accuracy consistent with frame-center accuracy | FT-P-10 | Covered |
| AC-23 | Satellite imagery ≥ 0.5 m/pixel | NFT-RES-LIM-03 | Covered (tile storage validation uses zoom 18 = 0.6m/px) |
| AC-24 | Satellite imagery pre-loaded before flight | NFT-RES-LIM-03 | Covered |
| AC-25 | MRE < 1.0 pixels | — | NOT COVERED — requires cuVSLAM internal reprojection metric; not observable via black-box interfaces. Covered at component test level (Step 5). |
| AC-26 | Satellite imagery < 2 years old | — | NOT COVERED — operational procurement constraint; not runtime-testable. Validated during offline tile preparation. |
| AC ID | Acceptance Criterion Summary | Test IDs | Coverage |
|-------|------------------------------|----------|----------|
| AC-1.1 | >=80% frame centers within 50 m | FT-P-01, FT-P-04 | Covered |
| AC-1.2 | >=50% frame centers within 20 m | FT-P-01, FT-P-04 | Covered |
| AC-1.3 | Drift and anchor age reporting | FT-P-02, FT-P-03, NFT-RES-02 | Covered |
| AC-1.4 | Quantitative confidence and source label | FT-P-02, FT-N-01, FT-P-03 | Covered |
| AC-2.1a | VO registration >95% on normal segments | FT-P-03, NFT-PERF-02 | Covered |
| AC-2.1b | Satellite-anchor registration measured separately | FT-P-04 | Covered |
| AC-2.2 | MRE <1 px VO and <2.5 px satellite anchor | FT-P-03, FT-P-04 | Covered |
| AC-3.1 | Handles up to 350 m outliers | FT-N-01, NFT-RES-02 | Covered |
| AC-3.2 | Sharp turn relocalization | FT-P-04, NFT-RES-02, NFT-PERF-03 | Covered |
| AC-3.3 | >=3 disconnected segments via retrieval/relocalization | NFT-RES-02, FT-P-04 | Covered |
| AC-3.4 | Relocalization request after loss threshold | NFT-RES-02 | Covered |
| AC-3.5 | Total visual blackout/occlusion + GPS spoofing degraded mode | FT-N-02, NFT-RES-01, NFT-SEC-03 | Covered |
| AC-4.1 | End-to-end latency <400 ms p95 | NFT-PERF-01, NFT-PERF-02, NFT-PERF-03 | Covered |
| AC-4.2 | Memory below 8 GB shared | NFT-RES-LIM-01, NFT-RES-LIM-05 | Covered |
| AC-4.3 | MAVLink `GPS_INPUT` output for ArduPilot | FT-N-02, NFT-SEC-03 | Covered |
| AC-4.4 | Frame-by-frame streaming | FT-P-02, NFT-PERF-01 | Covered |
| AC-4.5 | Previous estimate corrections | FT-P-02 | Covered |
| AC-5.1 | Initialize from last trusted FC state | NFT-RES-03, NFT-PERF-04 | Covered |
| AC-5.2 | >3 s no-estimate fallback behavior | FT-N-02, NFT-RES-01 | Covered |
| AC-5.3 | Re-initialize after companion reboot | NFT-RES-03 | Covered |
| AC-6.1 | QGC status at 1-2 Hz | FT-N-02, NFT-SEC-03 | Covered |
| AC-6.2 | Ground station command ingress | FT-P-04, NFT-RES-02 | Covered |
| AC-6.3 | WGS84 output | FT-P-01, FT-P-02 | Covered |
| AC-7.1 | Object localization accuracy consistent with frame center in level flight | FT-P-01 | Covered at system boundary; detailed AI-camera tests deferred to component specs |
| AC-7.2 | Object coordinates from UAV position, gimbal angle, zoom, altitude | FT-P-01 | Covered at system boundary; detailed AI-camera tests deferred to component specs |
| AC-8.1 | Cache imagery 0.5 m/px minimum, 0.3 m/px ideal | FT-P-04, NFT-SEC-04 | Covered |
| AC-8.2 | Tile freshness thresholds | FT-N-03, NFT-RES-04, NFT-SEC-01 | Covered |
| AC-8.3 | Preloaded/preprocessed offline cache | NFT-SEC-04, NFT-RES-LIM-03 | Covered |
| AC-8.4 | Mid-flight tile generation and write-back | NFT-SEC-02, NFT-RES-LIM-04 | Covered |
| AC-8.5 | No raw frame retention | NFT-RES-LIM-04 | Covered |
| AC-8.6 | VPR retrieval chunks, multi-scale, dynamic K | FT-P-04, NFT-PERF-03 | Covered |
| AC-NEW-1 | Cold start first fix <30 s | NFT-PERF-04, NFT-RES-LIM-05, NFT-RES-03 | Covered |
| AC-NEW-2 | Spoofing promotion <3 s | FT-N-02, NFT-SEC-03 | Covered |
| AC-NEW-3 | FDR <=64 GB and payload retention | NFT-RES-LIM-04 | Covered |
| AC-NEW-4 | False-position safety budget | FT-N-01, NFT-SEC-01 | Covered |
| AC-NEW-5 | Thermal envelope and no throttle | NFT-RES-LIM-02 | Covered |
| AC-NEW-6 | Imagery freshness enforcement | FT-N-03, NFT-RES-04 | Covered |
| AC-NEW-7 | Cache-poisoning safety budget | FT-N-03, NFT-SEC-02 | Covered |
| AC-NEW-8 | Pre-VIO total occlusion gate, IMU-only blackout propagation, and spoofing degraded-mode budget | FT-N-02, NFT-RES-01 | Covered |
## Restrictions Coverage
| Restriction ID | Restriction | Test IDs | Coverage |
|---------------|-------------|----------|----------|
| RESTRICT-01 | Fixed-wing UAV only | FT-P-01 through FT-P-06 (test data from fixed-wing flight) | Covered |
| RESTRICT-02 | Camera pointing downwards, not autostabilized | FT-P-10, FT-P-11 (coordinate transforms account for non-stabilized mount) | Covered |
| RESTRICT-03 | Eastern/southern Ukraine operational area | FT-P-01 (test coordinates at 48.25-48.28°N, 37.34-37.39°E) | Covered |
| RESTRICT-04 | Altitude ≤ 1km, terrain height negligible | FT-P-01, FT-P-10 (test data at 400m altitude, flat terrain assumed) | Covered |
| RESTRICT-05 | Mostly sunny weather | — | NOT COVERED — environmental condition; cannot be tested in Docker. Mitigated by feature-based matching robustness. |
| RESTRICT-06 | Sharp turns (no common points possible, exceptional) | FT-N-01, FT-N-02, NFT-RES-02 | Covered |
| RESTRICT-07 | Up to 3000 photos per flight | NFT-RES-LIM-04 | Covered |
| RESTRICT-08 | Two cameras: navigation (fixed nadir) + AI (configurable) | FT-P-10 (object localization uses AI camera angles) | Covered |
| RESTRICT-09 | Navigation camera: FullHD to 6252x4168, known parameters | FT-P-01 (test data at 6252x4168) | Covered |
| RESTRICT-10 | Jetson Orin Nano Super: 67 TOPS, 8GB, 25W TDP | NFT-RES-LIM-01, NFT-RES-LIM-02, NFT-PERF-01 | Covered |
| RESTRICT-11 | Thermal throttling at sustained GPU load | NFT-RES-LIM-02 | Covered |
| RESTRICT-12 | IMU data via flight controller (MAVLink) | FT-P-05, FT-P-12 (SITL provides IMU via MAVLink) | Covered |
| RESTRICT-13 | GPS_INPUT message to flight controller | FT-P-05, FT-P-09 | Covered |
| RESTRICT-14 | Telemetry link bandwidth-limited | FT-P-13 (1Hz telemetry rate is bandwidth-appropriate) | Covered |
| RESTRICT-15 | Google Maps satellite (potentially outdated) | NFT-RES-04 (tile server failure resilience) | Covered |
| RESTRICT-16 | Onboard storage limited for satellite tiles | NFT-RES-LIM-03 | Covered |
| Restriction ID | Restriction Summary | Test IDs | Coverage |
|----------------|---------------------|----------|----------|
| R-UAV-01 | Fixed-wing UAV mission profile and 8-hour operations | NFT-RES-LIM-02, NFT-RES-LIM-04 | Covered |
| R-CAM-01 | Fixed downward navigation camera | FT-P-01, FT-P-03 | Covered |
| R-CAM-02 | ADTi 20MP 20L V1 and calibration assumptions | FT-P-01, NFT-PERF-01 | Covered |
| R-SAT-01 | Offline-only Satellite Service cache, no in-flight provider fetch | NFT-SEC-04 | Covered |
| R-SAT-02 | Cache resolution/freshness/metadata conventions | FT-N-03, NFT-RES-LIM-03 | Covered |
| R-HW-01 | Jetson Orin Nano Super 8 GB / 25 W | NFT-RES-LIM-01, NFT-RES-LIM-02 | Covered |
| R-SENSOR-01 | FC IMU available; original still-image sample lacks synchronized IMU; Derkachi fixture provides video/IMU/GPS trajectory but calibration is pending | FT-P-03, NFT-PERF-02 | Covered through Derkachi representative replay plus public/calibrated dataset plan |
| R-MAV-01 | MAVLink, ArduPilot only, GPS_INPUT via pymavlink | FT-N-02, NFT-SEC-03 | Covered |
| R-GCS-01 | QGroundControl supported GCS | FT-N-02, NFT-SEC-03 | Covered |
| R-SAFETY-01 | False-position, cold-start, spoofing, and failsafe constraints | FT-N-01, FT-N-02, NFT-PERF-04, NFT-RES-01 | Covered |
## Coverage Summary
| Category | Total Items | Covered | Not Covered | Coverage % |
|----------|-----------|---------|-------------|-----------|
| Acceptance Criteria | 26 | 24 | 2 | 92% |
| Restrictions | 16 | 15 | 1 | 94% |
| **Total** | **42** | **39** | **3** | **93%** |
|----------|-------------|---------|-------------|------------|
| Acceptance Criteria | 39 | 39 | 0 | 100% |
| Restrictions | 10 | 10 | 0 | 100% |
| **Total** | 49 | 49 | 0 | 100% |
## Uncovered Items Analysis
| Item | Reason Not Covered | Risk | Mitigation |
|------|-------------------|------|-----------|
| AC-25 (MRE < 1.0 pixels) | cuVSLAM reprojection error is an internal metric not exposed via public API or MAVLink; cannot be observed at black-box level | LOW — MRE is a proxy for VO quality; position accuracy tests (FT-P-01/02) validate the end result | Covered at component test level (Step 5) with cuVSLAM-specific unit tests |
| AC-26 (Satellite imagery < 2 years old) | Operational procurement constraint validated during offline tile preparation, not at runtime | LOW — outdated imagery degrades matching but is caught by satellite matching accuracy tests | Validated during pre-flight tile download workflow; documented in operational procedures |
| RESTRICT-05 (Sunny weather) | Environmental condition that cannot be reproduced in Docker or controlled test environment | LOW — system uses feature-based matching (LiteSAM/XFeat) robust to lighting variation; not a software test | Validated during real-world flight tests; feature-based matching provides robustness |
|------|--------------------|------|------------|
| None | All current AC and restriction groups have black-box coverage | N/A | Continue to Phase 3 validation gate |
## Data Coverage Caveats
- Current project data fully supports still-image frame-center checks for 60 mapped images.
- Derkachi project data supports synchronized video/IMU/GPS trajectory replay for FT-P-03 and NFT-PERF-02.
- Derkachi project data is calibration-limited: raw camera intrinsics, lens distortion, and camera-to-body transform are still required before final absolute accuracy thresholds can be treated as production acceptance.
- Phase 3 must validate camera calibration inputs and public/calibrated dataset acquisition before FT-P-03, FT-P-04, and NFT-PERF-02 can be used for final signoff.
_docs/02_tasks/_dependencies_table.md
+49
View File
@@ -0,0 +1,49 @@
# Dependencies Table
**Date**: 2026-05-04
**Total Tasks**: 24
**Total Complexity Points**: 108
**Lessons applied**: No `_docs/LESSONS.md` file exists; no prior estimation or dependency lessons were available.
| Task | Name | Complexity | Dependencies | Epic |
|------|------|------------|--------------|------|
| AZ-219 | initial_structure | 5 | None | AZ-206 |
| AZ-220 | shared_runtime_contracts | 3 | AZ-219 | AZ-206 |
| AZ-221 | shared_geometry_time_sync | 3 | AZ-219, AZ-220 | AZ-207 |
| AZ-222 | runtime_config_errors_telemetry | 3 | AZ-219, AZ-220 | AZ-208 |
| AZ-223 | camera_ingest_calibration | 5 | AZ-220, AZ-221, AZ-222 | AZ-209 |
| AZ-224 | mavlink_gcs_gateway | 3 | AZ-220, AZ-222 | AZ-210 |
| AZ-225 | tile_manager_cache_manifest | 5 | AZ-220, AZ-221, AZ-222 | AZ-211 |
| AZ-226 | generated_tile_orthorectification | 5 | AZ-223, AZ-225 | AZ-211 |
| AZ-227 | fdr_event_recorder | 5 | AZ-220, AZ-222 | AZ-212 |
| AZ-228 | vio_adapter | 5 | AZ-221, AZ-222, AZ-223, AZ-224 | AZ-213 |
| AZ-229 | satellite_service_sync | 3 | AZ-222, AZ-225, AZ-226 | AZ-214 |
| AZ-230 | satellite_service_vpr_retrieval | 5 | AZ-223, AZ-225, AZ-229 | AZ-214 |
| AZ-231 | anchor_verification_matching | 5 | AZ-223, AZ-225, AZ-230 | AZ-215 |
| AZ-232 | safety_anchor_state_machine | 5 | AZ-223, AZ-224, AZ-227, AZ-228, AZ-231 | AZ-216 |
| AZ-240 | native_vio_backend_integration | 5 | AZ-228 | AZ-213 |
| AZ-241 | real_satellite_vpr_descriptor_retrieval | 5 | AZ-230 | AZ-214 |
| AZ-242 | real_anchor_feature_matching_ransac | 5 | AZ-231, AZ-241 | AZ-215 |
| AZ-233 | test_infrastructure | 5 | AZ-240, AZ-241, AZ-242 | AZ-218 |
| AZ-234 | replay_geolocation_confidence_tests | 3 | AZ-233 | AZ-218 |
| AZ-235 | vio_replay_performance_tests | 5 | AZ-233, AZ-240 | AZ-218 |
| AZ-236 | satellite_anchor_cache_tests | 5 | AZ-233, AZ-241, AZ-242 | AZ-218 |
| AZ-237 | mavlink_blackout_spoofing_tests | 5 | AZ-233 | AZ-218 |
| AZ-238 | cold_start_restart_tests | 5 | AZ-233 | AZ-218 |
| AZ-239 | jetson_resource_endurance_tests | 5 | AZ-233 | AZ-218 |
## Verification Notes
- No task exceeds 5 complexity points.
- Test implementation tasks are appended under Blackbox Tests (AZ-218); the test infrastructure bootstrap now depends on the product remediation tasks so tests do not validate scaffold behavior.
- The graph is acyclic: product foundations precede adapters/stores, then VIO/retrieval/matching, then safety wrapper orchestration; remediation tasks close native VIO, real VPR, and real matching gaps before affected blackbox tests run.
## Test Coverage Verification
- AZ-234 covers FT-P-01, FT-P-02, and NFT-PERF-01.
- AZ-235 covers FT-P-03 and NFT-PERF-02 after AZ-240 provides the real native VIO path.
- AZ-236 covers FT-P-04, FT-N-01, FT-N-03, NFT-PERF-03, NFT-RES-04, NFT-SEC-01, NFT-SEC-02, NFT-SEC-04, and NFT-RES-LIM-03 after AZ-241 and AZ-242 provide real VPR retrieval and anchor matching.
- AZ-237 covers FT-N-02, NFT-RES-01, and NFT-SEC-03.
- AZ-238 covers NFT-RES-02, NFT-RES-03, NFT-PERF-04, and NFT-RES-LIM-05.
- AZ-239 covers NFT-RES-LIM-01, NFT-RES-LIM-02, and NFT-RES-LIM-04.
- All traceability-matrix AC and restriction groups remain covered by at least one test task.
_docs/02_tasks/done/AZ-219_initial_structure.md
+289
View File
@@ -0,0 +1,289 @@
# Initial Project Structure
**Task**: AZ-219_initial_structure
**Name**: Initial Structure
**Description**: Scaffold the project skeleton for the Jetson-hosted GPS-denied localization runtime, replay harness, local infrastructure, CI, and deployment evidence paths.
**Complexity**: 5 points
**Dependencies**: None
**Component**: Bootstrap
**Tracker**: AZ-219
**Epic**: AZ-206
## Project Folder Layout
```text
project-root/
├── src/
│ ├── __init__.py
│ ├── camera_ingest_calibration/
│ ├── vio_adapter/
│ │ └── native/
│ ├── safety_anchor_wrapper/
│ ├── satellite_service/
│ │ └── native/
│ ├── anchor_verification/
│ │ └── native/
│ ├── tile_manager/
│ ├── mavlink_gcs_integration/
│ ├── fdr_observability/
│ ├── shared/
│ │ ├── contracts/
│ │ ├── geo_geometry/
│ │ ├── time_sync/
│ │ ├── config/
│ │ ├── errors/
│ │ └── telemetry/
├── migrations/
│ ├── postgresql/
│ └── seed/
├── tests/
│ ├── unit/
│ ├── integration/
│ ├── blackbox/
│ ├── fixtures/
│ ├── sitl/
│ └── e2e/
├── e2e/
│ ├── replay/
│ └── reports/
├── deployment/
│ ├── docker/
│ ├── compose/
│ ├── jetson/
│ └── scripts/
├── config/
│ ├── development/
│ ├── ci/
│ ├── jetson/
│ └── production/
├── data/
│ ├── input/
│ ├── expected/
│ ├── cache/
│ ├── fdr/
│ └── test-results/
├── .github/
│ └── workflows/
│ └── ci.yml
├── docker-compose.yml
├── docker-compose.test.yml
├── .dockerignore
└── .env.example
```
### Layout Rationale
The runtime is organized directly under `src/` because this repository already represents the GPS-denied onboard system. Component directories live at the source root, and native bridges stay inside the component folder that owns the backend or hot path. Shared contracts, geometry, time-sync, configuration, error envelopes, and telemetry DTOs are centralized so component tasks consume a single public interface instead of duplicating cross-cutting logic.
The scaffold separates runtime source, migrations, tests, deployment assets, configuration, and mutable data. Production runs on Jetson hardware, while Docker/compose is used for replay, SITL, and deterministic CI environments.
## DTOs and Interfaces
### Shared DTOs
| DTO Name | Used By Components | Fields Summary |
|----------|--------------------|----------------|
| `FramePacket` | Camera ingest, BASALT VIO, Satellite Service, anchor verification, Tile Manager, FDR | Frame ID, timestamp, image reference, camera calibration ID, occlusion status, quality metrics |
| `TelemetrySample` | MAVLink/GCS, BASALT VIO, safety wrapper, FDR, e2e tests | Timestamp, IMU, attitude, airspeed, altitude, GPS health |
| `VioStatePacket` | BASALT VIO, safety wrapper, FDR, e2e tests | Timestamp, relative pose, velocity, bias, tracking quality, covariance hint |
| `PositionEstimate` | Safety wrapper, MAVLink/GCS, Tile Manager, FDR, e2e tests | WGS84 coordinates, covariance semi-major axis, source label, fix type, horizontal accuracy, anchor age |
| `VprCandidate` | Satellite Service, anchor verification, FDR | Chunk ID, tile ID, score, footprint, freshness status |
| `AnchorDecision` | Anchor verification, safety wrapper, FDR | Candidate ID, acceptance result, estimated pose, inliers, MRE, rejection reason |
| `CacheTileRecord` | Tile Manager, Satellite Service, anchor verification, FDR | Tile ID, type, CRS, meters per pixel, capture date, signature/hash status, trust level |
| `FdrEvent` | All runtime components, e2e tests | Event type, timestamp, component, severity, payload reference, mission/run ID |
| `ScenarioReport` | Separate e2e test suite, CI/CD, release evidence | Scenario ID, result, metrics, artifacts, failure reason |
### Component Interfaces
| Component | Interface | Methods | Exposed To |
|-----------|-----------|---------|------------|
| Camera ingest/calibration | `FrameProvider` | `next_frame`, `detect_occlusion`, `classify_quality` | VIO Adapter, Satellite Service, anchor verification, Tile Manager |
| VIO adapter | `VioAdapter` | `initialize`, `process`, `health` | Safety wrapper, e2e tests |
| Safety/anchor wrapper | `LocalizationStateMachine` | `update_vio`, `consider_anchor`, `degrade`, `propagate_imu_only`, `tile_write_eligibility` | MAVLink/GCS, Tile Manager, FDR, e2e tests |
| Satellite Service | `SatelliteService` | `import_mission_cache`, `load_index`, `retrieve`, `upload_generated_tiles` | Safety wrapper, anchor verification, Tile Manager |
| Anchor verification | `AnchorVerifier` | `verify`, `benchmark_matcher` | Safety wrapper, FDR |
| Tile Manager | `TileManager` | `validate_cache`, `get_tile_window`, `orthorectify_frame`, `write_generated_tile`, `package_sync` | Satellite Service, anchor verification, post-flight sync |
| MAVLink/GCS integration | `MavlinkGateway` | `subscribe_telemetry`, `emit_gps_input`, `emit_status` | BASALT VIO, safety wrapper, QGC, FDR |
| FDR/observability | `FlightRecorder` | `append_event`, `rollover`, `export` | All runtime components, e2e tests |
## CI/CD Pipeline
| Stage | Purpose | Trigger |
|-------|---------|---------|
| Format / lint | Enforce code style and static quality | Every PR and push to `dev` |
| Unit tests | Validate component-local behavior and shared contracts | Every PR and push to `dev` |
| Replay black-box smoke | Run deterministic still-image/cache/SITL subsets | Every PR |
| Cache/security fixture tests | Validate signed manifests, stale-tile rejection, and no mid-flight provider/Satellite Service calls | Every PR |
| Plane SITL spoof/failsafe | Validate ArduPilot Plane `GPS_INPUT`, failsafe, spoofing promotion | Nightly and release candidate |
| Public dataset replay | Exercise VIO, retrieval, and anchor behavior against pinned public slices | Nightly and release candidate |
| Jetson latency/resource tests | Measure p95 latency, memory, cold start, TensorRT/ONNX fidelity | Release candidate |
| Thermal/FDR endurance | Prove 8-hour 25 W no-throttle and <=64 GB FDR rollover | Hardware qualification |
| Build / package | Produce replay-compatible and Jetson deployment artifacts | Release candidate |
| Deploy staging evidence | Publish reports, tlogs, FDR summaries, cache validation artifacts | Release candidate |
### Pipeline Configuration Notes
CI uses Docker/compose for replay and SITL gates. Jetson, thermal, camera, and representative replay gates run on dedicated hardware runners and block release rather than every PR. Dataset downloads are license-tagged and not baked into images. Secrets, signing keys, and Satellite Service credentials are never cached.
## Environment Strategy
| Environment | Purpose | Configuration Notes |
|-------------|---------|---------------------|
| Development replay | Fast local iteration | Small fixture cache, local PostgreSQL/PostGIS, replay frames, test keys only |
| CI replay | Deterministic PR checks | Docker services for runtime, replay consumer, cache stub, and reports |
| Public dataset replay | Algorithm de-risking | Pinned dataset slices with license metadata and ground-truth reports |
| Plane SITL | MAVLink/failsafe validation | ArduPilot Plane SITL, QGC/tlog observer, production-like `GPS_INPUT` parameters |
| Jetson hardware validation | Production path profiling | JetPack/CUDA/TensorRT, local camera/GPU/MAVLink access, resource monitoring |
| Representative flight/replay | Final acceptance evidence | Target-like UAV, FC, camera, synchronized telemetry, ground truth |
| Production | Onboard mission runtime | Preloaded signed cache, local PostGIS, per-flight FDR, no provider network calls |
### Environment Variables
| Variable | Dev | Staging | Production | Description |
|----------|-----|---------|------------|-------------|
| `GPSD_ENV` | `development` | `staging` | `production` | Selects runtime profile |
| `GPSD_CONFIG_DIR` | local path | staged config path | onboard config path | Versioned configuration root |
| `GPSD_CACHE_DIR` | fixture cache | staged mission cache | onboard mission cache | COG, manifest, sidecar, descriptor root |
| `GPSD_FDR_DIR` | temp output | staged evidence path | per-flight NVMe path | Flight data recorder output |
| `GPSD_DATABASE_URL` | local PostGIS | staging PostGIS | onboard local PostGIS | Manifest, mission state, FDR event index |
| `GPSD_MAVLINK_URL` | SITL/replay | SITL/hardware rig | FC telemetry link | MAVLink connection endpoint |
| `GPSD_CAMERA_SOURCE` | fixture/replay | fixture or hardware | live camera | Navigation frame source |
| `GPSD_SIGNING_KEY_REF` | test key ref | staging secret ref | mission secret ref | Cache/sidecar signature verification |
| `GPSD_MAX_FDR_BYTES` | small fixture cap | release-like cap | `68719476736` | FDR rollover threshold |
| `GPSD_LOG_LEVEL` | debug/info | info | info/warn | Runtime logging level |
## Database Migration Approach
**Migration tool**: Versioned PostgreSQL migration scripts, with PostGIS extension setup and deterministic seed scripts.
**Strategy**: Additive migrations by default. Database, table, and column renames require explicit approval before implementation. Runtime rejects unknown required schema versions loudly.
### Initial Schema
- Mission profile tables for route geometry, sector classification, altitude band, and cache budget.
- Camera calibration tables for camera model, intrinsics, extrinsics, and verification status.
- Cache manifest tables for COG tiles, generated tiles, freshness, signatures, sidecar hashes, trust level, and PostGIS footprints.
- VPR chunk tables for retrieval footprints, descriptor metadata, multi-scale index references, and sector top-K policy.
- FDR event index tables for mission/run IDs, timestamps, event type, component, severity, and CBOR segment references.
- Validation run tables or report records for scenario IDs, metrics, artifacts, and release evidence pointers.
## Test Structure
```text
tests/
├── unit/
│ ├── shared/
│ ├── camera_ingest_calibration/
│ ├── vio_adapter/
│ ├── safety_anchor_wrapper/
│ ├── satellite_service/
│ ├── anchor_verification/
│ ├── tile_manager/
│ ├── mavlink_gcs_integration/
│ ├── fdr_observability/
├── integration/
│ ├── contracts/
│ ├── cache_postgis/
│ ├── mavlink/
│ └── fdr/
├── blackbox/
│ ├── still_image_geolocation/
│ ├── satellite_anchor/
│ ├── visual_blackout_spoofing/
│ ├── cache_freshness/
│ └── resource_limits/
├── fixtures/
│ ├── project_60_images/
│ ├── expected_results/
│ ├── satellite_cache/
│ ├── telemetry/
│ └── public_dataset_slices/
├── sitl/
│ ├── plane_gps_input/
│ ├── spoofing_promotion/
│ └── failsafe/
└── e2e/
├── replay/
├── reports/
└── release_evidence/
```
### Test Configuration Notes
Unit tests validate contracts and component behavior without external hardware. Integration tests exercise PostGIS, MAVLink, FDR segments, and cache fixtures. Black-box tests interact only through public inputs and outputs: frames, telemetry, offline cache, `GPS_INPUT`, QGC status, and FDR. Release gates add Jetson hardware, thermal/endurance, and representative replay evidence.
## Health Checks
| Service / Component | Liveness | Readiness |
|---------------------|----------|-----------|
| GPS-denied service | Process event loop responsive | Config, PostGIS, cache, FDR path, MAVLink, and camera/replay source validated |
| Replay consumer | Runner process responsive | Fixtures, expected results, and output path available |
| Satellite cache stub | Fixture volume mounted | Manifests, sidecars, descriptors, and signatures validated |
| ArduPilot Plane SITL | SITL process responsive | MAVLink ports accepting telemetry and production parameters loaded |
| QGC observer/log parser | Parser process responsive | Tlog/status stream connected |
Each deployable service exposes `/health/live`, `/health/ready`, and `/metrics` where HTTP is available. Non-HTTP hardware processes write equivalent structured health events to FDR and CI reports.
## Docker And Compose Requirements
- Create a replay-compatible Dockerfile for the Python runtime and native optional stubs.
- Create a replay-consumer Dockerfile for black-box test execution.
- Create a satellite-cache-stub fixture image or volume contract.
- Create an ArduPilot Plane SITL service for integration tests.
- Use non-root users, pinned base images, multi-stage builds where native compilation is needed, and health checks for long-running services.
- Provide `docker-compose.yml` for local development/replay and `docker-compose.test.yml` for black-box/SITL test execution.
- Do not bake secrets, provider credentials, mission signing keys, or public dataset payloads into images.
## Implementation Order
| Order | Component | Reason |
|-------|-----------|--------|
| 1 | Bootstrap and shared contracts | Establish package, configuration, migrations, CI, DTOs, and error envelopes |
| 2 | Shared geometry/time helpers | Foundational math and timestamp contracts used by camera, VIO, cache, wrapper, and validation |
| 3 | Runtime configuration and error handling | Prevent duplicated config/error behavior across components |
| 4 | Camera ingest/calibration | Produces the frame and occlusion signals required by VIO, anchor, cache, and tests |
| 5 | MAVLink/GCS integration | Supplies FC telemetry DTOs and validates `GPS_INPUT` output contract early |
| 6 | Tile Manager | Owns PostGIS cache manifest, sidecars, COG access, freshness gates, and generated-tile orthorectification |
| 7 | FDR/observability | Provides audit path for all components and validation reports |
| 8 | VIO adapter | Depends on frame and telemetry contracts, blocks wrapper integration |
| 9 | Satellite Service | Depends on tile schema and frame DTOs, feeds anchor verification, and handles pre-flight/post-flight package sync |
| 10 | Anchor verification | Depends on retrieval candidates and cache tile access |
| 11 | Safety/anchor wrapper | Consumes VIO, anchor, camera degradation, MAVLink, and FDR contracts |
| 12 | Validation harness | Uses public interfaces once contracts and runtime components are stable |
| 13 | Black-box, SITL, Jetson, and endurance test tasks | Exercise release gates and acceptance criteria end to end |
## Acceptance Criteria
**AC-1: Project scaffolded**
Given the structure plan above
When the implementer executes this task
Then source, test, migration, deployment, configuration, and data directories exist with placeholder files where needed so empty directories are retained.
**AC-2: Shared contracts initialized**
Given the planned component interfaces
When the scaffold is complete
Then shared DTO, error, configuration, telemetry, geometry, and time-sync contract locations exist and are importable from component skeletons.
**AC-3: Local infrastructure defined**
Given the system requires local PostGIS, replay, SITL, cache, and FDR paths
When the scaffold is complete
Then `docker-compose.yml`, `docker-compose.test.yml`, `.env.example`, and migration seed directories describe all required local services and volumes.
**AC-4: CI/CD configured**
Given the pipeline stages in the planning artifacts
When CI runs on a PR
Then format/lint, unit tests, replay black-box smoke, cache/security fixture tests, and artifact collection stages are defined.
**AC-5: Test harness skeleton ready**
Given black-box tests must use public interfaces only
When the scaffold is complete
Then unit, integration, black-box, fixtures, and SITL test locations exist with runner entry points ready for implementation tasks.
**AC-6: Deployment evidence paths ready**
Given release requires Jetson, SITL, FDR, cache, and representative replay evidence
When the scaffold is complete
Then deployment scripts, report output paths, and documentation placeholders exist for those evidence artifacts.
**AC-7: No production secrets or raw frames committed**
Given onboard runtime must not retain raw frames and must protect signing credentials
When the scaffold is reviewed
Then `.gitignore`, `.dockerignore`, and `.env.example` exclude secrets, generated FDR payloads, raw frame dumps, cache payloads, and test result artifacts unless explicitly fixture-scoped.
_docs/02_tasks/done/AZ-220_shared_runtime_contracts.md
+78
View File
@@ -0,0 +1,78 @@
# Shared Runtime Contracts
**Task**: AZ-220_shared_runtime_contracts
**Name**: Shared Runtime Contracts
**Description**: Define the shared DTO and event contract surface used across the onboard runtime.
**Complexity**: 3 points
**Dependencies**: AZ-219_initial_structure
**Component**: Bootstrap / Shared Contracts
**Tracker**: AZ-220
**Epic**: AZ-206
## Problem
Runtime components need common shapes for frames, telemetry, VIO state, position estimates, candidates, anchors, cache tiles, and FDR events.
## Outcome
- Components consume one documented shared contract surface.
- Cross-component DTO drift is prevented before implementation begins.
## Scope
### Included
- Shared runtime DTO names, fields, invariants, versioning rules, and contract tests.
- Public contract document for consumers.
### Excluded
- Component-specific algorithms.
- E2E runner-only report schemas.
## Acceptance Criteria
**AC-1: Shared contract exists**
Given runtime components need common DTOs
When the shared contracts are implemented
Then each runtime component can import or reference the same contract surface.
**AC-2: Contract validation exists**
Given a malformed shared DTO
When validation runs
Then the input is rejected with a structured error.
## Non-Functional Requirements
**Compatibility**
- Contract changes follow documented versioning rules.
**Reliability**
- Invalid or missing required fields are not silently ignored.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | DTO construction for valid minimal values | Accepted |
| AC-2 | Missing required timestamp or ID | Rejected with structured error |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Runtime started with contract consumers | Public interfaces exchange typed records | No component-specific duplicate shape required | Compatibility |
## Constraints
- Contracts are shared API surface and must be consumed through documented public modules.
- Raw frame payloads are referenced, not retained in shared DTOs.
## Risks & Mitigation
**Risk 1: Contract drift**
- *Risk*: Consumers create local incompatible DTOs.
- *Mitigation*: Contract file is required dependency for consuming tasks.
## Contract
This task produces/implements the contract at `_docs/02_document/contracts/shared/runtime_contracts.md`.
Consumers MUST read that file — not this task spec — to discover the interface.
_docs/02_tasks/done/AZ-221_shared_geometry_time_sync.md
+78
View File
@@ -0,0 +1,78 @@
# Shared Geometry And Time Sync
**Task**: AZ-221_shared_geometry_time_sync
**Name**: Shared Geometry And Time Sync
**Description**: Provide shared geospatial and timestamp helper behavior for runtime components.
**Complexity**: 3 points
**Dependencies**: AZ-219_initial_structure, AZ-220_shared_runtime_contracts
**Component**: Shared Geometry And Time Sync
**Tracker**: AZ-221
**Epic**: AZ-207
## Problem
Camera, VIO, Tile Manager, Satellite Service, Anchor Verification, safety, and FDR need consistent coordinate, footprint, homography, and timestamp behavior.
## Outcome
- Shared helpers provide deterministic geometry and time-sync behavior.
- Components do not duplicate geospatial or timestamp alignment logic.
## Scope
### Included
- WGS84/local conversions, distance/GSD helpers, footprint projection, homography/covariance conversion support.
- Monotonic timestamp checks, frame-to-IMU window selection, replay ordering, gap/jitter metrics.
### Excluded
- VIO estimation.
- Cache policy and safety degrade decisions.
## Acceptance Criteria
**AC-1: Geometry helpers are deterministic**
Given the same calibration, attitude, altitude, and coordinates
When geometry helpers run
Then they return repeatable footprint and metric outputs.
**AC-2: Time-sync violations are explicit**
Given frame and telemetry timestamps with a gap or mismatch
When alignment runs
Then the result reports the violation instead of dropping data silently.
## Non-Functional Requirements
**Performance**
- Helpers are suitable for hot-path use under the system latency budget.
**Reliability**
- Invalid calibration, CRS, or timestamp inputs fail explicitly.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | WGS84/local round-trip | Error within tolerance |
| AC-2 | Non-monotonic timestamps | Explicit mismatch result |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Derkachi/replay frame metadata | Footprint and ordering evidence | Deterministic helper outputs in reports | Performance |
## Constraints
- Shared helpers do not own policy decisions.
- Timestamp units must match the shared runtime contract.
## Risks & Mitigation
**Risk 1: Coordinate-frame confusion**
- *Risk*: Components interpret helper outputs differently.
- *Mitigation*: Contract documents units, frames, and error semantics.
## Contract
This task produces/implements the contract at `_docs/02_document/contracts/shared/geometry_time_sync.md`.
Consumers MUST read that file — not this task spec — to discover the interface.
_docs/02_tasks/done/AZ-222_runtime_config_errors_telemetry.md
+80
View File
@@ -0,0 +1,80 @@
# Runtime Config Errors And Telemetry
**Task**: AZ-222_runtime_config_errors_telemetry
**Name**: Runtime Config Errors And Telemetry
**Description**: Provide shared configuration, error envelope, health, and telemetry behavior for runtime components.
**Complexity**: 3 points
**Dependencies**: AZ-219_initial_structure, AZ-220_shared_runtime_contracts
**Component**: Runtime Configuration And Errors
**Tracker**: AZ-222
**Epic**: AZ-208
## Problem
Runtime components need common configuration loading, readiness validation, error handling, and telemetry metadata.
## Outcome
- Environment profiles are validated before runtime use.
- Errors and health events are structured and FDR-safe.
- Secrets are referenced without leaking into logs or reports.
## Scope
### Included
- Runtime profile validation.
- Shared error/result envelopes.
- Health and metrics metadata.
### Excluded
- Component-specific business logic.
- FDR storage implementation.
## Acceptance Criteria
**AC-1: Required settings are validated**
Given a production profile is missing a required setting
When startup/readiness validation runs
Then the component reports a structured failure.
**AC-2: Errors are not silent**
Given a dependency failure occurs
When a component reports it
Then the error includes component, category, severity, and retryability.
## Non-Functional Requirements
**Reliability**
- Missing configuration and dependency failures are never silently ignored.
**Compatibility**
- Error categories and config keys follow the contract versioning rules.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Missing production cache dir | Readiness failure |
| AC-2 | Dependency error envelope | Structured fields present |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Invalid env profile | Runtime readiness | Service refuses ready state | Reliability |
## Constraints
- Secrets must not be serialized into logs, FDR, or metrics.
- Error handling must not use silent suppression.
## Risks & Mitigation
**Risk 1: Environment-specific behavior drifts**
- *Risk*: Development and production profiles behave differently in unsafe ways.
- *Mitigation*: Profile validation is shared and tested.
## Contract
This task produces/implements the contract at `_docs/02_document/contracts/shared/config_errors_telemetry.md`.
Consumers MUST read that file — not this task spec — to discover the interface.
_docs/02_tasks/done/AZ-223_camera_ingest_calibration.md
+89
View File
@@ -0,0 +1,89 @@
# Camera Ingest Calibration And Frame Quality
**Task**: AZ-223_camera_ingest_calibration
**Name**: Camera Ingest Calibration And Frame Quality
**Description**: Ingest navigation frames, attach calibration/timestamp metadata, classify quality, detect occlusion, and provide north-up normalization hints.
**Complexity**: 5 points
**Dependencies**: AZ-220_shared_runtime_contracts, AZ-221_shared_geometry_time_sync, AZ-222_runtime_config_errors_telemetry
**Component**: Camera Ingest And Calibration
**Tracker**: AZ-223
**Epic**: AZ-209
## Problem
Downstream VIO, retrieval, anchor verification, Tile Manager, and FDR need trustworthy frame metadata and quality decisions before using image data.
## Outcome
- Replay and live-source frames are exposed with timestamps, calibration ID, quality, occlusion, and normalization hints.
- Total occlusion/blackout frames are marked unusable for VIO and anchor paths.
## Scope
### Included
- Frame source abstraction for replay/live camera boundary.
- Calibration metadata loading and validation.
- Quality and occlusion reports.
- North-up/orthorectification hints as metadata, not unconditional frame mutation.
### Excluded
- Tile writing and generated COG persistence.
- BASALT processing.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Usable frame packet emitted**
Given a valid replay frame and calibration
When the frame is ingested
Then a frame packet includes timestamp, calibration ID, quality report, occlusion report, and normalization hint.
**AC-2: Blackout bypass signal emitted**
Given an unreadable, covered, or total-occlusion frame
When quality classification runs
Then the frame is marked unusable for VIO and anchor matching.
**AC-3: Raw frames are not retained**
Given normal runtime operation
When a frame is processed
Then only allowed metadata/references are retained outside explicit fixture/test paths.
## Non-Functional Requirements
**Performance**
- Frame preprocessing must fit within the system p95 latency budget.
**Reliability**
- Missing calibration blocks production readiness.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid frame with calibration | Frame packet contains required metadata |
| AC-2 | Total occlusion input | `usable_for_vio=false` and `usable_for_anchor=false` |
| AC-3 | Processed frame cleanup | No raw-frame persistence |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Derkachi replay fixture | Frame/telemetry alignment | Accepted frame packets are timestamped | Performance |
| AC-2 | Blackout fixture | Occlusion handling | Visual path degrades safely | Reliability |
## Constraints
- Do not rotate every frame in ingest; emit normalization hints for downstream use.
- ADTi hardware assumptions remain blocked until exact specs are verified.
## Risks & Mitigation
**Risk 1: False-negative occlusion**
- *Risk*: Bad frames enter VIO or matching.
- *Mitigation*: Conservative occlusion gate and explicit quality flags.
_docs/02_tasks/done/AZ-224_mavlink_gcs_gateway.md
+84
View File
@@ -0,0 +1,84 @@
# MAVLink GCS Gateway
**Task**: AZ-224_mavlink_gcs_gateway
**Name**: MAVLink GCS Gateway
**Description**: Subscribe to FC telemetry, emit `GPS_INPUT`, and publish QGC-visible status/failsafe messages.
**Complexity**: 3 points
**Dependencies**: AZ-220_shared_runtime_contracts, AZ-222_runtime_config_errors_telemetry
**Component**: MAVLink And GCS Integration
**Tracker**: AZ-224
**Epic**: AZ-210
## Problem
The runtime needs a strict protocol boundary for ArduPilot Plane telemetry in and GPS-denied estimates/status out.
## Outcome
- FC telemetry is normalized into shared samples.
- `GPS_INPUT` is emitted only from validated position estimates.
- QGC status is rate-limited and safety-relevant.
## Scope
### Included
- Telemetry subscription behavior.
- `GPS_INPUT` field validation and emission result.
- QGC status/failsafe message emission.
### Excluded
- Safety policy and covariance calculation.
- SITL test runner implementation.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Telemetry sample emitted**
Given a valid FC telemetry stream
When the gateway subscribes
Then normalized telemetry samples are available to consumers.
**AC-2: Invalid GPS_INPUT is rejected**
Given a malformed or unsafe position estimate
When emission is requested
Then no invalid `GPS_INPUT` packet is emitted.
**AC-3: Operator status is rate-limited**
Given repeated mode changes or warnings
When status is emitted
Then QGC-visible messages stay within the configured rate.
## Non-Functional Requirements
**Reliability**
- Connection loss is surfaced to wrapper/FDR.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Telemetry normalization | Shared sample emitted |
| AC-2 | Invalid fix type/accuracy | Emission rejected |
| AC-3 | Status burst | Rate limit enforced |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-2 | Plane SITL | `GPS_INPUT` output | Fields match ArduPilot expectations | Reliability |
## Constraints
- v1 emits `GPS_INPUT` only.
- Do not hide MAVLink disconnects or invalid output errors.
## Risks & Mitigation
**Risk 1: ArduPilot parameter mismatch**
- *Risk*: Plane ignores or mishandles emitted estimates.
- *Mitigation*: SITL validation remains a release gate.
_docs/02_tasks/done/AZ-225_tile_manager_cache_manifest.md
+89
View File
@@ -0,0 +1,89 @@
# Tile Manager Cache Manifest
**Task**: AZ-225_tile_manager_cache_manifest
**Name**: Tile Manager Cache Manifest
**Description**: Validate local cache manifests, signed sidecars, tile freshness, and spatial/descriptor metadata access.
**Complexity**: 5 points
**Dependencies**: AZ-220_shared_runtime_contracts, AZ-221_shared_geometry_time_sync, AZ-222_runtime_config_errors_telemetry
**Component**: Tile Manager
**Tracker**: AZ-225
**Epic**: AZ-211
## Problem
The runtime must trust only preloaded offline cache tiles with valid signatures, hashes, freshness, resolution, and spatial metadata.
## Outcome
- Mission cache validation blocks invalid cache usage.
- Tile windows and descriptor metadata are queryable through local PostGIS-backed metadata.
- Stale or tampered tiles cannot become trusted anchor inputs.
## Scope
### Included
- Cache package validation.
- Signed sidecar/hash/freshness checks.
- Tile window and descriptor metadata lookup.
- Cache validation report events.
### Excluded
- Satellite Service package transfer.
- Generated tile orthorectification/write-back.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Valid cache activates**
Given a cache package with valid manifests, sidecars, hashes, freshness, and spatial coverage
When cache validation runs
Then the cache becomes available to retrieval and anchor paths.
**AC-2: Invalid cache is rejected**
Given a stale, unsigned, or hash-mismatched tile
When validation runs
Then the tile is rejected and an auditable reason is recorded.
**AC-3: Tile metadata is queryable**
Given a candidate footprint or chunk ID
When a consumer requests tile metadata
Then the Tile Manager returns a valid local record or explicit rejection.
## Non-Functional Requirements
**Performance**
- Indexed tile lookups meet relocalization-path latency needs.
**Reliability**
- Local PostgreSQL/PostGIS unavailability blocks mission cache activation.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid fixture | Cache available |
| AC-2 | Tampered sidecar | Tile rejected |
| AC-3 | Chunk metadata lookup | Correct record or explicit error |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-2 | Stale/unsigned fixtures | Cache security gate | No invalid tile reaches retrieval | Reliability |
## Constraints
- Runtime never fetches satellite data mid-flight.
- Large imagery/descriptors stay as files, not DB blobs.
## Risks & Mitigation
**Risk 1: Cache poisoning**
- *Risk*: Bad cache entries influence anchor decisions.
- *Mitigation*: Signature, hash, freshness, and provenance gates are mandatory.
_docs/02_tasks/done/AZ-226_generated_tile_orthorectification.md
+88
View File
@@ -0,0 +1,88 @@
# Generated Tile Orthorectification And Sync Package
**Task**: AZ-226_generated_tile_orthorectification
**Name**: Generated Tile Orthorectification And Sync Package
**Description**: Convert eligible nadir frames into generated orthorectified COG tiles and prepare post-flight sync packages.
**Complexity**: 5 points
**Dependencies**: AZ-223_camera_ingest_calibration, AZ-225_tile_manager_cache_manifest
**Component**: Tile Manager
**Tracker**: AZ-226
**Epic**: AZ-211
## Problem
Generated tiles must be written only when pose, frame quality, and provenance gates make them safe to retain for post-flight Satellite Service review.
## Outcome
- Eligible nadir frames can become generated COG candidates with sidecars.
- Unsafe or over-confident tile writes are rejected.
- Post-flight generated-tile packages carry enough metadata for Satellite Service ingest/voting.
## Scope
### Included
- Orthorectification request handling.
- Generated COG + sidecar metadata creation.
- Covariance/quality gates and trust level assignment.
- Sync package preparation.
### Excluded
- Satellite Service upload transport.
- Promotion to trusted basemap onboard.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
## Acceptance Criteria
**AC-1: Eligible frame writes generated tile**
Given a high-confidence pose and usable frame
When tile generation runs
Then a generated COG candidate and sidecar are staged.
**AC-2: Unsafe frame is rejected**
Given high covariance or unusable frame quality
When tile generation runs
Then no trusted tile is written.
**AC-3: Sync package is auditable**
Given generated candidate tiles exist
When a package is prepared
Then it includes manifest delta, sidecars, parent covariance, and trust level.
## Non-Functional Requirements
**Reliability**
- Generated tiles are never promoted directly to trusted basemap onboard.
**Performance**
- Tile generation must not block localization output.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid tile request | COG + sidecar staged |
| AC-2 | Covariance too high | Write rejected |
| AC-3 | Package creation | Required metadata present |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-2 | Poisoning fixture | Generated tile gate | No direct trusted promotion | Reliability |
## Constraints
- Tile Manager performs generated tile writes; camera ingest only provides metadata/hints.
- Missing sidecars make generated tiles ineligible for upload.
## Risks & Mitigation
**Risk 1: Misaligned generated tile**
- *Risk*: Wrong pose creates harmful future anchor data.
- *Mitigation*: Parent covariance, frame quality, and post-flight validation gates.
_docs/02_tasks/done/AZ-227_fdr_event_recorder.md
+88
View File
@@ -0,0 +1,88 @@
# FDR Event Recorder And Export Surface
**Task**: AZ-227_fdr_event_recorder
**Name**: FDR Event Recorder And Export Surface
**Description**: Record bounded replayable mission evidence and expose exportable post-flight summaries.
**Complexity**: 5 points
**Dependencies**: AZ-220_shared_runtime_contracts, AZ-222_runtime_config_errors_telemetry
**Component**: FDR And Observability
**Tracker**: AZ-227
**Epic**: AZ-212
## Problem
The system needs compact, bounded, queryable evidence for estimates, inputs, health, anchors, tile writes, and safety transitions.
## Outcome
- Runtime components can append structured FDR events.
- Storage rollover and health behavior are explicit.
- Post-flight exports are available for analysis and release evidence.
## Scope
### Included
- Append event behavior.
- PostgreSQL event index and CBOR segment payload contract.
- Rollover and storage-full handling.
- Export request/result behavior.
### Excluded
- Component-specific event payload generation.
- E2E report collation.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Events are appendable**
Given a valid FDR event
When a component appends it
Then metadata is indexed and payload is stored within bounds.
**AC-2: Storage pressure is handled**
Given segment or storage limits are reached
When append continues
Then rollover or critical status behavior is explicit.
**AC-3: Export produces evidence**
Given a completed run
When export is requested
Then queryable evidence and optional analytics artifacts are produced.
## Non-Functional Requirements
**Performance**
- FDR appends must not block hot-path localization.
**Reliability**
- Append failures are surfaced to callers and health state.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid event append | Indexed metadata and payload reference |
| AC-2 | Rollover threshold | Rollover event recorded |
| AC-3 | Export request | Evidence artifact produced |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-2 | 8-hour synthetic load | FDR cap and rollover | <=64 GB or explicit rollover evidence | Performance |
## Constraints
- Raw frames are not retained by default.
- Secrets are not logged in FDR payloads.
## Risks & Mitigation
**Risk 1: FDR affects latency**
- *Risk*: Append pressure slows localization.
- *Mitigation*: Bounded segments and async-friendly append semantics.
_docs/02_tasks/done/AZ-228_vio_adapter.md
+90
View File
@@ -0,0 +1,90 @@
# VIO Adapter
**Task**: AZ-228_vio_adapter
**Name**: VIO Adapter
**Description**: Wrap the selected relative VIO backend as a replaceable component with health and error behavior.
**Complexity**: 5 points
**Dependencies**: AZ-221_shared_geometry_time_sync, AZ-222_runtime_config_errors_telemetry, AZ-223_camera_ingest_calibration, AZ-224_mavlink_gcs_gateway
**Component**: VIO Adapter
**Tracker**: AZ-228
**Epic**: AZ-213
## Problem
The safety wrapper needs relative VIO state from calibrated frames and FC IMU without inheriting backend-specific internals.
## Outcome
- Backend initialization, processing, and health behavior are exposed through a replaceable adapter.
- Tracking loss and timestamp mismatch are explicit.
- The adapter never emits WGS84 coordinates or safety decisions.
## Scope
### Included
- Initialization and runtime health.
- Frame + IMU processing behavior.
- Relative pose/velocity/bias output and quality metadata.
- Native bridge boundary.
### Excluded
- Absolute anchor fusion and covariance authority.
- Satellite matching fallback implementation.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: VIO state emitted**
Given synchronized frame and IMU samples
When processing succeeds
Then a relative VIO state packet with tracking quality is emitted.
**AC-2: Timestamp mismatch is explicit**
Given frame/IMU timestamps are inconsistent
When processing is requested
Then the adapter rejects the packet with a timestamp mismatch error.
**AC-3: Health is observable**
Given initialization or tracking quality changes
When health is requested
Then the adapter reports current VIO readiness and degradation state.
## Non-Functional Requirements
**Performance**
- Adapter processing must be profiled against Jetson latency/memory limits.
**Reliability**
- Backend failures are surfaced, not hidden.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid synchronized packet | VIO state emitted |
| AC-2 | Bad timestamp window | Explicit error |
| AC-3 | Tracking loss | Health reports degraded |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Derkachi/public replay | Relative VIO path | Continuous estimates where data supports it | Performance |
## Constraints
- BASALT remains the current selected backend, but package and folder names must stay backend-neutral.
- The VIO adapter is not the safety authority.
- GPL VIO dependencies remain reference-only unless explicitly approved.
## Risks & Mitigation
**Risk 1: Nadir fixed-wing fit**
- *Risk*: The selected VIO backend underperforms on low-parallax terrain.
- *Mitigation*: Representative replay and reference comparisons gate acceptance.
_docs/02_tasks/done/AZ-229_satellite_service_sync.md
+87
View File
@@ -0,0 +1,87 @@
# Satellite Service Sync Boundary
**Task**: AZ-229_satellite_service_sync
**Name**: Satellite Service Sync Boundary
**Description**: Import mission cache packages before flight and upload generated-tile packages after flight.
**Complexity**: 3 points
**Dependencies**: AZ-222_runtime_config_errors_telemetry, AZ-225_tile_manager_cache_manifest, AZ-226_generated_tile_orthorectification
**Component**: Satellite Service
**Tracker**: AZ-229
**Epic**: AZ-214
## Problem
The onboard runtime needs a clear boundary for Satellite Service package exchange without allowing mid-flight network calls.
## Outcome
- Pre-flight cache packages can be imported and handed to Tile Manager validation.
- Post-flight generated-tile packages can be uploaded/retried.
- Mid-flight provider or Satellite Service calls are explicitly blocked.
## Scope
### Included
- Pre-flight import behavior.
- Post-flight generated-tile upload behavior.
- Package status and retry/error reporting.
### Excluded
- In-flight retrieval ranking.
- Tile manifest validation and generated tile creation.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Pre-flight package import succeeds**
Given a valid Satellite Service mission cache package
When import runs before flight
Then the package is available for Tile Manager validation.
**AC-2: Post-flight upload is auditable**
Given a generated-tile package exists after landing
When upload runs
Then success, rejection, or retryable failure is recorded.
**AC-3: Mid-flight network calls are blocked**
Given the runtime is in flight mode
When cache data is missing
Then the component does not call a satellite provider or suite service.
## Non-Functional Requirements
**Reliability**
- Upload failures retain packages for retry.
**Security**
- Signing credentials are never logged.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid import package | Package ready for validation |
| AC-2 | Upload unavailable | Retryable failure recorded |
| AC-3 | Flight-mode missing data | No network call |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-3 | Network disabled in replay | Relocalization missing data | Degraded/no-candidate behavior, no fetch | Security |
## Constraints
- No in-flight Satellite Service or satellite-provider network dependency.
- Package sync is before takeoff or after landing only.
## Risks & Mitigation
**Risk 1: Boundary confusion**
- *Risk*: Retrieval code starts fetching provider data mid-flight.
- *Mitigation*: Flight-mode invariant is acceptance-tested.
_docs/02_tasks/done/AZ-230_satellite_service_vpr_retrieval.md
+89
View File
@@ -0,0 +1,89 @@
# Satellite Service Local VPR Retrieval
**Task**: AZ-230_satellite_service_vpr_retrieval
**Name**: Satellite Service Local VPR Retrieval
**Description**: Retrieve local VPR candidates from preloaded descriptors and FAISS indexes.
**Complexity**: 5 points
**Dependencies**: AZ-223_camera_ingest_calibration, AZ-225_tile_manager_cache_manifest, AZ-229_satellite_service_sync
**Component**: Satellite Service
**Tracker**: AZ-230
**Epic**: AZ-214
## Problem
Relocalization needs ranked satellite/cache candidates, but retrieval must be trigger-based and use only local preloaded cache/index data.
## Outcome
- DINOv2-VLAD descriptor extraction and CPU FAISS top-K candidate retrieval are available.
- Candidate freshness and dynamic top-K policy are carried forward.
- No-candidate/index failures produce degraded behavior rather than unsafe anchors.
## Scope
### Included
- Index load/readiness behavior.
- Query descriptor and top-K candidate retrieval.
- Freshness tagging and retrieval result metadata.
- Descriptor fidelity gate for optimized engines.
### Excluded
- Local matching/RANSAC.
- In-flight network fetches.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Index loads before retrieval**
Given a valid local descriptor/index package
When index loading runs
Then retrieval readiness is reported.
**AC-2: Top-K candidates returned**
Given a relocalization request and loaded local index
When retrieval runs
Then bounded candidates include tile/chunk IDs, scores, footprints, and freshness status.
**AC-3: Missing candidates degrade safely**
Given no valid candidates or index failure
When retrieval runs
Then the result is explicit no-candidate/degraded behavior.
## Non-Functional Requirements
**Performance**
- Retrieval is trigger-based and profiled against Jetson limits.
**Security**
- Retrieval never performs mid-flight provider calls.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid index | Ready status |
| AC-2 | Query frame | Candidate list with freshness |
| AC-3 | Missing index | Explicit degraded/no-candidate result |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-2 | Public/cache fixture | VPR recall and top-K policy | Correct candidate bounds and freshness | Performance |
## Constraints
- Retrieval is not per-frame steady-state work.
- TensorRT/ONNX path is accepted only after descriptor-fidelity tests pass.
## Risks & Mitigation
**Risk 1: Trigger path exceeds Jetson budget**
- *Risk*: Descriptor extraction or FAISS query is too slow.
- *Mitigation*: CPU-first FAISS, bounded top-K, and profiling gates.
_docs/02_tasks/done/AZ-231_anchor_verification_matching.md
+88
View File
@@ -0,0 +1,88 @@
# Anchor Verification Matching And Geometry Gates
**Task**: AZ-231_anchor_verification_matching
**Name**: Anchor Verification Matching And Geometry Gates
**Description**: Verify retrieved candidates with ALIKED/DISK + LightGlue and geometric safety gates.
**Complexity**: 5 points
**Dependencies**: AZ-223_camera_ingest_calibration, AZ-225_tile_manager_cache_manifest, AZ-230_satellite_service_vpr_retrieval
**Component**: Anchor Verification
**Tracker**: AZ-231
**Epic**: AZ-215
## Problem
VPR candidates are not trusted fixes; they require local feature matching, RANSAC geometry, provenance checks, and measurable rejection evidence.
## Outcome
- Anchor candidates are accepted or rejected with MRE, inlier count, homography, freshness/provenance, and reason metadata.
- Matcher profiles can be benchmarked without making learned matching a per-frame VIO hot path.
## Scope
### Included
- ALIKED/DISK + LightGlue matching profile behavior.
- SIFT/ORB baseline profile.
- OpenCV RANSAC/USAC geometry checks.
- Anchor decision output and rejection reasons.
### Excluded
- VPR retrieval ranking.
- Safety wrapper anchor fusion.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Candidate verification emits evidence**
Given retrieved candidates and a usable frame
When verification runs
Then each result includes acceptance state, MRE, inlier count, and rejection/acceptance reason.
**AC-2: Unsafe candidates are rejected**
Given low inliers, high MRE, stale provenance, or geometry failure
When verification runs
Then no accepted anchor decision is emitted for that candidate.
**AC-3: Matcher benchmark is reportable**
Given configured matcher profiles
When benchmark runs
Then profile runtime and quality metrics are reported.
## Non-Functional Requirements
**Performance**
- Learned matching is trigger-based and profiled separately from BASALT.
**Reliability**
- SuperPoint is excluded from production unless legal approval exists.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Valid matching fixture | Evidence fields present |
| AC-2 | Bad geometry fixture | Rejected decision |
| AC-3 | Benchmark profiles | Metrics emitted |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Aerial/cache fixture | Anchor verification path | Accepted anchors meet MRE/inlier gates | Performance |
## Constraints
- ALIKED/DISK + LightGlue supplies correspondences, not full VIO.
- Anchor verification does not decide final safety fusion.
## Risks & Mitigation
**Risk 1: False anchor acceptance**
- *Risk*: Cross-domain match looks plausible but is wrong.
- *Mitigation*: Geometry, freshness, provenance, and downstream Mahalanobis gates.
_docs/02_tasks/done/AZ-232_safety_anchor_state_machine.md
+97
View File
@@ -0,0 +1,97 @@
# Safety Anchor State Machine
**Task**: AZ-232_safety_anchor_state_machine
**Name**: Safety Anchor State Machine
**Description**: Own authoritative localization state, confidence, anchor fusion, degraded modes, tile-write eligibility, and MAVLink output semantics.
**Complexity**: 5 points
**Dependencies**: AZ-223_camera_ingest_calibration, AZ-224_mavlink_gcs_gateway, AZ-227_fdr_event_recorder, AZ-228_vio_adapter, AZ-231_anchor_verification_matching
**Component**: Safety And Anchor Wrapper
**Tracker**: AZ-232
**Epic**: AZ-216
## Problem
The product needs one safety authority that converts VIO, telemetry, camera quality, and anchor evidence into honest localization outputs and degraded/failsafe behavior.
## Outcome
- Wrapper updates localization state from VIO and anchors.
- Covariance grows honestly in degraded modes.
- `GPS_INPUT` semantics, source labels, and tile-write eligibility are controlled by the wrapper.
## Scope
### Included
- VIO update handling.
- Anchor acceptance/rejection integration.
- IMU-only degraded propagation.
- Covariance/source-label/fix-type behavior.
- Tile-write eligibility decisions.
### Excluded
- VIO backend internals.
- MAVLink transport implementation.
- Generated tile writing.
## Dependencies
### Document Dependencies
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: VIO state updates position estimate**
Given valid VIO state and telemetry context
When the wrapper processes the update
Then it emits an estimate with source label and honest covariance.
**AC-2: Accepted anchor can correct state**
Given an anchor decision that passes safety gates
When the wrapper considers it
Then the state may become `satellite_anchored` with recorded evidence.
**AC-3: Blackout degrades safely**
Given total visual blackout or tracking loss
When degraded propagation runs
Then covariance grows monotonically and failsafe/no-fix semantics occur at thresholds.
**AC-4: Tile-write eligibility is conservative**
Given a frame and current estimate
When tile eligibility is requested
Then only sufficiently trusted poses are eligible for generated tile writes.
## Non-Functional Requirements
**Safety**
- Confidence must not be optimistic relative to known error/covariance.
**Reliability**
- State invariant violations are surfaced and recorded.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Healthy VIO update | Estimate emitted |
| AC-2 | Accepted/rejected anchors | State updates only on accepted evidence |
| AC-3 | Blackout thresholds | `dead_reckoned` then no-fix/failsafe |
| AC-4 | Tile eligibility | Covariance/quality gates enforced |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-3 | Blackout + spoofing SITL | Degraded behavior | QGC/FDR/fix fields match thresholds | Safety |
## Constraints
- The VIO backend is not the safety authority.
- The wrapper does not call Tile Manager directly during anchor acceptance; freshness/provenance arrives through anchor evidence.
## Risks & Mitigation
**Risk 1: Over-trusted dead reckoning**
- *Risk*: IMU-only propagation is believed for too long.
- *Mitigation*: Monotonic covariance growth and no-fix/failsafe thresholds.
_docs/02_tasks/todo/AZ-233_test_infrastructure.md
+163
View File
@@ -0,0 +1,163 @@
# Test Infrastructure
**Task**: AZ-233_test_infrastructure
**Name**: Test Infrastructure
**Description**: Scaffold the blackbox and e2e test project: runner, deterministic fixtures, isolated replay/SITL environment, reporting, and external dependency stubs.
**Complexity**: 5 points
**Dependencies**: AZ-240_native_vio_backend_integration, AZ-241_real_satellite_vpr_descriptor_retrieval, AZ-242_real_anchor_feature_matching_ransac
**Component**: Blackbox Tests
**Tracker**: AZ-233
**Epic**: AZ-218
## Test Project Folder Layout
```text
e2e/
├── replay/
│ ├── run_replay.py
│ ├── scenarios/
│ └── reports/
├── fixtures/
│ ├── cache/
│ ├── mavlink/
│ ├── telemetry/
│ └── expected/
├── tests/
│ ├── test_still_image_replay.py
│ ├── test_vio_replay.py
│ ├── test_satellite_anchor.py
│ ├── test_blackout_spoofing.py
│ ├── test_resource_limits.py
│ └── test_security_gates.py
├── mocks/
│ ├── satellite_cache_stub/
│ ├── ardupilot_sitl/
│ └── qgc_observer/
└── reports/
```
### Layout Rationale
The test project keeps blackbox/e2e runner code outside product runtime internals. Scenario definitions, fixtures, mocks, and reports are separated so tests can reset state between runs and produce release evidence without importing private component modules.
Test implementation starts only after remediation tasks AZ-240, AZ-241, and AZ-242 close the native VIO, real satellite VPR, and real anchor matching gaps found during autodev verification.
## Mock Services
| Mock Service | Replaces | Interfaces | Behavior |
|-------------|----------|------------|----------|
| `satellite_cache_stub` | Offline Azaion Suite Satellite Service cache package | Local COG/manifest/descriptor fixture volume | Serves preloaded valid, stale, unsigned, hash-mismatched, and low-resolution cache fixtures; never performs network fetches during flight-mode tests. |
| `ardupilot_sitl` | ArduPilot Plane flight controller | MAVLink telemetry and `GPS_INPUT` receiving path | Emits generated IMU, attitude, GPS health, spoofing, and failsafe traces; records injected `GPS_INPUT` for assertions. |
| `qgc_observer` | QGroundControl status consumer | MAVLink/tlog parser | Records downsampled `STATUSTEXT`, status, and failsafe messages for rate and content assertions. |
### Mock Control API
Each mock or runner fixture must expose deterministic scenario controls for normal replay, stale cache, missing cache, spoofed GPS, blackout, restart, and resource-load modes. Recorded interactions must be queryable after each test run for assertions.
## Docker Test Environment
### `docker-compose.test.yml` Structure
| Service | Image / Build | Purpose | Depends On |
|---------|---------------|---------|------------|
| `gps-denied-service` | Project runtime image or local package mount | System under test | `satellite-cache-stub` |
| `replay-consumer` | Python replay/test harness | Feeds frames, telemetry, cache data, and faults | `gps-denied-service`, mock services |
| `satellite-cache-stub` | Fixture volume/service | Provides offline cache manifests, sidecars, descriptors, and generated invalid variants | none |
| `ardupilot-plane-sitl` | SITL container or local process wrapper | Validates `GPS_INPUT`, spoofing, and failsafe behavior | `gps-denied-service` |
| `qgc-observer` | MAVLink log parser | Verifies GCS-visible status output | `ardupilot-plane-sitl` |
### Networks and Volumes
- `replay-net`: connects the runtime, replay consumer, and satellite-cache stub.
- `sitl-net`: connects the runtime, ArduPilot Plane SITL, and QGC observer.
- `input-data`: read-only mount for `_docs/00_problem/input_data/`.
- `expected-results`: read-only mount for expected coordinate and report fixtures.
- `derkachi-replay`: read-only mount for `flight_derkachi.mp4` and `data_imu.csv`.
- `satellite-cache`: fixture cache volume with valid and invalid manifests.
- `fdr-output`: fresh per-run output volume for FDR and report artifacts.
## Test Runner Configuration
**Framework**: Python pytest-style replay harness.
**Entry point**: `run-blackbox-replay` or equivalent pytest command that executes scenario groups and writes reports.
**Reports**: CSV summary plus FDR validation Markdown.
### Fixture Strategy
| Fixture | Scope | Purpose |
|---------|-------|---------|
| `project_60_still_images` | session | Provides 60 nadir images and expected WGS84 centers. |
| `derkachi_video_telemetry` | session | Provides synchronized video, IMU, and `GLOBAL_POSITION_INT` replay data. |
| `cache_integrity_fixtures` | function | Provides valid, stale, unsigned, hash-mismatched, and low-resolution cache variants. |
| `sitl_spoofing_scenarios` | function | Provides generated GPS loss/spoofing and blackout traces. |
| `public_nadir_vio_candidates` | optional/session | Provides public or representative synchronized datasets when available. |
## Test Data Fixtures
| Data Set | Source | Format | Used By |
|----------|--------|--------|---------|
| `project_60_still_images` | `_docs/00_problem/input_data/` | JPG + metadata | Still-image accuracy, confidence, latency smoke |
| `expected_frame_centers` | `_docs/00_problem/input_data/coordinates.csv` and expected-results report | CSV/Markdown | Geolocation assertions |
| `derkachi_video_telemetry` | `_docs/00_problem/input_data/flight_derkachi/` | MP4 + CSV | VIO replay, latency, resilience |
| `cache_integrity_fixtures` | generated fixture volume | COG/manifest/sidecar/index fixtures | Cache freshness, poisoning, no-fetch tests |
| `sitl_spoofing_scenarios` | generated by SITL harness | MAVLink/tlog traces | Spoofing, blackout, failsafe, GCS status |
| `public_nadir_vio_candidates` | pinned external fixtures | dataset-specific | Final VIO and satellite-anchor validation |
### Data Isolation
Every run uses read-only input fixtures and fresh run-scoped output directories. FDR, generated tiles, tlogs, and reports are written only to per-run output volumes. Mock state and generated fixtures are reset before each scenario group.
## Test Reporting
**Format**: CSV summary and Markdown evidence report.
**Output paths**: `test-results/blackbox-report.csv` and `test-results/fdr-validation-summary.md`.
**Required columns**: Test ID, test name, input dataset, execution time, result, error distance, source label, covariance 95% semi-major, `GPS_INPUT.fix_type`, and error message.
## Acceptance Criteria
**AC-1: Test environment starts**
Given the Docker/replay test environment
When the test stack starts
Then the runtime, replay consumer, cache fixture, SITL, and observer services are reachable or report a clear blocked prerequisite.
**AC-2: External dependency stubs are deterministic**
Given a scenario config for cache, MAVLink, QGC, or fixture behavior
When the replay consumer executes it
Then mocks produce repeatable responses and expose recorded interactions for assertions.
**AC-3: Test runner executes scenario groups**
Given valid fixtures and a running test environment
When the test runner starts
Then it discovers and executes blackbox, performance, resilience, security, and resource-limit scenario groups.
**AC-4: Reports are generated**
Given a completed or blocked test run
When reporting finishes
Then CSV and Markdown evidence files are written with the required columns, metrics, artifact paths, and blocked-prerequisite reasons.
## Non-Functional Requirements
**Reliability**
- Missing hardware, public datasets, calibration, or SITL prerequisites are reported as `blocked`, not `passed`.
**Security**
- Fixture stubs must not access external satellite-provider or Suite service networks during in-flight test scenarios.
**Data Isolation**
- No test may mutate source fixtures or write FDR/generated-tile artifacts outside run-scoped output paths.
## Constraints
- The test suite must use public runtime boundaries only: navigation frames, telemetry, offline cache, MAVLink output, QGC status, and FDR outputs.
- The suite must not import private estimator, BASALT, wrapper, or tile-manager internals.
- Hardware-specific Jetson gates remain release-gate tests and may be skipped or blocked in ordinary local replay.
## Risks & Mitigation
**Risk 1: Environment prerequisites hide real failures**
- *Risk*: Missing hardware, calibration, or datasets could be treated as success.
- *Mitigation*: Report unavailable prerequisites as `blocked` with explicit artifact evidence.
**Risk 2: Fixture mutation contaminates later runs**
- *Risk*: Generated FDR, cache, or SITL output changes expected input fixtures.
- *Mitigation*: Use read-only fixture mounts and fresh run-scoped output volumes for every execution.
_docs/02_tasks/todo/AZ-234_replay_geolocation_confidence_tests.md
+88
View File
@@ -0,0 +1,88 @@
# Replay Geolocation And Confidence Tests
**Task**: AZ-234_replay_geolocation_confidence_tests
**Name**: Replay Geolocation And Confidence Tests
**Description**: Implement blackbox tests for still-image geolocation, confidence/source-label output, and replay latency smoke.
**Complexity**: 3 points
**Dependencies**: AZ-233_test_infrastructure
**Component**: Blackbox Tests
**Tracker**: AZ-234
**Epic**: AZ-218
## Problem
The project needs deterministic blackbox evidence that the 60-image replay path emits WGS84 frame-center estimates with required confidence fields and latency metrics.
## Outcome
- Still-image replay reports per-frame coordinate error and aggregate threshold results.
- Every emitted estimate includes covariance, source label, and anchor-age fields.
- Replay smoke latency and dropped-frame metrics are captured in the shared report format.
## Scope
### Included
- FT-P-01 Still-Image Frame Center Geolocation.
- FT-P-02 Position Confidence Output Contract.
- NFT-PERF-01 Per-Frame Latency On Project Still Images.
- CSV and Markdown evidence output for these scenarios.
### Excluded
- Synchronized VIO video/IMU replay.
- Satellite-anchor VPR/local matching.
- Jetson-only release-gate profiling.
## Acceptance Criteria
**AC-1: Still-image coordinates are validated**
Given the 60-image project fixture and expected frame-center coordinates
When the replay test runs
Then per-frame WGS84 error is reported and aggregate 50 m / 20 m thresholds are evaluated.
**AC-2: Confidence output contract is validated**
Given emitted position estimates from the replay
When the test inspects public output fields
Then each estimate includes WGS84 coordinates, 95% covariance semi-major axis, source label, and anchor age.
**AC-3: Replay latency is measured**
Given the still-image replay runs at the configured smoke rate
When processing completes
Then capture-to-output latency and dropped-frame rate are recorded with pass/fail or blocked status.
## Non-Functional Requirements
**Performance**
- Replay smoke evidence includes p50/p95/p99 latency and dropped-frame rate.
**Reliability**
- Missing or invalid expected-coordinate fixtures fail fixture validation before scenario execution.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Expected-coordinate loader validation | Invalid coordinates are rejected before replay |
| AC-2 | Report field validation | Missing confidence/source fields fail the scenario |
| AC-3 | Latency metric aggregation | p50/p95/p99 and dropped-frame metrics are emitted |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | `project_60_still_images`, `expected_frame_centers` | FT-P-01 | >=80% within 50 m and >=50% within 20 m or explicit failure | Reliability |
| AC-2 | Same replay output | FT-P-02 | 100% of emitted estimates include required confidence fields | Reliability |
| AC-3 | Replay smoke run | NFT-PERF-01 | Latency and drop-rate metrics are recorded | Performance |
## Constraints
- Tests must use public replay input and output artifacts only.
- Input fixtures must be mounted read-only.
- Blocked prerequisites must be reported as `blocked`, not `passed`.
## Risks & Mitigation
**Risk 1: Calibration limits are mistaken for product failure**
- *Risk*: Fixture limits can make absolute accuracy inconclusive.
- *Mitigation*: Report the fixture source and threshold basis with each failure.
_docs/02_tasks/todo/AZ-235_vio_replay_performance_tests.md
+89
View File
@@ -0,0 +1,89 @@
# VIO Replay Performance Tests
**Task**: AZ-235_vio_replay_performance_tests
**Name**: VIO Replay Performance Tests
**Description**: Implement synchronized video/IMU replay tests for VIO output, covariance evidence, and replay performance metrics.
**Complexity**: 5 points
**Dependencies**: AZ-233_test_infrastructure, AZ-240_native_vio_backend_integration
**Component**: Blackbox Tests
**Tracker**: AZ-235
**Epic**: AZ-218
## Problem
The runtime needs blackbox evidence that synchronized navigation video and flight-controller telemetry can drive VIO/wrapper output with honest confidence and measurable performance.
This test task must run after AZ-240 so it validates the real native VIO path rather than the deterministic scaffold.
## Outcome
- Derkachi video/telemetry fixture alignment is validated before replay.
- Synchronized replay produces frame-by-frame output or a clear blocked/failure reason.
- Latency, completion rate, memory, trajectory comparison, and calibration-gated checks are reported.
## Scope
### Included
- FT-P-03 BASALT VIO Replay With Synchronized Video/Telemetry.
- NFT-PERF-02 BASALT + Wrapper Replay Latency.
- Public/representative dataset prerequisite reporting.
### Excluded
- Satellite-anchor local verification.
- SITL spoofing/failsafe scenarios.
- Thermal/endurance release gates.
## Acceptance Criteria
**AC-1: Replay fixture alignment is validated**
Given the Derkachi MP4 and telemetry CSV
When fixture validation runs
Then duration, frame-to-telemetry ratio, and timestamp monotonicity are verified before replay.
**AC-2: Synchronized replay emits estimates**
Given a valid synchronized video/IMU replay fixture
When replay executes
Then estimates are emitted frame-by-frame with source labels, covariance, and segment evidence.
**AC-3: VIO performance evidence is reported**
Given replay completed or blocked
When reporting finishes
Then latency, completion rate, memory, and calibration/public-dataset prerequisite status are written.
## Non-Functional Requirements
**Performance**
- Reports include per-frame latency and memory metrics where the environment can measure them.
**Reliability**
- Calibration-gated absolute accuracy checks must be marked explicitly instead of silently passing.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Video/telemetry validator | Invalid duration or timestamp alignment blocks replay |
| AC-2 | Replay result parser | Missing per-frame confidence fields fail the scenario |
| AC-3 | Calibration gate reporting | Missing calibration/public data is reported as blocked |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | `derkachi_video_telemetry` | FT-P-03 fixture validation | Fixture accepted only when alignment rules pass | Reliability |
| AC-2 | Valid synchronized replay | FT-P-03 output | Continuous estimates for normal overlapping segments or explicit degradation | Reliability |
| AC-3 | Replay performance run | NFT-PERF-02 | Latency, completion rate, and memory evidence are recorded | Performance |
## Constraints
- Tests must not import BASALT/OpenVINS/Kimera internals directly.
- Public/representative datasets are optional prerequisites and may produce blocked results.
- Raw input video and telemetry fixtures remain read-only.
## Risks & Mitigation
**Risk 1: Hardware or dataset prerequisites are unavailable**
- *Risk*: The scenario cannot produce final accuracy evidence locally.
- *Mitigation*: Emit blocked results with exact missing prerequisite and continue other scenario groups.
_docs/02_tasks/todo/AZ-236_satellite_anchor_cache_tests.md
+102
View File
@@ -0,0 +1,102 @@
# Satellite Anchor Cache Tests
**Task**: AZ-236_satellite_anchor_cache_tests
**Name**: Satellite Anchor Cache Tests
**Description**: Implement blackbox, security, and performance tests for satellite-anchor retrieval, local verification, cache integrity, and no in-flight external access.
**Complexity**: 5 points
**Dependencies**: AZ-233_test_infrastructure, AZ-241_real_satellite_vpr_descriptor_retrieval, AZ-242_real_anchor_feature_matching_ransac
**Component**: Blackbox Tests
**Tracker**: AZ-236
**Epic**: AZ-218
## Problem
Satellite anchors and cache fixtures are safety-critical: invalid, stale, poisoned, or externally fetched data must not become trusted localization output.
This test task must run after AZ-241 and AZ-242 so it validates real local VPR retrieval and real anchor feature matching rather than scaffold evidence gates.
## Outcome
- Accepted anchors include retrieval, matching, geometry, freshness, and provenance evidence.
- Invalid/stale/poisoned cache fixtures cannot produce trusted anchors or trusted generated tiles.
- No in-flight Satellite Service or provider access occurs when cache data is missing.
## Scope
### Included
- FT-P-04 Satellite Service And Anchor Verification.
- FT-N-01 Repetitive Or Low-Texture Imagery.
- FT-N-03 Invalid Or Stale Satellite Cache.
- NFT-PERF-03 Relocalization Trigger Path Latency.
- NFT-RES-04 Tile Cache Freshness Degradation.
- NFT-SEC-01 Signed Cache Manifest Enforcement.
- NFT-SEC-02 Cache Poisoning Write Gate.
- NFT-SEC-04 No In-Flight Satellite Provider Access.
- NFT-RES-LIM-03 Satellite Cache Storage Budget.
### Excluded
- VIO synchronized replay.
- MAVLink spoofing/failsafe behavior.
- Jetson thermal endurance.
## Acceptance Criteria
**AC-1: Verified anchors include evidence**
Given a valid local cache/index fixture and relocalization trigger
When retrieval and verification run
Then accepted anchors include candidate IDs, scores, MRE, inliers, covariance, and tile provenance.
**AC-2: Unsafe candidates are rejected**
Given low-texture, stale, unsigned, hash-mismatched, or low-resolution fixtures
When anchor/cache tests run
Then no invalid candidate emits a trusted `satellite_anchored` estimate or trusted generated tile.
**AC-3: No in-flight external access occurs**
Given flight-mode replay with missing cache data
When relocalization is requested
Then the system reports degraded/no-candidate behavior without satellite-provider or Suite service network calls.
**AC-4: Cache and trigger-path metrics are reported**
Given cache and relocalization scenarios complete
When reporting finishes
Then latency, MRE, trust level, freshness, and storage-budget evidence are written.
## Non-Functional Requirements
**Security**
- Invalid cache data must not be trusted or promoted.
**Performance**
- Trigger-path latency and bounded top-K behavior are measured.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Anchor evidence parser | Required evidence fields are present |
| AC-2 | Invalid cache fixture generator | Stale/unsigned/hash-mismatched fixtures are produced deterministically |
| AC-3 | Network-block assertion | Unexpected external calls fail the scenario |
| AC-4 | Cache metrics report | Latency, freshness, and storage metrics are present |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Public/cache fixture | FT-P-04 | Accepted anchors meet MRE/evidence requirements | Performance |
| AC-2 | Ambiguous and invalid cache fixtures | FT-N-01, FT-N-03, NFT-SEC-01, NFT-SEC-02 | 0 unsafe trusted outputs | Security |
| AC-3 | Network-blocked flight-mode replay | NFT-SEC-04 | Missing cache causes degraded behavior, not fetch | Security |
| AC-4 | Relocalization/cache runs | NFT-PERF-03, NFT-RES-04, NFT-RES-LIM-03 | Metrics and storage evidence are recorded | Performance |
## Constraints
- Tests must use local preloaded cache/index fixtures only.
- External network access during flight-mode scenarios is a failure.
- VPAir and UZH FPV licensing must be respected before use as commercial acceptance evidence.
## Risks & Mitigation
**Risk 1: Dataset licensing blocks final anchor evidence**
- *Risk*: Public dataset terms prevent commercial acceptance use.
- *Mitigation*: Mark dataset-specific checks blocked and keep generated cache fixtures for deterministic security coverage.
_docs/02_tasks/todo/AZ-237_mavlink_blackout_spoofing_tests.md
+94
View File
@@ -0,0 +1,94 @@
# MAVLink Blackout Spoofing Tests
**Task**: AZ-237_mavlink_blackout_spoofing_tests
**Name**: MAVLink Blackout Spoofing Tests
**Description**: Implement SITL/replay tests for visual blackout, spoofed GPS, MAVLink source validation, degraded covariance, no-fix thresholds, and QGC status.
**Complexity**: 5 points
**Dependencies**: AZ-233_test_infrastructure
**Component**: Blackbox Tests
**Tracker**: AZ-237
**Epic**: AZ-218
## Problem
The system must prove that spoofed GPS and unauthorized MAVLink messages cannot override estimator state during visual blackout or degraded operation.
## Outcome
- Blackout and spoofing traces drive visible degraded-mode transitions.
- Covariance, `GPS_INPUT`, QGC status, and FDR evidence match the safety thresholds.
- Unauthorized MAVLink sources are rejected and recorded.
## Scope
### Included
- FT-N-02 GPS Spoofing During Total Visual Blackout.
- NFT-RES-01 Total Visual Blackout With GPS Spoofing.
- NFT-SEC-03 MAVLink Source And Spoofing Rejection.
### Excluded
- Still-image geolocation accuracy.
- Satellite-anchor cache poisoning.
- Cold-start and restart trials.
## Acceptance Criteria
**AC-1: Blackout transitions to dead reckoning**
Given a replay/SITL trace with total camera blackout and spoofed GPS
When the scenario runs
Then the system enters `dead_reckoned` mode within the required frame or timing threshold.
**AC-2: Degraded output thresholds are enforced**
Given blackout continues beyond configured thresholds
When estimates are emitted
Then covariance grows monotonically and `GPS_INPUT` fields degrade to no-fix/failsafe values at the specified limits.
**AC-3: Spoofed or unauthorized MAVLink inputs are rejected**
Given spoofed real-GPS measurements or unauthorized MAVLink source IDs
When messages arrive during normal or blackout operation
Then no confident position estimate is produced from those inputs.
**AC-4: Operator and FDR evidence is visible**
Given degraded-mode transitions occur
When reporting completes
Then QGC status and FDR evidence show promotion, demotion, blackout, and failsafe events at expected rates.
## Non-Functional Requirements
**Safety**
- Spoofed GPS must not be promoted during blackout without the documented recovery gates.
**Reliability**
- Missing SITL prerequisites are reported as blocked with exact setup evidence.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Scenario trigger builder | Blackout and spoofing events are generated deterministically |
| AC-2 | Threshold assertion logic | Fix type, covariance, and `horiz_accuracy` thresholds are checked |
| AC-3 | MAVLink source filter assertion | Unauthorized source messages fail the scenario |
| AC-4 | Status/FDR parser | Expected status events and rates are validated |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | SITL or replay spoofing trace | FT-N-02, NFT-RES-01 | Dead-reckoned transition within timing threshold | Safety |
| AC-2 | Continued blackout | FT-N-02, NFT-RES-01 | Monotonic covariance and no-fix/failsafe fields | Safety |
| AC-3 | Unauthorized/spoofed MAVLink messages | NFT-SEC-03 | No confident estimate from bad source | Safety |
| AC-4 | QGC/FDR outputs | FT-N-02, NFT-SEC-03 | Status and evidence are visible and rate-limited | Reliability |
## Constraints
- ArduPilot Plane SITL is the authoritative autopilot target.
- v1 asserts `GPS_INPUT` output and intentional absence of ODOMETRY.
- Tests must not depend on Mission Planner or PX4 behavior.
## Risks & Mitigation
**Risk 1: SITL setup varies by environment**
- *Risk*: Local runs may not have SITL installed or configured.
- *Mitigation*: Report blocked prerequisites clearly and keep replay-level assertions runnable where possible.
_docs/02_tasks/todo/AZ-238_cold_start_restart_tests.md
+95
View File
@@ -0,0 +1,95 @@
# Cold Start Restart Tests
**Task**: AZ-238_cold_start_restart_tests
**Name**: Cold Start Restart Tests
**Description**: Implement tests for cold start, companion restart, sharp-turn/disconnected relocalization, and first-fix resource spikes.
**Complexity**: 5 points
**Dependencies**: AZ-233_test_infrastructure
**Component**: Blackbox Tests
**Tracker**: AZ-238
**Epic**: AZ-218
## Problem
The test suite must prove that the runtime recovers from disconnected visual segments and companion restarts without hiding missing prerequisites or unsafe degraded behavior.
## Outcome
- Sharp-turn/disconnected-segment scenarios trigger relocalization or explicit degraded output.
- Companion restart scenarios measure first valid output timing and FDR evidence.
- Cold-start trials record first-fix latency and resource spikes.
## Scope
### Included
- NFT-RES-02 Sharp Turn And Disconnected Segment Relocalization.
- NFT-RES-03 Companion Computer Restart Mid-Flight.
- NFT-PERF-04 Cold Boot Time To First Fix.
- NFT-RES-LIM-05 Cold Start Resource Spike.
### Excluded
- Long thermal endurance.
- FDR 8-hour rollover load.
- Cache poisoning and no-fetch security tests.
## Acceptance Criteria
**AC-1: Disconnected segments trigger relocalization**
Given a sharp-turn or disconnected segment fixture
When replay reaches the low-overlap transition
Then relocalization is requested and the system either reconnects via verified anchor or reports degraded status.
**AC-2: Companion restart recovery is measured**
Given a replay/SITL mission in progress
When the GPS-denied service is restarted
Then first valid output timing, FC-state handoff behavior, and FDR restart evidence are recorded.
**AC-3: Cold-start trials report first-fix timing**
Given cold-start conditions and local cache/index prerequisites
When 50 trials run or are blocked
Then the p95 time-to-first-fix result or exact blocked prerequisite is reported.
**AC-4: Cold-start resource spikes are captured**
Given initialization begins
When engines/indexes/cache are loaded
Then peak memory and initialization-stage timing are recorded where measurable.
## Non-Functional Requirements
**Reliability**
- Missing calibration, public datasets, or hardware prerequisites must not be treated as passing.
**Performance**
- First-fix timing and peak memory are reported with percentile summaries where enough trials run.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Relocalization trigger assertion | Missing-position thresholds trigger request checks |
| AC-2 | Restart report parser | Restart and first-output events are present |
| AC-3 | Trial aggregation | p95 first-fix summary or blocked reason is emitted |
| AC-4 | Resource metric parser | Peak memory and stage timings are captured |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Sharp-turn/disconnected replay | NFT-RES-02 | Verified relocalization or degraded evidence | Reliability |
| AC-2 | Mission restart trace | NFT-RES-03 | First valid output and FDR restart evidence | Reliability |
| AC-3 | Cold-start harness | NFT-PERF-04 | p95 first fix <30 s or blocked prerequisite | Performance |
| AC-4 | Cold-start resource monitoring | NFT-RES-LIM-05 | Peak memory <8 GB or blocked/failure evidence | Performance |
## Constraints
- Restart tests must preserve fixture read-only guarantees.
- Trial loops must be bounded and report partial results if interrupted.
- Hardware-only assertions must be clearly marked when not runnable locally.
## Risks & Mitigation
**Risk 1: Long cold-start trials are expensive**
- *Risk*: Full 50-run evidence may not be practical on every PR.
- *Mitigation*: Support smoke mode for PRs and full mode for release gates, with clear report labels.
_docs/02_tasks/todo/AZ-239_jetson_resource_endurance_tests.md
+94
View File
@@ -0,0 +1,94 @@
# Jetson Resource Endurance Tests
**Task**: AZ-239_jetson_resource_endurance_tests
**Name**: Jetson Resource Endurance Tests
**Description**: Implement release-gate resource and endurance tests for Jetson memory, thermal/power behavior, and FDR rollover.
**Complexity**: 5 points
**Dependencies**: AZ-233_test_infrastructure
**Component**: Blackbox Tests
**Tracker**: AZ-239
**Epic**: AZ-218
## Problem
Release readiness requires hardware/resource evidence that cannot be proven by ordinary unit tests or short local replay runs.
## Outcome
- Jetson memory and thermal/power metrics are captured where hardware is available.
- FDR 8-hour synthetic load verifies rollover, storage cap, and retained payload classes.
- Hardware-only prerequisites are reported as blocked when not available.
## Scope
### Included
- NFT-RES-LIM-01 Jetson Memory Budget.
- NFT-RES-LIM-02 Thermal And Power Envelope.
- NFT-RES-LIM-04 Flight Data Recorder Rollover.
### Excluded
- Still-image replay accuracy.
- Satellite anchor/cache security tests.
- Cold-start first-fix trials.
## Acceptance Criteria
**AC-1: Jetson memory budget is measured**
Given Jetson hardware or equivalent production target is available
When sustained replay and trigger-path workload runs
Then CPU/GPU shared memory, process RSS, CUDA allocations, and OOM/throttle status are recorded.
**AC-2: Thermal and power endurance is validated or blocked**
Given thermal test prerequisites are available
When the sustained 25 W workload runs
Then throttle flags, temperatures, clocks, and latency are recorded for the required duration; otherwise the run reports blocked prerequisites.
**AC-3: FDR rollover is validated**
Given an 8-hour synthetic mission load
When FDR output reaches rollover conditions
Then storage remains within the cap, rollover is logged, and no payload class is silently dropped.
**AC-4: Evidence artifacts are complete**
Given resource/endurance scenarios complete or block
When reporting finishes
Then metrics, duration, environment, status, and artifact paths are written.
## Non-Functional Requirements
**Performance**
- Resource evidence must include duration and sampling interval.
**Reliability**
- Hardware-unavailable results are `blocked`, not `passed`.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Resource metric parser | Memory and throttle fields are present |
| AC-2 | Blocked prerequisite reporter | Missing hardware/thermal setup records blocked status |
| AC-3 | FDR rollover report parser | Storage, rollover, and payload-class fields are validated |
| AC-4 | Evidence manifest writer | Artifact paths and run metadata are present |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Jetson/prod-equivalent hardware | NFT-RES-LIM-01 | Peak memory <8 GB or explicit failure | Performance |
| AC-2 | Thermal/power test setup | NFT-RES-LIM-02 | No throttle over required duration or blocked/failure | Performance |
| AC-3 | Synthetic 8-hour mission load | NFT-RES-LIM-04 | FDR cap and rollover behavior are evidenced | Reliability |
| AC-4 | Resource/endurance reports | All included scenarios | Complete artifact manifest and status | Reliability |
## Constraints
- These tests are release-gate oriented and may be skipped or blocked in ordinary PR mode.
- Raw frames must not be retained during FDR load tests.
- Resource tests must not write outside run-scoped output directories.
## Risks & Mitigation
**Risk 1: Hardware gates are unavailable during local development**
- *Risk*: Developers cannot run full evidence locally.
- *Mitigation*: Support blocked status and separate PR smoke mode from release-gate execution.
_docs/02_tasks/todo/AZ-240_native_vio_backend_integration.md
+95
View File
@@ -0,0 +1,95 @@
# Native VIO Backend Integration
**Task**: AZ-240_native_vio_backend_integration
**Name**: Native VIO Backend Integration
**Description**: Replace the deterministic VIO placeholder path with a real native backend integration boundary for representative replay.
**Complexity**: 5 points
**Dependencies**: AZ-228_vio_adapter
**Component**: VIO Adapter
**Tracker**: AZ-240
**Epic**: AZ-213
## Problem
The current VIO adapter satisfies the public contract with deterministic scaffold behavior, but it does not exercise a real native VIO backend for synchronized replay.
## Outcome
- A production-capable native VIO bridge is available behind the existing `VioBackend` protocol.
- Backend-specific setup remains isolated from the public VIO adapter boundary.
- Existing timestamp mismatch, tracking-loss, health, and no-WGS84-authority behavior is preserved.
## Scope
### Included
- Native/backend bridge implementation behind `VioBackend`.
- Backend initialization and runtime failure mapping into explicit health/error states.
- Replay-driven relative pose, velocity, bias, tracking quality, and covariance output.
- Tests that prove the real backend path is selected when configured.
### Excluded
- Absolute WGS84 authority or safety fusion.
- Satellite-anchor fallback logic.
- Direct test imports of backend internals.
## Dependencies
### Document Dependencies
- `_docs/02_document/components/02_vio_adapter/description.md`
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/geometry_time_sync.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
## Acceptance Criteria
**AC-1: Native backend path emits VIO state**
Given synchronized replay frames and telemetry
When VIO processing runs with the native backend enabled
Then the adapter emits a relative VIO state packet from the native path.
**AC-2: Backend failures are explicit**
Given backend initialization or runtime failure
When VIO processing or health reporting runs
Then the adapter surfaces an explicit error and degraded or failed health state.
**AC-3: Existing safety boundaries remain intact**
Given timestamp mismatch, low tracking quality, or successful native output
When the adapter returns a result
Then degraded behavior, tracking quality, and absence of WGS84 authority remain intact.
## Non-Functional Requirements
**Performance**
- Replay execution must expose latency and memory metrics for later Jetson profiling gates.
**Reliability**
- Backend failures must not be hidden behind deterministic fallback success.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Configured native backend path | Native estimate is used, not deterministic fallback |
| AC-2 | Backend init/runtime failure | Explicit error and degraded/failed health |
| AC-3 | Timestamp/quality boundaries | Existing degraded/no-WGS84 behavior preserved |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Derkachi or representative synchronized replay | Native VIO replay path | Relative estimates are emitted or blocked with a real prerequisite reason | Performance |
## Constraints
- Keep backend-specific dependencies behind the `vio_adapter` native boundary.
- Do not make the VIO adapter the safety or WGS84 authority.
- If required native packages are unavailable locally, tests must skip or block with explicit prerequisite evidence rather than passing through the deterministic fallback.
## Risks & Mitigation
**Risk 1: Native dependency unavailable in local CI**
- *Risk*: The real backend cannot run on all developer machines.
- *Mitigation*: Provide dependency-gated tests that fail only when the backend is configured but broken, and report blocked prerequisites for full replay gates.
_docs/02_tasks/todo/AZ-241_real_satellite_vpr_descriptor_retrieval.md
+95
View File
@@ -0,0 +1,95 @@
# Real Satellite VPR Descriptor Retrieval
**Task**: AZ-241_real_satellite_vpr_descriptor_retrieval
**Name**: Real Satellite VPR Descriptor Retrieval
**Description**: Replace the tuple-similarity satellite retrieval scaffold with the real local descriptor/index retrieval path promised by the Satellite Service design.
**Complexity**: 5 points
**Dependencies**: AZ-230_satellite_service_vpr_retrieval
**Component**: Satellite Service
**Tracker**: AZ-241
**Epic**: AZ-214
## Problem
The current Satellite Service can load in-memory descriptor records and rank them with local tuple similarity, but it does not yet integrate the real offline descriptor/index retrieval path.
## Outcome
- Local mission cache descriptor/index packages can be loaded by the runtime retrieval path.
- Retrieval uses the selected CPU FAISS/DINOv2-VLAD-compatible boundary where available.
- Freshness filtering, bounded top-K output, descriptor-fidelity checks, and no in-flight network behavior remain intact.
## Scope
### Included
- Local descriptor/index package loading from the offline cache boundary.
- Real local VPR retrieval implementation behind the public Satellite Service API.
- Explicit degraded/no-candidate/index failure behavior.
- Tests that distinguish the real retrieval path from the current tuple-similarity scaffold.
### Excluded
- Local feature matching, RANSAC, or anchor acceptance.
- In-flight provider or Suite service calls.
- TensorRT/ONNX optimization unless descriptor-fidelity gates are in place.
## Dependencies
### Document Dependencies
- `_docs/02_document/components/04_satellite_retrieval/description.md`
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/contracts/shared/config_errors_telemetry.md`
- `_docs/02_document/components/06_cache_tile_lifecycle/description.md`
## Acceptance Criteria
**AC-1: Real local index readiness is reported**
Given a valid local descriptor/index package
When the Satellite Service loads the package
Then readiness reflects the real local index and loaded record count.
**AC-2: Real top-K retrieval returns candidates**
Given a relocalization request and loaded local index
When retrieval runs
Then bounded candidates come from the real local descriptor/index path with scores, footprints, and freshness state.
**AC-3: Missing or invalid indexes degrade safely**
Given missing, corrupt, incompatible, or empty local index data
When retrieval runs
Then the result is explicit degraded/no-candidate behavior without unsafe anchors or network calls.
## Non-Functional Requirements
**Performance**
- Retrieval remains trigger-based and exposes latency metrics for Jetson profiling.
**Security**
- Retrieval must not perform in-flight provider or Suite service calls.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Real index package load | Ready status references loaded real index data |
| AC-2 | Query against fixture index | Candidates come from the real retrieval path |
| AC-3 | Missing/corrupt index | Explicit degraded/no-candidate result |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-2 | Public/cache fixture with descriptor index | VPR recall and top-K policy | Candidate bounds, freshness, and latency evidence are reported | Performance |
## Constraints
- Use only local preloaded cache/index data during flight-mode retrieval.
- Keep optional optimized engines behind descriptor-fidelity gates.
- Missing native/index prerequisites must be reported as blocked, not silently passed by the scaffold path.
## Risks & Mitigation
**Risk 1: Heavy native/index dependencies do not run in ordinary CI**
- *Risk*: The real retrieval path needs packages or data unavailable in local CI.
- *Mitigation*: Keep fast contract tests for package parsing and dependency-gated integration tests for real index execution.
_docs/02_tasks/todo/AZ-242_real_anchor_feature_matching_ransac.md
+94
View File
@@ -0,0 +1,94 @@
# Real Anchor Feature Matching And RANSAC
**Task**: AZ-242_real_anchor_feature_matching_ransac
**Name**: Real Anchor Feature Matching And RANSAC
**Description**: Replace the precomputed evidence gate-only scaffold with real local feature matching and geometry verification behind the Anchor Verification boundary.
**Complexity**: 5 points
**Dependencies**: AZ-231_anchor_verification_matching, AZ-241_real_satellite_vpr_descriptor_retrieval
**Component**: Anchor Verification
**Tracker**: AZ-242
**Epic**: AZ-215
## Problem
The current Anchor Verification component can classify precomputed `MatchEvidence`, but it does not yet run real feature extraction, matching, homography estimation, or RANSAC/USAC geometry checks.
## Outcome
- Approved matcher profiles can compute correspondence evidence from frame imagery and candidate tile data.
- Geometry verification produces inliers, MRE, homography/provenance, runtime, and rejection reasons.
- Existing safety gates continue to reject unsafe candidates before any anchor is trusted.
## Scope
### Included
- Matcher bridge for approved ALIKED/DISK + LightGlue and SIFT/ORB baseline profiles where dependencies are available.
- Homography and RANSAC/USAC evidence generation from local imagery/tile fixtures.
- Integration with existing `GeometryGatedAnchorVerifier` decision output.
- Benchmark reporting from actual matching paths.
### Excluded
- VPR candidate ranking.
- Safety wrapper fusion/promotion policy.
- Per-frame steady-state VIO hot path execution.
## Dependencies
### Document Dependencies
- `_docs/02_document/components/05_anchor_verification/description.md`
- `_docs/02_document/contracts/shared/runtime_contracts.md`
- `_docs/02_document/components/04_satellite_retrieval/description.md`
## Acceptance Criteria
**AC-1: Matching path computes evidence**
Given a usable frame and fresh candidate tile
When anchor verification runs
Then matcher evidence is computed from local imagery and includes inliers, MRE, homography, provenance, and runtime.
**AC-2: Unsafe candidates are rejected**
Given low inliers, high reprojection error, stale or untrusted provenance, or geometry failure
When verification runs
Then no accepted anchor decision is emitted for that candidate.
**AC-3: Real matcher benchmark is reportable**
Given configured matcher profiles and fixture inputs
When benchmark runs
Then runtime and quality metrics are reported from actual matching paths.
## Non-Functional Requirements
**Performance**
- Learned matching remains trigger-based and benchmarked separately from the VIO hot path.
**Reliability**
- Missing matcher dependencies or fixture data must be explicit blocked prerequisites, not passing scaffold behavior.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | Fixture matching path | Evidence is computed from imagery/tile input |
| AC-2 | Bad geometry/provenance | Candidate is rejected with reason |
| AC-3 | Matcher benchmark | Runtime and quality metrics come from real path |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|-------------------------|--------------|-------------------|----------------|
| AC-1 | Aerial/cache fixture pair | Anchor verification path | Accepted anchors meet MRE/inlier gates with real evidence | Performance |
## Constraints
- Keep native feature extraction and RANSAC acceleration under `anchor_verification`.
- Do not trust precomputed evidence in production paths without provenance checks.
- SuperPoint or other legally restricted models remain excluded unless explicitly approved.
## Risks & Mitigation
**Risk 1: False anchor acceptance**
- *Risk*: Real cross-domain matching can produce plausible but unsafe geometry.
- *Mitigation*: Preserve freshness, provenance, inlier, MRE, and downstream safety gates; add negative fixtures for low-texture and stale-cache cases.
_docs/03_implementation/batch_01_cycle1_report.md
+41
View File
@@ -0,0 +1,41 @@
# Batch Report
**Batch**: 1
**Tasks**: AZ-219_initial_structure
**Date**: 2026-05-03
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-219_initial_structure | Done | 98 files | Pass | 7/7 ACs covered | None |
## AC Test Coverage: All covered
| AC Ref | Coverage |
|--------|----------|
| AC-1 | `test_scaffold_paths_cover_runtime_test_and_evidence_layout` verifies source, tests, migrations, deployment, configuration, data, CI, and compose scaffold paths. |
| AC-2 | `test_runtime_component_public_modules_are_importable` and `test_shared_contract_locations_are_importable` verify public component and shared contract namespaces. |
| AC-3 | `test_scaffold_paths_cover_runtime_test_and_evidence_layout` verifies compose, env template, and migration paths; compose config validation passed. |
| AC-4 | `test_scaffold_paths_cover_runtime_test_and_evidence_layout` verifies `.github/workflows/ci.yml`; the workflow defines format, lint, unit test, compose config, and artifact placeholder jobs. |
| AC-5 | `test_scaffold_paths_cover_runtime_test_and_evidence_layout` verifies unit, integration, black-box, fixture, SITL, and e2e runner paths. |
| AC-6 | `test_scaffold_paths_cover_runtime_test_and_evidence_layout` verifies deployment scripts and evidence report paths. |
| AC-7 | `test_ignore_rules_exclude_runtime_payloads_and_secrets` verifies secrets, raw frames, cache/FDR payloads, and test result artifacts are ignored. |
## Code Review Verdict: PASS
Review report: `_docs/03_implementation/reviews/batch_01_review.md`
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Verification
- `.venv/bin/python -m black --check src tests e2e/replay` passed.
- `.venv/bin/python -m ruff check src tests e2e/replay` passed.
- `.venv/bin/python -m pytest` passed: 5 tests.
- `docker compose -f docker-compose.yml config` passed.
- `docker compose -f docker-compose.test.yml config` passed.
## Next Batch: AZ-220_shared_runtime_contracts
_docs/03_implementation/batch_02_cycle1_report.md
+34
View File
@@ -0,0 +1,34 @@
# Batch Report
**Batch**: 2
**Tasks**: AZ-220_shared_runtime_contracts
**Date**: 2026-05-03
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-220_shared_runtime_contracts | Done | 8 files | Pass | 2/2 ACs covered | None |
## AC Test Coverage: All covered
| AC Ref | Coverage |
|--------|----------|
| AC-1 | `test_runtime_dtos_accept_valid_minimal_values` verifies the shared DTO contract surface can be imported and constructed. |
| AC-2 | `test_missing_required_timestamp_is_rejected_with_structured_error`, `test_raw_frame_retention_is_rejected`, `test_position_accuracy_cannot_under_report_covariance`, and `test_accepted_anchor_requires_estimated_pose` verify malformed DTOs are rejected with structured Pydantic validation errors. |
## Code Review Verdict: PASS
Review report: `_docs/03_implementation/reviews/batch_02_review.md`
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Verification
- `.venv/bin/python -m black --check src tests e2e/replay` passed.
- `.venv/bin/python -m ruff check src tests e2e/replay` passed.
- `.venv/bin/python -m pytest` passed: 11 tests.
## Next Batch: AZ-221_shared_geometry_time_sync, AZ-222_runtime_config_errors_telemetry
_docs/03_implementation/batch_03_cycle1_report.md
+37
View File
@@ -0,0 +1,37 @@
# Batch Report
**Batch**: 3
**Tasks**: AZ-221_shared_geometry_time_sync, AZ-222_runtime_config_errors_telemetry
**Date**: 2026-05-03
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-221_shared_geometry_time_sync | Done | 5 files | Pass | 2/2 ACs covered | None |
| AZ-222_runtime_config_errors_telemetry | Done | 7 files | Pass | 2/2 ACs covered | None |
## AC Test Coverage: All covered
| AC Ref | Coverage |
|--------|----------|
| AZ-221 AC-1 | `test_wgs84_local_round_trip_is_deterministic` verifies deterministic WGS84/local conversion and metric output. |
| AZ-221 AC-2 | `test_non_monotonic_timestamps_return_explicit_violation` and `test_time_window_reports_gap_instead_of_dropping_silently` verify explicit time-sync violation results. |
| AZ-222 AC-1 | `test_missing_production_cache_dir_returns_readiness_failure` verifies missing production settings produce a structured readiness failure. |
| AZ-222 AC-2 | `test_dependency_error_envelope_has_required_structured_fields` verifies dependency errors include component, category, severity, and retryability. |
## Code Review Verdict: PASS
Review report: `_docs/03_implementation/reviews/batch_03_review.md`
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Verification
- `.venv/bin/python -m black --check src tests e2e/replay` passed.
- `.venv/bin/python -m ruff check src tests e2e/replay` passed.
- `.venv/bin/python -m pytest` passed: 17 tests.
## Next Batch: AZ-223_camera_ingest_calibration, AZ-224_mavlink_gcs_gateway, AZ-225_tile_manager_cache_manifest, AZ-227_fdr_event_recorder
_docs/03_implementation/batch_04_cycle1_report.md
+47
View File
@@ -0,0 +1,47 @@
# Batch Report
**Batch**: 4
**Tasks**: AZ-223_camera_ingest_calibration, AZ-224_mavlink_gcs_gateway, AZ-225_tile_manager_cache_manifest, AZ-227_fdr_event_recorder
**Date**: 2026-05-03
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-223_camera_ingest_calibration | Done | 4 files | Pass | 3/3 ACs covered | None |
| AZ-224_mavlink_gcs_gateway | Done | 4 files | Pass | 3/3 ACs covered | None |
| AZ-225_tile_manager_cache_manifest | Done | 4 files | Pass | 3/3 ACs covered | None |
| AZ-227_fdr_event_recorder | Done | 4 files | Pass | 3/3 ACs covered | None |
## AC Test Coverage: All covered
| AC Ref | Coverage |
|--------|----------|
| AZ-223 AC-1 | `test_valid_frame_packet_contains_metadata_reports_and_normalization_hint` verifies timestamp, calibration, quality, occlusion, and normalization metadata. |
| AZ-223 AC-2 | `test_total_occlusion_marks_frame_unusable_for_vio_and_anchor` verifies blackout frames are unavailable for visual paths. |
| AZ-223 AC-3 | `test_raw_frame_payload_retention_is_rejected` verifies raw frame payload retention is rejected. |
| AZ-224 AC-1 | `test_telemetry_subscription_emits_normalized_sample` verifies normalized shared telemetry samples. |
| AZ-224 AC-2 | `test_invalid_gps_input_estimate_is_rejected_without_emission` verifies unsafe `GPS_INPUT` requests are rejected without emission. |
| AZ-224 AC-3 | `test_operator_status_messages_are_rate_limited_by_text` verifies QGC-visible status rate limiting. |
| AZ-225 AC-1 | `test_valid_cache_manifest_activates_trusted_records` verifies valid cache activation. |
| AZ-225 AC-2 | `test_tampered_or_stale_tile_is_rejected_with_auditable_reason` verifies hash and freshness rejection reasons. |
| AZ-225 AC-3 | `test_tile_metadata_lookup_returns_record_or_explicit_rejection` verifies trusted metadata lookup and explicit rejection. |
| AZ-227 AC-1 | `test_valid_event_append_indexes_metadata_and_payload_reference` verifies event metadata and payload references are stored within bounds. |
| AZ-227 AC-2 | `test_rollover_threshold_records_explicit_rollover_result` verifies rollover is explicit. |
| AZ-227 AC-3 | `test_export_request_produces_queryable_evidence_artifacts` verifies export evidence and analytics references. |
## Code Review Verdict: PASS
Review report: `_docs/03_implementation/reviews/batch_04_review.md`
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Verification
- `.venv/bin/python -m black --check src tests e2e/replay` passed.
- `.venv/bin/python -m ruff check src tests e2e/replay` passed.
- `.venv/bin/python -m pytest` passed: 29 tests.
## Next Batch: AZ-226_generated_tile_orthorectification
_docs/03_implementation/batch_05_cycle1_report.md
+35
View File
@@ -0,0 +1,35 @@
# Batch Report
**Batch**: 5
**Tasks**: AZ-226_generated_tile_orthorectification
**Date**: 2026-05-03
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-226_generated_tile_orthorectification | Done | 4 files | Pass | 3/3 ACs covered | None |
## AC Test Coverage: All covered
| AC Ref | Coverage |
|--------|----------|
| AZ-226 AC-1 | `test_eligible_frame_stages_generated_cog_and_sidecar` verifies generated COG and sidecar staging for eligible frames. |
| AZ-226 AC-2 | `test_high_covariance_generated_tile_write_is_rejected` verifies unsafe high-covariance writes are rejected and not packaged. |
| AZ-226 AC-3 | `test_sync_package_includes_manifest_delta_sidecar_covariance_and_trust_level` verifies sync package audit metadata. |
## Code Review Verdict: PASS
Review report: `_docs/03_implementation/reviews/batch_05_review.md`
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Verification
- `.venv/bin/python -m black --check src tests e2e/replay` passed.
- `.venv/bin/python -m ruff check src tests e2e/replay` passed.
- `.venv/bin/python -m pytest` passed: 32 tests.
## Next Batch: AZ-228_vio_adapter, AZ-229_satellite_service_sync
_docs/03_implementation/batch_06_cycle1_report.md
+39
View File
@@ -0,0 +1,39 @@
# Batch Report
**Batch**: 6
**Tasks**: AZ-228_vio_adapter, AZ-229_satellite_service_sync
**Date**: 2026-05-03
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-228_vio_adapter | Done | 4 files | Pass | 3/3 ACs covered | None |
| AZ-229_satellite_service_sync | Done | 4 files | Pass | 3/3 ACs covered | None |
## AC Test Coverage: All covered
| AC Ref | Coverage |
|--------|----------|
| AZ-228 AC-1 | `test_valid_synchronized_packet_emits_vio_state` verifies synchronized frame/IMU processing emits a relative VIO state packet. |
| AZ-228 AC-2 | `test_timestamp_mismatch_is_explicit_validation_error` verifies timestamp mismatch is rejected with an explicit error. |
| AZ-228 AC-3 | `test_tracking_loss_degrades_health_without_emitting_absolute_position` verifies health reports degraded tracking state. |
| AZ-229 AC-1 | `test_pre_flight_import_returns_package_for_tile_manager_validation` verifies mission cache packages are exposed for Tile Manager validation. |
| AZ-229 AC-2 | `test_post_flight_upload_records_retryable_failure_for_audit` verifies upload outcomes are auditable and retryable failures retain packages. |
| AZ-229 AC-3 | `test_in_flight_sync_is_blocked_without_calling_network_boundary` verifies in-flight sync is blocked before network/uploader calls. |
## Code Review Verdict: PASS
Review report: `_docs/03_implementation/reviews/batch_06_review.md`
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Verification
- `.venv/bin/python -m black --check src tests e2e/replay` passed.
- `.venv/bin/python -m ruff check src tests e2e/replay` passed.
- `.venv/bin/python -m pytest` passed: 38 tests.
## Next Batch: AZ-230_satellite_service_vpr_retrieval
_docs/03_implementation/batch_07_cycle1_report.md
+35
View File
@@ -0,0 +1,35 @@
# Batch Report
**Batch**: 7
**Tasks**: AZ-230_satellite_service_vpr_retrieval
**Date**: 2026-05-03
## Task Results
| Task | Status | Files Modified | Tests | AC Coverage | Issues |
|------|--------|----------------|-------|-------------|--------|
| AZ-230_satellite_service_vpr_retrieval | Done | 4 files | Pass | 3/3 ACs covered | None |
## AC Test Coverage: All covered
| AC Ref | Coverage |
|--------|----------|
| AZ-230 AC-1 | `test_valid_local_index_load_reports_ready_status` verifies local index loading reports readiness and record count. |
| AZ-230 AC-2 | `test_loaded_index_returns_bounded_candidates_with_freshness` verifies bounded top-K candidate output with tile/chunk IDs, score, footprint, and freshness. |
| AZ-230 AC-3 | `test_missing_index_degrades_with_explicit_no_candidate_result` verifies missing index produces explicit degraded behavior. |
## Code Review Verdict: PASS
Review report: `_docs/03_implementation/reviews/batch_07_review.md`
## Auto-Fix Attempts: 0
## Stuck Agents: None
## Verification
- `.venv/bin/python -m black --check src tests e2e/replay` passed.
- `.venv/bin/python -m ruff check src tests e2e/replay` passed.
- `.venv/bin/python -m pytest` passed: 42 tests.
## Next Batch: AZ-231_anchor_verification_matching

Some files were not shown because too many files have changed in this diff Show More