Merge branch 'try02' into dev

This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-05 05:51:29 +03:00
259 changed files with 19947 additions and 2778 deletions
+160 -35
View File
@@ -1,50 +1,175 @@
# Position Accuracy
# Acceptance Criteria
- The system should determine GPS coordinates of frame centers for 80% of photos within 50m error compared to real GPS
- The system should determine GPS coordinates of frame centers for 60% of photos within 20m error compared to real GPS
- Maximum cumulative VO drift between satellite correction anchors should be less than 100 meters
- System should report a confidence score per position estimate (high = satellite-anchored, low = VO-extrapolated with drift)
> **Last revised**: 2026-05-01 (Phase 1 AC/restrictions assessment clarifications).
> Changes vs. previous version (2026-04-25): AC-1.2 split into hard-floor + stretch; AC-1.4 made quantitative; AC-2.2 split per pipeline stage; AC-3.4 dual-trigger; AC-4.3 autopilot-pinned; AC-5.2 N pinned; AC-7.1 scoped to level flight; AC-8.2 freshness by sector; six new AC added (AC-NEW-1 … AC-NEW-6).
> Changes 2026-04-26: AC-4.3 extended to dual-channel hybrid (GPS_INPUT primary + ODOMETRY auxiliary); AC-8.6 added (VPR retrieval-unit + change-robustness); AC-NEW-7 added with confirmed numeric thresholds (cache-poisoning safety budget).
> Changes 2026-04-29: AC-3.5 and AC-NEW-8 added for temporary visual blackout/cloud occlusion during GPS spoofing, including IMU-only degraded navigation, covariance growth, and failover limits.
> Changes 2026-05-01: AC-1.3 anchor-age reporting clarified; AC-2.1 split so the >95% rate applies to VO registration, not every satellite re-anchor; AC-5.2 and AC-NEW-2 now require ArduPilot Plane SITL trigger verification; AC-8.3 storage accounting and AC-NEW-7 Satellite Service ownership clarified.
# Image Processing Quality
## Position Accuracy
- Image Registration Rate > 95% for normal flight segments. The system can find enough matching features to confidently calculate the camera's 6-DoF pose and stitch that image into the trajectory
- Mean Reprojection Error (MRE) < 1.0 pixels
- **AC-1.1** — The system shall determine GPS coordinates of frame centers within **50 m** of true GPS for **≥80%** of photos in normal flight segments.
- **AC-1.2** — The system shall determine GPS coordinates of frame centers within **20 m** of true GPS for **≥50%** of photos in normal flight segments.
- **AC-1.3** — Maximum cumulative VO drift between two consecutive satellite-anchored fixes shall be **<100 m** (VO-only fallback) or **<50 m** (when IMU is fused). Drift is measured as ‖VO-extrapolated centre next anchor centre‖ at the moment of the anchor fix. Every emitted estimate shall include `last_satellite_anchor_age_ms`; validation results shall be binned by anchor age, and the solution draft must define the maximum anchor age after which estimates are treated as degraded (`vo_extrapolated` or `dead_reckoned`) with monotonically growing covariance.
- **AC-1.4** — The system shall report a **quantitative confidence score** per position estimate, comprising:
- the 95% covariance ellipse semi-major axis in meters, AND
- a categorical label `{satellite_anchored, vo_extrapolated, dead_reckoned}`.
# Resilience & Edge Cases
## Image Processing Quality
- The system should correctly continue work even in the presence of up to 350m outlier between 2 consecutive photos (due to tilt of the plane)
- System should correctly continue work during sharp turns, where the next photo doesn't overlap at all or overlaps less than 5%. The next photo should be within 200m drift and at an angle of less than 70 degrees. Sharp-turn frames are expected to fail VO and should be handled by satellite-based re-localization
- System should operate when UAV makes a sharp turn and next photos have no common points with previous route. It should figure out the location of the new route segment and connect it to the previous route. There could be more than 2 such disconnected segments, so this strategy must be core to the system
- In case the system cannot determine the position of 3 consecutive frames by any means, it should send a re-localization request to the ground station operator via telemetry link. While waiting for operator input, the system continues attempting VO/IMU dead reckoning and the flight controller uses last known position + IMU extrapolation
- **AC-2.1** — Image registration rate is split by registration type:
- **AC-2.1a — VO registration**: frame-to-frame visual registration shall succeed for **>95%** of normal flight segments (defined as: nadir flight ±10° bank / pitch, ≥40% overlap with prior frame, daytime, usable texture, no full visual blackout).
- **AC-2.1b — Satellite-anchor registration**: cross-domain UAV-photo to satellite/cache registration is measured separately and is not hidden inside AC-2.1a. Satellite anchoring must satisfy AC-1.1 / AC-1.2 position accuracy, AC-2.2 cross-domain MRE, AC-8.2 freshness, and AC-8.6 retrieval behavior on season-matched tiles.
- **AC-2.2** — Mean Reprojection Error (MRE):
- **<1.0 px** for VO frame-to-frame homography on overlapping aerial pairs;
- **<2.5 px** for satellite-anchored cross-domain (UAV photo ↔ ortho satellite tile) registration.
# Real-Time Onboard Performance
## Resilience & Edge Cases
- Less than 400ms end-to-end per frame: from camera capture to GPS coordinate output to the flight controller (camera shoots at ~3fps)
- Memory usage should stay below 8GB shared memory (Jetson Orin Nano Super — CPU and GPU share the same 8GB LPDDR5 pool)
- The system must output calculated GPS coordinates directly to the flight controller via MAVLink GPS_INPUT messages (using MAVSDK)
- Position estimates are streamed to the flight controller frame-by-frame; the system does not batch or delay output
- The system may refine previously calculated positions and send corrections to the flight controller as updated estimates
- **AC-3.1** — The system shall correctly continue work in the presence of up to **350 m** outliers between two consecutive photos (caused by airframe tilt up to ±20°).
- **AC-3.2** — The system shall correctly continue work during sharp turns where the next photo overlaps **<5%** with the previous, drifts **<200 m**, and changes heading **<70°**. Sharp-turn frames are expected to fail VO and shall be handled by satellite-based re-localization (place recognition over the satellite tile cache).
- **AC-3.3** — The system shall handle **≥3 disconnected segments** per flight, connecting each new segment to the previous trajectory via global descriptor retrieval + RANSAC pose-graph relocalization. This is a core capability, not a degraded mode.
- **AC-3.4** — When the system cannot determine position for **≥3 consecutive frames AND ≥2 s**, it shall send a re-localization request to the ground station via telemetry. While waiting, it continues VO/IMU dead reckoning and the flight controller uses last known position + IMU extrapolation.
- **AC-3.5** — During temporary **visual blackout** where the navigation camera provides no usable ground signal (e.g., clouds/occlusion/whiteout) while GPS is denied or spoofed, the system shall switch to `{dead_reckoned}` within **≤1 processed frame OR ≤400 ms**, reject the spoofed GPS as an estimator input, and propagate position solely from the last trusted state + flight-controller IMU/attitude/airspeed/altitude inputs until visual or satellite anchoring recovers. During this mode, covariance shall grow monotonically, `GPS_INPUT.horiz_accuracy` shall not under-report the 95% covariance semi-major axis, and QGroundControl shall receive a `VISUAL_BLACKOUT_IMU_ONLY` status at **12 Hz**.
# Startup & Failsafe
## Real-Time Onboard Performance
- The system initializes using the last known valid GPS position from the flight controller before GPS denial begins
- If the system completely fails to produce any position estimate for more than N seconds (TBD), the flight controller should fall back to IMU-only dead reckoning and the system should log the failure
- On companion computer reboot mid-flight, the system should attempt to re-initialize from the flight controller's current IMU-extrapolated position
- **AC-4.1** — End-to-end latency from camera capture to GPS coordinate output to the flight controller shall be **<400 ms p95**. Up to ~10% of frames may be dropped under sustained load (skip-allowed). Heavy global VPR / cross-domain re-ranking shall be conditional, not part of the steady-state per-frame path, unless profiling proves the full path stays inside the latency and memory budgets on the target Jetson.
- **AC-4.2** — Memory usage shall remain below **8 GB** shared on Jetson Orin Nano Super (CPU and GPU share the same 8 GB LPDDR5 pool).
- **AC-4.3** — The system shall output its position estimate to the flight controller via **two parallel MAVLink channels**, both emitted by **pymavlink** (general telemetry uses MAVSDK):
- **Primary**: `GPS_INPUT` targeting **ArduPilot** with `GPS1_TYPE=14` (MAVLink GPS substitute). Matches the "replacement for the GPS module" framing of the build.
- **Auxiliary** (when the EKF emits a fix with full 6-DoF covariance and quality > VISO_QUAL_MIN): `ODOMETRY` so EKF3 can fuse the richer covariance + native yaw error + quality field. ArduPilot's own dev docs designate ODOMETRY as the preferred external-nav channel for non-GPS substitution; we hybridise to keep AC-4.3's GPS-substitute framing while not throwing away the covariance fidelity that AC-NEW-4 depends on.
- FC source priorities are configured so GPS_INPUT remains the failover path if ODOMETRY trips a parameter gate.
- **v1 scope clause (added 2026-04-26 — see solution_draft03 finding M-30)**: v1 ships **GPS_INPUT only**; the ODOMETRY auxiliary channel is intentionally **disabled** in v1 because feeding both `GPS_INPUT` and `ODOMETRY` for overlapping axes triggers ArduPilot EKF3 double-fusion bugs (issues #30076 / #32506). `EK3_SRC1_*=GPS+Compass`; ODOMETRY emission re-enables in v1.1 once F-T9 SITL confirms PR #30080-class clean source-switching. Tests therefore assert v1 emits GPS_INPUT only and that ODOMETRY is *intentionally absent* on the wire.
- (Decision rationale: MAVSDK has no native GPS_INPUT support — see `_docs/00_research/00_ac_assessment.md` Q-1; ODOMETRY hybrid rationale — see Mode B finding M-1 in `_docs/00_research/02_fact_cards.md`; v1 single-channel rationale — see Mode B round-2 finding M-30 in `_docs/00_research/02_fact_cards.md` / solution_draft03.)
- **AC-4.4** — Position estimates are streamed to the flight controller frame-by-frame; the system shall not batch or delay output.
- **AC-4.5** — The system may refine previously calculated positions and send corrections to the flight controller as updated estimates.
# Ground Station & Telemetry
## Startup & Failsafe
- Position estimates and confidence scores should be streamed to the ground station via telemetry link for operator situational awareness
- The ground station can send commands to the onboard system (e.g., operator-assisted re-localization hint with approximate coordinates)
- Output coordinates in WGS84 format
- **AC-5.1** — The system shall initialise using the last known valid GPS position from the flight controller's EKF, plus IMU-extrapolated position at the moment of GPS denial.
- **AC-5.2** — If the system fails to produce any position estimate for **>3 s**, the flight controller shall fall back to IMU-only dead reckoning and the system shall log the failure. Because ArduPilot failsafe timing depends on vehicle type and parameters, this fallback behavior must be verified specifically in ArduPilot Plane SITL with the production parameter set; Copter defaults are reference evidence only.
- **AC-5.3** — On companion computer reboot mid-flight, the system shall attempt to re-initialise from the flight controller's current IMU-extrapolated position. See AC-NEW-1 for the cold-start time-to-first-fix budget.
# Object Localization
## Ground Station & Telemetry
- Other onboard AI systems can request GPS coordinates of objects detected by the AI camera
- The GPS-Denied system calculates object coordinates trigonometrically using: current UAV GPS position (from GPS-Denied), known AI camera angle, zoom, and current flight altitude. Flat terrain is assumed
- Accuracy is consistent with the frame-center position accuracy of the GPS-Denied system
- **AC-6.1** — Position estimates and confidence scores shall be streamed to **QGroundControl** via the MAVLink telemetry link. High-rate (per-frame) content stays on the local link for forensics; the GCS link is downsampled to **12 Hz** for situational awareness.
- **AC-6.2** — The ground station can send commands to the onboard system (e.g., operator-assisted re-localization hint with approximate coordinates) via STATUSTEXT, NAMED_VALUE_FLOAT, or a custom MAVLink dialect.
- **AC-6.3** — Output coordinates are in **WGS84** format (matches GPS_INPUT spec).
# Satellite Reference Imagery
## Object Localization (AI Camera)
- Satellite reference imagery resolution must be at least 0.5 m/pixel, ideally 0.3 m/pixel
- Satellite imagery for the operational area should be less than 2 years old where possible
- Satellite imagery must be pre-processed and loaded onto the companion computer before flight. Offline preprocessing time is not time-critical (can take minutes/hours)
- **AC-7.1** — Other onboard AI systems may request GPS coordinates of objects detected by the AI camera. Localization accuracy is **consistent with the frame-center accuracy of the GPS-Denied system in level flight (bank/pitch <5°)**. In maneuvering flight, ground-projection error is bounded by `altitude × |sin(unknown_bank_or_pitch)|` and the system shall publish that bound alongside the estimate.
- **AC-7.2** — The system computes object coordinates trigonometrically using: current UAV GPS position (from GPS-Denied), known AI-camera gimbal angle, zoom, and current flight altitude. Flat-terrain assumption applies.
## Satellite Reference Imagery
- **AC-8.1** — Satellite reference imagery is provided by the **Azaion Suite Satellite Service** (a separate component of the Suite). The runtime onboard system consumes this service through an offline tile cache interface; it does **not** call commercial providers (Maxar, Airbus, Planet, etc.) directly. The Satellite Service is responsible for upstream sourcing and is out of scope for this build. Required resolution at the cache interface: **at least 0.5 m/pixel, ideally 0.3 m/pixel**.
- **AC-8.2** — Satellite tiles consumed at runtime shall be:
- **<6 months old** for active-conflict sectors;
- **<12 months old** for stable rear sectors.
System shall reject or downgrade-confidence on tiles older than these thresholds (see AC-NEW-6).
- **AC-8.3** — Satellite imagery for the operational area shall be **pre-loaded and pre-processed** onto the companion computer before flight. Offline preprocessing time is not time-critical (minutes/hours). Pre-extracted tile descriptors (e.g., SuperPoint keypoints/descriptors and DINOv2-VLAD global descriptors) are part of the cache and count against the storage budget unless the solution draft explicitly defines a separate descriptor/index budget.
- **AC-8.4** — **Mid-flight tile generation & write-back**: during flight, the system shall continuously orthorectify navigation-camera frames into tiles aligned with the basemap projection and store them in the local cache, **deduplicated** so each ground sector is stored at most once (latest / highest-quality tile wins). On landing, the companion computer shall upload newly generated tiles back to the Azaion Suite Satellite Service so that the next mission cache contains imagery refreshed by the previous flight.
- **AC-8.5** — **Storage policy**: the system shall **not** retain raw navigation-camera frames or AI-camera frames as part of normal operation. Tiles are the only persistent imagery artifact. Forensic exception: a low-rate (≤0.1 Hz) thumbnail log of frames that **failed** tile generation may be retained for debugging within the FDR budget (AC-NEW-3).
- **AC-8.6** — **VPR retrieval unit + change-robustness**:
- The Visual Place Recognition (Component 2) FAISS index shall be built over **ground-footprint-sized "VPR chunks"** (~600800 m at the deployment altitude band, with **4050 % overlap** between adjacent chunks), **decoupled from the slippy-XYZ storage tile** (z=20). Any UAV frame footprint shall fall fully inside ≥1 chunk regardless of position.
- The index shall be **multi-scale**: in addition to fine-scale chunks (derived from z=20 storage), a coarser-scale chunk descriptor set (z=17 or z=18 effective scale) shall be maintained for change-robust retrieval in **active-conflict sectors** where building destruction or major scene change is expected.
- VPR top-K shall be **dynamically sized** by sector classification (AC-NEW-6) and EKF position covariance: K=5 in stable sectors with σ_xy ≤ 20 m; K=20 in active-conflict sectors; K=50 on expanding-window fallback.
- VPR shall be **invoked conditionally**, not on every frame: in steady state (last anchor age < 2 s, σ_xy < 20 m, VO healthy), the system uses a geometric prior from IMU+VO predicted position to rank candidate chunks by distance alone. VPR's DINOv2 forward is invoked on **re-loc triggers** (cold start AC-NEW-1, sharp turn AC-3.2, disconnected segment AC-3.3, σ_xy > 50 m, or VO failure for ≥2 frames).
## New AC (added in Phase 1 assessment, expanded with rationale & validation)
### AC-NEW-1 — Time-to-first-fix on cold start
**Statement.** From companion-computer boot, the system shall emit its first valid `GPS_INPUT` message in **<30 s**, given an IMU-extrapolated initial position handed over from the flight controller's EKF.
**Why it matters.** A mid-flight reboot (brown-out, watchdog reset, OS panic) is a realistic scenario on a fixed-wing UAV running an 8-hour mission. The autopilot continues to fly on IMU dead reckoning during the gap; a 30 s budget keeps that drift under ~500 m at 60 km/h cruise, which the EKF can absorb when our first fix arrives.
**Implementation drivers.** TensorRT engines must be built at install time (not at first run); CUDA / TRT init <5 s; tile-cache mmap warm at start; FAISS index loaded before MAVLink connect; first VPR retrieval + cross-view match must succeed at full resolution within the remaining budget.
**Validation.** Bench: cold-boot the companion 50× with simulated FC-pose input; record time from boot to first valid `GPS_INPUT` MAVLink frame. Pass = 95% percentile <30 s.
### AC-NEW-2 — Spoofing-promotion latency
**Statement.** When the flight controller signals GPS denial or spoofing (ArduPilot fix-loss / EKF lane-switch event; PX4 `EKF2_GPS_SPOOFED` flag if PX4 ever returns to scope), the GPS-Denied system shall promote its own estimate to the FC's primary GPS source within **<3 s**.
**Why it matters.** Without this gate, the FC may continue to follow a spoofed real-GPS source while our valid estimate sits idle. 3 s is short enough to keep the FC from acting on a malicious heading change but long enough to ride out a single-frame anomaly.
**Implementation drivers.** Subscribe to `GPS_RAW_INT`, `EKF_STATUS_REPORT`, `SYS_STATUS`, and any ArduPilot Plane EKF/GPS status messages available in the production firmware. Maintain an internal "real-GPS health" rolling average; switch to "primary" mode (raise our `GPS_INPUT` `fix_type` to 3D and assert) when the verified Plane-specific health trigger stays below threshold for >=1 s. Emit `STATUSTEXT` to QGC on every promotion / demotion.
**Validation.** ArduPilot Plane SITL: simulate spoofing (inject false `GPS_RAW_INT` from a malicious node); verify the exact trigger signals used by the production parameter set; measure time from spoof onset to our promotion. Pass = 95% percentile <3 s.
### AC-NEW-3 — Flight Data Recorder
**Statement.** The system shall retain to non-volatile storage, per flight: per-frame position estimates with covariance and source-label, IMU traces from the FC at full rate, all emitted `GPS_INPUT` frames, MAVLink raw stream (tlog), system health (CPU / GPU / temp / throttle), tiles generated mid-flight (AC-8.4), and a low-rate (≤0.1 Hz) thumbnail log of frames that failed tile generation. **Raw nav-cam frames and AI-cam frames are NOT retained** (AC-8.5). Storage cap **64 GB / flight**; recorder rolls over (oldest segment dropped first) after cap.
**Why it matters.** Tiles, telemetry traces, and IMU are the operationally useful artifacts: they reproduce the mission, feed the next mission's cache (AC-8.4), and let post-mission analysis explain any false-position event (AC-NEW-4). Raw frames are large and redundant once tiles exist.
**Implementation drivers.** Per-day directory layout; fixed-size segment files; rollover policy on segment-close, not on every write. NVMe ≥64 GB on top of the persistent satellite-tile cache.
**Validation.** Bench: run an 8-hour synthetic load (3 Hz nav frames replayed from disk), assert the FDR ends ≤64 GB and no payload class is silently dropped without a logged rollover event.
### AC-NEW-4 — False-position safety budget
**Statement.**
- P(reported estimate error > **500 m**) **<0.1 %** per flight.
- P(reported estimate error > **1 km**) **<0.01 %** per flight.
**Why it matters.** A single 1-km-off `GPS_INPUT` frame can hand the FC a heading that flies the UAV outside the geofence in seconds. The covariance carried in `GPS_INPUT` (`h_acc`) is the FC's only defense; this AC bounds the **probability** of our covariance under-reporting reality.
**Implementation drivers.** EKF covariance must be calibrated, not optimistic. Cross-view fixes with low inlier ratio must be **rejected**, not down-weighted to "small but non-zero". Outlier rejection at the EKF stage (Mahalanobis gate) is mandatory.
**Validation.** Monte Carlo over the AerialVL public dataset (S03) and our own recorded Mavic flights, with synthetic IMU injection where applicable; report error CDF; pass = both probabilities below budget across ≥100 simulated flights worth of frames.
### AC-NEW-5 — Operational environmental envelope
**Statement.** Operating temperature **20 °C to +50 °C**; vibration / shock per RTCA DO-160G low-altitude UAV-class envelope. The cooling solution shall sustain the **25 W** power mode at the upper temperature bound for the full **8-hour duty cycle** without thermal throttling.
**Why it matters.** Without this, all latency / accuracy ACs are conditional on a benign thermal day. Eastern/southern Ukraine summers easily exceed +35 °C ambient inside a UAV bay; without active cooling, the Jetson throttles to 15 W mode and our 400 ms latency budget collapses.
**Implementation drivers.** Forced-air or active heatsink sized for 25 W continuous at +50 °C ambient bay temperature; thermal sensors logged in FDR (AC-NEW-3); throttle event = automatic `STATUSTEXT` warning to QGC.
**Validation.** Hot-soak chamber test: 25 W workload at +50 °C ambient for 8 h; assert no throttle. Cold-soak: 20 °C cold-start to first fix within AC-NEW-1 budget.
### AC-NEW-6 — Imagery freshness enforcement
**Statement.** The system shall reject (or downgrade confidence on) any satellite tile whose capture date violates AC-8.2 (>6 months old in active-conflict sectors; >12 months old in stable rear sectors). Tiles generated mid-flight (AC-8.4) and not yet uploaded to the Suite Satellite Service are timestamped with the current flight date and treated as fresh.
**Why it matters.** Stale satellite tiles are the dominant cross-view-matching failure mode in active-conflict sectors (cratering, dam destruction, road realignment). A confident match against a stale tile is worse than no match.
**Implementation drivers.** Each tile carries `capture_date` metadata in the cache index. Sector classification (active vs stable) is part of the operational area definition handed in pre-flight. Confidence weight = 1.0 if within freshness budget, linearly decayed to 0.0 over a 30-day grace zone past the budget, hard reject beyond the grace.
**Validation.** Inject tiles with synthetic age into the cache; verify rejection / decay curve matches spec; verify a stale-tile match never produces a `satellite_anchored` source label.
### AC-NEW-7 — Cache-poisoning safety budget
**Statement.** Per flight, across all onboard tiles written by Component 1b (in-flight ortho-tile generator):
- P(onboard tile geo-misaligned > **30 m**) **<1 %**.
- P(onboard tile geo-misaligned > **100 m**) **<0.1 %**.
**Why it matters.** Onboard tiles feed back into the Suite Satellite Service's basemap (AC-8.4). Without this AC, a confidently-bad EKF pose can write a misaligned tile that, after Service ingest, becomes the next flight's satellite anchor — producing cross-flight error compounding that AC-NEW-4 (single-flight false-position budget) does not capture. This AC bounds the **probability** that an onboard tile's claimed geo-alignment is wrong by a margin that would propagate to a downstream flight.
**Implementation drivers.**
- Service-source tiles are immutable within freshness budget (AC-8.2); onboard tiles overwrite only stale or other-onboard tiles.
- The onboard GPS-Denied system writes tile-quality metadata required by the Suite Satellite Service. The Service-side ingest applies a **2-flight voting layer**: an onboard tile gets promoted to "trusted basemap" only after **N>=2 independent flights** confirm consistent geo-alignment within X m of each other. (Active sectors per AC-NEW-6 may use single-flight promotion when σ_xy <= 3 m AND OSM-road-overlap >= 70 %.) The voting layer is an external Suite Satellite Service dependency, not implemented inside this onboard build, but its contract is required for AC-NEW-7 to pass end-to-end.
- The Component-1b parent-pose covariance is a **hard gate** in the local quality score: σ_xy ≤ 5 m for a hard write (`trust_level = candidate`); σ_xy ≤ 3 m for `trust_level = candidate` with full quality; tiles written in the σ_xy ∈ (3, 5] m band are marked `trust_level = soft` in the sidecar.
- Eligibility check (Component 1b) tightens generation gate from σ_xy ≤ 10 m to σ_xy ≤ 5 m.
**Validation.** Multi-flight Monte Carlo replay over AerialVL + Mavic + AerialExtreMatch with **synthetic over-confidence injection** (artificially deflate EKF covariance by 1.5×–3×): assert both probabilities below budget across ≥100 simulated flights worth of frames. Independently, Service-side voting layer is exercised in F-T3 to verify candidate tiles are not promoted to trusted basemap before N-flight confirmation.
### AC-NEW-8 — Visual blackout + GPS spoofing degraded-mode budget
**Statement.** When the navigation camera is fully unusable for visual localization and the flight controller simultaneously reports GPS denial/spoofing, the onboard system shall:
- continue emitting `GPS_INPUT` from IMU-only propagation for **up to 30 s** after the last trusted visual/satellite anchor, unless the estimator covariance exceeds the fail threshold earlier;
- label every estimate `{dead_reckoned}` and set `fix_type=2` or lower when the 95% covariance semi-major axis exceeds **100 m**;
- emit `fix_type=0`, `horiz_accuracy=999.0`, and `STATUSTEXT: VISUAL_BLACKOUT_FAILSAFE` when the 95% covariance semi-major axis exceeds **500 m** OR visual blackout exceeds **30 s** without a trusted re-anchor;
- never promote spoofed real-GPS measurements back into the estimator during blackout unless the FC GPS health has been stable and non-spoofed for **≥10 s** and a visual/satellite consistency check has succeeded.
**Why it matters.** A cloud/whiteout period removes all visual correction exactly when spoofed GPS cannot be trusted. The only safe behavior is honest IMU-only dead reckoning with rapidly growing uncertainty, not pretending that a stale visual position or spoofed GPS remains valid.
**Implementation drivers.** Add an image-quality/occlusion classifier before VO/VPR, a blackout state in the ESKF mode machine, covariance floors for IMU-only propagation, strict GPS health gating, and QGC/FDR logging for blackout start, every degraded estimate, and blackout recovery/failsafe.
**Validation.** SITL/replay: inject a 5 s, 15 s, and 35 s full-camera blackout while spoofing `GPS_RAW_INT`; assert mode transition ≤400 ms, spoofed GPS is ignored, covariance grows monotonically, `GPS_INPUT` fields degrade at the thresholds above, and recovery only occurs after a trusted visual/satellite anchor or the 10 s GPS-health + visual-consistency gate.
@@ -1,8 +1,2 @@
- Height
- 400m
- Camera:
- Name: ADTi Surveyor Lite 26S v2
- Resolution: 26MP
- Image resolution: 6252*4168
- Focal length: 25mm
- Sensor width: 23.5
- Height: 400m
- Camera: ADTi Surveyor Lite 20MP 20L V1
@@ -1,166 +1,97 @@
# Expected Results
# Expected Results Mapping
Maps every input data item to its quantifiable expected result.
Tests use this mapping to compare actual system output against known-correct answers.
## Scope
## Result Format Legend
`coordinates.csv` is the current source of truth for the provided still-image nadir set. It gives expected WGS84 frame-center coordinates for `AD000001.jpg` through `AD000060.jpg`.
| Result Type | When to Use | Example |
|-------------|-------------|---------|
| Exact value | Output must match precisely | `fix_type: 3`, `satellites_visible: 10` |
| Tolerance range | Numeric output with acceptable variance | `lat: 48.275292 ± 50m` |
| Threshold | Output must exceed or stay below a limit | `latency < 400ms`, `memory < 8GB` |
| Pattern match | Output must match a string/regex pattern | `RELOC_REQ: last_lat=.* last_lon=.* uncertainty=.*m` |
| File reference | Complex output compared against a reference file | `match expected_results/position_accuracy.csv` |
| Set/count | Output must contain specific items or counts | `registered_frames / total_frames > 0.95` |
This data is sufficient for black-box frame-center geolocation tests against still images. The Derkachi representative fixture in `input_data/flight_derkachi/` adds cropped nadir video plus synchronized `SCALED_IMU2` and `GLOBAL_POSITION_INT` telemetry. It is sufficient for fixture validation, video/telemetry synchronization, replay, latency, VIO smoke tests, and trajectory comparison against the tlog GPS path. It is not sufficient by itself for final production accuracy because raw camera calibration, lens distortion, and exact camera-to-body calibration are still pending.
## Comparison Methods
## Pass / Fail Rules
| Method | Description | Tolerance Syntax |
|--------|-------------|-----------------|
| `numeric_tolerance` | abs(actual - expected) ≤ tolerance | `± <value>` |
| `threshold_min` | actual ≥ threshold | `≥ <value>` |
| `threshold_max` | actual ≤ threshold | `≤ <value>` |
| `percentage` | percentage of items meeting criterion | `≥ N%` |
| `exact` | actual == expected | N/A |
| `regex` | actual matches regex pattern | regex string |
| `file_reference` | compare against reference file | file path |
- **Normal frame-center geolocation**: estimated frame center is within 50 m of the expected WGS84 coordinate.
- **Stretch accuracy bin**: estimated frame center is within 20 m of the expected WGS84 coordinate.
- **Dataset aggregate**: at least 80% of mapped images pass the 50 m threshold and at least 50% pass the 20 m threshold.
- **Output shape**: each result must include image name, estimated `lat`, estimated `lon`, error in meters, source label, 95% covariance semi-major axis, and `last_satellite_anchor_age_ms`.
## Input Expected Result Mapping
## Input To Expected Output Map
### Position Accuracy (60-image flight sequence)
### Still-Image Frame Centers
Ground truth GPS coordinates for each frame are in `coordinates.csv`. The system processes these frames sequentially (simulating a real flight) with corresponding IMU data (200Hz, from SITL ArduPilot or synthetic generation from trajectory) and satellite tile matches. The system outputs estimated GPS coordinates per frame. Expected results compare estimated positions against ground truth.
| Input image | Expected latitude | Expected longitude | Primary threshold | Stretch threshold |
|-------------|-------------------|--------------------|-------------------|-------------------|
| AD000001.jpg | 48.275292 | 37.385220 | <= 50 m | <= 20 m |
| AD000002.jpg | 48.275001 | 37.382922 | <= 50 m | <= 20 m |
| AD000003.jpg | 48.274520 | 37.381657 | <= 50 m | <= 20 m |
| AD000004.jpg | 48.274956 | 37.379004 | <= 50 m | <= 20 m |
| AD000005.jpg | 48.273997 | 37.379828 | <= 50 m | <= 20 m |
| AD000006.jpg | 48.272538 | 37.380294 | <= 50 m | <= 20 m |
| AD000007.jpg | 48.272408 | 37.379153 | <= 50 m | <= 20 m |
| AD000008.jpg | 48.271992 | 37.377572 | <= 50 m | <= 20 m |
| AD000009.jpg | 48.271376 | 37.376671 | <= 50 m | <= 20 m |
| AD000010.jpg | 48.271233 | 37.374806 | <= 50 m | <= 20 m |
| AD000011.jpg | 48.270334 | 37.374442 | <= 50 m | <= 20 m |
| AD000012.jpg | 48.269922 | 37.373284 | <= 50 m | <= 20 m |
| AD000013.jpg | 48.269366 | 37.372134 | <= 50 m | <= 20 m |
| AD000014.jpg | 48.268759 | 37.370940 | <= 50 m | <= 20 m |
| AD000015.jpg | 48.268291 | 37.369815 | <= 50 m | <= 20 m |
| AD000016.jpg | 48.267719 | 37.368469 | <= 50 m | <= 20 m |
| AD000017.jpg | 48.267461 | 37.367255 | <= 50 m | <= 20 m |
| AD000018.jpg | 48.266663 | 37.365888 | <= 50 m | <= 20 m |
| AD000019.jpg | 48.266135 | 37.365460 | <= 50 m | <= 20 m |
| AD000020.jpg | 48.265574 | 37.364211 | <= 50 m | <= 20 m |
| AD000021.jpg | 48.264892 | 37.362998 | <= 50 m | <= 20 m |
| AD000022.jpg | 48.264393 | 37.361086 | <= 50 m | <= 20 m |
| AD000023.jpg | 48.263803 | 37.361028 | <= 50 m | <= 20 m |
| AD000024.jpg | 48.263014 | 37.359878 | <= 50 m | <= 20 m |
| AD000025.jpg | 48.262635 | 37.358277 | <= 50 m | <= 20 m |
| AD000026.jpg | 48.261819 | 37.357116 | <= 50 m | <= 20 m |
| AD000027.jpg | 48.261182 | 37.355907 | <= 50 m | <= 20 m |
| AD000028.jpg | 48.260727 | 37.354723 | <= 50 m | <= 20 m |
| AD000029.jpg | 48.260117 | 37.353469 | <= 50 m | <= 20 m |
| AD000030.jpg | 48.259677 | 37.352165 | <= 50 m | <= 20 m |
| AD000031.jpg | 48.258881 | 37.351376 | <= 50 m | <= 20 m |
| AD000032.jpg | 48.258425 | 37.349964 | <= 50 m | <= 20 m |
| AD000033.jpg | 48.258653 | 37.347004 | <= 50 m | <= 20 m |
| AD000034.jpg | 48.257879 | 37.347711 | <= 50 m | <= 20 m |
| AD000035.jpg | 48.256777 | 37.348444 | <= 50 m | <= 20 m |
| AD000036.jpg | 48.255756 | 37.348098 | <= 50 m | <= 20 m |
| AD000037.jpg | 48.255375 | 37.346549 | <= 50 m | <= 20 m |
| AD000038.jpg | 48.254799 | 37.345603 | <= 50 m | <= 20 m |
| AD000039.jpg | 48.254557 | 37.344566 | <= 50 m | <= 20 m |
| AD000040.jpg | 48.254380 | 37.344375 | <= 50 m | <= 20 m |
| AD000041.jpg | 48.253722 | 37.343093 | <= 50 m | <= 20 m |
| AD000042.jpg | 48.254205 | 37.340532 | <= 50 m | <= 20 m |
| AD000043.jpg | 48.252380 | 37.342112 | <= 50 m | <= 20 m |
| AD000044.jpg | 48.251489 | 37.343079 | <= 50 m | <= 20 m |
| AD000045.jpg | 48.251085 | 37.346128 | <= 50 m | <= 20 m |
| AD000046.jpg | 48.250413 | 37.344034 | <= 50 m | <= 20 m |
| AD000047.jpg | 48.249414 | 37.343296 | <= 50 m | <= 20 m |
| AD000048.jpg | 48.249114 | 37.346895 | <= 50 m | <= 20 m |
| AD000049.jpg | 48.250241 | 37.347741 | <= 50 m | <= 20 m |
| AD000050.jpg | 48.250974 | 37.348379 | <= 50 m | <= 20 m |
| AD000051.jpg | 48.251528 | 37.349468 | <= 50 m | <= 20 m |
| AD000052.jpg | 48.251873 | 37.350485 | <= 50 m | <= 20 m |
| AD000053.jpg | 48.252161 | 37.351491 | <= 50 m | <= 20 m |
| AD000054.jpg | 48.252685 | 37.352343 | <= 50 m | <= 20 m |
| AD000055.jpg | 48.253268 | 37.353119 | <= 50 m | <= 20 m |
| AD000056.jpg | 48.253767 | 37.354246 | <= 50 m | <= 20 m |
| AD000057.jpg | 48.254329 | 37.354946 | <= 50 m | <= 20 m |
| AD000058.jpg | 48.254874 | 37.355765 | <= 50 m | <= 20 m |
| AD000059.jpg | 48.255481 | 37.356501 | <= 50 m | <= 20 m |
| AD000060.jpg | 48.256246 | 37.357485 | <= 50 m | <= 20 m |
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 1 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | ≥ 80% of frames have position error < 50m from ground truth | percentage | ≥ 80% of frames within 50m | `expected_results/position_accuracy.csv` |
| 2 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | ≥ 60% of frames have position error < 20m from ground truth | percentage | ≥ 60% of frames within 20m | `expected_results/position_accuracy.csv` |
| 3 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | Per-frame position output in WGS84 (lat, lon) | numeric_tolerance | each frame ± 100m max (no single frame exceeds 100m error) | `expected_results/position_accuracy.csv` |
| 4 | `coordinates.csv` (all 60 frames) | Sequential flight images with ground truth GPS | Cumulative VO drift between satellite anchors < 100m | threshold_max | ≤ 100m drift between anchors | N/A |
### Representative Derkachi Video/IMU Fixture
### GPS_INPUT Message Correctness
| Input fixture | Expected validation result | Threshold |
|---------------|----------------------------|-----------|
| `flight_derkachi/data_imu.csv` | Telemetry CSV has required `timestamp(ms)`, `Time`, `SCALED_IMU2.*`, and `GLOBAL_POSITION_INT.*` columns; non-empty rows are monotonic from `Time=0.0` to `489.9` | 0 missing required columns; 0 decreasing timestamps; 4,900 nonblank rows |
| `flight_derkachi/flight_derkachi.mp4` | Video stream is readable as cropped nadir footage for replay | H.264, 880 x 720, 30 fps, approximately 490.07 s |
| Video/telemetry alignment | Video has 14,700 frames and telemetry has 4,900 rows | Exactly 3 video frames per telemetry row; duration delta <=250 ms |
| Derkachi trajectory comparison | Replay output can be compared to `GLOBAL_POSITION_INT.lat`, `GLOBAL_POSITION_INT.lon`, `GLOBAL_POSITION_INT.alt`, `GLOBAL_POSITION_INT.relative_alt`, velocity, and heading | Thresholds are calibration-gated; use for smoke/relative trajectory validation until intrinsics and camera-to-body calibration are pinned |
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 5 | Single frame + IMU data | Normal tracking frame with recent satellite match | `fix_type: 3`, `horiz_accuracy: 5-20m`, `satellites_visible: 10`, lat/lon populated | exact (fix_type, sat), numeric_tolerance (accuracy) | fix_type == 3, horiz_accuracy ∈ [1, 50] | N/A |
| 6 | Frame sequence, no satellite match for >30s | VO-only tracking, no recent satellite anchor | `fix_type: 3`, `horiz_accuracy: 20-50m` | exact (fix_type), range (accuracy) | fix_type == 3, horiz_accuracy ∈ [20, 100] | N/A |
| 7 | Frame sequence, VO lost + no satellite | IMU-only dead reckoning | `fix_type: 2`, `horiz_accuracy: 50-200m+` (growing over time) | exact (fix_type), threshold_min (accuracy) | fix_type == 2, horiz_accuracy ≥ 50 | N/A |
| 8 | VO lost + 3 consecutive satellite failures | Total position failure | `fix_type: 0`, `horiz_accuracy: 999.0` | exact | fix_type == 0, horiz_accuracy == 999.0 | N/A |
| 9 | Any valid frame | GPS_INPUT output rate | GPS_INPUT messages at 5-10Hz continuous | range | 5 ≤ rate_hz ≤ 10 | N/A |
## Known Gaps
### Confidence Tier Transitions
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 10 | Frame with satellite match <30s ago, covariance <400m² | HIGH confidence conditions | Confidence tier: HIGH, SSE confidence: "HIGH" | exact | N/A | N/A |
| 11 | Frame with cuVSLAM OK, no satellite match >30s | MEDIUM confidence conditions | Confidence tier: MEDIUM, SSE confidence: "MEDIUM" | exact | N/A | N/A |
| 12 | Frame with cuVSLAM lost, IMU-only | LOW confidence conditions | Confidence tier: LOW, SSE confidence: "LOW" | exact | N/A | N/A |
| 13 | 3+ consecutive total failures | FAILED conditions | Confidence tier: FAILED, SSE confidence: "FAILED", fix_type: 0 | exact | N/A | N/A |
### Image Registration & Visual Odometry
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 14 | 60 sequential flight images | Normal flight (no sharp turns) | Image registration rate ≥ 95% (≥ 57 of 60 registered) | percentage | ≥ 95% | N/A |
| 15 | 60 sequential flight images | Normal flight images | Mean reprojection error < 1.0 pixels | threshold_max | MRE < 1.0 px | N/A |
### Disconnected Route Segments & Sharp Turns
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 16 | Frames 32-43 from coordinates.csv | Trajectory with direction change (turn area) | System continues producing position estimates through the turn | threshold_min | ≥ 1 position output per frame | N/A |
| 17 | Simulated consecutive frames with 350m gap | Outlier between 2 consecutive photos due to tilt | System handles outlier, position estimate not corrupted (error < 100m for next valid frame) | threshold_max | ≤ 100m error after recovery | N/A |
| 18 | Simulated sharp turn (no overlap, <5% overlap, <70° angle, <200m drift) | Sharp turn where VO fails | Satellite re-localization triggers, position recovered within 3 frames after turn | threshold_max | position error ≤ 50m after re-localization | N/A |
| 19 | Simulated VO loss + satellite match success | Tracking loss → re-localization | cuVSLAM restarts, ESKF position corrected, tracking_state returns to NORMAL | exact | tracking_state == NORMAL after recovery | N/A |
### 3-Consecutive-Failure Re-Localization
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 20 | Simulated VO loss + 3 satellite match failures | Cannot determine position by any means | Re-localization request sent: `RELOC_REQ: last_lat=.* last_lon=.* uncertainty=.*m` | regex | message matches pattern | N/A |
| 21 | Re-localization request active | System waiting for operator | GPS_INPUT fix_type=0, system continues IMU prediction, continues satellite matching attempts | exact (fix_type) | fix_type == 0 | N/A |
| 22 | Operator sends approximate coordinates (lat, lon) | Operator re-localization hint | System uses hint as ESKF measurement (high covariance ~500m), attempts satellite match in new area | threshold_max | position error ≤ 500m initially, ≤ 50m after satellite match | N/A |
### Startup & Handoff
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 23 | System boot with GLOBAL_POSITION_INT available | Normal startup | System reads initial position, initializes ESKF, starts GPS_INPUT output | exact | GPS_INPUT output begins within 60s of boot | N/A |
| 24 | System boot + first satellite match | Startup validation | First satellite match validates initial position, position error drops | threshold_max | position error ≤ 50m after first satellite match | N/A |
### Mid-Flight Reboot Recovery
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 25 | System process killed mid-flight | Companion computer reboot | System recovers: reads FC position, inits ESKF with high uncertainty, loads TRT engines, starts cuVSLAM, performs satellite match | threshold_max | total recovery time ≤ 70s | N/A |
| 26 | Post-reboot first satellite match | Recovery validation | Position accuracy restored after first satellite match | threshold_max | position error ≤ 50m after first satellite match | N/A |
### Object Localization
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 27 | POST /objects/locate with pixel_x, pixel_y, gimbal angles, zoom, known UAV position | Object at known ground GPS | Response: `{ lat, lon, alt, accuracy_m, confidence }` with lat/lon matching ground truth | numeric_tolerance | lat/lon within accuracy_m of ground truth (consistent with frame-center accuracy) | N/A |
| 28 | POST /objects/locate with invalid pixel coordinates | Out-of-frame pixel | HTTP 422 or error response indicating invalid input | exact | HTTP status 422 | N/A |
### Coordinate Transform Chain
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 29 | Known GPS → NED → pixel → GPS round-trip | Coordinate transform validation | Round-trip error < 0.1m | threshold_max | ≤ 0.1m | N/A |
### API & Communication
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 30 | GET /health | Health check endpoint | HTTP 200, JSON with memory_mb, gpu_temp_c, status fields | exact (status code), regex (body) | status == 200, body contains `"status"` | N/A |
| 31 | POST /sessions | Start session | HTTP 200/201 with session ID | exact | status ∈ {200, 201} | N/A |
| 32 | GET /sessions/{id}/stream | SSE position stream | SSE events at ~1Hz with fields: type, timestamp, lat, lon, alt, accuracy_h, confidence, vo_status | regex | each event matches SSE schema | N/A |
| 33 | Unauthenticated request to /sessions | No JWT token | HTTP 401 Unauthorized | exact | status == 401 | N/A |
### Performance Thresholds
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 34 | Single camera frame (6252x4168) | End-to-end processing time | Total pipeline latency < 400ms (capture → GPS coordinate output) | threshold_max | ≤ 400ms | N/A |
| 35 | 30-minute sustained operation | Memory usage over time | Peak memory < 8GB, no memory leaks (growth < 50MB over 30min) | threshold_max | peak < 8192MB, growth ≤ 50MB | N/A |
| 36 | 30-minute sustained operation | GPU thermal | SoC junction temperature stays below 80°C (no throttling) | threshold_max | ≤ 80°C | N/A |
| 37 | cuVSLAM single frame | VO processing time | cuVSLAM inference ≤ 20ms per frame | threshold_max | ≤ 20ms | N/A |
| 38 | Satellite matching single frame | Satellite matching time (async) | LiteSAM/XFeat inference ≤ 330ms | threshold_max | ≤ 330ms | N/A |
| 39 | TRT engine load | Engine initialization time | All TRT engines loaded within 10s total | threshold_max | ≤ 10s | N/A |
### Satellite Tile Management
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 40 | Mission area definition (200km path, ±2km buffer, zoom 18) | Tile storage calculation | Total storage 500-800MB for zoom 18 + zoom 19 flight path | range | [300MB, 1000MB] | N/A |
| 41 | ESKF position ± 3σ search radius | Tile selection | Tiles covering search area loaded, mosaic assembled, covers at least 500m radius | threshold_min | coverage radius ≥ 500m | N/A |
### TRT Engine Validation
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 42 | LiteSAM PyTorch model → ONNX → TRT FP16 | TRT engine conversion | Engine builds successfully on Jetson Orin Nano Super | exact | exit_code == 0 | N/A |
| 43 | TRT engine output vs PyTorch reference (same input) | Inference correctness | Max L1 error between TRT and PyTorch output < 0.01 | threshold_max | L1_max < 0.01 | N/A |
| 44 | LiteSAM MinGRU operations | TRT compatibility check | All MinGRU ops supported in TRT 10.3 (polygraphy inspect) | exact | unsupported_ops == 0 | N/A |
### Telemetry
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| 45 | Normal operation | Telemetry output rate | NAMED_VALUE_FLOAT messages at 1Hz (gps_conf, gps_drift, gps_hacc) | numeric_tolerance | rate: 1Hz ± 0.2Hz | N/A |
| 46 | VO tracking lost + 3 satellite failures | Re-localization telemetry | STATUSTEXT with RELOC_REQ sent to ground station | regex | message matches `RELOC_REQ:.*` | N/A |
## Expected Result Reference Files
### position_accuracy.csv
Reference file: `expected_results/position_accuracy.csv`
Contains the ground truth GPS coordinate for each frame in the 60-image test sequence (copied from `coordinates.csv`) plus the acceptance thresholds. Test harness computes haversine distance between estimated and ground truth positions, then applies aggregate criteria.
Thresholds applied to the full 60-frame sequence:
- ≥ 80% of frames: error < 50m
- ≥ 60% of frames: error < 20m
- 0% of frames: error > 100m (no single frame exceeds 100m)
- Cumulative VO drift between satellite anchors: < 100m
- The still-image set has expected WGS84 centers but no synchronized IMU, attitude, airspeed, altitude, or timestamp stream.
- The Derkachi fixture has synchronized video, IMU, and GPS trajectory, but no raw camera calibration, lens distortion, exact camera-to-body transform, attitude, or airspeed columns.
- The still-image sample cadence is slower than the target 3 fps runtime profile; the Derkachi video is 30 fps and must be sampled to target replay cadence for runtime tests.
- Final production acceptance requires camera calibration and representative datasets with synchronized camera/IMU plus ground-truth trajectory.
@@ -0,0 +1,14 @@
# Derkachi Representative Flight Fixture
## Files
| File | Description | Observed Metadata |
|------|-------------|-------------------|
| `flight_derkachi.mp4` | Cropped nadir flight footage for replay | H.264, 880 x 720, 30 fps, about 490.07 s |
| `data_imu.csv` | Flight-controller telemetry trace exported from the tlog | 4,900 rows at 10 Hz from `Time=0.0` to `489.9`; includes `SCALED_IMU2` and `GLOBAL_POSITION_INT` trajectory fields |
## Test Use
Use this fixture for video/telemetry synchronization checks, representative replay smoke tests, VIO hot-path latency, frame-drop accounting, and trajectory comparison against `GLOBAL_POSITION_INT`. The video and telemetry align at exactly three video frames per telemetry row. Camera intrinsics, lens distortion, raw camera resolution, and exact camera-to-body calibration are still unknown, so this fixture is not sufficient by itself for final production camera calibration or satellite-anchor accuracy claims.
For the test recording, the rotating camera was mechanically fixed in a downward/nadir orientation. Treat the MP4 as a cleaned/cropped replay fixture rather than the raw camera feed.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9acb97042fc648301d73d3c0fe7d80f7e3e2697000c0d33afa8a7b7a74a20005
size 282207328
+2 -2
View File
@@ -1,2 +1,2 @@
We have a wing-type UAV with a camera pointing downwards that can take photos 3 times per second with a resolution 6200*4100. Also plane has flight controller with IMU. During the plane flight, we know GPS coordinates initially. During the flight, GPS could be disabled or spoofed. We need to determine the GPS of the centers of the next frame from the camera. And also the coordinates of the center of any object in these photos. We can use an external satellite provider for ground checks on the existing photos. So, before the flight, UAV's operator should upload the satellite photos to the plane's companion PC.
The real world examples are in input_data folder, but the distance between each photo is way bigger than it will be from a real plane. On that particular example photos were taken 1 photo per 2-3 seconds. But in real-world scenario frames would appear within the interval no more than 500ms or even 400 ms.
We have a wing-type UAV with a fixed downward navigation camera that can take photos 3 times per second. The authoritative navigation-camera spec is defined in `restrictions.md` as the ADTi 20MP 20L V1, APS-C sensor, about 5472 x 3648 px; older higher-resolution references are superseded. Also plane has flight controller with IMU. During the plane flight, we know GPS coordinates initially. During the flight, GPS could be disabled or spoofed. We need to determine the GPS of the centers of the next frame from the camera. And also the coordinates of the center of any object in these photos. We can use an external satellite provider for ground checks on the existing photos. So, before the flight, UAV's operator should upload the satellite photos to the plane's companion PC.
The real world examples are in input_data folder, but the original still-image set has a much larger distance between photos than the target aircraft will have. On that particular example photos were taken 1 photo per 2-3 seconds. But in real-world scenario frames would appear within the interval no more than 500ms. Additional representative data is available in `input_data/flight_derkachi/`: cropped nadir flight footage plus synchronized `SCALED_IMU2` and `GLOBAL_POSITION_INT` telemetry. This supports video/telemetry synchronization, replay, latency, VIO smoke tests, and trajectory comparison against the tlog GPS path. Camera intrinsics, lens distortion, raw camera feed parameters, and exact camera-to-body calibration are still pending, so final production accuracy claims remain gated on calibration data or a separately surveyed representative dataset.
+50 -28
View File
@@ -1,37 +1,59 @@
# UAV & Flight
# Restrictions
- Photos are taken by only airplane (fixed-wing) type UAVs
- Photos are taken by the camera pointing downwards and fixed, but it is not autostabilized
- The flying range is restricted by the eastern and southern parts of Ukraine (to the left of the Dnipro River)
- Altitude is predefined and no more than 1km. The height of the terrain can be neglected
- Flights are done mostly in sunny weather
- During the flight, UAVs can make sharp turns, so that the next photo may be absolutely different from the previous one (no same objects), but it is rather an exception than the rule
- Number of photos per flight could be up to 3000, usually in the 500-1500 range
> **Last revised**: 2026-05-01 (post Phase 1 AC/restrictions assessment clarifications).
# Cameras
## UAV & Flight
- UAV has two cameras:
1. **Navigation camera** — fixed, pointing downwards, not autostabilized. Used by GPS-Denied system for position estimation
2. **AI camera** — main camera with configurable angle and zoom, used by onboard AI detection systems
- Navigation camera resolution: FullHD to 6252*4168. Camera parameters are known: focal length, sensor width, resolution, etc.
- Cameras are connected to the companion computer (interface TBD: USB, CSI, or GigE)
- Terrain is assumed flat (eastern/southern Ukraine operational area); height differences are negligible
- Photos are taken by airplane (fixed-wing) type UAVs only.
- Photos are taken by the navigation camera pointing downwards and fixed (not gimbal-stabilized).
- Operational area is the eastern and southern parts of Ukraine (east/left of the Dnipro River).
- Mission profile: 8-hour flights at ~60 km/h cruise. Two route shapes coexist:
- **Sector**: up to **10 × 15 km = 150 km²** of dense coverage.
- **Transit corridor**: ~**50 km × 1 km = 50 km²** strip in/out of the sector.
- **Total operational area: up to ~400 km²** of pre-cached satellite imagery per mission. Cache is **persistent across flights** (not redownloaded each mission). Storage budget **~10 GB** for the satellite tile cache; see AC-NEW-3 for flight-data-recorder budget.
- Altitude: pre-defined, **≤1 km AGL**. Terrain is assumed flat (operational area is rolling steppe / agricultural land); height differences are negligible.
- Weather: predominantly sunny daytime operations. Validation must still cover the seasonal/visibility classes that affect visual matching in the operational area: summer crop/field patterns, autumn/winter bare fields, cloud/smoke/haze, snow if missions can occur in winter, and low-texture agricultural repetition.
- Sharp turns occur but are the exception, not the rule. Two consecutive photos may share <5% overlap during a turn (see AC-3.2).
- **No photo-count cap.** The previously stated "up to 3000 photos per flight" was a legacy operator number from a Mavic-class workflow; it is dropped because (a) it is inconsistent with 8 h × 3 fps, and (b) the system does **not store raw photos at all** (see AC-8.5). Storage is bounded by the tile-cache + FDR caps (~10 GB persistent + 64 GB / flight, AC-NEW-3).
# Satellite Imagery
## Cameras
- We can use satellite providers, but we're limited right now to Google Maps, which could be outdated for some regions
- Satellite imagery for the operational area must be pre-loaded onto the companion computer before flight
- The UAV carries **two cameras**:
1. **Navigation camera** — fixed, downward-pointing, not autostabilized. Consumed by the GPS-Denied system for position estimation.
2. **AI camera** — main mission camera with operator-controllable gimbal angle and zoom. Consumed by onboard AI detection systems.
- **Navigation camera**: **ADTi 20MP 20L V1, APS-C sensor, ~5472 × 3648 px (≈20 MP)**. APS-C sensor (~23.6 × 15.7 mm). Lens TBD — selected during solution-draft phase to land GSD in the **1020 cm/px band at 1 km AGL** (drives a frame ground footprint of ~470 m × 314 m to ~980 m × 655 m depending on focal length). Other intrinsics (focal length, exact sensor dimensions, distortion coefficients) are pinned at module-selection time and used by Component-1b orthorectification (pre-flight checkerboard calibration, F-F2).
- **AI camera pose information available to the GPS-Denied system**: gimbal angle and zoom only. The UAV's instantaneous bank/pitch is **not** published from the autopilot to the AI-camera reasoning path. Object-localization accuracy is therefore scoped to level flight (AC-7.1).
- Cameras connect to the companion computer over USB, MIPI-CSI, or GigE (specific interface TBD at solution-draft phase, dependent on chosen module).
# Onboard Hardware
## Satellite Imagery
- Processing is done on a Jetson Orin Nano Super (67 TOPS, 8GB shared LPDDR5, 25W TDP)
- The companion computer runs JetPack (Ubuntu-based) with CUDA/TensorRT available
- Onboard storage for satellite imagery is limited (exact capacity TBD, but must be accounted for in tile preparation)
- Sustained GPU load may cause thermal throttling; the processing pipeline must stay within thermal envelope
- **Source: Azaion Suite Satellite Service** (a separate component of the wider Suite). The onboard system is a **consumer** of this service, not a direct customer of commercial providers. Upstream sourcing (Maxar / Airbus / partner agencies / commissioned tasking) is the Satellite Service's concern, not this build's.
- **Onboard interface to the Service is offline-only**: the companion computer holds a local cache populated **before flight** by syncing from the Service for the operational area (AC-8.3). No satellite imagery is fetched in-flight from the Service.
- **Mid-flight tile generation (AC-8.4)**: during the mission the companion computer generates fresh tiles from the navigation camera, orthorectified into the basemap projection, deduplicated against the existing cache, and stored locally. On landing, those new tiles are uploaded back to the Suite Satellite Service for ingestion, so the next mission's cache is refreshed by the previous flight.
- **No raw photo storage** (AC-8.5): the tile is the unit of persistence. Raw nav-camera and AI-camera frames are not retained (except a low-rate failure-thumbnail log for forensics).
- **Resolution at the cache interface**: 0.5 m/pixel minimum, 0.3 m/pixel ideal (AC-8.1). The architecture is provider-agnostic at the cache boundary; whatever the Suite Satellite Service supplies must meet that bar.
- **Storage tile resolution convention**: cache imagery is specified by source pixel size, not by assuming a universal zoom-to-meter mapping. The cache interface accepts **0.5 m/px minimum, 0.3 m/px ideal** imagery, and every tile manifest records CRS, tile matrix convention, tile dimension, latitude-adjusted meters-per-pixel, capture date, source, and compression. If an XYZ/WebMercator tile pyramid is used, its zoom level is documented as a provider convention rather than treated as proof of physical resolution. The matcher (Component 3) needs <=~4x scale ratio between the UAV frame (~12 cm/px GSD at 1 km AGL with the 20 MP APS-C camera) and the reference; 0.3-0.5 m/px reference imagery gives a ~2.5-4.2x ratio. Storage budget across the 400 km² operational area remains capped at **10 GB** for the persistent cache and must be validated against the final provider format/compression. The 10 GB budget includes cache imagery, manifests, overviews, and any precomputed global/local descriptors unless the solution draft explicitly splits a separate descriptor/index budget. **VPR retrieval unit is decoupled from the storage tile** (see AC-8.6 below): VPR chunks are derived from the tile cache at ground-footprint scale (~600-800 m chunks with 40-50 % overlap), independent of the storage tile convention.
- **Freshness gates** (AC-8.2 / AC-NEW-6) are enforced at runtime: tiles older than 6 months (active-conflict sectors) or 12 months (stable rear sectors) are rejected or down-confidence-weighted. Tiles generated mid-flight are timestamped with the current flight date and treated as fresh.
- **Free public imagery (Sentinel-2 etc.)** is not on the runtime path. If the Suite Satellite Service ever returns Sentinel-class tiles, the cache rejects them as below the 0.5 m/px floor.
# Sensors & Integration
## Onboard Hardware
- There is a lot of data from IMU (via the flight controller)
- The system communicates with the flight controller via MAVLink protocol using MAVSDK library
- The system must output GPS coordinates to the flight controller as a replacement for the real GPS module (MAVLink GPS_INPUT message)
- Ground station telemetry link is available but bandwidth-limited; it is not the primary output channel
- Processing platform: **Jetson Orin Nano Super** — 67 TOPS sparse INT8, **8 GB shared LPDDR5** (CPU and GPU share the same memory pool), **25 W TDP**.
- Companion computer runs JetPack (Ubuntu-based) with CUDA / TensorRT available.
- Sustained GPU load may cause thermal throttling; the processing pipeline must stay within the thermal envelope. The cooling solution shall sustain the 25 W power mode for the 8-hour duty cycle at the upper environmental-envelope temperature (AC-NEW-5).
- Onboard non-volatile storage: budget at least the satellite-cache (~10 GB) **plus** the flight-data-recorder cap (64 GB / flight, AC-NEW-3). Reuse-across-flights tile cache stays resident; per-flight FDR rolls over after cap.
## Sensors & Integration
- High-rate **IMU** data is available from the flight controller via MAVLink.
- The original still-image sample does **not** include synchronized IMU or ground-truth pose. The Derkachi representative fixture adds cropped nadir video plus synchronized `SCALED_IMU2` and `GLOBAL_POSITION_INT` telemetry, which is enough for replay, synchronization, latency, VIO smoke tests, and trajectory comparison against the tlog GPS path. Final production acceptance still requires camera intrinsics, lens distortion, exact camera-to-body calibration, and representative synchronized navigation-camera frames, FC IMU/attitude/airspeed/altitude, emitted MAVLink messages, and ground-truth trajectory from a representative flight or replay rig.
- The system communicates with the flight controller via MAVLink. Telemetry plumbing uses **MAVSDK**; the `GPS_INPUT` injection path is implemented via **pymavlink**, since MAVSDK does not expose a native `GPS_INPUT` API.
- **Autopilot target: ArduPilot only** (with `GPS1_TYPE=14` for MAVLink GPS injection). PX4 is out of scope for the build; if it ever returns to scope it will use `VISION_POSITION_ESTIMATE`, not `GPS_INPUT`. (See `_docs/00_research/00_ac_assessment.md` Q-1.)
- The system outputs WGS84 GPS coordinates to the flight controller as a replacement for the real GPS module (MAVLink GPS_INPUT, AC-4.3).
- **Ground station: QGroundControl** is the supported GCS. Mission Planner is not in scope. Telemetry link is bandwidth-limited and is not the primary output channel; per-frame data stays on the local FDR (AC-NEW-3), GCS sees a 12 Hz downsampled summary (AC-6.1).
## Failsafe & Safety
- If the GPS-Denied system fails to produce any position estimate for **>3 s**, the autopilot falls back to IMU-only dead reckoning (AC-5.2). N=3 s rides through one sharp turn at cruise speed without tripping the failsafe.
- The system must satisfy the false-position safety budget in AC-NEW-4 (P(error >500 m) <0.1%, P(error >1 km) <0.01% per flight).
- Cold-start time-to-first-fix budget is **<30 s** from companion-computer boot (AC-NEW-1); spoofing-promotion latency is **<3 s** from FC's GPS-loss signal (AC-NEW-2).