mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 08:41:12 +00:00
Update autodev state documentation to reflect progress in the Decompose phase, changing the current step from 5 to 6. Revise sub-step details to indicate a shift to phase 2, focusing on module layout for the Satellite Service and Tile Manager, and awaiting confirmation before product task decomposition. Additionally, enhance problem documentation to clarify the original still-image sample limitations and introduce the Derkachi representative fixture for improved data validation. Update references to the Tile Manager and Satellite Service throughout the documentation for consistency.
This commit is contained in:
@@ -52,33 +52,38 @@
|
||||
|
||||
---
|
||||
|
||||
### FT-P-03: BASALT VIO Replay With Public Synchronized Data
|
||||
### FT-P-03: BASALT VIO Replay With Synchronized Video/Telemetry
|
||||
|
||||
**Summary**: Validate that BASALT + safety/anchor wrapper can process synchronized camera/IMU data and produce trajectory estimates with calibrated confidence.
|
||||
**Summary**: Validate that BASALT + safety/anchor wrapper can process synchronized nadir video, IMU, and trajectory telemetry and produce frame-by-frame estimates with honest confidence.
|
||||
|
||||
**Traces to**: AC-1.3, AC-2.1a, AC-2.2, AC-4.1, AC-4.2
|
||||
|
||||
**Category**: VO / IMU Propagation
|
||||
|
||||
**Preconditions**:
|
||||
- Public synchronized dataset slice is pinned during implementation. Strongest candidates: MUN-FRL, ALTO, EPFL fixed-wing, Kagaru; EuRoC/UZH FPV are proxy-only.
|
||||
- Ground-truth trajectory or frame poses are available.
|
||||
- Derkachi replay fixture is mounted from `input_data/flight_derkachi/`.
|
||||
- `flight_derkachi.mp4` is readable as cropped nadir video: 880 x 720, 30 fps, approximately 490.07 s.
|
||||
- `data_imu.csv` contains monotonic 10 Hz `Time`, `timestamp(ms)`, `SCALED_IMU2.*`, and `GLOBAL_POSITION_INT.*` fields for 4,900 rows.
|
||||
- Camera intrinsics, lens distortion, and camera-to-body transform are either pinned or the run is marked as calibration-limited.
|
||||
- Public synchronized dataset slice remains useful for calibrated final comparison. Strongest candidates: MUN-FRL, ALTO, EPFL fixed-wing, Kagaru; EuRoC/UZH FPV are proxy-only.
|
||||
|
||||
**Input data**: `public_nadir_vio_candidates`
|
||||
**Input data**: `derkachi_video_telemetry`, `public_nadir_vio_candidates`
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|-----------------|--------------------------|
|
||||
| 1 | Replay synchronized camera and IMU stream | System emits frame-by-frame `vo_extrapolated` or `satellite_anchored` estimates |
|
||||
| 2 | Compare output trajectory to dataset ground truth | Error and covariance calibration are reported per segment |
|
||||
| 3 | Compare against OpenVINS reference replay | BASALT + wrapper does not materially under-report uncertainty relative to error |
|
||||
| 1 | Validate Derkachi video/telemetry alignment | Harness accepts the fixture only if MP4 duration and CSV duration differ by <=250 ms and there are exactly 3 video frames per telemetry row |
|
||||
| 2 | Replay synchronized video frames and IMU stream | System emits frame-by-frame `vo_extrapolated` or `satellite_anchored` estimates without batching |
|
||||
| 3 | Compare output trajectory to `GLOBAL_POSITION_INT` lat/lon/alt/heading | Error, covariance, source label, and anchor age are reported per segment |
|
||||
| 4 | Compare calibrated public/representative replay against ground truth when available | BASALT + wrapper does not materially under-report uncertainty relative to error |
|
||||
| 5 | Compare against OpenVINS reference replay when available | BASALT + wrapper does not materially under-report uncertainty relative to error |
|
||||
|
||||
**Expected outcome**: VO registration succeeds for >95% of normal overlapping frames in dataset-supported normal segments; VO homography MRE is <1.0 px where homography validation is applicable.
|
||||
**Expected outcome**: Derkachi replay is accepted as a synchronized representative fixture and produces continuous estimates for >95% of normal overlapping frames. Absolute geolocation and covariance pass/fail thresholds are calibration-gated until camera intrinsics, distortion, and camera-to-body transform are pinned. For calibrated datasets, VO homography MRE is <1.0 px where homography validation is applicable.
|
||||
|
||||
**Max execution time**: Dataset-dependent, but replay must report per-frame latency.
|
||||
|
||||
---
|
||||
|
||||
### FT-P-04: Satellite Retrieval And Anchor Verification
|
||||
### FT-P-04: Satellite Service And Anchor Verification
|
||||
|
||||
**Summary**: Validate that relocalization uses global retrieval plus local verification and emits only verified satellite anchors.
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
| Environment | Purpose | Required for |
|
||||
|-------------|---------|--------------|
|
||||
| Local replay workstation | Fast still-image and dataset replay validation | Frame-center geolocation, satellite retrieval, stale-tile rejection |
|
||||
| Local replay workstation | Fast still-image and dataset replay validation | Frame-center geolocation, Satellite Service local retrieval, stale-tile rejection |
|
||||
| Jetson Orin Nano Super | Production-like latency, memory, thermal, and TensorRT/ONNX profiling | AC-4.1, AC-4.2, AC-NEW-1, AC-NEW-5 |
|
||||
| ArduPilot Plane SITL + QGroundControl | MAVLink `GPS_INPUT`, spoofing, failsafe, and GCS status validation | AC-4.3, AC-5.2, AC-NEW-2, AC-NEW-8 |
|
||||
| Representative flight/replay rig | Final acceptance evidence with synchronized nav camera, FC IMU/attitude/airspeed/altitude, MAVLink logs, and ground truth | Final AC signoff |
|
||||
@@ -38,6 +38,7 @@
|
||||
|--------|------------|---------|
|
||||
| input-data | `/data/input` | `_docs/00_problem/input_data/` and public dataset slices |
|
||||
| expected-results | `/data/expected` | `_docs/00_problem/input_data/expected_results/` |
|
||||
| derkachi-replay | `/data/input/flight_derkachi` | Cropped nadir MP4 plus synchronized IMU and `GLOBAL_POSITION_INT` trajectory |
|
||||
| satellite-cache | `/cache/satellite` | COG tiles, manifests, descriptor index fixtures |
|
||||
| fdr-output | `/fdr` | Flight-data-recorder outputs for validation |
|
||||
|
||||
@@ -98,6 +99,7 @@
|
||||
Use Docker or local host replay for deterministic, reproducible tests that do not require physical Jetson hardware:
|
||||
|
||||
- Still-image frame-center geolocation.
|
||||
- Derkachi synchronized video/telemetry replay, including alignment and VIO smoke checks.
|
||||
- Satellite-cache freshness and integrity fixtures.
|
||||
- FAISS descriptor/index behavior.
|
||||
- Public dataset replay where GPU/hardware timing is not the assertion.
|
||||
|
||||
@@ -25,23 +25,25 @@
|
||||
|
||||
### NFT-PERF-02: BASALT + Wrapper Replay Latency
|
||||
|
||||
**Summary**: Validate relative VIO hot-path latency using synchronized public or representative camera/IMU data.
|
||||
**Summary**: Validate relative VIO hot-path latency using synchronized Derkachi video/telemetry and public or representative camera/IMU data.
|
||||
|
||||
**Traces to**: AC-2.1a, AC-4.1, AC-4.2
|
||||
|
||||
**Metric**: Per-frame VIO latency, completion rate, and memory usage.
|
||||
|
||||
**Preconditions**:
|
||||
- MUN-FRL/ALTO/EPFL/Kagaru or representative synchronized dataset slice is pinned.
|
||||
- OpenVINS reference replay is available for comparison.
|
||||
- Derkachi `flight_derkachi.mp4` and `data_imu.csv` are mounted and pass fixture validation.
|
||||
- MUN-FRL/ALTO/EPFL/Kagaru or another representative synchronized dataset slice is pinned for calibrated final comparison.
|
||||
- OpenVINS reference replay is available for comparison when the dataset supports it.
|
||||
|
||||
| Step | Consumer Action | Measurement |
|
||||
|------|-----------------|-------------|
|
||||
| 1 | Replay synchronized camera/IMU stream through BASALT + wrapper | Measure per-frame processing time |
|
||||
| 2 | Compare emitted trajectory against ground truth | Measure completion rate and error distribution |
|
||||
| 3 | Monitor memory | Track CPU/GPU shared memory peak |
|
||||
| 1 | Replay Derkachi video at target 3 fps and stress rates from the 30 fps source | Measure per-frame processing time, dropped frames, and telemetry alignment |
|
||||
| 2 | Replay synchronized camera/IMU stream through BASALT + wrapper | Measure VIO processing time and completion rate |
|
||||
| 3 | Compare emitted trajectory against Derkachi `GLOBAL_POSITION_INT` and calibrated dataset ground truth where available | Measure completion rate and error distribution |
|
||||
| 4 | Monitor memory | Track CPU/GPU shared memory peak |
|
||||
|
||||
**Pass criteria**: Normal-frame VO registration >95%; p95 processing latency <400 ms for the hot path; memory <8 GB shared.
|
||||
**Pass criteria**: Normal-frame VO registration >95% on calibration-supported segments; p95 processing latency <400 ms for the hot path; memory <8 GB shared; Derkachi replay maintains stable 3-video-frames-per-telemetry-row alignment with <=10% dropped frames under sustained target-rate replay.
|
||||
|
||||
**Duration**: Dataset-dependent; at least one normal segment and one challenging segment.
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
| `project_60_still_images` | 60 nadir images with WGS84 frame-center coordinates from `coordinates.csv`; height 400 m | FT-P-01, FT-P-02, FT-N-01, NFT-PERF-01 | Mounted from `_docs/00_problem/input_data/` | Read-only |
|
||||
| `project_gmaps_reference_subset` | Google Maps reference images available for the first sample frames | FT-P-02, FT-N-01 | Mounted from `_docs/00_problem/input_data/` | Read-only |
|
||||
| `expected_frame_centers` | Expected lat/lon and thresholds derived from `coordinates.csv` | FT-P-01, FT-P-02 | `_docs/00_problem/input_data/expected_results/results_report.md` | Read-only |
|
||||
| `derkachi_video_telemetry` | Cropped nadir MP4 synchronized with IMU and `GLOBAL_POSITION_INT` trajectory: 880 x 720, 30 fps, ~490.07 s; telemetry 10 Hz, 4,900 rows | FT-P-03, NFT-PERF-02, NFT-RES-02 | Mounted from `_docs/00_problem/input_data/flight_derkachi/` | Read-only |
|
||||
| `public_nadir_vio_candidates` | MUN-FRL, ALTO, EPFL fixed-wing, Kagaru, AerialVL/VPAir slices, EuRoC/UZH FPV proxy slices | FT-P-03, FT-P-04, NFT-PERF-02, NFT-RES-02 | Downloaded or mounted by replay harness; exact files pinned during implementation | Reset fixture volume |
|
||||
| `sitl_spoofing_scenarios` | Generated ArduPilot Plane SITL GPS loss/spoofing traces | FT-N-02, NFT-RES-01, NFT-SEC-03 | Generated by test harness | Discard generated logs after report |
|
||||
| `cache_integrity_fixtures` | Fresh, stale, unsigned, hash-mismatched, and low-resolution cache manifests | FT-N-03, NFT-SEC-01, NFT-SEC-02 | Mounted fixture volume | Read-only |
|
||||
@@ -36,6 +37,7 @@ Every replay test uses read-only fixture mounts and writes results to a fresh `t
|
||||
| `coordinates.csv` | `_docs/00_problem/input_data/coordinates.csv` | Machine-readable expected frame centers | FT-P-01, FT-P-02 |
|
||||
| `data_parameters.md` | `_docs/00_problem/input_data/data_parameters.md` | Height 400 m and camera model | FT-P-01, NFT-PERF-01 |
|
||||
| `AD000001_gmaps.png`, `AD000002_gmaps.png` | `_docs/00_problem/input_data/` | Reference map screenshots for sample sanity checks | FT-P-02 |
|
||||
| `flight_derkachi/flight_derkachi.mp4` + `flight_derkachi/data_imu.csv` | `_docs/00_problem/input_data/flight_derkachi/` | Cropped nadir video synchronized with IMU and `GLOBAL_POSITION_INT` GPS trajectory | FT-P-03, NFT-PERF-02, NFT-RES-02 |
|
||||
| Public dataset slices | External fixture paths pinned during implementation | Synchronized camera/IMU/GNSS/ground truth where available | FT-P-03, FT-P-04, NFT-PERF-02, NFT-RES-02 |
|
||||
|
||||
## Expected Results Mapping
|
||||
@@ -44,7 +46,7 @@ Every replay test uses read-only fixture mounts and writes results to a fresh `t
|
||||
|------------------|------------|-----------------|-------------------|-----------|------------------------|
|
||||
| FT-P-01 | `AD000001.jpg` ... `AD000060.jpg` | Output WGS84 frame center per mapped row; >=80% within 50 m, >=50% within 20 m | Haversine distance threshold + aggregate pass rate | 50 m primary, 20 m stretch | `input_data/expected_results/results_report.md` |
|
||||
| FT-P-02 | Same 60 images + map references where present | Output includes source label, covariance semi-major axis, and anchor age for every emitted estimate | Required-field validation + geolocation threshold | Required fields present; geolocation thresholds as above | `input_data/expected_results/results_report.md` |
|
||||
| FT-P-03 | Public synchronized VIO dataset slice | BASALT + wrapper emits trajectory with calibrated covariance and no optimistic under-reporting | Dataset ground-truth trajectory comparison | Dataset-specific threshold pinned at implementation; must support AC-1/AC-2 signoff only after representative data | Public dataset ground truth |
|
||||
| FT-P-03 | `derkachi_video_telemetry` plus public synchronized VIO dataset slice when available | BASALT + wrapper emits trajectory with calibrated covariance and no optimistic under-reporting | Compare Derkachi output to `GLOBAL_POSITION_INT` trajectory for smoke/relative validation; compare public/representative calibrated runs to ground truth for final accuracy | Derkachi threshold is calibration-gated; final threshold is dataset-specific and pinned after camera calibration | `data_imu.csv` trajectory plus public dataset ground truth |
|
||||
| FT-P-04 | AerialVL/ALTO/VPAir-style aerial localization slice | Satellite retrieval returns candidate chunks and local verification produces accepted/rejected anchors | Georeference error + MRE + source-label checks | AC-1.1/1.2 and AC-2.2 thresholds where dataset supports them | Public dataset ground truth/reference map |
|
||||
| FT-N-01 | Low-texture/repetitive frames from sample or public data | System emits degraded confidence or rejects anchor rather than confident false fix | Source label and covariance threshold | No `satellite_anchored` label unless gates pass | Fixture-specific |
|
||||
| FT-N-02 | Plane SITL GPS spoof/loss trace | Spoofed GPS rejected; system promotes own estimate within <3 s when trigger conditions are met | Event timing and MAVLink field checks | <3 s promotion; blackout thresholds from AC-NEW-8 | Generated SITL trace |
|
||||
@@ -64,7 +66,9 @@ Every replay test uses read-only fixture mounts and writes results to a fresh `t
|
||||
|-----------|------------|------------------|--------------------------|
|
||||
| Image frame | Existing file, readable image, expected timestamp/order metadata if sequence replay | Missing image, corrupt image, unsupported resolution | Mark estimate unavailable/degraded, log error, continue if possible |
|
||||
| Expected coordinate | Valid WGS84 latitude/longitude | Out-of-range lat/lon, missing row | Reject test fixture before replay |
|
||||
| Video/telemetry pair | MP4 duration matches telemetry duration, frame-to-telemetry ratio is stable, timestamps are monotonic | Duration drift >250 ms, missing trajectory columns, non-monotonic timestamps | Reject fixture before replay |
|
||||
| IMU trace | Monotonic timestamps, angular rate/accel fields, calibrated units | Non-monotonic timestamps, missing samples | Reject fixture or enter degraded mode depending scenario |
|
||||
| GPS trajectory trace | Valid WGS84 lat/lon, altitude, velocity, and heading fields | Out-of-range lat/lon, impossible altitude, missing `GLOBAL_POSITION_INT` columns | Reject trajectory comparison while allowing pure video replay if applicable |
|
||||
| Cache tile manifest | CRS, m/px, capture date, source, hashes, signature/provenance | Stale, unsigned, hash mismatch, low resolution | Reject or down-confidence per AC-8.2 and AC-NEW-6 |
|
||||
| MAVLink output | Valid `GPS_INPUT` fields and fix type/accuracy semantics | Missing `horiz_accuracy`, impossible fix type | Fail test; output contract violated |
|
||||
|
||||
@@ -74,13 +78,13 @@ Every replay test uses read-only fixture mounts and writes results to a fresh `t
|
||||
|------------------|-------|---------------------|--------------------------|-----------------|---------------------------|---------------------|
|
||||
| FT-P-01 | Input/output | 60 project images + `coordinates.csv` | WGS84 center per image with 50 m / 20 m thresholds | Yes | Yes | Keep |
|
||||
| FT-P-02 | Input/output | 60 project images + output schema expectations | Required confidence/source-label fields and thresholds | Yes | Yes | Keep |
|
||||
| FT-P-03 | Input/output | Public or representative synchronized camera/IMU dataset | Dataset ground truth and calibrated covariance checks | Accepted as required external fixture | Accepted as dataset ground truth | Keep with acquisition task |
|
||||
| FT-P-03 | Input/output | Derkachi synchronized video/IMU/GPS fixture; public or calibrated representative dataset for final accuracy | Derkachi `GLOBAL_POSITION_INT` trajectory for smoke/relative validation; calibrated ground truth for final covariance checks | Yes for Derkachi; public/calibrated dataset still useful for final signoff | Yes for Derkachi GPS trajectory; calibrated camera thresholds pending | Keep with calibration gate |
|
||||
| FT-P-04 | Input/output | Public aerial localization or project cache fixture | Georeference, MRE, and source-label checks | Accepted as required external fixture | Accepted as dataset/reference-map ground truth | Keep with acquisition task |
|
||||
| FT-N-01 | Behavioral/input-output | Ambiguous low-texture/repetitive frames | 0 confident false anchors | Accepted as project/public fixture | Yes | Keep |
|
||||
| FT-N-02 | Behavioral | Generated Plane SITL spoof/blackout trace | Timing and MAVLink field thresholds from AC-NEW-8 | Generated by test harness | Yes | Keep |
|
||||
| FT-N-03 | Behavioral/input-output | Cache integrity fixtures | 0 trusted anchors from stale/invalid tiles | Generated fixture | Yes | Keep |
|
||||
| NFT-PERF-01 | Input/output | 60 project images | p95 latency and drop-rate thresholds | Yes | Yes | Keep |
|
||||
| NFT-PERF-02 | Input/output | Public/representative synchronized camera/IMU dataset | VO registration, latency, memory thresholds | Accepted as required external fixture | Yes | Keep with acquisition task |
|
||||
| NFT-PERF-02 | Input/output | Derkachi synchronized video/IMU/GPS fixture; public/representative synchronized camera/IMU dataset | VO registration, latency, memory thresholds | Yes for Derkachi | Yes | Keep with calibration gate |
|
||||
| NFT-PERF-03 | Behavioral/input-output | Precomputed descriptor/cache fixture | Trigger-path latency and MRE thresholds | Generated fixture | Yes | Keep |
|
||||
| NFT-PERF-04 | Behavioral | Cold-start harness and cache fixture | <30 s p95 over 50 runs | Generated by test harness | Yes | Keep |
|
||||
| NFT-RES-* | Behavioral | Fault triggers and generated traces | AC-defined timing/status thresholds | Generated by test harness | Yes | Keep |
|
||||
@@ -90,6 +94,7 @@ Every replay test uses read-only fixture mounts and writes results to a fresh `t
|
||||
**Coverage after validation**: 49/49 AC and restriction groups remain covered. No tests were removed.
|
||||
|
||||
**Acquisition tasks required downstream**:
|
||||
- Pin camera intrinsics, lens distortion, raw camera feed parameters, and camera-to-body mounting transform for the Derkachi fixture or future representative recordings.
|
||||
- Pin and download at least one strong synchronized nadir camera + IMU + ground-truth dataset, preferably MUN-FRL or ALTO, with EPFL fixed-wing and Kagaru as fixed-wing/farmland candidates.
|
||||
- Pin license-compatible VPR/localization datasets for satellite anchor tests; VPAir and UZH FPV have non-commercial restrictions and must not be used for commercial acceptance unless license terms allow it.
|
||||
- Create generated fixtures for Plane SITL spoofing, stale cache manifests, signed/unsigned manifests, FDR load, and thermal/resource monitoring during implementation.
|
||||
|
||||
@@ -54,7 +54,7 @@
|
||||
| R-SAT-01 | Offline-only Satellite Service cache, no in-flight provider fetch | NFT-SEC-04 | Covered |
|
||||
| R-SAT-02 | Cache resolution/freshness/metadata conventions | FT-N-03, NFT-RES-LIM-03 | Covered |
|
||||
| R-HW-01 | Jetson Orin Nano Super 8 GB / 25 W | NFT-RES-LIM-01, NFT-RES-LIM-02 | Covered |
|
||||
| R-SENSOR-01 | FC IMU available; sample data lacks synchronized IMU | FT-P-03, NFT-PERF-02 | Covered through public/representative dataset plan |
|
||||
| R-SENSOR-01 | FC IMU available; original still-image sample lacks synchronized IMU; Derkachi fixture provides video/IMU/GPS trajectory but calibration is pending | FT-P-03, NFT-PERF-02 | Covered through Derkachi representative replay plus public/calibrated dataset plan |
|
||||
| R-MAV-01 | MAVLink, ArduPilot only, GPS_INPUT via pymavlink | FT-N-02, NFT-SEC-03 | Covered |
|
||||
| R-GCS-01 | QGroundControl supported GCS | FT-N-02, NFT-SEC-03 | Covered |
|
||||
| R-SAFETY-01 | False-position, cold-start, spoofing, and failsafe constraints | FT-N-01, FT-N-02, NFT-PERF-04, NFT-RES-01 | Covered |
|
||||
@@ -76,5 +76,6 @@
|
||||
## Data Coverage Caveats
|
||||
|
||||
- Current project data fully supports still-image frame-center checks for 60 mapped images.
|
||||
- Current project data does not contain synchronized IMU or ground-truth trajectory for BASALT VIO validation.
|
||||
- Phase 3 must validate public dataset acquisition before FT-P-03, FT-P-04, and NFT-PERF-02 can remain in the executable test plan.
|
||||
- Derkachi project data supports synchronized video/IMU/GPS trajectory replay for FT-P-03 and NFT-PERF-02.
|
||||
- Derkachi project data is calibration-limited: raw camera intrinsics, lens distortion, and camera-to-body transform are still required before final absolute accuracy thresholds can be treated as production acceptance.
|
||||
- Phase 3 must validate camera calibration inputs and public/calibrated dataset acquisition before FT-P-03, FT-P-04, and NFT-PERF-02 can be used for final signoff.
|
||||
|
||||
Reference in New Issue
Block a user