mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 21:21:13 +00:00
Oleksandr Bezdieniezhnykh
72a9df6b57
Keep VIO package and native bridge paths backend-neutral so BASALT remains an implementation choice rather than a component boundary. Co-authored-by: Cursor <cursoragent@cursor.com>
322 lines
13 KiB
Markdown
322 lines
13 KiB
Markdown
# GPS-Denied Onboard Localization — System Flows
|
|
|
|
## Flow Inventory
|
|
|
|
| # | Flow Name | Trigger | Primary Components | Criticality |
|
|
|---|-----------|---------|--------------------|-------------|
|
|
| F1 | Pre-flight cache preparation | Operator sync before mission | Satellite Service, Tile Manager | High |
|
|
| F2 | Normal frame processing | Navigation frame + FC telemetry | Camera ingest, VIO adapter, safety/anchor wrapper, MAVLink, FDR | High |
|
|
| F3 | Satellite relocalization | Cold start, VO failure, sharp turn, covariance growth, stale anchor | Satellite Service, anchor verification, safety/anchor wrapper | High |
|
|
| F4 | Visual blackout / spoofing degraded mode | Image-quality failure and GPS health failure | Camera ingest, MAVLink telemetry, safety/anchor wrapper, QGC, FDR | Critical |
|
|
| F5 | Generated tile lifecycle | High-confidence pose + usable frame | Camera ingest, safety/anchor wrapper, Tile Manager, FDR | Medium |
|
|
| F6 | Post-flight sync and audit | Landing / operator offload | Tile Manager, Satellite Service, FDR | Medium |
|
|
| F7 | E2E validation replay | Test-suite invocation | Separate e2e test suite, system runtime, public datasets, SITL | High |
|
|
|
|
## Flow Dependencies
|
|
|
|
| Flow | Depends On | Shares Data With |
|
|
|------|------------|------------------|
|
|
| F1 | Satellite Service cache export and Tile Manager validation | F2, F3, F5 |
|
|
| F2 | F1 for cache availability; FC telemetry | F3, F4, F5, FDR |
|
|
| F3 | F1 cache/index; F2 state estimate | F2, F5 |
|
|
| F4 | F2 telemetry and quality signals | F2, QGC/FDR |
|
|
| F5 | Accepted state/covariance from F2/F3 | F6 |
|
|
| F6 | F5 generated tiles and FDR | Satellite Service |
|
|
| F7 | Test fixtures and selected execution environment | All flows |
|
|
|
|
---
|
|
|
|
## Flow F1: Pre-Flight Cache Preparation
|
|
|
|
### Description
|
|
|
|
Before flight, the Satellite Service imports an offline cache package for the operational area, including COG tiles, manifests, sidecars, VPR chunks, descriptors, and FAISS index files. No Satellite Service or satellite-provider calls are allowed during flight.
|
|
|
|
### Preconditions
|
|
|
|
- Operational area and sector freshness classification are known.
|
|
- Cache imagery meets 0.5 m/px minimum and ideally 0.3 m/px.
|
|
- Cache package fits storage budget or has approved split descriptor budget.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Operator
|
|
participant SatelliteService
|
|
participant TileManager
|
|
|
|
Operator->>SatelliteService: Request mission cache
|
|
SatelliteService-->>TileManager: COG tiles + manifests + sidecars
|
|
TileManager->>TileManager: Verify signatures, hashes, freshness, resolution
|
|
TileManager-->>SatelliteService: Local cache/index ready
|
|
TileManager-->>Operator: Cache validation report
|
|
```
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Satellite Service | Tile Manager | Tiles and metadata | COG + PostgreSQL/PostGIS manifest + signed JSON sidecars |
|
|
| 2 | Tile Manager | Satellite Service | Descriptor/index readiness | FAISS index + descriptor sidecars |
|
|
| 3 | Tile Manager | Operator/FDR | Validation report | Markdown/CSV/log |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Stale tile | Cache validation | Capture date exceeds sector threshold | Reject/down-confidence tile |
|
|
| Hash mismatch | Cache validation | Sidecar hash mismatch | Reject tile and report security event |
|
|
| Cache too large | Cache load | Storage accounting > budget | Require cache rebuild or approved split budget |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| Runtime network calls | 0 | No in-flight Satellite Service or provider calls |
|
|
| Cache load | Within cold-start budget contribution | Exact threshold set during implementation |
|
|
|
|
---
|
|
|
|
## Flow F2: Normal Frame Processing
|
|
|
|
### Description
|
|
|
|
During normal flight, the system processes each navigation frame and FC telemetry sample. The camera component first checks for total occlusion/blackout. Usable frames go to the VIO adapter; total-occlusion frames bypass VIO and send the wrapper into IMU-only degraded propagation.
|
|
|
|
### Preconditions
|
|
|
|
- Camera calibration/extrinsics are loaded.
|
|
- VIO adapter and wrapper are initialized.
|
|
- FC telemetry stream is healthy.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant CameraIngest
|
|
participant FCTelemetry
|
|
participant BasaltAdapter
|
|
participant SafetyWrapper
|
|
participant MavlinkOutput
|
|
participant FDR
|
|
|
|
CameraIngest->>CameraIngest: Total occlusion / blackout check
|
|
CameraIngest->>BasaltAdapter: Usable frame + timestamp + calibration
|
|
CameraIngest-->>SafetyWrapper: Degradation signal if total occlusion
|
|
FCTelemetry->>BasaltAdapter: IMU/attitude/altitude
|
|
BasaltAdapter-->>SafetyWrapper: Relative VIO state + quality
|
|
SafetyWrapper->>SafetyWrapper: Calibrate covariance + source label
|
|
SafetyWrapper-->>MavlinkOutput: GPS_INPUT estimate
|
|
SafetyWrapper-->>FDR: Estimate + inputs + health
|
|
```
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Camera ingest | VIO adapter or safety wrapper | Frame metadata, image, occlusion status | Frame DTO / DegradationSignal |
|
|
| 2 | FC telemetry | VIO adapter | IMU/attitude/altitude | MAVLink-derived telemetry DTO |
|
|
| 3 | VIO adapter | Safety wrapper | Relative VIO state | VioState DTO |
|
|
| 4 | Safety wrapper | MAVLink output | WGS84 estimate | `GPS_INPUT` |
|
|
| 5 | Safety wrapper | FDR | Inputs/outputs/audit | FDR segment event |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Total occlusion / blackout | Camera ingest | Occlusion status, exposure/texture/decode checks | Bypass VIO, enter IMU-only `dead_reckoned` propagation |
|
|
| Frame unreadable | Camera ingest | Decode/quality failure | Mark visual signal degraded and bypass VIO for that frame |
|
|
| VIO quality low | VIO adapter | Tracking/completion metrics | Trigger relocalization or dead reckoning |
|
|
| Covariance grows | Safety wrapper | Covariance threshold | Degrade fix type/source label |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| End-to-end latency | <400 ms p95 | Frame input to emitted estimate |
|
|
| Dropped frames | <=10% sustained | Under load |
|
|
| Memory | <8 GB shared | Jetson limit |
|
|
|
|
---
|
|
|
|
## Flow F3: Satellite Relocalization
|
|
|
|
### Description
|
|
|
|
When the state becomes uncertain or disconnected, the system retrieves satellite/cache candidates and accepts an anchor only after local verification and safety gates pass.
|
|
|
|
### Preconditions
|
|
|
|
- Offline VPR chunks and FAISS index are loaded.
|
|
- Trigger condition is met: cold start, VO failure, sharp turn, disconnected segment, covariance growth, or stale anchor.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant SafetyWrapper
|
|
participant SatelliteService
|
|
participant AnchorVerification
|
|
participant TileManager
|
|
participant FDR
|
|
|
|
SafetyWrapper->>SatelliteService: Relocalization request
|
|
SatelliteService->>TileManager: Read candidate chunk metadata
|
|
SatelliteService-->>AnchorVerification: Top-K candidates
|
|
AnchorVerification->>AnchorVerification: ALIKED/DISK+LightGlue + RANSAC
|
|
AnchorVerification-->>SafetyWrapper: Accepted/rejected anchor
|
|
SafetyWrapper->>SafetyWrapper: Mahalanobis + freshness + provenance gates
|
|
SafetyWrapper-->>FDR: Anchor decision audit
|
|
```
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Safety wrapper | Satellite Service | Query frame and prior/covariance | Relocalization DTO |
|
|
| 2 | Satellite Service | Anchor verification | Top-K chunks from local cache/index | Candidate list |
|
|
| 3 | Anchor verification | Safety wrapper | MRE, inliers, homography, provenance | AnchorDecision DTO |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| No good candidate | Retrieval/verification | Low score or failed RANSAC | Continue degraded and request GCS hint after threshold |
|
|
| Stale candidate | Tile Manager | Capture date gate | Reject/down-confidence |
|
|
| Implausible anchor | Safety wrapper | Mahalanobis/impossible velocity gate | Reject and log |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| Invocation frequency | Trigger-based only | Not per-frame |
|
|
| Cross-domain MRE | <2.5 px for accepted anchors | AC-2.2 |
|
|
|
|
---
|
|
|
|
## Flow F4: Visual Blackout / Spoofing Degraded Mode
|
|
|
|
### Description
|
|
|
|
When visual localization is unavailable due to total occlusion/blackout and GPS is denied/spoofed, the wrapper switches to honest IMU-only propagation from the last trusted state and degrades MAVLink output based on covariance/time thresholds.
|
|
|
|
### Preconditions
|
|
|
|
- Last trusted state exists.
|
|
- FC telemetry continues.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant CameraIngest
|
|
participant FCTelemetry
|
|
participant SafetyWrapper
|
|
participant MavlinkOutput
|
|
participant QGC
|
|
participant FDR
|
|
|
|
CameraIngest-->>SafetyWrapper: Total occlusion / visual blackout signal
|
|
FCTelemetry-->>SafetyWrapper: GPS health/spoofing signal
|
|
SafetyWrapper->>SafetyWrapper: IMU-only propagation + monotonic covariance growth
|
|
SafetyWrapper->>SafetyWrapper: Switch source_label to dead_reckoned
|
|
SafetyWrapper-->>MavlinkOutput: Degraded GPS_INPUT
|
|
SafetyWrapper-->>QGC: VISUAL_BLACKOUT_IMU_ONLY / FAILSAFE
|
|
SafetyWrapper-->>FDR: Blackout and spoofing audit events
|
|
```
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Blackout >30 s | Safety wrapper | Timer threshold | Emit no-fix/failsafe |
|
|
| Covariance >500 m | Safety wrapper | Covariance threshold | `fix_type=0`, `horiz_accuracy=999.0` |
|
|
| Spoofed GPS recovers | Safety wrapper | FC health + visual consistency gate | Re-enable only after required stable interval and visual/satellite consistency |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| Mode transition | <=1 processed frame or <=400 ms | AC-3.5 |
|
|
| QGC status | 1-2 Hz | Downsampled operator awareness |
|
|
|
|
---
|
|
|
|
## Flow F5: Generated Tile Lifecycle
|
|
|
|
### Description
|
|
|
|
When pose confidence is strong enough, the system orthorectifies navigation imagery into write-new generated tiles and records quality/provenance sidecars.
|
|
|
|
### Preconditions
|
|
|
|
- Parent pose covariance passes tile-write gate.
|
|
- Frame quality supports orthorectification.
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Safety wrapper | Tile Manager | Pose/covariance + frame metadata | TileGenerationRequest |
|
|
| 2 | Tile Manager | Local storage | Orthorectified generated COG + sidecar | COG + signed JSON |
|
|
| 3 | Tile Manager | FDR | Tile write event | FDR event |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Parent covariance too high | Safety wrapper | Sigma gate | Do not write tile |
|
|
| Duplicate sector | Tile Manager | Spatial deduplication | Keep latest/highest-quality tile |
|
|
| Sidecar write failure | Tile Manager | I/O error | Log and do not mark tile eligible |
|
|
|
|
---
|
|
|
|
## Flow F6: Post-Flight Sync And Audit
|
|
|
|
### Description
|
|
|
|
After landing, generated tiles and FDR evidence are exported through Satellite Service sync for ingest and incident analysis.
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Tile Manager | Satellite Service | Generated tile package | COG + sidecar + manifest delta |
|
|
| 2 | FDR | Operator/audit tools | Mission replay evidence | Segmented logs + optional Parquet export |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Upload unavailable | Post-flight sync | Network/service failure | Retain package for retry |
|
|
| Candidate rejected by Service voting | Satellite Service | Ingest rules | Keep as candidate/soft trust, not trusted basemap |
|
|
|
|
---
|
|
|
|
## Flow F7: Validation Replay
|
|
|
|
### Description
|
|
|
|
The separate e2e test suite runs deterministic still-image, public dataset, SITL, Jetson, and representative replay scenarios against public interfaces.
|
|
|
|
### Preconditions
|
|
|
|
- Test data and expected results are pinned.
|
|
- Execution mode is selected: Docker/replay and local Jetson hardware.
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | E2E test suite | Runtime | Images/telemetry/cache fixtures | File/stream/MAVLink |
|
|
| 2 | Runtime | E2E test suite | GPS_INPUT/FDR/status | MAVLink/log files |
|
|
| 3 | E2E test suite | Reports | Pass/fail metrics | CSV/Markdown |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| PR smoke | <=15 min | Still-image/cache/SITL subset |
|
|
| Release gate | Hardware-dependent | Jetson and representative replay required |
|