mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 13:11:14 +00:00
Oleksandr Bezdieniezhnykh
323 lines
13 KiB
Markdown
323 lines
13 KiB
Markdown
# GPS-Denied Onboard Localization — System Flows
|
|
|
|
## Flow Inventory
|
|
|
|
| # | Flow Name | Trigger | Primary Components | Criticality |
|
|
|---|-----------|---------|--------------------|-------------|
|
|
| F1 | Pre-flight cache preparation | Operator sync before mission | Satellite Service, cache/tile lifecycle, satellite retrieval | High |
|
|
| F2 | Normal frame processing | Navigation frame + FC telemetry | Camera ingest, BASALT VIO adapter, safety/anchor wrapper, MAVLink, FDR | High |
|
|
| F3 | Satellite relocalization | Cold start, VO failure, sharp turn, covariance growth, stale anchor | Satellite retrieval, anchor verification, safety/anchor wrapper | High |
|
|
| F4 | Visual blackout / spoofing degraded mode | Image-quality failure and GPS health failure | Camera ingest, MAVLink telemetry, safety/anchor wrapper, QGC, FDR | Critical |
|
|
| F5 | Generated tile lifecycle | High-confidence pose + usable frame | Camera ingest, safety/anchor wrapper, cache/tile lifecycle, FDR | Medium |
|
|
| F6 | Post-flight sync and audit | Landing / operator offload | Cache/tile lifecycle, Satellite Service, FDR | Medium |
|
|
| F7 | Validation replay | Test harness invocation | Validation harness, system runtime, public datasets, SITL | High |
|
|
|
|
## Flow Dependencies
|
|
|
|
| Flow | Depends On | Shares Data With |
|
|
|------|------------|------------------|
|
|
| F1 | Satellite Service cache export | F2, F3, F5 |
|
|
| F2 | F1 for cache availability; FC telemetry | F3, F4, F5, FDR |
|
|
| F3 | F1 cache/index; F2 state estimate | F2, F5 |
|
|
| F4 | F2 telemetry and quality signals | F2, QGC/FDR |
|
|
| F5 | Accepted state/covariance from F2/F3 | F6 |
|
|
| F6 | F5 generated tiles and FDR | Satellite Service |
|
|
| F7 | Test fixtures and selected execution environment | All flows |
|
|
|
|
---
|
|
|
|
## Flow F1: Pre-Flight Cache Preparation
|
|
|
|
### Description
|
|
|
|
Before flight, the Satellite Service provides an offline cache for the operational area, including COG tiles, manifests, sidecars, VPR chunks, descriptors, and FAISS index files.
|
|
|
|
### Preconditions
|
|
|
|
- Operational area and sector freshness classification are known.
|
|
- Cache imagery meets 0.5 m/px minimum and ideally 0.3 m/px.
|
|
- Cache package fits storage budget or has approved split descriptor budget.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Operator
|
|
participant SatelliteService
|
|
participant CacheLifecycle
|
|
participant SatelliteRetrieval
|
|
|
|
Operator->>SatelliteService: Request mission cache
|
|
SatelliteService-->>CacheLifecycle: COG tiles + manifests + sidecars
|
|
CacheLifecycle->>CacheLifecycle: Verify signatures, hashes, freshness, resolution
|
|
CacheLifecycle->>SatelliteRetrieval: Load VPR chunks + FAISS index
|
|
CacheLifecycle-->>Operator: Cache validation report
|
|
```
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Satellite Service | Cache lifecycle | Tiles and metadata | COG + PostgreSQL/PostGIS manifest + signed JSON sidecars |
|
|
| 2 | Cache lifecycle | Satellite retrieval | Descriptor/index files | FAISS index + descriptor sidecars |
|
|
| 3 | Cache lifecycle | Operator/FDR | Validation report | Markdown/CSV/log |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Stale tile | Cache validation | Capture date exceeds sector threshold | Reject/down-confidence tile |
|
|
| Hash mismatch | Cache validation | Sidecar hash mismatch | Reject tile and report security event |
|
|
| Cache too large | Cache load | Storage accounting > budget | Require cache rebuild or approved split budget |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| Runtime network calls | 0 | No in-flight provider calls |
|
|
| Cache load | Within cold-start budget contribution | Exact threshold set during implementation |
|
|
|
|
---
|
|
|
|
## Flow F2: Normal Frame Processing
|
|
|
|
### Description
|
|
|
|
During normal flight, the system processes each navigation frame and FC telemetry sample. The camera component first checks for total occlusion/blackout. Usable frames go to BASALT VIO; total-occlusion frames bypass VIO and send the wrapper into IMU-only degraded propagation.
|
|
|
|
### Preconditions
|
|
|
|
- Camera calibration/extrinsics are loaded.
|
|
- BASALT and wrapper are initialized.
|
|
- FC telemetry stream is healthy.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant CameraIngest
|
|
participant FCTelemetry
|
|
participant BasaltAdapter
|
|
participant SafetyWrapper
|
|
participant MavlinkOutput
|
|
participant FDR
|
|
|
|
CameraIngest->>CameraIngest: Total occlusion / blackout check
|
|
CameraIngest->>BasaltAdapter: Usable frame + timestamp + calibration
|
|
CameraIngest-->>SafetyWrapper: Degradation signal if total occlusion
|
|
FCTelemetry->>BasaltAdapter: IMU/attitude/altitude
|
|
BasaltAdapter-->>SafetyWrapper: Relative VIO state + quality
|
|
SafetyWrapper->>SafetyWrapper: Calibrate covariance + source label
|
|
SafetyWrapper-->>MavlinkOutput: GPS_INPUT estimate
|
|
SafetyWrapper-->>FDR: Estimate + inputs + health
|
|
```
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Camera ingest | BASALT adapter or safety wrapper | Frame metadata, image, occlusion status | Frame DTO / DegradationSignal |
|
|
| 2 | FC telemetry | BASALT adapter | IMU/attitude/altitude | MAVLink-derived telemetry DTO |
|
|
| 3 | BASALT adapter | Safety wrapper | Relative VIO state | VioState DTO |
|
|
| 4 | Safety wrapper | MAVLink output | WGS84 estimate | `GPS_INPUT` |
|
|
| 5 | Safety wrapper | FDR | Inputs/outputs/audit | FDR segment event |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Total occlusion / blackout | Camera ingest | Occlusion status, exposure/texture/decode checks | Bypass BASALT, enter IMU-only `dead_reckoned` propagation |
|
|
| Frame unreadable | Camera ingest | Decode/quality failure | Mark visual signal degraded and bypass BASALT for that frame |
|
|
| VIO quality low | BASALT adapter | Tracking/completion metrics | Trigger relocalization or dead reckoning |
|
|
| Covariance grows | Safety wrapper | Covariance threshold | Degrade fix type/source label |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| End-to-end latency | <400 ms p95 | Frame input to emitted estimate |
|
|
| Dropped frames | <=10% sustained | Under load |
|
|
| Memory | <8 GB shared | Jetson limit |
|
|
|
|
---
|
|
|
|
## Flow F3: Satellite Relocalization
|
|
|
|
### Description
|
|
|
|
When the state becomes uncertain or disconnected, the system retrieves satellite/cache candidates and accepts an anchor only after local verification and safety gates pass.
|
|
|
|
### Preconditions
|
|
|
|
- Offline VPR chunks and FAISS index are loaded.
|
|
- Trigger condition is met: cold start, VO failure, sharp turn, disconnected segment, covariance growth, or stale anchor.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant SafetyWrapper
|
|
participant SatelliteRetrieval
|
|
participant AnchorVerification
|
|
participant CacheLifecycle
|
|
participant FDR
|
|
|
|
SafetyWrapper->>SatelliteRetrieval: Relocalization request
|
|
SatelliteRetrieval->>CacheLifecycle: Read candidate chunk metadata
|
|
SatelliteRetrieval-->>AnchorVerification: Top-K candidates
|
|
AnchorVerification->>AnchorVerification: ALIKED/DISK+LightGlue + RANSAC
|
|
AnchorVerification-->>SafetyWrapper: Accepted/rejected anchor
|
|
SafetyWrapper->>SafetyWrapper: Mahalanobis + freshness + provenance gates
|
|
SafetyWrapper-->>FDR: Anchor decision audit
|
|
```
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Safety wrapper | Satellite retrieval | Query frame and prior/covariance | Relocalization DTO |
|
|
| 2 | Satellite retrieval | Anchor verification | Top-K chunks | Candidate list |
|
|
| 3 | Anchor verification | Safety wrapper | MRE, inliers, homography, provenance | AnchorDecision DTO |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| No good candidate | Retrieval/verification | Low score or failed RANSAC | Continue degraded and request GCS hint after threshold |
|
|
| Stale candidate | Cache lifecycle | Capture date gate | Reject/down-confidence |
|
|
| Implausible anchor | Safety wrapper | Mahalanobis/impossible velocity gate | Reject and log |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| Invocation frequency | Trigger-based only | Not per-frame |
|
|
| Cross-domain MRE | <2.5 px for accepted anchors | AC-2.2 |
|
|
|
|
---
|
|
|
|
## Flow F4: Visual Blackout / Spoofing Degraded Mode
|
|
|
|
### Description
|
|
|
|
When visual localization is unavailable due to total occlusion/blackout and GPS is denied/spoofed, the wrapper switches to honest IMU-only propagation from the last trusted state and degrades MAVLink output based on covariance/time thresholds.
|
|
|
|
### Preconditions
|
|
|
|
- Last trusted state exists.
|
|
- FC telemetry continues.
|
|
|
|
### Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant CameraIngest
|
|
participant FCTelemetry
|
|
participant SafetyWrapper
|
|
participant MavlinkOutput
|
|
participant QGC
|
|
participant FDR
|
|
|
|
CameraIngest-->>SafetyWrapper: Total occlusion / visual blackout signal
|
|
FCTelemetry-->>SafetyWrapper: GPS health/spoofing signal
|
|
SafetyWrapper->>SafetyWrapper: IMU-only propagation + monotonic covariance growth
|
|
SafetyWrapper->>SafetyWrapper: Switch source_label to dead_reckoned
|
|
SafetyWrapper-->>MavlinkOutput: Degraded GPS_INPUT
|
|
SafetyWrapper-->>QGC: VISUAL_BLACKOUT_IMU_ONLY / FAILSAFE
|
|
SafetyWrapper-->>FDR: Blackout and spoofing audit events
|
|
```
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Blackout >30 s | Safety wrapper | Timer threshold | Emit no-fix/failsafe |
|
|
| Covariance >500 m | Safety wrapper | Covariance threshold | `fix_type=0`, `horiz_accuracy=999.0` |
|
|
| Spoofed GPS recovers | Safety wrapper | FC health + visual consistency gate | Re-enable only after required stable interval and visual/satellite consistency |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| Mode transition | <=1 processed frame or <=400 ms | AC-3.5 |
|
|
| QGC status | 1-2 Hz | Downsampled operator awareness |
|
|
|
|
---
|
|
|
|
## Flow F5: Generated Tile Lifecycle
|
|
|
|
### Description
|
|
|
|
When pose confidence is strong enough, the system orthorectifies navigation imagery into write-new generated tiles and records quality/provenance sidecars.
|
|
|
|
### Preconditions
|
|
|
|
- Parent pose covariance passes tile-write gate.
|
|
- Frame quality supports orthorectification.
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Safety wrapper | Cache lifecycle | Pose/covariance + frame metadata | TileGenerationRequest |
|
|
| 2 | Cache lifecycle | Local storage | Generated COG + sidecar | COG + signed JSON |
|
|
| 3 | Cache lifecycle | FDR | Tile write event | FDR event |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Parent covariance too high | Safety wrapper | Sigma gate | Do not write tile |
|
|
| Duplicate sector | Cache lifecycle | Spatial deduplication | Keep latest/highest-quality tile |
|
|
| Sidecar write failure | Cache lifecycle | I/O error | Log and do not mark tile eligible |
|
|
|
|
---
|
|
|
|
## Flow F6: Post-Flight Sync And Audit
|
|
|
|
### Description
|
|
|
|
After landing, generated tiles and FDR evidence are exported for Satellite Service ingest and incident analysis.
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Cache lifecycle | Satellite Service | Generated tile package | COG + sidecar + manifest delta |
|
|
| 2 | FDR | Operator/audit tools | Mission replay evidence | Segmented logs + optional Parquet export |
|
|
|
|
### Error Scenarios
|
|
|
|
| Error | Where | Detection | Recovery |
|
|
|-------|-------|-----------|----------|
|
|
| Upload unavailable | Post-flight sync | Network/service failure | Retain package for retry |
|
|
| Candidate rejected by Service voting | Satellite Service | Ingest rules | Keep as candidate/soft trust, not trusted basemap |
|
|
|
|
---
|
|
|
|
## Flow F7: Validation Replay
|
|
|
|
### Description
|
|
|
|
The validation harness runs deterministic still-image, public dataset, SITL, Jetson, and representative replay scenarios against public interfaces.
|
|
|
|
### Preconditions
|
|
|
|
- Test data and expected results are pinned.
|
|
- Execution mode is selected: Docker/replay and local Jetson hardware.
|
|
|
|
### Data Flow
|
|
|
|
| Step | From | To | Data | Format |
|
|
|------|------|----|------|--------|
|
|
| 1 | Validation harness | Runtime | Images/telemetry/cache fixtures | File/stream/MAVLink |
|
|
| 2 | Runtime | Validation harness | GPS_INPUT/FDR/status | MAVLink/log files |
|
|
| 3 | Validation harness | Reports | Pass/fail metrics | CSV/Markdown |
|
|
|
|
### Performance Expectations
|
|
|
|
| Metric | Target | Notes |
|
|
|--------|--------|-------|
|
|
| PR smoke | <=15 min | Still-image/cache/SITL subset |
|
|
| Release gate | Hardware-dependent | Jetson and representative replay required |
|