Revise acceptance criteria and restrictions documentation to clarify recent updates and specifications. Key changes include enhanced definitions for position accuracy, image processing quality, and operational parameters, as well as updates to camera specifications and validation requirements. This revision aims to improve clarity and ensure alignment with project goals.

This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-01 16:24:46 +03:00
parent 3f173c1bb7
commit 7e15868d39
62 changed files with 6878 additions and 13 deletions
@@ -0,0 +1,115 @@
# Camera Ingest And Calibration
## 1. High-Level Overview
**Purpose**: Ingest navigation-camera frames, attach timestamps and calibration metadata, undistort/normalize imagery, detect total occlusion/blackout before VIO, and classify image quality before VIO or satellite matching consumes it.
**Architectural Pattern**: Streaming adapter + validation gate.
**Upstream dependencies**: Navigation camera, camera calibration files.
**Downstream consumers**: BASALT VIO adapter, satellite retrieval, anchor verification, cache/tile lifecycle, FDR.
## 2. Internal Interfaces
### Interface: `FrameProvider`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `next_frame` | `FrameRequest` | `FramePacket` | Yes | `FrameUnavailable`, `CalibrationMissing`, `InvalidFrame` |
| `detect_occlusion` | `FramePacket` | `OcclusionReport` | No | `InvalidFrame` |
| `classify_quality` | `FramePacket` | `ImageQualityReport` | No | `InvalidFrame` |
**Input DTOs**:
```yaml
FrameRequest:
source: enum(live_camera, replay_file)
timestamp_ns: integer optional
```
**Output DTOs**:
```yaml
FramePacket:
frame_id: string
timestamp_ns: integer
image_ref: path_or_buffer
camera_calibration_id: string
altitude_hint_m: number optional
occlusion: OcclusionReport
quality: ImageQualityReport
OcclusionReport:
status: enum(clear, partial_occlusion, total_occlusion, blackout)
reason: enum(cloud, lens_cover, whiteout, decode_failure, underexposed, overexposed, unknown) optional
usable_for_vio: boolean
usable_for_anchor: boolean
ImageQualityReport:
usable_for_vio: boolean
usable_for_anchor: boolean
blackout_detected: boolean
blur_score: number
texture_score: number
```
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Load calibration by ID | Startup | Yes | No |
| Read replay frame | Test/replay | Yes | File order |
## 4. Implementation Details
**State Management**: Maintains current camera calibration version and frame sequence state.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| OpenCV | Decode, undistort, homography support, image metrics |
| Camera SDK / V4L2 / GigE SDK | Live camera access once interface is selected |
**Error Handling Strategy**:
- Camera or decode errors emit degraded quality and FDR events.
- Total occlusion or blackout sets `usable_for_vio=false`, bypasses BASALT for that frame, and emits a degradation signal to the safety/anchor wrapper.
- Missing calibration blocks production startup.
- ADTi public spec mismatch is tracked as a verification blocker until manufacturer spec is pinned.
## 5. Extensions and Helpers
| Helper | Purpose | Used By |
|--------|---------|---------|
| `geo_geometry_helper` | Coordinate transforms, GSD, WGS84/local conversions | Camera ingest, safety wrapper, cache lifecycle |
## 6. Caveats & Edge Cases
**Known limitations**:
- Public ADTi pages list 2 fps continuous capture and -10..40 C operating range; project assumptions need manufacturer verification.
- Live camera interface is TBD.
- Total occlusion detection must be conservative: false positives cause temporary IMU-only degradation, while false negatives can feed unusable frames into VIO.
**Performance bottlenecks**:
- Full-resolution preprocessing must stay inside the <400 ms p95 pipeline budget.
## 7. Dependency Graph
**Must be implemented after**: none.
**Can be implemented in parallel with**: cache/tile lifecycle, MAVLink/GCS integration.
**Blocks**: BASALT VIO adapter, anchor verification, generated tile lifecycle.
## 8. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Camera unavailable or calibration missing | `camera_calibration_missing id=...` |
| WARN | Frame degraded or occluded | `frame_quality_degraded occlusion=total_occlusion blur=... texture=...` |
| INFO | Calibration loaded | `camera_calibration_loaded version=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment and health log.
@@ -0,0 +1,139 @@
# Test Specification — Camera Ingest And Calibration
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-2.1a | VO registration uses only normal usable frames | IT-01, AT-01 | Covered |
| AC-3.5 | Visual blackout switches to degraded mode quickly | IT-02, AT-02 | Covered |
| AC-4.1 | Capture-to-output latency budget | PT-01 | Covered |
| AC-4.2 | Jetson memory budget | PT-01 | Covered |
| AC-8.4 | Mid-flight tile generation input eligibility | IT-03 | Covered |
| AC-8.5 | No raw frame retention | ST-01 | Covered |
| AC-NEW-8 | Full blackout/occlusion degraded-mode trigger | IT-02, AT-02 | Covered |
## Blackbox Tests
### IT-01: Usable Frame Classification
**Summary**: Verify normal nadir frames are marked usable for VIO and anchor matching.
**Traces to**: AC-2.1a
**Input data**: Project still images plus calibration metadata.
**Expected result**: `FramePacket.quality.usable_for_vio=true`, `usable_for_anchor=true`, and `occlusion.status=clear` for normal daytime textured frames.
**Max execution time**: 100 ms per frame.
**Dependencies**: Calibration file, replay fixture.
---
### IT-02: Total Occlusion Gate Before VIO
**Summary**: Verify total occlusion is detected before BASALT receives the frame.
**Traces to**: AC-3.5, AC-NEW-8
**Input data**: Black/white/covered-lens frames, corrupt frame fixture, low-texture whiteout frames.
**Expected result**: `occlusion.status` is `total_occlusion` or `blackout`, `usable_for_vio=false`, `usable_for_anchor=false`, and a degradation signal is emitted to the safety wrapper.
**Max execution time**: 100 ms per frame.
**Dependencies**: Safety wrapper test double.
---
### IT-03: Tile Generation Eligibility Metadata
**Summary**: Verify frame metadata required for generated tiles is available without persisting raw frames.
**Traces to**: AC-8.4, AC-8.5
**Input data**: Valid frame, altitude hint, calibration, pose fixture.
**Expected result**: Frame metadata supports orthorectification handoff; raw frame is not retained after processing outside allowed FDR thumbnail exception.
**Max execution time**: 100 ms per frame.
**Dependencies**: Cache lifecycle test double.
## Performance Tests
### PT-01: Ingest And Quality Latency
**Summary**: Verify decode, calibration lookup, occlusion detection, and quality classification stay inside the system latency budget.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Input rate: target camera/replay rate.
- Duration: 30 minutes replay.
- Dataset: project still images plus synthetic occlusion frames.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Per-frame ingest p95 | <=100 ms | >150 ms |
| Memory contribution | <=1 GB | >1.5 GB |
| Dropped frames | <=10% under sustained load | >10% |
**Resource limits**: Jetson shared memory remains below system 8 GB cap.
## Security Tests
### ST-01: Raw Frame Retention Check
**Summary**: Verify normal operation does not persist raw navigation frames.
**Traces to**: AC-8.5
**Attack vector**: Sensitive raw imagery remains on disk after processing.
**Test procedure**:
1. Run replay with normal and failed tile-generation frames.
2. Inspect output directories and FDR artifacts.
**Expected behavior**: Only metadata, generated tiles, and allowed low-rate failed-frame thumbnails are retained.
**Pass criteria**: No raw full-resolution frames remain after teardown.
## Acceptance Tests
### AT-01: Normal Frame Enters VIO
**Summary**: Confirm normal usable frames are passed to BASALT.
**Traces to**: AC-2.1a
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Feed a calibrated normal frame | Occlusion status is `clear` |
| 2 | Process quality gate | Frame is emitted to BASALT adapter |
---
### AT-02: Occluded Frame Bypasses VIO
**Summary**: Confirm full occlusion triggers IMU-only degraded path.
**Traces to**: AC-3.5, AC-NEW-8
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Feed total-occlusion frame | `usable_for_vio=false` |
| 2 | Observe downstream routing | BASALT is bypassed and safety wrapper receives degradation signal |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `project_60_still_images` | Normal nadir frames | `_docs/00_problem/input_data/` | Project data |
| `synthetic_occlusion_frames` | Black/white/corrupt/whiteout frames | Generated fixture | Small |
**Setup procedure**: Load calibration fixture and mount read-only frame data.
**Teardown procedure**: Remove run-scoped temp directories and verify no raw-frame persistence.
**Data isolation strategy**: Each run writes to a unique `test-results/<run-id>/` directory.
@@ -0,0 +1,92 @@
# BASALT VIO Adapter
## 1. High-Level Overview
**Purpose**: Wrap BASALT as a replaceable relative VIO component that consumes calibrated frames and FC IMU data, then emits relative pose/velocity/bias state and tracking quality.
**Architectural Pattern**: Adapter / anti-corruption layer.
**Upstream dependencies**: Camera ingest/calibration, MAVLink telemetry stream.
**Downstream consumers**: Safety/anchor wrapper, validation harness, FDR.
## 2. Internal Interfaces
### Interface: `VioAdapter`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `initialize` | `VioInitRequest` | `VioInitResult` | No | `CalibrationInvalid`, `DatasetUnsupported` |
| `process` | `VioInputPacket` | `VioStatePacket` | Yes | `TrackingLost`, `TimestampMismatch` |
| `health` | none | `VioHealth` | No | none |
**Input DTOs**:
```yaml
VioInputPacket:
frame: FramePacket
imu_samples: list[TelemetrySample]
attitude_sample: TelemetrySample optional
```
**Output DTOs**:
```yaml
VioStatePacket:
timestamp_ns: integer
relative_pose: transform
velocity: vector3
bias_estimate: object optional
tracking_quality: number
completed: boolean
covariance_hint: matrix optional
```
## 3. Data Access Patterns
No persistent production data ownership. Reads calibration/config at startup and emits state to downstream consumers.
## 4. Implementation Details
**State Management**: Owns BASALT runtime state and resets only through explicit wrapper command.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| BASALT | Relative visual-inertial odometry |
| Eigen/Sophus or BASALT native math stack | Pose and transform representation |
**Error Handling Strategy**:
- Tracking loss is surfaced to the safety/anchor wrapper, not hidden.
- Timestamp/camera-IMU sync violations fail the packet and are logged.
- The adapter never emits WGS84 coordinates; absolute semantics belong to the wrapper.
## 5. Caveats & Edge Cases
**Known limitations**:
- BASALT has no special fixed-wing nadir mode; validation must prove fit under low-parallax/planar terrain.
- BASALT covariance/confidence output is not the product authority; wrapper calibration is required.
**Performance bottlenecks**:
- Native VIO runtime and image resolution can exceed Jetson budget if not tuned.
## 6. Dependency Graph
**Must be implemented after**: Camera ingest/calibration, MAVLink telemetry DTO definitions.
**Can be implemented in parallel with**: satellite retrieval, cache lifecycle.
**Blocks**: Safety/anchor wrapper final integration.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | BASALT initialization fails | `basalt_init_failed reason=...` |
| WARN | Tracking quality drops | `vio_tracking_degraded quality=...` |
| INFO | VIO reset/reinitialized | `vio_reset cause=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment.
@@ -0,0 +1,140 @@
# Test Specification — BASALT VIO Adapter
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.3 | Drift between anchors and anchor-age reporting input | IT-02, AT-01 | Covered |
| AC-2.1a | >95% VO registration on normal segments | IT-01, AT-01 | Covered |
| AC-2.2 | <1.0 px VO MRE | IT-01 | Covered |
| AC-3.1 | Handles 350 m outliers/tilt | IT-03 | Covered |
| AC-3.2 | Sharp turns trigger relocalization path | IT-04 | Covered |
| AC-3.4 | Loss threshold feeds relocalization/dead reckoning | IT-04 | Covered |
| AC-4.1 | Hot-path latency | PT-01 | Covered |
| AC-4.2 | Jetson memory budget | PT-01 | Covered |
## Blackbox Tests
### IT-01: Public Dataset VIO Replay
**Summary**: Verify BASALT adapter produces relative motion for synchronized camera/IMU replay.
**Traces to**: AC-2.1a, AC-2.2
**Input data**: MUN-FRL preferred slice, or representative synchronized nav-camera + IMU + ground truth.
**Expected result**: VO registration succeeds for >95% of normal usable frames; frame-to-frame MRE <1.0 px where ground-truth/feature evaluation supports it.
**Max execution time**: Dataset-dependent; report per-frame latency.
**Dependencies**: Camera ingest, MAVLink telemetry/replay, calibration fixtures.
---
### IT-02: Relative Drift Reporting
**Summary**: Verify adapter emits state needed for wrapper drift and anchor-age accounting.
**Traces to**: AC-1.3
**Input data**: Segment with two known satellite anchors and IMU samples.
**Expected result**: Adapter emits continuous `VioStatePacket` values with timestamps and quality, enabling wrapper to compare VO extrapolation to next anchor.
**Max execution time**: Dataset-dependent.
**Dependencies**: Safety wrapper test harness.
---
### IT-03: Tilt/Outlier Robustness
**Summary**: Verify adapter reports degraded tracking without false success under tilt/outlier cases.
**Traces to**: AC-3.1
**Input data**: Replay segment with synthetic ±20° tilt and up to 350 m apparent outlier.
**Expected result**: Adapter either tracks with quality metadata or emits `TrackingLost`; it never hides a failure as high-quality VIO.
**Max execution time**: 15 minutes per fixture.
---
### IT-04: Sharp Turn / Loss Signal
**Summary**: Verify sharp turns and disconnected visual overlap produce wrapper-visible failure signals.
**Traces to**: AC-3.2, AC-3.4
**Input data**: <5% overlap sequence with heading change <70°.
**Expected result**: Adapter emits low tracking quality or `TrackingLost` within the loss window, allowing relocalization trigger.
**Max execution time**: 10 minutes.
## Performance Tests
### PT-01: BASALT Adapter Runtime Budget
**Summary**: Verify VIO processing does not consume the full <400 ms system p95 budget.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Input: synchronized public/representative replay.
- Duration: 30 minutes plus release long-run slice.
- Target: Jetson Orin Nano Super.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Adapter p95 latency | <=250 ms | >300 ms |
| Memory contribution | <=3 GB | >4 GB |
| Tracking failure on normal segments | <5% | >=5% |
**Resource limits**: Total system memory remains below 8 GB.
## Security Tests
### ST-01: Timestamp Injection Rejection
**Summary**: Verify malformed or non-monotonic timestamps do not produce trusted VIO state.
**Traces to**: AC-NEW-4
**Attack vector**: Replay or telemetry timestamp manipulation.
**Test procedure**:
1. Feed non-monotonic frame and IMU timestamps.
2. Observe adapter output.
**Expected behavior**: Adapter returns `TimestampMismatch` or low-quality failure; wrapper does not trust the state.
**Pass criteria**: No high-quality VIO state is emitted from malformed timing.
## Acceptance Tests
### AT-01: Normal VIO State Contract
**Summary**: Confirm adapter output contract supports downstream localization.
**Traces to**: AC-1.3, AC-2.1a
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Initialize with calibrated frame/IMU config | `VioInitResult` succeeds |
| 2 | Replay normal frames | `VioStatePacket` includes timestamp, relative pose, velocity, tracking quality |
| 3 | End segment | State stream is continuous enough for wrapper drift accounting |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `public_nadir_vio_candidates` | MUN-FRL/ALTO/Kagaru/EPFL slices | Public pinned fixtures | Dataset-dependent |
| `representative_sync_replay` | Target camera + FC IMU + ground truth | Project collection | TBD |
**Setup procedure**: Pin calibration/extrinsics and mount read-only synchronized replay data.
**Teardown procedure**: Remove generated result reports and adapter temp state.
**Data isolation strategy**: One run directory per dataset slice and configuration hash.
@@ -0,0 +1,106 @@
# Safety And Anchor Wrapper
## 1. High-Level Overview
**Purpose**: Own the authoritative localization state, confidence calibration, source labels, anchor fusion, degraded modes, tile-write gates, and MAVLink output semantics.
**Architectural Pattern**: Stateful coordinator / safety facade.
**Upstream dependencies**: BASALT VIO adapter, anchor verification, MAVLink telemetry, camera quality reports.
**Downstream consumers**: MAVLink/GCS integration, FDR, cache/tile lifecycle, validation harness.
## 2. Internal Interfaces
### Interface: `LocalizationStateMachine`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `update_vio` | `VioStatePacket` | `PositionEstimate` | Yes | `StateInconsistent` |
| `consider_anchor` | `AnchorDecision` | `AnchorAcceptanceResult` | No | `AnchorRejected` |
| `degrade` | `DegradationSignal` | `PositionEstimate` | No | none |
| `propagate_imu_only` | `ImuOnlyPropagationRequest` | `PositionEstimate` | Yes | `InsufficientTelemetry` |
| `tile_write_eligibility` | `FramePacket` | `TileWriteDecision` | No | none |
**Input DTOs**:
```yaml
AnchorDecision:
candidate_id: string
timestamp_ns: integer
estimated_pose_wgs84: object
inlier_count: integer
mre_px: number
tile_freshness_status: enum
provenance_status: enum
DegradationSignal:
type: enum(total_occlusion, visual_blackout, gps_spoofing, covariance_growth, tracking_lost)
timestamp_ns: integer
ImuOnlyPropagationRequest:
last_trusted_estimate: PositionEstimate
imu_samples: list[TelemetrySample]
elapsed_blackout_ms: integer
reason: enum(total_occlusion, visual_blackout, tracking_lost)
```
**Output DTOs**:
```yaml
PositionEstimate:
timestamp_ns: integer
lat_deg: number
lon_deg: number
alt_msl_m: number
covariance_95_semi_major_m: number
source_label: enum(satellite_anchored, vo_extrapolated, dead_reckoned)
fix_type: integer
horiz_accuracy_m: number
last_satellite_anchor_age_ms: integer
```
## 3. Data Access Patterns
No direct tile/image storage ownership. Writes all decisions to FDR via observability component.
## 4. Implementation Details
**State Management**: Owns the authoritative state machine and covariance growth model.
**Error Handling Strategy**:
- Reject uncertain anchors by default.
- Never emit optimistic accuracy when confidence is degraded.
- On total occlusion or visual blackout, do not call VIO for that frame; propagate from the last trusted state with IMU-only dynamics, set `source_label=dead_reckoned`, and grow covariance monotonically.
- If covariance or blackout thresholds exceed AC limits, emit no-fix/failsafe semantics.
- Treat cache freshness and provenance as evidence carried by `AnchorDecision`; do not call the cache lifecycle component directly during anchor acceptance.
## 5. Caveats & Edge Cases
**Known limitations**:
- Final covariance calibration requires representative synchronized data.
- The wrapper must stay independent of BASALT internals so VIO can be replaced.
- IMU-only propagation is an emergency bridge, not a reliable long-duration localization mode; the spec requires failsafe/no-fix once time or covariance thresholds are exceeded.
**Potential race conditions**:
- A delayed anchor result must be checked against current timestamp/state before acceptance.
## 6. Dependency Graph
**Must be implemented after**: VIO DTOs, anchor DTOs, MAVLink output contract.
**Can be implemented in parallel with**: FDR schema after DTOs stabilize.
**Blocks**: MAVLink production output, tile-write lifecycle, end-to-end validation.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | State invariant violation | `localization_state_inconsistent reason=...` |
| WARN | Anchor rejected or mode degraded | `anchor_rejected reason=mahalanobis_gate` |
| INFO | Mode transition | `source_label_changed from=vo_extrapolated to=dead_reckoned reason=total_occlusion` |
**Log format**: FDR structured event.
**Log storage**: FDR segment and QGC status for operator-critical events.
@@ -0,0 +1,208 @@
# Test Specification — Safety And Anchor Wrapper
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.1 | >=80% within 50 m | AT-01 | Covered |
| AC-1.2 | >=50% within 20 m | AT-01 | Covered |
| AC-1.3 | Drift and anchor age | IT-01 | Covered |
| AC-1.4 | Quantitative confidence + label | IT-02, AT-02 | Covered |
| AC-3.4 | Relocalization after no-position window | IT-03 | Covered |
| AC-3.5 | Blackout to dead reckoned | IT-04, AT-03 | Covered |
| AC-4.3 | GPS_INPUT semantics source fields | IT-05 | Covered |
| AC-4.4 | Frame-by-frame streaming | PT-01 | Covered |
| AC-4.5 | Correction updates | IT-06 | Covered |
| AC-5.1 | Initialize from FC state | IT-07 | Covered |
| AC-5.2 | >3 s no-estimate fallback | IT-03 | Covered |
| AC-5.3 | Reinitialize after reboot | IT-07 | Covered |
| AC-NEW-2 | Spoofing promotion <3 s | IT-05 | Covered |
| AC-NEW-4 | False-position safety budget | ST-01, AT-02 | Covered |
| AC-NEW-8 | IMU-only blackout thresholds | IT-04, AT-03 | Covered |
## Blackbox Tests
### IT-01: Drift And Anchor Age Accounting
**Summary**: Verify VO extrapolation drift and anchor age are tracked per estimate.
**Traces to**: AC-1.3
**Input data**: VIO state stream with two accepted anchor decisions.
**Expected result**: Every `PositionEstimate` includes `last_satellite_anchor_age_ms`; drift at next anchor is measured and binned by age.
**Max execution time**: 5 minutes.
---
### IT-02: Confidence Output Contract
**Summary**: Verify every estimate has covariance and source label.
**Traces to**: AC-1.4
**Input data**: satellite-anchored, VO-extrapolated, and dead-reckoned state fixtures.
**Expected result**: Each output contains `covariance_95_semi_major_m`, `source_label`, `fix_type`, and `horiz_accuracy_m`; `horiz_accuracy_m` never under-reports covariance.
**Max execution time**: 2 minutes.
---
### IT-03: No-Position Relocalization Trigger
**Summary**: Verify no position for >=3 frames and >=2 s triggers relocalization/degraded behavior.
**Traces to**: AC-3.4, AC-5.2
**Input data**: VIO loss sequence with no accepted anchors.
**Expected result**: Relocalization request is emitted and wrapper continues dead reckoning until fail threshold.
**Max execution time**: 5 minutes.
---
### IT-04: Total Blackout IMU-Only Propagation
**Summary**: Verify total occlusion produces honest IMU-only estimates and failsafe thresholds.
**Traces to**: AC-3.5, AC-NEW-8
**Input data**: Last trusted estimate, IMU/attitude/airspeed/altitude stream, total-occlusion signal for 5 s, 15 s, and 35 s.
**Expected result**: Wrapper emits `dead_reckoned` within <=1 frame or <=400 ms, covariance grows monotonically, and no-fix/failsafe is emitted when blackout >30 s or covariance >500 m.
**Max execution time**: 10 minutes.
---
### IT-05: Spoofing Promotion And GPS_INPUT Mapping
**Summary**: Verify wrapper promotes own estimate and maps output fields correctly under GPS spoofing.
**Traces to**: AC-4.3, AC-NEW-2
**Input data**: Plane SITL spoofing telemetry and trusted wrapper estimate.
**Expected result**: Own estimate is promoted within <3 s; v1 emits `GPS_INPUT` only; source labels and accuracy fields are correct.
**Max execution time**: 10 minutes.
---
### IT-06: Correction Update
**Summary**: Verify refined estimates can correct previous positions.
**Traces to**: AC-4.5
**Input data**: VO estimate followed by accepted satellite anchor for same segment.
**Expected result**: Wrapper emits updated estimate/correction with improved source label and covariance.
**Max execution time**: 5 minutes.
---
### IT-07: Initialization And Reboot Recovery
**Summary**: Verify wrapper initializes from FC state and can reinitialize after reboot.
**Traces to**: AC-5.1, AC-5.3
**Input data**: FC EKF position, IMU-extrapolated state, simulated companion restart.
**Expected result**: Wrapper resumes from current FC state and reports degraded confidence until re-anchored.
**Max execution time**: 5 minutes.
## Performance Tests
### PT-01: Wrapper Streaming Overhead
**Summary**: Verify state-machine processing does not delay frame-by-frame output.
**Traces to**: AC-4.4
**Load scenario**:
- Input: 3 Hz estimate stream with anchor and blackout events.
- Duration: 8-hour synthetic run.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Wrapper p95 processing | <=25 ms | >50 ms |
| Missed frame outputs | 0 except skip-allowed upstream drops | Any silent batch/delay |
**Resource limits**: Negligible memory growth across run.
## Security Tests
### ST-01: False Anchor Rejection
**Summary**: Verify impossible anchors do not become trusted estimates.
**Traces to**: AC-NEW-4
**Attack vector**: Anchor decision with plausible inliers but impossible jump or stale provenance.
**Test procedure**:
1. Feed an anchor >1 km from predicted state with low covariance.
2. Feed stale/provenance-failed anchor evidence.
**Expected behavior**: Anchor is rejected, FDR logs reason, source label remains degraded or VO extrapolated.
**Pass criteria**: 0 accepted impossible/stale anchors.
## Acceptance Tests
### AT-01: Position Accuracy Aggregation
**Summary**: Verify wrapper outputs can be scored against frame-center thresholds.
**Traces to**: AC-1.1, AC-1.2
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Replay mapped frame-center estimates | Report computes >=80% within 50 m and >=50% within 20 m |
| 2 | Include source labels | Accuracy is binned by source label |
---
### AT-02: Confidence Honesty
**Summary**: Verify reported confidence is conservative relative to measured error.
**Traces to**: AC-1.4, AC-NEW-4
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Replay ground-truth trajectory | Measured error is not systematically above reported covariance |
| 2 | Inject over-confident anchors | Mahalanobis gate rejects them |
---
### AT-03: Blackout Failsafe
**Summary**: Verify operator-visible blackout/failsafe behavior.
**Traces to**: AC-3.5, AC-NEW-8
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Inject total visual blackout | `dead_reckoned` within <=400 ms |
| 2 | Continue >30 s or covariance >500 m | `fix_type=0`, `horiz_accuracy=999.0`, QGC failsafe status |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `wrapper_state_fixtures` | VIO states, anchors, blackouts, spoofing signals | Generated fixtures | Small |
| `representative_replay` | Target synchronized replay and ground truth | Project collection | TBD |
**Setup procedure**: Load deterministic VIO/anchor/telemetry fixtures and run with isolated PostgreSQL schema.
**Teardown procedure**: Drop run schema and remove FDR segments.
**Data isolation strategy**: Use per-run mission IDs and database schemas.
@@ -0,0 +1,96 @@
# Satellite Retrieval
## 1. High-Level Overview
**Purpose**: Convert a query frame and prior state into ranked satellite/cache VPR chunk candidates using DINOv2-VLAD descriptors and FAISS.
**Architectural Pattern**: Query service / retrieval index adapter.
**Upstream dependencies**: Camera ingest/calibration, cache/tile lifecycle, safety/anchor wrapper.
**Downstream consumers**: Anchor verification, FDR.
## 2. Internal Interfaces
### Interface: `CandidateRetriever`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `retrieve` | `RetrievalRequest` | `RetrievalResult` | Yes | `IndexUnavailable`, `DescriptorFailed` |
| `load_index` | `IndexLoadRequest` | `IndexStatus` | No | `ManifestInvalid`, `IndexUnavailable` |
**Input DTOs**:
```yaml
RetrievalRequest:
frame: FramePacket
prior_estimate: PositionEstimate optional
search_radius_m: number optional
max_candidates: integer
```
**Output DTOs**:
```yaml
RetrievalResult:
query_descriptor_id: string
candidates: list[VprCandidate]
VprCandidate:
chunk_id: string
tile_id: string
score: number
footprint: geometry
freshness_status: enum
```
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Top-K FAISS search | Triggered only | No steady-state | FAISS index |
| Load chunk metadata | Per candidate | No | PostgreSQL/PostGIS spatial and chunk indexes |
## 4. Implementation Details
**State Management**: Holds loaded descriptor model and FAISS index handles.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| DINOv2 / ONNX / TensorRT candidate path | Query descriptor extraction |
| FAISS CPU | Top-K retrieval |
**Error Handling Strategy**:
- If descriptor extraction or index load fails, return no candidates and trigger degraded mode.
- Optimized engines are allowed only after descriptor-fidelity tests pass.
## 5. Caveats & Edge Cases
**Known limitations**:
- VPR result is only a candidate, never an accepted fix.
- Cross-domain retrieval can be wrong under seasonal, lighting, or terrain ambiguity.
**Performance bottlenecks**:
- Descriptor extraction on Jetson must be trigger-limited and profiled separately from BASALT.
## 6. Dependency Graph
**Must be implemented after**: cache manifest/index schema, camera frame DTOs.
**Can be implemented in parallel with**: anchor verification.
**Blocks**: satellite relocalization flow.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Index unavailable | `faiss_index_unavailable id=...` |
| WARN | No candidates | `vpr_no_candidates frame_id=...` |
| INFO | Retrieval invoked | `vpr_query candidates=... latency_ms=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment.
@@ -0,0 +1,153 @@
# Test Specification — Satellite Retrieval
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-3.2 | Sharp-turn relocalization | IT-02, AT-01 | Covered |
| AC-3.3 | >=3 disconnected segments | IT-03 | Covered |
| AC-3.4 | Relocalization request after loss | IT-02 | Covered |
| AC-4.1 | Trigger path latency | PT-01 | Covered |
| AC-4.2 | Memory budget | PT-01 | Covered |
| AC-8.1 | Cache imagery interface | IT-01 | Covered |
| AC-8.3 | Preloaded/preprocessed cache | IT-01 | Covered |
| AC-8.6 | VPR chunks, multi-scale, dynamic K | IT-01, IT-04 | Covered |
| AC-NEW-1 | Cold-start first fix support | PT-02 | Covered |
| AC-NEW-6 | Freshness-aware retrieval | IT-04, ST-01 | Covered |
## Blackbox Tests
### IT-01: Index Load And Chunk Coverage
**Summary**: Verify preloaded VPR chunks and FAISS index cover the operational area.
**Traces to**: AC-8.1, AC-8.3, AC-8.6
**Input data**: PostgreSQL/PostGIS cache manifest, VPR chunk metadata, FAISS index.
**Expected result**: Every test frame footprint falls inside at least one VPR chunk; fine and coarse descriptors are present where required.
**Max execution time**: 2 minutes per mission cache.
---
### IT-02: Sharp-Turn Retrieval Trigger
**Summary**: Verify sharp-turn state requests candidates rather than relying on frame-to-frame VO.
**Traces to**: AC-3.2, AC-3.4
**Input data**: Wrapper relocalization request with sharp-turn/loss reason.
**Expected result**: Retrieval returns bounded top-K candidates based on sector/covariance policy.
**Max execution time**: 2 seconds per query.
---
### IT-03: Disconnected Segment Retrieval
**Summary**: Verify at least three disconnected segments can each retrieve candidate chunks.
**Traces to**: AC-3.3
**Input data**: Three disconnected query frames with approximate prior/covariance.
**Expected result**: Each query returns a candidate set including the ground-truth region when covered by the cache fixture.
**Max execution time**: Dataset-dependent.
---
### IT-04: Dynamic K And Freshness Filter
**Summary**: Verify K varies by sector and covariance, and stale candidates are tagged.
**Traces to**: AC-8.6, AC-NEW-6
**Input data**: Stable and active-conflict sector cache fixtures with fresh/stale tiles.
**Expected result**: K=5 for stable low-covariance, K=20 for active-conflict, K=50 fallback; stale candidates are flagged for rejection/down-confidence.
**Max execution time**: 2 seconds per query.
## Performance Tests
### PT-01: Retrieval Query Runtime
**Summary**: Verify descriptor extraction and FAISS query fit trigger-path budget.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Query set: 100 representative relocalization frames.
- Environment: Jetson and replay workstation.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Retrieval p95 | <=300 ms trigger path share | >400 ms |
| Memory contribution | <=2 GB | >3 GB |
| Candidate count policy | Exact | Any wrong K |
**Resource limits**: Total system memory remains below 8 GB.
---
### PT-02: Cold-Start Index Load
**Summary**: Verify retrieval readiness supports first fix <30 s.
**Traces to**: AC-NEW-1
**Load scenario**:
- Cold boot 50 runs.
- Cache/index mounted locally.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Index ready p95 | <=10 s | >15 s |
| First retrieval p95 | Fits <30 s first-fix budget | Exceeds budget |
## Security Tests
### ST-01: Stale Candidate Handling
**Summary**: Verify stale imagery cannot silently appear as a trusted retrieval candidate.
**Traces to**: AC-NEW-6
**Attack vector**: Manipulated cache manifest marks stale tile as available.
**Test procedure**:
1. Load cache fixture with stale capture dates.
2. Query against stale region.
**Expected behavior**: Candidate carries stale status; anchor path cannot accept it as `satellite_anchored`.
**Pass criteria**: 0 stale candidates without explicit stale/down-confidence metadata.
## Acceptance Tests
### AT-01: Relocalization Candidate Returned
**Summary**: Verify a relocalization request returns usable candidates for anchor verification.
**Traces to**: AC-3.2, AC-8.6
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Submit sharp-turn query | Retrieval invokes VPR |
| 2 | Read output | Candidate list includes chunk IDs, tile IDs, scores, footprints, freshness |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `cache_vpr_fixture` | PostGIS manifest, COGs, descriptors, FAISS index | Generated/cache fixture | Mission-dependent |
| `aerial_vpr_queries` | Aerial query frames with ground-truth regions | ALTO/AerialVL/representative | Dataset-dependent |
**Setup procedure**: Restore isolated PostgreSQL schema and mount read-only descriptor/index files.
**Teardown procedure**: Drop schema and remove generated retrieval reports.
**Data isolation strategy**: Per-run schema and read-only cache fixture volume.
@@ -0,0 +1,93 @@
# Anchor Verification
## 1. High-Level Overview
**Purpose**: Verify retrieved cache candidates with local feature matching and geometric checks before the safety wrapper considers an absolute anchor.
**Architectural Pattern**: Validation pipeline.
**Upstream dependencies**: Satellite retrieval, camera ingest/calibration, cache/tile lifecycle.
**Downstream consumers**: Safety/anchor wrapper, FDR.
## 2. Internal Interfaces
### Interface: `AnchorVerifier`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `verify` | `AnchorVerificationRequest` | `AnchorDecision` | Yes | `TileUnavailable`, `MatchFailed`, `GeometryFailed` |
| `benchmark_matcher` | `MatcherBenchmarkRequest` | `MatcherBenchmarkReport` | Yes | `ModelUnavailable` |
**Input DTOs**:
```yaml
AnchorVerificationRequest:
frame: FramePacket
candidates: list[VprCandidate]
matcher_profile: enum(aliked, disk, sift_orb_baseline)
```
**Output DTOs**:
```yaml
AnchorDecision:
candidate_id: string
accepted_by_geometry: boolean
estimated_pose_wgs84: object optional
inlier_count: integer
mre_px: number
homography: matrix optional
rejection_reason: string optional
```
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Read candidate COG footprint/window | Triggered only | No | Tile spatial metadata |
| Read matcher model | Startup | No | No |
## 4. Implementation Details
**State Management**: Loads matcher/extractor models and tracks benchmark-selected profile.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| ALIKED/DISK + LightGlue | Learned local matching |
| OpenCV | RANSAC/USAC geometry and error metrics |
**Error Handling Strategy**:
- Low inlier count, high MRE, stale tile, or provenance failure returns a rejected decision with reason.
- SuperPoint can be benchmarked only if legal approval allows its license use.
## 5. Caveats & Edge Cases
**Known limitations**:
- ALIKED-LightGlue is not VIO by itself; it supplies correspondences. A full VIO path still needs state estimation and IMU fusion.
- Optional frame-to-frame VO fallback must be benchmarked separately from cross-domain anchor verification.
**Performance bottlenecks**:
- Learned matching can exceed the Jetson budget if run per-frame; default invocation is trigger-based.
## 6. Dependency Graph
**Must be implemented after**: satellite retrieval candidate DTOs, cache tile access.
**Can be implemented in parallel with**: BASALT VIO adapter.
**Blocks**: accepted satellite-anchor path.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Matcher model unavailable | `lightglue_model_unavailable profile=aliked` |
| WARN | Candidate rejected | `anchor_geometry_rejected mre_px=... inliers=...` |
| INFO | Anchor verified | `anchor_verified tile_id=... mre_px=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment.
@@ -0,0 +1,124 @@
# Test Specification — Anchor Verification
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.1 | 50 m frame-center accuracy via accepted anchors | AT-01 | Covered |
| AC-1.2 | 20 m stretch accuracy via accepted anchors | AT-01 | Covered |
| AC-2.1b | Satellite-anchor registration measured separately | IT-01 | Covered |
| AC-2.2 | <2.5 px cross-domain MRE | IT-01, AT-01 | Covered |
| AC-3.1 | Outlier rejection | ST-01 | Covered |
| AC-4.1 | Trigger-path latency | PT-01 | Covered |
| AC-4.2 | Memory budget | PT-01 | Covered |
| AC-NEW-4 | False-position safety budget | ST-01 | Covered |
| AC-NEW-6 | Stale imagery rejection evidence | IT-02, ST-02 | Covered |
## Blackbox Tests
### IT-01: Cross-Domain Match And RANSAC
**Summary**: Verify ALIKED/DISK-LightGlue plus RANSAC produces measurable anchor evidence.
**Traces to**: AC-2.1b, AC-2.2
**Input data**: UAV frame, retrieved COG candidate window, ground-truth georegistration.
**Expected result**: Accepted anchors have MRE <2.5 px, sufficient inliers, and homography/pose evidence.
**Max execution time**: 2 seconds per candidate set.
---
### IT-02: Stale Candidate Verification
**Summary**: Verify stale or provenance-failed candidates are rejected or marked unsafe.
**Traces to**: AC-NEW-6
**Input data**: Candidate list with stale tile metadata and valid-looking image content.
**Expected result**: `AnchorDecision` is rejected or carries stale/provenance failure; safety wrapper cannot accept it as trusted.
**Max execution time**: 2 seconds per candidate.
## Performance Tests
### PT-01: Local Matcher Runtime
**Summary**: Verify learned matching stays bounded on Jetson when invoked.
**Traces to**: AC-4.1, AC-4.2
**Load scenario**:
- Candidate sets: K=5, K=20, K=50.
- Matcher profiles: ALIKED, DISK, SIFT/ORB baseline.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| K=5 p95 | <=300 ms | >500 ms |
| K=20 p95 | Reported and bounded | Unbounded/no report |
| Memory contribution | <=2 GB | >3 GB |
**Resource limits**: Total Jetson shared memory remains below 8 GB.
## Security Tests
### ST-01: False Match / Impossible Jump Rejection
**Summary**: Verify visually plausible but geographically impossible matches are rejected.
**Traces to**: AC-3.1, AC-NEW-4
**Attack vector**: Candidate tile from wrong region with repetitive texture.
**Test procedure**:
1. Pair query with wrong-region candidate.
2. Run matcher and RANSAC.
3. Pass result to safety wrapper fixture.
**Expected behavior**: Anchor is rejected by geometry or downstream consistency gates.
**Pass criteria**: 0 accepted anchors from wrong-region fixtures.
---
### ST-02: Provenance Failure
**Summary**: Verify unsigned/hash-failed candidate tiles cannot produce accepted anchors.
**Traces to**: AC-NEW-6
**Attack vector**: Tampered COG or sidecar.
**Test procedure**: Run verification against tampered tile fixture.
**Expected behavior**: `AnchorDecision` includes provenance failure and is not accepted.
**Pass criteria**: 0 trusted anchors from tampered tiles.
## Acceptance Tests
### AT-01: Accepted Anchor Accuracy
**Summary**: Verify accepted anchors support position accuracy thresholds.
**Traces to**: AC-1.1, AC-1.2, AC-2.2
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Verify satellite candidate against query | MRE <2.5 px |
| 2 | Convert accepted anchor to WGS84 evidence | Error supports 50 m / 20 m aggregate thresholds |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `anchor_match_fixture` | Query frames, COG windows, expected georegistration | ALTO/AerialVL/project cache | Dataset-dependent |
| `tampered_tile_fixture` | Hash/signature/stale cases | Generated fixture | Small |
**Setup procedure**: Load cache fixture and matcher model profile.
**Teardown procedure**: Remove matcher outputs and reports.
**Data isolation strategy**: Read-only imagery with per-run output folders.
@@ -0,0 +1,90 @@
# Cache And Tile Lifecycle
## 1. High-Level Overview
**Purpose**: Manage offline service-source tiles, manifests, descriptor metadata, freshness/provenance checks, generated tile writes, and post-flight sync packaging.
**Architectural Pattern**: Repository + policy gate.
**Upstream dependencies**: Satellite Service cache packages, safety/anchor wrapper, camera ingest/calibration.
**Downstream consumers**: Satellite retrieval, anchor verification, FDR, post-flight sync.
## 2. Internal Interfaces
### Interface: `CacheRepository`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `validate_cache` | `CacheValidationRequest` | `CacheValidationReport` | No | `ManifestInvalid`, `SignatureInvalid` |
| `get_tile_window` | `TileWindowRequest` | `TileWindow` | No | `TileUnavailable`, `TileRejected` |
| `write_generated_tile` | `GeneratedTileRequest` | `GeneratedTileRecord` | Yes | `TileWriteRejected`, `StorageFull` |
| `package_sync` | `SyncPackageRequest` | `SyncPackage` | Yes | `PackageFailed` |
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Tile by footprint/time/freshness | Per retrieval/anchor | Yes during relocalization | Spatial/time indexes |
| Descriptor metadata by chunk | Per retrieval | Yes during relocalization | Chunk ID index |
| Generated tile by mission/sector | Post-flight | No | Mission ID index |
### Caching Strategy
| Data | Cache Type | TTL | Invalidation |
|------|------------|-----|--------------|
| Manifest metadata | PostgreSQL/PostGIS query cache / process cache | Mission duration | New mission cache load |
| Sidecar verification | In-memory result cache | Mission duration | File hash change |
### Storage Estimates
| Table/Collection | Est. Row Count | Row Size | Total Size | Growth Rate |
|------------------|----------------|----------|------------|-------------|
| Cache manifest tiles | Mission-dependent | Small metadata | Within ~10 GB package with imagery | Per mission |
| Generated tiles | Flight-dependent | Metadata + COG payload | FDR/cache budget constrained | Per flight |
## 4. Implementation Details
**State Management**: Owns PostgreSQL/PostGIS manifest connection, sidecar verification state, and generated tile staging area.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| PostgreSQL + PostGIS | Manifest, spatial metadata, freshness queries, and generated-tile metadata |
| GDAL/rasterio candidate | COG read/write |
| Cryptographic hash/signature library | Sidecar validation |
**Error Handling Strategy**:
- Invalid signatures/hashes reject tiles.
- Storage-full blocks generated tile writes without affecting localization output.
- Cache validation failure blocks mission cache usage.
## 5. Caveats & Edge Cases
**Known limitations**:
- JSON-only manifests are avoided for scale and queryability, but signed JSON sidecars remain required for audit/interchange.
- PostgreSQL/PostGIS must be available locally before flight; runtime cannot depend on a remote DB link.
**Potential race conditions**:
- Generated tile and PostgreSQL manifest update must be atomic enough to avoid orphan trusted metadata.
## 6. Dependency Graph
**Must be implemented after**: data model schema decisions.
**Can be implemented in parallel with**: camera ingest, MAVLink integration.
**Blocks**: satellite retrieval, anchor verification, generated tile lifecycle.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Cache package invalid | `cache_manifest_invalid reason=signature` |
| WARN | Tile rejected | `tile_rejected reason=stale tile_id=...` |
| INFO | Generated tile staged | `generated_tile_written tile_id=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment plus cache validation report.
@@ -0,0 +1,167 @@
# Test Specification — Cache And Tile Lifecycle
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-4.2 | Memory/storage pressure | PT-01 | Covered |
| AC-8.1 | Resolution at cache interface | IT-01 | Covered |
| AC-8.2 | Freshness thresholds | IT-02, ST-01 | Covered |
| AC-8.3 | Preloaded/preprocessed offline cache | IT-01 | Covered |
| AC-8.4 | Mid-flight tile generation/write-back | IT-03, AT-01 | Covered |
| AC-8.5 | Persistent imagery policy | ST-02 | Covered |
| AC-8.6 | VPR chunk metadata | IT-04 | Covered |
| AC-NEW-3 | FDR/tile storage cap interaction | PT-01 | Covered |
| AC-NEW-6 | Imagery freshness enforcement | IT-02, ST-01 | Covered |
| AC-NEW-7 | Cache-poisoning safety budget | ST-03, AT-01 | Covered |
## Blackbox Tests
### IT-01: Mission Cache Validation
**Summary**: Verify preloaded COGs, PostGIS metadata, sidecars, descriptors, and indexes validate before flight.
**Traces to**: AC-8.1, AC-8.3
**Input data**: Mission cache package with COGs, signed JSON sidecars, PostGIS manifest seed, FAISS index files.
**Expected result**: Valid cache passes resolution, hash, signature, descriptor-reference, and spatial coverage checks.
**Max execution time**: 5 minutes per cache fixture.
---
### IT-02: Freshness Gate
**Summary**: Verify active-conflict and stable-rear freshness rules.
**Traces to**: AC-8.2, AC-NEW-6
**Input data**: Tiles at fresh, grace, and stale ages for both sector classes.
**Expected result**: Fresh tiles pass, grace tiles are down-confidence weighted if allowed, stale tiles are rejected and cannot emit `satellite_anchored`.
**Max execution time**: 2 minutes.
---
### IT-03: Generated Tile Write
**Summary**: Verify generated tiles are written only when pose and frame quality gates pass.
**Traces to**: AC-8.4
**Input data**: Frame metadata, pose covariance <=3 m, <=5 m, and >5 m.
**Expected result**: <=3 m writes full-quality candidate, 3-5 m writes soft candidate, >5 m rejects write.
**Max execution time**: 2 minutes.
---
### IT-04: VPR Chunk Metadata
**Summary**: Verify chunk metadata supports retrieval rules.
**Traces to**: AC-8.6
**Input data**: Operational-area cache manifest.
**Expected result**: Chunks are 600-800 m equivalent footprint with 40-50% overlap and multi-scale active-sector descriptors.
**Max execution time**: 2 minutes.
## Performance Tests
### PT-01: Cache And FDR Storage Budget
**Summary**: Verify cache metadata and generated tile writes stay within storage/memory budgets.
**Traces to**: AC-4.2, AC-NEW-3
**Load scenario**:
- Mission cache: up to operational budget.
- Generated tiles: 8-hour synthetic flight.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Persistent cache | <=10 GB unless split budget approved | >budget without report |
| FDR + generated artifacts | <=64 GB per flight | >64 GB without rollover |
| DB query p95 | <=50 ms for indexed tile lookup | >150 ms |
**Resource limits**: PostgreSQL/PostGIS stays within total system 8 GB memory budget.
## Security Tests
### ST-01: Signed Manifest Enforcement
**Summary**: Verify unsigned/tampered/stale manifests are rejected.
**Traces to**: AC-8.2, AC-NEW-6
**Attack vector**: Tampered sidecar, bad hash, unsigned manifest.
**Test procedure**: Load invalid cache variants and run validation.
**Expected behavior**: Invalid tiles are rejected and logged.
**Pass criteria**: 0 invalid cache entries become available to retrieval/anchor verification.
---
### ST-02: Raw Frame Persistence Check
**Summary**: Verify cache lifecycle persists tiles, not raw frames.
**Traces to**: AC-8.5
**Attack vector**: Raw frames accidentally stored as generated artifacts.
**Test procedure**: Run tile generation and inspect cache/FDR outputs.
**Expected behavior**: Only COG tiles, sidecars, manifests, and allowed failed-frame thumbnails exist.
**Pass criteria**: No raw full-resolution frames retained.
---
### ST-03: Cache Poisoning Gate
**Summary**: Verify misaligned generated tiles cannot become trusted basemap.
**Traces to**: AC-NEW-7
**Attack vector**: Over-confident pose writes misaligned generated tile.
**Test procedure**: Inject deflated covariance and wrong pose during tile write.
**Expected behavior**: Tile is rejected or marked candidate/soft; never promoted to trusted by onboard component.
**Pass criteria**: 0 direct trusted basemap promotions onboard.
## Acceptance Tests
### AT-01: Generated Tile Package For Satellite Service
**Summary**: Verify post-flight sync package contains valid generated tiles and metadata.
**Traces to**: AC-8.4, AC-NEW-7
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Write generated candidate tile | COG + sidecar + PostGIS manifest row created |
| 2 | Package post-flight sync | Manifest delta includes trust level and parent covariance |
| 3 | Inspect package | No tile is marked trusted basemap by onboard runtime |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `cache_integrity_fixtures` | Valid/stale/unsigned/hash-mismatched manifests | Generated fixture | Small |
| `mission_cache_fixture` | COGs, descriptors, PostGIS seed | Satellite Service stub | Mission-dependent |
**Setup procedure**: Restore isolated PostgreSQL/PostGIS schema and mount cache fixture read-only except generated-tile staging.
**Teardown procedure**: Drop schema and delete generated staging volume.
**Data isolation strategy**: Per-run mission ID, schema, and staging directory.
@@ -0,0 +1,69 @@
# MAVLink And GCS Integration
## 1. High-Level Overview
**Purpose**: Subscribe to flight-controller telemetry, emit `GPS_INPUT`, and send downsampled QGroundControl status/failsafe messages.
**Architectural Pattern**: Protocol adapter.
**Upstream dependencies**: ArduPilot Plane FC, safety/anchor wrapper.
**Downstream consumers**: BASALT VIO adapter, safety/anchor wrapper, QGC, FDR.
## 2. Internal Interfaces
### Interface: `MavlinkGateway`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `subscribe_telemetry` | `TelemetrySubscriptionRequest` | `TelemetrySample` | Yes | `MavlinkDisconnected` |
| `emit_gps_input` | `PositionEstimate` | `EmitResult` | Yes | `MavlinkDisconnected`, `InvalidGpsInput` |
| `emit_status` | `GcsStatusMessage` | `EmitResult` | Yes | `MavlinkDisconnected` |
## 3. Data Access Patterns
No persistent data ownership; telemetry and emitted packets are mirrored to FDR.
## 4. Implementation Details
**State Management**: Maintains MAVLink connection status, source/system IDs, and rate limiters for QGC status.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| MAVSDK | Telemetry subscriptions |
| pymavlink | Exact `GPS_INPUT` field emission |
**Error Handling Strategy**:
- Invalid `GPS_INPUT` fields are rejected before emission.
- Connection loss is surfaced to wrapper/FDR and does not silently drop safety events.
## 5. Caveats & Edge Cases
**Known limitations**:
- v1 emits `GPS_INPUT` only, not velocity-target navigation commands.
- Plane parameter configuration must be validated in SITL before hardware use.
**Performance bottlenecks**:
- Status text must be rate-limited to avoid telemetry noise.
## 6. Dependency Graph
**Must be implemented after**: position estimate DTO and MAVLink output contract.
**Can be implemented in parallel with**: cache lifecycle, camera ingest.
**Blocks**: SITL integration and production FC output.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | MAVLink disconnected | `mavlink_disconnected endpoint=...` |
| WARN | Invalid output rejected | `gps_input_invalid reason=...` |
| INFO | FC link established | `mavlink_connected system_id=...` |
**Log format**: FDR structured event.
**Log storage**: FDR segment and optional tlog.
@@ -0,0 +1,176 @@
# Test Specification — MAVLink And GCS Integration
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-4.3 | v1 GPS_INPUT only for ArduPilot Plane | IT-01, AT-01 | Covered |
| AC-4.4 | Frame-by-frame streaming | PT-01 | Covered |
| AC-4.5 | Updated estimates/corrections | IT-02 | Covered |
| AC-5.1 | FC state initialization telemetry | IT-03 | Covered |
| AC-5.2 | Plane SITL fallback | IT-04 | Covered |
| AC-6.1 | QGC status 1-2 Hz | IT-05, PT-02 | Covered |
| AC-6.2 | GCS command ingress | IT-06, ST-01 | Covered |
| AC-6.3 | WGS84 output | IT-01 | Covered |
| AC-NEW-2 | Spoofing promotion <3 s | IT-04 | Covered |
| AC-NEW-8 | Blackout/failsafe status | IT-05 | Covered |
## Blackbox Tests
### IT-01: GPS_INPUT Field Mapping
**Summary**: Verify `PositionEstimate` maps to valid MAVLink `GPS_INPUT`.
**Traces to**: AC-4.3, AC-6.3
**Input data**: Position estimates across all source labels.
**Expected result**: v1 emits `GPS_INPUT` only, no `ODOMETRY`; WGS84 lat/lon/alt, fix type, ignore flags, and accuracy fields match contract.
**Max execution time**: 2 minutes.
---
### IT-02: Correction Emission
**Summary**: Verify updated estimates can be emitted without batching.
**Traces to**: AC-4.5
**Input data**: Original VO estimate followed by anchor-corrected estimate.
**Expected result**: Both estimates are emitted in order with updated accuracy/source label.
**Max execution time**: 2 minutes.
---
### IT-03: FC Telemetry Subscription
**Summary**: Verify telemetry needed for initialization and VIO is available.
**Traces to**: AC-5.1
**Input data**: Plane SITL or MAVLink replay with EKF position, IMU, attitude, airspeed, altitude.
**Expected result**: Normalized `TelemetrySample` stream includes required fields and timestamps.
**Max execution time**: 5 minutes.
---
### IT-04: Spoofing And Fallback In Plane SITL
**Summary**: Verify spoofing and no-estimate behavior in ArduPilot Plane SITL.
**Traces to**: AC-5.2, AC-NEW-2
**Input data**: Plane SITL production parameter set and spoofing trace.
**Expected result**: Own estimate promotion occurs within <3 s; fallback/no-estimate behavior matches Plane parameters.
**Max execution time**: 10 minutes.
---
### IT-05: QGC Blackout Status
**Summary**: Verify degraded-mode messages are visible at required rate.
**Traces to**: AC-6.1, AC-NEW-8
**Input data**: Safety wrapper emits blackout and failsafe statuses.
**Expected result**: QGC observer sees `VISUAL_BLACKOUT_IMU_ONLY` at 1-2 Hz and `VISUAL_BLACKOUT_FAILSAFE` at threshold.
**Max execution time**: 10 minutes.
---
### IT-06: Operator Relocalization Hint
**Summary**: Verify GCS command ingress can carry approximate relocalization hints.
**Traces to**: AC-6.2
**Input data**: STATUSTEXT/NAMED_VALUE_FLOAT/custom dialect hint fixture.
**Expected result**: Valid hint is parsed and forwarded to retrieval/safety logic; invalid hint is rejected.
**Max execution time**: 5 minutes.
## Performance Tests
### PT-01: Frame-Rate Emission
**Summary**: Verify output is streamed frame-by-frame and not batched.
**Traces to**: AC-4.4
**Load scenario**:
- Input estimate rate: target frame rate.
- Duration: 30 minutes.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Output delay p95 | <=25 ms after wrapper output | >100 ms |
| Missing messages | 0 except upstream dropped frames | Any silent drop |
---
### PT-02: QGC Status Rate Limit
**Summary**: Verify QGC status is downsampled without losing critical transitions.
**Traces to**: AC-6.1
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Status rate | 1-2 Hz while active | <1 Hz or >2 Hz sustained |
| Critical transition delay | <=1 s | >2 s |
## Security Tests
### ST-01: MAVLink Source And Command Validation
**Summary**: Verify unauthorized or malformed MAVLink messages are rejected.
**Traces to**: AC-6.2
**Attack vector**: Malicious source sends spoofed command or GPS data.
**Test procedure**:
1. Send valid command from allowed source.
2. Send same command from disallowed source/system ID.
3. Send malformed values.
**Expected behavior**: Allowed command is accepted; disallowed/malformed messages are rejected and logged.
**Pass criteria**: 0 unauthorized commands affect localization state.
## Acceptance Tests
### AT-01: Plane SITL Output Acceptance
**Summary**: Verify ArduPilot Plane receives and uses v1 `GPS_INPUT` as configured.
**Traces to**: AC-4.3
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Start Plane SITL with production params | FC accepts external GPS substitute config |
| 2 | Emit `GPS_INPUT` estimate | Message is received with expected fields |
| 3 | Observe wire | `ODOMETRY` is absent in v1 |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `sitl_spoofing_scenarios` | GPS loss/spoofing traces | Generated SITL | Small |
| `mavlink_output_fixtures` | PositionEstimate cases | Generated fixture | Small |
**Setup procedure**: Start SITL/QGC observer or replay MAVLink log.
**Teardown procedure**: Stop processes and archive tlogs.
**Data isolation strategy**: Unique MAVLink ports and run IDs per test.
@@ -0,0 +1,79 @@
# FDR And Observability
## 1. High-Level Overview
**Purpose**: Record bounded, replayable mission evidence and expose runtime health/status events for analysis and operator awareness.
**Architectural Pattern**: Append-only event sink + exporter.
**Upstream dependencies**: All runtime components.
**Downstream consumers**: Validation harness, post-flight audit tools, QGC status through MAVLink component.
## 2. Internal Interfaces
### Interface: `FlightRecorder`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `append_event` | `FdrEvent` | `AppendResult` | Yes | `RecorderUnavailable`, `StorageFull` |
| `rollover` | `RolloverRequest` | `FdrSegmentInfo` | No | `RolloverFailed` |
| `export` | `ExportRequest` | `ExportResult` | Yes | `ExportFailed` |
## 3. Data Access Patterns
| Query | Frequency | Hot Path | Index Needed |
|-------|-----------|----------|--------------|
| Append event | High | Yes | Append index only |
| Export by time/type | Post-flight | No | Time/type index |
### Storage Estimates
| Table/Collection | Est. Row Count | Row Size | Total Size | Growth Rate |
|------------------|----------------|----------|------------|-------------|
| FDR events | Flight-dependent | Mixed | <=64 GB per 8 h | Per flight |
## 4. Implementation Details
**State Management**: Owns active segment, rollover policy, and export state.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| PostgreSQL client | Event metadata, time/type indexes, mission query surface |
| CBOR writer | Bounded runtime payload segments |
| Parquet writer | Optional post-flight export |
**Error Handling Strategy**:
- Storage-full emits critical status and starts rollover/retention behavior.
- Append failures are surfaced to the caller and health system.
## 5. Caveats & Edge Cases
**Known limitations**:
- Raw frames are not retained by default; only metadata, decisions, hashes, and occlusion/blackout status are recorded.
- PostgreSQL availability is required for indexed FDR metadata; CBOR payload segments preserve bounded append behavior for high-volume data.
**Performance bottlenecks**:
- FDR appends must not block hot-path localization.
## 6. Dependency Graph
**Must be implemented after**: event schema and key DTOs.
**Can be implemented in parallel with**: MAVLink integration.
**Blocks**: release evidence and most validation reports.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Recorder unavailable | `fdr_unavailable path=...` |
| WARN | Rollover occurs | `fdr_rollover segment=...` |
| INFO | Export complete | `fdr_export_complete format=parquet` |
**Log format**: FDR event metadata plus local health logs.
**Log storage**: PostgreSQL FDR event tables plus CBOR segment payloads.
@@ -0,0 +1,166 @@
# Test Specification — FDR And Observability
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.3 | Anchor age/drift evidence | IT-01 | Covered |
| AC-1.4 | Confidence/source label retained | IT-01 | Covered |
| AC-4.4 | Per-frame local stream evidence | IT-01, PT-01 | Covered |
| AC-5.2 | Failure logging | IT-02 | Covered |
| AC-6.1 | QGC/status evidence | IT-03 | Covered |
| AC-8.4 | Generated tile audit | IT-04 | Covered |
| AC-8.5 | No raw frame retention | ST-01 | Covered |
| AC-NEW-3 | FDR retention and 64 GB cap | PT-01, AT-01 | Covered |
| AC-NEW-4 | False-position forensics | IT-05 | Covered |
| AC-NEW-5 | Thermal/throttle logging | IT-06 | Covered |
| AC-NEW-8 | Blackout/failsafe logging | IT-02, IT-03 | Covered |
## Blackbox Tests
### IT-01: Per-Estimate Event Capture
**Summary**: Verify every estimate stores covariance, source label, anchor age, and emitted output metadata.
**Traces to**: AC-1.3, AC-1.4, AC-4.4
**Input data**: Position estimate stream with satellite, VO, and dead-reckoned labels.
**Expected result**: PostgreSQL event index and CBOR payload segments contain all required fields with monotonic timestamps.
**Max execution time**: 5 minutes.
---
### IT-02: Failure And Blackout Logging
**Summary**: Verify no-estimate and blackout transitions are recorded.
**Traces to**: AC-5.2, AC-NEW-8
**Input data**: No-estimate gap and total blackout sequence.
**Expected result**: FDR records start, every degraded estimate, failsafe threshold, and recovery reason.
**Max execution time**: 10 minutes.
---
### IT-03: QGC Status Audit
**Summary**: Verify operator-visible status has matching FDR evidence.
**Traces to**: AC-6.1, AC-NEW-8
**Input data**: QGC status messages from MAVLink component.
**Expected result**: FDR contains status text, timestamp, and mode context.
**Max execution time**: 5 minutes.
---
### IT-04: Generated Tile Audit Trail
**Summary**: Verify tile-write decisions are recorded with parent covariance and trust level.
**Traces to**: AC-8.4
**Input data**: Accepted and rejected generated tile write decisions.
**Expected result**: FDR includes tile ID, parent covariance, trust level, sidecar hash, and rejection reason where applicable.
**Max execution time**: 5 minutes.
---
### IT-05: False-Position Investigation Bundle
**Summary**: Verify enough evidence exists to investigate a false-position event.
**Traces to**: AC-NEW-4
**Input data**: Simulated false anchor rejection and covariance growth sequence.
**Expected result**: Export includes estimates, anchor decisions, residuals, covariance, and emitted MAVLink fields.
**Max execution time**: 5 minutes.
---
### IT-06: Thermal/Throttle Event Capture
**Summary**: Verify resource health events are recorded.
**Traces to**: AC-NEW-5
**Input data**: Synthetic thermal/throttle metric stream.
**Expected result**: FDR records CPU/GPU/temp/throttle status and QGC warning trigger.
**Max execution time**: 5 minutes.
## Performance Tests
### PT-01: 8-Hour FDR Load
**Summary**: Verify FDR storage and append behavior under full mission load.
**Traces to**: AC-4.4, AC-NEW-3
**Load scenario**:
- Duration: 8 hours synthetic.
- Inputs: 3 Hz estimates, full-rate IMU, MAVLink tlog, health metrics, tile events.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| Total FDR size | <=64 GB | >64 GB without rollover |
| Append latency p95 | <=10 ms async enqueue | >25 ms |
| Silent payload loss | 0 | Any unlogged loss |
**Resource limits**: FDR must not block hot-path localization.
## Security Tests
### ST-01: Raw Frame Retention Audit
**Summary**: Verify FDR does not store raw full-resolution frames.
**Traces to**: AC-8.5
**Attack vector**: Debug logging accidentally persists raw camera frames.
**Test procedure**:
1. Run normal replay and failed tile-generation replay.
2. Inspect FDR payloads and output directories.
**Expected behavior**: Only metadata, hashes, estimates, tiles, and allowed low-rate failed-frame thumbnails are retained.
**Pass criteria**: No raw nav/AI camera frame payloads in normal FDR.
## Acceptance Tests
### AT-01: FDR Export
**Summary**: Verify post-flight export creates usable audit artifacts.
**Traces to**: AC-NEW-3
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Complete synthetic flight | Segment rollover is logged and cap respected |
| 2 | Export FDR summary | Markdown/CSV/Parquet optional artifacts are produced |
| 3 | Query PostgreSQL index | Events can be filtered by time/type/mission |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `fdr_synthetic_load` | Estimate, IMU, MAVLink, health, tile events | Generated fixture | Large |
| `incident_fixture` | False-position and blackout evidence | Generated fixture | Small |
**Setup procedure**: Create isolated PostgreSQL schema and FDR segment directory.
**Teardown procedure**: Export report, then remove schema and segment directory.
**Data isolation strategy**: Per-run mission ID, schema, and FDR directory.
@@ -0,0 +1,86 @@
# Validation Harness
## 1. High-Level Overview
**Purpose**: Drive black-box replay, public dataset, SITL, Jetson, and representative validation through the runtime's public interfaces.
**Architectural Pattern**: Test harness / scenario runner.
**Upstream dependencies**: Test data fixtures, public datasets, SITL, Jetson environment.
**Downstream consumers**: CI/CD pipeline, release evidence review.
## 2. Internal Interfaces
### Interface: `ScenarioRunner`
| Method | Input | Output | Async | Error Types |
|--------|-------|--------|-------|-------------|
| `run_scenario` | `ScenarioRequest` | `ScenarioReport` | Yes | `FixtureInvalid`, `RuntimeFailed`, `ThresholdFailed` |
| `validate_fixture` | `FixtureRequest` | `FixtureValidationReport` | No | `FixtureInvalid` |
**Input DTOs**:
```yaml
ScenarioRequest:
scenario_id: string
execution_environment: enum(replay, sitl, jetson, representative)
fixture_paths: list[string]
```
**Output DTOs**:
```yaml
ScenarioReport:
scenario_id: string
result: enum(pass, fail, blocked)
metrics: object
artifacts: list[path]
failure_reason: string optional
```
## 3. Data Access Patterns
Reads versioned fixtures and writes reports. Does not import runtime internals.
## 4. Implementation Details
**State Management**: Per-run temporary directories and report aggregation.
**Key Dependencies**:
| Library | Purpose |
|---------|---------|
| pytest or equivalent | Test orchestration |
| pymavlink/log parser | SITL and output validation |
| Docker/compose runner | Replay/SITL environment |
**Error Handling Strategy**:
- Fixture gaps are reported as blocked, not passed.
- Threshold failures include metrics and artifacts.
## 5. Caveats & Edge Cases
**Known limitations**:
- Public datasets are not final acceptance evidence unless representative and license-compatible.
- Missing synchronized target data remains a final acceptance blocker.
## 6. Dependency Graph
**Must be implemented after**: public interfaces are defined.
**Can be implemented in parallel with**: runtime components using mocks/fixtures only after interfaces are stable.
**Blocks**: CI/release gates.
## 7. Logging Strategy
| Log Level | When | Example |
|-----------|------|---------|
| ERROR | Runtime/test process fails | `scenario_failed id=... reason=...` |
| WARN | Fixture blocked | `fixture_blocked missing=...` |
| INFO | Scenario complete | `scenario_complete id=... result=pass` |
**Log format**: Test report CSV/Markdown plus structured runner logs.
**Log storage**: `test-results/`.
@@ -0,0 +1,232 @@
# Test Specification — Validation Harness
## Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|-------|---------------------|----------|----------|
| AC-1.1 through AC-1.4 | Position accuracy, drift, confidence | IT-01, AT-01 | Covered |
| AC-2.1a/b, AC-2.2 | VO and satellite registration | IT-02, IT-03 | Covered |
| AC-3.1 through AC-3.5 | Resilience edge cases | IT-04, IT-05 | Covered |
| AC-4.1 through AC-4.5 | Latency, memory, MAVLink streaming | PT-01, IT-06 | Covered |
| AC-5.1 through AC-5.3 | Startup/failsafe/reboot | IT-07 | Covered |
| AC-6.1 through AC-6.3 | QGC/GCS/WGS84 | IT-06 | Covered |
| AC-7.1, AC-7.2 | Object coordinate contract | IT-08 | Covered at system boundary |
| AC-8.1 through AC-8.6 | Offline cache, freshness, tiles, VPR | IT-03, IT-09, ST-01 | Covered |
| AC-NEW-1 through AC-NEW-8 | Cold start, spoofing, FDR, false-position, thermal, freshness, poisoning, blackout | IT-05, IT-07, PT-02, ST-01, AT-02 | Covered |
## Blackbox Tests
### IT-01: Still-Image Accuracy Runner
**Summary**: Verify project still-image replay reports frame-center accuracy.
**Traces to**: AC-1.1, AC-1.2, AC-1.4
**Input data**: Project mapped images and `expected_results/results_report.md`.
**Expected result**: Report includes per-image error, aggregate 50 m/20 m pass rates, covariance, source label, and anchor age.
**Max execution time**: 15 minutes.
---
### IT-02: Public VIO Replay Runner
**Summary**: Verify public/representative synchronized data can drive BASALT/wrapper tests.
**Traces to**: AC-1.3, AC-2.1a, AC-2.2
**Input data**: MUN-FRL preferred slice or representative synchronized dataset.
**Expected result**: Runner validates trajectory, VIO registration, latency, and covariance calibration.
**Max execution time**: Dataset-dependent.
---
### IT-03: Satellite Anchor Replay Runner
**Summary**: Verify VPR and anchor verification test scenarios are executable.
**Traces to**: AC-2.1b, AC-2.2, AC-8.1, AC-8.2, AC-8.6
**Input data**: ALTO/AerialVL/representative aerial localization fixture plus cache.
**Expected result**: Runner reports retrieval recall, MRE, accepted/rejected anchors, and freshness behavior.
**Max execution time**: Dataset-dependent.
---
### IT-04: Outlier/Sharp-Turn/Disconnected Runner
**Summary**: Verify resilience scenarios are executable and reported.
**Traces to**: AC-3.1, AC-3.2, AC-3.3, AC-3.4
**Input data**: Synthetic and public disconnected-segment fixtures.
**Expected result**: Runner validates relocalization and records degraded-mode timelines.
**Max execution time**: 30 minutes.
---
### IT-05: Blackout And Spoofing Runner
**Summary**: Verify total blackout plus spoofing scenarios can be driven through SITL/replay.
**Traces to**: AC-3.5, AC-NEW-2, AC-NEW-8
**Input data**: Plane SITL spoofing scenario with 5 s, 15 s, and 35 s blackout windows.
**Expected result**: Runner measures <=400 ms mode switch, <3 s promotion, monotonic covariance, and failsafe thresholds.
**Max execution time**: 30 minutes.
---
### IT-06: MAVLink/QGC Contract Runner
**Summary**: Verify MAVLink output and GCS status assertions are automated.
**Traces to**: AC-4.3, AC-4.4, AC-4.5, AC-6.1, AC-6.2, AC-6.3
**Input data**: Plane SITL, QGC observer/log parser, position fixtures.
**Expected result**: Runner validates v1 GPS_INPUT-only output, WGS84 coordinates, status rate, and command ingress.
**Max execution time**: 60 minutes.
---
### IT-07: Startup/Reboot Runner
**Summary**: Verify cold-start and reboot scenarios are measurable.
**Traces to**: AC-5.1, AC-5.2, AC-5.3, AC-NEW-1
**Input data**: 50 cold-start runs and companion reboot trace.
**Expected result**: First valid `GPS_INPUT` <30 s p95; reboot reinitializes from FC state.
**Max execution time**: Runset-dependent.
---
### IT-08: Object Coordinate Contract Runner
**Summary**: Verify AI-camera object coordinate request contract at system boundary.
**Traces to**: AC-7.1, AC-7.2
**Input data**: Frame-center estimate, object pixel/angle fixture, gimbal angle, altitude.
**Expected result**: Output coordinate includes frame-center-consistent accuracy and maneuvering-flight projection error bound.
**Max execution time**: 5 minutes.
---
### IT-09: Cache And Tile Lifecycle Runner
**Summary**: Verify cache, generated tiles, and storage tests are executable.
**Traces to**: AC-8.3, AC-8.4, AC-8.5, AC-NEW-6, AC-NEW-7
**Input data**: Cache integrity fixtures, generated tile scenarios, PostGIS manifest.
**Expected result**: Runner validates cache load, tile write gates, no raw-frame retention, stale rejection, and poisoning budget evidence.
**Max execution time**: Dataset-dependent.
## Performance Tests
### PT-01: End-To-End Release Gate Runner
**Summary**: Verify performance and resource tests can run in the proper environment.
**Traces to**: AC-4.1, AC-4.2, AC-NEW-5
**Load scenario**:
- Environments: replay, Jetson hardware, SITL.
- Duration: smoke, nightly, and release-gate profiles.
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| End-to-end p95 | <400 ms | >=400 ms |
| Memory | <8 GB | >=8 GB |
| Thermal throttle | 0 events in release gate | Any throttle event |
---
### PT-02: FDR/Storage Runner
**Summary**: Verify 8-hour storage/endurance test orchestration.
**Traces to**: AC-NEW-3
| Metric | Target | Failure Threshold |
|--------|--------|-------------------|
| FDR cap | <=64 GB | >64 GB |
| Rollover logging | Complete | Missing rollover event |
## Security Tests
### ST-01: Security Fixture Runner
**Summary**: Verify stale/tampered cache, spoofed MAVLink, and false-anchor scenarios are automated.
**Traces to**: AC-NEW-4, AC-NEW-6, AC-NEW-7
**Attack vector**: Cache tampering, stale imagery, spoofed GPS, impossible anchors.
**Test procedure**:
1. Load each security fixture.
2. Run scenario through public runtime interfaces.
3. Validate output labels, FDR, and rejection reasons.
**Expected behavior**: No tampered/stale/spoofed input produces a trusted false fix.
**Pass criteria**: 0 accepted unsafe anchors or spoofed GPS promotions outside gates.
## Acceptance Tests
### AT-01: Traceability Completeness Report
**Summary**: Verify every AC has executable or explicitly blocked test coverage.
**Traces to**: All ACs
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Read traceability matrix | All ACs mapped to tests |
| 2 | Run fixture validation | Missing public/representative data is reported as blocked, not passed |
---
### AT-02: Release Evidence Bundle
**Summary**: Verify release evidence can be assembled.
**Traces to**: AC-NEW-1 through AC-NEW-8
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | Run release profile | Reports, tlogs, FDR summaries, cache reports are produced |
| 2 | Collate artifacts | Bundle contains pass/fail status and residual blockers |
## Test Data Management
| Data Set | Description | Source | Size |
|----------|-------------|--------|------|
| `project_60_still_images` | Frame-center geolocation smoke | Project data | Project size |
| `public_dataset_slices` | MUN-FRL/ALTO/Kagaru/EPFL/AerialVL as licensed | Public pinned fixtures | Dataset-dependent |
| `sitl_scenarios` | Plane spoofing/failsafe traces | Generated | Small |
| `security_fixtures` | Stale/tampered/cache poisoning cases | Generated | Small |
**Setup procedure**: Create isolated run directory, restore PostgreSQL schema, mount fixtures read-only, and start requested environment.
**Teardown procedure**: Stop environments, archive reports, drop run schema, and delete temp volumes.
**Data isolation strategy**: Unique run ID, schema, ports, cache staging directory, and FDR directory per scenario.