mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 10:31:12 +00:00
[AZ-219] [AZ-228] Generalize VIO component layout
Keep VIO package and native bridge paths backend-neutral so BASALT remains an implementation choice rather than a component boundary. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,141 @@
|
||||
# Test Specification — VIO Adapter
|
||||
|
||||
## Acceptance Criteria Traceability
|
||||
|
||||
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|
||||
|-------|---------------------|----------|----------|
|
||||
| AC-1.3 | Drift between anchors and anchor-age reporting input | IT-02, AT-01 | Covered |
|
||||
| AC-2.1a | >95% VO registration on normal segments | IT-01, AT-01 | Covered |
|
||||
| AC-2.2 | <1.0 px VO MRE | IT-01 | Covered |
|
||||
| AC-3.1 | Handles 350 m outliers/tilt | IT-03 | Covered |
|
||||
| AC-3.2 | Sharp turns trigger relocalization path | IT-04 | Covered |
|
||||
| AC-3.4 | Loss threshold feeds relocalization/dead reckoning | IT-04 | Covered |
|
||||
| AC-4.1 | Hot-path latency | PT-01 | Covered |
|
||||
| AC-4.2 | Jetson memory budget | PT-01 | Covered |
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
### IT-01: Public Dataset VIO Replay
|
||||
|
||||
**Summary**: Verify the VIO adapter produces relative motion for synchronized camera/IMU replay.
|
||||
|
||||
**Traces to**: AC-2.1a, AC-2.2
|
||||
|
||||
**Input data**: Derkachi cropped nadir video + `SCALED_IMU2` + `GLOBAL_POSITION_INT`, MUN-FRL preferred slice, or representative synchronized nav-camera + IMU + ground truth.
|
||||
|
||||
**Expected result**: VO registration succeeds for >95% of normal usable frames; frame-to-frame MRE <1.0 px where ground-truth/feature evaluation supports it. Derkachi runs are accepted as calibration-limited until intrinsics, distortion, and camera-to-body transform are pinned.
|
||||
|
||||
**Max execution time**: Dataset-dependent; report per-frame latency.
|
||||
|
||||
**Dependencies**: Camera ingest, MAVLink telemetry/replay, calibration fixtures.
|
||||
|
||||
---
|
||||
|
||||
### IT-02: Relative Drift Reporting
|
||||
|
||||
**Summary**: Verify adapter emits state needed for wrapper drift and anchor-age accounting.
|
||||
|
||||
**Traces to**: AC-1.3
|
||||
|
||||
**Input data**: Segment with two known satellite anchors and IMU samples.
|
||||
|
||||
**Expected result**: Adapter emits continuous `VioStatePacket` values with timestamps and quality, enabling wrapper to compare VO extrapolation to next anchor.
|
||||
|
||||
**Max execution time**: Dataset-dependent.
|
||||
|
||||
**Dependencies**: Safety wrapper test harness.
|
||||
|
||||
---
|
||||
|
||||
### IT-03: Tilt/Outlier Robustness
|
||||
|
||||
**Summary**: Verify adapter reports degraded tracking without false success under tilt/outlier cases.
|
||||
|
||||
**Traces to**: AC-3.1
|
||||
|
||||
**Input data**: Replay segment with synthetic +/-20 degree tilt and up to 350 m apparent outlier.
|
||||
|
||||
**Expected result**: Adapter either tracks with quality metadata or emits `TrackingLost`; it never hides a failure as high-quality VIO.
|
||||
|
||||
**Max execution time**: 15 minutes per fixture.
|
||||
|
||||
---
|
||||
|
||||
### IT-04: Sharp Turn / Loss Signal
|
||||
|
||||
**Summary**: Verify sharp turns and disconnected visual overlap produce wrapper-visible failure signals.
|
||||
|
||||
**Traces to**: AC-3.2, AC-3.4
|
||||
|
||||
**Input data**: <5% overlap sequence with heading change <70 degrees.
|
||||
|
||||
**Expected result**: Adapter emits low tracking quality or `TrackingLost` within the loss window, allowing relocalization trigger.
|
||||
|
||||
**Max execution time**: 10 minutes.
|
||||
|
||||
## Performance Tests
|
||||
|
||||
### PT-01: VIO Adapter Runtime Budget
|
||||
|
||||
**Summary**: Verify VIO processing does not consume the full <400 ms system p95 budget.
|
||||
|
||||
**Traces to**: AC-4.1, AC-4.2
|
||||
|
||||
**Load scenario**:
|
||||
- Input: Derkachi synchronized replay and public/representative replay.
|
||||
- Duration: 30 minutes plus release long-run slice.
|
||||
- Target: Jetson Orin Nano Super.
|
||||
|
||||
| Metric | Target | Failure Threshold |
|
||||
|--------|--------|-------------------|
|
||||
| Adapter p95 latency | <=250 ms | >300 ms |
|
||||
| Memory contribution | <=3 GB | >4 GB |
|
||||
| Tracking failure on normal segments | <5% | >=5% |
|
||||
|
||||
**Resource limits**: Total system memory remains below 8 GB.
|
||||
|
||||
## Security Tests
|
||||
|
||||
### ST-01: Timestamp Injection Rejection
|
||||
|
||||
**Summary**: Verify malformed or non-monotonic timestamps do not produce trusted VIO state.
|
||||
|
||||
**Traces to**: AC-NEW-4
|
||||
|
||||
**Attack vector**: Replay or telemetry timestamp manipulation.
|
||||
|
||||
**Test procedure**:
|
||||
1. Feed non-monotonic frame and IMU timestamps.
|
||||
2. Observe adapter output.
|
||||
|
||||
**Expected behavior**: Adapter returns `TimestampMismatch` or low-quality failure; wrapper does not trust the state.
|
||||
|
||||
**Pass criteria**: No high-quality VIO state is emitted from malformed timing.
|
||||
|
||||
## Acceptance Tests
|
||||
|
||||
### AT-01: Normal VIO State Contract
|
||||
|
||||
**Summary**: Confirm adapter output contract supports downstream localization.
|
||||
|
||||
**Traces to**: AC-1.3, AC-2.1a
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | Initialize with calibrated frame/IMU config | `VioInitResult` succeeds |
|
||||
| 2 | Replay normal frames | `VioStatePacket` includes timestamp, relative pose, velocity, tracking quality |
|
||||
| 3 | End segment | State stream is continuous enough for wrapper drift accounting |
|
||||
|
||||
## Test Data Management
|
||||
|
||||
| Data Set | Description | Source | Size |
|
||||
|----------|-------------|--------|------|
|
||||
| `derkachi_video_telemetry` | Cropped nadir MP4 + synchronized IMU and `GLOBAL_POSITION_INT` trajectory | Project fixture | ~282 MB video + CSV |
|
||||
| `public_nadir_vio_candidates` | MUN-FRL/ALTO/Kagaru/EPFL slices | Public pinned fixtures | Dataset-dependent |
|
||||
| `representative_sync_replay` | Target camera + FC IMU + calibrated ground truth | Project collection | TBD |
|
||||
|
||||
**Setup procedure**: Pin calibration/extrinsics and mount read-only synchronized replay data.
|
||||
|
||||
**Teardown procedure**: Remove generated result reports and adapter temp state.
|
||||
|
||||
**Data isolation strategy**: One run directory per dataset slice and configuration hash.
|
||||
Reference in New Issue
Block a user