mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 17:21:13 +00:00
Update autodev state, architecture documentation, and glossary terms
Transitioned the autodev state to phase 21, reflecting the completion of Step 5 and the drafting of Step 6 epics. Revised the architecture documentation to clarify the roles of the Tile Manager and its components, ensuring accurate representation of the system's operational flow. Updated glossary entries for Flight State and Operator to incorporate recent changes and enhance clarity on component interactions and responsibilities.
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,176 @@
|
||||
# Test Specification — C1 Visual / Visual-Inertial Odometry
|
||||
|
||||
This file is component-scoped. AC-level coverage is in the suite-level test docs (`_docs/02_document/tests/*.md`) and the canonical traceability map is `_docs/02_document/tests/traceability-matrix.md`. The tables below cite test IDs from those files; the `Component-Internal Tests` section adds C1-specific unit/contract tests that the suite-level scenarios do not cover.
|
||||
|
||||
## Acceptance Criteria Traceability
|
||||
|
||||
| AC ID | Acceptance Criterion (one-line) | Test IDs (suite-level + this file) | Coverage |
|
||||
|-------|---------------------------------|-----------------------------------|----------|
|
||||
| AC-1.3 | Cumulative drift between satellite-anchored fixes <100 m visual / <50 m IMU-fused | FT-P-02, **C1-IT-01** | Covered |
|
||||
| AC-1.4 | Estimate reports 95% covariance + source label | FT-P-03, **C1-IT-02** | Covered |
|
||||
| AC-2.1a | Frame-to-frame registration ≥95% on normal segments | FT-P-04, **C1-IT-03** | Covered |
|
||||
| AC-2.2 (frame-to-frame portion) | MRE <1 px frame-to-frame | FT-P-05, **C1-IT-04** | Covered |
|
||||
| AC-3.2 | Tolerate sharp turns; recovery via satellite re-loc | FT-P-07, FT-N-02 | Covered (C1 contributes track-loss detection) |
|
||||
| AC-4.1 | E2E latency <400 ms p95 | NFT-PERF-01 (Tier-2), **C1-PT-01** | Covered |
|
||||
| AC-5.1 | Init from FC EKF's last valid GPS + IMU-extrapolated | FT-P-11, **C1-IT-05** | Covered |
|
||||
| AC-5.3 | On reboot, re-init from FC IMU-extrapolated pose | NFT-RES-02, **C1-IT-06** | Covered |
|
||||
|
||||
---
|
||||
|
||||
## Component-Internal Tests
|
||||
|
||||
### C1-IT-01: VioStrategy contract — `process_frame` honest covariance under degradation
|
||||
|
||||
**Summary**: every concrete `VioStrategy` implementation (Okvis2, VinsMono, KltRansac) must produce a 6×6 covariance whose norm grows monotonically when feature tracking degrades.
|
||||
|
||||
**Traces to**: AC-1.3, AC-1.4
|
||||
|
||||
**Description**: feed each strategy a synthetic 60 s nav-camera + IMU sequence with a controlled feature-loss event at t=30 s (50% feature drop). Assert that `pose_covariance_6x6` Frobenius norm before t=30 s is below the steady-state threshold and rises monotonically for ≥3 s after the event. No strategy may emit a tightened covariance during a degradation event (catches honest-covariance-violation regressions).
|
||||
|
||||
**Input data**: `tests/fixtures/synthetic_vio/normal_then_feature_drop_60s/` (nav-cam frames at 3 Hz + IMU at 200 Hz; feature-loss event injected via image masking).
|
||||
|
||||
**Expected result**: `||cov||_F` curve has a rising shoulder ≥1.5× steady-state norm within 3 s of the event for all three strategies; `VioHealth.state` transitions `TRACKING → DEGRADED` within the same window.
|
||||
|
||||
**Max execution time**: 30 s per strategy on Tier-1.
|
||||
**Dependencies**: helper `ImuPreintegrator` (shared with C5).
|
||||
|
||||
---
|
||||
|
||||
### C1-IT-02: `VioOutput` schema invariants
|
||||
|
||||
**Summary**: every `VioOutput` carries a 6×6 SPD covariance and a non-empty `feature_quality`.
|
||||
|
||||
**Traces to**: AC-1.4
|
||||
|
||||
**Description**: drive each strategy through 100 frames of a synthetic loop; for each emitted `VioOutput`, assert (a) `pose_covariance_6x6` is symmetric and positive-definite, (b) `feature_quality.tracked + new ≥ 0`, (c) `frame_id` matches the input `NavCameraFrame.frame_id`. Any single violation fails the test.
|
||||
|
||||
**Input data**: `tests/fixtures/synthetic_vio/loop_100f/`.
|
||||
|
||||
**Expected result**: 100/100 frames pass all invariants per strategy.
|
||||
|
||||
**Max execution time**: 10 s.
|
||||
|
||||
---
|
||||
|
||||
### C1-IT-03: KltRansac mandatory simple-baseline registration ≥95%
|
||||
|
||||
**Summary**: the simple-baseline strategy must hit AC-2.1a's 95% threshold on the Derkachi normal segment so the engine rule's mandatory baseline is met.
|
||||
|
||||
**Traces to**: AC-2.1a (engine rule)
|
||||
|
||||
**Description**: replay the Derkachi normal-segment fixture (the same 60 stills C8 fixture that FT-P-04 uses) through the KltRansac strategy only; count frames with `VioHealth.state == TRACKING` at emission. Pass if ≥95%.
|
||||
|
||||
**Input data**: `tests/fixtures/flight_derkachi/normal_segment_60_stills/`.
|
||||
|
||||
**Expected result**: tracked frame ratio ≥ 0.95.
|
||||
|
||||
**Max execution time**: 60 s.
|
||||
|
||||
---
|
||||
|
||||
### C1-IT-04: Frame-to-frame MRE bound
|
||||
|
||||
**Summary**: each strategy's per-frame mean reprojection error stays under 1 px on normal segments.
|
||||
|
||||
**Traces to**: AC-2.2 (frame-to-frame portion)
|
||||
|
||||
**Description**: same Derkachi normal-segment fixture as C1-IT-03. Compute MRE per frame from the strategy's internal residual; assert MRE p95 < 1 px per AC-2.2.
|
||||
|
||||
**Input data**: as above.
|
||||
|
||||
**Expected result**: MRE p95 < 1 px for Okvis2 + KltRansac (production-default + simple-baseline). VinsMono is research-only and exempt from MRE bound (only IT-12 comparative-study coverage).
|
||||
|
||||
**Max execution time**: 60 s.
|
||||
|
||||
---
|
||||
|
||||
### C1-IT-05: Warm-start from `WarmStartPose` converges within configured budget
|
||||
|
||||
**Summary**: `reset_to_warm_start` followed by 5 frames of input must converge to `state == TRACKING` for every strategy.
|
||||
|
||||
**Traces to**: AC-5.1
|
||||
|
||||
**Description**: prepare a `WarmStartPose` derived from the FC EKF's last-valid-GPS fixture; call `reset_to_warm_start`, then push 5 frames of normal-segment input; assert health transitions `INIT → TRACKING` within 5 frames.
|
||||
|
||||
**Input data**: `tests/fixtures/flight_derkachi/takeoff_warmstart/`.
|
||||
|
||||
**Expected result**: TRACKING by frame 5 for all three strategies.
|
||||
|
||||
**Max execution time**: 5 s.
|
||||
|
||||
---
|
||||
|
||||
### C1-IT-06: F8 reboot recovery via warm-start hint
|
||||
|
||||
**Summary**: simulate a mid-flight reboot — the strategy must re-init from the warm-start hint without crashing or producing covariance < pre-reboot.
|
||||
|
||||
**Traces to**: AC-5.3
|
||||
|
||||
**Description**: run normal-segment input for 30 s; capture last `VioOutput`; reset the strategy with that pose as `WarmStartPose`; resume input; assert next 5 emitted `VioOutput` have `pose_covariance_6x6` Frobenius norm ≥ the pre-reboot value (no fake confidence after reboot).
|
||||
|
||||
**Input data**: as C1-IT-03.
|
||||
|
||||
**Expected result**: pass per assertion above for all three strategies.
|
||||
|
||||
**Max execution time**: 60 s.
|
||||
|
||||
---
|
||||
|
||||
## Performance Tests
|
||||
|
||||
### C1-PT-01: per-frame latency budget on Tier-2
|
||||
|
||||
**Summary**: `process_frame` p95 latency on Jetson under nominal thermal.
|
||||
|
||||
**Traces to**: AC-4.1 (component-level partition; suite-level NFT-PERF-01 owns the e2e budget).
|
||||
|
||||
**Load scenario**:
|
||||
- Single ingest thread, 3 Hz frame rate, 10 min replay of Derkachi normal segment.
|
||||
- Concurrent C2 backbone forward pass running on the same Jetson (realistic load contention).
|
||||
|
||||
**Expected results**:
|
||||
|
||||
| Metric | Target | Failure Threshold |
|
||||
|--------|--------|-------------------|
|
||||
| `process_frame` latency p50 | ≤ 25 ms (Okvis2 production-default) | 60 ms |
|
||||
| `process_frame` latency p95 | ≤ 80 ms | 120 ms |
|
||||
| Throughput | ≥ 3 Hz sustained | < 2.5 Hz |
|
||||
|
||||
**Resource limits**:
|
||||
- CPU: ≤ 30% of one core (Okvis2 is multi-threaded internally; bound at 30% per ADR-002 budget partition).
|
||||
- Memory: ≤ 1.5 GB resident.
|
||||
|
||||
---
|
||||
|
||||
## Security Tests
|
||||
|
||||
C1 has no externally-reachable surface (internal-only component); suite-level NFT-SEC-02 (no in-flight egress) and NFT-SEC-05 (DNS blackholing) cover the airborne process broadly. No C1-specific security tests.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Tests
|
||||
|
||||
C1 contributes to AC-1.3 / 1.4 / 2.1a / 5.1 / 5.3 via the suite-level FT scenarios cited in the traceability table. No additional C1-only acceptance tests are needed.
|
||||
|
||||
---
|
||||
|
||||
## Test Data Management
|
||||
|
||||
**Required test data**:
|
||||
|
||||
| Data Set | Description | Source | Size |
|
||||
|----------|-------------|--------|------|
|
||||
| `synthetic_vio/normal_then_feature_drop_60s/` | 60 s nav-cam + IMU with controlled feature-loss event at t=30 s | generated by `scripts/gen_synthetic_vio.py` (deterministic seed) | ~50 MB |
|
||||
| `synthetic_vio/loop_100f/` | 100-frame synthetic closed loop for invariant checks | generated, deterministic | ~30 MB |
|
||||
| `flight_derkachi/normal_segment_60_stills/` | the project's canonical normal-segment fixture | curated subset of Derkachi raw drop | ~80 MB |
|
||||
| `flight_derkachi/takeoff_warmstart/` | last-valid-GPS + IMU window from FC EKF for warm-start tests | recorded once, replayed | ~5 MB |
|
||||
|
||||
**Setup procedure**:
|
||||
1. Run `scripts/gen_synthetic_vio.py` once per fixture to populate `tests/fixtures/synthetic_vio/`.
|
||||
2. Mount `tests/fixtures/flight_derkachi/` from the project's data archive (read-only).
|
||||
|
||||
**Teardown procedure**:
|
||||
1. Synthetic fixtures persist between runs (deterministic; no per-run mutation).
|
||||
2. The Derkachi fixture is read-only; nothing to clean up.
|
||||
|
||||
**Data isolation strategy**: every test runs in its own temp directory under `tests/tmp/c1/<test-id>/`; per-strategy state is constructed fresh in each test (no shared state across tests).
|
||||
Reference in New Issue
Block a user