Oleksandr Bezdieniezhnykh
5.1 KiB
Test Specification — FDR And Observability
Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|---|---|---|---|
| AC-1.3 | Anchor age/drift evidence | IT-01 | Covered |
| AC-1.4 | Confidence/source label retained | IT-01 | Covered |
| AC-4.4 | Per-frame local stream evidence | IT-01, PT-01 | Covered |
| AC-5.2 | Failure logging | IT-02 | Covered |
| AC-6.1 | QGC/status evidence | IT-03 | Covered |
| AC-8.4 | Generated tile audit | IT-04 | Covered |
| AC-8.5 | No raw frame retention | ST-01 | Covered |
| AC-NEW-3 | FDR retention and 64 GB cap | PT-01, AT-01 | Covered |
| AC-NEW-4 | False-position forensics | IT-05 | Covered |
| AC-NEW-5 | Thermal/throttle logging | IT-06 | Covered |
| AC-NEW-8 | Blackout/failsafe logging | IT-02, IT-03 | Covered |
Blackbox Tests
IT-01: Per-Estimate Event Capture
Summary: Verify every estimate stores covariance, source label, anchor age, and emitted output metadata.
Traces to: AC-1.3, AC-1.4, AC-4.4
Input data: Position estimate stream with satellite, VO, and dead-reckoned labels.
Expected result: PostgreSQL event index and CBOR payload segments contain all required fields with monotonic timestamps.
Max execution time: 5 minutes.
IT-02: Failure And Blackout Logging
Summary: Verify no-estimate and blackout transitions are recorded.
Traces to: AC-5.2, AC-NEW-8
Input data: No-estimate gap and total blackout sequence.
Expected result: FDR records start, every degraded estimate, failsafe threshold, and recovery reason.
Max execution time: 10 minutes.
IT-03: QGC Status Audit
Summary: Verify operator-visible status has matching FDR evidence.
Traces to: AC-6.1, AC-NEW-8
Input data: QGC status messages from MAVLink component.
Expected result: FDR contains status text, timestamp, and mode context.
Max execution time: 5 minutes.
IT-04: Generated Tile Audit Trail
Summary: Verify tile-write decisions are recorded with parent covariance and trust level.
Traces to: AC-8.4
Input data: Accepted and rejected generated tile write decisions.
Expected result: FDR includes tile ID, parent covariance, trust level, sidecar hash, and rejection reason where applicable.
Max execution time: 5 minutes.
IT-05: False-Position Investigation Bundle
Summary: Verify enough evidence exists to investigate a false-position event.
Traces to: AC-NEW-4
Input data: Simulated false anchor rejection and covariance growth sequence.
Expected result: Export includes estimates, anchor decisions, residuals, covariance, and emitted MAVLink fields.
Max execution time: 5 minutes.
IT-06: Thermal/Throttle Event Capture
Summary: Verify resource health events are recorded.
Traces to: AC-NEW-5
Input data: Synthetic thermal/throttle metric stream.
Expected result: FDR records CPU/GPU/temp/throttle status and QGC warning trigger.
Max execution time: 5 minutes.
Performance Tests
PT-01: 8-Hour FDR Load
Summary: Verify FDR storage and append behavior under full mission load.
Traces to: AC-4.4, AC-NEW-3
Load scenario:
- Duration: 8 hours synthetic.
- Inputs: 3 Hz estimates, full-rate IMU, MAVLink tlog, health metrics, tile events.
| Metric | Target | Failure Threshold |
|---|---|---|
| Total FDR size | <=64 GB | >64 GB without rollover |
| Append latency p95 | <=10 ms async enqueue | >25 ms |
| Silent payload loss | 0 | Any unlogged loss |
Resource limits: FDR must not block hot-path localization.
Security Tests
ST-01: Raw Frame Retention Audit
Summary: Verify FDR does not store raw full-resolution frames.
Traces to: AC-8.5
Attack vector: Debug logging accidentally persists raw camera frames.
Test procedure:
- Run normal replay and failed tile-generation replay.
- Inspect FDR payloads and output directories.
Expected behavior: Only metadata, hashes, estimates, tiles, and allowed low-rate failed-frame thumbnails are retained.
Pass criteria: No raw nav/AI camera frame payloads in normal FDR.
Acceptance Tests
AT-01: FDR Export
Summary: Verify post-flight export creates usable audit artifacts.
Traces to: AC-NEW-3
| Step | Action | Expected Result |
|---|---|---|
| 1 | Complete synthetic flight | Segment rollover is logged and cap respected |
| 2 | Export FDR summary | Markdown/CSV/Parquet optional artifacts are produced |
| 3 | Query PostgreSQL index | Events can be filtered by time/type/mission |
Test Data Management
| Data Set | Description | Source | Size |
|---|---|---|---|
fdr_synthetic_load |
Estimate, IMU, MAVLink, health, tile events | Generated fixture | Large |
incident_fixture |
False-position and blackout evidence | Generated fixture | Small |
Setup procedure: Create isolated PostgreSQL schema and FDR segment directory.
Teardown procedure: Export report, then remove schema and segment directory.
Data isolation strategy: Per-run mission ID, schema, and FDR directory.