Oleksandr Bezdieniezhnykh
64542d32fc
Transitioned the autodev state to phase 21, reflecting the completion of Step 5 and the drafting of Step 6 epics. Revised the architecture documentation to clarify the roles of the Tile Manager and its components, ensuring accurate representation of the system's operational flow. Updated glossary entries for Flight State and Operator to incorporate recent changes and enhance clarity on component interactions and responsibilities.
6.3 KiB
Test Specification — C13 Flight Data Recorder
Component-scoped. Suite-level coverage in _docs/02_document/tests/*.md.
Acceptance Criteria Traceability
| AC ID | Acceptance Criterion (one-line) | Test IDs | Coverage |
|---|---|---|---|
| AC-1.4 | 95% covariance + source label (FDR record class) | FT-P-03, C13-IT-01 | Covered |
| AC-4.5 (revised) | Internal smoothing past keyframes (FDR-only path) | FT-P-10, C13-IT-02 | Covered |
| AC-8.5 | No raw nav/AI-cam frame retention except thumbnail log | FT-P-18, C13-IT-03 | Covered |
| AC-NEW-3 | FDR ≤64 GB / flight, no silent drops | NFT-LIM-02, C13-IT-04 | Covered |
| RESTRICT-UAV-4 | No raw-photo storage; tile cache + FDR only | FT-P-18, NFT-LIM-03 | Covered |
Component-Internal Tests
C13-IT-01: every payload class produces an FDR record
Summary: every component documented as an FDR producer publishes records of the right kind; missing producer = test failure.
Traces to: AC-1.4, AC-NEW-3 (every-payload-class-from-t=0 invariant)
Description: spin up a minimal in-process test harness wiring all 14 components (mocked where heavy, real where light); drive 10 s of synthetic flight; assert the FDR contains records of every documented kind (estimate, vio_health, vpr_health, match_health, pose_estimate, source_label_change, fc_emit, fc_inbound, signing_key_event, spoof_promotion_block, mid_flight_tile, failed_tile_thumbnail, system_health, segment_rollover).
Input data: synthetic 10 s flight.
Expected result: every kind present with at least one record.
Max execution time: 60 s.
C13-IT-02: smoothed past-keyframe records land in FDR (NOT in FC stream)
Summary: when C5 publishes a smoothed past-keyframe revision, it lands in FDR but is NOT forwarded to C8's emission path.
Traces to: AC-4.5 (revised)
Description: per C5-IT-04 — re-run that scenario; assert (a) FDR contains the smoothed-history record class, (b) the C8 emission stream contains the original (unshifted) value.
Input data: shared with C5-IT-04.
Expected result: per assertion.
Max execution time: 60 s.
C13-IT-03: AC-8.5 forensic-thumbnail-only enforcement
Summary: C13 refuses to write a raw nav-cam or AI-cam frame; the only allowed exception is the failed-tile thumbnail at ≤0.1 Hz cap.
Traces to: AC-8.5
Description: attempt to write an FdrRecord of kind = "raw_nav_frame"; assert C13 raises RawFrameWriteForbiddenError. Attempt kind = "failed_tile_thumbnail" at 0.05 Hz; assert accepted. Attempt the same kind at 0.5 Hz (above cap); assert rate-limited (drop with cap log).
Input data: scripted producer.
Expected result: per assertion.
Max execution time: 30 s.
C13-IT-04: 64 GB cap holds without silent drop
Summary: under synthetic 8 h replay producing > 64 GB worth of records, C13 enforces the cap via oldest-segment-dropped + always logs the rollover event.
Traces to: AC-NEW-3
Description: scripted producer that emits at peak rates known to cross 64 GB in 8 h; replay; assert (a) total disk usage stays ≤ 64 GB, (b) every segment-rollover-with-drop is recorded with producer-id + dropped count, (c) the FlightFooter shows non-zero records_dropped_overrun matching the rollover events.
Input data: synthetic high-rate producer.
Expected result: cap held; every drop visible.
Max execution time: 8 h on a Tier-1 runner (NFT-LIM-02 budget).
C13-IT-05: queue overrun produces a structured drop record (never silent)
Summary: when a producer overruns its in-process queue, C13 writes a structured "overrun" record naming the producer and the dropped count.
Traces to: AC-NEW-3 (no-silent-drop)
Description: artificially throttle the writer thread; flood one producer's queue; assert (a) the first dropped record triggers an "overrun" record, (b) the overrun record includes producer-id + dropped count, (c) when throttling is removed, normal writes resume.
Input data: scripted producer + writer-thread throttle harness.
Expected result: overrun record present + accurate count.
Max execution time: 60 s.
C13-IT-06: refuse takeoff if open_flight fails
Summary: per AC-NEW-3, every payload class must be present from t=0 — if C13 cannot open the segment file, takeoff is aborted.
Traces to: AC-NEW-3
Description: configure flight_root to a directory the process cannot write to; call open_flight; assert FdrOpenError raised; assert the calling code (compositional root) refuses to open the FC adapter.
Input data: read-only flight_root directory.
Expected result: takeoff aborts; error logged.
Max execution time: 5 s.
Performance Tests
C13-PT-01: writer-thread throughput vs peak producer rate
Traces to: AC-NEW-3
Load scenario: aggregated peak producer rate (~100 Hz combined records).
Expected results:
| Metric | Target | Failure Threshold |
|---|---|---|
| Writer throughput | ≥ 200 Hz sustained | < 100 Hz |
| Per-record serialise + write p95 | ≤ 5 ms | 20 ms |
Security Tests
C13-ST-01: FDR record cannot be silenced via config
Summary: there is no config flag that disables the spoofing-promotion-block, signing-key-rotation, or rollover-drop record kinds — they are mandatory per AC-NEW-3 + ADR-008.
Traces to: defensive (AC-NEW-3, ADR-008)
Test procedure:
- Search the config schema for any flag that could disable any of those record kinds.
- Assert no such flag exists.
- Inject events of each kind under every documented config preset; assert all land in FDR.
Pass criteria: no disabling flag found; all events land. Fail criteria: any disabling flag exists or any event suppressed.
Acceptance Tests
Covered transitively via FT-P-03 / FT-P-10 / FT-P-18 / NFT-LIM-02.
Test Data Management
| Data Set | Source | Size |
|---|---|---|
| Synthetic 10 s flight harness | scripted | <10 MB |
| 8 h synthetic high-rate producer | scripted | runtime-generated |
Read-only flight_root fixture |
scripted | n/a (fs perms) |
Setup: per-test flight_root under tests/tmp/c13/<test-id>/.
Teardown: drop tmp directory.
Data isolation: per-test flight_root.