mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-22 03:21:14 +00:00
Revise acceptance criteria and restrictions documentation to clarify recent updates and specifications. Key changes include enhanced definitions for position accuracy, image processing quality, and operational parameters, as well as updates to camera specifications and validation requirements. This revision aims to improve clarity and ensure alignment with project goals.
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,79 @@
|
||||
# FDR And Observability
|
||||
|
||||
## 1. High-Level Overview
|
||||
|
||||
**Purpose**: Record bounded, replayable mission evidence and expose runtime health/status events for analysis and operator awareness.
|
||||
|
||||
**Architectural Pattern**: Append-only event sink + exporter.
|
||||
|
||||
**Upstream dependencies**: All runtime components.
|
||||
|
||||
**Downstream consumers**: Validation harness, post-flight audit tools, QGC status through MAVLink component.
|
||||
|
||||
## 2. Internal Interfaces
|
||||
|
||||
### Interface: `FlightRecorder`
|
||||
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
| `append_event` | `FdrEvent` | `AppendResult` | Yes | `RecorderUnavailable`, `StorageFull` |
|
||||
| `rollover` | `RolloverRequest` | `FdrSegmentInfo` | No | `RolloverFailed` |
|
||||
| `export` | `ExportRequest` | `ExportResult` | Yes | `ExportFailed` |
|
||||
|
||||
## 3. Data Access Patterns
|
||||
|
||||
| Query | Frequency | Hot Path | Index Needed |
|
||||
|-------|-----------|----------|--------------|
|
||||
| Append event | High | Yes | Append index only |
|
||||
| Export by time/type | Post-flight | No | Time/type index |
|
||||
|
||||
### Storage Estimates
|
||||
|
||||
| Table/Collection | Est. Row Count | Row Size | Total Size | Growth Rate |
|
||||
|------------------|----------------|----------|------------|-------------|
|
||||
| FDR events | Flight-dependent | Mixed | <=64 GB per 8 h | Per flight |
|
||||
|
||||
## 4. Implementation Details
|
||||
|
||||
**State Management**: Owns active segment, rollover policy, and export state.
|
||||
|
||||
**Key Dependencies**:
|
||||
|
||||
| Library | Purpose |
|
||||
|---------|---------|
|
||||
| PostgreSQL client | Event metadata, time/type indexes, mission query surface |
|
||||
| CBOR writer | Bounded runtime payload segments |
|
||||
| Parquet writer | Optional post-flight export |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
- Storage-full emits critical status and starts rollover/retention behavior.
|
||||
- Append failures are surfaced to the caller and health system.
|
||||
|
||||
## 5. Caveats & Edge Cases
|
||||
|
||||
**Known limitations**:
|
||||
- Raw frames are not retained by default; only metadata, decisions, hashes, and occlusion/blackout status are recorded.
|
||||
- PostgreSQL availability is required for indexed FDR metadata; CBOR payload segments preserve bounded append behavior for high-volume data.
|
||||
|
||||
**Performance bottlenecks**:
|
||||
- FDR appends must not block hot-path localization.
|
||||
|
||||
## 6. Dependency Graph
|
||||
|
||||
**Must be implemented after**: event schema and key DTOs.
|
||||
|
||||
**Can be implemented in parallel with**: MAVLink integration.
|
||||
|
||||
**Blocks**: release evidence and most validation reports.
|
||||
|
||||
## 7. Logging Strategy
|
||||
|
||||
| Log Level | When | Example |
|
||||
|-----------|------|---------|
|
||||
| ERROR | Recorder unavailable | `fdr_unavailable path=...` |
|
||||
| WARN | Rollover occurs | `fdr_rollover segment=...` |
|
||||
| INFO | Export complete | `fdr_export_complete format=parquet` |
|
||||
|
||||
**Log format**: FDR event metadata plus local health logs.
|
||||
|
||||
**Log storage**: PostgreSQL FDR event tables plus CBOR segment payloads.
|
||||
@@ -0,0 +1,166 @@
|
||||
# Test Specification — FDR And Observability
|
||||
|
||||
## Acceptance Criteria Traceability
|
||||
|
||||
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|
||||
|-------|---------------------|----------|----------|
|
||||
| AC-1.3 | Anchor age/drift evidence | IT-01 | Covered |
|
||||
| AC-1.4 | Confidence/source label retained | IT-01 | Covered |
|
||||
| AC-4.4 | Per-frame local stream evidence | IT-01, PT-01 | Covered |
|
||||
| AC-5.2 | Failure logging | IT-02 | Covered |
|
||||
| AC-6.1 | QGC/status evidence | IT-03 | Covered |
|
||||
| AC-8.4 | Generated tile audit | IT-04 | Covered |
|
||||
| AC-8.5 | No raw frame retention | ST-01 | Covered |
|
||||
| AC-NEW-3 | FDR retention and 64 GB cap | PT-01, AT-01 | Covered |
|
||||
| AC-NEW-4 | False-position forensics | IT-05 | Covered |
|
||||
| AC-NEW-5 | Thermal/throttle logging | IT-06 | Covered |
|
||||
| AC-NEW-8 | Blackout/failsafe logging | IT-02, IT-03 | Covered |
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
### IT-01: Per-Estimate Event Capture
|
||||
|
||||
**Summary**: Verify every estimate stores covariance, source label, anchor age, and emitted output metadata.
|
||||
|
||||
**Traces to**: AC-1.3, AC-1.4, AC-4.4
|
||||
|
||||
**Input data**: Position estimate stream with satellite, VO, and dead-reckoned labels.
|
||||
|
||||
**Expected result**: PostgreSQL event index and CBOR payload segments contain all required fields with monotonic timestamps.
|
||||
|
||||
**Max execution time**: 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
### IT-02: Failure And Blackout Logging
|
||||
|
||||
**Summary**: Verify no-estimate and blackout transitions are recorded.
|
||||
|
||||
**Traces to**: AC-5.2, AC-NEW-8
|
||||
|
||||
**Input data**: No-estimate gap and total blackout sequence.
|
||||
|
||||
**Expected result**: FDR records start, every degraded estimate, failsafe threshold, and recovery reason.
|
||||
|
||||
**Max execution time**: 10 minutes.
|
||||
|
||||
---
|
||||
|
||||
### IT-03: QGC Status Audit
|
||||
|
||||
**Summary**: Verify operator-visible status has matching FDR evidence.
|
||||
|
||||
**Traces to**: AC-6.1, AC-NEW-8
|
||||
|
||||
**Input data**: QGC status messages from MAVLink component.
|
||||
|
||||
**Expected result**: FDR contains status text, timestamp, and mode context.
|
||||
|
||||
**Max execution time**: 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
### IT-04: Generated Tile Audit Trail
|
||||
|
||||
**Summary**: Verify tile-write decisions are recorded with parent covariance and trust level.
|
||||
|
||||
**Traces to**: AC-8.4
|
||||
|
||||
**Input data**: Accepted and rejected generated tile write decisions.
|
||||
|
||||
**Expected result**: FDR includes tile ID, parent covariance, trust level, sidecar hash, and rejection reason where applicable.
|
||||
|
||||
**Max execution time**: 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
### IT-05: False-Position Investigation Bundle
|
||||
|
||||
**Summary**: Verify enough evidence exists to investigate a false-position event.
|
||||
|
||||
**Traces to**: AC-NEW-4
|
||||
|
||||
**Input data**: Simulated false anchor rejection and covariance growth sequence.
|
||||
|
||||
**Expected result**: Export includes estimates, anchor decisions, residuals, covariance, and emitted MAVLink fields.
|
||||
|
||||
**Max execution time**: 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
### IT-06: Thermal/Throttle Event Capture
|
||||
|
||||
**Summary**: Verify resource health events are recorded.
|
||||
|
||||
**Traces to**: AC-NEW-5
|
||||
|
||||
**Input data**: Synthetic thermal/throttle metric stream.
|
||||
|
||||
**Expected result**: FDR records CPU/GPU/temp/throttle status and QGC warning trigger.
|
||||
|
||||
**Max execution time**: 5 minutes.
|
||||
|
||||
## Performance Tests
|
||||
|
||||
### PT-01: 8-Hour FDR Load
|
||||
|
||||
**Summary**: Verify FDR storage and append behavior under full mission load.
|
||||
|
||||
**Traces to**: AC-4.4, AC-NEW-3
|
||||
|
||||
**Load scenario**:
|
||||
- Duration: 8 hours synthetic.
|
||||
- Inputs: 3 Hz estimates, full-rate IMU, MAVLink tlog, health metrics, tile events.
|
||||
|
||||
| Metric | Target | Failure Threshold |
|
||||
|--------|--------|-------------------|
|
||||
| Total FDR size | <=64 GB | >64 GB without rollover |
|
||||
| Append latency p95 | <=10 ms async enqueue | >25 ms |
|
||||
| Silent payload loss | 0 | Any unlogged loss |
|
||||
|
||||
**Resource limits**: FDR must not block hot-path localization.
|
||||
|
||||
## Security Tests
|
||||
|
||||
### ST-01: Raw Frame Retention Audit
|
||||
|
||||
**Summary**: Verify FDR does not store raw full-resolution frames.
|
||||
|
||||
**Traces to**: AC-8.5
|
||||
|
||||
**Attack vector**: Debug logging accidentally persists raw camera frames.
|
||||
|
||||
**Test procedure**:
|
||||
1. Run normal replay and failed tile-generation replay.
|
||||
2. Inspect FDR payloads and output directories.
|
||||
|
||||
**Expected behavior**: Only metadata, hashes, estimates, tiles, and allowed low-rate failed-frame thumbnails are retained.
|
||||
|
||||
**Pass criteria**: No raw nav/AI camera frame payloads in normal FDR.
|
||||
|
||||
## Acceptance Tests
|
||||
|
||||
### AT-01: FDR Export
|
||||
|
||||
**Summary**: Verify post-flight export creates usable audit artifacts.
|
||||
|
||||
**Traces to**: AC-NEW-3
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | Complete synthetic flight | Segment rollover is logged and cap respected |
|
||||
| 2 | Export FDR summary | Markdown/CSV/Parquet optional artifacts are produced |
|
||||
| 3 | Query PostgreSQL index | Events can be filtered by time/type/mission |
|
||||
|
||||
## Test Data Management
|
||||
|
||||
| Data Set | Description | Source | Size |
|
||||
|----------|-------------|--------|------|
|
||||
| `fdr_synthetic_load` | Estimate, IMU, MAVLink, health, tile events | Generated fixture | Large |
|
||||
| `incident_fixture` | False-position and blackout evidence | Generated fixture | Small |
|
||||
|
||||
**Setup procedure**: Create isolated PostgreSQL schema and FDR segment directory.
|
||||
|
||||
**Teardown procedure**: Export report, then remove schema and segment directory.
|
||||
|
||||
**Data isolation strategy**: Per-run mission ID, schema, and FDR directory.
|
||||
Reference in New Issue
Block a user