Revise skills documentation to incorporate updated directory structure and terminology. Replace references to integration tests with blackbox tests in SKILL.md files and templates. Adjust paths in planning and deployment documentation to align with the new _docs/02_document/ structure, ensuring consistency and clarity throughout the documentation.

2026-04-22 11:16:37 +00:00 · 2026-03-25 06:35:41 +02:00
parent 1c6e8f47b1
commit 531a1301d5
12 changed files with 2190 additions and 0 deletions
@@ -0,0 +1,138 @@
+# Performance Tests
+
+### NFT-PERF-01: End-to-End Per-Frame Latency
+
+**Summary**: Validate total pipeline latency from camera capture to GPS_INPUT output is <400ms.
+**Traces to**: AC-07 (< 400ms end-to-end per frame)
+**Metric**: End-to-end latency (camera frame timestamp → GPS_INPUT message timestamp)
+
+**Preconditions**:
+- System running on Jetson Orin Nano Super (GPU-mode)
+- Camera-replay serving frames at 0.7fps
+- System in steady state (warm-up: ≥10 frames processed)
+
+**Steps**:
+
+| Step | Consumer Action | Measurement |
+|------|----------------|-------------|
+| 1 | Camera-replay sends frame with known timestamp | Record t_capture |
+| 2 | Monitor GPS_INPUT messages at mavlink-inspector | Record t_gps_input for first GPS_INPUT update after t_capture |
+| 3 | Compute latency = t_gps_input - t_capture | Per-frame latency |
+| 4 | Repeat for 30 consecutive frames | Array of 30 latency values |
+
+**Pass criteria**: p95 latency < 400ms; max latency < 500ms
+**Duration**: 50s (~30 frames at 0.7fps + warm-up)
+
+---
+
+### NFT-PERF-02: GPS_INPUT Output Rate Consistency
+
+**Summary**: Validate GPS_INPUT messages are delivered at a sustained 5-10Hz with no gaps.
+**Traces to**: AC-08 (GPS_INPUT via MAVLink at 5-10Hz)
+**Metric**: Message rate (Hz), maximum inter-message gap (ms)
+
+**Preconditions**:
+- System in steady state
+- Camera-replay active
+
+**Steps**:
+
+| Step | Consumer Action | Measurement |
+|------|----------------|-------------|
+| 1 | Capture GPS_INPUT messages for 60 seconds | Count messages, record timestamps |
+| 2 | Compute rate: count / 60 | 5 ≤ rate ≤ 10 Hz |
+| 3 | Compute max gap between consecutive messages | max_gap ≤ 250ms |
+| 4 | Compute jitter: std_dev of inter-message intervals | jitter < 50ms |
+
+**Pass criteria**: Rate 5-10Hz; max gap ≤ 250ms; jitter < 50ms
+**Duration**: 60s
+
+---
+
+### NFT-PERF-03: cuVSLAM Visual Odometry Processing Time
+
+**Summary**: Validate cuVSLAM processes each frame within 20ms.
+**Traces to**: AC-07 (real-time processing budget)
+**Metric**: Per-frame cuVSLAM inference time (ms)
+
+**Preconditions**:
+- System running on Jetson Orin Nano Super
+- Steady state (≥10 frames processed)
+
+**Steps**:
+
+| Step | Consumer Action | Measurement |
+|------|----------------|-------------|
+| 1 | Replay 30 frames, read processing time from SSE events or health endpoint metrics | Per-frame VO time |
+| 2 | Compute p95 of VO time | p95 ≤ 20ms |
+
+**Pass criteria**: p95 cuVSLAM inference time ≤ 20ms
+**Duration**: 50s
+
+---
+
+### NFT-PERF-04: Satellite Matching Latency (Async)
+
+**Summary**: Validate satellite matching completes within 330ms per keyframe (async, does not block VO).
+**Traces to**: AC-07 (within frame budget), solution processing time budget
+**Metric**: Per-keyframe satellite matching latency (ms)
+
+**Preconditions**:
+- System running on Jetson Orin Nano Super
+- Satellite tiles loaded
+
+**Steps**:
+
+| Step | Consumer Action | Measurement |
+|------|----------------|-------------|
+| 1 | Monitor satellite match events over 60s (expect ~4-8 matches at 0.07-0.14Hz) | Per-match latency from health/metrics endpoint |
+| 2 | Verify no VO frame was blocked during satellite matching | VO timestamps maintain 0.7fps cadence |
+
+**Pass criteria**: p95 satellite matching ≤ 330ms; VO cadence unaffected
+**Duration**: 60s
+
+---
+
+### NFT-PERF-05: TRT Engine Load Time
+
+**Summary**: Validate all TensorRT engines load within 10 seconds total.
+**Traces to**: AC-11 (startup), solution startup sequence
+**Metric**: Engine load time (seconds)
+
+**Preconditions**:
+- Cold start on Jetson Orin Nano Super
+- Engines pre-built and available on storage
+
+**Steps**:
+
+| Step | Consumer Action | Measurement |
+|------|----------------|-------------|
+| 1 | Start system, monitor startup log for engine load timestamps | t_start_load, t_end_load per engine |
+| 2 | Compute total: sum of all engine load times | ≤ 10s total |
+
+**Pass criteria**: Total TRT engine load time ≤ 10s
+**Duration**: 30s (includes boot time)
+
+---
+
+### NFT-PERF-06: Sustained 30-Minute Processing
+
+**Summary**: Validate the system maintains consistent performance over a 30-minute continuous session without degradation.
+**Traces to**: AC-07 (real-time), AC-08 (memory < 8GB)
+**Metric**: Per-frame latency, GPS_INPUT rate, position accuracy over time
+
+**Preconditions**:
+- System running on Jetson Orin Nano Super
+- Camera-replay looping flight-sequence-60 (re-starts after frame 60)
+- Satellite tiles available
+
+**Steps**:
+
+| Step | Consumer Action | Measurement |
+|------|----------------|-------------|
+| 1 | Run for 30 minutes, collect per-minute stats | Latency, rate, accuracy |
+| 2 | Compare first-5-min stats vs last-5-min stats | No degradation >10% |
+| 3 | Monitor for any position output gaps > 1s | Count gaps |
+
+**Pass criteria**: No latency degradation >10% over 30 min; GPS_INPUT rate remains 5-10Hz; no output gaps >1s
+**Duration**: 30 minutes