Oleksandr Bezdieniezhnykh
cab7b5d020
- Modified the Docker Compose configuration to include an input root for replay tests and added an environment variable for enabling SITL. - Enhanced documentation for various testing processes, including the addition of a Runtime Completeness Decomposition Gate and clarifications on internal module testing requirements. - Updated the implementation completeness report to reflect the current state and added new test cases for performance and resilience scenarios. Co-authored-by: Cursor <cursoragent@cursor.com>
3.4 KiB
Resource Limit Tests
NFT-RES-LIM-01: Jetson Memory Budget
Summary: Validate that runtime memory stays below the 8 GB shared LPDDR5 limit.
Traces to: AC-4.2, Restrictions Onboard Hardware
Preconditions:
- Jetson Orin Nano Super in production power/thermal mode.
- BASALT + wrapper, cache index, FAISS CPU index, and FDR enabled.
Monitoring:
- CPU/GPU shared memory, process RSS, CUDA allocations, FAISS index memory.
Duration: Minimum 60 minutes steady-state replay plus relocalization triggers.
Pass criteria: Peak memory <8 GB shared; no OOM kill; no silent descriptor/index eviction.
NFT-RES-LIM-02: Thermal And Power Envelope
Summary: Validate sustained 25 W operation without thermal throttling across the environmental envelope.
Traces to: AC-NEW-5
Preconditions:
- Jetson cooling solution installed.
- Hot-soak chamber or production thermal test setup at +50 °C.
Monitoring:
- Power mode, temperature sensors, throttle flags, CPU/GPU clocks, per-frame latency.
Duration: 8 hours at sustained representative workload.
Pass criteria: No thermal throttle event; p95 latency remains <400 ms; QGC receives thermal warning if any threshold is approached.
NFT-RES-LIM-03: Satellite Cache Storage Budget
Summary: Validate persistent satellite cache footprint for up to 400 km² operational area.
Traces to: AC-8.3, Restrictions Satellite Imagery
Monitoring:
- Cache imagery, overviews, manifests, sidecars, FAISS descriptors/indexes.
Duration: Full cache build/load test.
Pass criteria: Persistent cache is <=10 GB unless the implementation explicitly defines and gets approval for a separate descriptor/index budget.
NFT-RES-LIM-04: Flight Data Recorder Rollover
Summary: Validate FDR storage cap and rollover behavior under an 8-hour synthetic mission.
Traces to: AC-NEW-3, AC-8.5
Preconditions:
- Synthetic 8-hour load with 3 fps navigation frames, full-rate IMU, emitted
GPS_INPUT, health telemetry, tile writes, and failure thumbnails.
Monitoring:
- FDR segment sizes, rollover events, retained payload classes.
Duration: 8 hours.
Pass criteria: FDR remains <=64 GB per flight; rollover is logged; no raw nav/AI frames are retained; no payload class is silently dropped.
NFT-RES-LIM-05: Cold Start Resource Spike
Summary: Validate that CUDA/TensorRT/ONNX/FAISS initialization does not violate boot or memory budgets.
Traces to: AC-NEW-1, AC-4.2
Monitoring:
- Initialization time, peak memory, engine/index load time.
Duration: 50 cold-start trials.
Pass criteria: First valid GPS_INPUT <30 s p95; peak memory <8 GB; no first-run engine build occurs at runtime.
NFT-RES-LIM-INFRA: Jetson Hardware Prerequisite Smoke
Summary: Validate that local replay reports Jetson-only resource gates as blocked unless target hardware is explicitly enabled.
Traces to: AZ-239 AC-1, AZ-239 AC-2, AZ-239 AC-4, AZ-233 Reliability NFR
Monitoring:
- Replay report status, blocked reason, and run-scoped artifact path.
Duration: One Docker replay smoke run.
Pass criteria: On non-Jetson local runners, the scenario reports blocked with Jetson prerequisite blocked: set GPSD_ENABLE_JETSON=1 on target hardware; on Jetson release-gate runners, it must collect the metrics required by NFT-RES-LIM-01, NFT-RES-LIM-02, and NFT-RES-LIM-05.