# Test Environment

## Overview

**System under test**: Onboard GPS-denied localization service. Public interfaces are navigation-camera frame input, flight-controller telemetry input, offline satellite-cache input, `GPS_INPUT` MAVLink output, QGroundControl status output, and flight-data-recorder output.

**Consumer app purpose**: A black-box replay harness that feeds image frames, telemetry traces, cache manifests, and fault triggers into the service, then validates emitted coordinates, confidence fields, telemetry, and logs without importing internal modules.

## Execution Environments

| Environment | Purpose | Required for |
|-------------|---------|--------------|
| Local replay workstation | Fast still-image and dataset replay validation | Frame-center geolocation, Satellite Service local retrieval, stale-tile rejection |
| Jetson Orin Nano Super | Production-like latency, memory, thermal, and TensorRT/ONNX profiling | AC-4.1, AC-4.2, AC-NEW-1, AC-NEW-5 |
| ArduPilot Plane SITL + QGroundControl | MAVLink `GPS_INPUT`, spoofing, failsafe, and GCS status validation | AC-4.3, AC-5.2, AC-NEW-2, AC-NEW-8 |
| Representative flight/replay rig | Final acceptance evidence with synchronized nav camera, FC IMU/attitude/airspeed/altitude, MAVLink logs, and ground truth | Final AC signoff |

## Docker / Compose Structure

| Service | Image / Build | Purpose | Ports |
|---------|---------------|---------|-------|
| gps-denied-service | Project build image for JetPack-compatible target or replay-compatible host | System under test | MAVLink UDP/TCP and health/status endpoints TBD |
| replay-consumer | Python replay/test harness | Feeds images, telemetry, cache data, and fault triggers | none |
| satellite-cache-stub | Local COG/manifest/descriptor fixture volume | Provides offline tile cache and signed/unsigned manifests | none |
| ardupilot-plane-sitl | ArduPilot Plane SITL image or local process | Validates `GPS_INPUT`, spoofing/failsafe behavior | MAVLink SITL ports |
| qgc-observer | QGC/tlog-compatible observer or MAVLink log parser | Verifies GCS-visible status output | none |

## Networks

| Network | Services | Purpose |
|---------|----------|---------|
| replay-net | gps-denied-service, replay-consumer, satellite-cache-stub | Offline replay and black-box validation |
| sitl-net | gps-denied-service, ardupilot-plane-sitl, qgc-observer | MAVLink integration and failsafe validation |

## Volumes

| Volume | Mounted to | Purpose |
|--------|------------|---------|
| input-data | `/data/input` | `_docs/00_problem/input_data/` and public dataset slices |
| expected-results | `/data/expected` | `_docs/00_problem/input_data/expected_results/` |
| derkachi-replay | `/data/input/flight_derkachi` | Cropped nadir MP4 plus synchronized IMU and `GLOBAL_POSITION_INT` trajectory |
| satellite-cache | `/cache/satellite` | COG tiles, manifests, descriptor index fixtures |
| fdr-output | `/fdr` | Flight-data-recorder outputs for validation |

## Consumer Application

**Tech stack**: Python replay harness with pytest-style assertions, Docker/compose orchestration, deterministic cache/SITL/QGC stubs, and CSV/Markdown report generation.

**Entry points**:
- Local functional suite: `python3 -m pytest`
- Replay harness: `python -m e2e.replay.run_replay --output-dir <dir> --input-root <fixture-root>`
- Docker replay gate: `docker compose -f docker-compose.test.yml run --build --rm replay-consumer`

### Communication With System Under Test

| Interface | Protocol | Endpoint / Topic | Authentication |
|-----------|----------|------------------|----------------|
| Navigation frames | File/stream replay | Ordered image frames with timestamps | Local fixture access |
| FC telemetry | MAVLink replay or generated stream | IMU, attitude, airspeed, altitude, GPS health | Local MAVLink link |
| Satellite cache | Local filesystem contract | COG + manifest + descriptors | Signed manifest validation |
| GPS output | MAVLink | `GPS_INPUT` to ArduPilot Plane | MAVLink source/system ID allowlist |
| Status output | MAVLink/QGC | `STATUSTEXT` / status summary | MAVLink source/system ID allowlist |
| FDR | Filesystem output | Per-flight segmented logs | Local fixture access |

### What The Consumer Does Not Access

- No internal estimator modules.
- No direct BASALT/OpenVINS/Kimera APIs.
- No direct mutation of internal state.
- No bypass of public cache, MAVLink, replay, or FDR interfaces.

## CI/CD Integration

| Suite | When to run | Gate behavior | Timeout |
|-------|-------------|---------------|---------|
| Still-image geolocation smoke | Every PR after implementation exists | Block merge | <= 15 min |
| Public VIO dataset replay | Nightly and before release | Block release | Dataset-dependent |
| Jetson performance/resource | Before release and after runtime dependency changes | Block release | <= 8 h for endurance/thermal |
| Plane SITL failsafe/spoofing | Every release candidate | Block release | <= 60 min |

## Reporting

**Format**: CSV and FDR validation summary.

**Columns**: Test ID, Test Name, Input Dataset, Execution Time (ms), Result, Error Distance (m), Source Label, Covariance 95% Semi-Major (m), `GPS_INPUT.fix_type`, Error Message.

**Output path**: `data/test-results/<run-id>/blackbox-report.csv` and `data/test-results/<run-id>/fdr-validation-summary.md` on the host; `/app/data/test-results/<run-id>/...` inside the replay container.

## Test Execution

**Decision**: Both Docker/replay and local hardware execution.

**Hardware dependencies found**:
- Jetson Orin Nano Super with 8 GB shared LPDDR5 and 25 W power mode.
- CUDA/TensorRT/ONNX acceleration for DINOv2 and local-matcher profiling.
- Camera ingestion paths over USB, MIPI-CSI, or GigE.
- ArduPilot Plane SITL and MAVLink `GPS_INPUT` behavior.
- Thermal, power, FDR, and storage limits that require target-like execution.

### Docker / Replay Mode

Use Docker or local host replay for deterministic, reproducible tests that do not require physical Jetson hardware:

- Still-image frame-center geolocation.
- Derkachi synchronized video/telemetry replay, including alignment and VIO smoke checks.
- Satellite-cache freshness and integrity fixtures.
- FAISS descriptor/index behavior.
- Public dataset replay where GPU/hardware timing is not the assertion.
- Plane SITL tests where SITL and MAVLink behavior are the target.

Docker/replay mode is suitable for PR checks and nightly validation, but it does not prove Jetson latency, memory, thermal, or camera-driver behavior.

Current Docker replay smoke evidence is expected to pass `FT-P-01`, `NFT-PERF-INFRA`, `NFT-RES-INFRA`, and `NFT-SEC-INFRA`. `NFT-RES-LIM-INFRA` remains blocked on local non-Jetson runners with an explicit target-hardware prerequisite.

### Local Hardware Mode

Use local Jetson hardware for release gates:

- BASALT + wrapper latency and memory profiling.
- DINOv2/ONNX/TensorRT descriptor-fidelity and runtime profiling.
- ALIKED/DISK + LightGlue runtime profiling.
- Cold-start time to first valid `GPS_INPUT`.
- 8-hour thermal and FDR endurance tests.
- Camera interface validation once the exact module interface is selected.

### Gate Policy

- PR gate: Docker/replay smoke and deterministic fixture tests.
- Nightly gate: Docker/replay public dataset slices and SITL scenarios.
- Release gate: local Jetson hardware, Plane SITL, thermal/resource tests, and representative replay data.