gps-denied-onboard/_docs/02_document/tests/environment.md

# Test Environment

## Overview

**System under test (SUT)**: `gps-denied-onboard` companion-PC service that produces WGS84 position estimates from nav-camera frames + FC IMU/attitude and emits them to the FC over its native external-positioning interface. Public boundaries (the only surfaces tests interact with):

- **Inbound — nav-camera frames**: V4L2 / GStreamer source (production: USB / MIPI-CSI / GigE per `restrictions.md`; tests: file-backed source replaying `_docs/00_problem/input_data/AD0000NN.jpg` or `flight_derkachi/flight_derkachi.mp4`).
- **Inbound — FC telemetry**: MAVLink (ArduPilot) or MSP2 (iNav) inbound stream carrying `SCALED_IMU2`, `ATTITUDE`, `GLOBAL_POSITION_INT` (or MSP equivalents). Tests replay `flight_derkachi/data_imu.csv` through a thin replayer.
- **Inbound — satellite tile cache**: filesystem + on-disk index (FAISS HNSW + tile manifest). Tests load a fixture cache mounted as a Docker volume.
- **Outbound — FC external-positioning**: MAVLink `GPS_INPUT` (ArduPilot Plane) OR MSP2 `MSP2_SENSOR_GPS` (iNav). Tests observe these by spinning up the corresponding open-source SITL and reading what reaches the FC.
- **Outbound — GCS telemetry**: MAVLink to QGroundControl (1-2 Hz downsample of estimates + STATUSTEXT). Tests subscribe via a passive MAVLink listener.
- **Outbound — Flight Data Recorder**: NVM filesystem (per AC-NEW-3). Tests read the resulting FDR archive after the run.

**Consumer app purpose**: The e2e harness drives the SUT through these public boundaries — replaying frames + telemetry, mounting tile-cache fixtures, observing FC-side acceptance via SITL, and parsing FDR output. It NEVER imports SUT modules, NEVER queries SUT internal state, and NEVER touches the SUT's filesystem outside the FDR output directory.

## Two-tier execution profile

This project requires two distinct test environments because the production target is Jetson hardware and AC-4.1/AC-4.2/AC-NEW-5 cannot be honestly validated on a generic x86 dev workstation.

| Tier | Hardware | What it covers | What it skips |
|------|----------|----------------|---------------|
| **Tier-1 (workstation Docker)** | x86 dev workstation, optional NVIDIA dGPU for TensorRT validation | All `FT-*` correctness, schema, `NFT-RES-*` resilience scenarios, `NFT-SEC-*` security scenarios, `NFT-LIM-*` storage budgets | Any AC whose pass criterion is bound to Jetson Orin Nano Super wall-clock latency or thermal envelope: AC-4.1 / AC-4.2 / AC-NEW-1 / AC-NEW-5 |
| **Tier-2 (Jetson hardware loop)** | Jetson Orin Nano Super (pinned hardware per `restrictions.md`), thermal chamber for AC-NEW-5 | AC-4.1 latency p95, AC-4.2 memory, AC-NEW-1 cold-start TTFF, AC-NEW-5 thermal envelope (chamber-only) | Iteration speed (manual hardware time) |

CI runs Tier-1 on every PR. Tier-2 runs on hardware-attached runners on a nightly cadence and pre-release gate; results are imported into the same CSV report format as Tier-1.

## Docker Environment (Tier-1)

### Services

| Service | Image / Build | Purpose | Ports |
|---------|--------------|---------|-------|
| `gps-denied-onboard` | local build (`docker/Dockerfile`) | The SUT. Production binary built with `BUILD_VINS_MONO=OFF` per locked sub-decision D-C1-1-SUB-A; research builds run a parallel job with `BUILD_VINS_MONO=ON` | 14550/udp (MAVLink to GCS), 5760/tcp (MSP2 to iNav SITL) |
| `ardupilot-plane-sitl` | `ardupilot/ardupilot-sitl:plane-stable` | ArduPilot Plane SITL. Receives `GPS_INPUT` from the SUT; we read its EKF source-set state to validate AC-4.3, AC-NEW-2, AC-5.x | 14550/udp (MAVLink) |
| `inav-sitl` | `inavflight/inav-sitl:9.0.0` | iNav SITL. Receives `MSP2_SENSOR_GPS` from the SUT; we read its GPS provider state | 5760/tcp (MSP2 over TCP per iNav SITL convention) |
| `mock-suite-sat-service` | local build (`e2e/fixtures/mock-suite-sat`) | Stubs the parent-suite Satellite Service tile-publish API (read-only ingest contract for AC-NEW-7 voting layer). Returns deterministic fixture tiles | 8080/tcp |
| `e2e-runner` | local build (`e2e/runner`) | Pytest-based harness. Drives all replays, reads FDR output, spins SITL scenarios. See § Harness Implementation Layout below for the per-evaluator inventory. | — |
| `mavproxy-listener` | `ardupilot/mavproxy:latest` | Passive MAVLink listener that captures the SUT → GCS stream into a per-run `.tlog` for assertions | 14551/udp |

### Networks

| Network | Services | Purpose |
|---------|----------|---------|
| `e2e-net` | all | Isolated test network. No host networking, no internet. Per RESTRICT-SAT-1, the SUT must NEVER reach an external satellite provider during a flight; a deny-all egress rule on `e2e-net` enforces this and is itself a security test (NFT-SEC-02). |

### Volumes

| Volume | Mounted to | Purpose |
|--------|-----------|---------|
| `tile-cache-fixture` | `gps-denied-onboard:/var/azaion/tile-cache:ro` | Pre-built FAISS HNSW index + tile filesystem. Built once per test run from `e2e/fixtures/tile-cache-builder/` from the 60 still-image satellite references and the Derkachi route bbox. Read-only mount mirrors AC-8.3 pre-flight load behavior. |
| `fdr-output` | `gps-denied-onboard:/var/azaion/fdr` | Per-flight FDR write target (AC-NEW-3 64 GB cap enforced via Docker `--storage-opt size=64g` on this volume) |
| `input-data` | `e2e-runner:/test-data:ro` | Bind mount of `_docs/00_problem/input_data/` for replay |
| `expected-results` | `e2e-runner:/expected:ro` | Bind mount of `_docs/00_problem/input_data/expected_results/` for assertions |

### docker-compose structure

```yaml
services:
  gps-denied-onboard:
    build:
      context: ../..
      dockerfile: docker/Dockerfile
      args:
        BUILD_VINS_MONO: "OFF"
    networks: [e2e-net]
    volumes:
      - tile-cache-fixture:/var/azaion/tile-cache:ro
      - fdr-output:/var/azaion/fdr
    environment:
      ONBOARD_FC_ADAPTER: ${FC_ADAPTER}        # ardupilot | inav, set per scenario
      ONBOARD_VIO_STRATEGY: ${VIO_STRATEGY}    # okvis2 | klt_ransac (production); vins_mono only in research build
      MAVLINK_SIGNING_PASSKEY_FILE: /run/secrets/mavlink_passkey
    depends_on:
      - mock-suite-sat-service

  ardupilot-plane-sitl:
    image: ardupilot/ardupilot-sitl:plane-stable
    networks: [e2e-net]
    command: ["--vehicle=ArduPlane", "--gps-type=14"]   # GPS_TYPE=14 = MAV per ArduPilot SITL_simulation_parameters.html

  inav-sitl:
    image: inavflight/inav-sitl:9.0.0
    networks: [e2e-net]
    # iNav SITL exposes MSP on TCP 5760 (UART1) per docs/SITL/SITL.md

  mock-suite-sat-service:
    build: ../fixtures/mock-suite-sat
    networks: [e2e-net]
    # Egress restriction enforced at network level, not service level

  e2e-runner:
    build: ../runner
    networks: [e2e-net]
    volumes:
      - input-data:/test-data:ro
      - expected-results:/expected:ro
      - fdr-output:/fdr:ro
    depends_on:
      - gps-denied-onboard
      - ardupilot-plane-sitl
      - inav-sitl
      - mavproxy-listener

  mavproxy-listener:
    image: ardupilot/mavproxy:latest
    networks: [e2e-net]

networks:
  e2e-net:
    driver: bridge
    internal: true   # NO external connectivity (enforces RESTRICT-SAT-1)

volumes:
  tile-cache-fixture: {}
  fdr-output: {}
```

## Consumer Application

**Tech stack**: Python 3.12, pytest 8.x, pymavlink (MAVLink ground side), `msp_gps_toy` (MSP2 ground side, Rust binary called via subprocess), OpenCV ≥4.11.0,<4.12 (frame source replay; see `_docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md` — pin is held below 4.12 until gtsam ships numpy-2 wheels; D-CROSS-CVE-1 leftover remains open), numpy ≥1.26,<2.0 + scipy (geodesic-distance assertions in WGS84).

**Entry point**: `pytest e2e/tests/` from inside `e2e-runner`. Each scenario is a parameterized pytest case keyed by FC adapter (`ardupilot` / `inav`) and VioStrategy (`okvis2` / `klt_ransac`) via the session-scoped conftest fixtures.

### Harness Implementation Layout

The blackbox harness implementation lives under `e2e/` (NOT the SUT source tree — public-boundary discipline enforced by `e2e/README.md`):

```
e2e/
├── docker/                            Tier-1 entrypoint
│   ├── docker-compose.test.yml        Compose stack (services from § Services above)
│   ├── docker-compose.tier2-bridge.yml  Compose override for paired-host Tier-2 SITL bridging
│   ├── run-tier1.sh                   AZ-444 selector-parity wrapper
│   └── secrets/                       Mounted Docker secrets (mavlink-passkey)
├── jetson/                            Tier-2 entrypoint
│   ├── run-tier2.sh                   AZ-444 selector-parity wrapper (control-host side)
│   ├── tier2-on-jetson.sh             SSH-orchestrated on-Jetson half
│   ├── tier2.service                  systemd unit template
│   ├── jtop_parser.py                 jetson_stats / jtop telemetry parser (NFT-LIM-01)
│   └── tegrastats_parser.py           tegrastats parser (NFT-LIM-04)
├── runner/                            e2e-runner image
│   ├── Dockerfile, conftest.py, pytest.ini, requirements.txt
│   ├── helpers/                       Per-AC evaluator + observer modules (47 evaluators
│   │                                  covering accuracy, AP/iNav contract, blackout-spoof,
│   │                                  cache poisoning, cold-start, companion reboot,
│   │                                  CVE probe, e2e latency, egress observer, escalation
│   │                                  ladder, FDR reader, frame-source replay, IMU replay,
│   │                                  injector fixtures, MAVLink signing, MAVProxy tlog,
│   │                                  memory budget, mid-flight tile, mock suite-sat audit,
│   │                                  Monte Carlo envelope, MRE, multi-segment, outage
│   │                                  request, outlier tolerance, registration classifier,
│   │                                  retrieval, sharp-turn, sitl_observer, smoothing,
│   │                                  spoof promotion, storage budget, streaming, thermal
│   │                                  envelope, tile-cache inspector, TTFF — see
│   │                                  `e2e/runner/helpers/` for the authoritative list)
│   └── reporting/                     CSV reporter + evidence bundler (AZ-445/446)
│       ├── csv_reporter.py            Emits `report.csv` per § Reporting
│       ├── evidence_bundler.py        Collects per-run `.tlog`, FDR, telemetry CSVs
│       └── nfr_recorder.py            NFR per-stage latency + budget recorder
├── fixtures/                          Fixture builders + captured fixtures
│   ├── tile-cache-builder/            `tile-cache-fixture` builder
│   ├── age-injector/                  `synth-age-tile-set` builder (FT-N-05)
│   ├── injectors/                     Runtime injectors:
│   │   ├── outlier.py                 `outlier-injection-derkachi` (FT-N-01)
│   │   ├── blackout_spoof.py          `blackout-spoof-derkachi` (FT-N-04, NFT-RES-04)
│   │   ├── multi_segment.py           `multi-segment-derkachi` (FT-P-08)
│   │   ├── cold_boot.py               `cold-boot-fixture` (NFT-PERF-03)
│   │   └── fc_proxy.py                FC-inbound blackout/spoof proxy (FT-N-04 driver)
│   ├── sitl_replay/                   Captured offline FDR-replay fixtures
│   │   └── p01/                       FT-P-01 capture set (see test-data.md)
│   ├── sitl_replay_builder/           Captured-fixture builder framework (AZ-598-600)
│   │   ├── builder.py                 VideoSource × TlogSource × FdrProjection strategies
│   │   ├── build_p01_fixtures.py      FT-P-01 still-image builder
│   │   └── build_p02_fixtures.py      FT-P-02 Derkachi builder
│   ├── mock-suite-sat/                `mock-suite-sat-service` Docker image
│   ├── secrets/                       Test-only secrets (mavlink-test-passkey.txt)
│   └── security/                      Security fixtures (cve-2025-53644.jpg)
├── tests/                             Pytest target: positive/, negative/, performance/,
│                                      resilience/, security/, resource_limit/
└── _unit_tests/                       Out-of-container unit tests for harness internals
                                       (runs as part of project pytest, no Docker required)
```

### Replay-Mode Skip Gating

Several FT-* and FT-N-* scenarios rely on a pre-captured FDR-replay fixture instead of a live SITL run. When the `E2E_SITL_REPLAY_DIR` environment variable is unset, those scenarios skip cleanly via a `sitl_replay_ready` pytest marker (per AZ-594/595/598/599). To activate them:

```bash
E2E_SITL_REPLAY_DIR=e2e/fixtures/sitl_replay/p01 \
    pytest e2e/tests/positive/test_ft_p_01_still_image_accuracy.py
```

The captured-fixture builder framework (`e2e/fixtures/sitl_replay_builder/`) regenerates these fixtures from `_docs/00_problem/input_data/` against a live compose stack; the captured artifacts are then committed under `e2e/fixtures/sitl_replay/<scenario>/`. See `e2e/fixtures/sitl_replay_builder/README.md` for the framework, supported scenarios, and per-scenario builder invocations.

### Communication with system under test

| Interface | Protocol | Endpoint / Topic | Authentication |
|-----------|----------|-----------------|----------------|
| Frame source | V4L2 / GStreamer file source | UNIX domain socket / shared `/test-data` mount | none (local) |
| FC telemetry inbound | MAVLink (AP) or MSP2 (iNav) | `udp:gps-denied-onboard:14550` (AP) or `tcp:gps-denied-onboard:5760` (iNav) | MAVLink 2.0 message signing on AP per D-C8-9 (passkey via Docker secret); iNav unsigned per accepted residual risk |
| Tile cache | Filesystem read | `/var/azaion/tile-cache` (read-only mount) | filesystem perms |
| FC external-pos outbound observation | Read SITL EKF source-set + GLOBAL_POSITION_INT replay back from SITL | `udp:ardupilot-plane-sitl:14550` or `tcp:inav-sitl:5760` | passive listener |
| GCS telemetry observation | MAVLink listener | `udp:mavproxy-listener:14551` (forwarded from SUT 14550) | none |
| FDR output | Filesystem read post-run | `/fdr` (read-only mount) | filesystem perms |
| Suite Sat Service mock | HTTP/JSON | `http://mock-suite-sat-service:8080` | none (test) |

### What the consumer does NOT have access to

- No direct access to the SUT's internal state (GTSAM iSAM2 graph, FAISS index in-memory, OpenCV intermediate buffers, VioStrategy implementation pointer).
- No internal Python/C++ module imports from the SUT.
- No shared memory or filesystem with the SUT outside the four explicit mounts (`tile-cache-fixture` r/o, `fdr-output` r/o from runner side, `input-data` r/o, `expected-results` r/o).
- No bypass of the FC-side acceptance check — every AC-4.3 assertion goes through SITL.

## CI/CD Integration

**When to run**:
- Tier-1 (workstation Docker): on every PR to `dev` branch and nightly on `dev` HEAD.
- Tier-2 (Jetson hardware loop): nightly on `dev`, and as a hard gate before any release tag.
- AC-NEW-5 thermal envelope: monthly on chamber-attached Jetson runner; failures block release tags only.

**Pipeline stage**:
- Tier-1 fits in the standard CI matrix as a single job (~30-45 min wall-clock for the full suite at first cut).
- Tier-2 is a separate workflow on `self-hosted-jetson-orin` runner.

**Gate behavior**: Tier-1 blocks PR merge on any test failure. Tier-2 blocks release tag on any test failure. Chamber tests are warning-only on PRs and blocking on release tags.

**Timeout**:
- Tier-1: 60 min per matrix entry.
- Tier-2: 4 hr per matrix entry (allows for full Derkachi 8 min replay × ~10 scenarios + cold-boot loops).
- Thermal chamber AC-NEW-5: 9 hr (8 h hot-soak + setup/teardown).

## Reporting

**Format**: CSV (one row per test).

**Columns**: `test_id, test_name, traces_to, fc_adapter, vio_strategy, tier, started_at_utc, execution_time_ms, result, error_message, evidence_paths`

- `traces_to`: comma-separated AC/RESTRICT IDs from the traceability matrix.
- `fc_adapter`: `ardupilot` | `inav` | `n/a`.
- `vio_strategy`: `okvis2` | `klt_ransac` | `vins_mono` | `n/a` (research-build only for `vins_mono`).
- `tier`: `tier1-docker` | `tier2-jetson` | `tier2-chamber`.
- `result`: `PASS` | `FAIL` | `SKIP` | `XFAIL` (XFAIL only allowed for AC explicitly marked NOT COVERED in the traceability matrix and not yet promoted to a real test).
- `evidence_paths`: comma-separated paths inside the run-output bundle (`.tlog` files, FDR archives, screenshots, profiler traces) supporting the verdict.

**Output path**: `e2e-results/run-${RUN_ID}/report.csv` plus a per-run bundle of evidence at `e2e-results/run-${RUN_ID}/evidence/`.

## Test Execution

**Decision (2026-05-09)**: **both** — Tier-1 Docker + Tier-2 Jetson hardware loop. Confirmed at the Hardware-Dependency Assessment Step 4 gate.

### Hardware dependencies found (Phase 3 → Hardware Assessment scan)

| Category | Indicator | Source file |
|---|---|---|
| GPU / CUDA | TensorRT engines (`.engine`, SM 87, JetPack 6.2, TRT 10.3) | `_docs/01_solution/solution.md` PRE-FLIGHT block |
| GPU / CUDA | DISK+LightGlue FP16 inference | `_docs/01_solution/solution.md` RUNTIME block (C3) |
| GPU / CUDA pin | Jetson Orin Nano Super (67 TOPS sparse INT8, 8 GB shared LPDDR5, 25 W) | `_docs/00_problem/restrictions.md` § Onboard Hardware |
| Sensors / Cameras | ADTi 20MP 20L V1 nadir camera over USB / MIPI-CSI / GigE | `_docs/00_problem/restrictions.md` § Cameras |
| Sensors / Cameras | V4L2 / GStreamer frame source (production) | `_docs/02_document/tests/environment.md` § Overview |
| OS-specific services | High-rate IMU via UART/MAVLink to FC | `_docs/00_problem/restrictions.md` § Sensors & Integration |
| OS-specific services | Per-FC inbound (MAVLink GPS_INPUT for AP, MSP2 over UART for iNav) | `_docs/00_problem/restrictions.md` § Sensors & Integration |
| OS-specific services | tegrastats / jetson_stats for thermal telemetry | `_docs/02_document/tests/resource-limit-tests.md` NFT-LIM-04 |
| Thermal envelope | -20 °C to +50 °C operating envelope, 25 W TDP, 8 h duty cycle | `_docs/00_problem/restrictions.md` § Failsafe & Safety + AC-NEW-5 |

(Step 2 Code scan from the planning phase returned zero indicators because no source code existed yet. Post-implementation: `pyproject.toml` confirms `tensorrt`, `pymavlink`, `gtsam==4.2.1`, `faiss-gpu`, `opencv-python>=4.11.0.86,<4.12` (cycle-1 relaxation per `_docs/_process_leftovers/2026-05-11_d_cross_cve_1_opencv_pin_deferred.md` — the original `>=4.12.0` target replays once gtsam ships numpy-2 wheels), and `jetson-stats`. `pycuda` was NOT added — TensorRT EP is invoked via ONNX Runtime + the `onnx_trt_ep_runtime` factory, which uses TensorRT's Python bindings directly without `pycuda`.)

### Execution instructions — Tier-1 (Docker)

**Prerequisites**:
- Docker 24+ with Compose v2.
- NVIDIA Container Toolkit if the workstation has an NVIDIA dGPU (lets the SUT exercise the TensorRT path; otherwise falls back to CPU TensorRT).
- ≥16 GB host RAM, ≥80 GB free disk for `tile-cache-fixture` + `fdr-output` + image build cache.

**How to start** (preferred — selector-parity wrapper from AZ-444):
```bash
./e2e/docker/run-tier1.sh \
    --fc-adapter ardupilot \
    --vio-strategy okvis2 \
    [-k <pytest selector>] \
    [--build-kind production|asan] \
    [--enable-chamber]
```

`run-tier1.sh` and `e2e/jetson/run-tier2.sh` accept the same `-k <selector>` flag and emit the same pytest invocation modulo the `TIER` env var (AZ-444 AC-1).

Raw-compose equivalent (when bypassing the wrapper for debugging):
```bash
cd e2e/docker
export FC_ADAPTER=ardupilot VIO_STRATEGY=okvis2
docker compose -f docker-compose.test.yml up --build --abort-on-container-exit e2e-runner
```

The run reports to `./e2e-results/run-${RUN_ID}/report.csv` (see § Reporting). Exit code matches the test verdict.

**Environment variables**:
- `FC_ADAPTER` ∈ `{ardupilot, inav}` — selects which SITL the SUT talks to.
- `VIO_STRATEGY` ∈ `{okvis2, klt_ransac}` for production binary; `vins_mono` only when the research binary `BUILD_VINS_MONO=ON` is the build.
- `MAVLINK_SIGNING_PASSKEY_FILE` — path to the Docker secret loaded with the test passkey for FT-P-09-AP / NFT-SEC-03.
- `E2E_SITL_REPLAY_DIR` — when set, activates captured-fixture FDR-replay mode for scenarios that gate on `sitl_replay_ready`; unset → those scenarios skip cleanly (see § Replay-Mode Skip Gating above).
- `RUN_ID` — per-invocation run identifier; defaults to `local-${USER}-${EPOCH}` in development, CI sets it from the workflow run id. Determines the `e2e-results/run-${RUN_ID}/` output directory.

**Skipped on Tier-1**: `NFT-PERF-01` (AC-4.1 latency p95 — Jetson-bound), `NFT-LIM-01` (AC-4.2 memory — Jetson-bound), `NFT-PERF-03` (AC-NEW-1 cold-start — Jetson-bound), `NFT-LIM-04` (AC-NEW-5 chamber baseline — Jetson-bound), AC-NEW-5 chamber portion (chamber-bound).

### Execution instructions — Tier-2 (Jetson hardware loop)

**Prerequisites**:
- Jetson Orin Nano Super (per `restrictions.md` § Onboard Hardware).
- JetPack 6.2 + CUDA + TensorRT 10.3 + cuDNN per D-C7-9.
- Workstation thermal-day environment for NFT-LIM-04 baseline. Chamber-attached runner for AC-NEW-5 chamber portion (separate quarterly job; not run in standard CI).
- ArduPilot Plane SITL + iNav SITL run on the same Jetson, OR on a paired x86 host on the same network — both are supported.
- Real ADTi 20MP 20L V1 camera connected via USB/MIPI-CSI/GigE; OR file-replay source if camera unavailable (in which case all `AC-2.x` cross-validation is `XFAIL` for that run).

**How to start** (AZ-444 selector-parity wrapper):
```bash
./e2e/jetson/run-tier2.sh \
    --fc-adapter ardupilot \
    --vio-strategy okvis2 \
    [-k <pytest selector>] \
    [--build-kind production|asan] \
    [--duration 5min|8h] \
    [--enable-chamber] \
    [--reflash]
```

The Tier-2 SITL stack runs on a paired x86 host via:
```bash
docker compose \
    -f e2e/docker/docker-compose.test.yml \
    -f e2e/docker/docker-compose.tier2-bridge.yml up ...
```

When invoked on a control host (typical), the script SSH-orchestrates the Jetson half (`tier2-on-jetson.sh`). When `TIER2_HOST=localhost` and the script runs on the Jetson itself, it delegates directly without SSH. Outputs the same CSV format as Tier-1 (one report.csv per run) plus tegrastats + jtop CSVs in the evidence bundle.

**Environment variables**: same as Tier-1 plus:
- `TIER2_HOST` / `TIER2_USER` / `TIER2_KEY_PATH` — control-host → Jetson SSH wiring (required when `TIER2_HOST != localhost`).
- `TIER2_CHAMBER_AMBIENT_C` — ambient temperature for AC-NEW-5 chamber runs.
- `TIER2_CAMERA_DEVICE` — `/dev/video0` (production) or file path for replay mode.

`gps-denied-onboard.service` (or `gps-denied-onboard-asan.service` for `--build-kind=asan`) MUST be installed via systemd on the Jetson — `e2e/jetson/tier2.service` is the template. See `_docs/03_implementation/jetson_harness_setup.md` for the physical provisioning steps.

### CI runner mapping

- `ubuntu-24.04` (GitHub-hosted) → Tier-1 Docker, every PR + nightly. ~30-45 min per matrix entry.
- `self-hosted-jetson-orin` → Tier-2 Jetson, nightly on `dev` HEAD + pre-release gate. ~4 hr per matrix entry.
- `self-hosted-jetson-orin-chamber` → AC-NEW-5 hot-soak. Quarterly + before any release tag. ~9 hr.

**Matrix dimensions**: `FC_ADAPTER × VIO_STRATEGY × build_kind` where `build_kind ∈ {production, research}`. Production `vins_mono` is excluded (D-C1-1-SUB-A locked); research includes all three VioStrategy values.