[AZ-406] Blackbox test harness bootstrap (Tier-1 + Tier-2 scaffold)

Bootstraps the public-boundary blackbox test harness owned by epic
AZ-262 (E-BBT). Establishes the e2e/ directory tree at the repo root,
fully separated from src/gps_denied_onboard/** and from the in-process
tests/** tree, and commits to the contracts every subsequent test
ticket (AZ-407..AZ-446) builds against.

Tier-1 (workstation Docker):
- docker/docker-compose.test.yml wires SUT + ArduPilot SITL + iNav SITL
  + mock Suite Sat Service + mavproxy listener + e2e-runner onto one
  e2e-net bridge with internal: true (enforces RESTRICT-SAT-1 /
  NFT-SEC-02 egress isolation at the network layer).
- docker/docker-compose.tier2-bridge.yml override disables the in-
  compose SUT so Tier-2 pairs SITLs + mock + runner on an x86 host
  while the SUT runs natively on the Jetson under systemd.

Tier-2 (Jetson):
- jetson/run-tier2.sh + tier2.service systemd unit + tegrastats /
  jtop parsers feed per-sample telemetry into the evidence bundle.

Runner image (e2e/runner/):
- Dockerfile + requirements.txt install ONLY ground-side libs
  (pymavlink, opencv-python>=4.12, numpy/scipy/geopy/pyproj, httpx,
  orjson, pydantic, structlog, pytest 8.x). The runner deliberately
  does NOT install the SUT package.
- conftest.py implements the AC-9 skip-rule mapping (tier2_only,
  chamber_only, vins_mono, deferred_ac) tied to environment.md
  parametrize axes.
- reporting/csv_reporter.py is a pytest plugin emitting one row per
  test with the exact 11-column schema from environment.md §
  Reporting (test_id, test_name, traces_to, fc_adapter, vio_strategy,
  tier, started_at_utc, execution_time_ms, result, error_message,
  evidence_paths). XFAIL surfaced only when a test carries
  @pytest.mark.deferred_ac(verdict="xfail", reason=...).
- reporting/evidence_bundler.py exposes the attach_evidence fixture
  that copies per-test artifacts (.tlog, FDR archives, screenshots,
  tegrastats / jtop CSVs) into the run bundle and records relative
  paths into the reporter's evidence_paths column.
- helpers/{frame_source_replay,imu_replay,sitl_observer,
  mavproxy_tlog_reader,fdr_reader}.py declare the public surfaces
  (concrete implementations owned by AZ-407 / AZ-408 / AZ-416 /
  AZ-417 / AZ-441 per the dependency table); helpers/geo.py ships
  today (no downstream task dep) — WGS84 distance / forward-bearing
  / offset via pyproj with NaN rejection.

Mock Suite Sat Service (e2e/fixtures/mock-suite-sat/):
- FastAPI app: POST /tiles (ingest contract from D-PROJ-2 follow-up),
  GET /tiles/audit + /mock/audit (per-run read-back), POST
  /mock/config (force-status, response delay), POST /mock/reset
  (clears audit between tests), GET /mock/health.

Fixture scaffolds (e2e/fixtures/{tile-cache-builder, age-injector,
injectors, cold-boot, secrets, security}/):
- Public surfaces only. Concrete builders land in AZ-407 (static
  fixtures), AZ-408 (runtime synthetic injection), AZ-419 (cold-boot
  fixture), AZ-439 (CVE-2025-53644 JPEG generator).

Test tree (e2e/tests/{positive,negative,performance,resilience,
security,resource_limit}/):
- Mirror of the test-spec category grouping in
  _docs/02_document/tests/*-tests.md.
- tests/positive/test_smoke.py is the AC-1 harness-boot smoke run
  inside the e2e-runner image once Docker brings everything up.

Out-of-container unit tests (e2e/_unit_tests/):
- Exercises the harness internals (CSV reporter plugin lifecycle,
  conftest skip rules, helper modules, parsers, mock app, compose
  YAML structural contract, public-boundary enforcement) without
  Docker / SITL. 97 unit tests, all passing.

Build / config:
- pyproject.toml: testpaths extended with e2e/_unit_tests; pythonpath
  extended with e2e; fastapi>=0.111,<0.120 added to dev extras for the
  mock-app TestClient unit test.

AC coverage:
- AC-1 (Tier-1 boot)         → compose YAML test + directory layout
                                + smoke test (Docker-bound)
- AC-2 (mock services)       → 6 FastAPI TestClient unit tests
- AC-3 (SITLs accept output) → contract present; concrete check
                                deferred to AZ-416 / AZ-417
- AC-4 (CSV columns)         → in-process plugin lifecycle test
                                emits the exact 11-column schema
- AC-5 (egress isolation)    → static config test + runtime probe
                                in Docker-bound smoke
- AC-6 (Tier-2 contract)     → tegrastats + jtop parser unit tests
                                + jetson/* layout test; full Tier-2
                                contract is AZ-444
- AC-7 (fixture reproducibility) → deferred to AZ-407 per task spec
- AC-8 (parametrize matrix)  → vins_mono skip-rule cases +
                                tests/positive/test_smoke
- AC-9 (skip semantics)      → 9 conftest skip-rule unit tests

Module layout entry for blackbox_tests was added in 2026-05-16
preparatory commit d7a17a8 so this diff stays focused on the harness
scaffold. AZ-406 advances to In Testing on commit.

Co-authored-by: Cursor <cursoragent@cursor.com>

e2e/docker/docker-compose.test.yml

@@ -0,0 +1,149 @@
+# Tier-1 docker-compose entrypoint for the gps-denied-onboard blackbox e2e harness.
+#
+# Spec sources (single source of truth):
+#   _docs/02_document/tests/environment.md   § Docker Environment
+#   _docs/02_tasks/todo/AZ-406_test_infrastructure.md
+#
+# Layout note: AZ-406 introduces this file; later test-task batches may add
+# per-scenario override files alongside it (e.g. negative path injectors).
+# This base file MUST stay self-contained — every override is purely additive.
+#
+# Build context (`build.context: ../..`) is the repo root, so the SUT image
+# build sees `src/`, `cpp/`, `docker/Dockerfile`, and `pyproject.toml`.
+
+services:
+
+  gps-denied-onboard:
+    build:
+      context: ../..
+      dockerfile: docker/Dockerfile
+      args:
+        BUILD_VINS_MONO: "OFF"
+    image: gps-denied-onboard:e2e
+    networks: [e2e-net]
+    volumes:
+      - tile-cache-fixture:/var/azaion/tile-cache:ro
+      - fdr-output:/var/azaion/fdr
+    environment:
+      ONBOARD_FC_ADAPTER: ${FC_ADAPTER:-ardupilot}
+      ONBOARD_VIO_STRATEGY: ${VIO_STRATEGY:-okvis2}
+      MAVLINK_SIGNING_PASSKEY_FILE: /run/secrets/mavlink_passkey
+    secrets:
+      - mavlink_passkey
+    depends_on:
+      - mock-suite-sat-service
+    healthcheck:
+      test: ["CMD", "python", "-c", "from gps_denied_onboard.healthcheck import check; check()"]
+      interval: 5s
+      retries: 12
+
+  ardupilot-plane-sitl:
+    image: ardupilot/ardupilot-sitl:plane-stable
+    networks: [e2e-net]
+    command: ["--vehicle=ArduPlane", "--gps-type=14"]
+    environment:
+      # GPS_TYPE=14 selects MAV (external positioning) per ArduPilot SITL params.
+      AP_PARAM_GPS_TYPE: "14"
+
+  inav-sitl:
+    image: inavflight/inav-sitl:9.0.0
+    networks: [e2e-net]
+    # iNav SITL exposes MSP on TCP 5760 (UART1) per docs/SITL/SITL.md
+
+  mock-suite-sat-service:
+    build: ../fixtures/mock-suite-sat
+    image: mock-suite-sat-service:e2e
+    networks: [e2e-net]
+    environment:
+      MOCK_SUITE_SAT_AUDIT_PATH: /audit
+    volumes:
+      - mock-audit:/audit
+    healthcheck:
+      test: ["CMD", "python", "-c", "import urllib.request, sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8080/mock/health', timeout=2).status==200 else 1)"]
+      interval: 5s
+      retries: 12
+
+  mavproxy-listener:
+    image: ardupilot/mavproxy:latest
+    networks: [e2e-net]
+    command:
+      - "--master=udp:0.0.0.0:14551"
+      - "--logfile=/var/log/tlogs/${RUN_ID:-local}.tlog"
+      - "--out=udp:e2e-runner:14552"
+    volumes:
+      - tlog-output:/var/log/tlogs
+
+  e2e-runner:
+    build: ../runner
+    image: gps-denied-onboard-e2e-runner:latest
+    networks: [e2e-net]
+    environment:
+      RUN_ID: ${RUN_ID:-local}
+      FC_ADAPTER: ${FC_ADAPTER:-ardupilot}
+      VIO_STRATEGY: ${VIO_STRATEGY:-okvis2}
+      TIER: tier1-docker
+      MAVLINK_PASSKEY_PATH: /test-fixtures/secrets/mavlink-test-passkey.txt
+      MOCK_SUITE_SAT_URL: http://mock-suite-sat-service:8080
+      AP_SITL_HOST: ardupilot-plane-sitl
+      INAV_SITL_HOST: inav-sitl
+      MAVPROXY_LISTENER_HOST: mavproxy-listener
+    volumes:
+      - ../../_docs/00_problem/input_data:/test-data:ro
+      - ../../_docs/00_problem/input_data/expected_results:/expected:ro
+      - ../fixtures:/test-fixtures:ro
+      - ../tests:/test-suite:ro
+      - fdr-output:/fdr:ro
+      - tlog-output:/tlogs:ro
+      - e2e-results:/e2e-results
+      - mock-audit:/mock-audit:ro
+    command:
+      - "pytest"
+      - "/test-suite"
+      - "--csv=/e2e-results/run-${RUN_ID:-local}/report.csv"
+      - "--csv-columns=test_id,test_name,traces_to,fc_adapter,vio_strategy,tier,started_at_utc,execution_time_ms,result,error_message,evidence_paths"
+      - "--evidence-out=/e2e-results/run-${RUN_ID:-local}/evidence"
+    depends_on:
+      gps-denied-onboard:
+        condition: service_healthy
+      mock-suite-sat-service:
+        condition: service_healthy
+      ardupilot-plane-sitl:
+        condition: service_started
+      inav-sitl:
+        condition: service_started
+      mavproxy-listener:
+        condition: service_started
+
+networks:
+  e2e-net:
+    driver: bridge
+    # CRITICAL: enforces RESTRICT-SAT-1 / NFT-SEC-02 / NFT-SEC-05 at the network layer.
+    # The SUT, mock, runner, and SITLs can talk to each other but none of them can
+    # reach the public internet (no DNS, no egress). The e2e-runner verifies this
+    # at runtime by attempting a TCP connect to 1.1.1.1:443 (AC-5).
+    internal: true
+
+volumes:
+  # Size cap follows AC-NEW-3: each FDR file ≤ 64 GB. The volume layer cap is
+  # belt-and-suspenders; the SUT enforces the cap internally per NFT-LIM-02.
+  # `--storage-opt size=64g` requires overlay2 with xfs backing on the host; CI
+  # YAML notes the fallback for CI runners that lack that driver combination.
+  fdr-output:
+    driver: local
+    driver_opts:
+      type: tmpfs
+      device: tmpfs
+      o: "size=64g"
+  tile-cache-fixture: {}
+  tlog-output: {}
+  mock-audit: {}
+  e2e-results:
+    driver: local
+    driver_opts:
+      type: none
+      device: ${PWD}/../../e2e-results
+      o: bind
+
+secrets:
+  mavlink_passkey:
+    file: ./secrets/mavlink_passkey

e2e/docker/docker-compose.tier2-bridge.yml

@@ -0,0 +1,36 @@
+# Tier-2 bridge override. Used when the SITLs and the runner run on a paired
+# x86 host while the SUT runs natively on the Jetson under systemd. Provisions
+# only the SITLs + mock + listener + runner; the SUT block is intentionally
+# omitted because Tier-2 owns the SUT lifecycle via `systemctl`.
+#
+# Usage (Tier-2):
+#   cd e2e/docker
+#   docker compose -f docker-compose.test.yml -f docker-compose.tier2-bridge.yml up \
+#       --build --abort-on-container-exit e2e-runner ardupilot-plane-sitl inav-sitl
+#
+# The override removes the `gps-denied-onboard` service entirely (the override
+# below sets `profiles: ["disabled"]`) and points the runner at the Jetson host
+# via `JETSON_HOST` so the FC adapter target is the real device.
+
+services:
+
+  gps-denied-onboard:
+    profiles: ["disabled"]
+
+  e2e-runner:
+    environment:
+      TIER: tier2-jetson
+      # The Jetson host's reachable hostname / IP — operator sets this when
+      # invoking docker compose on the paired x86 box.
+      JETSON_HOST: ${JETSON_HOST:?must set JETSON_HOST when using tier2-bridge}
+      # The SUT is no longer in compose; the runner does NOT depend on the
+      # `gps-denied-onboard` service and observes it only via SITL + FDR.
+    depends_on:
+      mock-suite-sat-service:
+        condition: service_healthy
+      ardupilot-plane-sitl:
+        condition: service_started
+      inav-sitl:
+        condition: service_started
+      mavproxy-listener:
+        condition: service_started

e2e/docker/secrets/README.md

@@ -0,0 +1,14 @@
+# Docker secrets (TEST ONLY)
+
+This directory mounts as Docker secrets into the `gps-denied-onboard` service.
+The `mavlink_passkey` file is a deterministic 32-byte hex string used solely
+for FT-P-09-AP / NFT-SEC-03 testing of MAVLink 2.0 message signing.
+
+**Production deployments MUST NOT use this file.** Production wires the
+passkey via `/run/secrets/mavlink_passkey` from a real secret store; the test
+fixture path here is intercepted at compose build time so the production
+artifact never sees this value.
+
+The matching key on the runner side lives at
+`e2e/fixtures/secrets/mavlink-test-passkey.txt` (same bytes) — pymavlink
+loads it from there when constructing the signed-message peer.

e2e/docker/secrets/mavlink_passkey

@@ -0,0 +1 @@
+0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef