[AZ-406] Blackbox test harness bootstrap (Tier-1 + Tier-2 scaffold)

Bootstraps the public-boundary blackbox test harness owned by epic
AZ-262 (E-BBT). Establishes the e2e/ directory tree at the repo root,
fully separated from src/gps_denied_onboard/** and from the in-process
tests/** tree, and commits to the contracts every subsequent test
ticket (AZ-407..AZ-446) builds against.

Tier-1 (workstation Docker):
- docker/docker-compose.test.yml wires SUT + ArduPilot SITL + iNav SITL
  + mock Suite Sat Service + mavproxy listener + e2e-runner onto one
  e2e-net bridge with internal: true (enforces RESTRICT-SAT-1 /
  NFT-SEC-02 egress isolation at the network layer).
- docker/docker-compose.tier2-bridge.yml override disables the in-
  compose SUT so Tier-2 pairs SITLs + mock + runner on an x86 host
  while the SUT runs natively on the Jetson under systemd.

Tier-2 (Jetson):
- jetson/run-tier2.sh + tier2.service systemd unit + tegrastats /
  jtop parsers feed per-sample telemetry into the evidence bundle.

Runner image (e2e/runner/):
- Dockerfile + requirements.txt install ONLY ground-side libs
  (pymavlink, opencv-python>=4.12, numpy/scipy/geopy/pyproj, httpx,
  orjson, pydantic, structlog, pytest 8.x). The runner deliberately
  does NOT install the SUT package.
- conftest.py implements the AC-9 skip-rule mapping (tier2_only,
  chamber_only, vins_mono, deferred_ac) tied to environment.md
  parametrize axes.
- reporting/csv_reporter.py is a pytest plugin emitting one row per
  test with the exact 11-column schema from environment.md §
  Reporting (test_id, test_name, traces_to, fc_adapter, vio_strategy,
  tier, started_at_utc, execution_time_ms, result, error_message,
  evidence_paths). XFAIL surfaced only when a test carries
  @pytest.mark.deferred_ac(verdict="xfail", reason=...).
- reporting/evidence_bundler.py exposes the attach_evidence fixture
  that copies per-test artifacts (.tlog, FDR archives, screenshots,
  tegrastats / jtop CSVs) into the run bundle and records relative
  paths into the reporter's evidence_paths column.
- helpers/{frame_source_replay,imu_replay,sitl_observer,
  mavproxy_tlog_reader,fdr_reader}.py declare the public surfaces
  (concrete implementations owned by AZ-407 / AZ-408 / AZ-416 /
  AZ-417 / AZ-441 per the dependency table); helpers/geo.py ships
  today (no downstream task dep) — WGS84 distance / forward-bearing
  / offset via pyproj with NaN rejection.

Mock Suite Sat Service (e2e/fixtures/mock-suite-sat/):
- FastAPI app: POST /tiles (ingest contract from D-PROJ-2 follow-up),
  GET /tiles/audit + /mock/audit (per-run read-back), POST
  /mock/config (force-status, response delay), POST /mock/reset
  (clears audit between tests), GET /mock/health.

Fixture scaffolds (e2e/fixtures/{tile-cache-builder, age-injector,
injectors, cold-boot, secrets, security}/):
- Public surfaces only. Concrete builders land in AZ-407 (static
  fixtures), AZ-408 (runtime synthetic injection), AZ-419 (cold-boot
  fixture), AZ-439 (CVE-2025-53644 JPEG generator).

Test tree (e2e/tests/{positive,negative,performance,resilience,
security,resource_limit}/):
- Mirror of the test-spec category grouping in
  _docs/02_document/tests/*-tests.md.
- tests/positive/test_smoke.py is the AC-1 harness-boot smoke run
  inside the e2e-runner image once Docker brings everything up.

Out-of-container unit tests (e2e/_unit_tests/):
- Exercises the harness internals (CSV reporter plugin lifecycle,
  conftest skip rules, helper modules, parsers, mock app, compose
  YAML structural contract, public-boundary enforcement) without
  Docker / SITL. 97 unit tests, all passing.

Build / config:
- pyproject.toml: testpaths extended with e2e/_unit_tests; pythonpath
  extended with e2e; fastapi>=0.111,<0.120 added to dev extras for the
  mock-app TestClient unit test.

AC coverage:
- AC-1 (Tier-1 boot)         → compose YAML test + directory layout
                                + smoke test (Docker-bound)
- AC-2 (mock services)       → 6 FastAPI TestClient unit tests
- AC-3 (SITLs accept output) → contract present; concrete check
                                deferred to AZ-416 / AZ-417
- AC-4 (CSV columns)         → in-process plugin lifecycle test
                                emits the exact 11-column schema
- AC-5 (egress isolation)    → static config test + runtime probe
                                in Docker-bound smoke
- AC-6 (Tier-2 contract)     → tegrastats + jtop parser unit tests
                                + jetson/* layout test; full Tier-2
                                contract is AZ-444
- AC-7 (fixture reproducibility) → deferred to AZ-407 per task spec
- AC-8 (parametrize matrix)  → vins_mono skip-rule cases +
                                tests/positive/test_smoke
- AC-9 (skip semantics)      → 9 conftest skip-rule unit tests

Module layout entry for blackbox_tests was added in 2026-05-16
preparatory commit d7a17a8 so this diff stays focused on the harness
scaffold. AZ-406 advances to In Testing on commit.

Co-authored-by: Cursor <cursoragent@cursor.com>

e2e/_unit_tests/reporting/test_csv_reporter.py

@@ -0,0 +1,204 @@
+"""Unit tests for `runner.reporting.csv_reporter`.
+
+Covers two layers:
+    1. `build_row` — pure function exercised with fake `Item` / `TestReport`
+       objects. Verifies the column set and result classification logic.
+    2. Plugin smoke-test — runs a tiny in-process pytest invocation against
+       a temporary test file with the plugin registered, then reads the CSV
+       output back and asserts the column ordering matches CSV_COLUMNS.
+"""
+
+from __future__ import annotations
+
+import csv
+import sys
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Any
+
+import pytest
+
+from runner.reporting.csv_reporter import CSV_COLUMNS, build_row
+
+
+class _FakeItem:
+    """Minimal duck-typed pytest.Item replacement for unit tests."""
+
+    def __init__(
+        self,
+        nodeid: str = "tests/test_x.py::test_y",
+        name: str = "test_y",
+        markers: list[SimpleNamespace] | None = None,
+        callspec: SimpleNamespace | None = None,
+    ) -> None:
+        self.nodeid = nodeid
+        self.name = name
+        self._markers = markers or []
+        self.callspec = callspec
+
+    def get_closest_marker(self, name: str) -> SimpleNamespace | None:
+        return next((m for m in self._markers if m.name == name), None)
+
+
+def _report(outcome: str, when: str = "call", longrepr: Any = "") -> SimpleNamespace:
+    return SimpleNamespace(
+        outcome=outcome,
+        when=when,
+        longreprtext=str(longrepr) if outcome == "failed" else "",
+        longrepr=longrepr,
+    )
+
+
+# ---------------------------------------------------------------------------
+# build_row unit tests
+# ---------------------------------------------------------------------------
+
+
+def test_build_row_pass_minimal() -> None:
+    # Arrange
+    item = _FakeItem()
+    report = _report("passed")
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 42, [])
+    # Assert
+    assert set(row.keys()) == set(CSV_COLUMNS)
+    assert row["result"] == "PASS"
+    assert row["test_id"] == "tests/test_x.py::test_y"
+    assert row["execution_time_ms"] == "42"
+    assert row["error_message"] == ""
+
+
+def test_build_row_fail_attaches_error_message() -> None:
+    # Arrange
+    item = _FakeItem()
+    report = _report("failed", longrepr="boom\nat line 4")
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 10, [])
+    # Assert
+    assert row["result"] == "FAIL"
+    assert "boom" in row["error_message"]
+    assert "\n" not in row["error_message"]  # collapsed for CSV friendliness
+
+
+def test_build_row_skip_records_reason() -> None:
+    # Arrange
+    item = _FakeItem()
+    report = _report("skipped", when="setup", longrepr=("file.py", 5, "deferred: AC-7.1"))
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 1)
+    # Assert
+    assert row["result"] == "SKIP"
+    assert row["error_message"] == "deferred: AC-7.1"
+
+
+def test_build_row_xfail_when_deferred_ac_xfail_verdict() -> None:
+    # Arrange
+    marker = SimpleNamespace(
+        name="deferred_ac", args=(), kwargs={"verdict": "xfail", "reason": "AC-8.6 scene-change PARTIAL"}
+    )
+    item = _FakeItem(markers=[marker])
+    report = _report("skipped", longrepr=("file.py", 5, "xfail strict=False"))
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 1)
+    # Assert
+    assert row["result"] == "XFAIL"
+
+
+def test_build_row_uses_test_id_marker_when_set() -> None:
+    # Arrange
+    marker = SimpleNamespace(name="test_id", args=("FT-P-01",), kwargs={})
+    item = _FakeItem(markers=[marker])
+    report = _report("passed")
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 1)
+    # Assert
+    assert row["test_id"] == "FT-P-01"
+
+
+def test_build_row_emits_traces_to_csv() -> None:
+    # Arrange
+    marker = SimpleNamespace(name="traces_to", args=(["AC-1.1", "AC-1.2"],), kwargs={})
+    item = _FakeItem(markers=[marker])
+    report = _report("passed")
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 1)
+    # Assert
+    assert row["traces_to"] == "AC-1.1,AC-1.2"
+
+
+def test_build_row_propagates_parametrize_ids() -> None:
+    # Arrange
+    callspec = SimpleNamespace(params={"fc_adapter": "ardupilot", "vio_strategy": "okvis2"})
+    item = _FakeItem(callspec=callspec)
+    report = _report("passed")
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 1)
+    # Assert
+    assert row["fc_adapter"] == "ardupilot"
+    assert row["vio_strategy"] == "okvis2"
+
+
+def test_build_row_records_evidence_paths() -> None:
+    # Arrange
+    item = _FakeItem()
+    report = _report("passed")
+    # Act
+    row = build_row(item, report, "2026-05-16T10:00:00+00:00", 1, ["evidence/a.tlog", "evidence/b.csv"])
+    # Assert
+    assert row["evidence_paths"] == "evidence/a.tlog,evidence/b.csv"
+
+
+# ---------------------------------------------------------------------------
+# In-process plugin integration
+# ---------------------------------------------------------------------------
+
+PLUGIN_INTEGRATION = """
+import pytest
+
+pytest_plugins = ["runner.reporting.csv_reporter"]
+
+
+@pytest.mark.traces_to(["AC-1"])
+@pytest.mark.test_id("UNIT-CSV-01")
+def test_passing():
+    assert 1 == 1
+
+
+def test_failing():
+    assert 1 == 2
+"""
+
+
+def test_csv_plugin_emits_required_columns(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    """Run pytest in-process with the CSV plugin and assert the column header matches CSV_COLUMNS."""
+    # Arrange
+    test_file = tmp_path / "test_plugin_smoke.py"
+    test_file.write_text(PLUGIN_INTEGRATION, encoding="utf-8")
+    csv_out = tmp_path / "report.csv"
+    monkeypatch.setenv("TIER", "tier1-docker")
+    # Make `runner.*` importable from the in-process pytest.
+    e2e_root = Path(__file__).resolve().parents[2]
+    monkeypatch.syspath_prepend(str(e2e_root))
+    # Act — `-p runner.reporting.csv_reporter` registers the plugin BEFORE option parsing,
+    # otherwise pytest rejects `--csv=...` as unrecognized.
+    rc = pytest.main([
+        "-p", "runner.reporting.csv_reporter",
+        str(test_file),
+        f"--csv={csv_out}",
+        "--no-header",
+        "-q",
+    ])
+    # Assert
+    # rc=1 is expected because test_failing intentionally fails.
+    assert rc in (0, 1), f"unexpected pytest rc={rc}"
+    assert csv_out.exists(), "csv_reporter did not write the report file"
+    with csv_out.open() as fh:
+        reader = csv.DictReader(fh)
+        rows = list(reader)
+        assert reader.fieldnames == list(CSV_COLUMNS)
+    # Both rows should be present (one passed, one failed).
+    assert len(rows) == 2
+    results = {row["test_id"]: row["result"] for row in rows}
+    assert "UNIT-CSV-01" in results and results["UNIT-CSV-01"] == "PASS"
+    failing_row = next(row for row in rows if row["result"] == "FAIL")
+    assert "assert" in failing_row["error_message"].lower()