# E2E Test Suite

## Scope

The e2e test suite is separate test tooling, not part of the onboard runtime. It drives black-box replay, public dataset, SITL, Jetson, and representative validation through public runtime interfaces only.

## Purpose

- Feed navigation frames, telemetry traces, cache manifests, and fault triggers into the system under test.
- Validate emitted coordinates, confidence fields, MAVLink `GPS_INPUT`, QGC status, FDR, and generated-tile evidence.
- Produce release evidence without importing runtime internals.

## Ownership

- **Epic**: AZ-217 (E2E Test Suite / test-support work, not product runtime)
- **Owns**:
  - `tests/blackbox/**`
  - `tests/e2e/**`
  - `e2e/replay/**`
  - `e2e/reports/**`
- **Does not own**:
  - `src/**`
  - runtime component internals
  - production deployment code

## Public Interfaces Under Test

| Interface | Protocol / Contract |
|-----------|---------------------|
| Navigation frames | Ordered image/video replay with timestamps |
| FC telemetry | MAVLink replay or generated stream |
| Satellite cache | Local COG + manifest + descriptor fixtures |
| GPS output | MAVLink `GPS_INPUT` |
| Operator status | QGC-visible MAVLink status |
| FDR | Filesystem/database-backed evidence outputs |

## Runner Contract

| Method | Input | Output | Error Types |
|--------|-------|--------|-------------|
| `run_scenario` | `ScenarioRequest` | `ScenarioReport` | `FixtureInvalid`, `RuntimeFailed`, `ThresholdFailed` |
| `validate_fixture` | `FixtureRequest` | `FixtureValidationReport` | `FixtureInvalid` |

```yaml
ScenarioRequest:
  scenario_id: string
  execution_environment: enum(replay, sitl, jetson, representative)
  fixture_paths: list[string]

ScenarioReport:
  scenario_id: string
  result: enum(pass, fail, blocked)
  metrics: object
  artifacts: list[path]
  failure_reason: string optional
```

## Scenario Coverage

| Scenario | Purpose | Evidence |
|----------|---------|----------|
| Still-image accuracy runner | Verify project still-image replay reports frame-center accuracy | Per-image error, aggregate pass rates, covariance, source label, anchor age |
| Synchronized VIO replay runner | Verify Derkachi and public/representative synchronized data drive BASALT/wrapper tests | Fixture alignment, trajectory comparison, VIO registration, latency, covariance calibration |
| Satellite anchor replay runner | Verify VPR and anchor verification scenarios are executable | Retrieval recall, MRE, accepted/rejected anchors, freshness behavior |
| Outlier/sharp-turn/disconnected runner | Verify relocalization resilience scenarios are executable | Degraded-mode timelines and relocalization outcomes |
| Blackout and spoofing runner | Verify total blackout plus spoofing through SITL/replay | Mode-switch timing, covariance growth, failsafe thresholds |
| MAVLink/QGC contract runner | Verify MAVLink output and GCS status assertions | `GPS_INPUT`, WGS84 coordinates, status rate, command ingress |
| Startup/reboot runner | Verify cold-start and companion reboot scenarios | First valid `GPS_INPUT` p95 and FC-state reinitialization |
| Object coordinate contract runner | Verify AI-camera object coordinate request at system boundary | Frame-center-consistent coordinate accuracy and projection bound |
| Tile Manager runner | Verify cache, generated tiles, and storage tests | Cache load, tile write gates, no raw-frame retention, stale rejection, poisoning evidence |

## Release Evidence

The suite assembles CSV, Markdown, MAVLink tlogs, FDR summaries, cache validation reports, and pass/fail metadata into release evidence bundles. Missing public or representative data is reported as `blocked`, not `passed`.

## Non-Responsibilities

- No onboard flight logic.
- No direct estimator, BASALT, wrapper, or tile-manager imports.
- No mutation of runtime internal state.
- No production service APIs.