# Replay-input CSV format (AZ-896)

**Status**: canonical operator-facing spec for the `--imu` argument of
`gps-denied-replay` (AZ-894).
**Audience**: operators preparing a (video, CSV) replay pair, plus engineers
implementing alternative replay backends.
**Companion artifacts**:

- `_docs/02_document/contracts/replay/example_data_imu.csv` — minimal valid
  example (20 rows = 2 s at 10 Hz).
- `_docs/00_problem/input_data/flight_derkachi/data_imu.csv` — full Derkachi
  fixture (4,900 rows = 489.9 s at 10 Hz).
- Parser implementation:
  `src/gps_denied_onboard/replay_input/csv_ground_truth.py`.

## Hard contract (read before generating a file)

The replay pipeline trusts the CSV blindly inside the loop. Violations of any
of the following will produce silently wrong outputs (the parser only catches
schema-level faults, not semantic ones), so the operator owns these
invariants:

1. **Nadir camera.** The companion `.mp4` must be a nadir (straight-down)
   recording. The C1 VIO and C2 VPR stages assume nadir framing; oblique
   imagery breaks the satellite-anchor and VIO scale recovery.
2. **Airborne at row 0.** The UAV must already be airborne at the first CSV
   row / first video frame. The replay pipeline does not implement a
   take-off detector — feeding a ground-roll segment yields garbage IMU
   integration.
3. **Aligned start.** Row 0's `Time = 0.0` must correspond to the first
   video frame. The CLI does not perform sub-frame alignment; offset the
   CSV/clip pair offline before invoking `gps-denied-replay`.
4. **Monotonic, uniformly-spaced `Time`.** Rows must be strictly increasing
   on `Time` and uniformly spaced (the Derkachi fixture is 10 Hz). The
   parser enforces monotonicity (AC-5); uniform spacing is the operator's
   responsibility — non-uniform spacing skews the ESKF prediction step
   without raising an error.

## Schema

The CSV must be header-first, comma-separated, UTF-8 encoded. Column order
does not matter — the parser uses `csv.DictReader` and looks up by name —
but the column **names** must match exactly (case-sensitive).

15 columns are required; up to 4 additional columns (mag fields,
`relative_alt`) are tolerated and ignored.

### Required columns

CSV columns use the MAVLink wire format (mG accel, mrad/s gyro, FRD
body frame). The parser converts to SI / FLU at the `ImuSample`
boundary via
`gps_denied_onboard.helpers.imu_units.mavlink_imu_to_si_flu` (AZ-918)
so downstream C5 ESKF + `imu_preintegrator` consumers see the contract
they were built for. **Operator-facing CSV files keep the raw scaling**
— the conversion is a parser-internal concern.

| # | Column | Unit (CSV) | Type | Notes |
|---|--------|------------|------|-------|
| 1 | `timestamp(ms)` | ms | float | Pixhawk wall clock at sample capture. **Ignored by the replay pipeline** — kept only for trace-back to the original tlog. |
| 2 | `Time` | s | float | **Canonical replay clock.** Must start at `0.0`, increase monotonically, and be uniformly spaced. The replay loop uses this column for every timestamp it emits. |
| 3 | `SCALED_IMU2.xacc` | mg, FRD | float | Body-frame X accelerometer, MAVLink `SCALED_IMU2` raw scaling. Converted by the parser to m/s² in `ImuSample.accel_xyz[0]` (FLU body). |
| 4 | `SCALED_IMU2.yacc` | mg, FRD | float | Body-frame Y accelerometer; sign-flipped during FRD→FLU. |
| 5 | `SCALED_IMU2.zacc` | mg, FRD | float | Body-frame Z accelerometer; sign-flipped during FRD→FLU. |
| 6 | `SCALED_IMU2.xgyro` | mrad/s, FRD | float | Body-frame X gyro, MAVLink `SCALED_IMU2` raw scaling. Converted to rad/s in `ImuSample.gyro_xyz[0]` (FLU body). |
| 7 | `SCALED_IMU2.ygyro` | mrad/s, FRD | float | Body-frame Y gyro; sign-flipped during FRD→FLU. |
| 8 | `SCALED_IMU2.zgyro` | mrad/s, FRD | float | Body-frame Z gyro; sign-flipped during FRD→FLU. |
| 9 | `GLOBAL_POSITION_INT.lat` | degrees | float | WGS84 latitude. **Already in decimal degrees** (Derkachi dump convention — pre-divided by 1e7 from MAVLink's int representation). |
| 10 | `GLOBAL_POSITION_INT.lon` | degrees | float | WGS84 longitude (same convention as `lat`). |
| 11 | `GLOBAL_POSITION_INT.alt` | mm | float | MSL altitude. Parser divides by 1000 to emit metres. |
| 12 | `GLOBAL_POSITION_INT.vx` | cm/s | float | NED north velocity. Parser divides by 100 to emit m/s. |
| 13 | `GLOBAL_POSITION_INT.vy` | cm/s | float | NED east velocity. |
| 14 | `GLOBAL_POSITION_INT.vz` | cm/s | float | NED down velocity. |
| 15 | `GLOBAL_POSITION_INT.hdg` | cdeg | float | Heading, 0–35999. Parser divides by 100 to emit degrees. |

### Tolerated extra columns

The following may be present but are not consumed:

| Column | Reason kept | Reason unused |
|--------|-------------|---------------|
| `SCALED_IMU2.xmag`, `.ymag`, `.zmag` | Symmetric with the accel/gyro triples in the Derkachi dump | The current ESKF does not integrate magnetometer; AZ-848 follow-up may add it |
| `GLOBAL_POSITION_INT.relative_alt` | Present in the MAVLink dump | The replay pipeline uses MSL `alt` only |

Additional columns beyond these are ignored without warning. Missing
required columns cause the load to raise
`ReplayInputAdapterError` before the replay loop starts (AC-5).

## Schema-level errors the parser catches

The parser raises `ReplayInputAdapterError` (CLI exit code 1) for any of:

- File does not exist or is not a regular file.
- File is empty (no header row).
- File has a header but no data rows.
- Any required column from the table above is missing from the header.
- The `Time` column at any row contains a non-numeric / NaN / Inf value.
- The `Time` column is non-monotonic (`Time[i] <= Time[i-1]`).
- Any required IMU or GPS column at any row contains a non-numeric / NaN /
  Inf value.

The error message includes the row number (1-based, where row 1 is the
header — so the first data row is row 2). Operators should treat the first
parse failure as authoritative and fix the source CSV; the parser does not
continue after the first invalid row.

## Operator workflow

```bash
gps-denied-replay \
  --video ./flight.mp4 \
  --imu ./data_imu.csv \
  --output ./estimator_output.jsonl \
  --camera-calibration ./calib.json \
  --config ./config.yaml \
  --mavlink-signing-key ./signing_key.bin
```

`--tlog` is accepted as a deprecated alias and will be removed by AZ-895.
When both `--imu` and `--tlog` are supplied, `--imu` wins and a deprecation
warning is printed to stderr.

## Deriving a new CSV from an ArduPilot tlog

The Derkachi fixture was produced with `pymavlink`'s `mavlogdump.py`. The
short version:

```bash
mavlogdump.py --format csv \
  --types SCALED_IMU2,GLOBAL_POSITION_INT \
  ./flight.tlog > ./raw_dump.csv
```

Then post-process to:

1. Rename / merge the per-message timestamp into a single `Time` column
   relative to the first row.
2. Drop pre-takeoff rows (the UAV must be airborne at row 0 — see the hard
   contract above).
3. Pre-divide `lat` / `lon` from the MAVLink `int * 1e7` representation
   into decimal degrees.
4. Re-sample to a uniform 10 Hz cadence if the tlog dump produced
   non-uniform spacing.

A reference post-processor script is **not** shipped — operators
historically write a one-off Python or Pandas pipeline per source aircraft.

## Cross-references

- AZ-894 — the CLI + adapter that consumes this format.
- AZ-895 — deletes the legacy `--tlog` argument once all callers migrate.
- AZ-897 — operator replay UI; links to this page and serves
  `example_data_imu.csv`.
- `_docs/02_document/contracts/replay/replay_protocol.md` — the broader
  replay orchestration contract.
- `_docs/00_problem/input_data/flight_derkachi/README.md` — fixture
  provenance and license caveats.
