azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-22 15:51:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-959] replay_api: POST /replay (video,csv) + /static/example-csv

Browse Source 
Extend the AZ-701 replay_api POST /replay endpoint so AZ-897 (now
in ../ui repo) can drive the AZ-894 CSV-replay path. The endpoint
keeps full back-compat for tlog clients and adds:

- (video, tlog) OR (video, csv) multipart with strict XOR enforced
  at the API boundary (AC-2 / AC-3 → 400 multipart_missing_field)
- validate_csv_kind: rejects malformed CSV schema at boundary by
  scanning the header line for AZ-896 required tokens; messages
  point at csv_replay_format.md (AC-4)
- ReplayInputs DTO: tlog_path / csv_path are now Path | None with
  XOR re-enforced in __post_init__ for internal callers
- JobStorage reserves both input.tlog and input.csv paths; handler
  writes exactly one
- SubprocessReplayRunner.run dispatches --imu vs --tlog argv (AC-1)
- _maybe_render_report dispatches load_csv_ground_truth vs
  load_tlog_ground_truth; CsvGpsFix and TlogGpsFix have
  field-compatible shapes for the GroundTruthRow adapter (AC-6)
- GET /static/example-csv serves the AZ-896 reference CSV; honours
  REPLAY_API_EXAMPLE_CSV_PATH env, falls back to source-checkout
  layout, returns 503 with example_csv_unavailable when neither
  resolves to a readable file. No auth required (AC-5)

Tests: 27/27 unit tests green:
- 18 pre-existing tlog-path tests unchanged (AC-7)
- 9 new tests covering ACs 1-6 + validate_csv_kind isolation

Deferred (NOT silently fixed; reported to user as end-of-turn
notes for scope discipline):

- gps-denied-render-map only consumes binary tlog truth today, so
  CSV-path jobs return map_html_url=None. Extending render-map to
  dispatch on truth-file extension is AZ-700 follow-up territory.
- ReportContext.tlog_path field is now overloaded as the
  "ground-truth source path"; the rendered report still labels
  the line "Tlog: <csv_path>" which is cosmetically misleading
  for CSV runs. Field rename + label fix is AZ-699 follow-up.

Bookkeeping: AZ-959 spec moved todo/ → done/, dep-table preamble
fifth bump documents what landed + what's deferred, state.md
records batch 5 complete and what comes next.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-29 12:45:25 +03:00
parent 05fcacffa3
commit 1d18e25cf4
8 changed files with 476 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_tasks/done/AZ-959_replay_api_csv_path_endpoint.md
+72
View File
@@ -0,0 +1,72 @@
# replay_api: extend POST /replay to accept (video, csv) multipart for AZ-897 UI
**Task**: AZ-959_replay_api_csv_path_endpoint
**Name**: Extend `replay_api` `POST /replay` to accept (video, csv) multipart so the AZ-897 UI in `../ui` can drive the CSV-replay path
**Description**: AZ-897 was relocated to the `../ui` repo (the single Azaion suite React 19 front-end). The UI uploads `(video, CSV)` per AZ-894's CSV path, but the existing `replay_api` `POST /replay` endpoint only accepts `(tlog, video, calibration)` — it predates AZ-894. This ticket extends the endpoint to accept either input pair (XOR), validates the CSV against the AZ-896 schema, dispatches the subprocess with the right CLI flag (`--imu` vs `--tlog`), and adds a `GET /static/example-csv` endpoint serving the AZ-896 reference CSV. Existing tlog-path callers continue to work unchanged for the cycle-4 demo + transitional clients; AZ-908 (cycle-5+ backlog) eventually removes the tlog branch.
**Complexity**: 3 SP
**Dependencies**: AZ-701 (existing `replay_api` package, done), AZ-894 (CSV adapter that the CLI consumes, done), AZ-896 (example CSV + format spec, done)
**Blocks**: AZ-897 (UI in `../ui` — HARD BLOCKER; the UI cannot ship until this endpoint exists)
**Component**: replay_api (existing FastAPI app)
**Tracker**: AZ-959 (https://denyspopov.atlassian.net/browse/AZ-959)
**Parent Epic**: (none — cycle-4 replay-input redesign, replacement for the original AZ-897 backend slice after the UI relocation)
Jira AZ-959 is the authoritative spec; this file is the in-workspace mirror.
## Goal
Extend the AZ-701 `replay_api` `POST /replay` endpoint to accept the `(video, csv)` input pair that the AZ-894 CLI introduced. AZ-897 (relocated to `../ui`) calls this endpoint with `(video, csv, calibration)` multipart to drive the CSV-replay path; the UI does not upload pymavlink tlog files.
## Scope
1. **`src/gps_denied_onboard/replay_api/app.py`** (`post_replay` handler):
- Add `csv: Annotated[UploadFile | None, File()] = None` parameter alongside the existing `tlog`.
- Make `tlog` optional (currently required).
- Enforce XOR: exactly one of `tlog` or `csv` must be present; both or neither → 400 with clear error pointing at the XOR contract.
- Validate csv bytes via new `validate_csv_kind`.
- Persist via `job_storage.csv_path` when csv route taken.
- Pass through to `SubprocessReplayRunner.run` via the extended `ReplayInputs` shape.
2. **`src/gps_denied_onboard/replay_api/handlers.py`**:
- New `validate_csv_kind(probe_bytes: bytes) -> None`: checks the CSV header line starts with `timestamp(ms),Time,SCALED_IMU2.xacc,...` matching the AZ-896 csv_replay_format.md schema. Raises `UnsupportedFileKindError` with a message pointing to the docs path.
3. **`src/gps_denied_onboard/replay_api/interface.py`**:
- `ReplayInputs`: change `tlog_path: Path` to `tlog_path: Path | None` and add `csv_path: Path | None`. Invariant: exactly one is None (raised in `__post_init__` if violated).
4. **`src/gps_denied_onboard/replay_api/storage.py`**:
- Per-job storage: add `csv_path` property pointing to `{job_dir}/input/data_imu.csv` (mirrors `tlog_path`).
5. **`SubprocessReplayRunner.run` in `app.py`**:
- When `inputs.csv_path is not None`, shell out with `--imu` flag; when `inputs.tlog_path is not None`, shell out with `--tlog`.
- Ground-truth extraction (`_maybe_render_report`) currently calls `load_tlog_ground_truth(inputs.tlog_path)`. For the CSV path, ground truth must come from the CSV's `GLOBAL_POSITION_INT.*` columns. Default proposal: extend the ground-truth loader to dispatch on file extension via a thin helper next to `load_tlog_ground_truth` (avoids branching inside `_maybe_render_report`).
6. **New `GET /static/example-csv` endpoint** in `app.py`:
- Serve `_docs/02_document/contracts/replay/example_data_imu.csv` with `Content-Type: text/csv; charset=utf-8`.
- 200 if file exists, 503 if missing (build/packaging issue — file is in the source tree, so a missing file means a deploy-misconfiguration).
- This is what AZ-897 UI's "Download example CSV" links to.
7. **`tests/unit/replay_api/test_az701_replay_api.py`**:
- Update existing tests to cover the XOR validation (both rejected, neither rejected).
- Add tests: CSV happy path, malformed CSV schema rejected, example-csv endpoint serves correct content + content-type.
## Acceptance Criteria
- **AC-1**: `POST /replay` with `(csv, video, calibration)` multipart is accepted; subprocess invocation uses `--imu CSV_PATH`. Same response shape as the tlog-path call.
- **AC-2**: `POST /replay` with both `tlog` AND `csv` returns 400 + clear error pointing at the XOR contract.
- **AC-3**: `POST /replay` with neither `tlog` nor `csv` returns 400 + clear error.
- **AC-4**: `POST /replay` with malformed CSV (missing required column, e.g. no `Time` column) returns 400 + error referencing the AZ-896 format docs.
- **AC-5**: `GET /static/example-csv` returns 200 + `text/csv; charset=utf-8` content-type + exact file bytes from `_docs/02_document/contracts/replay/example_data_imu.csv`.
- **AC-6**: Ground-truth extraction works for the CSV path — accuracy report renders against the CSV's `GLOBAL_POSITION_INT.*` columns when `csv_path` was used.
- **AC-7**: All existing AZ-701 tlog-path tests in `test_az701_replay_api.py` still pass unchanged.
## Out of scope
- Calibration handling changes — keep current behaviour (operator uploads `calibration` field; falls back to `REPLAY_API_DEFAULT_CALIBRATION` env if not provided).
- Removing the tlog path — AZ-895 deprecated `--tlog` in the CLI but tlog API support stays for backwards compat for the cycle-4 demo + any existing programmatic clients. AZ-908 (cycle-5+ backlog) will remove tlog from both CLI and API.
- The UI itself — that's AZ-897 in `../ui`.
- Format docs page rendering / serving — keep markdown source in `_docs/02_document/contracts/replay/csv_replay_format.md`; UI links to the docs URL when published.
## Notes
- The `--imu` flag was added to the CLI by AZ-894; this ticket exposes that path through the HTTP API. No changes to the CLI itself.
- `validate_csv_kind` should be schema-aware (checks the header line matches the AZ-896 format), not just content-type sniffing. Bad schema must fail fast at the API boundary, not deep in `gps-denied-replay`.
- The `GET /static/example-csv` endpoint should not require auth (the example CSV is a public reference document, not a secret).