Files
autopilot/_docs/02_tasks/todo/AZ-658_frame_ingest_decoder.md
T
Oleksandr Bezdieniezhnykh bc40ea7300 [AZ-626] Decompose complete: 47 tasks + docs + module layout
Greenfield Steps 1-6 baseline for the autopilot rewrite from legacy
Qt/C++ to a Rust workspace.

- Remove legacy Qt/C++ tree (ai_controller, drone_controller,
  misc/camera, python_scaffold, root Dockerfile, autopilot.pro,
  legacy main.py / requirements.txt).
- Add _docs/00_problem (problem, restrictions, acceptance criteria,
  security approach, input data + fixtures).
- Add _docs/01_solution/solution_draft01.
- Add _docs/02_document (architecture, system-flows, data_model,
  glossary, decision-rationale, deployment, 13 component descriptions,
  tests/ specs, FINAL_report, module-layout).
- Add _docs/02_tasks/todo with 47 task specs (AZ-640..AZ-686, one
  bootstrap + 46 component tasks) and _dependencies_table.md.
- Add .cursor/rules/artifact-srp.mdc (single-responsibility rule for
  canonical _docs artifacts).
- Track autodev state in _docs/_autodev_state.md (Step 6 completed,
  ready for Step 7 Implement).

Jira: bootstrap AZ-626; component epics AZ-627..AZ-639; tasks
AZ-640..AZ-686. Total complexity 173 points across 12 epics.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 11:02:01 +03:00

3.6 KiB

Frame Decoder (NVDEC + Software Fallback)

Task: AZ-658_frame_ingest_decoder Name: H.264/265 decoder (NVDEC primary, software fallback) + monotonic timestamps Description: Decode H.264/265 to raw frames using NVDEC on Jetson Orin Nano, with software fallback. Stamp each frame with a monotonic capture timestamp + sequence number at the earliest practical point in the pipeline. Complexity: 5 points Dependencies: AZ-640_initial_structure, AZ-657_frame_ingest_rtsp_session Component: frame_ingest Tracker: AZ-658 Epic: AZ-627

Problem

Every frame downstream needs a monotonic capture timestamp so movement_detector can detect telemetry skew. Decoding must use the hardware decoder (NVDEC on Jetson) where present and fall back to software otherwise, without changing the emitted Frame shape. Decode errors on a single frame must be dropped (counted), not abort the stream — cold-start latency is observable once but not an alert by itself.

Outcome

  • FrameDecoder::decode(packet) -> Result<Frame, DecodeError> emits a Frame { seq, capture_ts_monotonic, decode_ts_monotonic, pixels: Arc<Bytes>, width, height, pix_fmt, ai_locked }.
  • NVDEC code path is used when available; software fallback otherwise (selection is automatic and observable in health).
  • Single-frame errors are dropped and counted as decode_errors_total; the stream is never aborted on a single frame.
  • Cold-start latency (first-frame decode time) is surfaced as decode_ms_first_frame once per session open.
  • Health surface: decode_ms_p50, decode_ms_p99, decoder_backend ∈ {NVDEC, Software}, decode_errors_total.

Scope

Included

  • NVDEC binding (via Jetson Multimedia API or GStreamer nvv4l2decoder).
  • Software decoder fallback (FFmpeg libavcodec).
  • Monotonic timestamping at the earliest point in the decode pipeline.
  • Sequence-number generation (monotonic u64 per session).
  • Single-frame error handling.

Excluded

  • RTSP session lifecycle (task 18).
  • Multi-consumer publisher (task 20).

Acceptance Criteria

AC-1: Software-path decode of a sample stream Given a sample H.264 RTSP stream at 1080p / 30 fps and a host without NVDEC When the decoder runs for 10 s Then ≥285 frames are emitted; decoder_backend = "Software"; sequence numbers are strictly monotonic.

AC-2: NVDEC-path selection on Jetson Given the host has NVDEC available When the decoder is initialized Then decoder_backend = "NVDEC"; functional correctness is identical to software path.

AC-3: Single-frame decode error does not abort the stream Given the input contains one corrupted frame When the decoder runs Then that single frame is dropped, decode_errors_total increments by 1, and subsequent frames continue to be emitted.

AC-4: Monotonic timestamps Given a sequence of decoded frames When their capture_ts_monotonic is read Then values are strictly monotonically increasing.

Non-Functional Requirements

Performance

  • End-to-end RTSP-rx → publish ≤30 ms p99 on Jetson Orin Nano (per description.md §8); decoder portion of that budget ≤20 ms p99.

Reliability

  • Single-frame errors do not abort the stream.
  • Cold-start latency surfaced once; not an alert.

Runtime Completeness

  • Named capability: H.264/265 decode (NVDEC primary, software fallback) — production decode path required.
  • Production code that must exist: real NVDEC binding; real software fallback; real monotonic timestamping.
  • Unacceptable substitutes: software-only decode on Jetson is acceptable as fallback but the NVDEC code path MUST exist (otherwise the latency target cannot be met).