autopilot/_docs/02_tasks/todo/AZ-659_frame_ingest_publisher.md

Multi-Consumer Frame Publisher + Back-Pressure Drops

Task: AZ-659_frame_ingest_publisher Name: Tokio broadcast publisher + per-consumer drop counters + zero-copy Arc<Bytes> Description: Publish Frames through a single multi-consumer channel using Arc<Bytes> for pixel data so consumers do not copy. Drop frames when downstream consumers fall behind beyond a configured queue depth; record per-consumer drop counters with reason tags. Complexity: 3 points Dependencies: AZ-640_initial_structure, AZ-657_frame_ingest_rtsp_session, AZ-658_frame_ingest_decoder Component: frame_ingest Tracker: AZ-659 Epic: AZ-627

Problem

Three downstream consumers (detection_client, movement_detector, telemetry_stream) all need the same frames at the same rate. A single-consumer queue would serialise the slowest; a per-consumer fan-out with cloned pixel buffers would multiply memory. The right structure is a Tokio broadcast channel (or equivalent) carrying Arc<Bytes> so pixels are shared by reference. Slow consumers drop their oldest frame, with the drop counted (and reason-tagged) — never silently coalesced.

Outcome

• FramePublisher::subscribe() -> FrameReceiver returns a per-consumer receiver.
• Frame carries Arc<Bytes> for pixels so consumers do not copy.
• When a consumer falls behind beyond channel_depth (configurable, default 4), the oldest frame is dropped for THAT consumer; per-consumer counters increment with reason tag ({detection_client_slow, movement_detector_slow, telemetry_slow}).
• Health surface: per-consumer drop counters, total publish count.

Scope

Included

• tokio::sync::broadcast (or equivalent) with Arc<Bytes> payload.
• Per-consumer drop counter (statically known three consumer ids; future-extensible).
• Channel-depth config.

Excluded

• RTSP session (task 18).
• Decoder (task 19).

Acceptance Criteria

AC-1: Three consumers receive every frame at nominal rate Given three subscribers consuming at 30 fps and source at 30 fps When the publisher runs for 10 s Then each consumer observes ~300 frames; per-consumer drop counters = 0.

AC-2: Slow consumer drops, fast consumers unaffected Given a slow consumer that yields every 200 ms while source is 30 fps and channel_depth = 4 When the publisher runs for 5 s Then the slow consumer's drop counter increments and fast consumers continue to receive every frame.

AC-3: Zero-copy under load Given a publisher emitting at 30 fps for 60 s with three subscribers When peak memory is sampled Then memory does not scale linearly with consumer count (i.e. Arc<Bytes> is correctly shared).

Non-Functional Requirements

Performance

• Publish-to-consumer p99 ≤5 ms (helps keep total RTSP-rx-to-publish under the 30 ms p99 budget).

Reliability

• Drops are counted with reason; never silent.
• No unbounded memory growth on slow consumer.

Runtime Completeness

• Named capability: lossy multi-consumer frame fan-out with Arc<Bytes>.
• Production code that must exist: real broadcast channel; real per-consumer drop accounting.
• Unacceptable substitutes: cloning pixel buffers per consumer is unacceptable (multiplies memory); blocking the publisher on a slow consumer is unacceptable (gates the whole pipeline).