[AZ-178] Implement streaming video detection endpoint

- Added `/detect/video` endpoint for true streaming video detection, allowing inference to start as upload bytes arrive.
- Introduced `run_detect_video_stream` method in the inference module to handle video processing from a file-like object.
- Updated media hashing to include a new function for computing hashes directly from files with minimal I/O.
- Enhanced documentation to reflect changes in video processing and API behavior.

Made-with: Cursor
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-04-01 03:11:43 +03:00
parent e65d8da6a3
commit be4cab4fcb
42 changed files with 2983 additions and 29 deletions
+12 -1
View File
@@ -39,6 +39,7 @@ Core inference orchestrator — manages the AI engine lifecycle, preprocesses me
| `__init__` | `(loader_client)` | public | Initializes state, calls `init_ai()` |
| `run_detect_image` | `(bytes image_bytes, AIRecognitionConfig ai_config, str media_name, annotation_callback, status_callback=None)` | cpdef | Decodes image from bytes, runs tiling + inference + postprocessing |
| `run_detect_video` | `(bytes video_bytes, AIRecognitionConfig ai_config, str media_name, str save_path, annotation_callback, status_callback=None)` | cpdef | Processes video from in-memory bytes via PyAV, concurrently writes to save_path |
| `run_detect_video_stream` | `(object readable, AIRecognitionConfig ai_config, str media_name, annotation_callback, status_callback=None)` | cpdef | Processes video from a file-like readable (e.g. StreamingBuffer) via PyAV — true streaming, no bytes in RAM (AZ-178) |
| `stop` | `()` | cpdef | Sets stop_signal to True |
| `init_ai` | `()` | cdef | Engine initialization: tries TensorRT → falls back to ONNX → background TensorRT conversion |
| `preprocess` | `(frames) -> ndarray` | via engine | OpenCV blobFromImage: resize, normalize to 0..1, swap RGB, stack batch |
@@ -75,6 +76,15 @@ Both `run_detect_image` and `run_detect_video` accept raw bytes instead of file
6. Annotation validity heuristics (time gap, detection count increase, spatial movement, confidence improvement)
7. Valid frames get JPEG-encoded image attached
### Streaming Video Processing (`run_detect_video_stream` — AZ-178)
1. Accepts a file-like `readable` object (e.g. `StreamingBuffer`) instead of `bytes`
2. Opens directly via `av.open(readable)` — PyAV calls `read()`/`seek()` on the object
3. No writer thread needed — the caller (API layer) manages disk persistence via the same buffer
4. Reuses `_process_video_pyav` for frame decoding, batch inference, and annotation delivery
5. For faststart MP4/MKV/WebM: frames are decoded as bytes stream in (~500ms latency)
6. For standard MP4 (moov at end): PyAV's `seek(0, 2)` blocks until the buffer signals EOF, then decoding starts
### Ground Sampling Distance (GSD)
`GSD = sensor_width * altitude / (focal_length * image_width)` — meters per pixel, used for physical size filtering of aerial detections.
@@ -86,7 +96,7 @@ Both `run_detect_image` and `run_detect_video` accept raw bytes instead of file
## Consumers
- `main` — lazy-initializes Inference, calls `run_detect_image`/`run_detect_video`, reads `ai_availability_status` and `is_engine_ready`
- `main` — lazy-initializes Inference, calls `run_detect_image`/`run_detect_video`/`run_detect_video_stream`, reads `ai_availability_status` and `is_engine_ready`
## Data Models
@@ -107,5 +117,6 @@ None.
## Tests
- `tests/test_ai_config_from_dict.py` — tests `ai_config_from_dict` helper
- `tests/test_az178_streaming_video.py` — tests `run_detect_video_stream` via the `/detect/video` endpoint and `StreamingBuffer`
- `e2e/tests/test_video.py` — exercises `run_detect_video` via the full API
- `e2e/tests/test_single_image.py` — exercises `run_detect_image` via the full API