mirror of
https://github.com/azaion/detections.git
synced 2026-04-22 12:06:32 +00:00
[AZ-178] Implement streaming video detection endpoint
- Added `/detect/video` endpoint for true streaming video detection, allowing inference to start as upload bytes arrive. - Introduced `run_detect_video_stream` method in the inference module to handle video processing from a file-like object. - Updated media hashing to include a new function for computing hashes directly from files with minimal I/O. - Enhanced documentation to reflect changes in video processing and API behavior. Made-with: Cursor
This commit is contained in:
@@ -39,6 +39,7 @@ Core inference orchestrator — manages the AI engine lifecycle, preprocesses me
|
||||
| `__init__` | `(loader_client)` | public | Initializes state, calls `init_ai()` |
|
||||
| `run_detect_image` | `(bytes image_bytes, AIRecognitionConfig ai_config, str media_name, annotation_callback, status_callback=None)` | cpdef | Decodes image from bytes, runs tiling + inference + postprocessing |
|
||||
| `run_detect_video` | `(bytes video_bytes, AIRecognitionConfig ai_config, str media_name, str save_path, annotation_callback, status_callback=None)` | cpdef | Processes video from in-memory bytes via PyAV, concurrently writes to save_path |
|
||||
| `run_detect_video_stream` | `(object readable, AIRecognitionConfig ai_config, str media_name, annotation_callback, status_callback=None)` | cpdef | Processes video from a file-like readable (e.g. StreamingBuffer) via PyAV — true streaming, no bytes in RAM (AZ-178) |
|
||||
| `stop` | `()` | cpdef | Sets stop_signal to True |
|
||||
| `init_ai` | `()` | cdef | Engine initialization: tries TensorRT → falls back to ONNX → background TensorRT conversion |
|
||||
| `preprocess` | `(frames) -> ndarray` | via engine | OpenCV blobFromImage: resize, normalize to 0..1, swap RGB, stack batch |
|
||||
@@ -75,6 +76,15 @@ Both `run_detect_image` and `run_detect_video` accept raw bytes instead of file
|
||||
6. Annotation validity heuristics (time gap, detection count increase, spatial movement, confidence improvement)
|
||||
7. Valid frames get JPEG-encoded image attached
|
||||
|
||||
### Streaming Video Processing (`run_detect_video_stream` — AZ-178)
|
||||
|
||||
1. Accepts a file-like `readable` object (e.g. `StreamingBuffer`) instead of `bytes`
|
||||
2. Opens directly via `av.open(readable)` — PyAV calls `read()`/`seek()` on the object
|
||||
3. No writer thread needed — the caller (API layer) manages disk persistence via the same buffer
|
||||
4. Reuses `_process_video_pyav` for frame decoding, batch inference, and annotation delivery
|
||||
5. For faststart MP4/MKV/WebM: frames are decoded as bytes stream in (~500ms latency)
|
||||
6. For standard MP4 (moov at end): PyAV's `seek(0, 2)` blocks until the buffer signals EOF, then decoding starts
|
||||
|
||||
### Ground Sampling Distance (GSD)
|
||||
|
||||
`GSD = sensor_width * altitude / (focal_length * image_width)` — meters per pixel, used for physical size filtering of aerial detections.
|
||||
@@ -86,7 +96,7 @@ Both `run_detect_image` and `run_detect_video` accept raw bytes instead of file
|
||||
|
||||
## Consumers
|
||||
|
||||
- `main` — lazy-initializes Inference, calls `run_detect_image`/`run_detect_video`, reads `ai_availability_status` and `is_engine_ready`
|
||||
- `main` — lazy-initializes Inference, calls `run_detect_image`/`run_detect_video`/`run_detect_video_stream`, reads `ai_availability_status` and `is_engine_ready`
|
||||
|
||||
## Data Models
|
||||
|
||||
@@ -107,5 +117,6 @@ None.
|
||||
## Tests
|
||||
|
||||
- `tests/test_ai_config_from_dict.py` — tests `ai_config_from_dict` helper
|
||||
- `tests/test_az178_streaming_video.py` — tests `run_detect_video_stream` via the `/detect/video` endpoint and `StreamingBuffer`
|
||||
- `e2e/tests/test_video.py` — exercises `run_detect_video` via the full API
|
||||
- `e2e/tests/test_single_image.py` — exercises `run_detect_image` via the full API
|
||||
|
||||
Reference in New Issue
Block a user