azaion/detections
1
0
Fork 0
mirror of https://github.com/azaion/detections.git synced 2026-04-22 14:16:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-180] Update module and component docs for Jetson/INT8 changes

Browse Source 
Made-with: Cursor
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-04-02 07:25:22 +03:00
parent 2ed9ce3336
commit 7a7f2a4cdd
5 changed files with 37 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_document/modules/inference.md
+7 -4
View File
@@ -41,7 +41,8 @@ Core inference orchestrator — manages the AI engine lifecycle, preprocesses me
| `run_detect_video` | `(bytes video_bytes, AIRecognitionConfig ai_config, str media_name, str save_path, annotation_callback, status_callback=None)` | cpdef | Processes video from in-memory bytes via PyAV, concurrently writes to save_path |
| `run_detect_video_stream` | `(object readable, AIRecognitionConfig ai_config, str media_name, annotation_callback, status_callback=None)` | cpdef | Processes video from a file-like readable (e.g. StreamingBuffer) via PyAV — true streaming, no bytes in RAM (AZ-178) |
| `stop` | `()` | cpdef | Sets stop_signal to True |
| `init_ai` | `()` | cdef | Engine initialization: tries TensorRT → falls back to ONNX → background TensorRT conversion |
| `init_ai` | `()` | cdef | Engine initialization: tries INT8 engine → FP16 engine → background TensorRT conversion (with optional INT8 calibration cache) |
| `_try_download_calib_cache` | `(str models_dir) -> str or None` | cdef | Downloads `azaion.int8_calib.cache` from Loader; writes to a temp file; returns path or None if unavailable |
| `preprocess` | `(frames) -> ndarray` | via engine | OpenCV blobFromImage: resize, normalize to 0..1, swap RGB, stack batch |
| `postprocess` | `(output, ai_config) -> list[list[Detection]]` | via engine | Parses engine output to Detection objects, applies confidence threshold and overlap removal |
@@ -50,9 +51,11 @@ Core inference orchestrator — manages the AI engine lifecycle, preprocesses me
### Engine Initialization (`init_ai`)
1. If `_converted_model_bytes` exists → load TensorRT from those bytes
2. If GPU available → try downloading pre-built TensorRT engine from loader
3. If download fails → download ONNX model, start background thread for ONNX→TensorRT conversion
4. If no GPU → load OnnxEngine from ONNX model bytes
2. If GPU available → try downloading pre-built INT8 engine first (`*.int8.engine`), then FP16 engine (`*.engine`) from loader
3. If no cached engine found → download ONNX source, attempt to download INT8 calibration cache (`azaion.int8_calib.cache`) from loader, spawn background thread for ONNX→TensorRT conversion (INT8 if cache downloaded, FP16 fallback)
4. Calibration cache download failure is non-fatal — log warning and proceed with FP16
5. Temporary calibration cache file is deleted after conversion completes
6. If no GPU → load OnnxEngine from ONNX model bytes
### Stream-Based Media Processing (AZ-173)