mirror of
https://github.com/azaion/detections.git
synced 2026-04-22 09:36:32 +00:00
[AZ-180] Add Jetson Orin Nano support with INT8 TensorRT engine
- Dockerfile.jetson: JetPack 6.x L4T base image (aarch64), TensorRT and PyCUDA from apt - requirements-jetson.txt: derived from requirements.txt, no pip tensorrt/pycuda - docker-compose.jetson.yml: runtime: nvidia for NVIDIA Container Runtime - tensorrt_engine.pyx: convert_from_source accepts optional calib_cache_path; INT8 used when cache present, FP16 fallback; get_engine_filename encodes precision suffix to avoid engine cache confusion - inference.pyx: init_ai tries INT8 engine then FP16 on lookup; downloads calibration cache before conversion thread; passes cache path through to convert_from_source - constants_inf: add INT8_CALIB_CACHE_FILE constant - Unit tests for AC-3 (INT8 flag set when cache provided) and AC-4 (FP16 when no cache) Made-with: Cursor
This commit is contained in:
@@ -115,6 +115,30 @@ networks:
|
||||
|
||||
Already exists: `e2e/docker-compose.test.yml`. No changes needed — supports both `cpu` and `gpu` profiles with mock services and test runner.
|
||||
|
||||
### detections-jetson (Dockerfile.jetson)
|
||||
|
||||
| Aspect | Specification |
|
||||
|--------|--------------|
|
||||
| Base image | `nvcr.io/nvidia/l4t-base:r36.3.0` (JetPack 6.x, aarch64) |
|
||||
| TensorRT | Pre-installed via JetPack — `python3-libnvinfer` apt package (NOT pip) |
|
||||
| PyCUDA | Pre-installed via JetPack — `python3-pycuda` apt package (NOT pip) |
|
||||
| Build stages | Single stage (Cython compile requires gcc) |
|
||||
| Non-root user | `adduser --disabled-password --gecos '' appuser` + `USER appuser` |
|
||||
| Exposed ports | 8080 |
|
||||
| Entrypoint | `uvicorn main:app --host 0.0.0.0 --port 8080` |
|
||||
| Runtime | Requires NVIDIA Container Runtime (`runtime: nvidia` in docker-compose) |
|
||||
|
||||
**Jetson-specific behaviour**:
|
||||
- `requirements-jetson.txt` derives from `requirements.txt` — `tensorrt` and `pycuda` are excluded from pip and provided by JetPack
|
||||
- Engine filename auto-encodes CC+SM (e.g. `azaion.cc_8.7_sm_16.engine` for Orin Nano), ensuring the Jetson engine is distinct from any x86-cached engine
|
||||
- INT8 is used when `azaion.int8_calib.cache` is available on the Loader service; precision suffix appended to engine filename (`*.int8.engine`); FP16 fallback when cache is absent
|
||||
- `docker-compose.jetson.yml` uses `runtime: nvidia` for the NVIDIA Container Runtime
|
||||
|
||||
**Compose usage on Jetson**:
|
||||
```bash
|
||||
docker compose -f docker-compose.jetson.yml up
|
||||
```
|
||||
|
||||
## Image Tagging Strategy
|
||||
|
||||
| Context | Tag Format | Example |
|
||||
|
||||
Reference in New Issue
Block a user