azaion/detections
1
0
Fork 0
mirror of https://github.com/azaion/detections.git synced 2026-04-22 12:46:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-172] Update documentation for distributed architecture, add Update Docs step to workflow

Browse Source 
- Update module docs: main, inference, ai_config, loader_http_client
- Add new module doc: media_hash
- Update component docs: inference_pipeline, api
- Update system-flows (F2, F3) and data_parameters
- Add Task Mode to document skill for incremental doc updates
- Insert Step 11 (Update Docs) in existing-code flow, renumber 11-13 to 12-14

Made-with: Cursor
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-03-31 17:25:58 +03:00
parent e29606c313
commit 1fe9425aa8
12 changed files with 447 additions and 245 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_document/modules/main.md
+60 -84
View File
@@ -2,7 +2,7 @@
## Purpose
FastAPI application entry point — exposes HTTP API for object detection on images and video media, health checks, and Server-Sent Events (SSE) streaming of detection results.
FastAPI application entry point — exposes HTTP API for object detection on images and video media, health checks, and Server-Sent Events (SSE) streaming of detection results. Manages media lifecycle (content hashing, persistent storage, media record creation, status updates) and DB-driven AI configuration.
## Public Interface
@@ -11,8 +11,8 @@ FastAPI application entry point — exposes HTTP API for object detection on ima
| Method | Path | Description |
|--------|------|-------------|
| GET | `/health` | Returns AI engine availability status |
| POST | `/detect` | Single image detection (multipart file upload) |
| POST | `/detect/{media_id}` | Start async detection on media from loader service |
| POST | `/detect` | Image/video detection with media lifecycle management |
| POST | `/detect/{media_id}` | Start async detection on media resolved from Annotations service |
| GET | `/detect/stream` | SSE stream of detection events |
### DTOs (Pydantic Models)
@@ -21,8 +21,8 @@ FastAPI application entry point — exposes HTTP API for object detection on ima
|-------|--------|-------------|
| `DetectionDto` | centerX, centerY, width, height, classNum, label, confidence | Single detection result |
| `DetectionEvent` | annotations (list[DetectionDto]), mediaId, mediaStatus, mediaPercent | SSE event payload |
| `HealthResponse` | status, aiAvailability, errorMessage | Health check response |
| `AIConfigDto` | frame_period_recognition, frame_recognition_seconds, probability_threshold, tracking_*, model_batch_size, big_image_tile_overlap_percent, altitude, focal_length, sensor_width, paths | Configuration input for media detection |
| `HealthResponse` | status, aiAvailability, engineType, errorMessage | Health check response |
| `AIConfigDto` | frame_period_recognition, frame_recognition_seconds, probability_threshold, tracking_*, model_batch_size, big_image_tile_overlap_percent, altitude, focal_length, sensor_width | Configuration input (no `paths` field — removed in AZ-174) |
### Class: TokenManager
@@ -30,31 +30,55 @@ FastAPI application entry point — exposes HTTP API for object detection on ima
|--------|-----------|-------------|
| `__init__` | `(str access_token, str refresh_token)` | Stores tokens |
| `get_valid_token` | `() -> str` | Returns access_token; auto-refreshes if expiring within 60s |
| `decode_user_id` | `(str token) -> Optional[str]` | Static. Extracts user ID from JWT claims (sub, userId, user_id, nameid, or SAML nameidentifier) |
### Helper Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `_merged_annotation_settings_payload` | `(raw: object) -> dict` | Merges nested AI settings from Annotations service response (handles `aiRecognitionSettings`, `cameraSettings` sub-objects and PascalCase/camelCase/snake_case aliases) |
| `_resolve_media_for_detect` | `(media_id, token_mgr, override) -> tuple[dict, str]` | Fetches user AI settings + media path from Annotations service, merges with client overrides |
| `_detect_upload_kind` | `(filename, data) -> tuple[str, str]` | Determines if upload is image or video by extension, falls back to content probing (cv2/PyAV) |
| `_post_media_record` | `(payload, bearer) -> bool` | Creates media record via `POST /api/media` on Annotations service |
| `_put_media_status` | `(media_id, status, bearer) -> bool` | Updates media status via `PUT /api/media/{media_id}/status` on Annotations service |
| `compute_media_content_hash` | (imported from `media_hash`) | XxHash64 content hash with sampling |
## Internal Logic
### `/health`
Returns `HealthResponse` with `status="healthy"` always. `aiAvailability` reflects the engine's `AIAvailabilityStatus`. On exception, returns `aiAvailability="None"`.
Returns `HealthResponse` with `status="healthy"` always. `aiAvailability` reflects the engine's `AIAvailabilityStatus`. `engineType` reports the active engine name. On exception, returns `aiAvailability="None"`.
### `/detect` (single image)
### `/detect` (unified upload — AZ-173, AZ-175)
1. Reads uploaded file bytes
2. Parses optional JSON config
3. Runs `inference.detect_single_image` in ThreadPoolExecutor (max 2 workers)
4. Returns list of DetectionDto
1. Reads uploaded file bytes, rejects empty
2. Detects kind (image/video) via `_detect_upload_kind` (extension → content probe)
3. Validates image data with `cv2.imdecode` if kind is image
4. Parses optional JSON config
5. Extracts auth tokens; if authenticated:
a. Computes XxHash64 content hash
b. Persists file to `VIDEOS_DIR` or `IMAGES_DIR`
c. Creates media record via `POST /api/media`
d. Sets status to `AI_PROCESSING` via `PUT /api/media/{id}/status`
6. Runs `run_detect_image` or `run_detect_video` in ThreadPoolExecutor
7. On success: sets status to `AI_PROCESSED`
8. On failure: sets status to `ERROR`
9. Returns list of `DetectionDto`
Error mapping: RuntimeError("not available") → 503, RuntimeError → 422, ValueError → 400.
### `/detect/{media_id}` (async — AZ-174)
### `/detect/{media_id}` (async media)
1. Checks for duplicate active detection (409 if already running)
2. Extracts auth tokens from Authorization header and x-refresh-token header
3. Creates `asyncio.Task` for background detection
4. Detection runs `inference.run_detect` in ThreadPoolExecutor
5. Callbacks push `DetectionEvent` to all SSE queues
6. If auth token present, also POSTs annotations to the Annotations service
7. Returns immediately: `{"status": "started", "mediaId": media_id}`
1. Checks for duplicate active detection (409)
2. Extracts auth tokens
3. Resolves media via `_resolve_media_for_detect`:
a. Fetches user AI settings from `GET /api/users/{user_id}/ai-settings`
b. Merges with client overrides
c. Fetches media path from `GET /api/media/{media_id}`
4. Reads file bytes from resolved path
5. Creates `asyncio.Task` for background detection
6. Calls `run_detect_video` or `run_detect_image` depending on file extension
7. Callbacks push `DetectionEvent` to SSE queues and POST annotations to Annotations service
8. Updates media status via `PUT /api/media/{id}/status`
9. Returns immediately: `{"status": "started", "mediaId": media_id}`
### `/detect/stream` (SSE)
@@ -64,73 +88,18 @@ Error mapping: RuntimeError("not available") → 503, RuntimeError → 422, Valu
### Token Management
- Decodes JWT exp claim from base64 payload (no signature verification)
- `_decode_exp`: Decodes JWT exp claim from base64 payload (no signature verification)
- Auto-refreshes via POST to `{ANNOTATIONS_URL}/auth/refresh` when within 60s of expiry
- `decode_user_id`: Extracts user identity from multiple possible JWT claim keys
### Annotations Service Integration
Detections posts results to the Annotations service (`POST {ANNOTATIONS_URL}/annotations`) server-to-server during async media detection (F3). This only happens when an auth token is present in the original request.
**Endpoint:** `POST {ANNOTATIONS_URL}/annotations`
**Headers:**
| Header | Value |
|--------|-------|
| Authorization | `Bearer {accessToken}` (forwarded from the original client request) |
| Content-Type | `application/json` |
**Request body — payload sent by Detections:**
| Field | Type | Description |
|-------|------|-------------|
| mediaId | string | ID of the media being processed |
| source | int | `0` (AnnotationSource.AI) |
| videoTime | string | Video playback position formatted from ms as `"HH:MM:SS"` — mapped to `Annotations.Time` |
| detections | list | Detection results for this batch (see below) |
| image | string (base64) | Optional — base64-encoded frame image bytes |
`userId` is not included in the payload. The Annotations service resolves the user identity from the Bearer JWT.
**Detection object (as sent by Detections):**
| Field | Type | Description |
|-------|------|-------------|
| centerX | float | X center, normalized 0.0–1.0 |
| centerY | float | Y center, normalized 0.0–1.0 |
| width | float | Width, normalized 0.0–1.0 |
| height | float | Height, normalized 0.0–1.0 |
| classNum | int | Detection class number |
| label | string | Human-readable class name |
| confidence | float | Model confidence 0.0–1.0 |
The Annotations API contract (`CreateAnnotationRequest`) also accepts `description` (string), `affiliation` (AffiliationEnum), and `combatReadiness` (CombatReadinessEnum) on each Detection, but the Detections service does not populate these — the Annotations service uses defaults.
**Responses from Annotations service:**
| Status | Condition |
|--------|-----------|
| 201 | Annotation created |
| 400 | Neither image nor mediaId provided |
| 404 | MediaId not found in Annotations DB |
**Failure handling:** POST failures are silently caught — detection processing continues regardless. Annotations that fail to post are not retried.
**Downstream pipeline (Annotations service side):**
1. Saves annotation to local PostgreSQL (image → XxHash64 ID, label file in YOLO format)
2. Publishes SSE event to UI via `GET /annotations/events`
3. Enqueues annotation ID to `annotations_queue_records` buffer table (unless SilentDetection mode is enabled in system settings)
4. `FailsafeProducer` (BackgroundService) drains the buffer to RabbitMQ Stream (`azaion-annotations`) using MessagePack + Gzip
**Token refresh for long-running video:**
For video detection that may outlast the JWT lifetime, the `TokenManager` auto-refreshes via `POST {ANNOTATIONS_URL}/auth/refresh` when the token is within 60s of expiry. The refresh token is provided by the client in the `X-Refresh-Token` request header.
Detections posts results to `POST {ANNOTATIONS_URL}/annotations` during async media detection (F3). Media lifecycle (create record, update status) uses `POST /api/media` and `PUT /api/media/{media_id}/status`.
## Dependencies
- **External**: `asyncio`, `base64`, `json`, `os`, `time`, `concurrent.futures`, `typing`, `requests`, `fastapi`, `pydantic`
- **Internal**: `inference` (lazy import), `constants_inf` (label lookup), `loader_http_client` (client instantiation)
- **External**: `asyncio`, `base64`, `io`, `json`, `os`, `tempfile`, `time`, `concurrent.futures`, `pathlib`, `typing`, `av`, `cv2`, `numpy`, `requests`, `fastapi`, `pydantic`
- **Internal**: `inference` (lazy import), `constants_inf` (label lookup), `loader_http_client` (client instantiation), `media_hash` (content hashing)
## Consumers
@@ -147,22 +116,29 @@ None (entry point).
|---------|---------|-------------|
| `LOADER_URL` | `http://loader:8080` | Loader service base URL |
| `ANNOTATIONS_URL` | `http://annotations:8080` | Annotations service base URL |
| `VIDEOS_DIR` | `{cwd}/data/videos` | Persistent video storage directory |
| `IMAGES_DIR` | `{cwd}/data/images` | Persistent image storage directory |
## External Integrations
| Service | Protocol | Purpose |
|---------|----------|---------|
| Loader | HTTP (via LoaderHttpClient) | Model loading |
| Annotations | HTTP POST | Auth refresh (`/auth/refresh`), annotation posting (`/annotations`) |
| Annotations | HTTP GET | User AI settings (`/api/users/{id}/ai-settings`), media path resolution (`/api/media/{id}`) |
| Annotations | HTTP POST | Annotation posting (`/annotations`), media record creation (`/api/media`) |
| Annotations | HTTP PUT | Media status updates (`/api/media/{id}/status`) |
| Annotations | HTTP POST | Auth refresh (`/auth/refresh`) |
## Security
- Bearer token from request headers, refreshed via Annotations service
- JWT exp decoded (base64, no signature verification) — token validation is not performed locally
- Image data validated via `cv2.imdecode` before processing
- No CORS configuration
- No rate limiting
- No input validation on media_id path parameter beyond string type
## Tests
None found.
- `tests/test_az174_db_driven_config.py` — `decode_user_id`, `_merged_annotation_settings_payload`, `_resolve_media_for_detect`
- `tests/test_az175_api_calls.py` — `_post_media_record`, `_put_media_status`
- `e2e/tests/test_*.py` — full API e2e tests (health, single image, video, async, SSE, negative, security, performance, resilience)