mirror of
https://github.com/azaion/detections-semantic.git
synced 2026-04-23 06:36:37 +00:00
8e2ecf50fd
Made-with: Cursor
118 lines
4.5 KiB
Markdown
118 lines
4.5 KiB
Markdown
# E2E Test Environment
|
|
|
|
## Overview
|
|
|
|
**System under test**: Semantic Detection Service — a Cython + TensorRT module running within the existing FastAPI detections service on Jetson Orin Nano Super. Entry points: FastAPI REST API (image/video input), UART serial port (gimbal commands), Unix socket (VLM IPC).
|
|
|
|
**Consumer app purpose**: Standalone Python test runner that exercises the semantic detection pipeline through its public interfaces: submitting frames, injecting mock YOLO detections, capturing detection results, and monitoring gimbal command output. No access to internals.
|
|
|
|
## Docker Environment
|
|
|
|
### Services
|
|
|
|
| Service | Image / Build | Purpose | Ports |
|
|
|---------|--------------|---------|-------|
|
|
| semantic-detection | build: ./Dockerfile.test | Main semantic detection pipeline (Tier 1 + 2 + scan controller + gimbal driver + recorder) | 8080 (API) |
|
|
| mock-yolo | build: ./tests/mock_yolo/ | Provides deterministic YOLO detection output for test frames | 8081 (API) |
|
|
| mock-gimbal | build: ./tests/mock_gimbal/ | Simulates ViewPro A40 serial interface via TCP socket (replaces UART for testing) | 9090 (TCP) |
|
|
| vlm-stub | build: ./tests/vlm_stub/ | Deterministic VLM response stub via Unix socket | — (Unix socket) |
|
|
| e2e-consumer | build: ./tests/e2e/ | Black-box test runner (pytest) | — |
|
|
|
|
### Networks
|
|
|
|
| Network | Services | Purpose |
|
|
|---------|----------|---------|
|
|
| e2e-net | all | Isolated test network |
|
|
|
|
### Volumes
|
|
|
|
| Volume | Mounted to | Purpose |
|
|
|--------|-----------|---------|
|
|
| test-frames | semantic-detection:/data/frames, e2e-consumer:/data/frames | Shared test images (semantic01-04.png + synthetic frames) |
|
|
| test-output | semantic-detection:/data/output, e2e-consumer:/data/output | Detection logs, recorded frames, gimbal command log |
|
|
|
|
### docker-compose structure
|
|
|
|
```yaml
|
|
services:
|
|
semantic-detection:
|
|
build: .
|
|
environment:
|
|
- ENV=test
|
|
- GIMBAL_HOST=mock-gimbal
|
|
- GIMBAL_PORT=9090
|
|
- VLM_SOCKET=/tmp/vlm.sock
|
|
- YOLO_API=http://mock-yolo:8081
|
|
- RECORD_PATH=/data/output/frames
|
|
- LOG_PATH=/data/output/detections.jsonl
|
|
volumes:
|
|
- test-frames:/data/frames
|
|
- test-output:/data/output
|
|
depends_on:
|
|
- mock-yolo
|
|
- mock-gimbal
|
|
- vlm-stub
|
|
|
|
mock-yolo:
|
|
build: ./tests/mock_yolo
|
|
|
|
mock-gimbal:
|
|
build: ./tests/mock_gimbal
|
|
|
|
vlm-stub:
|
|
build: ./tests/vlm_stub
|
|
|
|
e2e-consumer:
|
|
build: ./tests/e2e
|
|
volumes:
|
|
- test-frames:/data/frames
|
|
- test-output:/data/output
|
|
depends_on:
|
|
- semantic-detection
|
|
```
|
|
|
|
## Consumer Application
|
|
|
|
**Tech stack**: Python 3.11, pytest, requests, struct (for gimbal protocol parsing)
|
|
**Entry point**: `pytest tests/e2e/ --junitxml=e2e-results/report.xml`
|
|
|
|
### Communication with system under test
|
|
|
|
| Interface | Protocol | Endpoint / Topic | Authentication |
|
|
|-----------|----------|-----------------|----------------|
|
|
| Frame submission | HTTP POST | http://semantic-detection:8080/api/v1/detect | None (internal network) |
|
|
| Detection results | HTTP GET | http://semantic-detection:8080/api/v1/results | None |
|
|
| Gimbal command log | File read | /data/output/gimbal_commands.log | None (shared volume) |
|
|
| Detection log | File read | /data/output/detections.jsonl | None (shared volume) |
|
|
| Recorded frames | File read | /data/output/frames/ | None (shared volume) |
|
|
|
|
### What the consumer does NOT have access to
|
|
|
|
- No direct access to TensorRT engine internals
|
|
- No access to YOLOE model weights or inference state
|
|
- No access to VLM process memory or internal prompts
|
|
- No direct UART/serial access (reads gimbal command log only)
|
|
- No access to scan controller state machine internals
|
|
|
|
## CI/CD Integration
|
|
|
|
**When to run**: On every PR to `dev` branch; nightly on `dev`
|
|
**Pipeline stage**: After unit tests pass, before merge approval
|
|
**Gate behavior**: Block merge on any FAIL
|
|
**Timeout**: 10 minutes total suite (most tests < 1s each; VLM tests up to 30s)
|
|
|
|
## Reporting
|
|
|
|
**Format**: JUnit XML + CSV summary
|
|
**Columns**: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message (if FAIL)
|
|
**Output path**: `./e2e-results/report.xml`, `./e2e-results/summary.csv`
|
|
|
|
## Hardware-in-the-Loop Test Track
|
|
|
|
Tests requiring actual Jetson Orin Nano Super hardware are marked with `[HIL]` in test IDs. These tests:
|
|
- Run on physical Jetson with real TensorRT engines
|
|
- Use real ViewPro A40 gimbal (or ViewPro simulator if available)
|
|
- Measure actual latency, memory, thermal, power
|
|
- Run separately from Docker-based E2E suite
|
|
- Triggered manually or on hardware CI runner (if available)
|