Made-with: Cursor
4.5 KiB
E2E Test Environment
Overview
System under test: Semantic Detection Service — a Cython + TensorRT module running within the existing FastAPI detections service on Jetson Orin Nano Super. Entry points: FastAPI REST API (image/video input), UART serial port (gimbal commands), Unix socket (VLM IPC).
Consumer app purpose: Standalone Python test runner that exercises the semantic detection pipeline through its public interfaces: submitting frames, injecting mock YOLO detections, capturing detection results, and monitoring gimbal command output. No access to internals.
Docker Environment
Services
| Service | Image / Build | Purpose | Ports |
|---|---|---|---|
| semantic-detection | build: ./Dockerfile.test | Main semantic detection pipeline (Tier 1 + 2 + scan controller + gimbal driver + recorder) | 8080 (API) |
| mock-yolo | build: ./tests/mock_yolo/ | Provides deterministic YOLO detection output for test frames | 8081 (API) |
| mock-gimbal | build: ./tests/mock_gimbal/ | Simulates ViewPro A40 serial interface via TCP socket (replaces UART for testing) | 9090 (TCP) |
| vlm-stub | build: ./tests/vlm_stub/ | Deterministic VLM response stub via Unix socket | — (Unix socket) |
| e2e-consumer | build: ./tests/e2e/ | Black-box test runner (pytest) | — |
Networks
| Network | Services | Purpose |
|---|---|---|
| e2e-net | all | Isolated test network |
Volumes
| Volume | Mounted to | Purpose |
|---|---|---|
| test-frames | semantic-detection:/data/frames, e2e-consumer:/data/frames | Shared test images (semantic01-04.png + synthetic frames) |
| test-output | semantic-detection:/data/output, e2e-consumer:/data/output | Detection logs, recorded frames, gimbal command log |
docker-compose structure
services:
semantic-detection:
build: .
environment:
- ENV=test
- GIMBAL_HOST=mock-gimbal
- GIMBAL_PORT=9090
- VLM_SOCKET=/tmp/vlm.sock
- YOLO_API=http://mock-yolo:8081
- RECORD_PATH=/data/output/frames
- LOG_PATH=/data/output/detections.jsonl
volumes:
- test-frames:/data/frames
- test-output:/data/output
depends_on:
- mock-yolo
- mock-gimbal
- vlm-stub
mock-yolo:
build: ./tests/mock_yolo
mock-gimbal:
build: ./tests/mock_gimbal
vlm-stub:
build: ./tests/vlm_stub
e2e-consumer:
build: ./tests/e2e
volumes:
- test-frames:/data/frames
- test-output:/data/output
depends_on:
- semantic-detection
Consumer Application
Tech stack: Python 3.11, pytest, requests, struct (for gimbal protocol parsing)
Entry point: pytest tests/e2e/ --junitxml=e2e-results/report.xml
Communication with system under test
| Interface | Protocol | Endpoint / Topic | Authentication |
|---|---|---|---|
| Frame submission | HTTP POST | http://semantic-detection:8080/api/v1/detect | None (internal network) |
| Detection results | HTTP GET | http://semantic-detection:8080/api/v1/results | None |
| Gimbal command log | File read | /data/output/gimbal_commands.log | None (shared volume) |
| Detection log | File read | /data/output/detections.jsonl | None (shared volume) |
| Recorded frames | File read | /data/output/frames/ | None (shared volume) |
What the consumer does NOT have access to
- No direct access to TensorRT engine internals
- No access to YOLOE model weights or inference state
- No access to VLM process memory or internal prompts
- No direct UART/serial access (reads gimbal command log only)
- No access to scan controller state machine internals
CI/CD Integration
When to run: On every PR to dev branch; nightly on dev
Pipeline stage: After unit tests pass, before merge approval
Gate behavior: Block merge on any FAIL
Timeout: 10 minutes total suite (most tests < 1s each; VLM tests up to 30s)
Reporting
Format: JUnit XML + CSV summary
Columns: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message (if FAIL)
Output path: ./e2e-results/report.xml, ./e2e-results/summary.csv
Hardware-in-the-Loop Test Track
Tests requiring actual Jetson Orin Nano Super hardware are marked with [HIL] in test IDs. These tests:
- Run on physical Jetson with real TensorRT engines
- Use real ViewPro A40 gimbal (or ViewPro simulator if available)
- Measure actual latency, memory, thermal, power
- Run separately from Docker-based E2E suite
- Triggered manually or on hardware CI runner (if available)