detections-semantic/_docs/02_tasks/01_initial_structure.md

Initial Project Structure

Task: 01_initial_structure Name: Initial Structure Description: Scaffold the project skeleton — folders, shared models, interfaces, stubs, Docker, CI/CD, test structure Complexity: 5 points Dependencies: None Component: Bootstrap Jira: TBD Epic: Bootstrap & Initial Structure

Project Folder Layout

detections-semantic/
├── src/
│   ├── __init__.py
│   ├── main.py
│   ├── config/
│   │   ├── __init__.py
│   │   ├── loader.py
│   │   └── schema.py
│   ├── types/
│   │   ├── __init__.py
│   │   ├── detection.py
│   │   ├── frame.py
│   │   ├── gimbal.py
│   │   ├── poi.py
│   │   ├── scenario.py
│   │   ├── spatial.py
│   │   └── vlm.py
│   ├── scan_controller/
│   │   ├── __init__.py
│   │   ├── tree.py
│   │   ├── behaviors/
│   │   │   └── __init__.py
│   │   └── subtrees/
│   │       └── __init__.py
│   ├── tier1_detector/
│   │   ├── __init__.py
│   │   └── detector.py
│   ├── tier2_spatial_analyzer/
│   │   ├── __init__.py
│   │   ├── mask_tracer.py
│   │   ├── cluster_tracer.py
│   │   └── roi_classifier.py
│   ├── vlm_client/
│   │   ├── __init__.py
│   │   └── client.py
│   ├── gimbal_driver/
│   │   ├── __init__.py
│   │   ├── driver.py
│   │   ├── protocol.py
│   │   └── pid.py
│   └── output_manager/
│       ├── __init__.py
│       └── manager.py
├── tests/
│   ├── unit/
│   │   ├── test_config.py
│   │   ├── test_types.py
│   │   ├── test_tier1.py
│   │   ├── test_tier2.py
│   │   ├── test_vlm.py
│   │   ├── test_gimbal.py
│   │   ├── test_output.py
│   │   └── test_scan_controller.py
│   ├── integration/
│   │   ├── conftest.py
│   │   └── test_data/
│   └── conftest.py
├── config/
│   ├── config.dev.yaml
│   ├── config.prod.yaml
│   └── config.test.yaml
├── docker/
│   ├── Dockerfile
│   ├── Dockerfile.dev
│   ├── mock-gimbal/
│   │   └── Dockerfile
│   └── .dockerignore
├── docker-compose.yml
├── docker-compose.test.yml
├── pyproject.toml
├── .env.example
├── .gitignore
└── azure-pipelines.yml

Layout Rationale

Python package structure following standard conventions. Components as subpackages under src/. Config and types are helpers used by all components. Tests mirror the source layout. Docker files separated into docker/ directory. Two compose files: one for dev (all services + mock gimbal), one for integration testing (black-box runner).

DTOs and Interfaces

Shared DTOs

DTO Name	Used By Components	Fields Summary
FrameContext	All	frame_id, timestamp, image, scan_level, quality_score, pan, tilt, zoom
Detection	Tier1→ScanController	centerX, centerY, width, height, classNum, label, confidence, mask
SemanticDetection	OutputManager	extends Detection with tier, freshness, tier2/3 results
POI	ScanController	poi_id, frame_id, trigger_class, scenario_name, investigation_type, confidence, bbox, priority, status
GimbalState	GimbalDriver→ScanController	pan, tilt, zoom, target_pan/tilt/zoom, last_heartbeat
CapabilityFlags	ScanController	vlm_available, gimbal_available, semantic_available
SpatialAnalysisResult	Tier2→ScanController	pattern_type, waypoints, trajectory, overall_direction, skeleton, cluster_bbox
Waypoint	Tier2→ScanController	x, y, dx, dy, label, confidence, freshness_tag, roi_thumbnail
VLMResponse	VLMClient→ScanController	text, confidence, latency_ms
SearchScenario	Config→ScanController	name, enabled, trigger/investigation params

Component Interfaces

Component	Interface	Methods	Exposed To
Tier1Detector	Tier1Detector	load, detect, is_ready	ScanController
Tier2SpatialAnalyzer	Tier2SpatialAnalyzer	trace_mask, trace_cluster, analyze_roi	ScanController
VLMClient	VLMClient	connect, disconnect, is_available, analyze, load_model, unload_model	ScanController
GimbalDriver	GimbalDriver	connect, disconnect, is_alive, set_angles, get_state, set_sweep_target, zoom_to_poi, follow_path, return_to_sweep	ScanController
OutputManager	OutputManager	init, log_detection, record_frame, log_health, log_gimbal_command, report_to_operator, get_storage_status	ScanController

CI/CD Pipeline

Stage	Purpose	Trigger
Lint	ruff + mypy	Every push
Unit Tests	pytest tests/unit/	Every push
Integration Tests	docker compose test	PR to dev
Security Scan	pip-audit + bandit	Every push
Build Docker	Build production image	Merge to dev

Pipeline Configuration Notes

Azure Pipelines (azure-pipelines.yml). Python 3.11 base. pip cache for dependencies. Unit tests run natively (no Docker needed). Integration tests spin up docker-compose.test.yml. Security scan includes pip-audit for known CVEs and bandit for SAST.

Environment Strategy

Environment	Purpose	Configuration Notes
Development	Local development on x86 workstation	ONNX Runtime fallback, mock gimbal TCP, console logging, local NVMe path
Production	Jetson Orin Nano Super on UAV	TensorRT FP16, real UART gimbal, JSON-lines to NVMe, tegrastats health

Environment Variables

Variable	Dev	Production	Description
CONFIG_PATH	config/config.dev.yaml	config/config.prod.yaml	Config file path
LOG_LEVEL	DEBUG	INFO	Logging verbosity
GIMBAL_MODE	mock_tcp	real_uart	Gimbal connection mode
VLM_SOCKET	/tmp/vlm.sock	/tmp/vlm.sock	VLM IPC socket path
OUTPUT_DIR	./output	/data/output	NVMe output directory

Database Migration Approach

Not applicable — no database. Data model uses runtime structs (in-memory) and append-only flat files (NVMe). Config changes handled by YAML version field with backward-compatible defaults.

Test Structure

tests/
├── unit/
│   ├── test_config.py          → Config loader + validation
│   ├── test_types.py           → Dataclass creation and field types
│   ├── test_tier1.py           → Detection output format, error handling
│   ├── test_tier2.py           → Mask tracing, cluster tracing, ROI classify
│   ├── test_vlm.py             → IPC protocol, timeout, availability
│   ├── test_gimbal.py          → Command format, PID, retry logic
│   ├── test_output.py          → Log format, storage status, write errors
│   └── test_scan_controller.py → POI queue, BT behavior, scenario matching
├── integration/
│   ├── conftest.py             → Docker compose fixtures
│   ├── test_data/              → Test images, masks, configs
│   ├── test_pipeline.py        → End-to-end L1→L2→L1 cycle
│   └── test_degradation.py     → Health degradation scenarios
└── conftest.py                 → Shared fixtures, mock factories

Test Configuration Notes

pytest with pytest-timeout for latency tests. Mock factories for all component interfaces (returns fixture data). Integration tests use docker-compose.test.yml with mock gimbal and mock VLM services. Test data stored in tests/integration/test_data/ (not committed to git if > 10MB — download script instead).

Implementation Order

Order	Component	Reason
1	Config helper	All components depend on config
2	Types helper	All components depend on shared types
3	Tier1Detector	Core inference, blocks ScanController
3	Tier2SpatialAnalyzer	Parallel with Tier1
3	VLMClient	Parallel with Tier1
3	GimbalDriver	Parallel with Tier1
3	OutputManager	Parallel with Tier1
4	ScanController	Integrates all components (last)
5	Integration Tests	Validates complete system

Acceptance Criteria

AC-1: Project scaffolded Given the structure plan above When the implementer executes this task Then all folders, stubs, and configuration files exist

AC-2: Tests runnable Given the scaffolded project When pytest tests/unit/ is executed Then all stub tests pass

AC-3: Docker dev environment boots Given docker-compose.yml When docker compose up -d is executed Then all services start and health endpoint returns 200

AC-4: CI pipeline configured Given azure-pipelines.yml When pipeline is triggered Then lint, test, and build stages complete successfully