mirror of
https://github.com/azaion/detections-semantic.git
synced 2026-04-22 19:26:38 +00:00
Initial commit
Made-with: Cursor
This commit is contained in:
@@ -0,0 +1,319 @@
|
||||
# ScanController
|
||||
|
||||
## 1. High-Level Overview
|
||||
|
||||
**Purpose**: Central orchestrator that drives the scan behavior tree — ticks the tree each cycle, which coordinates frame capture, inference dispatch, POI management, gimbal control, health monitoring, and L1/L2 scan transitions. Search behavior is data-driven via configurable **Search Scenarios**.
|
||||
|
||||
**Architectural Pattern**: Behavior Tree (py_trees) for high-level scan orchestration. Leaf nodes contain simple procedural logic that calls into other components. Search scenarios loaded from YAML config define what to look for and how to investigate.
|
||||
|
||||
**Upstream dependencies**: Tier1Detector, Tier2SpatialAnalyzer, VLMClient (optional), GimbalDriver, OutputManager, Config helper, Types helper
|
||||
|
||||
**Downstream consumers**: None — this is the top-level orchestrator. Exposes health API endpoint.
|
||||
|
||||
## 2. Search Scenarios (data-driven)
|
||||
|
||||
A **SearchScenario** defines what triggers a Level 2 investigation and how to investigate it. Multiple scenarios can be active simultaneously. Defined in YAML config:
|
||||
|
||||
```yaml
|
||||
search_scenarios:
|
||||
- name: winter_concealment
|
||||
enabled: true
|
||||
trigger:
|
||||
classes: [footpath_winter, branch_pile, dark_entrance]
|
||||
min_confidence: 0.5
|
||||
investigation:
|
||||
type: path_follow
|
||||
follow_class: footpath_winter
|
||||
target_classes: [concealed_position, branch_pile, dark_entrance, trash]
|
||||
use_vlm: true
|
||||
priority_boost: 1.0
|
||||
|
||||
- name: autumn_concealment
|
||||
enabled: true
|
||||
trigger:
|
||||
classes: [footpath_autumn, branch_pile, dark_entrance]
|
||||
min_confidence: 0.5
|
||||
investigation:
|
||||
type: path_follow
|
||||
follow_class: footpath_autumn
|
||||
target_classes: [concealed_position, branch_pile, dark_entrance]
|
||||
use_vlm: true
|
||||
priority_boost: 1.0
|
||||
|
||||
- name: building_area_search
|
||||
enabled: true
|
||||
trigger:
|
||||
classes: [building_block, road_with_traces, house_with_vehicle]
|
||||
min_confidence: 0.6
|
||||
investigation:
|
||||
type: area_sweep
|
||||
target_classes: [vehicle, military_vehicle, traces, dark_entrance]
|
||||
use_vlm: false
|
||||
priority_boost: 0.8
|
||||
|
||||
- name: aa_defense_network
|
||||
enabled: false
|
||||
trigger:
|
||||
classes: [radar_dish, aa_launcher, military_truck]
|
||||
min_confidence: 0.4
|
||||
min_cluster_size: 2
|
||||
investigation:
|
||||
type: cluster_follow
|
||||
target_classes: [radar_dish, aa_launcher, military_truck, command_vehicle]
|
||||
cluster_radius_px: 300
|
||||
use_vlm: true
|
||||
priority_boost: 1.5
|
||||
```
|
||||
|
||||
### Investigation Types
|
||||
|
||||
| Type | Description | Subtree Used | When |
|
||||
|------|-------------|-------------|------|
|
||||
| `path_follow` | Skeletonize footpath → PID follow → endpoint analysis | PathFollowSubtree | Footpath-based scenarios |
|
||||
| `area_sweep` | Slow pan across POI area at high zoom, Tier 1 continuously | AreaSweepSubtree | Building blocks, tree rows, clearings |
|
||||
| `zoom_classify` | Zoom to POI → run Tier 1 at high zoom → report | ZoomClassifySubtree | Single long-range targets |
|
||||
| `cluster_follow` | Cluster nearby detections → visit each in order → classify per point | ClusterFollowSubtree | AA defense networks, radar clusters, vehicle groups |
|
||||
|
||||
Adding a new investigation type requires a new BT subtree. Adding a new scenario that uses existing investigation types requires only YAML config changes.
|
||||
|
||||
## 3. Behavior Tree Structure
|
||||
|
||||
```
|
||||
Root (Selector — try highest priority first)
|
||||
│
|
||||
├── [1] HealthGuard (Decorator: checks capability flags)
|
||||
│ └── FallbackBehavior
|
||||
│ ├── If semantic_available=false → run existing YOLO only
|
||||
│ └── If gimbal_available=false → fixed camera, Tier 1 detect only
|
||||
│
|
||||
├── [2] L2Investigation (Sequence — runs if POI queue non-empty)
|
||||
│ ├── CheckPOIQueue (Condition: queue non-empty?)
|
||||
│ ├── PickHighestPOI (Action: pop from priority queue)
|
||||
│ ├── ZoomToPOI (Action: gimbal zoom + wait)
|
||||
│ ├── L2DetectLoop (Repeat until timeout)
|
||||
│ │ ├── CaptureFrame (Action)
|
||||
│ │ ├── RunTier1 (Action: YOLOE on zoomed frame)
|
||||
│ │ ├── RecordFrame (Action: L2 rate)
|
||||
│ │ └── InvestigateByScenario (Selector — picks subtree based on POI's scenario)
|
||||
│ │ ├── PathFollowSubtree (Sequence — if scenario.type == path_follow)
|
||||
│ │ │ ├── TraceMask (Action: Tier2.trace_mask → SpatialAnalysisResult)
|
||||
│ │ │ ├── PIDFollow (Action: gimbal PID along trajectory)
|
||||
│ │ │ └── WaypointAnalysis (Selector — for each waypoint)
|
||||
│ │ │ ├── HighConfidence (Condition: heuristic > threshold)
|
||||
│ │ │ │ └── LogDetection (Action: tier=2)
|
||||
│ │ │ └── AmbiguousWithVLM (Sequence — if scenario.use_vlm)
|
||||
│ │ │ ├── CheckVLMAvailable (Condition)
|
||||
│ │ │ ├── RunVLM (Action: VLMClient.analyze)
|
||||
│ │ │ └── LogDetection (Action: tier=3)
|
||||
│ │ ├── ClusterFollowSubtree (Sequence — if scenario.type == cluster_follow)
|
||||
│ │ │ ├── TraceCluster (Action: Tier2.trace_cluster → SpatialAnalysisResult)
|
||||
│ │ │ ├── VisitLoop (Repeat over waypoints)
|
||||
│ │ │ │ ├── MoveToWaypoint (Action: gimbal to next waypoint position)
|
||||
│ │ │ │ ├── CaptureFrame (Action)
|
||||
│ │ │ │ ├── RunTier1 (Action: YOLOE at high zoom)
|
||||
│ │ │ │ ├── ClassifyWaypoint (Selector — heuristic or VLM)
|
||||
│ │ │ │ │ ├── HighConfidence (Condition)
|
||||
│ │ │ │ │ │ └── LogDetection (Action: tier=2)
|
||||
│ │ │ │ │ └── AmbiguousWithVLM (Sequence)
|
||||
│ │ │ │ │ ├── CheckVLMAvailable (Condition)
|
||||
│ │ │ │ │ ├── RunVLM (Action)
|
||||
│ │ │ │ │ └── LogDetection (Action: tier=3)
|
||||
│ │ │ │ └── RecordFrame (Action)
|
||||
│ │ │ └── LogClusterSummary (Action: report cluster as a whole)
|
||||
│ │ ├── AreaSweepSubtree (Sequence — if scenario.type == area_sweep)
|
||||
│ │ │ ├── ComputeSweepPattern (Action: bounding box → pan/tilt waypoints)
|
||||
│ │ │ ├── SweepLoop (Repeat over waypoints)
|
||||
│ │ │ │ ├── SendGimbalCommand (Action)
|
||||
│ │ │ │ ├── CaptureFrame (Action)
|
||||
│ │ │ │ ├── RunTier1 (Action)
|
||||
│ │ │ │ └── CheckTargets (Action: match against scenario.target_classes)
|
||||
│ │ │ └── LogDetections (Action: all found targets)
|
||||
│ │ └── ZoomClassifySubtree (Sequence — if scenario.type == zoom_classify)
|
||||
│ │ ├── HoldZoom (Action: maintain zoom on POI)
|
||||
│ │ ├── CaptureMultipleFrames (Action: 3-5 frames for confidence)
|
||||
│ │ ├── RunTier1 (Action: on each frame)
|
||||
│ │ ├── AggregateResults (Action: majority vote on target_classes)
|
||||
│ │ └── LogDetection (Action)
|
||||
│ ├── ReportToOperator (Action)
|
||||
│ └── ReturnToSweep (Action: gimbal zoom out)
|
||||
│
|
||||
├── [3] L1Sweep (Sequence — default behavior)
|
||||
│ ├── HealthCheck (Action: read tegrastats, update capability flags)
|
||||
│ ├── AdvanceSweep (Action: compute next pan angle)
|
||||
│ ├── SendGimbalCommand (Action: set sweep target)
|
||||
│ ├── CaptureFrame (Action)
|
||||
│ ├── QualityGate (Condition: Laplacian variance > threshold)
|
||||
│ ├── RunTier1 (Action: YOLOE inference)
|
||||
│ ├── EvaluatePOI (Action: match detections against ALL active scenarios' trigger_classes)
|
||||
│ ├── RecordFrame (Action: L1 rate)
|
||||
│ └── LogDetections (Action)
|
||||
│
|
||||
└── [4] Idle (AlwaysSucceeds — fallback)
|
||||
```
|
||||
|
||||
### EvaluatePOI Logic (scenario-aware)
|
||||
|
||||
```
|
||||
for each detection in detections:
|
||||
for each scenario in active_scenarios:
|
||||
if detection.label in scenario.trigger.classes
|
||||
AND detection.confidence >= scenario.trigger.min_confidence:
|
||||
|
||||
if scenario.investigation.type == "cluster_follow":
|
||||
# Aggregate nearby detections into a single cluster POI
|
||||
matching = [d for d in detections
|
||||
if d.label in scenario.trigger.classes
|
||||
and d.confidence >= scenario.trigger.min_confidence]
|
||||
clusters = spatial_cluster(matching, scenario.cluster_radius_px)
|
||||
for cluster in clusters:
|
||||
if len(cluster) >= scenario.min_cluster_size:
|
||||
create POI with:
|
||||
trigger_class = cluster[0].label
|
||||
scenario_name = scenario.name
|
||||
investigation_type = "cluster_follow"
|
||||
cluster_detections = cluster
|
||||
priority = mean(d.confidence for d in cluster) * scenario.priority_boost
|
||||
add to POI queue (deduplicate by cluster overlap)
|
||||
break # scenario fully evaluated, don't create individual POIs
|
||||
else:
|
||||
create POI with:
|
||||
trigger_class = detection.label
|
||||
scenario_name = scenario.name
|
||||
investigation_type = scenario.investigation.type
|
||||
priority = detection.confidence * scenario.priority_boost * recency_factor
|
||||
add to POI queue (deduplicate by bbox overlap)
|
||||
```
|
||||
|
||||
### Blackboard Variables (py_trees shared state)
|
||||
|
||||
| Variable | Type | Written by | Read by |
|
||||
|----------|------|-----------|---------|
|
||||
| `frame` | FrameContext | CaptureFrame | RunTier1, RecordFrame, QualityGate |
|
||||
| `detections` | list[Detection] | RunTier1 | EvaluatePOI, InvestigateByScenario, LogDetections |
|
||||
| `poi_queue` | list[POI] | EvaluatePOI | CheckPOIQueue, PickHighestPOI |
|
||||
| `current_poi` | POI | PickHighestPOI | ZoomToPOI, InvestigateByScenario |
|
||||
| `active_scenarios` | list[SearchScenario] | Config load | EvaluatePOI, InvestigateByScenario |
|
||||
| `spatial_result` | SpatialAnalysisResult | TraceMask, TraceCluster | PIDFollow, WaypointAnalysis, VisitLoop |
|
||||
| `capability_flags` | CapabilityFlags | HealthCheck | HealthGuard, CheckVLMAvailable |
|
||||
| `scan_angle` | float | AdvanceSweep | SendGimbalCommand |
|
||||
|
||||
## 4. External API Specification
|
||||
|
||||
| Endpoint | Method | Auth | Rate Limit | Description |
|
||||
|----------|--------|------|------------|-------------|
|
||||
| `/api/v1/health` | GET | None | — | Returns health status + metrics |
|
||||
| `/api/v1/detect` | POST | None | Frame rate | Submit single frame for processing (dev/test mode) |
|
||||
|
||||
**Health response**:
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"tier1_ready": true,
|
||||
"gimbal_alive": true,
|
||||
"vlm_alive": false,
|
||||
"t_junction_c": 68.5,
|
||||
"capabilities": {"vlm_available": true, "gimbal_available": true, "semantic_available": true},
|
||||
"active_behavior": "L2Investigation.PathFollowSubtree.PIDFollow",
|
||||
"active_scenarios": ["winter_concealment", "building_area_search"],
|
||||
"frames_processed": 12345,
|
||||
"detections_total": 89,
|
||||
"poi_queue_depth": 2
|
||||
}
|
||||
```
|
||||
|
||||
## 5. Data Access Patterns
|
||||
|
||||
No database. State lives on the py_trees Blackboard:
|
||||
- POI queue: priority list on blackboard, max size from config (default 10)
|
||||
- Capability flags: 3 booleans on blackboard
|
||||
- Active scenarios: loaded from config at startup, stored on blackboard
|
||||
- Current frame: single FrameContext on blackboard (overwritten each tick)
|
||||
- Scan angle: float on blackboard (incremented each L1 tick)
|
||||
|
||||
## 6. Implementation Details
|
||||
|
||||
**Main Loop**:
|
||||
```python
|
||||
scenarios = config.get("search_scenarios")
|
||||
tree = create_scan_tree(config, components, scenarios)
|
||||
while running:
|
||||
tree.tick()
|
||||
```
|
||||
|
||||
Each `tick()` traverses the tree from root. The Selector tries HealthGuard first (preempts if degraded), then L2 (if POI queued), then L1 (default). Leaf nodes call into Tier1Detector, Tier2SpatialAnalyzer, VLMClient, GimbalDriver, OutputManager.
|
||||
|
||||
**InvestigateByScenario** dispatching: reads `current_poi.investigation_type` from blackboard, routes to the matching subtree (PathFollow / ClusterFollow / AreaSweep / ZoomClassify). Each subtree reads `current_poi.scenario` for target classes and VLM usage.
|
||||
|
||||
**Leaf Node Pattern**: Each leaf node is a simple py_trees.behaviour.Behaviour subclass. `setup()` gets component references. `update()` calls the component method and returns SUCCESS/FAILURE/RUNNING.
|
||||
|
||||
**Key Dependencies**:
|
||||
|
||||
| Library | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| py_trees | 2.4.0 | Behavior tree framework |
|
||||
| OpenCV | 4.x | Frame capture, Laplacian variance |
|
||||
| FastAPI | existing | Health + detect endpoints |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
- Leaf node exceptions → catch, log, return FAILURE → tree falls through to next Selector child
|
||||
- Component unavailable → Condition nodes gate access (CheckVLMAvailable, QualityGate)
|
||||
- Health degradation → HealthGuard decorator at root preempts all other behaviors
|
||||
- Invalid scenario config → log error at startup, skip invalid scenario, continue with valid ones
|
||||
|
||||
**Frame Quality Gate**: QualityGate is a Condition node in L1Sweep. If it returns FAILURE (blurry frame), the Sequence aborts and the tree ticks L1Sweep again next cycle (new frame).
|
||||
|
||||
## 7. Extensions and Helpers
|
||||
|
||||
| Helper | Purpose | Used By |
|
||||
|--------|---------|---------|
|
||||
| config | YAML config loading + validation | All components |
|
||||
| types | Shared structs (FrameContext, POI, SearchScenario, etc.) | All components |
|
||||
|
||||
### Adding a New Search Scenario (config only)
|
||||
|
||||
1. Define new detection classes in YOLOE (retrain if needed)
|
||||
2. Add YAML block under `search_scenarios` with trigger classes, investigation type, targets
|
||||
3. Restart service — new scenario is active
|
||||
|
||||
### Adding a New Investigation Type (code + config)
|
||||
|
||||
1. Create new BT subtree (e.g., `SpiralSearchSubtree`)
|
||||
2. Register it in InvestigateByScenario dispatcher
|
||||
3. Use `type: spiral_search` in scenario config
|
||||
|
||||
## 8. Caveats & Edge Cases
|
||||
|
||||
**Known limitations**:
|
||||
- Single-threaded tree ticking — throughput capped by slowest leaf per tick
|
||||
- py_trees Blackboard is not thread-safe (fine — single-threaded design)
|
||||
- POI queue doesn't persist across restarts
|
||||
- Scenario changes require service restart (no hot-reload)
|
||||
|
||||
**Performance bottlenecks**:
|
||||
- Tier 1 inference leaf is the bottleneck (~7-100ms)
|
||||
- Tree traversal overhead is negligible (<1ms)
|
||||
|
||||
## 9. Dependency Graph
|
||||
|
||||
**Must be implemented after**: Config helper, Types helper, ALL other components (02-06)
|
||||
**Can be implemented in parallel with**: None (top-level orchestrator)
|
||||
**Blocks**: Nothing (implemented last)
|
||||
|
||||
## 10. Logging Strategy
|
||||
|
||||
| Log Level | When | Example |
|
||||
|-----------|------|---------|
|
||||
| ERROR | Component crash, 3x retry exhausted, invalid scenario | `Gimbal UART failed 3 times, disabling gimbal` |
|
||||
| WARN | Frame skipped, VLM timeout, leaf FAILURE | `QualityGate FAILURE: Laplacian 12.3 < 50.0` |
|
||||
| INFO | State transitions, POI created, scenario match | `POI queued: winter_concealment triggered by footpath_winter (conf=0.72)` |
|
||||
|
||||
**py_trees built-in logging**: Tree can render active path as ASCII for debugging:
|
||||
```
|
||||
[o] Root
|
||||
[-] HealthGuard
|
||||
[o] L2Investigation
|
||||
[o] L2DetectLoop
|
||||
[o] InvestigateByScenario
|
||||
[o] PathFollowSubtree
|
||||
[*] PIDFollow (RUNNING)
|
||||
```
|
||||
@@ -0,0 +1,516 @@
|
||||
# Test Specification — ScanController
|
||||
|
||||
## Acceptance Criteria Traceability
|
||||
|
||||
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|
||||
|-------|---------------------|----------|----------|
|
||||
| AC-09 | L1 wide-area scan covers planned route with left-right camera sweep at medium zoom | IT-01, AT-01 | Covered |
|
||||
| AC-10 | POIs detected during L1: footpaths, tree rows, branch piles, black entrances, houses with vehicles/traces, roads | IT-02, IT-03, AT-02 | Covered |
|
||||
| AC-11 | L1→L2 transition within 2 seconds of POI detection | IT-04, PT-01, AT-03 | Covered |
|
||||
| AC-12 | L2 maintains camera lock on POI while UAV continues flight | IT-05, AT-04 | Covered |
|
||||
| AC-13 | Path-following mode: camera pans along footpath keeping it visible and centered | IT-06, AT-05 | Covered |
|
||||
| AC-14 | Endpoint hold: camera maintains position on path endpoint for VLM analysis (up to 2s) | IT-07, AT-06 | Covered |
|
||||
| AC-15 | Return to L1 after analysis completes or configurable timeout (default 5s) | IT-08, AT-07 | Covered |
|
||||
| AC-21 | POI queue: ordered by confidence and proximity | IT-09, IT-10 | Covered |
|
||||
| AC-22 | Semantic pipeline consumes YOLO detections as input | IT-02, IT-11 | Covered |
|
||||
| AC-27 | Coexist with YOLO pipeline without degrading YOLO performance | PT-02 | Covered |
|
||||
|
||||
---
|
||||
|
||||
## Integration Tests
|
||||
|
||||
### IT-01: L1 Sweep Cycle Completes
|
||||
|
||||
**Summary**: Verify a single L1 sweep tick advances the scan angle and invokes Tier1 inference.
|
||||
|
||||
**Traces to**: AC-09
|
||||
|
||||
**Input data**:
|
||||
- Mock Tier1Detector returning empty detection list
|
||||
- Mock GimbalDriver accepting set_sweep_target()
|
||||
- Config: sweep_angle_range=45, sweep_step=5
|
||||
|
||||
**Expected result**:
|
||||
- Scan angle increments by sweep_step
|
||||
- GimbalDriver.set_sweep_target called with new angle
|
||||
- Tier1Detector.detect called once
|
||||
- Tree returns SUCCESS from L1Sweep branch
|
||||
|
||||
**Max execution time**: 500ms
|
||||
|
||||
**Dependencies**: Mock Tier1Detector, Mock GimbalDriver, Mock OutputManager
|
||||
|
||||
---
|
||||
|
||||
### IT-02: EvaluatePOI Creates POI from Trigger Class Match
|
||||
|
||||
**Summary**: Verify that when Tier1 returns a detection matching a scenario trigger class, a POI is created and queued.
|
||||
|
||||
**Traces to**: AC-10, AC-22
|
||||
|
||||
**Input data**:
|
||||
- Detection: {label: "footpath_winter", confidence: 0.72, bbox: (0.5, 0.5, 0.1, 0.3)}
|
||||
- Active scenario: winter_concealment (trigger classes: [footpath_winter, branch_pile, dark_entrance], min_confidence: 0.5)
|
||||
|
||||
**Expected result**:
|
||||
- POI created with scenario_name="winter_concealment", investigation_type="path_follow"
|
||||
- POI priority = 0.72 * 1.0 (priority_boost)
|
||||
- POI added to blackboard poi_queue
|
||||
|
||||
**Max execution time**: 100ms
|
||||
|
||||
**Dependencies**: Mock Tier1Detector
|
||||
|
||||
---
|
||||
|
||||
### IT-03: EvaluatePOI Cluster Aggregation
|
||||
|
||||
**Summary**: Verify that cluster_follow scenarios aggregate multiple nearby detections into a single cluster POI.
|
||||
|
||||
**Traces to**: AC-10
|
||||
|
||||
**Input data**:
|
||||
- Detections: 3x {label: "radar_dish", confidence: 0.6}, centers within 200px of each other
|
||||
- Active scenario: aa_defense_network (type: cluster_follow, min_cluster_size: 2, cluster_radius_px: 300)
|
||||
|
||||
**Expected result**:
|
||||
- Single cluster POI created (not 3 individual POIs)
|
||||
- cluster_detections contains all 3 detections
|
||||
- priority = mean(confidences) * 1.5
|
||||
|
||||
**Max execution time**: 100ms
|
||||
|
||||
**Dependencies**: Mock Tier1Detector
|
||||
|
||||
---
|
||||
|
||||
### IT-04: L1→L2 Transition Timing
|
||||
|
||||
**Summary**: Verify that when a POI is queued, the next tree tick enters L2Investigation and issues a zoom command.
|
||||
|
||||
**Traces to**: AC-11
|
||||
|
||||
**Input data**:
|
||||
- POI already on blackboard queue
|
||||
- Mock GimbalDriver.zoom_to_poi returns success after simulated delay
|
||||
|
||||
**Expected result**:
|
||||
- Tree selects L2Investigation branch (not L1Sweep)
|
||||
- GimbalDriver.zoom_to_poi called
|
||||
- Transition from L1 tick to GimbalDriver call occurs within same tick cycle (<100ms in mock)
|
||||
|
||||
**Max execution time**: 500ms
|
||||
|
||||
**Dependencies**: Mock GimbalDriver, Mock Tier1Detector
|
||||
|
||||
---
|
||||
|
||||
### IT-05: L2 Investigation Maintains Zoom on POI
|
||||
|
||||
**Summary**: Verify L2DetectLoop continuously captures frames and runs Tier1 at high zoom while investigating a POI.
|
||||
|
||||
**Traces to**: AC-12
|
||||
|
||||
**Input data**:
|
||||
- POI with investigation_type="zoom_classify"
|
||||
- Mock Tier1Detector returns detections at high zoom
|
||||
- Investigation timeout: 10s
|
||||
|
||||
**Expected result**:
|
||||
- Multiple CaptureFrame + RunTier1 cycles within the L2DetectLoop
|
||||
- GimbalDriver maintains zoom level (no return_to_sweep called during investigation)
|
||||
|
||||
**Max execution time**: 2s
|
||||
|
||||
**Dependencies**: Mock Tier1Detector, Mock GimbalDriver, Mock OutputManager
|
||||
|
||||
---
|
||||
|
||||
### IT-06: PathFollowSubtree Invokes Tier2 and PID
|
||||
|
||||
**Summary**: Verify that for a path_follow investigation, TraceMask is called and gimbal PID follow is engaged along the skeleton trajectory.
|
||||
|
||||
**Traces to**: AC-13
|
||||
|
||||
**Input data**:
|
||||
- POI with investigation_type="path_follow", scenario=winter_concealment
|
||||
- Mock Tier2SpatialAnalyzer.trace_mask returns SpatialAnalysisResult with 3 waypoints and trajectory
|
||||
- Mock GimbalDriver.follow_path accepts direction commands
|
||||
|
||||
**Expected result**:
|
||||
- Tier2SpatialAnalyzer.trace_mask called with the frame's segmentation mask
|
||||
- GimbalDriver.follow_path called with direction from SpatialAnalysisResult.overall_direction
|
||||
- WaypointAnalysis evaluates each waypoint
|
||||
|
||||
**Max execution time**: 1s
|
||||
|
||||
**Dependencies**: Mock Tier2SpatialAnalyzer, Mock GimbalDriver
|
||||
|
||||
---
|
||||
|
||||
### IT-07: Endpoint Hold for VLM Analysis
|
||||
|
||||
**Summary**: Verify that at a path endpoint with ambiguous confidence, the system holds position and invokes VLMClient.
|
||||
|
||||
**Traces to**: AC-14
|
||||
|
||||
**Input data**:
|
||||
- Waypoint at skeleton endpoint with confidence=0.4 (below high-confidence threshold)
|
||||
- Scenario.use_vlm=true
|
||||
- Mock VLMClient.is_available=true
|
||||
- Mock VLMClient.analyze returns VLMResponse within 2s
|
||||
|
||||
**Expected result**:
|
||||
- CheckVLMAvailable returns SUCCESS
|
||||
- RunVLM action invokes VLMClient.analyze with ROI image and prompt
|
||||
- LogDetection records tier=3
|
||||
|
||||
**Max execution time**: 3s
|
||||
|
||||
**Dependencies**: Mock VLMClient, Mock OutputManager
|
||||
|
||||
---
|
||||
|
||||
### IT-08: Return to L1 After Investigation Completes
|
||||
|
||||
**Summary**: Verify the tree returns to L1Sweep after L2Investigation finishes.
|
||||
|
||||
**Traces to**: AC-15
|
||||
|
||||
**Input data**:
|
||||
- L2Investigation sequence completes (all waypoints analyzed)
|
||||
- Mock GimbalDriver.return_to_sweep returns success
|
||||
|
||||
**Expected result**:
|
||||
- ReturnToSweep action calls GimbalDriver.return_to_sweep
|
||||
- Next tree tick enters L1Sweep branch (POI queue now empty)
|
||||
- Scan angle resumes from where it left off
|
||||
|
||||
**Max execution time**: 500ms
|
||||
|
||||
**Dependencies**: Mock GimbalDriver
|
||||
|
||||
---
|
||||
|
||||
### IT-09: POI Queue Priority Ordering
|
||||
|
||||
**Summary**: Verify the POI queue returns the highest-priority POI first.
|
||||
|
||||
**Traces to**: AC-21
|
||||
|
||||
**Input data**:
|
||||
- 3 POIs: {priority: 0.5}, {priority: 0.9}, {priority: 0.3}
|
||||
|
||||
**Expected result**:
|
||||
- PickHighestPOI retrieves POI with priority=0.9 first
|
||||
- Subsequent picks return 0.5, then 0.3
|
||||
|
||||
**Max execution time**: 100ms
|
||||
|
||||
**Dependencies**: None
|
||||
|
||||
---
|
||||
|
||||
### IT-10: POI Queue Deduplication
|
||||
|
||||
**Summary**: Verify that overlapping POIs (same bbox area) are deduplicated.
|
||||
|
||||
**Traces to**: AC-21
|
||||
|
||||
**Input data**:
|
||||
- Two detections with >70% bbox overlap, same scenario trigger
|
||||
|
||||
**Expected result**:
|
||||
- Only one POI created; higher-confidence one kept
|
||||
|
||||
**Max execution time**: 100ms
|
||||
|
||||
**Dependencies**: None
|
||||
|
||||
---
|
||||
|
||||
### IT-11: HealthGuard Disables Semantic When Overheating
|
||||
|
||||
**Summary**: Verify the HealthGuard decorator routes to FallbackBehavior when capability flags degrade.
|
||||
|
||||
**Traces to**: AC-22, AC-27
|
||||
|
||||
**Input data**:
|
||||
- capability_flags: {semantic_available: false, gimbal_available: true, vlm_available: false}
|
||||
|
||||
**Expected result**:
|
||||
- HealthGuard activates FallbackBehavior
|
||||
- Tree runs existing YOLO only (no EvaluatePOI, no L2 investigation)
|
||||
|
||||
**Max execution time**: 500ms
|
||||
|
||||
**Dependencies**: Mock Tier1Detector
|
||||
|
||||
---
|
||||
|
||||
## Performance Tests
|
||||
|
||||
### PT-01: L1→L2 Transition Latency Under Load
|
||||
|
||||
**Summary**: Measure the time from POI detection to GimbalDriver.zoom_to_poi call under continuous inference load.
|
||||
|
||||
**Traces to**: AC-11
|
||||
|
||||
**Load scenario**:
|
||||
- Continuous L1 sweep at 30 FPS
|
||||
- POI injected at frame N
|
||||
- Duration: 60 seconds
|
||||
- Ramp-up: immediate
|
||||
|
||||
**Expected results**:
|
||||
|
||||
| Metric | Target | Failure Threshold |
|
||||
|--------|--------|-------------------|
|
||||
| Transition latency (p50) | ≤500ms | >2000ms |
|
||||
| Transition latency (p95) | ≤1500ms | >2000ms |
|
||||
| Transition latency (p99) | ≤2000ms | >2000ms |
|
||||
|
||||
**Resource limits**:
|
||||
- CPU: ≤80%
|
||||
- Memory: ≤6GB total (semantic module)
|
||||
|
||||
---
|
||||
|
||||
### PT-02: Sustained L1 Sweep Does Not Degrade YOLO Throughput
|
||||
|
||||
**Summary**: Verify that running the full behavior tree with L1 sweep does not reduce YOLO inference FPS below baseline.
|
||||
|
||||
**Traces to**: AC-27
|
||||
|
||||
**Load scenario**:
|
||||
- Baseline: YOLO only, 30 FPS, 300 frames
|
||||
- Test: YOLO + ScanController L1 sweep, 30 FPS, 300 frames
|
||||
- Duration: 10 seconds each
|
||||
- Ramp-up: none
|
||||
|
||||
**Expected results**:
|
||||
|
||||
| Metric | Target | Failure Threshold |
|
||||
|--------|--------|-------------------|
|
||||
| FPS delta (baseline vs test) | ≤5% reduction | >10% reduction |
|
||||
| Frame drop rate | ≤1% | >5% |
|
||||
|
||||
**Resource limits**:
|
||||
- GPU memory: ≤2.5GB for YOLO engine
|
||||
- CPU: ≤60% for tree overhead
|
||||
|
||||
---
|
||||
|
||||
## Security Tests
|
||||
|
||||
### ST-01: Health Endpoint Does Not Expose Sensitive Data
|
||||
|
||||
**Summary**: Verify /api/v1/health response contains only operational metrics, no file paths, secrets, or internal state.
|
||||
|
||||
**Traces to**: AC-27
|
||||
|
||||
**Attack vector**: Information disclosure via health endpoint
|
||||
|
||||
**Test procedure**:
|
||||
1. GET /api/v1/health
|
||||
2. Parse response JSON
|
||||
3. Check no field contains file system paths, config values, or credentials
|
||||
|
||||
**Expected behavior**: Response contains only status, readiness booleans, temperature, capability flags, counters.
|
||||
|
||||
**Pass criteria**: No field value matches regex for file paths (`/[a-z]+/`), env vars, or credential patterns.
|
||||
|
||||
**Fail criteria**: Any file path, secret, or config detail in response body.
|
||||
|
||||
---
|
||||
|
||||
### ST-02: Detect Endpoint Input Validation
|
||||
|
||||
**Summary**: Verify /api/v1/detect rejects malformed input gracefully.
|
||||
|
||||
**Traces to**: AC-22
|
||||
|
||||
**Attack vector**: Denial of service via oversized or malformed frame submission
|
||||
|
||||
**Test procedure**:
|
||||
1. POST /api/v1/detect with empty body → expect 400
|
||||
2. POST /api/v1/detect with 100MB payload → expect 413
|
||||
3. POST /api/v1/detect with non-image content type → expect 415
|
||||
|
||||
**Expected behavior**: Server returns appropriate HTTP error codes, does not crash.
|
||||
|
||||
**Pass criteria**: All 3 requests return expected error codes; server remains operational.
|
||||
|
||||
**Fail criteria**: Server crashes, hangs, or returns 500.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Tests
|
||||
|
||||
### AT-01: Full L1 Sweep Covers Angle Range
|
||||
|
||||
**Summary**: Verify the sweep completes from -sweep_angle_range to +sweep_angle_range and wraps around.
|
||||
|
||||
**Traces to**: AC-09
|
||||
|
||||
**Preconditions**:
|
||||
- System running with mock gimbal in dev environment
|
||||
- Config: sweep_angle_range=45, sweep_step=5
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | Start system | L1Sweep active, scan_angle starts at -45 |
|
||||
| 2 | Let system run for 18+ ticks | Scan angle reaches +45 |
|
||||
| 3 | Next tick | Scan angle wraps back to -45 |
|
||||
|
||||
---
|
||||
|
||||
### AT-02: POI Detection for All Trigger Classes
|
||||
|
||||
**Summary**: Verify that each configured trigger class type produces a POI when detected.
|
||||
|
||||
**Traces to**: AC-10
|
||||
|
||||
**Preconditions**:
|
||||
- All search scenarios enabled
|
||||
- Mock Tier1 returns one detection per trigger class sequentially
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | Inject footpath_winter detection (conf=0.7) | POI created with scenario=winter_concealment |
|
||||
| 2 | Inject branch_pile detection (conf=0.6) | POI created with scenario=winter_concealment |
|
||||
| 3 | Inject building_block detection (conf=0.8) | POI created with scenario=building_area_search |
|
||||
| 4 | Inject radar_dish + aa_launcher (conf=0.5 each, within 200px) | Cluster POI created with scenario=aa_defense_network |
|
||||
|
||||
---
|
||||
|
||||
### AT-03: End-to-End L1→L2→L1 Cycle
|
||||
|
||||
**Summary**: Verify complete investigation lifecycle from POI detection to return to sweep.
|
||||
|
||||
**Traces to**: AC-11
|
||||
|
||||
**Preconditions**:
|
||||
- System running with mock components
|
||||
- winter_concealment scenario active
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | Inject footpath_winter detection | POI queued, L2Investigation starts |
|
||||
| 2 | Wait for zoom_to_poi | GimbalDriver zooms to POI location |
|
||||
| 3 | Tier2.trace_mask returns waypoints | PathFollowSubtree engages |
|
||||
| 4 | Investigation completes | GimbalDriver.return_to_sweep called |
|
||||
| 5 | Next tick | L1Sweep resumes |
|
||||
|
||||
---
|
||||
|
||||
### AT-04: L2 Camera Lock During Investigation
|
||||
|
||||
**Summary**: Verify gimbal maintains zoom and tracking during L2 investigation.
|
||||
|
||||
**Traces to**: AC-12
|
||||
|
||||
**Preconditions**:
|
||||
- L2Investigation active on a POI
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | L2 investigation starts | Gimbal zoomed to POI |
|
||||
| 2 | Monitor gimbal state during investigation | Zoom level remains constant |
|
||||
| 3 | Investigation timeout reached | return_to_sweep called (not before) |
|
||||
|
||||
---
|
||||
|
||||
### AT-05: Path Following Stays Centered
|
||||
|
||||
**Summary**: Verify gimbal PID follows path trajectory keeping the path centered.
|
||||
|
||||
**Traces to**: AC-13
|
||||
|
||||
**Preconditions**:
|
||||
- PathFollowSubtree active with a mock skeleton trajectory
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | PIDFollow starts | follow_path called with trajectory direction |
|
||||
| 2 | Multiple PID updates (10 cycles) | Direction updates sent to GimbalDriver |
|
||||
| 3 | Path trajectory ends | PIDFollow returns SUCCESS |
|
||||
|
||||
---
|
||||
|
||||
### AT-06: VLM Analysis at Path Endpoint
|
||||
|
||||
**Summary**: Verify VLM is invoked for ambiguous endpoint classifications.
|
||||
|
||||
**Traces to**: AC-14
|
||||
|
||||
**Preconditions**:
|
||||
- PathFollowSubtree at WaypointAnalysis step
|
||||
- Waypoint confidence below threshold
|
||||
- VLM available
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | WaypointAnalysis evaluates endpoint | HighConfidence condition fails |
|
||||
| 2 | AmbiguousWithVLM sequence begins | CheckVLMAvailable returns SUCCESS |
|
||||
| 3 | RunVLM action | VLMClient.analyze called, response received |
|
||||
| 4 | LogDetection | Detection logged with tier=3 |
|
||||
|
||||
---
|
||||
|
||||
### AT-07: Timeout Returns to L1
|
||||
|
||||
**Summary**: Verify investigation times out and returns to L1 when timeout expires.
|
||||
|
||||
**Traces to**: AC-15
|
||||
|
||||
**Preconditions**:
|
||||
- L2Investigation active
|
||||
- Config: investigation_timeout_s=5
|
||||
- Mock Tier2 returns long-running analysis
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | L2DetectLoop starts | Investigation proceeds |
|
||||
| 2 | 5 seconds elapse | L2DetectLoop repeat terminates |
|
||||
| 3 | ReportToOperator called | Partial results reported |
|
||||
| 4 | ReturnToSweep | GimbalDriver.return_to_sweep called |
|
||||
|
||||
---
|
||||
|
||||
## Test Data Management
|
||||
|
||||
**Required test data**:
|
||||
|
||||
| Data Set | Description | Source | Size |
|
||||
|----------|-------------|--------|------|
|
||||
| mock_detections | Pre-defined detection lists per scenario type | Generated fixtures | ~10 KB |
|
||||
| mock_spatial_results | SpatialAnalysisResult objects with waypoints | Generated fixtures | ~5 KB |
|
||||
| mock_vlm_responses | VLMResponse objects for endpoint analysis | Generated fixtures | ~2 KB |
|
||||
| scenario_configs | YAML search scenario configurations (valid + invalid) | Generated fixtures | ~3 KB |
|
||||
|
||||
**Setup procedure**:
|
||||
1. Load mock component implementations that return fixture data
|
||||
2. Initialize BT with test config
|
||||
3. Set blackboard variables to known state
|
||||
|
||||
**Teardown procedure**:
|
||||
1. Shutdown tree
|
||||
2. Clear blackboard
|
||||
3. Reset mock call counters
|
||||
|
||||
**Data isolation strategy**: Each test initializes a fresh BT instance with clean blackboard. No shared mutable state between tests.
|
||||
Reference in New Issue
Block a user