# Test Specification — ScanController ## Acceptance Criteria Traceability | AC ID | Acceptance Criterion | Test IDs | Coverage | |-------|---------------------|----------|----------| | AC-09 | L1 wide-area scan covers planned route with left-right camera sweep at medium zoom | IT-01, AT-01 | Covered | | AC-10 | POIs detected during L1: footpaths, tree rows, branch piles, black entrances, houses with vehicles/traces, roads | IT-02, IT-03, AT-02 | Covered | | AC-11 | L1→L2 transition within 2 seconds of POI detection | IT-04, PT-01, AT-03 | Covered | | AC-12 | L2 maintains camera lock on POI while UAV continues flight | IT-05, AT-04 | Covered | | AC-13 | Path-following mode: camera pans along footpath keeping it visible and centered | IT-06, AT-05 | Covered | | AC-14 | Endpoint hold: camera maintains position on path endpoint for VLM analysis (up to 2s) | IT-07, AT-06 | Covered | | AC-15 | Return to L1 after analysis completes or configurable timeout (default 5s) | IT-08, AT-07 | Covered | | AC-21 | POI queue: ordered by confidence and proximity | IT-09, IT-10 | Covered | | AC-22 | Semantic pipeline consumes YOLO detections as input | IT-02, IT-11 | Covered | | AC-27 | Coexist with YOLO pipeline without degrading YOLO performance | PT-02 | Covered | --- ## Integration Tests ### IT-01: L1 Sweep Cycle Completes **Summary**: Verify a single L1 sweep tick advances the scan angle and invokes Tier1 inference. **Traces to**: AC-09 **Input data**: - Mock Tier1Detector returning empty detection list - Mock GimbalDriver accepting set_sweep_target() - Config: sweep_angle_range=45, sweep_step=5 **Expected result**: - Scan angle increments by sweep_step - GimbalDriver.set_sweep_target called with new angle - Tier1Detector.detect called once - Tree returns SUCCESS from L1Sweep branch **Max execution time**: 500ms **Dependencies**: Mock Tier1Detector, Mock GimbalDriver, Mock OutputManager --- ### IT-02: EvaluatePOI Creates POI from Trigger Class Match **Summary**: Verify that when Tier1 returns a detection matching a scenario trigger class, a POI is created and queued. **Traces to**: AC-10, AC-22 **Input data**: - Detection: {label: "footpath_winter", confidence: 0.72, bbox: (0.5, 0.5, 0.1, 0.3)} - Active scenario: winter_concealment (trigger classes: [footpath_winter, branch_pile, dark_entrance], min_confidence: 0.5) **Expected result**: - POI created with scenario_name="winter_concealment", investigation_type="path_follow" - POI priority = 0.72 * 1.0 (priority_boost) - POI added to blackboard poi_queue **Max execution time**: 100ms **Dependencies**: Mock Tier1Detector --- ### IT-03: EvaluatePOI Cluster Aggregation **Summary**: Verify that cluster_follow scenarios aggregate multiple nearby detections into a single cluster POI. **Traces to**: AC-10 **Input data**: - Detections: 3x {label: "radar_dish", confidence: 0.6}, centers within 200px of each other - Active scenario: aa_defense_network (type: cluster_follow, min_cluster_size: 2, cluster_radius_px: 300) **Expected result**: - Single cluster POI created (not 3 individual POIs) - cluster_detections contains all 3 detections - priority = mean(confidences) * 1.5 **Max execution time**: 100ms **Dependencies**: Mock Tier1Detector --- ### IT-04: L1→L2 Transition Timing **Summary**: Verify that when a POI is queued, the next tree tick enters L2Investigation and issues a zoom command. **Traces to**: AC-11 **Input data**: - POI already on blackboard queue - Mock GimbalDriver.zoom_to_poi returns success after simulated delay **Expected result**: - Tree selects L2Investigation branch (not L1Sweep) - GimbalDriver.zoom_to_poi called - Transition from L1 tick to GimbalDriver call occurs within same tick cycle (<100ms in mock) **Max execution time**: 500ms **Dependencies**: Mock GimbalDriver, Mock Tier1Detector --- ### IT-05: L2 Investigation Maintains Zoom on POI **Summary**: Verify L2DetectLoop continuously captures frames and runs Tier1 at high zoom while investigating a POI. **Traces to**: AC-12 **Input data**: - POI with investigation_type="zoom_classify" - Mock Tier1Detector returns detections at high zoom - Investigation timeout: 10s **Expected result**: - Multiple CaptureFrame + RunTier1 cycles within the L2DetectLoop - GimbalDriver maintains zoom level (no return_to_sweep called during investigation) **Max execution time**: 2s **Dependencies**: Mock Tier1Detector, Mock GimbalDriver, Mock OutputManager --- ### IT-06: PathFollowSubtree Invokes Tier2 and PID **Summary**: Verify that for a path_follow investigation, TraceMask is called and gimbal PID follow is engaged along the skeleton trajectory. **Traces to**: AC-13 **Input data**: - POI with investigation_type="path_follow", scenario=winter_concealment - Mock Tier2SpatialAnalyzer.trace_mask returns SpatialAnalysisResult with 3 waypoints and trajectory - Mock GimbalDriver.follow_path accepts direction commands **Expected result**: - Tier2SpatialAnalyzer.trace_mask called with the frame's segmentation mask - GimbalDriver.follow_path called with direction from SpatialAnalysisResult.overall_direction - WaypointAnalysis evaluates each waypoint **Max execution time**: 1s **Dependencies**: Mock Tier2SpatialAnalyzer, Mock GimbalDriver --- ### IT-07: Endpoint Hold for VLM Analysis **Summary**: Verify that at a path endpoint with ambiguous confidence, the system holds position and invokes VLMClient. **Traces to**: AC-14 **Input data**: - Waypoint at skeleton endpoint with confidence=0.4 (below high-confidence threshold) - Scenario.use_vlm=true - Mock VLMClient.is_available=true - Mock VLMClient.analyze returns VLMResponse within 2s **Expected result**: - CheckVLMAvailable returns SUCCESS - RunVLM action invokes VLMClient.analyze with ROI image and prompt - LogDetection records tier=3 **Max execution time**: 3s **Dependencies**: Mock VLMClient, Mock OutputManager --- ### IT-08: Return to L1 After Investigation Completes **Summary**: Verify the tree returns to L1Sweep after L2Investigation finishes. **Traces to**: AC-15 **Input data**: - L2Investigation sequence completes (all waypoints analyzed) - Mock GimbalDriver.return_to_sweep returns success **Expected result**: - ReturnToSweep action calls GimbalDriver.return_to_sweep - Next tree tick enters L1Sweep branch (POI queue now empty) - Scan angle resumes from where it left off **Max execution time**: 500ms **Dependencies**: Mock GimbalDriver --- ### IT-09: POI Queue Priority Ordering **Summary**: Verify the POI queue returns the highest-priority POI first. **Traces to**: AC-21 **Input data**: - 3 POIs: {priority: 0.5}, {priority: 0.9}, {priority: 0.3} **Expected result**: - PickHighestPOI retrieves POI with priority=0.9 first - Subsequent picks return 0.5, then 0.3 **Max execution time**: 100ms **Dependencies**: None --- ### IT-10: POI Queue Deduplication **Summary**: Verify that overlapping POIs (same bbox area) are deduplicated. **Traces to**: AC-21 **Input data**: - Two detections with >70% bbox overlap, same scenario trigger **Expected result**: - Only one POI created; higher-confidence one kept **Max execution time**: 100ms **Dependencies**: None --- ### IT-11: HealthGuard Disables Semantic When Overheating **Summary**: Verify the HealthGuard decorator routes to FallbackBehavior when capability flags degrade. **Traces to**: AC-22, AC-27 **Input data**: - capability_flags: {semantic_available: false, gimbal_available: true, vlm_available: false} **Expected result**: - HealthGuard activates FallbackBehavior - Tree runs existing YOLO only (no EvaluatePOI, no L2 investigation) **Max execution time**: 500ms **Dependencies**: Mock Tier1Detector --- ## Performance Tests ### PT-01: L1→L2 Transition Latency Under Load **Summary**: Measure the time from POI detection to GimbalDriver.zoom_to_poi call under continuous inference load. **Traces to**: AC-11 **Load scenario**: - Continuous L1 sweep at 30 FPS - POI injected at frame N - Duration: 60 seconds - Ramp-up: immediate **Expected results**: | Metric | Target | Failure Threshold | |--------|--------|-------------------| | Transition latency (p50) | ≤500ms | >2000ms | | Transition latency (p95) | ≤1500ms | >2000ms | | Transition latency (p99) | ≤2000ms | >2000ms | **Resource limits**: - CPU: ≤80% - Memory: ≤6GB total (semantic module) --- ### PT-02: Sustained L1 Sweep Does Not Degrade YOLO Throughput **Summary**: Verify that running the full behavior tree with L1 sweep does not reduce YOLO inference FPS below baseline. **Traces to**: AC-27 **Load scenario**: - Baseline: YOLO only, 30 FPS, 300 frames - Test: YOLO + ScanController L1 sweep, 30 FPS, 300 frames - Duration: 10 seconds each - Ramp-up: none **Expected results**: | Metric | Target | Failure Threshold | |--------|--------|-------------------| | FPS delta (baseline vs test) | ≤5% reduction | >10% reduction | | Frame drop rate | ≤1% | >5% | **Resource limits**: - GPU memory: ≤2.5GB for YOLO engine - CPU: ≤60% for tree overhead --- ## Security Tests ### ST-01: Health Endpoint Does Not Expose Sensitive Data **Summary**: Verify /api/v1/health response contains only operational metrics, no file paths, secrets, or internal state. **Traces to**: AC-27 **Attack vector**: Information disclosure via health endpoint **Test procedure**: 1. GET /api/v1/health 2. Parse response JSON 3. Check no field contains file system paths, config values, or credentials **Expected behavior**: Response contains only status, readiness booleans, temperature, capability flags, counters. **Pass criteria**: No field value matches regex for file paths (`/[a-z]+/`), env vars, or credential patterns. **Fail criteria**: Any file path, secret, or config detail in response body. --- ### ST-02: Detect Endpoint Input Validation **Summary**: Verify /api/v1/detect rejects malformed input gracefully. **Traces to**: AC-22 **Attack vector**: Denial of service via oversized or malformed frame submission **Test procedure**: 1. POST /api/v1/detect with empty body → expect 400 2. POST /api/v1/detect with 100MB payload → expect 413 3. POST /api/v1/detect with non-image content type → expect 415 **Expected behavior**: Server returns appropriate HTTP error codes, does not crash. **Pass criteria**: All 3 requests return expected error codes; server remains operational. **Fail criteria**: Server crashes, hangs, or returns 500. --- ## Acceptance Tests ### AT-01: Full L1 Sweep Covers Angle Range **Summary**: Verify the sweep completes from -sweep_angle_range to +sweep_angle_range and wraps around. **Traces to**: AC-09 **Preconditions**: - System running with mock gimbal in dev environment - Config: sweep_angle_range=45, sweep_step=5 **Steps**: | Step | Action | Expected Result | |------|--------|-----------------| | 1 | Start system | L1Sweep active, scan_angle starts at -45 | | 2 | Let system run for 18+ ticks | Scan angle reaches +45 | | 3 | Next tick | Scan angle wraps back to -45 | --- ### AT-02: POI Detection for All Trigger Classes **Summary**: Verify that each configured trigger class type produces a POI when detected. **Traces to**: AC-10 **Preconditions**: - All search scenarios enabled - Mock Tier1 returns one detection per trigger class sequentially **Steps**: | Step | Action | Expected Result | |------|--------|-----------------| | 1 | Inject footpath_winter detection (conf=0.7) | POI created with scenario=winter_concealment | | 2 | Inject branch_pile detection (conf=0.6) | POI created with scenario=winter_concealment | | 3 | Inject building_block detection (conf=0.8) | POI created with scenario=building_area_search | | 4 | Inject radar_dish + aa_launcher (conf=0.5 each, within 200px) | Cluster POI created with scenario=aa_defense_network | --- ### AT-03: End-to-End L1→L2→L1 Cycle **Summary**: Verify complete investigation lifecycle from POI detection to return to sweep. **Traces to**: AC-11 **Preconditions**: - System running with mock components - winter_concealment scenario active **Steps**: | Step | Action | Expected Result | |------|--------|-----------------| | 1 | Inject footpath_winter detection | POI queued, L2Investigation starts | | 2 | Wait for zoom_to_poi | GimbalDriver zooms to POI location | | 3 | Tier2.trace_mask returns waypoints | PathFollowSubtree engages | | 4 | Investigation completes | GimbalDriver.return_to_sweep called | | 5 | Next tick | L1Sweep resumes | --- ### AT-04: L2 Camera Lock During Investigation **Summary**: Verify gimbal maintains zoom and tracking during L2 investigation. **Traces to**: AC-12 **Preconditions**: - L2Investigation active on a POI **Steps**: | Step | Action | Expected Result | |------|--------|-----------------| | 1 | L2 investigation starts | Gimbal zoomed to POI | | 2 | Monitor gimbal state during investigation | Zoom level remains constant | | 3 | Investigation timeout reached | return_to_sweep called (not before) | --- ### AT-05: Path Following Stays Centered **Summary**: Verify gimbal PID follows path trajectory keeping the path centered. **Traces to**: AC-13 **Preconditions**: - PathFollowSubtree active with a mock skeleton trajectory **Steps**: | Step | Action | Expected Result | |------|--------|-----------------| | 1 | PIDFollow starts | follow_path called with trajectory direction | | 2 | Multiple PID updates (10 cycles) | Direction updates sent to GimbalDriver | | 3 | Path trajectory ends | PIDFollow returns SUCCESS | --- ### AT-06: VLM Analysis at Path Endpoint **Summary**: Verify VLM is invoked for ambiguous endpoint classifications. **Traces to**: AC-14 **Preconditions**: - PathFollowSubtree at WaypointAnalysis step - Waypoint confidence below threshold - VLM available **Steps**: | Step | Action | Expected Result | |------|--------|-----------------| | 1 | WaypointAnalysis evaluates endpoint | HighConfidence condition fails | | 2 | AmbiguousWithVLM sequence begins | CheckVLMAvailable returns SUCCESS | | 3 | RunVLM action | VLMClient.analyze called, response received | | 4 | LogDetection | Detection logged with tier=3 | --- ### AT-07: Timeout Returns to L1 **Summary**: Verify investigation times out and returns to L1 when timeout expires. **Traces to**: AC-15 **Preconditions**: - L2Investigation active - Config: investigation_timeout_s=5 - Mock Tier2 returns long-running analysis **Steps**: | Step | Action | Expected Result | |------|--------|-----------------| | 1 | L2DetectLoop starts | Investigation proceeds | | 2 | 5 seconds elapse | L2DetectLoop repeat terminates | | 3 | ReportToOperator called | Partial results reported | | 4 | ReturnToSweep | GimbalDriver.return_to_sweep called | --- ## Test Data Management **Required test data**: | Data Set | Description | Source | Size | |----------|-------------|--------|------| | mock_detections | Pre-defined detection lists per scenario type | Generated fixtures | ~10 KB | | mock_spatial_results | SpatialAnalysisResult objects with waypoints | Generated fixtures | ~5 KB | | mock_vlm_responses | VLMResponse objects for endpoint analysis | Generated fixtures | ~2 KB | | scenario_configs | YAML search scenario configurations (valid + invalid) | Generated fixtures | ~3 KB | **Setup procedure**: 1. Load mock component implementations that return fixture data 2. Initialize BT with test config 3. Set blackboard variables to known state **Teardown procedure**: 1. Shutdown tree 2. Clear blackboard 3. Reset mock call counters **Data isolation strategy**: Each test initializes a fresh BT instance with clean blackboard. No shared mutable state between tests.