Initial commit

Made-with: Cursor

_docs/02_plans/components/03_tier2_spatial_analyzer/description.md

@@ -0,0 +1,140 @@
+# Tier2SpatialAnalyzer
+
+## 1. High-Level Overview
+
+**Purpose**: Analyzes spatial patterns from Tier 1 detections — both continuous segmentation masks (footpaths) and discrete point clusters (defense systems, vehicle groups). Produces an ordered list of waypoints for the gimbal to follow, with a classification at each waypoint.
+
+**Architectural Pattern**: Stateless processing pipeline with two strategies (mask tracing, cluster tracing) producing a unified output.
+
+**Upstream dependencies**: Config helper, Types helper
+
+**Downstream consumers**: ScanController
+
+## 2. Internal Interfaces
+
+### Interface: Tier2SpatialAnalyzer
+
+| Method | Input | Output | Async | Error Types |
+|--------|-------|--------|-------|-------------|
+| `trace_mask(mask, gsd)` | numpy (H,W) binary mask, float gsd | SpatialAnalysisResult | No | TracingError |
+| `trace_cluster(detections, frame, scenario)` | list[Detection], numpy (H,W,3), SearchScenario | SpatialAnalysisResult | No | ClusterError |
+| `analyze_roi(frame, bbox)` | numpy (H,W,3), bbox tuple | WaypointClassification | No | ClassificationError |
+
+### Strategy: Mask Tracing (footpaths, roads, linear features)
+
+Input: binary segmentation mask from Tier 1
+Algorithm: skeletonize → prune → extract endpoints → classify each endpoint ROI
+Output: waypoints at skeleton endpoints, trajectory along skeleton centerline
+
+### Strategy: Cluster Tracing (AA systems, radar networks, vehicle groups)
+
+Input: list of point detections from Tier 1
+Algorithm: spatial clustering → visit order → per-point ROI classify
+Output: waypoints at each cluster member, trajectory as point-to-point path
+
+### Unified Output: SpatialAnalysisResult
+
+```
+pattern_type: str — "mask_trace" or "cluster_trace"
+waypoints: list[Waypoint] — ordered visit sequence
+trajectory: list[tuple(x, y)] — full gimbal trajectory
+overall_direction: (dx, dy) — for gimbal PID
+skeleton: numpy array (H,W) or None — only for mask_trace
+cluster_bbox: tuple(cx, cy, w, h) or None — bounding box of cluster, only for cluster_trace
+```
+
+### Waypoint
+
+```
+x: int
+y: int
+dx: float — direction vector x component
+dy: float — direction vector y component
+label: str — "concealed_position", "branch_pile", "radar_dish", "unknown", etc.
+confidence: float (0-1)
+freshness_tag: str or None — "high_contrast" / "low_contrast" (mask_trace only)
+roi_thumbnail: numpy array — cropped ROI for logging
+```
+
+## 5. Implementation Details
+
+### Mask Tracing Algorithm (footpaths)
+
+1. Morphological closing to connect nearby mask fragments
+2. Skeletonize mask using Zhang-Suen (scikit-image `skeletonize`)
+3. Prune short branches (< config `min_branch_length` pixels)
+4. Select longest connected skeleton component
+5. Find endpoints via hit-miss morphological operation
+6. For each endpoint: extract ROI (size = config `base_roi_px` * gsd_factor)
+7. Classify each endpoint via `analyze_roi` heuristic
+8. Build trajectory from skeleton pixel coordinates
+9. Compute overall_direction from skeleton start→end vector
+
+### Cluster Tracing Algorithm (discrete objects)
+
+1. Filter detections to scenario's `target_classes`
+2. Compute pairwise distances between detection centers (in pixels)
+3. Group detections within `cluster_radius_px` of each other (union-find on distance graph)
+4. Discard clusters smaller than `min_cluster_size`
+5. For the largest valid cluster: plan visit order via nearest-neighbor greedy traversal from current gimbal position
+6. For each waypoint: extract ROI around detection bbox, run `analyze_roi`
+7. Build trajectory as ordered point-to-point path
+8. Compute overall_direction from first waypoint to last
+9. Compute cluster_bbox as bounding box enclosing all cluster members
+
+### ROI Classification Heuristic (`analyze_roi`)
+
+Shared by both strategies:
+1. Extract ROI from frame at given bbox
+2. Compute: mean_darkness = mean intensity in ROI center 50%
+3. Compute: contrast = (surrounding_mean - center_mean) / surrounding_mean
+4. Compute: freshness_tag based on path-vs-terrain contrast ratio (mask_trace only, None for clusters)
+5. Classify: if darkness < threshold AND contrast > threshold → label from scenario target_classes; else "unknown"
+
+### Key Dependencies
+
+| Library | Version | Purpose |
+|---------|---------|---------|
+| scikit-image | — | Skeletonization (Zhang-Suen), morphology |
+| OpenCV | 4.x | ROI cropping, intensity calculations, morphological closing |
+| numpy | — | Mask and image operations, pairwise distance computation |
+| scipy.spatial | — | Distance matrix for cluster grouping (cdist) |
+
+### Error Handling Strategy
+
+- Empty mask → return empty SpatialAnalysisResult (no waypoints)
+- Skeleton has no endpoints (circular path) → fallback to mask centroid as single waypoint
+- ROI extends beyond frame → clip to frame boundaries
+- No detections match target_classes → return empty SpatialAnalysisResult
+- All clusters smaller than min_cluster_size → return empty SpatialAnalysisResult
+- Single detection (cluster_size=1, below min) → return empty result; single points handled by zoom_classify investigation type instead
+
+## 7. Caveats & Edge Cases
+
+**Known limitations**:
+- Mask heuristic will have high false positive rate (dark shadows, water puddles)
+- Skeletonization on noisy/fragmented masks produces spurious branches (hence pruning + closing)
+- Freshness assessment is contrast metadata, not a reliable classifier
+- Cluster tracing depends on Tier 1 detecting enough cluster members in a single frame — wide-area L1 frames at medium zoom may not resolve small objects
+- Nearest-neighbor visit order is not globally optimal (but adequate for <10 waypoints)
+
+**Performance bottlenecks**:
+- Skeletonization: ~10-20ms for a 1080p mask
+- Pruning + endpoint detection: ~5ms
+- Pairwise distance + clustering: ~1ms for <50 detections
+- Total mask_trace: ~30ms per mask
+- Total cluster_trace: ~5ms per cluster (no skeletonization)
+
+## 8. Dependency Graph
+
+**Must be implemented after**: Config helper, Types helper
+**Can be implemented in parallel with**: Tier1Detector, VLMClient, GimbalDriver, OutputManager
+**Blocks**: ScanController (needs Tier2 for L2 investigation)
+
+## 9. Logging Strategy
+
+| Log Level | When | Example |
+|-----------|------|---------|
+| ERROR | Skeletonization crash, distance computation failure | `Skeleton extraction failed on frame 1234` |
+| WARN | No endpoints found, cluster too small, fallback | `Cluster has 1 member (min=2), skipping` |
+| INFO | Waypoint classified, cluster formed | `mask_trace: 3 waypoints, direction=(0.7, 0.3)` / `cluster_trace: 4 waypoints (aa_launcher×2, radar_dish×2)` |

_docs/02_plans/components/03_tier2_spatial_analyzer/tests.md

@@ -0,0 +1,384 @@
+# Test Specification — Tier2SpatialAnalyzer
+
+## Acceptance Criteria Traceability
+
+| AC ID | Acceptance Criterion | Test IDs | Coverage |
+|-------|---------------------|----------|----------|
+| AC-02 | Tier 2 latency ≤200ms per ROI | PT-01, PT-02 | Covered |
+| AC-06 | Concealed position recall ≥60% | AT-01 | Covered |
+| AC-07 | Concealed position precision ≥20% initial | AT-01 | Covered |
+| AC-08 | Footpath detection recall ≥70% | AT-02 | Covered |
+| AC-23 | Distinguish fresh footpaths from stale ones | IT-03, AT-03 | Covered |
+| AC-24 | Trace footpaths to endpoints, identify concealed structures | IT-01, IT-02, AT-04 | Covered |
+| AC-25 | Handle path intersections by following freshest branch | IT-04 | Covered |
+
+---
+
+## Integration Tests
+
+### IT-01: trace_mask Produces Valid Skeleton and Waypoints
+
+**Summary**: Verify trace_mask correctly skeletonizes a clean binary footpath mask and returns waypoints at endpoints.
+
+**Traces to**: AC-24
+
+**Input data**:
+- Binary mask (1080x1920) with a single continuous footpath shape (L-shaped, ~300px long)
+- gsd=0.15 (meters per pixel)
+
+**Expected result**:
+- SpatialAnalysisResult with pattern_type="mask_trace"
+- waypoints list length ≥ 2 (at least start and end)
+- skeleton is non-None, shape matches input mask
+- trajectory has ≥ 10 points along the skeleton
+- overall_direction is a unit-ish vector (magnitude > 0)
+- Each waypoint has x, y within mask bounds, label is a string, confidence in [0,1]
+
+**Max execution time**: 200ms
+
+**Dependencies**: None (stateless)
+
+---
+
+### IT-02: trace_mask Handles Fragmented Mask
+
+**Summary**: Verify morphological closing connects nearby mask fragments before skeletonization.
+
+**Traces to**: AC-24
+
+**Input data**:
+- Binary mask with 5 disconnected fragments (gaps of ~10px) forming a roughly linear path
+
+**Expected result**:
+- Closing connects fragments into 1-2 components
+- Skeleton follows the general path direction
+- Waypoints at the two extremes of the longest connected component
+- No crash or empty result
+
+**Max execution time**: 200ms
+
+**Dependencies**: None
+
+---
+
+### IT-03: Freshness Tag Assignment
+
+**Summary**: Verify analyze_roi assigns freshness_tag based on path-vs-terrain contrast for mask traces.
+
+**Traces to**: AC-23
+
+**Input data**:
+- Frame with high-contrast footpath on snow (bright terrain, dark path) → expected "high_contrast"
+- Frame with low-contrast footpath on mud (similar intensity) → expected "low_contrast"
+
+**Expected result**:
+- High contrast ROI: freshness_tag="high_contrast"
+- Low contrast ROI: freshness_tag="low_contrast"
+
+**Max execution time**: 50ms per ROI
+
+**Dependencies**: None
+
+---
+
+### IT-04: trace_mask Handles Intersections by Selecting Longest Branch
+
+**Summary**: Verify that when a skeleton has multiple branches (intersection), the longest connected component is selected.
+
+**Traces to**: AC-25
+
+**Input data**:
+- Binary mask forming a T-intersection: main path ~400px, branch ~100px
+
+**Expected result**:
+- Longest skeleton component selected (~400px worth of skeleton)
+- Short branch pruned (< min_branch_length or shorter than main path)
+- Waypoints at the two endpoints of the main path
+
+**Max execution time**: 200ms
+
+**Dependencies**: None
+
+---
+
+### IT-05: trace_cluster Produces Waypoints for Valid Cluster
+
+**Summary**: Verify trace_cluster groups nearby detections and produces ordered waypoints.
+
+**Traces to**: AC-24
+
+**Input data**:
+- 4 detections: labels=["radar_dish", "aa_launcher", "radar_dish", "military_truck"]
+- Centers: (100,100), (200,150), (150,300), (250,250) — all within 300px cluster_radius
+- Scenario: cluster_follow, min_cluster_size=2, cluster_radius_px=300
+
+**Expected result**:
+- SpatialAnalysisResult with pattern_type="cluster_trace"
+- waypoints list length = 4 (one per detection)
+- Waypoints ordered by nearest-neighbor from a starting position
+- cluster_bbox covers all 4 detection centers
+- trajectory connects waypoints in visit order
+
+**Max execution time**: 50ms
+
+**Dependencies**: None
+
+---
+
+### IT-06: trace_cluster Returns Empty for Below-Minimum Cluster
+
+**Summary**: Verify trace_cluster returns empty result when cluster size is below minimum.
+
+**Traces to**: AC-24
+
+**Input data**:
+- 1 detection: {label: "radar_dish", center: (100,100)}
+- Scenario: min_cluster_size=2
+
+**Expected result**:
+- SpatialAnalysisResult with empty waypoints list
+- pattern_type="cluster_trace"
+- No exception
+
+**Max execution time**: 10ms
+
+**Dependencies**: None
+
+---
+
+### IT-07: trace_mask Empty Mask Returns Empty Result
+
+**Summary**: Verify trace_mask handles an all-zero mask gracefully.
+
+**Traces to**: AC-24
+
+**Input data**:
+- All-zero binary mask (1080x1920)
+
+**Expected result**:
+- SpatialAnalysisResult with empty waypoints list
+- skeleton is None or all-zero
+- No exception
+
+**Max execution time**: 50ms
+
+**Dependencies**: None
+
+---
+
+### IT-08: analyze_roi Classifies Dark ROI as Potential Concealment
+
+**Summary**: Verify the darkness + contrast heuristic labels a dark center with bright surrounding as a target class.
+
+**Traces to**: AC-06, AC-07
+
+**Input data**:
+- ROI (100x100) with center 50% at mean intensity 40 (dark), surrounding mean 180 (bright)
+- Config: darkness_threshold=80, contrast_threshold=0.3
+
+**Expected result**:
+- label from scenario's target_classes (not "unknown")
+- confidence > 0
+- Contrast = (180-40)/180 = 0.78 > 0.3 → passes threshold
+
+**Max execution time**: 10ms
+
+**Dependencies**: None
+
+---
+
+### IT-09: analyze_roi Rejects Bright ROI as Unknown
+
+**Summary**: Verify the heuristic does not flag bright, low-contrast regions.
+
+**Traces to**: AC-07
+
+**Input data**:
+- ROI (100x100) with center mean 160, surrounding mean 170
+- Config: darkness_threshold=80, contrast_threshold=0.3
+
+**Expected result**:
+- label="unknown"
+- darkness 160 > threshold 80 → fails darkness check
+
+**Max execution time**: 10ms
+
+**Dependencies**: None
+
+---
+
+## Performance Tests
+
+### PT-01: trace_mask Latency on Full-Resolution Mask
+
+**Summary**: Measure end-to-end trace_mask processing time on 1080p masks.
+
+**Traces to**: AC-02
+
+**Load scenario**:
+- 50 different binary masks (1080x1920), varying complexity
+- Sequential processing
+- Duration: ~5s
+
+**Expected results**:
+
+| Metric | Target | Failure Threshold |
+|--------|--------|-------------------|
+| Latency (p50) | ≤30ms | >200ms |
+| Latency (p95) | ≤100ms | >200ms |
+| Latency (p99) | ≤150ms | >200ms |
+
+**Resource limits**:
+- CPU: ≤50% single core
+- Memory: ≤200MB additional
+
+---
+
+### PT-02: trace_cluster Latency with Many Detections
+
+**Summary**: Measure trace_cluster processing time with increasing detection counts.
+
+**Traces to**: AC-02
+
+**Load scenario**:
+- Detection counts: 10, 20, 50 detections
+- cluster_radius_px=300
+- Sequential processing
+
+**Expected results**:
+
+| Metric | Target | Failure Threshold |
+|--------|--------|-------------------|
+| Latency (10 dets, p95) | ≤5ms | >50ms |
+| Latency (50 dets, p95) | ≤20ms | >200ms |
+
+**Resource limits**:
+- CPU: ≤30% single core
+- Memory: ≤50MB additional
+
+---
+
+## Security Tests
+
+### ST-01: ROI Boundary Clipping Prevents Out-of-Bounds Access
+
+**Summary**: Verify analyze_roi clips ROI to frame boundaries when bbox extends beyond frame edges.
+
+**Traces to**: AC-24
+
+**Attack vector**: Crafted detection with bbox extending beyond frame dimensions
+
+**Test procedure**:
+1. Call analyze_roi with bbox extending 50px beyond frame right edge
+2. Call analyze_roi with bbox at negative coordinates
+
+**Expected behavior**: ROI is clipped to valid frame area; no segfault, no array index error.
+
+**Pass criteria**: Function returns a classification without raising exceptions.
+
+**Fail criteria**: IndexError, segfault, or uncaught exception.
+
+---
+
+## Acceptance Tests
+
+### AT-01: Concealed Position Detection Rate on Validation Set
+
+**Summary**: Verify the heuristic achieves ≥60% recall and ≥20% precision on concealed position ROIs.
+
+**Traces to**: AC-06, AC-07
+
+**Preconditions**:
+- Validation set of 100+ annotated concealment ROIs (positive: actual concealed positions, negative: shadows, puddles, dark soil)
+- Config thresholds set to production defaults
+
+**Steps**:
+
+| Step | Action | Expected Result |
+|------|--------|-----------------|
+| 1 | Run analyze_roi on all positive ROIs | Count TP (label != "unknown") and FN |
+| 2 | Run analyze_roi on all negative ROIs | Count FP (label != "unknown") and TN |
+| 3 | Compute recall = TP/(TP+FN) | ≥ 60% |
+| 4 | Compute precision = TP/(TP+FP) | ≥ 20% |
+
+---
+
+### AT-02: Footpath Endpoint Detection Rate
+
+**Summary**: Verify trace_mask finds endpoints for ≥70% of annotated footpaths.
+
+**Traces to**: AC-08
+
+**Preconditions**:
+- 50+ annotated footpath masks with ground-truth endpoint locations
+
+**Steps**:
+
+| Step | Action | Expected Result |
+|------|--------|-----------------|
+| 1 | Run trace_mask on each mask | SpatialAnalysisResult per mask |
+| 2 | Match waypoints to ground-truth endpoints (within 30px) | Count matches |
+| 3 | Compute recall = matched / total GT endpoints | ≥ 70% |
+
+---
+
+### AT-03: Freshness Discrimination on Seasonal Data
+
+**Summary**: Verify freshness_tag distinguishes fresh high-contrast paths from stale low-contrast ones.
+
+**Traces to**: AC-23
+
+**Preconditions**:
+- 30 annotated ROIs: 15 fresh (high-contrast), 15 stale (low-contrast)
+
+**Steps**:
+
+| Step | Action | Expected Result |
+|------|--------|-----------------|
+| 1 | Run analyze_roi on all 30 ROIs | freshness_tag assigned to each |
+| 2 | Check fresh ROIs tagged "high_contrast" | ≥ 80% correctly tagged |
+| 3 | Check stale ROIs tagged "low_contrast" | ≥ 60% correctly tagged |
+
+---
+
+### AT-04: End-to-End Mask Trace Pipeline
+
+**Summary**: Verify the full pipeline from binary mask to classified waypoints.
+
+**Traces to**: AC-24
+
+**Preconditions**:
+- 10 real footpath masks from field imagery
+- Known endpoint locations
+
+**Steps**:
+
+| Step | Action | Expected Result |
+|------|--------|-----------------|
+| 1 | trace_mask on each mask | SpatialAnalysisResult returned |
+| 2 | Verify waypoints exist at path endpoints | ≥ 70% of endpoints found |
+| 3 | Verify trajectory follows skeleton | Trajectory points lie within 5px of skeleton |
+| 4 | Verify overall_direction matches path orientation | Angle error < 30° |
+
+---
+
+## Test Data Management
+
+**Required test data**:
+
+| Data Set | Description | Source | Size |
+|----------|-------------|--------|------|
+| footpath_masks | 50+ binary footpath masks (1080p) with GT endpoints | Annotated from Tier1 output | ~500 MB |
+| concealment_rois | 100+ ROI crops: positives (actual concealment) + negatives (shadows, puddles) | Annotated field imagery | ~200 MB |
+| freshness_rois | 30 ROI crops: 15 fresh, 15 stale | Annotated field imagery | ~60 MB |
+| cluster_fixtures | Synthetic detection lists for cluster tracing tests | Generated | ~1 KB |
+| fragmented_masks | Masks with deliberate gaps and noise | Generated from real masks | ~100 MB |
+
+**Setup procedure**:
+1. Load test masks/ROIs from fixture directory
+2. Load config with test thresholds
+
+**Teardown procedure**:
+1. No persistent state to clean (stateless component)
+
+**Data isolation strategy**: Each test creates its own input arrays. No shared mutable state.