detections-semantic/_docs/02_plans/risk_mitigations.md

Risk Assessment — Semantic Detection System — Iteration 01

Risk Scoring Matrix

	Low Impact	Medium Impact	High Impact
High Probability	Medium	High	Critical
Medium Probability	Low	Medium	High
Low Probability	Low	Low	Medium

Risk Register

ID	Risk	Category	Prob	Impact	Score	Mitigation	Status
R01	YOLOE backbone accuracy on aerial concealment data	Technical	Med	High	High	Benchmark sprint; dual backbone; empirical selection	Open
R02	VLM model load latency (5-10s) delays first L2 VLM analysis	Technical	High	Med	High	Predictive loading when first POI queued	Open
R03	V1 heuristic false positive rate overwhelms operator	Technical	High	Med	High	High initial thresholds; per-season tuning; VLM filter	Open
R04	GPU memory pressure during YOLOE→VLM transitions	Technical	Med	High	High	Sequential scheduling; explicit load/unload; memory monitoring	Open
R05	Seasonal model generalization failure	Technical	High	High	Critical	Phased rollout (winter first); config-driven season; continuous data collection	Open
R06	Search scenario config complexity causes runtime errors	Technical	Med	Med	Medium	Config validation at startup; scenario integration tests; good defaults	Open
R07	Path following on fragmented/noisy segmentation masks	Technical	Med	Med	Medium	Aggressive pruning; min-length filter; fallback to area_sweep	Open
R08	ViewLink protocol implementation effort underestimated	Schedule	Med	Med	Medium	Mock mode for parallel dev; allocate extra time; check for community implementations	Open
R09	Operator information overload from too many detections	Technical	Med	Med	Medium	Confidence thresholds; priority ranking; scenario-based filtering	Open
R10	Python GIL blocking in future code additions	Technical	Low	Low	Low	All hot-path compute in C/Cython/TRT (GIL released); document convention	Accepted
R11	NVMe write latency during high L2 recording rate	Technical	Low	Med	Low	3MB/s well within NVMe bandwidth; async writes; drop frames if queue backs up	Accepted
R12	py_trees performance overhead on complex trees	Technical	Low	Low	Low	<1ms measured for ~30 nodes; monitor if tree grows	Accepted
R13	Dynamic search scenarios not extensible enough	Technical	Low	Med	Low	BT architecture allows new subtree types without changing existing code	Accepted

Total: 13 risks — 1 Critical, 4 High, 4 Medium, 4 Low

---

Detailed Risk Analysis

R01: YOLOE Backbone Accuracy on Aerial Concealment Data

Description: Neither YOLO11 nor YOLO26 has been validated on aerial imagery of camouflaged/concealed military positions. YOLO26 has reported accuracy regression on custom datasets (GitHub #23206). Zero-shot YOLOE performance on concealment classes is unknown.

Trigger conditions: mAP50 on validation set < 50% for target classes (footpath, branch_pile, dark_entrance)

Affected components: Tier1Detector, all downstream pipeline

Mitigation strategy:

1. Sprint 1 benchmark: 200 annotated frames, both backbones, same hyperparameters
2. Evaluate zero-shot YOLOE with text/visual prompts first (no training data needed)
3. If both backbones underperform: fall back to standard YOLO with custom training
4. Keep both TRT engines on NVMe; config switch

Contingency plan: If YOLOE open-vocabulary approach fails entirely, train a standard YOLO model from scratch on annotated data (Phase 2 timeline).

Residual risk after mitigation: Medium — benchmark data will be limited initially

Documents updated: architecture.md ADR-003

---

R02: VLM Model Load Latency

Description: NanoLLM VILA1.5-3B takes 5-10s to load. During L1→L2 transition, if VLM is needed for the first time in a session, the investigation is delayed.

Trigger conditions: First POI requiring VLM analysis in a session

Affected components: VLMClient, ScanController

Mitigation strategy:

1. When first POI is queued (even before L2 starts), begin VLM loading in background
2. If VLM not ready when Tier 2 result is ambiguous, proceed with Tier 2 result only
3. Subsequent analyses will have VLM warm

Contingency plan: If load time is unacceptable, keep VLM loaded at startup and accept higher memory usage during L1 (requires verifying YOLOE fits in remaining memory).

Residual risk after mitigation: Low — only first VLM request affected

Documents updated: VLMClient description.md (lifecycle section), ScanController description.md (L2 subtree)

---

R03: V1 Heuristic False Positive Rate

Description: Darkness + contrast heuristic will flag shadows, water puddles, dark soil, tree shade as potential concealed positions. Operator may be overwhelmed.

Trigger conditions: FP rate > 80% during initial field testing

Affected components: Tier2SpatialAnalyzer, OutputManager, operator workflow

Mitigation strategy:

1. Start with conservative thresholds (higher darkness, higher contrast required)
2. Per-season threshold configs (winter/summer/autumn differ significantly)
3. VLM as secondary filter for ambiguous cases (when available)
4. Priority ranking: scenarios with higher confidence bubble up
5. Scenario-based filtering: operator sees scenario name, can mentally filter

Contingency plan: If heuristic is unusable, fast-track a simple binary classifier trained on collected FP/TP data from field testing.

Residual risk after mitigation: Medium — some FP expected and accepted per design

Documents updated: Tier2SpatialAnalyzer description.md, Config helper (per-season thresholds)

---

R04: GPU Memory Pressure During Transitions

Description: YOLOE TRT engine (~2GB GPU) must coexist with VLM (~3GB GPU) in 8GB shared LPDDR5. If both are loaded simultaneously, total exceeds available GPU memory.

Trigger conditions: YOLOE engine stays loaded while VLM loads, or VLM doesn't fully unload

Affected components: Tier1Detector, VLMClient, ScanController

Mitigation strategy:

1. Sequential GPU scheduling: YOLOE processes current frame → VLM loads → VLM analyzes → VLM unloads → YOLOE resumes
2. Explicit unload_model() call before any load_model() for different model
3. Monitor GPU memory via tegrastats in health check; set semantic_available=false if memory exceeds threshold

Contingency plan: If memory management is unreliable, use a smaller VLM (Obsidian-3B at ~1.5GB) that can coexist with YOLOE.

Residual risk after mitigation: Low — sequential scheduling is well-defined

Documents updated: architecture.md §5 (sequential GPU note), VLMClient description.md (lifecycle)

---

R05: Seasonal Model Generalization Failure (CRITICAL)

Description: Models trained on winter imagery will fail in spring/summer/autumn. Footpaths on snow look completely different from footpaths on mud/grass. Branch piles vary by season.

Trigger conditions: Deploying winter-trained model in spring/summer without retraining

Affected components: Tier1Detector, Tier2SpatialAnalyzer, all search scenarios

Mitigation strategy:

1. Phased rollout: winter only for initial release
2. Season config in YAML: season: winter — adjusts thresholds, enables season-specific scenarios
3. Continuous data collection from every flight via frame recorder
4. Season-specific YOLOE classes: footpath_winter, footpath_autumn etc.
5. Retraining pipeline per season (Phase 4 in training strategy)
6. Search scenarios are season-aware (different trigger classes per season)

Contingency plan: If multi-season models don't converge, maintain separate model files per season. Config-switch per deployment.

Residual risk after mitigation: Medium — phased approach manages exposure, but spring/summer models will need real data

Documents updated: Config helper (season field), ScanController (season-aware scenarios), training strategy in solution.md

---

R06: Search Scenario Config Complexity

Description: Data-driven search scenarios add YAML complexity. Invalid configs (missing follow_class for path_follow, unknown investigation type, empty trigger classes) could cause runtime errors or silent failures.

Trigger conditions: Operator modifies config YAML incorrectly

Affected components: ScanController, Config helper

Mitigation strategy:

1. Config validation at startup: reject invalid scenarios with clear error messages
2. Ship with well-tested default scenarios per season
3. Scenario integration tests: verify each investigation type with mock data
4. Unknown investigation type → log error, skip scenario, continue with others

Contingency plan: If config complexity proves too error-prone, build a simple scenario editor tool.

Residual risk after mitigation: Low — validation catches most errors

Documents updated: Config helper (validation rules expanded)

---

R07: Spatial Analysis on Noisy/Sparse Input

Description: For mask tracing: noisy or fragmented segmentation masks produce broken skeletons with spurious branches, leading to erratic path following. For cluster tracing: too few detections visible in a single frame (e.g., wide-area L1 at medium zoom can't resolve small objects), or false positive detections create phantom clusters.

Trigger conditions: YOLOE footpath segmentation has many disconnected components; or cluster_follow scenario triggers on <min_cluster_size detections

Affected components: Tier2SpatialAnalyzer, GimbalDriver (PID receives erratic targets)

Mitigation strategy:

1. Mask trace: morphological closing before skeletonization to connect nearby fragments
2. Mask trace: aggressive pruning of branches shorter than min_branch_length; select longest connected component
3. Mask trace: if skeleton quality too low, fall back to area_sweep
4. Cluster trace: configurable min_cluster_size (default 2) filters noise
5. Cluster trace: cluster_radius_px prevents grouping unrelated detections
6. Cluster trace: if no valid cluster found, return empty result (ScanController falls through to next investigation type or returns to L1)

Contingency plan: Mask trace: use centroid + bounding box direction as rough path direction. Cluster trace: fall back to zoom_classify on individual detections instead of cluster_follow.

Residual risk after mitigation: Low — multiple fallback layers for both strategies

Documents updated: Tier2SpatialAnalyzer description.md (preprocessing, cluster error handling)

---

R08: ViewLink Protocol Implementation Effort

Description: No open-source Python implementation of ViewLink Serial Protocol V3.3.3 exists. Custom implementation requires parsing the PDF specification, handling binary packet format, and testing on real hardware.

Trigger conditions: Implementation takes >2 weeks; edge cases in protocol not documented

Affected components: GimbalDriver

Mitigation strategy:

1. Mock mode (TCP socket) enables parallel development without real hardware
2. ArduPilot has a ViewPro driver in C++ — can reference for packet format
3. Allocate 2 weeks for GimbalDriver implementation + bench testing
4. Start with basic commands (pan/tilt/zoom) before advanced features

Contingency plan: If ViewLink implementation stalls, use MAVLink gimbal protocol via ArduPilot as an intermediary (less control but faster to implement).

Residual risk after mitigation: Low — ArduPilot reference reduces uncertainty

Documents updated: GimbalDriver description.md

---

R09: Operator Information Overload

Description: High FP rate + continuous scanning + multiple active scenarios = many detection candidates. Operator may miss real targets in the noise.

Trigger conditions: >20 detections per minute with FP rate >70%

Affected components: OutputManager (operator delivery), ScanController

Mitigation strategy:

1. Confidence threshold per scenario (configurable)
2. Priority ranking: higher-confidence, VLM-confirmed detections shown first
3. Scenario name in detection output: operator knows context
4. Configurable detection throttle: max N detections per minute to operator

Contingency plan: Add a simple client-side filter in operator display (by scenario, by confidence).

Residual risk after mitigation: Medium — some overload expected initially, tuning required

Documents updated: OutputManager description.md, Config helper

---

R10: Python GIL (Low — Accepted)

Description: Python's Global Interpreter Lock prevents true parallel execution of Python threads.

Why it's not a concern for this system:

1. All compute-heavy operations release the GIL: TensorRT inference (C++ backend), OpenCV (C backend), scikit-image skeletonization (Cython), pyserial I/O (C backend), NVMe file writes (OS-level I/O)
2. VLM runs in a separate Docker process — entirely outside the GIL
3. Architecture is deliberately single-threaded: BT tick loop processes one frame at a time; no need for threading
4. Pure Python in hot path: only py_trees traversal (~<1ms) and dict/list operations
5. I/O operations (UART, NVMe writes) release the GIL natively

Caveat: If future developers add Python-heavy computation in a BT leaf node without using C/Cython, it could block other operations. This is a coding practice issue, not an architectural one.

Mitigation: Document in coding guidelines: "All compute-heavy leaf nodes must use C-extension libraries or Cython. Pure Python processing must complete in <5ms."

Residual risk: Low

---

Architecture/Component Changes Applied (This Iteration)

Risk ID	Document Modified	Change Description
R01	architecture.md ADR-003	Already documented: dual backbone strategy
R02	components/04_vlm_client/description.md	Lifecycle notes: predictive loading when first POI queued
R03	components/03_tier2_spatial_analyzer/description.md	V2 CNN removed; heuristic is permanent Tier 2 approach
R03	architecture.md tech stack	MobileNetV3-Small removed
R04	architecture.md §2, §5	Sequential GPU scheduling documented
R05	common-helpers/01_helper_config.md	Season-specific search scenarios
R05	components/01_scan_controller/description.md	Dynamic search scenarios with season-aware trigger classes
R06	common-helpers/01_helper_config.md	Scenario validation rules expanded
R07	components/03_tier2_spatial_analyzer/description.md	Preprocessing, cluster error handling, and fallback documented
R10	—	No change needed; documented as accepted
—	data_model.md	Fixed: POI.status added "timeout", POI max size config-driven, HealthLogEntry uses capability flags
—	common-helpers/02_helper_types.md	Added SearchScenario struct; POI includes scenario_name and investigation_type

Summary

Total risks identified: 13 Critical: 1 (R05 — seasonal generalization) High: 4 (R01 backbone accuracy, R02 VLM load latency, R03 heuristic FP rate, R04 GPU memory) Medium: 4 (R06 config complexity, R07 fragmented masks, R08 ViewLink effort, R09 operator overload) Low: 4 (R10 GIL, R11 NVMe writes, R12 py_trees overhead, R13 scenario extensibility)

Risks mitigated this iteration: All 13 have mitigation strategies documented Risks requiring user decision: None — all mitigations are actionable without further input