azaion/autopilot
1
0
Fork 0
mirror of https://github.com/azaion/autopilot.git synced 2026-06-21 16:01:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-662] [AZ-669] Implement ego-motion estimator and primitive graph

Browse Source 
AZ-662: movement_detector ego-motion
- Add opencv + petgraph to workspace dependencies
- internal/zoom_bands: per-band telemetry skew tolerances
- internal/telemetry_sync: skew gate (check_skew)
- internal/optical_flow: frame→gray, degenerate detection,
  LK sparse flow + RANSAC homography estimation
- internal/ego_motion: EgoMotionEstimator + atomic counters

AZ-669: semantic_analyzer primitive graph
- internal/primitive_graph: NodeType, PrimitiveNode, PrimitiveGraph,
  PrimitiveGraphBuilder with proximity-adjacency + BFS connectivity check
- internal/scoring/freshness: FreshnessScorer (Laplacian variance,
  texture stddev, undisturbed-surroundings heuristic)
- All ACs covered by unit tests (AC-1/2/3 per task)

Note: native OpenCV not installed on macOS; authoritative test is
cargo test --workspace on Jetson (ssh jetson-e2e).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-20 19:00:39 +03:00
parent 9ed2842c00
commit db844db232
20 changed files with 1546 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_tasks/todo/AZ-662_movement_detector_ego_motion.md
-64
View File
@@ -1,64 +0,0 @@
# Ego-Motion Estimator + Telemetry Sync Gate
**Task**: AZ-662_movement_detector_ego_motion
**Name**: OpenCV optical-flow / global-motion estimator + telemetry-skew gate
**Description**: Compute per-frame ego-motion using OpenCV (Lucas–Kanade optical flow or feature-based homography), refined by the synchronised gimbal + UAV telemetry. Drop frames whose telemetry skew exceeds the per-zoom-band tolerance; never silent.
**Complexity**: 5 points
**Dependencies**: AZ-640_initial_structure, AZ-659_frame_ingest_publisher, AZ-656_gimbal_centre_on_target, AZ-649_mission_executor_telemetry_forwarding
**Component**: movement_detector
**Tracker**: AZ-662
**Epic**: AZ-629
## Problem
Naive frame differencing is rejected — the UAV and gimbal are moving, so most pixel motion is ego-motion. The estimator must (a) recover camera motion from the frame stream and (b) cross-check against telemetry (gimbal + UAV) within a per-zoom-band skew tolerance. Frames whose telemetry skew exceeds the tolerance MUST be dropped (with a counter), never silently consumed — otherwise the compensation is wrong and false positives flood the operator.
## Outcome
- `EgoMotionEstimator::estimate(frame, gimbal_state, uav_telemetry) -> Result<EgoMotion, SkewExceeded>` returns the per-frame ego-motion vector (or homography) refined by telemetry, OR rejects the frame as skewed.
- Per-zoom-band tolerance from config (defaults per `description.md §5`): zoom-out 50 ms frame↔gimbal / 100 ms frame↔UAV; zoom-in 25 ms / 50 ms.
- Health surface: `telemetry_skew_drops_total`, `optical_flow_degenerate_total`, `current_zoom_band`.
## Scope
### Included
- OpenCV bindings (Rust crate `opencv`).
- Optical-flow primary path (dense Lucas–Kanade or feature-based homography — `opencv::video::CalcOpticalFlow*` or `opencv::calib3d::findHomography`).
- Telemetry-skew gate per zoom band.
- Compensation output (the residual-pixel-motion field; downstream task 24 clusters it).
### Excluded
- Cluster persistence + candidate emission (task 24).
- Q14 fallback (task 25).
## Acceptance Criteria
**AC-1: Synthetic pure-pan: residual ≈ 0**
Given a synthetic frame pair where the camera panned by `dx` and the entire scene is static
When `estimate(frame, gimbal_state, uav_telemetry)` runs
Then the returned ego-motion captures `dx` and the residual motion field is ≈ 0 within epsilon.
**AC-2: Telemetry skew above zoom-out tolerance is dropped**
Given a frame whose gimbal-telemetry timestamp differs by 200 ms while `zoom_band = zoomed_out` (tolerance 50 ms)
When `estimate(...)` is called
Then it returns `Err(SkewExceeded)` and `telemetry_skew_drops_total{band="zoomed_out"}` increments by 1.
**AC-3: Optical-flow degenerate is observable**
Given a fully-saturated white frame
When `estimate(...)` runs
Then it returns `Err(OpticalFlowDegenerate)` and `optical_flow_degenerate_total` increments by 1.
## Non-Functional Requirements
**Performance**
- Per-frame ego-motion estimation: ≤30 ms p99 on Jetson Orin Nano (must coexist with Tier 1 + Tier 2 — per `description.md §9`).
**Reliability**
- Drops never silent.
## Runtime Completeness
- **Named capability**: ego-motion estimation using real OpenCV; telemetry-skew gating.
- **Production code that must exist**: real OpenCV optical-flow / homography path; real synchronisation logic.
- **Allowed external stubs**: synthetic frame pairs in tests; pinned `opencv` Rust crate in CI.
- **Unacceptable substitutes**: a fake/stub estimator that always returns "no motion" is unacceptable in production (would mask real movement candidates).
_docs/02_tasks/todo/AZ-669_semantic_analyzer_primitive_graph.md
-64
View File
@@ -1,64 +0,0 @@
# Primitive Graph Builder + Path Freshness Scoring
**Task**: AZ-669_semantic_analyzer_primitive_graph
**Name**: Primitive graph from Tier-1 detections + path-freshness scoring
**Description**: Build a small ROI-scoped primitive graph from Tier-1 detections (path nodes, endpoint nodes, context nodes). Score path freshness using texture, edge clarity, undisturbed-surroundings cues.
**Complexity**: 5 points
**Dependencies**: AZ-640_initial_structure, AZ-660_detection_client_grpc_stream, AZ-661_detection_client_schema_and_health
**Component**: semantic_analyzer
**Tracker**: AZ-669
**Epic**: AZ-630
## Problem
Tier 2 reasons over zoom-in crops using a primitive graph built from Tier-1 detections. The graph captures footpaths (path nodes), branch piles / dark entrances / dugouts (endpoint nodes), and trees / tree-blocks (context nodes). Path-freshness scoring combines surface texture, edge clarity, and undisturbed-surroundings cues into a single freshness score consumed by the recommended-action policy.
## Outcome
- `PrimitiveGraph::build(roi, detections) -> Graph` builds the graph from Tier-1 detections inside the ROI.
- `FreshnessScorer::score(graph, frame_crop) -> PathFreshnessScore` returns a normalized 0–1 score per path node.
- Graph validation: disconnected paths trigger an explicit warning (consumed by task 32).
- Health surface: `graphs_built_total`, `freshness_score_p50/p99`, `disconnected_graphs_total`.
## Scope
### Included
- Graph data structures (path / endpoint / context node types).
- Detection-to-node mapping (per-class).
- Freshness scoring (computer-vision-style: edge density, texture variance, surrounding undisturbed area).
- Graph validation.
### Excluded
- ROI CNN inference (task 31).
- Recommended-action policy (task 32).
- VLM (separate component).
## Acceptance Criteria
**AC-1: Graph contains all relevant detections**
Given a `DetectionBatch` with 3 footpath bboxes + 2 branch-pile bboxes + 5 tree bboxes inside the ROI
When `build(roi, batch)` runs
Then the graph contains 3 path nodes + 2 endpoint nodes + 5 context nodes.
**AC-2: Freshness score is bounded**
Given any valid graph + frame crop
When `score(graph, crop)` runs
Then every emitted freshness score is in `[0.0, 1.0]`.
**AC-3: Disconnected graph is flagged**
Given a graph with two unconnected path components
When validation runs
Then `disconnected_graphs_total` increments by 1 and the graph is marked invalid.
## Non-Functional Requirements
**Performance**
- Graph build: ≤30 ms per ROI on Jetson Orin Nano.
- Freshness scoring: ≤50 ms per ROI.
## Runtime Completeness
- **Named capability**: primitive graph construction + path-freshness scoring — production reasoning path.
- **Production code that must exist**: real graph construction; real freshness scorer.
- **Allowed external stubs**: `opencv` for texture/edge feature extraction.
- **Unacceptable substitutes**: a constant-score scorer in production is unacceptable.