azaion/gps-denied-desktop
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-desktop.git synced 2026-04-22 22:56:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

add chunking

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-27 03:43:19 +02:00
parent 4f8c18a066
commit 2037870f67
43 changed files with 7041 additions and 4135 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/01_solution/solution.md
+110 -20
View File
@@ -66,12 +66,39 @@ The system utilizes a **Factor Graph Optimization** (using libraries like GTSAM)
2. **Processing:** The factor graph optimizes the trajectory by minimizing the error between these conflicting constraints.
3. **Output:** A smoothed, globally consistent trajectory $(x, y, z, \\text{roll}, \\text{pitch}, \\text{yaw})$ for every image timestamp.
### **3.3 ZeroMQ Background Service Architecture**
### **3.3 Atlas Multi-Map Architecture**
As per the requirement, the system operates as a background service.
ASTRAL-Next implements an **"Atlas" multi-map architecture** where route chunks/fragments are first-class entities, not just recovery mechanisms. This architecture is critical for handling sharp turns (AC-4) and disconnected route segments (AC-5).
* **Communication Pattern:** The service utilizes a REP-REQ (Reply-Request) pattern for control commands (Start/Stop/Reset) and a PUB-SUB (Publish-Subscribe) pattern for the continuous stream of localization results.
* **Concurrency:** Layer 1 runs on a high-priority thread to ensure immediate feedback. Layers 2 and 3 run asynchronously; when a global match is found, the result is injected into the Factor Graph, which then "back-propagates" the correction to previous frames, refining the entire recent trajectory.
**Core Principles:**
- **Chunks are the primary unit of operation**: When tracking is lost (sharp turn, 350m outlier), the system immediately creates a new chunk and continues processing.
- **Proactive chunk creation**: Chunks are created proactively on tracking loss, not reactively after matching failures.
- **Independent chunk processing**: Each chunk has its own subgraph in the factor graph, optimized independently with local consistency.
- **Chunk matching and merging**: Unanchored chunks are matched semantically (aggregate DINOv2 features) and with LiteSAM (with rotation sweeps), then merged into the global trajectory via Sim(3) similarity transformation.
**Chunk Lifecycle:**
1. **Chunk Creation**: On tracking loss, new chunk created immediately in factor graph.
2. **Chunk Building**: Frames processed within chunk using sequential VO, factors added to chunk's subgraph.
3. **Chunk Matching**: When chunk ready (5-20 frames), semantic matching attempted (aggregate DINOv2 descriptor more robust than single-image).
4. **Chunk LiteSAM Matching**: Candidate tiles matched with LiteSAM, rotation sweeps handle unknown orientation from sharp turns.
5. **Chunk Merging**: Successful matches anchor chunks, which are merged into global trajectory via Sim(3) transform (translation, rotation, scale).
**Benefits:**
- System never "fails" - it fragments and continues processing.
- Chunk semantic matching succeeds where single-image matching fails (featureless terrain).
- Multiple chunks can exist simultaneously and be matched/merged asynchronously.
- Reduces user input requests by 50-70% in challenging scenarios.
### **3.4 REST API Background Service Architecture**
As per the requirement, the system operates as a background service exposed via a REST API.
* **Communication Pattern:** The service utilizes **REST API endpoints** (FastAPI) for all control operations and **Server-Sent Events (SSE)** for real-time streaming of localization results. This architecture provides:
* **REST Endpoints:** `POST /flights` (create flight), `GET /flights/{id}` (status), `POST /flights/{id}/images/batch` (upload images), `POST /flights/{id}/user-fix` (human-in-the-loop input)
* **SSE Streaming:** `GET /flights/{id}/stream` provides continuous, real-time updates of frame processing results, refinements, and status changes
* **Standard HTTP/HTTPS:** Enables easy integration with web clients, mobile apps, and existing infrastructure without requiring specialized messaging libraries
* **Concurrency:** Layer 1 runs on a high-priority thread to ensure immediate feedback. Layers 2 and 3 run asynchronously; when a global match is found, the result is injected into the Factor Graph, which then "back-propagates" the correction to previous frames, refining the entire recent trajectory. Results are immediately pushed via SSE to connected clients.
* **Future Enhancement:** For multi-client online SaaS deployments, ZeroMQ (PUB-SUB pattern) can be added as an alternative transport layer to support high-throughput, multi-tenant scenarios with lower latency and better scalability than HTTP-based SSE.
## **4. Layer 1: Robust Sequential Visual Odometry**
@@ -110,6 +137,31 @@ When the UAV executes a sharp turn, resulting in a completely new view (0% overl
3. **In-Flight Retrieval:** When Layer 1 reports a loss of tracking (or periodically), the current UAV image is processed by AnyLoc. The resulting vector is queried against the Faiss index.
4. **Result:** The system retrieves the top-5 most similar satellite tiles. These tiles represent the coarse global location of the UAV (e.g., "You are in Grid Square B7").2
### **5.3 Chunk-Based Processing**
When semantic matching fails on featureless terrain (plain agricultural fields), the system employs chunk-based processing as a more robust recovery strategy.
**Chunk Semantic Matching:**
- **Aggregate DINOv2 Features**: Instead of matching a single image, the system builds a route chunk (5-20 frames) using sequential VO and computes an aggregate DINOv2 descriptor from all chunk images.
- **Robustness**: Aggregate descriptors are more robust to featureless terrain where single-image matching fails. Multiple images provide more context and reduce false matches.
- **Implementation**: DINOv2 descriptors from all chunk images are aggregated (mean, VLAD, or max pooling) and queried against the Faiss index.
**Chunk LiteSAM Matching:**
- **Rotation Sweeps**: When matching chunks, the system rotates the entire chunk to all possible angles (0°, 30°, 60°, ..., 330°) because sharp turns change orientation and previous heading may not be relevant.
- **Aggregate Correspondences**: LiteSAM matches the entire chunk to satellite tiles, aggregating correspondences from multiple images for more robust matching.
- **Sim(3) Transform**: Successful matches provide Sim(3) transformation (translation, rotation, scale) for merging chunks into the global trajectory.
**Normal Operation:**
- Frames are processed within an active chunk context.
- Relative factors are added to the chunk's subgraph (not global graph).
- Chunks are optimized independently for local consistency.
- When chunks are anchored (GPS found), they are merged into the global trajectory.
**Chunk Merging:**
- Chunks are merged using Sim(3) similarity transformation, accounting for translation, rotation, and scale differences.
- This is critical for monocular VO where scale ambiguity exists.
- Merged chunks maintain global consistency while preserving internal consistency.
## **6. Layer 3: Fine-Grained Metric Localization (LiteSAM)**
Retrieving the correct satellite tile (Layer 2) gives a location error of roughly the tile size (e.g., 200 meters). To meet the "60% < 20m" and "80% < 50m" criteria, the system must precisely align the UAV image onto the satellite tile. ASTRAL-Next utilizes **LiteSAM**.
@@ -193,39 +245,77 @@ Meeting the <5 second per frame requirement on an RTX 2060 requires optimizing t
* **TensorRT Compilation:** The ONNX models are then compiled into **TensorRT Engines**. This process performs graph fusion (combining multiple layers into one) and kernel auto-tuning (selecting the fastest GPU instructions for the specific RTX 2060/3070 architecture).26
* **Precision:** The models should be quantized to **FP16** (16-bit floating point). Research shows that FP16 inference on NVIDIA RTX cards offers a 2x-3x speedup with negligible loss in matching accuracy for these specific networks.16
### **9.2 Background Service Architecture (ZeroMQ)**
### **9.2 Background Service Architecture (REST API + SSE)**
The system is encapsulated as a headless service.
The system is encapsulated as a headless service exposed via REST API.
**ZeroMQ Topology:**
**REST API Architecture:**
* **Socket 1 (REP - Port 5555):** Command Interface. Accepts JSON messages:
* {"cmd": "START", "config": {"lat": 48.1, "lon": 37.5}}
* {"cmd": "USER_FIX", "lat": 48.22, "lon": 37.66} (Human-in-the-loop input).
* **Socket 2 (PUB - Port 5556):** Data Stream. Publishes JSON results for every frame:
* {"frame_id": 1024, "gps": [48.123, 37.123], "object_centers": [...], "status": "LOCKED", "confidence": 0.98}.
* **FastAPI Framework:** Modern, high-performance Python web framework with automatic OpenAPI documentation
* **REST Endpoints:**
* `POST /flights` - Create flight with initial configuration (start GPS, camera params, altitude)
* `GET /flights/{flightId}` - Retrieve flight status and waypoints
* `POST /flights/{flightId}/images/batch` - Upload batch of 10-50 images for processing
* `POST /flights/{flightId}/user-fix` - Submit human-in-the-loop GPS anchor when system requests input
* `GET /flights/{flightId}/stream` - SSE stream for real-time frame results
* `DELETE /flights/{flightId}` - Cancel/delete flight
* **SSE Streaming:** Server-Sent Events provide real-time updates:
* Frame processing results: `{"event": "frame_processed", "data": {"frame_id": 1024, "gps": [48.123, 37.123], "confidence": 0.98}}`
* Refinement updates: `{"event": "frame_refined", "data": {"frame_id": 1000, "gps": [48.120, 37.120]}}`
* Status changes: `{"event": "status", "data": {"status": "REQ_INPUT", "message": "User input required"}}`
Asynchronous Pipeline:
The system utilizes a Python multiprocessing architecture. One process handles the camera/image ingest and ZeroMQ communication. A second process hosts the TensorRT engines and runs the Factor Graph. This ensures that the heavy computation of Bundle Adjustment does not block the receipt of new images or user commands.
**Asynchronous Pipeline:**
The system utilizes a Python multiprocessing architecture. One process handles the REST API server and SSE streaming. A second process hosts the TensorRT engines and runs the Factor Graph. This ensures that the heavy computation of Bundle Adjustment does not block the receipt of new images or user commands. Results are immediately pushed to connected SSE clients.
**Future Multi-Client SaaS Enhancement:**
For production deployments requiring multiple concurrent clients and higher throughput, ZeroMQ can be added as an alternative transport layer:
* **ZeroMQ PUB-SUB:** For high-frequency result streaming to multiple subscribers
* **ZeroMQ REQ-REP:** For low-latency command/response patterns
* **Hybrid Approach:** REST API for control operations, ZeroMQ for data streaming in multi-tenant scenarios
## **10. Human-in-the-Loop Strategy**
The requirement stipulates that for the "20% of the route" where automation fails, the user must intervene. The system must proactively detect its own failure.
### **10.1 Failure Detection with PDM@K**
### **10.1 Failure Detection and Recovery Stages**
The system monitors the **PDM@K** (Positioning Distance Measurement) metric continuously.
* **Definition:** PDM@K measures the percentage of queries localized within $K$ meters.3
* **Real-Time Proxy:** In flight, we cannot know the true PDM (as we don't have ground truth). Instead, we use the **Marginal Covariance** from the Factor Graph. If the uncertainty ellipse for the current position grows larger than a radius of 50 meters, or if the **Image Registration Rate** (percentage of inliers in LightGlue/LiteSAM) drops below 10% for 3 consecutive frames, the system triggers a **Critical Failure Mode**.19
**Recovery Stages:**
1. **Stage 1: Progressive Tile Search (Single Image)**
- Attempts single-image semantic matching (DINOv2) and LiteSAM matching.
- Progressive tile grid expansion (1→4→9→16→25 tiles).
- Fast recovery for transient tracking loss.
2. **Stage 2: Chunk Building and Semantic Matching (Proactive)**
- **Immediately creates new chunk** when tracking lost (proactive, not reactive).
- Continues processing frames, building chunk with sequential VO.
- When chunk ready (5-20 frames), attempts chunk semantic matching.
- Aggregate DINOv2 descriptor more robust than single-image matching.
- Handles featureless terrain where single-image matching fails.
3. **Stage 3: Chunk LiteSAM Matching with Rotation Sweeps**
- After chunk semantic matching succeeds, attempts LiteSAM matching.
- Rotates entire chunk to all angles (0°, 30°, ..., 330°) for matching.
- Critical for sharp turns where orientation unknown.
- Aggregate correspondences from multiple images for robustness.
4. **Stage 4: User Input (Last Resort)**
- Only triggered if all chunk matching strategies fail.
- System requests user-provided GPS anchor.
- User anchor applied as hard constraint, processing resumes.
### **10.2 The User Interaction Workflow**
1. **Trigger:** Critical Failure Mode activated.
2. **Action:** The Service publishes a status {"status": "REQ_INPUT"} via ZeroMQ.
3. **Data Payload:** It sends the current UAV image and the top-3 retrieved satellite tiles (from Layer 2) to the client UI.
4. **User Input:** The user clicks a distinctive feature (e.g., a specific crossroad) in the UAV image and the corresponding point on the satellite map.
5. **Recovery:** This pair of points is treated as a **Hard Constraint** in the Factor Graph. The optimizer immediately snaps the trajectory to this user-defined anchor, resetting the covariance and effectively "healing" the localized track.19
2. **Action:** The Service sends an SSE event `{"event": "user_input_needed", "data": {"status": "REQ_INPUT", "frame_id": 1024}}` to connected clients.
3. **Data Payload:** The client retrieves the current UAV image and top-3 retrieved satellite tiles via `GET /flights/{flightId}/frames/{frameId}/context` endpoint.
4. **User Input:** The user clicks a distinctive feature (e.g., a specific crossroad) in the UAV image and the corresponding point on the satellite map, then submits via `POST /flights/{flightId}/user-fix` with the GPS coordinate.
5. **Recovery:** This GPS coordinate is treated as a **Hard Constraint** in the Factor Graph. The optimizer immediately snaps the trajectory to this user-defined anchor, resetting the covariance and effectively "healing" the localized track. An SSE event confirms the recovery: `{"event": "user_fix_applied", "data": {"frame_id": 1024, "status": "PROCESSING"}}`.19
## **11. Performance Evaluation and Benchmarks**
@@ -267,7 +357,7 @@ A comprehensive test plan is required to validate compliance with all 10 Accepta
| **AC-9** | Image Registration Rate > 95% | V-SLAM (C-3) | **"Atlas" Multi-Map** (4.2). A "lost track" (AC-4) is *not* a registration failure; it's a *new map registration*. This ensures the rate > 95%. |
| **AC-10** | Mean Reprojection Error (MRE) < 1.0px | V-SLAM (C-3) + TOH (C-6) | Local BA (4.3) + Global BA (TOH14) + **Per-Keyframe Scale** (6.2) minimizes internal graph tension (Flaw 1.3), allowing the optimizer to converge to a low MRE. |
### **8.1 Rigorous Validation Methodology**
### **12.1 Rigorous Validation Methodology**
* **Test Harness:** A validation script will be created to compare the system's Pose_N^{Refined} output against a ground-truth coordinates.csv file, computing Haversine distance errors.
* **Test Datasets:**
docs/02_components/00_decomposition_plan.md
-364
View File
@@ -1,364 +0,0 @@
<!-- 31098ee5-58fb-474a-815e-fd9cbd17c063 9f609f9e-c80d-4c88-b618-3135b96a8333 -->
# ASTRAL-Next System Component Decomposition Plan
## Design Principle: Interface-Based Architecture
**CRITICAL REQUIREMENT**: Each component MUST implement a well-defined interface to ensure interchangeability with different implementations.
**Benefits**:
- Swap implementations (e.g., replace LiteSAM with TransFG, GTSAM with Ceres)
- Enable unit testing with mocks
- Support multiple backends (TensorRT vs ONNX, different databases)
- Facilitate future enhancements without breaking contracts
**Interface Specification**: Each component spec must define:
- Interface name (e.g., `ISatelliteDataManager`, `IMetricRefinement`)
- All public methods with strict contracts
- Input/output data structures
- Error conditions and exceptions
- Performance guarantees
---
## System Architecture Overview
**Two separate REST APIs in same repository:**
### Route API (Separate Project)
- Route/waypoint/geofence CRUD
- Shared by GPS-Denied and Mission Planner
- Does NOT call satellite provider
### GPS-Denied API (Main System)
- Tri-layer localization (SuperPoint+LightGlue, AnyLoc, LiteSAM)
- Calls satellite provider for tiles
- Rotation preprocessing (LiteSAM 45° limit)
- Per-frame Route API updates
- Progressive tile search (1→4→9→16→25)
---
## ROUTE API COMPONENTS (4 components)
### R01_route_rest_api
**Interface**: `IRouteRestAPI`
**Endpoints**: `POST /routes`, `GET /routes/{routeId}`, `PUT /routes/{routeId}/waypoints`, `DELETE /routes/{routeId}`
### R02_route_data_manager
**Interface**: `IRouteDataManager`
**API**: `save_route()`, `load_route()`, `update_waypoint()`, `delete_waypoint()`, `get_route_metadata()`
### R03_waypoint_validator
**Interface**: `IWaypointValidator`
**API**: `validate_waypoint()`, `validate_geofence()`, `check_bounds()`, `validate_route_continuity()`
### R04_route_database_layer
**Interface**: `IRouteDatabase`
**API**: `insert_route()`, `update_route()`, `query_routes()`, `get_waypoints()`
---
## GPS-DENIED API COMPONENTS (17 components)
### Core REST API Layer
**G01_gps_denied_rest_api**
**Interface**: `IGPSDeniedRestAPI`
**Endpoints**: `POST /gps-denied/flights`, `POST .../images/batch`, `POST .../user-fix`, `GET .../status`, `GET .../stream`
**G02_flight_manager**
**Interface**: `IFlightManager`
**API**: `create_flight()`, `get_flight_state()`, `link_to_route()`, `update_flight_status()`
**G03_route_api_client**
**Interface**: `IRouteAPIClient`
**API**: `update_route_waypoint()`, `get_route_info()`, `batch_update_waypoints()`
### Data Management
**G04_satellite_data_manager**
**Interface**: `ISatelliteDataManager`
**API**: `fetch_tile()`, `fetch_tile_grid()`, `prefetch_route_corridor()`, `progressive_fetch()`, `cache_tile()`, `get_cached_tile()`, `compute_tile_coords()`, `expand_search_grid()`
**Features**: Progressive retrieval, tile caching, grid calculations
**G05_image_input_pipeline**
**Interface**: `IImageInputPipeline`
**API**: `queue_batch()`, `process_next_batch()`, `validate_batch()`, `store_images()`, `get_next_image()`, `get_image_by_sequence()`
**Features**: FIFO queuing, validation, storage
**G06_image_rotation_manager**
**Interface**: `IImageRotationManager`
**API**: `rotate_image_360()`, `try_rotation_steps()`, `calculate_precise_angle()`, `get_current_heading()`, `update_heading()`, `detect_sharp_turn()`, `requires_rotation_sweep()`
**Features**: 30° rotation sweeps, heading tracking
### Visual Processing
**G07_sequential_visual_odometry**
**Interface**: `ISequentialVO`
**API**: `compute_relative_pose()`, `extract_features()`, `match_features()`, `estimate_motion()`
**G08_global_place_recognition**
**Interface**: `IGlobalPlaceRecognition`
**API**: `retrieve_candidate_tiles()`, `compute_location_descriptor()`, `query_database()`, `rank_candidates()`
**G09_metric_refinement**
**Interface**: `IMetricRefinement`
**API**: `align_to_satellite()`, `compute_homography()`, `extract_gps_from_alignment()`, `compute_match_confidence()`
### State Estimation
**G10_factor_graph_optimizer**
**Interface**: `IFactorGraphOptimizer`
**API**: `add_relative_factor()`, `add_absolute_factor()`, `add_altitude_prior()`, `optimize()`, `get_trajectory()`
**G11_failure_recovery_coordinator**
**Interface**: `IFailureRecoveryCoordinator`
**API**: `check_confidence()`, `detect_tracking_loss()`, `start_search()`, `expand_search_radius()`, `try_current_grid()`, `create_user_input_request()`, `apply_user_anchor()`
**G12_coordinate_transformer**
**Interface**: `ICoordinateTransformer`
**API**: `pixel_to_gps()`, `gps_to_pixel()`, `image_object_to_gps()`, `compute_gsd()`, `transform_points()`
### Results & Communication
**G13_result_manager**
**Interface**: `IResultManager`
**API**: `update_frame_result()`, `publish_to_route_api()`, `get_flight_results()`, `mark_refined()`
**G14_sse_event_streamer**
**Interface**: `ISSEEventStreamer`
**API**: `create_stream()`, `send_frame_result()`, `send_search_progress()`, `send_user_input_request()`, `send_refinement()`
### Infrastructure
**G15_model_manager**
**Interface**: `IModelManager`
**API**: `load_model()`, `get_inference_engine()`, `optimize_to_tensorrt()`, `fallback_to_onnx()`
**G16_configuration_manager**
**Interface**: `IConfigurationManager`
**API**: `load_config()`, `get_camera_params()`, `validate_config()`, `get_flight_config()`
**G17_gps_denied_database_layer**
**Interface**: `IGPSDeniedDatabase`
**API**: `save_flight_state()`, `load_flight_state()`, `query_processing_history()`
---
## HELPER COMPONENTS (8 components)
**H01_camera_model** - `ICameraModel`
**H02_gsd_calculator** - `IGSDCalculator`
**H03_robust_kernels** - `IRobustKernels`
**H04_faiss_index_manager** - `IFaissIndexManager`
**H05_performance_monitor** - `IPerformanceMonitor`
**H06_web_mercator_utils** - `IWebMercatorUtils`
**H07_image_rotation_utils** - `IImageRotationUtils`
**H08_batch_validator** - `IBatchValidator`
---
## Comprehensive Component Interaction Matrix
### System Initialization
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G02 | G15 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| G02 | G16 | `load_config()` | Load system configuration |
| G04 | G08 | Satellite tiles | G08 generates descriptors for Faiss |
| G08 | H04 | `build_index()` | Build satellite descriptor index |
| G08 | G15 | `get_inference_engine("DINOv2")` | Get model for descriptor generation |
### Flight Creation
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | G01 | `POST /gps-denied/flights` | Create flight |
| G01 | G02 | `create_flight()` | Initialize flight state |
| G02 | G16 | `get_flight_config()` | Get camera params, altitude |
| G02 | G03 | `get_route_info()` | Fetch route metadata |
| G03 | Route API | `GET /routes/{routeId}` | HTTP call |
| G02 | G04 | `prefetch_route_corridor()` | Prefetch tiles |
| G04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download |
| G04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations |
| G02 | G17 | `save_flight_state()` | Persist flight metadata |
| Client | G01 | `GET .../stream` | Open SSE connection |
| G01 | G14 | `create_stream()` | Establish SSE channel |
### Image Upload
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | G01 | `POST .../images/batch` | Upload 10-50 images |
| G01 | G05 | `queue_batch()` | Queue for processing |
| G05 | H08 | `validate_batch()` | Validate sequence, format |
| G05 | G17 | `store_images()` | Persist images |
### Per-Frame Processing (First Frame / Sharp Turn)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G05 | G06 | `get_next_image()` | Get image for processing |
| G06 | G06 | `requires_rotation_sweep()` | Check if sweep needed |
| G06 | H07 | `rotate_image()` × 12 | Rotate in 30° steps |
| G06 | G09 | `align_to_satellite()` × 12 | Try LiteSAM each rotation |
| G09 | G04 | `get_cached_tile()` | Get expected tile |
| G09 | G15 | `get_inference_engine("LiteSAM")` | Get model |
| G06 | H07 | `calculate_rotation_from_points()` | Precise angle from homography |
| G06 | Internal | `update_heading()` | Store UAV heading |
### Per-Frame Processing (Sequential VO)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G05 | G07 | `get_next_image()` | Provide image |
| G07 | G15 | `get_inference_engine("SuperPoint")` | Get feature extractor |
| G07 | G15 | `get_inference_engine("LightGlue")` | Get matcher |
| G07 | H05 | `start_timer()`, `end_timer()` | Monitor timing |
| G07 | G10 | `add_relative_factor()` | Add pose measurement |
### Tracking Good (Drift Correction)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G07 | G11 | `check_confidence()` | Check tracking quality |
| G11 | G09 | `align_to_satellite()` | Align to 1 tile |
| G09 | G04 | `get_tile_grid(1)` | Get single tile |
| G09 | G10 | `add_absolute_factor()` | Add GPS measurement |
### Tracking Lost (Progressive Search)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G07 | G11 | `check_confidence()` → FAIL | Low confidence |
| G11 | G06 | `requires_rotation_sweep()` | Trigger rotation sweep |
| G11 | G08 | `retrieve_candidate_tiles()` | Coarse localization |
| G08 | G15 | `get_inference_engine("DINOv2")` | Get model |
| G08 | H04 | `search()` | Query Faiss index |
| G08 | G04 | `get_tile_by_gps()` × 5 | Get candidate tiles |
| G11 | G04 | `expand_search_grid(4)` | Get 2×2 grid |
| G11 | G09 | `align_to_satellite()` | Try LiteSAM on 4 tiles |
| G11 (fail) | G04 | `expand_search_grid(9)` | Expand to 3×3 |
| G11 (fail) | G04 | `expand_search_grid(16)` | Expand to 4×4 |
| G11 (fail) | G04 | `expand_search_grid(25)` | Expand to 5×5 |
| G11 (fail) | G14 | `send_user_input_request()` | Request human help |
| G11 | G02 | `update_flight_status("BLOCKED")` | Block processing |
### Optimization & Results
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G10 | H03 | `huber_loss()`, `cauchy_loss()` | Apply robust kernels |
| G10 | Internal | `optimize()` | Run iSAM2 optimization |
| G10 | G12 | `get_trajectory()` | Get optimized poses |
| G12 | H01 | `project()`, `unproject()` | Camera operations |
| G12 | H02 | `compute_gsd()` | GSD calculations |
| G12 | H06 | `tile_to_latlon()` | Coordinate transforms |
| G12 | G13 | Frame GPS + object coords | Provide results |
| G13 | G03 | `update_route_waypoint()` | Per-frame Route API update |
| G03 | Route API | `PUT /routes/.../waypoints/...` | HTTP call |
| G13 | G14 | `send_frame_result()` | Publish to client |
| G14 | Client | SSE `frame_processed` | Real-time delivery |
| G13 | G17 | `save_flight_state()` | Persist state |
### User Input Recovery
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G14 | Client | SSE `user_input_needed` | Notify client |
| Client | G01 | `POST .../user-fix` | Provide anchor |
| G01 | G11 | `apply_user_anchor()` | Apply fix |
| G11 | G10 | `add_absolute_factor()` (high confidence) | Hard constraint |
| G10 | Internal | `optimize()` | Re-optimize |
| G11 | G02 | `update_flight_status("PROCESSING")` | Resume |
### Asynchronous Refinement
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G10 | Internal (background) | `optimize()` | Back-propagate anchors |
| G10 | G13 | `get_trajectory()` | Get refined poses |
| G13 | G03 | `batch_update_waypoints()` | Batch update Route API |
| G13 | G14 | `send_refinement()` × N | Send updates |
| G14 | Client | SSE `frame_refined` × N | Incremental updates |
### Cross-Cutting Concerns
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G16 | ALL | `get_*_config()` | Provide configuration |
| H05 | G07, G08, G09, G10, G11 | `start_timer()`, `end_timer()` | Performance monitoring |
---
## Interaction Coverage Verification
✅ **Initialization**: G02→G15, G16, G17; G04→G08→H04
✅ **Flight creation**: Client→G01→G02→G03,G04,G16,G17,G14
✅ **Image upload**: Client→G01→G05→H08,G17
✅ **Rotation sweep**: G06→H07,G09 (12 iterations)
✅ **Sequential VO**: G07→G15,G10,H05
✅ **Drift correction**: G11→G09→G04(1),G10
✅ **Tracking loss**: G11→G06,G08,G04(progressive),G09,G14,G02
✅ **Global PR**: G08→G15,H04,G04
✅ **Optimization**: G10→H03,G12
✅ **Coordinate transform**: G12→H01,H02,H06
✅ **Results**: G12→G13→G03,G14,G17
✅ **User input**: Client→G01→G11→G10,G02
✅ **Refinement**: G10→G13→G03,G14
✅ **Configuration**: G16→ALL
✅ **Performance**: H05→processing components
**All major component interactions are covered.**
---
## Deliverables
**Component Count**: 29 total
- Route API: 4 (R01-R04)
- GPS-Denied API: 17 (G01-G17)
- Helpers: 8 (H01-H08)
**For each component**, create `docs/02_components/[project]_[##]_[component_name]/[component_name]_spec.md`:
1. **Interface Definition** (interface name, methods, contracts)
2. **Component Description** (responsibilities, scope)
3. **API Methods** (inputs, outputs, errors, which components call it, test cases)
4. **Integration Tests**
5. **Non-Functional Requirements** (performance, accuracy targets)
6. **Dependencies** (which components it calls)
7. **Data Models**
**Generate draw.io diagram** showing:
- Two API projects (Route API, GPS-Denied API)
- All 29 components
- Route API ↔ GPS-Denied API communication
- GPS-Denied → Satellite Provider calls
- Rotation preprocessing flow
- Progressive search expansion (1→4→9→16→25)
- Per-frame Route API update flow
- Helper component usage
### To-dos
- [x] Create 4 Route API specs with interfaces (REST, data manager, validator, DB)
- [x] Create GPS-Denied core API specs with interfaces (REST, flight manager, Route client)
- [x] Create data management specs with interfaces (satellite, image pipeline, rotation)
- [x] Create visual processing specs with interfaces (VO, place recognition, LiteSAM)
- [x] Create coordination specs with interfaces (factor graph, failure recovery, transformer)
- [x] Create results/infrastructure specs with interfaces (result manager, SSE, models, config, DB)
- [x] Create 8 helper specs with interfaces
- [x] Generate draw.io with all components, interactions, flows
docs/02_components/01_flight_api/flight_api_spec.md
+649
View File
@@ -0,0 +1,649 @@
# Flight API
## Interface Definition
**Interface Name**: `IFlightAPI`
### Interface Methods
```python
class IFlightAPI(ABC):
@abstractmethod
def create_flight(self, flight_data: FlightCreateRequest) -> FlightResponse:
pass
@abstractmethod
def get_flight(self, flight_id: str) -> FlightDetailResponse:
pass
@abstractmethod
def delete_flight(self, flight_id: str) -> DeleteResponse:
pass
@abstractmethod
def update_waypoint(self, flight_id: str, waypoint_id: str, waypoint: Waypoint) -> UpdateResponse:
pass
@abstractmethod
def batch_update_waypoints(self, flight_id: str, waypoints: List[Waypoint]) -> BatchUpdateResponse:
pass
@abstractmethod
def upload_image_batch(self, flight_id: str, batch: ImageBatch) -> BatchResponse:
pass
@abstractmethod
def submit_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResponse:
pass
@abstractmethod
def get_flight_status(self, flight_id: str) -> FlightStatusResponse:
pass
@abstractmethod
def create_sse_stream(self, flight_id: str) -> SSEStream:
pass
```
## Component Description
### Responsibilities
- Expose REST API endpoints for complete flight lifecycle management
- Handle flight CRUD operations (create, read, update, delete)
- Manage waypoints and geofences within flights
- Handle satellite data prefetching on flight creation
- Accept batch image uploads (10-50 images per request)
- Accept user-provided GPS fixes for blocked flights
- Provide real-time status updates
- Stream results via Server-Sent Events (SSE)
### Scope
- FastAPI-based REST endpoints
- Request/response validation
- Coordinate with Flight Processor for all operations
- Multipart form data handling for image uploads
- SSE connection management
- Authentication and rate limiting
---
## Flight Management Endpoints
### `create_flight(flight_data: FlightCreateRequest) -> FlightResponse`
**REST Endpoint**: `POST /flights`
**Description**: Creates a new flight with initial waypoints, geofences, camera parameters, and triggers satellite data prefetching.
**Called By**:
- Client applications (Flight UI, Mission Planner UI)
**Input**:
```python
FlightCreateRequest:
name: str
description: str
start_gps: GPSPoint
rough_waypoints: List[GPSPoint]
geofences: Geofences
camera_params: CameraParameters
altitude: float
```
**Output**:
```python
FlightResponse:
flight_id: str
status: str # "prefetching", "ready", "error"
message: Optional[str]
created_at: datetime
```
**Processing Flow**:
1. Validate request data
2. Call F02 Flight Processor → create_flight()
3. Flight Processor triggers satellite prefetch
4. Return flight_id immediately (prefetch is async)
**Error Conditions**:
- `400 Bad Request`: Invalid input data (missing required fields, invalid GPS coordinates)
- `409 Conflict`: Flight with same ID already exists
- `500 Internal Server Error`: Database or internal error
**Test Cases**:
1. **Valid flight creation**: Provide valid flight data → returns 201 with flight_id
2. **Missing required field**: Omit name → returns 400 with error message
3. **Invalid GPS coordinates**: Provide lat > 90 → returns 400
4. **Concurrent flight creation**: Multiple flights → all succeed
---
### `get_flight(flight_id: str) -> FlightDetailResponse`
**REST Endpoint**: `GET /flights/{flightId}`
**Description**: Retrieves complete flight information including all waypoints, geofences, and processing status.
**Called By**:
- Client applications
**Input**:
```python
flight_id: str
```
**Output**:
```python
FlightDetailResponse:
flight_id: str
name: str
description: str
start_gps: GPSPoint
waypoints: List[Waypoint]
geofences: Geofences
camera_params: CameraParameters
altitude: float
status: str
frames_processed: int
frames_total: int
created_at: datetime
updated_at: datetime
```
**Error Conditions**:
- `404 Not Found`: Flight ID does not exist
- `500 Internal Server Error`: Database error
**Test Cases**:
1. **Existing flight**: Valid flightId → returns 200 with complete flight data
2. **Non-existent flight**: Invalid flightId → returns 404
3. **Flight with many waypoints**: Flight with 2000+ waypoints → returns 200 with all data
---
### `delete_flight(flight_id: str) -> DeleteResponse`
**REST Endpoint**: `DELETE /flights/{flightId}`
**Description**: Deletes a flight and all associated waypoints, images, and processing data.
**Called By**:
- Client applications
**Input**:
```python
flight_id: str
```
**Output**:
```python
DeleteResponse:
deleted: bool
flight_id: str
```
**Error Conditions**:
- `404 Not Found`: Flight does not exist
- `409 Conflict`: Flight is currently being processed
- `500 Internal Server Error`: Database error
**Test Cases**:
1. **Delete existing flight**: Valid flightId → returns 200
2. **Delete non-existent flight**: Invalid flightId → returns 404
3. **Delete processing flight**: Active processing → returns 409
---
### `update_waypoint(flight_id: str, waypoint_id: str, waypoint: Waypoint) -> UpdateResponse`
**REST Endpoint**: `PUT /flights/{flightId}/waypoints/{waypointId}`
**Description**: Updates a specific waypoint within a flight. Used for per-frame GPS refinement.
**Called By**:
- Internal (F13 Result Manager for per-frame updates)
- Client applications (manual corrections)
**Input**:
```python
flight_id: str
waypoint_id: str
waypoint: Waypoint:
lat: float
lon: float
altitude: Optional[float]
confidence: float
timestamp: datetime
refined: bool
```
**Output**:
```python
UpdateResponse:
updated: bool
waypoint_id: str
```
**Error Conditions**:
- `404 Not Found`: Flight or waypoint not found
- `400 Bad Request`: Invalid waypoint data
- `500 Internal Server Error`: Database error
**Test Cases**:
1. **Update existing waypoint**: Valid data → returns 200
2. **Refinement update**: Refined coordinates → updates successfully
3. **Invalid coordinates**: lat > 90 → returns 400
4. **Non-existent waypoint**: Invalid waypoint_id → returns 404
---
### `batch_update_waypoints(flight_id: str, waypoints: List[Waypoint]) -> BatchUpdateResponse`
**REST Endpoint**: `PUT /flights/{flightId}/waypoints/batch`
**Description**: Updates multiple waypoints in a single request. Used for trajectory refinements.
**Called By**:
- Internal (F13 Result Manager for asynchronous refinement updates)
**Input**:
```python
flight_id: str
waypoints: List[Waypoint]
```
**Output**:
```python
BatchUpdateResponse:
success: bool
updated_count: int
failed_ids: List[str]
```
**Error Conditions**:
- `404 Not Found`: Flight not found
- `400 Bad Request`: Invalid waypoint data
- `500 Internal Server Error`: Database error
**Test Cases**:
1. **Batch update 100 waypoints**: All succeed
2. **Partial failure**: 5 waypoints fail → returns failed_ids
3. **Empty batch**: Returns success=True, updated_count=0
4. **Large batch**: 500 waypoints → succeeds
---
## Image Processing Endpoints
### `upload_image_batch(flight_id: str, batch: ImageBatch) -> BatchResponse`
**REST Endpoint**: `POST /flights/{flightId}/images/batch`
**Description**: Uploads a batch of 10-50 UAV images for processing.
**Called By**:
- Client applications
**Input**:
```python
flight_id: str
ImageBatch: multipart/form-data
images: List[UploadFile]
metadata: BatchMetadata
start_sequence: int
end_sequence: int
```
**Output**:
```python
BatchResponse:
accepted: bool
sequences: List[int]
next_expected: int
message: Optional[str]
```
**Processing Flow**:
1. Validate flight_id exists
2. Validate batch size (10-50 images)
3. Validate sequence numbers (strict sequential)
4. Pass to F05 Image Input Pipeline
5. Return immediately (processing is async)
**Error Conditions**:
- `400 Bad Request`: Invalid batch size, out-of-sequence images
- `404 Not Found`: flight_id doesn't exist
- `413 Payload Too Large`: Batch exceeds size limit
- `429 Too Many Requests`: Rate limit exceeded
**Test Cases**:
1. **Valid batch upload**: 20 images → returns 202 Accepted
2. **Out-of-sequence batch**: Sequence gap detected → returns 400
3. **Too many images**: 60 images → returns 400
4. **Large images**: 50 × 8MB images → successfully uploads
---
### `submit_user_fix(flight_id: str, fix_data: UserFixRequest) -> UserFixResponse`
**REST Endpoint**: `POST /flights/{flightId}/user-fix`
**Description**: Submits user-provided GPS anchor point to unblock failed localization.
**Called By**:
- Client applications (when user responds to `user_input_needed` event)
**Input**:
```python
UserFixRequest:
frame_id: int
uav_pixel: Tuple[float, float]
satellite_gps: GPSPoint
```
**Output**:
```python
UserFixResponse:
accepted: bool
processing_resumed: bool
message: Optional[str]
```
**Processing Flow**:
1. Validate flight_id exists and is blocked
2. Pass to F11 Failure Recovery Coordinator
3. Coordinator applies anchor to Factor Graph
4. Resume processing pipeline
**Error Conditions**:
- `400 Bad Request`: Invalid fix data
- `404 Not Found`: flight_id or frame_id not found
- `409 Conflict`: Flight not in blocked state
**Test Cases**:
1. **Valid user fix**: Blocked flight → returns 200, processing resumes
2. **Fix for non-blocked flight**: Returns 409
3. **Invalid GPS coordinates**: Returns 400
---
### `get_flight_status(flight_id: str) -> FlightStatusResponse`
**REST Endpoint**: `GET /flights/{flightId}/status`
**Description**: Retrieves current processing status of a flight.
**Called By**:
- Client applications (polling for status)
**Input**:
```python
flight_id: str
```
**Output**:
```python
FlightStatusResponse:
status: str # "prefetching", "ready", "processing", "blocked", "completed", "failed"
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
message: Optional[str]
created_at: datetime
updated_at: datetime
```
**Error Conditions**:
- `404 Not Found`: flight_id doesn't exist
**Test Cases**:
1. **Processing flight**: Returns current progress
2. **Blocked flight**: Returns blocked=true with search_grid_size
3. **Completed flight**: Returns status="completed" with final counts
---
### `create_sse_stream(flight_id: str) -> SSEStream`
**REST Endpoint**: `GET /flights/{flightId}/stream`
**Description**: Opens Server-Sent Events connection for real-time result streaming.
**Called By**:
- Client applications
**Input**:
```python
flight_id: str
```
**Output**:
```python
SSE Stream with events:
- frame_processed
- frame_refined
- search_expanded
- user_input_needed
- processing_blocked
- flight_completed
```
**Event Format**:
```json
{
"event": "frame_processed",
"data": {
"frame_id": 237,
"gps": {"lat": 48.123, "lon": 37.456},
"altitude": 800.0,
"confidence": 0.95,
"heading": 87.3,
"timestamp": "2025-11-24T10:30:00Z"
}
}
```
**Error Conditions**:
- `404 Not Found`: flight_id doesn't exist
- Connection closed on client disconnect
**Test Cases**:
1. **Connect to stream**: Opens SSE connection successfully
2. **Receive frame events**: Process 100 frames → receive 100 events
3. **Receive user_input_needed**: Blocked frame → event sent
4. **Client reconnect**: Replay missed events from last_event_id
---
## Integration Tests
### Test 1: Complete Flight Lifecycle
1. POST /flights with valid data
2. GET /flights/{flightId} → verify data
3. GET /flights/{flightId}/stream (open SSE)
4. POST /flights/{flightId}/images/batch × 40
5. Receive frame_processed events via SSE
6. Receive flight_completed event
7. GET /flights/{flightId} → verify waypoints updated
8. DELETE /flights/{flightId}
### Test 2: User Fix Flow
1. Create flight and process images
2. Receive user_input_needed event
3. POST /flights/{flightId}/user-fix
4. Receive processing_resumed event
5. Continue receiving frame_processed events
### Test 3: Concurrent Flights
1. Create 10 flights concurrently
2. Upload batches to all flights in parallel
3. Stream results from all flights simultaneously
4. Verify no cross-contamination
### Test 4: Waypoint Updates
1. Create flight
2. Simulate per-frame updates via PUT /flights/{flightId}/waypoints/{waypointId} × 100
3. GET flight and verify all waypoints updated
4. Verify refined=true flag set
---
## Non-Functional Requirements
### Performance
- **create_flight**: < 500ms response (prefetch is async)
- **get_flight**: < 200ms for flights with < 2000 waypoints
- **update_waypoint**: < 100ms (critical for real-time updates)
- **upload_image_batch**: < 2 seconds for 50 × 2MB images
- **submit_user_fix**: < 200ms response
- **get_flight_status**: < 100ms
- **SSE latency**: < 500ms from event generation to client receipt
### Scalability
- Support 100 concurrent flight processing sessions
- Handle 1000+ concurrent SSE connections
- Handle flights with up to 3000 waypoints
- Support 10,000 requests per minute
### Reliability
- Request timeout: 30 seconds for batch uploads
- SSE keepalive: Ping every 30 seconds
- Automatic SSE reconnection with event replay
- Graceful handling of client disconnects
### Security
- API key authentication
- Rate limiting: 100 requests/minute per client
- Max upload size: 500MB per batch
- CORS configuration for web clients
- Input validation on all endpoints
- SQL injection prevention
---
## Dependencies
### Internal Components
- **F02 Flight Processor**: For all flight operations and processing orchestration
- **F05 Image Input Pipeline**: For batch processing
- **F11 Failure Recovery Coordinator**: For user fixes
- **F15 SSE Event Streamer**: For real-time streaming
- **F03 Flight Database**: For persistence (via F02)
### External Dependencies
- **FastAPI**: Web framework
- **Uvicorn**: ASGI server
- **Pydantic**: Validation
- **python-multipart**: Multipart form handling
---
## Data Models
### GPSPoint
```python
class GPSPoint(BaseModel):
lat: float # Latitude -90 to 90
lon: float # Longitude -180 to 180
```
### CameraParameters
```python
class CameraParameters(BaseModel):
focal_length: float # mm
sensor_width: float # mm
sensor_height: float # mm
resolution_width: int # pixels
resolution_height: int # pixels
distortion_coefficients: Optional[List[float]] = None
```
### Polygon
```python
class Polygon(BaseModel):
north_west: GPSPoint
south_east: GPSPoint
```
### Geofences
```python
class Geofences(BaseModel):
polygons: List[Polygon]
```
### FlightCreateRequest
```python
class FlightCreateRequest(BaseModel):
name: str
description: str
start_gps: GPSPoint
rough_waypoints: List[GPSPoint]
geofences: Geofences
camera_params: CameraParameters
altitude: float
```
### Waypoint
```python
class Waypoint(BaseModel):
id: str
lat: float
lon: float
altitude: Optional[float] = None
confidence: float
timestamp: datetime
refined: bool = False
```
### FlightDetailResponse
```python
class FlightDetailResponse(BaseModel):
flight_id: str
name: str
description: str
start_gps: GPSPoint
waypoints: List[Waypoint]
geofences: Geofences
camera_params: CameraParameters
altitude: float
status: str
frames_processed: int
frames_total: int
created_at: datetime
updated_at: datetime
```
### FlightStatusResponse
```python
class FlightStatusResponse(BaseModel):
status: str
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
message: Optional[str]
created_at: datetime
updated_at: datetime
```
### BatchMetadata
```python
class BatchMetadata(BaseModel):
start_sequence: int
end_sequence: int
batch_number: int
```
### BatchUpdateResponse
```python
class BatchUpdateResponse(BaseModel):
success: bool
updated_count: int
failed_ids: List[str]
errors: Optional[Dict[str, str]]
```
docs/02_components/02_flight_processor/flight_processor_spec.md
+1111
View File
File diff suppressed because it is too large Load Diff
docs/02_components/03_flight_database/flight_database_spec.md
+889
View File
@@ -0,0 +1,889 @@
# Flight Database Layer
## Interface Definition
**Interface Name**: `IFlightDatabase`
### Interface Methods
```python
class IFlightDatabase(ABC):
# Flight Operations
@abstractmethod
def insert_flight(self, flight: Flight) -> str:
pass
@abstractmethod
def update_flight(self, flight: Flight) -> bool:
pass
@abstractmethod
def query_flights(self, filters: Dict[str, Any], limit: int, offset: int) -> List[Flight]:
pass
@abstractmethod
def get_flight_by_id(self, flight_id: str) -> Optional[Flight]:
pass
@abstractmethod
def delete_flight(self, flight_id: str) -> bool:
pass
# Waypoint Operations
@abstractmethod
def get_waypoints(self, flight_id: str, limit: Optional[int] = None) -> List[Waypoint]:
pass
@abstractmethod
def insert_waypoint(self, flight_id: str, waypoint: Waypoint) -> str:
pass
@abstractmethod
def update_waypoint(self, flight_id: str, waypoint_id: str, waypoint: Waypoint) -> bool:
pass
@abstractmethod
def batch_update_waypoints(self, flight_id: str, waypoints: List[Waypoint]) -> BatchResult:
pass
# Flight State Operations
@abstractmethod
def save_flight_state(self, flight_state: FlightState) -> bool:
pass
@abstractmethod
def load_flight_state(self, flight_id: str) -> Optional[FlightState]:
pass
@abstractmethod
def query_processing_history(self, filters: Dict[str, Any]) -> List[FlightState]:
pass
# Frame Result Operations
@abstractmethod
def save_frame_result(self, flight_id: str, frame_result: FrameResult) -> bool:
pass
@abstractmethod
def get_frame_results(self, flight_id: str) -> List[FrameResult]:
pass
# Heading History Operations
@abstractmethod
def save_heading(self, flight_id: str, frame_id: int, heading: float, timestamp: datetime) -> bool:
pass
@abstractmethod
def get_heading_history(self, flight_id: str, last_n: Optional[int] = None) -> List[HeadingRecord]:
pass
@abstractmethod
def get_latest_heading(self, flight_id: str) -> Optional[float]:
pass
# Image Storage Operations
@abstractmethod
def save_image_metadata(self, flight_id: str, frame_id: int, file_path: str, metadata: Dict) -> bool:
pass
@abstractmethod
def get_image_path(self, flight_id: str, frame_id: int) -> Optional[str]:
pass
@abstractmethod
def get_image_metadata(self, flight_id: str, frame_id: int) -> Optional[Dict]:
pass
```
## Component Description
### Responsibilities
- Direct database access layer for all flight-related data
- Execute SQL queries and commands
- Manage database connections and transactions
- Handle connection pooling and retry logic
- Provide database abstraction (PostgreSQL, MySQL, etc.)
- Persist flight state, waypoints, frame results
- Store heading history for rotation management
- Store image file paths and metadata
### Scope
- CRUD operations on flights table
- CRUD operations on waypoints table
- CRUD operations on geofences table
- Flight state persistence
- Frame result storage
- Heading history tracking
- Image metadata storage
- Query optimization for large datasets
---
## Flight Operations
### `insert_flight(flight: Flight) -> str`
**Description**: Inserts a new flight with initial waypoints and geofences.
**Called By**:
- F02 Flight Processor
**Input**:
```python
Flight:
id: str
name: str
description: str
start_gps: GPSPoint
rough_waypoints: List[Waypoint]
geofences: Geofences
camera_params: CameraParameters
altitude: float
created_at: datetime
updated_at: datetime
```
**Output**:
```python
flight_id: str
```
**Database Operations**:
1. Begin transaction
2. INSERT INTO flights
3. INSERT INTO waypoints for each initial waypoint
4. INSERT INTO geofences for each polygon
5. INSERT INTO flight_state (initial state)
6. Commit transaction
**Error Conditions**:
- `IntegrityError`: Duplicate flight_id
- `DatabaseError`: Connection error, transaction failure
- Automatic rollback on error
**Test Cases**:
1. **Insert flight with 100 waypoints**: All data persisted
2. **Duplicate flight_id**: Raises IntegrityError
3. **Transaction rollback**: Error mid-insert → complete rollback
---
### `update_flight(flight: Flight) -> bool`
**Description**: Updates flight metadata.
**Called By**:
- F02 Flight Processor
**Input**:
```python
Flight with updated fields
```
**Output**:
```python
bool: True if updated, False if not found
```
**Database Operations**:
```sql
UPDATE flights
SET name = ?, description = ?, updated_at = ?
WHERE id = ?
```
**Test Cases**:
1. **Update existing flight**: Returns True
2. **Update non-existent flight**: Returns False
---
### `query_flights(filters: Dict[str, Any], limit: int, offset: int) -> List[Flight]`
**Description**: Queries flights with filtering and pagination.
**Called By**:
- F02 Flight Processor (listing)
- F01 Flight API
**Input**:
```python
filters: Dict[str, Any] # e.g., {"name": "Mission%", "status": "completed"}
limit: int
offset: int
```
**Output**:
```python
List[Flight] # Metadata only, without full waypoint data
```
**Test Cases**:
1. **Filter by name**: Returns matching flights
2. **Pagination**: offset=100, limit=50 → returns flights 100-149
3. **No matches**: Returns []
---
### `get_flight_by_id(flight_id: str) -> Optional[Flight]`
**Description**: Retrieves complete flight with all waypoints.
**Called By**:
- F02 Flight Processor
**Input**:
```python
flight_id: str
```
**Output**:
```python
Optional[Flight] # Complete flight with all waypoints
```
**Database Operations**:
1. SELECT FROM flights WHERE id = ?
2. SELECT FROM waypoints WHERE flight_id = ? ORDER BY timestamp
3. SELECT FROM geofences WHERE flight_id = ?
4. Assemble Flight object
**Test Cases**:
1. **Existing flight**: Returns complete Flight
2. **Non-existent flight**: Returns None
3. **Large flight (3000 waypoints)**: Returns within 150ms
---
### `delete_flight(flight_id: str) -> bool`
**Description**: Deletes a flight and cascades to all related data.
**Called By**:
- F02 Flight Processor
**Input**:
```python
flight_id: str
```
**Output**:
```python
bool: True if deleted, False if not found
```
**Database Operations**:
```sql
DELETE FROM flights WHERE id = ?
-- Cascade deletes via FK constraints:
-- waypoints, geofences, flight_state, frame_results,
-- heading_history, flight_images
```
**Test Cases**:
1. **Delete flight**: Cascades to all related tables
2. **Non-existent flight**: Returns False
---
## Waypoint Operations
### `get_waypoints(flight_id: str, limit: Optional[int] = None) -> List[Waypoint]`
**Description**: Retrieves waypoints for a flight.
**Called By**:
- F02 Flight Processor
**Input**:
```python
flight_id: str
limit: Optional[int]
```
**Output**:
```python
List[Waypoint]
```
**Test Cases**:
1. **All waypoints**: limit=None → returns all
2. **Limited**: limit=100 → returns first 100
---
### `insert_waypoint(flight_id: str, waypoint: Waypoint) -> str`
**Description**: Inserts a new waypoint.
**Called By**:
- F02 Flight Processor
**Input**:
```python
flight_id: str
waypoint: Waypoint
```
**Output**:
```python
waypoint_id: str
```
**Test Cases**:
1. **Valid insertion**: Returns waypoint_id
2. **Non-existent flight**: Raises ForeignKeyError
---
### `update_waypoint(flight_id: str, waypoint_id: str, waypoint: Waypoint) -> bool`
**Description**: Updates a waypoint. Critical path for GPS refinement updates.
**Called By**:
- F02 Flight Processor
- F13 Result Manager
**Input**:
```python
flight_id: str
waypoint_id: str
waypoint: Waypoint
```
**Output**:
```python
bool: True if updated, False if not found
```
**Database Operations**:
```sql
UPDATE waypoints
SET lat = ?, lon = ?, altitude = ?, confidence = ?, refined = ?
WHERE id = ? AND flight_id = ?
```
**Optimization**:
- Prepared statement caching
- Connection pooling
- Indexed on (flight_id, id)
**Test Cases**:
1. **Update existing**: Returns True
2. **Non-existent**: Returns False
3. **High-frequency**: 100 updates/sec sustained
---
### `batch_update_waypoints(flight_id: str, waypoints: List[Waypoint]) -> BatchResult`
**Description**: Updates multiple waypoints in a single transaction.
**Called By**:
- F02 Flight Processor (asynchronous refinements)
**Input**:
```python
flight_id: str
waypoints: List[Waypoint]
```
**Output**:
```python
BatchResult:
success: bool
updated_count: int
failed_ids: List[str]
```
**Test Cases**:
1. **Batch update 100**: All succeed
2. **Partial failure**: Returns failed_ids
---
## Flight State Operations
### `save_flight_state(flight_state: FlightState) -> bool`
**Description**: Saves or updates flight processing state.
**Called By**:
- F02 Flight Processor
**Input**:
```python
FlightState:
flight_id: str
status: str
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
created_at: datetime
updated_at: datetime
```
**Output**:
```python
bool: True if saved
```
**Test Cases**:
1. Save state → persisted
2. Update state → overwrites
---
### `load_flight_state(flight_id: str) -> Optional[FlightState]`
**Description**: Loads flight state (for crash recovery).
**Called By**:
- F02 Flight Processor
**Output**:
```python
Optional[FlightState]
```
**Test Cases**:
1. Load existing → returns state
2. Load non-existent → returns None
---
### `query_processing_history(filters: Dict[str, Any]) -> List[FlightState]`
**Description**: Queries historical processing data.
**Called By**:
- Analytics, admin tools
**Test Cases**:
1. Query by date range → returns flights
2. Query by status → returns filtered
---
## Frame Result Operations
### `save_frame_result(flight_id: str, frame_result: FrameResult) -> bool`
**Description**: Saves frame processing result.
**Called By**:
- F13 Result Manager
**Input**:
```python
FrameResult:
frame_id: int
gps_center: GPSPoint
altitude: float
heading: float
confidence: float
refined: bool
timestamp: datetime
```
**Output**:
```python
bool: True if saved
```
**Test Cases**:
1. Save result → persisted
2. Update on refinement → overwrites
---
### `get_frame_results(flight_id: str) -> List[FrameResult]`
**Description**: Gets all frame results for flight.
**Called By**:
- F13 Result Manager
**Test Cases**:
1. Get results → returns all frames
2. No results → returns empty list
---
## Heading History Operations
### `save_heading(flight_id: str, frame_id: int, heading: float, timestamp: datetime) -> bool`
**Description**: Saves heading value for temporal smoothing and recovery.
**Called By**:
- F06 Image Rotation Manager
**Input**:
```python
flight_id: str
frame_id: int
heading: float # Degrees 0-360
timestamp: datetime
```
**Output**:
```python
bool: True if saved
```
**Test Cases**:
1. **Save heading**: Persisted correctly
2. **Overwrite heading**: Same frame_id → updates value
---
### `get_heading_history(flight_id: str, last_n: Optional[int] = None) -> List[HeadingRecord]`
**Description**: Retrieves heading history for smoothing calculations.
**Called By**:
- F06 Image Rotation Manager
**Input**:
```python
flight_id: str
last_n: Optional[int] # Get last N headings, or all if None
```
**Output**:
```python
List[HeadingRecord]:
- frame_id: int
- heading: float
- timestamp: datetime
```
**Test Cases**:
1. **Get all**: Returns complete history
2. **Get last 10**: Returns 10 most recent
---
### `get_latest_heading(flight_id: str) -> Optional[float]`
**Description**: Gets most recent heading for pre-rotation.
**Called By**:
- F06 Image Rotation Manager
**Output**:
```python
Optional[float]: Heading in degrees, or None if no history
```
**Test Cases**:
1. **Has history**: Returns latest heading
2. **No history**: Returns None
---
## Image Storage Operations
### `save_image_metadata(flight_id: str, frame_id: int, file_path: str, metadata: Dict) -> bool`
**Description**: Saves image file path and metadata (original filename, dimensions, etc.).
**Called By**:
- F05 Image Input Pipeline
**Input**:
```python
flight_id: str
frame_id: int
file_path: str # Path where image is stored
metadata: Dict # {original_name, width, height, file_size, upload_time, ...}
```
**Output**:
```python
bool: True if saved
```
**Test Cases**:
1. **Save metadata**: Persisted with file_path
2. **Overwrite**: Same frame_id → updates
---
### `get_image_path(flight_id: str, frame_id: int) -> Optional[str]`
**Description**: Gets stored image file path.
**Called By**:
- F05 Image Input Pipeline
**Output**:
```python
Optional[str]: File path or None
```
**Test Cases**:
1. **Exists**: Returns file path
2. **Not exists**: Returns None
---
### `get_image_metadata(flight_id: str, frame_id: int) -> Optional[Dict]`
**Description**: Gets image metadata.
**Called By**:
- F05 Image Input Pipeline
**Output**:
```python
Optional[Dict]: Metadata dictionary or None
```
---
## Integration Tests
### Test 1: Complete Flight Lifecycle
1. insert_flight() with 500 waypoints
2. save_flight_state() with initial state
3. update_waypoint() × 100
4. save_frame_result() × 500
5. save_heading() × 500
6. get_flight_by_id() and verify all data
7. delete_flight() and verify cascade
### Test 2: High-Frequency Update Pattern
1. insert_flight() with 2000 waypoints
2. Concurrent: update_waypoint(), save_frame_result(), save_heading()
3. Measure throughput > 200 updates/sec
4. Verify all data persisted
### Test 3: Crash Recovery
1. Insert flight, process 500 frames
2. Simulate crash (kill process)
3. Restart, load_flight_state()
4. Verify state intact, resume processing
---
## Non-Functional Requirements
### Performance
- **insert_flight**: < 200ms for 100 waypoints
- **update_waypoint**: < 30ms (critical path)
- **get_flight_by_id**: < 100ms for 2000 waypoints
- **save_heading**: < 10ms
- **Throughput**: 200+ operations per second
### Scalability
- Connection pool: 50-100 connections
- Support 100+ concurrent flights
- Handle tables with millions of records
### Reliability
- ACID transaction guarantees
- Automatic retry on transient errors (3 attempts)
- Connection health checks
### Security
- SQL injection prevention (parameterized queries)
- Least privilege database permissions
- Connection string encryption
---
## Dependencies
### Internal Components
- None (lowest layer)
### External Dependencies
- **PostgreSQL** or **MySQL**
- **SQLAlchemy** or **psycopg2**
- **Alembic**: Schema migrations
---
## Database Schema
```sql
-- Flights table
CREATE TABLE flights (
id VARCHAR(36) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
description TEXT,
start_lat DECIMAL(10, 7) NOT NULL,
start_lon DECIMAL(11, 7) NOT NULL,
altitude DECIMAL(7, 2) NOT NULL,
camera_params JSONB NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
INDEX idx_created_at (created_at),
INDEX idx_name (name)
);
-- Waypoints table
CREATE TABLE waypoints (
id VARCHAR(36) PRIMARY KEY,
flight_id VARCHAR(36) NOT NULL,
lat DECIMAL(10, 7) NOT NULL,
lon DECIMAL(11, 7) NOT NULL,
altitude DECIMAL(7, 2),
confidence DECIMAL(3, 2) NOT NULL,
timestamp TIMESTAMP NOT NULL,
refined BOOLEAN NOT NULL DEFAULT FALSE,
FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE,
INDEX idx_flight_timestamp (flight_id, timestamp),
INDEX idx_flight_id (flight_id, id)
);
-- Geofences table
CREATE TABLE geofences (
id VARCHAR(36) PRIMARY KEY,
flight_id VARCHAR(36) NOT NULL,
nw_lat DECIMAL(10, 7) NOT NULL,
nw_lon DECIMAL(11, 7) NOT NULL,
se_lat DECIMAL(10, 7) NOT NULL,
se_lon DECIMAL(11, 7) NOT NULL,
FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE,
INDEX idx_geofence_flight (flight_id)
);
-- Flight state table
CREATE TABLE flight_state (
flight_id VARCHAR(36) PRIMARY KEY,
status VARCHAR(50) NOT NULL,
frames_processed INT NOT NULL DEFAULT 0,
frames_total INT NOT NULL DEFAULT 0,
current_frame INT,
current_heading FLOAT,
blocked BOOLEAN NOT NULL DEFAULT FALSE,
search_grid_size INT,
created_at TIMESTAMP NOT NULL,
updated_at TIMESTAMP NOT NULL,
FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE
);
-- Frame results table
CREATE TABLE frame_results (
id VARCHAR(36) PRIMARY KEY,
flight_id VARCHAR(36) NOT NULL,
frame_id INT NOT NULL,
gps_lat DECIMAL(10, 7),
gps_lon DECIMAL(11, 7),
altitude FLOAT,
heading FLOAT,
confidence FLOAT,
refined BOOLEAN DEFAULT FALSE,
timestamp TIMESTAMP,
updated_at TIMESTAMP,
FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE,
UNIQUE KEY (flight_id, frame_id),
INDEX idx_frame_flight (flight_id, frame_id)
);
-- Heading history table
CREATE TABLE heading_history (
flight_id VARCHAR(36) NOT NULL,
frame_id INT NOT NULL,
heading FLOAT NOT NULL,
timestamp TIMESTAMP NOT NULL,
PRIMARY KEY (flight_id, frame_id),
FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE,
INDEX idx_heading_flight (flight_id, frame_id DESC)
);
-- Flight images table
CREATE TABLE flight_images (
flight_id VARCHAR(36) NOT NULL,
frame_id INT NOT NULL,
file_path VARCHAR(500) NOT NULL,
metadata JSONB,
uploaded_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (flight_id, frame_id),
FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE,
INDEX idx_images_flight (flight_id, frame_id)
);
```
---
## Data Models
### Flight
```python
class Flight(BaseModel):
id: str
name: str
description: str
start_gps: GPSPoint
waypoints: List[Waypoint]
geofences: Geofences
camera_params: CameraParameters
altitude: float
created_at: datetime
updated_at: datetime
```
### FlightState
```python
class FlightState(BaseModel):
flight_id: str
status: str
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
created_at: datetime
updated_at: datetime
```
### FrameResult
```python
class FrameResult(BaseModel):
frame_id: int
gps_center: GPSPoint
altitude: float
heading: float
confidence: float
refined: bool
timestamp: datetime
updated_at: datetime
```
### HeadingRecord
```python
class HeadingRecord(BaseModel):
frame_id: int
heading: float
timestamp: datetime
```
### BatchResult
```python
class BatchResult(BaseModel):
success: bool
updated_count: int
failed_ids: List[str]
```
### DatabaseConfig
```python
class DatabaseConfig(BaseModel):
host: str
port: int
database: str
username: str
password: str
pool_size: int = 50
max_overflow: int = 50
pool_timeout: int = 30
pool_recycle: int = 3600
```
docs/02_components/gps_denied_04_satellite_data_manager/satellite_data_manager_spec.md → docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md
+10 -10
View File
@@ -77,7 +77,7 @@ class ISatelliteDataManager(ABC):
**Description**: Fetches a single satellite tile by GPS coordinates.
**Called By**:
- G09 Metric Refinement (single tile for drift correction)
- F09 Metric Refinement (single tile for drift correction)
- Internal (during prefetching)
**Input**:
@@ -121,8 +121,8 @@ GET /api/satellite/tiles/latlon?lat={lat}&lon={lon}&zoom={zoom}
**Description**: Fetches NxN grid of tiles centered on GPS coordinates.
**Called By**:
- G09 Metric Refinement (for progressive search)
- G11 Failure Recovery Coordinator
- F09 Metric Refinement (for progressive search)
- F11 Failure Recovery Coordinator
**Input**:
```python
@@ -167,7 +167,7 @@ GET /api/satellite/tiles/batch?tiles=[...]
**Description**: Prefetches satellite tiles along route corridor for a flight.
**Called By**:
- G02 Flight Manager (during flight creation)
- F02 Flight Manager (during flight creation)
**Input**:
```python
@@ -210,7 +210,7 @@ bool: True if prefetch completed, False on error
**Description**: Progressively fetches expanding tile grids for "kidnapped robot" recovery.
**Called By**:
- G11 Failure Recovery Coordinator (progressive search)
- F11 Failure Recovery Coordinator (progressive search)
**Input**:
```python
@@ -288,7 +288,7 @@ bool: True if cached successfully
**Called By**:
- Internal (before fetching from API)
- G09 Metric Refinement (direct cache lookup)
- F09 Metric Refinement (direct cache lookup)
**Input**:
```python
@@ -322,7 +322,7 @@ Optional[np.ndarray]: Tile image or None if not cached
**Called By**:
- Internal (for grid fetching)
- G11 Failure Recovery Coordinator
- F11 Failure Recovery Coordinator
**Input**:
```python
@@ -388,7 +388,7 @@ y = int((1 - log(tan(lat_rad) + sec(lat_rad)) / π) / 2 * n)
**Description**: Returns only NEW tiles when expanding from current grid to larger grid.
**Called By**:
- G11 Failure Recovery Coordinator (progressive search optimization)
- F11 Failure Recovery Coordinator (progressive search optimization)
**Input**:
```python
@@ -416,7 +416,7 @@ List[TileCoords] # Only tiles not in current_size grid
**Description**: Computes GPS bounding box of a tile.
**Called By**:
- G09 Metric Refinement (for homography calculations)
- F09 Metric Refinement (for homography calculations)
- H06 Web Mercator Utils (shared calculation)
**Input**:
@@ -450,7 +450,7 @@ TileBounds:
**Description**: Clears cached tiles for a completed flight.
**Called By**:
- G02 Flight Manager (cleanup after flight completion)
- F02 Flight Manager (cleanup after flight completion)
**Input**:
```python
docs/02_components/gps_denied_05_image_input_pipeline/image_input_pipeline_spec.md → docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md
+11 -11
View File
@@ -65,7 +65,7 @@ class IImageInputPipeline(ABC):
**Description**: Queues a batch of images for processing (FIFO).
**Called By**:
- G01 GPS-Denied REST API (after upload)
- F01 GPS-Denied REST API (after upload)
**Input**:
```python
@@ -207,7 +207,7 @@ bool: True if stored successfully
1. Create flight directory if not exists
2. Write each image to disk
3. Update metadata index
4. Persist to G17 Database Layer (metadata only)
4. Persist to F17 Database Layer (metadata only)
**Error Conditions**:
- `StorageError`: Disk full, permission error
@@ -224,8 +224,8 @@ bool: True if stored successfully
**Description**: Gets the next image in sequence for processing.
**Called By**:
- G06 Image Rotation Manager
- G07 Sequential VO
- F06 Image Rotation Manager
- F07 Sequential VO
- Processing pipeline (main loop)
**Input**:
@@ -265,8 +265,8 @@ ImageData:
**Description**: Retrieves a specific image by sequence number.
**Called By**:
- G11 Failure Recovery Coordinator (for user fix)
- G13 Result Manager (for refinement)
- F11 Failure Recovery Coordinator (for user fix)
- F13 Result Manager (for refinement)
**Input**:
```python
@@ -299,8 +299,8 @@ Optional[ImageData]
**Description**: Retrieves metadata without loading full image (lightweight).
**Called By**:
- G02 Flight Manager (status checks)
- G13 Result Manager (metadata-only queries)
- F02 Flight Manager (status checks)
- F13 Result Manager (metadata-only queries)
**Input**:
```python
@@ -330,8 +330,8 @@ ImageMetadata:
**Description**: Gets current processing status for a flight.
**Called By**:
- G01 GPS-Denied REST API (status endpoint)
- G02 Flight Manager
- F01 GPS-Denied REST API (status endpoint)
- F02 Flight Manager
**Input**:
```python
@@ -397,7 +397,7 @@ ProcessingStatus:
### Internal Components
- **H08 Batch Validator**: For validation logic
- **G17 Database Layer**: For metadata persistence
- **F17 Database Layer**: For metadata persistence
### External Dependencies
- **opencv-python**: Image I/O
docs/02_components/gps_denied_06_image_rotation_manager/image_rotation_manager_spec.md → docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md
+116 -5
View File
@@ -35,6 +35,14 @@ class IImageRotationManager(ABC):
@abstractmethod
def requires_rotation_sweep(self, flight_id: str) -> bool:
pass
@abstractmethod
def rotate_chunk_360(self, chunk_images: List[np.ndarray], angle: float) -> List[np.ndarray]:
pass
@abstractmethod
def try_chunk_rotation_steps(self, chunk_images: List[np.ndarray], satellite_tile: np.ndarray) -> Optional[RotationResult]:
pass
```
## Component Description
@@ -47,6 +55,8 @@ class IImageRotationManager(ABC):
- Calculate precise rotation angle from homography point correspondences
- Detect sharp turns requiring rotation sweep
- Pre-rotate images to known heading for subsequent frames
- **Chunk rotation operations (rotate all images in chunk)**
- **Chunk rotation sweeps for LiteSAM matching**
### Scope
- Image rotation operations
@@ -54,6 +64,7 @@ class IImageRotationManager(ABC):
- Sharp turn detection
- Rotation sweep coordination with LiteSAM matching
- Precise angle calculation from homography
- **Chunk-level rotation (all images rotated by same angle)**
## API Methods
@@ -132,7 +143,7 @@ return None # No match found
**Processing Flow**:
1. For each 30° step:
- Rotate image
- Call G09 Metric Refinement (LiteSAM)
- Call F09 Metric Refinement (LiteSAM)
- Check if match found
2. If match found:
- Calculate precise angle from homography
@@ -193,9 +204,9 @@ float: Precise rotation angle (e.g., 62.3° refined from 60° step)
**Description**: Gets current UAV heading angle for a flight.
**Called By**:
- G06 Internal (to check if pre-rotation needed)
- F06 Internal (to check if pre-rotation needed)
- Main processing loop (before LiteSAM)
- G11 Failure Recovery Coordinator (logging)
- F11 Failure Recovery Coordinator (logging)
**Input**:
```python
@@ -293,7 +304,7 @@ return delta > 45
**Called By**:
- Main processing loop (before each frame)
- G11 Failure Recovery Coordinator (after tracking loss)
- F11 Failure Recovery Coordinator (after tracking loss)
**Input**:
```python
@@ -317,6 +328,98 @@ bool: True if rotation sweep required
3. **Sharp turn detected**: Returns True
4. **Tracking loss**: Returns True
---
### `rotate_chunk_360(chunk_images: List[np.ndarray], angle: float) -> List[np.ndarray]`
**Description**: Rotates all images in a chunk by the same angle.
**Called By**:
- Internal (during try_chunk_rotation_steps)
- F11 Failure Recovery Coordinator (chunk rotation sweeps)
**Input**:
```python
chunk_images: List[np.ndarray] # 5-20 images from chunk
angle: float # Rotation angle in degrees (0-360)
```
**Output**:
```python
List[np.ndarray] # Rotated images (same dimensions)
```
**Processing Flow**:
1. For each image in chunk:
- rotate_image_360(image, angle) → rotated_image
2. Return list of rotated images
**Performance**:
- Rotation time: ~20ms × N images
- For 10 images: ~200ms total
**Test Cases**:
1. **Rotate chunk**: All images rotated correctly
2. **Angle consistency**: All images rotated by same angle
3. **Image preservation**: Original images unchanged
---
### `try_chunk_rotation_steps(chunk_images: List[np.ndarray], satellite_tile: np.ndarray) -> Optional[RotationResult]`
**Description**: Performs 30° rotation sweep on entire chunk, trying LiteSAM match for each rotation.
**Called By**:
- F11 Failure Recovery Coordinator (chunk LiteSAM matching with rotation)
**Input**:
```python
chunk_images: List[np.ndarray] # Chunk images
satellite_tile: np.ndarray # Reference satellite tile
```
**Output**:
```python
RotationResult:
matched: bool
initial_angle: float # Best matching step angle (0, 30, 60, ...)
precise_angle: float # Refined angle from homography
confidence: float
homography: np.ndarray
```
**Algorithm**:
```
For angle in [0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°]:
rotated_chunk = rotate_chunk_360(chunk_images, angle)
result = LiteSAM.align_chunk_to_satellite(rotated_chunk, satellite_tile)
if result.matched and result.confidence > threshold:
precise_angle = calculate_precise_angle(result.homography, angle)
return RotationResult(matched=True, initial_angle=angle, precise_angle=precise_angle, ...)
return None # No match found
```
**Processing Flow**:
1. For each 30° step:
- Rotate all chunk images
- Call F09 Metric Refinement.align_chunk_to_satellite()
- Check if match found
2. If match found:
- Calculate precise angle from homography
- Return RotationResult
3. If no match:
- Return None
**Performance**:
- 12 rotations × chunk LiteSAM (~60ms) = ~720ms
- Acceptable for chunk matching (async operation)
**Test Cases**:
1. **Match at 60°**: Finds match, returns result
2. **Match at 0°**: No rotation needed, finds match
3. **No match**: All 12 rotations tried, returns None
4. **Multiple matches**: Returns best confidence
## Integration Tests
### Test 1: First Frame Rotation Sweep
@@ -348,6 +451,13 @@ bool: True if rotation sweep required
3. try_rotation_steps() with all 12 rotations
4. Match found → heading updated
### Test 5: Chunk Rotation Sweeps
1. Build chunk with 10 images (unknown orientation)
2. try_chunk_rotation_steps() with satellite tile
3. Match found at 120° step
4. Precise angle calculated (122.5°)
5. Verify all images rotated consistently
## Non-Functional Requirements
### Performance
@@ -369,8 +479,9 @@ bool: True if rotation sweep required
## Dependencies
### Internal Components
- **G09 Metric Refinement**: For LiteSAM matching during rotation sweep
- **F09 Metric Refinement**: For LiteSAM matching during rotation sweep and chunk matching
- **H07 Image Rotation Utils**: For image rotation and angle calculations
- **F12 Route Chunk Manager**: For chunk image retrieval
### External Dependencies
- **opencv-python**: Image rotation (`cv2.warpAffine`)
docs/02_components/gps_denied_07_sequential_visual_odometry/sequential_visual_odometry_spec.md → docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md
+69 -9
View File
@@ -23,6 +23,10 @@ class ISequentialVO(ABC):
@abstractmethod
def estimate_motion(self, matches: Matches, camera_params: CameraParameters) -> Optional[Motion]:
pass
@abstractmethod
def compute_relative_pose_in_chunk(self, prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]:
pass
```
## Component Description
@@ -34,12 +38,14 @@ class ISequentialVO(ABC):
- Estimate relative pose (translation + rotation) between frames
- Return relative pose factors for Factor Graph Optimizer
- Detect tracking loss (low inlier count)
- **Chunk-aware VO operations (factors added to chunk subgraph)**
### Scope
- Frame-to-frame visual odometry
- Feature-based motion estimation
- Handles low overlap and challenging agricultural environments
- Provides relative measurements for trajectory optimization
- **Chunk-scoped operations (Atlas multi-map architecture)**
## API Methods
@@ -96,7 +102,7 @@ RelativePose:
**Called By**:
- Internal (during compute_relative_pose)
- G08 Global Place Recognition (for descriptor caching)
- F08 Global Place Recognition (for descriptor caching)
**Input**:
```python
@@ -112,7 +118,7 @@ Features:
```
**Processing Details**:
- Uses G15 Model Manager to get SuperPoint model
- Uses F16 Model Manager to get SuperPoint model
- Converts to grayscale if needed
- Non-maximum suppression for keypoint selection
- Typically extracts 500-2000 keypoints per image
@@ -153,7 +159,7 @@ Matches:
```
**Processing Details**:
- Uses G15 Model Manager to get LightGlue model
- Uses F16 Model Manager to get LightGlue model
- Transformer-based attention mechanism
- "Dustbin" mechanism for unmatched features
- Adaptive depth (exits early for easy matches)
@@ -203,10 +209,12 @@ Motion:
**Scale Ambiguity**:
- Monocular VO has inherent scale ambiguity
- Translation is unit vector (direction only)
- Scale resolved by:
- Altitude prior (from G10 Factor Graph)
- Absolute GPS measurements (from G09 LiteSAM)
- Translation is unit vector (direction only, magnitude = 1)
- **F07 does NOT resolve scale** - it only outputs unit translation vectors
- Scale resolution is handled by F10 Factor Graph Optimizer, which uses:
- Altitude priors (soft constraints)
- GSD-based expected displacement calculations (via H02)
- Absolute GPS anchors from F09 Metric Refinement
**Error Conditions**:
- Returns `None`: Insufficient inliers (< 8 points for Essential Matrix)
@@ -216,6 +224,49 @@ Motion:
2. **Low inliers**: May return None
3. **Degenerate motion**: Handles pure rotation
---
### `compute_relative_pose_in_chunk(prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]`
**Description**: Computes relative camera pose between consecutive frames within a chunk context.
**Called By**:
- F02 Flight Processor (chunk-aware processing)
**Input**:
```python
prev_image: np.ndarray # Previous frame (t-1)
curr_image: np.ndarray # Current frame (t)
chunk_id: str # Chunk identifier for context
```
**Output**:
```python
RelativePose:
translation: np.ndarray # (x, y, z) in meters
rotation: np.ndarray # 3×3 rotation matrix or quaternion
confidence: float # 0.0 to 1.0
inlier_count: int
total_matches: int
tracking_good: bool
chunk_id: str # Chunk context
```
**Processing Flow**:
1. Same as compute_relative_pose() (SuperPoint + LightGlue)
2. Return RelativePose with chunk_id context
3. Factor will be added to chunk's subgraph (not global graph)
**Chunk Context**:
- VO operations are chunk-scoped
- Factors added to chunk's subgraph via F10.add_relative_factor_to_chunk()
- Chunk isolation ensures independent optimization
**Test Cases**:
1. **Chunk-aware VO**: Returns RelativePose with chunk_id
2. **Chunk isolation**: Factors isolated to chunk
3. **Multiple chunks**: VO operations don't interfere between chunks
## Integration Tests
### Test 1: Normal Flight Sequence
@@ -241,6 +292,13 @@ Motion:
2. compute_relative_pose() → SuperPoint handles better than SIFT
3. Verify match quality
### Test 5: Chunk-Aware VO
1. Create chunk_1 and chunk_2
2. compute_relative_pose_in_chunk() for frames in chunk_1
3. compute_relative_pose_in_chunk() for frames in chunk_2
4. Verify factors added to respective chunks
5. Verify chunks optimized independently
## Non-Functional Requirements
### Performance
@@ -263,10 +321,11 @@ Motion:
## Dependencies
### Internal Components
- **G15 Model Manager**: For SuperPoint and LightGlue models
- **G16 Configuration Manager**: For camera parameters
- **F16 Model Manager**: For SuperPoint and LightGlue models
- **F17 Configuration Manager**: For camera parameters
- **H01 Camera Model**: For coordinate normalization
- **H05 Performance Monitor**: For timing measurements
- **F10 Factor Graph Optimizer**: For chunk-scoped factor addition
### External Dependencies
- **SuperPoint**: Feature extraction model
@@ -303,6 +362,7 @@ class RelativePose(BaseModel):
total_matches: int
tracking_good: bool
scale_ambiguous: bool = True
chunk_id: Optional[str] = None # Chunk context (if chunk-aware)
```
### Motion
docs/02_components/gps_denied_08_global_place_recognition/global_place_recognition_spec.md → docs/02_components/08_global_place_recognition/global_place_recognition_spec.md
+111 -6
View File
@@ -27,6 +27,14 @@ class IGlobalPlaceRecognition(ABC):
@abstractmethod
def initialize_database(self, satellite_tiles: List[SatelliteTile]) -> bool:
pass
@abstractmethod
def retrieve_candidate_tiles_for_chunk(self, chunk_images: List[np.ndarray], top_k: int) -> List[TileCandidate]:
pass
@abstractmethod
def compute_chunk_descriptor(self, chunk_images: List[np.ndarray]) -> np.ndarray:
pass
```
## Component Description
@@ -38,6 +46,8 @@ class IGlobalPlaceRecognition(ABC):
- Query Faiss index of satellite tile descriptors
- Return top-k candidate tile regions for progressive refinement
- Initialize satellite descriptor database during system startup
- **Chunk semantic matching (aggregate DINOv2 features)**
- **Chunk descriptor computation for robust matching**
### Scope
- Global localization (not frame-to-frame)
@@ -45,6 +55,7 @@ class IGlobalPlaceRecognition(ABC):
- Handles domain gap (UAV vs satellite imagery)
- Semantic feature extraction (DINOv2)
- Efficient similarity search (Faiss)
- **Chunk-level matching (more robust than single-image)**
## API Methods
@@ -53,7 +64,7 @@ class IGlobalPlaceRecognition(ABC):
**Description**: Retrieves top-k candidate satellite tiles for a UAV image.
**Called By**:
- G11 Failure Recovery Coordinator (after tracking loss)
- F11 Failure Recovery Coordinator (after tracking loss)
**Input**:
```python
@@ -112,7 +123,7 @@ np.ndarray: Descriptor vector (4096-dim or 8192-dim)
4. Return compact global descriptor
**Processing Details**:
- Uses G15 Model Manager to get DINOv2 model
- Uses F16 Model Manager to get DINOv2 model
- Dense features: extracts from multiple spatial locations
- VLAD codebook: pre-trained cluster centers
- Semantic features: invariant to texture/color changes
@@ -197,7 +208,7 @@ List[TileCandidate] # Re-ranked list
**Description**: Initializes satellite descriptor database during system startup.
**Called By**:
- G02 Flight Manager (during system initialization)
- F02 Flight Manager (during system initialization)
**Input**:
```python
@@ -228,13 +239,99 @@ bool: True if database initialized successfully
1. **Initialize with 1000 tiles**: Completes successfully
2. **Load pre-built index**: Fast startup (<10s)
---
### `retrieve_candidate_tiles_for_chunk(chunk_images: List[np.ndarray], top_k: int) -> List[TileCandidate]`
**Description**: Retrieves top-k candidate satellite tiles for a chunk using aggregate descriptor.
**Called By**:
- F11 Failure Recovery Coordinator (chunk semantic matching)
- F12 Route Chunk Manager (chunk matching coordination)
**Input**:
```python
chunk_images: List[np.ndarray] # 5-20 images from chunk
top_k: int # Number of candidates (typically 5)
```
**Output**:
```python
List[TileCandidate]:
tile_id: str
gps_center: GPSPoint
similarity_score: float
rank: int
```
**Processing Flow**:
1. compute_chunk_descriptor(chunk_images) → aggregate descriptor
2. query_database(descriptor, top_k) → database_matches
3. Retrieve tile metadata for matches
4. rank_candidates() → sorted by similarity
5. Return top-k candidates
**Advantages over Single-Image Matching**:
- Aggregate descriptor more robust to featureless terrain
- Multiple images provide more context
- Better handles plain fields where single-image matching fails
**Test Cases**:
1. **Chunk matching**: Returns relevant tiles
2. **Featureless terrain**: Succeeds where single-image fails
3. **Top-1 accuracy**: Correct tile in top-5 > 90% (better than single-image)
---
### `compute_chunk_descriptor(chunk_images: List[np.ndarray]) -> np.ndarray`
**Description**: Computes aggregate DINOv2 descriptor from multiple chunk images.
**Called By**:
- Internal (during retrieve_candidate_tiles_for_chunk)
- F12 Route Chunk Manager (chunk descriptor computation - delegates to F08)
**Input**:
```python
chunk_images: List[np.ndarray] # 5-20 images from chunk
```
**Output**:
```python
np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
```
**Algorithm**:
1. For each image in chunk:
- compute_location_descriptor(image) → descriptor (DINOv2 + VLAD)
2. Aggregate descriptors:
- **Mean aggregation**: Average all descriptors
- **VLAD aggregation**: Use VLAD codebook for aggregation
- **Max aggregation**: Element-wise maximum
3. L2-normalize aggregated descriptor
4. Return composite descriptor
**Aggregation Strategy**:
- **Mean**: Simple average (default)
- **VLAD**: More sophisticated, preserves spatial information
- **Max**: Emphasizes strongest features
**Performance**:
- Descriptor computation: ~150ms × N images (can be parallelized)
- Aggregation: ~10ms
**Test Cases**:
1. **Compute descriptor**: Returns aggregated descriptor
2. **Multiple images**: Descriptor aggregates correctly
3. **Descriptor quality**: More robust than single-image descriptor
## Integration Tests
### Test 1: Place Recognition Flow
1. Load UAV image from sharp turn
2. retrieve_candidate_tiles(top_k=5)
3. Verify correct tile in top-5
4. Pass candidates to G11 Failure Recovery
4. Pass candidates to F11 Failure Recovery
### Test 2: Season Invariance
1. Satellite tiles from summer
@@ -247,6 +344,13 @@ bool: True if database initialized successfully
3. Verify Faiss index built
4. Query with test image → returns matches
### Test 4: Chunk Semantic Matching
1. Build chunk with 10 images (plain field scenario)
2. compute_chunk_descriptor() → aggregate descriptor
3. retrieve_candidate_tiles_for_chunk() → returns candidates
4. Verify correct tile in top-5 (where single-image matching failed)
5. Verify chunk matching more robust than single-image
## Non-Functional Requirements
### Performance
@@ -267,9 +371,10 @@ bool: True if database initialized successfully
## Dependencies
### Internal Components
- **G15 Model Manager**: For DINOv2 model
- **F16 Model Manager**: For DINOv2 model
- **H04 Faiss Index Manager**: For similarity search
- **G04 Satellite Data Manager**: For tile metadata
- **F04 Satellite Data Manager**: For tile metadata
- **F12 Route Chunk Manager**: For chunk image retrieval
### External Dependencies
- **DINOv2**: Foundation vision model
docs/02_components/gps_denied_09_metric_refinement/metric_refinement_spec.md → docs/02_components/09_metric_refinement/metric_refinement_spec.md
+161 -13
View File
@@ -9,7 +9,7 @@
```python
class IMetricRefinement(ABC):
@abstractmethod
def align_to_satellite(self, uav_image: np.ndarray, satellite_tile: np.ndarray) -> Optional[AlignmentResult]:
def align_to_satellite(self, uav_image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[AlignmentResult]:
pass
@abstractmethod
@@ -23,6 +23,14 @@ class IMetricRefinement(ABC):
@abstractmethod
def compute_match_confidence(self, alignment: AlignmentResult) -> float:
pass
@abstractmethod
def align_chunk_to_satellite(self, chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[ChunkAlignmentResult]:
pass
@abstractmethod
def match_chunk_homography(self, chunk_images: List[np.ndarray], satellite_tile: np.ndarray) -> Optional[np.ndarray]:
pass
```
## Component Description
@@ -34,6 +42,8 @@ class IMetricRefinement(ABC):
- Extract absolute GPS coordinates from alignment
- Process against single tile (drift correction) or tile grid (progressive search)
- Achieve <20m accuracy requirement
- **Chunk-to-satellite matching (more robust than single-image)**
- **Chunk homography computation**
### Scope
- Cross-view geo-localization (UAV↔satellite)
@@ -41,22 +51,24 @@ class IMetricRefinement(ABC):
- Multi-scale processing for different GSDs
- Domain gap (UAV downward vs satellite nadir view)
- **Critical**: Fails if rotation >45° (handled by G06)
- **Chunk-level matching (aggregate correspondences from multiple images)**
## API Methods
### `align_to_satellite(uav_image: np.ndarray, satellite_tile: np.ndarray) -> Optional[AlignmentResult]`
### `align_to_satellite(uav_image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[AlignmentResult]`
**Description**: Aligns UAV image to satellite tile, returning GPS location.
**Called By**:
- G06 Image Rotation Manager (during rotation sweep)
- G11 Failure Recovery Coordinator (progressive search)
- Main processing loop (drift correction with single tile)
- F06 Image Rotation Manager (during rotation sweep)
- F11 Failure Recovery Coordinator (progressive search)
- F02 Flight Processor (drift correction with single tile)
**Input**:
```python
uav_image: np.ndarray # Pre-rotated UAV image
satellite_tile: np.ndarray # Reference satellite tile
tile_bounds: TileBounds # GPS bounds and GSD of the satellite tile
```
**Output**:
@@ -76,7 +88,7 @@ AlignmentResult:
3. Estimate homography from correspondences
4. Validate match quality (inlier count, reprojection error)
5. If valid match:
- Extract GPS from homography
- Extract GPS from homography using tile_bounds
- Return AlignmentResult
6. If no match:
- Return None
@@ -142,7 +154,7 @@ Optional[np.ndarray]: 3×3 homography matrix or None
**Called By**:
- Internal (during align_to_satellite)
- G06 Image Rotation Manager (for precise angle calculation)
- F06 Image Rotation Manager (for precise angle calculation)
**Input**:
```python
@@ -164,7 +176,7 @@ GPSPoint:
3. Convert satellite pixel to GPS using tile_bounds and GSD
4. Return GPS coordinates
**Uses**: G04 Satellite Data Manager for tile_bounds, H02 GSD Calculator
**Uses**: tile_bounds parameter, H02 GSD Calculator
**Test Cases**:
1. **Center alignment**: UAV center → correct GPS
@@ -179,7 +191,7 @@ GPSPoint:
**Called By**:
- Internal (during align_to_satellite)
- G11 Failure Recovery Coordinator (to decide if match acceptable)
- F11 Failure Recovery Coordinator (to decide if match acceptable)
**Input**:
```python
@@ -207,6 +219,104 @@ float: Confidence score (0.0 to 1.0)
2. **Weak match**: confidence 0.5-0.7
3. **Poor match**: confidence < 0.5
---
### `align_chunk_to_satellite(chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[ChunkAlignmentResult]`
**Description**: Aligns entire chunk to satellite tile, returning GPS location.
**Called By**:
- F06 Image Rotation Manager (during chunk rotation sweep)
- F11 Failure Recovery Coordinator (chunk LiteSAM matching)
**Input**:
```python
chunk_images: List[np.ndarray] # Pre-rotated chunk images (5-20 images)
satellite_tile: np.ndarray # Reference satellite tile
tile_bounds: TileBounds # GPS bounds and GSD of the satellite tile
```
**Output**:
```python
ChunkAlignmentResult:
matched: bool
chunk_id: str
chunk_center_gps: GPSPoint # GPS of chunk center (middle frame)
rotation_angle: float
confidence: float
inlier_count: int
transform: Sim3Transform
```
**Processing Flow**:
1. For each image in chunk:
- Extract features using LiteSAM encoder
- Compute correspondences with satellite tile
2. Aggregate correspondences from all images
3. Estimate homography from aggregate correspondences
4. Validate match quality (inlier count, reprojection error)
5. If valid match:
- Extract GPS from chunk center using tile_bounds
- Compute Sim(3) transform (translation, rotation, scale)
- Return ChunkAlignmentResult
6. If no match:
- Return None
**Match Criteria**:
- **Good match**: inlier_count > 50, confidence > 0.7
- **Weak match**: inlier_count 30-50, confidence 0.5-0.7
- **No match**: inlier_count < 30
**Advantages over Single-Image Matching**:
- More correspondences (aggregate from multiple images)
- More robust to featureless terrain
- Better handles partial occlusions
- Higher confidence scores
**Test Cases**:
1. **Chunk alignment**: Returns GPS within 20m of ground truth
2. **Featureless terrain**: Succeeds where single-image fails
3. **Rotation >45°**: Fails (requires pre-rotation via F06)
4. **Multi-scale**: Handles GSD mismatch
---
### `match_chunk_homography(chunk_images: List[np.ndarray], satellite_tile: np.ndarray) -> Optional[np.ndarray]`
**Description**: Computes homography transformation from chunk to satellite.
**Called By**:
- Internal (during align_chunk_to_satellite)
**Input**:
```python
chunk_images: List[np.ndarray]
satellite_tile: np.ndarray
```
**Output**:
```python
Optional[np.ndarray]: 3×3 homography matrix or None
```
**Algorithm (LiteSAM)**:
1. Extract multi-scale features from all chunk images using TAIFormer
2. Aggregate features (mean or max pooling)
3. Compute correlation via Convolutional Token Mixer (CTM)
4. Generate dense correspondences
5. Estimate homography using RANSAC
6. Refine with non-linear optimization
**Homography Properties**:
- Maps pixels from chunk center to satellite image
- Accounts for: scale, rotation, perspective
- 8 DoF (degrees of freedom)
**Test Cases**:
1. **Valid correspondence**: Returns 3×3 matrix
2. **Insufficient features**: Returns None
3. **Aggregate correspondences**: More robust than single-image
## Integration Tests
### Test 1: Single Tile Drift Correction
@@ -217,8 +327,8 @@ float: Confidence score (0.0 to 1.0)
### Test 2: Progressive Search (4 tiles)
1. Load UAV image from sharp turn
2. Get 2×2 tile grid from G04
3. align_to_satellite() for each tile
2. Get 2×2 tile grid from F04
3. align_to_satellite() for each tile (with tile_bounds)
4. First 3 tiles: No match
5. 4th tile: Match found → GPS extracted
@@ -233,6 +343,20 @@ float: Confidence score (0.0 to 1.0)
2. Satellite at zoom 19 (GSD=0.3m/pixel)
3. LiteSAM handles scale difference → match succeeds
### Test 5: Chunk LiteSAM Matching
1. Build chunk with 10 images (plain field scenario)
2. Pre-rotate chunk to known heading
3. align_chunk_to_satellite() → returns GPS
4. Verify GPS within 20m of ground truth
5. Verify chunk matching more robust than single-image
### Test 6: Chunk Rotation Sweeps
1. Build chunk with unknown orientation
2. Try chunk rotation steps (0°, 30°, ..., 330°)
3. align_chunk_to_satellite() for each rotation
4. Match found at 120° → GPS extracted
5. Verify Sim(3) transform computed correctly
## Non-Functional Requirements
### Performance
@@ -253,11 +377,13 @@ float: Confidence score (0.0 to 1.0)
## Dependencies
### Internal Components
- **G15 Model Manager**: For LiteSAM model
- **G04 Satellite Data Manager**: For tile_bounds and GSD
- **F16 Model Manager**: For LiteSAM model
- **H01 Camera Model**: For projection operations
- **H02 GSD Calculator**: For coordinate transformations
- **H05 Performance Monitor**: For timing
- **F12 Route Chunk Manager**: For chunk image retrieval
**Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager)
### External Dependencies
- **LiteSAM**: Cross-view matching model
@@ -304,5 +430,27 @@ class LiteSAMConfig(BaseModel):
min_inliers: int = 15
max_reprojection_error: float = 2.0 # pixels
multi_scale_levels: int = 3
chunk_min_inliers: int = 30 # Higher threshold for chunk matching
```
### ChunkAlignmentResult
```python
class ChunkAlignmentResult(BaseModel):
matched: bool
chunk_id: str
chunk_center_gps: GPSPoint
rotation_angle: float
confidence: float
inlier_count: int
transform: Sim3Transform # Translation, rotation, scale
reprojection_error: float # Mean error in pixels
```
### Sim3Transform
```python
class Sim3Transform(BaseModel):
translation: np.ndarray # (3,) - translation vector
rotation: np.ndarray # (3, 3) rotation matrix or (4,) quaternion
scale: float # Scale factor
```
docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md
+763
View File
@@ -0,0 +1,763 @@
# Factor Graph Optimizer
## Interface Definition
**Interface Name**: `IFactorGraphOptimizer`
### Interface Methods
```python
class IFactorGraphOptimizer(ABC):
@abstractmethod
def add_relative_factor(self, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
pass
@abstractmethod
def add_absolute_factor(self, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool:
pass
@abstractmethod
def add_altitude_prior(self, frame_id: int, altitude: float, covariance: float) -> bool:
pass
@abstractmethod
def optimize(self, iterations: int) -> OptimizationResult:
pass
@abstractmethod
def get_trajectory(self) -> Dict[int, Pose]:
pass
@abstractmethod
def get_marginal_covariance(self, frame_id: int) -> np.ndarray:
pass
@abstractmethod
def create_new_chunk(self, chunk_id: str, start_frame_id: int) -> ChunkHandle:
pass
@abstractmethod
def get_chunk_for_frame(self, frame_id: int) -> Optional[ChunkHandle]:
pass
@abstractmethod
def add_relative_factor_to_chunk(self, chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
pass
@abstractmethod
def add_chunk_anchor(self, chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool:
pass
@abstractmethod
def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool:
pass
@abstractmethod
def get_chunk_trajectory(self, chunk_id: str) -> Dict[int, Pose]:
pass
@abstractmethod
def get_all_chunks(self) -> List[ChunkHandle]:
pass
@abstractmethod
def optimize_chunk(self, chunk_id: str, iterations: int) -> OptimizationResult:
pass
@abstractmethod
def optimize_global(self, iterations: int) -> OptimizationResult:
pass
```
## Component Description
### Responsibilities
- GTSAM-based fusion of relative and absolute measurements
- Incremental optimization (iSAM2) for real-time performance
- Robust kernels (Huber/Cauchy) for 350m outlier handling
- Scale resolution through altitude priors and absolute GPS
- Trajectory smoothing and global consistency
- Back-propagation of refinements to previous frames
- **Native multi-chunk/multi-map support (Atlas architecture)**
- **Chunk lifecycle management (creation, optimization, merging)**
- **Sim(3) transformation for chunk merging**
### Scope
- Non-linear least squares optimization
- Factor graph representation of SLAM problem
- Handles monocular scale ambiguity
- Real-time incremental updates
- Asynchronous batch refinement
- **Multi-chunk factor graph with independent subgraphs**
- **Chunk-level optimization and global merging**
- **Sim(3) similarity transformation for chunk alignment**
## API Methods
### `add_relative_factor(frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool`
**Description**: Adds relative pose measurement between consecutive frames.
**Called By**:
- F07 Sequential VO (frame-to-frame odometry)
**Input**:
```python
frame_i: int # Previous frame ID
frame_j: int # Current frame ID (typically frame_i + 1)
relative_pose: RelativePose:
translation: np.ndarray # (3,) - unit vector (scale ambiguous from VO)
rotation: np.ndarray # (3, 3) or quaternion
covariance: np.ndarray # (6, 6) - uncertainty
```
**Scale Resolution**:
F07 returns unit translation vectors due to monocular scale ambiguity. F10 resolves scale by:
1. Using altitude prior to constrain Z-axis
2. Computing expected displacement from H02 GSD Calculator:
- GSD = (sensor_width × altitude) / (focal_length × resolution_width)
- expected_displacement ≈ frame_spacing × GSD (typically ~100m)
3. Scaling: scaled_translation = unit_translation × expected_displacement
4. Global refinement using absolute GPS factors from F09 LiteSAM
**Output**:
```python
bool: True if factor added successfully
```
**Processing Flow**:
1. Create BetweenFactor in GTSAM
2. Apply robust kernel (Huber) to handle outliers
3. Add to factor graph
4. Mark graph as needing optimization
**Robust Kernel**:
- **Huber loss**: Downweights large errors (>threshold)
- **Critical** for 350m outlier handling from tilt
**Test Cases**:
1. **Normal motion**: Factor added, contributes to optimization
2. **Large displacement** (350m outlier): Huber kernel reduces weight
3. **Consecutive factors**: Chain of relative factors builds trajectory
---
### `add_absolute_factor(frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool`
**Description**: Adds absolute GPS measurement for drift correction or user anchor.
**Called By**:
- F09 Metric Refinement (after LiteSAM alignment)
- F11 Failure Recovery Coordinator (user-provided anchors)
**Input**:
```python
frame_id: int
gps: GPSPoint:
lat: float
lon: float
covariance: np.ndarray # (2, 2) or (3, 3) - GPS uncertainty
is_user_anchor: bool # True for user-provided fixes (high confidence)
```
**Output**:
```python
bool: True if factor added
```
**Processing Flow**:
1. Convert GPS to local ENU coordinates (East-North-Up)
2. Create PriorFactor or UnaryFactor
3. Set covariance (low for user anchors, higher for LiteSAM)
4. Add to factor graph
5. Trigger optimization (immediate for user anchors)
**Covariance Settings**:
- **User anchor**: σ = 5m (high confidence)
- **LiteSAM match**: σ = 20-50m (depends on confidence)
**Test Cases**:
1. **LiteSAM GPS**: Adds absolute factor, corrects drift
2. **User anchor**: High confidence, immediately refines trajectory
3. **Multiple absolute factors**: Graph optimizes to balance all
---
### `add_altitude_prior(frame_id: int, altitude: float, covariance: float) -> bool`
**Description**: Adds altitude constraint to resolve monocular scale ambiguity.
**Called By**:
- Main processing loop (for each frame)
**Input**:
```python
frame_id: int
altitude: float # Predefined altitude in meters
covariance: float # Altitude uncertainty (e.g., 50m)
```
**Output**:
```python
bool: True if prior added
```
**Processing Flow**:
1. Create UnaryFactor for Z-coordinate
2. Set as soft constraint (not hard constraint)
3. Add to factor graph
**Purpose**:
- Resolves scale ambiguity in monocular VO
- Prevents scale drift (trajectory collapsing or exploding)
- Soft constraint allows adjustment based on absolute GPS
**Test Cases**:
1. **Without altitude prior**: Scale drifts over time
2. **With altitude prior**: Scale stabilizes
3. **Conflicting measurements**: Optimizer balances VO and altitude
---
### `optimize(iterations: int) -> OptimizationResult`
**Description**: Runs optimization to refine trajectory.
**Called By**:
- Main processing loop (incremental after each frame)
- Asynchronous refinement thread (batch optimization)
**Input**:
```python
iterations: int # Max iterations (typically 5-10 for incremental, 50-100 for batch)
```
**Output**:
```python
OptimizationResult:
converged: bool
final_error: float
iterations_used: int
optimized_frames: List[int] # Frames with updated poses
```
**Processing Details**:
- **Incremental** (iSAM2): Updates only affected nodes
- **Batch**: Re-optimizes entire trajectory when new absolute factors added
- **Robust M-estimation**: Automatically downweights outliers
**Optimization Algorithm** (Levenberg-Marquardt):
1. Linearize factor graph around current estimate
2. Solve linear system
3. Update pose estimates
4. Check convergence (error reduction < threshold)
**Test Cases**:
1. **Incremental optimization**: Fast (<100ms), local update
2. **Batch optimization**: Slower (~500ms), refines entire trajectory
3. **Convergence**: Error reduces, converges within iterations
---
### `get_trajectory() -> Dict[int, Pose]`
**Description**: Retrieves complete optimized trajectory.
**Called By**:
- F13 Result Manager (for publishing results)
- F12 Coordinate Transformer (for GPS conversion)
**Input**: None
**Output**:
```python
Dict[int, Pose]:
frame_id -> Pose:
position: np.ndarray # (x, y, z) in ENU
orientation: np.ndarray # Quaternion or rotation matrix
timestamp: datetime
```
**Processing Flow**:
1. Extract all pose estimates from graph
2. Convert to appropriate coordinate system
3. Return dictionary
**Test Cases**:
1. **After optimization**: Returns all frame poses
2. **Refined trajectory**: Poses updated after batch optimization
---
### `get_marginal_covariance(frame_id: int) -> np.ndarray`
**Description**: Gets uncertainty (covariance) of a pose estimate.
**Called By**:
- F11 Failure Recovery Coordinator (to detect high uncertainty)
**Input**:
```python
frame_id: int
```
**Output**:
```python
np.ndarray: (6, 6) covariance matrix [x, y, z, roll, pitch, yaw]
```
**Purpose**:
- Uncertainty quantification
- Trigger user input when uncertainty too high (> 50m radius)
**Test Cases**:
1. **Well-constrained pose**: Small covariance
2. **Unconstrained pose**: Large covariance
3. **After absolute factor**: Covariance reduces
---
### `create_new_chunk(chunk_id: str, start_frame_id: int) -> ChunkHandle`
**Description**: Creates a new map fragment/chunk with its own subgraph.
**Called By**:
- F02 Flight Processor (when tracking lost)
- F12 Route Chunk Manager (chunk lifecycle)
**Input**:
```python
chunk_id: str # Unique chunk identifier
start_frame_id: int # First frame in chunk
```
**Output**:
```python
ChunkHandle:
chunk_id: str
flight_id: str
start_frame_id: int
end_frame_id: Optional[int]
frames: List[int]
is_active: bool
has_anchor: bool
anchor_frame_id: Optional[int]
anchor_gps: Optional[GPSPoint]
matching_status: str # "unanchored", "matching", "anchored", "merged"
```
**Processing Flow**:
1. Create new subgraph for chunk
2. Initialize first frame pose in chunk's local coordinate system
3. Mark chunk as active
4. Return ChunkHandle
**Test Cases**:
1. **Create chunk**: Returns ChunkHandle with is_active=True
2. **Multiple chunks**: Can create multiple chunks simultaneously
3. **Chunk isolation**: Factors added to chunk don't affect other chunks
---
### `get_chunk_for_frame(frame_id: int) -> Optional[ChunkHandle]`
**Description**: Gets the chunk containing the specified frame (low-level factor graph query).
**Called By**:
- F12 Route Chunk Manager (for internal queries)
- F07 Sequential VO (to determine chunk context for factor graph operations)
**Note**: This is a low-level method for factor graph operations. For high-level chunk queries, use F12.get_active_chunk(flight_id).
**Input**:
```python
frame_id: int
```
**Output**:
```python
Optional[ChunkHandle] # Active chunk or None if frame not in any chunk
```
**Test Cases**:
1. **Frame in active chunk**: Returns ChunkHandle
2. **Frame not in chunk**: Returns None
3. **Multiple chunks**: Returns correct chunk for frame
---
### `add_relative_factor_to_chunk(chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool`
**Description**: Adds relative pose measurement to a specific chunk's subgraph.
**Called By**:
- F07 Sequential VO (chunk-scoped operations)
**Input**:
```python
chunk_id: str
frame_i: int # Previous frame ID
frame_j: int # Current frame ID
relative_pose: RelativePose
covariance: np.ndarray # (6, 6)
```
**Output**:
```python
bool: True if factor added successfully
```
**Processing Flow**:
1. Verify chunk exists and is active
2. Create BetweenFactor in chunk's subgraph
3. Apply robust kernel (Huber)
4. Add to chunk's factor graph
5. Mark chunk as needing optimization
**Test Cases**:
1. **Add to active chunk**: Factor added successfully
2. **Add to inactive chunk**: Returns False
3. **Multiple chunks**: Factors isolated to respective chunks
---
### `add_chunk_anchor(chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool`
**Description**: Adds absolute GPS anchor to a chunk, enabling global localization.
**Called By**:
- F09 Metric Refinement (after chunk LiteSAM matching)
- F11 Failure Recovery Coordinator (chunk matching)
**Input**:
```python
chunk_id: str
frame_id: int # Frame within chunk to anchor
gps: GPSPoint
covariance: np.ndarray # (2, 2) or (3, 3)
```
**Output**:
```python
bool: True if anchor added
```
**Processing Flow**:
1. Convert GPS to ENU coordinates
2. Create PriorFactor in chunk's subgraph
3. Mark chunk as anchored
4. Trigger chunk optimization
5. Enable chunk merging
**Test Cases**:
1. **Anchor unanchored chunk**: Anchor added, has_anchor=True
2. **Anchor already anchored chunk**: Updates anchor
3. **Chunk optimization**: Chunk optimized after anchor
---
### `merge_chunks(chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool`
**Description**: Merges two chunks using Sim(3) similarity transformation.
**Called By**:
- F11 Failure Recovery Coordinator (after chunk matching)
- Background optimization task
**Input**:
```python
chunk_id_1: str # Source chunk (typically newer)
chunk_id_2: str # Target chunk (typically older, merged into)
transform: Sim3Transform:
translation: np.ndarray # (3,)
rotation: np.ndarray # (3, 3) or quaternion
scale: float
```
**Output**:
```python
bool: True if merge successful
```
**Processing Flow**:
1. Verify both chunks exist and chunk_id_1 is anchored
2. Apply Sim(3) transform to all poses in chunk_id_1
3. Merge chunk_id_1's subgraph into chunk_id_2's subgraph
4. Update frame-to-chunk mapping
5. Mark chunk_id_1 as merged
6. Optimize merged graph globally
**Sim(3) Transformation**:
- Accounts for translation, rotation, and scale differences
- Critical for merging chunks with different scales (monocular VO)
- Preserves internal consistency of both chunks
**Test Cases**:
1. **Merge anchored chunks**: Chunks merged successfully
2. **Merge unanchored chunk**: Returns False
3. **Global consistency**: Merged trajectory is globally consistent
---
### `get_chunk_trajectory(chunk_id: str) -> Dict[int, Pose]`
**Description**: Retrieves optimized trajectory for a specific chunk.
**Called By**:
- F12 Route Chunk Manager (chunk state queries)
- F14 Result Manager (result publishing)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
Dict[int, Pose] # Frame ID -> Pose in chunk's local coordinate system
```
**Test Cases**:
1. **Get chunk trajectory**: Returns all poses in chunk
2. **Empty chunk**: Returns empty dict
3. **After optimization**: Returns optimized poses
---
### `get_all_chunks() -> List[ChunkHandle]`
**Description**: Retrieves all chunks in the factor graph.
**Called By**:
- F11 Failure Recovery Coordinator (chunk matching coordination)
- F12 Route Chunk Manager (chunk state queries)
**Input**: None
**Output**:
```python
List[ChunkHandle] # All chunks (active and inactive)
```
**Test Cases**:
1. **Multiple chunks**: Returns all chunks
2. **No chunks**: Returns empty list
3. **Mixed states**: Returns chunks in various states (active, anchored, merged)
---
### `optimize_chunk(chunk_id: str, iterations: int) -> OptimizationResult`
**Description**: Optimizes a specific chunk's subgraph independently.
**Called By**:
- F02 Flight Processor (after chunk anchor added)
- F12 Route Chunk Manager (periodic chunk optimization)
**Input**:
```python
chunk_id: str
iterations: int # Max iterations (typically 5-10 for incremental)
```
**Output**:
```python
OptimizationResult:
converged: bool
final_error: float
iterations_used: int
optimized_frames: List[int]
mean_reprojection_error: float
```
**Processing Flow**:
1. Extract chunk's subgraph
2. Run Levenberg-Marquardt optimization
3. Update poses in chunk's local coordinate system
4. Return optimization result
**Test Cases**:
1. **Optimize active chunk**: Chunk optimized successfully
2. **Optimize anchored chunk**: Optimization improves consistency
3. **Chunk isolation**: Other chunks unaffected
---
### `optimize_global(iterations: int) -> OptimizationResult`
**Description**: Optimizes all chunks and performs global merging.
**Called By**:
- Background optimization task (periodic)
- F11 Failure Recovery Coordinator (after chunk matching)
**Input**:
```python
iterations: int # Max iterations (typically 50-100 for global)
```
**Output**:
```python
OptimizationResult:
converged: bool
final_error: float
iterations_used: int
optimized_frames: List[int] # All frames across all chunks
mean_reprojection_error: float
```
**Processing Flow**:
1. Collect all chunks
2. For anchored chunks, transform to global coordinate system
3. Optimize merged global graph
4. Update all chunk trajectories
5. Return global optimization result
**Test Cases**:
1. **Global optimization**: All chunks optimized together
2. **Multiple anchored chunks**: Global consistency achieved
3. **Performance**: Completes within acceptable time (<500ms)
## Integration Tests
### Test 1: Incremental Trajectory Building
1. Initialize graph with first frame
2. Add relative factors from VO × 100
3. Add altitude priors × 100
4. Optimize incrementally after each frame
5. Verify smooth trajectory
### Test 2: Drift Correction with Absolute GPS
1. Build trajectory with VO only (will drift)
2. Add absolute GPS factor at frame 50
3. Optimize → trajectory corrects
4. Verify frames 1-49 also corrected (back-propagation)
### Test 3: Outlier Handling
1. Add normal relative factors
2. Add 350m outlier factor (tilt error)
3. Optimize with robust kernel
4. Verify outlier downweighted, trajectory smooth
### Test 4: User Anchor Integration
1. Processing blocked at frame 237
2. User provides anchor (high confidence)
3. add_absolute_factor(is_user_anchor=True)
4. Optimize → trajectory snaps to anchor
### Test 5: Multi-Chunk Creation and Isolation
1. Create chunk_1 with frames 1-10
2. Create chunk_2 with frames 20-30 (disconnected)
3. Add relative factors to each chunk
4. Verify chunks optimized independently
5. Verify factors isolated to respective chunks
### Test 6: Chunk Anchoring and Merging
1. Create chunk_1 (frames 1-10), chunk_2 (frames 20-30)
2. Add chunk_anchor to chunk_2 (frame 25)
3. Optimize chunk_2 → local consistency improved
4. Merge chunk_2 into chunk_1 with Sim(3) transform
5. Optimize global → both chunks globally consistent
6. Verify final trajectory coherent across chunks
### Test 7: Simultaneous Multi-Chunk Processing
1. Create 3 chunks simultaneously (disconnected segments)
2. Process frames in each chunk independently
3. Anchor each chunk asynchronously
4. Merge chunks as anchors become available
5. Verify final global trajectory consistent
## Non-Functional Requirements
### Performance
- **Incremental optimize**: < 100ms per frame (iSAM2)
- **Batch optimize**: < 500ms for 100 frames
- **get_trajectory**: < 10ms
- Real-time capable: 10 FPS processing
### Accuracy
- **Mean Reprojection Error (MRE)**: < 1.0 pixels
- **GPS accuracy**: Meet 80% < 50m, 60% < 20m criteria
- **Trajectory smoothness**: No sudden jumps (except user anchors)
### Reliability
- Numerical stability for 2000+ frame trajectories
- Graceful handling of degenerate configurations
- Robust to missing/corrupted measurements
## Dependencies
### Internal Components
- **H03 Robust Kernels**: For Huber/Cauchy loss functions
- **H02 GSD Calculator**: For coordinate conversions
### External Dependencies
- **GTSAM**: Graph optimization library
- **numpy**: Matrix operations
- **scipy**: Sparse matrix operations (optional)
## Data Models
### Pose
```python
class Pose(BaseModel):
frame_id: int
position: np.ndarray # (3,) - [x, y, z] in ENU
orientation: np.ndarray # (4,) quaternion or (3,3) rotation matrix
timestamp: datetime
covariance: Optional[np.ndarray] # (6, 6)
```
### RelativePose
```python
class RelativePose(BaseModel):
translation: np.ndarray # (3,)
rotation: np.ndarray # (3, 3) or (4,)
covariance: np.ndarray # (6, 6)
```
### OptimizationResult
```python
class OptimizationResult(BaseModel):
converged: bool
final_error: float
iterations_used: int
optimized_frames: List[int]
mean_reprojection_error: float
```
### FactorGraphConfig
```python
class FactorGraphConfig(BaseModel):
robust_kernel_type: str = "Huber" # or "Cauchy"
huber_threshold: float = 1.0 # pixels
cauchy_k: float = 0.1
isam2_relinearize_threshold: float = 0.1
isam2_relinearize_skip: int = 1
max_chunks: int = 100 # Maximum number of simultaneous chunks
chunk_merge_threshold: float = 0.1 # Error threshold for chunk merging
```
### ChunkHandle
```python
class ChunkHandle(BaseModel):
chunk_id: str
flight_id: str
start_frame_id: int
end_frame_id: Optional[int]
frames: List[int]
is_active: bool
has_anchor: bool
anchor_frame_id: Optional[int]
anchor_gps: Optional[GPSPoint]
matching_status: str # "unanchored", "matching", "anchored", "merged"
```
### Sim3Transform
```python
class Sim3Transform(BaseModel):
translation: np.ndarray # (3,) - translation vector
rotation: np.ndarray # (3, 3) rotation matrix or (4,) quaternion
scale: float # Scale factor
```
docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md
+700
View File
@@ -0,0 +1,700 @@
# Failure Recovery Coordinator
## Interface Definition
**Interface Name**: `IFailureRecoveryCoordinator`
### Interface Methods
```python
class IFailureRecoveryCoordinator(ABC):
@abstractmethod
def check_confidence(self, vo_result: RelativePose, litesam_result: Optional[AlignmentResult]) -> ConfidenceAssessment:
pass
@abstractmethod
def detect_tracking_loss(self, confidence: ConfidenceAssessment) -> bool:
pass
@abstractmethod
def start_search(self, flight_id: str, frame_id: int, estimated_gps: GPSPoint) -> SearchSession:
pass
@abstractmethod
def expand_search_radius(self, session: SearchSession) -> List[TileCoords]:
pass
@abstractmethod
def try_current_grid(self, session: SearchSession, tiles: Dict[str, np.ndarray]) -> Optional[AlignmentResult]:
pass
@abstractmethod
def mark_found(self, session: SearchSession, result: AlignmentResult) -> bool:
pass
@abstractmethod
def get_search_status(self, session: SearchSession) -> SearchStatus:
pass
@abstractmethod
def create_user_input_request(self, flight_id: str, frame_id: int, candidate_tiles: List[TileCandidate]) -> UserInputRequest:
pass
@abstractmethod
def apply_user_anchor(self, flight_id: str, frame_id: int, anchor: UserAnchor) -> bool:
pass
@abstractmethod
def create_chunk_on_tracking_loss(self, flight_id: str, frame_id: int) -> ChunkHandle:
pass
@abstractmethod
def try_chunk_semantic_matching(self, chunk_id: str) -> Optional[List[TileCandidate]]:
pass
@abstractmethod
def try_chunk_litesam_matching(self, chunk_id: str, candidate_tiles: List[TileCandidate]) -> Optional[ChunkAlignmentResult]:
pass
@abstractmethod
def merge_chunk_to_trajectory(self, chunk_id: str, alignment_result: ChunkAlignmentResult) -> bool:
pass
@abstractmethod
def process_unanchored_chunks(self, flight_id: str) -> None:
pass
```
## Component Description
### Responsibilities
- Monitor confidence metrics (inlier count, MRE, covariance)
- Detect tracking loss and trigger recovery
- Coordinate progressive tile search (1→4→9→16→25)
- Handle human-in-the-loop when all strategies exhausted
- Block flight processing when awaiting user input
- Apply user-provided anchors to Factor Graph
- **Proactive chunk creation on tracking loss**
- **Chunk semantic matching coordination**
- **Chunk LiteSAM matching with rotation sweeps**
- **Chunk merging orchestration**
- **Background chunk matching processing**
### Scope
- Confidence monitoring
- Progressive search coordination
- User input request/response handling
- Recovery strategy orchestration
- Integration point for G04, G06, G08, G09, F10
- **Chunk lifecycle and matching coordination**
- **Multi-chunk simultaneous processing**
## API Methods
### `check_confidence(vo_result: RelativePose, litesam_result: Optional[AlignmentResult]) -> ConfidenceAssessment`
**Description**: Assesses tracking confidence from VO and LiteSAM results.
**Called By**: Main processing loop (per frame)
**Input**:
```python
vo_result: RelativePose
litesam_result: Optional[AlignmentResult]
```
**Output**:
```python
ConfidenceAssessment:
overall_confidence: float # 0-1
vo_confidence: float
litesam_confidence: float
inlier_count: int
tracking_status: str # "good", "degraded", "lost"
```
**Confidence Metrics**:
- VO inlier count and ratio
- LiteSAM match confidence
- Factor graph marginal covariance
- Reprojection error
**Thresholds**:
- **Good**: VO inliers > 50, LiteSAM confidence > 0.7
- **Degraded**: VO inliers 20-50
- **Lost**: VO inliers < 20
**Test Cases**:
1. Good tracking → "good" status
2. Low overlap → "degraded"
3. Sharp turn → "lost"
---
### `detect_tracking_loss(confidence: ConfidenceAssessment) -> bool`
**Description**: Determines if tracking is lost.
**Called By**: Main processing loop
**Input**: `ConfidenceAssessment`
**Output**: `bool` - True if tracking lost
**Test Cases**:
1. Confidence good → False
2. Confidence lost → True
---
### `start_search(flight_id: str, frame_id: int, estimated_gps: GPSPoint) -> SearchSession`
**Description**: Initiates progressive search session.
**Called By**: Main processing loop (when tracking lost)
**Input**:
```python
flight_id: str
frame_id: int
estimated_gps: GPSPoint # Dead-reckoning estimate
```
**Output**:
```python
SearchSession:
session_id: str
flight_id: str
frame_id: int
center_gps: GPSPoint
current_grid_size: int # Starts at 1
max_grid_size: int # 25
found: bool
```
**Processing Flow**:
1. Create search session
2. Set center from estimated_gps
3. Set current_grid_size = 1
4. Return session
**Test Cases**:
1. Start search → session created with grid_size=1
---
### `expand_search_radius(session: SearchSession) -> List[TileCoords]`
**Description**: Expands search grid to next size (1→4→9→16→25).
**Called By**: Internal (after try_current_grid fails)
**Input**: `SearchSession`
**Output**: `List[TileCoords]` - Tiles for next grid size
**Processing Flow**:
1. Increment current_grid_size (1→4→9→16→25)
2. Call G04.expand_search_grid() to get new tiles only
3. Return new tile coordinates
**Test Cases**:
1. Expand 1→4 → returns 3 new tiles
2. Expand 4→9 → returns 5 new tiles
3. At grid_size=25 → no more expansion
---
### `try_current_grid(session: SearchSession, tiles: Dict[str, np.ndarray]) -> Optional[AlignmentResult]`
**Description**: Tries LiteSAM matching on current tile grid.
**Called By**: Internal (progressive search loop)
**Input**:
```python
session: SearchSession
tiles: Dict[str, np.ndarray] # From G04
```
**Output**: `Optional[AlignmentResult]` - Match result or None
**Processing Flow**:
1. Get UAV image for frame_id
2. For each tile in grid:
- Call G09.align_to_satellite(uav_image, tile)
- If match found with confidence > threshold:
- mark_found(session, result)
- Return result
3. Return None if no match
**Test Cases**:
1. Match on 3rd tile → returns result
2. No match in grid → returns None
---
### `mark_found(session: SearchSession, result: AlignmentResult) -> bool`
**Description**: Marks search session as successful.
**Called By**: Internal
**Input**:
```python
session: SearchSession
result: AlignmentResult
```
**Output**: `bool` - True
**Processing Flow**:
1. Set session.found = True
2. Log success (grid_size where found)
3. Resume processing
---
### `get_search_status(session: SearchSession) -> SearchStatus`
**Description**: Gets current search status.
**Called By**: F01 REST API (for status endpoint)
**Output**:
```python
SearchStatus:
current_grid_size: int
found: bool
exhausted: bool # Reached grid_size=25 without match
```
---
### `create_user_input_request(flight_id: str, frame_id: int, candidate_tiles: List[TileCandidate]) -> UserInputRequest`
**Description**: Creates user input request when all search strategies exhausted.
**Called By**: Internal (when grid_size=25 and no match)
**Input**:
```python
flight_id: str
frame_id: int
candidate_tiles: List[TileCandidate] # Top-5 from G08
```
**Output**:
```python
UserInputRequest:
request_id: str
flight_id: str
frame_id: int
uav_image: np.ndarray
candidate_tiles: List[TileCandidate]
message: str
```
**Processing Flow**:
1. Get UAV image for frame_id
2. Get top-5 candidates from G08
3. Create request
4. Send via F14 SSE → "user_input_needed" event
5. Update F02 flight_status("BLOCKED")
**Test Cases**:
1. All search failed → creates request
2. Request sent to client via SSE
---
### `apply_user_anchor(flight_id: str, frame_id: int, anchor: UserAnchor) -> bool`
**Description**: Applies user-provided GPS anchor.
**Called By**: F01 REST API (user-fix endpoint)
**Input**:
```python
flight_id: str
frame_id: int
anchor: UserAnchor:
uav_pixel: Tuple[float, float]
satellite_gps: GPSPoint
```
**Output**: `bool` - True if applied
**Processing Flow**:
1. Validate anchor data
2. Call F10.add_absolute_factor(frame_id, gps, is_user_anchor=True)
3. F10.optimize() → refines trajectory
4. Update F02 flight_status("PROCESSING")
5. Resume processing from next frame
**Test Cases**:
1. Valid anchor → applied, processing resumes
2. Invalid anchor → rejected
---
### `create_chunk_on_tracking_loss(flight_id: str, frame_id: int) -> ChunkHandle`
**Description**: Creates a new chunk proactively when tracking is lost.
**Called By**:
- F02 Flight Processor (when tracking lost detected)
**Input**:
```python
flight_id: str
frame_id: int # First frame in new chunk
```
**Output**:
```python
ChunkHandle:
chunk_id: str
flight_id: str
start_frame_id: int
end_frame_id: Optional[int]
frames: List[int]
is_active: bool
has_anchor: bool
anchor_frame_id: Optional[int]
anchor_gps: Optional[GPSPoint]
matching_status: str
```
**Processing Flow**:
1. Call F12 Route Chunk Manager.create_chunk()
2. F12 creates chunk in F10 Factor Graph Optimizer
3. Mark chunk as active
4. Return ChunkHandle
**Proactive Behavior**:
- Chunk created immediately (not waiting for matching to fail)
- Processing continues in new chunk
- Matching attempted asynchronously
**Test Cases**:
1. **Create chunk**: Chunk created successfully
2. **Proactive creation**: Chunk created before matching attempts
3. **Continue processing**: Processing continues in new chunk
---
### `try_chunk_semantic_matching(chunk_id: str) -> Optional[List[TileCandidate]]`
**Description**: Attempts semantic matching for a chunk using aggregate DINOv2 descriptor.
**Called By**:
- Internal (when chunk ready for matching)
- process_unanchored_chunks() (background task)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
Optional[List[TileCandidate]]: Top-k candidate tiles or None if matching failed
```
**Processing Flow**:
1. Get chunk images via F12.get_chunk_images()
2. Call F08 Global Place Recognition.retrieve_candidate_tiles_for_chunk()
3. F08 computes chunk descriptor and queries Faiss
4. Return candidate tiles if found, None otherwise
**Test Cases**:
1. **Chunk matching**: Returns candidate tiles
2. **Featureless terrain**: Succeeds where single-image fails
3. **No match**: Returns None
---
### `try_chunk_litesam_matching(chunk_id: str, candidate_tiles: List[TileCandidate]) -> Optional[ChunkAlignmentResult]`
**Description**: Attempts LiteSAM matching for chunk with rotation sweeps.
**Called By**:
- Internal (after chunk semantic matching succeeds)
- process_unanchored_chunks() (background task)
**Input**:
```python
chunk_id: str
candidate_tiles: List[TileCandidate] # From chunk semantic matching
```
**Output**:
```python
Optional[ChunkAlignmentResult]: Match result or None
```
**Processing Flow**:
1. Get chunk images via F12.get_chunk_images()
2. For each candidate tile:
- Get tile from F04 Satellite Data Manager
- Call F06.try_chunk_rotation_steps() (12 rotations: 0°, 30°, ..., 330°)
- F06 calls F09.align_chunk_to_satellite() for each rotation
- If match found with confidence > threshold:
- Return ChunkAlignmentResult
3. Return None if no match found
**Rotation Sweeps**:
- Critical for chunks from sharp turns (unknown orientation)
- Tries all 12 rotation angles
- Returns best matching rotation
**Test Cases**:
1. **Match on first tile**: Returns alignment result
2. **Match on 3rd tile**: Returns alignment result
3. **Match at 120° rotation**: Returns result with correct rotation angle
4. **No match**: Returns None
---
### `merge_chunk_to_trajectory(chunk_id: str, alignment_result: ChunkAlignmentResult) -> bool`
**Description**: Merges chunk into main trajectory after successful matching.
**Called By**:
- Internal (after chunk LiteSAM matching succeeds)
**Input**:
```python
chunk_id: str
alignment_result: ChunkAlignmentResult:
chunk_center_gps: GPSPoint
transform: Sim3Transform
confidence: float
```
**Output**:
```python
bool: True if merge successful
```
**Processing Flow**:
1. Get chunk anchor frame (middle frame or best frame)
2. Call F12.mark_chunk_anchored() with GPS (F12 coordinates with F10)
3. Find target chunk (previous chunk or main trajectory)
4. Call F12.merge_chunks(chunk_id, target_chunk_id, transform) (F12 coordinates with F10)
5. F12 handles chunk state updates (deactivation, status updates)
6. F10 optimizes merged graph globally (via F12.merge_chunks())
7. Return True
**Sim(3) Transform**:
- Translation: GPS offset
- Rotation: From alignment result
- Scale: Resolved from altitude and GSD
**Test Cases**:
1. **Merge chunk**: Chunk merged successfully
2. **Global consistency**: Merged trajectory globally consistent
3. **Multiple chunks**: Can merge multiple chunks sequentially
---
### `process_unanchored_chunks(flight_id: str) -> None`
**Description**: Background task that periodically attempts matching for unanchored chunks.
**Called By**:
- Background thread (periodic, e.g., every 5 seconds)
- F02 Flight Processor (after frame processing)
**Input**:
```python
flight_id: str
```
**Output**: None (runs asynchronously)
**Processing Flow**:
```
while flight_active:
unanchored_chunks = F12.get_chunks_for_matching(flight_id)
for chunk in unanchored_chunks:
if F12.is_chunk_ready_for_matching(chunk.chunk_id):
F12.mark_chunk_matching(chunk.chunk_id)
candidates = try_chunk_semantic_matching(chunk.chunk_id)
if candidates:
alignment = try_chunk_litesam_matching(chunk.chunk_id, candidates)
if alignment:
merge_chunk_to_trajectory(chunk.chunk_id, alignment)
sleep(5 seconds)
```
**Background Processing**:
- Runs asynchronously, doesn't block frame processing
- Periodically checks for ready chunks
- Attempts matching and merging
- Reduces user input requests
**Test Cases**:
1. **Background matching**: Unanchored chunks matched asynchronously
2. **Chunk merging**: Chunks merged when matches found
3. **Non-blocking**: Frame processing continues during matching
## Integration Tests
### Test 1: Progressive Search Flow
1. Tracking lost detected
2. start_search() → grid_size=1
3. try_current_grid(1 tile) → no match
4. expand_search_radius() → grid_size=4
5. try_current_grid(4 tiles) → match found
6. mark_found() → success
### Test 2: Full Search Exhaustion
1. start_search()
2. try grids: 1→4→9→16→25, all fail
3. create_user_input_request()
4. User provides anchor
5. apply_user_anchor() → processing resumes
### Test 3: Confidence Monitoring
1. Normal frames → confidence good
2. Low overlap frame → confidence degraded
3. Sharp turn → tracking lost, trigger search
### Test 4: Proactive Chunk Creation
1. Tracking lost detected
2. create_chunk_on_tracking_loss() → chunk created immediately
3. Processing continues in new chunk
4. Verify chunk matching attempted asynchronously
### Test 5: Chunk Semantic Matching
1. Build chunk with 10 images (plain field)
2. try_chunk_semantic_matching() → returns candidate tiles
3. Verify chunk matching succeeds where single-image fails
### Test 6: Chunk LiteSAM Matching with Rotation
1. Build chunk with unknown orientation
2. try_chunk_litesam_matching() with candidate tiles
3. Rotation sweep finds match at 120°
4. Returns ChunkAlignmentResult with correct GPS
### Test 7: Chunk Merging
1. Create chunk_1 (frames 1-10), chunk_2 (frames 20-30)
2. Anchor chunk_2 via chunk matching
3. merge_chunk_to_trajectory() → chunks merged
4. Verify global trajectory consistent
### Test 8: Background Chunk Processing
1. Create 3 unanchored chunks
2. process_unanchored_chunks() runs in background
3. Chunks matched and merged asynchronously
4. Frame processing continues uninterrupted
## Non-Functional Requirements
### Performance
- **check_confidence**: < 10ms
- **Progressive search (25 tiles)**: < 1.5s total
- **User input latency**: < 500ms from creation to SSE event
### Reliability
- Always exhausts all search strategies before requesting user input
- Guarantees processing block when awaiting user input
- Graceful recovery from all failure modes
## Dependencies
### Internal Components
- F04 Satellite Data Manager (tile grids)
- F06 Image Rotation Manager (rotation sweep and chunk rotation)
- F08 Global Place Recognition (candidates and chunk semantic matching)
- F09 Metric Refinement (LiteSAM and chunk LiteSAM matching)
- F10 Factor Graph Optimizer (anchor application and chunk merging)
- F02 Flight Manager (status updates)
- F14 SSE Event Streamer (user input events)
- F12 Route Chunk Manager (chunk lifecycle)
### External Dependencies
- None
## Data Models
### ConfidenceAssessment
```python
class ConfidenceAssessment(BaseModel):
overall_confidence: float
vo_confidence: float
litesam_confidence: float
inlier_count: int
tracking_status: str
```
### SearchSession
```python
class SearchSession(BaseModel):
session_id: str
flight_id: str
frame_id: int
center_gps: GPSPoint
current_grid_size: int
max_grid_size: int
found: bool
exhausted: bool
```
### UserInputRequest
```python
class UserInputRequest(BaseModel):
request_id: str
flight_id: str
frame_id: int
uav_image: np.ndarray
candidate_tiles: List[TileCandidate]
message: str
created_at: datetime
```
### UserAnchor
```python
class UserAnchor(BaseModel):
uav_pixel: Tuple[float, float]
satellite_gps: GPSPoint
confidence: float = 1.0
```
### ChunkHandle
```python
class ChunkHandle(BaseModel):
chunk_id: str
flight_id: str
start_frame_id: int
end_frame_id: Optional[int]
frames: List[int]
is_active: bool
has_anchor: bool
anchor_frame_id: Optional[int]
anchor_gps: Optional[GPSPoint]
matching_status: str # "unanchored", "matching", "anchored", "merged"
```
### ChunkAlignmentResult
```python
class ChunkAlignmentResult(BaseModel):
matched: bool
chunk_id: str
chunk_center_gps: GPSPoint
rotation_angle: float
confidence: float
inlier_count: int
transform: Sim3Transform
```
### Sim3Transform
```python
class Sim3Transform(BaseModel):
translation: np.ndarray # (3,)
rotation: np.ndarray # (3, 3) or (4,) quaternion
scale: float
```
docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md
+617
View File
@@ -0,0 +1,617 @@
# Route Chunk Manager
## Interface Definition
**Interface Name**: `IRouteChunkManager`
### Interface Methods
```python
class IRouteChunkManager(ABC):
@abstractmethod
def create_chunk(self, flight_id: str, start_frame_id: int) -> ChunkHandle:
pass
@abstractmethod
def add_frame_to_chunk(self, chunk_id: str, frame_id: int, vo_result: RelativePose) -> bool:
pass
@abstractmethod
def get_chunk_frames(self, chunk_id: str) -> List[int]:
pass
@abstractmethod
def get_chunk_images(self, chunk_id: str) -> List[np.ndarray]:
pass
@abstractmethod
def get_chunk_composite_descriptor(self, chunk_id: str) -> np.ndarray:
pass
@abstractmethod
def get_chunk_bounds(self, chunk_id: str) -> ChunkBounds:
pass
@abstractmethod
def is_chunk_ready_for_matching(self, chunk_id: str) -> bool:
pass
@abstractmethod
def mark_chunk_anchored(self, chunk_id: str, frame_id: int, gps: GPSPoint) -> bool:
pass
@abstractmethod
def get_chunks_for_matching(self, flight_id: str) -> List[ChunkHandle]:
pass
@abstractmethod
def get_active_chunk(self, flight_id: str) -> Optional[ChunkHandle]:
pass
@abstractmethod
def deactivate_chunk(self, chunk_id: str) -> bool:
pass
@abstractmethod
def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool:
pass
@abstractmethod
def mark_chunk_matching(self, chunk_id: str) -> bool:
pass
```
## Component Description
### Responsibilities
- Manage chunk lifecycle (creation, activation, deactivation, merging)
- Track chunk state (frames, anchors, matching status)
- Coordinate chunk semantic matching and LiteSAM matching
- Provide chunk representations for matching (composite images, descriptors)
- Determine chunk readiness for matching (min frames, consistency)
### Scope
- Chunk lifecycle management
- Chunk state tracking
- Chunk representation generation (descriptors, bounds)
- Integration point for chunk matching coordination
## API Methods
### `create_chunk(flight_id: str, start_frame_id: int) -> ChunkHandle`
**Description**: Creates a new route chunk and initializes it in the factor graph.
**Called By**:
- F02 Flight Processor (when tracking lost)
- F11 Failure Recovery Coordinator (proactive chunk creation)
**Input**:
```python
flight_id: str
start_frame_id: int # First frame in chunk
```
**Output**:
```python
ChunkHandle:
chunk_id: str
flight_id: str
start_frame_id: int
end_frame_id: Optional[int]
frames: List[int]
is_active: bool
has_anchor: bool
anchor_frame_id: Optional[int]
anchor_gps: Optional[GPSPoint]
matching_status: str
```
**Processing Flow**:
1. Generate unique chunk_id
2. Call F10 Factor Graph Optimizer.create_new_chunk()
3. Initialize chunk state (unanchored, active)
4. Store chunk metadata
5. Return ChunkHandle
**Test Cases**:
1. **Create chunk**: Returns ChunkHandle with is_active=True
2. **Multiple chunks**: Can create multiple chunks for same flight
3. **Chunk initialization**: Chunk initialized in factor graph
---
### `add_frame_to_chunk(chunk_id: str, frame_id: int, vo_result: RelativePose) -> bool`
**Description**: Adds a frame to an existing chunk with its VO result.
**Called By**:
- F02 Flight Processor (during frame processing)
**Input**:
```python
chunk_id: str
frame_id: int
vo_result: RelativePose # From F07 Sequential VO
```
**Output**:
```python
bool: True if frame added successfully
```
**Processing Flow**:
1. Verify chunk exists and is active
2. Add frame_id to chunk's frames list
3. Store vo_result for chunk
4. Call F10.add_relative_factor_to_chunk()
5. Update chunk's end_frame_id
6. Check if chunk ready for matching
**Test Cases**:
1. **Add frame to active chunk**: Frame added successfully
2. **Add frame to inactive chunk**: Returns False
3. **Chunk growth**: Chunk frames list updated
---
### `get_chunk_frames(chunk_id: str) -> List[int]`
**Description**: Retrieves list of frame IDs in a chunk.
**Called By**:
- F08 Global Place Recognition (for chunk descriptor computation)
- F09 Metric Refinement (for chunk LiteSAM matching)
- F11 Failure Recovery Coordinator (chunk state queries)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
List[int] # Frame IDs in chunk, ordered by sequence
```
**Test Cases**:
1. **Get frames**: Returns all frames in chunk
2. **Empty chunk**: Returns empty list
3. **Ordered frames**: Frames returned in sequence order
---
### `get_chunk_images(chunk_id: str) -> List[np.ndarray]`
**Description**: Retrieves images for all frames in a chunk.
**Called By**:
- F08 Global Place Recognition (chunk descriptor computation)
- F09 Metric Refinement (chunk LiteSAM matching)
- F06 Image Rotation Manager (chunk rotation)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
List[np.ndarray] # Images for each frame in chunk
```
**Processing Flow**:
1. Get chunk frames via get_chunk_frames()
2. Load images from F05 Image Input Pipeline
3. Return list of images
**Test Cases**:
1. **Get images**: Returns all images in chunk
2. **Image loading**: Images loaded correctly from pipeline
3. **Order consistency**: Images match frame order
---
### `get_chunk_composite_descriptor(chunk_id: str) -> np.ndarray`
**Description**: Computes aggregate DINOv2 descriptor for chunk (for semantic matching).
**Called By**:
- F08 Global Place Recognition (chunk semantic matching)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
```
**Processing Flow**:
1. Get chunk images via get_chunk_images()
2. Call F08.compute_chunk_descriptor(chunk_images) → aggregate descriptor
3. Return composite descriptor
**Delegation**:
- Delegates to F08.compute_chunk_descriptor() for descriptor computation
- F08 handles aggregation logic (mean, VLAD, or max)
- Single source of truth for chunk descriptor computation
**Test Cases**:
1. **Compute descriptor**: Returns aggregated descriptor
2. **Multiple images**: Descriptor aggregates correctly
3. **Descriptor quality**: More robust than single-image descriptor
---
### `get_chunk_bounds(chunk_id: str) -> ChunkBounds`
**Description**: Estimates GPS bounds of a chunk based on VO trajectory.
**Called By**:
- F11 Failure Recovery Coordinator (for tile search area)
- F04 Satellite Data Manager (for tile prefetching)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
ChunkBounds:
estimated_center: GPSPoint
estimated_radius: float # meters
confidence: float
```
**Processing Flow**:
1. Get chunk trajectory from F10.get_chunk_trajectory()
2. If chunk has anchor:
- Use anchor GPS as center
- Compute radius from trajectory extent
3. If chunk unanchored:
- Estimate center from VO trajectory (relative to start)
- Use dead-reckoning estimate
- Lower confidence
4. Return ChunkBounds
**Test Cases**:
1. **Anchored chunk**: Returns accurate bounds with high confidence
2. **Unanchored chunk**: Returns estimated bounds with lower confidence
3. **Bounds calculation**: Radius computed from trajectory extent
---
### `is_chunk_ready_for_matching(chunk_id: str) -> bool`
**Description**: Determines if chunk has enough frames and consistency for matching.
**Called By**:
- F11 Failure Recovery Coordinator (before attempting matching)
- Background matching task
**Input**:
```python
chunk_id: str
```
**Output**:
```python
bool: True if chunk ready for matching
```
**Criteria**:
- **Min frames**: >= 5 frames (configurable)
- **Max frames**: <= 20 frames (configurable, prevents oversized chunks)
- **Internal consistency**: VO factors have reasonable inlier counts
- **Not already matched**: matching_status != "anchored" or "merged"
**Test Cases**:
1. **Ready chunk**: 10 frames, good consistency → True
2. **Too few frames**: 3 frames → False
3. **Already anchored**: has_anchor=True → False
---
### `mark_chunk_anchored(chunk_id: str, frame_id: int, gps: GPSPoint) -> bool`
**Description**: Marks chunk as anchored with GPS coordinate.
**Called By**:
- F11 Failure Recovery Coordinator (after successful chunk matching)
**Input**:
```python
chunk_id: str
frame_id: int # Frame within chunk that was anchored
gps: GPSPoint
```
**Output**:
```python
bool: True if marked successfully
```
**Processing Flow**:
1. Verify chunk exists
2. Call F10.add_chunk_anchor()
3. Update chunk state (has_anchor=True, anchor_frame_id, anchor_gps)
4. Update matching_status to "anchored"
5. Trigger chunk optimization
**Test Cases**:
1. **Mark anchored**: Chunk state updated correctly
2. **Anchor in factor graph**: F10 anchor added
3. **Chunk optimization**: Chunk optimized after anchoring
---
### `get_chunks_for_matching(flight_id: str) -> List[ChunkHandle]`
**Description**: Retrieves all unanchored chunks ready for matching.
**Called By**:
- F11 Failure Recovery Coordinator (background matching task)
- Background processing task
**Input**:
```python
flight_id: str
```
**Output**:
```python
List[ChunkHandle] # Unanchored chunks ready for matching
```
**Processing Flow**:
1. Get all chunks for flight_id
2. Filter chunks where:
- has_anchor == False
- is_chunk_ready_for_matching() == True
- matching_status == "unanchored" or "matching"
3. Return filtered list
**Test Cases**:
1. **Get unanchored chunks**: Returns ready chunks
2. **Filter criteria**: Only returns chunks meeting criteria
3. **Empty result**: Returns empty list if no ready chunks
---
### `get_active_chunk(flight_id: str) -> Optional[ChunkHandle]`
**Description**: Gets the currently active chunk for a flight.
**Called By**:
- F02 Flight Processor (before processing frame)
**Input**:
```python
flight_id: str
```
**Output**:
```python
Optional[ChunkHandle] # Active chunk or None
```
**Test Cases**:
1. **Get active chunk**: Returns active chunk
2. **No active chunk**: Returns None
3. **Multiple chunks**: Returns only active chunk
---
### `deactivate_chunk(chunk_id: str) -> bool`
**Description**: Deactivates a chunk (typically after merging or completion).
**Called By**:
- F11 Failure Recovery Coordinator (after chunk merged)
- F02 Flight Processor (chunk lifecycle)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
bool: True if deactivated successfully
```
**Processing Flow**:
1. Verify chunk exists
2. Update chunk state (is_active=False)
3. Update matching_status to "merged" if merged
4. Return True
**Test Cases**:
1. **Deactivate chunk**: Chunk marked inactive
2. **After merge**: Matching status updated to "merged"
---
### `merge_chunks(chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool`
**Description**: Coordinates chunk merging by validating chunks, calling F10 for factor graph merge, and updating chunk states.
**Called By**:
- F11 Failure Recovery Coordinator (after successful chunk matching)
**Input**:
```python
chunk_id_1: str # Source chunk (typically newer, to be merged)
chunk_id_2: str # Target chunk (typically older, merged into)
transform: Sim3Transform:
translation: np.ndarray # (3,)
rotation: np.ndarray # (3, 3) or quaternion
scale: float
```
**Output**:
```python
bool: True if merge successful
```
**Processing Flow**:
1. Verify both chunks exist
2. Verify chunk_id_1 is anchored (has_anchor=True)
3. Validate chunks can be merged (not already merged, not same chunk)
4. Call F10.merge_chunks(chunk_id_1, chunk_id_2, transform)
5. Update chunk_id_1 state:
- Set is_active=False
- Set matching_status="merged"
- Call deactivate_chunk(chunk_id_1)
6. Update chunk_id_2 state (if needed)
7. Return True
**Validation**:
- Both chunks must exist
- chunk_id_1 must be anchored
- chunk_id_1 must not already be merged
- chunk_id_1 and chunk_id_2 must be different
**Test Cases**:
1. **Merge anchored chunks**: Chunks merged successfully, chunk_id_1 deactivated
2. **Merge unanchored chunk**: Returns False (validation fails)
3. **Merge already merged chunk**: Returns False (validation fails)
4. **State updates**: chunk_id_1 marked as merged and deactivated
---
### `mark_chunk_matching(chunk_id: str) -> bool`
**Description**: Explicitly marks chunk as being matched (updates matching_status to "matching").
**Called By**:
- F11 Failure Recovery Coordinator (when chunk matching starts)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
bool: True if marked successfully
```
**Processing Flow**:
1. Verify chunk exists
2. Verify chunk is unanchored (has_anchor=False)
3. Update matching_status to "matching"
4. Return True
**State Transition**:
- `unanchored` → `matching` (explicit transition)
**Test Cases**:
1. **Mark unanchored chunk**: Status updated to "matching"
2. **Mark already anchored chunk**: Returns False (invalid state)
3. **Mark non-existent chunk**: Returns False
## Integration Tests
### Test 1: Chunk Lifecycle
1. create_chunk() → chunk created
2. add_frame_to_chunk() × 10 → 10 frames added
3. is_chunk_ready_for_matching() → True
4. mark_chunk_anchored() → chunk anchored
5. deactivate_chunk() → chunk deactivated
### Test 2: Chunk Descriptor Computation
1. Create chunk with 10 frames
2. get_chunk_images() → 10 images
3. get_chunk_composite_descriptor() → aggregated descriptor
4. Verify descriptor more robust than single-image descriptor
### Test 3: Multiple Chunks
1. Create chunk_1 (frames 1-10)
2. Create chunk_2 (frames 20-30)
3. get_chunks_for_matching() → returns both chunks
4. mark_chunk_anchored(chunk_1) → chunk_1 anchored
5. get_chunks_for_matching() → returns only chunk_2
### Test 4: Chunk Merging
1. Create chunk_1 (frames 1-10), chunk_2 (frames 20-30)
2. Anchor chunk_1 via mark_chunk_anchored()
3. merge_chunks(chunk_1, chunk_2, transform) → chunks merged
4. Verify chunk_1 marked as merged and deactivated
5. Verify F10 merge_chunks() called
### Test 5: Chunk Matching Status
1. Create chunk
2. mark_chunk_matching() → status updated to "matching"
3. mark_chunk_anchored() → status updated to "anchored"
4. Verify explicit state transitions
## Non-Functional Requirements
### Performance
- **create_chunk**: < 10ms
- **add_frame_to_chunk**: < 5ms
- **get_chunk_composite_descriptor**: < 3s for 20 images (async)
- **get_chunk_bounds**: < 10ms
### Reliability
- Chunk state persisted across restarts
- Graceful handling of missing frames
- Thread-safe chunk operations
## Dependencies
### Internal Components
- **F10 Factor Graph Optimizer**: Chunk creation and factor management
- **F05 Image Input Pipeline**: Image retrieval
- **F08 Global Place Recognition**: Descriptor computation
- **F07 Sequential VO**: VO results for chunk building
### External Dependencies
- **numpy**: Array operations
## Data Models
### ChunkHandle
```python
class ChunkHandle(BaseModel):
chunk_id: str
flight_id: str
start_frame_id: int
end_frame_id: Optional[int]
frames: List[int]
is_active: bool
has_anchor: bool
anchor_frame_id: Optional[int]
anchor_gps: Optional[GPSPoint]
matching_status: str # "unanchored", "matching", "anchored", "merged"
```
### ChunkBounds
```python
class ChunkBounds(BaseModel):
estimated_center: GPSPoint
estimated_radius: float # meters
confidence: float # 0.0 to 1.0
```
### ChunkConfig
```python
class ChunkConfig(BaseModel):
min_frames_for_matching: int = 5
max_frames_per_chunk: int = 20
descriptor_aggregation: str = "mean" # "mean", "vlad", "max"
```
### Sim3Transform
```python
class Sim3Transform(BaseModel):
translation: np.ndarray # (3,) - translation vector
rotation: np.ndarray # (3, 3) rotation matrix or (4,) quaternion
scale: float # Scale factor
```
docs/02_components/gps_denied_12_coordinate_transformer/coordinate_transformer_spec.md → docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md
+147 -9
View File
@@ -8,6 +8,24 @@
```python
class ICoordinateTransformer(ABC):
# ENU Origin Management
@abstractmethod
def set_enu_origin(self, origin_gps: GPSPoint) -> None:
pass
@abstractmethod
def get_enu_origin(self) -> GPSPoint:
pass
@abstractmethod
def gps_to_enu(self, gps: GPSPoint) -> Tuple[float, float, float]:
pass
@abstractmethod
def enu_to_gps(self, enu: Tuple[float, float, float]) -> GPSPoint:
pass
# Pixel/GPS Conversions
@abstractmethod
def pixel_to_gps(self, pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint:
pass
@@ -48,12 +66,131 @@ class ICoordinateTransformer(ABC):
## API Methods
### ENU Origin Management
#### `set_enu_origin(origin_gps: GPSPoint) -> None`
**Description**: Sets the ENU (East-North-Up) coordinate system origin. Called once during flight creation using the flight's start_gps.
**Called By**:
- F02 Flight Processor (during create_flight)
**Input**:
```python
origin_gps: GPSPoint
lat: float # Origin latitude in WGS84
lon: float # Origin longitude in WGS84
```
**Processing Flow**:
1. Store origin_gps as ENU origin
2. Precompute conversion factors for lat/lon to meters
3. All subsequent ENU operations use this origin
**Important**: The ENU origin is set to the flight's start_gps during flight creation and remains constant for the entire flight duration.
**Test Cases**:
1. **Set origin**: Store origin GPS
2. **Subsequent gps_to_enu calls**: Use set origin
3. **Multiple flights**: Each flight has independent origin
---
#### `get_enu_origin() -> GPSPoint`
**Description**: Returns the currently set ENU origin.
**Called By**:
- F10 Factor Graph Optimizer (for coordinate checks)
- F13 Result Manager (for reference)
**Output**:
```python
GPSPoint: The ENU origin (flight start_gps)
```
**Error Conditions**:
- Raises `OriginNotSetError` if called before set_enu_origin()
**Test Cases**:
1. **After set_enu_origin**: Returns stored origin
2. **Before set_enu_origin**: Raises error
---
#### `gps_to_enu(gps: GPSPoint) -> Tuple[float, float, float]`
**Description**: Converts GPS coordinates to ENU (East, North, Up) relative to the set origin.
**Called By**:
- F10 Factor Graph Optimizer (for absolute factors)
- Internal (for pixel_to_gps)
**Input**:
```python
gps: GPSPoint
lat: float
lon: float
```
**Output**:
```python
Tuple[float, float, float]: (east, north, up) in meters relative to origin
```
**Algorithm**:
1. Compute delta_lat = gps.lat - origin.lat
2. Compute delta_lon = gps.lon - origin.lon
3. east = delta_lon × cos(origin.lat) × 111319.5 # meters/degree at equator
4. north = delta_lat × 111319.5
5. up = 0 (or computed from elevation if available)
**Test Cases**:
1. **Origin GPS**: Returns (0, 0, 0)
2. **1km East**: Returns (~1000, 0, 0)
3. **1km North**: Returns (0, ~1000, 0)
4. **Diagonal**: Returns correct east/north components
---
#### `enu_to_gps(enu: Tuple[float, float, float]) -> GPSPoint`
**Description**: Converts ENU coordinates back to GPS.
**Called By**:
- F10 Factor Graph Optimizer (for get_trajectory)
- F13 Result Manager (for publishing GPS results)
**Input**:
```python
enu: Tuple[float, float, float] # (east, north, up) in meters
```
**Output**:
```python
GPSPoint: WGS84 coordinates
```
**Algorithm**:
1. delta_lon = east / (cos(origin.lat) × 111319.5)
2. delta_lat = north / 111319.5
3. lat = origin.lat + delta_lat
4. lon = origin.lon + delta_lon
**Test Cases**:
1. **Origin (0, 0, 0)**: Returns origin GPS
2. **Round-trip**: gps → enu → gps matches original (within precision)
---
### Pixel/GPS Conversions
### `pixel_to_gps(pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint`
**Description**: Converts pixel coordinates to GPS using camera pose and ground plane assumption.
**Called By**:
- G13 Result Manager (for frame center GPS)
- F13 Result Manager (for frame center GPS)
- Internal (for image_object_to_gps)
**Input**:
@@ -132,7 +269,7 @@ Tuple[float, float]: (x, y) pixel coordinates
**Called By**:
- External object detection system (provides pixel coordinates)
- G13 Result Manager (converts objects to GPS for output)
- F13 Result Manager (converts objects to GPS for output)
**Input**:
```python
@@ -146,8 +283,8 @@ GPSPoint: GPS coordinates of object center
```
**Processing Flow**:
1. Get frame_pose from G10 Factor Graph
2. Get camera_params from G16 Configuration Manager
1. Get frame_pose from F10 Factor Graph
2. Get camera_params from F17 Configuration Manager
3. Get altitude from configuration
4. Call pixel_to_gps(object_pixel, frame_pose, camera_params, altitude)
5. Return GPS
@@ -172,7 +309,7 @@ GPSPoint: GPS coordinates of object center
**Called By**:
- Internal (for pixel_to_gps)
- G09 Metric Refinement (for scale calculations)
- F09 Metric Refinement (for scale calculations)
- H02 GSD Calculator (may delegate to)
**Input**:
@@ -212,8 +349,8 @@ GSD = (altitude * sensor_width) / (focal_length * image_width)
**Description**: Applies homography or affine transformation to list of points.
**Called By**:
- G06 Image Rotation Manager (for rotation transforms)
- G09 Metric Refinement (homography application)
- F06 Image Rotation Manager (for rotation transforms)
- F09 Metric Refinement (homography application)
**Input**:
```python
@@ -282,8 +419,9 @@ List[Tuple[float, float]]: Transformed points
## Dependencies
### Internal Components
- **G10 Factor Graph Optimizer**: For frame poses
- **G16 Configuration Manager**: For camera parameters
- **F10 Factor Graph Optimizer**: For frame poses
- **F12 Route Chunk Manager**: For chunk context (chunk-aware coordinate transforms)
- **F17 Configuration Manager**: For camera parameters
- **H01 Camera Model**: For projection operations
- **H02 GSD Calculator**: For GSD calculations
- **H06 Web Mercator Utils**: For coordinate conversions
docs/02_components/gps_denied_13_result_manager/result_manager_spec.md → docs/02_components/14_result_manager/result_manager_spec.md
+11 -11
View File
@@ -35,7 +35,7 @@ class IResultManager(ABC):
- Manage trajectory results per flight
- Track frame refinements and changes
- Trigger per-frame Route API updates via G03
- Send incremental updates via G14 SSE
- Send incremental updates via F14 SSE
- Maintain result versioning for audit trail
- Convert optimized poses to GPS coordinates
@@ -54,7 +54,7 @@ class IResultManager(ABC):
**Called By**:
- Main processing loop (after each frame)
- G10 Factor Graph (after refinement)
- F10 Factor Graph (after refinement)
**Input**:
```python
@@ -116,7 +116,7 @@ frame_id: int
**Description**: Retrieves all results for a flight.
**Called By**:
- G01 REST API (results endpoint)
- F01 REST API (results endpoint)
- Testing/validation
**Input**: `flight_id: str`
@@ -139,7 +139,7 @@ FlightResults:
**Description**: Marks frames as refined after batch optimization.
**Called By**:
- G10 Factor Graph (after asynchronous refinement)
- F10 Factor Graph (after asynchronous refinement)
**Input**:
```python
@@ -151,7 +151,7 @@ frame_ids: List[int] # Frames with updated poses
**Processing Flow**:
1. For each frame_id:
- Get refined pose from G10
- Get refined pose from F10
- Convert to GPS via G12
- Update result with refined=True
- publish_to_route_api()
@@ -167,7 +167,7 @@ frame_ids: List[int] # Frames with updated poses
**Description**: Gets frames changed since timestamp (for incremental updates).
**Called By**:
- G14 SSE Event Streamer (for reconnection replay)
- F14 SSE Event Streamer (for reconnection replay)
**Input**:
```python
@@ -187,7 +187,7 @@ since: datetime
1. Process frame 237
2. update_frame_result() → stores result
3. Verify publish_to_route_api() called
4. Verify G14 SSE event sent
4. Verify F14 SSE event sent
### Test 2: Batch Refinement
1. Process 100 frames
@@ -219,10 +219,10 @@ since: datetime
### Internal Components
- G03 Route API Client
- G10 Factor Graph Optimizer
- G12 Coordinate Transformer
- G14 SSE Event Streamer
- G17 Database Layer
- F10 Factor Graph Optimizer
- F12 Coordinate Transformer
- F14 SSE Event Streamer
- F17 Database Layer
### External Dependencies
- None
docs/02_components/gps_denied_14_sse_event_streamer/sse_event_streamer_spec.md → docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md
+6 -6
View File
@@ -56,7 +56,7 @@ class ISSEEventStreamer(ABC):
**Description**: Creates SSE connection for a client.
**Called By**: G01 REST API (GET /stream endpoint)
**Called By**: F01 REST API (GET /stream endpoint)
**Output**:
```python
@@ -86,7 +86,7 @@ StreamConnection:
**Description**: Sends frame_processed event.
**Called By**: G13 Result Manager
**Called By**: F13 Result Manager
**Event Format**:
```json
@@ -114,7 +114,7 @@ StreamConnection:
**Description**: Sends search_expanded event.
**Called By**: G11 Failure Recovery Coordinator
**Called By**: F11 Failure Recovery Coordinator
**Event Format**:
```json
@@ -134,7 +134,7 @@ StreamConnection:
**Description**: Sends user_input_needed event.
**Called By**: G11 Failure Recovery Coordinator
**Called By**: F11 Failure Recovery Coordinator
**Event Format**:
```json
@@ -154,7 +154,7 @@ StreamConnection:
**Description**: Sends frame_refined event.
**Called By**: G13 Result Manager
**Called By**: F13 Result Manager
**Event Format**:
```json
@@ -174,7 +174,7 @@ StreamConnection:
**Description**: Closes SSE connection.
**Called By**: G01 REST API (on client disconnect)
**Called By**: F01 REST API (on client disconnect)
## Integration Tests
docs/02_components/gps_denied_15_model_manager/model_manager_spec.md → docs/02_components/16_model_manager/model_manager_spec.md
+4 -4
View File
@@ -55,7 +55,7 @@ class IModelManager(ABC):
**Description**: Loads model in specified format.
**Called By**: G02 Flight Manager (during initialization)
**Called By**: F02 Flight Manager (during initialization)
**Input**:
```python
@@ -84,9 +84,9 @@ model_format: str # "tensorrt", "onnx", "pytorch"
**Description**: Gets inference engine for a model.
**Called By**:
- G07 Sequential VO (SuperPoint, LightGlue)
- G08 Global Place Recognition (DINOv2)
- G09 Metric Refinement (LiteSAM)
- F07 Sequential VO (SuperPoint, LightGlue)
- F08 Global Place Recognition (DINOv2)
- F09 Metric Refinement (LiteSAM)
**Output**:
```python
docs/02_components/gps_denied_16_configuration_manager/configuration_manager_spec.md → docs/02_components/17_configuration_manager/configuration_manager_spec.md
+1 -1
View File
@@ -111,7 +111,7 @@ CameraParameters:
**Description**: Gets flight-specific configuration.
**Called By**: G02 Flight Manager
**Called By**: F02 Flight Manager
**Output**:
```python
docs/02_components/chunking_system_assessment.md
+440
View File
@@ -0,0 +1,440 @@
# Chunking System Assessment: F12 Route Chunk Manager and Integration
## Executive Summary
The chunking system implements the Atlas multi-map architecture from the solution document, with **F12 Route Chunk Manager** serving as the high-level coordinator. The system demonstrates **strong coherence** with clear separation of concerns between F12 (lifecycle), F10 (factor graph), and F11 (matching coordination). However, there are some **integration inconsistencies** and **missing lifecycle transitions** that should be addressed.
**Overall Assessment: 8/10** - Well-designed with minor gaps
---
## 1. Architecture Coherence
### 1.1 Separation of Concerns ✅
The chunking system properly separates responsibilities:
| Component | Responsibility | Level |
|-----------|---------------|-------|
| **F12 Route Chunk Manager** | Chunk lifecycle, state tracking, matching coordination | High-level (flight context) |
| **F10 Factor Graph Optimizer** | Chunk subgraphs, factor management, optimization | Low-level (factor graph) |
| **F11 Failure Recovery Coordinator** | Chunk creation triggers, matching orchestration | Coordination layer |
| **F02 Flight Processor** | Chunk-aware frame processing | Application layer |
**Assessment:** ✅ **Excellent separation** - Clear boundaries, no overlap
### 1.2 Chunk Lifecycle Completeness
**Lifecycle States:**
- `unanchored` → `matching` → `anchored` → `merged`
**Lifecycle Methods:**
| State Transition | F12 Method | F10 Method | Status |
|------------------|------------|------------|--------|
| **Creation** | `create_chunk()` | `create_new_chunk()` | ✅ Complete |
| **Building** | `add_frame_to_chunk()` | `add_relative_factor_to_chunk()` | ✅ Complete |
| **Matching** | `is_chunk_ready_for_matching()` | N/A | ✅ Complete |
| **Anchoring** | `mark_chunk_anchored()` | `add_chunk_anchor()` | ✅ Complete |
| **Merging** | `deactivate_chunk()` | `merge_chunks()` | ⚠️ **Gap** |
| **Deactivation** | `deactivate_chunk()` | N/A | ✅ Complete |
**Status:** ✅ **Fixed** - Added `F12.merge_chunks()` method for chunk merging coordination
---
## 2. Integration Analysis
### 2.1 F12 ↔ F10 Integration
**F12 calls F10:**
- ✅ `create_chunk()` → `F10.create_new_chunk()` - Correct
- ✅ `add_frame_to_chunk()` → `F10.add_relative_factor_to_chunk()` - Correct
- ✅ `mark_chunk_anchored()` → `F10.add_chunk_anchor()` - Correct
- ✅ `get_chunk_bounds()` → `F10.get_chunk_trajectory()` - Correct
**F10 exposes chunk operations:**
- ✅ `create_new_chunk()` - Low-level factor graph operation
- ✅ `add_relative_factor_to_chunk()` - Factor management
- ✅ `add_chunk_anchor()` - Anchor management
- ✅ `merge_chunks()` - Sim(3) transformation
- ✅ `optimize_chunk()` - Chunk-level optimization
- ✅ `get_chunk_trajectory()` - Trajectory retrieval
**Assessment:** ✅ **Well-integrated** - F12 properly wraps F10 operations
**Status:** ✅ **Fixed** - F10's method renamed to `get_chunk_for_frame(frame_id)` for clarity, F12's `get_active_chunk(flight_id)` remains for high-level queries.
---
### 2.2 F12 ↔ F11 Integration
**F11 calls F12:**
- ✅ `create_chunk_on_tracking_loss()` → `F12.create_chunk()` - Correct (proactive)
- ✅ `try_chunk_semantic_matching()` → Uses F12 methods indirectly - Correct
- ✅ `try_chunk_litesam_matching()` → Uses F12 methods indirectly - Correct
- ✅ `merge_chunk_to_trajectory()` → Calls `F10.merge_chunks()` directly - ⚠️ **Bypasses F12**
**F12 provides for F11:**
- ✅ `get_chunks_for_matching()` - Returns unanchored chunks
- ✅ `get_chunk_images()` - Image retrieval
- ✅ `get_chunk_composite_descriptor()` - Descriptor computation
- ✅ `get_chunk_bounds()` - Bounds for tile search
**Assessment:** ⚠️ **Minor inconsistency** - F11 bypasses F12 for merging
**Status:** ✅ **Fixed** - F11 now calls `F12.merge_chunks()` which coordinates with F10 and updates chunk states.
---
### 2.3 F12 ↔ F02 Integration
**F02 calls F12:**
- ✅ `get_active_chunk(flight_id)` - Before processing frame
- ✅ `create_new_chunk(flight_id, frame_id)` - On tracking loss
- ✅ `add_frame_to_chunk()` - During frame processing
**F02 chunk-aware processing:**
- ✅ Gets active chunk before processing frame
- ✅ Creates new chunk on tracking loss
- ✅ Adds frames to chunk with VO results
**Assessment:** ✅ **Well-integrated** - F02 properly uses F12 for chunk management
---
### 2.4 F12 ↔ F08/F09 Integration
**F08 Global Place Recognition:**
- ✅ `get_chunk_images()` - Retrieves chunk images
- ✅ `get_chunk_composite_descriptor()` - Gets aggregate descriptor
- ⚠️ **Issue:** F12's `get_chunk_composite_descriptor()` calls `F08.compute_location_descriptor()` for each image, but F08 also has `compute_chunk_descriptor()` method. This creates duplication.
**F09 Metric Refinement:**
- ✅ `get_chunk_images()` - Retrieves chunk images
- ✅ `align_chunk_to_satellite()` - Chunk-to-satellite matching
**Assessment:** ⚠️ **Descriptor duplication** - F12 and F08 both compute chunk descriptors
---
## 3. Chunk Lifecycle Flow Analysis
### 3.1 Normal Chunk Lifecycle
```
1. Tracking Lost (F02/F11 detects)
↓
2. create_chunk() (F12) → create_new_chunk() (F10)
↓
3. add_frame_to_chunk() × N (F12) → add_relative_factor_to_chunk() (F10)
↓
4. is_chunk_ready_for_matching() (F12) → True (>=5 frames)
↓
5. get_chunks_for_matching() (F12) → Returns chunk
↓
6. try_chunk_semantic_matching() (F11) → Uses F12.get_chunk_composite_descriptor()
↓
7. try_chunk_litesam_matching() (F11) → Uses F12.get_chunk_images()
↓
8. mark_chunk_anchored() (F12) → add_chunk_anchor() (F10)
↓
9. merge_chunks() (F10) → Called directly by F11 (bypasses F12)
↓
10. deactivate_chunk() (F12) → Chunk marked as merged
```
**Status:** ✅ **Fixed** - Step 9 now goes through F12.merge_chunks() maintaining abstraction
---
### 3.2 Proactive Chunk Creation
**Solution Requirement:** Chunks created proactively on tracking loss, not reactively after matching failures.
**Implementation:**
- ✅ F11's `create_chunk_on_tracking_loss()` creates chunk immediately
- ✅ F02's `handle_tracking_loss()` creates chunk proactively
- ✅ Processing continues in new chunk while matching happens asynchronously
**Assessment:** ✅ **Correctly implements proactive creation**
---
### 3.3 Chunk Matching Strategy
**Solution Requirement:** Chunk semantic matching (aggregate DINOv2) → Chunk LiteSAM matching (rotation sweeps) → Chunk merging (Sim3)
**Implementation:**
- ✅ F12 provides `get_chunk_composite_descriptor()` for semantic matching
- ✅ F11 coordinates semantic matching via F08
- ✅ F11 coordinates LiteSAM matching via F09 with rotation sweeps
- ✅ F10 provides `merge_chunks()` with Sim(3) transform
**Assessment:** ✅ **Correctly implements matching strategy**
---
## 4. State Management Coherence
### 4.1 ChunkHandle State Fields
**F12 ChunkHandle:**
```python
chunk_id: str
flight_id: str
start_frame_id: int
end_frame_id: Optional[int]
frames: List[int]
is_active: bool
has_anchor: bool
anchor_frame_id: Optional[int]
anchor_gps: Optional[GPSPoint]
matching_status: str # "unanchored", "matching", "anchored", "merged"
```
**Assessment:** ✅ **Complete state representation** - All necessary fields present
**Issue:** `matching_status` and `has_anchor` are redundant (if `matching_status == "anchored"`, then `has_anchor == True`). Consider consolidating.
---
### 4.2 State Transitions
**Valid Transitions:**
- `unanchored` → `matching` (when matching starts)
- `matching` → `anchored` (when anchor found)
- `anchored` → `merged` (when merged to global trajectory)
- `unanchored` → `anchored` (direct anchor, e.g., user input)
**Assessment:** ✅ **State transitions are well-defined**
**Status:** ✅ **Fixed** - Added `F12.mark_chunk_matching(chunk_id)` method for explicit state transitions.
---
## 5. Missing Functionality
### 5.1 Chunk Merging Coordination
**Gap:** F12 doesn't have a method to coordinate chunk merging.
**Current:** F11 calls `F10.merge_chunks()` directly, bypassing F12.
**Recommendation:** Add `F12.merge_chunks(chunk_id_1, chunk_id_2, transform)` that:
1. Validates chunks can be merged
2. Calls `F10.merge_chunks()`
3. Updates chunk states (deactivates chunk_id_1, updates chunk_id_2)
4. Updates `matching_status` to "merged"
---
### 5.2 Chunk State Persistence
**Gap:** No explicit persistence of chunk state.
**Current:** Chunk state is in-memory only (via F12 and F10).
**Recommendation:** F12 should persist chunk state to F03 Flight Database for:
- Recovery after system restart
- Chunk state queries
- Debugging and analysis
---
### 5.3 Chunk Matching Status Updates
**Gap:** No explicit method to update `matching_status` to "matching".
**Current:** Status transitions happen implicitly in F11.
**Recommendation:** Add `F12.mark_chunk_matching(chunk_id)` to explicitly track matching state.
---
## 6. Inconsistencies
### 6.1 Descriptor Computation Duplication
**Issue:** Both F08 and F12 compute chunk descriptors.
- **F08:** `compute_chunk_descriptor(chunk_images)` - Computes aggregate DINOv2 descriptor
- **F12:** `get_chunk_composite_descriptor(chunk_id)` - Also computes aggregate descriptor
**Current Implementation (F12):**
```python
1. Get chunk images via get_chunk_images()
2. For each image:
- Call F08.compute_location_descriptor(image) → descriptor
3. Aggregate descriptors (mean, max, or VLAD)
```
**Status:** ✅ **Fixed** - F12 now calls `F08.compute_chunk_descriptor()` directly, eliminating duplication.
---
### 6.2 Active Chunk Query Inconsistency
**Issue:** F10 and F12 have different signatures for `get_active_chunk()`.
- **F10:** `get_active_chunk(frame_id: int)` - Query by frame ID
- **F12:** `get_active_chunk(flight_id: str)` - Query by flight ID
**Status:** ✅ **Fixed** - F10's method renamed to `get_chunk_for_frame(frame_id)` for clarity, clearly distinguishing from F12's `get_active_chunk(flight_id)`.
---
### 6.3 Chunk Merging Bypass
**Issue:** F11 bypasses F12 when merging chunks.
**Current:** `F11.merge_chunk_to_trajectory()` → `F10.merge_chunks()` directly
**Status:** ✅ **Fixed** - F11 now calls `F12.merge_chunks()` which properly coordinates merging and updates chunk states.
---
## 7. Strengths
### 7.1 Clear Abstraction Layers ✅
- **F12** provides high-level chunk lifecycle management
- **F10** provides low-level factor graph operations
- Clear separation of concerns
### 7.2 Proactive Chunk Creation ✅
- Chunks created immediately on tracking loss
- Processing continues while matching happens asynchronously
- Matches solution requirement perfectly
### 7.3 Complete Chunk State Tracking ✅
- ChunkHandle captures all necessary state
- State transitions are well-defined
- Matching status tracks chunk progress
### 7.4 Chunk Matching Integration ✅
- F12 provides chunk representations (images, descriptors, bounds)
- F11 coordinates matching strategies
- F08/F09 perform actual matching
### 7.5 Chunk Isolation ✅
- Each chunk has independent subgraph in F10
- Factors isolated to chunks
- Local optimization before global merging
---
## 8. Weaknesses
### 8.1 Missing Merging Coordination ✅ **FIXED**
- ✅ F12.merge_chunks() method added
- ✅ F11 now calls F12, maintaining abstraction
- ✅ State updates handled by F12
### 8.2 Descriptor Computation Duplication ✅ **FIXED**
- ✅ F12 now delegates to F08.compute_chunk_descriptor() directly
- ✅ Single source of truth for chunk descriptor computation
### 8.3 No State Persistence ⚠️
- Chunk state is in-memory only
- No recovery after restart
- No debugging/analysis capabilities
- **Note:** This is a Priority 2 enhancement, not critical
### 8.4 Implicit State Transitions ✅ **FIXED**
- ✅ F12.mark_chunk_matching() method added
- ✅ Explicit state transitions for matching status
- ✅ Easier to track chunk state changes
---
## 9. Recommendations
### Priority 1: Critical Fixes ✅ **COMPLETED**
1. ✅ **Add F12.merge_chunks() method** - **FIXED**
- F12.merge_chunks() method added
- Coordinates chunk merging and updates states
- F11 now calls F12, maintaining abstraction
2. ✅ **Fix descriptor computation duplication** - **FIXED**
- F12.get_chunk_composite_descriptor() now calls F08.compute_chunk_descriptor() directly
- Removed duplicate aggregation logic from F12
3. ✅ **Add explicit matching status updates** - **FIXED**
- F12.mark_chunk_matching(chunk_id) method added
- Explicitly tracks when matching starts
### Priority 2: Important Enhancements
4. **Add chunk state persistence**
- Persist ChunkHandle to F03 Flight Database
- Enable recovery after restart
- Support debugging and analysis
5. ✅ **Clarify method naming** - **FIXED**
- F10.get_active_chunk() renamed to get_chunk_for_frame() for clarity
- Documented different use cases (low-level vs high-level)
6. **Add chunk validation**
- Validate chunk state before operations
- Prevent invalid state transitions
- Better error messages
### Priority 3: Nice-to-Have
7. **Add chunk metrics**
- Track chunk creation time
- Track matching success rate
- Track merging statistics
8. **Add chunk query methods**
- Query chunks by status
- Query chunks by frame range
- Query chunks by matching status
---
## 10. Overall Assessment
### Coherence Score: 9/10 (Updated after fixes)
**Strengths:**
- ✅ Clear separation of concerns
- ✅ Proactive chunk creation
- ✅ Complete lifecycle coverage
- ✅ Well-integrated with F10, F11, F02
- ✅ Proper chunk isolation
- ✅ **FIXED:** Merging coordination through F12
- ✅ **FIXED:** Descriptor computation delegation
- ✅ **FIXED:** Explicit state transitions
**Remaining Enhancement:**
- ⚠️ No state persistence (Priority 2, not critical)
**Conclusion:** The chunking system is **excellent and coherent** with clear architectural boundaries. All Priority 1 issues have been **fixed**. The system now properly maintains abstraction layers with F12 coordinating all chunk lifecycle operations. Remaining enhancement (state persistence) is a nice-to-have feature for recovery and debugging.
---
## 11. Solution Alignment
### Atlas Multi-Map Architecture ✅
The chunking system correctly implements the Atlas architecture from the solution:
- ✅ **Chunks are first-class entities** - F12 manages chunks as primary units
- ✅ **Proactive chunk creation** - Chunks created immediately on tracking loss
- ✅ **Independent chunk processing** - Each chunk has its own subgraph in F10
- ✅ **Chunk matching and merging** - Semantic matching → LiteSAM → Sim(3) merging
- ✅ **Multiple chunks simultaneously** - System supports multiple unanchored chunks
**Assessment:** ✅ **Fully aligned with solution architecture**
docs/02_components/component_coverage_analysis.md
+526
View File
@@ -0,0 +1,526 @@
# Component Coverage Analysis: Solution, Problem, Acceptance Criteria, and Restrictions
## Executive Summary
This document analyzes how the ASTRAL-Next component architecture covers the solution design, addresses the original problem, meets acceptance criteria, and operates within restrictions.
**Key Findings:**
- ✅ Components comprehensively implement the tri-layer localization strategy (Sequential VO, Global PR, Metric Refinement)
- ✅ Atlas multi-map chunk architecture properly handles sharp turns and disconnected routes
- ✅ All 10 acceptance criteria are addressed by specific component capabilities
- ✅ Restrictions are respected through component design choices
- ⚠️ Some architectural concerns identified (see architecture_assessment.md)
---
## 1. Solution Coverage Analysis
### 1.1 Tri-Layer Localization Strategy
The solution document specifies three layers operating concurrently:
| Solution Layer | Component(s) | Implementation Status |
|----------------|--------------|----------------------|
| **L1: Sequential Tracking** | F07 Sequential Visual Odometry | ✅ Fully covered |
| **L2: Global Re-Localization** | F08 Global Place Recognition | ✅ Fully covered |
| **L3: Metric Refinement** | F09 Metric Refinement | ✅ Fully covered |
| **State Estimation** | F10 Factor Graph Optimizer | ✅ Fully covered |
**Coverage Details:**
**L1 - Sequential VO (F07):**
- Uses SuperPoint + LightGlue as specified
- Handles <5% overlap scenarios
- Provides relative pose factors to F10
- Chunk-aware operations (factors added to chunk subgraphs)
**L2 - Global PR (F08):**
- Implements AnyLoc (DINOv2 + VLAD) as specified
- Faiss indexing for efficient retrieval
- Chunk semantic matching (aggregate descriptors)
- Handles "kidnapped robot" scenarios
**L3 - Metric Refinement (F09):**
- Implements LiteSAM for cross-view matching
- Requires pre-rotation (handled by F06)
- Extracts GPS from homography
- Chunk-to-satellite matching support
**State Estimation (F10):**
- GTSAM-based factor graph optimization
- Robust kernels (Huber/Cauchy) for outlier handling
- Multi-chunk support (Atlas architecture)
- Sim(3) transformation for chunk merging
### 1.2 Atlas Multi-Map Architecture
**Solution Requirement:** Chunks are first-class entities, created proactively on tracking loss.
**Component Coverage:**
- ✅ **F12 Route Chunk Manager**: Manages chunk lifecycle (creation, activation, matching, merging)
- ✅ **F10 Factor Graph Optimizer**: Provides multi-chunk factor graph with independent subgraphs
- ✅ **F11 Failure Recovery Coordinator**: Proactively creates chunks on tracking loss
- ✅ **F02 Flight Processor**: Chunk-aware frame processing
**Chunk Lifecycle Flow:**
1. **Tracking Loss Detected** → F11 creates chunk proactively (not reactive)
2. **Chunk Building** → F07 adds VO factors to chunk subgraph via F10
3. **Chunk Matching** → F08 (semantic) + F09 (LiteSAM) match chunks
4. **Chunk Anchoring** → F10 anchors chunk with GPS
5. **Chunk Merging** → F10 merges chunks using Sim(3) transform
**Coverage Verification:**
- ✅ Chunks created proactively (not after matching failures)
- ✅ Chunks processed independently
- ✅ Chunk semantic matching (aggregate DINOv2)
- ✅ Chunk LiteSAM matching with rotation sweeps
- ✅ Chunk merging via Sim(3) transformation
### 1.3 REST API + SSE Architecture
**Solution Requirement:** Background service with REST API and SSE streaming.
**Component Coverage:**
- ✅ **F01 Flight API**: REST endpoints (FastAPI)
- ✅ **F15 SSE Event Streamer**: Real-time result streaming
- ✅ **F02 Flight Processor**: Background processing orchestration
- ✅ **F14 Result Manager**: Result publishing coordination
**API Coverage:**
- ✅ `POST /flights` - Flight creation
- ✅ `GET /flights/{id}` - Flight retrieval
- ✅ `POST /flights/{id}/images/batch` - Batch image upload
- ✅ `POST /flights/{id}/user-fix` - User anchor input
- ✅ `GET /flights/{id}/stream` - SSE streaming
**SSE Events:**
- ✅ `frame_processed` - Per-frame results
- ✅ `frame_refined` - Refinement updates
- ✅ `user_input_needed` - User intervention required
- ✅ `search_expanded` - Progressive search updates
### 1.4 Human-in-the-Loop Strategy
**Solution Requirement:** User input for 20% of route where automation fails.
**Component Coverage:**
- ✅ **F11 Failure Recovery Coordinator**: Monitors confidence, triggers user input
- ✅ **F01 Flight API**: Accepts user fixes via REST endpoint
- ✅ **F15 SSE Event Streamer**: Sends user input requests
- ✅ **F10 Factor Graph Optimizer**: Applies user anchors as hard constraints
**Recovery Stages:**
1. ✅ Stage 1: Progressive tile search (single-image)
2. ✅ Stage 2: Chunk building and semantic matching
3. ✅ Stage 3: Chunk LiteSAM matching with rotation sweeps
4. ✅ Stage 4: User input (last resort)
---
## 2. Original Problem Coverage
### 2.1 Problem Statement
**Original Problem:** Determine GPS coordinates of image centers from UAV flight, given only starting GPS coordinates.
**Component Coverage:**
- ✅ **F13 Coordinate Transformer**: Converts pixel coordinates to GPS
- ✅ **F09 Metric Refinement**: Extracts GPS from satellite alignment
- ✅ **F10 Factor Graph Optimizer**: Optimizes trajectory to GPS coordinates
- ✅ **F14 Result Manager**: Publishes GPS results per frame
**Coverage Verification:**
- ✅ Starting GPS used to initialize ENU coordinate system (F13)
- ✅ Per-frame GPS computed from trajectory (F10 → F13)
- ✅ Object coordinates computed via pixel-to-GPS transformation (F13)
### 2.2 Image Processing Requirements
**Requirement:** Process images taken consecutively within 100m spacing.
**Component Coverage:**
- ✅ **F05 Image Input Pipeline**: Handles sequential image batches
- ✅ **F07 Sequential VO**: Processes consecutive frames
- ✅ **F02 Flight Processor**: Validates sequence continuity
**Coverage Verification:**
- ✅ Batch validation ensures sequential ordering
- ✅ VO handles 100m spacing via relative pose estimation
- ✅ Factor graph maintains trajectory continuity
### 2.3 Satellite Data Usage
**Requirement:** Use external satellite provider for ground checks.
**Component Coverage:**
- ✅ **F04 Satellite Data Manager**: Fetches Google Maps tiles
- ✅ **F08 Global Place Recognition**: Matches UAV images to satellite tiles
- ✅ **F09 Metric Refinement**: Aligns UAV images to satellite tiles
**Coverage Verification:**
- ✅ Google Maps Static API integration (F04)
- ✅ Tile caching and prefetching (F04)
- ✅ Progressive tile search (1→4→9→16→25) (F04 + F11)
---
## 3. Acceptance Criteria Coverage
### AC-1: 80% of photos < 50m error
**Component Coverage:**
- **F09 Metric Refinement**: LiteSAM achieves ~17.86m RMSE (within 50m requirement)
- **F10 Factor Graph Optimizer**: Fuses measurements for accuracy
- **F13 Coordinate Transformer**: Accurate GPS conversion
**Implementation:**
- LiteSAM provides pixel-level alignment
- Factor graph optimization reduces drift
- Altitude priors resolve scale ambiguity
**Status:** ✅ Covered
---
### AC-2: 60% of photos < 20m error
**Component Coverage:**
- **F09 Metric Refinement**: LiteSAM RMSE ~17.86m (close to 20m requirement)
- **F10 Factor Graph Optimizer**: Global optimization improves precision
- **F04 Satellite Data Manager**: High-resolution tiles (Zoom Level 19, ~0.30m/pixel)
**Implementation:**
- Multi-scale LiteSAM processing
- Per-keyframe scale model in factor graph
- High-resolution satellite tiles
**Status:** ✅ Covered (may require Tier-2 commercial data per solution doc)
---
### AC-3: Robust to 350m outlier
**Component Coverage:**
- **F10 Factor Graph Optimizer**: Robust kernels (Huber/Cauchy) downweight outliers
- **F11 Failure Recovery Coordinator**: Detects outliers and triggers recovery
- **F07 Sequential VO**: Reports low confidence for outlier frames
**Implementation:**
- Huber loss function in factor graph
- M-estimation automatically rejects high-residual constraints
- Stage 2 failure logic discards outlier frames
**Status:** ✅ Covered
---
### AC-4: Robust to sharp turns (<5% overlap)
**Component Coverage:**
- **F12 Route Chunk Manager**: Creates new chunks on tracking loss
- **F08 Global Place Recognition**: Re-localizes after sharp turns
- **F06 Image Rotation Manager**: Handles unknown orientation
- **F11 Failure Recovery Coordinator**: Coordinates recovery
**Implementation:**
- Proactive chunk creation on tracking loss
- Rotation sweeps (0°, 30°, ..., 330°) for unknown orientation
- Chunk semantic matching handles featureless terrain
- Chunk LiteSAM matching aggregates correspondences
**Status:** ✅ Covered
---
### AC-5: < 10% outlier anchors
**Component Coverage:**
- **F10 Factor Graph Optimizer**: Robust M-estimation (Huber loss)
- **F09 Metric Refinement**: Match confidence filtering
- **F11 Failure Recovery Coordinator**: Validates matches before anchoring
**Implementation:**
- Huber loss automatically downweights bad anchors
- Match confidence threshold (0.7) filters outliers
- Inlier count validation before anchoring
**Status:** ✅ Covered
---
### AC-6: Connect route chunks; User input
**Component Coverage:**
- **F12 Route Chunk Manager**: Manages chunk lifecycle
- **F10 Factor Graph Optimizer**: Merges chunks via Sim(3) transform
- **F11 Failure Recovery Coordinator**: Coordinates chunk matching
- **F01 Flight API**: User input endpoint
- **F15 SSE Event Streamer**: User input requests
**Implementation:**
- Chunk semantic matching connects chunks
- Chunk LiteSAM matching provides Sim(3) transform
- Chunk merging maintains global consistency
- User input as last resort (Stage 4)
**Status:** ✅ Covered
---
### AC-7: < 5 seconds processing/image
**Component Coverage:**
- **F16 Model Manager**: TensorRT optimization (2-4x speedup)
- **F07 Sequential VO**: ~50ms (SuperPoint + LightGlue)
- **F08 Global Place Recognition**: ~150ms (DINOv2 + VLAD, keyframes only)
- **F09 Metric Refinement**: ~60ms (LiteSAM)
- **F10 Factor Graph Optimizer**: ~100ms (iSAM2 incremental)
**Performance Breakdown:**
- Sequential VO: ~50ms
- Global PR (keyframes): ~150ms
- Metric Refinement: ~60ms
- Factor Graph: ~100ms
- **Total (worst case): ~360ms << 5s**
**Status:** ✅ Covered (with TensorRT optimization)
---
### AC-8: Real-time stream + async refinement
**Component Coverage:**
- **F15 SSE Event Streamer**: Real-time frame results
- **F14 Result Manager**: Per-frame and refinement publishing
- **F10 Factor Graph Optimizer**: Asynchronous batch refinement
- **F02 Flight Processor**: Decoupled processing pipeline
**Implementation:**
- Immediate per-frame results via SSE
- Background refinement thread
- Batch waypoint updates for refinements
- Incremental SSE events for refinements
**Status:** ✅ Covered
---
### AC-9: Image Registration Rate > 95%
**Component Coverage:**
- **F07 Sequential VO**: Handles <5% overlap
- **F12 Route Chunk Manager**: Chunk creation prevents "lost" frames
- **F08 Global Place Recognition**: Re-localizes after tracking loss
- **F09 Metric Refinement**: Aligns frames to satellite
**Implementation:**
- "Lost track" creates new chunk (not registration failure)
- Chunk matching recovers disconnected segments
- System never "fails" - fragments and continues
**Status:** ✅ Covered (Atlas architecture ensures >95%)
---
### AC-10: Mean Reprojection Error (MRE) < 1.0px
**Component Coverage:**
- **F10 Factor Graph Optimizer**: Local and global bundle adjustment
- **F07 Sequential VO**: High-quality feature matching (SuperPoint + LightGlue)
- **F09 Metric Refinement**: Precise homography estimation
**Implementation:**
- Local BA in sequential VO
- Global BA in factor graph optimizer
- Per-keyframe scale model minimizes graph tension
- Robust kernels prevent outlier contamination
**Status:** ✅ Covered
---
## 4. Restrictions Compliance
### R-1: Photos from airplane-type UAVs only
**Component Coverage:**
- **F17 Configuration Manager**: Validates flight type
- **F02 Flight Processor**: Validates flight parameters
**Compliance:** ✅ Validated at flight creation
---
### R-2: Camera pointing downwards, fixed, not autostabilized
**Component Coverage:**
- **F06 Image Rotation Manager**: Handles rotation variations
- **F09 Metric Refinement**: Requires pre-rotation (handled by F06)
- **F07 Sequential VO**: Handles perspective variations
**Compliance:** ✅ Rotation sweeps handle fixed camera orientation
---
### R-3: Flying range restricted to Eastern/Southern Ukraine
**Component Coverage:**
- **F02 Flight Processor**: Validates waypoints within operational area
- **F04 Satellite Data Manager**: Prefetches tiles for operational area
- **F13 Coordinate Transformer**: ENU origin set to operational area
**Compliance:** ✅ Geofence validation, operational area constraints
---
### R-4: Image resolution FullHD to 6252×4168
**Component Coverage:**
- **F16 Model Manager**: TensorRT handles variable resolutions
- **F07 Sequential VO**: SuperPoint processes variable resolutions
- **F05 Image Input Pipeline**: Validates image dimensions
**Compliance:** ✅ Components handle variable resolutions
---
### R-5: Altitude predefined, no more than 1km
**Component Coverage:**
- **F10 Factor Graph Optimizer**: Altitude priors resolve scale
- **F13 Coordinate Transformer**: GSD calculations use altitude
- **F02 Flight Processor**: Validates altitude <= 1000m
**Compliance:** ✅ Altitude used as soft constraint in factor graph
---
### R-6: NO data from IMU
**Component Coverage:**
- **F10 Factor Graph Optimizer**: Monocular VO only (no IMU factors)
- **F07 Sequential VO**: Pure visual odometry
- **F13 Coordinate Transformer**: Scale resolved via altitude + satellite matching
**Compliance:** ✅ No IMU components, scale resolved via altitude priors
---
### R-7: Flights mostly in sunny weather
**Component Coverage:**
- **F08 Global Place Recognition**: DINOv2 handles appearance changes
- **F09 Metric Refinement**: LiteSAM robust to lighting variations
- **F07 Sequential VO**: SuperPoint handles texture variations
**Compliance:** ✅ Algorithms robust to lighting conditions
---
### R-8: Google Maps (may be outdated)
**Component Coverage:**
- **F04 Satellite Data Manager**: Google Maps Static API integration
- **F08 Global Place Recognition**: DINOv2 semantic features (invariant to temporal changes)
- **F09 Metric Refinement**: LiteSAM focuses on structural features
**Compliance:** ✅ Semantic matching handles outdated satellite data
---
### R-9: 500-1500 photos typically, up to 3000
**Component Coverage:**
- **F05 Image Input Pipeline**: Batch processing (10-50 images)
- **F10 Factor Graph Optimizer**: Efficient optimization (iSAM2)
- **F03 Flight Database**: Handles large flight datasets
**Compliance:** ✅ Components scale to 3000 images
---
### R-10: Sharp turns possible (exception, not rule)
**Component Coverage:**
- **F12 Route Chunk Manager**: Chunk architecture handles sharp turns
- **F11 Failure Recovery Coordinator**: Recovery strategies for sharp turns
- **F06 Image Rotation Manager**: Rotation sweeps for unknown orientation
**Compliance:** ✅ Chunk architecture handles exceptions gracefully
---
### R-11: Processing on RTX 2060/3070 (TensorRT required)
**Component Coverage:**
- **F16 Model Manager**: TensorRT optimization (FP16 quantization)
- **F07 Sequential VO**: TensorRT-optimized SuperPoint + LightGlue
- **F08 Global Place Recognition**: TensorRT-optimized DINOv2
- **F09 Metric Refinement**: TensorRT-optimized LiteSAM
**Compliance:** ✅ All models optimized for TensorRT, FP16 quantization
---
## 5. Coverage Gaps and Concerns
### 5.1 Architectural Concerns
See `architecture_assessment.md` for detailed concerns:
- Component numbering inconsistencies
- Circular dependencies (F14 → F01)
- Duplicate functionality (chunk descriptors)
- Missing component connections
### 5.2 Potential Gaps
1. **Performance Monitoring**: H05 Performance Monitor exists but integration unclear
2. **Error Recovery**: Comprehensive error handling not fully specified
3. **Concurrent Flights**: Multi-flight processing not fully validated
4. **Satellite Data Freshness**: Handling of outdated Google Maps data relies on semantic features (may need validation)
### 5.3 Recommendations
1. **Fix Architectural Issues**: Address concerns in architecture_assessment.md
2. **Performance Validation**: Validate <5s processing on RTX 2060
3. **Accuracy Validation**: Test against ground truth data (coordinates.csv)
4. **Chunk Matching Validation**: Validate chunk matching reduces user input by 50-70%
---
## 6. Summary Matrix
| Requirement Category | Coverage | Status |
|---------------------|----------|--------|
| **Solution Architecture** | Tri-layer + Atlas + REST/SSE | ✅ Complete |
| **Problem Statement** | GPS localization from images | ✅ Complete |
| **AC-1** (80% < 50m) | LiteSAM + Factor Graph | ✅ Covered |
| **AC-2** (60% < 20m) | LiteSAM + High-res tiles | ✅ Covered |
| **AC-3** (350m outlier) | Robust kernels | ✅ Covered |
| **AC-4** (Sharp turns) | Chunk architecture | ✅ Covered |
| **AC-5** (<10% outliers) | Robust M-estimation | ✅ Covered |
| **AC-6** (Chunk connection) | Chunk matching + User input | ✅ Covered |
| **AC-7** (<5s processing) | TensorRT optimization | ✅ Covered |
| **AC-8** (Real-time stream) | SSE + Async refinement | ✅ Covered |
| **AC-9** (>95% registration) | Atlas architecture | ✅ Covered |
| **AC-10** (MRE < 1.0px) | Bundle adjustment | ✅ Covered |
| **Restrictions** | All 11 restrictions | ✅ Compliant |
---
## 7. Conclusion
The component architecture comprehensively covers the solution design, addresses the original problem, meets all acceptance criteria, and operates within restrictions. The Atlas multi-map chunk architecture is properly implemented across F10, F11, and F12 components. The tri-layer localization strategy is fully covered by F07, F08, and F09.
**Key Strengths:**
- Complete solution coverage
- All acceptance criteria addressed
- Restrictions respected
- Chunk architecture properly implemented
**Areas for Improvement:**
- Fix architectural concerns (see architecture_assessment.md)
- Validate performance on target hardware
- Test accuracy against ground truth data
- Validate chunk matching effectiveness
docs/02_components/decomposition_plan.md
+214 -184
View File
@@ -1,4 +1,3 @@
<!-- 31098ee5-58fb-474a-815e-fd9cbd17c063 9f609f9e-c80d-4c88-b618-3135b96a8333 -->
# ASTRAL-Next System Component Decomposition Plan
## Design Principle: Interface-Based Architecture
@@ -24,133 +23,106 @@
## System Architecture Overview
**Two separate REST APIs in same repository:**
**Single unified Flight API:**
### Route API (Separate Project)
- Route/waypoint/geofence CRUD
- Shared by GPS-Denied and Mission Planner
- Does NOT call satellite provider
### GPS-Denied API (Main System)
- Tri-layer localization (SuperPoint+LightGlue, AnyLoc, LiteSAM)
- Flight CRUD operations (create, read, update, delete)
- Waypoint management within flights
- Geofence management
- Tri-layer localization (SuperPoint+LightGlue, DINOv2, LiteSAM)
- Calls satellite provider for tiles
- Rotation preprocessing (LiteSAM 45° limit)
- Per-frame Route API updates
- Per-frame waypoint updates
- Progressive tile search (1→4→9→16→25)
- SSE streaming for real-time results
---
## ROUTE API COMPONENTS (4 components)
## FLIGHT API COMPONENTS (17 components)
### R01_route_rest_api
### Core API Layer
**Interface**: `IRouteRestAPI`
**Endpoints**: `POST /routes`, `GET /routes/{routeId}`, `PUT /routes/{routeId}/waypoints`, `DELETE /routes/{routeId}`
**F01_flight_api**
**Interface**: `IFlightAPI`
**Endpoints**: `POST /flights`, `GET /flights/{flightId}`, `DELETE /flights/{flightId}`, `PUT /flights/{flightId}/waypoints/{waypointId}`, `POST .../images/batch`, `POST .../user-fix`, `GET .../status`, `GET .../stream`
### R02_route_data_manager
**F02_flight_processor**
**Interface**: `IFlightProcessor`
**API**: `create_flight()`, `get_flight()`, `get_flight_state()`, `delete_flight()`, `update_waypoint()`, `batch_update_waypoints()`, `validate_waypoint()`, `validate_geofence()`, `process_frame()`, `run_processing_loop()`, `handle_tracking_loss()`, `initialize_system()`
**Interface**: `IRouteDataManager`
**API**: `save_route()`, `load_route()`, `update_waypoint()`, `delete_waypoint()`, `get_route_metadata()`
### R03_waypoint_validator
**Interface**: `IWaypointValidator`
**API**: `validate_waypoint()`, `validate_geofence()`, `check_bounds()`, `validate_route_continuity()`
### R04_route_database_layer
**Interface**: `IRouteDatabase`
**API**: `insert_route()`, `update_route()`, `query_routes()`, `get_waypoints()`
---
## GPS-DENIED API COMPONENTS (17 components)
### Core REST API Layer
**G01_gps_denied_rest_api**
**Interface**: `IGPSDeniedRestAPI`
**Endpoints**: `POST /gps-denied/flights`, `POST .../images/batch`, `POST .../user-fix`, `GET .../status`, `GET .../stream`
**G02_flight_manager**
**Interface**: `IFlightManager`
**API**: `create_flight()`, `get_flight_state()`, `link_to_route()`, `update_flight_status()`
**G03_route_api_client**
**Interface**: `IRouteAPIClient`
**API**: `update_route_waypoint()`, `get_route_info()`, `batch_update_waypoints()`
**F03_flight_database**
**Interface**: `IFlightDatabase`
**API**: `insert_flight()`, `update_flight()`, `query_flights()`, `get_flight_by_id()`, `delete_flight()`, `get_waypoints()`, `insert_waypoint()`, `update_waypoint()`, `batch_update_waypoints()`, `save_flight_state()`, `load_flight_state()`, `save_frame_result()`, `get_frame_results()`, `save_heading()`, `get_heading_history()`, `get_latest_heading()`, `save_image_metadata()`, `get_image_path()`, `get_image_metadata()`
### Data Management
**G04_satellite_data_manager**
**F04_satellite_data_manager**
**Interface**: `ISatelliteDataManager`
**API**: `fetch_tile()`, `fetch_tile_grid()`, `prefetch_route_corridor()`, `progressive_fetch()`, `cache_tile()`, `get_cached_tile()`, `compute_tile_coords()`, `expand_search_grid()`
**API**: `fetch_tile()`, `fetch_tile_grid()`, `prefetch_route_corridor()`, `progressive_fetch()`, `cache_tile()`, `get_cached_tile()`, `compute_tile_coords()`, `expand_search_grid()`, `compute_tile_bounds()`
**Features**: Progressive retrieval, tile caching, grid calculations
**G05_image_input_pipeline**
**F05_image_input_pipeline**
**Interface**: `IImageInputPipeline`
**API**: `queue_batch()`, `process_next_batch()`, `validate_batch()`, `store_images()`, `get_next_image()`, `get_image_by_sequence()`
**Features**: FIFO queuing, validation, storage
**G06_image_rotation_manager**
**F06_image_rotation_manager**
**Interface**: `IImageRotationManager`
**API**: `rotate_image_360()`, `try_rotation_steps()`, `calculate_precise_angle()`, `get_current_heading()`, `update_heading()`, `detect_sharp_turn()`, `requires_rotation_sweep()`
**Features**: 30° rotation sweeps, heading tracking
### Visual Processing
**G07_sequential_visual_odometry**
**F07_sequential_visual_odometry**
**Interface**: `ISequentialVO`
**API**: `compute_relative_pose()`, `extract_features()`, `match_features()`, `estimate_motion()`
**G08_global_place_recognition**
**F08_global_place_recognition**
**Interface**: `IGlobalPlaceRecognition`
**API**: `retrieve_candidate_tiles()`, `compute_location_descriptor()`, `query_database()`, `rank_candidates()`
**G09_metric_refinement**
**F09_metric_refinement**
**Interface**: `IMetricRefinement`
**API**: `align_to_satellite()`, `compute_homography()`, `extract_gps_from_alignment()`, `compute_match_confidence()`
**API**: `align_to_satellite(uav_image, satellite_tile, tile_bounds)`, `compute_homography()`, `extract_gps_from_alignment()`, `compute_match_confidence()`
### State Estimation
**G10_factor_graph_optimizer**
**F10_factor_graph_optimizer**
**Interface**: `IFactorGraphOptimizer`
**API**: `add_relative_factor()`, `add_absolute_factor()`, `add_altitude_prior()`, `optimize()`, `get_trajectory()`
**API**: `add_relative_factor()`, `add_absolute_factor()`, `add_altitude_prior()`, `optimize()`, `get_trajectory()`, `get_marginal_covariance()`
**G11_failure_recovery_coordinator**
**F11_failure_recovery_coordinator**
**Interface**: `IFailureRecoveryCoordinator`
**API**: `check_confidence()`, `detect_tracking_loss()`, `start_search()`, `expand_search_radius()`, `try_current_grid()`, `create_user_input_request()`, `apply_user_anchor()`
**G12_coordinate_transformer**
**F12_route_chunk_manager**
**Interface**: `IRouteChunkManager`
**API**: `create_chunk()`, `add_frame_to_chunk()`, `get_chunk_frames()`, `get_chunk_images()`, `get_chunk_composite_descriptor()`, `get_chunk_bounds()`, `is_chunk_ready_for_matching()`, `mark_chunk_anchored()`, `get_chunks_for_matching()`, `get_active_chunk()`, `deactivate_chunk()`
**Features**: Chunk lifecycle management, chunk state tracking, chunk matching coordination
**F13_coordinate_transformer**
**Interface**: `ICoordinateTransformer`
**API**: `pixel_to_gps()`, `gps_to_pixel()`, `image_object_to_gps()`, `compute_gsd()`, `transform_points()`
**API**: `set_enu_origin()`, `get_enu_origin()`, `gps_to_enu()`, `enu_to_gps()`, `pixel_to_gps()`, `gps_to_pixel()`, `image_object_to_gps()`, `compute_gsd()`, `transform_points()`
### Results & Communication
**G13_result_manager**
**F14_result_manager**
**Interface**: `IResultManager`
**API**: `update_frame_result()`, `publish_to_route_api()`, `get_flight_results()`, `mark_refined()`
**API**: `update_frame_result()`, `publish_waypoint_update()`, `get_flight_results()`, `mark_refined()`
**G14_sse_event_streamer**
**F15_sse_event_streamer**
**Interface**: `ISSEEventStreamer`
**API**: `create_stream()`, `send_frame_result()`, `send_search_progress()`, `send_user_input_request()`, `send_refinement()`
### Infrastructure
**G15_model_manager**
**F16_model_manager**
**Interface**: `IModelManager`
**API**: `load_model()`, `get_inference_engine()`, `optimize_to_tensorrt()`, `fallback_to_onnx()`
**G16_configuration_manager**
**F17_configuration_manager**
**Interface**: `IConfigurationManager`
**API**: `load_config()`, `get_camera_params()`, `validate_config()`, `get_flight_config()`
**G17_gps_denied_database_layer**
**Interface**: `IGPSDeniedDatabase`
**API**: `save_flight_state()`, `load_flight_state()`, `query_processing_history()`
---
## HELPER COMPONENTS (8 components)
@@ -166,156 +138,217 @@
---
## System Startup Initialization Order
**Startup sequence** (blocking, sequential):
| Order | Component | Method | Purpose | Dependencies |
|-------|-----------|--------|---------|--------------|
| 1 | F16 Configuration Manager | `load_config()` | Load system configuration | None |
| 2 | F03 Flight Database | Initialize connections | Establish DB connection pool | F16 |
| 3 | F15 Model Manager | `load_model("SuperPoint")` | Load SuperPoint feature extractor | F16 |
| 4 | F15 Model Manager | `load_model("LightGlue")` | Load LightGlue matcher | F16 |
| 5 | F15 Model Manager | `load_model("DINOv2")` | Load DINOv2 for place recognition | F16 |
| 6 | F15 Model Manager | `load_model("LiteSAM")` | Load LiteSAM for cross-view matching | F16 |
| 7 | F04 Satellite Data Manager | Initialize cache | Initialize tile cache directory | F16 |
| 8 | F08 Global Place Recognition | `build_index()` or `load_index()` | Build/load Faiss index from satellite tiles | F04, F15, H04 |
| 9 | F12 Route Chunk Manager | Initialize | Initialize chunk state tracking | F10 |
| 10 | F02 Flight Processor | Ready | Ready to accept flights | All above |
| 11 | F01 Flight API | Start server | Start FastAPI/Uvicorn | F02 |
**Estimated total startup time**: ~30 seconds (dominated by model loading)
**Shutdown sequence** (reverse order):
1. F01 Flight API - Stop accepting requests
2. F02 Flight Processor - Complete or cancel active flights
3. F11 Failure Recovery Coordinator - Stop background chunk matching
4. F12 Route Chunk Manager - Save chunk state
5. F16 Model Manager - Unload models
6. F03 Flight Database - Close connections
7. F04 Satellite Data Manager - Flush cache
---
## Comprehensive Component Interaction Matrix
### System Initialization
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G02 | G15 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| G02 | G16 | `load_config()` | Load system configuration |
| G04 | G08 | Satellite tiles | G08 generates descriptors for Faiss |
| G08 | H04 | `build_index()` | Build satellite descriptor index |
| G08 | G15 | `get_inference_engine("DINOv2")` | Get model for descriptor generation |
| F02 | F16 | `load_config()` | Load system configuration |
| F02 | F15 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| F04 | F08 | Satellite tiles | F08 generates descriptors for Faiss |
| F08 | H04 | `build_index()` | Build satellite descriptor index |
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model for descriptor generation |
### Flight Creation
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | G01 | `POST /gps-denied/flights` | Create flight |
| G01 | G02 | `create_flight()` | Initialize flight state |
| G02 | G16 | `get_flight_config()` | Get camera params, altitude |
| G02 | G03 | `get_route_info()` | Fetch route metadata |
| G03 | Route API | `GET /routes/{routeId}` | HTTP call |
| G02 | G04 | `prefetch_route_corridor()` | Prefetch tiles |
| G04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download |
| G04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations |
| G02 | G17 | `save_flight_state()` | Persist flight metadata |
| Client | G01 | `GET .../stream` | Open SSE connection |
| G01 | G14 | `create_stream()` | Establish SSE channel |
| Client | F01 | `POST /flights` | Create flight |
| F01 | F02 | `create_flight()` | Initialize flight state |
| F02 | F16 | `get_flight_config()` | Get camera params, altitude |
| F02 | F12 | `set_enu_origin(start_gps)` | Set ENU coordinate origin |
| F02 | F04 | `prefetch_route_corridor()` | Prefetch tiles |
| F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download |
| F04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations |
| F02 | F03 | `insert_flight()` | Persist flight data |
| Client | F01 | `GET .../stream` | Open SSE connection |
| F01 | F14 | `create_stream()` | Establish SSE channel |
### Image Upload
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | G01 | `POST .../images/batch` | Upload 10-50 images |
| G01 | G05 | `queue_batch()` | Queue for processing |
| G05 | H08 | `validate_batch()` | Validate sequence, format |
| G05 | G17 | `store_images()` | Persist images |
| Client | F01 | `POST .../images/batch` | Upload 10-50 images |
| F01 | F05 | `queue_batch()` | Queue for processing |
| F05 | H08 | `validate_batch()` | Validate sequence, format |
| F05 | F03 | `save_image_metadata()` | Persist image metadata |
### Per-Frame Processing (First Frame / Sharp Turn)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G05 | G06 | `get_next_image()` | Get image for processing |
| G06 | G06 | `requires_rotation_sweep()` | Check if sweep needed |
| G06 | H07 | `rotate_image()` × 12 | Rotate in 30° steps |
| G06 | G09 | `align_to_satellite()` × 12 | Try LiteSAM each rotation |
| G09 | G04 | `get_cached_tile()` | Get expected tile |
| G09 | G15 | `get_inference_engine("LiteSAM")` | Get model |
| G06 | H07 | `calculate_rotation_from_points()` | Precise angle from homography |
| G06 | Internal | `update_heading()` | Store UAV heading |
| F02 | F05 | `get_next_image()` | Get image for processing |
| F02 | F06 | `requires_rotation_sweep()` | Check if sweep needed |
| F06 | H07 | `rotate_image()` × 12 | Rotate in 30° steps |
| F06 | F09 | `align_to_satellite(img, tile, bounds)` × 12 | Try LiteSAM each rotation |
| F06 | F04 | `get_cached_tile()` + `compute_tile_bounds()` | Get expected tile with bounds |
| F09 | F15 | `get_inference_engine("LiteSAM")` | Get model |
| F06 | H07 | `calculate_rotation_from_points()` | Precise angle from homography |
| F06 | F03 | `save_heading()` | Store UAV heading |
### Per-Frame Processing (Sequential VO)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G05 | G07 | `get_next_image()` | Provide image |
| G07 | G15 | `get_inference_engine("SuperPoint")` | Get feature extractor |
| G07 | G15 | `get_inference_engine("LightGlue")` | Get matcher |
| G07 | H05 | `start_timer()`, `end_timer()` | Monitor timing |
| G07 | G10 | `add_relative_factor()` | Add pose measurement |
| F02 | F12 | `get_active_chunk()` | Get active chunk for frame |
| F02 | F07 | Process frame | Provide image and chunk context |
| F07 | F16 | `get_inference_engine("SuperPoint")` | Get feature extractor |
| F07 | F16 | `get_inference_engine("LightGlue")` | Get matcher |
| F07 | H05 | `start_timer()`, `end_timer()` | Monitor timing |
| F07 | F10 | `add_relative_factor_to_chunk()` | Add pose measurement to chunk subgraph |
| F02 | F12 | `add_frame_to_chunk()` | Add frame to chunk |
### Tracking Good (Drift Correction)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G07 | G11 | `check_confidence()` | Check tracking quality |
| G11 | G09 | `align_to_satellite()` | Align to 1 tile |
| G09 | G04 | `get_tile_grid(1)` | Get single tile |
| G09 | G10 | `add_absolute_factor()` | Add GPS measurement |
| F02 | F11 | `check_confidence()` | Check tracking quality |
| F02 | F04 | `fetch_tile()` + `compute_tile_bounds()` | Get single tile with bounds |
| F02 | F09 | `align_to_satellite(img, tile, bounds)` | Align to 1 tile |
| F02 | F10 | `add_absolute_factor()` | Add GPS measurement |
### Tracking Lost (Progressive Search)
### Tracking Lost (Progressive Search + Chunk Building)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G07 | G11 | `check_confidence()` → FAIL | Low confidence |
| G11 | G06 | `requires_rotation_sweep()` | Trigger rotation sweep |
| G11 | G08 | `retrieve_candidate_tiles()` | Coarse localization |
| G08 | G15 | `get_inference_engine("DINOv2")` | Get model |
| G08 | H04 | `search()` | Query Faiss index |
| G08 | G04 | `get_tile_by_gps()` × 5 | Get candidate tiles |
| G11 | G04 | `expand_search_grid(4)` | Get 2×2 grid |
| G11 | G09 | `align_to_satellite()` | Try LiteSAM on 4 tiles |
| G11 (fail) | G04 | `expand_search_grid(9)` | Expand to 3×3 |
| G11 (fail) | G04 | `expand_search_grid(16)` | Expand to 4×4 |
| G11 (fail) | G04 | `expand_search_grid(25)` | Expand to 5×5 |
| G11 (fail) | G14 | `send_user_input_request()` | Request human help |
| G11 | G02 | `update_flight_status("BLOCKED")` | Block processing |
| F02 | F11 | `check_confidence()` → FAIL | Low confidence |
| F11 | F12 | `create_chunk_on_tracking_loss()` | **Proactive chunk creation** |
| F12 | F10 | `create_new_chunk()` | Create chunk in factor graph |
| F02 | F12 | `get_active_chunk()` | Get new active chunk |
| F11 | F06 | `requires_rotation_sweep()` | Trigger rotation sweep (single-image) |
| F11 | F08 | `retrieve_candidate_tiles()` | Coarse localization (single-image) |
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model |
| F08 | H04 | `search()` | Query Faiss index |
| F08 | F04 | `get_tile_by_gps()` × 5 | Get candidate tiles |
| F11 | F04 | `expand_search_grid(4)` | Get 2×2 grid |
| F11 | F09 | `align_to_satellite(img, tile, bounds)` | Try LiteSAM on tiles |
| F11 (fail) | F04 | `expand_search_grid(9)` | Expand to 3×3 |
| F11 (fail) | F04 | `expand_search_grid(16)` | Expand to 4×4 |
| F11 (fail) | F04 | `expand_search_grid(25)` | Expand to 5×5 |
| F11 (fail) | F03 | Continue building chunk | **Chunk building continues** |
| F11 (background) | F03 | `get_chunks_for_matching()` | Get unanchored chunks |
| F11 (background) | F08 | `retrieve_candidate_tiles_for_chunk()` | **Chunk semantic matching** |
| F11 (background) | F06 | `try_chunk_rotation_steps()` | **Chunk rotation sweeps** |
| F11 (background) | F09 | `align_chunk_to_satellite()` | **Chunk LiteSAM matching** |
| F11 (background) | F10 | `add_chunk_anchor()` + `merge_chunks()` | **Chunk merging** |
| F11 (fail) | F14 | `send_user_input_request()` | Request human help (last resort) |
| F11 | F02 | `update_flight_status("BLOCKED")` | Block processing |
### Optimization & Results
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G10 | H03 | `huber_loss()`, `cauchy_loss()` | Apply robust kernels |
| G10 | Internal | `optimize()` | Run iSAM2 optimization |
| G10 | G12 | `get_trajectory()` | Get optimized poses |
| G12 | H01 | `project()`, `unproject()` | Camera operations |
| G12 | H02 | `compute_gsd()` | GSD calculations |
| G12 | H06 | `tile_to_latlon()` | Coordinate transforms |
| G12 | G13 | Frame GPS + object coords | Provide results |
| G13 | G03 | `update_route_waypoint()` | Per-frame Route API update |
| G03 | Route API | `PUT /routes/.../waypoints/...` | HTTP call |
| G13 | G14 | `send_frame_result()` | Publish to client |
| G14 | Client | SSE `frame_processed` | Real-time delivery |
| G13 | G17 | `save_flight_state()` | Persist state |
| F10 | H03 | `huber_loss()`, `cauchy_loss()` | Apply robust kernels |
| F10 | Internal | `optimize()` | Run iSAM2 optimization |
| F02 | F10 | `get_trajectory()` | Get optimized poses |
| F02 | F13 | `enu_to_gps()` | Convert ENU to GPS |
| F13 | H01 | `project()`, `unproject()` | Camera operations |
| F13 | H02 | `compute_gsd()` | GSD calculations |
| F13 | H06 | `tile_to_latlon()` | Coordinate transforms |
| F02 | F14 | Frame GPS + object coords | Provide results |
| F14 | F02 | `update_waypoint()` | Per-frame waypoint update |
| F14 | F15 | `send_frame_result()` | Publish to client |
| F15 | Client | SSE `frame_processed` | Real-time delivery |
| F14 | F03 | `save_frame_result()` | Persist frame result |
### User Input Recovery
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G14 | Client | SSE `user_input_needed` | Notify client |
| Client | G01 | `POST .../user-fix` | Provide anchor |
| G01 | G11 | `apply_user_anchor()` | Apply fix |
| G11 | G10 | `add_absolute_factor()` (high confidence) | Hard constraint |
| G10 | Internal | `optimize()` | Re-optimize |
| G11 | G02 | `update_flight_status("PROCESSING")` | Resume |
| F15 | Client | SSE `user_input_needed` | Notify client |
| Client | F01 | `POST .../user-fix` | Provide anchor |
| F01 | F11 | `apply_user_anchor()` | Apply fix |
| F11 | F10 | `add_absolute_factor()` (high confidence) | Hard constraint |
| F10 | Internal | `optimize()` | Re-optimize |
| F11 | F02 | `update_flight_status("PROCESSING")` | Resume |
### Asynchronous Refinement
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G10 | Internal (background) | `optimize()` | Back-propagate anchors |
| G10 | G13 | `get_trajectory()` | Get refined poses |
| G13 | G03 | `batch_update_waypoints()` | Batch update Route API |
| G13 | G14 | `send_refinement()` × N | Send updates |
| G14 | Client | SSE `frame_refined` × N | Incremental updates |
| F10 | Internal (background) | `optimize()` | Back-propagate anchors |
| F10 | F14 | `get_trajectory()` | Get refined poses |
| F14 | F02 | `batch_update_waypoints()` | Batch update waypoints |
| F14 | F15 | `send_refinement()` × N | Send updates |
| F15 | Client | SSE `frame_refined` × N | Incremental updates |
### Chunk Matching (Background)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F11 (background) | F12 | `get_chunks_for_matching()` | Get unanchored chunks ready for matching |
| F11 | F12 | `get_chunk_images()` | Get chunk images |
| F11 | F08 | `retrieve_candidate_tiles_for_chunk()` | Chunk semantic matching (aggregate DINOv2) |
| F08 | F16 | `get_inference_engine("DINOv2")` | Get model for descriptor computation |
| F08 | H04 | `search()` | Query Faiss with chunk descriptor |
| F11 | F06 | `try_chunk_rotation_steps()` | Chunk rotation sweeps (12 rotations) |
| F06 | F09 | `align_chunk_to_satellite()` × 12 | Try LiteSAM for each rotation |
| F11 | F10 | `add_chunk_anchor()` | Anchor chunk with GPS |
| F11 | F10 | `merge_chunks()` | Merge chunk to main trajectory (Sim3) |
| F10 | Internal | `optimize_global()` | Global optimization after merging |
| F11 | F12 | `mark_chunk_anchored()` | Update chunk state |
### Cross-Cutting Concerns
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| G16 | ALL | `get_*_config()` | Provide configuration |
| H05 | G07, G08, G09, G10, G11 | `start_timer()`, `end_timer()` | Performance monitoring |
| F16 | ALL | `get_*_config()` | Provide configuration |
| H05 | F07, F08, F09, F10, F11 | `start_timer()`, `end_timer()` | Performance monitoring |
---
## Interaction Coverage Verification
✅ **Initialization**: G02→G15, G16, G17; G04→G08→H04
✅ **Flight creation**: Client→G01→G02→G03,G04,G16,G17,G14
✅ **Image upload**: Client→G01→G05→H08,G17
✅ **Rotation sweep**: G06→H07,G09 (12 iterations)
✅ **Sequential VO**: G07→G15,G10,H05
✅ **Drift correction**: G11→G09→G04(1),G10
✅ **Tracking loss**: G11→G06,G08,G04(progressive),G09,G14,G02
✅ **Global PR**: G08→G15,H04,G04
✅ **Optimization**: G10→H03,G12
✅ **Coordinate transform**: G12→H01,H02,H06
✅ **Results**: G12→G13→G03,G14,G17
✅ **User input**: Client→G01→G11→G10,G02
✅ **Refinement**: G10→G13→G03,G14
✅ **Configuration**: G16→ALL
✅ **Initialization**: F02→F15, F16, F17; F04→F08→H04
✅ **Flight creation**: Client→F01→F02→F04,F12,F16,F17,F14
✅ **Image upload**: Client→F01→F05→H08,F17
✅ **Rotation sweep**: F06→H07,F09 (12 iterations)
✅ **Sequential VO**: F07→F16,F10(chunk),F12,H05
✅ **Drift correction**: F02→F04,F09,F10
✅ **Tracking loss**: F11→F12(proactive chunk),F06,F08,F04(progressive),F09,F15,F02
✅ **Chunk building**: F02→F12→F10,F07
✅ **Chunk semantic matching**: F11→F12→F08(chunk descriptor)
✅ **Chunk LiteSAM matching**: F11→F06(chunk rotation)→F09(chunk alignment)
✅ **Chunk merging**: F11→F10(Sim3 transform)
✅ **Global PR**: F08→F16,H04,F04
✅ **Optimization**: F10→H03(chunk optimization, global optimization)
✅ **Coordinate transform**: F13→H01,H02,H06
✅ **Results**: F02→F13→F14→F15,F03
✅ **User input**: Client→F01→F11→F10,F02
✅ **Refinement**: F10→F14→F02,F15
✅ **Configuration**: F17→ALL
✅ **Performance**: H05→processing components
**All major component interactions are covered.**
@@ -324,13 +357,12 @@
## Deliverables
**Component Count**: 29 total
**Component Count**: 25 total
- Route API: 4 (R01-R04)
- GPS-Denied API: 17 (G01-G17)
- Flight API: 17 (F01-F17)
- Helpers: 8 (H01-H08)
**For each component**, create `docs/02_components/[project]_[##]_[component_name]/[component_name]_spec.md`:
**For each component**, create `docs/02_components/[##]_[component_name]/[component_name]_spec.md`:
1. **Interface Definition** (interface name, methods, contracts)
2. **Component Description** (responsibilities, scope)
@@ -340,25 +372,23 @@
6. **Dependencies** (which components it calls)
7. **Data Models**
**Generate draw.io diagram** showing:
- Two API projects (Route API, GPS-Denied API)
- All 29 components
- Route API ↔ GPS-Denied API communication
- GPS-Denied → Satellite Provider calls
- Rotation preprocessing flow
- Progressive search expansion (1→4→9→16→25)
- Per-frame Route API update flow
- Helper component usage
### To-dos
- [x] Create 4 Route API specs with interfaces (REST, data manager, validator, DB)
- [x] Create GPS-Denied core API specs with interfaces (REST, flight manager, Route client)
- [x] Create data management specs with interfaces (satellite, image pipeline, rotation)
- [x] Create visual processing specs with interfaces (VO, place recognition, LiteSAM)
- [x] Create coordination specs with interfaces (factor graph, failure recovery, transformer)
- [x] Create results/infrastructure specs with interfaces (result manager, SSE, models, config, DB)
- [x] Create 8 helper specs with interfaces
- [x] Generate draw.io with all components, interactions, flows
### Component Specifications Status
- [x] F01 Flight API (merged from R01 Route REST API)
- [x] F02 Flight Processor (merged from R02, R03, G02)
- [x] F04 Satellite Data Manager
- [x] F05 Image Input Pipeline
- [x] F06 Image Rotation Manager
- [x] F07 Sequential Visual Odometry
- [x] F08 Global Place Recognition
- [x] F09 Metric Refinement
- [x] F10 Factor Graph Optimizer
- [x] F11 Failure Recovery Coordinator
- [x] F12 Route Chunk Manager (Atlas multi-map chunk lifecycle)
- [x] F13 Coordinate Transformer (with ENU origin)
- [x] F14 Result Manager
- [x] F15 SSE Event Streamer
- [x] F16 Model Manager
- [x] F17 Configuration Manager
- [x] F03 Flight Database (merged from R04, G17)
- [x] Helper components (H01-H08)
docs/02_components/gps_denied_01_gps_denied_rest_api/gps_denied_rest_api_spec.md
-387
View File
@@ -1,387 +0,0 @@
# GPS-Denied REST API
## Interface Definition
**Interface Name**: `IGPSDeniedRestAPI`
### Interface Methods
```python
class IGPSDeniedRestAPI(ABC):
@abstractmethod
def create_flight(self, flight_data: FlightCreateRequest) -> FlightResponse:
pass
@abstractmethod
def upload_image_batch(self, flight_id: str, batch: ImageBatch) -> BatchResponse:
pass
@abstractmethod
def submit_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResponse:
pass
@abstractmethod
def get_flight_status(self, flight_id: str) -> FlightStatusResponse:
pass
@abstractmethod
def create_sse_stream(self, flight_id: str) -> SSEStream:
pass
```
## Component Description
### Responsibilities
- Expose REST API endpoints for GPS-Denied image processing pipeline
- Handle flight creation with satellite data prefetching
- Accept batch image uploads (10-50 images per request)
- Accept user-provided GPS fixes for blocked flights
- Provide real-time status updates
- Stream results via Server-Sent Events (SSE)
### Scope
- FastAPI-based REST endpoints
- Request/response validation
- Coordinate with Flight Manager for processing
- Multipart form data handling for image uploads
- SSE connection management
- Authentication and rate limiting
## API Methods
### `create_flight(flight_data: FlightCreateRequest) -> FlightResponse`
**REST Endpoint**: `POST /gps-denied/flights`
**Description**: Creates a new flight processing session, links to Route API, and prefetches satellite data.
**Called By**:
- Client applications (GPS-Denied UI)
**Input**:
```python
FlightCreateRequest:
route_id: str # UUID from Route API
start_gps: GPSPoint # Starting GPS coordinates (approximate)
camera_params: CameraParameters
rough_waypoints: List[GPSPoint] # Rough route for prefetching
```
**Output**:
```python
FlightResponse:
flight_id: str # UUID
status: str # "prefetching", "ready", "error"
message: Optional[str]
```
**Processing Flow**:
1. Validate request data
2. Call G02 Flight Manager → create_flight()
3. Flight Manager triggers satellite prefetch
4. Return flight_id immediately (prefetch is async)
**Error Conditions**:
- `400 Bad Request`: Invalid input data
- `404 Not Found`: route_id doesn't exist in Route API
- `500 Internal Server Error`: System error
**Test Cases**:
1. **Valid flight creation**: Returns 201 with flight_id
2. **Invalid route_id**: Returns 404
3. **Missing camera_params**: Returns 400
4. **Concurrent flight creation**: Multiple flights for same route → all succeed
---
### `upload_image_batch(flight_id: str, batch: ImageBatch) -> BatchResponse`
**REST Endpoint**: `POST /gps-denied/flights/{flightId}/images/batch`
**Description**: Uploads a batch of 10-50 UAV images for processing.
**Called By**:
- Client applications
**Input**:
```python
flight_id: str # Path parameter
ImageBatch: multipart/form-data
images: List[UploadFile] # 10-50 images
metadata: BatchMetadata
start_sequence: int
end_sequence: int
```
**Output**:
```python
BatchResponse:
accepted: bool
sequences: List[int] # [start, end]
next_expected: int
message: Optional[str]
```
**Processing Flow**:
1. Validate flight_id exists
2. Validate batch size (10-50 images)
3. Validate sequence numbers (strict sequential)
4. Pass to G05 Image Input Pipeline
5. Return immediately (processing is async)
**Error Conditions**:
- `400 Bad Request`: Invalid batch size, out-of-sequence images
- `404 Not Found`: flight_id doesn't exist
- `413 Payload Too Large`: Batch exceeds size limit
- `429 Too Many Requests`: Rate limit exceeded
**Test Cases**:
1. **Valid batch upload**: 20 images → returns 202 Accepted
2. **Out-of-sequence batch**: Sequence gap detected → returns 400
3. **Too many images**: 60 images → returns 400
4. **Large images**: 50 × 8MB images → successfully uploads
5. **Client-side resized images**: 2048×1536 → optimal processing
---
### `submit_user_fix(flight_id: str, fix_data: UserFixRequest) -> UserFixResponse`
**REST Endpoint**: `POST /gps-denied/flights/{flightId}/user-fix`
**Description**: Submits user-provided GPS anchor point to unblock failed localization.
**Called By**:
- Client applications (when user responds to `user_input_needed` event)
**Input**:
```python
UserFixRequest:
frame_id: int # Frame sequence number
uav_pixel: Tuple[float, float] # Pixel coordinates in UAV image
satellite_gps: GPSPoint # GPS corresponding to pixel location
```
**Output**:
```python
UserFixResponse:
accepted: bool
processing_resumed: bool
message: Optional[str]
```
**Processing Flow**:
1. Validate flight_id exists and is blocked
2. Pass to G11 Failure Recovery Coordinator
3. Coordinator applies anchor to Factor Graph
4. Resume processing pipeline
**Error Conditions**:
- `400 Bad Request`: Invalid fix data
- `404 Not Found`: flight_id or frame_id not found
- `409 Conflict`: Flight not in blocked state
**Test Cases**:
1. **Valid user fix**: Blocked flight → returns 200, processing resumes
2. **Fix for non-blocked flight**: Returns 409
3. **Invalid GPS coordinates**: Returns 400
4. **Multiple fixes**: Sequential fixes for different frames → all accepted
---
### `get_flight_status(flight_id: str) -> FlightStatusResponse`
**REST Endpoint**: `GET /gps-denied/flights/{flightId}/status`
**Description**: Retrieves current processing status of a flight.
**Called By**:
- Client applications (polling for status)
**Input**:
```python
flight_id: str
```
**Output**:
```python
FlightStatusResponse:
status: str # "prefetching", "processing", "blocked", "completed", "failed"
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float] # UAV heading in degrees
blocked: bool
search_grid_size: Optional[int] # 1, 4, 9, 16, or 25
message: Optional[str]
```
**Error Conditions**:
- `404 Not Found`: flight_id doesn't exist
**Test Cases**:
1. **Processing flight**: Returns current progress
2. **Blocked flight**: Returns blocked=true with search_grid_size
3. **Completed flight**: Returns status="completed" with final counts
---
### `create_sse_stream(flight_id: str) -> SSEStream`
**REST Endpoint**: `GET /gps-denied/flights/{flightId}/stream`
**Description**: Opens Server-Sent Events connection for real-time result streaming.
**Called By**:
- Client applications
**Input**:
```python
flight_id: str
```
**Output**:
```python
SSE Stream with events:
- frame_processed
- frame_refined
- search_expanded
- user_input_needed
- processing_blocked
- route_api_updated
- route_completed
```
**Event Format**:
```json
{
"event": "frame_processed",
"data": {
"frame_id": 237,
"gps": {"lat": 48.123, "lon": 37.456},
"altitude": 800.0,
"confidence": 0.95,
"heading": 87.3,
"timestamp": "2025-11-24T10:30:00Z"
}
}
```
**Error Conditions**:
- `404 Not Found`: flight_id doesn't exist
- Connection closed on client disconnect
**Test Cases**:
1. **Connect to stream**: Opens SSE connection successfully
2. **Receive frame events**: Process 100 frames → receive 100 events
3. **Receive user_input_needed**: Blocked frame → event sent
4. **Client reconnect**: Replay missed events from last_event_id
## Integration Tests
### Test 1: Complete Flight Processing Flow
1. POST /gps-denied/flights
2. GET /gps-denied/flights/{flightId}/stream (open SSE)
3. POST /gps-denied/flights/{flightId}/images/batch × 40 (2000 images total)
4. Receive frame_processed events via SSE
5. Receive route_completed event
### Test 2: User Fix Flow
1. Create flight and process images
2. Receive user_input_needed event
3. POST /gps-denied/flights/{flightId}/user-fix
4. Receive processing_resumed event
5. Continue receiving frame_processed events
### Test 3: Multiple Concurrent Flights
1. Create 10 flights concurrently
2. Upload batches to all flights in parallel
3. Stream results from all flights simultaneously
4. Verify no cross-contamination
## Non-Functional Requirements
### Performance
- **create_flight**: < 500ms response (prefetch is async)
- **upload_image_batch**: < 2 seconds for 50 × 2MB images
- **submit_user_fix**: < 200ms response
- **get_flight_status**: < 100ms
- **SSE latency**: < 500ms from event generation to client receipt
### Scalability
- Support 100 concurrent flight processing sessions
- Handle 1000+ concurrent SSE connections
- Support 10,000 requests per minute
### Reliability
- Request timeout: 30 seconds for batch uploads
- SSE keepalive: Ping every 30 seconds
- Automatic SSE reconnection with event replay
- Graceful handling of client disconnects
### Security
- API key authentication
- Rate limiting: 100 requests/minute per client
- Max upload size: 500MB per batch
- CORS configuration for web clients
## Dependencies
### Internal Components
- **G02 Flight Manager**: For all flight operations
- **G05 Image Input Pipeline**: For batch processing
- **G11 Failure Recovery Coordinator**: For user fixes
- **G14 SSE Event Streamer**: For real-time streaming
### External Dependencies
- **FastAPI**: Web framework
- **Uvicorn**: ASGI server
- **Pydantic**: Validation
- **python-multipart**: Multipart form handling
## Data Models
### FlightCreateRequest
```python
class GPSPoint(BaseModel):
lat: float
lon: float
class CameraParameters(BaseModel):
focal_length: float # mm
sensor_width: float # mm
sensor_height: float # mm
resolution_width: int # pixels
resolution_height: int # pixels
distortion_coefficients: Optional[List[float]] = None
class FlightCreateRequest(BaseModel):
route_id: str
start_gps: GPSPoint
camera_params: CameraParameters
rough_waypoints: List[GPSPoint]
altitude: float # Predefined altitude in meters
```
### BatchMetadata
```python
class BatchMetadata(BaseModel):
start_sequence: int # e.g., 101
end_sequence: int # e.g., 150
batch_number: int # e.g., 3
```
### FlightStatusResponse
```python
class FlightStatusResponse(BaseModel):
status: str
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
message: Optional[str]
created_at: datetime
updated_at: datetime
```
docs/02_components/gps_denied_02_flight_manager/flight_manager_spec.md
-358
View File
@@ -1,358 +0,0 @@
# Flight Manager
## Interface Definition
**Interface Name**: `IFlightManager`
### Interface Methods
```python
class IFlightManager(ABC):
@abstractmethod
def create_flight(self, flight_data: FlightData) -> str:
pass
@abstractmethod
def get_flight_state(self, flight_id: str) -> Optional[FlightState]:
pass
@abstractmethod
def link_to_route(self, flight_id: str, route_id: str) -> bool:
pass
@abstractmethod
def update_flight_status(self, flight_id: str, status: FlightStatus) -> bool:
pass
@abstractmethod
def initialize_system(self) -> bool:
pass
```
## Component Description
### Responsibilities
- Manage flight lifecycle (creation, state tracking, completion)
- Link flights to Route API routes
- Initialize system components (models, configurations, satellite database)
- Coordinate satellite data prefetching
- Track flight processing status and statistics
- Manage flight metadata persistence
### Scope
- Central coordinator for flight processing sessions
- System initialization and resource management
- Flight state machine management
- Integration point between REST API and processing components
## API Methods
### `create_flight(flight_data: FlightData) -> str`
**Description**: Creates a new flight processing session, initializes state, and triggers satellite prefetching.
**Called By**:
- G01 GPS-Denied REST API
**Input**:
```python
FlightData:
route_id: str
start_gps: GPSPoint
camera_params: CameraParameters
rough_waypoints: List[GPSPoint]
altitude: float
```
**Output**:
```python
flight_id: str # UUID
```
**Processing Flow**:
1. Generate flight_id (UUID)
2. Get flight configuration from G16 Configuration Manager
3. Get route info from G03 Route API Client
4. Initialize flight state
5. Trigger G04 Satellite Data Manager → prefetch_route_corridor()
6. Save flight state to G17 Database Layer
7. Return flight_id
**Error Conditions**:
- `RouteNotFoundError`: route_id doesn't exist
- `ConfigurationError`: Invalid camera parameters
- `DatabaseError`: Failed to persist state
**Test Cases**:
1. **Valid flight creation**: Returns flight_id, state persisted
2. **Invalid route_id**: Raises RouteNotFoundError
3. **Prefetch triggered**: Satellite manager receives prefetch request
4. **Concurrent creation**: 10 flights created simultaneously → all succeed
---
### `get_flight_state(flight_id: str) -> Optional[FlightState]`
**Description**: Retrieves current flight state including processing statistics.
**Called By**:
- G01 GPS-Denied REST API (for status endpoint)
- G05 Image Input Pipeline (to check flight exists)
- G11 Failure Recovery Coordinator (for state updates)
**Input**:
```python
flight_id: str
```
**Output**:
```python
FlightState:
flight_id: str
route_id: str
status: str # "prefetching", "ready", "processing", "blocked", "completed", "failed"
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
created_at: datetime
updated_at: datetime
cache_reference: str # Satellite data cache identifier
```
**Error Conditions**:
- Returns `None`: Flight not found (not an error condition)
**Test Cases**:
1. **Get existing flight**: Returns complete FlightState
2. **Get non-existent flight**: Returns None
3. **Get during processing**: Returns accurate frame count
---
### `link_to_route(flight_id: str, route_id: str) -> bool`
**Description**: Links a flight to its Route API route for waypoint updates.
**Called By**:
- Internal (during create_flight)
**Input**:
```python
flight_id: str
route_id: str
```
**Output**:
```python
bool: True if linked, False if flight doesn't exist
```
**Processing Flow**:
1. Verify flight exists
2. Verify route exists via G03 Route API Client
3. Store linkage in flight state
4. Update database
**Error Conditions**:
- `RouteNotFoundError`: route_id invalid
**Test Cases**:
1. **Valid linkage**: Returns True
2. **Invalid flight_id**: Returns False
3. **Invalid route_id**: Raises RouteNotFoundError
---
### `update_flight_status(flight_id: str, status: FlightStatus) -> bool`
**Description**: Updates flight processing status (processing, blocked, completed, etc.).
**Called By**:
- G05 Image Input Pipeline (status transitions)
- G11 Failure Recovery Coordinator (blocked/resumed)
- G13 Result Manager (completed)
**Input**:
```python
flight_id: str
status: FlightStatus:
status_type: str # "processing", "blocked", "completed", "failed"
frames_processed: Optional[int]
current_frame: Optional[int]
current_heading: Optional[float]
blocked: Optional[bool]
search_grid_size: Optional[int]
message: Optional[str]
```
**Output**:
```python
bool: True if updated, False if flight not found
```
**Processing Flow**:
1. Load current flight state
2. Update relevant fields
3. Update updated_at timestamp
4. Persist to G17 Database Layer
5. Return success
**Error Conditions**:
- Returns False if flight not found
**Test Cases**:
1. **Update to processing**: status="processing" → updates successfully
2. **Update to blocked**: blocked=True, search_grid_size=9 → updates
3. **Resume from blocked**: blocked=False → processing continues
4. **Concurrent updates**: Multiple simultaneous updates → all persist correctly
---
### `initialize_system() -> bool`
**Description**: Initializes system components on startup (models, configurations, satellite database).
**Called By**:
- System startup (main application)
**Input**:
- None
**Output**:
```python
bool: True if initialization successful
```
**Processing Flow**:
1. Load system configuration from G16 Configuration Manager
2. Initialize ML models via G15 Model Manager:
- Load SuperPoint model
- Load LightGlue model
- Load DINOv2 model
- Load LiteSAM model
3. Initialize G08 Global Place Recognition → build satellite descriptor database
4. Initialize G04 Satellite Data Manager cache
5. Verify all components ready
**Error Conditions**:
- `InitializationError`: Component initialization failed
- `ModelLoadError`: ML model loading failed
**Test Cases**:
1. **Clean startup**: All models load successfully
2. **Missing model file**: Raises ModelLoadError
3. **Configuration error**: Raises InitializationError
4. **Partial initialization failure**: Cleanup and raise error
## Integration Tests
### Test 1: Flight Lifecycle
1. initialize_system()
2. create_flight() with valid data
3. get_flight_state() → verify "prefetching"
4. Wait for prefetch completion
5. update_flight_status("processing")
6. get_flight_state() → verify "processing"
7. update_flight_status("completed")
### Test 2: Multiple Concurrent Flights
1. create_flight() × 10 concurrently
2. update_flight_status() for all flights in parallel
3. get_flight_state() for all flights
4. Verify no state cross-contamination
### Test 3: System Initialization
1. initialize_system()
2. Verify all 4 models loaded
3. Verify satellite database ready
4. Create flight immediately after init → succeeds
## Non-Functional Requirements
### Performance
- **create_flight**: < 300ms (excluding prefetch which is async)
- **get_flight_state**: < 50ms
- **update_flight_status**: < 30ms
- **initialize_system**: < 30 seconds (one-time startup cost)
### Scalability
- Support 1000+ concurrent flight sessions
- Handle 100 status updates per second
- Maintain state for up to 10,000 flights (historical data)
### Reliability
- Graceful handling of component initialization failures
- Flight state persistence survives process restarts
- Transaction safety for concurrent updates
- Automatic cleanup of completed flights after 7 days
## Dependencies
### Internal Components
- **G03 Route API Client**: For route validation and metadata
- **G04 Satellite Data Manager**: For prefetch operations
- **G08 Global Place Recognition**: For descriptor database initialization
- **G15 Model Manager**: For ML model loading
- **G16 Configuration Manager**: For system configuration
- **G17 GPS-Denied Database Layer**: For state persistence
### External Dependencies
- None (coordinates with internal components)
## Data Models
### FlightData
```python
class FlightData(BaseModel):
route_id: str
start_gps: GPSPoint
camera_params: CameraParameters
rough_waypoints: List[GPSPoint]
altitude: float
```
### FlightState
```python
class FlightState(BaseModel):
flight_id: str
route_id: str
status: str
frames_processed: int = 0
frames_total: int = 0
current_frame: Optional[int] = None
current_heading: Optional[float] = None
blocked: bool = False
search_grid_size: Optional[int] = None
created_at: datetime
updated_at: datetime
cache_reference: str
camera_params: CameraParameters
altitude: float
start_gps: GPSPoint
```
### FlightStatus (Update DTO)
```python
class FlightStatus(BaseModel):
status_type: str
frames_processed: Optional[int] = None
current_frame: Optional[int] = None
current_heading: Optional[float] = None
blocked: Optional[bool] = None
search_grid_size: Optional[int] = None
message: Optional[str] = None
```
### SystemState
```python
class SystemState(BaseModel):
initialized: bool
models_loaded: Dict[str, bool] # {"SuperPoint": True, "LightGlue": True, ...}
satellite_db_ready: bool
active_flights_count: int
initialization_timestamp: datetime
```
docs/02_components/gps_denied_03_route_api_client/route_api_client_spec.md
-331
View File
@@ -1,331 +0,0 @@
# Route API Client
## Interface Definition
**Interface Name**: `IRouteAPIClient`
### Interface Methods
```python
class IRouteAPIClient(ABC):
@abstractmethod
def update_route_waypoint(self, route_id: str, waypoint_id: str, waypoint: Waypoint) -> bool:
pass
@abstractmethod
def get_route_info(self, route_id: str) -> Optional[RouteInfo]:
pass
@abstractmethod
def batch_update_waypoints(self, route_id: str, waypoints: List[Waypoint]) -> BatchUpdateResult:
pass
```
## Component Description
### Responsibilities
- HTTP client for communicating with Route API
- Send per-frame GPS refinements to Route API
- Retrieve route metadata and waypoints
- Handle batch waypoint updates for trajectory refinements
- Manage connection pooling and retry logic
- Handle HTTP errors and timeouts
### Scope
- Synchronous HTTP client (requests library)
- Waypoint update operations
- Route metadata retrieval
- Error handling and retries
- Rate limiting and backpressure management
## API Methods
### `update_route_waypoint(route_id: str, waypoint_id: str, waypoint: Waypoint) -> bool`
**Description**: Updates a single waypoint in Route API. Called per-frame after GPS calculation.
**Called By**:
- G13 Result Manager (per-frame update)
**Input**:
```python
route_id: str
waypoint_id: str # Frame sequence number
waypoint: Waypoint:
lat: float
lon: float
altitude: Optional[float]
confidence: float
timestamp: datetime
refined: bool # Always True for GPS-Denied updates
```
**Output**:
```python
bool: True if updated successfully, False on failure
```
**HTTP Request**:
```
PUT /routes/{route_id}/waypoints/{waypoint_id}
Content-Type: application/json
{
"lat": 48.123456,
"lon": 37.654321,
"altitude": 800.0,
"confidence": 0.95,
"timestamp": "2025-11-24T10:30:00Z",
"refined": true
}
```
**Error Handling**:
- **Retry**: 3 attempts with exponential backoff (1s, 2s, 4s)
- **Timeout**: 5 seconds per request
- **404 Not Found**: Log warning, return False
- **429 Too Many Requests**: Backoff and retry
- **500 Server Error**: Retry with backoff
**Error Conditions**:
- Returns `False`: Update failed after retries
- Logs errors but doesn't raise exceptions (non-critical path)
**Test Cases**:
1. **Successful update**: Returns True
2. **Route API unavailable**: Retries 3 times, returns False
3. **Waypoint not found**: Returns False
4. **Network timeout**: Retries, returns False if all fail
5. **High-frequency updates**: 100 updates/sec sustained
---
### `get_route_info(route_id: str) -> Optional[RouteInfo]`
**Description**: Retrieves route metadata including rough waypoints and geofences.
**Called By**:
- G02 Flight Manager (during flight creation)
**Input**:
```python
route_id: str
```
**Output**:
```python
RouteInfo:
route_id: str
name: str
description: str
points: List[GPSPoint] # Rough waypoints
geofences: Geofences
waypoint_count: int
created_at: datetime
```
**HTTP Request**:
```
GET /routes/{route_id}
```
**Error Handling**:
- **Retry**: 3 attempts for transient errors
- **Timeout**: 10 seconds
- **404 Not Found**: Return None
- **500 Server Error**: Retry with backoff
**Error Conditions**:
- Returns `None`: Route not found or error after retries
- `RouteAPIError`: Critical error retrieving route
**Test Cases**:
1. **Existing route**: Returns complete RouteInfo
2. **Non-existent route**: Returns None
3. **Large route**: 2000+ waypoints → returns successfully
4. **Concurrent requests**: 10 simultaneous requests → all succeed
---
### `batch_update_waypoints(route_id: str, waypoints: List[Waypoint]) -> BatchUpdateResult`
**Description**: Updates multiple waypoints in a single request. Used for trajectory refinements.
**Called By**:
- G13 Result Manager (asynchronous refinement updates)
**Input**:
```python
route_id: str
waypoints: List[Waypoint] # Refined waypoints
```
**Output**:
```python
BatchUpdateResult:
success: bool
updated_count: int
failed_ids: List[str]
```
**HTTP Request**:
```
PUT /routes/{route_id}/waypoints/batch
Content-Type: application/json
{
"waypoints": [
{
"id": "AD000237",
"lat": 48.123,
"lon": 37.654,
"altitude": 800.0,
"confidence": 0.97,
"timestamp": "2025-11-24T10:30:00Z",
"refined": true
},
...
]
}
```
**Error Handling**:
- **Partial success**: Some waypoints update, some fail
- **Retry**: 3 attempts for complete batch
- **Timeout**: 30 seconds (larger batches)
- Returns updated_count and failed_ids
**Error Conditions**:
- Returns `success=False` with failed_ids list
**Test Cases**:
1. **Batch update 100 waypoints**: All succeed
2. **Partial failure**: 5 waypoints fail → returns failed_ids
3. **Empty batch**: Returns success=True, updated_count=0
4. **Large batch**: 500 waypoints → splits into sub-batches
## Integration Tests
### Test 1: Per-Frame Update Flow
1. Create flight and process 100 frames
2. update_route_waypoint() × 100 sequentially
3. Verify all updates successful via get_route_info()
4. Verify waypoints marked as refined=True
### Test 2: Refinement Batch Update
1. Process route, track 200 frames needing refinement
2. batch_update_waypoints() with 200 waypoints
3. Verify all updates applied
4. Handle partial failures gracefully
### Test 3: Error Recovery
1. Simulate Route API downtime
2. Attempt update_route_waypoint() → retries 3 times
3. Route API comes back online
4. Next update succeeds
### Test 4: High-Frequency Updates
1. Send 200 waypoint updates sequentially
2. Measure throughput and success rate
3. Verify no rate limiting issues
4. Verify all updates persisted
## Non-Functional Requirements
### Performance
- **update_route_waypoint**: < 100ms average latency (critical path)
- **get_route_info**: < 200ms
- **batch_update_waypoints**: < 2 seconds for 100 waypoints
- **Throughput**: Support 100 waypoint updates per second
### Scalability
- Connection pool: 20-50 connections
- Handle 1000+ waypoint updates per flight (2000+ frame flight)
- Support concurrent updates from multiple flights
### Reliability
- **Retry strategy**: 3 attempts with exponential backoff
- **Circuit breaker**: Temporarily stop requests after 5 consecutive failures
- **Timeout management**: Progressive timeouts (5s, 10s, 30s)
- **Graceful degradation**: Continue processing even if Route API unavailable
### Monitoring
- Track success/failure rates
- Monitor latency percentiles (p50, p95, p99)
- Alert on high failure rates
- Log all HTTP errors
## Dependencies
### Internal Components
- None (external HTTP client)
### External Dependencies
- **Route API**: External REST API service
- **requests** or **httpx**: HTTP client library
- **tenacity**: Retry library
- **urllib3**: Connection pooling
## Data Models
### RouteInfo
```python
class GPSPoint(BaseModel):
lat: float
lon: float
class Geofences(BaseModel):
polygons: List[Polygon]
class RouteInfo(BaseModel):
route_id: str
name: str
description: str
points: List[GPSPoint]
geofences: Geofences
waypoint_count: int
created_at: datetime
```
### Waypoint
```python
class Waypoint(BaseModel):
id: str
lat: float
lon: float
altitude: Optional[float]
confidence: float
timestamp: datetime
refined: bool
```
### BatchUpdateResult
```python
class BatchUpdateResult(BaseModel):
success: bool
updated_count: int
failed_ids: List[str]
errors: Optional[Dict[str, str]] # waypoint_id -> error_message
```
### HTTPConfig
```python
class HTTPConfig(BaseModel):
route_api_base_url: str # e.g., "http://localhost:8000"
timeout: int = 5 # seconds
max_retries: int = 3
retry_backoff: float = 1.0 # seconds
connection_pool_size: int = 50
max_batch_size: int = 500
```
### Retry Strategy
```python
retry_strategy = {
"stop": "stop_after_attempt(3)",
"wait": "wait_exponential(multiplier=1, min=1, max=10)",
"retry": "retry_if_exception_type((ConnectionError, Timeout, HTTPError))",
"reraise": True
}
```
docs/02_components/gps_denied_10_factor_graph_optimizer/factor_graph_optimizer_spec.md
-362
View File
@@ -1,362 +0,0 @@
# Factor Graph Optimizer
## Interface Definition
**Interface Name**: `IFactorGraphOptimizer`
### Interface Methods
```python
class IFactorGraphOptimizer(ABC):
@abstractmethod
def add_relative_factor(self, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
pass
@abstractmethod
def add_absolute_factor(self, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool:
pass
@abstractmethod
def add_altitude_prior(self, frame_id: int, altitude: float, covariance: float) -> bool:
pass
@abstractmethod
def optimize(self, iterations: int) -> OptimizationResult:
pass
@abstractmethod
def get_trajectory(self) -> Dict[int, Pose]:
pass
@abstractmethod
def get_marginal_covariance(self, frame_id: int) -> np.ndarray:
pass
```
## Component Description
### Responsibilities
- GTSAM-based fusion of relative and absolute measurements
- Incremental optimization (iSAM2) for real-time performance
- Robust kernels (Huber/Cauchy) for 350m outlier handling
- Scale resolution through altitude priors and absolute GPS
- Trajectory smoothing and global consistency
- Back-propagation of refinements to previous frames
### Scope
- Non-linear least squares optimization
- Factor graph representation of SLAM problem
- Handles monocular scale ambiguity
- Real-time incremental updates
- Asynchronous batch refinement
## API Methods
### `add_relative_factor(frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool`
**Description**: Adds relative pose measurement between consecutive frames.
**Called By**:
- G07 Sequential VO (frame-to-frame odometry)
**Input**:
```python
frame_i: int # Previous frame ID
frame_j: int # Current frame ID (typically frame_i + 1)
relative_pose: RelativePose:
translation: np.ndarray # (3,) - in meters (scale from altitude prior)
rotation: np.ndarray # (3, 3) or quaternion
covariance: np.ndarray # (6, 6) - uncertainty
```
**Output**:
```python
bool: True if factor added successfully
```
**Processing Flow**:
1. Create BetweenFactor in GTSAM
2. Apply robust kernel (Huber) to handle outliers
3. Add to factor graph
4. Mark graph as needing optimization
**Robust Kernel**:
- **Huber loss**: Downweights large errors (>threshold)
- **Critical** for 350m outlier handling from tilt
**Test Cases**:
1. **Normal motion**: Factor added, contributes to optimization
2. **Large displacement** (350m outlier): Huber kernel reduces weight
3. **Consecutive factors**: Chain of relative factors builds trajectory
---
### `add_absolute_factor(frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool`
**Description**: Adds absolute GPS measurement for drift correction or user anchor.
**Called By**:
- G09 Metric Refinement (after LiteSAM alignment)
- G11 Failure Recovery Coordinator (user-provided anchors)
**Input**:
```python
frame_id: int
gps: GPSPoint:
lat: float
lon: float
covariance: np.ndarray # (2, 2) or (3, 3) - GPS uncertainty
is_user_anchor: bool # True for user-provided fixes (high confidence)
```
**Output**:
```python
bool: True if factor added
```
**Processing Flow**:
1. Convert GPS to local ENU coordinates (East-North-Up)
2. Create PriorFactor or UnaryFactor
3. Set covariance (low for user anchors, higher for LiteSAM)
4. Add to factor graph
5. Trigger optimization (immediate for user anchors)
**Covariance Settings**:
- **User anchor**: σ = 5m (high confidence)
- **LiteSAM match**: σ = 20-50m (depends on confidence)
**Test Cases**:
1. **LiteSAM GPS**: Adds absolute factor, corrects drift
2. **User anchor**: High confidence, immediately refines trajectory
3. **Multiple absolute factors**: Graph optimizes to balance all
---
### `add_altitude_prior(frame_id: int, altitude: float, covariance: float) -> bool`
**Description**: Adds altitude constraint to resolve monocular scale ambiguity.
**Called By**:
- Main processing loop (for each frame)
**Input**:
```python
frame_id: int
altitude: float # Predefined altitude in meters
covariance: float # Altitude uncertainty (e.g., 50m)
```
**Output**:
```python
bool: True if prior added
```
**Processing Flow**:
1. Create UnaryFactor for Z-coordinate
2. Set as soft constraint (not hard constraint)
3. Add to factor graph
**Purpose**:
- Resolves scale ambiguity in monocular VO
- Prevents scale drift (trajectory collapsing or exploding)
- Soft constraint allows adjustment based on absolute GPS
**Test Cases**:
1. **Without altitude prior**: Scale drifts over time
2. **With altitude prior**: Scale stabilizes
3. **Conflicting measurements**: Optimizer balances VO and altitude
---
### `optimize(iterations: int) -> OptimizationResult`
**Description**: Runs optimization to refine trajectory.
**Called By**:
- Main processing loop (incremental after each frame)
- Asynchronous refinement thread (batch optimization)
**Input**:
```python
iterations: int # Max iterations (typically 5-10 for incremental, 50-100 for batch)
```
**Output**:
```python
OptimizationResult:
converged: bool
final_error: float
iterations_used: int
optimized_frames: List[int] # Frames with updated poses
```
**Processing Details**:
- **Incremental** (iSAM2): Updates only affected nodes
- **Batch**: Re-optimizes entire trajectory when new absolute factors added
- **Robust M-estimation**: Automatically downweights outliers
**Optimization Algorithm** (Levenberg-Marquardt):
1. Linearize factor graph around current estimate
2. Solve linear system
3. Update pose estimates
4. Check convergence (error reduction < threshold)
**Test Cases**:
1. **Incremental optimization**: Fast (<100ms), local update
2. **Batch optimization**: Slower (~500ms), refines entire trajectory
3. **Convergence**: Error reduces, converges within iterations
---
### `get_trajectory() -> Dict[int, Pose]`
**Description**: Retrieves complete optimized trajectory.
**Called By**:
- G13 Result Manager (for publishing results)
- G12 Coordinate Transformer (for GPS conversion)
**Input**: None
**Output**:
```python
Dict[int, Pose]:
frame_id -> Pose:
position: np.ndarray # (x, y, z) in ENU
orientation: np.ndarray # Quaternion or rotation matrix
timestamp: datetime
```
**Processing Flow**:
1. Extract all pose estimates from graph
2. Convert to appropriate coordinate system
3. Return dictionary
**Test Cases**:
1. **After optimization**: Returns all frame poses
2. **Refined trajectory**: Poses updated after batch optimization
---
### `get_marginal_covariance(frame_id: int) -> np.ndarray`
**Description**: Gets uncertainty (covariance) of a pose estimate.
**Called By**:
- G11 Failure Recovery Coordinator (to detect high uncertainty)
**Input**:
```python
frame_id: int
```
**Output**:
```python
np.ndarray: (6, 6) covariance matrix [x, y, z, roll, pitch, yaw]
```
**Purpose**:
- Uncertainty quantification
- Trigger user input when uncertainty too high (> 50m radius)
**Test Cases**:
1. **Well-constrained pose**: Small covariance
2. **Unconstrained pose**: Large covariance
3. **After absolute factor**: Covariance reduces
## Integration Tests
### Test 1: Incremental Trajectory Building
1. Initialize graph with first frame
2. Add relative factors from VO × 100
3. Add altitude priors × 100
4. Optimize incrementally after each frame
5. Verify smooth trajectory
### Test 2: Drift Correction with Absolute GPS
1. Build trajectory with VO only (will drift)
2. Add absolute GPS factor at frame 50
3. Optimize → trajectory corrects
4. Verify frames 1-49 also corrected (back-propagation)
### Test 3: Outlier Handling
1. Add normal relative factors
2. Add 350m outlier factor (tilt error)
3. Optimize with robust kernel
4. Verify outlier downweighted, trajectory smooth
### Test 4: User Anchor Integration
1. Processing blocked at frame 237
2. User provides anchor (high confidence)
3. add_absolute_factor(is_user_anchor=True)
4. Optimize → trajectory snaps to anchor
## Non-Functional Requirements
### Performance
- **Incremental optimize**: < 100ms per frame (iSAM2)
- **Batch optimize**: < 500ms for 100 frames
- **get_trajectory**: < 10ms
- Real-time capable: 10 FPS processing
### Accuracy
- **Mean Reprojection Error (MRE)**: < 1.0 pixels
- **GPS accuracy**: Meet 80% < 50m, 60% < 20m criteria
- **Trajectory smoothness**: No sudden jumps (except user anchors)
### Reliability
- Numerical stability for 2000+ frame trajectories
- Graceful handling of degenerate configurations
- Robust to missing/corrupted measurements
## Dependencies
### Internal Components
- **H03 Robust Kernels**: For Huber/Cauchy loss functions
- **H02 GSD Calculator**: For coordinate conversions
### External Dependencies
- **GTSAM**: Graph optimization library
- **numpy**: Matrix operations
- **scipy**: Sparse matrix operations (optional)
## Data Models
### Pose
```python
class Pose(BaseModel):
frame_id: int
position: np.ndarray # (3,) - [x, y, z] in ENU
orientation: np.ndarray # (4,) quaternion or (3,3) rotation matrix
timestamp: datetime
covariance: Optional[np.ndarray] # (6, 6)
```
### RelativePose
```python
class RelativePose(BaseModel):
translation: np.ndarray # (3,)
rotation: np.ndarray # (3, 3) or (4,)
covariance: np.ndarray # (6, 6)
```
### OptimizationResult
```python
class OptimizationResult(BaseModel):
converged: bool
final_error: float
iterations_used: int
optimized_frames: List[int]
mean_reprojection_error: float
```
### FactorGraphConfig
```python
class FactorGraphConfig(BaseModel):
robust_kernel_type: str = "Huber" # or "Cauchy"
huber_threshold: float = 1.0 # pixels
cauchy_k: float = 0.1
isam2_relinearize_threshold: float = 0.1
isam2_relinearize_skip: int = 1
```
docs/02_components/gps_denied_11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md
-404
View File
@@ -1,404 +0,0 @@
# Failure Recovery Coordinator
## Interface Definition
**Interface Name**: `IFailureRecoveryCoordinator`
### Interface Methods
```python
class IFailureRecoveryCoordinator(ABC):
@abstractmethod
def check_confidence(self, vo_result: RelativePose, litesam_result: Optional[AlignmentResult]) -> ConfidenceAssessment:
pass
@abstractmethod
def detect_tracking_loss(self, confidence: ConfidenceAssessment) -> bool:
pass
@abstractmethod
def start_search(self, flight_id: str, frame_id: int, estimated_gps: GPSPoint) -> SearchSession:
pass
@abstractmethod
def expand_search_radius(self, session: SearchSession) -> List[TileCoords]:
pass
@abstractmethod
def try_current_grid(self, session: SearchSession, tiles: Dict[str, np.ndarray]) -> Optional[AlignmentResult]:
pass
@abstractmethod
def mark_found(self, session: SearchSession, result: AlignmentResult) -> bool:
pass
@abstractmethod
def get_search_status(self, session: SearchSession) -> SearchStatus:
pass
@abstractmethod
def create_user_input_request(self, flight_id: str, frame_id: int, candidate_tiles: List[TileCandidate]) -> UserInputRequest:
pass
@abstractmethod
def apply_user_anchor(self, flight_id: str, frame_id: int, anchor: UserAnchor) -> bool:
pass
```
## Component Description
### Responsibilities
- Monitor confidence metrics (inlier count, MRE, covariance)
- Detect tracking loss and trigger recovery
- Coordinate progressive tile search (1→4→9→16→25)
- Handle human-in-the-loop when all strategies exhausted
- Block flight processing when awaiting user input
- Apply user-provided anchors to Factor Graph
### Scope
- Confidence monitoring
- Progressive search coordination
- User input request/response handling
- Recovery strategy orchestration
- Integration point for G04, G06, G08, G09, G10
## API Methods
### `check_confidence(vo_result: RelativePose, litesam_result: Optional[AlignmentResult]) -> ConfidenceAssessment`
**Description**: Assesses tracking confidence from VO and LiteSAM results.
**Called By**: Main processing loop (per frame)
**Input**:
```python
vo_result: RelativePose
litesam_result: Optional[AlignmentResult]
```
**Output**:
```python
ConfidenceAssessment:
overall_confidence: float # 0-1
vo_confidence: float
litesam_confidence: float
inlier_count: int
tracking_status: str # "good", "degraded", "lost"
```
**Confidence Metrics**:
- VO inlier count and ratio
- LiteSAM match confidence
- Factor graph marginal covariance
- Reprojection error
**Thresholds**:
- **Good**: VO inliers > 50, LiteSAM confidence > 0.7
- **Degraded**: VO inliers 20-50
- **Lost**: VO inliers < 20
**Test Cases**:
1. Good tracking → "good" status
2. Low overlap → "degraded"
3. Sharp turn → "lost"
---
### `detect_tracking_loss(confidence: ConfidenceAssessment) -> bool`
**Description**: Determines if tracking is lost.
**Called By**: Main processing loop
**Input**: `ConfidenceAssessment`
**Output**: `bool` - True if tracking lost
**Test Cases**:
1. Confidence good → False
2. Confidence lost → True
---
### `start_search(flight_id: str, frame_id: int, estimated_gps: GPSPoint) -> SearchSession`
**Description**: Initiates progressive search session.
**Called By**: Main processing loop (when tracking lost)
**Input**:
```python
flight_id: str
frame_id: int
estimated_gps: GPSPoint # Dead-reckoning estimate
```
**Output**:
```python
SearchSession:
session_id: str
flight_id: str
frame_id: int
center_gps: GPSPoint
current_grid_size: int # Starts at 1
max_grid_size: int # 25
found: bool
```
**Processing Flow**:
1. Create search session
2. Set center from estimated_gps
3. Set current_grid_size = 1
4. Return session
**Test Cases**:
1. Start search → session created with grid_size=1
---
### `expand_search_radius(session: SearchSession) -> List[TileCoords]`
**Description**: Expands search grid to next size (1→4→9→16→25).
**Called By**: Internal (after try_current_grid fails)
**Input**: `SearchSession`
**Output**: `List[TileCoords]` - Tiles for next grid size
**Processing Flow**:
1. Increment current_grid_size (1→4→9→16→25)
2. Call G04.expand_search_grid() to get new tiles only
3. Return new tile coordinates
**Test Cases**:
1. Expand 1→4 → returns 3 new tiles
2. Expand 4→9 → returns 5 new tiles
3. At grid_size=25 → no more expansion
---
### `try_current_grid(session: SearchSession, tiles: Dict[str, np.ndarray]) -> Optional[AlignmentResult]`
**Description**: Tries LiteSAM matching on current tile grid.
**Called By**: Internal (progressive search loop)
**Input**:
```python
session: SearchSession
tiles: Dict[str, np.ndarray] # From G04
```
**Output**: `Optional[AlignmentResult]` - Match result or None
**Processing Flow**:
1. Get UAV image for frame_id
2. For each tile in grid:
- Call G09.align_to_satellite(uav_image, tile)
- If match found with confidence > threshold:
- mark_found(session, result)
- Return result
3. Return None if no match
**Test Cases**:
1. Match on 3rd tile → returns result
2. No match in grid → returns None
---
### `mark_found(session: SearchSession, result: AlignmentResult) -> bool`
**Description**: Marks search session as successful.
**Called By**: Internal
**Input**:
```python
session: SearchSession
result: AlignmentResult
```
**Output**: `bool` - True
**Processing Flow**:
1. Set session.found = True
2. Log success (grid_size where found)
3. Resume processing
---
### `get_search_status(session: SearchSession) -> SearchStatus`
**Description**: Gets current search status.
**Called By**: G01 REST API (for status endpoint)
**Output**:
```python
SearchStatus:
current_grid_size: int
found: bool
exhausted: bool # Reached grid_size=25 without match
```
---
### `create_user_input_request(flight_id: str, frame_id: int, candidate_tiles: List[TileCandidate]) -> UserInputRequest`
**Description**: Creates user input request when all search strategies exhausted.
**Called By**: Internal (when grid_size=25 and no match)
**Input**:
```python
flight_id: str
frame_id: int
candidate_tiles: List[TileCandidate] # Top-5 from G08
```
**Output**:
```python
UserInputRequest:
request_id: str
flight_id: str
frame_id: int
uav_image: np.ndarray
candidate_tiles: List[TileCandidate]
message: str
```
**Processing Flow**:
1. Get UAV image for frame_id
2. Get top-5 candidates from G08
3. Create request
4. Send via G14 SSE → "user_input_needed" event
5. Update G02 flight_status("BLOCKED")
**Test Cases**:
1. All search failed → creates request
2. Request sent to client via SSE
---
### `apply_user_anchor(flight_id: str, frame_id: int, anchor: UserAnchor) -> bool`
**Description**: Applies user-provided GPS anchor.
**Called By**: G01 REST API (user-fix endpoint)
**Input**:
```python
flight_id: str
frame_id: int
anchor: UserAnchor:
uav_pixel: Tuple[float, float]
satellite_gps: GPSPoint
```
**Output**: `bool` - True if applied
**Processing Flow**:
1. Validate anchor data
2. Call G10.add_absolute_factor(frame_id, gps, is_user_anchor=True)
3. G10.optimize() → refines trajectory
4. Update G02 flight_status("PROCESSING")
5. Resume processing from next frame
**Test Cases**:
1. Valid anchor → applied, processing resumes
2. Invalid anchor → rejected
## Integration Tests
### Test 1: Progressive Search Flow
1. Tracking lost detected
2. start_search() → grid_size=1
3. try_current_grid(1 tile) → no match
4. expand_search_radius() → grid_size=4
5. try_current_grid(4 tiles) → match found
6. mark_found() → success
### Test 2: Full Search Exhaustion
1. start_search()
2. try grids: 1→4→9→16→25, all fail
3. create_user_input_request()
4. User provides anchor
5. apply_user_anchor() → processing resumes
### Test 3: Confidence Monitoring
1. Normal frames → confidence good
2. Low overlap frame → confidence degraded
3. Sharp turn → tracking lost, trigger search
## Non-Functional Requirements
### Performance
- **check_confidence**: < 10ms
- **Progressive search (25 tiles)**: < 1.5s total
- **User input latency**: < 500ms from creation to SSE event
### Reliability
- Always exhausts all search strategies before requesting user input
- Guarantees processing block when awaiting user input
- Graceful recovery from all failure modes
## Dependencies
### Internal Components
- G04 Satellite Data Manager (tile grids)
- G06 Image Rotation Manager (rotation sweep)
- G08 Global Place Recognition (candidates)
- G09 Metric Refinement (LiteSAM)
- G10 Factor Graph Optimizer (anchor application)
- G02 Flight Manager (status updates)
- G14 SSE Event Streamer (user input events)
### External Dependencies
- None
## Data Models
### ConfidenceAssessment
```python
class ConfidenceAssessment(BaseModel):
overall_confidence: float
vo_confidence: float
litesam_confidence: float
inlier_count: int
tracking_status: str
```
### SearchSession
```python
class SearchSession(BaseModel):
session_id: str
flight_id: str
frame_id: int
center_gps: GPSPoint
current_grid_size: int
max_grid_size: int
found: bool
exhausted: bool
```
### UserInputRequest
```python
class UserInputRequest(BaseModel):
request_id: str
flight_id: str
frame_id: int
uav_image: np.ndarray
candidate_tiles: List[TileCandidate]
message: str
created_at: datetime
```
### UserAnchor
```python
class UserAnchor(BaseModel):
uav_pixel: Tuple[float, float]
satellite_gps: GPSPoint
confidence: float = 1.0
```
docs/02_components/gps_denied_17_gps_denied_database_layer/gps_denied_database_layer_spec.md
-193
View File
@@ -1,193 +0,0 @@
# GPS-Denied Database Layer
## Interface Definition
**Interface Name**: `IGPSDeniedDatabase`
### Interface Methods
```python
class IGPSDeniedDatabase(ABC):
@abstractmethod
def save_flight_state(self, flight_state: FlightState) -> bool:
pass
@abstractmethod
def load_flight_state(self, flight_id: str) -> Optional[FlightState]:
pass
@abstractmethod
def query_processing_history(self, filters: Dict[str, Any]) -> List[FlightState]:
pass
@abstractmethod
def save_frame_result(self, flight_id: str, frame_result: FrameResult) -> bool:
pass
@abstractmethod
def get_frame_results(self, flight_id: str) -> List[FrameResult]:
pass
```
## Component Description
### Responsibilities
- Database access for GPS-Denied processing state
- Separate schema from Route API database
- Persist flight state, frame results
- Query processing history
- Support crash recovery
### Scope
- Flight state persistence
- Frame result storage
- Processing history queries
- Connection management
- Transaction handling
## API Methods
### `save_flight_state(flight_state: FlightState) -> bool`
**Description**: Saves flight processing state.
**Called By**: G02 Flight Manager
**Input**:
```python
FlightState:
flight_id: str
route_id: str
status: str
frames_processed: int
frames_total: int
current_heading: float
blocked: bool
...
```
**Output**: `bool` - True if saved
**Test Cases**:
1. Save state → persisted
2. Update state → overwrites
---
### `load_flight_state(flight_id: str) -> Optional[FlightState]`
**Description**: Loads flight state.
**Called By**: G02 Flight Manager (crash recovery)
**Output**: `Optional[FlightState]`
**Test Cases**:
1. Load existing → returns state
2. Load non-existent → returns None
---
### `query_processing_history(filters: Dict[str, Any]) -> List[FlightState]`
**Description**: Queries historical processing data.
**Called By**: Analytics, admin tools
**Test Cases**:
1. Query by date range → returns flights
2. Query by status → returns filtered
---
### `save_frame_result(flight_id: str, frame_result: FrameResult) -> bool`
**Description**: Saves frame processing result.
**Called By**: G13 Result Manager
**Test Cases**:
1. Save result → persisted
2. Update result (refinement) → overwrites
---
### `get_frame_results(flight_id: str) -> List[FrameResult]`
**Description**: Gets all frame results for flight.
**Called By**: G13 Result Manager
**Test Cases**:
1. Get results → returns all frames
2. No results → returns empty list
## Database Schema
```sql
-- Flights table
CREATE TABLE gps_denied_flights (
flight_id VARCHAR(36) PRIMARY KEY,
route_id VARCHAR(36) NOT NULL,
status VARCHAR(50),
frames_processed INT,
frames_total INT,
current_heading FLOAT,
blocked BOOLEAN,
created_at TIMESTAMP,
updated_at TIMESTAMP
);
-- Frame results table
CREATE TABLE frame_results (
id VARCHAR(36) PRIMARY KEY,
flight_id VARCHAR(36) NOT NULL,
frame_id INT NOT NULL,
gps_lat DECIMAL(10, 7),
gps_lon DECIMAL(11, 7),
altitude FLOAT,
heading FLOAT,
confidence FLOAT,
refined BOOLEAN,
timestamp TIMESTAMP,
updated_at TIMESTAMP,
FOREIGN KEY (flight_id) REFERENCES gps_denied_flights(flight_id) ON DELETE CASCADE,
UNIQUE KEY (flight_id, frame_id)
);
```
## Dependencies
### External Dependencies
- **PostgreSQL** or **MySQL**
- **SQLAlchemy** or **psycopg2**
## Data Models
### FlightState
```python
class FlightState(BaseModel):
flight_id: str
route_id: str
status: str
frames_processed: int
frames_total: int
current_heading: Optional[float]
blocked: bool
created_at: datetime
updated_at: datetime
```
### FrameResult
```python
class FrameResult(BaseModel):
frame_id: int
gps_center: GPSPoint
altitude: float
heading: float
confidence: float
refined: bool
timestamp: datetime
updated_at: datetime
```
docs/02_components/helpers/h08_batch_validator_spec.md
+5 -5
View File
@@ -35,7 +35,7 @@ class IBatchValidator(ABC):
- Support strict sequential ordering (ADxxxxxx.jpg)
### Scope
- Batch validation for G05 Image Input Pipeline
- Batch validation for F05 Image Input Pipeline
- Image format validation
- Filename pattern matching
- Sequence gap detection
@@ -47,7 +47,7 @@ class IBatchValidator(ABC):
**Description**: Validates batch contains 10-50 images.
**Called By**:
- G05 Image Input Pipeline (before queuing)
- F05 Image Input Pipeline (before queuing)
**Input**:
```python
@@ -86,7 +86,7 @@ ValidationResult:
**Description**: Validates images form consecutive sequence with no gaps.
**Called By**:
- G05 Image Input Pipeline (before queuing)
- F05 Image Input Pipeline (before queuing)
**Input**:
```python
@@ -135,7 +135,7 @@ return valid()
**Called By**:
- Internal (during check_sequence_continuity)
- G05 Image Input Pipeline
- F05 Image Input Pipeline
**Input**:
```python
@@ -174,7 +174,7 @@ ValidationResult:
**Description**: Validates image file format and properties.
**Called By**:
- G05 Image Input Pipeline (per-image validation)
- F05 Image Input Pipeline (per-image validation)
**Input**:
```python
docs/02_components/route_01_route_rest_api/route_rest_api_spec.md
-289
View File
@@ -1,289 +0,0 @@
# Route REST API
## Interface Definition
**Interface Name**: `IRouteRestAPI`
### Interface Methods
```python
class IRouteRestAPI(ABC):
@abstractmethod
def create_route(self, route_data: RouteCreateRequest) -> RouteResponse:
pass
@abstractmethod
def get_route(self, route_id: str) -> RouteResponse:
pass
@abstractmethod
def update_waypoints(self, route_id: str, waypoints: List[Waypoint]) -> UpdateResponse:
pass
@abstractmethod
def delete_route(self, route_id: str) -> DeleteResponse:
pass
```
## Component Description
### Responsibilities
- Expose REST API endpoints for route lifecycle management
- Handle HTTP request validation and routing
- Coordinate with Route Data Manager for persistence operations
- Validate incoming requests through Waypoint Validator
- Return appropriate HTTP responses with proper status codes
### Scope
- CRUD operations for routes
- Waypoint management within routes
- Geofence management
- Route metadata retrieval
- Used by both GPS-Denied system and Mission Planner
## API Methods
### `create_route(route_data: RouteCreateRequest) -> RouteResponse`
**Description**: Creates a new route with initial waypoints and geofences.
**Called By**:
- Client applications (GPS-Denied UI, Mission Planner UI)
**Input**:
```python
RouteCreateRequest:
id: Optional[str] # UUID, generated if not provided
name: str
description: str
points: List[GPSPoint] # Initial rough waypoints
geofences: Geofences
```
**Output**:
```python
RouteResponse:
route_id: str
created: bool
timestamp: datetime
```
**Error Conditions**:
- `400 Bad Request`: Invalid input data (missing required fields, invalid GPS coordinates)
- `409 Conflict`: Route with same ID already exists
- `500 Internal Server Error`: Database or internal error
**Test Cases**:
1. **Valid route creation**: Provide valid route data → returns 201 with routeId
2. **Missing required field**: Omit name → returns 400 with error message
3. **Invalid GPS coordinates**: Provide lat > 90 → returns 400
4. **Duplicate route ID**: Create route with existing ID → returns 409
---
### `get_route(route_id: str) -> RouteResponse`
**Description**: Retrieves complete route information including all waypoints and geofences.
**Called By**:
- Client applications
- G03 Route API Client (from GPS-Denied system)
**Input**:
```python
route_id: str # UUID
```
**Output**:
```python
RouteResponse:
route_id: str
name: str
description: str
points: List[Waypoint] # All waypoints with metadata
geofences: Geofences
created_at: datetime
updated_at: datetime
```
**Error Conditions**:
- `404 Not Found`: Route ID does not exist
- `500 Internal Server Error`: Database error
**Test Cases**:
1. **Existing route**: Valid routeId → returns 200 with complete route data
2. **Non-existent route**: Invalid routeId → returns 404
3. **Route with many waypoints**: Route with 2000+ waypoints → returns 200 with all data
---
### `update_waypoints(route_id: str, waypoint_id: str, waypoint_data: Waypoint) -> UpdateResponse`
**Description**: Updates a specific waypoint within a route. Used for per-frame GPS refinement from GPS-Denied system.
**Called By**:
- G03 Route API Client (per-frame updates)
- Client applications (manual corrections)
**Input**:
```python
route_id: str
waypoint_id: str # Frame sequence number or waypoint ID
waypoint_data: Waypoint:
lat: float
lon: float
altitude: Optional[float]
confidence: float
timestamp: datetime
refined: bool # True if updated by GPS-Denied refinement
```
**Output**:
```python
UpdateResponse:
updated: bool
waypoint_id: str
```
**Error Conditions**:
- `404 Not Found`: Route or waypoint not found
- `400 Bad Request`: Invalid waypoint data
- `500 Internal Server Error`: Database error
**Test Cases**:
1. **Update existing waypoint**: Valid data → returns 200
2. **Refinement update**: GPS-Denied sends refined coordinates → updates successfully
3. **Invalid coordinates**: lat > 90 → returns 400
4. **Non-existent waypoint**: Invalid waypoint_id → returns 404
---
### `delete_route(route_id: str) -> DeleteResponse`
**Description**: Deletes a route and all associated waypoints.
**Called By**:
- Client applications
**Input**:
```python
route_id: str
```
**Output**:
```python
DeleteResponse:
deleted: bool
route_id: str
```
**Error Conditions**:
- `404 Not Found`: Route does not exist
- `500 Internal Server Error`: Database error
**Test Cases**:
1. **Delete existing route**: Valid routeId → returns 200
2. **Delete non-existent route**: Invalid routeId → returns 404
3. **Delete route with active flight**: Route linked to processing flight → returns 200 (cascade handling in DB)
## Integration Tests
### Test 1: Route Creation and Retrieval Flow
1. POST `/routes` with valid data
2. Verify 201 response with routeId
3. GET `/routes/{routeId}`
4. Verify returned data matches created data
### Test 2: GPS-Denied Integration Flow
1. Create route via POST
2. Simulate GPS-Denied per-frame updates via PUT `/routes/{routeId}/waypoints/{waypointId}` × 100
3. GET route and verify all waypoints updated
4. Verify `refined: true` flag set
### Test 3: Concurrent Waypoint Updates
1. Create route
2. Send 50 concurrent PUT requests to different waypoints
3. Verify all updates succeed
4. Verify data consistency
## Non-Functional Requirements
### Performance
- **Create route**: < 500ms response time
- **Get route**: < 200ms for routes with < 2000 waypoints
- **Update waypoint**: < 100ms (critical for GPS-Denied real-time updates)
- **Delete route**: < 300ms
- **Throughput**: Handle 100 concurrent waypoint updates per second
### Scalability
- Support 1000+ concurrent route processing sessions
- Handle routes with up to 3000 waypoints
### Availability
- 99.9% uptime SLA
- Graceful degradation under load
### Security
- Input validation on all endpoints
- SQL injection prevention
- Rate limiting: 1000 requests/minute per client
## Dependencies
### Internal Components
- **R02 Route Data Manager**: For all data persistence operations
- **R03 Waypoint Validator**: For input validation
- **R04 Route Database Layer**: Indirectly through Data Manager
### External Dependencies
- **FastAPI/Flask**: Web framework
- **Pydantic**: Request/response validation
- **Uvicorn**: ASGI server
## Data Models
### RouteCreateRequest
```python
class GPSPoint(BaseModel):
lat: float # Latitude -90 to 90
lon: float # Longitude -180 to 180
class Polygon(BaseModel):
north_west: GPSPoint
south_east: GPSPoint
class Geofences(BaseModel):
polygons: List[Polygon]
class RouteCreateRequest(BaseModel):
id: Optional[str] = None # UUID
name: str
description: str
points: List[GPSPoint] # Initial rough waypoints
geofences: Geofences
```
### Waypoint
```python
class Waypoint(BaseModel):
id: str # Sequence number or UUID
lat: float
lon: float
altitude: Optional[float] = None
confidence: float # 0.0 to 1.0
timestamp: datetime
refined: bool = False
```
### RouteResponse
```python
class RouteResponse(BaseModel):
route_id: str
name: str
description: str
points: List[Waypoint]
geofences: Geofences
created_at: datetime
updated_at: datetime
```
docs/02_components/route_02_route_data_manager/route_data_manager_spec.md
-338
View File
@@ -1,338 +0,0 @@
# Route Data Manager
## Interface Definition
**Interface Name**: `IRouteDataManager`
### Interface Methods
```python
class IRouteDataManager(ABC):
@abstractmethod
def save_route(self, route: Route) -> str:
pass
@abstractmethod
def load_route(self, route_id: str) -> Optional[Route]:
pass
@abstractmethod
def update_waypoint(self, route_id: str, waypoint_id: str, waypoint: Waypoint) -> bool:
pass
@abstractmethod
def delete_waypoint(self, route_id: str, waypoint_id: str) -> bool:
pass
@abstractmethod
def get_route_metadata(self, route_id: str) -> Optional[RouteMetadata]:
pass
@abstractmethod
def delete_route(self, route_id: str) -> bool:
pass
```
## Component Description
### Responsibilities
- Manage route persistence and retrieval
- Coordinate with Route Database Layer for data operations
- Handle waypoint CRUD operations within routes
- Manage route metadata (timestamps, statistics)
- Ensure data consistency and transaction management
### Scope
- Business logic layer between REST API and Database Layer
- Route lifecycle management
- Waypoint batch operations
- Query optimization for large route datasets
- Caching layer for frequently accessed routes (optional)
## API Methods
### `save_route(route: Route) -> str`
**Description**: Persists a new route with initial waypoints and geofences.
**Called By**:
- R01 Route REST API
**Input**:
```python
Route:
id: Optional[str] # Generated if not provided
name: str
description: str
points: List[Waypoint]
geofences: Geofences
```
**Output**:
```python
route_id: str # UUID of saved route
```
**Error Conditions**:
- `DuplicateRouteError`: Route with same ID exists
- `ValidationError`: Invalid route data
- `DatabaseError`: Database connection or constraint violation
**Test Cases**:
1. **New route**: Valid route → returns routeId, verifies in DB
2. **Route with 1000 waypoints**: Large route → saves successfully
3. **Duplicate ID**: Existing route ID → raises DuplicateRouteError
4. **Transaction rollback**: DB error mid-save → no partial data persisted
---
### `load_route(route_id: str) -> Optional[Route]`
**Description**: Retrieves complete route data including all waypoints.
**Called By**:
- R01 Route REST API
- R03 Waypoint Validator (for context validation)
**Input**:
```python
route_id: str
```
**Output**:
```python
Route or None if not found
```
**Error Conditions**:
- `DatabaseError`: Database connection error
- Returns `None`: Route not found (not an error condition)
**Test Cases**:
1. **Existing route**: Valid ID → returns complete Route object
2. **Non-existent route**: Invalid ID → returns None
3. **Large route**: 3000 waypoints → returns all data efficiently
4. **Concurrent reads**: Multiple simultaneous loads → all succeed
---
### `update_waypoint(route_id: str, waypoint_id: str, waypoint: Waypoint) -> bool`
**Description**: Updates a single waypoint within a route. Optimized for high-frequency GPS-Denied updates.
**Called By**:
- R01 Route REST API
**Input**:
```python
route_id: str
waypoint_id: str
waypoint: Waypoint
```
**Output**:
```python
bool: True if updated, False if route/waypoint not found
```
**Error Conditions**:
- `ValidationError`: Invalid waypoint data
- `DatabaseError`: Database error
**Test Cases**:
1. **Update existing waypoint**: Valid data → returns True
2. **Non-existent waypoint**: Invalid waypoint_id → returns False
3. **Concurrent updates**: 100 simultaneous updates to different waypoints → all succeed
4. **Update timestamp**: Automatically updates route.updated_at
---
### `delete_waypoint(route_id: str, waypoint_id: str) -> bool`
**Description**: Deletes a specific waypoint from a route.
**Called By**:
- R01 Route REST API (rare, for manual corrections)
**Input**:
```python
route_id: str
waypoint_id: str
```
**Output**:
```python
bool: True if deleted, False if not found
```
**Error Conditions**:
- `DatabaseError`: Database error
**Test Cases**:
1. **Delete existing waypoint**: Valid IDs → returns True
2. **Delete non-existent waypoint**: Invalid ID → returns False
3. **Delete all waypoints**: Delete all waypoints one by one → succeeds
---
### `get_route_metadata(route_id: str) -> Optional[RouteMetadata]`
**Description**: Retrieves route metadata without loading all waypoints (lightweight operation).
**Called By**:
- R01 Route REST API
- Client applications (route listing)
**Input**:
```python
route_id: str
```
**Output**:
```python
RouteMetadata:
route_id: str
name: str
description: str
waypoint_count: int
created_at: datetime
updated_at: datetime
```
**Error Conditions**:
- Returns `None`: Route not found
**Test Cases**:
1. **Get metadata**: Valid ID → returns metadata without waypoints
2. **Performance**: Metadata retrieval < 50ms even for 3000-waypoint route
---
### `delete_route(route_id: str) -> bool`
**Description**: Deletes a route and all associated waypoints.
**Called By**:
- R01 Route REST API
**Input**:
```python
route_id: str
```
**Output**:
```python
bool: True if deleted, False if not found
```
**Error Conditions**:
- `DatabaseError`: Database error
**Test Cases**:
1. **Delete route**: Valid ID → deletes route and all waypoints
2. **Cascade delete**: Verify all waypoints deleted
3. **Non-existent route**: Invalid ID → returns False
## Integration Tests
### Test 1: Complete Route Lifecycle
1. save_route() with 100 waypoints
2. load_route() and verify all data
3. update_waypoint() for 50 waypoints
4. delete_waypoint() for 10 waypoints
5. get_route_metadata() and verify count
6. delete_route() and verify removal
### Test 2: High-Frequency Update Simulation (GPS-Denied Pattern)
1. save_route() with 2000 waypoints
2. Simulate per-frame updates: update_waypoint() × 2000 in sequence
3. Verify all updates persisted correctly
4. Measure total time < 200 seconds (100ms per update)
### Test 3: Concurrent Route Operations
1. Create 10 routes concurrently
2. Update different waypoints in parallel (100 concurrent updates)
3. Delete 5 routes concurrently while updating others
4. Verify data consistency
## Non-Functional Requirements
### Performance
- **save_route**: < 300ms for routes with 100 waypoints
- **load_route**: < 150ms for routes with 2000 waypoints
- **update_waypoint**: < 50ms (critical path for GPS-Denied)
- **get_route_metadata**: < 30ms
- **delete_route**: < 200ms
### Scalability
- Support 1000+ concurrent route operations
- Handle routes with up to 3000 waypoints efficiently
- Optimize for read-heavy workload (90% reads, 10% writes)
### Reliability
- ACID transaction guarantees
- Automatic retry on transient database errors (3 attempts)
- Data validation before persistence
### Maintainability
- Clear separation from database implementation
- Support for future caching layer integration
- Comprehensive error handling and logging
## Dependencies
### Internal Components
- **R03 Waypoint Validator**: Validates waypoints before persistence
- **R04 Route Database Layer**: For all database operations
### External Dependencies
- None (pure business logic layer)
## Data Models
### Route
```python
class Route(BaseModel):
id: str # UUID
name: str
description: str
points: List[Waypoint]
geofences: Geofences
created_at: datetime
updated_at: datetime
```
### RouteMetadata
```python
class RouteMetadata(BaseModel):
route_id: str
name: str
description: str
waypoint_count: int
geofence_count: int
created_at: datetime
updated_at: datetime
```
### Waypoint
```python
class Waypoint(BaseModel):
id: str
lat: float
lon: float
altitude: Optional[float]
confidence: float
timestamp: datetime
refined: bool
```
### Geofences
```python
class Polygon(BaseModel):
north_west: GPSPoint
south_east: GPSPoint
class Geofences(BaseModel):
polygons: List[Polygon]
```
docs/02_components/route_03_waypoint_validator/waypoint_validator_spec.md
-294
View File
@@ -1,294 +0,0 @@
# Waypoint Validator
## Interface Definition
**Interface Name**: `IWaypointValidator`
### Interface Methods
```python
class IWaypointValidator(ABC):
@abstractmethod
def validate_waypoint(self, waypoint: Waypoint) -> ValidationResult:
pass
@abstractmethod
def validate_geofence(self, geofence: Geofences) -> ValidationResult:
pass
@abstractmethod
def check_bounds(self, waypoint: Waypoint, geofences: Geofences) -> bool:
pass
@abstractmethod
def validate_route_continuity(self, waypoints: List[Waypoint]) -> ValidationResult:
pass
```
## Component Description
### Responsibilities
- Validate individual waypoint data (GPS coordinates, altitude, confidence)
- Validate geofence definitions (polygon bounds, topology)
- Check waypoints against geofence boundaries
- Validate route continuity (detect large gaps, validate sequencing)
- Provide detailed validation error messages
### Scope
- Input validation for Route API
- Business rule enforcement (operational area restrictions for Ukraine)
- Geospatial boundary checking
- Data quality assurance
## API Methods
### `validate_waypoint(waypoint: Waypoint) -> ValidationResult`
**Description**: Validates a single waypoint's data integrity and constraints.
**Called By**:
- R01 Route REST API (before creating/updating)
- R02 Route Data Manager (pre-persistence validation)
**Input**:
```python
Waypoint:
id: str
lat: float
lon: float
altitude: Optional[float]
confidence: float
timestamp: datetime
refined: bool
```
**Output**:
```python
ValidationResult:
valid: bool
errors: List[str]
```
**Validation Rules**:
1. **Latitude**: -90.0 <= lat <= 90.0
2. **Longitude**: -180.0 <= lon <= 180.0
3. **Altitude**: 0 <= altitude <= 1000 meters (if provided)
4. **Confidence**: 0.0 <= confidence <= 1.0
5. **Timestamp**: Not in future, not older than 1 year
6. **Operational area** (Ukraine restriction): Latitude ~45-52N, Longitude ~22-40E
7. **ID**: Non-empty string
**Error Conditions**:
- Returns `ValidationResult` with `valid=False` and error list (not an exception)
**Test Cases**:
1. **Valid waypoint**: All fields correct → returns `valid=True`
2. **Invalid latitude**: lat=100 → returns `valid=False`, error="Latitude out of range"
3. **Invalid longitude**: lon=200 → returns `valid=False`
4. **Invalid confidence**: confidence=1.5 → returns `valid=False`
5. **Future timestamp**: timestamp=tomorrow → returns `valid=False`
6. **Outside operational area**: lat=10 (not Ukraine) → returns `valid=False`
7. **Valid altitude**: altitude=500 → returns `valid=True`
8. **Invalid altitude**: altitude=1500 → returns `valid=False`
---
### `validate_geofence(geofence: Geofences) -> ValidationResult`
**Description**: Validates geofence polygon definitions.
**Called By**:
- R01 Route REST API (during route creation)
**Input**:
```python
Geofences:
polygons: List[Polygon]
Polygon:
north_west: GPSPoint
south_east: GPSPoint
```
**Output**:
```python
ValidationResult:
valid: bool
errors: List[str]
```
**Validation Rules**:
1. **North-West corner**: NW.lat > SE.lat
2. **North-West corner**: NW.lon < SE.lon (for Eastern Ukraine)
3. **Polygon size**: Max 500km × 500km
4. **Polygon count**: 1 <= len(polygons) <= 10
5. **No self-intersection**: Polygons should not overlap
6. **Within operational area**: All corners within Ukraine bounds
**Error Conditions**:
- Returns `ValidationResult` with validation errors
**Test Cases**:
1. **Valid geofence**: Single polygon in Ukraine → valid=True
2. **Invalid corners**: NW.lat < SE.lat → valid=False
3. **Too large**: 600km × 600km → valid=False
4. **Too many polygons**: 15 polygons → valid=False
5. **Overlapping polygons**: Two overlapping → valid=False (warning)
---
### `check_bounds(waypoint: Waypoint, geofences: Geofences) -> bool`
**Description**: Checks if a waypoint falls within geofence boundaries.
**Called By**:
- R01 Route REST API (optional check during waypoint updates)
- R02 Route Data Manager (business rule enforcement)
**Input**:
```python
waypoint: Waypoint
geofences: Geofences
```
**Output**:
```python
bool: True if waypoint is within any geofence polygon
```
**Algorithm**:
- Point-in-polygon test for each geofence polygon
- Returns True if point is inside at least one polygon
**Error Conditions**:
- None (returns False if outside all geofences)
**Test Cases**:
1. **Inside geofence**: Waypoint in polygon center → returns True
2. **Outside geofence**: Waypoint 10km outside → returns False
3. **On boundary**: Waypoint on polygon edge → returns True
4. **Multiple geofences**: Waypoint in second polygon → returns True
---
### `validate_route_continuity(waypoints: List[Waypoint]) -> ValidationResult`
**Description**: Validates route continuity, detecting large gaps and sequence issues.
**Called By**:
- R01 Route REST API (during route creation)
- R02 Route Data Manager (route quality check)
**Input**:
```python
waypoints: List[Waypoint] # Should be ordered by sequence/timestamp
```
**Output**:
```python
ValidationResult:
valid: bool
errors: List[str]
warnings: List[str]
```
**Validation Rules**:
1. **Minimum waypoints**: len(waypoints) >= 2
2. **Maximum waypoints**: len(waypoints) <= 3000
3. **Timestamp ordering**: waypoints[i].timestamp < waypoints[i+1].timestamp
4. **Distance gaps**: Consecutive waypoints < 500 meters apart (warning if violated)
5. **Large gap detection**: Flag gaps > 1km (warning for potential data loss)
6. **No duplicate timestamps**: All timestamps unique
**Error Conditions**:
- Returns `ValidationResult` with errors and warnings
**Test Cases**:
1. **Valid route**: 100 waypoints, 100m spacing → valid=True
2. **Too few waypoints**: 1 waypoint → valid=False
3. **Too many waypoints**: 3500 waypoints → valid=False
4. **Unordered timestamps**: waypoints out of order → valid=False
5. **Large gap**: 2km gap between waypoints → valid=True with warning
6. **Duplicate timestamps**: Two waypoints same time → valid=False
## Integration Tests
### Test 1: Complete Validation Pipeline
1. Create waypoint with all valid data
2. validate_waypoint() → passes
3. Create geofence for Eastern Ukraine
4. validate_geofence() → passes
5. check_bounds() → waypoint inside geofence
### Test 2: Route Validation Flow
1. Create 500 waypoints with 100m spacing
2. validate_route_continuity() → passes
3. Add waypoint 2km away
4. validate_route_continuity() → passes with warning
5. Add waypoint with past timestamp
6. validate_route_continuity() → fails
### Test 3: Edge Cases
1. Waypoint on geofence boundary
2. Waypoint at North Pole (lat=90)
3. Waypoint at dateline (lon=180)
4. Route with exactly 3000 waypoints
## Non-Functional Requirements
### Performance
- **validate_waypoint**: < 1ms per waypoint
- **validate_geofence**: < 10ms per geofence
- **check_bounds**: < 2ms per check
- **validate_route_continuity**: < 100ms for 2000 waypoints
### Accuracy
- GPS coordinate validation: 6 decimal places precision (0.1m)
- Geofence boundary check: 1-meter precision
### Maintainability
- Validation rules configurable via configuration file
- Easy to add new validation rules
- Clear error messages for debugging
## Dependencies
### Internal Components
- **R04 Route Database Layer**: For loading existing route data (optional context)
- **H06 Web Mercator Utils**: For distance calculations (optional)
### External Dependencies
- **Shapely** (optional): For advanced polygon operations
- **Geopy**: For geodesic distance calculations
## Data Models
### ValidationResult
```python
class ValidationResult(BaseModel):
valid: bool
errors: List[str] = []
warnings: List[str] = []
```
### OperationalArea (Configuration)
```python
class OperationalArea(BaseModel):
name: str = "Eastern Ukraine"
min_lat: float = 45.0
max_lat: float = 52.0
min_lon: float = 22.0
max_lon: float = 40.0
```
### ValidationRules (Configuration)
```python
class ValidationRules(BaseModel):
max_altitude: float = 1000.0 # meters
max_waypoint_gap: float = 500.0 # meters
max_route_waypoints: int = 3000
min_route_waypoints: int = 2
max_geofence_size: float = 500000.0 # meters (500km)
max_geofences: int = 10
```
docs/02_components/route_04_route_database_layer/route_database_layer_spec.md
-475
View File
@@ -1,475 +0,0 @@
# Route Database Layer
## Interface Definition
**Interface Name**: `IRouteDatabase`
### Interface Methods
```python
class IRouteDatabase(ABC):
@abstractmethod
def insert_route(self, route: Route) -> str:
pass
@abstractmethod
def update_route(self, route: Route) -> bool:
pass
@abstractmethod
def query_routes(self, filters: Dict[str, Any], limit: int, offset: int) -> List[Route]:
pass
@abstractmethod
def get_route_by_id(self, route_id: str) -> Optional[Route]:
pass
@abstractmethod
def get_waypoints(self, route_id: str, limit: Optional[int] = None) -> List[Waypoint]:
pass
@abstractmethod
def insert_waypoint(self, route_id: str, waypoint: Waypoint) -> str:
pass
@abstractmethod
def update_waypoint(self, route_id: str, waypoint_id: str, waypoint: Waypoint) -> bool:
pass
@abstractmethod
def delete_route(self, route_id: str) -> bool:
pass
```
## Component Description
### Responsibilities
- Direct database access layer for route data
- Execute SQL queries and commands
- Manage database connections and transactions
- Handle connection pooling and retry logic
- Provide database abstraction for potential migration (PostgreSQL, MySQL, etc.)
### Scope
- CRUD operations on routes table
- CRUD operations on waypoints table
- CRUD operations on geofences table
- Query optimization for large datasets
- Database schema management
- Separate schema from GPS-Denied API database
## API Methods
### `insert_route(route: Route) -> str`
**Description**: Inserts a new route with initial waypoints and geofences into the database.
**Called By**:
- R02 Route Data Manager
**Input**:
```python
Route:
id: str
name: str
description: str
points: List[Waypoint]
geofences: Geofences
created_at: datetime
updated_at: datetime
```
**Output**:
```python
route_id: str # Inserted route ID
```
**Database Operations**:
1. Begin transaction
2. INSERT INTO routes (id, name, description, created_at, updated_at)
3. INSERT INTO waypoints (route_id, ...) for each waypoint
4. INSERT INTO geofences (route_id, ...) for each polygon
5. Commit transaction
**Error Conditions**:
- `IntegrityError`: Duplicate route_id (unique constraint violation)
- `DatabaseError`: Connection error, transaction failure
- Automatic rollback on any error
**Test Cases**:
1. **Insert route with 100 waypoints**: Successful insertion, all waypoints persisted
2. **Duplicate route_id**: Raises IntegrityError
3. **Transaction rollback**: Error on waypoint insertion → route also rolled back
4. **Connection loss**: Mid-transaction error → graceful rollback
---
### `update_route(route: Route) -> bool`
**Description**: Updates route metadata (name, description, updated_at).
**Called By**:
- R02 Route Data Manager
**Input**:
```python
Route with updated fields
```
**Output**:
```python
bool: True if updated, False if route not found
```
**Database Operations**:
```sql
UPDATE routes
SET name = ?, description = ?, updated_at = ?
WHERE id = ?
```
**Error Conditions**:
- `DatabaseError`: Connection or query error
**Test Cases**:
1. **Update existing route**: Returns True
2. **Update non-existent route**: Returns False
3. **Update with same data**: Succeeds, updates timestamp
---
### `query_routes(filters: Dict[str, Any], limit: int, offset: int) -> List[Route]`
**Description**: Queries routes with filtering, pagination for route listing.
**Called By**:
- R02 Route Data Manager
- R01 Route REST API (list endpoints)
**Input**:
```python
filters: Dict[str, Any] # e.g., {"name": "Mission%", "created_after": datetime}
limit: int # Max results
offset: int # For pagination
```
**Output**:
```python
List[Route] # Routes without full waypoint data (metadata only)
```
**Database Operations**:
```sql
SELECT * FROM routes
WHERE name LIKE ? AND created_at > ?
ORDER BY created_at DESC
LIMIT ? OFFSET ?
```
**Error Conditions**:
- `DatabaseError`: Query error
**Test Cases**:
1. **Filter by name**: Returns matching routes
2. **Pagination**: offset=100, limit=50 → returns routes 100-149
3. **Empty result**: No matches → returns []
4. **No filters**: Returns all routes (with limit)
---
### `get_route_by_id(route_id: str) -> Optional[Route]`
**Description**: Retrieves complete route with all waypoints by ID.
**Called By**:
- R02 Route Data Manager
**Input**:
```python
route_id: str
```
**Output**:
```python
Optional[Route] # Complete route with all waypoints, or None
```
**Database Operations**:
1. SELECT FROM routes WHERE id = ?
2. SELECT FROM waypoints WHERE route_id = ? ORDER BY timestamp
3. SELECT FROM geofences WHERE route_id = ?
4. Assemble Route object
**Error Conditions**:
- `DatabaseError`: Query error
- Returns None if route not found
**Test Cases**:
1. **Existing route**: Returns complete Route object
2. **Non-existent route**: Returns None
3. **Large route (3000 waypoints)**: Returns all data within 150ms
4. **Route with no waypoints**: Returns route with empty points list
---
### `get_waypoints(route_id: str, limit: Optional[int] = None) -> List[Waypoint]`
**Description**: Retrieves waypoints for a route, optionally limited.
**Called By**:
- R02 Route Data Manager
**Input**:
```python
route_id: str
limit: Optional[int] # For pagination or preview
```
**Output**:
```python
List[Waypoint]
```
**Database Operations**:
```sql
SELECT * FROM waypoints
WHERE route_id = ?
ORDER BY timestamp ASC
LIMIT ? -- if limit provided
```
**Error Conditions**:
- `DatabaseError`: Query error
**Test Cases**:
1. **All waypoints**: limit=None → returns all
2. **Limited waypoints**: limit=100 → returns first 100
3. **No waypoints**: Empty list
---
### `insert_waypoint(route_id: str, waypoint: Waypoint) -> str`
**Description**: Inserts a new waypoint into a route.
**Called By**:
- R02 Route Data Manager
**Input**:
```python
route_id: str
waypoint: Waypoint
```
**Output**:
```python
waypoint_id: str
```
**Database Operations**:
```sql
INSERT INTO waypoints (id, route_id, lat, lon, altitude, confidence, timestamp, refined)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
```
**Error Conditions**:
- `ForeignKeyError`: route_id doesn't exist
- `IntegrityError`: Duplicate waypoint_id
**Test Cases**:
1. **Valid insertion**: Returns waypoint_id
2. **Non-existent route**: Raises ForeignKeyError
---
### `update_waypoint(route_id: str, waypoint_id: str, waypoint: Waypoint) -> bool`
**Description**: Updates a waypoint. Critical path for GPS-Denied per-frame updates.
**Called By**:
- R02 Route Data Manager
**Input**:
```python
route_id: str
waypoint_id: str
waypoint: Waypoint
```
**Output**:
```python
bool: True if updated, False if not found
```
**Database Operations**:
```sql
UPDATE waypoints
SET lat = ?, lon = ?, altitude = ?, confidence = ?, refined = ?
WHERE id = ? AND route_id = ?
```
**Optimization**:
- Prepared statement caching
- Connection pooling for high throughput
- Indexed on (route_id, id) for fast lookups
**Error Conditions**:
- `DatabaseError`: Query error
**Test Cases**:
1. **Update existing waypoint**: Returns True, updates data
2. **Non-existent waypoint**: Returns False
3. **High-frequency updates**: 100 updates/sec sustained for 20 seconds
---
### `delete_route(route_id: str) -> bool`
**Description**: Deletes a route and cascades to waypoints and geofences.
**Called By**:
- R02 Route Data Manager
**Input**:
```python
route_id: str
```
**Output**:
```python
bool: True if deleted, False if not found
```
**Database Operations**:
```sql
DELETE FROM routes WHERE id = ?
-- Cascade deletes from waypoints and geofences via FK constraints
```
**Error Conditions**:
- `DatabaseError`: Query error
**Test Cases**:
1. **Delete route with waypoints**: Deletes route and all waypoints
2. **Verify cascade**: Check waypoints table empty for route_id
3. **Non-existent route**: Returns False
## Integration Tests
### Test 1: Complete Database Lifecycle
1. insert_route() with 500 waypoints
2. get_route_by_id() and verify all data
3. update_waypoint() × 100
4. query_routes() with filters
5. delete_route() and verify removal
### Test 2: High-Frequency Update Pattern (GPS-Denied Simulation)
1. insert_route() with 2000 waypoints
2. update_waypoint() × 2000 sequentially
3. Measure total time and throughput
4. Verify all updates persisted correctly
### Test 3: Concurrent Access
1. Insert 10 routes concurrently
2. Update waypoints in parallel (100 concurrent connections)
3. Query routes while updates occurring
4. Verify no deadlocks or data corruption
### Test 4: Transaction Integrity
1. Begin insert_route() transaction
2. Simulate error mid-waypoint insertion
3. Verify complete rollback (no partial data)
## Non-Functional Requirements
### Performance
- **insert_route**: < 200ms for 100 waypoints
- **update_waypoint**: < 30ms (critical path)
- **get_route_by_id**: < 100ms for 2000 waypoints
- **query_routes**: < 150ms for pagination queries
- **Throughput**: 200+ waypoint updates per second
### Scalability
- Connection pool: 50-100 connections
- Support 1000+ concurrent operations
- Handle tables with millions of waypoints
### Reliability
- ACID transaction guarantees
- Automatic retry on transient errors (3 attempts with exponential backoff)
- Connection health checks
- Graceful degradation on connection pool exhaustion
### Security
- SQL injection prevention (parameterized queries only)
- Principle of least privilege (database user permissions)
- Connection string encryption
## Dependencies
### Internal Components
- None (lowest layer)
### External Dependencies
- **PostgreSQL** or **MySQL**: Relational database
- **SQLAlchemy** or **psycopg2**: Database driver
- **Alembic**: Schema migration tool
## Data Models
### Database Schema
```sql
-- Routes table
CREATE TABLE routes (
id VARCHAR(36) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
description TEXT,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
INDEX idx_created_at (created_at),
INDEX idx_name (name)
);
-- Waypoints table
CREATE TABLE waypoints (
id VARCHAR(36) PRIMARY KEY,
route_id VARCHAR(36) NOT NULL,
lat DECIMAL(10, 7) NOT NULL,
lon DECIMAL(11, 7) NOT NULL,
altitude DECIMAL(7, 2),
confidence DECIMAL(3, 2) NOT NULL,
timestamp TIMESTAMP NOT NULL,
refined BOOLEAN NOT NULL DEFAULT FALSE,
FOREIGN KEY (route_id) REFERENCES routes(id) ON DELETE CASCADE,
INDEX idx_route_timestamp (route_id, timestamp),
INDEX idx_route_id (route_id, id)
);
-- Geofences table
CREATE TABLE geofences (
id VARCHAR(36) PRIMARY KEY,
route_id VARCHAR(36) NOT NULL,
nw_lat DECIMAL(10, 7) NOT NULL,
nw_lon DECIMAL(11, 7) NOT NULL,
se_lat DECIMAL(10, 7) NOT NULL,
se_lon DECIMAL(11, 7) NOT NULL,
FOREIGN KEY (route_id) REFERENCES routes(id) ON DELETE CASCADE,
INDEX idx_route_id (route_id)
);
```
### Connection Configuration
```python
class DatabaseConfig(BaseModel):
host: str
port: int
database: str
username: str
password: str
pool_size: int = 50
max_overflow: int = 50
pool_timeout: int = 30
pool_recycle: int = 3600
```
docs/03_tests/00_test_summary.md
+9 -5
View File
@@ -3,7 +3,7 @@
## Overview
Comprehensive test specifications for the GPS-denied navigation system following the QA testing pyramid approach.
**Total Test Specifications**: 54
**Total Test Specifications**: 56
## Test Organization
@@ -23,8 +23,8 @@ Tests individual system components in isolation with their dependencies.
- 08: Image Rotation Manager
**Service Infrastructure (09-12)**:
- 09: REST API
- 10: SSE Event Streamer
- 09: REST API (FastAPI endpoints for flight management, image uploads, user fixes)
- 10: SSE Event Streamer (Server-Sent Events for real-time result streaming)
- 11: Flight Manager
- 12: Result Manager
@@ -71,6 +71,10 @@ Tests mapped to 10 acceptance criteria.
- 42: Degraded satellite data
- 43: Complete system validation
**Chunk-Based Recovery (55-56)**:
- 55: Chunk rotation recovery (rotation sweeps for chunks)
- 56: Multi-chunk simultaneous processing (Atlas architecture)
### GPS-Analyzed Scenario Tests (51-54): Real Data
Tests using GPS-analyzed test datasets.
@@ -108,8 +112,8 @@ Tests using GPS-analyzed test datasets.
| AC-1 | 80% < 50m error | 31, 43, 51, 54 | ✓ Covered |
| AC-2 | 60% < 20m error | 32, 43, 51, 54 | ✓ Covered |
| AC-3 | 350m outlier robust | 33, 43, 52, 54 | ✓ Covered |
| AC-4 | Sharp turn <5% overlap | 34, 43, 53, 54 | ✓ Covered |
| AC-5 | Multi-fragment connection | 35, 43 | ✓ Covered |
| AC-4 | Sharp turn <5% overlap | 34, 43, 53, 54, 55 | ✓ Covered |
| AC-5 | Multi-fragment connection | 35, 39, 43, 56 | ✓ Covered |
| AC-6 | User input after 3 failures | 36, 43 | ✓ Covered |
| AC-7 | <5s per image | 37, 43, 51, 54 | ✓ Covered |
| AC-8 | Real-time + refinement | 38, 43 | ✓ Covered |
docs/03_tests/09_rest_api_integration_spec.md
+16 -15
View File
@@ -31,7 +31,7 @@ The API is the primary interface for external clients to interact with the GPS-d
## Input Data
### Test Case 1: Create Flight
- **Endpoint**: POST /api/v1/flights
- **Endpoint**: POST /flights
- **Payload**:
```json
{
@@ -48,60 +48,61 @@ The API is the primary interface for external clients to interact with the GPS-d
- **Expected**: 201 Created, returns flight_id
### Test Case 2: Upload Single Image
- **Endpoint**: POST /api/v1/flights/{flight_id}/images
- **Endpoint**: POST /flights/{flightId}/images
- **Payload**: Multipart form-data with AD000001.jpg
- **Headers**: Content-Type: multipart/form-data
- **Expected**: 202 Accepted, processing started
### Test Case 3: Upload Multiple Images
- **Endpoint**: POST /api/v1/flights/{flight_id}/images/batch
- **Endpoint**: POST /flights/{flightId}/images/batch
- **Payload**: AD000001-AD000010 (10 images)
- **Expected**: 202 Accepted, all images queued
### Test Case 4: Get Flight Status
- **Endpoint**: GET /api/v1/flights/{flight_id}/status
- **Endpoint**: GET /flights/{flightId}
- **Expected**: 200 OK, returns processing status and statistics
### Test Case 5: Get Results
- **Endpoint**: GET /api/v1/flights/{flight_id}/results
- **Endpoint**: GET /flights/{flightId}/results
- **Query Params**: ?format=json&include_refined=true
- **Expected**: 200 OK, returns GPS coordinates for all processed images
### Test Case 6: Get Specific Image Result
- **Endpoint**: GET /api/v1/flights/{flight_id}/images/{image_id}/result
- **Endpoint**: GET /flights/{flightId}/results?image_id={imageId}
- **Expected**: 200 OK, returns detailed result for one image
### Test Case 7: Submit User Fix (AC-6)
- **Endpoint**: POST /api/v1/flights/{flight_id}/images/{image_id}/user_fix
- **Endpoint**: POST /flights/{flightId}/user-fix
- **Payload**:
```json
{
"gps": {"lat": 48.270334, "lon": 37.374442},
"confidence": "high"
"frame_id": 15,
"uav_pixel": [3126, 2084],
"satellite_gps": {"lat": 48.270334, "lon": 37.374442}
}
```
- **Expected**: 200 OK, system incorporates user fix
- **Expected**: 200 OK, system incorporates user fix, processing resumes
### Test Case 8: List Flights
- **Endpoint**: GET /api/v1/flights
- **Endpoint**: GET /flights
- **Query Params**: ?status=active&limit=10
- **Expected**: 200 OK, returns list of flights
### Test Case 9: Delete Flight
- **Endpoint**: DELETE /api/v1/flights/{flight_id}
- **Endpoint**: DELETE /flights/{flightId}
- **Expected**: 204 No Content, flight and associated data deleted
### Test Case 10: Error Handling - Invalid Input
- **Endpoint**: POST /api/v1/flights
- **Endpoint**: POST /flights
- **Payload**: Invalid JSON or missing required fields
- **Expected**: 400 Bad Request with error details
### Test Case 11: Error Handling - Not Found
- **Endpoint**: GET /api/v1/flights/nonexistent_id
- **Endpoint**: GET /flights/nonexistent_id
- **Expected**: 404 Not Found
### Test Case 12: SSE Connection
- **Endpoint**: GET /api/v1/flights/{flight_id}/stream
- **Endpoint**: GET /flights/{flightId}/stream
- **Headers**: Accept: text/event-stream
- **Expected**: 200 OK, establishes SSE stream for real-time updates
docs/03_tests/10_sse_event_streamer_integration_spec.md
+1 -1
View File
@@ -213,7 +213,7 @@ For each test case, verify:
c. Prepare SSE test clients
2. **Test Case 1 - Single Client**:
a. Open SSE connection to /api/v1/flights/{flight_id}/stream
a. Open SSE connection to GET /flights/{flightId}/stream
b. Upload 10 images to flight
c. Collect all events
d. Verify event count and content
docs/03_tests/21_end_to_end_normal_flight_spec.md
+6 -6
View File
@@ -33,27 +33,27 @@ This is a comprehensive end-to-end test simulating real operational usage. Tests
- Prepare satellite tiles
2. **Create Flight (< 1s)**:
- POST /api/v1/flights
- POST /flights
- Payload: start_gps, altitude=400m, camera_params
- Verify: flight_id returned, state="created"
3. **Upload Images (< 30s)**:
- POST /api/v1/flights/{id}/images/batch
- POST /flights/{flightId}/images/batch
- Upload AD000001-AD000030
- Verify: All 30 queued, processing starts
4. **Monitor Processing via SSE (< 150s)**:
- Connect to /api/v1/flights/{id}/stream
- Connect to GET /flights/{flightId}/stream
- Receive "image_processed" events as each image completes
- Verify: Events received in < 1s of processing
5. **Await Completion (< 150s total)**:
- Monitor flight status
- Monitor flight status via GET /flights/{flightId}
- Wait for state="completed"
- Verify: No failures, all 30 images processed
6. **Retrieve Results (< 2s)**:
- GET /api/v1/flights/{id}/results?format=json
- GET /flights/{flightId}/results?format=json
- Download GPS coordinates for all images
- Compare with ground truth
@@ -64,7 +64,7 @@ This is a comprehensive end-to-end test simulating real operational usage. Tests
- Check: Mean error < 30m
8. **Export Results (< 5s)**:
- GET /api/v1/flights/{id}/results?format=csv
- GET /flights/{flightId}/results?format=csv
- Verify CSV matches coordinates.csv format
## Expected Results
docs/03_tests/31_accuracy_50m_baseline_spec.md
+1 -1
View File
@@ -38,7 +38,7 @@ Process baseline flight of 30 images with normal spacing (~120m between images).
- Registration rate > 95%
### Step 5: Retrieve Results
- **Action**: GET /api/v1/flights/{id}/results
- **Action**: GET /flights/{flightId}/results
- **Expected Result**: Results for all 30 images returned
### Step 6: Calculate Errors
docs/03_tests/33_accuracy_20m_high_precision_spec.md
+1 -1
View File
@@ -38,7 +38,7 @@ Process same baseline flight as AC-1 test, but now validate the more stringent c
- Factor graph well-constrained
### Step 5: Retrieve Final Results
- **Action**: GET /api/v1/flights/{id}/results with latest versions
- **Action**: GET /flights/{flightId}/results?include_refined=true
- **Expected Result**: Refined GPS coordinates (post-optimization)
### Step 6: Calculate Errors
docs/03_tests/39_route_chunk_connection_spec.md
+27 -15
View File
@@ -7,10 +7,13 @@ Validate Acceptance Criterion 5 (partial): "System should try to operate when UA
**AC-5**: Connect multiple disconnected route fragments
## Preconditions
1. System with "Atlas" multi-map capability (factor graph)
2. L2 global place recognition functional
3. Geodetic map-merging logic implemented
4. Test dataset: Simulate 3 disconnected route fragments
1. System with "Atlas" multi-map capability (factor graph with native chunk support)
2. F12 Route Chunk Manager functional
3. F10 Factor Graph Optimizer with multi-chunk support
4. L2 global place recognition functional (chunk semantic matching)
5. L3 metric refinement functional (chunk LiteSAM matching)
6. Geodetic map-merging logic implemented (Sim(3) transform)
7. Test dataset: Simulate 3 disconnected route fragments
## Test Description
Test system's ability to handle completely disconnected route segments (no overlap between segments) and eventually connect them into a coherent trajectory using global GPS anchors.
@@ -36,15 +39,19 @@ Test system's ability to handle completely disconnected route segments (no overl
- **Action**: Process AD000025 after AD000010
- **Expected Result**:
- L1 fails (no overlap, large displacement ~2km)
- System detects new fragment
- Initializes Map_Fragment_2
- System **proactively creates new chunk** (Map_Fragment_2)
- Processing continues immediately in new chunk
- Chunk matching attempted asynchronously
### Step 4: Process Fragment 2 Independently
- **Action**: Process AD000025-030
- **Expected Result**:
- New sequential tracking starts
- L2 provides global localization for AD000025
- Map_Fragment_2 created with local consistency
- New sequential tracking starts in chunk_2
- Frames processed within chunk_2 context
- Relative factors added to chunk_2's subgraph
- Chunk_2 optimized independently for local consistency
- Chunk semantic matching attempted when ready (5-20 frames)
- Chunk LiteSAM matching with rotation sweeps attempted
### Step 5: Process Fragment 3
- **Action**: Process AD000050-055 after AD000030
@@ -56,9 +63,11 @@ Test system's ability to handle completely disconnected route segments (no overl
### Step 6: Global Map Merging
- **Action**: Factor graph attempts geodetic map-merging
- **Expected Result**:
- All 3 fragments have GPS anchors from L3
- All 3 chunks have GPS anchors from chunk LiteSAM matching
- Chunks merged via Sim(3) transform (translation, rotation, scale)
- Fragments aligned in global coordinate frame
- Single consistent trajectory created
- Global optimization performed
### Step 7: Validate Fragment Connections
- **Action**: Verify all 18 images have global GPS coordinates
@@ -138,15 +147,18 @@ Processing Mode: Multi-Map Atlas
- Geodetic merging aligns all maps
**Recovery Mechanisms**:
- L2 (AnyLoc) finds location for first image of new fragment
- L3 (LiteSAM) refines GPS anchor
- Factor graph creates new map fragment
- Global alignment via GPS coordinates
- **Proactive chunk creation** on tracking loss (immediate, not reactive)
- Chunk semantic matching (aggregate DINOv2) finds location for chunk
- Chunk LiteSAM matching (with rotation sweeps) refines GPS anchor
- Factor graph creates new chunk subgraph
- Sim(3) transform merges chunks into global trajectory
**Fragment Detection**:
- Large displacement (> 500m) from last image
- Low/zero overlap
- L1 failure triggers new fragment initialization
- L1 failure triggers **proactive** new chunk creation
- Chunks processed independently with local optimization
- Multiple chunks can exist simultaneously
## Notes
- AC-5 describes realistic operational scenario (multiple turns, disconnected segments)
docs/03_tests/40_user_input_recovery_spec.md
+3 -2
View File
@@ -17,8 +17,9 @@ Validate Acceptance Criterion 6: "In case of being absolutely incapable of deter
- **Expected Result**: Event includes image needing fix (AD000005), top-3 satellite tiles for reference
### Step 3: User Provides GPS Fix
- **Action**: User submits GPS for AD000005: POST /api/v1/flights/{id}/images/AD000005/user_fix
- **Expected Result**: Fix accepted, confidence=1.0, processing resumes
- **Action**: User submits GPS for AD000005: POST /flights/{flightId}/user-fix
- **Payload**: `{"frame_id": 5, "uav_pixel": [3126, 2084], "satellite_gps": {"lat": 48.273997, "lon": 37.379828}}`
- **Expected Result**: Fix accepted, processing resumes, SSE event "user_fix_applied" sent
### Step 4: System Incorporates Fix
- **Action**: Factor graph adds user fix as high-confidence GPS anchor
docs/03_tests/55_chunk_rotation_recovery_spec.md
+136
View File
@@ -0,0 +1,136 @@
# Acceptance Test: Chunk Rotation Recovery
## Summary
Validate chunk LiteSAM matching with rotation sweeps for chunks with unknown orientation (sharp turns).
## Linked Acceptance Criteria
**AC-4**: Robust to sharp turns (<5% overlap)
**AC-5**: Connect route chunks
## Preconditions
1. F12 Route Chunk Manager functional
2. F06 Image Rotation Manager with chunk rotation support
3. F09 Metric Refinement with chunk LiteSAM matching
4. F10 Factor Graph Optimizer with chunk merging
5. Test dataset: Chunk with unknown orientation (simulated sharp turn)
## Test Description
Test system's ability to match chunks with unknown orientation using rotation sweeps. When a chunk is created after a sharp turn, its orientation relative to the satellite map is unknown. The system must rotate the entire chunk to all possible angles and attempt LiteSAM matching.
## Test Steps
### Step 1: Create Chunk with Unknown Orientation
- **Action**: Simulate sharp turn scenario
- Process frames 1-10 (normal flight, heading 0°)
- Sharp turn at frame 11 (heading changes to 120°)
- Tracking lost, chunk_2 created proactively
- Process frames 11-20 in chunk_2
- **Expected Result**:
- Chunk_2 created with frames 11-20
- Chunk orientation unknown (previous heading not relevant)
- Chunk ready for matching (10 frames)
### Step 2: Chunk Semantic Matching
- **Action**: Attempt chunk semantic matching
- **Expected Result**:
- F08.compute_chunk_descriptor() → aggregate DINOv2 descriptor
- F08.retrieve_candidate_tiles_for_chunk() → returns top-5 candidate tiles
- Correct tile in top-5 candidates
### Step 3: Chunk Rotation Sweeps
- **Action**: Attempt chunk LiteSAM matching with rotation sweeps
- **Expected Result**:
- F06.try_chunk_rotation_steps() called
- For each rotation angle (0°, 30°, 60°, ..., 330°):
- F06.rotate_chunk_360() rotates all 10 images
- F09.align_chunk_to_satellite() attempts matching
- Match found at 120° rotation (correct orientation)
- Returns ChunkAlignmentResult with:
- rotation_angle: 120°
- chunk_center_gps: correct GPS
- confidence > 0.7
- Sim(3) transform computed
### Step 4: Chunk Merging
- **Action**: Merge chunk_2 to main trajectory
- **Expected Result**:
- F10.add_chunk_anchor() anchors chunk_2
- F10.merge_chunks() merges chunk_2 into chunk_1
- Sim(3) transform applied correctly
- Global trajectory consistent
### Step 5: Verify Final Trajectory
- **Action**: Verify all frames have GPS coordinates
- **Expected Result**:
- All 20 frames have GPS coordinates
- Frames 1-10: Original trajectory
- Frames 11-20: Merged chunk trajectory
- Global consistency maintained
- Accuracy: 18/20 < 50m (90%)
## Success Criteria
**Primary Criterion**:
- Chunk rotation sweeps find correct orientation (120°)
- Chunk LiteSAM matching succeeds with rotation
- Chunk merged correctly to main trajectory
**Supporting Criteria**:
- All 12 rotation angles tried
- Match found at correct angle
- Sim(3) transform computed correctly
- Final trajectory globally consistent
## Expected Results
```
Chunk Rotation Recovery:
- Chunk_2 created: frames 11-20
- Unknown orientation: previous heading (0°) not relevant
- Chunk semantic matching: correct tile in top-5
- Rotation sweeps: 12 rotations tried (0°, 30°, ..., 330°)
- Match found: 120° rotation
- Chunk center GPS: Accurate (within 20m)
- Chunk merged: Sim(3) transform applied
- Final trajectory: Globally consistent
- Accuracy: 18/20 < 50m (90%)
```
## Pass/Fail Criteria
**TEST PASSES IF**:
- Chunk rotation sweeps find match at correct angle (120°)
- Chunk LiteSAM matching succeeds
- Chunk merged correctly
- Final trajectory globally consistent
- Accuracy acceptable (≥ 80% < 50m)
**TEST FAILS IF**:
- Rotation sweeps don't find correct angle
- Chunk LiteSAM matching fails
- Chunk merging produces inconsistent trajectory
- Final accuracy below threshold
## Architecture Elements
**Chunk Rotation**:
- F06 rotates all images in chunk by same angle
- 12 rotation steps: 0°, 30°, 60°, ..., 330°
- F09 attempts LiteSAM matching for each rotation
**Chunk LiteSAM Matching**:
- Aggregate correspondences from multiple images
- More robust than single-image matching
- Handles featureless terrain better
**Chunk Merging**:
- Sim(3) transform (translation, rotation, scale)
- Critical for monocular VO scale ambiguity
- Preserves internal consistency
## Notes
- Rotation sweeps are critical for chunks from sharp turns
- Previous heading may not be relevant after sharp turn
- Chunk matching more robust than single-image matching
- Sim(3) transform accounts for scale differences
docs/03_tests/56_multi_chunk_simultaneous_spec.md
+170
View File
@@ -0,0 +1,170 @@
# Acceptance Test: Multi-Chunk Simultaneous Processing
## Summary
Validate system's ability to process multiple chunks simultaneously, matching and merging them asynchronously.
## Linked Acceptance Criteria
**AC-4**: Robust to sharp turns (<5% overlap)
**AC-5**: Connect route chunks (multiple chunks)
## Preconditions
1. F10 Factor Graph Optimizer with native multi-chunk support
2. F12 Route Chunk Manager functional
3. F11 Failure Recovery Coordinator with chunk orchestration
4. Test dataset: Flight with 3 disconnected segments
## Test Description
Test system's ability to handle multiple disconnected route segments simultaneously. The system should create chunks proactively, process them independently, and match/merge them asynchronously without blocking frame processing.
## Test Steps
### Step 1: Create Multi-Segment Flight
- **Action**: Create flight with 3 disconnected segments:
- Segment 1: AD000001-010 (10 frames, sequential)
- Segment 2: AD000025-030 (6 frames, no overlap with Segment 1)
- Segment 3: AD000050-055 (6 frames, no overlap with Segments 1 or 2)
- **Expected Result**: Flight created with all 22 images
### Step 2: Process Segment 1
- **Action**: Process AD000001-010
- **Expected Result**:
- Chunk_1 created (frames 1-10)
- Sequential VO provides relative factors
- Factors added to chunk_1's subgraph
- Chunk_1 optimized independently
- GPS anchors from LiteSAM matching
- Chunk_1 anchored and merged to main trajectory
### Step 3: Detect Discontinuity (Segment 1 → 2)
- **Action**: Process AD000025 after AD000010
- **Expected Result**:
- Tracking lost detected (large displacement ~2km)
- **Chunk_2 created proactively** (immediate, not reactive)
- Processing continues immediately in chunk_2
- Chunk_1 remains in factor graph (not deleted)
### Step 4: Process Segment 2 Independently
- **Action**: Process AD000025-030
- **Expected Result**:
- Frames processed in chunk_2 context
- Relative factors added to chunk_2's subgraph
- Chunk_2 optimized independently
- Chunk_1 and chunk_2 exist simultaneously
- Chunk_2 matching attempted asynchronously (background)
### Step 5: Detect Discontinuity (Segment 2 → 3)
- **Action**: Process AD000050 after AD000030
- **Expected Result**:
- Tracking lost detected again
- **Chunk_3 created proactively**
- Processing continues in chunk_3
- Chunk_1, chunk_2, chunk_3 all exist simultaneously
### Step 6: Process Segment 3 Independently
- **Action**: Process AD000050-055
- **Expected Result**:
- Frames processed in chunk_3 context
- Chunk_3 optimized independently
- All 3 chunks exist simultaneously
- Each chunk processed independently
### Step 7: Asynchronous Chunk Matching
- **Action**: Background task attempts matching for unanchored chunks
- **Expected Result**:
- Chunk_2 semantic matching attempted
- Chunk_2 LiteSAM matching attempted
- Chunk_2 anchored when match found
- Chunk_3 semantic matching attempted
- Chunk_3 LiteSAM matching attempted
- Chunk_3 anchored when match found
- Matching happens asynchronously (doesn't block frame processing)
### Step 8: Chunk Merging
- **Action**: Merge chunks as they become anchored
- **Expected Result**:
- Chunk_2 merged to chunk_1 when anchored
- Chunk_3 merged to merged trajectory when anchored
- Sim(3) transforms applied correctly
- Global optimization performed
- All chunks merged into single trajectory
### Step 9: Verify Final Trajectory
- **Action**: Verify all 22 frames have GPS coordinates
- **Expected Result**:
- All frames have GPS coordinates
- Trajectory globally consistent
- No systematic bias between segments
- Accuracy: 20/22 < 50m (90.9%)
## Success Criteria
**Primary Criterion**:
- Multiple chunks created simultaneously
- Chunks processed independently
- Chunks matched and merged asynchronously
- Final trajectory globally consistent
**Supporting Criteria**:
- Chunk creation is proactive (immediate)
- Frame processing continues during chunk matching
- Chunk matching doesn't block processing
- Sim(3) transforms computed correctly
## Expected Results
```
Multi-Chunk Simultaneous Processing:
- Chunk_1: frames 1-10, anchored, merged
- Chunk_2: frames 25-30, anchored asynchronously, merged
- Chunk_3: frames 50-55, anchored asynchronously, merged
- Simultaneous existence: All 3 chunks exist at same time
- Independent processing: Each chunk optimized independently
- Asynchronous matching: Matching doesn't block frame processing
- Final trajectory: Globally consistent
- Accuracy: 20/22 < 50m (90.9%)
```
## Pass/Fail Criteria
**TEST PASSES IF**:
- Multiple chunks created simultaneously
- Chunks processed independently
- Chunks matched and merged asynchronously
- Final trajectory globally consistent
- Accuracy acceptable (≥ 80% < 50m)
**TEST FAILS IF**:
- Chunks not created simultaneously
- Chunk processing blocks frame processing
- Chunk matching blocks processing
- Merging produces inconsistent trajectory
- Final accuracy below threshold
## Architecture Elements
**Multi-Chunk Support**:
- F10 Factor Graph Optimizer supports multiple chunks simultaneously
- Each chunk has own subgraph
- Chunks optimized independently
**Proactive Chunk Creation**:
- Chunks created immediately on tracking loss
- Not reactive (doesn't wait for matching to fail)
- Processing continues in new chunk
**Asynchronous Matching**:
- Background task processes unanchored chunks
- Matching doesn't block frame processing
- Chunks matched and merged asynchronously
**Chunk Merging**:
- Sim(3) transform for merging
- Accounts for translation, rotation, scale
- Global optimization after merging
## Notes
- Multiple chunks can exist simultaneously
- Chunk processing is independent and non-blocking
- Asynchronous matching reduces user input requests
- Sim(3) transform critical for scale ambiguity resolution