diff --git a/docs/02_components/02_flight_processor/flight_processor_spec.md b/docs/02_components/02_flight_processor/flight_processor_spec.md index 1fd5a61..27a61ba 100644 --- a/docs/02_components/02_flight_processor/flight_processor_spec.md +++ b/docs/02_components/02_flight_processor/flight_processor_spec.md @@ -137,19 +137,20 @@ class IFlightProcessor(ABC): - **Background task management for each flight** - **Event subscription for F11 recovery events** -### Event-Based Communication +### Direct Call Communication -F02 subscribes to events from F11 Failure Recovery Coordinator instead of F11 directly calling F02's status update methods. This decouples recovery logic from flight state management. +F02 calls F11 methods directly and F11 returns results synchronously. This provides clear data flow and simpler debugging compared to event-based patterns. -**Events Subscribed (from F11)**: -- `RecoveryStarted`: Update flight status to "recovering" -- `RecoverySucceeded`: Update flight status to "processing", resume processing loop -- `RecoveryFailed`: Update flight status to "blocked", blocked=True -- `UserInputNeeded`: Update flight status to "blocked", blocked=True, await user fix -- `UserFixApplied`: Update flight status to "processing", resume processing loop -- `ChunkCreated`: Log chunk creation, update internal tracking -- `ChunkAnchored`: Log chunk anchor, update statistics -- `ChunkMerged`: Trigger result updates via F14 +**F02 calls F11 for recovery operations**: +- `F11.start_search()` → returns `SearchSession` (F02 updates status to "recovering") +- `F11.try_current_grid()` → returns `Optional[AlignmentResult]` +- `F11.mark_found()` → F02 updates status to "processing" +- `F11.create_user_input_request()` → F02 updates status to "blocked" +- `F11.apply_user_anchor()` → returns success, F02 updates status to "processing" +- `F11.create_chunk_on_tracking_loss()` → returns `ChunkHandle` +- `F11.try_chunk_semantic_matching()` → returns candidates +- `F11.try_chunk_litesam_matching()` → returns `Optional[ChunkAlignmentResult]` +- `F11.merge_chunk_to_trajectory()` → F02 then calls F14.update_results_after_chunk_merge() ### Background Task Management @@ -755,18 +756,22 @@ FrameResult: - F06.try_rotation_steps() → establish heading 5. Else: - Pre-rotate image to current heading -6. Compute relative pose via F07 Sequential VO (chunk-aware) +6. Compute relative pose via F07 Sequential VO (chunk-agnostic - returns RelativePose) 7. Add frame to chunk via F12 Route Chunk Manager.add_frame_to_chunk() (F12 handles F10 internally) -9. Get satellite tile from F04 Satellite Data Manager +8. Get satellite tile from F04 Satellite Data Manager +9. Get tile_bounds via F04.compute_tile_bounds(tile_coords) 10. Align to satellite via F09 Metric Refinement (with tile_bounds) 11. If alignment successful: - Add chunk anchor via F10.add_chunk_anchor() - - Update heading via F06 -12. Optimize chunk locally via F10.optimize_chunk() -13. Transform to GPS via F13 Coordinate Transformer -14. Update waypoint via update_waypoint() -15. Publish result via F14 Result Manager → F15 SSE Event Streamer -16. Periodically attempt global optimization and chunk merging + - Update heading via F06.update_heading() + - **Persist heading**: Call F03.save_heading(flight_id, frame_id, heading, timestamp) +12. **Add altitude prior**: Call F10.add_altitude_prior(flight_id, frame_id, altitude, covariance) for scale resolution +13. Optimize chunk locally via F10.optimize_chunk() +14. Get optimized pose from F10.get_trajectory(flight_id) +15. Transform ENU to GPS via F13.enu_to_gps(flight_id, enu_point) +16. Update waypoint via update_waypoint() +17. Publish result via F14.update_frame_result() → F15 SSE Event Streamer +18. Periodically attempt global optimization and chunk merging **Error Conditions**: - `TrackingLossError`: Triggers handle_tracking_loss() diff --git a/docs/02_components/03_flight_database/flight_database_spec.md b/docs/02_components/03_flight_database/flight_database_spec.md index 0021292..6aaec7e 100644 --- a/docs/02_components/03_flight_database/flight_database_spec.md +++ b/docs/02_components/03_flight_database/flight_database_spec.md @@ -1,4 +1,4 @@ -# Flight Database Layer +# Flight Database ## Interface Definition @@ -548,7 +548,7 @@ bool: True if saved **Description**: Saves heading value for temporal smoothing and recovery. **Called By**: -- F06 Image Rotation Manager +- F02 Flight Processor (after F06.update_heading() returns, F02 persists to F03) **Input**: ```python @@ -955,6 +955,11 @@ CREATE TABLE chunks ( created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE, + -- Foreign key to ensure anchor_frame_id references valid frame in flight_images + CONSTRAINT fk_anchor_frame + FOREIGN KEY (flight_id, anchor_frame_id) + REFERENCES flight_images(flight_id, frame_id) + ON DELETE SET NULL, INDEX idx_chunks_flight (flight_id), INDEX idx_chunks_active (flight_id, is_active), INDEX idx_chunks_matching (flight_id, matching_status) diff --git a/docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md b/docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md index 2c8627a..9ce704d 100644 --- a/docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md +++ b/docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md @@ -454,7 +454,7 @@ TileBounds: **Description**: Clears cached tiles for a completed flight. **Called By**: -- F02 Flight Manager (cleanup after flight completion) +- F02 Flight Processor (cleanup after flight completion) **Input**: ```python @@ -516,6 +516,13 @@ bool: True if cleared successfully ### Internal Components - **H06 Web Mercator Utils**: Tile coordinate calculations +**Note on Tile Coordinate Calculations**: F04 delegates ALL tile coordinate calculations to H06 Web Mercator Utils: +- `compute_tile_coords()` → internally calls `H06.latlon_to_tile()` +- `compute_tile_bounds()` → internally calls `H06.compute_tile_bounds()` +- `get_tile_grid()` → uses H06 for coordinate math + +This ensures single source of truth for Web Mercator projection logic and avoids duplication with H06. + ### External Dependencies - **Satellite Provider API**: HTTP tile source - **requests** or **httpx**: HTTP client diff --git a/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md b/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md index a4b02b6..e41def0 100644 --- a/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md +++ b/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md @@ -207,7 +207,7 @@ bool: True if stored successfully 1. Create flight directory if not exists 2. Write each image to disk 3. Update metadata index -4. Persist to F17 Database Layer (metadata only) +4. Persist to F03 Database Layer (metadata only) **Error Conditions**: - `StorageError`: Disk full, permission error diff --git a/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md b/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md index e10c23a..c555dca 100644 --- a/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md +++ b/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md @@ -263,7 +263,9 @@ bool: True if updated successfully 1. Normalize angle to 0-360 range 2. Add to heading history (last 10 headings) 3. Update current_heading for flight -4. **Persist to database**: Call F03 Flight Database.save_heading(flight_id, frame_id, heading, timestamp) +4. Return True (caller F02 is responsible for persistence via F03) + +**Note**: Heading persistence is the caller's responsibility (F02 Flight Processor calls F03.save_heading() after receiving the updated heading). **Test Cases**: 1. **Update heading**: Sets new heading @@ -495,13 +497,14 @@ return None # No match found ## Dependencies ### Internal Components -- **F04 Satellite Data Manager**: For tile fetching (`fetch_tile`) and tile bounds computation (`compute_tile_bounds`) - **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching. - **H07 Image Rotation Utils**: For image rotation and angle calculations -- **F12 Route Chunk Manager**: For chunk image retrieval -- **F03 Flight Database**: For heading persistence -**Note**: `TileBounds` data model is imported from F09 Metric Refinement. +**Note**: +- `TileBounds` data model is imported from F09 Metric Refinement. +- F06 does NOT call F04 directly. The caller (F02 or F11) provides satellite tiles and tile_bounds. +- F06 does NOT persist heading to database. The caller (F02) is responsible for calling F03.save_heading(). +- Chunk rotation orchestration (calling try_chunk_rotation_steps in recovery flow) is done by F11 Failure Recovery Coordinator. ### External Dependencies - **opencv-python**: Image rotation (`cv2.warpAffine`) diff --git a/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md b/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md index 7cd5211..12ba7f5 100644 --- a/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md +++ b/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md @@ -2,12 +2,12 @@ ## Interface Definition -**Interface Name**: `ISequentialVO` +**Interface Name**: `ISequentialVisualOdometry` ### Interface Methods ```python -class ISequentialVO(ABC): +class ISequentialVisualOdometry(ABC): @abstractmethod def compute_relative_pose(self, prev_image: np.ndarray, curr_image: np.ndarray) -> Optional[RelativePose]: pass @@ -279,7 +279,8 @@ Motion: - **F17 Configuration Manager**: For camera parameters - **H01 Camera Model**: For coordinate normalization - **H05 Performance Monitor**: For timing measurements -- **F10 Factor Graph Optimizer**: For chunk-scoped factor addition + +**Note**: F07 is chunk-agnostic and does NOT depend on F10 Factor Graph Optimizer. F07 only computes relative poses between images and returns them to the caller (F02). The caller (F02) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10. ### External Dependencies - **SuperPoint**: Feature extraction model diff --git a/docs/02_components/08_global_place_recognition/global_place_recognition_spec.md b/docs/02_components/08_global_place_recognition/global_place_recognition_spec.md index 34c9344..2118e9d 100644 --- a/docs/02_components/08_global_place_recognition/global_place_recognition_spec.md +++ b/docs/02_components/08_global_place_recognition/global_place_recognition_spec.md @@ -229,8 +229,10 @@ bool: True if database loaded successfully **Satellite Provider Responsibility**: - Satellite provider builds the semantic index offline using DINOv2 + VLAD - Provider delivers index file along with satellite tiles -- Index format: Faiss IVF or HNSW index + tile metadata JSON +- **Index format**: Faiss IVF1000 (Inverted File with 1000 clusters) + tile metadata JSON - Provider is responsible for index updates when satellite data changes +- Index is rebuilt by provider whenever new satellite tiles are fetched on demand +- Supported providers: Maxar, Google Maps, Copernicus, etc. **Error Conditions**: - Raises `IndexNotFoundError`: Index file not found diff --git a/docs/02_components/09_metric_refinement/metric_refinement_spec.md b/docs/02_components/09_metric_refinement/metric_refinement_spec.md index 0556707..0fcfd06 100644 --- a/docs/02_components/09_metric_refinement/metric_refinement_spec.md +++ b/docs/02_components/09_metric_refinement/metric_refinement_spec.md @@ -50,7 +50,7 @@ class IMetricRefinement(ABC): - Handles altitude variations (<1km) - Multi-scale processing for different GSDs - Domain gap (UAV downward vs satellite nadir view) -- **Critical**: Fails if rotation >45° (handled by G06) +- **Critical**: Fails if rotation >45° (handled by F06) - **Chunk-level matching (aggregate correspondences from multiple images)** ## API Methods @@ -383,6 +383,12 @@ Optional[np.ndarray]: 3×3 homography matrix or None - **H02 GSD Calculator**: For coordinate transformations - **H05 Performance Monitor**: For timing +**Critical Dependency on F06 Image Rotation Manager**: +- F09 requires pre-rotated images (rotation <45° from north) +- Caller (F06 or F11) must pre-rotate images using F06.rotate_image_360() before calling F09.align_to_satellite() +- If rotation >45°, F09 will fail to match (by design) +- F06 handles the rotation sweep (trying 0°, 30°, 60°, etc.) and calls F09 for each rotation + **Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager) ### External Dependencies diff --git a/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md b/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md index 7707606..53dbe42 100644 --- a/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md +++ b/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md @@ -91,51 +91,42 @@ class IFailureRecoveryCoordinator(ABC): **F10 provides low-level factor graph operations** (see F10 spec) -### Event-Based Communication +### Direct Call Communication -F11 emits events instead of directly calling F02's status update methods. This decouples recovery logic from flight state management. +F11 is called directly by F02 and returns results synchronously. F02 is responsible for updating flight status based on F11's return values. This provides clear data flow and simpler debugging. -### Internal Events (Component-to-Component) +**F02 calls F11 and handles responses**: +- `start_search()` → returns `SearchSession`, F02 updates status to "recovering" +- `try_current_grid()` → returns `Optional[AlignmentResult]`, F02 checks if found +- `mark_found()` → F02 updates status to "processing" +- `create_user_input_request()` → returns `UserInputRequest`, F02 updates status to "blocked" and sends via F15 +- `apply_user_anchor()` → returns `bool`, F02 updates status to "processing" on success +- `create_chunk_on_tracking_loss()` → returns `ChunkHandle`, F02 tracks chunk +- `try_chunk_semantic_matching()` → returns `Optional[List[TileCandidate]]` +- `try_chunk_litesam_matching()` → returns `Optional[ChunkAlignmentResult]` +- `merge_chunk_to_trajectory()` → returns `bool`, F02 calls F14.update_results_after_chunk_merge() -F11 emits events for internal component communication. These are NOT directly sent to clients. - -**Events Emitted (internal)**: -- `RecoveryStarted`: When tracking loss detected and recovery begins -- `RecoverySucceeded`: When recovery finds a match (single-image or chunk) -- `RecoveryFailed`: When all recovery strategies exhausted -- `UserInputNeeded`: When user input is required -- `UserFixApplied`: When user-provided anchor successfully applied -- `ChunkCreated`: When new chunk created on tracking loss -- `ChunkAnchored`: When chunk successfully matched and anchored -- `ChunkMerged`: When chunk merged into main trajectory (includes flight_id, chunk_id, merged_frames) -- `ChunkMatchingFailed`: When chunk matching exhausts all candidates and fails - -**Event Listeners** (F02 Flight Processor subscribes to these): -- On `RecoveryStarted`: Update status to "recovering" -- On `RecoveryFailed`: Update status to "blocked" -- On `RecoverySucceeded`: Update status to "processing" -- On `UserInputNeeded`: Update status to "blocked", blocked=True -- On `ChunkMatchingFailed`: Re-queue chunk for user input or continue building +**Note**: F11 does NOT emit events. All communication is through direct method calls with return values. F02 orchestrates the recovery flow by calling F11 methods and handling responses. ### External SSE Events (to Clients) -F11 does NOT directly send events to clients. External events are routed through F14 Result Manager which manages SSE streaming via F15: +F11 does NOT directly send events to clients. F02 orchestrates client communication based on F11's return values: -- When F11 emits `UserInputNeeded` → F02 receives → F02 calls F14.publish_user_input_request() → F14 sends via F15 SSE -- When F11 emits `ChunkMerged` → F02 receives → F02 calls F14.update_results_after_chunk_merge() → F14 sends via F15 SSE -- When F11 emits `RecoveryFailed` → F02 receives → F02 calls F14.publish_processing_blocked() → F14 sends via F15 SSE +- When F11.create_user_input_request() returns `UserInputRequest` → F02 calls F15.send_user_input_request() +- When F11.merge_chunk_to_trajectory() returns True → F02 calls F14.update_results_after_chunk_merge() +- When recovery fails (all F11 methods return None/False) → F02 calls F15.send_processing_blocked() This separation ensures: -1. F11 is decoupled from SSE implementation -2. F14 controls all client-facing communication -3. Consistent event format for clients +1. F11 is a pure business logic component (no I/O dependencies) +2. F02 controls all coordination and state management +3. F14/F15 handle all client-facing communication ### Scope - Confidence monitoring - Progressive search coordination - User input request/response handling - Recovery strategy orchestration -- Integration point for G04, G06, G08, G09, F10 +- Integration point for F04, F06, F08, F09, F10 - **Chunk lifecycle and matching coordination** - **Multi-chunk simultaneous processing** @@ -245,7 +236,7 @@ SearchSession: **Processing Flow**: 1. Increment current_grid_size (1→4→9→16→25) -2. Call G04.expand_search_grid() to get new tiles only +2. Call F04.expand_search_grid() to get new tiles only 3. Return new tile coordinates **Test Cases**: @@ -264,7 +255,7 @@ SearchSession: **Input**: ```python session: SearchSession -tiles: Dict[str, np.ndarray] # From G04 +tiles: Dict[str, np.ndarray] # From F04 ``` **Output**: `Optional[AlignmentResult]` - Match result or None @@ -779,14 +770,16 @@ class Sim3Transform(BaseModel): scale: float ``` -### RecoveryEvent +### RecoveryStatus (Returned by recovery methods) ```python -class RecoveryEvent(BaseModel): - event_type: str # "RecoveryStarted", "RecoverySucceeded", "RecoveryFailed", "UserInputNeeded", "ChunkCreated", "ChunkAnchored", "ChunkMerged" - flight_id: str - frame_id: Optional[int] +class RecoveryStatus(BaseModel): + """Status returned by F11 methods, used by F02 to determine next action.""" + success: bool + method: str # "rotation_sweep", "progressive_search", "chunk_matching", "user_input" + gps: Optional[GPSPoint] chunk_id: Optional[str] - data: Optional[Dict[str, Any]] - timestamp: datetime + message: Optional[str] ``` +**Note**: F11 uses direct return values instead of events. F02 inspects return values to determine status updates and next actions. + diff --git a/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md b/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md index 14125a7..bfc2835 100644 --- a/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md +++ b/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md @@ -27,11 +27,11 @@ class ICoordinateTransformer(ABC): # Pixel/GPS Conversions @abstractmethod - def pixel_to_gps(self, pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint: + def pixel_to_gps(self, flight_id: str, pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint: pass @abstractmethod - def gps_to_pixel(self, gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]: + def gps_to_pixel(self, flight_id: str, gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]: pass @abstractmethod @@ -195,16 +195,17 @@ GPSPoint: WGS84 coordinates ### Pixel/GPS Conversions -### `pixel_to_gps(pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint` +### `pixel_to_gps(flight_id: str, pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint` **Description**: Converts pixel coordinates to GPS using camera pose and ground plane assumption. **Called By**: -- F13 Result Manager (for frame center GPS) +- F02 Flight Processor (for frame center GPS) - Internal (for image_object_to_gps) **Input**: ```python +flight_id: str # Flight identifier (required for ENU origin lookup) pixel: Tuple[float, float] # (x, y) in image coordinates frame_pose: Pose # From Factor Graph (ENU coordinates) camera_params: CameraParameters @@ -222,7 +223,7 @@ GPSPoint: 1. Unproject pixel to 3D ray using H01 Camera Model 2. Intersect ray with ground plane at altitude 3. Transform 3D point from camera frame to ENU using frame_pose -4. Convert ENU to WGS84 GPS using H06 Web Mercator Utils +4. Convert ENU to WGS84 GPS using enu_to_gps(flight_id, enu_point) **Assumptions**: - Ground plane assumption (terrain height negligible) @@ -240,7 +241,7 @@ GPSPoint: --- -### `gps_to_pixel(gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]` +### `gps_to_pixel(flight_id: str, gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]` **Description**: Inverse projection from GPS to image pixel coordinates. @@ -250,6 +251,7 @@ GPSPoint: **Input**: ```python +flight_id: str # Flight identifier (required for ENU origin lookup) gps: GPSPoint frame_pose: Pose camera_params: CameraParameters @@ -262,7 +264,7 @@ Tuple[float, float]: (x, y) pixel coordinates ``` **Algorithm**: -1. Convert GPS to ENU using H06 +1. Convert GPS to ENU using gps_to_enu(flight_id, gps) 2. Transform ENU point to camera frame using frame_pose 3. Project 3D point to image plane using H01 Camera Model 4. Return pixel coordinates @@ -390,11 +392,12 @@ List[Tuple[float, float]]: Transformed points ## Dependencies ### Internal Components -- **F10 Factor Graph Optimizer**: For frame poses -- **F17 Configuration Manager**: For camera parameters +- **F10 Factor Graph Optimizer**: For frame poses via get_trajectory(flight_id) +- **F17 Configuration Manager**: For camera parameters via get_flight_config(flight_id) - **H01 Camera Model**: For projection operations - **H02 GSD Calculator**: For GSD calculations -- **H06 Web Mercator Utils**: For coordinate conversions + +**Note**: F13 uses internal enu_to_gps() and gps_to_enu() methods that rely on the ENU origin set via set_enu_origin(flight_id, origin_gps). H06 Web Mercator Utils is NOT used for ENU conversions - ENU is a local Cartesian coordinate system. ### External Dependencies - **numpy**: Matrix operations diff --git a/docs/02_components/14_result_manager/result_manager_spec.md b/docs/02_components/14_result_manager/result_manager_spec.md index f505e05..a768316 100644 --- a/docs/02_components/14_result_manager/result_manager_spec.md +++ b/docs/02_components/14_result_manager/result_manager_spec.md @@ -41,7 +41,8 @@ class IResultManager(ABC): - Store waypoint updates via F03 Flight Database - Send incremental updates via F15 SSE Event Streamer - Maintain result versioning for audit trail -- Convert optimized poses to GPS coordinates + +**Note**: F14 receives GPS coordinates directly from the caller (F02). F14 does NOT call F10 Factor Graph Optimizer or F13 Coordinate Transformer for pose retrieval/conversion. This ensures unidirectional data flow: F02 → F14 → F03/F15. ### Scope - Result state management @@ -153,11 +154,14 @@ frame_ids: List[int] # Frames with updated poses **Output**: `bool` +**Note**: F14 does NOT call F10 or F13. The caller (F02) performs the following steps before calling mark_refined(): +1. F02 gets refined poses from F10.get_trajectory(flight_id) +2. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple) +3. F02 calls F14.mark_refined() with the GPS-converted results + **Processing Flow**: -1. For each frame_id: - - Get refined pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position - - Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])` - - Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint +1. For each frame_id in frame_ids: + - Receive GPS coordinates from caller (already converted) - Update result with refined=True via F03 Flight Database.save_frame_result() - Update waypoint via F03 Flight Database.update_waypoint() - Call F15 SSE Event Streamer.send_refinement() @@ -204,11 +208,15 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge **Output**: `bool` - True if updated successfully +**Note**: F14 does NOT call F10 or F13. The caller (F02) performs these steps before calling update_results_after_chunk_merge(): +1. F02 receives ChunkMerged event from F11 with merged_frames list +2. F02 gets updated poses from F10.get_trajectory(flight_id) +3. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple) for each frame +4. F02 calls F14.update_results_after_chunk_merge() with GPS-converted results + **Processing Flow**: 1. For each frame_id in merged_frames: - - Get updated pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position - - Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])` - - Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint + - Receive GPS coordinates from caller (already converted) - Update frame result via F03 Flight Database.save_frame_result() - Update waypoint via F03 Flight Database.update_waypoint() - Send refinement event via F15 SSE Event Streamer.send_refinement() @@ -256,10 +264,9 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge ### Internal Components - **F03 Flight Database**: For waypoint and frame result persistence -- **F10 Factor Graph Optimizer**: For refined pose retrieval -- **F13 Coordinate Transformer**: For ENU to GPS conversion - **F15 SSE Event Streamer**: For real-time result streaming -- **F11 Failure Recovery Coordinator**: Triggers chunk merge result updates + +**Note**: F14 does NOT depend on F10, F13, or F11. The caller (F02) coordinates with those components and provides GPS-converted results to F14. This ensures unidirectional data flow and eliminates circular dependencies. ### External Dependencies - None diff --git a/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md b/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md index 7dbf843..a6aa711 100644 --- a/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md +++ b/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md @@ -96,7 +96,7 @@ StreamConnection: **Description**: Sends frame_processed event. -**Called By**: F13 Result Manager +**Called By**: F14 Result Manager **Event Format**: ```json @@ -164,7 +164,7 @@ StreamConnection: **Description**: Sends frame_refined event. -**Called By**: F13 Result Manager +**Called By**: F14 Result Manager **Event Format**: ```json diff --git a/docs/02_components/17_configuration_manager/configuration_manager_spec.md b/docs/02_components/17_configuration_manager/configuration_manager_spec.md index 028f76f..519041a 100644 --- a/docs/02_components/17_configuration_manager/configuration_manager_spec.md +++ b/docs/02_components/17_configuration_manager/configuration_manager_spec.md @@ -241,3 +241,43 @@ class OperationalArea(BaseModel): max_lon: float = 40.0 ``` +### RecoveryConfig +```python +class RecoveryConfig(BaseModel): + """Configuration for failure recovery and progressive search.""" + search_grid_sizes: List[int] = [1, 4, 9, 16, 25] # Progressive tile search grid + min_chunk_frames_for_matching: int = 5 # Minimum frames before chunk matching + max_chunk_frames_for_matching: int = 20 # Maximum frames per chunk + user_input_threshold_tiles: int = 25 # Request user input after this many tiles + chunk_matching_interval_seconds: float = 5.0 # Background matching interval + confidence_threshold_good: float = 0.7 # Confidence for good tracking + confidence_threshold_degraded: float = 0.5 # Confidence for degraded tracking + min_inlier_count_good: int = 50 # Inliers for good tracking + min_inlier_count_tracking: int = 20 # Minimum inliers before tracking loss +``` + +### RotationConfig +```python +class RotationConfig(BaseModel): + """Configuration for image rotation and heading tracking.""" + rotation_step_degrees: float = 30.0 # Degrees per rotation step + litesam_max_rotation_tolerance: float = 45.0 # Max rotation LiteSAM handles + sharp_turn_threshold_degrees: float = 45.0 # Threshold for sharp turn detection + heading_history_size: int = 10 # Number of headings to track + confidence_threshold: float = 0.7 # For accepting rotation match + + @property + def rotation_iterations(self) -> int: + """Number of rotation steps (360 / step_degrees).""" + return int(360 / self.rotation_step_degrees) +``` + +## Dependencies + +### Internal Components +- **F03 Flight Database**: For flight-specific configuration persistence (save_flight_config stores to F03) + +### External Dependencies +- **pydantic**: Data model validation +- **PyYAML**: Configuration file parsing + diff --git a/docs/02_components/decomposition_plan.md b/docs/02_components/decomposition_plan.md index d13c97b..d24cea7 100644 --- a/docs/02_components/decomposition_plan.md +++ b/docs/02_components/decomposition_plan.md @@ -73,7 +73,7 @@ ### Visual Processing **F07_sequential_visual_odometry** -**Interface**: `ISequentialVO` +**Interface**: `ISequentialVisualOdometry` **API**: `compute_relative_pose()`, `extract_features()`, `match_features()`, `estimate_motion()` **F08_global_place_recognition** @@ -319,6 +319,7 @@ |--------|--------|--------|---------| | F11 (background) | F12 | `get_chunks_for_matching()` | Get unanchored chunks ready for matching | | F11 | F12 | `get_chunk_images()` | Get chunk images | +| F12 | F05 | `get_image_by_sequence()` | **Load images for chunk** (F12 delegates to F05 for actual image retrieval) | | F11 | F08 | `retrieve_candidate_tiles_for_chunk()` | Chunk semantic matching (aggregate DINOv2) | | F08 | F16 | `get_inference_engine("DINOv2")` | Get model for descriptor computation | | F08 | H04 | `search()` | Query Faiss with chunk descriptor | @@ -348,6 +349,7 @@ ✅ **Drift correction**: F02→F04,F09,F10 ✅ **Tracking loss**: F11→F12(proactive chunk),F06,F08,F04(progressive),F09,F15,F02 ✅ **Chunk building**: F02→F12→F10,F07 +✅ **Chunk image retrieval**: F12→F05(get_image_by_sequence for chunk images) ✅ **Chunk semantic matching**: F11→F12→F08(chunk descriptor) ✅ **Chunk LiteSAM matching**: F11→F06(chunk rotation)→F09(chunk alignment) ✅ **Chunk merging**: F11→F10(Sim3 transform) diff --git a/docs/02_components/helpers/h04_faiss_index_manager_spec.md b/docs/02_components/helpers/h04_faiss_index_manager_spec.md index ae9e05a..a965881 100644 --- a/docs/02_components/helpers/h04_faiss_index_manager_spec.md +++ b/docs/02_components/helpers/h04_faiss_index_manager_spec.md @@ -40,12 +40,22 @@ class IFaissIndexManager(ABC): ## Component Description -Manages Faiss indices for DINOv2 descriptor similarity search. H04 builds indexes from UAV image descriptors for: -1. **Loop closure detection**: Find when UAV revisits previously seen areas within the same flight -2. **Chunk-to-chunk matching**: Match disconnected chunks to each other -3. **Flight-to-flight matching**: Match current flight to previous flights in same area +Manages Faiss indices for DINOv2 descriptor similarity search. H04 provides generic Faiss index operations used by: -**Index Source**: Descriptors are computed from UAV images using F08's DINOv2 encoder, NOT from satellite images. The index enables finding similar UAV viewpoints. +### Satellite Index (Primary Use Case) +- **Index format**: IVF1000 (Inverted File with 1000 clusters) +- **Index source**: Pre-built by external Satellite Provider (Maxar, Google Maps, Copernicus, etc.) +- **Index delivery**: Provider delivers index file + tile metadata when tiles are fetched on demand +- **Index updates**: Provider rebuilds index when new satellite tiles become available +- **Usage**: F08 Global Place Recognition loads this index via H04.load_index() + +### UAV Index (Optional, Future Use) +For loop closure and chunk-to-chunk matching: +1. **Loop closure detection**: Find when UAV revisits previously seen areas +2. **Chunk-to-chunk matching**: Match disconnected chunks to each other +3. **Flight-to-flight matching**: Match current flight to previous flights + +**Note**: H04 is a low-level utility that manages ANY Faiss index. It does NOT know whether the index contains satellite or UAV descriptors. ## API Methods diff --git a/docs/02_components/system_flows.md b/docs/02_components/system_flows.md index 55ca711..ec55c09 100644 --- a/docs/02_components/system_flows.md +++ b/docs/02_components/system_flows.md @@ -16,7 +16,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match | F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search | | F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage | | F06 | Image Rotation Manager | `IImageRotationManager` | Rotation sweeps, heading tracking | -| F07 | Sequential Visual Odometry | `ISequentialVO` | Frame-to-frame VO (SuperPoint+LightGlue) | +| F07 | Sequential Visual Odometry | `ISequentialVisualOdometry` | Frame-to-frame VO (SuperPoint+LightGlue) | | F08 | Global Place Recognition | `IGlobalPlaceRecognition` | Coarse localization (DINOv2+VLAD) | | F09 | Metric Refinement | `IMetricRefinement` | Precise alignment (LiteSAM) | | F10 | Factor Graph Optimizer | `IFactorGraphOptimizer` | GTSAM-based trajectory optimization |