improving components consistency

This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-30 08:44:28 +02:00
parent 310cf78ee7
commit 700d00a1bc
16 changed files with 186 additions and 102 deletions
@@ -137,19 +137,20 @@ class IFlightProcessor(ABC):
- **Background task management for each flight** - **Background task management for each flight**
- **Event subscription for F11 recovery events** - **Event subscription for F11 recovery events**
### Event-Based Communication ### Direct Call Communication
F02 subscribes to events from F11 Failure Recovery Coordinator instead of F11 directly calling F02's status update methods. This decouples recovery logic from flight state management. F02 calls F11 methods directly and F11 returns results synchronously. This provides clear data flow and simpler debugging compared to event-based patterns.
**Events Subscribed (from F11)**: **F02 calls F11 for recovery operations**:
- `RecoveryStarted`: Update flight status to "recovering" - `F11.start_search()` → returns `SearchSession` (F02 updates status to "recovering")
- `RecoverySucceeded`: Update flight status to "processing", resume processing loop - `F11.try_current_grid()` → returns `Optional[AlignmentResult]`
- `RecoveryFailed`: Update flight status to "blocked", blocked=True - `F11.mark_found()` → F02 updates status to "processing"
- `UserInputNeeded`: Update flight status to "blocked", blocked=True, await user fix - `F11.create_user_input_request()` → F02 updates status to "blocked"
- `UserFixApplied`: Update flight status to "processing", resume processing loop - `F11.apply_user_anchor()` → returns success, F02 updates status to "processing"
- `ChunkCreated`: Log chunk creation, update internal tracking - `F11.create_chunk_on_tracking_loss()` → returns `ChunkHandle`
- `ChunkAnchored`: Log chunk anchor, update statistics - `F11.try_chunk_semantic_matching()` → returns candidates
- `ChunkMerged`: Trigger result updates via F14 - `F11.try_chunk_litesam_matching()` → returns `Optional[ChunkAlignmentResult]`
- `F11.merge_chunk_to_trajectory()` → F02 then calls F14.update_results_after_chunk_merge()
### Background Task Management ### Background Task Management
@@ -755,18 +756,22 @@ FrameResult:
- F06.try_rotation_steps() → establish heading - F06.try_rotation_steps() → establish heading
5. Else: 5. Else:
- Pre-rotate image to current heading - Pre-rotate image to current heading
6. Compute relative pose via F07 Sequential VO (chunk-aware) 6. Compute relative pose via F07 Sequential VO (chunk-agnostic - returns RelativePose)
7. Add frame to chunk via F12 Route Chunk Manager.add_frame_to_chunk() (F12 handles F10 internally) 7. Add frame to chunk via F12 Route Chunk Manager.add_frame_to_chunk() (F12 handles F10 internally)
9. Get satellite tile from F04 Satellite Data Manager 8. Get satellite tile from F04 Satellite Data Manager
9. Get tile_bounds via F04.compute_tile_bounds(tile_coords)
10. Align to satellite via F09 Metric Refinement (with tile_bounds) 10. Align to satellite via F09 Metric Refinement (with tile_bounds)
11. If alignment successful: 11. If alignment successful:
- Add chunk anchor via F10.add_chunk_anchor() - Add chunk anchor via F10.add_chunk_anchor()
- Update heading via F06 - Update heading via F06.update_heading()
12. Optimize chunk locally via F10.optimize_chunk() - **Persist heading**: Call F03.save_heading(flight_id, frame_id, heading, timestamp)
13. Transform to GPS via F13 Coordinate Transformer 12. **Add altitude prior**: Call F10.add_altitude_prior(flight_id, frame_id, altitude, covariance) for scale resolution
14. Update waypoint via update_waypoint() 13. Optimize chunk locally via F10.optimize_chunk()
15. Publish result via F14 Result Manager → F15 SSE Event Streamer 14. Get optimized pose from F10.get_trajectory(flight_id)
16. Periodically attempt global optimization and chunk merging 15. Transform ENU to GPS via F13.enu_to_gps(flight_id, enu_point)
16. Update waypoint via update_waypoint()
17. Publish result via F14.update_frame_result() → F15 SSE Event Streamer
18. Periodically attempt global optimization and chunk merging
**Error Conditions**: **Error Conditions**:
- `TrackingLossError`: Triggers handle_tracking_loss() - `TrackingLossError`: Triggers handle_tracking_loss()
@@ -1,4 +1,4 @@
# Flight Database Layer # Flight Database
## Interface Definition ## Interface Definition
@@ -548,7 +548,7 @@ bool: True if saved
**Description**: Saves heading value for temporal smoothing and recovery. **Description**: Saves heading value for temporal smoothing and recovery.
**Called By**: **Called By**:
- F06 Image Rotation Manager - F02 Flight Processor (after F06.update_heading() returns, F02 persists to F03)
**Input**: **Input**:
```python ```python
@@ -955,6 +955,11 @@ CREATE TABLE chunks (
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE, FOREIGN KEY (flight_id) REFERENCES flights(id) ON DELETE CASCADE,
-- Foreign key to ensure anchor_frame_id references valid frame in flight_images
CONSTRAINT fk_anchor_frame
FOREIGN KEY (flight_id, anchor_frame_id)
REFERENCES flight_images(flight_id, frame_id)
ON DELETE SET NULL,
INDEX idx_chunks_flight (flight_id), INDEX idx_chunks_flight (flight_id),
INDEX idx_chunks_active (flight_id, is_active), INDEX idx_chunks_active (flight_id, is_active),
INDEX idx_chunks_matching (flight_id, matching_status) INDEX idx_chunks_matching (flight_id, matching_status)
@@ -454,7 +454,7 @@ TileBounds:
**Description**: Clears cached tiles for a completed flight. **Description**: Clears cached tiles for a completed flight.
**Called By**: **Called By**:
- F02 Flight Manager (cleanup after flight completion) - F02 Flight Processor (cleanup after flight completion)
**Input**: **Input**:
```python ```python
@@ -516,6 +516,13 @@ bool: True if cleared successfully
### Internal Components ### Internal Components
- **H06 Web Mercator Utils**: Tile coordinate calculations - **H06 Web Mercator Utils**: Tile coordinate calculations
**Note on Tile Coordinate Calculations**: F04 delegates ALL tile coordinate calculations to H06 Web Mercator Utils:
- `compute_tile_coords()` → internally calls `H06.latlon_to_tile()`
- `compute_tile_bounds()` → internally calls `H06.compute_tile_bounds()`
- `get_tile_grid()` → uses H06 for coordinate math
This ensures single source of truth for Web Mercator projection logic and avoids duplication with H06.
### External Dependencies ### External Dependencies
- **Satellite Provider API**: HTTP tile source - **Satellite Provider API**: HTTP tile source
- **requests** or **httpx**: HTTP client - **requests** or **httpx**: HTTP client
@@ -207,7 +207,7 @@ bool: True if stored successfully
1. Create flight directory if not exists 1. Create flight directory if not exists
2. Write each image to disk 2. Write each image to disk
3. Update metadata index 3. Update metadata index
4. Persist to F17 Database Layer (metadata only) 4. Persist to F03 Database Layer (metadata only)
**Error Conditions**: **Error Conditions**:
- `StorageError`: Disk full, permission error - `StorageError`: Disk full, permission error
@@ -263,7 +263,9 @@ bool: True if updated successfully
1. Normalize angle to 0-360 range 1. Normalize angle to 0-360 range
2. Add to heading history (last 10 headings) 2. Add to heading history (last 10 headings)
3. Update current_heading for flight 3. Update current_heading for flight
4. **Persist to database**: Call F03 Flight Database.save_heading(flight_id, frame_id, heading, timestamp) 4. Return True (caller F02 is responsible for persistence via F03)
**Note**: Heading persistence is the caller's responsibility (F02 Flight Processor calls F03.save_heading() after receiving the updated heading).
**Test Cases**: **Test Cases**:
1. **Update heading**: Sets new heading 1. **Update heading**: Sets new heading
@@ -495,13 +497,14 @@ return None # No match found
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- **F04 Satellite Data Manager**: For tile fetching (`fetch_tile`) and tile bounds computation (`compute_tile_bounds`)
- **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching. - **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching.
- **H07 Image Rotation Utils**: For image rotation and angle calculations - **H07 Image Rotation Utils**: For image rotation and angle calculations
- **F12 Route Chunk Manager**: For chunk image retrieval
- **F03 Flight Database**: For heading persistence
**Note**: `TileBounds` data model is imported from F09 Metric Refinement. **Note**:
- `TileBounds` data model is imported from F09 Metric Refinement.
- F06 does NOT call F04 directly. The caller (F02 or F11) provides satellite tiles and tile_bounds.
- F06 does NOT persist heading to database. The caller (F02) is responsible for calling F03.save_heading().
- Chunk rotation orchestration (calling try_chunk_rotation_steps in recovery flow) is done by F11 Failure Recovery Coordinator.
### External Dependencies ### External Dependencies
- **opencv-python**: Image rotation (`cv2.warpAffine`) - **opencv-python**: Image rotation (`cv2.warpAffine`)
@@ -2,12 +2,12 @@
## Interface Definition ## Interface Definition
**Interface Name**: `ISequentialVO` **Interface Name**: `ISequentialVisualOdometry`
### Interface Methods ### Interface Methods
```python ```python
class ISequentialVO(ABC): class ISequentialVisualOdometry(ABC):
@abstractmethod @abstractmethod
def compute_relative_pose(self, prev_image: np.ndarray, curr_image: np.ndarray) -> Optional[RelativePose]: def compute_relative_pose(self, prev_image: np.ndarray, curr_image: np.ndarray) -> Optional[RelativePose]:
pass pass
@@ -279,7 +279,8 @@ Motion:
- **F17 Configuration Manager**: For camera parameters - **F17 Configuration Manager**: For camera parameters
- **H01 Camera Model**: For coordinate normalization - **H01 Camera Model**: For coordinate normalization
- **H05 Performance Monitor**: For timing measurements - **H05 Performance Monitor**: For timing measurements
- **F10 Factor Graph Optimizer**: For chunk-scoped factor addition
**Note**: F07 is chunk-agnostic and does NOT depend on F10 Factor Graph Optimizer. F07 only computes relative poses between images and returns them to the caller (F02). The caller (F02) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
### External Dependencies ### External Dependencies
- **SuperPoint**: Feature extraction model - **SuperPoint**: Feature extraction model
@@ -229,8 +229,10 @@ bool: True if database loaded successfully
**Satellite Provider Responsibility**: **Satellite Provider Responsibility**:
- Satellite provider builds the semantic index offline using DINOv2 + VLAD - Satellite provider builds the semantic index offline using DINOv2 + VLAD
- Provider delivers index file along with satellite tiles - Provider delivers index file along with satellite tiles
- Index format: Faiss IVF or HNSW index + tile metadata JSON - **Index format**: Faiss IVF1000 (Inverted File with 1000 clusters) + tile metadata JSON
- Provider is responsible for index updates when satellite data changes - Provider is responsible for index updates when satellite data changes
- Index is rebuilt by provider whenever new satellite tiles are fetched on demand
- Supported providers: Maxar, Google Maps, Copernicus, etc.
**Error Conditions**: **Error Conditions**:
- Raises `IndexNotFoundError`: Index file not found - Raises `IndexNotFoundError`: Index file not found
@@ -50,7 +50,7 @@ class IMetricRefinement(ABC):
- Handles altitude variations (<1km) - Handles altitude variations (<1km)
- Multi-scale processing for different GSDs - Multi-scale processing for different GSDs
- Domain gap (UAV downward vs satellite nadir view) - Domain gap (UAV downward vs satellite nadir view)
- **Critical**: Fails if rotation >45° (handled by G06) - **Critical**: Fails if rotation >45° (handled by F06)
- **Chunk-level matching (aggregate correspondences from multiple images)** - **Chunk-level matching (aggregate correspondences from multiple images)**
## API Methods ## API Methods
@@ -383,6 +383,12 @@ Optional[np.ndarray]: 3×3 homography matrix or None
- **H02 GSD Calculator**: For coordinate transformations - **H02 GSD Calculator**: For coordinate transformations
- **H05 Performance Monitor**: For timing - **H05 Performance Monitor**: For timing
**Critical Dependency on F06 Image Rotation Manager**:
- F09 requires pre-rotated images (rotation <45° from north)
- Caller (F06 or F11) must pre-rotate images using F06.rotate_image_360() before calling F09.align_to_satellite()
- If rotation >45°, F09 will fail to match (by design)
- F06 handles the rotation sweep (trying 0°, 30°, 60°, etc.) and calls F09 for each rotation
**Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager) **Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager)
### External Dependencies ### External Dependencies
@@ -91,51 +91,42 @@ class IFailureRecoveryCoordinator(ABC):
**F10 provides low-level factor graph operations** (see F10 spec) **F10 provides low-level factor graph operations** (see F10 spec)
### Event-Based Communication ### Direct Call Communication
F11 emits events instead of directly calling F02's status update methods. This decouples recovery logic from flight state management. F11 is called directly by F02 and returns results synchronously. F02 is responsible for updating flight status based on F11's return values. This provides clear data flow and simpler debugging.
### Internal Events (Component-to-Component) **F02 calls F11 and handles responses**:
- `start_search()` → returns `SearchSession`, F02 updates status to "recovering"
- `try_current_grid()` → returns `Optional[AlignmentResult]`, F02 checks if found
- `mark_found()` → F02 updates status to "processing"
- `create_user_input_request()` → returns `UserInputRequest`, F02 updates status to "blocked" and sends via F15
- `apply_user_anchor()` → returns `bool`, F02 updates status to "processing" on success
- `create_chunk_on_tracking_loss()` → returns `ChunkHandle`, F02 tracks chunk
- `try_chunk_semantic_matching()` → returns `Optional[List[TileCandidate]]`
- `try_chunk_litesam_matching()` → returns `Optional[ChunkAlignmentResult]`
- `merge_chunk_to_trajectory()` → returns `bool`, F02 calls F14.update_results_after_chunk_merge()
F11 emits events for internal component communication. These are NOT directly sent to clients. **Note**: F11 does NOT emit events. All communication is through direct method calls with return values. F02 orchestrates the recovery flow by calling F11 methods and handling responses.
**Events Emitted (internal)**:
- `RecoveryStarted`: When tracking loss detected and recovery begins
- `RecoverySucceeded`: When recovery finds a match (single-image or chunk)
- `RecoveryFailed`: When all recovery strategies exhausted
- `UserInputNeeded`: When user input is required
- `UserFixApplied`: When user-provided anchor successfully applied
- `ChunkCreated`: When new chunk created on tracking loss
- `ChunkAnchored`: When chunk successfully matched and anchored
- `ChunkMerged`: When chunk merged into main trajectory (includes flight_id, chunk_id, merged_frames)
- `ChunkMatchingFailed`: When chunk matching exhausts all candidates and fails
**Event Listeners** (F02 Flight Processor subscribes to these):
- On `RecoveryStarted`: Update status to "recovering"
- On `RecoveryFailed`: Update status to "blocked"
- On `RecoverySucceeded`: Update status to "processing"
- On `UserInputNeeded`: Update status to "blocked", blocked=True
- On `ChunkMatchingFailed`: Re-queue chunk for user input or continue building
### External SSE Events (to Clients) ### External SSE Events (to Clients)
F11 does NOT directly send events to clients. External events are routed through F14 Result Manager which manages SSE streaming via F15: F11 does NOT directly send events to clients. F02 orchestrates client communication based on F11's return values:
- When F11 emits `UserInputNeeded` → F02 receives → F02 calls F14.publish_user_input_request() → F14 sends via F15 SSE - When F11.create_user_input_request() returns `UserInputRequest` → F02 calls F15.send_user_input_request()
- When F11 emits `ChunkMerged` → F02 receives → F02 calls F14.update_results_after_chunk_merge() → F14 sends via F15 SSE - When F11.merge_chunk_to_trajectory() returns True → F02 calls F14.update_results_after_chunk_merge()
- When F11 emits `RecoveryFailed` → F02 receives → F02 calls F14.publish_processing_blocked() → F14 sends via F15 SSE - When recovery fails (all F11 methods return None/False) → F02 calls F15.send_processing_blocked()
This separation ensures: This separation ensures:
1. F11 is decoupled from SSE implementation 1. F11 is a pure business logic component (no I/O dependencies)
2. F14 controls all client-facing communication 2. F02 controls all coordination and state management
3. Consistent event format for clients 3. F14/F15 handle all client-facing communication
### Scope ### Scope
- Confidence monitoring - Confidence monitoring
- Progressive search coordination - Progressive search coordination
- User input request/response handling - User input request/response handling
- Recovery strategy orchestration - Recovery strategy orchestration
- Integration point for G04, G06, G08, G09, F10 - Integration point for F04, F06, F08, F09, F10
- **Chunk lifecycle and matching coordination** - **Chunk lifecycle and matching coordination**
- **Multi-chunk simultaneous processing** - **Multi-chunk simultaneous processing**
@@ -245,7 +236,7 @@ SearchSession:
**Processing Flow**: **Processing Flow**:
1. Increment current_grid_size (1→4→9→16→25) 1. Increment current_grid_size (1→4→9→16→25)
2. Call G04.expand_search_grid() to get new tiles only 2. Call F04.expand_search_grid() to get new tiles only
3. Return new tile coordinates 3. Return new tile coordinates
**Test Cases**: **Test Cases**:
@@ -264,7 +255,7 @@ SearchSession:
**Input**: **Input**:
```python ```python
session: SearchSession session: SearchSession
tiles: Dict[str, np.ndarray] # From G04 tiles: Dict[str, np.ndarray] # From F04
``` ```
**Output**: `Optional[AlignmentResult]` - Match result or None **Output**: `Optional[AlignmentResult]` - Match result or None
@@ -779,14 +770,16 @@ class Sim3Transform(BaseModel):
scale: float scale: float
``` ```
### RecoveryEvent ### RecoveryStatus (Returned by recovery methods)
```python ```python
class RecoveryEvent(BaseModel): class RecoveryStatus(BaseModel):
event_type: str # "RecoveryStarted", "RecoverySucceeded", "RecoveryFailed", "UserInputNeeded", "ChunkCreated", "ChunkAnchored", "ChunkMerged" """Status returned by F11 methods, used by F02 to determine next action."""
flight_id: str success: bool
frame_id: Optional[int] method: str # "rotation_sweep", "progressive_search", "chunk_matching", "user_input"
gps: Optional[GPSPoint]
chunk_id: Optional[str] chunk_id: Optional[str]
data: Optional[Dict[str, Any]] message: Optional[str]
timestamp: datetime
``` ```
**Note**: F11 uses direct return values instead of events. F02 inspects return values to determine status updates and next actions.
@@ -27,11 +27,11 @@ class ICoordinateTransformer(ABC):
# Pixel/GPS Conversions # Pixel/GPS Conversions
@abstractmethod @abstractmethod
def pixel_to_gps(self, pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint: def pixel_to_gps(self, flight_id: str, pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint:
pass pass
@abstractmethod @abstractmethod
def gps_to_pixel(self, gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]: def gps_to_pixel(self, flight_id: str, gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]:
pass pass
@abstractmethod @abstractmethod
@@ -195,16 +195,17 @@ GPSPoint: WGS84 coordinates
### Pixel/GPS Conversions ### Pixel/GPS Conversions
### `pixel_to_gps(pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint` ### `pixel_to_gps(flight_id: str, pixel: Tuple[float, float], frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> GPSPoint`
**Description**: Converts pixel coordinates to GPS using camera pose and ground plane assumption. **Description**: Converts pixel coordinates to GPS using camera pose and ground plane assumption.
**Called By**: **Called By**:
- F13 Result Manager (for frame center GPS) - F02 Flight Processor (for frame center GPS)
- Internal (for image_object_to_gps) - Internal (for image_object_to_gps)
**Input**: **Input**:
```python ```python
flight_id: str # Flight identifier (required for ENU origin lookup)
pixel: Tuple[float, float] # (x, y) in image coordinates pixel: Tuple[float, float] # (x, y) in image coordinates
frame_pose: Pose # From Factor Graph (ENU coordinates) frame_pose: Pose # From Factor Graph (ENU coordinates)
camera_params: CameraParameters camera_params: CameraParameters
@@ -222,7 +223,7 @@ GPSPoint:
1. Unproject pixel to 3D ray using H01 Camera Model 1. Unproject pixel to 3D ray using H01 Camera Model
2. Intersect ray with ground plane at altitude 2. Intersect ray with ground plane at altitude
3. Transform 3D point from camera frame to ENU using frame_pose 3. Transform 3D point from camera frame to ENU using frame_pose
4. Convert ENU to WGS84 GPS using H06 Web Mercator Utils 4. Convert ENU to WGS84 GPS using enu_to_gps(flight_id, enu_point)
**Assumptions**: **Assumptions**:
- Ground plane assumption (terrain height negligible) - Ground plane assumption (terrain height negligible)
@@ -240,7 +241,7 @@ GPSPoint:
--- ---
### `gps_to_pixel(gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]` ### `gps_to_pixel(flight_id: str, gps: GPSPoint, frame_pose: Pose, camera_params: CameraParameters, altitude: float) -> Tuple[float, float]`
**Description**: Inverse projection from GPS to image pixel coordinates. **Description**: Inverse projection from GPS to image pixel coordinates.
@@ -250,6 +251,7 @@ GPSPoint:
**Input**: **Input**:
```python ```python
flight_id: str # Flight identifier (required for ENU origin lookup)
gps: GPSPoint gps: GPSPoint
frame_pose: Pose frame_pose: Pose
camera_params: CameraParameters camera_params: CameraParameters
@@ -262,7 +264,7 @@ Tuple[float, float]: (x, y) pixel coordinates
``` ```
**Algorithm**: **Algorithm**:
1. Convert GPS to ENU using H06 1. Convert GPS to ENU using gps_to_enu(flight_id, gps)
2. Transform ENU point to camera frame using frame_pose 2. Transform ENU point to camera frame using frame_pose
3. Project 3D point to image plane using H01 Camera Model 3. Project 3D point to image plane using H01 Camera Model
4. Return pixel coordinates 4. Return pixel coordinates
@@ -390,11 +392,12 @@ List[Tuple[float, float]]: Transformed points
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- **F10 Factor Graph Optimizer**: For frame poses - **F10 Factor Graph Optimizer**: For frame poses via get_trajectory(flight_id)
- **F17 Configuration Manager**: For camera parameters - **F17 Configuration Manager**: For camera parameters via get_flight_config(flight_id)
- **H01 Camera Model**: For projection operations - **H01 Camera Model**: For projection operations
- **H02 GSD Calculator**: For GSD calculations - **H02 GSD Calculator**: For GSD calculations
- **H06 Web Mercator Utils**: For coordinate conversions
**Note**: F13 uses internal enu_to_gps() and gps_to_enu() methods that rely on the ENU origin set via set_enu_origin(flight_id, origin_gps). H06 Web Mercator Utils is NOT used for ENU conversions - ENU is a local Cartesian coordinate system.
### External Dependencies ### External Dependencies
- **numpy**: Matrix operations - **numpy**: Matrix operations
@@ -41,7 +41,8 @@ class IResultManager(ABC):
- Store waypoint updates via F03 Flight Database - Store waypoint updates via F03 Flight Database
- Send incremental updates via F15 SSE Event Streamer - Send incremental updates via F15 SSE Event Streamer
- Maintain result versioning for audit trail - Maintain result versioning for audit trail
- Convert optimized poses to GPS coordinates
**Note**: F14 receives GPS coordinates directly from the caller (F02). F14 does NOT call F10 Factor Graph Optimizer or F13 Coordinate Transformer for pose retrieval/conversion. This ensures unidirectional data flow: F02 → F14 → F03/F15.
### Scope ### Scope
- Result state management - Result state management
@@ -153,11 +154,14 @@ frame_ids: List[int] # Frames with updated poses
**Output**: `bool` **Output**: `bool`
**Note**: F14 does NOT call F10 or F13. The caller (F02) performs the following steps before calling mark_refined():
1. F02 gets refined poses from F10.get_trajectory(flight_id)
2. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple)
3. F02 calls F14.mark_refined() with the GPS-converted results
**Processing Flow**: **Processing Flow**:
1. For each frame_id: 1. For each frame_id in frame_ids:
- Get refined pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position - Receive GPS coordinates from caller (already converted)
- Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])`
- Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint
- Update result with refined=True via F03 Flight Database.save_frame_result() - Update result with refined=True via F03 Flight Database.save_frame_result()
- Update waypoint via F03 Flight Database.update_waypoint() - Update waypoint via F03 Flight Database.update_waypoint()
- Call F15 SSE Event Streamer.send_refinement() - Call F15 SSE Event Streamer.send_refinement()
@@ -204,11 +208,15 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
**Output**: `bool` - True if updated successfully **Output**: `bool` - True if updated successfully
**Note**: F14 does NOT call F10 or F13. The caller (F02) performs these steps before calling update_results_after_chunk_merge():
1. F02 receives ChunkMerged event from F11 with merged_frames list
2. F02 gets updated poses from F10.get_trajectory(flight_id)
3. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple) for each frame
4. F02 calls F14.update_results_after_chunk_merge() with GPS-converted results
**Processing Flow**: **Processing Flow**:
1. For each frame_id in merged_frames: 1. For each frame_id in merged_frames:
- Get updated pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position - Receive GPS coordinates from caller (already converted)
- Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])`
- Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint
- Update frame result via F03 Flight Database.save_frame_result() - Update frame result via F03 Flight Database.save_frame_result()
- Update waypoint via F03 Flight Database.update_waypoint() - Update waypoint via F03 Flight Database.update_waypoint()
- Send refinement event via F15 SSE Event Streamer.send_refinement() - Send refinement event via F15 SSE Event Streamer.send_refinement()
@@ -256,10 +264,9 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
### Internal Components ### Internal Components
- **F03 Flight Database**: For waypoint and frame result persistence - **F03 Flight Database**: For waypoint and frame result persistence
- **F10 Factor Graph Optimizer**: For refined pose retrieval
- **F13 Coordinate Transformer**: For ENU to GPS conversion
- **F15 SSE Event Streamer**: For real-time result streaming - **F15 SSE Event Streamer**: For real-time result streaming
- **F11 Failure Recovery Coordinator**: Triggers chunk merge result updates
**Note**: F14 does NOT depend on F10, F13, or F11. The caller (F02) coordinates with those components and provides GPS-converted results to F14. This ensures unidirectional data flow and eliminates circular dependencies.
### External Dependencies ### External Dependencies
- None - None
@@ -96,7 +96,7 @@ StreamConnection:
**Description**: Sends frame_processed event. **Description**: Sends frame_processed event.
**Called By**: F13 Result Manager **Called By**: F14 Result Manager
**Event Format**: **Event Format**:
```json ```json
@@ -164,7 +164,7 @@ StreamConnection:
**Description**: Sends frame_refined event. **Description**: Sends frame_refined event.
**Called By**: F13 Result Manager **Called By**: F14 Result Manager
**Event Format**: **Event Format**:
```json ```json
@@ -241,3 +241,43 @@ class OperationalArea(BaseModel):
max_lon: float = 40.0 max_lon: float = 40.0
``` ```
### RecoveryConfig
```python
class RecoveryConfig(BaseModel):
"""Configuration for failure recovery and progressive search."""
search_grid_sizes: List[int] = [1, 4, 9, 16, 25] # Progressive tile search grid
min_chunk_frames_for_matching: int = 5 # Minimum frames before chunk matching
max_chunk_frames_for_matching: int = 20 # Maximum frames per chunk
user_input_threshold_tiles: int = 25 # Request user input after this many tiles
chunk_matching_interval_seconds: float = 5.0 # Background matching interval
confidence_threshold_good: float = 0.7 # Confidence for good tracking
confidence_threshold_degraded: float = 0.5 # Confidence for degraded tracking
min_inlier_count_good: int = 50 # Inliers for good tracking
min_inlier_count_tracking: int = 20 # Minimum inliers before tracking loss
```
### RotationConfig
```python
class RotationConfig(BaseModel):
"""Configuration for image rotation and heading tracking."""
rotation_step_degrees: float = 30.0 # Degrees per rotation step
litesam_max_rotation_tolerance: float = 45.0 # Max rotation LiteSAM handles
sharp_turn_threshold_degrees: float = 45.0 # Threshold for sharp turn detection
heading_history_size: int = 10 # Number of headings to track
confidence_threshold: float = 0.7 # For accepting rotation match
@property
def rotation_iterations(self) -> int:
"""Number of rotation steps (360 / step_degrees)."""
return int(360 / self.rotation_step_degrees)
```
## Dependencies
### Internal Components
- **F03 Flight Database**: For flight-specific configuration persistence (save_flight_config stores to F03)
### External Dependencies
- **pydantic**: Data model validation
- **PyYAML**: Configuration file parsing
+3 -1
View File
@@ -73,7 +73,7 @@
### Visual Processing ### Visual Processing
**F07_sequential_visual_odometry** **F07_sequential_visual_odometry**
**Interface**: `ISequentialVO` **Interface**: `ISequentialVisualOdometry`
**API**: `compute_relative_pose()`, `extract_features()`, `match_features()`, `estimate_motion()` **API**: `compute_relative_pose()`, `extract_features()`, `match_features()`, `estimate_motion()`
**F08_global_place_recognition** **F08_global_place_recognition**
@@ -319,6 +319,7 @@
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F11 (background) | F12 | `get_chunks_for_matching()` | Get unanchored chunks ready for matching | | F11 (background) | F12 | `get_chunks_for_matching()` | Get unanchored chunks ready for matching |
| F11 | F12 | `get_chunk_images()` | Get chunk images | | F11 | F12 | `get_chunk_images()` | Get chunk images |
| F12 | F05 | `get_image_by_sequence()` | **Load images for chunk** (F12 delegates to F05 for actual image retrieval) |
| F11 | F08 | `retrieve_candidate_tiles_for_chunk()` | Chunk semantic matching (aggregate DINOv2) | | F11 | F08 | `retrieve_candidate_tiles_for_chunk()` | Chunk semantic matching (aggregate DINOv2) |
| F08 | F16 | `get_inference_engine("DINOv2")` | Get model for descriptor computation | | F08 | F16 | `get_inference_engine("DINOv2")` | Get model for descriptor computation |
| F08 | H04 | `search()` | Query Faiss with chunk descriptor | | F08 | H04 | `search()` | Query Faiss with chunk descriptor |
@@ -348,6 +349,7 @@
**Drift correction**: F02→F04,F09,F10 **Drift correction**: F02→F04,F09,F10
**Tracking loss**: F11→F12(proactive chunk),F06,F08,F04(progressive),F09,F15,F02 **Tracking loss**: F11→F12(proactive chunk),F06,F08,F04(progressive),F09,F15,F02
**Chunk building**: F02→F12→F10,F07 **Chunk building**: F02→F12→F10,F07
**Chunk image retrieval**: F12→F05(get_image_by_sequence for chunk images)
**Chunk semantic matching**: F11→F12→F08(chunk descriptor) **Chunk semantic matching**: F11→F12→F08(chunk descriptor)
**Chunk LiteSAM matching**: F11→F06(chunk rotation)→F09(chunk alignment) **Chunk LiteSAM matching**: F11→F06(chunk rotation)→F09(chunk alignment)
**Chunk merging**: F11→F10(Sim3 transform) **Chunk merging**: F11→F10(Sim3 transform)
@@ -40,12 +40,22 @@ class IFaissIndexManager(ABC):
## Component Description ## Component Description
Manages Faiss indices for DINOv2 descriptor similarity search. H04 builds indexes from UAV image descriptors for: Manages Faiss indices for DINOv2 descriptor similarity search. H04 provides generic Faiss index operations used by:
1. **Loop closure detection**: Find when UAV revisits previously seen areas within the same flight
2. **Chunk-to-chunk matching**: Match disconnected chunks to each other
3. **Flight-to-flight matching**: Match current flight to previous flights in same area
**Index Source**: Descriptors are computed from UAV images using F08's DINOv2 encoder, NOT from satellite images. The index enables finding similar UAV viewpoints. ### Satellite Index (Primary Use Case)
- **Index format**: IVF1000 (Inverted File with 1000 clusters)
- **Index source**: Pre-built by external Satellite Provider (Maxar, Google Maps, Copernicus, etc.)
- **Index delivery**: Provider delivers index file + tile metadata when tiles are fetched on demand
- **Index updates**: Provider rebuilds index when new satellite tiles become available
- **Usage**: F08 Global Place Recognition loads this index via H04.load_index()
### UAV Index (Optional, Future Use)
For loop closure and chunk-to-chunk matching:
1. **Loop closure detection**: Find when UAV revisits previously seen areas
2. **Chunk-to-chunk matching**: Match disconnected chunks to each other
3. **Flight-to-flight matching**: Match current flight to previous flights
**Note**: H04 is a low-level utility that manages ANY Faiss index. It does NOT know whether the index contains satellite or UAV descriptors.
## API Methods ## API Methods
+1 -1
View File
@@ -16,7 +16,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
| F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search | | F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search |
| F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage | | F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage |
| F06 | Image Rotation Manager | `IImageRotationManager` | Rotation sweeps, heading tracking | | F06 | Image Rotation Manager | `IImageRotationManager` | Rotation sweeps, heading tracking |
| F07 | Sequential Visual Odometry | `ISequentialVO` | Frame-to-frame VO (SuperPoint+LightGlue) | | F07 | Sequential Visual Odometry | `ISequentialVisualOdometry` | Frame-to-frame VO (SuperPoint+LightGlue) |
| F08 | Global Place Recognition | `IGlobalPlaceRecognition` | Coarse localization (DINOv2+VLAD) | | F08 | Global Place Recognition | `IGlobalPlaceRecognition` | Coarse localization (DINOv2+VLAD) |
| F09 | Metric Refinement | `IMetricRefinement` | Precise alignment (LiteSAM) | | F09 | Metric Refinement | `IMetricRefinement` | Precise alignment (LiteSAM) |
| F10 | Factor Graph Optimizer | `IFactorGraphOptimizer` | GTSAM-based trajectory optimization | | F10 | Factor Graph Optimizer | `IFactorGraphOptimizer` | GTSAM-based trajectory optimization |