azaion/gps-denied-desktop
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-desktop.git synced 2026-04-22 07:06:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

component assesment and fixes done

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-30 16:09:31 +02:00
parent a172b21aac
commit ce9760fcbe
22 changed files with 681 additions and 1844 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -1 +1,2 @@
.DS_Store
.idea
docs/02_components/02_flight_processor/flight_lifecycle_manager_spec.md
+146
View File
@@ -0,0 +1,146 @@
# Flight Lifecycle Manager
## Interface Definition
**Interface Name**: `IFlightLifecycleManager`
### Interface Methods
```python
class IFlightLifecycleManager(ABC):
# Flight Lifecycle
@abstractmethod
def create_flight(self, flight_data: FlightData) -> str:
pass
@abstractmethod
def get_flight(self, flight_id: str) -> Optional[Flight]:
pass
@abstractmethod
def get_flight_state(self, flight_id: str) -> Optional[FlightState]:
pass
@abstractmethod
def delete_flight(self, flight_id: str) -> bool:
pass
@abstractmethod
def update_flight_status(self, flight_id: str, status: FlightStatusUpdate) -> bool:
pass
# Waypoint Management
@abstractmethod
def update_waypoint(self, flight_id: str, waypoint_id: str, waypoint: Waypoint) -> bool:
pass
@abstractmethod
def batch_update_waypoints(self, flight_id: str, waypoints: List[Waypoint]) -> BatchUpdateResult:
pass
@abstractmethod
def get_flight_metadata(self, flight_id: str) -> Optional[FlightMetadata]:
pass
# Validation
@abstractmethod
def validate_waypoint(self, waypoint: Waypoint) -> ValidationResult:
pass
@abstractmethod
def validate_geofence(self, geofence: Geofences) -> ValidationResult:
pass
@abstractmethod
def validate_flight_continuity(self, waypoints: List[Waypoint]) -> ValidationResult:
pass
# API Delegation Methods (called by F01)
@abstractmethod
def queue_images(self, flight_id: str, batch: ImageBatch) -> BatchQueueResult:
"""Delegates to F05 Image Input Pipeline and triggers F02.2 Processing Engine."""
pass
@abstractmethod
def handle_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResult:
"""Delegates to F02.2 Processing Engine to apply fix."""
pass
@abstractmethod
def create_client_stream(self, flight_id: str, client_id: str) -> StreamConnection:
"""Delegates to F15 SSE Event Streamer."""
pass
@abstractmethod
def convert_object_to_gps(self, flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint:
"""Delegates to F13 Coordinate Transformer."""
pass
# System Initialization
@abstractmethod
def initialize_system(self) -> bool:
pass
```
## Component Description
### Responsibilities
- **Component ID**: F02.1
- **Flight Lifecycle**: Manage flight creation, persistence, and deletion.
- **System Entry Point**: Main interface for the REST API (F01).
- **Resource Management**: Initializes system components and manages `F02.2 Flight Processing Engine` instances.
- **Data Validation**: Validates inputs before processing.
- **Satellite Prefetching**: Triggers F04 prefetching on flight creation.
### Scope
- CRUD operations for Flights and Waypoints.
- Coordination of system startup.
- Hand-off of active processing tasks to F02.2.
- Does **NOT** contain the processing loop or recovery logic.
## API Methods
### `create_flight(flight_data: FlightData) -> str`
**Description**: Creates a new flight, validates data, sets ENU origin, and triggers satellite prefetching.
**Called By**: F01 Flight API
**Processing Flow**:
1. Generate flight_id.
2. Validate geofences and waypoints.
3. Call F13.set_enu_origin(flight_id, start_gps).
4. Trigger F04.prefetch_route_corridor().
5. Save to F03 Flight Database.
6. Return flight_id.
### `queue_images(flight_id: str, batch: ImageBatch) -> BatchQueueResult`
**Description**: Queues images and ensures a Processing Engine is active for this flight.
**Processing Flow**:
1. Delegate to F05.queue_batch().
2. Retrieve or create instance of `F02.2 Flight Processing Engine` for this flight.
3. Trigger `engine.start_processing()` (async).
4. Return result.
### `handle_user_fix(flight_id: str, fix_data: UserFixRequest) -> UserFixResult`
**Description**: Forwards user fix to the active processing engine.
**Processing Flow**:
1. Get active engine for flight_id.
2. Call `engine.apply_user_fix(fix_data)`.
3. Return result.
### `initialize_system() -> bool`
**Description**: startup routine.
**Processing Flow**:
1. Load config (F17).
2. Initialize models (F16).
3. Initialize DB (F03).
4. Initialize Cache (F04).
5. Load Indexes (F08).
## Dependencies
- **F02.2 Flight Processing Engine**: Managed child component.
- **F03 Flight Database**: Persistence.
- **F04 Satellite Data Manager**: Prefetching.
- **F05 Image Input Pipeline**: Image queuing.
- **F13 Coordinate Transformer**: ENU origin setting.
- **F15 SSE Event Streamer**: Stream creation.
- **F17 Configuration Manager**: Config loading.
docs/02_components/02_flight_processor/flight_processing_engine_spec.md
+108
View File
@@ -0,0 +1,108 @@
# Flight Processing Engine
## Interface Definition
**Interface Name**: `IFlightProcessingEngine`
### Interface Methods
```python
class IFlightProcessingEngine(ABC):
@abstractmethod
def start_processing(self, flight_id: str) -> None:
"""Starts the main processing loop in a background thread/task."""
pass
@abstractmethod
def stop_processing(self, flight_id: str) -> None:
pass
@abstractmethod
def process_frame(self, flight_id: str, frame_id: int) -> FrameResult:
pass
@abstractmethod
def apply_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResult:
pass
@abstractmethod
def handle_tracking_loss(self, flight_id: str, frame_id: int) -> RecoveryStatus:
pass
# Chunk Lifecycle (Delegates to F12 but orchestrated here)
@abstractmethod
def get_active_chunk(self, flight_id: str) -> Optional[ChunkHandle]:
pass
@abstractmethod
def create_new_chunk(self, flight_id: str, frame_id: int) -> ChunkHandle:
pass
```
## Component Description
### Responsibilities
- **Component ID**: F02.2
- **Processing Orchestration**: Runs the main loop: Image -> VO -> Graph -> Result.
- **State Machine**: Manages flight status (Processing, Blocked, Recovering, Completed).
- **Recovery Coordination**: Calls F11 methods and acts on return values.
- **Chunk Management**: Calls F12 to manage chunk lifecycle based on tracking status.
- **Background Tasks**: Manages background chunk matching tasks.
### Scope
- Per-flight processing logic.
- Interaction with Visual Pipeline (F06, F07, F09).
- Interaction with State Estimation (F10).
- Interaction with Recovery (F11).
- Result Publishing (F14, F15).
## Processing Flow
### `run_processing_loop(flight_id: str)`
1. **Loop**: While `F05.get_next_image()` returns image:
2. **Chunk Check**:
* `active_chunk = F12.get_active_chunk()`
* If `detect_chunk_boundary()`: `active_chunk = F12.create_chunk()`.
3. **Visual Odometry**:
* Call `F06.requires_rotation_sweep()`.
* Call `F07.compute_relative_pose()`.
4. **Tracking Check**:
* If tracking good:
* `F12.add_frame_to_chunk()`.
* `F09.align_to_satellite()` (optional drift correction).
* `F10.add_relative_factor()`.
* `F10.optimize_chunk()`.
* `F14.update_frame_result()`.
* If tracking lost:
* **Proactive Chunking**: `F11.create_chunk_on_tracking_loss()`.
* **Recovery**: Call `handle_tracking_loss()`.
### `handle_tracking_loss(flight_id, frame_id)`
1. Call `F11.start_search()`.
2. Loop progressive search (1..25 tiles):
* `result = F11.try_current_grid()`.
* If found: `F11.mark_found()`, break.
* If not found: `F11.expand_search_radius()`.
3. If still not found:
* `req = F11.create_user_input_request()`.
* `F15.send_user_input_request(req)`.
* Set status to **BLOCKED**.
* Wait for `apply_user_fix`.
### `apply_user_fix(fix_data)`
1. Call `F11.apply_user_anchor(fix_data)`.
2. If success:
* Set status to **PROCESSING**.
* Resume loop.
## Dependencies
- **F05 Image Input Pipeline**: Image source.
- **F06 Image Rotation Manager**: Pre-processing.
- **F07 Sequential Visual Odometry**: Motion estimation.
- **F09 Metric Refinement**: Satellite alignment.
- **F10 Factor Graph Optimizer**: State estimation.
- **F11 Failure Recovery Coordinator**: Recovery logic.
- **F12 Route Chunk Manager**: Chunk state.
- **F14 Result Manager**: Saving results.
- **F15 SSE Event Streamer**: Real-time updates.
docs/02_components/02_flight_processor/flight_processor_spec.md
-1285
View File
File diff suppressed because it is too large Load Diff
docs/02_components/03_flight_database/flight_database_spec.md
+30 -15
View File
@@ -8,6 +8,12 @@
```python
class IFlightDatabase(ABC):
# Transaction Support
@abstractmethod
def execute_transaction(self, operations: List[Callable]) -> bool:
"""Executes a list of DB operations atomically."""
pass
# Flight Operations
@abstractmethod
def insert_flight(self, flight: Flight) -> str:
@@ -111,6 +117,8 @@ class IFlightDatabase(ABC):
## Component Description
### Responsibilities
- Persistence layer.
- **Consistency**: Implements explicit transaction support to ensure `waypoints` and `frame_results` (and other related tables) stay synchronized when updated by F14 Result Manager.
- Direct database access layer for all flight-related data
- Execute SQL queries and commands
- Manage database connections and transactions
@@ -120,6 +128,10 @@ class IFlightDatabase(ABC):
- Store heading history for rotation management
- Store image file paths and metadata
### Transactional Integrity
- **Atomic Updates**: Critical for preventing partial state updates during chunk merges or frame refinements.
- **Pattern**: Usage of database transactions (BEGIN, COMMIT, ROLLBACK) for batch operations.
### Scope
- CRUD operations on flights table
- CRUD operations on waypoints table
@@ -154,7 +166,7 @@ The schema uses strategic denormalization to optimize for the most common access
**Description**: Inserts a new flight with initial waypoints and geofences.
**Called By**:
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager
**Input**:
```python
@@ -201,7 +213,7 @@ flight_id: str
**Description**: Updates flight metadata.
**Called By**:
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager
**Input**:
```python
@@ -231,7 +243,7 @@ WHERE id = ?
**Description**: Queries flights with filtering and pagination.
**Called By**:
- F02 Flight Processor (listing)
- F02.1 Flight Lifecycle Manager (listing)
- F01 Flight API
**Input**:
@@ -258,7 +270,7 @@ List[Flight] # Metadata only, without full waypoint data
**Description**: Retrieves complete flight with all waypoints.
**Called By**:
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager
**Input**:
```python
@@ -288,7 +300,7 @@ Optional[Flight] # Complete flight with all waypoints
**Description**: Deletes a flight and cascades to all related data.
**Called By**:
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager
**Input**:
```python
@@ -321,7 +333,7 @@ DELETE FROM flights WHERE id = ?
**Description**: Retrieves waypoints for a flight.
**Called By**:
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager
**Input**:
```python
@@ -345,7 +357,7 @@ List[Waypoint]
**Description**: Inserts a new waypoint.
**Called By**:
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager
**Input**:
```python
@@ -369,7 +381,7 @@ waypoint_id: str
**Description**: Updates a waypoint. Critical path for GPS refinement updates.
**Called By**:
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager
- F14 Result Manager
**Input**:
@@ -408,7 +420,7 @@ WHERE id = ? AND flight_id = ?
**Description**: Updates multiple waypoints in a single transaction.
**Called By**:
- F02 Flight Processor (asynchronous refinements)
- F02.2 Flight Processing Engine (asynchronous refinements)
**Input**:
```python
@@ -437,7 +449,7 @@ BatchResult:
**Description**: Saves or updates flight processing state.
**Called By**:
- F02 Flight Processor
- F02.2 Flight Processing Engine
**Input**:
```python
@@ -447,13 +459,14 @@ FlightState:
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
created_at: datetime
updated_at: datetime
```
**Note**: Heading is NOT stored in flight_state. Use the dedicated `heading_history` table via `save_heading()` and `get_latest_heading()` for all heading operations. This avoids data duplication and ensures a single source of truth for heading data.
**Output**:
```python
bool: True if saved
@@ -470,7 +483,7 @@ bool: True if saved
**Description**: Loads flight state (for crash recovery).
**Called By**:
- F02 Flight Processor
- F02.2 Flight Processing Engine
**Output**:
```python
@@ -548,7 +561,7 @@ bool: True if saved
**Description**: Saves heading value for temporal smoothing and recovery.
**Called By**:
- F02 Flight Processor (after F06.update_heading() returns, F02 persists to F03)
- F02.2 Flight Processing Engine (after F06.update_heading() returns, F02 persists to F03)
**Input**:
```python
@@ -884,13 +897,14 @@ CREATE TABLE geofences (
);
-- Flight state table
-- NOTE: Heading is NOT stored here. Use heading_history table for heading data.
-- This avoids duplication and ensures single source of truth.
CREATE TABLE flight_state (
flight_id VARCHAR(36) PRIMARY KEY,
status VARCHAR(50) NOT NULL,
frames_processed INT NOT NULL DEFAULT 0,
frames_total INT NOT NULL DEFAULT 0,
current_frame INT,
current_heading FLOAT,
blocked BOOLEAN NOT NULL DEFAULT FALSE,
search_grid_size INT,
created_at TIMESTAMP NOT NULL,
@@ -993,11 +1007,12 @@ class FlightState(BaseModel):
frames_processed: int
frames_total: int
current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool
search_grid_size: Optional[int]
created_at: datetime
updated_at: datetime
# NOTE: Heading is NOT stored in FlightState.
# Use get_latest_heading(flight_id) from heading_history table.
```
### FrameResult
docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md
+2 -2
View File
@@ -167,7 +167,7 @@ GET /api/satellite/tiles/batch?tiles=[...]
**Description**: Prefetches satellite tiles along route corridor for a flight.
**Called By**:
- F02 Flight Manager (during flight creation)
- F02.1 Flight Lifecycle Manager (during flight creation)
**Input**:
```python
@@ -454,7 +454,7 @@ TileBounds:
**Description**: Clears cached tiles for a completed flight.
**Called By**:
- F02 Flight Processor (cleanup after flight completion)
- F02.1 Flight Lifecycle Manager (cleanup after flight completion)
**Input**:
```python
docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md
+7 -7
View File
@@ -65,7 +65,7 @@ class IImageInputPipeline(ABC):
**Description**: Queues a batch of images for processing (FIFO).
**Called By**:
- F01 GPS-Denied REST API (after upload)
- F02.1 Flight Lifecycle Manager (via F01 Flight API image upload route)
**Input**:
```python
@@ -224,9 +224,9 @@ bool: True if stored successfully
**Description**: Gets the next image in sequence for processing.
**Called By**:
- F06 Image Rotation Manager
- F07 Sequential VO
- Processing pipeline (main loop)
- F02.2 Flight Processing Engine (main processing loop)
- F06 Image Rotation Manager (via F02.2)
- F07 Sequential VO (via F02.2)
**Input**:
```python
@@ -299,7 +299,7 @@ Optional[ImageData]
**Description**: Retrieves metadata without loading full image (lightweight).
**Called By**:
- F02 Flight Manager (status checks)
- F02.1 Flight Lifecycle Manager (status checks)
- F13 Result Manager (metadata-only queries)
**Input**:
@@ -330,8 +330,8 @@ ImageMetadata:
**Description**: Gets current processing status for a flight.
**Called By**:
- F01 Flight API (status endpoint)
- F02 Flight Processor
- F02.1 Flight Lifecycle Manager (status queries via F01 Flight API)
- F02.2 Flight Processing Engine (processing loop status)
**Input**:
```python
docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md
+29 -19
View File
@@ -13,7 +13,11 @@ class IImageRotationManager(ABC):
pass
@abstractmethod
def try_rotation_steps(self, flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime) -> Optional[RotationResult]:
def try_rotation_steps(self, flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime, matcher: IImageMatcher) -> Optional[RotationResult]:
"""
Performs rotation sweep.
'matcher' is an injected dependency (usually F09) to avoid direct coupling.
"""
pass
@abstractmethod
@@ -41,22 +45,27 @@ class IImageRotationManager(ABC):
pass
@abstractmethod
def try_chunk_rotation_steps(self, chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[RotationResult]:
def try_chunk_rotation_steps(self, chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds, matcher: IImageMatcher) -> Optional[RotationResult]:
pass
```
## Component Description
### Responsibilities
- Handle UAV image rotation preprocessing
- **Critical**: LiteSAM (F09 Metric Refinement) fails if images rotated >45°, requires preprocessing
- Perform 30° step rotation sweeps (12 rotations: 0°, 30°, 60°, ..., 330°)
- Track UAV heading angle across flight
- Calculate precise rotation angle from homography point correspondences
- Detect sharp turns requiring rotation sweep
- Pre-rotate images to known heading for subsequent frames
- **Chunk rotation operations (rotate all images in chunk)**
- **Chunk rotation sweeps (delegates matching to F09 Metric Refinement)**
- Image rotation utility.
- Heading tracking.
- Coordination of rotation sweeps.
### Decoupling Fix
- **Problem**: F06 previously depended directly on `F09 Metric Refinement`.
- **Fix**: Methods `try_rotation_steps` and `try_chunk_rotation_steps` now accept a `matcher` argument conforming to `IImageMatcher`.
- **IImageMatcher Interface**:
```python
class IImageMatcher(ABC):
def align_to_satellite(self, uav_image, satellite_tile, tile_bounds) -> AlignmentResult: pass
def align_chunk_to_satellite(self, chunk_images, satellite_tile, tile_bounds) -> ChunkAlignmentResult: pass
```
- **Runtime**: F02.2 injects F09 instance when calling F06 methods.
### Scope
- Image rotation operations (pure rotation, no matching)
@@ -103,7 +112,7 @@ np.ndarray # Rotated image (same dimensions)
---
### `try_rotation_steps(flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime) -> Optional[RotationResult]`
### `try_rotation_steps(flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime, matcher: IImageMatcher) -> Optional[RotationResult]`
**Description**: Performs 30° rotation sweep, rotating image at each step and delegating matching to F09 Metric Refinement.
@@ -119,6 +128,7 @@ image: np.ndarray # UAV image
satellite_tile: np.ndarray # Satellite reference tile
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (passed to F09)
timestamp: datetime # Timestamp for heading persistence
matcher: IImageMatcher # Injected matcher (F09)
```
**About tile_bounds**: `TileBounds` contains the GPS bounding box of the satellite tile:
@@ -142,7 +152,7 @@ RotationResult:
```
For angle in [0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°]:
rotated_image = rotate_image_360(image, angle)
result = F09.align_to_satellite(rotated_image, satellite_tile, tile_bounds)
result = matcher.align_to_satellite(rotated_image, satellite_tile, tile_bounds)
if result.matched and result.confidence > threshold:
precise_angle = calculate_precise_angle(result.homography, angle)
update_heading(flight_id, frame_id, precise_angle, timestamp)
@@ -153,7 +163,7 @@ return None # No match found
**Processing Flow**:
1. For each 30° step:
- Rotate image via rotate_image_360()
- Call F09 Metric Refinement.align_to_satellite(rotated_image, satellite_tile, tile_bounds)
- Call matcher.align_to_satellite(rotated_image, satellite_tile, tile_bounds)
- Check if match found
2. If match found:
- Calculate precise angle from homography via calculate_precise_angle()
@@ -379,7 +389,7 @@ List[np.ndarray] # Rotated images (same dimensions)
---
### `try_chunk_rotation_steps(chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[RotationResult]`
### `try_chunk_rotation_steps(chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds, matcher: IImageMatcher) -> Optional[RotationResult]`
**Description**: Performs 30° rotation sweep on entire chunk, rotating all images at each step and delegating matching to F09 Metric Refinement.
@@ -391,6 +401,7 @@ List[np.ndarray] # Rotated images (same dimensions)
chunk_images: List[np.ndarray] # Chunk images
satellite_tile: np.ndarray # Reference satellite tile
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (for F09)
matcher: IImageMatcher # Injected matcher
```
**Output**:
@@ -407,7 +418,7 @@ RotationResult:
```
For angle in [0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°]:
rotated_chunk = rotate_chunk_360(chunk_images, angle)
result = F09.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds)
result = matcher.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds)
if result.matched and result.confidence > threshold:
precise_angle = calculate_precise_angle(result.homography, angle)
return RotationResult(matched=True, initial_angle=angle, precise_angle=precise_angle, ...)
@@ -417,7 +428,7 @@ return None # No match found
**Processing Flow**:
1. For each 30° step:
- Rotate all chunk images via rotate_chunk_360()
- Call F09 Metric Refinement.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds)
- Call matcher.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds)
- Check if match found
2. If match found:
- Calculate precise angle from homography via calculate_precise_angle()
@@ -497,8 +508,8 @@ return None # No match found
## Dependencies
### Internal Components
- **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching.
- **H07 Image Rotation Utils**: For image rotation and angle calculations
- **Injected Matcher (F09)**.
**Note**:
- `TileBounds` data model is imported from F09 Metric Refinement.
@@ -541,4 +552,3 @@ class RotationConfig(BaseModel):
confidence_threshold: float = 0.7 # For accepting match
history_size: int = 10 # Number of headings to track
```
docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md
+24 -9
View File
@@ -25,7 +25,7 @@ class ISequentialVisualOdometry(ABC):
pass
```
**Note**: F07 is chunk-agnostic. It only computes relative poses between images. The caller (F02 Flight Processor) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
**Note**: F07 is chunk-agnostic. It only computes relative poses between images. The caller (F02.2 Flight Processing Engine) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
## Component Description
@@ -42,7 +42,7 @@ class ISequentialVisualOdometry(ABC):
- Feature-based motion estimation
- Handles low overlap and challenging agricultural environments
- Provides relative measurements for trajectory optimization
- **Chunk-agnostic**: F07 doesn't know about chunks. Caller (F02) routes results to appropriate chunk subgraph.
- **Chunk-agnostic**: F07 doesn't know about chunks. Caller (F02.2) routes results to appropriate chunk subgraph.
## API Methods
@@ -213,6 +213,21 @@ Motion:
- GSD-based expected displacement calculations (via H02)
- Absolute GPS anchors from F09 Metric Refinement
**Critical Handoff to F10**:
The caller (F02.2) must pass the unit translation to F10 for scale resolution:
```python
# F02.2 receives RelativePose from F07
vo_result = F07.compute_relative_pose(prev_image, curr_image)
# vo_result.translation is a UNIT VECTOR (||t|| = 1)
# F02.2 passes to F10 which scales using:
# 1. altitude = F17.get_operational_altitude(flight_id)
# 2. gsd = H02.compute_gsd(altitude, camera_params)
# 3. expected_displacement = frame_spacing * gsd
# 4. scaled_translation = vo_result.translation * expected_displacement
F10.add_relative_factor(flight_id, frame_i, frame_j, vo_result, covariance)
```
**Error Conditions**:
- Returns `None`: Insufficient inliers (< 8 points for Essential Matrix)
@@ -246,12 +261,12 @@ Motion:
2. compute_relative_pose() → SuperPoint handles better than SIFT
3. Verify match quality
### Test 5: Chunk-Aware VO
1. Create chunk_1 and chunk_2
2. compute_relative_pose_in_chunk() for frames in chunk_1
3. compute_relative_pose_in_chunk() for frames in chunk_2
4. Verify factors added to respective chunks
5. Verify chunks optimized independently
### Test 5: VO with Chunk Routing
1. Create chunk_1 and chunk_2 via F12
2. compute_relative_pose() for frames in chunk_1, F02.2 routes to chunk_1
3. compute_relative_pose() for frames in chunk_2, F02.2 routes to chunk_2
4. Verify F02.2 calls F12.add_frame_to_chunk() with correct chunk_id
5. Verify chunks optimized independently via F10
## Non-Functional Requirements
@@ -280,7 +295,7 @@ Motion:
- **H01 Camera Model**: For coordinate normalization
- **H05 Performance Monitor**: For timing measurements
**Note**: F07 is chunk-agnostic and does NOT depend on F10 Factor Graph Optimizer. F07 only computes relative poses between images and returns them to the caller (F02). The caller (F02) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
**Note**: F07 is chunk-agnostic and does NOT depend on F10 Factor Graph Optimizer. F07 only computes relative poses between images and returns them to the caller (F02.2). The caller (F02.2) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
### External Dependencies
- **SuperPoint**: Feature extraction model
docs/02_components/08_global_place_recognition/global_place_recognition_spec.md
+5 -5
View File
@@ -208,7 +208,7 @@ List[TileCandidate] # Re-ranked list
**Description**: Loads pre-built satellite descriptor database from file. **Note**: The semantic index (DINOv2 descriptors + Faiss index) MUST be provided by the satellite data provider. F08 does NOT build the index - it only loads it.
**Called By**:
- F02 Flight Processor (during flight initialization, index_path from F04 Satellite Data Manager)
- F02.1 Flight Lifecycle Manager (during flight initialization, index_path from F04 Satellite Data Manager)
**Input**:
```python
@@ -380,10 +380,10 @@ np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
## Dependencies
### Internal Components
- **F16 Model Manager**: For DINOv2 model
- **H04 Faiss Index Manager**: For similarity search
- **F04 Satellite Data Manager**: For tile metadata
- **F12 Route Chunk Manager**: For chunk image retrieval
- **F16 Model Manager**: For DINOv2 inference engine via `get_inference_engine("DINOv2")`.
- **H04 Faiss Index Manager**: For similarity search via `load_index()`, `search()`. Critical for `query_database()`.
- **F04 Satellite Data Manager**: For tile metadata retrieval after Faiss search returns tile indices.
- **F12 Route Chunk Manager**: For chunk image retrieval during chunk descriptor computation.
### External Dependencies
- **DINOv2**: Foundation vision model
docs/02_components/09_metric_refinement/metric_refinement_spec.md
+2 -2
View File
@@ -62,7 +62,7 @@ class IMetricRefinement(ABC):
**Called By**:
- F06 Image Rotation Manager (during rotation sweep)
- F11 Failure Recovery Coordinator (progressive search)
- F02 Flight Processor (drift correction with single tile)
- F02.2 Flight Processing Engine (drift correction with single tile)
**Input**:
```python
@@ -389,7 +389,7 @@ Optional[np.ndarray]: 3×3 homography matrix or None
- If rotation >45°, F09 will fail to match (by design)
- F06 handles the rotation sweep (trying 0°, 30°, 60°, etc.) and calls F09 for each rotation
**Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager)
**Note**: tile_bounds is passed as parameter from caller (F02.2 Flight Processing Engine gets it from F04 Satellite Data Manager)
### External Dependencies
- **LiteSAM**: Cross-view matching model
docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md
+13 -2
View File
@@ -8,8 +8,19 @@
```python
class IFactorGraphOptimizer(ABC):
# All methods take flight_id to support concurrent flights
# F10 maintains Dict[str, FactorGraph] keyed by flight_id internally
"""
GTSAM-based factor graph optimizer for trajectory estimation.
## Multi-Flight Support
All state-modifying methods require `flight_id` parameter.
Each flight maintains an independent factor graph state.
F10 internally manages Dict[str, FactorGraph] keyed by flight_id.
This enables:
- Concurrent processing of multiple flights
- Flight-scoped optimization without cross-contamination
- Independent cleanup via delete_flight_graph()
"""
@abstractmethod
def add_relative_factor(self, flight_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md
+44 -63
View File
@@ -8,6 +8,7 @@
```python
class IFailureRecoveryCoordinator(ABC):
# Status Checks
@abstractmethod
def check_confidence(self, vo_result: RelativePose, litesam_result: Optional[AlignmentResult]) -> ConfidenceAssessment:
pass
@@ -16,6 +17,7 @@ class IFailureRecoveryCoordinator(ABC):
def detect_tracking_loss(self, confidence: ConfidenceAssessment) -> bool:
pass
# Search & Recovery (Synchronous returns, NO EVENTS)
@abstractmethod
def start_search(self, flight_id: str, frame_id: int, estimated_gps: GPSPoint) -> SearchSession:
pass
@@ -38,12 +40,14 @@ class IFailureRecoveryCoordinator(ABC):
@abstractmethod
def create_user_input_request(self, flight_id: str, frame_id: int, candidate_tiles: List[TileCandidate]) -> UserInputRequest:
"""Returns request object. Caller (F02.2) is responsible for sending to F15."""
pass
@abstractmethod
def apply_user_anchor(self, flight_id: str, frame_id: int, anchor: UserAnchor) -> bool:
pass
# Chunk Recovery
@abstractmethod
def create_chunk_on_tracking_loss(self, flight_id: str, frame_id: int) -> ChunkHandle:
pass
@@ -62,63 +66,52 @@ class IFailureRecoveryCoordinator(ABC):
@abstractmethod
def process_unanchored_chunks(self, flight_id: str) -> None:
"""Background task logic. Should be invoked/managed by F02.2."""
pass
```
## Component Description
### Responsibilities
- **Pure Logic Component**: Coordinates recovery strategies but delegates execution and communication to F02.2.
- **No Events**: Returns status objects or booleans. F02.2 decides state transitions based on these returns.
- **Chunk Orchestration**: Coordinates F12 and F10 operations during recovery but does not own the thread/task.
- Monitor confidence metrics (inlier count, MRE, covariance)
- Detect tracking loss and trigger recovery
- Coordinate progressive tile search (1→4→9→16→25)
- Handle human-in-the-loop when all strategies exhausted
- **Emit recovery events** (RecoveryStarted, RecoveryFailed, RecoverySucceeded, UserInputNeeded)
- Apply user-provided anchors to Factor Graph
- **Proactive chunk creation on tracking loss** (via F12)
- **Chunk LiteSAM matching with rotation sweeps**
- **Chunk merging orchestration**
- **Background chunk matching processing**
### Chunk Responsibility Clarification
### Interaction Pattern (Direct Calls)
**Caller**: F02.2 Flight Processing Engine
**F11 coordinates chunk-based recovery workflow only**:
- Detects when to create chunks (on tracking loss)
- Calls F12 for ALL chunk operations (F11 NEVER directly calls F10 chunk methods)
- Coordinates matching workflows
- Emits chunk-related events (ChunkCreated, ChunkAnchored, ChunkMerged)
1. **Tracking Loss**:
* F02.2 calls `start_search()`.
* F02.2 calls `try_current_grid()` iteratively.
* If found, F02.2 calls `mark_found()`.
* If exhausted, F02.2 calls `create_user_input_request()` -> gets `Request` object -> F02.2 calls F15.
**F12 is the source of truth for chunk state** (see F12 spec)
**F10 provides low-level factor graph operations** (see F10 spec)
### Direct Call Communication
F11 is called directly by F02 and returns results synchronously. F02 is responsible for updating flight status based on F11's return values. This provides clear data flow and simpler debugging.
**F02 calls F11 and handles responses**:
- `start_search()` → returns `SearchSession`, F02 updates status to "recovering"
- `try_current_grid()` → returns `Optional[AlignmentResult]`, F02 checks if found
- `mark_found()` → F02 updates status to "processing"
- `create_user_input_request()` → returns `UserInputRequest`, F02 updates status to "blocked" and sends via F15
- `apply_user_anchor()` → returns `bool`, F02 updates status to "processing" on success
- `create_chunk_on_tracking_loss()` → returns `ChunkHandle`, F02 tracks chunk
- `try_chunk_semantic_matching()` → returns `Optional[List[TileCandidate]]`
- `try_chunk_litesam_matching()` → returns `Optional[ChunkAlignmentResult]`
- `merge_chunk_to_trajectory()` → returns `bool`, F02 calls F14.update_results_after_chunk_merge()
**Note**: F11 does NOT emit events. All communication is through direct method calls with return values. F02 orchestrates the recovery flow by calling F11 methods and handling responses.
2. **Chunk Matching**:
* F02.2 calls `create_chunk_on_tracking_loss()`.
* F02.2 (background task) calls `try_chunk_semantic_matching()`.
* F02.2 calls `try_chunk_litesam_matching()`.
* F02.2 calls `merge_chunk_to_trajectory()`.
### External SSE Events (to Clients)
F11 does NOT directly send events to clients. F02 orchestrates client communication based on F11's return values:
F11 does NOT directly send events to clients. F02.2 orchestrates client communication based on F11's return values:
- When F11.create_user_input_request() returns `UserInputRequest` → F02 calls F15.send_user_input_request()
- When F11.merge_chunk_to_trajectory() returns True → F02 calls F14.update_results_after_chunk_merge()
- When recovery fails (all F11 methods return None/False) → F02 calls F15.send_processing_blocked()
- When F11.create_user_input_request() returns `UserInputRequest` → F02.2 calls F15.send_user_input_request()
- When F11.merge_chunk_to_trajectory() returns True → F02.2 calls F14.update_results_after_chunk_merge()
- When recovery fails (all F11 methods return None/False) → F02.2 calls F15.send_processing_blocked()
This separation ensures:
1. F11 is a pure business logic component (no I/O dependencies)
2. F02 controls all coordination and state management
2. F02.2 controls all coordination and state management
3. F14/F15 handle all client-facing communication
### Scope
@@ -341,12 +334,11 @@ UserInputRequest:
1. Get UAV image for frame_id
2. Get top-5 candidates from F08
3. Create request
4. Send via F15 SSE → "user_input_needed" event
5. Emit `UserInputNeeded` event (F02 subscribes and updates status to "BLOCKED")
4. Return Request object (Caller F02.2 handles sending)
**Test Cases**:
1. All search failed → creates request
2. Request sent to client via SSE
2. Request returned to caller
---
@@ -371,8 +363,7 @@ anchor: UserAnchor:
1. Validate anchor data
2. Call F10.add_absolute_factor(frame_id, gps, is_user_anchor=True)
3. F10.optimize() → refines trajectory
4. Emit `UserFixApplied` event (F02 subscribes and updates status to "PROCESSING")
5. Resume processing from next frame
4. Return True (Caller F02.2 updates status)
**Test Cases**:
1. Valid anchor → applied, processing resumes
@@ -385,7 +376,7 @@ anchor: UserAnchor:
**Description**: Creates a new chunk proactively when tracking is lost.
**Called By**:
- F02 Flight Processor (when tracking lost detected)
- F02.2 Flight Processing Engine (when tracking lost detected)
**Input**:
```python
@@ -535,9 +526,7 @@ bool: True if merge successful
- chunk_id (source) is merged into target_chunk_id
6. F12 handles chunk state updates (deactivation, status updates)
7. F10 optimizes merged graph globally (via F12.merge_chunks())
8. **Emit ChunkMerged event** with flight_id and merged_frames
- F14 subscribes and updates results: retrieves poses from F10, converts ENU to GPS, updates database, publishes via SSE
9. Return True
8. Return True (Caller F02.2 coordinates result updates)
**Sim(3) Transform**:
- Translation: GPS offset
@@ -545,10 +534,9 @@ bool: True if merge successful
- Scale: Resolved from altitude and GSD
**Test Cases**:
1. **Merge successful**: Chunks merged successfully, results updated via F14
1. **Merge successful**: Chunks merged successfully
2. **Global consistency**: Merged trajectory globally consistent
3. **Multiple chunks**: Can merge multiple chunks sequentially
4. **Result updates**: All merged frames have updated GPS coordinates published
---
@@ -558,7 +546,7 @@ bool: True if merge successful
**Called By**:
- Background thread (periodic, e.g., every 5 seconds)
- F02 Flight Processor (after frame processing)
- F02.2 Flight Processing Engine (after frame processing)
**Input**:
```python
@@ -583,7 +571,7 @@ while flight_active:
```
**Background Processing**:
- **Trigger**: Started by F02 Flight Processor after flight creation
- **Trigger**: Started by F02.2 Flight Processing Engine after flight creation
- Runs asynchronously, doesn't block frame processing
- Periodically checks for ready chunks (every 5 seconds)
- Attempts matching and merging
@@ -592,15 +580,15 @@ while flight_active:
**Chunk Failure Flow**:
When chunk matching fails after trying all candidates:
1. Emit `ChunkMatchingFailed` event with chunk_id and flight_id
2. F02 receives event and decides next action:
1. Emit `ChunkMatchingFailed` event with chunk_id and flight_id (to F02.2 internal queue)
2. F02.2 receives event and decides next action:
- **Option A**: Wait for more frames and retry later (chunk may gain more distinctive features)
- **Option B**: Create user input request for the chunk
3. If user input requested:
- F02 calls F14.publish_user_input_request() with chunk context
- F02.2 calls F15.send_user_input_request() with chunk context
- Client receives via SSE
- User provides GPS anchor for one frame in chunk
- F02 receives user fix via handle_user_fix()
- F02.2 receives user fix via handle_user_fix()
- F11.apply_user_anchor() anchors the chunk
- Processing resumes
@@ -608,7 +596,7 @@ When chunk matching fails after trying all candidates:
1. **Background matching**: Unanchored chunks matched asynchronously
2. **Chunk merging**: Chunks merged when matches found
3. **Non-blocking**: Frame processing continues during matching
4. **Chunk matching fails**: ChunkMatchingFailed emitted, user input requested if needed
4. **Chunk matching fails**: Failure handled, user input requested if needed
## Integration Tests
@@ -676,16 +664,12 @@ When chunk matching fails after trying all candidates:
## Dependencies
### Internal Components
- F04 Satellite Data Manager (tile grids and tile_bounds computation)
- F06 Image Rotation Manager (rotation sweep and chunk rotation)
- F08 Global Place Recognition (candidates and chunk semantic matching)
- F09 Metric Refinement (LiteSAM and chunk LiteSAM matching)
- F10 Factor Graph Optimizer (anchor application and chunk merging)
- F12 Route Chunk Manager (chunk lifecycle)
- F14 Result Manager (result updates after chunk merging)
- F15 SSE Event Streamer (user input events)
**Note**: F11 does NOT directly call F02. Instead, F11 emits events (RecoveryStarted, RecoveryFailed, etc.) which F02 subscribes to for status updates. This decouples recovery logic from flight state management.
- **F12 Route Chunk Manager**: Chunk operations.
- **F10 Factor Graph Optimizer**: Anchor application.
- **F04 Satellite Data Manager**: Search grids.
- **F08 Global Place Recognition**: Candidate retrieval.
- **F09 Metric Refinement**: Alignment.
- **F06 Image Rotation Manager**: Rotation sweeps.
### External Dependencies
- None
@@ -780,6 +764,3 @@ class RecoveryStatus(BaseModel):
chunk_id: Optional[str]
message: Optional[str]
```
**Note**: F11 uses direct return values instead of events. F02 inspects return values to determine status updates and next actions.
docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md
+57 -254
View File
@@ -38,6 +38,12 @@ class IRouteChunkManager(ABC):
@abstractmethod
def mark_chunk_anchored(self, chunk_id: str, frame_id: int, gps: GPSPoint) -> bool:
"""
Transactional update:
1. Calls F10.add_chunk_anchor().
2. IF success: Updates internal state to 'anchored'.
3. Returns success status.
"""
pass
@abstractmethod
@@ -54,7 +60,12 @@ class IRouteChunkManager(ABC):
@abstractmethod
def merge_chunks(self, target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool:
"""Merges source_chunk INTO target_chunk. Result is stored in target_chunk."""
"""
Transactional update:
1. Calls F10.merge_chunk_subgraphs().
2. IF success: Updates internal state (source merged/deactivated).
3. Returns success status.
"""
pass
@abstractmethod
@@ -63,40 +74,23 @@ class IRouteChunkManager(ABC):
@abstractmethod
def save_chunk_state(self, flight_id: str) -> bool:
"""Persist all chunk state to F03 for crash recovery."""
pass
@abstractmethod
def load_chunk_state(self, flight_id: str) -> bool:
"""Load chunk state from F03 on restart."""
pass
```
## Component Description
### Responsibilities
- **Source of truth for chunk state** (active, anchored, merged status)
- Manage chunk lifecycle (creation, activation, deactivation, merging)
- Track chunk state (frames, anchors, matching status)
- Provide chunk representations for matching (composite images, descriptors)
- Determine chunk readiness for matching (min frames, consistency)
- Persist chunk state via F03 Flight Database
- **Source of Truth**: Manages chunk states (Active, Matching, Anchored, Merged).
- **Transactional Integrity**: Ensures internal state updates are atomic with respect to Factor Graph (F10) operations.
- **Implementation**: Uses "Check-Act" pattern. Calls F10 first; if F10 fails, F12 does not update state and returns error.
### Chunk Responsibility Clarification
**F12 is the high-level chunk manager**:
- Owns chunk state (ChunkHandle with matching_status, is_active, has_anchor)
- Provides high-level queries: `get_active_chunk()`, `get_chunks_for_matching()`
- Coordinates with F10 for factor graph operations
- Persists chunk state for crash recovery
**F10 provides low-level factor graph operations** (see F10 spec):
- Subgraph creation and factor management
- Does NOT own chunk state - only factor graph data
**F11 coordinates recovery** (see F11 spec):
- Calls F12 for chunk operations
- F11 NEVER directly calls F10 chunk methods
### Interaction
- Called by **F02.2 Flight Processing Engine** and **F11 Failure Recovery Coordinator** (via F02.2 or direct delegation).
- Calls **F10 Factor Graph Optimizer**.
### Scope
- Chunk lifecycle management
@@ -108,10 +102,10 @@ class IRouteChunkManager(ABC):
### `create_chunk(flight_id: str, start_frame_id: int) -> ChunkHandle`
**Description**: Creates a new route chunk and initializes it in the factor graph.
**Description**: Initializes a new route chunk.
**Called By**:
- F02 Flight Processor (when tracking lost)
- F02.2 Flight Processing Engine (when tracking lost)
- F11 Failure Recovery Coordinator (proactive chunk creation)
**Input**:
@@ -151,10 +145,10 @@ ChunkHandle:
### `add_frame_to_chunk(chunk_id: str, frame_id: int, vo_result: RelativePose) -> bool`
**Description**: Adds a frame to an existing chunk with its VO result.
**Description**: Adds a frame to an active chunk.
**Called By**:
- F02 Flight Processor (during frame processing)
- F02.2 Flight Processing Engine (during frame processing)
**Input**:
```python
@@ -192,20 +186,9 @@ bool: True if frame added successfully
- F09 Metric Refinement (for chunk LiteSAM matching)
- F11 Failure Recovery Coordinator (chunk state queries)
**Input**:
```python
chunk_id: str
```
**Input**: `chunk_id: str`
**Output**:
```python
List[int] # Frame IDs in chunk, ordered by sequence
```
**Test Cases**:
1. **Get frames**: Returns all frames in chunk
2. **Empty chunk**: Returns empty list
3. **Ordered frames**: Frames returned in sequence order
**Output**: `List[int]` # Frame IDs in chunk, ordered by sequence
---
@@ -218,25 +201,7 @@ List[int] # Frame IDs in chunk, ordered by sequence
- F09 Metric Refinement (chunk LiteSAM matching)
- F06 Image Rotation Manager (chunk rotation)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
List[np.ndarray] # Images for each frame in chunk
```
**Processing Flow**:
1. Get chunk frames via get_chunk_frames()
2. Load images from F05 Image Input Pipeline
3. Return list of images
**Test Cases**:
1. **Get images**: Returns all images in chunk
2. **Image loading**: Images loaded correctly from pipeline
3. **Order consistency**: Images match frame order
**Output**: `List[np.ndarray]` # Images for each frame in chunk
---
@@ -247,30 +212,7 @@ List[np.ndarray] # Images for each frame in chunk
**Called By**:
- F08 Global Place Recognition (chunk semantic matching)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
```
**Processing Flow**:
1. Get chunk images via get_chunk_images()
2. Call F08.compute_chunk_descriptor(chunk_images) → aggregate descriptor
3. Return composite descriptor
**Delegation**:
- Delegates to F08.compute_chunk_descriptor() for descriptor computation
- F08 handles aggregation logic (mean, VLAD, or max)
- Single source of truth for chunk descriptor computation
**Test Cases**:
1. **Compute descriptor**: Returns aggregated descriptor
2. **Multiple images**: Descriptor aggregates correctly
3. **Descriptor quality**: More robust than single-image descriptor
**Output**: `np.ndarray`: Aggregated descriptor vector (4096-dim or 8192-dim)
---
@@ -282,11 +224,6 @@ np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
- F11 Failure Recovery Coordinator (for tile search area)
- F04 Satellite Data Manager (for tile prefetching)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
ChunkBounds:
@@ -295,58 +232,23 @@ ChunkBounds:
confidence: float
```
**Processing Flow**:
1. Get chunk trajectory from F10.get_chunk_trajectory()
2. If chunk has anchor:
- Use anchor GPS as center
- Compute radius from trajectory extent
3. If chunk unanchored:
- Estimate center from VO trajectory (relative to start)
- Use dead-reckoning estimate
- Lower confidence
4. Return ChunkBounds
**Test Cases**:
1. **Anchored chunk**: Returns accurate bounds with high confidence
2. **Unanchored chunk**: Returns estimated bounds with lower confidence
3. **Bounds calculation**: Radius computed from trajectory extent
---
### `is_chunk_ready_for_matching(chunk_id: str) -> bool`
**Description**: Determines if chunk has enough frames and consistency for matching.
**Called By**:
- F11 Failure Recovery Coordinator (before attempting matching)
- Background matching task
**Input**:
```python
chunk_id: str
```
**Output**:
```python
bool: True if chunk ready for matching
```
**Description**: Checks if a chunk has enough data (frames, spread) to attempt satellite matching.
**Criteria**:
- **Min frames**: >= 5 frames (configurable)
- **Max frames**: <= 20 frames (configurable, prevents oversized chunks)
- **Internal consistency**: VO factors have reasonable inlier counts
- **Not already matched**: matching_status != "anchored" or "merged"
**Test Cases**:
1. **Ready chunk**: 10 frames, good consistency → True
2. **Too few frames**: 3 frames → False
3. **Already anchored**: has_anchor=True → False
- Min frames: >= 5 frames (configurable)
- Max frames: <= 20 frames (configurable, prevents oversized chunks)
- Internal consistency: VO factors have reasonable inlier counts
- Not already matched: matching_status != "anchored" or "merged"
---
### `mark_chunk_anchored(chunk_id: str, frame_id: int, gps: GPSPoint) -> bool`
**Description**: Marks chunk as anchored with GPS coordinate.
**Description**: Anchors a chunk to a specific GPS coordinate (e.g., from successful satellite matching).
**Called By**:
- F11 Failure Recovery Coordinator (after successful chunk matching)
@@ -358,17 +260,15 @@ frame_id: int # Frame within chunk that was anchored
gps: GPSPoint
```
**Output**:
```python
bool: True if marked successfully
```
**Output**: `bool` - True if marked successfully
**Processing Flow**:
1. Verify chunk exists
2. Call F10.add_chunk_anchor()
3. Update chunk state (has_anchor=True, anchor_frame_id, anchor_gps)
4. Update matching_status to "anchored"
5. Trigger chunk optimization
3. If successful:
- Update chunk state (has_anchor=True, anchor_frame_id, anchor_gps)
- Update matching_status to "anchored"
- Trigger chunk optimization
**Test Cases**:
1. **Mark anchored**: Chunk state updated correctly
@@ -383,30 +283,8 @@ bool: True if marked successfully
**Called By**:
- F11 Failure Recovery Coordinator (background matching task)
- Background processing task
**Input**:
```python
flight_id: str
```
**Output**:
```python
List[ChunkHandle] # Unanchored chunks ready for matching
```
**Processing Flow**:
1. Get all chunks for flight_id
2. Filter chunks where:
- has_anchor == False
- is_chunk_ready_for_matching() == True
- matching_status == "unanchored" or "matching"
3. Return filtered list
**Test Cases**:
1. **Get unanchored chunks**: Returns ready chunks
2. **Filter criteria**: Only returns chunks meeting criteria
3. **Empty result**: Returns empty list if no ready chunks
**Output**: `List[ChunkHandle]` # Unanchored chunks ready for matching
---
@@ -415,22 +293,9 @@ List[ChunkHandle] # Unanchored chunks ready for matching
**Description**: Gets the currently active chunk for a flight.
**Called By**:
- F02 Flight Processor (before processing frame)
- F02.2 Flight Processing Engine (before processing frame)
**Input**:
```python
flight_id: str
```
**Output**:
```python
Optional[ChunkHandle] # Active chunk or None
```
**Test Cases**:
1. **Get active chunk**: Returns active chunk
2. **No active chunk**: Returns None
3. **Multiple chunks**: Returns only active chunk
**Output**: `Optional[ChunkHandle]` # Active chunk or None
---
@@ -440,27 +305,9 @@ Optional[ChunkHandle] # Active chunk or None
**Called By**:
- F11 Failure Recovery Coordinator (after chunk merged)
- F02 Flight Processor (chunk lifecycle)
- F02.2 Flight Processing Engine (chunk lifecycle)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
bool: True if deactivated successfully
```
**Processing Flow**:
1. Verify chunk exists
2. Update chunk state (is_active=False)
3. Update matching_status to "merged" if merged
4. Return True
**Test Cases**:
1. **Deactivate chunk**: Chunk marked inactive
2. **After merge**: Matching status updated to "merged"
**Output**: `bool` - True if deactivated successfully
---
@@ -481,35 +328,18 @@ transform: Sim3Transform:
scale: float
```
**Output**:
```python
bool: True if merge successful
```
**Output**: `bool` - True if merge successful
**Processing Flow**:
1. Verify both chunks exist
2. Verify source_chunk_id is anchored (has_anchor=True)
3. Validate chunks can be merged (not already merged, not same chunk)
4. Call F10.merge_chunk_subgraphs(flight_id, source_chunk_id, target_chunk_id, transform)
5. Update source_chunk_id state:
- Set is_active=False
- Set matching_status="merged"
- Call deactivate_chunk(source_chunk_id)
6. target_chunk remains active (now contains merged frames)
7. Persist chunk state via F03 Flight Database.save_chunk_state()
8. Return True
**Merge Convention**:
- `merge_chunks(target, source)` → source is merged INTO target
- Result is stored in target_chunk
- source_chunk is deactivated after merge
- Example: `merge_chunks("main", "chunk_3")` merges chunk_3 into main trajectory
**Validation**:
- Both chunks must exist
- source_chunk must be anchored
- source_chunk must not already be merged
- target_chunk and source_chunk must be different
1. Call F10.merge_chunk_subgraphs(flight_id, source_chunk_id, target_chunk_id, transform)
2. If successful:
- Update source_chunk_id state:
- Set is_active=False
- Set matching_status="merged"
- Call deactivate_chunk(source_chunk_id)
- target_chunk remains active (now contains merged frames)
- Persist chunk state via F03 Flight Database.save_chunk_state()
3. Return True
**Test Cases**:
1. **Merge anchored chunk**: source_chunk merged into target_chunk
@@ -525,29 +355,7 @@ bool: True if merge successful
**Called By**:
- F11 Failure Recovery Coordinator (when chunk matching starts)
**Input**:
```python
chunk_id: str
```
**Output**:
```python
bool: True if marked successfully
```
**Processing Flow**:
1. Verify chunk exists
2. Verify chunk is unanchored (has_anchor=False)
3. Update matching_status to "matching"
4. Return True
**State Transition**:
- `unanchored` → `matching` (explicit transition)
**Test Cases**:
1. **Mark unanchored chunk**: Status updated to "matching"
2. **Mark already anchored chunk**: Returns False (invalid state)
3. **Mark non-existent chunk**: Returns False
**Output**: `bool` - True if marked successfully
## Integration Tests
@@ -600,14 +408,10 @@ bool: True if marked successfully
## Dependencies
### Internal Components
- **F10 Factor Graph Optimizer**: Chunk creation and factor management
- **F05 Image Input Pipeline**: Image retrieval
- **F08 Global Place Recognition**: Descriptor computation
- **F07 Sequential VO**: VO results for chunk building
- **F03 Flight Database**: Chunk state persistence
### External Dependencies
- **numpy**: Array operations
- **F10 Factor Graph Optimizer**: Critical dependency for subgraph operations (`create_chunk_subgraph`, `add_relative_factor_to_chunk`, `merge_chunk_subgraphs`).
- **F03 Flight Database**: Persistence via `save_chunk_state()`, `load_chunk_states()`.
- **F05 Image Input Pipeline**: Image retrieval via `get_image_by_sequence()` for `get_chunk_images()`.
- **F08 Global Place Recognition**: Descriptor computation via `compute_chunk_descriptor()` for `get_chunk_composite_descriptor()`.
## Data Models
@@ -649,4 +453,3 @@ class Sim3Transform(BaseModel):
rotation: np.ndarray # (3, 3) rotation matrix or (4,) quaternion
scale: float # Scale factor
```
docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md
+7 -7
View File
@@ -72,7 +72,7 @@ class ICoordinateTransformer(ABC):
**Why ENU Origin?**: Factor graph optimization works in Cartesian coordinates (meters) for better numerical stability. ENU converts GPS (degrees) to local meters relative to an origin. See `helpers/enu_origin_explanation.md` for details.
**Called By**:
- F02 Flight Processor (during create_flight)
- F02.1 Flight Lifecycle Manager (during create_flight)
**Input**:
```python
@@ -200,7 +200,7 @@ GPSPoint: WGS84 coordinates
**Description**: Converts pixel coordinates to GPS using camera pose and ground plane assumption.
**Called By**:
- F02 Flight Processor (for frame center GPS)
- F02.2 Flight Processing Engine (for frame center GPS)
- Internal (for image_object_to_gps)
**Input**:
@@ -392,12 +392,12 @@ List[Tuple[float, float]]: Transformed points
## Dependencies
### Internal Components
- **F10 Factor Graph Optimizer**: For frame poses via get_trajectory(flight_id)
- **F17 Configuration Manager**: For camera parameters via get_flight_config(flight_id)
- **H01 Camera Model**: For projection operations
- **H02 GSD Calculator**: For GSD calculations
- **F10 Factor Graph Optimizer**: For frame poses via `get_trajectory(flight_id)` - required for `pixel_to_gps()` and `image_object_to_gps()` to get frame pose estimates.
- **F17 Configuration Manager**: For camera parameters via `get_flight_config(flight_id)`.
- **H01 Camera Model**: For projection/unprojection operations (`project()`, `unproject()`).
- **H02 GSD Calculator**: For GSD calculations in coordinate conversions.
**Note**: F13 uses internal enu_to_gps() and gps_to_enu() methods that rely on the ENU origin set via set_enu_origin(flight_id, origin_gps). H06 Web Mercator Utils is NOT used for ENU conversions - ENU is a local Cartesian coordinate system.
**Note**: F13 uses internal `enu_to_gps()` and `gps_to_enu()` methods that rely on the ENU origin set via `set_enu_origin(flight_id, origin_gps)`. H06 Web Mercator Utils is NOT used for ENU conversions - ENU is a local Cartesian coordinate system.
### External Dependencies
- **numpy**: Matrix operations
docs/02_components/14_result_manager/result_manager_spec.md
+33 -42
View File
@@ -10,6 +10,12 @@
class IResultManager(ABC):
@abstractmethod
def update_frame_result(self, flight_id: str, frame_id: int, result: FrameResult) -> bool:
"""
Atomic update:
1. Saves result to frame_results table.
2. Updates waypoint in waypoints table.
3. All within a single transaction via F03.
"""
pass
@abstractmethod
@@ -36,13 +42,8 @@ class IResultManager(ABC):
## Component Description
### Responsibilities
- Manage trajectory results per flight
- Track frame refinements and changes
- Store waypoint updates via F03 Flight Database
- Send incremental updates via F15 SSE Event Streamer
- Maintain result versioning for audit trail
**Note**: F14 receives GPS coordinates directly from the caller (F02). F14 does NOT call F10 Factor Graph Optimizer or F13 Coordinate Transformer for pose retrieval/conversion. This ensures unidirectional data flow: F02 → F14 → F03/F15.
- Result consistency and publishing.
- **Atomic Updates**: Ensures consistency between normalized `waypoints` and denormalized `frame_results` via transaction requests to F03.
### Scope
- Result state management
@@ -55,7 +56,7 @@ class IResultManager(ABC):
### `update_frame_result(flight_id: str, frame_id: int, result: FrameResult) -> bool`
**Description**: Updates result for a processed frame.
**Description**: Persists and publishes the result of a processed frame.
**Called By**:
- Main processing loop (after each frame)
@@ -78,10 +79,12 @@ result: FrameResult:
**Output**: `bool` - True if updated
**Processing Flow**:
1. Store result via F03 Flight Database.save_frame_result()
2. Call publish_waypoint_update()
3. Call F15 SSE Event Streamer.send_frame_result()
4. Update flight statistics
1. Construct DB transaction:
- Insert/Update `frame_results`.
- Update `waypoints` (latest position).
2. Call `F03.execute_transaction()`.
3. If success: call `F15.send_frame_result()`.
4. Update flight statistics.
**Test Cases**:
1. New frame result → stored and published
@@ -91,7 +94,7 @@ result: FrameResult:
### `publish_waypoint_update(flight_id: str, frame_id: int) -> bool`
**Description**: Updates waypoint in Flight Database via F03.
**Description**: Specifically triggers an update for the waypoint visualization.
**Called By**:
- Internal (after update_frame_result)
@@ -105,10 +108,9 @@ frame_id: int
**Output**: `bool` - True if updated successfully
**Processing Flow**:
1. Get result for frame_id
2. Convert to Waypoint format
3. Call F03 Flight Database.update_waypoint()
4. Handle errors (retry if transient)
1. Fetch latest data.
2. Call `F15.send_frame_result()` (or a specific lightweight event).
3. Handle errors (retry if transient)
**Test Cases**:
1. Successful update → Waypoint stored in database
@@ -141,7 +143,7 @@ FlightResults:
### `mark_refined(flight_id: str, frame_ids: List[int]) -> bool`
**Description**: Marks frames as refined after batch optimization.
**Description**: Updates results for frames that have been retrospectively improved (e.g., after loop closure or chunk merge).
**Called By**:
- F10 Factor Graph (after asynchronous refinement)
@@ -154,16 +156,16 @@ frame_ids: List[int] # Frames with updated poses
**Output**: `bool`
**Note**: F14 does NOT call F10 or F13. The caller (F02) performs the following steps before calling mark_refined():
1. F02 gets refined poses from F10.get_trajectory(flight_id)
2. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple)
3. F02 calls F14.mark_refined() with the GPS-converted results
**Note**: F14 does NOT call F10 or F13. The caller (F02.2) performs the following steps before calling mark_refined():
1. F02.2 gets refined poses from F10.get_trajectory(flight_id)
2. F02.2 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple)
3. F02.2 calls F14.mark_refined() with the GPS-converted results
**Processing Flow**:
1. For each frame_id in frame_ids:
- Receive GPS coordinates from caller (already converted)
- Update result with refined=True via F03 Flight Database.save_frame_result()
- Update waypoint via F03 Flight Database.update_waypoint()
- Update result with refined=True via F03 Flight Database (part of transaction)
- Update waypoint via F03 Flight Database (part of transaction)
- Call F15 SSE Event Streamer.send_refinement()
**Test Cases**:
@@ -194,11 +196,11 @@ since: datetime
### `update_results_after_chunk_merge(flight_id: str, merged_frames: List[int]) -> bool`
**Description**: Updates frame results after chunk merging changes frame poses.
**Description**: Handling specific to chunk merging events.
**Triggered By**:
- `ChunkMerged` event from F11 (F14 subscribes to this event)
- Alternative: F02 Flight Processor can coordinate this call after receiving ChunkMerged event
- Alternative: F02.2 Flight Processing Engine can coordinate this call after receiving ChunkMerged event
**Input**:
```python
@@ -208,19 +210,9 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
**Output**: `bool` - True if updated successfully
**Note**: F14 does NOT call F10 or F13. The caller (F02) performs these steps before calling update_results_after_chunk_merge():
1. F02 receives ChunkMerged event from F11 with merged_frames list
2. F02 gets updated poses from F10.get_trajectory(flight_id)
3. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple) for each frame
4. F02 calls F14.update_results_after_chunk_merge() with GPS-converted results
**Processing Flow**:
1. For each frame_id in merged_frames:
- Receive GPS coordinates from caller (already converted)
- Update frame result via F03 Flight Database.save_frame_result()
- Update waypoint via F03 Flight Database.update_waypoint()
- Send refinement event via F15 SSE Event Streamer.send_refinement()
2. Return True
1. Similar to `mark_refined` but triggered by `ChunkMerged` event context.
2. Ensures all frames in the merged chunk are updated to the global coordinate system.
**Test Cases**:
1. **Chunk merge updates**: All merged frames updated and published
@@ -263,10 +255,10 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
## Dependencies
### Internal Components
- **F03 Flight Database**: For waypoint and frame result persistence
- **F15 SSE Event Streamer**: For real-time result streaming
- **F03 Flight Database**: Must support transactional updates.
- **F15 SSE Event Streamer**: For real-time result streaming.
**Note**: F14 does NOT depend on F10, F13, or F11. The caller (F02) coordinates with those components and provides GPS-converted results to F14. This ensures unidirectional data flow and eliminates circular dependencies.
**Note**: F14 does NOT depend on F10, F13, or F11. The caller (F02.2) coordinates with those components and provides GPS-converted results to F14. This ensures unidirectional data flow and eliminates circular dependencies.
### External Dependencies
- None
@@ -308,4 +300,3 @@ class FlightResults(BaseModel):
frames: List[FrameResult]
statistics: FlightStatistics
```
docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md
+16 -15
View File
@@ -30,7 +30,6 @@ class ISSEEventStreamer(ABC):
@abstractmethod
def send_heartbeat(self, flight_id: str) -> bool:
"""Sends heartbeat/keepalive to all clients subscribed to flight."""
pass
@abstractmethod
@@ -39,19 +38,22 @@ class ISSEEventStreamer(ABC):
@abstractmethod
def get_active_connections(self, flight_id: str) -> int:
"""Returns count of active SSE connections for a flight."""
pass
```
## Component Description
### Responsibilities
- Server-Sent Events broadcaster for real-time results
- Stream per-frame processing results to clients
- Send refinement updates asynchronously
- Request user input when processing blocked
- Handle client connections and reconnections
- Event replay from last received event
- Real-time communication with clients.
- Buffering events for disconnected clients.
### Callers
- **F02.1 Flight Lifecycle Manager**: Calls `create_stream` (delegated from F01).
- **F02.2 Flight Processing Engine**: Calls `send_user_input_request`, `send_search_progress`.
- **F14 Result Manager**: Calls `send_frame_result`, `send_refinement`.
### Consistency Fix
- Previously listed F11 as caller. **Correction**: F11 returns request objects to F02.2. **F02.2 is the sole caller** for user input requests and search status updates. F11 has no dependencies on F15.
### Scope
- SSE protocol implementation
@@ -64,7 +66,7 @@ class ISSEEventStreamer(ABC):
### `create_stream(flight_id: str, client_id: str) -> StreamConnection`
**Description**: Creates SSE connection for a client.
**Description**: Establishes a server-sent events connection.
**Called By**: F01 REST API (GET /stream endpoint)
@@ -124,7 +126,7 @@ StreamConnection:
**Description**: Sends search_expanded event.
**Called By**: F11 Failure Recovery Coordinator
**Called By**: F02.2 Flight Processing Engine (via F11 status return)
**Event Format**:
```json
@@ -144,7 +146,7 @@ StreamConnection:
**Description**: Sends user_input_needed event.
**Called By**: F11 Failure Recovery Coordinator
**Called By**: F02.2 Flight Processing Engine (via F11 request object)
**Event Format**:
```json
@@ -182,11 +184,11 @@ StreamConnection:
### `send_heartbeat(flight_id: str) -> bool`
**Description**: Sends heartbeat/keepalive ping to all clients subscribed to a flight. Keeps connections alive and helps detect stale connections.
**Description**: Sends heartbeat/keepalive to all clients subscribed to flight.
**Called By**:
- Background heartbeat task (every 30 seconds)
- F02 Flight Processor (periodically during processing)
- F02.2 Flight Processing Engine (periodically during processing)
**Event Format**:
```
@@ -217,7 +219,7 @@ StreamConnection:
**Description**: Returns count of active SSE connections for a flight.
**Called By**:
- F02 Flight Processor (monitoring)
- F02.2 Flight Processing Engine (monitoring)
- Admin tools
**Test Cases**:
@@ -287,4 +289,3 @@ class SSEEvent(BaseModel):
id: Optional[str]
data: Dict[str, Any]
```
docs/02_components/16_model_manager/model_manager_spec.md
+1 -1
View File
@@ -55,7 +55,7 @@ class IModelManager(ABC):
**Description**: Loads model in specified format.
**Called By**: F02 Flight Manager (during initialization)
**Called By**: F02.1 Flight Lifecycle Manager (during system initialization)
**Input**:
```python
docs/02_components/17_configuration_manager/configuration_manager_spec.md
+2 -2
View File
@@ -126,7 +126,7 @@ CameraParameters:
**Description**: Gets flight-specific configuration.
**Called By**: F02 Flight Manager
**Called By**: F02.1 Flight Lifecycle Manager
**Output**:
```python
@@ -193,7 +193,7 @@ FlightConfig:
**Description**: Persists flight-specific configuration when a flight is created.
**Called By**:
- F02 Flight Processor (during flight creation)
- F02.1 Flight Lifecycle Manager (during flight creation)
**Input**:
```python
docs/02_components/decomposition_plan.md
+85 -72
View File
@@ -45,9 +45,13 @@
**Interface**: `IFlightAPI`
**Endpoints**: `POST /flights`, `GET /flights/{flightId}`, `DELETE /flights/{flightId}`, `PUT /flights/{flightId}/waypoints/{waypointId}`, `POST .../images/batch`, `POST .../user-fix`, `GET .../status`, `GET .../stream`
**F02_flight_processor**
**Interface**: `IFlightProcessor`
**API**: `create_flight()`, `get_flight()`, `get_flight_state()`, `delete_flight()`, `update_waypoint()`, `batch_update_waypoints()`, `validate_waypoint()`, `validate_geofence()`, `process_frame()`, `run_processing_loop()`, `handle_tracking_loss()`, `initialize_system()`
**F02.1_flight_lifecycle_manager**
**Interface**: `IFlightLifecycleManager`
**API**: `create_flight()`, `get_flight()`, `get_flight_state()`, `delete_flight()`, `update_waypoint()`, `batch_update_waypoints()`, `validate_waypoint()`, `validate_geofence()`, `queue_images()`, `handle_user_fix()`, `create_client_stream()`, `convert_object_to_gps()`, `initialize_system()`
**F02.2_flight_processing_engine**
**Interface**: `IFlightProcessingEngine`
**API**: `start_processing()`, `stop_processing()`, `process_frame()`, `apply_user_fix()`, `handle_tracking_loss()`, `get_active_chunk()`, `create_new_chunk()`
**F03_flight_database**
**Interface**: `IFlightDatabase`
@@ -176,40 +180,40 @@
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F02 | F16 | `load_config()` | Load system configuration |
| F02 | F16 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| F04 | F08 | Satellite tiles | F08 generates descriptors for Faiss |
| F08 | H04 | `build_index()` | Build satellite descriptor index |
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model for descriptor generation |
| F02.1 | F17 | `load_config()` | Load system configuration |
| F02.1 | F16 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| F04 | F08 | Satellite tiles + index | F08 loads pre-built Faiss index from provider |
| F08 | H04 | `load_index()` | Load satellite descriptor index |
| F08 | F16 | `get_inference_engine("DINOv2")` | Get model for descriptor computation |
### Flight Creation
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | F01 | `POST /flights` | Create flight |
| F01 | F02 | `create_flight()` | Initialize flight state |
| F02 | F16 | `get_flight_config()` | Get camera params, altitude |
| F02 | F13 | `set_enu_origin(flight_id, start_gps)` | Set ENU coordinate origin |
| F02 | F04 | `prefetch_route_corridor()` | Prefetch tiles |
| F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download |
| F01 | F02.1 | `create_flight()` | Initialize flight state |
| F02.1 | F17 | `get_flight_config()` | Get camera params, altitude |
| F02.1 | F13 | `set_enu_origin(flight_id, start_gps)` | Set ENU coordinate origin |
| F02.1 | F04 | `prefetch_route_corridor()` | Prefetch tiles |
| F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download (includes tile metadata) |
| F04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations |
| F02 | F03 | `insert_flight()` | Persist flight data |
| F02.1 | F03 | `insert_flight()` | Persist flight data |
### SSE Stream Creation
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | F01 | `GET .../stream` | Open SSE connection |
| F01 | F02 | `create_client_stream()` | Route through coordinator |
| F02 | F15 | `create_stream()` | Establish SSE channel |
| F01 | F02.1 | `create_client_stream()` | Route through lifecycle manager |
| F02.1 | F15 | `create_stream()` | Establish SSE channel |
### Image Upload
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | F01 | `POST .../images/batch` | Upload 10-50 images |
| F01 | F02 | `queue_images()` | Route through coordinator |
| F02 | F05 | `queue_batch()` | Queue for processing |
| F01 | F02.1 | `queue_images()` | Route through lifecycle manager |
| F02.1 | F05 | `queue_batch()` | Queue for processing |
| F05 | H08 | `validate_batch()` | Validate sequence, format |
| F05 | F03 | `save_image_metadata()` | Persist image metadata |
@@ -217,48 +221,48 @@
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F02 | F05 | `get_next_image()` | Get image for processing |
| F02 | F06 | `requires_rotation_sweep()` | Check if sweep needed |
| F02.2 | F05 | `get_next_image()` | Get image for processing |
| F02.2 | F06 | `requires_rotation_sweep()` | Check if sweep needed |
| F06 | H07 | `rotate_image()` × 12 | Rotate in 30° steps |
| F06 | F09 | `align_to_satellite(img, tile, bounds)` × 12 | Try LiteSAM each rotation |
| F06 | F04 | `get_cached_tile()` + `compute_tile_bounds()` | Get expected tile with bounds |
| F09 | F15 | `get_inference_engine("LiteSAM")` | Get model |
| F02.2 | F04 | `get_cached_tile()` + `compute_tile_bounds()` | Get expected tile with bounds |
| F09 | F16 | `get_inference_engine("LiteSAM")` | Get model |
| F06 | H07 | `calculate_rotation_from_points()` | Precise angle from homography |
| F06 | F03 | `save_heading()` | Store UAV heading |
| F02.2 | F03 | `save_heading()` | Store UAV heading |
### Per-Frame Processing (Sequential VO)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F02 | F12 | `get_active_chunk()` | Get active chunk for frame |
| F02 | F07 | Process frame | Provide image and chunk context |
| F02.2 | F12 | `get_active_chunk()` | Get active chunk for frame |
| F02.2 | F07 | `compute_relative_pose()` | Provide image and chunk context |
| F07 | F16 | `get_inference_engine("SuperPoint")` | Get feature extractor |
| F07 | F16 | `get_inference_engine("LightGlue")` | Get matcher |
| F07 | H05 | `start_timer()`, `end_timer()` | Monitor timing |
| F07 | F10 | `add_relative_factor_to_chunk()` | Add pose measurement to chunk subgraph |
| F02 | F12 | `add_frame_to_chunk()` | Add frame to chunk |
| F02.2 | F10 | `add_relative_factor_to_chunk()` | Add pose measurement to chunk subgraph |
| F02.2 | F12 | `add_frame_to_chunk()` | Add frame to chunk |
### Tracking Good (Drift Correction)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F02 | F11 | `check_confidence()` | Check tracking quality |
| F02 | F04 | `fetch_tile()` + `compute_tile_bounds()` | Get single tile with bounds |
| F02 | F09 | `align_to_satellite(img, tile, bounds)` | Align to 1 tile |
| F02 | F10 | `add_absolute_factor()` | Add GPS measurement |
| F02.2 | F11 | `check_confidence()` | Check tracking quality |
| F02.2 | F04 | `fetch_tile()` + `compute_tile_bounds()` | Get single tile with bounds |
| F02.2 | F09 | `align_to_satellite(img, tile, bounds)` | Align to 1 tile |
| F02.2 | F10 | `add_absolute_factor()` | Add GPS measurement |
### Tracking Lost (Progressive Search + Chunk Building)
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F02 | F11 | `check_confidence()` → FAIL | Low confidence |
| F02.2 | F11 | `check_confidence()` → FAIL | Low confidence |
| F11 | F12 | `create_chunk_on_tracking_loss()` | **Proactive chunk creation** |
| F12 | F10 | `create_new_chunk()` | Create chunk in factor graph |
| F12 | F10 | `create_chunk_subgraph()` | Create chunk in factor graph |
| F12 | F03 | `save_chunk_state()` | Persist chunk state for recovery |
| F02 | F12 | `get_active_chunk()` | Get new active chunk |
| F02.2 | F12 | `get_active_chunk()` | Get new active chunk |
| F11 | F06 | `requires_rotation_sweep()` | Trigger rotation sweep (single-image) |
| F11 | F08 | `retrieve_candidate_tiles()` | Coarse localization (single-image) |
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model |
| F08 | F16 | `get_inference_engine("DINOv2")` | Get model |
| F08 | H04 | `search()` | Query Faiss index |
| F08 | F04 | `get_tile_by_gps()` × 5 | Get candidate tiles |
| F11 | F04 | `expand_search_grid(4)` | Get 2×2 grid |
@@ -266,14 +270,14 @@
| F11 (fail) | F04 | `expand_search_grid(9)` | Expand to 3×3 |
| F11 (fail) | F04 | `expand_search_grid(16)` | Expand to 4×4 |
| F11 (fail) | F04 | `expand_search_grid(25)` | Expand to 5×5 |
| F11 (fail) | F03 | Continue building chunk | **Chunk building continues** |
| F11 (background) | F03 | `get_chunks_for_matching()` | Get unanchored chunks |
| F11 (fail) | F12 | Continue building chunk | **Chunk building continues** |
| F11 (background) | F12 | `get_chunks_for_matching()` | Get unanchored chunks |
| F11 (background) | F08 | `retrieve_candidate_tiles_for_chunk()` | **Chunk semantic matching** |
| F11 (background) | F06 | `try_chunk_rotation_steps()` | **Chunk rotation sweeps** |
| F11 (background) | F09 | `align_chunk_to_satellite()` | **Chunk LiteSAM matching** |
| F11 (background) | F10 | `add_chunk_anchor()` + `merge_chunks()` | **Chunk merging** |
| F11 (fail) | F14 | `send_user_input_request()` | Request human help (last resort) |
| F11 | F02 | `update_flight_status("BLOCKED")` | Block processing |
| F11 (background) | F10 | `add_chunk_anchor()` + `merge_chunk_subgraphs()` | **Chunk merging** |
| F11 (fail) | F02.2 | Returns `UserInputRequest` | Request human help (last resort) |
| F02.2 | F15 | `send_user_input_request()` | Send SSE event to client |
### Optimization & Results
@@ -281,13 +285,13 @@
|--------|--------|--------|---------|
| F10 | H03 | `huber_loss()`, `cauchy_loss()` | Apply robust kernels |
| F10 | Internal | `optimize()` | Run iSAM2 optimization |
| F02 | F10 | `get_trajectory()` | Get optimized poses |
| F02 | F13 | `enu_to_gps()` | Convert ENU to GPS |
| F02.2 | F10 | `get_trajectory()` | Get optimized poses |
| F02.2 | F13 | `enu_to_gps()` | Convert ENU to GPS |
| F13 | H01 | `project()`, `unproject()` | Camera operations |
| F13 | H02 | `compute_gsd()` | GSD calculations |
| F13 | H06 | `tile_to_latlon()` | Coordinate transforms |
| F02 | F14 | Frame GPS + object coords | Provide results |
| F14 | F02 | `update_waypoint()` | Per-frame waypoint update |
| F02.2 | F14 | Frame GPS + object coords | Provide results |
| F14 | F03 | `update_waypoint()` | Per-frame waypoint update |
| F14 | F15 | `send_frame_result()` | Publish to client |
| F15 | Client | SSE `frame_processed` | Real-time delivery |
| F14 | F03 | `save_frame_result()` | Persist frame result |
@@ -298,18 +302,20 @@
|--------|--------|--------|---------|
| F15 | Client | SSE `user_input_needed` | Notify client |
| Client | F01 | `POST .../user-fix` | Provide anchor |
| F01 | F11 | `apply_user_anchor()` | Apply fix |
| F01 | F02.1 | `handle_user_fix()` | Route through lifecycle manager |
| F02.1 | F02.2 | `apply_user_fix()` | Delegate to processing engine |
| F02.2 | F11 | `apply_user_anchor()` | Apply fix |
| F11 | F10 | `add_absolute_factor()` (high confidence) | Hard constraint |
| F10 | Internal | `optimize()` | Re-optimize |
| F11 | Event | Emit `UserFixApplied` | F02 subscribes and resumes |
| F02.2 | F15 | `send_frame_result()` | Publish result via SSE |
### Asynchronous Refinement
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F10 | Internal (background) | `optimize()` | Back-propagate anchors |
| F10 | F14 | `get_trajectory()` | Get refined poses |
| F14 | F02 | `batch_update_waypoints()` | Batch update waypoints |
| F14 | F10 | `get_trajectory()` | Get refined poses |
| F14 | F03 | `batch_update_waypoints()` | Batch update waypoints |
| F14 | F15 | `send_refinement()` × N | Send updates |
| F15 | Client | SSE `frame_refined` × N | Incremental updates |
@@ -334,31 +340,31 @@
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F16 | ALL | `get_*_config()` | Provide configuration |
| F17 | ALL | `get_*_config()` | Provide configuration |
| H05 | F07, F08, F09, F10, F11 | `start_timer()`, `end_timer()` | Performance monitoring |
---
## Interaction Coverage Verification
✅ **Initialization**: F02→F16, F17; F04→F08→H04
✅ **Flight creation**: Client→F01→F02→F04,F12,F16,F17,F14
✅ **Image upload**: Client→F01→F05→H08,F17
✅ **Initialization**: F02.1→F16, F17; F04→F08→H04
✅ **Flight creation**: Client→F01→F02.1→F04,F12,F16,F17,F14
✅ **Image upload**: Client→F01→F02.1→F05→H08
✅ **Rotation sweep**: F06→H07,F09 (12 iterations)
✅ **Sequential VO**: F07→F16,F10(chunk),F12,H05
✅ **Drift correction**: F02→F04,F09,F10
✅ **Tracking loss**: F11→F12(proactive chunk),F06,F08,F04(progressive),F09,F15,F02
✅ **Chunk building**: F02→F12→F10,F07
✅ **Sequential VO**: F02.2→F07→F16,F10(chunk),F12,H05
✅ **Drift correction**: F02.2→F04,F09,F10
✅ **Tracking loss**: F02.2→F11→F12(proactive chunk),F06,F08,F04(progressive),F09
✅ **Chunk building**: F02.2→F12→F10,F07
✅ **Chunk image retrieval**: F12→F05(get_image_by_sequence for chunk images)
✅ **Chunk semantic matching**: F11→F12→F08(chunk descriptor)
✅ **Chunk LiteSAM matching**: F11→F06(chunk rotation)→F09(chunk alignment)
✅ **Chunk merging**: F11→F10(Sim3 transform)
✅ **Global PR**: F08→F16,H04,F04
✅ **Optimization**: F10→H03(chunk optimization, global optimization)
✅ **Coordinate transform**: F13→H01,H02,H06
✅ **Results**: F02→F13→F14→F15,F03
✅ **User input**: Client→F01→F11→F10,F02
✅ **Refinement**: F10→F14→F02,F15
✅ **Coordinate transform**: F13→F10,H01,H02,H06
✅ **Results**: F02.2→F13→F14→F15,F03
✅ **User input**: Client→F01→F02.1→F02.2→F11→F10
✅ **Refinement**: F10→F14→F03,F15
✅ **Configuration**: F17→ALL
✅ **Performance**: H05→processing components
@@ -386,7 +392,9 @@
### Component Specifications Status
- [x] F01 Flight API (merged from R01 Route REST API)
- [x] F02 Flight Processor (merged from R02, R03, G02)
- [x] F02.1 Flight Lifecycle Manager (split from F02 Flight Processor)
- [x] F02.2 Flight Processing Engine (split from F02 Flight Processor)
- [x] F03 Flight Database (merged from R04, G17)
- [x] F04 Satellite Data Manager
- [x] F05 Image Input Pipeline
- [x] F06 Image Rotation Manager
@@ -401,26 +409,31 @@
- [x] F15 SSE Event Streamer
- [x] F16 Model Manager
- [x] F17 Configuration Manager
- [x] F03 Flight Database (merged from R04, G17)
- [x] Helper components (H01-H08)
---
## Architecture Notes
### F02 Flight Processor Complexity
### F02 Split: Lifecycle Manager + Processing Engine
F02 Flight Processor handles multiple concerns:
- Flight lifecycle management
- Processing loop orchestration
- Event subscription and state updates
- Coordination of F07/F08/F09/F10
F02 has been split into two components with clear Single Responsibility:
**Current Status**: Acceptable for MVP. The responsibilities are related (all flight processing) and the component acts as a coordinator rather than implementing logic directly.
**F02.1 Flight Lifecycle Manager**:
- Flight CRUD operations
- System initialization
- API request routing
- SSE stream creation
- User fix delegation
**Future Consideration**: If complexity grows, consider splitting into:
- F02a Flight State Manager (lifecycle, status)
- F02b Processing Loop Coordinator (frame processing orchestration)
**F02.2 Flight Processing Engine**:
- Main processing loop
- Frame-by-frame processing orchestration
- Recovery coordination with F11
- Chunk management coordination with F12
- Runs in background thread per active flight
**Rationale**: This split aligns with Single Responsibility Principle. F02.1 handles external-facing operations (API layer), while F02.2 handles internal processing (background engine).
**Decision**: Keep as single component. Clear internal organization with separate methods for state vs processing concerns. Event-based communication with F11 keeps recovery logic separate.
docs/02_components/helpers/h04_faiss_index_manager_spec.md
+14
View File
@@ -107,6 +107,20 @@ H04 supports automatic fallback from GPU to CPU:
- `set_device("gpu")`: Use GPU acceleration (faster for large indexes)
- `set_device("cpu")`: Use CPU (fallback when GPU unavailable)
## Current vs Future Use Cases
### Current Use (MVP)
- **Satellite Index Loading**: F08 uses `load_index()` to load pre-built satellite descriptor index from provider.
- **Similarity Search**: F08 uses `search()` to find candidate satellite tiles.
### Future Use Cases (build_index, add_descriptors)
The `build_index()` and `add_descriptors()` methods are reserved for future features:
1. **UAV Loop Closure Detection**: Build index of UAV frame descriptors to detect when UAV revisits previously seen areas.
2. **Chunk-to-Chunk Matching**: Build index of chunk descriptors for matching disconnected trajectory segments.
3. **Flight-to-Flight Matching**: Match current flight to previous flights for multi-flight consistency.
**Note**: For MVP, F08 does NOT build satellite indexes - they are provided pre-built by the satellite data provider.
## Test Cases
1. Build index with 10,000 UAV image descriptors → succeeds
docs/02_components/system_flows.md
+55 -42
View File
@@ -11,7 +11,8 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
| ID | Component | Interface | Purpose |
|----|-----------|-----------|---------|
| F01 | Flight API | `IFlightAPI` | REST endpoints, SSE streaming |
| F02 | Flight Processor | `IFlightProcessor` | Central coordinator, processing loop |
| F02.1 | Flight Lifecycle Manager | `IFlightLifecycleManager` | Flight CRUD, init, API delegation |
| F02.2 | Flight Processing Engine | `IFlightProcessingEngine` | Processing loop, recovery orchestration |
| F03 | Flight Database | `IFlightDatabase` | Persistence layer |
| F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search |
| F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage |
@@ -86,7 +87,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
└────────┬────────┘
▼
┌─────────────────┐
│ F02 Ready │ ← Ready to accept flights
│ F02.1 Ready │ ← Ready to accept flights
└────────┬────────┘
▼
┌─────────────────┐
@@ -113,7 +114,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ create_flight()
▼
┌─────────┐
│ F02 │ Flight Processor
│ F02.1 │ Flight Lifecycle Manager
└────┬────┘
┌────────────────────────┬─┴─┬────────────────────────┐
│ │ │ │
@@ -186,7 +187,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌──────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │
│ F02.2 Flight Processing Engine │
│ │
│ ┌─────────────┐ │
│ │ get_next │◄───────────────────────────────────────────────────────┐│
@@ -205,7 +206,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ └──────────────────────────────┬──────────────────────────────┘ ││
│ ▼ ││
│ ┌─────────────────────────────────────────────────────────────┐ ││
│ │ F07 compute_relative_pose_in_chunk() │ ││
│ │ F07 compute_relative_pose() │ ││
│ │ ├─ SuperPoint extract (prev + curr) via F16 │ ││
│ │ └─ LightGlue match via F16 │ ││
│ └──────────────────────────────┬──────────────────────────────┘ ││
@@ -230,15 +231,17 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ ┌─────────────────┴─────────────────┐ ││
│ │ Match Found? │ ││
│ ▼ ▼ ││
│ ┌─────────────────────────┐ ┌─────────────────────────┐ ││
│ │F10 add_absolute_factor()│ │ Skip absolute anchor │ ││
│ │F06 update_heading() │ │ (VO-only frame) │ ││
│ └────────────┬────────────┘ └─────────────────────────┘ ││
│ ┌──────────────────────────────────┐ ┌─────────────────────────┐ ││
│ │F10 add_absolute_factor(flight_id,│ │ Skip absolute anchor │ ││
│ │ frame_id, gps, covariance, │ │ (VO-only frame) │ ││
│ │ is_user_anchor=False) │ └─────────────────────────┘ ││
│ │F06 update_heading() │ ││
│ └────────────┬─────────────────────┘ ││
│ │ ││
│ └─────────────────────┬───────────────────────────────────┘│
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ F10 optimize_chunk() │ │
│ │ F10 optimize_chunk(flight_id, chunk_id, iterations) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -324,7 +327,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. EMIT RecoveryStarted event │ │
│ │ └─ F02 subscribes → Update status to "recovering" │ │
│ │ └─ F02.2 updates status to "recovering" │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -381,7 +384,8 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ create_chunk(flight_id, start_frame_id) │ │
│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │
│ │ ├─ F10 create_chunk_subgraph(flight_id, chunk_id, │ │
│ │ │ start_frame_id) ← Factor graph subgraph │ │
│ │ ├─ Initialize chunk state (unanchored, active) │ │
│ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
@@ -390,16 +394,19 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ For each frame in chunk: │ │
│ │ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F07 compute_relative_pose_in_chunk()│ │ │
│ │ │ F07 compute_relative_pose() │ │ │
│ │ └───────────────────┬─────────────────┘ │ │
│ │ ▼ │ │
│ │ ┌──────────────────────────────────────┐ │ │
│ │ │ F12 add_frame_to_chunk() │ │ │
│ │ │ └─ F10 add_relative_factor_to_chunk│ │ │
│ │ │ (flight_id, chunk_id, frame_i, │ │ │
│ │ │ frame_j, relative_pose, cov) │ │ │
│ │ └───────────────────┬──────────────────┘ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F10 optimize_chunk() (local) │ │ │
│ │ │ F10 optimize_chunk(flight_id, │ │ │
│ │ │ chunk_id, iterations) (local) │ │ │
│ │ └─────────────────────────────────────┘ │ │
│ │ │ │
│ └────────────────────────────────────────────────────┘ │
@@ -481,7 +488,8 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 2. Anchor chunk: F12 mark_chunk_anchored(chunk_id, gps) │ │
│ │ └─ F10 add_chunk_anchor() │ │
│ │ └─ F10 add_chunk_anchor(flight_id, chunk_id, frame_id, │ │
│ │ gps, covariance) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -490,20 +498,22 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 4. Merge: F12 merge_chunks(chunk_id, target_chunk_id, Sim3) │ │
│ │ ├─ F10 merge_chunks() ← Apply Sim(3) transform │ │
│ │ 4. Merge: F12 merge_chunks(target_chunk_id, chunk_id, Sim3) │ │
│ │ ├─ F10 merge_chunk_subgraphs(flight_id, chunk_id, │ │
│ │ │ target_chunk_id, transform) ← Apply Sim(3) │ │
│ │ ├─ F12 deactivate_chunk(chunk_id) │ │
│ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 5. Optimize global: F10 optimize_global() │ │
│ │ 5. Optimize global: F10 optimize_global(flight_id, │ │
│ │ iterations) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 6. EMIT ChunkMerged event (flight_id,chunk_id,merged_frames)│ │
│ │ └─ F14 subscribes → update_results_after_chunk_merge() │ │
│ │ ├─ F10 get_trajectory() → ENU poses │ │
│ │ ├─ F10 get_trajectory(flight_id) → ENU poses │ │
│ │ ├─ F13 enu_to_gps() for each frame │ │
│ │ ├─ F03 save_frame_result() + update_waypoint() │ │
│ │ └─ F15 send_refinement() → SSE "frame_refined" × N │ │
@@ -532,7 +542,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ 3. Create UserInputRequest │ │
│ │ 4. F15 send_user_input_request() → SSE "user_input_needed" │ │
│ │ 5. EMIT UserInputNeeded event │ │
│ │ └─ F02 subscribes → Update status to "BLOCKED" │ │
│ │ └─ F02.2 updates status to "BLOCKED" │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│
@@ -561,7 +571,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ 2. F10 add_absolute_factor(is_user_anchor=True) ← σ=5m │ │
│ │ 3. F10 optimize() │ │
│ │ 4. EMIT UserFixApplied event │ │
│ │ └─ F02 subscribes → Update status to "PROCESSING" │ │
│ │ └─ F02.2 updates status to "PROCESSING" │ │
│ │ 5. Resume processing loop │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
@@ -591,7 +601,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ 3. Identify refined frames │ │
│ │ 4. F14 mark_refined(frame_ids) │ │
│ │ ├─ For each frame: │ │
│ │ │ ├─ F10 get_trajectory() → ENU pose │ │
│ │ │ ├─ F10 get_trajectory(flight_id) → ENU pose │ │
│ │ │ ├─ F13 enu_to_gps(flight_id, enu_pose) │ │
│ │ │ ├─ F03 save_frame_result(refined=True) │ │
│ │ │ └─ F03 update_waypoint() │ │
@@ -651,12 +661,13 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌──────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │
│ F02.1 Flight Lifecycle Manager │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. All frames processed │ │
│ │ 1. All frames processed (Signal from F02.2) │ │
│ │ 2. Wait for pending chunk merges (if any) │ │
│ │ 3. F10 optimize_global(iterations=100) ← Final optimization │ │
│ │ 3. F10 optimize_global(flight_id, iterations=100) │ │
│ │ ← Final optimization │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -699,7 +710,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
└────────┬────────┘
▼
┌─────────────────┐
│ F02 Complete/ │ ← Complete or cancel active flights
│ F02.1 Complete/ │ ← Complete or cancel active flights
│ Cancel │
└────────┬────────┘
▼
@@ -760,8 +771,13 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
┌─────────────────────────────────────────────────────────────────────────────┐
│ ORCHESTRATION LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F02 Flight Processor │ │
│ │ (Central coordinator, event subscriber, background task manager) │ │
│ │ F02.1 Flight Lifecycle Manager │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ (Spawns/Manages) │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F02.2 Flight Processing Engine │ │
│ │ (Processing loop, State machine, Recovery orchestration) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
@@ -772,7 +788,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ ┌─────────────┐ ┌─────────────┐ │ │ ┌───────────────────────┐ │
│ │ F04 │ │ F05 │ │ │ │ F11 │ │
│ │ Satellite │ │ Image │ │ │ │ Failure Recovery │ │
│ │ Data │ │ Input │ │ │ │ (Event emitter) │ │
│ │ Data │ │ Input │ │ │ │ (Logic & Strategies) │ │
│ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │
│ │ │ │ │
│ ┌─────────────┐ ┌─────────────┐ │ │ ▼ │
@@ -827,20 +843,18 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
---
## Event-Based Communication
## Status-Based Recovery Communication
F11 Failure Recovery Coordinator emits events instead of directly calling F02:
F11 Failure Recovery Coordinator uses **Direct Call Returns** to communicate with F02.2 Flight Processing Engine. Events have been removed to eliminate circular dependencies and clarify control flow.
| Event | Publisher | Subscribers | Action |
|-------|-----------|-------------|--------|
| `RecoveryStarted` | F11 | F02 | Update status to "recovering" |
| `RecoverySucceeded` | F11 | F02 | Update status to "processing", resume |
| `RecoveryFailed` | F11 | F02 | Update status to "blocked" |
| `UserInputNeeded` | F11 | F02 | Update status to "blocked", await fix |
| `UserFixApplied` | F11 | F02 | Update status to "processing", resume |
| `ChunkCreated` | F11 | F02 | Log chunk creation |
| `ChunkAnchored` | F11 | F02 | Log chunk anchor |
| `ChunkMerged` | F11 | F02, F14 | F14 updates results for merged frames |
| Action | Caller | Called | Return/Outcome |
|--------|--------|--------|----------------|
| `start_search` | F02.2 | F11 | `SearchSession` object |
| `try_current_grid` | F02.2 | F11 | `AlignmentResult` (or None) |
| `create_user_input_request` | F02.2 | F11 | `UserInputRequest` object |
| `apply_user_anchor` | F02.2 | F11 | Success boolean |
F02.2 is responsible for updating the flight status (e.g., "recovering", "blocked", "processing") based on these return values.
---
@@ -880,4 +894,3 @@ F11 Failure Recovery Coordinator emits events instead of directly calling F02:
| Mean Reprojection Error | < 1.0 pixels |
| Place Recognition Recall@5 | > 85% |
| LiteSAM Success Rate | > 95% (when rotation correct) |