azaion/gps-denied-desktop
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-desktop.git synced 2026-04-23 01:06:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

component assesment and fixes done

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-30 16:09:31 +02:00
parent a172b21aac
commit ce9760fcbe
22 changed files with 681 additions and 1844 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -1 +1,2 @@
.DS_Store .DS_Store
.idea
docs/02_components/02_flight_processor/flight_lifecycle_manager_spec.md
+146
View File
@@ -0,0 +1,146 @@
# Flight Lifecycle Manager
## Interface Definition
**Interface Name**: `IFlightLifecycleManager`
### Interface Methods
```python
class IFlightLifecycleManager(ABC):
# Flight Lifecycle
@abstractmethod
def create_flight(self, flight_data: FlightData) -> str:
pass
@abstractmethod
def get_flight(self, flight_id: str) -> Optional[Flight]:
pass
@abstractmethod
def get_flight_state(self, flight_id: str) -> Optional[FlightState]:
pass
@abstractmethod
def delete_flight(self, flight_id: str) -> bool:
pass
@abstractmethod
def update_flight_status(self, flight_id: str, status: FlightStatusUpdate) -> bool:
pass
# Waypoint Management
@abstractmethod
def update_waypoint(self, flight_id: str, waypoint_id: str, waypoint: Waypoint) -> bool:
pass
@abstractmethod
def batch_update_waypoints(self, flight_id: str, waypoints: List[Waypoint]) -> BatchUpdateResult:
pass
@abstractmethod
def get_flight_metadata(self, flight_id: str) -> Optional[FlightMetadata]:
pass
# Validation
@abstractmethod
def validate_waypoint(self, waypoint: Waypoint) -> ValidationResult:
pass
@abstractmethod
def validate_geofence(self, geofence: Geofences) -> ValidationResult:
pass
@abstractmethod
def validate_flight_continuity(self, waypoints: List[Waypoint]) -> ValidationResult:
pass
# API Delegation Methods (called by F01)
@abstractmethod
def queue_images(self, flight_id: str, batch: ImageBatch) -> BatchQueueResult:
"""Delegates to F05 Image Input Pipeline and triggers F02.2 Processing Engine."""
pass
@abstractmethod
def handle_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResult:
"""Delegates to F02.2 Processing Engine to apply fix."""
pass
@abstractmethod
def create_client_stream(self, flight_id: str, client_id: str) -> StreamConnection:
"""Delegates to F15 SSE Event Streamer."""
pass
@abstractmethod
def convert_object_to_gps(self, flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint:
"""Delegates to F13 Coordinate Transformer."""
pass
# System Initialization
@abstractmethod
def initialize_system(self) -> bool:
pass
```
## Component Description
### Responsibilities
- **Component ID**: F02.1
- **Flight Lifecycle**: Manage flight creation, persistence, and deletion.
- **System Entry Point**: Main interface for the REST API (F01).
- **Resource Management**: Initializes system components and manages `F02.2 Flight Processing Engine` instances.
- **Data Validation**: Validates inputs before processing.
- **Satellite Prefetching**: Triggers F04 prefetching on flight creation.
### Scope
- CRUD operations for Flights and Waypoints.
- Coordination of system startup.
- Hand-off of active processing tasks to F02.2.
- Does **NOT** contain the processing loop or recovery logic.
## API Methods
### `create_flight(flight_data: FlightData) -> str`
**Description**: Creates a new flight, validates data, sets ENU origin, and triggers satellite prefetching.
**Called By**: F01 Flight API
**Processing Flow**:
1. Generate flight_id.
2. Validate geofences and waypoints.
3. Call F13.set_enu_origin(flight_id, start_gps).
4. Trigger F04.prefetch_route_corridor().
5. Save to F03 Flight Database.
6. Return flight_id.
### `queue_images(flight_id: str, batch: ImageBatch) -> BatchQueueResult`
**Description**: Queues images and ensures a Processing Engine is active for this flight.
**Processing Flow**:
1. Delegate to F05.queue_batch().
2. Retrieve or create instance of `F02.2 Flight Processing Engine` for this flight.
3. Trigger `engine.start_processing()` (async).
4. Return result.
### `handle_user_fix(flight_id: str, fix_data: UserFixRequest) -> UserFixResult`
**Description**: Forwards user fix to the active processing engine.
**Processing Flow**:
1. Get active engine for flight_id.
2. Call `engine.apply_user_fix(fix_data)`.
3. Return result.
### `initialize_system() -> bool`
**Description**: startup routine.
**Processing Flow**:
1. Load config (F17).
2. Initialize models (F16).
3. Initialize DB (F03).
4. Initialize Cache (F04).
5. Load Indexes (F08).
## Dependencies
- **F02.2 Flight Processing Engine**: Managed child component.
- **F03 Flight Database**: Persistence.
- **F04 Satellite Data Manager**: Prefetching.
- **F05 Image Input Pipeline**: Image queuing.
- **F13 Coordinate Transformer**: ENU origin setting.
- **F15 SSE Event Streamer**: Stream creation.
- **F17 Configuration Manager**: Config loading.
docs/02_components/02_flight_processor/flight_processing_engine_spec.md
+108
View File
@@ -0,0 +1,108 @@
# Flight Processing Engine
## Interface Definition
**Interface Name**: `IFlightProcessingEngine`
### Interface Methods
```python
class IFlightProcessingEngine(ABC):
@abstractmethod
def start_processing(self, flight_id: str) -> None:
"""Starts the main processing loop in a background thread/task."""
pass
@abstractmethod
def stop_processing(self, flight_id: str) -> None:
pass
@abstractmethod
def process_frame(self, flight_id: str, frame_id: int) -> FrameResult:
pass
@abstractmethod
def apply_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResult:
pass
@abstractmethod
def handle_tracking_loss(self, flight_id: str, frame_id: int) -> RecoveryStatus:
pass
# Chunk Lifecycle (Delegates to F12 but orchestrated here)
@abstractmethod
def get_active_chunk(self, flight_id: str) -> Optional[ChunkHandle]:
pass
@abstractmethod
def create_new_chunk(self, flight_id: str, frame_id: int) -> ChunkHandle:
pass
```
## Component Description
### Responsibilities
- **Component ID**: F02.2
- **Processing Orchestration**: Runs the main loop: Image -> VO -> Graph -> Result.
- **State Machine**: Manages flight status (Processing, Blocked, Recovering, Completed).
- **Recovery Coordination**: Calls F11 methods and acts on return values.
- **Chunk Management**: Calls F12 to manage chunk lifecycle based on tracking status.
- **Background Tasks**: Manages background chunk matching tasks.
### Scope
- Per-flight processing logic.
- Interaction with Visual Pipeline (F06, F07, F09).
- Interaction with State Estimation (F10).
- Interaction with Recovery (F11).
- Result Publishing (F14, F15).
## Processing Flow
### `run_processing_loop(flight_id: str)`
1. **Loop**: While `F05.get_next_image()` returns image:
2. **Chunk Check**:
* `active_chunk = F12.get_active_chunk()`
* If `detect_chunk_boundary()`: `active_chunk = F12.create_chunk()`.
3. **Visual Odometry**:
* Call `F06.requires_rotation_sweep()`.
* Call `F07.compute_relative_pose()`.
4. **Tracking Check**:
* If tracking good:
* `F12.add_frame_to_chunk()`.
* `F09.align_to_satellite()` (optional drift correction).
* `F10.add_relative_factor()`.
* `F10.optimize_chunk()`.
* `F14.update_frame_result()`.
* If tracking lost:
* **Proactive Chunking**: `F11.create_chunk_on_tracking_loss()`.
* **Recovery**: Call `handle_tracking_loss()`.
### `handle_tracking_loss(flight_id, frame_id)`
1. Call `F11.start_search()`.
2. Loop progressive search (1..25 tiles):
* `result = F11.try_current_grid()`.
* If found: `F11.mark_found()`, break.
* If not found: `F11.expand_search_radius()`.
3. If still not found:
* `req = F11.create_user_input_request()`.
* `F15.send_user_input_request(req)`.
* Set status to **BLOCKED**.
* Wait for `apply_user_fix`.
### `apply_user_fix(fix_data)`
1. Call `F11.apply_user_anchor(fix_data)`.
2. If success:
* Set status to **PROCESSING**.
* Resume loop.
## Dependencies
- **F05 Image Input Pipeline**: Image source.
- **F06 Image Rotation Manager**: Pre-processing.
- **F07 Sequential Visual Odometry**: Motion estimation.
- **F09 Metric Refinement**: Satellite alignment.
- **F10 Factor Graph Optimizer**: State estimation.
- **F11 Failure Recovery Coordinator**: Recovery logic.
- **F12 Route Chunk Manager**: Chunk state.
- **F14 Result Manager**: Saving results.
- **F15 SSE Event Streamer**: Real-time updates.
docs/02_components/02_flight_processor/flight_processor_spec.md
-1285
View File
File diff suppressed because it is too large Load Diff
docs/02_components/03_flight_database/flight_database_spec.md
+30 -15
View File
@@ -8,6 +8,12 @@
```python ```python
class IFlightDatabase(ABC): class IFlightDatabase(ABC):
# Transaction Support
@abstractmethod
def execute_transaction(self, operations: List[Callable]) -> bool:
"""Executes a list of DB operations atomically."""
pass
# Flight Operations # Flight Operations
@abstractmethod @abstractmethod
def insert_flight(self, flight: Flight) -> str: def insert_flight(self, flight: Flight) -> str:
@@ -111,6 +117,8 @@ class IFlightDatabase(ABC):
## Component Description ## Component Description
### Responsibilities ### Responsibilities
- Persistence layer.
- **Consistency**: Implements explicit transaction support to ensure `waypoints` and `frame_results` (and other related tables) stay synchronized when updated by F14 Result Manager.
- Direct database access layer for all flight-related data - Direct database access layer for all flight-related data
- Execute SQL queries and commands - Execute SQL queries and commands
- Manage database connections and transactions - Manage database connections and transactions
@@ -120,6 +128,10 @@ class IFlightDatabase(ABC):
- Store heading history for rotation management - Store heading history for rotation management
- Store image file paths and metadata - Store image file paths and metadata
### Transactional Integrity
- **Atomic Updates**: Critical for preventing partial state updates during chunk merges or frame refinements.
- **Pattern**: Usage of database transactions (BEGIN, COMMIT, ROLLBACK) for batch operations.
### Scope ### Scope
- CRUD operations on flights table - CRUD operations on flights table
- CRUD operations on waypoints table - CRUD operations on waypoints table
@@ -154,7 +166,7 @@ The schema uses strategic denormalization to optimize for the most common access
**Description**: Inserts a new flight with initial waypoints and geofences. **Description**: Inserts a new flight with initial waypoints and geofences.
**Called By**: **Called By**:
- F02 Flight Processor - F02.1 Flight Lifecycle Manager
**Input**: **Input**:
```python ```python
@@ -201,7 +213,7 @@ flight_id: str
**Description**: Updates flight metadata. **Description**: Updates flight metadata.
**Called By**: **Called By**:
- F02 Flight Processor - F02.1 Flight Lifecycle Manager
**Input**: **Input**:
```python ```python
@@ -231,7 +243,7 @@ WHERE id = ?
**Description**: Queries flights with filtering and pagination. **Description**: Queries flights with filtering and pagination.
**Called By**: **Called By**:
- F02 Flight Processor (listing) - F02.1 Flight Lifecycle Manager (listing)
- F01 Flight API - F01 Flight API
**Input**: **Input**:
@@ -258,7 +270,7 @@ List[Flight] # Metadata only, without full waypoint data
**Description**: Retrieves complete flight with all waypoints. **Description**: Retrieves complete flight with all waypoints.
**Called By**: **Called By**:
- F02 Flight Processor - F02.1 Flight Lifecycle Manager
**Input**: **Input**:
```python ```python
@@ -288,7 +300,7 @@ Optional[Flight] # Complete flight with all waypoints
**Description**: Deletes a flight and cascades to all related data. **Description**: Deletes a flight and cascades to all related data.
**Called By**: **Called By**:
- F02 Flight Processor - F02.1 Flight Lifecycle Manager
**Input**: **Input**:
```python ```python
@@ -321,7 +333,7 @@ DELETE FROM flights WHERE id = ?
**Description**: Retrieves waypoints for a flight. **Description**: Retrieves waypoints for a flight.
**Called By**: **Called By**:
- F02 Flight Processor - F02.1 Flight Lifecycle Manager
**Input**: **Input**:
```python ```python
@@ -345,7 +357,7 @@ List[Waypoint]
**Description**: Inserts a new waypoint. **Description**: Inserts a new waypoint.
**Called By**: **Called By**:
- F02 Flight Processor - F02.1 Flight Lifecycle Manager
**Input**: **Input**:
```python ```python
@@ -369,7 +381,7 @@ waypoint_id: str
**Description**: Updates a waypoint. Critical path for GPS refinement updates. **Description**: Updates a waypoint. Critical path for GPS refinement updates.
**Called By**: **Called By**:
- F02 Flight Processor - F02.1 Flight Lifecycle Manager
- F14 Result Manager - F14 Result Manager
**Input**: **Input**:
@@ -408,7 +420,7 @@ WHERE id = ? AND flight_id = ?
**Description**: Updates multiple waypoints in a single transaction. **Description**: Updates multiple waypoints in a single transaction.
**Called By**: **Called By**:
- F02 Flight Processor (asynchronous refinements) - F02.2 Flight Processing Engine (asynchronous refinements)
**Input**: **Input**:
```python ```python
@@ -437,7 +449,7 @@ BatchResult:
**Description**: Saves or updates flight processing state. **Description**: Saves or updates flight processing state.
**Called By**: **Called By**:
- F02 Flight Processor - F02.2 Flight Processing Engine
**Input**: **Input**:
```python ```python
@@ -447,13 +459,14 @@ FlightState:
frames_processed: int frames_processed: int
frames_total: int frames_total: int
current_frame: Optional[int] current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool blocked: bool
search_grid_size: Optional[int] search_grid_size: Optional[int]
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
``` ```
**Note**: Heading is NOT stored in flight_state. Use the dedicated `heading_history` table via `save_heading()` and `get_latest_heading()` for all heading operations. This avoids data duplication and ensures a single source of truth for heading data.
**Output**: **Output**:
```python ```python
bool: True if saved bool: True if saved
@@ -470,7 +483,7 @@ bool: True if saved
**Description**: Loads flight state (for crash recovery). **Description**: Loads flight state (for crash recovery).
**Called By**: **Called By**:
- F02 Flight Processor - F02.2 Flight Processing Engine
**Output**: **Output**:
```python ```python
@@ -548,7 +561,7 @@ bool: True if saved
**Description**: Saves heading value for temporal smoothing and recovery. **Description**: Saves heading value for temporal smoothing and recovery.
**Called By**: **Called By**:
- F02 Flight Processor (after F06.update_heading() returns, F02 persists to F03) - F02.2 Flight Processing Engine (after F06.update_heading() returns, F02 persists to F03)
**Input**: **Input**:
```python ```python
@@ -884,13 +897,14 @@ CREATE TABLE geofences (
); );
-- Flight state table -- Flight state table
-- NOTE: Heading is NOT stored here. Use heading_history table for heading data.
-- This avoids duplication and ensures single source of truth.
CREATE TABLE flight_state ( CREATE TABLE flight_state (
flight_id VARCHAR(36) PRIMARY KEY, flight_id VARCHAR(36) PRIMARY KEY,
status VARCHAR(50) NOT NULL, status VARCHAR(50) NOT NULL,
frames_processed INT NOT NULL DEFAULT 0, frames_processed INT NOT NULL DEFAULT 0,
frames_total INT NOT NULL DEFAULT 0, frames_total INT NOT NULL DEFAULT 0,
current_frame INT, current_frame INT,
current_heading FLOAT,
blocked BOOLEAN NOT NULL DEFAULT FALSE, blocked BOOLEAN NOT NULL DEFAULT FALSE,
search_grid_size INT, search_grid_size INT,
created_at TIMESTAMP NOT NULL, created_at TIMESTAMP NOT NULL,
@@ -993,11 +1007,12 @@ class FlightState(BaseModel):
frames_processed: int frames_processed: int
frames_total: int frames_total: int
current_frame: Optional[int] current_frame: Optional[int]
current_heading: Optional[float]
blocked: bool blocked: bool
search_grid_size: Optional[int] search_grid_size: Optional[int]
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
# NOTE: Heading is NOT stored in FlightState.
# Use get_latest_heading(flight_id) from heading_history table.
``` ```
### FrameResult ### FrameResult
docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md
+2 -2
View File
@@ -167,7 +167,7 @@ GET /api/satellite/tiles/batch?tiles=[...]
**Description**: Prefetches satellite tiles along route corridor for a flight. **Description**: Prefetches satellite tiles along route corridor for a flight.
**Called By**: **Called By**:
- F02 Flight Manager (during flight creation) - F02.1 Flight Lifecycle Manager (during flight creation)
**Input**: **Input**:
```python ```python
@@ -454,7 +454,7 @@ TileBounds:
**Description**: Clears cached tiles for a completed flight. **Description**: Clears cached tiles for a completed flight.
**Called By**: **Called By**:
- F02 Flight Processor (cleanup after flight completion) - F02.1 Flight Lifecycle Manager (cleanup after flight completion)
**Input**: **Input**:
```python ```python
docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md
+7 -7
View File
@@ -65,7 +65,7 @@ class IImageInputPipeline(ABC):
**Description**: Queues a batch of images for processing (FIFO). **Description**: Queues a batch of images for processing (FIFO).
**Called By**: **Called By**:
- F01 GPS-Denied REST API (after upload) - F02.1 Flight Lifecycle Manager (via F01 Flight API image upload route)
**Input**: **Input**:
```python ```python
@@ -224,9 +224,9 @@ bool: True if stored successfully
**Description**: Gets the next image in sequence for processing. **Description**: Gets the next image in sequence for processing.
**Called By**: **Called By**:
- F06 Image Rotation Manager - F02.2 Flight Processing Engine (main processing loop)
- F07 Sequential VO - F06 Image Rotation Manager (via F02.2)
- Processing pipeline (main loop) - F07 Sequential VO (via F02.2)
**Input**: **Input**:
```python ```python
@@ -299,7 +299,7 @@ Optional[ImageData]
**Description**: Retrieves metadata without loading full image (lightweight). **Description**: Retrieves metadata without loading full image (lightweight).
**Called By**: **Called By**:
- F02 Flight Manager (status checks) - F02.1 Flight Lifecycle Manager (status checks)
- F13 Result Manager (metadata-only queries) - F13 Result Manager (metadata-only queries)
**Input**: **Input**:
@@ -330,8 +330,8 @@ ImageMetadata:
**Description**: Gets current processing status for a flight. **Description**: Gets current processing status for a flight.
**Called By**: **Called By**:
- F01 Flight API (status endpoint) - F02.1 Flight Lifecycle Manager (status queries via F01 Flight API)
- F02 Flight Processor - F02.2 Flight Processing Engine (processing loop status)
**Input**: **Input**:
```python ```python
docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md
+29 -19
View File
@@ -13,7 +13,11 @@ class IImageRotationManager(ABC):
pass pass
@abstractmethod @abstractmethod
def try_rotation_steps(self, flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime) -> Optional[RotationResult]: def try_rotation_steps(self, flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime, matcher: IImageMatcher) -> Optional[RotationResult]:
"""
Performs rotation sweep.
'matcher' is an injected dependency (usually F09) to avoid direct coupling.
"""
pass pass
@abstractmethod @abstractmethod
@@ -41,22 +45,27 @@ class IImageRotationManager(ABC):
pass pass
@abstractmethod @abstractmethod
def try_chunk_rotation_steps(self, chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[RotationResult]: def try_chunk_rotation_steps(self, chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds, matcher: IImageMatcher) -> Optional[RotationResult]:
pass pass
``` ```
## Component Description ## Component Description
### Responsibilities ### Responsibilities
- Handle UAV image rotation preprocessing - Image rotation utility.
- **Critical**: LiteSAM (F09 Metric Refinement) fails if images rotated >45°, requires preprocessing - Heading tracking.
- Perform 30° step rotation sweeps (12 rotations: 0°, 30°, 60°, ..., 330°) - Coordination of rotation sweeps.
- Track UAV heading angle across flight
- Calculate precise rotation angle from homography point correspondences ### Decoupling Fix
- Detect sharp turns requiring rotation sweep - **Problem**: F06 previously depended directly on `F09 Metric Refinement`.
- Pre-rotate images to known heading for subsequent frames - **Fix**: Methods `try_rotation_steps` and `try_chunk_rotation_steps` now accept a `matcher` argument conforming to `IImageMatcher`.
- **Chunk rotation operations (rotate all images in chunk)** - **IImageMatcher Interface**:
- **Chunk rotation sweeps (delegates matching to F09 Metric Refinement)** ```python
class IImageMatcher(ABC):
def align_to_satellite(self, uav_image, satellite_tile, tile_bounds) -> AlignmentResult: pass
def align_chunk_to_satellite(self, chunk_images, satellite_tile, tile_bounds) -> ChunkAlignmentResult: pass
```
- **Runtime**: F02.2 injects F09 instance when calling F06 methods.
### Scope ### Scope
- Image rotation operations (pure rotation, no matching) - Image rotation operations (pure rotation, no matching)
@@ -103,7 +112,7 @@ np.ndarray # Rotated image (same dimensions)
--- ---
### `try_rotation_steps(flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime) -> Optional[RotationResult]` ### `try_rotation_steps(flight_id: str, frame_id: int, image: np.ndarray, satellite_tile: np.ndarray, tile_bounds: TileBounds, timestamp: datetime, matcher: IImageMatcher) -> Optional[RotationResult]`
**Description**: Performs 30° rotation sweep, rotating image at each step and delegating matching to F09 Metric Refinement. **Description**: Performs 30° rotation sweep, rotating image at each step and delegating matching to F09 Metric Refinement.
@@ -119,6 +128,7 @@ image: np.ndarray # UAV image
satellite_tile: np.ndarray # Satellite reference tile satellite_tile: np.ndarray # Satellite reference tile
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (passed to F09) tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (passed to F09)
timestamp: datetime # Timestamp for heading persistence timestamp: datetime # Timestamp for heading persistence
matcher: IImageMatcher # Injected matcher (F09)
``` ```
**About tile_bounds**: `TileBounds` contains the GPS bounding box of the satellite tile: **About tile_bounds**: `TileBounds` contains the GPS bounding box of the satellite tile:
@@ -142,7 +152,7 @@ RotationResult:
``` ```
For angle in [0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°]: For angle in [0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°]:
rotated_image = rotate_image_360(image, angle) rotated_image = rotate_image_360(image, angle)
result = F09.align_to_satellite(rotated_image, satellite_tile, tile_bounds) result = matcher.align_to_satellite(rotated_image, satellite_tile, tile_bounds)
if result.matched and result.confidence > threshold: if result.matched and result.confidence > threshold:
precise_angle = calculate_precise_angle(result.homography, angle) precise_angle = calculate_precise_angle(result.homography, angle)
update_heading(flight_id, frame_id, precise_angle, timestamp) update_heading(flight_id, frame_id, precise_angle, timestamp)
@@ -153,7 +163,7 @@ return None # No match found
**Processing Flow**: **Processing Flow**:
1. For each 30° step: 1. For each 30° step:
- Rotate image via rotate_image_360() - Rotate image via rotate_image_360()
- Call F09 Metric Refinement.align_to_satellite(rotated_image, satellite_tile, tile_bounds) - Call matcher.align_to_satellite(rotated_image, satellite_tile, tile_bounds)
- Check if match found - Check if match found
2. If match found: 2. If match found:
- Calculate precise angle from homography via calculate_precise_angle() - Calculate precise angle from homography via calculate_precise_angle()
@@ -379,7 +389,7 @@ List[np.ndarray] # Rotated images (same dimensions)
--- ---
### `try_chunk_rotation_steps(chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds) -> Optional[RotationResult]` ### `try_chunk_rotation_steps(chunk_images: List[np.ndarray], satellite_tile: np.ndarray, tile_bounds: TileBounds, matcher: IImageMatcher) -> Optional[RotationResult]`
**Description**: Performs 30° rotation sweep on entire chunk, rotating all images at each step and delegating matching to F09 Metric Refinement. **Description**: Performs 30° rotation sweep on entire chunk, rotating all images at each step and delegating matching to F09 Metric Refinement.
@@ -391,6 +401,7 @@ List[np.ndarray] # Rotated images (same dimensions)
chunk_images: List[np.ndarray] # Chunk images chunk_images: List[np.ndarray] # Chunk images
satellite_tile: np.ndarray # Reference satellite tile satellite_tile: np.ndarray # Reference satellite tile
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (for F09) tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (for F09)
matcher: IImageMatcher # Injected matcher
``` ```
**Output**: **Output**:
@@ -407,7 +418,7 @@ RotationResult:
``` ```
For angle in [0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°]: For angle in [0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°]:
rotated_chunk = rotate_chunk_360(chunk_images, angle) rotated_chunk = rotate_chunk_360(chunk_images, angle)
result = F09.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds) result = matcher.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds)
if result.matched and result.confidence > threshold: if result.matched and result.confidence > threshold:
precise_angle = calculate_precise_angle(result.homography, angle) precise_angle = calculate_precise_angle(result.homography, angle)
return RotationResult(matched=True, initial_angle=angle, precise_angle=precise_angle, ...) return RotationResult(matched=True, initial_angle=angle, precise_angle=precise_angle, ...)
@@ -417,7 +428,7 @@ return None # No match found
**Processing Flow**: **Processing Flow**:
1. For each 30° step: 1. For each 30° step:
- Rotate all chunk images via rotate_chunk_360() - Rotate all chunk images via rotate_chunk_360()
- Call F09 Metric Refinement.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds) - Call matcher.align_chunk_to_satellite(rotated_chunk, satellite_tile, tile_bounds)
- Check if match found - Check if match found
2. If match found: 2. If match found:
- Calculate precise angle from homography via calculate_precise_angle() - Calculate precise angle from homography via calculate_precise_angle()
@@ -497,8 +508,8 @@ return None # No match found
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching.
- **H07 Image Rotation Utils**: For image rotation and angle calculations - **H07 Image Rotation Utils**: For image rotation and angle calculations
- **Injected Matcher (F09)**.
**Note**: **Note**:
- `TileBounds` data model is imported from F09 Metric Refinement. - `TileBounds` data model is imported from F09 Metric Refinement.
@@ -541,4 +552,3 @@ class RotationConfig(BaseModel):
confidence_threshold: float = 0.7 # For accepting match confidence_threshold: float = 0.7 # For accepting match
history_size: int = 10 # Number of headings to track history_size: int = 10 # Number of headings to track
``` ```
docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md
+24 -9
View File
@@ -25,7 +25,7 @@ class ISequentialVisualOdometry(ABC):
pass pass
``` ```
**Note**: F07 is chunk-agnostic. It only computes relative poses between images. The caller (F02 Flight Processor) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10. **Note**: F07 is chunk-agnostic. It only computes relative poses between images. The caller (F02.2 Flight Processing Engine) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
## Component Description ## Component Description
@@ -42,7 +42,7 @@ class ISequentialVisualOdometry(ABC):
- Feature-based motion estimation - Feature-based motion estimation
- Handles low overlap and challenging agricultural environments - Handles low overlap and challenging agricultural environments
- Provides relative measurements for trajectory optimization - Provides relative measurements for trajectory optimization
- **Chunk-agnostic**: F07 doesn't know about chunks. Caller (F02) routes results to appropriate chunk subgraph. - **Chunk-agnostic**: F07 doesn't know about chunks. Caller (F02.2) routes results to appropriate chunk subgraph.
## API Methods ## API Methods
@@ -213,6 +213,21 @@ Motion:
- GSD-based expected displacement calculations (via H02) - GSD-based expected displacement calculations (via H02)
- Absolute GPS anchors from F09 Metric Refinement - Absolute GPS anchors from F09 Metric Refinement
**Critical Handoff to F10**:
The caller (F02.2) must pass the unit translation to F10 for scale resolution:
```python
# F02.2 receives RelativePose from F07
vo_result = F07.compute_relative_pose(prev_image, curr_image)
# vo_result.translation is a UNIT VECTOR (||t|| = 1)
# F02.2 passes to F10 which scales using:
# 1. altitude = F17.get_operational_altitude(flight_id)
# 2. gsd = H02.compute_gsd(altitude, camera_params)
# 3. expected_displacement = frame_spacing * gsd
# 4. scaled_translation = vo_result.translation * expected_displacement
F10.add_relative_factor(flight_id, frame_i, frame_j, vo_result, covariance)
```
**Error Conditions**: **Error Conditions**:
- Returns `None`: Insufficient inliers (< 8 points for Essential Matrix) - Returns `None`: Insufficient inliers (< 8 points for Essential Matrix)
@@ -246,12 +261,12 @@ Motion:
2. compute_relative_pose() → SuperPoint handles better than SIFT 2. compute_relative_pose() → SuperPoint handles better than SIFT
3. Verify match quality 3. Verify match quality
### Test 5: Chunk-Aware VO ### Test 5: VO with Chunk Routing
1. Create chunk_1 and chunk_2 1. Create chunk_1 and chunk_2 via F12
2. compute_relative_pose_in_chunk() for frames in chunk_1 2. compute_relative_pose() for frames in chunk_1, F02.2 routes to chunk_1
3. compute_relative_pose_in_chunk() for frames in chunk_2 3. compute_relative_pose() for frames in chunk_2, F02.2 routes to chunk_2
4. Verify factors added to respective chunks 4. Verify F02.2 calls F12.add_frame_to_chunk() with correct chunk_id
5. Verify chunks optimized independently 5. Verify chunks optimized independently via F10
## Non-Functional Requirements ## Non-Functional Requirements
@@ -280,7 +295,7 @@ Motion:
- **H01 Camera Model**: For coordinate normalization - **H01 Camera Model**: For coordinate normalization
- **H05 Performance Monitor**: For timing measurements - **H05 Performance Monitor**: For timing measurements
**Note**: F07 is chunk-agnostic and does NOT depend on F10 Factor Graph Optimizer. F07 only computes relative poses between images and returns them to the caller (F02). The caller (F02) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10. **Note**: F07 is chunk-agnostic and does NOT depend on F10 Factor Graph Optimizer. F07 only computes relative poses between images and returns them to the caller (F02.2). The caller (F02.2) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
### External Dependencies ### External Dependencies
- **SuperPoint**: Feature extraction model - **SuperPoint**: Feature extraction model
docs/02_components/08_global_place_recognition/global_place_recognition_spec.md
+5 -5
View File
@@ -208,7 +208,7 @@ List[TileCandidate] # Re-ranked list
**Description**: Loads pre-built satellite descriptor database from file. **Note**: The semantic index (DINOv2 descriptors + Faiss index) MUST be provided by the satellite data provider. F08 does NOT build the index - it only loads it. **Description**: Loads pre-built satellite descriptor database from file. **Note**: The semantic index (DINOv2 descriptors + Faiss index) MUST be provided by the satellite data provider. F08 does NOT build the index - it only loads it.
**Called By**: **Called By**:
- F02 Flight Processor (during flight initialization, index_path from F04 Satellite Data Manager) - F02.1 Flight Lifecycle Manager (during flight initialization, index_path from F04 Satellite Data Manager)
**Input**: **Input**:
```python ```python
@@ -380,10 +380,10 @@ np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- **F16 Model Manager**: For DINOv2 model - **F16 Model Manager**: For DINOv2 inference engine via `get_inference_engine("DINOv2")`.
- **H04 Faiss Index Manager**: For similarity search - **H04 Faiss Index Manager**: For similarity search via `load_index()`, `search()`. Critical for `query_database()`.
- **F04 Satellite Data Manager**: For tile metadata - **F04 Satellite Data Manager**: For tile metadata retrieval after Faiss search returns tile indices.
- **F12 Route Chunk Manager**: For chunk image retrieval - **F12 Route Chunk Manager**: For chunk image retrieval during chunk descriptor computation.
### External Dependencies ### External Dependencies
- **DINOv2**: Foundation vision model - **DINOv2**: Foundation vision model
docs/02_components/09_metric_refinement/metric_refinement_spec.md
+2 -2
View File
@@ -62,7 +62,7 @@ class IMetricRefinement(ABC):
**Called By**: **Called By**:
- F06 Image Rotation Manager (during rotation sweep) - F06 Image Rotation Manager (during rotation sweep)
- F11 Failure Recovery Coordinator (progressive search) - F11 Failure Recovery Coordinator (progressive search)
- F02 Flight Processor (drift correction with single tile) - F02.2 Flight Processing Engine (drift correction with single tile)
**Input**: **Input**:
```python ```python
@@ -389,7 +389,7 @@ Optional[np.ndarray]: 3×3 homography matrix or None
- If rotation >45°, F09 will fail to match (by design) - If rotation >45°, F09 will fail to match (by design)
- F06 handles the rotation sweep (trying 0°, 30°, 60°, etc.) and calls F09 for each rotation - F06 handles the rotation sweep (trying 0°, 30°, 60°, etc.) and calls F09 for each rotation
**Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager) **Note**: tile_bounds is passed as parameter from caller (F02.2 Flight Processing Engine gets it from F04 Satellite Data Manager)
### External Dependencies ### External Dependencies
- **LiteSAM**: Cross-view matching model - **LiteSAM**: Cross-view matching model
docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md
+13 -2
View File
@@ -8,8 +8,19 @@
```python ```python
class IFactorGraphOptimizer(ABC): class IFactorGraphOptimizer(ABC):
# All methods take flight_id to support concurrent flights """
# F10 maintains Dict[str, FactorGraph] keyed by flight_id internally GTSAM-based factor graph optimizer for trajectory estimation.
## Multi-Flight Support
All state-modifying methods require `flight_id` parameter.
Each flight maintains an independent factor graph state.
F10 internally manages Dict[str, FactorGraph] keyed by flight_id.
This enables:
- Concurrent processing of multiple flights
- Flight-scoped optimization without cross-contamination
- Independent cleanup via delete_flight_graph()
"""
@abstractmethod @abstractmethod
def add_relative_factor(self, flight_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool: def add_relative_factor(self, flight_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md
+44 -63
View File
@@ -8,6 +8,7 @@
```python ```python
class IFailureRecoveryCoordinator(ABC): class IFailureRecoveryCoordinator(ABC):
# Status Checks
@abstractmethod @abstractmethod
def check_confidence(self, vo_result: RelativePose, litesam_result: Optional[AlignmentResult]) -> ConfidenceAssessment: def check_confidence(self, vo_result: RelativePose, litesam_result: Optional[AlignmentResult]) -> ConfidenceAssessment:
pass pass
@@ -16,6 +17,7 @@ class IFailureRecoveryCoordinator(ABC):
def detect_tracking_loss(self, confidence: ConfidenceAssessment) -> bool: def detect_tracking_loss(self, confidence: ConfidenceAssessment) -> bool:
pass pass
# Search & Recovery (Synchronous returns, NO EVENTS)
@abstractmethod @abstractmethod
def start_search(self, flight_id: str, frame_id: int, estimated_gps: GPSPoint) -> SearchSession: def start_search(self, flight_id: str, frame_id: int, estimated_gps: GPSPoint) -> SearchSession:
pass pass
@@ -38,12 +40,14 @@ class IFailureRecoveryCoordinator(ABC):
@abstractmethod @abstractmethod
def create_user_input_request(self, flight_id: str, frame_id: int, candidate_tiles: List[TileCandidate]) -> UserInputRequest: def create_user_input_request(self, flight_id: str, frame_id: int, candidate_tiles: List[TileCandidate]) -> UserInputRequest:
"""Returns request object. Caller (F02.2) is responsible for sending to F15."""
pass pass
@abstractmethod @abstractmethod
def apply_user_anchor(self, flight_id: str, frame_id: int, anchor: UserAnchor) -> bool: def apply_user_anchor(self, flight_id: str, frame_id: int, anchor: UserAnchor) -> bool:
pass pass
# Chunk Recovery
@abstractmethod @abstractmethod
def create_chunk_on_tracking_loss(self, flight_id: str, frame_id: int) -> ChunkHandle: def create_chunk_on_tracking_loss(self, flight_id: str, frame_id: int) -> ChunkHandle:
pass pass
@@ -62,63 +66,52 @@ class IFailureRecoveryCoordinator(ABC):
@abstractmethod @abstractmethod
def process_unanchored_chunks(self, flight_id: str) -> None: def process_unanchored_chunks(self, flight_id: str) -> None:
"""Background task logic. Should be invoked/managed by F02.2."""
pass pass
``` ```
## Component Description ## Component Description
### Responsibilities ### Responsibilities
- **Pure Logic Component**: Coordinates recovery strategies but delegates execution and communication to F02.2.
- **No Events**: Returns status objects or booleans. F02.2 decides state transitions based on these returns.
- **Chunk Orchestration**: Coordinates F12 and F10 operations during recovery but does not own the thread/task.
- Monitor confidence metrics (inlier count, MRE, covariance) - Monitor confidence metrics (inlier count, MRE, covariance)
- Detect tracking loss and trigger recovery - Detect tracking loss and trigger recovery
- Coordinate progressive tile search (1→4→9→16→25) - Coordinate progressive tile search (1→4→9→16→25)
- Handle human-in-the-loop when all strategies exhausted - Handle human-in-the-loop when all strategies exhausted
- **Emit recovery events** (RecoveryStarted, RecoveryFailed, RecoverySucceeded, UserInputNeeded)
- Apply user-provided anchors to Factor Graph - Apply user-provided anchors to Factor Graph
- **Proactive chunk creation on tracking loss** (via F12) - **Proactive chunk creation on tracking loss** (via F12)
- **Chunk LiteSAM matching with rotation sweeps** - **Chunk LiteSAM matching with rotation sweeps**
- **Chunk merging orchestration** - **Chunk merging orchestration**
- **Background chunk matching processing** - **Background chunk matching processing**
### Chunk Responsibility Clarification ### Interaction Pattern (Direct Calls)
**Caller**: F02.2 Flight Processing Engine
**F11 coordinates chunk-based recovery workflow only**: 1. **Tracking Loss**:
- Detects when to create chunks (on tracking loss) * F02.2 calls `start_search()`.
- Calls F12 for ALL chunk operations (F11 NEVER directly calls F10 chunk methods) * F02.2 calls `try_current_grid()` iteratively.
- Coordinates matching workflows * If found, F02.2 calls `mark_found()`.
- Emits chunk-related events (ChunkCreated, ChunkAnchored, ChunkMerged) * If exhausted, F02.2 calls `create_user_input_request()` -> gets `Request` object -> F02.2 calls F15.
**F12 is the source of truth for chunk state** (see F12 spec) 2. **Chunk Matching**:
* F02.2 calls `create_chunk_on_tracking_loss()`.
**F10 provides low-level factor graph operations** (see F10 spec) * F02.2 (background task) calls `try_chunk_semantic_matching()`.
* F02.2 calls `try_chunk_litesam_matching()`.
### Direct Call Communication * F02.2 calls `merge_chunk_to_trajectory()`.
F11 is called directly by F02 and returns results synchronously. F02 is responsible for updating flight status based on F11's return values. This provides clear data flow and simpler debugging.
**F02 calls F11 and handles responses**:
- `start_search()` → returns `SearchSession`, F02 updates status to "recovering"
- `try_current_grid()` → returns `Optional[AlignmentResult]`, F02 checks if found
- `mark_found()` → F02 updates status to "processing"
- `create_user_input_request()` → returns `UserInputRequest`, F02 updates status to "blocked" and sends via F15
- `apply_user_anchor()` → returns `bool`, F02 updates status to "processing" on success
- `create_chunk_on_tracking_loss()` → returns `ChunkHandle`, F02 tracks chunk
- `try_chunk_semantic_matching()` → returns `Optional[List[TileCandidate]]`
- `try_chunk_litesam_matching()` → returns `Optional[ChunkAlignmentResult]`
- `merge_chunk_to_trajectory()` → returns `bool`, F02 calls F14.update_results_after_chunk_merge()
**Note**: F11 does NOT emit events. All communication is through direct method calls with return values. F02 orchestrates the recovery flow by calling F11 methods and handling responses.
### External SSE Events (to Clients) ### External SSE Events (to Clients)
F11 does NOT directly send events to clients. F02 orchestrates client communication based on F11's return values: F11 does NOT directly send events to clients. F02.2 orchestrates client communication based on F11's return values:
- When F11.create_user_input_request() returns `UserInputRequest` → F02 calls F15.send_user_input_request() - When F11.create_user_input_request() returns `UserInputRequest` → F02.2 calls F15.send_user_input_request()
- When F11.merge_chunk_to_trajectory() returns True → F02 calls F14.update_results_after_chunk_merge() - When F11.merge_chunk_to_trajectory() returns True → F02.2 calls F14.update_results_after_chunk_merge()
- When recovery fails (all F11 methods return None/False) → F02 calls F15.send_processing_blocked() - When recovery fails (all F11 methods return None/False) → F02.2 calls F15.send_processing_blocked()
This separation ensures: This separation ensures:
1. F11 is a pure business logic component (no I/O dependencies) 1. F11 is a pure business logic component (no I/O dependencies)
2. F02 controls all coordination and state management 2. F02.2 controls all coordination and state management
3. F14/F15 handle all client-facing communication 3. F14/F15 handle all client-facing communication
### Scope ### Scope
@@ -341,12 +334,11 @@ UserInputRequest:
1. Get UAV image for frame_id 1. Get UAV image for frame_id
2. Get top-5 candidates from F08 2. Get top-5 candidates from F08
3. Create request 3. Create request
4. Send via F15 SSE → "user_input_needed" event 4. Return Request object (Caller F02.2 handles sending)
5. Emit `UserInputNeeded` event (F02 subscribes and updates status to "BLOCKED")
**Test Cases**: **Test Cases**:
1. All search failed → creates request 1. All search failed → creates request
2. Request sent to client via SSE 2. Request returned to caller
--- ---
@@ -371,8 +363,7 @@ anchor: UserAnchor:
1. Validate anchor data 1. Validate anchor data
2. Call F10.add_absolute_factor(frame_id, gps, is_user_anchor=True) 2. Call F10.add_absolute_factor(frame_id, gps, is_user_anchor=True)
3. F10.optimize() → refines trajectory 3. F10.optimize() → refines trajectory
4. Emit `UserFixApplied` event (F02 subscribes and updates status to "PROCESSING") 4. Return True (Caller F02.2 updates status)
5. Resume processing from next frame
**Test Cases**: **Test Cases**:
1. Valid anchor → applied, processing resumes 1. Valid anchor → applied, processing resumes
@@ -385,7 +376,7 @@ anchor: UserAnchor:
**Description**: Creates a new chunk proactively when tracking is lost. **Description**: Creates a new chunk proactively when tracking is lost.
**Called By**: **Called By**:
- F02 Flight Processor (when tracking lost detected) - F02.2 Flight Processing Engine (when tracking lost detected)
**Input**: **Input**:
```python ```python
@@ -535,9 +526,7 @@ bool: True if merge successful
- chunk_id (source) is merged into target_chunk_id - chunk_id (source) is merged into target_chunk_id
6. F12 handles chunk state updates (deactivation, status updates) 6. F12 handles chunk state updates (deactivation, status updates)
7. F10 optimizes merged graph globally (via F12.merge_chunks()) 7. F10 optimizes merged graph globally (via F12.merge_chunks())
8. **Emit ChunkMerged event** with flight_id and merged_frames 8. Return True (Caller F02.2 coordinates result updates)
- F14 subscribes and updates results: retrieves poses from F10, converts ENU to GPS, updates database, publishes via SSE
9. Return True
**Sim(3) Transform**: **Sim(3) Transform**:
- Translation: GPS offset - Translation: GPS offset
@@ -545,10 +534,9 @@ bool: True if merge successful
- Scale: Resolved from altitude and GSD - Scale: Resolved from altitude and GSD
**Test Cases**: **Test Cases**:
1. **Merge successful**: Chunks merged successfully, results updated via F14 1. **Merge successful**: Chunks merged successfully
2. **Global consistency**: Merged trajectory globally consistent 2. **Global consistency**: Merged trajectory globally consistent
3. **Multiple chunks**: Can merge multiple chunks sequentially 3. **Multiple chunks**: Can merge multiple chunks sequentially
4. **Result updates**: All merged frames have updated GPS coordinates published
--- ---
@@ -558,7 +546,7 @@ bool: True if merge successful
**Called By**: **Called By**:
- Background thread (periodic, e.g., every 5 seconds) - Background thread (periodic, e.g., every 5 seconds)
- F02 Flight Processor (after frame processing) - F02.2 Flight Processing Engine (after frame processing)
**Input**: **Input**:
```python ```python
@@ -583,7 +571,7 @@ while flight_active:
``` ```
**Background Processing**: **Background Processing**:
- **Trigger**: Started by F02 Flight Processor after flight creation - **Trigger**: Started by F02.2 Flight Processing Engine after flight creation
- Runs asynchronously, doesn't block frame processing - Runs asynchronously, doesn't block frame processing
- Periodically checks for ready chunks (every 5 seconds) - Periodically checks for ready chunks (every 5 seconds)
- Attempts matching and merging - Attempts matching and merging
@@ -592,15 +580,15 @@ while flight_active:
**Chunk Failure Flow**: **Chunk Failure Flow**:
When chunk matching fails after trying all candidates: When chunk matching fails after trying all candidates:
1. Emit `ChunkMatchingFailed` event with chunk_id and flight_id 1. Emit `ChunkMatchingFailed` event with chunk_id and flight_id (to F02.2 internal queue)
2. F02 receives event and decides next action: 2. F02.2 receives event and decides next action:
- **Option A**: Wait for more frames and retry later (chunk may gain more distinctive features) - **Option A**: Wait for more frames and retry later (chunk may gain more distinctive features)
- **Option B**: Create user input request for the chunk - **Option B**: Create user input request for the chunk
3. If user input requested: 3. If user input requested:
- F02 calls F14.publish_user_input_request() with chunk context - F02.2 calls F15.send_user_input_request() with chunk context
- Client receives via SSE - Client receives via SSE
- User provides GPS anchor for one frame in chunk - User provides GPS anchor for one frame in chunk
- F02 receives user fix via handle_user_fix() - F02.2 receives user fix via handle_user_fix()
- F11.apply_user_anchor() anchors the chunk - F11.apply_user_anchor() anchors the chunk
- Processing resumes - Processing resumes
@@ -608,7 +596,7 @@ When chunk matching fails after trying all candidates:
1. **Background matching**: Unanchored chunks matched asynchronously 1. **Background matching**: Unanchored chunks matched asynchronously
2. **Chunk merging**: Chunks merged when matches found 2. **Chunk merging**: Chunks merged when matches found
3. **Non-blocking**: Frame processing continues during matching 3. **Non-blocking**: Frame processing continues during matching
4. **Chunk matching fails**: ChunkMatchingFailed emitted, user input requested if needed 4. **Chunk matching fails**: Failure handled, user input requested if needed
## Integration Tests ## Integration Tests
@@ -676,16 +664,12 @@ When chunk matching fails after trying all candidates:
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- F04 Satellite Data Manager (tile grids and tile_bounds computation) - **F12 Route Chunk Manager**: Chunk operations.
- F06 Image Rotation Manager (rotation sweep and chunk rotation) - **F10 Factor Graph Optimizer**: Anchor application.
- F08 Global Place Recognition (candidates and chunk semantic matching) - **F04 Satellite Data Manager**: Search grids.
- F09 Metric Refinement (LiteSAM and chunk LiteSAM matching) - **F08 Global Place Recognition**: Candidate retrieval.
- F10 Factor Graph Optimizer (anchor application and chunk merging) - **F09 Metric Refinement**: Alignment.
- F12 Route Chunk Manager (chunk lifecycle) - **F06 Image Rotation Manager**: Rotation sweeps.
- F14 Result Manager (result updates after chunk merging)
- F15 SSE Event Streamer (user input events)
**Note**: F11 does NOT directly call F02. Instead, F11 emits events (RecoveryStarted, RecoveryFailed, etc.) which F02 subscribes to for status updates. This decouples recovery logic from flight state management.
### External Dependencies ### External Dependencies
- None - None
@@ -780,6 +764,3 @@ class RecoveryStatus(BaseModel):
chunk_id: Optional[str] chunk_id: Optional[str]
message: Optional[str] message: Optional[str]
``` ```
**Note**: F11 uses direct return values instead of events. F02 inspects return values to determine status updates and next actions.
docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md
+54 -251
View File
@@ -38,6 +38,12 @@ class IRouteChunkManager(ABC):
@abstractmethod @abstractmethod
def mark_chunk_anchored(self, chunk_id: str, frame_id: int, gps: GPSPoint) -> bool: def mark_chunk_anchored(self, chunk_id: str, frame_id: int, gps: GPSPoint) -> bool:
"""
Transactional update:
1. Calls F10.add_chunk_anchor().
2. IF success: Updates internal state to 'anchored'.
3. Returns success status.
"""
pass pass
@abstractmethod @abstractmethod
@@ -54,7 +60,12 @@ class IRouteChunkManager(ABC):
@abstractmethod @abstractmethod
def merge_chunks(self, target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool: def merge_chunks(self, target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool:
"""Merges source_chunk INTO target_chunk. Result is stored in target_chunk.""" """
Transactional update:
1. Calls F10.merge_chunk_subgraphs().
2. IF success: Updates internal state (source merged/deactivated).
3. Returns success status.
"""
pass pass
@abstractmethod @abstractmethod
@@ -63,40 +74,23 @@ class IRouteChunkManager(ABC):
@abstractmethod @abstractmethod
def save_chunk_state(self, flight_id: str) -> bool: def save_chunk_state(self, flight_id: str) -> bool:
"""Persist all chunk state to F03 for crash recovery."""
pass pass
@abstractmethod @abstractmethod
def load_chunk_state(self, flight_id: str) -> bool: def load_chunk_state(self, flight_id: str) -> bool:
"""Load chunk state from F03 on restart."""
pass pass
``` ```
## Component Description ## Component Description
### Responsibilities ### Responsibilities
- **Source of truth for chunk state** (active, anchored, merged status) - **Source of Truth**: Manages chunk states (Active, Matching, Anchored, Merged).
- Manage chunk lifecycle (creation, activation, deactivation, merging) - **Transactional Integrity**: Ensures internal state updates are atomic with respect to Factor Graph (F10) operations.
- Track chunk state (frames, anchors, matching status) - **Implementation**: Uses "Check-Act" pattern. Calls F10 first; if F10 fails, F12 does not update state and returns error.
- Provide chunk representations for matching (composite images, descriptors)
- Determine chunk readiness for matching (min frames, consistency)
- Persist chunk state via F03 Flight Database
### Chunk Responsibility Clarification ### Interaction
- Called by **F02.2 Flight Processing Engine** and **F11 Failure Recovery Coordinator** (via F02.2 or direct delegation).
**F12 is the high-level chunk manager**: - Calls **F10 Factor Graph Optimizer**.
- Owns chunk state (ChunkHandle with matching_status, is_active, has_anchor)
- Provides high-level queries: `get_active_chunk()`, `get_chunks_for_matching()`
- Coordinates with F10 for factor graph operations
- Persists chunk state for crash recovery
**F10 provides low-level factor graph operations** (see F10 spec):
- Subgraph creation and factor management
- Does NOT own chunk state - only factor graph data
**F11 coordinates recovery** (see F11 spec):
- Calls F12 for chunk operations
- F11 NEVER directly calls F10 chunk methods
### Scope ### Scope
- Chunk lifecycle management - Chunk lifecycle management
@@ -108,10 +102,10 @@ class IRouteChunkManager(ABC):
### `create_chunk(flight_id: str, start_frame_id: int) -> ChunkHandle` ### `create_chunk(flight_id: str, start_frame_id: int) -> ChunkHandle`
**Description**: Creates a new route chunk and initializes it in the factor graph. **Description**: Initializes a new route chunk.
**Called By**: **Called By**:
- F02 Flight Processor (when tracking lost) - F02.2 Flight Processing Engine (when tracking lost)
- F11 Failure Recovery Coordinator (proactive chunk creation) - F11 Failure Recovery Coordinator (proactive chunk creation)
**Input**: **Input**:
@@ -151,10 +145,10 @@ ChunkHandle:
### `add_frame_to_chunk(chunk_id: str, frame_id: int, vo_result: RelativePose) -> bool` ### `add_frame_to_chunk(chunk_id: str, frame_id: int, vo_result: RelativePose) -> bool`
**Description**: Adds a frame to an existing chunk with its VO result. **Description**: Adds a frame to an active chunk.
**Called By**: **Called By**:
- F02 Flight Processor (during frame processing) - F02.2 Flight Processing Engine (during frame processing)
**Input**: **Input**:
```python ```python
@@ -192,20 +186,9 @@ bool: True if frame added successfully
- F09 Metric Refinement (for chunk LiteSAM matching) - F09 Metric Refinement (for chunk LiteSAM matching)
- F11 Failure Recovery Coordinator (chunk state queries) - F11 Failure Recovery Coordinator (chunk state queries)
**Input**: **Input**: `chunk_id: str`
```python
chunk_id: str
```
**Output**: **Output**: `List[int]` # Frame IDs in chunk, ordered by sequence
```python
List[int] # Frame IDs in chunk, ordered by sequence
```
**Test Cases**:
1. **Get frames**: Returns all frames in chunk
2. **Empty chunk**: Returns empty list
3. **Ordered frames**: Frames returned in sequence order
--- ---
@@ -218,25 +201,7 @@ List[int] # Frame IDs in chunk, ordered by sequence
- F09 Metric Refinement (chunk LiteSAM matching) - F09 Metric Refinement (chunk LiteSAM matching)
- F06 Image Rotation Manager (chunk rotation) - F06 Image Rotation Manager (chunk rotation)
**Input**: **Output**: `List[np.ndarray]` # Images for each frame in chunk
```python
chunk_id: str
```
**Output**:
```python
List[np.ndarray] # Images for each frame in chunk
```
**Processing Flow**:
1. Get chunk frames via get_chunk_frames()
2. Load images from F05 Image Input Pipeline
3. Return list of images
**Test Cases**:
1. **Get images**: Returns all images in chunk
2. **Image loading**: Images loaded correctly from pipeline
3. **Order consistency**: Images match frame order
--- ---
@@ -247,30 +212,7 @@ List[np.ndarray] # Images for each frame in chunk
**Called By**: **Called By**:
- F08 Global Place Recognition (chunk semantic matching) - F08 Global Place Recognition (chunk semantic matching)
**Input**: **Output**: `np.ndarray`: Aggregated descriptor vector (4096-dim or 8192-dim)
```python
chunk_id: str
```
**Output**:
```python
np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
```
**Processing Flow**:
1. Get chunk images via get_chunk_images()
2. Call F08.compute_chunk_descriptor(chunk_images) → aggregate descriptor
3. Return composite descriptor
**Delegation**:
- Delegates to F08.compute_chunk_descriptor() for descriptor computation
- F08 handles aggregation logic (mean, VLAD, or max)
- Single source of truth for chunk descriptor computation
**Test Cases**:
1. **Compute descriptor**: Returns aggregated descriptor
2. **Multiple images**: Descriptor aggregates correctly
3. **Descriptor quality**: More robust than single-image descriptor
--- ---
@@ -282,11 +224,6 @@ np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
- F11 Failure Recovery Coordinator (for tile search area) - F11 Failure Recovery Coordinator (for tile search area)
- F04 Satellite Data Manager (for tile prefetching) - F04 Satellite Data Manager (for tile prefetching)
**Input**:
```python
chunk_id: str
```
**Output**: **Output**:
```python ```python
ChunkBounds: ChunkBounds:
@@ -295,58 +232,23 @@ ChunkBounds:
confidence: float confidence: float
``` ```
**Processing Flow**:
1. Get chunk trajectory from F10.get_chunk_trajectory()
2. If chunk has anchor:
- Use anchor GPS as center
- Compute radius from trajectory extent
3. If chunk unanchored:
- Estimate center from VO trajectory (relative to start)
- Use dead-reckoning estimate
- Lower confidence
4. Return ChunkBounds
**Test Cases**:
1. **Anchored chunk**: Returns accurate bounds with high confidence
2. **Unanchored chunk**: Returns estimated bounds with lower confidence
3. **Bounds calculation**: Radius computed from trajectory extent
--- ---
### `is_chunk_ready_for_matching(chunk_id: str) -> bool` ### `is_chunk_ready_for_matching(chunk_id: str) -> bool`
**Description**: Determines if chunk has enough frames and consistency for matching. **Description**: Checks if a chunk has enough data (frames, spread) to attempt satellite matching.
**Called By**:
- F11 Failure Recovery Coordinator (before attempting matching)
- Background matching task
**Input**:
```python
chunk_id: str
```
**Output**:
```python
bool: True if chunk ready for matching
```
**Criteria**: **Criteria**:
- **Min frames**: >= 5 frames (configurable) - Min frames: >= 5 frames (configurable)
- **Max frames**: <= 20 frames (configurable, prevents oversized chunks) - Max frames: <= 20 frames (configurable, prevents oversized chunks)
- **Internal consistency**: VO factors have reasonable inlier counts - Internal consistency: VO factors have reasonable inlier counts
- **Not already matched**: matching_status != "anchored" or "merged" - Not already matched: matching_status != "anchored" or "merged"
**Test Cases**:
1. **Ready chunk**: 10 frames, good consistency → True
2. **Too few frames**: 3 frames → False
3. **Already anchored**: has_anchor=True → False
--- ---
### `mark_chunk_anchored(chunk_id: str, frame_id: int, gps: GPSPoint) -> bool` ### `mark_chunk_anchored(chunk_id: str, frame_id: int, gps: GPSPoint) -> bool`
**Description**: Marks chunk as anchored with GPS coordinate. **Description**: Anchors a chunk to a specific GPS coordinate (e.g., from successful satellite matching).
**Called By**: **Called By**:
- F11 Failure Recovery Coordinator (after successful chunk matching) - F11 Failure Recovery Coordinator (after successful chunk matching)
@@ -358,17 +260,15 @@ frame_id: int # Frame within chunk that was anchored
gps: GPSPoint gps: GPSPoint
``` ```
**Output**: **Output**: `bool` - True if marked successfully
```python
bool: True if marked successfully
```
**Processing Flow**: **Processing Flow**:
1. Verify chunk exists 1. Verify chunk exists
2. Call F10.add_chunk_anchor() 2. Call F10.add_chunk_anchor()
3. Update chunk state (has_anchor=True, anchor_frame_id, anchor_gps) 3. If successful:
4. Update matching_status to "anchored" - Update chunk state (has_anchor=True, anchor_frame_id, anchor_gps)
5. Trigger chunk optimization - Update matching_status to "anchored"
- Trigger chunk optimization
**Test Cases**: **Test Cases**:
1. **Mark anchored**: Chunk state updated correctly 1. **Mark anchored**: Chunk state updated correctly
@@ -383,30 +283,8 @@ bool: True if marked successfully
**Called By**: **Called By**:
- F11 Failure Recovery Coordinator (background matching task) - F11 Failure Recovery Coordinator (background matching task)
- Background processing task
**Input**: **Output**: `List[ChunkHandle]` # Unanchored chunks ready for matching
```python
flight_id: str
```
**Output**:
```python
List[ChunkHandle] # Unanchored chunks ready for matching
```
**Processing Flow**:
1. Get all chunks for flight_id
2. Filter chunks where:
- has_anchor == False
- is_chunk_ready_for_matching() == True
- matching_status == "unanchored" or "matching"
3. Return filtered list
**Test Cases**:
1. **Get unanchored chunks**: Returns ready chunks
2. **Filter criteria**: Only returns chunks meeting criteria
3. **Empty result**: Returns empty list if no ready chunks
--- ---
@@ -415,22 +293,9 @@ List[ChunkHandle] # Unanchored chunks ready for matching
**Description**: Gets the currently active chunk for a flight. **Description**: Gets the currently active chunk for a flight.
**Called By**: **Called By**:
- F02 Flight Processor (before processing frame) - F02.2 Flight Processing Engine (before processing frame)
**Input**: **Output**: `Optional[ChunkHandle]` # Active chunk or None
```python
flight_id: str
```
**Output**:
```python
Optional[ChunkHandle] # Active chunk or None
```
**Test Cases**:
1. **Get active chunk**: Returns active chunk
2. **No active chunk**: Returns None
3. **Multiple chunks**: Returns only active chunk
--- ---
@@ -440,27 +305,9 @@ Optional[ChunkHandle] # Active chunk or None
**Called By**: **Called By**:
- F11 Failure Recovery Coordinator (after chunk merged) - F11 Failure Recovery Coordinator (after chunk merged)
- F02 Flight Processor (chunk lifecycle) - F02.2 Flight Processing Engine (chunk lifecycle)
**Input**: **Output**: `bool` - True if deactivated successfully
```python
chunk_id: str
```
**Output**:
```python
bool: True if deactivated successfully
```
**Processing Flow**:
1. Verify chunk exists
2. Update chunk state (is_active=False)
3. Update matching_status to "merged" if merged
4. Return True
**Test Cases**:
1. **Deactivate chunk**: Chunk marked inactive
2. **After merge**: Matching status updated to "merged"
--- ---
@@ -481,35 +328,18 @@ transform: Sim3Transform:
scale: float scale: float
``` ```
**Output**: **Output**: `bool` - True if merge successful
```python
bool: True if merge successful
```
**Processing Flow**: **Processing Flow**:
1. Verify both chunks exist 1. Call F10.merge_chunk_subgraphs(flight_id, source_chunk_id, target_chunk_id, transform)
2. Verify source_chunk_id is anchored (has_anchor=True) 2. If successful:
3. Validate chunks can be merged (not already merged, not same chunk) - Update source_chunk_id state:
4. Call F10.merge_chunk_subgraphs(flight_id, source_chunk_id, target_chunk_id, transform)
5. Update source_chunk_id state:
- Set is_active=False - Set is_active=False
- Set matching_status="merged" - Set matching_status="merged"
- Call deactivate_chunk(source_chunk_id) - Call deactivate_chunk(source_chunk_id)
6. target_chunk remains active (now contains merged frames) - target_chunk remains active (now contains merged frames)
7. Persist chunk state via F03 Flight Database.save_chunk_state() - Persist chunk state via F03 Flight Database.save_chunk_state()
8. Return True 3. Return True
**Merge Convention**:
- `merge_chunks(target, source)` → source is merged INTO target
- Result is stored in target_chunk
- source_chunk is deactivated after merge
- Example: `merge_chunks("main", "chunk_3")` merges chunk_3 into main trajectory
**Validation**:
- Both chunks must exist
- source_chunk must be anchored
- source_chunk must not already be merged
- target_chunk and source_chunk must be different
**Test Cases**: **Test Cases**:
1. **Merge anchored chunk**: source_chunk merged into target_chunk 1. **Merge anchored chunk**: source_chunk merged into target_chunk
@@ -525,29 +355,7 @@ bool: True if merge successful
**Called By**: **Called By**:
- F11 Failure Recovery Coordinator (when chunk matching starts) - F11 Failure Recovery Coordinator (when chunk matching starts)
**Input**: **Output**: `bool` - True if marked successfully
```python
chunk_id: str
```
**Output**:
```python
bool: True if marked successfully
```
**Processing Flow**:
1. Verify chunk exists
2. Verify chunk is unanchored (has_anchor=False)
3. Update matching_status to "matching"
4. Return True
**State Transition**:
- `unanchored` → `matching` (explicit transition)
**Test Cases**:
1. **Mark unanchored chunk**: Status updated to "matching"
2. **Mark already anchored chunk**: Returns False (invalid state)
3. **Mark non-existent chunk**: Returns False
## Integration Tests ## Integration Tests
@@ -600,14 +408,10 @@ bool: True if marked successfully
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- **F10 Factor Graph Optimizer**: Chunk creation and factor management - **F10 Factor Graph Optimizer**: Critical dependency for subgraph operations (`create_chunk_subgraph`, `add_relative_factor_to_chunk`, `merge_chunk_subgraphs`).
- **F05 Image Input Pipeline**: Image retrieval - **F03 Flight Database**: Persistence via `save_chunk_state()`, `load_chunk_states()`.
- **F08 Global Place Recognition**: Descriptor computation - **F05 Image Input Pipeline**: Image retrieval via `get_image_by_sequence()` for `get_chunk_images()`.
- **F07 Sequential VO**: VO results for chunk building - **F08 Global Place Recognition**: Descriptor computation via `compute_chunk_descriptor()` for `get_chunk_composite_descriptor()`.
- **F03 Flight Database**: Chunk state persistence
### External Dependencies
- **numpy**: Array operations
## Data Models ## Data Models
@@ -649,4 +453,3 @@ class Sim3Transform(BaseModel):
rotation: np.ndarray # (3, 3) rotation matrix or (4,) quaternion rotation: np.ndarray # (3, 3) rotation matrix or (4,) quaternion
scale: float # Scale factor scale: float # Scale factor
``` ```
docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md
+7 -7
View File
@@ -72,7 +72,7 @@ class ICoordinateTransformer(ABC):
**Why ENU Origin?**: Factor graph optimization works in Cartesian coordinates (meters) for better numerical stability. ENU converts GPS (degrees) to local meters relative to an origin. See `helpers/enu_origin_explanation.md` for details. **Why ENU Origin?**: Factor graph optimization works in Cartesian coordinates (meters) for better numerical stability. ENU converts GPS (degrees) to local meters relative to an origin. See `helpers/enu_origin_explanation.md` for details.
**Called By**: **Called By**:
- F02 Flight Processor (during create_flight) - F02.1 Flight Lifecycle Manager (during create_flight)
**Input**: **Input**:
```python ```python
@@ -200,7 +200,7 @@ GPSPoint: WGS84 coordinates
**Description**: Converts pixel coordinates to GPS using camera pose and ground plane assumption. **Description**: Converts pixel coordinates to GPS using camera pose and ground plane assumption.
**Called By**: **Called By**:
- F02 Flight Processor (for frame center GPS) - F02.2 Flight Processing Engine (for frame center GPS)
- Internal (for image_object_to_gps) - Internal (for image_object_to_gps)
**Input**: **Input**:
@@ -392,12 +392,12 @@ List[Tuple[float, float]]: Transformed points
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- **F10 Factor Graph Optimizer**: For frame poses via get_trajectory(flight_id) - **F10 Factor Graph Optimizer**: For frame poses via `get_trajectory(flight_id)` - required for `pixel_to_gps()` and `image_object_to_gps()` to get frame pose estimates.
- **F17 Configuration Manager**: For camera parameters via get_flight_config(flight_id) - **F17 Configuration Manager**: For camera parameters via `get_flight_config(flight_id)`.
- **H01 Camera Model**: For projection operations - **H01 Camera Model**: For projection/unprojection operations (`project()`, `unproject()`).
- **H02 GSD Calculator**: For GSD calculations - **H02 GSD Calculator**: For GSD calculations in coordinate conversions.
**Note**: F13 uses internal enu_to_gps() and gps_to_enu() methods that rely on the ENU origin set via set_enu_origin(flight_id, origin_gps). H06 Web Mercator Utils is NOT used for ENU conversions - ENU is a local Cartesian coordinate system. **Note**: F13 uses internal `enu_to_gps()` and `gps_to_enu()` methods that rely on the ENU origin set via `set_enu_origin(flight_id, origin_gps)`. H06 Web Mercator Utils is NOT used for ENU conversions - ENU is a local Cartesian coordinate system.
### External Dependencies ### External Dependencies
- **numpy**: Matrix operations - **numpy**: Matrix operations
docs/02_components/14_result_manager/result_manager_spec.md
+33 -42
View File
@@ -10,6 +10,12 @@
class IResultManager(ABC): class IResultManager(ABC):
@abstractmethod @abstractmethod
def update_frame_result(self, flight_id: str, frame_id: int, result: FrameResult) -> bool: def update_frame_result(self, flight_id: str, frame_id: int, result: FrameResult) -> bool:
"""
Atomic update:
1. Saves result to frame_results table.
2. Updates waypoint in waypoints table.
3. All within a single transaction via F03.
"""
pass pass
@abstractmethod @abstractmethod
@@ -36,13 +42,8 @@ class IResultManager(ABC):
## Component Description ## Component Description
### Responsibilities ### Responsibilities
- Manage trajectory results per flight - Result consistency and publishing.
- Track frame refinements and changes - **Atomic Updates**: Ensures consistency between normalized `waypoints` and denormalized `frame_results` via transaction requests to F03.
- Store waypoint updates via F03 Flight Database
- Send incremental updates via F15 SSE Event Streamer
- Maintain result versioning for audit trail
**Note**: F14 receives GPS coordinates directly from the caller (F02). F14 does NOT call F10 Factor Graph Optimizer or F13 Coordinate Transformer for pose retrieval/conversion. This ensures unidirectional data flow: F02 → F14 → F03/F15.
### Scope ### Scope
- Result state management - Result state management
@@ -55,7 +56,7 @@ class IResultManager(ABC):
### `update_frame_result(flight_id: str, frame_id: int, result: FrameResult) -> bool` ### `update_frame_result(flight_id: str, frame_id: int, result: FrameResult) -> bool`
**Description**: Updates result for a processed frame. **Description**: Persists and publishes the result of a processed frame.
**Called By**: **Called By**:
- Main processing loop (after each frame) - Main processing loop (after each frame)
@@ -78,10 +79,12 @@ result: FrameResult:
**Output**: `bool` - True if updated **Output**: `bool` - True if updated
**Processing Flow**: **Processing Flow**:
1. Store result via F03 Flight Database.save_frame_result() 1. Construct DB transaction:
2. Call publish_waypoint_update() - Insert/Update `frame_results`.
3. Call F15 SSE Event Streamer.send_frame_result() - Update `waypoints` (latest position).
4. Update flight statistics 2. Call `F03.execute_transaction()`.
3. If success: call `F15.send_frame_result()`.
4. Update flight statistics.
**Test Cases**: **Test Cases**:
1. New frame result → stored and published 1. New frame result → stored and published
@@ -91,7 +94,7 @@ result: FrameResult:
### `publish_waypoint_update(flight_id: str, frame_id: int) -> bool` ### `publish_waypoint_update(flight_id: str, frame_id: int) -> bool`
**Description**: Updates waypoint in Flight Database via F03. **Description**: Specifically triggers an update for the waypoint visualization.
**Called By**: **Called By**:
- Internal (after update_frame_result) - Internal (after update_frame_result)
@@ -105,10 +108,9 @@ frame_id: int
**Output**: `bool` - True if updated successfully **Output**: `bool` - True if updated successfully
**Processing Flow**: **Processing Flow**:
1. Get result for frame_id 1. Fetch latest data.
2. Convert to Waypoint format 2. Call `F15.send_frame_result()` (or a specific lightweight event).
3. Call F03 Flight Database.update_waypoint() 3. Handle errors (retry if transient)
4. Handle errors (retry if transient)
**Test Cases**: **Test Cases**:
1. Successful update → Waypoint stored in database 1. Successful update → Waypoint stored in database
@@ -141,7 +143,7 @@ FlightResults:
### `mark_refined(flight_id: str, frame_ids: List[int]) -> bool` ### `mark_refined(flight_id: str, frame_ids: List[int]) -> bool`
**Description**: Marks frames as refined after batch optimization. **Description**: Updates results for frames that have been retrospectively improved (e.g., after loop closure or chunk merge).
**Called By**: **Called By**:
- F10 Factor Graph (after asynchronous refinement) - F10 Factor Graph (after asynchronous refinement)
@@ -154,16 +156,16 @@ frame_ids: List[int] # Frames with updated poses
**Output**: `bool` **Output**: `bool`
**Note**: F14 does NOT call F10 or F13. The caller (F02) performs the following steps before calling mark_refined(): **Note**: F14 does NOT call F10 or F13. The caller (F02.2) performs the following steps before calling mark_refined():
1. F02 gets refined poses from F10.get_trajectory(flight_id) 1. F02.2 gets refined poses from F10.get_trajectory(flight_id)
2. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple) 2. F02.2 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple)
3. F02 calls F14.mark_refined() with the GPS-converted results 3. F02.2 calls F14.mark_refined() with the GPS-converted results
**Processing Flow**: **Processing Flow**:
1. For each frame_id in frame_ids: 1. For each frame_id in frame_ids:
- Receive GPS coordinates from caller (already converted) - Receive GPS coordinates from caller (already converted)
- Update result with refined=True via F03 Flight Database.save_frame_result() - Update result with refined=True via F03 Flight Database (part of transaction)
- Update waypoint via F03 Flight Database.update_waypoint() - Update waypoint via F03 Flight Database (part of transaction)
- Call F15 SSE Event Streamer.send_refinement() - Call F15 SSE Event Streamer.send_refinement()
**Test Cases**: **Test Cases**:
@@ -194,11 +196,11 @@ since: datetime
### `update_results_after_chunk_merge(flight_id: str, merged_frames: List[int]) -> bool` ### `update_results_after_chunk_merge(flight_id: str, merged_frames: List[int]) -> bool`
**Description**: Updates frame results after chunk merging changes frame poses. **Description**: Handling specific to chunk merging events.
**Triggered By**: **Triggered By**:
- `ChunkMerged` event from F11 (F14 subscribes to this event) - `ChunkMerged` event from F11 (F14 subscribes to this event)
- Alternative: F02 Flight Processor can coordinate this call after receiving ChunkMerged event - Alternative: F02.2 Flight Processing Engine can coordinate this call after receiving ChunkMerged event
**Input**: **Input**:
```python ```python
@@ -208,19 +210,9 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
**Output**: `bool` - True if updated successfully **Output**: `bool` - True if updated successfully
**Note**: F14 does NOT call F10 or F13. The caller (F02) performs these steps before calling update_results_after_chunk_merge():
1. F02 receives ChunkMerged event from F11 with merged_frames list
2. F02 gets updated poses from F10.get_trajectory(flight_id)
3. F02 converts ENU to GPS via F13.enu_to_gps(flight_id, enu_tuple) for each frame
4. F02 calls F14.update_results_after_chunk_merge() with GPS-converted results
**Processing Flow**: **Processing Flow**:
1. For each frame_id in merged_frames: 1. Similar to `mark_refined` but triggered by `ChunkMerged` event context.
- Receive GPS coordinates from caller (already converted) 2. Ensures all frames in the merged chunk are updated to the global coordinate system.
- Update frame result via F03 Flight Database.save_frame_result()
- Update waypoint via F03 Flight Database.update_waypoint()
- Send refinement event via F15 SSE Event Streamer.send_refinement()
2. Return True
**Test Cases**: **Test Cases**:
1. **Chunk merge updates**: All merged frames updated and published 1. **Chunk merge updates**: All merged frames updated and published
@@ -263,10 +255,10 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
## Dependencies ## Dependencies
### Internal Components ### Internal Components
- **F03 Flight Database**: For waypoint and frame result persistence - **F03 Flight Database**: Must support transactional updates.
- **F15 SSE Event Streamer**: For real-time result streaming - **F15 SSE Event Streamer**: For real-time result streaming.
**Note**: F14 does NOT depend on F10, F13, or F11. The caller (F02) coordinates with those components and provides GPS-converted results to F14. This ensures unidirectional data flow and eliminates circular dependencies. **Note**: F14 does NOT depend on F10, F13, or F11. The caller (F02.2) coordinates with those components and provides GPS-converted results to F14. This ensures unidirectional data flow and eliminates circular dependencies.
### External Dependencies ### External Dependencies
- None - None
@@ -308,4 +300,3 @@ class FlightResults(BaseModel):
frames: List[FrameResult] frames: List[FrameResult]
statistics: FlightStatistics statistics: FlightStatistics
``` ```
docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md
+16 -15
View File
@@ -30,7 +30,6 @@ class ISSEEventStreamer(ABC):
@abstractmethod @abstractmethod
def send_heartbeat(self, flight_id: str) -> bool: def send_heartbeat(self, flight_id: str) -> bool:
"""Sends heartbeat/keepalive to all clients subscribed to flight."""
pass pass
@abstractmethod @abstractmethod
@@ -39,19 +38,22 @@ class ISSEEventStreamer(ABC):
@abstractmethod @abstractmethod
def get_active_connections(self, flight_id: str) -> int: def get_active_connections(self, flight_id: str) -> int:
"""Returns count of active SSE connections for a flight."""
pass pass
``` ```
## Component Description ## Component Description
### Responsibilities ### Responsibilities
- Server-Sent Events broadcaster for real-time results - Real-time communication with clients.
- Stream per-frame processing results to clients - Buffering events for disconnected clients.
- Send refinement updates asynchronously
- Request user input when processing blocked ### Callers
- Handle client connections and reconnections - **F02.1 Flight Lifecycle Manager**: Calls `create_stream` (delegated from F01).
- Event replay from last received event - **F02.2 Flight Processing Engine**: Calls `send_user_input_request`, `send_search_progress`.
- **F14 Result Manager**: Calls `send_frame_result`, `send_refinement`.
### Consistency Fix
- Previously listed F11 as caller. **Correction**: F11 returns request objects to F02.2. **F02.2 is the sole caller** for user input requests and search status updates. F11 has no dependencies on F15.
### Scope ### Scope
- SSE protocol implementation - SSE protocol implementation
@@ -64,7 +66,7 @@ class ISSEEventStreamer(ABC):
### `create_stream(flight_id: str, client_id: str) -> StreamConnection` ### `create_stream(flight_id: str, client_id: str) -> StreamConnection`
**Description**: Creates SSE connection for a client. **Description**: Establishes a server-sent events connection.
**Called By**: F01 REST API (GET /stream endpoint) **Called By**: F01 REST API (GET /stream endpoint)
@@ -124,7 +126,7 @@ StreamConnection:
**Description**: Sends search_expanded event. **Description**: Sends search_expanded event.
**Called By**: F11 Failure Recovery Coordinator **Called By**: F02.2 Flight Processing Engine (via F11 status return)
**Event Format**: **Event Format**:
```json ```json
@@ -144,7 +146,7 @@ StreamConnection:
**Description**: Sends user_input_needed event. **Description**: Sends user_input_needed event.
**Called By**: F11 Failure Recovery Coordinator **Called By**: F02.2 Flight Processing Engine (via F11 request object)
**Event Format**: **Event Format**:
```json ```json
@@ -182,11 +184,11 @@ StreamConnection:
### `send_heartbeat(flight_id: str) -> bool` ### `send_heartbeat(flight_id: str) -> bool`
**Description**: Sends heartbeat/keepalive ping to all clients subscribed to a flight. Keeps connections alive and helps detect stale connections. **Description**: Sends heartbeat/keepalive to all clients subscribed to flight.
**Called By**: **Called By**:
- Background heartbeat task (every 30 seconds) - Background heartbeat task (every 30 seconds)
- F02 Flight Processor (periodically during processing) - F02.2 Flight Processing Engine (periodically during processing)
**Event Format**: **Event Format**:
``` ```
@@ -217,7 +219,7 @@ StreamConnection:
**Description**: Returns count of active SSE connections for a flight. **Description**: Returns count of active SSE connections for a flight.
**Called By**: **Called By**:
- F02 Flight Processor (monitoring) - F02.2 Flight Processing Engine (monitoring)
- Admin tools - Admin tools
**Test Cases**: **Test Cases**:
@@ -287,4 +289,3 @@ class SSEEvent(BaseModel):
id: Optional[str] id: Optional[str]
data: Dict[str, Any] data: Dict[str, Any]
``` ```
docs/02_components/16_model_manager/model_manager_spec.md
+1 -1
View File
@@ -55,7 +55,7 @@ class IModelManager(ABC):
**Description**: Loads model in specified format. **Description**: Loads model in specified format.
**Called By**: F02 Flight Manager (during initialization) **Called By**: F02.1 Flight Lifecycle Manager (during system initialization)
**Input**: **Input**:
```python ```python
docs/02_components/17_configuration_manager/configuration_manager_spec.md
+2 -2
View File
@@ -126,7 +126,7 @@ CameraParameters:
**Description**: Gets flight-specific configuration. **Description**: Gets flight-specific configuration.
**Called By**: F02 Flight Manager **Called By**: F02.1 Flight Lifecycle Manager
**Output**: **Output**:
```python ```python
@@ -193,7 +193,7 @@ FlightConfig:
**Description**: Persists flight-specific configuration when a flight is created. **Description**: Persists flight-specific configuration when a flight is created.
**Called By**: **Called By**:
- F02 Flight Processor (during flight creation) - F02.1 Flight Lifecycle Manager (during flight creation)
**Input**: **Input**:
```python ```python
docs/02_components/decomposition_plan.md
+85 -72
View File
@@ -45,9 +45,13 @@
**Interface**: `IFlightAPI` **Interface**: `IFlightAPI`
**Endpoints**: `POST /flights`, `GET /flights/{flightId}`, `DELETE /flights/{flightId}`, `PUT /flights/{flightId}/waypoints/{waypointId}`, `POST .../images/batch`, `POST .../user-fix`, `GET .../status`, `GET .../stream` **Endpoints**: `POST /flights`, `GET /flights/{flightId}`, `DELETE /flights/{flightId}`, `PUT /flights/{flightId}/waypoints/{waypointId}`, `POST .../images/batch`, `POST .../user-fix`, `GET .../status`, `GET .../stream`
**F02_flight_processor** **F02.1_flight_lifecycle_manager**
**Interface**: `IFlightProcessor` **Interface**: `IFlightLifecycleManager`
**API**: `create_flight()`, `get_flight()`, `get_flight_state()`, `delete_flight()`, `update_waypoint()`, `batch_update_waypoints()`, `validate_waypoint()`, `validate_geofence()`, `process_frame()`, `run_processing_loop()`, `handle_tracking_loss()`, `initialize_system()` **API**: `create_flight()`, `get_flight()`, `get_flight_state()`, `delete_flight()`, `update_waypoint()`, `batch_update_waypoints()`, `validate_waypoint()`, `validate_geofence()`, `queue_images()`, `handle_user_fix()`, `create_client_stream()`, `convert_object_to_gps()`, `initialize_system()`
**F02.2_flight_processing_engine**
**Interface**: `IFlightProcessingEngine`
**API**: `start_processing()`, `stop_processing()`, `process_frame()`, `apply_user_fix()`, `handle_tracking_loss()`, `get_active_chunk()`, `create_new_chunk()`
**F03_flight_database** **F03_flight_database**
**Interface**: `IFlightDatabase` **Interface**: `IFlightDatabase`
@@ -176,40 +180,40 @@
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F02 | F16 | `load_config()` | Load system configuration | | F02.1 | F17 | `load_config()` | Load system configuration |
| F02 | F16 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM | | F02.1 | F16 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| F04 | F08 | Satellite tiles | F08 generates descriptors for Faiss | | F04 | F08 | Satellite tiles + index | F08 loads pre-built Faiss index from provider |
| F08 | H04 | `build_index()` | Build satellite descriptor index | | F08 | H04 | `load_index()` | Load satellite descriptor index |
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model for descriptor generation | | F08 | F16 | `get_inference_engine("DINOv2")` | Get model for descriptor computation |
### Flight Creation ### Flight Creation
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| Client | F01 | `POST /flights` | Create flight | | Client | F01 | `POST /flights` | Create flight |
| F01 | F02 | `create_flight()` | Initialize flight state | | F01 | F02.1 | `create_flight()` | Initialize flight state |
| F02 | F16 | `get_flight_config()` | Get camera params, altitude | | F02.1 | F17 | `get_flight_config()` | Get camera params, altitude |
| F02 | F13 | `set_enu_origin(flight_id, start_gps)` | Set ENU coordinate origin | | F02.1 | F13 | `set_enu_origin(flight_id, start_gps)` | Set ENU coordinate origin |
| F02 | F04 | `prefetch_route_corridor()` | Prefetch tiles | | F02.1 | F04 | `prefetch_route_corridor()` | Prefetch tiles |
| F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download | | F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download (includes tile metadata) |
| F04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations | | F04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations |
| F02 | F03 | `insert_flight()` | Persist flight data | | F02.1 | F03 | `insert_flight()` | Persist flight data |
### SSE Stream Creation ### SSE Stream Creation
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| Client | F01 | `GET .../stream` | Open SSE connection | | Client | F01 | `GET .../stream` | Open SSE connection |
| F01 | F02 | `create_client_stream()` | Route through coordinator | | F01 | F02.1 | `create_client_stream()` | Route through lifecycle manager |
| F02 | F15 | `create_stream()` | Establish SSE channel | | F02.1 | F15 | `create_stream()` | Establish SSE channel |
### Image Upload ### Image Upload
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| Client | F01 | `POST .../images/batch` | Upload 10-50 images | | Client | F01 | `POST .../images/batch` | Upload 10-50 images |
| F01 | F02 | `queue_images()` | Route through coordinator | | F01 | F02.1 | `queue_images()` | Route through lifecycle manager |
| F02 | F05 | `queue_batch()` | Queue for processing | | F02.1 | F05 | `queue_batch()` | Queue for processing |
| F05 | H08 | `validate_batch()` | Validate sequence, format | | F05 | H08 | `validate_batch()` | Validate sequence, format |
| F05 | F03 | `save_image_metadata()` | Persist image metadata | | F05 | F03 | `save_image_metadata()` | Persist image metadata |
@@ -217,48 +221,48 @@
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F02 | F05 | `get_next_image()` | Get image for processing | | F02.2 | F05 | `get_next_image()` | Get image for processing |
| F02 | F06 | `requires_rotation_sweep()` | Check if sweep needed | | F02.2 | F06 | `requires_rotation_sweep()` | Check if sweep needed |
| F06 | H07 | `rotate_image()` × 12 | Rotate in 30° steps | | F06 | H07 | `rotate_image()` × 12 | Rotate in 30° steps |
| F06 | F09 | `align_to_satellite(img, tile, bounds)` × 12 | Try LiteSAM each rotation | | F06 | F09 | `align_to_satellite(img, tile, bounds)` × 12 | Try LiteSAM each rotation |
| F06 | F04 | `get_cached_tile()` + `compute_tile_bounds()` | Get expected tile with bounds | | F02.2 | F04 | `get_cached_tile()` + `compute_tile_bounds()` | Get expected tile with bounds |
| F09 | F15 | `get_inference_engine("LiteSAM")` | Get model | | F09 | F16 | `get_inference_engine("LiteSAM")` | Get model |
| F06 | H07 | `calculate_rotation_from_points()` | Precise angle from homography | | F06 | H07 | `calculate_rotation_from_points()` | Precise angle from homography |
| F06 | F03 | `save_heading()` | Store UAV heading | | F02.2 | F03 | `save_heading()` | Store UAV heading |
### Per-Frame Processing (Sequential VO) ### Per-Frame Processing (Sequential VO)
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F02 | F12 | `get_active_chunk()` | Get active chunk for frame | | F02.2 | F12 | `get_active_chunk()` | Get active chunk for frame |
| F02 | F07 | Process frame | Provide image and chunk context | | F02.2 | F07 | `compute_relative_pose()` | Provide image and chunk context |
| F07 | F16 | `get_inference_engine("SuperPoint")` | Get feature extractor | | F07 | F16 | `get_inference_engine("SuperPoint")` | Get feature extractor |
| F07 | F16 | `get_inference_engine("LightGlue")` | Get matcher | | F07 | F16 | `get_inference_engine("LightGlue")` | Get matcher |
| F07 | H05 | `start_timer()`, `end_timer()` | Monitor timing | | F07 | H05 | `start_timer()`, `end_timer()` | Monitor timing |
| F07 | F10 | `add_relative_factor_to_chunk()` | Add pose measurement to chunk subgraph | | F02.2 | F10 | `add_relative_factor_to_chunk()` | Add pose measurement to chunk subgraph |
| F02 | F12 | `add_frame_to_chunk()` | Add frame to chunk | | F02.2 | F12 | `add_frame_to_chunk()` | Add frame to chunk |
### Tracking Good (Drift Correction) ### Tracking Good (Drift Correction)
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F02 | F11 | `check_confidence()` | Check tracking quality | | F02.2 | F11 | `check_confidence()` | Check tracking quality |
| F02 | F04 | `fetch_tile()` + `compute_tile_bounds()` | Get single tile with bounds | | F02.2 | F04 | `fetch_tile()` + `compute_tile_bounds()` | Get single tile with bounds |
| F02 | F09 | `align_to_satellite(img, tile, bounds)` | Align to 1 tile | | F02.2 | F09 | `align_to_satellite(img, tile, bounds)` | Align to 1 tile |
| F02 | F10 | `add_absolute_factor()` | Add GPS measurement | | F02.2 | F10 | `add_absolute_factor()` | Add GPS measurement |
### Tracking Lost (Progressive Search + Chunk Building) ### Tracking Lost (Progressive Search + Chunk Building)
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F02 | F11 | `check_confidence()` → FAIL | Low confidence | | F02.2 | F11 | `check_confidence()` → FAIL | Low confidence |
| F11 | F12 | `create_chunk_on_tracking_loss()` | **Proactive chunk creation** | | F11 | F12 | `create_chunk_on_tracking_loss()` | **Proactive chunk creation** |
| F12 | F10 | `create_new_chunk()` | Create chunk in factor graph | | F12 | F10 | `create_chunk_subgraph()` | Create chunk in factor graph |
| F12 | F03 | `save_chunk_state()` | Persist chunk state for recovery | | F12 | F03 | `save_chunk_state()` | Persist chunk state for recovery |
| F02 | F12 | `get_active_chunk()` | Get new active chunk | | F02.2 | F12 | `get_active_chunk()` | Get new active chunk |
| F11 | F06 | `requires_rotation_sweep()` | Trigger rotation sweep (single-image) | | F11 | F06 | `requires_rotation_sweep()` | Trigger rotation sweep (single-image) |
| F11 | F08 | `retrieve_candidate_tiles()` | Coarse localization (single-image) | | F11 | F08 | `retrieve_candidate_tiles()` | Coarse localization (single-image) |
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model | | F08 | F16 | `get_inference_engine("DINOv2")` | Get model |
| F08 | H04 | `search()` | Query Faiss index | | F08 | H04 | `search()` | Query Faiss index |
| F08 | F04 | `get_tile_by_gps()` × 5 | Get candidate tiles | | F08 | F04 | `get_tile_by_gps()` × 5 | Get candidate tiles |
| F11 | F04 | `expand_search_grid(4)` | Get 2×2 grid | | F11 | F04 | `expand_search_grid(4)` | Get 2×2 grid |
@@ -266,14 +270,14 @@
| F11 (fail) | F04 | `expand_search_grid(9)` | Expand to 3×3 | | F11 (fail) | F04 | `expand_search_grid(9)` | Expand to 3×3 |
| F11 (fail) | F04 | `expand_search_grid(16)` | Expand to 4×4 | | F11 (fail) | F04 | `expand_search_grid(16)` | Expand to 4×4 |
| F11 (fail) | F04 | `expand_search_grid(25)` | Expand to 5×5 | | F11 (fail) | F04 | `expand_search_grid(25)` | Expand to 5×5 |
| F11 (fail) | F03 | Continue building chunk | **Chunk building continues** | | F11 (fail) | F12 | Continue building chunk | **Chunk building continues** |
| F11 (background) | F03 | `get_chunks_for_matching()` | Get unanchored chunks | | F11 (background) | F12 | `get_chunks_for_matching()` | Get unanchored chunks |
| F11 (background) | F08 | `retrieve_candidate_tiles_for_chunk()` | **Chunk semantic matching** | | F11 (background) | F08 | `retrieve_candidate_tiles_for_chunk()` | **Chunk semantic matching** |
| F11 (background) | F06 | `try_chunk_rotation_steps()` | **Chunk rotation sweeps** | | F11 (background) | F06 | `try_chunk_rotation_steps()` | **Chunk rotation sweeps** |
| F11 (background) | F09 | `align_chunk_to_satellite()` | **Chunk LiteSAM matching** | | F11 (background) | F09 | `align_chunk_to_satellite()` | **Chunk LiteSAM matching** |
| F11 (background) | F10 | `add_chunk_anchor()` + `merge_chunks()` | **Chunk merging** | | F11 (background) | F10 | `add_chunk_anchor()` + `merge_chunk_subgraphs()` | **Chunk merging** |
| F11 (fail) | F14 | `send_user_input_request()` | Request human help (last resort) | | F11 (fail) | F02.2 | Returns `UserInputRequest` | Request human help (last resort) |
| F11 | F02 | `update_flight_status("BLOCKED")` | Block processing | | F02.2 | F15 | `send_user_input_request()` | Send SSE event to client |
### Optimization & Results ### Optimization & Results
@@ -281,13 +285,13 @@
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F10 | H03 | `huber_loss()`, `cauchy_loss()` | Apply robust kernels | | F10 | H03 | `huber_loss()`, `cauchy_loss()` | Apply robust kernels |
| F10 | Internal | `optimize()` | Run iSAM2 optimization | | F10 | Internal | `optimize()` | Run iSAM2 optimization |
| F02 | F10 | `get_trajectory()` | Get optimized poses | | F02.2 | F10 | `get_trajectory()` | Get optimized poses |
| F02 | F13 | `enu_to_gps()` | Convert ENU to GPS | | F02.2 | F13 | `enu_to_gps()` | Convert ENU to GPS |
| F13 | H01 | `project()`, `unproject()` | Camera operations | | F13 | H01 | `project()`, `unproject()` | Camera operations |
| F13 | H02 | `compute_gsd()` | GSD calculations | | F13 | H02 | `compute_gsd()` | GSD calculations |
| F13 | H06 | `tile_to_latlon()` | Coordinate transforms | | F13 | H06 | `tile_to_latlon()` | Coordinate transforms |
| F02 | F14 | Frame GPS + object coords | Provide results | | F02.2 | F14 | Frame GPS + object coords | Provide results |
| F14 | F02 | `update_waypoint()` | Per-frame waypoint update | | F14 | F03 | `update_waypoint()` | Per-frame waypoint update |
| F14 | F15 | `send_frame_result()` | Publish to client | | F14 | F15 | `send_frame_result()` | Publish to client |
| F15 | Client | SSE `frame_processed` | Real-time delivery | | F15 | Client | SSE `frame_processed` | Real-time delivery |
| F14 | F03 | `save_frame_result()` | Persist frame result | | F14 | F03 | `save_frame_result()` | Persist frame result |
@@ -298,18 +302,20 @@
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F15 | Client | SSE `user_input_needed` | Notify client | | F15 | Client | SSE `user_input_needed` | Notify client |
| Client | F01 | `POST .../user-fix` | Provide anchor | | Client | F01 | `POST .../user-fix` | Provide anchor |
| F01 | F11 | `apply_user_anchor()` | Apply fix | | F01 | F02.1 | `handle_user_fix()` | Route through lifecycle manager |
| F02.1 | F02.2 | `apply_user_fix()` | Delegate to processing engine |
| F02.2 | F11 | `apply_user_anchor()` | Apply fix |
| F11 | F10 | `add_absolute_factor()` (high confidence) | Hard constraint | | F11 | F10 | `add_absolute_factor()` (high confidence) | Hard constraint |
| F10 | Internal | `optimize()` | Re-optimize | | F10 | Internal | `optimize()` | Re-optimize |
| F11 | Event | Emit `UserFixApplied` | F02 subscribes and resumes | | F02.2 | F15 | `send_frame_result()` | Publish result via SSE |
### Asynchronous Refinement ### Asynchronous Refinement
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F10 | Internal (background) | `optimize()` | Back-propagate anchors | | F10 | Internal (background) | `optimize()` | Back-propagate anchors |
| F10 | F14 | `get_trajectory()` | Get refined poses | | F14 | F10 | `get_trajectory()` | Get refined poses |
| F14 | F02 | `batch_update_waypoints()` | Batch update waypoints | | F14 | F03 | `batch_update_waypoints()` | Batch update waypoints |
| F14 | F15 | `send_refinement()` × N | Send updates | | F14 | F15 | `send_refinement()` × N | Send updates |
| F15 | Client | SSE `frame_refined` × N | Incremental updates | | F15 | Client | SSE `frame_refined` × N | Incremental updates |
@@ -334,31 +340,31 @@
| Source | Target | Method | Purpose | | Source | Target | Method | Purpose |
|--------|--------|--------|---------| |--------|--------|--------|---------|
| F16 | ALL | `get_*_config()` | Provide configuration | | F17 | ALL | `get_*_config()` | Provide configuration |
| H05 | F07, F08, F09, F10, F11 | `start_timer()`, `end_timer()` | Performance monitoring | | H05 | F07, F08, F09, F10, F11 | `start_timer()`, `end_timer()` | Performance monitoring |
--- ---
## Interaction Coverage Verification ## Interaction Coverage Verification
✅ **Initialization**: F02→F16, F17; F04→F08→H04 ✅ **Initialization**: F02.1→F16, F17; F04→F08→H04
✅ **Flight creation**: Client→F01→F02→F04,F12,F16,F17,F14 ✅ **Flight creation**: Client→F01→F02.1→F04,F12,F16,F17,F14
✅ **Image upload**: Client→F01→F05→H08,F17 ✅ **Image upload**: Client→F01→F02.1→F05→H08
✅ **Rotation sweep**: F06→H07,F09 (12 iterations) ✅ **Rotation sweep**: F06→H07,F09 (12 iterations)
✅ **Sequential VO**: F07→F16,F10(chunk),F12,H05 ✅ **Sequential VO**: F02.2→F07→F16,F10(chunk),F12,H05
✅ **Drift correction**: F02→F04,F09,F10 ✅ **Drift correction**: F02.2→F04,F09,F10
✅ **Tracking loss**: F11→F12(proactive chunk),F06,F08,F04(progressive),F09,F15,F02 ✅ **Tracking loss**: F02.2→F11→F12(proactive chunk),F06,F08,F04(progressive),F09
✅ **Chunk building**: F02→F12→F10,F07 ✅ **Chunk building**: F02.2→F12→F10,F07
✅ **Chunk image retrieval**: F12→F05(get_image_by_sequence for chunk images) ✅ **Chunk image retrieval**: F12→F05(get_image_by_sequence for chunk images)
✅ **Chunk semantic matching**: F11→F12→F08(chunk descriptor) ✅ **Chunk semantic matching**: F11→F12→F08(chunk descriptor)
✅ **Chunk LiteSAM matching**: F11→F06(chunk rotation)→F09(chunk alignment) ✅ **Chunk LiteSAM matching**: F11→F06(chunk rotation)→F09(chunk alignment)
✅ **Chunk merging**: F11→F10(Sim3 transform) ✅ **Chunk merging**: F11→F10(Sim3 transform)
✅ **Global PR**: F08→F16,H04,F04 ✅ **Global PR**: F08→F16,H04,F04
✅ **Optimization**: F10→H03(chunk optimization, global optimization) ✅ **Optimization**: F10→H03(chunk optimization, global optimization)
✅ **Coordinate transform**: F13→H01,H02,H06 ✅ **Coordinate transform**: F13→F10,H01,H02,H06
✅ **Results**: F02→F13→F14→F15,F03 ✅ **Results**: F02.2→F13→F14→F15,F03
✅ **User input**: Client→F01→F11→F10,F02 ✅ **User input**: Client→F01→F02.1→F02.2→F11→F10
✅ **Refinement**: F10→F14→F02,F15 ✅ **Refinement**: F10→F14→F03,F15
✅ **Configuration**: F17→ALL ✅ **Configuration**: F17→ALL
✅ **Performance**: H05→processing components ✅ **Performance**: H05→processing components
@@ -386,7 +392,9 @@
### Component Specifications Status ### Component Specifications Status
- [x] F01 Flight API (merged from R01 Route REST API) - [x] F01 Flight API (merged from R01 Route REST API)
- [x] F02 Flight Processor (merged from R02, R03, G02) - [x] F02.1 Flight Lifecycle Manager (split from F02 Flight Processor)
- [x] F02.2 Flight Processing Engine (split from F02 Flight Processor)
- [x] F03 Flight Database (merged from R04, G17)
- [x] F04 Satellite Data Manager - [x] F04 Satellite Data Manager
- [x] F05 Image Input Pipeline - [x] F05 Image Input Pipeline
- [x] F06 Image Rotation Manager - [x] F06 Image Rotation Manager
@@ -401,26 +409,31 @@
- [x] F15 SSE Event Streamer - [x] F15 SSE Event Streamer
- [x] F16 Model Manager - [x] F16 Model Manager
- [x] F17 Configuration Manager - [x] F17 Configuration Manager
- [x] F03 Flight Database (merged from R04, G17)
- [x] Helper components (H01-H08) - [x] Helper components (H01-H08)
--- ---
## Architecture Notes ## Architecture Notes
### F02 Flight Processor Complexity ### F02 Split: Lifecycle Manager + Processing Engine
F02 Flight Processor handles multiple concerns: F02 has been split into two components with clear Single Responsibility:
- Flight lifecycle management
- Processing loop orchestration
- Event subscription and state updates
- Coordination of F07/F08/F09/F10
**Current Status**: Acceptable for MVP. The responsibilities are related (all flight processing) and the component acts as a coordinator rather than implementing logic directly. **F02.1 Flight Lifecycle Manager**:
- Flight CRUD operations
- System initialization
- API request routing
- SSE stream creation
- User fix delegation
**Future Consideration**: If complexity grows, consider splitting into: **F02.2 Flight Processing Engine**:
- F02a Flight State Manager (lifecycle, status) - Main processing loop
- F02b Processing Loop Coordinator (frame processing orchestration) - Frame-by-frame processing orchestration
- Recovery coordination with F11
- Chunk management coordination with F12
- Runs in background thread per active flight
**Rationale**: This split aligns with Single Responsibility Principle. F02.1 handles external-facing operations (API layer), while F02.2 handles internal processing (background engine).
**Decision**: Keep as single component. Clear internal organization with separate methods for state vs processing concerns. Event-based communication with F11 keeps recovery logic separate. **Decision**: Keep as single component. Clear internal organization with separate methods for state vs processing concerns. Event-based communication with F11 keeps recovery logic separate.
docs/02_components/helpers/h04_faiss_index_manager_spec.md
+14
View File
@@ -107,6 +107,20 @@ H04 supports automatic fallback from GPU to CPU:
- `set_device("gpu")`: Use GPU acceleration (faster for large indexes) - `set_device("gpu")`: Use GPU acceleration (faster for large indexes)
- `set_device("cpu")`: Use CPU (fallback when GPU unavailable) - `set_device("cpu")`: Use CPU (fallback when GPU unavailable)
## Current vs Future Use Cases
### Current Use (MVP)
- **Satellite Index Loading**: F08 uses `load_index()` to load pre-built satellite descriptor index from provider.
- **Similarity Search**: F08 uses `search()` to find candidate satellite tiles.
### Future Use Cases (build_index, add_descriptors)
The `build_index()` and `add_descriptors()` methods are reserved for future features:
1. **UAV Loop Closure Detection**: Build index of UAV frame descriptors to detect when UAV revisits previously seen areas.
2. **Chunk-to-Chunk Matching**: Build index of chunk descriptors for matching disconnected trajectory segments.
3. **Flight-to-Flight Matching**: Match current flight to previous flights for multi-flight consistency.
**Note**: For MVP, F08 does NOT build satellite indexes - they are provided pre-built by the satellite data provider.
## Test Cases ## Test Cases
1. Build index with 10,000 UAV image descriptors → succeeds 1. Build index with 10,000 UAV image descriptors → succeeds
docs/02_components/system_flows.md
+55 -42
View File
@@ -11,7 +11,8 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
| ID | Component | Interface | Purpose | | ID | Component | Interface | Purpose |
|----|-----------|-----------|---------| |----|-----------|-----------|---------|
| F01 | Flight API | `IFlightAPI` | REST endpoints, SSE streaming | | F01 | Flight API | `IFlightAPI` | REST endpoints, SSE streaming |
| F02 | Flight Processor | `IFlightProcessor` | Central coordinator, processing loop | | F02.1 | Flight Lifecycle Manager | `IFlightLifecycleManager` | Flight CRUD, init, API delegation |
| F02.2 | Flight Processing Engine | `IFlightProcessingEngine` | Processing loop, recovery orchestration |
| F03 | Flight Database | `IFlightDatabase` | Persistence layer | | F03 | Flight Database | `IFlightDatabase` | Persistence layer |
| F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search | | F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search |
| F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage | | F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage |
@@ -86,7 +87,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
└────────┬────────┘ └────────┬────────┘
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ F02 Ready │ ← Ready to accept flights │ F02.1 Ready │ ← Ready to accept flights
└────────┬────────┘ └────────┬────────┘
▼ ▼
┌─────────────────┐ ┌─────────────────┐
@@ -113,7 +114,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ create_flight() │ create_flight()
▼ ▼
┌─────────┐ ┌─────────┐
│ F02 │ Flight Processor │ F02.1 │ Flight Lifecycle Manager
└────┬────┘ └────┬────┘
┌────────────────────────┬─┴─┬────────────────────────┐ ┌────────────────────────┬─┴─┬────────────────────────┐
│ │ │ │ │ │ │ │
@@ -186,7 +187,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**: **Sequence**:
``` ```
┌──────────────────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │ │ F02.2 Flight Processing Engine │
│ │ │ │
│ ┌─────────────┐ │ │ ┌─────────────┐ │
│ │ get_next │◄───────────────────────────────────────────────────────┐│ │ │ get_next │◄───────────────────────────────────────────────────────┐│
@@ -205,7 +206,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ └──────────────────────────────┬──────────────────────────────┘ ││ │ └──────────────────────────────┬──────────────────────────────┘ ││
│ ▼ ││ │ ▼ ││
│ ┌─────────────────────────────────────────────────────────────┐ ││ │ ┌─────────────────────────────────────────────────────────────┐ ││
│ │ F07 compute_relative_pose_in_chunk() │ ││ │ │ F07 compute_relative_pose() │ ││
│ │ ├─ SuperPoint extract (prev + curr) via F16 │ ││ │ │ ├─ SuperPoint extract (prev + curr) via F16 │ ││
│ │ └─ LightGlue match via F16 │ ││ │ │ └─ LightGlue match via F16 │ ││
│ └──────────────────────────────┬──────────────────────────────┘ ││ │ └──────────────────────────────┬──────────────────────────────┘ ││
@@ -230,15 +231,17 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ ┌─────────────────┴─────────────────┐ ││ │ ┌─────────────────┴─────────────────┐ ││
│ │ Match Found? │ ││ │ │ Match Found? │ ││
│ ▼ ▼ ││ │ ▼ ▼ ││
│ ┌─────────────────────────┐ ┌─────────────────────────┐ ││ │ ┌──────────────────────────────────┐ ┌─────────────────────────┐ ││
│ │F10 add_absolute_factor()│ │ Skip absolute anchor │ ││ │ │F10 add_absolute_factor(flight_id,│ │ Skip absolute anchor │ ││
│ │F06 update_heading() │ │ (VO-only frame) │ ││ │ │ frame_id, gps, covariance, │ │ (VO-only frame) │ ││
│ └────────────┬────────────┘ └─────────────────────────┘ ││ │ │ is_user_anchor=False) │ └─────────────────────────┘ ││
│ │F06 update_heading() │ ││
│ └────────────┬─────────────────────┘ ││
│ │ ││ │ │ ││
│ └─────────────────────┬───────────────────────────────────┘│ │ └─────────────────────┬───────────────────────────────────┘│
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ F10 optimize_chunk() │ │ │ │ F10 optimize_chunk(flight_id, chunk_id, iterations) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
@@ -324,7 +327,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ │ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. EMIT RecoveryStarted event │ │ │ │ 1. EMIT RecoveryStarted event │ │
│ │ └─ F02 subscribes → Update status to "recovering" │ │ │ │ └─ F02.2 updates status to "recovering" │ │
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
@@ -381,7 +384,8 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ │ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ create_chunk(flight_id, start_frame_id) │ │ │ │ create_chunk(flight_id, start_frame_id) │ │
│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │ │ │ ├─ F10 create_chunk_subgraph(flight_id, chunk_id, │ │
│ │ │ start_frame_id) ← Factor graph subgraph │ │
│ │ ├─ Initialize chunk state (unanchored, active) │ │ │ │ ├─ Initialize chunk state (unanchored, active) │ │
│ │ └─ F03 save_chunk_state() │ │ │ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
@@ -390,16 +394,19 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ For each frame in chunk: │ │ │ │ For each frame in chunk: │ │
│ │ │ │ │ │ │ │
│ │ ┌─────────────────────────────────────┐ │ │ │ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F07 compute_relative_pose_in_chunk()│ │ │ │ │ │ F07 compute_relative_pose() │ │ │
│ │ └───────────────────┬─────────────────┘ │ │ │ │ └───────────────────┬─────────────────┘ │ │
│ │ ▼ │ │ │ │ ▼ │ │
│ │ ┌──────────────────────────────────────┐ │ │ │ │ ┌──────────────────────────────────────┐ │ │
│ │ │ F12 add_frame_to_chunk() │ │ │ │ │ │ F12 add_frame_to_chunk() │ │ │
│ │ │ └─ F10 add_relative_factor_to_chunk│ │ │ │ │ │ └─ F10 add_relative_factor_to_chunk│ │ │
│ │ │ (flight_id, chunk_id, frame_i, │ │ │
│ │ │ frame_j, relative_pose, cov) │ │ │
│ │ └───────────────────┬──────────────────┘ │ │ │ │ └───────────────────┬──────────────────┘ │ │
│ │ ▼ │ │ │ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │ │ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F10 optimize_chunk() (local) │ │ │ │ │ │ F10 optimize_chunk(flight_id, │ │ │
│ │ │ chunk_id, iterations) (local) │ │ │
│ │ └─────────────────────────────────────┘ │ │ │ │ └─────────────────────────────────────┘ │ │
│ │ │ │ │ │ │ │
│ └────────────────────────────────────────────────────┘ │ │ └────────────────────────────────────────────────────┘ │
@@ -481,7 +488,8 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 2. Anchor chunk: F12 mark_chunk_anchored(chunk_id, gps) │ │ │ │ 2. Anchor chunk: F12 mark_chunk_anchored(chunk_id, gps) │ │
│ │ └─ F10 add_chunk_anchor() │ │ │ │ └─ F10 add_chunk_anchor(flight_id, chunk_id, frame_id, │ │
│ │ gps, covariance) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
@@ -490,20 +498,22 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 4. Merge: F12 merge_chunks(chunk_id, target_chunk_id, Sim3) │ │ │ │ 4. Merge: F12 merge_chunks(target_chunk_id, chunk_id, Sim3) │ │
│ │ ├─ F10 merge_chunks() ← Apply Sim(3) transform │ │ │ │ ├─ F10 merge_chunk_subgraphs(flight_id, chunk_id, │ │
│ │ │ target_chunk_id, transform) ← Apply Sim(3) │ │
│ │ ├─ F12 deactivate_chunk(chunk_id) │ │ │ │ ├─ F12 deactivate_chunk(chunk_id) │ │
│ │ └─ F03 save_chunk_state() │ │ │ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 5. Optimize global: F10 optimize_global() │ │ │ │ 5. Optimize global: F10 optimize_global(flight_id, │ │
│ │ iterations) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 6. EMIT ChunkMerged event (flight_id,chunk_id,merged_frames)│ │ │ │ 6. EMIT ChunkMerged event (flight_id,chunk_id,merged_frames)│ │
│ │ └─ F14 subscribes → update_results_after_chunk_merge() │ │ │ │ └─ F14 subscribes → update_results_after_chunk_merge() │ │
│ │ ├─ F10 get_trajectory() → ENU poses │ │ │ │ ├─ F10 get_trajectory(flight_id) → ENU poses │ │
│ │ ├─ F13 enu_to_gps() for each frame │ │ │ │ ├─ F13 enu_to_gps() for each frame │ │
│ │ ├─ F03 save_frame_result() + update_waypoint() │ │ │ │ ├─ F03 save_frame_result() + update_waypoint() │ │
│ │ └─ F15 send_refinement() → SSE "frame_refined" × N │ │ │ │ └─ F15 send_refinement() → SSE "frame_refined" × N │ │
@@ -532,7 +542,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ 3. Create UserInputRequest │ │ │ │ 3. Create UserInputRequest │ │
│ │ 4. F15 send_user_input_request() → SSE "user_input_needed" │ │ │ │ 4. F15 send_user_input_request() → SSE "user_input_needed" │ │
│ │ 5. EMIT UserInputNeeded event │ │ │ │ 5. EMIT UserInputNeeded event │ │
│ │ └─ F02 subscribes → Update status to "BLOCKED" │ │ │ │ └─ F02.2 updates status to "BLOCKED" │ │
│ └─────────────────────────────────────────────────────────────┘ │ │ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘ └──────────────────────────────────────────────────────────────────────────┘
│ │
@@ -561,7 +571,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ 2. F10 add_absolute_factor(is_user_anchor=True) ← σ=5m │ │ │ │ 2. F10 add_absolute_factor(is_user_anchor=True) ← σ=5m │ │
│ │ 3. F10 optimize() │ │ │ │ 3. F10 optimize() │ │
│ │ 4. EMIT UserFixApplied event │ │ │ │ 4. EMIT UserFixApplied event │ │
│ │ └─ F02 subscribes → Update status to "PROCESSING" │ │ │ │ └─ F02.2 updates status to "PROCESSING" │ │
│ │ 5. Resume processing loop │ │ │ │ 5. Resume processing loop │ │
│ └─────────────────────────────────────────────────────────────┘ │ │ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘ └──────────────────────────────────────────────────────────────────────────┘
@@ -591,7 +601,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ 3. Identify refined frames │ │ │ │ 3. Identify refined frames │ │
│ │ 4. F14 mark_refined(frame_ids) │ │ │ │ 4. F14 mark_refined(frame_ids) │ │
│ │ ├─ For each frame: │ │ │ │ ├─ For each frame: │ │
│ │ │ ├─ F10 get_trajectory() → ENU pose │ │ │ │ │ ├─ F10 get_trajectory(flight_id) → ENU pose │ │
│ │ │ ├─ F13 enu_to_gps(flight_id, enu_pose) │ │ │ │ │ ├─ F13 enu_to_gps(flight_id, enu_pose) │ │
│ │ │ ├─ F03 save_frame_result(refined=True) │ │ │ │ │ ├─ F03 save_frame_result(refined=True) │ │
│ │ │ └─ F03 update_waypoint() │ │ │ │ │ └─ F03 update_waypoint() │ │
@@ -651,12 +661,13 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**: **Sequence**:
``` ```
┌──────────────────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │ │ F02.1 Flight Lifecycle Manager │
│ │ │ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. All frames processed │ │ │ │ 1. All frames processed (Signal from F02.2) │ │
│ │ 2. Wait for pending chunk merges (if any) │ │ │ │ 2. Wait for pending chunk merges (if any) │ │
│ │ 3. F10 optimize_global(iterations=100) ← Final optimization │ │ │ │ 3. F10 optimize_global(flight_id, iterations=100) │ │
│ │ ← Final optimization │ │
│ └──────────────────────────────┬──────────────────────────────┘ │ │ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────┐ │
@@ -699,7 +710,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
└────────┬────────┘ └────────┬────────┘
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ F02 Complete/ │ ← Complete or cancel active flights │ F02.1 Complete/ │ ← Complete or cancel active flights
│ Cancel │ │ Cancel │
└────────┬────────┘ └────────┬────────┘
▼ ▼
@@ -760,8 +771,13 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
┌─────────────────────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────────────────────┐
│ ORCHESTRATION LAYER │ │ ORCHESTRATION LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F02 Flight Processor │ │ │ │ F02.1 Flight Lifecycle Manager │ │
│ │ (Central coordinator, event subscriber, background task manager) │ │ │ └─────────────────────────────────────────────────────────────────────┘ │
│ │ (Spawns/Manages) │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F02.2 Flight Processing Engine │ │
│ │ (Processing loop, State machine, Recovery orchestration) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │ │ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────────────────┘
│ │
@@ -772,7 +788,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ ┌─────────────┐ ┌─────────────┐ │ │ ┌───────────────────────┐ │ │ ┌─────────────┐ ┌─────────────┐ │ │ ┌───────────────────────┐ │
│ │ F04 │ │ F05 │ │ │ │ F11 │ │ │ │ F04 │ │ F05 │ │ │ │ F11 │ │
│ │ Satellite │ │ Image │ │ │ │ Failure Recovery │ │ │ │ Satellite │ │ Image │ │ │ │ Failure Recovery │ │
│ │ Data │ │ Input │ │ │ │ (Event emitter) │ │ │ │ Data │ │ Input │ │ │ │ (Logic & Strategies) │ │
│ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │ │ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │
│ │ │ │ │ │ │ │ │ │
│ ┌─────────────┐ ┌─────────────┐ │ │ ▼ │ │ ┌─────────────┐ ┌─────────────┐ │ │ ▼ │
@@ -827,20 +843,18 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
--- ---
## Event-Based Communication ## Status-Based Recovery Communication
F11 Failure Recovery Coordinator emits events instead of directly calling F02: F11 Failure Recovery Coordinator uses **Direct Call Returns** to communicate with F02.2 Flight Processing Engine. Events have been removed to eliminate circular dependencies and clarify control flow.
| Event | Publisher | Subscribers | Action | | Action | Caller | Called | Return/Outcome |
|-------|-----------|-------------|--------| |--------|--------|--------|----------------|
| `RecoveryStarted` | F11 | F02 | Update status to "recovering" | | `start_search` | F02.2 | F11 | `SearchSession` object |
| `RecoverySucceeded` | F11 | F02 | Update status to "processing", resume | | `try_current_grid` | F02.2 | F11 | `AlignmentResult` (or None) |
| `RecoveryFailed` | F11 | F02 | Update status to "blocked" | | `create_user_input_request` | F02.2 | F11 | `UserInputRequest` object |
| `UserInputNeeded` | F11 | F02 | Update status to "blocked", await fix | | `apply_user_anchor` | F02.2 | F11 | Success boolean |
| `UserFixApplied` | F11 | F02 | Update status to "processing", resume |
| `ChunkCreated` | F11 | F02 | Log chunk creation | F02.2 is responsible for updating the flight status (e.g., "recovering", "blocked", "processing") based on these return values.
| `ChunkAnchored` | F11 | F02 | Log chunk anchor |
| `ChunkMerged` | F11 | F02, F14 | F14 updates results for merged frames |
--- ---
@@ -880,4 +894,3 @@ F11 Failure Recovery Coordinator emits events instead of directly calling F02:
| Mean Reprojection Error | < 1.0 pixels | | Mean Reprojection Error | < 1.0 pixels |
| Place Recognition Recall@5 | > 85% | | Place Recognition Recall@5 | > 85% |
| LiteSAM Success Rate | > 95% (when rotation correct) | | LiteSAM Success Rate | > 95% (when rotation correct) |