diff --git a/docs/02_components/01_flight_api/flight_api_spec.md b/docs/02_components/01_flight_api/flight_api_spec.md index 8173af4..65e778d 100644 --- a/docs/02_components/01_flight_api/flight_api_spec.md +++ b/docs/02_components/01_flight_api/flight_api_spec.md @@ -311,8 +311,9 @@ BatchResponse: 1. Validate flight_id exists 2. Validate batch size (10-50 images) 3. Validate sequence numbers (strict sequential) -4. Pass to F05 Image Input Pipeline -5. Return immediately (processing is async) +4. Call F02 Flight Processor → queue_images(flight_id, batch) +5. F02 delegates to F05 Image Input Pipeline +6. Return immediately (processing is async) **Error Conditions**: - `400 Bad Request`: Invalid batch size, out-of-sequence images @@ -355,9 +356,10 @@ UserFixResponse: **Processing Flow**: 1. Validate flight_id exists and is blocked -2. Pass to F11 Failure Recovery Coordinator -3. Coordinator applies anchor to Factor Graph -4. Resume processing pipeline +2. Call F02 Flight Processor → handle_user_fix(flight_id, fix_data) +3. F02 delegates to F11 Failure Recovery Coordinator +4. Coordinator applies anchor to Factor Graph +5. Resume processing pipeline **Error Conditions**: - `400 Bad Request`: Invalid fix data @@ -400,8 +402,9 @@ ObjectGPSResponse: **Processing Flow**: 1. Validate flight_id and frame_id exist 2. Validate frame has been processed (has pose in Factor Graph) -3. Call F13.image_object_to_gps(pixel, frame_id) -4. Return GPS with accuracy estimate +3. Call F02 Flight Processor → convert_object_to_gps(flight_id, frame_id, pixel) +4. F02 delegates to F13.image_object_to_gps(flight_id, frame_id, pixel) +5. Return GPS with accuracy estimate **Error Conditions**: - `400 Bad Request`: Invalid pixel coordinates @@ -479,6 +482,12 @@ SSE Stream with events: - flight_completed ``` +**Processing Flow**: +1. Validate flight_id exists +2. Call F02 Flight Processor → create_client_stream(flight_id, client_id) +3. F02 delegates to F15 SSE Event Streamer → create_stream() +4. Return SSE stream to client + **Event Format**: ```json { @@ -575,11 +584,9 @@ SSE Stream with events: ## Dependencies ### Internal Components -- **F02 Flight Processor**: For all flight operations and processing orchestration -- **F05 Image Input Pipeline**: For batch processing -- **F11 Failure Recovery Coordinator**: For user fixes -- **F15 SSE Event Streamer**: For real-time streaming -- **F03 Flight Database**: For persistence (via F02) +- **F02 Flight Processor**: For ALL operations (flight CRUD, image batching, user fixes, SSE streams, object-to-GPS conversion). F01 is a thin REST layer that delegates all business logic to F02. + +**Note**: F01 does NOT directly call F05, F11, F13, or F15. All operations are routed through F02 to maintain a single coordinator pattern. ### External Dependencies - **FastAPI**: Web framework diff --git a/docs/02_components/02_flight_processor/flight_processor_spec.md b/docs/02_components/02_flight_processor/flight_processor_spec.md index fdfbada..1fd5a61 100644 --- a/docs/02_components/02_flight_processor/flight_processor_spec.md +++ b/docs/02_components/02_flight_processor/flight_processor_spec.md @@ -59,6 +59,27 @@ class IFlightProcessor(ABC): def validate_flight_continuity(self, waypoints: List[Waypoint]) -> ValidationResult: pass + # API Delegation Methods (called by F01) + @abstractmethod + def queue_images(self, flight_id: str, batch: ImageBatch) -> BatchQueueResult: + """Delegates to F05 Image Input Pipeline.""" + pass + + @abstractmethod + def handle_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResult: + """Delegates to F11 Failure Recovery Coordinator.""" + pass + + @abstractmethod + def create_client_stream(self, flight_id: str, client_id: str) -> StreamConnection: + """Delegates to F15 SSE Event Streamer.""" + pass + + @abstractmethod + def convert_object_to_gps(self, flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint: + """Delegates to F13 Coordinate Transformer.""" + pass + # Processing Orchestration @abstractmethod def process_frame(self, flight_id: str, frame_id: int) -> FrameResult: @@ -576,6 +597,129 @@ ValidationResult: --- +## API Delegation Methods + +These methods are called by F01 Flight API and delegate to specialized components. This maintains F02 as the single coordinator for all operations. + +### `queue_images(flight_id: str, batch: ImageBatch) -> BatchQueueResult` + +**Description**: Queues image batch for processing. Delegates to F05 Image Input Pipeline. + +**Called By**: +- F01 Flight API (upload_image_batch endpoint) + +**Input**: +```python +flight_id: str +batch: ImageBatch +``` + +**Output**: +```python +BatchQueueResult: + accepted: bool + sequences: List[int] + next_expected: int + message: Optional[str] +``` + +**Processing Flow**: +1. Validate flight exists and is in valid state +2. Delegate to F05 Image Input Pipeline → queue_batch() +3. Return result to F01 + +--- + +### `handle_user_fix(flight_id: str, fix_data: UserFixRequest) -> UserFixResult` + +**Description**: Handles user-provided GPS fix. Delegates to F11 Failure Recovery Coordinator. + +**Called By**: +- F01 Flight API (submit_user_fix endpoint) + +**Input**: +```python +flight_id: str +fix_data: UserFixRequest: + frame_id: int + uav_pixel: Tuple[float, float] + satellite_gps: GPSPoint +``` + +**Output**: +```python +UserFixResult: + accepted: bool + processing_resumed: bool + message: Optional[str] +``` + +**Processing Flow**: +1. Validate flight exists and is blocked +2. Delegate to F11 Failure Recovery Coordinator → apply_user_anchor() +3. F11 emits UserFixApplied event (F02 subscribes and resumes processing) +4. Return result to F01 + +--- + +### `create_client_stream(flight_id: str, client_id: str) -> StreamConnection` + +**Description**: Creates SSE stream for client. Delegates to F15 SSE Event Streamer. + +**Called By**: +- F01 Flight API (create_sse_stream endpoint) + +**Input**: +```python +flight_id: str +client_id: str +``` + +**Output**: +```python +StreamConnection: + stream_id: str + flight_id: str + client_id: str + last_event_id: Optional[str] +``` + +**Processing Flow**: +1. Validate flight exists +2. Delegate to F15 SSE Event Streamer → create_stream() +3. Return StreamConnection to F01 + +--- + +### `convert_object_to_gps(flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint` + +**Description**: Converts object pixel to GPS. Delegates to F13 Coordinate Transformer. + +**Called By**: +- F01 Flight API (convert_object_to_gps endpoint) + +**Input**: +```python +flight_id: str +frame_id: int +pixel: Tuple[float, float] +``` + +**Output**: +```python +GPSPoint: + lat: float + lon: float +``` + +**Processing Flow**: +1. Validate flight and frame exist +2. Validate frame has been processed (has pose) +3. Delegate to F13 Coordinate Transformer → image_object_to_gps(flight_id, frame_id, pixel) +4. Return GPSPoint to F01 + +--- + ## Processing Orchestration Methods ### `process_frame(flight_id: str, frame_id: int) -> FrameResult` diff --git a/docs/02_components/03_flight_database/flight_database_spec.md b/docs/02_components/03_flight_database/flight_database_spec.md index e0b1465..0021292 100644 --- a/docs/02_components/03_flight_database/flight_database_spec.md +++ b/docs/02_components/03_flight_database/flight_database_spec.md @@ -676,6 +676,13 @@ Optional[Dict]: Metadata dictionary or None ## Chunk State Operations +**Necessity**: These methods are **required** for crash recovery. Without them: +- Chunk state would be lost on system restart +- Processing would need to start from scratch +- Background chunk matching progress would be lost + +F12 Route Chunk Manager delegates persistence to F03 to maintain separation of concerns (F12 manages chunk logic, F03 handles storage). + ### `save_chunk_state(flight_id: str, chunk: ChunkHandle) -> bool` **Description**: Saves chunk state to database for crash recovery. diff --git a/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md b/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md index 8353a62..a4b02b6 100644 --- a/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md +++ b/docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md @@ -401,8 +401,8 @@ ProcessingStatus: ## Dependencies ### Internal Components -- **H08 Batch Validator**: For validation logic - **F03 Flight Database**: For metadata persistence and flight state information +- **H08 Batch Validator**: For batch validation (naming convention, sequence continuity, format, dimensions) ### External Dependencies - **opencv-python**: Image I/O diff --git a/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md b/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md index 8875235..e10c23a 100644 --- a/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md +++ b/docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md @@ -495,6 +495,7 @@ return None # No match found ## Dependencies ### Internal Components +- **F04 Satellite Data Manager**: For tile fetching (`fetch_tile`) and tile bounds computation (`compute_tile_bounds`) - **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching. - **H07 Image Rotation Utils**: For image rotation and angle calculations - **F12 Route Chunk Manager**: For chunk image retrieval diff --git a/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md b/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md index 0675ab8..7cd5211 100644 --- a/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md +++ b/docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md @@ -23,12 +23,10 @@ class ISequentialVO(ABC): @abstractmethod def estimate_motion(self, matches: Matches, camera_params: CameraParameters) -> Optional[Motion]: pass - - @abstractmethod - def compute_relative_pose_in_chunk(self, prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]: - pass ``` +**Note**: F07 is chunk-agnostic. It only computes relative poses between images. The caller (F02 Flight Processor) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10. + ## Component Description ### Responsibilities @@ -38,14 +36,13 @@ class ISequentialVO(ABC): - Estimate relative pose (translation + rotation) between frames - Return relative pose factors for Factor Graph Optimizer - Detect tracking loss (low inlier count) -- **Chunk-aware VO operations (factors added to chunk subgraph)** ### Scope - Frame-to-frame visual odometry - Feature-based motion estimation - Handles low overlap and challenging agricultural environments - Provides relative measurements for trajectory optimization -- **Chunk-scoped operations (Atlas multi-map architecture)** +- **Chunk-agnostic**: F07 doesn't know about chunks. Caller (F02) routes results to appropriate chunk subgraph. ## API Methods @@ -224,49 +221,6 @@ Motion: 2. **Low inliers**: May return None 3. **Degenerate motion**: Handles pure rotation ---- - -### `compute_relative_pose_in_chunk(prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]` - -**Description**: Computes relative camera pose between consecutive frames within a chunk context. - -**Called By**: -- F02 Flight Processor (chunk-aware processing) - -**Input**: -```python -prev_image: np.ndarray # Previous frame (t-1) -curr_image: np.ndarray # Current frame (t) -chunk_id: str # Chunk identifier for context -``` - -**Output**: -```python -RelativePose: - translation: np.ndarray # (x, y, z) in meters - rotation: np.ndarray # 3×3 rotation matrix or quaternion - confidence: float # 0.0 to 1.0 - inlier_count: int - total_matches: int - tracking_good: bool - chunk_id: str # Chunk context -``` - -**Processing Flow**: -1. Same as compute_relative_pose() (SuperPoint + LightGlue) -2. Return RelativePose with chunk_id context -3. Factor will be added to chunk's subgraph (not global graph) - -**Chunk Context**: -- VO operations are chunk-scoped -- Factors added to chunk's subgraph via F10.add_relative_factor_to_chunk() -- Chunk isolation ensures independent optimization - -**Test Cases**: -1. **Chunk-aware VO**: Returns RelativePose with chunk_id -2. **Chunk isolation**: Factors isolated to chunk -3. **Multiple chunks**: VO operations don't interfere between chunks - ## Integration Tests ### Test 1: Normal Flight Sequence diff --git a/docs/02_components/09_metric_refinement/metric_refinement_spec.md b/docs/02_components/09_metric_refinement/metric_refinement_spec.md index 138433f..0556707 100644 --- a/docs/02_components/09_metric_refinement/metric_refinement_spec.md +++ b/docs/02_components/09_metric_refinement/metric_refinement_spec.md @@ -377,11 +377,11 @@ Optional[np.ndarray]: 3×3 homography matrix or None ## Dependencies ### Internal Components +- **F12 Route Chunk Manager**: For chunk image retrieval and chunk operations - **F16 Model Manager**: For LiteSAM model - **H01 Camera Model**: For projection operations - **H02 GSD Calculator**: For coordinate transformations - **H05 Performance Monitor**: For timing -- **F12 Route Chunk Manager**: For chunk image retrieval **Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager) diff --git a/docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md b/docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md index bb53291..afa6f81 100644 --- a/docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md +++ b/docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md @@ -8,64 +8,67 @@ ```python class IFactorGraphOptimizer(ABC): + # All methods take flight_id to support concurrent flights + # F10 maintains Dict[str, FactorGraph] keyed by flight_id internally + @abstractmethod - def add_relative_factor(self, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool: + def add_relative_factor(self, flight_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool: pass @abstractmethod - def add_absolute_factor(self, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool: + def add_absolute_factor(self, flight_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool: pass @abstractmethod - def add_altitude_prior(self, frame_id: int, altitude: float, covariance: float) -> bool: + def add_altitude_prior(self, flight_id: str, frame_id: int, altitude: float, covariance: float) -> bool: pass @abstractmethod - def optimize(self, iterations: int) -> OptimizationResult: + def optimize(self, flight_id: str, iterations: int) -> OptimizationResult: pass @abstractmethod - def get_trajectory(self) -> Dict[int, Pose]: + def get_trajectory(self, flight_id: str) -> Dict[int, Pose]: pass @abstractmethod - def get_marginal_covariance(self, frame_id: int) -> np.ndarray: + def get_marginal_covariance(self, flight_id: str, frame_id: int) -> np.ndarray: + pass + + # Chunk operations - F10 only manages factor graph subgraphs + # F12 owns chunk metadata (status, is_active, etc.) + @abstractmethod + def create_chunk_subgraph(self, flight_id: str, chunk_id: str, start_frame_id: int) -> bool: pass @abstractmethod - def create_new_chunk(self, chunk_id: str, start_frame_id: int) -> ChunkHandle: + def add_relative_factor_to_chunk(self, flight_id: str, chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool: pass @abstractmethod - def get_chunk_for_frame(self, frame_id: int) -> Optional[ChunkHandle]: + def add_chunk_anchor(self, flight_id: str, chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool: pass @abstractmethod - def add_relative_factor_to_chunk(self, chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool: + def merge_chunk_subgraphs(self, flight_id: str, source_chunk_id: str, target_chunk_id: str, transform: Sim3Transform) -> bool: + """Merges source_chunk INTO target_chunk. Source chunk subgraph is merged into target.""" pass @abstractmethod - def add_chunk_anchor(self, chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool: + def get_chunk_trajectory(self, flight_id: str, chunk_id: str) -> Dict[int, Pose]: pass @abstractmethod - def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool: + def optimize_chunk(self, flight_id: str, chunk_id: str, iterations: int) -> OptimizationResult: pass @abstractmethod - def get_chunk_trajectory(self, chunk_id: str) -> Dict[int, Pose]: + def optimize_global(self, flight_id: str, iterations: int) -> OptimizationResult: pass @abstractmethod - def get_all_chunks(self) -> List[ChunkHandle]: - pass - - @abstractmethod - def optimize_chunk(self, chunk_id: str, iterations: int) -> OptimizationResult: - pass - - @abstractmethod - def optimize_global(self, iterations: int) -> OptimizationResult: + def delete_flight_graph(self, flight_id: str) -> bool: + """Cleanup factor graph when flight is deleted.""" pass ``` @@ -84,16 +87,20 @@ class IFactorGraphOptimizer(ABC): ### Chunk Responsibility Clarification **F10 provides low-level factor graph operations only**: -- `create_new_chunk()`: Creates subgraph in factor graph +- `create_chunk_subgraph()`: Creates subgraph in factor graph (returns bool, not ChunkHandle) - `add_relative_factor_to_chunk()`: Adds factors to chunk's subgraph -- `add_chunk_anchor()`: Adds GPS anchor to chunk -- `merge_chunks()`: Applies Sim(3) transform and merges subgraphs +- `add_chunk_anchor()`: Adds GPS anchor to chunk's subgraph +- `merge_chunk_subgraphs()`: Applies Sim(3) transform and merges subgraphs - `optimize_chunk()`, `optimize_global()`: Runs optimization -**F12 is the source of truth for chunk state** (see F12 spec): -- Chunk lifecycle management (active, anchored, merged status) +**F10 does NOT own chunk metadata** - only factor graph data structures. + +**F12 is the source of truth for ALL chunk state** (see F12 spec): +- ChunkHandle with all metadata (is_active, has_anchor, matching_status) +- Chunk lifecycle management - Chunk readiness determination - High-level chunk queries +- F12 calls F10 for factor graph operations **F11 coordinates recovery** (see F11 spec): - Triggers chunk creation via F12 @@ -155,16 +162,19 @@ F07 returns unit translation vectors due to monocular scale ambiguity. F10 resol **Explicit Flow**: ```python # In add_relative_factor(): -# altitude comes from F05 Image Input Pipeline (extracted from EXIF metadata) +# altitude comes from F17 Configuration Manager (predefined operational altitude) # focal_length, sensor_width from F17 Configuration Manager -gsd = H02.compute_gsd(altitude, focal_length, sensor_width, image_width) +config = F17.get_flight_config(flight_id) +altitude = config.altitude # Predefined altitude, NOT from EXIF +gsd = H02.compute_gsd(altitude, config.camera_params.focal_length, + config.camera_params.sensor_width, + config.camera_params.resolution_width) expected_displacement = frame_spacing * gsd # ~100m typical at 300m altitude scaled_translation = relative_pose.translation * expected_displacement # Add scaled_translation to factor graph ``` -**Note**: Altitude is passed through the processing chain: -- F05 extracts altitude from EXIF → F02 includes in FrameData → F10 receives with add_relative_factor() +**Note**: Altitude comes from F17 Configuration Manager (predefined operational altitude), NOT from EXIF metadata. The problem statement specifies images don't have GPS metadata. **Output**: ```python diff --git a/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md b/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md index 92c9760..7707606 100644 --- a/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md +++ b/docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md @@ -95,7 +95,11 @@ class IFailureRecoveryCoordinator(ABC): F11 emits events instead of directly calling F02's status update methods. This decouples recovery logic from flight state management. -**Events Emitted**: +### Internal Events (Component-to-Component) + +F11 emits events for internal component communication. These are NOT directly sent to clients. + +**Events Emitted (internal)**: - `RecoveryStarted`: When tracking loss detected and recovery begins - `RecoverySucceeded`: When recovery finds a match (single-image or chunk) - `RecoveryFailed`: When all recovery strategies exhausted @@ -104,12 +108,27 @@ F11 emits events instead of directly calling F02's status update methods. This d - `ChunkCreated`: When new chunk created on tracking loss - `ChunkAnchored`: When chunk successfully matched and anchored - `ChunkMerged`: When chunk merged into main trajectory (includes flight_id, chunk_id, merged_frames) +- `ChunkMatchingFailed`: When chunk matching exhausts all candidates and fails **Event Listeners** (F02 Flight Processor subscribes to these): - On `RecoveryStarted`: Update status to "recovering" - On `RecoveryFailed`: Update status to "blocked" - On `RecoverySucceeded`: Update status to "processing" - On `UserInputNeeded`: Update status to "blocked", blocked=True +- On `ChunkMatchingFailed`: Re-queue chunk for user input or continue building + +### External SSE Events (to Clients) + +F11 does NOT directly send events to clients. External events are routed through F14 Result Manager which manages SSE streaming via F15: + +- When F11 emits `UserInputNeeded` → F02 receives → F02 calls F14.publish_user_input_request() → F14 sends via F15 SSE +- When F11 emits `ChunkMerged` → F02 receives → F02 calls F14.update_results_after_chunk_merge() → F14 sends via F15 SSE +- When F11 emits `RecoveryFailed` → F02 receives → F02 calls F14.publish_processing_blocked() → F14 sends via F15 SSE + +This separation ensures: +1. F11 is decoupled from SSE implementation +2. F14 controls all client-facing communication +3. Consistent event format for clients ### Scope - Confidence monitoring @@ -516,13 +535,13 @@ bool: True if merge successful 1. Get chunk frames via F12.get_chunk_frames(chunk_id) → merged_frames 2. Get chunk anchor frame (middle frame or best frame) 3. Call F12.mark_chunk_anchored() with GPS (F12 coordinates with F10) -4. **Resolve target chunk**: - - Query F12.get_merge_target(chunk_id) → returns target_chunk_id - - Target selection logic (inside F12): - - If chunk has temporal predecessor (previous chunk by frame_id order): merge to predecessor - - If no predecessor: merge to main trajectory (chunk_id="main") - - F12 maintains chunk ordering based on first frame_id in each chunk -5. Call F12.merge_chunks(chunk_id, target_chunk_id, transform) (F12 coordinates with F10) +4. **Determine merge target**: + - Target is typically the temporal predecessor (previous chunk by frame_id order) + - If no predecessor: merge to main trajectory (target_chunk_id="main") + - F11 determines target based on chunk frame_id ordering +5. Call F12.merge_chunks(target_chunk_id, chunk_id, transform) + - Note: `merge_chunks(target, source)` merges source INTO target + - chunk_id (source) is merged into target_chunk_id 6. F12 handles chunk state updates (deactivation, status updates) 7. F10 optimizes merged graph globally (via F12.merge_chunks()) 8. **Emit ChunkMerged event** with flight_id and merged_frames @@ -580,10 +599,25 @@ while flight_active: - Reduces user input requests - **Lifecycle**: Starts when flight becomes active, stops when flight completed +**Chunk Failure Flow**: +When chunk matching fails after trying all candidates: +1. Emit `ChunkMatchingFailed` event with chunk_id and flight_id +2. F02 receives event and decides next action: + - **Option A**: Wait for more frames and retry later (chunk may gain more distinctive features) + - **Option B**: Create user input request for the chunk +3. If user input requested: + - F02 calls F14.publish_user_input_request() with chunk context + - Client receives via SSE + - User provides GPS anchor for one frame in chunk + - F02 receives user fix via handle_user_fix() + - F11.apply_user_anchor() anchors the chunk + - Processing resumes + **Test Cases**: 1. **Background matching**: Unanchored chunks matched asynchronously 2. **Chunk merging**: Chunks merged when matches found 3. **Non-blocking**: Frame processing continues during matching +4. **Chunk matching fails**: ChunkMatchingFailed emitted, user input requested if needed ## Integration Tests diff --git a/docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md b/docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md index ff3d0e7..8a40701 100644 --- a/docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md +++ b/docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md @@ -53,12 +53,8 @@ class IRouteChunkManager(ABC): pass @abstractmethod - def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool: - pass - - @abstractmethod - def get_merge_target(self, chunk_id: str) -> str: - """Returns target chunk_id for merging. Returns 'main' for main trajectory.""" + def merge_chunks(self, target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool: + """Merges source_chunk INTO target_chunk. Result is stored in target_chunk.""" pass @abstractmethod @@ -468,17 +464,17 @@ bool: True if deactivated successfully --- -### `merge_chunks(chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool` +### `merge_chunks(target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool` -**Description**: Coordinates chunk merging by validating chunks, calling F10 for factor graph merge, and updating chunk states. +**Description**: Merges source_chunk INTO target_chunk. The resulting merged chunk is target_chunk. Source chunk is deactivated after merge. **Called By**: - F11 Failure Recovery Coordinator (after successful chunk matching) **Input**: ```python -chunk_id_1: str # Source chunk (typically newer, to be merged) -chunk_id_2: str # Target chunk (typically older, merged into) +target_chunk_id: str # Target chunk (receives the merge, typically older/main) +source_chunk_id: str # Source chunk (being merged in, typically newer) transform: Sim3Transform: translation: np.ndarray # (3,) rotation: np.ndarray # (3, 3) or quaternion @@ -492,34 +488,33 @@ bool: True if merge successful **Processing Flow**: 1. Verify both chunks exist -2. Verify chunk_id_1 is anchored (has_anchor=True) +2. Verify source_chunk_id is anchored (has_anchor=True) 3. Validate chunks can be merged (not already merged, not same chunk) -4. **Merge direction**: chunk_id_1 (newer, source) merges INTO chunk_id_2 (older, target) -5. Call F10.merge_chunks(chunk_id_1, chunk_id_2, transform) -6. Update chunk_id_1 state: +4. Call F10.merge_chunk_subgraphs(flight_id, source_chunk_id, target_chunk_id, transform) +5. Update source_chunk_id state: - Set is_active=False - Set matching_status="merged" - - Call deactivate_chunk(chunk_id_1) -7. Update chunk_id_2 state (if needed) -8. Persist chunk state via F03 Flight Database.save_chunk_state() -9. Return True + - Call deactivate_chunk(source_chunk_id) +6. target_chunk remains active (now contains merged frames) +7. Persist chunk state via F03 Flight Database.save_chunk_state() +8. Return True + +**Merge Convention**: +- `merge_chunks(target, source)` → source is merged INTO target +- Result is stored in target_chunk +- source_chunk is deactivated after merge +- Example: `merge_chunks("main", "chunk_3")` merges chunk_3 into main trajectory **Validation**: - Both chunks must exist -- chunk_id_1 must be anchored -- chunk_id_1 must not already be merged -- chunk_id_1 and chunk_id_2 must be different - -**Merge Direction**: -- **chunk_id_1**: Source chunk (newer, recently anchored) -- **chunk_id_2**: Target chunk (older, main trajectory or previous chunk) -- Newer chunks merge INTO older chunks to maintain chronological consistency +- source_chunk must be anchored +- source_chunk must not already be merged +- target_chunk and source_chunk must be different **Test Cases**: -1. **Merge anchored chunks**: Chunks merged successfully, chunk_id_1 deactivated -2. **Merge unanchored chunk**: Returns False (validation fails) -3. **Merge already merged chunk**: Returns False (validation fails) -4. **State updates**: chunk_id_1 marked as merged and deactivated +1. **Merge anchored chunk**: source_chunk merged into target_chunk +2. **Source deactivated**: source_chunk marked as merged and deactivated +3. **Target unchanged**: target_chunk remains active with new frames --- diff --git a/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md b/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md index a98b486..14125a7 100644 --- a/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md +++ b/docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md @@ -35,7 +35,7 @@ class ICoordinateTransformer(ABC): pass @abstractmethod - def image_object_to_gps(self, object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint: + def image_object_to_gps(self, flight_id: str, frame_id: int, object_pixel: Tuple[float, float]) -> GPSPoint: pass @abstractmethod @@ -273,18 +273,19 @@ Tuple[float, float]: (x, y) pixel coordinates --- -### `image_object_to_gps(object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint` +### `image_object_to_gps(flight_id: str, frame_id: int, object_pixel: Tuple[float, float]) -> GPSPoint` **Description**: **Critical method** - Converts object pixel coordinates to GPS. Used for external object detection integration. **Called By**: -- F01 Flight API (via `POST /flights/{flightId}/frames/{frameId}/object-to-gps` endpoint) +- F02 Flight Processor (via convert_object_to_gps delegation from F01) - F14 Result Manager (converts objects to GPS for output) **Input**: ```python -object_pixel: Tuple[float, float] # Pixel coordinates from object detector +flight_id: str # Flight identifier (needed for ENU origin and factor graph) frame_id: int # Frame containing object +object_pixel: Tuple[float, float] # Pixel coordinates from object detector ``` **Output**: @@ -293,15 +294,17 @@ GPSPoint: GPS coordinates of object center ``` **Processing Flow**: -1. Get frame_pose from F10 Factor Graph -2. Get camera_params from F17 Configuration Manager -3. Get altitude from configuration +1. Get frame_pose from F10 Factor Graph Optimizer.get_trajectory(flight_id)[frame_id] +2. Get camera_params from F17 Configuration Manager.get_flight_config(flight_id) +3. Get altitude from F17 Configuration Manager.get_flight_config(flight_id).altitude 4. Call pixel_to_gps(object_pixel, frame_pose, camera_params, altitude) -5. Return GPS +5. Use enu_to_gps(flight_id, enu_point) for final GPS conversion +6. Return GPS **User Story**: - External system detects object in UAV image at pixel (1024, 768) -- Calls image_object_to_gps(frame_id=237, object_pixel=(1024, 768)) +- Calls F02.convert_object_to_gps(flight_id="abc", frame_id=237, pixel=(1024, 768)) +- F02 delegates to F13.image_object_to_gps(flight_id="abc", frame_id=237, object_pixel=(1024, 768)) - Returns GPSPoint(lat=48.123, lon=37.456) - Object GPS can be used for navigation, targeting, etc. diff --git a/docs/02_components/14_result_manager/result_manager_spec.md b/docs/02_components/14_result_manager/result_manager_spec.md index 1606033..f505e05 100644 --- a/docs/02_components/14_result_manager/result_manager_spec.md +++ b/docs/02_components/14_result_manager/result_manager_spec.md @@ -155,8 +155,9 @@ frame_ids: List[int] # Frames with updated poses **Processing Flow**: 1. For each frame_id: - - Get refined pose from F10 Factor Graph Optimizer → ENU pose - - Convert ENU pose to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_pose) + - Get refined pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position + - Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])` + - Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint - Update result with refined=True via F03 Flight Database.save_frame_result() - Update waypoint via F03 Flight Database.update_waypoint() - Call F15 SSE Event Streamer.send_refinement() @@ -205,8 +206,9 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge **Processing Flow**: 1. For each frame_id in merged_frames: - - Get updated pose from F10 Factor Graph Optimizer.get_trajectory() → ENU pose - - Convert ENU pose to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_pose) + - Get updated pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position + - Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])` + - Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint - Update frame result via F03 Flight Database.save_frame_result() - Update waypoint via F03 Flight Database.update_waypoint() - Send refinement event via F15 SSE Event Streamer.send_refinement() diff --git a/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md b/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md index e0bdfd1..7dbf843 100644 --- a/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md +++ b/docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md @@ -28,9 +28,19 @@ class ISSEEventStreamer(ABC): def send_refinement(self, flight_id: str, frame_id: int, updated_result: FrameResult) -> bool: pass + @abstractmethod + def send_heartbeat(self, flight_id: str) -> bool: + """Sends heartbeat/keepalive to all clients subscribed to flight.""" + pass + @abstractmethod def close_stream(self, flight_id: str, client_id: str) -> bool: pass + + @abstractmethod + def get_active_connections(self, flight_id: str) -> int: + """Returns count of active SSE connections for a flight.""" + pass ``` ## Component Description @@ -170,12 +180,50 @@ StreamConnection: --- +### `send_heartbeat(flight_id: str) -> bool` + +**Description**: Sends heartbeat/keepalive ping to all clients subscribed to a flight. Keeps connections alive and helps detect stale connections. + +**Called By**: +- Background heartbeat task (every 30 seconds) +- F02 Flight Processor (periodically during processing) + +**Event Format**: +``` +:heartbeat +``` + +**Behavior**: +- Sends SSE comment (`:heartbeat`) which doesn't trigger event handlers +- Keeps TCP connection alive +- Client can use to detect connection health + +**Test Cases**: +1. Send heartbeat → all clients receive ping +2. Client timeout → connection marked stale + +--- + ### `close_stream(flight_id: str, client_id: str) -> bool` **Description**: Closes SSE connection. **Called By**: F01 REST API (on client disconnect) +--- + +### `get_active_connections(flight_id: str) -> int` + +**Description**: Returns count of active SSE connections for a flight. + +**Called By**: +- F02 Flight Processor (monitoring) +- Admin tools + +**Test Cases**: +1. No connections → returns 0 +2. 5 clients connected → returns 5 + ## Integration Tests ### Test 1: Real-Time Streaming diff --git a/docs/02_components/17_configuration_manager/configuration_manager_spec.md b/docs/02_components/17_configuration_manager/configuration_manager_spec.md index 006b486..028f76f 100644 --- a/docs/02_components/17_configuration_manager/configuration_manager_spec.md +++ b/docs/02_components/17_configuration_manager/configuration_manager_spec.md @@ -27,6 +27,21 @@ class IConfigurationManager(ABC): @abstractmethod def update_config(self, section: str, key: str, value: Any) -> bool: pass + + @abstractmethod + def get_operational_altitude(self, flight_id: str) -> float: + """Returns predefined operational altitude for the flight (NOT from EXIF).""" + pass + + @abstractmethod + def get_frame_spacing(self, flight_id: str) -> float: + """Returns expected distance between consecutive frames in meters.""" + pass + + @abstractmethod + def save_flight_config(self, flight_id: str, config: FlightConfig) -> bool: + """Persists flight-specific configuration.""" + pass ``` ## Component Description @@ -136,6 +151,62 @@ FlightConfig: 1. Update value → succeeds 2. Invalid key → fails +--- + +### `get_operational_altitude(flight_id: str) -> float` + +**Description**: Returns predefined operational altitude for the flight in meters. This is the altitude provided during flight creation, NOT extracted from EXIF metadata (images don't have GPS/altitude metadata per problem constraints). + +**Called By**: +- F10 Factor Graph Optimizer (for scale resolution) +- H02 GSD Calculator (for GSD computation) +- F09 Metric Refinement (for alignment) + +**Input**: `flight_id: str` + +**Output**: `float` - Altitude in meters (typically 100-500m) + +**Test Cases**: +1. Get existing flight altitude → returns value +2. Non-existent flight → raises error + +--- + +### `get_frame_spacing(flight_id: str) -> float` + +**Description**: Returns expected distance between consecutive frames in meters. Used for scale estimation in visual odometry. + +**Called By**: +- F10 Factor Graph Optimizer (for expected displacement calculation) + +**Input**: `flight_id: str` + +**Output**: `float` - Expected frame spacing in meters (typically ~100m) + +**Test Cases**: +1. Get frame spacing → returns expected distance + +--- + +### `save_flight_config(flight_id: str, config: FlightConfig) -> bool` + +**Description**: Persists flight-specific configuration when a flight is created. + +**Called By**: +- F02 Flight Processor (during flight creation) + +**Input**: +```python +flight_id: str +config: FlightConfig +``` + +**Output**: `bool` - True if saved successfully + +**Test Cases**: +1. Save valid config → succeeds +2. Invalid flight_id → fails + ## Data Models ### SystemConfig diff --git a/docs/02_components/decomposition_plan.md b/docs/02_components/decomposition_plan.md index 4bf409a..d13c97b 100644 --- a/docs/02_components/decomposition_plan.md +++ b/docs/02_components/decomposition_plan.md @@ -189,20 +189,27 @@ | Client | F01 | `POST /flights` | Create flight | | F01 | F02 | `create_flight()` | Initialize flight state | | F02 | F16 | `get_flight_config()` | Get camera params, altitude | -| F02 | F12 | `set_enu_origin(start_gps)` | Set ENU coordinate origin | +| F02 | F13 | `set_enu_origin(flight_id, start_gps)` | Set ENU coordinate origin | | F02 | F04 | `prefetch_route_corridor()` | Prefetch tiles | | F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download | | F04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations | | F02 | F03 | `insert_flight()` | Persist flight data | + +### SSE Stream Creation + +| Source | Target | Method | Purpose | +|--------|--------|--------|---------| | Client | F01 | `GET .../stream` | Open SSE connection | -| F01 | F14 | `create_stream()` | Establish SSE channel | +| F01 | F02 | `create_client_stream()` | Route through coordinator | +| F02 | F15 | `create_stream()` | Establish SSE channel | ### Image Upload | Source | Target | Method | Purpose | |--------|--------|--------|---------| | Client | F01 | `POST .../images/batch` | Upload 10-50 images | -| F01 | F05 | `queue_batch()` | Queue for processing | +| F01 | F02 | `queue_images()` | Route through coordinator | +| F02 | F05 | `queue_batch()` | Queue for processing | | F05 | H08 | `validate_batch()` | Validate sequence, format | | F05 | F03 | `save_image_metadata()` | Persist image metadata | @@ -433,3 +440,41 @@ F02 Flight Processor handles multiple concerns: **Event Recovery**: - Events are fire-and-forget (no persistence) - Subscribers rebuild state from F03 on restart + +### Error Propagation Strategy + +**Principle**: Errors propagate upward through the component hierarchy. Lower-level components throw exceptions or return error results; higher-level coordinators handle recovery. + +**Error Propagation Chain**: +``` +H01-H08 (Helpers) → F07/F08/F09 (Visual Processing) → F11 (Recovery) → F02 (Coordinator) → F01 (API) → Client + ↓ + Events → F14 → F15 → SSE → Client +``` + +**Error Categories**: +1. **Recoverable** (F11 handles): + - Tracking loss → Progressive search + - Low confidence → Rotation sweep + - Chunk matching fails → User input request + +2. **Propagated to Client** (via SSE): + - User input needed → `user_input_needed` event + - Processing blocked → `processing_blocked` event + - Flight completed → `flight_completed` event + +3. **Fatal** (F02 handles, returns to F01): + - Database connection lost → HTTP 503 + - Model loading failed → HTTP 500 + - Invalid configuration → HTTP 400 + +**User Fix Flow**: +``` +Client ---> F01 (POST /user-fix) ---> F02.handle_user_fix() ---> F11.apply_user_anchor() + ↓ + F10.add_chunk_anchor() + ↓ + F02 receives UserFixApplied event + ↓ + F14.publish_result() ---> F15 ---> SSE ---> Client +``` diff --git a/docs/02_components/helpers/h04_faiss_index_manager_spec.md b/docs/02_components/helpers/h04_faiss_index_manager_spec.md index ad956d0..ae9e05a 100644 --- a/docs/02_components/helpers/h04_faiss_index_manager_spec.md +++ b/docs/02_components/helpers/h04_faiss_index_manager_spec.md @@ -27,11 +27,25 @@ class IFaissIndexManager(ABC): @abstractmethod def load_index(self, path: str) -> FaissIndex: pass + + @abstractmethod + def is_gpu_available(self) -> bool: + pass + + @abstractmethod + def set_device(self, device: str) -> bool: + """Set device: 'gpu' or 'cpu'.""" + pass ``` ## Component Description -Manages Faiss indices for AnyLoc retrieval (IVF, HNSW options). +Manages Faiss indices for DINOv2 descriptor similarity search. H04 builds indexes from UAV image descriptors for: +1. **Loop closure detection**: Find when UAV revisits previously seen areas within the same flight +2. **Chunk-to-chunk matching**: Match disconnected chunks to each other +3. **Flight-to-flight matching**: Match current flight to previous flights in same area + +**Index Source**: Descriptors are computed from UAV images using F08's DINOv2 encoder, NOT from satellite images. The index enables finding similar UAV viewpoints. ## API Methods @@ -76,9 +90,18 @@ Manages Faiss indices for AnyLoc retrieval (IVF, HNSW options). **External**: faiss-gpu or faiss-cpu +## GPU/CPU Fallback + +H04 supports automatic fallback from GPU to CPU: +- `is_gpu_available()`: Returns True if faiss-gpu is available and CUDA works +- `set_device("gpu")`: Use GPU acceleration (faster for large indexes) +- `set_device("cpu")`: Use CPU (fallback when GPU unavailable) + ## Test Cases -1. Build index with 10,000 descriptors → succeeds -2. Search query → returns top-k matches +1. Build index with 10,000 UAV image descriptors → succeeds +2. Search query UAV descriptor → returns top-k similar UAV frames 3. Save/load index → index restored correctly +4. GPU unavailable → automatically falls back to CPU +5. Add descriptors incrementally → index grows correctly diff --git a/docs/02_components/system_flows.md b/docs/02_components/system_flows.md index 07889d9..55ca711 100644 --- a/docs/02_components/system_flows.md +++ b/docs/02_components/system_flows.md @@ -185,7 +185,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ +┌──────────────────────────────────────────────────────────────────────────┐ │ F02 Flight Processor │ │ │ │ ┌─────────────┐ │ @@ -265,15 +265,15 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ -│ F06 Image Rotation Manager │ -│ │ -│ ┌─────────────────────────────────────────────────────────────┐ │ -│ │ try_rotation_steps(image, satellite_tile, tile_bounds) │ │ -│ └──────────────────────────────┬──────────────────────────────┘ │ -│ │ │ -│ ┌───────────────────────┴───────────────────────────┐ │ -│ │ For angle in [0°, 30°, 60°, ... 330°]: │ │ +┌───────────────────────────────────────────────────────────────────────────┐ +│ F06 Image Rotation Manager │ +│ │ +│ ┌─────────────────────────────────────────────────────────────┐ │ +│ │ try_rotation_steps(image, satellite_tile, tile_bounds) │ │ +│ └──────────────────────────────┬──────────────────────────────┘ │ +│ │ │ +│ ┌───────────────────────┴────────────────────────────┐ │ +│ │ For angle in [0°, 30°, 60°, ... 330°]: │ │ │ │ │ │ │ │ ┌─────────────────────────────────────┐ │ │ │ │ │ H07 rotate_image(image, angle) │ │ │ @@ -300,11 +300,11 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ │ │ │ │ └────────────────────────────────────────────────────┘ │ │ │ │ -│ ┌────────────┴────────────┐ │ -│ │ Return RotationResult │ │ -│ │ or None │ │ -│ └─────────────────────────┘ │ -└──────────────────────────────────────────────────────────────────────────┘ +│ ┌────────────┴────────────┐ │ +│ │ Return RotationResult │ │ +│ │ or None │ │ +│ └─────────────────────────┘ │ +└───────────────────────────────────────────────────────────────────────────┘ ``` **Output**: RotationResult with precise heading angle @@ -319,7 +319,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ +┌──────────────────────────────────────────────────────────────────────────┐ │ F11 Failure Recovery Coordinator │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ @@ -337,34 +337,34 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ │ ├─ F06 requires_rotation_sweep() → trigger sweep │ │ │ │ ├─ F08 retrieve_candidate_tiles() (DINOv2) │ │ │ │ └─ Progressive tile search (1→4→9→16→25): │ │ -│ │ │ │ -│ │ ┌───────────────────────────────────────────────────┐ │ │ -│ │ │ For grid_size in [1, 4, 9, 16, 25]: │ │ │ -│ │ │ ├─ F04 expand_search_grid() │ │ │ -│ │ │ ├─ For each tile: │ │ │ -│ │ │ │ ├─ F04 compute_tile_bounds() │ │ │ -│ │ │ │ └─ F09 align_to_satellite(img, tile, bounds)│ │ │ -│ │ │ │ │ │ │ -│ │ │ └─ If match found: BREAK │ │ │ -│ │ └───────────────────────────────────────────────────┘ │ │ -│ │ │ │ +│ │ │ │ +│ │ ┌───────────────────────────────────────────────────┐ │ │ +│ │ │ For grid_size in [1, 4, 9, 16, 25]: │ │ │ +│ │ │ ├─ F04 expand_search_grid() │ │ │ +│ │ │ ├─ For each tile: │ │ │ +│ │ │ │ ├─ F04 compute_tile_bounds() │ │ │ +│ │ │ │ └─ F09 align_to_satellite(img, tile, bounds)│ │ │ +│ │ │ │ │ │ │ +│ │ │ └─ If match found: BREAK │ │ │ +│ │ └───────────────────────────────────────────────────┘ │ │ +│ │ │ │ │ └──────────────────────────────┬──────────────────────────────┘ │ │ │ │ │ ┌────────────────────┴────────────────────┐ │ │ │ Single-image match found? │ │ │ ▼ ▼ │ -│ ┌─────────────────────┐ ┌─────────────────────────────┐ │ -│ │ EMIT RecoverySucceeded│ │ Continue chunk building │ │ -│ │ Resume normal flow │ │ → Flow 7 (Chunk Building) │ │ -│ └─────────────────────┘ │ → Flow 8 (Chunk Matching) │ │ -│ │ (Background) │ │ -│ └─────────────────────────────┘ │ +│ ┌─────────────────────┐ ┌────────────────────────────┐ │ +│ │ EMIT RecoverySucceeded│ │ Continue chunk building │ │ +│ │ Resume normal flow │ │ → Flow 7 (Chunk Building) │ │ +│ └─────────────────────┘ │ → Flow 8 (Chunk Matching) │ │ +│ │ (Background) │ │ +│ └────────────────────────────┘ │ │ │ │ │ ▼ │ -│ ┌─────────────────────────────────────┐ │ -│ │ If all strategies exhausted: │ │ -│ │ → Flow 10 (User Input Recovery) │ │ -│ └─────────────────────────────────────┘ │ +│ ┌─────────────────────────────────────┐ │ +│ │ If all strategies exhausted: │ │ +│ │ → Flow 10 (User Input Recovery) │ │ +│ └─────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ ``` @@ -376,42 +376,42 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ -│ F12 Route Chunk Manager │ -│ │ -│ ┌─────────────────────────────────────────────────────────────┐ │ -│ │ create_chunk(flight_id, start_frame_id) │ │ -│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │ -│ │ ├─ Initialize chunk state (unanchored, active) │ │ -│ │ └─ F03 save_chunk_state() │ │ -│ └──────────────────────────────┬──────────────────────────────┘ │ -│ │ │ -│ ┌───────────────────────┴───────────────────────────┐ │ -│ │ For each frame in chunk: │ │ +┌───────────────────────────────────────────────────────────────────────────┐ +│ F12 Route Chunk Manager │ +│ │ +│ ┌─────────────────────────────────────────────────────────────┐ │ +│ │ create_chunk(flight_id, start_frame_id) │ │ +│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │ +│ │ ├─ Initialize chunk state (unanchored, active) │ │ +│ │ └─ F03 save_chunk_state() │ │ +│ └──────────────────────────────┬──────────────────────────────┘ │ +│ │ │ +│ ┌───────────────────────┴────────────────────────────┐ │ +│ │ For each frame in chunk: │ │ │ │ │ │ │ │ ┌─────────────────────────────────────┐ │ │ │ │ │ F07 compute_relative_pose_in_chunk()│ │ │ │ │ └───────────────────┬─────────────────┘ │ │ │ │ ▼ │ │ -│ │ ┌─────────────────────────────────────┐ │ │ -│ │ │ F12 add_frame_to_chunk() │ │ │ +│ │ ┌──────────────────────────────────────┐ │ │ +│ │ │ F12 add_frame_to_chunk() │ │ │ │ │ │ └─ F10 add_relative_factor_to_chunk│ │ │ -│ │ └───────────────────┬─────────────────┘ │ │ +│ │ └───────────────────┬──────────────────┘ │ │ │ │ ▼ │ │ │ │ ┌─────────────────────────────────────┐ │ │ │ │ │ F10 optimize_chunk() (local) │ │ │ │ │ └─────────────────────────────────────┘ │ │ │ │ │ │ │ └────────────────────────────────────────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌─────────────────────────────────────────────────────────────┐ │ -│ │ is_chunk_ready_for_matching()? │ │ -│ │ ├─ Min 5 frames │ │ -│ │ ├─ Max 20 frames │ │ -│ │ └─ Internal consistency (good VO inlier counts) │ │ -│ └─────────────────────────────────────────────────────────────┘ │ -└──────────────────────────────────────────────────────────────────────────┘ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────┐ │ +│ │ is_chunk_ready_for_matching()? │ │ +│ │ ├─ Min 5 frames │ │ +│ │ ├─ Max 20 frames │ │ +│ │ └─ Internal consistency (good VO inlier counts) │ │ +│ └─────────────────────────────────────────────────────────────┘ │ +└───────────────────────────────────────────────────────────────────────────┘ ``` --- @@ -424,7 +424,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ +┌──────────────────────────────────────────────────────────────────────────┐ │ F11 process_unanchored_chunks() (Background Task) │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ @@ -472,7 +472,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ +┌───────────────────────────────────────────────────────────────────────-──┐ │ F11 merge_chunk_to_trajectory() │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ @@ -485,7 +485,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ └──────────────────────────────┬──────────────────────────────┘ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────┐ │ -│ │ 3. Resolve target: F12 get_merge_target(chunk_id) │ │ +│ │ 3. Determine target chunk (predecessor or "main") │ │ │ │ └─ Returns target_chunk_id (predecessor or "main") │ │ │ └──────────────────────────────┬──────────────────────────────┘ │ │ ▼ │ @@ -501,7 +501,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ └──────────────────────────────┬──────────────────────────────┘ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────┐ │ -│ │ 6. EMIT ChunkMerged event (flight_id, chunk_id, merged_frames)│ │ +│ │ 6. EMIT ChunkMerged event (flight_id,chunk_id,merged_frames)│ │ │ │ └─ F14 subscribes → update_results_after_chunk_merge() │ │ │ │ ├─ F10 get_trajectory() → ENU poses │ │ │ │ ├─ F13 enu_to_gps() for each frame │ │ @@ -523,7 +523,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ +┌──────────────────────────────────────────────────────────────────────────┐ │ F11 create_user_input_request() │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ @@ -537,7 +537,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match └──────────────────────────────────────────────────────────────────────────┘ │ ▼ Client receives SSE event -┌──────────────────────────────────────────────────────────────────────────┐ +┌───────────────────────────────────────────────────────────────────────────┐ │ Client UI │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ @@ -550,7 +550,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ │ POST /flights/{flightId}/user-fix │ │ │ │ body: { frame_id, uav_pixel, satellite_gps } │ │ │ └─────────────────────────────────────────────────────────────┘ │ -└──────────────────────────────────────────────────────────────────────────┘ +└───────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────┐ @@ -577,7 +577,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ +┌─────────────────────────────────────────────────────────────-────────────┐ │ F10 Background Optimization │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ @@ -650,7 +650,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match **Sequence**: ``` -┌─────────────────────────────────────────────────────────────────────────┐ +┌──────────────────────────────────────────────────────────────────────────┐ │ F02 Flight Processor │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ @@ -741,7 +741,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match ``` ┌─────────────────────────────────────────────────────────────────────────────┐ -│ EXTERNAL │ +│ EXTERNAL │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Client │ │ Satellite │ │ External │ │ │ │ (UI) │ │ Provider │ │ Detector │ │ @@ -750,7 +750,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ REST/SSE │ HTTP │ REST ▼ ▼ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ -│ API LAYER │ +│ API LAYER │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ F01 Flight API │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ @@ -758,23 +758,23 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ -│ ORCHESTRATION LAYER │ +│ ORCHESTRATION LAYER │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ F02 Flight Processor │ │ -│ │ (Central coordinator, event subscriber, background task manager) │ │ +│ │ F02 Flight Processor │ │ +│ │ (Central coordinator, event subscriber, background task manager) │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────┐ ▼ ▼ ┌─────────────────────────────────────────────┐ ┌────────────────────────────┐ -│ DATA MANAGEMENT LAYER │ │ RECOVERY LAYER │ +│ DATA MANAGEMENT LAYER │ │ RECOVERY LAYER │ │ ┌─────────────┐ ┌─────────────┐ │ │ ┌───────────────────────┐ │ │ │ F04 │ │ F05 │ │ │ │ F11 │ │ │ │ Satellite │ │ Image │ │ │ │ Failure Recovery │ │ │ │ Data │ │ Input │ │ │ │ (Event emitter) │ │ │ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │ -│ │ │ │ │ +│ │ │ │ │ │ ┌─────────────┐ ┌─────────────┐ │ │ ▼ │ │ │ F12 │ │ F03 │ │ │ ┌───────────────────────┐ │ │ │Route Chunk │ │ Flight │ │ │ │ F12 │ │ @@ -784,7 +784,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ -│ VISUAL PROCESSING LAYER │ +│ VISUAL PROCESSING LAYER │ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ┌──────────────┐ │ │ │ F06 │ │ F07 │ │ F08 │ │ F09 │ │ │ │ Rotation │ │ Sequential VO │ │ Global │ │ Metric │ │ @@ -796,16 +796,16 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ -│ STATE ESTIMATION LAYER │ -│ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ F10 Factor Graph Optimizer │ │ -│ │ (GTSAM, iSAM2, Robust kernels, Chunk subgraphs, Sim(3) merging) │ │ +│ STATE ESTIMATION LAYER │ +│ ┌──────────────────────────────────────────────────────────────────k──┐ │ +│ │ F10 Factor Graph Optimizer │ │ +│ │ (GTSAM, iSAM2, Robust kernels, Chunk subgraphs, Sim(3) merging) │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ -│ OUTPUT LAYER │ +│ OUTPUT LAYER │ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ │ │ F13 │ │ F14 │ │ F15 │ │ │ │ Coordinate │ │ Result │ │ SSE │ │ @@ -815,14 +815,14 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ +┌─────────────────────────────────────────────────────────────────────────────-┐ │ INFRASTRUCTURE LAYER │ -│ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────────────────┐│ -│ │ F16 │ │ F17 │ │ HELPERS ││ -│ │ Model │ │Configuration │ │ H01-H08 (Camera, GSD, Kernels, ││ -│ │ Manager │ │ Manager │ │ Faiss, Monitor, Mercator, etc) ││ -│ └───────────────┘ └───────────────┘ └───────────────────────────────────┘│ -└─────────────────────────────────────────────────────────────────────────────┘ +│ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────────────────-┐│ +│ │ F16 │ │ F17 │ │ HELPERS ││ +│ │ Model │ │Configuration │ │ H01-H08 (Camera, GSD, Kernels, ││ +│ │ Manager │ │ Manager │ │ Faiss, Monitor, Mercator, etc) ││ +│ └───────────────┘ └───────────────┘ └────────────────────────────────────┘│ +└──────────────────────────────────────────────────────────────────────────────┘ ``` ---