azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-04-22 22:06:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix issues

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-30 01:43:23 +02:00
parent 1082316660
commit 310cf78ee7
17 changed files with 584 additions and 240 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/02_components/01_flight_api/flight_api_spec.md
+19 -12
View File
@@ -311,8 +311,9 @@ BatchResponse:
1. Validate flight_id exists
2. Validate batch size (10-50 images)
3. Validate sequence numbers (strict sequential)
4. Pass to F05 Image Input Pipeline
5. Return immediately (processing is async)
4. Call F02 Flight Processor → queue_images(flight_id, batch)
5. F02 delegates to F05 Image Input Pipeline
6. Return immediately (processing is async)
**Error Conditions**:
- `400 Bad Request`: Invalid batch size, out-of-sequence images
@@ -355,9 +356,10 @@ UserFixResponse:
**Processing Flow**:
1. Validate flight_id exists and is blocked
2. Pass to F11 Failure Recovery Coordinator
3. Coordinator applies anchor to Factor Graph
4. Resume processing pipeline
2. Call F02 Flight Processor → handle_user_fix(flight_id, fix_data)
3. F02 delegates to F11 Failure Recovery Coordinator
4. Coordinator applies anchor to Factor Graph
5. Resume processing pipeline
**Error Conditions**:
- `400 Bad Request`: Invalid fix data
@@ -400,8 +402,9 @@ ObjectGPSResponse:
**Processing Flow**:
1. Validate flight_id and frame_id exist
2. Validate frame has been processed (has pose in Factor Graph)
3. Call F13.image_object_to_gps(pixel, frame_id)
4. Return GPS with accuracy estimate
3. Call F02 Flight Processor → convert_object_to_gps(flight_id, frame_id, pixel)
4. F02 delegates to F13.image_object_to_gps(flight_id, frame_id, pixel)
5. Return GPS with accuracy estimate
**Error Conditions**:
- `400 Bad Request`: Invalid pixel coordinates
@@ -479,6 +482,12 @@ SSE Stream with events:
- flight_completed
```
**Processing Flow**:
1. Validate flight_id exists
2. Call F02 Flight Processor → create_client_stream(flight_id, client_id)
3. F02 delegates to F15 SSE Event Streamer → create_stream()
4. Return SSE stream to client
**Event Format**:
```json
{
@@ -575,11 +584,9 @@ SSE Stream with events:
## Dependencies
### Internal Components
- **F02 Flight Processor**: For all flight operations and processing orchestration
- **F05 Image Input Pipeline**: For batch processing
- **F11 Failure Recovery Coordinator**: For user fixes
- **F15 SSE Event Streamer**: For real-time streaming
- **F03 Flight Database**: For persistence (via F02)
- **F02 Flight Processor**: For ALL operations (flight CRUD, image batching, user fixes, SSE streams, object-to-GPS conversion). F01 is a thin REST layer that delegates all business logic to F02.
**Note**: F01 does NOT directly call F05, F11, F13, or F15. All operations are routed through F02 to maintain a single coordinator pattern.
### External Dependencies
- **FastAPI**: Web framework
docs/02_components/02_flight_processor/flight_processor_spec.md
+144
View File
@@ -59,6 +59,27 @@ class IFlightProcessor(ABC):
def validate_flight_continuity(self, waypoints: List[Waypoint]) -> ValidationResult:
pass
# API Delegation Methods (called by F01)
@abstractmethod
def queue_images(self, flight_id: str, batch: ImageBatch) -> BatchQueueResult:
"""Delegates to F05 Image Input Pipeline."""
pass
@abstractmethod
def handle_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResult:
"""Delegates to F11 Failure Recovery Coordinator."""
pass
@abstractmethod
def create_client_stream(self, flight_id: str, client_id: str) -> StreamConnection:
"""Delegates to F15 SSE Event Streamer."""
pass
@abstractmethod
def convert_object_to_gps(self, flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint:
"""Delegates to F13 Coordinate Transformer."""
pass
# Processing Orchestration
@abstractmethod
def process_frame(self, flight_id: str, frame_id: int) -> FrameResult:
@@ -576,6 +597,129 @@ ValidationResult:
---
## API Delegation Methods
These methods are called by F01 Flight API and delegate to specialized components. This maintains F02 as the single coordinator for all operations.
### `queue_images(flight_id: str, batch: ImageBatch) -> BatchQueueResult`
**Description**: Queues image batch for processing. Delegates to F05 Image Input Pipeline.
**Called By**:
- F01 Flight API (upload_image_batch endpoint)
**Input**:
```python
flight_id: str
batch: ImageBatch
```
**Output**:
```python
BatchQueueResult:
accepted: bool
sequences: List[int]
next_expected: int
message: Optional[str]
```
**Processing Flow**:
1. Validate flight exists and is in valid state
2. Delegate to F05 Image Input Pipeline → queue_batch()
3. Return result to F01
---
### `handle_user_fix(flight_id: str, fix_data: UserFixRequest) -> UserFixResult`
**Description**: Handles user-provided GPS fix. Delegates to F11 Failure Recovery Coordinator.
**Called By**:
- F01 Flight API (submit_user_fix endpoint)
**Input**:
```python
flight_id: str
fix_data: UserFixRequest:
frame_id: int
uav_pixel: Tuple[float, float]
satellite_gps: GPSPoint
```
**Output**:
```python
UserFixResult:
accepted: bool
processing_resumed: bool
message: Optional[str]
```
**Processing Flow**:
1. Validate flight exists and is blocked
2. Delegate to F11 Failure Recovery Coordinator → apply_user_anchor()
3. F11 emits UserFixApplied event (F02 subscribes and resumes processing)
4. Return result to F01
---
### `create_client_stream(flight_id: str, client_id: str) -> StreamConnection`
**Description**: Creates SSE stream for client. Delegates to F15 SSE Event Streamer.
**Called By**:
- F01 Flight API (create_sse_stream endpoint)
**Input**:
```python
flight_id: str
client_id: str
```
**Output**:
```python
StreamConnection:
stream_id: str
flight_id: str
client_id: str
last_event_id: Optional[str]
```
**Processing Flow**:
1. Validate flight exists
2. Delegate to F15 SSE Event Streamer → create_stream()
3. Return StreamConnection to F01
---
### `convert_object_to_gps(flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint`
**Description**: Converts object pixel to GPS. Delegates to F13 Coordinate Transformer.
**Called By**:
- F01 Flight API (convert_object_to_gps endpoint)
**Input**:
```python
flight_id: str
frame_id: int
pixel: Tuple[float, float]
```
**Output**:
```python
GPSPoint:
lat: float
lon: float
```
**Processing Flow**:
1. Validate flight and frame exist
2. Validate frame has been processed (has pose)
3. Delegate to F13 Coordinate Transformer → image_object_to_gps(flight_id, frame_id, pixel)
4. Return GPSPoint to F01
---
## Processing Orchestration Methods
### `process_frame(flight_id: str, frame_id: int) -> FrameResult`
docs/02_components/03_flight_database/flight_database_spec.md
+7
View File
@@ -676,6 +676,13 @@ Optional[Dict]: Metadata dictionary or None
## Chunk State Operations
**Necessity**: These methods are **required** for crash recovery. Without them:
- Chunk state would be lost on system restart
- Processing would need to start from scratch
- Background chunk matching progress would be lost
F12 Route Chunk Manager delegates persistence to F03 to maintain separation of concerns (F12 manages chunk logic, F03 handles storage).
### `save_chunk_state(flight_id: str, chunk: ChunkHandle) -> bool`
**Description**: Saves chunk state to database for crash recovery.
docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md
+1 -1
View File
@@ -401,8 +401,8 @@ ProcessingStatus:
## Dependencies
### Internal Components
- **H08 Batch Validator**: For validation logic
- **F03 Flight Database**: For metadata persistence and flight state information
- **H08 Batch Validator**: For batch validation (naming convention, sequence continuity, format, dimensions)
### External Dependencies
- **opencv-python**: Image I/O
docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md
+1
View File
@@ -495,6 +495,7 @@ return None # No match found
## Dependencies
### Internal Components
- **F04 Satellite Data Manager**: For tile fetching (`fetch_tile`) and tile bounds computation (`compute_tile_bounds`)
- **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching.
- **H07 Image Rotation Utils**: For image rotation and angle calculations
- **F12 Route Chunk Manager**: For chunk image retrieval
docs/02_components/07_sequential_visual_odometry/sequential_visual_odometry_spec.md
+3 -49
View File
@@ -23,12 +23,10 @@ class ISequentialVO(ABC):
@abstractmethod
def estimate_motion(self, matches: Matches, camera_params: CameraParameters) -> Optional[Motion]:
pass
@abstractmethod
def compute_relative_pose_in_chunk(self, prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]:
pass
```
**Note**: F07 is chunk-agnostic. It only computes relative poses between images. The caller (F02 Flight Processor) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
## Component Description
### Responsibilities
@@ -38,14 +36,13 @@ class ISequentialVO(ABC):
- Estimate relative pose (translation + rotation) between frames
- Return relative pose factors for Factor Graph Optimizer
- Detect tracking loss (low inlier count)
- **Chunk-aware VO operations (factors added to chunk subgraph)**
### Scope
- Frame-to-frame visual odometry
- Feature-based motion estimation
- Handles low overlap and challenging agricultural environments
- Provides relative measurements for trajectory optimization
- **Chunk-scoped operations (Atlas multi-map architecture)**
- **Chunk-agnostic**: F07 doesn't know about chunks. Caller (F02) routes results to appropriate chunk subgraph.
## API Methods
@@ -224,49 +221,6 @@ Motion:
2. **Low inliers**: May return None
3. **Degenerate motion**: Handles pure rotation
---
### `compute_relative_pose_in_chunk(prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]`
**Description**: Computes relative camera pose between consecutive frames within a chunk context.
**Called By**:
- F02 Flight Processor (chunk-aware processing)
**Input**:
```python
prev_image: np.ndarray # Previous frame (t-1)
curr_image: np.ndarray # Current frame (t)
chunk_id: str # Chunk identifier for context
```
**Output**:
```python
RelativePose:
translation: np.ndarray # (x, y, z) in meters
rotation: np.ndarray # 3×3 rotation matrix or quaternion
confidence: float # 0.0 to 1.0
inlier_count: int
total_matches: int
tracking_good: bool
chunk_id: str # Chunk context
```
**Processing Flow**:
1. Same as compute_relative_pose() (SuperPoint + LightGlue)
2. Return RelativePose with chunk_id context
3. Factor will be added to chunk's subgraph (not global graph)
**Chunk Context**:
- VO operations are chunk-scoped
- Factors added to chunk's subgraph via F10.add_relative_factor_to_chunk()
- Chunk isolation ensures independent optimization
**Test Cases**:
1. **Chunk-aware VO**: Returns RelativePose with chunk_id
2. **Chunk isolation**: Factors isolated to chunk
3. **Multiple chunks**: VO operations don't interfere between chunks
## Integration Tests
### Test 1: Normal Flight Sequence
docs/02_components/09_metric_refinement/metric_refinement_spec.md
+1 -1
View File
@@ -377,11 +377,11 @@ Optional[np.ndarray]: 3×3 homography matrix or None
## Dependencies
### Internal Components
- **F12 Route Chunk Manager**: For chunk image retrieval and chunk operations
- **F16 Model Manager**: For LiteSAM model
- **H01 Camera Model**: For projection operations
- **H02 GSD Calculator**: For coordinate transformations
- **H05 Performance Monitor**: For timing
- **F12 Route Chunk Manager**: For chunk image retrieval
**Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager)
docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md
+40 -30
View File
@@ -8,64 +8,67 @@
```python
class IFactorGraphOptimizer(ABC):
# All methods take flight_id to support concurrent flights
# F10 maintains Dict[str, FactorGraph] keyed by flight_id internally
@abstractmethod
def add_relative_factor(self, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
def add_relative_factor(self, flight_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
pass
@abstractmethod
def add_absolute_factor(self, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool:
def add_absolute_factor(self, flight_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool:
pass
@abstractmethod
def add_altitude_prior(self, frame_id: int, altitude: float, covariance: float) -> bool:
def add_altitude_prior(self, flight_id: str, frame_id: int, altitude: float, covariance: float) -> bool:
pass
@abstractmethod
def optimize(self, iterations: int) -> OptimizationResult:
def optimize(self, flight_id: str, iterations: int) -> OptimizationResult:
pass
@abstractmethod
def get_trajectory(self) -> Dict[int, Pose]:
def get_trajectory(self, flight_id: str) -> Dict[int, Pose]:
pass
@abstractmethod
def get_marginal_covariance(self, frame_id: int) -> np.ndarray:
def get_marginal_covariance(self, flight_id: str, frame_id: int) -> np.ndarray:
pass
# Chunk operations - F10 only manages factor graph subgraphs
# F12 owns chunk metadata (status, is_active, etc.)
@abstractmethod
def create_chunk_subgraph(self, flight_id: str, chunk_id: str, start_frame_id: int) -> bool:
pass
@abstractmethod
def create_new_chunk(self, chunk_id: str, start_frame_id: int) -> ChunkHandle:
def add_relative_factor_to_chunk(self, flight_id: str, chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
pass
@abstractmethod
def get_chunk_for_frame(self, frame_id: int) -> Optional[ChunkHandle]:
def add_chunk_anchor(self, flight_id: str, chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool:
pass
@abstractmethod
def add_relative_factor_to_chunk(self, chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
def merge_chunk_subgraphs(self, flight_id: str, source_chunk_id: str, target_chunk_id: str, transform: Sim3Transform) -> bool:
"""Merges source_chunk INTO target_chunk. Source chunk subgraph is merged into target."""
pass
@abstractmethod
def add_chunk_anchor(self, chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool:
def get_chunk_trajectory(self, flight_id: str, chunk_id: str) -> Dict[int, Pose]:
pass
@abstractmethod
def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool:
def optimize_chunk(self, flight_id: str, chunk_id: str, iterations: int) -> OptimizationResult:
pass
@abstractmethod
def get_chunk_trajectory(self, chunk_id: str) -> Dict[int, Pose]:
def optimize_global(self, flight_id: str, iterations: int) -> OptimizationResult:
pass
@abstractmethod
def get_all_chunks(self) -> List[ChunkHandle]:
pass
@abstractmethod
def optimize_chunk(self, chunk_id: str, iterations: int) -> OptimizationResult:
pass
@abstractmethod
def optimize_global(self, iterations: int) -> OptimizationResult:
def delete_flight_graph(self, flight_id: str) -> bool:
"""Cleanup factor graph when flight is deleted."""
pass
```
@@ -84,16 +87,20 @@ class IFactorGraphOptimizer(ABC):
### Chunk Responsibility Clarification
**F10 provides low-level factor graph operations only**:
- `create_new_chunk()`: Creates subgraph in factor graph
- `create_chunk_subgraph()`: Creates subgraph in factor graph (returns bool, not ChunkHandle)
- `add_relative_factor_to_chunk()`: Adds factors to chunk's subgraph
- `add_chunk_anchor()`: Adds GPS anchor to chunk
- `merge_chunks()`: Applies Sim(3) transform and merges subgraphs
- `add_chunk_anchor()`: Adds GPS anchor to chunk's subgraph
- `merge_chunk_subgraphs()`: Applies Sim(3) transform and merges subgraphs
- `optimize_chunk()`, `optimize_global()`: Runs optimization
**F12 is the source of truth for chunk state** (see F12 spec):
- Chunk lifecycle management (active, anchored, merged status)
**F10 does NOT own chunk metadata** - only factor graph data structures.
**F12 is the source of truth for ALL chunk state** (see F12 spec):
- ChunkHandle with all metadata (is_active, has_anchor, matching_status)
- Chunk lifecycle management
- Chunk readiness determination
- High-level chunk queries
- F12 calls F10 for factor graph operations
**F11 coordinates recovery** (see F11 spec):
- Triggers chunk creation via F12
@@ -155,16 +162,19 @@ F07 returns unit translation vectors due to monocular scale ambiguity. F10 resol
**Explicit Flow**:
```python
# In add_relative_factor():
# altitude comes from F05 Image Input Pipeline (extracted from EXIF metadata)
# altitude comes from F17 Configuration Manager (predefined operational altitude)
# focal_length, sensor_width from F17 Configuration Manager
gsd = H02.compute_gsd(altitude, focal_length, sensor_width, image_width)
config = F17.get_flight_config(flight_id)
altitude = config.altitude # Predefined altitude, NOT from EXIF
gsd = H02.compute_gsd(altitude, config.camera_params.focal_length,
config.camera_params.sensor_width,
config.camera_params.resolution_width)
expected_displacement = frame_spacing * gsd # ~100m typical at 300m altitude
scaled_translation = relative_pose.translation * expected_displacement
# Add scaled_translation to factor graph
```
**Note**: Altitude is passed through the processing chain:
- F05 extracts altitude from EXIF → F02 includes in FrameData → F10 receives with add_relative_factor()
**Note**: Altitude comes from F17 Configuration Manager (predefined operational altitude), NOT from EXIF metadata. The problem statement specifies images don't have GPS metadata.
**Output**:
```python
docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md
+42 -8
View File
@@ -95,7 +95,11 @@ class IFailureRecoveryCoordinator(ABC):
F11 emits events instead of directly calling F02's status update methods. This decouples recovery logic from flight state management.
**Events Emitted**:
### Internal Events (Component-to-Component)
F11 emits events for internal component communication. These are NOT directly sent to clients.
**Events Emitted (internal)**:
- `RecoveryStarted`: When tracking loss detected and recovery begins
- `RecoverySucceeded`: When recovery finds a match (single-image or chunk)
- `RecoveryFailed`: When all recovery strategies exhausted
@@ -104,12 +108,27 @@ F11 emits events instead of directly calling F02's status update methods. This d
- `ChunkCreated`: When new chunk created on tracking loss
- `ChunkAnchored`: When chunk successfully matched and anchored
- `ChunkMerged`: When chunk merged into main trajectory (includes flight_id, chunk_id, merged_frames)
- `ChunkMatchingFailed`: When chunk matching exhausts all candidates and fails
**Event Listeners** (F02 Flight Processor subscribes to these):
- On `RecoveryStarted`: Update status to "recovering"
- On `RecoveryFailed`: Update status to "blocked"
- On `RecoverySucceeded`: Update status to "processing"
- On `UserInputNeeded`: Update status to "blocked", blocked=True
- On `ChunkMatchingFailed`: Re-queue chunk for user input or continue building
### External SSE Events (to Clients)
F11 does NOT directly send events to clients. External events are routed through F14 Result Manager which manages SSE streaming via F15:
- When F11 emits `UserInputNeeded` → F02 receives → F02 calls F14.publish_user_input_request() → F14 sends via F15 SSE
- When F11 emits `ChunkMerged` → F02 receives → F02 calls F14.update_results_after_chunk_merge() → F14 sends via F15 SSE
- When F11 emits `RecoveryFailed` → F02 receives → F02 calls F14.publish_processing_blocked() → F14 sends via F15 SSE
This separation ensures:
1. F11 is decoupled from SSE implementation
2. F14 controls all client-facing communication
3. Consistent event format for clients
### Scope
- Confidence monitoring
@@ -516,13 +535,13 @@ bool: True if merge successful
1. Get chunk frames via F12.get_chunk_frames(chunk_id) → merged_frames
2. Get chunk anchor frame (middle frame or best frame)
3. Call F12.mark_chunk_anchored() with GPS (F12 coordinates with F10)
4. **Resolve target chunk**:
- Query F12.get_merge_target(chunk_id) → returns target_chunk_id
- Target selection logic (inside F12):
- If chunk has temporal predecessor (previous chunk by frame_id order): merge to predecessor
- If no predecessor: merge to main trajectory (chunk_id="main")
- F12 maintains chunk ordering based on first frame_id in each chunk
5. Call F12.merge_chunks(chunk_id, target_chunk_id, transform) (F12 coordinates with F10)
4. **Determine merge target**:
- Target is typically the temporal predecessor (previous chunk by frame_id order)
- If no predecessor: merge to main trajectory (target_chunk_id="main")
- F11 determines target based on chunk frame_id ordering
5. Call F12.merge_chunks(target_chunk_id, chunk_id, transform)
- Note: `merge_chunks(target, source)` merges source INTO target
- chunk_id (source) is merged into target_chunk_id
6. F12 handles chunk state updates (deactivation, status updates)
7. F10 optimizes merged graph globally (via F12.merge_chunks())
8. **Emit ChunkMerged event** with flight_id and merged_frames
@@ -580,10 +599,25 @@ while flight_active:
- Reduces user input requests
- **Lifecycle**: Starts when flight becomes active, stops when flight completed
**Chunk Failure Flow**:
When chunk matching fails after trying all candidates:
1. Emit `ChunkMatchingFailed` event with chunk_id and flight_id
2. F02 receives event and decides next action:
- **Option A**: Wait for more frames and retry later (chunk may gain more distinctive features)
- **Option B**: Create user input request for the chunk
3. If user input requested:
- F02 calls F14.publish_user_input_request() with chunk context
- Client receives via SSE
- User provides GPS anchor for one frame in chunk
- F02 receives user fix via handle_user_fix()
- F11.apply_user_anchor() anchors the chunk
- Processing resumes
**Test Cases**:
1. **Background matching**: Unanchored chunks matched asynchronously
2. **Chunk merging**: Chunks merged when matches found
3. **Non-blocking**: Frame processing continues during matching
4. **Chunk matching fails**: ChunkMatchingFailed emitted, user input requested if needed
## Integration Tests
docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md
+25 -30
View File
@@ -53,12 +53,8 @@ class IRouteChunkManager(ABC):
pass
@abstractmethod
def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool:
pass
@abstractmethod
def get_merge_target(self, chunk_id: str) -> str:
"""Returns target chunk_id for merging. Returns 'main' for main trajectory."""
def merge_chunks(self, target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool:
"""Merges source_chunk INTO target_chunk. Result is stored in target_chunk."""
pass
@abstractmethod
@@ -468,17 +464,17 @@ bool: True if deactivated successfully
---
### `merge_chunks(chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool`
### `merge_chunks(target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool`
**Description**: Coordinates chunk merging by validating chunks, calling F10 for factor graph merge, and updating chunk states.
**Description**: Merges source_chunk INTO target_chunk. The resulting merged chunk is target_chunk. Source chunk is deactivated after merge.
**Called By**:
- F11 Failure Recovery Coordinator (after successful chunk matching)
**Input**:
```python
chunk_id_1: str # Source chunk (typically newer, to be merged)
chunk_id_2: str # Target chunk (typically older, merged into)
target_chunk_id: str # Target chunk (receives the merge, typically older/main)
source_chunk_id: str # Source chunk (being merged in, typically newer)
transform: Sim3Transform:
translation: np.ndarray # (3,)
rotation: np.ndarray # (3, 3) or quaternion
@@ -492,34 +488,33 @@ bool: True if merge successful
**Processing Flow**:
1. Verify both chunks exist
2. Verify chunk_id_1 is anchored (has_anchor=True)
2. Verify source_chunk_id is anchored (has_anchor=True)
3. Validate chunks can be merged (not already merged, not same chunk)
4. **Merge direction**: chunk_id_1 (newer, source) merges INTO chunk_id_2 (older, target)
5. Call F10.merge_chunks(chunk_id_1, chunk_id_2, transform)
6. Update chunk_id_1 state:
4. Call F10.merge_chunk_subgraphs(flight_id, source_chunk_id, target_chunk_id, transform)
5. Update source_chunk_id state:
- Set is_active=False
- Set matching_status="merged"
- Call deactivate_chunk(chunk_id_1)
7. Update chunk_id_2 state (if needed)
8. Persist chunk state via F03 Flight Database.save_chunk_state()
9. Return True
- Call deactivate_chunk(source_chunk_id)
6. target_chunk remains active (now contains merged frames)
7. Persist chunk state via F03 Flight Database.save_chunk_state()
8. Return True
**Merge Convention**:
- `merge_chunks(target, source)` → source is merged INTO target
- Result is stored in target_chunk
- source_chunk is deactivated after merge
- Example: `merge_chunks("main", "chunk_3")` merges chunk_3 into main trajectory
**Validation**:
- Both chunks must exist
- chunk_id_1 must be anchored
- chunk_id_1 must not already be merged
- chunk_id_1 and chunk_id_2 must be different
**Merge Direction**:
- **chunk_id_1**: Source chunk (newer, recently anchored)
- **chunk_id_2**: Target chunk (older, main trajectory or previous chunk)
- Newer chunks merge INTO older chunks to maintain chronological consistency
- source_chunk must be anchored
- source_chunk must not already be merged
- target_chunk and source_chunk must be different
**Test Cases**:
1. **Merge anchored chunks**: Chunks merged successfully, chunk_id_1 deactivated
2. **Merge unanchored chunk**: Returns False (validation fails)
3. **Merge already merged chunk**: Returns False (validation fails)
4. **State updates**: chunk_id_1 marked as merged and deactivated
1. **Merge anchored chunk**: source_chunk merged into target_chunk
2. **Source deactivated**: source_chunk marked as merged and deactivated
3. **Target unchanged**: target_chunk remains active with new frames
---
docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md
+12 -9
View File
@@ -35,7 +35,7 @@ class ICoordinateTransformer(ABC):
pass
@abstractmethod
def image_object_to_gps(self, object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint:
def image_object_to_gps(self, flight_id: str, frame_id: int, object_pixel: Tuple[float, float]) -> GPSPoint:
pass
@abstractmethod
@@ -273,18 +273,19 @@ Tuple[float, float]: (x, y) pixel coordinates
---
### `image_object_to_gps(object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint`
### `image_object_to_gps(flight_id: str, frame_id: int, object_pixel: Tuple[float, float]) -> GPSPoint`
**Description**: **Critical method** - Converts object pixel coordinates to GPS. Used for external object detection integration.
**Called By**:
- F01 Flight API (via `POST /flights/{flightId}/frames/{frameId}/object-to-gps` endpoint)
- F02 Flight Processor (via convert_object_to_gps delegation from F01)
- F14 Result Manager (converts objects to GPS for output)
**Input**:
```python
object_pixel: Tuple[float, float] # Pixel coordinates from object detector
flight_id: str # Flight identifier (needed for ENU origin and factor graph)
frame_id: int # Frame containing object
object_pixel: Tuple[float, float] # Pixel coordinates from object detector
```
**Output**:
@@ -293,15 +294,17 @@ GPSPoint: GPS coordinates of object center
```
**Processing Flow**:
1. Get frame_pose from F10 Factor Graph
2. Get camera_params from F17 Configuration Manager
3. Get altitude from configuration
1. Get frame_pose from F10 Factor Graph Optimizer.get_trajectory(flight_id)[frame_id]
2. Get camera_params from F17 Configuration Manager.get_flight_config(flight_id)
3. Get altitude from F17 Configuration Manager.get_flight_config(flight_id).altitude
4. Call pixel_to_gps(object_pixel, frame_pose, camera_params, altitude)
5. Return GPS
5. Use enu_to_gps(flight_id, enu_point) for final GPS conversion
6. Return GPS
**User Story**:
- External system detects object in UAV image at pixel (1024, 768)
- Calls image_object_to_gps(frame_id=237, object_pixel=(1024, 768))
- Calls F02.convert_object_to_gps(flight_id="abc", frame_id=237, pixel=(1024, 768))
- F02 delegates to F13.image_object_to_gps(flight_id="abc", frame_id=237, object_pixel=(1024, 768))
- Returns GPSPoint(lat=48.123, lon=37.456)
- Object GPS can be used for navigation, targeting, etc.
docs/02_components/14_result_manager/result_manager_spec.md
+6 -4
View File
@@ -155,8 +155,9 @@ frame_ids: List[int] # Frames with updated poses
**Processing Flow**:
1. For each frame_id:
- Get refined pose from F10 Factor Graph Optimizer → ENU pose
- Convert ENU pose to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_pose)
- Get refined pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position
- Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])`
- Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint
- Update result with refined=True via F03 Flight Database.save_frame_result()
- Update waypoint via F03 Flight Database.update_waypoint()
- Call F15 SSE Event Streamer.send_refinement()
@@ -205,8 +206,9 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
**Processing Flow**:
1. For each frame_id in merged_frames:
- Get updated pose from F10 Factor Graph Optimizer.get_trajectory() → ENU pose
- Convert ENU pose to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_pose)
- Get updated pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position
- Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])`
- Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint
- Update frame result via F03 Flight Database.save_frame_result()
- Update waypoint via F03 Flight Database.update_waypoint()
- Send refinement event via F15 SSE Event Streamer.send_refinement()
docs/02_components/15_sse_event_streamer/sse_event_streamer_spec.md
+48
View File
@@ -28,9 +28,19 @@ class ISSEEventStreamer(ABC):
def send_refinement(self, flight_id: str, frame_id: int, updated_result: FrameResult) -> bool:
pass
@abstractmethod
def send_heartbeat(self, flight_id: str) -> bool:
"""Sends heartbeat/keepalive to all clients subscribed to flight."""
pass
@abstractmethod
def close_stream(self, flight_id: str, client_id: str) -> bool:
pass
@abstractmethod
def get_active_connections(self, flight_id: str) -> int:
"""Returns count of active SSE connections for a flight."""
pass
```
## Component Description
@@ -170,12 +180,50 @@ StreamConnection:
---
### `send_heartbeat(flight_id: str) -> bool`
**Description**: Sends heartbeat/keepalive ping to all clients subscribed to a flight. Keeps connections alive and helps detect stale connections.
**Called By**:
- Background heartbeat task (every 30 seconds)
- F02 Flight Processor (periodically during processing)
**Event Format**:
```
:heartbeat
```
**Behavior**:
- Sends SSE comment (`:heartbeat`) which doesn't trigger event handlers
- Keeps TCP connection alive
- Client can use to detect connection health
**Test Cases**:
1. Send heartbeat → all clients receive ping
2. Client timeout → connection marked stale
---
### `close_stream(flight_id: str, client_id: str) -> bool`
**Description**: Closes SSE connection.
**Called By**: F01 REST API (on client disconnect)
---
### `get_active_connections(flight_id: str) -> int`
**Description**: Returns count of active SSE connections for a flight.
**Called By**:
- F02 Flight Processor (monitoring)
- Admin tools
**Test Cases**:
1. No connections → returns 0
2. 5 clients connected → returns 5
## Integration Tests
### Test 1: Real-Time Streaming
docs/02_components/17_configuration_manager/configuration_manager_spec.md
+71
View File
@@ -27,6 +27,21 @@ class IConfigurationManager(ABC):
@abstractmethod
def update_config(self, section: str, key: str, value: Any) -> bool:
pass
@abstractmethod
def get_operational_altitude(self, flight_id: str) -> float:
"""Returns predefined operational altitude for the flight (NOT from EXIF)."""
pass
@abstractmethod
def get_frame_spacing(self, flight_id: str) -> float:
"""Returns expected distance between consecutive frames in meters."""
pass
@abstractmethod
def save_flight_config(self, flight_id: str, config: FlightConfig) -> bool:
"""Persists flight-specific configuration."""
pass
```
## Component Description
@@ -136,6 +151,62 @@ FlightConfig:
1. Update value → succeeds
2. Invalid key → fails
---
### `get_operational_altitude(flight_id: str) -> float`
**Description**: Returns predefined operational altitude for the flight in meters. This is the altitude provided during flight creation, NOT extracted from EXIF metadata (images don't have GPS/altitude metadata per problem constraints).
**Called By**:
- F10 Factor Graph Optimizer (for scale resolution)
- H02 GSD Calculator (for GSD computation)
- F09 Metric Refinement (for alignment)
**Input**: `flight_id: str`
**Output**: `float` - Altitude in meters (typically 100-500m)
**Test Cases**:
1. Get existing flight altitude → returns value
2. Non-existent flight → raises error
---
### `get_frame_spacing(flight_id: str) -> float`
**Description**: Returns expected distance between consecutive frames in meters. Used for scale estimation in visual odometry.
**Called By**:
- F10 Factor Graph Optimizer (for expected displacement calculation)
**Input**: `flight_id: str`
**Output**: `float` - Expected frame spacing in meters (typically ~100m)
**Test Cases**:
1. Get frame spacing → returns expected distance
---
### `save_flight_config(flight_id: str, config: FlightConfig) -> bool`
**Description**: Persists flight-specific configuration when a flight is created.
**Called By**:
- F02 Flight Processor (during flight creation)
**Input**:
```python
flight_id: str
config: FlightConfig
```
**Output**: `bool` - True if saved successfully
**Test Cases**:
1. Save valid config → succeeds
2. Invalid flight_id → fails
## Data Models
### SystemConfig
docs/02_components/decomposition_plan.md
+48 -3
View File
@@ -189,20 +189,27 @@
| Client | F01 | `POST /flights` | Create flight |
| F01 | F02 | `create_flight()` | Initialize flight state |
| F02 | F16 | `get_flight_config()` | Get camera params, altitude |
| F02 | F12 | `set_enu_origin(start_gps)` | Set ENU coordinate origin |
| F02 | F13 | `set_enu_origin(flight_id, start_gps)` | Set ENU coordinate origin |
| F02 | F04 | `prefetch_route_corridor()` | Prefetch tiles |
| F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download |
| F04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations |
| F02 | F03 | `insert_flight()` | Persist flight data |
### SSE Stream Creation
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | F01 | `GET .../stream` | Open SSE connection |
| F01 | F14 | `create_stream()` | Establish SSE channel |
| F01 | F02 | `create_client_stream()` | Route through coordinator |
| F02 | F15 | `create_stream()` | Establish SSE channel |
### Image Upload
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| Client | F01 | `POST .../images/batch` | Upload 10-50 images |
| F01 | F05 | `queue_batch()` | Queue for processing |
| F01 | F02 | `queue_images()` | Route through coordinator |
| F02 | F05 | `queue_batch()` | Queue for processing |
| F05 | H08 | `validate_batch()` | Validate sequence, format |
| F05 | F03 | `save_image_metadata()` | Persist image metadata |
@@ -433,3 +440,41 @@ F02 Flight Processor handles multiple concerns:
**Event Recovery**:
- Events are fire-and-forget (no persistence)
- Subscribers rebuild state from F03 on restart
### Error Propagation Strategy
**Principle**: Errors propagate upward through the component hierarchy. Lower-level components throw exceptions or return error results; higher-level coordinators handle recovery.
**Error Propagation Chain**:
```
H01-H08 (Helpers) → F07/F08/F09 (Visual Processing) → F11 (Recovery) → F02 (Coordinator) → F01 (API) → Client
↓
Events → F14 → F15 → SSE → Client
```
**Error Categories**:
1. **Recoverable** (F11 handles):
- Tracking loss → Progressive search
- Low confidence → Rotation sweep
- Chunk matching fails → User input request
2. **Propagated to Client** (via SSE):
- User input needed → `user_input_needed` event
- Processing blocked → `processing_blocked` event
- Flight completed → `flight_completed` event
3. **Fatal** (F02 handles, returns to F01):
- Database connection lost → HTTP 503
- Model loading failed → HTTP 500
- Invalid configuration → HTTP 400
**User Fix Flow**:
```
Client ---> F01 (POST /user-fix) ---> F02.handle_user_fix() ---> F11.apply_user_anchor()
↓
F10.add_chunk_anchor()
↓
F02 receives UserFixApplied event
↓
F14.publish_result() ---> F15 ---> SSE ---> Client
```
docs/02_components/helpers/h04_faiss_index_manager_spec.md
+26 -3
View File
@@ -27,11 +27,25 @@ class IFaissIndexManager(ABC):
@abstractmethod
def load_index(self, path: str) -> FaissIndex:
pass
@abstractmethod
def is_gpu_available(self) -> bool:
pass
@abstractmethod
def set_device(self, device: str) -> bool:
"""Set device: 'gpu' or 'cpu'."""
pass
```
## Component Description
Manages Faiss indices for AnyLoc retrieval (IVF, HNSW options).
Manages Faiss indices for DINOv2 descriptor similarity search. H04 builds indexes from UAV image descriptors for:
1. **Loop closure detection**: Find when UAV revisits previously seen areas within the same flight
2. **Chunk-to-chunk matching**: Match disconnected chunks to each other
3. **Flight-to-flight matching**: Match current flight to previous flights in same area
**Index Source**: Descriptors are computed from UAV images using F08's DINOv2 encoder, NOT from satellite images. The index enables finding similar UAV viewpoints.
## API Methods
@@ -76,9 +90,18 @@ Manages Faiss indices for AnyLoc retrieval (IVF, HNSW options).
**External**: faiss-gpu or faiss-cpu
## GPU/CPU Fallback
H04 supports automatic fallback from GPU to CPU:
- `is_gpu_available()`: Returns True if faiss-gpu is available and CUDA works
- `set_device("gpu")`: Use GPU acceleration (faster for large indexes)
- `set_device("cpu")`: Use CPU (fallback when GPU unavailable)
## Test Cases
1. Build index with 10,000 descriptors → succeeds
2. Search query → returns top-k matches
1. Build index with 10,000 UAV image descriptors → succeeds
2. Search query UAV descriptor → returns top-k similar UAV frames
3. Save/load index → index restored correctly
4. GPU unavailable → automatically falls back to CPU
5. Add descriptors incrementally → index grows correctly
docs/02_components/system_flows.md
+90 -90
View File
@@ -185,7 +185,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
┌──────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │
│ │
│ ┌─────────────┐ │
@@ -265,15 +265,15 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F06 Image Rotation Manager │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ try_rotation_steps(image, satellite_tile, tile_bounds) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┴───────────────────────────┐ │
│ │ For angle in [0°, 30°, 60°, ... 330°]: │ │
┌───────────────────────────────────────────────────────────────────────────┐
│ F06 Image Rotation Manager │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ try_rotation_steps(image, satellite_tile, tile_bounds) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┴────────────────────────────┐ │
│ │ For angle in [0°, 30°, 60°, ... 330°]: │ │
│ │ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ H07 rotate_image(image, angle) │ │ │
@@ -300,11 +300,11 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ │ │
│ └────────────────────────────────────────────────────┘ │
│ │ │
│ ┌────────────┴────────────┐ │
│ │ Return RotationResult │ │
│ │ or None │ │
│ └─────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│ ┌────────────┴────────────┐ │
│ │ Return RotationResult │ │
│ │ or None │ │
│ └─────────────────────────┘ │
└───────────────────────────────────────────────────────────────────────────┘
```
**Output**: RotationResult with precise heading angle
@@ -319,7 +319,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
┌──────────────────────────────────────────────────────────────────────────┐
│ F11 Failure Recovery Coordinator │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -337,34 +337,34 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ ├─ F06 requires_rotation_sweep() → trigger sweep │ │
│ │ ├─ F08 retrieve_candidate_tiles() (DINOv2) │ │
│ │ └─ Progressive tile search (1→4→9→16→25): │ │
│ │ │ │
│ │ ┌───────────────────────────────────────────────────┐ │ │
│ │ │ For grid_size in [1, 4, 9, 16, 25]: │ │ │
│ │ │ ├─ F04 expand_search_grid() │ │ │
│ │ │ ├─ For each tile: │ │ │
│ │ │ │ ├─ F04 compute_tile_bounds() │ │ │
│ │ │ │ └─ F09 align_to_satellite(img, tile, bounds)│ │ │
│ │ │ │ │ │ │
│ │ │ └─ If match found: BREAK │ │ │
│ │ └───────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ │ │
│ │ ┌───────────────────────────────────────────────────┐ │ │
│ │ │ For grid_size in [1, 4, 9, 16, 25]: │ │ │
│ │ │ ├─ F04 expand_search_grid() │ │ │
│ │ │ ├─ For each tile: │ │ │
│ │ │ │ ├─ F04 compute_tile_bounds() │ │ │
│ │ │ │ └─ F09 align_to_satellite(img, tile, bounds)│ │ │
│ │ │ │ │ │ │
│ │ │ └─ If match found: BREAK │ │ │
│ │ └───────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌────────────────────┴────────────────────┐ │
│ │ Single-image match found? │ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌─────────────────────────────┐ │
│ │ EMIT RecoverySucceeded│ │ Continue chunk building │ │
│ │ Resume normal flow │ │ → Flow 7 (Chunk Building) │ │
│ └─────────────────────┘ │ → Flow 8 (Chunk Matching) │ │
│ │ (Background) │ │
│ └─────────────────────────────┘ │
│ ┌─────────────────────┐ ┌────────────────────────────┐ │
│ │ EMIT RecoverySucceeded│ │ Continue chunk building │ │
│ │ Resume normal flow │ │ → Flow 7 (Chunk Building) │ │
│ └─────────────────────┘ │ → Flow 8 (Chunk Matching) │ │
│ │ (Background) │ │
│ └────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────┐ │
│ │ If all strategies exhausted: │ │
│ │ → Flow 10 (User Input Recovery) │ │
│ └─────────────────────────────────────┘ │
│ ┌─────────────────────────────────────┐ │
│ │ If all strategies exhausted: │ │
│ │ → Flow 10 (User Input Recovery) │ │
│ └─────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
@@ -376,42 +376,42 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F12 Route Chunk Manager │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ create_chunk(flight_id, start_frame_id) │ │
│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │
│ │ ├─ Initialize chunk state (unanchored, active) │ │
│ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┴───────────────────────────┐ │
│ │ For each frame in chunk: │ │
┌───────────────────────────────────────────────────────────────────────────┐
│ F12 Route Chunk Manager │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ create_chunk(flight_id, start_frame_id) │ │
│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │
│ │ ├─ Initialize chunk state (unanchored, active) │ │
│ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┴────────────────────────────┐ │
│ │ For each frame in chunk: │ │
│ │ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F07 compute_relative_pose_in_chunk()│ │ │
│ │ └───────────────────┬─────────────────┘ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F12 add_frame_to_chunk() │ │ │
│ │ ┌──────────────────────────────────────┐ │ │
│ │ │ F12 add_frame_to_chunk() │ │ │
│ │ │ └─ F10 add_relative_factor_to_chunk│ │ │
│ │ └───────────────────┬─────────────────┘ │ │
│ │ └───────────────────┬──────────────────┘ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F10 optimize_chunk() (local) │ │ │
│ │ └─────────────────────────────────────┘ │ │
│ │ │ │
│ └────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ is_chunk_ready_for_matching()? │ │
│ │ ├─ Min 5 frames │ │
│ │ ├─ Max 20 frames │ │
│ │ └─ Internal consistency (good VO inlier counts) │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ is_chunk_ready_for_matching()? │ │
│ │ ├─ Min 5 frames │ │
│ │ ├─ Max 20 frames │ │
│ │ └─ Internal consistency (good VO inlier counts) │ │
│ └─────────────────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────────────────────┘
```
---
@@ -424,7 +424,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
┌──────────────────────────────────────────────────────────────────────────┐
│ F11 process_unanchored_chunks() (Background Task) │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -472,7 +472,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
┌───────────────────────────────────────────────────────────────────────-──┐
│ F11 merge_chunk_to_trajectory() │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -485,7 +485,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 3. Resolve target: F12 get_merge_target(chunk_id) │ │
│ │ 3. Determine target chunk (predecessor or "main") │ │
│ │ └─ Returns target_chunk_id (predecessor or "main") │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
@@ -501,7 +501,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 6. EMIT ChunkMerged event (flight_id, chunk_id, merged_frames)│ │
│ │ 6. EMIT ChunkMerged event (flight_id,chunk_id,merged_frames)│ │
│ │ └─ F14 subscribes → update_results_after_chunk_merge() │ │
│ │ ├─ F10 get_trajectory() → ENU poses │ │
│ │ ├─ F13 enu_to_gps() for each frame │ │
@@ -523,7 +523,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
┌──────────────────────────────────────────────────────────────────────────┐
│ F11 create_user_input_request() │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -537,7 +537,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
└──────────────────────────────────────────────────────────────────────────┘
│
▼ Client receives SSE event
┌──────────────────────────────────────────────────────────────────────────┐
┌───────────────────────────────────────────────────────────────────────────┐
│ Client UI │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -550,7 +550,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ │ POST /flights/{flightId}/user-fix │ │
│ │ body: { frame_id, uav_pixel, satellite_gps } │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
└───────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
@@ -577,7 +577,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
┌─────────────────────────────────────────────────────────────-────────────┐
│ F10 Background Optimization │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -650,7 +650,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
┌──────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
@@ -741,7 +741,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ EXTERNAL │
│ EXTERNAL │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Client │ │ Satellite │ │ External │ │
│ │ (UI) │ │ Provider │ │ Detector │ │
@@ -750,7 +750,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│ REST/SSE │ HTTP │ REST
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ API LAYER │
│ API LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F01 Flight API │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
@@ -758,23 +758,23 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ ORCHESTRATION LAYER │
│ ORCHESTRATION LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F02 Flight Processor │ │
│ │ (Central coordinator, event subscriber, background task manager) │ │
│ │ F02 Flight Processor │ │
│ │ (Central coordinator, event subscriber, background task manager) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
├─────────────────────────────────────────────────────────────┐
▼ ▼
┌─────────────────────────────────────────────┐ ┌────────────────────────────┐
│ DATA MANAGEMENT LAYER │ │ RECOVERY LAYER │
│ DATA MANAGEMENT LAYER │ │ RECOVERY LAYER │
│ ┌─────────────┐ ┌─────────────┐ │ │ ┌───────────────────────┐ │
│ │ F04 │ │ F05 │ │ │ │ F11 │ │
│ │ Satellite │ │ Image │ │ │ │ Failure Recovery │ │
│ │ Data │ │ Input │ │ │ │ (Event emitter) │ │
│ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │
│ │ │ │ │
│ │ │ │ │
│ ┌─────────────┐ ┌─────────────┐ │ │ ▼ │
│ │ F12 │ │ F03 │ │ │ ┌───────────────────────┐ │
│ │Route Chunk │ │ Flight │ │ │ │ F12 │ │
@@ -784,7 +784,7 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ VISUAL PROCESSING LAYER │
│ VISUAL PROCESSING LAYER │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ┌──────────────┐ │
│ │ F06 │ │ F07 │ │ F08 │ │ F09 │ │
│ │ Rotation │ │ Sequential VO │ │ Global │ │ Metric │ │
@@ -796,16 +796,16 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ STATE ESTIMATION LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F10 Factor Graph Optimizer │ │
│ │ (GTSAM, iSAM2, Robust kernels, Chunk subgraphs, Sim(3) merging) │ │
│ STATE ESTIMATION LAYER │
│ ┌──────────────────────────────────────────────────────────────────k──┐ │
│ │ F10 Factor Graph Optimizer │ │
│ │ (GTSAM, iSAM2, Robust kernels, Chunk subgraphs, Sim(3) merging) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ OUTPUT LAYER │
│ OUTPUT LAYER │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ F13 │ │ F14 │ │ F15 │ │
│ │ Coordinate │ │ Result │ │ SSE │ │
@@ -815,14 +815,14 @@ ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer match
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
┌─────────────────────────────────────────────────────────────────────────────-┐
│ INFRASTRUCTURE LAYER │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────────────────┐│
│ │ F16 │ │ F17 │ │ HELPERS ││
│ │ Model │ │Configuration │ │ H01-H08 (Camera, GSD, Kernels, ││
│ │ Manager │ │ Manager │ │ Faiss, Monitor, Mercator, etc) ││
│ └───────────────┘ └───────────────┘ └───────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────────────────┘
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────────────────-┐│
│ │ F16 │ │ F17 │ │ HELPERS ││
│ │ Model │ │Configuration │ │ H01-H08 (Camera, GSD, Kernels, ││
│ │ Manager │ │ Manager │ │ Faiss, Monitor, Mercator, etc) ││
│ └───────────────┘ └───────────────┘ └────────────────────────────────────┘│
└──────────────────────────────────────────────────────────────────────────────┘
```
---