mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-04-22 07:16:38 +00:00
Merge branch 'main' of https://bitbucket.org/zxsanny/gps-denied
This commit is contained in:
@@ -18,4 +18,4 @@
|
||||
|
||||
- Mean Reprojection Error (MRE) < 1.0 pixels. The distance, in pixels, between the original pixel location of the object and the re-projected pixel location.
|
||||
|
||||
- The whole system should work as a background service. The interaction should be done by zeromq. Sevice should be up and running and awaiting for the initial input message. On the input message processing should started, and immediately after the first results system should provide them to the client
|
||||
- The whole system should work as a background service exposed via REST API with Server-Sent Events (SSE) for real-time streaming. Service should be up and running and awaiting for the initial request. On the request processing should start, and immediately after the first results system should provide them to the client via SSE stream
|
||||
@@ -43,6 +43,10 @@ class IFlightAPI(ABC):
|
||||
@abstractmethod
|
||||
def create_sse_stream(self, flight_id: str) -> SSEStream:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def convert_object_to_gps(self, flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> ObjectGPSResponse:
|
||||
pass
|
||||
```
|
||||
|
||||
## Component Description
|
||||
@@ -307,8 +311,9 @@ BatchResponse:
|
||||
1. Validate flight_id exists
|
||||
2. Validate batch size (10-50 images)
|
||||
3. Validate sequence numbers (strict sequential)
|
||||
4. Pass to F05 Image Input Pipeline
|
||||
5. Return immediately (processing is async)
|
||||
4. Call F02 Flight Processor → queue_images(flight_id, batch)
|
||||
5. F02 delegates to F05 Image Input Pipeline
|
||||
6. Return immediately (processing is async)
|
||||
|
||||
**Error Conditions**:
|
||||
- `400 Bad Request`: Invalid batch size, out-of-sequence images
|
||||
@@ -351,9 +356,10 @@ UserFixResponse:
|
||||
|
||||
**Processing Flow**:
|
||||
1. Validate flight_id exists and is blocked
|
||||
2. Pass to F11 Failure Recovery Coordinator
|
||||
3. Coordinator applies anchor to Factor Graph
|
||||
4. Resume processing pipeline
|
||||
2. Call F02 Flight Processor → handle_user_fix(flight_id, fix_data)
|
||||
3. F02 delegates to F11 Failure Recovery Coordinator
|
||||
4. Coordinator applies anchor to Factor Graph
|
||||
5. Resume processing pipeline
|
||||
|
||||
**Error Conditions**:
|
||||
- `400 Bad Request`: Invalid fix data
|
||||
@@ -367,6 +373,51 @@ UserFixResponse:
|
||||
|
||||
---
|
||||
|
||||
### `convert_object_to_gps(flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> ObjectGPSResponse`
|
||||
|
||||
**REST Endpoint**: `POST /flights/{flightId}/frames/{frameId}/object-to-gps`
|
||||
|
||||
**Description**: Converts object pixel coordinates to GPS. Used by external object detection systems (e.g., Azaion.Inference) to get GPS coordinates for detected objects.
|
||||
|
||||
**Called By**:
|
||||
- External object detection systems (Azaion.Inference)
|
||||
- Any system needing pixel-to-GPS conversion for a specific frame
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
ObjectToGPSRequest:
|
||||
pixel_x: float # X coordinate in image
|
||||
pixel_y: float # Y coordinate in image
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
ObjectGPSResponse:
|
||||
gps: GPSPoint
|
||||
accuracy_meters: float # Estimated accuracy
|
||||
frame_id: int
|
||||
pixel: Tuple[float, float]
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Validate flight_id and frame_id exist
|
||||
2. Validate frame has been processed (has pose in Factor Graph)
|
||||
3. Call F02 Flight Processor → convert_object_to_gps(flight_id, frame_id, pixel)
|
||||
4. F02 delegates to F13.image_object_to_gps(flight_id, frame_id, pixel)
|
||||
5. Return GPS with accuracy estimate
|
||||
|
||||
**Error Conditions**:
|
||||
- `400 Bad Request`: Invalid pixel coordinates
|
||||
- `404 Not Found`: flight_id or frame_id not found
|
||||
- `409 Conflict`: Frame not yet processed (no pose available)
|
||||
|
||||
**Test Cases**:
|
||||
1. **Valid conversion**: Object at (1024, 768) → returns GPS
|
||||
2. **Unprocessed frame**: Frame not in Factor Graph → returns 409
|
||||
3. **Invalid pixel**: Negative coordinates → returns 400
|
||||
|
||||
---
|
||||
|
||||
### `get_flight_status(flight_id: str) -> FlightStatusResponse`
|
||||
|
||||
**REST Endpoint**: `GET /flights/{flightId}/status`
|
||||
@@ -431,6 +482,12 @@ SSE Stream with events:
|
||||
- flight_completed
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Validate flight_id exists
|
||||
2. Call F02 Flight Processor → create_client_stream(flight_id, client_id)
|
||||
3. F02 delegates to F15 SSE Event Streamer → create_stream()
|
||||
4. Return SSE stream to client
|
||||
|
||||
**Event Format**:
|
||||
```json
|
||||
{
|
||||
@@ -527,11 +584,9 @@ SSE Stream with events:
|
||||
## Dependencies
|
||||
|
||||
### Internal Components
|
||||
- **F02 Flight Processor**: For all flight operations and processing orchestration
|
||||
- **F05 Image Input Pipeline**: For batch processing
|
||||
- **F11 Failure Recovery Coordinator**: For user fixes
|
||||
- **F15 SSE Event Streamer**: For real-time streaming
|
||||
- **F03 Flight Database**: For persistence (via F02)
|
||||
- **F02 Flight Processor**: For ALL operations (flight CRUD, image batching, user fixes, SSE streams, object-to-GPS conversion). F01 is a thin REST layer that delegates all business logic to F02.
|
||||
|
||||
**Note**: F01 does NOT directly call F05, F11, F13, or F15. All operations are routed through F02 to maintain a single coordinator pattern.
|
||||
|
||||
### External Dependencies
|
||||
- **FastAPI**: Web framework
|
||||
|
||||
@@ -59,6 +59,27 @@ class IFlightProcessor(ABC):
|
||||
def validate_flight_continuity(self, waypoints: List[Waypoint]) -> ValidationResult:
|
||||
pass
|
||||
|
||||
# API Delegation Methods (called by F01)
|
||||
@abstractmethod
|
||||
def queue_images(self, flight_id: str, batch: ImageBatch) -> BatchQueueResult:
|
||||
"""Delegates to F05 Image Input Pipeline."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def handle_user_fix(self, flight_id: str, fix_data: UserFixRequest) -> UserFixResult:
|
||||
"""Delegates to F11 Failure Recovery Coordinator."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def create_client_stream(self, flight_id: str, client_id: str) -> StreamConnection:
|
||||
"""Delegates to F15 SSE Event Streamer."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def convert_object_to_gps(self, flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint:
|
||||
"""Delegates to F13 Coordinate Transformer."""
|
||||
pass
|
||||
|
||||
# Processing Orchestration
|
||||
@abstractmethod
|
||||
def process_frame(self, flight_id: str, frame_id: int) -> FrameResult:
|
||||
@@ -113,6 +134,31 @@ class IFlightProcessor(ABC):
|
||||
- System initialization and resource management
|
||||
- Flight state machine management
|
||||
- **Chunk-aware frame processing (Atlas multi-map architecture)**
|
||||
- **Background task management for each flight**
|
||||
- **Event subscription for F11 recovery events**
|
||||
|
||||
### Event-Based Communication
|
||||
|
||||
F02 subscribes to events from F11 Failure Recovery Coordinator instead of F11 directly calling F02's status update methods. This decouples recovery logic from flight state management.
|
||||
|
||||
**Events Subscribed (from F11)**:
|
||||
- `RecoveryStarted`: Update flight status to "recovering"
|
||||
- `RecoverySucceeded`: Update flight status to "processing", resume processing loop
|
||||
- `RecoveryFailed`: Update flight status to "blocked", blocked=True
|
||||
- `UserInputNeeded`: Update flight status to "blocked", blocked=True, await user fix
|
||||
- `UserFixApplied`: Update flight status to "processing", resume processing loop
|
||||
- `ChunkCreated`: Log chunk creation, update internal tracking
|
||||
- `ChunkAnchored`: Log chunk anchor, update statistics
|
||||
- `ChunkMerged`: Trigger result updates via F14
|
||||
|
||||
### Background Task Management
|
||||
|
||||
F02 is responsible for managing background tasks per flight:
|
||||
- **Chunk matching task**: Background task for matching unanchored chunks (delegated to F11)
|
||||
- **Asynchronous refinement task**: Background optimization and result publishing (via F10, F14)
|
||||
- **Task lifecycle**: Tasks created on flight start, cancelled on flight completion/deletion
|
||||
|
||||
Background tasks are coordinated via Python `asyncio` or `ThreadPoolExecutor`, with F02 as the owner. F11 executes chunk matching logic but F02 schedules and monitors the tasks.
|
||||
|
||||
---
|
||||
|
||||
@@ -551,6 +597,129 @@ ValidationResult:
|
||||
|
||||
---
|
||||
|
||||
## API Delegation Methods
|
||||
|
||||
These methods are called by F01 Flight API and delegate to specialized components. This maintains F02 as the single coordinator for all operations.
|
||||
|
||||
### `queue_images(flight_id: str, batch: ImageBatch) -> BatchQueueResult`
|
||||
|
||||
**Description**: Queues image batch for processing. Delegates to F05 Image Input Pipeline.
|
||||
|
||||
**Called By**:
|
||||
- F01 Flight API (upload_image_batch endpoint)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
flight_id: str
|
||||
batch: ImageBatch
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
BatchQueueResult:
|
||||
accepted: bool
|
||||
sequences: List[int]
|
||||
next_expected: int
|
||||
message: Optional[str]
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Validate flight exists and is in valid state
|
||||
2. Delegate to F05 Image Input Pipeline → queue_batch()
|
||||
3. Return result to F01
|
||||
|
||||
---
|
||||
|
||||
### `handle_user_fix(flight_id: str, fix_data: UserFixRequest) -> UserFixResult`
|
||||
|
||||
**Description**: Handles user-provided GPS fix. Delegates to F11 Failure Recovery Coordinator.
|
||||
|
||||
**Called By**:
|
||||
- F01 Flight API (submit_user_fix endpoint)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
flight_id: str
|
||||
fix_data: UserFixRequest:
|
||||
frame_id: int
|
||||
uav_pixel: Tuple[float, float]
|
||||
satellite_gps: GPSPoint
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
UserFixResult:
|
||||
accepted: bool
|
||||
processing_resumed: bool
|
||||
message: Optional[str]
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Validate flight exists and is blocked
|
||||
2. Delegate to F11 Failure Recovery Coordinator → apply_user_anchor()
|
||||
3. F11 emits UserFixApplied event (F02 subscribes and resumes processing)
|
||||
4. Return result to F01
|
||||
|
||||
---
|
||||
|
||||
### `create_client_stream(flight_id: str, client_id: str) -> StreamConnection`
|
||||
|
||||
**Description**: Creates SSE stream for client. Delegates to F15 SSE Event Streamer.
|
||||
|
||||
**Called By**:
|
||||
- F01 Flight API (create_sse_stream endpoint)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
flight_id: str
|
||||
client_id: str
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
StreamConnection:
|
||||
stream_id: str
|
||||
flight_id: str
|
||||
client_id: str
|
||||
last_event_id: Optional[str]
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Validate flight exists
|
||||
2. Delegate to F15 SSE Event Streamer → create_stream()
|
||||
3. Return StreamConnection to F01
|
||||
|
||||
---
|
||||
|
||||
### `convert_object_to_gps(flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> GPSPoint`
|
||||
|
||||
**Description**: Converts object pixel to GPS. Delegates to F13 Coordinate Transformer.
|
||||
|
||||
**Called By**:
|
||||
- F01 Flight API (convert_object_to_gps endpoint)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
flight_id: str
|
||||
frame_id: int
|
||||
pixel: Tuple[float, float]
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
GPSPoint:
|
||||
lat: float
|
||||
lon: float
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Validate flight and frame exist
|
||||
2. Validate frame has been processed (has pose)
|
||||
3. Delegate to F13 Coordinate Transformer → image_object_to_gps(flight_id, frame_id, pixel)
|
||||
4. Return GPSPoint to F01
|
||||
|
||||
---
|
||||
|
||||
## Processing Orchestration Methods
|
||||
|
||||
### `process_frame(flight_id: str, frame_id: int) -> FrameResult`
|
||||
|
||||
@@ -130,6 +130,21 @@ class IFlightDatabase(ABC):
|
||||
- Image metadata storage
|
||||
- Query optimization for large datasets
|
||||
|
||||
### Design Decision: Denormalization
|
||||
|
||||
The schema uses strategic denormalization to optimize for the most common access patterns:
|
||||
|
||||
**Denormalized Fields**:
|
||||
- `frame_results` stores `gps_lat`, `gps_lon` directly (not as foreign key to waypoints)
|
||||
- `flight_state` duplicates `frames_processed` (could be computed from frame_results)
|
||||
- `chunks.frames` stored as JSONB array (not normalized into separate frame_chunk mapping table)
|
||||
|
||||
**Rationale**:
|
||||
- Read-heavy workload: Frame results are read 100x more than written
|
||||
- Avoids JOINs in critical path (per-frame processing)
|
||||
- Simplifies chunk lifecycle (all chunk data in single row)
|
||||
- Acceptable trade-off: Slightly increased storage, significantly faster reads
|
||||
|
||||
---
|
||||
|
||||
## Flight Operations
|
||||
@@ -661,6 +676,13 @@ Optional[Dict]: Metadata dictionary or None
|
||||
|
||||
## Chunk State Operations
|
||||
|
||||
**Necessity**: These methods are **required** for crash recovery. Without them:
|
||||
- Chunk state would be lost on system restart
|
||||
- Processing would need to start from scratch
|
||||
- Background chunk matching progress would be lost
|
||||
|
||||
F12 Route Chunk Manager delegates persistence to F03 to maintain separation of concerns (F12 manages chunk logic, F03 handles storage).
|
||||
|
||||
### `save_chunk_state(flight_id: str, chunk: ChunkHandle) -> bool`
|
||||
|
||||
**Description**: Saves chunk state to database for crash recovery.
|
||||
|
||||
@@ -25,11 +25,11 @@ class ISatelliteDataManager(ABC):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def cache_tile(self, tile_coords: TileCoords, tile_data: np.ndarray) -> bool:
|
||||
def cache_tile(self, flight_id: str, tile_coords: TileCoords, tile_data: np.ndarray) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_cached_tile(self, tile_coords: TileCoords) -> Optional[np.ndarray]:
|
||||
def get_cached_tile(self, flight_id: str, tile_coords: TileCoords) -> Optional[np.ndarray]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
|
||||
@@ -350,7 +350,7 @@ ProcessingStatus:
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Get flight state via F02 Flight Processor.get_flight_state(flight_id)
|
||||
1. Get flight state via F03 Flight Database.get_flight(flight_id).status
|
||||
2. Combine with internal queue status
|
||||
3. Return ProcessingStatus
|
||||
|
||||
@@ -401,9 +401,8 @@ ProcessingStatus:
|
||||
## Dependencies
|
||||
|
||||
### Internal Components
|
||||
- **H08 Batch Validator**: For validation logic
|
||||
- **F03 Flight Database**: For metadata persistence
|
||||
- **F02 Flight Processor**: For flight state information
|
||||
- **F03 Flight Database**: For metadata persistence and flight state information
|
||||
- **H08 Batch Validator**: For batch validation (naming convention, sequence continuity, format, dimensions)
|
||||
|
||||
### External Dependencies
|
||||
- **opencv-python**: Image I/O
|
||||
|
||||
@@ -117,10 +117,17 @@ flight_id: str
|
||||
frame_id: int # Frame identifier for heading persistence
|
||||
image: np.ndarray # UAV image
|
||||
satellite_tile: np.ndarray # Satellite reference tile
|
||||
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (for F09)
|
||||
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (passed to F09)
|
||||
timestamp: datetime # Timestamp for heading persistence
|
||||
```
|
||||
|
||||
**About tile_bounds**: `TileBounds` contains the GPS bounding box of the satellite tile:
|
||||
- `nw`, `ne`, `sw`, `se`: GPS coordinates of tile corners
|
||||
- `center`: GPS coordinate of tile center
|
||||
- `gsd`: Ground Sampling Distance (meters/pixel)
|
||||
|
||||
The caller (F02 Flight Processor) obtains tile_bounds by calling `F04.compute_tile_bounds(tile_coords)` before calling this method. F06 passes tile_bounds to F09.align_to_satellite() which uses it to convert pixel coordinates to GPS.
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
RotationResult:
|
||||
@@ -488,6 +495,7 @@ return None # No match found
|
||||
## Dependencies
|
||||
|
||||
### Internal Components
|
||||
- **F04 Satellite Data Manager**: For tile fetching (`fetch_tile`) and tile bounds computation (`compute_tile_bounds`)
|
||||
- **F09 Metric Refinement**: For matching during rotation sweep (align_to_satellite, align_chunk_to_satellite). F06 rotates images, F09 performs the actual matching.
|
||||
- **H07 Image Rotation Utils**: For image rotation and angle calculations
|
||||
- **F12 Route Chunk Manager**: For chunk image retrieval
|
||||
|
||||
+3
-49
@@ -23,12 +23,10 @@ class ISequentialVO(ABC):
|
||||
@abstractmethod
|
||||
def estimate_motion(self, matches: Matches, camera_params: CameraParameters) -> Optional[Motion]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def compute_relative_pose_in_chunk(self, prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]:
|
||||
pass
|
||||
```
|
||||
|
||||
**Note**: F07 is chunk-agnostic. It only computes relative poses between images. The caller (F02 Flight Processor) determines which chunk the frames belong to and routes factors to the appropriate subgraph via F12 → F10.
|
||||
|
||||
## Component Description
|
||||
|
||||
### Responsibilities
|
||||
@@ -38,14 +36,13 @@ class ISequentialVO(ABC):
|
||||
- Estimate relative pose (translation + rotation) between frames
|
||||
- Return relative pose factors for Factor Graph Optimizer
|
||||
- Detect tracking loss (low inlier count)
|
||||
- **Chunk-aware VO operations (factors added to chunk subgraph)**
|
||||
|
||||
### Scope
|
||||
- Frame-to-frame visual odometry
|
||||
- Feature-based motion estimation
|
||||
- Handles low overlap and challenging agricultural environments
|
||||
- Provides relative measurements for trajectory optimization
|
||||
- **Chunk-scoped operations (Atlas multi-map architecture)**
|
||||
- **Chunk-agnostic**: F07 doesn't know about chunks. Caller (F02) routes results to appropriate chunk subgraph.
|
||||
|
||||
## API Methods
|
||||
|
||||
@@ -224,49 +221,6 @@ Motion:
|
||||
2. **Low inliers**: May return None
|
||||
3. **Degenerate motion**: Handles pure rotation
|
||||
|
||||
---
|
||||
|
||||
### `compute_relative_pose_in_chunk(prev_image: np.ndarray, curr_image: np.ndarray, chunk_id: str) -> Optional[RelativePose]`
|
||||
|
||||
**Description**: Computes relative camera pose between consecutive frames within a chunk context.
|
||||
|
||||
**Called By**:
|
||||
- F02 Flight Processor (chunk-aware processing)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
prev_image: np.ndarray # Previous frame (t-1)
|
||||
curr_image: np.ndarray # Current frame (t)
|
||||
chunk_id: str # Chunk identifier for context
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
RelativePose:
|
||||
translation: np.ndarray # (x, y, z) in meters
|
||||
rotation: np.ndarray # 3×3 rotation matrix or quaternion
|
||||
confidence: float # 0.0 to 1.0
|
||||
inlier_count: int
|
||||
total_matches: int
|
||||
tracking_good: bool
|
||||
chunk_id: str # Chunk context
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Same as compute_relative_pose() (SuperPoint + LightGlue)
|
||||
2. Return RelativePose with chunk_id context
|
||||
3. Factor will be added to chunk's subgraph (not global graph)
|
||||
|
||||
**Chunk Context**:
|
||||
- VO operations are chunk-scoped
|
||||
- Factors added to chunk's subgraph via F10.add_relative_factor_to_chunk()
|
||||
- Chunk isolation ensures independent optimization
|
||||
|
||||
**Test Cases**:
|
||||
1. **Chunk-aware VO**: Returns RelativePose with chunk_id
|
||||
2. **Chunk isolation**: Factors isolated to chunk
|
||||
3. **Multiple chunks**: VO operations don't interfere between chunks
|
||||
|
||||
## Integration Tests
|
||||
|
||||
### Test 1: Normal Flight Sequence
|
||||
|
||||
@@ -25,7 +25,7 @@ class IGlobalPlaceRecognition(ABC):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def initialize_database(self, satellite_tiles: List[SatelliteTile]) -> bool:
|
||||
def load_index(self, flight_id: str, index_path: str) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
@@ -45,7 +45,7 @@ class IGlobalPlaceRecognition(ABC):
|
||||
- Compute image descriptors robust to season/appearance changes
|
||||
- Query Faiss index of satellite tile descriptors
|
||||
- Return top-k candidate tile regions for progressive refinement
|
||||
- Initialize satellite descriptor database during system startup
|
||||
- **Load pre-built satellite descriptor index** (index is built by satellite provider, NOT by F08)
|
||||
- **Chunk semantic matching (aggregate DINOv2 features)**
|
||||
- **Chunk descriptor computation for robust matching**
|
||||
|
||||
@@ -203,54 +203,48 @@ List[TileCandidate] # Re-ranked list
|
||||
|
||||
---
|
||||
|
||||
### `initialize_database(satellite_tiles: List[SatelliteTile]) -> bool`
|
||||
### `load_index(flight_id: str, index_path: str) -> bool`
|
||||
|
||||
**Description**: Loads pre-built satellite descriptor database. **Note**: Semantic index building (DINOv2 descriptors) is performed by the satellite provider service, not during system startup.
|
||||
**Description**: Loads pre-built satellite descriptor database from file. **Note**: The semantic index (DINOv2 descriptors + Faiss index) MUST be provided by the satellite data provider. F08 does NOT build the index - it only loads it.
|
||||
|
||||
**Called By**:
|
||||
- F02 Flight Processor (during system initialization)
|
||||
- F02 Flight Processor (during flight initialization, index_path from F04 Satellite Data Manager)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
List[SatelliteTile]:
|
||||
tile_id: str
|
||||
image: np.ndarray # Optional - only if building index locally
|
||||
gps_center: GPSPoint
|
||||
bounds: TileBounds
|
||||
descriptor: Optional[np.ndarray] # Pre-computed descriptor from provider
|
||||
index_path: str # Path to pre-built Faiss index file from satellite provider
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
bool: True if database initialized successfully
|
||||
bool: True if database loaded successfully
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. **Load pre-built index**: Satellite provider provides pre-computed DINOv2 descriptors
|
||||
2. If descriptors provided:
|
||||
- Load descriptors directly
|
||||
- Build Faiss index using H04 Faiss Index Manager
|
||||
3. If descriptors not provided (fallback):
|
||||
- For each satellite tile:
|
||||
- compute_location_descriptor(tile.image) → descriptor
|
||||
- Store descriptor with tile metadata
|
||||
- Build Faiss index
|
||||
4. Persist index to disk for fast startup
|
||||
1. Load pre-built Faiss index from index_path
|
||||
2. Load tile metadata (tile_id → gps_center, bounds mapping)
|
||||
3. Validate index integrity (check descriptor dimensions, tile count)
|
||||
4. Return True if loaded successfully
|
||||
|
||||
**Satellite Provider Integration**:
|
||||
- **Primary**: Satellite provider builds semantic index offline
|
||||
- Provider exposes index via API or file download
|
||||
- F08 loads pre-built index at startup
|
||||
- **Fallback**: F08 can build index locally if provider doesn't supply it
|
||||
**Satellite Provider Responsibility**:
|
||||
- Satellite provider builds the semantic index offline using DINOv2 + VLAD
|
||||
- Provider delivers index file along with satellite tiles
|
||||
- Index format: Faiss IVF or HNSW index + tile metadata JSON
|
||||
- Provider is responsible for index updates when satellite data changes
|
||||
|
||||
**Error Conditions**:
|
||||
- Raises `IndexNotFoundError`: Index file not found
|
||||
- Raises `IndexCorruptedError`: Index file corrupted or invalid format
|
||||
- Raises `MetadataMismatchError`: Metadata doesn't match index
|
||||
|
||||
**Performance**:
|
||||
- **Load pre-built index**: <10 seconds (fast startup)
|
||||
- **Build index locally**: ~10-30 minutes for 10,000 tiles (fallback only)
|
||||
- **Load time**: <10 seconds for 10,000+ tiles
|
||||
|
||||
**Test Cases**:
|
||||
1. **Load pre-built index**: Completes successfully, fast startup
|
||||
2. **Fallback local building**: Builds index if provider doesn't supply it
|
||||
3. **Index query**: Works correctly after loading
|
||||
1. **Load valid index**: Completes successfully, index operational
|
||||
2. **Index not found**: Raises IndexNotFoundError
|
||||
3. **Corrupted index**: Raises IndexCorruptedError
|
||||
4. **Index query after load**: Works correctly
|
||||
|
||||
---
|
||||
|
||||
@@ -351,10 +345,10 @@ np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
|
||||
2. UAV images from autumn
|
||||
3. retrieve_candidate_tiles() → correct match despite appearance change
|
||||
|
||||
### Test 3: Database Initialization
|
||||
1. Prepare 500 satellite tiles
|
||||
2. initialize_database(tiles)
|
||||
3. Verify Faiss index built
|
||||
### Test 3: Index Loading
|
||||
1. Prepare pre-built index file from satellite provider
|
||||
2. load_index(index_path)
|
||||
3. Verify Faiss index loaded correctly
|
||||
4. Query with test image → returns matches
|
||||
|
||||
### Test 4: Chunk Semantic Matching
|
||||
|
||||
@@ -377,11 +377,11 @@ Optional[np.ndarray]: 3×3 homography matrix or None
|
||||
## Dependencies
|
||||
|
||||
### Internal Components
|
||||
- **F12 Route Chunk Manager**: For chunk image retrieval and chunk operations
|
||||
- **F16 Model Manager**: For LiteSAM model
|
||||
- **H01 Camera Model**: For projection operations
|
||||
- **H02 GSD Calculator**: For coordinate transformations
|
||||
- **H05 Performance Monitor**: For timing
|
||||
- **F12 Route Chunk Manager**: For chunk image retrieval
|
||||
|
||||
**Note**: tile_bounds is passed as parameter from caller (F02 Flight Processor gets it from F04 Satellite Data Manager)
|
||||
|
||||
|
||||
@@ -8,64 +8,67 @@
|
||||
|
||||
```python
|
||||
class IFactorGraphOptimizer(ABC):
|
||||
# All methods take flight_id to support concurrent flights
|
||||
# F10 maintains Dict[str, FactorGraph] keyed by flight_id internally
|
||||
|
||||
@abstractmethod
|
||||
def add_relative_factor(self, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
|
||||
def add_relative_factor(self, flight_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def add_absolute_factor(self, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool:
|
||||
def add_absolute_factor(self, flight_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray, is_user_anchor: bool) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def add_altitude_prior(self, frame_id: int, altitude: float, covariance: float) -> bool:
|
||||
def add_altitude_prior(self, flight_id: str, frame_id: int, altitude: float, covariance: float) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def optimize(self, iterations: int) -> OptimizationResult:
|
||||
def optimize(self, flight_id: str, iterations: int) -> OptimizationResult:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_trajectory(self) -> Dict[int, Pose]:
|
||||
def get_trajectory(self, flight_id: str) -> Dict[int, Pose]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_marginal_covariance(self, frame_id: int) -> np.ndarray:
|
||||
def get_marginal_covariance(self, flight_id: str, frame_id: int) -> np.ndarray:
|
||||
pass
|
||||
|
||||
# Chunk operations - F10 only manages factor graph subgraphs
|
||||
# F12 owns chunk metadata (status, is_active, etc.)
|
||||
@abstractmethod
|
||||
def create_chunk_subgraph(self, flight_id: str, chunk_id: str, start_frame_id: int) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def create_new_chunk(self, chunk_id: str, start_frame_id: int) -> ChunkHandle:
|
||||
def add_relative_factor_to_chunk(self, flight_id: str, chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_chunk_for_frame(self, frame_id: int) -> Optional[ChunkHandle]:
|
||||
def add_chunk_anchor(self, flight_id: str, chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def add_relative_factor_to_chunk(self, chunk_id: str, frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool:
|
||||
def merge_chunk_subgraphs(self, flight_id: str, source_chunk_id: str, target_chunk_id: str, transform: Sim3Transform) -> bool:
|
||||
"""Merges source_chunk INTO target_chunk. Source chunk subgraph is merged into target."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def add_chunk_anchor(self, chunk_id: str, frame_id: int, gps: GPSPoint, covariance: np.ndarray) -> bool:
|
||||
def get_chunk_trajectory(self, flight_id: str, chunk_id: str) -> Dict[int, Pose]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool:
|
||||
def optimize_chunk(self, flight_id: str, chunk_id: str, iterations: int) -> OptimizationResult:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_chunk_trajectory(self, chunk_id: str) -> Dict[int, Pose]:
|
||||
def optimize_global(self, flight_id: str, iterations: int) -> OptimizationResult:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_all_chunks(self) -> List[ChunkHandle]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def optimize_chunk(self, chunk_id: str, iterations: int) -> OptimizationResult:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def optimize_global(self, iterations: int) -> OptimizationResult:
|
||||
def delete_flight_graph(self, flight_id: str) -> bool:
|
||||
"""Cleanup factor graph when flight is deleted."""
|
||||
pass
|
||||
```
|
||||
|
||||
@@ -78,10 +81,32 @@ class IFactorGraphOptimizer(ABC):
|
||||
- Scale resolution through altitude priors and absolute GPS
|
||||
- Trajectory smoothing and global consistency
|
||||
- Back-propagation of refinements to previous frames
|
||||
- **Native multi-chunk/multi-map support (Atlas architecture)**
|
||||
- **Chunk lifecycle management (creation, optimization, merging)**
|
||||
- **Low-level factor graph chunk operations** (subgraph creation, factor addition, optimization)
|
||||
- **Sim(3) transformation for chunk merging**
|
||||
|
||||
### Chunk Responsibility Clarification
|
||||
|
||||
**F10 provides low-level factor graph operations only**:
|
||||
- `create_chunk_subgraph()`: Creates subgraph in factor graph (returns bool, not ChunkHandle)
|
||||
- `add_relative_factor_to_chunk()`: Adds factors to chunk's subgraph
|
||||
- `add_chunk_anchor()`: Adds GPS anchor to chunk's subgraph
|
||||
- `merge_chunk_subgraphs()`: Applies Sim(3) transform and merges subgraphs
|
||||
- `optimize_chunk()`, `optimize_global()`: Runs optimization
|
||||
|
||||
**F10 does NOT own chunk metadata** - only factor graph data structures.
|
||||
|
||||
**F12 is the source of truth for ALL chunk state** (see F12 spec):
|
||||
- ChunkHandle with all metadata (is_active, has_anchor, matching_status)
|
||||
- Chunk lifecycle management
|
||||
- Chunk readiness determination
|
||||
- High-level chunk queries
|
||||
- F12 calls F10 for factor graph operations
|
||||
|
||||
**F11 coordinates recovery** (see F11 spec):
|
||||
- Triggers chunk creation via F12
|
||||
- Coordinates matching workflows
|
||||
- Emits chunk-related events
|
||||
|
||||
### Scope
|
||||
- Non-linear least squares optimization
|
||||
- Factor graph representation of SLAM problem
|
||||
@@ -92,6 +117,20 @@ class IFactorGraphOptimizer(ABC):
|
||||
- **Chunk-level optimization and global merging**
|
||||
- **Sim(3) similarity transformation for chunk alignment**
|
||||
|
||||
### Design Pattern: Composition Over Complex Interface
|
||||
|
||||
F10 uses **composition** to keep the interface manageable. Rather than exposing 20+ methods in a monolithic interface, complex operations are composed from simpler primitives:
|
||||
|
||||
**Primitive Operations**:
|
||||
- `add_relative_factor()`, `add_absolute_factor()`, `add_altitude_prior()` - Factor management
|
||||
- `optimize()`, `get_trajectory()`, `get_marginal_covariance()` - Core optimization
|
||||
|
||||
**Chunk Operations** (composed from primitives):
|
||||
- `create_new_chunk()`, `add_relative_factor_to_chunk()`, `add_chunk_anchor()` - Chunk factor management
|
||||
- `merge_chunks()`, `optimize_chunk()`, `optimize_global()` - Chunk optimization
|
||||
|
||||
**Callers compose these primitives** for complex workflows (e.g., F11 composes anchor + merge + optimize_global).
|
||||
|
||||
## API Methods
|
||||
|
||||
### `add_relative_factor(frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool`
|
||||
@@ -123,12 +162,20 @@ F07 returns unit translation vectors due to monocular scale ambiguity. F10 resol
|
||||
**Explicit Flow**:
|
||||
```python
|
||||
# In add_relative_factor():
|
||||
gsd = H02.compute_gsd(altitude, focal_length, sensor_width, image_width)
|
||||
expected_displacement = frame_spacing * gsd # ~100m
|
||||
# altitude comes from F17 Configuration Manager (predefined operational altitude)
|
||||
# focal_length, sensor_width from F17 Configuration Manager
|
||||
config = F17.get_flight_config(flight_id)
|
||||
altitude = config.altitude # Predefined altitude, NOT from EXIF
|
||||
gsd = H02.compute_gsd(altitude, config.camera_params.focal_length,
|
||||
config.camera_params.sensor_width,
|
||||
config.camera_params.resolution_width)
|
||||
expected_displacement = frame_spacing * gsd # ~100m typical at 300m altitude
|
||||
scaled_translation = relative_pose.translation * expected_displacement
|
||||
# Add scaled_translation to factor graph
|
||||
```
|
||||
|
||||
**Note**: Altitude comes from F17 Configuration Manager (predefined operational altitude), NOT from EXIF metadata. The problem statement specifies images don't have GPS metadata.
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
bool: True if factor added successfully
|
||||
|
||||
+97
-15
@@ -57,7 +57,7 @@ class IFailureRecoveryCoordinator(ABC):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def merge_chunk_to_trajectory(self, chunk_id: str, alignment_result: ChunkAlignmentResult) -> bool:
|
||||
def merge_chunk_to_trajectory(self, flight_id: str, chunk_id: str, alignment_result: ChunkAlignmentResult) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
@@ -72,14 +72,64 @@ class IFailureRecoveryCoordinator(ABC):
|
||||
- Detect tracking loss and trigger recovery
|
||||
- Coordinate progressive tile search (1→4→9→16→25)
|
||||
- Handle human-in-the-loop when all strategies exhausted
|
||||
- Block flight processing when awaiting user input
|
||||
- **Emit recovery events** (RecoveryStarted, RecoveryFailed, RecoverySucceeded, UserInputNeeded)
|
||||
- Apply user-provided anchors to Factor Graph
|
||||
- **Proactive chunk creation on tracking loss**
|
||||
- **Chunk semantic matching coordination**
|
||||
- **Proactive chunk creation on tracking loss** (via F12)
|
||||
- **Chunk LiteSAM matching with rotation sweeps**
|
||||
- **Chunk merging orchestration**
|
||||
- **Background chunk matching processing**
|
||||
|
||||
### Chunk Responsibility Clarification
|
||||
|
||||
**F11 coordinates chunk-based recovery workflow only**:
|
||||
- Detects when to create chunks (on tracking loss)
|
||||
- Calls F12 for ALL chunk operations (F11 NEVER directly calls F10 chunk methods)
|
||||
- Coordinates matching workflows
|
||||
- Emits chunk-related events (ChunkCreated, ChunkAnchored, ChunkMerged)
|
||||
|
||||
**F12 is the source of truth for chunk state** (see F12 spec)
|
||||
|
||||
**F10 provides low-level factor graph operations** (see F10 spec)
|
||||
|
||||
### Event-Based Communication
|
||||
|
||||
F11 emits events instead of directly calling F02's status update methods. This decouples recovery logic from flight state management.
|
||||
|
||||
### Internal Events (Component-to-Component)
|
||||
|
||||
F11 emits events for internal component communication. These are NOT directly sent to clients.
|
||||
|
||||
**Events Emitted (internal)**:
|
||||
- `RecoveryStarted`: When tracking loss detected and recovery begins
|
||||
- `RecoverySucceeded`: When recovery finds a match (single-image or chunk)
|
||||
- `RecoveryFailed`: When all recovery strategies exhausted
|
||||
- `UserInputNeeded`: When user input is required
|
||||
- `UserFixApplied`: When user-provided anchor successfully applied
|
||||
- `ChunkCreated`: When new chunk created on tracking loss
|
||||
- `ChunkAnchored`: When chunk successfully matched and anchored
|
||||
- `ChunkMerged`: When chunk merged into main trajectory (includes flight_id, chunk_id, merged_frames)
|
||||
- `ChunkMatchingFailed`: When chunk matching exhausts all candidates and fails
|
||||
|
||||
**Event Listeners** (F02 Flight Processor subscribes to these):
|
||||
- On `RecoveryStarted`: Update status to "recovering"
|
||||
- On `RecoveryFailed`: Update status to "blocked"
|
||||
- On `RecoverySucceeded`: Update status to "processing"
|
||||
- On `UserInputNeeded`: Update status to "blocked", blocked=True
|
||||
- On `ChunkMatchingFailed`: Re-queue chunk for user input or continue building
|
||||
|
||||
### External SSE Events (to Clients)
|
||||
|
||||
F11 does NOT directly send events to clients. External events are routed through F14 Result Manager which manages SSE streaming via F15:
|
||||
|
||||
- When F11 emits `UserInputNeeded` → F02 receives → F02 calls F14.publish_user_input_request() → F14 sends via F15 SSE
|
||||
- When F11 emits `ChunkMerged` → F02 receives → F02 calls F14.update_results_after_chunk_merge() → F14 sends via F15 SSE
|
||||
- When F11 emits `RecoveryFailed` → F02 receives → F02 calls F14.publish_processing_blocked() → F14 sends via F15 SSE
|
||||
|
||||
This separation ensures:
|
||||
1. F11 is decoupled from SSE implementation
|
||||
2. F14 controls all client-facing communication
|
||||
3. Consistent event format for clients
|
||||
|
||||
### Scope
|
||||
- Confidence monitoring
|
||||
- Progressive search coordination
|
||||
@@ -298,10 +348,10 @@ UserInputRequest:
|
||||
|
||||
**Processing Flow**:
|
||||
1. Get UAV image for frame_id
|
||||
2. Get top-5 candidates from G08
|
||||
2. Get top-5 candidates from F08
|
||||
3. Create request
|
||||
4. Send via F14 SSE → "user_input_needed" event
|
||||
5. Update F02 flight_status("BLOCKED")
|
||||
4. Send via F15 SSE → "user_input_needed" event
|
||||
5. Emit `UserInputNeeded` event (F02 subscribes and updates status to "BLOCKED")
|
||||
|
||||
**Test Cases**:
|
||||
1. All search failed → creates request
|
||||
@@ -330,7 +380,7 @@ anchor: UserAnchor:
|
||||
1. Validate anchor data
|
||||
2. Call F10.add_absolute_factor(frame_id, gps, is_user_anchor=True)
|
||||
3. F10.optimize() → refines trajectory
|
||||
4. Update F02 flight_status("PROCESSING")
|
||||
4. Emit `UserFixApplied` event (F02 subscribes and updates status to "PROCESSING")
|
||||
5. Resume processing from next frame
|
||||
|
||||
**Test Cases**:
|
||||
@@ -482,15 +532,20 @@ bool: True if merge successful
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Get chunk frames via F12.get_chunk_frames(chunk_id) → merged_frames (all frames in chunk that will be updated)
|
||||
1. Get chunk frames via F12.get_chunk_frames(chunk_id) → merged_frames
|
||||
2. Get chunk anchor frame (middle frame or best frame)
|
||||
3. Call F12.mark_chunk_anchored() with GPS (F12 coordinates with F10)
|
||||
4. Find target chunk (previous chunk or main trajectory)
|
||||
5. Call F12.merge_chunks(chunk_id, target_chunk_id, transform) (F12 coordinates with F10)
|
||||
4. **Determine merge target**:
|
||||
- Target is typically the temporal predecessor (previous chunk by frame_id order)
|
||||
- If no predecessor: merge to main trajectory (target_chunk_id="main")
|
||||
- F11 determines target based on chunk frame_id ordering
|
||||
5. Call F12.merge_chunks(target_chunk_id, chunk_id, transform)
|
||||
- Note: `merge_chunks(target, source)` merges source INTO target
|
||||
- chunk_id (source) is merged into target_chunk_id
|
||||
6. F12 handles chunk state updates (deactivation, status updates)
|
||||
7. F10 optimizes merged graph globally (via F12.merge_chunks())
|
||||
8. **Update results**: Call F14 Result Manager.update_results_after_chunk_merge(flight_id, merged_frames)
|
||||
- F14 retrieves updated poses from F10, converts ENU to GPS, updates database, and publishes via SSE
|
||||
8. **Emit ChunkMerged event** with flight_id and merged_frames
|
||||
- F14 subscribes and updates results: retrieves poses from F10, converts ENU to GPS, updates database, publishes via SSE
|
||||
9. Return True
|
||||
|
||||
**Sim(3) Transform**:
|
||||
@@ -499,7 +554,7 @@ bool: True if merge successful
|
||||
- Scale: Resolved from altitude and GSD
|
||||
|
||||
**Test Cases**:
|
||||
1. **Merge chunk**: Chunk merged successfully, results updated via F14
|
||||
1. **Merge successful**: Chunks merged successfully, results updated via F14
|
||||
2. **Global consistency**: Merged trajectory globally consistent
|
||||
3. **Multiple chunks**: Can merge multiple chunks sequentially
|
||||
4. **Result updates**: All merged frames have updated GPS coordinates published
|
||||
@@ -544,10 +599,25 @@ while flight_active:
|
||||
- Reduces user input requests
|
||||
- **Lifecycle**: Starts when flight becomes active, stops when flight completed
|
||||
|
||||
**Chunk Failure Flow**:
|
||||
When chunk matching fails after trying all candidates:
|
||||
1. Emit `ChunkMatchingFailed` event with chunk_id and flight_id
|
||||
2. F02 receives event and decides next action:
|
||||
- **Option A**: Wait for more frames and retry later (chunk may gain more distinctive features)
|
||||
- **Option B**: Create user input request for the chunk
|
||||
3. If user input requested:
|
||||
- F02 calls F14.publish_user_input_request() with chunk context
|
||||
- Client receives via SSE
|
||||
- User provides GPS anchor for one frame in chunk
|
||||
- F02 receives user fix via handle_user_fix()
|
||||
- F11.apply_user_anchor() anchors the chunk
|
||||
- Processing resumes
|
||||
|
||||
**Test Cases**:
|
||||
1. **Background matching**: Unanchored chunks matched asynchronously
|
||||
2. **Chunk merging**: Chunks merged when matches found
|
||||
3. **Non-blocking**: Frame processing continues during matching
|
||||
4. **Chunk matching fails**: ChunkMatchingFailed emitted, user input requested if needed
|
||||
|
||||
## Integration Tests
|
||||
|
||||
@@ -622,9 +692,10 @@ while flight_active:
|
||||
- F10 Factor Graph Optimizer (anchor application and chunk merging)
|
||||
- F12 Route Chunk Manager (chunk lifecycle)
|
||||
- F14 Result Manager (result updates after chunk merging)
|
||||
- F02 Flight Manager (status updates)
|
||||
- F15 SSE Event Streamer (user input events)
|
||||
|
||||
**Note**: F11 does NOT directly call F02. Instead, F11 emits events (RecoveryStarted, RecoveryFailed, etc.) which F02 subscribes to for status updates. This decouples recovery logic from flight state management.
|
||||
|
||||
### External Dependencies
|
||||
- None
|
||||
|
||||
@@ -708,3 +779,14 @@ class Sim3Transform(BaseModel):
|
||||
scale: float
|
||||
```
|
||||
|
||||
### RecoveryEvent
|
||||
```python
|
||||
class RecoveryEvent(BaseModel):
|
||||
event_type: str # "RecoveryStarted", "RecoverySucceeded", "RecoveryFailed", "UserInputNeeded", "ChunkCreated", "ChunkAnchored", "ChunkMerged"
|
||||
flight_id: str
|
||||
frame_id: Optional[int]
|
||||
chunk_id: Optional[str]
|
||||
data: Optional[Dict[str, Any]]
|
||||
timestamp: datetime
|
||||
```
|
||||
|
||||
|
||||
@@ -53,22 +53,50 @@ class IRouteChunkManager(ABC):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool:
|
||||
def merge_chunks(self, target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool:
|
||||
"""Merges source_chunk INTO target_chunk. Result is stored in target_chunk."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def mark_chunk_matching(self, chunk_id: str) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def save_chunk_state(self, flight_id: str) -> bool:
|
||||
"""Persist all chunk state to F03 for crash recovery."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def load_chunk_state(self, flight_id: str) -> bool:
|
||||
"""Load chunk state from F03 on restart."""
|
||||
pass
|
||||
```
|
||||
|
||||
## Component Description
|
||||
|
||||
### Responsibilities
|
||||
- **Source of truth for chunk state** (active, anchored, merged status)
|
||||
- Manage chunk lifecycle (creation, activation, deactivation, merging)
|
||||
- Track chunk state (frames, anchors, matching status)
|
||||
- Coordinate chunk semantic matching and LiteSAM matching
|
||||
- Provide chunk representations for matching (composite images, descriptors)
|
||||
- Determine chunk readiness for matching (min frames, consistency)
|
||||
- Persist chunk state via F03 Flight Database
|
||||
|
||||
### Chunk Responsibility Clarification
|
||||
|
||||
**F12 is the high-level chunk manager**:
|
||||
- Owns chunk state (ChunkHandle with matching_status, is_active, has_anchor)
|
||||
- Provides high-level queries: `get_active_chunk()`, `get_chunks_for_matching()`
|
||||
- Coordinates with F10 for factor graph operations
|
||||
- Persists chunk state for crash recovery
|
||||
|
||||
**F10 provides low-level factor graph operations** (see F10 spec):
|
||||
- Subgraph creation and factor management
|
||||
- Does NOT own chunk state - only factor graph data
|
||||
|
||||
**F11 coordinates recovery** (see F11 spec):
|
||||
- Calls F12 for chunk operations
|
||||
- F11 NEVER directly calls F10 chunk methods
|
||||
|
||||
### Scope
|
||||
- Chunk lifecycle management
|
||||
@@ -436,17 +464,17 @@ bool: True if deactivated successfully
|
||||
|
||||
---
|
||||
|
||||
### `merge_chunks(chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool`
|
||||
### `merge_chunks(target_chunk_id: str, source_chunk_id: str, transform: Sim3Transform) -> bool`
|
||||
|
||||
**Description**: Coordinates chunk merging by validating chunks, calling F10 for factor graph merge, and updating chunk states.
|
||||
**Description**: Merges source_chunk INTO target_chunk. The resulting merged chunk is target_chunk. Source chunk is deactivated after merge.
|
||||
|
||||
**Called By**:
|
||||
- F11 Failure Recovery Coordinator (after successful chunk matching)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
chunk_id_1: str # Source chunk (typically newer, to be merged)
|
||||
chunk_id_2: str # Target chunk (typically older, merged into)
|
||||
target_chunk_id: str # Target chunk (receives the merge, typically older/main)
|
||||
source_chunk_id: str # Source chunk (being merged in, typically newer)
|
||||
transform: Sim3Transform:
|
||||
translation: np.ndarray # (3,)
|
||||
rotation: np.ndarray # (3, 3) or quaternion
|
||||
@@ -460,34 +488,33 @@ bool: True if merge successful
|
||||
|
||||
**Processing Flow**:
|
||||
1. Verify both chunks exist
|
||||
2. Verify chunk_id_1 is anchored (has_anchor=True)
|
||||
2. Verify source_chunk_id is anchored (has_anchor=True)
|
||||
3. Validate chunks can be merged (not already merged, not same chunk)
|
||||
4. **Merge direction**: chunk_id_1 (newer, source) merges INTO chunk_id_2 (older, target)
|
||||
5. Call F10.merge_chunks(chunk_id_1, chunk_id_2, transform)
|
||||
6. Update chunk_id_1 state:
|
||||
4. Call F10.merge_chunk_subgraphs(flight_id, source_chunk_id, target_chunk_id, transform)
|
||||
5. Update source_chunk_id state:
|
||||
- Set is_active=False
|
||||
- Set matching_status="merged"
|
||||
- Call deactivate_chunk(chunk_id_1)
|
||||
7. Update chunk_id_2 state (if needed)
|
||||
8. Persist chunk state via F03 Flight Database.save_chunk_state()
|
||||
9. Return True
|
||||
- Call deactivate_chunk(source_chunk_id)
|
||||
6. target_chunk remains active (now contains merged frames)
|
||||
7. Persist chunk state via F03 Flight Database.save_chunk_state()
|
||||
8. Return True
|
||||
|
||||
**Merge Convention**:
|
||||
- `merge_chunks(target, source)` → source is merged INTO target
|
||||
- Result is stored in target_chunk
|
||||
- source_chunk is deactivated after merge
|
||||
- Example: `merge_chunks("main", "chunk_3")` merges chunk_3 into main trajectory
|
||||
|
||||
**Validation**:
|
||||
- Both chunks must exist
|
||||
- chunk_id_1 must be anchored
|
||||
- chunk_id_1 must not already be merged
|
||||
- chunk_id_1 and chunk_id_2 must be different
|
||||
|
||||
**Merge Direction**:
|
||||
- **chunk_id_1**: Source chunk (newer, recently anchored)
|
||||
- **chunk_id_2**: Target chunk (older, main trajectory or previous chunk)
|
||||
- Newer chunks merge INTO older chunks to maintain chronological consistency
|
||||
- source_chunk must be anchored
|
||||
- source_chunk must not already be merged
|
||||
- target_chunk and source_chunk must be different
|
||||
|
||||
**Test Cases**:
|
||||
1. **Merge anchored chunks**: Chunks merged successfully, chunk_id_1 deactivated
|
||||
2. **Merge unanchored chunk**: Returns False (validation fails)
|
||||
3. **Merge already merged chunk**: Returns False (validation fails)
|
||||
4. **State updates**: chunk_id_1 marked as merged and deactivated
|
||||
1. **Merge anchored chunk**: source_chunk merged into target_chunk
|
||||
2. **Source deactivated**: source_chunk marked as merged and deactivated
|
||||
3. **Target unchanged**: target_chunk remains active with new frames
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -10,19 +10,19 @@
|
||||
class ICoordinateTransformer(ABC):
|
||||
# ENU Origin Management
|
||||
@abstractmethod
|
||||
def set_enu_origin(self, origin_gps: GPSPoint) -> None:
|
||||
def set_enu_origin(self, flight_id: str, origin_gps: GPSPoint) -> None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_enu_origin(self) -> GPSPoint:
|
||||
def get_enu_origin(self, flight_id: str) -> GPSPoint:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def gps_to_enu(self, gps: GPSPoint) -> Tuple[float, float, float]:
|
||||
def gps_to_enu(self, flight_id: str, gps: GPSPoint) -> Tuple[float, float, float]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def enu_to_gps(self, enu: Tuple[float, float, float]) -> GPSPoint:
|
||||
def enu_to_gps(self, flight_id: str, enu: Tuple[float, float, float]) -> GPSPoint:
|
||||
pass
|
||||
|
||||
# Pixel/GPS Conversions
|
||||
@@ -35,11 +35,7 @@ class ICoordinateTransformer(ABC):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def image_object_to_gps(self, object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def compute_gsd(self, altitude: float, focal_length: float, sensor_width: float, image_width: int) -> float:
|
||||
def image_object_to_gps(self, flight_id: str, frame_id: int, object_pixel: Tuple[float, float]) -> GPSPoint:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
@@ -53,10 +49,11 @@ class ICoordinateTransformer(ABC):
|
||||
- Pixel-to-GPS coordinate conversions
|
||||
- GPS-to-pixel inverse projections
|
||||
- **Critical**: Convert object pixel coordinates (from external detection system) to GPS
|
||||
- Ground Sampling Distance (GSD) calculations
|
||||
- Handle multiple coordinate systems: WGS84, Web Mercator, ENU, image pixels, rotated coordinates
|
||||
- Camera model integration for projection operations
|
||||
|
||||
**Note**: GSD calculations are delegated to H02 GSD Calculator helper.
|
||||
|
||||
### Scope
|
||||
- Coordinate system transformations
|
||||
- Camera projection mathematics
|
||||
@@ -276,18 +273,19 @@ Tuple[float, float]: (x, y) pixel coordinates
|
||||
|
||||
---
|
||||
|
||||
### `image_object_to_gps(object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint`
|
||||
### `image_object_to_gps(flight_id: str, frame_id: int, object_pixel: Tuple[float, float]) -> GPSPoint`
|
||||
|
||||
**Description**: **Critical method** - Converts object pixel coordinates to GPS. Used by external object detection system.
|
||||
**Description**: **Critical method** - Converts object pixel coordinates to GPS. Used for external object detection integration.
|
||||
|
||||
**Called By**:
|
||||
- External object detection system (provides pixel coordinates)
|
||||
- F13 Result Manager (converts objects to GPS for output)
|
||||
- F02 Flight Processor (via convert_object_to_gps delegation from F01)
|
||||
- F14 Result Manager (converts objects to GPS for output)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
object_pixel: Tuple[float, float] # Pixel coordinates from object detector
|
||||
flight_id: str # Flight identifier (needed for ENU origin and factor graph)
|
||||
frame_id: int # Frame containing object
|
||||
object_pixel: Tuple[float, float] # Pixel coordinates from object detector
|
||||
```
|
||||
|
||||
**Output**:
|
||||
@@ -296,15 +294,17 @@ GPSPoint: GPS coordinates of object center
|
||||
```
|
||||
|
||||
**Processing Flow**:
|
||||
1. Get frame_pose from F10 Factor Graph
|
||||
2. Get camera_params from F17 Configuration Manager
|
||||
3. Get altitude from configuration
|
||||
1. Get frame_pose from F10 Factor Graph Optimizer.get_trajectory(flight_id)[frame_id]
|
||||
2. Get camera_params from F17 Configuration Manager.get_flight_config(flight_id)
|
||||
3. Get altitude from F17 Configuration Manager.get_flight_config(flight_id).altitude
|
||||
4. Call pixel_to_gps(object_pixel, frame_pose, camera_params, altitude)
|
||||
5. Return GPS
|
||||
5. Use enu_to_gps(flight_id, enu_point) for final GPS conversion
|
||||
6. Return GPS
|
||||
|
||||
**User Story**:
|
||||
- External system detects object in UAV image at pixel (1024, 768)
|
||||
- Calls image_object_to_gps(frame_id=237, object_pixel=(1024, 768))
|
||||
- Calls F02.convert_object_to_gps(flight_id="abc", frame_id=237, pixel=(1024, 768))
|
||||
- F02 delegates to F13.image_object_to_gps(flight_id="abc", frame_id=237, object_pixel=(1024, 768))
|
||||
- Returns GPSPoint(lat=48.123, lon=37.456)
|
||||
- Object GPS can be used for navigation, targeting, etc.
|
||||
|
||||
@@ -316,47 +316,6 @@ GPSPoint: GPS coordinates of object center
|
||||
|
||||
---
|
||||
|
||||
### `compute_gsd(altitude: float, focal_length: float, sensor_width: float, image_width: int) -> float`
|
||||
|
||||
**Description**: Computes Ground Sampling Distance (meters per pixel).
|
||||
|
||||
**Called By**:
|
||||
- Internal (for pixel_to_gps)
|
||||
- F09 Metric Refinement (for scale calculations)
|
||||
- H02 GSD Calculator (may delegate to)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
altitude: float # meters
|
||||
focal_length: float # mm
|
||||
sensor_width: float # mm
|
||||
image_width: int # pixels
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```python
|
||||
float: GSD in meters/pixel
|
||||
```
|
||||
|
||||
**Formula**:
|
||||
```
|
||||
GSD = (altitude * sensor_width) / (focal_length * image_width)
|
||||
```
|
||||
|
||||
**Example**:
|
||||
- altitude = 800m
|
||||
- focal_length = 24mm
|
||||
- sensor_width = 36mm
|
||||
- image_width = 6000px
|
||||
- GSD = (800 * 36) / (24 * 6000) = 0.2 m/pixel
|
||||
|
||||
**Test Cases**:
|
||||
1. **Standard parameters**: Returns reasonable GSD (~0.1-0.3 m/pixel)
|
||||
2. **Higher altitude**: GSD increases
|
||||
3. **Longer focal length**: GSD decreases
|
||||
|
||||
---
|
||||
|
||||
### `transform_points(points: List[Tuple[float, float]], transformation: np.ndarray) -> List[Tuple[float, float]]`
|
||||
|
||||
**Description**: Applies homography or affine transformation to list of points.
|
||||
@@ -406,8 +365,8 @@ List[Tuple[float, float]]: Transformed points
|
||||
3. pixel_to_gps() → GPS
|
||||
4. Verify GPS matches original (within tolerance)
|
||||
|
||||
### Test 4: GSD Calculation
|
||||
1. compute_gsd() with known parameters
|
||||
### Test 4: GSD via H02
|
||||
1. Call H02.compute_gsd() with known parameters
|
||||
2. Verify matches expected value
|
||||
3. Test at different altitudes
|
||||
|
||||
@@ -417,7 +376,6 @@ List[Tuple[float, float]]: Transformed points
|
||||
- **pixel_to_gps**: < 5ms
|
||||
- **gps_to_pixel**: < 5ms
|
||||
- **image_object_to_gps**: < 10ms
|
||||
- **compute_gsd**: < 1ms
|
||||
|
||||
### Accuracy
|
||||
- **GPS accuracy**: Inherits from Factor Graph accuracy (~20m)
|
||||
|
||||
@@ -27,6 +27,10 @@ class IResultManager(ABC):
|
||||
@abstractmethod
|
||||
def get_changed_frames(self, flight_id: str, since: datetime) -> List[int]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def update_results_after_chunk_merge(self, flight_id: str, merged_frames: List[int]) -> bool:
|
||||
pass
|
||||
```
|
||||
|
||||
## Component Description
|
||||
@@ -151,8 +155,9 @@ frame_ids: List[int] # Frames with updated poses
|
||||
|
||||
**Processing Flow**:
|
||||
1. For each frame_id:
|
||||
- Get refined pose from F10 Factor Graph Optimizer → ENU pose
|
||||
- Convert ENU pose to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_pose)
|
||||
- Get refined pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position
|
||||
- Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])`
|
||||
- Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint
|
||||
- Update result with refined=True via F03 Flight Database.save_frame_result()
|
||||
- Update waypoint via F03 Flight Database.update_waypoint()
|
||||
- Call F15 SSE Event Streamer.send_refinement()
|
||||
@@ -187,8 +192,9 @@ since: datetime
|
||||
|
||||
**Description**: Updates frame results after chunk merging changes frame poses.
|
||||
|
||||
**Called By**:
|
||||
- F11 Failure Recovery Coordinator (after chunk merging)
|
||||
**Triggered By**:
|
||||
- `ChunkMerged` event from F11 (F14 subscribes to this event)
|
||||
- Alternative: F02 Flight Processor can coordinate this call after receiving ChunkMerged event
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
@@ -200,8 +206,9 @@ merged_frames: List[int] # Frames whose poses changed due to chunk merge
|
||||
|
||||
**Processing Flow**:
|
||||
1. For each frame_id in merged_frames:
|
||||
- Get updated pose from F10 Factor Graph Optimizer.get_trajectory() → ENU pose
|
||||
- Convert ENU pose to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_pose)
|
||||
- Get updated pose from F10 Factor Graph Optimizer.get_trajectory(flight_id) → Pose object with ENU position
|
||||
- Extract ENU tuple: `enu_tuple = (pose.position[0], pose.position[1], pose.position[2])`
|
||||
- Convert ENU to GPS via F13 Coordinate Transformer.enu_to_gps(flight_id, enu_tuple) → GPSPoint
|
||||
- Update frame result via F03 Flight Database.save_frame_result()
|
||||
- Update waypoint via F03 Flight Database.update_waypoint()
|
||||
- Send refinement event via F15 SSE Event Streamer.send_refinement()
|
||||
|
||||
@@ -28,9 +28,19 @@ class ISSEEventStreamer(ABC):
|
||||
def send_refinement(self, flight_id: str, frame_id: int, updated_result: FrameResult) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def send_heartbeat(self, flight_id: str) -> bool:
|
||||
"""Sends heartbeat/keepalive to all clients subscribed to flight."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def close_stream(self, flight_id: str, client_id: str) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_active_connections(self, flight_id: str) -> int:
|
||||
"""Returns count of active SSE connections for a flight."""
|
||||
pass
|
||||
```
|
||||
|
||||
## Component Description
|
||||
@@ -170,12 +180,50 @@ StreamConnection:
|
||||
|
||||
---
|
||||
|
||||
### `send_heartbeat(flight_id: str) -> bool`
|
||||
|
||||
**Description**: Sends heartbeat/keepalive ping to all clients subscribed to a flight. Keeps connections alive and helps detect stale connections.
|
||||
|
||||
**Called By**:
|
||||
- Background heartbeat task (every 30 seconds)
|
||||
- F02 Flight Processor (periodically during processing)
|
||||
|
||||
**Event Format**:
|
||||
```
|
||||
:heartbeat
|
||||
```
|
||||
|
||||
**Behavior**:
|
||||
- Sends SSE comment (`:heartbeat`) which doesn't trigger event handlers
|
||||
- Keeps TCP connection alive
|
||||
- Client can use to detect connection health
|
||||
|
||||
**Test Cases**:
|
||||
1. Send heartbeat → all clients receive ping
|
||||
2. Client timeout → connection marked stale
|
||||
|
||||
---
|
||||
|
||||
### `close_stream(flight_id: str, client_id: str) -> bool`
|
||||
|
||||
**Description**: Closes SSE connection.
|
||||
|
||||
**Called By**: F01 REST API (on client disconnect)
|
||||
|
||||
---
|
||||
|
||||
### `get_active_connections(flight_id: str) -> int`
|
||||
|
||||
**Description**: Returns count of active SSE connections for a flight.
|
||||
|
||||
**Called By**:
|
||||
- F02 Flight Processor (monitoring)
|
||||
- Admin tools
|
||||
|
||||
**Test Cases**:
|
||||
1. No connections → returns 0
|
||||
2. 5 clients connected → returns 5
|
||||
|
||||
## Integration Tests
|
||||
|
||||
### Test 1: Real-Time Streaming
|
||||
|
||||
@@ -27,6 +27,21 @@ class IConfigurationManager(ABC):
|
||||
@abstractmethod
|
||||
def update_config(self, section: str, key: str, value: Any) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_operational_altitude(self, flight_id: str) -> float:
|
||||
"""Returns predefined operational altitude for the flight (NOT from EXIF)."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_frame_spacing(self, flight_id: str) -> float:
|
||||
"""Returns expected distance between consecutive frames in meters."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def save_flight_config(self, flight_id: str, config: FlightConfig) -> bool:
|
||||
"""Persists flight-specific configuration."""
|
||||
pass
|
||||
```
|
||||
|
||||
## Component Description
|
||||
@@ -136,6 +151,62 @@ FlightConfig:
|
||||
1. Update value → succeeds
|
||||
2. Invalid key → fails
|
||||
|
||||
---
|
||||
|
||||
### `get_operational_altitude(flight_id: str) -> float`
|
||||
|
||||
**Description**: Returns predefined operational altitude for the flight in meters. This is the altitude provided during flight creation, NOT extracted from EXIF metadata (images don't have GPS/altitude metadata per problem constraints).
|
||||
|
||||
**Called By**:
|
||||
- F10 Factor Graph Optimizer (for scale resolution)
|
||||
- H02 GSD Calculator (for GSD computation)
|
||||
- F09 Metric Refinement (for alignment)
|
||||
|
||||
**Input**: `flight_id: str`
|
||||
|
||||
**Output**: `float` - Altitude in meters (typically 100-500m)
|
||||
|
||||
**Test Cases**:
|
||||
1. Get existing flight altitude → returns value
|
||||
2. Non-existent flight → raises error
|
||||
|
||||
---
|
||||
|
||||
### `get_frame_spacing(flight_id: str) -> float`
|
||||
|
||||
**Description**: Returns expected distance between consecutive frames in meters. Used for scale estimation in visual odometry.
|
||||
|
||||
**Called By**:
|
||||
- F10 Factor Graph Optimizer (for expected displacement calculation)
|
||||
|
||||
**Input**: `flight_id: str`
|
||||
|
||||
**Output**: `float` - Expected frame spacing in meters (typically ~100m)
|
||||
|
||||
**Test Cases**:
|
||||
1. Get frame spacing → returns expected distance
|
||||
|
||||
---
|
||||
|
||||
### `save_flight_config(flight_id: str, config: FlightConfig) -> bool`
|
||||
|
||||
**Description**: Persists flight-specific configuration when a flight is created.
|
||||
|
||||
**Called By**:
|
||||
- F02 Flight Processor (during flight creation)
|
||||
|
||||
**Input**:
|
||||
```python
|
||||
flight_id: str
|
||||
config: FlightConfig
|
||||
```
|
||||
|
||||
**Output**: `bool` - True if saved successfully
|
||||
|
||||
**Test Cases**:
|
||||
1. Save valid config → succeeds
|
||||
2. Invalid flight_id → fails
|
||||
|
||||
## Data Models
|
||||
|
||||
### SystemConfig
|
||||
|
||||
@@ -1,355 +1,135 @@
|
||||
<mxfile host="65bd71144e">
|
||||
<diagram name="ASTRAL-Next Components" id="astral-next-components">
|
||||
<mxGraphModel dx="1440" dy="1201" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1800" pageHeight="2800" math="0" shadow="0">
|
||||
<mxGraphModel dx="771" dy="632" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="900" pageHeight="500" math="0" shadow="0">
|
||||
<root>
|
||||
<mxCell id="0"/>
|
||||
<mxCell id="1" parent="0"/>
|
||||
<mxCell id="title" value="ASTRAL-Next System Architecture GPS-Denied Localization for UAVs (Atlas Multi-Map Chunk Architecture) 25 Components: Route API (4) + Flight API (17) + Helpers (8)" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=20;fontStyle=1;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="200" y="20" width="1200" height="80" as="geometry"/>
|
||||
<mxCell id="title" value="ASTRAL-Next Component Connections" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="280" y="10" width="300" height="25" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="route-api-lane" value="Route API (Separate Project)" style="swimlane;horizontal=1;whiteSpace=wrap;html=1;fontSize=14;fontStyle=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="200" y="110" width="600" height="350" as="geometry"/>
|
||||
<mxCell id="client" value="Client" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="20" y="50" width="60" height="35" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r01" value="R01 Route REST API" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
|
||||
<mxGeometry x="40" y="50" width="120" height="60" as="geometry"/>
|
||||
<mxCell id="f01" value="F01 API" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="100" y="50" width="70" height="35" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r02" value="R02 Route Data Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
|
||||
<mxGeometry x="200" y="50" width="140" height="60" as="geometry"/>
|
||||
<mxCell id="f02" value="F02 Flight Processor" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;fontStyle=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="195" y="70" width="110" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r03" value="R03 Waypoint Validator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
|
||||
<mxGeometry x="380" y="50" width="140" height="60" as="geometry"/>
|
||||
<mxCell id="f03" value="F03 DB" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="340" y="50" width="70" height="35" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r04" value="R04 Route Database Layer" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
|
||||
<mxGeometry x="200" y="150" width="140" height="60" as="geometry"/>
|
||||
<mxCell id="f04" value="F04 Satellite Mgr" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="60" y="367.5" width="80" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r01-r02" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r01" target="r02" edge="1">
|
||||
<mxCell id="f05" value="F05 Image Pipeline" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="550" y="45" width="80" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="satellite" value="Satellite Provider" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFB300;fontColor=#ffffff;dashed=1;" vertex="1" parent="1">
|
||||
<mxGeometry x="660" y="45" width="70" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f15" value="F15 SSE" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="20" y="120" width="60" height="35" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f11" value="F11 Failure Recovery" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="310" y="120" width="110" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f06" value="F06 Rotation" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="100" y="195" width="70" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f07" value="F07 Sequential VO" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="200" y="195" width="90" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f08" value="F08 Place Recog" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="100" y="120" width="80" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f09" value="F09 Metric Refine" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="340" y="195" width="80" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f12" value="F12 Chunk Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="450" y="120" width="100" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f10" value="F10 Factor Graph" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;fontStyle=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="330" y="280" width="100" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f13" value="F13 Coord Transform" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="390" y="355" width="100" height="45" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f14" value="F14 Result Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="190" y="372.5" width="120" height="35" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="c1" value="HTTP" style="strokeColor=#FFFFFF;fontColor=#ffffff;fontSize=8;" edge="1" parent="1" source="client" target="f01">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r02-r03" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r02" target="r03" edge="1">
|
||||
<mxCell id="c2" value="SSE" style="strokeColor=#FFFFFF;fontColor=#ffffff;fontSize=8;dashed=1;" edge="1" parent="1" source="f15" target="client">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r02-r04" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r02" target="r04" edge="1">
|
||||
<mxCell id="c3" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f01" target="f02">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="route-db" value="Route DB (Separate Schema)" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
|
||||
<mxGeometry x="225" y="240" width="90" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="r04-db" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r04" target="route-db" edge="1">
|
||||
<mxCell id="c4" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f03">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="gps-denied-lane" value="Flight API (Main Processing System - Atlas Multi-Map Architecture)" style="swimlane;horizontal=1;whiteSpace=wrap;html=1;fontSize=14;fontStyle=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="130" y="500" width="1400" height="2400" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="core-layer" value="Core API Layer" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="560" height="140" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f01" value="F01 Flight REST API" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="core-layer" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="150" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f02" value="F02 Flight Processor (chunk-aware)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="core-layer" vertex="1">
|
||||
<mxGeometry x="200" y="40" width="150" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f03" value="F03 Flight Database" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="core-layer" vertex="1">
|
||||
<mxGeometry x="380" y="40" width="150" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f01-f02" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="core-layer" source="f01" target="f02" edge="1">
|
||||
<mxCell id="c5" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f11">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="data-layer" value="Data Management" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="600" y="40" width="560" height="140" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f04" value="F04 Satellite Data Manager (fetch, cache, grid)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="data-layer" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="160" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f05" value="F05 Image Input Pipeline (queue, validate, store)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="data-layer" vertex="1">
|
||||
<mxGeometry x="200" y="40" width="160" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f06" value="F06 Image Rotation Mgr (30° sweep, chunk rotation)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="data-layer" vertex="1">
|
||||
<mxGeometry x="380" y="40" width="160" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="visual-layer" value="Visual Processing (Tri-Layer Architecture)" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="20" y="200" width="560" height="140" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f07" value="F07 Sequential VO (SuperPoint+LightGlue chunk-scoped)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="visual-layer" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="160" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f08" value="F08 Global Place Recognition (AnyLoc DINOv2 chunk descriptors)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="visual-layer" vertex="1">
|
||||
<mxGeometry x="200" y="40" width="160" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f09" value="F09 Metric Refinement (LiteSAM chunk matching)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="visual-layer" vertex="1">
|
||||
<mxGeometry x="380" y="40" width="160" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="state-layer" value="State Estimation & Coordination (Atlas Multi-Map)" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="600" y="200" width="760" height="200" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f10" value="F10 Factor Graph Optimizer (GTSAM iSAM2 multi-chunk, Sim3)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="state-layer" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="160" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f11" value="F11 Failure Recovery (Progressive 1→25 chunk matching)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="state-layer" vertex="1">
|
||||
<mxGeometry x="200" y="40" width="160" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f12" value="F12 Route Chunk Manager (Atlas lifecycle chunk matching)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="state-layer" vertex="1">
|
||||
<mxGeometry x="380" y="40" width="160" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f13" value="F13 Coordinate Transform (Pixel↔GPS)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="state-layer" vertex="1">
|
||||
<mxGeometry x="560" y="40" width="160" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="results-layer" value="Results & Communication" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="20" y="360" width="380" height="140" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f14" value="F14 Result Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="results-layer" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="150" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f15" value="F15 SSE Event Streamer" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="results-layer" vertex="1">
|
||||
<mxGeometry x="200" y="40" width="150" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f14-f15" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="results-layer" source="f14" target="f15" edge="1">
|
||||
<mxCell id="c6" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f07">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="infra-layer" value="Infrastructure" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="420" y="360" width="740" height="140" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f16" value="F16 Model Manager (TensorRT/ONNX)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="infra-layer" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="160" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f17" value="F17 Configuration Mgr" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="infra-layer" vertex="1">
|
||||
<mxGeometry x="200" y="40" width="160" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="gps-db" value="GPS-Denied DB (Separate Schema) Flights, Frame Results" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="infra-layer" vertex="1">
|
||||
<mxGeometry x="570" y="30" width="140" height="90" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="helpers-lane" value="Helper Components (Shared Utilities)" style="swimlane;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="20" y="520" width="1140" height="160" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h01" value="H01 Camera Model" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="20" y="40" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h02" value="H02 GSD Calculator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="160" y="40" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h03" value="H03 Robust Kernels" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="300" y="40" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h04" value="H04 Faiss Index Mgr" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="440" y="40" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h05" value="H05 Performance Monitor" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="580" y="40" width="130" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h06" value="H06 Web Mercator Utils" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="730" y="40" width="130" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h07" value="H07 Image Rotation Utils" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="880" y="40" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="h08" value="H08 Batch Validator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
|
||||
<mxGeometry x="1020" y="40" width="100" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-title" value="Main Processing Flow" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="20" y="700" width="200" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-box" value="" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#1E1E1E;strokeColor=#FFFFFF;dashed=1;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="20" y="740" width="1340" height="800" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-1" value="1. Client uploads batch F01 → F05" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="760" width="160" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-2" value="2. Get next image F05 → F06" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="230" y="760" width="160" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-3" value="3. Rotation preprocessing F06 (30° sweep if needed)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="420" y="760" width="180" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-4" value="4. Sequential VO (chunk-aware) F07 → F12 (get active chunk) F07 → F10 (add to chunk subgraph)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="630" y="760" width="200" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-5" value="5. Check confidence F11 Failure Recovery" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="860" y="760" width="180" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-6a" value="6a. IF GOOD: LiteSAM (1 tile) F09 drift correction" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="880" width="200" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-6b" value="6b. IF LOST: Create chunk (proactive) F11 → F12 (create_chunk) F12 → F10 (create_new_chunk) Continue processing in chunk" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="270" y="880" width="240" height="100" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-6c" value="6c. Progressive search (single-image) F11 → F04 (1→4→9→16→25) F08 Global PR + F09 LiteSAM" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="540" y="880" width="220" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-7" value="7. Factor Graph optimize F10 (chunk optimization) Fuse VO + GPS in chunk" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="790" y="880" width="200" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-8" value="8. Coordinate transform F13 (Pixel → GPS)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="1020" y="880" width="180" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="flow-9" value="9. Publish results F14 → F03 (Route API) F14 → F15 (SSE to client)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="1230" y="880" width="180" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-1" target="flow-2" edge="1">
|
||||
<mxCell id="c7" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f10">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-2" target="flow-3" edge="1">
|
||||
<mxCell id="c10" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f11" target="f08">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-3-4" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-3" target="flow-4" edge="1">
|
||||
<mxCell id="c11" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f11" target="f09">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-4-5" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-4" target="flow-5" edge="1">
|
||||
<mxCell id="c12" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f11" target="f12">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-5-6a" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-5" target="flow-6a" edge="1">
|
||||
<mxCell id="c14" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f07" target="f10">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-5-6b" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-5" target="flow-6b" edge="1">
|
||||
<mxCell id="c15" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f09" target="f10">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-5-6c" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-5" target="flow-6c" edge="1">
|
||||
<mxCell id="c16" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f12" target="f10">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-6a-7" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-6a" target="flow-7" edge="1">
|
||||
<mxCell id="c17" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f10" target="f13">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-6b-7" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-6b" target="flow-7" edge="1">
|
||||
<mxCell id="c18" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f14" target="f13">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-6c-7" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-6c" target="flow-7" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-7-8" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-7" target="flow-8" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-8-9" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-8" target="flow-9" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="chunk-title" value="Chunk Matching (Background - Atlas Multi-Map)" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="1000" width="500" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="chunk-1" value="F11 (background) → F12 get_chunks_for_matching() (unanchored, ready)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="1040" width="220" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="chunk-2" value="F11 → F08 Chunk semantic matching (aggregate DINOv2)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="290" y="1040" width="220" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="chunk-3" value="F11 → F06 Chunk rotation sweeps (12 angles: 0°-330°)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="540" y="1040" width="220" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="chunk-4" value="F11 → F09 Chunk LiteSAM matching (aggregate correspondences)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="790" y="1040" width="240" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="chunk-5" value="F11 → F10 add_chunk_anchor() merge_chunks(Sim3)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="1060" y="1040" width="200" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-chunk-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-1" target="chunk-2" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-chunk-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-2" target="chunk-3" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-chunk-3-4" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-3" target="chunk-4" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-chunk-4-5" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-4" target="chunk-5" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="user-input-title" value="User Input Recovery (when all search fails)" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="1160" width="400" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="user-1" value="F11 exhausted (grid=25) → F15 send user_input_needed" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="1200" width="240" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="user-2" value="Client responds F01 → F11 apply_user_anchor" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="310" y="1200" width="240" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="user-3" value="F11 → F10 add_absolute_factor (high confidence) Processing resumes" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="580" y="1200" width="240" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-user-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="user-1" target="user-2" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-user-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="user-2" target="user-3" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="async-title" value="Asynchronous Trajectory Refinement" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="1310" width="350" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="async-1" value="F10 back-propagates new absolute factors (chunk + global optimization)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="40" y="1350" width="220" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="async-2" value="F14 detect changed frames → F03 batch_update_waypoints" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="290" y="1350" width="240" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="async-3" value="F14 → F15 send frame_refined events" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
|
||||
<mxGeometry x="560" y="1350" width="220" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-async-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="async-1" target="async-2" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="arrow-async-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="async-2" target="async-3" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="client" value="Client (GPS-Denied UI)" style="shape=actor;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="10" y="560" width="90" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="satellite-provider" value="Satellite Provider API /api/satellite/tiles/..." style="ellipse;shape=cloud;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="900" y="290" width="250" height="140" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="external-detector" value="External<br>Object Detector (Azaion.Inference)<br>(provides pixel coords)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#C62828;strokeColor=#EF5350;dashed=1;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="1340" y="800" width="160" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="client-f01" value="HTTP REST" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="client" target="f01" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f14-route" value="Per-frame waypoint updates" style="edgeStyle=orthogonalEdgeStyle;curved=1;dashed=1;strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f14" target="r01" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f04-sat" value="Fetch tiles" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f04" target="satellite-provider" edge="1">
|
||||
<mxGeometry x="0.3759" y="-10" relative="1" as="geometry">
|
||||
<mxPoint as="offset"/>
|
||||
<mxCell id="c19" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f14" target="f15">
|
||||
<mxGeometry relative="1" as="geometry">
|
||||
<Array as="points">
|
||||
<mxPoint x="80" y="270"/>
|
||||
</Array>
|
||||
</mxGeometry>
|
||||
</mxCell>
|
||||
<mxCell id="f15-client" value="SSE Events" style="dashed=1;strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f15" target="client" edge="1">
|
||||
<mxCell id="c13" style="strokeColor=#66BB6A;dashed=1;" edge="1" parent="1" source="f11" target="f14">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="detector-f13" value="Object pixels → GPS" style="dashed=1;strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="external-detector" target="f13" edge="1">
|
||||
<mxCell id="c21" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f05" target="satellite">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f12-f10" value="Chunk lifecycle" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f12" target="f10" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
<mxCell id="legend" value="── sync - - event" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=9;fontColor=#888888;" vertex="1" parent="1">
|
||||
<mxGeometry x="600" y="380" width="100" height="20" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="f12-f08" value="Chunk descriptors" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f12" target="f08" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
<mxCell id="3" value="" style="strokeColor=#FFFFFF;endArrow=none;" edge="1" parent="1" source="f11" target="f02">
|
||||
<mxGeometry relative="1" as="geometry">
|
||||
<mxPoint x="507.02127659574467" y="100.00000000000011" as="sourcePoint"/>
|
||||
<mxPoint x="256.3829787234042" y="289.9999999999999" as="targetPoint"/>
|
||||
</mxGeometry>
|
||||
</mxCell>
|
||||
<mxCell id="f11-f12" value="Chunk matching" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f11" target="f12" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-title" value="Legend" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="100" y="2730" width="100" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-box" value="" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#1E1E1E;strokeColor=#FFFFFF;" parent="1" vertex="1">
|
||||
<mxGeometry x="100" y="2770" width="700" height="120" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-1" value="Route API Components" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="120" y="2790" width="140" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-2" value="Flight API Core" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="280" y="2790" width="140" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-8" value="Chunk Management" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="600" y="2840" width="140" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-3" value="Visual Processing" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="440" y="2790" width="140" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-4" value="Helper Components" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="120" y="2840" width="140" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-5" value="State Estimation" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="280" y="2840" width="140" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-6" value="Infrastructure" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="1" vertex="1">
|
||||
<mxGeometry x="440" y="2840" width="140" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="legend-7" value="External Systems" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#C62828;strokeColor=#EF5350;fontColor=#ffffff;" vertex="1" parent="1">
|
||||
<mxGeometry x="600" y="2790" width="140" height="40" as="geometry"/>
|
||||
<mxCell id="2" value="" style="strokeColor=#FFFFFF;fontColor=#ffffff;endArrow=none;" edge="1" parent="1" source="f04" target="f14">
|
||||
<mxGeometry relative="1" as="geometry">
|
||||
<mxPoint x="715" y="121.22222222222206" as="sourcePoint"/>
|
||||
<mxPoint x="125" y="212.9999999999999" as="targetPoint"/>
|
||||
</mxGeometry>
|
||||
</mxCell>
|
||||
</root>
|
||||
</mxGraphModel>
|
||||
|
||||
@@ -101,7 +101,8 @@
|
||||
|
||||
**F13_coordinate_transformer**
|
||||
**Interface**: `ICoordinateTransformer`
|
||||
**API**: `set_enu_origin()`, `get_enu_origin()`, `gps_to_enu()`, `enu_to_gps()`, `pixel_to_gps()`, `gps_to_pixel()`, `image_object_to_gps()`, `compute_gsd()`, `transform_points()`
|
||||
**API**: `set_enu_origin()`, `get_enu_origin()`, `gps_to_enu()`, `enu_to_gps()`, `pixel_to_gps()`, `gps_to_pixel()`, `image_object_to_gps()`, `transform_points()`
|
||||
**Dependencies**: F10 Factor Graph Optimizer (for frame poses), H02 GSD Calculator (for GSD computation)
|
||||
|
||||
### Results & Communication
|
||||
|
||||
@@ -144,14 +145,14 @@
|
||||
|
||||
| Order | Component | Method | Purpose | Dependencies |
|
||||
|-------|-----------|--------|---------|--------------|
|
||||
| 1 | F16 Configuration Manager | `load_config()` | Load system configuration | None |
|
||||
| 2 | F03 Flight Database | Initialize connections | Establish DB connection pool | F16 |
|
||||
| 3 | F15 Model Manager | `load_model("SuperPoint")` | Load SuperPoint feature extractor | F16 |
|
||||
| 4 | F15 Model Manager | `load_model("LightGlue")` | Load LightGlue matcher | F16 |
|
||||
| 5 | F15 Model Manager | `load_model("DINOv2")` | Load DINOv2 for place recognition | F16 |
|
||||
| 6 | F15 Model Manager | `load_model("LiteSAM")` | Load LiteSAM for cross-view matching | F16 |
|
||||
| 7 | F04 Satellite Data Manager | Initialize cache | Initialize tile cache directory | F16 |
|
||||
| 8 | F08 Global Place Recognition | `build_index()` or `load_index()` | Build/load Faiss index from satellite tiles | F04, F15, H04 |
|
||||
| 1 | F17 Configuration Manager | `load_config()` | Load system configuration | None |
|
||||
| 2 | F03 Flight Database | Initialize connections | Establish DB connection pool | F17 |
|
||||
| 3 | F16 Model Manager | `load_model("SuperPoint")` | Load SuperPoint feature extractor | F17 |
|
||||
| 4 | F16 Model Manager | `load_model("LightGlue")` | Load LightGlue matcher | F17 |
|
||||
| 5 | F16 Model Manager | `load_model("DINOv2")` | Load DINOv2 for place recognition | F17 |
|
||||
| 6 | F16 Model Manager | `load_model("LiteSAM")` | Load LiteSAM for cross-view matching | F17 |
|
||||
| 7 | F04 Satellite Data Manager | Initialize cache | Initialize tile cache directory | F17 |
|
||||
| 8 | F08 Global Place Recognition | `load_index()` | Load pre-built Faiss index from satellite provider | F04, F16, H04 |
|
||||
| 9 | F12 Route Chunk Manager | Initialize | Initialize chunk state tracking | F10 |
|
||||
| 10 | F02 Flight Processor | Ready | Ready to accept flights | All above |
|
||||
| 11 | F01 Flight API | Start server | Start FastAPI/Uvicorn | F02 |
|
||||
@@ -176,7 +177,7 @@
|
||||
| Source | Target | Method | Purpose |
|
||||
|--------|--------|--------|---------|
|
||||
| F02 | F16 | `load_config()` | Load system configuration |
|
||||
| F02 | F15 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
|
||||
| F02 | F16 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
|
||||
| F04 | F08 | Satellite tiles | F08 generates descriptors for Faiss |
|
||||
| F08 | H04 | `build_index()` | Build satellite descriptor index |
|
||||
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model for descriptor generation |
|
||||
@@ -188,20 +189,27 @@
|
||||
| Client | F01 | `POST /flights` | Create flight |
|
||||
| F01 | F02 | `create_flight()` | Initialize flight state |
|
||||
| F02 | F16 | `get_flight_config()` | Get camera params, altitude |
|
||||
| F02 | F12 | `set_enu_origin(start_gps)` | Set ENU coordinate origin |
|
||||
| F02 | F13 | `set_enu_origin(flight_id, start_gps)` | Set ENU coordinate origin |
|
||||
| F02 | F04 | `prefetch_route_corridor()` | Prefetch tiles |
|
||||
| F04 | Satellite Provider | `GET /api/satellite/tiles/batch` | HTTP batch download |
|
||||
| F04 | H06 | `compute_tile_bounds()` | Tile coordinate calculations |
|
||||
| F02 | F03 | `insert_flight()` | Persist flight data |
|
||||
|
||||
### SSE Stream Creation
|
||||
|
||||
| Source | Target | Method | Purpose |
|
||||
|--------|--------|--------|---------|
|
||||
| Client | F01 | `GET .../stream` | Open SSE connection |
|
||||
| F01 | F14 | `create_stream()` | Establish SSE channel |
|
||||
| F01 | F02 | `create_client_stream()` | Route through coordinator |
|
||||
| F02 | F15 | `create_stream()` | Establish SSE channel |
|
||||
|
||||
### Image Upload
|
||||
|
||||
| Source | Target | Method | Purpose |
|
||||
|--------|--------|--------|---------|
|
||||
| Client | F01 | `POST .../images/batch` | Upload 10-50 images |
|
||||
| F01 | F05 | `queue_batch()` | Queue for processing |
|
||||
| F01 | F02 | `queue_images()` | Route through coordinator |
|
||||
| F02 | F05 | `queue_batch()` | Queue for processing |
|
||||
| F05 | H08 | `validate_batch()` | Validate sequence, format |
|
||||
| F05 | F03 | `save_image_metadata()` | Persist image metadata |
|
||||
|
||||
@@ -246,6 +254,7 @@
|
||||
| F02 | F11 | `check_confidence()` → FAIL | Low confidence |
|
||||
| F11 | F12 | `create_chunk_on_tracking_loss()` | **Proactive chunk creation** |
|
||||
| F12 | F10 | `create_new_chunk()` | Create chunk in factor graph |
|
||||
| F12 | F03 | `save_chunk_state()` | Persist chunk state for recovery |
|
||||
| F02 | F12 | `get_active_chunk()` | Get new active chunk |
|
||||
| F11 | F06 | `requires_rotation_sweep()` | Trigger rotation sweep (single-image) |
|
||||
| F11 | F08 | `retrieve_candidate_tiles()` | Coarse localization (single-image) |
|
||||
@@ -292,7 +301,7 @@
|
||||
| F01 | F11 | `apply_user_anchor()` | Apply fix |
|
||||
| F11 | F10 | `add_absolute_factor()` (high confidence) | Hard constraint |
|
||||
| F10 | Internal | `optimize()` | Re-optimize |
|
||||
| F11 | F02 | `update_flight_status("PROCESSING")` | Resume |
|
||||
| F11 | Event | Emit `UserFixApplied` | F02 subscribes and resumes |
|
||||
|
||||
### Asynchronous Refinement
|
||||
|
||||
@@ -331,7 +340,7 @@
|
||||
|
||||
## Interaction Coverage Verification
|
||||
|
||||
✅ **Initialization**: F02→F15, F16, F17; F04→F08→H04
|
||||
✅ **Initialization**: F02→F16, F17; F04→F08→H04
|
||||
✅ **Flight creation**: Client→F01→F02→F04,F12,F16,F17,F14
|
||||
✅ **Image upload**: Client→F01→F05→H08,F17
|
||||
✅ **Rotation sweep**: F06→H07,F09 (12 iterations)
|
||||
@@ -392,3 +401,80 @@
|
||||
- [x] F17 Configuration Manager
|
||||
- [x] F03 Flight Database (merged from R04, G17)
|
||||
- [x] Helper components (H01-H08)
|
||||
|
||||
---
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
### F02 Flight Processor Complexity
|
||||
|
||||
F02 Flight Processor handles multiple concerns:
|
||||
- Flight lifecycle management
|
||||
- Processing loop orchestration
|
||||
- Event subscription and state updates
|
||||
- Coordination of F07/F08/F09/F10
|
||||
|
||||
**Current Status**: Acceptable for MVP. The responsibilities are related (all flight processing) and the component acts as a coordinator rather than implementing logic directly.
|
||||
|
||||
**Future Consideration**: If complexity grows, consider splitting into:
|
||||
- F02a Flight State Manager (lifecycle, status)
|
||||
- F02b Processing Loop Coordinator (frame processing orchestration)
|
||||
|
||||
**Decision**: Keep as single component. Clear internal organization with separate methods for state vs processing concerns. Event-based communication with F11 keeps recovery logic separate.
|
||||
|
||||
### Error Recovery Strategy
|
||||
|
||||
**Per-Component Recovery**:
|
||||
- **F02**: Persists flight state via F03 on each significant update. On restart, loads last known state.
|
||||
- **F07**: Stateless - reprocesses frame if VO fails
|
||||
- **F10**: Factor graph state persisted periodically. On restart, rebuilds from F03 checkpoint.
|
||||
- **F11**: Chunk state persisted via F12→F03. Recovery continues from last chunk state.
|
||||
- **F12**: All chunk state persisted to F03. Restores chunk handles on restart.
|
||||
|
||||
**System-Wide Recovery**:
|
||||
1. On crash, F02 loads flight state from F03
|
||||
2. F12 restores chunk state from F03
|
||||
3. Processing resumes from last successfully processed frame
|
||||
4. Incomplete chunks continue building/matching
|
||||
|
||||
**Event Recovery**:
|
||||
- Events are fire-and-forget (no persistence)
|
||||
- Subscribers rebuild state from F03 on restart
|
||||
|
||||
### Error Propagation Strategy
|
||||
|
||||
**Principle**: Errors propagate upward through the component hierarchy. Lower-level components throw exceptions or return error results; higher-level coordinators handle recovery.
|
||||
|
||||
**Error Propagation Chain**:
|
||||
```
|
||||
H01-H08 (Helpers) → F07/F08/F09 (Visual Processing) → F11 (Recovery) → F02 (Coordinator) → F01 (API) → Client
|
||||
↓
|
||||
Events → F14 → F15 → SSE → Client
|
||||
```
|
||||
|
||||
**Error Categories**:
|
||||
1. **Recoverable** (F11 handles):
|
||||
- Tracking loss → Progressive search
|
||||
- Low confidence → Rotation sweep
|
||||
- Chunk matching fails → User input request
|
||||
|
||||
2. **Propagated to Client** (via SSE):
|
||||
- User input needed → `user_input_needed` event
|
||||
- Processing blocked → `processing_blocked` event
|
||||
- Flight completed → `flight_completed` event
|
||||
|
||||
3. **Fatal** (F02 handles, returns to F01):
|
||||
- Database connection lost → HTTP 503
|
||||
- Model loading failed → HTTP 500
|
||||
- Invalid configuration → HTTP 400
|
||||
|
||||
**User Fix Flow**:
|
||||
```
|
||||
Client ---> F01 (POST /user-fix) ---> F02.handle_user_fix() ---> F11.apply_user_anchor()
|
||||
↓
|
||||
F10.add_chunk_anchor()
|
||||
↓
|
||||
F02 receives UserFixApplied event
|
||||
↓
|
||||
F14.publish_result() ---> F15 ---> SSE ---> Client
|
||||
```
|
||||
|
||||
@@ -27,11 +27,25 @@ class IFaissIndexManager(ABC):
|
||||
@abstractmethod
|
||||
def load_index(self, path: str) -> FaissIndex:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def is_gpu_available(self) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def set_device(self, device: str) -> bool:
|
||||
"""Set device: 'gpu' or 'cpu'."""
|
||||
pass
|
||||
```
|
||||
|
||||
## Component Description
|
||||
|
||||
Manages Faiss indices for AnyLoc retrieval (IVF, HNSW options).
|
||||
Manages Faiss indices for DINOv2 descriptor similarity search. H04 builds indexes from UAV image descriptors for:
|
||||
1. **Loop closure detection**: Find when UAV revisits previously seen areas within the same flight
|
||||
2. **Chunk-to-chunk matching**: Match disconnected chunks to each other
|
||||
3. **Flight-to-flight matching**: Match current flight to previous flights in same area
|
||||
|
||||
**Index Source**: Descriptors are computed from UAV images using F08's DINOv2 encoder, NOT from satellite images. The index enables finding similar UAV viewpoints.
|
||||
|
||||
## API Methods
|
||||
|
||||
@@ -76,9 +90,18 @@ Manages Faiss indices for AnyLoc retrieval (IVF, HNSW options).
|
||||
|
||||
**External**: faiss-gpu or faiss-cpu
|
||||
|
||||
## GPU/CPU Fallback
|
||||
|
||||
H04 supports automatic fallback from GPU to CPU:
|
||||
- `is_gpu_available()`: Returns True if faiss-gpu is available and CUDA works
|
||||
- `set_device("gpu")`: Use GPU acceleration (faster for large indexes)
|
||||
- `set_device("cpu")`: Use CPU (fallback when GPU unavailable)
|
||||
|
||||
## Test Cases
|
||||
|
||||
1. Build index with 10,000 descriptors → succeeds
|
||||
2. Search query → returns top-k matches
|
||||
1. Build index with 10,000 UAV image descriptors → succeeds
|
||||
2. Search query UAV descriptor → returns top-k similar UAV frames
|
||||
3. Save/load index → index restored correctly
|
||||
4. GPU unavailable → automatically falls back to CPU
|
||||
5. Add descriptors incrementally → index grows correctly
|
||||
|
||||
|
||||
@@ -0,0 +1,883 @@
|
||||
# ASTRAL-Next System Flows
|
||||
|
||||
> **See also**: [System Flow Diagrams (Mermaid)](./system_flows_diagrams.md) for interactive visual diagrams.
|
||||
|
||||
## System Overview
|
||||
|
||||
ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer matching (SuperPoint+LightGlue for VO, DINOv2 for place recognition, LiteSAM for cross-view matching) with an Atlas-style multi-map chunk architecture for robust tracking and recovery.
|
||||
|
||||
## Components Summary
|
||||
|
||||
| ID | Component | Interface | Purpose |
|
||||
|----|-----------|-----------|---------|
|
||||
| F01 | Flight API | `IFlightAPI` | REST endpoints, SSE streaming |
|
||||
| F02 | Flight Processor | `IFlightProcessor` | Central coordinator, processing loop |
|
||||
| F03 | Flight Database | `IFlightDatabase` | Persistence layer |
|
||||
| F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search |
|
||||
| F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage |
|
||||
| F06 | Image Rotation Manager | `IImageRotationManager` | Rotation sweeps, heading tracking |
|
||||
| F07 | Sequential Visual Odometry | `ISequentialVO` | Frame-to-frame VO (SuperPoint+LightGlue) |
|
||||
| F08 | Global Place Recognition | `IGlobalPlaceRecognition` | Coarse localization (DINOv2+VLAD) |
|
||||
| F09 | Metric Refinement | `IMetricRefinement` | Precise alignment (LiteSAM) |
|
||||
| F10 | Factor Graph Optimizer | `IFactorGraphOptimizer` | GTSAM-based trajectory optimization |
|
||||
| F11 | Failure Recovery Coordinator | `IFailureRecoveryCoordinator` | Recovery orchestration, chunk matching |
|
||||
| F12 | Route Chunk Manager | `IRouteChunkManager` | Chunk lifecycle management |
|
||||
| F13 | Coordinate Transformer | `ICoordinateTransformer` | Coordinate conversions (GPS↔ENU↔Pixel) |
|
||||
| F14 | Result Manager | `IResultManager` | Result tracking and publishing |
|
||||
| F15 | SSE Event Streamer | `ISSEEventStreamer` | Real-time event streaming |
|
||||
| F16 | Model Manager | `IModelManager` | ML model loading (TensorRT/ONNX) |
|
||||
| F17 | Configuration Manager | `IConfigurationManager` | System configuration |
|
||||
|
||||
### Helper Components
|
||||
|
||||
| ID | Helper | Interface | Purpose |
|
||||
|----|--------|-----------|---------|
|
||||
| H01 | Camera Model | `ICameraModel` | Projection/unprojection |
|
||||
| H02 | GSD Calculator | `IGSDCalculator` | Ground sampling distance |
|
||||
| H03 | Robust Kernels | `IRobustKernels` | Huber/Cauchy loss functions |
|
||||
| H04 | Faiss Index Manager | `IFaissIndexManager` | Similarity search |
|
||||
| H05 | Performance Monitor | `IPerformanceMonitor` | Timing measurements |
|
||||
| H06 | Web Mercator Utils | `IWebMercatorUtils` | Tile coordinate calculations |
|
||||
| H07 | Image Rotation Utils | `IImageRotationUtils` | Image rotation operations |
|
||||
| H08 | Batch Validator | `IBatchValidator` | Image batch validation |
|
||||
|
||||
---
|
||||
|
||||
## Flow 1: System Initialization
|
||||
|
||||
**Purpose**: Initialize all system components on startup.
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ System Start │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F17 load_config │ ← Load system configuration
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F03 Initialize │ ← Establish DB connection pool
|
||||
│ connections │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────┐
|
||||
│ F16 load_model() × 4 │
|
||||
│ ├─ SuperPoint (feature extraction) │
|
||||
│ ├─ LightGlue (feature matching) │
|
||||
│ ├─ DINOv2 (place recognition) │
|
||||
│ └─ LiteSAM (cross-view matching) │
|
||||
└────────┬────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│F04 Initialize │ ← Initialize tile cache
|
||||
│ cache │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────┐
|
||||
│ F08 load_index() │
|
||||
│ ← Load pre-built Faiss index from │
|
||||
│ satellite provider │
|
||||
└────────┬────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F12 Initialize │ ← Initialize chunk state tracking
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F02 Ready │ ← Ready to accept flights
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F01 Start │ ← Start FastAPI/Uvicorn
|
||||
│ server │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
**Duration**: ~30 seconds (dominated by model loading)
|
||||
|
||||
---
|
||||
|
||||
## Flow 2: Flight Creation
|
||||
|
||||
**Purpose**: Create a new flight with initial configuration and prefetch satellite data.
|
||||
|
||||
**Trigger**: `POST /flights`
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────┐ POST /flights ┌─────┐
|
||||
│ Client │ ───────────────────► │ F01 │
|
||||
└──────────┘ └──┬──┘
|
||||
│ create_flight()
|
||||
▼
|
||||
┌─────────┐
|
||||
│ F02 │ Flight Processor
|
||||
└────┬────┘
|
||||
┌────────────────────────┬─┴─┬────────────────────────┐
|
||||
│ │ │ │
|
||||
▼ │ ▼ ▼
|
||||
┌───────────────┐ │ ┌───────────────┐ ┌───────────────┐
|
||||
│F17 get_flight │ │ │F13 set_enu │ │F04 prefetch │
|
||||
│ _config() │ │ │ _origin() │ │ _route │
|
||||
└───────────────┘ │ └───────────────┘ │ _corridor() │
|
||||
│ └───────┬───────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────┐ ┌─────────────────┐
|
||||
│F03 insert │ │ Satellite │
|
||||
│ _flight() │ │ Provider API │
|
||||
└─────────────┘ │ GET tiles/batch │
|
||||
└─────────────────┘
|
||||
│
|
||||
┌──────┴──────┐
|
||||
│ F14 │
|
||||
│create_stream│
|
||||
└─────────────┘
|
||||
│
|
||||
┌──────┴──────┐
|
||||
SSE Connection │ Client │
|
||||
◄─────────────────────────────│ GET stream │
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
**Output**: `flight_id`, SSE stream established
|
||||
|
||||
---
|
||||
|
||||
## Flow 3: Image Upload
|
||||
|
||||
**Purpose**: Upload batch of UAV images for processing.
|
||||
|
||||
**Trigger**: `POST /flights/{flightId}/images/batch`
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────┐ POST images/batch ┌─────┐
|
||||
│ Client │ ──────────────────► │ F01 │
|
||||
│ │ (10-50 images) └──┬──┘
|
||||
└──────────┘ │ queue_batch()
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ F05 │ Image Input Pipeline
|
||||
└──────┬──────┘
|
||||
┌───────────────┼───────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│H08 validate │ │ Store to │ │F03 save │
|
||||
│ _batch() │ │ disk │ │ _image │
|
||||
└─────────────┘ └─────────────┘ │ _metadata()│
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
**Validation Rules**:
|
||||
- Batch size: 10-50 images
|
||||
- Naming convention: ADxxxxxx.jpg
|
||||
- Sequential numbering
|
||||
- Image dimensions: 640×480 to 6252×4168
|
||||
|
||||
---
|
||||
|
||||
## Flow 4: Normal Frame Processing (Tracking Good)
|
||||
|
||||
**Purpose**: Process a single frame when tracking quality is good.
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ F02 Flight Processor │
|
||||
│ │
|
||||
│ ┌─────────────┐ │
|
||||
│ │ get_next │◄───────────────────────────────────────────────────────┐│
|
||||
│ │ _image() │ ││
|
||||
│ └──────┬──────┘ ││
|
||||
│ │ F05 ││
|
||||
│ ▼ ││
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ ││
|
||||
│ │ get_active_chunk() via F12 │ ││
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ ││
|
||||
│ ▼ ││
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ ││
|
||||
│ │ requires_rotation_sweep()? via F06 │ ││
|
||||
│ │ ├─ YES → Flow 5 (First Frame/Sharp Turn) │ ││
|
||||
│ │ └─ NO → Pre-rotate to current heading │ ││
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ ││
|
||||
│ ▼ ││
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ ││
|
||||
│ │ F07 compute_relative_pose_in_chunk() │ ││
|
||||
│ │ ├─ SuperPoint extract (prev + curr) via F16 │ ││
|
||||
│ │ └─ LightGlue match via F16 │ ││
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ ││
|
||||
│ │ ││
|
||||
│ ┌─────────────────┼─────────────────┐ ││
|
||||
│ │ Tracking Good? │ Tracking Lost? │ ││
|
||||
│ ▼ ▼ ▼ ││
|
||||
│ ┌─────────────────┐ ┌─────────────────────────────────┐ ││
|
||||
│ │F12 add_frame │ │ →Flow 6 (Tracking Loss/Recovery)│ ││
|
||||
│ │ _to_chunk() │ └─────────────────────────────────┘ ││
|
||||
│ └────────┬────────┘ ││
|
||||
│ ▼ ││
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ ││
|
||||
│ │ F04 fetch_tile() + compute_tile_bounds() │ ││
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ ││
|
||||
│ ▼ ││
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ ││
|
||||
│ │ F09 align_to_satellite(image, tile, tile_bounds) │ ││
|
||||
│ │ └─ LiteSAM matching via F16 │ ││
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ ││
|
||||
│ │ ││
|
||||
│ ┌─────────────────┴─────────────────┐ ││
|
||||
│ │ Match Found? │ ││
|
||||
│ ▼ ▼ ││
|
||||
│ ┌─────────────────────────┐ ┌─────────────────────────┐ ││
|
||||
│ │F10 add_absolute_factor()│ │ Skip absolute anchor │ ││
|
||||
│ │F06 update_heading() │ │ (VO-only frame) │ ││
|
||||
│ └────────────┬────────────┘ └─────────────────────────┘ ││
|
||||
│ │ ││
|
||||
│ └─────────────────────┬───────────────────────────────────┘│
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ F10 optimize_chunk() │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ F13 enu_to_gps() → Convert ENU pose to GPS │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ F14 update_frame_result() + publish_waypoint_update() │ │
|
||||
│ │ └─ F15 send_frame_result() → SSE "frame_processed" │ │
|
||||
│ │ └─ F03 save_frame_result() │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Performance**: < 5 seconds per frame
|
||||
|
||||
---
|
||||
|
||||
## Flow 5: First Frame / Sharp Turn (Rotation Sweep)
|
||||
|
||||
**Purpose**: Establish UAV heading when unknown or after sharp turn.
|
||||
|
||||
**Trigger**: `requires_rotation_sweep()` returns True
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌───────────────────────────────────────────────────────────────────────────┐
|
||||
│ F06 Image Rotation Manager │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ try_rotation_steps(image, satellite_tile, tile_bounds) │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────────────────────┴────────────────────────────┐ │
|
||||
│ │ For angle in [0°, 30°, 60°, ... 330°]: │ │
|
||||
│ │ │ │
|
||||
│ │ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ │ H07 rotate_image(image, angle) │ │ │
|
||||
│ │ └───────────────────┬─────────────────┘ │ │
|
||||
│ │ ▼ │ │
|
||||
│ │ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ │ F09 align_to_satellite(rotated, │ │ │
|
||||
│ │ │ satellite_tile, tile_bounds) │ │ │
|
||||
│ │ └───────────────────┬─────────────────┘ │ │
|
||||
│ │ │ │ │
|
||||
│ │ ┌──────────────┴──────────────┐ │ │
|
||||
│ │ │ Match Found? │ │ │
|
||||
│ │ ▼ ▼ │ │
|
||||
│ │ ┌───────────────┐ ┌───────────────┐ │ │
|
||||
│ │ │ Calculate │ │ Try next │ │ │
|
||||
│ │ │ precise angle │ │ rotation │ │ │
|
||||
│ │ └───────┬───────┘ └───────────────┘ │ │
|
||||
│ │ │ │ │
|
||||
│ │ ▼ │ │
|
||||
│ │ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ │ F03 save_heading() via F06 │ │ │
|
||||
│ │ │ update_heading() │ │ │
|
||||
│ │ └─────────────────────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────────┴────────────┐ │
|
||||
│ │ Return RotationResult │ │
|
||||
│ │ or None │ │
|
||||
│ └─────────────────────────┘ │
|
||||
└───────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Output**: RotationResult with precise heading angle
|
||||
|
||||
---
|
||||
|
||||
## Flow 6: Tracking Loss / Recovery (Progressive Search)
|
||||
|
||||
**Purpose**: Recover localization after tracking loss.
|
||||
|
||||
**Trigger**: VO inlier count < 20 or LiteSAM match fails
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ F11 Failure Recovery Coordinator │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 1. EMIT RecoveryStarted event │ │
|
||||
│ │ └─ F02 subscribes → Update status to "recovering" │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 2. create_chunk_on_tracking_loss() via F12 │ │
|
||||
│ │ └─ Proactive chunk creation (processing continues) │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 3. Single-image recovery attempt: │ │
|
||||
│ │ ├─ F06 requires_rotation_sweep() → trigger sweep │ │
|
||||
│ │ ├─ F08 retrieve_candidate_tiles() (DINOv2) │ │
|
||||
│ │ └─ Progressive tile search (1→4→9→16→25): │ │
|
||||
│ │ │ │
|
||||
│ │ ┌───────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ For grid_size in [1, 4, 9, 16, 25]: │ │ │
|
||||
│ │ │ ├─ F04 expand_search_grid() │ │ │
|
||||
│ │ │ ├─ For each tile: │ │ │
|
||||
│ │ │ │ ├─ F04 compute_tile_bounds() │ │ │
|
||||
│ │ │ │ └─ F09 align_to_satellite(img, tile, bounds)│ │ │
|
||||
│ │ │ │ │ │ │
|
||||
│ │ │ └─ If match found: BREAK │ │ │
|
||||
│ │ └───────────────────────────────────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────────────────┴────────────────────┐ │
|
||||
│ │ Single-image match found? │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌─────────────────────┐ ┌────────────────────────────┐ │
|
||||
│ │ EMIT RecoverySucceeded│ │ Continue chunk building │ │
|
||||
│ │ Resume normal flow │ │ → Flow 7 (Chunk Building) │ │
|
||||
│ └─────────────────────┘ │ → Flow 8 (Chunk Matching) │ │
|
||||
│ │ (Background) │ │
|
||||
│ └────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────┐ │
|
||||
│ │ If all strategies exhausted: │ │
|
||||
│ │ → Flow 10 (User Input Recovery) │ │
|
||||
│ └─────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow 7: Chunk Building
|
||||
|
||||
**Purpose**: Build route chunks when tracking is lost, continuing VO within chunk.
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌───────────────────────────────────────────────────────────────────────────┐
|
||||
│ F12 Route Chunk Manager │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ create_chunk(flight_id, start_frame_id) │ │
|
||||
│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │
|
||||
│ │ ├─ Initialize chunk state (unanchored, active) │ │
|
||||
│ │ └─ F03 save_chunk_state() │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────────────────────┴────────────────────────────┐ │
|
||||
│ │ For each frame in chunk: │ │
|
||||
│ │ │ │
|
||||
│ │ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ │ F07 compute_relative_pose_in_chunk()│ │ │
|
||||
│ │ └───────────────────┬─────────────────┘ │ │
|
||||
│ │ ▼ │ │
|
||||
│ │ ┌──────────────────────────────────────┐ │ │
|
||||
│ │ │ F12 add_frame_to_chunk() │ │ │
|
||||
│ │ │ └─ F10 add_relative_factor_to_chunk│ │ │
|
||||
│ │ └───────────────────┬──────────────────┘ │ │
|
||||
│ │ ▼ │ │
|
||||
│ │ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ │ F10 optimize_chunk() (local) │ │ │
|
||||
│ │ └─────────────────────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ is_chunk_ready_for_matching()? │ │
|
||||
│ │ ├─ Min 5 frames │ │
|
||||
│ │ ├─ Max 20 frames │ │
|
||||
│ │ └─ Internal consistency (good VO inlier counts) │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└───────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow 8: Background Chunk Matching
|
||||
|
||||
**Purpose**: Asynchronously match unanchored chunks to satellite data.
|
||||
|
||||
**Trigger**: Background task (every 5 seconds)
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ F11 process_unanchored_chunks() (Background Task) │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ while flight_active: │ │
|
||||
│ │ ├─ unanchored = F12 get_chunks_for_matching() │ │
|
||||
│ │ │ │ │
|
||||
│ │ └─ for chunk in unanchored: │ │
|
||||
│ │ │ │ │
|
||||
│ │ ▼ │ │
|
||||
│ │ ┌─────────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ if F12 is_chunk_ready_for_matching(chunk_id): │ │ │
|
||||
│ │ │ ├─ F12 mark_chunk_matching(chunk_id) │ │ │
|
||||
│ │ │ │ │ │ │
|
||||
│ │ │ ├─ STEP 1: Chunk Semantic Matching │ │ │
|
||||
│ │ │ │ ├─ F12 get_chunk_images() │ │ │
|
||||
│ │ │ │ └─ F08 retrieve_candidate_tiles_for_chunk() │ │ │
|
||||
│ │ │ │ └─ F08 compute_chunk_descriptor() │ │ │
|
||||
│ │ │ │ └─ H04 Faiss search() │ │ │
|
||||
│ │ │ │ │ │ │
|
||||
│ │ │ ├─ STEP 2: Chunk LiteSAM Matching (with rotation) │ │ │
|
||||
│ │ │ │ ├─ For each candidate tile: │ │ │
|
||||
│ │ │ │ │ ├─ F04 get_tile + compute_tile_bounds() │ │ │
|
||||
│ │ │ │ │ ├─ F06 try_chunk_rotation_steps() │ │ │
|
||||
│ │ │ │ │ │ └─ F09 align_chunk_to_satellite() │ │ │
|
||||
│ │ │ │ │ └─ If match: BREAK │ │ │
|
||||
│ │ │ │ │ │ │
|
||||
│ │ │ └─ If match found: │ │ │
|
||||
│ │ │ └─ → Flow 9 (Chunk Merging) │ │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ └─────────────────────────────────────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ │ sleep(5 seconds) │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow 9: Chunk Merging
|
||||
|
||||
**Purpose**: Merge anchored chunk into main trajectory.
|
||||
|
||||
**Trigger**: Successful chunk matching
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌───────────────────────────────────────────────────────────────────────-──┐
|
||||
│ F11 merge_chunk_to_trajectory() │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 1. Get chunk frames: F12 get_chunk_frames(chunk_id) │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 2. Anchor chunk: F12 mark_chunk_anchored(chunk_id, gps) │ │
|
||||
│ │ └─ F10 add_chunk_anchor() │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 3. Determine target chunk (predecessor or "main") │ │
|
||||
│ │ └─ Returns target_chunk_id (predecessor or "main") │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 4. Merge: F12 merge_chunks(chunk_id, target_chunk_id, Sim3) │ │
|
||||
│ │ ├─ F10 merge_chunks() ← Apply Sim(3) transform │ │
|
||||
│ │ ├─ F12 deactivate_chunk(chunk_id) │ │
|
||||
│ │ └─ F03 save_chunk_state() │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 5. Optimize global: F10 optimize_global() │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 6. EMIT ChunkMerged event (flight_id,chunk_id,merged_frames)│ │
|
||||
│ │ └─ F14 subscribes → update_results_after_chunk_merge() │ │
|
||||
│ │ ├─ F10 get_trajectory() → ENU poses │ │
|
||||
│ │ ├─ F13 enu_to_gps() for each frame │ │
|
||||
│ │ ├─ F03 save_frame_result() + update_waypoint() │ │
|
||||
│ │ └─ F15 send_refinement() → SSE "frame_refined" × N │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Sim(3) Transform**: Translation + Rotation + Scale alignment between chunks
|
||||
|
||||
---
|
||||
|
||||
## Flow 10: User Input Recovery
|
||||
|
||||
**Purpose**: Request human assistance when all automatic recovery fails.
|
||||
|
||||
**Trigger**: Progressive search exhausted (25 tiles), chunk matching failed
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ F11 create_user_input_request() │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 1. Get UAV image for frame_id │ │
|
||||
│ │ 2. Get top-5 candidates from F08 │ │
|
||||
│ │ 3. Create UserInputRequest │ │
|
||||
│ │ 4. F15 send_user_input_request() → SSE "user_input_needed" │ │
|
||||
│ │ 5. EMIT UserInputNeeded event │ │
|
||||
│ │ └─ F02 subscribes → Update status to "BLOCKED" │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼ Client receives SSE event
|
||||
┌───────────────────────────────────────────────────────────────────────────┐
|
||||
│ Client UI │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Display UAV image + candidate satellite tiles │ │
|
||||
│ │ User clicks corresponding point on satellite tile │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ POST /flights/{flightId}/user-fix │ │
|
||||
│ │ body: { frame_id, uav_pixel, satellite_gps } │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└───────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ F11 apply_user_anchor() │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 1. Validate anchor data │ │
|
||||
│ │ 2. F10 add_absolute_factor(is_user_anchor=True) ← σ=5m │ │
|
||||
│ │ 3. F10 optimize() │ │
|
||||
│ │ 4. EMIT UserFixApplied event │ │
|
||||
│ │ └─ F02 subscribes → Update status to "PROCESSING" │ │
|
||||
│ │ 5. Resume processing loop │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow 11: Asynchronous Refinement
|
||||
|
||||
**Purpose**: Back-propagate optimization improvements to previous frames.
|
||||
|
||||
**Trigger**: New absolute GPS factor added, chunk merged
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────-────────────┐
|
||||
│ F10 Background Optimization │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 1. New absolute factor triggers batch optimization │ │
|
||||
│ │ 2. F10 optimize(iterations=50-100) │ │
|
||||
│ │ └─ Levenberg-Marquardt with robust kernels │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 3. Identify refined frames │ │
|
||||
│ │ 4. F14 mark_refined(frame_ids) │ │
|
||||
│ │ ├─ For each frame: │ │
|
||||
│ │ │ ├─ F10 get_trajectory() → ENU pose │ │
|
||||
│ │ │ ├─ F13 enu_to_gps(flight_id, enu_pose) │ │
|
||||
│ │ │ ├─ F03 save_frame_result(refined=True) │ │
|
||||
│ │ │ └─ F03 update_waypoint() │ │
|
||||
│ │ │ │ │
|
||||
│ │ └─ F15 send_refinement() × N → SSE "frame_refined" × N │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow 12: Object to GPS Conversion
|
||||
|
||||
**Purpose**: Convert detected object pixel coordinates to GPS (external integration).
|
||||
|
||||
**Trigger**: `POST /flights/{flightId}/frames/{frameId}/object-to-gps`
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────────────┐ POST object-to-gps ┌─────┐
|
||||
│ External System │ ────────────────────►│ F01 │
|
||||
│ (Azaion.Inference│ {pixel_x, pixel_y} └──┬──┘
|
||||
└──────────────────┘ │
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ F13 │ Coordinate Transformer
|
||||
└──────┬──────┘
|
||||
┌───────────────────┼───────────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│F10 get_pose │ │F17 camera │ │ H01 Camera │
|
||||
│ (frame_id) │ │ _params │ │ Model │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘
|
||||
│ │ │
|
||||
└───────────────────┼───────────────────┘
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ pixel_to_gps│
|
||||
│ calculation │
|
||||
└──────┬──────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────┐
|
||||
│ Return ObjectGPSResponse │
|
||||
│ { gps, accuracy_meters }│
|
||||
└───────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow 13: Flight Completion
|
||||
|
||||
**Purpose**: Complete flight processing and clean up resources.
|
||||
|
||||
**Trigger**: All images processed successfully
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ F02 Flight Processor │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 1. All frames processed │ │
|
||||
│ │ 2. Wait for pending chunk merges (if any) │ │
|
||||
│ │ 3. F10 optimize_global(iterations=100) ← Final optimization │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 4. F14 mark_refined(all_frames) │ │
|
||||
│ │ └─ Final refinement updates │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 5. F03 save_flight_state(status="completed") │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 6. F15 send event → SSE "flight_completed" │ │
|
||||
│ └──────────────────────────────┬──────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 7. Cleanup: │ │
|
||||
│ │ ├─ Stop background chunk matching task │ │
|
||||
│ │ ├─ F04 clear_flight_cache(flight_id) (optional) │ │
|
||||
│ │ └─ Release flight resources │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow 14: System Shutdown
|
||||
|
||||
**Purpose**: Gracefully shutdown all components.
|
||||
|
||||
**Sequence**:
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Shutdown Signal │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F01 Stop │ ← Stop accepting requests
|
||||
│ accepting │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F02 Complete/ │ ← Complete or cancel active flights
|
||||
│ Cancel │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F11 Stop │ ← Stop background chunk matching
|
||||
│ background │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F12 Save chunk │ ← Save chunk state for recovery
|
||||
│ state │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F16 Unload │ ← Unload ML models, free GPU memory
|
||||
│ models │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F03 Close │ ← Close database connections
|
||||
│ connections │
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ F04 Flush cache │ ← Flush satellite tile cache
|
||||
└────────┬────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Shutdown │
|
||||
│ Complete │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Component Interaction Summary
|
||||
|
||||
### Data Flow Direction
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ EXTERNAL │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Client │ │ Satellite │ │ External │ │
|
||||
│ │ (UI) │ │ Provider │ │ Detector │ │
|
||||
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
|
||||
└─────────┼───────────────────┼───────────────────┼───────────────────────────┘
|
||||
│ REST/SSE │ HTTP │ REST
|
||||
▼ ▼ ▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ API LAYER │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ F01 Flight API │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ ORCHESTRATION LAYER │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ F02 Flight Processor │ │
|
||||
│ │ (Central coordinator, event subscriber, background task manager) │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
├─────────────────────────────────────────────────────────────┐
|
||||
▼ ▼
|
||||
┌─────────────────────────────────────────────┐ ┌────────────────────────────┐
|
||||
│ DATA MANAGEMENT LAYER │ │ RECOVERY LAYER │
|
||||
│ ┌─────────────┐ ┌─────────────┐ │ │ ┌───────────────────────┐ │
|
||||
│ │ F04 │ │ F05 │ │ │ │ F11 │ │
|
||||
│ │ Satellite │ │ Image │ │ │ │ Failure Recovery │ │
|
||||
│ │ Data │ │ Input │ │ │ │ (Event emitter) │ │
|
||||
│ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ │ │ ▼ │
|
||||
│ │ F12 │ │ F03 │ │ │ ┌───────────────────────┐ │
|
||||
│ │Route Chunk │ │ Flight │ │ │ │ F12 │ │
|
||||
│ │ Manager │ │ Database │ │ │ │ (Chunk state source) │ │
|
||||
│ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │
|
||||
└─────────────────────────────────────────────┘ └────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ VISUAL PROCESSING LAYER │
|
||||
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ┌──────────────┐ │
|
||||
│ │ F06 │ │ F07 │ │ F08 │ │ F09 │ │
|
||||
│ │ Rotation │ │ Sequential VO │ │ Global │ │ Metric │ │
|
||||
│ │ Manager │ │(SuperPoint+ │ │ Place │ │ Refinement │ │
|
||||
│ │ (30° sweeps) │ │ LightGlue) │ │ Recognition │ │ (LiteSAM) │ │
|
||||
│ │ │ │ │ │ (DINOv2) │ │ │ │
|
||||
│ └───────────────┘ └───────────────┘ └───────────────┘ └──────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ STATE ESTIMATION LAYER │
|
||||
│ ┌──────────────────────────────────────────────────────────────────k──┐ │
|
||||
│ │ F10 Factor Graph Optimizer │ │
|
||||
│ │ (GTSAM, iSAM2, Robust kernels, Chunk subgraphs, Sim(3) merging) │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ OUTPUT LAYER │
|
||||
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
|
||||
│ │ F13 │ │ F14 │ │ F15 │ │
|
||||
│ │ Coordinate │ │ Result │ │ SSE │ │
|
||||
│ │ Transformer │ │ Manager │ │ Streamer │ │
|
||||
│ │ (ENU↔GPS↔Px) │ │ │ │ │ │
|
||||
│ └───────────────┘ └───────────────┘ └───────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────-┐
|
||||
│ INFRASTRUCTURE LAYER │
|
||||
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────────────────-┐│
|
||||
│ │ F16 │ │ F17 │ │ HELPERS ││
|
||||
│ │ Model │ │Configuration │ │ H01-H08 (Camera, GSD, Kernels, ││
|
||||
│ │ Manager │ │ Manager │ │ Faiss, Monitor, Mercator, etc) ││
|
||||
│ └───────────────┘ └───────────────┘ └────────────────────────────────────┘│
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Event-Based Communication
|
||||
|
||||
F11 Failure Recovery Coordinator emits events instead of directly calling F02:
|
||||
|
||||
| Event | Publisher | Subscribers | Action |
|
||||
|-------|-----------|-------------|--------|
|
||||
| `RecoveryStarted` | F11 | F02 | Update status to "recovering" |
|
||||
| `RecoverySucceeded` | F11 | F02 | Update status to "processing", resume |
|
||||
| `RecoveryFailed` | F11 | F02 | Update status to "blocked" |
|
||||
| `UserInputNeeded` | F11 | F02 | Update status to "blocked", await fix |
|
||||
| `UserFixApplied` | F11 | F02 | Update status to "processing", resume |
|
||||
| `ChunkCreated` | F11 | F02 | Log chunk creation |
|
||||
| `ChunkAnchored` | F11 | F02 | Log chunk anchor |
|
||||
| `ChunkMerged` | F11 | F02, F14 | F14 updates results for merged frames |
|
||||
|
||||
---
|
||||
|
||||
## SSE Events to Client
|
||||
|
||||
| Event | Trigger | Data |
|
||||
|-------|---------|------|
|
||||
| `frame_processed` | Frame completed | frame_id, gps, altitude, heading, confidence |
|
||||
| `frame_refined` | Trajectory refined | frame_id, updated gps, refined=true |
|
||||
| `search_expanded` | Progressive search | frame_id, grid_size, status |
|
||||
| `user_input_needed` | Recovery exhausted | request_id, frame_id, candidate_tiles |
|
||||
| `processing_blocked` | Status change | reason, frame_id |
|
||||
| `flight_completed` | Flight done | statistics |
|
||||
|
||||
---
|
||||
|
||||
## Performance Targets
|
||||
|
||||
| Flow | Target |
|
||||
|------|--------|
|
||||
| System Initialization | < 30 seconds |
|
||||
| Flight Creation | < 500ms response |
|
||||
| Image Batch Upload | < 2 seconds (50 images) |
|
||||
| Per-Frame Processing | < 5 seconds |
|
||||
| Rotation Sweep (12 rotations) | < 1.2 seconds |
|
||||
| Progressive Search (25 tiles) | < 1.5 seconds |
|
||||
| Chunk Matching | < 3 seconds (background) |
|
||||
| SSE Event Latency | < 500ms |
|
||||
|
||||
---
|
||||
|
||||
## Accuracy Targets
|
||||
|
||||
| Metric | Target |
|
||||
|--------|--------|
|
||||
| GPS Accuracy | 60% < 20m, 80% < 50m |
|
||||
| Mean Reprojection Error | < 1.0 pixels |
|
||||
| Place Recognition Recall@5 | > 85% |
|
||||
| LiteSAM Success Rate | > 95% (when rotation correct) |
|
||||
|
||||
@@ -0,0 +1,603 @@
|
||||
# ASTRAL-Next System Flow Diagrams
|
||||
|
||||
## Component Architecture Diagram
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph External["External Systems"]
|
||||
Client[Client UI]
|
||||
SatProv[Satellite Provider]
|
||||
ExtDet[External Detector]
|
||||
end
|
||||
|
||||
subgraph API["API Layer"]
|
||||
F01[F01 Flight API]
|
||||
end
|
||||
|
||||
subgraph Orchestration["Orchestration Layer"]
|
||||
F02[F02 Flight Processor]
|
||||
end
|
||||
|
||||
subgraph Data["Data Management"]
|
||||
F03[F03 Flight Database]
|
||||
F04[F04 Satellite Data Manager]
|
||||
F05[F05 Image Input Pipeline]
|
||||
F12[F12 Route Chunk Manager]
|
||||
end
|
||||
|
||||
subgraph Recovery["Recovery Layer"]
|
||||
F11[F11 Failure Recovery Coordinator]
|
||||
end
|
||||
|
||||
subgraph Visual["Visual Processing"]
|
||||
F06[F06 Image Rotation Manager]
|
||||
F07[F07 Sequential Visual Odometry]
|
||||
F08[F08 Global Place Recognition]
|
||||
F09[F09 Metric Refinement]
|
||||
end
|
||||
|
||||
subgraph State["State Estimation"]
|
||||
F10[F10 Factor Graph Optimizer]
|
||||
end
|
||||
|
||||
subgraph Output["Output Layer"]
|
||||
F13[F13 Coordinate Transformer]
|
||||
F14[F14 Result Manager]
|
||||
F15[F15 SSE Event Streamer]
|
||||
end
|
||||
|
||||
subgraph Infra["Infrastructure"]
|
||||
F16[F16 Model Manager]
|
||||
F17[F17 Configuration Manager]
|
||||
end
|
||||
|
||||
Client -->|REST| F01
|
||||
F15 -->|SSE| Client
|
||||
ExtDet -->|REST| F01
|
||||
F04 -->|HTTP| SatProv
|
||||
|
||||
F01 --> F02
|
||||
F02 --> F03
|
||||
F02 --> F05
|
||||
F02 --> F11
|
||||
F02 --> F07
|
||||
F02 --> F10
|
||||
F02 --> F12
|
||||
F02 --> F04
|
||||
|
||||
F11 --> F08
|
||||
F11 --> F09
|
||||
F11 --> F12
|
||||
F11 -.->|events| F02
|
||||
F11 -.->|events| F14
|
||||
|
||||
F06 --> F09
|
||||
F07 --> F10
|
||||
F09 --> F10
|
||||
F12 --> F10
|
||||
F10 --> F13
|
||||
F14 --> F13
|
||||
F14 --> F15
|
||||
F14 --> F03
|
||||
|
||||
F07 --> F16
|
||||
F08 --> F16
|
||||
F09 --> F16
|
||||
F02 --> F17
|
||||
```
|
||||
|
||||
## Flow 1: System Initialization
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Main as System
|
||||
participant F17 as Config Manager
|
||||
participant F03 as Flight Database
|
||||
participant F16 as Model Manager
|
||||
participant F04 as Satellite Manager
|
||||
participant F08 as Place Recognition
|
||||
participant F12 as Chunk Manager
|
||||
participant F02 as Flight Processor
|
||||
participant F01 as Flight API
|
||||
|
||||
Main->>F17: load_config()
|
||||
F17-->>Main: SystemConfig
|
||||
|
||||
Main->>F03: Initialize connections
|
||||
F03-->>Main: Connection pool ready
|
||||
|
||||
Main->>F16: load_model("SuperPoint")
|
||||
Main->>F16: load_model("LightGlue")
|
||||
Main->>F16: load_model("DINOv2")
|
||||
Main->>F16: load_model("LiteSAM")
|
||||
F16-->>Main: Models loaded (~25s)
|
||||
|
||||
Main->>F04: Initialize cache
|
||||
F04-->>Main: Cache ready
|
||||
|
||||
Main->>F08: load_index()
|
||||
F08-->>Main: Faiss index loaded
|
||||
|
||||
Main->>F12: Initialize
|
||||
F12-->>Main: Chunk tracking ready
|
||||
|
||||
Main->>F02: Ready
|
||||
F02-->>Main: Ready to accept flights
|
||||
|
||||
Main->>F01: Start server
|
||||
F01-->>Main: FastAPI running
|
||||
```
|
||||
|
||||
## Flow 2: Flight Creation
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C as Client
|
||||
participant F01 as Flight API
|
||||
participant F02 as Flight Processor
|
||||
participant F17 as Config Manager
|
||||
participant F13 as Coord Transformer
|
||||
participant F04 as Satellite Manager
|
||||
participant F03 as Flight Database
|
||||
participant F15 as SSE Streamer
|
||||
|
||||
C->>F01: POST /flights
|
||||
F01->>F02: create_flight()
|
||||
|
||||
F02->>F17: get_flight_config()
|
||||
F17-->>F02: CameraParams, Altitude
|
||||
|
||||
F02->>F13: set_enu_origin(start_gps)
|
||||
|
||||
F02->>F04: prefetch_route_corridor()
|
||||
F04-->>F02: Prefetching async
|
||||
|
||||
F02->>F03: insert_flight()
|
||||
F03-->>F02: flight_id
|
||||
|
||||
F01-->>C: 201 Created {flight_id}
|
||||
|
||||
C->>F01: GET /flights/{id}/stream
|
||||
F01->>F15: create_stream()
|
||||
F15-->>C: SSE Connection
|
||||
```
|
||||
|
||||
## Flow 3: Normal Frame Processing
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F02 as Flight Processor
|
||||
participant F05 as Image Pipeline
|
||||
participant F12 as Chunk Manager
|
||||
participant F06 as Rotation Manager
|
||||
participant F07 as Sequential VO
|
||||
participant F04 as Satellite Manager
|
||||
participant F09 as Metric Refinement
|
||||
participant F10 as Factor Graph
|
||||
participant F13 as Coord Transformer
|
||||
participant F14 as Result Manager
|
||||
participant F15 as SSE Streamer
|
||||
|
||||
F02->>F05: get_next_image()
|
||||
F05-->>F02: ImageData
|
||||
|
||||
F02->>F12: get_active_chunk()
|
||||
F12-->>F02: ChunkHandle
|
||||
|
||||
F02->>F06: requires_rotation_sweep()
|
||||
F06-->>F02: false (heading known)
|
||||
|
||||
F02->>F07: compute_relative_pose_in_chunk()
|
||||
F07-->>F02: RelativePose
|
||||
|
||||
F02->>F12: add_frame_to_chunk()
|
||||
F12->>F10: add_relative_factor_to_chunk()
|
||||
|
||||
F02->>F04: fetch_tile() + compute_tile_bounds()
|
||||
F04-->>F02: tile, tile_bounds
|
||||
|
||||
F02->>F09: align_to_satellite(img, tile, bounds)
|
||||
F09-->>F02: AlignmentResult (GPS)
|
||||
|
||||
F02->>F10: add_absolute_factor()
|
||||
F02->>F10: optimize_chunk()
|
||||
F10-->>F02: OptimizationResult
|
||||
|
||||
F02->>F13: enu_to_gps()
|
||||
F13-->>F02: GPSPoint
|
||||
|
||||
F02->>F14: update_frame_result()
|
||||
F14->>F15: send_frame_result()
|
||||
F15-->>Client: SSE "frame_processed"
|
||||
```
|
||||
|
||||
## Flow 4: Rotation Sweep (First Frame / Sharp Turn)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F02 as Flight Processor
|
||||
participant F06 as Rotation Manager
|
||||
participant H07 as Rotation Utils
|
||||
participant F09 as Metric Refinement
|
||||
participant F03 as Flight Database
|
||||
|
||||
F02->>F06: try_rotation_steps(img, tile, bounds)
|
||||
|
||||
loop For angle in [0°, 30°, ... 330°]
|
||||
F06->>H07: rotate_image(img, angle)
|
||||
H07-->>F06: rotated_img
|
||||
|
||||
F06->>F09: align_to_satellite(rotated_img, tile, bounds)
|
||||
F09-->>F06: AlignmentResult
|
||||
|
||||
alt Match Found (confidence > 0.7)
|
||||
F06->>F06: calculate_precise_angle()
|
||||
F06->>F03: save_heading()
|
||||
F06-->>F02: RotationResult
|
||||
end
|
||||
end
|
||||
|
||||
alt No Match Found
|
||||
F06-->>F02: None (trigger recovery)
|
||||
end
|
||||
```
|
||||
|
||||
## Flow 5: Tracking Loss Recovery
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F02 as Flight Processor
|
||||
participant F11 as Failure Recovery
|
||||
participant F12 as Chunk Manager
|
||||
participant F06 as Rotation Manager
|
||||
participant F08 as Place Recognition
|
||||
participant F04 as Satellite Manager
|
||||
participant F09 as Metric Refinement
|
||||
|
||||
F02->>F11: start_search(frame_id, estimated_gps)
|
||||
F11-->>F02: SearchSession
|
||||
|
||||
Note over F11: Emit RecoveryStarted event
|
||||
|
||||
F11->>F12: create_chunk_on_tracking_loss()
|
||||
F12-->>F11: ChunkHandle (processing continues)
|
||||
|
||||
F11->>F06: requires_rotation_sweep()
|
||||
F11->>F08: retrieve_candidate_tiles()
|
||||
|
||||
loop Progressive Search [1, 4, 9, 16, 25]
|
||||
F11->>F04: expand_search_grid(grid_size)
|
||||
F04-->>F11: tiles
|
||||
|
||||
loop For each tile
|
||||
F11->>F04: compute_tile_bounds()
|
||||
F11->>F09: align_to_satellite(img, tile, bounds)
|
||||
|
||||
alt Match Found
|
||||
Note over F11: Emit RecoverySucceeded
|
||||
F11-->>F02: RecoveryResult(success=true)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
alt All Failed
|
||||
Note over F11: Emit UserInputNeeded
|
||||
F11-->>F02: RecoveryResult(success=false)
|
||||
end
|
||||
```
|
||||
|
||||
## Flow 6: Chunk Matching (Background)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F11 as Failure Recovery
|
||||
participant F12 as Chunk Manager
|
||||
participant F08 as Place Recognition
|
||||
participant F06 as Rotation Manager
|
||||
participant F04 as Satellite Manager
|
||||
participant F09 as Metric Refinement
|
||||
participant F10 as Factor Graph
|
||||
|
||||
loop Every 5 seconds
|
||||
F11->>F12: get_chunks_for_matching()
|
||||
F12-->>F11: List[ChunkHandle]
|
||||
|
||||
loop For each unanchored chunk
|
||||
F11->>F12: is_chunk_ready_for_matching()
|
||||
|
||||
alt Chunk Ready
|
||||
F11->>F12: mark_chunk_matching()
|
||||
|
||||
Note over F11: Step 1: Semantic Matching
|
||||
F11->>F12: get_chunk_images()
|
||||
F11->>F08: retrieve_candidate_tiles_for_chunk()
|
||||
F08-->>F11: List[TileCandidate]
|
||||
|
||||
Note over F11: Step 2: LiteSAM with Rotation
|
||||
loop For each candidate tile
|
||||
F11->>F04: get_tile + compute_tile_bounds()
|
||||
F11->>F06: try_chunk_rotation_steps()
|
||||
F06->>F09: align_chunk_to_satellite()
|
||||
|
||||
alt Match Found
|
||||
F09-->>F11: ChunkAlignmentResult
|
||||
end
|
||||
end
|
||||
|
||||
alt Match Found
|
||||
F11->>F12: mark_chunk_anchored()
|
||||
F12->>F10: add_chunk_anchor()
|
||||
|
||||
F11->>F12: merge_chunks()
|
||||
F12->>F10: merge_chunks(Sim3)
|
||||
|
||||
F11->>F10: optimize_global()
|
||||
|
||||
Note over F11: Emit ChunkMerged event
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Flow 7: User Input Recovery
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F11 as Failure Recovery
|
||||
participant F08 as Place Recognition
|
||||
participant F15 as SSE Streamer
|
||||
participant C as Client
|
||||
participant F01 as Flight API
|
||||
participant F10 as Factor Graph
|
||||
participant F02 as Flight Processor
|
||||
|
||||
F11->>F08: retrieve_candidate_tiles()
|
||||
F08-->>F11: Top-5 candidates
|
||||
|
||||
F11->>F15: send_user_input_request()
|
||||
F15-->>C: SSE "user_input_needed"
|
||||
|
||||
Note over F11: Emit UserInputNeeded event
|
||||
Note over F02: Status = BLOCKED
|
||||
|
||||
C->>F01: POST /user-fix {pixel, gps}
|
||||
F01->>F11: apply_user_anchor()
|
||||
|
||||
F11->>F10: add_absolute_factor(is_user_anchor=true)
|
||||
F11->>F10: optimize()
|
||||
|
||||
Note over F11: Emit UserFixApplied event
|
||||
Note over F02: Status = PROCESSING
|
||||
|
||||
F11-->>F01: Success
|
||||
F01-->>C: 200 OK
|
||||
```
|
||||
|
||||
## Flow 8: Result Publishing & Refinement
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F10 as Factor Graph
|
||||
participant F14 as Result Manager
|
||||
participant F13 as Coord Transformer
|
||||
participant F03 as Flight Database
|
||||
participant F15 as SSE Streamer
|
||||
participant C as Client
|
||||
|
||||
Note over F10: New absolute factor added
|
||||
|
||||
F10->>F10: optimize(batch)
|
||||
|
||||
F10->>F14: mark_refined(frame_ids)
|
||||
|
||||
loop For each refined frame
|
||||
F14->>F10: get_trajectory()
|
||||
F10-->>F14: Pose (ENU)
|
||||
|
||||
F14->>F13: enu_to_gps(flight_id, enu)
|
||||
F13-->>F14: GPSPoint
|
||||
|
||||
F14->>F03: save_frame_result(refined=true)
|
||||
F14->>F03: update_waypoint()
|
||||
|
||||
F14->>F15: send_refinement()
|
||||
F15-->>C: SSE "frame_refined"
|
||||
end
|
||||
```
|
||||
|
||||
## Flow 9: Object to GPS Conversion
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Ext as External Detector
|
||||
participant F01 as Flight API
|
||||
participant F13 as Coord Transformer
|
||||
participant F10 as Factor Graph
|
||||
participant F17 as Config Manager
|
||||
participant H01 as Camera Model
|
||||
|
||||
Ext->>F01: POST /object-to-gps {pixel_x, pixel_y}
|
||||
F01->>F13: image_object_to_gps(pixel, frame_id)
|
||||
|
||||
F13->>F10: get_pose(frame_id)
|
||||
F10-->>F13: Pose (ENU)
|
||||
|
||||
F13->>F17: get_camera_params()
|
||||
F17-->>F13: CameraParameters
|
||||
|
||||
F13->>H01: unproject(pixel)
|
||||
F13->>F13: intersect_ground_plane()
|
||||
F13->>F13: enu_to_gps()
|
||||
|
||||
F13-->>F01: GPSPoint
|
||||
F01-->>Ext: {gps, accuracy_meters}
|
||||
```
|
||||
|
||||
## Complete System Flow Overview
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Init["System Initialization"]
|
||||
direction TB
|
||||
I1[Load Config F17]
|
||||
I2[Init DB F03]
|
||||
I3[Load Models F16]
|
||||
I4[Init Cache F04]
|
||||
I5[Load Faiss F08]
|
||||
I6[Start API F01]
|
||||
I1 --> I2 --> I3 --> I4 --> I5 --> I6
|
||||
end
|
||||
|
||||
subgraph Flight["Flight Lifecycle"]
|
||||
direction TB
|
||||
FL1[Create Flight]
|
||||
FL2[Upload Images]
|
||||
FL3[Process Frames]
|
||||
FL4[Complete Flight]
|
||||
FL1 --> FL2 --> FL3 --> FL4
|
||||
end
|
||||
|
||||
subgraph Process["Frame Processing"]
|
||||
direction TB
|
||||
P1{First Frame?}
|
||||
P2[Rotation Sweep]
|
||||
P3[Sequential VO]
|
||||
P4{Tracking OK?}
|
||||
P5[Single Tile Match]
|
||||
P6[Optimize]
|
||||
P7[Publish Result]
|
||||
|
||||
P1 -->|Yes| P2
|
||||
P1 -->|No| P3
|
||||
P2 --> P3
|
||||
P3 --> P4
|
||||
P4 -->|Yes| P5
|
||||
P5 --> P6
|
||||
P6 --> P7
|
||||
end
|
||||
|
||||
subgraph Recovery["Recovery Flow"]
|
||||
direction TB
|
||||
R1[Create Chunk]
|
||||
R2[Progressive Search]
|
||||
R3{Match Found?}
|
||||
R4[Build Chunk]
|
||||
R5[Chunk Matching]
|
||||
R6{Chunk Match?}
|
||||
R7[Merge Chunk]
|
||||
R8[Request User Input]
|
||||
R9[Apply User Anchor]
|
||||
|
||||
P4 -->|No| R1
|
||||
R1 --> R2
|
||||
R2 --> R3
|
||||
R3 -->|Yes| P6
|
||||
R3 -->|No| R4
|
||||
R4 --> R5
|
||||
R5 --> R6
|
||||
R6 -->|Yes| R7
|
||||
R7 --> P6
|
||||
R6 -->|No| R8
|
||||
R8 --> R9
|
||||
R9 --> P6
|
||||
end
|
||||
|
||||
Init --> Flight
|
||||
FL3 --> Process
|
||||
```
|
||||
|
||||
## Event Flow Diagram
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Events["F11 Emits Events"]
|
||||
E1[RecoveryStarted]
|
||||
E2[RecoverySucceeded]
|
||||
E3[RecoveryFailed]
|
||||
E4[UserInputNeeded]
|
||||
E5[UserFixApplied]
|
||||
E6[ChunkCreated]
|
||||
E7[ChunkAnchored]
|
||||
E8[ChunkMerged]
|
||||
end
|
||||
|
||||
subgraph F02Sub["F02 Subscribes"]
|
||||
S1[Update status: recovering]
|
||||
S2[Update status: processing]
|
||||
S3[Update status: blocked]
|
||||
S4[Update status: blocked]
|
||||
S5[Resume processing]
|
||||
S6[Log chunk creation]
|
||||
S7[Log anchor]
|
||||
S8[Trigger result update]
|
||||
end
|
||||
|
||||
subgraph F14Sub["F14 Subscribes"]
|
||||
R1[Update merged frame results]
|
||||
end
|
||||
|
||||
E1 --> S1
|
||||
E2 --> S2
|
||||
E3 --> S3
|
||||
E4 --> S4
|
||||
E5 --> S5
|
||||
E6 --> S6
|
||||
E7 --> S7
|
||||
E8 --> S8
|
||||
E8 --> R1
|
||||
```
|
||||
|
||||
## Data Flow Through Layers
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Input["Input"]
|
||||
IMG[UAV Images]
|
||||
SAT[Satellite Tiles]
|
||||
USR[User Anchors]
|
||||
end
|
||||
|
||||
subgraph Processing["Processing"]
|
||||
SP[SuperPoint Features]
|
||||
LG[LightGlue Matches]
|
||||
DINO[DINOv2 Descriptors]
|
||||
LITE[LiteSAM Homography]
|
||||
end
|
||||
|
||||
subgraph State["State Estimation"]
|
||||
REL[Relative Factors]
|
||||
ABS[Absolute Factors]
|
||||
CHUNK[Chunk Subgraphs]
|
||||
OPT[iSAM2 Optimization]
|
||||
end
|
||||
|
||||
subgraph Output["Output"]
|
||||
ENU[ENU Trajectory]
|
||||
GPS[GPS Coordinates]
|
||||
SSE[SSE Events]
|
||||
DB[Database]
|
||||
end
|
||||
|
||||
IMG --> SP
|
||||
IMG --> DINO
|
||||
IMG --> LITE
|
||||
SAT --> LITE
|
||||
SAT --> DINO
|
||||
USR --> ABS
|
||||
|
||||
SP --> LG
|
||||
LG --> REL
|
||||
DINO --> ABS
|
||||
LITE --> ABS
|
||||
|
||||
REL --> CHUNK
|
||||
ABS --> CHUNK
|
||||
CHUNK --> OPT
|
||||
|
||||
OPT --> ENU
|
||||
ENU --> GPS
|
||||
GPS --> SSE
|
||||
GPS --> DB
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user