azaion/gps-denied-desktop
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-desktop.git synced 2026-04-23 06:16:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

put rest and sse to acceptance criteria. revise components. add system flows diagram

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-30 01:02:07 +02:00
parent ef75cc5877
commit 1082316660
17 changed files with 1906 additions and 434 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/02_components/01_flight_api/flight_api_spec.md
+48
View File
@@ -43,6 +43,10 @@ class IFlightAPI(ABC):
@abstractmethod
def create_sse_stream(self, flight_id: str) -> SSEStream:
pass
@abstractmethod
def convert_object_to_gps(self, flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> ObjectGPSResponse:
pass
```
## Component Description
@@ -367,6 +371,50 @@ UserFixResponse:
---
### `convert_object_to_gps(flight_id: str, frame_id: int, pixel: Tuple[float, float]) -> ObjectGPSResponse`
**REST Endpoint**: `POST /flights/{flightId}/frames/{frameId}/object-to-gps`
**Description**: Converts object pixel coordinates to GPS. Used by external object detection systems (e.g., Azaion.Inference) to get GPS coordinates for detected objects.
**Called By**:
- External object detection systems (Azaion.Inference)
- Any system needing pixel-to-GPS conversion for a specific frame
**Input**:
```python
ObjectToGPSRequest:
pixel_x: float # X coordinate in image
pixel_y: float # Y coordinate in image
```
**Output**:
```python
ObjectGPSResponse:
gps: GPSPoint
accuracy_meters: float # Estimated accuracy
frame_id: int
pixel: Tuple[float, float]
```
**Processing Flow**:
1. Validate flight_id and frame_id exist
2. Validate frame has been processed (has pose in Factor Graph)
3. Call F13.image_object_to_gps(pixel, frame_id)
4. Return GPS with accuracy estimate
**Error Conditions**:
- `400 Bad Request`: Invalid pixel coordinates
- `404 Not Found`: flight_id or frame_id not found
- `409 Conflict`: Frame not yet processed (no pose available)
**Test Cases**:
1. **Valid conversion**: Object at (1024, 768) → returns GPS
2. **Unprocessed frame**: Frame not in Factor Graph → returns 409
3. **Invalid pixel**: Negative coordinates → returns 400
---
### `get_flight_status(flight_id: str) -> FlightStatusResponse`
**REST Endpoint**: `GET /flights/{flightId}/status`
docs/02_components/02_flight_processor/flight_processor_spec.md
+25
View File
@@ -113,6 +113,31 @@ class IFlightProcessor(ABC):
- System initialization and resource management
- Flight state machine management
- **Chunk-aware frame processing (Atlas multi-map architecture)**
- **Background task management for each flight**
- **Event subscription for F11 recovery events**
### Event-Based Communication
F02 subscribes to events from F11 Failure Recovery Coordinator instead of F11 directly calling F02's status update methods. This decouples recovery logic from flight state management.
**Events Subscribed (from F11)**:
- `RecoveryStarted`: Update flight status to "recovering"
- `RecoverySucceeded`: Update flight status to "processing", resume processing loop
- `RecoveryFailed`: Update flight status to "blocked", blocked=True
- `UserInputNeeded`: Update flight status to "blocked", blocked=True, await user fix
- `UserFixApplied`: Update flight status to "processing", resume processing loop
- `ChunkCreated`: Log chunk creation, update internal tracking
- `ChunkAnchored`: Log chunk anchor, update statistics
- `ChunkMerged`: Trigger result updates via F14
### Background Task Management
F02 is responsible for managing background tasks per flight:
- **Chunk matching task**: Background task for matching unanchored chunks (delegated to F11)
- **Asynchronous refinement task**: Background optimization and result publishing (via F10, F14)
- **Task lifecycle**: Tasks created on flight start, cancelled on flight completion/deletion
Background tasks are coordinated via Python `asyncio` or `ThreadPoolExecutor`, with F02 as the owner. F11 executes chunk matching logic but F02 schedules and monitors the tasks.
---
docs/02_components/03_flight_database/flight_database_spec.md
+15
View File
@@ -130,6 +130,21 @@ class IFlightDatabase(ABC):
- Image metadata storage
- Query optimization for large datasets
### Design Decision: Denormalization
The schema uses strategic denormalization to optimize for the most common access patterns:
**Denormalized Fields**:
- `frame_results` stores `gps_lat`, `gps_lon` directly (not as foreign key to waypoints)
- `flight_state` duplicates `frames_processed` (could be computed from frame_results)
- `chunks.frames` stored as JSONB array (not normalized into separate frame_chunk mapping table)
**Rationale**:
- Read-heavy workload: Frame results are read 100x more than written
- Avoids JOINs in critical path (per-frame processing)
- Simplifies chunk lifecycle (all chunk data in single row)
- Acceptable trade-off: Slightly increased storage, significantly faster reads
---
## Flight Operations
docs/02_components/04_satellite_data_manager/satellite_data_manager_spec.md
+2 -2
View File
@@ -25,11 +25,11 @@ class ISatelliteDataManager(ABC):
pass
@abstractmethod
def cache_tile(self, tile_coords: TileCoords, tile_data: np.ndarray) -> bool:
def cache_tile(self, flight_id: str, tile_coords: TileCoords, tile_data: np.ndarray) -> bool:
pass
@abstractmethod
def get_cached_tile(self, tile_coords: TileCoords) -> Optional[np.ndarray]:
def get_cached_tile(self, flight_id: str, tile_coords: TileCoords) -> Optional[np.ndarray]:
pass
@abstractmethod
docs/02_components/05_image_input_pipeline/image_input_pipeline_spec.md
+2 -3
View File
@@ -350,7 +350,7 @@ ProcessingStatus:
```
**Processing Flow**:
1. Get flight state via F02 Flight Processor.get_flight_state(flight_id)
1. Get flight state via F03 Flight Database.get_flight(flight_id).status
2. Combine with internal queue status
3. Return ProcessingStatus
@@ -402,8 +402,7 @@ ProcessingStatus:
### Internal Components
- **H08 Batch Validator**: For validation logic
- **F03 Flight Database**: For metadata persistence
- **F02 Flight Processor**: For flight state information
- **F03 Flight Database**: For metadata persistence and flight state information
### External Dependencies
- **opencv-python**: Image I/O
docs/02_components/06_image_rotation_manager/image_rotation_manager_spec.md
+8 -1
View File
@@ -117,10 +117,17 @@ flight_id: str
frame_id: int # Frame identifier for heading persistence
image: np.ndarray # UAV image
satellite_tile: np.ndarray # Satellite reference tile
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (for F09)
tile_bounds: TileBounds # GPS bounds and GSD of satellite tile (passed to F09)
timestamp: datetime # Timestamp for heading persistence
```
**About tile_bounds**: `TileBounds` contains the GPS bounding box of the satellite tile:
- `nw`, `ne`, `sw`, `se`: GPS coordinates of tile corners
- `center`: GPS coordinate of tile center
- `gsd`: Ground Sampling Distance (meters/pixel)
The caller (F02 Flight Processor) obtains tile_bounds by calling `F04.compute_tile_bounds(tile_coords)` before calling this method. F06 passes tile_bounds to F09.align_to_satellite() which uses it to convert pixel coordinates to GPS.
**Output**:
```python
RotationResult:
docs/02_components/08_global_place_recognition/global_place_recognition_spec.md
+30 -36
View File
@@ -25,7 +25,7 @@ class IGlobalPlaceRecognition(ABC):
pass
@abstractmethod
def initialize_database(self, satellite_tiles: List[SatelliteTile]) -> bool:
def load_index(self, flight_id: str, index_path: str) -> bool:
pass
@abstractmethod
@@ -45,7 +45,7 @@ class IGlobalPlaceRecognition(ABC):
- Compute image descriptors robust to season/appearance changes
- Query Faiss index of satellite tile descriptors
- Return top-k candidate tile regions for progressive refinement
- Initialize satellite descriptor database during system startup
- **Load pre-built satellite descriptor index** (index is built by satellite provider, NOT by F08)
- **Chunk semantic matching (aggregate DINOv2 features)**
- **Chunk descriptor computation for robust matching**
@@ -203,54 +203,48 @@ List[TileCandidate] # Re-ranked list
---
### `initialize_database(satellite_tiles: List[SatelliteTile]) -> bool`
### `load_index(flight_id: str, index_path: str) -> bool`
**Description**: Loads pre-built satellite descriptor database. **Note**: Semantic index building (DINOv2 descriptors) is performed by the satellite provider service, not during system startup.
**Description**: Loads pre-built satellite descriptor database from file. **Note**: The semantic index (DINOv2 descriptors + Faiss index) MUST be provided by the satellite data provider. F08 does NOT build the index - it only loads it.
**Called By**:
- F02 Flight Processor (during system initialization)
- F02 Flight Processor (during flight initialization, index_path from F04 Satellite Data Manager)
**Input**:
```python
List[SatelliteTile]:
tile_id: str
image: np.ndarray # Optional - only if building index locally
gps_center: GPSPoint
bounds: TileBounds
descriptor: Optional[np.ndarray] # Pre-computed descriptor from provider
index_path: str # Path to pre-built Faiss index file from satellite provider
```
**Output**:
```python
bool: True if database initialized successfully
bool: True if database loaded successfully
```
**Processing Flow**:
1. **Load pre-built index**: Satellite provider provides pre-computed DINOv2 descriptors
2. If descriptors provided:
- Load descriptors directly
- Build Faiss index using H04 Faiss Index Manager
3. If descriptors not provided (fallback):
- For each satellite tile:
- compute_location_descriptor(tile.image) → descriptor
- Store descriptor with tile metadata
- Build Faiss index
4. Persist index to disk for fast startup
1. Load pre-built Faiss index from index_path
2. Load tile metadata (tile_id → gps_center, bounds mapping)
3. Validate index integrity (check descriptor dimensions, tile count)
4. Return True if loaded successfully
**Satellite Provider Integration**:
- **Primary**: Satellite provider builds semantic index offline
- Provider exposes index via API or file download
- F08 loads pre-built index at startup
- **Fallback**: F08 can build index locally if provider doesn't supply it
**Satellite Provider Responsibility**:
- Satellite provider builds the semantic index offline using DINOv2 + VLAD
- Provider delivers index file along with satellite tiles
- Index format: Faiss IVF or HNSW index + tile metadata JSON
- Provider is responsible for index updates when satellite data changes
**Error Conditions**:
- Raises `IndexNotFoundError`: Index file not found
- Raises `IndexCorruptedError`: Index file corrupted or invalid format
- Raises `MetadataMismatchError`: Metadata doesn't match index
**Performance**:
- **Load pre-built index**: <10 seconds (fast startup)
- **Build index locally**: ~10-30 minutes for 10,000 tiles (fallback only)
- **Load time**: <10 seconds for 10,000+ tiles
**Test Cases**:
1. **Load pre-built index**: Completes successfully, fast startup
2. **Fallback local building**: Builds index if provider doesn't supply it
3. **Index query**: Works correctly after loading
1. **Load valid index**: Completes successfully, index operational
2. **Index not found**: Raises IndexNotFoundError
3. **Corrupted index**: Raises IndexCorruptedError
4. **Index query after load**: Works correctly
---
@@ -351,10 +345,10 @@ np.ndarray: Aggregated descriptor vector (4096-dim or 8192-dim)
2. UAV images from autumn
3. retrieve_candidate_tiles() → correct match despite appearance change
### Test 3: Database Initialization
1. Prepare 500 satellite tiles
2. initialize_database(tiles)
3. Verify Faiss index built
### Test 3: Index Loading
1. Prepare pre-built index file from satellite provider
2. load_index(index_path)
3. Verify Faiss index loaded correctly
4. Query with test image → returns matches
### Test 4: Chunk Semantic Matching
docs/02_components/10_factor_graph_optimizer/factor_graph_optimizer_spec.md
+40 -3
View File
@@ -78,10 +78,28 @@ class IFactorGraphOptimizer(ABC):
- Scale resolution through altitude priors and absolute GPS
- Trajectory smoothing and global consistency
- Back-propagation of refinements to previous frames
- **Native multi-chunk/multi-map support (Atlas architecture)**
- **Chunk lifecycle management (creation, optimization, merging)**
- **Low-level factor graph chunk operations** (subgraph creation, factor addition, optimization)
- **Sim(3) transformation for chunk merging**
### Chunk Responsibility Clarification
**F10 provides low-level factor graph operations only**:
- `create_new_chunk()`: Creates subgraph in factor graph
- `add_relative_factor_to_chunk()`: Adds factors to chunk's subgraph
- `add_chunk_anchor()`: Adds GPS anchor to chunk
- `merge_chunks()`: Applies Sim(3) transform and merges subgraphs
- `optimize_chunk()`, `optimize_global()`: Runs optimization
**F12 is the source of truth for chunk state** (see F12 spec):
- Chunk lifecycle management (active, anchored, merged status)
- Chunk readiness determination
- High-level chunk queries
**F11 coordinates recovery** (see F11 spec):
- Triggers chunk creation via F12
- Coordinates matching workflows
- Emits chunk-related events
### Scope
- Non-linear least squares optimization
- Factor graph representation of SLAM problem
@@ -92,6 +110,20 @@ class IFactorGraphOptimizer(ABC):
- **Chunk-level optimization and global merging**
- **Sim(3) similarity transformation for chunk alignment**
### Design Pattern: Composition Over Complex Interface
F10 uses **composition** to keep the interface manageable. Rather than exposing 20+ methods in a monolithic interface, complex operations are composed from simpler primitives:
**Primitive Operations**:
- `add_relative_factor()`, `add_absolute_factor()`, `add_altitude_prior()` - Factor management
- `optimize()`, `get_trajectory()`, `get_marginal_covariance()` - Core optimization
**Chunk Operations** (composed from primitives):
- `create_new_chunk()`, `add_relative_factor_to_chunk()`, `add_chunk_anchor()` - Chunk factor management
- `merge_chunks()`, `optimize_chunk()`, `optimize_global()` - Chunk optimization
**Callers compose these primitives** for complex workflows (e.g., F11 composes anchor + merge + optimize_global).
## API Methods
### `add_relative_factor(frame_i: int, frame_j: int, relative_pose: RelativePose, covariance: np.ndarray) -> bool`
@@ -123,12 +155,17 @@ F07 returns unit translation vectors due to monocular scale ambiguity. F10 resol
**Explicit Flow**:
```python
# In add_relative_factor():
# altitude comes from F05 Image Input Pipeline (extracted from EXIF metadata)
# focal_length, sensor_width from F17 Configuration Manager
gsd = H02.compute_gsd(altitude, focal_length, sensor_width, image_width)
expected_displacement = frame_spacing * gsd # ~100m
expected_displacement = frame_spacing * gsd # ~100m typical at 300m altitude
scaled_translation = relative_pose.translation * expected_displacement
# Add scaled_translation to factor graph
```
**Note**: Altitude is passed through the processing chain:
- F05 extracts altitude from EXIF → F02 includes in FrameData → F10 receives with add_relative_factor()
**Output**:
```python
bool: True if factor added successfully
docs/02_components/11_failure_recovery_coordinator/failure_recovery_coordinator_spec.md
+62 -14
View File
@@ -57,7 +57,7 @@ class IFailureRecoveryCoordinator(ABC):
pass
@abstractmethod
def merge_chunk_to_trajectory(self, chunk_id: str, alignment_result: ChunkAlignmentResult) -> bool:
def merge_chunk_to_trajectory(self, flight_id: str, chunk_id: str, alignment_result: ChunkAlignmentResult) -> bool:
pass
@abstractmethod
@@ -72,14 +72,45 @@ class IFailureRecoveryCoordinator(ABC):
- Detect tracking loss and trigger recovery
- Coordinate progressive tile search (1→4→9→16→25)
- Handle human-in-the-loop when all strategies exhausted
- Block flight processing when awaiting user input
- **Emit recovery events** (RecoveryStarted, RecoveryFailed, RecoverySucceeded, UserInputNeeded)
- Apply user-provided anchors to Factor Graph
- **Proactive chunk creation on tracking loss**
- **Chunk semantic matching coordination**
- **Proactive chunk creation on tracking loss** (via F12)
- **Chunk LiteSAM matching with rotation sweeps**
- **Chunk merging orchestration**
- **Background chunk matching processing**
### Chunk Responsibility Clarification
**F11 coordinates chunk-based recovery workflow only**:
- Detects when to create chunks (on tracking loss)
- Calls F12 for ALL chunk operations (F11 NEVER directly calls F10 chunk methods)
- Coordinates matching workflows
- Emits chunk-related events (ChunkCreated, ChunkAnchored, ChunkMerged)
**F12 is the source of truth for chunk state** (see F12 spec)
**F10 provides low-level factor graph operations** (see F10 spec)
### Event-Based Communication
F11 emits events instead of directly calling F02's status update methods. This decouples recovery logic from flight state management.
**Events Emitted**:
- `RecoveryStarted`: When tracking loss detected and recovery begins
- `RecoverySucceeded`: When recovery finds a match (single-image or chunk)
- `RecoveryFailed`: When all recovery strategies exhausted
- `UserInputNeeded`: When user input is required
- `UserFixApplied`: When user-provided anchor successfully applied
- `ChunkCreated`: When new chunk created on tracking loss
- `ChunkAnchored`: When chunk successfully matched and anchored
- `ChunkMerged`: When chunk merged into main trajectory (includes flight_id, chunk_id, merged_frames)
**Event Listeners** (F02 Flight Processor subscribes to these):
- On `RecoveryStarted`: Update status to "recovering"
- On `RecoveryFailed`: Update status to "blocked"
- On `RecoverySucceeded`: Update status to "processing"
- On `UserInputNeeded`: Update status to "blocked", blocked=True
### Scope
- Confidence monitoring
- Progressive search coordination
@@ -298,10 +329,10 @@ UserInputRequest:
**Processing Flow**:
1. Get UAV image for frame_id
2. Get top-5 candidates from G08
2. Get top-5 candidates from F08
3. Create request
4. Send via F14 SSE → "user_input_needed" event
5. Update F02 flight_status("BLOCKED")
4. Send via F15 SSE → "user_input_needed" event
5. Emit `UserInputNeeded` event (F02 subscribes and updates status to "BLOCKED")
**Test Cases**:
1. All search failed → creates request
@@ -330,7 +361,7 @@ anchor: UserAnchor:
1. Validate anchor data
2. Call F10.add_absolute_factor(frame_id, gps, is_user_anchor=True)
3. F10.optimize() → refines trajectory
4. Update F02 flight_status("PROCESSING")
4. Emit `UserFixApplied` event (F02 subscribes and updates status to "PROCESSING")
5. Resume processing from next frame
**Test Cases**:
@@ -482,15 +513,20 @@ bool: True if merge successful
```
**Processing Flow**:
1. Get chunk frames via F12.get_chunk_frames(chunk_id) → merged_frames (all frames in chunk that will be updated)
1. Get chunk frames via F12.get_chunk_frames(chunk_id) → merged_frames
2. Get chunk anchor frame (middle frame or best frame)
3. Call F12.mark_chunk_anchored() with GPS (F12 coordinates with F10)
4. Find target chunk (previous chunk or main trajectory)
4. **Resolve target chunk**:
- Query F12.get_merge_target(chunk_id) → returns target_chunk_id
- Target selection logic (inside F12):
- If chunk has temporal predecessor (previous chunk by frame_id order): merge to predecessor
- If no predecessor: merge to main trajectory (chunk_id="main")
- F12 maintains chunk ordering based on first frame_id in each chunk
5. Call F12.merge_chunks(chunk_id, target_chunk_id, transform) (F12 coordinates with F10)
6. F12 handles chunk state updates (deactivation, status updates)
7. F10 optimizes merged graph globally (via F12.merge_chunks())
8. **Update results**: Call F14 Result Manager.update_results_after_chunk_merge(flight_id, merged_frames)
- F14 retrieves updated poses from F10, converts ENU to GPS, updates database, and publishes via SSE
8. **Emit ChunkMerged event** with flight_id and merged_frames
- F14 subscribes and updates results: retrieves poses from F10, converts ENU to GPS, updates database, publishes via SSE
9. Return True
**Sim(3) Transform**:
@@ -499,7 +535,7 @@ bool: True if merge successful
- Scale: Resolved from altitude and GSD
**Test Cases**:
1. **Merge chunk**: Chunk merged successfully, results updated via F14
1. **Merge successful**: Chunks merged successfully, results updated via F14
2. **Global consistency**: Merged trajectory globally consistent
3. **Multiple chunks**: Can merge multiple chunks sequentially
4. **Result updates**: All merged frames have updated GPS coordinates published
@@ -622,9 +658,10 @@ while flight_active:
- F10 Factor Graph Optimizer (anchor application and chunk merging)
- F12 Route Chunk Manager (chunk lifecycle)
- F14 Result Manager (result updates after chunk merging)
- F02 Flight Manager (status updates)
- F15 SSE Event Streamer (user input events)
**Note**: F11 does NOT directly call F02. Instead, F11 emits events (RecoveryStarted, RecoveryFailed, etc.) which F02 subscribes to for status updates. This decouples recovery logic from flight state management.
### External Dependencies
- None
@@ -708,3 +745,14 @@ class Sim3Transform(BaseModel):
scale: float
```
### RecoveryEvent
```python
class RecoveryEvent(BaseModel):
event_type: str # "RecoveryStarted", "RecoverySucceeded", "RecoveryFailed", "UserInputNeeded", "ChunkCreated", "ChunkAnchored", "ChunkMerged"
flight_id: str
frame_id: Optional[int]
chunk_id: Optional[str]
data: Optional[Dict[str, Any]]
timestamp: datetime
```
docs/02_components/12_route_chunk_manager/route_chunk_manager_spec.md
+33 -1
View File
@@ -56,19 +56,51 @@ class IRouteChunkManager(ABC):
def merge_chunks(self, chunk_id_1: str, chunk_id_2: str, transform: Sim3Transform) -> bool:
pass
@abstractmethod
def get_merge_target(self, chunk_id: str) -> str:
"""Returns target chunk_id for merging. Returns 'main' for main trajectory."""
pass
@abstractmethod
def mark_chunk_matching(self, chunk_id: str) -> bool:
pass
@abstractmethod
def save_chunk_state(self, flight_id: str) -> bool:
"""Persist all chunk state to F03 for crash recovery."""
pass
@abstractmethod
def load_chunk_state(self, flight_id: str) -> bool:
"""Load chunk state from F03 on restart."""
pass
```
## Component Description
### Responsibilities
- **Source of truth for chunk state** (active, anchored, merged status)
- Manage chunk lifecycle (creation, activation, deactivation, merging)
- Track chunk state (frames, anchors, matching status)
- Coordinate chunk semantic matching and LiteSAM matching
- Provide chunk representations for matching (composite images, descriptors)
- Determine chunk readiness for matching (min frames, consistency)
- Persist chunk state via F03 Flight Database
### Chunk Responsibility Clarification
**F12 is the high-level chunk manager**:
- Owns chunk state (ChunkHandle with matching_status, is_active, has_anchor)
- Provides high-level queries: `get_active_chunk()`, `get_chunks_for_matching()`
- Coordinates with F10 for factor graph operations
- Persists chunk state for crash recovery
**F10 provides low-level factor graph operations** (see F10 spec):
- Subgraph creation and factor management
- Does NOT own chunk state - only factor graph data
**F11 coordinates recovery** (see F11 spec):
- Calls F12 for chunk operations
- F11 NEVER directly calls F10 chunk methods
### Scope
- Chunk lifecycle management
docs/02_components/13_coordinate_transformer/coordinate_transformer_spec.md
+11 -56
View File
@@ -10,19 +10,19 @@
class ICoordinateTransformer(ABC):
# ENU Origin Management
@abstractmethod
def set_enu_origin(self, origin_gps: GPSPoint) -> None:
def set_enu_origin(self, flight_id: str, origin_gps: GPSPoint) -> None:
pass
@abstractmethod
def get_enu_origin(self) -> GPSPoint:
def get_enu_origin(self, flight_id: str) -> GPSPoint:
pass
@abstractmethod
def gps_to_enu(self, gps: GPSPoint) -> Tuple[float, float, float]:
def gps_to_enu(self, flight_id: str, gps: GPSPoint) -> Tuple[float, float, float]:
pass
@abstractmethod
def enu_to_gps(self, enu: Tuple[float, float, float]) -> GPSPoint:
def enu_to_gps(self, flight_id: str, enu: Tuple[float, float, float]) -> GPSPoint:
pass
# Pixel/GPS Conversions
@@ -38,10 +38,6 @@ class ICoordinateTransformer(ABC):
def image_object_to_gps(self, object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint:
pass
@abstractmethod
def compute_gsd(self, altitude: float, focal_length: float, sensor_width: float, image_width: int) -> float:
pass
@abstractmethod
def transform_points(self, points: List[Tuple[float, float]], transformation: np.ndarray) -> List[Tuple[float, float]]:
pass
@@ -53,10 +49,11 @@ class ICoordinateTransformer(ABC):
- Pixel-to-GPS coordinate conversions
- GPS-to-pixel inverse projections
- **Critical**: Convert object pixel coordinates (from external detection system) to GPS
- Ground Sampling Distance (GSD) calculations
- Handle multiple coordinate systems: WGS84, Web Mercator, ENU, image pixels, rotated coordinates
- Camera model integration for projection operations
**Note**: GSD calculations are delegated to H02 GSD Calculator helper.
### Scope
- Coordinate system transformations
- Camera projection mathematics
@@ -278,11 +275,11 @@ Tuple[float, float]: (x, y) pixel coordinates
### `image_object_to_gps(object_pixel: Tuple[float, float], frame_id: int) -> GPSPoint`
**Description**: **Critical method** - Converts object pixel coordinates to GPS. Used by external object detection system.
**Description**: **Critical method** - Converts object pixel coordinates to GPS. Used for external object detection integration.
**Called By**:
- External object detection system (provides pixel coordinates)
- F13 Result Manager (converts objects to GPS for output)
- F01 Flight API (via `POST /flights/{flightId}/frames/{frameId}/object-to-gps` endpoint)
- F14 Result Manager (converts objects to GPS for output)
**Input**:
```python
@@ -316,47 +313,6 @@ GPSPoint: GPS coordinates of object center
---
### `compute_gsd(altitude: float, focal_length: float, sensor_width: float, image_width: int) -> float`
**Description**: Computes Ground Sampling Distance (meters per pixel).
**Called By**:
- Internal (for pixel_to_gps)
- F09 Metric Refinement (for scale calculations)
- H02 GSD Calculator (may delegate to)
**Input**:
```python
altitude: float # meters
focal_length: float # mm
sensor_width: float # mm
image_width: int # pixels
```
**Output**:
```python
float: GSD in meters/pixel
```
**Formula**:
```
GSD = (altitude * sensor_width) / (focal_length * image_width)
```
**Example**:
- altitude = 800m
- focal_length = 24mm
- sensor_width = 36mm
- image_width = 6000px
- GSD = (800 * 36) / (24 * 6000) = 0.2 m/pixel
**Test Cases**:
1. **Standard parameters**: Returns reasonable GSD (~0.1-0.3 m/pixel)
2. **Higher altitude**: GSD increases
3. **Longer focal length**: GSD decreases
---
### `transform_points(points: List[Tuple[float, float]], transformation: np.ndarray) -> List[Tuple[float, float]]`
**Description**: Applies homography or affine transformation to list of points.
@@ -406,8 +362,8 @@ List[Tuple[float, float]]: Transformed points
3. pixel_to_gps() → GPS
4. Verify GPS matches original (within tolerance)
### Test 4: GSD Calculation
1. compute_gsd() with known parameters
### Test 4: GSD via H02
1. Call H02.compute_gsd() with known parameters
2. Verify matches expected value
3. Test at different altitudes
@@ -417,7 +373,6 @@ List[Tuple[float, float]]: Transformed points
- **pixel_to_gps**: < 5ms
- **gps_to_pixel**: < 5ms
- **image_object_to_gps**: < 10ms
- **compute_gsd**: < 1ms
### Accuracy
- **GPS accuracy**: Inherits from Factor Graph accuracy (~20m)
docs/02_components/14_result_manager/result_manager_spec.md
+7 -2
View File
@@ -27,6 +27,10 @@ class IResultManager(ABC):
@abstractmethod
def get_changed_frames(self, flight_id: str, since: datetime) -> List[int]:
pass
@abstractmethod
def update_results_after_chunk_merge(self, flight_id: str, merged_frames: List[int]) -> bool:
pass
```
## Component Description
@@ -187,8 +191,9 @@ since: datetime
**Description**: Updates frame results after chunk merging changes frame poses.
**Called By**:
- F11 Failure Recovery Coordinator (after chunk merging)
**Triggered By**:
- `ChunkMerged` event from F11 (F14 subscribes to this event)
- Alternative: F02 Flight Processor can coordinate this call after receiving ChunkMerged event
**Input**:
```python
docs/02_components/astral_next_components_diagram.drawio
+83 -303
View File
@@ -1,355 +1,135 @@
<mxfile host="65bd71144e">
<diagram name="ASTRAL-Next Components" id="astral-next-components">
<mxGraphModel dx="1440" dy="1201" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1800" pageHeight="2800" math="0" shadow="0">
<mxGraphModel dx="771" dy="632" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="900" pageHeight="500" math="0" shadow="0">
<root>
<mxCell id="0"/>
<mxCell id="1" parent="0"/>
<mxCell id="title" value="ASTRAL-Next System Architecture&#10;GPS-Denied Localization for UAVs (Atlas Multi-Map Chunk Architecture)&#10;25 Components: Route API (4) + Flight API (17) + Helpers (8)" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=20;fontStyle=1;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="200" y="20" width="1200" height="80" as="geometry"/>
<mxCell id="title" value="ASTRAL-Next Component Connections" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="280" y="10" width="300" height="25" as="geometry"/>
</mxCell>
<mxCell id="route-api-lane" value="Route API (Separate Project)" style="swimlane;horizontal=1;whiteSpace=wrap;html=1;fontSize=14;fontStyle=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="200" y="110" width="600" height="350" as="geometry"/>
<mxCell id="client" value="Client" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="20" y="50" width="60" height="35" as="geometry"/>
</mxCell>
<mxCell id="r01" value="R01&#10;Route REST API" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
<mxGeometry x="40" y="50" width="120" height="60" as="geometry"/>
<mxCell id="f01" value="F01 API" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="100" y="50" width="70" height="35" as="geometry"/>
</mxCell>
<mxCell id="r02" value="R02&#10;Route Data Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
<mxGeometry x="200" y="50" width="140" height="60" as="geometry"/>
<mxCell id="f02" value="F02&#10;Flight Processor" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;fontStyle=1;" parent="1" vertex="1">
<mxGeometry x="195" y="70" width="110" height="45" as="geometry"/>
</mxCell>
<mxCell id="r03" value="R03&#10;Waypoint Validator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
<mxGeometry x="380" y="50" width="140" height="60" as="geometry"/>
<mxCell id="f03" value="F03 DB" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="340" y="50" width="70" height="35" as="geometry"/>
</mxCell>
<mxCell id="r04" value="R04&#10;Route Database Layer" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
<mxGeometry x="200" y="150" width="140" height="60" as="geometry"/>
<mxCell id="f04" value="F04&#10;Satellite Mgr" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="60" y="367.5" width="80" height="45" as="geometry"/>
</mxCell>
<mxCell id="r01-r02" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r01" target="r02" edge="1">
<mxCell id="f05" value="F05&#10;Image Pipeline" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="550" y="45" width="80" height="45" as="geometry"/>
</mxCell>
<mxCell id="satellite" value="Satellite&#10;Provider" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFB300;fontColor=#ffffff;dashed=1;" vertex="1" parent="1">
<mxGeometry x="660" y="45" width="70" height="45" as="geometry"/>
</mxCell>
<mxCell id="f15" value="F15 SSE" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="20" y="120" width="60" height="35" as="geometry"/>
</mxCell>
<mxCell id="f11" value="F11&#10;Failure Recovery" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="310" y="120" width="110" height="45" as="geometry"/>
</mxCell>
<mxCell id="f06" value="F06&#10;Rotation" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="100" y="195" width="70" height="45" as="geometry"/>
</mxCell>
<mxCell id="f07" value="F07&#10;Sequential VO" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="200" y="195" width="90" height="45" as="geometry"/>
</mxCell>
<mxCell id="f08" value="F08&#10;Place Recog" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="100" y="120" width="80" height="45" as="geometry"/>
</mxCell>
<mxCell id="f09" value="F09&#10;Metric Refine" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="340" y="195" width="80" height="45" as="geometry"/>
</mxCell>
<mxCell id="f12" value="F12&#10;Chunk Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="450" y="120" width="100" height="45" as="geometry"/>
</mxCell>
<mxCell id="f10" value="F10&#10;Factor Graph" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;fontStyle=1;" parent="1" vertex="1">
<mxGeometry x="330" y="280" width="100" height="45" as="geometry"/>
</mxCell>
<mxCell id="f13" value="F13&#10;Coord Transform" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="390" y="355" width="100" height="45" as="geometry"/>
</mxCell>
<mxCell id="f14" value="F14 Result Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="190" y="372.5" width="120" height="35" as="geometry"/>
</mxCell>
<mxCell id="c1" value="HTTP" style="strokeColor=#FFFFFF;fontColor=#ffffff;fontSize=8;" edge="1" parent="1" source="client" target="f01">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="r02-r03" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r02" target="r03" edge="1">
<mxCell id="c2" value="SSE" style="strokeColor=#FFFFFF;fontColor=#ffffff;fontSize=8;dashed=1;" edge="1" parent="1" source="f15" target="client">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="r02-r04" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r02" target="r04" edge="1">
<mxCell id="c3" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f01" target="f02">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="route-db" value="Route DB&#10;(Separate Schema)" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="route-api-lane" vertex="1">
<mxGeometry x="225" y="240" width="90" height="80" as="geometry"/>
</mxCell>
<mxCell id="r04-db" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="route-api-lane" source="r04" target="route-db" edge="1">
<mxCell id="c4" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f03">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="gps-denied-lane" value="Flight API (Main Processing System - Atlas Multi-Map Architecture)" style="swimlane;horizontal=1;whiteSpace=wrap;html=1;fontSize=14;fontStyle=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="130" y="500" width="1400" height="2400" as="geometry"/>
</mxCell>
<mxCell id="core-layer" value="Core API Layer" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="20" y="40" width="560" height="140" as="geometry"/>
</mxCell>
<mxCell id="f01" value="F01&#10;Flight REST API" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="core-layer" vertex="1">
<mxGeometry x="20" y="40" width="150" height="60" as="geometry"/>
</mxCell>
<mxCell id="f02" value="F02&#10;Flight Processor&#10;(chunk-aware)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="core-layer" vertex="1">
<mxGeometry x="200" y="40" width="150" height="70" as="geometry"/>
</mxCell>
<mxCell id="f03" value="F03&#10;Flight Database" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="core-layer" vertex="1">
<mxGeometry x="380" y="40" width="150" height="60" as="geometry"/>
</mxCell>
<mxCell id="f01-f02" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="core-layer" source="f01" target="f02" edge="1">
<mxCell id="c5" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f11">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="data-layer" value="Data Management" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="600" y="40" width="560" height="140" as="geometry"/>
</mxCell>
<mxCell id="f04" value="F04&#10;Satellite Data Manager&#10;(fetch, cache, grid)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="data-layer" vertex="1">
<mxGeometry x="20" y="40" width="160" height="70" as="geometry"/>
</mxCell>
<mxCell id="f05" value="F05&#10;Image Input Pipeline&#10;(queue, validate, store)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="data-layer" vertex="1">
<mxGeometry x="200" y="40" width="160" height="70" as="geometry"/>
</mxCell>
<mxCell id="f06" value="F06&#10;Image Rotation Mgr&#10;(30° sweep, chunk rotation)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="data-layer" vertex="1">
<mxGeometry x="380" y="40" width="160" height="70" as="geometry"/>
</mxCell>
<mxCell id="visual-layer" value="Visual Processing (Tri-Layer Architecture)" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="20" y="200" width="560" height="140" as="geometry"/>
</mxCell>
<mxCell id="f07" value="F07&#10;Sequential VO&#10;(SuperPoint+LightGlue&#10;chunk-scoped)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="visual-layer" vertex="1">
<mxGeometry x="20" y="40" width="160" height="80" as="geometry"/>
</mxCell>
<mxCell id="f08" value="F08&#10;Global Place Recognition&#10;(AnyLoc DINOv2&#10;chunk descriptors)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="visual-layer" vertex="1">
<mxGeometry x="200" y="40" width="160" height="80" as="geometry"/>
</mxCell>
<mxCell id="f09" value="F09&#10;Metric Refinement&#10;(LiteSAM&#10;chunk matching)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="visual-layer" vertex="1">
<mxGeometry x="380" y="40" width="160" height="80" as="geometry"/>
</mxCell>
<mxCell id="state-layer" value="State Estimation &amp; Coordination (Atlas Multi-Map)" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="600" y="200" width="760" height="200" as="geometry"/>
</mxCell>
<mxCell id="f10" value="F10&#10;Factor Graph Optimizer&#10;(GTSAM iSAM2&#10;multi-chunk, Sim3)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="state-layer" vertex="1">
<mxGeometry x="20" y="40" width="160" height="80" as="geometry"/>
</mxCell>
<mxCell id="f11" value="F11&#10;Failure Recovery&#10;(Progressive 1→25&#10;chunk matching)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="state-layer" vertex="1">
<mxGeometry x="200" y="40" width="160" height="80" as="geometry"/>
</mxCell>
<mxCell id="f12" value="F12&#10;Route Chunk Manager&#10;(Atlas lifecycle&#10;chunk matching)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="state-layer" vertex="1">
<mxGeometry x="380" y="40" width="160" height="80" as="geometry"/>
</mxCell>
<mxCell id="f13" value="F13&#10;Coordinate Transform&#10;(Pixel↔GPS)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="state-layer" vertex="1">
<mxGeometry x="560" y="40" width="160" height="70" as="geometry"/>
</mxCell>
<mxCell id="results-layer" value="Results &amp; Communication" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="20" y="360" width="380" height="140" as="geometry"/>
</mxCell>
<mxCell id="f14" value="F14&#10;Result Manager" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="results-layer" vertex="1">
<mxGeometry x="20" y="40" width="150" height="60" as="geometry"/>
</mxCell>
<mxCell id="f15" value="F15&#10;SSE Event Streamer" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="results-layer" vertex="1">
<mxGeometry x="200" y="40" width="150" height="60" as="geometry"/>
</mxCell>
<mxCell id="f14-f15" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="results-layer" source="f14" target="f15" edge="1">
<mxCell id="c6" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f07">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="infra-layer" value="Infrastructure" style="swimlane;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="420" y="360" width="740" height="140" as="geometry"/>
</mxCell>
<mxCell id="f16" value="F16&#10;Model Manager&#10;(TensorRT/ONNX)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="infra-layer" vertex="1">
<mxGeometry x="20" y="40" width="160" height="70" as="geometry"/>
</mxCell>
<mxCell id="f17" value="F17&#10;Configuration Mgr" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="infra-layer" vertex="1">
<mxGeometry x="200" y="40" width="160" height="70" as="geometry"/>
</mxCell>
<mxCell id="gps-db" value="GPS-Denied DB&#10;(Separate Schema)&#10;Flights, Frame Results" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="infra-layer" vertex="1">
<mxGeometry x="570" y="30" width="140" height="90" as="geometry"/>
</mxCell>
<mxCell id="helpers-lane" value="Helper Components (Shared Utilities)" style="swimlane;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="20" y="520" width="1140" height="160" as="geometry"/>
</mxCell>
<mxCell id="h01" value="H01&#10;Camera Model" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="20" y="40" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="h02" value="H02&#10;GSD Calculator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="160" y="40" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="h03" value="H03&#10;Robust Kernels" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="300" y="40" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="h04" value="H04&#10;Faiss Index Mgr" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="440" y="40" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="h05" value="H05&#10;Performance Monitor" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="580" y="40" width="130" height="60" as="geometry"/>
</mxCell>
<mxCell id="h06" value="H06&#10;Web Mercator Utils" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="730" y="40" width="130" height="60" as="geometry"/>
</mxCell>
<mxCell id="h07" value="H07&#10;Image Rotation Utils" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="880" y="40" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="h08" value="H08&#10;Batch Validator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="helpers-lane" vertex="1">
<mxGeometry x="1020" y="40" width="100" height="60" as="geometry"/>
</mxCell>
<mxCell id="flow-title" value="Main Processing Flow" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="20" y="700" width="200" height="30" as="geometry"/>
</mxCell>
<mxCell id="flow-box" value="" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#1E1E1E;strokeColor=#FFFFFF;dashed=1;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="20" y="740" width="1340" height="800" as="geometry"/>
</mxCell>
<mxCell id="flow-1" value="1. Client uploads batch&#10;F01 → F05" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="760" width="160" height="60" as="geometry"/>
</mxCell>
<mxCell id="flow-2" value="2. Get next image&#10;F05 → F06" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="230" y="760" width="160" height="60" as="geometry"/>
</mxCell>
<mxCell id="flow-3" value="3. Rotation preprocessing&#10;F06 (30° sweep if needed)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="420" y="760" width="180" height="60" as="geometry"/>
</mxCell>
<mxCell id="flow-4" value="4. Sequential VO (chunk-aware)&#10;F07 → F12 (get active chunk)&#10;F07 → F10 (add to chunk subgraph)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="630" y="760" width="200" height="80" as="geometry"/>
</mxCell>
<mxCell id="flow-5" value="5. Check confidence&#10;F11 Failure Recovery" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="860" y="760" width="180" height="60" as="geometry"/>
</mxCell>
<mxCell id="flow-6a" value="6a. IF GOOD: LiteSAM (1 tile)&#10;F09 drift correction" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="880" width="200" height="60" as="geometry"/>
</mxCell>
<mxCell id="flow-6b" value="6b. IF LOST: Create chunk (proactive)&#10;F11 → F12 (create_chunk)&#10;F12 → F10 (create_new_chunk)&#10;Continue processing in chunk" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="270" y="880" width="240" height="100" as="geometry"/>
</mxCell>
<mxCell id="flow-6c" value="6c. Progressive search (single-image)&#10;F11 → F04 (1→4→9→16→25)&#10;F08 Global PR + F09 LiteSAM" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="540" y="880" width="220" height="80" as="geometry"/>
</mxCell>
<mxCell id="flow-7" value="7. Factor Graph optimize&#10;F10 (chunk optimization)&#10;Fuse VO + GPS in chunk" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="790" y="880" width="200" height="80" as="geometry"/>
</mxCell>
<mxCell id="flow-8" value="8. Coordinate transform&#10;F13 (Pixel → GPS)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="1020" y="880" width="180" height="60" as="geometry"/>
</mxCell>
<mxCell id="flow-9" value="9. Publish results&#10;F14 → F03 (Route API)&#10;F14 → F15 (SSE to client)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="1230" y="880" width="180" height="80" as="geometry"/>
</mxCell>
<mxCell id="arrow-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-1" target="flow-2" edge="1">
<mxCell id="c7" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f02" target="f10">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-2" target="flow-3" edge="1">
<mxCell id="c10" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f11" target="f08">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-3-4" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-3" target="flow-4" edge="1">
<mxCell id="c11" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f11" target="f09">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-4-5" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-4" target="flow-5" edge="1">
<mxCell id="c12" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f11" target="f12">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-5-6a" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-5" target="flow-6a" edge="1">
<mxCell id="c14" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f07" target="f10">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-5-6b" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-5" target="flow-6b" edge="1">
<mxCell id="c15" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f09" target="f10">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-5-6c" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-5" target="flow-6c" edge="1">
<mxCell id="c16" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f12" target="f10">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-6a-7" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-6a" target="flow-7" edge="1">
<mxCell id="c17" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f10" target="f13">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-6b-7" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-6b" target="flow-7" edge="1">
<mxCell id="c18" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f14" target="f13">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-6c-7" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-6c" target="flow-7" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-7-8" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-7" target="flow-8" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-8-9" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="flow-8" target="flow-9" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="chunk-title" value="Chunk Matching (Background - Atlas Multi-Map)" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="1000" width="500" height="30" as="geometry"/>
</mxCell>
<mxCell id="chunk-1" value="F11 (background) → F12&#10;get_chunks_for_matching()&#10;(unanchored, ready)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="1040" width="220" height="80" as="geometry"/>
</mxCell>
<mxCell id="chunk-2" value="F11 → F08&#10;Chunk semantic matching&#10;(aggregate DINOv2)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="290" y="1040" width="220" height="80" as="geometry"/>
</mxCell>
<mxCell id="chunk-3" value="F11 → F06&#10;Chunk rotation sweeps&#10;(12 angles: 0°-330°)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#CC6600;strokeColor=#FFB300;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="540" y="1040" width="220" height="80" as="geometry"/>
</mxCell>
<mxCell id="chunk-4" value="F11 → F09&#10;Chunk LiteSAM matching&#10;(aggregate correspondences)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="790" y="1040" width="240" height="80" as="geometry"/>
</mxCell>
<mxCell id="chunk-5" value="F11 → F10&#10;add_chunk_anchor()&#10;merge_chunks(Sim3)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="1060" y="1040" width="200" height="80" as="geometry"/>
</mxCell>
<mxCell id="arrow-chunk-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-1" target="chunk-2" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-chunk-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-2" target="chunk-3" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-chunk-3-4" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-3" target="chunk-4" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-chunk-4-5" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="chunk-4" target="chunk-5" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="user-input-title" value="User Input Recovery (when all search fails)" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="1160" width="400" height="30" as="geometry"/>
</mxCell>
<mxCell id="user-1" value="F11 exhausted (grid=25)&#10;→ F15 send user_input_needed" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="1200" width="240" height="70" as="geometry"/>
</mxCell>
<mxCell id="user-2" value="Client responds&#10;F01 → F11 apply_user_anchor" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="310" y="1200" width="240" height="70" as="geometry"/>
</mxCell>
<mxCell id="user-3" value="F11 → F10 add_absolute_factor&#10;(high confidence)&#10;Processing resumes" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="580" y="1200" width="240" height="70" as="geometry"/>
</mxCell>
<mxCell id="arrow-user-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="user-1" target="user-2" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-user-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="user-2" target="user-3" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="async-title" value="Asynchronous Trajectory Refinement" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=14;fontStyle=1;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="1310" width="350" height="30" as="geometry"/>
</mxCell>
<mxCell id="async-1" value="F10 back-propagates&#10;new absolute factors&#10;(chunk + global optimization)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#6A1B9A;strokeColor=#BA68C8;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="40" y="1350" width="220" height="80" as="geometry"/>
</mxCell>
<mxCell id="async-2" value="F14 detect changed frames&#10;→ F03 batch_update_waypoints" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="290" y="1350" width="240" height="60" as="geometry"/>
</mxCell>
<mxCell id="async-3" value="F14 → F15&#10;send frame_refined events" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFA726;fontColor=#ffffff;" parent="gps-denied-lane" vertex="1">
<mxGeometry x="560" y="1350" width="220" height="60" as="geometry"/>
</mxCell>
<mxCell id="arrow-async-1-2" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="async-1" target="async-2" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="arrow-async-2-3" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="gps-denied-lane" source="async-2" target="async-3" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="client" value="Client&#10;(GPS-Denied UI)" style="shape=actor;whiteSpace=wrap;html=1;fillColor=#1565C0;strokeColor=#64B5F6;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="10" y="560" width="90" height="80" as="geometry"/>
</mxCell>
<mxCell id="satellite-provider" value="Satellite Provider API&#10;/api/satellite/tiles/..." style="ellipse;shape=cloud;whiteSpace=wrap;html=1;fillColor=#E65100;strokeColor=#FFB300;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="900" y="290" width="250" height="140" as="geometry"/>
</mxCell>
<mxCell id="external-detector" value="External&lt;br&gt;Object Detector (Azaion.Inference)&lt;br&gt;(provides pixel coords)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#C62828;strokeColor=#EF5350;dashed=1;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="1340" y="800" width="160" height="80" as="geometry"/>
</mxCell>
<mxCell id="client-f01" value="HTTP REST" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="client" target="f01" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="f14-route" value="Per-frame&#10;waypoint updates" style="edgeStyle=orthogonalEdgeStyle;curved=1;dashed=1;strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f14" target="r01" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="f04-sat" value="Fetch tiles" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f04" target="satellite-provider" edge="1">
<mxGeometry x="0.3759" y="-10" relative="1" as="geometry">
<mxPoint as="offset"/>
<mxCell id="c19" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f14" target="f15">
<mxGeometry relative="1" as="geometry">
<Array as="points">
<mxPoint x="80" y="270"/>
</Array>
</mxGeometry>
</mxCell>
<mxCell id="f15-client" value="SSE Events" style="dashed=1;strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f15" target="client" edge="1">
<mxCell id="c13" style="strokeColor=#66BB6A;dashed=1;" edge="1" parent="1" source="f11" target="f14">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="detector-f13" value="Object pixels → GPS" style="dashed=1;strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="external-detector" target="f13" edge="1">
<mxCell id="c21" style="strokeColor=#FFFFFF;" edge="1" parent="1" source="f05" target="satellite">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="f12-f10" value="Chunk lifecycle" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f12" target="f10" edge="1">
<mxGeometry relative="1" as="geometry"/>
<mxCell id="legend" value="── sync - - event" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=9;fontColor=#888888;" vertex="1" parent="1">
<mxGeometry x="600" y="380" width="100" height="20" as="geometry"/>
</mxCell>
<mxCell id="f12-f08" value="Chunk descriptors" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f12" target="f08" edge="1">
<mxGeometry relative="1" as="geometry"/>
<mxCell id="3" value="" style="strokeColor=#FFFFFF;endArrow=none;" edge="1" parent="1" source="f11" target="f02">
<mxGeometry relative="1" as="geometry">
<mxPoint x="507.02127659574467" y="100.00000000000011" as="sourcePoint"/>
<mxPoint x="256.3829787234042" y="289.9999999999999" as="targetPoint"/>
</mxGeometry>
</mxCell>
<mxCell id="f11-f12" value="Chunk matching" style="strokeColor=#FFFFFF;fontColor=#ffffff;" parent="1" source="f11" target="f12" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="legend-title" value="Legend" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="100" y="2730" width="100" height="30" as="geometry"/>
</mxCell>
<mxCell id="legend-box" value="" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#1E1E1E;strokeColor=#FFFFFF;" parent="1" vertex="1">
<mxGeometry x="100" y="2770" width="700" height="120" as="geometry"/>
</mxCell>
<mxCell id="legend-1" value="Route API Components" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#B8860B;strokeColor=#FFD54F;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="120" y="2790" width="140" height="40" as="geometry"/>
</mxCell>
<mxCell id="legend-2" value="Flight API Core" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#8B1A1A;strokeColor=#EF5350;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="280" y="2790" width="140" height="40" as="geometry"/>
</mxCell>
<mxCell id="legend-8" value="Chunk Management" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#7B1FA2;strokeColor=#BA68C8;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="600" y="2840" width="140" height="40" as="geometry"/>
</mxCell>
<mxCell id="legend-3" value="Visual Processing" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#1E88E5;strokeColor=#42A5F5;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="440" y="2790" width="140" height="40" as="geometry"/>
</mxCell>
<mxCell id="legend-4" value="Helper Components" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#689F38;strokeColor=#9CCC65;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="120" y="2840" width="140" height="40" as="geometry"/>
</mxCell>
<mxCell id="legend-5" value="State Estimation" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#388E3C;strokeColor=#66BB6A;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="280" y="2840" width="140" height="40" as="geometry"/>
</mxCell>
<mxCell id="legend-6" value="Infrastructure" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#424242;strokeColor=#BDBDBD;fontColor=#ffffff;" parent="1" vertex="1">
<mxGeometry x="440" y="2840" width="140" height="40" as="geometry"/>
</mxCell>
<mxCell id="legend-7" value="External Systems" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#C62828;strokeColor=#EF5350;fontColor=#ffffff;" vertex="1" parent="1">
<mxGeometry x="600" y="2790" width="140" height="40" as="geometry"/>
<mxCell id="2" value="" style="strokeColor=#FFFFFF;fontColor=#ffffff;endArrow=none;" edge="1" parent="1" source="f04" target="f14">
<mxGeometry relative="1" as="geometry">
<mxPoint x="715" y="121.22222222222206" as="sourcePoint"/>
<mxPoint x="125" y="212.9999999999999" as="targetPoint"/>
</mxGeometry>
</mxCell>
</root>
</mxGraphModel>
docs/02_components/decomposition_plan.md
+53 -12
View File
@@ -101,7 +101,8 @@
**F13_coordinate_transformer**
**Interface**: `ICoordinateTransformer`
**API**: `set_enu_origin()`, `get_enu_origin()`, `gps_to_enu()`, `enu_to_gps()`, `pixel_to_gps()`, `gps_to_pixel()`, `image_object_to_gps()`, `compute_gsd()`, `transform_points()`
**API**: `set_enu_origin()`, `get_enu_origin()`, `gps_to_enu()`, `enu_to_gps()`, `pixel_to_gps()`, `gps_to_pixel()`, `image_object_to_gps()`, `transform_points()`
**Dependencies**: F10 Factor Graph Optimizer (for frame poses), H02 GSD Calculator (for GSD computation)
### Results & Communication
@@ -144,14 +145,14 @@
| Order | Component | Method | Purpose | Dependencies |
|-------|-----------|--------|---------|--------------|
| 1 | F16 Configuration Manager | `load_config()` | Load system configuration | None |
| 2 | F03 Flight Database | Initialize connections | Establish DB connection pool | F16 |
| 3 | F15 Model Manager | `load_model("SuperPoint")` | Load SuperPoint feature extractor | F16 |
| 4 | F15 Model Manager | `load_model("LightGlue")` | Load LightGlue matcher | F16 |
| 5 | F15 Model Manager | `load_model("DINOv2")` | Load DINOv2 for place recognition | F16 |
| 6 | F15 Model Manager | `load_model("LiteSAM")` | Load LiteSAM for cross-view matching | F16 |
| 7 | F04 Satellite Data Manager | Initialize cache | Initialize tile cache directory | F16 |
| 8 | F08 Global Place Recognition | `build_index()` or `load_index()` | Build/load Faiss index from satellite tiles | F04, F15, H04 |
| 1 | F17 Configuration Manager | `load_config()` | Load system configuration | None |
| 2 | F03 Flight Database | Initialize connections | Establish DB connection pool | F17 |
| 3 | F16 Model Manager | `load_model("SuperPoint")` | Load SuperPoint feature extractor | F17 |
| 4 | F16 Model Manager | `load_model("LightGlue")` | Load LightGlue matcher | F17 |
| 5 | F16 Model Manager | `load_model("DINOv2")` | Load DINOv2 for place recognition | F17 |
| 6 | F16 Model Manager | `load_model("LiteSAM")` | Load LiteSAM for cross-view matching | F17 |
| 7 | F04 Satellite Data Manager | Initialize cache | Initialize tile cache directory | F17 |
| 8 | F08 Global Place Recognition | `load_index()` | Load pre-built Faiss index from satellite provider | F04, F16, H04 |
| 9 | F12 Route Chunk Manager | Initialize | Initialize chunk state tracking | F10 |
| 10 | F02 Flight Processor | Ready | Ready to accept flights | All above |
| 11 | F01 Flight API | Start server | Start FastAPI/Uvicorn | F02 |
@@ -176,7 +177,7 @@
| Source | Target | Method | Purpose |
|--------|--------|--------|---------|
| F02 | F16 | `load_config()` | Load system configuration |
| F02 | F15 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| F02 | F16 | `load_model()` × 4 | Load SuperPoint, LightGlue, DINOv2, LiteSAM |
| F04 | F08 | Satellite tiles | F08 generates descriptors for Faiss |
| F08 | H04 | `build_index()` | Build satellite descriptor index |
| F08 | F15 | `get_inference_engine("DINOv2")` | Get model for descriptor generation |
@@ -246,6 +247,7 @@
| F02 | F11 | `check_confidence()` → FAIL | Low confidence |
| F11 | F12 | `create_chunk_on_tracking_loss()` | **Proactive chunk creation** |
| F12 | F10 | `create_new_chunk()` | Create chunk in factor graph |
| F12 | F03 | `save_chunk_state()` | Persist chunk state for recovery |
| F02 | F12 | `get_active_chunk()` | Get new active chunk |
| F11 | F06 | `requires_rotation_sweep()` | Trigger rotation sweep (single-image) |
| F11 | F08 | `retrieve_candidate_tiles()` | Coarse localization (single-image) |
@@ -292,7 +294,7 @@
| F01 | F11 | `apply_user_anchor()` | Apply fix |
| F11 | F10 | `add_absolute_factor()` (high confidence) | Hard constraint |
| F10 | Internal | `optimize()` | Re-optimize |
| F11 | F02 | `update_flight_status("PROCESSING")` | Resume |
| F11 | Event | Emit `UserFixApplied` | F02 subscribes and resumes |
### Asynchronous Refinement
@@ -331,7 +333,7 @@
## Interaction Coverage Verification
✅ **Initialization**: F02→F15, F16, F17; F04→F08→H04
✅ **Initialization**: F02→F16, F17; F04→F08→H04
✅ **Flight creation**: Client→F01→F02→F04,F12,F16,F17,F14
✅ **Image upload**: Client→F01→F05→H08,F17
✅ **Rotation sweep**: F06→H07,F09 (12 iterations)
@@ -392,3 +394,42 @@
- [x] F17 Configuration Manager
- [x] F03 Flight Database (merged from R04, G17)
- [x] Helper components (H01-H08)
---
## Architecture Notes
### F02 Flight Processor Complexity
F02 Flight Processor handles multiple concerns:
- Flight lifecycle management
- Processing loop orchestration
- Event subscription and state updates
- Coordination of F07/F08/F09/F10
**Current Status**: Acceptable for MVP. The responsibilities are related (all flight processing) and the component acts as a coordinator rather than implementing logic directly.
**Future Consideration**: If complexity grows, consider splitting into:
- F02a Flight State Manager (lifecycle, status)
- F02b Processing Loop Coordinator (frame processing orchestration)
**Decision**: Keep as single component. Clear internal organization with separate methods for state vs processing concerns. Event-based communication with F11 keeps recovery logic separate.
### Error Recovery Strategy
**Per-Component Recovery**:
- **F02**: Persists flight state via F03 on each significant update. On restart, loads last known state.
- **F07**: Stateless - reprocesses frame if VO fails
- **F10**: Factor graph state persisted periodically. On restart, rebuilds from F03 checkpoint.
- **F11**: Chunk state persisted via F12→F03. Recovery continues from last chunk state.
- **F12**: All chunk state persisted to F03. Restores chunk handles on restart.
**System-Wide Recovery**:
1. On crash, F02 loads flight state from F03
2. F12 restores chunk state from F03
3. Processing resumes from last successfully processed frame
4. Incomplete chunks continue building/matching
**Event Recovery**:
- Events are fire-and-forget (no persistence)
- Subscribers rebuild state from F03 on restart
docs/02_components/system_flows.md
+883
View File
@@ -0,0 +1,883 @@
# ASTRAL-Next System Flows
> **See also**: [System Flow Diagrams (Mermaid)](./system_flows_diagrams.md) for interactive visual diagrams.
## System Overview
ASTRAL-Next is a GPS-denied UAV visual localization system using tri-layer matching (SuperPoint+LightGlue for VO, DINOv2 for place recognition, LiteSAM for cross-view matching) with an Atlas-style multi-map chunk architecture for robust tracking and recovery.
## Components Summary
| ID | Component | Interface | Purpose |
|----|-----------|-----------|---------|
| F01 | Flight API | `IFlightAPI` | REST endpoints, SSE streaming |
| F02 | Flight Processor | `IFlightProcessor` | Central coordinator, processing loop |
| F03 | Flight Database | `IFlightDatabase` | Persistence layer |
| F04 | Satellite Data Manager | `ISatelliteDataManager` | Tile fetching, caching, progressive search |
| F05 | Image Input Pipeline | `IImageInputPipeline` | Image ingestion, validation, storage |
| F06 | Image Rotation Manager | `IImageRotationManager` | Rotation sweeps, heading tracking |
| F07 | Sequential Visual Odometry | `ISequentialVO` | Frame-to-frame VO (SuperPoint+LightGlue) |
| F08 | Global Place Recognition | `IGlobalPlaceRecognition` | Coarse localization (DINOv2+VLAD) |
| F09 | Metric Refinement | `IMetricRefinement` | Precise alignment (LiteSAM) |
| F10 | Factor Graph Optimizer | `IFactorGraphOptimizer` | GTSAM-based trajectory optimization |
| F11 | Failure Recovery Coordinator | `IFailureRecoveryCoordinator` | Recovery orchestration, chunk matching |
| F12 | Route Chunk Manager | `IRouteChunkManager` | Chunk lifecycle management |
| F13 | Coordinate Transformer | `ICoordinateTransformer` | Coordinate conversions (GPS↔ENU↔Pixel) |
| F14 | Result Manager | `IResultManager` | Result tracking and publishing |
| F15 | SSE Event Streamer | `ISSEEventStreamer` | Real-time event streaming |
| F16 | Model Manager | `IModelManager` | ML model loading (TensorRT/ONNX) |
| F17 | Configuration Manager | `IConfigurationManager` | System configuration |
### Helper Components
| ID | Helper | Interface | Purpose |
|----|--------|-----------|---------|
| H01 | Camera Model | `ICameraModel` | Projection/unprojection |
| H02 | GSD Calculator | `IGSDCalculator` | Ground sampling distance |
| H03 | Robust Kernels | `IRobustKernels` | Huber/Cauchy loss functions |
| H04 | Faiss Index Manager | `IFaissIndexManager` | Similarity search |
| H05 | Performance Monitor | `IPerformanceMonitor` | Timing measurements |
| H06 | Web Mercator Utils | `IWebMercatorUtils` | Tile coordinate calculations |
| H07 | Image Rotation Utils | `IImageRotationUtils` | Image rotation operations |
| H08 | Batch Validator | `IBatchValidator` | Image batch validation |
---
## Flow 1: System Initialization
**Purpose**: Initialize all system components on startup.
**Sequence**:
```
┌─────────────────┐
│ System Start │
└────────┬────────┘
▼
┌─────────────────┐
│ F17 load_config │ ← Load system configuration
└────────┬────────┘
▼
┌─────────────────┐
│ F03 Initialize │ ← Establish DB connection pool
│ connections │
└────────┬────────┘
▼
┌─────────────────────────────────────────┐
│ F16 load_model() × 4 │
│ ├─ SuperPoint (feature extraction) │
│ ├─ LightGlue (feature matching) │
│ ├─ DINOv2 (place recognition) │
│ └─ LiteSAM (cross-view matching) │
└────────┬────────────────────────────────┘
▼
┌─────────────────┐
│F04 Initialize │ ← Initialize tile cache
│ cache │
└────────┬────────┘
▼
┌─────────────────────────────────────────┐
│ F08 load_index() │
│ ← Load pre-built Faiss index from │
│ satellite provider │
└────────┬────────────────────────────────┘
▼
┌─────────────────┐
│ F12 Initialize │ ← Initialize chunk state tracking
└────────┬────────┘
▼
┌─────────────────┐
│ F02 Ready │ ← Ready to accept flights
└────────┬────────┘
▼
┌─────────────────┐
│ F01 Start │ ← Start FastAPI/Uvicorn
│ server │
└─────────────────┘
```
**Duration**: ~30 seconds (dominated by model loading)
---
## Flow 2: Flight Creation
**Purpose**: Create a new flight with initial configuration and prefetch satellite data.
**Trigger**: `POST /flights`
**Sequence**:
```
┌──────────┐ POST /flights ┌─────┐
│ Client │ ───────────────────► │ F01 │
└──────────┘ └──┬──┘
│ create_flight()
▼
┌─────────┐
│ F02 │ Flight Processor
└────┬────┘
┌────────────────────────┬─┴─┬────────────────────────┐
│ │ │ │
▼ │ ▼ ▼
┌───────────────┐ │ ┌───────────────┐ ┌───────────────┐
│F17 get_flight │ │ │F13 set_enu │ │F04 prefetch │
│ _config() │ │ │ _origin() │ │ _route │
└───────────────┘ │ └───────────────┘ │ _corridor() │
│ └───────┬───────┘
│ │
▼ ▼
┌─────────────┐ ┌─────────────────┐
│F03 insert │ │ Satellite │
│ _flight() │ │ Provider API │
└─────────────┘ │ GET tiles/batch │
└─────────────────┘
│
┌──────┴──────┐
│ F14 │
│create_stream│
└─────────────┘
│
┌──────┴──────┐
SSE Connection │ Client │
◄─────────────────────────────│ GET stream │
└─────────────┘
```
**Output**: `flight_id`, SSE stream established
---
## Flow 3: Image Upload
**Purpose**: Upload batch of UAV images for processing.
**Trigger**: `POST /flights/{flightId}/images/batch`
**Sequence**:
```
┌──────────┐ POST images/batch ┌─────┐
│ Client │ ──────────────────► │ F01 │
│ │ (10-50 images) └──┬──┘
└──────────┘ │ queue_batch()
▼
┌─────────────┐
│ F05 │ Image Input Pipeline
└──────┬──────┘
┌───────────────┼───────────────┐
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│H08 validate │ │ Store to │ │F03 save │
│ _batch() │ │ disk │ │ _image │
└─────────────┘ └─────────────┘ │ _metadata()│
└─────────────┘
```
**Validation Rules**:
- Batch size: 10-50 images
- Naming convention: ADxxxxxx.jpg
- Sequential numbering
- Image dimensions: 640×480 to 6252×4168
---
## Flow 4: Normal Frame Processing (Tracking Good)
**Purpose**: Process a single frame when tracking quality is good.
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │
│ │
│ ┌─────────────┐ │
│ │ get_next │◄───────────────────────────────────────────────────────┐│
│ │ _image() │ ││
│ └──────┬──────┘ ││
│ │ F05 ││
│ ▼ ││
│ ┌─────────────────────────────────────────────────────────────┐ ││
│ │ get_active_chunk() via F12 │ ││
│ └──────────────────────────────┬──────────────────────────────┘ ││
│ ▼ ││
│ ┌─────────────────────────────────────────────────────────────┐ ││
│ │ requires_rotation_sweep()? via F06 │ ││
│ │ ├─ YES → Flow 5 (First Frame/Sharp Turn) │ ││
│ │ └─ NO → Pre-rotate to current heading │ ││
│ └──────────────────────────────┬──────────────────────────────┘ ││
│ ▼ ││
│ ┌─────────────────────────────────────────────────────────────┐ ││
│ │ F07 compute_relative_pose_in_chunk() │ ││
│ │ ├─ SuperPoint extract (prev + curr) via F16 │ ││
│ │ └─ LightGlue match via F16 │ ││
│ └──────────────────────────────┬──────────────────────────────┘ ││
│ │ ││
│ ┌─────────────────┼─────────────────┐ ││
│ │ Tracking Good? │ Tracking Lost? │ ││
│ ▼ ▼ ▼ ││
│ ┌─────────────────┐ ┌─────────────────────────────────┐ ││
│ │F12 add_frame │ │ →Flow 6 (Tracking Loss/Recovery)│ ││
│ │ _to_chunk() │ └─────────────────────────────────┘ ││
│ └────────┬────────┘ ││
│ ▼ ││
│ ┌─────────────────────────────────────────────────────────────┐ ││
│ │ F04 fetch_tile() + compute_tile_bounds() │ ││
│ └──────────────────────────────┬──────────────────────────────┘ ││
│ ▼ ││
│ ┌─────────────────────────────────────────────────────────────┐ ││
│ │ F09 align_to_satellite(image, tile, tile_bounds) │ ││
│ │ └─ LiteSAM matching via F16 │ ││
│ └──────────────────────────────┬──────────────────────────────┘ ││
│ │ ││
│ ┌─────────────────┴─────────────────┐ ││
│ │ Match Found? │ ││
│ ▼ ▼ ││
│ ┌─────────────────────────┐ ┌─────────────────────────┐ ││
│ │F10 add_absolute_factor()│ │ Skip absolute anchor │ ││
│ │F06 update_heading() │ │ (VO-only frame) │ ││
│ └────────────┬────────────┘ └─────────────────────────┘ ││
│ │ ││
│ └─────────────────────┬───────────────────────────────────┘│
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ F10 optimize_chunk() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ F13 enu_to_gps() → Convert ENU pose to GPS │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ F14 update_frame_result() + publish_waypoint_update() │ │
│ │ └─ F15 send_frame_result() → SSE "frame_processed" │ │
│ │ └─ F03 save_frame_result() │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
**Performance**: < 5 seconds per frame
---
## Flow 5: First Frame / Sharp Turn (Rotation Sweep)
**Purpose**: Establish UAV heading when unknown or after sharp turn.
**Trigger**: `requires_rotation_sweep()` returns True
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F06 Image Rotation Manager │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ try_rotation_steps(image, satellite_tile, tile_bounds) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┴───────────────────────────┐ │
│ │ For angle in [0°, 30°, 60°, ... 330°]: │ │
│ │ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ H07 rotate_image(image, angle) │ │ │
│ │ └───────────────────┬─────────────────┘ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F09 align_to_satellite(rotated, │ │ │
│ │ │ satellite_tile, tile_bounds) │ │ │
│ │ └───────────────────┬─────────────────┘ │ │
│ │ │ │ │
│ │ ┌──────────────┴──────────────┐ │ │
│ │ │ Match Found? │ │ │
│ │ ▼ ▼ │ │
│ │ ┌───────────────┐ ┌───────────────┐ │ │
│ │ │ Calculate │ │ Try next │ │ │
│ │ │ precise angle │ │ rotation │ │ │
│ │ └───────┬───────┘ └───────────────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F03 save_heading() via F06 │ │ │
│ │ │ update_heading() │ │ │
│ │ └─────────────────────────────────────┘ │ │
│ │ │ │
│ └────────────────────────────────────────────────────┘ │
│ │ │
│ ┌────────────┴────────────┐ │
│ │ Return RotationResult │ │
│ │ or None │ │
│ └─────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
**Output**: RotationResult with precise heading angle
---
## Flow 6: Tracking Loss / Recovery (Progressive Search)
**Purpose**: Recover localization after tracking loss.
**Trigger**: VO inlier count < 20 or LiteSAM match fails
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F11 Failure Recovery Coordinator │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. EMIT RecoveryStarted event │ │
│ │ └─ F02 subscribes → Update status to "recovering" │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 2. create_chunk_on_tracking_loss() via F12 │ │
│ │ └─ Proactive chunk creation (processing continues) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 3. Single-image recovery attempt: │ │
│ │ ├─ F06 requires_rotation_sweep() → trigger sweep │ │
│ │ ├─ F08 retrieve_candidate_tiles() (DINOv2) │ │
│ │ └─ Progressive tile search (1→4→9→16→25): │ │
│ │ │ │
│ │ ┌───────────────────────────────────────────────────┐ │ │
│ │ │ For grid_size in [1, 4, 9, 16, 25]: │ │ │
│ │ │ ├─ F04 expand_search_grid() │ │ │
│ │ │ ├─ For each tile: │ │ │
│ │ │ │ ├─ F04 compute_tile_bounds() │ │ │
│ │ │ │ └─ F09 align_to_satellite(img, tile, bounds)│ │ │
│ │ │ │ │ │ │
│ │ │ └─ If match found: BREAK │ │ │
│ │ └───────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌────────────────────┴────────────────────┐ │
│ │ Single-image match found? │ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌─────────────────────────────┐ │
│ │ EMIT RecoverySucceeded│ │ Continue chunk building │ │
│ │ Resume normal flow │ │ → Flow 7 (Chunk Building) │ │
│ └─────────────────────┘ │ → Flow 8 (Chunk Matching) │ │
│ │ (Background) │ │
│ └─────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────┐ │
│ │ If all strategies exhausted: │ │
│ │ → Flow 10 (User Input Recovery) │ │
│ └─────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
---
## Flow 7: Chunk Building
**Purpose**: Build route chunks when tracking is lost, continuing VO within chunk.
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F12 Route Chunk Manager │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ create_chunk(flight_id, start_frame_id) │ │
│ │ ├─ F10 create_new_chunk() ← Factor graph subgraph │ │
│ │ ├─ Initialize chunk state (unanchored, active) │ │
│ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┴───────────────────────────┐ │
│ │ For each frame in chunk: │ │
│ │ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F07 compute_relative_pose_in_chunk()│ │ │
│ │ └───────────────────┬─────────────────┘ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F12 add_frame_to_chunk() │ │ │
│ │ │ └─ F10 add_relative_factor_to_chunk│ │ │
│ │ └───────────────────┬─────────────────┘ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ F10 optimize_chunk() (local) │ │ │
│ │ └─────────────────────────────────────┘ │ │
│ │ │ │
│ └────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ is_chunk_ready_for_matching()? │ │
│ │ ├─ Min 5 frames │ │
│ │ ├─ Max 20 frames │ │
│ │ └─ Internal consistency (good VO inlier counts) │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
---
## Flow 8: Background Chunk Matching
**Purpose**: Asynchronously match unanchored chunks to satellite data.
**Trigger**: Background task (every 5 seconds)
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F11 process_unanchored_chunks() (Background Task) │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ while flight_active: │ │
│ │ ├─ unanchored = F12 get_chunks_for_matching() │ │
│ │ │ │ │
│ │ └─ for chunk in unanchored: │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────────────────────┐ │ │
│ │ │ if F12 is_chunk_ready_for_matching(chunk_id): │ │ │
│ │ │ ├─ F12 mark_chunk_matching(chunk_id) │ │ │
│ │ │ │ │ │ │
│ │ │ ├─ STEP 1: Chunk Semantic Matching │ │ │
│ │ │ │ ├─ F12 get_chunk_images() │ │ │
│ │ │ │ └─ F08 retrieve_candidate_tiles_for_chunk() │ │ │
│ │ │ │ └─ F08 compute_chunk_descriptor() │ │ │
│ │ │ │ └─ H04 Faiss search() │ │ │
│ │ │ │ │ │ │
│ │ │ ├─ STEP 2: Chunk LiteSAM Matching (with rotation) │ │ │
│ │ │ │ ├─ For each candidate tile: │ │ │
│ │ │ │ │ ├─ F04 get_tile + compute_tile_bounds() │ │ │
│ │ │ │ │ ├─ F06 try_chunk_rotation_steps() │ │ │
│ │ │ │ │ │ └─ F09 align_chunk_to_satellite() │ │ │
│ │ │ │ │ └─ If match: BREAK │ │ │
│ │ │ │ │ │ │
│ │ │ └─ If match found: │ │ │
│ │ │ └─ → Flow 9 (Chunk Merging) │ │ │
│ │ │ │ │ │
│ │ └─────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ sleep(5 seconds) │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
---
## Flow 9: Chunk Merging
**Purpose**: Merge anchored chunk into main trajectory.
**Trigger**: Successful chunk matching
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F11 merge_chunk_to_trajectory() │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. Get chunk frames: F12 get_chunk_frames(chunk_id) │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 2. Anchor chunk: F12 mark_chunk_anchored(chunk_id, gps) │ │
│ │ └─ F10 add_chunk_anchor() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 3. Resolve target: F12 get_merge_target(chunk_id) │ │
│ │ └─ Returns target_chunk_id (predecessor or "main") │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 4. Merge: F12 merge_chunks(chunk_id, target_chunk_id, Sim3) │ │
│ │ ├─ F10 merge_chunks() ← Apply Sim(3) transform │ │
│ │ ├─ F12 deactivate_chunk(chunk_id) │ │
│ │ └─ F03 save_chunk_state() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 5. Optimize global: F10 optimize_global() │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 6. EMIT ChunkMerged event (flight_id, chunk_id, merged_frames)│ │
│ │ └─ F14 subscribes → update_results_after_chunk_merge() │ │
│ │ ├─ F10 get_trajectory() → ENU poses │ │
│ │ ├─ F13 enu_to_gps() for each frame │ │
│ │ ├─ F03 save_frame_result() + update_waypoint() │ │
│ │ └─ F15 send_refinement() → SSE "frame_refined" × N │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
**Sim(3) Transform**: Translation + Rotation + Scale alignment between chunks
---
## Flow 10: User Input Recovery
**Purpose**: Request human assistance when all automatic recovery fails.
**Trigger**: Progressive search exhausted (25 tiles), chunk matching failed
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F11 create_user_input_request() │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. Get UAV image for frame_id │ │
│ │ 2. Get top-5 candidates from F08 │ │
│ │ 3. Create UserInputRequest │ │
│ │ 4. F15 send_user_input_request() → SSE "user_input_needed" │ │
│ │ 5. EMIT UserInputNeeded event │ │
│ │ └─ F02 subscribes → Update status to "BLOCKED" │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│
▼ Client receives SSE event
┌──────────────────────────────────────────────────────────────────────────┐
│ Client UI │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Display UAV image + candidate satellite tiles │ │
│ │ User clicks corresponding point on satellite tile │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ POST /flights/{flightId}/user-fix │ │
│ │ body: { frame_id, uav_pixel, satellite_gps } │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ F11 apply_user_anchor() │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. Validate anchor data │ │
│ │ 2. F10 add_absolute_factor(is_user_anchor=True) ← σ=5m │ │
│ │ 3. F10 optimize() │ │
│ │ 4. EMIT UserFixApplied event │ │
│ │ └─ F02 subscribes → Update status to "PROCESSING" │ │
│ │ 5. Resume processing loop │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
---
## Flow 11: Asynchronous Refinement
**Purpose**: Back-propagate optimization improvements to previous frames.
**Trigger**: New absolute GPS factor added, chunk merged
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F10 Background Optimization │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. New absolute factor triggers batch optimization │ │
│ │ 2. F10 optimize(iterations=50-100) │ │
│ │ └─ Levenberg-Marquardt with robust kernels │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 3. Identify refined frames │ │
│ │ 4. F14 mark_refined(frame_ids) │ │
│ │ ├─ For each frame: │ │
│ │ │ ├─ F10 get_trajectory() → ENU pose │ │
│ │ │ ├─ F13 enu_to_gps(flight_id, enu_pose) │ │
│ │ │ ├─ F03 save_frame_result(refined=True) │ │
│ │ │ └─ F03 update_waypoint() │ │
│ │ │ │ │
│ │ └─ F15 send_refinement() × N → SSE "frame_refined" × N │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
---
## Flow 12: Object to GPS Conversion
**Purpose**: Convert detected object pixel coordinates to GPS (external integration).
**Trigger**: `POST /flights/{flightId}/frames/{frameId}/object-to-gps`
**Sequence**:
```
┌──────────────────┐ POST object-to-gps ┌─────┐
│ External System │ ────────────────────►│ F01 │
│ (Azaion.Inference│ {pixel_x, pixel_y} └──┬──┘
└──────────────────┘ │
▼
┌─────────────┐
│ F13 │ Coordinate Transformer
└──────┬──────┘
┌───────────────────┼───────────────────┐
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│F10 get_pose │ │F17 camera │ │ H01 Camera │
│ (frame_id) │ │ _params │ │ Model │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└───────────────────┼───────────────────┘
▼
┌─────────────┐
│ pixel_to_gps│
│ calculation │
└──────┬──────┘
│
▼
┌───────────────────────────┐
│ Return ObjectGPSResponse │
│ { gps, accuracy_meters }│
└───────────────────────────┘
```
---
## Flow 13: Flight Completion
**Purpose**: Complete flight processing and clean up resources.
**Trigger**: All images processed successfully
**Sequence**:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ F02 Flight Processor │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 1. All frames processed │ │
│ │ 2. Wait for pending chunk merges (if any) │ │
│ │ 3. F10 optimize_global(iterations=100) ← Final optimization │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 4. F14 mark_refined(all_frames) │ │
│ │ └─ Final refinement updates │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 5. F03 save_flight_state(status="completed") │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 6. F15 send event → SSE "flight_completed" │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ 7. Cleanup: │ │
│ │ ├─ Stop background chunk matching task │ │
│ │ ├─ F04 clear_flight_cache(flight_id) (optional) │ │
│ │ └─ Release flight resources │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
---
## Flow 14: System Shutdown
**Purpose**: Gracefully shutdown all components.
**Sequence**:
```
┌─────────────────┐
│ Shutdown Signal │
└────────┬────────┘
▼
┌─────────────────┐
│ F01 Stop │ ← Stop accepting requests
│ accepting │
└────────┬────────┘
▼
┌─────────────────┐
│ F02 Complete/ │ ← Complete or cancel active flights
│ Cancel │
└────────┬────────┘
▼
┌─────────────────┐
│ F11 Stop │ ← Stop background chunk matching
│ background │
└────────┬────────┘
▼
┌─────────────────┐
│ F12 Save chunk │ ← Save chunk state for recovery
│ state │
└────────┬────────┘
▼
┌─────────────────┐
│ F16 Unload │ ← Unload ML models, free GPU memory
│ models │
└────────┬────────┘
▼
┌─────────────────┐
│ F03 Close │ ← Close database connections
│ connections │
└────────┬────────┘
▼
┌─────────────────┐
│ F04 Flush cache │ ← Flush satellite tile cache
└────────┬────────┘
▼
┌─────────────────┐
│ Shutdown │
│ Complete │
└─────────────────┘
```
---
## Component Interaction Summary
### Data Flow Direction
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ EXTERNAL │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Client │ │ Satellite │ │ External │ │
│ │ (UI) │ │ Provider │ │ Detector │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
└─────────┼───────────────────┼───────────────────┼───────────────────────────┘
│ REST/SSE │ HTTP │ REST
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ API LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F01 Flight API │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ ORCHESTRATION LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F02 Flight Processor │ │
│ │ (Central coordinator, event subscriber, background task manager) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
├─────────────────────────────────────────────────────────────┐
▼ ▼
┌─────────────────────────────────────────────┐ ┌────────────────────────────┐
│ DATA MANAGEMENT LAYER │ │ RECOVERY LAYER │
│ ┌─────────────┐ ┌─────────────┐ │ │ ┌───────────────────────┐ │
│ │ F04 │ │ F05 │ │ │ │ F11 │ │
│ │ Satellite │ │ Image │ │ │ │ Failure Recovery │ │
│ │ Data │ │ Input │ │ │ │ (Event emitter) │ │
│ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │
│ │ │ │ │
│ ┌─────────────┐ ┌─────────────┐ │ │ ▼ │
│ │ F12 │ │ F03 │ │ │ ┌───────────────────────┐ │
│ │Route Chunk │ │ Flight │ │ │ │ F12 │ │
│ │ Manager │ │ Database │ │ │ │ (Chunk state source) │ │
│ └─────────────┘ └─────────────┘ │ │ └───────────────────────┘ │
└─────────────────────────────────────────────┘ └────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ VISUAL PROCESSING LAYER │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ┌──────────────┐ │
│ │ F06 │ │ F07 │ │ F08 │ │ F09 │ │
│ │ Rotation │ │ Sequential VO │ │ Global │ │ Metric │ │
│ │ Manager │ │(SuperPoint+ │ │ Place │ │ Refinement │ │
│ │ (30° sweeps) │ │ LightGlue) │ │ Recognition │ │ (LiteSAM) │ │
│ │ │ │ │ │ (DINOv2) │ │ │ │
│ └───────────────┘ └───────────────┘ └───────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ STATE ESTIMATION LAYER │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ F10 Factor Graph Optimizer │ │
│ │ (GTSAM, iSAM2, Robust kernels, Chunk subgraphs, Sim(3) merging) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ OUTPUT LAYER │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ F13 │ │ F14 │ │ F15 │ │
│ │ Coordinate │ │ Result │ │ SSE │ │
│ │ Transformer │ │ Manager │ │ Streamer │ │
│ │ (ENU↔GPS↔Px) │ │ │ │ │ │
│ └───────────────┘ └───────────────┘ └───────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ INFRASTRUCTURE LAYER │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────────────────┐│
│ │ F16 │ │ F17 │ │ HELPERS ││
│ │ Model │ │Configuration │ │ H01-H08 (Camera, GSD, Kernels, ││
│ │ Manager │ │ Manager │ │ Faiss, Monitor, Mercator, etc) ││
│ └───────────────┘ └───────────────┘ └───────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Event-Based Communication
F11 Failure Recovery Coordinator emits events instead of directly calling F02:
| Event | Publisher | Subscribers | Action |
|-------|-----------|-------------|--------|
| `RecoveryStarted` | F11 | F02 | Update status to "recovering" |
| `RecoverySucceeded` | F11 | F02 | Update status to "processing", resume |
| `RecoveryFailed` | F11 | F02 | Update status to "blocked" |
| `UserInputNeeded` | F11 | F02 | Update status to "blocked", await fix |
| `UserFixApplied` | F11 | F02 | Update status to "processing", resume |
| `ChunkCreated` | F11 | F02 | Log chunk creation |
| `ChunkAnchored` | F11 | F02 | Log chunk anchor |
| `ChunkMerged` | F11 | F02, F14 | F14 updates results for merged frames |
---
## SSE Events to Client
| Event | Trigger | Data |
|-------|---------|------|
| `frame_processed` | Frame completed | frame_id, gps, altitude, heading, confidence |
| `frame_refined` | Trajectory refined | frame_id, updated gps, refined=true |
| `search_expanded` | Progressive search | frame_id, grid_size, status |
| `user_input_needed` | Recovery exhausted | request_id, frame_id, candidate_tiles |
| `processing_blocked` | Status change | reason, frame_id |
| `flight_completed` | Flight done | statistics |
---
## Performance Targets
| Flow | Target |
|------|--------|
| System Initialization | < 30 seconds |
| Flight Creation | < 500ms response |
| Image Batch Upload | < 2 seconds (50 images) |
| Per-Frame Processing | < 5 seconds |
| Rotation Sweep (12 rotations) | < 1.2 seconds |
| Progressive Search (25 tiles) | < 1.5 seconds |
| Chunk Matching | < 3 seconds (background) |
| SSE Event Latency | < 500ms |
---
## Accuracy Targets
| Metric | Target |
|--------|--------|
| GPS Accuracy | 60% < 20m, 80% < 50m |
| Mean Reprojection Error | < 1.0 pixels |
| Place Recognition Recall@5 | > 85% |
| LiteSAM Success Rate | > 95% (when rotation correct) |
docs/02_components/system_flows_diagrams.md
+603
View File
@@ -0,0 +1,603 @@
# ASTRAL-Next System Flow Diagrams
## Component Architecture Diagram
```mermaid
graph TB
subgraph External["External Systems"]
Client[Client UI]
SatProv[Satellite Provider]
ExtDet[External Detector]
end
subgraph API["API Layer"]
F01[F01 Flight API]
end
subgraph Orchestration["Orchestration Layer"]
F02[F02 Flight Processor]
end
subgraph Data["Data Management"]
F03[F03 Flight Database]
F04[F04 Satellite Data Manager]
F05[F05 Image Input Pipeline]
F12[F12 Route Chunk Manager]
end
subgraph Recovery["Recovery Layer"]
F11[F11 Failure Recovery Coordinator]
end
subgraph Visual["Visual Processing"]
F06[F06 Image Rotation Manager]
F07[F07 Sequential Visual Odometry]
F08[F08 Global Place Recognition]
F09[F09 Metric Refinement]
end
subgraph State["State Estimation"]
F10[F10 Factor Graph Optimizer]
end
subgraph Output["Output Layer"]
F13[F13 Coordinate Transformer]
F14[F14 Result Manager]
F15[F15 SSE Event Streamer]
end
subgraph Infra["Infrastructure"]
F16[F16 Model Manager]
F17[F17 Configuration Manager]
end
Client -->|REST| F01
F15 -->|SSE| Client
ExtDet -->|REST| F01
F04 -->|HTTP| SatProv
F01 --> F02
F02 --> F03
F02 --> F05
F02 --> F11
F02 --> F07
F02 --> F10
F02 --> F12
F02 --> F04
F11 --> F08
F11 --> F09
F11 --> F12
F11 -.->|events| F02
F11 -.->|events| F14
F06 --> F09
F07 --> F10
F09 --> F10
F12 --> F10
F10 --> F13
F14 --> F13
F14 --> F15
F14 --> F03
F07 --> F16
F08 --> F16
F09 --> F16
F02 --> F17
```
## Flow 1: System Initialization
```mermaid
sequenceDiagram
participant Main as System
participant F17 as Config Manager
participant F03 as Flight Database
participant F16 as Model Manager
participant F04 as Satellite Manager
participant F08 as Place Recognition
participant F12 as Chunk Manager
participant F02 as Flight Processor
participant F01 as Flight API
Main->>F17: load_config()
F17-->>Main: SystemConfig
Main->>F03: Initialize connections
F03-->>Main: Connection pool ready
Main->>F16: load_model("SuperPoint")
Main->>F16: load_model("LightGlue")
Main->>F16: load_model("DINOv2")
Main->>F16: load_model("LiteSAM")
F16-->>Main: Models loaded (~25s)
Main->>F04: Initialize cache
F04-->>Main: Cache ready
Main->>F08: load_index()
F08-->>Main: Faiss index loaded
Main->>F12: Initialize
F12-->>Main: Chunk tracking ready
Main->>F02: Ready
F02-->>Main: Ready to accept flights
Main->>F01: Start server
F01-->>Main: FastAPI running
```
## Flow 2: Flight Creation
```mermaid
sequenceDiagram
participant C as Client
participant F01 as Flight API
participant F02 as Flight Processor
participant F17 as Config Manager
participant F13 as Coord Transformer
participant F04 as Satellite Manager
participant F03 as Flight Database
participant F15 as SSE Streamer
C->>F01: POST /flights
F01->>F02: create_flight()
F02->>F17: get_flight_config()
F17-->>F02: CameraParams, Altitude
F02->>F13: set_enu_origin(start_gps)
F02->>F04: prefetch_route_corridor()
F04-->>F02: Prefetching async
F02->>F03: insert_flight()
F03-->>F02: flight_id
F01-->>C: 201 Created {flight_id}
C->>F01: GET /flights/{id}/stream
F01->>F15: create_stream()
F15-->>C: SSE Connection
```
## Flow 3: Normal Frame Processing
```mermaid
sequenceDiagram
participant F02 as Flight Processor
participant F05 as Image Pipeline
participant F12 as Chunk Manager
participant F06 as Rotation Manager
participant F07 as Sequential VO
participant F04 as Satellite Manager
participant F09 as Metric Refinement
participant F10 as Factor Graph
participant F13 as Coord Transformer
participant F14 as Result Manager
participant F15 as SSE Streamer
F02->>F05: get_next_image()
F05-->>F02: ImageData
F02->>F12: get_active_chunk()
F12-->>F02: ChunkHandle
F02->>F06: requires_rotation_sweep()
F06-->>F02: false (heading known)
F02->>F07: compute_relative_pose_in_chunk()
F07-->>F02: RelativePose
F02->>F12: add_frame_to_chunk()
F12->>F10: add_relative_factor_to_chunk()
F02->>F04: fetch_tile() + compute_tile_bounds()
F04-->>F02: tile, tile_bounds
F02->>F09: align_to_satellite(img, tile, bounds)
F09-->>F02: AlignmentResult (GPS)
F02->>F10: add_absolute_factor()
F02->>F10: optimize_chunk()
F10-->>F02: OptimizationResult
F02->>F13: enu_to_gps()
F13-->>F02: GPSPoint
F02->>F14: update_frame_result()
F14->>F15: send_frame_result()
F15-->>Client: SSE "frame_processed"
```
## Flow 4: Rotation Sweep (First Frame / Sharp Turn)
```mermaid
sequenceDiagram
participant F02 as Flight Processor
participant F06 as Rotation Manager
participant H07 as Rotation Utils
participant F09 as Metric Refinement
participant F03 as Flight Database
F02->>F06: try_rotation_steps(img, tile, bounds)
loop For angle in [0°, 30°, ... 330°]
F06->>H07: rotate_image(img, angle)
H07-->>F06: rotated_img
F06->>F09: align_to_satellite(rotated_img, tile, bounds)
F09-->>F06: AlignmentResult
alt Match Found (confidence > 0.7)
F06->>F06: calculate_precise_angle()
F06->>F03: save_heading()
F06-->>F02: RotationResult
end
end
alt No Match Found
F06-->>F02: None (trigger recovery)
end
```
## Flow 5: Tracking Loss Recovery
```mermaid
sequenceDiagram
participant F02 as Flight Processor
participant F11 as Failure Recovery
participant F12 as Chunk Manager
participant F06 as Rotation Manager
participant F08 as Place Recognition
participant F04 as Satellite Manager
participant F09 as Metric Refinement
F02->>F11: start_search(frame_id, estimated_gps)
F11-->>F02: SearchSession
Note over F11: Emit RecoveryStarted event
F11->>F12: create_chunk_on_tracking_loss()
F12-->>F11: ChunkHandle (processing continues)
F11->>F06: requires_rotation_sweep()
F11->>F08: retrieve_candidate_tiles()
loop Progressive Search [1, 4, 9, 16, 25]
F11->>F04: expand_search_grid(grid_size)
F04-->>F11: tiles
loop For each tile
F11->>F04: compute_tile_bounds()
F11->>F09: align_to_satellite(img, tile, bounds)
alt Match Found
Note over F11: Emit RecoverySucceeded
F11-->>F02: RecoveryResult(success=true)
end
end
end
alt All Failed
Note over F11: Emit UserInputNeeded
F11-->>F02: RecoveryResult(success=false)
end
```
## Flow 6: Chunk Matching (Background)
```mermaid
sequenceDiagram
participant F11 as Failure Recovery
participant F12 as Chunk Manager
participant F08 as Place Recognition
participant F06 as Rotation Manager
participant F04 as Satellite Manager
participant F09 as Metric Refinement
participant F10 as Factor Graph
loop Every 5 seconds
F11->>F12: get_chunks_for_matching()
F12-->>F11: List[ChunkHandle]
loop For each unanchored chunk
F11->>F12: is_chunk_ready_for_matching()
alt Chunk Ready
F11->>F12: mark_chunk_matching()
Note over F11: Step 1: Semantic Matching
F11->>F12: get_chunk_images()
F11->>F08: retrieve_candidate_tiles_for_chunk()
F08-->>F11: List[TileCandidate]
Note over F11: Step 2: LiteSAM with Rotation
loop For each candidate tile
F11->>F04: get_tile + compute_tile_bounds()
F11->>F06: try_chunk_rotation_steps()
F06->>F09: align_chunk_to_satellite()
alt Match Found
F09-->>F11: ChunkAlignmentResult
end
end
alt Match Found
F11->>F12: mark_chunk_anchored()
F12->>F10: add_chunk_anchor()
F11->>F12: merge_chunks()
F12->>F10: merge_chunks(Sim3)
F11->>F10: optimize_global()
Note over F11: Emit ChunkMerged event
end
end
end
end
```
## Flow 7: User Input Recovery
```mermaid
sequenceDiagram
participant F11 as Failure Recovery
participant F08 as Place Recognition
participant F15 as SSE Streamer
participant C as Client
participant F01 as Flight API
participant F10 as Factor Graph
participant F02 as Flight Processor
F11->>F08: retrieve_candidate_tiles()
F08-->>F11: Top-5 candidates
F11->>F15: send_user_input_request()
F15-->>C: SSE "user_input_needed"
Note over F11: Emit UserInputNeeded event
Note over F02: Status = BLOCKED
C->>F01: POST /user-fix {pixel, gps}
F01->>F11: apply_user_anchor()
F11->>F10: add_absolute_factor(is_user_anchor=true)
F11->>F10: optimize()
Note over F11: Emit UserFixApplied event
Note over F02: Status = PROCESSING
F11-->>F01: Success
F01-->>C: 200 OK
```
## Flow 8: Result Publishing & Refinement
```mermaid
sequenceDiagram
participant F10 as Factor Graph
participant F14 as Result Manager
participant F13 as Coord Transformer
participant F03 as Flight Database
participant F15 as SSE Streamer
participant C as Client
Note over F10: New absolute factor added
F10->>F10: optimize(batch)
F10->>F14: mark_refined(frame_ids)
loop For each refined frame
F14->>F10: get_trajectory()
F10-->>F14: Pose (ENU)
F14->>F13: enu_to_gps(flight_id, enu)
F13-->>F14: GPSPoint
F14->>F03: save_frame_result(refined=true)
F14->>F03: update_waypoint()
F14->>F15: send_refinement()
F15-->>C: SSE "frame_refined"
end
```
## Flow 9: Object to GPS Conversion
```mermaid
sequenceDiagram
participant Ext as External Detector
participant F01 as Flight API
participant F13 as Coord Transformer
participant F10 as Factor Graph
participant F17 as Config Manager
participant H01 as Camera Model
Ext->>F01: POST /object-to-gps {pixel_x, pixel_y}
F01->>F13: image_object_to_gps(pixel, frame_id)
F13->>F10: get_pose(frame_id)
F10-->>F13: Pose (ENU)
F13->>F17: get_camera_params()
F17-->>F13: CameraParameters
F13->>H01: unproject(pixel)
F13->>F13: intersect_ground_plane()
F13->>F13: enu_to_gps()
F13-->>F01: GPSPoint
F01-->>Ext: {gps, accuracy_meters}
```
## Complete System Flow Overview
```mermaid
flowchart TB
subgraph Init["System Initialization"]
direction TB
I1[Load Config F17]
I2[Init DB F03]
I3[Load Models F16]
I4[Init Cache F04]
I5[Load Faiss F08]
I6[Start API F01]
I1 --> I2 --> I3 --> I4 --> I5 --> I6
end
subgraph Flight["Flight Lifecycle"]
direction TB
FL1[Create Flight]
FL2[Upload Images]
FL3[Process Frames]
FL4[Complete Flight]
FL1 --> FL2 --> FL3 --> FL4
end
subgraph Process["Frame Processing"]
direction TB
P1{First Frame?}
P2[Rotation Sweep]
P3[Sequential VO]
P4{Tracking OK?}
P5[Single Tile Match]
P6[Optimize]
P7[Publish Result]
P1 -->|Yes| P2
P1 -->|No| P3
P2 --> P3
P3 --> P4
P4 -->|Yes| P5
P5 --> P6
P6 --> P7
end
subgraph Recovery["Recovery Flow"]
direction TB
R1[Create Chunk]
R2[Progressive Search]
R3{Match Found?}
R4[Build Chunk]
R5[Chunk Matching]
R6{Chunk Match?}
R7[Merge Chunk]
R8[Request User Input]
R9[Apply User Anchor]
P4 -->|No| R1
R1 --> R2
R2 --> R3
R3 -->|Yes| P6
R3 -->|No| R4
R4 --> R5
R5 --> R6
R6 -->|Yes| R7
R7 --> P6
R6 -->|No| R8
R8 --> R9
R9 --> P6
end
Init --> Flight
FL3 --> Process
```
## Event Flow Diagram
```mermaid
flowchart LR
subgraph Events["F11 Emits Events"]
E1[RecoveryStarted]
E2[RecoverySucceeded]
E3[RecoveryFailed]
E4[UserInputNeeded]
E5[UserFixApplied]
E6[ChunkCreated]
E7[ChunkAnchored]
E8[ChunkMerged]
end
subgraph F02Sub["F02 Subscribes"]
S1[Update status: recovering]
S2[Update status: processing]
S3[Update status: blocked]
S4[Update status: blocked]
S5[Resume processing]
S6[Log chunk creation]
S7[Log anchor]
S8[Trigger result update]
end
subgraph F14Sub["F14 Subscribes"]
R1[Update merged frame results]
end
E1 --> S1
E2 --> S2
E3 --> S3
E4 --> S4
E5 --> S5
E6 --> S6
E7 --> S7
E8 --> S8
E8 --> R1
```
## Data Flow Through Layers
```mermaid
flowchart TB
subgraph Input["Input"]
IMG[UAV Images]
SAT[Satellite Tiles]
USR[User Anchors]
end
subgraph Processing["Processing"]
SP[SuperPoint Features]
LG[LightGlue Matches]
DINO[DINOv2 Descriptors]
LITE[LiteSAM Homography]
end
subgraph State["State Estimation"]
REL[Relative Factors]
ABS[Absolute Factors]
CHUNK[Chunk Subgraphs]
OPT[iSAM2 Optimization]
end
subgraph Output["Output"]
ENU[ENU Trajectory]
GPS[GPS Coordinates]
SSE[SSE Events]
DB[Database]
end
IMG --> SP
IMG --> DINO
IMG --> LITE
SAT --> LITE
SAT --> DINO
USR --> ABS
SP --> LG
LG --> REL
DINO --> ABS
LITE --> ABS
REL --> CHUNK
ABS --> CHUNK
CHUNK --> OPT
OPT --> ENU
ENU --> GPS
GPS --> SSE
GPS --> DB
```