gps-denied-onboard/docs/02_components/system_flows_diagrams.md

ASTRAL-Next System Flow Diagrams

Component Architecture Diagram

graph TB
    subgraph External["External Systems"]
        Client[Client UI]
        SatProv[Satellite Provider]
        ExtDet[External Detector]
    end

    subgraph API["API Layer"]
        F01[F01 Flight API]
    end

    subgraph Orchestration["Orchestration Layer"]
        F02[F02 Flight Processor]
    end

    subgraph Data["Data Management"]
        F03[F03 Flight Database]
        F04[F04 Satellite Data Manager]
        F05[F05 Image Input Pipeline]
        F12[F12 Route Chunk Manager]
    end

    subgraph Recovery["Recovery Layer"]
        F11[F11 Failure Recovery Coordinator]
    end

    subgraph Visual["Visual Processing"]
        F06[F06 Image Rotation Manager]
        F07[F07 Sequential Visual Odometry]
        F08[F08 Global Place Recognition]
        F09[F09 Metric Refinement]
    end

    subgraph State["State Estimation"]
        F10[F10 Factor Graph Optimizer]
    end

    subgraph Output["Output Layer"]
        F13[F13 Coordinate Transformer]
        F14[F14 Result Manager]
        F15[F15 SSE Event Streamer]
    end

    subgraph Infra["Infrastructure"]
        F16[F16 Model Manager]
        F17[F17 Configuration Manager]
    end

    Client -->|REST| F01
    F15 -->|SSE| Client
    ExtDet -->|REST| F01
    F04 -->|HTTP| SatProv

    F01 --> F02
    F02 --> F03
    F02 --> F05
    F02 --> F11
    F02 --> F07
    F02 --> F10
    F02 --> F12
    F02 --> F04

    F11 --> F08
    F11 --> F09
    F11 --> F12
    F11 -.->|events| F02
    F11 -.->|events| F14

    F06 --> F09
    F07 --> F10
    F09 --> F10
    F12 --> F10
    F10 --> F13
    F14 --> F13
    F14 --> F15
    F14 --> F03

    F07 --> F16
    F08 --> F16
    F09 --> F16
    F02 --> F17

Flow 1: System Initialization

sequenceDiagram
    participant Main as System
    participant F17 as Config Manager
    participant F03 as Flight Database
    participant F16 as Model Manager
    participant F04 as Satellite Manager
    participant F08 as Place Recognition
    participant F12 as Chunk Manager
    participant F02 as Flight Processor
    participant F01 as Flight API

    Main->>F17: load_config()
    F17-->>Main: SystemConfig
    
    Main->>F03: Initialize connections
    F03-->>Main: Connection pool ready
    
    Main->>F16: load_model("SuperPoint")
    Main->>F16: load_model("LightGlue")
    Main->>F16: load_model("DINOv2")
    Main->>F16: load_model("LiteSAM")
    F16-->>Main: Models loaded (~25s)
    
    Main->>F04: Initialize cache
    F04-->>Main: Cache ready
    
    Main->>F08: load_index()
    F08-->>Main: Faiss index loaded
    
    Main->>F12: Initialize
    F12-->>Main: Chunk tracking ready
    
    Main->>F02: Ready
    F02-->>Main: Ready to accept flights
    
    Main->>F01: Start server
    F01-->>Main: FastAPI running

Flow 2: Flight Creation

sequenceDiagram
    participant C as Client
    participant F01 as Flight API
    participant F02 as Flight Processor
    participant F17 as Config Manager
    participant F13 as Coord Transformer
    participant F04 as Satellite Manager
    participant F03 as Flight Database
    participant F15 as SSE Streamer

    C->>F01: POST /flights
    F01->>F02: create_flight()
    
    F02->>F17: get_flight_config()
    F17-->>F02: CameraParams, Altitude
    
    F02->>F13: set_enu_origin(start_gps)
    
    F02->>F04: prefetch_route_corridor()
    F04-->>F02: Prefetching async
    
    F02->>F03: insert_flight()
    F03-->>F02: flight_id
    
    F01-->>C: 201 Created {flight_id}
    
    C->>F01: GET /flights/{id}/stream
    F01->>F15: create_stream()
    F15-->>C: SSE Connection

Flow 3: Normal Frame Processing

sequenceDiagram
    participant F02 as Flight Processor
    participant F05 as Image Pipeline
    participant F12 as Chunk Manager
    participant F06 as Rotation Manager
    participant F07 as Sequential VO
    participant F04 as Satellite Manager
    participant F09 as Metric Refinement
    participant F10 as Factor Graph
    participant F13 as Coord Transformer
    participant F14 as Result Manager
    participant F15 as SSE Streamer

    F02->>F05: get_next_image()
    F05-->>F02: ImageData
    
    F02->>F12: get_active_chunk()
    F12-->>F02: ChunkHandle
    
    F02->>F06: requires_rotation_sweep()
    F06-->>F02: false (heading known)
    
    F02->>F07: compute_relative_pose_in_chunk()
    F07-->>F02: RelativePose
    
    F02->>F12: add_frame_to_chunk()
    F12->>F10: add_relative_factor_to_chunk()
    
    F02->>F04: fetch_tile() + compute_tile_bounds()
    F04-->>F02: tile, tile_bounds
    
    F02->>F09: align_to_satellite(img, tile, bounds)
    F09-->>F02: AlignmentResult (GPS)
    
    F02->>F10: add_absolute_factor()
    F02->>F10: optimize_chunk()
    F10-->>F02: OptimizationResult
    
    F02->>F13: enu_to_gps()
    F13-->>F02: GPSPoint
    
    F02->>F14: update_frame_result()
    F14->>F15: send_frame_result()
    F15-->>Client: SSE "frame_processed"

Flow 4: Rotation Sweep (First Frame / Sharp Turn)

sequenceDiagram
    participant F02 as Flight Processor
    participant F06 as Rotation Manager
    participant H07 as Rotation Utils
    participant F09 as Metric Refinement
    participant F03 as Flight Database

    F02->>F06: try_rotation_steps(img, tile, bounds)
    
    loop For angle in [0°, 30°, ... 330°]
        F06->>H07: rotate_image(img, angle)
        H07-->>F06: rotated_img
        
        F06->>F09: align_to_satellite(rotated_img, tile, bounds)
        F09-->>F06: AlignmentResult
        
        alt Match Found (confidence > 0.7)
            F06->>F06: calculate_precise_angle()
            F06->>F03: save_heading()
            F06-->>F02: RotationResult
        end
    end
    
    alt No Match Found
        F06-->>F02: None (trigger recovery)
    end

Flow 5: Tracking Loss Recovery

sequenceDiagram
    participant F02 as Flight Processor
    participant F11 as Failure Recovery
    participant F12 as Chunk Manager
    participant F06 as Rotation Manager
    participant F08 as Place Recognition
    participant F04 as Satellite Manager
    participant F09 as Metric Refinement

    F02->>F11: start_search(frame_id, estimated_gps)
    F11-->>F02: SearchSession
    
    Note over F11: Emit RecoveryStarted event
    
    F11->>F12: create_chunk_on_tracking_loss()
    F12-->>F11: ChunkHandle (processing continues)
    
    F11->>F06: requires_rotation_sweep()
    F11->>F08: retrieve_candidate_tiles()
    
    loop Progressive Search [1, 4, 9, 16, 25]
        F11->>F04: expand_search_grid(grid_size)
        F04-->>F11: tiles
        
        loop For each tile
            F11->>F04: compute_tile_bounds()
            F11->>F09: align_to_satellite(img, tile, bounds)
            
            alt Match Found
                Note over F11: Emit RecoverySucceeded
                F11-->>F02: RecoveryResult(success=true)
            end
        end
    end
    
    alt All Failed
        Note over F11: Emit UserInputNeeded
        F11-->>F02: RecoveryResult(success=false)
    end

Flow 6: Chunk Matching (Background)

sequenceDiagram
    participant F11 as Failure Recovery
    participant F12 as Chunk Manager
    participant F08 as Place Recognition
    participant F06 as Rotation Manager
    participant F04 as Satellite Manager
    participant F09 as Metric Refinement
    participant F10 as Factor Graph

    loop Every 5 seconds
        F11->>F12: get_chunks_for_matching()
        F12-->>F11: List[ChunkHandle]
        
        loop For each unanchored chunk
            F11->>F12: is_chunk_ready_for_matching()
            
            alt Chunk Ready
                F11->>F12: mark_chunk_matching()
                
                Note over F11: Step 1: Semantic Matching
                F11->>F12: get_chunk_images()
                F11->>F08: retrieve_candidate_tiles_for_chunk()
                F08-->>F11: List[TileCandidate]
                
                Note over F11: Step 2: LiteSAM with Rotation
                loop For each candidate tile
                    F11->>F04: get_tile + compute_tile_bounds()
                    F11->>F06: try_chunk_rotation_steps()
                    F06->>F09: align_chunk_to_satellite()
                    
                    alt Match Found
                        F09-->>F11: ChunkAlignmentResult
                    end
                end
                
                alt Match Found
                    F11->>F12: mark_chunk_anchored()
                    F12->>F10: add_chunk_anchor()
                    
                    F11->>F12: merge_chunks()
                    F12->>F10: merge_chunks(Sim3)
                    
                    F11->>F10: optimize_global()
                    
                    Note over F11: Emit ChunkMerged event
                end
            end
        end
    end

Flow 7: User Input Recovery

sequenceDiagram
    participant F11 as Failure Recovery
    participant F08 as Place Recognition
    participant F15 as SSE Streamer
    participant C as Client
    participant F01 as Flight API
    participant F10 as Factor Graph
    participant F02 as Flight Processor

    F11->>F08: retrieve_candidate_tiles()
    F08-->>F11: Top-5 candidates
    
    F11->>F15: send_user_input_request()
    F15-->>C: SSE "user_input_needed"
    
    Note over F11: Emit UserInputNeeded event
    Note over F02: Status = BLOCKED
    
    C->>F01: POST /user-fix {pixel, gps}
    F01->>F11: apply_user_anchor()
    
    F11->>F10: add_absolute_factor(is_user_anchor=true)
    F11->>F10: optimize()
    
    Note over F11: Emit UserFixApplied event
    Note over F02: Status = PROCESSING
    
    F11-->>F01: Success
    F01-->>C: 200 OK

Flow 8: Result Publishing & Refinement

sequenceDiagram
    participant F10 as Factor Graph
    participant F14 as Result Manager
    participant F13 as Coord Transformer
    participant F03 as Flight Database
    participant F15 as SSE Streamer
    participant C as Client

    Note over F10: New absolute factor added
    
    F10->>F10: optimize(batch)
    
    F10->>F14: mark_refined(frame_ids)
    
    loop For each refined frame
        F14->>F10: get_trajectory()
        F10-->>F14: Pose (ENU)
        
        F14->>F13: enu_to_gps(flight_id, enu)
        F13-->>F14: GPSPoint
        
        F14->>F03: save_frame_result(refined=true)
        F14->>F03: update_waypoint()
        
        F14->>F15: send_refinement()
        F15-->>C: SSE "frame_refined"
    end

Flow 9: Object to GPS Conversion

sequenceDiagram
    participant Ext as External Detector
    participant F01 as Flight API
    participant F13 as Coord Transformer
    participant F10 as Factor Graph
    participant F17 as Config Manager
    participant H01 as Camera Model

    Ext->>F01: POST /object-to-gps {pixel_x, pixel_y}
    F01->>F13: image_object_to_gps(pixel, frame_id)
    
    F13->>F10: get_pose(frame_id)
    F10-->>F13: Pose (ENU)
    
    F13->>F17: get_camera_params()
    F17-->>F13: CameraParameters
    
    F13->>H01: unproject(pixel)
    F13->>F13: intersect_ground_plane()
    F13->>F13: enu_to_gps()
    
    F13-->>F01: GPSPoint
    F01-->>Ext: {gps, accuracy_meters}

Complete System Flow Overview

flowchart TB
    subgraph Init["System Initialization"]
        direction TB
        I1[Load Config F17]
        I2[Init DB F03]
        I3[Load Models F16]
        I4[Init Cache F04]
        I5[Load Faiss F08]
        I6[Start API F01]
        I1 --> I2 --> I3 --> I4 --> I5 --> I6
    end

    subgraph Flight["Flight Lifecycle"]
        direction TB
        FL1[Create Flight]
        FL2[Upload Images]
        FL3[Process Frames]
        FL4[Complete Flight]
        FL1 --> FL2 --> FL3 --> FL4
    end

    subgraph Process["Frame Processing"]
        direction TB
        P1{First Frame?}
        P2[Rotation Sweep]
        P3[Sequential VO]
        P4{Tracking OK?}
        P5[Single Tile Match]
        P6[Optimize]
        P7[Publish Result]
        
        P1 -->|Yes| P2
        P1 -->|No| P3
        P2 --> P3
        P3 --> P4
        P4 -->|Yes| P5
        P5 --> P6
        P6 --> P7
    end

    subgraph Recovery["Recovery Flow"]
        direction TB
        R1[Create Chunk]
        R2[Progressive Search]
        R3{Match Found?}
        R4[Build Chunk]
        R5[Chunk Matching]
        R6{Chunk Match?}
        R7[Merge Chunk]
        R8[Request User Input]
        R9[Apply User Anchor]
        
        P4 -->|No| R1
        R1 --> R2
        R2 --> R3
        R3 -->|Yes| P6
        R3 -->|No| R4
        R4 --> R5
        R5 --> R6
        R6 -->|Yes| R7
        R7 --> P6
        R6 -->|No| R8
        R8 --> R9
        R9 --> P6
    end

    Init --> Flight
    FL3 --> Process

Event Flow Diagram

flowchart LR
    subgraph Events["F11 Emits Events"]
        E1[RecoveryStarted]
        E2[RecoverySucceeded]
        E3[RecoveryFailed]
        E4[UserInputNeeded]
        E5[UserFixApplied]
        E6[ChunkCreated]
        E7[ChunkAnchored]
        E8[ChunkMerged]
    end

    subgraph F02Sub["F02 Subscribes"]
        S1[Update status: recovering]
        S2[Update status: processing]
        S3[Update status: blocked]
        S4[Update status: blocked]
        S5[Resume processing]
        S6[Log chunk creation]
        S7[Log anchor]
        S8[Trigger result update]
    end

    subgraph F14Sub["F14 Subscribes"]
        R1[Update merged frame results]
    end

    E1 --> S1
    E2 --> S2
    E3 --> S3
    E4 --> S4
    E5 --> S5
    E6 --> S6
    E7 --> S7
    E8 --> S8
    E8 --> R1

Data Flow Through Layers

flowchart TB
    subgraph Input["Input"]
        IMG[UAV Images]
        SAT[Satellite Tiles]
        USR[User Anchors]
    end

    subgraph Processing["Processing"]
        SP[SuperPoint Features]
        LG[LightGlue Matches]
        DINO[DINOv2 Descriptors]
        LITE[LiteSAM Homography]
    end

    subgraph State["State Estimation"]
        REL[Relative Factors]
        ABS[Absolute Factors]
        CHUNK[Chunk Subgraphs]
        OPT[iSAM2 Optimization]
    end

    subgraph Output["Output"]
        ENU[ENU Trajectory]
        GPS[GPS Coordinates]
        SSE[SSE Events]
        DB[Database]
    end

    IMG --> SP
    IMG --> DINO
    IMG --> LITE
    SAT --> LITE
    SAT --> DINO
    USR --> ABS

    SP --> LG
    LG --> REL
    DINO --> ABS
    LITE --> ABS

    REL --> CHUNK
    ABS --> CHUNK
    CHUNK --> OPT

    OPT --> ENU
    ENU --> GPS
    GPS --> SSE
    GPS --> DB