azaion/gps-denied-desktop
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-desktop.git synced 2026-04-22 22:06:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
634e6d7574fbc562e4a5fdc9dc5d9c9585d7d49d
gps-denied-desktop/_docs/02_components/decomposition_plan.md
T
Oleksandr Bezdieniezhnykh abc26d5c20 initial structure implemented 
docs -> _docs
2025-12-01 14:20:56 +02:00

23 KiB
Raw Blame History

ASTRAL-Next System Component Decomposition Plan

Design Principle: Interface-Based Architecture

CRITICAL REQUIREMENT: Each component MUST implement a well-defined interface to ensure interchangeability with different implementations.

Benefits:

  • Swap implementations (e.g., replace LiteSAM with TransFG, GTSAM with Ceres)
  • Enable unit testing with mocks
  • Support multiple backends (TensorRT vs ONNX, different databases)
  • Facilitate future enhancements without breaking contracts

Interface Specification: Each component spec must define:

  • Interface name (e.g., ISatelliteDataManager, IMetricRefinement)
  • All public methods with strict contracts
  • Input/output data structures
  • Error conditions and exceptions
  • Performance guarantees

System Architecture Overview

Single unified Flight API:

  • Flight CRUD operations (create, read, update, delete)
  • Waypoint management within flights
  • Geofence management
  • Tri-layer localization (SuperPoint+LightGlue, DINOv2, LiteSAM)
  • Calls satellite provider for tiles
  • Rotation preprocessing (LiteSAM 45° limit)
  • Per-frame waypoint updates
  • Progressive tile search (1→4→9→16→25)
  • SSE streaming for real-time results

FLIGHT API COMPONENTS (17 components)

Core API Layer

F01_flight_api Interface: IFlightAPI Endpoints: POST /flights, GET /flights/{flightId}, DELETE /flights/{flightId}, PUT /flights/{flightId}/waypoints/{waypointId}, POST .../images/batch, POST .../user-fix, GET .../status, GET .../stream

F02.1_flight_lifecycle_manager Interface: IFlightLifecycleManager API: create_flight(), get_flight(), get_flight_state(), delete_flight(), update_waypoint(), batch_update_waypoints(), validate_waypoint(), validate_geofence(), queue_images(), handle_user_fix(), create_client_stream(), convert_object_to_gps(), initialize_system()

F02.2_flight_processing_engine Interface: IFlightProcessingEngine API: start_processing(), stop_processing(), process_frame(), apply_user_fix(), handle_tracking_loss(), get_active_chunk(), create_new_chunk()

F03_flight_database Interface: IFlightDatabase API: insert_flight(), update_flight(), query_flights(), get_flight_by_id(), delete_flight(), get_waypoints(), insert_waypoint(), update_waypoint(), batch_update_waypoints(), save_flight_state(), load_flight_state(), save_frame_result(), get_frame_results(), save_heading(), get_heading_history(), get_latest_heading(), save_image_metadata(), get_image_path(), get_image_metadata()

Data Management

F04_satellite_data_manager Interface: ISatelliteDataManager API: fetch_tile(), fetch_tile_grid(), prefetch_route_corridor(), progressive_fetch(), cache_tile(), get_cached_tile(), compute_tile_coords(), expand_search_grid(), compute_tile_bounds() Features: Progressive retrieval, tile caching, grid calculations

F05_image_input_pipeline Interface: IImageInputPipeline API: queue_batch(), process_next_batch(), validate_batch(), store_images(), get_next_image(), get_image_by_sequence() Features: FIFO queuing, validation, storage

F06_image_rotation_manager Interface: IImageRotationManager API: rotate_image_360(), try_rotation_steps(), calculate_precise_angle(), get_current_heading(), update_heading(), detect_sharp_turn(), requires_rotation_sweep() Features: 30° rotation sweeps, heading tracking

Visual Processing

F07_sequential_visual_odometry Interface: ISequentialVisualOdometry API: compute_relative_pose(), extract_features(), match_features(), estimate_motion()

F08_global_place_recognition Interface: IGlobalPlaceRecognition API: retrieve_candidate_tiles(), compute_location_descriptor(), query_database(), rank_candidates()

F09_metric_refinement Interface: IMetricRefinement API: align_to_satellite(uav_image, satellite_tile, tile_bounds), compute_homography(), extract_gps_from_alignment(), compute_match_confidence()

State Estimation

F10_factor_graph_optimizer Interface: IFactorGraphOptimizer API: add_relative_factor(), add_absolute_factor(), add_altitude_prior(), optimize(), get_trajectory(), get_marginal_covariance()

F11_failure_recovery_coordinator Interface: IFailureRecoveryCoordinator API: check_confidence(), detect_tracking_loss(), start_search(), expand_search_radius(), try_current_grid(), create_user_input_request(), apply_user_anchor()

F12_route_chunk_manager Interface: IRouteChunkManager API: create_chunk(), add_frame_to_chunk(), get_chunk_frames(), get_chunk_images(), get_chunk_composite_descriptor(), get_chunk_bounds(), is_chunk_ready_for_matching(), mark_chunk_anchored(), get_chunks_for_matching(), get_active_chunk(), deactivate_chunk() Features: Chunk lifecycle management, chunk state tracking, chunk matching coordination

F13_coordinate_transformer Interface: ICoordinateTransformer API: set_enu_origin(), get_enu_origin(), gps_to_enu(), enu_to_gps(), pixel_to_gps(), gps_to_pixel(), image_object_to_gps(), transform_points() Dependencies: F10 Factor Graph Optimizer (for frame poses), H02 GSD Calculator (for GSD computation)

Results & Communication

F14_result_manager Interface: IResultManager API: update_frame_result(), publish_waypoint_update(), get_flight_results(), mark_refined()

F15_sse_event_streamer Interface: ISSEEventStreamer API: create_stream(), send_frame_result(), send_search_progress(), send_user_input_request(), send_refinement()

Infrastructure

F16_model_manager Interface: IModelManager API: load_model(), get_inference_engine(), optimize_to_tensorrt(), fallback_to_onnx()

F17_configuration_manager Interface: IConfigurationManager API: load_config(), get_camera_params(), validate_config(), get_flight_config()

HELPER COMPONENTS (8 components)

H01_camera_model - ICameraModel H02_gsd_calculator - IGSDCalculator H03_robust_kernels - IRobustKernels H04_faiss_index_manager - IFaissIndexManager H05_performance_monitor - IPerformanceMonitor H06_web_mercator_utils - IWebMercatorUtils H07_image_rotation_utils - IImageRotationUtils H08_batch_validator - IBatchValidator

System Startup Initialization Order

Startup sequence (blocking, sequential):

Order Component Method Purpose Dependencies
1 F17 Configuration Manager load_config() Load system configuration None
2 F03 Flight Database Initialize connections Establish DB connection pool F17
3 F16 Model Manager load_model("SuperPoint") Load SuperPoint feature extractor F17
4 F16 Model Manager load_model("LightGlue") Load LightGlue matcher F17
5 F16 Model Manager load_model("DINOv2") Load DINOv2 for place recognition F17
6 F16 Model Manager load_model("LiteSAM") Load LiteSAM for cross-view matching F17
7 F04 Satellite Data Manager Initialize cache Initialize tile cache directory F17
8 F08 Global Place Recognition load_index() Load pre-built Faiss index from satellite provider F04, F16, H04
9 F12 Route Chunk Manager Initialize Initialize chunk state tracking F10
10 F02 Flight Processor Ready Ready to accept flights All above
11 F01 Flight API Start server Start FastAPI/Uvicorn F02

Estimated total startup time: ~30 seconds (dominated by model loading)

Shutdown sequence (reverse order):

  1. F01 Flight API - Stop accepting requests
  2. F02 Flight Processor - Complete or cancel active flights
  3. F11 Failure Recovery Coordinator - Stop background chunk matching
  4. F12 Route Chunk Manager - Save chunk state
  5. F16 Model Manager - Unload models
  6. F03 Flight Database - Close connections
  7. F04 Satellite Data Manager - Flush cache

Comprehensive Component Interaction Matrix

System Initialization

Source Target Method Purpose
F02.1 F17 load_config() Load system configuration
F02.1 F16 load_model() × 4 Load SuperPoint, LightGlue, DINOv2, LiteSAM
F04 F08 Satellite tiles + index F08 loads pre-built Faiss index from provider
F08 H04 load_index() Load satellite descriptor index
F08 F16 get_inference_engine("DINOv2") Get model for descriptor computation

Flight Creation

Source Target Method Purpose
Client F01 POST /flights Create flight
F01 F02.1 create_flight() Initialize flight state
F02.1 F17 get_flight_config() Get camera params, altitude
F02.1 F13 set_enu_origin(flight_id, start_gps) Set ENU coordinate origin
F02.1 F04 prefetch_route_corridor() Prefetch tiles
F04 Satellite Provider GET /api/satellite/tiles/batch HTTP batch download (includes tile metadata)
F04 H06 compute_tile_bounds() Tile coordinate calculations
F02.1 F03 insert_flight() Persist flight data

SSE Stream Creation

Source Target Method Purpose
Client F01 GET .../stream Open SSE connection
F01 F02.1 create_client_stream() Route through lifecycle manager
F02.1 F15 create_stream() Establish SSE channel

Image Upload

Source Target Method Purpose
Client F01 POST .../images/batch Upload 10-50 images
F01 F02.1 queue_images() Route through lifecycle manager
F02.1 F05 queue_batch() Queue for processing
F05 H08 validate_batch() Validate sequence, format
F05 F03 save_image_metadata() Persist image metadata

Per-Frame Processing (First Frame / Sharp Turn)

Source Target Method Purpose
F02.2 F05 get_next_image() Get image for processing
F02.2 F06 requires_rotation_sweep() Check if sweep needed
F06 H07 rotate_image() × 12 Rotate in 30° steps
F06 F09 align_to_satellite(img, tile, bounds) × 12 Try LiteSAM each rotation
F02.2 F04 get_cached_tile() + compute_tile_bounds() Get expected tile with bounds
F09 F16 get_inference_engine("LiteSAM") Get model
F06 H07 calculate_rotation_from_points() Precise angle from homography
F02.2 F03 save_heading() Store UAV heading

Per-Frame Processing (Sequential VO)

Source Target Method Purpose
F02.2 F12 get_active_chunk() Get active chunk for frame
F02.2 F07 compute_relative_pose() Provide image and chunk context
F07 F16 get_inference_engine("SuperPoint") Get feature extractor
F07 F16 get_inference_engine("LightGlue") Get matcher
F07 H05 start_timer(), end_timer() Monitor timing
F02.2 F10 add_relative_factor_to_chunk() Add pose measurement to chunk subgraph
F02.2 F12 add_frame_to_chunk() Add frame to chunk

Tracking Good (Drift Correction)

Source Target Method Purpose
F02.2 F11 check_confidence() Check tracking quality
F02.2 F04 fetch_tile() + compute_tile_bounds() Get single tile with bounds
F02.2 F09 align_to_satellite(img, tile, bounds) Align to 1 tile
F02.2 F10 add_absolute_factor() Add GPS measurement

Tracking Lost (Progressive Search + Chunk Building)

Source Target Method Purpose
F02.2 F11 check_confidence() → FAIL Low confidence
F11 F12 create_chunk_on_tracking_loss() Proactive chunk creation
F12 F10 create_chunk_subgraph() Create chunk in factor graph
F12 F03 save_chunk_state() Persist chunk state for recovery
F02.2 F12 get_active_chunk() Get new active chunk
F11 F06 requires_rotation_sweep() Trigger rotation sweep (single-image)
F11 F08 retrieve_candidate_tiles() Coarse localization (single-image)
F08 F16 get_inference_engine("DINOv2") Get model
F08 H04 search() Query Faiss index
F08 F04 get_tile_by_gps() × 5 Get candidate tiles
F11 F04 expand_search_grid(4) Get 2×2 grid
F11 F09 align_to_satellite(img, tile, bounds) Try LiteSAM on tiles
F11 (fail) F04 expand_search_grid(9) Expand to 3×3
F11 (fail) F04 expand_search_grid(16) Expand to 4×4
F11 (fail) F04 expand_search_grid(25) Expand to 5×5
F11 (fail) F12 Continue building chunk Chunk building continues
F11 (background) F12 get_chunks_for_matching() Get unanchored chunks
F11 (background) F08 retrieve_candidate_tiles_for_chunk() Chunk semantic matching
F11 (background) F06 try_chunk_rotation_steps() Chunk rotation sweeps
F11 (background) F09 align_chunk_to_satellite() Chunk LiteSAM matching
F11 (background) F12 mark_chunk_anchored() + merge_chunks(main, new) Chunk merging via F12
F11 (fail) F02.2 Returns UserInputRequest Request human help (last resort)
F02.2 F15 send_user_input_request() Send SSE event to client

Optimization & Results

Source Target Method Purpose
F10 H03 huber_loss(), cauchy_loss() Apply robust kernels
F10 Internal optimize() Run iSAM2 optimization
F02.2 F10 get_trajectory() Get optimized poses
F02.2 F13 enu_to_gps() Convert ENU to GPS
F13 H01 project(), unproject() Camera operations
F13 H02 compute_gsd() GSD calculations
F13 H06 tile_to_latlon() Coordinate transforms
F02.2 F14 Frame GPS + object coords Provide results
F14 F03 update_waypoint() Per-frame waypoint update
F14 F15 send_frame_result() Publish to client
F15 Client SSE frame_processed Real-time delivery
F14 F03 save_frame_result() Persist frame result

User Input Recovery

Source Target Method Purpose
F15 Client SSE user_input_needed Notify client
Client F01 POST .../user-fix Provide anchor
F01 F02.1 handle_user_fix() Route through lifecycle manager
F02.1 F02.2 apply_user_fix() Delegate to processing engine
F02.2 F11 apply_user_anchor() Apply fix
F11 F10 add_absolute_factor() (high confidence) Hard constraint
F10 Internal optimize() Re-optimize
F02.2 F15 send_frame_result() Publish result via SSE

Asynchronous Refinement

Source Target Method Purpose
F10 Internal (background) optimize() Back-propagate anchors
F02.2 F10 get_trajectory() Get refined poses (ENU)
F02.2 F13 enu_to_gps() Convert ENU poses to GPS
F02.2 F14 mark_refined(List[RefinedFrameResult]) Pass GPS-converted results
F14 F03 batch_update_waypoints() Batch update waypoints
F14 F15 send_refinement() × N Send updates
F15 Client SSE frame_refined × N Incremental updates

Note: F14 does NOT call F10 or F13. F02.2 performs coordinate conversion and provides GPS results to F14.

Chunk Matching (Background)

Source Target Method Purpose
F11 (background) F12 get_chunks_for_matching() Get unanchored chunks ready for matching
F11 F12 get_chunk_images() Get chunk images
F12 F05 get_image_by_sequence() Load images for chunk (F12 delegates to F05 for actual image retrieval)
F11 F08 retrieve_candidate_tiles_for_chunk() Chunk semantic matching (aggregate DINOv2)
F08 F16 get_inference_engine("DINOv2") Get model for descriptor computation
F08 H04 search() Query Faiss with chunk descriptor
F11 F06 try_chunk_rotation_steps() Chunk rotation sweeps (12 rotations)
F06 F09 align_chunk_to_satellite() × 12 Try LiteSAM for each rotation
F11 F10 add_chunk_anchor() Anchor chunk with GPS
F11 F12 merge_chunks(main_chunk, new_chunk) Merge new_chunk into main_chunk (Sim3 transform)
F10 Internal optimize_global() Global optimization after merging
F11 F12 mark_chunk_anchored() Update chunk state

Cross-Cutting Concerns

Source Target Method Purpose
F17 ALL get_*_config() Provide configuration
H05 F07, F08, F09, F10, F11 start_timer(), end_timer() Performance monitoring

Interaction Coverage Verification

✅ Initialization: F02.1→F16, F17; F04→F08→H04 ✅ Flight creation: Client→F01→F02.1→F04,F12,F16,F17,F14 ✅ Image upload: Client→F01→F02.1→F05→H08 ✅ Rotation sweep: F06→H07,F09 (12 iterations) ✅ Sequential VO: F02.2→F07→F16,F10(chunk),F12,H05 ✅ Drift correction: F02.2→F04,F09,F10 ✅ Tracking loss: F02.2→F11→F12(proactive chunk),F06,F08,F04(progressive),F09 ✅ Chunk building: F02.2→F12→F10,F07 ✅ Chunk image retrieval: F12→F05(get_image_by_sequence for chunk images) ✅ Chunk semantic matching: F11→F12→F08(chunk descriptor) ✅ Chunk LiteSAM matching: F11→F06(chunk rotation)→F09(chunk alignment) ✅ Chunk merging: F11→F10(Sim3 transform) ✅ Global PR: F08→F16,H04,F04 ✅ Optimization: F10→H03(chunk optimization, global optimization) ✅ Coordinate transform: F13→F10,H01,H02,H06 ✅ Results: F02.2→F13→F14→F15,F03 ✅ User input: Client→F01→F02.1→F02.2→F11→F10 ✅ Refinement: F10→F14→F03,F15 ✅ Configuration: F17→ALL ✅ Performance: H05→processing components

All major component interactions are covered.

Deliverables

Component Count: 25 total

  • Flight API: 17 (F01-F17)
  • Helpers: 8 (H01-H08)

For each component, create docs/02_components/[##]_[component_name]/[component_name]_spec.md:

  1. Interface Definition (interface name, methods, contracts)
  2. Component Description (responsibilities, scope)
  3. API Methods (inputs, outputs, errors, which components call it, test cases)
  4. Integration Tests
  5. Non-Functional Requirements (performance, accuracy targets)
  6. Dependencies (which components it calls)
  7. Data Models

Component Specifications Status

  • F01 Flight API (merged from R01 Route REST API)
  • F02.1 Flight Lifecycle Manager (split from F02 Flight Processor)
  • F02.2 Flight Processing Engine (split from F02 Flight Processor)
  • F03 Flight Database (merged from R04, G17)
  • F04 Satellite Data Manager
  • F05 Image Input Pipeline
  • F06 Image Rotation Manager
  • F07 Sequential Visual Odometry
  • F08 Global Place Recognition
  • F09 Metric Refinement
  • F10 Factor Graph Optimizer
  • F11 Failure Recovery Coordinator
  • F12 Route Chunk Manager (Atlas multi-map chunk lifecycle)
  • F13 Coordinate Transformer (with ENU origin)
  • F14 Result Manager
  • F15 SSE Event Streamer
  • F16 Model Manager
  • F17 Configuration Manager
  • Helper components (H01-H08)

Architecture Notes

F02 Split: Lifecycle Manager + Processing Engine

F02 has been split into two components with clear Single Responsibility:

F02.1 Flight Lifecycle Manager:

  • Flight CRUD operations
  • System initialization
  • API request routing
  • SSE stream creation
  • User fix delegation

F02.2 Flight Processing Engine:

  • Main processing loop
  • Frame-by-frame processing orchestration
  • Recovery coordination with F11
  • Chunk management coordination with F12
  • Runs in background thread per active flight

Rationale: This split aligns with Single Responsibility Principle. F02.1 handles external-facing operations (API layer), while F02.2 handles internal processing (background engine).

Decision: Keep as single component. Clear internal organization with separate methods for state vs processing concerns. Event-based communication with F11 keeps recovery logic separate.

Error Recovery Strategy

Per-Component Recovery:

  • F02: Persists flight state via F03 on each significant update. On restart, loads last known state.
  • F07: Stateless - reprocesses frame if VO fails
  • F10: Factor graph state persisted periodically. On restart, rebuilds from F03 checkpoint.
  • F11: Chunk state persisted via F12→F03. Recovery continues from last chunk state.
  • F12: All chunk state persisted to F03. Restores chunk handles on restart.

System-Wide Recovery:

  1. On crash, F02 loads flight state from F03
  2. F12 restores chunk state from F03
  3. Processing resumes from last successfully processed frame
  4. Incomplete chunks continue building/matching

Event Recovery:

  • Events are fire-and-forget (no persistence)
  • Subscribers rebuild state from F03 on restart

Error Propagation Strategy

Principle: Errors propagate upward through the component hierarchy. Lower-level components throw exceptions or return error results; higher-level coordinators handle recovery.

Error Propagation Chain:

H01-H08 (Helpers) → F07/F08/F09 (Visual Processing) → F11 (Recovery) → F02 (Coordinator) → F01 (API) → Client
                                                                  ↓
                                                               Events → F14 → F15 → SSE → Client

Error Categories:

  1. Recoverable (F11 handles):

    • Tracking loss → Progressive search
    • Low confidence → Rotation sweep
    • Chunk matching fails → User input request

  2. Propagated to Client (via SSE):

    • User input needed → user_input_needed event
    • Processing blocked → processing_blocked event
    • Flight completed → flight_completed event

  3. Fatal (F02 handles, returns to F01):

    • Database connection lost → HTTP 503
    • Model loading failed → HTTP 500
    • Invalid configuration → HTTP 400

User Fix Flow:

Client ---> F01 (POST /user-fix) ---> F02.handle_user_fix() ---> F11.apply_user_anchor()
                                                                      ↓
                                                              F10.add_chunk_anchor()
                                                                      ↓
                                                              F02 receives UserFixApplied event
                                                                      ↓
                                                              F14.publish_result() ---> F15 ---> SSE ---> Client
Reference in New Issue View Git Blame Copy Permalink