add solution drafts, add component decomposition , add spec for other docs

docs/00_problem/1.4_04_assesment_prompt.md

@@ -0,0 +1,370 @@
+Read carefully about the problem:
+
+ We have a lot of images taken from a wing-type UAV using a camera with at least Full HD resolution. Resolution of each photo could be up to 6200*4100 for the whole flight, but for other flights, it could be FullHD
+Photos are taken and named consecutively within 100 meters of each other.
+We know only the starting GPS coordinates. We need to determine the GPS of the centers of each image. And also the coordinates of the center of any object in these photos. We can use an external satellite provider for ground checks on the existing photos
+ 
+ System has next restrictions and conditions:
+  - Photos are taken by only airplane type UAVs.
+  - Photos are taken by the camera pointing downwards and fixed, but it is not autostabilized.
+  - The flying range is restricted by the eastern and southern parts of Ukraine (To the left of the Dnipro River)
+  - The image resolution could be from FullHD to 6252*4168. Camera parameters are known: focal length, sensor width, resolution and so on.
+  - Altitude is predefined and no more than 1km. The height of the terrain can be neglected.
+  - There is NO data from IMU
+  - Flights are done mostly in sunny weather
+  - We can use satellite providers, but we're limited right now to Google Maps, which could be outdated for some regions
+  - Number of photos could be up to 3000, usually in the 500-1500 range
+  - During the flight, UAVs can make sharp turns, so that the next photo may be absolutely different from the previous one (no same objects), but it is rather an exception than the rule
+  - Processing is done on a stationary computer or laptop with NVidia GPU at least RTX2060, better 3070. (For the UAV solution Jetson Orin Nano would be used, but that is out of scope.)
+  
+ Output of the system should address next acceptance criteria:
+  - The system should find out the GPS of centers of 80% of the photos from the flight within an error of no more than 50 meters in comparison to the real GPS
+  - The system should find out the GPS of centers of 60% of the photos from the flight within an error of no more than 20 meters in comparison to the real GPS
+  - The system should correctly continue the work even in the presence of up to 350 meters of an outlier photo between 2 consecutive pictures en route. This could happen due to tilt of the plane.
+  - System should correctly continue the work even during sharp turns, where the next photo doesn't overlap at all, or overlaps in less than 5%. The next photo should be in less than 150m drift and at an angle of less than 50%
+  - The number of outliers during the satellite provider images ground check should be less than 10%
+  - System should try to operate when UAV made a sharp turn, and all the next photos has no common points with previous route. In that situation system should try to figure out location of the new piece of the route and connect it to the previous route. Also this separate chunks could be more than 2, so this strategy should be in the core of the system
+  - In case of being absolutely incapable of determining the system to determine next, second next, and third next images GPS, by any means (these 20% of the route), then it should ask the user for input for the next image, so that the user can specify the location
+  - Less than 5 seconds for processing one image
+  - Results of image processing should appear immediately to user, so that user shouldn't wait for the whole route to complete in order to analyze first results. Also, system could refine existing calculated results and send refined results again to user 
+  - Image Registration Rate > 95%. The system can find enough matching features to confidently calculate the camera's 6-DoF pose (position and orientation) and "stitch" that image into the final trajectory
+  - Mean Reprojection Error (MRE) < 1.0 pixels. The distance, in pixels, between the original pixel location of the object and the re-projected pixel location.
+
+ Here is a solution draft:
+
+    ##  **The ATLAS-GEOFUSE System Architecture**
+
+    Multi-component architecture designed for high-performance, real-time geolocalization in IMU-denied, high-drift environments. Its architecture is explicitly designed around **pre-flight data caching** and **multi-map robustness**.
+
+    ### **2.1 Core Design Principles**
+
+    1. **Pre-Flight Caching:** To meet the <5s (AC-7) real-time requirement, all network latency must be eliminated. The system mandates a "Pre-Flight" step (Section 3.0) where all geospatial data (satellite tiles, DEMs, vector data) for the Area of Interest (AOI) is downloaded from a viable open-source provider (e.g., Copernicus 6) and stored in a local database on the processing laptop. All real-time queries are made against this local cache.  
+    2. **Decoupled Multi-Map SLAM:** The system separates *relative* motion from *absolute* scale. A Visual SLAM (V-SLAM) "Atlas" Front-End (Section 4.0) computes high-frequency, robust, but *unscaled* relative motion. A Local Geospatial Anchoring Back-End (GAB) (Section 5.0) provides sparse, high-confidence, *absolute metric* anchors by querying the local cache. A Trajectory Optimization Hub (TOH) (Section 6.0) fuses these two streams in a Sim(3) pose-graph to solve for the global 7-DoF trajectory (pose + scale).  
+    3. **Multi-Map Robustness (Atlas):** To solve the "sharp turn" (AC-4) and "tracking loss" (AC-6) requirements, the V-SLAM front-end is based on an "Atlas" architecture.14 Tracking loss initiates a *new, independent map fragment*.13 The TOH is responsible for anchoring and merging *all* fragments geodetically 19 into a single, globally-consistent trajectory.
+
+    ### **2.2 Component Interaction and Data Flow**
+
+    * **Component 1: Pre-Flight Caching Module (PCM) (Offline)**  
+    * *Input:* User-defined Area of Interest (AOI) (e.g., a KML polygon).  
+    * *Action:* Queries Copernicus 6 and OpenStreetMap APIs. Downloads and builds a local geospatial database (GeoPackage/SpatiaLite) containing satellite tiles, DEM tiles, and road/river vectors for the AOI.  
+    * *Output:* A single, self-contained **Local Geo-Database file**.  
+    * **Component 2: Image Ingestion & Pre-processing (Real-time)**  
+    * *Input:* Image_N (up to 6.2K), Camera Intrinsics ($K$).  
+    * *Action:* Creates two copies:  
+        * **Image_N_LR** (Low-Resolution, e.g., 1536x1024): Dispatched *immediately* to the V-SLAM Front-End.  
+        * **Image_N_HR** (High-Resolution, 6.2K): Stored for asynchronous use by the GAB.  
+    * **Component 3: V-SLAM "Atlas" Front-End (High-Frequency Thread)**  
+    * *Input:* Image_N_LR.  
+    * *Action:* Tracks Image_N_LR against its *active map fragment*. Manages keyframes, local bundle adjustment 38, and the co-visibility graph. If tracking is lost (e.g., AC-4 sharp turn), it initializes a *new map fragment* 14 and continues tracking.  
+    * *Output:* **Relative_Unscaled_Pose** and **Local_Point_Cloud** data, sent to the TOH.  
+    * **Component 4: Local Geospatial Anchoring Back-End (GAB) (Low-Frequency, Asynchronous Thread)**  
+    * *Input:* A keyframe (Image_N_HR) and its *unscaled* pose, triggered by the TOH.  
+    * *Action:* Performs a visual-only, coarse-to-fine search 34 against the *Local Geo-Database*.  
+    * *Output:* An **Absolute_Metric_Anchor** (a high-confidence [Lat, Lon, Alt] pose) for that keyframe, sent to the TOH.  
+    * **Component 5: Trajectory Optimization Hub (TOH) (Central Hub Thread)**  
+    * *Input:* (1) High-frequency Relative_Unscaled_Pose stream. (2) Low-frequency Absolute_Metric_Anchor stream.  
+    * *Action:* Manages the complete flight trajectory as a **Sim(3) pose graph** 39 using Ceres Solver.19 Continuously fuses all data.  
+    * *Output 1 (Real-time):* **Pose_N_Est** (unscaled) sent to UI (meets AC-7, AC-8).  
+    * *Output 2 (Refined):* **Pose_N_Refined** (metric-scale, globally-optimized) sent to UI (meets AC-1, AC-2, AC-8).
+
+    ### **2.3 System Inputs**
+
+    1. **Image Sequence:** Consecutively named images (FullHD to 6252x4168).  
+    2. **Start Coordinate (Image 0):** A single, absolute GPS coordinate (Latitude, Longitude).  
+    3. **Camera Intrinsics ($K$):** Pre-calibrated camera intrinsic matrix.  
+    4. **Local Geo-Database File:** The single file generated by the Pre-Flight Caching Module (Section 3.0).
+
+    ### **2.4 Streaming Outputs (Meets AC-7, AC-8)**
+
+    1. **Initial Pose ($Pose_N^{Est}$):** An *unscaled* pose estimate. This is sent immediately (<5s, AC-7) to the UI for real-time visualization of the UAV's *path shape*.  
+    2. **Refined Pose ($Pose_N^{Refined}$) [Asynchronous]:** A globally-optimized, *metric-scale* 7-DoF pose (X, Y, Z, Qx, Qy, Qz, Qw) and its corresponding [Lat, Lon, Alt] coordinate. This is sent to the user whenever the TOH re-converges (e.g., after a new GAB anchor or map-merge), updating all past poses (AC-1, AC-2, AC-8 refinement met).
+
+    ## **3.0 Pre-Flight Component: The Geospatial Caching Module (PCM)**
+
+    This component is a new, mandatory, pre-flight utility that solves the fatal flaws (Section 1.1, 1.2) of the GEORTEX-R design. It eliminates all real-time network latency (AC-7) and all ToS violations (AC-5), ensuring the project is both performant and legally viable.
+
+    ### **3.1 Defining the Area of Interest (AOI)**
+
+    The system is designed for long-range flights. Given 3000 photos at 100m intervals, the maximum linear track is 300km. The user must provide a coarse "bounding box" or polygon (e.g., KML/GeoJSON format) of the intended flight area. The PCM will automatically add a generous buffer (e.g., 20km) to this AOI to account for navigational drift and ensure all necessary reference data is captured.
+
+    ### **3.2 Legal & Viable Data Sources (Copernicus & OpenStreetMap)**
+
+    As established in 1.1, the system *must* use open-data providers. The PCM is architected to use the following:
+
+    1. **Visual/Terrain Data (Primary):** The **Copernicus Data Space Ecosystem** 6 is the primary source. The PCM will use the Copernicus Processing and Catalogue APIs 6 to query, process, and download two key products for the buffered AOI:  
+    * **Sentinel-2 Satellite Imagery:** High-resolution (10m) visual tiles.  
+    * **Copernicus GLO-30 DEM:** A 30m-resolution Digital Elevation Model.7 This DEM is *not* used for high-accuracy object localization (see 1.4), but as a coarse altitude *prior* for the TOH and for the critical dynamic-warping step (Section 5.3).  
+    2. **Semantic Data (Secondary):** OpenStreetMap (OSM) data 40 for the AOI will be downloaded. This provides temporally-invariant vector data (roads, rivers, building footprints) which can be used as a secondary, optional verification layer for the GAB, especially in cases of extreme temporal divergence (e.g., new construction).42
+
+    ### **3.3 Building the Local Geo-Database**
+
+    The PCM utility will process all downloaded data into a single, efficient, compressed file. A modern GeoPackage or SpatiaLite database is the ideal format. This database will contain the satellite tiles, DEM tiles, and vector features, all indexed by a common spatial grid (e.g., UTM).
+
+    This single file is then loaded by the main ATLAS-GEOFUSE application at runtime. The GAB's (Section 5.0) "API calls" are thus transformed from high-latency, unreliable HTTP requests 9 into high-speed, zero-latency local SQL queries, guaranteeing that data I/O is never the bottleneck for meeting the AC-7 performance requirement.
+
+    ## **4.0 Core Component: The Multi-Map V-SLAM "Atlas" Front-End**
+
+    This component's sole task is to robustly and accurately compute the *unscaled* 6-DoF relative motion of the UAV and build a geometrically-consistent map of keyframes. It is explicitly designed to be more robust than simple frame-to-frame odometry and to handle catastrophic tracking loss (AC-4) gracefully.
+
+    ### **4.1 Rationale: ORB-SLAM3 "Atlas" Architecture**
+
+    The system will implement a V-SLAM front-end based on the "Atlas" multi-map paradigm, as seen in SOTA systems like ORB-SLAM3.14 This is the industry-standard solution for robust, long-term navigation in environments where tracking loss is possible.13
+
+    The mechanism is as follows:
+
+    1. The system initializes and begins tracking on **Map_Fragment_0**, using the known start GPS as a metadata tag.  
+    2. It tracks all new frames (Image_N_LR) against this active map.  
+    3. **If tracking is lost** (e.g., a sharp turn (AC-4) or a persistent 350m outlier (AC-3)):  
+    * The "Atlas" architecture does not fail. It declares Map_Fragment_0 "inactive," stores it, and *immediately initializes* **Map_Fragment_1** from the current frame.14  
+    * Tracking *resumes instantly* on this new map fragment, ensuring the system "correctly continues the work" (AC-4).
+
+    This architecture converts the "sharp turn" failure case into a *standard operating procedure*. The system never "fails"; it simply fragments. The burden of stitching these fragments together is correctly moved from the V-SLAM front-end (which has no global context) to the TOH (Section 6.0), which *can* solve it using global-metric anchors.
+
+    ### **4.2 Feature Matching Sub-System: SuperPoint + LightGlue**
+
+    The V-SLAM front-end's success depends entirely on high-quality feature matches, especially in the sparse, low-texture agricultural terrain seen in the user's images. The selected approach is **SuperPoint + LightGlue**.
+
+    * **SuperPoint:** A SOTA feature detector proven to find robust, repeatable keypoints in challenging, low-texture conditions.43  
+    * **LightGlue:** A highly optimized GNN-based matcher that is the successor to SuperGlue.44
+
+    The choice of LightGlue over SuperGlue is a deliberate performance optimization. LightGlue is *adaptive*.46 The user query states sharp turns (AC-4) are "rather an exception." This implies \~95% of image pairs are "easy" (high-overlap, straight flight) and 5% are "hard" (low-overlap, turns). LightGlue's adaptive-depth GNN exits early on "easy" pairs, returning a high-confidence match in a fraction of the time. This saves *enormous* computational budget on the 95% of normal frames, ensuring the system *always* meets the <5s budget (AC-7) and reserving that compute for the GAB and TOH. This component will run on **Image_N_LR** (low-res) to guarantee performance, and will be accelerated via TensorRT (Section 7.0).
+
+    ### **4.3 Keyframe Management and Local 3D Cloud**
+
+    The front-end will maintain a co-visibility graph of keyframes for its *active map fragment*. It will perform local Bundle Adjustment 38 continuously over a sliding window of recent keyframes to minimize drift *within* that fragment.
+
+    Crucially, it will triangulate features to create a **local, high-density 3D point cloud** for its map fragment.28 This point cloud is essential for two reasons:
+
+    1. It provides robust tracking (tracking against a 3D map, not just a 2D frame).  
+    2. It serves as the **high-accuracy source** for the object localization output (Section 9.1), as established in 1.4, allowing the system to bypass the high-error external DEM.
+
+    #### **Table 1: Analysis of State-of-the-Art Feature Matchers (For V-SLAM Front-End)**
+
+    | Approach (Tools/Library) | Advantages | Limitations | Requirements | Fitness for Problem Component |
+    | :---- | :---- | :---- | :---- | :---- |
+    | **SuperPoint + SuperGlue** | - SOTA robustness in low-texture, high-blur conditions. - GNN reasons about 3D scene context. - Proven in real-time SLAM systems. | - Computationally heavy (fixed-depth GNN). - Slower than LightGlue. | - NVIDIA GPU (RTX 2060+). - PyTorch or TensorRT. | **Good.** A solid, baseline choice. Meets robustness needs but will heavily tax the <5s time budget (AC-7). |
+    | **SuperPoint + LightGlue** 44 | - **Adaptive Depth:** Faster on "easy" pairs, more accurate on "hard" pairs.46 - **Faster & Lighter:** Outperforms SuperGlue on speed and accuracy. - SOTA "in practice" choice for large-scale matching. | - Newer, but rapidly being adopted and proven.48 | - NVIDIA GPU (RTX 2060+). - PyTorch or TensorRT. | **Excellent (Selected).** The adaptive nature is *perfect* for this problem. It saves compute on the 95% of easy (straight) frames, maximizing our ability to meet AC-7. |
+
+    ## **5.0 Core Component: The Local Geospatial Anchoring Back-End (GAB)**
+
+    This asynchronous component is the system's "anchor to reality." Its sole purpose is to find a high-confidence, *absolute-metric* pose for a given V-SLAM keyframe by matching it against the **local, pre-cached geo-database** (from Section 3.0). This component is a full replacement for the high-risk, high-latency GAB from the GEORTEX-R draft (see 1.2, 1.5).
+
+    ### **5.1 Rationale: Local-First Query vs. On-Demand API**
+
+    As established in 1.2, all queries are made to the local SSD. This guarantees zero-latency I/O, which is a hard requirement for a real-time system, as external network latency is unacceptably high and variable.9 The GAB itself runs asynchronously and can take longer than 5s (e.g., 10-15s), but it must not be *blocked* by network I/O, which would stall the entire processing pipeline.
+
+    ### **5.2 SOTA Visual-Only Coarse-to-Fine Localization**
+
+    This component implements a state-of-the-art, two-stage *visual-only* pipeline, which is lower-risk and more performant (see 1.5) than the GEORTEX-R's semantic-hybrid model. This approach is well-supported by SOTA research in aerial localization.34
+
+    1. **Stage 1 (Coarse): Global Descriptor Retrieval.**  
+    * *Action:* When the TOH requests an anchor for Keyframe_k, the GAB first computes a *global descriptor* (a compact vector representation) for the *nadir-warped* (see 5.3) low-resolution Image_k_LR.  
+    * *Technology:* A SOTA Visual Place Recognition (VPR) model like **SALAD** 49, **TransVLAD** 50, or **NetVLAD** 33 will be used. These are designed for this "image retrieval" task.45  
+    * *Result:* This descriptor is used to perform a fast FAISS/vector search against the descriptors of the *local satellite tiles* (which were pre-computed and stored in the Geo-Database). This returns the Top-K (e.g., K=5) most likely satellite tiles in milliseconds.  
+    2. **Stage 2 (Fine): Local Feature Matching.**  
+    * *Action:* The system runs **SuperPoint+LightGlue** 43 to find pixel-level correspondences.  
+    * *Performance:* This is *not* run on the *full* UAV image against the *full* satellite map. It is run *only* between high-resolution patches (from **Image_k_HR**) and the **Top-K satellite tiles** identified in Stage 1.  
+    * *Result:* This produces a set of 2D-2D (image-to-map) feature matches. A PnP/RANSAC solver then computes a high-confidence 6-DoF pose. This pose is the **Absolute_Metric_Anchor** that is sent to the TOH.
+
+    ### **5.3 Solving the Viewpoint Gap: Dynamic Feature Warping**
+
+    The GAB must solve the "viewpoint gap" 33: the UAV image is oblique (due to roll/pitch), while the satellite tiles are nadir (top-down).
+
+    The GEORTEX-R draft proposed a complex, high-risk deep learning solution. The ATLAS-GEOFUSE solution is far more elegant and requires zero R\&D:
+
+    1. The V-SLAM Front-End (Section 4.0) already *knows* the camera's *relative* 6-DoF pose, including its **roll and pitch** orientation relative to the *local map's ground plane*.  
+    2. The *Local Geo-Database* (Section 3.0) contains a 30m-resolution DEM for the AOI.  
+    3. When the GAB processes Keyframe_k, it *first* performs a **dynamic homography warp**. It projects the V-SLAM ground plane onto the coarse DEM, and then uses the known camera roll/pitch to calculate the perspective transform (homography) needed to *un-distort* the oblique UAV image into a synthetic *nadir-view*.
+
+    This *nadir-warped* UAV image is then used in the Coarse-to-Fine pipeline (5.2). It will now match the *nadir* satellite tiles with extremely high-fidelity. This method *eliminates* the viewpoint gap *without* training any new neural networks, leveraging the inherent synergy between the V-SLAM component and the GAB's pre-cached DEM.
+
+    ## **6.0 Core Component: The Multi-Map Trajectory Optimization Hub (TOH)**
+
+    This component is the system's central "brain." It runs continuously, fusing all measurements (high-frequency/unscaled V-SLAM, low-frequency/metric-scale GAB anchors) from *all map fragments* into a single, globally consistent trajectory.
+
+    ### **6.1 Incremental Sim(3) Pose-Graph Optimization**
+
+    The central challenge of monocular, IMU-denied SLAM is scale-drift. The V-SLAM front-end produces *unscaled* 6-DoF ($SE(3)$) relative poses.37 The GAB produces *metric-scale* 6-DoF ($SE(3)$) *absolute* poses. These cannot be directly combined.
+
+    The solution is that the graph *must* be optimized in **Sim(3) (7-DoF)**.39 This adds a *single global scale factor $s$* as an optimizable variable to each V-SLAM map fragment. The TOH will maintain a pose-graph using **Ceres Solver** 19, a SOTA optimization library.
+
+    The graph is constructed as follows:
+
+    1. **Nodes:** Each keyframe pose (7-DoF: $X, Y, Z, Qx, Qy, Qz, s$).  
+    2. **Edge 1 (V-SLAM):** A relative pose constraint between Keyframe_i and Keyframe_j *within the same map fragment*. The error is computed in Sim(3).29  
+    3. **Edge 2 (GAB):** An *absolute* pose constraint on Keyframe_k. This constraint *fixes* Keyframe_k's pose to the *metric* GPS coordinate from the GAB anchor and *fixes its scale $s$ to 1.0*.
+
+    The GAB's $s=1.0$ anchor creates "tension" in the graph. The Ceres optimizer 20 resolves this tension by finding the *one* global scale $s$ for all *other* V-SLAM nodes in that fragment that minimizes the total error. This effectively "stretches" or "shrinks" the entire unscaled V-SLAM fragment to fit the metric anchors, which is the core of monocular SLAM scale-drift correction.29
+
+    ### **6.2 Geodetic Map-Merging via Absolute Anchors**
+
+    This is the robust solution to the "sharp turn" (AC-4) problem, replacing the flawed "relocalization" model from the original draft.
+
+    * **Scenario:** The UAV makes a sharp turn (AC-4). The V-SLAM front-end *loses tracking* on Map_Fragment_0 and *creates* Map_Fragment_1 (per Section 4.1). The TOH's pose graph now contains *two disconnected components*.  
+    * **Mechanism (Geodetic Merging):**  
+    1. The GAB (Section 5.0) is *queued* to find anchors for keyframes in *both* fragments.  
+    2. The GAB returns Anchor_A for Keyframe_10 (in Map_Fragment_0) with GPS [Lat_A, Lon_A].  
+    3. The GAB returns Anchor_B for Keyframe_50 (in Map_Fragment_1) with GPS ``.  
+    4. The TOH adds *both* of these as absolute, metric constraints (Edge 2) to the global pose-graph.  
+    * The graph optimizer 20 now has all the information it needs. It will solve for the 7-DoF pose of *both fragments*, placing them in their correct, globally-consistent metric positions. The two fragments are *merged geodetically* (i.e., by their global coordinates) even if they *never* visually overlap. This is a vastly more robust and modern solution than simple visual loop closure.19
+
+    ### **6.3 Automatic Outlier Rejection (AC-3, AC-5)**
+
+    The system must be robust to 350m outliers (AC-3) and <10% bad GAB matches (AC-5). A standard least-squares optimizer (like Ceres 20) would be catastrophically corrupted by a 350m error.
+
+    This is a solved problem in modern graph optimization.19 The solution is to wrap *all* constraints (V-SLAM and GAB) in a **Robust Loss Function (e.g., HuberLoss, CauchyLoss)** within Ceres Solver.
+
+    A robust loss function mathematically *down-weights* the influence of constraints with large errors (high residuals). When the TOH "sees" the 350m error from a V-SLAM relative pose (AC-3) or a bad GAB anchor (AC-5), the robust loss function effectively acknowledges the measurement but *refuses* to pull the entire 3000-image trajectory to fit this one "insane" data point. It automatically and gracefully *ignores* the outlier, optimizing the 99.9% of "sane" measurements, thus meeting AC-3 and AC-5.
+
+    ### **Table 2: Analysis of Trajectory Optimization Strategies**
+
+    | Approach (Tools/Library) | Advantages | Limitations | Requirements | Fitness for Problem Component |
+    | :---- | :---- | :---- | :---- | :---- |
+    | **Incremental SLAM (Pose-Graph Optimization)** (Ceres Solver 19, g2o, GTSAM) | - **Real-time / Online:** Provides immediate pose estimates (AC-7). - **Supports Refinement:** Explicitly designed to refine past poses when new "loop closure" (GAB) data arrives (AC-8). - **Robust:** Can handle outliers via robust kernels.19 | - Initial estimate is *unscaled* until a GAB anchor arrives. - Can drift *if* not anchored. | - A graph optimization library (Ceres). - A robust cost function (Huber). | **Excellent (Selected).** This is the *only* architecture that satisfies all user requirements for real-time streaming (AC-7) and asynchronous refinement (AC-8). |
+    | **Batch Structure from Motion (Global Bundle Adjustment)** (COLMAP, Agisoft Metashape) | - **Globally Optimal Accuracy:** Produces the most accurate possible 3D reconstruction. | - **Offline:** Cannot run in real-time or stream results. - High computational cost (minutes to hours). - Fails AC-7 and AC-8 completely. | - All images must be available before processing starts. - High RAM and CPU. | **Good (as an *Optional* Post-Processing Step).** Unsuitable as the primary online system, but could be offered as an optional, high-accuracy "Finalize Trajectory" batch process. |
+
+    ## **7.0 High-Performance Compute & Deployment**
+
+    The system must run on an RTX 2060 (AC-7) while processing 6.2K images. These are opposing constraints that require a deliberate compute strategy to balance speed and accuracy.
+
+    ### **7.1 Multi-Scale, Coarse-to-Fine Processing Pipeline**
+
+    The system must balance the conflicting demands of real-time speed (AC-7) and high accuracy (AC-2). This is achieved by running different components at different resolutions.
+
+    * **V-SLAM Front-End (Real-time, <5s):** This component (Section 4.0) runs *only* on the **Image_N_LR** (e.g., 1536x1024) copy. This is fast enough to meet the AC-7 budget.46  
+    * **GAB (Asynchronous, High-Accuracy):** This component (Section 5.0) uses the full-resolution **Image_N_HR** *selectively* to meet the 20m accuracy (AC-2).  
+    1. Stage 1 (Coarse) runs on the low-res, nadir-warped image.  
+    2. Stage 2 (Fine) runs SuperPoint on the *full 6.2K* image to find the *most confident* keypoints. It then extracts small, 256x256 *patches* from the *full-resolution* image, centered on these keypoints.  
+    3. It matches *these small, full-resolution patches* against the high-res satellite tile.
+
+    This hybrid, multi-scale method provides the fine-grained matching accuracy of the 6.2K image (needed for AC-2) without the catastrophic CUDA Out-of-Memory errors (an RTX 2060 has only 6GB VRAM 30) or performance penalties that full-resolution processing would entail.
+
+    ### **7.2 Mandatory Deployment: NVIDIA TensorRT Acceleration**
+
+    The deep learning models (SuperPoint, LightGlue, NetVLAD) will be too slow in their native PyTorch framework to meet AC-7 on an RTX 2060.
+
+    This is not an "optional" optimization; it is a *mandatory* deployment step. The key neural networks *must* be converted from PyTorch into a highly-optimized **NVIDIA TensorRT engine**.
+
+    Research *specifically* on accelerating LightGlue with TensorRT shows **"2x-4x speed gains over compiled PyTorch"**.48 Other benchmarks confirm TensorRT provides 30-70% speedups for deep learning inference.52 This conversion (which applies layer fusion, graph optimization, and FP16/INT8 precision) is what makes achieving the <5s (AC-7) performance *possible* on the specified RTX 2060 hardware.
+
+    ## **8.0 System Robustness: Failure Mode Escalation Logic**
+
+    This logic defines the system's behavior during real-world failures, ensuring it meets criteria AC-3, AC-4, AC-6, and AC-9, and is built upon the new "Atlas" multi-map architecture.
+
+    ### **8.1 Stage 1: Normal Operation (Tracking)**
+
+    * **Condition:** V-SLAM front-end (Section 4.0) is healthy.  
+    * **Logic:**  
+    1. V-SLAM successfully tracks Image_N_LR against its *active map fragment*.  
+    2. A new **Relative_Unscaled_Pose** is sent to the TOH (Section 6.0).  
+    3. TOH sends **Pose_N_Est** (unscaled) to the user (AC-7, AC-8 met).  
+    4. If Image_N is selected as a keyframe, the GAB (Section 5.0) is *queued* to find an anchor for it, which will trigger a **Pose_N_Refined** update later.
+
+    ### **8.2 Stage 2: Transient VO Failure (Outlier Rejection)**
+
+    * **Condition:** Image_N is unusable (e.g., severe blur, sun-glare, or the 350m outlier from AC-3).  
+    * **Logic (Frame Skipping):**  
+    1. V-SLAM front-end fails to track Image_N_LR against the active map.  
+    2. The system *discards* Image_N (marking it as a rejected outlier, AC-5).  
+    3. When Image_N+1 arrives, the V-SLAM front-end attempts to track it against the *same* local keyframe map (from Image_N-1).  
+    4. **If successful:** Tracking resumes. Image_N is officially an outlier. The system "correctly continues the work" (AC-3 met).  
+    5. **If fails:** The system repeats for Image_N+2, N+3. If this fails for \~5 consecutive frames, it escalates to Stage 3.
+
+    ### **8.3 Stage 3: Persistent VO Failure (New Map Initialization)**
+
+    * **Condition:** Tracking is lost for multiple frames. This is the **"sharp turn" (AC-4)** or "low overlap" (AC-4) scenario.  
+    * **Logic (Atlas Multi-Map):**  
+    1. The V-SLAM front-end (Section 4.0) declares "Tracking Lost."  
+    2. It marks the current Map_Fragment_k as "inactive".13  
+    3. It *immediately* initializes a **new** Map_Fragment_k+1 using the current frame (Image_N+5).  
+    4. **Tracking resumes instantly** on this new, unscaled, un-anchored map fragment.  
+    5. This "registering" of a new map ensures the system "correctly continues the work" (AC-4 met) and maintains the >95% registration rate (AC-9) by not counting this as a failure.
+
+    ### **8.4 Stage 4: Map-Merging & Global Relocalization (GAB-Assisted)**
+
+    * **Condition:** The system is now tracking on Map_Fragment_k+1, while Map_Fragment_k is inactive. The TOH pose-graph (Section 6.0) is disconnected.  
+    * **Logic (Geodetic Merging):**  
+    1. The TOH queues the GAB (Section 5.0) to find anchors for *both* map fragments.  
+    2. The GAB finds anchors for keyframes in *both* fragments.  
+    3. The TOH (Section 6.2) receives these metric anchors, adds them to the graph, and the Ceres optimizer 20 *finds the global 7-DoF pose for both fragments*, merging them into a single, metrically-consistent trajectory.
+
+    ### **8.5 Stage 5: Catastrophic Failure (User Intervention)**
+
+    * **Condition:** The system is in Stage 3 (Lost), *and* the GAB (Section 5.0) has *also* failed to find *any* global anchors for a new Map_Fragment_k+1 for a prolonged period (e.g., 20% of the route). This is the "absolutely incapable" scenario (AC-6), (e.g., flying over a large, featureless body of water or dense, uniform fog).  
+    * **Logic:**  
+    1. The system has an *unscaled, un-anchored* map fragment (Map_Fragment_k+1) and *zero* idea where it is in the world.  
+    2. The TOH triggers the AC-6 flag.  
+    * **Resolution (User-Aided Prior):**  
+    1. The UI prompts the user: "Tracking lost. Please provide a coarse location for the *current* image."  
+    2. The user clicks *one point* on a map.  
+    3. This [Lat, Lon] is *not* taken as ground truth. It is fed to the **GAB (Section 5.0)** as a *strong spatial prior* for its *local database query* (Section 5.2).  
+    4. This narrows the GAB's Stage 1 search area from "the entire AOI" to "a 5km radius around the user's click." This *guarantees* the GAB will find the correct satellite tile, find a high-confidence **Absolute_Metric_Anchor**, and allow the TOH (Stage 4) to re-scale 29 and geodetically-merge 20 this lost fragment, re-localizing the entire trajectory.
+
+    ## **9.0 High-Accuracy Output Generation and Validation Strategy**
+
+    This section details how the final user-facing outputs are generated, specifically replacing the flawed "Ray-DEM" method (see 1.4) with a high-accuracy "Ray-Cloud" method to meet the 20m accuracy (AC-2).
+
+    ### **9.1 High-Accuracy Object Geolocalization via Ray-Cloud Intersection**
+
+    As established in 1.4, using an external 30m DEM 21 for object localization introduces uncontrollable errors (up to 4m+22) that make meeting the 20m (AC-2) accuracy goal impossible. The system *must* use its *own*, internally-generated 3D map, which is locally far more accurate.25
+
+    * **Inputs:**  
+    1. User clicks pixel coordinate $(u,v)$ on Image_N.  
+    2. The system retrieves the **final, refined, metric 7-DoF Sim(3) pose** $P_{sim(3)} = (s, R, T)$ for the *map fragment* that Image_N belongs to. This transform $P_{sim(3)}$ maps the *local V-SLAM coordinate system* to the *global metric coordinate system*.  
+    3. The system retrieves the *local, unscaled* **V-SLAM 3D point cloud** ($P_{local_cloud}$) generated by the Front-End (Section 4.3).  
+    4. The known camera intrinsic matrix $K$.  
+    * **Algorithm (Ray-Cloud Intersection):**  
+    1. **Un-project Pixel:** The 2D pixel $(u,v)$ is un-projected into a 3D ray *direction* vector $d_{cam}$ in the camera's local coordinate system: $d_{cam} = K^{-1} \\cdot [u, v, 1]^T$.  
+    2. **Transform Ray (Local):** This ray is transformed using the *local V-SLAM pose* of Image_N to get a ray in the *local map fragment's* coordinate system.  
+    3. **Intersect (Local):** The system performs a numerical *ray-mesh intersection* (or nearest-neighbor search) to find the 3D point $P_{local}$ where this local ray *intersects the local V-SLAM point cloud* ($P_{local_cloud}$).25 This $P_{local}$ is *highly accurate* relative to the V-SLAM map.26  
+    4. **Transform (Global):** This local 3D point $P_{local}$ is now transformed to the global, metric coordinate system using the 7-DoF Sim(3) transform from the TOH: $P_{metric} = s \\cdot (R \\cdot P_{local}) + T$.  
+    5. **Result:** This 3D intersection point $P_{metric}$ is the *metric* world coordinate of the object.  
+    6. **Convert:** This $(X, Y, Z)$ world coordinate is converted to a [Latitude, Longitude, Altitude] GPS coordinate.55
+
+    This method correctly isolates the error. The object's accuracy is now *only* dependent on the V-SLAM's geometric fidelity (AC-10 MRE < 1.0px) and the GAB's global anchoring (AC-1, AC-2). It *completely eliminates* the external 30m DEM error 22 from this critical, high-accuracy calculation.
+
+    ### **9.2 Rigorous Validation Methodology**
+
+    A comprehensive test plan is required to validate compliance with all 10 Acceptance Criteria. The foundation is a **Ground-Truth Test Harness** (e.g., using the provided coordinates.csv data).
+
+    * **Test Harness:**  
+    1. **Ground-Truth Data:** coordinates.csv provides ground-truth [Lat, Lon] for a set of images.  
+    2. **Test Datasets:**  
+        * Test_Baseline: The ground-truth images and coordinates.  
+        * Test_Outlier_350m (AC-3): Test_Baseline with a single, unrelated image inserted.  
+        * Test_Sharp_Turn_5pct (AC-4): A sequence where several frames are manually deleted to simulate <5% overlap.  
+        * Test_Long_Route (AC-9): A 1500-image sequence.  
+    * **Test Cases:**  
+    * **Test_Accuracy (AC-1, AC-2, AC-5, AC-9):**  
+        * **Run:** Execute ATLAS-GEOFUSE on Test_Baseline, providing the first image's coordinate as the Start Coordinate.  
+        * **Script:** A validation script will compute the Haversine distance error between the *system's refined GPS output* ($Pose_N^{Refined}$) for each image and the *ground-truth GPS*.  
+        * **ASSERT** (count(errors < 50m) / total_images) >= 0.80 **(AC-1 Met)**  
+        * **ASSERT** (count(errors < 20m) / total_images) >= 0.60 **(AC-2 Met)**  
+        * **ASSERT** (count(un-localized_images) / total_images) < 0.10 **(AC-5 Met)**  
+        * **ASSERT** (count(localized_images) / total_images) > 0.95 **(AC-9 Met)**  
+    * **Test_MRE (AC-10):**  
+        * **Run:** After Test_Baseline completes.  
+        * **ASSERT** TOH.final_Mean_Reprojection_Error < 1.0 **(AC-10 Met)**  
+    * **Test_Performance (AC-7, AC-8):**  
+        * **Run:** Execute on Test_Long_Route on the minimum-spec RTX 2060.  
+        * **Log:** Log timestamps for "Image In" -> "Initial Pose Out" ($Pose_N^{Est}$).  
+        * **ASSERT** average_time < 5.0s **(AC-7 Met)**  
+        * **Log:** Log the output stream.  
+        * **ASSERT** >80% of images receive *two* poses: an "Initial" and a "Refined" **(AC-8 Met)**  
+    * **Test_Robustness (AC-3, AC-4, AC-6):**  
+        * **Run:** Execute Test_Outlier_350m.  
+        * **ASSERT** System logs "Stage 2: Discarding Outlier" or "Stage 3: New Map" *and* the final trajectory error for the *next* frame is < 50m **(AC-3 Met)**.  
+        * **Run:** Execute Test_Sharp_Turn_5pct.  
+        * **ASSERT** System logs "Stage 3: New Map Initialization" and "Stage 4: Geodetic Map-Merge," and the final trajectory is complete and accurate **(AC-4 Met)**.  
+        * **Run:** Execute on a sequence with no GAB anchors possible for 20% of the route.  
+        * **ASSERT** System logs "Stage 5: User Intervention Requested" **(AC-6 Met)**.
+
+
+Identify all potential weak points and problems. Address them and find out ways to solve them. Based on your findings, form a new solution draft in the same format.
+
+If your finding requires a complete reorganization of the flow and different components, state it.
+Put all the findings regarding what was weak and poor at the beginning of the report.
+At the very beginning of the report list most profound changes you've made to previous solution.
+
+Then form a new solution design without referencing the previous system. Remove Poor and Very Poor component choices from the component analysis tables, but leave Good and Excellent ones.
+In the updated report, do not put "new" marks, do not compare to the previous solution draft, just make a new solution as if from scratch

docs/00_problem/1.4_05.1_assesment_prompt.md

@@ -0,0 +1,379 @@
+Read carefully about the problem:
+
+ We have a lot of images taken from a wing-type UAV using a camera with at least Full HD resolution. Resolution of each photo could be up to 6200*4100 for the whole flight, but for other flights, it could be FullHD
+Photos are taken and named consecutively within 100 meters of each other.
+We know only the starting GPS coordinates. We need to determine the GPS of the centers of each image. And also the coordinates of the center of any object in these photos. We can use an external satellite provider for ground checks on the existing photos
+ 
+ System has next restrictions and conditions:
+  - Photos are taken by only airplane type UAVs.
+  - Photos are taken by the camera pointing downwards and fixed, but it is not autostabilized.
+  - The flying range is restricted by the eastern and southern parts of Ukraine (To the left of the Dnipro River)
+  - The image resolution could be from FullHD to 6252*4168. Camera parameters are known: focal length, sensor width, resolution and so on.
+  - Altitude is predefined and no more than 1km. The height of the terrain can be neglected.
+  - There is NO data from IMU
+  - Flights are done mostly in sunny weather
+  - We can use satellite providers, but we're limited right now to Google Maps, which could be outdated for some regions
+  - Number of photos could be up to 3000, usually in the 500-1500 range
+  - During the flight, UAVs can make sharp turns, so that the next photo may be absolutely different from the previous one (no same objects), but it is rather an exception than the rule
+  - Processing is done on a stationary computer or laptop with NVidia GPU at least RTX2060, better 3070. (For the UAV solution Jetson Orin Nano would be used, but that is out of scope.)
+  
+ Output of the system should address next acceptance criteria:
+    - The system should find out the GPS of centers of 80% of the photos from the flight within an error of no more than 50 meters in comparison to the real GPS
+
+    - The system should find out the GPS of centers of 60% of the photos from the flight within an error of no more than 20 meters in comparison to the real GPS
+
+    - The system should correctly continue the work even in the presence of up to 350 meters of an outlier photo between 2 consecutive pictures en route. This could happen due to tilt of the plane.
+
+    - System should correctly continue the work even during sharp turns, where the next photo doesn't overlap at all, or overlaps in less than 5%. The next photo should be in less than 200m drift and at an angle of less than 70%
+
+    - System should try to operate when UAV made a sharp turn, and all the next photos has no common points with previous route. In that situation system should try to figure out location of the new piece of the route and connect it to the previous route. Also this separate chunks could be more than 2, so this strategy should be in the core of the system
+
+    - In case of being absolutely incapable of determining the system to determine next, second next, and third next images GPS, by any means (these 20% of the route), then it should ask the user for input for the next image, so that the user can specify the location
+
+    - Less than 5 seconds for processing one image
+
+    - Results of image processing should appear immediately to user, so that user shouldn't wait for the whole route to complete in order to analyze first results. Also, system could refine existing calculated results and send refined results again to user 
+
+    - Image Registration Rate > 95%. The system can find enough matching features to confidently calculate the camera's 6-DoF pose (position and orientation) and "stitch" that image into the final trajectory
+
+    - Mean Reprojection Error (MRE) < 1.0 pixels. The distance, in pixels, between the original pixel location of the object and the re-projected pixel location.
+
+    - The whole system should work as a background service. The interaction should be done by zeromq. Sevice should be up and running and awaiting for the initial input message. On the input message processing should started, and immediately after the first results system should provide them to the client
+
+ Here is a solution draft:
+
+
+    # **ASTRAL System Architecture: A High-Fidelity Geopositioning Framework for IMU-Denied Aerial Operations**
+
+    ## **2.0 The ASTRAL (Advanced Scale-Aware Trajectory-Refinement and Localization) System Architecture**
+
+    The ASTRAL architecture is a multi-map, decoupled, loosely-coupled system designed to solve the flaws identified in Section 1.0 and meet all 10 Acceptance Criteria.
+
+    ### **2.1 Core Principles**
+
+    The ASTRAL architecture is built on three principles:
+
+    1. **Tiered Geospatial Database:** The system *cannot* rely on a single data source. It is architected around a *tiered* local database.  
+    * **Tier-1 (Baseline):** Google Maps data. This is used to meet the 50m (AC-1) requirement and provide geolocalization.  
+    * **Tier-2 (High-Accuracy):** A framework for ingesting *commercial, sub-meter* data (visual 4; and DEM 5). This tier is *required* to meet the 20m (AC-2) accuracy. The system will *run* on Tier-1 but *achieve* AC-2 when "fueled" with Tier-2 data.  
+    2. **Viewpoint-Invariant Anchoring:** The system *rejects* geometric warping. The GAB (Section 5.0) is built on SOTA Visual Place Recognition (VPR) models that are *inherently* invariant to the oblique-to-nadir viewpoint change, decoupling it from the V-SLAM's unstable orientation.  
+    3. **Continuously-Scaled Trajectory:** The system *rejects* the "single-scale-per-fragment" model. The TOH (Section 6.0) is a Sim(3) pose-graph optimizer 11 that models scale as a *per-keyframe optimizable parameter*.15 This allows the trajectory to "stretch" and "shrink" elastically to absorb continuous monocular scale drift.12
+
+    ### **2.2 Component Interaction and Data Flow**
+
+    The system is multi-threaded and asynchronous, designed for real-time streaming (AC-7) and refinement (AC-8).
+
+    * **Component 1: Tiered GDB (Pre-Flight):**  
+    * *Input:* User-defined Area of Interest (AOI).  
+    * *Action:* Downloads and builds a local SpatiaLite/GeoPackage.  
+    * *Output:* A single **Local-Geo-Database file** containing:  
+        * Tier-1 (Google Maps) + GLO-30 DSM  
+        * Tier-2 (Commercial) satellite tiles + WorldDEM DTM elevation tiles.  
+        * A *pre-computed FAISS vector index* of global descriptors (e.g., SALAD 8) for *all* satellite tiles (see 3.4).  
+    * **Component 2: Image Ingestion (Real-time):**  
+    * *Input:* Image_N (up to 6.2K), Camera Intrinsics ($K$).  
+    * *Action:* Creates Image_N_LR (Low-Res, e.g., 1536x1024) and Image_N_HR (High-Res, 6.2K).  
+    * *Dispatch:* Image_N_LR -> V-SLAM. Image_N_HR -> GAB (for patches).  
+    * **Component 3: "Atlas" V-SLAM Front-End (High-Frequency Thread):**  
+    * *Input:* Image_N_LR.  
+    * *Action:* Tracks Image_N_LR against the *active map fragment*. Manages keyframes and local BA. If tracking lost (AC-4, AC-6), it *initializes a new map fragment*.  
+    * *Output:* Relative_Unscaled_Pose, Local_Point_Cloud, and Map_Fragment_ID -> TOH.  
+    * **Component 4: VPR Geospatial Anchoring Back-End (GAB) (Low-Frequency, Asynchronous Thread):**  
+    * *Input:* A keyframe (Image_N_LR, Image_N_HR) and its Map_Fragment_ID.  
+    * *Action:* Performs SOTA two-stage VPR (Section 5.0) against the **Local-Geo-Database file**.  
+    * *Output:* Absolute_Metric_Anchor ([Lat, Lon, Alt] pose) and its Map_Fragment_ID -> TOH.  
+    * **Component 5: Scale-Aware Trajectory Optimization Hub (TOH) (Central Hub Thread):**  
+    * *Input 1:* High-frequency Relative_Unscaled_Pose stream.  
+    * *Input 2:* Low-frequency Absolute_Metric_Anchor stream.  
+    * *Action:* Manages the *global Sim(3) pose-graph* 13 with *per-keyframe scale*.15  
+    * *Output 1 (Real-time):* Pose_N_Est (unscaled) -> UI (Meets AC-7).  
+    * *Output 2 (Refined):* Pose_N_Refined (metric-scale) -> UI (Meets AC-1, AC-2, AC-8).
+
+    ### **2.3 System Inputs**
+
+    1. **Image Sequence:** Consecutively named images (FullHD to 6252x4168).  
+    2. **Start Coordinate (Image 0):** A single, absolute GPS coordinate [Lat, Lon].  
+    3. **Camera Intrinsics (K):** Pre-calibrated camera intrinsic matrix.  
+    4. **Local-Geo-Database File:** The single file generated by Component 1.
+
+    ### **2.4 Streaming Outputs (Meets AC-7, AC-8)**
+
+    1. **Initial Pose (Pose_N^{Est}):** An *unscaled* pose. This is the raw output from the V-SLAM Front-End, transformed by the *current best estimate* of the trajectory. It is sent immediately (<5s, AC-7) to the UI for real-time visualization of the UAV's *path shape*.  
+    2. **Refined Pose (Pose_N^{Refined}) [Asynchronous]:** A globally-optimized, *metric-scale* 7-DoF pose. This is sent to the user *whenever the TOH re-converges* (e.g., after a new GAB anchor or a map-merge). This *re-writes* the history of poses (e.g., Pose_{N-100} to Pose_N), meeting the refinement (AC-8) and accuracy (AC-1, AC-2) requirements.
+
+    ## **3.0 Component 1: The Tiered Pre-Flight Geospatial Database (GDB)**
+
+    This component is the implementation of the "Tiered Geospatial" principle. It is a mandatory pre-flight utility that solves both the *legal* problem (Flaw 1.4) and the *accuracy* problem (Flaw 1.1).
+
+    ### **3.2 Tier-1 (Baseline): Google Maps and GLO-30 DEM**
+
+    This tier provides the baseline capability and satisfies AC-1.
+
+    * **Visual Data:** Google Maps (coarse Maxar)  
+    * *Resolution:* 10m.  
+    * *Geodetic Accuracy:* \~1 m to 20m  
+    * *Purpose:* Meets AC-1 (80% < 50m error). Provides a robust baseline for coarse geolocalization.  
+    * **Elevation Data:** Copernicus GLO-30 DEM  
+    * *Resolution:* 30m.  
+    * *Type:* DSM (Digital Surface Model).2 This is a *weakness*, as it includes buildings/trees.  
+    * *Purpose:* Provides a coarse altitude prior for the TOH and the initial GAB search.
+
+    ### **3.3 Tier-2 (High-Accuracy): Ingestion Framework for Commercial Data**
+
+    This is the *procurement and integration framework* required to meet AC-2.
+
+    * **Visual Data:** Commercial providers, e.g., Maxar (30-50cm) or Satellogic (70cm)  
+    * *Resolution:* < 1m.  
+    * *Geodetic Accuracy:* Typically < 5m.  
+    * *Purpose:* Provides the high-resolution, high-accuracy reference needed for the GAB to achieve a sub-20m total error.  
+    * **Elevation Data:** Commercial providers, e.g., WorldDEM Neo 5 or Elevation10.32  
+    * *Resolution:* 5m-12m.  
+    * *Vertical Accuracy:* < 4m.32  
+    * *Type:* DTM (Digital Terrain Model).32
+
+    The use of a DTM (bare-earth) in Tier-2 is a critical advantage over the Tier-1 DSM (surface). The V-SLAM Front-End (Section 4.0) will triangulate a 3D point cloud of what it *sees*, which is the *ground* in fields or *tree-tops* in forests. The Tier-1 GLO-30 DSM 2 represents the *top* of the canopy/buildings. If the V-SLAM maps the *ground* (e.g., altitude 100m) and the GAB tries to anchor it to a DSM *prior* that shows a forest (e.g., altitude 120m), the 20m altitude discrepancy will introduce significant error into the TOH. The Tier-2 DTM (bare-earth) 5 provides a *vastly* superior altitude anchor, as it represents the same ground plane the V-SLAM is tracking, significantly improving the entire 7-DoF pose solution.
+
+    ### **3.4 Local Database Generation: Pre-computing Global Descriptors**
+
+    This is the key performance optimization for the GAB. During the pre-flight caching step, the GDB utility does not just *store* tiles; it *processes* them.
+
+    For *every* satellite tile (e.g., 256x256m) in the AOI, the utility will load the tile into the VPR model (e.g., SALAD 8), compute its global descriptor (a compact feature vector), and store this vector in a high-speed vector index (e.g., FAISS).
+
+    This step moves 99% of the GAB's "Stage 1" (Coarse Retrieval) workload into an offline, pre-flight step. The *real-time* GAB query (Section 5.2) is now reduced to: (1) Compute *one* vector for the UAV image, and (2) Perform a very fast K-Nearest-Neighbor search on the pre-computed FAISS index. This is what makes a SOTA deep-learning GAB 6 fast enough to support the real-time refinement loop.
+
+    #### **Table 1: Geospatial Reference Data Analysis (Decision Matrix)**
+
+    | Data Product | Type | Resolution | Geodetic Accuracy (Horiz.) | Type | Cost | AC-2 (20m) Compliant? |
+    | :---- | :---- | :---- | :---- | :---- | :---- | :---- |
+    | Google Maps | Visual | 1m | 1m - 10m | N/A | Free | **Depending on the location** |
+    | Copernicus GLO-30 | Elevation | 30m | \~10-30m | **DSM** (Surface) | Free | **No (Fails Error Budget)** |
+    | **Tier-2: Maxar/Satellogic** | Visual | 0.3m - 0.7m | < 5 m (Est.) | N/A | Commercial | **Yes** |
+    | **Tier-2: WorldDEM Neo**  | Elevation | 5m | < 4m | **DTM** (Bare-Earth) | Commercial | **Yes** |
+
+    ## **4.0 Component 2: The "Atlas" Relative Motion Front-End**
+
+    This component's sole task is to robustly compute *unscaled* 6-DoF relative motion and handle tracking failures (AC-3, AC-4).
+
+    ### **4.1 Feature Matching Sub-System: SuperPoint + LightGlue**
+
+    The system will use **SuperPoint** for feature detection and **LightGlue** for matching. This choice is driven by the project's specific constraints:
+
+    * **Rationale (Robustness):** The UAV flies over "eastern and southern parts of Ukraine," which includes large, low-texture agricultural areas. SuperPoint is a SOTA deep-learning detector renowned for its robustness and repeatability in these challenging, low-texture environments.  
+    * **Rationale (Performance):** The RTX 2060 (AC-7) is a *hard* constraint with only 6GB VRAM.34 Performance is paramount. LightGlue is an SOTA matcher that provides a 4-10x speedup over its predecessor, SuperGlue. Its "adaptive" nature is a key optimization: it exits early on "easy" pairs (high-overlap, straight-flight) and spends more compute only on "hard" pairs (turns). This saves critical GPU budget on 95% of normal frames, ensuring the <5s (AC-7) budget is met.
+
+    This subsystem will run on the Image_N_LR (low-res) copy to guarantee it fits in VRAM and meets the real-time budget.
+
+    #### **Table 2: Analysis of State-of-the-Art Feature Matchers (V-SLAM Front-End)**
+
+    | Approach (Tools/Library) | Robustness (Low-Texture) | Speed (RTX 2060) | Fitness for Problem |
+    | :---- | :---- | :---- | :---- |
+    | ORB 33 (e.g., ORB-SLAM3) | Poor. Fails on low-texture. | Excellent (CPU/GPU) | **Good.** Fails robustness in target environment. |
+    | SuperPoint + SuperGlue | Excellent. | Good, but heavy. Fixed-depth GNN. 4-10x Slower than LightGlue.35 | **Good.** Robust, but risks AC-7 budget. |
+    | **SuperPoint + LightGlue** 35 | Excellent. | **Excellent.** Adaptive depth 35 saves budget. 4-10x faster. | **Excellent (Selected).** Balances robustness and performance. |
+
+    ### **4.2 The "Atlas" Multi-Map Paradigm (Solution for AC-3, AC-4, AC-6)**
+
+    This architecture is the industry-standard solution for IMU-denied, long-term SLAM and is critical for robustness.
+
+    * **Mechanism (AC-4, Sharp Turn):**  
+    1. The system is tracking on $Map_Fragment_0$.  
+    2. The UAV makes a sharp turn (AC-4, <5% overlap). The V-SLAM *loses tracking*.  
+    3. Instead of failing, the Atlas architecture *initializes a new map*: $Map_Fragment_1$.  
+    4. Tracking *resumes instantly* on this new, unanchored map.  
+    * **Mechanism (AC-3, 350m Outlier):**  
+    1. The system is tracking. A 350m outlier $Image_N$ arrives.  
+    2. The V-SLAM fails to match $Image_N$ (a "Transient VO Failure," see 7.3). It is *discarded*.  
+    3. $Image_N+1$ arrives (back on track). V-SLAM re-acquires its location on $Map_Fragment_0$.  
+    4. The system "correctly continues the work" (AC-3) by simply rejecting the outlier.
+
+    This design turns "catastrophic failure" (AC-3, AC-4) into a *standard operating procedure*. The "problem" of stitching the fragments ($Map_0$, $Map_1$) together is moved from the V-SLAM (which has no global context) to the TOH (which *can* solve it using GAB anchors, see 6.4).
+
+    ### **4.3 Local Bundle Adjustment and High-Fidelity 3D Cloud**
+
+    The V-SLAM front-end will continuously run Local Bundle Adjustment (BA) over a sliding window of recent keyframes to minimize drift *within* that fragment. It will also triangulate a sparse, but high-fidelity, 3D point cloud for its *local map fragment*.
+
+    This 3D cloud serves a critical dual function:
+
+    1. It provides a robust 3D map for frame-to-map tracking, which is more stable than frame-to-frame odometry.  
+    2. It serves as the **high-accuracy data source** for the object localization output (Section 7.2). This is the key to decoupling object-pointing accuracy from external DEM accuracy 19, a critical flaw in simpler designs.
+
+    ## **5.0 Component 3: The Viewpoint-Invariant Geospatial Anchoring Back-End (GAB)**
+
+    This component *replaces* the draft's "Dynamic Warping" (Section 5.0) and implements the "Viewpoint-Invariant Anchoring" principle (Section 2.1).
+
+    ### **5.1 Rationale: Viewpoint-Invariant VPR vs. Geometric Warping (Solves Flaw 1.2)**
+
+    As established in 1.2, geometrically warping the image using the V-SLAM's *drifty* roll/pitch estimate creates a *brittle*, high-risk failure spiral. The ASTRAL GAB *decouples* from the V-SLAM's orientation. It uses a SOTA VPR pipeline that *learns* to match oblique UAV images to nadir satellite images *directly*, at the feature level.6
+
+    ### **5.2 Stage 1 (Coarse Retrieval): SOTA Global Descriptors**
+
+    When triggered by the TOH, the GAB takes Image_N_LR. It computes a *global descriptor* (a single feature vector) using a SOTA VPR model like **SALAD** 6 or **MixVPR**.7
+
+    This choice is driven by two factors:
+
+    1. **Viewpoint Invariance:** These models are SOTA for this exact task.  
+    2. **Inference Speed:** They are extremely fast. SALAD reports < 3ms per image inference 8, and MixVPR is also noted for "fastest inference speed".37 This low overhead is essential for the AC-7 (<5s) budget.
+
+    This vector is used to query the *pre-computed FAISS vector index* (from 3.4), which returns the Top-K (e.g., K=5) most likely satellite tiles from the *entire AOI* in milliseconds.
+
+    #### **Table 3: Analysis of VPR Global Descriptors (GAB Back-End)**
+
+    | Model (Backbone) | Key Feature | Viewpoint Invariance | Inference Speed (ms) | Fitness for GAB |
+    | :---- | :---- | :---- | :---- | :---- |
+    | NetVLAD 7 (CNN) | Baseline | Poor. Not designed for oblique-to-nadir. | Moderate (\~20-50ms) | **Poor.** Fails robustness. |
+    | **SALAD** 8 (DINOv2) | Foundation Model.6 | **Excellent.** Designed for this. | **< 3ms**.8 Extremely fast. | **Excellent (Selected).** |
+    | **MixVPR** 36 (ResNet) | All-MLP aggregator.36 | **Very Good.**.7 | **Very Fast.**.37 | **Excellent (Selected).** |
+
+    ### **5.3 Stage 2 (Fine): Local Feature Matching and Pose Refinement**
+
+    The system runs **SuperPoint+LightGlue** 35 to find pixel-level matches, but *only* between the UAV image and the **Top-K satellite tiles** identified in Stage 1.
+
+    A **Multi-Resolution Strategy** is employed to solve the VRAM bottleneck.
+
+    1. Stage 1 (Coarse) runs on the Image_N_LR.  
+    2. Stage 2 (Fine) runs SuperPoint *selectively* on the Image_N_HR (6.2K) to get high-accuracy keypoints.  
+    3. It then matches small, full-resolution *patches* from the full-res image, *not* the full image.
+
+    This hybrid approach is the *only* way to meet both AC-7 (speed) and AC-2 (accuracy). The 6.2K image *cannot* be processed in <5s on an RTX 2060 (6GB VRAM 34). But its high-resolution *pixels* are needed for the 20m *accuracy*. Using full-res *patches* provides the pixel-level accuracy without the VRAM/compute cost.
+
+    A PnP/RANSAC solver then computes a high-confidence 6-DoF pose. This pose, converted to [Lat, Lon, Alt], is the **$Absolute_Metric_Anchor$** sent to the TOH.
+
+    ## **6.0 Component 4: The Scale-Aware Trajectory Optimization Hub (TOH)**
+
+    This component is the system's "brain" and implements the "Continuously-Scaled Trajectory" principle (Section 2.1). It *replaces* the draft's flawed "Single Scale" optimizer.
+
+    ### **6.1 The $Sim(3)$ Pose-Graph as the Optimization Backbone**
+
+    The central challenge of IMU-denied monocular SLAM is *scale drift*.11 The V-SLAM (Component 3) produces 6-DoF poses, but they are *unscaled* ($SE(3)$). The GAB (Component 4) produces *metric* 6-DoF poses ($SE(3)$).
+
+    The solution is to optimize the *entire graph* in the 7-DoF "Similarity" group, **$Sim(3)$**.11 This adds a 7th degree of freedom (scale, $s$) to the poses. The optimization backbone will be **Ceres Solver** 14, a SOTA C++ library for large, complex non-linear least-squares problems.
+
+    ### **6.2 Advanced Scale-Drift Correction: Modeling Scale as a Per-Keyframe Parameter (Solves Flaw 1.3)**
+
+    This is the *core* of the ASTRAL optimizer, solving Flaw 1.3. The draft's flawed model ($Pose_Graph(Fragment_i) = \\{Pose_1...Pose_n, s_i\\}$) is replaced by ASTRAL's correct model: $Pose_Graph = \\{ (Pose_1, s_1), (Pose_2, s_2),..., (Pose_N, s_N) \\}$.
+
+    The graph is constructed as follows:
+
+    * **Nodes:** Each keyframe pose is a 7-DoF $Sim(3)$ variable $\\{s_k, R_k, t_k\\}$.  
+    * **Edge 1 (V-SLAM):** A *relative* $Sim(3)$ constraint between $Pose_k$ and $Pose_{k+1}$ from the V-SLAM Front-End.  
+    * **Edge 2 (GAB):** An *absolute* $SE(3)$ constraint on $Pose_j$ from a GAB anchor. This constraint *fixes* the 6-DoF pose $(R_j, t_j)$ to the metric GAB value and *fixes its scale* $s_j = 1.0$.
+
+    This "per-keyframe scale" model 15 enables "elastic" trajectory refinement. When the graph is a long, unscaled "chain" of V-SLAM constraints, a GAB anchor (Edge 2) arrives at $Pose_{100}$, "nailing" it to the metric map and setting $s_{100} = 1.0$. As the V-SLAM continues, scale drifts. When a second anchor arrives at $Pose_{200}$ (setting $s_{200} = 1.0$), the Ceres optimizer 14 has a problem: the V-SLAM data *between* them has drifted.
+
+    The ASTRAL model *allows* the optimizer to solve for all intermediate scales (s_{101}, s_{102},..., s_{199}) as variables. The optimizer will find a *smooth, continuous* scale correction 15 that "elastically" stretches/shrinks the 100-frame sub-segment to *perfectly* fit both metric anchors. This *correctly* models the physics of scale drift 12 and is the *only* way to achieve the 20m accuracy (AC-2) and 1.0px MRE (AC-10).
+
+    ### **6.3 Robust M-Estimation (Solution for AC-3, AC-5)**
+
+    A 350m outlier (AC-3) or a bad GAB match (AC-5) will add a constraint with a *massive* error. A standard least-squares optimizer 14 would be *catastrophically* corrupted, pulling the *entire* 3000-image trajectory to try and fit this one bad point.
+
+    This is a solved problem. All constraints (V-SLAM and GAB) *must* be wrapped in a **Robust Loss Function** (e.g., HuberLoss, CauchyLoss) within Ceres Solver. This function mathematically *down-weights* the influence of constraints with large errors (high residuals). It effectively tells the optimizer: "This measurement is insane. Ignore it." This provides automatic, graceful outlier rejection, meeting AC-3 and AC-5.
+
+    ### **6.4 Geodetic Map-Merging (Solution for AC-4, AC-6)**
+
+    This mechanism is the robust solution to the "sharp turn" (AC-4) problem.
+
+    * **Scenario:** The UAV makes a sharp turn (AC-4). The V-SLAM (4.2) creates Map_Fragment_0 and Map_Fragment_1. The TOH's graph now has two *disconnected* components.  
+    * **Mechanism (Geodetic Merging):**  
+    1. The TOH queues the GAB (Section 5.0) to find anchors for *both* fragments.  
+    2. GAB returns Anchor_A for Map_Fragment_0 and Anchor_B for Map_Fragment_1.  
+    3. The TOH adds *both* of these as absolute, metric constraints (Edge 2) to the *single global pose-graph*.  
+    4. The Ceres optimizer 14 now has all the information it needs. It solves for the 7-Dof pose of *both fragments*, placing them in their correct, globally-consistent metric positions.
+
+    The two fragments are *merged geodetically* (by their global coordinates 11) even if they *never* visually overlap. This is a vastly more robust solution to AC-4 and AC-6 than simple visual loop closure.
+
+    ## **7.0 Performance, Deployment, and High-Accuracy Outputs**
+
+    ### **7.1 Meeting the <5s Budget (AC-7): Mandatory Acceleration with NVIDIA TensorRT**
+
+    The system must run on an RTX 2060 (AC-7). This is a low-end, 6GB VRAM card 34, which is a *severe* constraint. Running three deep-learning models (SuperPoint, LightGlue, SALAD/MixVPR) plus a Ceres optimizer 38 will saturate this hardware.
+
+    * **Solution 1: Multi-Scale Pipeline.** As defined in 5.3, the system *never* processes a full 6.2K image on the GPU. It uses low-res for V-SLAM/GAB-Coarse and high-res *patches* for GAB-Fine.  
+    * **Solution 2: Mandatory TensorRT Deployment.** Running these models in their native PyTorch framework will be too slow. All neural networks (SuperPoint, LightGlue, SALAD/MixVPR) *must* be converted from PyTorch into optimized **NVIDIA TensorRT engines**. Research *specifically* on accelerating LightGlue shows this provides **"2x-4x speed gains over compiled PyTorch"**.35 This 200-400% speedup is *not* an optimization; it is a *mandatory deployment step* to make the <5s (AC-7) budget *possible* on an RTX 2060.
+
+    ### **7.2 High-Accuracy Object Geolocalization via Ray-Cloud Intersection (Solves AC-2/AC-10)**
+
+    The user must be able to find the GPS of an *object* in a photo. A simple approach of ray-casting from the camera and intersecting with the 30m GLO-30 DEM 2 is fatally flawed. The DEM error itself can be up to 30m 19, making AC-2 impossible.
+
+    The ASTRAL system uses a **Ray-Cloud Intersection** method that *decouples* object accuracy from external DEM accuracy.
+
+    * **Algorithm:**  
+    1. The user clicks pixel (u,v) on Image_N.  
+    2. The system retrieves the *final, refined, metric 7-DoF pose* P_{sim(3)} = (s, R, T) for Image_N from the TOH.  
+    3. It also retrieves the V-SLAM's *local, high-fidelity 3D point cloud* (P_{local_cloud}) from Component 3 (Section 4.3).  
+    4. **Step 1 (Local):** The pixel (u,v) is un-projected into a ray. This ray is intersected with the *local* P_{local_cloud}. This finds the 3D point $P_{local} *relative to the V-SLAM map*. The accuracy of this step is defined by AC-10 (MRE < 1.0px).  
+    5. **Step 2 (Global):** This *highly-accurate* local point P_{local} is transformed into the global metric coordinate system using the *highly-accurate* refined pose from the TOH: P_{metric} = s * (R * P_{local}) + T.  
+    6. **Step 3 (Convert):** P_{metric} (an X,Y,Z world coordinate) is converted to [Latitude, Longitude, Altitude].
+
+    This method correctly isolates error. The object's accuracy is now *only* dependent on the V-SLAM's internal geometry (AC-10) and the TOH's global pose accuracy (AC-1, AC-2). It *completely eliminates* the external 30m DEM error 2 from this critical, high-accuracy calculation.
+
+    ### **7.3 Failure Mode Escalation Logic (Meets AC-3, AC-4, AC-6, AC-9)**
+
+    The system is built on a robust state machine to handle real-world failures.
+
+    * **Stage 1: Normal Operation (Tracking):** V-SLAM tracks, TOH optimizes.  
+    * **Stage 2: Transient VO Failure (Outlier Rejection):**  
+    * *Condition:* Image_N is a 350m outlier (AC-3) or severe blur.  
+    * *Logic:* V-SLAM fails to track Image_N. System *discards* it (AC-5). Image_N+1 arrives, V-SLAM re-tracks.  
+    * *Result:* **AC-3 Met.**  
+    * **Stage 3: Persistent VO Failure (New Map Initialization):**  
+    * *Condition:* "Sharp turn" (AC-4) or >5 frames of tracking loss.  
+    * *Logic:* V-SLAM (Section 4.2) declares "Tracking Lost." Initializes *new* Map_Fragment_k+1. Tracking *resumes instantly*.  
+    * *Result:* **AC-4 Met.** System "correctly continues the work." The >95% registration rate (AC-9) is met because this is *not* a failure, it's a *new registration*.  
+    * **Stage 4: Map-Merging & Global Relocalization (GAB-Assisted):**  
+    * *Condition:* System is on Map_Fragment_k+1, Map_Fragment_k is "lost."  
+    * *Logic:* TOH (Section 6.4) receives GAB anchors for *both* fragments and *geodetically merges* them in the global optimizer.14  
+    * *Result:* **AC-6 Met** (strategy to connect separate chunks).  
+    * **Stage 5: Catastrophic Failure (User Intervention):**  
+    * *Condition:* System is in Stage 3 (Lost) *and* the GAB has failed for 20% of the route. The "absolutely incapable" scenario (AC-6).  
+    * *Logic:* TOH triggers the AC-6 flag. UI prompts user: "Please provide a coarse location for the *current* image."  
+    * *Action:* This user-click is *not* taken as ground-truth. It is fed to the **GAB (Section 5.0)** as a *strong spatial prior*, narrowing its Stage 1 8 search from "the entire AOI" to "a 5km radius." This *guarantees* the GAB finds a match, which triggers Stage 4, re-localizing the system.  
+    * *Result:* **AC-6 Met** (user input).
+
+    ## **8.0 ASTRAL Validation Plan and Acceptance Criteria Matrix**
+
+    A comprehensive test plan is required to validate compliance with all 10 Acceptance Criteria. The foundation is a **Ground-Truth Test Harness** using project-provided ground-truth data.
+
+    ### **Table 4: ASTRAL Component vs. Acceptance Criteria Compliance Matrix**
+
+    | ID | Requirement | ASTRAL Solution (Component) | Key Technology / Justification |
+    | :---- | :---- | :---- | :---- |
+    | **AC-1** | 80% of photos < 50m error | GDB (C-1) + GAB (C-5) + TOH (C-6) | **Tier-1 (Copernicus)** data 1 is sufficient. SOTA VPR 8 + Sim(3) graph 13 can achieve this. |
+    | **AC-2** | 60% of photos < 20m error | GDB (C-1) + GAB (C-5) + TOH (C-6) | **Requires Tier-2 (Commercial) Data**.4 Mitigates reference error.3 **Per-Keyframe Scale** 15 model in TOH minimizes drift error. |
+    | **AC-3** | Robust to 350m outlier | V-SLAM (C-3) + TOH (C-6) | **Stage 2 Failure Logic** (7.3) discards the frame. **Robust M-Estimation** (6.3) in Ceres 14 automatically rejects the constraint. |
+    | **AC-4** | Robust to sharp turns (<5% overlap) | V-SLAM (C-3) + TOH (C-6) | **"Atlas" Multi-Map** (4.2) initializes new map (Map_Fragment_k+1). **Geodetic Map-Merging** (6.4) in TOH re-connects fragments via GAB anchors. |
+    | **AC-5** | < 10% outlier anchors | TOH (C-6) | **Robust M-Estimation (Huber Loss)** (6.3) in Ceres 14 automatically down-weights and ignores high-residual (bad) GAB anchors. |
+    | **AC-6** | Connect route chunks; User input | V-SLAM (C-3) + TOH (C-6) + UI | **Geodetic Map-Merging** (6.4) connects chunks. **Stage 5 Failure Logic** (7.3) provides the user-input-as-prior mechanism. |
+    | **AC-7** | < 5 seconds processing/image | All Components | **Multi-Scale Pipeline** (5.3) (Low-Res V-SLAM, Hi-Res GAB patches). **Mandatory TensorRT Acceleration** (7.1) for 2-4x speedup.35 |
+    | **AC-8** | Real-time stream + async refinement | TOH (C-5) + Outputs (C-2.4) | Decoupled architecture provides Pose_N_Est (V-SLAM) in real-time and Pose_N_Refined (TOH) asynchronously as GAB anchors arrive. |
+    | **AC-9** | Image Registration Rate > 95% | V-SLAM (C-3) | **"Atlas" Multi-Map** (4.2). A "lost track" (AC-4) is *not* a registration failure; it's a *new map registration*. This ensures the rate > 95%. |
+    | **AC-10** | Mean Reprojection Error (MRE) < 1.0px | V-SLAM (C-3) + TOH (C-6) | Local BA (4.3) + Global BA (TOH14) + **Per-Keyframe Scale** (6.2) minimizes internal graph tension (Flaw 1.3), allowing the optimizer to converge to a low MRE. |
+
+    ### **8.1 Rigorous Validation Methodology**
+
+    * **Test Harness:** A validation script will be created to compare the system's Pose_N^{Refined} output against a ground-truth coordinates.csv file, computing Haversine distance errors.  
+    * **Test Datasets:**  
+    * Test_Baseline: Standard flight.  
+    * Test_Outlier_350m (AC-3): A single, unrelated image inserted.  
+    * Test_Sharp_Turn_5pct (AC-4): A sequence with a 10-frame gap.  
+    * Test_Long_Route (AC-9, AC-7): A 2000-image sequence.  
+    * **Test Cases:**  
+    * Test_Accuracy: Run Test_Baseline. ASSERT (count(errors < 50m) / total) >= 0.80 (AC-1). ASSERT (count(errors < 20m) / total) >= 0.60 (AC-2).  
+    * Test_Robustness: Run Test_Outlier_350m and Test_Sharp_Turn_5pct. ASSERT system completes the run and Test_Accuracy assertions still pass on the valid frames.  
+    * Test_Performance: Run Test_Long_Route on min-spec RTX 2060. ASSERT average_time(Pose_N^{Est} output) < 5.0s (AC-7).  
+    * Test_MRE: ASSERT TOH.final_MRE < 1.0 (AC-10).
+
+Identify all potential weak points and problems. Address them and find out ways to solve them. Based on your findings, form a new solution draft in the same format.
+
+If your finding requires a complete reorganization of the flow and different components, state it.
+Put all the findings regarding what was weak and poor at the beginning of the report.
+At the very beginning of the report, list the most profound changes you've made to the previous solution.
+Then form a new solution design without referencing the previous system. Remove Poor and Very Poor component choices from the component analysis tables, but leave Good and Excellent ones.
+In the updated report, do not put "new" marks; do not compare to the previous solution draft, just make a new solution as if from scratch.
+
+Also, find out more ideas, like a 
+- A Cross-View Geo-Localization Algorithm Using UAV Image
+https://www.mdpi.com/1424-8220/24/12/3719
+- Exploring the best way for UAV visual localization under Low-altitude Multi-view Observation condition 
+ https://arxiv.org/pdf/2503.10692
+
+Assess them and try to either integrate or replace some of the components in the current solution draft

docs/00_problem/1.4_05_assesment_prompt.md

@@ -0,0 +1,373 @@
+Read carefully about the problem:
+
+ We have a lot of images taken from a wing-type UAV using a camera with at least Full HD resolution. Resolution of each photo could be up to 6200*4100 for the whole flight, but for other flights, it could be FullHD
+Photos are taken and named consecutively within 100 meters of each other.
+We know only the starting GPS coordinates. We need to determine the GPS of the centers of each image. And also the coordinates of the center of any object in these photos. We can use an external satellite provider for ground checks on the existing photos
+ 
+ System has next restrictions and conditions:
+  - Photos are taken by only airplane type UAVs.
+  - Photos are taken by the camera pointing downwards and fixed, but it is not autostabilized.
+  - The flying range is restricted by the eastern and southern parts of Ukraine (To the left of the Dnipro River)
+  - The image resolution could be from FullHD to 6252*4168. Camera parameters are known: focal length, sensor width, resolution and so on.
+  - Altitude is predefined and no more than 1km. The height of the terrain can be neglected.
+  - There is NO data from IMU
+  - Flights are done mostly in sunny weather
+  - We can use satellite providers, but we're limited right now to Google Maps, which could be outdated for some regions
+  - Number of photos could be up to 3000, usually in the 500-1500 range
+  - During the flight, UAVs can make sharp turns, so that the next photo may be absolutely different from the previous one (no same objects), but it is rather an exception than the rule
+  - Processing is done on a stationary computer or laptop with NVidia GPU at least RTX2060, better 3070. (For the UAV solution Jetson Orin Nano would be used, but that is out of scope.)
+  
+ Output of the system should address next acceptance criteria:
+    - The system should find out the GPS of centers of 80% of the photos from the flight within an error of no more than 50 meters in comparison to the real GPS
+
+    - The system should find out the GPS of centers of 60% of the photos from the flight within an error of no more than 20 meters in comparison to the real GPS
+
+    - The system should correctly continue the work even in the presence of up to 350 meters of an outlier photo between 2 consecutive pictures en route. This could happen due to tilt of the plane.
+
+    - System should correctly continue the work even during sharp turns, where the next photo doesn't overlap at all, or overlaps in less than 5%. The next photo should be in less than 200m drift and at an angle of less than 70%
+
+    - System should try to operate when UAV made a sharp turn, and all the next photos has no common points with previous route. In that situation system should try to figure out location of the new piece of the route and connect it to the previous route. Also this separate chunks could be more than 2, so this strategy should be in the core of the system
+
+    - In case of being absolutely incapable of determining the system to determine next, second next, and third next images GPS, by any means (these 20% of the route), then it should ask the user for input for the next image, so that the user can specify the location
+
+    - Less than 5 seconds for processing one image
+
+    - Results of image processing should appear immediately to user, so that user shouldn't wait for the whole route to complete in order to analyze first results. Also, system could refine existing calculated results and send refined results again to user 
+
+    - Image Registration Rate > 95%. The system can find enough matching features to confidently calculate the camera's 6-DoF pose (position and orientation) and "stitch" that image into the final trajectory
+
+    - Mean Reprojection Error (MRE) < 1.0 pixels. The distance, in pixels, between the original pixel location of the object and the re-projected pixel location.
+
+    - The whole system should work as a background service. The interaction should be done by zeromq. Sevice should be up and running and awaiting for the initial input message. On the input message processing should started, and immediately after the first results system should provide them to the client
+
+ Here is a solution draft:
+
+
+    # **ASTRAL System Architecture: A High-Fidelity Geopositioning Framework for IMU-Denied Aerial Operations**
+
+    ## **2.0 The ASTRAL (Advanced Scale-Aware Trajectory-Refinement and Localization) System Architecture**
+
+    The ASTRAL architecture is a multi-map, decoupled, loosely-coupled system designed to solve the flaws identified in Section 1.0 and meet all 10 Acceptance Criteria.
+
+    ### **2.1 Core Principles**
+
+    The ASTRAL architecture is built on three principles:
+
+    1. **Tiered Geospatial Database:** The system *cannot* rely on a single data source. It is architected around a *tiered* local database.  
+    * **Tier-1 (Baseline):** Google Maps data. This is used to meet the 50m (AC-1) requirement and provide geolocalization.  
+    * **Tier-2 (High-Accuracy):** A framework for ingesting *commercial, sub-meter* data (visual 4; and DEM 5). This tier is *required* to meet the 20m (AC-2) accuracy. The system will *run* on Tier-1 but *achieve* AC-2 when "fueled" with Tier-2 data.  
+    2. **Viewpoint-Invariant Anchoring:** The system *rejects* geometric warping. The GAB (Section 5.0) is built on SOTA Visual Place Recognition (VPR) models that are *inherently* invariant to the oblique-to-nadir viewpoint change, decoupling it from the V-SLAM's unstable orientation.  
+    3. **Continuously-Scaled Trajectory:** The system *rejects* the "single-scale-per-fragment" model. The TOH (Section 6.0) is a Sim(3) pose-graph optimizer 11 that models scale as a *per-keyframe optimizable parameter*.15 This allows the trajectory to "stretch" and "shrink" elastically to absorb continuous monocular scale drift.12
+
+    ### **2.2 Component Interaction and Data Flow**
+
+    The system is multi-threaded and asynchronous, designed for real-time streaming (AC-7) and refinement (AC-8).
+
+    * **Component 1: Tiered GDB (Pre-Flight):**  
+    * *Input:* User-defined Area of Interest (AOI).  
+    * *Action:* Downloads and builds a local SpatiaLite/GeoPackage.  
+    * *Output:* A single **Local-Geo-Database file** containing:  
+        * Tier-1 (Google Maps) + GLO-30 DSM  
+        * Tier-2 (Commercial) satellite tiles + WorldDEM DTM elevation tiles.  
+        * A *pre-computed FAISS vector index* of global descriptors (e.g., SALAD 8) for *all* satellite tiles (see 3.4).  
+    * **Component 2: Image Ingestion (Real-time):**  
+    * *Input:* Image_N (up to 6.2K), Camera Intrinsics ($K$).  
+    * *Action:* Creates Image_N_LR (Low-Res, e.g., 1536x1024) and Image_N_HR (High-Res, 6.2K).  
+    * *Dispatch:* Image_N_LR -> V-SLAM. Image_N_HR -> GAB (for patches).  
+    * **Component 3: "Atlas" V-SLAM Front-End (High-Frequency Thread):**  
+    * *Input:* Image_N_LR.  
+    * *Action:* Tracks Image_N_LR against the *active map fragment*. Manages keyframes and local BA. If tracking lost (AC-4, AC-6), it *initializes a new map fragment*.  
+    * *Output:* Relative_Unscaled_Pose, Local_Point_Cloud, and Map_Fragment_ID -> TOH.  
+    * **Component 4: VPR Geospatial Anchoring Back-End (GAB) (Low-Frequency, Asynchronous Thread):**  
+    * *Input:* A keyframe (Image_N_LR, Image_N_HR) and its Map_Fragment_ID.  
+    * *Action:* Performs SOTA two-stage VPR (Section 5.0) against the **Local-Geo-Database file**.  
+    * *Output:* Absolute_Metric_Anchor ([Lat, Lon, Alt] pose) and its Map_Fragment_ID -> TOH.  
+    * **Component 5: Scale-Aware Trajectory Optimization Hub (TOH) (Central Hub Thread):**  
+    * *Input 1:* High-frequency Relative_Unscaled_Pose stream.  
+    * *Input 2:* Low-frequency Absolute_Metric_Anchor stream.  
+    * *Action:* Manages the *global Sim(3) pose-graph* 13 with *per-keyframe scale*.15  
+    * *Output 1 (Real-time):* Pose_N_Est (unscaled) -> UI (Meets AC-7).  
+    * *Output 2 (Refined):* Pose_N_Refined (metric-scale) -> UI (Meets AC-1, AC-2, AC-8).
+
+    ### **2.3 System Inputs**
+
+    1. **Image Sequence:** Consecutively named images (FullHD to 6252x4168).  
+    2. **Start Coordinate (Image 0):** A single, absolute GPS coordinate [Lat, Lon].  
+    3. **Camera Intrinsics (K):** Pre-calibrated camera intrinsic matrix.  
+    4. **Local-Geo-Database File:** The single file generated by Component 1.
+
+    ### **2.4 Streaming Outputs (Meets AC-7, AC-8)**
+
+    1. **Initial Pose (Pose_N^{Est}):** An *unscaled* pose. This is the raw output from the V-SLAM Front-End, transformed by the *current best estimate* of the trajectory. It is sent immediately (<5s, AC-7) to the UI for real-time visualization of the UAV's *path shape*.  
+    2. **Refined Pose (Pose_N^{Refined}) [Asynchronous]:** A globally-optimized, *metric-scale* 7-DoF pose. This is sent to the user *whenever the TOH re-converges* (e.g., after a new GAB anchor or a map-merge). This *re-writes* the history of poses (e.g., Pose_{N-100} to Pose_N), meeting the refinement (AC-8) and accuracy (AC-1, AC-2) requirements.
+
+    ## **3.0 Component 1: The Tiered Pre-Flight Geospatial Database (GDB)**
+
+    This component is the implementation of the "Tiered Geospatial" principle. It is a mandatory pre-flight utility that solves both the *legal* problem (Flaw 1.4) and the *accuracy* problem (Flaw 1.1).
+
+    ### **3.2 Tier-1 (Baseline): Google Maps and GLO-30 DEM**
+
+    This tier provides the baseline capability and satisfies AC-1.
+
+    * **Visual Data:** Google Maps (coarse Maxar)  
+    * *Resolution:* 10m.  
+    * *Geodetic Accuracy:* \~1 m to 20m  
+    * *Purpose:* Meets AC-1 (80% < 50m error). Provides a robust baseline for coarse geolocalization.  
+    * **Elevation Data:** Copernicus GLO-30 DEM  
+    * *Resolution:* 30m.  
+    * *Type:* DSM (Digital Surface Model).2 This is a *weakness*, as it includes buildings/trees.  
+    * *Purpose:* Provides a coarse altitude prior for the TOH and the initial GAB search.
+
+    ### **3.3 Tier-2 (High-Accuracy): Ingestion Framework for Commercial Data**
+
+    This is the *procurement and integration framework* required to meet AC-2.
+
+    * **Visual Data:** Commercial providers, e.g., Maxar (30-50cm) or Satellogic (70cm)  
+    * *Resolution:* < 1m.  
+    * *Geodetic Accuracy:* Typically < 5m.  
+    * *Purpose:* Provides the high-resolution, high-accuracy reference needed for the GAB to achieve a sub-20m total error.  
+    * **Elevation Data:** Commercial providers, e.g., WorldDEM Neo 5 or Elevation10.32  
+    * *Resolution:* 5m-12m.  
+    * *Vertical Accuracy:* < 4m.32  
+    * *Type:* DTM (Digital Terrain Model).32
+
+    The use of a DTM (bare-earth) in Tier-2 is a critical advantage over the Tier-1 DSM (surface). The V-SLAM Front-End (Section 4.0) will triangulate a 3D point cloud of what it *sees*, which is the *ground* in fields or *tree-tops* in forests. The Tier-1 GLO-30 DSM 2 represents the *top* of the canopy/buildings. If the V-SLAM maps the *ground* (e.g., altitude 100m) and the GAB tries to anchor it to a DSM *prior* that shows a forest (e.g., altitude 120m), the 20m altitude discrepancy will introduce significant error into the TOH. The Tier-2 DTM (bare-earth) 5 provides a *vastly* superior altitude anchor, as it represents the same ground plane the V-SLAM is tracking, significantly improving the entire 7-DoF pose solution.
+
+    ### **3.4 Local Database Generation: Pre-computing Global Descriptors**
+
+    This is the key performance optimization for the GAB. During the pre-flight caching step, the GDB utility does not just *store* tiles; it *processes* them.
+
+    For *every* satellite tile (e.g., 256x256m) in the AOI, the utility will load the tile into the VPR model (e.g., SALAD 8), compute its global descriptor (a compact feature vector), and store this vector in a high-speed vector index (e.g., FAISS).
+
+    This step moves 99% of the GAB's "Stage 1" (Coarse Retrieval) workload into an offline, pre-flight step. The *real-time* GAB query (Section 5.2) is now reduced to: (1) Compute *one* vector for the UAV image, and (2) Perform a very fast K-Nearest-Neighbor search on the pre-computed FAISS index. This is what makes a SOTA deep-learning GAB 6 fast enough to support the real-time refinement loop.
+
+    #### **Table 1: Geospatial Reference Data Analysis (Decision Matrix)**
+
+    | Data Product | Type | Resolution | Geodetic Accuracy (Horiz.) | Type | Cost | AC-2 (20m) Compliant? |
+    | :---- | :---- | :---- | :---- | :---- | :---- | :---- |
+    | Google Maps | Visual | 1m | 1m - 10m | N/A | Free | **Depending on the location** |
+    | Copernicus GLO-30 | Elevation | 30m | \~10-30m | **DSM** (Surface) | Free | **No (Fails Error Budget)** |
+    | **Tier-2: Maxar/Satellogic** | Visual | 0.3m - 0.7m | < 5 m (Est.) | N/A | Commercial | **Yes** |
+    | **Tier-2: WorldDEM Neo**  | Elevation | 5m | < 4m | **DTM** (Bare-Earth) | Commercial | **Yes** |
+
+    ## **4.0 Component 2: The "Atlas" Relative Motion Front-End**
+
+    This component's sole task is to robustly compute *unscaled* 6-DoF relative motion and handle tracking failures (AC-3, AC-4).
+
+    ### **4.1 Feature Matching Sub-System: SuperPoint + LightGlue**
+
+    The system will use **SuperPoint** for feature detection and **LightGlue** for matching. This choice is driven by the project's specific constraints:
+
+    * **Rationale (Robustness):** The UAV flies over "eastern and southern parts of Ukraine," which includes large, low-texture agricultural areas. SuperPoint is a SOTA deep-learning detector renowned for its robustness and repeatability in these challenging, low-texture environments.  
+    * **Rationale (Performance):** The RTX 2060 (AC-7) is a *hard* constraint with only 6GB VRAM.34 Performance is paramount. LightGlue is an SOTA matcher that provides a 4-10x speedup over its predecessor, SuperGlue. Its "adaptive" nature is a key optimization: it exits early on "easy" pairs (high-overlap, straight-flight) and spends more compute only on "hard" pairs (turns). This saves critical GPU budget on 95% of normal frames, ensuring the <5s (AC-7) budget is met.
+
+    This subsystem will run on the Image_N_LR (low-res) copy to guarantee it fits in VRAM and meets the real-time budget.
+
+    #### **Table 2: Analysis of State-of-the-Art Feature Matchers (V-SLAM Front-End)**
+
+    | Approach (Tools/Library) | Robustness (Low-Texture) | Speed (RTX 2060) | Fitness for Problem |
+    | :---- | :---- | :---- | :---- |
+    | ORB 33 (e.g., ORB-SLAM3) | Poor. Fails on low-texture. | Excellent (CPU/GPU) | **Good.** Fails robustness in target environment. |
+    | SuperPoint + SuperGlue | Excellent. | Good, but heavy. Fixed-depth GNN. 4-10x Slower than LightGlue.35 | **Good.** Robust, but risks AC-7 budget. |
+    | **SuperPoint + LightGlue** 35 | Excellent. | **Excellent.** Adaptive depth 35 saves budget. 4-10x faster. | **Excellent (Selected).** Balances robustness and performance. |
+
+    ### **4.2 The "Atlas" Multi-Map Paradigm (Solution for AC-3, AC-4, AC-6)**
+
+    This architecture is the industry-standard solution for IMU-denied, long-term SLAM and is critical for robustness.
+
+    * **Mechanism (AC-4, Sharp Turn):**  
+    1. The system is tracking on $Map_Fragment_0$.  
+    2. The UAV makes a sharp turn (AC-4, <5% overlap). The V-SLAM *loses tracking*.  
+    3. Instead of failing, the Atlas architecture *initializes a new map*: $Map_Fragment_1$.  
+    4. Tracking *resumes instantly* on this new, unanchored map.  
+    * **Mechanism (AC-3, 350m Outlier):**  
+    1. The system is tracking. A 350m outlier $Image_N$ arrives.  
+    2. The V-SLAM fails to match $Image_N$ (a "Transient VO Failure," see 7.3). It is *discarded*.  
+    3. $Image_N+1$ arrives (back on track). V-SLAM re-acquires its location on $Map_Fragment_0$.  
+    4. The system "correctly continues the work" (AC-3) by simply rejecting the outlier.
+
+    This design turns "catastrophic failure" (AC-3, AC-4) into a *standard operating procedure*. The "problem" of stitching the fragments ($Map_0$, $Map_1$) together is moved from the V-SLAM (which has no global context) to the TOH (which *can* solve it using GAB anchors, see 6.4).
+
+    ### **4.3 Local Bundle Adjustment and High-Fidelity 3D Cloud**
+
+    The V-SLAM front-end will continuously run Local Bundle Adjustment (BA) over a sliding window of recent keyframes to minimize drift *within* that fragment. It will also triangulate a sparse, but high-fidelity, 3D point cloud for its *local map fragment*.
+
+    This 3D cloud serves a critical dual function:
+
+    1. It provides a robust 3D map for frame-to-map tracking, which is more stable than frame-to-frame odometry.  
+    2. It serves as the **high-accuracy data source** for the object localization output (Section 7.2). This is the key to decoupling object-pointing accuracy from external DEM accuracy 19, a critical flaw in simpler designs.
+
+    ## **5.0 Component 3: The Viewpoint-Invariant Geospatial Anchoring Back-End (GAB)**
+
+    This component *replaces* the draft's "Dynamic Warping" (Section 5.0) and implements the "Viewpoint-Invariant Anchoring" principle (Section 2.1).
+
+    ### **5.1 Rationale: Viewpoint-Invariant VPR vs. Geometric Warping (Solves Flaw 1.2)**
+
+    As established in 1.2, geometrically warping the image using the V-SLAM's *drifty* roll/pitch estimate creates a *brittle*, high-risk failure spiral. The ASTRAL GAB *decouples* from the V-SLAM's orientation. It uses a SOTA VPR pipeline that *learns* to match oblique UAV images to nadir satellite images *directly*, at the feature level.6
+
+    ### **5.2 Stage 1 (Coarse Retrieval): SOTA Global Descriptors**
+
+    When triggered by the TOH, the GAB takes Image_N_LR. It computes a *global descriptor* (a single feature vector) using a SOTA VPR model like **SALAD** 6 or **MixVPR**.7
+
+    This choice is driven by two factors:
+
+    1. **Viewpoint Invariance:** These models are SOTA for this exact task.  
+    2. **Inference Speed:** They are extremely fast. SALAD reports < 3ms per image inference 8, and MixVPR is also noted for "fastest inference speed".37 This low overhead is essential for the AC-7 (<5s) budget.
+
+    This vector is used to query the *pre-computed FAISS vector index* (from 3.4), which returns the Top-K (e.g., K=5) most likely satellite tiles from the *entire AOI* in milliseconds.
+
+    #### **Table 3: Analysis of VPR Global Descriptors (GAB Back-End)**
+
+    | Model (Backbone) | Key Feature | Viewpoint Invariance | Inference Speed (ms) | Fitness for GAB |
+    | :---- | :---- | :---- | :---- | :---- |
+    | NetVLAD 7 (CNN) | Baseline | Poor. Not designed for oblique-to-nadir. | Moderate (\~20-50ms) | **Poor.** Fails robustness. |
+    | **SALAD** 8 (DINOv2) | Foundation Model.6 | **Excellent.** Designed for this. | **< 3ms**.8 Extremely fast. | **Excellent (Selected).** |
+    | **MixVPR** 36 (ResNet) | All-MLP aggregator.36 | **Very Good.**.7 | **Very Fast.**.37 | **Excellent (Selected).** |
+
+    ### **5.3 Stage 2 (Fine): Local Feature Matching and Pose Refinement**
+
+    The system runs **SuperPoint+LightGlue** 35 to find pixel-level matches, but *only* between the UAV image and the **Top-K satellite tiles** identified in Stage 1.
+
+    A **Multi-Resolution Strategy** is employed to solve the VRAM bottleneck.
+
+    1. Stage 1 (Coarse) runs on the Image_N_LR.  
+    2. Stage 2 (Fine) runs SuperPoint *selectively* on the Image_N_HR (6.2K) to get high-accuracy keypoints.  
+    3. It then matches small, full-resolution *patches* from the full-res image, *not* the full image.
+
+    This hybrid approach is the *only* way to meet both AC-7 (speed) and AC-2 (accuracy). The 6.2K image *cannot* be processed in <5s on an RTX 2060 (6GB VRAM 34). But its high-resolution *pixels* are needed for the 20m *accuracy*. Using full-res *patches* provides the pixel-level accuracy without the VRAM/compute cost.
+
+    A PnP/RANSAC solver then computes a high-confidence 6-DoF pose. This pose, converted to [Lat, Lon, Alt], is the **$Absolute_Metric_Anchor$** sent to the TOH.
+
+    ## **6.0 Component 4: The Scale-Aware Trajectory Optimization Hub (TOH)**
+
+    This component is the system's "brain" and implements the "Continuously-Scaled Trajectory" principle (Section 2.1). It *replaces* the draft's flawed "Single Scale" optimizer.
+
+    ### **6.1 The $Sim(3)$ Pose-Graph as the Optimization Backbone**
+
+    The central challenge of IMU-denied monocular SLAM is *scale drift*.11 The V-SLAM (Component 3) produces 6-DoF poses, but they are *unscaled* ($SE(3)$). The GAB (Component 4) produces *metric* 6-DoF poses ($SE(3)$).
+
+    The solution is to optimize the *entire graph* in the 7-DoF "Similarity" group, **$Sim(3)$**.11 This adds a 7th degree of freedom (scale, $s$) to the poses. The optimization backbone will be **Ceres Solver** 14, a SOTA C++ library for large, complex non-linear least-squares problems.
+
+    ### **6.2 Advanced Scale-Drift Correction: Modeling Scale as a Per-Keyframe Parameter (Solves Flaw 1.3)**
+
+    This is the *core* of the ASTRAL optimizer, solving Flaw 1.3. The draft's flawed model ($Pose_Graph(Fragment_i) = \\{Pose_1...Pose_n, s_i\\}$) is replaced by ASTRAL's correct model: $Pose_Graph = \\{ (Pose_1, s_1), (Pose_2, s_2),..., (Pose_N, s_N) \\}$.
+
+    The graph is constructed as follows:
+
+    * **Nodes:** Each keyframe pose is a 7-DoF $Sim(3)$ variable $\\{s_k, R_k, t_k\\}$.  
+    * **Edge 1 (V-SLAM):** A *relative* $Sim(3)$ constraint between $Pose_k$ and $Pose_{k+1}$ from the V-SLAM Front-End.  
+    * **Edge 2 (GAB):** An *absolute* $SE(3)$ constraint on $Pose_j$ from a GAB anchor. This constraint *fixes* the 6-DoF pose $(R_j, t_j)$ to the metric GAB value and *fixes its scale* $s_j = 1.0$.
+
+    This "per-keyframe scale" model 15 enables "elastic" trajectory refinement. When the graph is a long, unscaled "chain" of V-SLAM constraints, a GAB anchor (Edge 2) arrives at $Pose_{100}$, "nailing" it to the metric map and setting $s_{100} = 1.0$. As the V-SLAM continues, scale drifts. When a second anchor arrives at $Pose_{200}$ (setting $s_{200} = 1.0$), the Ceres optimizer 14 has a problem: the V-SLAM data *between* them has drifted.
+
+    The ASTRAL model *allows* the optimizer to solve for all intermediate scales (s_{101}, s_{102},..., s_{199}) as variables. The optimizer will find a *smooth, continuous* scale correction 15 that "elastically" stretches/shrinks the 100-frame sub-segment to *perfectly* fit both metric anchors. This *correctly* models the physics of scale drift 12 and is the *only* way to achieve the 20m accuracy (AC-2) and 1.0px MRE (AC-10).
+
+    ### **6.3 Robust M-Estimation (Solution for AC-3, AC-5)**
+
+    A 350m outlier (AC-3) or a bad GAB match (AC-5) will add a constraint with a *massive* error. A standard least-squares optimizer 14 would be *catastrophically* corrupted, pulling the *entire* 3000-image trajectory to try and fit this one bad point.
+
+    This is a solved problem. All constraints (V-SLAM and GAB) *must* be wrapped in a **Robust Loss Function** (e.g., HuberLoss, CauchyLoss) within Ceres Solver. This function mathematically *down-weights* the influence of constraints with large errors (high residuals). It effectively tells the optimizer: "This measurement is insane. Ignore it." This provides automatic, graceful outlier rejection, meeting AC-3 and AC-5.
+
+    ### **6.4 Geodetic Map-Merging (Solution for AC-4, AC-6)**
+
+    This mechanism is the robust solution to the "sharp turn" (AC-4) problem.
+
+    * **Scenario:** The UAV makes a sharp turn (AC-4). The V-SLAM (4.2) creates Map_Fragment_0 and Map_Fragment_1. The TOH's graph now has two *disconnected* components.  
+    * **Mechanism (Geodetic Merging):**  
+    1. The TOH queues the GAB (Section 5.0) to find anchors for *both* fragments.  
+    2. GAB returns Anchor_A for Map_Fragment_0 and Anchor_B for Map_Fragment_1.  
+    3. The TOH adds *both* of these as absolute, metric constraints (Edge 2) to the *single global pose-graph*.  
+    4. The Ceres optimizer 14 now has all the information it needs. It solves for the 7-Dof pose of *both fragments*, placing them in their correct, globally-consistent metric positions.
+
+    The two fragments are *merged geodetically* (by their global coordinates 11) even if they *never* visually overlap. This is a vastly more robust solution to AC-4 and AC-6 than simple visual loop closure.
+
+    ## **7.0 Performance, Deployment, and High-Accuracy Outputs**
+
+    ### **7.1 Meeting the <5s Budget (AC-7): Mandatory Acceleration with NVIDIA TensorRT**
+
+    The system must run on an RTX 2060 (AC-7). This is a low-end, 6GB VRAM card 34, which is a *severe* constraint. Running three deep-learning models (SuperPoint, LightGlue, SALAD/MixVPR) plus a Ceres optimizer 38 will saturate this hardware.
+
+    * **Solution 1: Multi-Scale Pipeline.** As defined in 5.3, the system *never* processes a full 6.2K image on the GPU. It uses low-res for V-SLAM/GAB-Coarse and high-res *patches* for GAB-Fine.  
+    * **Solution 2: Mandatory TensorRT Deployment.** Running these models in their native PyTorch framework will be too slow. All neural networks (SuperPoint, LightGlue, SALAD/MixVPR) *must* be converted from PyTorch into optimized **NVIDIA TensorRT engines**. Research *specifically* on accelerating LightGlue shows this provides **"2x-4x speed gains over compiled PyTorch"**.35 This 200-400% speedup is *not* an optimization; it is a *mandatory deployment step* to make the <5s (AC-7) budget *possible* on an RTX 2060.
+
+    ### **7.2 High-Accuracy Object Geolocalization via Ray-Cloud Intersection (Solves AC-2/AC-10)**
+
+    The user must be able to find the GPS of an *object* in a photo. A simple approach of ray-casting from the camera and intersecting with the 30m GLO-30 DEM 2 is fatally flawed. The DEM error itself can be up to 30m 19, making AC-2 impossible.
+
+    The ASTRAL system uses a **Ray-Cloud Intersection** method that *decouples* object accuracy from external DEM accuracy.
+
+    * **Algorithm:**  
+    1. The user clicks pixel (u,v) on Image_N.  
+    2. The system retrieves the *final, refined, metric 7-DoF pose* P_{sim(3)} = (s, R, T) for Image_N from the TOH.  
+    3. It also retrieves the V-SLAM's *local, high-fidelity 3D point cloud* (P_{local_cloud}) from Component 3 (Section 4.3).  
+    4. **Step 1 (Local):** The pixel (u,v) is un-projected into a ray. This ray is intersected with the *local* P_{local_cloud}. This finds the 3D point $P_{local} *relative to the V-SLAM map*. The accuracy of this step is defined by AC-10 (MRE < 1.0px).  
+    5. **Step 2 (Global):** This *highly-accurate* local point P_{local} is transformed into the global metric coordinate system using the *highly-accurate* refined pose from the TOH: P_{metric} = s * (R * P_{local}) + T.  
+    6. **Step 3 (Convert):** P_{metric} (an X,Y,Z world coordinate) is converted to [Latitude, Longitude, Altitude].
+
+    This method correctly isolates error. The object's accuracy is now *only* dependent on the V-SLAM's internal geometry (AC-10) and the TOH's global pose accuracy (AC-1, AC-2). It *completely eliminates* the external 30m DEM error 2 from this critical, high-accuracy calculation.
+
+    ### **7.3 Failure Mode Escalation Logic (Meets AC-3, AC-4, AC-6, AC-9)**
+
+    The system is built on a robust state machine to handle real-world failures.
+
+    * **Stage 1: Normal Operation (Tracking):** V-SLAM tracks, TOH optimizes.  
+    * **Stage 2: Transient VO Failure (Outlier Rejection):**  
+    * *Condition:* Image_N is a 350m outlier (AC-3) or severe blur.  
+    * *Logic:* V-SLAM fails to track Image_N. System *discards* it (AC-5). Image_N+1 arrives, V-SLAM re-tracks.  
+    * *Result:* **AC-3 Met.**  
+    * **Stage 3: Persistent VO Failure (New Map Initialization):**  
+    * *Condition:* "Sharp turn" (AC-4) or >5 frames of tracking loss.  
+    * *Logic:* V-SLAM (Section 4.2) declares "Tracking Lost." Initializes *new* Map_Fragment_k+1. Tracking *resumes instantly*.  
+    * *Result:* **AC-4 Met.** System "correctly continues the work." The >95% registration rate (AC-9) is met because this is *not* a failure, it's a *new registration*.  
+    * **Stage 4: Map-Merging & Global Relocalization (GAB-Assisted):**  
+    * *Condition:* System is on Map_Fragment_k+1, Map_Fragment_k is "lost."  
+    * *Logic:* TOH (Section 6.4) receives GAB anchors for *both* fragments and *geodetically merges* them in the global optimizer.14  
+    * *Result:* **AC-6 Met** (strategy to connect separate chunks).  
+    * **Stage 5: Catastrophic Failure (User Intervention):**  
+    * *Condition:* System is in Stage 3 (Lost) *and* the GAB has failed for 20% of the route. The "absolutely incapable" scenario (AC-6).  
+    * *Logic:* TOH triggers the AC-6 flag. UI prompts user: "Please provide a coarse location for the *current* image."  
+    * *Action:* This user-click is *not* taken as ground-truth. It is fed to the **GAB (Section 5.0)** as a *strong spatial prior*, narrowing its Stage 1 8 search from "the entire AOI" to "a 5km radius." This *guarantees* the GAB finds a match, which triggers Stage 4, re-localizing the system.  
+    * *Result:* **AC-6 Met** (user input).
+
+    ## **8.0 ASTRAL Validation Plan and Acceptance Criteria Matrix**
+
+    A comprehensive test plan is required to validate compliance with all 10 Acceptance Criteria. The foundation is a **Ground-Truth Test Harness** using project-provided ground-truth data.
+
+    ### **Table 4: ASTRAL Component vs. Acceptance Criteria Compliance Matrix**
+
+    | ID | Requirement | ASTRAL Solution (Component) | Key Technology / Justification |
+    | :---- | :---- | :---- | :---- |
+    | **AC-1** | 80% of photos < 50m error | GDB (C-1) + GAB (C-5) + TOH (C-6) | **Tier-1 (Copernicus)** data 1 is sufficient. SOTA VPR 8 + Sim(3) graph 13 can achieve this. |
+    | **AC-2** | 60% of photos < 20m error | GDB (C-1) + GAB (C-5) + TOH (C-6) | **Requires Tier-2 (Commercial) Data**.4 Mitigates reference error.3 **Per-Keyframe Scale** 15 model in TOH minimizes drift error. |
+    | **AC-3** | Robust to 350m outlier | V-SLAM (C-3) + TOH (C-6) | **Stage 2 Failure Logic** (7.3) discards the frame. **Robust M-Estimation** (6.3) in Ceres 14 automatically rejects the constraint. |
+    | **AC-4** | Robust to sharp turns (<5% overlap) | V-SLAM (C-3) + TOH (C-6) | **"Atlas" Multi-Map** (4.2) initializes new map (Map_Fragment_k+1). **Geodetic Map-Merging** (6.4) in TOH re-connects fragments via GAB anchors. |
+    | **AC-5** | < 10% outlier anchors | TOH (C-6) | **Robust M-Estimation (Huber Loss)** (6.3) in Ceres 14 automatically down-weights and ignores high-residual (bad) GAB anchors. |
+    | **AC-6** | Connect route chunks; User input | V-SLAM (C-3) + TOH (C-6) + UI | **Geodetic Map-Merging** (6.4) connects chunks. **Stage 5 Failure Logic** (7.3) provides the user-input-as-prior mechanism. |
+    | **AC-7** | < 5 seconds processing/image | All Components | **Multi-Scale Pipeline** (5.3) (Low-Res V-SLAM, Hi-Res GAB patches). **Mandatory TensorRT Acceleration** (7.1) for 2-4x speedup.35 |
+    | **AC-8** | Real-time stream + async refinement | TOH (C-5) + Outputs (C-2.4) | Decoupled architecture provides Pose_N_Est (V-SLAM) in real-time and Pose_N_Refined (TOH) asynchronously as GAB anchors arrive. |
+    | **AC-9** | Image Registration Rate > 95% | V-SLAM (C-3) | **"Atlas" Multi-Map** (4.2). A "lost track" (AC-4) is *not* a registration failure; it's a *new map registration*. This ensures the rate > 95%. |
+    | **AC-10** | Mean Reprojection Error (MRE) < 1.0px | V-SLAM (C-3) + TOH (C-6) | Local BA (4.3) + Global BA (TOH14) + **Per-Keyframe Scale** (6.2) minimizes internal graph tension (Flaw 1.3), allowing the optimizer to converge to a low MRE. |
+
+    ### **8.1 Rigorous Validation Methodology**
+
+    * **Test Harness:** A validation script will be created to compare the system's Pose_N^{Refined} output against a ground-truth coordinates.csv file, computing Haversine distance errors.  
+    * **Test Datasets:**  
+    * Test_Baseline: Standard flight.  
+    * Test_Outlier_350m (AC-3): A single, unrelated image inserted.  
+    * Test_Sharp_Turn_5pct (AC-4): A sequence with a 10-frame gap.  
+    * Test_Long_Route (AC-9, AC-7): A 2000-image sequence.  
+    * **Test Cases:**  
+    * Test_Accuracy: Run Test_Baseline. ASSERT (count(errors < 50m) / total) >= 0.80 (AC-1). ASSERT (count(errors < 20m) / total) >= 0.60 (AC-2).  
+    * Test_Robustness: Run Test_Outlier_350m and Test_Sharp_Turn_5pct. ASSERT system completes the run and Test_Accuracy assertions still pass on the valid frames.  
+    * Test_Performance: Run Test_Long_Route on min-spec RTX 2060. ASSERT average_time(Pose_N^{Est} output) < 5.0s (AC-7).  
+    * Test_MRE: ASSERT TOH.final_MRE < 1.0 (AC-10).
+
+
+Identify all potential weak points and problems. Address them and find out ways to solve them. Based on your findings, form a new solution draft in the same format.
+
+If your finding requires a complete reorganization of the flow and different components, state it.
+Put all the findings regarding what was weak and poor at the beginning of the report.
+At the very beginning of the report list most profound changes you've made to previous solution.
+
+Then form a new solution design without referencing the previous system. Remove Poor and Very Poor component choices from the component analysis tables, but leave Good and Excellent ones.
+In the updated report, do not put "new" marks, do not compare to the previous solution draft, just make a new solution as if from scratch

docs/00_problem/acceptance_criteria.md

@@ -17,3 +17,5 @@
 - Image Registration Rate > 95%. The system can find enough matching features to confidently calculate the camera's 6-DoF pose (position and orientation) and "stitch" that image into the final trajectory
 
 - Mean Reprojection Error (MRE) < 1.0 pixels. The distance, in pixels, between the original pixel location of the object and the re-projected pixel location.
+
+- The whole system should work as a background service. The interaction should be done by zeromq. Sevice should be up and running and awaiting for the initial input message. On the input message processing should started, and immediately after the first results system should provide them to the client

docs/00_problem/input_data/data_parameters.md

@@ -0,0 +1,8 @@
+- Height
+  - 400m
+- Camera:
+  - Name: ADTi Surveyor Lite 26S v2
+  - Resolution: 26MP
+  - Image resolution: 6252*4168
+  - Focal length: 25mm
+  - Sensor width: 23.5