component decomposition is done

2026-04-23 05:26:37 +00:00 · 2025-11-24 14:09:23 +02:00
parent acec83018b
commit f50006d100
34 changed files with 8637 additions and 0 deletions
@@ -0,0 +1,308 @@
+# Metric Refinement
+
+## Interface Definition
+
+**Interface Name**: `IMetricRefinement`
+
+### Interface Methods
+
+```python
+class IMetricRefinement(ABC):
+    @abstractmethod
+    def align_to_satellite(self, uav_image: np.ndarray, satellite_tile: np.ndarray) -> Optional[AlignmentResult]:
+        pass
+    
+    @abstractmethod
+    def compute_homography(self, uav_image: np.ndarray, satellite_tile: np.ndarray) -> Optional[np.ndarray]:
+        pass
+    
+    @abstractmethod
+    def extract_gps_from_alignment(self, homography: np.ndarray, tile_bounds: TileBounds, image_center: Tuple[int, int]) -> GPSPoint:
+        pass
+    
+    @abstractmethod
+    def compute_match_confidence(self, alignment: AlignmentResult) -> float:
+        pass
+```
+
+## Component Description
+
+### Responsibilities
+- LiteSAM for precise UAV-to-satellite cross-view matching
+- **Requires pre-rotated images** from Image Rotation Manager
+- Compute homography mapping UAV image to satellite tile
+- Extract absolute GPS coordinates from alignment
+- Process against single tile (drift correction) or tile grid (progressive search)
+- Achieve <20m accuracy requirement
+
+### Scope
+- Cross-view geo-localization (UAV↔satellite)
+- Handles altitude variations (<1km)
+- Multi-scale processing for different GSDs
+- Domain gap (UAV downward vs satellite nadir view)
+- **Critical**: Fails if rotation >45° (handled by G06)
+
+## API Methods
+
+### `align_to_satellite(uav_image: np.ndarray, satellite_tile: np.ndarray) -> Optional[AlignmentResult]`
+
+**Description**: Aligns UAV image to satellite tile, returning GPS location.
+
+**Called By**:
+- G06 Image Rotation Manager (during rotation sweep)
+- G11 Failure Recovery Coordinator (progressive search)
+- Main processing loop (drift correction with single tile)
+
+**Input**:
+```python
+uav_image: np.ndarray  # Pre-rotated UAV image
+satellite_tile: np.ndarray  # Reference satellite tile
+```
+
+**Output**:
+```python
+AlignmentResult:
+    matched: bool
+    homography: np.ndarray  # 3×3 transformation matrix
+    gps_center: GPSPoint  # UAV image center GPS
+    confidence: float
+    inlier_count: int
+    total_correspondences: int
+```
+
+**Processing Flow**:
+1. Extract features from both images using LiteSAM encoder
+2. Compute dense correspondence field
+3. Estimate homography from correspondences
+4. Validate match quality (inlier count, reprojection error)
+5. If valid match:
+   - Extract GPS from homography
+   - Return AlignmentResult
+6. If no match:
+   - Return None
+
+**Match Criteria**:
+- **Good match**: inlier_count > 30, confidence > 0.7
+- **Weak match**: inlier_count 15-30, confidence 0.5-0.7
+- **No match**: inlier_count < 15
+
+**Error Conditions**:
+- Returns `None`: No match found, rotation >45° (should be pre-rotated)
+
+**Test Cases**:
+1. **Good alignment**: Returns GPS within 20m of ground truth
+2. **Altitude variation**: Handles GSD mismatch
+3. **Rotation >45°**: Fails (by design, requires pre-rotation)
+4. **Multi-scale**: Processes at multiple scales
+
+---
+
+### `compute_homography(uav_image: np.ndarray, satellite_tile: np.ndarray) -> Optional[np.ndarray]`
+
+**Description**: Computes homography transformation from UAV to satellite.
+
+**Called By**:
+- Internal (during align_to_satellite)
+
+**Input**:
+```python
+uav_image: np.ndarray
+satellite_tile: np.ndarray
+```
+
+**Output**:
+```python
+Optional[np.ndarray]: 3×3 homography matrix or None
+```
+
+**Algorithm (LiteSAM)**:
+1. Extract multi-scale features using TAIFormer
+2. Compute correlation via Convolutional Token Mixer (CTM)
+3. Generate dense correspondences
+4. Estimate homography using RANSAC
+5. Refine with non-linear optimization
+
+**Homography Properties**:
+- Maps pixels from UAV image to satellite image
+- Accounts for: scale, rotation, perspective
+- 8 DoF (degrees of freedom)
+
+**Error Conditions**:
+- Returns `None`: Insufficient correspondences
+
+**Test Cases**:
+1. **Valid correspondence**: Returns 3×3 matrix
+2. **Insufficient features**: Returns None
+
+---
+
+### `extract_gps_from_alignment(homography: np.ndarray, tile_bounds: TileBounds, image_center: Tuple[int, int]) -> GPSPoint`
+
+**Description**: Extracts GPS coordinates from homography and tile georeferencing.
+
+**Called By**:
+- Internal (during align_to_satellite)
+- G06 Image Rotation Manager (for precise angle calculation)
+
+**Input**:
+```python
+homography: np.ndarray  # 3×3 matrix
+tile_bounds: TileBounds  # GPS bounds of satellite tile
+image_center: Tuple[int, int]  # Center pixel of UAV image
+```
+
+**Output**:
+```python
+GPSPoint:
+    lat: float
+    lon: float
+```
+
+**Algorithm**:
+1. Apply homography to UAV image center point
+2. Get pixel coordinates in satellite tile
+3. Convert satellite pixel to GPS using tile_bounds and GSD
+4. Return GPS coordinates
+
+**Uses**: G04 Satellite Data Manager for tile_bounds, H02 GSD Calculator
+
+**Test Cases**:
+1. **Center alignment**: UAV center → correct GPS
+2. **Corner alignment**: UAV corner → correct GPS
+3. **Multiple points**: All points consistent
+
+---
+
+### `compute_match_confidence(alignment: AlignmentResult) -> float`
+
+**Description**: Computes match confidence score from alignment quality.
+
+**Called By**:
+- Internal (during align_to_satellite)
+- G11 Failure Recovery Coordinator (to decide if match acceptable)
+
+**Input**:
+```python
+alignment: AlignmentResult
+```
+
+**Output**:
+```python
+float: Confidence score (0.0 to 1.0)
+```
+
+**Confidence Factors**:
+1. **Inlier ratio**: inliers / total_correspondences
+2. **Inlier count**: Absolute number of inliers
+3. **Reprojection error**: Mean error of inliers (in pixels)
+4. **Spatial distribution**: Inliers well-distributed vs clustered
+
+**Thresholds**:
+- **High confidence (>0.8)**: inlier_ratio > 0.6, inlier_count > 50, MRE < 0.5px
+- **Medium confidence (0.5-0.8)**: inlier_ratio > 0.4, inlier_count > 30
+- **Low confidence (<0.5)**: Reject match
+
+**Test Cases**:
+1. **Good match**: confidence > 0.8
+2. **Weak match**: confidence 0.5-0.7
+3. **Poor match**: confidence < 0.5
+
+## Integration Tests
+
+### Test 1: Single Tile Drift Correction
+1. Load UAV image and expected satellite tile
+2. Pre-rotate UAV image to known heading
+3. align_to_satellite() → returns GPS
+4. Verify GPS within 20m of ground truth
+
+### Test 2: Progressive Search (4 tiles)
+1. Load UAV image from sharp turn
+2. Get 2×2 tile grid from G04
+3. align_to_satellite() for each tile
+4. First 3 tiles: No match
+5. 4th tile: Match found → GPS extracted
+
+### Test 3: Rotation Sensitivity
+1. Rotate UAV image by 60° (not pre-rotated)
+2. align_to_satellite() → returns None (fails as expected)
+3. Pre-rotate to 60°
+4. align_to_satellite() → succeeds
+
+### Test 4: Multi-Scale Robustness
+1. UAV at 500m altitude (GSD=0.1m/pixel)
+2. Satellite at zoom 19 (GSD=0.3m/pixel)
+3. LiteSAM handles scale difference → match succeeds
+
+## Non-Functional Requirements
+
+### Performance
+- **align_to_satellite**: ~60ms per tile (TensorRT optimized)
+- **Progressive search 25 tiles**: ~1.5 seconds total (25 × 60ms)
+- Meets <5s per frame requirement
+
+### Accuracy
+- **GPS accuracy**: 60% of frames < 20m error, 80% < 50m error
+- **Mean Reprojection Error (MRE)**: < 1.0 pixels
+- **Alignment success rate**: > 95% when rotation correct
+
+### Reliability
+- Graceful failure when no match
+- Robust to altitude variations (<1km)
+- Handles seasonal appearance changes (to extent possible)
+
+## Dependencies
+
+### Internal Components
+- **G15 Model Manager**: For LiteSAM model
+- **G04 Satellite Data Manager**: For tile_bounds and GSD
+- **H01 Camera Model**: For projection operations
+- **H02 GSD Calculator**: For coordinate transformations
+- **H05 Performance Monitor**: For timing
+
+### External Dependencies
+- **LiteSAM**: Cross-view matching model
+- **opencv-python**: Homography estimation
+- **numpy**: Matrix operations
+
+## Data Models
+
+### AlignmentResult
+```python
+class AlignmentResult(BaseModel):
+    matched: bool
+    homography: np.ndarray  # (3, 3)
+    gps_center: GPSPoint
+    confidence: float
+    inlier_count: int
+    total_correspondences: int
+    reprojection_error: float  # Mean error in pixels
+```
+
+### GPSPoint
+```python
+class GPSPoint(BaseModel):
+    lat: float
+    lon: float
+```
+
+### TileBounds
+```python
+class TileBounds(BaseModel):
+    nw: GPSPoint
+    ne: GPSPoint
+    sw: GPSPoint
+    se: GPSPoint
+    center: GPSPoint
+    gsd: float  # Ground Sampling Distance (m/pixel)
+```
+
+### LiteSAMConfig
+```python
+class LiteSAMConfig(BaseModel):
+    model_path: str
+    confidence_threshold: float = 0.7
+    min_inliers: int = 15
+    max_reprojection_error: float = 2.0  # pixels
+    multi_scale_levels: int = 3
+```
+