mirror of
https://github.com/azaion/gps-denied-desktop.git
synced 2026-04-22 09:26:37 +00:00
update metodology, add claude solution draft
This commit is contained in:
Oleksandr Bezdieniezhnykh
Vendored
BIN
Binary file not shown.
@@ -3,6 +3,7 @@
|
||||
- The flying range is restricted by eastern and southern part of Ukraine (To the left of Dnipro river)
|
||||
- The image resolution could be from FullHd to 6252*4168
|
||||
- Altitude is prefefined and no more than 1km
|
||||
- There is NO data from IMU
|
||||
- Flights are done mostly in sunny weather
|
||||
- We can use satellite providers, but we're limited right now to Google Maps, which could be possibly outdated for some regions
|
||||
- Number of photos could be up to 3000, usually in 500-1500 range
|
||||
|
||||
@@ -0,0 +1,559 @@
|
||||
# UAV Aerial Image Geolocation System - Solution Draft
|
||||
|
||||
## 1. Product Solution Description
|
||||
|
||||
### Overview
|
||||
The system is a **hybrid Visual Odometry + Cross-View Matching pipeline** for GPS-denied aerial image geolocation. It combines:
|
||||
- **Incremental Visual Odometry (VO)** for relative pose estimation between consecutive frames
|
||||
- **Periodic Satellite Map Registration** to correct accumulated drift
|
||||
- **Structure from Motion (SfM)** for trajectory refinement
|
||||
- **Deep Learning-based Cross-View Matching** for absolute geolocation
|
||||
|
||||
### Core Components
|
||||
|
||||
#### 1.1 Visual Odometry Pipeline
|
||||
Modern visual odometry approaches for UAVs use downward-facing cameras to track motion by analyzing changes in feature positions between consecutive frames, with correction methods using satellite imagery to reduce accumulated error.
|
||||
|
||||
**Key Features:**
|
||||
- Monocular camera with planar ground assumption
|
||||
- Feature tracking using modern deep learning approaches
|
||||
- Scale recovery using altitude information (≤1km)
|
||||
- Drift correction via satellite image matching
|
||||
|
||||
#### 1.2 Cross-View Matching Engine
|
||||
Cross-view geolocation matches aerial UAV images with georeferenced satellite images through coarse-to-fine matching stages, using deep learning networks to handle scale and illumination differences.
|
||||
|
||||
**Workflow:**
|
||||
1. **Coarse Matching**: Global descriptor extraction (NetVLAD) to find candidate regions
|
||||
2. **Fine Matching**: Local feature matching within candidates
|
||||
3. **Pose Estimation**: Homography/EPnP+RANSAC for geographic pose
|
||||
|
||||
#### 1.3 Structure from Motion (SfM)
|
||||
Structure from Motion uses multiple overlapping images to reconstruct 3D structure and camera poses, automatically performing camera calibration and requiring only 60% vertical overlap between images.
|
||||
|
||||
**Implementation:**
|
||||
- Bundle adjustment for trajectory optimization
|
||||
- Incremental reconstruction for online processing
|
||||
- Multi-view stereo for terrain modeling (optional)
|
||||
|
||||
## 2. Architecture Approach
|
||||
|
||||
### 2.1 System Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Input Layer │
|
||||
│ - Sequential UAV Images (500-3000) │
|
||||
│ - Starting GPS Coordinates │
|
||||
│ - Flight Metadata (altitude, camera params) │
|
||||
└──────────────────┬──────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────▼──────────────────────────────────────────┐
|
||||
│ Feature Extraction Module │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ Primary: SuperPoint + LightGlue (GPU) │ │
|
||||
│ │ Fallback: SIFT + FLANN (CPU) │ │
|
||||
│ │ Target: 1024-2048 keypoints/image │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
└──────────────────┬──────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────▼──────────────────────────────────────────┐
|
||||
│ Sequential Processing Pipeline │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ 1. Visual Odometry Tracker │ │
|
||||
│ │ - Frame-to-frame matching │ │
|
||||
│ │ - Relative pose estimation │ │
|
||||
│ │ - Scale recovery (altitude) │ │
|
||||
│ │ - Outlier detection (350m check) │ │
|
||||
│ └──────────────┬─────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────────────▼─────────────────────────┐ │
|
||||
│ │ 2. Incremental SfM (COLMAP-based) │ │
|
||||
│ │ - Bundle adjustment every N frames │ │
|
||||
│ │ - Track management │ │
|
||||
│ │ - Camera pose refinement │ │
|
||||
│ └──────────────┬─────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────────────▼─────────────────────────┐ │
|
||||
│ │ 3. Satellite Registration Module │ │
|
||||
│ │ - Triggered every 10-20 frames │ │
|
||||
│ │ - Cross-view matching │ │
|
||||
│ │ - Drift correction │ │
|
||||
│ │ - GPS coordinate assignment │ │
|
||||
│ └──────────────┬─────────────────────────┘ │
|
||||
└─────────────────┼─────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────────▼─────────────────────────────────────────┐
|
||||
│ Fallback & Quality Control │
|
||||
│ - Sharp turn detection (overlap <5%) │
|
||||
│ - User intervention request (<20% failure cases) │
|
||||
│ - Quality metrics logging (MRE, registration rate) │
|
||||
└─────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────────▼─────────────────────────────────────────┐
|
||||
│ Output Layer │
|
||||
│ - GPS coordinates for each image center │
|
||||
│ - 6-DoF camera poses │
|
||||
│ - Confidence scores │
|
||||
│ - Sparse 3D point cloud │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 2.2 Technical Implementation
|
||||
|
||||
#### Feature Extraction & Matching
|
||||
LightGlue provides efficient local feature matching with adaptive inference, processing at 150 FPS for 1024 keypoints and outperforming SuperGlue in both speed and accuracy, making it suitable for real-time applications.
|
||||
|
||||
**Primary Stack:**
|
||||
- **Feature Detector**: SuperPoint (256-D descriptors, rotation invariant)
|
||||
- **Feature Matcher**: LightGlue (adaptive inference, early termination)
|
||||
- **Alternative**: DISK + LightGlue for better outdoor performance
|
||||
|
||||
**Configuration:**
|
||||
```python
|
||||
# SuperPoint + LightGlue configuration
|
||||
extractor = SuperPoint(max_num_keypoints=1024)
|
||||
matcher = LightGlue(
|
||||
features='superpoint',
|
||||
depth_confidence=0.9,
|
||||
width_confidence=0.95,
|
||||
flash_attention=True # 4-10x speedup
|
||||
)
|
||||
```
|
||||
|
||||
#### Visual Odometry Component
|
||||
Visual odometry for high-altitude flights often assumes locally flat ground and solves motion through planar homography between ground images, with the scale determined by vehicle elevation.
|
||||
|
||||
**Method:**
|
||||
1. Extract features from consecutive frames (i, i+1)
|
||||
2. Match features using LightGlue
|
||||
3. Apply RANSAC for outlier rejection
|
||||
4. Compute essential matrix
|
||||
5. Recover relative pose (R, t)
|
||||
6. Scale using altitude: `scale = altitude / focal_length`
|
||||
7. Update trajectory
|
||||
|
||||
**Outlier Handling:**
|
||||
- Distance check: reject if displacement >350m between consecutive frames
|
||||
- Overlap check: require >5% feature overlap or trigger satellite matching
|
||||
- Angle threshold: <50° rotation between frames
|
||||
|
||||
#### Cross-View Satellite Matching
|
||||
Cross-view geolocation uses transformers with self-attention and cross-attention mechanisms to match drone images with satellite imagery, employing coarse-to-fine strategies with global descriptors like NetVLAD.
|
||||
|
||||
**Architecture:**
|
||||
```
|
||||
Offline Preparation:
|
||||
1. Download Google Maps tiles for flight region
|
||||
2. Build spatial quad-tree index
|
||||
3. Extract NetVLAD global descriptors (4096-D)
|
||||
4. Store in efficient retrieval database
|
||||
|
||||
Online Processing (every 10-20 frames):
|
||||
1. Extract global descriptor from current aerial image
|
||||
2. Retrieve top-K candidates (K=5-10) using L2 distance
|
||||
3. Fine matching using local features (SuperPoint+LightGlue)
|
||||
4. Homography estimation with RANSAC
|
||||
5. GPS coordinate calculation
|
||||
6. Apply correction to trajectory
|
||||
```
|
||||
|
||||
#### Bundle Adjustment
|
||||
COLMAP provides incremental Structure-from-Motion with automatic camera calibration and bundle adjustment, reconstructing 3D structure and camera poses from overlapping images.
|
||||
|
||||
**Strategy:**
|
||||
- **Local BA**: Every 20 frames (maintain <2s processing time)
|
||||
- **Global BA**: After every 100 frames or satellite correction
|
||||
- **Fixed Parameters**: Altitude constraint, camera intrinsics (if known)
|
||||
- **Optimization**: Ceres Solver with Levenberg-Marquardt
|
||||
|
||||
### 2.3 Meeting Acceptance Criteria
|
||||
|
||||
| Criterion | Implementation Strategy |
|
||||
|-----------|------------------------|
|
||||
| 80% within 50m accuracy | VO + Satellite correction every 10-20 frames |
|
||||
| 60% within 20m accuracy | Fine-tuned cross-view matching + bundle adjustment |
|
||||
| Handle 350m outliers | RANSAC outlier rejection + distance threshold |
|
||||
| Handle sharp turns (<5% overlap) | Trigger satellite matching, skip VO |
|
||||
| <10% satellite outliers | Confidence scoring + verification matches |
|
||||
| User fallback (20% cases) | Automatic detection + GUI for manual GPS input |
|
||||
| <2 seconds per image | GPU acceleration, adaptive LightGlue, parallel processing |
|
||||
| >95% registration rate | Robust feature matching + multiple fallback strategies |
|
||||
| MRE <1.0 pixels | Iterative bundle adjustment + outlier filtering |
|
||||
|
||||
### 2.4 Technology Stack
|
||||
|
||||
**Core Libraries:**
|
||||
- **COLMAP**: SfM and bundle adjustment
|
||||
- **Kornia/PyTorch**: Deep learning feature extraction/matching
|
||||
- **OpenCV**: Image processing and classical CV
|
||||
- **NumPy/SciPy**: Numerical computations
|
||||
- **GDAL**: Geospatial data handling
|
||||
|
||||
**Recommended Hardware:**
|
||||
- **CPU**: 8+ cores (Intel i7/AMD Ryzen 7)
|
||||
- **GPU**: NVIDIA RTX 3080 or better (12GB+ VRAM)
|
||||
- **RAM**: 32GB minimum
|
||||
- **Storage**: SSD for fast I/O
|
||||
|
||||
## 3. Testing Strategy
|
||||
|
||||
### 3.1 Functional Testing
|
||||
|
||||
#### 3.1.1 Feature Extraction & Matching Tests
|
||||
**Objective**: Verify robust feature detection and matching
|
||||
|
||||
**Test Cases:**
|
||||
1. **Varied Illumination**
|
||||
- Sunny conditions (baseline)
|
||||
- Overcast conditions
|
||||
- Shadow-heavy areas
|
||||
- Different times of day
|
||||
|
||||
2. **Terrain Variations**
|
||||
- Urban areas (buildings, roads)
|
||||
- Rural areas (fields, forests)
|
||||
- Mixed terrain
|
||||
- Water bodies
|
||||
|
||||
3. **Image Quality**
|
||||
- FullHD (1920×1080)
|
||||
- 4K (3840×2160)
|
||||
- Maximum resolution (6252×4168)
|
||||
- Simulated motion blur
|
||||
|
||||
**Metrics:**
|
||||
- Number of keypoints detected per image
|
||||
- Matching ratio (inliers/total matches)
|
||||
- Repeatability score
|
||||
- Processing time per image
|
||||
|
||||
**Tools:**
|
||||
- Custom Python test suite
|
||||
- Benchmark datasets (MegaDepth, HPatches)
|
||||
|
||||
#### 3.1.2 Visual Odometry Tests
|
||||
**Objective**: Validate trajectory estimation accuracy
|
||||
|
||||
**Test Cases:**
|
||||
1. **Normal Flight Path**
|
||||
- Straight line flight (100m spacing)
|
||||
- Gradual turns (>20° overlap)
|
||||
- Consistent altitude
|
||||
|
||||
2. **Challenging Scenarios**
|
||||
- Sharp turns (trigger satellite matching)
|
||||
- Variable altitude (if applicable)
|
||||
- Low-texture areas (fields)
|
||||
- Repetitive structures (urban grid)
|
||||
|
||||
3. **Outlier Handling**
|
||||
- Inject 350m displacement
|
||||
- Non-overlapping consecutive frames
|
||||
- Verify recovery mechanism
|
||||
|
||||
**Metrics:**
|
||||
- Relative pose error (rotation and translation)
|
||||
- Trajectory drift (compared to ground truth)
|
||||
- Recovery time after outlier
|
||||
- Scale estimation accuracy
|
||||
|
||||
#### 3.1.3 Cross-View Matching Tests
|
||||
**Objective**: Ensure accurate satellite registration
|
||||
|
||||
**Test Cases:**
|
||||
1. **Scale Variations**
|
||||
- Different altitudes (500m, 750m, 1000m)
|
||||
- Various GSD (Ground Sample Distance)
|
||||
|
||||
2. **Environmental Changes**
|
||||
- Temporal differences (satellite data age)
|
||||
- Seasonal variations
|
||||
- Construction/development changes
|
||||
|
||||
3. **Geographic Regions**
|
||||
- Test on multiple locations in Eastern/Southern Ukraine
|
||||
- Urban vs rural performance
|
||||
- Different Google Maps update frequencies
|
||||
|
||||
**Metrics:**
|
||||
- Localization accuracy (meters)
|
||||
- Retrieval success rate (top-K candidates)
|
||||
- False positive rate
|
||||
- Processing time per registration
|
||||
|
||||
#### 3.1.4 Integration Tests
|
||||
**Objective**: Validate end-to-end pipeline
|
||||
|
||||
**Test Cases:**
|
||||
1. **Complete Flight Sequences**
|
||||
- Process 500-image dataset
|
||||
- Process 1500-image dataset
|
||||
- Process 3000-image dataset
|
||||
|
||||
2. **User Fallback Mechanism**
|
||||
- Simulate failure cases
|
||||
- Test manual GPS input interface
|
||||
- Verify trajectory continuation
|
||||
|
||||
3. **Sharp Turn Recovery**
|
||||
- Multiple consecutive sharp turns
|
||||
- Recovery after extended non-overlap
|
||||
|
||||
**Metrics:**
|
||||
- Overall GPS accuracy (80% within 50m, 60% within 20m)
|
||||
- Total processing time
|
||||
- User intervention frequency
|
||||
- System stability (memory usage, crashes)
|
||||
|
||||
### 3.2 Non-Functional Testing
|
||||
|
||||
#### 3.2.1 Performance Testing
|
||||
**Objective**: Meet <2 seconds per image requirement
|
||||
|
||||
**Test Scenarios:**
|
||||
1. **Processing Speed**
|
||||
- Measure per-image processing time
|
||||
- Identify bottlenecks (profiling)
|
||||
- Test with different hardware configurations
|
||||
|
||||
2. **Scalability**
|
||||
- 500 images
|
||||
- 1500 images
|
||||
- 3000 images
|
||||
- Monitor memory usage and CPU/GPU utilization
|
||||
|
||||
3. **Optimization**
|
||||
- GPU vs CPU performance
|
||||
- Batch processing efficiency
|
||||
- Parallel processing gains
|
||||
|
||||
**Tools:**
|
||||
- Python cProfile
|
||||
- NVIDIA Nsight
|
||||
- Memory profilers
|
||||
|
||||
**Target Metrics:**
|
||||
- Average: <1.5 seconds per image
|
||||
- 95th percentile: <2.0 seconds per image
|
||||
- Peak memory: <16GB RAM
|
||||
|
||||
#### 3.2.2 Accuracy Testing
|
||||
**Objective**: Validate GPS accuracy requirements
|
||||
|
||||
**Methodology:**
|
||||
1. **Ground Truth Collection**
|
||||
- Use high-accuracy GNSS/RTK measurements
|
||||
- Collect control points throughout flight path
|
||||
- Minimum 50 ground truth points per test flight
|
||||
|
||||
2. **Error Analysis**
|
||||
- Calculate 2D position error for each image
|
||||
- Generate error distribution histograms
|
||||
- Identify systematic errors
|
||||
|
||||
3. **Statistical Validation**
|
||||
- Verify 80% within 50m threshold
|
||||
- Verify 60% within 20m threshold
|
||||
- Calculate RMSE, mean, and median errors
|
||||
|
||||
**Test Flights:**
|
||||
- Minimum 10 different flights
|
||||
- Various conditions (time of day, terrain)
|
||||
- Different regions in operational area
|
||||
|
||||
#### 3.2.3 Robustness Testing
|
||||
**Objective**: Ensure system reliability under adverse conditions
|
||||
|
||||
**Test Cases:**
|
||||
1. **Image Registration Rate**
|
||||
- Target: >95% successful registration
|
||||
- Test with challenging image sequences
|
||||
- Analyze failure modes
|
||||
|
||||
2. **Mean Reprojection Error**
|
||||
- Target: <1.0 pixels
|
||||
- Test bundle adjustment convergence
|
||||
- Verify 3D point quality
|
||||
|
||||
3. **Outlier Detection**
|
||||
- Inject various outlier types
|
||||
- Measure detection rate
|
||||
- Verify no false negatives (missed outliers)
|
||||
|
||||
4. **Satellite Map Quality**
|
||||
- Test with outdated satellite imagery
|
||||
- Regions with limited coverage
|
||||
- Urban development changes
|
||||
|
||||
#### 3.2.4 Stress Testing
|
||||
**Objective**: Test system limits and failure modes
|
||||
|
||||
**Scenarios:**
|
||||
1. **Extreme Conditions**
|
||||
- Maximum 3000 images
|
||||
- Highest resolution (6252×4168)
|
||||
- Extended flight duration
|
||||
|
||||
2. **Resource Constraints**
|
||||
- Limited GPU memory
|
||||
- CPU-only processing
|
||||
- Concurrent processing tasks
|
||||
|
||||
3. **Edge Cases**
|
||||
- All images in same location (no motion)
|
||||
- Completely featureless terrain
|
||||
- Extreme weather effects (if data available)
|
||||
|
||||
### 3.3 Test Data Requirements
|
||||
|
||||
#### 3.3.1 Synthetic Data
|
||||
**Purpose**: Controlled testing environment
|
||||
|
||||
**Generation:**
|
||||
- Simulate flights using game engines (Unreal Engine/Unity)
|
||||
- Generate ground truth poses
|
||||
- Vary parameters (altitude, speed, terrain)
|
||||
- Add realistic noise and artifacts
|
||||
|
||||
#### 3.3.2 Real-World Data
|
||||
**Collection Requirements:**
|
||||
- 10+ flights with ground truth GPS
|
||||
- Diverse terrains (urban, rural, mixed)
|
||||
- Different times of day
|
||||
- Various weather conditions (within restrictions)
|
||||
- Coverage across operational area
|
||||
|
||||
**Annotation:**
|
||||
- Manual verification of GPS coordinates
|
||||
- Quality ratings for each image
|
||||
- Terrain type classification
|
||||
- Known challenging sections
|
||||
|
||||
### 3.4 Continuous Testing Strategy
|
||||
|
||||
#### 3.4.1 Unit Tests
|
||||
- Feature extraction modules
|
||||
- Matching algorithms
|
||||
- Coordinate transformations
|
||||
- Utility functions
|
||||
- >80% code coverage target
|
||||
|
||||
#### 3.4.2 Integration Tests
|
||||
- Component interactions
|
||||
- Data flow validation
|
||||
- Error handling
|
||||
- API consistency
|
||||
|
||||
#### 3.4.3 Regression Tests
|
||||
- Performance benchmarks
|
||||
- Accuracy baselines
|
||||
- Automated on each code change
|
||||
- Prevent degradation
|
||||
|
||||
#### 3.4.4 Test Automation
|
||||
**CI/CD Pipeline:**
|
||||
```yaml
|
||||
Pipeline:
|
||||
1. Code commit
|
||||
2. Unit tests (pytest)
|
||||
3. Integration tests
|
||||
4. Performance benchmarks
|
||||
5. Generate test report
|
||||
6. Deploy if all pass
|
||||
```
|
||||
|
||||
**Tools:**
|
||||
- pytest for Python testing
|
||||
- GitHub Actions / GitLab CI
|
||||
- Docker for environment consistency
|
||||
- Custom validation scripts
|
||||
|
||||
### 3.5 Test Metrics & Success Criteria
|
||||
|
||||
| Metric | Target | Test Method |
|
||||
|--------|--------|-------------|
|
||||
| GPS Accuracy (50m) | 80% | Real flight validation |
|
||||
| GPS Accuracy (20m) | 60% | Real flight validation |
|
||||
| Processing Speed | <2s/image | Performance profiling |
|
||||
| Registration Rate | >95% | Feature matching tests |
|
||||
| MRE | <1.0 pixels | Bundle adjustment analysis |
|
||||
| Outlier Detection | >99% | Synthetic outlier injection |
|
||||
| User Intervention | <20% | Complete flight processing |
|
||||
| System Uptime | >99% | Stress testing |
|
||||
|
||||
### 3.6 Test Documentation
|
||||
|
||||
**Required Documentation:**
|
||||
1. **Test Plan**: Comprehensive testing strategy
|
||||
2. **Test Cases**: Detailed test scenarios and steps
|
||||
3. **Test Data**: Description and location of datasets
|
||||
4. **Test Results**: Logs, metrics, and analysis
|
||||
5. **Bug Reports**: Issue tracking and resolution
|
||||
6. **Performance Reports**: Benchmarking results
|
||||
7. **User Acceptance Testing**: Validation with stakeholders
|
||||
|
||||
### 3.7 Best Practices
|
||||
|
||||
1. **Iterative Testing**: Test early and often throughout development
|
||||
2. **Realistic Data**: Use real flight data as much as possible
|
||||
3. **Version Control**: Track test data and results
|
||||
4. **Reproducibility**: Ensure tests can be replicated
|
||||
5. **Automation**: Automate repetitive tests
|
||||
6. **Monitoring**: Continuous performance tracking
|
||||
7. **Feedback Loop**: Incorporate test results into development
|
||||
|
||||
## 4. Implementation Roadmap
|
||||
|
||||
### Phase 1: Core Development (Weeks 1-4)
|
||||
- Feature extraction pipeline (SuperPoint/LightGlue)
|
||||
- Visual odometry implementation
|
||||
- Basic bundle adjustment integration
|
||||
|
||||
### Phase 2: Cross-View Matching (Weeks 5-8)
|
||||
- Satellite tile download and indexing
|
||||
- NetVLAD descriptor extraction
|
||||
- Coarse-to-fine matching pipeline
|
||||
|
||||
### Phase 3: Integration & Optimization (Weeks 9-12)
|
||||
- End-to-end pipeline integration
|
||||
- Performance optimization (GPU, parallelization)
|
||||
- User fallback interface
|
||||
|
||||
### Phase 4: Testing & Validation (Weeks 13-16)
|
||||
- Comprehensive testing (all test cases)
|
||||
- Real-world validation flights
|
||||
- Performance tuning
|
||||
|
||||
### Phase 5: Deployment (Weeks 17-18)
|
||||
- Documentation
|
||||
- Deployment setup
|
||||
- Training materials
|
||||
|
||||
## 5. Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Google Maps outdated | Multiple satellite sources, manual verification |
|
||||
| GPU unavailable | CPU fallback with SIFT |
|
||||
| Sharp turns | Automatic satellite matching trigger |
|
||||
| Featureless terrain | Reduced keypoint threshold, larger search radius |
|
||||
| Processing time > 2s | Adaptive LightGlue, parallel processing |
|
||||
| Poor lighting | Image enhancement preprocessing |
|
||||
|
||||
## 6. References & Resources
|
||||
|
||||
**Key Papers:**
|
||||
- SuperPoint: Self-Supervised Interest Point Detection and Description (DeTone et al., 2018)
|
||||
- LightGlue: Local Feature Matching at Light Speed (Lindenberger et al., 2023)
|
||||
- CVM-Net: Cross-View Matching Network (Hu et al., 2018)
|
||||
- COLMAP: Structure-from-Motion Revisited (Schönberger et al., 2016)
|
||||
|
||||
**Software & Libraries:**
|
||||
- COLMAP: https://colmap.github.io/
|
||||
- Kornia: https://kornia.readthedocs.io/
|
||||
- Hierarchical Localization: https://github.com/cvg/Hierarchical-Localization
|
||||
- LightGlue: https://github.com/cvg/LightGlue
|
||||
|
||||
This solution provides a robust, scalable approach that meets all acceptance criteria while leveraging state-of-the-art computer vision and deep learning techniques.
|
||||
@@ -1,128 +0,0 @@
|
||||
# UAV Aerial Image Geolocalization System: Improved Solution Draft
|
||||
|
||||
## Executive Summary
|
||||
This improved system addresses all identified weak points in the previous design for UAV-based aerial image geolocalization in GPS-denied scenarios. Key improvements include robust initialization without GPS, mitigation of scale drift, integration of IMU/barometric data, adaptive feature detection, drift suppression through loop closure and global optimization, scalable processing for large datasets, and explicit accuracy validation protocols.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem Analysis & Critical Improvements
|
||||
|
||||
### 1.1 Key Constraints & Challenges (with Mitigation)
|
||||
- **No onboard GPS:** Initialization via visual place recognition or satellite/map-based rough localization. Fallback to user landmark selection if both fail.
|
||||
- **Camera calibration unknown:** Mission begins with field/in-flight self-calibration using geometric patterns; results stored and reused.
|
||||
- **Altitude & scale ambiguity:** Estimate via stereo shadow analysis/barometric sensor if present; continuously refined with satellite anchor points and GCPs.
|
||||
- **Low-texture regions:** Automatic switch to global descriptors or semantic deep features; spatial/temporal priors used for matching.
|
||||
- **Extreme pose/turns or <5% overlap:** Expanded skip/interleaved matching windows; classifier triggers fallback matching when sharp turn detected.
|
||||
- **Outlier scenarios:** Predictive analytics, not just retrospective detection. Early anomaly detection to prevent error propagation.
|
||||
- **Accuracy validation:** Ground truth via surveyed GCPs or pseudo-checkpoints (road intersections, corners) when unavailable. Incorporate empirical validation.
|
||||
- **Satellite API limits:** Batch pre-fetch and use open data portals; avoid hitting commercial API rate limits.
|
||||
|
||||
---
|
||||
|
||||
## 2. State-of-the-Art: Enhanced Benchmarking & Algorithm Selection
|
||||
|
||||
- **Feature extraction:** Benchmark AKAZE, ORB, SIFT, SuperPoint, and select best for context (full-res performance profiled per mission).
|
||||
- **Cross-view matching:** Employ deep learning networks (CVPR2025, HC-Net) for robust aerial/satellite registration, tolerating more domain and season variations.
|
||||
- **Global optimization:** Periodic global or keyframe-based bundle adjustment. Loop closure via NetVLAD-style place recognition suppresses drift.
|
||||
- **Visual-inertial fusion:** Mandatory IMU/barometer integration with visual odometry for scale/orientation stability.
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture: Robust, Drift-Resistant System Design
|
||||
|
||||
### 3.1 Initialization Module
|
||||
- Coarse matching to map/satellite (not GPS), visual landmark picking, or user manual anchor.
|
||||
- Self-calibration procedure; field edges/runway as calibration targets.
|
||||
|
||||
### 3.2 Feature Extraction & Matching Module
|
||||
- Adaptively select the fastest robust algorithm per detected texture/scene.
|
||||
- Deep descriptors/deep semantic matching switch in low-feature areas.
|
||||
|
||||
### 3.3 Sequential & Wide-baseline Matching
|
||||
- Skip/interleaved window strategy during sharp turns/low overlap; classifier to select mode.
|
||||
- Periodic absolute anchors (GCP, satellite, landmark) to pin scale and orientation.
|
||||
|
||||
### 3.4 Pose Estimation & Bundle Adjustment
|
||||
- Visual-inertial fusion for incremental stabilization.
|
||||
- Keyframe-based and periodic global BA; loop closure detection and global optimization.
|
||||
|
||||
### 3.5 Satellite Georeferencing Module
|
||||
- Batch caching and use of non-commercial open source imagery where possible.
|
||||
- Preprocessing to common GSD; deep-learning cross-view registration for robust matching.
|
||||
|
||||
### 3.6 Outlier & Anomaly Detection
|
||||
- Predictive outlier detection—anomaly scores tracked per-frame and alert before severe divergence.
|
||||
|
||||
### 3.7 User Intervention, Feedback, & Incremental Output
|
||||
- User can intervene at any stage with manual correction; preview trajectory and labeled anomalies during flight (not only after full sequence).
|
||||
- Incremental outputs streamed during processing.
|
||||
|
||||
---
|
||||
|
||||
## 4. Testing & Validation Protocols
|
||||
|
||||
### 4.1 Data Collection & Quality Control
|
||||
- Validate calibration and initialization at start by test images against known patterns/landmarks.
|
||||
- Mandate 3–9 accurately surveyed GCPs or pseudo-checkpoints for true accuracy benchmarks.
|
||||
- Run dedicated benchmark flights over controlled areas every development cycle.
|
||||
|
||||
### 4.2 Performance & Scale Testing
|
||||
- Profile all components at mission scale (1000–3000 images); parallelize all viable steps and break datasets into clusters for batch processing.
|
||||
- Use RAM-efficient out-of-core databases for features/trajectories.
|
||||
|
||||
### 4.3 Real-world Edge Case Testing
|
||||
- Low-texture, sharp-turn, water/snow scenarios simulated with edge missions and field datasets.
|
||||
- Outlier detection tested on both synthetic and real injected events; accuracy measured empirically.
|
||||
|
||||
### 4.4 API/Resource Limitation Testing
|
||||
- For satellite imagery, pre-load, regional cache, and batch API keys under compliant usage where necessary. Prefer open repositories for large missions.
|
||||
|
||||
---
|
||||
|
||||
## 5. Module Specifications (Improvements Incorporated)
|
||||
- **Image Preprocessor:** Calibration step at startup and periodic recalibration; correction for lens/altitude uncertainty.
|
||||
- **Feature Matcher:** Profiled selection per context; adapts to low-feature case, deep CNN fallback.
|
||||
- **Pose Solver:** Visual-inertial fusion standard, no monocular-only solution; scale pinned via anchors.
|
||||
- **Bundle Adjuster:** Keyframe-based, periodic global BA; incremental optimization and drift suppression.
|
||||
- **Satellite Module:** Batch requests only; no per-image dependent on commercial rate limits; open imagery preferred.
|
||||
- **Outlier Detector:** Predictive analytics; triggers remediation early versus reactive correction.
|
||||
- **User Interface:** Live streaming of results and anomaly flags; interactive corrections before mission completion.
|
||||
|
||||
---
|
||||
|
||||
## 6. Acceptance & Success Criteria
|
||||
- **Absolute Accuracy:** Validated against GCPs or reference points; not just internal consistency/satellite.
|
||||
- **Robustness:** System continues under extreme conditions; drift suppressed by anchors; predictive outlier recovery.
|
||||
- **Performance:** Latency and scale measured for clusters and full mission; targets empirically validated.
|
||||
|
||||
---
|
||||
|
||||
## 7. Architecture Diagram
|
||||
A revised annotation for the system should include:
|
||||
- Initialization (without GPS)
|
||||
- Self-calibration
|
||||
- Visual-inertial fusion
|
||||
- Adaptive feature extraction
|
||||
- Multi-window matching and global optimization
|
||||
- Deep cross-view registration
|
||||
- Predictive outlier detection
|
||||
- User feedback anytime
|
||||
|
||||
---
|
||||
|
||||
## Summary of Key Fixes Over Previous Draft
|
||||
- No dependence on onboard GPS at any stage
|
||||
- Scale/altitude ambiguity suppressed by periodic anchors/model fusion
|
||||
- Memory and performance scalability included by design
|
||||
- Satellite matching and API rate limits explicitly managed
|
||||
- Empirical validation protocols with external ground truth incorporated
|
||||
- Robustness for low-feature, extreme pose, and unstructured data scenarios
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
Design incorporates field best practices, research findings, and expert recommendations on photogrammetry, visual-inertial navigation, and cross-view localization for GPS-denied UAV missions.
|
||||
|
||||
---
|
||||
|
||||
*This improved document is structured in the same style and covers all problem areas, specifying practical and state-of-the-art solutions for each identified weakness.*
|
||||
@@ -12,9 +12,10 @@
|
||||
- orthophoto images from the UAV for the analysis
|
||||
- list of expected GPS for the centers for each picture in csv format: picture name, lat, lon
|
||||
- video from the UAV for the analysis
|
||||
- list of expected GPS for the centers of video per timeframe in csv format: timestamp, lat, lon for each 1-2 seconds of the
|
||||
- list of expected GPS for the centers of video per timeframe in csv format: timestamp, lat, lon for each 1-2 seconds
|
||||
- ...
|
||||
Analyze very thoroughly input data and form system's restrictions for input data and acceptance ctiteria
|
||||
|
||||
Analyze very thoroughly input data and form system's restrictions and acceptance ctiteria
|
||||
|
||||
- `restrictions.md`: Restrictions we have in real world in the -dashed list format. For example:
|
||||
- We're limiting our solution to airplane type UAVs.
|
||||
@@ -31,89 +32,68 @@
|
||||
|
||||
|
||||
## 1.2 **✨AI Research**: Restrictions and Acceptance Criteria assesment
|
||||
Form the next prompt:
|
||||
- Replace *.md to the corresponding content
|
||||
In the new context form the next prompt:
|
||||
- Add *.md to the context
|
||||
- Add sample files to the prompt
|
||||
- Put proper sample filenames in the text of the prompt
|
||||
and run it in DeepResearch tool (Gemini, ChatGPT, or other)
|
||||
**📝prompt template:**
|
||||
We have this problem:
|
||||
|
||||
`docs/00_problem/problem_description.md`
|
||||
|
||||
System should process this data samples:
|
||||
|
||||
`sample1.jpg`, `sample2.jpg`, `coordinates.csv`, ...
|
||||
|
||||
We have next restrictions for the input data:
|
||||
|
||||
`docs/00_problem/restrictions.md`
|
||||
|
||||
Output of our system should meet these acceptance criteria:
|
||||
|
||||
`docs/00_problem/acceptance_criteria.md`
|
||||
|
||||
and run it in DeepResearch tool (Gemini, DeepSeek, or other)
|
||||
```
|
||||
We have the problem described in `problem_description.md`.
|
||||
- System should process data samples in the attached files (if any). They are for reference only.
|
||||
- Restrictions for the input data are stated in `restrictions.md`.
|
||||
- Acceptance criteria for the output of the system are stated in `acceptance_criteria.md`.
|
||||
Check how realistic these acceptance criteria are.
|
||||
Check how critical is each criterion. Also find out more acceptance criteria for this specific domain.
|
||||
Research impact of each value in acceptance criteria to the whole system quality.
|
||||
Assess acceptable ranges for each value in each acceptance criteria in the state of the art solutions, and propose corrections in the next table:
|
||||
Check how critical each criterion is. Find out more acceptance criteria for this specific domain.
|
||||
Research the impact of each value in the acceptance criteria on the whole system quality.
|
||||
Assess acceptable ranges for each value in each acceptance criterion in the state-of-the-art solutions, and propose corrections in the next table:
|
||||
- Acceptance criterion name
|
||||
- Our values
|
||||
- Your researched criterion values
|
||||
- Status: Is criterion was added by your research to our system, or modified, or removed, cause not necessary
|
||||
Also asses restrictions we've put for the system. Are they realistic? Propose corrections in the next table:
|
||||
- Status: Is the criterion added by your research to our system, modified, or removed
|
||||
Assess the restrictions we've put on the system. Are they realistic? Propose corrections in the next table:
|
||||
- Restriction name
|
||||
- Our values
|
||||
- Your researched restriction values
|
||||
- Status: Is restriction was added by your research to our system, or modified, or removed, cause not necessary
|
||||
- Status: Is a restriction added by your research to our system, modified, or removed
|
||||
```
|
||||
|
||||
**👨💻Developers**: Revise the result, discuss them and overwrite `docs/00_problem/acceptance_criteria.md`
|
||||
|
||||
|
||||
## 1.3 **✨AI Research**: Research the problem in great detail
|
||||
Form the next prompt:
|
||||
- Replace *.md to the corresponding content
|
||||
In the new context form the next prompt:
|
||||
- Add *.md to the context
|
||||
- Add sample files to the prompt
|
||||
- Put proper sample filenames in the text of the prompt
|
||||
and run it in DeepResearch tool (Gemini, ChatGPT, or other)
|
||||
**📝prompt template:**
|
||||
Research this problem:
|
||||
|
||||
`docs/00_problem/problem_description.md`
|
||||
|
||||
System should process this data samples. They are for references only:
|
||||
|
||||
`sample1.jpg`, `sample2.jpg`, `coordinates.csv`
|
||||
|
||||
We have next restrictions for the input data:
|
||||
|
||||
`docs/00_problem/restrictions.md`
|
||||
|
||||
Output of our system should meet these acceptance criteria:
|
||||
|
||||
`docs/00_problem/acceptance_criteria.md`
|
||||
|
||||
Find out all the state of the art solutions for this problem and produce the resulting solution draft in the next format:
|
||||
- Product solution description
|
||||
- Architecture approach that meet restrictions and acceptance criteria
|
||||
- Testing strategy. Research the best approaches to cover all the acceptance criteria, functional and non-functional tests
|
||||
and run it in DeepResearch tool (Gemini, DeepSeek, or other)
|
||||
```
|
||||
Research this problem in `problem_description.md`.
|
||||
- The system should process data samples in attached files (if any). They are for reference only.
|
||||
- We have the next restrictions for the input data in `restrictions.md`.
|
||||
- Output of our system should meet these acceptance criteria in `acceptance_criteria.md`.
|
||||
- Find out all the state-of-the-art solutions for this problem and produce the resulting solution draft in the next format:
|
||||
- Short Product solution description. Brief component interaction diagram.
|
||||
- Architecture approach that meets restrictions and acceptance criteria. Tables to compare possible component decisions.
|
||||
- Testing strategy. Research the best approaches to cover all the acceptance criteria, functional, and non-functional tests
|
||||
Be concise in formulating, The less words the better, but do not miss any important details.
|
||||
```
|
||||
|
||||
**👨💻Developer**: Revise the result from AI. Research the problem as well, and add/modify/remove some solution details in the draft.
|
||||
Store it to the `docs/01_solution/solution_draft_01.md`
|
||||
|
||||
|
||||
## 1.4 **✨AI Research**: Solution draft assessment
|
||||
Form the next prompt by replacing *.md with corresponding content and run it in DeepResearch tool (Gemini, ChatGPT, or other)
|
||||
**📝prompt template:**
|
||||
In the solution here:
|
||||
`docs/01_solution/solution_draft_xx.md`
|
||||
find out all the possible weak points and problems. Address them and find out ways to solve them. Based on your findings, form new solution draft in the same format.
|
||||
Add *.md to the context and run it in DeepResearch tool (Gemini, DeepSeek, or other)
|
||||
```
|
||||
We have a problem here in `problem_description.md` with restrictions `restrictions.md`. We've presented the solution here in `solution_draft_xx.md`.
|
||||
Identify all potential weak points and problems. Address them and find out ways to solve them. Based on your findings, form a new solution draft in the same format without mentioning or comparison with first solution draft. Also, in case of replacing some components with better one, do not just throw away the previous, form a table with comparison for the component.
|
||||
```
|
||||
|
||||
**👨💻Developer**: Research by yourself as well - how to solve additional problems which AI figured out, and add them to the result. Store the result draft to the `docs/01_solution/solution_draft_xx+1.md`, and repeat the process. When the next solution wouldn't differ much from the previous one, store the last draft as `docs/01_solution/solution.md`
|
||||
|
||||
|
||||
## 1.5 **🤖📋AI plan**: Solution Decomposition
|
||||
**📝prompt:**
|
||||
```
|
||||
Decompose the solution `@docs/01_solution/solution.md` to the components.
|
||||
Store description of each component to the file `docs/02_components/[##]_[component_name]/spec.md` with the next structure:
|
||||
- Component Name
|
||||
@@ -141,14 +121,16 @@
|
||||
- StepN - Expected resultN
|
||||
|
||||
Do not put any code yet. Ask as many questions as needed.
|
||||
|
||||
```
|
||||
**👨💻Developer**: Answer the questions AI asked, put as many details as possible
|
||||
|
||||
|
||||
## 1.6 **🤖📋AI plan**: Business Requirement generation
|
||||
```
|
||||
From the initial requirements above generate Jira epics:
|
||||
- Ask the Jira project key and latest created jira task number. For example AZ-412
|
||||
- Access the solution `@docs/01_solution/solution.md` and description of each component in the files `docs/02_components/[##]_[component_name]/spec.md`
|
||||
- Generate Jira Epics from the Components
|
||||
- Ensure each epic has clear goal and acceptance criteria, verify the criteria with `@docs/00_initial/acceptance_criteria.md`
|
||||
- Generate draw.io components diagram based on previous diagram shows relations between components and Jira Epic numbers corresponding to each component.
|
||||
```
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## 2.1 **🤖📋AI plan**: Component Decomposition
|
||||
For each component in `docs/02_components` do next:
|
||||
**📝prompt:**
|
||||
```
|
||||
Decompose `@docs/02_components/[##]_[component_name]/spec.md` to the features. If component is simple enough, make only 1 feature, if complex - separate per features. Feature can contain 0 or more APIs. Create `docs/02_components/[##]_[component_name]/[##]_[feature_name]_feature.md` with the next structure:
|
||||
- Name
|
||||
- Description
|
||||
@@ -14,12 +14,12 @@
|
||||
|
||||
Do NOT generate any code yet, only brief explanations what should be done.
|
||||
Ask as many questions as needed.
|
||||
|
||||
```
|
||||
**👨💻Developer**: Answer the questions AI asked, put as many details as possible
|
||||
|
||||
## 2.2 **🤖AI agent**: Feature implementation
|
||||
For each component in `docs/02_components/[##]_[component_name]/` folder do next:
|
||||
**📝prompt:**
|
||||
```
|
||||
Read component description `@docs/02_components/[##]_[component_name]/spec.md`.
|
||||
Read all features in the folder `@docs/02_components/[##]_[component_name]`. For each feature do next:
|
||||
- Implement it
|
||||
@@ -27,9 +27,10 @@
|
||||
- Create unit tests from the Test cases description, run it and make sure the result is a success
|
||||
- Create integration test for the feature, run and make sure the result is a success
|
||||
If integration tests are specified in component spec, then write them and run, and make sure that component working correctly
|
||||
```
|
||||
|
||||
## 2.3 **🤖AI agent**: Solution composition and integration tests
|
||||
**📝prompt:**
|
||||
```
|
||||
Read all the files here `docs/03_tests/` and for each file write down tests and run it.
|
||||
Compose a final test results in a csv with the next format:
|
||||
- Test filename
|
||||
@@ -38,4 +39,5 @@
|
||||
|
||||
Fix all problems if tests failed until we got a successful result.
|
||||
In case if one or more tests was failed due to missing data from user or API or other system, ask it from developer.
|
||||
Repeat test cycle until no failed tests.
|
||||
Repeat test cycle until no failed tests.
|
||||
```
|
||||
@@ -1,45 +0,0 @@
|
||||
We have this problem:
|
||||
|
||||
We have a lot of images taken from wing-type UAV by camera with at least FullHD resolution. Resolution of each photo could be up to 6200*4100 for the whole flight, but for other flight could be FullHD, f.e.
|
||||
Photos are taken and named consecutively within 100 meters distance between each other.
|
||||
We know only starting GPS coordinates. We need to determine coordinates of centers of each image. And also coordinates of the center of any object on these photos. We can use external satellite provider for ground check the existing photos
|
||||
|
||||
System should process these data samples. They are for references only:
|
||||
|
||||
AD000001.jpg, AD000002.jpg, AD000003.jpg, AD000004.jpg, AD000005.jpg, AD000006.jpg, AD000007.jpg, AD000008.jpg, AD000009.jpg, AD000010.jpg, AD000011.jpg, AD000012.jpg, AD000013.jpg, AD000014.jpg, AD000015.jpg, AD000016.jpg, AD000017.jpg, AD000018.jpg, AD000019.jpg, AD000020.jpg, AD000021.jpg, AD000022.jpg, AD000023.jpg, AD000024.jpg, AD000025.jpg, AD000026.jpg, AD000027.jpg, AD000028.jpg, AD000029.jpg, AD000030.jpg, AD000031.jpg, AD000032.jpg, AD000033.jpg, AD000034.jpg, AD000035.jpg, AD000036.jpg, AD000037.jpg, AD000038.jpg, AD000039.jpg, AD000040.jpg, AD000041.jpg, AD000042.jpg, AD000043.jpg, AD000044.jpg, AD000045.jpg, AD000046.jpg, AD000047.jpg, AD000048.jpg, AD000049.jpg, AD000050.jpg, AD000051.jpg, AD000052.jpg, AD000053.jpg, AD000054.jpg, AD000055.jpg, AD000056.jpg, AD000057.jpg, AD000058.jpg, AD000059.jpg, AD000060.jpg, coordinates.csv
|
||||
|
||||
We have next restrictions for the input data:
|
||||
|
||||
- Photos are taken by only airplane type UAVs.
|
||||
- Photos are taken by the camera pointing downwards and fixed, but it is not autostabilized.
|
||||
- The flying range is restricted by eastern and southern part of Ukraine (To the left of Dnipro river)
|
||||
- The image resolution could be from FullHd to 6252*4168
|
||||
- Altitude is prefefined and no more than 1km
|
||||
- Flights are done mostly in sunny weather
|
||||
- We can use satellite providers, but we're limited right now to Google Maps, which could be possibly outdated for some regions
|
||||
- Number of photos could be up to 3000, usually in 500-1500 range
|
||||
- During the flight UAV can make sharp turns, so that it is possible that next photo is absolutely different from the previous one (no same objects), but it is rather exception than the rule
|
||||
|
||||
|
||||
Output of our system should meet these acceptance criteria:
|
||||
|
||||
- System should find out GPS of centers of 80% of the photos from the flight within error no more than 50 meters in comparison to the real GPS
|
||||
- System should find out GPS of centers of 60% of the photos from the flight within error no more than 10 meters in comparison to the real GPS
|
||||
- System should correctly continue the work even in a presence of up to 350 meters outlier photo between 2 consecutive photos en route. This could happen due to tilt of the plane.
|
||||
- System should correctly continue the work even during sharp turns, where the next photo doesn't overlap at all, or overlaps in less than 5%. Next photo should be in less than 150m drift and angle less than 50%
|
||||
- Number of outliers during the satellite provider images ground check should be less than 10%
|
||||
- In case of absolute incapable of the system to determine next, second next, and third next images gps, by any methods, (these 20% of the route), then it should ask user for an input for the next image, so that user can specify location
|
||||
|
||||
Check how realistic these acceptance criteria are.
|
||||
Check how critical is each criterion. Also find out more acceptance criteria for this specific domain.
|
||||
Research impact of each value in acceptance criteria to the whole system quality.
|
||||
Assess acceptable ranges for each value in each acceptance criteria in the state of the art solutions, and propose corrections in the next table:
|
||||
- Acceptance criterion name
|
||||
- Our values
|
||||
- Your researched criterion values
|
||||
- Status: Is criterion was added by your research to our system, or modified, or removed, cause not necessary
|
||||
Also asses restrictions we've put for the system. Are they realistic? Propose corrections in the next table:
|
||||
- Restriction name
|
||||
- Our values
|
||||
- Your researched restriction values
|
||||
- Status: Is restriction was added by your research to our system, or modified, or removed, cause not necessary
|
||||
@@ -1,39 +0,0 @@
|
||||
Research this problem:
|
||||
|
||||
We have a lot of images taken from wing-type UAV by camera with at least FullHD resolution. Resolution of each photo could be up to 6200*4100 for the whole flight, but for other flight could be FullHD, f.e.
|
||||
Photos are taken and named consecutively within 100 meters distance between each other.
|
||||
We know only starting GPS coordinates. We need to determine coordinates of centers of each image. And also coordinates of the center of any object on these photos. We can use external satellite provider for ground check the existing photos
|
||||
|
||||
System should process these data samples. They are for references only:
|
||||
|
||||
AD000001.jpg, AD000002.jpg, AD000003.jpg, AD000004.jpg, AD000005.jpg, AD000006.jpg, AD000007.jpg, AD000008.jpg, AD000009.jpg, AD000010.jpg, AD000011.jpg, AD000012.jpg, AD000013.jpg, AD000014.jpg, AD000015.jpg, AD000016.jpg, AD000017.jpg, AD000018.jpg, AD000019.jpg, AD000020.jpg, AD000021.jpg, AD000022.jpg, AD000023.jpg, AD000024.jpg, AD000025.jpg, AD000026.jpg, AD000027.jpg, AD000028.jpg, AD000029.jpg, AD000030.jpg, AD000031.jpg, AD000032.jpg, AD000033.jpg, AD000034.jpg, AD000035.jpg, AD000036.jpg, AD000037.jpg, AD000038.jpg, AD000039.jpg, AD000040.jpg, AD000041.jpg, AD000042.jpg, AD000043.jpg, AD000044.jpg, AD000045.jpg, AD000046.jpg, AD000047.jpg, AD000048.jpg, AD000049.jpg, AD000050.jpg, AD000051.jpg, AD000052.jpg, AD000053.jpg, AD000054.jpg, AD000055.jpg, AD000056.jpg, AD000057.jpg, AD000058.jpg, AD000059.jpg, AD000060.jpg, coordinates.csv
|
||||
|
||||
|
||||
We have next restrictions for the input data:
|
||||
|
||||
- Photos are taken by only airplane type UAVs.
|
||||
- Photos are taken by the camera pointing downwards and fixed, but it is not autostabilized.
|
||||
- The flying range is restricted by eastern and southern part of Ukraine (To the left of Dnipro river)
|
||||
- The image resolution could be from FullHd to 6252*4168
|
||||
- Altitude is prefefined and no more than 1km
|
||||
- Flights are done mostly in sunny weather
|
||||
- We can use satellite providers, but we're limited right now to Google Maps, which could be possibly outdated for some regions
|
||||
- Number of photos could be up to 3000, usually in 500-1500 range
|
||||
- During the flight UAV can make sharp turns, so that it is possible that next photo is absolutely different from the previous one (no same objects), but it is rather exception than the rule
|
||||
|
||||
Output of our system should meet these acceptance criteria:
|
||||
|
||||
- System should find out GPS of centers of 80% of the photos from the flight within error no more than 50 meters in comparison to the real GPS
|
||||
- System should find out GPS of centers of 60% of the photos from the flight within error no more than 20 meters in comparison to the real GPS
|
||||
- System should correctly continue the work even in a presence of up to 350 meters outlier photo between 2 consecutive photos en route. This could happen due to tilt of the plane.
|
||||
- System should correctly continue the work even during sharp turns, where the next photo doesn't overlap at all, or overlaps in less than 5%. Next photo should be in less than 150m drift and angle less than 50%
|
||||
- Number of outliers during the satellite provider images ground check should be less than 10%
|
||||
- In case of absolute incapable of the system to determine next, second next, and third next images gps, by any methods, (these 20% of the route), then it should ask user for an input for the next image, so that user can specify location
|
||||
- Less than 2 seconds for processing one image
|
||||
- Image Registration Rate > 95%. System can find enough matching features to confidently calculate the camera's 6-DoF pose (position and orientation) and "stitch" that image into the final trajectory
|
||||
- Mean Reprojection Error (MRE) < 1.0 pixels. The distance, in pixels, between the original pixel location of the object and the re-projected pixel location.
|
||||
|
||||
Find out all the state of the art solutions for this problem and produce the resulting solution draft in the next format:
|
||||
- Product solution description
|
||||
- Architecture approach that meet restrictions and acceptance criteria
|
||||
- Testing strategy. Research the best approaches to cover all the acceptance criteria, functional and non-functional tests
|
||||
Reference in New Issue
Block a user