mirror of
https://github.com/azaion/gps-denied-desktop.git
synced 2026-04-23 03:06:36 +00:00
add tests
gen_tests updated solution.md updated
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,291 @@
|
||||
# Integration Test: Image Rotation Manager
|
||||
|
||||
## Summary
|
||||
Validate the Image Rotation Manager component for detecting and correcting image orientation issues from non-stabilized UAV camera.
|
||||
|
||||
## Component Under Test
|
||||
**Component**: Image Rotation Manager
|
||||
**Location**: `gps_denied_06_image_rotation_manager`
|
||||
**Dependencies**:
|
||||
- Image Input Pipeline
|
||||
- EXIF metadata reading
|
||||
- OpenCV rotation functions
|
||||
- Configuration Manager
|
||||
|
||||
## Detailed Description
|
||||
This test validates that the Image Rotation Manager can:
|
||||
1. Detect rotation/orientation from EXIF metadata
|
||||
2. Detect rotation from image content analysis (horizon detection)
|
||||
3. Apply rotation correction efficiently
|
||||
4. Handle pitch and roll variations from wing-type UAV
|
||||
5. Maintain image quality during rotation
|
||||
6. Estimate rotation angle for images without metadata
|
||||
7. Handle edge cases (upside-down, 90° rotations)
|
||||
8. Coordinate with vision algorithms that expect specific orientations
|
||||
|
||||
Since the UAV camera is "pointing downwards and fixed, but not auto-stabilized" (per restrictions), images may have varying orientations due to aircraft banking and pitch during flight, especially during turns.
|
||||
|
||||
## Input Data
|
||||
|
||||
### Test Case 1: EXIF Orientation
|
||||
- **Image**: AD000001.jpg
|
||||
- **Check**: EXIF orientation tag if present
|
||||
- **Expected**: Detect and report orientation metadata
|
||||
|
||||
### Test Case 2: Straight Flight (No Rotation)
|
||||
- **Images**: AD000001-AD000010 (straight segment)
|
||||
- **Expected**: Little to no rotation needed (< 5°)
|
||||
|
||||
### Test Case 3: Before/After Sharp Turn
|
||||
- **Images**: AD000042, AD000044 (sharp turn, skip AD000043)
|
||||
- **Expected**: Possible rotation difference due to banking
|
||||
|
||||
### Test Case 4: Outlier Image
|
||||
- **Image**: AD000048 (after 268m jump)
|
||||
- **Expected**: May have significant rotation due to aircraft tilt
|
||||
|
||||
### Test Case 5: Content-Based Rotation Detection
|
||||
- **Image**: AD000025.jpg (mid-route)
|
||||
- **Method**: Horizon detection or feature alignment
|
||||
- **Expected**: Estimate rotation angle without metadata
|
||||
|
||||
### Test Case 6: Rotation Correction Application
|
||||
- **Image**: Artificially rotated AD000010.jpg by known angles (15°, 30°, 45°, 90°)
|
||||
- **Expected**: Detect and correct rotation to within 2°
|
||||
|
||||
### Test Case 7: Batch Processing
|
||||
- **Images**: All 60 images
|
||||
- **Expected**: Process all images, identify which need rotation correction
|
||||
|
||||
### Test Case 8: Extreme Rotation
|
||||
- **Image**: AD000015.jpg rotated 180° (upside-down)
|
||||
- **Expected**: Detect and correct significant rotation
|
||||
|
||||
## Expected Output
|
||||
|
||||
For each test case:
|
||||
```json
|
||||
{
|
||||
"success": true/false,
|
||||
"image_path": "path/to/image.jpg",
|
||||
"rotation_detected": true/false,
|
||||
"rotation_angle_deg": <float>,
|
||||
"rotation_source": "exif|content_analysis|none",
|
||||
"confidence": <float 0-1>,
|
||||
"rotation_applied": true/false,
|
||||
"corrected_image_path": "path/to/corrected_image.jpg",
|
||||
"processing_time_ms": <float>,
|
||||
"quality_metrics": {
|
||||
"sharpness_before": <float>,
|
||||
"sharpness_after": <float>
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
|
||||
**Test Case 1 (EXIF)**:
|
||||
- success = true
|
||||
- EXIF orientation read if present
|
||||
- Reported correctly
|
||||
- processing_time_ms < 50
|
||||
|
||||
**Test Case 2 (Straight Flight)**:
|
||||
- All images show rotation_angle_deg < 5°
|
||||
- rotation_applied = false or minimal correction
|
||||
- Consistent orientations across sequence
|
||||
|
||||
**Test Case 3 (Sharp Turn)**:
|
||||
- Rotation angles detected if present
|
||||
- Banking during turn may cause rotation
|
||||
- Both images processed successfully
|
||||
|
||||
**Test Case 4 (Outlier)**:
|
||||
- Image processed successfully
|
||||
- Rotation detected if present
|
||||
- No crash despite unusual geometry
|
||||
|
||||
**Test Case 5 (Content-Based)**:
|
||||
- Rotation angle estimated (confidence > 0.6)
|
||||
- Estimate within ±5° of ground truth if available
|
||||
- processing_time_ms < 500
|
||||
|
||||
**Test Case 6 (Correction)**:
|
||||
- All artificially rotated images corrected
|
||||
- Corrected angle within ±2° of target (0°)
|
||||
- Image quality preserved (PSNR > 25dB)
|
||||
- 90° rotations detected and corrected exactly
|
||||
|
||||
**Test Case 7 (Batch)**:
|
||||
- All 60 images processed
|
||||
- Statistics reported (mean rotation, max rotation)
|
||||
- Total processing time < 30 seconds
|
||||
- No crashes or failures
|
||||
|
||||
**Test Case 8 (Extreme)**:
|
||||
- 180° rotation detected (confidence > 0.8)
|
||||
- Correction applied successfully
|
||||
- Resulting image is right-side-up
|
||||
|
||||
## Maximum Expected Time
|
||||
- **EXIF reading**: < 50ms
|
||||
- **Content-based detection**: < 500ms
|
||||
- **Rotation application**: < 200ms
|
||||
- **Batch 60 images**: < 30 seconds
|
||||
- **Total test suite**: < 60 seconds
|
||||
|
||||
## Test Execution Steps
|
||||
|
||||
1. **Setup Phase**:
|
||||
a. Initialize Image Rotation Manager
|
||||
b. Load test images
|
||||
c. Create artificially rotated test images for Test Case 6
|
||||
d. Configure detection parameters
|
||||
|
||||
2. **Test Case 1 - EXIF**:
|
||||
a. Load AD000001.jpg
|
||||
b. Read EXIF orientation tag
|
||||
c. Report findings
|
||||
d. Measure timing
|
||||
|
||||
3. **Test Case 2 - Straight Flight**:
|
||||
a. Process AD000001-AD000010
|
||||
b. Detect rotation for each
|
||||
c. Verify low rotation angles
|
||||
d. Check consistency
|
||||
|
||||
4. **Test Case 3 - Sharp Turn**:
|
||||
a. Process AD000042, AD000044
|
||||
b. Compare rotation between images
|
||||
c. Check for banking-induced rotation
|
||||
d. Verify successful processing
|
||||
|
||||
5. **Test Case 4 - Outlier**:
|
||||
a. Process AD000048
|
||||
b. Handle any unusual orientation
|
||||
c. Verify robustness
|
||||
d. Check error handling
|
||||
|
||||
6. **Test Case 5 - Content-Based**:
|
||||
a. Process AD000025.jpg
|
||||
b. Apply content-based rotation detection
|
||||
c. Estimate rotation angle
|
||||
d. Validate confidence score
|
||||
|
||||
7. **Test Case 6 - Correction**:
|
||||
a. Load artificially rotated images
|
||||
b. Detect rotation
|
||||
c. Apply correction
|
||||
d. Verify accuracy
|
||||
e. Check image quality
|
||||
|
||||
8. **Test Case 7 - Batch**:
|
||||
a. Process all 60 images
|
||||
b. Collect statistics
|
||||
c. Identify outliers
|
||||
d. Report summary
|
||||
|
||||
9. **Test Case 8 - Extreme**:
|
||||
a. Load 180° rotated image
|
||||
b. Detect rotation
|
||||
c. Apply correction
|
||||
d. Verify result
|
||||
|
||||
## Pass/Fail Criteria
|
||||
|
||||
**Overall Test Passes If**:
|
||||
- All test cases meet their success criteria
|
||||
- No crashes on any image
|
||||
- Rotation detection accuracy > 85%
|
||||
- Rotation correction accuracy within ±3°
|
||||
- Image quality maintained (PSNR > 25dB)
|
||||
|
||||
**Test Fails If**:
|
||||
- Any image causes crash
|
||||
- Rotation detection fails on obviously rotated images
|
||||
- Correction introduces artifacts or quality loss > 5dB
|
||||
- Processing time exceeds 2x maximum expected
|
||||
- False positive rate > 20% (detecting rotation when none exists)
|
||||
|
||||
## Additional Validation
|
||||
|
||||
**Rotation Detection Methods**:
|
||||
1. **EXIF Tag**: Quickest, but may not be present
|
||||
2. **Horizon Detection**: Use Hough lines to find horizon
|
||||
3. **Feature Alignment**: Compare to reference orientation
|
||||
4. **Gravity Vector**: Use vanishing points (vertical structures)
|
||||
|
||||
**Accuracy Testing**:
|
||||
For Test Case 6, create rotated versions at:
|
||||
- 0° (control)
|
||||
- 5°, 10°, 15° (small rotations)
|
||||
- 30°, 45° (moderate rotations)
|
||||
- 90°, 180°, 270° (cardinal rotations)
|
||||
- -15°, -30° (counter-clockwise)
|
||||
|
||||
Measure detection and correction accuracy for each.
|
||||
|
||||
**Quality Preservation**:
|
||||
- **Interpolation Method**: Use bicubic or lanczos for high quality
|
||||
- **Border Handling**: Crop or pad appropriately
|
||||
- **Metadata Preservation**: Maintain EXIF data in corrected image
|
||||
|
||||
**Performance Optimization**:
|
||||
- Content-based detection should be skippable if EXIF reliable
|
||||
- GPU acceleration for batch rotation if available
|
||||
- Parallel processing for batch operations
|
||||
|
||||
**Edge Cases**:
|
||||
1. **No Clear Horizon**: Agricultural fields may have no clear horizon - should handle gracefully
|
||||
2. **All-Sky Image**: If camera points up instead of down (error case)
|
||||
3. **Blank/Dark Image**: Insufficient features for detection
|
||||
4. **High Texture Image**: Many features but no clear orientation cues
|
||||
|
||||
**Integration with Vision Pipeline**:
|
||||
- SuperPoint should work regardless of minor rotation
|
||||
- LightGlue should match features despite rotation
|
||||
- AnyLoc DINOv2 features should be rotation-invariant
|
||||
- LiteSAM may benefit from consistent orientation
|
||||
|
||||
**False Positive Analysis**:
|
||||
Test images that are correctly oriented:
|
||||
- Should not detect false rotation
|
||||
- Should report confidence < 0.5 for "no rotation needed"
|
||||
- Should not apply unnecessary corrections
|
||||
|
||||
**Rotation Statistics for Full Dataset**:
|
||||
Expected findings:
|
||||
- Mean rotation angle: < 5° (mostly straight)
|
||||
- Max rotation angle: ~15-30° during banking
|
||||
- Images needing correction: < 30%
|
||||
- Straight segments (01-30): minimal rotation
|
||||
- Turn segments (42-48): possible higher rotation
|
||||
|
||||
## Camera Characteristics
|
||||
|
||||
Given UAV camera is:
|
||||
- "pointing downwards and fixed"
|
||||
- "not auto-stabilized"
|
||||
- Wing-type UAV (banks to turn)
|
||||
|
||||
Expected rotation scenarios:
|
||||
- **Roll**: Banking during turns (up to ±30°)
|
||||
- **Pitch**: Climbing/descending (usually minimal)
|
||||
- **Yaw**: Heading change (doesn't affect image rotation)
|
||||
|
||||
Most critical: **Roll detection and correction**
|
||||
|
||||
## Validation Against Ground Truth
|
||||
|
||||
If GPS coordinates and consecutive images are used:
|
||||
- Direction of flight can be inferred
|
||||
- Expected "up" direction in image can be estimated
|
||||
- Validate detected rotation against flight path geometry
|
||||
|
||||
## Performance Benchmarks
|
||||
|
||||
- 1000 EXIF reads in < 5 seconds
|
||||
- 100 content-based detections in < 30 seconds
|
||||
- 100 rotation applications in < 10 seconds
|
||||
- Memory usage per image < 100MB
|
||||
|
||||
Reference in New Issue
Block a user