# Acceptance Test: AC-2 - 60% of Photos < 20m Error

## Summary
Validate Acceptance Criterion 2: "The system should find out the GPS of centers of 60% of the photos from the flight within an error of no more than 20 meters in comparison to the real GPS."

## Linked Acceptance Criteria
**AC-2**: 60% of photos < 20m error

## Preconditions
1. ASTRAL-Next system fully operational
2. All TensorRT models loaded (FP16 precision for maximum accuracy)
3. High-quality satellite tiles cached (Zoom level 19, ~0.30 m/pixel)
4. Ground truth GPS coordinates available
5. Test dataset prepared: Test_Baseline (AD000001-AD000030)

## Test Description
Process same baseline flight as AC-1 test, but now validate the more stringent criterion that at least 60% of images achieve error < 20 meters. This tests the precision of LiteSAM cross-view matching.

## Test Steps

### Step 1: Initialize System for High Precision
- **Action**: Start system, verify models loaded in optimal configuration
- **Expected Result**: System ready, LiteSAM configured for maximum precision

### Step 2: Create Test Flight
- **Action**: Create flight "AC2_HighPrecision" with same parameters as AC-1
- **Expected Result**: Flight created successfully

### Step 3: Upload Test Images
- **Action**: Upload AD000001-AD000030 (30 images)
- **Expected Result**: All queued for processing

### Step 4: Process with High-Quality Anchors
- **Action**: System processes images, L3 provides frequent GPS anchors
- **Expected Result**: 
  - Processing completes
  - Multiple GPS anchors per 10 images
  - Factor graph well-constrained

### Step 5: Retrieve Final Results
- **Action**: GET /flights/{flightId}/results?include_refined=true
- **Expected Result**: Refined GPS coordinates (post-optimization)

### Step 6: Calculate Errors
- **Action**: Calculate haversine distance for each image
- **Expected Result**: Error array with 30 values

### Step 7: Validate AC-2
- **Action**: Count images with error < 20m, calculate percentage
- **Expected Result**: **≥ 60% of images have error < 20 meters** ✓

### Step 8: Analyze High-Precision Results
- **Action**: Identify which images achieve < 20m vs 20-50m vs > 50m
- **Expected Result**: 
  - Category 1 (< 20m): ≥ 18 images (60%)
  - Category 2 (20-50m): ~10 images
  - Category 3 (> 50m): < 2 images

### Step 9: Generate Detailed Report
- **Action**: Create comprehensive accuracy report
- **Expected Result**:
  - Percentage breakdown by error thresholds
  - Distribution histogram
  - Correlation between accuracy and image features
  - Compliance matrix for AC-1 and AC-2

## Success Criteria

**Primary Criterion (AC-2)**:
- ≥ 18 out of 30 images (60%) have GPS error < 20 meters

**Supporting Criteria**:
- Also meets AC-1 (≥ 80% < 50m)
- Mean error < 30 meters
- RMSE < 35 meters
- No catastrophic failures (errors > 200m)

## Expected Results

```
Total Images: 30
Successfully Processed: 30 (100%)
Images with error < 10m: 8 (26.7%)
Images with error < 20m: 20 (66.7%)
Images with error < 50m: 28 (93.3%)
Images with error > 50m: 2 (6.7%)
Mean Error: 24.5m
Median Error: 18.2m
RMSE: 28.3m
90th Percentile: 42.1m
AC-2 Status: PASS (66.7% > 60%)
AC-1 Status: PASS (93.3% > 80%)
```

## Pass/Fail Criteria

**TEST PASSES IF**:
- ≥ 60% of images achieve error < 20m
- Also passes AC-1 (≥ 80% < 50m)
- System performance stable across multiple runs

**TEST FAILS IF**:
- < 60% of images achieve error < 20m
- Fails AC-1 (would be critical failure)
- Results not reproducible (high variance)

## Error Analysis

If test fails or borderline:

**Investigate**:
1. **Satellite Data Quality**: Check zoom level, age of imagery, resolution
2. **LiteSAM Performance**: Review correspondence counts, homography quality
3. **Factor Graph**: Check if GPS anchors frequent enough
4. **Image Quality**: Verify no motion blur, good lighting conditions
5. **Altitude Variation**: Check if altitude assumption (400m) accurate

**Potential Improvements**:
- Use Tier-2 commercial satellite data (higher resolution)
- Increase GPS anchor frequency (every 3rd image vs every 5th)
- Tune LiteSAM confidence threshold
- Apply per-keyframe scale adjustment in factor graph

## Notes
- AC-2 is more stringent than AC-1 (20m vs 50m)
- Achieving 60% at 20m while maintaining 80% at 50m validates solution design
- LiteSAM reported RMSE of 17.86m on UAV-VisLoc dataset supports feasibility
- Test represents high-precision navigation requirement