mirror of
https://github.com/azaion/gps-denied-desktop.git
synced 2026-04-23 03:26:37 +00:00
add tests
gen_tests updated solution.md updated
This commit is contained in:
@@ -0,0 +1,105 @@
|
||||
# Acceptance Test: AC-1 - 80% of Photos < 50m Error
|
||||
|
||||
## Summary
|
||||
Validate Acceptance Criterion 1: "The system should find out the GPS of centers of 80% of the photos from the flight within an error of no more than 50 meters in comparison to the real GPS."
|
||||
|
||||
## Linked Acceptance Criteria
|
||||
**AC-1**: 80% of photos < 50m error
|
||||
|
||||
## Preconditions
|
||||
1. ASTRAL-Next system fully operational
|
||||
2. All TensorRT models loaded
|
||||
3. Satellite tiles cached for test area (48.25-48.28°N, 37.34-37.39°E)
|
||||
4. Ground truth GPS coordinates available (coordinates.csv)
|
||||
5. Test dataset prepared: Test_Baseline (AD000001-AD000030)
|
||||
|
||||
## Test Description
|
||||
Process baseline flight of 30 images with normal spacing (~120m between images). Compare estimated GPS coordinates against ground truth and verify that at least 80% achieve error < 50 meters.
|
||||
|
||||
## Test Steps
|
||||
|
||||
### Step 1: Initialize System
|
||||
- **Action**: Start ASTRAL-Next system, verify all components ready
|
||||
- **Expected Result**: System state = "ready", all models loaded, no errors
|
||||
|
||||
### Step 2: Create Test Flight
|
||||
- **Action**: Create flight "AC1_Baseline" with start_gps=48.275292, 37.385220, altitude=400m
|
||||
- **Expected Result**: Flight created, flight_id returned
|
||||
|
||||
### Step 3: Upload Test Images
|
||||
- **Action**: Upload AD000001-AD000030 (30 images) in order
|
||||
- **Expected Result**: All 30 images queued, sequence maintained
|
||||
|
||||
### Step 4: Monitor Processing
|
||||
- **Action**: Monitor flight status until completed
|
||||
- **Expected Result**:
|
||||
- Processing completes within 150 seconds (5s per image)
|
||||
- No system errors
|
||||
- Registration rate > 95%
|
||||
|
||||
### Step 5: Retrieve Results
|
||||
- **Action**: GET /api/v1/flights/{id}/results
|
||||
- **Expected Result**: Results for all 30 images returned
|
||||
|
||||
### Step 6: Calculate Errors
|
||||
- **Action**: For each image, calculate haversine distance between estimated and ground truth GPS
|
||||
- **Expected Result**: Error array with 30 values
|
||||
|
||||
### Step 7: Validate AC-1
|
||||
- **Action**: Count images with error < 50m, calculate percentage
|
||||
- **Expected Result**: **≥ 80% of images have error < 50 meters** ✓
|
||||
|
||||
### Step 8: Generate Report
|
||||
- **Action**: Create test report with statistics
|
||||
- **Expected Result**:
|
||||
- Total images: 30
|
||||
- Images < 50m: ≥ 24
|
||||
- Percentage: ≥ 80.0%
|
||||
- Mean error: documented
|
||||
- Median error: documented
|
||||
- Max error: documented
|
||||
|
||||
## Success Criteria
|
||||
|
||||
**Primary Criterion (AC-1)**:
|
||||
- ≥ 24 out of 30 images (80%) have GPS error < 50 meters
|
||||
|
||||
**Supporting Criteria**:
|
||||
- All 30 images processed (or user input requested if failures occur)
|
||||
- Processing time < 150 seconds total
|
||||
- No system crashes or unhandled errors
|
||||
- Registration rate > 95% (AC-9)
|
||||
|
||||
## Expected Results
|
||||
|
||||
Based on solution architecture (LiteSAM RMSE ~18m), expected performance:
|
||||
```
|
||||
Total Images: 30
|
||||
Successfully Processed: 30 (100%)
|
||||
Images with error < 50m: 28 (93.3%)
|
||||
Images with error < 20m: 20 (66.7%)
|
||||
Mean Error: 24.5m
|
||||
Median Error: 18.2m
|
||||
RMSE: 28.3m
|
||||
Max Error: 48.7m
|
||||
AC-1 Status: PASS (93.3% > 80%)
|
||||
```
|
||||
|
||||
## Pass/Fail Criteria
|
||||
|
||||
**TEST PASSES IF**:
|
||||
- ≥ 80% of images achieve error < 50m
|
||||
- System completes processing without critical failures
|
||||
- Results reproducible across multiple test runs
|
||||
|
||||
**TEST FAILS IF**:
|
||||
- < 80% of images achieve error < 50m
|
||||
- System crashes or becomes unresponsive
|
||||
- More than 5% of images fail to process (violates AC-9)
|
||||
|
||||
## Notes
|
||||
- This test uses Test_Baseline dataset (AD000001-030) with consistent spacing
|
||||
- No sharp turns or outliers in this dataset
|
||||
- Represents ideal operating conditions
|
||||
- If test fails, investigate: satellite data quality, model accuracy, georeferencing precision
|
||||
|
||||
Reference in New Issue
Block a user