mirror of
https://github.com/azaion/gps-denied-desktop.git
synced 2026-04-22 22:46:36 +00:00
Oleksandr Bezdieniezhnykh
168 lines
5.3 KiB
Markdown
168 lines
5.3 KiB
Markdown
# ASTRAL-Next Test Specifications Summary
|
|
|
|
## Overview
|
|
Comprehensive test specifications for the GPS-denied navigation system following the QA testing pyramid approach.
|
|
|
|
**Total Test Specifications**: 56
|
|
|
|
## Test Organization
|
|
|
|
### Integration Tests (01-16): Component Level
|
|
Tests individual system components in isolation with their dependencies.
|
|
|
|
**Vision Pipeline (01-04)**:
|
|
- 01: Sequential Visual Odometry (SuperPoint + LightGlue)
|
|
- 02: Global Place Recognition (AnyLoc)
|
|
- 03: Metric Refinement (LiteSAM)
|
|
- 04: Factor Graph Optimizer (GTSAM)
|
|
|
|
**Data Management (05-08)**:
|
|
- 05: Satellite Data Manager
|
|
- 06: Coordinate Transformer
|
|
- 07: Image Input Pipeline
|
|
- 08: Image Rotation Manager
|
|
|
|
**Service Infrastructure (09-12)**:
|
|
- 09: REST API (FastAPI endpoints for flight management, image uploads, user fixes)
|
|
- 10: SSE Event Streamer (Server-Sent Events for real-time result streaming)
|
|
- 11: Flight Manager
|
|
- 12: Result Manager
|
|
|
|
**Support Components (13-16)**:
|
|
- 13: Model Manager (TensorRT)
|
|
- 14: Failure Recovery Coordinator
|
|
- 15: Configuration Manager
|
|
- 16: Database Layer
|
|
|
|
### System Integration Tests (21-25): Multi-Component Flows
|
|
Tests integration between multiple components.
|
|
|
|
- 21: End-to-End Normal Flight
|
|
- 22: Satellite to Vision Pipeline
|
|
- 23: Vision to Optimization Pipeline
|
|
- 24: Multi-Component Error Propagation
|
|
- 25: Real-Time Streaming Pipeline
|
|
|
|
### Acceptance Tests (31-43): Requirements Validation
|
|
Tests mapped to 10 acceptance criteria.
|
|
|
|
**Accuracy (31-32)**:
|
|
- 31: AC-1 - 80% < 50m error
|
|
- 32: AC-2 - 60% < 20m error
|
|
|
|
**Robustness (33-35)**:
|
|
- 33: AC-3 - 350m outlier handling
|
|
- 34: AC-4 - Sharp turn recovery
|
|
- 35: AC-5 - Multi-fragment route connection
|
|
|
|
**User Interaction (36)**:
|
|
- 36: AC-6 - User input after 3 failures
|
|
|
|
**Performance (37-38)**:
|
|
- 37: AC-7 - <5s per image
|
|
- 38: AC-8 - Real-time streaming + refinement
|
|
|
|
**Quality Metrics (39-40)**:
|
|
- 39: AC-9 - Registration rate >95%
|
|
- 40: AC-10 - MRE <1.0 pixels
|
|
|
|
**Cross-Cutting (41-43)**:
|
|
- 41: Long flight (3000 images)
|
|
- 42: Degraded satellite data
|
|
- 43: Complete system validation
|
|
|
|
**Chunk-Based Recovery (55-56)**:
|
|
- 55: Chunk rotation recovery (rotation sweeps for chunks)
|
|
- 56: Multi-chunk simultaneous processing (Atlas architecture)
|
|
|
|
### GPS-Analyzed Scenario Tests (51-54): Real Data
|
|
Tests using GPS-analyzed test datasets.
|
|
|
|
- 51: Test_Baseline (AD000001-030) - Standard flight
|
|
- 52: Test_Outlier_350m (AD000045-050) - Outlier scenario
|
|
- 53: Test_Sharp_Turn - Multiple sharp turn datasets
|
|
- 54: Test_Long_Flight (AD000001-060) - Full dataset
|
|
|
|
## Test Data
|
|
|
|
### GPS Analysis Results
|
|
- Mean distance: 120.8m
|
|
- Min distance: 24.2m
|
|
- Max distance: 268.6m
|
|
|
|
**Identified Sharp Turns (>200m)**:
|
|
- AD000003 → AD000004: 202.2m
|
|
- AD000032 → AD000033: 220.6m
|
|
- AD000042 → AD000043: 234.2m
|
|
- AD000044 → AD000045: 230.2m
|
|
- AD000047 → AD000048: 268.6m (largest outlier)
|
|
|
|
### Test Datasets
|
|
**Test_Baseline**: AD000001-030 (30 images, normal spacing)
|
|
**Test_Outlier_350m**: AD000045-050 (6 images, 268.6m outlier)
|
|
**Test_Sharp_Turn_A**: AD000042, AD000044, AD000045, AD000046 (skip 043)
|
|
**Test_Sharp_Turn_B**: AD000032-035 (220m jump)
|
|
**Test_Sharp_Turn_C**: AD000003, AD000009 (5-frame gap)
|
|
**Test_Long_Flight**: AD000001-060 (all 60 images, all variations)
|
|
|
|
## Acceptance Criteria Coverage
|
|
|
|
| AC | Requirement | Test Specs | Status |
|
|
|----|-------------|------------|--------|
|
|
| AC-1 | 80% < 50m error | 31, 43, 51, 54 | ✓ Covered |
|
|
| AC-2 | 60% < 20m error | 32, 43, 51, 54 | ✓ Covered |
|
|
| AC-3 | 350m outlier robust | 33, 43, 52, 54 | ✓ Covered |
|
|
| AC-4 | Sharp turn <5% overlap | 34, 43, 53, 54, 55 | ✓ Covered |
|
|
| AC-5 | Multi-fragment connection | 35, 39, 43, 56 | ✓ Covered |
|
|
| AC-6 | User input after 3 failures | 36, 43 | ✓ Covered |
|
|
| AC-7 | <5s per image | 37, 43, 51, 54 | ✓ Covered |
|
|
| AC-8 | Real-time + refinement | 38, 43 | ✓ Covered |
|
|
| AC-9 | Registration >95% | 39, 43, 51, 54 | ✓ Covered |
|
|
| AC-10 | MRE <1.0px | 40, 43 | ✓ Covered |
|
|
|
|
## Test Execution Strategy
|
|
|
|
### Phase 1: Component Integration (01-16)
|
|
- Validate each component individually
|
|
- Verify interfaces and dependencies
|
|
- Establish baseline performance metrics
|
|
|
|
### Phase 2: System Integration (21-25)
|
|
- Test multi-component interactions
|
|
- Validate end-to-end flows
|
|
- Verify error handling across components
|
|
|
|
### Phase 3: Acceptance Testing (31-43)
|
|
- Validate all acceptance criteria
|
|
- Use GPS-analyzed real data
|
|
- Measure against requirements
|
|
|
|
### Phase 4: Special Scenarios (51-54)
|
|
- Test specific GPS-identified situations
|
|
- Validate outliers and sharp turns
|
|
- Full system validation
|
|
|
|
## Success Criteria Summary
|
|
|
|
**Integration Tests**: All components pass individual tests, interfaces work correctly
|
|
**System Tests**: Multi-component flows work, errors handled properly
|
|
**Acceptance Tests**: All 10 ACs met with real data
|
|
**Overall**: System meets all requirements, ready for deployment
|
|
|
|
## Test Metrics to Track
|
|
|
|
- **Accuracy**: Mean error, RMSE, percentiles
|
|
- **Performance**: Processing time per image, total time
|
|
- **Reliability**: Registration rate, success rate
|
|
- **Quality**: MRE, confidence scores
|
|
- **Robustness**: Outlier handling, error recovery
|
|
|
|
## Notes
|
|
|
|
- All test specs follow standard format (Integration vs Acceptance)
|
|
- GPS-analyzed datasets based on actual test data coordinates
|
|
- Specifications ready for QA team implementation
|
|
- No code included per requirement
|
|
- Tests cover all components and all acceptance criteria
|
|
|