azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-04-23 00:26:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

update tests

Browse Source
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-30 16:21:03 +02:00
parent ce9760fcbe
commit b12f37ab01
12 changed files with 1435 additions and 603 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/03_tests/00_test_summary.md
+88 -53
View File
@@ -3,7 +3,7 @@
## Overview
Comprehensive test specifications for the GPS-denied navigation system following the QA testing pyramid approach.
**Total Test Specifications**: 56
**Total Test Specifications**: 49
## Test Organization
@@ -11,65 +11,77 @@ Comprehensive test specifications for the GPS-denied navigation system following
Tests individual system components in isolation with their dependencies.
**Vision Pipeline (01-04)**:
- 01: Sequential Visual Odometry (SuperPoint + LightGlue)
- 02: Global Place Recognition (AnyLoc)
- 03: Metric Refinement (LiteSAM)
- 04: Factor Graph Optimizer (GTSAM)
- 01: Sequential Visual Odometry (F07 - SuperPoint + LightGlue)
- 02: Global Place Recognition (F08 - AnyLoc/DINOv2)
- 03: Metric Refinement (F09 - LiteSAM)
- 04: Factor Graph Optimizer (F10 - GTSAM)
**Data Management (05-08)**:
- 05: Satellite Data Manager
- 06: Coordinate Transformer
- 07: Image Input Pipeline
- 08: Image Rotation Manager
- 05: Satellite Data Manager (F04)
- 06: Coordinate Transformer (F13)
- 07: Image Input Pipeline (F05)
- 08: Image Rotation Manager (F06)
**Service Infrastructure (09-12)**:
- 09: REST API (FastAPI endpoints for flight management, image uploads, user fixes)
- 10: SSE Event Streamer (Server-Sent Events for real-time result streaming)
- 11: Flight Manager
- 12: Result Manager
- 09: REST API (F01 - FastAPI endpoints)
- 10: SSE Event Streamer (F15 - real-time streaming)
- 11a: Flight Lifecycle Manager (F02.1 - CRUD, initialization, API delegation)
- 11b: Flight Processing Engine (F02.2 - processing loop, recovery coordination)
- 12: Result Manager (F14)
**Support Components (13-16)**:
- 13: Model Manager (TensorRT)
- 14: Failure Recovery Coordinator
- 15: Configuration Manager
- 16: Database Layer
- 13: Model Manager (F16 - TensorRT)
- 14: Failure Recovery Coordinator (F11)
- 15: Configuration Manager (F17)
- 16: Database Layer (F03)
### System Integration Tests (21-25): Multi-Component Flows
Tests integration between multiple components.
- 21: End-to-End Normal Flight
- 22: Satellite to Vision Pipeline
- 23: Vision to Optimization Pipeline
- 22: Satellite to Vision Pipeline (F04 → F07/F08/F09)
- 23: Vision to Optimization Pipeline (F07/F08/F09 → F10)
- 24: Multi-Component Error Propagation
- 25: Real-Time Streaming Pipeline
- 25: Real-Time Streaming Pipeline (F02 → F14 → F15)
### Acceptance Tests (31-43): Requirements Validation
### Acceptance Tests (31-50): Requirements Validation
Tests mapped to 10 acceptance criteria.
**Accuracy (31-32)**:
- 31: AC-1 - 80% < 50m error
- 32: AC-2 - 60% < 20m error
**Accuracy (31-33)**:
- 31: AC-1 - 80% < 50m error (baseline)
- 32: AC-1 - 80% < 50m error (varied terrain)
- 33: AC-2 - 60% < 20m error (high precision)
**Robustness (33-35)**:
- 33: AC-3 - 350m outlier handling
- 34: AC-4 - Sharp turn recovery
- 35: AC-5 - Multi-fragment route connection
**Robustness - Outliers (34-35)**:
- 34: AC-3 - Single 350m outlier handling
- 35: AC-3 - Multiple outliers handling
**User Interaction (36)**:
- 36: AC-6 - User input after 3 failures
**Robustness - Sharp Turns (36-38)**:
- 36: AC-4 - Sharp turn zero overlap recovery
- 37: AC-4 - Sharp turn minimal overlap (<5%)
- 38: Outlier anchor detection
**Performance (37-38)**:
- 37: AC-7 - <5s per image
- 38: AC-8 - Real-time streaming + refinement
**Multi-Fragment (39)**:
- 39: AC-5 - Multi-fragment route connection (chunk architecture)
**Quality Metrics (39-40)**:
- 39: AC-9 - Registration rate >95%
- 40: AC-10 - MRE <1.0 pixels
**User Interaction (40)**:
- 40: AC-6 - User input after 3 consecutive failures
**Cross-Cutting (41-43)**:
- 41: Long flight (3000 images)
- 42: Degraded satellite data
- 43: Complete system validation
**Performance (41-44)**:
- 41: AC-7 - <5s single image processing
- 42: AC-7 - Sustained throughput performance
- 43: AC-8 - Real-time streaming results
- 44: AC-8 - Async refinement delivery
**Quality Metrics (45-47)**:
- 45: AC-9 - Registration rate >95% (baseline)
- 46: AC-9 - Registration rate >95% (challenging conditions)
- 47: AC-10 - Mean Reprojection Error <1.0 pixels
**Cross-Cutting (48-50)**:
- 48: Long flight (3000 images)
- 49: Degraded satellite data
- 50: Complete system acceptance validation
**Chunk-Based Recovery (55-56)**:
- 55: Chunk rotation recovery (rotation sweeps for chunks)
@@ -109,16 +121,39 @@ Tests using GPS-analyzed test datasets.
| AC | Requirement | Test Specs | Status |
|----|-------------|------------|--------|
| AC-1 | 80% < 50m error | 31, 43, 51, 54 | ✓ Covered |
| AC-2 | 60% < 20m error | 32, 43, 51, 54 | ✓ Covered |
| AC-3 | 350m outlier robust | 33, 43, 52, 54 | ✓ Covered |
| AC-4 | Sharp turn <5% overlap | 34, 43, 53, 54, 55 | ✓ Covered |
| AC-5 | Multi-fragment connection | 35, 39, 43, 56 | ✓ Covered |
| AC-6 | User input after 3 failures | 36, 43 | ✓ Covered |
| AC-7 | <5s per image | 37, 43, 51, 54 | ✓ Covered |
| AC-8 | Real-time + refinement | 38, 43 | ✓ Covered |
| AC-9 | Registration >95% | 39, 43, 51, 54 | ✓ Covered |
| AC-10 | MRE <1.0px | 40, 43 | ✓ Covered |
| AC-1 | 80% < 50m error | 31, 32, 50, 51, 54 | ✓ Covered |
| AC-2 | 60% < 20m error | 33, 50, 51, 54 | ✓ Covered |
| AC-3 | 350m outlier robust | 34, 35, 50, 52, 54 | ✓ Covered |
| AC-4 | Sharp turn <5% overlap | 36, 37, 50, 53, 54, 55 | ✓ Covered |
| AC-5 | Multi-fragment connection | 39, 50, 56 | ✓ Covered |
| AC-6 | User input after 3 failures | 40, 50 | ✓ Covered |
| AC-7 | <5s per image | 41, 42, 50, 51, 54 | ✓ Covered |
| AC-8 | Real-time + refinement | 43, 44, 50 | ✓ Covered |
| AC-9 | Registration >95% | 45, 46, 50, 51, 54 | ✓ Covered |
| AC-10 | MRE <1.0px | 47, 50 | ✓ Covered |
## Component to Test Mapping
| Component | ID | Integration Test |
|-----------|-----|------------------|
| Flight API | F01 | 09 |
| Flight Lifecycle Manager | F02.1 | 11a |
| Flight Processing Engine | F02.2 | 11b |
| Flight Database | F03 | 16 |
| Satellite Data Manager | F04 | 05 |
| Image Input Pipeline | F05 | 07 |
| Image Rotation Manager | F06 | 08 |
| Sequential Visual Odometry | F07 | 01 |
| Global Place Recognition | F08 | 02 |
| Metric Refinement | F09 | 03 |
| Factor Graph Optimizer | F10 | 04 |
| Failure Recovery Coordinator | F11 | 14 |
| Route Chunk Manager | F12 | 39, 55, 56 |
| Coordinate Transformer | F13 | 06 |
| Result Manager | F14 | 12 |
| SSE Event Streamer | F15 | 10 |
| Model Manager | F16 | 13 |
| Configuration Manager | F17 | 15 |
## Test Execution Strategy
@@ -132,14 +167,15 @@ Tests using GPS-analyzed test datasets.
- Validate end-to-end flows
- Verify error handling across components
### Phase 3: Acceptance Testing (31-43)
### Phase 3: Acceptance Testing (31-50)
- Validate all acceptance criteria
- Use GPS-analyzed real data
- Measure against requirements
### Phase 4: Special Scenarios (51-54)
### Phase 4: Special Scenarios (51-56)
- Test specific GPS-identified situations
- Validate outliers and sharp turns
- Chunk-based recovery scenarios
- Full system validation
## Success Criteria Summary
@@ -164,4 +200,3 @@ Tests using GPS-analyzed test datasets.
- Specifications ready for QA team implementation
- No code included per requirement
- Tests cover all components and all acceptance criteria