mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-04-23 01:46:38 +00:00
add tests
gen_tests updated solution.md updated
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
# Acceptance Test: Single 350m Outlier Robustness
|
||||
|
||||
## Summary
|
||||
Validate AC-3 requirement that the system handles a single outlier image with up to 350m position deviation caused by aircraft tilt without trajectory divergence.
|
||||
|
||||
## Linked Acceptance Criteria
|
||||
**AC-3**: The system should correctly continue the work even in the presence of up to 350 meters of an outlier photo between 2 consecutive pictures en route. This could happen due to tilt of the plane.
|
||||
|
||||
## Preconditions
|
||||
- ASTRAL-Next system operational
|
||||
- Factor graph optimizer configured with robust cost functions (Cauchy/Huber kernel)
|
||||
- Test dataset with known large position jump
|
||||
- Ground truth available for validation
|
||||
|
||||
## Test Data
|
||||
- **Dataset**: AD000045-AD000050 (6 images)
|
||||
- **Outlier**: AD000047 → AD000048 (268.6m jump, closest to 350m requirement)
|
||||
- **Ground Truth**: coordinates.csv
|
||||
- **Purpose**: Verify robust handling of tilt-induced position jumps
|
||||
|
||||
## Test Steps
|
||||
|
||||
### Step 1: Establish Baseline Trajectory
|
||||
**Action**: Process AD000045-AD000046 to establish stable tracking
|
||||
**Expected Result**:
|
||||
- L1 sequential tracking successful
|
||||
- Factor graph initialized with first 2 poses
|
||||
- Baseline trajectory established
|
||||
- Status: TRACKING
|
||||
|
||||
### Step 2: Process Outlier Frame
|
||||
**Action**: Process AD000047 (before the 268.6m jump)
|
||||
**Expected Result**:
|
||||
- Frame processed normally
|
||||
- Factor graph updated
|
||||
- Position estimate within expected range
|
||||
- Status: TRACKING
|
||||
|
||||
### Step 3: Encounter Large Position Jump
|
||||
**Action**: Process AD000048 (268.6m from AD000047)
|
||||
**Expected Result**:
|
||||
- L1 reports low confidence or matching failure
|
||||
- Large translation vector detected (>>120m expected)
|
||||
- Robust cost function activates (error penalty changes from quadratic to linear/log)
|
||||
- Factor graph does NOT diverge
|
||||
- Status: OUTLIER_DETECTED
|
||||
|
||||
### Step 4: Verify Robust Handling
|
||||
**Action**: Examine factor graph's treatment of the outlier constraint
|
||||
**Expected Result**:
|
||||
- Outlier factor down-weighted (residual weight <0.3)
|
||||
- OR outlier treated as rotation+normal translation
|
||||
- Trajectory maintains consistency with other constraints
|
||||
- No catastrophic error propagation
|
||||
- Status: OUTLIER_HANDLED
|
||||
|
||||
### Step 5: Continue Processing Post-Outlier
|
||||
**Action**: Process AD000049-AD000050
|
||||
**Expected Result**:
|
||||
- L2 global relocalization may activate to verify position
|
||||
- L1 tracking resumes successfully
|
||||
- Factor graph optimization converges
|
||||
- Post-outlier trajectory accurate (<50m error)
|
||||
- Status: RECOVERED
|
||||
|
||||
### Step 6: Validate Final Trajectory
|
||||
**Action**: Compare entire sequence with ground truth
|
||||
**Expected Result**:
|
||||
```
|
||||
AD000045: Error <50m
|
||||
AD000046: Error <50m
|
||||
AD000047: Error <50m
|
||||
AD000048: Error may be 50-100m (outlier frame)
|
||||
AD000049: Error <50m (recovered)
|
||||
AD000050: Error <50m (recovered)
|
||||
Overall: System continues operation successfully
|
||||
```
|
||||
|
||||
## Pass/Fail Criteria
|
||||
|
||||
**PASS if**:
|
||||
- System processes all 6 images without crashing
|
||||
- Outlier frame (AD000048) does not cause trajectory divergence
|
||||
- Post-outlier frames (049-050) achieve <50m error
|
||||
- Factor graph converges with MRE <2.0px
|
||||
- No manual intervention required
|
||||
|
||||
**FAIL if**:
|
||||
- System crashes on outlier frame
|
||||
- Trajectory diverges (errors >200m persist for >2 frames)
|
||||
- Factor graph fails to converge
|
||||
- Manual user input required for recovery
|
||||
|
||||
## Technical Validation
|
||||
- **Robust Kernel Active**: Verify Cauchy/Huber kernel applied to outlier factor
|
||||
- **Residual Analysis**: Outlier residual >3σ from mean
|
||||
- **Weight Analysis**: Outlier factor weight <0.3 vs normal ~1.0
|
||||
- **Convergence**: Final optimization converges in <50 iterations
|
||||
|
||||
## Notes
|
||||
- Test uses 268.6m outlier (real data) as proxy for 350m requirement
|
||||
- Robust M-estimation is critical for AC-3 compliance
|
||||
- Single outlier is easier than multiple; see AC-3 test 35 for multiple outliers
|
||||
|
||||
Reference in New Issue
Block a user