add tests

gen_tests updated
solution.md updated
This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-11-24 22:57:46 +02:00
parent f50006d100
commit 4f8c18a066
49 changed files with 7209 additions and 3 deletions
@@ -0,0 +1,111 @@
# Acceptance Test: Sharp Turns with Minimal Overlap (<5%)
## Summary
Validate AC-4 requirement for handling sharp turns where consecutive images have minimal overlap (1-5%) but do share some common features, testing the boundaries of L1 sequential tracking before L2 activation.
## Linked Acceptance Criteria
**AC-4**: System should correctly continue the work even during sharp turns, where the next photo doesn't overlap at all, or overlaps in less than 5%. The next photo should be in less than 200m drift and at an angle of less than 70%.
## Preconditions
- ASTRAL-Next system operational
- LightGlue configured with adaptive depth mechanism
- L2 global place recognition ready for fallback
- Minimal overlap test dataset prepared
## Test Data
- **Dataset A**: AD000032-AD000035 (contains 220.6m jump)
- **Dataset B**: AD000042, AD000044, AD000045 (skip 043, natural gap)
- **Overlap Characteristics**: 2-5% estimated (edges of consecutive frames)
- **Distance**: All jumps <230m (within 200m drift requirement)
## Test Steps
### Step 1: Test Minimal Overlap Scenario A
**Action**: Process AD000032 → AD000033 (220.6m jump)
**Expected Result**:
- L1 SuperPoint+LightGlue attempts matching
- Feature count low (10-50 matches vs normal 100+)
- LightGlue adaptive depth increases (more attention layers)
- Confidence score reported as LOW
- Status: MINIMAL_OVERLAP_L1
### Step 2: Verify L1 Handling or L2 Activation
**Action**: Analyze matching result and system response
**Expected Result**:
- IF ≥10 high-quality matches: L1 succeeds, pose estimated
- IF <10 matches: L1 fails gracefully, L2 activates
- No system crash or undefined behavior
- Transition between L1/L2 smooth
- Status: L1_OR_L2_SUCCESS
### Step 3: Test Minimal Overlap Scenario B
**Action**: Process AD000042 → AD000044 (skipping 043)
**Expected Result**:
- Larger gap increases difficulty
- L1 likely fails (overlap too low)
- L2 AnyLoc retrieval activates
- DINOv2 features match despite view change
- Global position estimated from satellite database
- Status: L2_RELOCALIZATION
### Step 4: Validate Position Accuracy
**Action**: Compare estimated positions with ground truth
**Expected Result**:
```
AD000033: Error <100m (minimal overlap, higher uncertainty)
AD000044: Error <100m (L2 retrieval, coarse localization)
Drift: <200m for both scenarios (AC-4 requirement met)
```
### Step 5: Test Continuous Minimal Overlap Sequence
**Action**: Process AD000032-035 as continuous sequence
**Expected Result**:
- Initial minimal overlap (032→033) handled
- Subsequent frames (033→034→035) easier (normal overlap)
- Factor graph smooths trajectory
- Final accuracy improved through optimization
- All frames: Error <50m after optimization
- Status: SEQUENCE_COMPLETED
### Step 6: Verify Angular Constraint (<70°)
**Action**: Analyze estimated rotation between frames
**Expected Result**:
- Rotation estimates <70° for all minimal overlap scenarios
- Rotation estimates consistent with wing-type UAV banking
- No unrealistic 180° rotations estimated
- Status: ANGULAR_CONSTRAINT_MET
## Pass/Fail Criteria
**PASS if**:
- Both minimal overlap scenarios processed without crash
- Position drift <200m for each minimal overlap frame
- Angular changes <70° estimated
- System demonstrates graceful L1→L2 transition
- Final optimized trajectory <50m error
**FAIL if**:
- System crashes on minimal overlap
- Position drift >200m for any frame
- Angular changes >70° estimated
- No L2 fallback when L1 fails
- Trajectory diverges
## Technical Validation
- **Match Count**: Minimal overlap yields 10-50 matches (vs normal 100+)
- **LightGlue Depth**: Adaptive mechanism uses more layers (8+ vs typical 6)
- **L1 Confidence**: <0.5 for minimal overlap (vs >0.8 normal)
- **L2 Activation**: Triggered when L1 confidence <0.4
- **Retrieval Accuracy**: Top-1 satellite tile correct in ≥90% cases
## Edge Case Analysis
- **1-5% overlap**: Boundary between L1 possible and L1 impossible
- **5-10% overlap**: L1 should succeed with high confidence
- **0% overlap**: Covered in separate test (36_sharp_turn_zero_overlap_spec.md)
## Notes
- This test validates the smooth transition zone between L1 and L2
- LightGlue's attention mechanism is crucial for minimal overlap success
- Real flights may have brief periods of minimal overlap during banking
- AC-4 allows up to 200m drift, giving system reasonable tolerance