gen_tests updated solution.md updated
4.3 KiB
Acceptance Test: Sharp Turns with Minimal Overlap (<5%)
Summary
Validate AC-4 requirement for handling sharp turns where consecutive images have minimal overlap (1-5%) but do share some common features, testing the boundaries of L1 sequential tracking before L2 activation.
Linked Acceptance Criteria
AC-4: System should correctly continue the work even during sharp turns, where the next photo doesn't overlap at all, or overlaps in less than 5%. The next photo should be in less than 200m drift and at an angle of less than 70%.
Preconditions
- ASTRAL-Next system operational
- LightGlue configured with adaptive depth mechanism
- L2 global place recognition ready for fallback
- Minimal overlap test dataset prepared
Test Data
- Dataset A: AD000032-AD000035 (contains 220.6m jump)
- Dataset B: AD000042, AD000044, AD000045 (skip 043, natural gap)
- Overlap Characteristics: 2-5% estimated (edges of consecutive frames)
- Distance: All jumps <230m (within 200m drift requirement)
Test Steps
Step 1: Test Minimal Overlap Scenario A
Action: Process AD000032 → AD000033 (220.6m jump) Expected Result:
- L1 SuperPoint+LightGlue attempts matching
- Feature count low (10-50 matches vs normal 100+)
- LightGlue adaptive depth increases (more attention layers)
- Confidence score reported as LOW
- Status: MINIMAL_OVERLAP_L1
Step 2: Verify L1 Handling or L2 Activation
Action: Analyze matching result and system response Expected Result:
- IF ≥10 high-quality matches: L1 succeeds, pose estimated
- IF <10 matches: L1 fails gracefully, L2 activates
- No system crash or undefined behavior
- Transition between L1/L2 smooth
- Status: L1_OR_L2_SUCCESS
Step 3: Test Minimal Overlap Scenario B
Action: Process AD000042 → AD000044 (skipping 043) Expected Result:
- Larger gap increases difficulty
- L1 likely fails (overlap too low)
- L2 AnyLoc retrieval activates
- DINOv2 features match despite view change
- Global position estimated from satellite database
- Status: L2_RELOCALIZATION
Step 4: Validate Position Accuracy
Action: Compare estimated positions with ground truth Expected Result:
AD000033: Error <100m (minimal overlap, higher uncertainty)
AD000044: Error <100m (L2 retrieval, coarse localization)
Drift: <200m for both scenarios (AC-4 requirement met)
Step 5: Test Continuous Minimal Overlap Sequence
Action: Process AD000032-035 as continuous sequence Expected Result:
- Initial minimal overlap (032→033) handled
- Subsequent frames (033→034→035) easier (normal overlap)
- Factor graph smooths trajectory
- Final accuracy improved through optimization
- All frames: Error <50m after optimization
- Status: SEQUENCE_COMPLETED
Step 6: Verify Angular Constraint (<70°)
Action: Analyze estimated rotation between frames Expected Result:
- Rotation estimates <70° for all minimal overlap scenarios
- Rotation estimates consistent with wing-type UAV banking
- No unrealistic 180° rotations estimated
- Status: ANGULAR_CONSTRAINT_MET
Pass/Fail Criteria
PASS if:
- Both minimal overlap scenarios processed without crash
- Position drift <200m for each minimal overlap frame
- Angular changes <70° estimated
- System demonstrates graceful L1→L2 transition
- Final optimized trajectory <50m error
FAIL if:
- System crashes on minimal overlap
- Position drift >200m for any frame
- Angular changes >70° estimated
- No L2 fallback when L1 fails
- Trajectory diverges
Technical Validation
- Match Count: Minimal overlap yields 10-50 matches (vs normal 100+)
- LightGlue Depth: Adaptive mechanism uses more layers (8+ vs typical 6)
- L1 Confidence: <0.5 for minimal overlap (vs >0.8 normal)
- L2 Activation: Triggered when L1 confidence <0.4
- Retrieval Accuracy: Top-1 satellite tile correct in ≥90% cases
Edge Case Analysis
- 1-5% overlap: Boundary between L1 possible and L1 impossible
- 5-10% overlap: L1 should succeed with high confidence
- 0% overlap: Covered in separate test (36_sharp_turn_zero_overlap_spec.md)
Notes
- This test validates the smooth transition zone between L1 and L2
- LightGlue's attention mechanism is crucial for minimal overlap success
- Real flights may have brief periods of minimal overlap during banking
- AC-4 allows up to 200m drift, giving system reasonable tolerance