Oleksandr Bezdieniezhnykh
4.1 KiB
Validation Log
Validation Scenario 1: Normal Flight (500 images, 60-80% overlap, mild turns)
Expected Based on Conclusions
- XFeat VO: ~15ms/frame → total VO time for 500 images: ~7.5s
- LiteSAM satellite matching (overlapped): ~200ms/frame
- Total processing: ~100s (under 5s/image budget)
- Most images get satellite anchors → HIGH confidence
- VRAM peak: ~900MB (XFeat 200 + DINOv2 300 + LiteSAM 400)
Actual Validation Results
Consistent with SatLoc-Fusion results on similar setup. XFeat handles consecutive frames well. Time budget is well within 5s AC.
Counterexamples
None for normal flight.
Validation Scenario 2: Flight over outdated satellite imagery area (eastern Ukraine conflict zone)
Expected Based on Conclusions
- DINOv2 coarse retrieval: semantic matching should still identify approximate area despite 1-3 year imagery age
- LiteSAM fine matching: likely degraded if buildings destroyed/rebuilt. Hit rate could drop 10-20pp from baseline.
- Many frames may be VO-only → drift accumulates
- Drift monitoring triggers warnings at 100m, user input at 200m
Actual Validation Results
System degrades gracefully. VO chain continues providing relative positioning. Satellite anchors become sparse. Confidence reporting reflects this via exponential decay formula.
Counterexamples
If entire flight is over heavily changed terrain, ALL satellite matches may fail. System falls back to pure VO + user manual anchoring. This is handled by the segment manager but accuracy degrades significantly.
Validation Scenario 3: LiteSAM startup failure (weights corrupted or unavailable)
Expected Based on Conclusions
- SHA256 checksum verification catches corruption at startup
- System falls back to EfficientLoFTR (or SP+LG) for satellite fine matching
- Warning logged, system continues
Actual Validation Results
Fallback mechanism ensures system availability. EfficientLoFTR has proven quality (CVPR 2024).
Counterexamples
If BOTH LiteSAM and fallback fail to load → system should still start but without satellite matching (VO-only mode). Not currently handled — SHOULD add this graceful degradation.
Validation Scenario 4: Sharp turn with 5+ disconnected segments
Expected Based on Conclusions
- Each segment tracks independently with VO
- Satellite anchoring attempts run for each segment
- ANCHORED segments check for nearby FLOATING segments
- With XFeat VO at 15ms, segment transitions are detected quickly
Actual Validation Results
Strategy works for 2-3 segments. With 5+ segments, reconnection order matters. Should process segments in proximity order. If satellite imagery is outdated for the area, many segments remain FLOATING.
Counterexamples
All segments FLOATING in a poor satellite imagery area. User must manually anchor at least one image per segment. Current system handles this but UX could be improved — suggest a "batch anchor" endpoint.
Validation Scenario 5: iSAM2 exception during optimization
Expected Based on Conclusions
- IndeterminantLinearSystemException caught
- Skip problematic factor, retry with relaxed noise
- Pipeline continues
Actual Validation Results
Error handling prevents crash. Position for affected frame derived from VO only.
Counterexamples
If exception happens on first frame's prior factor → entire optimization fails. Need special handling for initial factor.
Review Checklist
- Draft conclusions consistent with fact cards
- No important dimensions missed
- No over-extrapolation
- Conclusions actionable/verifiable
- LiteSAM hit rate correctly attributed to proper dataset
- VO regression identified and fix proposed
- Security CVEs addressed with version pinning
- Issue: Need to add EfficientLoFTR fallback and graceful degradation for model loading failures
- Issue: Need to add iSAM2 error handling for initial factor edge case
Conclusions Requiring Revision
- Add graceful degradation when ALL matchers fail to load (VO-only mode)
- Add special iSAM2 error handling for initial prior factor
- Consider "batch anchor" API endpoint for multi-segment manual anchoring UX