# Reasoning Chain — Draft02 Assessment

## Dimension 1: Factor Graph Design Correctness

### Fact Confirmation
According to Fact #1, GTSAM's `GPSFactor` class works exclusively with `Pose3` variables. `GPSFactor2` works with `NavState`. Neither accepts `Pose2`. According to Fact #19, `PriorFactorPoint2` provides 2D position constraints, and `BetweenFactorPose2` provides 2D odometry constraints.

### Problem
Draft02 specifies `Pose2 (x, y, heading)` variables but lists `GPSFactor` for satellite anchors. This is an API mismatch — the code would fail at runtime.

### Solution
Two valid approaches:
1. **Pose2 graph (recommended)**: Use `Pose2` variables + `BetweenFactorPose2` for VO + `PriorFactorPose2` for satellite anchors (constraining full pose when heading is available) or use a custom partial factor that constrains only the position part of Pose2. Convert WGS84 to local ENU coordinates centered on starting GPS.
2. **Pose3 graph**: Use `Pose3` with fixed altitude. More accurate but adds unnecessary complexity for 2D problem.

The custom DEM terrain factor and drift limit factor also need reconsideration: Python custom factors invoke a Python callback per optimization step, which is slow. DEM terrain is irrelevant for 2D Pose2 (altitude is not a variable). Drift should be managed by the Segment Manager logic, not as a factor.

### Conclusion
Switch to Pose2 + BetweenFactorPose2 + PriorFactorPose2 (or partial position prior). Remove DEM terrain factor (handle elevation in GSD calculation outside the graph). Remove drift limit factor (handle in Segment Manager). This simplifies the factor graph and avoids Python callback overhead.

### Confidence
✅ High — based on official GTSAM documentation

---

## Dimension 2: VRAM/Memory Budget Feasibility

### Fact Confirmation
According to Fact #8: DINOv2 ViT-S/14 ~300MB, ViT-B/14 ~600MB. Fact #16: faiss GPU uses ~2GB scratch. Fact #3: LightGlue ONNX FP16 works on RTX 2060.

### Problem
Draft02 doesn't specify DINOv2 model size. Proposing faiss GPU would consume 2GB of 6GB VRAM. Combined with other models, total could exceed 6GB. Draft02 estimates ~1.5GB per frame for XFeat/SuperPoint + LightGlue, but doesn't account for DINOv2 or faiss.

### Budget Analysis
Concurrent peak VRAM (worst case):
- XFeat inference: ~200MB
- LightGlue ONNX (FP16): ~500MB
- DINOv2 ViT-B/14: ~600MB
- SuperPoint: ~400MB
- faiss GPU: ~2GB
- ONNX Runtime overhead: ~300MB
- **Total: ~4.0GB** (without faiss GPU: ~2.0GB)

Sequential loading (recommended):
- Step 1: XFeat + XFeat matcher (VO): ~400MB
- Step 2: DINOv2-S (coarse retrieval): ~300MB → unload
- Step 3: SuperPoint + LightGlue ONNX (fine matching): ~900MB
- Peak: ~1.3GB (with model switching)

### Conclusion
Use DINOv2 ViT-S/14 (300MB, 0.05s/img — fast enough for coarse retrieval). Run faiss on CPU (the embedding vectors are small, CPU search is <1ms for ~2000 vectors). Sequential model loading for GPU: VO models first, then satellite matching models. Keep models loaded but process sequentially (no concurrent GPU inference).

### Confidence
✅ High — based on documented VRAM numbers

---

## Dimension 3: Rotation Handling Completeness

### Fact Confirmation
According to Fact #4, SuperPoint+LightGlue fail at 90°/180° rotations. According to Fact #6 (ISPRS 2025), SIFT+LightGlue outperforms SuperPoint+LightGlue in high-rotation UAV scenarios.

### Problem
Draft02 says "estimate heading from VO chain, rectify images before satellite matching, SIFT fallback for rotation-heavy cases." But:
1. At segment start, there's no heading from VO chain — no rectification possible.
2. No criteria for when to trigger SIFT fallback.
3. Multi-rotation matching strategy ({0°, 90°, 180°, 270°}) not mentioned.

### Conclusion
Three-tier rotation handling:
1. **Segment start (no heading)**: Try DINOv2 coarse retrieval (more rotation-robust than local features) → if match found, estimate heading from satellite alignment → proceed normally.
2. **Normal operation (heading available)**: Rectify to approximate north-up using accumulated heading → SuperPoint+LightGlue.
3. **Match failure fallback**: Try 4 rotations {0°, 90°, 180°, 270°} with SuperPoint. If still fails → SIFT+LightGlue (rotation-invariant).

Trigger for SIFT: SuperPoint inlier ratio < 0.15 after rotation retry.

### Confidence
✅ High — based on confirmed LightGlue limitation + proven SIFT+LightGlue alternative

---

## Dimension 4: VO Robustness (Homography Decomposition)

### Fact Confirmation
According to Fact #13, cv2.decomposeHomographyMat returns 4 solutions. Can return non-orthogonal matrices. Calibration matrix K precision is critical.

### Problem
Draft02 specifies selection by "motion consistent with previous direction + positive depth." This is underspecified for the first frame pair in a segment (no previous direction). Non-orthogonal R detection is missing.

### Conclusion
Disambiguation procedure:
1. Compute all 4 decompositions.
2. **Filter by positive depth**: triangulate a few matched points, reject solutions where points are behind camera.
3. **Filter by plane normal**: for downward-looking camera, the normal should approximately point up (positive z component in camera frame).
4. **Motion consistency**: if previous direction available, prefer solution consistent with expected motion direction.
5. **Orthogonality check**: verify R'R ≈ I, det(R) ≈ 1. If not, re-orthogonalize via SVD.
6. For first frame pair: rely on filters 2+3 only.

### Confidence
✅ High — based on well-documented decomposition ambiguity

---

## Dimension 5: Satellite Coarse Retrieval

### Fact Confirmation
According to Fact #15, DINOv2+SALAD outperforms raw DINOv2 CLS token for retrieval. According to Fact #5, DINOv2 achieves 86.27 R@1 with adaptive enhancement.

### Problem
Draft02 proposes "DINOv2 global retrieval + faiss cosine similarity." Using raw CLS token is suboptimal. SALAD or patch-level feature aggregation would improve retrieval accuracy.

### Conclusion
DINOv2+SALAD is the better approach but adds a training/fine-tuning dependency. For a production system without the ability to fine-tune: use DINOv2 patch tokens (not just CLS) with spatial pooling, then cosine similarity via faiss. This captures more spatial information than CLS alone. If time permits, train SALAD head (30 minutes on appropriate dataset).

Alternatively, consider SatDINO (DINOv2 pre-trained on satellite imagery) if available as a checkpoint.

### Confidence
⚠️ Medium — SALAD is proven but adding training dependency may not be worth the complexity for this use case

---

## Dimension 6: Concurrency & Pipeline Architecture

### Fact Confirmation
Single GPU (RTX 2060) cannot run two models concurrently. Fact #8 shows sequential model inference times. Fact #3 shows LightGlue ONNX at ~50-100ms.

### Problem
Draft02 says satellite matching is "async — don't block VO pipeline" but on a single GPU, you can't parallelize GPU inference.

### Conclusion
Pipeline design:
1. **VO (synchronous, per-frame)**: XFeat extract + match (~30ms total) → homography estimation (~5ms) → GTSAM update (~5ms) → emit position via SSE. **Total: ~40ms per frame.**
2. **Satellite matching (asynchronous, overlapped with next frame's VO)**: DINOv2 coarse (~50ms) → SuperPoint+LightGlue fine (~150ms) → GTSAM update (~5ms) → emit refined position. **Total: ~205ms but overlapped.**
3. **I/O (fully async)**: Tile download, DEM fetch, cache management — all via asyncio.
4. **CPU tasks (parallel)**: faiss search (CPU), homography RANSAC (CPU-bound but fast).

The GPU processes frames sequentially. The "async" part is that satellite matching for frame N happens while VO for frame N+1 proceeds. Since satellite matching (~205ms) is longer than VO (~40ms), the pipeline is satellite-matching-bound but VO results stream immediately.

### Confidence
✅ High — based on documented inference times

---

## Dimension 7: Security Attack Surface

### Fact Confirmation
According to Fact #14, Pillow CVE-2025-48379 affects image loading. Fact #12 confirms SSE cleanup issues.

### Problem
Draft02 has JWT + rate limiting + CORS but misses:
- Image format/magic byte validation before loading
- Pillow version pinning
- Memory-limited image loading (a 100,000 × 100,000 pixel image could OOM)
- SSE heartbeat for connection health
- No mention of directory traversal prevention depth

### Conclusion
Additional security measures:
1. Pin Pillow ≥11.3.0 in requirements.
2. Validate image magic bytes (JPEG/PNG/TIFF) before loading with PIL.
3. Check image dimensions before loading: reject if either dimension > 10,000px.
4. Use OpenCV for loading (separate from PIL) — validate separately.
5. Resolve image_folder path to canonical form (os.path.realpath) and verify it's under allowed base directories.
6. Add Content-Security-Policy headers.
7. SSE heartbeat every 15s to detect stale connections.
8. Implement asyncio.Queue-based event publisher for SSE.

### Confidence
✅ High — based on documented CVE + known SSE issues

---

## Dimension 8: Drift Management Strategy

### Fact Confirmation
According to Fact #17, NaviLoc demonstrates that trajectory-level optimization with noisy VPR measurements achieves 16x better accuracy than per-frame approaches. SatLoc-Fusion uses adaptive confidence metrics.

### Problem
Draft02's "drift limit factor" as a GTSAM custom factor is problematic: (1) custom Python factors are slow, (2) if no satellite anchors arrive for extended period, the drift factor has nothing to constrain against.

### Conclusion
Replace GTSAM drift factor with Segment Manager logic:
1. Track cumulative VO displacement since last satellite anchor.
2. If cumulative displacement > 100m without anchor: emit warning SSE event, increase satellite matching frequency/radius.
3. If cumulative displacement > 200m: request user input with timeout.
4. If cumulative displacement > 500m: mark segment as LOW confidence, continue but warn.
5. Confidence score per position: decays exponentially with distance from nearest anchor.

This is simpler, faster, and more controllable than a GTSAM custom factor.

### Confidence
✅ High — engineering judgment supported by SatLoc-Fusion's confidence-based approach
