# Feature: Combined Neural Inference

## Description
Single-pass SuperPoint+LightGlue TensorRT inference for feature extraction and matching. Takes two images as input and outputs matched keypoints directly, eliminating intermediate feature transfer overhead.

## Component APIs Implemented
- `extract_and_match(image1: np.ndarray, image2: np.ndarray) -> Matches`

## External Tools and Services
- **Combined SuperPoint+LightGlue TensorRT Engine**: Single model combining extraction and matching
- **F16 Model Manager**: Provides pre-loaded TensorRT engine instance
- **Reference**: [D_VINS](https://github.com/kajo-kurisu/D_VINS/) for TensorRT optimization patterns

## Internal Methods

### `_preprocess_images(image1: np.ndarray, image2: np.ndarray) -> Tuple[np.ndarray, np.ndarray]`
Converts images to grayscale if needed, normalizes pixel values, resizes to model input dimensions.

### `_run_combined_inference(img1_tensor: np.ndarray, img2_tensor: np.ndarray) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]`
Executes combined TensorRT engine. Returns matched keypoints from both images and match confidence scores.

**Internal Pipeline** (within single inference):
1. SuperPoint extracts keypoints + descriptors from both images
2. LightGlue performs attention-based matching with adaptive depth
3. Dustbin mechanism filters unmatched features
4. Returns only matched keypoint pairs

### `_filter_matches_by_confidence(keypoints1: np.ndarray, keypoints2: np.ndarray, scores: np.ndarray, threshold: float) -> Matches`
Filters low-confidence matches, constructs final Matches object.

## Architecture Notes

### Combined Model Benefits
- Single GPU memory transfer (both images together)
- No intermediate descriptor serialization
- Optimized attention layers for batch processing
- Adaptive depth exits early for easy (high-overlap) pairs

### TensorRT Engine Configuration
```
Input shapes:
  image1: (1, 1, H, W) - grayscale
  image2: (1, 1, H, W) - grayscale
  
Output shapes:
  keypoints1: (1, M, 2) - matched keypoints from image1
  keypoints2: (1, M, 2) - matched keypoints from image2
  scores: (1, M) - match confidence scores
```

### Model Export (reference from D_VINS)
```bash
trtexec --onnx='superpoint_lightglue_combined.onnx' \
  --fp16 \
  --minShapes=image1:1x1x480x752,image2:1x1x480x752 \
  --optShapes=image1:1x1x480x752,image2:1x1x480x752 \
  --maxShapes=image1:1x1x480x752,image2:1x1x480x752 \
  --saveEngine=sp_lg_combined.engine \
  --warmUp=500 --duration=10
```

## Unit Tests

### Test: Grayscale Conversion
- Input: Two RGB images (H×W×3)
- Verify: _preprocess_images returns grayscale tensors

### Test: Grayscale Passthrough
- Input: Two grayscale images (H×W)
- Verify: _preprocess_images returns unchanged

### Test: High Overlap Matching
- Input: Two images with >50% overlap
- Verify: Returns 500+ matches
- Verify: Inference time ~35-50ms (adaptive depth fast path)

### Test: Low Overlap Matching
- Input: Two images with 5-10% overlap
- Verify: Returns 20-50 matches
- Verify: Inference time ~80ms (full depth)

### Test: No Overlap Handling
- Input: Two non-overlapping images
- Verify: Returns <10 matches
- Verify: No exception raised

### Test: Confidence Score Range
- Input: Any valid image pair
- Verify: All scores in [0, 1] range

### Test: Empty/Invalid Image Handling
- Input: Black/invalid image pair
- Verify: Returns empty Matches (never raises exception)

### Test: High Resolution Images
- Input: Two 6252×4168 images
- Verify: Preprocessing resizes appropriately
- Verify: Completes within performance budget

### Test: Output Shape Consistency
- Input: Any valid image pair
- Verify: keypoints1.shape[0] == keypoints2.shape[0] == scores.shape[0]

## Integration Tests

### Test: Model Manager Integration
- Verify: Successfully retrieves combined SP+LG engine from F16
- Verify: Engine loaded with correct TensorRT backend

### Test: Performance Budget
- Input: Two FullHD images
- Verify: Combined inference completes in <80ms on RTX 2060

### Test: Adaptive Depth Behavior
- Input: High overlap pair, then low overlap pair
- Verify: High overlap completes faster than low overlap

### Test: Agricultural Texture Handling
- Input: Two wheat field images with repetitive patterns
- Verify: Produces valid matches despite repetitive textures

### Test: Memory Efficiency
- Verify: Single GPU memory allocation vs two separate models
- Verify: No intermediate descriptor buffer allocation

### Test: Batch Consistency
- Input: Same image pair multiple times
- Verify: Consistent match results (deterministic)

