add features

This commit is contained in:
Oleksandr Bezdieniezhnykh
2025-12-01 01:07:46 +02:00
parent 97f558b3d7
commit 54be35fde7
81 changed files with 4618 additions and 10 deletions
@@ -0,0 +1,64 @@
# Feature: Index Management
## Description
Load and manage pre-built satellite descriptor database (Faiss index). The semantic index is built by the satellite data provider offline using DINOv2 + VLAD - F08 only loads and validates the index. This is the foundation for all place recognition queries.
## Component APIs Implemented
- `load_index(flight_id: str, index_path: str) -> bool`
## External Tools and Services
- **H04 Faiss Index Manager**: For `load_index()`, `validate_index()` operations
- **Faiss**: Facebook similarity search library
## Internal Methods
### `_validate_index_integrity(index) -> bool`
Validates loaded Faiss index: checks descriptor dimensions (4096 or 8192), verifies tile count matches metadata.
### `_load_tile_metadata(metadata_path: str) -> Dict[int, TileMetadata]`
Loads tile_id → gps_center, bounds mapping from JSON file provided by satellite provider.
### `_verify_metadata_alignment(index, metadata: Dict) -> bool`
Ensures metadata entries match index size (same number of tiles).
## Unit Tests
### Test: Load Valid Index
- Input: Valid index file path from satellite provider
- Verify: Returns True, index operational
### Test: Index Not Found
- Input: Non-existent path
- Verify: Raises `IndexNotFoundError`
### Test: Corrupted Index
- Input: Corrupted/truncated index file
- Verify: Raises `IndexCorruptedError`
### Test: Dimension Validation
- Input: Index with wrong descriptor dimensions
- Verify: Raises `IndexCorruptedError` with descriptive message
### Test: Metadata Mismatch
- Input: Index with 1000 entries, metadata with 500 entries
- Verify: Raises `MetadataMismatchError`
### Test: Empty Metadata File
- Input: Valid index, empty metadata JSON
- Verify: Raises `MetadataMismatchError`
### Test: Load Performance
- Input: Index with 10,000 tiles
- Verify: Load completes in <10 seconds
## Integration Tests
### Test: Faiss Manager Integration
- Verify: Successfully delegates index loading to H04 Faiss Index Manager
- Verify: Index accessible for subsequent queries
### Test: Query After Load
- Setup: Load valid index
- Action: Query with random descriptor
- Verify: Returns valid matches (not empty, valid indices)
@@ -0,0 +1,95 @@
# Feature: Descriptor Computation
## Description
Compute global location descriptors using DINOv2 + VLAD aggregation. Supports both single-image descriptors and aggregate chunk descriptors for robust matching. DINOv2 features are semantic and invariant to season/texture changes, critical for UAV-to-satellite domain gap.
## Component APIs Implemented
- `compute_location_descriptor(image: np.ndarray) -> np.ndarray`
- `compute_chunk_descriptor(chunk_images: List[np.ndarray]) -> np.ndarray`
## External Tools and Services
- **F16 Model Manager**: Provides DINOv2 inference engine via `get_inference_engine("DINOv2")`
- **DINOv2**: Meta's foundation vision model for semantic feature extraction
- **numpy**: Array operations and L2 normalization
## Internal Methods
### `_preprocess_image(image: np.ndarray) -> np.ndarray`
Resizes and normalizes image for DINOv2 input (typically 224x224 or 518x518).
### `_extract_dense_features(preprocessed: np.ndarray) -> np.ndarray`
Runs DINOv2 inference, extracts dense feature map from multiple spatial locations.
### `_vlad_aggregate(dense_features: np.ndarray, codebook: np.ndarray) -> np.ndarray`
Applies VLAD (Vector of Locally Aggregated Descriptors) aggregation using pre-trained cluster centers.
### `_l2_normalize(descriptor: np.ndarray) -> np.ndarray`
L2-normalizes descriptor vector for cosine similarity search.
### `_aggregate_chunk_descriptors(descriptors: List[np.ndarray], strategy: str) -> np.ndarray`
Aggregates multiple descriptors into one using strategy: "mean" (default), "vlad", or "max".
## Unit Tests
### compute_location_descriptor
#### Test: Output Dimensions
- Input: UAV image (any resolution)
- Verify: Returns 4096-dim or 8192-dim vector
#### Test: Normalization
- Input: Any valid image
- Verify: Output L2 norm equals 1.0
#### Test: Deterministic Output
- Input: Same image twice
- Verify: Identical descriptors (no randomness)
#### Test: Season Invariance
- Input: Two images of same location, different seasons
- Verify: Cosine similarity > 0.7
#### Test: Location Discrimination
- Input: Two images of different locations
- Verify: Cosine similarity < 0.5
#### Test: Domain Invariance
- Input: UAV image and satellite image of same location
- Verify: Cosine similarity > 0.6 (cross-domain)
### compute_chunk_descriptor
#### Test: Empty Chunk
- Input: Empty list
- Verify: Raises ValueError
#### Test: Single Image Chunk
- Input: List with one image
- Verify: Equivalent to compute_location_descriptor
#### Test: Multiple Images Mean Aggregation
- Input: 5 images from chunk
- Verify: Descriptor is mean of individual descriptors
#### Test: Aggregated Normalization
- Input: Any chunk
- Verify: Output L2 norm equals 1.0
#### Test: Chunk More Robust Than Single
- Input: Featureless terrain chunk (10 images)
- Verify: Chunk descriptor has lower variance than individual descriptors
## Integration Tests
### Test: Model Manager Integration
- Verify: Successfully retrieves DINOv2 from F16 Model Manager
- Verify: Model loaded with correct TensorRT/ONNX backend
### Test: Performance Budget
- Input: FullHD image
- Verify: Single descriptor computed in ~150ms
### Test: Chunk Performance
- Input: 10 images
- Verify: Chunk descriptor in <2s (parallelizable)
@@ -0,0 +1,130 @@
# Feature: Candidate Retrieval
## Description
Query satellite database and retrieve ranked tile candidates. Orchestrates the full place recognition pipeline: descriptor computation → database query → candidate ranking. Supports both single-image and chunk-based retrieval for "kidnapped robot" recovery.
## Component APIs Implemented
- `query_database(descriptor: np.ndarray, top_k: int) -> List[DatabaseMatch]`
- `rank_candidates(candidates: List[TileCandidate]) -> List[TileCandidate]`
- `retrieve_candidate_tiles(image: np.ndarray, top_k: int) -> List[TileCandidate]`
- `retrieve_candidate_tiles_for_chunk(chunk_images: List[np.ndarray], top_k: int) -> List[TileCandidate]`
## External Tools and Services
- **H04 Faiss Index Manager**: For `search()` operation
- **F04 Satellite Data Manager**: For tile metadata retrieval (gps_center, bounds) after Faiss returns indices
- **F12 Route Chunk Manager**: Provides chunk images for chunk-based retrieval
## Internal Methods
### `_distance_to_similarity(distance: float) -> float`
Converts L2 distance to normalized similarity score [0, 1].
### `_retrieve_tile_metadata(indices: List[int]) -> List[TileMetadata]`
Fetches GPS center and bounds for tile indices from F04 Satellite Data Manager.
### `_apply_spatial_reranking(candidates: List[TileCandidate], dead_reckoning_estimate: Optional[GPSPoint]) -> List[TileCandidate]`
Re-ranks candidates based on proximity to dead-reckoning estimate if available.
### `_apply_trajectory_reranking(candidates: List[TileCandidate], previous_trajectory: Optional[List[GPSPoint]]) -> List[TileCandidate]`
Favors tiles that continue the previous trajectory direction.
### `_filter_by_geofence(candidates: List[TileCandidate], geofence: Optional[BoundingBox]) -> List[TileCandidate]`
Removes candidates outside operational geofence.
## Unit Tests
### query_database
#### Test: Returns Top-K Matches
- Input: Valid descriptor, top_k=5
- Verify: Returns exactly 5 DatabaseMatch objects
#### Test: Ordered by Distance
- Input: Any descriptor
- Verify: Matches sorted by ascending distance
#### Test: Similarity Score Range
- Input: Any query
- Verify: All similarity_score values in [0, 1]
#### Test: Empty Database
- Input: Query when no index loaded
- Verify: Returns empty list (not exception)
#### Test: Query Performance
- Input: Large database (10,000 tiles)
- Verify: Query completes in <50ms
### rank_candidates
#### Test: Preserves Order Without Heuristics
- Input: Candidates without dead-reckoning estimate
- Verify: Order unchanged (similarity-based)
#### Test: Spatial Reranking Applied
- Input: Candidates + dead-reckoning estimate
- Verify: Closer tile promoted in ranking
#### Test: Tie Breaking
- Input: Two candidates with similar similarity scores
- Verify: Spatial proximity breaks tie
#### Test: Geofence Filtering
- Input: Candidates with some outside geofence
- Verify: Out-of-bounds candidates removed
### retrieve_candidate_tiles
#### Test: End-to-End Single Image
- Input: UAV image, top_k=5
- Verify: Returns 5 TileCandidate with valid gps_center
#### Test: Correct Tile in Top-5
- Input: UAV image with known location
- Verify: Correct tile appears in top-5 (Recall@5 test)
#### Test: Performance Budget
- Input: FullHD UAV image
- Verify: Total time <200ms (descriptor ~150ms + query ~50ms)
### retrieve_candidate_tiles_for_chunk
#### Test: End-to-End Chunk
- Input: 10 chunk images, top_k=5
- Verify: Returns 5 TileCandidate
#### Test: Chunk More Accurate Than Single
- Input: Featureless terrain images
- Verify: Chunk retrieval finds correct tile where single-image fails
#### Test: Recall@5 > 90%
- Input: Various chunk scenarios
- Verify: Correct tile in top-5 at least 90% of test cases
## Integration Tests
### Test: Faiss Manager Integration
- Verify: query_database correctly delegates to H04 Faiss Index Manager
### Test: Satellite Data Manager Integration
- Verify: Tile metadata correctly retrieved from F04 after Faiss query
### Test: Full Pipeline Single Image
- Setup: Load index, prepare UAV image
- Action: retrieve_candidate_tiles()
- Verify: Returns valid candidates with GPS coordinates
### Test: Full Pipeline Chunk
- Setup: Load index, prepare chunk images
- Action: retrieve_candidate_tiles_for_chunk()
- Verify: Returns valid candidates, more robust than single-image
### Test: Season Invariance
- Setup: Satellite tiles from summer, UAV image from autumn
- Action: retrieve_candidate_tiles()
- Verify: Correct match despite appearance change
### Test: Recall@5 Benchmark
- Input: Test dataset of 100 UAV images with ground truth
- Verify: Recall@5 > 85% for single-image, > 90% for chunk