mirror of
https://github.com/azaion/gps-denied-desktop.git
synced 2026-04-23 02:46:35 +00:00
add features
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
# Feature: Index Management
|
||||
|
||||
## Description
|
||||
Load and manage pre-built satellite descriptor database (Faiss index). The semantic index is built by the satellite data provider offline using DINOv2 + VLAD - F08 only loads and validates the index. This is the foundation for all place recognition queries.
|
||||
|
||||
## Component APIs Implemented
|
||||
- `load_index(flight_id: str, index_path: str) -> bool`
|
||||
|
||||
## External Tools and Services
|
||||
- **H04 Faiss Index Manager**: For `load_index()`, `validate_index()` operations
|
||||
- **Faiss**: Facebook similarity search library
|
||||
|
||||
## Internal Methods
|
||||
|
||||
### `_validate_index_integrity(index) -> bool`
|
||||
Validates loaded Faiss index: checks descriptor dimensions (4096 or 8192), verifies tile count matches metadata.
|
||||
|
||||
### `_load_tile_metadata(metadata_path: str) -> Dict[int, TileMetadata]`
|
||||
Loads tile_id → gps_center, bounds mapping from JSON file provided by satellite provider.
|
||||
|
||||
### `_verify_metadata_alignment(index, metadata: Dict) -> bool`
|
||||
Ensures metadata entries match index size (same number of tiles).
|
||||
|
||||
## Unit Tests
|
||||
|
||||
### Test: Load Valid Index
|
||||
- Input: Valid index file path from satellite provider
|
||||
- Verify: Returns True, index operational
|
||||
|
||||
### Test: Index Not Found
|
||||
- Input: Non-existent path
|
||||
- Verify: Raises `IndexNotFoundError`
|
||||
|
||||
### Test: Corrupted Index
|
||||
- Input: Corrupted/truncated index file
|
||||
- Verify: Raises `IndexCorruptedError`
|
||||
|
||||
### Test: Dimension Validation
|
||||
- Input: Index with wrong descriptor dimensions
|
||||
- Verify: Raises `IndexCorruptedError` with descriptive message
|
||||
|
||||
### Test: Metadata Mismatch
|
||||
- Input: Index with 1000 entries, metadata with 500 entries
|
||||
- Verify: Raises `MetadataMismatchError`
|
||||
|
||||
### Test: Empty Metadata File
|
||||
- Input: Valid index, empty metadata JSON
|
||||
- Verify: Raises `MetadataMismatchError`
|
||||
|
||||
### Test: Load Performance
|
||||
- Input: Index with 10,000 tiles
|
||||
- Verify: Load completes in <10 seconds
|
||||
|
||||
## Integration Tests
|
||||
|
||||
### Test: Faiss Manager Integration
|
||||
- Verify: Successfully delegates index loading to H04 Faiss Index Manager
|
||||
- Verify: Index accessible for subsequent queries
|
||||
|
||||
### Test: Query After Load
|
||||
- Setup: Load valid index
|
||||
- Action: Query with random descriptor
|
||||
- Verify: Returns valid matches (not empty, valid indices)
|
||||
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
# Feature: Descriptor Computation
|
||||
|
||||
## Description
|
||||
Compute global location descriptors using DINOv2 + VLAD aggregation. Supports both single-image descriptors and aggregate chunk descriptors for robust matching. DINOv2 features are semantic and invariant to season/texture changes, critical for UAV-to-satellite domain gap.
|
||||
|
||||
## Component APIs Implemented
|
||||
- `compute_location_descriptor(image: np.ndarray) -> np.ndarray`
|
||||
- `compute_chunk_descriptor(chunk_images: List[np.ndarray]) -> np.ndarray`
|
||||
|
||||
## External Tools and Services
|
||||
- **F16 Model Manager**: Provides DINOv2 inference engine via `get_inference_engine("DINOv2")`
|
||||
- **DINOv2**: Meta's foundation vision model for semantic feature extraction
|
||||
- **numpy**: Array operations and L2 normalization
|
||||
|
||||
## Internal Methods
|
||||
|
||||
### `_preprocess_image(image: np.ndarray) -> np.ndarray`
|
||||
Resizes and normalizes image for DINOv2 input (typically 224x224 or 518x518).
|
||||
|
||||
### `_extract_dense_features(preprocessed: np.ndarray) -> np.ndarray`
|
||||
Runs DINOv2 inference, extracts dense feature map from multiple spatial locations.
|
||||
|
||||
### `_vlad_aggregate(dense_features: np.ndarray, codebook: np.ndarray) -> np.ndarray`
|
||||
Applies VLAD (Vector of Locally Aggregated Descriptors) aggregation using pre-trained cluster centers.
|
||||
|
||||
### `_l2_normalize(descriptor: np.ndarray) -> np.ndarray`
|
||||
L2-normalizes descriptor vector for cosine similarity search.
|
||||
|
||||
### `_aggregate_chunk_descriptors(descriptors: List[np.ndarray], strategy: str) -> np.ndarray`
|
||||
Aggregates multiple descriptors into one using strategy: "mean" (default), "vlad", or "max".
|
||||
|
||||
## Unit Tests
|
||||
|
||||
### compute_location_descriptor
|
||||
|
||||
#### Test: Output Dimensions
|
||||
- Input: UAV image (any resolution)
|
||||
- Verify: Returns 4096-dim or 8192-dim vector
|
||||
|
||||
#### Test: Normalization
|
||||
- Input: Any valid image
|
||||
- Verify: Output L2 norm equals 1.0
|
||||
|
||||
#### Test: Deterministic Output
|
||||
- Input: Same image twice
|
||||
- Verify: Identical descriptors (no randomness)
|
||||
|
||||
#### Test: Season Invariance
|
||||
- Input: Two images of same location, different seasons
|
||||
- Verify: Cosine similarity > 0.7
|
||||
|
||||
#### Test: Location Discrimination
|
||||
- Input: Two images of different locations
|
||||
- Verify: Cosine similarity < 0.5
|
||||
|
||||
#### Test: Domain Invariance
|
||||
- Input: UAV image and satellite image of same location
|
||||
- Verify: Cosine similarity > 0.6 (cross-domain)
|
||||
|
||||
### compute_chunk_descriptor
|
||||
|
||||
#### Test: Empty Chunk
|
||||
- Input: Empty list
|
||||
- Verify: Raises ValueError
|
||||
|
||||
#### Test: Single Image Chunk
|
||||
- Input: List with one image
|
||||
- Verify: Equivalent to compute_location_descriptor
|
||||
|
||||
#### Test: Multiple Images Mean Aggregation
|
||||
- Input: 5 images from chunk
|
||||
- Verify: Descriptor is mean of individual descriptors
|
||||
|
||||
#### Test: Aggregated Normalization
|
||||
- Input: Any chunk
|
||||
- Verify: Output L2 norm equals 1.0
|
||||
|
||||
#### Test: Chunk More Robust Than Single
|
||||
- Input: Featureless terrain chunk (10 images)
|
||||
- Verify: Chunk descriptor has lower variance than individual descriptors
|
||||
|
||||
## Integration Tests
|
||||
|
||||
### Test: Model Manager Integration
|
||||
- Verify: Successfully retrieves DINOv2 from F16 Model Manager
|
||||
- Verify: Model loaded with correct TensorRT/ONNX backend
|
||||
|
||||
### Test: Performance Budget
|
||||
- Input: FullHD image
|
||||
- Verify: Single descriptor computed in ~150ms
|
||||
|
||||
### Test: Chunk Performance
|
||||
- Input: 10 images
|
||||
- Verify: Chunk descriptor in <2s (parallelizable)
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
# Feature: Candidate Retrieval
|
||||
|
||||
## Description
|
||||
Query satellite database and retrieve ranked tile candidates. Orchestrates the full place recognition pipeline: descriptor computation → database query → candidate ranking. Supports both single-image and chunk-based retrieval for "kidnapped robot" recovery.
|
||||
|
||||
## Component APIs Implemented
|
||||
- `query_database(descriptor: np.ndarray, top_k: int) -> List[DatabaseMatch]`
|
||||
- `rank_candidates(candidates: List[TileCandidate]) -> List[TileCandidate]`
|
||||
- `retrieve_candidate_tiles(image: np.ndarray, top_k: int) -> List[TileCandidate]`
|
||||
- `retrieve_candidate_tiles_for_chunk(chunk_images: List[np.ndarray], top_k: int) -> List[TileCandidate]`
|
||||
|
||||
## External Tools and Services
|
||||
- **H04 Faiss Index Manager**: For `search()` operation
|
||||
- **F04 Satellite Data Manager**: For tile metadata retrieval (gps_center, bounds) after Faiss returns indices
|
||||
- **F12 Route Chunk Manager**: Provides chunk images for chunk-based retrieval
|
||||
|
||||
## Internal Methods
|
||||
|
||||
### `_distance_to_similarity(distance: float) -> float`
|
||||
Converts L2 distance to normalized similarity score [0, 1].
|
||||
|
||||
### `_retrieve_tile_metadata(indices: List[int]) -> List[TileMetadata]`
|
||||
Fetches GPS center and bounds for tile indices from F04 Satellite Data Manager.
|
||||
|
||||
### `_apply_spatial_reranking(candidates: List[TileCandidate], dead_reckoning_estimate: Optional[GPSPoint]) -> List[TileCandidate]`
|
||||
Re-ranks candidates based on proximity to dead-reckoning estimate if available.
|
||||
|
||||
### `_apply_trajectory_reranking(candidates: List[TileCandidate], previous_trajectory: Optional[List[GPSPoint]]) -> List[TileCandidate]`
|
||||
Favors tiles that continue the previous trajectory direction.
|
||||
|
||||
### `_filter_by_geofence(candidates: List[TileCandidate], geofence: Optional[BoundingBox]) -> List[TileCandidate]`
|
||||
Removes candidates outside operational geofence.
|
||||
|
||||
## Unit Tests
|
||||
|
||||
### query_database
|
||||
|
||||
#### Test: Returns Top-K Matches
|
||||
- Input: Valid descriptor, top_k=5
|
||||
- Verify: Returns exactly 5 DatabaseMatch objects
|
||||
|
||||
#### Test: Ordered by Distance
|
||||
- Input: Any descriptor
|
||||
- Verify: Matches sorted by ascending distance
|
||||
|
||||
#### Test: Similarity Score Range
|
||||
- Input: Any query
|
||||
- Verify: All similarity_score values in [0, 1]
|
||||
|
||||
#### Test: Empty Database
|
||||
- Input: Query when no index loaded
|
||||
- Verify: Returns empty list (not exception)
|
||||
|
||||
#### Test: Query Performance
|
||||
- Input: Large database (10,000 tiles)
|
||||
- Verify: Query completes in <50ms
|
||||
|
||||
### rank_candidates
|
||||
|
||||
#### Test: Preserves Order Without Heuristics
|
||||
- Input: Candidates without dead-reckoning estimate
|
||||
- Verify: Order unchanged (similarity-based)
|
||||
|
||||
#### Test: Spatial Reranking Applied
|
||||
- Input: Candidates + dead-reckoning estimate
|
||||
- Verify: Closer tile promoted in ranking
|
||||
|
||||
#### Test: Tie Breaking
|
||||
- Input: Two candidates with similar similarity scores
|
||||
- Verify: Spatial proximity breaks tie
|
||||
|
||||
#### Test: Geofence Filtering
|
||||
- Input: Candidates with some outside geofence
|
||||
- Verify: Out-of-bounds candidates removed
|
||||
|
||||
### retrieve_candidate_tiles
|
||||
|
||||
#### Test: End-to-End Single Image
|
||||
- Input: UAV image, top_k=5
|
||||
- Verify: Returns 5 TileCandidate with valid gps_center
|
||||
|
||||
#### Test: Correct Tile in Top-5
|
||||
- Input: UAV image with known location
|
||||
- Verify: Correct tile appears in top-5 (Recall@5 test)
|
||||
|
||||
#### Test: Performance Budget
|
||||
- Input: FullHD UAV image
|
||||
- Verify: Total time <200ms (descriptor ~150ms + query ~50ms)
|
||||
|
||||
### retrieve_candidate_tiles_for_chunk
|
||||
|
||||
#### Test: End-to-End Chunk
|
||||
- Input: 10 chunk images, top_k=5
|
||||
- Verify: Returns 5 TileCandidate
|
||||
|
||||
#### Test: Chunk More Accurate Than Single
|
||||
- Input: Featureless terrain images
|
||||
- Verify: Chunk retrieval finds correct tile where single-image fails
|
||||
|
||||
#### Test: Recall@5 > 90%
|
||||
- Input: Various chunk scenarios
|
||||
- Verify: Correct tile in top-5 at least 90% of test cases
|
||||
|
||||
## Integration Tests
|
||||
|
||||
### Test: Faiss Manager Integration
|
||||
- Verify: query_database correctly delegates to H04 Faiss Index Manager
|
||||
|
||||
### Test: Satellite Data Manager Integration
|
||||
- Verify: Tile metadata correctly retrieved from F04 after Faiss query
|
||||
|
||||
### Test: Full Pipeline Single Image
|
||||
- Setup: Load index, prepare UAV image
|
||||
- Action: retrieve_candidate_tiles()
|
||||
- Verify: Returns valid candidates with GPS coordinates
|
||||
|
||||
### Test: Full Pipeline Chunk
|
||||
- Setup: Load index, prepare chunk images
|
||||
- Action: retrieve_candidate_tiles_for_chunk()
|
||||
- Verify: Returns valid candidates, more robust than single-image
|
||||
|
||||
### Test: Season Invariance
|
||||
- Setup: Satellite tiles from summer, UAV image from autumn
|
||||
- Action: retrieve_candidate_tiles()
|
||||
- Verify: Correct match despite appearance change
|
||||
|
||||
### Test: Recall@5 Benchmark
|
||||
- Input: Test dataset of 100 UAV images with ground truth
|
||||
- Verify: Recall@5 > 85% for single-image, > 90% for chunk
|
||||
|
||||
Reference in New Issue
Block a user