add features

docs/02_components/05_image_input_pipeline/05.01_feature_batch_queue_management.md

@@ -0,0 +1,61 @@
+# Feature: Batch Queue Management
+
+## Description
+
+Handles ingestion, validation, and FIFO queuing of image batches. This feature manages the entry point for all images into the system, ensuring sequence integrity and proper queuing for downstream processing.
+
+## Component APIs Implemented
+
+- `queue_batch(flight_id: str, batch: ImageBatch) -> bool`
+- `validate_batch(batch: ImageBatch) -> ValidationResult`
+- `process_next_batch(flight_id: str) -> Optional[ProcessedBatch]`
+
+## External Tools and Services
+
+- **H08 Batch Validator**: Delegated validation for naming convention, sequence continuity, format, dimensions
+- **Pillow**: Image decoding and metadata extraction
+- **opencv-python**: Image I/O operations
+
+## Internal Methods
+
+| Method | Purpose |
+|--------|---------|
+| `_add_to_queue(flight_id, batch)` | Adds validated batch to flight's FIFO queue |
+| `_dequeue_batch(flight_id)` | Removes and returns next batch from queue |
+| `_check_sequence_continuity(flight_id, batch)` | Validates batch continues from last processed sequence |
+| `_decode_images(batch)` | Decompresses/decodes raw image bytes to ImageData |
+| `_extract_metadata(image_bytes)` | Extracts EXIF, dimensions from raw image |
+| `_get_queue_capacity(flight_id)` | Returns remaining queue capacity for backpressure |
+
+## Unit Tests
+
+| Test | Description |
+|------|-------------|
+| `test_queue_batch_valid` | Valid batch queued successfully, returns True |
+| `test_queue_batch_sequence_gap` | Batch with sequence gap from last processed → ValidationError |
+| `test_queue_batch_invalid_naming` | Non-consecutive filenames → ValidationError |
+| `test_queue_batch_queue_full` | Queue at capacity → QueueFullError |
+| `test_validate_batch_size_min` | 9 images → invalid (min 10) |
+| `test_validate_batch_size_max` | 51 images → invalid (max 50) |
+| `test_validate_batch_naming_convention` | ADxxxxxx.jpg format validated |
+| `test_validate_batch_invalid_format` | IMG_0001.jpg → invalid |
+| `test_validate_batch_non_consecutive` | AD000101, AD000103 → invalid |
+| `test_validate_batch_file_format` | JPEG/PNG accepted, others rejected |
+| `test_validate_batch_dimensions` | Within 640x480 to 6252x4168 |
+| `test_validate_batch_file_size` | < 10MB per image |
+| `test_process_next_batch_dequeue` | Returns ProcessedBatch with decoded images |
+| `test_process_next_batch_empty_queue` | Empty queue → returns None |
+| `test_process_next_batch_corrupted_image` | Corrupted image skipped, others processed |
+| `test_process_next_batch_metadata_extraction` | EXIF and dimensions extracted correctly |
+| `test_fifo_order` | Multiple batches processed in queue order |
+
+## Integration Tests
+
+| Test | Description |
+|------|-------------|
+| `test_batch_flow_queue_to_process` | queue_batch → process_next_batch → verify ImageData list |
+| `test_multiple_batches_fifo` | Queue 5 batches, process in order, verify sequence maintained |
+| `test_batch_validation_with_h08` | Integration with H08 Batch Validator |
+| `test_concurrent_queue_access` | Multiple flights queuing simultaneously |
+| `test_backpressure_handling` | Queue fills up, backpressure signal returned |
+

docs/02_components/05_image_input_pipeline/05.02_feature_image_storage_retrieval.md

@@ -0,0 +1,70 @@
+# Feature: Image Storage and Retrieval
+
+## Description
+
+Handles persistent storage of processed images and provides retrieval mechanisms for sequential processing, random access, and metadata queries. Manages disk storage structure, maintains sequence tracking per flight, and provides processing status information.
+
+## Component APIs Implemented
+
+- `store_images(flight_id: str, images: List[ImageData]) -> bool`
+- `get_next_image(flight_id: str) -> Optional[ImageData]`
+- `get_image_by_sequence(flight_id: str, sequence: int) -> Optional[ImageData]`
+- `get_image_metadata(flight_id: str, sequence: int) -> Optional[ImageMetadata]`
+- `get_processing_status(flight_id: str) -> ProcessingStatus`
+
+## External Tools and Services
+
+- **F03 Flight Database**: Metadata persistence, flight state queries
+- **opencv-python**: Image I/O (cv2.imread, cv2.imwrite)
+- **numpy**: Image array handling
+
+## Internal Methods
+
+| Method | Purpose |
+|--------|---------|
+| `_create_flight_directory(flight_id)` | Creates storage directory structure for flight |
+| `_write_image(flight_id, filename, image_data)` | Writes single image to disk |
+| `_update_metadata_index(flight_id, metadata_list)` | Updates metadata.json with new image metadata |
+| `_load_image_from_disk(flight_id, filename)` | Reads image file and returns np.ndarray |
+| `_construct_filename(sequence)` | Converts sequence number to ADxxxxxx.jpg format |
+| `_get_sequence_tracker(flight_id)` | Gets/initializes current sequence position for flight |
+| `_increment_sequence(flight_id)` | Advances sequence counter after get_next_image |
+| `_load_metadata_from_index(flight_id, sequence)` | Reads metadata from index without loading image |
+| `_calculate_processing_rate(flight_id)` | Computes images/second processing rate |
+
+## Unit Tests
+
+| Test | Description |
+|------|-------------|
+| `test_store_images_success` | All images written to correct paths |
+| `test_store_images_creates_directory` | Flight directory created if not exists |
+| `test_store_images_updates_metadata` | metadata.json updated with image info |
+| `test_store_images_disk_full` | Storage error returns False |
+| `test_get_next_image_sequential` | Returns images in sequence order |
+| `test_get_next_image_increments_counter` | Sequence counter advances after each call |
+| `test_get_next_image_end_of_sequence` | Returns None when no more images |
+| `test_get_next_image_missing_file` | Handles missing image gracefully |
+| `test_get_image_by_sequence_valid` | Returns correct image for sequence number |
+| `test_get_image_by_sequence_invalid` | Invalid sequence returns None |
+| `test_get_image_by_sequence_constructs_filename` | Sequence 101 → AD000101.jpg |
+| `test_get_image_metadata_fast` | Returns metadata without loading full image |
+| `test_get_image_metadata_missing` | Missing image returns None |
+| `test_get_image_metadata_contains_fields` | Returns sequence, filename, dimensions, file_size, timestamp, exif |
+| `test_get_processing_status_counts` | Accurate total_images, processed_images counts |
+| `test_get_processing_status_current_sequence` | Reflects current processing position |
+| `test_get_processing_status_queued_batches` | Includes queue depth |
+| `test_get_processing_status_rate` | Processing rate calculation |
+
+## Integration Tests
+
+| Test | Description |
+|------|-------------|
+| `test_store_then_retrieve_sequential` | store_images → get_next_image × N → all images retrieved |
+| `test_store_then_retrieve_by_sequence` | store_images → get_image_by_sequence → correct image |
+| `test_metadata_persistence_f03` | store_images → metadata persisted to F03 Flight Database |
+| `test_crash_recovery_resume` | Restart processing from last stored sequence |
+| `test_concurrent_retrieval` | Multiple consumers retrieving images simultaneously |
+| `test_storage_large_batch` | Store and retrieve 3000 images for single flight |
+| `test_multiple_flights_isolation` | Multiple flights don't interfere with each other's storage |
+| `test_status_updates_realtime` | Status reflects current state during active processing |
+