# Acceptance Test: AC-6 - User Input Recovery ## Summary Validate Acceptance Criterion 6: "In case of being absolutely incapable of determining the system to determine next, second next, and third next images GPS, by any means (these 20% of the route), then it should ask the user for input for the next image, so that the user can specify the location." ## Linked Acceptance Criteria **AC-6**: User input requested after 3 consecutive failures ## Preconditions 1. ASTRAL-Next system operational 2. F11 Failure Recovery Coordinator configured with failure threshold = 3 3. F15 SSE Event Streamer functional 4. F01 Flight API accepting user-fix endpoint 5. F10 Factor Graph Optimizer ready to accept high-confidence anchors 6. Test environment configured to simulate L1/L2/L3 failures 7. SSE client connected and monitoring events ## Test Data - **Dataset**: AD000001-060 (60 images) - **Failure Injection**: Configure mock failures for specific frames - **Ground Truth**: coordinates.csv for validation ## Test Steps ### Step 1: Setup Failure Injection - **Action**: Configure system to fail L1, L2, L3 for frames AD000020, AD000021, AD000022 - **Expected Result**: - L1 (SuperPoint+LightGlue): Returns match_count < 10 - L2 (AnyLoc): Returns confidence < 0.3 - L3 (LiteSAM): Returns alignment_score < 0.2 ### Step 2: Process Normal Frames (1-19) - **Action**: Process AD000001-AD000019 normally - **Expected Result**: - All 19 frames processed successfully - No user input requests - SSE events: 19 × `frame_processed` ### Step 3: First Consecutive Failure - **Action**: Process AD000020 - **Expected Result**: - L1 fails (low match count) - L2 fallback fails (low confidence) - L3 fallback fails (low alignment) - System increments failure_count to 1 - SSE event: `frame_processing_failed` with frame_id=20 - **No user input request yet** ### Step 4: Second Consecutive Failure - **Action**: Process AD000021 - **Expected Result**: - All layers fail - failure_count incremented to 2 - SSE event: `frame_processing_failed` with frame_id=21 - **No user input request yet** ### Step 5: Third Consecutive Failure - Triggers User Input - **Action**: Process AD000022 - **Expected Result**: - All layers fail - failure_count reaches threshold (3) - F11 calls `create_user_input_request()` - SSE event: `user_input_required` - Event payload contains: ```json { "type": "user_input_required", "flight_id": "", "frame_id": 22, "failed_frames": [20, 21, 22], "candidate_tiles": [ {"tile_id": "xyz", "gps": {"lat": 48.27, "lon": 37.38}, "thumbnail_url": "..."}, {"tile_id": "abc", "gps": {"lat": 48.26, "lon": 37.37}, "thumbnail_url": "..."}, {"tile_id": "def", "gps": {"lat": 48.28, "lon": 37.39}, "thumbnail_url": "..."} ], "uav_image_url": "/flights//images/22", "message": "System unable to locate 3 consecutive images. Please provide GPS fix." } ``` ### Step 6: Validate Threshold Behavior - **Action**: Verify user input NOT requested before 3 failures - **Expected Result**: - Review event log: no `user_input_required` before frame 22 - Threshold is exactly 3 consecutive failures, not 2 or 4 ### Step 7: User Provides GPS Fix - **Action**: POST /flights/{flightId}/user-fix - **Payload**: ```json { "frame_id": 22, "uav_pixel": [3126, 2084], "satellite_gps": {"lat": 48.273997, "lon": 37.379828}, "confidence": "high" } ``` - **Expected Result**: - HTTP 200 OK - Response: `{"status": "accepted", "frame_id": 22}` ### Step 8: System Incorporates User Fix - **Action**: F11 processes user fix via `apply_user_anchor()` - **Expected Result**: - F10 adds GPS anchor with high confidence (weight = 10.0) - Factor graph re-optimizes - SSE event: `user_fix_applied` - Event payload: ```json { "type": "user_fix_applied", "frame_id": 22, "estimated_gps": {"lat": 48.273997, "lon": 37.379828}, "affected_frames": [20, 21, 22] } ``` ### Step 9: Trajectory Refinement - **Action**: Factor graph back-propagates fix to frames 20, 21 - **Expected Result**: - SSE event: `trajectory_refined` for frames 20, 21 - All 3 failed frames now have GPS estimates - failure_count reset to 0 ### Step 10: Processing Resumes Automatically - **Action**: System processes AD000023 and beyond - **Expected Result**: - Processing resumes without manual restart - AD000023+ processed normally (no more injected failures) - SSE events continue: `frame_processed` ### Step 11: Validate 20% Route Allowance - **Action**: Calculate maximum allowed user inputs for 60-image flight - **Expected Result**: - 20% of 60 = 12 images maximum can need user input - System tracks user_input_count per flight - If user_input_count > 12, system logs warning but continues ### Step 12: Test Multiple User Input Cycles - **Action**: Inject failures for frames AD000040, AD000041, AD000042 - **Expected Result**: - Second `user_input_required` event triggered - User provides second fix - System continues processing - Total user inputs: 2 cycles (6 frames aided) ### Step 13: Test User Input Timeout - **Action**: Trigger user input request, wait 5 minutes without response - **Expected Result**: - System sends reminder: `user_input_reminder` at 2 minutes - Processing remains paused for affected chunk - Other chunks (if any) continue processing - No timeout crash ### Step 14: Test Invalid User Fix - **Action**: Submit user fix with invalid GPS (outside geofence) - **Payload**: ```json { "frame_id": 22, "satellite_gps": {"lat": 0.0, "lon": 0.0} } ``` - **Expected Result**: - HTTP 400 Bad Request - Error: "GPS coordinates outside flight geofence" - System re-requests user input ### Step 15: Validate Final Flight Statistics - **Action**: GET /flights/{flightId}/status - **Expected Result**: ```json { "flight_id": "", "total_frames": 60, "processed_frames": 60, "user_input_requests": 2, "user_inputs_provided": 2, "frames_aided_by_user": 6, "user_input_percentage": 10.0 } ``` ## Success Criteria **Primary Criteria (AC-6)**: - User input requested after exactly 3 consecutive failures (not 2, not 4) - User notified via SSE with relevant context (candidate tiles, image URL) - User fix accepted via REST API - User fix incorporated as high-confidence GPS anchor - Processing resumes automatically after fix - System allows up to 20% of route to need user input **Supporting Criteria**: - SSE events delivered within 1 second - Factor graph incorporates fix within 2 seconds - Back-propagation refines earlier failed frames - failure_count resets after successful fix - System handles multiple user input cycles per flight ## Pass/Fail Criteria **TEST PASSES IF**: - User input request triggered at exactly 3 consecutive failures - SSE event contains all required info (frame_id, candidate tiles) - User fix accepted and incorporated - Processing resumes automatically - 20% allowance calculated correctly - Multiple cycles work correctly - Invalid fixes rejected gracefully **TEST FAILS IF**: - User input requested before 3 failures - User input NOT requested after 3 failures - SSE event missing required fields - User fix causes system error - Processing does not resume after fix - System crashes on invalid user input - Timeout causes system hang ## Error Scenarios ### Scenario A: User Provides Wrong GPS - User fix GPS is 500m from actual location - System accepts fix (user has authority) - Subsequent frames may fail again - Second user input cycle may be needed ### Scenario B: SSE Connection Lost - Client disconnects during user input wait - System buffers events - Client reconnects, receives pending events - Processing state preserved ### Scenario C: Database Failure During Fix - User fix received but DB write fails - System retries 3 times - If all retries fail, returns HTTP 503 - User can retry submission ## Components Involved - F01 Flight API: `POST /flights/{id}/user-fix` - F02.1 Flight Lifecycle Manager: `handle_user_fix()` - F02.2 Flight Processing Engine: `apply_user_fix()` - F10 Factor Graph Optimizer: `add_absolute_factor()` with high confidence - F11 Failure Recovery Coordinator: `create_user_input_request()`, `apply_user_anchor()` - F15 SSE Event Streamer: `send_user_input_request()`, `send_user_fix_applied()` ## Notes - AC-6 is the human-in-the-loop fallback for extreme failures - 3-failure threshold balances automation with user intervention - 20% allowance (12 of 60 images) is operational constraint - User fixes are trusted (high confidence weight in factor graph) - System should minimize user inputs via L1/L2/L3 layer defense