gps-denied-onboard/docs/03_tests/40_user_input_recovery_spec.md

Acceptance Test: AC-6 - User Input Recovery

Summary

Validate Acceptance Criterion 6: "In case of being absolutely incapable of determining the system to determine next, second next, and third next images GPS, by any means (these 20% of the route), then it should ask the user for input for the next image, so that the user can specify the location."

Linked Acceptance Criteria

AC-6: User input requested after 3 consecutive failures

Preconditions

1. ASTRAL-Next system operational
2. F11 Failure Recovery Coordinator configured with failure threshold = 3
3. F15 SSE Event Streamer functional
4. F01 Flight API accepting user-fix endpoint
5. F10 Factor Graph Optimizer ready to accept high-confidence anchors
6. Test environment configured to simulate L1/L2/L3 failures
7. SSE client connected and monitoring events

Test Data

• Dataset: AD000001-060 (60 images)
• Failure Injection: Configure mock failures for specific frames
• Ground Truth: coordinates.csv for validation

Test Steps

Step 1: Setup Failure Injection

• Action: Configure system to fail L1, L2, L3 for frames AD000020, AD000021, AD000022
• Expected Result:
  • L1 (SuperPoint+LightGlue): Returns match_count < 10
  • L2 (AnyLoc): Returns confidence < 0.3
  • L3 (LiteSAM): Returns alignment_score < 0.2

Step 2: Process Normal Frames (1-19)

• Action: Process AD000001-AD000019 normally
• Expected Result:
  • All 19 frames processed successfully
  • No user input requests
  • SSE events: 19 × frame_processed

Step 3: First Consecutive Failure

• Action: Process AD000020
• Expected Result:
  • L1 fails (low match count)
  • L2 fallback fails (low confidence)
  • L3 fallback fails (low alignment)
  • System increments failure_count to 1
  • SSE event: frame_processing_failed with frame_id=20
  • No user input request yet

Step 4: Second Consecutive Failure

• Action: Process AD000021
• Expected Result:
  • All layers fail
  • failure_count incremented to 2
  • SSE event: frame_processing_failed with frame_id=21
  • No user input request yet

Step 5: Third Consecutive Failure - Triggers User Input

• Action: Process AD000022
• Expected Result:
  • All layers fail
  • failure_count reaches threshold (3)
  • F11 calls create_user_input_request()
  • SSE event: user_input_required
  • Event payload contains:

    {
      "type": "user_input_required",
      "flight_id": "<flight_id>",
      "frame_id": 22,
      "failed_frames": [20, 21, 22],
      "candidate_tiles": [
        {"tile_id": "xyz", "gps": {"lat": 48.27, "lon": 37.38}, "thumbnail_url": "..."},
        {"tile_id": "abc", "gps": {"lat": 48.26, "lon": 37.37}, "thumbnail_url": "..."},
        {"tile_id": "def", "gps": {"lat": 48.28, "lon": 37.39}, "thumbnail_url": "..."}
      ],
      "uav_image_url": "/flights/<id>/images/22",
      "message": "System unable to locate 3 consecutive images. Please provide GPS fix."
    }
    

Step 6: Validate Threshold Behavior

• Action: Verify user input NOT requested before 3 failures
• Expected Result:
  • Review event log: no user_input_required before frame 22
  • Threshold is exactly 3 consecutive failures, not 2 or 4

Step 7: User Provides GPS Fix

• Action: POST /flights/{flightId}/user-fix
• Payload:

  {
    "frame_id": 22,
    "uav_pixel": [3126, 2084],
    "satellite_gps": {"lat": 48.273997, "lon": 37.379828},
    "confidence": "high"
  }
  

• Expected Result:
  • HTTP 200 OK
  • Response: {"status": "accepted", "frame_id": 22}

Step 8: System Incorporates User Fix

• Action: F11 processes user fix via apply_user_anchor()
• Expected Result:
  • F10 adds GPS anchor with high confidence (weight = 10.0)
  • Factor graph re-optimizes
  • SSE event: user_fix_applied
  • Event payload:

    {
      "type": "user_fix_applied",
      "frame_id": 22,
      "estimated_gps": {"lat": 48.273997, "lon": 37.379828},
      "affected_frames": [20, 21, 22]
    }
    

Step 9: Trajectory Refinement

• Action: Factor graph back-propagates fix to frames 20, 21
• Expected Result:
  • SSE event: trajectory_refined for frames 20, 21
  • All 3 failed frames now have GPS estimates
  • failure_count reset to 0

Step 10: Processing Resumes Automatically

• Action: System processes AD000023 and beyond
• Expected Result:
  • Processing resumes without manual restart
  • AD000023+ processed normally (no more injected failures)
  • SSE events continue: frame_processed

Step 11: Validate 20% Route Allowance

• Action: Calculate maximum allowed user inputs for 60-image flight
• Expected Result:
  • 20% of 60 = 12 images maximum can need user input
  • System tracks user_input_count per flight
  • If user_input_count > 12, system logs warning but continues

Step 12: Test Multiple User Input Cycles

• Action: Inject failures for frames AD000040, AD000041, AD000042
• Expected Result:
  • Second user_input_required event triggered
  • User provides second fix
  • System continues processing
  • Total user inputs: 2 cycles (6 frames aided)

Step 13: Test User Input Timeout

• Action: Trigger user input request, wait 5 minutes without response
• Expected Result:
  • System sends reminder: user_input_reminder at 2 minutes
  • Processing remains paused for affected chunk
  • Other chunks (if any) continue processing
  • No timeout crash

Step 14: Test Invalid User Fix

• Action: Submit user fix with invalid GPS (outside geofence)
• Payload:

  {
    "frame_id": 22,
    "satellite_gps": {"lat": 0.0, "lon": 0.0}
  }
  

• Expected Result:
  • HTTP 400 Bad Request
  • Error: "GPS coordinates outside flight geofence"
  • System re-requests user input

Step 15: Validate Final Flight Statistics

• Action: GET /flights/{flightId}/status
• Expected Result:

  {
    "flight_id": "<id>",
    "total_frames": 60,
    "processed_frames": 60,
    "user_input_requests": 2,
    "user_inputs_provided": 2,
    "frames_aided_by_user": 6,
    "user_input_percentage": 10.0
  }
  

Success Criteria

Primary Criteria (AC-6):

• User input requested after exactly 3 consecutive failures (not 2, not 4)
• User notified via SSE with relevant context (candidate tiles, image URL)
• User fix accepted via REST API
• User fix incorporated as high-confidence GPS anchor
• Processing resumes automatically after fix
• System allows up to 20% of route to need user input

Supporting Criteria:

• SSE events delivered within 1 second
• Factor graph incorporates fix within 2 seconds
• Back-propagation refines earlier failed frames
• failure_count resets after successful fix
• System handles multiple user input cycles per flight

Pass/Fail Criteria

TEST PASSES IF:

• User input request triggered at exactly 3 consecutive failures
• SSE event contains all required info (frame_id, candidate tiles)
• User fix accepted and incorporated
• Processing resumes automatically
• 20% allowance calculated correctly
• Multiple cycles work correctly
• Invalid fixes rejected gracefully

TEST FAILS IF:

• User input requested before 3 failures
• User input NOT requested after 3 failures
• SSE event missing required fields
• User fix causes system error
• Processing does not resume after fix
• System crashes on invalid user input
• Timeout causes system hang

Error Scenarios

Scenario A: User Provides Wrong GPS

• User fix GPS is 500m from actual location
• System accepts fix (user has authority)
• Subsequent frames may fail again
• Second user input cycle may be needed

Scenario B: SSE Connection Lost

• Client disconnects during user input wait
• System buffers events
• Client reconnects, receives pending events
• Processing state preserved

Scenario C: Database Failure During Fix

• User fix received but DB write fails
• System retries 3 times
• If all retries fail, returns HTTP 503
• User can retry submission

Components Involved

• F01 Flight API: POST /flights/{id}/user-fix
• F02.1 Flight Lifecycle Manager: handle_user_fix()
• F02.2 Flight Processing Engine: apply_user_fix()
• F10 Factor Graph Optimizer: add_absolute_factor() with high confidence
• F11 Failure Recovery Coordinator: create_user_input_request(), apply_user_anchor()
• F15 SSE Event Streamer: send_user_input_request(), send_user_fix_applied()

Notes

• AC-6 is the human-in-the-loop fallback for extreme failures
• 3-failure threshold balances automation with user intervention
• 20% allowance (12 of 60 images) is operational constraint
• User fixes are trusted (high confidence weight in factor graph)
• System should minimize user inputs via L1/L2/L3 layer defense