gps-denied-onboard/docs/03_tests/11b_flight_processing_engine_integration_spec.md

Integration Test: Flight Processing Engine (F02.2)

Summary

Validate the Flight Processing Engine component responsible for the main processing loop, frame-by-frame orchestration, recovery coordination, and chunk management.

Component Under Test

Component: Flight Processing Engine (F02.2) Interface: IFlightProcessingEngine Dependencies:

• F05 Image Input Pipeline (image source)
• F06 Image Rotation Manager (pre-processing)
• F07 Sequential Visual Odometry (motion estimation)
• F09 Metric Refinement (satellite alignment)
• F10 Factor Graph Optimizer (state estimation)
• F11 Failure Recovery Coordinator (recovery logic)
• F12 Route Chunk Manager (chunk state)
• F14 Result Manager (saving results)
• F15 SSE Event Streamer (real-time updates)

Detailed Description

This test validates that the Flight Processing Engine can:

1. Run the main processing loop (Image → VO → Graph → Result)
2. Manage flight status (Processing, Blocked, Recovering, Completed)
3. Coordinate chunk lifecycle with F12
4. Handle tracking loss and delegate to F11
5. Apply user fixes and resume processing
6. Publish results via F14/F15
7. Manage background chunk matching tasks
8. Handle concurrent processing gracefully

The Processing Engine runs in a background thread per active flight.

Input Data

Test Case 1: Start Processing

• Flight: Test_Baseline_Flight with 10 queued images
• Action: Call start_processing(flight_id)
• Expected:
  • Background processing thread started
  • First image retrieved from F05
  • Processing loop begins

Test Case 2: Stop Processing

• Flight: Active flight with processing in progress
• Action: Call stop_processing(flight_id)
• Expected:
  • Processing loop stopped gracefully
  • Current frame completed or cancelled
  • State saved

Test Case 3: Process Single Frame (Normal)

• Input: Single frame with good tracking
• Expected:
  • F06.requires_rotation_sweep() checked
  • F07.compute_relative_pose() called
  • F12.add_frame_to_chunk() called
  • F10.add_relative_factor() called
  • F10.optimize_chunk() called
  • F14.update_frame_result() called
  • SSE event sent

Test Case 4: Process Frame (First Frame / Sharp Turn)

• Input: First frame or frame after sharp turn
• Expected:
  • F06.requires_rotation_sweep() returns True
  • F06.rotate_image_360() called (12 rotations)
  • F09.align_to_satellite() called for each rotation
  • Best rotation selected
  • Heading updated

Test Case 5: Process Frame (Tracking Lost)

• Input: Frame with low VO confidence
• Expected:
  • F11.check_confidence() returns LOST
  • F11.create_chunk_on_tracking_loss() called
  • New chunk created proactively
  • handle_tracking_loss() invoked

Test Case 6: Handle Tracking Loss (Progressive Search)

• Input: Frame with tracking lost, recoverable
• Expected:
  • F11.start_search() called
  • F11.try_current_grid() called iteratively
  • Grid expansion (1→4→9→16→25)
  • Match found, F11.mark_found() called
  • Processing continues

Test Case 7: Handle Tracking Loss (User Input Needed)

• Input: Frame with tracking lost, not recoverable
• Expected:
  • Progressive search exhausted (25 tiles)
  • F11.create_user_input_request() called
  • Engine receives UserInputRequest
  • F15.send_user_input_request() called
  • Status set to BLOCKED
  • Processing paused

Test Case 8: Apply User Fix

• Input: UserFixRequest with GPS anchor
• Action: Call apply_user_fix(flight_id, fix_data)
• Expected:
  • F11.apply_user_anchor() called
  • Anchor applied to factor graph
  • Status set to PROCESSING
  • Processing loop resumes

Test Case 9: Get Active Chunk

• Flight: Active flight with chunks
• Action: Call get_active_chunk(flight_id)
• Expected:
  • F12.get_active_chunk() called
  • Returns current active chunk or None

Test Case 10: Create New Chunk

• Input: Tracking loss detected
• Action: Call create_new_chunk(flight_id, frame_id)
• Expected:
  • F12.create_chunk() called
  • New chunk created in factor graph
  • Returns ChunkHandle

Test Case 11: Process Flight (Full - Normal)

• Flight: 30 images (AD000001-030)
• Expected:
  • All 30 images processed
  • Status transitions: Processing → Completed
  • Results published for all frames
  • Processing time < 150 seconds (5s per image)

Test Case 12: Process Flight (With Sharp Turn)

• Flight: AD000042, AD000044, AD000045, AD000046 (skip AD000043)
• Expected:
  • Tracking lost at AD000044
  • New chunk created
  • Recovery succeeds (L2/L3)
  • Flight completes

Test Case 13: Process Flight (With Outlier)

• Flight: AD000045-050 (includes 268m outlier)
• Expected:
  • Outlier detected by factor graph
  • Robust kernel handles outlier
  • Other images processed correctly

Test Case 14: Process Flight (Long)

• Flight: All 60 images (AD000001-060)
• Expected:
  • Processing completes
  • Registration rate > 95% (AC-9)
  • Processing time < 300 seconds (AC-7)

Test Case 15: Background Chunk Matching

• Flight: Flight with multiple unanchored chunks
• Expected:
  • Background task processes chunks
  • F11.process_unanchored_chunks() called periodically
  • Chunks matched and merged asynchronously
  • Frame processing not blocked

Test Case 16: State Persistence and Recovery

• Scenario:
  • Process 15 frames
  • Simulate restart
  • Resume processing
• Expected:
  • State saved to F03 before restart
  • State restored on resume
  • Processing continues from frame 16

Expected Output

For each frame processed:

{
  "flight_id": "string",
  "frame_id": <integer>,
  "status": "processed|failed|skipped|blocked",
  "gps": {
    "latitude": <float>,
    "longitude": <float>
  },
  "confidence": <float>,
  "chunk_id": "string",
  "processing_time_ms": <float>
}

Success Criteria

Test Cases 1-2 (Start/Stop):

• Processing starts/stops correctly
• No resource leaks
• Graceful shutdown

Test Cases 3-5 (Frame Processing):

• Correct components called in order
• State updates correctly
• Results published

Test Cases 6-8 (Recovery):

• Progressive search works
• User input flow works
• Recovery successful

Test Cases 9-10 (Chunk Management):

• Chunks created/managed correctly
• F12 integration works

Test Cases 11-14 (Full Flights):

• All acceptance criteria met
• Processing completes successfully

Test Cases 15-16 (Background/Recovery):

• Background tasks work
• State persistence works

Maximum Expected Time

• Start/stop processing: < 500ms
• Process single frame: < 5 seconds (AC-7)
• Handle tracking loss: < 2 seconds
• Apply user fix: < 1 second
• Process 30 images: < 150 seconds
• Process 60 images: < 300 seconds
• Total test suite: < 600 seconds

Pass/Fail Criteria

Overall Test Passes If:

• All 16 test cases pass
• Processing loop works correctly
• Recovery mechanisms work
• Chunk management works
• Performance targets met

Test Fails If:

• Processing loop crashes
• Recovery fails when it should succeed
• User input not requested when needed
• Performance exceeds 5s per image
• State persistence fails