gps-denied-onboard/_docs/02_tests/45_registration_rate_baseline_spec.md

Acceptance Test: Image Registration Rate >95% - Baseline

Summary

Validate AC-9 requirement that ≥95% of images successfully register (find enough matching features for pose estimation) under normal flight conditions.

Linked Acceptance Criteria

AC-9: Image Registration Rate > 95%. The system can find enough matching features to confidently calculate the camera's 6-DoF pose (position and orientation) and "stitch" that image into the final trajectory.

Preconditions

• ASTRAL-Next system operational
• Normal flight conditions (no extreme weather, lighting, or terrain)
• Multi-layer architecture (L1, L2, L3) active
• Registration success criteria defined

Registration Success Definition

An image is successfully registered if:

1. L1 sequential tracking succeeds (≥50 inlier matches), OR
2. L2 global retrieval succeeds (correct satellite tile, confidence >0.6), OR
3. L3 metric refinement succeeds (cross-view match confidence >0.5)

An image fails registration only if ALL three layers fail.

Test Data

• Primary Dataset: AD000001-AD000030 (baseline segment, normal conditions)
• Secondary Dataset: AD000001-AD000060 (full flight, includes challenges)
• Expected: ≥95% registration rate for both datasets

Test Steps

Step 1: Process Baseline Segment (Normal Conditions)

Action: Process AD000001-AD000030, track registration outcomes Expected Result:

Total images: 30
L1 successful: 28 (93.3%)
L2 successful (L1 failed): 2 (6.7%)
L3 successful (L1+L2 failed): 0 (0%)
Total registered: 30 (100%)
Registration rate: 100% > 95% (AC-9 PASS)
Status: BASELINE_EXCELLENT

Step 2: Analyze L1 Registration Performance

Action: Examine L1 sequential tracking success rate Expected Result:

L1 attempts: 29 (frame-to-frame pairs)
L1 successes: 28
L1 failures: 1
L1 success rate: 96.6%
Failure case: AD000003→004 (202.2m jump, expected)
L1 failure handling: L2 activated successfully
Status: L1_BASELINE_PASS

Step 3: Analyze L2 Fallback Performance

Action: Examine L2 global retrieval when L1 fails Expected Result:

L2 activations: 2 (L1 failure triggers)
L2 successes: 2
L2 failures: 0
L2 success rate: 100%
Cases: AD000004 (after 202m jump), AD000033 (after 220m jump)
L2 contribution: Prevented 2 registration failures
Status: L2_FALLBACK_WORKING

Step 4: Process Full Flight with Challenges

Action: Process AD000001-AD000060, including sharp turns and outliers Expected Result:

Total images: 60
L1 successful: 53 (88.3%)
L2 successful (L1 failed): 6 (10%)
L3 successful (L1+L2 failed): 0 (0%)
Registration failures: 1 (1.7%)
Registration rate: 98.3% > 95% (AC-9 PASS)
Status: FULL_FLIGHT_PASS

Step 5: Identify and Analyze Registration Failures

Action: Investigate any frames that failed all three registration layers Expected Result:

Failed frame: AD000048 (hypothetical example)
L1 failure: 268.6m jump from AD000047, overlap ~0%
L2 failure: Satellite data outdated, wrong tile retrieved
L3 failure: Cross-view match confidence 0.3 (below 0.5 threshold)
Outcome: User intervention requested (AC-6 human-in-the-loop)
Acceptable: <5% failure rate allows for extremely difficult cases
Status: FAILURE_WITHIN_TOLERANCE

Step 6: Calculate Final Registration Statistics

Action: Compute comprehensive registration metrics Expected Result:

Dataset: AD000001-060 (60 images)
Successfully registered: 59 images
Registration failures: 1 image
Registration rate: 98.3%

AC-9 Requirement: >95%
Actual Performance: 98.3%
Margin: +3.3 percentage points
Status: AC-9 PASS with margin

Pass/Fail Criteria

PASS if:

• Registration rate ≥95% on baseline dataset (AD000001-030)
• Registration rate ≥95% on full flight dataset (AD000001-060)
• Multi-layer fallback working (L2 activates when L1 fails)
• Registration failures <5% (allowing for extremely difficult frames)

FAIL if:

• Registration rate <95% on either dataset
• L2 fallback not activating when L1 fails
• Registration failures >5%
• System crashes on unregistered frames (should handle gracefully)

Layer Contribution Analysis

Layer 1 (Sequential Tracking)

• Role: Primary registration method (fastest)
• Success Rate: 85-95% (normal overlap conditions)
• Failure Cases: Sharp turns, low overlap

Layer 2 (Global Retrieval)

• Role: Fallback for L1 failures (slower but robust)
• Success Rate: 85-95% when activated
• Failure Cases: Stale satellite data, ambiguous locations

Layer 3 (Metric Refinement)

• Role: Precision improvement, not primary registration
• Success Rate: 80-90% when attempted
• Failure Cases: Large view angle difference, seasonal mismatch

Multi-Layer Defense

Registration Success = L1 OR L2 OR L3
P(success) = 1 - P(all fail)
P(success) = 1 - (0.10 × 0.10 × 0.15)
P(success) = 1 - 0.0015
P(success) = 99.85% > 95% (AC-9)

Registration Failure Handling

When registration fails for a frame:

1. System flags frame as UNREGISTERED
2. Continues processing subsequent frames
3. Attempts to re-register after later frames provide context
4. If still fails after 3 attempts, requests user input (AC-6)
5. Does not crash or halt processing

Quality Metrics Beyond Registration Rate

• Mean inlier count: >100 matches (L1 successful cases)
• Mean confidence: >0.8 (registered frames)
• Pose covariance: <50m uncertainty (registered frames)
• Trajectory continuity: No gaps >3 frames

Notes

• AC-9 threshold of 95% balances operational utility with real-world challenges
• 5% failure allowance (~3 frames per 60) accommodates extreme cases
• Multi-layer architecture critical for achieving high registration rate
• "Atlas" multi-map approach counts disconnected fragments as registered
• Registration rate > positioning accuracy (AC-1/AC-2) requirements