gps-denied-onboard/_docs/02_document/components/04_satellite_retrieval/tests.md

Test Specification — Satellite Service

Acceptance Criteria Traceability

AC ID	Acceptance Criterion	Test IDs	Coverage
AC-3.2	Sharp-turn relocalization	IT-02, AT-01	Covered
AC-3.3	>=3 disconnected segments	IT-03	Covered
AC-3.4	Relocalization request after loss	IT-02	Covered
AC-4.1	Trigger path latency	PT-01	Covered
AC-4.2	Memory budget	PT-01	Covered
AC-8.1	Cache imagery interface	IT-01	Covered
AC-8.3	Preloaded/preprocessed cache	IT-01	Covered
AC-8.6	VPR chunks, multi-scale, dynamic K	IT-01, IT-04	Covered
AC-NEW-1	Cold-start first fix support	PT-02	Covered
AC-NEW-6	Freshness-aware retrieval	IT-04, ST-01	Covered

Blackbox Tests

IT-01: Index Load And Chunk Coverage

Summary: Verify preloaded VPR chunks and FAISS index cover the operational area.

Traces to: AC-8.1, AC-8.3, AC-8.6

Input data: PostgreSQL/PostGIS cache manifest, VPR chunk metadata, FAISS index.

Expected result: Every test frame footprint falls inside at least one VPR chunk; fine and coarse descriptors are present where required.

Max execution time: 2 minutes per mission cache.

---

IT-02: Sharp-Turn Local Retrieval Trigger

Summary: Verify sharp-turn state requests candidates rather than relying on frame-to-frame VO.

Traces to: AC-3.2, AC-3.4

Input data: Wrapper relocalization request with sharp-turn/loss reason.

Expected result: Satellite Service returns bounded top-K candidates from preloaded local indexes based on sector/covariance policy.

Max execution time: 2 seconds per query.

---

IT-03: Disconnected Segment Retrieval

Summary: Verify at least three disconnected segments can each retrieve candidate chunks.

Traces to: AC-3.3

Input data: Three disconnected query frames with approximate prior/covariance.

Expected result: Each query returns a candidate set including the ground-truth region when covered by the cache fixture.

Max execution time: Dataset-dependent.

---

IT-04: Dynamic K And Freshness Filter

Summary: Verify K varies by sector and covariance, and stale candidates are tagged.

Traces to: AC-8.6, AC-NEW-6

Input data: Stable and active-conflict sector cache fixtures with fresh/stale tiles.

Expected result: K=5 for stable low-covariance, K=20 for active-conflict, K=50 fallback; stale candidates are flagged for rejection/down-confidence.

Max execution time: 2 seconds per query.

Performance Tests

PT-01: Retrieval Query Runtime

Summary: Verify descriptor extraction and FAISS query fit trigger-path budget.

Traces to: AC-4.1, AC-4.2

Load scenario:

• Query set: 100 representative relocalization frames.
• Environment: Jetson and replay workstation.

Metric	Target	Failure Threshold
Retrieval p95	<=300 ms trigger path share	>400 ms
Memory contribution	<=2 GB	>3 GB
Candidate count policy	Exact	Any wrong K

Resource limits: Total system memory remains below 8 GB.

---

PT-02: Cold-Start Index Load

Summary: Verify retrieval readiness supports first fix <30 s.

Traces to: AC-NEW-1

Load scenario:

• Cold boot 50 runs.
• Cache/index mounted locally.

Metric	Target	Failure Threshold
Index ready p95	<=10 s	>15 s
First retrieval p95	Fits <30 s first-fix budget	Exceeds budget

Security Tests

ST-01: Stale Candidate Handling

Summary: Verify stale imagery cannot silently appear as a trusted retrieval candidate.

Traces to: AC-NEW-6

Attack vector: Manipulated cache manifest marks stale tile as available.

Test procedure:

1. Load cache fixture with stale capture dates.
2. Query against stale region.

Expected behavior: Candidate carries stale status; anchor path cannot accept it as satellite_anchored.

Pass criteria: 0 stale candidates without explicit stale/down-confidence metadata.

---

ST-02: No Mid-Flight Satellite Service Calls

Summary: Verify relocalization never performs satellite-provider or suite Satellite Service network calls during flight.

Traces to: AC-8.3, R-SAT-01

Attack vector: Runtime attempts to fetch missing cache/index data over the network during relocalization.

Test procedure:

1. Disable external network access during a replay scenario.
2. Trigger relocalization against preloaded cache fixtures.
3. Inspect network call logs and Satellite Service client telemetry.

Expected behavior: Retrieval uses only mounted local cache/index data; missing data produces degraded/no-candidate behavior, not a network fetch.

Pass criteria: 0 mid-flight Satellite Service or satellite-provider calls.

Acceptance Tests

AT-01: Relocalization Candidate Returned

Summary: Verify a relocalization request returns usable candidates for anchor verification.

Traces to: AC-3.2, AC-8.6

Step	Action	Expected Result
1	Submit sharp-turn query	Retrieval invokes VPR
2	Read output	Candidate list includes chunk IDs, tile IDs, scores, footprints, freshness

Test Data Management

Data Set	Description	Source	Size
cache_vpr_fixture	PostGIS manifest, COGs, descriptors, FAISS index	Generated/cache fixture	Mission-dependent
aerial_vpr_queries	Aerial query frames with ground-truth regions	ALTO/AerialVL/representative	Dataset-dependent

Setup procedure: Restore isolated PostgreSQL schema and mount read-only descriptor/index files.

Teardown procedure: Drop schema and remove generated retrieval reports.

Data isolation strategy: Per-run schema and read-only cache fixture volume.