gps-denied-onboard/_docs/02_document/tests/test-data.md

Test Data Management

Seed Data Sets

Data Set	Description	Used by Tests	How Loaded	Cleanup
flight-sequence-60	60 aerial images (AD000001-060.jpg) with ground truth GPS from coordinates.csv, captured at ~1 photo per 2-3s from a fixed-wing UAV at 400m altitude	FT-P-01 through FT-P-06, FT-N-01, FT-N-02, NFT-PERF-01, NFT-RES-01	Volume mount to camera-replay service; coordinates.csv loaded by e2e-consumer for ground truth comparison	Container restart between test groups
camera-params	Camera parameters: ADTi Surveyor Lite 26S v2, 26MP (6252x4168), 25mm focal length, 23.5mm sensor width	All position accuracy tests, object localization tests	Volume mount; read by gps-denied-system at startup	N/A (read-only)
satellite-tiles-test	Pre-processed satellite tiles (zoom 18) covering test flight area: 48.249-48.276°N, 37.340-37.386°E	All tests requiring satellite matching	Volume mount to satellite-tile-server and gps-denied-system	Container restart
ardupilot-params	ArduPilot SITL parameters: GPS1_TYPE=14, GPS_RATE=5, EK3_SRC1_POSXY=1, EK3_SRC1_VELXY=1, fixed-wing frame	All tests requiring flight controller interaction	Baked into ardupilot-sitl Docker image	Container restart
imu-replay-data	Synthetic IMU data (accelerometer + gyroscope at 200Hz) matching the flight-sequence-60 trajectory. Generated from coordinates.csv ground truth positions by computing velocities/accelerations between frames, interpolating to 200Hz, adding noise consistent with ICM-42688-P datasheet (gyro: 3.0e-3 °/s/√Hz, accel: 70 µg/√Hz). Stored as input_data/imu_synthetic_200hz.csv with columns: timestamp_us, accel_x, accel_y, accel_z, gyro_x, gyro_y, gyro_z. Alternative: SITL ArduPilot flies a waypoint mission following the coordinates.csv trajectory and internally generates physically consistent IMU data at 200Hz.	FT-P-01 through FT-P-06, NFT-PERF-01, NFT-RES-01 through NFT-RES-05	Primary: SITL ArduPilot flies the trajectory and generates IMU internally via MAVLink. Fallback: pre-generated CSV replayed via MAVLink injector	Container restart
invalid-inputs	Malformed images (truncated JPEG, wrong resolution), invalid API payloads, corrupted IMU streams	FT-N-03 through FT-N-06, NFT-SEC-01 through NFT-SEC-04	Volume mount to e2e-consumer; injected via API calls	Container restart

Data Isolation Strategy

Each test group runs against a fresh container restart of the gps-denied-system and ardupilot-sitl services. The camera-replay service is restarted and configured per test group (different frame subsets, different replay speeds, or different fault injection modes). MAVLink capture logs are isolated per test run via timestamped directories.

Input Data Mapping

Input Data File	Source Location	Description	Covers Scenarios
AD000001-060.jpg	_docs/00_problem/input_data/	60 aerial images (6252x4168, 26MP) from a fixed-wing UAV at 400m altitude. Images taken at ~1 photo/2-3s (wider spacing than real 0.7fps but usable for functional tests)	FT-P-01 to FT-P-06, FT-N-01, FT-N-02, NFT-PERF-01, NFT-RES-01, NFT-RES-LIM-01
AD000001_gmaps.png, AD000002_gmaps.png	_docs/00_problem/input_data/	Google Maps satellite reference images for frames 1-2 (used as sample satellite tile data for satellite matching validation)	FT-P-01, NFT-RES-02, NFT-RES-04
coordinates.csv	_docs/00_problem/input_data/coordinates.csv	Ground truth GPS (lat, lon) for each of the 60 images	FT-P-01, FT-P-02, FT-P-03, FT-P-04 (comparison baseline)
data_parameters.md	_docs/00_problem/input_data/data_parameters.md	Camera specs: 400m altitude, ADTi Surveyor Lite 26S v2, 26MP, 25mm focal length, 23.5mm sensor	Test environment configuration
position_accuracy.csv	_docs/00_problem/input_data/expected_results/position_accuracy.csv	Per-frame ground truth with acceptance thresholds	FT-P-01, FT-P-02 (expected result comparison)
imu_synthetic_200hz.csv	_docs/00_problem/input_data/ (TO BE GENERATED)	Synthetic 200Hz IMU data (accel + gyro) derived from coordinates.csv trajectory. Matches ICM-42688-P noise characteristics. Required for ESKF sensor fusion testing outside SITL.	FT-P-01 to FT-P-06, NFT-PERF-01, NFT-RES-01 to NFT-RES-06

IMU Data Generation

No real IMU recordings exist for the 60-image flight sequence. Two approaches for providing IMU data during tests:

Approach A — SITL-generated (primary): ArduPilot SITL flies a waypoint mission following the coordinates.csv trajectory. SITL's internal physics engine generates physically consistent IMU data at 200Hz, delivered via MAVLink to the GPS-denied system. This is the most realistic approach and requires no pre-generated files.

Approach B — Synthetic CSV (fallback/replay): Generate imu_synthetic_200hz.csv offline from coordinates.csv:

1. Compute inter-frame velocities from GPS positions and timestamps
2. Interpolate position/velocity to 200Hz using cubic splines
3. Compute accelerations (body frame) accounting for gravity + flight dynamics
4. Add sensor noise matching ICM-42688-P specs (gyro: 3.0e-3 °/s/√Hz, accel: 70 µg/√Hz)
5. Add bias random walks (gyro: 5.0e-5 °/s²/√Hz, accel: 2.0e-3 m/s³/√Hz)
6. Replay via MAVLink injector service at 200Hz

Approach A is recommended for integration tests. Approach B is useful for deterministic unit-level ESKF tests where reproducible IMU streams are needed.

Satellite Tile Data

Only 2 Google Maps screenshots exist (AD000001_gmaps.png, AD000002_gmaps.png). Full satellite tile coverage for the test area must be prepared:

1. Download Google Maps tiles at zoom 18 for the bounding box: 48.249-48.276°N, 37.340-37.386°E
2. Store as 256x256 JPEG tiles with geohash-based naming
3. Load into satellite-tile-server Docker service
4. Estimated: ~50-100 tiles for the test area (~1-2MB total)

Expected Results Mapping

Test Scenario ID	Input Data	Expected Result	Comparison Method	Tolerance	Expected Result Source
FT-P-01	flight-sequence-60 (60 frames)	≥80% of frames within 50m of ground truth	percentage	≥80%	expected_results/position_accuracy.csv
FT-P-02	flight-sequence-60 (60 frames)	≥60% of frames within 20m of ground truth	percentage	≥60%	expected_results/position_accuracy.csv
FT-P-03	flight-sequence-60 (60 frames)	No single frame exceeds 100m error	threshold_max	≤100m	expected_results/position_accuracy.csv
FT-P-04	flight-sequence-60 (selected satellite anchor pairs)	VO drift between satellite anchors <100m	threshold_max	≤100m	inline
FT-P-05	Single frame + satellite match	GPS_INPUT: fix_type=3, horiz_accuracy 5-20m, satellites_visible=10	exact + range	fix_type==3, accuracy∈[1,50]	expected_results/results_report.md #5
FT-P-06	flight-sequence-60	≥57 of 60 frames registered (≥95%)	percentage	≥95%	inline
FT-P-07	Normal operation, satellite match <30s	Confidence tier: HIGH	exact	N/A	inline
FT-P-08	VO tracking, no satellite >30s	Confidence tier: MEDIUM	exact	N/A	inline
FT-P-09	GPS_INPUT stream	Messages at 5-10Hz	range	[5,10] Hz	inline
FT-P-10	POST /objects/locate (known pixel, gimbal, zoom, UAV position)	lat/lon within accuracy_m of ground truth	numeric_tolerance	within accuracy_m	expected_results/results_report.md #27
FT-P-11	Known GPS → NED → pixel → GPS	Round-trip error <0.1m	threshold_max	≤0.1m	inline
FT-P-12	System boot + GLOBAL_POSITION_INT	GPS_INPUT output within 60s	threshold_max	≤60s	inline
FT-N-01	Frames 32-43 (direction change area)	System continues producing estimates	threshold_min	≥1 output per frame	inline
FT-N-02	Simulated 350m gap between frames	System handles outlier, next valid frame <100m error	threshold_max	≤100m	inline
FT-N-03	POST /objects/locate with invalid pixels	HTTP 422	exact	status==422	inline
FT-N-04	Unauthenticated request to /sessions	HTTP 401	exact	status==401	inline
FT-N-05	VO lost + 3 satellite failures	fix_type=0, horiz_accuracy=999.0, RELOC_REQ sent	exact + regex	fix_type==0, matches RELOC_REQ:.*	inline
FT-N-06	VO lost + IMU-only	fix_type=2, horiz_accuracy≥50m growing	exact + threshold_min	fix_type==2, accuracy≥50	inline

External Dependency Mocks

External Service	Mock/Stub	How Provided	Behavior
Flight Controller (ArduPilot)	ArduPilot SITL	Docker service (ardupilot-sitl)	Full MAVLink protocol: heartbeat, GLOBAL_POSITION_INT, IMU data at 200Hz, accepts GPS_INPUT, responds to COMMAND_LONG
Camera hardware (ADTI 20L V1)	Frame replay server	Docker service (camera-replay)	Serves frames from input_data/ at configurable rate (0.7fps default); supports fault injection (frame drop, delayed frame, corrupted JPEG)
Satellite imagery (Google Maps)	Static tile server	Docker service (satellite-tile-server)	Serves pre-cached tiles via HTTP; supports fault injection (404 for missing tiles, slow response)
Ground station	MAVLink inspector	Docker service (mavlink-inspector)	Captures STATUSTEXT and NAMED_VALUE_FLOAT; can inject COMMAND_LONG (operator re-localization hint)
GPU (CUDA/TensorRT)	CPU fallback or Jetson hardware	Conditional	Docker: CPU-mode stubs for TRT inference (slower but functionally equivalent). Jetson: real GPU

Data Validation Rules

Data Type	Validation	Invalid Examples	Expected System Behavior
Camera frame (JPEG)	Valid JPEG, resolution ≥ 1280x720	Truncated JPEG, 0-byte file, BMP format	Log warning, skip frame, continue with IMU-only ESKF prediction
GPS coordinate	lat ∈ [-90, 90], lon ∈ [-180, 180]	lat=999, lon=NaN	Reject, use last valid position
IMU data	Acceleration ∈ [-160, 160] m/s², gyro ∈ [-35, 35] rad/s	All zeros, NaN values, extreme spikes	Filter outliers via ESKF process noise, log warning
Satellite tile	Valid JPEG/PNG, 256x256 px	Missing tile (404), corrupted image	Skip tile, expand search radius, fall back to wider area
API request body	Valid JSON, required fields present	Missing pixel_x, non-numeric zoom_factor	HTTP 422 with validation error details
JWT token	Valid signature, not expired	Expired token, invalid signature, missing token	HTTP 401 Unauthorized