azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-22 20:21:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-219] Scaffold onboard runtime project

Browse Source 
Add the initial source, test, infrastructure, CI, configuration, and evidence-path scaffold so dependent implementation tasks have stable package and runtime boundaries.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-03 12:04:22 +03:00
parent dd9afe2797
commit 79997e39ac
97 changed files with 753 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_tasks/todo/AZ-219_initial_structure.md
-290
View File
@@ -1,290 +0,0 @@
# Initial Project Structure
**Task**: AZ-219_initial_structure
**Name**: Initial Structure
**Description**: Scaffold the project skeleton for the Jetson-hosted GPS-denied localization runtime, replay harness, local infrastructure, CI, and deployment evidence paths.
**Complexity**: 5 points
**Dependencies**: None
**Component**: Bootstrap
**Tracker**: AZ-219
**Epic**: AZ-206
## Project Folder Layout
```text
project-root/
├── src/
│ ├── __init__.py
│ ├── camera_ingest_calibration/
│ ├── basalt_vio_adapter/
│ ├── safety_anchor_wrapper/
│ ├── satellite_service/
│ ├── anchor_verification/
│ ├── tile_manager/
│ ├── mavlink_gcs_integration/
│ ├── fdr_observability/
│ ├── shared/
│ │ ├── contracts/
│ │ ├── geo_geometry/
│ │ ├── time_sync/
│ │ ├── config/
│ │ ├── errors/
│ │ └── telemetry/
│ └── native/
│ ├── basalt_bridge/
│ ├── feature_matching/
│ └── tensor_rt/
├── migrations/
│ ├── postgresql/
│ └── seed/
├── tests/
│ ├── unit/
│ ├── integration/
│ ├── blackbox/
│ ├── fixtures/
│ ├── sitl/
│ └── e2e/
├── e2e/
│ ├── replay/
│ └── reports/
├── deployment/
│ ├── docker/
│ ├── compose/
│ ├── jetson/
│ └── scripts/
├── config/
│ ├── development/
│ ├── ci/
│ ├── jetson/
│ └── production/
├── data/
│ ├── input/
│ ├── expected/
│ ├── cache/
│ ├── fdr/
│ └── test-results/
├── .github/
│ └── workflows/
│ └── ci.yml
├── docker-compose.yml
├── docker-compose.test.yml
├── .dockerignore
└── .env.example
```
### Layout Rationale
The runtime is organized directly under `src/` because this repository already represents the GPS-denied onboard system. Component directories live at the source root, with native bridges isolated under `src/native/` for BASALT, feature matching, and TensorRT-sensitive paths. Shared contracts, geometry, time-sync, configuration, error envelopes, and telemetry DTOs are centralized so component tasks consume a single public interface instead of duplicating cross-cutting logic.
The scaffold separates runtime source, migrations, tests, deployment assets, configuration, and mutable data. Production runs on Jetson hardware, while Docker/compose is used for replay, SITL, and deterministic CI environments.
## DTOs and Interfaces
### Shared DTOs
| DTO Name | Used By Components | Fields Summary |
|----------|--------------------|----------------|
| `FramePacket` | Camera ingest, BASALT VIO, Satellite Service, anchor verification, Tile Manager, FDR | Frame ID, timestamp, image reference, camera calibration ID, occlusion status, quality metrics |
| `TelemetrySample` | MAVLink/GCS, BASALT VIO, safety wrapper, FDR, e2e tests | Timestamp, IMU, attitude, airspeed, altitude, GPS health |
| `VioStatePacket` | BASALT VIO, safety wrapper, FDR, e2e tests | Timestamp, relative pose, velocity, bias, tracking quality, covariance hint |
| `PositionEstimate` | Safety wrapper, MAVLink/GCS, Tile Manager, FDR, e2e tests | WGS84 coordinates, covariance semi-major axis, source label, fix type, horizontal accuracy, anchor age |
| `VprCandidate` | Satellite Service, anchor verification, FDR | Chunk ID, tile ID, score, footprint, freshness status |
| `AnchorDecision` | Anchor verification, safety wrapper, FDR | Candidate ID, acceptance result, estimated pose, inliers, MRE, rejection reason |
| `CacheTileRecord` | Tile Manager, Satellite Service, anchor verification, FDR | Tile ID, type, CRS, meters per pixel, capture date, signature/hash status, trust level |
| `FdrEvent` | All runtime components, e2e tests | Event type, timestamp, component, severity, payload reference, mission/run ID |
| `ScenarioReport` | Separate e2e test suite, CI/CD, release evidence | Scenario ID, result, metrics, artifacts, failure reason |
### Component Interfaces
| Component | Interface | Methods | Exposed To |
|-----------|-----------|---------|------------|
| Camera ingest/calibration | `FrameProvider` | `next_frame`, `detect_occlusion`, `classify_quality` | BASALT VIO, Satellite Service, anchor verification, Tile Manager |
| BASALT VIO adapter | `VioAdapter` | `initialize`, `process`, `health` | Safety wrapper, e2e tests |
| Safety/anchor wrapper | `LocalizationStateMachine` | `update_vio`, `consider_anchor`, `degrade`, `propagate_imu_only`, `tile_write_eligibility` | MAVLink/GCS, Tile Manager, FDR, e2e tests |
| Satellite Service | `SatelliteService` | `import_mission_cache`, `load_index`, `retrieve`, `upload_generated_tiles` | Safety wrapper, anchor verification, Tile Manager |
| Anchor verification | `AnchorVerifier` | `verify`, `benchmark_matcher` | Safety wrapper, FDR |
| Tile Manager | `TileManager` | `validate_cache`, `get_tile_window`, `orthorectify_frame`, `write_generated_tile`, `package_sync` | Satellite Service, anchor verification, post-flight sync |
| MAVLink/GCS integration | `MavlinkGateway` | `subscribe_telemetry`, `emit_gps_input`, `emit_status` | BASALT VIO, safety wrapper, QGC, FDR |
| FDR/observability | `FlightRecorder` | `append_event`, `rollover`, `export` | All runtime components, e2e tests |
## CI/CD Pipeline
| Stage | Purpose | Trigger |
|-------|---------|---------|
| Format / lint | Enforce code style and static quality | Every PR and push to `dev` |
| Unit tests | Validate component-local behavior and shared contracts | Every PR and push to `dev` |
| Replay black-box smoke | Run deterministic still-image/cache/SITL subsets | Every PR |
| Cache/security fixture tests | Validate signed manifests, stale-tile rejection, and no mid-flight provider/Satellite Service calls | Every PR |
| Plane SITL spoof/failsafe | Validate ArduPilot Plane `GPS_INPUT`, failsafe, spoofing promotion | Nightly and release candidate |
| Public dataset replay | Exercise VIO, retrieval, and anchor behavior against pinned public slices | Nightly and release candidate |
| Jetson latency/resource tests | Measure p95 latency, memory, cold start, TensorRT/ONNX fidelity | Release candidate |
| Thermal/FDR endurance | Prove 8-hour 25 W no-throttle and <=64 GB FDR rollover | Hardware qualification |
| Build / package | Produce replay-compatible and Jetson deployment artifacts | Release candidate |
| Deploy staging evidence | Publish reports, tlogs, FDR summaries, cache validation artifacts | Release candidate |
### Pipeline Configuration Notes
CI uses Docker/compose for replay and SITL gates. Jetson, thermal, camera, and representative replay gates run on dedicated hardware runners and block release rather than every PR. Dataset downloads are license-tagged and not baked into images. Secrets, signing keys, and Satellite Service credentials are never cached.
## Environment Strategy
| Environment | Purpose | Configuration Notes |
|-------------|---------|---------------------|
| Development replay | Fast local iteration | Small fixture cache, local PostgreSQL/PostGIS, replay frames, test keys only |
| CI replay | Deterministic PR checks | Docker services for runtime, replay consumer, cache stub, and reports |
| Public dataset replay | Algorithm de-risking | Pinned dataset slices with license metadata and ground-truth reports |
| Plane SITL | MAVLink/failsafe validation | ArduPilot Plane SITL, QGC/tlog observer, production-like `GPS_INPUT` parameters |
| Jetson hardware validation | Production path profiling | JetPack/CUDA/TensorRT, local camera/GPU/MAVLink access, resource monitoring |
| Representative flight/replay | Final acceptance evidence | Target-like UAV, FC, camera, synchronized telemetry, ground truth |
| Production | Onboard mission runtime | Preloaded signed cache, local PostGIS, per-flight FDR, no provider network calls |
### Environment Variables
| Variable | Dev | Staging | Production | Description |
|----------|-----|---------|------------|-------------|
| `GPSD_ENV` | `development` | `staging` | `production` | Selects runtime profile |
| `GPSD_CONFIG_DIR` | local path | staged config path | onboard config path | Versioned configuration root |
| `GPSD_CACHE_DIR` | fixture cache | staged mission cache | onboard mission cache | COG, manifest, sidecar, descriptor root |
| `GPSD_FDR_DIR` | temp output | staged evidence path | per-flight NVMe path | Flight data recorder output |
| `GPSD_DATABASE_URL` | local PostGIS | staging PostGIS | onboard local PostGIS | Manifest, mission state, FDR event index |
| `GPSD_MAVLINK_URL` | SITL/replay | SITL/hardware rig | FC telemetry link | MAVLink connection endpoint |
| `GPSD_CAMERA_SOURCE` | fixture/replay | fixture or hardware | live camera | Navigation frame source |
| `GPSD_SIGNING_KEY_REF` | test key ref | staging secret ref | mission secret ref | Cache/sidecar signature verification |
| `GPSD_MAX_FDR_BYTES` | small fixture cap | release-like cap | `68719476736` | FDR rollover threshold |
| `GPSD_LOG_LEVEL` | debug/info | info | info/warn | Runtime logging level |
## Database Migration Approach
**Migration tool**: Versioned PostgreSQL migration scripts, with PostGIS extension setup and deterministic seed scripts.
**Strategy**: Additive migrations by default. Database, table, and column renames require explicit approval before implementation. Runtime rejects unknown required schema versions loudly.
### Initial Schema
- Mission profile tables for route geometry, sector classification, altitude band, and cache budget.
- Camera calibration tables for camera model, intrinsics, extrinsics, and verification status.
- Cache manifest tables for COG tiles, generated tiles, freshness, signatures, sidecar hashes, trust level, and PostGIS footprints.
- VPR chunk tables for retrieval footprints, descriptor metadata, multi-scale index references, and sector top-K policy.
- FDR event index tables for mission/run IDs, timestamps, event type, component, severity, and CBOR segment references.
- Validation run tables or report records for scenario IDs, metrics, artifacts, and release evidence pointers.
## Test Structure
```text
tests/
├── unit/
│ ├── shared/
│ ├── camera_ingest_calibration/
│ ├── basalt_vio_adapter/
│ ├── safety_anchor_wrapper/
│ ├── satellite_service/
│ ├── anchor_verification/
│ ├── tile_manager/
│ ├── mavlink_gcs_integration/
│ ├── fdr_observability/
├── integration/
│ ├── contracts/
│ ├── cache_postgis/
│ ├── mavlink/
│ └── fdr/
├── blackbox/
│ ├── still_image_geolocation/
│ ├── satellite_anchor/
│ ├── visual_blackout_spoofing/
│ ├── cache_freshness/
│ └── resource_limits/
├── fixtures/
│ ├── project_60_images/
│ ├── expected_results/
│ ├── satellite_cache/
│ ├── telemetry/
│ └── public_dataset_slices/
├── sitl/
│ ├── plane_gps_input/
│ ├── spoofing_promotion/
│ └── failsafe/
└── e2e/
├── replay/
├── reports/
└── release_evidence/
```
### Test Configuration Notes
Unit tests validate contracts and component behavior without external hardware. Integration tests exercise PostGIS, MAVLink, FDR segments, and cache fixtures. Black-box tests interact only through public inputs and outputs: frames, telemetry, offline cache, `GPS_INPUT`, QGC status, and FDR. Release gates add Jetson hardware, thermal/endurance, and representative replay evidence.
## Health Checks
| Service / Component | Liveness | Readiness |
|---------------------|----------|-----------|
| GPS-denied service | Process event loop responsive | Config, PostGIS, cache, FDR path, MAVLink, and camera/replay source validated |
| Replay consumer | Runner process responsive | Fixtures, expected results, and output path available |
| Satellite cache stub | Fixture volume mounted | Manifests, sidecars, descriptors, and signatures validated |
| ArduPilot Plane SITL | SITL process responsive | MAVLink ports accepting telemetry and production parameters loaded |
| QGC observer/log parser | Parser process responsive | Tlog/status stream connected |
Each deployable service exposes `/health/live`, `/health/ready`, and `/metrics` where HTTP is available. Non-HTTP hardware processes write equivalent structured health events to FDR and CI reports.
## Docker And Compose Requirements
- Create a replay-compatible Dockerfile for the Python runtime and native optional stubs.
- Create a replay-consumer Dockerfile for black-box test execution.
- Create a satellite-cache-stub fixture image or volume contract.
- Create an ArduPilot Plane SITL service for integration tests.
- Use non-root users, pinned base images, multi-stage builds where native compilation is needed, and health checks for long-running services.
- Provide `docker-compose.yml` for local development/replay and `docker-compose.test.yml` for black-box/SITL test execution.
- Do not bake secrets, provider credentials, mission signing keys, or public dataset payloads into images.
## Implementation Order
| Order | Component | Reason |
|-------|-----------|--------|
| 1 | Bootstrap and shared contracts | Establish package, configuration, migrations, CI, DTOs, and error envelopes |
| 2 | Shared geometry/time helpers | Foundational math and timestamp contracts used by camera, VIO, cache, wrapper, and validation |
| 3 | Runtime configuration and error handling | Prevent duplicated config/error behavior across components |
| 4 | Camera ingest/calibration | Produces the frame and occlusion signals required by VIO, anchor, cache, and tests |
| 5 | MAVLink/GCS integration | Supplies FC telemetry DTOs and validates `GPS_INPUT` output contract early |
| 6 | Tile Manager | Owns PostGIS cache manifest, sidecars, COG access, freshness gates, and generated-tile orthorectification |
| 7 | FDR/observability | Provides audit path for all components and validation reports |
| 8 | BASALT VIO adapter | Depends on frame and telemetry contracts, blocks wrapper integration |
| 9 | Satellite Service | Depends on tile schema and frame DTOs, feeds anchor verification, and handles pre-flight/post-flight package sync |
| 10 | Anchor verification | Depends on retrieval candidates and cache tile access |
| 11 | Safety/anchor wrapper | Consumes VIO, anchor, camera degradation, MAVLink, and FDR contracts |
| 12 | Validation harness | Uses public interfaces once contracts and runtime components are stable |
| 13 | Black-box, SITL, Jetson, and endurance test tasks | Exercise release gates and acceptance criteria end to end |
## Acceptance Criteria
**AC-1: Project scaffolded**
Given the structure plan above
When the implementer executes this task
Then source, test, migration, deployment, configuration, and data directories exist with placeholder files where needed so empty directories are retained.
**AC-2: Shared contracts initialized**
Given the planned component interfaces
When the scaffold is complete
Then shared DTO, error, configuration, telemetry, geometry, and time-sync contract locations exist and are importable from component skeletons.
**AC-3: Local infrastructure defined**
Given the system requires local PostGIS, replay, SITL, cache, and FDR paths
When the scaffold is complete
Then `docker-compose.yml`, `docker-compose.test.yml`, `.env.example`, and migration seed directories describe all required local services and volumes.
**AC-4: CI/CD configured**
Given the pipeline stages in the planning artifacts
When CI runs on a PR
Then format/lint, unit tests, replay black-box smoke, cache/security fixture tests, and artifact collection stages are defined.
**AC-5: Test harness skeleton ready**
Given black-box tests must use public interfaces only
When the scaffold is complete
Then unit, integration, black-box, fixtures, and SITL test locations exist with runner entry points ready for implementation tasks.
**AC-6: Deployment evidence paths ready**
Given release requires Jetson, SITL, FDR, cache, and representative replay evidence
When the scaffold is complete
Then deployment scripts, report output paths, and documentation placeholders exist for those evidence artifacts.
**AC-7: No production secrets or raw frames committed**
Given onboard runtime must not retain raw frames and must protect signing credentials
When the scaffold is reviewed
Then `.gitignore`, `.dockerignore`, and `.env.example` exclude secrets, generated FDR payloads, raw frame dumps, cache payloads, and test result artifacts unless explicitly fixture-scoped.