Files
gps-denied-onboard/_docs/02_tasks/done/AZ-263_initial_structure.md
T
Oleksandr Bezdieniezhnykh b12db61444 [AZ-263] Bootstrap: repo skeleton + Docker + CI + Alembic + Tier-1 tests
Implements the AZ-263 / E-BOOT initial structure task:

- Python src/-layout package `gps_denied_onboard/` with per-component
  interface stubs (14 components), type-only DTOs under `_types/`,
  shared helpers under `helpers/` (R14 LightGlue ownership), structured
  JSON logging, runtime composition root with env-var fail-fast gate,
  healthcheck module shared by Docker and CI smoke.
- CMake top-level + `cmake/{build_options,dependencies,strategies}.cmake`
  with the BUILD_* per-binary flags (ADR-002) and pinned external git
  refs for OKVIS2 / VINS-Mono / GTSAM / FAISS / OpenCV >=4.12.0.
- Three Dockerfiles (companion-tier1, operator-tooling,
  mock-suite-sat-service) + two compose files (dev + Tier-1 test).
- Four GitHub Actions workflows: ci.yml (lint/unit/integration/dual
  binary build/SBOM diff/security), ci-tier2.yml (self-hosted Jetson
  AC-bound NFTs), release.yml, cve-rescan.yml.
- Two CI gate scripts: `ci/sbom_diff.py` (deployment SBOM subset +
  R02 exclusion), `ci/opencv_pin_gate.py` (>=4.12.0 enforcement,
  D-CROSS-CVE-1).
- Alembic-driven Postgres 16 initial migration `0001_initial.py`
  mirroring satellite-provider tiles + flights + sector_classifications
  + manifests + engine_cache_entries (data_model.md s 2).
- Tier-1 test scaffolding: 95 passing unit tests covering every AC,
  per-component smoke tests, structured logging JSON output check,
  env-var gate check, healthcheck import check. Two CI-gated tests
  (cmake configure, actionlint) skip locally with explicit reasons.
- Batch report + code review report under `_docs/03_implementation/`.

Verdict: PASS_WITH_WARNINGS (two Low findings, both informational).
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 01:00:28 +03:00

33 KiB
Raw Blame History

Initial Project Structure

Task: AZ-263_initial_structure Name: Initial Structure Description: Scaffold the gps-denied-onboard companion project skeleton — Python+C++ monolith with per-binary CMake BUILD_* flags (ADR-002), interface-first components under src/gps_denied_onboard/components/<component>/ (ADR-009), three Dockerfiles + two compose files for Tier-1, GitHub Actions workflow with dual-binary emit + SBOM diff, PostgreSQL 16 schema mirrored from satellite-provider, and the test structure that the per-component test specs already require. Complexity: 5 points Dependencies: None Component: Bootstrap Tracker: AZ-263 Epic: AZ-244 (E-BOOT)

Project Folder Layout

gps-denied-onboard/
├── pyproject.toml                       # Python project (ruff, mypy --strict, pytest, coverage gates per ci_cd_pipeline.md)
├── CMakeLists.txt                       # Top-level CMake — drives BUILD_VINS_MONO / BUILD_SALAD / etc.
├── cmake/
│   ├── dependencies.cmake               # Pinned commits/refs for OKVIS2, VINS-Mono, GTSAM, FAISS, OpenCV ≥4.12.0 (D-CROSS-CVE-1)
│   ├── build_options.cmake              # The BUILD_* option() set — single source of truth for ADR-002
│   └── strategies.cmake                 # Helper to register a strategy implementation behind its BUILD_* flag
├── .clang-format
├── .clang-tidy
├── .cmake-format.yaml
├── .editorconfig
├── .env.example                         # All env vars from § Environment Variables below
├── .dockerignore                        # Verbatim from deployment/containerization.md § .dockerignore
├── .gitignore                           # Excludes build/, *.engine, *.calib, *.index, .venv/, __pycache__/, tests/fixtures/large_replays/
├── README.md                            # Pointers into _docs/02_document/
│
├── docker/
│   ├── companion-tier1.Dockerfile       # Multi-stage: system-deps → python-deps → cpp-build → runtime
│   ├── operator-tooling.Dockerfile      # python:3.10-slim; installs C11 + C12 + healthcheck
│   ├── mock-suite-sat-service.Dockerfile  # Lives next to fixtures/mock-suite-sat-service/ but referenced from compose
│   └── db-init/
│       └── 01_seed.sql.example          # Template — real seed lives under tests/fixtures/seed-db.sql
│
├── docker-compose.yml                   # Local dev: companion + mock-sat + db + operator-tooling (verbatim from containerization.md § Local Development)
├── docker-compose.test.yml              # Tier-1 integration/blackbox: extends docker-compose.yml + e2e-runner sidecar
│
├── .github/
│   └── workflows/
│       ├── ci.yml                       # Tier-1 lint + unit + integration + dual-binary build + SBOM diff + security
│       ├── ci-tier2.yml                 # Self-hosted Jetson runner (labels: [self-hosted, jetson, orin-nano-super]); AC-bound NFTs
│       ├── release.yml                  # Tag-on-main: JetPack image build + operator tooling tarball
│       └── cve-rescan.yml               # Monthly scheduled CVE re-scan (D-CROSS-CVE-1 + GTSAM)
│
├── ci/
│   ├── sbom_diff.py                     # ADR-002 enforcement — deployment SBOM ⊂ research SBOM, vins_mono/salad/etc. excluded
│   └── opencv_pin_gate.py               # Asserts resolved OpenCV ≥ 4.12.0
│
├── src/
│   └── gps_denied_onboard/              # Top-level Python package (single package per src-layout convention)
│       ├── __init__.py
│       ├── runtime_root.py              # Composition root — config → strategy resolution → graph wiring (ADR-009)
│       ├── healthcheck.py               # Used by Dockerfile HEALTHCHECK + CI smoke
│       ├── config/
│       │   ├── loader.py                # YAML config loader + validation (per-flight + camera calibration JSON)
│       │   └── schema.py                # Config dataclasses
│       ├── logging/
│       │   └── structured.py            # Structured JSON logging (E-CC-LOG / AZ-245); no narrative log lines
│       ├── fdr_client/                  # E-CC-FDR-CLIENT / AZ-247: lock-free queue + record schema; producer-side API
│       │   ├── client.py
│       │   ├── records.py               # FdrRecord schema (estimates / IMU / MAVLink / health / tile / thumbnail)
│       │   └── queue.py                 # Lock-free SPSC queue per producer, drop-oldest on overrun
│       ├── _types/                      # Cross-component DTOs (Protocol/dataclass — no implementation here)
│       │   ├── nav.py                   # NavCameraFrame, ImuSample, ImuWindow, AttitudeWindow, FlightStateSignal, GpsHealth
│       │   ├── vio.py                   # VioOutput
│       │   ├── vpr.py                   # VprQuery, VprResult, RerankResult
│       │   ├── matching.py              # MatchResult
│       │   ├── pose.py                  # PoseEstimate, EstimatorOutput, EstimatorHealth
│       │   ├── tile.py                  # Tile, TileQualityMetadata, TileRecord, SectorClassification
│       │   ├── calibration.py           # CameraCalibration (intrinsics + distortion + body-to-camera + acquisition_method)
│       │   ├── emitted.py               # EmittedExternalPosition (per-FC encoded)
│       │   └── manifests.py             # Manifest, EngineCacheEntry
│       ├── helpers/                     # Common helpers (shared utilities — no component owns them)
│       │   ├── imu_preintegrator.py     # _docs/02_document/common-helpers/01_helper_imu_preintegrator.md
│       │   ├── se3_utils.py             # 02_helper_se3_utils.md
│       │   ├── lightglue_runtime.py     # 03_helper_lightglue_runtime.md (shared by C2.5 + C3 — owned by helper, NOT C3, R14 fix)
│       │   ├── wgs_converter.py         # 04_helper_wgs_converter.md
│       │   ├── sha256_sidecar.py        # 05_helper_sha256_sidecar.md (D-C10-3 content-hash gate)
│       │   ├── engine_filename_schema.py # 06_helper_engine_filename_schema.md (D-C10-7 self-describing engine names)
│       │   ├── ransac_filter.py         # 07_helper_ransac_filter.md
│       │   └── descriptor_normaliser.py # 08_helper_descriptor_normaliser.md
│       └── components/                  # Subpackage — every component has interface.py + default impl + optional alt impls + tests/
│           ├── c1_vio/                  # AZ-254: VioStrategy (OKVIS2 default, VINS-Mono research-only, KltRansac mandatory baseline)
│           ├── c2_vpr/                  # AZ-255: VprStrategy (UltraVPR primary; MegaLoc/MixVPR/SelaVPR/EigenPlaces/NetVLAD/SALAD)
│           ├── c2_5_rerank/             # AZ-256: InlierBasedReranker (single-pair LightGlue inlier count K=10→N=3)
│           ├── c3_matcher/              # AZ-257: CrossDomainMatcher (DISK+LightGlue / ALIKED+LightGlue / XFeat)
│           ├── c3_5_adhop/              # AZ-258: AdHoPRefinementStrategy
│           ├── c4_pose/                 # AZ-259: PoseEstimator (OpenCV solvePnPRansac + GTSAM Marginals)
│           ├── c5_state/                # AZ-260: StateEstimator (GTSAM iSAM2 + IncrementalFixedLagSmoother + ESKF baseline)
│           ├── c6_tile_cache/           # AZ-250: TileStore (Postgres mirror + filesystem mmap + FAISS HNSW)
│           ├── c7_inference/            # AZ-249: InferenceRuntime (TensorRT 10.3 / ONNX Runtime+TRT EP / PyTorch FP16)
│           ├── c8_fc_adapter/           # AZ-261: FcAdapter (PymavlinkArdupilotAdapter + Msp2InavAdapter) + GcsAdapter
│           ├── c10_provisioning/        # AZ-252: CacheProvisioner (engine compile + descriptors + manifest + content-hash)
│           ├── c11_tile_manager/        # AZ-251: TileDownloader + TileUploader (operator-side ONLY — excluded from airborne via CMake)
│           ├── c12_operator_tooling/    # AZ-253: CacheBuildWorkflow + OperatorReLocService (CLI; GUI deferred)
│           └── c13_fdr/                 # AZ-248: FdrWriter (writer thread + segment rotation + ≤64 GB cap)
│
├── cpp/                                 # Native libraries linked from src/gps_denied_onboard/components/* via pybind11
│   ├── okvis2/                          # OKVIS2 wrapper, exposes Python C-extension; C1 production-default
│   ├── vins_mono/                       # Built only when BUILD_VINS_MONO=ON (research binary)
│   ├── klt_ransac/                      # Mandatory simple-baseline; always linked
│   ├── gtsam_bindings/                  # Thin wrapper over GTSAM iSAM2 + IncrementalFixedLagSmoother + Marginals
│   ├── faiss_index/                     # FAISS HNSW wrapper (mmap-backed)
│   └── pybind11/                        # Submodule
│
├── tests/
│   ├── conftest.py                      # Pytest fixtures (Postgres bring-up, SITL containers, Derkachi corpus mount)
│   ├── unit/                            # Per-component pytest tests (mirrors src/gps_denied_onboard/components/ structure)
│   │   ├── c1_vio/   ...                # one folder per component
│   ├── integration/                     # IT-* scenarios from _docs/02_document/tests/blackbox-tests.md (Tier-1 capable)
│   ├── e2e/                             # End-to-end runner — entrypoint for docker-compose.test.yml
│   │   ├── Dockerfile                   # Slim pytest container
│   │   ├── conftest.py
│   │   └── scenarios/                   # FT-P-* / FT-N-* / NFT-LIM (Tier-1) scenarios
│   ├── perf/                            # NFT-PERF-* (Tier-2) scenarios
│   ├── security/                        # NFT-SEC-* scenarios (incl. NFT-SEC-02 network egress, NFT-SEC-05 DNS blackhole)
│   ├── resilience/                      # NFT-RES-* scenarios
│   └── fixtures/
│       ├── flight_derkachi/             # Multi-GB; gitignored; provisioned via DVC or out-of-band
│       ├── tiles_corpus/                # Curated tile snapshots for FT-P-15/16
│       ├── seed-db.sql                  # PostgreSQL seed mirroring satellite-provider's tile rows
│       ├── calibration/
│       │   ├── adti26.json              # Test-fixture camera calibration
│       │   └── adti20.json              # Production calibration (D-PROJ-1 hybrid output, when available)
│       ├── mock-suite-sat-service/      # .NET 8 minimal API implementing the planned D-PROJ-2 ingest contract (e2e fixture only)
│       └── ardupilot_sitl/              # Helper compose for AP SITL (used by IT-3, NFT-PERF-04 AP)
│
├── scripts/
│   ├── run-tests.sh                     # Tier-1 wrapper around docker-compose.test.yml (deferred from test-spec Phase 4)
│   └── run-performance-tests.sh         # Tier-2 wrapper (deferred from test-spec Phase 4)
│
└── _docs/                               # Already exists (problem, solution, document, tasks)

Layout Rationale

  • src/gps_denied_onboard/ is the single top-level Python package (per Python src/-layout convention). All imports are rooted here: from gps_denied_onboard.components.c5_state import StateEstimator, from gps_denied_onboard.helpers.lightglue_runtime import LightGlueRuntime, from gps_denied_onboard._types.pose import EstimatorOutput. One package per repo keeps pip install -e ., mypy --strict, and coverage globs unambiguous.
  • gps_denied_onboard/components/<component>/ — every component is its own folder with interface.py (Protocol/ABC) + one or more concrete impls + a private _internal/ if needed. The components/ subpackage exists (rather than flattening) because (a) per-component tooling globs (--cov-fail-under=90 on src/gps_denied_onboard/components/c5_state/** and .../c8_fc_adapter/** per ci_cd_pipeline.md § Unit) need a clean prefix, (b) it preserves the ADR-009 architectural boundary between "components" (strategy-pattern units) and cross-cutting infra (config/, logging/, _types/, helpers/, fdr_client/), and (c) it gives a 1:1 mirror with _docs/02_document/components/. No reaching across folders for siblings; collaborators are constructor-injected. Build-time exclusion is enforced by CMake reading cmake/build_options.cmake and by the composition root validator refusing to wire a strategy whose BUILD_* flag is OFF.
  • cpp/ is parallel to src/ rather than nested under it because CMake is the source of truth for native builds; the Python package imports the resulting .so files via pybind11 wrappers stored in src/gps_denied_onboard/components/<component>/_native/ (not shown above to keep the tree readable — added per component task in Step 2).
  • gps_denied_onboard/helpers/ is the home of every shared utility called out in _docs/02_document/common-helpers/*.md. R14 (LightGlue circular dep) is structurally prevented because both C2.5 and C3 import gps_denied_onboard.helpers.lightglue_runtime, never each other.
  • tests/ mirrors the component graph 1:1 so a developer landing on src/gps_denied_onboard/components/c5_state/ can find its tests at tests/unit/c5_state/. Cross-component scenarios live in tests/integration/, tests/e2e/, tests/perf/, tests/security/, tests/resilience/ matching the system-level _docs/02_document/tests/*.md files.
  • docker/ and docker-compose*.yml correspond to _docs/02_document/deployment/containerization.md. The bootstrap implementer copies the YAML bodies from there verbatim (see § Acceptance Criteria below).
  • .github/workflows/ mirrors _docs/02_document/deployment/ci_cd_pipeline.md. Two workflows: Tier-1 (everything except AC-bound NFTs) and Tier-2 (AC-bound NFTs on self-hosted Jetson).

DTOs and Interfaces

Detailed DTO field lists are already in architecture.md § 4 Data Model Overview and per-component description.md § 2 Internal Interfaces. The bootstrap task creates type-only stubs (no implementation logic) for the cross-component DTOs listed in src/gps_denied_onboard/_types/ so that downstream component tasks can import and depend on them.

Shared DTOs (cross-component — bootstrap creates type stubs)

DTO File Used By Source
NavCameraFrame _types/nav.py C1, C2, C13 architecture.md § 4
ImuSample, ImuWindow, AttitudeWindow _types/nav.py C1, C5, C8 C8 inbound spec
FlightStateSignal, GpsHealth _types/nav.py C5, C8, C11 C8 inbound spec
VioOutput _types/vio.py C1 → C5; C13 (FDR) C1 spec
VprQuery, VprResult _types/vpr.py C2 → C2.5 C2 spec
RerankResult _types/vpr.py C2.5 → C3 C2.5 spec
MatchResult _types/matching.py C3, C3.5 → C4 C3 spec
PoseEstimate _types/pose.py C4 → C5; C13 C4 spec
EstimatorOutput, EstimatorHealth _types/pose.py C5 → C8, C13 C5 spec
Tile, TileRecord, TileQualityMetadata, SectorClassification _types/tile.py C6, C10, C11 C6 / data_model.md § 2.1
CameraCalibration _types/calibration.py C1, C3, C4 architecture.md § 4
EmittedExternalPosition _types/emitted.py C8 outbound C8 spec
Manifest, EngineCacheEntry _types/manifests.py C10, C7 C10 spec / data_model.md § 2.42.5
FdrRecord (and per-record-type discriminator) gps_denied_onboard/fdr_client/records.py producer side: every component; consumer side: C13 C13 spec

Component Interfaces (bootstrap creates empty interface.py per component)

For each src/gps_denied_onboard/components/<component>/, bootstrap creates:

  • interface.py — the Protocol / ABC taken verbatim from the component spec's § 2 Internal Interfaces
  • __init__.py exporting the interface symbol
  • _native/ placeholder if the component has a C++ binding (C1, C5, C6, C7)
  • tests/__init__.py placeholder under tests/unit/<component>/

Concrete implementations are NOT created here — they are the subject of Step 2 component tasks.

Component Interface Defined in (spec)
C1 VioStrategy components/01_c1_vio/description.md § 2
C2 VprStrategy, VprPipeline components/02_c2_vpr/description.md § 2
C2.5 Reranker components/03_c2_5_rerank/description.md § 2
C3 CrossDomainMatcher components/04_c3_matcher/description.md § 2
C3.5 AdHoPRefinementStrategy components/05_c3_5_adhop/description.md § 2
C4 PoseEstimator components/06_c4_pose/description.md § 2
C5 StateEstimator components/07_c5_state/description.md § 2
C6 TileStore, DescriptorIndex components/08_c6_tile_cache/description.md § 2
C7 InferenceRuntime, EngineLoader components/09_c7_inference/description.md § 2
C8 FcAdapter, GcsAdapter components/10_c8_fc_adapter/description.md § 2
C10 CacheProvisioner components/11_c10_provisioning/description.md § 2
C11 TileDownloader, TileUploader components/12_c11_tilemanager/description.md § 2
C12 CacheBuildWorkflow, OperatorReLocService components/13_c12_operator_tooling/description.md § 2
C13 FdrWriter (consumer side) components/14_c13_fdr/description.md § 2

CI/CD Pipeline

Bootstrap creates the workflow files matching _docs/02_document/deployment/ci_cd_pipeline.md. Stages (verbatim mapping):

Stage File Trigger Quality Gate
Lint .github/workflows/ci.yml every push, every PR ruff + mypy --strict (Python); clang-format --dry-run + clang-tidy (C++); cmake-format --check (CMake); yamllint + markdownlint-cli (YAML/MD)
Unit .github/workflows/ci.yml every push, every PR pytest --cov-fail-under=75 per component; --cov-fail-under=90 on C5, C8; C++ gtest + lcov ≥75% (90% on klt_ransac)
Integration (Tier-1) .github/workflows/ci.yml every push, every PR docker compose -f docker-compose.test.yml up --abort-on-container-exit --exit-code-from e2e-runner --build
Build (dual binary) .github/workflows/ci.yml every push, every PR matrix: [deployment (BUILD_VINS_MONO=OFF, BUILD_SALAD=OFF), research (all ON)] — both must build green
SBOM diff .github/workflows/ci.yml after build ci/sbom_diff.py: deployment SBOM excludes vins_mono, salad, c11_tilemanager (R02 enforcement); deployment ⊂ research
Security .github/workflows/ci.yml after build pip-audit, dotnet list package --vulnerable for mock-sat, Trivy on images, ci/opencv_pin_gate.py ≥4.12.0
Push images (Tier-1) .github/workflows/ci.yml merge to dev/stage/main tag ${BRANCH}-${BUILD_KIND}-${SHORT_SHA}, push to GHCR; PRs do NOT push
Build (Tier-2 deployment) .github/workflows/ci-tier2.yml merge to dev/stage/main native build on [self-hosted, jetson, orin-nano-super]; SBOM byte-equal to Tier-1 deployment
AC-bound NFTs (Tier-2) .github/workflows/ci-tier2.yml merge events; manual on PR NFT-PERF-01..04, NFT-LIM-01..04, NFT-RES-03/04, NFT-SEC-01/03/05, IT-12 — thresholds in tests/traceability-matrix.md
JetPack image build .github/workflows/release.yml tag on main image built + signed + attested; checksum signed by Tier-1 secret manager
Operator tooling tarball .github/workflows/release.yml tag on main bundle: operator-tooling image + mock-suite-sat-service image + compose + verification script + _docs/02_document/
CVE re-scan (monthly) .github/workflows/cve-rescan.yml scheduled cron re-runs security stage against pinned versions; pages onboard team on new High/Critical

Pipeline Configuration Notes

  • Manual-trigger override: AC-bound NFTs initially run on manual trigger only while Tier-2 runner is being provisioned (see deployment_procedures.md § Tier-2 enablement). The merge gate on dev excludes Tier-2; stage/main retain the full gate.
  • Caching keys per ci_cd_pipeline.md § Caching Strategy: Python deps keyed by pyproject.toml hash; C++ build deps keyed by cmake/dependencies.cmake hash; Docker layers keyed by Dockerfile + dep-file hashes; TRT engine cache (Tier-2) keyed by engine_cache_bundle_hash from data_model.md § 2.4.
  • Notifications: Slack #gps-denied-ci for build/Tier-2 failures + SBOM diff fail PR comments; email + page on JetPack signature mismatch.

Environment Strategy

Per architecture.md § 3 Deployment Model. Bootstrap defines dev-tier1 and production-operator-workstation for local execution; staging-tier2 and production are bare JetPack and have no Dockerfile (the JetPack image is assembled by release.yml).

Environment Purpose Configuration Notes
dev-tier1 Local dev, lint, unit, most integration Workstation Docker; LOG_LEVEL=DEBUG; mock-suite-sat-service Docker; test-fixture calibration adti26.json; both BUILD_* tracks available
staging-tier1 CI runs that don't require Jetson GitHub-hosted x86_64 runner; same Docker as dev-tier1
staging-tier2 CI runs that require Jetson (AC-bound) Self-hosted Jetson runner; bare JetPack 6.2; no Docker; both binary tracks built
production Deployed companion image on a UAV Jetson Orin Nano Super; bare JetPack; LOG_LEVEL=INFO to FDR; per-flight MAVLink signing key generated at takeoff load + zeroised at FDR rollover; no inbound network listening + DNS blackhole + iptables OUTPUT REJECT (NFT-SEC-05)
production-operator-workstation Pre-flight tile download + cache build (C10) + post-landing tile upload (C11) + FDR retrieval Operator's Linux workstation; Docker for satellite-provider mirror + operator-tooling

Environment Variables (.env.example)

Variable dev-tier1 staging-tier2 production Description
GPS_DENIED_FC_PROFILE ardupilot_plane ardupilot_plane or inav (matrix) exactly one of ardupilot_plane / inav per build Selects FC adapter at composition root
GPS_DENIED_TIER 1 2 2 Read by composition root + tests to gate hardware-dependent paths
DB_URL postgresql://gps_denied:dev@db:5432/gps_denied postgresql://gps_denied:dev@db:5432/gps_denied local Postgres on companion (NVM) C6 PostgreSQL connection
SATELLITE_PROVIDER_URL http://mock-sat:5100 mix: real satellite-provider (download) + mock-sat:5100 (upload until D-PROJ-2 ships) not set in flight (no egress); operator workstation only Pre-flight tile fetch
CAMERA_CALIBRATION_PATH /fixtures/calibration/adti26.json /fixtures/calibration/adti26.json /etc/gps-denied/calibration/adti20.json Loaded once at startup
LOG_LEVEL DEBUG INFO INFO Structured JSON output
LOG_SINK console journald + fdr fdr only (per-flight ≤64 GB) E-CC-LOG sink selection
MAVLINK_SIGNING_KEY dev key from tests/fixtures/mavlink_signing/dev_key per-flight key from test config generated at takeoff load; rotated per flight; logged to FDR D-C8-9 / R03
BUILD_VINS_MONO both both (matrix) OFF (production-only) CMake-time exclusion (ADR-002)
BUILD_SALAD both both OFF ADR-002
BUILD_C11_TILE_MANAGER ON (operator tooling image) OFF (airborne) / ON (operator tooling) OFF (R02 process isolation) Excludes operator-only code path from airborne image
INFERENCE_BACKEND tensorrt if GPU; else pytorch_fp16 tensorrt tensorrt C7 backend selection
FDR_PATH /var/lib/gps-denied/fdr /var/lib/gps-denied/fdr host NVM mount (≤64 GB) C13 FDR ring path
TILE_CACHE_PATH /var/lib/gps-denied/tiles /var/lib/gps-denied/tiles host NVM mount C6 filesystem tile body store

Database Migration Approach

Migration tool: alembic (Python; matches Python+C++ stack and integrates with pyproject.toml). Strategy: versioned migration scripts under db/migrations/. Additive-only by default per data_model.md § 1 principle #5 — migrations may add tables/columns/indexes/CHECK constraints, may NEVER rename or drop existing columns without an ADR-recorded deprecation window. The tiles schema specifically is frozen on the canonical columns (mirrored from satellite-provider) and only extensible via additive onboard-only columns.

Initial Schema (migration 0001_initial.sql)

Per data_model.md § 2:

  • tiles — mirrored from satellite-provider (canonical columns) + onboard-only additive columns (source CHECK in ('googlemaps', 'onboard_ingest'), tile_quality_metadata jsonb on onboard_ingest rows, voting_status enum)
  • flights — per-flight metadata; FK target for engine_cache_entries and manifests
  • sector_classifications — operator-set; drives freshness threshold via apply_freshness_threshold
  • manifests — D-C10-1 idempotence (hash of model + calibration + corpus + sector classification)
  • engine_cache_entries — TRT engine + INT8 calibration cache keyed by SM/JP/TRT/precision tuple (D-C10-7)

Seed data (tests/fixtures/seed-db.sql): a curated subset of satellite-provider tile rows for tests/fixtures/tiles_corpus/ so Tier-1 integration tests have deterministic input.

Test Structure

tests/
├── conftest.py                          # Postgres bring-up via docker-compose; AP/iNav SITL fixtures; Derkachi corpus mount
├── unit/<component>/                    # Per-component pytest tests; coverage gate per ci_cd_pipeline.md § Unit
├── integration/                         # IT-* scenarios (Tier-1 capable); pytest-driven; uses docker-compose.test.yml
├── e2e/                                 # FT-P-* / FT-N-* / NFT-LIM (Tier-1) scenarios — runs inside e2e-runner sidecar
│   ├── Dockerfile
│   ├── conftest.py
│   └── scenarios/
├── perf/                                # NFT-PERF-* (Tier-2 only); pytest + jetson-stats integration
├── security/                            # NFT-SEC-* incl. NFT-SEC-02 network egress + NFT-SEC-05 DNS blackhole + NFT-SEC-01 cache poisoning
├── resilience/                          # NFT-RES-* — Monte-Carlo + scheduled-perturbation
└── fixtures/
    ├── flight_derkachi/                 # Multi-GB; gitignored; provisioned via DVC or out-of-band
    ├── tiles_corpus/                    # Curated tile snapshots for FT-P-15/16
    ├── seed-db.sql
    ├── calibration/{adti26,adti20}.json
    ├── mavlink_signing/dev_key
    ├── mock-suite-sat-service/          # .NET 8 minimal API; ASP.NET Core; implements POST /api/satellite/tiles/ingest
    └── ardupilot_sitl/

Test Configuration Notes

  • Test runner: pytest for everything Python; gtest for C++ unit tests built under CMake BUILD_TESTING=ON. Both run from the same Tier-1 CI workflow.
  • Fixture lifecycle: conftest.py brings Postgres + mock-sat + ArduPilot SITL up via docker-compose.test.yml; tests that need real Jetson hardware (NFT-PERF, NFT-LIM, NFT-RES on real fixtures, IT-12) live under tests/perf/ + tests/resilience/ and are guarded by pytest.mark.tier2 so they're auto-skipped on Tier-1.
  • Test isolation: Postgres uses transaction-rollback fixtures for unit/integration tests; e2e tests get a fresh DB volume per scenario. Tile cache and FDR mount tmpfs in CI.
  • Test data management: per tests/test-data.md. Large fixtures (Derkachi, tile corpus) live under tests/fixtures/large_replays/ which is gitignored + .dockerignore'd (mounted as volume in docker-compose.test.yml).
  • Source of truth for scenario specs: _docs/02_document/tests/*.md — implementation tasks (Step 9 Decompose Tests) bind these specs to actual test files. Bootstrap only creates the directory scaffolding.

Implementation Order

Driven by the epic dependency graph in _docs/02_document/epics.md § Implementation order:

Order Component Reason
1 Bootstrap (this task) — repo skeleton, Docker, CI, DB schema foundation; no dependencies
2 E-CC-LOG (AZ-245), E-CC-CONF (AZ-246) — cross-cutting unblocks every component's structured logging and config loading
3 E-CC-FDR-CLIENT (AZ-247) — FDR producer client every component's FDR producer depends on the lock-free queue + record schema
4 E-C13 (AZ-248) — FDR writer consumer side; needs records.py from #3; also unblocks E-C7 instrumentation
5 E-C7 (AZ-249) — inference runtime + E-C6 (AZ-250) — tile cache parallel; enable C2 / C3 / C3.5 development
6 E-C11 (AZ-251) — Tile Manager (operator-side) unblocks F1 + F10 tooling
7 E-C10 (AZ-252) — pre-flight cache provisioning → E-C12 (AZ-253) — operator tooling sequential; F1 chain
8 E-C1 (AZ-254) — VIO independent of #5#7; can start in parallel after #2
9 E-C2 (AZ-255) → E-C2.5 (AZ-256) → E-C3 (AZ-257) → E-C3.5 (AZ-258) → E-C4 (AZ-259) F3 hot path; sequential per pipeline order
10 E-C5 (AZ-260) — state estimator needs C1 + C4; the largest single epic
11 E-C8 (AZ-261) — FC + GCS adapter needs C5; gates IT-3 (R03 lock)
12 E-BBT (AZ-262) — system-level FT/NFT scenarios hardens behind every other epic; per-component tests live in their own epics

Acceptance Criteria

AC-1: Project scaffolded matching the layout above Given the structure plan above When the implementer executes this task Then every folder shown in ## Project Folder Layout exists, every __init__.py / interface.py placeholder exists, and pyproject.toml + CMakeLists.txt + cmake/dependencies.cmake + cmake/build_options.cmake parse without error.

AC-2: DTO type stubs importable Given the bootstrap output When python -c "from gps_denied_onboard._types import nav, vio, vpr, matching, pose, tile, calibration, emitted, manifests" is run Then no ImportError raises (the modules need not implement methods — only declare the classes/Protocols/dataclasses).

AC-3: Three Dockerfiles + two compose files match containerization.md Given the bootstrap output When docker compose -f docker-compose.yml config --quiet and docker compose -f docker-compose.test.yml config --quiet are run Then both exit zero (compose files are syntactically valid + service references resolve). The Dockerfile bodies match the per-image specifications in _docs/02_document/deployment/containerization.md § Component Dockerfiles.

AC-4: CI workflow stubs are syntactically valid Given the bootstrap output When the GitHub Actions workflows under .github/workflows/ are validated (actionlint or equivalent) Then they parse without error AND the dual-binary build matrix (deployment + research) is present in ci.yml per ADR-002.

AC-5: Database migration tool wired Given the bootstrap output When alembic check is run against an empty database Then it reports 0001_initial.sql as the head revision and the schema matches the canonical tiles columns from satellite-provider (with the additive onboard-only columns documented in data_model.md § 2.1.1).

AC-6: Health checks defined Given the bootstrap output Then src/gps_denied_onboard/healthcheck.py exists and is importable; the operator-tooling Dockerfile + companion-tier1 Dockerfile both reference it via HEALTHCHECK CMD; mock-suite-sat-service Dockerfile defines GET /healthz returning 200 when storage backend is mounted.

AC-7: Structured logging entrypoint exists Given the bootstrap output Then src/gps_denied_onboard/logging/structured.py exists, exports get_logger(name), and emits one JSON object per log line (no narrative log lines per E-CC-LOG / AZ-245 contract).

AC-8: .env.example documents every variable used by the composition root Given the bootstrap output Then .env.example matches the table in § Environment Strategy above; missing required variables cause runtime_root.py to refuse startup with a clear error pointing at the offending variable name.

AC-9: Stub tests pass Given the scaffolded project When pytest -q runs against tests/unit/ Then every component folder has at least one test_smoke.py whose only assertion is that the component's interface is importable; the run exits zero.

AC-10: SBOM diff script + OpenCV pin gate exist and run on stub builds Given the bootstrap output Then ci/sbom_diff.py --deployment <stub> --research <stub> runs without exception; ci/opencv_pin_gate.py exits zero when fed a pyproject.toml with opencv-python ≥ 4.12.0.

Out of Scope (deferred to other tasks)

  • Concrete implementations of any component interface — that's Step 2 component tasks (one task per child issue from epics.md).
  • Module layout — that's Decompose Step 1.5 (module-layout.md).
  • Test scenario implementation — that's Step 9 Decompose Tests + Step 10 Implement Tests.
  • D-PROJ-2 ingest contract implementation in mock-suite-sat-service — initial Dockerfile + minimal API placeholder only; the actual contract is implemented as a scoped task once the parent-suite design lands.
  • TensorRT engine compilation, GTSAM iSAM2 substrate wiring, FAISS HNSW index build — owned by C7, C5, C6 epics respectively.