azaion/gps-denied-onboard
1
0
Fork 0
mirror of https://github.com/azaion/gps-denied-onboard.git synced 2026-06-21 15:01:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
6ca8d78190fcaa4a5a2dc78da0a77932d4e2145d
gps-denied-onboard/_docs/02_document/module-layout.md
T
Oleksandr Bezdieniezhnykh 0dfe7c5301 [AZ-321] C10 EngineCompiler: hardware-tied TRT compile + cache reuse 
Land the C10 per-model engine compile + cache-reuse orchestrator.
`EngineCompiler.compile_engines_for_corpus(request)` walks the
corpus, computes the canonical engine filename via AZ-281
`EngineFilenameSchema.build`, and either reuses the cached binary
(cache hit, AZ-280 `Sha256Sidecar.verify` returns True) or delegates
to the AZ-297 `compile_engine` on the injected runtime (cache miss;
the runtime owns the write path). Returns one `EngineCompileResult`
per backbone carrying the canonical `EngineCacheEntry`, outcome
(BUILT / REUSED), and `compile_duration_s` (None on reuse).
Hardware-tied reuse (D-C10-6 / D-C10-7) falls out of the filename
schema — a host change rebuilds at the new path and leaves the old
files untouched (AC-4).

Design corrections vs. the task spec body:
- The spec proposed a c10-local `EngineCacheEntry` carrying outcome
  and duration; that name is already taken by the AZ-297 canonical
  DTO. The wrapper is renamed `EngineCompileResult`; the canonical
  shape wins.
- The spec called `InferenceRuntime.host_info()`, which is not in
  the AZ-297 Protocol. `HostCapabilities` is threaded through
  `EngineCompileRequest` instead so the composition root owns host
  probing and the compiler stays decoupled.
- The c10 layer cannot import `components.c7_inference` (arch rule
  `test_az270_compose_root.test_ac6`). `engine_compiler.py` defines
  `CompileEngineCallable` — a structural Protocol cut of
  `InferenceRuntime` exposing only `compile_engine` — and catches
  broad `Exception` (re-raising preserves the original type;
  `error_class` is recorded in the ERROR log payload).

Production
- engine_compiler.py: `CompileOutcome` enum, `BackboneSpec`,
  `EngineCompileRequest`, `EngineCompileResult`,
  `EngineCompileSummary` DTOs; `CompileEngineCallable` Protocol;
  `EngineCompiler` with the single public method.
- config.py: `BackboneConfig` + `C10ProvisioningConfig`
  (`workspace_mb` default 4 GiB to match C7 NFT-LIM-01); validate
  positive shape dims and duplicate model_name detection in
  `__post_init__`.
- runtime_root/c10_factory.py: `build_engine_compiler(config)` wires
  the existing `build_inference_runtime` factory through;
  `build_backbone_specs(config)` materialises the `BackboneSpec`
  tuple from the config block.
- components/c10_provisioning/__init__.py: re-exports the AZ-321
  surface and registers the new config block.

Tests
- test_engine_compiler.py: covers AC-1..AC-10 + missing-sidecar
  sibling case for AC-5. Tier-1 via fake runtime that writes through
  the REAL `Sha256Sidecar.write_atomic_and_sidecar`. Tier-2
  placeholders for the cache-hit p99 NFR (200 MB engine sweep) and
  kill-during-compile atomic-write NFR.

Docs
- module-layout.md: c10_provisioning Per-Component Mapping lists the
  new internal modules (engine_compiler.py, config.py), the
  composition-root c10_factory.py, the AZ-321 public re-export
  surface, and the registered config block.
- batch_33_cycle1_report.md + reviews/batch_33_review.md:
  PASS_WITH_WARNINGS (4 Low findings accepted).

Tests run: c10_provisioning 13 passing + 2 Tier-2 skips; combined
unit suite (excluding pending components) 543 passing, 21
env-skipped.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-13 00:09:53 +03:00

36 KiB
Raw Blame History

Module Layout

Language: python (with C++ native libraries linked via pybind11 from a parallel cpp/ tree) Layout Convention: src-layout (single top-level package src/gps_denied_onboard/) Root: src/gps_denied_onboard/ Last Updated: 2026-05-10

This file is the authoritative file-ownership map consumed by the /implement skill (Step 4 File Ownership). Per-task specs in _docs/02_tasks/ remain purely behavioral — they do NOT carry file paths. All component → filesystem mapping lives here.

Bootstrap reference: _docs/02_tasks/todo/AZ-263_initial_structure.md. Architecture reference: _docs/02_document/architecture.md (ADR-001 monolith, ADR-002 build-time exclusion, ADR-009 interface-first DI).

Layout Rules

  1. The single top-level Python package is src/gps_denied_onboard/. All imports are rooted there. No sibling packages live under src/.
  2. Each component owns ONE folder under src/gps_denied_onboard/components/. Folder name = component slug (lowercase, snake_case, e.g. c1_vio, c2_vpr, c2_5_rerank).
  3. Cross-cutting concerns own ONE folder each directly under src/gps_denied_onboard/: _types/, helpers/, config/, logging/, fdr_client/, frame_source/, clock/. Plus runtime_root.py and healthcheck.py at the package root.
  4. Native (C++) libraries live under cpp/ (parallel to src/, NOT nested), built by CMake; per-component pybind11 wrappers live at src/gps_denied_onboard/components/<component>/_native/<name>.py and import the resulting .so from a CMake-known path.
  5. Public API surface per component = the files listed in each component's Public API list below. Anything not listed is internal and MUST NOT be imported from another component.
  6. The composition root is src/gps_denied_onboard/runtime_root.py. It is the ONLY place that may import concrete strategy implementations across components — every other cross-component dependency is constructor-injected against an interface (ADR-009).
  7. Tests mirror the component graph 1:1 at tests/unit/<component>/. Cross-component scenarios live in tests/integration/, tests/e2e/, tests/perf/, tests/security/, tests/resilience/.
  8. Build-time exclusion (ADR-002): each <component>/_native/ and the corresponding cpp/<lib>/ carry a CMake BUILD_<NAME> flag. The composition root validator refuses to wire a strategy whose flag is OFF.

Per-Component Mapping

Component: c1_vio

  • Epic: AZ-254 (E-C1 VIO)
  • Directory: src/gps_denied_onboard/components/c1_vio/
  • Public API:
    • src/gps_denied_onboard/components/c1_vio/__init__.py (re-exports VioStrategy, VioOutput)
    • src/gps_denied_onboard/components/c1_vio/interface.py (VioStrategy Protocol)
  • Internal (do NOT import from other components):
    • src/gps_denied_onboard/components/c1_vio/okvis2.py (production-default; links cpp/okvis2/)
    • src/gps_denied_onboard/components/c1_vio/vins_mono.py (research-only; gated by BUILD_VINS_MONO=ON)
    • src/gps_denied_onboard/components/c1_vio/klt_ransac.py (mandatory simple-baseline)
    • src/gps_denied_onboard/components/c1_vio/_native/
  • Owns (exclusive write during implementation): src/gps_denied_onboard/components/c1_vio/**, cpp/okvis2/**, cpp/vins_mono/**, cpp/klt_ransac/**, tests/unit/c1_vio/**
  • Imports from: _types, helpers.imu_preintegrator, helpers.se3_utils, config, logging, fdr_client
  • Consumed by: c2_vpr, c5_state, c13_fdr, runtime_root

Component: c2_vpr

  • Epic: AZ-255 (E-C2 VPR)
  • Directory: src/gps_denied_onboard/components/c2_vpr/
  • Public API:
    • __init__.py (re-exports VprStrategy, VprQuery, VprResult)
    • interface.py (VprStrategy Protocol)
  • Internal:
    • ultra_vpr.py (primary), mega_loc.py, mix_vpr.py, sela_vpr.py, eigen_places.py, net_vlad.py, salad.py
    • _native/
  • Owns: src/gps_denied_onboard/components/c2_vpr/**, tests/unit/c2_vpr/**
  • Imports from: _types, helpers.descriptor_normaliser, components.c6_tile_cache (Public API only — TileStore query interface), components.c7_inference (InferenceRuntime), config, logging, fdr_client
  • Consumed by: c2_5_rerank, runtime_root

Component: c2_5_rerank

  • Epic: AZ-256 (E-C2.5 Rerank)
  • Directory: src/gps_denied_onboard/components/c2_5_rerank/
  • Public API:
    • __init__.py (re-exports ReRankStrategy, RerankResult, RerankCandidate, RerankError family, C2_5RerankConfig)
    • interface.py (ReRankStrategy Protocol)
    • config.py (C2_5RerankConfig dataclass; registered on import; strategy, top_n, debug_per_frame_log fields)
    • errors.py (RerankError, RerankBackboneError, RerankAllCandidatesFailedError)
  • Internal:
    • inlier_based_reranker.py (InlierCountReRanker — single-pair LightGlue inlier count K=10→N=3, AZ-343; module-level create() factory entry-point consumed by runtime_root.rerank_factory.build_rerank_strategy; gated by BUILD_RERANK_INLIER_COUNT)
  • Owns: src/gps_denied_onboard/components/c2_5_rerank/**, src/gps_denied_onboard/runtime_root/rerank_factory.py, tests/unit/c2_5_rerank/**
  • Imports from: _types, helpers.lightglue_runtime, helpers.feature_extractor (AZ-343 scope expansion), helpers.descriptor_normaliser, helpers.ransac_filter, helpers.se3_utils, components.c6_tile_cache (Public API only — TileStore, TilePixelHandle, TileCacheError family), clock, config, logging, fdr_client
  • Consumed by: c3_matcher, runtime_root

Component: c3_matcher

  • Epic: AZ-257 (E-C3 Cross-Domain Matcher)
  • Directory: src/gps_denied_onboard/components/c3_matcher/
  • Public API:
    • __init__.py (re-exports CrossDomainMatcher, MatchResult, MatcherHealth, CandidateMatchSet, MatcherError, MatcherBackboneError, InsufficientInliersError, C3MatcherConfig)
    • interface.py (CrossDomainMatcher Protocol)
    • config.py (C3MatcherConfig)
    • errors.py (error hierarchy)
  • Internal:
    • _health_window.py (RollingHealthWindow accumulator; constructor-injected into every concrete matcher)
    • disk_lightglue.py (DISK + LightGlue, AZ-345)
    • aliked_lightglue.py (ALIKED + LightGlue, AZ-346)
    • xfeat.py (XFeat, AZ-347)
    • _native/
  • Owns: src/gps_denied_onboard/components/c3_matcher/**, tests/unit/c3_matcher/**, src/gps_denied_onboard/runtime_root/matcher_factory.py
  • Imports from: _types, helpers.lightglue_runtime (R14: SHARED with C2.5 — owned by helper, NOT by C3), helpers.ransac_filter, helpers.descriptor_normaliser, helpers.se3_utils, components.c7_inference, config, logging, fdr_client
  • Consumed by: c3_5_adhop, runtime_root

Component: c3_5_adhop

  • Epic: AZ-258 (E-C3.5 AdHoP Refinement)
  • Directory: src/gps_denied_onboard/components/c3_5_adhop/
  • Public API:
    • __init__.py (re-exports ConditionalRefiner, C3_5RefinerConfig)
    • interface.py (ConditionalRefiner Protocol)
    • config.py (C3_5RefinerConfig)
    • errors.py (RefinerError, RefinerBackboneError, RefinerConfigError — held internal to the component; consumers reach them only via tests)
  • Internal:
    • passthrough_refiner.py (reference baseline; AZ-348)
    • adhop_refiner.py (production-default; AZ-349 pending)
  • Owns: src/gps_denied_onboard/components/c3_5_adhop/**, tests/unit/c3_5_adhop/**, src/gps_denied_onboard/runtime_root/refiner_factory.py
  • Imports from: _types, helpers.ransac_filter (R14: SHARED with C3 and C4 — owned by helper, NOT by C3.5), helpers.se3_utils, components.c7_inference, config, logging, fdr_client
  • Consumed by: c4_pose, runtime_root

Component: c4_pose

  • Epic: AZ-259 (E-C4 Pose Estimator)
  • Directory: src/gps_denied_onboard/components/c4_pose/
  • Public API:
    • __init__.py (re-exports PoseEstimator, PoseEstimate, EstimatorOutput)
    • interface.py (PoseEstimator Protocol)
  • Internal:
    • opencv_pnp_estimator.py (OpenCV solvePnPRansac + GTSAM Marginals for covariance)
    • _native/ (GTSAM bindings via cpp/gtsam_bindings/)
  • Owns: src/gps_denied_onboard/components/c4_pose/**, cpp/gtsam_bindings/** (shared with c5_state — see ownership note below), tests/unit/c4_pose/**
  • Imports from: _types, helpers.ransac_filter, helpers.se3_utils, helpers.wgs_converter, config, logging, fdr_client
  • Consumed by: c5_state, runtime_root

Joint native ownership note: cpp/gtsam_bindings/ is a thin pybind11 wrapper used by both C4 (Marginals for covariance) and C5 (iSAM2 + IncrementalFixedLagSmoother). Implementation task for cpp/gtsam_bindings/ is owned by c5_state (the heavier consumer); c4_pose imports it READ-ONLY. See Layering table below.

Component: c5_state

  • Epic: AZ-260 (E-C5 State Estimator)
  • Directory: src/gps_denied_onboard/components/c5_state/
  • Public API:
    • __init__.py (re-exports StateEstimator, EstimatorOutput, EstimatorHealth)
    • interface.py (StateEstimator Protocol)
  • Internal:
    • gtsam_isam2_estimator.py (production-default; iSAM2 + IncrementalFixedLagSmoother)
    • eskf_baseline.py (mandatory simple-baseline)
    • _native/
  • Owns: src/gps_denied_onboard/components/c5_state/**, cpp/gtsam_bindings/** (primary owner; see joint-native note above), tests/unit/c5_state/**
  • Imports from: _types, helpers.imu_preintegrator, helpers.se3_utils, helpers.wgs_converter, components.c4_pose (Public API: PoseEstimate), config, logging, fdr_client
  • Consumed by: c8_fc_adapter, c13_fdr, runtime_root

Component: c6_tile_cache

  • Epic: AZ-250 (E-C6 Tile Cache & Vector Index)
  • Directory: src/gps_denied_onboard/components/c6_tile_cache/
  • Public API (__init__.py __all__; consult the module for the canonical list):
    • interface.py (three Protocols: TileStore, TileMetadataStore, DescriptorIndex — query/get/put surface; concrete impls swappable)
    • DTOs: TileId, TileMetadata, TileMetadataPersistent, TileQualityMetadata, Bbox, SectorBoundary, HnswParams, IndexMetadata
    • Enums: TileSource, FreshnessLabel, VotingStatus, SectorClassification
    • ABC: TilePixelHandle
    • Config block: C6TileCacheConfig (registered on import)
    • Error family rooted at TileCacheError with documented subtypes (TileNotFoundError, TileFsError, TileMetadataError, ContentHashMismatchError, FreshnessRejectionError, CacheBudgetExhaustedError, IndexUnavailableError) + sibling IndexBuildError (offline build envelope, not in the TileCacheError family)
  • Internal:
    • _native/ (FAISS C++ wrapper, planned)
    • _tile_pixel_handle.py (TilePixelHandle ABC)
    • _types.py (DTOs / enums; consumed via the Public API re-exports)
    • _uuid_namespace.py (AZ-304 — pinned TILE_NAMESPACE_UUID + derive_tile_id / derive_location_hash helpers; cross-repo coordinated with satellite-provider)
    • migrations.py (AZ-304 — apply_migrations(config) -> MigrationResult runner invoked by the composition root at startup)
    • postgres_filesystem_store.py (AZ-305 — production-default TileStore + TileMetadataStore impl over Postgres mirror + filesystem; PostgresFilesystemStore.from_config opens its own psycopg_pool.ConnectionPool and constructs an AZ-307 FreshnessGate)
    • freshness_gate.py (AZ-307 — ACTIVE_CONFLICT reject + STABLE_REAR downgrade per freshness_gate.md v1.0.0)
    • cache_budget_enforcer.py (AZ-308 — RESTRICT-SAT-2 10 GiB hard cap; CacheBudgetEnforcer.reserve_headroom + BudgetEnforcedTileStore write-decorator)
    • tools.py (AZ-305 — operator dump CLI invoked via python -m gps_denied_onboard.components.c6_tile_cache.tools ...)
    • errors.py, config.py (component plumbing)
  • Owns: src/gps_denied_onboard/components/c6_tile_cache/**, cpp/faiss_index/**, tests/unit/c6_tile_cache/**, db/migrations/** (project-level Alembic env owned by c6 — alembic.ini at repo root points here; 0001_initial.py shipped by AZ-263 bootstrap, 0002_c6_tile_identity_and_lru.py and forward owned by AZ-304+ migrations)
  • Imports from: _types, helpers.sha256_sidecar, helpers.wgs_converter, clock, config, logging, fdr_client
  • Consumed by: c2_vpr, c2_5_rerank, c3_matcher, c10_provisioning, c11_tile_manager, runtime_root

Component: c7_inference

  • Epic: AZ-249 (E-C7 Inference Runtime)
  • Directory: src/gps_denied_onboard/components/c7_inference/
  • Public API (__init__.py __all__; consult the module for the canonical list):
    • interface.py (InferenceRuntime Protocol)
    • DTOs re-exported from _types.inference + _types.thermal: BuildConfig, EngineCacheEntry, EngineHandle, OptimizationProfile, PrecisionMode, ThermalState
    • Component services: EngineGate + HostTuple (AZ-301), ThermalStatePublisher + ThermalReading + ThermalSource Protocol (AZ-302), ManifestReader + ManifestReaderProtocol + DeploymentManifest (AZ-301), ArchitectureFactory + default_registry + register_architecture (AZ-300)
    • Config block: C7InferenceConfig (registered on import)
    • Error family rooted at RuntimeError with documented subtypes (InferenceError, EngineBuildError, EngineDeserializeError, EngineHashMismatchError, EngineSchemaMismatchError, EngineSidecarMissingError, CalibrationCacheError, OutOfMemoryError, TelemetryUnavailableError)
  • Internal:
    • architecture_registry.py (AZ-300; family of registered ArchitectureFactory callables consumed by PytorchFp16Runtime)
    • config.py (C7InferenceConfig dataclass; registered on import)
    • engine_gate.py (AZ-301; D-C10-3 + D-C10-7 takeoff validator)
    • errors.py (component error family)
    • manifest.py (AZ-301; DeploymentManifest + ManifestReader for engine sidecar manifests)
    • onnx_trt_ep_runtime.py (AZ-299; ONNX Runtime + TensorRT EP fallback strategy + per-flight ORT TRT subgraph cache + one-shot fallback WARN/FDR/GCS alert + CPU-fallback gate)
    • pytorch_fp16_runtime.py (AZ-300; research-only / simple-baseline strategy)
    • tensorrt_runtime.py (AZ-298; production-default TensorRT 10.3 strategy + INT8 calibration cache trust + GPU memory budget enforcement + python -m ...tensorrt_runtime compile ... CLI)
    • thermal_publisher.py (AZ-302; 1 Hz background poller, jtop/NVML fallback)
  • Owns: src/gps_denied_onboard/components/c7_inference/**, tests/unit/c7_inference/**
  • Imports from: _types, helpers.engine_filename_schema, helpers.sha256_sidecar, config, logging, fdr_client
  • Consumed by: c2_vpr, c2_5_rerank, c3_matcher, c10_provisioning, runtime_root

Component: c8_fc_adapter

  • Epic: AZ-261 (E-C8 FC + GCS Adapter)
  • Replay extensions epic: AZ-265 (E-DEMO-REPLAY) — adds tlog_replay_adapter.py + replay_sink.py as gated strategies
  • Directory: src/gps_denied_onboard/components/c8_fc_adapter/
  • Public API:
    • __init__.py (re-exports FcAdapter, GcsAdapter, ReplaySink, EmittedExternalPosition)
    • interface.py (FcAdapter, GcsAdapter, ReplaySink Protocols)
  • Internal:
    • pymavlink_ardupilot_adapter.py (ArduPilot Plane via pymavlink)
    • msp2_inav_adapter.py (iNav via MSP2)
    • mavlink_gcs_adapter.py (1–2 Hz downsampled summary to QGroundControl)
    • tlog_replay_adapter.py (replay-only FcAdapter; gated BUILD_TLOG_REPLAY_ADAPTER; AZ-265)
    • replay_sink.py (ReplaySink interface + JsonlReplaySink impl; gated BUILD_REPLAY_SINK_JSONL; AZ-265)
  • Owns: src/gps_denied_onboard/components/c8_fc_adapter/**, tests/unit/c8_fc_adapter/**
  • Imports from: _types, helpers.wgs_converter, helpers.se3_utils, components.c5_state (Public API: EstimatorOutput), config, logging, fdr_client, clock (for replay timer-injection)
  • Consumed by: c1_vio (back-channel: ImuSample, AttitudeWindow), c5_state (back-channel: ImuSample, FlightStateSignal, GpsHealth), runtime_root (live + operator + replay binaries)

Back-channel note: C8 is the source of inbound IMU / attitude / GPS-health signals from the FC. C1 and C5 receive these via constructor-injected FcAdapter (typed against the interface, not the concrete adapter). This is NOT a layering violation — C8's role spans both the outbound emit path AND the inbound telemetry source.

Component: c10_provisioning

  • Epic: AZ-252 (E-C10 Cache Provisioner)
  • Directory: src/gps_denied_onboard/components/c10_provisioning/
  • Public API:
    • __init__.py (re-exports CacheProvisioner, Manifest, EngineCacheEntry, plus AZ-321 surface: EngineCompiler, BackboneSpec, EngineCompileRequest, EngineCompileResult, CompileOutcome, EngineCompileSummary, CompileEngineCallable, BackboneConfig, C10ProvisioningConfig)
    • interface.py (CacheProvisioner Protocol)
    • Config block: C10ProvisioningConfig (registered on import)
  • Internal:
    • engine_compiler.py (AZ-321; per-model TRT compile + hardware-tied cache reuse + CompileEngineCallable structural cut of the C7 InferenceRuntime)
    • config.py (AZ-321; BackboneConfig + C10ProvisioningConfig dataclasses)
    • default_provisioner.py (engine compile + descriptors + manifest + content-hash gate, pending)
    • Composition root: runtime_root/c10_factory.py (build_engine_compiler, build_backbone_specs)
  • Owns: src/gps_denied_onboard/components/c10_provisioning/**, tests/unit/c10_provisioning/**
  • Imports from: _types, helpers.sha256_sidecar, helpers.engine_filename_schema, helpers.wgs_converter, components.c6_tile_cache (Public API), components.c7_inference (Public API: engine compile surface), config, logging, fdr_client
  • Consumed by: c12_operator_tooling, runtime_root (operator binary only — excluded from airborne via BUILD_C10_PROVISIONING=OFF for airborne build per ADR-002)

Component: c11_tile_manager

  • Epic: AZ-251 (E-C11 Tile Downloader/Uploader)
  • Directory: src/gps_denied_onboard/components/c11_tile_manager/
  • Public API:
    • __init__.py (re-exports TileDownloader, TileUploader)
    • interface.py (TileDownloader, TileUploader Protocols)
  • Internal:
    • satellite_provider_downloader.py (REST client against parent-suite satellite-provider)
    • satellite_provider_uploader.py (post-landing batch upload, D-PROJ-2 ingest contract)
  • Owns: src/gps_denied_onboard/components/c11_tile_manager/**, tests/unit/c11_tile_manager/**
  • Imports from: _types, helpers.sha256_sidecar, helpers.wgs_converter, components.c6_tile_cache (Public API), config, logging, fdr_client
  • Consumed by: c12_operator_tooling, runtime_root (operator binary only — BUILD_C11_TILE_MANAGER=OFF for airborne)

Component: c12_operator_tooling

  • Epic: AZ-253 (E-C12 Operator Pre-flight Tooling)
  • Directory: src/gps_denied_onboard/components/c12_operator_tooling/
  • Public API:
    • __init__.py (re-exports CacheBuildWorkflow, OperatorReLocService)
    • interface.py
  • Internal:
    • cache_build_workflow.py (CLI orchestrator)
    • operator_reloc_service.py (CLI; GUI deferred per epic)
    • sector_classifier.py (operator sets SectorClassification → C6)
  • Owns: src/gps_denied_onboard/components/c12_operator_tooling/**, tests/unit/c12_operator_tooling/**
  • Imports from: _types, helpers.wgs_converter, components.c6_tile_cache (Public API), components.c10_provisioning (Public API), components.c11_tile_manager (Public API), config, logging, fdr_client
  • Consumed by: runtime_root (operator binary only — BUILD_C12_OPERATOR_TOOLING=OFF for airborne)

Component: c13_fdr

  • Epic: AZ-248 (E-C13 FDR Writer)
  • Directory: src/gps_denied_onboard/components/c13_fdr/
  • Public API:
    • __init__.py (re-exports FdrWriter)
    • interface.py (FdrWriter Protocol)
  • Internal:
    • default_fdr_writer.py (writer thread + segment rotation + ≤64 GB cap)
  • Owns: src/gps_denied_onboard/components/c13_fdr/**, tests/unit/c13_fdr/**
  • Imports from: _types, fdr_client.records (FdrRecord schema only — schema lives in cross-cutting fdr_client; the consumer-side writer lives here), config, logging
  • Consumed by: runtime_root (every component's fdr_client producer ultimately writes to the C13 writer at process root)

C13 / fdr_client split: the producer-side FdrClient (lock-free SPSC queue + record schema) lives in src/gps_denied_onboard/fdr_client/ (cross-cutting; AZ-247 / E-CC-FDR-CLIENT). The consumer-side FdrWriter (writer thread + segment rotation) lives in components/c13_fdr/ (AZ-248 / E-C13). This split is intentional: every component depends on the producer interface, but only the writer process implements the consumer.

Shared / Cross-Cutting

shared/_types

  • Directory: src/gps_denied_onboard/_types/
  • Purpose: Cross-component DTOs (NavCameraFrame, ImuSample, ImuWindow, AttitudeWindow, FlightStateSignal, GpsHealth, VioOutput, VprQuery, VprResult, RerankResult, MatchResult, PoseEstimate, EstimatorOutput, EstimatorHealth, Tile, TileQualityMetadata, TileRecord, SectorClassification, CameraCalibration, EmittedExternalPosition, Manifest, EngineCacheEntry). Type-only stubs: zero implementation logic.
  • Owned by: AZ-263 (Bootstrap task); subsequent additions are type-only edits owned by the proposing component task.
  • Consumed by: every component, every cross-cutting module, the composition root.

shared/config

  • Directory: src/gps_denied_onboard/config/
  • Purpose: YAML config loader + validation + dataclass schemas (per-flight config + camera calibration JSON loader).
  • Owned by: AZ-263 (Bootstrap); subsequent schema fields added by the consuming component task touching schema.py only.
  • Consumed by: composition root + every component constructor that reads config.

shared/logging

  • Directory: src/gps_denied_onboard/logging/
  • Purpose: Structured JSON logging (one JSON object per line; no narrative log lines).
  • Owned by: AZ-245 (E-CC-LOG — Cross-Cutting Logging) — bootstrap creates the entrypoint stub satisfying the contract.
  • Consumed by: every component (via from gps_denied_onboard.logging.structured import get_logger).

shared/fdr_client

  • Directory: src/gps_denied_onboard/fdr_client/
  • Purpose: Producer-side API for FDR records (lock-free SPSC queue per producer, drop-oldest on overrun) + the FdrRecord schema.
  • Owned by: AZ-247 (E-CC-FDR-CLIENT — Cross-Cutting FDR Client).
  • Consumed by: every component's producer code path; the consumer (writer thread) is C13.

shared/helpers/imu_preintegrator

  • Directory: src/gps_denied_onboard/helpers/imu_preintegrator.py
  • Purpose: IMU preintegration utility (see _docs/02_document/common-helpers/01_helper_imu_preintegrator.md).
  • Owned by: AZ-264 (E-CC-HELPERS — Common Helpers); per-helper tasks live under that epic.
  • Consumed by: c1_vio, c5_state.

shared/helpers/se3_utils

  • Directory: src/gps_denied_onboard/helpers/se3_utils.py
  • Purpose: SE(3) math utilities (02_helper_se3_utils.md).
  • Owned by: AZ-264.
  • Consumed by: c1_vio, c2_5_rerank, c3_matcher, c3_5_adhop, c4_pose, c5_state, c8_fc_adapter.

shared/helpers/lightglue_runtime

  • Directory: src/gps_denied_onboard/helpers/lightglue_runtime.py
  • Purpose: Shared LightGlue inference runtime (03_helper_lightglue_runtime.md). R14 fix: this helper is the single owner; both C2.5 (single-pair inlier counter) and C3 (matcher) import it. Neither depends on the other.
  • Owned by: AZ-264.
  • Consumed by: c2_5_rerank, c3_matcher.

shared/helpers/feature_extractor

  • Directory: src/gps_denied_onboard/helpers/feature_extractor.py
  • Purpose: Shared image → KeypointSet Protocol + placeholder OpenCvOrbExtractor impl (AZ-343 scope expansion). Lets every consumer that feeds LightGlueRuntime.match reach for the SAME extractor (same descriptor distribution, same descriptor_dim) without each strategy reinventing its own preprocessing.
  • Owned by: AZ-343.
  • Consumed by: c2_5_rerank (today via InlierCountReRanker), c3_matcher (future concrete strategies in AZ-345 / AZ-346 / AZ-347).

shared/helpers/wgs_converter

  • Directory: src/gps_denied_onboard/helpers/wgs_converter.py
  • Purpose: WGS84 ↔ local-tangent-plane conversion utilities (04_helper_wgs_converter.md).
  • Owned by: AZ-264.
  • Consumed by: c4_pose, c5_state, c6_tile_cache, c8_fc_adapter, c10_provisioning, c11_tile_manager, c12_operator_tooling.

shared/helpers/sha256_sidecar

  • Directory: src/gps_denied_onboard/helpers/sha256_sidecar.py
  • Purpose: Content-hash sidecar files (D-C10-3 content-hash gate; 05_helper_sha256_sidecar.md).
  • Owned by: AZ-264.
  • Consumed by: c6_tile_cache, c7_inference, c10_provisioning, c11_tile_manager.

shared/helpers/engine_filename_schema

  • Directory: src/gps_denied_onboard/helpers/engine_filename_schema.py
  • Purpose: Self-describing TensorRT engine filename schema (D-C10-7; 06_helper_engine_filename_schema.md).
  • Owned by: AZ-264.
  • Consumed by: c7_inference, c10_provisioning.

shared/helpers/ransac_filter

  • Directory: src/gps_denied_onboard/helpers/ransac_filter.py
  • Purpose: Generic RANSAC inlier filter (07_helper_ransac_filter.md).
  • Owned by: AZ-264.
  • Consumed by: c2_5_rerank, c3_5_adhop, c4_pose.

shared/helpers/descriptor_normaliser

  • Directory: src/gps_denied_onboard/helpers/descriptor_normaliser.py
  • Purpose: Descriptor normalisation utility (08_helper_descriptor_normaliser.md).
  • Owned by: AZ-264.
  • Consumed by: c2_vpr, c2_5_rerank, c3_matcher.

shared/frame_source

  • Directory: src/gps_denied_onboard/frame_source/
  • Purpose: FrameSource interface (formalised cross-cutting; previously implicit "camera ingest thread" in architecture) + LiveCameraFrameSource (existing live path, retrofitted) + VideoFileFrameSource (replay-only; reads .mp4 / .h264 and emits NavCameraFrame at configured FPS).
  • Owned by: AZ-265 (E-DEMO-REPLAY); the interface itself + LiveCameraFrameSource retrofit are cycle-1 deliverables under AZ-265 child task #1 (Decompose Step 2 amendment — interface was previously implicit).
  • Consumed by: c1_vio (constructor-injected), runtime_root (composes the right strategy per binary).

shared/clock

  • Directory: src/gps_denied_onboard/clock/
  • Purpose: Clock interface + WallClock (live) + TlogDerivedClock (replay). Per R-DEMO-4: production C1–C5 paths bake real-time-cadence assumptions (e.g., AC-5.2 3 s no-estimate fallback timer); injected Clock lets replay mode trip those timers consistently against tlog timestamps rather than wall-clock.
  • Owned by: AZ-265 (E-DEMO-REPLAY) — child task #4 (compose_replay + Clock injection).
  • Consumed by: c1_vio, c5_state, c8_fc_adapter, any component with timer-driven fallback logic; runtime_root (selects WallClock for live/research/operator, TlogDerivedClock for replay).

shared/runtime_root

  • File: src/gps_denied_onboard/runtime_root.py
  • Purpose: Composition root — config → strategy resolution → graph wiring (ADR-009). The ONLY place that may import concrete strategy classes across components. Per-binary CMake BUILD_* flags + composition root validator enforce ADR-002 build-time exclusion. Hosts compose_root(config) (airborne), compose_operator(config) (operator), and compose_replay(config) (replay-cli).
  • Owned by: AZ-263 (Bootstrap stub); per-component additions that wire a new strategy are owned jointly by the bootstrap epic and the consuming component task (touching runtime_root.py is allowed only via the explicit "wire-in" task in each component's epic). The compose_replay extension is owned by AZ-265 child task #4.
  • Consumed by: the airborne binary entrypoint + the operator-tooling binary entrypoint + the research/comparative binary entrypoint + the replay-cli binary entrypoint.

shared/cli/replay

  • File: src/gps_denied_onboard/cli/replay.py
  • Purpose: gps-denied-replay CLI entrypoint. Args: --video PATH --tlog PATH --output results.jsonl --camera-calibration calib.json --config config.yaml --pace {realtime,asap} [--time-offset-ms N].
  • Owned by: AZ-265 (E-DEMO-REPLAY) — child task #5.
  • Consumed by: the gps-denied-replay-cli Docker image entrypoint; parent-suite UI backend (subprocess shell-out per AZ-265 architecture decision).

shared/healthcheck

  • File: src/gps_denied_onboard/healthcheck.py
  • Purpose: Importable healthcheck callable used by Dockerfile HEALTHCHECK CMD and CI smoke.
  • Owned by: AZ-263.
  • Consumed by: companion-tier1 Dockerfile, operator-tooling Dockerfile, CI smoke job.

Allowed Dependencies (Layering)

Read top-to-bottom; an upper layer may import from a lower layer but NEVER the reverse. Cross-layer violations are Architecture findings in code-review (High severity).

Layer Components / Modules May import from
5. Entry / Composition runtime_root, cli/replay, healthcheck 1, 2, 3, 4
4. Adapters c8_fc_adapter (incl. tlog_replay_adapter + replay_sink), c11_tile_manager, c10_provisioning, c12_operator_tooling, frame_source/VideoFileFrameSource + frame_source/LiveCameraFrameSource 1, 2, 3 (limited — see notes)
3. Domain (runtime path) c1_vio, c2_vpr, c2_5_rerank, c3_matcher, c3_5_adhop, c4_pose, c5_state, c13_fdr 1, 2
2. Infrastructure c6_tile_cache, c7_inference 1
1. Foundation (shared) _types, config, logging, fdr_client, helpers/*, frame_source (interface only), clock (none)

Layer-specific notes:

  • Layer 3 → Layer 4 is BANNED. Domain components must not import adapter-layer components. C1's reception of FC telemetry happens via a constructor-injected FcAdapter interface (the interface lives in c8_fc_adapter Public API) — C1 imports the interface from a Layer-4 component's Public API, which is technically a downward-pointing import on the dependency graph, but the runtime data flow is Layer 4 → Layer 3 (FC → C1). This is the standard "interface lives at the producer" Hexagonal pattern; flagged here so the cross-verification step (Step 4) doesn't false-positive it.
  • C3 → C2.5 is BANNED at runtime (R14): both must import helpers.lightglue_runtime instead. Enforced by the absence of any from gps_denied_onboard.components.c2_5_rerank import ... line inside c3_matcher/.
  • runtime_root.py may import any component's concrete impl; everywhere else, cross-component imports go through the consumed component's Public API only.

Build-Time Exclusion Map (ADR-002)

Four binaries are built from this codebase: airborne (Tier-1 + Tier-2 production), research (IT-12 comparative-study, links every strategy), operator-tooling (pre-flight workflows on operator workstation), replay-cli (offline gps-denied-replay against video + tlog; AZ-265).

CMake flag Components / native libs gated Airborne Research Operator-tooling Replay-cli
BUILD_OKVIS2 c1_vio/okvis2, cpp/okvis2 ON ON OFF ON
BUILD_VINS_MONO c1_vio/vins_mono, cpp/vins_mono OFF ON OFF OFF
BUILD_KLT_RANSAC c1_vio/klt_ransac, cpp/klt_ransac ON (mandatory baseline) ON OFF ON
BUILD_VPR_<variant> (UltraVPR, MegaLoc, MixVPR, SelaVPR, EigenPlaces, NetVLAD, SALAD) c2_vpr/ UltraVPR ON, others OFF all ON OFF UltraVPR ON, others OFF
BUILD_TENSORRT_RUNTIME c7_inference/tensorrt_runtime ON ON ON (operator pre-compiles engines) ON
BUILD_PYTORCH_RUNTIME c7_inference/pytorch_fp16_runtime OFF ON OFF OFF
BUILD_C10_PROVISIONING c10_provisioning OFF OFF ON OFF
BUILD_C11_TILE_MANAGER c11_tile_manager OFF OFF ON OFF
BUILD_C12_OPERATOR_TOOLING c12_operator_tooling OFF OFF ON OFF
BUILD_GTSAM_BINDINGS cpp/gtsam_bindings (used by c4_pose + c5_state) ON ON OFF ON
BUILD_FAISS_INDEX cpp/faiss_index (used by c6_tile_cache) ON ON ON OFF (replay reads pre-built cache only)
BUILD_VIDEO_FILE_FRAME_SOURCE frame_source/VideoFileFrameSource (AZ-265) OFF OFF OFF ON
BUILD_TLOG_REPLAY_ADAPTER c8_fc_adapter/tlog_replay_adapter (AZ-265) OFF OFF OFF ON
BUILD_REPLAY_SINK_JSONL c8_fc_adapter/replay_sink (AZ-265) OFF OFF OFF ON
BUILD_REPLAY_CLI cli/replay.py entrypoint + compose_replay wiring (AZ-265) OFF OFF OFF ON
BUILD_LIVE_CAMERA_FRAME_SOURCE frame_source/LiveCameraFrameSource (AZ-265 retrofit) ON ON OFF OFF

The composition root validator at startup refuses to wire a strategy whose BUILD_* flag is OFF (raises ConfigurationError pointing at the offending strategy name + the missing flag).

Build-time exclusion is enforced by:

  • CMake reading cmake/build_options.cmake per binary target.
  • Per-binary CI matrix entry in .github/workflows/ci.yml (4 parallel build jobs).
  • ci/sbom_diff.py step asserting each binary's SBOM contains exactly the expected component set (e.g., the airborne SBOM MUST NOT contain c11_tile_manager; the replay-cli SBOM MUST contain C1–C5 + replay strategies and MUST NOT contain c10_provisioning).

Layout Conventions (reference)

Language Root Per-component path Public API file Test path
Python (this project) src/gps_denied_onboard/ src/gps_denied_onboard/components/<component>/ src/gps_denied_onboard/components/<component>/__init__.py (re-exports) + interface.py tests/unit/<component>/
Python (generic) src/<pkg>/ src/<pkg>/<component>/ src/<pkg>/<component>/__init__.py tests/<component>/
C# (.NET) src/ src/<Component>/ src/<Component>/<Component>.cs tests/<Component>.Tests/
Rust crates/ crates/<component>/ crates/<component>/src/lib.rs crates/<component>/tests/
TypeScript / React packages/ or src/ src/<component>/ src/<component>/index.ts src/<component>/__tests__/
Go ./ internal/<component>/ or pkg/<component>/ internal/<component>/doc.go internal/<component>/*_test.go

Self-Verification Checklist

  • Every component in _docs/02_document/components/ has a Per-Component Mapping entry (14 components: c1_vio, c2_vpr, c2_5_rerank, c3_matcher, c3_5_adhop, c4_pose, c5_state, c6_tile_cache, c7_inference, c8_fc_adapter, c10_provisioning, c11_tile_manager, c12_operator_tooling, c13_fdr).
  • Every shared / cross-cutting concern has a Shared section entry (_types, config, logging, fdr_client, frame_source, clock, helpers/* × 8, runtime_root, cli/replay, healthcheck).
  • Layering table covers every component; foundation at Layer 1.
  • No component's Imports from list points at a component in a higher layer (back-channel exception for C8 → C1/C5 documented as interface-at-producer pattern).
  • Paths follow Python src/-layout convention with single top-level package gps_denied_onboard/.
  • No two components own overlapping paths. Joint native ownership of cpp/gtsam_bindings/ resolved: c5_state is primary owner; c4_pose READ-ONLY.
  • Replay-mode additions (AZ-265) covered: new frame_source/ and clock/ cross-cuttings, new C8 strategies (tlog_replay_adapter, replay_sink), new cli/replay.py entrypoint, and a fourth replay-cli binary added to the Build-Time Exclusion Map.

How the implement skill consumes this

The /implement skill's Step 4 (File Ownership) reads this file and, for each task in the batch:

  1. Resolve the task's Component field to a Per-Component Mapping entry.
  2. Set OWNED = the component's Owns glob.
  3. Set READ-ONLY = the Public API files of every component listed in Imports from, plus all shared/* Public API files.
  4. Set FORBIDDEN = every other component's Owns glob.

Execution inside a batch is already sequential. This mapping is still required because it enforces scope discipline per task — preventing a task from drifting into files that belong to another component.

Reference in New Issue View Git Blame Copy Permalink