diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 4a4a8de..df5785f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -13,12 +13,12 @@ jobs: - name: Build JetPack image run: echo "JetPack image build + sign + attest — concrete wiring lands per deploy task" - operator-tooling-tarball: + operator-orchestrator-tarball: runs-on: ubuntu-22.04 needs: jetpack-image steps: - uses: actions/checkout@v4 - - name: Bundle operator-tooling tarball + - name: Bundle operator-orchestrator tarball run: | mkdir -p dist - tar -czf dist/operator-tooling.tar.gz docker-compose.yml docker/ _docs/ + tar -czf dist/operator-orchestrator.tar.gz docker-compose.yml docker/ _docs/ diff --git a/README.md b/README.md index ac698a5..ba3433f 100644 --- a/README.md +++ b/README.md @@ -23,4 +23,4 @@ For full Tier-1 integration via Docker, see [`_docs/02_document/deployment/conta ## Build matrix -Four binaries built from this codebase: **airborne**, **research**, **operator-tooling**, **replay-cli**. CMake `BUILD_*` flags gate component inclusion per binary — see [`cmake/build_options.cmake`](cmake/build_options.cmake) and [`_docs/02_document/module-layout.md` § Build-Time Exclusion Map](_docs/02_document/module-layout.md#build-time-exclusion-map-adr-002). +Four binaries built from this codebase: **airborne**, **research**, **operator-orchestrator**, **replay-cli**. CMake `BUILD_*` flags gate component inclusion per binary — see [`cmake/build_options.cmake`](cmake/build_options.cmake) and [`_docs/02_document/module-layout.md` § Build-Time Exclusion Map](_docs/02_document/module-layout.md#build-time-exclusion-map-adr-002). diff --git a/_docs/02_document/FINAL_report.md b/_docs/02_document/FINAL_report.md index 029dd4f..6df929a 100644 --- a/_docs/02_document/FINAL_report.md +++ b/_docs/02_document/FINAL_report.md @@ -37,7 +37,7 @@ See `architecture.md` for the full ADR set (ADR-001..ADR-009), 12 architectural | 10 | C8 FC + GCS Adapter | `pymavlink` `GPS_INPUT` for ArduPilot (signed) + `MSP2_SENSOR_GPS` for iNav (unsigned, accepted residual risk); honest 6×6 → 2×2 covariance projection; GCS 1–2 Hz downsampled telemetry | C5, E-CC-CONF, E-CC-LOG | AZ-261 | | 11 | C10 Pre-flight Cache Provisioning | Builds model-derived cache (descriptors, engines, manifest, content hashes); F2 takeoff verifier; does NOT touch `satellite-provider` (network I/O lives in C11) | C6, C7, E-CC-LOG | AZ-252 | | 12 | C11 Tile Manager | Operator-side `TileDownloader` (pre-flight) + `TileUploader` (post-landing, gated `flight_state == ON_GROUND`); excluded from airborne image | C6, E-CC-CONF, E-CC-LOG | AZ-251 | -| 13 | C12 Operator Pre-flight Tooling | CLI subcommands (`download`, `build-cache`, `upload-pending`, `reloc-confirm`); sector classification UI hook; FDR retrieval helpers | C10, C11, E-CC-LOG | AZ-253 | +| 13 | C12 Operator Pre-flight Orchestrator | CLI subcommands (`download`, `build-cache`, `upload-pending`, `reloc-confirm`); sector classification UI hook; FDR retrieval helpers | C10, C11, E-CC-LOG | AZ-253 | | 14 | C13 Flight Data Recorder | Per-flight ≤64 GB NVM ring (estimates + IMU + emitted MAVLink + health + mid-flight tiles + ≤0.1 Hz failed-tile thumbnails); raw nav/AI-cam frames excluded | E-BOOT, E-CC-LOG, E-CC-CONF, E-CC-FDR-CLIENT | AZ-248 | **Cross-cutting epics** (not components, but shared concerns): E-BOOT (AZ-244), E-CC-LOG (AZ-245), E-CC-CONF (AZ-246), E-CC-FDR-CLIENT (AZ-247). @@ -103,7 +103,7 @@ The test suite is organised as scenario specs (no source code yet). Per-componen | C8 | `components/10_c8_fc_adapter/tests.md` | | C10 | `components/11_c10_provisioning/tests.md` | | C11 | `components/12_c11_tilemanager/tests.md` | -| C12 | `components/13_c12_operator_tooling/tests.md` | +| C12 | `components/13_c12_operator_orchestrator/tests.md` | | C13 | `components/14_c13_fdr/tests.md` | ### System-level scenario suites (`_docs/02_document/tests/`) @@ -142,7 +142,7 @@ Both the inclusive reading (PARTIAL = covered) and the strict reading clear the | 7 | AZ-250: E-C6 — Tile Cache + Spatial Index | C6 | M | 13–21 | E-BOOT, E-CC-LOG, E-CC-CONF | | 8 | AZ-251: E-C11 — Tile Manager | C11 | M | 13–21 | E-C6, E-CC-CONF, E-CC-LOG | | 9 | AZ-252: E-C10 — Pre-flight Cache Provisioning | C10 | M | 13–21 | E-C6, E-C7, E-CC-LOG | -| 10 | AZ-253: E-C12 — Operator Pre-flight Tooling | C12 | M | 13–21 | E-C10, E-C11, E-CC-LOG | +| 10 | AZ-253: E-C12 — Operator Pre-flight Orchestrator | C12 | M | 13–21 | E-C10, E-C11, E-CC-LOG | | 11 | AZ-254: E-C1 — Visual / Visual-Inertial Odometry | C1 | XL | 34–55 | E-BOOT, E-CC-FDR-CLIENT, E-C7 | | 12 | AZ-255: E-C2 — Visual Place Recognition | C2 | L | 21–34 | E-C6, E-C7, E-CC-FDR-CLIENT | | 13 | AZ-256: E-C2.5 — Inlier-based Re-rank | C2.5 | S | 5–8 | E-C2, E-C7, E-C6 (shared LightGlue helper) | diff --git a/_docs/02_document/architecture.md b/_docs/02_document/architecture.md index 1b9b926..b62b5e9 100644 --- a/_docs/02_document/architecture.md +++ b/_docs/02_document/architecture.md @@ -139,9 +139,9 @@ The system is a **Jetson Orin Nano Super-hosted onboard companion** that deliver **Infrastructure**: -- **No cloud orchestration**. The companion is an embedded edge device; the operator's workstation is a single host that runs the operator tooling (C11 Tile Manager + C12 Operator Pre-flight Tooling) and a local `satellite-provider` mirror or VPN-reaches the lab `satellite-provider`. +- **No cloud orchestration**. The companion is an embedded edge device; the operator's workstation is a single host that runs the operator tooling (C11 Tile Manager + C12 Operator Pre-flight Orchestrator) and a local `satellite-provider` mirror or VPN-reaches the lab `satellite-provider`. - **Two binaries shipped on every PR** (ADR-002): `deployment-binary` (links the production-default strategy on each component + the mandatory simple-baseline; CMake `BUILD_VINS_MONO=OFF`, `BUILD_SALAD=OFF`, …) and `research-binary` (links every available strategy on every component; all `BUILD_*` flags `ON`, used for the IT-12 comparative study). The deployment binary is what installs onto an operational Jetson; the research binary runs on dev/lab Jetson hardware for the comparative-study report. The same code base produces both — ADR-002 mechanism scales to additional binary variants later if packaging strategy requires it. -- **Container scope**: Tier-1 uses Docker (`docker compose` for the developer setup including a `mock-suite-sat-service` container, the operator-tool container, and a Postgres for C6). **Tier-2 (Jetson) does NOT use Docker** — TensorRT INT8 calibration caches and `jetson-stats` thermal telemetry are most reliable without a container layer, per D-C7-9 + D-C10-6. The deployed image on the Jetson is a JetPack-based system image with the deployment binary preinstalled. +- **Container scope**: Tier-1 uses Docker (`docker compose` for the developer setup including a `mock-suite-sat-service` container, the operator-orchestrator container, and a Postgres for C6). **Tier-2 (Jetson) does NOT use Docker** — TensorRT INT8 calibration caches and `jetson-stats` thermal telemetry are most reliable without a container layer, per D-C7-9 + D-C10-6. The deployed image on the Jetson is a JetPack-based system image with the deployment binary preinstalled. - **Scaling**: not applicable (per-UAV, single companion). Failover is per-airframe (the FC's IMU-only fallback at AC-5.2 is the system's "scale-out"). **Environment-specific configuration**: @@ -170,7 +170,7 @@ source repo │ ├─ deployment-binary tarball (production-default strategies + mandatory baselines, ADR-002) │ ├─ research-binary tarball (all strategies linked; for IT-12 comparative study) │ ├─ JetPack image (deployment-binary preinstalled) - │ └─ operator-tooling tarball (C11 + C12 + e2e-test mock-suite-sat-service compose for offline integration testing) + │ └─ operator-orchestrator tarball (C11 + C12 + e2e-test mock-suite-sat-service compose for offline integration testing) │ └─→ deploy paths: ├─ Jetson operational deploy: JetPack image flash (deployment-binary) @@ -321,7 +321,7 @@ The onboard side of D-PROJ-2 is fully specified in `_docs/_process_leftovers/202 | Companion ↔ GCS (AP profile) | MAVLink 2.0 signing inherited from the FC channel | | Operator workstation ↔ `satellite-provider` (pre-flight) | TLS + service-internal API key (workstation only; never on the airborne companion) | | Companion ↔ `satellite-provider` (post-landing upload, **D-PROJ-2 planned**) | Per-flight onboard signing key carried with each uploaded tile; the planned ingest endpoint verifies the key | -| Operator workstation pre-flight stage | OS-level (operator login + workstation hardening — operator-tooling concern, C12) | +| Operator workstation pre-flight stage | OS-level (operator login + workstation hardening — operator-orchestrator concern, C12) | **Authorization**: @@ -424,7 +424,7 @@ This decision is made on **technical grounds only**. Component licenses (BSD/Apa 1. Single binary with software-only guard — rejected on principle: a runtime guard cannot be the primary control for an "is the system airborne?" safety property. 2. Hardware-level switch (e.g., physical write-enable jumper) — rejected: adds operations cost; software-image-isolation gives equivalent assurance for this threat model. -**Consequences**: Two binaries to maintain (companion image + operator-tooling image). CI builds and tests both. The operator workflow has an explicit post-landing step ("run the upload tool") which is itself a feature, not a bug. +**Consequences**: Two binaries to maintain (companion image + operator-orchestrator image). CI builds and tests both. The operator workflow has an explicit post-landing step ("run the upload tool") which is itself a feature, not a bug. ### ADR-005 — Two execution tiers (Tier-1 / Tier-2) are first-class architectural concerns (F6) @@ -462,7 +462,7 @@ This decision is made on **technical grounds only**. Component licenses (BSD/Apa 1. Keep ADR-007 as originally written — rejected: see "Why reversed". 2. Wait for D-PROJ-2 service-side implementation before any tests — rejected: blocks the onboard cycle. -**Consequences**: The mock continues to ship in the operator-tooling tarball's compose file as a test-time service, but it is no longer documented under `_docs/02_document/components/`. Test specs and CI references treat it as a fixture. When `satellite-provider` ships the real endpoint, the fixture is replaced by pointing tests at the real service; no architectural changes flow from that switch. +**Consequences**: The mock continues to ship in the operator-orchestrator tarball's compose file as a test-time service, but it is no longer documented under `_docs/02_document/components/`. Test specs and CI references treat it as a fixture. When `satellite-provider` ships the real endpoint, the fixture is replaced by pointing tests at the real service; no architectural changes flow from that switch. ### ADR-008 — D-C8-2 source-set switch is `Selected with runtime gate` (Mode B Fact #111) diff --git a/_docs/02_document/components/11_c10_provisioning/description.md b/_docs/02_document/components/11_c10_provisioning/description.md index 40a0923..6bec13f 100644 --- a/_docs/02_document/components/11_c10_provisioning/description.md +++ b/_docs/02_document/components/11_c10_provisioning/description.md @@ -145,7 +145,7 @@ would break AC-6. **Potential race conditions**: -- Concurrent `build_cache_artifacts` invocations on the same cache root would corrupt state. Single-process operator-tool wraps with a filesystem lockfile (the same lockfile C11 honours); if a second invocation tries to start, fail with explicit error. +- Concurrent `build_cache_artifacts` invocations on the same cache root would corrupt state. Single-process operator-orchestrator wraps with a filesystem lockfile (the same lockfile C11 honours); if a second invocation tries to start, fail with explicit error. **Performance bottlenecks**: diff --git a/_docs/02_document/components/12_c11_tilemanager/description.md b/_docs/02_document/components/12_c11_tilemanager/description.md index b4434ee..355a067 100644 --- a/_docs/02_document/components/12_c11_tilemanager/description.md +++ b/_docs/02_document/components/12_c11_tilemanager/description.md @@ -5,7 +5,7 @@ **Purpose**: own the operator-side network I/O against `satellite-provider` for the onboard tile corpus, in **both directions**: - **Download** (pre-flight, F1): fetch tiles from `satellite-provider` for the operational area, apply AC-NEW-6 freshness gating, and write into C6 (`TileStore` + `TileMetadataStore`). C11 is the **only** path that crosses the workstation/companion enclave to the parent suite for tile pixels — C10 reads from the populated C6 store and never touches `satellite-provider` itself. -- **Upload** (post-landing, F10): when `flight_state == ON_GROUND` is confirmed, read pending mid-flight tiles from C6 and POST to `satellite-provider`'s ingest endpoint (D-PROJ-2 contract sketch). +- **Upload** (post-landing, F10): read pending mid-flight tiles from C6 and POST to `satellite-provider`'s ingest endpoint (D-PROJ-2 contract sketch). C11 itself does NOT gate on flight state — it is a dumb pipe; the post-landing safety gate is owned by C12's `PostLandingUploadOrchestrator` (AZ-329 / Batch 44), which checks the C13 `flight_footer` FDR record for `clean_shutdown=True` before invoking `TileUploader.upload_pending_tiles`. C11 is a **separate operator-side binary / image**. The airborne companion image's CMake target deliberately excludes the entire `c11_tilemanager/` source tree so the airborne process cannot accidentally execute either the download path or the upload path even via reflection or config error (ADR-004 process-level isolation, AC-8.4). Both directions of tile I/O are operator-driven on the operator workstation; the companion only consumes the populated C6 store while airborne. @@ -36,10 +36,11 @@ C11 is a **separate operator-side binary / image**. The airborne companion image | Method | Input | Output | Async | Error Types | |--------|-------|--------|-------|-------------| -| `confirm_flight_state` | `()` | `FlightStateSignal` (must be ON_GROUND) | No | `FlightStateNotOnGroundError` | | `enumerate_pending_tiles` | `flight_id: uuid (optional)` | `list[TileMetadata]` | No | `TileMetadataError` | | `upload_pending_tiles` | `UploadRequest` | `UploadBatchReport` | No | `SatelliteProviderError`, `RateLimitedError`, `SignatureRejectedError` | +C11 no longer exposes `confirm_flight_state` — the post-landing flight-state gate moved to C12 (`PostLandingUploadOrchestrator`, AZ-329) per Batch 44. `FlightStateNotOnGroundError` is retired from C11; the corresponding refusal now lives at the C12 boundary as `FlightStateNotConfirmedError`. + **Input/Output DTOs**: ``` @@ -65,8 +66,6 @@ UploadRequest: batch_size: int satellite_provider_url: URL -FlightStateSignal: see C8 — must be ON_GROUND for any upload to proceed - UploadBatchReport: batch_uuid: uuid (assigned by satellite-provider per D-PROJ-2 contract) per_tile_status: list[(tile_id, status: enum {queued, rejected, duplicate, superseded})] @@ -154,9 +153,10 @@ C11 reads from / writes to C6 (the local store) and reads from / writes to `sate - `RateLimitedError` (429): obey `Retry-After`; the operator can also re-invoke later. Same handling either direction. - `FreshnessRejectionError` / `ResolutionRejectionError`: download-side only. Per AC-NEW-6 / RESTRICT-SAT-4 — never silently downgrade fresh-required tiles in `active_conflict` sectors. Surface counts in the `DownloadBatchReport`. - `CacheBudgetExceededError`: download-side only. Pre-flight free-space check against AC-8.3 (≤ 10 GB). Fail fast with explicit budget delta; no partial write. -- `FlightStateNotOnGroundError`: upload-side only. Refuse to start; log + show explicit reason. ADR-004 process-level isolation means C11 should never run when the FC believes it's airborne — this error is a defense-in-depth, not the primary control. - `SignatureRejectedError`: upload-side only. Per-flight signing key was rejected by `satellite-provider`. This is a security-critical event — do NOT silently drop; surface to operator + log to FDR. +Post-landing safety: C11's upload path no longer gates on flight state internally. The check now lives in C12's `PostLandingUploadOrchestrator` (AZ-329 / Batch 44), which refuses to invoke `TileUploader.upload_pending_tiles` unless the C13 `flight_footer` FDR record records `clean_shutdown=True` for the target flight. ADR-004 process-level isolation remains the primary control — C11 should never run on the companion at all. + ## 6. Extensions and Helpers | Helper | Purpose | Used By | @@ -192,7 +192,7 @@ C11 reads from / writes to C6 (the local store) and reads from / writes to `sate | Log Level | When | Example | |-----------|------|---------| -| ERROR | `FlightStateNotOnGroundError`, `SignatureRejectedError`, persistent `SatelliteProviderError`, `CacheBudgetExceededError` | `C11 refused to start: flight_state=IN_AIR; safeguard active` | +| ERROR | `SignatureRejectedError`, persistent `SatelliteProviderError`, `CacheBudgetExceededError` | `C11 upload failure: signature rejected by satellite-provider` | | WARN | one-off network failure, scheduled retry, freshness-driven rejections (counts) | `C11 batch upload retry: batch_uuid=…; next_retry_in_s=30` | | INFO | session start/end; per-batch report (download + upload) | `C11 download complete: 87654 tiles, 12 stale-rejected; bbox=…` | | DEBUG | per-tile request/response | `C11 tile uploaded: tile_id=(z=18,lat=…,lon=…); status=queued` | diff --git a/_docs/02_document/components/12_c11_tilemanager/tests.md b/_docs/02_document/components/12_c11_tilemanager/tests.md index ff887b5..d9bb3ba 100644 --- a/_docs/02_document/components/12_c11_tilemanager/tests.md +++ b/_docs/02_document/components/12_c11_tilemanager/tests.md @@ -66,19 +66,13 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C11 wa --- -### C11-IT-04: TileUploader gates on `flight_state == ON_GROUND` +### C11-IT-04: post-landing safety gate lives in C12 (cross-reference) -**Summary**: `TileUploader.upload_pending` refuses to run if `FlightStateSignal != ON_GROUND` (defense-in-depth atop ADR-004 process isolation). +**Summary**: post-landing safety is owned by C12, not C11. The gate that historically lived in `TileUploader.upload_pending_tiles` was removed in Batch 44 (supersedes AZ-317); the equivalent check now lives in C12's `PostLandingUploadOrchestrator` (AZ-329) and refuses to invoke `TileUploader.upload_pending_tiles` unless the C13 `flight_footer` FDR record records `clean_shutdown=True` for the target flight. -**Traces to**: AC-8.4 (defensive — ADR-004's secondary guard) +**Traces to**: see `_docs/02_document/components/13_c12_operator_orchestrator/tests.md` → C12-IT-03 for the post-landing safety test. -**Description**: call `upload_pending` with `FlightStateSignal == IN_FLIGHT`; assert `UploadGateBlockedError`. Same with `UNKNOWN`. Set `ON_GROUND` and assert upload proceeds. - -**Input data**: scripted FlightStateSignal source. - -**Expected result**: upload blocked except in `ON_GROUND`. - -**Max execution time**: 30 s. +**Status**: cross-reference only. C11's `TileUploader` no longer exposes `confirm_flight_state` or raises `FlightStateNotOnGroundError`. --- @@ -193,10 +187,10 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C11 wa | Step | Action | Expected Result | |------|--------|-----------------| -| 1 | `operator-tool download --area derkachi.geojson --since 2026-01` | `DownloadBatchReport` printed; tiles in C6 | -| 2 | `operator-tool build-cache` | C10 builds engines + descriptors + Manifest | +| 1 | `operator-orchestrator download --area derkachi.geojson --since 2026-01` | `DownloadBatchReport` printed; tiles in C6 | +| 2 | `operator-orchestrator build-cache` | C10 builds engines + descriptors + Manifest | | 3 | (simulate flight) | (covered by other tests) | -| 4 | `operator-tool upload-pending` | Pending-upload tiles POSTed; report printed | +| 4 | `operator-orchestrator upload-pending` | Pending-upload tiles POSTed; report printed | --- diff --git a/_docs/02_document/components/13_c12_operator_tooling/description.md b/_docs/02_document/components/13_c12_operator_orchestrator/description.md similarity index 96% rename from _docs/02_document/components/13_c12_operator_tooling/description.md rename to _docs/02_document/components/13_c12_operator_orchestrator/description.md index b41b221..d509fe5 100644 --- a/_docs/02_document/components/13_c12_operator_tooling/description.md +++ b/_docs/02_document/components/13_c12_operator_orchestrator/description.md @@ -1,4 +1,4 @@ -# C12 — Operator Pre-flight Tooling +# C12 — Operator Pre-flight Orchestrator ## 1. High-Level Overview @@ -26,7 +26,7 @@ | Method | Input | Output | Async | Error Types | |--------|-------|--------|-------|-------------| | `build_cache` | `flight_id` (online) OR `flight_file: Path` (offline), `sector_class`, `calibration_path`, `satellite_provider_url`, `api_key` | `CacheBuildReport` (wraps `FlightResolveReport` + C11 `DownloadBatchReport` + C10 `BuildReport`) | No (operator-facing; minutes) | `CacheBuildError` (wraps `FlightNotFoundError`, `FlightsApiUnreachableError`, `SatelliteProviderError`, `EngineBuildError`, etc.) | -| `trigger_post_landing_upload` | `flight_id` | C11 `UploadBatchReport` | No (operator-facing; minutes) | `CacheBuildError` wrapper around `FlightStateNotOnGroundError`, `SignatureRejectedError`, etc. | +| `trigger_post_landing_upload` | `PostLandingUploadRequest` (`flight_id`, `satellite_provider_url`, `api_key`, `batch_size`) | C11 `UploadBatchReport` (re-exposed as `UploadBatchReportCut`) | No (operator-facing; minutes) | `FlightStateNotConfirmedError` (footer missing / unclean / fdr-unreadable / flight-id not found), `SatelliteProviderError`, `SignatureRejectedError` (passthrough from C11) | | `verify_companion_ready` | `companion_address` | `ReadinessReport` | No | `CompanionUnreachableError`, `ContentHashMismatchError` | | `set_sector_classification` | `area, sector_class` | `None` | No | — | | `apply_freshness_threshold` | `sector_class` | `int (months)` | No | — | diff --git a/_docs/02_document/components/13_c12_operator_tooling/tests.md b/_docs/02_document/components/13_c12_operator_orchestrator/tests.md similarity index 75% rename from _docs/02_document/components/13_c12_operator_tooling/tests.md rename to _docs/02_document/components/13_c12_operator_orchestrator/tests.md index 0e7325a..49bbdc1 100644 --- a/_docs/02_document/components/13_c12_operator_tooling/tests.md +++ b/_docs/02_document/components/13_c12_operator_orchestrator/tests.md @@ -1,4 +1,4 @@ -# Test Specification — C12 Operator Pre-flight Tooling +# Test Specification — C12 Operator Pre-flight Orchestrator Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 sequences the F1 (C11 download → C10 build) and F10 (C11 upload trigger) operator-side flows. @@ -47,17 +47,17 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 se --- -### C12-IT-03: trigger_post_landing_upload invokes C11 TileUploader on confirmed ON_GROUND +### C12-IT-03: trigger_post_landing_upload invokes C11 TileUploader on confirmed clean-shutdown footer -**Summary**: `trigger_post_landing_upload` reads the most recent `FlightStateSignal` from the post-flight FDR; if `ON_GROUND` is confirmed for ≥ a configurable safety threshold (default 30 s), it invokes `C11.TileUploader.upload_pending`. If `ON_GROUND` is not confirmed, it refuses and returns a clear error. +**Summary**: `trigger_post_landing_upload` reads the post-flight FDR newest-segment-first looking for a `flight_footer` record (kind registered by C13 in AZ-292; emitted exactly once per flight on `close_flight()`); if found with `payload["clean_shutdown"] == True`, it invokes `C11.TileUploader.upload_pending_tiles(UploadRequest(flight_id=..., ...))`. If the footer is absent (truncation / crash) or carries `clean_shutdown == False`, it refuses with `FlightStateNotConfirmedError`. **Traces to**: AC-8.4 -**Description**: stage two flight FDR fixtures — one ending with confirmed ON_GROUND for 60 s, one ending with `IN_FLIGHT` (incomplete log). Call `trigger_post_landing_upload`; assert (a) first case invokes upload, (b) second case refuses with `FlightStateNotConfirmedError`. +**Description**: stage two flight FDR fixtures produced by C13's `FileFdrWriter` — one with a clean-shutdown footer (the writer's standard `close_flight()` path, which always sets `clean_shutdown=True` in the current AZ-292 implementation), one truncated (writer terminated before `close_flight()` ran, so no footer record). Call `trigger_post_landing_upload`; assert (a) first case invokes upload via the `TileUploaderCut` and returns the recorded `UploadBatchReport`, (b) second case refuses with `FlightStateNotConfirmedError(not_confirmed_reason="footer_missing")`. -**Input data**: 2 scripted FDR fixtures. +**Input data**: 2 FDR fixtures generated by `FileFdrWriter` (one closed cleanly; one with the close skipped). -**Expected result**: per assertion. +**Expected result**: per assertion. No 30-second ON_GROUND threshold is consulted — the footer's existence + `clean_shutdown` flag is the sole signal. **Max execution time**: 60 s. @@ -99,7 +99,7 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 se ### C12-ST-01: CLI rejects writes to airborne images -**Summary**: the operator-tool CLI has no command path that writes into the airborne `production-binary` image (defends against operator-side mistakes that would defeat ADR-004). +**Summary**: the operator-orchestrator CLI has no command path that writes into the airborne `production-binary` image (defends against operator-side mistakes that would defeat ADR-004). **Traces to**: ADR-004 R02 enforcement (C12 side) @@ -135,7 +135,7 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 se | Data Set | Source | Size | |----------|--------|------| | Operator-tooling tarball | CI build artifact | varies | -| FDR fixtures (ON_GROUND-confirmed and IN_FLIGHT) | scripted | <100 MB each | +| FDR fixtures (clean-shutdown footer present / footer absent) | generated by C13 `FileFdrWriter` | <100 MB each | | Small Derkachi sub-area for C12-IT-02 | scripted | <500 MB | **Setup**: extract operator-tooling tarball; bring up Docker compose. diff --git a/_docs/02_document/components/14_c13_fdr/description.md b/_docs/02_document/components/14_c13_fdr/description.md index d868d28..b70c2aa 100644 --- a/_docs/02_document/components/14_c13_fdr/description.md +++ b/_docs/02_document/components/14_c13_fdr/description.md @@ -90,7 +90,7 @@ Not applicable. |---------|---------|---------| | orjson / msgpack | per project pin | Record serialisation (serialised format choice during decompose phase) | | atomicwrites | latest | Segment file rotation (atomic open of new segment + close of previous) | -| filelock | per project pin | Cross-process safety for the FDR root (operator-tool reads while companion writes — companion-only access during flight) | +| filelock | per project pin | Cross-process safety for the FDR root (operator-orchestrator reads while companion writes — companion-only access during flight) | **Error Handling Strategy**: - `FdrOpenError` at takeoff: refuse takeoff (per AC-NEW-3 every payload class must be present from t=0). diff --git a/_docs/02_document/contracts/c11_tilemanager/tile_uploader.md b/_docs/02_document/contracts/c11_tilemanager/tile_uploader.md index b83fdb6..6a3f075 100644 --- a/_docs/02_document/contracts/c11_tilemanager/tile_uploader.md +++ b/_docs/02_document/contracts/c11_tilemanager/tile_uploader.md @@ -1,17 +1,23 @@ # Contract: tile_uploader **Component**: c11_tilemanager -**Producer task**: AZ-319_c11_tile_uploader -**Consumer tasks**: AZ-253 (E-C12 Operator Pre-flight Tooling — TBD at C12 decompose time) -**Version**: 1.0.0 -**Status**: draft -**Last Updated**: 2026-05-10 +**Producer task**: AZ-319_c11_tile_uploader (initial), Batch 44 C11-SRP-revert (v2.0.0 gate removal) +**Consumer tasks**: AZ-329 (C12 `PostLandingUploadOrchestrator`) — see `_docs/02_document/contracts/c12_operator_orchestrator/` for the C12 surface that owns the post-landing safety gate. +**Version**: 2.0.0 +**Status**: frozen +**Last Updated**: 2026-05-13 + +## Migration note — v1.0.0 → v2.0.0 + +Batch 44 removed C11's internal post-landing safety gate per SRP. v1.0.0 exposed `confirm_flight_state(): FlightStateSignal` and raised `FlightStateNotOnGroundError` from `upload_pending_tiles`. v2.0.0 drops both — the equivalent check moved to C12's `PostLandingUploadOrchestrator` (AZ-329), which inspects the C13 `flight_footer` FDR record and refuses to invoke `upload_pending_tiles` unless `clean_shutdown=True` is recorded. C11 is now a dumb pipe. + +Consumers that still call `confirm_flight_state` or catch `FlightStateNotOnGroundError` MUST migrate to consuming C12's `FlightStateNotConfirmedError` family instead. ADR-004 process-level isolation remains the primary control — C11 never runs on the companion at all. ## Purpose -The `TileUploader` Protocol is C11's operator-side post-landing upload interface. C12 invokes it during F10 (post-landing) to read mid-flight tiles flagged pending-upload from C6 (`source = onboard_ingest`, `voting_status = pending`), package them per the D-PROJ-2 ingest contract sketch, sign each tile payload with the per-flight ephemeral key (AZ-318), and POST to `satellite-provider`'s `/api/satellite/tiles/ingest` endpoint. Acknowledged tiles are marked uploaded in C6. +The `TileUploader` Protocol is C11's operator-side post-landing upload interface. C12's `PostLandingUploadOrchestrator` (AZ-329) invokes it during F10 (post-landing) AFTER it has confirmed `clean_shutdown=True` from the C13 `flight_footer` FDR record. C11 then reads mid-flight tiles flagged pending-upload from C6 (`source = onboard_ingest`, `voting_status = pending`), packages them per the D-PROJ-2 ingest contract sketch, signs each tile payload with the per-flight ephemeral key (AZ-318), and POSTs to `satellite-provider`'s `/api/satellite/tiles/ingest` endpoint. Acknowledged tiles are marked uploaded in C6. -The uploader gates on `flight_state == ON_GROUND` (AZ-317) before any network egress. C11 is operator-side ONLY; ADR-004 forbids the airborne companion image from importing this module. +C11 is operator-side ONLY; ADR-004 forbids the airborne companion image from importing this module. ## Shape @@ -24,14 +30,12 @@ from typing import Protocol, runtime_checkable class TileUploader(Protocol): def upload_pending_tiles(self, request: UploadRequest) -> UploadBatchReport: ... def enumerate_pending_tiles(self, flight_id: uuid.UUID | None = None) -> list[TileMetadata]: ... - def confirm_flight_state(self) -> FlightStateSignal: ... ``` | Name | Signature | Throws / Errors | Blocking? | |------|-----------|-----------------|-----------| -| `upload_pending_tiles` | `(request: UploadRequest) -> UploadBatchReport` | `FlightStateNotOnGroundError`, `SatelliteProviderError`, `RateLimitedError`, `SignatureRejectedError`, `TileMetadataError` | sync (post-landing; minutes) | +| `upload_pending_tiles` | `(request: UploadRequest) -> UploadBatchReport` | `SatelliteProviderError`, `RateLimitedError`, `SignatureRejectedError`, `TileMetadataError` | sync (post-landing; minutes) | | `enumerate_pending_tiles` | `(flight_id: uuid.UUID \| None) -> list[TileMetadata]` | `TileMetadataError` | sync (seconds) | -| `confirm_flight_state` | `() -> FlightStateSignal` | `FlightStateNotOnGroundError` | sync (≤ 1 ms) | ### Data DTOs @@ -70,7 +74,7 @@ class PerTileStatus: ## Invariants -- I-1: `confirm_flight_state` is called by `upload_pending_tiles` BEFORE any C6 read or network egress; if `FlightStateNotOnGroundError` is raised, NO tiles are read, NO POSTs are issued, NO C6 mutation occurs. The gate is closed by default. +- I-1 (v2.0.0): C11 itself does NOT gate on flight state. The pre-call gate is C12's `PostLandingUploadOrchestrator` (AZ-329), which inspects the C13 `flight_footer` FDR record for `clean_shutdown=True` BEFORE invoking `upload_pending_tiles`. C11 is a dumb pipe — once called, it proceeds to read C6 + POST to the satellite-provider with no internal short-circuit. ADR-004 process-level isolation remains the primary defence (C11 never runs on the companion). - I-2: Every uploaded tile carries a signature produced by the AZ-318 per-flight key manager's `sign(payload)`. The parent suite verifies against the public key it received via the safety officer's pre-flight enrolment OR the `kind="c11.upload.session.key.public"` FDR record. - I-3: A tile acknowledged as `queued`, `duplicate`, or `superseded` by the parent suite is marked `uploaded` in C6 (`mark_uploaded(tile_id)`); a tile acknowledged as `rejected` is NOT marked uploaded — it remains `pending` for human review. - I-4: The per-flight signing key is zeroised at the end of `upload_pending_tiles` regardless of success or failure (try/finally in the caller; AZ-318's `end_session()`). @@ -98,8 +102,8 @@ class PerTileStatus: | Case | Input | Expected | Notes | |------|-------|----------|-------| -| upload-happy-path | 50 pending tiles, ON_GROUND, parent-suite returns 202 with all `queued` | `UploadBatchReport.outcome = success`; all 50 marked `uploaded` in C6; signature verifies on each | C11-IT-03 | -| flight-state-blocks | `FlightStateSource` returns `IN_FLIGHT` | `FlightStateNotOnGroundError`; zero C6 reads; zero POSTs | C11-IT-04 | +| upload-happy-path | 50 pending tiles, parent-suite returns 202 with all `queued` | `UploadBatchReport.outcome = success`; all 50 marked `uploaded` in C6; signature verifies on each | C11-IT-03 | +| post-landing-gate-in-c12 | C12 `PostLandingUploadOrchestrator` invocation flow | The flight-state gate lives in C12 (`FlightStateNotConfirmedError`), not C11. v2.0.0 removed the C11 internal gate. | See `c12_operator_orchestrator` contract + AZ-329 spec | | signature-rejected | Parent suite returns `rejected` for 1 tile with reason `"invalid signature"` | `PerTileStatus.status = rejected`; `outcome = partial`; FDR `c11.upload.signature_rejected` emitted; the tile NOT marked uploaded | I-5 | | duplicate-acknowledged | Parent suite returns `duplicate` for 5 tiles (already ingested in a prior batch) | All 5 marked `uploaded`; `outcome = success` | I-3 | | signing-key-zeroised | Run a successful upload, then assert the AZ-318 manager's `_private_key is None` | Always zeroised; FDR `c11.upload.session.key.zeroised` recorded | I-4 | @@ -112,3 +116,4 @@ class PerTileStatus: | Version | Date | Change | Author | |---------|------|--------|--------| | 1.0.0 | 2026-05-10 | Initial contract — produced by AZ-319 (E-C11 decomposition) | autodev | +| 2.0.0 | 2026-05-13 | Batch 44: remove C11 internal flight-state gate per SRP. `confirm_flight_state` method dropped; `FlightStateNotOnGroundError` retired; post-landing safety gate now owned by C12's `PostLandingUploadOrchestrator` (AZ-329). Breaking — consumers MUST migrate to C12's `FlightStateNotConfirmedError`. | autodev (Batch 44) | diff --git a/_docs/02_document/contracts/c12_operator_tooling/flights_api_client.md b/_docs/02_document/contracts/c12_operator_orchestrator/flights_api_client.md similarity index 99% rename from _docs/02_document/contracts/c12_operator_tooling/flights_api_client.md rename to _docs/02_document/contracts/c12_operator_orchestrator/flights_api_client.md index bf4cd14..a20bbc8 100644 --- a/_docs/02_document/contracts/c12_operator_tooling/flights_api_client.md +++ b/_docs/02_document/contracts/c12_operator_orchestrator/flights_api_client.md @@ -1,6 +1,6 @@ # Contract: flights_api_client -**Component**: c12_operator_tooling +**Component**: c12_operator_orchestrator **Producer task**: AZ-489 — `_docs/02_tasks/todo/AZ-489_c12_flights_api_client.md` **Consumer tasks**: AZ-326 (CLI app — wires `--flight-id` / `--flight-file` flags), AZ-328 (build-cache orchestrator — calls `fetch_flight` / `load_flight_file`, then `bbox_from_waypoints` + `takeoff_origin_from_flight`) **Version**: 1.0.0 diff --git a/_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md b/_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md similarity index 98% rename from _docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md rename to _docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md index 4d62bb4..e85ac50 100644 --- a/_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md +++ b/_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md @@ -1,6 +1,6 @@ # Contract: operator_command_transport -**Component**: c12_operator_tooling +**Component**: c12_operator_orchestrator **Producer task**: AZ-330 — `_docs/02_tasks/todo/AZ-330_c12_operator_reloc_service.md` **Consumer tasks**: TBD — a future E-C8 (AZ-261) task implements `MavlinkOperatorCommandTransport` against pymavlink **Version**: 1.0.0 @@ -9,7 +9,7 @@ ## Purpose -Defines the operator-workstation ↔ companion command channel for AC-3.4 operator-relocalization. C12 owns the Protocol shape; E-C8 (AZ-261) ships the pymavlink-backed concrete implementation that encodes the hint into a MAVLink message and transmits it over the GCS link to the airborne companion. Decoupling the two sides through this Protocol prevents C12 from having to know MAVLink details, and prevents E-C8 from having to know operator-tool internals — they meet at this contract. +Defines the operator-workstation ↔ companion command channel for AC-3.4 operator-relocalization. C12 owns the Protocol shape; E-C8 (AZ-261) ships the pymavlink-backed concrete implementation that encodes the hint into a MAVLink message and transmits it over the GCS link to the airborne companion. Decoupling the two sides through this Protocol prevents C12 from having to know MAVLink details, and prevents E-C8 from having to know operator-orchestrator internals — they meet at this contract. ## Shape diff --git a/_docs/02_document/data_model.md b/_docs/02_document/data_model.md index 6e86a25..ef7ef8e 100644 --- a/_docs/02_document/data_model.md +++ b/_docs/02_document/data_model.md @@ -73,7 +73,7 @@ The tile is the single most important persistent entity. The schema deliberately - B-tree on `(zoom_level, tile_x, tile_y)` — primary spatial lookup path for VPR retrieval and pre-flight cache hydration. - B-tree on `(latitude, longitude)` — bounding-box queries for sector classification and spatial-coverage reports. -- B-tree on `voting_status` partial WHERE `source = 'onboard_ingest'` — operator-tooling queries for "which mid-flight tiles are still pending promotion?". +- B-tree on `voting_status` partial WHERE `source = 'onboard_ingest'` — operator-orchestrator queries for "which mid-flight tiles are still pending promotion?". - B-tree on `flight_id` — FDR cross-reference; post-landing upload batching. - B-tree on `created_at` — pruning / rollover queries. @@ -141,7 +141,7 @@ A lightweight tracking row per flight, used by the FDR's manifest, the Tile Mana ### 2.3 `sector_classifications` (PostgreSQL — operator-set, onboard-side cache) -Mirrors operator-tooling C12's authoritative sector classification onto the companion so the freshness gate (AC-8.2 / AC-NEW-6) can be evaluated locally without a network call. +Mirrors operator-orchestrator C12's authoritative sector classification onto the companion so the freshness gate (AC-8.2 / AC-NEW-6) can be evaluated locally without a network call. | Column | Type | Constraints | Description | |---|---|---|---| @@ -318,13 +318,13 @@ record_crc32 u32 **Backward compatibility**: new record types are appended; readers MUST skip records they don't recognise (the `record_header` length is enough to advance the cursor). No record type is ever renumbered or removed; deprecation is by ceasing to emit. -**Retention**: per-flight ring; on `IN_AIR → ON_GROUND` transition, the ring is sealed and the operator-tooling FDR-retrieval workflow (C12) copies it off the companion. The companion auto-prunes flights older than the configured retention window (default: 30 days) — the prune log itself is its own FDR record on the next flight. +**Retention**: per-flight ring; on `IN_AIR → ON_GROUND` transition, the ring is sealed and the operator-orchestrator FDR-retrieval workflow (C12) copies it off the companion. The companion auto-prunes flights older than the configured retention window (default: 30 days) — the prune log itself is its own FDR record on the next flight. --- ### 2.9 Tile JPEG bodies (filesystem) -JPEG bodies live at `./tiles/{zoomLevel}/{x}/{y}.jpg`. A sidecar `./tiles/{zoomLevel}/{x}/{y}.json` carries the full row content for upload-time payload assembly. Both files are atomic-written (via `atomicwrites`); both are removed only after the corresponding `tiles` row's lifecycle says it is safe (see § 2.1.2). Filesystem and PostgreSQL drift is treated as a defect: the operator-tooling C12 has a periodic `consistency_audit` that reports any orphan files / missing files. +JPEG bodies live at `./tiles/{zoomLevel}/{x}/{y}.jpg`. A sidecar `./tiles/{zoomLevel}/{x}/{y}.json` carries the full row content for upload-time payload assembly. Both files are atomic-written (via `atomicwrites`); both are removed only after the corresponding `tiles` row's lifecycle says it is safe (see § 2.1.2). Filesystem and PostgreSQL drift is treated as a defect: the operator-orchestrator C12 has a periodic `consistency_audit` that reports any orphan files / missing files. --- @@ -533,7 +533,7 @@ Schema-version bumps are tracked in `_docs/02_document/schemas/` (a new `tiles_q ### 6.5 FDR file-format compatibility -The FDR `record_header` is fixed at version 1. Every FDR reader (operator-tooling, replay tools) MUST: +The FDR `record_header` is fixed at version 1. Every FDR reader (operator-orchestrator, replay tools) MUST: - Validate `magic == 0x47464452` and skip a corrupt segment. - Read the `version` field; on `version != 1`, refuse to interpret the body and emit a "unknown FDR version" diagnostic. @@ -568,4 +568,4 @@ The following DTOs flow through the per-frame pipeline in memory and are **NOT** - **D-PROJ-2 #1 ingest-endpoint contract**: the `signature` column's exact algorithm (Ed25519 vs ECDSA) and the per-flight key distribution is a parent-suite design decision; onboard side is contract-flexible and treats `signature` as opaque `bytea`. - **D-PROJ-2 #2 voting-layer schema**: parent-suite-side; this onboard data model writes `voting_status='pending'` and reads `'trusted'` only — the actual promotion table lives in `satellite-provider`'s schema and is out of scope here. - **GeoJSON polygon precision** (`sector_classifications.polygon_geojson`): GeoJSON is precision-bounded by JSON number representation; if AC-NEW-7 cache-poisoning safety needs sub-metre polygon edges, a future migration can switch to PostGIS `geography(Polygon, 4326)`. Captured as carryforward (currently no AC requirement to do so). -- **FDR retention policy default**: 30 days post-landing is a reasonable default but is not pinned in any AC; carryforward to the operator-tooling spec (C12) for confirmation. +- **FDR retention policy default**: 30 days post-landing is a reasonable default but is not pinned in any AC; carryforward to the operator-orchestrator spec (C12) for confirmation. diff --git a/_docs/02_document/deployment/ci_cd_pipeline.md b/_docs/02_document/deployment/ci_cd_pipeline.md index 8128ca6..e98203e 100644 --- a/_docs/02_document/deployment/ci_cd_pipeline.md +++ b/_docs/02_document/deployment/ci_cd_pipeline.md @@ -19,7 +19,7 @@ The pipeline has **two execution tiers** (architecture.md ADR-005), reflected in | Build (Tier-2 deployment binary) | PR merge to `dev`, `stage`, `main` | Tier-2 (self-hosted Jetson) | Native build on Jetson green; deployment binary SBOM matches Tier-1 deployment SBOM | | AC-bound NFTs (Tier-2) | PR merge to `dev`, `stage`, `main`; manual on PR | Tier-2 | NFT-PERF-* (AC-4.1, AC-NEW-1, AC-NEW-2), NFT-LIM-* (AC-4.2, AC-NEW-3), NFT-RES-* (AC-NEW-4, AC-NEW-7), IT-12 (comparative study) all pass thresholds in `tests/traceability-matrix.md` | | JetPack image build | Tag on `main` | Tier-2 | JetPack 6.2 image built with deployment binary preinstalled, signed, and attested | -| Operator tooling tarball | Tag on `main` | Tier-1 | Tarball contains C11 Tile Manager (both `TileDownloader` and `TileUploader`) + C12 Operator Pre-flight Tooling + mock-sat-service compose + verification script | +| Operator tooling tarball | Tag on `main` | Tier-1 | Tarball contains C11 Tile Manager (both `TileDownloader` and `TileUploader`) + C12 Operator Pre-flight Orchestrator + mock-sat-service compose + verification script | Tier-2 jobs are the **only** AC-bound jobs. Everything else runs on Tier-1. @@ -146,7 +146,7 @@ Runs on tag push to `main`. Produces `gps-denied-jetpack--.img` (th ### Operator tooling tarball (release-only) -Bundles `operator-tooling` Docker image + `mock-suite-sat-service` Docker image + their compose file + a verification script + the documentation under `_docs/02_document/`. The tarball is uploaded to the release bucket alongside the JetPack image. +Bundles `operator-orchestrator` Docker image + `mock-suite-sat-service` Docker image + their compose file + a verification script + the documentation under `_docs/02_document/`. The tarball is uploaded to the release bucket alongside the JetPack image. ## Caching Strategy diff --git a/_docs/02_document/deployment/containerization.md b/_docs/02_document/deployment/containerization.md index d2ab4e8..17d22ff 100644 --- a/_docs/02_document/deployment/containerization.md +++ b/_docs/02_document/deployment/containerization.md @@ -9,7 +9,7 @@ This project has **asymmetric containerization** by design (architecture.md § 3 - **Tier-1** (workstation): Docker is the universal runtime. Dev, lint, unit, most integration, and `mock-suite-sat-service` all run in Docker compose. - **Tier-2 (Jetson)**: **NO Docker**. The deployed JetPack image runs the deployment binary natively. TensorRT INT8 calibration caches and `jetson-stats` thermal telemetry are most reliable without a container layer (D-C7-9 + D-C10-6). The "image" is a JetPack 6.2 system image with the deployment binary preinstalled. -- **Operator workstation**: Docker is used for the local `satellite-provider` mirror, the `mock-suite-sat-service` (when offline), and the operator-tooling stack (C11 Tile Manager + C12 Operator Pre-flight Tooling). +- **Operator workstation**: Docker is used for the local `satellite-provider` mirror, the `mock-suite-sat-service` (when offline), and the operator-orchestrator stack (C11 Tile Manager + C12 Operator Pre-flight Orchestrator). Three Dockerfiles are maintained; the airborne companion uses **none of them** in production. @@ -43,9 +43,9 @@ e2e-test fixture only — implements the planned D-PROJ-2 ingest contract (`POST | Health check | HTTP `GET /healthz` (returns 200 if listening + storage backend mounted). 10 s interval. | | Exposed ports | `5100/tcp` (matches `satellite-provider`'s port so the same client config works) | | Key build args | `MOCK_FAILURE_PROFILE` (default `none`; used by NFT-SEC-01 to inject latency / 5xx / partial responses) | -| Notes | The mock is a release artifact (operator-tooling tarball includes its compose file). When the real `satellite-provider` D-PROJ-2 endpoint ships, the mock is retired. | +| Notes | The mock is a release artifact (operator-orchestrator tarball includes its compose file). When the real `satellite-provider` D-PROJ-2 endpoint ships, the mock is retired. | -### `operator-tooling` (Operator workstation Tile Manager + pre-flight UI, C11 + C12) +### `operator-orchestrator` (Operator workstation Tile Manager + pre-flight UI, C11 + C12) | Property | Value | |----------|-------| @@ -53,7 +53,7 @@ e2e-test fixture only — implements the planned D-PROJ-2 ingest contract (`POST | Build image | `python:3.10-slim` (no native deps; pure Python plus `httpx` for both download and upload, `psycopg` for read/write of C6 mirror, `cryptography` for upload signing) | | Stages | `python-deps` → `runtime` | | User | `operator` (non-root) | -| Health check | `python -m operator_tooling.healthcheck` (validates `satellite-provider` reachable). 30 s interval. | +| Health check | `python -m operator_orchestrator.healthcheck` (validates `satellite-provider` reachable). 30 s interval. | | Exposed ports | `8080/tcp` (operator pre-flight UI, C12); no inbound network for C11 Tile Manager (it's a CLI / one-shot tool, both directions) | | Key build args | `INCLUDE_PRE_FLIGHT_UI=true` (default; can be turned off for headless CLI-only deployments) | | Notes | **C11 Tile Manager (both `TileDownloader` and `TileUploader`) is in this image, NEVER in `gps-denied-companion-tier1`** (ADR-004 process-level isolation). The airborne deployment binary on Tier-2 also does not contain C11. | @@ -120,11 +120,11 @@ services: interval: 5s networks: [ gps-denied-net ] - operator-tooling: + operator-orchestrator: build: context: . - dockerfile: docker/operator-tooling.Dockerfile - image: gps-denied/operator-tooling:dev + dockerfile: docker/operator-orchestrator.Dockerfile + image: gps-denied/operator-orchestrator:dev environment: - SATELLITE_PROVIDER_URL=http://mock-sat:5100 - COMPANION_DB_URL=postgresql://gps_denied:dev@db:5432/gps_denied @@ -207,7 +207,7 @@ Tier-2 CI runs the same deployment binary directly on the self-hosted Jetson run | CI build (deployment binary) | `/gps-denied/companion-tier1:deployment-` | `ghcr.io/azaion/gps-denied/companion-tier1:deployment-a1b2c3d` | | CI build (research binary) | `/gps-denied/companion-tier1:research-` | `ghcr.io/azaion/gps-denied/companion-tier1:research-a1b2c3d` | | Mock sat service | `/gps-denied/mock-suite-sat-service:` | `ghcr.io/azaion/gps-denied/mock-suite-sat-service:a1b2c3d` | -| Operator tooling | `/gps-denied/operator-tooling:` | `ghcr.io/azaion/gps-denied/operator-tooling:a1b2c3d` | +| Operator tooling | `/gps-denied/operator-orchestrator:` | `ghcr.io/azaion/gps-denied/operator-orchestrator:a1b2c3d` | | Release | `/gps-denied/:` | `ghcr.io/azaion/gps-denied/companion-tier1:deployment-1.2.0` | | Local dev | `gps-denied/:dev` | `gps-denied/companion-tier1:dev` | | JetPack image (Tier-2) | `gps-denied-jetpack--.img` | `gps-denied-jetpack-1.2.0-a1b2c3d.img` (file artifact, not a container tag) | diff --git a/_docs/02_document/deployment/deployment_procedures.md b/_docs/02_document/deployment/deployment_procedures.md index 53fb01c..10ea8ad 100644 --- a/_docs/02_document/deployment/deployment_procedures.md +++ b/_docs/02_document/deployment/deployment_procedures.md @@ -5,12 +5,12 @@ ## Deployment scope and model -This project does **not** ship a service; it ships an **embedded edge image** plus an **operator-tooling bundle**. The "deployment" patterns from the standard template (blue-green / rolling / canary) are not applicable. Deployment for this project means: +This project does **not** ship a service; it ships an **embedded edge image** plus an **operator-orchestrator bundle**. The "deployment" patterns from the standard template (blue-green / rolling / canary) are not applicable. Deployment for this project means: | Artifact | Target | Deployment mechanism | |---|---|---| | **JetPack image** (`gps-denied-jetpack--.img`) | Production Jetson Orin Nano Super on a UAV | Operator flashes the image onto the Jetson via NVIDIA `sdkmanager` or `Etcher`-style `dd` from the operator workstation | -| **Operator tooling tarball** | Operator workstation | Operator extracts; `docker compose up -d` brings up `mock-suite-sat-service` (when offline) + `operator-tooling` | +| **Operator tooling tarball** | Operator workstation | Operator extracts; `docker compose up -d` brings up `mock-suite-sat-service` (when offline) + `operator-orchestrator` | | **Tier-1 dev compose** | Developer workstation | Developer runs `docker compose up` from repo root | **Zero-downtime is not a goal**: a UAV is not in service while it is being re-flashed. The deployment cadence is per-airframe maintenance, not per-request availability. @@ -25,9 +25,9 @@ Performed once per release on Tier-1 + Tier-2 CI; produces signed artifacts stor 2. **Tier-1 produces**: - `companion-tier1:deployment-` and `companion-tier1:research-` Docker images (pushed to registry). - `mock-suite-sat-service:` Docker image. - - `operator-tooling:` Docker image. + - `operator-orchestrator:` Docker image. - SBOM artifacts for both binaries (deployment and research). - - `operator-tooling--.tar.gz` containing the operator-tooling image + mock-sat image + their compose file + verification script + relevant docs. + - `operator-orchestrator--.tar.gz` containing the operator-orchestrator image + mock-sat image + their compose file + verification script + relevant docs. 3. **Tier-2 produces**: - Native deployment-binary build on the self-hosted Jetson runner. - SBOM verification: byte-equal (after canonicalization) to Tier-1's deployment-binary SBOM. Mismatch fails the release. @@ -35,7 +35,7 @@ Performed once per release on Tier-1 + Tier-2 CI; produces signed artifacts stor 4. **Signing** (Tier-1): - Both Docker image manifests are signed with the project's release key. - The JetPack image is signed; checksum is published as a separate signed file (`gps-denied-jetpack--.img.sha256.sig`). - - The operator-tooling tarball is signed. + - The operator-orchestrator tarball is signed. 5. **Release bucket**: artifacts uploaded; release notes published; the previous release's artifacts retained for at least 90 days for rollback support. A release fails if any step above fails — including any AC-bound NFT failure on Tier-2 (`ci_cd_pipeline.md` § AC-bound NFTs). @@ -85,19 +85,19 @@ cosign verify-blob \ sha256sum -c gps-denied-jetpack--.img.sha256 -# Verify the operator-tooling tarball. +# Verify the operator-orchestrator tarball. cosign verify-blob \ - --signature operator-tooling--.tar.gz.sig \ + --signature operator-orchestrator--.tar.gz.sig \ --key gps-denied-release-key.pub \ - operator-tooling--.tar.gz + operator-orchestrator--.tar.gz ``` -### 3. Pre-flight cache build (operator-tooling C12) +### 3. Pre-flight cache build (operator-orchestrator C12) Performed on the operator workstation, with `satellite-provider` reachable (locally mirrored or via lab VPN). ```sh -docker compose -f operator-tooling-compose.yml up -d +docker compose -f operator-orchestrator-compose.yml up -d # Operator opens http://127.0.0.1:8080 ``` @@ -164,7 +164,7 @@ The first flight on a freshly-deployed airframe is a **commissioning flight**, n Post first commissioning flight: -- [ ] FDR retrieved and visualized on operator workstation (operator-tooling C12 dashboard, observability.md § 5.1). +- [ ] FDR retrieved and visualized on operator workstation (operator-orchestrator C12 dashboard, observability.md § 5.1). - [ ] AC-NEW-4 statistics for the commissioning flight reviewed; outliers investigated. - [ ] No FDR segment drops; no `ContentHashGateFail` events. - [ ] Mid-flight tile generation working (post-landing upload — handle that separately). @@ -172,12 +172,12 @@ Post first commissioning flight: ## Post-landing tile upload (per-flight, ADR-004) -Per AC-8.4 + ADR-004, mid-flight tile upload to `satellite-provider` is **post-landing only**, and uses the operator-tooling's C11 Tile Manager (`TileUploader` interface; a separate binary, never linked into the airborne image). +Per AC-8.4 + ADR-004, mid-flight tile upload to `satellite-provider` is **post-landing only**, and uses the operator-orchestrator's C11 Tile Manager (`TileUploader` interface; a separate binary, never linked into the airborne image). ```sh # Operator plugs the companion's NVM into the workstation OR ssh's into the powered-off-then-re-booted Jetson. -docker compose run operator-tooling \ - python -m operator_tooling.tilemanager upload \ +docker compose run operator-orchestrator \ + python -m operator_orchestrator.tilemanager upload \ --flight-id \ --satellite-provider $SATELLITE_PROVIDER_URL \ --signing-pubkey-fingerprint @@ -210,7 +210,7 @@ When the parent-suite voting layer (D-PROJ-2 design task #2) ships, this flow do ### Rollback steps (per-airframe) 1. **Re-flash** the previous release's JetPack image onto the affected Jetson (same procedure as § 4 with the previous artifact). -2. **Re-stage** the previous release's pre-flight bundle (the operator workstation retains it in the operator-tooling cache for ≥ 30 days). +2. **Re-stage** the previous release's pre-flight bundle (the operator workstation retains it in the operator-orchestrator cache for ≥ 30 days). 3. **Re-run** the pre-takeoff readiness gate. 4. **Confirm** AC-5.2 fallback is still functional (it is FC firmware behavior; rolling back the companion image cannot break it, but verify on the GCS). 5. **Document** the rollback in the post-mortem template; include FDR snapshots from the offending flight (if any) plus the rollback artifacts versions. diff --git a/_docs/02_document/deployment/environment_strategy.md b/_docs/02_document/deployment/environment_strategy.md index ed62b56..c9c0563 100644 --- a/_docs/02_document/deployment/environment_strategy.md +++ b/_docs/02_document/deployment/environment_strategy.md @@ -141,7 +141,7 @@ This means the threat surface on a captured companion reduces to "what is in the |---|---|---| | Per-flight MAVLink signing key | Every flight (per-flight ephemeral) | Automated at takeoff load | | Per-flight onboard tile-signing key | Every flight (per-flight ephemeral) | Automated at takeoff load | -| `SATELLITE_PROVIDER_API_KEY` | Operator-managed; rotated when an operator workstation is reissued or compromised is suspected | Operator workstation hardening procedure (out of scope of this document; operator-tooling C12 owns it) | +| `SATELLITE_PROVIDER_API_KEY` | Operator-managed; rotated when an operator workstation is reissued or compromised is suspected | Operator workstation hardening procedure (out of scope of this document; operator-orchestrator C12 owns it) | | Production binary signing key | Per release cycle or on suspected compromise | Release engineer rotates; new key fingerprint is published in release notes; verification scripts on the operator workstation pull the latest fingerprint | | JetPack image signing key | Same as production binary signing key | Same | diff --git a/_docs/02_document/deployment/observability.md b/_docs/02_document/deployment/observability.md index 94db9a7..b13f5d5 100644 --- a/_docs/02_document/deployment/observability.md +++ b/_docs/02_document/deployment/observability.md @@ -12,7 +12,7 @@ Observability therefore splits into three regimes: | Regime | Where | Live or post-flight | Primary mechanism | |---|---|---|---| | **In-flight onboard** | Production Jetson, in flight | Live (to FDR ring) + best-effort live (to GCS) | FDR binary record stream + GCS STATUSTEXT / NAMED_VALUE_FLOAT | -| **Post-flight onboard** | Operator workstation after pulling the FDR | Post-flight | FDR replay + visualization in operator-tooling C12 | +| **Post-flight onboard** | Operator workstation after pulling the FDR | Post-flight | FDR replay + visualization in operator-orchestrator C12 | | **CI / dev (Tier-1, Tier-2)** | Workstation Docker / Jetson CI runner | Live | Standard structured logging + Prometheus metrics endpoint where applicable | The sections below are organized by regime. @@ -85,7 +85,7 @@ There is no Prometheus endpoint on the production airborne companion. The justif When the operator plugs the companion in post-landing: -1. **FDR retrieval** (operator tooling C12 — feature, not in scope of this document's structure but observability-impacting): operator-tooling reads the FDR ring, copies it to the workstation, and seals the in-flight ring. The companion's per-flight ephemeral keys are deleted at this step (environment_strategy.md § Per-flight key lifecycle). +1. **FDR retrieval** (operator tooling C12 — feature, not in scope of this document's structure but observability-impacting): operator-orchestrator reads the FDR ring, copies it to the workstation, and seals the in-flight ring. The companion's per-flight ephemeral keys are deleted at this step (environment_strategy.md § Per-flight key lifecycle). 2. **Visualization** (operator tooling C12): the workstation renders: - Time-series of `horiz_accuracy`, `vert_accuracy`, `last_anchor_age_ms`, source label timeline, thermal-throttle hybrid switches, and CPU / GPU / temp. - Map view: emitted positions vs. (when available) FC `GLOBAL_POSITION_INT` ground truth. @@ -173,7 +173,7 @@ Collection interval: 15 s (typical Prometheus default; Tier-2 NFT runs may use 1 The runtime is a single in-process Python program with no cross-service hops in flight (architecture.md § 5 internal communication is all in-process). Distributed tracing is therefore not applicable to the production runtime. -The Tier-1 integration setup DOES involve cross-container hops (companion ↔ mock-sat ↔ db ↔ e2e-runner), but those are exercised by the e2e test framework's own log + status capture; OpenTelemetry is not provisioned for this project. If a future cycle introduces a multi-process companion (which ADR-004 explicitly rejected for the airborne profile but might appear on the operator workstation for C11 Tile Manager + C12 Operator Pre-flight Tooling), tracing can be reconsidered then. +The Tier-1 integration setup DOES involve cross-container hops (companion ↔ mock-sat ↔ db ↔ e2e-runner), but those are exercised by the e2e test framework's own log + status capture; OpenTelemetry is not provisioned for this project. If a future cycle introduces a multi-process companion (which ADR-004 explicitly rejected for the airborne profile but might appear on the operator workstation for C11 Tile Manager + C12 Operator Pre-flight Orchestrator), tracing can be reconsidered then. ## 4. Alerting (post-flight, not in-flight) @@ -201,7 +201,7 @@ There is no PagerDuty / on-call rotation for this project; in-flight failures ar ### 5.1 Operator workstation post-flight dashboard -Built into operator-tooling C12. Per flight: +Built into operator-orchestrator C12. Per flight: - Time series: source label, `horiz_accuracy`, `last_anchor_age_ms`, CPU%, GPU%, temp. - Event markers: VISUAL_BLACKOUT entries, spoofing events, signing key rotations, thermal hybrid switches. @@ -227,6 +227,6 @@ Out of scope by design. The GCS is the only live operator surface; all other ins ## 6. Open Items / Plan-Phase Carryforward -- **Long-term FDR archive** (multi-flight statistical headroom): D-PROJ-3 (multi-flight fixture acquisition for AC-NEW-4 / AC-NEW-7) is not pursued this cycle. If pursued in a future cycle, post-flight FDR archives become a corpus contribution path; the operator-tooling FDR-retrieval step would need an explicit "contribute to corpus" toggle. +- **Long-term FDR archive** (multi-flight statistical headroom): D-PROJ-3 (multi-flight fixture acquisition for AC-NEW-4 / AC-NEW-7) is not pursued this cycle. If pursued in a future cycle, post-flight FDR archives become a corpus contribution path; the operator-orchestrator FDR-retrieval step would need an explicit "contribute to corpus" toggle. - **Telemetry-link encryption** beyond MAVLink-2.0 signing: out of scope; addressed by physical link assumptions in the threat model (architecture.md § 7). - **iNav signing**: still has no equivalent to MAVLink-2.0 signing (Mode B Source #129). Carryforward Plan-phase action: file a feature request upstream; meanwhile observability for iNav-profile flights is the same as AP-profile minus the `MavlinkSigningKeyRotated` records (which are NULL on iNav flights per data_model.md § 2.2). diff --git a/_docs/02_document/epics.md b/_docs/02_document/epics.md index fb5784e..3286526 100644 --- a/_docs/02_document/epics.md +++ b/_docs/02_document/epics.md @@ -27,7 +27,7 @@ Row 20 (E-CC-HELPERS / AZ-264) was added during Decompose Step 2 to comply with | 7 | E-C6 | C6 Tile Cache + Spatial Index | component | AZ-250 | M | 13–21 | E-BOOT, E-CC-LOG, E-CC-CONF | | 8 | E-C11 | C11 Tile Manager (TileDownloader + TileUploader) | component | AZ-251 | M | 13–21 | E-C6, E-CC-CONF, E-CC-LOG | | 9 | E-C10 | C10 Pre-flight Cache Provisioning | component | AZ-252 | M | 13–21 | E-C6, E-C7, E-CC-LOG | -| 10 | E-C12 | C12 Operator Pre-flight Tooling | component | AZ-253 | M | 13–21 | E-C10, E-C11, E-CC-LOG | +| 10 | E-C12 | C12 Operator Pre-flight Orchestrator | component | AZ-253 | M | 13–21 | E-C10, E-C11, E-CC-LOG | | 11 | E-C1 | C1 Visual / Visual-Inertial Odometry | component | AZ-254 | XL | 34–55 | E-BOOT, E-CC-FDR-CLIENT, E-C7 | | 12 | E-C2 | C2 Visual Place Recognition | component | AZ-255 | L | 21–34 | E-C6, E-C7, E-CC-FDR-CLIENT | | 13 | E-C2.5 | C2.5 Inlier-based Re-rank | component | AZ-256 | S | 5–8 | E-C2, E-C7, E-C6 (LightGlue helper shared with C3) | @@ -127,7 +127,7 @@ flowchart LR ### Problem / Context -No source layout exists yet. Every downstream epic assumes a defined repo skeleton: `src/components/_/`, `src/shared//`, `tests/`, `tests/fixtures/`, plus the Tier-1 Docker compose, the Tier-2 CI job, the Postgres init scripts that match `data_model.md`, and the operator-tooling tarball build path. Until this exists, no other epic can start. +No source layout exists yet. Every downstream epic assumes a defined repo skeleton: `src/components/_/`, `src/shared//`, `tests/`, `tests/fixtures/`, plus the Tier-1 Docker compose, the Tier-2 CI job, the Postgres init scripts that match `data_model.md`, and the operator-orchestrator tarball build path. Until this exists, no other epic can start. ### Scope @@ -1047,7 +1047,7 @@ Per `components/11_c10_provisioning/tests.md`. --- -## E-C12 — C12 Operator Pre-flight Tooling +## E-C12 — C12 Operator Pre-flight Orchestrator **Tracker**: AZ-253 | **Type**: component | **T-shirt**: M | **Story points**: 13–21 @@ -1055,7 +1055,7 @@ Per `components/11_c10_provisioning/tests.md`. ```mermaid flowchart LR - CLI[operator-tool CLI] + CLI[operator-orchestrator CLI] CLI --> C11D[C11 TileDownloader] CLI --> C10[C10 CacheProvisioner] CLI --> C11U[C11 TileUploader] @@ -1065,7 +1065,7 @@ flowchart LR ### Problem / Context -Operator-facing CLI that sequences pre-flight (C11 download → C10 build) and post-landing (C11 upload), surfaces actionable failures, and handles the AC-3.4 re-localization workflow. Delivered as part of the operator-tooling tarball. +Operator-facing CLI that sequences pre-flight (C11 download → C10 build) and post-landing (C11 upload), surfaces actionable failures, and handles the AC-3.4 re-localization workflow. Delivered as part of the operator-orchestrator tarball. ### Scope @@ -1075,7 +1075,7 @@ Operator-facing CLI that sequences pre-flight (C11 download → C10 build) and p ### Architecture notes -- File: `components/13_c12_operator_tooling/description.md`. +- File: `components/13_c12_operator_orchestrator/description.md`. - Strict process boundary: C12 is operator-side only, in the same image as C11, but never airborne. ### Interface specification @@ -1144,7 +1144,7 @@ T-shirt M; 13–21 points. ### Testing strategy -Per `components/13_c12_operator_tooling/tests.md`. +Per `components/13_c12_operator_orchestrator/tests.md`. --- @@ -1616,7 +1616,7 @@ sequenceDiagram ### Risks & mitigations -- **R10** (latency under throttle) — threshold tunable via operator-tooling pre-flight. +- **R10** (latency under throttle) — threshold tunable via operator-orchestrator pre-flight. ### Effort @@ -2124,7 +2124,7 @@ ROS as the input transport was considered and rejected: the system is MAVLink-na ### Architecture notes - ADR-001 / ADR-002 / ADR-009 all apply unchanged. -- New `BUILD_*` flags: `BUILD_VIDEO_FILE_FRAME_SOURCE`, `BUILD_TLOG_REPLAY_ADAPTER`, `BUILD_REPLAY_SINK_JSONL`. Default ON for the new replay-cli binary; OFF for airborne, research, and operator-tooling. +- New `BUILD_*` flags: `BUILD_VIDEO_FILE_FRAME_SOURCE`, `BUILD_TLOG_REPLAY_ADAPTER`, `BUILD_REPLAY_SINK_JSONL`. Default ON for the new replay-cli binary; OFF for airborne, research, and operator-orchestrator. - New cross-cutting `FrameSource` interface lives at `src/gps_denied_onboard/frame_source/` (Layer 1 Foundation per `module-layout.md` § layering). - `compose_replay` lives in `runtime_root.py` alongside `compose_root` and `compose_operator`. @@ -2209,7 +2209,7 @@ T-shirt M; 27–32 points across 8 child tasks. - ADR-001 / ADR-002 / ADR-009. - C1–C5 components MUST remain mode-agnostic; replay-aware logic lives only in the composition root, the new strategies, and the CLI. -- No HTTP server in any companion binary (airborne or replay); HTTP wrapper, if added later, lives in operator-tooling per `module-layout.md` Layer-4 placement. +- No HTTP server in any companion binary (airborne or replay); HTTP wrapper, if added later, lives in operator-orchestrator per `module-layout.md` Layer-4 placement. ### Testing strategy diff --git a/_docs/02_document/glossary.md b/_docs/02_document/glossary.md index c591f79..6694f69 100644 --- a/_docs/02_document/glossary.md +++ b/_docs/02_document/glossary.md @@ -76,7 +76,7 @@ Terms are alphabetical. Each entry: one-line definition + parenthetical source. **Satellite anchored** — Source label `satellite_anchored`: estimate produced by matching the current nav frame against pre-cached satellite tiles. Highest confidence among the three labels. (source: AC-1.4) -**Sector classification** — Pre-flight operator decision: active-conflict (6-month tile-freshness threshold) vs stable rear (12-month threshold). Drives the freshness gate at ingest and during runtime tile use. (source: AC-8.2, AC-NEW-6, `solution.md` operator-tooling section) +**Sector classification** — Pre-flight operator decision: active-conflict (6-month tile-freshness threshold) vs stable rear (12-month threshold). Drives the freshness gate at ingest and during runtime tile use. (source: AC-8.2, AC-NEW-6, `solution.md` operator-orchestrator section) **Source label** — Provenance tag carried with every emitted estimate: `{satellite_anchored | visual_propagated | dead_reckoned}`. (source: AC-1.4) diff --git a/_docs/02_document/module-layout.md b/_docs/02_document/module-layout.md index f8b3f14..c3998f8 100644 --- a/_docs/02_document/module-layout.md +++ b/_docs/02_document/module-layout.md @@ -221,7 +221,7 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec - Composition root: `runtime_root/c10_factory.py` (`build_engine_compiler`, `build_backbone_specs`, `build_manifest_builder`, `build_manifest_verifier`, `build_descriptor_batcher` + the C6→C10 adapters `c6_tile_metadata_store_to_tiles_batch_query`, `c6_tile_store_to_pixel_opener`, `c6_descriptor_index_to_rebuilder`) - **Owns**: `src/gps_denied_onboard/components/c10_provisioning/**`, `tests/unit/c10_provisioning/**` - **Imports from**: `_types` (cross-component DTOs `EngineCacheEntry`, `BuildConfig`, `PrecisionMode`, `OptimizationProfile`, `HostCapabilities`, `TileMetadata`, etc.), `_types.inference_errors` (AZ-507 typed-error envelope for `EngineBuildError` + `CalibrationCacheError`), `helpers.sha256_sidecar`, `helpers.engine_filename_schema`, `helpers.wgs_converter`, `config`, `logging`, `fdr_client`. The `InferenceRuntime.compile_engine` surface (c7) and the `TileMetadataStore.query_by_bbox` surface (c6) are obtained via constructor-injected consumer-side structural Protocol cuts (the `CompileEngineCallable` cut already lives in `engine_compiler.py`; AZ-323 / AZ-324 will define analogous `query_by_bbox` cuts inside `c10_provisioning/`). NEVER `from gps_denied_onboard.components.c6_tile_cache import ...` or `from gps_denied_onboard.components.c7_inference import ...` inside `c10_provisioning/*.py`. -- **Consumed by**: `c12_operator_tooling`, `runtime_root` (operator binary only — excluded from airborne via `BUILD_C10_PROVISIONING=OFF` for airborne build per ADR-002) +- **Consumed by**: `c12_operator_orchestrator`, `runtime_root` (operator binary only — excluded from airborne via `BUILD_C10_PROVISIONING=OFF` for airborne build per ADR-002) ### Component: c11_tile_manager @@ -235,12 +235,12 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec - `satellite_provider_uploader.py` (post-landing batch upload, D-PROJ-2 ingest contract) - **Owns**: `src/gps_denied_onboard/components/c11_tile_manager/**`, `tests/unit/c11_tile_manager/**` - **Imports from**: `_types`, `helpers.sha256_sidecar`, `helpers.wgs_converter`, `config`, `logging`, `fdr_client`. The c6 storage surface (`TileStore`, `TileMetadataStore`) is obtained via constructor-injected consumer-side structural Protocol cuts (see AZ-507 cross-component rule below); composition root wires the concrete c6 strategy in. NEVER `from gps_denied_onboard.components.c6_tile_cache import ...` inside `c11_tile_manager/*.py`. -- **Consumed by**: `c12_operator_tooling`, `runtime_root` (operator binary only — `BUILD_C11_TILE_MANAGER=OFF` for airborne) +- **Consumed by**: `c12_operator_orchestrator`, `runtime_root` (operator binary only — `BUILD_C11_TILE_MANAGER=OFF` for airborne) -### Component: c12_operator_tooling +### Component: c12_operator_orchestrator -- **Epic**: AZ-253 (E-C12 Operator Pre-flight Tooling) -- **Directory**: `src/gps_denied_onboard/components/c12_operator_tooling/` +- **Epic**: AZ-253 (E-C12 Operator Pre-flight Orchestrator) +- **Directory**: `src/gps_denied_onboard/components/c12_operator_orchestrator/` - **Public API**: - `__init__.py` (re-exports `CacheBuildWorkflow`, `OperatorReLocService`) - `interface.py` @@ -248,9 +248,9 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec - `cache_build_workflow.py` (CLI orchestrator) - `operator_reloc_service.py` (CLI; GUI deferred per epic) - `sector_classifier.py` (operator sets `SectorClassification` → C6) -- **Owns**: `src/gps_denied_onboard/components/c12_operator_tooling/**`, `tests/unit/c12_operator_tooling/**` -- **Imports from**: `_types`, `helpers.wgs_converter`, `config`, `logging`, `fdr_client`. The c6 / c10 / c11 surfaces (`TileStore`, `TileMetadataStore`, `CacheProvisioner`, `TileDownloader`, `TileUploader`) are obtained via constructor-injected consumer-side structural Protocol cuts (see AZ-507 cross-component rule below); composition root wires the concrete c6/c10/c11 strategies in. NEVER `from gps_denied_onboard.components.c6_tile_cache import ...`, `from gps_denied_onboard.components.c10_provisioning import ...`, or `from gps_denied_onboard.components.c11_tile_manager import ...` inside `c12_operator_tooling/*.py`. -- **Consumed by**: `runtime_root` (operator binary only — `BUILD_C12_OPERATOR_TOOLING=OFF` for airborne) +- **Owns**: `src/gps_denied_onboard/components/c12_operator_orchestrator/**`, `tests/unit/c12_operator_orchestrator/**` +- **Imports from**: `_types`, `helpers.wgs_converter`, `config`, `logging`, `fdr_client`. The c6 / c10 / c11 surfaces (`TileStore`, `TileMetadataStore`, `CacheProvisioner`, `TileDownloader`, `TileUploader`) are obtained via constructor-injected consumer-side structural Protocol cuts (see AZ-507 cross-component rule below); composition root wires the concrete c6/c10/c11 strategies in. NEVER `from gps_denied_onboard.components.c6_tile_cache import ...`, `from gps_denied_onboard.components.c10_provisioning import ...`, or `from gps_denied_onboard.components.c11_tile_manager import ...` inside `c12_operator_orchestrator/*.py`. +- **Consumed by**: `runtime_root` (operator binary only — `BUILD_C12_OPERATOR_ORCHESTRATOR=OFF` for airborne) ### Component: c13_fdr @@ -330,7 +330,7 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec - **Directory**: `src/gps_denied_onboard/helpers/wgs_converter.py` - **Purpose**: WGS84 ↔ local-tangent-plane conversion utilities (`04_helper_wgs_converter.md`). - **Owned by**: AZ-264. -- **Consumed by**: c4_pose, c5_state, c6_tile_cache, c8_fc_adapter, c10_provisioning, c11_tile_manager, c12_operator_tooling. +- **Consumed by**: c4_pose, c5_state, c6_tile_cache, c8_fc_adapter, c10_provisioning, c11_tile_manager, c12_operator_orchestrator. ### shared/helpers/sha256_sidecar @@ -379,7 +379,7 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec - **File**: `src/gps_denied_onboard/runtime_root.py` - **Purpose**: Composition root — config → strategy resolution → graph wiring (ADR-009). The ONLY place that may import concrete strategy classes across components. Per-binary CMake `BUILD_*` flags + composition root validator enforce ADR-002 build-time exclusion. Hosts `compose_root(config)` (airborne), `compose_operator(config)` (operator), and `compose_replay(config)` (replay-cli). - **Owned by**: AZ-263 (Bootstrap stub); per-component additions that wire a new strategy are owned jointly by the bootstrap epic and the consuming component task (touching `runtime_root.py` is allowed only via the explicit "wire-in" task in each component's epic). The `compose_replay` extension is owned by AZ-265 child task #4. -- **Consumed by**: the airborne binary entrypoint + the operator-tooling binary entrypoint + the research/comparative binary entrypoint + the replay-cli binary entrypoint. +- **Consumed by**: the airborne binary entrypoint + the operator-orchestrator binary entrypoint + the research/comparative binary entrypoint + the replay-cli binary entrypoint. ### shared/cli/replay @@ -393,7 +393,7 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec - **File**: `src/gps_denied_onboard/healthcheck.py` - **Purpose**: Importable healthcheck callable used by Dockerfile `HEALTHCHECK CMD` and CI smoke. - **Owned by**: AZ-263. -- **Consumed by**: companion-tier1 Dockerfile, operator-tooling Dockerfile, CI smoke job. +- **Consumed by**: companion-tier1 Dockerfile, operator-orchestrator Dockerfile, CI smoke job. ## Allowed Dependencies (Layering) @@ -402,7 +402,7 @@ Read top-to-bottom; an upper layer may import from a lower layer but NEVER the r | Layer | Components / Modules | May import from | |-------|---------------------|-----------------| | 5. Entry / Composition | `runtime_root`, `cli/replay`, `healthcheck` | 1, 2, 3, 4 | -| 4. Adapters | c8_fc_adapter (incl. `tlog_replay_adapter` + `replay_sink`), c11_tile_manager, c10_provisioning, c12_operator_tooling, `frame_source/VideoFileFrameSource` + `frame_source/LiveCameraFrameSource` | 1, 2, 3 (limited — see notes) | +| 4. Adapters | c8_fc_adapter (incl. `tlog_replay_adapter` + `replay_sink`), c11_tile_manager, c10_provisioning, c12_operator_orchestrator, `frame_source/VideoFileFrameSource` + `frame_source/LiveCameraFrameSource` | 1, 2, 3 (limited — see notes) | | 3. Domain (runtime path) | c1_vio, c2_vpr, c2_5_rerank, c3_matcher, c3_5_adhop, c4_pose, c5_state, c13_fdr | 1, 2 | | 2. Infrastructure | c6_tile_cache, c7_inference | 1 | | 1. Foundation (shared) | `_types`, `config`, `logging`, `fdr_client`, `helpers/*`, `frame_source` (interface only), `clock` | (none) | @@ -415,7 +415,7 @@ Read top-to-bottom; an upper layer may import from a lower layer but NEVER the r ## Build-Time Exclusion Map (ADR-002) -Four binaries are built from this codebase: **airborne** (Tier-1 + Tier-2 production), **research** (IT-12 comparative-study, links every strategy), **operator-tooling** (pre-flight workflows on operator workstation), **replay-cli** (offline `gps-denied-replay` against video + tlog; AZ-265). +Four binaries are built from this codebase: **airborne** (Tier-1 + Tier-2 production), **research** (IT-12 comparative-study, links every strategy), **operator-orchestrator** (pre-flight workflows on operator workstation), **replay-cli** (offline `gps-denied-replay` against video + tlog; AZ-265). | CMake flag | Components / native libs gated | Airborne | Research | Operator-tooling | Replay-cli | |-----------|-------------------------------|----------|----------|------------------|------------| @@ -427,7 +427,7 @@ Four binaries are built from this codebase: **airborne** (Tier-1 + Tier-2 produc | `BUILD_PYTORCH_RUNTIME` | c7_inference/pytorch_fp16_runtime | OFF | ON | OFF | OFF | | `BUILD_C10_PROVISIONING` | c10_provisioning | OFF | OFF | ON | OFF | | `BUILD_C11_TILE_MANAGER` | c11_tile_manager | OFF | OFF | ON | OFF | -| `BUILD_C12_OPERATOR_TOOLING` | c12_operator_tooling | OFF | OFF | ON | OFF | +| `BUILD_C12_OPERATOR_ORCHESTRATOR` | c12_operator_orchestrator | OFF | OFF | ON | OFF | | `BUILD_GTSAM_BINDINGS` | cpp/gtsam_bindings (used by c4_pose + c5_state) | ON | ON | OFF | ON | | `BUILD_FAISS_INDEX` | c6_tile_cache `FaissDescriptorIndex` (faiss-cpu wheel; runtime gate at `runtime_root.storage_factory` — no native target) | ON | ON | ON | OFF (replay reads pre-built cache only) | | `BUILD_VIDEO_FILE_FRAME_SOURCE` | `frame_source/VideoFileFrameSource` (AZ-265) | OFF | OFF | OFF | ON | @@ -456,7 +456,7 @@ Build-time exclusion is enforced by: ## Self-Verification Checklist -- [x] Every component in `_docs/02_document/components/` has a Per-Component Mapping entry (14 components: c1_vio, c2_vpr, c2_5_rerank, c3_matcher, c3_5_adhop, c4_pose, c5_state, c6_tile_cache, c7_inference, c8_fc_adapter, c10_provisioning, c11_tile_manager, c12_operator_tooling, c13_fdr). +- [x] Every component in `_docs/02_document/components/` has a Per-Component Mapping entry (14 components: c1_vio, c2_vpr, c2_5_rerank, c3_matcher, c3_5_adhop, c4_pose, c5_state, c6_tile_cache, c7_inference, c8_fc_adapter, c10_provisioning, c11_tile_manager, c12_operator_orchestrator, c13_fdr). - [x] Every shared / cross-cutting concern has a Shared section entry (_types, config, logging, fdr_client, frame_source, clock, helpers/* × 8, runtime_root, cli/replay, healthcheck). - [x] Layering table covers every component; foundation at Layer 1. - [x] No component's `Imports from` list points at a component in a higher layer (back-channel exception for C8 → C1/C5 documented as interface-at-producer pattern). diff --git a/_docs/02_document/system-flows.md b/_docs/02_document/system-flows.md index 476fa3f..d7f8726 100644 --- a/_docs/02_document/system-flows.md +++ b/_docs/02_document/system-flows.md @@ -153,7 +153,7 @@ flowchart TD | 3 | `satellite-provider` | C11 | Paged tile blobs + metadata rows | JPEG + JSON metadata | | 4 | C11 | C6 filesystem (over USB/Eth) | Tile JPEG bodies | `./tiles/{zoomLevel}/{x}/{y}.jpg` | | 5 | C11 | C6 PostgreSQL | Tile metadata rows (`source='googlemaps'`) | SQL INSERT (mirror of `satellite-provider`'s `tiles` table) | -| 6 | C12 | C10 `CacheProvisioner` | `BuildRequest(bbox, zoom_levels, sector_class, calibration_path, takeoff_origin, flight_id)` | in-process call (operator-tool side); RPC over USB/Eth to companion runner | +| 6 | C12 | C10 `CacheProvisioner` | `BuildRequest(bbox, zoom_levels, sector_class, calibration_path, takeoff_origin, flight_id)` | in-process call (operator-orchestrator side); RPC over USB/Eth to companion runner | | 7 | C10 → C7 | TRT engine cache | TRT engines | `.engine` files keyed by `(SM, JP, TRT, precision)` (D-C10-7) | | 8 | C2 backbone (driven by C10) | C6 FAISS index | Descriptor matrix | `.index` (FAISS HNSW), atomicwrites, SHA-256 sidecar | | 9 | C10 | filesystem | Manifest (carries `takeoff_origin` + hashes) | YAML or JSON | diff --git a/_docs/02_tasks/_dependencies_table.md b/_docs/02_tasks/_dependencies_table.md index 5291409..6257b7d 100644 --- a/_docs/02_tasks/_dependencies_table.md +++ b/_docs/02_tasks/_dependencies_table.md @@ -1,8 +1,8 @@ # Dependencies Table -**Date**: 2026-05-13 (refreshed after AZ-507 + AZ-508 hygiene-PBI onboarding from cumulative review batches 31-33; previously 2026-05-11 for AZ-489 + AZ-490 ADR-010 operator-origin path) -**Total Tasks**: 144 (103 product + 41 blackbox-test) -**Total Complexity Points**: 482 (349 product + 133 blackbox-test) +**Date**: 2026-05-13 (refreshed after Batch 44 SRP refactor: AZ-317 superseded; AZ-329 + AZ-330 specs rewritten; AZ-523 + AZ-524 audit-trail tickets added; E-C12 epic renamed `Operator Pre-flight Tooling` → `Operator Pre-flight Orchestrator`; earlier same-day refresh: AZ-507 + AZ-508 hygiene PBIs from cumulative review batches 31-33; 2026-05-11: AZ-489 + AZ-490 ADR-010 operator-origin path) +**Total Tasks**: 146 (105 product + 41 blackbox-test) — AZ-317 retained in the table marked SUPERSEDED for audit; AZ-523 (C11 gate removal) + AZ-524 (C12 rename) added as 2 closed audit-trail tasks +**Total Complexity Points**: 487 (354 product + 133 blackbox-test) — AZ-523 = 3pt, AZ-524 = 2pt Dependencies columns list only the tracker-ID portion (descriptive tail text in each task spec is omitted here for table-readability). The @@ -52,9 +52,9 @@ are all declared and documented below under **Cycle Check**. | AZ-307 | C6 Freshness Gate | 2 | AZ-303, AZ-304, AZ-305, AZ-263, AZ-269, AZ-266, AZ-273 | AZ-250 | | AZ-308 | C6 Cache Budget Eviction | 3 | AZ-303, AZ-305, AZ-263, AZ-269, AZ-266, AZ-273 | AZ-250 | | AZ-316 | C11 TileDownloader | 5 | AZ-263, AZ-269, AZ-266, AZ-303, AZ-305, AZ-307, AZ-308 | AZ-251 | -| AZ-317 | C11 Flight-State Gate | 2 | AZ-263, AZ-269, AZ-266 | AZ-251 | +| AZ-317 | C11 Flight-State Gate (SUPERSEDED by Batch 44 / AZ-523; gate moved to C12 AZ-329) | 2 | AZ-263, AZ-269, AZ-266 | AZ-251 | | AZ-318 | C11 Per-Flight Signing Key | 3 | AZ-263, AZ-269, AZ-266, AZ-273 | AZ-251 | -| AZ-319 | C11 TileUploader | 5 | AZ-263, AZ-269, AZ-266, AZ-273, AZ-303, AZ-305, AZ-317, AZ-318 | AZ-251 | +| AZ-319 | C11 TileUploader (contract v2.0.0 — internal flight-state gate removed in Batch 44) | 5 | AZ-263, AZ-269, AZ-266, AZ-273, AZ-303, AZ-305, AZ-318 | AZ-251 | | AZ-320 | C11 Idempotent Retry Decorator | 3 | AZ-263, AZ-269, AZ-266, AZ-273, AZ-303, AZ-319 | AZ-251 | | AZ-321 | C10 Engine Compiler | 5 | AZ-263, AZ-269, AZ-266, AZ-280, AZ-281, AZ-298 | AZ-252 | | AZ-322 | C10 Descriptor Batcher | 3 | AZ-263, AZ-269, AZ-266, AZ-303, AZ-306, AZ-321 | AZ-252 | @@ -64,8 +64,8 @@ are all declared and documented below under **Cycle Check**. | AZ-326 | C12 CLI App | 3 | AZ-263, AZ-269, AZ-266, AZ-489 | AZ-253 | | AZ-327 | C12 Companion Bringup | 3 | AZ-263, AZ-269, AZ-266 | AZ-253 | | AZ-328 | C12 Build-Cache Orchestrator | 5 | AZ-326, AZ-327, AZ-316, AZ-325, AZ-489, AZ-263, AZ-269, AZ-266 | AZ-253 | -| AZ-329 | C12 Post-Landing Upload | 3 | AZ-326, AZ-319, AZ-272, AZ-263, AZ-269, AZ-266 | AZ-253 | -| AZ-330 | C12 OperatorReLocService | 3 | AZ-326, AZ-273, AZ-263, AZ-269, AZ-266 | AZ-253 | +| AZ-329 | C12 PostLandingUploadOrchestrator (flight_footer FDR gate; Batch 44 design pivot) | 3 | AZ-326, AZ-319, AZ-272, AZ-273, AZ-292, AZ-263, AZ-269, AZ-266 | AZ-253 | +| AZ-330 | C12 OperatorReLocService | 3 | AZ-326, AZ-273, AZ-272, AZ-263, AZ-269, AZ-266 | AZ-253 | | AZ-331 | C1 VioStrategy Protocol | 3 | AZ-263, AZ-269, AZ-266, AZ-270, AZ-272, AZ-276, AZ-277 | AZ-254 | | AZ-332 | C1 OKVIS2 Strategy | 5 | AZ-331, AZ-263, AZ-269, AZ-266, AZ-276, AZ-277, AZ-272, AZ-273 | AZ-254 | | AZ-333 | C1 VINS-Mono Strategy | 5 | AZ-331, AZ-263, AZ-269, AZ-266, AZ-276, AZ-277, AZ-272, AZ-273 | AZ-254 | @@ -158,6 +158,8 @@ are all declared and documented below under **Cycle Check**. | AZ-490 | C5 set_takeoff_origin entrypoint — accept operator origin from C10 Manifest | 3 | AZ-263, AZ-269, AZ-266, AZ-272, AZ-273, AZ-279, AZ-381, AZ-383, AZ-384, AZ-385, AZ-386 | AZ-260 | | AZ-507 | Hygiene — align module-layout.md cross-component import rules with AZ-270 lint | 2 | AZ-263, AZ-270, AZ-321 | AZ-246 | | AZ-508 | Hygiene — consolidate `_iso_ts_now` helpers into `helpers/iso_timestamps.py` | 2 | AZ-263 | AZ-264 | +| AZ-523 | Batch 44 — C11 internal flight-state gate removal (SRP refactor; audit-trail; closed) | 3 | AZ-317, AZ-319, AZ-329 | AZ-251 | +| AZ-524 | Batch 44 — C12 package rename: c12_operator_tooling → c12_operator_orchestrator (audit; closed)| 2 | AZ-263, AZ-326, AZ-327, AZ-328, AZ-329, AZ-330, AZ-489 | AZ-253 | ## Notes @@ -213,6 +215,36 @@ are all declared and documented below under **Cycle Check**. - **All E-BBT tasks depend on AZ-406 (test infrastructure)**; this is by design — AZ-406 is the foundation every blackbox test depends on (analogous to AZ-263 for the product side). +- **Batch 44 SRP refactor + C12 rename** (added 2026-05-13): + - **AZ-317 (C11 Flight-State Gate)** is **superseded**. The + C11-internal gate (`confirm_flight_state` / + `FlightStateSignal` / `FlightStateNotOnGroundError`) was removed + in Batch 44 Phase B; the post-landing safety responsibility + moved to C12's new `PostLandingUploadOrchestrator` (AZ-329). + The row is retained in the table for audit; the ticket is in + `_docs/02_tasks/done/` with a SUPERSEDED banner. + - **AZ-319 (C11 TileUploader)** lost its dependency on AZ-317 + (gate removed) and the `TileUploader` Protocol contract was + bumped to **v2.0.0 (frozen)** with the gate parameters removed. + Migration note in + `_docs/02_document/contracts/c11_tilemanager/tile_uploader.md`. + - **AZ-329 (C12 PostLandingUploadOrchestrator)** specification + was rewritten in Phase C to gate on the `flight_footer` FDR + record's `clean_shutdown` field instead of counting consecutive + `FlightStateSignal` records. Added explicit dependency on + AZ-292 (C13 footer write) since the orchestrator reads the + footer record produced there. + - **AZ-330 (C12 OperatorReLocService)** added an explicit + dependency on AZ-272 (FDR schema) since the service emits a + new `c12.reloc.requested` FDR record kind. + - **AZ-523 (C11 gate removal audit-trail)** and **AZ-524 (C12 + package rename audit-trail)** are post-hoc tickets closed + on creation. Their dependencies (AZ-317/319/329 for AZ-523; + the C12 task set for AZ-524) are listed for traceability; + these tickets are not gates on any future work. + - **E-C12 epic (AZ-253) summary renamed**: + `C12 Operator Pre-flight Tooling` → + `C12 Operator Pre-flight Orchestrator`. - **Hygiene PBIs from cumulative review batches 31-33** (added 2026-05-13): - **AZ-507** (E-CC-CONF / AZ-246) — module-layout.md ↔ AZ-270 lint @@ -240,8 +272,8 @@ are all declared and documented below under **Cycle Check**. - C7 `InferenceRuntime` → AZ-297 (Protocol) + AZ-298/299/300/301/302 - C8 `FcAdapter` / `GcsAdapter` → AZ-390 (Protocols) + AZ-391..AZ-397 - C10 Provisioning → AZ-321/322/323/324/325 - - C11 Tile Manager → AZ-316/317/318/319/320 - - C12 Operator Tooling → AZ-326/327/328/329/330 + AZ-489 (FlightsApiClient) + - C11 Tile Manager → AZ-316/318/319/320 + AZ-523 (Batch 44 gate-removal audit; AZ-317 superseded) + - C12 Operator Pre-flight Orchestrator → AZ-326/327/328/329/330 + AZ-489 (FlightsApiClient) + AZ-524 (Batch 44 rename audit) - C13 FDR Writer → AZ-291..AZ-296 - **Cross-cutting product modules**: diff --git a/_docs/02_tasks/done/AZ-263_initial_structure.md b/_docs/02_tasks/done/AZ-263_initial_structure.md index 295c307..002694b 100644 --- a/_docs/02_tasks/done/AZ-263_initial_structure.md +++ b/_docs/02_tasks/done/AZ-263_initial_structure.md @@ -95,7 +95,7 @@ gps-denied-onboard/ │ ├── c8_fc_adapter/ # AZ-261: FcAdapter (PymavlinkArdupilotAdapter + Msp2InavAdapter) + GcsAdapter │ ├── c10_provisioning/ # AZ-252: CacheProvisioner (engine compile + descriptors + manifest + content-hash) │ ├── c11_tile_manager/ # AZ-251: TileDownloader + TileUploader (operator-side ONLY — excluded from airborne via CMake) -│ ├── c12_operator_tooling/ # AZ-253: CacheBuildWorkflow + OperatorReLocService (CLI; GUI deferred) +│ ├── c12_operator_orchestrator/ # AZ-253: CacheBuildWorkflow + OperatorReLocService (CLI; GUI deferred) │ └── c13_fdr/ # AZ-248: FdrWriter (writer thread + segment rotation + ≤64 GB cap) │ ├── cpp/ # Native libraries linked from src/gps_denied_onboard/components/* via pybind11 @@ -193,7 +193,7 @@ Concrete implementations are NOT created here — they are the subject of Step 2 | C8 | `FcAdapter`, `GcsAdapter` | `components/10_c8_fc_adapter/description.md § 2` | | C10 | `CacheProvisioner` | `components/11_c10_provisioning/description.md § 2` | | C11 | `TileDownloader`, `TileUploader` | `components/12_c11_tilemanager/description.md § 2` | -| C12 | `CacheBuildWorkflow`, `OperatorReLocService` | `components/13_c12_operator_tooling/description.md § 2` | +| C12 | `CacheBuildWorkflow`, `OperatorReLocService` | `components/13_c12_operator_orchestrator/description.md § 2` | | C13 | `FdrWriter` (consumer side) | `components/14_c13_fdr/description.md § 2` | ## CI/CD Pipeline diff --git a/_docs/02_tasks/done/AZ-317_c11_flight_state_gate.md b/_docs/02_tasks/done/AZ-317_c11_flight_state_gate.md index 4a1987b..233280c 100644 --- a/_docs/02_tasks/done/AZ-317_c11_flight_state_gate.md +++ b/_docs/02_tasks/done/AZ-317_c11_flight_state_gate.md @@ -1,5 +1,7 @@ # C11 Flight-State Gate — ON_GROUND Defence-in-Depth for Upload +> **Status (2026-05-13): SUPERSEDED by Batch 44.** This task originally placed an `ON_GROUND` gate inside `HttpTileUploader` (C11). Batch 44's SRP refactor removed that gate — "upload bytes" and "decide when uploading is safe" are different responsibilities. The post-landing safety check now lives in C12's `PostLandingUploadOrchestrator` (AZ-329), which inspects the C13 `flight_footer` FDR record and refuses to invoke `TileUploader.upload_pending_tiles` unless `clean_shutdown=True` is recorded. The `FlightStateGate`, `FlightStateSource` Protocol, and `FlightStateNotOnGroundError` have been deleted from C11; this spec is kept here as historical record. See `_docs/03_implementation/batch_44_implementation_plan.md` Phase B for the deletion details. + **Task**: AZ-317_c11_flight_state_gate **Name**: C11 Flight-State Gate **Description**: Implement the `flight_state == ON_GROUND` precondition check that `TileUploader.upload_pending_tiles` calls before any network egress. Defines a thin C11-internal `FlightStateSource` Protocol with one method `current_flight_state() -> FlightStateSignal`; the concrete impl is supplied by E-C8 later (subscribes to the FC adapter's flight-state stream). The gate raises `FlightStateNotOnGroundError` if the current state is anything other than `ON_GROUND` (`IN_FLIGHT`, `UNKNOWN`, `TAKING_OFF`, `LANDING` all block). Logs an ERROR with the observed state and refuses to proceed; this is defence-in-depth atop ADR-004's process-level isolation, NOT the primary control. diff --git a/_docs/02_tasks/done/AZ-326_c12_cli_app.md b/_docs/02_tasks/done/AZ-326_c12_cli_app.md index a993dfd..1b99c7f 100644 --- a/_docs/02_tasks/done/AZ-326_c12_cli_app.md +++ b/_docs/02_tasks/done/AZ-326_c12_cli_app.md @@ -5,13 +5,13 @@ **Description**: Implement the operator-tooling CLI shell that operators run on the workstation. Wires Typer (per the Click/Typer project pin) into `operator_tool/__main__.py`, registers six subcommands (`download`, `build-cache`, `upload-pending`, `reloc-confirm`, `verify-ready`, `set-sector`), wires the E-CC-LOG (AZ-266) logger to a workstation-side structured-JSON log file (`~/.azaion/onboard/c12-tooling.log`), and ships the two trivial operator-side helpers from description.md § 2 — `set_sector_classification(area, sector_class)` (persists per-area classification to a local JSON file under the operator workstation's home directory) and `apply_freshness_threshold(sector_class) -> int (months)` (a pure-data lookup that maps the sector classification enum to the AC-NEW-6 months freshness budget). Each subcommand is a thin shell that resolves its service collaborator (`flights_api_client`, `build_cache`, `companion_bringup`, `post_landing_upload`, `operator_reloc_service` — all owned by sibling tasks AZ-489 / AZ-NNN T2..T5) from the composition root and delegates to it; on success returns 0; on a known error type maps to a documented non-zero exit code with a one-line operator-friendly message + remediation hint pulled from the underlying error's `remediation` attribute. The CLI app does NOT own any workflow logic itself — only command registration, argument parsing, logger wiring, exit-code mapping, and the two simple operator helpers. **ADR-010 amendment**: the `build-cache` subcommand accepts a mutually-exclusive pair `--flight-id | --flight-file ` and forwards the resolved `FlightDto` (via AZ-489 `FlightsApiClient`) to the orchestrator (AZ-328), which derives the bbox + takeoff origin from it. The legacy `--bbox` flag is dropped because the bbox is now derived; passing it is an error. **Complexity**: 3 points **Dependencies**: AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module, AZ-489_c12_flights_api_client (for the `FlightsApiClient` service collaborator + DTO definitions surfaced via `--flight-id` / `--flight-file`) -**Component**: c12_operator_tooling (epic AZ-253 / E-C12) +**Component**: c12_operator_orchestrator (epic AZ-253 / E-C12) **Tracker**: AZ-326 **Epic**: AZ-253 (E-C12) ### Document Dependencies -- `_docs/02_document/components/13_c12_operator_tooling/description.md` — § 2 (`set_sector_classification`, `apply_freshness_threshold` from `CacheBuildWorkflow`), § 5 (logging strategy table), § 7 (CLI-only this cycle, GUI deferred). +- `_docs/02_document/components/13_c12_operator_orchestrator/description.md` — § 2 (`set_sector_classification`, `apply_freshness_threshold` from `CacheBuildWorkflow`), § 5 (logging strategy table), § 7 (CLI-only this cycle, GUI deferred). - `_docs/02_document/contracts/shared_logging/log_record_schema.md` — INFO/WARN/ERROR log shapes for operator events. ## Problem @@ -23,7 +23,7 @@ Without a real CLI shell: - Sector classification (active-conflict vs stable-rear) per description.md § 1 has no persistent surface; operator restarts lose all classifications. - Logging from C12 is silent — without the wiring of E-CC-LOG to the workstation-side log file, every operator action is invisible during incident review. - Sibling tasks T2..T5 have no consumer; their service classes ship but no end-to-end CLI flow exercises them. -- Exit codes are inconsistent across subcommands — operators script `operator-tool` runs and need `$?` to mean something specific per failure category. +- Exit codes are inconsistent across subcommands — operators script `operator-orchestrator` runs and need `$?` to mean something specific per failure category. This task delivers the CLI shell + the two trivial operator helpers. It does NOT own `build_cache`, `verify_companion_ready`, `trigger_post_landing_upload`, or `OperatorReLocService` — those are sibling tasks invoked through the CLI. @@ -31,7 +31,7 @@ This task delivers the CLI shell + the two trivial operator helpers. It does NOT - A Typer-based CLI app at `src/operator_tool/`: - `src/operator_tool/__main__.py` — module entry point: `from operator_tool.cli import app; app()`. - - `src/operator_tool/cli.py` — Typer `app = typer.Typer(name="operator-tool", help="GPS-denied onboard pre-flight tooling (operator workstation)")`. Registers six subcommands via `@app.command(...)`. Each subcommand opens a logging context, calls into its service collaborator, catches the documented exception family for that command, maps to the documented exit code, and `raise typer.Exit(code=N)`. + - `src/operator_tool/cli.py` — Typer `app = typer.Typer(name="operator-orchestrator", help="GPS-denied onboard pre-flight tooling (operator workstation)")`. Registers six subcommands via `@app.command(...)`. Each subcommand opens a logging context, calls into its service collaborator, catches the documented exception family for that command, maps to the documented exit code, and `raise typer.Exit(code=N)`. - `src/operator_tool/sector_classification_store.py` — `SectorClassificationStore` class: - Constructor: `__init__(self, *, store_path: Path, logger: Logger)`. - `set_classification(area: AreaIdentifier, sector_class: SectorClassification) -> None` — persists `{area_id: sector_class}` mapping to `store_path` (default: `~/.azaion/onboard/sector-classifications.json`) using atomic write (`tempfile + os.replace`). @@ -44,7 +44,7 @@ This task delivers the CLI shell + the two trivial operator helpers. It does NOT - Module-level constant: `FRESHNESS_TABLE: dict[SectorClassification, int]`. - `src/operator_tool/exit_codes.py` — module-level constants: `EXIT_OK = 0`, `EXIT_GENERIC_ERROR = 1`, `EXIT_USAGE = 2`, `EXIT_COMPANION_UNREACHABLE = 10`, `EXIT_CONTENT_HASH_MISMATCH = 11`, `EXIT_DOWNLOAD_FAILURE = 20`, `EXIT_BUILD_FAILURE = 21`, `EXIT_FLIGHT_STATE_NOT_CONFIRMED = 30`, `EXIT_UPLOAD_FAILURE = 31`, `EXIT_GCS_LINK_ERROR = 40`, `EXIT_LOCK_HELD = 50`, `EXIT_FLIGHTS_API_UNREACHABLE = 60`, `EXIT_FLIGHTS_API_AUTH = 61`, `EXIT_FLIGHT_NOT_FOUND = 62`, `EXIT_FLIGHT_SCHEMA = 63`, `EXIT_EMPTY_WAYPOINTS = 64`. Sibling tasks may extend with documented additions. - A composition root entry at `src/gps_denied_onboard/runtime_root/c12_factory.py`: - - `build_operator_tool(config: Config) -> OperatorToolServices` — pure factory that constructs the `SectorClassificationStore` + a logger configured to write to `~/.azaion/onboard/c12-tooling.log`. Returns a frozen dataclass aggregating the operator-tool service handles. Sibling tasks T2..T5 each add their service to this dataclass without renaming or moving it. + - `build_operator_tool(config: Config) -> OperatorOrchestratorServices` — pure factory that constructs the `SectorClassificationStore` + a logger configured to write to `~/.azaion/onboard/c12-tooling.log`. Returns a frozen dataclass aggregating the operator-orchestrator service handles. Sibling tasks T2..T5 each add their service to this dataclass without renaming or moving it. - Subcommand surface (each subcommand body lives in `cli.py`; service implementations live in sibling task files): - `download` — delegates to `tile_downloader.fetch(...)` (AZ-316). Maps `SatelliteProviderError → EXIT_DOWNLOAD_FAILURE`. - `build-cache` — accepts a mutually-exclusive pair `--flight-id | --flight-file ` (Typer-enforced via a callback that rejects both-set / neither-set with `EXIT_USAGE`), plus `--sector-class`, `--calibration-path`. Delegates to `build_cache_orchestrator.build_cache(...)` (sibling AZ-328) passing the resolved `FlightDto` (the orchestrator computes bbox + takeoff origin from it via AZ-489 helpers). Maps `CacheBuildError → EXIT_DOWNLOAD_FAILURE | EXIT_BUILD_FAILURE` (per `failure_phase`); `BuildLockHeldError → EXIT_LOCK_HELD`; `FlightsApiUnreachableError → EXIT_FLIGHTS_API_UNREACHABLE`; `FlightsApiAuthError → EXIT_FLIGHTS_API_AUTH`; `FlightNotFoundError → EXIT_FLIGHT_NOT_FOUND`; `FlightsApiSchemaError | FlightFileNotFoundError | WaypointSchemaError → EXIT_FLIGHT_SCHEMA`; `EmptyWaypointsError → EXIT_EMPTY_WAYPOINTS`. @@ -54,7 +54,7 @@ This task delivers the CLI shell + the two trivial operator helpers. It does NOT - `set-sector` — delegates to `SectorClassificationStore.set_classification(...)`. - Each subcommand's `--help` includes a one-line summary + the AC IDs it supports (e.g. `build-cache: orchestrate F1 (AC-8.3, AC-NEW-1)`). - Logging is wired at app startup: a single rotating file handler at `~/.azaion/onboard/c12-tooling.log`, structured JSON formatter from E-CC-LOG (AZ-266). Console (stderr) handler at WARN level for operator visibility. -- `pyproject.toml` registers `operator-tool` as a console script entry point pointing at `operator_tool.__main__:main`. The `main` function in `__main__.py` calls `app()`. +- `pyproject.toml` registers `operator-orchestrator` as a console script entry point pointing at `operator_tool.__main__:main`. The `main` function in `__main__.py` calls `app()`. ## Scope @@ -82,8 +82,8 @@ This task delivers the CLI shell + the two trivial operator helpers. It does NOT ## Acceptance Criteria **AC-1: All six subcommands register and appear in `--help`** -Given the `operator-tool` console script is installed -When the operator runs `operator-tool --help` +Given the `operator-orchestrator` console script is installed +When the operator runs `operator-orchestrator --help` Then the listed subcommands include exactly `download`, `build-cache`, `upload-pending`, `reloc-confirm`, `verify-ready`, `set-sector`; no extras **AC-2: Successful subcommand exits 0** @@ -118,12 +118,12 @@ Then a `c12-tooling.log` file exists at `~/.azaion/onboard/`; its lines parse as **AC-8: Console-script entry point is installed and runnable** Given the package is installed via `pip install -e .` -When the shell runs `operator-tool --help` +When the shell runs `operator-orchestrator --help` Then the help text is printed; the exit code is 0; the binary resolves through the entry-point declared in `pyproject.toml` **AC-9: Subcommand `--help` references the relevant AC IDs** Given any subcommand -When `operator-tool --help` is run +When `operator-orchestrator --help` is run Then the help text body includes the AC IDs the subcommand supports (e.g. `build-cache` mentions `AC-8.3, AC-NEW-1`); operators reading `--help` can cross-reference to `acceptance_criteria.md` **AC-10: `set-sector` is idempotent for the same input** @@ -133,20 +133,20 @@ Then the on-disk JSON file is byte-identical (or has only timestamp diffs in the **AC-11: `build-cache --flight-id` happy path delegates to orchestrator with `FlightDto` (ADR-010)** Given a fake `FlightsApiClient.fetch_flight` returns a 3-waypoint `FlightDto` -When `operator-tool build-cache --flight-id 00000000-0000-0000-0000-000000000001 --sector-class stable_rear --calibration-path /tmp/cal.json` runs +When `operator-orchestrator build-cache --flight-id 00000000-0000-0000-0000-000000000001 --sector-class stable_rear --calibration-path /tmp/cal.json` runs Then `build_cache_orchestrator.build_cache(...)` is called once with the resolved `FlightDto` (or its `(flight_id, bbox, takeoff_origin)` projection per AZ-328 signature); ZERO calls to `--bbox` legacy parsing **AC-12: `build-cache --flight-file` happy path uses offline loader** Given a local JSON file in the documented schema is on disk -When `operator-tool build-cache --flight-file /tmp/flight.json --sector-class stable_rear --calibration-path /tmp/cal.json` runs +When `operator-orchestrator build-cache --flight-file /tmp/flight.json --sector-class stable_rear --calibration-path /tmp/cal.json` runs Then `FlightsApiClient.load_flight_file(/tmp/flight.json)` is called once; `fetch_flight` is NOT called; the orchestrator receives the same DTO shape **AC-13: `build-cache` with both `--flight-id` and `--flight-file` errors out** -When `operator-tool build-cache --flight-id 00000000-0000-0000-0000-000000000001 --flight-file /tmp/flight.json ...` runs +When `operator-orchestrator build-cache --flight-id 00000000-0000-0000-0000-000000000001 --flight-file /tmp/flight.json ...` runs Then exit code is `EXIT_USAGE = 2`; stderr names the conflict; ZERO calls to either client method **AC-14: `build-cache` with neither `--flight-id` nor `--flight-file` errors out** -When `operator-tool build-cache --sector-class stable_rear --calibration-path /tmp/cal.json` runs (no flight source) +When `operator-orchestrator build-cache --sector-class stable_rear --calibration-path /tmp/cal.json` runs (no flight source) Then exit code is `EXIT_USAGE = 2`; stderr lists which flag must be supplied **AC-15: `FlightNotFoundError` maps to `EXIT_FLIGHT_NOT_FOUND`** @@ -167,7 +167,7 @@ Then exit code is `64`; the stderr message instructs the operator to re-plan in ## Non-Functional Requirements **Performance** -- CLI cold start (`operator-tool --help`) ≤ 500 ms on a developer laptop. The Typer app must avoid eager-importing heavy dependencies (httpx, pymavlink, paramiko) — sibling tasks expose lazy-import accessors used by their respective subcommands, not at module load time. +- CLI cold start (`operator-orchestrator --help`) ≤ 500 ms on a developer laptop. The Typer app must avoid eager-importing heavy dependencies (httpx, pymavlink, paramiko) — sibling tasks expose lazy-import accessors used by their respective subcommands, not at module load time. **Compatibility** - Click/Typer per the project pin (no version override). @@ -181,14 +181,14 @@ Then exit code is `64`; the stderr message instructs the operator to re-plan in | AC Ref | What to Test | Required Outcome | |--------|-------------|-----------------| -| AC-1 | `operator-tool --help` output | All 6 subcommands listed | +| AC-1 | `operator-orchestrator --help` output | All 6 subcommands listed | | AC-2 | Subcommand with success-returning fake service | Exit 0, INFO log, no stderr | | AC-3 | Subcommand with raising fake (each documented exception family) | Exit code matches `exit_codes.py`; ERROR log; one-line stderr | | AC-4 | Round-trip `SectorClassificationStore` set → read | Matches input | | AC-5 | Patched `os.replace` to raise mid-write | Original file intact, no `*.tmp` lingers | | AC-6 | `freshness_threshold_months` for both enums | `active_conflict → 1`, `stable_rear → 12` | | AC-7 | Subcommand run, then read log file | Each line parses as JSON; required fields present | -| AC-8 | `subprocess.run(["operator-tool", "--help"])` after `pip install -e .` | Exit 0, help text printed | +| AC-8 | `subprocess.run(["operator-orchestrator", "--help"])` after `pip install -e .` | Exit 0, help text printed | | AC-9 | Per-subcommand `--help` text | Includes documented AC IDs | | AC-10 | Repeated `set-sector` for same area/class | On-disk JSON byte-identical | | AC-11 | `build-cache --flight-id` happy path | Orchestrator called once with resolved DTO | @@ -198,7 +198,7 @@ Then exit code is `64`; the stderr message instructs the operator to re-plan in | AC-15 | `FlightNotFoundError` | Exit 62; flight_id in log | | AC-16 | `FlightsApiAuthError` | Exit 61; auth_token NOT in log | | AC-17 | `EmptyWaypointsError` | Exit 64; Mission Planner UI hint | -| NFR-perf-cold-start | Microbench `operator-tool --help` × 10 | p99 ≤ 500 ms | +| NFR-perf-cold-start | Microbench `operator-orchestrator --help` × 10 | p99 ≤ 500 ms | ## Constraints @@ -216,11 +216,11 @@ Then exit code is `64`; the stderr message instructs the operator to re-plan in - *Mitigation*: AC-NFR-perf-cold-start microbenches startup; CI hooks the test. If a regression appears, the offending import is surfaced by `python -X importtime`. **Risk 2: Operator runs `set-sector` against a stale store path after upgrade** -- *Risk*: An operator upgrades the operator-tool tarball; the new version changes the default `store_path`; classifications appear lost. +- *Risk*: An operator upgrades the operator-orchestrator tarball; the new version changes the default `store_path`; classifications appear lost. - *Mitigation*: The default path is fixed at `~/.azaion/onboard/sector-classifications.json` and treated as a stable contract. A future cycle that needs to migrate runs an explicit migration; this cycle does NOT change the path. **Risk 3: Console script collides with another tool** -- *Risk*: The name `operator-tool` is generic; another package on the operator's workstation could shadow it. +- *Risk*: The name `operator-orchestrator` is generic; another package on the operator's workstation could shadow it. - *Mitigation*: The package is shipped as part of the operator-tooling tarball with its own venv; no global install. README documents the tarball install procedure. **Risk 4: Atomic-write corner case — disk full mid-tempfile** diff --git a/_docs/02_tasks/done/AZ-327_c12_companion_bringup.md b/_docs/02_tasks/done/AZ-327_c12_companion_bringup.md index 8b25814..d5e2517 100644 --- a/_docs/02_tasks/done/AZ-327_c12_companion_bringup.md +++ b/_docs/02_tasks/done/AZ-327_c12_companion_bringup.md @@ -5,13 +5,13 @@ **Description**: Implement `CompanionBringup`, the C12-internal helper that opens an SSH session against the companion (paramiko per project pin), inspects the companion-side filesystem for the four required pre-flight artifacts (Manifest.json, .engine files + AZ-280 sidecars, calibration JSON), runs sidecar verification on the engines via a remote `sha256sum` over the engine path (compared against the sidecar's hex digest), and returns a `ReadinessReport` per description.md § 2 (`manifest_present`, `content_hashes_pass`, `engines_present`, `calibration_present`, `outcome ∈ {ready, not_ready}`, `not_ready_reasons: list[str]`). Owns the two error families: `CompanionUnreachableError` (SSH session-open failure: TCP refused, auth failed, host key mismatch, socket timeout) and `ContentHashMismatchError` (sidecar verification fails on at least one engine — distinct from "engine missing", which is a not-ready signal not an exception). Public surface is one method `verify_companion_ready(companion_address: CompanionAddress) -> ReadinessReport`. SSH user, key file, host-key policy, connect-timeout, and the canonical companion-side cache root come from config (`config.c12.companion_ssh_user`, `config.c12.companion_ssh_keyfile`, `config.c12.companion_host_key_policy`, `config.c12.companion_connect_timeout_s`, `config.c12.companion_cache_root`) per AZ-269. The session is opened in a `try/finally` block; the connection is always closed even if the four checks raise. INFO log on every successful call (with the four boolean flags + outcome); WARN on degraded readiness (any 3-of-4); ERROR on the two error families. **Complexity**: 3 points **Dependencies**: AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module -**Component**: c12_operator_tooling (epic AZ-253 / E-C12) +**Component**: c12_operator_orchestrator (epic AZ-253 / E-C12) **Tracker**: AZ-327 **Epic**: AZ-253 (E-C12) ### Document Dependencies -- `_docs/02_document/components/13_c12_operator_tooling/description.md` — § 2 (`verify_companion_ready` interface + `ReadinessReport` DTO shape), § 5 (`CompanionUnreachableError`, `ContentHashMismatchError`), § 7 (filesystem lockfile note — relevant for orchestrator T3 not this task). +- `_docs/02_document/components/13_c12_operator_orchestrator/description.md` — § 2 (`verify_companion_ready` interface + `ReadinessReport` DTO shape), § 5 (`CompanionUnreachableError`, `ContentHashMismatchError`), § 7 (filesystem lockfile note — relevant for orchestrator T3 not this task). - `_docs/02_document/contracts/shared_helpers/sha256_sidecar.md` — sidecar file format (this task verifies remotely; does not import the helper but reuses the schema). - `_docs/02_document/contracts/shared_helpers/engine_filename_schema.md` — engine filename layout used to enumerate the expected engines list. - `_docs/02_document/contracts/shared_logging/log_record_schema.md` — INFO/WARN/ERROR log shapes. @@ -39,7 +39,7 @@ This task delivers the bring-up + verification layer. It does NOT orchestrate th - `ReadinessReport` (`@dataclass(frozen=True)`): `manifest_present: bool`, `content_hashes_pass: bool`, `engines_present: bool`, `calibration_present: bool`, `outcome: enum {ready, not_ready}`, `not_ready_reasons: tuple[str, ...]`, `companion_cache_root: str`, `engines_inspected_count: int`. - Errors at `src/operator_tool/errors.py`: - `CompanionUnreachableError(Exception)`: attributes `host: str`, `port: int`, `reason: enum {connect_refused, auth_failed, host_key_mismatch, timeout, other}`, `underlying_exception_repr: str`. `remediation` attribute returns a one-line operator-friendly hint per `reason`. - - `ContentHashMismatchError(Exception)`: attributes `engine_path: str`, `expected_sha256_hex: str`, `actual_sha256_hex: str`. `remediation` attribute returns "Re-run the cache build (`operator-tool build-cache --area ...`) to repopulate the affected engine.". + - `ContentHashMismatchError(Exception)`: attributes `engine_path: str`, `expected_sha256_hex: str`, `actual_sha256_hex: str`. `remediation` attribute returns "Re-run the cache build (`operator-orchestrator build-cache --area ...`) to repopulate the affected engine.". - A `SshSessionFactory` Protocol at `src/operator_tool/ssh_session.py`: ```python @runtime_checkable @@ -65,7 +65,7 @@ This task delivers the bring-up + verification layer. It does NOT orchestrate th 6. Compute `outcome`: `ready` iff all four booleans are `True`; `not_ready` otherwise. 7. Emit log: INFO `kind="c12.companion.ready"` with the four flags + outcome on success; WARN `kind="c12.companion.degraded"` if any check failed without raising (i.e. `outcome=not_ready` due to a missing artifact, not a hash mismatch). 8. Return the `ReadinessReport`. -- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py` extends T1's `OperatorToolServices` dataclass with a `companion_bringup: CompanionBringup` field. The factory `build_companion_bringup(config) -> CompanionBringup` constructs the paramiko-backed session factory + remote sidecar verifier + logger. +- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py` extends T1's `OperatorOrchestratorServices` dataclass with a `companion_bringup: CompanionBringup` field. The factory `build_companion_bringup(config) -> CompanionBringup` constructs the paramiko-backed session factory + remote sidecar verifier + logger. ## Scope diff --git a/_docs/02_tasks/done/AZ-328_c12_build_cache_orchestrator.md b/_docs/02_tasks/done/AZ-328_c12_build_cache_orchestrator.md index 1993b1a..8df0412 100644 --- a/_docs/02_tasks/done/AZ-328_c12_build_cache_orchestrator.md +++ b/_docs/02_tasks/done/AZ-328_c12_build_cache_orchestrator.md @@ -5,7 +5,7 @@ **Description**: Implement `BuildCacheOrchestrator`, the public top-level F1 (pre-flight cache build) workflow. `build_cache(request: BuildCacheRequest) -> CacheBuildReport` does the following sequenced work, with strict ordering: **(0) Flight-resolve phase (ADR-010, AZ-489)** — the orchestrator either calls `flights_api_client.fetch_flight(flight_id, base_url, auth_token)` (online) or `flights_api_client.load_flight_file(path)` (offline) per the resolved CLI flag, then `bbox = flights_api_client.bbox_from_waypoints(flight.waypoints, buffer_m=config.flight_bbox_buffer_m)` and `takeoff_origin = flights_api_client.takeoff_origin_from_flight(flight)`. The resolved `(bbox, takeoff_origin, flight_id, raw_flight_dto)` is captured into `FlightResolveReport` for FDR/debug and forwarded into the downstream phases; any `FlightsApiUnreachableError` / `FlightsApiAuthError` / `FlightNotFoundError` / `FlightsApiSchemaError` / `FlightFileNotFoundError` / `EmptyWaypointsError` / `WaypointSchemaError` is wrapped as `CacheBuildError(failure_phase=flight_resolve, ...)` and aborts BEFORE the lockfile is even acquired (no point holding the lock while diagnosing operator inputs). (1) acquire a filesystem lockfile at `/.c12.lock` per description.md § 7 (prevents concurrent F1 runs from stomping each other); (2) call `tile_downloader.fetch(...)` (AZ-316) on the operator workstation with `bbox` (computed in phase 0), `sector_class`, `freshness_threshold_months`, `satellite_provider_url`, `api_key`; (3) on download `failure` outcome → wrap as `CacheBuildError(failure_phase=download, ...)` and return `CacheBuildReport(outcome=failure, failure_phase=download, flight_resolve_report=..., download_report=..., build_report=None)` WITHOUT invoking C10; (4) on download `success` → call `companion_bringup.verify_companion_ready(...)` (AZ-327) — if `not_ready` → wrap and return `CacheBuildReport(outcome=failure, failure_phase=download, ...)`; (5) SSH-invoke `C10.CacheProvisioner.build_cache_artifacts` (AZ-325) on the companion via the `RemoteCacheProvisionerInvoker` helper, **passing `takeoff_origin` + `flight_id` along with bbox/sector_class** so AZ-325 / AZ-323 bake them into the Manifest. Stream the C10 stdout/stderr lines back as DEBUG logs and parse the final `BuildReport` JSON document the C10 process emits on stdout; (6) aggregate into `CacheBuildReport`; (7) release the lockfile in `finally`. Wraps any underlying error from C11/C10/C7/C6 as `CacheBuildError` with a `remediation` attribute populated per `failure_phase`. Owns the operator-facing C12-IT-02 acceptance test contract. **Complexity**: 5 points **Dependencies**: AZ-326_c12_cli_app, AZ-327_c12_companion_bringup, AZ-316_c11_tile_downloader, AZ-325_c10_cache_provisioner, AZ-489_c12_flights_api_client (Flight resolve + bbox-from-waypoints + takeoff origin), AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module -**Component**: c12_operator_tooling (epic AZ-253 / E-C12) +**Component**: c12_operator_orchestrator (epic AZ-253 / E-C12) **Tracker**: AZ-328 **Epic**: AZ-253 (E-C12) @@ -13,7 +13,7 @@ - `_docs/02_document/contracts/c11_tilemanager/tile_downloader.md` — consumed: `fetch` API + `DownloadBatchReport` shape. - `_docs/02_document/contracts/c10_provisioning/cache_provisioner.md` — consumed: `build_cache_artifacts` API + `BuildReport` shape (this task invokes the contract over SSH; the contract values are passed back as a JSON document). -- `_docs/02_document/components/13_c12_operator_tooling/description.md` — § 1 (Coordinator), § 2 (`build_cache`, `CacheBuildReport`), § 5 (`CacheBuildError`), § 7 (lockfile), § 8 (depends on C10 + C11). +- `_docs/02_document/components/13_c12_operator_orchestrator/description.md` — § 1 (Coordinator), § 2 (`build_cache`, `CacheBuildReport`), § 5 (`CacheBuildError`), § 7 (lockfile), § 8 (depends on C10 + C11). - `_docs/02_document/contracts/shared_logging/log_record_schema.md` — INFO/WARN/ERROR + DEBUG log shapes (DEBUG is used for streamed C10 progress). - `_docs/_process_leftovers/2026-05-09_satellite-provider-design-tasks.md` — the parent-suite `satellite-provider` URL + auth surface this task wires through (informational, no direct dep). @@ -80,7 +80,7 @@ This task delivers the F1 orchestrator + the remote C10 invoker + the lockfile + 10. INFO log `kind="c12.build_cache.success"` with the aggregated counts (tiles_downloaded, engines_built, engines_reused, descriptors_generated). 11. Return `CacheBuildReport(outcome=success, failure_phase=none, download_report=..., build_report=..., failure_reason=None, wall_clock_s=...)`. 12. Lockfile released by `__exit__` of the `with` block. -- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py` extends T1's `OperatorToolServices` dataclass with a `build_cache_orchestrator: BuildCacheOrchestrator` field. The factory `build_build_cache_orchestrator(config, services) -> BuildCacheOrchestrator` constructs the lock factory, the remote C10 invoker, and pulls T1's `freshness_table` + T2's `companion_bringup` from the existing services dataclass. +- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py` extends T1's `OperatorOrchestratorServices` dataclass with a `build_cache_orchestrator: BuildCacheOrchestrator` field. The factory `build_build_cache_orchestrator(config, services) -> BuildCacheOrchestrator` constructs the lock factory, the remote C10 invoker, and pulls T1's `freshness_table` + T2's `companion_bringup` from the existing services dataclass. - T1's `cli.py` `build-cache` subcommand resolves `services.build_cache_orchestrator` and calls `.build_cache(request)`. Maps `CacheBuildError(failure_phase=download) → exit 20`; `CacheBuildError(failure_phase=build) → exit 21`; `BuildLockHeldError → exit 50`. ## Scope diff --git a/_docs/02_tasks/done/AZ-489_c12_flights_api_client.md b/_docs/02_tasks/done/AZ-489_c12_flights_api_client.md index d2e32e5..78e289a 100644 --- a/_docs/02_tasks/done/AZ-489_c12_flights_api_client.md +++ b/_docs/02_tasks/done/AZ-489_c12_flights_api_client.md @@ -2,17 +2,17 @@ **Task**: AZ-489_c12_flights_api_client **Name**: C12 FlightsApiClient — fetch Flight from suite flights service + offline JSON fallback -**Description**: Add a typed client module to C12 that fetches a parent-suite `Flight` (route + waypoints + altitudes) from the parent-suite `flights` REST service so C12 can derive the cache bbox and the takeoff origin directly from the operator-planned mission (ADR-010). The operator runs `operator-tool build-cache --flight-id `; C12 calls `GET /flights/{id}` and `GET /flights/{id}/waypoints`, parses into local pydantic DTOs (`FlightDto`, `WaypointDto`) mirroring `suite/flights/Database/Entities/{Flight,Waypoint}.cs`, computes the bbox as the envelope of waypoint lat/lon plus a configurable buffer (default 1 km, horizontal-distance — not degree-space — via `WgsConverter`), and exposes the first-ordered waypoint as the takeoff origin. An `--flight-file ` alternative reads the same DTO shape from a local JSON export so the workflow stays usable when the workstation has no path to the flights service. The client is read-only, raises typed errors for every documented failure path, redacts the auth token in all log output, and is consumed by AZ-326 (CLI flags) + AZ-328 (orchestrator phase 0). +**Description**: Add a typed client module to C12 that fetches a parent-suite `Flight` (route + waypoints + altitudes) from the parent-suite `flights` REST service so C12 can derive the cache bbox and the takeoff origin directly from the operator-planned mission (ADR-010). The operator runs `operator-orchestrator build-cache --flight-id `; C12 calls `GET /flights/{id}` and `GET /flights/{id}/waypoints`, parses into local pydantic DTOs (`FlightDto`, `WaypointDto`) mirroring `suite/flights/Database/Entities/{Flight,Waypoint}.cs`, computes the bbox as the envelope of waypoint lat/lon plus a configurable buffer (default 1 km, horizontal-distance — not degree-space — via `WgsConverter`), and exposes the first-ordered waypoint as the takeoff origin. An `--flight-file ` alternative reads the same DTO shape from a local JSON export so the workflow stays usable when the workstation has no path to the flights service. The client is read-only, raises typed errors for every documented failure path, redacts the auth token in all log output, and is consumed by AZ-326 (CLI flags) + AZ-328 (orchestrator phase 0). **Complexity**: 3 points **Dependencies**: AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module, AZ-279_wgs_converter (for the bbox buffer math) -**Component**: c12_operator_tooling (epic AZ-253 / E-C12) +**Component**: c12_operator_orchestrator (epic AZ-253 / E-C12) **Tracker**: AZ-489 **Epic**: AZ-253 (E-C12) ### Document Dependencies -- `_docs/02_document/contracts/c12_operator_tooling/flights_api_client.md` — produced by this task (frozen Protocol + DTOs + invariants + test cases). -- `_docs/02_document/components/13_c12_operator_tooling/description.md` — § 2 (FlightsApiClient interface), § 5 (httpx + pydantic dependencies). +- `_docs/02_document/contracts/c12_operator_orchestrator/flights_api_client.md` — produced by this task (frozen Protocol + DTOs + invariants + test cases). +- `_docs/02_document/components/13_c12_operator_orchestrator/description.md` — § 2 (FlightsApiClient interface), § 5 (httpx + pydantic dependencies). - `_docs/02_document/architecture.md` — ADR-010 (operator-planned mission as cold-start trust anchor). - Parent-suite reference (read-only): `suite/flights/Database/Entities/Flight.cs`, `suite/flights/Database/Entities/Waypoint.cs`, `suite/flights/Controllers/FlightsController.cs`. @@ -44,7 +44,7 @@ This task delivers the client + its frozen contract. It does NOT modify the CLI - Error hierarchy at `src/operator_tool/flights_api_errors.py`: - `FlightsApiError` (base) → `FlightsApiUnreachableError`, `FlightsApiAuthError`, `FlightNotFoundError`, `FlightsApiSchemaError`, `FlightFileNotFoundError`, `EmptyWaypointsError`, `WaypointSchemaError`. - Composition-root factory entry at `src/gps_denied_onboard/runtime_root/c12_factory.py`: - - Extend the `OperatorToolServices` dataclass with `flights_api_client: FlightsApiClient`. + - Extend the `OperatorOrchestratorServices` dataclass with `flights_api_client: FlightsApiClient`. - `build_flights_api_client(config) -> FlightsApiClient` constructs the httpx client with TLS verify on (no `verify=False`), default timeout `10.0 s`, and the project's `WgsConverter`. - Logging: - INFO on every successful fetch (`kind="c12.flights.fetch.success"`) with `flight_id`, `waypoint_count`, `bbox` summary. NO `auth_token` in any log line. diff --git a/_docs/02_tasks/todo/AZ-329_c12_post_landing_upload.md b/_docs/02_tasks/todo/AZ-329_c12_post_landing_upload.md index 560715e..a6ba95f 100644 --- a/_docs/02_tasks/todo/AZ-329_c12_post_landing_upload.md +++ b/_docs/02_tasks/todo/AZ-329_c12_post_landing_upload.md @@ -1,216 +1,217 @@ -# C12 Post-Landing Upload — `trigger_post_landing_upload` + FDR ON_GROUND Confirmation +# C12 Post-Landing Upload — `trigger_post_landing_upload` + FDR `flight_footer` Confirmation **Task**: AZ-329_c12_post_landing_upload **Name**: C12 Post-Landing Upload -**Description**: Implement `PostLandingUploadOrchestrator`, the C12 post-flight (F10) workflow that gates `C11.TileUploader.upload_pending_tiles` (AZ-319) on a confirmed-ON_GROUND signal from the post-flight FDR. `trigger_post_landing_upload(request: PostLandingUploadRequest) -> UploadBatchReport` does the following: (1) locate the FDR segments for the given `flight_id` under `config.c12.fdr_root` (segment layout: `//segment_.fdr` per the C13 conventions); (2) iterate the segments from newest to oldest, parsing records via AZ-272's `FdrRecord.parse(...)`; (3) collect all `state.tick` records carrying a `flight_state` payload field (or a dedicated `flight_state.tick` kind if the schema names it that way — defer to AZ-272's contract); (4) walking the collected records backwards from the most recent (chronologically), count contiguous `ON_GROUND` records and compute the contiguous ON_GROUND duration as `(latest_record.ts − first_consecutive_on_ground_record.ts)` seconds; (5) compare against `config.c12.upload_min_on_ground_s` (default 30 s per description.md C12-IT-03); (6) on confirmed ≥ threshold → construct a `FlightStateSignal(state=ON_GROUND, since_ts=)` and call `tile_uploader.upload_pending_tiles(flight_state=...)`; (7) on any refusal mode → raise `FlightStateNotConfirmedError(not_confirmed_reason=...)` with one of the four documented reason strings (`"never_landed"`, `"insufficient_duration: s < s"`, `"flight_id_not_found"`, `"fdr_unreadable: "`). Owns AC-8.4's defense-in-depth check on the operator-tooling side — the airborne C11 ALSO blocks via `UploadGateBlockedError` per AZ-319; this task is the operator-side gate that prevents the upload command from even being issued. Returns C11's `UploadBatchReport` unchanged on success. Logs every decision (INFO on confirmed; ERROR on each refusal mode) including the inferred contiguous ON_GROUND duration in seconds. +**Description**: Implement `PostLandingUploadOrchestrator`, the C12 post-flight (F10) workflow that gates `C11.TileUploader.upload_pending_tiles` (AZ-319) on the presence of a clean-shutdown `flight_footer` FDR record for `flight_id`. `trigger_post_landing_upload(request: PostLandingUploadRequest) -> UploadBatchReportCut` does the following: (1) resolve `//` and confirm the directory exists; (2) iterate the segment files from newest to oldest, streaming length-prefixed records via AZ-272's `FdrRecord.parse(...)`; (3) short-circuit on the first record whose `kind == "flight_footer"` (the C13 writer in AZ-292 emits exactly one such record per flight, on `close_flight()`); (4) inspect `payload["clean_shutdown"]` — `True` → the flight terminated gracefully → invoke `tile_uploader.upload_pending_tiles(UploadRequestCut(flight_id=..., batch_size=..., satellite_provider_url=...))` and return its `UploadBatchReportCut` unchanged; `False` → operator inspection required → refuse with `FlightStateNotConfirmedError("unclean_shutdown")`; (5) footer absent across every segment → power-loss truncation or mid-flight crash → refuse with `FlightStateNotConfirmedError("footer_missing")`; (6) FDR parse error mid-stream → refuse with `FlightStateNotConfirmedError("fdr_unreadable: ")`; (7) `//` does not exist → refuse with `FlightStateNotConfirmedError("flight_id_not_found")`. Owns AC-8.4's defense-in-depth check on the operator-orchestrator side — C11 is now a dumb pipe (the airborne internal gate was removed in batch 44 — see superseded AZ-317); this task is the only gate that prevents the upload command from being issued when the flight didn't terminate cleanly. Returns C11's `UploadBatchReport` (passthrough via the cut) on success. Logs every decision (INFO on confirmed; ERROR on each refusal mode); the `api_key` carried inside `PostLandingUploadRequest` is a plain `str` field but the orchestrator + CLI MUST redact it from every log line (matching the existing AZ-328 `BuildCacheRequest.api_key` pattern — `"api_key": "REDACTED"`). Introducing a Pydantic-backed `SecretStr` type would require adding `pydantic` as a runtime dependency, which the project explicitly avoids; the runtime-redaction contract is enforced by AC-8. + **Complexity**: 3 points -**Dependencies**: AZ-326_c12_cli_app, AZ-319_c11_tile_uploader, AZ-272_fdr_record_schema, AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module -**Component**: c12_operator_tooling (epic AZ-253 / E-C12) +**Dependencies**: AZ-326_c12_cli_app, AZ-319_c11_tile_uploader (post batch 44 gate removal), AZ-272_fdr_record_schema, AZ-292_c13_flight_header_footer, AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module +**Component**: c12_operator_orchestrator (epic AZ-253 / E-C12) **Tracker**: AZ-329 **Epic**: AZ-253 (E-C12) ### Document Dependencies -- `_docs/02_document/contracts/c11_tilemanager/tile_uploader.md` — consumed: `upload_pending_tiles` API + `UploadBatchReport` shape + `FlightStateSignal` DTO. -- `_docs/02_document/contracts/shared_fdr_client/fdr_record_schema.md` — consumed: `parse(buf: bytes) -> FdrRecord` + the `state.tick` / `flight_state.tick` kind shape (defer to the contract for the exact `kind` name and `flight_state` field). -- `_docs/02_document/components/13_c12_operator_tooling/description.md` — § 2 (`trigger_post_landing_upload` interface, `FlightStateNotConfirmedError`). -- `_docs/02_document/components/13_c12_operator_tooling/tests.md` — C12-IT-03 specifies the 30-s ON_GROUND threshold. -- `_docs/02_document/components/14_c13_fdr/description.md` — § 1 segment file layout (informational). +- `_docs/02_document/contracts/c11_tilemanager/tile_uploader.md` v2.0.0 — consumed: `upload_pending_tiles(UploadRequest) -> UploadBatchReport` API (post batch 44 — no `FlightStateSignal` parameter, no `confirm_flight_state` method). +- `_docs/02_document/contracts/shared_fdr_client/fdr_record_schema.md` — consumed: `parse(buf: bytes) -> FdrRecord` + the `flight_footer` kind shape (`flight_id`, `flight_ended_at_iso`, `clean_shutdown`, and the four AC-NEW-3 counters). +- `_docs/02_document/components/13_c12_operator_orchestrator/description.md` — § 2 (`trigger_post_landing_upload` interface, `FlightStateNotConfirmedError`). +- `_docs/02_document/components/13_c12_operator_orchestrator/tests.md` — C12-IT-03 specifies the `flight_footer`-based check. +- `_docs/02_document/components/14_c13_fdr/description.md` — § 1 segment file layout (informational) + § 2 `FlightFooter` shape (authoritative producer). ## Problem Without a real `PostLandingUploadOrchestrator`: -- F10 has no head — operators cannot trigger post-landing tile upload; AC-8.4 (mid-flight tile upload trigger, post-landing) collapses; the pending-upload journal in C6 grows unboundedly across flights. -- The operator-side ON_GROUND gate (defense-in-depth on top of C11's airborne gate) does not exist — operators can manually invoke `C11.TileUploader.upload_pending_tiles` with a fabricated `FlightStateSignal`, defeating the AC-NEW-7 / AC-8.4 architectural intent that mid-flight tiles only upload when the aircraft has landed. -- C12-IT-03 (`trigger_post_landing_upload` requires ≥ 30 s confirmed ON_GROUND in FDR) has no implementation. +- F10 has no head — operators cannot trigger post-landing tile upload; AC-8.4 collapses; C6's pending-upload journal grows unboundedly across flights. +- The operator-side gate (the *only* remaining gate after batch 44's removal of C11's internal `FlightStateGate`) does not exist — operators can manually invoke `C11.TileUploader.upload_pending_tiles(UploadRequest(...))` directly, defeating the AC-NEW-7 / AC-8.4 architectural intent that mid-flight tiles only upload after a clean landing. +- C12-IT-03 (`trigger_post_landing_upload` requires a `flight_footer` with `clean_shutdown=True`) has no implementation. - `FlightStateNotConfirmedError` is concept-only in description.md § 5 with no producer. - The CLI's `upload-pending` subcommand has nothing to delegate to. -- An incomplete flight log (FDR ends with `IN_FLIGHT` because the aircraft crashed or never landed) silently passes through to C11 if there's no operator-side gate; the airborne gate is the last line of defense and may itself be unavailable on the operator workstation. +- A truncated FDR (no footer; the aircraft crashed or lost power) would silently pass through to C11 if there were no operator-side gate. -This task delivers the operator-side gate. It does NOT own the actual upload (AZ-319), the FDR record schema (AZ-272), or the FDR write side (AZ-291..296) — it composes them. +This task delivers the operator-side gate. It does NOT own the actual upload (AZ-319), the FDR record schema (AZ-272), the FDR write side / footer producer (AZ-291..296, AZ-292) — it composes them. ## Outcome -- A `PostLandingUploadOrchestrator` class at `src/operator_tool/post_landing_upload.py`: - - Constructor: `__init__(self, *, tile_uploader: TileUploader, fdr_segment_reader: FdrSegmentReader, logger: Logger, clock: Clock, config: C12PostLandingConfig)`. - - `C12PostLandingConfig` (`@dataclass(frozen=True)`): `fdr_root: Path`, `upload_min_on_ground_s: float = 30.0`, `flight_state_record_kind: str = "state.tick"`, `flight_state_payload_field: str = "flight_state"`. +- A `PostLandingUploadOrchestrator` class at `src/gps_denied_onboard/components/c12_operator_orchestrator/post_landing_upload.py`: + - Constructor: `__init__(self, *, tile_uploader: TileUploaderCut, fdr_footer_reader: FdrFooterReader, logger: Logger, config: C12PostLandingConfig)`. + - `C12PostLandingConfig` (`@dataclass(frozen=True)`): `fdr_root: Path`. - Public method: `trigger_post_landing_upload(request: PostLandingUploadRequest) -> UploadBatchReport`. -- DTOs at `src/operator_tool/_types.py`: - - `PostLandingUploadRequest` (`@dataclass(frozen=True)`): `flight_id: str`. - - Reuses C11's `UploadBatchReport`. -- Errors at `src/operator_tool/errors.py`: - - `FlightStateNotConfirmedError(Exception)`: attributes `flight_id: str`, `not_confirmed_reason: str` (one of the four documented strings), `inferred_on_ground_duration_s: float | None` (populated when the reason is `insufficient_duration`), `remediation: str` (per-reason hint, e.g. for `flight_id_not_found`: "Verify the flight ID matches the FDR directory name; check `//`."). -- An `FdrSegmentReader` Protocol + `LocalFdrSegmentReader` concrete at `src/operator_tool/fdr_segment_reader.py`: - - `Protocol`: `iter_records_for_flight(flight_id: str, *, kind_filter: str | None = None) -> Iterator[FdrRecord]` — yields records ordered by `ts` ASCENDING; the orchestrator reverses on its own. `kind_filter` if non-None restricts to that record kind for efficiency. - - `LocalFdrSegmentReader.iter_records_for_flight(...)` — opens `//segment_*.fdr` files in numerical order, reads each as a stream of length-prefixed `FdrRecord` blobs (per AZ-272's serialisation), parses via `FdrRecord.parse(...)`, optionally filters by `kind`, yields one record at a time. Files are mmap'd or buffered-iterated so the operator workstation does not load multi-GB segments fully into memory. +- DTOs at `src/gps_denied_onboard/components/c12_operator_orchestrator/_types.py`: + - `PostLandingUploadRequest` (`@dataclass(frozen=True)`): `flight_id: UUID`, `satellite_provider_url: str`, `api_key: str`, `batch_size: int = 50`. The `api_key` field is plain `str` for consistency with `BuildCacheRequest`; redaction is a runtime guarantee enforced by AC-8 and the CLI's `_emit_invoked` redaction (matching the AZ-328 pattern). + - `UploadBatchReportCut` — local consumer-side AZ-507 Protocol mirroring C11's `UploadBatchReport` shape (no import from c11). Used only as the return-type annotation for `TileUploaderCut.upload_pending_tiles`. +- `TileUploaderCut` Protocol at `src/gps_denied_onboard/components/c12_operator_orchestrator/post_landing_upload.py` (or a sibling `_cuts.py`): `def upload_pending_tiles(self, request: UploadRequestCut) -> UploadBatchReportCut: ...`. `UploadRequestCut` mirrors C11's `UploadRequest(batch_size, satellite_provider_url, flight_id)`. This is the AZ-507 consumer-side cut; the composition root binds a real `HttpTileUploader` here, and the structural typing prevents a direct c11 import from c12. +- Errors at `src/gps_denied_onboard/components/c12_operator_orchestrator/errors.py`: + - `FlightStateNotConfirmedError(Exception)`: attributes `flight_id: str`, `not_confirmed_reason: Literal["flight_id_not_found", "footer_missing", "unclean_shutdown", "fdr_unreadable"]`, `detail: str` (for `unclean_shutdown` carries the four AC-NEW-3 counters; for `fdr_unreadable` carries the inner exception `repr`; empty string otherwise), `remediation: str` (per-reason hint). +- An `FdrFooterReader` Protocol + `LocalFdrFooterReader` concrete at `src/gps_denied_onboard/components/c12_operator_orchestrator/fdr_footer_reader.py`: + - `Protocol`: `read_footer(flight_id: UUID) -> FlightFooterRecord | None` — returns the `flight_footer` record's payload (as a typed `FlightFooterRecord` dataclass owned by this module — NOT C13's `FlightFooter` — preserving the c12↔c13 cut), or `None` if no footer record is found across any segment. + - `LocalFdrFooterReader.read_footer(flight_id)` — opens `//segment-NNNN.fdr` files (the C13 naming convention: hyphen separator, 4-digit zero-padded index — see `c13_fdr/writer.py::_segment_path`) in DESCENDING numerical order (newest first), streams length-prefixed `FdrRecord` blobs via `FdrRecord.parse(...)` (each frame is `uint32 LE length` + JSON body — see `c13_fdr/writer.py::_LENGTH_PREFIX`), returns the first one whose `kind == "flight_footer"`, or `None` if none found. Each segment is read with a buffered file iterator — NEVER fully `read()`-ed into memory. - On any I/O or parse error → raises `FdrUnreadableError(reason: str)` (a sibling helper exception caught by the orchestrator and rewrapped as `FlightStateNotConfirmedError("fdr_unreadable: ...")`). +- `FlightFooterRecord` (`@dataclass(frozen=True)`) at `_types.py`: `flight_id: UUID`, `flight_ended_at_iso: str`, `records_written: int`, `records_dropped_overrun: int`, `bytes_written: int`, `rollover_count: int`, `clean_shutdown: bool`. Built from `FdrRecord.payload` inside `LocalFdrFooterReader`; the orchestrator only reads `clean_shutdown` + the four counters (for `unclean_shutdown` log/error detail). - Method flow for `trigger_post_landing_upload`: - 1. `flight_dir = config.fdr_root / request.flight_id`. If `not flight_dir.exists()` → raise `FlightStateNotConfirmedError(flight_id, "flight_id_not_found", remediation="Verify // exists; check `config.c12.fdr_root`.")`. - 2. Collect all `flight_state` records: `records = list(fdr_segment_reader.iter_records_for_flight(request.flight_id, kind_filter=config.flight_state_record_kind))`. Catch `FdrUnreadableError` → raise `FlightStateNotConfirmedError(flight_id, f"fdr_unreadable: {e!r}", ...)`. - 3. If `not records` → raise `FlightStateNotConfirmedError(flight_id, "never_landed", remediation="No flight state records in FDR for this flight; check the flight produced state.tick records.")` (treat absence of any state record as never-landed since we have no positive ON_GROUND signal). - 4. Walk `records` backward from the last (most recent `ts`): - - `latest = records[-1]`. - - If `latest.payload[config.flight_state_payload_field] != "ON_GROUND"` → raise `FlightStateNotConfirmedError(flight_id, "never_landed", remediation="Most recent flight_state in FDR is not ON_GROUND; the flight may have ended in IN_FLIGHT (e.g. crash, log truncation).")`. - - Walk backward through `records[:-1]` while `record.payload[...] == "ON_GROUND"`; the first non-`ON_GROUND` (or the start of the list) bounds the contiguous ON_GROUND run. - - `since = first_contiguous_on_ground_record.ts`; `duration_s = (parse_iso(latest.ts) - parse_iso(since)).total_seconds()`. - 5. If `duration_s < config.upload_min_on_ground_s` → raise `FlightStateNotConfirmedError(flight_id, f"insufficient_duration: {duration_s:.1f}s < {config.upload_min_on_ground_s:.1f}s", inferred_on_ground_duration_s=duration_s, remediation="Wait for the aircraft to be confirmed ON_GROUND for the required duration, then re-run.")`. - 6. INFO log `kind="c12.upload.confirmed_on_ground"` with `flight_id`, `inferred_on_ground_duration_s`. - 7. Construct `flight_state = FlightStateSignal(state=ON_GROUND, since_ts=since)` (the DTO comes from C11 per AZ-319's contract). - 8. Call `report = tile_uploader.upload_pending_tiles(flight_state=flight_state)`. Propagate `UploadGateBlockedError` (defense-in-depth on the airborne side; this should never happen if step 6 confirmed; if it does, log ERROR and re-raise as-is). - 9. INFO log `kind="c12.upload.complete"` with `tiles_acked`, `tiles_rejected` from `report`. - 10. Return `report` unchanged. -- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py` extends T1's `OperatorToolServices` dataclass with `post_landing_upload_orchestrator: PostLandingUploadOrchestrator`. The factory `build_post_landing_upload_orchestrator(config, services) -> PostLandingUploadOrchestrator` constructs the `LocalFdrSegmentReader` over `config.c12.fdr_root` and pulls C11's `tile_uploader` from the wider service registry. -- T1's `cli.py` `upload-pending` subcommand resolves `services.post_landing_upload_orchestrator` and calls `.trigger_post_landing_upload(...)`. Maps `FlightStateNotConfirmedError → exit 30`; `UploadGateBlockedError → exit 31`. + 1. `flight_dir = config.fdr_root / str(request.flight_id)`. If `not flight_dir.exists()` → raise `FlightStateNotConfirmedError(flight_id=str(request.flight_id), not_confirmed_reason="flight_id_not_found", detail="", remediation="Verify // exists; check `config.c12_operator_orchestrator.fdr_root`.")`. ERROR log `kind="c12.upload.refused.flight_id_not_found"`. + 2. `footer = fdr_footer_reader.read_footer(request.flight_id)`. Catch `FdrUnreadableError` → raise `FlightStateNotConfirmedError(flight_id, "fdr_unreadable", detail=f"{e!r}", remediation="Inspect FDR segment files manually; the parser failed mid-stream.")`. ERROR log `kind="c12.upload.refused.fdr_unreadable"`. + 3. If `footer is None` → raise `FlightStateNotConfirmedError(flight_id, "footer_missing", detail="", remediation="No flight_footer record found in any segment — the flight likely terminated abnormally (power loss, crash, or close_flight() never ran). Inspect FDR manually; upload requires a clean shutdown.")`. ERROR log `kind="c12.upload.refused.footer_missing"`. + 4. If `footer.clean_shutdown is False` → raise `FlightStateNotConfirmedError(flight_id, "unclean_shutdown", detail=f"records_dropped_overrun={footer.records_dropped_overrun}, bytes_written={footer.bytes_written}", remediation="The flight footer reports an unclean shutdown. Operator must manually verify the flight outcome before authorising tile upload.")`. ERROR log `kind="c12.upload.refused.unclean_shutdown"` with the four counters in `kv`. + 5. INFO log `kind="c12.upload.confirmed_clean_shutdown"` with `flight_id`, `flight_ended_at_iso`, `records_written`. + 6. `inner_request = UploadRequestCut(batch_size=request.batch_size, satellite_provider_url=request.satellite_provider_url, flight_id=request.flight_id)`. `api_key` is not passed to C11 — C11 picks up the satellite-provider auth from its own configuration (per the AZ-319 contract); `api_key` here is for forward-compat with the F10 operator workflow that may sign the upload command itself. + 7. `report = tile_uploader.upload_pending_tiles(inner_request)`. Any exception from C11 propagates unchanged. + 8. INFO log `kind="c12.upload.complete"` with `outcome=report.outcome`, `tiles_acked=count(SUCCESS)`, `tiles_rejected=count(REJECTED)`, `batch_uuid=str(report.batch_uuid)`, `public_key_fingerprint=report.public_key_fingerprint`. + 9. Return `report` unchanged. +- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py`: + - `build_post_landing_upload_orchestrator(config: C12Config, *, tile_uploader: TileUploaderCut) -> PostLandingUploadOrchestrator` — constructs `LocalFdrFooterReader(config.post_landing.fdr_root)` + the orchestrator. + - Extends `OperatorOrchestratorServices` dataclass with `post_landing_upload_orchestrator: PostLandingUploadOrchestrator | None = None`. + - `build_operator_orchestrator(...)` aggregator: when a `tile_uploader` is passed in, build and wire the orchestrator; otherwise leave the field `None`. +- `cli.py` `upload-pending` subcommand resolves `services.post_landing_upload_orchestrator` and calls `.trigger_post_landing_upload(...)`. Maps `FlightStateNotConfirmedError → exit 30` (already defined as `EXIT_FLIGHT_STATE_NOT_CONFIRMED`); any other exception → exit 1. +- `__init__.py` re-exports `PostLandingUploadOrchestrator`, `PostLandingUploadRequest`, `FlightStateNotConfirmedError`, `FdrFooterReader`, `LocalFdrFooterReader`, `C12PostLandingConfig`. ## Scope ### Included - `PostLandingUploadOrchestrator` class with the single public method. -- `PostLandingUploadRequest` DTO. -- `FlightStateNotConfirmedError` with the four documented `not_confirmed_reason` strings + per-reason `remediation`. -- `FdrSegmentReader` Protocol. -- `LocalFdrSegmentReader` concrete reading on-disk FDR segments. +- `PostLandingUploadRequest` DTO (with `SecretStr` `api_key`). +- `FlightFooterRecord` DTO (local c12-owned mirror of C13's footer payload). +- `FlightStateNotConfirmedError` with the four `not_confirmed_reason` values + per-reason `detail` + `remediation`. +- `FdrFooterReader` Protocol. +- `LocalFdrFooterReader` concrete reading on-disk FDR segments newest-first. - `FdrUnreadableError` helper exception (caught and rewrapped at the orchestrator boundary). -- Composition-root factory. -- Wiring of T1's `upload-pending` subcommand to this service. -- Conformance unit tests using a fake `FdrSegmentReader` returning scripted record sequences for all 7 acceptance criteria. -- Two end-to-end integration tests using real FDR segment fixtures (one ending with confirmed ON_GROUND for 60 s, one ending with IN_FLIGHT) — these are the C12-IT-03 fixtures. +- `TileUploaderCut` + `UploadRequestCut` + `UploadBatchReportCut` AZ-507 consumer-side cuts (no direct c11 import from c12 source). +- Composition-root factory `build_post_landing_upload_orchestrator(...)` + `OperatorOrchestratorServices.post_landing_upload_orchestrator` field. +- Wiring of the `upload-pending` CLI subcommand. +- Conformance unit tests using a fake `FdrFooterReader` returning scripted footer records for AC-1..AC-8. +- Two integration tests using real FDR fixture files generated via the C13 `FileFdrWriter` (AC-9 clean shutdown, AC-10 unclean shutdown). ### Excluded -- The actual upload HTTP machinery (AZ-319). +- The actual upload HTTP machinery (AZ-319 / C11). - The FDR record schema or serialiser (AZ-272). -- The FDR write side / segment rotation (AZ-291..296). -- A "force-upload" override flag to bypass the gate — explicitly NOT supported (defeats the operator-side gate's purpose). -- Reading mid-flight tile snapshots from FDR — the upload itself reads tiles from C6 per AZ-319. +- The FDR write side / segment rotation / `flight_footer` producer (AZ-291..296, AZ-292). +- Any 30-second / contiguous-ON_GROUND threshold logic (REMOVED in batch 44 — the footer is the on-ground signal). +- Reading `state.tick` / `flight_state.tick` payloads (REMOVED in batch 44 — the footer's existence + `clean_shutdown` flag is the sole signal). +- A "force-upload" override — explicitly NOT supported. - Cross-flight aggregation — one `flight_id` per call. ## Acceptance Criteria -**AC-1: ≥ 30 s confirmed ON_GROUND → upload invoked** -Given a fake `FdrSegmentReader` returning 60 records, the last 60 of them with `flight_state=ON_GROUND` spanning 60 s of timestamps +**AC-1: `flight_footer` with `clean_shutdown=True` → upload invoked** +Given a fake `FdrFooterReader` returning `FlightFooterRecord(clean_shutdown=True, records_written=12345, ...)` When `trigger_post_landing_upload(request)` is called -Then `tile_uploader.upload_pending_tiles` is called exactly once with `flight_state.state=ON_GROUND` and `flight_state.since_ts` equal to the first contiguous ON_GROUND record's ts; the returned `UploadBatchReport` is the one C11 produced; ONE INFO log `kind="c12.upload.confirmed_on_ground"` with `inferred_on_ground_duration_s ≈ 60.0`; ONE INFO log `kind="c12.upload.complete"` +Then `tile_uploader.upload_pending_tiles` is called exactly once with `UploadRequestCut(flight_id=request.flight_id, batch_size=request.batch_size, satellite_provider_url=request.satellite_provider_url)`; the returned `UploadBatchReport` is the one C11 produced; ONE INFO log `kind="c12.upload.confirmed_clean_shutdown"`; ONE INFO log `kind="c12.upload.complete"` -**AC-2: Insufficient duration → `FlightStateNotConfirmedError("insufficient_duration: ...")`** -Given the FDR ends with 15 s contiguous ON_GROUND records (less than the 30 s threshold) +**AC-2: `flight_footer` absent → `FlightStateNotConfirmedError("footer_missing")`** +Given a fake `FdrFooterReader` returning `None` (no footer record found across any segment) When `trigger_post_landing_upload(request)` is called -Then `FlightStateNotConfirmedError(not_confirmed_reason="insufficient_duration: 15.0s < 30.0s", inferred_on_ground_duration_s≈15.0)` is raised; `tile_uploader.upload_pending_tiles` is NEVER called; ONE ERROR log `kind="c12.upload.refused.insufficient_duration"` +Then `FlightStateNotConfirmedError(not_confirmed_reason="footer_missing", detail="", remediation contains "No flight_footer record found")` is raised; `tile_uploader.upload_pending_tiles` is NEVER called; ONE ERROR log `kind="c12.upload.refused.footer_missing"` -**AC-3: Never-landed (last record is IN_FLIGHT) → `FlightStateNotConfirmedError("never_landed")`** -Given the FDR's most recent `state.tick` record has `flight_state=IN_FLIGHT` +**AC-3: `flight_footer` with `clean_shutdown=False` → `FlightStateNotConfirmedError("unclean_shutdown")`** +Given a fake `FdrFooterReader` returning `FlightFooterRecord(clean_shutdown=False, records_dropped_overrun=42, bytes_written=987654, ...)` When `trigger_post_landing_upload(request)` is called -Then `FlightStateNotConfirmedError(not_confirmed_reason="never_landed", inferred_on_ground_duration_s=None)` is raised; uploader NOT called; ONE ERROR log `kind="c12.upload.refused.never_landed"` +Then `FlightStateNotConfirmedError(not_confirmed_reason="unclean_shutdown", detail contains "records_dropped_overrun=42")` is raised; uploader NOT called; ONE ERROR log `kind="c12.upload.refused.unclean_shutdown"` containing all four AC-NEW-3 counters in `kv` -**AC-4: `flight_id` not found in FDR → `FlightStateNotConfirmedError("flight_id_not_found")`** -Given `//` does not exist +**AC-4: `//` does not exist → `FlightStateNotConfirmedError("flight_id_not_found")`** +Given `config.post_landing.fdr_root / str(request.flight_id)` does not exist When `trigger_post_landing_upload(request)` is called -Then `FlightStateNotConfirmedError(not_confirmed_reason="flight_id_not_found")` is raised; uploader NOT called; ONE ERROR log `kind="c12.upload.refused.flight_id_not_found"` +Then `FlightStateNotConfirmedError(not_confirmed_reason="flight_id_not_found")` is raised; the `FdrFooterReader` is NOT called; uploader NOT called; ONE ERROR log `kind="c12.upload.refused.flight_id_not_found"` -**AC-5: FDR unreadable → `FlightStateNotConfirmedError("fdr_unreadable: ")`** -Given the FDR segments exist but parsing raises `OSError("input/output error")` mid-stream +**AC-5: FDR unreadable → `FlightStateNotConfirmedError("fdr_unreadable")`** +Given the FDR segments exist but `LocalFdrFooterReader.read_footer` raises `FdrUnreadableError("OSError('input/output error')")` mid-stream When `trigger_post_landing_upload(request)` is called -Then `FlightStateNotConfirmedError(not_confirmed_reason=re.compile(r"^fdr_unreadable: .*OSError.*"))` is raised; uploader NOT called; ONE ERROR log `kind="c12.upload.refused.fdr_unreadable"` including the inner repr +Then `FlightStateNotConfirmedError(not_confirmed_reason="fdr_unreadable", detail matches r".*OSError.*")` is raised; uploader NOT called; ONE ERROR log `kind="c12.upload.refused.fdr_unreadable"` including the inner repr -**AC-6: Threshold is configurable** -Given `config.c12.upload_min_on_ground_s = 5.0` (override) and the FDR ends with 6 s contiguous ON_GROUND records -When `trigger_post_landing_upload(request)` is called -Then the call succeeds (uploader invoked); the threshold is read from config, NOT a hardcoded literal +**AC-6: Newest-segment-first short-circuit** +Given the FDR for `` has three segments (`segment-0000.fdr`, `segment-0001.fdr`, `segment-0002.fdr`) and the `flight_footer` record is in `segment-0002.fdr` (the most recent) +When `LocalFdrFooterReader.read_footer(flight_id)` is called +Then the reader opens `segment-0002.fdr` FIRST, finds the footer, and never opens `segment-0001.fdr` or `segment-0000.fdr` (assert via a spy on `open(...)` or a custom segment-iteration hook); the call returns in well under 100 ms even when the older segments are >100 MB each **AC-7: Returns C11's `UploadBatchReport` unchanged** -Given a successful upload returning `UploadBatchReport(tiles_acked=42, tiles_rejected=3, ...)` +Given a successful upload returning a `UploadBatchReport` with specific `batch_uuid`, `per_tile_status`, `outcome`, `public_key_fingerprint` values When the caller inspects the return value of `trigger_post_landing_upload` -Then it is byte-for-byte the `UploadBatchReport` C11 returned (same dataclass instance via passthrough); no field is added, removed, or renamed +Then it is the same object (passthrough) returned by `tile_uploader.upload_pending_tiles`; no field is mutated, added, removed, or renamed -**AC-8: Contiguous ON_GROUND counting starts from the most recent record only** -Given the FDR contains a sequence `IN_FLIGHT, ON_GROUND, IN_FLIGHT, ON_GROUND × 60s` (an aborted go-around landing) -When `trigger_post_landing_upload(request)` is called -Then the contiguous ON_GROUND block counted is the LAST one (60 s), not the earlier ON_GROUND record; the upload is invoked since 60 s ≥ 30 s +**AC-8: `api_key` is REDACTED in every log line** +Given `PostLandingUploadRequest(api_key="super-secret-token-123", ...)` and an end-to-end run through every refusal mode + the success path +When the log records are inspected (via `caplog` capture) +Then NO log record's `msg`, `kv`, `extra`, or any string field contains the substring `"super-secret-token-123"`; the CLI's `_emit_invoked` writes `"api_key": "REDACTED"` (matching the AZ-328 `BuildCacheRequest` pattern); the orchestrator never includes `api_key` in any log payload -**AC-9: Empty `flight_state` records → `never_landed`** -Given `iter_records_for_flight(...)` yields zero records (no `state.tick` records ever emitted) -When `trigger_post_landing_upload(request)` is called -Then `FlightStateNotConfirmedError(not_confirmed_reason="never_landed")` is raised (treated as "we have no positive ON_GROUND signal") +**AC-9: Real FDR fixture C12-IT-03(a) (clean-shutdown footer) → upload invoked** +Given an FDR fixture written by the C13 `FileFdrWriter`'s `close_flight()` path (which always sets `clean_shutdown=True` in the current AZ-292 implementation) at `tests/fixtures/c12_operator_orchestrator/fdr/clean_shutdown//segment-NNNN.fdr` +When `trigger_post_landing_upload(PostLandingUploadRequest(flight_id=, ...))` is called against a `LocalFdrFooterReader` over the fixture and a fake `TileUploaderCut` that records the call +Then the upload is invoked exactly once with `flight_id=`; the fake's recorded `UploadBatchReport` is returned unchanged -**AC-10: Real FDR fixture C12-IT-03(a) (60 s confirmed) → upload invoked** -Given the C12-IT-03 fixture FDR with confirmed ON_GROUND for 60 s -When `trigger_post_landing_upload(request)` is called against the LocalFdrSegmentReader on the fixture -Then the upload is invoked; the returned `UploadBatchReport` matches the fixture's expected counts - -**AC-11: Real FDR fixture C12-IT-03(b) (IN_FLIGHT, incomplete log) → refused** -Given the C12-IT-03 fixture FDR ending with IN_FLIGHT (truncated) -When `trigger_post_landing_upload(request)` is called against the LocalFdrSegmentReader on the fixture -Then `FlightStateNotConfirmedError(not_confirmed_reason="never_landed")` is raised; the upload is NOT invoked +**AC-10: Real FDR fixture C12-IT-03(b) (no-footer truncation) → refused** +Given an FDR fixture WITHOUT a `flight_footer` record (simulate truncation by writing segments via the writer thread and forcibly terminating before `close_flight()` runs — i.e. drop the last segment after the writer's `close_flight()` would have appended the footer record) +When `trigger_post_landing_upload(...)` is called against a `LocalFdrFooterReader` over this fixture +Then `FlightStateNotConfirmedError(not_confirmed_reason="footer_missing")` is raised; the upload is NOT invoked ## Non-Functional Requirements **Performance** -- For an 8-hour flight (≤ 64 GB FDR per AC-NEW-3) the orchestrator's read of `state.tick` records completes in ≤ 30 s wall-clock on a developer laptop with NVMe (the records are sparse — `state.tick` is one of many record kinds; the `kind_filter` argument lets the reader skip non-state records cheaply). -- Memory peak ≤ 200 MB even with multi-GB FDR segments — `LocalFdrSegmentReader` is a streaming generator, NOT a list-in-memory. +- `LocalFdrFooterReader.read_footer(flight_id)` completes in ≤ 1 s wall-clock on a developer laptop with NVMe even when the flight's FDR is 64 GB across many segments — the newest-segment-first short-circuit means a clean-shutdown flight reads only the tail of the last segment. +- Memory peak ≤ 50 MB even with multi-GB segments — `LocalFdrFooterReader` is a streaming reader: opens one segment at a time, reads length-prefixed blobs in a bounded buffer, releases the file handle before opening the next. **Compatibility** - AZ-272's `FdrRecord.parse` API is the only parser path; this task does NOT re-implement record parsing. -- C11's `FlightStateSignal` DTO is consumed unchanged; this task does NOT redefine it. +- C13's `flight_footer` record kind + payload shape (AZ-292) is consumed via the schema in `KNOWN_PAYLOAD_KEYS`; this task does NOT redefine the payload keys. +- `C12.PostLandingUploadOrchestrator` does NOT import from `c11_tile_manager`; the AZ-507 consumer-side cuts (`TileUploaderCut`, `UploadRequestCut`, `UploadBatchReportCut`) are the only contract. **Reliability** -- Catches and rewraps the four refusal modes deterministically — operators can script against the four documented `not_confirmed_reason` prefix strings. +- Catches and rewraps the four refusal modes deterministically — operators can script against the four documented `not_confirmed_reason` values (`flight_id_not_found`, `footer_missing`, `unclean_shutdown`, `fdr_unreadable`) which form a closed `Literal` type. - Streaming I/O on FDR segments — multi-GB segments do not blow memory. -- The threshold default (30.0 s) matches description.md C12-IT-03 exactly. +- No background threads, no global state, no caching — every call re-reads the FDR. +- `api_key` is `SecretStr` — the type system prevents accidental string concatenation into log messages. ## Unit Tests | AC Ref | What to Test | Required Outcome | |--------|-------------|-----------------| -| AC-1 | Fake reader with 60 ON_GROUND records spanning 60 s | Uploader called once, INFO logs, returns `UploadBatchReport` | -| AC-2 | Fake reader with 15 s ON_GROUND tail | `FlightStateNotConfirmedError("insufficient_duration: 15.0s < 30.0s")` | -| AC-3 | Fake reader whose last record is IN_FLIGHT | `FlightStateNotConfirmedError("never_landed")` | -| AC-4 | Path doesn't exist | `FlightStateNotConfirmedError("flight_id_not_found")` | -| AC-5 | Fake reader raises `FdrUnreadableError("OSError(...)")` | `FlightStateNotConfirmedError(re.match("^fdr_unreadable: .*"))` | -| AC-6 | Override `upload_min_on_ground_s=5.0` + 6 s ON_GROUND | Upload invoked | -| AC-7 | Successful upload, inspect return | Same `UploadBatchReport` instance/fields | -| AC-8 | Sequence with go-around (IN_FLIGHT in middle) | Contiguous count is the LAST run only | -| AC-9 | Empty `iter_records_for_flight` | `FlightStateNotConfirmedError("never_landed")` | -| AC-10 | C12-IT-03(a) fixture | Upload invoked | -| AC-11 | C12-IT-03(b) fixture | `FlightStateNotConfirmedError("never_landed")` | -| NFR-perf-streaming | Microbench `LocalFdrSegmentReader` over 1 GB synthetic segment | Memory peak ≤ 200 MB; parse rate ≥ 100 MB/s | +| AC-1 | Fake reader returns `clean_shutdown=True` | Uploader called once, INFO logs, returns `UploadBatchReport` | +| AC-2 | Fake reader returns `None` | `FlightStateNotConfirmedError("footer_missing")` | +| AC-3 | Fake reader returns `clean_shutdown=False` | `FlightStateNotConfirmedError("unclean_shutdown")` with counters in `detail` + log `kv` | +| AC-4 | `//` missing | `FlightStateNotConfirmedError("flight_id_not_found")` | +| AC-5 | Fake reader raises `FdrUnreadableError("OSError(...)")` | `FlightStateNotConfirmedError("fdr_unreadable")` w/ inner repr | +| AC-6 | Three-segment fixture, footer in newest | `LocalFdrFooterReader` opens only the newest segment | +| AC-7 | Success path; inspect return | Same `UploadBatchReport` instance | +| AC-8 | `caplog` capture across every code path | `api_key.get_secret_value()` never appears in any log | +| AC-9 | C12-IT-03(a) fixture (writer-produced clean footer) | Upload invoked | +| AC-10 | C12-IT-03(b) fixture (truncated; no footer) | `FlightStateNotConfirmedError("footer_missing")` | +| NFR-perf-streaming | Microbench `LocalFdrFooterReader` over a 1 GB synthetic segment with footer at the end | Memory peak ≤ 50 MB; wall-clock ≤ 1 s | ## Constraints -- The four `not_confirmed_reason` strings (`"never_landed"`, `"insufficient_duration: ..."`, `"flight_id_not_found"`, `"fdr_unreadable: ..."`) are a closed contract — adding a new value requires Plan-cycle approval (operators script against these prefixes). -- The threshold default 30.0 s matches description.md C12-IT-03 EXACTLY; changing it requires a spec amendment, not just a config change. -- The "contiguous ON_GROUND from most recent only" semantic (AC-8) is non-negotiable — counting the union of all ON_GROUND windows would defeat the gate by allowing an aborted-go-around aircraft to qualify based on the brief earlier landing. +- The four `not_confirmed_reason` values form a closed `Literal["flight_id_not_found", "footer_missing", "unclean_shutdown", "fdr_unreadable"]` type — adding a new value requires Plan-cycle approval (operators script against these values). - A "force-upload" override is explicitly NOT supported — operators who legitimately need to upload after a non-conforming flight must use a separate forensic path (out of scope this cycle). -- `LocalFdrSegmentReader` MUST stream; loading a multi-GB segment fully into memory is a NFR violation (NFR-perf-streaming). -- C11's `FlightStateSignal` DTO is the source of truth for the gate signal — this task does NOT define a parallel C12-internal `FlightStateSignal`. -- The threshold is a `float`; comparison uses `>=` (so exactly 30.0 s qualifies). +- `LocalFdrFooterReader` MUST stream and MUST iterate segments newest-first; loading any segment fully into memory is a NFR violation, and iterating oldest-first defeats AC-6's short-circuit. +- C13's `flight_footer` kind + payload schema (`KNOWN_PAYLOAD_KEYS["flight_footer"]`) is the source of truth — this task does NOT duplicate the schema; the local `FlightFooterRecord` dataclass extracts only the fields the orchestrator inspects. +- `api_key` is plain `str` (matching `BuildCacheRequest.api_key`); redaction is a runtime guarantee enforced by AC-8 (caught by `caplog` substring assertion). The CLI's `_emit_invoked` writes `"REDACTED"` and the orchestrator never includes `api_key` in any log payload. +- C12 does NOT import C11 directly — the AZ-507 consumer-side cuts pattern is enforced (the linter / import-cycle check should fail if `c12_operator_orchestrator/*.py` adds `from gps_denied_onboard.components.c11_tile_manager import ...`). +- The orchestrator does NOT consult any `state.tick` / `flight_state.tick` payloads — those are out of scope post batch 44. ## Risks & Mitigation -**Risk 1: AZ-272's record schema names the field something other than `flight_state`** -- *Risk*: AZ-272's contract may use `state` or `flight.state` instead of `flight_state`; this task hardcodes the field name in `config.c12.flight_state_payload_field`. -- *Mitigation*: The field name is a config knob (default `"flight_state"`); during integration with AZ-272, the default is updated to match AZ-272's actual contract. Tests use the default; integration tests against real FDR fixtures catch a mismatch immediately. +**Risk 1: C13 writes the footer to a segment that's not the most recent on disk** +- *Risk*: If `close_flight()` triggers a rollover concurrently, the footer might land in `segment_NNN+1.fdr` while older `segment_NNN.fdr` files are still on disk. The reader must still iterate newest-first by integer segment index, not by mtime, to correctly find the footer. +- *Mitigation*: `LocalFdrFooterReader` sorts segments by the integer `NNN` in `segment_.fdr` (descending), not by filesystem mtime. AC-6 covers the multi-segment case directly. Document the segment-naming dependency on `_docs/02_document/components/14_c13_fdr/description.md` § 1. -**Risk 2: The aircraft logs ON_GROUND briefly during taxi before takeoff** -- *Risk*: The flight starts ON_GROUND, transitions to IN_FLIGHT, lands ON_GROUND again. The "contiguous from most recent" semantic correctly handles this — but if the FDR is truncated mid-flight, the most recent record might be from the taxi phase, falsely suggesting a landed flight. -- *Mitigation*: The truncation case is captured by AC-3 / AC-11 — a truncated log ending in IN_FLIGHT correctly refuses. A truncated log ending in the early ON_GROUND taxi phase is indistinguishable from a real landing, but this is an FDR integrity concern out of scope; in practice the FDR writes are continuous. +**Risk 2: A future cycle introduces additional record kinds at the tail (e.g. `flight_audit`)** +- *Risk*: A new tail record kind could push the `flight_footer` deeper into the segment, increasing read latency. Currently the footer is the LAST record before file close, but the contract doesn't forbid later additions. +- *Mitigation*: The streaming reader scans the entire newest segment if needed; AC-6 only asserts "doesn't open older segments", not "reads only the last N bytes". A future cycle that adds tail records would still satisfy AC-6. -**Risk 3: FDR segment file naming convention drift** -- *Risk*: C13 (AZ-291..296) may name segments differently than `segment_.fdr`. -- *Mitigation*: The naming pattern is captured in `LocalFdrSegmentReader` with a `glob_pattern` constructor parameter (default `segment_*.fdr`); update the default if AZ-291 picks a different name. Tests cover both patterns. +**Risk 3: The footer's `flight_id` UUID doesn't match the directory name** +- *Risk*: An operator could rename the flight directory; the reader would still find a footer but its `flight_id` would mismatch. +- *Mitigation*: `LocalFdrFooterReader.read_footer(flight_id)` asserts `footer.flight_id == flight_id` and treats a mismatch as `FdrUnreadableError(f"footer flight_id mismatch: footer={footer.flight_id}, requested={flight_id}")`. The orchestrator rewraps as `FlightStateNotConfirmedError("fdr_unreadable")`. -**Risk 4: `parse_iso` timezone handling** -- *Risk*: Two records with the same wall-clock time but different timezones produce a wrong duration calculation. -- *Mitigation*: AZ-272's contract specifies all timestamps are ISO 8601 UTC microseconds; this task asserts UTC at parse time and raises `FdrUnreadableError("non-UTC timestamp in record")` otherwise. Defense-in-depth. +**Risk 4: A future cycle changes the `clean_shutdown` flag semantics** +- *Risk*: AZ-292 currently hardcodes `clean_shutdown=True` in `close_flight()`; a future cycle might emit `False` for graceful shutdowns that nonetheless lost some records. +- *Mitigation*: AC-3 already covers `clean_shutdown=False` → refused. The orchestrator does NOT interpret the four counters — operators do. If a future cycle wants to allow upload despite `clean_shutdown=False` under certain counter thresholds, that's a Plan-cycle change to this task. -**Risk 5: A future cycle adds a third flight state value (e.g. `EMERGENCY`)** -- *Risk*: The contiguous-counting code treats anything other than `ON_GROUND` as breaking the run; a new `EMERGENCY` value during landing rollout could shorten the inferred duration spuriously. -- *Mitigation*: Acceptable for this cycle — emergency states should not allow upload anyway. A future cycle that introduces such states must update this task's logic explicitly via a Plan-cycle change. +**Risk 5: Symlinks under `//`** +- *Risk*: An operator could symlink to a different flight's segments; the reader would still find a footer but it would belong to a different flight. +- *Mitigation*: Same as Risk 3 — the `flight_id` assertion catches it. Document that `` is operator-trusted territory; symlink escape is out of scope. ## Runtime Completeness -- **Named capability**: post-flight ON_GROUND-gated upload trigger per description.md § 2 (`trigger_post_landing_upload`) + AC-8.4 + C12-IT-03. -- **Production code that must exist**: real `PostLandingUploadOrchestrator` consuming real `TileUploader` (AZ-319) + real `LocalFdrSegmentReader` reading real on-disk FDR segments + real `FdrRecord.parse` (AZ-272). -- **Allowed external stubs**: tests MAY use fakes for `FdrSegmentReader` and `TileUploader`; the C12-IT-03 integration tests use real FDR fixture files + a fake `TileUploader` that records the call (no real network). -- **Unacceptable substitutes**: in-memory FDR (defeats the streaming guarantee NFR); a "force-upload" override (defeats the gate); shelling out to `cat ` instead of using `FdrRecord.parse` (no schema validation, no forward-compat); reading the FDR via the producer-side ring buffer (wrong API; ring buffer is for live producers, not post-flight reads). +- **Named capability**: post-flight clean-shutdown-gated upload trigger per description.md § 2 (`trigger_post_landing_upload`) + AC-8.4 + C12-IT-03. +- **Production code that must exist**: real `PostLandingUploadOrchestrator` consuming a real `HttpTileUploader` (AZ-319) via the `TileUploaderCut` Protocol + real `LocalFdrFooterReader` reading real on-disk FDR segments + real `FdrRecord.parse` (AZ-272). +- **Allowed external stubs**: tests MAY use fakes for `FdrFooterReader` and `TileUploaderCut`; the C12-IT-03 integration tests use real FDR fixture files (produced by C13's `FileFdrWriter`) + a fake `TileUploaderCut` that records the call (no real network). +- **Unacceptable substitutes**: in-memory FDR (defeats the streaming guarantee NFR); a "force-upload" override (defeats the gate); shelling out to `cat ` instead of using `FdrRecord.parse` (no schema validation, no forward-compat); reading the FDR via the producer-side ring buffer (wrong API; ring buffer is for live producers, not post-flight reads); importing `c11_tile_manager` directly from c12 source (violates AZ-507 consumer-side cuts). diff --git a/_docs/02_tasks/todo/AZ-330_c12_operator_reloc_service.md b/_docs/02_tasks/todo/AZ-330_c12_operator_reloc_service.md index 832e902..385eadc 100644 --- a/_docs/02_tasks/todo/AZ-330_c12_operator_reloc_service.md +++ b/_docs/02_tasks/todo/AZ-330_c12_operator_reloc_service.md @@ -2,19 +2,19 @@ **Task**: AZ-330_c12_operator_reloc_service **Name**: C12 OperatorReLocService -**Description**: Implement `OperatorReLocService`, the C12 operator-side of AC-3.4 (operator-relocalization on visual loss; the SUT requests a position hint from the operator after losing satellite anchoring; the operator confirms a candidate; the system re-anchors). Owns: (a) the `ReLocHint` DTO (`approximate_position_wgs84: LatLonAlt`, `confidence_radius_m: float`, `reason: str`) per description.md § 2; (b) the `OperatorCommandTransport` Protocol that E-C8 (a future task in AZ-261) will implement against pymavlink for the actual GCS-link MAVLink encoding + transmission; (c) the `request_reloc(reloc_hint: ReLocHint) -> None` public method that validates the hint at the C12 boundary, calls `transport.send_reloc_hint(...)`, catches the transport's `GcsLinkError` and re-raises with C12-specific context (operator action label, monotonic timestamp, hint summary as a redacted log line), emits an FDR record `kind="c12.reloc.requested"` via the AZ-273 FDR client so the post-flight log carries the operator's action chronologically, and writes an INFO log on success / ERROR log on failure. Best-effort semantics per description.md § 7 — if the GCS link is degraded the operator may need to re-issue manually; this task does NOT auto-retry. Publishes the Protocol contract at `_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md` so a future E-C8 task implements the same shape against pymavlink without re-negotiating fields. The pattern matches AZ-322's `BackboneEmbedder` Protocol (C10 owns the Protocol; C2 implements it later). +**Description**: Implement `OperatorReLocService`, the C12 operator-side of AC-3.4 (operator-relocalization on visual loss; the SUT requests a position hint from the operator after losing satellite anchoring; the operator confirms a candidate; the system re-anchors). Owns: (a) the `ReLocHint` DTO (`approximate_position_wgs84: LatLonAlt`, `confidence_radius_m: float`, `reason: str`) per description.md § 2; (b) the `OperatorCommandTransport` Protocol that E-C8 (a future task in AZ-261) will implement against pymavlink for the actual GCS-link MAVLink encoding + transmission; (c) the `request_reloc(reloc_hint: ReLocHint) -> None` public method that validates the hint at the C12 boundary, calls `transport.send_reloc_hint(...)`, catches the transport's `GcsLinkError` and re-raises with C12-specific context (operator action label, monotonic timestamp, hint summary as a redacted log line), emits an FDR record `kind="c12.reloc.requested"` via the AZ-273 FDR client so the post-flight log carries the operator's action chronologically, and writes an INFO log on success / ERROR log on failure. Best-effort semantics per description.md § 7 — if the GCS link is degraded the operator may need to re-issue manually; this task does NOT auto-retry. Publishes the Protocol contract at `_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md` so a future E-C8 task implements the same shape against pymavlink without re-negotiating fields. The pattern matches AZ-322's `BackboneEmbedder` Protocol (C10 owns the Protocol; C2 implements it later). **Complexity**: 3 points **Dependencies**: AZ-326_c12_cli_app, AZ-273_fdr_client_ringbuf, AZ-263_initial_structure, AZ-269_config_loader, AZ-266_log_module -**Component**: c12_operator_tooling (epic AZ-253 / E-C12) +**Component**: c12_operator_orchestrator (epic AZ-253 / E-C12) **Tracker**: AZ-330 **Epic**: AZ-253 (E-C12) ### Document Dependencies -- `_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md` — produced by this task (frozen Protocol + DTO shape, invariants, test cases for E-C8 to implement against). +- `_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md` — produced by this task (frozen Protocol + DTO shape, invariants, test cases for E-C8 to implement against). - `_docs/02_document/contracts/shared_fdr_client/fdr_record_schema.md` — consumed: the `c12.reloc.requested` record envelope. -- `_docs/02_document/components/13_c12_operator_tooling/description.md` — § 2 (`OperatorReLocService` interface, `ReLocHint` DTO), § 5 (`GcsLinkError` best-effort), § 7 (best-effort semantics; operator may re-issue). -- `_docs/02_document/components/13_c12_operator_tooling/tests.md` — C12-IT-01 (operator re-loc workflow returns SUT to satellite-anchored ≤ 30 s). +- `_docs/02_document/components/13_c12_operator_orchestrator/description.md` — § 2 (`OperatorReLocService` interface, `ReLocHint` DTO), § 5 (`GcsLinkError` best-effort), § 7 (best-effort semantics; operator may re-issue). +- `_docs/02_document/components/13_c12_operator_orchestrator/tests.md` — C12-IT-01 (operator re-loc workflow returns SUT to satellite-anchored ≤ 30 s). ## Problem @@ -58,8 +58,8 @@ This task delivers the C12 service surface + the Protocol contract + the FDR sid - ERROR log `kind="c12.reloc.failed"` with the redacted summary + `e.reason`. - `fdr_client.enqueue(FdrRecord(kind="c12.reloc.requested", payload={"hint": , "outcome": "failed", "failure_reason": e.reason, "ts_monotonic": clock.monotonic()}))` — the FDR record carries BOTH the attempt and the failure so the post-flight log shows the operator tried. - Re-raise `GcsLinkError(reason=f"C12 reloc-confirm: {e.reason}", wrapped_exception_repr=repr(e), remediation=e.remediation)` — wrap with C12 prefix in `reason`. -- The Protocol contract published at `_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md` per `templates/api-contract.md`. Includes Shape, Invariants, Non-Goals, Versioning Rules, and at least 3 Test Cases that E-C8's implementer can run against `MavlinkOperatorCommandTransport`. -- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py` extends T1's `OperatorToolServices` dataclass with `operator_reloc_service: OperatorReLocService`. The factory `build_operator_reloc_service(config, services) -> OperatorReLocService` constructs the service; the `OperatorCommandTransport` is resolved from a wider service registry that includes E-C8's `MavlinkOperatorCommandTransport` (or a fake `LoggingOnlyOperatorCommandTransport` until E-C8 is implemented — fake declared in tests, NOT in production wiring). +- The Protocol contract published at `_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md` per `templates/api-contract.md`. Includes Shape, Invariants, Non-Goals, Versioning Rules, and at least 3 Test Cases that E-C8's implementer can run against `MavlinkOperatorCommandTransport`. +- Composition-root factory at `src/gps_denied_onboard/runtime_root/c12_factory.py` extends T1's `OperatorOrchestratorServices` dataclass with `operator_reloc_service: OperatorReLocService`. The factory `build_operator_reloc_service(config, services) -> OperatorReLocService` constructs the service; the `OperatorCommandTransport` is resolved from a wider service registry that includes E-C8's `MavlinkOperatorCommandTransport` (or a fake `LoggingOnlyOperatorCommandTransport` until E-C8 is implemented — fake declared in tests, NOT in production wiring). - T1's `cli.py` `reloc-confirm` subcommand resolves `services.operator_reloc_service` and calls `.request_reloc(...)`. The CLI subcommand parses CLI flags `--lat`, `--lon`, `--alt`, `--radius`, `--reason` into a `ReLocHint`. Maps `GcsLinkError → exit 40`; `ValueError → exit 2 (usage)`. ## Scope @@ -70,7 +70,7 @@ This task delivers the C12 service surface + the Protocol contract + the FDR sid - `LatLonAlt` and `ReLocHint` DTOs (or import from `shared_helpers` if WgsConverter already defined `LatLonAlt`). - `OperatorCommandTransport` Protocol. - `GcsLinkError` error type with `reason`, `wrapped_exception_repr`, `remediation`. -- The Protocol contract document at `_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md`. +- The Protocol contract document at `_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md`. - FDR record emission via `fdr_client.enqueue` (both success and failure cases). - Composition-root factory. - Wiring of T1's `reloc-confirm` subcommand to this service. @@ -108,7 +108,7 @@ When `request_reloc(hint)` is called Then the transport's `send_reloc_hint` receives the hint with `reason` byte-for-byte equal to the input (no truncation, no normalization); the FDR record's `payload.hint.reason` is the same; the INFO log truncates the displayed reason to 200 chars (display-only) but the underlying transport call is unmodified **AC-5: Protocol contract document exists with the exact method signature** -Given the published contract at `_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md` +Given the published contract at `_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md` When E-C8's implementer reads the contract to build `MavlinkOperatorCommandTransport` Then the contract specifies the exact Protocol shape (`def send_reloc_hint(self, hint: ReLocHint) -> None`), the `ReLocHint` field shape, the documented `GcsLinkError` raise behaviour, the Versioning Rules, and at least 3 Test Cases @@ -133,7 +133,7 @@ When `request_reloc(hint)` is called Then the INFO log line shows `position_lat: 49.99877` and `position_lon: 36.12346` (rounded to 5 decimals); the underlying transport receives the full-precision value (no rounding before transport) **AC-10: Composition-root factory does not eager-construct the transport** -Given the operator-tool starts up (T1's `cli.py` lazily resolves services) +Given the operator-orchestrator starts up (T1's `cli.py` lazily resolves services) When the operator does NOT use the `reloc-confirm` subcommand in this session Then `OperatorCommandTransport` is NEVER instantiated (verifiable via spy on the factory); pymavlink is NEVER imported (NFR-perf-cold-start from T1 holds) @@ -202,4 +202,4 @@ Then `OperatorCommandTransport` is NEVER instantiated (verifiable via spy on the ## Contract -This task produces/implements the contract at `_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md`. Consumers (specifically the future E-C8 task implementing `MavlinkOperatorCommandTransport`) MUST read that file — not this task spec — to discover the interface. +This task produces/implements the contract at `_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md`. Consumers (specifically the future E-C8 task implementing `MavlinkOperatorCommandTransport`) MUST read that file — not this task spec — to discover the interface. diff --git a/_docs/02_tasks/todo/AZ-401_replay_compose.md b/_docs/02_tasks/todo/AZ-401_replay_compose.md index e009281..94e10bb 100644 --- a/_docs/02_tasks/todo/AZ-401_replay_compose.md +++ b/_docs/02_tasks/todo/AZ-401_replay_compose.md @@ -66,7 +66,7 @@ Without this task, the replay-only strategies (FrameSource + Clock + TlogReplayF **AC-7: Composition uses Public APIs only** — assert that `compose_replay` imports ONLY `__init__.py` re-exports of each component (per `module-layout.md` Layer-3 / Layer-4 rules). CI-style check via AST scan in the unit test. -**AC-8: No C6/C10/C11/C12 imports** — assert that `compose_replay` does NOT import any symbol from `components.c6_tile_cache`, `components.c10_provisioning`, `components.c11_tilemanager`, `components.c12_operator_tooling` (per epic scope). +**AC-8: No C6/C10/C11/C12 imports** — assert that `compose_replay` does NOT import any symbol from `components.c6_tile_cache`, `components.c10_provisioning`, `components.c11_tilemanager`, `components.c12_operator_orchestrator` (per epic scope). **AC-9: Configuration + calibration loading** — `compose_replay(config_with_invalid_calib_path)` → `ReplayCompositionError("camera-calibration not found at ...")`. diff --git a/_docs/02_tasks/todo/AZ-403_replay_dockerfile_ci.md b/_docs/02_tasks/todo/AZ-403_replay_dockerfile_ci.md index e75c47c..eac1f78 100644 --- a/_docs/02_tasks/todo/AZ-403_replay_dockerfile_ci.md +++ b/_docs/02_tasks/todo/AZ-403_replay_dockerfile_ci.md @@ -2,7 +2,7 @@ **Task**: AZ-403_replay_dockerfile_ci **Name**: `gps-denied-replay-cli` Dockerfile + GitHub Actions matrix entry + SBOM diff (excludes C6/C10/C11/C12) -**Description**: Add the fourth Docker image `gps-denied-replay-cli`: multi-stage build (Python + C1–C5 + cpp/* + replay strategies; NO C6/C10/C11/C12; NO HTTP server). Add a GitHub Actions matrix entry building and pushing this image alongside the existing 3 images (live / research / operator). Add an **SBOM diff CI step** that builds the SBOM (via `syft` or the project's existing SBOM tooling), parses it, and asserts the absence of `c6_tile_cache`, `c10_provisioning`, `c11_tilemanager`, `c12_operator_tooling` packages — verifies AC-4 of the epic. The SBOM diff fails the CI job if any excluded component leaks into the replay image. Image base: same Python + CUDA base as the live image (consistency with TensorRT engines from C7) but with `BUILD_C6=OFF`, `BUILD_C10=OFF`, `BUILD_C11=OFF`, `BUILD_C12=OFF`, `BUILD_VIDEO_FILE_FRAME_SOURCE=ON`, `BUILD_TLOG_REPLAY_ADAPTER=ON`, `BUILD_REPLAY_SINK_JSONL=ON` build args. +**Description**: Add the fourth Docker image `gps-denied-replay-cli`: multi-stage build (Python + C1–C5 + cpp/* + replay strategies; NO C6/C10/C11/C12; NO HTTP server). Add a GitHub Actions matrix entry building and pushing this image alongside the existing 3 images (live / research / operator). Add an **SBOM diff CI step** that builds the SBOM (via `syft` or the project's existing SBOM tooling), parses it, and asserts the absence of `c6_tile_cache`, `c10_provisioning`, `c11_tilemanager`, `c12_operator_orchestrator` packages — verifies AC-4 of the epic. The SBOM diff fails the CI job if any excluded component leaks into the replay image. Image base: same Python + CUDA base as the live image (consistency with TensorRT engines from C7) but with `BUILD_C6=OFF`, `BUILD_C10=OFF`, `BUILD_C11=OFF`, `BUILD_C12=OFF`, `BUILD_VIDEO_FILE_FRAME_SOURCE=ON`, `BUILD_TLOG_REPLAY_ADAPTER=ON`, `BUILD_REPLAY_SINK_JSONL=ON` build args. **Complexity**: 3 points **Dependencies**: AZ-402 (CLI entrypoint registered in pyproject); AZ-398 / AZ-399 / AZ-400 / AZ-401 (replay strategies); existing Dockerfile + CI plumbing for the live image (pattern to mirror); `module-layout.md` build-flag table; AZ-263, AZ-269, AZ-266 **Component**: replay-cicd (epic AZ-265 / E-DEMO-REPLAY) — Dockerfile at `docker/replay-cli/Dockerfile`; CI at `.github/workflows/build-images.yml` (or equivalent); SBOM-diff script at `ci/sbom_diff_replay.py` @@ -27,7 +27,7 @@ Without this task, the replay binary cannot ship — there's no CI matrix entry - Entrypoint: `gps-denied-replay`. - No HTTP server (no exposed ports; CLI only). - `.github/workflows/build-images.yml` matrix entry for `replay-cli` (image tag, build args, push to registry). -- `ci/sbom_diff_replay.py` — generates the SBOM via `syft packages dir:./ -o spdx-json` (or equivalent) on the built image, parses it, asserts the absence of `c6_tile_cache`, `c10_provisioning`, `c11_tilemanager`, `c12_operator_tooling` Python packages. Exit 0 on clean SBOM; exit 1 on leak (with the leaking package name printed). +- `ci/sbom_diff_replay.py` — generates the SBOM via `syft packages dir:./ -o spdx-json` (or equivalent) on the built image, parses it, asserts the absence of `c6_tile_cache`, `c10_provisioning`, `c11_tilemanager`, `c12_operator_orchestrator` Python packages. Exit 0 on clean SBOM; exit 1 on leak (with the leaking package name printed). - CI step `replay-cli-sbom-diff` invokes the script after the image build; fails the job on script exit 1. - Documentation: `docker/replay-cli/README.md` documents the image scope + build-args. - Unit / smoke tests: `docker buildx build` of the Dockerfile succeeds locally; SBOM-diff script runs against a pre-built test image fixture. diff --git a/_docs/03_implementation/batch_44_cycle1_report.md b/_docs/03_implementation/batch_44_cycle1_report.md new file mode 100644 index 0000000..7958452 --- /dev/null +++ b/_docs/03_implementation/batch_44_cycle1_report.md @@ -0,0 +1,191 @@ +# Batch 44 — Cycle 1 Report + +**Date**: 2026-05-13 +**Batch**: 44 +**Tasks**: AZ-329 (C12 PostLandingUploadOrchestrator, 3pt) + AZ-330 (C12 OperatorReLocService, 3pt) + AZ-523 (audit: C11 internal gate removal, 3pt) + AZ-524 (audit: C12 package rename, 2pt) +**Status**: complete; AZ-329 + AZ-330 in In Testing; AZ-317 superseded → Done; AZ-523 + AZ-524 created as audit-trail tickets and closed on creation. + +## Scope + +Batch 44 is an atomic refactor delivering two new C12 services AND a paired SRP rebalance between C11 and C12: + +1. **AZ-329 PostLandingUploadOrchestrator** — gates C11's `upload_pending` on a confirmed `flight_footer` FDR record (`clean_shutdown == True`) read via a new `FdrFooterReader` Protocol + `LocalFdrFooterReader` concrete impl. Surfaces four refusal modes (`footer_missing`, `unclean_shutdown`, `flight_id_not_found`, `fdr_unreadable: `) plus a `SatelliteProviderError` passthrough wrapper. +2. **AZ-330 OperatorReLocService** — operator-side surface for AC-3.4 visual-loss re-localization. Validates a `ReLocHint` (reuses shared `LatLonAlt`; lat/lon/radius/reason invariants), forwards it via a new `OperatorCommandTransport` Protocol cut (E-C8 owns the future pymavlink concrete; pattern matches AZ-322's `BackboneEmbedder`), and emits a `c12.reloc.requested` FDR record with `outcome ∈ {sent, failed}`. +3. **C11 internal flight-state gate removal (SRP)** — the previously-shipped `confirm_flight_state` / `FlightStateSignal` / `FlightStateNotOnGroundError` surface in `c11_tile_manager` is **removed**. The post-landing safety responsibility now lives in C12 (single source of truth). The `TileUploader` Protocol contract is bumped to **v2.0.0 (frozen)**. +4. **C12 package rename** — `c12_operator_tooling` → `c12_operator_orchestrator` across source, tests, configs, CMake flag, CLI binary (`operator-tool` → `operator-orchestrator`), runtime-root services class (`OperatorToolServices` → `OperatorOrchestratorServices`), factory function (`build_operator_tool` → `build_operator_orchestrator`), logger namespaces, documentation directories, and the E-C12 epic summary on Jira. + +## Architectural Decisions + +### 1. Single-source-of-truth for the post-landing gate (SRP refactor) + +The previous design had C11's `TileUploader` consume a `FlightStateSignal` from C8 and refuse to upload when `MAV_STATE != ON_GROUND`. C12 was also expected to confirm `ON_GROUND` independently before invoking C11. This duplicated the safety invariant on both sides of the C11/C12 boundary — a "defence-in-depth" justification that did not survive review: the safety invariant is "the vehicle has fully stopped and shut down cleanly", and the single authoritative observer of that state is C13 (the FDR writer), which emits a `flight_footer` record only on clean shutdown. + +Resolution: C11 stops gating. The C12 `PostLandingUploadOrchestrator` reads the footer C13 wrote and either invokes C11 (which no longer gates) or refuses with an actionable error. Each side has exactly one responsibility. + +### 2. Footer-based gate (Phase C design pivot) + +The original AZ-329 spec described counting consecutive `FlightStateSignal` records and asserting a contiguous `ON_GROUND` duration ≥ 30 s. Phase C pivoted to reading the single `flight_footer` FDR record because: + +- The footer is the authoritative "vehicle stopped cleanly" signal (written by C13 only on clean shutdown). +- Counting consecutive signals duplicates state-machine logic C13 already encodes. +- The 30-second hold-down was an arbitrary heuristic; `clean_shutdown` is exact. + +The new design is mechanically simpler (read one record, check one boolean), removes a configurable threshold (`upload_min_on_ground_s`), and aligns with the SRP rebalance. + +### 3. Cross-component cut for the GCS-link transport (AZ-507) + +AZ-330 needs to send a re-loc hint to the airborne companion over the GCS link, which is C8's territory. C12 cannot import C8 directly (AZ-507 boundary policy). Resolution: + +- C12 owns `OperatorCommandTransport` Protocol (`operator_command_transport.py`) with one method `send_reloc_hint(hint: ReLocHint) -> None`. +- Concrete `MavlinkOperatorCommandTransport` (pymavlink-backed) will land in a future E-C8 task. Pattern matches AZ-322's `BackboneEmbedder` (C10 owns Protocol; C2 implements later). +- C12's `build_operator_orchestrator` accepts the transport as a constructor parameter; when omitted, `operator_reloc_service` stays `None` (AC-10 lazy composition — pymavlink is never imported on the operator-tool happy path). + +### 4. Log redaction policy + +- Live INFO/ERROR logs: lat/lon rounded to 5 decimals (~1 m precision), `reason` truncated to 200 chars, no `api_key` / `auth_token` substrings ever logged. +- FDR records: full hint un-redacted (post-flight forensics requirement; FDR is operator-only-readable). +- API-key leak coverage: parametrized tests verify the key never appears in logs across all five post-landing outcomes (success + four refusal modes). + +### 5. Best-effort FDR-record enqueue (AC-8) + +Both new services emit FDR records, but neither raises if the FDR ring buffer overruns — the primary user-visible action (upload triggered / reloc sent) is the contract; the FDR record is for post-flight forensics. Overrun returns `(record_id=None, overrun=True)` and is silently dropped. Unit-tested. + +### 6. Lazy service construction (NFR-perf-cold-start) + +`build_operator_orchestrator` builds each service only when its required collaborators are provided. `operator-orchestrator --help` cold-start stays ≤ 500 ms p99 (matched by the same regression test from AZ-326). The reloc service in particular avoids importing pymavlink unless a transport is wired. + +### 7. New FDR record kind: `c12.reloc.requested` + +Registered in `fdr_client/records.py` `KNOWN_PAYLOAD_KEYS` with fields `{hint, outcome, failure_reason, ts_monotonic_ns}`. The AZ-272 schema roundtrip fixture (`test_az272_fdr_record_schema.py`) was extended with a sample payload so the unknown-kind assertion stays green. + +### 8. Renamed package: scope of the rename + +Renaming `c12_operator_tooling` was driven by the broader responsibility shift — the component no longer ONLY does pre-flight tooling; it now also owns the post-landing safety gate and the operator re-loc service. "Operator orchestrator" reflects that. The rename touched: Python package, test directory, CLI binary, runtime-root services class + factory function, logger namespaces, config slug, CMake build flag, deployment Dockerfile name, documentation component + contract directories, and the E-C12 epic title on Jira. + +## Files Changed + +### Production source (new — AZ-329) + +- `src/gps_denied_onboard/components/c12_operator_orchestrator/post_landing_upload.py` — `PostLandingUploadOrchestrator` + `trigger_post_landing_upload(request) -> UploadBatchReportCut`. +- `src/gps_denied_onboard/components/c12_operator_orchestrator/fdr_footer_reader.py` — `FdrFooterReader` Protocol + `LocalFdrFooterReader` concrete (walks newest→oldest segments, parses length-prefixed footer record, validates `flight_id` match). +- `src/gps_denied_onboard/components/c12_operator_orchestrator/tile_uploader_cut.py` — `TileUploaderCut` Protocol + `UploadBatchReportCut` DTO (consumer-side cut for C11 `TileUploader`). + +### Production source (new — AZ-330) + +- `src/gps_denied_onboard/components/c12_operator_orchestrator/operator_reloc_service.py` — `OperatorReLocService.request_reloc(hint)` with INFO/ERROR logging + redaction + FDR enqueue. +- `src/gps_denied_onboard/components/c12_operator_orchestrator/operator_command_transport.py` — `OperatorCommandTransport` runtime_checkable Protocol. + +### Production source (modified) + +- `src/gps_denied_onboard/components/c12_operator_orchestrator/_types.py` — added `PostLandingUploadRequest`, `ReLocHint` (with `__post_init__` validation reusing shared `LatLonAlt`). +- `src/gps_denied_onboard/components/c12_operator_orchestrator/errors.py` — added `FlightStateNotConfirmedError` (4 sub-reasons + `remediation`), `SatelliteProviderError`, `FdrUnreadableError`, `GcsLinkError` (with `remediation` + wrapped-exception `repr` capture). +- `src/gps_denied_onboard/components/c12_operator_orchestrator/cli.py` — added `upload-pending` and `reloc-confirm` subcommands; CLI-side ValueError → usage-error mapping; exit codes `EXIT_FOOTER_MISSING`, `EXIT_UNCLEAN_SHUTDOWN`, `EXIT_FLIGHT_ID_NOT_FOUND`, `EXIT_FDR_UNREADABLE`, `EXIT_GCS_LINK_ERROR`, `EXIT_SATELLITE_PROVIDER_ERROR`. +- `src/gps_denied_onboard/components/c12_operator_orchestrator/config.py` — added `C12PostLandingUploadConfig`; reloc service has no static config (pure DI). +- `src/gps_denied_onboard/components/c12_operator_orchestrator/__init__.py` — re-exports new types; PEP 562 lazy machinery extended. +- `src/gps_denied_onboard/components/c12_operator_orchestrator/interface.py` — removed stale Protocol placeholder for `OperatorReLocService` (now a concrete class in its own module). +- `src/gps_denied_onboard/runtime_root/c12_factory.py` — extended `OperatorOrchestratorServices` with `post_landing_upload_orchestrator` + `operator_reloc_service`; added `build_post_landing_upload_orchestrator(...)` + `build_operator_reloc_service(...)`; `build_operator_orchestrator(...)` (renamed from `build_operator_tool`) accepts optional `tile_uploader`, `operator_command_transport`, `fdr_client` — each gates one service field. +- `src/gps_denied_onboard/fdr_client/records.py` — registered `c12.reloc.requested` payload keys. + +### Production source (removed — C11 gate revert / AZ-523) + +- `src/gps_denied_onboard/components/c11_tile_manager/flight_state_gate.py` — **deleted**. +- `src/gps_denied_onboard/components/c11_tile_manager/_types.py` — removed `FlightStateSignal` import (still defined in `_types/fc.py` for C8 consumption; only the C11 *use* is removed). +- `src/gps_denied_onboard/components/c11_tile_manager/errors.py` — removed `FlightStateNotOnGroundError`. +- `src/gps_denied_onboard/components/c11_tile_manager/interface.py` — removed `confirm_flight_state` from `TileUploader` Protocol. +- `src/gps_denied_onboard/components/c11_tile_manager/tile_uploader.py` — removed the gate call from `upload_pending`. +- `src/gps_denied_onboard/components/c11_tile_manager/__init__.py` + `idempotent_retry.py` — adjusted re-exports and decorator boundaries. + +### Production source (Phase A rename — AZ-524) + +- All paths under `src/gps_denied_onboard/components/c12_operator_tooling/` → `src/gps_denied_onboard/components/c12_operator_orchestrator/` (git mv). +- `pyproject.toml` `[project.scripts]` entry: `operator-tool` → `operator-orchestrator`. +- `cmake/build_options.cmake`: `BUILD_C12_OPERATOR_TOOLING` → `BUILD_C12_OPERATOR_ORCHESTRATOR`. +- `docker/operator-tooling.Dockerfile` → `docker/operator-orchestrator.Dockerfile` (git mv). +- `docker-compose.yml`, `docker-compose.test.yml`, `.github/workflows/release.yml`, `README.md` — string sweep. +- Logger namespaces: `c12.operator_tool.*` → `c12.operator_orchestrator.*`. +- Config slug under `Config.components`: `operator_tool` → `c12_operator_orchestrator`. + +### Tests (new) + +- `tests/unit/c12_operator_orchestrator/test_post_landing_upload_orchestrator.py` — 11 tests covering AC-1..AC-7 + AC-8 (api-key redaction across 5 outcomes). +- `tests/unit/c12_operator_orchestrator/test_fdr_footer_reader.py` — 11 tests covering AC-6 (segment walk + short-circuit) + AC-9/AC-10 fixture integration + 7 error-path tests. +- `tests/unit/c12_operator_orchestrator/test_operator_reloc_service.py` — 15 tests covering AC-1..AC-9 + AC-10 lazy composition. +- `tests/unit/test_az272_fdr_record_schema.py` — added `c12.reloc.requested` fixture entry (schema roundtrip). + +### Tests (removed) + +- `tests/unit/c11_tile_manager/test_flight_state_gate.py` — **deleted** along with the gate module. + +### Tests (Phase A rename — AZ-524) + +- `tests/unit/c12_operator_tooling/` → `tests/unit/c12_operator_orchestrator/` (git mv). +- Test-internal references to the renamed factory + class + binary updated (`build_operator_tool` → `build_operator_orchestrator`; `operator_tool_binary` fixture → `operator_orchestrator_binary`). + +### Documentation + +- `_docs/02_document/components/13_c12_operator_tooling/` → `13_c12_operator_orchestrator/` (git mv); description.md + tests.md rewritten for the new gate design + interface table updates. +- `_docs/02_document/contracts/c12_operator_tooling/` → `c12_operator_orchestrator/` (git mv); added `operator_command_transport.md` contract for the new Protocol. +- `_docs/02_document/contracts/c11_tilemanager/tile_uploader.md` — bumped to v2.0.0 (frozen); migration note documents the gate removal. +- `_docs/02_document/components/12_c11_tilemanager/description.md` + `tests.md` — gate references removed; C11-IT-04 retargeted to cross-reference the C12 gate. +- `_docs/02_tasks/done/AZ-317_c11_flight_state_gate.md` — SUPERSEDED banner added. +- `_docs/02_tasks/todo/AZ-329_c12_post_landing_upload.md` + `AZ-330_c12_operator_reloc_service.md` — task specs rewritten to reflect Phase C design + AZ-507 cuts. +- `_docs/02_tasks/_dependencies_table.md` — AZ-329/AZ-330 dep edges updated; AZ-317 marked SUPERSEDED in-table; AZ-523 + AZ-524 added; coverage-verification section updated. +- Cross-cutting docs swept for old names: `architecture.md`, `module-layout.md`, `FINAL_report.md`, `epics.md`, `glossary.md`, `data_model.md`, `deployment/*.md`, `system-flows.md`. + +## Task Results + +| Task | Status | Files (new / mod / del) | Tests added | AC Coverage | Issues | +|------|--------|-------------------------|-------------|-------------|--------| +| AZ-329 | In Testing | 3 / 8 / 0 | 22 (test_post_landing_upload + test_fdr_footer_reader) | 10/10 ACs | None | +| AZ-330 | In Testing | 2 / 5 / 0 | 15 (test_operator_reloc_service) | 10/10 ACs | None | +| AZ-523 (audit: C11 gate removal) | Done | 0 / 6 / 2 | n/a (existing C11 tests still green) | n/a | None | +| AZ-524 (audit: C12 package rename) | Done | git-mv only | n/a (1543 tests green post-rename) | n/a | None | +| AZ-317 (superseded) | Done | 0 / 1 / 0 (annotation only) | n/a | n/a | Superseded by AZ-523 | +| AZ-319 (TileUploader contract v2.0.0) | unchanged status (In Testing) | covered by AZ-523 deletes | n/a | n/a | None | + +## AC Test Coverage: All covered + +- **AZ-329 (AC-1..AC-10)**: every AC has a directly-validating test in `test_post_landing_upload_orchestrator.py` or `test_fdr_footer_reader.py`. AC-8 is parametrized across all five outcomes (1 success + 4 refusal modes) for api-key-leak coverage. AC-9 + AC-10 are full-stack fixture integration tests against on-disk FDR fixtures. +- **AZ-330 (AC-1..AC-10)**: every AC has a directly-validating test in `test_operator_reloc_service.py`. AC-7 (lat/lon range), AC-3 (radius), and AC-6 (reason) DTO validation are parametrized; AC-10 lazy composition has its own factory-level test (`test_build_operator_orchestrator_does_not_construct_operator_reloc_service_without_transport`). + +## Code Review Verdict: PASS + +### Findings + +None of severity Low or higher. + +### Notes (informational) + +- `tests/unit/c12_operator_orchestrator/test_cli_console_script.py` has the same flake-prone `test_cold_start_under_500ms_p99` documented in batch 42's report. The minimal imports added in Batch 44 (`OperatorCommandTransport`, `OperatorReLocService`, `ReLocHint`, `GcsLinkError`) are all pure-Python and add no measurable startup cost. Test passes when run individually; the flake is from system noise on the eager-aggregated test runs. +- One pre-existing leftover from Phase A (the factory function `build_operator_tool` and the test fixture name `operator_tool_binary`) was caught in the Phase H verification sweep and corrected in this batch — completing the Phase A rename intent. + +## Tracker Updates (Phase G) + +- **AZ-317** → **Done** with SUPERSEDED comment + annotated task spec in `_docs/02_tasks/done/`. +- **AZ-319** → comment added documenting the v2.0.0 contract bump + the four breaking removals from the `TileUploader` surface (no status change; already In Testing). +- **AZ-329** → summary updated; design-pivot + implementation-complete comment added; transitioned **To Do → In Testing**. +- **AZ-330** → implementation-complete comment added; transitioned **To Do → In Testing**. +- **AZ-253 (E-C12 epic)** → summary renamed `C12 Operator Pre-flight Tooling` → `C12 Operator Pre-flight Orchestrator`. +- **AZ-523** created and closed: "C11 internal flight-state gate removal (SRP refactor)", parent AZ-251, 3pt. +- **AZ-524** created and closed: "C12 package rename: c12_operator_tooling → c12_operator_orchestrator", parent AZ-253, 2pt. +- **`_docs/02_tasks/_dependencies_table.md`** refreshed: AZ-329 + AZ-330 dep edges updated; AZ-317 marked SUPERSEDED; AZ-523 + AZ-524 rows added; new "Batch 44 SRP refactor + C12 rename" Notes paragraph documents the rebalance. + +## Auto-Fix Attempts: 0 + +## Stuck Agents: None + +## Test Suite + +- **Full repository unit suite**: 1543 passed, 80 skipped, 3 warnings in ~64 s (skipped: pre-existing Docker / CUDA / Jetson / TensorRT / actionlint environment gates). +- **Targeted AC suite** (AZ-329 + AZ-330 + FDR-footer-reader): 37 passed in 1.24 s. +- **C11 post-gate-removal**: zero regressions; all pre-existing C11 unit tests still green. +- `python -X importtime` cold-start: `operator-orchestrator --help` consistently ≤ 200 ms locally; CLI console-script test asserts ≤ 500 ms p99 (test still green; one statistical-noise flake noted above). + +## Next Batch + +Natural follow-ups: + +- **E-C8** task to implement `MavlinkOperatorCommandTransport` (concrete pymavlink-backed `OperatorCommandTransport`) — unblocks end-to-end AC-3.4 with a real GCS link. +- **C12-IT-03 / C12-IT-04** end-to-end integration tests against a Tier-1 footer fixture + a stubbed `TileUploader` — the Batch 44 unit tests already exercise every AC, but an end-to-end pass would close the C12 epic's integration coverage line. + +Both are independent of each other and can be batched in any order. Confirm with `_docs/02_tasks/_dependencies_table.md` at the start of Batch 45. diff --git a/_docs/_autodev_state.md b/_docs/_autodev_state.md index 559b15c..75cbca5 100644 --- a/_docs/_autodev_state.md +++ b/_docs/_autodev_state.md @@ -6,13 +6,13 @@ step: 7 name: Implement status: in_progress sub_step: - phase: 11 + phase: 6 name: implement-tasks-sequentially - detail: "" + detail: "batch 44 phase H (closeout)" retry_count: 0 cycle: 1 tracker: jira last_completed_batch: 43 last_cumulative_review: batches_40-42 -in_flight_batch: null -in_flight_tasks: null +in_flight_batch: 44 +in_flight_tasks: ["AZ-329", "AZ-330", "c11-gate-revert", "c12-rename"] diff --git a/cmake/build_options.cmake b/cmake/build_options.cmake index b723a65..f9d156c 100644 --- a/cmake/build_options.cmake +++ b/cmake/build_options.cmake @@ -28,7 +28,7 @@ option(BUILD_PYTORCH_RUNTIME "Build C7 PyTorch FP16 inference runtime" O option(BUILD_C10_PROVISIONING "Build C10 (operator-only)" OFF) option(BUILD_C11_TILE_MANAGER "Build C11 (operator-only)" OFF) -option(BUILD_C12_OPERATOR_TOOLING "Build C12 (operator-only)" OFF) +option(BUILD_C12_OPERATOR_ORCHESTRATOR "Build C12 (operator-only)" OFF) option(BUILD_GTSAM_BINDINGS "Build cpp/gtsam_bindings (C4+C5)" ON) option(BUILD_FAISS_INDEX "Enable C6 FAISS descriptor index (faiss-cpu PyPI; runtime gate, no native target — AZ-306)" ON) diff --git a/docker-compose.test.yml b/docker-compose.test.yml index c8d19ce..d05f381 100644 --- a/docker-compose.test.yml +++ b/docker-compose.test.yml @@ -6,10 +6,10 @@ services: environment: LOG_LEVEL: INFO - operator-tooling: + operator-orchestrator: extends: file: docker-compose.yml - service: operator-tooling + service: operator-orchestrator mock-sat: extends: diff --git a/docker-compose.yml b/docker-compose.yml index 8d75337..e355640 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -31,11 +31,11 @@ services: timeout: 3s retries: 3 - operator-tooling: + operator-orchestrator: build: context: . - dockerfile: docker/operator-tooling.Dockerfile - image: gps-denied-onboard/operator-tooling:dev + dockerfile: docker/operator-orchestrator.Dockerfile + image: gps-denied-onboard/operator-orchestrator:dev depends_on: db: condition: service_healthy diff --git a/docker/operator-tooling.Dockerfile b/docker/operator-orchestrator.Dockerfile similarity index 90% rename from docker/operator-tooling.Dockerfile rename to docker/operator-orchestrator.Dockerfile index b0fd660..57af92a 100644 --- a/docker/operator-tooling.Dockerfile +++ b/docker/operator-orchestrator.Dockerfile @@ -1,4 +1,4 @@ -# Operator-tooling image — installs C11 + C12 + healthcheck. +# Operator-orchestrator image — installs C11 + C12 + healthcheck. # Per `_docs/02_document/deployment/containerization.md`. FROM python:3.10-slim AS runtime diff --git a/pyproject.toml b/pyproject.toml index f114aee..42328cd 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -119,7 +119,7 @@ telemetry = [ [project.scripts] gps-denied-replay = "gps_denied_onboard.cli.replay:main" -operator-tool = "gps_denied_onboard.components.c12_operator_tooling.cli:main" +operator-orchestrator = "gps_denied_onboard.components.c12_operator_orchestrator.cli:main" [tool.setuptools] package-dir = {"" = "src"} diff --git a/src/gps_denied_onboard/components/c10_provisioning/__init__.py b/src/gps_denied_onboard/components/c10_provisioning/__init__.py index 90dde33..4e264b5 100644 --- a/src/gps_denied_onboard/components/c10_provisioning/__init__.py +++ b/src/gps_denied_onboard/components/c10_provisioning/__init__.py @@ -5,7 +5,7 @@ lives at the L1 ``_types`` layer so C10 can re-export it without crossing the components.* boundary (architecture rule AC-6). The AZ-321 ``EngineCompiler`` plus its DTOs are re-exported here so -the composition root and downstream operator-tooling code consume +the composition root and downstream operator-orchestrator code consume them through this single contract surface. """ diff --git a/src/gps_denied_onboard/components/c10_provisioning/manifest_verifier.py b/src/gps_denied_onboard/components/c10_provisioning/manifest_verifier.py index 9323e70..4f1773a 100644 --- a/src/gps_denied_onboard/components/c10_provisioning/manifest_verifier.py +++ b/src/gps_denied_onboard/components/c10_provisioning/manifest_verifier.py @@ -9,7 +9,7 @@ a verify failure — callers branch on ``outcome`` (per the contract at The Protocol + DTOs live alongside the implementation here; the public re-export surface lives in ``c10_provisioning/__init__.py``. -Cross-component consumers (C5 takeoff arming, C12 operator tooling) +Cross-component consumers (C5 takeoff arming, C12 operator orchestrator) will import via a future ``_types/manifest_verify.py`` shim if and when they wire up — the AZ-270 lint forbids direct ``components.c10_provisioning`` imports from other components. diff --git a/src/gps_denied_onboard/components/c11_tile_manager/__init__.py b/src/gps_denied_onboard/components/c11_tile_manager/__init__.py index 82b9a80..f488c14 100644 --- a/src/gps_denied_onboard/components/c11_tile_manager/__init__.py +++ b/src/gps_denied_onboard/components/c11_tile_manager/__init__.py @@ -1,10 +1,10 @@ """C11 Tile Manager component — Public API. -Re-exports the Protocol surface (``TileDownloader``, ``TileUploader``, -``FlightStateSource``), the operator-side services that have landed -(``FlightStateGate`` from AZ-317, ``PerFlightKeyManager`` from AZ-318, -``HttpTileUploader`` from AZ-319, ``HttpTileDownloader`` from AZ-316), -the C11 internal DTOs / enums, the C11 error family, and the +Re-exports the Protocol surface (``TileDownloader``, ``TileUploader``), +the operator-side services that have landed (``PerFlightKeyManager`` +from AZ-318, ``HttpTileUploader`` from AZ-319 — flight-state gating is +now C12's responsibility per batch 44; ``HttpTileDownloader`` from +AZ-316), the C11 internal DTOs / enums, the C11 error family, and the per-component config block. """ @@ -12,7 +12,6 @@ from gps_denied_onboard.components.c11_tile_manager._types import ( DownloadBatchReport, DownloadOutcome, DownloadRequest, - FlightStateSignal, IngestStatus, PerTileStatus, PublicKeyFingerprint, @@ -28,7 +27,6 @@ from gps_denied_onboard.components.c11_tile_manager.config import ( ) from gps_denied_onboard.components.c11_tile_manager.errors import ( CacheBudgetExceededError, - FlightStateNotOnGroundError, RateLimitedError, ResolutionRejectionError, SatelliteProviderError, @@ -36,14 +34,10 @@ from gps_denied_onboard.components.c11_tile_manager.errors import ( SignatureRejectedError, TileManagerError, ) -from gps_denied_onboard.components.c11_tile_manager.flight_state_gate import ( - FlightStateGate, -) from gps_denied_onboard.components.c11_tile_manager.idempotent_retry import ( IdempotentRetryTileUploader, ) from gps_denied_onboard.components.c11_tile_manager.interface import ( - FlightStateSource, TileDownloader, TileUploader, ) @@ -71,10 +65,6 @@ __all__ = [ "DownloadBatchReport", "DownloadOutcome", "DownloadRequest", - "FlightStateGate", - "FlightStateNotOnGroundError", - "FlightStateSignal", - "FlightStateSource", "HttpTileDownloader", "HttpTileUploader", "IdempotentRetryTileUploader", diff --git a/src/gps_denied_onboard/components/c11_tile_manager/_types.py b/src/gps_denied_onboard/components/c11_tile_manager/_types.py index 30d1204..60e7250 100644 --- a/src/gps_denied_onboard/components/c11_tile_manager/_types.py +++ b/src/gps_denied_onboard/components/c11_tile_manager/_types.py @@ -1,14 +1,12 @@ -"""C11 internal DTOs (AZ-316, AZ-317, AZ-318, AZ-319). +"""C11 internal DTOs (AZ-316, AZ-318, AZ-319). -* :class:`FlightStateSignal` — five flight-state signals consumed by the - upload-side flight-state gate (AZ-317). * :class:`PublicKeyFingerprint` — per-flight Ed25519 keypair fingerprint envelope returned by :meth:`PerFlightKeyManager.start_session` (AZ-318). * :class:`UploadRequest`, :class:`UploadBatchReport`, :class:`PerTileStatus`, :class:`IngestStatus`, :class:`UploadOutcome` — upload-side DTOs and enums consumed and produced by the AZ-319 :class:`HttpTileUploader` (contract - ``_docs/02_document/contracts/c11_tilemanager/tile_uploader.md`` v1.0.0). + ``_docs/02_document/contracts/c11_tilemanager/tile_uploader.md`` v2.0.0). * :class:`DownloadRequest`, :class:`DownloadBatchReport`, :class:`TileSummary`, :class:`DownloadOutcome`, :class:`SectorClassification` — download-side DTOs and enums consumed @@ -33,7 +31,6 @@ __all__ = [ "DownloadBatchReport", "DownloadOutcome", "DownloadRequest", - "FlightStateSignal", "IngestStatus", "PerTileStatus", "PublicKeyFingerprint", @@ -45,20 +42,6 @@ __all__ = [ ] -class FlightStateSignal(str, Enum): - """Five flight-state signals C11's upload-side gate accepts. - - Only :attr:`ON_GROUND` permits an upload; every other value is - fail-closed by the AZ-317 gate (AC-2..AC-5). - """ - - ON_GROUND = "on_ground" - TAKING_OFF = "taking_off" - IN_FLIGHT = "in_flight" - LANDING = "landing" - UNKNOWN = "unknown" - - @dataclass(frozen=True) class PublicKeyFingerprint: """Public-key envelope returned by :meth:`PerFlightKeyManager.start_session`. @@ -99,10 +82,9 @@ class UploadOutcome(str, Enum): ``DUPLICATE`` / ``SUPERSEDED``. * ``PARTIAL`` — some tiles were ``REJECTED`` while others were acknowledged; the caller may re-invoke for the rejected set. - * ``FAILURE`` — the flight-state gate blocked or zero tiles could - be POSTed (TLS / 401 / 403 / persistent 5xx surface as raised - :class:`SatelliteProviderError`, NOT as ``FAILURE`` in a returned - report). + * ``FAILURE`` — zero tiles could be POSTed (TLS / 401 / 403 / + persistent 5xx surface as raised :class:`SatelliteProviderError`, + NOT as ``FAILURE`` in a returned report). """ SUCCESS = "success" @@ -292,7 +274,7 @@ class DownloadRequest: class DownloadBatchReport: """Aggregate report returned by :meth:`TileDownloader.download_tiles_for_area`. - Per-tile counts let the operator-tooling CLI render the post-run + Per-tile counts let the operator-orchestrator CLI render the post-run summary without re-reading the journal: * ``tiles_requested`` — total tiles enumerated by diff --git a/src/gps_denied_onboard/components/c11_tile_manager/errors.py b/src/gps_denied_onboard/components/c11_tile_manager/errors.py index 1e18e52..28966b6 100644 --- a/src/gps_denied_onboard/components/c11_tile_manager/errors.py +++ b/src/gps_denied_onboard/components/c11_tile_manager/errors.py @@ -1,13 +1,10 @@ -"""C11 TileManager error family (AZ-316, AZ-317, AZ-318, AZ-319). +"""C11 TileManager error family (AZ-316, AZ-318, AZ-319). Rooted at :class:`TileManagerError`. Both the upload (AZ-319) and download (AZ-316) paths share the family parent so cross-path callers can ``except TileManagerError`` to catch any C11-side terminal failure without enumerating subclasses. -* :class:`FlightStateNotOnGroundError` (AZ-317) — defence-in-depth - refusal when the flight controller reports anything other than - ``ON_GROUND`` at upload entry. * :class:`SessionNotActiveError` (AZ-318) — :meth:`PerFlightKeyManager.sign` / :meth:`record_signature_rejection` called outside an active session. * :class:`SignatureRejectedError` (AZ-318/AZ-319 envelope) — surfaced @@ -28,17 +25,8 @@ without enumerating subclasses. from __future__ import annotations -from datetime import datetime -from typing import TYPE_CHECKING - -if TYPE_CHECKING: - from gps_denied_onboard.components.c11_tile_manager._types import ( - FlightStateSignal, - ) - __all__ = [ "CacheBudgetExceededError", - "FlightStateNotOnGroundError", "RateLimitedError", "ResolutionRejectionError", "SatelliteProviderError", @@ -52,27 +40,6 @@ class TileManagerError(Exception): """Base class for the C11 TileManager error family.""" -class FlightStateNotOnGroundError(TileManagerError): - """Upload was attempted when the flight controller is not on ground. - - Carries the observed :class:`FlightStateSignal` and the diagnostic - ``observed_at`` timestamp. The original source exception (if the - refusal was caused by a :class:`FlightStateSource` failure mapped - to ``UNKNOWN`` per AC-5) is preserved on ``__cause__``. - """ - - def __init__( - self, - observed: FlightStateSignal, - observed_at: datetime, - ) -> None: - self.observed: FlightStateSignal = observed - self.observed_at: datetime = observed_at - super().__init__( - f"Upload refused: flight state is {observed.name}" - ) - - class SessionNotActiveError(TileManagerError): """:meth:`PerFlightKeyManager.sign` called without a live session. @@ -89,7 +56,7 @@ class SignatureRejectedError(TileManagerError): ``TileUploader`` raises the canonical type. The upload-side handler calls :meth:`PerFlightKeyManager.record_signature_rejection` to surface the FDR + ERROR log envelope per AZ-318 AC-8 before - re-raising this exception to the operator-tooling layer. + re-raising this exception to the operator-orchestrator layer. """ diff --git a/src/gps_denied_onboard/components/c11_tile_manager/flight_state_gate.py b/src/gps_denied_onboard/components/c11_tile_manager/flight_state_gate.py deleted file mode 100644 index 116707a..0000000 --- a/src/gps_denied_onboard/components/c11_tile_manager/flight_state_gate.py +++ /dev/null @@ -1,129 +0,0 @@ -"""C11 ``FlightStateGate`` (AZ-317). - -Defence-in-depth ON_GROUND gate for the upload entry point. The -primary control is ADR-004 process-level isolation — the airborne -binary has the entire ``c11_tile_manager`` source tree excluded at -build time. The gate is the runtime backstop: if the operator -workstation triggers an upload while the flight controller reports -anything other than ``ON_GROUND``, the gate refuses with -:class:`FlightStateNotOnGroundError`. - -Fail-closed by design — ``UNKNOWN``, transition states, and source -failures all block. AZ-317 acceptance criteria spell out the full -matrix. -""" - -from __future__ import annotations - -import logging -from datetime import datetime, timezone - -from gps_denied_onboard.components.c11_tile_manager._types import ( - FlightStateSignal, -) -from gps_denied_onboard.components.c11_tile_manager.errors import ( - FlightStateNotOnGroundError, -) -from gps_denied_onboard.components.c11_tile_manager.interface import ( - FlightStateSource, -) - -__all__ = ["FlightStateGate"] - - -_LOG_KIND_PASS = "c11.upload.flight_state_confirmed" -_LOG_KIND_REFUSED = "c11.upload.refused.flight_state" -_COMPONENT = "c11_tile_manager.flight_state_gate" - - -def _utcnow_second_precision() -> datetime: - """Diagnostic UTC timestamp truncated to seconds (AC-7).""" - return datetime.now(timezone.utc).replace(microsecond=0) - - -class FlightStateGate: - """Single-shot ON_GROUND check called by the upload entry point. - - The gate is constructed once at composition time and called once - per :meth:`upload_pending_tiles` invocation by the AZ-319 - :class:`TileUploader`. It performs no caching, no retries, and no - polling — :meth:`current_flight_state` is invoked exactly once per - :meth:`confirm_on_ground` call (AC-8). - """ - - def __init__( - self, - *, - source: FlightStateSource, - logger: logging.Logger, - ) -> None: - self._source = source - self._logger = logger - - def confirm_on_ground(self) -> FlightStateSignal: - """Return :attr:`FlightStateSignal.ON_GROUND` or raise. - - Behaviour matrix: - - * ``ON_GROUND`` → return + INFO log (AC-1). - * ``IN_FLIGHT`` / ``TAKING_OFF`` / ``LANDING`` / ``UNKNOWN`` → - raise :class:`FlightStateNotOnGroundError` + ERROR log - (AC-2..AC-4). - * Source raises → map to ``UNKNOWN`` + chain the original - exception via ``__cause__`` + ERROR log carrying the - original message (AC-5). - """ - - try: - observed = self._source.current_flight_state() - except Exception as exc: - observed_at = _utcnow_second_precision() - error = FlightStateNotOnGroundError( - observed=FlightStateSignal.UNKNOWN, - observed_at=observed_at, - ) - error.__cause__ = exc - self._logger.error( - "Upload refused: flight state source failed", - extra={ - "component": _COMPONENT, - "kind": _LOG_KIND_REFUSED, - "kv": { - "observed": FlightStateSignal.UNKNOWN.value, - "observed_at_iso": observed_at.isoformat(), - "source_error": str(exc), - }, - }, - ) - raise error - - observed_at = _utcnow_second_precision() - if observed is FlightStateSignal.ON_GROUND: - self._logger.info( - "Upload entry permitted: flight state is ON_GROUND", - extra={ - "component": _COMPONENT, - "kind": _LOG_KIND_PASS, - "kv": { - "observed": observed.value, - "observed_at_iso": observed_at.isoformat(), - }, - }, - ) - return observed - - self._logger.error( - f"Upload refused: flight state is {observed.name}", - extra={ - "component": _COMPONENT, - "kind": _LOG_KIND_REFUSED, - "kv": { - "observed": observed.value, - "observed_at_iso": observed_at.isoformat(), - }, - }, - ) - raise FlightStateNotOnGroundError( - observed=observed, - observed_at=observed_at, - ) diff --git a/src/gps_denied_onboard/components/c11_tile_manager/idempotent_retry.py b/src/gps_denied_onboard/components/c11_tile_manager/idempotent_retry.py index 1982ae3..3d70d3b 100644 --- a/src/gps_denied_onboard/components/c11_tile_manager/idempotent_retry.py +++ b/src/gps_denied_onboard/components/c11_tile_manager/idempotent_retry.py @@ -46,7 +46,6 @@ from uuid import UUID from gps_denied_onboard.clock.interface import Clock from gps_denied_onboard.components.c11_tile_manager._types import ( - FlightStateSignal, IngestStatus, PerTileStatus, UploadBatchReport, @@ -240,11 +239,6 @@ class IdempotentRetryTileUploader: return list(self._inner.enumerate_pending_tiles(flight_id)) - def confirm_flight_state(self) -> FlightStateSignal: - """Pass-through to the inner uploader (AC-11).""" - - return self._inner.confirm_flight_state() - # ------------------------------------------------------------------ # Internal helpers # ------------------------------------------------------------------ diff --git a/src/gps_denied_onboard/components/c11_tile_manager/interface.py b/src/gps_denied_onboard/components/c11_tile_manager/interface.py index 3c6a045..f17b0c0 100644 --- a/src/gps_denied_onboard/components/c11_tile_manager/interface.py +++ b/src/gps_denied_onboard/components/c11_tile_manager/interface.py @@ -1,4 +1,4 @@ -"""C11 ``TileDownloader`` + ``TileUploader`` + ``FlightStateSource`` Protocols. +"""C11 ``TileDownloader`` + ``TileUploader`` Protocols. Operator-side ONLY — excluded from airborne via CMake (`BUILD_C11_TILE_MANAGER=OFF`). See `_docs/02_document/components/12_c11_tilemanager/`. @@ -10,13 +10,9 @@ See `_docs/02_document/components/12_c11_tilemanager/`. * :class:`TileUploader` — post-landing upload path (AZ-319) — the authoritative shape lives in ``_docs/02_document/contracts/c11_tilemanager/tile_uploader.md`` - v1.0.0 and is mirrored 1:1 here. -* :class:`FlightStateSource` — thin C11-facing adapter the upload-side - flight-state gate (AZ-317) calls to read "what is the FC saying right - now?". A concrete impl ships with E-C8 (subscribes to the FC adapter's - flight-state stream); composition root wires it via the AZ-507 - consumer-side cut pattern (see `_docs/02_document/module-layout.md` - Rule 9). C11 NEVER imports ``components.c8_fc_adapter`` directly. + v2.0.0 (post-batch-44 removal of the internal flight-state gate) and + is mirrored 1:1 here. Flight-state confirmation is the caller's + responsibility (C12 ``PostLandingUploadOrchestrator``). """ from __future__ import annotations @@ -28,14 +24,12 @@ from uuid import UUID from gps_denied_onboard.components.c11_tile_manager._types import ( DownloadBatchReport, DownloadRequest, - FlightStateSignal, TileSummary, UploadBatchReport, UploadRequest, ) __all__ = [ - "FlightStateSource", "TileDownloader", "TileUploader", ] @@ -69,7 +63,7 @@ class TileUploader(Protocol): """Post-landing batch upload to ``satellite-provider`` ingest (D-PROJ-2). See ``_docs/02_document/contracts/c11_tilemanager/tile_uploader.md`` - v1.0.0 for invariants I-1 .. I-8 and the per-method error matrix. + v2.0.0 for invariants I-1 .. I-7 and the per-method error matrix. The :meth:`enumerate_pending_tiles` return type is the consumer- side structural metadata shape (mirrors c6's ``TileMetadata``; declared as ``Sequence[Any]`` here to keep C11 free of cross- @@ -81,20 +75,3 @@ class TileUploader(Protocol): def enumerate_pending_tiles( self, flight_id: UUID | None = None ) -> Sequence[Any]: ... - - def confirm_flight_state(self) -> FlightStateSignal: ... - - -@runtime_checkable -class FlightStateSource(Protocol): - """Consumer-side cut: "what is the flight controller saying now?". - - The AZ-317 :class:`FlightStateGate` calls - :meth:`current_flight_state` once per :meth:`confirm_on_ground` - invocation; no polling, no caching. The concrete impl that - subscribes to MAVLink heartbeats lives in E-C8 and is wrapped by a - composition-root adapter so C11 never imports - ``components.c8_fc_adapter``. - """ - - def current_flight_state(self) -> FlightStateSignal: ... diff --git a/src/gps_denied_onboard/components/c11_tile_manager/tile_uploader.py b/src/gps_denied_onboard/components/c11_tile_manager/tile_uploader.py index e8d197e..8ca8460 100644 --- a/src/gps_denied_onboard/components/c11_tile_manager/tile_uploader.py +++ b/src/gps_denied_onboard/components/c11_tile_manager/tile_uploader.py @@ -1,12 +1,14 @@ -"""C11 ``HttpTileUploader`` (AZ-319) — concrete :class:`TileUploader`. +"""C11 ``HttpTileUploader`` — concrete :class:`TileUploader`. Operator-side post-landing upload path. Reads pending mid-flight tiles from C6 (``source = onboard_ingest``, ``uploaded_at IS NULL``), packages each per the D-PROJ-2 multipart contract sketch, signs with the per-flight ephemeral key (AZ-318), POSTs to ``satellite-provider``'s ingest -endpoint, and marks acknowledged tiles uploaded. Gates on ``ON_GROUND`` -(AZ-317) before any C6 read or network egress; zeroes the signing key -in a try/finally regardless of outcome. +endpoint, and marks acknowledged tiles uploaded. Zeroes the signing key +in a try/finally regardless of outcome. Flight-state gating is a C12 +orchestrator policy (post-landing confirmation via the C13 +``flight_footer`` FDR record); this uploader is a dumb pipe and trusts +its caller. Architecture ------------ @@ -49,9 +51,6 @@ from gps_denied_onboard.components.c11_tile_manager.errors import ( SatelliteProviderError, SignatureRejectedError, ) -from gps_denied_onboard.components.c11_tile_manager.flight_state_gate import ( - FlightStateGate, -) from gps_denied_onboard.components.c11_tile_manager.signing_key import ( PerFlightKeyManager, ) @@ -245,9 +244,9 @@ class _SessionState: class HttpTileUploader: """Concrete :class:`TileUploader` against ``satellite-provider``'s ingest endpoint. - All cross-component dependencies (``flight_state_gate``, - ``key_manager``, ``tile_store``, ``tile_metadata_store``) are - constructor-injected via Protocol cuts. The ``http_client`` is an + All cross-component dependencies (``key_manager``, ``tile_store``, + ``tile_metadata_store``) are constructor-injected via Protocol cuts. + The ``http_client`` is an :class:`httpx.Client` the caller owns; ``HttpTileUploader`` does NOT close it — production wiring uses a long-lived client per process; tests inject ``httpx.Client(transport=httpx.MockTransport)`` @@ -260,7 +259,6 @@ class HttpTileUploader: http_client: httpx.Client, tile_store: _TileBytesReader, tile_metadata_store: _PendingMetadataReader, - flight_state_gate: FlightStateGate, key_manager: PerFlightKeyManager, fdr_client: FdrClient, logger: logging.Logger, @@ -270,7 +268,6 @@ class HttpTileUploader: self._http_client = http_client self._tile_store = tile_store self._metadata_store = tile_metadata_store - self._gate = flight_state_gate self._key_manager = key_manager self._fdr = fdr_client self._logger = logger @@ -282,15 +279,15 @@ class HttpTileUploader: # ------------------------------------------------------------------ def upload_pending_tiles(self, request: UploadRequest) -> UploadBatchReport: - """Gate → start_session → enumerate → batch loop → finally end_session. + """start_session → enumerate → batch loop → finally end_session. Order is FROZEN per Reliability constraint in the task spec — - re-ordering is a High Reliability finding at code-review time - because it breaks I-1 (gate before any read / network) or I-4 - (zeroisation guarantee on every exit path). + re-ordering would break I-4 (zeroisation guarantee on every exit + path). Flight-state confirmation is the caller's responsibility + (C12 ``PostLandingUploadOrchestrator``); this uploader is a dumb + pipe. """ - self._gate.confirm_on_ground() flight_id_for_session = request.flight_id or uuid4() fingerprint = self._key_manager.start_session(flight_id_for_session) state = _SessionState( @@ -362,15 +359,10 @@ class HttpTileUploader: def enumerate_pending_tiles( self, flight_id: UUID | None = None ) -> list[Any]: - """Read-only enumeration; does NOT call the gate (per contract).""" + """Read-only enumeration.""" return self._filter_by_flight(self._metadata_store.pending_uploads(), flight_id) - def confirm_flight_state(self) -> Any: - """Pass-through to :meth:`FlightStateGate.confirm_on_ground`.""" - - return self._gate.confirm_on_ground() - # ------------------------------------------------------------------ # Internal helpers # ------------------------------------------------------------------ diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/__init__.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/__init__.py similarity index 65% rename from src/gps_denied_onboard/components/c12_operator_tooling/__init__.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/__init__.py index 8163f77..9534f04 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/__init__.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/__init__.py @@ -11,15 +11,15 @@ Re-exports: Protocols, and the production :class:`ParamikoSshSessionFactory`. Also registers ``C12Config`` with :func:`register_component_block` so -the composition root sees the ``c12_operator_tooling`` slug under +the composition root sees the ``c12_operator_orchestrator`` slug under ``config.components``. NOTE on lazy imports (AZ-326 NFR-perf-cold-start, ≤500 ms p99 for -``operator-tool --help``): the heavy adapters +``operator-orchestrator --help``): the heavy adapters :class:`ParamikoSshSessionFactory` (pulls in ``paramiko`` + ``cryptography``) and :class:`HttpxFlightsApiClient` (pulls in ``httpx``) are exposed via a PEP 562 :func:`__getattr__` hook rather than top-level imports. Importing -them from this module — `from gps_denied_onboard.components.c12_operator_tooling +them from this module — `from gps_denied_onboard.components.c12_operator_orchestrator import HttpxFlightsApiClient` — still works for callers, but the heavy ``import paramiko`` / ``import httpx`` only fires on first access. The project spec's Constraints section forbids eager-importing these libs @@ -30,7 +30,7 @@ from __future__ import annotations from typing import TYPE_CHECKING, Any -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( AreaIdentifier, BuildCacheOutcome, BuildCacheRequest, @@ -42,36 +42,59 @@ from gps_denied_onboard.components.c12_operator_tooling._types import ( DownloadRequestCut, FailurePhase, FlightById, + FlightFooterRecord, FlightFromFile, FlightResolveReport, FlightResolveSource, FlightSource, + IngestStatusCut, + PerTileStatusCut, + PostLandingUploadRequest, ReadinessOutcome, ReadinessReport, + ReLocHint, RemoteBuildOutcome, RemoteBuildReport, SectorClassification, + UploadBatchReportCut, + UploadOutcomeCut, + UploadRequestCut, ) -from gps_denied_onboard.components.c12_operator_tooling.build_cache import ( +from gps_denied_onboard.components.c12_operator_orchestrator.build_cache import ( BuildCacheOrchestrator, ) -from gps_denied_onboard.components.c12_operator_tooling.companion_bringup import ( +from gps_denied_onboard.components.c12_operator_orchestrator.companion_bringup import ( CompanionBringup, ) -from gps_denied_onboard.components.c12_operator_tooling.config import ( +from gps_denied_onboard.components.c12_operator_orchestrator.config import ( C12BuildCacheConfig, C12CompanionConfig, C12Config, + C12PostLandingConfig, HostKeyPolicy, ) -from gps_denied_onboard.components.c12_operator_tooling.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( BuildLockHeldError, BuildReportParseError, CacheBuildError, CompanionUnreachableError, ContentHashMismatchError, + FdrUnreadableError, + FlightStateNotConfirmedError, + GcsLinkError, + NotConfirmedReason, ) -from gps_denied_onboard.components.c12_operator_tooling.exit_codes import ( +from gps_denied_onboard.components.c12_operator_orchestrator.fdr_footer_reader import ( + FdrFooterReader, + LocalFdrFooterReader, +) +from gps_denied_onboard.components.c12_operator_orchestrator.post_landing_upload import ( + PostLandingUploadOrchestrator, +) +from gps_denied_onboard.components.c12_operator_orchestrator.tile_uploader_cut import ( + TileUploaderCut, +) +from gps_denied_onboard.components.c12_operator_orchestrator.exit_codes import ( EXIT_BUILD_FAILURE, EXIT_COMPANION_UNREACHABLE, EXIT_CONTENT_HASH_MISMATCH, @@ -89,13 +112,13 @@ from gps_denied_onboard.components.c12_operator_tooling.exit_codes import ( EXIT_UPLOAD_FAILURE, EXIT_USAGE, ) -from gps_denied_onboard.components.c12_operator_tooling.file_lock import ( +from gps_denied_onboard.components.c12_operator_orchestrator.file_lock import ( FileLock, FileLockFactory, FilelockFileLockFactory, LockTimeout, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( EmptyWaypointsError, FlightFileNotFoundError, FlightNotFoundError, @@ -105,59 +128,64 @@ from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors impor FlightsApiUnreachableError, WaypointSchemaError, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.file_loader import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.file_loader import ( load_flight_file, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, FlightsApiClient, WaypointDto, WaypointObjective, WaypointSource, ) -from gps_denied_onboard.components.c12_operator_tooling.freshness_table import ( +from gps_denied_onboard.components.c12_operator_orchestrator.freshness_table import ( FRESHNESS_TABLE, freshness_threshold_months, ) -from gps_denied_onboard.components.c12_operator_tooling.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.interface import ( CacheBuildWorkflow, +) +from gps_denied_onboard.components.c12_operator_orchestrator.operator_command_transport import ( + OperatorCommandTransport, +) +from gps_denied_onboard.components.c12_operator_orchestrator.operator_reloc_service import ( OperatorReLocService, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_c10_invoker import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_c10_invoker import ( RemoteBuildRequest, RemoteCacheProvisionerInvoker, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_sidecar_verifier import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_sidecar_verifier import ( RemoteSidecarResult, RemoteSidecarVerifier, ) -from gps_denied_onboard.components.c12_operator_tooling.sector_classification_store import ( +from gps_denied_onboard.components.c12_operator_orchestrator.sector_classification_store import ( SectorClassificationStore, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( RemoteCommandResult, SshSession, SshSessionFactory, ) -from gps_denied_onboard.components.c12_operator_tooling.tile_downloader_cut import ( +from gps_denied_onboard.components.c12_operator_orchestrator.tile_downloader_cut import ( TileDownloaderCut, ) from gps_denied_onboard.config.schema import register_component_block if TYPE_CHECKING: - from gps_denied_onboard.components.c12_operator_tooling.flights_api.bbox import ( + from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.bbox import ( bbox_from_waypoints, takeoff_origin_from_flight, ) - from gps_denied_onboard.components.c12_operator_tooling.flights_api.httpx_client import ( + from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.httpx_client import ( HttpxFlightsApiClient, ) - from gps_denied_onboard.components.c12_operator_tooling.paramiko_ssh_session import ( + from gps_denied_onboard.components.c12_operator_orchestrator.paramiko_ssh_session import ( ParamikoSshSession, ParamikoSshSessionFactory, ) -register_component_block("c12_operator_tooling", C12Config) +register_component_block("c12_operator_orchestrator", C12Config) # --------------------------------------------------------------------------- # PEP 562 lazy re-exports for heavy adapters @@ -172,23 +200,23 @@ register_component_block("c12_operator_tooling", C12Config) _LAZY_NAMES: dict[str, tuple[str, str]] = { "HttpxFlightsApiClient": ( - "gps_denied_onboard.components.c12_operator_tooling.flights_api.httpx_client", + "gps_denied_onboard.components.c12_operator_orchestrator.flights_api.httpx_client", "HttpxFlightsApiClient", ), "ParamikoSshSession": ( - "gps_denied_onboard.components.c12_operator_tooling.paramiko_ssh_session", + "gps_denied_onboard.components.c12_operator_orchestrator.paramiko_ssh_session", "ParamikoSshSession", ), "ParamikoSshSessionFactory": ( - "gps_denied_onboard.components.c12_operator_tooling.paramiko_ssh_session", + "gps_denied_onboard.components.c12_operator_orchestrator.paramiko_ssh_session", "ParamikoSshSessionFactory", ), "bbox_from_waypoints": ( - "gps_denied_onboard.components.c12_operator_tooling.flights_api.bbox", + "gps_denied_onboard.components.c12_operator_orchestrator.flights_api.bbox", "bbox_from_waypoints", ), "takeoff_origin_from_flight": ( - "gps_denied_onboard.components.c12_operator_tooling.flights_api.bbox", + "gps_denied_onboard.components.c12_operator_orchestrator.flights_api.bbox", "takeoff_origin_from_flight", ), } @@ -234,6 +262,7 @@ __all__ = [ "C12BuildCacheConfig", "C12CompanionConfig", "C12Config", + "C12PostLandingConfig", "CacheBuildError", "CacheBuildReport", "CacheBuildWorkflow", @@ -247,28 +276,41 @@ __all__ = [ "DownloadRequestCut", "EmptyWaypointsError", "FailurePhase", + "FdrFooterReader", + "FdrUnreadableError", "FileLock", "FileLockFactory", "FilelockFileLockFactory", "FlightById", "FlightDto", "FlightFileNotFoundError", + "FlightFooterRecord", "FlightFromFile", "FlightNotFoundError", "FlightResolveReport", "FlightResolveSource", "FlightSource", + "FlightStateNotConfirmedError", "FlightsApiAuthError", "FlightsApiClient", "FlightsApiError", "FlightsApiSchemaError", "FlightsApiUnreachableError", + "GcsLinkError", "HostKeyPolicy", "HttpxFlightsApiClient", + "IngestStatusCut", + "LocalFdrFooterReader", "LockTimeout", + "NotConfirmedReason", + "OperatorCommandTransport", "OperatorReLocService", "ParamikoSshSession", "ParamikoSshSessionFactory", + "PerTileStatusCut", + "PostLandingUploadOrchestrator", + "PostLandingUploadRequest", + "ReLocHint", "ReadinessOutcome", "ReadinessReport", "RemoteBuildOutcome", @@ -283,6 +325,10 @@ __all__ = [ "SshSession", "SshSessionFactory", "TileDownloaderCut", + "TileUploaderCut", + "UploadBatchReportCut", + "UploadOutcomeCut", + "UploadRequestCut", "WaypointDto", "WaypointObjective", "WaypointSchemaError", diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/__main__.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/__main__.py similarity index 64% rename from src/gps_denied_onboard/components/c12_operator_tooling/__main__.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/__main__.py index 1d746e3..0a21bfc 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/__main__.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/__main__.py @@ -1,6 +1,6 @@ -"""Module entry point for ``python -m gps_denied_onboard.components.c12_operator_tooling``. +"""Module entry point for ``python -m gps_denied_onboard.components.c12_operator_orchestrator``. -The console script declared in ``pyproject.toml`` (``operator-tool``) +The console script declared in ``pyproject.toml`` (``operator-orchestrator``) points at :func:`cli.main` directly; this module is the convenience entry for ``python -m ...`` invocations during development and for operators who prefer the explicit form. @@ -8,7 +8,7 @@ operators who prefer the explicit form. from __future__ import annotations -from gps_denied_onboard.components.c12_operator_tooling.cli import main +from gps_denied_onboard.components.c12_operator_orchestrator.cli import main if __name__ == "__main__": raise SystemExit(main()) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/_types.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/_types.py similarity index 62% rename from src/gps_denied_onboard/components/c12_operator_tooling/_types.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/_types.py index 8fa8d99..15ea218 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/_types.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/_types.py @@ -1,4 +1,4 @@ -"""C12 operator-tooling shared DTOs / enums (AZ-326, AZ-327, AZ-328). +"""C12 operator-orchestrator shared DTOs / enums (AZ-326, AZ-327, AZ-328). ``SectorClassification`` is declared locally — c12 must not import the c6 / c10 / c11 enums (AZ-507 / module-layout cross-component rule); the @@ -26,7 +26,7 @@ from pathlib import Path from uuid import UUID from gps_denied_onboard._types.geo import BoundingBox, LatLonAlt -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, ) @@ -42,15 +42,23 @@ __all__ = [ "DownloadRequestCut", "FailurePhase", "FlightById", + "FlightFooterRecord", "FlightFromFile", "FlightResolveReport", "FlightResolveSource", "FlightSource", + "IngestStatusCut", + "PerTileStatusCut", + "PostLandingUploadRequest", + "ReLocHint", "ReadinessOutcome", "ReadinessReport", "RemoteBuildOutcome", "RemoteBuildReport", "SectorClassification", + "UploadBatchReportCut", + "UploadOutcomeCut", + "UploadRequestCut", ] @@ -63,7 +71,7 @@ AreaIdentifier = str class SectorClassification(str, Enum): """Operator-set classification of a geographic sector (AZ-326). - Mirrors the c6 enum at the c12 boundary so the operator-tool never + Mirrors the c6 enum at the c12 boundary so the operator-orchestrator never imports ``components.c6_tile_cache``. The string values are identical so the composition root can round-trip via ``.value``. """ @@ -83,7 +91,7 @@ class CompanionUnreachableReason(str, Enum): """SSH-session-open failure category (AZ-327). Drives the per-reason ``remediation`` hint on - :class:`~gps_denied_onboard.components.c12_operator_tooling.errors.CompanionUnreachableError`. + :class:`~gps_denied_onboard.components.c12_operator_orchestrator.errors.CompanionUnreachableError`. """ CONNECT_REFUSED = "connect_refused" @@ -226,7 +234,7 @@ class FlightResolveReport: # --------------------------------------------------------------------------- # Consumer-side structural cuts of C11 shapes (AZ-507) # -# c12_operator_tooling MAY NOT import from c11_tile_manager directly. The +# c12_operator_orchestrator MAY NOT import from c11_tile_manager directly. The # composition root maps these local cuts to / from the real c11 DTOs at # the wiring boundary (``runtime_root.c12_factory``). # --------------------------------------------------------------------------- @@ -300,6 +308,162 @@ class RemoteBuildReport: elapsed_s: float +# --------------------------------------------------------------------------- +# AZ-329: PostLandingUploadOrchestrator surface +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True, slots=True) +class PostLandingUploadRequest: + """Operator-supplied input to :meth:`PostLandingUploadOrchestrator.trigger_post_landing_upload` (AZ-329). + + The orchestrator inspects the C13 ``flight_footer`` record for + ``flight_id`` and, if found with ``clean_shutdown=True``, delegates + the upload to a c11 :class:`TileUploaderCut` collaborator. ``api_key`` + is plain :class:`str` for consistency with + :class:`BuildCacheRequest.api_key`; the CLI redacts it (``"REDACTED"``) + in the ``operator invoked subcommand`` log record and the orchestrator + never includes it in any log payload (AC-8). + + ``batch_size`` defaults to 50 — the same default the c11 + ``UploadRequest`` carries — and is bounded to ``[1, 200]`` by C11's + own ``__post_init__`` validation; this DTO does NOT re-validate. + """ + + flight_id: UUID + satellite_provider_url: str + api_key: str + batch_size: int = 50 + + +@dataclass(frozen=True, slots=True) +class FlightFooterRecord: + """C12-local mirror of the C13 ``flight_footer`` payload (AZ-292). + + Owned by C12 to preserve the c12 ↔ c13 cross-component cut — this + task does NOT import :class:`c13_fdr.headers.FlightFooter`. Only the + fields the orchestrator inspects (``clean_shutdown`` + the four + AC-NEW-3 counters) are mirrored; the orchestrator never touches + ``flight_ended_at_monotonic_ns`` because the operator workstation + does not share the airborne monotonic clock. + """ + + flight_id: UUID + flight_ended_at_iso: str + records_written: int + records_dropped_overrun: int + bytes_written: int + rollover_count: int + clean_shutdown: bool + + +# --------------------------------------------------------------------------- +# Consumer-side structural cuts of C11 TileUploader shapes (AZ-507) +# +# AZ-329 + AZ-330 forbid importing ``c11_tile_manager`` directly from +# c12. The composition root translates between the local cuts and the +# real C11 DTOs at the wiring boundary (``runtime_root.c12_factory``). +# --------------------------------------------------------------------------- + + +class IngestStatusCut(str, Enum): + """Mirror of c11 ``IngestStatus`` for C12's consumer-side cut.""" + + ACCEPTED = "accepted" + REJECTED = "rejected" + + +class UploadOutcomeCut(str, Enum): + """Mirror of c11 ``UploadOutcome`` for C12's consumer-side cut.""" + + SUCCESS = "success" + PARTIAL = "partial" + FAILURE = "failure" + + +@dataclass(frozen=True, slots=True) +class UploadRequestCut: + """C12-local mirror of c11 ``UploadRequest`` (AZ-507 cut). + + ``flight_id`` is required here (C12 always issues per-flight + uploads); the c11 DTO allows ``None`` for the "all pending across + every flight" path used elsewhere. The composition-root mapper + forwards this UUID into c11's ``UploadRequest.flight_id``. + """ + + flight_id: UUID + batch_size: int + satellite_provider_url: str + + +@dataclass(frozen=True, slots=True) +class PerTileStatusCut: + """C12-local mirror of c11 ``PerTileStatus`` (AZ-507 cut).""" + + tile_id: str + status: IngestStatusCut + rejection_reason: str | None = None + + +# --------------------------------------------------------------------------- +# AZ-330: OperatorReLocService surface +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True, slots=True) +class ReLocHint: + """Operator-supplied position hint for AC-3.4 re-localization (AZ-330). + + ``approximate_position_wgs84`` reuses the shared + :class:`gps_denied_onboard._types.geo.LatLonAlt` DTO (per the + cross-cutting rule); the shared shape has no range validation, so + this DTO validates lat/lon at construction (AC-7). + ``confidence_radius_m`` must be strictly positive (AC-3); + ``reason`` must be non-empty (AC-6). The full DTO is persisted to + FDR un-redacted; the live log redacts (rounds lat/lon to 5 decimals, + truncates ``reason`` to 200 chars) — see AC-9 + AC-4. + """ + + approximate_position_wgs84: LatLonAlt + confidence_radius_m: float + reason: str + + def __post_init__(self) -> None: + lat = self.approximate_position_wgs84.lat_deg + lon = self.approximate_position_wgs84.lon_deg + if not -90.0 <= lat <= 90.0: + raise ValueError( + f"approximate_position_wgs84.lat_deg must be in [-90, 90]; got {lat}" + ) + if not -180.0 < lon <= 180.0: + raise ValueError( + f"approximate_position_wgs84.lon_deg must be in (-180, 180]; got {lon}" + ) + if not self.confidence_radius_m > 0: + raise ValueError( + f"confidence_radius_m must be > 0; got {self.confidence_radius_m}" + ) + if not self.reason: + raise ValueError("reason must be non-empty") + + +@dataclass(frozen=True, slots=True) +class UploadBatchReportCut: + """C12-local mirror of c11 ``UploadBatchReport`` (AZ-507 cut). + + The orchestrator returns this passthrough; the composition root + maps c11's real ``UploadBatchReport`` into this cut at the wiring + boundary so c12 source never imports from c11. + """ + + batch_uuid: UUID + per_tile_status: tuple[PerTileStatusCut, ...] + retry_count: int + next_retry_at_s: int | None + outcome: UploadOutcomeCut + public_key_fingerprint: str + + @dataclass(frozen=True, slots=True) class CacheBuildReport: """Aggregated result of one :meth:`BuildCacheOrchestrator.build_cache` call. diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/build_cache.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/build_cache.py similarity index 96% rename from src/gps_denied_onboard/components/c12_operator_tooling/build_cache.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/build_cache.py index 292c5ee..c4122eb 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/build_cache.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/build_cache.py @@ -31,7 +31,7 @@ import logging from collections.abc import Callable from gps_denied_onboard.clock import Clock -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( BuildCacheOutcome, BuildCacheRequest, CacheBuildReport, @@ -47,24 +47,24 @@ from gps_denied_onboard.components.c12_operator_tooling._types import ( RemoteBuildReport, SectorClassification, ) -from gps_denied_onboard.components.c12_operator_tooling.companion_bringup import ( +from gps_denied_onboard.components.c12_operator_orchestrator.companion_bringup import ( CompanionBringup, ) -from gps_denied_onboard.components.c12_operator_tooling.config import ( +from gps_denied_onboard.components.c12_operator_orchestrator.config import ( C12BuildCacheConfig, ) -from gps_denied_onboard.components.c12_operator_tooling.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( BuildLockHeldError, BuildReportParseError, CacheBuildError, CompanionUnreachableError, ContentHashMismatchError, ) -from gps_denied_onboard.components.c12_operator_tooling.file_lock import ( +from gps_denied_onboard.components.c12_operator_orchestrator.file_lock import ( FileLockFactory, LockTimeout, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( EmptyWaypointsError, FlightFileNotFoundError, FlightNotFoundError, @@ -74,21 +74,21 @@ from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors impor FlightsApiUnreachableError, WaypointSchemaError, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, FlightsApiClient, ) -from gps_denied_onboard.components.c12_operator_tooling.freshness_table import ( +from gps_denied_onboard.components.c12_operator_orchestrator.freshness_table import ( freshness_threshold_months as _default_freshness_threshold, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_c10_invoker import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_c10_invoker import ( RemoteBuildRequest, RemoteCacheProvisionerInvoker, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( SshSessionFactory, ) -from gps_denied_onboard.components.c12_operator_tooling.tile_downloader_cut import ( +from gps_denied_onboard.components.c12_operator_orchestrator.tile_downloader_cut import ( TileDownloaderCut, ) @@ -151,7 +151,7 @@ _BUILD_RECOGNISED_NAMES: frozenset[str] = frozenset( class BuildCacheOrchestrator: """F1 pre-flight cache-build orchestrator (AZ-328). - Constructed once per ``OperatorToolServices`` from the composition + Constructed once per ``OperatorOrchestratorServices`` from the composition root; the CLI ``build-cache`` subcommand resolves it from the services dataclass and calls :meth:`build_cache` exactly once per invocation. diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/cli.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/cli.py similarity index 82% rename from src/gps_denied_onboard/components/c12_operator_tooling/cli.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/cli.py index b5de996..fde1aad 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/cli.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/cli.py @@ -1,4 +1,4 @@ -"""``operator-tool`` CLI shell — Click app + six subcommands (AZ-326). +"""``operator-orchestrator`` CLI shell — Click app + six subcommands (AZ-326). The task spec calls for a Typer-based shell. Typer is not pinned by the project (only ``click>=8.1`` is in ``pyproject.toml``); the spec's @@ -37,7 +37,7 @@ from uuid import UUID import click -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( BuildCacheOutcome, BuildCacheRequest, CacheBuildReport, @@ -46,18 +46,23 @@ from gps_denied_onboard.components.c12_operator_tooling._types import ( FlightById, FlightFromFile, FlightSource, + PostLandingUploadRequest, + ReLocHint, SectorClassification, ) -from gps_denied_onboard.components.c12_operator_tooling.config import ( +from gps_denied_onboard.components.c12_operator_orchestrator.config import ( C12Config, ) -from gps_denied_onboard.components.c12_operator_tooling.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( BuildLockHeldError, CacheBuildError, CompanionUnreachableError, ContentHashMismatchError, + FlightStateNotConfirmedError, + GcsLinkError, ) -from gps_denied_onboard.components.c12_operator_tooling.exit_codes import ( +from gps_denied_onboard._types.geo import LatLonAlt +from gps_denied_onboard.components.c12_operator_orchestrator.exit_codes import ( EXIT_BUILD_FAILURE, EXIT_COMPANION_UNREACHABLE, EXIT_CONTENT_HASH_MISMATCH, @@ -78,7 +83,7 @@ from gps_denied_onboard.components.c12_operator_tooling.exit_codes import ( # Import flights_api types from leaf modules — going through the # ``flights_api`` package ``__init__.py`` would eagerly load ``bbox.py`` # which pulls in numpy / pyproj (NFR-perf-cold-start regression). -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( EmptyWaypointsError, FlightFileNotFoundError, FlightNotFoundError, @@ -87,7 +92,7 @@ from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors impor FlightsApiUnreachableError, WaypointSchemaError, ) -from gps_denied_onboard.components.c12_operator_tooling.sector_classification_store import ( +from gps_denied_onboard.components.c12_operator_orchestrator.sector_classification_store import ( SectorClassificationStore, ) from gps_denied_onboard.logging import JsonFormatter @@ -97,7 +102,7 @@ __all__ = ["app", "build_app", "main"] # Service-collaborator placeholder for sibling tasks. Each subcommand # resolves its concrete collaborator via a factory the test injects; -# production wiring lives in runtime_root.c12_factory.OperatorToolServices. +# production wiring lives in runtime_root.c12_factory.OperatorOrchestratorServices. ServiceFactory = Callable[[], Any] @@ -110,7 +115,7 @@ _LOG_KIND_OK = "c12.cli.ok" _LOG_KIND_ERROR = "c12.cli.error" _LOG_KIND_USAGE = "c12.cli.usage" -_CLI_LOGGER_NAME = "c12_operator_tooling.cli" +_CLI_LOGGER_NAME = "c12_operator_orchestrator.cli" _HANDLER_MARKER = "_c12_cli_file_handler" @@ -254,7 +259,7 @@ _FLIGHTS_API_HINTS: dict[type, tuple[int, str]] = { @click.group( - name="operator-tool", + name="operator-orchestrator", help="GPS-denied onboard pre-flight tooling (operator workstation).", ) @click.option( @@ -514,22 +519,93 @@ def build_cache( "upload-pending", help="Trigger post-landing upload of pending tiles (AC-NEW-7).", ) +@click.option( + "--flight-id", + type=str, + required=True, + help="UUID of the flight whose pending tiles should be uploaded.", +) +@click.option( + "--satellite-provider-url", + type=str, + required=True, + help="Parent-suite ingest endpoint base URL.", +) +@click.option( + "--api-key", + type=str, + required=True, + help="Parent-suite ingest API key (NEVER logged; AC-8 redaction guarantee).", +) +@click.option( + "--batch-size", + type=int, + default=50, + show_default=True, + help="Tiles per ingest POST (forwarded to C11 UploadRequest).", +) @click.pass_context -def upload_pending(ctx: click.Context) -> None: - """Delegates to ``post_landing_upload.trigger_post_landing_upload`` (AZ-329).""" +def upload_pending( + ctx: click.Context, + flight_id: str, + satellite_provider_url: str, + api_key: str, + batch_size: int, +) -> None: + """Delegate to ``post_landing_upload_orchestrator.trigger_post_landing_upload`` (AZ-329).""" state = ctx.obj logger = state["logger"] - _emit_invoked(logger, "upload-pending") + _emit_invoked( + logger, + "upload-pending", + { + "flight_id": flight_id, + "satellite_provider_url": satellite_provider_url, + "api_key": "REDACTED", + "batch_size": batch_size, + }, + ) services = state.get("services") - if services is None or not hasattr(services, "post_landing_upload"): + if services is None or not hasattr(services, "post_landing_upload_orchestrator"): _emit_ok( logger, "upload-pending", - {"note": "no post_landing_upload wired (sibling AZ-329)"}, + {"note": "no post_landing_upload_orchestrator wired (composition-root pending)"}, ) ctx.exit(EXIT_OK) + orchestrator = services.post_landing_upload_orchestrator + if orchestrator is None: + _emit_ok( + logger, + "upload-pending", + {"note": "post_landing_upload_orchestrator is None (no tile_uploader wired)"}, + ) + ctx.exit(EXIT_OK) + request = PostLandingUploadRequest( + flight_id=UUID(flight_id), + satellite_provider_url=satellite_provider_url, + api_key=api_key, + batch_size=batch_size, + ) try: - services.post_landing_upload.trigger_post_landing_upload() + orchestrator.trigger_post_landing_upload(request) + except FlightStateNotConfirmedError as exc: + _emit_error( + logger, + "upload-pending", + exit_code=EXIT_FLIGHT_STATE_NOT_CONFIRMED, + exception=exc, + remediation=exc.remediation, + kv={ + "flight_id": flight_id, + "not_confirmed_reason": exc.not_confirmed_reason, + }, + ) + click.echo( + f"upload refused ({exc.not_confirmed_reason}): {exc.remediation}", + err=True, + ) + ctx.exit(EXIT_FLIGHT_STATE_NOT_CONFIRMED) except Exception as exc: _handle_known_exception( ctx, @@ -537,10 +613,6 @@ def upload_pending(ctx: click.Context) -> None: "upload-pending", exc, extra_table={ - "FlightStateNotConfirmedError": ( - EXIT_FLIGHT_STATE_NOT_CONFIRMED, - "Flight state has not been confirmed yet; retry after landing is logged.", - ), "UploadGateBlockedError": ( EXIT_UPLOAD_FAILURE, "Upload gate blocked the request; consult c11 logs for details.", @@ -548,7 +620,7 @@ def upload_pending(ctx: click.Context) -> None: }, ) return - _emit_ok(logger, "upload-pending") + _emit_ok(logger, "upload-pending", {"flight_id": flight_id}) ctx.exit(EXIT_OK) @@ -556,37 +628,84 @@ def upload_pending(ctx: click.Context) -> None: "reloc-confirm", help="Request operator-driven re-localization via GCS (AC-3.4, AC-7.3).", ) -@click.option("--hint", default="", help="Optional textual hint forwarded to the GCS link.") +@click.option("--lat", type=float, required=True, help="WGS84 latitude in degrees (-90..90).") +@click.option("--lon", type=float, required=True, help="WGS84 longitude in degrees (-180..180].") +@click.option("--alt", type=float, required=True, help="WGS84 ellipsoidal altitude in metres.") +@click.option( + "--radius", + type=float, + required=True, + help="Operator confidence radius in metres (must be > 0).", +) +@click.option( + "--reason", + type=str, + required=True, + help="Free-text operator note explaining the re-loc decision (non-empty).", +) @click.pass_context -def reloc_confirm(ctx: click.Context, hint: str) -> None: - """Delegates to ``operator_reloc_service.request_relocalization`` (AZ-330).""" +def reloc_confirm( + ctx: click.Context, + lat: float, + lon: float, + alt: float, + radius: float, + reason: str, +) -> None: + """Delegates to ``operator_reloc_service.request_reloc`` (AZ-330).""" state = ctx.obj logger = state["logger"] - _emit_invoked(logger, "reloc-confirm", {"hint": hint}) + # AC-4 + AC-9: log-side redaction at the CLI boundary mirrors the + # service redaction so the invoked-event line and the sent-event + # line agree on what's redacted. + _emit_invoked( + logger, + "reloc-confirm", + { + "position_lat": round(lat, 5), + "position_lon": round(lon, 5), + "altitude_m": alt, + "confidence_radius_m": radius, + "reason": reason[:200], + }, + ) services = state.get("services") if services is None or not hasattr(services, "operator_reloc_service"): _emit_ok( logger, "reloc-confirm", - {"note": "no operator_reloc_service wired (sibling AZ-330)"}, + {"note": "no operator_reloc_service wired (composition-root pending)"}, + ) + ctx.exit(EXIT_OK) + reloc_service = services.operator_reloc_service + if reloc_service is None: + _emit_ok( + logger, + "reloc-confirm", + {"note": "operator_reloc_service is None (no transport wired)"}, ) ctx.exit(EXIT_OK) try: - services.operator_reloc_service.request_relocalization(hint=hint) - except Exception as exc: - _handle_known_exception( - ctx, + hint = ReLocHint( + approximate_position_wgs84=LatLonAlt(lat_deg=lat, lon_deg=lon, alt_m=alt), + confidence_radius_m=radius, + reason=reason, + ) + except ValueError as exc: + _exit_with_usage(ctx, logger, "reloc-confirm", str(exc)) + try: + reloc_service.request_reloc(hint) + except GcsLinkError as exc: + _emit_error( logger, "reloc-confirm", - exc, - extra_table={ - "GcsLinkError": ( - EXIT_GCS_LINK_ERROR, - "GCS link unavailable; check pymavlink connectivity and signing key.", - ), - }, + exit_code=EXIT_GCS_LINK_ERROR, + exception=exc, + remediation=exc.remediation, + kv={"failure_reason": exc.reason}, ) - return + click.echo(f"GcsLinkError: {exc.remediation}", err=True) + ctx.exit(EXIT_GCS_LINK_ERROR) _emit_ok(logger, "reloc-confirm") ctx.exit(EXIT_OK) @@ -600,7 +719,7 @@ def reloc_confirm(ctx: click.Context, hint: str) -> None: @click.pass_context def verify_ready(ctx: click.Context, host: str, port: int) -> None: """Delegates to :class:`CompanionBringup.verify_companion_ready` (AZ-327).""" - from gps_denied_onboard.components.c12_operator_tooling._types import ( + from gps_denied_onboard.components.c12_operator_orchestrator._types import ( CompanionAddress, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/companion_bringup.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/companion_bringup.py similarity index 95% rename from src/gps_denied_onboard/components/c12_operator_tooling/companion_bringup.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/companion_bringup.py index cfd4696..c98edb0 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/companion_bringup.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/companion_bringup.py @@ -28,21 +28,21 @@ from __future__ import annotations import logging from pathlib import PurePosixPath -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( CompanionAddress, ReadinessOutcome, ReadinessReport, ) -from gps_denied_onboard.components.c12_operator_tooling.config import ( +from gps_denied_onboard.components.c12_operator_orchestrator.config import ( C12CompanionConfig, ) -from gps_denied_onboard.components.c12_operator_tooling.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( ContentHashMismatchError, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_sidecar_verifier import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_sidecar_verifier import ( RemoteSidecarVerifier, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( SshSession, SshSessionFactory, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/config.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/config.py similarity index 87% rename from src/gps_denied_onboard/components/c12_operator_tooling/config.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/config.py index 214f09a..e7e9c1d 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/config.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/config.py @@ -1,10 +1,10 @@ -"""C12 operator-tooling config block (AZ-326, AZ-327). +"""C12 operator-orchestrator config block (AZ-326, AZ-327). -Registered into ``config.components['c12_operator_tooling']`` by the +Registered into ``config.components['c12_operator_orchestrator']`` by the package ``__init__.py``. Two composition-root factories read this block: -* :func:`gps_denied_onboard.runtime_root.c12_factory.build_operator_tool` +* :func:`gps_denied_onboard.runtime_root.c12_factory.build_operator_orchestrator` reads the workstation-side service knobs (log path, sector classification store path). * :func:`gps_denied_onboard.runtime_root.c12_factory.build_companion_bringup` @@ -30,6 +30,7 @@ __all__ = [ "C12BuildCacheConfig", "C12CompanionConfig", "C12Config", + "C12PostLandingConfig", "HostKeyPolicy", ] @@ -51,6 +52,7 @@ _DEFAULT_LOG_PATH = Path("~/.azaion/onboard/c12-tooling.log").expanduser() _DEFAULT_SECTOR_STORE_PATH = Path("~/.azaion/onboard/sector-classifications.json").expanduser() _DEFAULT_COMPANION_CACHE_ROOT = PurePosixPath("/var/lib/azaion/c10/cache") _DEFAULT_CACHE_STAGING_ROOT = Path("~/.azaion/onboard/cache-staging").expanduser() +_DEFAULT_FDR_ROOT = Path("~/.azaion/onboard/fdr").expanduser() _DEFAULT_CONNECT_TIMEOUT_S = 10.0 _DEFAULT_SHA256SUM_TIMEOUT_S = 60.0 _DEFAULT_LOCK_TIMEOUT_S = 5.0 @@ -158,9 +160,22 @@ class C12BuildCacheConfig: raise ConfigError("C12BuildCacheConfig.lock_filename must be non-empty") +@dataclass(frozen=True) +class C12PostLandingConfig: + """Knobs consumed by :class:`PostLandingUploadOrchestrator` (AZ-329). + + * ``fdr_root`` — workstation-side root directory under which + per-flight FDR sub-directories live (``//``). + ``LocalFdrFooterReader`` scans this for the ``flight_footer`` + record. Defaults to ``~/.azaion/onboard/fdr``. + """ + + fdr_root: Path = _DEFAULT_FDR_ROOT + + @dataclass(frozen=True) class C12Config: - """Per-component config for C12 operator tooling. + """Per-component config for C12 operator orchestrator. * ``log_path`` — workstation-side rotating log file fed by the AZ-266 :class:`JsonFormatter`. Defaults to @@ -172,12 +187,15 @@ class C12Config: * ``companion`` — nested AZ-327 SSH config block. * ``build_cache`` — nested AZ-328 orchestrator knobs (lockfile, flights service URL/token, bbox buffer). + * ``post_landing`` — nested AZ-329 orchestrator knobs + (``fdr_root``). """ log_path: Path = _DEFAULT_LOG_PATH sector_classification_store_path: Path = _DEFAULT_SECTOR_STORE_PATH companion: C12CompanionConfig = field(default_factory=C12CompanionConfig) build_cache: C12BuildCacheConfig = field(default_factory=C12BuildCacheConfig) + post_landing: C12PostLandingConfig = field(default_factory=C12PostLandingConfig) def __post_init__(self) -> None: if not isinstance(self.companion, C12CompanionConfig): @@ -190,3 +208,8 @@ class C12Config: "C12Config.build_cache must be a C12BuildCacheConfig; got " f"{type(self.build_cache).__name__}" ) + if not isinstance(self.post_landing, C12PostLandingConfig): + raise ConfigError( + "C12Config.post_landing must be a C12PostLandingConfig; got " + f"{type(self.post_landing).__name__}" + ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/errors.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/errors.py similarity index 65% rename from src/gps_denied_onboard/components/c12_operator_tooling/errors.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/errors.py index d9f7e75..cf7142c 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/errors.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/errors.py @@ -1,7 +1,7 @@ """C12 ``CompanionBringup`` error hierarchy (AZ-327, AZ-328). Two failure modes own dedicated exit codes in -:mod:`gps_denied_onboard.components.c12_operator_tooling.exit_codes`: +:mod:`gps_denied_onboard.components.c12_operator_orchestrator.exit_codes`: * :class:`CompanionUnreachableError` — SSH session-open failure. Mapped 1:1 from the underlying paramiko / socket exception via the @@ -25,7 +25,7 @@ AZ-328 adds the ``BuildCacheOrchestrator`` family: ``BuildReport`` JSON document; surfaced as ``failure_phase=build``. All errors expose a ``remediation`` property the -:func:`gps_denied_onboard.components.c12_operator_tooling.cli.main` +:func:`gps_denied_onboard.components.c12_operator_orchestrator.cli.main` layer reads to print a one-line operator-friendly hint to stderr. The flights-API errors (AZ-489) deliberately do NOT carry a @@ -38,18 +38,30 @@ discipline by keeping the hint table in c12. from __future__ import annotations from pathlib import Path +from typing import Literal -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( CompanionUnreachableReason, FailurePhase, ) +NotConfirmedReason = Literal[ + "flight_id_not_found", + "footer_missing", + "unclean_shutdown", + "fdr_unreadable", +] + __all__ = [ "BuildLockHeldError", "BuildReportParseError", "CacheBuildError", "CompanionUnreachableError", "ContentHashMismatchError", + "FdrUnreadableError", + "FlightStateNotConfirmedError", + "GcsLinkError", + "NotConfirmedReason", ] @@ -140,7 +152,7 @@ class ContentHashMismatchError(Exception): @property def remediation(self) -> str: return ( - "Re-run the cache build (`operator-tool build-cache --flight-id ...`) " + "Re-run the cache build (`operator-orchestrator build-cache --flight-id ...`) " "to repopulate the affected engine." ) @@ -227,7 +239,7 @@ class BuildLockHeldError(CacheBuildError): failure_phase=FailurePhase.DOWNLOAD, wrapped_exception_repr=f"LockTimeout(path={lock_path!s}, timeout_s={timeout_s})", message=( - f"build-cache lock held: another `operator-tool build-cache` is in " + f"build-cache lock held: another `operator-orchestrator build-cache` is in " f"progress (lock={lock_path}, waited {timeout_s:.1f} s)" ), remediation=( @@ -239,6 +251,127 @@ class BuildLockHeldError(CacheBuildError): self.timeout_s = timeout_s +# --------------------------------------------------------------------------- +# AZ-329: PostLandingUploadOrchestrator error family +# --------------------------------------------------------------------------- + + +_POST_LANDING_REMEDIATIONS: dict[str, str] = { + "flight_id_not_found": ( + "Verify // exists; check " + "`config.c12_operator_orchestrator.post_landing.fdr_root` and the " + "flight UUID." + ), + "footer_missing": ( + "No flight_footer record found in any segment — the flight likely " + "terminated abnormally (power loss, crash, or close_flight() never " + "ran). Inspect FDR manually; upload requires a clean shutdown." + ), + "unclean_shutdown": ( + "The flight footer reports an unclean shutdown. Operator must " + "manually verify the flight outcome before authorising tile upload." + ), + "fdr_unreadable": ( + "Inspect FDR segment files manually; the parser failed mid-stream. " + "The wrapped exception repr is on the error object's `detail` field." + ), +} + + +class FdrUnreadableError(Exception): + """Sibling exception raised by :class:`LocalFdrFooterReader` on I/O or parse failure. + + Caught at the :class:`PostLandingUploadOrchestrator` boundary and + rewrapped as :class:`FlightStateNotConfirmedError` with + ``not_confirmed_reason="fdr_unreadable"``. Operators do not see this + exception directly; the orchestrator's typed refusal is the + operator-facing contract. + """ + + def __init__(self, reason: str) -> None: + super().__init__(reason) + self.reason = reason + + +class FlightStateNotConfirmedError(Exception): + """Operator-side refusal raised by :class:`PostLandingUploadOrchestrator` (AZ-329). + + The four valid ``not_confirmed_reason`` values form a closed + :class:`NotConfirmedReason` ``Literal`` — operators script against + these values. Adding a new value requires Plan-cycle approval. + + * ``flight_id_not_found`` — ``//`` does not exist + * ``footer_missing`` — no ``flight_footer`` record anywhere in the FDR + * ``unclean_shutdown`` — footer present but ``clean_shutdown=False`` + * ``fdr_unreadable`` — I/O or parse error while scanning segments + + ``detail`` is reason-specific extra context: + * ``unclean_shutdown`` — carries the four AC-NEW-3 counter values + * ``fdr_unreadable`` — carries the inner :class:`FdrUnreadableError` repr + * Other reasons — empty string. + """ + + def __init__( + self, + *, + flight_id: str, + not_confirmed_reason: NotConfirmedReason, + detail: str = "", + ) -> None: + super().__init__( + f"flight state not confirmed: flight_id={flight_id} " + f"reason={not_confirmed_reason}" + + (f" detail={detail}" if detail else "") + ) + self.flight_id = flight_id + self.not_confirmed_reason: NotConfirmedReason = not_confirmed_reason + self.detail = detail + + @property + def remediation(self) -> str: + return _POST_LANDING_REMEDIATIONS[self.not_confirmed_reason] + + +# --------------------------------------------------------------------------- +# AZ-330: OperatorReLocService error family +# --------------------------------------------------------------------------- + + +_GCS_LINK_DEFAULT_REMEDIATION: str = ( + "Check GCS link signal strength; re-issue the re-loc command when " + "the link recovers." +) + + +class GcsLinkError(Exception): + """Raised when the GCS link transport cannot send the operator's re-loc hint. + + Producer: the concrete :class:`OperatorCommandTransport` (E-C8's + pymavlink-backed implementation, future task). Consumer: C12's + :class:`OperatorReLocService.request_reloc`, which catches and + re-raises with a ``"C12 reloc-confirm: "`` prefix while preserving + the original exception as ``__cause__``. Best-effort semantics — + the operator may need to re-issue manually; this layer does NOT + auto-retry. + """ + + def __init__( + self, + *, + reason: str, + wrapped_exception_repr: str | None = None, + remediation: str = _GCS_LINK_DEFAULT_REMEDIATION, + ) -> None: + super().__init__(f"gcs link error: {reason}") + self.reason = reason + self.wrapped_exception_repr = wrapped_exception_repr + self._remediation = remediation + + @property + def remediation(self) -> str: + return self._remediation + + class BuildReportParseError(CacheBuildError): """C10's companion-side stdout did not contain a parseable BuildReport JSON. diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/exit_codes.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/exit_codes.py similarity index 96% rename from src/gps_denied_onboard/components/c12_operator_tooling/exit_codes.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/exit_codes.py index e144f31..b00c8ca 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/exit_codes.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/exit_codes.py @@ -1,4 +1,4 @@ -"""Exit-code constants for the ``operator-tool`` console script (AZ-326). +"""Exit-code constants for the ``operator-orchestrator`` console script (AZ-326). The CLI shell maps each documented service-collaborator exception family to a specific exit code so operator scripts can branch on ``$?``. The diff --git a/src/gps_denied_onboard/components/c12_operator_orchestrator/fdr_footer_reader.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/fdr_footer_reader.py new file mode 100644 index 0000000..baed91c --- /dev/null +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/fdr_footer_reader.py @@ -0,0 +1,195 @@ +"""C12 FDR footer reader (AZ-329). + +Reads the C13-emitted ``flight_footer`` record (AZ-292) from a flight's +FDR segment directory, newest-segment-first, with bounded memory. The +reader is the orchestrator's collaborator — the post-landing +orchestrator (:class:`PostLandingUploadOrchestrator`) decides what to +do with the result (or its absence). + +Segment naming convention (matches C13's +:func:`c13_fdr.writer.FileFdrWriter._segment_path`): each closed segment +is written to ``//segment-NNNN.fdr`` where ``NNNN`` +is a zero-padded 4-digit integer. The reader sorts by the integer +index, not by filesystem mtime, so a concurrent rollover during +``close_flight()`` cannot misorder the scan. + +Frame format (matches C13's +:func:`c13_fdr.writer.FileFdrWriter._write_record_frame`): each record +is a 4-byte little-endian uint32 length prefix followed by the AZ-272 +``serialise(...)`` body. The reader reads one length, exactly that +many body bytes, parses, and either keeps walking or short-circuits on +a matching ``kind``. +""" + +from __future__ import annotations + +import re +import struct +from pathlib import Path +from typing import BinaryIO, Iterator, Protocol, runtime_checkable +from uuid import UUID + +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + FlightFooterRecord, +) +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( + FdrUnreadableError, +) +from gps_denied_onboard.fdr_client.records import FdrRecord, FdrSchemaError, parse + +__all__ = [ + "FdrFooterReader", + "LocalFdrFooterReader", +] + + +_LENGTH_PREFIX = struct.Struct(" FlightFooterRecord | None: ... + + +class LocalFdrFooterReader: + """On-disk implementation of :class:`FdrFooterReader`. + + Streams length-prefixed records from each segment file in + DESCENDING numerical order, parses via AZ-272's + :func:`fdr_client.records.parse`, and returns the first record whose + ``kind == "flight_footer"`` as a :class:`FlightFooterRecord` (the + c12-local mirror). The returned ``flight_id`` is asserted to match + the requested UUID; a mismatch raises :class:`FdrUnreadableError`. + """ + + def __init__(self, fdr_root: Path) -> None: + self._fdr_root = fdr_root + + def read_footer(self, flight_id: UUID) -> FlightFooterRecord | None: + flight_dir = self._fdr_root / str(flight_id) + for segment_path in self._iter_segments_newest_first(flight_dir): + footer = self._scan_segment_for_footer(segment_path, flight_id) + if footer is not None: + return footer + return None + + def _iter_segments_newest_first(self, flight_dir: Path) -> list[Path]: + # Sort by integer index parsed from `segment-NNNN.fdr`. Filesystem + # mtime is NOT reliable — a concurrent rollover during close_flight() + # could land the footer in a newer segment whose mtime is older + # than an in-progress write to the previous segment. + try: + entries = list(flight_dir.iterdir()) + except OSError as exc: + raise FdrUnreadableError( + f"failed to list FDR segment directory {flight_dir}: {exc!r}" + ) from exc + + indexed: list[tuple[int, Path]] = [] + for entry in entries: + if not entry.is_file(): + continue + match = _SEGMENT_FILENAME_RE.match(entry.name) + if match is None: + continue + indexed.append((int(match.group(1)), entry)) + indexed.sort(key=lambda pair: pair[0], reverse=True) + return [path for _index, path in indexed] + + def _scan_segment_for_footer( + self, segment_path: Path, expected_flight_id: UUID + ) -> FlightFooterRecord | None: + try: + handle: BinaryIO = open(segment_path, "rb") # noqa: SIM115 — manual close below + except OSError as exc: + raise FdrUnreadableError( + f"failed to open FDR segment {segment_path}: {exc!r}" + ) from exc + try: + for record in self._iter_records(handle, segment_path): + if record.kind == _FLIGHT_FOOTER_KIND: + return _build_footer_record(record, expected_flight_id) + return None + finally: + handle.close() + + def _iter_records( + self, handle: BinaryIO, segment_path: Path + ) -> Iterator[FdrRecord]: + prefix_size = _LENGTH_PREFIX.size + while True: + prefix = handle.read(prefix_size) + if not prefix: + return + if len(prefix) != prefix_size: + raise FdrUnreadableError( + f"truncated length prefix in {segment_path}: " + f"expected {prefix_size} bytes, got {len(prefix)}" + ) + (length,) = _LENGTH_PREFIX.unpack(prefix) + body = handle.read(length) + if len(body) != length: + raise FdrUnreadableError( + f"truncated record body in {segment_path}: " + f"expected {length} bytes, got {len(body)}" + ) + try: + yield parse(body) + except FdrSchemaError as exc: + raise FdrUnreadableError( + f"failed to parse record in {segment_path}: {exc!r}" + ) from exc + + +def _build_footer_record( + record: FdrRecord, expected_flight_id: UUID +) -> FlightFooterRecord: + payload = record.payload + try: + footer_flight_id_str = str(payload["flight_id"]) + flight_ended_at_iso = str(payload["flight_ended_at_iso"]) + records_written = int(payload["records_written"]) + records_dropped_overrun = int(payload["records_dropped_overrun"]) + bytes_written = int(payload["bytes_written"]) + rollover_count = int(payload["rollover_count"]) + clean_shutdown = bool(payload["clean_shutdown"]) + except (KeyError, TypeError, ValueError) as exc: + raise FdrUnreadableError( + f"flight_footer payload schema violation: {exc!r}" + ) from exc + + try: + footer_flight_id = UUID(footer_flight_id_str) + except (TypeError, ValueError) as exc: + raise FdrUnreadableError( + f"flight_footer.flight_id is not a UUID: {footer_flight_id_str!r}" + ) from exc + + if footer_flight_id != expected_flight_id: + raise FdrUnreadableError( + f"flight_footer.flight_id mismatch: footer={footer_flight_id}, " + f"requested={expected_flight_id}" + ) + + return FlightFooterRecord( + flight_id=footer_flight_id, + flight_ended_at_iso=flight_ended_at_iso, + records_written=records_written, + records_dropped_overrun=records_dropped_overrun, + bytes_written=bytes_written, + rollover_count=rollover_count, + clean_shutdown=clean_shutdown, + ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/file_lock.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/file_lock.py similarity index 95% rename from src/gps_denied_onboard/components/c12_operator_tooling/file_lock.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/file_lock.py index 038d226..3a20eb0 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/file_lock.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/file_lock.py @@ -1,7 +1,7 @@ """Workstation-side file-lock protocols + ``filelock``-backed concrete (AZ-328). The C12 ``BuildCacheOrchestrator`` acquires ``cache_staging_root/.c12.lock`` -to serialise concurrent operator runs of ``operator-tool build-cache`` +to serialise concurrent operator runs of ``operator-orchestrator build-cache`` (description.md § 7). C10's own lockfile lives on the companion under ``companion_cache_root/.c10.lock`` (CP-INV-4) — these are independent; the workstation lock prevents two workstation processes from racing on @@ -10,7 +10,7 @@ from racing on the engines+manifest root. Why a separate factory rather than reusing c10's: the AZ-507 cross- component rule forbids importing ``c10_provisioning`` from -``c12_operator_tooling``. Both factories thinly wrap the same +``c12_operator_orchestrator``. Both factories thinly wrap the same ``filelock`` library; the contract Protocol below is the consumer-side cut for c12. diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/__init__.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/__init__.py similarity index 78% rename from src/gps_denied_onboard/components/c12_operator_tooling/flights_api/__init__.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/__init__.py index 895b2b4..9c1fc3a 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/__init__.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/__init__.py @@ -18,7 +18,7 @@ Two sources produce the same DTO shape: * :meth:`FlightsApiClient.load_flight_file` — JSON on disk (offline path). Public surface is frozen by -``_docs/02_document/contracts/c12_operator_tooling/flights_api_client.md`` +``_docs/02_document/contracts/c12_operator_orchestrator/flights_api_client.md`` v1.0.0. NOTE on lazy imports (AZ-326 NFR-perf-cold-start): :class:`HttpxFlightsApiClient` @@ -33,7 +33,7 @@ from __future__ import annotations from typing import TYPE_CHECKING, Any -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( EmptyWaypointsError, FlightFileNotFoundError, FlightNotFoundError, @@ -43,10 +43,10 @@ from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors impor FlightsApiUnreachableError, WaypointSchemaError, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.file_loader import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.file_loader import ( load_flight_file, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, FlightsApiClient, WaypointDto, @@ -55,26 +55,26 @@ from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface im ) if TYPE_CHECKING: - from gps_denied_onboard.components.c12_operator_tooling.flights_api.bbox import ( + from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.bbox import ( bbox_from_waypoints, takeoff_origin_from_flight, ) - from gps_denied_onboard.components.c12_operator_tooling.flights_api.httpx_client import ( + from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.httpx_client import ( HttpxFlightsApiClient, ) _LAZY_NAMES: dict[str, tuple[str, str]] = { "HttpxFlightsApiClient": ( - "gps_denied_onboard.components.c12_operator_tooling.flights_api.httpx_client", + "gps_denied_onboard.components.c12_operator_orchestrator.flights_api.httpx_client", "HttpxFlightsApiClient", ), "bbox_from_waypoints": ( - "gps_denied_onboard.components.c12_operator_tooling.flights_api.bbox", + "gps_denied_onboard.components.c12_operator_orchestrator.flights_api.bbox", "bbox_from_waypoints", ), "takeoff_origin_from_flight": ( - "gps_denied_onboard.components.c12_operator_tooling.flights_api.bbox", + "gps_denied_onboard.components.c12_operator_orchestrator.flights_api.bbox", "takeoff_origin_from_flight", ), } diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/_parser.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/_parser.py similarity index 97% rename from src/gps_denied_onboard/components/c12_operator_tooling/flights_api/_parser.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/_parser.py index d5d02bd..6bd9436 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/_parser.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/_parser.py @@ -10,11 +10,11 @@ import math from typing import Any from uuid import UUID -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( FlightsApiSchemaError, WaypointSchemaError, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, WaypointDto, WaypointObjective, diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/bbox.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/bbox.py similarity index 94% rename from src/gps_denied_onboard/components/c12_operator_tooling/flights_api/bbox.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/bbox.py index ac5e5e0..2ce01ae 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/bbox.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/bbox.py @@ -11,10 +11,10 @@ import math import numpy as np from gps_denied_onboard._types.geo import BoundingBox, LatLonAlt -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( EmptyWaypointsError, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, WaypointDto, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/errors.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/errors.py similarity index 96% rename from src/gps_denied_onboard/components/c12_operator_tooling/flights_api/errors.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/errors.py index d1c18ed..e1a6f7e 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/errors.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/errors.py @@ -1,7 +1,7 @@ """C12 ``FlightsApiClient`` error hierarchy (AZ-489). Mapped 1:1 to the failure modes in the -``_docs/02_document/contracts/c12_operator_tooling/flights_api_client.md`` +``_docs/02_document/contracts/c12_operator_orchestrator/flights_api_client.md`` exception table. FAC-INV-7 (auth-token redaction): ``FlightsApiAuthError`` overrides diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/file_loader.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/file_loader.py similarity index 82% rename from src/gps_denied_onboard/components/c12_operator_tooling/flights_api/file_loader.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/file_loader.py index 3a9ef96..5eda688 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/file_loader.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/file_loader.py @@ -11,14 +11,14 @@ from pathlib import Path import orjson -from gps_denied_onboard.components.c12_operator_tooling.flights_api._parser import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api._parser import ( parse_flight_payload, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( FlightFileNotFoundError, FlightsApiSchemaError, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/httpx_client.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/httpx_client.py similarity index 95% rename from src/gps_denied_onboard/components/c12_operator_tooling/flights_api/httpx_client.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/httpx_client.py index 0604fea..4853e1c 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/httpx_client.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/httpx_client.py @@ -22,23 +22,23 @@ import httpx from gps_denied_onboard.clock.wall_clock import WallClock from gps_denied_onboard._types.geo import BoundingBox, LatLonAlt -from gps_denied_onboard.components.c12_operator_tooling.flights_api._parser import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api._parser import ( parse_flight_payload, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.bbox import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.bbox import ( bbox_from_waypoints, takeoff_origin_from_flight, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.errors import ( FlightNotFoundError, FlightsApiAuthError, FlightsApiSchemaError, FlightsApiUnreachableError, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.file_loader import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.file_loader import ( load_flight_file, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, WaypointDto, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/interface.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/interface.py similarity index 97% rename from src/gps_denied_onboard/components/c12_operator_tooling/flights_api/interface.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/interface.py index 6728b22..e038276 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/flights_api/interface.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/flights_api/interface.py @@ -1,6 +1,6 @@ """C12 ``FlightsApiClient`` Protocol + DTOs + enums (AZ-489). -Frozen by ``_docs/02_document/contracts/c12_operator_tooling/flights_api_client.md`` +Frozen by ``_docs/02_document/contracts/c12_operator_orchestrator/flights_api_client.md`` v1.0.0. The DTOs mirror ``suite/flights/Database/Entities/{Flight,Waypoint}.cs``; adding a new field on the parent-suite C# side requires a new minor-version bump here (FAC-INV-1: online + offline produce the same shape). diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/freshness_table.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/freshness_table.py similarity index 94% rename from src/gps_denied_onboard/components/c12_operator_tooling/freshness_table.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/freshness_table.py index 070f1ff..61d06e8 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/freshness_table.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/freshness_table.py @@ -12,7 +12,7 @@ from __future__ import annotations from typing import Final -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( SectorClassification, ) diff --git a/src/gps_denied_onboard/components/c12_operator_orchestrator/interface.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/interface.py new file mode 100644 index 0000000..ab806d0 --- /dev/null +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/interface.py @@ -0,0 +1,24 @@ +"""C12 ``CacheBuildWorkflow`` Protocol. + +The placeholder :class:`OperatorReLocService` Protocol that used to live +here has been superseded by the AZ-330 concrete class in +:mod:`operator_reloc_service`. The package re-exports the concrete +class under the same public name; consumers continue to import +``OperatorReLocService`` from +``gps_denied_onboard.components.c12_operator_orchestrator`` unchanged. + +See `_docs/02_document/components/13_c12_operator_orchestrator/`. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Protocol + +__all__ = ["CacheBuildWorkflow"] + + +class CacheBuildWorkflow(Protocol): + """Operator CLI workflow that orchestrates C11 download → C10 provisioning.""" + + def run(self, flight_id: str, output_root: Path) -> None: ... diff --git a/src/gps_denied_onboard/components/c12_operator_orchestrator/operator_command_transport.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/operator_command_transport.py new file mode 100644 index 0000000..f2cbf76 --- /dev/null +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/operator_command_transport.py @@ -0,0 +1,37 @@ +"""C12 ``OperatorCommandTransport`` Protocol (AZ-330). + +The C12 ↔ C8 contract for operator-driven commands sent over the GCS +link. C12 owns the Protocol shape; E-C8 will own the concrete +``MavlinkOperatorCommandTransport`` against pymavlink in a future +task. The pattern matches AZ-322's ``BackboneEmbedder`` (C10 owns the +Protocol; C2 implements it later). + +The Protocol contract document at +``_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md`` +pins the shape, invariants, and test cases the E-C8 implementer reads. +""" + +from __future__ import annotations + +from typing import Protocol, runtime_checkable + +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + ReLocHint, +) + +__all__ = ["OperatorCommandTransport"] + + +@runtime_checkable +class OperatorCommandTransport(Protocol): + """Send operator-side commands to the airborne companion over the GCS link. + + Implementations MUST raise :class:`GcsLinkError` on any link-level + failure (timeout, signal loss, serial-port error). The method is + non-blocking with respect to operator-side waiting — the transport + may block briefly inside MAVLink serialisation but MUST NOT block + waiting for an ack from the companion (best-effort semantics per + description.md § 7). + """ + + def send_reloc_hint(self, hint: ReLocHint) -> None: ... diff --git a/src/gps_denied_onboard/components/c12_operator_orchestrator/operator_reloc_service.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/operator_reloc_service.py new file mode 100644 index 0000000..ff53527 --- /dev/null +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/operator_reloc_service.py @@ -0,0 +1,196 @@ +"""C12 ``OperatorReLocService`` (AZ-330). + +Operator-side surface for AC-3.4 (visual-loss re-localization). The +operator workstation issues a position hint; this service validates, +forwards to the GCS-link :class:`OperatorCommandTransport` (E-C8 ships +the pymavlink-backed concrete impl in a future task), and records the +action in FDR so post-flight forensics retains it. + +Best-effort semantics per description.md § 7 — a single attempt; on +:class:`GcsLinkError` the failure is logged + FDR-recorded but never +auto-retried. The operator decides when to re-issue. +""" + +from __future__ import annotations + +import logging +from datetime import datetime, timezone + +from gps_denied_onboard.clock import Clock +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + ReLocHint, +) +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( + GcsLinkError, +) +from gps_denied_onboard.components.c12_operator_orchestrator.operator_command_transport import ( + OperatorCommandTransport, +) +from gps_denied_onboard.fdr_client import EnqueueResult, FdrClient +from gps_denied_onboard.fdr_client.records import ( + CURRENT_SCHEMA_VERSION, + FdrRecord, +) + +__all__ = ["OperatorReLocService"] + + +_COMPONENT = "c12_operator_orchestrator" + +_LOG_KIND_SENT = "c12.reloc.sent" +_LOG_KIND_FAILED = "c12.reloc.failed" +_FDR_KIND_REQUESTED = "c12.reloc.requested" + +# AC-4 + AC-9: live-log redaction tweaks. The FULL hint is persisted +# verbatim to FDR (post-flight forensics) and forwarded verbatim to +# the transport (operator action is byte-preserving). +_REASON_LOG_TRUNCATE_CHARS: int = 200 +_POSITION_LOG_PRECISION: int = 5 + + +class OperatorReLocService: + """Single-method service: validate → transmit → log → FDR (AC-3.4). + + The flow is intentionally linear and stateless. Construction is + cheap (no transport probe, no FDR enqueue) so the composition root + can build it eagerly without violating NFR-perf-cold-start. The + transport — pymavlink-backed in production — is only touched when + the operator hits the CLI's ``reloc-confirm`` subcommand. + """ + + def __init__( + self, + *, + transport: OperatorCommandTransport, + fdr_client: FdrClient, + logger: logging.Logger, + clock: Clock, + ) -> None: + self._transport = transport + self._fdr_client = fdr_client + self._logger = logger + self._clock = clock + + def request_reloc(self, reloc_hint: ReLocHint) -> None: + if not reloc_hint.confidence_radius_m > 0: + raise ValueError( + f"confidence_radius_m must be > 0; got {reloc_hint.confidence_radius_m}" + ) + if not reloc_hint.reason: + raise ValueError("reason must be non-empty") + + ts_monotonic_ns = self._clock.monotonic_ns() + try: + self._transport.send_reloc_hint(reloc_hint) + except GcsLinkError as exc: + self._log_failure(reloc_hint, exc) + self._emit_fdr( + reloc_hint, + outcome="failed", + failure_reason=exc.reason, + ts_monotonic_ns=ts_monotonic_ns, + ) + raise GcsLinkError( + reason=f"C12 reloc-confirm: {exc.reason}", + wrapped_exception_repr=repr(exc), + remediation=exc.remediation, + ) from exc + + self._log_success(reloc_hint) + self._emit_fdr( + reloc_hint, + outcome="sent", + failure_reason=None, + ts_monotonic_ns=ts_monotonic_ns, + ) + + def _log_success(self, hint: ReLocHint) -> None: + self._logger.info( + "operator re-loc hint sent", + extra={ + "component": _COMPONENT, + "kind": _LOG_KIND_SENT, + "kv": _redacted_log_kv(hint), + }, + ) + + def _log_failure(self, hint: ReLocHint, exc: GcsLinkError) -> None: + kv = _redacted_log_kv(hint) + kv["failure_reason"] = exc.reason + if exc.wrapped_exception_repr is not None: + kv["wrapped_exception_repr"] = exc.wrapped_exception_repr + self._logger.error( + "operator re-loc hint transmission failed", + extra={ + "component": _COMPONENT, + "kind": _LOG_KIND_FAILED, + "kv": kv, + }, + ) + + def _emit_fdr( + self, + hint: ReLocHint, + *, + outcome: str, + failure_reason: str | None, + ts_monotonic_ns: int, + ) -> None: + payload: dict[str, object] = { + "hint": _hint_to_payload(hint), + "outcome": outcome, + "ts_monotonic_ns": ts_monotonic_ns, + } + if failure_reason is not None: + payload["failure_reason"] = failure_reason + record = FdrRecord( + schema_version=CURRENT_SCHEMA_VERSION, + ts=self._iso_ts_from_clock(), + producer_id=self._fdr_client.producer_id, + kind=_FDR_KIND_REQUESTED, + payload=payload, + ) + # AC-8: FDR best-effort. Overrun is observable in tests via spy + # but never raises; the operator's transport call already + # completed (success or failure) before this point. + result = self._fdr_client.enqueue(record) + if result == EnqueueResult.OVERRUN: + self._logger.warning( + "FDR enqueue dropped operator re-loc record (buffer overrun)", + extra={ + "component": _COMPONENT, + "kind": "c12.reloc.fdr_overrun", + "kv": {"outcome": outcome}, + }, + ) + + def _iso_ts_from_clock(self) -> str: + ns = int(self._clock.time_ns()) + seconds, fraction_ns = divmod(ns, 1_000_000_000) + dt = datetime.fromtimestamp(seconds, tz=timezone.utc) + return f"{dt.strftime('%Y-%m-%dT%H:%M:%S')}.{fraction_ns:09d}+00:00" + + +def _hint_to_payload(hint: ReLocHint) -> dict[str, object]: + """Full-precision FDR-side serialisation. No redaction (AC-4 + § 5).""" + position = hint.approximate_position_wgs84 + return { + "lat_deg": position.lat_deg, + "lon_deg": position.lon_deg, + "alt_m": position.alt_m, + "confidence_radius_m": hint.confidence_radius_m, + "reason": hint.reason, + } + + +def _redacted_log_kv(hint: ReLocHint) -> dict[str, object]: + """Live-log redaction: 5-decimal position + 200-char reason cap (AC-4 + AC-9).""" + position = hint.approximate_position_wgs84 + truncated_reason = hint.reason[:_REASON_LOG_TRUNCATE_CHARS] + return { + "position_lat": round(position.lat_deg, _POSITION_LOG_PRECISION), + "position_lon": round(position.lon_deg, _POSITION_LOG_PRECISION), + "altitude_m": position.alt_m, + "confidence_radius_m": hint.confidence_radius_m, + "reason": truncated_reason, + } diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/paramiko_ssh_session.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/paramiko_ssh_session.py similarity index 96% rename from src/gps_denied_onboard/components/c12_operator_tooling/paramiko_ssh_session.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/paramiko_ssh_session.py index 08e40aa..18f0823 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/paramiko_ssh_session.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/paramiko_ssh_session.py @@ -23,17 +23,17 @@ from typing import Final import paramiko -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( CompanionAddress, CompanionUnreachableReason, ) -from gps_denied_onboard.components.c12_operator_tooling.config import ( +from gps_denied_onboard.components.c12_operator_orchestrator.config import ( HostKeyPolicy, ) -from gps_denied_onboard.components.c12_operator_tooling.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( CompanionUnreachableError, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( RemoteCommandResult, SshSession, SshSessionFactory, diff --git a/src/gps_denied_onboard/components/c12_operator_orchestrator/post_landing_upload.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/post_landing_upload.py new file mode 100644 index 0000000..0a76102 --- /dev/null +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/post_landing_upload.py @@ -0,0 +1,193 @@ +"""C12 ``PostLandingUploadOrchestrator`` (AZ-329). + +Operator-side gate on the post-landing tile upload. Reads the C13 +``flight_footer`` record for ``flight_id`` and, when present with +``clean_shutdown=True``, delegates the actual upload to a C11 +:class:`TileUploaderCut` collaborator. Any other state (missing +directory, missing footer, ``clean_shutdown=False``, parse error) +refuses with :class:`FlightStateNotConfirmedError`. + +C12 does NOT import ``c11_tile_manager`` here — the AZ-507 consumer-side +cut pattern enforces structural typing via :class:`TileUploaderCut`. +""" + +from __future__ import annotations + +import logging + +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + FlightFooterRecord, + PostLandingUploadRequest, + UploadBatchReportCut, + UploadRequestCut, +) +from gps_denied_onboard.components.c12_operator_orchestrator.config import ( + C12PostLandingConfig, +) +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( + FdrUnreadableError, + FlightStateNotConfirmedError, +) +from gps_denied_onboard.components.c12_operator_orchestrator.fdr_footer_reader import ( + FdrFooterReader, +) +from gps_denied_onboard.components.c12_operator_orchestrator.tile_uploader_cut import ( + TileUploaderCut, +) + +__all__ = ["PostLandingUploadOrchestrator"] + + +_COMPONENT = "c12_operator_orchestrator" + +_LOG_KIND_CONFIRMED = "c12.upload.confirmed_clean_shutdown" +_LOG_KIND_COMPLETE = "c12.upload.complete" +_LOG_KIND_REFUSED_FLIGHT_NOT_FOUND = "c12.upload.refused.flight_id_not_found" +_LOG_KIND_REFUSED_FOOTER_MISSING = "c12.upload.refused.footer_missing" +_LOG_KIND_REFUSED_UNCLEAN_SHUTDOWN = "c12.upload.refused.unclean_shutdown" +_LOG_KIND_REFUSED_FDR_UNREADABLE = "c12.upload.refused.fdr_unreadable" + + +class PostLandingUploadOrchestrator: + """Operator-side gate on the post-landing tile upload (AZ-329). + + Single public method :meth:`trigger_post_landing_upload`. The + decision tree is deterministic and exhaustive across the four + :class:`NotConfirmedReason` values; the orchestrator never silently + proceeds when it cannot positively confirm a clean shutdown. + """ + + def __init__( + self, + *, + tile_uploader: TileUploaderCut, + fdr_footer_reader: FdrFooterReader, + logger: logging.Logger, + config: C12PostLandingConfig, + ) -> None: + self._tile_uploader = tile_uploader + self._fdr_footer_reader = fdr_footer_reader + self._logger = logger + self._config = config + + def trigger_post_landing_upload( + self, request: PostLandingUploadRequest + ) -> UploadBatchReportCut: + flight_id_str = str(request.flight_id) + flight_dir = self._config.fdr_root / flight_id_str + + if not flight_dir.exists(): + self._log_refusal( + _LOG_KIND_REFUSED_FLIGHT_NOT_FOUND, + "flight_id directory not found in FDR root", + kv={"flight_id": flight_id_str, "flight_dir": str(flight_dir)}, + ) + raise FlightStateNotConfirmedError( + flight_id=flight_id_str, + not_confirmed_reason="flight_id_not_found", + ) + + try: + footer = self._fdr_footer_reader.read_footer(request.flight_id) + except FdrUnreadableError as exc: + self._log_refusal( + _LOG_KIND_REFUSED_FDR_UNREADABLE, + "FDR segment scan failed", + kv={"flight_id": flight_id_str, "fdr_unreadable_repr": repr(exc)}, + ) + raise FlightStateNotConfirmedError( + flight_id=flight_id_str, + not_confirmed_reason="fdr_unreadable", + detail=repr(exc), + ) from exc + + if footer is None: + self._log_refusal( + _LOG_KIND_REFUSED_FOOTER_MISSING, + "no flight_footer record found in any FDR segment", + kv={"flight_id": flight_id_str}, + ) + raise FlightStateNotConfirmedError( + flight_id=flight_id_str, + not_confirmed_reason="footer_missing", + ) + + if not footer.clean_shutdown: + counters_kv = _footer_counters_kv(footer) + self._log_refusal( + _LOG_KIND_REFUSED_UNCLEAN_SHUTDOWN, + "flight_footer.clean_shutdown is False", + kv={"flight_id": flight_id_str, **counters_kv}, + ) + detail = ( + f"records_dropped_overrun={footer.records_dropped_overrun}, " + f"bytes_written={footer.bytes_written}, " + f"records_written={footer.records_written}, " + f"rollover_count={footer.rollover_count}" + ) + raise FlightStateNotConfirmedError( + flight_id=flight_id_str, + not_confirmed_reason="unclean_shutdown", + detail=detail, + ) + + self._logger.info( + "post-landing upload confirmed", + extra={ + "component": _COMPONENT, + "kind": _LOG_KIND_CONFIRMED, + "kv": { + "flight_id": flight_id_str, + "flight_ended_at_iso": footer.flight_ended_at_iso, + "records_written": footer.records_written, + }, + }, + ) + + inner_request = UploadRequestCut( + flight_id=request.flight_id, + batch_size=request.batch_size, + satellite_provider_url=request.satellite_provider_url, + ) + report = self._tile_uploader.upload_pending_tiles(inner_request) + + tiles_acked = sum( + 1 for tile in report.per_tile_status if tile.status.value == "accepted" + ) + tiles_rejected = sum( + 1 for tile in report.per_tile_status if tile.status.value == "rejected" + ) + self._logger.info( + "post-landing upload complete", + extra={ + "component": _COMPONENT, + "kind": _LOG_KIND_COMPLETE, + "kv": { + "flight_id": flight_id_str, + "outcome": report.outcome.value, + "tiles_acked": tiles_acked, + "tiles_rejected": tiles_rejected, + "batch_uuid": str(report.batch_uuid), + "public_key_fingerprint": report.public_key_fingerprint, + "retry_count": report.retry_count, + }, + }, + ) + return report + + def _log_refusal( + self, kind: str, message: str, *, kv: dict[str, object] + ) -> None: + self._logger.error( + message, + extra={"component": _COMPONENT, "kind": kind, "kv": kv}, + ) + + +def _footer_counters_kv(footer: FlightFooterRecord) -> dict[str, int]: + return { + "records_written": footer.records_written, + "records_dropped_overrun": footer.records_dropped_overrun, + "bytes_written": footer.bytes_written, + "rollover_count": footer.rollover_count, + } diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/remote_c10_invoker.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/remote_c10_invoker.py similarity index 97% rename from src/gps_denied_onboard/components/c12_operator_tooling/remote_c10_invoker.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/remote_c10_invoker.py index 16c64b4..39eefaa 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/remote_c10_invoker.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/remote_c10_invoker.py @@ -30,15 +30,15 @@ from pathlib import Path, PurePosixPath from uuid import UUID from gps_denied_onboard._types.geo import BoundingBox, LatLonAlt -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( RemoteBuildOutcome, RemoteBuildReport, SectorClassification, ) -from gps_denied_onboard.components.c12_operator_tooling.errors import ( +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( BuildReportParseError, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( SshSession, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/remote_sidecar_verifier.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/remote_sidecar_verifier.py similarity index 98% rename from src/gps_denied_onboard/components/c12_operator_tooling/remote_sidecar_verifier.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/remote_sidecar_verifier.py index 03257cd..0ee4177 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/remote_sidecar_verifier.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/remote_sidecar_verifier.py @@ -22,7 +22,7 @@ from dataclasses import dataclass from pathlib import PurePosixPath from typing import Final -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( SshSession, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/sector_classification_store.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/sector_classification_store.py similarity index 97% rename from src/gps_denied_onboard/components/c12_operator_tooling/sector_classification_store.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/sector_classification_store.py index 7170091..864b386 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/sector_classification_store.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/sector_classification_store.py @@ -1,7 +1,7 @@ """Persistent ``{area_id: SectorClassification}`` store (AZ-326). Atomic-write JSON file kept in the operator's home directory so a -restart of ``operator-tool`` recovers every classification the operator +restart of ``operator-orchestrator`` recovers every classification the operator ever ran ``set-sector`` against. The atomic-write pattern uses ``tempfile.NamedTemporaryFile(dir=...) + os.replace(...)`` per AC-5; see :mod:`gps_denied_onboard.helpers.sha256_sidecar` for the heavier @@ -22,7 +22,7 @@ import os import tempfile from pathlib import Path -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( AreaIdentifier, SectorClassification, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/ssh_session.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/ssh_session.py similarity index 97% rename from src/gps_denied_onboard/components/c12_operator_tooling/ssh_session.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/ssh_session.py index 29b2a78..ab35906 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/ssh_session.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/ssh_session.py @@ -17,7 +17,7 @@ from dataclasses import dataclass from pathlib import PurePosixPath from typing import Protocol, runtime_checkable -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( CompanionAddress, ) diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/tile_downloader_cut.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/tile_downloader_cut.py similarity index 90% rename from src/gps_denied_onboard/components/c12_operator_tooling/tile_downloader_cut.py rename to src/gps_denied_onboard/components/c12_operator_orchestrator/tile_downloader_cut.py index 8a1620a..043ec87 100644 --- a/src/gps_denied_onboard/components/c12_operator_tooling/tile_downloader_cut.py +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/tile_downloader_cut.py @@ -1,7 +1,7 @@ """C12 consumer-side structural cut of c11 ``TileDownloader`` (AZ-507). The AZ-507 cross-component rule (see ``_docs/02_document/module-layout.md`` -line 252) forbids ``c12_operator_tooling/*.py`` from importing +line 252) forbids ``c12_operator_orchestrator/*.py`` from importing ``components.c11_tile_manager`` directly. The ``BuildCacheOrchestrator`` needs the download surface to drive the F1 download phase, so we declare a local Protocol that mirrors the shape of c11's @@ -17,7 +17,7 @@ from __future__ import annotations from typing import Protocol, runtime_checkable -from gps_denied_onboard.components.c12_operator_tooling._types import ( +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( DownloadBatchReportCut, DownloadRequestCut, ) diff --git a/src/gps_denied_onboard/components/c12_operator_orchestrator/tile_uploader_cut.py b/src/gps_denied_onboard/components/c12_operator_orchestrator/tile_uploader_cut.py new file mode 100644 index 0000000..0ba74fc --- /dev/null +++ b/src/gps_denied_onboard/components/c12_operator_orchestrator/tile_uploader_cut.py @@ -0,0 +1,42 @@ +"""C12 consumer-side structural cut of c11 ``TileUploader`` (AZ-507). + +The AZ-507 cross-component rule (see ``_docs/02_document/module-layout.md``) +forbids ``c12_operator_orchestrator/*.py`` from importing +``components.c11_tile_manager`` directly. The +:class:`PostLandingUploadOrchestrator` needs the upload surface to drive +the F10 post-landing upload phase, so we declare a local Protocol that +mirrors the shape of c11's +:class:`gps_denied_onboard.components.c11_tile_manager.interface.TileUploader.upload_pending_tiles` +method. + +The composition root (``runtime_root.c12_factory``'s caller — the +suite-level runtime root) wires the concrete c11 strategy in via a thin +adapter that maps :class:`UploadRequestCut` to c11's ``UploadRequest`` +and ``UploadBatchReport`` back to :class:`UploadBatchReportCut`. Tests +inject a fake that returns a :class:`UploadBatchReportCut` directly, so +they never touch c11 either. +""" + +from __future__ import annotations + +from typing import Protocol, runtime_checkable + +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + UploadBatchReportCut, + UploadRequestCut, +) + +__all__ = ["TileUploaderCut"] + + +@runtime_checkable +class TileUploaderCut(Protocol): + """Single-method consumer-side cut of c11 ``TileUploader``. + + The orchestrator constructs a :class:`UploadRequestCut` and the + composition-root wiring translates it into c11's real + ``UploadRequest`` (and the returned ``UploadBatchReport`` back into + a :class:`UploadBatchReportCut`). + """ + + def upload_pending_tiles(self, request: UploadRequestCut) -> UploadBatchReportCut: ... diff --git a/src/gps_denied_onboard/components/c12_operator_tooling/interface.py b/src/gps_denied_onboard/components/c12_operator_tooling/interface.py deleted file mode 100644 index fa08d52..0000000 --- a/src/gps_denied_onboard/components/c12_operator_tooling/interface.py +++ /dev/null @@ -1,21 +0,0 @@ -"""C12 `CacheBuildWorkflow` + `OperatorReLocService` Protocols. - -See `_docs/02_document/components/13_c12_operator_tooling/`. -""" - -from __future__ import annotations - -from pathlib import Path -from typing import Protocol - - -class CacheBuildWorkflow(Protocol): - """Operator CLI workflow that orchestrates C11 download → C10 provisioning.""" - - def run(self, flight_id: str, output_root: Path) -> None: ... - - -class OperatorReLocService(Protocol): - """Operator-side re-localization request service (GUI deferred per epic).""" - - def request_relocalization(self, flight_id: str, hint: dict) -> None: ... diff --git a/src/gps_denied_onboard/components/c3_5_adhop/config.py b/src/gps_denied_onboard/components/c3_5_adhop/config.py index dfe0df6..f42a147 100644 --- a/src/gps_denied_onboard/components/c3_5_adhop/config.py +++ b/src/gps_denied_onboard/components/c3_5_adhop/config.py @@ -15,7 +15,7 @@ territory). Runtime selection only. :class:`MatchResult` whose ``reprojection_residual_px <= threshold`` is passed through unchanged; ``>`` invokes the strategy's refinement procedure. Default 2.5 px (the AC-NEW-5 / -R10 tunable from operator tooling). +R10 tunable from operator orchestrator). ``invocation_rate_warn_threshold`` is the rolling-60 s invocation-rate ceiling above which a WARN log fires diff --git a/src/gps_denied_onboard/components/c6_tile_cache/_tile_pixel_handle.py b/src/gps_denied_onboard/components/c6_tile_cache/_tile_pixel_handle.py index c4a0363..3db2eb6 100644 --- a/src/gps_denied_onboard/components/c6_tile_cache/_tile_pixel_handle.py +++ b/src/gps_denied_onboard/components/c6_tile_cache/_tile_pixel_handle.py @@ -31,7 +31,7 @@ class TilePixelHandle(ABC): def filesystem_path(self) -> Path: """Absolute path to the JPEG file backing this handle. - Used only by C12 operator tooling (post-flight inspection) + Used only by C12 operator orchestrator (post-flight inspection) and the C11 ``TileUploader`` post-landing copy. In-flight consumers MUST NOT open a second handle to the same path; they MUST use this :class:`TilePixelHandle`. diff --git a/src/gps_denied_onboard/fdr_client/records.py b/src/gps_denied_onboard/fdr_client/records.py index e07ea65..cf078cf 100644 --- a/src/gps_denied_onboard/fdr_client/records.py +++ b/src/gps_denied_onboard/fdr_client/records.py @@ -280,6 +280,22 @@ KNOWN_PAYLOAD_KEYS: Final[dict[str, frozenset[str]]] = { "last_rejection_reason", } ), + # AZ-330 / E-C12: emitted by the C12 OperatorReLocService on every + # operator-driven re-loc command (AC-3.4). ``outcome`` is "sent" on + # transport success, "failed" when the transport raised + # ``GcsLinkError``. ``hint`` carries the FULL ReLocHint (no + # redaction — post-flight forensics need the exact action the + # operator took). ``failure_reason`` is populated only on + # ``outcome="failed"``. ``ts_monotonic_ns`` is the orchestrator-side + # ``Clock.monotonic_ns()`` reading at the moment of the call. + "c12.reloc.requested": frozenset( + { + "hint", + "outcome", + "failure_reason", + "ts_monotonic_ns", + } + ), } KNOWN_KINDS: Final[frozenset[str]] = frozenset(KNOWN_PAYLOAD_KEYS.keys()) diff --git a/src/gps_denied_onboard/healthcheck.py b/src/gps_denied_onboard/healthcheck.py index 6938412..197130b 100644 --- a/src/gps_denied_onboard/healthcheck.py +++ b/src/gps_denied_onboard/healthcheck.py @@ -1,6 +1,6 @@ """Bootstrap healthcheck callable. -Used by both `companion-tier1` and `operator-tooling` Dockerfiles via +Used by both `companion-tier1` and `operator-orchestrator` Dockerfiles via `HEALTHCHECK CMD python -m gps_denied_onboard.healthcheck`. Returns a non-zero exit code on any failure so Docker's healthcheck loop marks the container unhealthy. diff --git a/src/gps_denied_onboard/runtime_root/__init__.py b/src/gps_denied_onboard/runtime_root/__init__.py index 462e8df..c862df8 100644 --- a/src/gps_denied_onboard/runtime_root/__init__.py +++ b/src/gps_denied_onboard/runtime_root/__init__.py @@ -409,15 +409,15 @@ def compose_root(config: Config) -> RuntimeRoot: def compose_operator(config: Config) -> OperatorRoot: - """Compose the operator-tooling runtime graph (per contract v1.0.0).""" + """Compose the operator-orchestrator runtime graph (per contract v1.0.0).""" components, order = _compose( config, - binary="operator-tooling", + binary="operator-orchestrator", allowed_tiers=frozenset({"operator", "shared"}), extra_required_env=("SATELLITE_PROVIDER_URL",), ) return OperatorRoot( - binary="operator-tooling", + binary="operator-orchestrator", profile=os.environ["GPS_DENIED_FC_PROFILE"], components=components, construction_order=order, diff --git a/src/gps_denied_onboard/runtime_root/c11_factory.py b/src/gps_denied_onboard/runtime_root/c11_factory.py index 4f6cfb4..3138253 100644 --- a/src/gps_denied_onboard/runtime_root/c11_factory.py +++ b/src/gps_denied_onboard/runtime_root/c11_factory.py @@ -1,17 +1,15 @@ -"""C11 TileManager composition-root factories (AZ-316, AZ-317, AZ-318, AZ-319). +"""C11 TileManager composition-root factories (AZ-316, AZ-318, AZ-319). Wires the operator-side services: -* :func:`build_flight_state_gate` (AZ-317) — adapts an injected - ``FlightStateSource`` (typically an E-C8 FC adapter wrapper) into - the C11 ``FlightStateGate``. * :func:`build_per_flight_key_manager` (AZ-318) — wires the AZ-273 :class:`FdrClient` and the project ``Clock`` strategy into the ephemeral signing-key manager. -* :func:`build_tile_uploader` (AZ-319) — composes the gate, the - key manager, the c6 storage cuts, an :class:`httpx.Client`, and - the :class:`C11Config` block into the production - :class:`HttpTileUploader`. +* :func:`build_tile_uploader` (AZ-319) — composes the key manager, + the c6 storage cuts, an :class:`httpx.Client`, and the + :class:`C11Config` block into the production + :class:`HttpTileUploader`. Flight-state confirmation is the + caller's responsibility (C12 ``PostLandingUploadOrchestrator``). * :func:`build_tile_downloader` (AZ-316) — composes the c6 store + metadata-store + budget-enforcer (wrapped in a single composition-root adapter that hides c6's :class:`TileMetadata` @@ -32,8 +30,6 @@ import httpx from gps_denied_onboard.components.c11_tile_manager import ( C11Config, - FlightStateGate, - FlightStateSource, HttpTileDownloader, HttpTileUploader, IdempotentRetryTileUploader, @@ -50,14 +46,12 @@ if TYPE_CHECKING: from gps_denied_onboard.config.schema import Config __all__ = [ - "build_flight_state_gate", "build_per_flight_key_manager", "build_tile_downloader", "build_tile_uploader", ] -_C11_GATE_LOGGER = "c11_tile_manager.flight_state_gate" _C11_SIGNING_LOGGER = "c11_tile_manager.signing_key" _C11_SIGNING_PRODUCER_ID = "c11_tile_manager.signing_key" _C11_UPLOADER_LOGGER = "c11_tile_manager.tile_uploader" @@ -65,19 +59,6 @@ _C11_UPLOADER_PRODUCER_ID = "c11_tile_manager.tile_uploader" _C11_DOWNLOADER_LOGGER = "c11_tile_manager.tile_downloader" -def build_flight_state_gate(*, source: FlightStateSource) -> FlightStateGate: - """Construct a wired :class:`FlightStateGate` (AZ-317). - - The ``source`` argument is the consumer-side cut over E-C8's FC - adapter; the composition root supplies a concrete adapter wrapping - the actual C8 instance once E-C8 ships. Until then operator - tooling tests inject a fake source that returns a fixed signal. - """ - - logger = get_logger(_C11_GATE_LOGGER) - return FlightStateGate(source=source, logger=logger) - - def build_per_flight_key_manager( config: Config, *, @@ -108,7 +89,6 @@ def build_tile_uploader( http_client: httpx.Client, tile_store: Any, tile_metadata_store: Any, - flight_state_gate: FlightStateGate, key_manager: PerFlightKeyManager, clock: ClockProtocol | None = None, fdr_client: FdrClient | None = None, @@ -162,7 +142,6 @@ def build_tile_uploader( http_client=http_client, tile_store=tile_store, tile_metadata_store=tile_metadata_store, - flight_state_gate=flight_state_gate, key_manager=key_manager, fdr_client=fdr_client, logger=logger, @@ -235,7 +214,7 @@ def build_tile_downloader( if not block.service_api_key: raise ConfigError( "build_tile_downloader: C11Config.service_api_key must be " - "set; the operator-tooling deploy MUST inject the bearer " + "set; the operator-orchestrator deploy MUST inject the bearer " "token via env override" ) logger = get_logger(_C11_DOWNLOADER_LOGGER) diff --git a/src/gps_denied_onboard/runtime_root/c12_factory.py b/src/gps_denied_onboard/runtime_root/c12_factory.py index 7d604d7..8776177 100644 --- a/src/gps_denied_onboard/runtime_root/c12_factory.py +++ b/src/gps_denied_onboard/runtime_root/c12_factory.py @@ -1,4 +1,4 @@ -"""Composition-root factories for C12 operator-tooling services. +"""Composition-root factories for C12 operator-orchestrator services. * :func:`build_flights_api_client` — AZ-489 ``FlightsApiClient`` (online + offline path). @@ -12,14 +12,14 @@ AZ-327 / AZ-489 services. The AZ-507 cross-component cut means we translate c11's real ``DownloadRequest`` / ``DownloadBatchReport`` to the local ``DownloadRequestCut`` / ``DownloadBatchReportCut`` here. -* :func:`build_operator_tool` — aggregator that returns the - :class:`OperatorToolServices` dataclass the AZ-326 CLI consumes. +* :func:`build_operator_orchestrator` — aggregator that returns the + :class:`OperatorOrchestratorServices` dataclass the AZ-326 CLI consumes. Each ``build_*`` function is intentionally tiny — there is one production strategy per service today and the CLI wiring just plugs the concrete instance into the same composition root method. Sibling tasks AZ-329 / AZ-330 will each add a single field to -:class:`OperatorToolServices` without renaming or moving the +:class:`OperatorOrchestratorServices` without renaming or moving the dataclass. """ @@ -30,60 +30,80 @@ from dataclasses import dataclass from typing import TYPE_CHECKING from gps_denied_onboard.clock import Clock -from gps_denied_onboard.components.c12_operator_tooling.build_cache import ( +from gps_denied_onboard.components.c12_operator_orchestrator.build_cache import ( BuildCacheOrchestrator, ) -from gps_denied_onboard.components.c12_operator_tooling.companion_bringup import ( +from gps_denied_onboard.components.c12_operator_orchestrator.companion_bringup import ( CompanionBringup, ) -from gps_denied_onboard.components.c12_operator_tooling.file_lock import ( +from gps_denied_onboard.components.c12_operator_orchestrator.fdr_footer_reader import ( + LocalFdrFooterReader, +) +from gps_denied_onboard.components.c12_operator_orchestrator.file_lock import ( FilelockFileLockFactory, ) -from gps_denied_onboard.components.c12_operator_tooling.flights_api import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api import ( FlightsApiClient, HttpxFlightsApiClient, ) -from gps_denied_onboard.components.c12_operator_tooling.paramiko_ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.operator_command_transport import ( + OperatorCommandTransport, +) +from gps_denied_onboard.components.c12_operator_orchestrator.operator_reloc_service import ( + OperatorReLocService, +) +from gps_denied_onboard.components.c12_operator_orchestrator.paramiko_ssh_session import ( ParamikoSshSessionFactory, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_c10_invoker import ( +from gps_denied_onboard.components.c12_operator_orchestrator.post_landing_upload import ( + PostLandingUploadOrchestrator, +) +from gps_denied_onboard.fdr_client import FdrClient +from gps_denied_onboard.components.c12_operator_orchestrator.remote_c10_invoker import ( RemoteCacheProvisionerInvoker, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_sidecar_verifier import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_sidecar_verifier import ( RemoteSidecarVerifier, ) -from gps_denied_onboard.components.c12_operator_tooling.sector_classification_store import ( +from gps_denied_onboard.components.c12_operator_orchestrator.sector_classification_store import ( SectorClassificationStore, ) -from gps_denied_onboard.components.c12_operator_tooling.tile_downloader_cut import ( +from gps_denied_onboard.components.c12_operator_orchestrator.tile_downloader_cut import ( TileDownloaderCut, ) +from gps_denied_onboard.components.c12_operator_orchestrator.tile_uploader_cut import ( + TileUploaderCut, +) if TYPE_CHECKING: - from gps_denied_onboard.components.c12_operator_tooling.config import ( + from gps_denied_onboard.components.c12_operator_orchestrator.config import ( C12Config, ) from gps_denied_onboard.config import Config __all__ = [ - "OperatorToolServices", + "OperatorOrchestratorServices", "build_build_cache_orchestrator", "build_companion_bringup", "build_flights_api_client", - "build_operator_tool", + "build_operator_reloc_service", + "build_operator_orchestrator", + "build_post_landing_upload_orchestrator", "build_sector_classification_store", ] -_C12_LOGGER_NAME = "c12_operator_tooling" -_COMPANION_LOGGER_NAME = "c12_operator_tooling.companion_bringup" -_BUILD_CACHE_LOGGER_NAME = "c12_operator_tooling.build_cache" -_REMOTE_C10_LOGGER_NAME = "c12_operator_tooling.remote_c10_invoker" +_C12_LOGGER_NAME = "c12_operator_orchestrator" +_COMPANION_LOGGER_NAME = "c12_operator_orchestrator.companion_bringup" +_BUILD_CACHE_LOGGER_NAME = "c12_operator_orchestrator.build_cache" +_REMOTE_C10_LOGGER_NAME = "c12_operator_orchestrator.remote_c10_invoker" +_POST_LANDING_LOGGER_NAME = "c12_operator_orchestrator.post_landing_upload" +_OPERATOR_RELOC_LOGGER_NAME = "c12_operator_orchestrator.operator_reloc_service" @dataclass(frozen=True) -class OperatorToolServices: - """Aggregated service handles the operator-tool CLI consumes (AZ-326). +class OperatorOrchestratorServices: + """Aggregated service handles the operator-orchestrator CLI consumes (AZ-326). AZ-326 introduced the dataclass and now owns three services (``flights_api_client``, ``sector_classification_store``, @@ -103,6 +123,8 @@ class OperatorToolServices: sector_classification_store: SectorClassificationStore companion_bringup: CompanionBringup build_cache_orchestrator: BuildCacheOrchestrator | None = None + post_landing_upload_orchestrator: PostLandingUploadOrchestrator | None = None + operator_reloc_service: OperatorReLocService | None = None def build_flights_api_client(config: Config) -> FlightsApiClient: @@ -162,7 +184,7 @@ def build_companion_bringup( def build_build_cache_orchestrator( config: Config, *, - services: OperatorToolServices, + services: OperatorOrchestratorServices, tile_downloader: TileDownloaderCut, clock: Clock, logger: logging.Logger | None = None, @@ -171,7 +193,7 @@ def build_build_cache_orchestrator( Caller (production runtime root) is responsible for translating the real c11 ``TileDownloader`` to a :class:`TileDownloaderCut` adapter - here — ``c12_operator_tooling`` cannot import c11 directly per + here — ``c12_operator_orchestrator`` cannot import c11 directly per AZ-507. The lockfile factory + remote-C10 invoker + SSH factory are constructed in-place; the SSH factory MUST be the same instance as the one wired into ``services.companion_bringup`` (single @@ -207,51 +229,142 @@ def build_build_cache_orchestrator( ) -def build_operator_tool( +def build_post_landing_upload_orchestrator( + config: Config, + *, + tile_uploader: TileUploaderCut, + logger: logging.Logger | None = None, +) -> PostLandingUploadOrchestrator: + """Build the AZ-329 :class:`PostLandingUploadOrchestrator` from config + a c11 uploader cut. + + Caller (production suite-level runtime root) is responsible for + translating the real c11 ``HttpTileUploader`` to a + :class:`TileUploaderCut` adapter here — ``c12_operator_orchestrator`` + cannot import c11 directly per AZ-507. The adapter maps + :class:`UploadRequestCut` ↔ c11's ``UploadRequest`` and + :class:`UploadBatchReportCut` ↔ c11's ``UploadBatchReport``. + """ + c12_config = _resolve_c12_config(config) + return PostLandingUploadOrchestrator( + tile_uploader=tile_uploader, + fdr_footer_reader=LocalFdrFooterReader(c12_config.post_landing.fdr_root), + logger=logger or logging.getLogger(_POST_LANDING_LOGGER_NAME), + config=c12_config.post_landing, + ) + + +def build_operator_reloc_service( + config: Config, + *, + transport: OperatorCommandTransport, + fdr_client: FdrClient, + clock: Clock, + logger: logging.Logger | None = None, +) -> OperatorReLocService: + """Build the AZ-330 :class:`OperatorReLocService`. + + The :class:`OperatorCommandTransport` (E-C8's pymavlink-backed + ``MavlinkOperatorCommandTransport`` in production; a + ``LoggingOnlyOperatorCommandTransport`` in dev environments without + a companion) is resolved by the suite-level runtime root and + injected here — c12 cannot import c8 directly per AZ-507. The + ``fdr_client`` is the shared AZ-273 instance keyed to producer + ``c12_operator_orchestrator`` so the post-flight FDR captures the + operator's re-loc actions chronologically alongside other onboard + records. + """ + _ = config # reserved for future composition-time tuning + return OperatorReLocService( + transport=transport, + fdr_client=fdr_client, + logger=logger or logging.getLogger(_OPERATOR_RELOC_LOGGER_NAME), + clock=clock, + ) + + +def build_operator_orchestrator( config: Config, *, tile_downloader: TileDownloaderCut | None = None, + tile_uploader: TileUploaderCut | None = None, clock: Clock | None = None, -) -> OperatorToolServices: - """Aggregate the AZ-326 / AZ-327 / AZ-328 / AZ-489 service handles. + operator_command_transport: OperatorCommandTransport | None = None, + fdr_client: FdrClient | None = None, +) -> OperatorOrchestratorServices: + """Aggregate the AZ-326 / AZ-327 / AZ-328 / AZ-329 / AZ-330 / AZ-489 service handles. - ``tile_downloader`` and ``clock`` are optional — without them, the - ``build_cache_orchestrator`` field is left as ``None`` and the CLI's - ``build-cache`` subcommand short-circuits gracefully. Production - wiring (the suite-level runtime root) supplies real instances. + Optional collaborators (each gates one service field): + + * ``tile_downloader`` + ``clock`` → ``build_cache_orchestrator`` + (AZ-328); CLI ``build-cache`` short-circuits when missing. + * ``tile_uploader`` → ``post_landing_upload_orchestrator`` (AZ-329); + CLI ``upload-pending`` short-circuits when missing. + * ``operator_command_transport`` + ``fdr_client`` + ``clock`` → + ``operator_reloc_service`` (AZ-330); CLI ``reloc-confirm`` + short-circuits when missing. AC-10: lazy construction — when the + transport is not supplied, no transport instance is created + (pymavlink stays unimported). """ - base = OperatorToolServices( - flights_api_client=build_flights_api_client(config), - sector_classification_store=build_sector_classification_store(config), - companion_bringup=build_companion_bringup(config), + flights_api_client = build_flights_api_client(config) + sector_store = build_sector_classification_store(config) + companion_bringup = build_companion_bringup(config) + + base_for_build_cache = OperatorOrchestratorServices( + flights_api_client=flights_api_client, + sector_classification_store=sector_store, + companion_bringup=companion_bringup, ) - if tile_downloader is None or clock is None: - return base - orchestrator = build_build_cache_orchestrator( - config, - services=base, - tile_downloader=tile_downloader, - clock=clock, - ) - return OperatorToolServices( - flights_api_client=base.flights_api_client, - sector_classification_store=base.sector_classification_store, - companion_bringup=base.companion_bringup, - build_cache_orchestrator=orchestrator, + + build_cache_orchestrator: BuildCacheOrchestrator | None = None + if tile_downloader is not None and clock is not None: + build_cache_orchestrator = build_build_cache_orchestrator( + config, + services=base_for_build_cache, + tile_downloader=tile_downloader, + clock=clock, + ) + + post_landing_orchestrator: PostLandingUploadOrchestrator | None = None + if tile_uploader is not None: + post_landing_orchestrator = build_post_landing_upload_orchestrator( + config, + tile_uploader=tile_uploader, + ) + + operator_reloc_service: OperatorReLocService | None = None + if ( + operator_command_transport is not None + and fdr_client is not None + and clock is not None + ): + operator_reloc_service = build_operator_reloc_service( + config, + transport=operator_command_transport, + fdr_client=fdr_client, + clock=clock, + ) + + return OperatorOrchestratorServices( + flights_api_client=flights_api_client, + sector_classification_store=sector_store, + companion_bringup=companion_bringup, + build_cache_orchestrator=build_cache_orchestrator, + post_landing_upload_orchestrator=post_landing_orchestrator, + operator_reloc_service=operator_reloc_service, ) def _resolve_c12_config(config: Config) -> C12Config: - from gps_denied_onboard.components.c12_operator_tooling.config import ( + from gps_denied_onboard.components.c12_operator_orchestrator.config import ( C12Config, ) - block = config.components.get("c12_operator_tooling") + block = config.components.get("c12_operator_orchestrator") if block is None: return C12Config() if not isinstance(block, C12Config): raise TypeError( - "config.components['c12_operator_tooling'] must be a C12Config; got " + "config.components['c12_operator_orchestrator'] must be a C12Config; got " f"{type(block).__name__}" ) return block diff --git a/tests/unit/c11_tile_manager/test_flight_state_gate.py b/tests/unit/c11_tile_manager/test_flight_state_gate.py deleted file mode 100644 index f355ff4..0000000 --- a/tests/unit/c11_tile_manager/test_flight_state_gate.py +++ /dev/null @@ -1,297 +0,0 @@ -"""AZ-317 ``FlightStateGate`` unit tests. - -Covers all eight acceptance criteria + NFRs from -``_docs/02_tasks/done/AZ-317_c11_flight_state_gate.md`` (after the -batch-38 archive). Uses a hand-rolled fake :class:`FlightStateSource` -and a list-backed log handler so assertions stay close to the -captured records. -""" - -from __future__ import annotations - -import logging -import time -from datetime import datetime, timezone - -import pytest - -from gps_denied_onboard.components.c11_tile_manager import ( - FlightStateGate, - FlightStateNotOnGroundError, - FlightStateSignal, - FlightStateSource, -) - - -# ---------------------------------------------------------------------- -# Helpers -# ---------------------------------------------------------------------- - - -class _FakeSource: - """Hand-rolled :class:`FlightStateSource` returning a fixed signal. - - Spies on every ``current_flight_state`` call so AC-8 can assert - the gate calls the source exactly once per ``confirm_on_ground``. - """ - - def __init__(self, signal: FlightStateSignal) -> None: - self._signal = signal - self.call_count = 0 - - def current_flight_state(self) -> FlightStateSignal: - self.call_count += 1 - return self._signal - - -class _RaisingSource: - """:class:`FlightStateSource` whose ``current_flight_state`` raises.""" - - def __init__(self, exc: Exception) -> None: - self._exc = exc - self.call_count = 0 - - def current_flight_state(self) -> FlightStateSignal: - self.call_count += 1 - raise self._exc - - -class _PartialFake: - """Type stub WITHOUT ``current_flight_state`` for AC-6 negative case.""" - - def something_else(self) -> str: - return "noop" - - -def _build_gate( - *, - source: FlightStateSource, -) -> tuple[FlightStateGate, list[logging.LogRecord]]: - records: list[logging.LogRecord] = [] - - class _ListHandler(logging.Handler): - def emit(self, record: logging.LogRecord) -> None: - records.append(record) - - logger = logging.getLogger(f"test_az317_{id(records)}") - logger.handlers.clear() - logger.addHandler(_ListHandler()) - logger.setLevel(logging.DEBUG) - logger.propagate = False - - return FlightStateGate(source=source, logger=logger), records - - -def _kinds(records: list[logging.LogRecord]) -> list[str]: - return [getattr(r, "kind", None) for r in records] - - -# ---------------------------------------------------------------------- -# AC-1: ON_GROUND passes -# ---------------------------------------------------------------------- - - -def test_ac1_on_ground_returns_signal_and_emits_info_log() -> None: - # Arrange - source = _FakeSource(FlightStateSignal.ON_GROUND) - gate, records = _build_gate(source=source) - - # Act - result = gate.confirm_on_ground() - - # Assert - assert result is FlightStateSignal.ON_GROUND - assert _kinds(records) == ["c11.upload.flight_state_confirmed"] - assert records[0].levelname == "INFO" - assert source.call_count == 1 - - -# ---------------------------------------------------------------------- -# AC-2: IN_FLIGHT raises -# ---------------------------------------------------------------------- - - -def test_ac2_in_flight_raises_with_observed_and_error_log() -> None: - # Arrange - source = _FakeSource(FlightStateSignal.IN_FLIGHT) - gate, records = _build_gate(source=source) - - # Act + Assert - with pytest.raises(FlightStateNotOnGroundError) as excinfo: - gate.confirm_on_ground() - - assert excinfo.value.observed is FlightStateSignal.IN_FLIGHT - assert "IN_FLIGHT" in str(excinfo.value) - assert _kinds(records) == ["c11.upload.refused.flight_state"] - assert records[0].levelname == "ERROR" - - -# ---------------------------------------------------------------------- -# AC-3: UNKNOWN raises (fail-closed) -# ---------------------------------------------------------------------- - - -def test_ac3_unknown_raises_fail_closed() -> None: - # Arrange - source = _FakeSource(FlightStateSignal.UNKNOWN) - gate, records = _build_gate(source=source) - - # Act + Assert - with pytest.raises(FlightStateNotOnGroundError) as excinfo: - gate.confirm_on_ground() - - assert excinfo.value.observed is FlightStateSignal.UNKNOWN - assert _kinds(records) == ["c11.upload.refused.flight_state"] - - -# ---------------------------------------------------------------------- -# AC-4: TAKING_OFF and LANDING raise -# ---------------------------------------------------------------------- - - -@pytest.mark.parametrize( - "transition_signal", - [FlightStateSignal.TAKING_OFF, FlightStateSignal.LANDING], -) -def test_ac4_transition_states_raise( - transition_signal: FlightStateSignal, -) -> None: - # Arrange - source = _FakeSource(transition_signal) - gate, records = _build_gate(source=source) - - # Act + Assert - with pytest.raises(FlightStateNotOnGroundError) as excinfo: - gate.confirm_on_ground() - - assert excinfo.value.observed is transition_signal - assert _kinds(records) == ["c11.upload.refused.flight_state"] - - -# ---------------------------------------------------------------------- -# AC-5: source exception → UNKNOWN with __cause__ chained -# ---------------------------------------------------------------------- - - -def test_ac5_source_exception_maps_to_unknown_and_preserves_cause() -> None: - # Arrange - original = RuntimeError("FC disconnected") - source = _RaisingSource(original) - gate, records = _build_gate(source=source) - - # Act + Assert - with pytest.raises(FlightStateNotOnGroundError) as excinfo: - gate.confirm_on_ground() - - assert excinfo.value.observed is FlightStateSignal.UNKNOWN - assert excinfo.value.__cause__ is original - assert _kinds(records) == ["c11.upload.refused.flight_state"] - assert records[0].levelname == "ERROR" - assert "FC disconnected" in records[0].kv["source_error"] - - -# ---------------------------------------------------------------------- -# AC-6: FlightStateSource Protocol is conformance-checkable -# ---------------------------------------------------------------------- - - -def test_ac6_protocol_isinstance_check_distinguishes_conforming_from_partial() -> None: - # Arrange - conforming = _FakeSource(FlightStateSignal.ON_GROUND) - non_conforming = _PartialFake() - - # Assert - assert isinstance(conforming, FlightStateSource) - assert not isinstance(non_conforming, FlightStateSource) - - -# ---------------------------------------------------------------------- -# AC-7: Error carries diagnostic fields -# ---------------------------------------------------------------------- - - -def test_ac7_error_carries_observed_and_observed_at_with_message_format() -> None: - # Arrange - source = _FakeSource(FlightStateSignal.IN_FLIGHT) - gate, _ = _build_gate(source=source) - - # Act - with pytest.raises(FlightStateNotOnGroundError) as excinfo: - gate.confirm_on_ground() - - # Assert - assert excinfo.value.observed is FlightStateSignal.IN_FLIGHT - assert isinstance(excinfo.value.observed_at, datetime) - assert excinfo.value.observed_at.tzinfo == timezone.utc - assert excinfo.value.observed_at.microsecond == 0 - assert str(excinfo.value).startswith("Upload refused: flight state is ") - - -# ---------------------------------------------------------------------- -# AC-8: Gate calls source exactly once -# ---------------------------------------------------------------------- - - -def test_ac8_gate_calls_source_exactly_once_no_retry() -> None: - # Arrange - source = _FakeSource(FlightStateSignal.IN_FLIGHT) - gate, _ = _build_gate(source=source) - - # Act - with pytest.raises(FlightStateNotOnGroundError): - gate.confirm_on_ground() - - # Assert - assert source.call_count == 1 - - -# ---------------------------------------------------------------------- -# NFR-perf: confirm_on_ground microbench p99 ≤ 1 ms -# ---------------------------------------------------------------------- - - -def test_nfr_perf_microbench_under_one_ms_p99() -> None: - # Arrange - source = _FakeSource(FlightStateSignal.ON_GROUND) - gate, _ = _build_gate(source=source) - iterations = 5_000 - - # Act - samples_ns: list[int] = [] - for _ in range(iterations): - start = time.perf_counter_ns() - gate.confirm_on_ground() - samples_ns.append(time.perf_counter_ns() - start) - - # Assert - samples_ns.sort() - p99_ns = samples_ns[int(iterations * 0.99) - 1] - assert p99_ns < 1_000_000, ( - f"p99 latency {p99_ns} ns exceeds 1 ms (1_000_000 ns) NFR budget" - ) - - -# ---------------------------------------------------------------------- -# NFR-reliability-fail-closed: every non-ON_GROUND state raises -# ---------------------------------------------------------------------- - - -@pytest.mark.parametrize( - "non_on_ground_signal", - [ - FlightStateSignal.IN_FLIGHT, - FlightStateSignal.TAKING_OFF, - FlightStateSignal.LANDING, - FlightStateSignal.UNKNOWN, - ], -) -def test_nfr_reliability_fail_closed_matrix_complete( - non_on_ground_signal: FlightStateSignal, -) -> None: - # Arrange - source = _FakeSource(non_on_ground_signal) - gate, _ = _build_gate(source=source) - - # Act + Assert - with pytest.raises(FlightStateNotOnGroundError): - gate.confirm_on_ground() diff --git a/tests/unit/c11_tile_manager/test_idempotent_retry.py b/tests/unit/c11_tile_manager/test_idempotent_retry.py index 7d6e656..d2ccdfa 100644 --- a/tests/unit/c11_tile_manager/test_idempotent_retry.py +++ b/tests/unit/c11_tile_manager/test_idempotent_retry.py @@ -26,8 +26,6 @@ import pytest from gps_denied_onboard.components.c11_tile_manager import ( C11RetryConfig, - FlightStateNotOnGroundError, - FlightStateSignal, IdempotentRetryTileUploader, IngestStatus, PerTileStatus, @@ -76,7 +74,6 @@ class _ScriptedInner: self.raises = list(raise_on_call or []) self.calls: list[UploadRequest] = [] self.enumerate_calls: list[Any] = [] - self.confirm_calls: int = 0 def upload_pending_tiles(self, request: UploadRequest) -> UploadBatchReport: self.calls.append(request) @@ -94,10 +91,6 @@ class _ScriptedInner: self.enumerate_calls.append(flight_id) return [{"sentinel": True, "flight_id": flight_id}] - def confirm_flight_state(self) -> FlightStateSignal: - self.confirm_calls += 1 - return FlightStateSignal.ON_GROUND - @dataclass class _FakeMetadataStore: @@ -388,39 +381,11 @@ def test_ac11_enumerate_pending_passes_through() -> None: assert out == [{"sentinel": True, "flight_id": fid}] -def test_ac11_confirm_flight_state_passes_through() -> None: - # Arrange - inner = _ScriptedInner(reports=[_success(0)]) - (decorator, _logs, _store, _clk, _fdr) = _build_decorator(inner=inner) - - # Act - state = decorator.confirm_flight_state() - - # Assert - assert state == FlightStateSignal.ON_GROUND - assert inner.confirm_calls == 1 - - # ---------------------------------------------------------------------- # AC-12 — inner exception propagates without retry # ---------------------------------------------------------------------- -def test_ac12_flight_state_not_on_ground_propagates_without_retry() -> None: - # Arrange - from datetime import datetime, timezone - - err = FlightStateNotOnGroundError(FlightStateSignal.IN_FLIGHT, datetime.now(timezone.utc)) - inner = _ScriptedInner(raise_on_call=[err]) - (decorator, _logs, _store, clk, _fdr) = _build_decorator(inner=inner) - - # Act / Assert - with pytest.raises(FlightStateNotOnGroundError): - decorator.upload_pending_tiles(_request()) - assert clk.sleep_calls == [] - assert len(inner.calls) == 1 - - def test_ac12_satellite_provider_error_propagates_without_retry() -> None: # Arrange inner = _ScriptedInner(raise_on_call=[SatelliteProviderError("boom")]) @@ -493,7 +458,6 @@ def test_ac10_factory_returns_decorated_uploader_by_default() -> None: http_client=_httpx.Client(transport=transport), tile_store=object(), tile_metadata_store=object(), - flight_state_gate=object(), # type: ignore[arg-type] key_manager=object(), # type: ignore[arg-type] ) @@ -516,7 +480,6 @@ def test_ac10_factory_bypasses_decorator_when_flag_set() -> None: http_client=_httpx.Client(transport=transport), tile_store=object(), tile_metadata_store=object(), - flight_state_gate=object(), # type: ignore[arg-type] key_manager=object(), # type: ignore[arg-type] ) diff --git a/tests/unit/c11_tile_manager/test_protocol_conformance.py b/tests/unit/c11_tile_manager/test_protocol_conformance.py index 84bd2c6..bffc3c1 100644 --- a/tests/unit/c11_tile_manager/test_protocol_conformance.py +++ b/tests/unit/c11_tile_manager/test_protocol_conformance.py @@ -33,17 +33,12 @@ class _NullSleep: return None -class _PartialFakeMissingConfirm: - """Conformance counterexample: missing ``confirm_flight_state``.""" +class _PartialFakeMissingEnumerate: + """Conformance counterexample: missing ``enumerate_pending_tiles``.""" def upload_pending_tiles(self, request: object) -> object: # noqa: ARG002 return None - def enumerate_pending_tiles( - self, flight_id: object | None = None - ) -> list[object]: # noqa: ARG002 - return [] - class _PartialDownloaderMissingEnumerate: """Conformance counterexample: missing ``enumerate_remote_coverage``.""" @@ -67,7 +62,6 @@ def test_ac12_concrete_uploader_satisfies_protocol() -> None: http_client=httpx.Client(transport=transport), tile_store=object(), # type: ignore[arg-type] tile_metadata_store=object(), # type: ignore[arg-type] - flight_state_gate=object(), # type: ignore[arg-type] key_manager=object(), # type: ignore[arg-type] fdr_client=FakeFdrSink(_PRODUCER_ID), # type: ignore[arg-type] logger=logging.getLogger("test_az319_conformance"), @@ -81,7 +75,7 @@ def test_ac12_concrete_uploader_satisfies_protocol() -> None: def test_ac12_partial_fake_is_not_protocol_conformant() -> None: # Assert - assert not isinstance(_PartialFakeMissingConfirm(), TileUploader) + assert not isinstance(_PartialFakeMissingEnumerate(), TileUploader) def test_ac10_concrete_downloader_satisfies_protocol() -> None: @@ -129,7 +123,6 @@ def test_ac9_idempotent_retry_decorator_satisfies_uploader_protocol() -> None: http_client=httpx.Client(transport=transport), tile_store=object(), # type: ignore[arg-type] tile_metadata_store=object(), # type: ignore[arg-type] - flight_state_gate=object(), # type: ignore[arg-type] key_manager=object(), # type: ignore[arg-type] fdr_client=FakeFdrSink(_PRODUCER_ID), # type: ignore[arg-type] logger=logging.getLogger("test_az320_inner"), diff --git a/tests/unit/c11_tile_manager/test_tile_uploader.py b/tests/unit/c11_tile_manager/test_tile_uploader.py index 5e760cd..dc720d2 100644 --- a/tests/unit/c11_tile_manager/test_tile_uploader.py +++ b/tests/unit/c11_tile_manager/test_tile_uploader.py @@ -1,12 +1,14 @@ """AZ-319 ``HttpTileUploader`` unit tests. -Covers AC-1 .. AC-14 and the upload-throughput NFR from -``_docs/02_tasks/todo/AZ-319_c11_tile_uploader.md``. +Covers AC-1, AC-3 .. AC-14 and the upload-throughput NFR from +``_docs/02_tasks/done/AZ-319_c11_tile_uploader.md``. AC-2 (the legacy +ON_GROUND gate) was removed in batch 44 — gating is now C12's +``PostLandingUploadOrchestrator`` responsibility. Uses :class:`httpx.MockTransport` for deterministic HTTP responses, :class:`FakeFdrSink` for FDR capture, a list-backed ``logging.Handler`` -for log capture, and stub C6 stores / gate / key manager so this -suite never drags in AZ-303 / AZ-305 / AZ-317 / AZ-318 internals. +for log capture, and stub C6 stores / key manager so this suite never +drags in AZ-303 / AZ-305 / AZ-318 internals. """ from __future__ import annotations @@ -25,8 +27,6 @@ import pytest from gps_denied_onboard.components.c11_tile_manager import ( C11Config, - FlightStateNotOnGroundError, - FlightStateSignal, HttpTileUploader, IngestStatus, PerFlightKeyManager, @@ -37,9 +37,6 @@ from gps_denied_onboard.components.c11_tile_manager import ( UploadRequest, canonical_payload_bytes, ) -from gps_denied_onboard.components.c11_tile_manager.flight_state_gate import ( - FlightStateGate, -) from gps_denied_onboard.fdr_client import FdrRecord from gps_denied_onboard.fdr_client.fakes import FakeFdrSink @@ -125,25 +122,6 @@ class _FakeMetadataStore: self.mark_calls.append((tile_id, uploaded_at)) -class _StubGate: - """Stand-in for AZ-317 ``FlightStateGate``.""" - - def __init__( - self, signal: FlightStateSignal = FlightStateSignal.ON_GROUND - ) -> None: - self._signal = signal - self.confirm_calls: int = 0 - - def confirm_on_ground(self) -> FlightStateSignal: - self.confirm_calls += 1 - if self._signal != FlightStateSignal.ON_GROUND: - raise FlightStateNotOnGroundError( - self._signal, - datetime.now(timezone.utc), - ) - return self._signal - - class _StubKeyManager: """Stand-in for AZ-318 ``PerFlightKeyManager``. @@ -222,7 +200,6 @@ def _build_uploader( transport: httpx.MockTransport, pending: list[_FakeTile] | None = None, blobs: dict[str, bytes] | None = None, - gate_signal: FlightStateSignal = FlightStateSignal.ON_GROUND, fingerprint_hex: str = "0123456789abcdef", config: C11Config | None = None, sleep_recorder: list[float] | None = None, @@ -230,7 +207,6 @@ def _build_uploader( HttpTileUploader, FakeFdrSink, list[logging.LogRecord], - _StubGate, _StubKeyManager, _FakeTileStore, _FakeMetadataStore, @@ -249,7 +225,6 @@ def _build_uploader( logger.setLevel(logging.DEBUG) logger.propagate = False - gate = _StubGate(signal=gate_signal) key_manager = _StubKeyManager(fingerprint_hex=fingerprint_hex) tile_store = _FakeTileStore(blobs=blobs) metadata_store = _FakeMetadataStore(pending=pending) @@ -272,14 +247,13 @@ def _build_uploader( http_client=client, tile_store=tile_store, tile_metadata_store=metadata_store, - flight_state_gate=gate, # type: ignore[arg-type] key_manager=key_manager, # type: ignore[arg-type] fdr_client=fdr, # type: ignore[arg-type] logger=logger, config=cfg, sleep=_sleep, ) - return uploader, fdr, log_records, gate, key_manager, tile_store, metadata_store, sleeps + return uploader, fdr, log_records, key_manager, tile_store, metadata_store, sleeps def _make_request(*, batch_size: int = 10, flight_id: UUID | None = None) -> UploadRequest: @@ -361,7 +335,6 @@ def test_ac1_50_tile_happy_path_marks_all_uploaded() -> None: uploader, fdr, _logs, - _gate, key_manager, _tile_store, metadata_store, @@ -385,48 +358,6 @@ def test_ac1_50_tile_happy_path_marks_all_uploaded() -> None: assert key_manager.end_calls == 1 -# ---------------------------------------------------------------------- -# AC-2: gate blocks before any read or POST -# ---------------------------------------------------------------------- - - -def test_ac2_gate_blocks_before_any_read_or_post() -> None: - # Arrange - pending = [_make_tile()] - posted: list[httpx.Request] = [] - - def _handler(request: httpx.Request) -> httpx.Response: - posted.append(request) - return httpx.Response(202, json={"batch_uuid": str(uuid4()), "per_tile_status": []}) - - transport = httpx.MockTransport(_handler) - ( - uploader, - _fdr, - _logs, - gate, - key_manager, - tile_store, - metadata_store, - _sleeps, - ) = _build_uploader( - transport=transport, - pending=pending, - gate_signal=FlightStateSignal.IN_FLIGHT, - ) - - # Act / Assert - with pytest.raises(FlightStateNotOnGroundError): - uploader.upload_pending_tiles(_make_request()) - - assert gate.confirm_calls == 1 - assert metadata_store.pending_calls == 0 - assert tile_store.read_calls == [] - assert key_manager.start_calls == [] - assert key_manager.end_calls == 0 - assert posted == [] - - # ---------------------------------------------------------------------- # AC-3: signature rejection — record + skip mark_uploaded; outcome=partial # ---------------------------------------------------------------------- @@ -457,7 +388,6 @@ def test_ac3_signature_rejection_records_and_keeps_pending() -> None: uploader, fdr, _logs, - _gate, key_manager, _tile_store, metadata_store, @@ -504,7 +434,6 @@ def test_ac4_duplicate_and_superseded_are_success() -> None: uploader, _fdr, _logs, - _gate, _key_manager, _tile_store, metadata_store, @@ -536,7 +465,6 @@ def test_ac5_signing_key_zeroised_on_success() -> None: uploader, _fdr, _logs, - _gate, key_manager, _tile_store, _metadata_store, @@ -570,7 +498,6 @@ def test_ac6_signing_key_zeroised_on_failure() -> None: uploader, _fdr, _logs, - _gate, key_manager, _tile_store, metadata_store, @@ -605,7 +532,6 @@ def test_ac7_public_key_fdr_precedes_tile_fdr() -> None: uploader, fdr, _logs, - _gate, _key_manager, _tile_store, _metadata_store, @@ -661,7 +587,6 @@ def test_ac8_429_honours_retry_after_seconds() -> None: uploader, _fdr, _logs, - _gate, _key_manager, _tile_store, _metadata_store, @@ -697,7 +622,6 @@ def test_ac9_persistent_5xx_raises_satellite_provider_error() -> None: uploader, _fdr, _logs, - _gate, key_manager, _tile_store, _metadata_store, @@ -731,7 +655,6 @@ def test_ac10_401_fails_fast_no_retry() -> None: uploader, _fdr, log_records, - _gate, _key_manager, _tile_store, _metadata_store, @@ -766,7 +689,6 @@ def test_ac11_empty_pending_set_is_success_no_posts() -> None: uploader, fdr, _logs, - _gate, key_manager, _tile_store, _metadata_store, @@ -868,7 +790,6 @@ def test_ac14_partial_success_batch_does_not_raise() -> None: uploader, _fdr, _logs, - _gate, _key_manager, _tile_store, metadata_store, @@ -910,7 +831,6 @@ def test_429_budget_exhaustion_raises_rate_limited_error() -> None: uploader, _fdr, _logs, - _gate, key_manager, _tile_store, _metadata_store, @@ -951,7 +871,7 @@ def test_nfr_throughput_1000_tiles_under_budget() -> None: ) transport = httpx.MockTransport(_handler) - (uploader, _fdr, _logs, _gate, _km, _ts, _ms, _sleeps) = _build_uploader( + (uploader, _fdr, _logs, _km, _ts, _ms, _sleeps) = _build_uploader( transport=transport, pending=pending ) diff --git a/tests/unit/c12_operator_tooling/__init__.py b/tests/unit/c12_operator_orchestrator/__init__.py similarity index 100% rename from tests/unit/c12_operator_tooling/__init__.py rename to tests/unit/c12_operator_orchestrator/__init__.py diff --git a/tests/unit/c12_operator_tooling/test_az489_flights_api_client.py b/tests/unit/c12_operator_orchestrator/test_az489_flights_api_client.py similarity index 99% rename from tests/unit/c12_operator_tooling/test_az489_flights_api_client.py rename to tests/unit/c12_operator_orchestrator/test_az489_flights_api_client.py index a8fa2d0..637cd4a 100644 --- a/tests/unit/c12_operator_tooling/test_az489_flights_api_client.py +++ b/tests/unit/c12_operator_orchestrator/test_az489_flights_api_client.py @@ -22,7 +22,7 @@ import httpx import pytest from gps_denied_onboard._types.geo import BoundingBox, LatLonAlt -from gps_denied_onboard.components.c12_operator_tooling.flights_api import ( +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api import ( EmptyWaypointsError, FlightDto, FlightFileNotFoundError, diff --git a/tests/unit/c12_operator_tooling/test_build_cache_orchestrator.py b/tests/unit/c12_operator_orchestrator/test_build_cache_orchestrator.py similarity index 98% rename from tests/unit/c12_operator_tooling/test_build_cache_orchestrator.py rename to tests/unit/c12_operator_orchestrator/test_build_cache_orchestrator.py index a271588..269fc9f 100644 --- a/tests/unit/c12_operator_tooling/test_build_cache_orchestrator.py +++ b/tests/unit/c12_operator_orchestrator/test_build_cache_orchestrator.py @@ -18,7 +18,7 @@ from uuid import UUID import pytest from gps_denied_onboard._types.geo import BoundingBox, LatLonAlt -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( BuildCacheOrchestrator, BuildCacheOutcome, BuildCacheRequest, @@ -51,20 +51,20 @@ from gps_denied_onboard.components.c12_operator_tooling import ( WaypointObjective, WaypointSource, ) -from gps_denied_onboard.components.c12_operator_tooling.file_lock import LockTimeout -from gps_denied_onboard.components.c12_operator_tooling.flights_api.interface import ( +from gps_denied_onboard.components.c12_operator_orchestrator.file_lock import LockTimeout +from gps_denied_onboard.components.c12_operator_orchestrator.flights_api.interface import ( FlightDto, FlightsApiClient, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_c10_invoker import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_c10_invoker import ( RemoteBuildRequest, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( RemoteCommandResult, SshSession, SshSessionFactory, ) -from gps_denied_onboard.components.c12_operator_tooling.tile_downloader_cut import ( +from gps_denied_onboard.components.c12_operator_orchestrator.tile_downloader_cut import ( TileDownloaderCut, ) @@ -948,7 +948,7 @@ class TestCompositionRootSmoke: # Reasonable smoke: real CompanionBringup with a fake SSH factory # constructs without raising; the orchestrator pulls the same # instance via the services dataclass. - from gps_denied_onboard.components.c12_operator_tooling.remote_sidecar_verifier import ( + from gps_denied_onboard.components.c12_operator_orchestrator.remote_sidecar_verifier import ( RemoteSidecarVerifier, ) diff --git a/tests/unit/c12_operator_tooling/test_cli_build_cache.py b/tests/unit/c12_operator_orchestrator/test_cli_build_cache.py similarity index 98% rename from tests/unit/c12_operator_tooling/test_cli_build_cache.py rename to tests/unit/c12_operator_orchestrator/test_cli_build_cache.py index f7d5131..98cb155 100644 --- a/tests/unit/c12_operator_tooling/test_cli_build_cache.py +++ b/tests/unit/c12_operator_orchestrator/test_cli_build_cache.py @@ -23,7 +23,7 @@ from uuid import UUID import pytest from click.testing import CliRunner -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( EXIT_BUILD_FAILURE, EXIT_DOWNLOAD_FAILURE, EXIT_EMPTY_WAYPOINTS, @@ -42,7 +42,7 @@ from gps_denied_onboard.components.c12_operator_tooling import ( FlightFromFile, SectorClassification, ) -from gps_denied_onboard.components.c12_operator_tooling.cli import app +from gps_denied_onboard.components.c12_operator_orchestrator.cli import app _FLIGHT_ID = UUID("00000000-0000-0000-0000-000000000001") _API_KEY = "super-secret-api-key" diff --git a/tests/unit/c12_operator_tooling/test_cli_console_script.py b/tests/unit/c12_operator_orchestrator/test_cli_console_script.py similarity index 70% rename from tests/unit/c12_operator_tooling/test_cli_console_script.py rename to tests/unit/c12_operator_orchestrator/test_cli_console_script.py index 342c89a..9022ef2 100644 --- a/tests/unit/c12_operator_tooling/test_cli_console_script.py +++ b/tests/unit/c12_operator_orchestrator/test_cli_console_script.py @@ -1,4 +1,4 @@ -"""AZ-326 AC-8 — `operator-tool` console script is installed and runnable.""" +"""AZ-326 AC-8 — `operator-orchestrator` console script is installed and runnable.""" from __future__ import annotations @@ -12,35 +12,35 @@ import pytest @pytest.fixture(scope="module") -def operator_tool_binary() -> str: +def operator_orchestrator_binary() -> str: # Prefer PATH (mimics operator install). Fall back to the active Python # interpreter's bin directory so the test still runs in an unactivated # venv (`.venv/bin/pytest ...`), which is the common CI invocation. - candidate = shutil.which("operator-tool") + candidate = shutil.which("operator-orchestrator") if candidate is not None: return candidate - venv_bin = Path(sys.executable).parent / "operator-tool" + venv_bin = Path(sys.executable).parent / "operator-orchestrator" if venv_bin.exists(): return str(venv_bin) - pytest.skip("operator-tool console script not on PATH or in venv bin") + pytest.skip("operator-orchestrator console script not on PATH or in venv bin") class TestConsoleScript: - def test_help_exits_zero(self, operator_tool_binary: str) -> None: + def test_help_exits_zero(self, operator_orchestrator_binary: str) -> None: # Act result = subprocess.run( - [operator_tool_binary, "--help"], + [operator_orchestrator_binary, "--help"], capture_output=True, text=True, timeout=10, ) # Assert assert result.returncode == 0, result.stderr - assert "operator-tool" in result.stdout + assert "operator-orchestrator" in result.stdout @pytest.mark.slow - def test_cold_start_under_500ms_p99(self, operator_tool_binary: str) -> None: - """NFR-perf-cold-start — `operator-tool --help` ≤ 500 ms p99 over 11 runs. + def test_cold_start_under_500ms_p99(self, operator_orchestrator_binary: str) -> None: + """NFR-perf-cold-start — `operator-orchestrator --help` ≤ 500 ms p99 over 11 runs. Methodology: 11 cold-start subprocess runs, drop the single worst sample (system noise: OS context switch, disk cache @@ -55,7 +55,7 @@ class TestConsoleScript: for _ in range(11): start = time.monotonic() subprocess.run( - [operator_tool_binary, "--help"], + [operator_orchestrator_binary, "--help"], capture_output=True, text=True, check=True, diff --git a/tests/unit/c12_operator_tooling/test_cli_help_and_logging.py b/tests/unit/c12_operator_orchestrator/test_cli_help_and_logging.py similarity index 91% rename from tests/unit/c12_operator_tooling/test_cli_help_and_logging.py rename to tests/unit/c12_operator_orchestrator/test_cli_help_and_logging.py index d315db5..6d15504 100644 --- a/tests/unit/c12_operator_tooling/test_cli_help_and_logging.py +++ b/tests/unit/c12_operator_orchestrator/test_cli_help_and_logging.py @@ -14,10 +14,10 @@ from types import SimpleNamespace import pytest from click.testing import CliRunner -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( EXIT_OK, ) -from gps_denied_onboard.components.c12_operator_tooling.cli import app +from gps_denied_onboard.components.c12_operator_orchestrator.cli import app _EXPECTED_SUBCOMMANDS = { "download", @@ -42,7 +42,7 @@ def isolated_log(tmp_path: Path) -> Path: class TestSubcommandRegistration: - """AC-1 — `operator-tool --help` lists exactly the six subcommands.""" + """AC-1 — `operator-orchestrator --help` lists exactly the six subcommands.""" def test_top_level_help_lists_all_six_subcommands(self, runner: CliRunner) -> None: # Act @@ -92,11 +92,11 @@ class TestSuccessfulSetSectorAcTwo: config_obj = SimpleNamespace() # Inject a config via the --log-path override + per-test sector store # by calling the underlying Click command directly with a custom obj. - from gps_denied_onboard.components.c12_operator_tooling import ( + from gps_denied_onboard.components.c12_operator_orchestrator import ( C12Config, HostKeyPolicy, ) - from gps_denied_onboard.components.c12_operator_tooling.config import ( + from gps_denied_onboard.components.c12_operator_orchestrator.config import ( C12CompanionConfig, ) @@ -142,7 +142,7 @@ class TestStructuredLoggingShapeAcSeven: ) -> None: # Arrange store_path = tmp_path / "sector.json" - from gps_denied_onboard.components.c12_operator_tooling import C12Config + from gps_denied_onboard.components.c12_operator_orchestrator import C12Config # Act result = runner.invoke( diff --git a/tests/unit/c12_operator_tooling/test_companion_bringup.py b/tests/unit/c12_operator_orchestrator/test_companion_bringup.py similarity index 98% rename from tests/unit/c12_operator_tooling/test_companion_bringup.py rename to tests/unit/c12_operator_orchestrator/test_companion_bringup.py index 7bb9416..a33c325 100644 --- a/tests/unit/c12_operator_tooling/test_companion_bringup.py +++ b/tests/unit/c12_operator_orchestrator/test_companion_bringup.py @@ -9,7 +9,7 @@ from pathlib import Path, PurePosixPath import pytest -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( C12CompanionConfig, CompanionAddress, CompanionBringup, @@ -19,10 +19,10 @@ from gps_denied_onboard.components.c12_operator_tooling import ( HostKeyPolicy, ReadinessOutcome, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_sidecar_verifier import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_sidecar_verifier import ( RemoteSidecarResult, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( RemoteCommandResult, SshSession, SshSessionFactory, diff --git a/tests/unit/c12_operator_tooling/test_exit_codes.py b/tests/unit/c12_operator_orchestrator/test_exit_codes.py similarity index 93% rename from tests/unit/c12_operator_tooling/test_exit_codes.py rename to tests/unit/c12_operator_orchestrator/test_exit_codes.py index 4bf2988..4a12148 100644 --- a/tests/unit/c12_operator_tooling/test_exit_codes.py +++ b/tests/unit/c12_operator_orchestrator/test_exit_codes.py @@ -2,7 +2,7 @@ from __future__ import annotations -from gps_denied_onboard.components.c12_operator_tooling import exit_codes +from gps_denied_onboard.components.c12_operator_orchestrator import exit_codes class TestExitCodes: diff --git a/tests/unit/c12_operator_orchestrator/test_fdr_footer_reader.py b/tests/unit/c12_operator_orchestrator/test_fdr_footer_reader.py new file mode 100644 index 0000000..fdc8df3 --- /dev/null +++ b/tests/unit/c12_operator_orchestrator/test_fdr_footer_reader.py @@ -0,0 +1,430 @@ +"""AZ-329 LocalFdrFooterReader unit + integration tests. + +Covers: +* AC-6 — newest-segment-first short-circuit (the reader opens only the + newest segment when the footer lives there). +* AC-9 — real FDR fixture C12-IT-03(a): clean-shutdown footer present → + the reader returns the parsed :class:`FlightFooterRecord`. +* AC-10 — real FDR fixture C12-IT-03(b): no-footer truncation → the + reader returns ``None`` and the orchestrator refuses with + ``footer_missing`` (integration via a tiny composition of orchestrator + + reader on the truncated fixture). +* Frame-corruption paths (truncated length prefix, truncated body, + invalid JSON, wrong-flight UUID, payload schema mismatch) → + :class:`FdrUnreadableError`. + +The fixtures are generated in-test using the same length-prefixed +serialisation the C13 writer uses (``struct.Struct(' bytes: + body = serialise(record) + return _LENGTH_PREFIX.pack(len(body)) + body + + +def _make_record(*, kind: str, payload: dict[str, Any]) -> FdrRecord: + return FdrRecord( + schema_version=1, + ts="2026-05-13T12:00:00+00:00", + producer_id="test.producer", + kind=kind, + payload=payload, + ) + + +def _make_footer_record(*, flight_id: UUID, clean_shutdown: bool) -> FdrRecord: + return _make_record( + kind="flight_footer", + payload={ + "flight_id": str(flight_id), + "flight_ended_at_iso": "2026-05-13T12:00:00+00:00", + "flight_ended_at_monotonic_ns": 1234567890, + "records_written": 9876, + "records_dropped_overrun": 0 if clean_shutdown else 5, + "bytes_written": 1024 * 1024, + "rollover_count": 0, + "clean_shutdown": clean_shutdown, + }, + ) + + +def _write_segment( + flight_dir: Path, segment_index: int, records: list[FdrRecord] +) -> Path: + path = flight_dir / f"segment-{segment_index:04d}.fdr" + with open(path, "wb") as fh: + for record in records: + fh.write(_frame_payload(record)) + return path + + +def _setup_flight_dir(tmp_path: Path, flight_id: UUID) -> tuple[Path, Path]: + fdr_root = tmp_path / "fdr" + fdr_root.mkdir() + flight_dir = fdr_root / str(flight_id) + flight_dir.mkdir() + return fdr_root, flight_dir + + +# ---------------------------------------------------------------------- +# AC-6 — newest-segment-first short-circuit +# ---------------------------------------------------------------------- + + +def test_ac6_reader_short_circuits_on_newest_segment(tmp_path, monkeypatch) -> None: + # Arrange — three segments; footer lives in the newest (index 2). + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + _write_segment( + flight_dir, 0, [_make_record(kind="log", payload={"msg": "early"})] + ) + _write_segment( + flight_dir, 1, [_make_record(kind="log", payload={"msg": "middle"})] + ) + _write_segment( + flight_dir, + 2, + [_make_footer_record(flight_id=flight_id, clean_shutdown=True)], + ) + + opened_paths: list[str] = [] + real_open = open + + def _tracking_open(file, *args, **kwargs): + if str(file).endswith(".fdr"): + opened_paths.append(str(file)) + return real_open(file, *args, **kwargs) + + monkeypatch.setattr( + "gps_denied_onboard.components.c12_operator_orchestrator.fdr_footer_reader.open", + _tracking_open, + raising=False, + ) + + reader = LocalFdrFooterReader(fdr_root) + + # Act + footer = reader.read_footer(flight_id) + + # Assert + assert footer is not None + assert footer.flight_id == flight_id + assert footer.clean_shutdown is True + assert len(opened_paths) == 1 + assert opened_paths[0].endswith("segment-0002.fdr") + + +def test_ac6_reader_walks_older_segments_when_newest_has_no_footer(tmp_path) -> None: + # Arrange — footer lives in segment 0 (oldest); newer segments have no footer. + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + _write_segment( + flight_dir, + 0, + [_make_footer_record(flight_id=flight_id, clean_shutdown=True)], + ) + _write_segment( + flight_dir, 1, [_make_record(kind="log", payload={"msg": "stray"})] + ) + _write_segment( + flight_dir, 2, [_make_record(kind="log", payload={"msg": "stray2"})] + ) + + reader = LocalFdrFooterReader(fdr_root) + + # Act + footer = reader.read_footer(flight_id) + + # Assert + assert footer is not None + assert footer.clean_shutdown is True + + +def test_reader_returns_none_when_no_footer_anywhere(tmp_path) -> None: + # Arrange + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + _write_segment( + flight_dir, 0, [_make_record(kind="log", payload={"msg": "only-log"})] + ) + + reader = LocalFdrFooterReader(fdr_root) + + # Act + footer = reader.read_footer(flight_id) + + # Assert + assert footer is None + + +# ---------------------------------------------------------------------- +# Parse / framing corruption → FdrUnreadableError +# ---------------------------------------------------------------------- + + +def test_reader_raises_on_truncated_length_prefix(tmp_path) -> None: + # Arrange — segment file ends with a single byte (not a complete prefix). + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + path = flight_dir / "segment-0000.fdr" + path.write_bytes(b"\x01") + + reader = LocalFdrFooterReader(fdr_root) + + # Act / Assert + with pytest.raises(FdrUnreadableError, match="truncated length prefix"): + reader.read_footer(flight_id) + + +def test_reader_raises_on_truncated_body(tmp_path) -> None: + # Arrange — length prefix claims 100 bytes; only 10 are present. + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + path = flight_dir / "segment-0000.fdr" + path.write_bytes(_LENGTH_PREFIX.pack(100) + b"x" * 10) + + reader = LocalFdrFooterReader(fdr_root) + + # Act / Assert + with pytest.raises(FdrUnreadableError, match="truncated record body"): + reader.read_footer(flight_id) + + +def test_reader_raises_on_invalid_json_body(tmp_path) -> None: + # Arrange — length matches but JSON is garbage. + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + body = b"this is not json" + path = flight_dir / "segment-0000.fdr" + path.write_bytes(_LENGTH_PREFIX.pack(len(body)) + body) + + reader = LocalFdrFooterReader(fdr_root) + + # Act / Assert + with pytest.raises(FdrUnreadableError, match="failed to parse record"): + reader.read_footer(flight_id) + + +def test_reader_raises_when_footer_flight_id_mismatches(tmp_path) -> None: + # Arrange — footer carries a different flight_id than the requested one. + requested = uuid4() + other = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, requested) + _write_segment( + flight_dir, 0, [_make_footer_record(flight_id=other, clean_shutdown=True)] + ) + + reader = LocalFdrFooterReader(fdr_root) + + # Act / Assert + with pytest.raises(FdrUnreadableError, match="flight_footer.flight_id mismatch"): + reader.read_footer(requested) + + +def test_reader_raises_when_footer_payload_misses_required_field(tmp_path) -> None: + # Arrange — footer payload missing `clean_shutdown`; build the bytes + # by hand so we bypass the serialise validator (which would not catch + # this because `flight_footer` allows known fields without requiring + # all of them per AZ-272's forward-compat policy). + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + bad_payload = { + "flight_id": str(flight_id), + "flight_ended_at_iso": "2026-05-13T12:00:00+00:00", + "flight_ended_at_monotonic_ns": 0, + "records_written": 1, + "records_dropped_overrun": 0, + "bytes_written": 0, + "rollover_count": 0, + # clean_shutdown intentionally omitted + } + envelope = { + "schema_version": 1, + "ts": "2026-05-13T12:00:00+00:00", + "producer_id": "test.producer", + "kind": "flight_footer", + "payload": bad_payload, + } + body = orjson.dumps(envelope) + path = flight_dir / "segment-0000.fdr" + path.write_bytes(_LENGTH_PREFIX.pack(len(body)) + body) + + reader = LocalFdrFooterReader(fdr_root) + + # Act / Assert + with pytest.raises(FdrUnreadableError, match="flight_footer payload schema violation"): + reader.read_footer(flight_id) + + +def test_reader_ignores_non_segment_files_in_flight_dir(tmp_path) -> None: + # Arrange — directory contains a stray `.log` file alongside the + # segment with the footer. + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + _write_segment( + flight_dir, + 0, + [_make_footer_record(flight_id=flight_id, clean_shutdown=True)], + ) + (flight_dir / "operator-notes.log").write_text("ignore me") + (flight_dir / "segment-bad.txt").write_text("ignore me too") + + reader = LocalFdrFooterReader(fdr_root) + + # Act + footer = reader.read_footer(flight_id) + + # Assert + assert footer is not None + assert footer.clean_shutdown is True + + +# ---------------------------------------------------------------------- +# AC-9 — integration: clean-shutdown footer fixture → upload invoked +# ---------------------------------------------------------------------- + + +class _RecordingUploader: + def __init__(self, report) -> None: + self.report = report + self.calls: list = [] + + def upload_pending_tiles(self, request): + self.calls.append(request) + return self.report + + +def test_ac9_integration_clean_shutdown_fixture_triggers_upload(tmp_path) -> None: + # Arrange — real segment files on disk + LocalFdrFooterReader + recording uploader. + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + _write_segment( + flight_dir, 0, [_make_record(kind="log", payload={"msg": "pre-takeoff"})] + ) + _write_segment( + flight_dir, 1, [_make_record(kind="log", payload={"msg": "mid-flight"})] + ) + _write_segment( + flight_dir, + 2, + [ + _make_record(kind="log", payload={"msg": "landing rollout"}), + _make_footer_record(flight_id=flight_id, clean_shutdown=True), + ], + ) + reader = LocalFdrFooterReader(fdr_root) + + from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + IngestStatusCut, + PerTileStatusCut, + UploadBatchReportCut, + UploadOutcomeCut, + ) + + fake_report = UploadBatchReportCut( + batch_uuid=uuid4(), + per_tile_status=( + PerTileStatusCut(tile_id="tile-A", status=IngestStatusCut.ACCEPTED), + ), + retry_count=0, + next_retry_at_s=None, + outcome=UploadOutcomeCut.SUCCESS, + public_key_fingerprint="cd" * 8, + ) + uploader = _RecordingUploader(fake_report) + orchestrator = PostLandingUploadOrchestrator( + tile_uploader=uploader, + fdr_footer_reader=reader, + logger=logging.getLogger("test.c12.it03a"), + config=C12PostLandingConfig(fdr_root=fdr_root), + ) + request = PostLandingUploadRequest( + flight_id=flight_id, + satellite_provider_url="https://parent.example/ingest", + api_key="key-it03a", + batch_size=25, + ) + + # Act + returned = orchestrator.trigger_post_landing_upload(request) + + # Assert + assert returned is fake_report + assert len(uploader.calls) == 1 + assert uploader.calls[0].flight_id == flight_id + assert uploader.calls[0].batch_size == 25 + + +# ---------------------------------------------------------------------- +# AC-10 — integration: truncated fixture (no footer anywhere) → refusal +# ---------------------------------------------------------------------- + + +def test_ac10_integration_truncated_fixture_refuses_with_footer_missing(tmp_path) -> None: + # Arrange — simulate a truncated flight: segments exist with `log` + # records but the writer terminated before close_flight() emitted the + # footer record. + flight_id = uuid4() + fdr_root, flight_dir = _setup_flight_dir(tmp_path, flight_id) + _write_segment( + flight_dir, 0, [_make_record(kind="log", payload={"msg": "in flight"})] + ) + _write_segment( + flight_dir, 1, [_make_record(kind="log", payload={"msg": "still flying"})] + ) + reader = LocalFdrFooterReader(fdr_root) + uploader = _RecordingUploader(None) + orchestrator = PostLandingUploadOrchestrator( + tile_uploader=uploader, + fdr_footer_reader=reader, + logger=logging.getLogger("test.c12.it03b"), + config=C12PostLandingConfig(fdr_root=fdr_root), + ) + request = PostLandingUploadRequest( + flight_id=flight_id, + satellite_provider_url="https://parent.example/ingest", + api_key="key-it03b", + batch_size=25, + ) + + # Act / Assert + with pytest.raises(FlightStateNotConfirmedError) as exc_info: + orchestrator.trigger_post_landing_upload(request) + assert exc_info.value.not_confirmed_reason == "footer_missing" + assert uploader.calls == [] diff --git a/tests/unit/c12_operator_tooling/test_file_lock.py b/tests/unit/c12_operator_orchestrator/test_file_lock.py similarity index 96% rename from tests/unit/c12_operator_tooling/test_file_lock.py rename to tests/unit/c12_operator_orchestrator/test_file_lock.py index da5b039..20fa741 100644 --- a/tests/unit/c12_operator_tooling/test_file_lock.py +++ b/tests/unit/c12_operator_orchestrator/test_file_lock.py @@ -6,7 +6,7 @@ from pathlib import Path import pytest -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( FilelockFileLockFactory, LockTimeout, ) diff --git a/tests/unit/c12_operator_tooling/test_freshness_table.py b/tests/unit/c12_operator_orchestrator/test_freshness_table.py similarity index 95% rename from tests/unit/c12_operator_tooling/test_freshness_table.py rename to tests/unit/c12_operator_orchestrator/test_freshness_table.py index ecca16e..2536668 100644 --- a/tests/unit/c12_operator_tooling/test_freshness_table.py +++ b/tests/unit/c12_operator_orchestrator/test_freshness_table.py @@ -4,7 +4,7 @@ from __future__ import annotations import pytest -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( FRESHNESS_TABLE, SectorClassification, freshness_threshold_months, diff --git a/tests/unit/c12_operator_orchestrator/test_operator_reloc_service.py b/tests/unit/c12_operator_orchestrator/test_operator_reloc_service.py new file mode 100644 index 0000000..19ee3f3 --- /dev/null +++ b/tests/unit/c12_operator_orchestrator/test_operator_reloc_service.py @@ -0,0 +1,432 @@ +"""AZ-330 OperatorReLocService unit tests. + +Covers AC-1..AC-9 directly against the service with fakes. AC-10 (lazy +construction of the transport in the composition root) lives in +:mod:`test_cli_help_and_logging` / a composition-root regression — the +factory is verified to NOT call the transport constructor unless the +operator supplies one. +""" + +from __future__ import annotations + +import logging +from dataclasses import dataclass, field + +import pytest + +from gps_denied_onboard._types.geo import LatLonAlt +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + ReLocHint, +) +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( + GcsLinkError, +) +from gps_denied_onboard.components.c12_operator_orchestrator.operator_reloc_service import ( + OperatorReLocService, +) +from gps_denied_onboard.fdr_client import EnqueueResult, FdrClient + + +@dataclass +class _FakeTransport: + """Configurable :class:`OperatorCommandTransport` for the unit tests.""" + + raises: GcsLinkError | None = None + calls: list[ReLocHint] = field(default_factory=list) + + def send_reloc_hint(self, hint: ReLocHint) -> None: + self.calls.append(hint) + if self.raises is not None: + raise self.raises + + +@dataclass +class _FakeClock: + """Deterministic Clock — ``monotonic_ns`` increments on every call.""" + + next_monotonic_ns: int = 1_000_000_000 + fixed_time_ns: int = 1_715_600_000_000_000_000 # ~2024-05-13T12:53:20Z + + def monotonic_ns(self) -> int: + v = self.next_monotonic_ns + self.next_monotonic_ns += 1 + return v + + def time_ns(self) -> int: + return self.fixed_time_ns + + def sleep_until_ns(self, target_ns: int) -> None: + _ = target_ns + + +def _make_fdr_client(*, force_overrun: bool = False) -> FdrClient: + client = FdrClient( + producer_id="c12_operator_orchestrator", + capacity=4, + _emit_diag_log=False, + ) + if force_overrun: + # Fill the buffer to its rounded-up power-of-two capacity so the + # next enqueue returns OVERRUN (AC-8). + from gps_denied_onboard.fdr_client.records import ( + CURRENT_SCHEMA_VERSION, + FdrRecord, + ) + + filler = FdrRecord( + schema_version=CURRENT_SCHEMA_VERSION, + ts="2026-05-13T00:00:00.000000+00:00", + producer_id=client.producer_id, + kind="log", + payload={ + "level": "INFO", + "component": "test", + "frame_id": "", + "kind": "test", + "msg": "filler", + }, + ) + for _ in range(client._capacity()): + client.enqueue(filler) + return client + + +def _make_hint( + *, + lat: float = 49.99876543, + lon: float = 36.12345678, + alt: float = 1234.5, + radius: float = 50.0, + reason: str = "lost track at WP3", +) -> ReLocHint: + return ReLocHint( + approximate_position_wgs84=LatLonAlt(lat_deg=lat, lon_deg=lon, alt_m=alt), + confidence_radius_m=radius, + reason=reason, + ) + + +def _build_service( + *, + transport: _FakeTransport, + fdr_client: FdrClient, + clock: _FakeClock, +) -> OperatorReLocService: + logger = logging.getLogger("c12.operator_reloc_service.test") + return OperatorReLocService( + transport=transport, + fdr_client=fdr_client, + logger=logger, + clock=clock, + ) + + +# --------------------------------------------------------------------------- +# AC-1: success → transport called once + INFO log + FDR record "sent" +# --------------------------------------------------------------------------- + + +def test_request_reloc_success_calls_transport_once_and_emits_fdr( + caplog: pytest.LogCaptureFixture, +) -> None: + # Arrange + transport = _FakeTransport() + fdr_client = _make_fdr_client() + clock = _FakeClock() + service = _build_service(transport=transport, fdr_client=fdr_client, clock=clock) + hint = _make_hint() + + # Act + with caplog.at_level(logging.INFO, logger="c12.operator_reloc_service.test"): + service.request_reloc(hint) + + # Assert — transport + assert len(transport.calls) == 1 + assert transport.calls[0] is hint + + # Assert — INFO log + sent_records = [r for r in caplog.records if r.kind == "c12.reloc.sent"] + assert len(sent_records) == 1 + log_kv = sent_records[0].kv + assert log_kv["position_lat"] == pytest.approx(49.99877) + assert log_kv["position_lon"] == pytest.approx(36.12346) + assert log_kv["confidence_radius_m"] == 50.0 + assert log_kv["reason"] == "lost track at WP3" + assert log_kv["altitude_m"] == 1234.5 + + # Assert — exactly one FDR record with outcome="sent" + record = fdr_client.pop_one() + assert record is not None + assert record.kind == "c12.reloc.requested" + assert record.payload["outcome"] == "sent" + assert record.payload["hint"]["reason"] == "lost track at WP3" + assert record.payload["hint"]["lat_deg"] == 49.99876543 + assert record.payload["hint"]["lon_deg"] == 36.12345678 + assert record.payload["hint"]["alt_m"] == 1234.5 + assert record.payload["hint"]["confidence_radius_m"] == 50.0 + assert "failure_reason" not in record.payload + assert isinstance(record.payload["ts_monotonic_ns"], int) + + +# --------------------------------------------------------------------------- +# AC-2: transport raises GcsLinkError → re-raise + ERROR log + FDR "failed" +# --------------------------------------------------------------------------- + + +def test_request_reloc_link_failure_reraises_with_c12_prefix_and_records_failure( + caplog: pytest.LogCaptureFixture, +) -> None: + # Arrange + inner = GcsLinkError( + reason="link signal lost", + wrapped_exception_repr="SerialTimeout(...)", + ) + transport = _FakeTransport(raises=inner) + fdr_client = _make_fdr_client() + clock = _FakeClock() + service = _build_service(transport=transport, fdr_client=fdr_client, clock=clock) + hint = _make_hint() + + # Act + with caplog.at_level(logging.ERROR, logger="c12.operator_reloc_service.test"): + with pytest.raises(GcsLinkError) as exc_info: + service.request_reloc(hint) + + # Assert — re-raise + cause chain preserves the original + outer = exc_info.value + assert outer.reason == "C12 reloc-confirm: link signal lost" + assert outer.__cause__ is inner + assert outer.wrapped_exception_repr is not None + assert "GcsLinkError" in outer.wrapped_exception_repr + + # Assert — ERROR log + failed_records = [r for r in caplog.records if r.kind == "c12.reloc.failed"] + assert len(failed_records) == 1 + log_kv = failed_records[0].kv + assert log_kv["failure_reason"] == "link signal lost" + assert log_kv["wrapped_exception_repr"] == "SerialTimeout(...)" + + # Assert — FDR record records the failure + record = fdr_client.pop_one() + assert record is not None + assert record.kind == "c12.reloc.requested" + assert record.payload["outcome"] == "failed" + assert record.payload["failure_reason"] == "link signal lost" + assert record.payload["hint"]["reason"] == "lost track at WP3" + + +# --------------------------------------------------------------------------- +# AC-3: ReLocHint(confidence_radius_m=0.0) → ValueError at construction +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("bad_radius", [0.0, -1.0, -0.0001]) +def test_reloc_hint_rejects_non_positive_radius(bad_radius: float) -> None: + with pytest.raises(ValueError, match="confidence_radius_m must be > 0"): + ReLocHint( + approximate_position_wgs84=LatLonAlt(lat_deg=49.0, lon_deg=36.0, alt_m=100.0), + confidence_radius_m=bad_radius, + reason="test", + ) + + +# --------------------------------------------------------------------------- +# AC-4: reason is preserved byte-for-byte through the transport call +# --------------------------------------------------------------------------- + + +def test_reason_byte_for_byte_through_transport_log_truncated( + caplog: pytest.LogCaptureFixture, +) -> None: + # Arrange + transport = _FakeTransport() + fdr_client = _make_fdr_client() + clock = _FakeClock() + service = _build_service(transport=transport, fdr_client=fdr_client, clock=clock) + long_reason = "x" * 300 + hint = _make_hint(reason=long_reason) + + # Act + with caplog.at_level(logging.INFO, logger="c12.operator_reloc_service.test"): + service.request_reloc(hint) + + # Assert — transport sees the unchanged hint + assert transport.calls[0].reason == long_reason + + # Assert — FDR record preserves the full reason + record = fdr_client.pop_one() + assert record is not None + assert record.payload["hint"]["reason"] == long_reason + + # Assert — INFO log truncates to 200 chars + sent_records = [r for r in caplog.records if r.kind == "c12.reloc.sent"] + assert len(sent_records) == 1 + assert sent_records[0].kv["reason"] == "x" * 200 + + +# --------------------------------------------------------------------------- +# AC-5: Contract document exists with the exact method signature +# --------------------------------------------------------------------------- + + +def test_operator_command_transport_contract_document_exists() -> None: + # Arrange + from pathlib import Path + + # The path is the current contract location; Phase F may move it + # to ``c12_operator_orchestrator/`` — both layouts are accepted so + # this test survives the Phase F rename. + candidates = [ + Path(__file__).resolve().parents[3] + / "_docs/02_document/contracts/c12_operator_orchestrator/operator_command_transport.md", + Path(__file__).resolve().parents[3] + / "_docs/02_document/contracts/c12_operator_tooling/operator_command_transport.md", + ] + + # Act + existing = [p for p in candidates if p.exists()] + + # Assert + assert existing, "expected operator_command_transport contract to exist in one of the known paths" + content = existing[0].read_text(encoding="utf-8") + assert "send_reloc_hint" in content + assert "ReLocHint" in content + assert "GcsLinkError" in content + assert "Versioning Rules" in content + assert content.count("TC-") >= 3 + + +# --------------------------------------------------------------------------- +# AC-6: ReLocHint(reason="") → ValueError +# --------------------------------------------------------------------------- + + +def test_reloc_hint_rejects_empty_reason() -> None: + with pytest.raises(ValueError, match="reason must be non-empty"): + ReLocHint( + approximate_position_wgs84=LatLonAlt(lat_deg=49.0, lon_deg=36.0, alt_m=100.0), + confidence_radius_m=50.0, + reason="", + ) + + +# --------------------------------------------------------------------------- +# AC-7: lat/lon out of range → ValueError at ReLocHint construction +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "lat, lon", + [ + (91.0, 36.0), + (-91.0, 36.0), + (49.0, 181.0), + (49.0, -180.0), # lower bound is strict + ], +) +def test_reloc_hint_rejects_out_of_range_lat_lon(lat: float, lon: float) -> None: + with pytest.raises(ValueError): + ReLocHint( + approximate_position_wgs84=LatLonAlt(lat_deg=lat, lon_deg=lon, alt_m=100.0), + confidence_radius_m=50.0, + reason="test", + ) + + +# --------------------------------------------------------------------------- +# AC-8: FDR enqueue OVERRUN does NOT raise +# --------------------------------------------------------------------------- + + +def test_request_reloc_succeeds_when_fdr_buffer_overruns() -> None: + # Arrange + transport = _FakeTransport() + fdr_client = _make_fdr_client(force_overrun=True) + clock = _FakeClock() + service = _build_service(transport=transport, fdr_client=fdr_client, clock=clock) + hint = _make_hint() + + enqueue_results: list[str] = [] + original_enqueue = fdr_client.enqueue + + def spy_enqueue(record): + result = original_enqueue(record) + enqueue_results.append(result) + return result + + fdr_client.enqueue = spy_enqueue # type: ignore[method-assign] + + # Act + service.request_reloc(hint) + + # Assert — transport call unaffected; enqueue observed OVERRUN + assert len(transport.calls) == 1 + assert enqueue_results == [EnqueueResult.OVERRUN] + + +# --------------------------------------------------------------------------- +# AC-9: Position logged at 5 decimals; transport sees full precision +# --------------------------------------------------------------------------- + + +def test_position_logged_at_5_decimals_transport_sees_full_precision( + caplog: pytest.LogCaptureFixture, +) -> None: + # Arrange + transport = _FakeTransport() + fdr_client = _make_fdr_client() + clock = _FakeClock() + service = _build_service(transport=transport, fdr_client=fdr_client, clock=clock) + hint = _make_hint(lat=49.99876543, lon=36.12345678, alt=42.0) + + # Act + with caplog.at_level(logging.INFO, logger="c12.operator_reloc_service.test"): + service.request_reloc(hint) + + # Assert — transport receives full precision + sent_hint = transport.calls[0] + assert sent_hint.approximate_position_wgs84.lat_deg == 49.99876543 + assert sent_hint.approximate_position_wgs84.lon_deg == 36.12345678 + + # Assert — log is rounded to 5 decimals + sent_records = [r for r in caplog.records if r.kind == "c12.reloc.sent"] + assert sent_records[0].kv["position_lat"] == pytest.approx(49.99877) + assert sent_records[0].kv["position_lon"] == pytest.approx(36.12346) + + +# --------------------------------------------------------------------------- +# AC-10: factory does NOT construct the transport unless one is passed in +# --------------------------------------------------------------------------- + + +def test_build_operator_orchestrator_does_not_construct_operator_reloc_service_without_transport( + tmp_path, +) -> None: + # Arrange + from pathlib import Path + + from gps_denied_onboard.components.c12_operator_orchestrator.config import ( + C12CompanionConfig, + C12Config, + HostKeyPolicy, + ) + from gps_denied_onboard.config import Config + + config = Config() + config.components["c12_operator_orchestrator"] = C12Config( + log_path=tmp_path / "c12.log", + sector_classification_store_path=tmp_path / "sectors.json", + companion=C12CompanionConfig( + ssh_user="op", + ssh_keyfile=Path("/tmp/fake-key"), + host_key_policy=HostKeyPolicy.STRICT, + ), + ) + + from gps_denied_onboard.runtime_root.c12_factory import build_operator_orchestrator + + # Act — no operator_command_transport supplied + services = build_operator_orchestrator(config) + + # Assert + assert services.operator_reloc_service is None diff --git a/tests/unit/c12_operator_tooling/test_paramiko_factory_smoke.py b/tests/unit/c12_operator_orchestrator/test_paramiko_factory_smoke.py similarity index 95% rename from tests/unit/c12_operator_tooling/test_paramiko_factory_smoke.py rename to tests/unit/c12_operator_orchestrator/test_paramiko_factory_smoke.py index 3d7d52a..566ea01 100644 --- a/tests/unit/c12_operator_tooling/test_paramiko_factory_smoke.py +++ b/tests/unit/c12_operator_orchestrator/test_paramiko_factory_smoke.py @@ -12,7 +12,7 @@ from pathlib import Path import paramiko import pytest -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( HostKeyPolicy, ParamikoSshSessionFactory, ) diff --git a/tests/unit/c12_operator_orchestrator/test_post_landing_upload_orchestrator.py b/tests/unit/c12_operator_orchestrator/test_post_landing_upload_orchestrator.py new file mode 100644 index 0000000..b0b5016 --- /dev/null +++ b/tests/unit/c12_operator_orchestrator/test_post_landing_upload_orchestrator.py @@ -0,0 +1,387 @@ +"""AZ-329 PostLandingUploadOrchestrator unit tests. + +Covers AC-1..AC-8 (the orchestrator-level ACs); AC-9/AC-10 live in +:mod:`test_post_landing_upload_integration` against real FDR fixtures. +""" + +from __future__ import annotations + +import logging +from dataclasses import dataclass, field +from uuid import UUID, uuid4 + +import pytest + +from gps_denied_onboard.components.c12_operator_orchestrator._types import ( + FlightFooterRecord, + IngestStatusCut, + PerTileStatusCut, + PostLandingUploadRequest, + UploadBatchReportCut, + UploadOutcomeCut, + UploadRequestCut, +) +from gps_denied_onboard.components.c12_operator_orchestrator.config import ( + C12PostLandingConfig, +) +from gps_denied_onboard.components.c12_operator_orchestrator.errors import ( + FdrUnreadableError, + FlightStateNotConfirmedError, +) +from gps_denied_onboard.components.c12_operator_orchestrator.post_landing_upload import ( + PostLandingUploadOrchestrator, +) + +_API_KEY_LITERAL = "super-secret-token-123" + + +@dataclass +class _FakeFooterReader: + """Configurable :class:`FdrFooterReader` for the orchestrator unit tests.""" + + footer: FlightFooterRecord | None = None + raises: FdrUnreadableError | None = None + calls: list[UUID] = field(default_factory=list) + + def read_footer(self, flight_id: UUID) -> FlightFooterRecord | None: + self.calls.append(flight_id) + if self.raises is not None: + raise self.raises + return self.footer + + +@dataclass +class _FakeTileUploader: + """Configurable :class:`TileUploaderCut` recording each request.""" + + report: UploadBatchReportCut | None = None + calls: list[UploadRequestCut] = field(default_factory=list) + + def upload_pending_tiles(self, request: UploadRequestCut) -> UploadBatchReportCut: + self.calls.append(request) + assert self.report is not None, "test must wire .report before the call" + return self.report + + +def _make_request(flight_id: UUID | None = None) -> PostLandingUploadRequest: + return PostLandingUploadRequest( + flight_id=flight_id or uuid4(), + satellite_provider_url="https://parent.example/ingest", + api_key=_API_KEY_LITERAL, + batch_size=50, + ) + + +def _make_footer(*, flight_id: UUID, clean_shutdown: bool) -> FlightFooterRecord: + return FlightFooterRecord( + flight_id=flight_id, + flight_ended_at_iso="2026-05-13T12:00:00+00:00", + records_written=12345, + records_dropped_overrun=0 if clean_shutdown else 42, + bytes_written=987654 if not clean_shutdown else 654321, + rollover_count=0, + clean_shutdown=clean_shutdown, + ) + + +def _make_report() -> UploadBatchReportCut: + return UploadBatchReportCut( + batch_uuid=uuid4(), + per_tile_status=( + PerTileStatusCut(tile_id="tile-0", status=IngestStatusCut.ACCEPTED), + PerTileStatusCut(tile_id="tile-1", status=IngestStatusCut.ACCEPTED), + PerTileStatusCut( + tile_id="tile-2", + status=IngestStatusCut.REJECTED, + rejection_reason="invalid signature", + ), + ), + retry_count=0, + next_retry_at_s=None, + outcome=UploadOutcomeCut.PARTIAL, + public_key_fingerprint="ab" * 8, + ) + + +def _build_orchestrator( + *, + tmp_path, + footer: FlightFooterRecord | None = None, + reader_raises: FdrUnreadableError | None = None, + report: UploadBatchReportCut | None = None, + create_flight_dir: bool = True, + flight_id: UUID | None = None, +) -> tuple[ + PostLandingUploadOrchestrator, + _FakeFooterReader, + _FakeTileUploader, + UUID, + logging.Logger, +]: + fdr_root = tmp_path / "fdr" + fdr_root.mkdir() + actual_flight_id = flight_id or uuid4() + if create_flight_dir: + (fdr_root / str(actual_flight_id)).mkdir() + reader = _FakeFooterReader(footer=footer, raises=reader_raises) + uploader = _FakeTileUploader(report=report) + logger = logging.getLogger(f"test.c12.post_landing.{actual_flight_id}") + logger.setLevel(logging.DEBUG) + orchestrator = PostLandingUploadOrchestrator( + tile_uploader=uploader, + fdr_footer_reader=reader, + logger=logger, + config=C12PostLandingConfig(fdr_root=fdr_root), + ) + return orchestrator, reader, uploader, actual_flight_id, logger + + +# ---------------------------------------------------------------------- +# AC-1: clean-shutdown footer → upload invoked +# ---------------------------------------------------------------------- + + +def test_ac1_clean_shutdown_footer_triggers_upload(tmp_path, caplog) -> None: + # Arrange + flight_id = uuid4() + report = _make_report() + orchestrator, reader, uploader, _flight_id, _logger = _build_orchestrator( + tmp_path=tmp_path, + footer=_make_footer(flight_id=flight_id, clean_shutdown=True), + report=report, + flight_id=flight_id, + ) + request = _make_request(flight_id=flight_id) + + # Act + with caplog.at_level(logging.DEBUG): + returned = orchestrator.trigger_post_landing_upload(request) + + # Assert + assert returned is report + assert reader.calls == [flight_id] + assert len(uploader.calls) == 1 + inner = uploader.calls[0] + assert inner.flight_id == flight_id + assert inner.satellite_provider_url == request.satellite_provider_url + assert inner.batch_size == request.batch_size + confirmed = [ + r for r in caplog.records if getattr(r, "kind", None) == "c12.upload.confirmed_clean_shutdown" + ] + complete = [ + r for r in caplog.records if getattr(r, "kind", None) == "c12.upload.complete" + ] + assert len(confirmed) == 1 + assert len(complete) == 1 + + +# ---------------------------------------------------------------------- +# AC-2: footer absent → footer_missing +# ---------------------------------------------------------------------- + + +def test_ac2_footer_absent_refuses_with_footer_missing(tmp_path, caplog) -> None: + # Arrange + orchestrator, _reader, uploader, flight_id, _logger = _build_orchestrator( + tmp_path=tmp_path, footer=None + ) + request = _make_request(flight_id=flight_id) + + # Act / Assert + with caplog.at_level(logging.DEBUG): + with pytest.raises(FlightStateNotConfirmedError) as exc_info: + orchestrator.trigger_post_landing_upload(request) + + assert exc_info.value.not_confirmed_reason == "footer_missing" + assert exc_info.value.flight_id == str(flight_id) + assert exc_info.value.detail == "" + assert "No flight_footer record" in exc_info.value.remediation + assert uploader.calls == [] + refusal_logs = [ + r for r in caplog.records if getattr(r, "kind", None) == "c12.upload.refused.footer_missing" + ] + assert len(refusal_logs) == 1 + + +# ---------------------------------------------------------------------- +# AC-3: footer with clean_shutdown=False → unclean_shutdown +# ---------------------------------------------------------------------- + + +def test_ac3_unclean_shutdown_refuses_with_counters_in_detail(tmp_path, caplog) -> None: + # Arrange + flight_id = uuid4() + footer = _make_footer(flight_id=flight_id, clean_shutdown=False) + orchestrator, _reader, uploader, _flight_id, _logger = _build_orchestrator( + tmp_path=tmp_path, + footer=footer, + flight_id=flight_id, + ) + request = _make_request(flight_id=flight_id) + + # Act / Assert + with caplog.at_level(logging.DEBUG): + with pytest.raises(FlightStateNotConfirmedError) as exc_info: + orchestrator.trigger_post_landing_upload(request) + + err = exc_info.value + assert err.not_confirmed_reason == "unclean_shutdown" + assert f"records_dropped_overrun={footer.records_dropped_overrun}" in err.detail + assert f"bytes_written={footer.bytes_written}" in err.detail + assert uploader.calls == [] + refusal_logs = [ + r for r in caplog.records if getattr(r, "kind", None) == "c12.upload.refused.unclean_shutdown" + ] + assert len(refusal_logs) == 1 + refusal = refusal_logs[0] + kv = getattr(refusal, "kv", {}) + assert kv["records_written"] == footer.records_written + assert kv["records_dropped_overrun"] == footer.records_dropped_overrun + assert kv["bytes_written"] == footer.bytes_written + assert kv["rollover_count"] == footer.rollover_count + + +# ---------------------------------------------------------------------- +# AC-4: // does not exist → flight_id_not_found +# ---------------------------------------------------------------------- + + +def test_ac4_flight_dir_missing_refuses_with_flight_id_not_found(tmp_path, caplog) -> None: + # Arrange + orchestrator, reader, uploader, flight_id, _logger = _build_orchestrator( + tmp_path=tmp_path, + create_flight_dir=False, + ) + request = _make_request(flight_id=flight_id) + + # Act / Assert + with caplog.at_level(logging.DEBUG): + with pytest.raises(FlightStateNotConfirmedError) as exc_info: + orchestrator.trigger_post_landing_upload(request) + + assert exc_info.value.not_confirmed_reason == "flight_id_not_found" + assert reader.calls == [] + assert uploader.calls == [] + refusal_logs = [ + r for r in caplog.records if getattr(r, "kind", None) == "c12.upload.refused.flight_id_not_found" + ] + assert len(refusal_logs) == 1 + + +# ---------------------------------------------------------------------- +# AC-5: reader raises FdrUnreadableError → fdr_unreadable +# ---------------------------------------------------------------------- + + +def test_ac5_fdr_unreadable_refuses_with_inner_repr(tmp_path, caplog) -> None: + # Arrange + inner_exc = FdrUnreadableError("OSError('input/output error') at segment-0001.fdr") + orchestrator, _reader, uploader, flight_id, _logger = _build_orchestrator( + tmp_path=tmp_path, + reader_raises=inner_exc, + ) + request = _make_request(flight_id=flight_id) + + # Act / Assert + with caplog.at_level(logging.DEBUG): + with pytest.raises(FlightStateNotConfirmedError) as exc_info: + orchestrator.trigger_post_landing_upload(request) + + err = exc_info.value + assert err.not_confirmed_reason == "fdr_unreadable" + assert "OSError" in err.detail + assert err.__cause__ is inner_exc + assert uploader.calls == [] + refusal_logs = [ + r for r in caplog.records if getattr(r, "kind", None) == "c12.upload.refused.fdr_unreadable" + ] + assert len(refusal_logs) == 1 + kv = getattr(refusal_logs[0], "kv", {}) + assert "OSError" in kv["fdr_unreadable_repr"] + + +# ---------------------------------------------------------------------- +# AC-7: passthrough — returns the exact UploadBatchReportCut instance +# ---------------------------------------------------------------------- + + +def test_ac7_upload_batch_report_is_passed_through_unchanged(tmp_path) -> None: + # Arrange + flight_id = uuid4() + report = _make_report() + orchestrator, _reader, _uploader, _flight_id, _logger = _build_orchestrator( + tmp_path=tmp_path, + footer=_make_footer(flight_id=flight_id, clean_shutdown=True), + report=report, + flight_id=flight_id, + ) + + # Act + returned = orchestrator.trigger_post_landing_upload(_make_request(flight_id=flight_id)) + + # Assert + assert returned is report + assert returned.batch_uuid == report.batch_uuid + assert returned.per_tile_status == report.per_tile_status + assert returned.outcome is report.outcome + assert returned.public_key_fingerprint == report.public_key_fingerprint + + +# ---------------------------------------------------------------------- +# AC-8: api_key never appears in any log record across every code path +# ---------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "scenario", + ["success", "footer_missing", "unclean_shutdown", "flight_id_not_found", "fdr_unreadable"], +) +def test_ac8_api_key_never_appears_in_logs(tmp_path, caplog, scenario) -> None: + # Arrange + flight_id = uuid4() + if scenario == "success": + footer = _make_footer(flight_id=flight_id, clean_shutdown=True) + orchestrator, _r, _u, _fid, _l = _build_orchestrator( + tmp_path=tmp_path, footer=footer, report=_make_report(), flight_id=flight_id + ) + elif scenario == "footer_missing": + orchestrator, _r, _u, _fid, _l = _build_orchestrator( + tmp_path=tmp_path, footer=None, flight_id=flight_id + ) + elif scenario == "unclean_shutdown": + footer = _make_footer(flight_id=flight_id, clean_shutdown=False) + orchestrator, _r, _u, _fid, _l = _build_orchestrator( + tmp_path=tmp_path, footer=footer, flight_id=flight_id + ) + elif scenario == "flight_id_not_found": + orchestrator, _r, _u, _fid, _l = _build_orchestrator( + tmp_path=tmp_path, create_flight_dir=False, flight_id=flight_id + ) + else: # fdr_unreadable + orchestrator, _r, _u, _fid, _l = _build_orchestrator( + tmp_path=tmp_path, + reader_raises=FdrUnreadableError("parse failure with token glimpse"), + flight_id=flight_id, + ) + request = _make_request(flight_id=flight_id) + + # Act + with caplog.at_level(logging.DEBUG): + try: + orchestrator.trigger_post_landing_upload(request) + except FlightStateNotConfirmedError: + pass + + # Assert — the literal api_key value MUST NOT appear in any log record + for record in caplog.records: + for field_name in ("msg", "message"): + value = getattr(record, field_name, None) + assert value is None or _API_KEY_LITERAL not in str(value), ( + f"api_key leaked into record.{field_name}: {value!r}" + ) + kv = getattr(record, "kv", None) + if kv is not None: + for k, v in kv.items(): + assert _API_KEY_LITERAL not in str(v), ( + f"api_key leaked into record.kv[{k!r}]: {v!r}" + ) diff --git a/tests/unit/c12_operator_tooling/test_remote_c10_invoker.py b/tests/unit/c12_operator_orchestrator/test_remote_c10_invoker.py similarity index 96% rename from tests/unit/c12_operator_tooling/test_remote_c10_invoker.py rename to tests/unit/c12_operator_orchestrator/test_remote_c10_invoker.py index 23e2b6c..f71bf02 100644 --- a/tests/unit/c12_operator_tooling/test_remote_c10_invoker.py +++ b/tests/unit/c12_operator_orchestrator/test_remote_c10_invoker.py @@ -11,17 +11,17 @@ from uuid import UUID import pytest from gps_denied_onboard._types.geo import BoundingBox, LatLonAlt -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( BuildReportParseError, RemoteBuildOutcome, RemoteCacheProvisionerInvoker, SectorClassification, ) -from gps_denied_onboard.components.c12_operator_tooling.remote_c10_invoker import ( +from gps_denied_onboard.components.c12_operator_orchestrator.remote_c10_invoker import ( REDACTED_PLACEHOLDER, RemoteBuildRequest, ) -from gps_denied_onboard.components.c12_operator_tooling.ssh_session import ( +from gps_denied_onboard.components.c12_operator_orchestrator.ssh_session import ( RemoteCommandResult, SshSession, ) diff --git a/tests/unit/c12_operator_tooling/test_sector_classification_store.py b/tests/unit/c12_operator_orchestrator/test_sector_classification_store.py similarity index 97% rename from tests/unit/c12_operator_tooling/test_sector_classification_store.py rename to tests/unit/c12_operator_orchestrator/test_sector_classification_store.py index cf2eecf..bb6f533 100644 --- a/tests/unit/c12_operator_tooling/test_sector_classification_store.py +++ b/tests/unit/c12_operator_orchestrator/test_sector_classification_store.py @@ -9,7 +9,7 @@ from pathlib import Path import pytest -from gps_denied_onboard.components.c12_operator_tooling import ( +from gps_denied_onboard.components.c12_operator_orchestrator import ( SectorClassification, SectorClassificationStore, ) @@ -88,7 +88,7 @@ class TestAtomicWriteUnderCrash: raise OSError("simulated kill mid-replace") monkeypatch.setattr( - "gps_denied_onboard.components.c12_operator_tooling." + "gps_denied_onboard.components.c12_operator_orchestrator." "sector_classification_store.os.replace", _raise_replace, ) diff --git a/tests/unit/c12_operator_tooling/test_smoke.py b/tests/unit/c12_operator_orchestrator/test_smoke.py similarity index 77% rename from tests/unit/c12_operator_tooling/test_smoke.py rename to tests/unit/c12_operator_orchestrator/test_smoke.py index 3d0027b..06dd415 100644 --- a/tests/unit/c12_operator_tooling/test_smoke.py +++ b/tests/unit/c12_operator_orchestrator/test_smoke.py @@ -3,7 +3,7 @@ def test_interface_importable() -> None: # Assert - from gps_denied_onboard.components.c12_operator_tooling import ( + from gps_denied_onboard.components.c12_operator_orchestrator import ( CacheBuildWorkflow, OperatorReLocService, ) diff --git a/tests/unit/test_ac1_scaffold_layout.py b/tests/unit/test_ac1_scaffold_layout.py index f899120..5629a31 100644 --- a/tests/unit/test_ac1_scaffold_layout.py +++ b/tests/unit/test_ac1_scaffold_layout.py @@ -33,7 +33,7 @@ REQUIRED_PATHS: tuple[str, ...] = ( ".gitignore", "README.md", "docker/companion-tier1.Dockerfile", - "docker/operator-tooling.Dockerfile", + "docker/operator-orchestrator.Dockerfile", "docker/mock-suite-sat-service.Dockerfile", "docker-compose.yml", "docker-compose.test.yml", @@ -71,7 +71,7 @@ COMPONENT_DIRS: tuple[str, ...] = ( "c8_fc_adapter", "c10_provisioning", "c11_tile_manager", - "c12_operator_tooling", + "c12_operator_orchestrator", "c13_fdr", ) diff --git a/tests/unit/test_ac3_compose_files.py b/tests/unit/test_ac3_compose_files.py index a1c9904..55c6ef5 100644 --- a/tests/unit/test_ac3_compose_files.py +++ b/tests/unit/test_ac3_compose_files.py @@ -35,7 +35,7 @@ def test_compose_yml_declares_required_services() -> None: data = yaml.safe_load((REPO_ROOT / "docker-compose.yml").read_text()) services = data["services"] # Assert - for required in ("companion", "operator-tooling", "mock-sat", "db"): + for required in ("companion", "operator-orchestrator", "mock-sat", "db"): assert required in services, f"docker-compose.yml missing service: {required}" diff --git a/tests/unit/test_az272_fdr_record_schema.py b/tests/unit/test_az272_fdr_record_schema.py index f4c1f56..428d261 100644 --- a/tests/unit/test_az272_fdr_record_schema.py +++ b/tests/unit/test_az272_fdr_record_schema.py @@ -255,6 +255,19 @@ def _kind_payload(kind: str) -> dict[str, object]: "attempts": 5, "last_rejection_reason": "invalid signature", } + if kind == "c12.reloc.requested": + return { + "hint": { + "lat_deg": 49.99876543, + "lon_deg": 36.12345678, + "alt_m": 1234.5, + "confidence_radius_m": 50.0, + "reason": "lost track at WP3", + }, + "outcome": "sent", + "failure_reason": None, + "ts_monotonic_ns": 1_234_567_890_123, + } raise AssertionError(f"unhandled kind in fixture: {kind!r}")