diff --git a/_docs/02_document/FINAL_report.md b/_docs/02_document/FINAL_report.md index 6df929a..eec62d2 100644 --- a/_docs/02_document/FINAL_report.md +++ b/_docs/02_document/FINAL_report.md @@ -36,8 +36,8 @@ See `architecture.md` for the full ADR set (ADR-001..ADR-009), 12 architectural | 09 | C7 Inference Runtime | TensorRT 10.3 engines (Polygraphy / trtexec / IBuilderConfig hybrid); ORT+TRT EP fallback; PyTorch FP16 baseline | E-BOOT, E-CC-CONF, E-CC-FDR-CLIENT | AZ-249 | | 10 | C8 FC + GCS Adapter | `pymavlink` `GPS_INPUT` for ArduPilot (signed) + `MSP2_SENSOR_GPS` for iNav (unsigned, accepted residual risk); honest 6×6 → 2×2 covariance projection; GCS 1–2 Hz downsampled telemetry | C5, E-CC-CONF, E-CC-LOG | AZ-261 | | 11 | C10 Pre-flight Cache Provisioning | Builds model-derived cache (descriptors, engines, manifest, content hashes); F2 takeoff verifier; does NOT touch `satellite-provider` (network I/O lives in C11) | C6, C7, E-CC-LOG | AZ-252 | -| 12 | C11 Tile Manager | Operator-side `TileDownloader` (pre-flight) + `TileUploader` (post-landing, gated `flight_state == ON_GROUND`); excluded from airborne image | C6, E-CC-CONF, E-CC-LOG | AZ-251 | -| 13 | C12 Operator Pre-flight Orchestrator | CLI subcommands (`download`, `build-cache`, `upload-pending`, `reloc-confirm`); sector classification UI hook; FDR retrieval helpers | C10, C11, E-CC-LOG | AZ-253 | +| 12 | C11 Tile Manager | Operator-side `TileDownloader` (pre-flight) + `TileUploader` (post-landing — no internal flight-state gate after Batch 44; gating lives in C12); excluded from airborne image | C6, E-CC-CONF, E-CC-LOG | AZ-251 | +| 13 | C12 Operator Pre-flight Orchestrator | CLI subcommands (`download`, `build-cache`, `upload-pending`, `reloc-confirm`); `PostLandingUploadOrchestrator` (gates on `flight_footer.clean_shutdown`); `OperatorReLocService` (AC-3.4 hint via `OperatorCommandTransport`); sector classification UI hook; FDR retrieval helpers | C10, C11, E-CC-LOG | AZ-253 | | 14 | C13 Flight Data Recorder | Per-flight ≤64 GB NVM ring (estimates + IMU + emitted MAVLink + health + mid-flight tiles + ≤0.1 Hz failed-tile thumbnails); raw nav/AI-cam frames excluded | E-BOOT, E-CC-LOG, E-CC-CONF, E-CC-FDR-CLIENT | AZ-248 | **Cross-cutting epics** (not components, but shared concerns): E-BOOT (AZ-244), E-CC-LOG (AZ-245), E-CC-CONF (AZ-246), E-CC-FDR-CLIENT (AZ-247). diff --git a/_docs/02_document/architecture.md b/_docs/02_document/architecture.md index b62b5e9..422bec6 100644 --- a/_docs/02_document/architecture.md +++ b/_docs/02_document/architecture.md @@ -22,8 +22,8 @@ The system is a **Jetson Orin Nano Super-hosted onboard companion** that deliver - **C7 — On-Jetson inference runtime**: TensorRT 10.3 engines (Polygraphy / trtexec / IBuilderConfig hybrid orchestration), JetPack 6.2, SM 87; ONNX Runtime + TRT EP fallback; pure PyTorch FP16 baseline. - **C8 — Flight-Controller adapter**: `pymavlink` `GPS_INPUT` for ArduPilot Plane (MAVLink 2.0 message signing on the companion ↔ AP wired channel, D-C8-9 = (d)) and `YAMSPy` / INAV-Toolkit `MSP2_SENSOR_GPS` for iNav (signing-gap accepted residual risk). - **C10 — Pre-flight cache provisioning**: builds the **model-derived** cache artifacts (descriptor generation, engine compilation, manifest + content-hash) on top of an already-populated tile store; F2 takeoff verifier (D-C10-1, D-C10-3, D-C10-6, D-C10-7). C10 does NOT touch `satellite-provider` — tile network I/O lives in C11. -- **C11 — Tile Manager** (operator-side, distinct binary/image, ADR-004 process-isolated): owns operator-side network I/O against `satellite-provider` in **both directions**. `TileDownloader` interface fetches tiles into C6 during F1 (TLS + service-internal API key); `TileUploader` interface, gated on `flight_state == ON_GROUND`, pushes mid-flight tiles to `satellite-provider`'s ingest endpoint (D-PROJ-2 contract; not yet implemented service-side). The component bundles both interfaces because they share auth, HTTP client, deployment unit, and the airborne-exclusion property. -- **C12 — Operator pre-flight tooling** (Plan-phase carryforward, deferred from research): cache provisioning UI, sector classification (active-conflict vs stable rear), freshness pipeline workflow. +- **C11 — Tile Manager** (operator-side, distinct binary/image, ADR-004 process-isolated): owns operator-side network I/O against `satellite-provider` in **both directions**. `TileDownloader` interface fetches tiles into C6 during F1 (TLS + service-internal API key); `TileUploader` interface pushes mid-flight tiles to `satellite-provider`'s ingest endpoint (D-PROJ-2 contract; not yet implemented service-side). C11 carries **no flight-state gating** of its own (Batch 44 SRP refactor) — the post-landing safety check lives in C12 (single source of truth). The component bundles both interfaces because they share auth, HTTP client, deployment unit, and the airborne-exclusion property. +- **C12 — Operator Pre-flight Orchestrator** (operator-side, same image as C11): orchestrates the operator-side workflows that C11 implements. Hosts the pre-flight cache provisioning UI, sector classification (active-conflict vs stable rear), the `Flight` resolver (`FlightsApiClient` → bbox + takeoff origin), the **post-landing upload orchestrator** (gates `TileUploader` on the `flight_footer` FDR record's `clean_shutdown` field — AZ-329), and the **operator re-localization service** (AC-3.4 visual-loss hint dispatched to the airborne companion over the GCS link via the `OperatorCommandTransport` Protocol; concrete pymavlink-backed impl is an E-C8 deliverable — AZ-330). The C12 ⇄ C11 boundary is a thin Protocol cut (`TileUploaderCut`) so C12 does not import C11 directly (AZ-507). - **C13 — Flight Data Recorder (FDR)**: per-flight ≤64 GB NVM record of estimates + IMU traces + emitted MAVLink + system health + mid-flight tiles + ≤0.1 Hz failed-tile thumbnails; raw nav/AI-cam frames excluded (AC-8.5, AC-NEW-3). - **External: `satellite-provider`** (parent-suite .NET 8 service): tile producer pre-flight; tile sink post-landing (D-PROJ-2). Treated as a planned external dependency on the upload + voting paths. @@ -32,7 +32,7 @@ The system is a **Jetson Orin Nano Super-hosted onboard companion** that deliver 1. **Camera-specific math enters only via a `Camera calibration artifact` JSON** (intrinsics + distortion + body-to-camera extrinsics + acquisition method `factory_sheet | checkerboard_refined | hybrid`). No hard-coded camera math anywhere; test fixtures (`adti26`) and production deployments (`adti20`) load different artifacts on the same code path. 2. **VioStrategy is selected at startup via config; not hot-swappable mid-flight.** 3. **Build-time exclusion of unused `Strategy` implementations.** A given binary links only the implementations it actually uses at runtime. The default deployment binary links the production-default strategies (e.g. OKVIS2 on C1) plus the engine-rule-mandatory simple-baseline (KltRansac on C1); the IT-12 comparative-study binary links all C1 implementations side-by-side. The mechanism is per-component CMake `BUILD_*` flags (`BUILD_VINS_MONO`, `BUILD_SALAD`, …) plus the per-binary composition root choosing among the linked implementations at startup. **Justification is technical** — binary size on the 8 GB shared Jetson, boot/load time inside the AC-NEW-1 30 s budget, deployed dependency / attack surface, and accidental-selection risk reduction (a binary with only OKVIS2 + KltRansac linked cannot be misconfigured into running VINS-Mono). **Component licenses do not drive this decision** — see ADR-002. CI emits both the deployment binary and the research binary on every PR. -4. **In-air network I/O against `satellite-provider` is forbidden — in BOTH directions.** Enforced primarily by **process-level isolation** — the Tile Manager (C11), which carries both the `TileDownloader` and the `TileUploader` interfaces, is not loaded in the airborne companion image. Software guard on `flight_state == ON_GROUND` (upload) is a defense-in-depth check, not the primary control. The companion is read-only against C6 in flight; both pre-flight tile fetching and post-landing tile upload happen on the operator workstation. +4. **In-air network I/O against `satellite-provider` is forbidden — in BOTH directions.** Enforced primarily by **process-level isolation** — the Tile Manager (C11), which carries both the `TileDownloader` and the `TileUploader` interfaces, is not loaded in the airborne companion image. The defense-in-depth software guard is a C12-side `flight_footer.clean_shutdown == True` check (read by `PostLandingUploadOrchestrator` from the post-flight FDR via `FdrFooterReader`); C11 itself no longer gates (Batch 44 SRP refactor). The companion is read-only against C6 in flight; both pre-flight tile fetching and post-landing tile upload happen on the operator workstation. 5. **All persistent imagery is in `satellite-provider`'s on-disk tile format** (`./tiles/{zoomLevel}/{x}/{y}.jpg` + matching metadata) so post-landing upload is byte-identical. No raw frames on disk except the AC-8.5 forensic ≤0.1 Hz failed-tile thumbnail log inside FDR. 6. **Honest 6×6 posterior covariance via GTSAM `Marginals`** is the safety floor for AC-NEW-4 and AC-NEW-7. Under-reported `horiz_accuracy` is a defect, not a tuning knob. 7. **MAVLink 2.0 message signing on the companion ↔ ArduPilot wired channel**, with per-flight key rotation (D-C8-9 = (d)). iNav has no signing equivalent — accepted residual risk, Plan-phase carryforward proposes an iNav firmware feature request. @@ -66,7 +66,7 @@ The system is a **Jetson Orin Nano Super-hosted onboard companion** that deliver | All onboard pose-estimation logic (C1–C8, C13) | Parent-suite `satellite-provider` (.NET 8 REST microservice) | | Pre-flight cache artifact build (C10 — engines + descriptors + manifest) | Parent-suite `flights` REST service (.NET 8; owns the `Flight` + `Waypoint` DTOs) | | Operator-side Tile Manager (C11 — pre-flight download + post-landing upload) | Parent-suite Mission Planner UI (`suite/ui` — where operators plan the route) | -| Operator pre-flight tooling (C12) | GCS (QGroundControl) | +| Operator Pre-flight Orchestrator (C12) | GCS (QGroundControl) | | FDR writer (C13) | Nav camera hardware (`adti20`); AI-camera hardware | | Camera calibration artifact format + loader | UAV airframe / FC IMU / sensors | | | Operator's workstation OS / authentication | @@ -135,7 +135,7 @@ The system is a **Jetson Orin Nano Super-hosted onboard companion** that deliver | `staging-tier1` | CI runs that don't require Jetson hardware | GitHub-hosted runner (x86_64); Docker | | `staging-tier2` | CI runs that require Jetson (AC-bound jobs only) | Self-hosted Jetson runner; bare JetPack (no Docker) | | `production` | Deployed companion image on a UAV | Jetson Orin Nano Super (pinned); bare JetPack; no inbound network listening (defense-in-depth, NFT-SEC-05) | -| `production-operator-workstation` | Pre-flight tile download + cache artifact build (C10) + post-landing tile upload (C11) + FDR retrieval | Operator's Linux workstation; Docker for `satellite-provider` mirror | +| `production-operator-workstation` | Operator-side workflows orchestrated by C12: pre-flight tile download (C11 `TileDownloader`), cache artifact build (C10), post-landing tile upload (C12 `PostLandingUploadOrchestrator` → C11 `TileUploader`), AC-3.4 re-loc hint dispatch (C12 `OperatorReLocService`), FDR retrieval | Operator's Linux workstation; Docker for `satellite-provider` mirror | **Infrastructure**: @@ -200,7 +200,10 @@ source repo | `Tile` | JPEG body + center lat/lon + zoomLevel + tile_size_meters/pixels + capture_timestamp + source + freshness flag + (mid-flight only) quality_metadata | C6 | | `TileQualityMetadata` | `estimator_label`, 2×2 covariance sub-matrix, `last_anchor_age_ms`, MRE, IMU bias norm — sufficient for D-PROJ-2 voting | C6 (write side from C5/C4 outputs) | | `EmittedExternalPosition` | WGS84 + honest `horiz_accuracy` + per-FC encoding (MAVLink `GPS_INPUT` for AP, MSP2 `MSP2_SENSOR_GPS` for iNav) | C8 | -| `FlightStateSignal` | `IN_AIR | ON_GROUND` boolean derived from FC `MAV_STATE` | C8 inbound side; published to C11 only post-landing | +| `FlightStateSignal` | `IN_AIR | ON_GROUND` boolean derived from FC `MAV_STATE` | C8 inbound side; used internally by C8/C5 for live-flight state machines. **Not** consumed by C11/C12 — post-landing gating reads the C13-written `flight_footer` FDR record instead (Batch 44 SRP refactor) | +| `FlightFooterRecord` | `{flight_id, clean_shutdown, total_records, segment_count, …}` — single FDR record written by C13 on clean shutdown | C13 (writer) → C12 `PostLandingUploadOrchestrator` (reader, via `FdrFooterReader`) | +| `PostLandingUploadRequest` | `{flight_id, satellite_provider_url, api_key, batch_size}` | C12 CLI → C12 `PostLandingUploadOrchestrator` | +| `ReLocHint` | Operator-supplied position hint for AC-3.4 visual-loss re-localization: `{approximate_position_wgs84: LatLonAlt, confidence_radius_m, reason}`; validated at construction (lat ∈ [-90,90]; lon ∈ (-180,180]; radius > 0; reason non-empty); emitted to airborne companion via `OperatorCommandTransport` Protocol (E-C8 concrete) | Operator CLI → C12 `OperatorReLocService` → (GCS link) airborne companion | | `FdrRecord` | Estimates + IMU traces + emitted MAVLink + system health + tiles + thumbnails (≤ 64 GB / flight) | C13 | | `Manifest` | Hash of (model + calibration + corpus + sector classification + takeoff origin) for D-C10-1 idempotence | C10 | | `EngineCacheEntry` | TRT engine + INT8 calibration cache keyed by SM/JP/TRT/precision tuple (D-C10-7) | C10, C7 | @@ -227,7 +230,8 @@ source repo - Mid-flight tile gen: `NavCameraFrame` + `PoseEstimate` → orthorectify → dedup → write to local C6 (no upload). - GCS telemetry: C5 → C8 → 1–2 Hz downsampled summary to QGroundControl. - FDR: every emitted/received stream → C13 ring with per-flight ≤ 64 GB cap. -- Post-landing: operator triggers C11 `TileUploader` → reads C6 → uploads to `satellite-provider` ingest endpoint (D-PROJ-2 contract). +- Post-landing: operator triggers C12 `PostLandingUploadOrchestrator` → reads `flight_footer` from FDR via `FdrFooterReader` → on `clean_shutdown == True` invokes C11 `TileUploader` (via `TileUploaderCut` Protocol) → reads C6 → uploads to `satellite-provider` ingest endpoint (D-PROJ-2 contract). Refusal modes (`footer_missing`, `unclean_shutdown`, `flight_id_not_found`, `fdr_unreadable`) raise `FlightStateNotConfirmedError` with operator-actionable remediation text and a distinct CLI exit code per mode. +- Operator re-loc (AC-3.4 visual-loss path): operator submits `ReLocHint` via the `reloc-confirm` CLI → C12 `OperatorReLocService` validates the DTO → forwards to airborne companion via `OperatorCommandTransport` (E-C8 concrete) → records `c12.reloc.requested` FDR record (`outcome ∈ {sent, failed}`). Live log redaction (lat/lon rounded to 5 decimals; `reason` truncated to 200 chars); FDR record persists the full hint un-redacted for post-flight forensics. --- @@ -326,7 +330,7 @@ The onboard side of D-PROJ-2 is fully specified in `_docs/_process_leftovers/202 **Authorization**: - **Onboard runtime**: a single principal (the runtime process); no in-process privilege boundaries. The Tile Manager (C11) runs as a different principal on the operator workstation, holding the only credentials that reach `satellite-provider` (TLS API key for download; per-flight onboard signing key for post-landing upload). The airborne image does not contain the C11 binary at all. -- **GCS**: operator commands (`AC-6.2`) are best-effort hints; the operator cannot promote a pose, override covariance, or reach the `satellite-provider` write path. Operator re-loc requests trigger the satellite re-localization flow (F6) but do not bypass any safety gate. +- **GCS**: operator commands (`AC-6.2`) are best-effort hints; the operator cannot promote a pose, override covariance, or reach the `satellite-provider` write path. Operator re-loc requests (C12 `OperatorReLocService` → `OperatorCommandTransport` over the GCS link) trigger the satellite re-localization flow (F6) but do not bypass any safety gate — the airborne pipeline still validates the hint against the visual/satellite consistency check before promoting any pose. **Data protection**: @@ -413,7 +417,9 @@ This decision is made on **technical grounds only**. Component licenses (BSD/Apa **Context**: AC-8.4 forbids in-air outbound writes to `satellite-provider` for drone-security reasons. The companion is also read-only against `satellite-provider` while airborne — there is no operational reason to fetch tiles in flight either, since the pre-flight cache is the contract. A software guard checking `flight_state == ON_GROUND` can be bypassed by code injection if the network I/O code path is ever loaded. -**Decision**: The Tile Manager (C11) is a **separate binary / image** that runs only on the operator's workstation; the airborne companion image does not contain the C11 binary at all — neither the `TileDownloader` (pre-flight) nor the `TileUploader` (post-landing) code paths can be reached from the airborne process. The `flight_state == ON_GROUND` software guard inside the `TileUploader` remains as defense-in-depth for the upload direction. The local mid-flight tile format is byte-identical to `satellite-provider`'s on-disk layout so no transformation is needed at upload time. +**Decision**: The Tile Manager (C11) is a **separate binary / image** that runs only on the operator's workstation; the airborne companion image does not contain the C11 binary at all — neither the `TileDownloader` (pre-flight) nor the `TileUploader` (post-landing) code paths can be reached from the airborne process. The defense-in-depth software guard is owned by **C12's `PostLandingUploadOrchestrator`**, which reads the `flight_footer` FDR record's `clean_shutdown` field before invoking C11's `TileUploader` (Batch 44 SRP refactor — the gate's single source of truth is the FDR footer C13 writes only on clean shutdown; C11 itself no longer gates). The local mid-flight tile format is byte-identical to `satellite-provider`'s on-disk layout so no transformation is needed at upload time. + +**Why the gate moved to C12 (Batch 44)**: An earlier iteration placed the gate inside C11's `TileUploader` (consuming a live `FlightStateSignal` from C8). That duplicated the safety invariant on both sides of the C11/C12 boundary and coupled C11 to C8 just for the post-landing check. The current design (a) consolidates ownership on the operator-side workflow head (C12) — single responsibility per component, single source of truth for "vehicle is fully stopped" (= C13's footer write decision), and (b) collapses an arbitrary 30-second hold-down heuristic to an exact boolean (`clean_shutdown`). The `TileUploader` Protocol contract is frozen at v2.0.0 with the gate parameters removed; AZ-317 is superseded. **Enforcement gates (per R02 risk register)**: 1. **CI SBOM diff**: the build pipeline fails the airborne `production-binary` artifact if any symbol from `c11_tilemanager/` (or any module that transitively imports `c11_tilemanager`) appears in the linked image. This is an extension of the per-implementation SBOM enforcement already in ADR-002. diff --git a/_docs/02_document/components/10_c8_fc_adapter/description.md b/_docs/02_document/components/10_c8_fc_adapter/description.md index 5935498..179ab7c 100644 --- a/_docs/02_document/components/10_c8_fc_adapter/description.md +++ b/_docs/02_document/components/10_c8_fc_adapter/description.md @@ -14,7 +14,7 @@ - C5 StateEstimator (consumes `ImuWindow`, `AttitudeWindow`, `GpsHealth`, `FlightStateSignal`). - C1 VIO (consumes `ImuWindow`). - C13 FDR (consumes raw inbound + emitted outbound MAVLink/MSP2 streams; signing key rotation events; spoof-promotion events). -- C11 `TileUploader` (consumes `FlightStateSignal == ON_GROUND` confirmation; runs on a different process / image, so the signal flows out-of-band via the FDR or a small bus the operator tool subscribes to post-flight). The C11 `TileDownloader` does NOT depend on `FlightStateSignal` — it runs pre-flight when the companion is plugged into the operator workstation. +- C11 `TileUploader` does **NOT** depend on `FlightStateSignal` after Batch 44 (SRP refactor — the post-landing safety gate now lives in C12's `PostLandingUploadOrchestrator`, which reads the `flight_footer` FDR record's `clean_shutdown` field, not the live `FlightStateSignal`). The C11 `TileDownloader` likewise does not depend on `FlightStateSignal` — it runs pre-flight when the companion is plugged into the operator workstation. ## 2. Internal Interfaces diff --git a/_docs/02_document/contracts/c6_tile_cache/tile_metadata_store.md b/_docs/02_document/contracts/c6_tile_cache/tile_metadata_store.md index 152ee47..e9d27d0 100644 --- a/_docs/02_document/contracts/c6_tile_cache/tile_metadata_store.md +++ b/_docs/02_document/contracts/c6_tile_cache/tile_metadata_store.md @@ -6,7 +6,7 @@ - AZ-TBD-c6-postgres-filesystem-store (implements) - AZ-TBD-c6-freshness-gate (insert hook + sector classification reader) - AZ-TBD-c6-cache-budget-eviction (LRU candidate enumeration + delete coordination) -- TBD at decompose time: E-C10 (AZ-252 — manifest + provisioning), E-C11 (AZ-251 — both `TileDownloader` insert and `TileUploader` reader queries), E-C12 (AZ-253 — operator pre-flight tooling) +- TBD at decompose time: E-C10 (AZ-252 — manifest + provisioning), E-C11 (AZ-251 — both `TileDownloader` insert and `TileUploader` reader queries), E-C12 (AZ-253 — operator pre-flight orchestrator) **Version**: 1.3.0 **Status**: draft **Last Updated**: 2026-05-13 diff --git a/_docs/02_document/data_model.md b/_docs/02_document/data_model.md index ef7ef8e..694459c 100644 --- a/_docs/02_document/data_model.md +++ b/_docs/02_document/data_model.md @@ -557,7 +557,10 @@ The following DTOs flow through the per-frame pipeline in memory and are **NOT** | `MatchResult` | C3 / C3.5 | C4 | Never | | `PoseEstimate` | C4 → C5 | C8, C13 | FDR `EmittedExternalPosition` records (post-emission); also feeds `tiles.quality_metadata` for in-flight orthorectified tiles | | `EmittedExternalPosition` | C8 | FC, FDR | FDR record | -| `FlightStateSignal` | C8 inbound side | flight-state guard, FDR | FDR `ComponentLifecycleEvent` on transition | +| `FlightStateSignal` | C8 inbound side | C5/C8 internal state machines, FDR | FDR `ComponentLifecycleEvent` on transition. **Not** consumed by C11/C12 post-landing — C12 reads the `flight_footer` FDR record's `clean_shutdown` field via `FdrFooterReader` instead (Batch 44 SRP refactor) | +| `FlightFooterRecord` | C13 (writer, on clean shutdown only) | C12 `PostLandingUploadOrchestrator` (reader, via `FdrFooterReader`) | FDR `flight_footer` record kind — `{flight_id, clean_shutdown, total_records, segment_count, …}` | +| `PostLandingUploadRequest` | C12 CLI (`upload-pending` subcommand) | C12 `PostLandingUploadOrchestrator` | Never persisted — composed inline from CLI args | +| `ReLocHint` | C12 CLI (`reloc-confirm` subcommand) | C12 `OperatorReLocService` → `OperatorCommandTransport` (E-C8 concrete) → airborne companion | FDR `c12.reloc.requested` record (full hint un-redacted; `outcome ∈ {sent, failed}`) | | `CameraCalibration` (loaded once) | calibration loader | C1, C3, C4 | NOT in PostgreSQL — see § 2.6 | --- diff --git a/_docs/02_document/epics.md b/_docs/02_document/epics.md index 3286526..2e3e275 100644 --- a/_docs/02_document/epics.md +++ b/_docs/02_document/epics.md @@ -857,9 +857,9 @@ Sole operator-side network I/O against `satellite-provider`, both directions. St ### Scope -**In scope**: `TileDownloader.fetch` (download → freshness gate → write to C6), `TileUploader.upload_pending` (read C6 pending → sign → POST → mark uploaded), per-flight ephemeral signing key, idempotent retry on partial-success batches, `flight_state == ON_GROUND` gate (defense-in-depth atop ADR-004). +**In scope**: `TileDownloader.fetch` (download → freshness gate → write to C6), `TileUploader.upload_pending` (read C6 pending → sign → POST → mark uploaded), per-flight ephemeral signing key, idempotent retry on partial-success batches. -**Out of scope**: any airborne code; cache artifact build (E-C10); orchestration (E-C12). +**Out of scope**: any airborne code; cache artifact build (E-C10); orchestration (E-C12 — including the post-landing safety gate, which moved to C12 in Batch 44). ### Architecture notes @@ -874,8 +874,10 @@ class TileDownloader: def fetch(req: FetchRequest) -> DownloadBatchReport: ... class TileUploader: - def upload_pending(flight_state: FlightStateSignal) -> UploadBatchReport: ... - # raises UploadGateBlockedError if flight_state != ON_GROUND + def upload_pending(req: UploadRequest) -> UploadBatchReport: ... + # contract v2.0.0 (frozen) — C11 no longer gates on flight state; + # the post-landing safety check lives in C12's PostLandingUploadOrchestrator + # (reads flight_footer.clean_shutdown from FDR) per Batch 44 SRP refactor. ``` ### Data flow @@ -903,7 +905,7 @@ sequenceDiagram - C11-IT-01: TileDownloader fetch + freshness gate + C6 write byte-identical layout. - C11-IT-02: stale-rejection counts surface in `DownloadBatchReport`. - C11-IT-03: TileUploader posts pending, signs payloads, marks uploaded on 202. -- C11-IT-04: `UploadGateBlockedError` when not ON_GROUND. +- C11-IT-04: post-landing safety gate is now a C12 concern — see `_docs/02_document/components/13_c12_operator_orchestrator/tests.md` C12-IT-03 (Batch 44 SRP refactor; AZ-317 superseded). - C11-IT-05: idempotent retry — already-acked tiles not re-sent. - C11-ST-01: airborne process cannot import `c11_tilemanager` (R02 enforcement). - C11-ST-02: NFT-SEC-02 network-egress test passes. @@ -931,7 +933,7 @@ T-shirt M; 13–21 points. | 1 | TileDownloader: GET + freshness gate + C6 write | 5 | | 2 | TileUploader: read pending + sign + POST + mark uploaded | 5 | | 3 | Idempotent retry on partial-success batch | 3 | -| 4 | `flight_state == ON_GROUND` gate (defense-in-depth) | 2 | +| 4 | ~~`flight_state == ON_GROUND` gate~~ — moved to C12 `PostLandingUploadOrchestrator` (Batch 44 SRP refactor; AZ-317 superseded) | n/a | | 5 | Per-flight ephemeral signing key + zeroisation | 3 | | 6 | Component-internal tests C11-IT-01..05 + C11-PT-01..02 + C11-ST-01..03 + C11-AT-01 | 5 | @@ -1069,9 +1071,9 @@ Operator-facing CLI that sequences pre-flight (C11 download → C10 build) and p ### Scope -**In scope**: CLI subcommands (`download`, `build-cache`, `upload-pending`, `reloc-confirm`), `CacheBuildReport` aggregation, post-landing `flight_state == ON_GROUND` confirmation from FDR, sector-classification UI hook, FDR retrieval helpers. +**In scope**: CLI subcommands (`download`, `build-cache`, `upload-pending`, `reloc-confirm`), `CacheBuildReport` aggregation, `PostLandingUploadOrchestrator` (post-landing safety gate reading `flight_footer.clean_shutdown` from FDR via `FdrFooterReader` — Batch 44 SRP refactor; supersedes the former C11-internal gate), `OperatorReLocService` (AC-3.4 visual-loss hint dispatched via `OperatorCommandTransport` Protocol — E-C8 ships the concrete pymavlink-backed impl), sector-classification UI hook, FDR retrieval helpers. -**Out of scope**: actual download/upload (E-C11); engine compile (E-C10); FDR write side (E-C13). +**Out of scope**: actual download/upload (E-C11; C11 no longer gates internally); engine compile (E-C10); FDR write side (E-C13); concrete `OperatorCommandTransport` (E-C8). ### Architecture notes @@ -1081,10 +1083,26 @@ Operator-facing CLI that sequences pre-flight (C11 download → C10 build) and p ### Interface specification ```python -class OperatorTool: - def build_cache(area: Area, sector_classification: SectorMap) -> CacheBuildReport: ... - def trigger_post_landing_upload(fdr_root: Path) -> UploadBatchReport: ... - def confirm_relocation(candidate: ReLocCandidate) -> None: ... +class BuildCacheOrchestrator: + def build_cache(request: BuildCacheRequest) -> CacheBuildReport: ... + +class PostLandingUploadOrchestrator: + def trigger_post_landing_upload(request: PostLandingUploadRequest) -> UploadBatchReportCut: ... + # raises FlightStateNotConfirmedError(reason) for {footer_missing, + # unclean_shutdown, flight_id_not_found, fdr_unreadable: } + # or SatelliteProviderError on C11 transport failures. + +class OperatorReLocService: + def request_reloc(reloc_hint: ReLocHint) -> None: ... + # raises GcsLinkError with "C12 reloc-confirm: " prefix on link failure. + +class FdrFooterReader(Protocol): + def read_flight_footer(flight_id: FlightId) -> FlightFooterRecord | None: ... + +class OperatorCommandTransport(Protocol): + def send_reloc_hint(hint: ReLocHint) -> None: ... + # concrete impl owned by E-C8 (pymavlink-backed); pattern matches + # AZ-322 BackboneEmbedder (C10 owns Protocol; C2 implements later). ``` ### Data flow @@ -1109,9 +1127,9 @@ sequenceDiagram ### Acceptance criteria -- C12-IT-01: operator re-loc workflow returns SUT to `satellite_anchored` ≤ 30 s (AC-3.4). +- C12-IT-01: operator re-loc workflow (`OperatorReLocService.request_reloc`) returns SUT to `satellite_anchored` ≤ 30 s (AC-3.4); on `GcsLinkError`, CLI exits with `EXIT_GCS_LINK_ERROR` and operator-actionable remediation text. - C12-IT-02: `build_cache` orchestrates C11 then C10; download failure aborts before C10. -- C12-IT-03: `trigger_post_landing_upload` requires ≥ 30 s confirmed ON_GROUND in FDR. +- C12-IT-03: `trigger_post_landing_upload` reads `flight_footer.clean_shutdown` from FDR via `FdrFooterReader` (Batch 44 footer-based gate; replaces the prior 30-s ON_GROUND heuristic). Refusal modes: `footer_missing`, `unclean_shutdown`, `flight_id_not_found`, `fdr_unreadable: ` — each maps to a distinct CLI exit code. - C12-IT-04: actionable failure messages + non-zero exit on stale-tile rate > 30% or manifest signature failure. - C12-ST-01: no CLI command path imports into airborne package boundary. @@ -1131,12 +1149,13 @@ T-shirt M; 13–21 points. | # | Title | Pts | |---|-------|-----| -| 1 | CLI scaffolding + subcommand routing | 3 | -| 2 | `build_cache` orchestration (C11 then C10) | 3 | -| 3 | `trigger_post_landing_upload` with FDR-state confirmation | 3 | -| 4 | AC-3.4 re-localization workflow | 3 | -| 5 | Actionable failure surfacing in CacheBuildReport | 2 | -| 6 | Component-internal tests C12-IT-01..04 + C12-PT-01 + C12-ST-01 + C12-AT-01 | 5 | +| 1 | CLI scaffolding + subcommand routing (AZ-326) | 3 | +| 2 | `BuildCacheOrchestrator` — C11 then C10 sequenced flow + lockfile (AZ-328) | 5 | +| 3 | `PostLandingUploadOrchestrator` + `FdrFooterReader` — Batch 44 footer-based gate (AZ-329) | 3 | +| 4 | `OperatorReLocService` + `OperatorCommandTransport` Protocol — AC-3.4 (AZ-330) | 3 | +| 5 | Companion bringup (SSH-based pre-flight verification) (AZ-327) | 3 | +| 6 | `FlightsApiClient` — operator-origin path (AZ-489) | 3 | +| 7 | Component-internal tests C12-IT-01..04 + C12-PT-01 + C12-ST-01 + C12-AT-01 | 5 | ### Key constraints diff --git a/_docs/02_document/glossary.md b/_docs/02_document/glossary.md index 6694f69..96a6996 100644 --- a/_docs/02_document/glossary.md +++ b/_docs/02_document/glossary.md @@ -70,7 +70,7 @@ Terms are alphabetical. Each entry: one-line definition + parenthetical source. **Operator** — Pre-flight and post-flight human role: authors the flight route in the **Mission Planner UI** (`suite/ui`), classifies the operational area (active-conflict vs stable rear), drives C12 cache provisioning (which reads the `Flight` from the parent-suite `flights` REST service, downloads satellite tiles via the **Tile Manager** for the route bbox, and bakes the takeoff origin into the C10 Manifest), stages calibration onto the companion before takeoff, and after landing triggers the **Tile Manager** upload run. (source: `problem.md`, AC-3.4 / AC-6.2, ADR-010, user confirmation 2026-05-09 + 2026-05-11) -**Tile Manager** — Operator-side component (C11) that owns both directions of network I/O against `satellite-provider`: pre-flight download (F1) into the local C6 store via the `TileDownloader` interface, and post-landing upload (F10) from C6 to the parent-suite ingest endpoint via the `TileUploader` interface (gated on `flight state == ON_GROUND`). Implemented as a separate binary / image so neither network path is loaded in the airborne companion (ADR-004 process-level isolation). Replaces the earlier "post-landing upload tool" naming after Plan-cycle scope expansion 2026-05-09. (source: user directive 2026-05-09) +**Tile Manager** — Operator-side component (C11) that owns both directions of network I/O against `satellite-provider`: pre-flight download (F1) into the local C6 store via the `TileDownloader` interface, and post-landing upload (F10) from C6 to the parent-suite ingest endpoint via the `TileUploader` interface. Carries **no internal flight-state gating** (Batch 44 SRP refactor — the post-landing safety check lives in C12's `PostLandingUploadOrchestrator`, which reads the `flight_footer.clean_shutdown` field). Implemented as a separate binary / image so neither network path is loaded in the airborne companion (ADR-004 process-level isolation). Replaces the earlier "post-landing upload tool" naming after Plan-cycle scope expansion 2026-05-09. (source: user directive 2026-05-09; Batch 44 SRP refactor 2026-05-13) **`satellite-provider`** — First-class architecture boundary: the suite's existing .NET 8 REST microservice at `/Users/obezdienie001/dev/azaion/suite/satellite-provider/`. Runs in Docker (`:5100`, OpenAPI at `/swagger`); downloads Google Maps tiles; stores them in PostgreSQL + filesystem (`./tiles/{zoomLevel}/{x}/{y}.jpg`). Read-only from the onboard runtime; receives post-landing tile uploads via a yet-to-be-designed ingest endpoint (parent-suite work, D-PROJ-2). Synonym in older docs: "Suite Sat Service" / "Azaion Suite Satellite Service". (source: parent-suite `satellite-provider/README.md`, user confirmation 2026-05-09) diff --git a/_docs/02_document/system-flows.md b/_docs/02_document/system-flows.md index d7f8726..683df97 100644 --- a/_docs/02_document/system-flows.md +++ b/_docs/02_document/system-flows.md @@ -18,7 +18,7 @@ | F7 | Spoofing-promotion via EKF source-set switch | FC reports GPS denial/spoof while companion estimate is healthy | C5, C8, [[ArduPilot Plane FC]] | High | | F8 | Companion reboot recovery | Companion process restart while FC remains armed | C8 (FC IMU pose ingest), C5, C10 (warm-cache verify), C13 | Medium | | F9 | GCS telemetry stream | Per-frame estimate available + GCS link healthy | C5, C8, [[QGroundControl]] | Medium | -| F10 | Post-landing tile upload | Operator triggers C11 `TileUploader` with `flight_state == ON_GROUND` confirmed | C11 `TileUploader` (operator-side), C6 (read), [[`satellite-provider`]] (D-PROJ-2 endpoint, planned) | High | +| F10 | Post-landing tile upload | Operator triggers C12 `PostLandingUploadOrchestrator`; orchestrator confirms `flight_footer.clean_shutdown == True` and invokes C11 `TileUploader` | C12 `PostLandingUploadOrchestrator` (operator-side; reads FDR footer), C11 `TileUploader` (operator-side), C6 (read), [[`satellite-provider`]] (D-PROJ-2 endpoint, planned) | High | ## Flow Dependencies @@ -33,7 +33,7 @@ | F7 | F3 (companion estimate health), F8 IMU prior | F3 (becomes primary FC source after switch) | | F8 | F1 + F2 (warm cache survives reboot via content-hash verify) | F3 (resumes once warm), F5 (degraded mode if recovery fails) | | F9 | F3 | n/a (read-only outbound) | -| F10 | F4 (locally-saved tiles), F8 confirmed `flight_state == ON_GROUND`, parent-suite D-PROJ-2 endpoint availability | F1 of the next flight (uploaded tiles enter the basemap once promoted to `trusted`) | +| F10 | F4 (locally-saved tiles), C13 `flight_footer` written on clean shutdown, parent-suite D-PROJ-2 endpoint availability | F1 of the next flight (uploaded tiles enter the basemap once promoted to `trusted`) | **Cross-cutting**: F13 FDR-write is not a flow per se — every flow above has an FDR write side-effect. AC-NEW-3 requires every payload class (estimate, IMU, MAVLink, mid-flight tile, system health, failed-tile thumbnail) to be present; rollover is logged, never silent. @@ -965,11 +965,11 @@ sequenceDiagram ### Description -After the UAV has landed and `flight_state == ON_GROUND` is confirmed, the operator triggers the C11 Tile Manager's `TileUploader` (a separate operator-side process / image — **not present in the airborne companion image**, ADR-004) which reads locally-saved mid-flight tiles from C6 and uploads them to `satellite-provider`'s ingest endpoint per the D-PROJ-2 contract sketch. Each tile carries quality metadata sufficient for the parent-suite voting layer to decide promotion `pending → trusted` (D-PROJ-2 design task #2; not yet implemented service-side). Until the real endpoint ships, integration tests target the e2e-test `mock-suite-sat-service` fixture under `tests/fixtures/`; production never reaches the fixture. +After the UAV has landed, the operator triggers C12's `PostLandingUploadOrchestrator` (`operator-orchestrator upload-pending --flight-id ...`). The orchestrator reads the `flight_footer` FDR record for the given `flight_id` via `FdrFooterReader`; if the footer is present AND `clean_shutdown == True`, it invokes C11's `TileUploader` (via the `TileUploaderCut` Protocol) which reads locally-saved mid-flight tiles from C6 and uploads them to `satellite-provider`'s ingest endpoint per the D-PROJ-2 contract sketch. The C11 Tile Manager is a separate operator-side process / image — **not present in the airborne companion image**, ADR-004. Each tile carries quality metadata sufficient for the parent-suite voting layer to decide promotion `pending → trusted` (D-PROJ-2 design task #2; not yet implemented service-side). Until the real endpoint ships, integration tests target the e2e-test `mock-suite-sat-service` fixture under `tests/fixtures/`; production never reaches the fixture. ### Preconditions -- `flight_state == ON_GROUND` confirmed by the FC's `MAV_STATE` (operator's workstation reads this off the FC or from the FDR). +- C13 has written the `flight_footer` FDR record for the requested `flight_id` with `clean_shutdown == True`. This is the single safety invariant; the operator does not query FC `MAV_STATE` directly (Batch 44 SRP refactor — the footer is the authoritative "vehicle stopped cleanly" signal). - Operator workstation has network reach to `satellite-provider` (in tests, the e2e `mock-suite-sat-service` fixture stands in for the not-yet-shipped POST endpoint). - Local C6 tile store has mid-flight tiles with `voting_status=pending` and quality metadata. - Per-flight onboard signing key (generated at takeoff load, baked into tile metadata) is available to C11 `TileUploader` for payload signing. @@ -1001,9 +1001,10 @@ sequenceDiagram ```mermaid flowchart TD - Start([Operator triggers C11 TileUploader with flight_id]) --> StateCheck{flight_state == ON_GROUND confirmed?} - StateCheck -->|no| Refuse[Refuse to upload; report to operator] - StateCheck -->|yes| ReadTiles[Read mid-flight tiles voting_status=pending] + Start([Operator triggers C12 PostLandingUploadOrchestrator with flight_id]) --> FooterRead[C12 FdrFooterReader reads flight_footer FDR record] + FooterRead --> StateCheck{footer present AND clean_shutdown == True?} + StateCheck -->|no — footer_missing / unclean_shutdown / flight_id_not_found / fdr_unreadable| Refuse[Raise FlightStateNotConfirmedError with sub-reason + operator-actionable remediation] + StateCheck -->|yes| ReadTiles[C11 TileUploader reads mid-flight tiles voting_status=pending] ReadTiles --> Empty{Any tiles to upload?} Empty -->|no| Done([No-op; report]) Empty -->|yes| Batch[Batch by configurable size] @@ -1038,7 +1039,7 @@ flowchart TD | Error | Where | Detection | Recovery | |-------|-------|-----------|----------| -| `flight_state != ON_GROUND` | Step 1 | FC `MAV_STATE` query | Refuse upload; never proceed (architectural invariant) | +| `flight_footer` missing OR `clean_shutdown == False` OR `flight_id` not found OR FDR unreadable | Step 1 (C12 `FdrFooterReader`) | Raises `FlightStateNotConfirmedError(reason=...)` with one of 4 sub-reasons (`footer_missing`, `unclean_shutdown`, `flight_id_not_found`, `fdr_unreadable: `) | Refuse upload; CLI exits with a distinct exit code per sub-reason; operator must fix the FDR or pick the right `flight_id` before retrying (architectural invariant — no auto-retry) | | `satellite-provider` ingest endpoint not yet implemented (D-PROJ-2 open) | Step 3 | 404 / 501 / connection refused | Keep batches queued locally; report to operator; retry on next operator trigger | | Network rate-limit (429) | Step 3 | HTTP 429 | Back off + retry | | Per-tile rejected by service | Step 4 | per-tile status `rejected` | Mark `voting_status=rejected_by_service`; FDR logs reason; do not retry that tile |