mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 08:21:13 +00:00
[AZ-329] [AZ-330] [AZ-523] [AZ-524] Batch 44 atomic refactor
Implements two new C12 services and rebalances the C11/C12 boundary in one atomic commit: * AZ-329 PostLandingUploadOrchestrator — gates C11 upload on the `flight_footer` FDR record's `clean_shutdown` field; 4 refusal modes; new FdrFooterReader Protocol + LocalFdrFooterReader. * AZ-330 OperatorReLocService — AC-3.4 visual-loss re-localization hint; reuses shared LatLonAlt; OperatorCommandTransport Protocol cut (E-C8 owns the future pymavlink concrete); new FDR record kind `c12.reloc.requested`; log redaction (lat/lon 5 decimals, reason 200 chars). * AZ-523 C11 internal flight-state gate removed (SRP refactor): `confirm_flight_state` / `FlightStateSignal` use / `FlightStateNotOnGroundError` deleted from C11; TileUploader contract bumped to v2.0.0 (frozen) with migration note; AZ-317 superseded. * AZ-524 Package rename `c12_operator_tooling` → `c12_operator_orchestrator` across source, tests, pyproject, CMake, Dockerfile, compose, CI, runtime-root services class (`OperatorOrchestratorServices`) + factory function (`build_operator_orchestrator`), logger namespaces, config slug, docs, and the E-C12 epic title. Tests: 1543 passed, 80 skipped (all environment gates). Targeted AC suite (AZ-329 + AZ-330 + FdrFooterReader): 37 passed. Cold-start NFR-perf still ≤ 500 ms p99. Tracker: AZ-317 → Done (superseded); AZ-319 v2.0.0 contract bump comment; AZ-329/AZ-330 → In Testing; AZ-253 epic renamed; AZ-523 + AZ-524 created and closed as audit-trail tickets. See `_docs/03_implementation/batch_44_cycle1_report.md`. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -145,7 +145,7 @@ would break AC-6.
|
||||
|
||||
**Potential race conditions**:
|
||||
|
||||
- Concurrent `build_cache_artifacts` invocations on the same cache root would corrupt state. Single-process operator-tool wraps with a filesystem lockfile (the same lockfile C11 honours); if a second invocation tries to start, fail with explicit error.
|
||||
- Concurrent `build_cache_artifacts` invocations on the same cache root would corrupt state. Single-process operator-orchestrator wraps with a filesystem lockfile (the same lockfile C11 honours); if a second invocation tries to start, fail with explicit error.
|
||||
|
||||
**Performance bottlenecks**:
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
**Purpose**: own the operator-side network I/O against `satellite-provider` for the onboard tile corpus, in **both directions**:
|
||||
|
||||
- **Download** (pre-flight, F1): fetch tiles from `satellite-provider` for the operational area, apply AC-NEW-6 freshness gating, and write into C6 (`TileStore` + `TileMetadataStore`). C11 is the **only** path that crosses the workstation/companion enclave to the parent suite for tile pixels — C10 reads from the populated C6 store and never touches `satellite-provider` itself.
|
||||
- **Upload** (post-landing, F10): when `flight_state == ON_GROUND` is confirmed, read pending mid-flight tiles from C6 and POST to `satellite-provider`'s ingest endpoint (D-PROJ-2 contract sketch).
|
||||
- **Upload** (post-landing, F10): read pending mid-flight tiles from C6 and POST to `satellite-provider`'s ingest endpoint (D-PROJ-2 contract sketch). C11 itself does NOT gate on flight state — it is a dumb pipe; the post-landing safety gate is owned by C12's `PostLandingUploadOrchestrator` (AZ-329 / Batch 44), which checks the C13 `flight_footer` FDR record for `clean_shutdown=True` before invoking `TileUploader.upload_pending_tiles`.
|
||||
|
||||
C11 is a **separate operator-side binary / image**. The airborne companion image's CMake target deliberately excludes the entire `c11_tilemanager/` source tree so the airborne process cannot accidentally execute either the download path or the upload path even via reflection or config error (ADR-004 process-level isolation, AC-8.4). Both directions of tile I/O are operator-driven on the operator workstation; the companion only consumes the populated C6 store while airborne.
|
||||
|
||||
@@ -36,10 +36,11 @@ C11 is a **separate operator-side binary / image**. The airborne companion image
|
||||
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
| `confirm_flight_state` | `()` | `FlightStateSignal` (must be ON_GROUND) | No | `FlightStateNotOnGroundError` |
|
||||
| `enumerate_pending_tiles` | `flight_id: uuid (optional)` | `list[TileMetadata]` | No | `TileMetadataError` |
|
||||
| `upload_pending_tiles` | `UploadRequest` | `UploadBatchReport` | No | `SatelliteProviderError`, `RateLimitedError`, `SignatureRejectedError` |
|
||||
|
||||
C11 no longer exposes `confirm_flight_state` — the post-landing flight-state gate moved to C12 (`PostLandingUploadOrchestrator`, AZ-329) per Batch 44. `FlightStateNotOnGroundError` is retired from C11; the corresponding refusal now lives at the C12 boundary as `FlightStateNotConfirmedError`.
|
||||
|
||||
**Input/Output DTOs**:
|
||||
|
||||
```
|
||||
@@ -65,8 +66,6 @@ UploadRequest:
|
||||
batch_size: int
|
||||
satellite_provider_url: URL
|
||||
|
||||
FlightStateSignal: see C8 — must be ON_GROUND for any upload to proceed
|
||||
|
||||
UploadBatchReport:
|
||||
batch_uuid: uuid (assigned by satellite-provider per D-PROJ-2 contract)
|
||||
per_tile_status: list[(tile_id, status: enum {queued, rejected, duplicate, superseded})]
|
||||
@@ -154,9 +153,10 @@ C11 reads from / writes to C6 (the local store) and reads from / writes to `sate
|
||||
- `RateLimitedError` (429): obey `Retry-After`; the operator can also re-invoke later. Same handling either direction.
|
||||
- `FreshnessRejectionError` / `ResolutionRejectionError`: download-side only. Per AC-NEW-6 / RESTRICT-SAT-4 — never silently downgrade fresh-required tiles in `active_conflict` sectors. Surface counts in the `DownloadBatchReport`.
|
||||
- `CacheBudgetExceededError`: download-side only. Pre-flight free-space check against AC-8.3 (≤ 10 GB). Fail fast with explicit budget delta; no partial write.
|
||||
- `FlightStateNotOnGroundError`: upload-side only. Refuse to start; log + show explicit reason. ADR-004 process-level isolation means C11 should never run when the FC believes it's airborne — this error is a defense-in-depth, not the primary control.
|
||||
- `SignatureRejectedError`: upload-side only. Per-flight signing key was rejected by `satellite-provider`. This is a security-critical event — do NOT silently drop; surface to operator + log to FDR.
|
||||
|
||||
Post-landing safety: C11's upload path no longer gates on flight state internally. The check now lives in C12's `PostLandingUploadOrchestrator` (AZ-329 / Batch 44), which refuses to invoke `TileUploader.upload_pending_tiles` unless the C13 `flight_footer` FDR record records `clean_shutdown=True` for the target flight. ADR-004 process-level isolation remains the primary control — C11 should never run on the companion at all.
|
||||
|
||||
## 6. Extensions and Helpers
|
||||
|
||||
| Helper | Purpose | Used By |
|
||||
@@ -192,7 +192,7 @@ C11 reads from / writes to C6 (the local store) and reads from / writes to `sate
|
||||
|
||||
| Log Level | When | Example |
|
||||
|-----------|------|---------|
|
||||
| ERROR | `FlightStateNotOnGroundError`, `SignatureRejectedError`, persistent `SatelliteProviderError`, `CacheBudgetExceededError` | `C11 refused to start: flight_state=IN_AIR; safeguard active` |
|
||||
| ERROR | `SignatureRejectedError`, persistent `SatelliteProviderError`, `CacheBudgetExceededError` | `C11 upload failure: signature rejected by satellite-provider` |
|
||||
| WARN | one-off network failure, scheduled retry, freshness-driven rejections (counts) | `C11 batch upload retry: batch_uuid=…; next_retry_in_s=30` |
|
||||
| INFO | session start/end; per-batch report (download + upload) | `C11 download complete: 87654 tiles, 12 stale-rejected; bbox=…` |
|
||||
| DEBUG | per-tile request/response | `C11 tile uploaded: tile_id=(z=18,lat=…,lon=…); status=queued` |
|
||||
|
||||
@@ -66,19 +66,13 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C11 wa
|
||||
|
||||
---
|
||||
|
||||
### C11-IT-04: TileUploader gates on `flight_state == ON_GROUND`
|
||||
### C11-IT-04: post-landing safety gate lives in C12 (cross-reference)
|
||||
|
||||
**Summary**: `TileUploader.upload_pending` refuses to run if `FlightStateSignal != ON_GROUND` (defense-in-depth atop ADR-004 process isolation).
|
||||
**Summary**: post-landing safety is owned by C12, not C11. The gate that historically lived in `TileUploader.upload_pending_tiles` was removed in Batch 44 (supersedes AZ-317); the equivalent check now lives in C12's `PostLandingUploadOrchestrator` (AZ-329) and refuses to invoke `TileUploader.upload_pending_tiles` unless the C13 `flight_footer` FDR record records `clean_shutdown=True` for the target flight.
|
||||
|
||||
**Traces to**: AC-8.4 (defensive — ADR-004's secondary guard)
|
||||
**Traces to**: see `_docs/02_document/components/13_c12_operator_orchestrator/tests.md` → C12-IT-03 for the post-landing safety test.
|
||||
|
||||
**Description**: call `upload_pending` with `FlightStateSignal == IN_FLIGHT`; assert `UploadGateBlockedError`. Same with `UNKNOWN`. Set `ON_GROUND` and assert upload proceeds.
|
||||
|
||||
**Input data**: scripted FlightStateSignal source.
|
||||
|
||||
**Expected result**: upload blocked except in `ON_GROUND`.
|
||||
|
||||
**Max execution time**: 30 s.
|
||||
**Status**: cross-reference only. C11's `TileUploader` no longer exposes `confirm_flight_state` or raises `FlightStateNotOnGroundError`.
|
||||
|
||||
---
|
||||
|
||||
@@ -193,10 +187,10 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C11 wa
|
||||
|
||||
| Step | Action | Expected Result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | `operator-tool download --area derkachi.geojson --since 2026-01` | `DownloadBatchReport` printed; tiles in C6 |
|
||||
| 2 | `operator-tool build-cache` | C10 builds engines + descriptors + Manifest |
|
||||
| 1 | `operator-orchestrator download --area derkachi.geojson --since 2026-01` | `DownloadBatchReport` printed; tiles in C6 |
|
||||
| 2 | `operator-orchestrator build-cache` | C10 builds engines + descriptors + Manifest |
|
||||
| 3 | (simulate flight) | (covered by other tests) |
|
||||
| 4 | `operator-tool upload-pending` | Pending-upload tiles POSTed; report printed |
|
||||
| 4 | `operator-orchestrator upload-pending` | Pending-upload tiles POSTed; report printed |
|
||||
|
||||
---
|
||||
|
||||
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
# C12 — Operator Pre-flight Tooling
|
||||
# C12 — Operator Pre-flight Orchestrator
|
||||
|
||||
## 1. High-Level Overview
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
| `build_cache` | `flight_id` (online) OR `flight_file: Path` (offline), `sector_class`, `calibration_path`, `satellite_provider_url`, `api_key` | `CacheBuildReport` (wraps `FlightResolveReport` + C11 `DownloadBatchReport` + C10 `BuildReport`) | No (operator-facing; minutes) | `CacheBuildError` (wraps `FlightNotFoundError`, `FlightsApiUnreachableError`, `SatelliteProviderError`, `EngineBuildError`, etc.) |
|
||||
| `trigger_post_landing_upload` | `flight_id` | C11 `UploadBatchReport` | No (operator-facing; minutes) | `CacheBuildError` wrapper around `FlightStateNotOnGroundError`, `SignatureRejectedError`, etc. |
|
||||
| `trigger_post_landing_upload` | `PostLandingUploadRequest` (`flight_id`, `satellite_provider_url`, `api_key`, `batch_size`) | C11 `UploadBatchReport` (re-exposed as `UploadBatchReportCut`) | No (operator-facing; minutes) | `FlightStateNotConfirmedError` (footer missing / unclean / fdr-unreadable / flight-id not found), `SatelliteProviderError`, `SignatureRejectedError` (passthrough from C11) |
|
||||
| `verify_companion_ready` | `companion_address` | `ReadinessReport` | No | `CompanionUnreachableError`, `ContentHashMismatchError` |
|
||||
| `set_sector_classification` | `area, sector_class` | `None` | No | — |
|
||||
| `apply_freshness_threshold` | `sector_class` | `int (months)` | No | — |
|
||||
+8
-8
@@ -1,4 +1,4 @@
|
||||
# Test Specification — C12 Operator Pre-flight Tooling
|
||||
# Test Specification — C12 Operator Pre-flight Orchestrator
|
||||
|
||||
Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 sequences the F1 (C11 download → C10 build) and F10 (C11 upload trigger) operator-side flows.
|
||||
|
||||
@@ -47,17 +47,17 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 se
|
||||
|
||||
---
|
||||
|
||||
### C12-IT-03: trigger_post_landing_upload invokes C11 TileUploader on confirmed ON_GROUND
|
||||
### C12-IT-03: trigger_post_landing_upload invokes C11 TileUploader on confirmed clean-shutdown footer
|
||||
|
||||
**Summary**: `trigger_post_landing_upload` reads the most recent `FlightStateSignal` from the post-flight FDR; if `ON_GROUND` is confirmed for ≥ a configurable safety threshold (default 30 s), it invokes `C11.TileUploader.upload_pending`. If `ON_GROUND` is not confirmed, it refuses and returns a clear error.
|
||||
**Summary**: `trigger_post_landing_upload` reads the post-flight FDR newest-segment-first looking for a `flight_footer` record (kind registered by C13 in AZ-292; emitted exactly once per flight on `close_flight()`); if found with `payload["clean_shutdown"] == True`, it invokes `C11.TileUploader.upload_pending_tiles(UploadRequest(flight_id=..., ...))`. If the footer is absent (truncation / crash) or carries `clean_shutdown == False`, it refuses with `FlightStateNotConfirmedError`.
|
||||
|
||||
**Traces to**: AC-8.4
|
||||
|
||||
**Description**: stage two flight FDR fixtures — one ending with confirmed ON_GROUND for 60 s, one ending with `IN_FLIGHT` (incomplete log). Call `trigger_post_landing_upload`; assert (a) first case invokes upload, (b) second case refuses with `FlightStateNotConfirmedError`.
|
||||
**Description**: stage two flight FDR fixtures produced by C13's `FileFdrWriter` — one with a clean-shutdown footer (the writer's standard `close_flight()` path, which always sets `clean_shutdown=True` in the current AZ-292 implementation), one truncated (writer terminated before `close_flight()` ran, so no footer record). Call `trigger_post_landing_upload`; assert (a) first case invokes upload via the `TileUploaderCut` and returns the recorded `UploadBatchReport`, (b) second case refuses with `FlightStateNotConfirmedError(not_confirmed_reason="footer_missing")`.
|
||||
|
||||
**Input data**: 2 scripted FDR fixtures.
|
||||
**Input data**: 2 FDR fixtures generated by `FileFdrWriter` (one closed cleanly; one with the close skipped).
|
||||
|
||||
**Expected result**: per assertion.
|
||||
**Expected result**: per assertion. No 30-second ON_GROUND threshold is consulted — the footer's existence + `clean_shutdown` flag is the sole signal.
|
||||
|
||||
**Max execution time**: 60 s.
|
||||
|
||||
@@ -99,7 +99,7 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 se
|
||||
|
||||
### C12-ST-01: CLI rejects writes to airborne images
|
||||
|
||||
**Summary**: the operator-tool CLI has no command path that writes into the airborne `production-binary` image (defends against operator-side mistakes that would defeat ADR-004).
|
||||
**Summary**: the operator-orchestrator CLI has no command path that writes into the airborne `production-binary` image (defends against operator-side mistakes that would defeat ADR-004).
|
||||
|
||||
**Traces to**: ADR-004 R02 enforcement (C12 side)
|
||||
|
||||
@@ -135,7 +135,7 @@ Component-scoped. Suite-level coverage in `_docs/02_document/tests/*.md`. C12 se
|
||||
| Data Set | Source | Size |
|
||||
|----------|--------|------|
|
||||
| Operator-tooling tarball | CI build artifact | varies |
|
||||
| FDR fixtures (ON_GROUND-confirmed and IN_FLIGHT) | scripted | <100 MB each |
|
||||
| FDR fixtures (clean-shutdown footer present / footer absent) | generated by C13 `FileFdrWriter` | <100 MB each |
|
||||
| Small Derkachi sub-area for C12-IT-02 | scripted | <500 MB |
|
||||
|
||||
**Setup**: extract operator-tooling tarball; bring up Docker compose.
|
||||
@@ -90,7 +90,7 @@ Not applicable.
|
||||
|---------|---------|---------|
|
||||
| orjson / msgpack | per project pin | Record serialisation (serialised format choice during decompose phase) |
|
||||
| atomicwrites | latest | Segment file rotation (atomic open of new segment + close of previous) |
|
||||
| filelock | per project pin | Cross-process safety for the FDR root (operator-tool reads while companion writes — companion-only access during flight) |
|
||||
| filelock | per project pin | Cross-process safety for the FDR root (operator-orchestrator reads while companion writes — companion-only access during flight) |
|
||||
|
||||
**Error Handling Strategy**:
|
||||
- `FdrOpenError` at takeoff: refuse takeoff (per AC-NEW-3 every payload class must be present from t=0).
|
||||
|
||||
Reference in New Issue
Block a user