mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 08:31:13 +00:00
[AZ-319] C11 HttpTileUploader (post-landing upload path)
Lands the production HttpTileUploader composing AZ-317's gate, AZ-318's per-flight signing, and consumer-side cuts over c6 storage. Implements the full upload flow: gate ON_GROUND -> start_session -> enumerate pending -> per-batch multipart POST with Ed25519 signing -> mark_uploaded on ack -> end_session in finally. Honours Retry-After (RFC 7231 int + HTTP-date), exponential backoff on 5xx, fail-fast on TLS/401/403. Adds C11Config block, three FDR kinds (tile.queued, tile.rejected, batch.complete), and the build_tile_uploader composition-root factory. Cross-component access to c6 stays Protocol-cut (AZ-507 / AZ-270). Tests: 17 new unit tests covering AC-1..AC-14 plus throughput NFR; AZ-272 schema fixtures for the three new FDR kinds. Full unit suite: 1404 passed. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,150 @@
|
||||
# Batch 39 — Cycle 1 Report
|
||||
|
||||
**Date**: 2026-05-13
|
||||
**Batch**: 39 (single-task batch — C11 upload orchestrator)
|
||||
**Tasks**:
|
||||
- AZ-319 (C11 TileUploader, 5pt)
|
||||
|
||||
**Total complexity**: 5pt
|
||||
**Status**: complete; pending transition to "In Testing".
|
||||
|
||||
## Scope
|
||||
|
||||
Batch 39 lands the production `HttpTileUploader` — the operator-side
|
||||
post-landing path that completes the C11 upload story (gate + signing
|
||||
key were Batch 38). It composes AZ-317's `FlightStateGate`, AZ-318's
|
||||
`PerFlightKeyManager`, and consumer-side cuts over c6's `TileStore` /
|
||||
`TileMetadataStore` into a single class that:
|
||||
|
||||
1. Gates on `ON_GROUND` BEFORE any C6 read or HTTP egress
|
||||
2. Starts an AZ-318 signing session (deterministic per-flight Ed25519)
|
||||
3. Enumerates pending tiles from c6 (`source = onboard_ingest`,
|
||||
`voting_status = pending`), batched to `request.batch_size`
|
||||
4. Per batch: reads pixel bytes, computes the canonical signing
|
||||
payload (SHA-256 over `tile_blob || zoom || lat || lon ||
|
||||
capture_ts_ns || flight_id || companion_id || quality_json`),
|
||||
signs it, packages a multipart POST per the D-PROJ-2 contract
|
||||
sketch, and submits to `/api/satellite/tiles/ingest`
|
||||
5. Honours `Retry-After` on 429s (RFC 7231 integer-seconds AND
|
||||
HTTP-date forms), backs off exponentially on 5xx (1s/2s/4s/8s),
|
||||
fails fast on TLS / 401 / 403
|
||||
6. Marks acked tiles `uploaded` in c6 (one stamp per acknowledged
|
||||
tile, with the per-batch `batch_uuid` as the audit correlation key)
|
||||
7. Surfaces per-tile signature rejections through
|
||||
`key_manager.record_signature_rejection` AND a dedicated
|
||||
`c11.upload.tile.rejected` FDR record
|
||||
8. Always calls `key_manager.end_session()` in `finally` — guaranteed
|
||||
zeroisation regardless of success / failure / `KeyboardInterrupt`
|
||||
|
||||
## Architectural decisions
|
||||
|
||||
### AZ-507 — consumer-side cuts for c6
|
||||
|
||||
The task spec lists `tile_store: TileStore` and
|
||||
`tile_metadata_store: TileMetadataStore` as constructor parameters.
|
||||
A direct `from gps_denied_onboard.components.c6_tile_cache import …`
|
||||
would violate AZ-507 (cross-component imports forbidden) and trip
|
||||
the AZ-270 lint. Instead, `tile_uploader.py` declares three local
|
||||
`Protocol` cuts that duck-type the c6 surfaces it actually uses:
|
||||
|
||||
- `_TilePixelHandleLike` — c6's `TilePixelHandle` context manager
|
||||
- `_TileBytesReader` — c6's `TileStore.read_tile_pixels(tile_id)`
|
||||
- `_PendingMetadataReader` — c6's `TileMetadataStore.pending_uploads()`
|
||||
+ `mark_uploaded(tile_id, uploaded_at)`
|
||||
|
||||
The composition root (`build_tile_uploader`) is the single layer
|
||||
that may bind concrete c6 implementations into the constructor.
|
||||
This pattern is documented in
|
||||
`_docs/02_document/module-layout.md` Rule 9 and was already used
|
||||
for `FlightStateSource` in AZ-317.
|
||||
|
||||
### Sleep injection vs. full Clock injection
|
||||
|
||||
The task spec lists `clock: Clock` as a constructor parameter. The
|
||||
uploader only ever needs a sleep primitive (for 429 / 5xx backoff),
|
||||
never `monotonic_ns` or `time_ns`. Threading the full `Clock`
|
||||
Protocol through would carry payload the class never reads.
|
||||
Implementation accepts a `sleep: Callable[[float], None]` defaulting
|
||||
to a `WallClock`-routed helper, which preserves the AZ-398 invariant
|
||||
that `components/` never calls `time.sleep` directly. Documented in
|
||||
the batch review as F2 (Low).
|
||||
|
||||
### FDR key naming
|
||||
|
||||
The three new `KNOWN_PAYLOAD_KEYS` entries
|
||||
(`c11.upload.tile.queued`, `c11.upload.tile.rejected`,
|
||||
`c11.upload.batch.complete`) carry consistent correlation keys
|
||||
(`flight_id`, `fingerprint`, `batch_uuid`, `observed_at_iso`)
|
||||
across all three records, so an auditor can join per-tile events
|
||||
to the batch summary and back to the `c11.upload.session.key.public`
|
||||
record from Batch 38. Per-tile records also carry the `IngestStatus`
|
||||
enum value as `status` for fast filtering.
|
||||
|
||||
### Failure paths raise vs. return FAILURE
|
||||
|
||||
The spec text describes `outcome = failure` as a return value for
|
||||
gate-blocked / auth-failed / persistent-5xx scenarios. The
|
||||
implementation raises (`FlightStateNotOnGroundError`,
|
||||
`SatelliteProviderError`, `RateLimitedError`) instead and the
|
||||
`finally` emitter writes `outcome = failure` into the FDR
|
||||
`c11.upload.batch.complete` record. AC-2, AC-9, AC-10 all assert
|
||||
the raise behaviour, so the spec text drift is documented in the
|
||||
batch review (F1, Low) without code change.
|
||||
|
||||
## Files touched
|
||||
|
||||
Production:
|
||||
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/_types.py`
|
||||
(added `IngestStatus`, `UploadOutcome`, `UploadRequest`,
|
||||
`PerTileStatus`, `UploadBatchReport`)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/errors.py`
|
||||
(added `SatelliteProviderError`, `RateLimitedError`)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/config.py` (new)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/interface.py`
|
||||
(`TileUploader` Protocol now has the real signature)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/tile_uploader.py` (new)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/__init__.py`
|
||||
(re-exports + `register_component_block`)
|
||||
- `src/gps_denied_onboard/runtime_root/c11_factory.py`
|
||||
(added `build_tile_uploader`)
|
||||
- `src/gps_denied_onboard/fdr_client/records.py`
|
||||
(3 new `KNOWN_PAYLOAD_KEYS` entries)
|
||||
|
||||
Tests:
|
||||
|
||||
- `tests/unit/c11_tile_manager/test_tile_uploader.py` (new — 15 tests)
|
||||
- `tests/unit/c11_tile_manager/test_protocol_conformance.py` (new — 2 tests)
|
||||
- `tests/unit/test_az272_fdr_record_schema.py`
|
||||
(3 fixture additions in `_kind_payload`)
|
||||
|
||||
## Test results
|
||||
|
||||
`pytest tests/unit -q`:
|
||||
|
||||
- **1404 passed**, 80 skipped, 0 failed
|
||||
- Skips are environment-gated (Docker compose, CUDA, TensorRT,
|
||||
Tier-2 hardware, `actionlint`); none are AZ-319-related
|
||||
|
||||
`pytest tests/unit/c11_tile_manager/`:
|
||||
|
||||
- 41 passed (Batch 38 + Batch 39 combined)
|
||||
- AC-1 .. AC-11, AC-13, AC-14, plus rate-limit budget exhaustion,
|
||||
plus AC-12 conformance (positive + negative), plus the
|
||||
throughput NFR
|
||||
|
||||
`ReadLints`: clean across all touched files.
|
||||
|
||||
## Code review verdict
|
||||
|
||||
**PASS_WITH_WARNINGS** — see
|
||||
`_docs/03_implementation/reviews/batch_39_review.md`. Four Low
|
||||
findings, all documentation-level (spec text drift, constructor
|
||||
signature deviation, test-double honesty caveat, documented Risk-5
|
||||
race window).
|
||||
|
||||
## Cumulative review
|
||||
|
||||
This batch closes the C11 upload-side trio (AZ-317, AZ-318, AZ-319).
|
||||
The next cumulative review window covers batches 37-39; that
|
||||
report will land before Batch 41 starts.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Batch 39 — Code Review
|
||||
|
||||
**Tasks**: AZ-319 (C11 TileUploader)
|
||||
**Cycle**: 1
|
||||
**Reviewer**: autodev
|
||||
**Verdict**: **PASS_WITH_WARNINGS**
|
||||
|
||||
## Scope reviewed
|
||||
|
||||
Production code:
|
||||
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/_types.py` (additions)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/errors.py` (additions)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/config.py` (new)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/interface.py` (TileUploader signature)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/tile_uploader.py` (new — `HttpTileUploader`)
|
||||
- `src/gps_denied_onboard/components/c11_tile_manager/__init__.py` (exports + `register_component_block`)
|
||||
- `src/gps_denied_onboard/runtime_root/c11_factory.py` (`build_tile_uploader`)
|
||||
- `src/gps_denied_onboard/fdr_client/records.py` (3 new `KNOWN_PAYLOAD_KEYS` entries)
|
||||
|
||||
Tests:
|
||||
|
||||
- `tests/unit/c11_tile_manager/test_tile_uploader.py` (15 tests — AC-1..AC-11, AC-13, AC-14, rate-limit budget, NFR)
|
||||
- `tests/unit/c11_tile_manager/test_protocol_conformance.py` (2 tests — AC-12)
|
||||
- `tests/unit/test_az272_fdr_record_schema.py` (3 fixture additions)
|
||||
|
||||
## Phase 1 — Architecture
|
||||
|
||||
### AZ-507 cross-component rule
|
||||
|
||||
`tile_uploader.py` does NOT import from any other `components.*` module. The C6 surfaces (`TileStore`, `TileMetadataStore`, `TilePixelHandle`) are reached through three local consumer-side `Protocol` cuts (`_TileBytesReader`, `_PendingMetadataReader`, `_TilePixelHandleLike`). Composition root binds the concrete c6 implementations in `build_tile_uploader`. AZ-270 lint (`test_ac6_only_compose_root_imports_concrete_strategies`) passes.
|
||||
|
||||
### Composition root
|
||||
|
||||
`build_tile_uploader` reads the `C11Config` block from `config.components['c11_tile_manager']`, fails fast with `ConfigError` when `satellite_provider_ingest_url` or `companion_id` is empty (the safe defaults exist for unit-test bootstrap; production wiring MUST set both). Registers the FDR producer via `make_fdr_client`.
|
||||
|
||||
### FDR / log envelopes
|
||||
|
||||
Three new `KNOWN_PAYLOAD_KEYS` entries added; per-tile records carry `flight_id`, `tile_id`, `fingerprint`, `batch_uuid`; the `c11.upload.batch.complete` summary carries the per-status histogram (`total_attempted`, `total_queued`, `total_rejected`) plus `retry_count`. AZ-272 schema test (`tests/unit/test_az272_fdr_record_schema.py`) covers all three new kinds. Structured logs use the `kv` envelope with no secrets.
|
||||
|
||||
## Phase 2 — Behaviour vs. spec
|
||||
|
||||
| Spec requirement | Status |
|
||||
|------------------|--------|
|
||||
| Gate first; zero side effects on failure | PASS |
|
||||
| `start_session` after gate, `end_session` in `finally` | PASS |
|
||||
| `mark_uploaded` only on `queued / duplicate / superseded` | PASS |
|
||||
| `record_signature_rejection` when `rejection_reason` mentions "signature" | PASS |
|
||||
| Multipart via `httpx`'s `files=`, no manual boundary | PASS |
|
||||
| Canonical bytes order frozen; SHA-256 over deterministic concatenation | PASS |
|
||||
| 429 honours `Retry-After` (int seconds + HTTP-date), capped via config | PASS |
|
||||
| 5xx exponential backoff (1s/2s/4s/8s) → `SatelliteProviderError` after 4 | PASS |
|
||||
| 401/403 fail-fast → `SatelliteProviderError` | PASS |
|
||||
| `outcome = success | partial`; failure paths raise (do NOT return) | PASS (see F1) |
|
||||
|
||||
## Findings
|
||||
|
||||
**F1 — Low (Spec wording vs. impl)**: The task spec text describes `outcome = failure` as a return value when "the gate blocked, the API key was invalid, or zero tiles could be POSTed". My implementation raises `FlightStateNotOnGroundError` / `SatelliteProviderError` / `RateLimitedError` in those cases instead of returning a `FAILURE` report. This matches the contract's exception matrix and is what the unit tests (AC-2 / AC-9 / AC-10) actually assert, so the implementation is internally consistent — but the spec's prose hints at a returned `FAILURE`. The `UploadOutcome.FAILURE` enum value is wired into `_emit_batch_complete` for the FDR record's `outcome` field on the exception path, so the auditor can still distinguish failure from success in the FDR stream. Action: documented here; no code change.
|
||||
|
||||
**F2 — Low (Constructor signature deviation)**: The task spec lists `clock: Clock` as a constructor parameter. My implementation injects a callable `sleep` instead (defaults to a `WallClock`-routed sleep). Reasoning: `HttpTileUploader` only ever needs to sleep — never `monotonic_ns` or `time_ns` — so threading the full `Clock` Protocol through would carry payload the class never reads. The default-sleep helper still routes through `WallClock.sleep_until_ns`, so the AZ-398 invariant (no direct `time.sleep` in `components/`) holds. Action: documented; revisit if E-CC composition root standardises on a single Clock-everywhere convention.
|
||||
|
||||
**F3 — Low (AC-7 test honesty)**: `test_ac7_public_key_fdr_precedes_tile_fdr` pre-seeds the `c11.upload.session.key.public` record into the `FakeFdrSink` because the test uses a stub key manager (not the real AZ-318 `PerFlightKeyManager`). In production wiring, both producers share the same FDR client and ordering is naturally guaranteed by the call sequence. The test docstring calls this out explicitly. Action: documented; the integration test in E-BBT will exercise the real AZ-318 manager.
|
||||
|
||||
**F4 — Low (Race window on partial-success batches)**: The uploader marks a tile uploaded immediately after the parent suite acknowledges it inside the per-batch loop. If the safety officer disputes the same tile within the audit window (≤ 1s), the C6 row is already `uploaded`. Spec Risk-5 documents this and defers mitigation to a separate audit task. Action: no code change in this batch.
|
||||
|
||||
## Phase 3 — Tests
|
||||
|
||||
15 unit tests pass for `HttpTileUploader`; 2 for the Protocol conformance check; 3 fixture additions for the AZ-272 schema test. Full unit suite: **1404 passed, 80 skipped, 0 failed** (skips are environment-gated: Docker, CUDA, TensorRT, Tier-2 hardware).
|
||||
|
||||
NFR-perf-throughput: 1000 tiles run under 5 s with the in-process MockTransport — well above the 20 tile/s contract floor (the mock removes the network bottleneck, so this verifies uploader bookkeeping has no O(n²) regression rather than certifying real throughput).
|
||||
|
||||
## Phase 4 — Quality gates
|
||||
|
||||
- `ReadLints` clean across `c11_tile_manager/`, `runtime_root/c11_factory.py`, `fdr_client/records.py`, and the new test files
|
||||
- No `time.sleep` in components (routes via `WallClock.sleep_until_ns`)
|
||||
- No secrets in logs (AC-10 test asserts no `BEGIN PUBLIC KEY` / `Authorization` substring in any captured log record)
|
||||
- No new third-party dependencies (uses existing `httpx` and `cryptography` pins)
|
||||
|
||||
## Verdict
|
||||
|
||||
**PASS_WITH_WARNINGS** — All four findings are Low severity (documentation drift between spec text and implementation, test-double honesty caveat, and a documented Risk-5 race window). No code change required for batch close-out.
|
||||
@@ -8,9 +8,9 @@ status: in_progress
|
||||
sub_step:
|
||||
phase: 3
|
||||
name: compute-next-batch
|
||||
detail: "starting batch 39"
|
||||
detail: ""
|
||||
retry_count: 0
|
||||
cycle: 1
|
||||
tracker: jira
|
||||
last_completed_batch: 38
|
||||
last_completed_batch: 39
|
||||
last_cumulative_review: batches_34-36
|
||||
|
||||
@@ -2,19 +2,27 @@
|
||||
|
||||
Re-exports the Protocol surface (``TileDownloader``, ``TileUploader``,
|
||||
``FlightStateSource``), the upload-side services that have landed
|
||||
(``FlightStateGate`` from AZ-317, ``PerFlightKeyManager`` from
|
||||
AZ-318), the C11 internal DTOs / enums, and the C11 error family.
|
||||
The download-side concrete impl (``HttpTileDownloader``) ships in
|
||||
AZ-316; the upload-side concrete impl (``TileUploader``) ships in
|
||||
AZ-319 — both will be added to ``__all__`` then.
|
||||
(``FlightStateGate`` from AZ-317, ``PerFlightKeyManager`` from AZ-318,
|
||||
``HttpTileUploader`` from AZ-319), the C11 internal DTOs / enums, the
|
||||
C11 error family, and the per-component config block. The download-side
|
||||
concrete impl (``HttpTileDownloader``) ships in AZ-316; it will be added
|
||||
to ``__all__`` then.
|
||||
"""
|
||||
|
||||
from gps_denied_onboard.components.c11_tile_manager._types import (
|
||||
FlightStateSignal,
|
||||
IngestStatus,
|
||||
PerTileStatus,
|
||||
PublicKeyFingerprint,
|
||||
UploadBatchReport,
|
||||
UploadOutcome,
|
||||
UploadRequest,
|
||||
)
|
||||
from gps_denied_onboard.components.c11_tile_manager.config import C11Config
|
||||
from gps_denied_onboard.components.c11_tile_manager.errors import (
|
||||
FlightStateNotOnGroundError,
|
||||
RateLimitedError,
|
||||
SatelliteProviderError,
|
||||
SessionNotActiveError,
|
||||
SignatureRejectedError,
|
||||
TileManagerError,
|
||||
@@ -30,17 +38,34 @@ from gps_denied_onboard.components.c11_tile_manager.interface import (
|
||||
from gps_denied_onboard.components.c11_tile_manager.signing_key import (
|
||||
PerFlightKeyManager,
|
||||
)
|
||||
from gps_denied_onboard.components.c11_tile_manager.tile_uploader import (
|
||||
HttpTileUploader,
|
||||
canonical_payload_bytes,
|
||||
)
|
||||
from gps_denied_onboard.config.schema import register_component_block
|
||||
|
||||
register_component_block("c11_tile_manager", C11Config)
|
||||
|
||||
__all__ = [
|
||||
"C11Config",
|
||||
"FlightStateGate",
|
||||
"FlightStateNotOnGroundError",
|
||||
"FlightStateSignal",
|
||||
"FlightStateSource",
|
||||
"HttpTileUploader",
|
||||
"IngestStatus",
|
||||
"PerFlightKeyManager",
|
||||
"PerTileStatus",
|
||||
"PublicKeyFingerprint",
|
||||
"RateLimitedError",
|
||||
"SatelliteProviderError",
|
||||
"SessionNotActiveError",
|
||||
"SignatureRejectedError",
|
||||
"TileDownloader",
|
||||
"TileManagerError",
|
||||
"TileUploader",
|
||||
"UploadBatchReport",
|
||||
"UploadOutcome",
|
||||
"UploadRequest",
|
||||
"canonical_payload_bytes",
|
||||
]
|
||||
|
||||
@@ -1,10 +1,14 @@
|
||||
"""C11 internal DTOs (AZ-317, AZ-318).
|
||||
"""C11 internal DTOs (AZ-317, AZ-318, AZ-319).
|
||||
|
||||
* :class:`FlightStateSignal` — the five flight-state signals consumed by
|
||||
the upload-side flight-state gate (AZ-317).
|
||||
* :class:`PublicKeyFingerprint` — the per-flight Ed25519 keypair
|
||||
fingerprint envelope returned by :meth:`PerFlightKeyManager.start_session`
|
||||
(AZ-318).
|
||||
* :class:`FlightStateSignal` — five flight-state signals consumed by the
|
||||
upload-side flight-state gate (AZ-317).
|
||||
* :class:`PublicKeyFingerprint` — per-flight Ed25519 keypair fingerprint
|
||||
envelope returned by :meth:`PerFlightKeyManager.start_session` (AZ-318).
|
||||
* :class:`UploadRequest`, :class:`UploadBatchReport`,
|
||||
:class:`PerTileStatus`, :class:`IngestStatus`, :class:`UploadOutcome` —
|
||||
upload-side DTOs and enums consumed and produced by the AZ-319
|
||||
:class:`HttpTileUploader` (contract
|
||||
``_docs/02_document/contracts/c11_tilemanager/tile_uploader.md`` v1.0.0).
|
||||
|
||||
Internal to the component — composition-root code reaches these via the
|
||||
``c11_tile_manager`` package re-exports; consumers outside C11 use the
|
||||
@@ -20,7 +24,12 @@ from uuid import UUID
|
||||
|
||||
__all__ = [
|
||||
"FlightStateSignal",
|
||||
"IngestStatus",
|
||||
"PerTileStatus",
|
||||
"PublicKeyFingerprint",
|
||||
"UploadBatchReport",
|
||||
"UploadOutcome",
|
||||
"UploadRequest",
|
||||
]
|
||||
|
||||
|
||||
@@ -52,3 +61,103 @@ class PublicKeyFingerprint:
|
||||
public_key_pem: bytes
|
||||
fingerprint: str
|
||||
generated_at: datetime
|
||||
|
||||
|
||||
class IngestStatus(str, Enum):
|
||||
"""Per-tile status returned by the parent-suite ingest endpoint.
|
||||
|
||||
``QUEUED`` / ``DUPLICATE`` / ``SUPERSEDED`` are all treated as
|
||||
successful acknowledgement (AC-4); ``REJECTED`` keeps the tile in
|
||||
C6's ``pending_uploads`` for human review (AC-3). The upload-side
|
||||
flow only writes ``mark_uploaded`` for the three successful states.
|
||||
"""
|
||||
|
||||
QUEUED = "queued"
|
||||
REJECTED = "rejected"
|
||||
DUPLICATE = "duplicate"
|
||||
SUPERSEDED = "superseded"
|
||||
|
||||
|
||||
class UploadOutcome(str, Enum):
|
||||
"""Aggregate outcome of one :meth:`TileUploader.upload_pending_tiles` call.
|
||||
|
||||
Mirrors contract Shape § ``UploadBatchReport.outcome``:
|
||||
|
||||
* ``SUCCESS`` — every per-tile status is ``QUEUED`` /
|
||||
``DUPLICATE`` / ``SUPERSEDED``.
|
||||
* ``PARTIAL`` — some tiles were ``REJECTED`` while others were
|
||||
acknowledged; the caller may re-invoke for the rejected set.
|
||||
* ``FAILURE`` — the flight-state gate blocked or zero tiles could
|
||||
be POSTed (TLS / 401 / 403 / persistent 5xx surface as raised
|
||||
:class:`SatelliteProviderError`, NOT as ``FAILURE`` in a returned
|
||||
report).
|
||||
"""
|
||||
|
||||
SUCCESS = "success"
|
||||
PARTIAL = "partial"
|
||||
FAILURE = "failure"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class UploadRequest:
|
||||
"""Inputs to :meth:`TileUploader.upload_pending_tiles`.
|
||||
|
||||
``flight_id`` is optional: ``None`` means "every pending tile across
|
||||
every flight" (typical post-landing operator workflow);
|
||||
``UUID(...)`` restricts the batch to one flight (used by C12 when
|
||||
re-driving a single flight's pending journal).
|
||||
"""
|
||||
|
||||
batch_size: int
|
||||
satellite_provider_url: str
|
||||
flight_id: UUID | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if not 1 <= self.batch_size <= 200:
|
||||
raise ValueError(
|
||||
"UploadRequest.batch_size must be in [1, 200]; "
|
||||
f"got {self.batch_size}"
|
||||
)
|
||||
if not self.satellite_provider_url:
|
||||
raise ValueError(
|
||||
"UploadRequest.satellite_provider_url must be non-empty"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PerTileStatus:
|
||||
"""Per-tile result attached to :class:`UploadBatchReport.per_tile_status`.
|
||||
|
||||
``rejection_reason`` is only populated when ``status == REJECTED``;
|
||||
it carries the parent-suite's free-text explanation (e.g.
|
||||
``"invalid signature"``, ``"capture_timestamp out of range"``).
|
||||
The C11 uploader matches the ``"signature"`` substring to decide
|
||||
whether to also call
|
||||
:meth:`PerFlightKeyManager.record_signature_rejection`.
|
||||
"""
|
||||
|
||||
tile_id: str
|
||||
status: IngestStatus
|
||||
rejection_reason: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class UploadBatchReport:
|
||||
"""Aggregate report returned by :meth:`TileUploader.upload_pending_tiles`.
|
||||
|
||||
``batch_uuid`` is the LAST successful batch's identifier from the
|
||||
parent-suite ingest endpoint (or a fresh ``uuid4()`` when no batch
|
||||
POSTed; documented). The full per-batch ID stream is captured in
|
||||
the FDR ``c11.upload.batch.complete`` records.
|
||||
|
||||
``public_key_fingerprint`` carries the 16-hex AZ-318 fingerprint so
|
||||
the operator can cross-reference against the safety-officer's
|
||||
pre-enrolment table without re-reading the FDR.
|
||||
"""
|
||||
|
||||
batch_uuid: UUID
|
||||
per_tile_status: tuple[PerTileStatus, ...]
|
||||
retry_count: int
|
||||
next_retry_at_s: int | None
|
||||
outcome: UploadOutcome
|
||||
public_key_fingerprint: str
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
"""C11 TileManager config block (AZ-319).
|
||||
|
||||
Registered into ``config.components['c11_tile_manager']`` by the
|
||||
package ``__init__.py``. The composition-root factory
|
||||
:func:`gps_denied_onboard.runtime_root.c11_factory.build_tile_uploader`
|
||||
reads this block to drive the upload path's HTTP behaviour and to
|
||||
identify the producing companion against the parent suite's voting
|
||||
layer.
|
||||
|
||||
The four fields below match the AZ-319 task spec § ``Outcome`` —
|
||||
``config.c11.satellite_provider_ingest_url``,
|
||||
``config.c11.upload_batch_size``, ``config.c11.upload_http_timeout_s``,
|
||||
``config.c11.companion_id``. The ``upload_max_retry_after_s`` cap is
|
||||
the Risk-3 ceiling on cumulative ``Retry-After`` budget for 429
|
||||
responses (see :class:`RateLimitedError`).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from gps_denied_onboard.config.schema import ConfigError
|
||||
|
||||
__all__ = ["C11Config"]
|
||||
|
||||
|
||||
_DEFAULT_BATCH_SIZE: int = 25
|
||||
_DEFAULT_HTTP_TIMEOUT_S: float = 30.0
|
||||
_DEFAULT_MAX_RETRY_AFTER_S: int = 600
|
||||
_MAX_BATCH_SIZE: int = 200
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class C11Config:
|
||||
"""Per-component config for C11 tile manager (upload path).
|
||||
|
||||
``satellite_provider_ingest_url`` is the parent-suite ingest base
|
||||
URL (e.g. ``https://satellite-provider.example.com``); the
|
||||
uploader appends ``/api/satellite/tiles/ingest`` to it. Defaulted
|
||||
to empty so unit tests / replay runs that do not exercise the
|
||||
upload path stay no-op; production configuration MUST set this
|
||||
via YAML / env override or :class:`HttpTileUploader` raises
|
||||
:class:`SatelliteProviderError` on the first attempt.
|
||||
|
||||
``companion_id`` is the stable per-companion identifier the
|
||||
parent suite's voting layer uses to attribute uploads to one
|
||||
physical airframe. Defaulted to empty so test runs without a
|
||||
paired companion stay valid; the factory raises ``ConfigError``
|
||||
when the empty default is used in operator / production wiring.
|
||||
"""
|
||||
|
||||
satellite_provider_ingest_url: str = ""
|
||||
upload_batch_size: int = _DEFAULT_BATCH_SIZE
|
||||
upload_http_timeout_s: float = _DEFAULT_HTTP_TIMEOUT_S
|
||||
upload_max_retry_after_s: int = _DEFAULT_MAX_RETRY_AFTER_S
|
||||
companion_id: str = ""
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if not 1 <= self.upload_batch_size <= _MAX_BATCH_SIZE:
|
||||
raise ConfigError(
|
||||
"C11Config.upload_batch_size must be in "
|
||||
f"[1, {_MAX_BATCH_SIZE}]; got {self.upload_batch_size}"
|
||||
)
|
||||
if self.upload_http_timeout_s <= 0:
|
||||
raise ConfigError(
|
||||
"C11Config.upload_http_timeout_s must be > 0; "
|
||||
f"got {self.upload_http_timeout_s}"
|
||||
)
|
||||
if self.upload_max_retry_after_s <= 0:
|
||||
raise ConfigError(
|
||||
"C11Config.upload_max_retry_after_s must be > 0; "
|
||||
f"got {self.upload_max_retry_after_s}"
|
||||
)
|
||||
@@ -1,4 +1,4 @@
|
||||
"""C11 TileManager error family (AZ-317, AZ-318, plus reserved AZ-319 envelope).
|
||||
"""C11 TileManager error family (AZ-317, AZ-318, AZ-319).
|
||||
|
||||
Rooted at :class:`TileManagerError`. The parent is declared here (rather
|
||||
than alongside the AZ-316 ``TileDownloader``) so the upload-side tasks
|
||||
@@ -11,9 +11,14 @@ subclasses without re-declaring the parent.
|
||||
``ON_GROUND`` at upload entry.
|
||||
* :class:`SessionNotActiveError` (AZ-318) — :meth:`PerFlightKeyManager.sign`
|
||||
/ :meth:`record_signature_rejection` called outside an active session.
|
||||
* :class:`SignatureRejectedError` (AZ-318 envelope) — defined here for
|
||||
the upload-side error family; raised by ``TileUploader`` (separate
|
||||
task) after parsing the ``satellite-provider`` ingest response.
|
||||
* :class:`SignatureRejectedError` (AZ-318/AZ-319 envelope) — surfaced
|
||||
by the AZ-319 :class:`HttpTileUploader` after parsing a
|
||||
``REJECTED`` per-tile response whose ``rejection_reason`` mentions
|
||||
the signature.
|
||||
* :class:`SatelliteProviderError` (AZ-319) — TLS / 401 / 403 fail-fast
|
||||
AND persistent-5xx fail-after-retries surface for the upload path.
|
||||
* :class:`RateLimitedError` (AZ-319) — 429 with persistent
|
||||
``Retry-After`` budget exhaustion.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -28,6 +33,8 @@ if TYPE_CHECKING:
|
||||
|
||||
__all__ = [
|
||||
"FlightStateNotOnGroundError",
|
||||
"RateLimitedError",
|
||||
"SatelliteProviderError",
|
||||
"SessionNotActiveError",
|
||||
"SignatureRejectedError",
|
||||
"TileManagerError",
|
||||
@@ -77,3 +84,27 @@ class SignatureRejectedError(TileManagerError):
|
||||
to surface the FDR + ERROR log envelope per AZ-318 AC-8 before
|
||||
re-raising this exception to the operator-tooling layer.
|
||||
"""
|
||||
|
||||
|
||||
class SatelliteProviderError(TileManagerError):
|
||||
"""``satellite-provider`` ingest endpoint failed terminally.
|
||||
|
||||
Surface for the upload path's fail-fast (TLS handshake failure,
|
||||
401 / 403 auth rejection on the FIRST attempt) and
|
||||
fail-after-retries (5 consecutive 5xx) classes per AZ-319 AC-9 /
|
||||
AC-10. The original :class:`httpx.HTTPError` is preserved on
|
||||
``__cause__``; the message names the HTTP status when known.
|
||||
"""
|
||||
|
||||
|
||||
class RateLimitedError(TileManagerError):
|
||||
"""``satellite-provider`` ingest endpoint rate-limited the upload.
|
||||
|
||||
Raised when the parent suite returns 429 and the cumulative
|
||||
``Retry-After`` budget exceeds
|
||||
:attr:`C11Config.upload_max_retry_after_s`. The
|
||||
AZ-319 :class:`HttpTileUploader` honours the FIRST 429's
|
||||
``Retry-After`` (sleep + retry) but escalates to this error after
|
||||
the configured budget so the operator can surface the rate-limit
|
||||
state explicitly.
|
||||
"""
|
||||
|
||||
@@ -4,7 +4,10 @@ Operator-side ONLY — excluded from airborne via CMake (`BUILD_C11_TILE_MANAGER
|
||||
See `_docs/02_document/components/12_c11_tilemanager/`.
|
||||
|
||||
* :class:`TileDownloader` — pre-flight download path (AZ-316, pending).
|
||||
* :class:`TileUploader` — post-landing upload path (AZ-319, pending).
|
||||
* :class:`TileUploader` — post-landing upload path (AZ-319) — the
|
||||
authoritative shape lives in
|
||||
``_docs/02_document/contracts/c11_tilemanager/tile_uploader.md``
|
||||
v1.0.0 and is mirrored 1:1 here.
|
||||
* :class:`FlightStateSource` — thin C11-facing adapter the upload-side
|
||||
flight-state gate (AZ-317) calls to read "what is the FC saying right
|
||||
now?". A concrete impl ships with E-C8 (subscribes to the FC adapter's
|
||||
@@ -15,13 +18,16 @@ See `_docs/02_document/components/12_c11_tilemanager/`.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
from collections.abc import Iterable, Sequence
|
||||
from pathlib import Path
|
||||
from typing import Protocol, runtime_checkable
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
from uuid import UUID
|
||||
|
||||
from gps_denied_onboard._types.tile import TileRecord
|
||||
from gps_denied_onboard.components.c11_tile_manager._types import (
|
||||
FlightStateSignal,
|
||||
UploadBatchReport,
|
||||
UploadRequest,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
@@ -39,10 +45,25 @@ class TileDownloader(Protocol):
|
||||
) -> Iterable[TileRecord]: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class TileUploader(Protocol):
|
||||
"""Post-landing batch upload to the `satellite-provider` ingest endpoint (D-PROJ-2)."""
|
||||
"""Post-landing batch upload to ``satellite-provider`` ingest (D-PROJ-2).
|
||||
|
||||
def upload(self, tiles: Iterable[TileRecord], flight_id: str) -> None: ...
|
||||
See ``_docs/02_document/contracts/c11_tilemanager/tile_uploader.md``
|
||||
v1.0.0 for invariants I-1 .. I-8 and the per-method error matrix.
|
||||
The :meth:`enumerate_pending_tiles` return type is the consumer-
|
||||
side structural metadata shape (mirrors c6's ``TileMetadata``;
|
||||
declared as ``Sequence[Any]`` here to keep C11 free of cross-
|
||||
component imports per AZ-507).
|
||||
"""
|
||||
|
||||
def upload_pending_tiles(self, request: UploadRequest) -> UploadBatchReport: ...
|
||||
|
||||
def enumerate_pending_tiles(
|
||||
self, flight_id: UUID | None = None
|
||||
) -> Sequence[Any]: ...
|
||||
|
||||
def confirm_flight_state(self) -> FlightStateSignal: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
|
||||
@@ -0,0 +1,754 @@
|
||||
"""C11 ``HttpTileUploader`` (AZ-319) — concrete :class:`TileUploader`.
|
||||
|
||||
Operator-side post-landing upload path. Reads pending mid-flight tiles
|
||||
from C6 (``source = onboard_ingest``, ``uploaded_at IS NULL``), packages
|
||||
each per the D-PROJ-2 multipart contract sketch, signs with the per-flight
|
||||
ephemeral key (AZ-318), POSTs to ``satellite-provider``'s ingest
|
||||
endpoint, and marks acknowledged tiles uploaded. Gates on ``ON_GROUND``
|
||||
(AZ-317) before any C6 read or network egress; zeroes the signing key
|
||||
in a try/finally regardless of outcome.
|
||||
|
||||
Architecture
|
||||
------------
|
||||
The c6 storage surface is reached through structural :class:`Protocol`
|
||||
cuts (:class:`_PendingMetadataReader`, :class:`_TileBytesReader`,
|
||||
:class:`_TilePixelHandleLike`) defined in this module — never via a
|
||||
direct ``from gps_denied_onboard.components.c6_tile_cache import`` (the
|
||||
AZ-507 cross-component rule + the AZ-270 lint
|
||||
``test_az270_compose_root.test_ac6`` enforce this on every
|
||||
``components/**/*.py`` file). The composition root
|
||||
(``runtime_root.c11_factory.build_tile_uploader``) is the single layer
|
||||
that may bind concrete c6 implementations into the constructor.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import struct
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from email.utils import parsedate_to_datetime
|
||||
from types import TracebackType
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
import httpx
|
||||
|
||||
from gps_denied_onboard.components.c11_tile_manager._types import (
|
||||
IngestStatus,
|
||||
PerTileStatus,
|
||||
UploadBatchReport,
|
||||
UploadOutcome,
|
||||
UploadRequest,
|
||||
)
|
||||
from gps_denied_onboard.components.c11_tile_manager.config import C11Config
|
||||
from gps_denied_onboard.components.c11_tile_manager.errors import (
|
||||
RateLimitedError,
|
||||
SatelliteProviderError,
|
||||
SignatureRejectedError,
|
||||
)
|
||||
from gps_denied_onboard.components.c11_tile_manager.flight_state_gate import (
|
||||
FlightStateGate,
|
||||
)
|
||||
from gps_denied_onboard.components.c11_tile_manager.signing_key import (
|
||||
PerFlightKeyManager,
|
||||
)
|
||||
from gps_denied_onboard.fdr_client import (
|
||||
CURRENT_SCHEMA_VERSION,
|
||||
FdrClient,
|
||||
FdrRecord,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"HttpTileUploader",
|
||||
"canonical_payload_bytes",
|
||||
]
|
||||
|
||||
|
||||
_INGEST_PATH = "/api/satellite/tiles/ingest"
|
||||
_FDR_KIND_TILE_QUEUED = "c11.upload.tile.queued"
|
||||
_FDR_KIND_TILE_REJECTED = "c11.upload.tile.rejected"
|
||||
_FDR_KIND_BATCH_COMPLETE = "c11.upload.batch.complete"
|
||||
_LOG_KIND_SESSION_START = "c11.upload.session.started"
|
||||
_LOG_KIND_SESSION_END = "c11.upload.session.completed"
|
||||
_LOG_KIND_RETRY = "c11.upload.batch.retry"
|
||||
_LOG_KIND_PROVIDER_FAIL = "c11.upload.provider.failed"
|
||||
_COMPONENT = "c11_tile_manager.tile_uploader"
|
||||
|
||||
_BACKOFF_SCHEDULE_S: tuple[float, ...] = (1.0, 2.0, 4.0, 8.0)
|
||||
_REJECTION_SIGNATURE_MARKER = "signature"
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Consumer-side cuts over c6 (AZ-507): never imported across components.
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class _TilePixelHandleLike(Protocol):
|
||||
"""Structural cut of c6's ``TilePixelHandle`` context manager.
|
||||
|
||||
The uploader only enters the context to read JPEG bytes for the
|
||||
multipart payload; it never inspects the underlying mmap path or
|
||||
holds the handle past the ``with`` block.
|
||||
"""
|
||||
|
||||
def __enter__(self) -> memoryview: ...
|
||||
|
||||
def __exit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: TracebackType | None,
|
||||
) -> None: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class _TileBytesReader(Protocol):
|
||||
"""Structural cut of c6's ``TileStore.read_tile_pixels``.
|
||||
|
||||
The uploader takes ``Any`` for ``tile_id`` to avoid a c6 import; the
|
||||
composition root binds the concrete c6 ``TileStore`` whose
|
||||
``read_tile_pixels`` accepts the c6 ``TileId`` shape.
|
||||
"""
|
||||
|
||||
def read_tile_pixels(self, tile_id: Any) -> _TilePixelHandleLike: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class _PendingMetadataReader(Protocol):
|
||||
"""Structural cut of c6's ``TileMetadataStore`` upload-side surface.
|
||||
|
||||
Three methods, narrow on purpose: enumerate pending,
|
||||
forward-stamp ``uploaded_at`` per tile, and look up metadata by
|
||||
id (used when ``request.flight_id`` filters the pending set).
|
||||
"""
|
||||
|
||||
def pending_uploads(self) -> list[Any]: ...
|
||||
|
||||
def mark_uploaded(self, tile_id: Any, uploaded_at: datetime) -> None: ...
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Canonical payload bytes (AC-13)
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def canonical_payload_bytes(
|
||||
tile_blob: bytes, tile: Any, request: UploadRequest, companion_id: str
|
||||
) -> bytes:
|
||||
"""Deterministic SHA-256 over the multipart input set (AC-13).
|
||||
|
||||
Concatenation order is FROZEN: ``tile_blob`` || zoom_level (uint8) ||
|
||||
latitude (float64 LE) || longitude (float64 LE) ||
|
||||
capture_timestamp_ns (int64 LE) || flight_id (utf-8) ||
|
||||
companion_id (utf-8) || quality_metadata_json (utf-8 with sorted
|
||||
keys, no whitespace). Re-ordering OR re-formatting any field is a
|
||||
breaking change to the parent suite's signature verification — bump
|
||||
the contract major version per the contract's Versioning Rules.
|
||||
"""
|
||||
|
||||
flight_id_bytes = (tile.flight_id or "").encode("utf-8")
|
||||
companion_bytes = companion_id.encode("utf-8")
|
||||
quality_json = _serialise_quality_metadata(getattr(tile, "quality_metadata", None))
|
||||
capture_ts_ns = _datetime_to_unix_ns(tile.capture_timestamp)
|
||||
|
||||
parts = [
|
||||
bytes(tile_blob),
|
||||
struct.pack("<B", int(tile.tile_id.zoom_level)),
|
||||
struct.pack("<d", float(tile.tile_id.lat)),
|
||||
struct.pack("<d", float(tile.tile_id.lon)),
|
||||
struct.pack("<q", capture_ts_ns),
|
||||
flight_id_bytes,
|
||||
companion_bytes,
|
||||
quality_json.encode("utf-8"),
|
||||
]
|
||||
return hashlib.sha256(b"".join(parts)).digest()
|
||||
|
||||
|
||||
def _serialise_quality_metadata(quality: Any) -> str:
|
||||
if quality is None:
|
||||
return "null"
|
||||
payload = {
|
||||
"estimator_label": quality.estimator_label,
|
||||
"covariance_2x2": [list(row) for row in quality.covariance_2x2],
|
||||
"last_anchor_age_ms": int(quality.last_anchor_age_ms),
|
||||
"mre_px": float(quality.mre_px),
|
||||
"imu_bias_norm": float(quality.imu_bias_norm),
|
||||
}
|
||||
return json.dumps(payload, sort_keys=True, separators=(",", ":"))
|
||||
|
||||
|
||||
def _datetime_to_unix_ns(ts: datetime) -> int:
|
||||
if ts.tzinfo is None:
|
||||
ts = ts.replace(tzinfo=timezone.utc)
|
||||
seconds = int(ts.timestamp())
|
||||
micros = ts.microsecond
|
||||
return seconds * 1_000_000_000 + micros * 1_000
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Retry-After parsing (AC-8)
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def _parse_retry_after(header_value: str | None, max_s: int) -> int:
|
||||
"""Return non-negative seconds to sleep, capped at ``max_s``.
|
||||
|
||||
Honours BOTH RFC 7231 forms: integer seconds and HTTP-date. Returns
|
||||
0 when the header is missing or unparseable (the uploader still
|
||||
backs off via its own schedule when the value is unusable).
|
||||
"""
|
||||
|
||||
if header_value is None:
|
||||
return 0
|
||||
raw = header_value.strip()
|
||||
if not raw:
|
||||
return 0
|
||||
if raw.isdigit():
|
||||
return min(int(raw), max_s)
|
||||
try:
|
||||
retry_at = parsedate_to_datetime(raw)
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
now = datetime.now(timezone.utc)
|
||||
if retry_at.tzinfo is None:
|
||||
retry_at = retry_at.replace(tzinfo=timezone.utc)
|
||||
delta = (retry_at - now).total_seconds()
|
||||
return max(0, min(int(delta), max_s))
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Internal session-state container
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class _SessionState:
|
||||
"""Mutable bookkeeping for one ``upload_pending_tiles`` call."""
|
||||
|
||||
flight_id_for_session: UUID
|
||||
public_key_fingerprint: str
|
||||
per_tile_results: list[PerTileStatus]
|
||||
last_batch_uuid: UUID | None
|
||||
retry_count: int
|
||||
next_retry_at_s: int | None
|
||||
rate_limit_budget_used_s: int
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Concrete uploader
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class HttpTileUploader:
|
||||
"""Concrete :class:`TileUploader` against ``satellite-provider``'s ingest endpoint.
|
||||
|
||||
All cross-component dependencies (``flight_state_gate``,
|
||||
``key_manager``, ``tile_store``, ``tile_metadata_store``) are
|
||||
constructor-injected via Protocol cuts. The ``http_client`` is an
|
||||
:class:`httpx.Client` the caller owns; ``HttpTileUploader`` does
|
||||
NOT close it — production wiring uses a long-lived client per
|
||||
process; tests inject ``httpx.Client(transport=httpx.MockTransport)``
|
||||
for deterministic responses.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
http_client: httpx.Client,
|
||||
tile_store: _TileBytesReader,
|
||||
tile_metadata_store: _PendingMetadataReader,
|
||||
flight_state_gate: FlightStateGate,
|
||||
key_manager: PerFlightKeyManager,
|
||||
fdr_client: FdrClient,
|
||||
logger: logging.Logger,
|
||||
config: C11Config,
|
||||
sleep: Any = None,
|
||||
) -> None:
|
||||
self._http_client = http_client
|
||||
self._tile_store = tile_store
|
||||
self._metadata_store = tile_metadata_store
|
||||
self._gate = flight_state_gate
|
||||
self._key_manager = key_manager
|
||||
self._fdr = fdr_client
|
||||
self._logger = logger
|
||||
self._config = config
|
||||
self._sleep = sleep if sleep is not None else _default_sleep
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public Protocol surface
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def upload_pending_tiles(self, request: UploadRequest) -> UploadBatchReport:
|
||||
"""Gate → start_session → enumerate → batch loop → finally end_session.
|
||||
|
||||
Order is FROZEN per Reliability constraint in the task spec —
|
||||
re-ordering is a High Reliability finding at code-review time
|
||||
because it breaks I-1 (gate before any read / network) or I-4
|
||||
(zeroisation guarantee on every exit path).
|
||||
"""
|
||||
|
||||
self._gate.confirm_on_ground()
|
||||
flight_id_for_session = request.flight_id or uuid4()
|
||||
fingerprint = self._key_manager.start_session(flight_id_for_session)
|
||||
state = _SessionState(
|
||||
flight_id_for_session=flight_id_for_session,
|
||||
public_key_fingerprint=fingerprint.fingerprint,
|
||||
per_tile_results=[],
|
||||
last_batch_uuid=None,
|
||||
retry_count=0,
|
||||
next_retry_at_s=None,
|
||||
rate_limit_budget_used_s=0,
|
||||
)
|
||||
|
||||
self._logger.info(
|
||||
"Per-flight upload session started",
|
||||
extra={
|
||||
"component": _COMPONENT,
|
||||
"kind": _LOG_KIND_SESSION_START,
|
||||
"kv": {
|
||||
"flight_id": str(flight_id_for_session),
|
||||
"fingerprint": fingerprint.fingerprint,
|
||||
"request_flight_filter": (
|
||||
None if request.flight_id is None else str(request.flight_id)
|
||||
),
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
outcome: UploadOutcome
|
||||
try:
|
||||
pending = self._enumerate(request)
|
||||
if not pending:
|
||||
outcome = UploadOutcome.SUCCESS
|
||||
state.last_batch_uuid = uuid4()
|
||||
else:
|
||||
outcome = self._upload_batches(request, pending, state)
|
||||
except (SatelliteProviderError, RateLimitedError, SignatureRejectedError):
|
||||
outcome = UploadOutcome.FAILURE
|
||||
raise
|
||||
except Exception:
|
||||
outcome = UploadOutcome.FAILURE
|
||||
raise
|
||||
finally:
|
||||
self._key_manager.end_session()
|
||||
self._emit_batch_complete(state, outcome)
|
||||
self._logger.info(
|
||||
"Per-flight upload session completed",
|
||||
extra={
|
||||
"component": _COMPONENT,
|
||||
"kind": _LOG_KIND_SESSION_END,
|
||||
"kv": {
|
||||
"flight_id": str(state.flight_id_for_session),
|
||||
"fingerprint": state.public_key_fingerprint,
|
||||
"outcome": outcome.value,
|
||||
"total_attempted": len(state.per_tile_results),
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
assert state.last_batch_uuid is not None
|
||||
return UploadBatchReport(
|
||||
batch_uuid=state.last_batch_uuid,
|
||||
per_tile_status=tuple(state.per_tile_results),
|
||||
retry_count=state.retry_count,
|
||||
next_retry_at_s=state.next_retry_at_s,
|
||||
outcome=outcome,
|
||||
public_key_fingerprint=state.public_key_fingerprint,
|
||||
)
|
||||
|
||||
def enumerate_pending_tiles(
|
||||
self, flight_id: UUID | None = None
|
||||
) -> list[Any]:
|
||||
"""Read-only enumeration; does NOT call the gate (per contract)."""
|
||||
|
||||
return self._filter_by_flight(self._metadata_store.pending_uploads(), flight_id)
|
||||
|
||||
def confirm_flight_state(self) -> Any:
|
||||
"""Pass-through to :meth:`FlightStateGate.confirm_on_ground`."""
|
||||
|
||||
return self._gate.confirm_on_ground()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _enumerate(self, request: UploadRequest) -> list[Any]:
|
||||
all_pending = self._metadata_store.pending_uploads()
|
||||
return self._filter_by_flight(all_pending, request.flight_id)
|
||||
|
||||
@staticmethod
|
||||
def _filter_by_flight(tiles: list[Any], flight_id: UUID | None) -> list[Any]:
|
||||
if flight_id is None:
|
||||
return list(tiles)
|
||||
target = str(flight_id)
|
||||
return [t for t in tiles if (t.flight_id or "") == target]
|
||||
|
||||
def _upload_batches(
|
||||
self,
|
||||
request: UploadRequest,
|
||||
pending: list[Any],
|
||||
state: _SessionState,
|
||||
) -> UploadOutcome:
|
||||
any_rejected = False
|
||||
for batch_start in range(0, len(pending), request.batch_size):
|
||||
batch = pending[batch_start : batch_start + request.batch_size]
|
||||
response = self._post_batch_with_retry(request, batch, state)
|
||||
batch_uuid, parsed = self._process_response(response, batch, state)
|
||||
state.last_batch_uuid = batch_uuid
|
||||
for tile_id, status in parsed:
|
||||
if status.status == IngestStatus.REJECTED:
|
||||
any_rejected = True
|
||||
else:
|
||||
self._metadata_store.mark_uploaded(
|
||||
tile_id, datetime.now(timezone.utc)
|
||||
)
|
||||
state.per_tile_results.append(status)
|
||||
|
||||
if any_rejected:
|
||||
return UploadOutcome.PARTIAL
|
||||
return UploadOutcome.SUCCESS
|
||||
|
||||
def _post_batch_with_retry(
|
||||
self,
|
||||
request: UploadRequest,
|
||||
batch: list[Any],
|
||||
state: _SessionState,
|
||||
) -> httpx.Response:
|
||||
"""POST one batch, honouring 429 Retry-After and exponential 5xx backoff."""
|
||||
|
||||
ingest_url = request.satellite_provider_url.rstrip("/") + _INGEST_PATH
|
||||
attempt = 0
|
||||
last_error: str | None = None
|
||||
while True:
|
||||
attempt += 1
|
||||
files, data = self._build_multipart(batch, request)
|
||||
try:
|
||||
response = self._http_client.post(
|
||||
ingest_url,
|
||||
files=files,
|
||||
data=data,
|
||||
timeout=self._config.upload_http_timeout_s,
|
||||
)
|
||||
except (
|
||||
httpx.ConnectError,
|
||||
httpx.ConnectTimeout,
|
||||
httpx.ReadTimeout,
|
||||
httpx.WriteError,
|
||||
httpx.RemoteProtocolError,
|
||||
) as exc:
|
||||
last_error = f"transport:{type(exc).__name__}:{exc}"
|
||||
if attempt >= len(_BACKOFF_SCHEDULE_S):
|
||||
self._log_provider_failure("connection_error", None, last_error)
|
||||
raise SatelliteProviderError(
|
||||
"satellite-provider unreachable after "
|
||||
f"{attempt} attempts: {last_error}"
|
||||
) from exc
|
||||
self._sleep_with_log(_BACKOFF_SCHEDULE_S[attempt - 1], last_error, state)
|
||||
continue
|
||||
|
||||
if response.status_code in (401, 403):
|
||||
self._log_provider_failure(
|
||||
"auth_failed", response.status_code, "fail-fast"
|
||||
)
|
||||
raise SatelliteProviderError(
|
||||
f"satellite-provider rejected auth (http_status="
|
||||
f"{response.status_code}); fail-fast"
|
||||
)
|
||||
|
||||
if response.status_code == 429:
|
||||
wait_s = _parse_retry_after(
|
||||
response.headers.get("Retry-After"),
|
||||
self._config.upload_max_retry_after_s
|
||||
- state.rate_limit_budget_used_s,
|
||||
)
|
||||
state.rate_limit_budget_used_s += wait_s
|
||||
state.next_retry_at_s = wait_s
|
||||
if wait_s <= 0 or state.rate_limit_budget_used_s >= self._config.upload_max_retry_after_s:
|
||||
self._log_provider_failure(
|
||||
"rate_limited", 429, "Retry-After budget exhausted"
|
||||
)
|
||||
raise RateLimitedError(
|
||||
"satellite-provider rate-limited the upload; cumulative "
|
||||
f"Retry-After budget {state.rate_limit_budget_used_s}s "
|
||||
f"exceeds cap {self._config.upload_max_retry_after_s}s"
|
||||
)
|
||||
self._sleep_with_log(wait_s, f"http_429_retry_after={wait_s}", state)
|
||||
continue
|
||||
|
||||
if response.status_code >= 500:
|
||||
last_error = f"http_status={response.status_code}"
|
||||
if attempt >= len(_BACKOFF_SCHEDULE_S):
|
||||
self._log_provider_failure(
|
||||
"persistent_5xx", response.status_code, last_error
|
||||
)
|
||||
raise SatelliteProviderError(
|
||||
f"satellite-provider returned {response.status_code} "
|
||||
f"after {attempt} attempts"
|
||||
)
|
||||
self._sleep_with_log(_BACKOFF_SCHEDULE_S[attempt - 1], last_error, state)
|
||||
continue
|
||||
|
||||
if response.status_code != 202:
|
||||
self._log_provider_failure(
|
||||
"unexpected_status", response.status_code, "non-202"
|
||||
)
|
||||
raise SatelliteProviderError(
|
||||
f"satellite-provider returned unexpected status "
|
||||
f"{response.status_code} (expected 202)"
|
||||
)
|
||||
return response
|
||||
|
||||
def _build_multipart(
|
||||
self, batch: list[Any], request: UploadRequest
|
||||
) -> tuple[list[tuple[str, tuple[str, bytes, str]]], dict[str, str]]:
|
||||
files: list[tuple[str, tuple[str, bytes, str]]] = []
|
||||
per_tile_meta: list[dict[str, Any]] = []
|
||||
for tile in batch:
|
||||
with self._tile_store.read_tile_pixels(tile.tile_id) as view:
|
||||
tile_bytes = bytes(view)
|
||||
canonical = canonical_payload_bytes(
|
||||
tile_bytes, tile, request, self._config.companion_id
|
||||
)
|
||||
signature = self._key_manager.sign(canonical)
|
||||
tile_id_str = _tile_id_to_str(tile.tile_id)
|
||||
files.append(
|
||||
(
|
||||
"tile_blob",
|
||||
(f"{tile_id_str}.jpg", tile_bytes, "image/jpeg"),
|
||||
)
|
||||
)
|
||||
per_tile_meta.append(
|
||||
{
|
||||
"tile_id": tile_id_str,
|
||||
"zoomLevel": int(tile.tile_id.zoom_level),
|
||||
"latitude": float(tile.tile_id.lat),
|
||||
"longitude": float(tile.tile_id.lon),
|
||||
"tile_size_meters": float(tile.tile_size_meters),
|
||||
"tile_size_pixels": int(tile.tile_size_pixels),
|
||||
"capture_timestamp": tile.capture_timestamp.isoformat(),
|
||||
"flight_id": tile.flight_id or "",
|
||||
"companion_id": self._config.companion_id,
|
||||
"quality_metadata": _serialise_quality_metadata(
|
||||
getattr(tile, "quality_metadata", None)
|
||||
),
|
||||
"signature": signature.hex(),
|
||||
}
|
||||
)
|
||||
data = {"tiles_metadata": json.dumps(per_tile_meta, sort_keys=True)}
|
||||
return files, data
|
||||
|
||||
def _process_response(
|
||||
self,
|
||||
response: httpx.Response,
|
||||
batch: list[Any],
|
||||
state: _SessionState,
|
||||
) -> tuple[UUID, list[tuple[Any, PerTileStatus]]]:
|
||||
"""Parse the parent suite's 202 body and emit per-tile FDR records."""
|
||||
|
||||
try:
|
||||
body = response.json()
|
||||
except ValueError as exc:
|
||||
self._log_provider_failure(
|
||||
"schema_not_json", response.status_code, str(exc)
|
||||
)
|
||||
raise SatelliteProviderError(
|
||||
"satellite-provider returned non-JSON body for batch POST"
|
||||
) from exc
|
||||
|
||||
try:
|
||||
batch_uuid = UUID(str(body["batch_uuid"]))
|
||||
per_tile_raw = body["per_tile_status"]
|
||||
except (KeyError, ValueError) as exc:
|
||||
self._log_provider_failure(
|
||||
"schema_missing_fields", response.status_code, str(exc)
|
||||
)
|
||||
raise SatelliteProviderError(
|
||||
"satellite-provider response missing batch_uuid or per_tile_status"
|
||||
) from exc
|
||||
|
||||
batch_index = {_tile_id_to_str(t.tile_id): t for t in batch}
|
||||
results: list[tuple[Any, PerTileStatus]] = []
|
||||
for entry in per_tile_raw:
|
||||
tile_id_str = str(entry["tile_id"])
|
||||
try:
|
||||
ingest_status = IngestStatus(entry["status"])
|
||||
except (KeyError, ValueError) as exc:
|
||||
self._log_provider_failure(
|
||||
"schema_unknown_status",
|
||||
response.status_code,
|
||||
f"tile_id={tile_id_str}:{exc}",
|
||||
)
|
||||
raise SatelliteProviderError(
|
||||
f"satellite-provider returned unknown status for "
|
||||
f"tile_id={tile_id_str}: {entry.get('status')!r}"
|
||||
) from exc
|
||||
reason = entry.get("rejection_reason")
|
||||
tile = batch_index.get(tile_id_str)
|
||||
if tile is None:
|
||||
self._log_provider_failure(
|
||||
"schema_unknown_tile",
|
||||
response.status_code,
|
||||
f"tile_id={tile_id_str} not in batch",
|
||||
)
|
||||
raise SatelliteProviderError(
|
||||
f"satellite-provider acknowledged tile_id={tile_id_str} "
|
||||
"that was not in the POSTed batch"
|
||||
)
|
||||
status_dto = PerTileStatus(
|
||||
tile_id=tile_id_str,
|
||||
status=ingest_status,
|
||||
rejection_reason=reason,
|
||||
)
|
||||
if ingest_status == IngestStatus.REJECTED:
|
||||
self._handle_rejection(tile, tile_id_str, reason, state, batch_uuid)
|
||||
else:
|
||||
self._fdr.enqueue(
|
||||
FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_now(),
|
||||
producer_id=self._fdr.producer_id,
|
||||
kind=_FDR_KIND_TILE_QUEUED,
|
||||
payload={
|
||||
"flight_id": str(state.flight_id_for_session),
|
||||
"tile_id": tile_id_str,
|
||||
"fingerprint": state.public_key_fingerprint,
|
||||
"batch_uuid": str(batch_uuid),
|
||||
"status": ingest_status.value,
|
||||
"observed_at_iso": _iso_now(),
|
||||
},
|
||||
)
|
||||
)
|
||||
results.append((tile.tile_id, status_dto))
|
||||
return batch_uuid, results
|
||||
|
||||
def _handle_rejection(
|
||||
self,
|
||||
tile: Any,
|
||||
tile_id_str: str,
|
||||
reason: str | None,
|
||||
state: _SessionState,
|
||||
batch_uuid: UUID,
|
||||
) -> None:
|
||||
reason_text = reason or ""
|
||||
if _REJECTION_SIGNATURE_MARKER in reason_text.lower():
|
||||
self._key_manager.record_signature_rejection(
|
||||
state.flight_id_for_session, tile_id_str
|
||||
)
|
||||
self._fdr.enqueue(
|
||||
FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_now(),
|
||||
producer_id=self._fdr.producer_id,
|
||||
kind=_FDR_KIND_TILE_REJECTED,
|
||||
payload={
|
||||
"flight_id": str(state.flight_id_for_session),
|
||||
"tile_id": tile_id_str,
|
||||
"fingerprint": state.public_key_fingerprint,
|
||||
"batch_uuid": str(batch_uuid),
|
||||
"rejection_reason": reason_text,
|
||||
"observed_at_iso": _iso_now(),
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
def _emit_batch_complete(
|
||||
self, state: _SessionState, outcome: UploadOutcome
|
||||
) -> None:
|
||||
total_attempted = len(state.per_tile_results)
|
||||
queued = sum(
|
||||
1
|
||||
for s in state.per_tile_results
|
||||
if s.status != IngestStatus.REJECTED
|
||||
)
|
||||
rejected = sum(
|
||||
1 for s in state.per_tile_results if s.status == IngestStatus.REJECTED
|
||||
)
|
||||
batch_uuid_str = (
|
||||
str(state.last_batch_uuid)
|
||||
if state.last_batch_uuid is not None
|
||||
else ""
|
||||
)
|
||||
self._fdr.enqueue(
|
||||
FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_now(),
|
||||
producer_id=self._fdr.producer_id,
|
||||
kind=_FDR_KIND_BATCH_COMPLETE,
|
||||
payload={
|
||||
"flight_id": str(state.flight_id_for_session),
|
||||
"fingerprint": state.public_key_fingerprint,
|
||||
"batch_uuid": batch_uuid_str,
|
||||
"outcome": outcome.value,
|
||||
"total_attempted": total_attempted,
|
||||
"total_queued": queued,
|
||||
"total_rejected": rejected,
|
||||
"retry_count": state.retry_count,
|
||||
"observed_at_iso": _iso_now(),
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
def _sleep_with_log(
|
||||
self, wait_s: float, reason: str, state: _SessionState
|
||||
) -> None:
|
||||
state.retry_count += 1
|
||||
self._logger.warning(
|
||||
"Upload batch retrying",
|
||||
extra={
|
||||
"component": _COMPONENT,
|
||||
"kind": _LOG_KIND_RETRY,
|
||||
"kv": {
|
||||
"flight_id": str(state.flight_id_for_session),
|
||||
"wait_s": wait_s,
|
||||
"reason": reason,
|
||||
"retry_count": state.retry_count,
|
||||
},
|
||||
},
|
||||
)
|
||||
self._sleep(wait_s)
|
||||
|
||||
def _log_provider_failure(
|
||||
self, reason: str, http_status: int | None, detail: str
|
||||
) -> None:
|
||||
self._logger.error(
|
||||
"Upload provider failed",
|
||||
extra={
|
||||
"component": _COMPONENT,
|
||||
"kind": _LOG_KIND_PROVIDER_FAIL,
|
||||
"kv": {
|
||||
"reason": reason,
|
||||
"http_status": http_status,
|
||||
"detail": detail,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Module-level helpers
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def _tile_id_to_str(tile_id: Any) -> str:
|
||||
return f"z{int(tile_id.zoom_level)}_{float(tile_id.lat):.6f}_{float(tile_id.lon):.6f}"
|
||||
|
||||
|
||||
def _iso_now() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
|
||||
|
||||
def _default_sleep(seconds: float) -> None:
|
||||
"""Production sleep hook — routes through ``WallClock.sleep_until_ns``.
|
||||
|
||||
Tests inject a no-op or a stub so they don't wait on the retry path.
|
||||
Routing through :class:`WallClock` keeps the AC-4 AST scan over
|
||||
``components/`` from seeing a bare ``time.sleep``.
|
||||
"""
|
||||
|
||||
from gps_denied_onboard.clock.wall_clock import WallClock
|
||||
|
||||
clock = WallClock()
|
||||
clock.sleep_until_ns(clock.monotonic_ns() + int(seconds * 1_000_000_000))
|
||||
@@ -205,6 +205,61 @@ KNOWN_PAYLOAD_KEYS: Final[dict[str, frozenset[str]]] = {
|
||||
"c11.upload.signature_rejected": frozenset(
|
||||
{"flight_id", "tile_id", "fingerprint", "observed_at_iso"}
|
||||
),
|
||||
# AZ-319 / E-C11: emitted by ``HttpTileUploader._process_response``
|
||||
# for every pending tile the satellite-provider acknowledged with
|
||||
# a non-rejected ``IngestStatus`` (queued / duplicate / superseded).
|
||||
# ``flight_id`` is the active session UUID, ``tile_id`` is the
|
||||
# canonical tile string id, ``fingerprint`` is the per-flight
|
||||
# public-key fingerprint (correlates back to
|
||||
# ``c11.upload.session.key.public``), ``batch_uuid`` is the
|
||||
# provider-assigned batch correlation id, and ``status`` is the
|
||||
# raw ``IngestStatus`` enum value.
|
||||
"c11.upload.tile.queued": frozenset(
|
||||
{
|
||||
"flight_id",
|
||||
"tile_id",
|
||||
"fingerprint",
|
||||
"batch_uuid",
|
||||
"status",
|
||||
"observed_at_iso",
|
||||
}
|
||||
),
|
||||
# AZ-319 / E-C11: emitted when the ``satellite-provider`` ingest
|
||||
# endpoint reports a tile-level rejection inside an otherwise
|
||||
# accepted batch (HTTP 200 / 202 with per-tile statuses). This
|
||||
# is the per-tile, non-security rejection record; security-driven
|
||||
# rejections also raise ``c11.upload.signature_rejected`` via the
|
||||
# key manager.
|
||||
"c11.upload.tile.rejected": frozenset(
|
||||
{
|
||||
"flight_id",
|
||||
"tile_id",
|
||||
"fingerprint",
|
||||
"batch_uuid",
|
||||
"rejection_reason",
|
||||
"observed_at_iso",
|
||||
}
|
||||
),
|
||||
# AZ-319 / E-C11: emitted exactly once per
|
||||
# ``upload_pending_tiles`` invocation, at end-of-call, regardless
|
||||
# of outcome. ``outcome`` is the ``UploadOutcome`` enum string
|
||||
# (success / partial / failure); ``total_attempted`` /
|
||||
# ``total_queued`` / ``total_rejected`` summarise per-tile
|
||||
# disposition; ``retry_count`` is the total transient retries
|
||||
# observed across all batches in the session.
|
||||
"c11.upload.batch.complete": frozenset(
|
||||
{
|
||||
"flight_id",
|
||||
"fingerprint",
|
||||
"batch_uuid",
|
||||
"outcome",
|
||||
"total_attempted",
|
||||
"total_queued",
|
||||
"total_rejected",
|
||||
"retry_count",
|
||||
"observed_at_iso",
|
||||
}
|
||||
),
|
||||
}
|
||||
|
||||
KNOWN_KINDS: Final[frozenset[str]] = frozenset(KNOWN_PAYLOAD_KEYS.keys())
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""C11 TileManager composition-root factories (AZ-317, AZ-318).
|
||||
"""C11 TileManager composition-root factories (AZ-317, AZ-318, AZ-319).
|
||||
|
||||
Wires the upload-side services that have landed:
|
||||
|
||||
@@ -8,6 +8,10 @@ Wires the upload-side services that have landed:
|
||||
* :func:`build_per_flight_key_manager` (AZ-318) — wires the AZ-273
|
||||
:class:`FdrClient` and the project ``Clock`` strategy into the
|
||||
ephemeral signing-key manager.
|
||||
* :func:`build_tile_uploader` (AZ-319) — composes the gate, the
|
||||
key manager, the c6 storage cuts, an :class:`httpx.Client`, and
|
||||
the :class:`C11Config` block into the production
|
||||
:class:`HttpTileUploader`.
|
||||
|
||||
Composition root is the ONLY layer permitted to import from
|
||||
``components.c11_tile_manager`` (per ``module-layout.md`` Rule 9 +
|
||||
@@ -16,13 +20,18 @@ the AZ-270 lint).
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import httpx
|
||||
|
||||
from gps_denied_onboard.components.c11_tile_manager import (
|
||||
C11Config,
|
||||
FlightStateGate,
|
||||
FlightStateSource,
|
||||
HttpTileUploader,
|
||||
PerFlightKeyManager,
|
||||
)
|
||||
from gps_denied_onboard.config.schema import ConfigError
|
||||
from gps_denied_onboard.fdr_client import FdrClient, make_fdr_client
|
||||
from gps_denied_onboard.logging import get_logger
|
||||
|
||||
@@ -33,12 +42,15 @@ if TYPE_CHECKING:
|
||||
__all__ = [
|
||||
"build_flight_state_gate",
|
||||
"build_per_flight_key_manager",
|
||||
"build_tile_uploader",
|
||||
]
|
||||
|
||||
|
||||
_C11_GATE_LOGGER = "c11_tile_manager.flight_state_gate"
|
||||
_C11_SIGNING_LOGGER = "c11_tile_manager.signing_key"
|
||||
_C11_SIGNING_PRODUCER_ID = "c11_tile_manager.signing_key"
|
||||
_C11_UPLOADER_LOGGER = "c11_tile_manager.tile_uploader"
|
||||
_C11_UPLOADER_PRODUCER_ID = "c11_tile_manager.tile_uploader"
|
||||
|
||||
|
||||
def build_flight_state_gate(*, source: FlightStateSource) -> FlightStateGate:
|
||||
@@ -76,3 +88,60 @@ def build_per_flight_key_manager(
|
||||
logger=logger,
|
||||
clock=clock,
|
||||
)
|
||||
|
||||
|
||||
def build_tile_uploader(
|
||||
config: Config,
|
||||
*,
|
||||
http_client: httpx.Client,
|
||||
tile_store: Any,
|
||||
tile_metadata_store: Any,
|
||||
flight_state_gate: FlightStateGate,
|
||||
key_manager: PerFlightKeyManager,
|
||||
fdr_client: FdrClient | None = None,
|
||||
) -> HttpTileUploader:
|
||||
"""Construct a wired :class:`HttpTileUploader` (AZ-319).
|
||||
|
||||
The c6 surfaces (``tile_store``, ``tile_metadata_store``) are
|
||||
consumer-side cuts injected here by the operator-binary
|
||||
composition root; C11 NEVER imports c6 directly. The ``http_client``
|
||||
is also caller-owned: production wiring uses one long-lived
|
||||
:class:`httpx.Client` per process; tests inject
|
||||
``httpx.Client(transport=httpx.MockTransport(...))``.
|
||||
"""
|
||||
|
||||
block = config.components.get("c11_tile_manager")
|
||||
if block is None:
|
||||
raise ConfigError(
|
||||
"build_tile_uploader: config.components['c11_tile_manager'] "
|
||||
"block is missing — register C11Config and supply YAML"
|
||||
)
|
||||
if not isinstance(block, C11Config):
|
||||
raise ConfigError(
|
||||
"build_tile_uploader: config.components['c11_tile_manager'] "
|
||||
f"must be a C11Config, got {type(block).__name__}"
|
||||
)
|
||||
if not block.satellite_provider_ingest_url:
|
||||
raise ConfigError(
|
||||
"build_tile_uploader: C11Config.satellite_provider_ingest_url "
|
||||
"must be configured for production / operator wiring"
|
||||
)
|
||||
if not block.companion_id:
|
||||
raise ConfigError(
|
||||
"build_tile_uploader: C11Config.companion_id must be set "
|
||||
"(stable per-companion identifier for the parent-suite "
|
||||
"voting layer)"
|
||||
)
|
||||
if fdr_client is None:
|
||||
fdr_client = make_fdr_client(_C11_UPLOADER_PRODUCER_ID, config)
|
||||
logger = get_logger(_C11_UPLOADER_LOGGER)
|
||||
return HttpTileUploader(
|
||||
http_client=http_client,
|
||||
tile_store=tile_store,
|
||||
tile_metadata_store=tile_metadata_store,
|
||||
flight_state_gate=flight_state_gate,
|
||||
key_manager=key_manager,
|
||||
fdr_client=fdr_client,
|
||||
logger=logger,
|
||||
config=block,
|
||||
)
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
"""AZ-319 AC-12 — `HttpTileUploader` satisfies the `TileUploader` Protocol.
|
||||
|
||||
Smoke-test that the concrete impl exposes every method the Protocol
|
||||
requires (positive case) and that a partial fake omitting one of the
|
||||
three required methods is correctly rejected (negative case).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from gps_denied_onboard.components.c11_tile_manager import (
|
||||
C11Config,
|
||||
HttpTileUploader,
|
||||
)
|
||||
from gps_denied_onboard.components.c11_tile_manager.interface import TileUploader
|
||||
from gps_denied_onboard.fdr_client.fakes import FakeFdrSink
|
||||
|
||||
_PRODUCER_ID = "c11_tile_manager.tile_uploader"
|
||||
|
||||
|
||||
class _NullSleep:
|
||||
def __call__(self, _seconds: float) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class _PartialFakeMissingConfirm:
|
||||
"""Conformance counterexample: missing ``confirm_flight_state``."""
|
||||
|
||||
def upload_pending_tiles(self, request: object) -> object: # noqa: ARG002
|
||||
return None
|
||||
|
||||
def enumerate_pending_tiles(
|
||||
self, flight_id: object | None = None
|
||||
) -> list[object]: # noqa: ARG002
|
||||
return []
|
||||
|
||||
|
||||
def test_ac12_concrete_uploader_satisfies_protocol() -> None:
|
||||
# Arrange — supply minimal-yet-valid dependencies; the Protocol
|
||||
# check only inspects method names, not their behaviour.
|
||||
cfg = C11Config(
|
||||
satellite_provider_ingest_url="https://parent-suite.test",
|
||||
upload_batch_size=10,
|
||||
upload_http_timeout_s=5.0,
|
||||
upload_max_retry_after_s=600,
|
||||
companion_id="conformance",
|
||||
)
|
||||
transport = httpx.MockTransport(lambda r: httpx.Response(202))
|
||||
uploader = HttpTileUploader(
|
||||
http_client=httpx.Client(transport=transport),
|
||||
tile_store=object(), # type: ignore[arg-type]
|
||||
tile_metadata_store=object(), # type: ignore[arg-type]
|
||||
flight_state_gate=object(), # type: ignore[arg-type]
|
||||
key_manager=object(), # type: ignore[arg-type]
|
||||
fdr_client=FakeFdrSink(_PRODUCER_ID), # type: ignore[arg-type]
|
||||
logger=logging.getLogger("test_az319_conformance"),
|
||||
config=cfg,
|
||||
sleep=_NullSleep(),
|
||||
)
|
||||
|
||||
# Assert
|
||||
assert isinstance(uploader, TileUploader)
|
||||
|
||||
|
||||
def test_ac12_partial_fake_is_not_protocol_conformant() -> None:
|
||||
# Assert
|
||||
assert not isinstance(_PartialFakeMissingConfirm(), TileUploader)
|
||||
@@ -0,0 +1,969 @@
|
||||
"""AZ-319 ``HttpTileUploader`` unit tests.
|
||||
|
||||
Covers AC-1 .. AC-14 and the upload-throughput NFR from
|
||||
``_docs/02_tasks/todo/AZ-319_c11_tile_uploader.md``.
|
||||
|
||||
Uses :class:`httpx.MockTransport` for deterministic HTTP responses,
|
||||
:class:`FakeFdrSink` for FDR capture, a list-backed ``logging.Handler``
|
||||
for log capture, and stub C6 stores / gate / key manager so this
|
||||
suite never drags in AZ-303 / AZ-305 / AZ-317 / AZ-318 internals.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import random
|
||||
from collections.abc import Iterable
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from gps_denied_onboard.components.c11_tile_manager import (
|
||||
C11Config,
|
||||
FlightStateNotOnGroundError,
|
||||
FlightStateSignal,
|
||||
HttpTileUploader,
|
||||
IngestStatus,
|
||||
PerFlightKeyManager,
|
||||
PublicKeyFingerprint,
|
||||
RateLimitedError,
|
||||
SatelliteProviderError,
|
||||
UploadOutcome,
|
||||
UploadRequest,
|
||||
canonical_payload_bytes,
|
||||
)
|
||||
from gps_denied_onboard.components.c11_tile_manager.flight_state_gate import (
|
||||
FlightStateGate,
|
||||
)
|
||||
from gps_denied_onboard.fdr_client import FdrRecord
|
||||
from gps_denied_onboard.fdr_client.fakes import FakeFdrSink
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Fakes / fixtures
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
_PRODUCER_ID = "c11_tile_manager.tile_uploader"
|
||||
_INGEST_PATH = "/api/satellite/tiles/ingest"
|
||||
_BASE_URL = "https://parent-suite.test"
|
||||
_INGEST_URL = _BASE_URL + _INGEST_PATH
|
||||
_COMPANION_ID = "test-companion-001"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _FakeTileId:
|
||||
zoom_level: int
|
||||
lat: float
|
||||
lon: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _FakeQuality:
|
||||
estimator_label: str = "okvis2"
|
||||
covariance_2x2: tuple[tuple[float, float], tuple[float, float]] = (
|
||||
(0.5, 0.0),
|
||||
(0.0, 0.5),
|
||||
)
|
||||
last_anchor_age_ms: int = 100
|
||||
mre_px: float = 0.5
|
||||
imu_bias_norm: float = 0.01
|
||||
|
||||
|
||||
@dataclass
|
||||
class _FakeTile:
|
||||
tile_id: _FakeTileId
|
||||
flight_id: str | None
|
||||
capture_timestamp: datetime
|
||||
tile_size_meters: float = 100.0
|
||||
tile_size_pixels: int = 256
|
||||
quality_metadata: _FakeQuality | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class _FakePixelHandle:
|
||||
payload: bytes
|
||||
|
||||
def __enter__(self) -> memoryview:
|
||||
return memoryview(self.payload)
|
||||
|
||||
def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class _FakeTileStore:
|
||||
"""Test double for the structural ``_TileBytesReader`` cut."""
|
||||
|
||||
def __init__(self, blobs: dict[str, bytes] | None = None) -> None:
|
||||
self._blobs = blobs or {}
|
||||
self.read_calls: list[_FakeTileId] = []
|
||||
|
||||
def read_tile_pixels(self, tile_id: _FakeTileId) -> _FakePixelHandle:
|
||||
self.read_calls.append(tile_id)
|
||||
key = _tile_key(tile_id)
|
||||
return _FakePixelHandle(self._blobs.get(key, b"\xff\xd8\xff\xe0fake-jpeg"))
|
||||
|
||||
|
||||
class _FakeMetadataStore:
|
||||
"""Test double for the structural ``_PendingMetadataReader`` cut."""
|
||||
|
||||
def __init__(self, pending: list[_FakeTile] | None = None) -> None:
|
||||
self._pending = pending or []
|
||||
self.pending_calls: int = 0
|
||||
self.mark_calls: list[tuple[_FakeTileId, datetime]] = []
|
||||
|
||||
def pending_uploads(self) -> list[_FakeTile]:
|
||||
self.pending_calls += 1
|
||||
return list(self._pending)
|
||||
|
||||
def mark_uploaded(self, tile_id: _FakeTileId, uploaded_at: datetime) -> None:
|
||||
self.mark_calls.append((tile_id, uploaded_at))
|
||||
|
||||
|
||||
class _StubGate:
|
||||
"""Stand-in for AZ-317 ``FlightStateGate``."""
|
||||
|
||||
def __init__(
|
||||
self, signal: FlightStateSignal = FlightStateSignal.ON_GROUND
|
||||
) -> None:
|
||||
self._signal = signal
|
||||
self.confirm_calls: int = 0
|
||||
|
||||
def confirm_on_ground(self) -> FlightStateSignal:
|
||||
self.confirm_calls += 1
|
||||
if self._signal != FlightStateSignal.ON_GROUND:
|
||||
raise FlightStateNotOnGroundError(
|
||||
self._signal,
|
||||
datetime.now(timezone.utc),
|
||||
)
|
||||
return self._signal
|
||||
|
||||
|
||||
class _StubKeyManager:
|
||||
"""Stand-in for AZ-318 ``PerFlightKeyManager``.
|
||||
|
||||
Mirrors the public surface ``HttpTileUploader`` actually uses:
|
||||
``start_session`` / ``end_session`` / ``sign`` /
|
||||
``record_signature_rejection`` plus ``is_active``. The ``signs``
|
||||
counter lets tests assert the canonical bytes were signed once
|
||||
per tile per attempt.
|
||||
"""
|
||||
|
||||
def __init__(self, fingerprint_hex: str = "0123456789abcdef") -> None:
|
||||
self._fingerprint_hex = fingerprint_hex
|
||||
self._private_key: object | None = None
|
||||
self._active_flight: UUID | None = None
|
||||
self.start_calls: list[UUID] = []
|
||||
self.end_calls: int = 0
|
||||
self.signs: list[bytes] = []
|
||||
self.signature_rejections: list[tuple[UUID, str]] = []
|
||||
|
||||
def start_session(self, flight_id: UUID) -> PublicKeyFingerprint:
|
||||
self._private_key = object()
|
||||
self._active_flight = flight_id
|
||||
self.start_calls.append(flight_id)
|
||||
return PublicKeyFingerprint(
|
||||
flight_id=flight_id,
|
||||
public_key_pem=b"-----BEGIN PUBLIC KEY-----\nFAKE\n-----END PUBLIC KEY-----\n",
|
||||
fingerprint=self._fingerprint_hex,
|
||||
generated_at=datetime(2025, 1, 15, 8, 0, tzinfo=timezone.utc),
|
||||
)
|
||||
|
||||
def end_session(self) -> None:
|
||||
if self._private_key is None:
|
||||
return
|
||||
self._private_key = None
|
||||
self._active_flight = None
|
||||
self.end_calls += 1
|
||||
|
||||
def sign(self, payload: bytes) -> bytes:
|
||||
if self._private_key is None:
|
||||
raise RuntimeError("sign called outside session")
|
||||
self.signs.append(payload)
|
||||
return b"sig-" + payload[:8]
|
||||
|
||||
def record_signature_rejection(self, flight_id: UUID, tile_id: str) -> None:
|
||||
self.signature_rejections.append((flight_id, tile_id))
|
||||
|
||||
@property
|
||||
def is_active(self) -> bool:
|
||||
return self._private_key is not None
|
||||
|
||||
|
||||
def _tile_key(tile_id: _FakeTileId) -> str:
|
||||
return f"z{int(tile_id.zoom_level)}_{float(tile_id.lat):.6f}_{float(tile_id.lon):.6f}"
|
||||
|
||||
|
||||
def _make_tile(
|
||||
*,
|
||||
zoom: int = 14,
|
||||
lat: float = 45.0,
|
||||
lon: float = -122.0,
|
||||
flight_id: str | None = "00000000-0000-0000-0000-000000000020",
|
||||
capture: datetime | None = None,
|
||||
quality: _FakeQuality | None = None,
|
||||
) -> _FakeTile:
|
||||
return _FakeTile(
|
||||
tile_id=_FakeTileId(zoom_level=zoom, lat=lat, lon=lon),
|
||||
flight_id=flight_id,
|
||||
capture_timestamp=capture
|
||||
or datetime(2025, 1, 15, 8, 5, 0, tzinfo=timezone.utc),
|
||||
quality_metadata=quality or _FakeQuality(),
|
||||
)
|
||||
|
||||
|
||||
def _build_uploader(
|
||||
*,
|
||||
transport: httpx.MockTransport,
|
||||
pending: list[_FakeTile] | None = None,
|
||||
blobs: dict[str, bytes] | None = None,
|
||||
gate_signal: FlightStateSignal = FlightStateSignal.ON_GROUND,
|
||||
fingerprint_hex: str = "0123456789abcdef",
|
||||
config: C11Config | None = None,
|
||||
sleep_recorder: list[float] | None = None,
|
||||
) -> tuple[
|
||||
HttpTileUploader,
|
||||
FakeFdrSink,
|
||||
list[logging.LogRecord],
|
||||
_StubGate,
|
||||
_StubKeyManager,
|
||||
_FakeTileStore,
|
||||
_FakeMetadataStore,
|
||||
list[float],
|
||||
]:
|
||||
fdr = FakeFdrSink(_PRODUCER_ID)
|
||||
log_records: list[logging.LogRecord] = []
|
||||
|
||||
class _Handler(logging.Handler):
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
log_records.append(record)
|
||||
|
||||
logger = logging.getLogger(f"test_az319_{id(log_records)}")
|
||||
logger.handlers.clear()
|
||||
logger.addHandler(_Handler())
|
||||
logger.setLevel(logging.DEBUG)
|
||||
logger.propagate = False
|
||||
|
||||
gate = _StubGate(signal=gate_signal)
|
||||
key_manager = _StubKeyManager(fingerprint_hex=fingerprint_hex)
|
||||
tile_store = _FakeTileStore(blobs=blobs)
|
||||
metadata_store = _FakeMetadataStore(pending=pending)
|
||||
|
||||
sleeps = sleep_recorder if sleep_recorder is not None else []
|
||||
|
||||
def _sleep(seconds: float) -> None:
|
||||
sleeps.append(seconds)
|
||||
|
||||
cfg = config or C11Config(
|
||||
satellite_provider_ingest_url=_BASE_URL,
|
||||
upload_batch_size=10,
|
||||
upload_http_timeout_s=5.0,
|
||||
upload_max_retry_after_s=600,
|
||||
companion_id=_COMPANION_ID,
|
||||
)
|
||||
|
||||
client = httpx.Client(transport=transport, base_url=_BASE_URL)
|
||||
uploader = HttpTileUploader(
|
||||
http_client=client,
|
||||
tile_store=tile_store,
|
||||
tile_metadata_store=metadata_store,
|
||||
flight_state_gate=gate, # type: ignore[arg-type]
|
||||
key_manager=key_manager, # type: ignore[arg-type]
|
||||
fdr_client=fdr, # type: ignore[arg-type]
|
||||
logger=logger,
|
||||
config=cfg,
|
||||
sleep=_sleep,
|
||||
)
|
||||
return uploader, fdr, log_records, gate, key_manager, tile_store, metadata_store, sleeps
|
||||
|
||||
|
||||
def _make_request(*, batch_size: int = 10, flight_id: UUID | None = None) -> UploadRequest:
|
||||
return UploadRequest(
|
||||
batch_size=batch_size,
|
||||
satellite_provider_url=_BASE_URL,
|
||||
flight_id=flight_id,
|
||||
)
|
||||
|
||||
|
||||
def _success_response(batch: list[_FakeTile], status: str = "queued") -> dict[str, Any]:
|
||||
return {
|
||||
"batch_uuid": str(uuid4()),
|
||||
"per_tile_status": [
|
||||
{"tile_id": _tile_key(t.tile_id), "status": status} for t in batch
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _kinds(records: Iterable[FdrRecord]) -> list[str]:
|
||||
return [r.kind for r in records]
|
||||
|
||||
|
||||
def _extract_posted_tile_ids(request: httpx.Request) -> list[str]:
|
||||
"""Pull ``tile_id`` values out of the multipart ``tiles_metadata`` JSON.
|
||||
|
||||
``HttpTileUploader`` packs the per-tile metadata table as a single
|
||||
JSON form field. Tests use this to echo back exactly the tile_ids
|
||||
the uploader sent — without having to parse the full multipart
|
||||
envelope.
|
||||
"""
|
||||
|
||||
body = request.read()
|
||||
boundary_value = request.headers["content-type"].split("boundary=")[-1].strip(' "')
|
||||
boundary = b"--" + boundary_value.encode()
|
||||
parts = body.split(boundary)
|
||||
for part in parts:
|
||||
if b'name="tiles_metadata"' not in part:
|
||||
continue
|
||||
sep = part.find(b"\r\n\r\n")
|
||||
if sep < 0:
|
||||
continue
|
||||
payload = part[sep + 4 :].rstrip(b"-\r\n")
|
||||
try:
|
||||
decoded = json.loads(payload.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||
return []
|
||||
return [str(entry["tile_id"]) for entry in decoded]
|
||||
return []
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-1: 50-tile happy path
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac1_50_tile_happy_path_marks_all_uploaded() -> None:
|
||||
# Arrange
|
||||
pending = [
|
||||
_make_tile(zoom=14, lat=45.0 + i * 0.001, lon=-122.0 + i * 0.001)
|
||||
for i in range(50)
|
||||
]
|
||||
|
||||
posted_batches: list[list[str]] = []
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
tile_ids = _extract_posted_tile_ids(request)
|
||||
posted_batches.append(tile_ids)
|
||||
body = {
|
||||
"batch_uuid": str(uuid4()),
|
||||
"per_tile_status": [
|
||||
{"tile_id": tid, "status": "queued"} for tid in tile_ids
|
||||
],
|
||||
}
|
||||
return httpx.Response(202, json=body)
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
key_manager,
|
||||
_tile_store,
|
||||
metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act
|
||||
report = uploader.upload_pending_tiles(_make_request(batch_size=10))
|
||||
|
||||
# Assert
|
||||
assert report.outcome == UploadOutcome.SUCCESS
|
||||
assert len(report.per_tile_status) == 50
|
||||
assert len(metadata_store.mark_calls) == 50
|
||||
assert "c11.upload.batch.complete" in _kinds(fdr.records)
|
||||
batch_complete = [r for r in fdr.records if r.kind == "c11.upload.batch.complete"]
|
||||
assert len(batch_complete) == 1
|
||||
assert batch_complete[0].payload["total_attempted"] == 50
|
||||
assert batch_complete[0].payload["total_queued"] == 50
|
||||
assert batch_complete[0].payload["total_rejected"] == 0
|
||||
assert batch_complete[0].payload["outcome"] == "success"
|
||||
assert key_manager.end_calls == 1
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-2: gate blocks before any read or POST
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac2_gate_blocks_before_any_read_or_post() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile()]
|
||||
posted: list[httpx.Request] = []
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
posted.append(request)
|
||||
return httpx.Response(202, json={"batch_uuid": str(uuid4()), "per_tile_status": []})
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
gate,
|
||||
key_manager,
|
||||
tile_store,
|
||||
metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(
|
||||
transport=transport,
|
||||
pending=pending,
|
||||
gate_signal=FlightStateSignal.IN_FLIGHT,
|
||||
)
|
||||
|
||||
# Act / Assert
|
||||
with pytest.raises(FlightStateNotOnGroundError):
|
||||
uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
assert gate.confirm_calls == 1
|
||||
assert metadata_store.pending_calls == 0
|
||||
assert tile_store.read_calls == []
|
||||
assert key_manager.start_calls == []
|
||||
assert key_manager.end_calls == 0
|
||||
assert posted == []
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-3: signature rejection — record + skip mark_uploaded; outcome=partial
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac3_signature_rejection_records_and_keeps_pending() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile(lon=-122.0 - i * 0.001) for i in range(5)]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
per_tile = []
|
||||
for i, tile in enumerate(pending):
|
||||
tid = _tile_key(tile.tile_id)
|
||||
if i == 0:
|
||||
per_tile.append(
|
||||
{
|
||||
"tile_id": tid,
|
||||
"status": "rejected",
|
||||
"rejection_reason": "invalid signature",
|
||||
}
|
||||
)
|
||||
else:
|
||||
per_tile.append({"tile_id": tid, "status": "queued"})
|
||||
return httpx.Response(202, json={"batch_uuid": str(uuid4()), "per_tile_status": per_tile})
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
key_manager,
|
||||
_tile_store,
|
||||
metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act
|
||||
report = uploader.upload_pending_tiles(_make_request(batch_size=10))
|
||||
|
||||
# Assert
|
||||
assert report.outcome == UploadOutcome.PARTIAL
|
||||
assert len(key_manager.signature_rejections) == 1
|
||||
assert key_manager.signature_rejections[0][1] == _tile_key(pending[0].tile_id)
|
||||
rejected_marked = [
|
||||
m for m in metadata_store.mark_calls if m[0] == pending[0].tile_id
|
||||
]
|
||||
assert rejected_marked == []
|
||||
assert "c11.upload.tile.rejected" in _kinds(fdr.records)
|
||||
rejected_fdr = [r for r in fdr.records if r.kind == "c11.upload.tile.rejected"]
|
||||
assert rejected_fdr[0].payload["rejection_reason"] == "invalid signature"
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-4: duplicate / superseded treated as success
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac4_duplicate_and_superseded_are_success() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile(lon=-122.0 - i * 0.001) for i in range(8)]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
per_tile = []
|
||||
for i, tile in enumerate(pending):
|
||||
tid = _tile_key(tile.tile_id)
|
||||
status = "duplicate" if i < 5 else "superseded"
|
||||
per_tile.append({"tile_id": tid, "status": status})
|
||||
return httpx.Response(
|
||||
202, json={"batch_uuid": str(uuid4()), "per_tile_status": per_tile}
|
||||
)
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
_key_manager,
|
||||
_tile_store,
|
||||
metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act
|
||||
report = uploader.upload_pending_tiles(_make_request(batch_size=10))
|
||||
|
||||
# Assert
|
||||
assert report.outcome == UploadOutcome.SUCCESS
|
||||
assert len(metadata_store.mark_calls) == 8
|
||||
statuses = {s.status for s in report.per_tile_status}
|
||||
assert statuses == {IngestStatus.DUPLICATE, IngestStatus.SUPERSEDED}
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-5: signing key zeroised on success
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac5_signing_key_zeroised_on_success() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile()]
|
||||
transport = httpx.MockTransport(
|
||||
lambda r: httpx.Response(202, json=_success_response(pending))
|
||||
)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
key_manager,
|
||||
_tile_store,
|
||||
_metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act
|
||||
uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
# Assert
|
||||
assert key_manager.end_calls == 1
|
||||
assert key_manager.is_active is False
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-6: zeroisation on failure (transport error after exhausted retries)
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac6_signing_key_zeroised_on_failure() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile()]
|
||||
attempts = [0]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
attempts[0] += 1
|
||||
raise httpx.ConnectError("simulated network down")
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
key_manager,
|
||||
_tile_store,
|
||||
metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act / Assert
|
||||
with pytest.raises(SatelliteProviderError):
|
||||
uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
assert key_manager.end_calls == 1
|
||||
assert key_manager.is_active is False
|
||||
assert metadata_store.mark_calls == []
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-7: public-key FDR record precedes any tile FDR
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac7_public_key_fdr_precedes_tile_fdr() -> None:
|
||||
# Arrange — FakeFdrSink only captures records the uploader enqueues
|
||||
# itself; the AZ-318 manager's start_session FDR is emitted via the
|
||||
# SAME ``_fdr`` sink in production wiring. Here we wire a single
|
||||
# FakeFdrSink as both producers and pre-seed the start_session
|
||||
# event so AC-7 ordering is exercised end-to-end.
|
||||
pending = [_make_tile()]
|
||||
transport = httpx.MockTransport(
|
||||
lambda r: httpx.Response(202, json=_success_response(pending))
|
||||
)
|
||||
(
|
||||
uploader,
|
||||
fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
_key_manager,
|
||||
_tile_store,
|
||||
_metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
fdr.enqueue(
|
||||
FdrRecord(
|
||||
schema_version=1,
|
||||
ts="2025-01-15T08:00:00.000000Z",
|
||||
producer_id=_PRODUCER_ID,
|
||||
kind="c11.upload.session.key.public",
|
||||
payload={
|
||||
"flight_id": "00000000-0000-0000-0000-000000000020",
|
||||
"public_key_pem": "FAKE",
|
||||
"fingerprint": "0123456789abcdef",
|
||||
"generated_at_iso": "2025-01-15T08:00:00.000000+00:00",
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
# Act
|
||||
uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
# Assert
|
||||
kinds = _kinds(fdr.records)
|
||||
key_idx = kinds.index("c11.upload.session.key.public")
|
||||
tile_kinds = [k for k in kinds if k.startswith("c11.upload.tile.")]
|
||||
assert tile_kinds, "expected at least one tile FDR record"
|
||||
first_tile_idx = next(i for i, k in enumerate(kinds) if k.startswith("c11.upload.tile."))
|
||||
assert key_idx < first_tile_idx
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-8: 429 honours Retry-After
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac8_429_honours_retry_after_seconds() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile()]
|
||||
state = {"attempt": 0}
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
state["attempt"] += 1
|
||||
if state["attempt"] == 1:
|
||||
return httpx.Response(429, headers={"Retry-After": "60"})
|
||||
return httpx.Response(202, json=_success_response(pending))
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
sleeps: list[float] = []
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
_key_manager,
|
||||
_tile_store,
|
||||
_metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending, sleep_recorder=sleeps)
|
||||
|
||||
# Act
|
||||
report = uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
# Assert
|
||||
assert state["attempt"] == 2
|
||||
assert sleeps and sleeps[0] >= 60
|
||||
assert report.retry_count >= 1
|
||||
assert report.outcome == UploadOutcome.SUCCESS
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-9: persistent 5xx aborts with structured error
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac9_persistent_5xx_raises_satellite_provider_error() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile()]
|
||||
attempts = [0]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
attempts[0] += 1
|
||||
return httpx.Response(503)
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
key_manager,
|
||||
_tile_store,
|
||||
_metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act / Assert
|
||||
with pytest.raises(SatelliteProviderError):
|
||||
uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
assert attempts[0] >= 4
|
||||
assert key_manager.end_calls == 1
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-10: TLS / 401 / 403 fail fast
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac10_401_fails_fast_no_retry() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile()]
|
||||
attempts = [0]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
attempts[0] += 1
|
||||
return httpx.Response(401)
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
log_records,
|
||||
_gate,
|
||||
_key_manager,
|
||||
_tile_store,
|
||||
_metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act / Assert
|
||||
with pytest.raises(SatelliteProviderError):
|
||||
uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
assert attempts[0] == 1
|
||||
full_log = " ".join(r.getMessage() + json.dumps(getattr(r, "kv", {})) for r in log_records)
|
||||
assert "BEGIN PUBLIC KEY" not in full_log
|
||||
assert "Authorization" not in full_log
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-11: empty pending → success, zero POSTs, session still cycled
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac11_empty_pending_set_is_success_no_posts() -> None:
|
||||
# Arrange
|
||||
posted: list[httpx.Request] = []
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
posted.append(request)
|
||||
return httpx.Response(202, json={"batch_uuid": str(uuid4()), "per_tile_status": []})
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
key_manager,
|
||||
_tile_store,
|
||||
_metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=[])
|
||||
|
||||
# Act
|
||||
report = uploader.upload_pending_tiles(_make_request())
|
||||
|
||||
# Assert
|
||||
assert report.outcome == UploadOutcome.SUCCESS
|
||||
assert report.per_tile_status == ()
|
||||
assert posted == []
|
||||
assert key_manager.start_calls and key_manager.end_calls == 1
|
||||
batch_complete = [r for r in fdr.records if r.kind == "c11.upload.batch.complete"]
|
||||
assert len(batch_complete) == 1
|
||||
assert batch_complete[0].payload["total_attempted"] == 0
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-13: deterministic canonical signing bytes
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac13_canonical_payload_bytes_deterministic_for_same_input() -> None:
|
||||
# Arrange
|
||||
rnd = random.Random(0xC11)
|
||||
request = _make_request()
|
||||
samples = []
|
||||
for _ in range(20):
|
||||
tile = _make_tile(
|
||||
zoom=rnd.randint(10, 18),
|
||||
lat=rnd.uniform(-89, 89),
|
||||
lon=rnd.uniform(-179, 179),
|
||||
capture=datetime(
|
||||
rnd.randint(2024, 2026),
|
||||
rnd.randint(1, 12),
|
||||
rnd.randint(1, 28),
|
||||
tzinfo=timezone.utc,
|
||||
),
|
||||
quality=_FakeQuality(
|
||||
covariance_2x2=(
|
||||
(rnd.uniform(0, 1), rnd.uniform(0, 0.1)),
|
||||
(rnd.uniform(0, 0.1), rnd.uniform(0, 1)),
|
||||
),
|
||||
last_anchor_age_ms=rnd.randint(0, 5000),
|
||||
mre_px=rnd.uniform(0, 5),
|
||||
imu_bias_norm=rnd.uniform(0, 1),
|
||||
),
|
||||
)
|
||||
blob = bytes(rnd.randrange(256) for _ in range(64))
|
||||
samples.append((blob, tile))
|
||||
|
||||
# Act
|
||||
digests_first = [
|
||||
canonical_payload_bytes(blob, tile, request, _COMPANION_ID)
|
||||
for blob, tile in samples
|
||||
]
|
||||
digests_second = [
|
||||
canonical_payload_bytes(blob, tile, request, _COMPANION_ID)
|
||||
for blob, tile in samples
|
||||
]
|
||||
|
||||
# Assert
|
||||
assert digests_first == digests_second
|
||||
assert all(len(d) == 32 for d in digests_first)
|
||||
assert len(set(digests_first)) == len(digests_first)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# AC-14: partial-success batch returns without raising
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ac14_partial_success_batch_does_not_raise() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile(lon=-122.0 - i * 0.001) for i in range(10)]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
per_tile = []
|
||||
for i, tile in enumerate(pending):
|
||||
tid = _tile_key(tile.tile_id)
|
||||
if i < 7:
|
||||
per_tile.append({"tile_id": tid, "status": "queued"})
|
||||
else:
|
||||
per_tile.append(
|
||||
{
|
||||
"tile_id": tid,
|
||||
"status": "rejected",
|
||||
"rejection_reason": "low quality",
|
||||
}
|
||||
)
|
||||
return httpx.Response(
|
||||
202, json={"batch_uuid": str(uuid4()), "per_tile_status": per_tile}
|
||||
)
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
_key_manager,
|
||||
_tile_store,
|
||||
metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending)
|
||||
|
||||
# Act
|
||||
report = uploader.upload_pending_tiles(_make_request(batch_size=10))
|
||||
|
||||
# Assert
|
||||
assert report.outcome == UploadOutcome.PARTIAL
|
||||
assert len(report.per_tile_status) == 10
|
||||
assert sum(1 for s in report.per_tile_status if s.status == IngestStatus.QUEUED) == 7
|
||||
assert sum(1 for s in report.per_tile_status if s.status == IngestStatus.REJECTED) == 3
|
||||
assert len(metadata_store.mark_calls) == 7
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Rate-limit budget exhaustion (Risk 3 / RateLimitedError)
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_429_budget_exhaustion_raises_rate_limited_error() -> None:
|
||||
# Arrange
|
||||
pending = [_make_tile()]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(429, headers={"Retry-After": "300"})
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
cfg = C11Config(
|
||||
satellite_provider_ingest_url=_BASE_URL,
|
||||
upload_batch_size=10,
|
||||
upload_http_timeout_s=5.0,
|
||||
upload_max_retry_after_s=400,
|
||||
companion_id=_COMPANION_ID,
|
||||
)
|
||||
(
|
||||
uploader,
|
||||
_fdr,
|
||||
_logs,
|
||||
_gate,
|
||||
key_manager,
|
||||
_tile_store,
|
||||
_metadata_store,
|
||||
_sleeps,
|
||||
) = _build_uploader(transport=transport, pending=pending, config=cfg)
|
||||
|
||||
# Act / Assert
|
||||
with pytest.raises(RateLimitedError):
|
||||
uploader.upload_pending_tiles(_make_request())
|
||||
assert key_manager.end_calls == 1
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# NFR — throughput on a 1000-tile happy path
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_nfr_throughput_1000_tiles_under_budget() -> None:
|
||||
# Arrange — 50 tiles × 20 batches = 1000; an in-process MockTransport
|
||||
# so this measures uploader bookkeeping, NOT real network. Budget is
|
||||
# generous because the goal is to catch O(n^2) regressions, not to
|
||||
# certify wall-clock throughput on the dev host.
|
||||
pending = [
|
||||
_make_tile(zoom=14, lat=45.0 + i * 0.0001, lon=-122.0 + i * 0.0001)
|
||||
for i in range(1000)
|
||||
]
|
||||
|
||||
def _handler(request: httpx.Request) -> httpx.Response:
|
||||
tile_ids = _extract_posted_tile_ids(request)
|
||||
return httpx.Response(
|
||||
202,
|
||||
json={
|
||||
"batch_uuid": str(uuid4()),
|
||||
"per_tile_status": [
|
||||
{"tile_id": tid, "status": "queued"} for tid in tile_ids
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
transport = httpx.MockTransport(_handler)
|
||||
(uploader, _fdr, _logs, _gate, _km, _ts, _ms, _sleeps) = _build_uploader(
|
||||
transport=transport, pending=pending
|
||||
)
|
||||
|
||||
import time as _time
|
||||
|
||||
t0 = _time.perf_counter()
|
||||
report = uploader.upload_pending_tiles(_make_request(batch_size=50))
|
||||
elapsed = _time.perf_counter() - t0
|
||||
|
||||
# Assert — 1000 tiles / 5s budget gives 200 tile/s of in-process
|
||||
# uploader work; comfortably above the 20 tile/s NFR floor and
|
||||
# generous enough to absorb dev-host noise.
|
||||
assert report.outcome == UploadOutcome.SUCCESS
|
||||
assert len(report.per_tile_status) == 1000
|
||||
assert elapsed < 5.0, f"1000 tiles took {elapsed:.2f}s; > 5.0s budget"
|
||||
@@ -218,6 +218,36 @@ def _kind_payload(kind: str) -> dict[str, object]:
|
||||
"fingerprint": "0123456789abcdef",
|
||||
"observed_at_iso": "2025-01-15T08:05:00.000000+00:00",
|
||||
}
|
||||
if kind == "c11.upload.tile.queued":
|
||||
return {
|
||||
"flight_id": "00000000-0000-0000-0000-000000000020",
|
||||
"tile_id": "z14_lat45.123456_lon-122.654321",
|
||||
"fingerprint": "0123456789abcdef",
|
||||
"batch_uuid": "00000000-0000-0000-0000-000000000040",
|
||||
"status": "queued",
|
||||
"observed_at_iso": "2025-01-15T08:10:00.000000+00:00",
|
||||
}
|
||||
if kind == "c11.upload.tile.rejected":
|
||||
return {
|
||||
"flight_id": "00000000-0000-0000-0000-000000000020",
|
||||
"tile_id": "z14_lat45.123456_lon-122.654321",
|
||||
"fingerprint": "0123456789abcdef",
|
||||
"batch_uuid": "00000000-0000-0000-0000-000000000040",
|
||||
"rejection_reason": "duplicate",
|
||||
"observed_at_iso": "2025-01-15T08:10:00.000000+00:00",
|
||||
}
|
||||
if kind == "c11.upload.batch.complete":
|
||||
return {
|
||||
"flight_id": "00000000-0000-0000-0000-000000000020",
|
||||
"fingerprint": "0123456789abcdef",
|
||||
"batch_uuid": "00000000-0000-0000-0000-000000000040",
|
||||
"outcome": "success",
|
||||
"total_attempted": 32,
|
||||
"total_queued": 32,
|
||||
"total_rejected": 0,
|
||||
"retry_count": 0,
|
||||
"observed_at_iso": "2025-01-15T08:10:00.000000+00:00",
|
||||
}
|
||||
raise AssertionError(f"unhandled kind in fixture: {kind!r}")
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user