[AZ-320] Add C11 IdempotentRetryTileUploader decorator

Wraps HttpTileUploader (AZ-319) with two bounded retry budgets:

- In-call (per-batch) — re-invokes inner on PARTIAL outcome up to
  `max_in_call_retries` times with capped exponential backoff
  (`min(base ** attempt_number, cap)`). On exhaustion: surfaces an
  operator hint via `next_retry_at_s = now + backoff_cap_s`.
- Per-tile (cross-call) — atomically increments c6's
  `tiles.upload_attempts` counter for every rejection; once a tile
  hits `max_per_tile_attempts` it is forward-only transitioned to
  `voting_status = upload_giveup` (excluded from `pending_uploads`).
  Each transition emits FDR `kind="c11.upload.giveup"` plus an
  ERROR log.

C6 contract changes (AZ-303 v1.3.0):
- VotingStatus.UPLOAD_GIVEUP added (forward-only from PENDING/TRUSTED).
- TileMetadataStore.increment_upload_attempts(tile_id) -> int added
  with NotImplementedError default for backwards-compat.
- Migration 0003_c11_upload_attempts: additive column +
  widened ck_tiles_voting_status (preserves IS NULL clause).

C11 wiring:
- C11RetryConfig + disable_retry_decorator on C11Config.
- build_tile_uploader wraps in decorator by default; bypass flag
  returns the bare HttpTileUploader. New `clock` keyword.

Cross-component isolation honoured (AZ-507): the decorator declares
`_RetryMetadataStoreLike` Protocol cut over c6's TileMetadataStore
and references `UPLOAD_GIVEUP` via a local string constant — no c6
imports.

Tests: 13 decorator + 1 conformance + 2 factory bypass + AC-6 enum
update + alembic head bump + AZ-272 schema fixture. 238 passed across
c11/c6/fdr suites; pre-existing perf microbenches unrelated.

Code review: PASS_WITH_WARNINGS (5 Low/Informational findings,
docs-level or downstream-CI-blocked). See
_docs/03_implementation/reviews/batch_41_review.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-13 08:48:53 +03:00
parent 90f4ac78f4
commit a06b107fc3
19 changed files with 1788 additions and 21 deletions
@@ -11,10 +11,13 @@ import logging
import httpx
from gps_denied_onboard.clock.wall_clock import WallClock
from gps_denied_onboard.components.c11_tile_manager import (
C11Config,
C11RetryConfig,
HttpTileDownloader,
HttpTileUploader,
IdempotentRetryTileUploader,
)
from gps_denied_onboard.components.c11_tile_manager.interface import (
TileDownloader,
@@ -108,3 +111,39 @@ def test_ac10_concrete_downloader_satisfies_protocol() -> None:
def test_ac10_partial_downloader_is_not_protocol_conformant() -> None:
# Assert
assert not isinstance(_PartialDownloaderMissingEnumerate(), TileDownloader)
def test_ac9_idempotent_retry_decorator_satisfies_uploader_protocol() -> None:
# Arrange — wrap a Protocol-conformant inner uploader; the decorator
# must itself satisfy ``TileUploader`` so the composition root can
# bind it transparently in place of ``HttpTileUploader``.
cfg = C11Config(
satellite_provider_ingest_url="https://parent-suite.test",
upload_batch_size=10,
upload_http_timeout_s=5.0,
upload_max_retry_after_s=600,
companion_id="conformance",
)
transport = httpx.MockTransport(lambda r: httpx.Response(202))
inner = HttpTileUploader(
http_client=httpx.Client(transport=transport),
tile_store=object(), # type: ignore[arg-type]
tile_metadata_store=object(), # type: ignore[arg-type]
flight_state_gate=object(), # type: ignore[arg-type]
key_manager=object(), # type: ignore[arg-type]
fdr_client=FakeFdrSink(_PRODUCER_ID), # type: ignore[arg-type]
logger=logging.getLogger("test_az320_inner"),
config=cfg,
sleep=_NullSleep(),
)
decorator = IdempotentRetryTileUploader(
inner=inner,
tile_metadata_store=object(), # type: ignore[arg-type]
fdr_client=FakeFdrSink("c11_tile_manager.idempotent_retry"), # type: ignore[arg-type]
logger=logging.getLogger("test_az320_decorator"),
clock=WallClock(),
config=C11RetryConfig(),
)
# Assert
assert isinstance(decorator, TileUploader)