mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 23:11:13 +00:00
chore: c6 docs-hygiene from cumulative_review_batches_28-30
Land F1+F2+F3 from the PASS_WITH_WARNINGS cumulative review of batches 28-30 (AZ-305 / AZ-307 / AZ-308) before continuing to batch 31. All three are bounded by the c6_tile_cache component; no public API contract change beyond the new error re-export. F1 (Medium / Architecture): Re-export CacheBudgetExhaustedError from c6_tile_cache package __init__ so consumers can catch the AZ-308 budget-exhaustion variant without widening to TileCacheError (which drops the needed_bytes / available_bytes / evicted_count diagnostics). F2 (Medium / Architecture): Refresh the c6_tile_cache section of module-layout.md so the Public API line and the Internal-files list reflect what is actually on disk after batches 28-30 (drop the stale Tile / TileRecord / connection.py entries; add the AZ-305 postgres_filesystem_store + tools.py, AZ-307 freshness_gate, AZ-308 cache_budget_enforcer entries; pivot the Public API bullet to the __init__.__all__ as canonical, mirroring the c7_inference section format). F3 (Low / Maintainability): Promote the triplicate intra-module _iso_ts_now() helper into a single c6_tile_cache._timestamp.iso_ts_now and import it from postgres_filesystem_store, freshness_gate, and cache_budget_enforcer. FDR record envelope ts format now has one source of truth. Test impact: tests/unit/c6_tile_cache: 105 passed, 57 skipped (pre-existing Docker-compose skip markers). No new tests required for F1/F2 (re-export + doc) and F3 (pure refactor; existing tests assert FDR record shape, not the helper symbol). Autodev state advanced to awaiting-invocation; next session resumes greenfield Step 7 at batch 31 (AZ-298 TensorrtRuntime). Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -137,17 +137,26 @@ Bootstrap reference: `_docs/02_tasks/todo/AZ-263_initial_structure.md`. Architec
|
||||
|
||||
- **Epic**: AZ-250 (E-C6 Tile Cache & Vector Index)
|
||||
- **Directory**: `src/gps_denied_onboard/components/c6_tile_cache/`
|
||||
- **Public API**:
|
||||
- `__init__.py` (re-exports `TileStore`, `Tile`, `TileQualityMetadata`, `TileRecord`, `SectorClassification`)
|
||||
- `interface.py` (`TileStore` Protocol — query/get/put surface; concrete impls swappable)
|
||||
- **Public API** (`__init__.py` `__all__`; consult the module for the canonical list):
|
||||
- `interface.py` (three Protocols: `TileStore`, `TileMetadataStore`, `DescriptorIndex` — query/get/put surface; concrete impls swappable)
|
||||
- DTOs: `TileId`, `TileMetadata`, `TileMetadataPersistent`, `TileQualityMetadata`, `Bbox`, `SectorBoundary`, `HnswParams`, `IndexMetadata`
|
||||
- Enums: `TileSource`, `FreshnessLabel`, `VotingStatus`, `SectorClassification`
|
||||
- ABC: `TilePixelHandle`
|
||||
- Config block: `C6TileCacheConfig` (registered on import)
|
||||
- Error family rooted at `TileCacheError` with documented subtypes (`TileNotFoundError`, `TileFsError`, `TileMetadataError`, `ContentHashMismatchError`, `FreshnessRejectionError`, `CacheBudgetExhaustedError`, `IndexUnavailableError`) + sibling `IndexBuildError` (offline build envelope, not in the `TileCacheError` family)
|
||||
- **Internal**:
|
||||
- `postgres_filesystem_store.py` (Postgres mirror + filesystem mmap + FAISS HNSW; production-default)
|
||||
- `_native/` (`cpp/faiss_index/` wrapper)
|
||||
- `migrations.py` (`apply_migrations(config) -> MigrationResult` runner invoked by the composition root at startup; AZ-304 + later)
|
||||
- `_uuid_namespace.py` (pinned `TILE_NAMESPACE_UUID` + `derive_tile_id` / `derive_location_hash` helpers; cross-repo coordinated with `satellite-provider`; AZ-304)
|
||||
- `connection.py` (planned — `psycopg_pool` ConnectionPool helper; AZ-305, not landed yet)
|
||||
- `_native/` (FAISS C++ wrapper, planned)
|
||||
- `_tile_pixel_handle.py` (`TilePixelHandle` ABC)
|
||||
- `_types.py` (DTOs / enums; consumed via the Public API re-exports)
|
||||
- `_uuid_namespace.py` (AZ-304 — pinned `TILE_NAMESPACE_UUID` + `derive_tile_id` / `derive_location_hash` helpers; cross-repo coordinated with `satellite-provider`)
|
||||
- `migrations.py` (AZ-304 — `apply_migrations(config) -> MigrationResult` runner invoked by the composition root at startup)
|
||||
- `postgres_filesystem_store.py` (AZ-305 — production-default `TileStore` + `TileMetadataStore` impl over Postgres mirror + filesystem; `PostgresFilesystemStore.from_config` opens its own `psycopg_pool.ConnectionPool` and constructs an `AZ-307 FreshnessGate`)
|
||||
- `freshness_gate.py` (AZ-307 — `ACTIVE_CONFLICT` reject + `STABLE_REAR` downgrade per `freshness_gate.md` v1.0.0)
|
||||
- `cache_budget_enforcer.py` (AZ-308 — RESTRICT-SAT-2 10 GiB hard cap; `CacheBudgetEnforcer.reserve_headroom` + `BudgetEnforcedTileStore` write-decorator)
|
||||
- `tools.py` (AZ-305 — operator dump CLI invoked via `python -m gps_denied_onboard.components.c6_tile_cache.tools ...`)
|
||||
- `errors.py`, `config.py` (component plumbing)
|
||||
- **Owns**: `src/gps_denied_onboard/components/c6_tile_cache/**`, `cpp/faiss_index/**`, `tests/unit/c6_tile_cache/**`, `db/migrations/**` (project-level Alembic env owned by c6 — `alembic.ini` at repo root points here; `0001_initial.py` shipped by AZ-263 bootstrap, `0002_c6_tile_identity_and_lru.py` and forward owned by AZ-304+ migrations)
|
||||
- **Imports from**: `_types`, `helpers.sha256_sidecar`, `helpers.wgs_converter`, `config`, `logging`, `fdr_client`
|
||||
- **Imports from**: `_types`, `helpers.sha256_sidecar`, `helpers.wgs_converter`, `clock`, `config`, `logging`, `fdr_client`
|
||||
- **Consumed by**: `c2_vpr`, `c2_5_rerank`, `c3_matcher`, `c10_provisioning`, `c11_tile_manager`, `runtime_root`
|
||||
|
||||
### Component: c7_inference
|
||||
|
||||
@@ -6,9 +6,9 @@ step: 7
|
||||
name: Implement
|
||||
status: in_progress
|
||||
sub_step:
|
||||
phase: 14
|
||||
name: batch-loop
|
||||
detail: "next: docs-hygiene mini-batch (F1+F2+F3 of cumulative_review_batches_28-30)"
|
||||
phase: 0
|
||||
name: awaiting-invocation
|
||||
detail: "next batch 31: AZ-298 TensorrtRuntime"
|
||||
retry_count: 0
|
||||
cycle: 1
|
||||
tracker: jira
|
||||
|
||||
@@ -39,6 +39,7 @@ from gps_denied_onboard.components.c6_tile_cache._types import (
|
||||
)
|
||||
from gps_denied_onboard.components.c6_tile_cache.config import C6TileCacheConfig
|
||||
from gps_denied_onboard.components.c6_tile_cache.errors import (
|
||||
CacheBudgetExhaustedError,
|
||||
ContentHashMismatchError,
|
||||
FreshnessRejectionError,
|
||||
IndexBuildError,
|
||||
@@ -60,6 +61,7 @@ register_component_block("c6_tile_cache", C6TileCacheConfig)
|
||||
__all__ = [
|
||||
"Bbox",
|
||||
"C6TileCacheConfig",
|
||||
"CacheBudgetExhaustedError",
|
||||
"ContentHashMismatchError",
|
||||
"DescriptorIndex",
|
||||
"FreshnessLabel",
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
"""RFC 3339 UTC timestamp helper shared inside the c6_tile_cache component.
|
||||
|
||||
Single source for the FDR record envelope ``ts`` field across
|
||||
``postgres_filesystem_store``, ``freshness_gate``, and
|
||||
``cache_budget_enforcer`` — formerly a triplicate ``_iso_ts_now`` per
|
||||
module (`cumulative_review_batches_28-30` F3). The format is wall-clock
|
||||
metadata about WHEN the FDR record was emitted; producers that need a
|
||||
Clock-driven payload field (e.g., ``age_seconds`` derived from an
|
||||
injected clock) MUST NOT route through this helper.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
__all__ = ["iso_ts_now"]
|
||||
|
||||
|
||||
def iso_ts_now() -> str:
|
||||
"""Return an RFC 3339 UTC timestamp with microsecond precision and a ``Z`` suffix."""
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
@@ -34,6 +34,7 @@ from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING, Final
|
||||
|
||||
from gps_denied_onboard.components.c6_tile_cache._timestamp import iso_ts_now
|
||||
from gps_denied_onboard.components.c6_tile_cache._types import (
|
||||
TileId,
|
||||
TileMetadata,
|
||||
@@ -80,16 +81,6 @@ class EvictionResult:
|
||||
freed_bytes: int
|
||||
|
||||
|
||||
def _iso_ts_now() -> str:
|
||||
"""RFC 3339 UTC timestamp with microsecond precision and ``Z`` suffix.
|
||||
|
||||
Used only on the FDR record envelope ``ts`` field — distinct from the
|
||||
per-row ``accessed_at`` / ``evicted_at`` datetimes which use the same
|
||||
wall-clock source but carry the operator-facing semantics.
|
||||
"""
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
|
||||
|
||||
class CacheBudgetEnforcer:
|
||||
"""LRU-driven 10 GiB hard-cap enforcer for the C6 tile cache.
|
||||
|
||||
@@ -300,7 +291,7 @@ class CacheBudgetEnforcer:
|
||||
self._fdr_client.enqueue(
|
||||
FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_ts_now(),
|
||||
ts=iso_ts_now(),
|
||||
producer_id=_PRODUCER_ID,
|
||||
kind="c6.eviction_batch",
|
||||
payload={
|
||||
|
||||
@@ -47,6 +47,7 @@ from typing import TYPE_CHECKING, Any, Final
|
||||
import psycopg
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
from gps_denied_onboard.components.c6_tile_cache._timestamp import iso_ts_now
|
||||
from gps_denied_onboard.components.c6_tile_cache._types import (
|
||||
Bbox,
|
||||
FreshnessLabel,
|
||||
@@ -116,16 +117,6 @@ class _Sector:
|
||||
return (self.bbox.max_lat - self.bbox.min_lat) * (self.bbox.max_lon - self.bbox.min_lon)
|
||||
|
||||
|
||||
def _iso_ts_now() -> str:
|
||||
"""RFC 3339 UTC timestamp with microsecond precision and ``Z`` suffix.
|
||||
|
||||
Used only on the FDR record envelope ``ts`` field, which is wall-clock
|
||||
metadata about WHEN the record was emitted — distinct from the
|
||||
Clock-driven ``age_seconds`` payload which uses the injected clock.
|
||||
"""
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
|
||||
|
||||
class FreshnessGate:
|
||||
"""Per-flight freshness gate (AZ-307).
|
||||
|
||||
@@ -374,7 +365,7 @@ class FreshnessGate:
|
||||
self._fdr_client.enqueue(
|
||||
FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_ts_now(),
|
||||
ts=iso_ts_now(),
|
||||
producer_id=_PRODUCER_ID,
|
||||
kind="c6.freshness.rejected",
|
||||
payload={
|
||||
@@ -393,7 +384,7 @@ class FreshnessGate:
|
||||
self._fdr_client.enqueue(
|
||||
FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_ts_now(),
|
||||
ts=iso_ts_now(),
|
||||
producer_id=_PRODUCER_ID,
|
||||
kind="c6.freshness.downgraded",
|
||||
payload={
|
||||
|
||||
@@ -45,6 +45,7 @@ from psycopg.rows import dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
from gps_denied_onboard.components.c6_tile_cache._timestamp import iso_ts_now
|
||||
from gps_denied_onboard.components.c6_tile_cache._tile_pixel_handle import (
|
||||
TilePixelHandle,
|
||||
)
|
||||
@@ -103,11 +104,6 @@ _ALLOWED_VOTING_TRANSITIONS = frozenset(
|
||||
)
|
||||
|
||||
|
||||
def _iso_ts_now() -> str:
|
||||
"""RFC 3339 UTC timestamp with microsecond precision and ``Z`` suffix."""
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
|
||||
|
||||
class MmapTilePixelHandle(TilePixelHandle):
|
||||
"""Read-only mmap view of a tile JPEG (Invariant I-4 read-only).
|
||||
|
||||
@@ -992,7 +988,7 @@ class PostgresFilesystemStore:
|
||||
)
|
||||
record = FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_ts_now(),
|
||||
ts=iso_ts_now(),
|
||||
producer_id=_PRODUCER_ID,
|
||||
kind="c6.write",
|
||||
payload={
|
||||
@@ -1020,7 +1016,7 @@ class PostgresFilesystemStore:
|
||||
message = message[:_MAX_FDR_FAILURE_MSG_LEN] + "...<truncated>"
|
||||
record = FdrRecord(
|
||||
schema_version=CURRENT_SCHEMA_VERSION,
|
||||
ts=_iso_ts_now(),
|
||||
ts=iso_ts_now(),
|
||||
producer_id=_PRODUCER_ID,
|
||||
kind="c6.write_failed",
|
||||
payload={
|
||||
|
||||
Reference in New Issue
Block a user