- Changed autodev state to reflect the transition from batch 26 to batch 27, updating the phase and details for the compute-batch step. - Incremented the version of the tile metadata store from 1.0.0 to 1.1.0, refining the uniqueness invariant to use a natural key that includes flight_id, allowing coexistence of multiple rows for the same tile from different flights. - Updated the last modified date in the tile metadata store documentation to reflect recent changes. Co-authored-by: Cursor <cursoragent@cursor.com>
37 KiB
C6 Postgres Schema — Additive 0002 Migration (Identity + LRU + Freshness Rules)
Task: AZ-304_c6_postgres_schema
Name: C6 Postgres Schema — Identity, LRU, and Freshness Rules (additive 0002)
Description: Author the additive Alembic migration 0002_c6_tile_identity_and_lru.py on top of the AZ-263 bootstrap baseline (db/migrations/versions/0001_initial.py). This migration is strictly additive per data_model.md § 6.1 / § 6.3: every existing AZ-263 column, table, index, and CHECK is preserved unchanged. The migration adds the deterministic-identity columns (tile_uuid, location_hash), the content-hash chain (content_sha256), the LRU + disk-budget bookkeeping columns (disk_bytes, accessed_at, uploaded_at), the per-flight natural-key UNIQUE constraint with COALESCE-zero-uuid semantics, the sector-boundary geometry columns (NULLable additions on the existing sector_classifications table), the new tile_freshness_rules table seeded with the default thresholds, and the additive widening of the tiles.freshness_status CHECK to accept the AZ-303 DTO FreshnessLabel vocabulary alongside the existing AZ-263 values. Ships the pinned UUIDv5 namespace module used by both onboard inserts and cross-workspace satellite-provider integrations, the migration runner the composition root invokes at startup, the schema-shape diff test, and the AZ-303 contract bump from v1.1.0 → v1.2.0 (adds location_hash field to TileMetadata).
Complexity: 3 points
Dependencies: AZ-303_c6_storage_interfaces (DTO contract — bumped here), AZ-263_initial_structure (baseline 0001_initial migration this builds on), AZ-269_config_loader, AZ-266_log_module
Component: c6_tile_cache (epic AZ-250 / E-C6)
Tracker: AZ-304
Epic: AZ-250 (E-C6)
Document Dependencies
_docs/02_document/contracts/c6_tile_cache/tile_metadata_store.md— bumped here v1.1.0 → v1.2.0 (addslocation_hashfield). Invariant I-1 already expresses the integer-slippy-tile + COALESCE-zero-uuid natural key this task wires into the DB._docs/02_document/contracts/c6_tile_cache/tile_store.md— definescontent_sha256_hex(str) thetiles.content_sha256column carries; unchanged this cycle._docs/02_document/contracts/shared_config/composition_root_protocol.md—config.tile_cache.postgres_dsnfield._docs/02_document/contracts/shared_logging/log_record_schema.md— INFO log shape on migration apply / no-op._docs/02_document/data_model.md— system-wide data model; § 4.4 (migration baseline ordering), § 6.1 (additive-only by default), § 6.3 (tilescanonical columns frozen) are the governing rules this task obeys.db/migrations/versions/0001_initial.py— AZ-263 baseline this migration extends additively._docs/_process_leftovers/2026-05-12_tile-schema-scenario-analysis.md— cross-workspace scenario analysis that established the UUIDv5 +location_hashdesign; the cross-workspace counterpartAZ-TBD_tile_identity_uuidv5_bulk_listlives insatellite-provider/_docs/02_tasks/todo/.
Problem
The AZ-263 bootstrap baseline shipped the tiles, flights, sector_classifications, manifests, and engine_cache_entries tables, but the canonical AZ-303 TileMetadataStore contract (v1.1.0) requires the schema to also carry:
- a deterministic UUIDv5 identifier byte-equal to
satellite-provider's tile id, so cross-workspace correlation (D-PROJ-2 ingest, voting, post-landing upload) does not need a separate join table; - a
location_hashcell-bag identifier forWHERE location_hash = ?cell-bag queries (Scenario 1 UI lookup + Scenario 6 voting query in the 2026-05-12 leftover); - the per-flight natural-key UNIQUE constraint (
(zoom_level, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, zero_uuid))) — without it, twoonboard_ingestrows for the same cell from different flights collapse, destroying the per-flight evidence the future D-PROJ-2 voting layer needs; - the
content_sha256content-hash column the AZ-280 atomic-write/sidecar pattern chains into; - the LRU + disk-budget bookkeeping columns (
disk_bytes,accessed_at,uploaded_at) theTileMetadataStore.record_lru_access/total_disk_bytes/pending_uploadsmethods read and write; - a freshness-rules table the freshness gate (separate task) reads at insert time;
- geometry on
sector_boundariesrecords so the freshness gate can perform tile-in-sector containment checks againstSectorBoundaryDTO instances — AZ-263 shippedsector_classificationswith a flatsector_idtext column and no bbox geometry.
Additionally, the AZ-303 DTO vocabulary for FreshnessLabel (fresh, stale_active_conflict, stale_rear, downgraded) differs from the AZ-263 baseline freshness_status CHECK (fresh, stale_warn, stale_reject). Because data_model.md § 6.1 forbids removing CHECK enum values without an ADR, this task widens the CHECK to the UNION of both vocabularies (a CHECK loosening, which IS additive); a future cycle may deprecate the legacy values via ADR-gated cleanup.
This task delivers the strictly-additive 0002 migration that closes those gaps without renaming, retyping, or dropping any AZ-263 column, table, index, or CHECK.
Outcome
- An Alembic migration script at
db/migrations/versions/0002_c6_tile_identity_and_lru.py(forwardupgrade()is purely additive; reversedowngrade()drops the additions and restores the original AZ-263freshness_statusCHECK). The migration is idempotent against a DB at AZ-263 head; Alembic rejects double-application via the standardalembic_versionrow. - The migration runner
apply_migrations(config) -> MigrationResultatsrc/gps_denied_onboard/components/c6_tile_cache/migrations.py, invoked by the composition root at startup AFTER config load and BEFOREPostgresFilesystemStoreconstruction. ReturnsMigrationResult(applied: list[str], current_revision: str, no_op: bool). Logs INFO on every applied revision; logs INFO withno_op=Truewhen the DB is already at head. - The pinned UUIDv5 namespace module at
src/gps_denied_onboard/components/c6_tile_cache/_uuid_namespace.pyexportingTILE_NAMESPACE_UUID = UUID("5b8d0c2e-1a4f-4b3a-8c9d-e7f6a3b2c1d0"),derive_tile_id(zoom_level, tile_x, tile_y, source, flight_id) -> UUID, andderive_location_hash(zoom_level, tile_x, tile_y) -> UUID. The namespace value is cross-repo coordinated withsatellite-provider/SatelliteProvider.Common/Utils/Uuidv5.csperAZ-TBD_tile_identity_uuidv5_bulk_list; the sameuuidv5(NAMESPACE, name)MUST produce byte-identical output on both sides. - The psycopg_pool connection helper at
src/gps_denied_onboard/components/c6_tile_cache/connection.py(psycopg_pool(config) -> psycopg_pool.ConnectionPool), used by both this task's runner and the futurePostgresFilesystemStore. - After
apply_migrations(config)on an AZ-263-baselined DB:- The
tilestable has six additional columns (tile_uuid,location_hash,content_sha256,disk_bytes,accessed_at,uploaded_at); the AZ-263 columns are unchanged. - The
tilestable has a new UNIQUE indexidx_tiles_natural_keyover the COALESCE-zero-uuid natural key; the AZ-263 indices are unchanged. - The
tilestable has four new indices (idx_tiles_location_hash,idx_tiles_tile_uuidfrom the UNIQUE,idx_tiles_accessed_at,idx_tiles_pending_uploadpartial,idx_tiles_flight_capturedpartial — see Schema Additions). - The
tiles.freshness_statusCHECK is widened to the UNION vocabulary; AZ-263 rows (none exist on greenfield apply) would continue to validate. - The
sector_classificationstable has four additional NULLable bbox columns; the AZ-263 columns are unchanged. - A new
tile_freshness_rulestable exists, seeded with the two default rows.
- The
- The AZ-303 contract
tile_metadata_store.mdis bumped v1.1.0 → v1.2.0 with thelocation_hash: UUID | Nonefield added to the documentedTileMetadatashape (non-breaking minor — Optional defaultNone, populated byPostgresFilesystemStore.insert_metadatafrom_uuid_namespace.derive_location_hashwhen not supplied). - The DTO
TileMetadatainsrc/gps_denied_onboard/components/c6_tile_cache/_types.pygains the samelocation_hash: UUID | None = Nonefield (positional last, default value preserves existing constructor call sites). - A schema fixture
tests/fixtures/c6_postgres_schema_v2.sqlis the human-readable expected post-0002 DDL used by the schema-shape diff test.
Scope
Included
- The Alembic migration
0002_c6_tile_identity_and_lru.py(additive only; the migration body usesop.add_column,op.create_unique_constraint/op.create_indexfor the UNIQUE expression index,op.create_indexfor new btree indices,op.create_tablefortile_freshness_rules,op.bulk_insertfor the seed rows, and a singleop.drop_constraint+op.create_check_constraintpair to widenck_tiles_freshness_status). - A
MigrationResultdataclass@dataclass(frozen=True)atc6_tile_cache.migrations. - The
apply_migrations(config) -> MigrationResultrunner using the existing project-pinned Alembic env atdb/migrations/(no new alembic.ini, no new env.py — AZ-263 bootstrap owns those; this task only wirestarget_metadataintodb/migrations/env.pyso future autogenerate diffs work). - The pinned UUIDv5 namespace module
_uuid_namespace.pywithTILE_NAMESPACE_UUID,derive_tile_id,derive_location_hash. No Postgres dependency; pure stdlibuuid.uuid5. - The schema-shape diff test
tests/unit/c6_tile_cache/test_postgres_schema.pythat introspects a freshly-migrated test DB (Alembic upgraded to0002head) and asserts every column, index, CHECK constraint, and seed row matchestests/fixtures/c6_postgres_schema_v2.sql. Test usestestcontainers-managed Postgres 16; no Python logic beyondinformation_schemaqueries. - The UUIDv5 determinism test
tests/unit/c6_tile_cache/test_uuid_namespace.pythat locksTILE_NAMESPACE_UUIDand verifies ≥5 fixed(z, x, y, source, flight_id)input vectors produce the documented UUIDv5 outputs. These vectors are the cross-repo coordination evidence — the correspondingsatellite-providertest MUST produce byte-identical UUIDs. - The connection helper
connection.psycopg_pool(config) -> psycopg_pool.ConnectionPool. - Wiring of
db/migrations/env.pytarget_metadatato ac6_tile_cache.metadataSQLAlchemyMetaDataobject that reflects both AZ-263 and AZ-304 schema (so future autogenerate diffs are mechanically comparable). - The schema-fixture file
tests/fixtures/c6_postgres_schema_v2.sql— copy-pastable DDL the test diffs against. - DTO extension in
_types.py:TileMetadata.location_hash: UUID | None = None(positional last, defaultNone). - Contract bump
_docs/02_document/contracts/c6_tile_cache/tile_metadata_store.mdv1.1.0 → v1.2.0 with a Change Log entry.
Excluded
- Concrete
PostgresFilesystemStore(insert / query / mark methods) — separate task (c6_postgres_filesystem_store). That task is responsible for: (a) computingtile_x/tile_yfrom theTileIdWGS84 coordinates via the project's shared Web-Mercator helper, (b) derivingtile_uuid/location_hashvia_uuid_namespace.derive_*, (c) mapping DTO field names to AZ-263 column names (quality_metadata→tile_quality_metadata,freshness_label→freshness_status,(lat, lon)→(latitude, longitude)), and (d) validating cross-side UUIDv5 parity withsatellite-provider. - The freshness gate logic reading
sector_classifications+tile_freshness_rules— separate task (c6_freshness_gate). - The LRU eviction policy reading
accessed_at— separate task (c6_cache_budget_eviction). - FAISS index file format — separate task (
c6_faiss_descriptor_index). - Sector-boundary CRUD (operator-side INSERT/UPDATE of geometry columns) — owned by C12; this task only adds the columns as NULLable.
- Per-flight DB lifecycle (drop-and-rebuild between flights, freshness-rules reload) — owned by the composition root's startup orchestration; this task only applies migrations idempotently.
- A third migration revision — every future schema change is a NEW migration file; this task only ships
0002. - Postgres tuning (work_mem, shared_buffers) — handled by deployment / Dockerfile (E-DEPLOY); the schema is portable across reasonable Postgres 16 configurations.
- Postgres-version migration (16 → 17) — out of scope this cycle; the schema MUST work on 16.x.
- Backfill of the legacy
freshness_statusenum values (stale_warn,stale_reject) to the new vocabulary — deferred until an ADR explicitly decides the deprecation path; this task widens the CHECK without committing to a value-mapping rewrite. - Renaming any AZ-263 column or table — forbidden per
coderule.mdcanddata_model.md§ 6.2; the DTO-to-column mapping lives inPostgresFilesystemStore.
Schema Additions (0002 on top of AZ-263 0001_initial)
All additions below are strictly additive. The migration MUST NOT drop, rename, or retype any AZ-263 column, table, index, or CHECK; the only constraint change is a CHECK widening (loosening) for tiles.freshness_status, which is additive per data_model.md § 6.1.
tiles — additive columns
| Column | Type | Nullable | Default | Notes |
|---|---|---|---|---|
tile_uuid |
UUID |
NO | — | Deterministic uuidv5(TILE_NAMESPACE_UUID, "{zoom_level}/{tile_x}/{tile_y}/{source}/{flight_id_or_zero_uuid}"). Cross-repo correlation key with satellite-provider. Populated at INSERT by PostgresFilesystemStore. UNIQUE. |
location_hash |
UUID |
NO | — | uuidv5(TILE_NAMESPACE_UUID, "{zoom_level}/{tile_x}/{tile_y}"). Identifies a Web-Mercator cell across sources / flights. Populated at INSERT. |
content_sha256 |
TEXT |
NO | — | 64 hex chars; matches tile_store.md content_sha256_hex invariant. CHECK length(content_sha256) = 64. |
disk_bytes |
BIGINT |
NO | — | On-disk JPEG body size; populated by write_tile. CHECK disk_bytes >= 0. |
accessed_at |
TIMESTAMPTZ |
NO | now() |
LRU clock; updated by record_lru_access. |
uploaded_at |
TIMESTAMPTZ |
YES | NULL | Set by mark_uploaded; NULL until C11 TileUploader confirms post-flight upload. |
Empty-DB assumption: at 0002 apply time the tiles table is empty (greenfield); the four NOT-NULL columns without server defaults are added with nullable=False directly. On a hypothetical non-empty DB the migration would need a 2-phase split (add nullable → backfill → alter NOT NULL); this is documented but explicitly out of scope for this cycle.
tiles — natural-key UNIQUE (additive index)
A UNIQUE expression index idx_tiles_natural_key over
(zoom_level, tile_x, tile_y, tile_size_meters, source,
COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid))
Two onboard_ingest rows for the same (z, tile_x, tile_y, tile_size_meters) from different flight_id values coexist (required by the future D-PROJ-2 voting layer). Two googlemaps rows for the same cell (both flight_id = NULL → both coalesce to the zero UUID) cannot coexist. The index is implemented via op.create_index(..., unique=True, postgresql_using='btree') with the expression list literal.
tiles — CHECK widening (additive loosen)
The existing ck_tiles_freshness_status CHECK is dropped and recreated with the UNION vocabulary:
freshness_status IN (
'fresh', 'stale_warn', 'stale_reject', -- AZ-263 vocabulary (legacy)
'stale_active_conflict', 'stale_rear', 'downgraded' -- AZ-303 FreshnessLabel vocabulary
)
A CHECK widening (more values accepted) is an additive loosening; any row that satisfied the prior CHECK still satisfies the new one. The legacy values are retained until an ADR-gated future cycle deprecates them. The constraint name stays ck_tiles_freshness_status to keep the AZ-263 fixture diffable.
tiles — new indices (additive)
| Index | Definition | Purpose |
|---|---|---|
idx_tiles_natural_key |
UNIQUE btree over the COALESCE natural-key expression (above) | I-1 of tile_metadata_store.md |
idx_tiles_location_hash |
btree over (location_hash) |
Scenario 1 UI cell-bag lookup, Scenario 6 voting query |
idx_tiles_accessed_at |
btree over (accessed_at) |
lru_candidates ordering |
idx_tiles_pending_upload |
partial btree over (uploaded_at) WHERE source = 'onboard_ingest' AND uploaded_at IS NULL |
pending_uploads |
idx_tiles_flight_captured |
partial btree over (flight_id, capture_timestamp) WHERE flight_id IS NOT NULL |
Scenario 8 per-flight inspection |
The AZ-263 indices (ix_tiles_zxy, ix_tiles_lat_lon, ix_tiles_voting_status_onboard, ix_tiles_flight_id, ix_tiles_created_at) are unchanged.
sector_classifications — additive columns (geometry, NULLable)
| Column | Type | Nullable | Default | Notes |
|---|---|---|---|---|
min_lat |
DOUBLE PRECISION |
YES | NULL | bbox south edge — populated pre-flight by C12 |
min_lon |
DOUBLE PRECISION |
YES | NULL | bbox west edge |
max_lat |
DOUBLE PRECISION |
YES | NULL | bbox north edge |
max_lon |
DOUBLE PRECISION |
YES | NULL | bbox east edge |
All NULLable because existing AZ-263 rows (if any) lack geometry; C12 populates pre-flight before the freshness gate reads. The AZ-263 columns (sector_id, classification, freshness_threshold_days) are unchanged. No CHECK on min ≤ max yet — that arrives when C12 enforces population (would be a tightening, ADR-gated).
The SectorBoundary DTO from _types.py maps bbox.min_lat → min_lat, etc.; classification maps directly. The DTO field sector_id is not present in the DTO today because the in-memory SectorBoundary does not need it; the impl task resolves DB row ↔ DTO via the sector_id text PK component.
New table: tile_freshness_rules
| Column | Type | Nullable | Default | Notes |
|---|---|---|---|---|
classification |
TEXT |
NO | — | PK; matches sector_classifications.classification |
max_age_seconds |
BIGINT |
NO | — | Per-classification threshold |
action |
TEXT |
NO | — | CHECK action IN ('reject','downgrade') |
set_at |
TIMESTAMPTZ |
NO | now() |
Constraints:
PRIMARY KEY (classification)CHECK (action IN ('reject', 'downgrade'))CHECK (max_age_seconds > 0)
Default rows seeded by the migration (AC-7):
('active_conflict', 15552000, 'reject')— 6 × 30 × 86400 seconds = 6 months('stable_rear', 31104000, 'downgrade')— 12 × 30 × 86400 seconds = 12 months
AZ-303 contract bump v1.1.0 → v1.2.0: TileMetadata.location_hash
- DTO change in
_types.py: addlocation_hash: UUID | None = NonetoTileMetadata(positional last, defaultNone).PostgresFilesystemStore.insert_metadatacomputes the value via_uuid_namespace.derive_location_hash(...)whenNone; uses the supplied value when present. - Contract change in
tile_metadata_store.md: bump version to v1.2.0; addlocation_hash: UUID | Nonerow to the DTO field table; add a "v1.2.0" Change Log row referencing this task and the leftover. - Non-breaking: existing constructors and
insinstance(impl, TileMetadataStore)checks continue to work.
Acceptance Criteria
AC-1: Migration is idempotent against an AZ-263-baselined DB
Given a Postgres 16 database at AZ-263 head (alembic_version = '0001_initial')
When apply_migrations(config) runs
Then the additive columns / indices / table / CHECK widening exist; the alembic_version row carries 0002_c6_tile_identity_and_lru; MigrationResult.applied == ['0002_c6_tile_identity_and_lru']; MigrationResult.no_op == False; AZ-263 columns / indices / CHECKs are byte-identical to their pre-migration state.
AC-2: Migration is no-op when at head
Given a Postgres DB already at 0002_c6_tile_identity_and_lru
When apply_migrations(config) runs again
Then MigrationResult.applied == []; MigrationResult.no_op == True; no DDL is emitted (verifiable via pg_stat_user_tables row counts unchanged).
AC-3: Schema shape matches the documented DDL
Given a DB upgraded through 0001 + 0002
When the schema-shape diff test introspects information_schema.columns / pg_indexes / pg_constraint / tile_freshness_rules row contents
Then every AZ-263 column / index / CHECK is present and unchanged; every additive column / index / CHECK from this task is present; the ck_tiles_freshness_status CHECK contains the UNION vocabulary; tile_freshness_rules has exactly two seeded rows with the documented values. Diff against tests/fixtures/c6_postgres_schema_v2.sql is empty.
AC-4: Natural-key uniqueness enforces per-flight separation
Given a tiles table after 0002
When two INSERTs with the same (zoom_level, tile_x, tile_y, tile_size_meters, source='onboard_ingest') and different flight_id values are attempted
Then both INSERTs succeed; the resulting rows have different tile_uuid values (different UUIDv5 inputs) and the same location_hash (UUIDv5 inputs differ only in flight_id/source).
AC-4b: Natural-key uniqueness rejects duplicate flight inserts
Given a tiles table after 0002
When two INSERTs with the same (zoom_level, tile_x, tile_y, tile_size_meters, source, flight_id) (or both flight_id = NULL for googlemaps) are attempted with different content_sha256 values
Then the second INSERT raises psycopg.errors.UniqueViolation (idx_tiles_natural_key); the first row is unaffected.
AC-5: AZ-263 CHECK constraints survive widening
Given the post-0002 tiles table
When INSERT INTO tiles (...freshness_status...) VALUES ('fresh') runs (and separately for 'stale_warn', 'stale_reject', 'stale_active_conflict', 'stale_rear', 'downgraded')
Then all six values are accepted; an INSERT ... VALUES ('bogus') is rejected by the widened ck_tiles_freshness_status CHECK.
AC-6: Down migration reverses cleanly
Given a DB at 0002_c6_tile_identity_and_lru
When alembic downgrade -1 runs (operator-only command; not exercised by the runtime)
Then the additive columns / indices / tile_freshness_rules table are dropped; the ck_tiles_freshness_status CHECK is restored to the AZ-263 vocabulary; the DB returns to byte-identical AZ-263 state; subsequent upgrade re-applies cleanly.
AC-7: Default freshness rules are seeded
Given a DB upgraded through 0002
When the schema-shape test queries tile_freshness_rules
Then exactly two rows exist: ('active_conflict', 15552000, 'reject') and ('stable_rear', 31104000, 'downgrade').
AC-8: Migration runner logs INFO on apply and no-op
Given a DB at AZ-263 head
When apply_migrations runs and then runs again
Then the first call emits an INFO log with kind="c6.migration.applied" carrying revisions=['0002_c6_tile_identity_and_lru']; the second call emits an INFO log with kind="c6.migration.no_op". Both calls also log the resolved TILE_NAMESPACE_UUID value once for post-mortem drift detection.
AC-9: AZ-263 columns are byte-identical after upgrade
Given a DB at AZ-263 head + a recorded snapshot of information_schema.columns for tiles, flights, sector_classifications, manifests, engine_cache_entries
When apply_migrations(config) runs
Then for every AZ-263 column the snapshot and the post-0002 row are byte-identical (column name, data type, nullability, default expression). New columns appear only as additions.
AC-10: UUIDv5 derivation is deterministic and cross-repo coordinated
Given the pinned TILE_NAMESPACE_UUID = 5b8d0c2e-1a4f-4b3a-8c9d-e7f6a3b2c1d0
When derive_tile_id and derive_location_hash are called for the documented fixed test vectors (≥5 (zoom_level, tile_x, tile_y, source, flight_id) tuples in tests/unit/c6_tile_cache/test_uuid_namespace.py)
Then each call returns the documented UUID byte-for-byte; the same Python expression run twice produces identical output; the locked test vectors are the cross-repo coordination evidence used to verify satellite-provider's C# implementation produces the same UUIDs.
AC-11: location_hash is invariant across source and flight_id
Given three rows for the same (zoom_level, tile_x, tile_y) — one source='googlemaps' and two source='onboard_ingest' from different flight_id values
When the test queries SELECT DISTINCT location_hash FROM tiles WHERE zoom_level=? AND tile_x=? AND tile_y=?
Then exactly ONE location_hash value is returned (the three rows share the same cell-bag identifier).
AC-12: TileMetadata.location_hash default is None (non-breaking DTO bump)
Given the v1.2.0 TileMetadata DTO
When existing AZ-303-style positional constructors run (without supplying location_hash)
Then construction succeeds with location_hash = None; when constructors supply an explicit UUID, the value is preserved on the frozen instance.
Non-Functional Requirements
Performance
- Migration apply ≤ 5 s on an AZ-263-baselined Postgres 16 database with empty
tiles. The migration body is bounded by index creation on an empty table, fourALTER TABLE … ADD COLUMNs, oneop.create_table, twoop.bulk_insertrows, and one CHECK widening — all O(1) in row count. apply_migrationsno-op call (DB at head) ≤ 100 ms.- The new UNIQUE expression index
idx_tiles_natural_keydoes not regress AZ-263 query plans on the existingix_tiles_zxybtree; both coexist and the planner selects based on predicate shape.
Compatibility
- Postgres 16.x.
psycopg_pool3.x — pinned by AZ-263; this task adds no new third-party dependencies.- Alembic 1.13+ — pinned by AZ-263.
- Cross-workspace UUIDv5 namespace:
TILE_NAMESPACE_UUIDMUST be byte-identical to the satellite-provider C# constant; any change requires a coordinated cross-repo release.
Reliability
- The migration is wrapped in a single Alembic transaction (Postgres default). A crash mid-migration leaves the DB at the prior revision (
0001_initial). - The runner catches
psycopg.errors.SerializationFailureand retries once with exponential backoff; after the second failure, raisesMigrationError(defined inc6_tile_cache.migrations, NOT inTileCacheError— migrations run before any runtime error consumer is constructed).
Unit Tests
| AC Ref | What to Test | Required Outcome |
|---|---|---|
| AC-1 | apply_migrations against fresh testcontainer DB previously upgraded to AZ-263 head |
All additive columns / indices / tile_freshness_rules table exist; alembic_version='0002_c6_tile_identity_and_lru'; result.applied=['0002_c6_tile_identity_and_lru']. AZ-263 columns / indices / CHECKs byte-identical to pre-migration snapshot. |
| AC-2 | apply_migrations against DB already at 0002 head |
result.applied=[]; result.no_op=True; no DDL emitted. |
| AC-3 | Introspect information_schema / pg_indexes / pg_constraint / tile_freshness_rules rows; diff against tests/fixtures/c6_postgres_schema_v2.sql |
Zero diff. |
| AC-4 | Two onboard_ingest INSERTs with same (z, tile_x, tile_y, tile_size_meters) and different flight_id |
Both succeed; same location_hash; different tile_uuid. |
| AC-4b | Two INSERTs with identical natural-key tuple (same flight_id or both NULL for googlemaps) |
Second INSERT raises psycopg.errors.UniqueViolation. |
| AC-5 | INSERT one row each for 'fresh', 'stale_warn', 'stale_reject', 'stale_active_conflict', 'stale_rear', 'downgraded'; INSERT one row with 'bogus' |
First six succeed; last raises psycopg.errors.CheckViolation. |
| AC-6 | alembic downgrade -1 then upgrade head |
DB returns to AZ-263-byte-identical state; subsequent upgrade re-applies cleanly. |
| AC-7 | SELECT tile_freshness_rules after migration |
Exactly 2 rows with documented values. |
| AC-8 | Capture log records during migration apply + no-op | INFO records with kind="c6.migration.applied" and kind="c6.migration.no_op"; namespace UUID emitted on apply. |
| AC-9 | Snapshot AZ-263 information_schema.columns before 0002; compare after |
Snapshot rows byte-identical post-migration; new column rows are additions only. |
| AC-10 | derive_tile_id / derive_location_hash for 5 fixed input vectors |
Outputs match the documented UUIDs byte-for-byte; idempotent on second call. |
| AC-11 | INSERT three rows sharing (z, tile_x, tile_y) from different (source, flight_id) |
Exactly one distinct location_hash value. |
| AC-12 | Construct TileMetadata without location_hash and with explicit location_hash=uuid4() |
First yields location_hash=None; second preserves the supplied value; both instances are frozen. |
| NFR-perf-apply | Migration apply on AZ-263-baselined empty tiles |
Wall ≤ 5 s |
| NFR-perf-noop | apply_migrations no-op timing |
Wall ≤ 100 ms |
| NFR-reliability-retry | Inject SerializationFailure once, then succeed |
Migration succeeds on retry; on second failure raises MigrationError. |
Constraints
- Postgres 16.x ONLY this cycle; no SQLite / no MySQL fallback.
- Alembic +
psycopg_poolare pinned by AZ-263; this task does NOT introduce new third-party dependencies. - The migration MUST be reversible (
downgradedrops the additions cleanly and restores the AZ-263 CHECK) — operator post-flight tooling depends on it for "drop-and-rebuild" flows. - The migration MUST be strictly additive on every AZ-263 column, table, index, and CHECK per
data_model.md§ 6.1 / § 6.3 andcoderule.mdc. The single allowed constraint mutation is theck_tiles_freshness_statusCHECK widening, which is additive in semantic effect. pgcryptoextension is NOT required by 0002 (nogen_random_uuid()use);tile_freshness_rules.classificationis a TEXT PK and the seeded rows are static.MigrationErroris NOT a member of theTileCacheErrorfamily — migrations run before anyc6_tile_cache.errorsconsumer is constructed.- The schema-fixture file
tests/fixtures/c6_postgres_schema_v2.sqlis the diff target; updating it without a new migration revision is a Spec-Gap finding (High) at code-review time. - The pinned
TILE_NAMESPACE_UUIDMUST NOT be regenerated by this task. The value5b8d0c2e-1a4f-4b3a-8c9d-e7f6a3b2c1d0is locked here; subsequent edits require a coordinated cross-workspace release. - The
latitude/longitudecolumns (AZ-263 names) remain advisory;lat/lonare NOT introduced. The DTOTileId(zoom_level, lat, lon)maps viaPostgresFilesystemStoreserialisation; the schema is NOT changed to match the DTO field name. - The
freshness_statuscolumn name (AZ-263) is NOT renamed tofreshness_label. DTO fieldfreshness_label: FreshnessLabelmaps viaPostgresFilesystemStoreserialisation. - The
tile_quality_metadatacolumn name (AZ-263) is NOT renamed toquality_metadata. DTO fieldquality_metadata: TileQualityMetadata | Nonemaps viaPostgresFilesystemStoreserialisation. - The
tiles.id BIGSERIALPK is NOT replaced. The deterministic UUIDv5 lives in the additivetile_uuidcolumn; cross-workspace correlation usestile_uuid(notid). Satellite-provider may use UUIDv5 as its own PK; the wire-format correlation key is the UUIDv5 value, not the column name. - The migration assumes empty
tilesat apply time (greenfield). NOT-NULL adds on existing rows would require a 2-phase migration (add nullable + backfill + alter NOT NULL); explicitly out of scope. - AZ-263 legacy
freshness_statusvalues (stale_warn,stale_reject) are NOT deprecated by this task. A future ADR-gated migration may tighten the CHECK once any rows carrying those values are backfilled.
Risks & Mitigation
Risk 1: AZ-263 baseline drift between dev and CI breaks the additive assumption
- Risk: A developer manually edits
0001_initial.pyafter AZ-263 merged; 0002 fails to apply because expected base state diverges. - Mitigation: AC-9 snapshots AZ-263 column metadata before 0002 and asserts byte-identity post-migration. AC-3 diffs full schema against
c6_postgres_schema_v2.sql. CI catches drift before merge.
Risk 2: Empty-DB assumption violated in dev fixtures
- Risk: A dev or test fixture inserts
tilesrows between0001_initialand0002_c6_tile_identity_and_lru; the NOT-NULL additive columns fail. - Mitigation: Document the empty-DB assumption in Constraints and in the migration's docstring. The composition root applies migrations BEFORE any seed step. Test fixtures use
apply_migrationsfirst, then INSERT.
Risk 3: Down-migration data loss is irreversible if rows exist
- Risk: Operator runs
alembic downgrade -1on a DB with live tiles; thetile_uuid/location_hash/content_sha256columns are dropped, destroying cross-workspace correlation evidence. - Mitigation: Downgrade is documented operator-only and destructive. The composition root only calls
upgrade headat runtime. Document in the migration docstring.
Risk 4: UNIQUE expression index on COALESCE is slow or unsupported
- Risk: The COALESCE-zero-uuid expression index is unusual; Postgres 16 supports it but plan-tooling may not recognise it for query optimisation.
- Mitigation: AC-4 / AC-4b exercise the index directly. The expression is deterministic and Postgres documents COALESCE-expression indices as supported since v8.0; no version-specific risk on 16.x.
Risk 5: CHECK widening creates ambiguity in downstream consumers
- Risk: Code reading
freshness_statusmay not handle the new 4-value subset and crash on legacystale_warn/stale_rejectrows (or vice-versa). - Mitigation: No legacy rows exist on greenfield. The DTO
FreshnessLabelenum is the canonical onboard vocabulary;PostgresFilesystemStoreis responsible for the value-mapping policy. Document the legacy values as deprecated-pending-ADR in the contract bump.
Risk 6: UUIDv5 namespace divergence between onboard Python and satellite-provider C#
- Risk: A subtle bug in the cross-repo C# UUIDv5 implementation produces UUIDs that differ from Python's
uuid.uuid5. Cross-repo lookups fail silently. - Mitigation: AC-10 locks 5+ fixed vectors with documented expected output. The corresponding
satellite-providertest (perAZ-TBD_tile_identity_uuidv5_bulk_listAC-1) asserts the same vectors produce byte-identical output.
Risk 7: AZ-303 DTO location_hash default None shadowing impl bugs
- Risk:
PostgresFilesystemStore.insert_metadataforgets to derivelocation_hashwhen the DTO field isNone; INSERT fails the NOT-NULL DB constraint at runtime. - Mitigation: The impl task's AC includes a "derive-on-None" unit test. The DB-side NOT-NULL is the safety net — fail-fast, not silent. Documented in the contract bump.
Risk 8: Future cross-workspace renegotiation of the namespace UUID
- Risk: A future cycle wants to rotate
TILE_NAMESPACE_UUID; doing so without coordination invalidates every existingtile_uuidandlocation_hash. - Mitigation: Constraint forbids regeneration in this task; future change requires an ADR + coordinated cross-workspace release. The migration runner logs the namespace value used at apply time.
Runtime Completeness
- Named capability: Postgres 16 deterministic-identity columns + per-flight natural-key UNIQUE + LRU/upload bookkeeping + sector-boundary geometry + per-classification freshness rules — all additive on the AZ-263 baseline (description.md / data_model.md / AC-NEW-3 / AC-NEW-6 / RESTRICT-SAT-2 / 2026-05-12 leftover § 4–5).
- Production code that must exist: real Alembic migration
0002_c6_tile_identity_and_lru.py, realapply_migrationsrunner, real schema-fixture diff test, realpsycopg_poolconnection helper, real_uuid_namespacemodule withTILE_NAMESPACE_UUIDconstant andderive_tile_id/derive_location_hashhelpers, real DTO extension in_types.py, real contract bump intile_metadata_store.md. - Allowed external stubs: tests use
testcontainers-managed Postgres 16 instances (already in the project's test infra per AZ-263); production wiring uses the operator's deployed Postgres. - Unacceptable substitutes: SQLite "for testing only" — production and test environments MUST both be Postgres 16; raw SQL DDL applied without Alembic (would defeat the version-tracking the runner depends on); a
tile_quality_metadatavalidation at the DB layer (would lock the schema to the JSONB shape — the application-side validation is the single source of truth); a non-deterministictile_uuidstrategy (would defeat the cross-workspace coordination the namespace pin establishes); any operation that renames, retypes, or drops an AZ-263 column / table / index / CHECK (forbidden percoderule.mdcanddata_model.md§ 6.2 / § 6.3); a parallel Alembic env atsrc/.../c6_tile_cache/_alembic/(forbidden — the project uses one alembic env atdb/migrations/per AZ-263 +alembic.ini).
Contract
This task does NOT produce a new contract file — it implements the tile_metadata_store.md contract's persistence surface and bumps its version v1.1.0 → v1.2.0 with one non-breaking minor addition (TileMetadata.location_hash: UUID | None = None).
The schema-fixture file tests/fixtures/c6_postgres_schema_v2.sql is the diff target referenced in tile_metadata_store.md § Test Cases (schema-shape-fixture-diff) — but the contract document of record stays the Protocol contract.