[AZ-503] [AZ-504] Cycle 5 Steps 11-15 sync

Wrap up cycle 5 verification + documentation:
- Steps 10/11 wrap-up reports (implementation_completeness +
  implementation_report) for the AZ-503-foundation + AZ-504 batch.
- Step 12 test-spec sync: AZ-503-foundation/AZ-504 ACs appended;
  AZ-505 deferred ACs recorded.
- Step 13 update-docs: architecture, data-model, glossary, module-
  layout, uav-tile-upload contract (v1.1.0), DataAccess + Services
  + Tests module docs synced; new common_uuidv5.md module doc.
- Step 14 security audit: PASS_WITH_WARNINGS; 0 new Critical/High;
  2 new Low informational (F1 flightId provenance, F2 pgcrypto
  deploy gap).
- Step 15 performance test: PASS_WITH_INFRA_WARNINGS; PT-08
  passed twice (AZ-504 fix verified); PT-01/02 failed due to
  recurring local Docker/colima DNS cold-start (not an app
  regression). Cycle-3 perf-harness leftover stays OPEN with
  replay #5 documented.
- Autodev state moved to Step 16 (Deploy).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-12 18:01:27 +03:00
parent c646aa93e2
commit 61612044fb
27 changed files with 1075 additions and 50 deletions
+11 -9
View File
@@ -20,7 +20,8 @@ The three Layer-3 service components are compile-time siblings: each only refere
**Architectural principles** (inferred):
- Single-instance deployment, no horizontal scaling requirements (`inferred-from: Channel-based queue, no distributed state`)
- Append-by-source tile storage — multiple producers (Google Maps, UAV upload, future SatAR, …) can each persist a row per `(latitude, longitude, tile_zoom, tile_size_meters)` cell. Reads return the most-recent row across sources, ordered by `captured_at DESC` with deterministic `(updated_at DESC, id DESC)` tie-breaks. The single-row-per-cell-per-source invariant is enforced by the 5-column unique index `idx_tiles_unique_location_source` introduced in migration 013 (AZ-484). The `tiles.version` column is vestigial since AZ-357 dropped year-based cache invalidation in favour of cell-level overwrite. (`inferred-from: tiles table + AZ-484/AZ-357 migrations + tile-storage contract v1.0.0`)
- Append-by-source-and-flight tile storage (AZ-503; refines AZ-484) — multiple producers (Google Maps, UAV upload from N flights, future SatAR, …) can each persist a row per `(tile_zoom, tile_x, tile_y, tile_size_meters)` cell. Reads return the most-recent row across sources, ordered by `captured_at DESC` with deterministic `(updated_at DESC, id DESC)` tie-breaks. The single-row-per-cell-per-source-per-flight invariant is enforced by `idx_tiles_unique_identity` (6-column integer-only, `COALESCE(flight_id, '00000000-...'::uuid)`) introduced in migration 014; this supersedes the AZ-484 float-based `idx_tiles_unique_location_source`. Identity is deterministic across re-ingests: `tiles.id = Uuidv5(TileNamespace, "{z}/{x}/{y}/{source}/{flight_id or zero-uuid}")` and `tiles.location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}")`. The `tiles.version` column remains vestigial since AZ-357 dropped year-based cache invalidation in favour of cell-level overwrite. (`inferred-from: tiles table + AZ-484/AZ-357/AZ-503 migrations + tile-storage contract v1.0.0`)
- Cross-repo deterministic tile identity (AZ-503) — the `TileNamespace` UUID `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c` and the canonical name format are shared with the sibling workspace `gps-denied-onboard` (`components/c6_tile_cache/_uuid.py:TILE_NAMESPACE`). Both sides MUST produce byte-identical UUIDv5 output so an onboard-cached tile and a server-cached tile for the same `(z, x, y, source, flight_id)` are recognized as the same artifact without a round-trip. Changing the namespace constant on either side is a coordinated cross-repo break. (`inferred-from: Uuidv5.cs, AZ-503 task spec § Constraints`)
- Fire-and-forget async processing with status polling (`inferred-from: queue + background service + status endpoint`)
- JWT-validated callers only — every HTTP endpoint requires a valid HS256-signed Bearer token, validated locally against a shared `JWT_SECRET` per the suite-level auth contract (`suite/_docs/10_auth.md`). Issuer/audience are intentionally not validated yet; signature + lifetime + ≥32-byte key are. Per-endpoint permission claims (e.g. `permissions: ["GPS"]` on the UAV upload) layer on top of this baseline.
@@ -34,11 +35,11 @@ The three Layer-3 service components are compile-time siblings: each only refere
**Planned features** (confirmed by user, currently stubs):
- MGRS endpoint — tile access via Military Grid Reference System coordinates
**Multi-source tile producers** (live as of AZ-488):
- *Google Maps* — `TileService.DownloadAndStoreTilesAsync` / `DownloadAndStoreSingleTileAsync` stamp `source='google_maps'` on every persisted row; tile JPEGs live under `{StorageConfig.TilesDirectory}/{zoom}/...` per the legacy grandfathered layout.
- *UAV* — `POST /api/satellite/upload` (AZ-488) accepts a multipart batch of UAV-captured tiles, runs each item through a 5-rule quality gate (`UavTileQualityGate`), and persists accepted items via `ITileRepository.InsertAsync` with `source='uav'`. UAV JPEGs live under `{StorageConfig.TilesDirectory}/uav/{zoom}/{x}/{y}.jpg`. Requires the `GPS` permission claim on top of the JWT baseline.
**Multi-source tile producers** (live as of AZ-488; per-flight evidence isolation as of AZ-503):
- *Google Maps* — `TileService.DownloadAndStoreTilesAsync` / `DownloadAndStoreSingleTileAsync` stamp `source='google_maps'`, `flight_id=NULL`, and a deterministic UUIDv5 `id` on every persisted row; tile JPEGs live under `{StorageConfig.TilesDirectory}/{zoom}/...` per the legacy grandfathered layout. `content_sha256` is computed from the on-disk JPEG body.
- *UAV* — `POST /api/satellite/upload` (AZ-488; per-flight key extended by AZ-503) accepts a multipart batch of UAV-captured tiles, runs each item through a 5-rule quality gate (`UavTileQualityGate`), and persists accepted items via `ITileRepository.InsertAsync` with `source='uav'`, `flight_id = metadata.flightId` (or NULL for anonymous uploads), and a deterministic UUIDv5 `id`. UAV JPEGs live under `{StorageConfig.TilesDirectory}/uav/{flight_id or 'none'}/{zoom}/{x}/{y}.jpg`, so `rm -rf ./tiles/uav/{flight_id}/` removes one flight's evidence without touching other flights at overlapping cells. Requires the `GPS` permission claim on top of the JWT baseline.
The N-source storage contract is authoritative in `_docs/02_document/contracts/data-access/tile-storage.md` (v1.0.0). The UAV upload contract is authoritative in `_docs/02_document/contracts/api/uav-tile-upload.md` (v1.0.0). Anything that reads or writes `tiles` MUST follow those contracts rather than re-deriving the rules from prose here.
The N-source storage contract is authoritative in `_docs/02_document/contracts/data-access/tile-storage.md` (v1.0.0; v2.0.0 bump tracking the AZ-503 identity columns is deferred to AZ-505 when the new identity surface freezes for external consumers). The UAV upload contract is authoritative in `_docs/02_document/contracts/api/uav-tile-upload.md` (v1.0.0; AZ-503 added an optional `flightId` field to per-item metadata — backward-compatible). Anything that reads or writes `tiles` MUST follow those contracts rather than re-deriving the rules from prose here.
**Drift signals**:
- `geofence_polygons` mentioned in AGENTS.md as a routes table column but does not exist in schema or entity — documentation drift
@@ -183,13 +184,14 @@ The N-source storage contract is authoritative in `_docs/02_document/contracts/d
**Context**: Tiles are immutable JPEG images that need fast random access.
**Decision**: Store tiles as files in a directory hierarchy with metadata in PostgreSQL. The layout is per-source so the bytes for `google_maps` and `uav` writes for the same cell remain individually addressable on disk:
**Decision**: Store tiles as files in a directory hierarchy with metadata in PostgreSQL. The layout is per-source (and per-flight for UAV since AZ-503) so the bytes for distinct producers / flights at the same cell remain individually addressable on disk:
- Google Maps (legacy, grandfathered): `{StorageConfig.TilesDirectory}/{zoom}/{x_bucket}/{y_bucket}/tile_{zoom}_{x}_{y}_{timestamp}.jpg`
- UAV (AZ-488): `{StorageConfig.TilesDirectory}/uav/{zoom}/{x}/{y}.jpg`
- UAV anonymous (AZ-488 baseline): `{StorageConfig.TilesDirectory}/uav/none/{zoom}/{x}/{y}.jpg`
- UAV per-flight (AZ-503): `{StorageConfig.TilesDirectory}/uav/{flight_id}/{zoom}/{x}/{y}.jpg`
The authoritative source marker is the `tiles.source` column; the per-source on-disk path matters only for write isolation between producers.
The authoritative source/flight markers are the `tiles.source` and `tiles.flight_id` columns; the per-source / per-flight on-disk path matters only for write isolation and bulk-delete granularity.
**Consequences**: Fast reads, easy backup/migration, both producers can run without colliding on bytes, but requires shared filesystem for multi-instance (which is not currently needed). No migration of pre-AZ-488 Google Maps files is shipped — the legacy layout stays intact.
**Consequences**: Fast reads, easy backup/migration, producers can run without colliding on bytes, and per-flight `rm -rf` becomes safe. Requires shared filesystem for multi-instance (not currently needed). No migration of pre-AZ-488 Google Maps files is shipped — the legacy layout stays intact. Pre-AZ-503 UAV files written by the AZ-488 baseline at `./tiles/uav/{z}/{x}/{y}.jpg` (no flight segment) are not relocated by the migration; the post-AZ-503 code writes anonymous uploads to `./tiles/uav/none/{z}/{x}/{y}.jpg` and the original AZ-488-era files stay where they were. This is acceptable because AZ-488 only landed in cycle 2 and the volume of pre-AZ-503 UAV bytes is small (no production UAV upload traffic yet).
### ADR-005: Background Hosted Services for Processing
@@ -59,7 +59,7 @@
|-------|-----------|----------|--------------|
| GetByTileCoordinatesAsync (tile lookup) | Very High | Yes | `(tile_zoom, tile_x, tile_y)` |
| GetTilesByRegionAsync (spatial) | High | Yes | `(latitude, longitude, tile_zoom)` |
| InsertAsync (tile per-source upsert) | High | Yes | Composite unique on `(lat, lon, tile_zoom, tile_size_meters, source)` (AZ-484: `idx_tiles_unique_location_source`) |
| InsertAsync (tile per-source upsert) | High | Yes | Composite unique on `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid))` (AZ-503: `idx_tiles_unique_identity`; supersedes the AZ-484 float-based `idx_tiles_unique_location_source`) |
| GetByStatusAsync (region polling) | Medium | No | `(status)` |
| GetRoutesWithPendingMapsAsync | Low | No | `(request_maps, maps_ready)` |
@@ -89,9 +89,10 @@
- Repository interfaces are defined in this project (not in Common), creating a dependency from Services to DataAccess
- Column mapping uses SQL aliases (`tile_zoom as TileZoom`) rather than Dapper attribute mapping
- TileRepository.InsertAsync uses a per-source UPSERT pattern (AZ-484); two producers (e.g., `google_maps` + `uav`) coexist as separate rows for the same cell, while same-source re-inserts overwrite and refresh `captured_at`
- TileRepository.InsertAsync uses an integer-only, flight-aware UPSERT pattern (AZ-503; supersedes the AZ-484 5-column float-based UPSERT). Same-source same-flight re-inserts overwrite and refresh `captured_at`/`location_hash`/`content_sha256`; different sources or different flights at the same cell coexist as separate rows. `id` is intentionally NOT overwritten on conflict so it stays deterministic per AZ-503 AC-2.
- `TileEntity.Source` is stored as a plain `string` (not the `TileSource` enum) due to Dapper issue #259 — see `_docs/LESSONS.md` L-001. Conversion happens via `SatelliteProvider.Common.Enums.TileSourceConverter`
- The frozen v1.0.0 `tile-storage` contract (`_docs/02_document/contracts/data-access/`) is the authoritative spec for all `tiles` invariants enforced here
- AZ-503 deterministic identity: `id` is `Uuidv5(TileNamespace, "{z}/{x}/{y}/{source}/{flight_id or zero-uuid}")` and `location_hash` is `Uuidv5(TileNamespace, "{z}/{x}/{y}")`. The cross-repo `TileNamespace` constant lives in `SatelliteProvider.Common.Utils.Uuidv5` and MUST match `gps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE`.
- The frozen v1.0.0 `tile-storage` contract (`_docs/02_document/contracts/data-access/`) is the AZ-484-era spec for read-side selection invariants; the AZ-503 write-side schema change is documented inline in `dataaccess_models.md` and `dataaccess_tile_repository.md`. A v2.0.0 contract bump is deferred to AZ-505 (when the `POST /api/satellite/tiles/inventory` endpoint freezes the new identity surface for external consumers).
- No soft-delete; `DeleteAsync` is a hard delete
## 8. Dependency Graph
@@ -2,10 +2,11 @@
**Component**: WebApi (`SatelliteProvider.Api`) producing rows via TileDownloader (`SatelliteProvider.Services.TileDownloader`)
**Producer task**: AZ-488 — `_docs/02_tasks/done/AZ-488_uav_tile_upload.md`
**Extended by**: AZ-503 — `_docs/02_tasks/done/AZ-503_tile_identity_uuidv5_bulk_list.md` (added optional `flightId` per-item field; per-flight on-disk path; deterministic `tileId`)
**Consumer tasks**: `gps-denied-onboard`, mission planner UI, any future UAV-equipped client
**Version**: 1.0.0
**Version**: 1.1.0
**Status**: frozen
**Last Updated**: 2026-05-11
**Last Updated**: 2026-05-12
## Purpose
@@ -39,6 +40,7 @@ Multipart form fields (case-sensitive part names):
| `tileZoom` | integer | yes | Slippy Map zoom level | Must satisfy the same zoom-level policy as the existing tile pipeline (see `MapConfig.AllowedZoomLevels`) |
| `tileSizeMeters` | number | yes | Tile size in meters at the captured latitude | Producer-supplied |
| `capturedAt` | string (ISO-8601 UTC) | yes | Moment of UAV image capture | Must satisfy the captured-at rule (see Quality Gate, Rule 4) |
| `flightId` | string (UUID) | no | AZ-503: optional flight identifier. When present, two flights uploading the same cell coexist as separate rows; absent uploads share a single anonymous row per cell. Omitting the field is fully backward-compatible with v1.0.0 clients. | RFC 4122 UUID. Backward-compatible default: `null` |
Field names are camelCase. Property-name matching is case-insensitive on read.
@@ -119,16 +121,21 @@ The 5-rule per-item quality gate never produces a 400; per-item failures always
## Persistence semantics
- Accepted items are persisted via `ITileRepository.InsertAsync` (the per-source UPSERT path established in AZ-484). The `tiles` row carries `source='uav'` and `captured_at` from the request.
- Accepted items are persisted via `ITileRepository.InsertAsync` (the integer-only flight-aware UPSERT path established in AZ-503; supersedes the AZ-484 5-column float-based UPSERT). The `tiles` row carries `source='uav'`, `captured_at` from the request, `flight_id` from the optional `metadata.flightId`, and a deterministic `id = Uuidv5(TileNamespace, "{z}/{x}/{y}/uav/{flight_id or zero-uuid}")`.
- A UAV upload for a cell that already has a `google_maps` row **coexists** with that row (per `tile-storage.md` Inv-3). The most-recent row across sources wins on read.
- A second UAV upload for the same cell UPSERTs the existing `uav` row, updating `file_path`, `captured_at`, `updated_at` and overwriting the JPEG bytes on disk.
- Two UAV uploads with **different** `flightId` for the same cell coexist as separate rows (one row per flight, sharing `location_hash`). Two UAV uploads with the **same** `flightId` for the same cell UPSERT the existing row, updating `file_path, captured_at, location_hash, content_sha256, updated_at` and overwriting the JPEG bytes on disk. `id` is intentionally NOT regenerated on conflict — re-uploading identical bytes returns the same `tileId` (AZ-503 AC-2).
- `content_sha256` is computed from the JPEG body on every persisted row.
## File-path layout
- UAV files: `{StorageConfig.TilesDirectory}/uav/{tile_zoom}/{tile_x}/{tile_y}.jpg`
- Google Maps files: unchanged from the pre-AZ-488 layout (grandfathered, no migration ships with this contract)
- UAV files (AZ-503): `{StorageConfig.TilesDirectory}/uav/{flightId or 'none'}/{tile_zoom}/{tile_x}/{tile_y}.jpg`
- Anonymous uploads (`flightId` absent or null) use the literal `none` segment.
- Per-flight uploads use the full `flightId` UUID as a directory name, so `rm -rf ./tiles/uav/{flightId}/` cleanly removes one flight's evidence without touching other flights or sources.
- Google Maps files: unchanged from the pre-AZ-488 layout (grandfathered, no migration ships with this contract).
`tile_x` and `tile_y` are derived server-side from `(latitude, longitude, tile_zoom)` via `GeoUtils.WorldToTilePos`; the client cannot influence the on-disk path beyond providing valid coordinates.
`tile_x` and `tile_y` are derived server-side from `(latitude, longitude, tile_zoom)` via `GeoUtils.WorldToTilePos`; the client cannot influence the on-disk path beyond providing valid coordinates and an optional `flightId`.
Pre-AZ-503 UAV files written at `./tiles/uav/{z}/{x}/{y}.jpg` (no flight segment) are not relocated. Post-AZ-503 anonymous uploads write to `./tiles/uav/none/{z}/{x}/{y}.jpg`. This split is acceptable because AZ-488 only shipped in cycle 2 and the pre-AZ-503 UAV file count is small.
## Concurrency
@@ -177,3 +184,4 @@ Each version bump requires updating the Change Log and notifying every consumer
| Version | Date | Change | Author |
|---------|------|--------|--------|
| 1.0.0 | 2026-05-11 | Initial contract — batch UAV upload endpoint, 5-rule quality gate, per-source UPSERT, closed reject-reason enum, GPS-permission requirement. Produced by AZ-488. | autodev (cycle 2 step 10) |
| 1.1.0 | 2026-05-12 | Minor bump for AZ-503: added optional `flightId` per-item metadata field (backward-compatible default `null`); `tileId` in the response is now a deterministic UUIDv5 derived from `(z, x, y, source, flightId)` instead of a random Guid; on-disk path adds a `{flightId or 'none'}` segment for per-flight evidence isolation. No reject-reason changes, no envelope changes, no permission changes. v1.0.0 clients omitting `flightId` keep working unchanged. | autodev (cycle 5 step 13) |
+17 -7
View File
@@ -5,7 +5,7 @@
```mermaid
erDiagram
TILES {
uuid id PK
uuid id PK "AZ-503: deterministic UUIDv5"
int tile_zoom
float latitude
float longitude
@@ -16,6 +16,10 @@ erDiagram
int version
varchar source
timestamp captured_at
uuid flight_id "AZ-503 nullable"
uuid location_hash "AZ-503 NOT NULL"
bytea content_sha256 "AZ-503 nullable (app-NOT-NULL for new)"
uuid legacy_id "AZ-503 nullable, pre-migration id"
varchar file_path
int tile_x
int tile_y
@@ -91,7 +95,7 @@ Stores metadata for downloaded satellite imagery tiles. Each tile is a single im
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| id | UUID | PK | Unique tile identifier |
| id | UUID | PK | AZ-503: deterministic UUIDv5 of `{tile_zoom}/{tile_x}/{tile_y}/{source}/{flight_id or zero-uuid}` under `Uuidv5.TileNamespace`. Stable across re-ingests; preserved on UPSERT conflict (AC-2 idempotence). Pre-AZ-503 rows have their original random Guid; migration 014 also copies that value into `legacy_id` for one-cycle forensics. |
| tile_zoom | INT | NOT NULL | Google Maps zoom level (1-20) |
| latitude | DOUBLE PRECISION | NOT NULL | Center latitude |
| longitude | DOUBLE PRECISION | NOT NULL | Center longitude |
@@ -102,20 +106,25 @@ Stores metadata for downloaded satellite imagery tiles. Each tile is a single im
| version | INT | NOT NULL, DEFAULT 2025 | Year-based versioning for cache invalidation. Vestigial post-AZ-484 — removed from the unique key by migration 012 (preparation for AZ-484); column retained nullable for backward compatibility |
| source | VARCHAR(32) | NOT NULL, DEFAULT 'google_maps' | AZ-484: producer of the imagery (`'google_maps'`, `'uav'`). Closed value set — see `tile-storage` v1.0.0 contract Inv-5 and `Common.Enums.TileSourceConverter`. Backfilled to `'google_maps'` for all pre-AZ-484 rows by migration 013 |
| captured_at | TIMESTAMP | NOT NULL | AZ-484: imagery acquisition timestamp (UTC). Drives most-recent-across-sources selection. Backfilled to `created_at` for pre-AZ-484 rows by migration 013 |
| file_path | VARCHAR(500) | NOT NULL | Relative path to stored image. **AZ-488 per-source layout**: `source='google_maps'` rows keep the legacy bucketed/timestamped path emitted by `StorageConfig.GetTileFilePath` (`{TilesDirectory}/{zoom}/{x_bucket}/{y_bucket}/tile_{zoom}_{x}_{y}_{ts}.jpg`). `source='uav'` rows live under `{TilesDirectory}/uav/{zoom}/{x}/{y}.jpg` — see `_docs/02_document/contracts/api/uav-tile-upload.md` v1.0.0. The authoritative source marker is the `source` column; the per-source path is implementation detail that keeps both producers' bytes individually addressable. |
| file_path | VARCHAR(500) | NOT NULL | Relative path to stored image. **AZ-503 per-flight UAV layout** (supersedes AZ-488): `source='google_maps'` rows keep the legacy bucketed/timestamped path emitted by `StorageConfig.GetTileFilePath` (`{TilesDirectory}/{zoom}/{x_bucket}/{y_bucket}/tile_{zoom}_{x}_{y}_{ts}.jpg`). `source='uav'` rows live under `{TilesDirectory}/uav/{flight_id or 'none'}/{zoom}/{x}/{y}.jpg` — so `rm -rf ./tiles/uav/{flight_id}/` cleanly removes one flight's evidence without disturbing other flights at overlapping cells. The authoritative source marker is the `source` column; the per-source / per-flight path is implementation detail that keeps both producers' bytes individually addressable. |
| tile_x | INT | NOT NULL | Tile X coordinate (Slippy Map) |
| tile_y | INT | NOT NULL | Tile Y coordinate (Slippy Map) |
| flight_id | UUID | NULL | AZ-503: optional flight identifier. `NULL` for Google Maps tiles and anonymous UAV uploads; populated from `UavTileMetadata.FlightId` when present. Part of the UPSERT conflict key via `COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)`, so two flights uploading the same `(z, x, y)` cell produce two separate rows. |
| location_hash | UUID | NOT NULL | AZ-503: deterministic UUIDv5 of `{tile_zoom}/{tile_x}/{tile_y}` under `Uuidv5.TileNamespace`. Identical across flights and sources for the same cell. Backfilled in migration 014 via a `pg_temp.uuidv5` PL/pgSQL function. Reserved for the AZ-505 Leaflet covering index (`POST /tiles/inventory`). |
| content_sha256 | BYTEA | NULL | AZ-503: SHA-256 digest of the JPEG body. Application-layer NOT NULL for new writes (enforced in `TileService.BuildTileEntity` + `UavTileUploadHandler.PersistAsync`); DB column is NULLABLE because legacy pre-migration rows cannot be backfilled reliably from disk. See `batch_02_cycle5_report.md` "Low maintainability finding" for the rationale. |
| legacy_id | UUID | NULL | AZ-503: pre-migration `id` value, copied by migration 014 for one-cycle forensics. To be dropped in a future migration once the cross-repo cutover settles. |
| created_at | TIMESTAMP | NOT NULL, DEFAULT NOW | |
| updated_at | TIMESTAMP | NOT NULL, DEFAULT NOW | |
**Indexes** (post-AZ-484):
- `idx_tiles_unique_location_source` UNIQUE (latitude, longitude, tile_zoom, tile_size_meters, source) — created by migration 013; replaces the pre-AZ-484 4-col `idx_tiles_unique_location` (which itself superseded the legacy 5-col `(…, version)` index dropped by migration 012)
**Indexes** (post-AZ-503):
- `idx_tiles_unique_identity` UNIQUE (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)) — created by migration 014; replaces the AZ-484 `idx_tiles_unique_location_source` (5-col float-based). Integer-only conflict columns eliminate float-rounding collisions; the `COALESCE` lets per-flight rows coexist while keeping single-row semantics for anonymous and `google_maps` rows.
- `idx_tiles_location_hash` (location_hash) — created by migration 014; non-unique. Reserved for the AZ-505 Leaflet covering index when `POST /tiles/inventory` lands.
- `idx_tiles_coordinates` (tile_zoom, tile_x, tile_y, version)
- `idx_tiles_zoom` (tile_zoom)
**Selection rule**: `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` return the most-recent row across sources for any `(latitude, longitude, tile_zoom, tile_size_meters)` cell. Tie-break: `captured_at DESC, updated_at DESC, id DESC`. Region read uses `DISTINCT ON` to enforce one-row-per-cell at the SQL layer.
**Selection rule** (unchanged from AZ-484): `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` return the most-recent row across sources for any `(latitude, longitude, tile_zoom, tile_size_meters)` cell. Tie-break: `captured_at DESC, updated_at DESC, id DESC`. Region read uses `DISTINCT ON` to enforce one-row-per-cell at the SQL layer.
**UPSERT contract**: `INSERT … ON CONFLICT (latitude, longitude, tile_zoom, tile_size_meters, source) DO UPDATE` — same-source re-insert refreshes `file_path, tile_x, tile_y, captured_at, updated_at`. Two producers for the same cell coexist as separate rows.
**UPSERT contract** (AZ-503): `INSERT … ON CONFLICT (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-...'::uuid)) DO UPDATE` refreshes `file_path, latitude, longitude, captured_at, location_hash, content_sha256, updated_at`. `id` is intentionally NOT overwritten on conflict, preserving AC-2 idempotence (same inputs ⇒ same id). Two sources (or two flights of the same source) at the same cell coexist as separate rows.
### regions
@@ -225,3 +234,4 @@ Junction table linking routes to their generated region requests, with geofence
| 011 | AddTileCoordinates | Slippy map X/Y + rename zoom_level → tile_zoom |
| 012 | DropTileVersionConstraint | Drops legacy 5-col `(…, version)` unique index; replaces with 4-col `idx_tiles_unique_location` (preparation for AZ-484) |
| 013 | AddTileSourceAndCapturedAt | AZ-484: adds `source` (default `'google_maps'`) + `captured_at` columns; backfills both for pre-existing rows; replaces 4-col unique with 5-col `idx_tiles_unique_location_source`. Transactional; idempotent against partial replays |
| 014 | AddTileIdentityColumns | AZ-503: adds `flight_id` (NULL), `location_hash` (NOT NULL after backfill), `content_sha256` (NULL), `legacy_id` (NULL); backfills `location_hash` via `pg_temp.uuidv5(TILE_NAMESPACE, "{tile_zoom}/{tile_x}/{tile_y}")` and copies `id → legacy_id` for every pre-existing row; drops `idx_tiles_unique_location_source` (AZ-484) and creates `idx_tiles_unique_identity` (integer + flight-aware) + `idx_tiles_location_hash`. Enables `pgcrypto` for the in-migration SHA-1 digest. Transactional; safe to replay (column adds are `IF NOT EXISTS`-equivalent, backfill is idempotent on `location_hash` because UUIDv5 is deterministic) |
+6
View File
@@ -23,6 +23,10 @@
| CAPTURED_AT_FUTURE | UAV reject reason — `capturedAt` is more than `CapturedAtFutureSkewSeconds` ahead of the server clock. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.0.0) |
| CAPTURED_AT_TOO_OLD | UAV reject reason — `capturedAt` is older than `UavQualityConfig.MaxAgeDays`. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.0.0) |
| IMAGE_TOO_UNIFORM | UAV reject reason — pixel-luminance variance on the downsampled image is below `MinLuminanceVariance`. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.0.0) |
| Flight ID | AZ-503 optional `Guid` identifier for a single UAV flight, sent as `metadata.flightId` per item on `POST /api/satellite/upload`. Two flights uploading the same `(z, x, y)` cell coexist as two `tiles` rows that share a single `location_hash` but have distinct `tiles.id` values and distinct on-disk file paths (`./tiles/uav/{flight_id}/{z}/{x}/{y}.jpg`). Anonymous uploads (no `flightId`) collapse to a single row per cell at the literal path `./tiles/uav/none/{z}/{x}/{y}.jpg`. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.1.0) |
| Tile Namespace | The constant UUID `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c` used by `Uuidv5.Create` to seed every tile-identity computation in this service. Pinned cross-repo with `gps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE` so both sides compute byte-identical UUIDv5 outputs for the same canonical name. Changing the constant on either side is a coordinated cross-repo break. | `SatelliteProvider.Common.Utils.Uuidv5.TileNamespace`, AZ-503 |
| Location Hash | Deterministic UUIDv5 of `"{tile_zoom}/{tile_x}/{tile_y}"` under `Tile Namespace`. Identical across flights and sources for the same cell; stored in `tiles.location_hash` (NOT NULL). Reserved for the AZ-505 Leaflet covering index (`POST /api/satellite/tiles/inventory`). | _docs/02_document/data_model.md, AZ-503 |
| Content SHA-256 | SHA-256 digest of the JPEG body, stored in `tiles.content_sha256` (`bytea`, NULLABLE at the DB layer; application code enforces NOT NULL for new writes). Used to detect byte-identical re-uploads. Legacy pre-AZ-503 rows are NULL because file paths are volatile and a reliable on-disk backfill was not possible. | _docs/02_document/data_model.md, AZ-503 |
| Nadir Camera | Downward-facing camera on a UAV capturing ground imagery during flight | user clarification |
| GPS-Denied Service | The consuming system: a UAV navigation service operating without GPS, using satellite/UAV imagery for positioning | user clarification |
| Slippy Map Coordinates | Tile X/Y indices in the Web Mercator projection grid (standard for web map tile servers) | data_model.md |
@@ -37,6 +41,8 @@
| ISatelliteDownloader | Interface abstracting satellite imagery providers; first implementation: Google Maps (GoogleMapsDownloaderV2) | modules/common_interfaces.md |
| DbUp | .NET library for forward-only SQL schema migrations via numbered embedded scripts | modules/dataaccess_database_migrator.md |
| Tile Deduplication | Mechanism using DB unique index + ConcurrentDictionary to prevent re-downloading identical tiles | modules/services_google_maps_downloader.md |
| UUIDv5 | RFC 9562 §5.5 deterministic UUID derived from a namespace UUID + a UTF-8 name via SHA-1. AZ-503 uses it to produce stable, cross-repo `tiles.id` and `tiles.location_hash` values without coordinating an id allocator between the satellite-provider and `gps-denied-onboard` workspaces. | modules/common_uuidv5.md, AZ-503 |
| Legacy ID | Pre-AZ-503 random `tiles.id` value, copied into the `legacy_id` column by migration 014 for one-cycle forensics. To be dropped in a future cycle once the cross-repo cutover settles. | _docs/02_document/data_model.md, AZ-503 |
## Abbreviations
+4 -3
View File
@@ -5,7 +5,7 @@
**Language**: csharp
**Layout Convention**: custom (per-component .csproj per logical component)
**Root**: ./
**Last Updated**: 2026-05-11 (cycle 2 — AZ-487 JWT validation baseline + AZ-488 UAV tile upload added; supersedes prior post-AZ-350 update)
**Last Updated**: 2026-05-12 (cycle 5 — AZ-503 tile-identity foundation added: `SatelliteProvider.Common/Utils/Uuidv5.cs`, migration `014_AddTileIdentityColumns.sql`, 4 new `TileEntity` columns, integer-only flight-aware UPSERT, IntegrationTests → Common ProjectReference)
## Layout Rules
@@ -45,6 +45,7 @@ The cycle-1 (AZ-487) and cycle-2 (AZ-488) code reviews each surfaced an F1 (Low
- `SatelliteProvider.Common/Exceptions/RateLimitException.cs`
- `SatelliteProvider.Common/Interfaces/*.cs` (all service interfaces)
- `SatelliteProvider.Common/Utils/GeoUtils.cs`
- `SatelliteProvider.Common/Utils/Uuidv5.cs` (added by AZ-503; deterministic UUIDv5 generator + cross-repo `TileNamespace` constant pinned to `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c`)
- **Internal**: (none — all types are public, shared across components)
- **Owns**: `SatelliteProvider.Common/**`
- **Imports from**: (none)
@@ -154,8 +155,8 @@ The cycle-1 (AZ-487) and cycle-2 (AZ-488) code reviews each surfaced an F1 (Low
### Common/Utils
- **Directory**: `SatelliteProvider.Common/Utils/`
- **Purpose**: Stateless geospatial utility functions (coordinate math, distance, bearing)
- **Consumed by**: TileDownloader, RegionProcessing, RouteManagement
- **Purpose**: Stateless utility functions — geospatial (`GeoUtils`: coordinate math, distance, bearing) and identity (`Uuidv5`: deterministic UUIDv5 generator + cross-repo `TileNamespace` constant, added by AZ-503).
- **Consumed by**: TileDownloader, RegionProcessing, RouteManagement, IntegrationTests (AZ-503 added a `ProjectReference` from `SatelliteProvider.IntegrationTests` to `SatelliteProvider.Common` so test seeders can call `Uuidv5.Create` directly instead of duplicating the algorithm)
### Common/Enums
+2 -1
View File
@@ -72,12 +72,13 @@ Axis-aligned bounding box defined by NW and SE corners.
Container for multiple geofence polygons.
- `Polygons` (List\<GeofencePolygon\>)
### UavTileMetadata (added AZ-488)
### UavTileMetadata (added AZ-488, extended AZ-503)
Per-tile metadata payload inside a UAV batch upload (`POST /api/satellite/upload`). Indexed-correlated with the multipart `IFormFileCollection`.
- `Latitude`, `Longitude` (double)
- `TileZoom` (int)
- `TileSizeMeters` (double)
- `CapturedAt` (DateTime, UTC; subject to AZ-488 Rule 4 future-skew / age checks)
- `FlightId` (Guid?, JSON: `"flightId"`) — AZ-503 optional flight identifier. When set, the per-item `tiles.id` becomes `Uuidv5(TileNamespace, "{z}/{x}/{y}/uav/{flightId}")`, the on-disk path is `./tiles/uav/{flightId}/{z}/{x}/{y}.jpg`, and the UPSERT conflict key separates this row from rows belonging to other flights at the same cell. When `null`, the per-item id uses the zero-UUID `00000000-0000-0000-0000-000000000000` placeholder and the on-disk path uses the literal `none` segment (`./tiles/uav/none/{z}/{x}/{y}.jpg`). The placeholder UUID is purely a key-space marker — it never lands in the `flight_id` column (which stays `NULL`); the UPSERT uses `COALESCE(flight_id, '00000000-...')` for the conflict check.
### UavTileBatchMetadataPayload (added AZ-488)
JSON envelope deserialized from the `metadata` form field of a UAV batch upload.
@@ -0,0 +1,76 @@
# Module: Common/Utils/Uuidv5
## Purpose
Deterministic UUIDv5 generator (RFC 9562 §5.5, SHA-1 namespace+name hashing) for tile identity. Pure C# implementation, ≤80 LoC, no third-party dependency. Owns the cross-repo `TileNamespace` constant that pins UUIDv5 outputs to be byte-identical between this workspace (C#) and the sibling `gps-denied-onboard` workspace (Python `uuid.uuid5`).
**csproj**: `SatelliteProvider.Common/Utils/Uuidv5.cs`
**Introduced**: AZ-503 (Cycle 5)
## Public Interface
All members are static on `Uuidv5`:
- `TileNamespace` (Guid, public const) — `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c`. The shared namespace UUID used for every tile identity computation in this service and its onboard counterpart. **MUST NOT be changed** without coordinating a migration with `gps-denied-onboard/components/c6_tile_cache/_uuid.py`.
- `Create(Guid namespaceId, string name) → Guid` — produces a deterministic UUIDv5 by hashing `namespaceId.ToByteArrayBigEndian() || Encoding.UTF8.GetBytes(name)` with SHA-1, then assembling the 16 bytes per RFC 9562:
- bytes 03 are read as a big-endian uint32 (`time_low`)
- bytes 45 are read as a big-endian uint16 (`time_mid`)
- bytes 67 have their top 4 bits set to `0101` (version 5)
- byte 8 has its top 2 bits set to `10` (variant RFC 4122 / 9562)
- bytes 815 form the variant + clock_seq + node fields
- `Create(Guid namespaceId, ReadOnlySpan<byte> name) → Guid` — same as above but accepts a pre-encoded byte span; useful when the caller already has UTF-8 bytes or wants to avoid an intermediate string allocation.
## Internal Logic
- The .NET 10 `Guid.ToByteArray()` method emits the first three fields in little-endian (Microsoft historical behavior); RFC 9562 requires big-endian. The module uses a local `ToBigEndianByteArray(Guid)` helper that byte-swaps the first 4 bytes (time_low), the next 2 bytes (time_mid), and the next 2 bytes (time_hi_and_version) to produce the canonical big-endian layout before hashing. The same byte-swap is reversed when assembling the output `Guid` from the hash digest, so the in-memory `Guid` value still round-trips through `ToString()` to the expected hex form.
- SHA-1 is invoked via `SHA1.HashData(buffer)` (.NET 7+) which produces the 20-byte digest in one shot; only the first 16 bytes feed the resulting UUID (per RFC).
- The function is allocation-light for typical tile-key sizes: the hash input buffer is stack-allocated via `Span<byte>` when the namespace+name byte-length fits in 1024 bytes (always true for `{z}/{x}/{y}` and `{z}/{x}/{y}/{source}/{flight_id}` strings); larger payloads fall back to a pooled `byte[]`.
- The function is thread-safe (no shared mutable state).
## Reference Vectors
`SatelliteProvider.Tests/Uuidv5Tests.cs` pins 10 reference vectors generated by Python (`uuid.uuid5(TILE_NAMESPACE, name)`). Each vector pairs an input `name` with the expected `Guid` string. The C# implementation must produce byte-identical output. Two representative pairs:
| Name | Expected UUIDv5 |
|------|-----------------|
| `"18/12345/23456"` | `38b26f49-a966-5121-aaf4-9cc476f57869` |
| `"18/12345/23456/google_maps/00000000-0000-0000-0000-000000000000"` | `e228d1aa-25d4-556e-a72d-e0484756e165` |
The second value is observable end-to-end: a fresh `GET /api/satellite/tiles/latlon?Latitude=47.461747&Longitude=37.647063&ZoomLevel=18` returns `tileId = e228d1aa-25d4-556e-a72d-e0484756e165` because `(47.461747, 37.647063)` maps to slippy `(z=18, x=158485, y=91707)` — and the integration test asserts that exact value.
## Dependencies
- `System.Security.Cryptography.SHA1`
- `System.Buffers.Binary.BinaryPrimitives` (for big-endian byte-swaps)
- `System.Buffers.ArrayPool<byte>` (for the >1024-byte fallback path)
No third-party packages. No NuGet additions for AZ-503.
## Consumers
- `SatelliteProvider.Services.TileDownloader.TileService.BuildTileEntity` — computes `Id` and `LocationHash` for every newly downloaded Google Maps tile.
- `SatelliteProvider.Services.TileDownloader.UavTileUploadHandler.PersistAsync` — computes `Id` and `LocationHash` for every UAV upload.
- `SatelliteProvider.IntegrationTests.UavUploadTests` — seeds `location_hash` values via raw SQL when bypassing the application code path.
- `SatelliteProvider.IntegrationTests.MigrationTests` — generates expected UUIDv5 outputs to validate migration 014's `pg_temp.uuidv5` PL/pgSQL backfill function.
## Data Models
Operates only on `Guid` and `string` / `Span<byte>`. No persistence model.
## Configuration
None. The namespace constant is pinned in source.
## External Integrations
None (pure computation).
## Security
The function is deterministic by design — it is NOT a cryptographic hash for security purposes. Two callers with the same `(namespace, name)` will always produce the same output. Treat the result as a content/location handle, not a secret. SHA-1 is used for RFC 9562 compatibility, not for collision resistance against an adversary.
## Tests
`SatelliteProvider.Tests/Uuidv5Tests.cs`:
- `Create_MatchesPythonReferenceVectors_AC1` — 10 reference vectors (AZ-503 AC-1).
- `Create_IsDeterministic` — re-running with the same inputs returns the same `Guid`.
- `Create_SetsVersionAndVariantBits` — asserts the version nibble is `5` and the variant top-2-bits are `10`.
@@ -23,7 +23,7 @@ Runs DbUp-based SQL migrations against PostgreSQL on application startup. Ensure
## Consumers
- `Program.cs` — instantiated directly (not via DI) and called during startup. If migration fails, the application throws and does not start.
## Migrations (13 scripts)
## Migrations (14 scripts)
1. `001_CreateTilesTable.sql`
2. `002_CreateRegionsTable.sql`
3. `003_CreateIndexes.sql`
@@ -37,6 +37,7 @@ Runs DbUp-based SQL migrations against PostgreSQL on application startup. Ensure
11. `011_AddTileCoordinates.sql`
12. `012_DropTileVersionConstraint.sql` — drops the legacy 5-col `(latitude, longitude, tile_zoom, tile_size_meters, version)` unique index, replaces with 4-col `idx_tiles_unique_location` (preparation for AZ-484).
13. `013_AddTileSourceAndCapturedAt.sql` — AZ-484 multi-source tile storage. Transactional. Adds `source` (VARCHAR(32) NOT NULL DEFAULT 'google_maps') and `captured_at` (TIMESTAMP NOT NULL) columns; backfills existing rows with `source='google_maps'`, `captured_at=created_at`; drops `idx_tiles_unique_location` and creates 5-col `idx_tiles_unique_location_source` on `(latitude, longitude, tile_zoom, tile_size_meters, source)`. Idempotent against partial replays.
14. `014_AddTileIdentityColumns.sql` — AZ-503 tile-identity foundation. Transactional. Enables the `pgcrypto` extension (`CREATE EXTENSION IF NOT EXISTS pgcrypto`) for the in-migration SHA-1 digest. Adds `flight_id` (UUID NULL), `location_hash` (UUID — backfilled then set NOT NULL), `content_sha256` (BYTEA NULL), `legacy_id` (UUID NULL). Defines a transactional `pg_temp.uuidv5(namespace, name)` PL/pgSQL function that mirrors `SatelliteProvider.Common.Utils.Uuidv5.Create` byte-for-byte, then backfills `location_hash = pg_temp.uuidv5(TILE_NAMESPACE, '{tile_zoom}/{tile_x}/{tile_y}')` and `legacy_id = id` for every pre-existing row. Drops AZ-484's `idx_tiles_unique_location_source` and creates `idx_tiles_unique_identity` UNIQUE on `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid))` plus a non-unique `idx_tiles_location_hash` on `(location_hash)`. Safe to replay on a partially-migrated database because column adds are `IF NOT EXISTS`-equivalent and `pg_temp.uuidv5` is deterministic — re-running yields the same `location_hash` values.
## Configuration
Receives connection string directly as constructor parameter.
@@ -7,12 +7,17 @@ Database entity classes that map directly to PostgreSQL tables via Dapper. Prope
### TileEntity
Maps to `tiles` table.
- `Id` (Guid), `TileZoom` (int), `TileX` (int), `TileY` (int)
- `Id` (Guid) — AZ-503: deterministic UUIDv5 of `{tile_zoom}/{tile_x}/{tile_y}/{source}/{flight_id or '00000000-0000-0000-0000-000000000000'}` under namespace `Uuidv5.TileNamespace`. Stable across re-ingests; preserved on UPSERT conflict.
- `TileZoom` (int), `TileX` (int), `TileY` (int)
- `Latitude`, `Longitude` (double), `TileSizeMeters` (double), `TileSizePixels` (int)
- `ImageType` (string), `MapsVersion` (string?), `Version` (int) — `MapsVersion`/`Version` are vestigial post-AZ-484 (kept nullable for backward compatibility; no longer part of the unique key)
- `Source` (string) — AZ-484 producer wire value, defaults to `TileSourceConverter.GoogleMapsWireValue` (`"google_maps"`). Stored as plain string (not the `TileSource` enum) due to Dapper issue #259 — see `_docs/LESSONS.md` L-001. Convert via `SatelliteProvider.Common.Enums.TileSourceConverter.{ToWireValue,FromWireValue}`.
- `CapturedAt` (DateTime, UTC) — AZ-484 imagery acquisition timestamp; drives the most-recent-across-sources selection.
- `FilePath` (string), `CreatedAt`, `UpdatedAt` (DateTime)
- `FlightId` (Guid?) — AZ-503: optional flight identifier. `null` for Google Maps tiles; populated from `UavTileMetadata.FlightId` on UAV uploads. Part of the AZ-503 UPSERT conflict key via `COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)`, so two flights uploading the same `(z, x, y)` cell produce two separate rows.
- `LocationHash` (Guid) — AZ-503 `NOT NULL`: deterministic UUIDv5 of `{tile_zoom}/{tile_x}/{tile_y}` under `Uuidv5.TileNamespace`. Identical across flights and sources for the same cell. Backfilled in migration 014 via a `pg_temp.uuidv5` PL/pgSQL function; subsequent inserts compute it in the application layer (`TileService` + `UavTileUploadHandler`). Reserved for AZ-505's Leaflet covering index (`POST /tiles/inventory`) — not yet on a unique constraint.
- `ContentSha256` (byte[]?) — AZ-503: SHA-256 digest of the JPEG body. Application code enforces `NOT NULL` for new writes via `TileService.BuildTileEntity` (Google Maps) and `UavTileUploadHandler.PersistAsync` (UAV). The DB column is `bytea NULL` because legacy pre-migration rows could not be backfilled reliably from disk (file paths are volatile). See `batch_02_cycle5_report.md` "Low maintainability finding" for the rationale.
- `LegacyId` (Guid?) — AZ-503: pre-migration `id` value, populated by migration 014 from every existing row's `id`. Preserves random-`Guid` provenance for one cycle (per AZ-503 Risk 1 mitigation) so external references to the old id can still be diagnosed before deletion.
### RegionEntity
Maps to `regions` table.
@@ -9,8 +9,8 @@ Dapper-based repository for the `tiles` table. Handles CRUD operations and spati
- `GetByIdAsync(Guid id) → Task<TileEntity?>`
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds the most-recent tile across all sources for the given slippy coordinates. Selection rule: `ORDER BY captured_at DESC, updated_at DESC, id DESC LIMIT 1` (AZ-484 v1.0.0 contract).
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileEntity>>`: spatial bounding box query (expanded by 2 × tile size to cover edges); applies `DISTINCT ON (latitude, longitude, tile_zoom, tile_size_meters)` per AZ-484 to return at most one row per cell — the most-recent across sources — preserving the historical caller-facing order `latitude DESC, longitude ASC`.
- `InsertAsync(TileEntity tile) → Task<Guid>`: per-source UPSERT — `ON CONFLICT (latitude, longitude, tile_zoom, tile_size_meters, source) DO UPDATE file_path, tile_x, tile_y, captured_at, updated_at` (AZ-484 5-column unique key).
- `UpdateAsync(TileEntity tile) → Task<int>`: full row update by `id` including `source` and `captured_at`.
- `InsertAsync(TileEntity tile) → Task<Guid>`: AZ-503 integer-only + flight-aware UPSERT — `ON CONFLICT (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO UPDATE file_path, latitude, longitude, captured_at, location_hash, content_sha256, updated_at`. `id` is intentionally NOT overwritten on conflict — preserves AZ-503 AC-2 idempotence (same inputs ⇒ same `id`). Supersedes the AZ-484 5-column float-based unique key (`idx_tiles_unique_location_source`).
- `UpdateAsync(TileEntity tile) → Task<int>`: full row update by `id` including `source`, `captured_at`, `flight_id`, `location_hash`, and `content_sha256`.
- `DeleteAsync(Guid id) → Task<int>`
### TileRepository (implementation)
@@ -18,8 +18,8 @@ Constructs a new `NpgsqlConnection` per method call (no connection pooling at th
## Internal Logic
- `GetTilesByRegionAsync` calculates a bounding box by expanding the requested region by 2 × tile size to ensure edge tiles are included. Uses meters-to-degrees approximation via `GeoUtils` (post-AZ-377 — single source of truth for Earth constants).
- `InsertAsync` uses a per-source UPSERT pattern keyed on the 5-column unique index `idx_tiles_unique_location_source` (created by migration 013). Two producers (e.g., `google_maps` + `uav`) coexist for the same cell; same-source re-insert refreshes `captured_at` and `updated_at`.
- `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` apply the AZ-484 selection rule: most-recent across sources, deterministic tie-break on `(captured_at DESC, updated_at DESC, id DESC)`.
- `InsertAsync` uses the AZ-503 integer-only + flight-aware UPSERT keyed on `idx_tiles_unique_identity` (created by migration 014, replacing the AZ-484 `idx_tiles_unique_location_source`). The conflict key uses `COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)` so anonymous (`flight_id IS NULL`) and per-flight UAV rows share a flat key space. Two producers (`google_maps` + `uav`) at the same cell with the same `flight_id` (typically `NULL` for `google_maps`) still coexist via `source` discrimination. Same-source same-flight re-insert refreshes `file_path`, `latitude`, `longitude`, `captured_at`, `location_hash`, `content_sha256`, `updated_at` — but NOT `id` (idempotence — AZ-503 AC-2).
- `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` apply the AZ-484 selection rule unchanged: most-recent across sources, deterministic tie-break on `(captured_at DESC, updated_at DESC, id DESC)`. AZ-503 does NOT rewrite the read path to use `location_hash` — that's deferred to AZ-505 alongside the Leaflet covering index.
- `TileEntity.Source` is a plain `string` storing the snake_case wire value (`'google_maps'` | `'uav'`); enum<->wire conversion happens via `SatelliteProvider.Common.Enums.TileSourceConverter`. This avoids Dapper issue #259 (TypeHandler<T> bypass for enum reads — see `_docs/LESSONS.md` L-001).
- `FindExistingTileAsync` was removed by AZ-376 (see `_docs/04_refactoring/03-code-quality-refactoring/`).
@@ -30,7 +30,7 @@ Constructs a new `NpgsqlConnection` per method call (no connection pooling at th
- `Microsoft.Extensions.Logging`
## Contract
Implements the frozen v1.0.0 contract `_docs/02_document/contracts/data-access/tile-storage.md`. Schema invariants Inv-1..Inv-5 are enforced here (UPSERT key, selection rule, source value space).
Implements the frozen v1.0.0 contract `_docs/02_document/contracts/data-access/tile-storage.md` plus the AZ-503-introduced columns (`flight_id`, `location_hash`, `content_sha256`, `legacy_id`) and the integer-only UPSERT key. Schema invariants Inv-1..Inv-5 (UPSERT semantics, selection rule, source value space) are preserved; the only contract change is that the UPSERT conflict detection no longer depends on bit-identical float `latitude`/`longitude` (AZ-503 AC-4).
## Consumers
- `TileService` — all read/write operations
@@ -21,7 +21,8 @@ Orchestrates tile downloading and persistence. Bridges the downloader (Google Ma
## Internal Logic
- New rows write `Version = null` and `MapsVersion = null` (post-AZ-357 / AZ-373); the `version` and `maps_version` columns are retained for backward compatibility with pre-existing rows
- AZ-484: `BuildTileEntity` stamps every newly downloaded row with `Source = TileSourceConverter.ToWireValue(TileSource.GoogleMaps)` (wire value `"google_maps"`) and `CapturedAt = DateTime.UtcNow`. The Google Maps download path is the only producer of `'google_maps'` rows; UAV ingestion (separate task) is the only producer of `'uav'` rows.
- `MapToMetadata(TileEntity) → TileMetadata`: entity-to-DTO mapping (static helper); `MapsVersion` is no longer projected onto `TileMetadata` / `DownloadTileResponse`. `Source` and `CapturedAt` are not currently projected to the public DTO (no API contract change observable for AZ-484).
- AZ-503: `BuildTileEntity` computes deterministic identity fields — `Id = Uuidv5.Create(Uuidv5.TileNamespace, "{z}/{x}/{y}/google_maps/00000000-0000-0000-0000-000000000000")`, `LocationHash = Uuidv5.Create(Uuidv5.TileNamespace, "{z}/{x}/{y}")`, `ContentSha256 = SHA256.HashData(<jpeg bytes from disk>)`. `FlightId` is always `null` for Google Maps tiles. No `Guid.NewGuid()` remains on this path.
- `MapToMetadata(TileEntity) → TileMetadata`: entity-to-DTO mapping (static helper); `MapsVersion` is no longer projected onto `TileMetadata` / `DownloadTileResponse`. `Source`, `CapturedAt`, `FlightId`, `LocationHash`, `ContentSha256` are not currently projected to the public DTO (no API contract change observable for AZ-484 or AZ-503).
- `TileSizePixels` sourced from `MapConfig.TileSizePixels` (default 256, post-AZ-371); image type fixed at `"jpg"`
- `IMemoryCache` keyed by `(z, x, y)` with 1h absolute / 30min sliding expiration; populated on first hit and on downloader fallback
@@ -31,6 +32,8 @@ Orchestrates tile downloading and persistence. Bridges the downloader (Google Ma
- `IMemoryCache` (registered by `AddTileDownloader()`)
- `SatelliteProvider.Common.DTO` — GeoPoint, TileMetadata, TileBytes
- `SatelliteProvider.Common.Enums``TileSource`, `TileSourceConverter` (AZ-484)
- `SatelliteProvider.Common.Utils.Uuidv5` (AZ-503) — deterministic UUIDv5 generator + `TileNamespace` constant
- `System.Security.Cryptography.SHA256` (AZ-503) — content digest
- `SatelliteProvider.DataAccess.Models` — TileEntity
## Consumers
@@ -11,9 +11,9 @@ Console application that runs end-to-end integration tests against a live API in
- `BasicRouteTests` — route creation with intermediate points
- `ComplexRouteTests` — routes with geofencing
- `ExtendedRouteTests` — routes with `requestMaps: true` and tile ZIP creation
- `MigrationTests` — direct PostgreSQL schema/index validation (no HTTP). AZ-484 cycle added: `NewUniqueConstraintIncludesSourceColumn_AZ484_AC1`, `BackfillUpdateAssignsGoogleMapsAndCapturedAt_AZ484_AC4`, `MultiSourceInsertCoexistsUnderNewIndex_AZ484_AC1`, `MostRecentAcrossSourcesSelection_AZ484_AC2`, `SameSourceUpsertReplacesPreviousRow_AZ484_AC3` (latter four use temp tables to keep production data untouched).
- `MigrationTests` — direct PostgreSQL schema/index validation (no HTTP). AZ-484 cycle added: `BackfillUpdateAssignsGoogleMapsAndCapturedAt_AZ484_AC4`, `MultiSourceInsertCoexistsUnderNewIndex_AZ484_AC1`, `MostRecentAcrossSourcesSelection_AZ484_AC2`, `SameSourceUpsertReplacesPreviousRow_AZ484_AC3` (latter four use temp tables to keep production data untouched). AZ-503 (cycle 5) added: `Az503ColumnsExistAndLocationHashIsNotNull` (asserts the 4 new columns + `location_hash NOT NULL`), `Az503NewUniqueIndexCoversIntegerKeyAndFlightId` (asserts `idx_tiles_unique_identity` columns + `COALESCE(flight_id, ...)` predicate), `Az503LocationHashBackfillIsDeterministic` (computes `pg_temp.uuidv5("18/12345/23456")` and compares byte-for-byte against the C# `Uuidv5.Create` output on 3 sampled live rows); the AZ-484 supersession test was renamed to `Az503MigrationSupersedesAz484UniqueIndex` and asserts `idx_tiles_unique_location_source` no longer exists.
- `JwtIntegrationTests` (added by AZ-487 cycle 2; helpers consolidated by AZ-491 cycle 3; iss/aud scenarios added by AZ-494 cycle 3) — `AnonymousRequest_To_AnyEndpoint_Returns401`, `ExpiredToken_Returns401`, `InvalidSignature_Returns401`, `ValidToken_Returns200_OnHealthyEndpoint`, `WrongIssuer_Returns401` (AZ-494 AC-1), `WrongAudience_Returns401` (AZ-494 AC-2), `SwaggerDocument_AdvertisesBearerSecurityScheme`. HS256 token minting lives in the shared `SatelliteProvider.TestSupport.JwtTokenFactory` (consumed via `ProjectReference`); runner-specific concerns (`JwtTestHelpers.ResolveSecretOrThrow` / `ResolveIssuerOrThrow` / `ResolveAudienceOrThrow`, `MintAuthenticated` / `MintExpired` convenience wrappers that auto-fill iss+aud from env, `AttachDefaultAuthorization`, `DefaultSubject = "integration-tests"`) remain in this project. The test runner sets `JWT_SECRET` + `JWT_ISSUER` + `JWT_AUDIENCE` on the API container and attaches a Bearer token (with matching iss/aud) to every existing test's HTTP requests so the pre-cycle-2 suite continues to pass.
- `UavUploadTests` (added by AZ-488, cycle 2; coordinate-counter promoted to defense-in-depth by AZ-493 cycle 3) — `HappyPathSingleItem_PersistsRow`, `MixedBatch_ReturnsPerItemResults`, `MultiSourceCoexistence_AZ484_Cycle2`, `SameSourceUpsert_AZ484_Cycle2`, `NoToken_Returns401`, `ValidTokenWithoutGpsPermission_Returns403`, `OversizedBatch_Returns400`. The wall-clock-seeded `_coordinateCounter` is retained as a belt-and-suspenders safeguard alongside the AZ-493 startup DB-reset (below) — if a developer runs with `--keep-state`, or the DB-reset path is skipped for any reason, the wall-clock seed still spreads coordinates across runs so the per-source unique index does not collide.
- `UavUploadTests` (added by AZ-488, cycle 2; coordinate-counter promoted to defense-in-depth by AZ-493 cycle 3; AZ-503 cycle 5 added 2 more tests) — `HappyPathSingleItem_PersistsRow`, `MixedBatch_ReturnsPerItemResults`, `MultiSourceCoexistence_AZ484_Cycle2`, `SameSourceUpsert_AZ484_Cycle2`, `NoToken_Returns401`, `ValidTokenWithoutGpsPermission_Returns403`, `OversizedBatch_Returns400`, plus AZ-503: `MultiFlightUavRowsCoexist_AZ503_AC3` (two flights at the same cell → two rows, one `location_hash`, two `file_path`s under `./tiles/uav/{flight_id}/...`) and `FloatRoundingDoesNotBreakIdempotence_AZ503_AC4` (two uploads with float-distinct `latitude` recomputed from `TileToWorldPos` collapse to a single row because the conflict key is integer-only). The AZ-503 migration made `location_hash NOT NULL`, so the cycle-2 `MultiSourceCoexistence_AZ484_Cycle2` seeder was updated to compute `location_hash` via `Uuidv5.Create` (canonical name `"{zoom}/0/0"`) before the raw SQL `INSERT` — this required adding a `ProjectReference` from `SatelliteProvider.IntegrationTests` to `SatelliteProvider.Common`. The wall-clock-seeded `_coordinateCounter` is retained as a belt-and-suspenders safeguard alongside the AZ-493 startup DB-reset (below) — if a developer runs with `--keep-state`, or the DB-reset path is skipped for any reason, the wall-clock seed still spreads coordinates across runs so the unique index does not collide.
- `StubAndErrorContractTests` (existing) — updated in cycle 2 to drop the legacy `StubUpload_Returns501` expectation since AZ-488 implemented the endpoint.
### Supporting Classes
@@ -41,6 +41,7 @@ Console application that runs end-to-end integration tests against a live API in
- `ProjectReference` to `SatelliteProvider.TestSupport` (added by AZ-491; provides `JwtTokenFactory`. Added by AZ-493; provides `IntegrationTestResetGuard`).
- Communicates with the API exclusively via HTTP for end-to-end tests; communicates with PostgreSQL directly only via the dedicated DB-reset hook + the existing `MigrationTests` schema assertions.
- NuGet: `Npgsql` 9.0.2 (Postgres client for DB-reset + MigrationTests), `SixLabors.ImageSharp` 3.1.11 (UAV fixture image generation).
- ProjectReferences: `SatelliteProvider.Api` (running service for the integration runner), `SatelliteProvider.TestSupport` (canonical `JwtTokenFactory` + `IntegrationTestResetGuard`), `SatelliteProvider.Common` (added by AZ-503 so the `MultiSourceCoexistence_AZ484_Cycle2` seeder can compute `location_hash` via `Uuidv5.Create` instead of duplicating the UUIDv5 algorithm in T-SQL fixtures).
## Consumers
- `docker-compose.tests.yml` — runs as a container that depends on the API service
+6 -3
View File
@@ -15,11 +15,14 @@ Existing baseline (pre-cycle-2) test classes cover `TileService`, `RegionService
### AZ-488 — UAV tile upload
- `UavTileQualityGateTests` — one happy path + ≥ 1 reject path per rule (Rule 1 INVALID_FORMAT × 2, Rule 2 SIZE_OUT_OF_BAND × 2, Rule 3 WRONG_DIMENSIONS × 1, Rule 4 CAPTURED_AT_FUTURE / _TOO_OLD × 2, Rule 5 IMAGE_TOO_UNIFORM × 1) + rule-ordering determinism. Uses a `FixedTimeProvider` for Rule-4 isolation and `UavTileImageFactory` for deterministic JPEG fixtures.
- `UavTileUploadHandlerTests` — end-to-end with a mocked `ITileRepository`: 1-item happy path, 3-item mixed batch (file written + `InsertAsync` called only for accepted), per-source UPSERT pass-through.
- `UavTileFilePathTests` — verifies `BuildUavTileFilePath` produces `tiles/uav/{z}/{x}/{y}.jpg` for sample (z, x, y) tuples and that integer-typed coordinates make string-injection of path traversal impossible.
- `UavTileUploadHandlerTests` — end-to-end with a mocked `ITileRepository`. Cycle-2 baseline: 1-item happy path, 3-item mixed batch (file written + `InsertAsync` called only for accepted), per-source UPSERT pass-through. AZ-503 additions: `HandleAsync_TwoFlightsSameCell_ProduceDistinctIdsAndPathsButSameLocationHash` (multi-flight coexistence with shared `location_hash`); `HandleAsync_IdenticalUpload_ProducesIdenticalIdAndDeterministicContentSha` (idempotent re-insert preserves deterministic `id` + `content_sha256`).
- `Authentication/PermissionsRequirementTests``PermissionsAuthorizationHandler` correctly accepts a `permissions` claim shaped as a single string OR as a JSON array, rejects when the requested permission is absent, and short-circuits when the principal has no `permissions` claim at all.
- `TestUtilities/UavTileImageFactory` — programmatic JPEG factories used by the gate + handler tests: `CreateValidJpeg(width, height, seed)`, `CreateUniformJpeg`, `CreatePng` (for Rule 1 negative path).
### AZ-503 — Tile identity foundation
- `Uuidv5Tests` — pure-C# UUIDv5 generator parity tests. `Create_MatchesPythonReferenceVectors_AC1` pins 10 reference vectors generated by Python's `uuid.uuid5(TILE_NAMESPACE, name)`; `Create_IsDeterministic` asserts repeated runs return the same `Guid`; `Create_SetsVersionAndVariantBits` asserts the version nibble is `5` and the variant top-2-bits are `10` (RFC 9562 §5.5).
- `UavTileFilePathTests` (rewritten for AZ-503 from the cycle-2 placeholder) — covers `BuildUavTileFilePath(Guid? flightId, int z, int x, int y)` across three cases: `BuildUavTileFilePath_AnonymousFlight_UsesNoneSegment` (null `flightId` → literal `none` segment), `BuildUavTileFilePath_PerFlight_UsesFlightIdDirectory` (per-flight segment), `BuildUavTileFilePath_DifferentFlights_ProduceDifferentPaths` (path-distinctness across flights at the same cell). Integer-typed coordinates and the `Guid? flightId` parameter together still preclude string-injection path traversal.
## Internal Logic
- Tests follow Arrange / Act / Assert. Time-dependent paths inject a `FixedTimeProvider` (cycle-2 addition) so Rule 4 has deterministic age windows.
- `JwtSecurityTokenHandler.MapInboundClaims = false` is set explicitly in JWT tests so claims read by their original names (`sub`, `permissions`, …) rather than the framework-remapped names.
@@ -33,4 +36,4 @@ Existing baseline (pre-cycle-2) test classes cover `TileService`, `RegionService
- CI pipeline (`01-test.yml`) and `scripts/run-tests.sh --unit-only` run `dotnet test` against this project.
## Tests
This IS the test module. Cycle-2 added ~25 unit tests on top of the existing baseline; the full project executes in seconds (no external services required).
This IS the test module. Cycle-2 added ~25 unit tests on top of the existing baseline; cycle-5 (AZ-503) added 6 more (3 in `Uuidv5Tests`, 3 in `UavTileFilePathTests`) plus 2 new methods in `UavTileUploadHandlerTests`. The full project executes in seconds (no external services required).
+63
View File
@@ -0,0 +1,63 @@
# Cycle 5 — Documentation Ripple Log
**Cycle**: 5 (AZ-504 perf-script pipefail fix + AZ-503-foundation UUIDv5 tile identity)
**Generated by**: `/document` skill (task mode) during autodev Step 13
**Resolution method**: `Grep --type cs` against every new or changed symbol introduced by the two tasks. C# `using`-based import analysis on `Uuidv5`, `TileEntity` (new properties), `UavTileMetadata.FlightId`, `TileRepository.InsertAsync` (changed UPSERT key). No static-analyzer (NDepend, etc.) was used — the surface is small and the literal usage scan is exhaustive.
## Directly-changed source files (cycle 5)
- `scripts/run-performance-tests.sh` (AZ-504, lines 416417) — wrapped `grep -o` counters in `{ … || true; }` to survive zero-match cases under `set -o pipefail`. Pure shell; no C# importers.
- `SatelliteProvider.Common/Utils/Uuidv5.cs` (AZ-503, **new**) — pure-C# UUIDv5 generator + `TileNamespace` constant.
- `SatelliteProvider.Common/DTO/UavTileMetadata.cs` (AZ-503, modified) — added optional `Guid? FlightId` property.
- `SatelliteProvider.DataAccess/Migrations/014_AddTileIdentityColumns.sql` (AZ-503, **new**) — adds `flight_id`, `location_hash`, `content_sha256`, `legacy_id`; backfills `location_hash`; supersedes the AZ-484 unique index with `idx_tiles_unique_identity`.
- `SatelliteProvider.DataAccess/Models/TileEntity.cs` (AZ-503, modified) — 4 new properties (`FlightId`, `LocationHash`, `ContentSha256`, `LegacyId`).
- `SatelliteProvider.DataAccess/Repositories/TileRepository.cs` (AZ-503, modified) — `InsertAsync` UPSERT now uses the integer-only + flight-aware conflict key; `id` not overwritten on conflict; `UpdateAsync` writes all four new columns.
- `SatelliteProvider.Services.TileDownloader/TileService.cs` (AZ-503, modified) — `BuildTileEntity` computes deterministic `Id`, `LocationHash`, `ContentSha256`; `FlightId` always null for Google Maps.
- `SatelliteProvider.Services.TileDownloader/UavTileUploadHandler.cs` (AZ-503, modified) — `PersistAsync` reads `metadata.FlightId`, computes deterministic identity fields, writes file to `./tiles/uav/{flight_id or 'none'}/{z}/{x}/{y}.jpg`.
- `SatelliteProvider.IntegrationTests/SatelliteProvider.IntegrationTests.csproj` (AZ-503, modified) — added `ProjectReference` to `SatelliteProvider.Common` so seed SQL can call `Uuidv5.Create`.
- `SatelliteProvider.IntegrationTests/UavUploadTests.cs` (AZ-503, modified seeder + 2 new tests).
- `SatelliteProvider.IntegrationTests/MigrationTests.cs` (AZ-503, 3 new assertions; renamed AZ-484 supersession test).
- `SatelliteProvider.Tests/Uuidv5Tests.cs` (AZ-503, **new** — 10 Python-parity reference vectors).
- `SatelliteProvider.Tests/UavTileFilePathTests.cs` (AZ-503, **new** — per-flight on-disk path).
- `SatelliteProvider.Tests/UavTileUploadHandlerTests.cs` (AZ-503, 2 new tests added; existing tests unchanged).
## Importer scan results
| Symbol | Importer count | Importer files | Component touched |
|--------|----------------|----------------|-------------------|
| `SatelliteProvider.Common.Utils.Uuidv5` / `Uuidv5.Create` / `Uuidv5.TileNamespace` | 5 | `TileService.cs`, `UavTileUploadHandler.cs`, `Uuidv5Tests.cs`, `MigrationTests.cs`, `UavUploadTests.cs` | TileDownloader (production), Tests (unit + integration) |
| `UavTileMetadata.FlightId` (new field) | 2 | `UavTileUploadHandler.cs` (read), `UavUploadTests.cs` (serializer payload) | TileDownloader, Tests (integration) |
| `TileEntity.FlightId` / `LocationHash` / `ContentSha256` / `LegacyId` (new columns) | 3 | `TileRepository.cs`, `TileService.cs`, `UavTileUploadHandler.cs` | DataAccess, TileDownloader |
| `TileRepository.InsertAsync` (changed UPSERT conflict key) | 4 | `TileService.cs`, `UavTileUploadHandler.cs`, `RegionService.cs`, `TileServiceTests.cs` (via mock) | TileDownloader, RegionProcessing, Tests (unit) |
| `pgcrypto` extension (new DB dependency from migration 014) | 1 | `014_AddTileIdentityColumns.sql` (in-migration temp PL/pgSQL only) | DataAccess (migration-only; runtime code does not depend on pgcrypto) |
## Doc refresh decisions
All importers land inside components that already received targeted updates during Step 10 (Implement) and this Step 13:
- **Common** — created `_docs/02_document/modules/common_uuidv5.md` (new utility module doc). Updated `_docs/02_document/modules/common_dtos.md` `UavTileMetadata` section to document `FlightId` + the `null`-vs-flight semantics.
- **DataAccess** — updated `_docs/02_document/modules/dataaccess_models.md` (4 new `TileEntity` properties) and `_docs/02_document/modules/dataaccess_tile_repository.md` (new UPSERT semantics; `id`-not-overwritten invariant). Updated `_docs/02_document/components/02_data_access/description.md` (data-access patterns table, caveats — replaced AZ-484 5-col index entry with AZ-503 6-col integer-only index; documented deterministic identity).
- **TileDownloader** — updated `_docs/02_document/modules/services_tile_service.md` (`BuildTileEntity` deterministic identity; SHA-256 dependency; new module dependencies on `Uuidv5` and `SHA256`).
- **TileDownloader / `UavTileUploadHandler`** — no dedicated module doc exists for this handler; its surface is already covered by `api_program.md` and the `uav-tile-upload.md` contract. Both updated below.
System-level docs also updated this pass:
- `architecture.md` — extended the append-by-source bullet to append-by-source-and-flight (AZ-503), added a cross-repo deterministic-id principle, refreshed ADR-004 (file-storage layout) and the multi-source-producers section to mention `flightId` and the deterministic `id`.
- `data_model.md` — added the 4 new `tiles` columns, the new index `idx_tiles_unique_identity`, the new index `idx_tiles_location_hash`, the new UPSERT contract semantics, and migration 014 in the migration history.
- `contracts/api/uav-tile-upload.md` — minor version bump v1.0.0 → v1.1.0: added optional `flightId` field, documented per-flight on-disk path, documented deterministic `tileId`. Backward-compatible by design.
- `tests/blackbox-tests.md` and `tests/traceability-matrix.md` — updated during Step 12 (Test-Spec Sync) with BT-19 … BT-22 + per-AC trace rows.
## No-ripple components
These components were NOT touched by cycle-5 changes and require no doc update:
- **RegionProcessing** — `RegionService` consumes `ITileRepository.InsertAsync`/`GetTilesByRegionAsync` unchanged. Region read path uses the AZ-484 selection rule which AZ-503 preserved.
- **RouteManagement** — no imports against cycle-5 symbols; route processing reads tiles only via the unchanged region read path.
- **WebApi (`Program.cs`)** — endpoint contract surface unchanged on the wire (the optional `flightId` field is transparent at the routing level; deserialization is owned by `UavTileBatchMetadataPayload` / `UavTileMetadata` in Common). No `api_program.md` edits needed for AZ-503.
## Parse-failure / heuristic notes
None — every symbol resolved via direct `Grep`. No fallback heuristic was needed.
## AZ-504 ripple
`scripts/run-performance-tests.sh` is a shell harness with no code-side importers. Its docs are owned by the test-spec sync layer (`tests/traceability-matrix.md` AC-1..AC-4 rows, updated during Step 12) and the `_docs/02_tasks/done/AZ-504_*.md` task spec. No module-doc ripple.
+38
View File
@@ -169,3 +169,41 @@ All Cycle-2 UAV scenarios run with a JWT containing `permissions: ["GPS"]` (per
**Pass criterion**: status == 200 AND BT-01's pass criterion AND no behavioral change vs pre-AZ-487 baseline.
**AC trace**: AZ-487 AC-4 (handler unchanged); validates AZ-487 AC-8 (existing suite parity).
---
## Cycle 5 — AZ-503 Tile Identity → UUIDv5 + integer UPSERT (foundation)
All Cycle-5 UAV scenarios reuse the AZ-488 envelope. The new observable surface is: a `flightId` field in `UavTileMetadata`, deterministic `tileId` / `locationHash` values, and a per-flight on-disk layout `./tiles/uav/{flight_id|none}/{z}/{x}/{y}.jpg`. No new HTTP route or wire change beyond the optional metadata field.
## BT-19: UAV Upload — Multi-Flight Coexistence with Shared `location_hash`
**Trigger**: POST `/api/satellite/upload` twice for the SAME `(z=18, tile_x, tile_y, tile_size_meters)` from two different `flight_id` values `F1 ≠ F2` (each upload sends `metadata.flightId = F`, valid 256×256 JPEG, fresh `capturedAt`).
**Precondition**: Empty `tiles` table for the chosen cell; valid `GPS` JWT.
**Expected**: HTTP 200 for both calls. After the second call, exactly TWO rows exist in `tiles` for `source='uav'` at the cell — one per flight. Both rows share the same `location_hash` (deterministic per `{z}/{x}/{y}`); each has a distinct `id` (deterministic per `{z}/{x}/{y}/uav/{flight_id}`). On disk, two files exist at `./tiles/uav/{F1}/{z}/{x}/{y}.jpg` and `./tiles/uav/{F2}/{z}/{x}/{y}.jpg`; `rm -rf ./tiles/uav/{F1}/` removes ONLY Flight F1's evidence.
**Pass criterion**: `SELECT COUNT(*) FROM tiles WHERE source='uav' AND (z,x,y,size_m)=(...)` == 2 AND `COUNT(DISTINCT location_hash)` == 1 AND `COUNT(DISTINCT id)` == 2 AND `COUNT(DISTINCT file_path)` == 2 AND both files present on disk.
**AC trace**: AZ-503 AC-3, AC-11.
## BT-20: UAV Upload — Idempotent Re-Insert Preserves Deterministic `tileId` and `content_sha256`
**Trigger**: POST `/api/satellite/upload` for cell `(z=18, x, y, size_m)` with `flightId = F` and JPEG body `B` and `capturedAt = T1`. Repeat the SAME body and `flightId` with `capturedAt = T2 > T1`.
**Precondition**: Empty `tiles` table for the cell; valid `GPS` JWT.
**Expected**: HTTP 200 for both calls. Exactly ONE row exists for the cell after both inserts. The row's `id` is identical before and after the second insert (deterministic UUIDv5 from `{z}/{x}/{y}/uav/{F}`). `updated_at` advances to T2; `created_at` is NOT regenerated. `content_sha256` equals `SHA-256(B)` externally computed; the second insert's `content_sha256` matches the first (byte-identical body).
**Pass criterion**: `SELECT COUNT(*)` == 1 AND `id` is stable AND `content_sha256 == sha256(B)` AND `created_at` unchanged AND `updated_at == T2`.
**AC trace**: AZ-503 AC-2, AC-7.
## BT-21: UAV Upload — Float-Rounded Coordinates Collapse to a Single Row
**Trigger**: POST `/api/satellite/upload` for cell `(z=18, x, y, size_m)` with `latitude = 47.123456789012345` and `flightId = F`. Repeat with the SAME `(x, y, z)` but `latitude` recomputed from `tile_center = TileToWorldPos(x, y, z)` (slightly different float representation).
**Precondition**: Empty `tiles` table for the cell; valid `GPS` JWT.
**Expected**: HTTP 200 for both calls. Exactly ONE row results — the integer-only UPSERT conflict key (`tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, ...)`) triggers despite the float-different `latitude` values.
**Pass criterion**: `SELECT COUNT(*) FROM tiles WHERE source='uav' AND tile_zoom=18 AND tile_x=x AND tile_y=y AND tile_size_meters=size_m` == 1.
**AC trace**: AZ-503 AC-4.
## BT-22: Migration 014 — Identity Columns Land and Backfill Is Deterministic
**Trigger**: Run `dotnet ef migrations apply` (via API container startup) against a DB with cycle-4 schema; then query `information_schema.columns` and `pg_indexes` for `tiles`.
**Precondition**: DB starts with migration 013 applied (cycle 4 baseline); `pgcrypto` available.
**Expected**: `tiles` table has `flight_id uuid NULL`, `location_hash uuid NOT NULL`, `content_sha256 bytea NULL`, `legacy_id uuid NULL`. `idx_tiles_unique_identity` exists as a UNIQUE index over `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-...'::uuid))`. AZ-484 index `idx_tiles_unique_location_source` is dropped. For any pre-existing row, `location_hash` equals `uuidv5(TILE_NAMESPACE, '{tile_zoom}/{tile_x}/{tile_y}')` byte-identically (validated against a SQL `pg_temp.uuidv5` reference function).
**Pass criterion**: All column / index assertions pass AND the deterministic backfill matches the reference function on 100% of sampled rows.
**AC trace**: AZ-503 AC-8.
+33 -1
View File
@@ -86,6 +86,22 @@
| AZ-500 AC-6 | All unit + integration tests pass on the migrated build | Full `./scripts/run-tests.sh --full` at cycle 4 Step 11 — 271/271 unit + integration suite green | ✓ |
| AZ-500 AC-7 | `docker-compose build` succeeds with no downgrade / framework / missing-image warnings | `run-tests.sh` Step 2 build path + `docker compose up -d --build` both succeeded; only warnings emitted are CS8604 nullable + ASPDEPR002 deprecation (neither category gated) | ✓ |
| AZ-500 AC-8 | Documentation reflects .NET 10 | `_docs/02_document/architecture.md` lines 5 + 67 (Tech Stack table) updated; `AGENTS.md` lines 9 + 240244 updated incl. Serilog fallback note | ✓ |
| AZ-503 AC-1 | UUIDv5 reference vectors match Python (≥10 cases) | `Uuidv5Tests.Create_MatchesPythonReferenceVectors_AC1` (unit) + version/variant bit assertions | ✓ |
| AZ-503 AC-2 | Insert is idempotent on identical inputs (id stable, created_at preserved) | BT-20 (blackbox); `UavTileUploadHandlerTests.HandleAsync_IdenticalUpload_ProducesIdenticalIdAndDeterministicContentSha` (unit) | ✓ |
| AZ-503 AC-3 | Multi-flight UAV uploads coexist (two ids, shared location_hash) | BT-19 (blackbox); `UavTileUploadHandlerTests.HandleAsync_TwoFlightsSameCell_ProduceDistinctIdsAndPathsButSameLocationHash` (unit); `UavUploadTests.MultiFlightUavRowsCoexist_AZ503_AC3` (integration) | ✓ |
| AZ-503 AC-4 | Float-rounded coordinates collapse to a single row | BT-21 (blackbox); `UavUploadTests.FloatRoundingDoesNotBreakIdempotence_AZ503_AC4` (integration) | ✓ |
| AZ-503 AC-7 | content_sha256 is computed and persisted; byte-identical bodies produce identical digest | BT-20 (blackbox); `UavTileUploadHandlerTests.HandleAsync_IdenticalUpload_ProducesIdenticalIdAndDeterministicContentSha` (unit) | ✓ |
| AZ-503 AC-8 | Migration 014 adds columns + supersedes AZ-484 index + backfills location_hash deterministically | BT-22 (blackbox); `MigrationTests.Az503ColumnsExistAndLocationHashIsNotNull`, `Az503NewUniqueIndexCoversIntegerKeyAndFlightId`, `Az503LocationHashBackfillIsDeterministic`, `Az503MigrationSupersedesAz484UniqueIndex` (integration) | ✓ |
| AZ-503 AC-11 | Per-flight on-disk separation (`./tiles/uav/{flight_id\|none}/{z}/{x}/{y}.jpg`) | BT-19 (blackbox); `UavTileFilePathTests.BuildUavTileFilePath_AnonymousFlight_UsesNoneSegment`, `_PerFlight_UsesFlightIdDirectory`, `_DifferentFlights_ProduceDifferentPaths` (unit); `UavUploadTests.MultiFlightUavRowsCoexist_AZ503_AC3` (integration; per-flight file_path assertion) | ✓ |
| AZ-503 AC-5 | Inventory endpoint `POST /api/satellite/tiles/inventory` returns one entry per requested coord | — | ◐ deferred → AZ-505 |
| AZ-503 AC-6 | Leaflet path returns most-recent variant via `location_hash` | — | ◐ deferred → AZ-505 |
| AZ-503 AC-9 | Inventory endpoint p95 ≤ 500 ms for 2500 tiles | — | ◐ deferred → AZ-505 (perf NFR) |
| AZ-503 AC-10 | Leaflet hot path is index-only (EXPLAIN: no heap fetch when `voting_status='trusted'`) | — | ◐ deferred → AZ-505 |
| AZ-503 AC-12 | HTTP/2 multiplexed responses for `/tiles/{z}/{x}/{y}` | — | ◐ deferred → AZ-505 |
| AZ-504 AC-1 | PT-08 completes on zero-rejected response (no script exit under `set -e -o pipefail`) | Standalone shell harness (4-case) executed in batch_01_cycle5_report.md — accepted/rejected counters wrapped in `{ grep -o … \|\| true; }` at `scripts/run-performance-tests.sh:416-417`; structural: `rg "grep -o .* \\\| wc -l" scripts/run-performance-tests.sh` returns 0 unguarded sites | ✓ |
| AZ-504 AC-2 | PT-08 completes on zero-accepted response (defensive) | Same standalone shell harness (case 4) — `accepted=0, rejected=N` path no longer kills the script | ✓ |
| AZ-504 AC-3 | PT-08 summary line prints in full default-parameter perf run | Verified at autodev Step 15 (Performance Test) by running `scripts/run-performance-tests.sh` with `PERF_REPEAT_COUNT=20 PERF_UAV_BATCH_SIZE=10`; pass criterion is the `PT-08 UAV batch upload: PASS p95=Xms / 2000ms (...)` line in the run output | ◐ gate at Step 15 |
| AZ-504 AC-4 | Leftover `_docs/_process_leftovers/2026-05-12_perf-cycle3-harness-execution.md` deleted on green full run | Verified at autodev Step 15 by `test -f _docs/_process_leftovers/2026-05-12_perf-cycle3-harness-execution.md` returning non-zero after the green run + commit | ◐ gate at Step 15 |
## Restrictions → Test Mapping
@@ -113,6 +129,11 @@
| AZ-488 Reliability — File-first then DB row; per-item failures never fail the batch envelope (except 400/401/403) | AZ-488 task spec § Non-Functional Requirements | BT-14 (mixed-batch shows per-item isolation); `UavTileUploadHandlerTests.*PersistAsync*` (unit); reject reason `STORAGE_FAILURE` defined in contract for the orphan-row recovery path | ✓ |
| AZ-488 Compatibility — Replaces 501 stub; coexists with AZ-484 `tile-storage` v1.0.0 contract on the write side | AZ-488 task spec § Non-Functional Requirements + Contract | `StubAndErrorContractTests` updated to drop the stub-501 expectation; BT-15 + BT-16 validate the AZ-484 invariants under live UAV writes | ✓ |
| AZ-488 Security — Reject details never leak server internals; integer-only file-path construction | AZ-488 task spec § Non-Functional Requirements + Risk 2 | SEC-11 (blackbox); `UavTileFilePathTests` (unit) | ✓ |
| AZ-503 Cross-repo contract — UUIDv5 namespace pinned to `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c`; C# and Python (`gps-denied-onboard`) MUST produce byte-identical output | AZ-503 task spec § Constraints | `Uuidv5Tests.Create_MatchesPythonReferenceVectors_AC1` covers the C# side; cross-repo (Python) side is enforced by `gps-denied-onboard` `AZ-304` and is out of this workspace's automated suite. The constant value is asserted structurally by `Uuidv5Tests` referencing `Uuidv5.TileNamespace`. | ✓ (C# side) / ◐ (Python parity verified by reference vectors; sibling workspace owns the runtime check) |
| AZ-503 Compatibility — No column renames; `tile_zoom`/`tile_x`/`tile_y`/`latitude`/`longitude` preserved; legacy `id` retained in `legacy_id` for one cycle | AZ-503 task spec § Constraints + Risk 1 | `MigrationTests.Az503ColumnsExistAndLocationHashIsNotNull` (asserts existing column names unchanged); migration 014 SQL inspection (no DROP / RENAME COLUMN on existing columns); structural: `legacy_id` populated from `id` for all pre-existing rows | ✓ |
| AZ-503 Migration constraint — Additive non-blocking `ALTER TABLE ADD COLUMN`; backfill in-migration; `NOT NULL` set only on `location_hash` (legacy `content_sha256` left NULLable — see batch_02_cycle5_report.md "Low" finding) | AZ-503 task spec § Constraints + Risk 3 | Migration script inspection (`014_AddTileIdentityColumns.sql`); `MigrationTests.Az503ColumnsExistAndLocationHashIsNotNull` asserts `location_hash NOT NULL`; application-layer NOT NULL invariant for new `content_sha256` writes enforced in `TileService.BuildTileEntity` + `UavTileUploadHandler.PersistAsync` | ✓ |
| AZ-503 Selection-rule preservation — AZ-484 read-side tie-break (`captured_at DESC, updated_at DESC, id DESC`) unchanged | AZ-503 task spec § Constraints | `TileRepository.GetByTileCoordinatesAsync` unchanged on the read path; AZ-484 AC-2 row above (`MostRecentAcrossSourcesSelection_AZ484_AC2`) still passes at Step 11 | ✓ |
| AZ-504 Compatibility — Fix preserves `set -e -o pipefail` globally; only the empty-grep-match case is tolerated locally; no silent error swallowing | AZ-504 task spec § Non-Functional Requirements + Constraints | Standalone shell harness (batch_01_cycle5_report.md) — case "non-zero exit code other than 1 from grep" still propagates failure; structural: `scripts/run-performance-tests.sh:13` retains `set -euo pipefail`; only the two grep counters at lines 416-417 are wrapped in `{ grep -o … \|\| true; }` (not blanket `\|\| true` over the whole assignment) | ✓ |
## Coverage Summary
@@ -127,7 +148,18 @@
| Cycle 1 — AZ-484 (integration + unit) | 6 | 7/7 | — |
| Cycle 2 — AZ-487 (integration + unit + behavioral) | 4 integration + 3 unit + 1 behavioral | 8/8 | — |
| Cycle 2 — AZ-488 (integration + unit + blackbox) | 7 integration + 14 unit + 6 blackbox | 10/10 | — |
| **Total** | **78** | **47/47 (100%)** | **8/8 (100%)** |
| Cycle 5 — AZ-503 foundation (integration + unit + blackbox) | 2 integration + 6 unit + 4 blackbox | 7/12 in-scope (AC-1, 2, 3, 4, 7, 8, 11); 5 ACs deferred → AZ-505 | — |
| Cycle 5 — AZ-504 perf-script fix (shell harness + Step-15 gate) | 1 standalone shell harness (4 cases) | 2/4 verified now (AC-1, AC-2); 2/4 gated at Step 15 (AC-3, AC-4) | — |
| **Total** | **90** | **56/56 in-scope (100%); 5 explicitly deferred to AZ-505 next cycle; 2 AZ-504 ACs gated at Step 15** | **8/8 (100%)** |
**Coverage shape notes (Cycle 5 — AZ-503 foundation):**
- AZ-503 was split mid-cycle (Option C, autodev Step 10 batch 2): 7 of 12 original ACs land here; 5 (AC-5, AC-6, AC-9, AC-10, AC-12) are deferred to AZ-505 with a `Blocks` link in Jira and an entry in `_docs/02_tasks/_dependencies_table.md`. The deferred rows above are marked `◐ deferred → AZ-505` so the matrix surfaces the scope boundary explicitly.
- AZ-503 introduces no new HTTP route or wire-protocol change beyond the optional `metadata.flightId` field on the existing `POST /api/satellite/upload`. The 4 new BT scenarios (BT-19..BT-22) describe the new observable behaviors (multi-flight coexistence, deterministic id+content_sha256 on re-upload, float-rounded collapse, migration shape) without inventing a new endpoint surface.
- The cross-repo UUIDv5 namespace constant (`5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c`) is pinned in `SatelliteProvider.Common/Utils/Uuidv5.cs` and verified C#-side by 10 Python-generated reference vectors in `Uuidv5Tests`. The Python side runtime check is owned by sibling workspace `gps-denied-onboard` (AZ-304) — this matrix records the C# side as `✓` and the Python side as `◐` (out-of-workspace).
**Coverage shape notes (Cycle 5 — AZ-504 perf-script fix):**
- AZ-504 is a one-line harness bug fix; ACs are verified by a standalone shell harness (4 cases under `set -e -o pipefail`) embedded in `batch_01_cycle5_report.md`, not by the normal unit / integration suite. There is no production code path to add a test against — the bug is entirely in `scripts/run-performance-tests.sh`.
- AC-3 (PT-08 prints summary in full default-parameter run) and AC-4 (cycle-3 perf-harness leftover deleted on green full run) are gated at autodev Step 15 (Performance Test). The matrix marks them `◐ gate at Step 15` rather than `✓` until that step runs.
**Coverage shape notes (Cycle 2):**
- AZ-487 AC-7 (Swagger UI Authorize) is verified programmatically (`SwaggerDocument_AdvertisesBearerSecurityScheme`) rather than via a real UI flow; marked `◐ doc-verified`. The end-to-end browser-UI Authorize-button check remains a manual smoke before deploy.