mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-22 05:41:15 +00:00
[AZ-505] Tile inventory endpoint + HTTP/2 + Leaflet covering index
Production code:
- POST /api/satellite/tiles/inventory (XOR body, 5000-cap,
most-recent-per-location_hash select, present/absent shaping).
- Kestrel HttpProtocols.Http1AndHttp2 on every listener (AC-5).
- Migration 015 creates tiles_leaflet_path covering index over
(location_hash, captured_at DESC, updated_at DESC, id DESC)
INCLUDE (file_path, source); drops superseded idx_tiles_location_hash.
- TileRepository.GetByTileCoordinatesAsync rewired to filter by
location_hash (Index Only Scan via tiles_leaflet_path).
- TileRepository.GetTilesByLocationHashesAsync added with Npgsql-
direct ANY($1::uuid[]) binding (Dapper IEnumerable expansion is
incompatible with the array form).
- Uuidv5.LocationHashForTile centralises the UUIDv5(TileNamespace,
"{z}/{x}/{y}") formula — single source of truth for the cross-repo
invariant (gps-denied-onboard parity).
Contracts:
- New: contracts/api/tile-inventory.md v1.0.0.
- Bumped: contracts/data-access/tile-storage.md to v2.0.0 (joint
ownership by AZ-503-foundation + AZ-505: schema + covering index +
GetByTileCoordinatesAsync rewrite).
Tests:
- TileInventoryTests covers AC-1, AC-2 (DB-level), AC-4, AC-6.
- Http2MultiplexingTests covers AC-5 (20 concurrent multiplexed GETs
over h2c via SocketsHttpHandler + AppContext Http2Unencrypted switch).
- LeafletPathIndexOnlyTests covers AC-3 (EXPLAIN (ANALYZE, BUFFERS)
asserts Index Only Scan over tiles_leaflet_path with heap_blocks=0).
Docs:
- architecture.md, system-flows.md, data_model.md, module-layout.md,
glossary.md, modules/api_program.md, modules/dataaccess_tile_repository.md,
components/02_data_access/description.md all updated to reference the
v2.0.0 tile-storage contract + new tile-inventory contract + AC-7.
Reports:
- batch_01_cycle6_report.md, batch_01_cycle6_review.md,
implementation_completeness_cycle6_report.md (PASS),
implementation_report_tile_inventory_cycle6.md.
Task spec moved todo/ -> done/.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -39,7 +39,7 @@ The three Layer-3 service components are compile-time siblings: each only refere
|
||||
- *Google Maps* — `TileService.DownloadAndStoreTilesAsync` / `DownloadAndStoreSingleTileAsync` stamp `source='google_maps'`, `flight_id=NULL`, and a deterministic UUIDv5 `id` on every persisted row; tile JPEGs live under `{StorageConfig.TilesDirectory}/{zoom}/...` per the legacy grandfathered layout. `content_sha256` is computed from the on-disk JPEG body.
|
||||
- *UAV* — `POST /api/satellite/upload` (AZ-488; per-flight key extended by AZ-503) accepts a multipart batch of UAV-captured tiles, runs each item through a 5-rule quality gate (`UavTileQualityGate`), and persists accepted items via `ITileRepository.InsertAsync` with `source='uav'`, `flight_id = metadata.flightId` (or NULL for anonymous uploads), and a deterministic UUIDv5 `id`. UAV JPEGs live under `{StorageConfig.TilesDirectory}/uav/{flight_id or 'none'}/{zoom}/{x}/{y}.jpg`, so `rm -rf ./tiles/uav/{flight_id}/` removes one flight's evidence without touching other flights at overlapping cells. Requires the `GPS` permission claim on top of the JWT baseline.
|
||||
|
||||
The N-source storage contract is authoritative in `_docs/02_document/contracts/data-access/tile-storage.md` (v1.0.0; v2.0.0 bump tracking the AZ-503 identity columns is deferred to AZ-505 when the new identity surface freezes for external consumers). The UAV upload contract is authoritative in `_docs/02_document/contracts/api/uav-tile-upload.md` (v1.0.0; AZ-503 added an optional `flightId` field to per-item metadata — backward-compatible). Anything that reads or writes `tiles` MUST follow those contracts rather than re-deriving the rules from prose here.
|
||||
The N-source storage contract is authoritative in `_docs/02_document/contracts/data-access/tile-storage.md` (v2.0.0 — bumped jointly by AZ-503-foundation and AZ-505 in cycle 6 to capture the identity columns, `tiles_leaflet_path` covering index, and `location_hash`-keyed leaflet read rule). The UAV upload contract is authoritative in `_docs/02_document/contracts/api/uav-tile-upload.md` (v1.1.0; AZ-503 added an optional `flightId` field to per-item metadata — backward-compatible). The bulk tile-inventory contract is authoritative in `_docs/02_document/contracts/api/tile-inventory.md` (v1.0.0; AZ-505). Anything that reads or writes `tiles` MUST follow those contracts rather than re-deriving the rules from prose here.
|
||||
|
||||
**Drift signals**:
|
||||
- `geofence_polygons` mentioned in AGENTS.md as a routes table column but does not exist in schema or entity — documentation drift
|
||||
|
||||
@@ -16,14 +16,17 @@
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
| `GetByIdAsync` | Guid | `TileEntity?` | Yes | NpgsqlException |
|
||||
| `GetByTileCoordinatesAsync` | zoom, x, y | `TileEntity?` (most-recent across sources, AZ-484) | Yes | NpgsqlException |
|
||||
| `GetByTileCoordinatesAsync` | zoom, x, y | `TileEntity?` (most-recent across sources/flights, AZ-505 rewired to filter on `location_hash` for `Index Only Scan` against `tiles_leaflet_path`; selection rule unchanged from AZ-484) | Yes | NpgsqlException |
|
||||
| `GetTilesByRegionAsync` | lat, lon, sizeM, zoom | `IEnumerable<TileEntity>` (one row per cell via `DISTINCT ON`, AZ-484) | Yes | NpgsqlException |
|
||||
| `InsertAsync` | `TileEntity` | Guid (per-source UPSERT, AZ-484) | Yes | NpgsqlException |
|
||||
| `GetTilesByLocationHashesAsync` | `IReadOnlyList<Guid>` location hashes | `IReadOnlyDictionary<Guid, TileEntity>` (one row per requested hash via `DISTINCT ON (location_hash)`, AZ-505) | Yes | NpgsqlException |
|
||||
| `InsertAsync` | `TileEntity` | Guid (integer-only flight-aware UPSERT, AZ-503-foundation; supersedes the AZ-484 5-column float-based UPSERT) | Yes | NpgsqlException |
|
||||
| `UpdateAsync` | `TileEntity` | int | Yes | NpgsqlException |
|
||||
| `DeleteAsync` | Guid | int | Yes | NpgsqlException |
|
||||
|
||||
`FindExistingTileAsync` was removed by AZ-376 (replaced by direct cell lookups through `GetByTileCoordinatesAsync` + `GetTilesByRegionAsync`).
|
||||
|
||||
`GetTilesByLocationHashesAsync` is intentionally NOT routed through Dapper. Npgsql binds `uuid[]` parameters to `ANY($1::uuid[])` queries as a single array column, while Dapper's parameter expander rewrites any `IEnumerable` parameter to a comma-separated list of scalar placeholders, producing `ANY((@p0, @p1, ...))` — which is invalid SQL. The method uses `NpgsqlCommand` with an explicit `NpgsqlParameter` typed `Array | Uuid`, and maps results manually from `NpgsqlDataReader`. This is the documented escape hatch for array-binding hot paths.
|
||||
|
||||
### Interface: IRegionRepository
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
@@ -57,9 +60,10 @@
|
||||
### Queries
|
||||
| Query | Frequency | Hot Path | Index Needed |
|
||||
|-------|-----------|----------|--------------|
|
||||
| GetByTileCoordinatesAsync (tile lookup) | Very High | Yes | `(tile_zoom, tile_x, tile_y)` |
|
||||
| GetByTileCoordinatesAsync (tile lookup, leaflet hot path) | Very High | Yes | `tiles_leaflet_path` covering index — `(location_hash, captured_at DESC, updated_at DESC, id DESC) INCLUDE (file_path, source)` (AZ-505). Target plan: `Index Only Scan` with `Heap Fetches = 0` after `VACUUM ANALYZE`. |
|
||||
| GetTilesByLocationHashesAsync (bulk inventory, AZ-505) | High | Yes | `tiles_leaflet_path` leading column. Inventory returns more columns than the INCLUDE list, so a bounded heap fetch is expected; AC-4 budget (≤ 1000 ms p95 / 2500 tiles) absorbs it. |
|
||||
| GetTilesByRegionAsync (spatial) | High | Yes | `(latitude, longitude, tile_zoom)` |
|
||||
| InsertAsync (tile per-source upsert) | High | Yes | Composite unique on `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid))` (AZ-503: `idx_tiles_unique_identity`; supersedes the AZ-484 float-based `idx_tiles_unique_location_source`) |
|
||||
| InsertAsync (tile per-source-per-flight upsert) | High | Yes | Composite unique on `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid))` (AZ-503-foundation: `idx_tiles_unique_identity`; supersedes the AZ-484 float-based `idx_tiles_unique_location_source`) |
|
||||
| GetByStatusAsync (region polling) | Medium | No | `(status)` |
|
||||
| GetRoutesWithPendingMapsAsync | Low | No | `(request_maps, maps_ready)` |
|
||||
|
||||
@@ -92,7 +96,7 @@
|
||||
- TileRepository.InsertAsync uses an integer-only, flight-aware UPSERT pattern (AZ-503; supersedes the AZ-484 5-column float-based UPSERT). Same-source same-flight re-inserts overwrite and refresh `captured_at`/`location_hash`/`content_sha256`; different sources or different flights at the same cell coexist as separate rows. `id` is intentionally NOT overwritten on conflict so it stays deterministic per AZ-503 AC-2.
|
||||
- `TileEntity.Source` is stored as a plain `string` (not the `TileSource` enum) due to Dapper issue #259 — see `_docs/LESSONS.md` L-001. Conversion happens via `SatelliteProvider.Common.Enums.TileSourceConverter`
|
||||
- AZ-503 deterministic identity: `id` is `Uuidv5(TileNamespace, "{z}/{x}/{y}/{source}/{flight_id or zero-uuid}")` and `location_hash` is `Uuidv5(TileNamespace, "{z}/{x}/{y}")`. The cross-repo `TileNamespace` constant lives in `SatelliteProvider.Common.Utils.Uuidv5` and MUST match `gps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE`.
|
||||
- The frozen v1.0.0 `tile-storage` contract (`_docs/02_document/contracts/data-access/`) is the AZ-484-era spec for read-side selection invariants; the AZ-503 write-side schema change is documented inline in `dataaccess_models.md` and `dataaccess_tile_repository.md`. A v2.0.0 contract bump is deferred to AZ-505 (when the `POST /api/satellite/tiles/inventory` endpoint freezes the new identity surface for external consumers).
|
||||
- The `tile-storage` contract (`_docs/02_document/contracts/data-access/tile-storage.md`) was bumped to v2.0.0 jointly by AZ-503-foundation (identity columns + integer UPSERT, cycle 5) and AZ-505 (covering index `tiles_leaflet_path` + `location_hash`-keyed reads + bulk inventory, cycle 6). The frozen v2.0.0 spec is the authoritative read-side / write-side / index contract for external consumers; the per-method shape table above mirrors it for in-component readers.
|
||||
- No soft-delete; `DeleteAsync` is a hard delete
|
||||
|
||||
## 8. Dependency Graph
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
# Contract: tile-inventory
|
||||
|
||||
**Component**: WebApi (`SatelliteProvider.Api`) producing rows via TileDownloader (`SatelliteProvider.Services.TileDownloader`)
|
||||
**Producer task**: AZ-505 — `_docs/02_tasks/todo/AZ-505_tile_inventory_http2_leaflet_index.md`
|
||||
**Consumer tasks**: `gps-denied-onboard` AZ-316 (`c11_tile_downloader`), future mission-planner UI cache-sizing flows
|
||||
**Version**: 1.0.0
|
||||
**Status**: frozen
|
||||
**Last Updated**: 2026-05-12
|
||||
|
||||
## Purpose
|
||||
|
||||
Defines the HTTP contract for the `POST /api/satellite/tiles/inventory` bulk-lookup endpoint. Callers submit either a list of slippy-map coords or a list of pre-computed `location_hash` UUIDs and receive one response entry per input — in the same order — telling them whether each tile is already cached server-side and (if so) which row to expect on a subsequent `GET /tiles/{z}/{x}/{y}`.
|
||||
|
||||
The endpoint is the consumer-facing payload that justifies the AZ-503-foundation schema work (the deterministic `location_hash` column) plus the AZ-505 `tiles_leaflet_path` covering index. It is designed for pre-flight cache sizing on the gps-denied-onboard side.
|
||||
|
||||
## Endpoint
|
||||
|
||||
```
|
||||
POST /api/satellite/tiles/inventory
|
||||
Content-Type: application/json
|
||||
Authorization: Bearer <JWT>
|
||||
```
|
||||
|
||||
The request MUST carry a valid JWT (AZ-487). No `permissions` claim is required — inventory is a metadata-only read; the GPS permission gate only applies to UAV writes. Anonymous requests are rejected with HTTP 401.
|
||||
|
||||
## Shape
|
||||
|
||||
### Request body
|
||||
|
||||
Exactly one of `tiles` OR `locationHashes` MUST be populated. Sending both, or neither, is HTTP 400.
|
||||
|
||||
```jsonc
|
||||
// Form A — coord-keyed
|
||||
{
|
||||
"tiles": [
|
||||
{ "tileZoom": 18, "tileX": 154321, "tileY": 95812 },
|
||||
{ "tileZoom": 18, "tileX": 154322, "tileY": 95812 }
|
||||
]
|
||||
}
|
||||
|
||||
// Form B — hash-keyed
|
||||
{
|
||||
"locationHashes": [
|
||||
"ad8c1c4c-2b27-5af4-902f-9c8baeed1e84",
|
||||
"5b8d0c2e-7f1a-5d3b-9c5e-1f3a8e7d2b6c"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Per-field constraints:
|
||||
|
||||
| Field | Type | Required | Description | Constraints |
|
||||
|-------|------|----------|-------------|-------------|
|
||||
| `tiles` | `TileCoord[]` | yes (XOR `locationHashes`) | Slippy-map tile coords | Up to 5000 entries per request. Each entry MUST have all three of `tileZoom`, `tileX`, `tileY`. |
|
||||
| `locationHashes` | `UUID[]` | yes (XOR `tiles`) | Pre-computed UUIDv5 `location_hash` values | Up to 5000 entries per request. Each entry MUST be RFC 4122 UUID. |
|
||||
|
||||
Hard cap: **5000 entries per request** (`SatelliteProvider.Common.DTO.TileInventoryLimits.MaxEntriesPerRequest`). Anything larger → HTTP 400. The cap is 2× the AC-4 perf gate (2500 tiles).
|
||||
|
||||
### `TileCoord` (per entry under `tiles`)
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `tileZoom` | integer | yes | Slippy-map zoom level |
|
||||
| `tileX` | integer | yes | Slippy-map tile column |
|
||||
| `tileY` | integer | yes | Slippy-map tile row |
|
||||
|
||||
### Response body
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"tileZoom": 18,
|
||||
"tileX": 154321,
|
||||
"tileY": 95812,
|
||||
"locationHash": "ad8c1c4c-2b27-5af4-902f-9c8baeed1e84",
|
||||
"present": true,
|
||||
"id": "5d83…",
|
||||
"capturedAt": "2026-05-12T13:24:50.123456Z",
|
||||
"source": "uav",
|
||||
"flightId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
|
||||
"resolutionMPerPx": 0.78125
|
||||
},
|
||||
{
|
||||
"tileZoom": 18,
|
||||
"tileX": 154322,
|
||||
"tileY": 95812,
|
||||
"locationHash": "5b8d0c2e-7f1a-5d3b-9c5e-1f3a8e7d2b6c",
|
||||
"present": false,
|
||||
"id": null,
|
||||
"capturedAt": null,
|
||||
"source": null,
|
||||
"flightId": null,
|
||||
"resolutionMPerPx": null
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Per-entry fields:
|
||||
|
||||
| Field | Type | Present when... | Description |
|
||||
|-------|------|-----------------|-------------|
|
||||
| `tileZoom` | integer | always (Form A); zeroed (Form B) | Echoes the request entry's `tileZoom` when input was `tiles`; `0` when input was `locationHashes` (caller already knows the cell). |
|
||||
| `tileX` | integer | always (Form A); zeroed (Form B) | Same as `tileZoom`. |
|
||||
| `tileY` | integer | always (Form A); zeroed (Form B) | Same as `tileZoom`. |
|
||||
| `locationHash` | UUIDv5 | always | `UUIDv5(TileNamespace, "{tileZoom}/{tileX}/{tileY}")`. Populated even when `present=false` so callers can persist the deterministic hash. |
|
||||
| `present` | bool | always | `true` iff a row exists in `tiles` with this `location_hash`. |
|
||||
| `id` | UUID | present=true | Most-recent row's `tiles.id`. Deterministic UUIDv5 for AZ-503+ rows; random for legacy rows. |
|
||||
| `capturedAt` | ISO-8601 UTC | present=true | `tiles.captured_at`. |
|
||||
| `source` | string enum | present=true | `tiles.source` wire value (`"google_maps"` or `"uav"`). |
|
||||
| `flightId` | UUID | present=true (may be null) | `tiles.flight_id`; null for `google_maps` rows and pre-AZ-503 legacy UAV rows. |
|
||||
| `resolutionMPerPx` | number | present=true | `tile_size_meters / tile_size_pixels`. Derived; not a stored column. |
|
||||
|
||||
Order invariant: `results[i]` corresponds to `request.tiles[i]` (or `request.locationHashes[i]`). Always. Even when entries are absent. Even when the request contains duplicates (each duplicate yields its own response entry).
|
||||
|
||||
### Endpoint summary
|
||||
|
||||
| Method | Path | Request body | Response | Status codes |
|
||||
|--------|------|--------------|----------|--------------|
|
||||
| `POST` | `/api/satellite/tiles/inventory` | `TileInventoryRequest` | `TileInventoryResponse` | 200, 400, 401 |
|
||||
|
||||
## Invariants
|
||||
|
||||
- **Inv-1**: Exactly one of `request.tiles` and `request.locationHashes` is populated and non-empty. Both-populated → 400; both-empty → 400.
|
||||
- **Inv-2**: `len(response.results) == len(request.tiles)` OR `len(request.locationHashes)` — never less, never more.
|
||||
- **Inv-3**: `response.results[i].locationHash` is deterministic from `request.tiles[i]` (UUIDv5 over `"{zoom}/{x}/{y}"` with `Uuidv5.TileNamespace`) when Form A is used, or equals `request.locationHashes[i]` when Form B is used.
|
||||
- **Inv-4**: `response.results[i].present == true` iff a row exists in `tiles` with `location_hash = response.results[i].locationHash`.
|
||||
- **Inv-5**: When `present=true`, the returned row is the most-recent across sources/flights ordered by `(captured_at DESC, updated_at DESC, id DESC)` — same rule as `ITileRepository.GetByTileCoordinatesAsync` per `tile-storage` v2.0.0.
|
||||
- **Inv-6**: When `present=false`, `id` / `capturedAt` / `source` / `flightId` / `resolutionMPerPx` are all `null`.
|
||||
- **Inv-7**: `request.tiles.length` and `request.locationHashes.length` MUST be ≤ `TileInventoryLimits.MaxEntriesPerRequest` (5000); over the cap → 400.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- **Not covered**: byte-size hints (`estimatedBytes`). Deferred until production profiling justifies the per-file `stat()` cost.
|
||||
- **Not covered**: voting / trust-promotion filtering. The `voting_status` filter is part of the future voting layer (`gps-denied-onboard` Design Task #2); inventory always returns the most-recent row regardless of any future trust state.
|
||||
- **Not covered**: tile body download. This endpoint returns metadata only; callers fetch bodies via `GET /tiles/{z}/{x}/{y}`.
|
||||
- **Not covered**: HTTP/3 / QUIC. Kestrel is set to `Http1AndHttp2`; the HTTP/3 plumbing requires ALPN + UDP verification that's deferred per AZ-505 scope.
|
||||
- **Not covered**: browser-side multiplexing. h2c (HTTP/2 over plaintext) is not supported by mainstream browsers; only programmatic clients (httpx http2=True, .NET HttpClient with `HttpVersionPolicy.RequestVersionExact`) realize the HTTP/2 multiplexing benefit. Browser Leaflet wins come from the covering-index hot path, not multiplexing.
|
||||
- **Not covered**: PMTiles or tar/multipart bundle endpoints. Rejected by AZ-503 parent rationale (HTTP/2 multistream is sufficient).
|
||||
- **Not covered**: write operations. Inventory is read-only; UAV writes go through `POST /api/satellite/upload` (`uav-tile-upload.md` v1.1.0).
|
||||
|
||||
## Versioning Rules
|
||||
|
||||
- **Patch (1.0.x)**: Documentation clarifications, additional invariants that do not change wire behavior.
|
||||
- **Minor (1.x.0)**: Adding an optional response field that consumers may safely ignore (e.g., the future `estimatedBytes`); raising the entry cap; adding a third request form alongside the current two.
|
||||
- **Major (2.0.0)**: Changing the response ordering rule; removing `present`; lowering the entry cap; making `flightId` required; adding voting / trust filtering to the read path.
|
||||
|
||||
## Test Cases
|
||||
|
||||
| Case | Input | Expected | Notes |
|
||||
|------|-------|----------|-------|
|
||||
| ordering-mixed-present-absent | 25 coords, 12 seeded + 13 absent, interleaved | 25 entries in request order; 12 present (id/capturedAt/source populated), 13 absent (only locationHash populated) | AC-1 |
|
||||
| most-recent-across-sources | Cell with `google_maps captured_at=T1` and `uav captured_at=T2 > T1`; coord request | `present=true`, `source='uav'`, `id` = UAV row's id | Inv-5 |
|
||||
| validation-both-populated | Body with both `tiles` and `locationHashes` | HTTP 400 | Inv-1 |
|
||||
| validation-neither-populated | Empty body or body with both fields empty | HTTP 400 | Inv-1 |
|
||||
| validation-over-cap | 5001 entries | HTTP 400 | Inv-7 |
|
||||
| auth-anonymous | No Bearer token | HTTP 401 | Standard `.RequireAuthorization()` baseline |
|
||||
| perf-2500-tiles | 2500-entry request against populated DB | p95 ≤ 1000 ms over 20 calls | AC-4 |
|
||||
| http2-multiplexing | 20 concurrent `GET /tiles/{z}/{x}/{y}` over a single H2 connection | All 20 responses `HttpResponseMessage.Version == 2.0`; ETag + Cache-Control preserved | AC-5; cross-references `tile-inventory.md` because Kestrel H2 is configured in the same PBI |
|
||||
|
||||
## Change Log
|
||||
|
||||
| Version | Date | Change | Author |
|
||||
|---------|------|--------|--------|
|
||||
| 1.0.0 | 2026-05-12 | Initial contract — `POST /api/satellite/tiles/inventory` with Form A (coords) / Form B (hashes) XOR validation, 5000-entry cap, most-recent-across-sources selection rule, ordering invariant. Produced by AZ-505. | autodev (Step 10, cycle 6) |
|
||||
@@ -1,22 +1,29 @@
|
||||
# Contract: tile-storage
|
||||
|
||||
**Component**: DataAccess
|
||||
**Producer task**: AZ-484 — `_docs/02_tasks/todo/AZ-484_multi_source_tile_storage.md`
|
||||
**Consumer tasks**: AZ-485 (planned T2 — UAV upload endpoint); future tasks adding additional sources (e.g., SatAR)
|
||||
**Version**: 1.0.0
|
||||
**Producer task**: AZ-484 (v1.0.0 — multi-source schema) + AZ-503-foundation (v2.0.0 — tile identity columns) + AZ-505 (v2.0.0 freeze — covering index + location_hash-keyed reads + bulk inventory)
|
||||
**Consumer tasks**: AZ-485 (UAV upload endpoint, cycle 5 — `uav-tile-upload.md` v1.1.0), AZ-505 (inventory endpoint, this cycle — `tile-inventory.md` v1.0.0), future SatAR / additional-source tasks
|
||||
**Version**: 2.0.0
|
||||
**Status**: frozen
|
||||
**Last Updated**: 2026-05-11
|
||||
**Last Updated**: 2026-05-12
|
||||
|
||||
## Purpose
|
||||
|
||||
Defines how satellite imagery tiles are persisted in the `tiles` table when more than one acquisition source can write to the same geographic cell. Producers must agree on the source enum, the captured-at semantics, and the per-source UPSERT contract. Readers must use the documented selection rule and tolerate the multi-source row layout.
|
||||
Defines how satellite imagery tiles are persisted in the `tiles` table when more than one acquisition source (and multiple UAV flights per source) can write to the same geographic cell, AND how the table is indexed for the two distinct read patterns it serves:
|
||||
|
||||
1. **Producer writes** (`POST /api/satellite/upload`, `GoogleMapsDownloaderV2`) — per-source, per-flight UPSERTs keyed by integer slippy coords.
|
||||
2. **Consumer reads**:
|
||||
- Leaflet hot path — `GET /tiles/{z}/{x}/{y}` returns the most-recent variant by `location_hash`.
|
||||
- Bulk inventory — `POST /api/satellite/tiles/inventory` returns one row per `location_hash` across many cells in one round trip.
|
||||
|
||||
Producers must agree on the source enum, `captured_at` semantics, `flight_id` semantics, and the per-(source × flight) UPSERT contract. Readers must use the `location_hash`-keyed selection rule and tolerate the multi-source / multi-flight row layout.
|
||||
|
||||
## Shape
|
||||
|
||||
### Schema (PostgreSQL `tiles` table — relevant columns only)
|
||||
|
||||
```sql
|
||||
-- Pre-existing columns (unchanged)
|
||||
-- Pre-existing columns (unchanged since AZ-484)
|
||||
id UUID PRIMARY KEY
|
||||
tile_zoom INT NOT NULL
|
||||
tile_x INT NOT NULL
|
||||
@@ -30,37 +37,60 @@ file_path VARCHAR(500) NOT NULL
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
|
||||
-- New in v1.0.0 (this contract)
|
||||
-- AZ-484 (v1.0.0)
|
||||
source VARCHAR(32) NOT NULL -- enum-stored: 'google_maps' | 'uav'
|
||||
captured_at TIMESTAMP NOT NULL -- UTC; producer-supplied semantics, see below
|
||||
|
||||
-- AZ-503-foundation (this contract — v2.0.0)
|
||||
flight_id UUID NULL -- per-UAV-flight identifier; NULL for google_maps + legacy uav
|
||||
location_hash UUID NOT NULL -- UUIDv5(TileNamespace, "{tile_zoom}/{tile_x}/{tile_y}")
|
||||
content_sha256 BYTEA NULL -- SHA-256 of JPEG body at insert time; NULL for legacy rows only
|
||||
legacy_id UUID NULL -- pre-AZ-503 random `id` preserved for one deprecation cycle
|
||||
|
||||
-- Vestigial columns (preserved per coderule.mdc; readers MUST NOT depend on them)
|
||||
maps_version VARCHAR(50) NULL
|
||||
version INT NULL
|
||||
```
|
||||
|
||||
### Field reference
|
||||
### Field reference (v2.0.0)
|
||||
|
||||
| Field | Type | Required | Description | Constraints |
|
||||
|-------|------|----------|-------------|-------------|
|
||||
| `source` | enum (`TileSource`) stored as `VARCHAR(32)` | yes | Producer of the tile | `'google_maps'` or `'uav'`. New values require a contract version bump. |
|
||||
| `captured_at` | `TIMESTAMP` UTC | yes | Producer-defined "moment the imagery represents" | For `google_maps`: `DateTime.UtcNow` at download time (provider does not expose original imagery date). For `uav`: the UAV capture timestamp supplied by the upload client. Must be UTC; non-UTC must be converted before write. |
|
||||
| `(latitude, longitude, tile_zoom, tile_size_meters, source)` | composite | yes | Per-source uniqueness | Enforced via `UNIQUE INDEX idx_tiles_unique_location_source`. |
|
||||
| `flight_id` | `UUID` | no (NULL for `google_maps` + legacy `uav`) | Per-flight identifier supplied by the UAV upload endpoint | When source = `'uav'` AND tile is AZ-503+ era → NOT NULL. When source = `'google_maps'` → MUST be NULL. Pre-AZ-503 `uav` rows may have NULL. UPSERT collapses NULL via `COALESCE(flight_id, '00000000-…'::uuid)`. |
|
||||
| `location_hash` | `UUID` (v5) | yes | Deterministic cell identifier | `UUIDv5(Uuidv5.TileNamespace, "{tile_zoom}/{tile_x}/{tile_y}")`. Cross-repo invariant — `TileNamespace = 5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c`. Identical byte-for-byte with `gps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE`. |
|
||||
| `content_sha256` | `BYTEA` (32) | yes for AZ-503+ writes (NULL only for pre-AZ-503 rows the migration could not re-hash) | SHA-256 of the JPEG body at insert time | Application invariant: enforced NOT NULL on new writes via `TileEntity.ContentSha256`. Migration 014 left the column nullable because it could not safely re-open tile files on disk during schema migration. |
|
||||
| `legacy_id` | `UUID` | no | Pre-AZ-503 random `id` preserved for one deprecation cycle | NULL for AZ-503+ rows; populated for rows that pre-date migration 014. Will be dropped in a follow-up migration once external references are confirmed flushed. |
|
||||
| `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-…'::uuid))` | composite | yes | Per-source-per-flight uniqueness | Enforced via `UNIQUE INDEX idx_tiles_unique_identity`. Replaces the AZ-484 lat/lon-keyed uniqueness from `idx_tiles_unique_location_source`. |
|
||||
|
||||
### Index
|
||||
### Indexes (v2.0.0)
|
||||
|
||||
```sql
|
||||
CREATE UNIQUE INDEX idx_tiles_unique_location_source
|
||||
ON tiles (latitude, longitude, tile_zoom, tile_size_meters, source);
|
||||
-- Per-source-per-flight uniqueness. NULL-safe via COALESCE.
|
||||
CREATE UNIQUE INDEX idx_tiles_unique_identity
|
||||
ON tiles (
|
||||
tile_zoom, tile_x, tile_y, tile_size_meters, source,
|
||||
COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)
|
||||
);
|
||||
|
||||
-- Leaflet hot-path covering index. AZ-505.
|
||||
CREATE INDEX tiles_leaflet_path
|
||||
ON tiles (location_hash, captured_at DESC, updated_at DESC, id DESC)
|
||||
INCLUDE (file_path, source);
|
||||
```
|
||||
|
||||
The previous 4-column unique index `(latitude, longitude, tile_zoom, tile_size_meters)` from migration 012 is dropped.
|
||||
Indexes dropped in v2.0.0:
|
||||
|
||||
- `idx_tiles_unique_location_source` (AZ-484 lat/lon-keyed uniqueness) — dropped by migration 014; superseded by `idx_tiles_unique_identity`.
|
||||
- `idx_tiles_unique_location` (pre-AZ-484 4-column uniqueness) — dropped by migration 013; included here for completeness.
|
||||
- `idx_tiles_location_hash` (lightweight lookup added by migration 014) — dropped by migration 015; superseded — equality lookups by `location_hash` use the leading column of `tiles_leaflet_path`.
|
||||
|
||||
### Producer write API
|
||||
|
||||
| Operation | Repository method | Conflict semantics |
|
||||
|-----------|-------------------|--------------------|
|
||||
| Insert / replace same-source row for a cell | `ITileRepository.InsertAsync(TileEntity)` | `ON CONFLICT (latitude, longitude, tile_zoom, tile_size_meters, source) DO UPDATE SET file_path, tile_x, tile_y, captured_at, updated_at`. Producers MUST set `Source` and `CapturedAt`. |
|
||||
| Insert / replace same-(source, flight) row for a cell | `ITileRepository.InsertAsync(TileEntity)` | `ON CONFLICT (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-…'::uuid)) DO UPDATE SET file_path, latitude, longitude, captured_at, updated_at, content_sha256`. Producers MUST set `Source`, `CapturedAt`, `LocationHash`, `ContentSha256`. Producers MUST set `FlightId` when source = `uav`. |
|
||||
| Update by primary key | `ITileRepository.UpdateAsync(TileEntity)` | Updates by `id` only. Caller's responsibility not to violate the unique index. |
|
||||
| Delete by primary key | `ITileRepository.DeleteAsync(Guid)` | Removes a single row by `id`; no cascade. |
|
||||
|
||||
@@ -68,34 +98,39 @@ The previous 4-column unique index `(latitude, longitude, tile_zoom, tile_size_m
|
||||
|
||||
| Operation | Repository method | Selection rule |
|
||||
|-----------|-------------------|----------------|
|
||||
| Read by `id` | `ITileRepository.GetByIdAsync(Guid)` | Returns the row identified by `id` (no source filter). |
|
||||
| Read most-recent for a cell by slippy coordinates | `ITileRepository.GetByTileCoordinatesAsync(zoom, x, y)` | Returns the row with the highest `(captured_at, updated_at, id)` tuple across all sources for that cell. At most one row. |
|
||||
| Read region | `ITileRepository.GetTilesByRegionAsync(lat, lon, sizeMeters, zoomLevel)` | Returns at most one row per `(latitude, longitude, tile_zoom, tile_size_meters)` group, selected by the same most-recent rule. |
|
||||
| Read by `id` | `ITileRepository.GetByIdAsync(Guid)` | Returns the row identified by `id` (no source/flight filter). |
|
||||
| Read most-recent for a cell by slippy coordinates | `ITileRepository.GetByTileCoordinatesAsync(zoom, x, y)` | Computes `location_hash = UUIDv5(TileNamespace, "{zoom}/{x}/{y}")` and returns the row with the highest `(captured_at, updated_at, id)` tuple for that hash across all sources/flights. At most one row. |
|
||||
| Read region | `ITileRepository.GetTilesByRegionAsync(lat, lon, sizeMeters, zoomLevel)` | Returns at most one row per `(tile_zoom, tile_x, tile_y, tile_size_meters)` group, selected by the same most-recent rule. |
|
||||
| Bulk inventory lookup | `ITileRepository.GetTilesByLocationHashesAsync(IReadOnlyList<Guid> locationHashes)` | Returns at most one row per requested `location_hash`, selected by `DISTINCT ON (location_hash) ... ORDER BY location_hash, captured_at DESC, updated_at DESC, id DESC`. Used by the AZ-505 inventory endpoint. |
|
||||
|
||||
The selection rule is **most-recent across all sources** ordered by `captured_at DESC`, with `(updated_at DESC, id DESC)` as deterministic tie-breakers.
|
||||
The selection rule is **most-recent across all sources and flights** ordered by `captured_at DESC`, with `(updated_at DESC, id DESC)` as deterministic tie-breakers. No voting / trust-promotion filter is applied at this layer.
|
||||
|
||||
## Invariants
|
||||
|
||||
- **Inv-1**: Every row has a non-null `source` whose string value is a member of `TileSource`. Rows with unknown source values are a contract violation.
|
||||
- **Inv-2**: Every row has a non-null `captured_at` in UTC.
|
||||
- **Inv-3**: At most one row exists per `(latitude, longitude, tile_zoom, tile_size_meters, source)`.
|
||||
- **Inv-3**: At most one row exists per `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-…'::uuid))`. NULL-coalesced flight_id is what makes `google_maps` rows (where flight_id is always NULL) deduplicate to one row per cell-and-size-and-source.
|
||||
- **Inv-4**: For any cell with one or more rows, the row returned by `GetByTileCoordinatesAsync` and the per-cell row returned by `GetTilesByRegionAsync` are identical.
|
||||
- **Inv-5**: The `source` column value space is closed: only the snake_case wire values defined in `SatelliteProvider.Common.Enums.TileSourceConverter` (`"google_maps"`, `"uav"`) are valid. Adding a new producer requires a new `TileSource` enum member, a corresponding wire value in `TileSourceConverter`, AND a contract version bump (minor). Note: `TileEntity.Source` is stored as the wire string (not the C# enum) because Dapper's `TypeHandler<T>` for enum types is bypassed during read deserialization (Dapper issue #259); `TileSourceConverter.{ToWireValue,FromWireValue}` is the documented bridge.
|
||||
- **Inv-6**: `captured_at` semantics are producer-defined per the Field Reference table above; consumers MUST NOT reinterpret it (e.g., consumers MUST NOT assume `captured_at` from `google_maps` reflects original imagery date).
|
||||
- **Inv-7** (new in v2.0.0): `location_hash` is functionally determined by `(tile_zoom, tile_x, tile_y)` — every row with the same slippy coords has the same hash, and that hash equals `UUIDv5(Uuidv5.TileNamespace, "{tile_zoom}/{tile_x}/{tile_y}")`. The namespace constant is a cross-repository invariant that must NOT be changed unilaterally — see `coderule.mdc` "cross-repo invariants" and the AZ-503 migration header.
|
||||
- **Inv-8** (new in v2.0.0): When `source = 'google_maps'`, `flight_id` MUST be NULL. When `source = 'uav'` AND the row is AZ-503+ era (created after migration 014), `flight_id` SHOULD be non-NULL; legacy `uav` rows with NULL `flight_id` are tolerated for one deprecation cycle.
|
||||
- **Inv-9** (new in v2.0.0): `GetByTileCoordinatesAsync` filters by `location_hash`, not by `(tile_zoom, tile_x, tile_y)` directly. Callers that pass the same `(z, x, y)` tuple get byte-identical results to v1.0.0 because the hash is deterministic; this is a behavior-preserving rewrite that exists to make the leaflet hot path index-only against `tiles_leaflet_path`.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- **Not covered**: Per-source historical revision retention. Same-source uploads to the same cell overwrite the previous row by design — this is not a versioned table. Consumers wanting season selection or rollback must propose a v2 schema.
|
||||
- **Not covered**: Cross-source merging or compositing at read time. Reads return exactly one row per cell.
|
||||
- **Not covered**: Quality scoring, threshold gating, or any policy beyond the selection rule. Quality enforcement happens upstream of the write (T2).
|
||||
- **Not covered**: Backwards-compatible reads against the legacy 4-column unique index. Migration 013 is mandatory before any consumer of v1.0.0 runs.
|
||||
- **Not covered**: Per-source / per-flight historical revision retention. Same-(source, flight) uploads to the same cell overwrite the previous row by design — this is not a versioned table. Consumers wanting season selection or rollback must propose a v3 schema.
|
||||
- **Not covered**: Cross-source / cross-flight merging or compositing at read time. Reads return exactly one row per cell.
|
||||
- **Not covered**: Quality scoring, threshold gating, or voting / trust-promotion at this layer. Voting is owned by `gps-denied-onboard` Design Task #2 and consumes `flight_id` from this contract.
|
||||
- **Not covered**: Backwards-compatible reads against the v1.0.0 unique index. Migration 014 is mandatory before any consumer of v2.0.0 runs.
|
||||
- **Not covered**: The vestigial `maps_version` and `version` columns. Consumers MUST NOT read them; producers MUST NOT write them in v1.0.0+.
|
||||
- **Not covered**: `content_sha256` integrity verification on read. The column is populated for new writes; downstream verification is a future-task concern.
|
||||
|
||||
## Versioning Rules
|
||||
|
||||
- **Patch (1.0.x)**: Documentation clarifications, additional invariants that do not change runtime behavior, expanded test cases.
|
||||
- **Minor (1.x.0)**: Adding a new `TileSource` enum member; adding optional columns that consumers may safely ignore; relaxing constraints in a backward-compatible way.
|
||||
- **Major (2.0.0)**: Removing or renaming a column; changing the unique index columns; changing the selection rule (e.g., adding source priority); changing `captured_at` from required to optional or vice versa; introducing per-source historical revisions.
|
||||
- **Patch (2.0.x)**: Documentation clarifications, additional invariants that do not change runtime behavior, expanded test cases.
|
||||
- **Minor (2.x.0)**: Adding a new `TileSource` enum member; adding optional columns that consumers may safely ignore; adding new repository read methods; widening the `tiles_leaflet_path` INCLUDE list to remove heap fetches from inventory.
|
||||
- **Major (3.0.0)**: Removing or renaming a column; changing the unique index columns; changing the selection rule (e.g., adding source priority or voting filter); changing `captured_at` from required to optional or vice versa; introducing per-(source, flight) historical revisions; changing the `Uuidv5.TileNamespace` constant (would also break sibling repos and require coordinated cross-repo work).
|
||||
|
||||
Each version bump requires updating the Change Log below and notifying every consumer listed in the header. If consumers' tasks have not yet been written, the producer task is responsible for surfacing the change to the user before merging.
|
||||
|
||||
@@ -103,11 +138,16 @@ Each version bump requires updating the Change Log below and notifying every con
|
||||
|
||||
| Case | Input | Expected | Notes |
|
||||
|------|-------|----------|-------|
|
||||
| valid-google-only | Insert `source='google_maps' captured_at=T1` for a fresh cell | Single row returned by region read; `source='google_maps'`, `captured_at=T1`. | Baseline regression case. |
|
||||
| valid-multi-source | Insert `google_maps captured_at=T1`, then `uav captured_at=T2 > T1` for same cell | Both rows persisted; `GetByTileCoordinatesAsync` returns the `uav` row. | AC-1 + AC-2 of producer task. |
|
||||
| same-source-upsert | Insert `uav captured_at=T1`, then `uav captured_at=T2 > T1` for same cell | Exactly one `uav` row remains, with `captured_at=T2` and updated `file_path`. | AC-3 of producer task. |
|
||||
| time-tiebreak | Insert `google_maps captured_at=T`, then `uav captured_at=T` (identical) for same cell | Selection deterministic by `(updated_at DESC, id DESC)` tie-break; result must be reproducible across two test runs with the same seed. | Inv-4 enforcement. |
|
||||
| backfill-completeness | Migration 013 against a snapshot DB with N pre-existing rows | Post-migration row count is N; every row has `source='google_maps'` and `captured_at = created_at`. | AC-4 of producer task. |
|
||||
| valid-google-only | Insert `source='google_maps' captured_at=T1 flight_id=NULL` for a fresh cell | Single row returned by region read; `source='google_maps'`, `captured_at=T1`. | v1.0.0 baseline regression case. |
|
||||
| valid-multi-source | Insert `google_maps captured_at=T1`, then `uav captured_at=T2 > T1 flight_id=F1` for same cell | Both rows persisted; `GetByTileCoordinatesAsync` returns the `uav` row. | AC-1 + AC-2 of AZ-484. |
|
||||
| valid-multi-flight | Insert two `uav` rows with distinct `flight_id`s for same `(z, x, y)`, `captured_at=T1` and `T2 > T1` | Both rows persisted under `idx_tiles_unique_identity`; most-recent rule returns the `T2` row. | v2.0.0 new — was a unique-index violation under v1.0.0. |
|
||||
| same-source-same-flight-upsert | Insert `uav captured_at=T1 flight_id=F1`, then `uav captured_at=T2 > T1 flight_id=F1` for same cell | Exactly one `uav/F1` row remains, with `captured_at=T2` and updated `file_path`. | AZ-484 AC-3, preserved through AZ-503 schema rewrite. |
|
||||
| time-tiebreak | Insert `google_maps captured_at=T`, then `uav captured_at=T flight_id=F1` (identical timestamps) for same cell | Selection deterministic by `(updated_at DESC, id DESC)` tie-break; result must be reproducible across two test runs with the same seed. | Inv-4 enforcement. |
|
||||
| location-hash-stability | Compute UUIDv5 for `(z=18, x=154321, y=95812)` both in C# (`Uuidv5.Create`) and in Postgres (migration 014 helper) | Identical 16 bytes. Both equal `gps-denied-onboard`'s Python `uuid5(TILE_NAMESPACE, "18/154321/95812")`. | Inv-7 cross-repo invariant. |
|
||||
| leaflet-index-only | Seed ≥ 100k rows, `VACUUM ANALYZE tiles`, then `EXPLAIN SELECT file_path FROM tiles WHERE location_hash = $1 ORDER BY captured_at DESC, updated_at DESC, id DESC LIMIT 1` | Plan contains `Index Only Scan using tiles_leaflet_path`; `Heap Fetches` ≤ 1. | AZ-505 AC-3. |
|
||||
| bulk-inventory-ordering | `GetTilesByLocationHashesAsync` with 2500 hashes (mix of present + absent) | Result is one-row-per-distinct-hash, most-recent across (source, flight). Order is hash-keyed; caller re-aligns to request order. | AZ-505 AC-1 / AC-4. |
|
||||
| backfill-completeness | Migration 013 against a snapshot DB with N pre-existing rows | Post-migration row count is N; every row has `source='google_maps'` and `captured_at = created_at`. | AZ-484 AC-4. |
|
||||
| location-hash-backfill | Migration 014 against a snapshot DB after AZ-484 has applied | Every row has non-NULL `location_hash` matching the application-side UUIDv5 for that row's `(tile_zoom, tile_x, tile_y)`. | AZ-503-foundation guarantee. |
|
||||
| invalid-source | Direct SQL insert with `source='satar'` (not in enum) | Repository read either rejects deserialization or raises a contract violation; behavior MUST surface the violation, not swallow it. | Inv-1 + `coderule.mdc` "never suppress errors silently". |
|
||||
|
||||
## Change Log
|
||||
@@ -115,3 +155,4 @@ Each version bump requires updating the Change Log below and notifying every con
|
||||
| Version | Date | Change | Author |
|
||||
|---------|------|--------|--------|
|
||||
| 1.0.0 | 2026-05-11 | Initial contract — multi-source schema (`source`, `captured_at`), 5-column unique key, most-recent-across-sources read rule. Produced by AZ-484. | autodev (Step 9) |
|
||||
| 2.0.0 | 2026-05-12 | **MAJOR**. Identity columns + covering-index freeze. Added columns: `flight_id` (per-UAV-flight, nullable), `location_hash` (UUIDv5, NOT NULL), `content_sha256` (BYTEA, app-NOT-NULL), `legacy_id` (pre-AZ-503 random id preserved one cycle). Replaced AZ-484 `idx_tiles_unique_location_source` (lat/lon-keyed) with `idx_tiles_unique_identity` (integer slippy + per-flight, NULL-coalesced) — migration 014. Added covering index `tiles_leaflet_path (location_hash, captured_at DESC, updated_at DESC, id DESC) INCLUDE (file_path, source)` — migration 015. Rewrote `GetByTileCoordinatesAsync` to filter on `location_hash` (behavior-preserving — same UUIDv5 deterministic on both ends — to enable index-only scan on the leaflet hot path). Added `GetTilesByLocationHashesAsync` for the AZ-505 bulk inventory endpoint. Introduced Inv-7 / Inv-8 / Inv-9. Produced jointly by AZ-503-foundation (cycle 5, columns + identity index) and AZ-505 (cycle 6, covering index + location_hash-keyed reads + bulk inventory). Consumers reviewed at bump time: AZ-485 (`uav-tile-upload.md` v1.1.0) — already aligned in cycle 5; AZ-505 (`tile-inventory.md` v1.0.0) — produced jointly. | autodev (Step 10, cycle 6) |
|
||||
|
||||
@@ -110,7 +110,7 @@ Stores metadata for downloaded satellite imagery tiles. Each tile is a single im
|
||||
| tile_x | INT | NOT NULL | Tile X coordinate (Slippy Map) |
|
||||
| tile_y | INT | NOT NULL | Tile Y coordinate (Slippy Map) |
|
||||
| flight_id | UUID | NULL | AZ-503: optional flight identifier. `NULL` for Google Maps tiles and anonymous UAV uploads; populated from `UavTileMetadata.FlightId` when present. Part of the UPSERT conflict key via `COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)`, so two flights uploading the same `(z, x, y)` cell produce two separate rows. |
|
||||
| location_hash | UUID | NOT NULL | AZ-503: deterministic UUIDv5 of `{tile_zoom}/{tile_x}/{tile_y}` under `Uuidv5.TileNamespace`. Identical across flights and sources for the same cell. Backfilled in migration 014 via a `pg_temp.uuidv5` PL/pgSQL function. Reserved for the AZ-505 Leaflet covering index (`POST /tiles/inventory`). |
|
||||
| location_hash | UUID | NOT NULL | AZ-503: deterministic UUIDv5 of `{tile_zoom}/{tile_x}/{tile_y}` under `Uuidv5.TileNamespace`. Identical across flights and sources for the same cell. Backfilled in migration 014 via a `pg_temp.uuidv5` PL/pgSQL function. AZ-505 made this the keyed read column for `GetByTileCoordinatesAsync` (leaflet hot path) and the bulk lookup column for `GetTilesByLocationHashesAsync` (`POST /api/satellite/tiles/inventory`); covered by the `tiles_leaflet_path` index. |
|
||||
| content_sha256 | BYTEA | NULL | AZ-503: SHA-256 digest of the JPEG body. Application-layer NOT NULL for new writes (enforced in `TileService.BuildTileEntity` + `UavTileUploadHandler.PersistAsync`); DB column is NULLABLE because legacy pre-migration rows cannot be backfilled reliably from disk. See `batch_02_cycle5_report.md` "Low maintainability finding" for the rationale. |
|
||||
| legacy_id | UUID | NULL | AZ-503: pre-migration `id` value, copied by migration 014 for one-cycle forensics. To be dropped in a future migration once the cross-repo cutover settles. |
|
||||
| created_at | TIMESTAMP | NOT NULL, DEFAULT NOW | |
|
||||
@@ -118,7 +118,7 @@ Stores metadata for downloaded satellite imagery tiles. Each tile is a single im
|
||||
|
||||
**Indexes** (post-AZ-503):
|
||||
- `idx_tiles_unique_identity` UNIQUE (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)) — created by migration 014; replaces the AZ-484 `idx_tiles_unique_location_source` (5-col float-based). Integer-only conflict columns eliminate float-rounding collisions; the `COALESCE` lets per-flight rows coexist while keeping single-row semantics for anonymous and `google_maps` rows.
|
||||
- `idx_tiles_location_hash` (location_hash) — created by migration 014; non-unique. Reserved for the AZ-505 Leaflet covering index when `POST /tiles/inventory` lands.
|
||||
- `tiles_leaflet_path` (location_hash, captured_at DESC, updated_at DESC, id DESC) INCLUDE (file_path, source) — created by AZ-505 migration 015. Drives `GET /tiles/{z}/{x}/{y}` (`Index Only Scan` for the leaflet hot path) and the `POST /api/satellite/tiles/inventory` bulk lookup (leading column matches the `WHERE location_hash = ANY($1::uuid[])` predicate). The lightweight `idx_tiles_location_hash` from migration 014 is dropped by migration 015 — equality lookups by `location_hash` use the leading column of the covering index, making the lookup-only index redundant.
|
||||
- `idx_tiles_coordinates` (tile_zoom, tile_x, tile_y, version)
|
||||
- `idx_tiles_zoom` (tile_zoom)
|
||||
|
||||
@@ -235,3 +235,4 @@ Junction table linking routes to their generated region requests, with geofence
|
||||
| 012 | DropTileVersionConstraint | Drops legacy 5-col `(…, version)` unique index; replaces with 4-col `idx_tiles_unique_location` (preparation for AZ-484) |
|
||||
| 013 | AddTileSourceAndCapturedAt | AZ-484: adds `source` (default `'google_maps'`) + `captured_at` columns; backfills both for pre-existing rows; replaces 4-col unique with 5-col `idx_tiles_unique_location_source`. Transactional; idempotent against partial replays |
|
||||
| 014 | AddTileIdentityColumns | AZ-503: adds `flight_id` (NULL), `location_hash` (NOT NULL after backfill), `content_sha256` (NULL), `legacy_id` (NULL); backfills `location_hash` via `pg_temp.uuidv5(TILE_NAMESPACE, "{tile_zoom}/{tile_x}/{tile_y}")` and copies `id → legacy_id` for every pre-existing row; drops `idx_tiles_unique_location_source` (AZ-484) and creates `idx_tiles_unique_identity` (integer + flight-aware) + `idx_tiles_location_hash`. Enables `pgcrypto` for the in-migration SHA-1 digest. Transactional; safe to replay (column adds are `IF NOT EXISTS`-equivalent, backfill is idempotent on `location_hash` because UUIDv5 is deterministic) |
|
||||
| 015 | AddTilesLeafletPathIndex | AZ-505: creates `tiles_leaflet_path (location_hash, captured_at DESC, updated_at DESC, id DESC) INCLUDE (file_path, source)` covering index for the leaflet hot path; drops the superseded `idx_tiles_location_hash` from migration 014 (equality lookups by `location_hash` now use the leading column of the covering index). Transactional; runs inside DbUp's per-script transaction (incompatible with `CREATE INDEX CONCURRENTLY`) — schedule deploys to a low-traffic window on populated tables. INCLUDE columns intentionally narrow (`file_path, source`); inventory queries that need more columns trigger a bounded heap fetch (per AZ-505 NFR-Perf-2 budget). |
|
||||
|
||||
@@ -13,8 +13,8 @@
|
||||
| Stitch | Compositing multiple tiles into a single larger image with optional markers/borders | modules/services_region_service.md |
|
||||
| Layer 1 | Historic name for satellite imagery from external providers (provider-agnostic; first implementation: Google Maps). Generalised in AZ-484 to one of N values of `Tile Source`; the term is retained for continuity with earlier docs and tickets. | user clarification, AZ-484 |
|
||||
| Layer 2 | Historic name for UAV-captured nadir camera imagery (orthogonal tiles uploaded post-flight). Generalised in AZ-484 to the `uav` `Tile Source` value; the term is retained for continuity with earlier docs and tickets. | user clarification, AZ-484 |
|
||||
| Tile Source | The producer of a tile row, persisted in `tiles.source` as a contract-defined string (`google_maps`, `uav`, …). Each cell may have at most one row per source; reads return the most-recent across sources. Adding a new source requires a new `TileSource` enum member and a tile-storage contract version bump. | _docs/02_document/contracts/data-access/tile-storage.md (v1.0.0) |
|
||||
| Captured At | Producer-defined UTC timestamp ("the moment this tile imagery represents") persisted in `tiles.captured_at`. For Google Maps it is `DateTime.UtcNow` at download time (provider does not expose original imagery date); for UAV it is the capture timestamp supplied by the upload client. Drives the most-recent-across-sources read selection rule. | _docs/02_document/contracts/data-access/tile-storage.md (v1.0.0) |
|
||||
| Tile Source | The producer of a tile row, persisted in `tiles.source` as a contract-defined string (`google_maps`, `uav`, …). Per AZ-503-foundation: each `(cell, source, flight)` triple may have at most one row; reads return the most-recent across sources AND flights. Adding a new source requires a new `TileSource` enum member and a tile-storage contract version bump. | _docs/02_document/contracts/data-access/tile-storage.md (v2.0.0) |
|
||||
| Captured At | Producer-defined UTC timestamp ("the moment this tile imagery represents") persisted in `tiles.captured_at`. For Google Maps it is `DateTime.UtcNow` at download time (provider does not expose original imagery date); for UAV it is the capture timestamp supplied by the upload client. Drives the most-recent-across-(source, flight) read selection rule. | _docs/02_document/contracts/data-access/tile-storage.md (v2.0.0) |
|
||||
| UAV Tile Upload | `POST /api/satellite/upload` batch endpoint (AZ-488) that ingests UAV-captured tiles. Multipart envelope with a JSON `metadata` field and an aligned `files` collection; per-item results returned in a single HTTP 200 response. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.0.0) |
|
||||
| Quality Gate | The 5-rule validator (`UavTileQualityGate`) applied to every UAV tile before persistence: Format, Size band, Dimensions, Captured-at age, Blank/uniform. The first failing rule produces a reject reason from the closed `UavTileRejectReasons` enumeration. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.0.0) |
|
||||
| INVALID_FORMAT | UAV reject reason — content-type is not `image/jpeg` OR the file's first three bytes are not the JPEG magic `FF D8 FF` OR the bytes fail to decode as JPEG. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.0.0) |
|
||||
@@ -25,7 +25,7 @@
|
||||
| IMAGE_TOO_UNIFORM | UAV reject reason — pixel-luminance variance on the downsampled image is below `MinLuminanceVariance`. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.0.0) |
|
||||
| Flight ID | AZ-503 optional `Guid` identifier for a single UAV flight, sent as `metadata.flightId` per item on `POST /api/satellite/upload`. Two flights uploading the same `(z, x, y)` cell coexist as two `tiles` rows that share a single `location_hash` but have distinct `tiles.id` values and distinct on-disk file paths (`./tiles/uav/{flight_id}/{z}/{x}/{y}.jpg`). Anonymous uploads (no `flightId`) collapse to a single row per cell at the literal path `./tiles/uav/none/{z}/{x}/{y}.jpg`. | _docs/02_document/contracts/api/uav-tile-upload.md (v1.1.0) |
|
||||
| Tile Namespace | The constant UUID `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c` used by `Uuidv5.Create` to seed every tile-identity computation in this service. Pinned cross-repo with `gps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE` so both sides compute byte-identical UUIDv5 outputs for the same canonical name. Changing the constant on either side is a coordinated cross-repo break. | `SatelliteProvider.Common.Utils.Uuidv5.TileNamespace`, AZ-503 |
|
||||
| Location Hash | Deterministic UUIDv5 of `"{tile_zoom}/{tile_x}/{tile_y}"` under `Tile Namespace`. Identical across flights and sources for the same cell; stored in `tiles.location_hash` (NOT NULL). Reserved for the AZ-505 Leaflet covering index (`POST /api/satellite/tiles/inventory`). | _docs/02_document/data_model.md, AZ-503 |
|
||||
| Location Hash | Deterministic UUIDv5 of `"{tile_zoom}/{tile_x}/{tile_y}"` under `Tile Namespace`. Identical across flights and sources for the same cell; stored in `tiles.location_hash` (NOT NULL). Drives the Leaflet covering index `tiles_leaflet_path` (used by `GET /tiles/{z}/{x}/{y}`) and the `POST /api/satellite/tiles/inventory` bulk-lookup endpoint. Same UUIDv5 is computed independently on both sides of the cross-repo boundary (`SatelliteProvider.Common.Utils.Uuidv5.Create` in C# and `gps-denied-onboard/components/c6_tile_cache/_uuid.py:location_hash` in Python). | _docs/02_document/contracts/data-access/tile-storage.md (v2.0.0), AZ-503-foundation + AZ-505 |
|
||||
| Content SHA-256 | SHA-256 digest of the JPEG body, stored in `tiles.content_sha256` (`bytea`, NULLABLE at the DB layer; application code enforces NOT NULL for new writes). Used to detect byte-identical re-uploads. Legacy pre-AZ-503 rows are NULL because file paths are volatile and a reliable on-disk backfill was not possible. | _docs/02_document/data_model.md, AZ-503 |
|
||||
| Nadir Camera | Downward-facing camera on a UAV capturing ground imagery during flight | user clarification |
|
||||
| GPS-Denied Service | The consuming system: a UAV navigation service operating without GPS, using satellite/UAV imagery for positioning | user clarification |
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
**Language**: csharp
|
||||
**Layout Convention**: custom (per-component .csproj per logical component)
|
||||
**Root**: ./
|
||||
**Last Updated**: 2026-05-12 (cycle 5 — AZ-503 tile-identity foundation added: `SatelliteProvider.Common/Utils/Uuidv5.cs`, migration `014_AddTileIdentityColumns.sql`, 4 new `TileEntity` columns, integer-only flight-aware UPSERT, IntegrationTests → Common ProjectReference)
|
||||
**Last Updated**: 2026-05-12 (cycle 6 — AZ-505 tile inventory + Leaflet covering index + HTTP/2: new `POST /api/satellite/tiles/inventory` endpoint, new `ITileRepository.GetTilesByLocationHashesAsync`, rewired `GetByTileCoordinatesAsync` to filter on `location_hash`, migration `015_AddTilesLeafletPathIndex.sql`, Kestrel `Http1AndHttp2`, new `TileInventory*` DTOs in Common; cycle 5 — AZ-503 tile-identity foundation added: `SatelliteProvider.Common/Utils/Uuidv5.cs`, migration `014_AddTileIdentityColumns.sql`, 4 new `TileEntity` columns, integer-only flight-aware UPSERT, IntegrationTests → Common ProjectReference)
|
||||
|
||||
## Layout Rules
|
||||
|
||||
@@ -37,7 +37,7 @@ The cycle-1 (AZ-487) and cycle-2 (AZ-488) code reviews each surfaced an F1 (Low
|
||||
- `SatelliteProvider.Common/Configs/ProcessingConfig.cs`
|
||||
- `SatelliteProvider.Common/Configs/DatabaseConfig.cs`
|
||||
- `SatelliteProvider.Common/Configs/UavQualityConfig.cs` (added by AZ-488; UAV quality-gate + request-envelope knobs)
|
||||
- `SatelliteProvider.Common/DTO/*.cs` (all DTOs; AZ-488 added `UavTileMetadata`, `UavTileBatchMetadataPayload`, `UavTileBatchUploadResponse`, `UavTileUploadResultItem`, `UavTileUploadStatus`, `UavTileRejectReasons` — placed in Common to keep `TileDownloader` from depending on the API layer)
|
||||
- `SatelliteProvider.Common/DTO/*.cs` (all DTOs; AZ-488 added `UavTileMetadata`, `UavTileBatchMetadataPayload`, `UavTileBatchUploadResponse`, `UavTileUploadResultItem`, `UavTileUploadStatus`, `UavTileRejectReasons` — placed in Common to keep `TileDownloader` from depending on the API layer; AZ-505 added `TileInventory.cs` housing `TileInventoryRequest`, `TileCoord`, `TileInventoryResponse`, `TileInventoryEntry`, `TileInventoryLimits` for the bulk-lookup endpoint)
|
||||
- `SatelliteProvider.Common/Enums/RegionStatus.cs`
|
||||
- `SatelliteProvider.Common/Enums/RoutePointType.cs`
|
||||
- `SatelliteProvider.Common/Enums/TileSource.cs` (added by AZ-484; backed by the `tile-storage` v1.0.0 contract)
|
||||
@@ -59,10 +59,10 @@ The cycle-1 (AZ-487) and cycle-2 (AZ-488) code reviews each surfaced an F1 (Low
|
||||
- `SatelliteProvider.DataAccess/Models/RegionEntity.cs`
|
||||
- `SatelliteProvider.DataAccess/Models/RouteEntity.cs`
|
||||
- `SatelliteProvider.DataAccess/Models/RoutePointEntity.cs`
|
||||
- `SatelliteProvider.DataAccess/Repositories/ITileRepository.cs`
|
||||
- `SatelliteProvider.DataAccess/Repositories/ITileRepository.cs` (AZ-505 added `GetTilesByLocationHashesAsync` for the bulk inventory hot path)
|
||||
- `SatelliteProvider.DataAccess/Repositories/IRegionRepository.cs`
|
||||
- `SatelliteProvider.DataAccess/Repositories/IRouteRepository.cs`
|
||||
- `SatelliteProvider.DataAccess/Repositories/TileRepository.cs`
|
||||
- `SatelliteProvider.DataAccess/Repositories/TileRepository.cs` (AZ-505 rewired `GetByTileCoordinatesAsync` to filter on `location_hash` for `Index Only Scan` against `tiles_leaflet_path`; added Npgsql-direct `GetTilesByLocationHashesAsync` to sidestep Dapper's `IEnumerable` parameter expansion against `ANY($1::uuid[])`)
|
||||
- `SatelliteProvider.DataAccess/Repositories/RegionRepository.cs`
|
||||
- `SatelliteProvider.DataAccess/Repositories/RouteRepository.cs`
|
||||
- `SatelliteProvider.DataAccess/DatabaseMigrator.cs`
|
||||
@@ -122,7 +122,7 @@ The cycle-1 (AZ-487) and cycle-2 (AZ-488) code reviews each surfaced an F1 (Low
|
||||
|
||||
- **Directory**: `SatelliteProvider.Api/`
|
||||
- **Public API**:
|
||||
- `SatelliteProvider.Api/Program.cs` (minimal API endpoints, DI setup, middleware chain — `UseAuthentication` + `UseAuthorization` added in AZ-487; `/api/satellite/upload` rewired in AZ-488)
|
||||
- `SatelliteProvider.Api/Program.cs` (minimal API endpoints, DI setup, middleware chain — `UseAuthentication` + `UseAuthorization` added in AZ-487; `/api/satellite/upload` rewired in AZ-488; AZ-505 added `POST /api/satellite/tiles/inventory` + `builder.WebHost.ConfigureKestrel(... Protocols = HttpProtocols.Http1AndHttp2)` for HTTP/2 over plaintext)
|
||||
- `SatelliteProvider.Api/Authentication/AuthenticationServiceCollectionExtensions.cs` (added by AZ-487; `AddSatelliteJwt(IConfiguration)` registers `JwtBearer` with the suite-wide HS256 contract from `suite/_docs/10_auth.md`; validates `JWT_SECRET` ≥ 32 bytes at startup)
|
||||
- `SatelliteProvider.Api/Authentication/PermissionsRequirement.cs` + `PermissionsAuthorizationHandler` + `SatellitePermissions` (added by AZ-488; custom requirement that accepts a `permissions` claim shaped as either a single string or a JSON array; powers the `UavUploadPolicy` requiring the `GPS` permission)
|
||||
- `SatelliteProvider.Api/DTOs/UavTileBatchUploadRequest.cs` (added by AZ-488; multipart form binding envelope — kept in WebApi because it depends on `IFormFileCollection` + `[FromForm]`, both API-layer types)
|
||||
|
||||
@@ -8,8 +8,9 @@ Application entry point. Configures DI container, sets up middleware, defines mi
|
||||
### API Endpoints
|
||||
| Method | Route | Handler | Description |
|
||||
|--------|-------|---------|-------------|
|
||||
| GET | `/tiles/{z}/{x}/{y}` | `ServeTile` | Slippy map tile server with in-memory caching |
|
||||
| GET | `/tiles/{z}/{x}/{y}` | `ServeTile` | Slippy map tile server with in-memory caching. AZ-505 rewired the DB lookup to filter on `location_hash` (deterministic UUIDv5) so the read becomes an `Index Only Scan` against `tiles_leaflet_path`; the wire response is byte-identical to pre-AZ-505. |
|
||||
| GET | `/api/satellite/tiles/latlon` | `GetTileByLatLon` | Download single tile by lat/lon/zoom |
|
||||
| POST | `/api/satellite/tiles/inventory` | `GetTilesInventory` | Bulk tile-existence/metadata lookup (AZ-505) — body is XOR of `tiles[{tileZoom,tileX,tileY}]` (Form A) and `locationHashes[uuid]` (Form B), each capped at 5000 entries. Response is one entry per request entry, in input order. Contract: `_docs/02_document/contracts/api/tile-inventory.md` v1.0.0. |
|
||||
| GET | `/api/satellite/tiles/mgrs` | `GetSatelliteTilesByMgrs` | MGRS stub (returns empty) |
|
||||
| POST | `/api/satellite/upload` | `UploadUavTileBatch` | UAV tile batch upload (AZ-488) — multipart envelope, 5-rule quality gate, per-source UPSERT with `source='uav'`. Requires the `RequiresGpsPermission` policy. |
|
||||
| POST | `/api/satellite/request` | `RequestRegion` | Queue region for async tile processing |
|
||||
@@ -31,6 +32,13 @@ Application entry point. Configures DI container, sets up middleware, defines mi
|
||||
- `UavTileBatchUploadResponse`, `UavTileUploadResultItem` — per-item response shape
|
||||
- `UavTileUploadStatus`, `UavTileRejectReasons` — string-constant enumerations exposed in the v1.0.0 contract
|
||||
|
||||
### Common/DTO (AZ-505)
|
||||
- `TileInventoryRequest` — XOR body envelope with `Tiles` (Form A) OR `LocationHashes` (Form B)
|
||||
- `TileCoord` — `{TileZoom, TileX, TileY}` per-entry coord under Form A
|
||||
- `TileInventoryResponse` — `{Results: TileInventoryEntry[]}` response shape; ordering matches request
|
||||
- `TileInventoryEntry` — per-entry response shape (`Present`, `LocationHash`, optional `Id`/`CapturedAt`/`Source`/`FlightId`/`ResolutionMPerPx`)
|
||||
- `TileInventoryLimits.MaxEntriesPerRequest` — hard cap (5000) consumed by request validation
|
||||
|
||||
## Internal Logic
|
||||
|
||||
### DI Registration
|
||||
@@ -44,6 +52,7 @@ Application entry point. Configures DI container, sets up middleware, defines mi
|
||||
8. CORS policy: `TilesCors` — configured origins from `CorsConfig:AllowedOrigins`, falls back to allow-any
|
||||
9. JSON options: camelCase, case-insensitive
|
||||
10. **JWT authentication (AZ-487 + AZ-494)**: `AddSatelliteJwt(builder.Configuration)` (extension in `SatelliteProvider.Api.Authentication`) registers `JwtBearer` with `TokenValidationParameters` set per the suite auth contract: signature + lifetime + issuer + audience validation, 30 s clock skew, ≥ 32-byte HMAC key. The `iss` value comes from `JWT_ISSUER` env (fallback `Jwt:Issuer` config); the `aud` value comes from `JWT_AUDIENCE` env (fallback `Jwt:Audience` config). All three values (secret, iss, aud) are fail-fast — the API throws `InvalidOperationException` at startup if any is unset or whitespace-only. Production deploys MUST set the env vars with admin-team-confirmed values; `appsettings.json` ships empty so the fail-fast triggers. `appsettings.Development.json` ships clearly-tagged DEV-ONLY values (`DEV-ONLY-iss-admin-azaion-local` / `DEV-ONLY-aud-satellite-provider`) so local dev works out-of-the-box. Followed by `AddAuthorization` with the `RequiresGpsPermission` policy (AZ-488).
|
||||
11. **Kestrel HTTP/2 (AZ-505)**: `builder.WebHost.ConfigureKestrel(opts => opts.ConfigureEndpointDefaults(lo => lo.Protocols = HttpProtocols.Http1AndHttp2))`. Enables HTTP/2 over plaintext (h2c) on the dev endpoint so programmatic clients (`HttpClient` with `Version20` + `RequestVersionExact`, httpx `http2=True`) can multiplex tile reads on a single TCP connection. Browsers still negotiate HTTP/1.1 over plaintext — browser Leaflet performance is unaffected by the H2 flip and depends instead on the `tiles_leaflet_path` covering index.
|
||||
|
||||
### Startup
|
||||
1. Database migration via `DatabaseMigrator.RunMigrations()` — throws on failure
|
||||
@@ -54,10 +63,17 @@ Application entry point. Configures DI container, sets up middleware, defines mi
|
||||
|
||||
### ServeTile Handler
|
||||
1. Checks `IMemoryCache` for tile bytes (1h absolute, 30min sliding expiration)
|
||||
2. If cache miss: queries `ITileRepository.GetByTileCoordinatesAsync`
|
||||
2. If cache miss: queries `ITileRepository.GetByTileCoordinatesAsync` — AZ-505 rewired this method to compute `location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}")` and filter by `WHERE location_hash = $1`, hitting `tiles_leaflet_path` as an `Index Only Scan` with `Heap Fetches ≤ 1`. Selection rule is unchanged (most-recent across sources/flights); wire response is byte-identical.
|
||||
3. If no DB record: downloads tile via `GoogleMapsDownloaderV2.DownloadSingleTileAsync`, creates `TileEntity`, inserts
|
||||
4. Returns image bytes with cache headers (`Cache-Control: public, max-age=86400`)
|
||||
|
||||
### GetTilesInventory Handler (AZ-505)
|
||||
1. Validates XOR body shape: 400 if both `tiles` and `locationHashes` are populated, 400 if neither is populated, 400 if either exceeds `TileInventoryLimits.MaxEntriesPerRequest` (5000)
|
||||
2. Delegates to `ITileService.GetInventoryAsync(request, ct)`
|
||||
3. Service computes `location_hash` for Form A entries via `Uuidv5.Create(TileNamespace, "{z}/{x}/{y}")`, calls `ITileRepository.GetTilesByLocationHashesAsync(IReadOnlyList<Guid>)`, re-aligns results back to input order
|
||||
4. Returns `TileInventoryResponse` with one entry per input — `present=true` entries carry `id` / `capturedAt` / `source` / `flightId` / `resolutionMPerPx`; `present=false` entries carry only `locationHash`
|
||||
5. Authenticated by `.RequireAuthorization()` (401 before handler for anonymous)
|
||||
|
||||
### GetTileByLatLon Handler
|
||||
Downloads a tile, persists it, returns metadata as `DownloadTileResponse`.
|
||||
|
||||
|
||||
@@ -7,8 +7,9 @@ Dapper-based repository for the `tiles` table. Handles CRUD operations and spati
|
||||
|
||||
### ITileRepository (interface)
|
||||
- `GetByIdAsync(Guid id) → Task<TileEntity?>`
|
||||
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds the most-recent tile across all sources for the given slippy coordinates. Selection rule: `ORDER BY captured_at DESC, updated_at DESC, id DESC LIMIT 1` (AZ-484 v1.0.0 contract).
|
||||
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds the most-recent tile across all sources / flights for the given slippy coordinates. AZ-505 rewired the predicate to compute `location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}")` and filter by `WHERE location_hash = $1` so the read becomes an `Index Only Scan` against `tiles_leaflet_path`. Selection rule preserved unchanged: `ORDER BY captured_at DESC, updated_at DESC, id DESC LIMIT 1` (v2.0.0 contract).
|
||||
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileEntity>>`: spatial bounding box query (expanded by 2 × tile size to cover edges); applies `DISTINCT ON (latitude, longitude, tile_zoom, tile_size_meters)` per AZ-484 to return at most one row per cell — the most-recent across sources — preserving the historical caller-facing order `latitude DESC, longitude ASC`.
|
||||
- `GetTilesByLocationHashesAsync(IReadOnlyList<Guid> locationHashes) → Task<IReadOnlyDictionary<Guid, TileEntity>>` (AZ-505): bulk lookup keyed by `location_hash` for the inventory endpoint. Single round-trip; `DISTINCT ON (location_hash) ... WHERE location_hash = ANY($1::uuid[]) ORDER BY location_hash, captured_at DESC, updated_at DESC, id DESC`. NOT routed through Dapper — uses `NpgsqlCommand` with an explicit `NpgsqlParameter` typed `Array | Uuid` because Dapper's parameter expander rewrites `IEnumerable` to scalar placeholders, which is invalid against `ANY($1::uuid[])`.
|
||||
- `InsertAsync(TileEntity tile) → Task<Guid>`: AZ-503 integer-only + flight-aware UPSERT — `ON CONFLICT (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO UPDATE file_path, latitude, longitude, captured_at, location_hash, content_sha256, updated_at`. `id` is intentionally NOT overwritten on conflict — preserves AZ-503 AC-2 idempotence (same inputs ⇒ same `id`). Supersedes the AZ-484 5-column float-based unique key (`idx_tiles_unique_location_source`).
|
||||
- `UpdateAsync(TileEntity tile) → Task<int>`: full row update by `id` including `source`, `captured_at`, `flight_id`, `location_hash`, and `content_sha256`.
|
||||
- `DeleteAsync(Guid id) → Task<int>`
|
||||
@@ -19,7 +20,8 @@ Constructs a new `NpgsqlConnection` per method call (no connection pooling at th
|
||||
## Internal Logic
|
||||
- `GetTilesByRegionAsync` calculates a bounding box by expanding the requested region by 2 × tile size to ensure edge tiles are included. Uses meters-to-degrees approximation via `GeoUtils` (post-AZ-377 — single source of truth for Earth constants).
|
||||
- `InsertAsync` uses the AZ-503 integer-only + flight-aware UPSERT keyed on `idx_tiles_unique_identity` (created by migration 014, replacing the AZ-484 `idx_tiles_unique_location_source`). The conflict key uses `COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)` so anonymous (`flight_id IS NULL`) and per-flight UAV rows share a flat key space. Two producers (`google_maps` + `uav`) at the same cell with the same `flight_id` (typically `NULL` for `google_maps`) still coexist via `source` discrimination. Same-source same-flight re-insert refreshes `file_path`, `latitude`, `longitude`, `captured_at`, `location_hash`, `content_sha256`, `updated_at` — but NOT `id` (idempotence — AZ-503 AC-2).
|
||||
- `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` apply the AZ-484 selection rule unchanged: most-recent across sources, deterministic tie-break on `(captured_at DESC, updated_at DESC, id DESC)`. AZ-503 does NOT rewrite the read path to use `location_hash` — that's deferred to AZ-505 alongside the Leaflet covering index.
|
||||
- `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` apply the AZ-484 selection rule unchanged: most-recent across sources, deterministic tie-break on `(captured_at DESC, updated_at DESC, id DESC)`. AZ-505 rewrote `GetByTileCoordinatesAsync` to filter on `location_hash` instead of `(tile_zoom, tile_x, tile_y)` — this is a behavior-preserving rewrite (deterministic UUIDv5) that enables `Index Only Scan` against `tiles_leaflet_path`. `GetTilesByRegionAsync` retains the lat/lon filter because spatial bounding-box queries don't reduce to a finite hash-set lookup.
|
||||
- `GetTilesByLocationHashesAsync` (AZ-505) is the inventory hot path. Deliberately bypasses Dapper because the `ANY($1::uuid[])` predicate requires array-typed parameter binding (Npgsql `NpgsqlDbType.Array | Uuid`) that Dapper's `IEnumerable` parameter expansion replaces with a comma-separated list of scalar placeholders, producing invalid SQL. Manual `NpgsqlDataReader` mapping to `TileEntity` is the trade-off. Slow-query threshold matches `GetTilesByRegionAsync` (500 ms).
|
||||
- `TileEntity.Source` is a plain `string` storing the snake_case wire value (`'google_maps'` | `'uav'`); enum<->wire conversion happens via `SatelliteProvider.Common.Enums.TileSourceConverter`. This avoids Dapper issue #259 (TypeHandler<T> bypass for enum reads — see `_docs/LESSONS.md` L-001).
|
||||
- `FindExistingTileAsync` was removed by AZ-376 (see `_docs/04_refactoring/03-code-quality-refactoring/`).
|
||||
|
||||
@@ -30,11 +32,11 @@ Constructs a new `NpgsqlConnection` per method call (no connection pooling at th
|
||||
- `Microsoft.Extensions.Logging`
|
||||
|
||||
## Contract
|
||||
Implements the frozen v1.0.0 contract `_docs/02_document/contracts/data-access/tile-storage.md` plus the AZ-503-introduced columns (`flight_id`, `location_hash`, `content_sha256`, `legacy_id`) and the integer-only UPSERT key. Schema invariants Inv-1..Inv-5 (UPSERT semantics, selection rule, source value space) are preserved; the only contract change is that the UPSERT conflict detection no longer depends on bit-identical float `latitude`/`longitude` (AZ-503 AC-4).
|
||||
Implements the frozen v2.0.0 contract `_docs/02_document/contracts/data-access/tile-storage.md` — captures the four AZ-503-foundation columns (`flight_id`, `location_hash`, `content_sha256`, `legacy_id`), the integer-only flight-aware UPSERT key (`idx_tiles_unique_identity`), the `tiles_leaflet_path` covering index (AZ-505 migration 015), the new `location_hash`-keyed `GetByTileCoordinatesAsync` read path (AZ-505), and the new `GetTilesByLocationHashesAsync` bulk lookup (AZ-505). Inv-1..Inv-9 from the v2.0.0 contract apply.
|
||||
|
||||
## Consumers
|
||||
- `TileService` — all read/write operations
|
||||
- `Program.cs` (ServeTile, GetTileByLatLon handlers) — `GetByTileCoordinatesAsync`, `InsertAsync`
|
||||
- `TileService` — all read/write operations including `GetInventoryAsync` (AZ-505) which routes through `GetTilesByLocationHashesAsync`
|
||||
- `Program.cs` (ServeTile, GetTileByLatLon, `GetTilesInventory` handlers) — `GetByTileCoordinatesAsync`, `InsertAsync`, indirectly `GetTilesByLocationHashesAsync` via `TileService.GetInventoryAsync`
|
||||
|
||||
## Data Models
|
||||
Operates on `TileEntity`.
|
||||
|
||||
Reference in New Issue
Block a user