[AZ-505] Tile inventory endpoint + HTTP/2 + Leaflet covering index

Production code:
- POST /api/satellite/tiles/inventory (XOR body, 5000-cap,
  most-recent-per-location_hash select, present/absent shaping).
- Kestrel HttpProtocols.Http1AndHttp2 on every listener (AC-5).
- Migration 015 creates tiles_leaflet_path covering index over
  (location_hash, captured_at DESC, updated_at DESC, id DESC)
  INCLUDE (file_path, source); drops superseded idx_tiles_location_hash.
- TileRepository.GetByTileCoordinatesAsync rewired to filter by
  location_hash (Index Only Scan via tiles_leaflet_path).
- TileRepository.GetTilesByLocationHashesAsync added with Npgsql-
  direct ANY($1::uuid[]) binding (Dapper IEnumerable expansion is
  incompatible with the array form).
- Uuidv5.LocationHashForTile centralises the UUIDv5(TileNamespace,
  "{z}/{x}/{y}") formula — single source of truth for the cross-repo
  invariant (gps-denied-onboard parity).

Contracts:
- New: contracts/api/tile-inventory.md v1.0.0.
- Bumped: contracts/data-access/tile-storage.md to v2.0.0 (joint
  ownership by AZ-503-foundation + AZ-505: schema + covering index +
  GetByTileCoordinatesAsync rewrite).

Tests:
- TileInventoryTests covers AC-1, AC-2 (DB-level), AC-4, AC-6.
- Http2MultiplexingTests covers AC-5 (20 concurrent multiplexed GETs
  over h2c via SocketsHttpHandler + AppContext Http2Unencrypted switch).
- LeafletPathIndexOnlyTests covers AC-3 (EXPLAIN (ANALYZE, BUFFERS)
  asserts Index Only Scan over tiles_leaflet_path with heap_blocks=0).

Docs:
- architecture.md, system-flows.md, data_model.md, module-layout.md,
  glossary.md, modules/api_program.md, modules/dataaccess_tile_repository.md,
  components/02_data_access/description.md all updated to reference the
  v2.0.0 tile-storage contract + new tile-inventory contract + AC-7.

Reports:
- batch_01_cycle6_report.md, batch_01_cycle6_review.md,
  implementation_completeness_cycle6_report.md (PASS),
  implementation_report_tile_inventory_cycle6.md.

Task spec moved todo/ -> done/.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-12 21:16:37 +03:00
parent 3c7cd4e56b
commit 909f69cb3a
26 changed files with 1780 additions and 65 deletions
+18 -2
View File
@@ -8,8 +8,9 @@ Application entry point. Configures DI container, sets up middleware, defines mi
### API Endpoints
| Method | Route | Handler | Description |
|--------|-------|---------|-------------|
| GET | `/tiles/{z}/{x}/{y}` | `ServeTile` | Slippy map tile server with in-memory caching |
| GET | `/tiles/{z}/{x}/{y}` | `ServeTile` | Slippy map tile server with in-memory caching. AZ-505 rewired the DB lookup to filter on `location_hash` (deterministic UUIDv5) so the read becomes an `Index Only Scan` against `tiles_leaflet_path`; the wire response is byte-identical to pre-AZ-505. |
| GET | `/api/satellite/tiles/latlon` | `GetTileByLatLon` | Download single tile by lat/lon/zoom |
| POST | `/api/satellite/tiles/inventory` | `GetTilesInventory` | Bulk tile-existence/metadata lookup (AZ-505) — body is XOR of `tiles[{tileZoom,tileX,tileY}]` (Form A) and `locationHashes[uuid]` (Form B), each capped at 5000 entries. Response is one entry per request entry, in input order. Contract: `_docs/02_document/contracts/api/tile-inventory.md` v1.0.0. |
| GET | `/api/satellite/tiles/mgrs` | `GetSatelliteTilesByMgrs` | MGRS stub (returns empty) |
| POST | `/api/satellite/upload` | `UploadUavTileBatch` | UAV tile batch upload (AZ-488) — multipart envelope, 5-rule quality gate, per-source UPSERT with `source='uav'`. Requires the `RequiresGpsPermission` policy. |
| POST | `/api/satellite/request` | `RequestRegion` | Queue region for async tile processing |
@@ -31,6 +32,13 @@ Application entry point. Configures DI container, sets up middleware, defines mi
- `UavTileBatchUploadResponse`, `UavTileUploadResultItem` — per-item response shape
- `UavTileUploadStatus`, `UavTileRejectReasons` — string-constant enumerations exposed in the v1.0.0 contract
### Common/DTO (AZ-505)
- `TileInventoryRequest` — XOR body envelope with `Tiles` (Form A) OR `LocationHashes` (Form B)
- `TileCoord``{TileZoom, TileX, TileY}` per-entry coord under Form A
- `TileInventoryResponse``{Results: TileInventoryEntry[]}` response shape; ordering matches request
- `TileInventoryEntry` — per-entry response shape (`Present`, `LocationHash`, optional `Id`/`CapturedAt`/`Source`/`FlightId`/`ResolutionMPerPx`)
- `TileInventoryLimits.MaxEntriesPerRequest` — hard cap (5000) consumed by request validation
## Internal Logic
### DI Registration
@@ -44,6 +52,7 @@ Application entry point. Configures DI container, sets up middleware, defines mi
8. CORS policy: `TilesCors` — configured origins from `CorsConfig:AllowedOrigins`, falls back to allow-any
9. JSON options: camelCase, case-insensitive
10. **JWT authentication (AZ-487 + AZ-494)**: `AddSatelliteJwt(builder.Configuration)` (extension in `SatelliteProvider.Api.Authentication`) registers `JwtBearer` with `TokenValidationParameters` set per the suite auth contract: signature + lifetime + issuer + audience validation, 30 s clock skew, ≥ 32-byte HMAC key. The `iss` value comes from `JWT_ISSUER` env (fallback `Jwt:Issuer` config); the `aud` value comes from `JWT_AUDIENCE` env (fallback `Jwt:Audience` config). All three values (secret, iss, aud) are fail-fast — the API throws `InvalidOperationException` at startup if any is unset or whitespace-only. Production deploys MUST set the env vars with admin-team-confirmed values; `appsettings.json` ships empty so the fail-fast triggers. `appsettings.Development.json` ships clearly-tagged DEV-ONLY values (`DEV-ONLY-iss-admin-azaion-local` / `DEV-ONLY-aud-satellite-provider`) so local dev works out-of-the-box. Followed by `AddAuthorization` with the `RequiresGpsPermission` policy (AZ-488).
11. **Kestrel HTTP/2 (AZ-505)**: `builder.WebHost.ConfigureKestrel(opts => opts.ConfigureEndpointDefaults(lo => lo.Protocols = HttpProtocols.Http1AndHttp2))`. Enables HTTP/2 over plaintext (h2c) on the dev endpoint so programmatic clients (`HttpClient` with `Version20` + `RequestVersionExact`, httpx `http2=True`) can multiplex tile reads on a single TCP connection. Browsers still negotiate HTTP/1.1 over plaintext — browser Leaflet performance is unaffected by the H2 flip and depends instead on the `tiles_leaflet_path` covering index.
### Startup
1. Database migration via `DatabaseMigrator.RunMigrations()` — throws on failure
@@ -54,10 +63,17 @@ Application entry point. Configures DI container, sets up middleware, defines mi
### ServeTile Handler
1. Checks `IMemoryCache` for tile bytes (1h absolute, 30min sliding expiration)
2. If cache miss: queries `ITileRepository.GetByTileCoordinatesAsync`
2. If cache miss: queries `ITileRepository.GetByTileCoordinatesAsync` — AZ-505 rewired this method to compute `location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}")` and filter by `WHERE location_hash = $1`, hitting `tiles_leaflet_path` as an `Index Only Scan` with `Heap Fetches ≤ 1`. Selection rule is unchanged (most-recent across sources/flights); wire response is byte-identical.
3. If no DB record: downloads tile via `GoogleMapsDownloaderV2.DownloadSingleTileAsync`, creates `TileEntity`, inserts
4. Returns image bytes with cache headers (`Cache-Control: public, max-age=86400`)
### GetTilesInventory Handler (AZ-505)
1. Validates XOR body shape: 400 if both `tiles` and `locationHashes` are populated, 400 if neither is populated, 400 if either exceeds `TileInventoryLimits.MaxEntriesPerRequest` (5000)
2. Delegates to `ITileService.GetInventoryAsync(request, ct)`
3. Service computes `location_hash` for Form A entries via `Uuidv5.Create(TileNamespace, "{z}/{x}/{y}")`, calls `ITileRepository.GetTilesByLocationHashesAsync(IReadOnlyList<Guid>)`, re-aligns results back to input order
4. Returns `TileInventoryResponse` with one entry per input — `present=true` entries carry `id` / `capturedAt` / `source` / `flightId` / `resolutionMPerPx`; `present=false` entries carry only `locationHash`
5. Authenticated by `.RequireAuthorization()` (401 before handler for anonymous)
### GetTileByLatLon Handler
Downloads a tile, persists it, returns metadata as `DownloadTileResponse`.
@@ -7,8 +7,9 @@ Dapper-based repository for the `tiles` table. Handles CRUD operations and spati
### ITileRepository (interface)
- `GetByIdAsync(Guid id) → Task<TileEntity?>`
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds the most-recent tile across all sources for the given slippy coordinates. Selection rule: `ORDER BY captured_at DESC, updated_at DESC, id DESC LIMIT 1` (AZ-484 v1.0.0 contract).
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds the most-recent tile across all sources / flights for the given slippy coordinates. AZ-505 rewired the predicate to compute `location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}")` and filter by `WHERE location_hash = $1` so the read becomes an `Index Only Scan` against `tiles_leaflet_path`. Selection rule preserved unchanged: `ORDER BY captured_at DESC, updated_at DESC, id DESC LIMIT 1` (v2.0.0 contract).
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileEntity>>`: spatial bounding box query (expanded by 2 × tile size to cover edges); applies `DISTINCT ON (latitude, longitude, tile_zoom, tile_size_meters)` per AZ-484 to return at most one row per cell — the most-recent across sources — preserving the historical caller-facing order `latitude DESC, longitude ASC`.
- `GetTilesByLocationHashesAsync(IReadOnlyList<Guid> locationHashes) → Task<IReadOnlyDictionary<Guid, TileEntity>>` (AZ-505): bulk lookup keyed by `location_hash` for the inventory endpoint. Single round-trip; `DISTINCT ON (location_hash) ... WHERE location_hash = ANY($1::uuid[]) ORDER BY location_hash, captured_at DESC, updated_at DESC, id DESC`. NOT routed through Dapper — uses `NpgsqlCommand` with an explicit `NpgsqlParameter` typed `Array | Uuid` because Dapper's parameter expander rewrites `IEnumerable` to scalar placeholders, which is invalid against `ANY($1::uuid[])`.
- `InsertAsync(TileEntity tile) → Task<Guid>`: AZ-503 integer-only + flight-aware UPSERT — `ON CONFLICT (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)) DO UPDATE file_path, latitude, longitude, captured_at, location_hash, content_sha256, updated_at`. `id` is intentionally NOT overwritten on conflict — preserves AZ-503 AC-2 idempotence (same inputs ⇒ same `id`). Supersedes the AZ-484 5-column float-based unique key (`idx_tiles_unique_location_source`).
- `UpdateAsync(TileEntity tile) → Task<int>`: full row update by `id` including `source`, `captured_at`, `flight_id`, `location_hash`, and `content_sha256`.
- `DeleteAsync(Guid id) → Task<int>`
@@ -19,7 +20,8 @@ Constructs a new `NpgsqlConnection` per method call (no connection pooling at th
## Internal Logic
- `GetTilesByRegionAsync` calculates a bounding box by expanding the requested region by 2 × tile size to ensure edge tiles are included. Uses meters-to-degrees approximation via `GeoUtils` (post-AZ-377 — single source of truth for Earth constants).
- `InsertAsync` uses the AZ-503 integer-only + flight-aware UPSERT keyed on `idx_tiles_unique_identity` (created by migration 014, replacing the AZ-484 `idx_tiles_unique_location_source`). The conflict key uses `COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)` so anonymous (`flight_id IS NULL`) and per-flight UAV rows share a flat key space. Two producers (`google_maps` + `uav`) at the same cell with the same `flight_id` (typically `NULL` for `google_maps`) still coexist via `source` discrimination. Same-source same-flight re-insert refreshes `file_path`, `latitude`, `longitude`, `captured_at`, `location_hash`, `content_sha256`, `updated_at` — but NOT `id` (idempotence — AZ-503 AC-2).
- `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` apply the AZ-484 selection rule unchanged: most-recent across sources, deterministic tie-break on `(captured_at DESC, updated_at DESC, id DESC)`. AZ-503 does NOT rewrite the read path to use `location_hash` — that's deferred to AZ-505 alongside the Leaflet covering index.
- `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` apply the AZ-484 selection rule unchanged: most-recent across sources, deterministic tie-break on `(captured_at DESC, updated_at DESC, id DESC)`. AZ-505 rewrote `GetByTileCoordinatesAsync` to filter on `location_hash` instead of `(tile_zoom, tile_x, tile_y)` — this is a behavior-preserving rewrite (deterministic UUIDv5) that enables `Index Only Scan` against `tiles_leaflet_path`. `GetTilesByRegionAsync` retains the lat/lon filter because spatial bounding-box queries don't reduce to a finite hash-set lookup.
- `GetTilesByLocationHashesAsync` (AZ-505) is the inventory hot path. Deliberately bypasses Dapper because the `ANY($1::uuid[])` predicate requires array-typed parameter binding (Npgsql `NpgsqlDbType.Array | Uuid`) that Dapper's `IEnumerable` parameter expansion replaces with a comma-separated list of scalar placeholders, producing invalid SQL. Manual `NpgsqlDataReader` mapping to `TileEntity` is the trade-off. Slow-query threshold matches `GetTilesByRegionAsync` (500 ms).
- `TileEntity.Source` is a plain `string` storing the snake_case wire value (`'google_maps'` | `'uav'`); enum<->wire conversion happens via `SatelliteProvider.Common.Enums.TileSourceConverter`. This avoids Dapper issue #259 (TypeHandler<T> bypass for enum reads — see `_docs/LESSONS.md` L-001).
- `FindExistingTileAsync` was removed by AZ-376 (see `_docs/04_refactoring/03-code-quality-refactoring/`).
@@ -30,11 +32,11 @@ Constructs a new `NpgsqlConnection` per method call (no connection pooling at th
- `Microsoft.Extensions.Logging`
## Contract
Implements the frozen v1.0.0 contract `_docs/02_document/contracts/data-access/tile-storage.md` plus the AZ-503-introduced columns (`flight_id`, `location_hash`, `content_sha256`, `legacy_id`) and the integer-only UPSERT key. Schema invariants Inv-1..Inv-5 (UPSERT semantics, selection rule, source value space) are preserved; the only contract change is that the UPSERT conflict detection no longer depends on bit-identical float `latitude`/`longitude` (AZ-503 AC-4).
Implements the frozen v2.0.0 contract `_docs/02_document/contracts/data-access/tile-storage.md` — captures the four AZ-503-foundation columns (`flight_id`, `location_hash`, `content_sha256`, `legacy_id`), the integer-only flight-aware UPSERT key (`idx_tiles_unique_identity`), the `tiles_leaflet_path` covering index (AZ-505 migration 015), the new `location_hash`-keyed `GetByTileCoordinatesAsync` read path (AZ-505), and the new `GetTilesByLocationHashesAsync` bulk lookup (AZ-505). Inv-1..Inv-9 from the v2.0.0 contract apply.
## Consumers
- `TileService` — all read/write operations
- `Program.cs` (ServeTile, GetTileByLatLon handlers) — `GetByTileCoordinatesAsync`, `InsertAsync`
- `TileService` — all read/write operations including `GetInventoryAsync` (AZ-505) which routes through `GetTilesByLocationHashesAsync`
- `Program.cs` (ServeTile, GetTileByLatLon, `GetTilesInventory` handlers) — `GetByTileCoordinatesAsync`, `InsertAsync`, indirectly `GetTilesByLocationHashesAsync` via `TileService.GetInventoryAsync`
## Data Models
Operates on `TileEntity`.