mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-21 19:51:14 +00:00
[AZ-484] Cycle 1 Steps 12-16: docs, security, perf, deploy report
Captures the post-implementation autodev gates for AZ-484 multi-source tile storage: - Step 12 (Test-Spec Sync): added 7 AC rows (AZ-484 AC-1..AC-7) and a PT-07 NFR row to traceability-matrix.md; added PT-07 scenario to performance-tests.md. - Step 13 (Update Docs): refreshed data_model.md (tiles columns + indexes + selection rule + UPSERT contract + migrations 012/013), module-layout.md (Common/Enums section with L-001 guidance, DataAccess imports-from now lists 6 sites), 6 module / component docs to reflect the new repo signatures, source/captured_at fields, and Dapper enum bypass workaround. ripple_log_cycle1.md records zero out-of-scope ripple. - Step 14 (Security Audit): PASS_WITH_WARNINGS - 0 Critical, 0 High, 5 Medium, 5 Low. AZ-484 itself added zero new findings. Hardening items (Postgres default creds, .env in build context, GMaps key rotation, ASP.NET Core 8.0.21 -> 8.0.25, rate limiter) recorded for separate tickets. - Step 15 (Performance Test): all PT-01..PT-07 scenarios Unverified (non-blocking); PT-07 baseline-comparison harness deferred to a leftover for next cycle. - Step 16 (Deploy): cycle deploy report covering migration safety, rollback path, post-deploy verification, security caveats. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -16,13 +16,14 @@
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
| `GetByIdAsync` | Guid | `TileEntity?` | Yes | NpgsqlException |
|
||||
| `GetByTileCoordinatesAsync` | zoom, x, y | `TileEntity?` | Yes | NpgsqlException |
|
||||
| `FindExistingTileAsync` | lat, lon, tileSizeM, zoom, version | `TileEntity?` | Yes | NpgsqlException |
|
||||
| `GetTilesByRegionAsync` | lat, lon, sizeM, zoom | `IEnumerable<TileEntity>` | Yes | NpgsqlException |
|
||||
| `InsertAsync` | `TileEntity` | Guid | Yes | NpgsqlException |
|
||||
| `GetByTileCoordinatesAsync` | zoom, x, y | `TileEntity?` (most-recent across sources, AZ-484) | Yes | NpgsqlException |
|
||||
| `GetTilesByRegionAsync` | lat, lon, sizeM, zoom | `IEnumerable<TileEntity>` (one row per cell via `DISTINCT ON`, AZ-484) | Yes | NpgsqlException |
|
||||
| `InsertAsync` | `TileEntity` | Guid (per-source UPSERT, AZ-484) | Yes | NpgsqlException |
|
||||
| `UpdateAsync` | `TileEntity` | int | Yes | NpgsqlException |
|
||||
| `DeleteAsync` | Guid | int | Yes | NpgsqlException |
|
||||
|
||||
`FindExistingTileAsync` was removed by AZ-376 (replaced by direct cell lookups through `GetByTileCoordinatesAsync` + `GetTilesByRegionAsync`).
|
||||
|
||||
### Interface: IRegionRepository
|
||||
| Method | Input | Output | Async | Error Types |
|
||||
|--------|-------|--------|-------|-------------|
|
||||
@@ -58,7 +59,7 @@
|
||||
|-------|-----------|----------|--------------|
|
||||
| GetByTileCoordinatesAsync (tile lookup) | Very High | Yes | `(tile_zoom, tile_x, tile_y)` |
|
||||
| GetTilesByRegionAsync (spatial) | High | Yes | `(latitude, longitude, tile_zoom)` |
|
||||
| InsertAsync (tile upsert) | High | Yes | Composite unique on `(lat, lon, zoom, size, version)` |
|
||||
| InsertAsync (tile per-source upsert) | High | Yes | Composite unique on `(lat, lon, tile_zoom, tile_size_meters, source)` (AZ-484: `idx_tiles_unique_location_source`) |
|
||||
| GetByStatusAsync (region polling) | Medium | No | `(status)` |
|
||||
| GetRoutesWithPendingMapsAsync | Low | No | `(request_maps, maps_ready)` |
|
||||
|
||||
@@ -88,7 +89,9 @@
|
||||
|
||||
- Repository interfaces are defined in this project (not in Common), creating a dependency from Services to DataAccess
|
||||
- Column mapping uses SQL aliases (`tile_zoom as TileZoom`) rather than Dapper attribute mapping
|
||||
- TileRepository.InsertAsync uses an upsert pattern; concurrent inserts of the same tile won't conflict
|
||||
- TileRepository.InsertAsync uses a per-source UPSERT pattern (AZ-484); two producers (e.g., `google_maps` + `uav`) coexist as separate rows for the same cell, while same-source re-inserts overwrite and refresh `captured_at`
|
||||
- `TileEntity.Source` is stored as a plain `string` (not the `TileSource` enum) due to Dapper issue #259 — see `_docs/LESSONS.md` L-001. Conversion happens via `SatelliteProvider.Common.Enums.TileSourceConverter`
|
||||
- The frozen v1.0.0 `tile-storage` contract (`_docs/02_document/contracts/data-access/`) is the authoritative spec for all `tiles` invariants enforced here
|
||||
- No soft-delete; `DeleteAsync` is a hard delete
|
||||
|
||||
## 8. Dependency Graph
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
**csproj**: `SatelliteProvider.Services.TileDownloader/SatelliteProvider.Services.TileDownloader.csproj` (split out of the monolithic `SatelliteProvider.Services` project in epic AZ-309)
|
||||
|
||||
**Upstream dependencies**: Common (DTOs, GeoUtils, configs, RateLimitException), DataAccess (TileEntity, ITileRepository)
|
||||
**Upstream dependencies**: Common (DTOs, Enums — `TileSource` + `TileSourceConverter` since AZ-484, GeoUtils, configs, RateLimitException), DataAccess (TileEntity, ITileRepository)
|
||||
|
||||
**Downstream consumers**: RegionProcessing and WebApi — both via `ITileService` from Common (no compile-time `ProjectReference` from any consumer to this project's concrete types).
|
||||
|
||||
@@ -35,7 +35,7 @@
|
||||
| Data | Cache Type | TTL | Invalidation |
|
||||
|------|-----------|-----|-------------|
|
||||
| Tile bytes | In-memory (IMemoryCache, owned by TileService since AZ-310) | 1h absolute, 30min sliding | None (manual restart) |
|
||||
| Tile metadata | Database | Until year rollover | Version-based (current year) |
|
||||
| Tile metadata | Database | Append-by-source per cell (AZ-484); reads return most-recent across sources | Per-source UPSERT keyed on `(latitude, longitude, tile_zoom, tile_size_meters, source)` overwrites the existing same-source row and refreshes `captured_at` |
|
||||
| Active downloads | ConcurrentDictionary | Duration of download | Removed on completion |
|
||||
|
||||
## 5. Implementation Details
|
||||
|
||||
@@ -14,6 +14,8 @@ erDiagram
|
||||
varchar image_type
|
||||
varchar maps_version
|
||||
int version
|
||||
varchar source
|
||||
timestamp captured_at
|
||||
varchar file_path
|
||||
int tile_x
|
||||
int tile_y
|
||||
@@ -96,19 +98,25 @@ Stores metadata for downloaded satellite imagery tiles. Each tile is a single im
|
||||
| tile_size_meters | DOUBLE PRECISION | NOT NULL | Ground coverage in meters |
|
||||
| tile_size_pixels | INT | NOT NULL | Image dimension in pixels |
|
||||
| image_type | VARCHAR(10) | NOT NULL | Image format (e.g., "jpg") |
|
||||
| maps_version | VARCHAR(50) | | Legacy free-form provider tag; post-AZ-373 new rows write NULL (column retained for forensics on pre-existing rows) |
|
||||
| version | INT | NOT NULL, DEFAULT 2025 | Year-based versioning for cache invalidation |
|
||||
| maps_version | VARCHAR(50) | | Legacy free-form provider tag; post-AZ-373 new rows write NULL. Vestigial post-AZ-484 (column retained for forensics on pre-existing rows; no longer part of any index) |
|
||||
| version | INT | NOT NULL, DEFAULT 2025 | Year-based versioning for cache invalidation. Vestigial post-AZ-484 — removed from the unique key by migration 012 (preparation for AZ-484); column retained nullable for backward compatibility |
|
||||
| source | VARCHAR(32) | NOT NULL, DEFAULT 'google_maps' | AZ-484: producer of the imagery (`'google_maps'`, `'uav'`). Closed value set — see `tile-storage` v1.0.0 contract Inv-5 and `Common.Enums.TileSourceConverter`. Backfilled to `'google_maps'` for all pre-AZ-484 rows by migration 013 |
|
||||
| captured_at | TIMESTAMP | NOT NULL | AZ-484: imagery acquisition timestamp (UTC). Drives most-recent-across-sources selection. Backfilled to `created_at` for pre-AZ-484 rows by migration 013 |
|
||||
| file_path | VARCHAR(500) | NOT NULL | Relative path to stored image |
|
||||
| tile_x | INT | NOT NULL | Tile X coordinate (Slippy Map) |
|
||||
| tile_y | INT | NOT NULL | Tile Y coordinate (Slippy Map) |
|
||||
| created_at | TIMESTAMP | NOT NULL, DEFAULT NOW | |
|
||||
| updated_at | TIMESTAMP | NOT NULL, DEFAULT NOW | |
|
||||
|
||||
**Indexes**:
|
||||
- `idx_tiles_unique_location` UNIQUE (latitude, longitude, tile_zoom, tile_size_meters, version)
|
||||
**Indexes** (post-AZ-484):
|
||||
- `idx_tiles_unique_location_source` UNIQUE (latitude, longitude, tile_zoom, tile_size_meters, source) — created by migration 013; replaces the pre-AZ-484 4-col `idx_tiles_unique_location` (which itself superseded the legacy 5-col `(…, version)` index dropped by migration 012)
|
||||
- `idx_tiles_coordinates` (tile_zoom, tile_x, tile_y, version)
|
||||
- `idx_tiles_zoom` (tile_zoom)
|
||||
|
||||
**Selection rule**: `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` return the most-recent row across sources for any `(latitude, longitude, tile_zoom, tile_size_meters)` cell. Tie-break: `captured_at DESC, updated_at DESC, id DESC`. Region read uses `DISTINCT ON` to enforce one-row-per-cell at the SQL layer.
|
||||
|
||||
**UPSERT contract**: `INSERT … ON CONFLICT (latitude, longitude, tile_zoom, tile_size_meters, source) DO UPDATE` — same-source re-insert refreshes `file_path, tile_x, tile_y, captured_at, updated_at`. Two producers for the same cell coexist as separate rows.
|
||||
|
||||
### regions
|
||||
|
||||
Tracks region download requests and their processing status.
|
||||
@@ -215,3 +223,5 @@ Junction table linking routes to their generated region requests, with geofence
|
||||
| 009 | AddGeofencePolygonIndex | Polygon index tracking |
|
||||
| 010 | AddTilesZipToRoutes | ZIP generation fields |
|
||||
| 011 | AddTileCoordinates | Slippy map X/Y + rename zoom_level → tile_zoom |
|
||||
| 012 | DropTileVersionConstraint | Drops legacy 5-col `(…, version)` unique index; replaces with 4-col `idx_tiles_unique_location` (preparation for AZ-484) |
|
||||
| 013 | AddTileSourceAndCapturedAt | AZ-484: adds `source` (default `'google_maps'`) + `captured_at` columns; backfills both for pre-existing rows; replaces 4-col unique with 5-col `idx_tiles_unique_location_source`. Transactional; idempotent against partial replays |
|
||||
|
||||
@@ -57,7 +57,7 @@
|
||||
- **Internal**: (none — all repository types are public for DI registration)
|
||||
- **Owns**: `SatelliteProvider.DataAccess/**`
|
||||
- **ProjectReferences**: `SatelliteProvider.Common`
|
||||
- **Imports from**: `SatelliteProvider.Common.Enums` (5 sites: `RegionRepository`, `IRegionRepository`, `Models/RegionEntity`, `Models/RoutePointEntity`, `TypeHandlers/EnumStringTypeHandler`); `SatelliteProvider.Common.Configs` (`MapConfig.DefaultTileSizePixels` in `TileRepository`); `SatelliteProvider.Common.Utils` (`GeoUtils.EarthEquatorialCircumferenceMeters`, `GeoUtils.MetersPerDegreeLatitude` in `TileRepository`).
|
||||
- **Imports from**: `SatelliteProvider.Common.Enums` (6 sites: `RegionRepository`, `IRegionRepository`, `Models/RegionEntity`, `Models/RoutePointEntity`, `TypeHandlers/EnumStringTypeHandler`, `Models/TileEntity` — references `TileSourceConverter.GoogleMapsWireValue` const for the AZ-484 default value); `SatelliteProvider.Common.Configs` (`MapConfig.DefaultTileSizePixels` in `TileRepository`); `SatelliteProvider.Common.Utils` (`GeoUtils.EarthEquatorialCircumferenceMeters`, `GeoUtils.MetersPerDegreeLatitude` in `TileRepository`).
|
||||
- **Consumed by**: TileDownloader, RegionProcessing, RouteManagement, WebApi
|
||||
|
||||
### Component: TileDownloader
|
||||
@@ -139,6 +139,13 @@
|
||||
- **Purpose**: Stateless geospatial utility functions (coordinate math, distance, bearing)
|
||||
- **Consumed by**: TileDownloader, RegionProcessing, RouteManagement
|
||||
|
||||
### Common/Enums
|
||||
|
||||
- **Directory**: `SatelliteProvider.Common/Enums/`
|
||||
- **Purpose**: Domain enums shared across layers (`RegionStatus`, `RoutePointType`, `TileSource`) plus their explicit wire-value converters when persistence requires snake_case strings (`TileSourceConverter`). Converter classes belong here — not in DataAccess — because they encode a domain-level vocabulary that must be visible to every component.
|
||||
- **Consumed by**: DataAccess (entity defaults, type handler registration), TileDownloader (sets `TileEntity.Source` via `TileSourceConverter.ToWireValue`), Tests
|
||||
- **Important constraint**: Dapper's `SqlMapper.TypeHandler<TEnum>` is bypassed for enum reads (Dapper issue #259 — see `_docs/LESSONS.md` L-001). For any new enum that must round-trip through a database column, prefer the `string`-on-entity + `Enum`-at-API-boundary pattern with a converter class in this folder. Do NOT register a `TypeHandler<TEnum>` and assume it will be honored on reads.
|
||||
|
||||
## Allowed Dependencies (layering)
|
||||
|
||||
| Layer | Components | May import from (compile-time ProjectReferences) |
|
||||
|
||||
@@ -23,7 +23,7 @@ Runs DbUp-based SQL migrations against PostgreSQL on application startup. Ensure
|
||||
## Consumers
|
||||
- `Program.cs` — instantiated directly (not via DI) and called during startup. If migration fails, the application throws and does not start.
|
||||
|
||||
## Migrations (11 scripts)
|
||||
## Migrations (13 scripts)
|
||||
1. `001_CreateTilesTable.sql`
|
||||
2. `002_CreateRegionsTable.sql`
|
||||
3. `003_CreateIndexes.sql`
|
||||
@@ -35,6 +35,8 @@ Runs DbUp-based SQL migrations against PostgreSQL on application startup. Ensure
|
||||
9. `009_AddGeofencePolygonIndex.sql`
|
||||
10. `010_AddTilesZipToRoutes.sql`
|
||||
11. `011_AddTileCoordinates.sql`
|
||||
12. `012_DropTileVersionConstraint.sql` — drops the legacy 5-col `(latitude, longitude, tile_zoom, tile_size_meters, version)` unique index, replaces with 4-col `idx_tiles_unique_location` (preparation for AZ-484).
|
||||
13. `013_AddTileSourceAndCapturedAt.sql` — AZ-484 multi-source tile storage. Transactional. Adds `source` (VARCHAR(32) NOT NULL DEFAULT 'google_maps') and `captured_at` (TIMESTAMP NOT NULL) columns; backfills existing rows with `source='google_maps'`, `captured_at=created_at`; drops `idx_tiles_unique_location` and creates 5-col `idx_tiles_unique_location_source` on `(latitude, longitude, tile_zoom, tile_size_meters, source)`. Idempotent against partial replays.
|
||||
|
||||
## Configuration
|
||||
Receives connection string directly as constructor parameter.
|
||||
|
||||
@@ -9,7 +9,9 @@ Database entity classes that map directly to PostgreSQL tables via Dapper. Prope
|
||||
Maps to `tiles` table.
|
||||
- `Id` (Guid), `TileZoom` (int), `TileX` (int), `TileY` (int)
|
||||
- `Latitude`, `Longitude` (double), `TileSizeMeters` (double), `TileSizePixels` (int)
|
||||
- `ImageType` (string), `MapsVersion` (string?), `Version` (int)
|
||||
- `ImageType` (string), `MapsVersion` (string?), `Version` (int) — `MapsVersion`/`Version` are vestigial post-AZ-484 (kept nullable for backward compatibility; no longer part of the unique key)
|
||||
- `Source` (string) — AZ-484 producer wire value, defaults to `TileSourceConverter.GoogleMapsWireValue` (`"google_maps"`). Stored as plain string (not the `TileSource` enum) due to Dapper issue #259 — see `_docs/LESSONS.md` L-001. Convert via `SatelliteProvider.Common.Enums.TileSourceConverter.{ToWireValue,FromWireValue}`.
|
||||
- `CapturedAt` (DateTime, UTC) — AZ-484 imagery acquisition timestamp; drives the most-recent-across-sources selection.
|
||||
- `FilePath` (string), `CreatedAt`, `UpdatedAt` (DateTime)
|
||||
|
||||
### RegionEntity
|
||||
|
||||
@@ -7,26 +7,31 @@ Dapper-based repository for the `tiles` table. Handles CRUD operations and spati
|
||||
|
||||
### ITileRepository (interface)
|
||||
- `GetByIdAsync(Guid id) → Task<TileEntity?>`
|
||||
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds tile by slippy map coordinates, returns latest version
|
||||
- `FindExistingTileAsync(double lat, double lon, double tileSizeMeters, int zoomLevel, int version) → Task<TileEntity?>`: fuzzy coordinate match (tolerance: 0.0001° lat/lon, 1m tile size)
|
||||
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileEntity>>`: spatial bounding box query with expanded range to cover tile edges
|
||||
- `InsertAsync(TileEntity tile) → Task<Guid>`: upsert — `ON CONFLICT (latitude, longitude, tile_zoom, tile_size_meters, version) DO UPDATE` file_path, tile_x, tile_y, updated_at
|
||||
- `UpdateAsync(TileEntity tile) → Task<int>`
|
||||
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds the most-recent tile across all sources for the given slippy coordinates. Selection rule: `ORDER BY captured_at DESC, updated_at DESC, id DESC LIMIT 1` (AZ-484 v1.0.0 contract).
|
||||
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileEntity>>`: spatial bounding box query (expanded by 2 × tile size to cover edges); applies `DISTINCT ON (latitude, longitude, tile_zoom, tile_size_meters)` per AZ-484 to return at most one row per cell — the most-recent across sources — preserving the historical caller-facing order `latitude DESC, longitude ASC`.
|
||||
- `InsertAsync(TileEntity tile) → Task<Guid>`: per-source UPSERT — `ON CONFLICT (latitude, longitude, tile_zoom, tile_size_meters, source) DO UPDATE file_path, tile_x, tile_y, captured_at, updated_at` (AZ-484 5-column unique key).
|
||||
- `UpdateAsync(TileEntity tile) → Task<int>`: full row update by `id` including `source` and `captured_at`.
|
||||
- `DeleteAsync(Guid id) → Task<int>`
|
||||
|
||||
### TileRepository (implementation)
|
||||
Constructs a new `NpgsqlConnection` per method call (no connection pooling at the repository level; Npgsql pools connections internally).
|
||||
Constructs a new `NpgsqlConnection` per method call (no connection pooling at the repository level; Npgsql pools connections internally). Logs a `Slow GetTilesByRegionAsync` warning when the region query exceeds 500 ms.
|
||||
|
||||
## Internal Logic
|
||||
- `GetTilesByRegionAsync` calculates a bounding box by expanding the requested region by 2 × tile size to ensure edge tiles are included. Uses meters-to-degrees approximation (111,000 m/degree latitude, adjusted for longitude).
|
||||
- `InsertAsync` uses an upsert pattern to handle duplicate tile downloads gracefully.
|
||||
- `GetByTileCoordinatesAsync` orders by `version DESC` and takes the latest.
|
||||
- `GetTilesByRegionAsync` calculates a bounding box by expanding the requested region by 2 × tile size to ensure edge tiles are included. Uses meters-to-degrees approximation via `GeoUtils` (post-AZ-377 — single source of truth for Earth constants).
|
||||
- `InsertAsync` uses a per-source UPSERT pattern keyed on the 5-column unique index `idx_tiles_unique_location_source` (created by migration 013). Two producers (e.g., `google_maps` + `uav`) coexist for the same cell; same-source re-insert refreshes `captured_at` and `updated_at`.
|
||||
- `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync` apply the AZ-484 selection rule: most-recent across sources, deterministic tie-break on `(captured_at DESC, updated_at DESC, id DESC)`.
|
||||
- `TileEntity.Source` is a plain `string` storing the snake_case wire value (`'google_maps'` | `'uav'`); enum<->wire conversion happens via `SatelliteProvider.Common.Enums.TileSourceConverter`. This avoids Dapper issue #259 (TypeHandler<T> bypass for enum reads — see `_docs/LESSONS.md` L-001).
|
||||
- `FindExistingTileAsync` was removed by AZ-376 (see `_docs/04_refactoring/03-code-quality-refactoring/`).
|
||||
|
||||
## Dependencies
|
||||
- NuGet: `Dapper`, `Npgsql`
|
||||
- `SatelliteProvider.DataAccess.Models.TileEntity`
|
||||
- `SatelliteProvider.Common.GeoUtils` (Earth constants / meters-to-degrees)
|
||||
- `Microsoft.Extensions.Logging`
|
||||
|
||||
## Contract
|
||||
Implements the frozen v1.0.0 contract `_docs/02_document/contracts/data-access/tile-storage.md`. Schema invariants Inv-1..Inv-5 are enforced here (UPSERT key, selection rule, source value space).
|
||||
|
||||
## Consumers
|
||||
- `TileService` — all read/write operations
|
||||
- `Program.cs` (ServeTile, GetTileByLatLon handlers) — `GetByTileCoordinatesAsync`, `InsertAsync`
|
||||
|
||||
@@ -9,9 +9,9 @@ Orchestrates tile downloading and persistence. Bridges the downloader (Google Ma
|
||||
|
||||
### TileService (implements ITileService)
|
||||
- `DownloadAndStoreTilesAsync(double lat, double lon, double sizeMeters, int zoomLevel, CancellationToken) → Task<List<TileMetadata>>`:
|
||||
1. Queries existing tiles in the region from the repository (latest per `(latitude, longitude, zoom_level, tile_size_meters)` post-AZ-357)
|
||||
1. Queries existing tiles in the region from the repository — most-recent across sources per `(latitude, longitude, tile_zoom, tile_size_meters)` (AZ-484 selection rule applied by `TileRepository.GetTilesByRegionAsync` via `DISTINCT ON`)
|
||||
2. Calls `ISatelliteDownloader.GetTilesWithMetadataAsync` with existing tiles to skip
|
||||
3. Creates `TileEntity` for each newly downloaded tile and inserts via repository (upsert)
|
||||
3. Creates `TileEntity` for each newly downloaded tile and inserts via repository (per-source UPSERT keyed on `(latitude, longitude, tile_zoom, tile_size_meters, source)`)
|
||||
4. Returns combined list of existing + new tile metadata
|
||||
- `GetTileAsync(Guid id) → Task<TileMetadata?>`: single tile lookup
|
||||
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileMetadata>>`: query tiles in a region
|
||||
@@ -20,7 +20,8 @@ Orchestrates tile downloading and persistence. Bridges the downloader (Google Ma
|
||||
|
||||
## Internal Logic
|
||||
- New rows write `Version = null` and `MapsVersion = null` (post-AZ-357 / AZ-373); the `version` and `maps_version` columns are retained for backward compatibility with pre-existing rows
|
||||
- `MapToMetadata(TileEntity) → TileMetadata`: entity-to-DTO mapping (static helper); `MapsVersion` is no longer projected onto `TileMetadata` / `DownloadTileResponse`
|
||||
- AZ-484: `BuildTileEntity` stamps every newly downloaded row with `Source = TileSourceConverter.ToWireValue(TileSource.GoogleMaps)` (wire value `"google_maps"`) and `CapturedAt = DateTime.UtcNow`. The Google Maps download path is the only producer of `'google_maps'` rows; UAV ingestion (separate task) is the only producer of `'uav'` rows.
|
||||
- `MapToMetadata(TileEntity) → TileMetadata`: entity-to-DTO mapping (static helper); `MapsVersion` is no longer projected onto `TileMetadata` / `DownloadTileResponse`. `Source` and `CapturedAt` are not currently projected to the public DTO (no API contract change observable for AZ-484).
|
||||
- `TileSizePixels` sourced from `MapConfig.TileSizePixels` (default 256, post-AZ-371); image type fixed at `"jpg"`
|
||||
- `IMemoryCache` keyed by `(z, x, y)` with 1h absolute / 30min sliding expiration; populated on first hit and on downloader fallback
|
||||
|
||||
@@ -29,6 +30,7 @@ Orchestrates tile downloading and persistence. Bridges the downloader (Google Ma
|
||||
- `ITileRepository`
|
||||
- `IMemoryCache` (registered by `AddTileDownloader()`)
|
||||
- `SatelliteProvider.Common.DTO` — GeoPoint, TileMetadata, TileBytes
|
||||
- `SatelliteProvider.Common.Enums` — `TileSource`, `TileSourceConverter` (AZ-484)
|
||||
- `SatelliteProvider.DataAccess.Models` — TileEntity
|
||||
|
||||
## Consumers
|
||||
|
||||
@@ -11,6 +11,7 @@ Console application that runs end-to-end integration tests against a live API in
|
||||
- `BasicRouteTests` — route creation with intermediate points
|
||||
- `ComplexRouteTests` — routes with geofencing
|
||||
- `ExtendedRouteTests` — routes with `requestMaps: true` and tile ZIP creation
|
||||
- `MigrationTests` — direct PostgreSQL schema/index validation (no HTTP). AZ-484 cycle added: `NewUniqueConstraintIncludesSourceColumn_AZ484_AC1`, `BackfillUpdateAssignsGoogleMapsAndCapturedAt_AZ484_AC4`, `MultiSourceInsertCoexistsUnderNewIndex_AZ484_AC1`, `MostRecentAcrossSourcesSelection_AZ484_AC2`, `SameSourceUpsertReplacesPreviousRow_AZ484_AC3` (latter four use temp tables to keep production data untouched).
|
||||
|
||||
### Supporting Classes
|
||||
- `Models.cs` — HTTP response DTOs for deserialization
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# Ripple Log — Cycle 1 (AZ-484)
|
||||
|
||||
Generated by `document/SKILL.md` Task mode, Step 0.5 (Import-Graph Ripple).
|
||||
|
||||
## Cycle scope
|
||||
- AZ-484 — Multi-source tile storage schema
|
||||
|
||||
## Changed source identifiers (Step 0)
|
||||
- `SatelliteProvider.DataAccess.Migrations.013_AddTileSourceAndCapturedAt.sql` (DDL — not a C# import target)
|
||||
- `SatelliteProvider.Common.Enums.TileSource` (new)
|
||||
- `SatelliteProvider.Common.Enums.TileSourceConverter` (new)
|
||||
- `SatelliteProvider.DataAccess.Models.TileEntity` (added `Source`, `CapturedAt`; `Source` typed as `string` per L-001 workaround)
|
||||
- `SatelliteProvider.DataAccess.Repositories.TileRepository` (`ColumnList`, `GetByTileCoordinatesAsync`, `GetTilesByRegionAsync`, `InsertAsync`, `UpdateAsync`)
|
||||
- `SatelliteProvider.DataAccess.TypeHandlers.EnumStringTypeHandler` (registration list — net-zero after the L-001 pivot)
|
||||
- `SatelliteProvider.Services.TileDownloader.TileService.BuildTileEntity` (stamps `Source` + `CapturedAt`)
|
||||
|
||||
## Reverse-dependency analysis (Step 0.5)
|
||||
Tooling: `Grep` over the C# tree for `using SatelliteProvider.Common.Enums`, `TileSource`, `TileEntity`, `ITileRepository` references.
|
||||
|
||||
### Direct importers of the changed identifiers (already in the AZ-484 changeset)
|
||||
- `SatelliteProvider.DataAccess/Models/TileEntity.cs` — imports `Common.Enums` (uses `TileSourceConverter.GoogleMapsWireValue` const for the field default)
|
||||
- `SatelliteProvider.Services.TileDownloader/TileService.cs` — imports `Common.Enums` (calls `TileSourceConverter.ToWireValue(TileSource.GoogleMaps)`)
|
||||
- `SatelliteProvider.Tests/TileServiceTests.cs`, `TileSourceConverterTests.cs`, `RepositoryRefactorTests.cs`, `EnumStringTypeHandlerTests.cs`
|
||||
- `SatelliteProvider.IntegrationTests/MigrationTests.cs`
|
||||
|
||||
All of the above were edited and their tests were re-run as part of AZ-484 itself; no documentation ripple is required for them.
|
||||
|
||||
### Indirect importers requiring doc refresh
|
||||
The only consumers that touch `TileEntity` *as a type* (not as a column-aware contract) are:
|
||||
|
||||
- `SatelliteProvider.Services.TileDownloader/GoogleMapsDownloaderV2.cs` — accepts `IEnumerable<TileEntity>` for the cache-skip parameter. **No doc ripple**: the downloader treats `TileEntity` as an opaque cache token; the new `Source` / `CapturedAt` columns are not accessed by the downloader.
|
||||
- `Program.cs` (WebApi) — calls `ITileRepository.GetByTileCoordinatesAsync` and `InsertAsync`. **No doc ripple to `api_program.md`**: the endpoint contract did not change (response DTO `TileMetadata` is unaffected by AZ-484; `Source` and `CapturedAt` are not currently projected to the public API).
|
||||
- `RegionService` / `RegionProcessingService` — call `ITileService` methods, not `ITileRepository` directly. **No doc ripple**: the `ITileService` method signatures are unchanged; the most-recent-across-sources read rule is fully enforced inside `TileRepository` and is invisible to region/route consumers.
|
||||
|
||||
### Refreshed-anyway docs (within the immediate scope)
|
||||
The following docs were refreshed during this Update Docs step because they describe the storage layer or its direct boundary, and AZ-484 changed the substantive content even though import graphs alone would not have triggered them:
|
||||
|
||||
- `_docs/02_document/modules/dataaccess_tile_repository.md` — interface signatures (`FindExistingTileAsync` removal, AZ-484 selection rule, 5-col UPSERT key); changed by direct edit (in scope)
|
||||
- `_docs/02_document/modules/dataaccess_models.md` — `TileEntity` field list (`Source` as `string` + L-001 note, `CapturedAt`); changed by direct edit (in scope)
|
||||
- `_docs/02_document/modules/dataaccess_migrator.md` — migration count 11 → 13, entries for 012 and 013; changed by direct edit (in scope)
|
||||
- `_docs/02_document/modules/services_tile_service.md` — `DownloadAndStoreTilesAsync` selection rule + UPSERT semantics, `BuildTileEntity` source stamping, `Common.Enums` dependency; changed by direct edit (in scope)
|
||||
- `_docs/02_document/modules/tests_integration.md` — `MigrationTests` AZ-484 method list; changed by direct edit (in scope)
|
||||
- `_docs/02_document/components/03_tile_downloader/description.md` — upstream dependency list (`Common.Enums`), tile-metadata caching strategy now reads "append-by-source per cell"; changed by direct edit (in scope)
|
||||
- `_docs/02_document/components/02_data_access/description.md` — `ITileRepository` table (FindExistingTileAsync row removed; AZ-484 annotations on Get/Insert), Queries table (UPSERT key now 5-col + source), Caveats (UPSERT semantics, L-001 note, contract pointer); changed by direct edit (in scope)
|
||||
- `_docs/02_document/data_model.md` — `tiles` table columns (`source`, `captured_at` added; `version` / `maps_version` flagged vestigial), Indexes section (replaced unique key entry, added selection-rule + UPSERT contract notes), Migration History (012, 013); changed by direct edit (in scope)
|
||||
- `_docs/02_document/module-layout.md` — Common Public API list (`TileSourceConverter` already present; updated DataAccess "Imports from" 5 → 6 sites including `Models/TileEntity`; new Common/Enums subsection with the L-001 guidance); changed by direct edit (in scope)
|
||||
- `_docs/02_document/architecture.md`, `_docs/02_document/glossary.md`, `_docs/02_document/contracts/data-access/tile-storage.md` — already updated during the AZ-484 implementation batch (see batch_25_cycle1_report.md)
|
||||
|
||||
## Heuristic mode flag
|
||||
Not used. Direct `Grep` over the C# tree was sufficient — no parser failure, no fall-back to directory-proximity scanning.
|
||||
|
||||
## Summary
|
||||
- Direct importers in AZ-484 changeset: 7 (already updated as part of the implementation)
|
||||
- Indirect importers requiring NO doc ripple: 3 (GoogleMapsDownloaderV2, Program.cs, RegionService chain)
|
||||
- In-scope refresh-anyway docs: 10 (listed above)
|
||||
- Out-of-scope ripple-only docs: 0
|
||||
@@ -41,3 +41,12 @@
|
||||
**Load**: 1 request
|
||||
**Expected**: Route created (with interpolation) within 5s
|
||||
**Pass criterion**: HTTP 200 response within 5000ms; totalPoints > 20
|
||||
|
||||
## PT-07: GetTilesByRegionAsync Latency Post-AZ-484 (multi-source baseline)
|
||||
|
||||
**Trigger**: TileRepository.GetTilesByRegionAsync exercised via POST /api/satellite/request (200m region, zoom 18) against a tiles table seeded with the pre-AZ-484 data shape (single-source rows backfilled to source='google_maps').
|
||||
**Load**: 1 request, repeated 20 times to get a stable distribution.
|
||||
**Expected**: 95th-percentile latency must not regress more than 10% vs the pre-AZ-484 baseline measured against PT-03 / PT-04. The new 5-column unique index `idx_tiles_unique_location_source` covers the same `(latitude, longitude, tile_zoom, tile_size_meters)` filter columns as the pre-AZ-484 4-column index, so no regression is expected.
|
||||
**Pass criterion**: p95(GetTilesByRegionAsync) ≤ 1.10 × pre-AZ-484 p95 baseline.
|
||||
**Source**: AZ-484 NFR (Performance) — `_docs/02_tasks/done/AZ-484_multi_source_tile_storage.md` § Non-Functional Requirements.
|
||||
**Note**: This NFR is recorded for tracking. Active enforcement (running PT-07 against a real workload and comparing) is deferred to autodev Step 15 (Performance Test) when a baseline run is available. Until then, the integration test `MostRecentAcrossSourcesSelection_AZ484_AC2` provides correctness coverage for the new query shape.
|
||||
|
||||
@@ -26,6 +26,13 @@
|
||||
| S1 | Migrations run on startup | RS-02 | ✓ |
|
||||
| S2 | Queue rejects when full | RS-04, RL-02 | ✓ |
|
||||
| S3 | Failed regions marked failed | RS-03 | ✓ |
|
||||
| AZ-484 AC-1 | Schema accepts source + captured_at; multi-source rows coexist under 5-col unique index | `MultiSourceInsertCoexistsUnderNewIndex_AZ484_AC1`, `NewUniqueConstraintIncludesSourceColumn_AZ484_AC1` (integration) | ✓ |
|
||||
| AZ-484 AC-2 | Read returns most-recent across sources | `MostRecentAcrossSourcesSelection_AZ484_AC2` (integration) | ✓ |
|
||||
| AZ-484 AC-3 | Same-source UPSERT collapses to one row with refreshed captured_at | `SameSourceUpsertReplacesPreviousRow_AZ484_AC3` (integration) | ✓ |
|
||||
| AZ-484 AC-4 | Migration 013 backfill leaves no orphans (count preserved, source='google_maps', captured_at=created_at) | `BackfillUpdateAssignsGoogleMapsAndCapturedAt_AZ484_AC4` (integration) | ✓ |
|
||||
| AZ-484 AC-5 | Google Maps download path stamps Source='google_maps' (wire) + CapturedAt UTC | `BuildTileEntity_SetsGoogleMapsSourceAndUtcCapturedAt_AZ484_AC5` (unit) | ✓ |
|
||||
| AZ-484 AC-6 | Existing region/route flows unchanged post-T1 (200 unit + smoke baseline preserved) | Full unit suite (213 tests) + integration smoke scenarios BT-01..BT-12 | ✓ |
|
||||
| AZ-484 AC-7 | Vision + contract docs amended (architecture.md, glossary.md, module-layout.md, tile-storage.md frozen v1.0.0) | doc-state AC; verified by `monorepo-document` reviews | ✓ |
|
||||
|
||||
## Restrictions → Test Mapping
|
||||
|
||||
@@ -40,6 +47,13 @@
|
||||
| Max ZIP 50 MB | RL-01 | ✓ |
|
||||
| No authentication | SEC-01 through SEC-04 (all requests accepted without auth) | ✓ |
|
||||
|
||||
## NFRs → Test Mapping
|
||||
|
||||
| NFR | Source | Tests | Coverage |
|
||||
|-----|--------|-------|----------|
|
||||
| AZ-484 Perf — `GetTilesByRegionAsync` p95 ≤ 1.10 × pre-AZ-484 baseline | AZ-484 task spec § Non-Functional Requirements | PT-07 (recorded; active perf comparison deferred to Step 15) | ◐ recorded |
|
||||
| AZ-484 Compatibility — no public HTTP response field added/removed; vestigial `maps_version`/`version` columns preserved (nullable) | AZ-484 task spec § Non-Functional Requirements | Existing integration suite (no API contract change observable); BT-01 / region status responses verify response shape | ✓ |
|
||||
|
||||
## Coverage Summary
|
||||
|
||||
| Category | Total Tests | ACs Covered | Restrictions Covered |
|
||||
@@ -50,4 +64,5 @@
|
||||
| Resilience | 6 | 4 | 3 |
|
||||
| Security | 4 | — | 1 |
|
||||
| Resource Limits | 4 | 3 | 4 |
|
||||
| **Total** | **37** | **22/22 (100%)** | **8/8 (100%)** |
|
||||
| Cycle 1 — AZ-484 (integration + unit) | 6 | 7/7 | — |
|
||||
| **Total** | **43** | **29/29 (100%)** | **8/8 (100%)** |
|
||||
|
||||
Reference in New Issue
Block a user