Wrap up cycle 5 verification + documentation: - Steps 10/11 wrap-up reports (implementation_completeness + implementation_report) for the AZ-503-foundation + AZ-504 batch. - Step 12 test-spec sync: AZ-503-foundation/AZ-504 ACs appended; AZ-505 deferred ACs recorded. - Step 13 update-docs: architecture, data-model, glossary, module- layout, uav-tile-upload contract (v1.1.0), DataAccess + Services + Tests module docs synced; new common_uuidv5.md module doc. - Step 14 security audit: PASS_WITH_WARNINGS; 0 new Critical/High; 2 new Low informational (F1 flightId provenance, F2 pgcrypto deploy gap). - Step 15 performance test: PASS_WITH_INFRA_WARNINGS; PT-08 passed twice (AZ-504 fix verified); PT-01/02 failed due to recurring local Docker/colima DNS cold-start (not an app regression). Cycle-3 perf-harness leftover stays OPEN with replay #5 documented. - Autodev state moved to Step 16 (Deploy). Co-authored-by: Cursor <cursoragent@cursor.com>
6.9 KiB
DataAccess (Persistence)
1. High-Level Overview
Purpose: Database persistence layer providing Dapper-based repositories for tiles, regions, routes, and route points, plus DbUp-driven schema migrations.
Architectural Pattern: Repository pattern with raw SQL (Dapper)
Upstream dependencies: None at project level (uses Microsoft.Extensions abstractions from NuGet)
Downstream consumers: TileDownloader (TileRepository), RegionProcessing (RegionRepository), RouteManagement (RouteRepository, RegionRepository), WebApi (TileRepository for ServeTile)
2. Internal Interfaces
Interface: ITileRepository
| Method | Input | Output | Async | Error Types |
|---|---|---|---|---|
GetByIdAsync |
Guid | TileEntity? |
Yes | NpgsqlException |
GetByTileCoordinatesAsync |
zoom, x, y | TileEntity? (most-recent across sources, AZ-484) |
Yes | NpgsqlException |
GetTilesByRegionAsync |
lat, lon, sizeM, zoom | IEnumerable<TileEntity> (one row per cell via DISTINCT ON, AZ-484) |
Yes | NpgsqlException |
InsertAsync |
TileEntity |
Guid (per-source UPSERT, AZ-484) | Yes | NpgsqlException |
UpdateAsync |
TileEntity |
int | Yes | NpgsqlException |
DeleteAsync |
Guid | int | Yes | NpgsqlException |
FindExistingTileAsync was removed by AZ-376 (replaced by direct cell lookups through GetByTileCoordinatesAsync + GetTilesByRegionAsync).
Interface: IRegionRepository
| Method | Input | Output | Async | Error Types |
|---|---|---|---|---|
GetByIdAsync |
Guid | RegionEntity? |
Yes | NpgsqlException |
GetByStatusAsync |
string | IEnumerable<RegionEntity> |
Yes | NpgsqlException |
InsertAsync |
RegionEntity |
Guid | Yes | NpgsqlException |
UpdateAsync |
RegionEntity |
int | Yes | NpgsqlException |
DeleteAsync |
Guid | int | Yes | NpgsqlException |
Interface: IRouteRepository
| Method | Input | Output | Async | Error Types |
|---|---|---|---|---|
GetByIdAsync |
Guid | RouteEntity? |
Yes | NpgsqlException |
GetRoutePointsAsync |
Guid routeId | IEnumerable<RoutePointEntity> |
Yes | NpgsqlException |
InsertRouteAsync |
RouteEntity |
Guid | Yes | NpgsqlException |
InsertRoutePointsAsync |
IEnumerable<RoutePointEntity> |
void | Yes | NpgsqlException |
UpdateRouteAsync |
RouteEntity |
int | Yes | NpgsqlException |
LinkRouteToRegionAsync |
routeId, regionId, isGeofence, polygonIndex | void | Yes | NpgsqlException |
GetRegionIdsByRouteAsync |
Guid routeId | IEnumerable<Guid> |
Yes | NpgsqlException |
GetGeofenceRegionIdsByRouteAsync |
Guid routeId | IEnumerable<Guid> |
Yes | NpgsqlException |
GetGeofenceRegionsByPolygonAsync |
Guid routeId | Dictionary<int, List<Guid>> |
Yes | NpgsqlException |
GetRoutesWithPendingMapsAsync |
— | IEnumerable<RouteEntity> |
Yes | NpgsqlException |
Class: DatabaseMigrator
| Method | Input | Output | Async | Error Types |
|---|---|---|---|---|
RunMigrations |
— | bool | No | Exception |
4. Data Access Patterns
Queries
| Query | Frequency | Hot Path | Index Needed |
|---|---|---|---|
| GetByTileCoordinatesAsync (tile lookup) | Very High | Yes | (tile_zoom, tile_x, tile_y) |
| GetTilesByRegionAsync (spatial) | High | Yes | (latitude, longitude, tile_zoom) |
| InsertAsync (tile per-source upsert) | High | Yes | Composite unique on (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)) (AZ-503: idx_tiles_unique_identity; supersedes the AZ-484 float-based idx_tiles_unique_location_source) |
| GetByStatusAsync (region polling) | Medium | No | (status) |
| GetRoutesWithPendingMapsAsync | Low | No | (request_maps, maps_ready) |
Storage Estimates
| Table | Est. Row Count (1yr) | Row Size | Growth Rate |
|---|---|---|---|
| tiles | ~100K–1M (depends on usage) | ~200B | Variable |
| regions | ~10K–50K | ~150B | Proportional to tile requests |
| routes | ~1K–5K | ~200B | Low |
| route_points | ~50K–500K | ~100B | Proportional to routes |
| route_regions | ~10K–100K | ~50B | Proportional to routes |
5. Implementation Details
State Management: Stateless — each repository creates a new Npgsql connection per method call. Npgsql handles internal connection pooling.
Key Dependencies:
| Library | Version | Purpose |
|---|---|---|
| Dapper | 2.1.35 | Micro-ORM for SQL queries |
| Npgsql | 9.0.2 | PostgreSQL ADO.NET driver |
| dbup-postgresql | 6.0.3 | Schema migration runner |
Error Handling: Exceptions propagate to callers. No retry logic at the repository level.
7. Caveats & Edge Cases
- Repository interfaces are defined in this project (not in Common), creating a dependency from Services to DataAccess
- Column mapping uses SQL aliases (
tile_zoom as TileZoom) rather than Dapper attribute mapping - TileRepository.InsertAsync uses an integer-only, flight-aware UPSERT pattern (AZ-503; supersedes the AZ-484 5-column float-based UPSERT). Same-source same-flight re-inserts overwrite and refresh
captured_at/location_hash/content_sha256; different sources or different flights at the same cell coexist as separate rows.idis intentionally NOT overwritten on conflict so it stays deterministic per AZ-503 AC-2. TileEntity.Sourceis stored as a plainstring(not theTileSourceenum) due to Dapper issue #259 — see_docs/LESSONS.mdL-001. Conversion happens viaSatelliteProvider.Common.Enums.TileSourceConverter- AZ-503 deterministic identity:
idisUuidv5(TileNamespace, "{z}/{x}/{y}/{source}/{flight_id or zero-uuid}")andlocation_hashisUuidv5(TileNamespace, "{z}/{x}/{y}"). The cross-repoTileNamespaceconstant lives inSatelliteProvider.Common.Utils.Uuidv5and MUST matchgps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE. - The frozen v1.0.0
tile-storagecontract (_docs/02_document/contracts/data-access/) is the AZ-484-era spec for read-side selection invariants; the AZ-503 write-side schema change is documented inline indataaccess_models.mdanddataaccess_tile_repository.md. A v2.0.0 contract bump is deferred to AZ-505 (when thePOST /api/satellite/tiles/inventoryendpoint freezes the new identity surface for external consumers). - No soft-delete;
DeleteAsyncis a hard delete
8. Dependency Graph
Must be implemented after: nothing (parallel with Common) Can be implemented in parallel with: Common Blocks: TileDownloader, RegionProcessing, RouteManagement, WebApi
9. Logging Strategy
| Log Level | When | Example |
|---|---|---|
| INFO | Migration start/complete | Starting database migrations... |
| ERROR | Migration failure | Database migration failed |
Structured logging via ILogger<T>. Logger injected but rarely used in repositories.