Files
satellite-provider/_docs/02_document/components/02_data_access/description.md
T
Oleksandr Bezdieniezhnykh 61612044fb [AZ-503] [AZ-504] Cycle 5 Steps 11-15 sync
Wrap up cycle 5 verification + documentation:
- Steps 10/11 wrap-up reports (implementation_completeness +
  implementation_report) for the AZ-503-foundation + AZ-504 batch.
- Step 12 test-spec sync: AZ-503-foundation/AZ-504 ACs appended;
  AZ-505 deferred ACs recorded.
- Step 13 update-docs: architecture, data-model, glossary, module-
  layout, uav-tile-upload contract (v1.1.0), DataAccess + Services
  + Tests module docs synced; new common_uuidv5.md module doc.
- Step 14 security audit: PASS_WITH_WARNINGS; 0 new Critical/High;
  2 new Low informational (F1 flightId provenance, F2 pgcrypto
  deploy gap).
- Step 15 performance test: PASS_WITH_INFRA_WARNINGS; PT-08
  passed twice (AZ-504 fix verified); PT-01/02 failed due to
  recurring local Docker/colima DNS cold-start (not an app
  regression). Cycle-3 perf-harness leftover stays OPEN with
  replay #5 documented.
- Autodev state moved to Step 16 (Deploy).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 18:01:27 +03:00

6.9 KiB
Raw Blame History

DataAccess (Persistence)

1. High-Level Overview

Purpose: Database persistence layer providing Dapper-based repositories for tiles, regions, routes, and route points, plus DbUp-driven schema migrations.

Architectural Pattern: Repository pattern with raw SQL (Dapper)

Upstream dependencies: None at project level (uses Microsoft.Extensions abstractions from NuGet)

Downstream consumers: TileDownloader (TileRepository), RegionProcessing (RegionRepository), RouteManagement (RouteRepository, RegionRepository), WebApi (TileRepository for ServeTile)

2. Internal Interfaces

Interface: ITileRepository

Method Input Output Async Error Types
GetByIdAsync Guid TileEntity? Yes NpgsqlException
GetByTileCoordinatesAsync zoom, x, y TileEntity? (most-recent across sources, AZ-484) Yes NpgsqlException
GetTilesByRegionAsync lat, lon, sizeM, zoom IEnumerable<TileEntity> (one row per cell via DISTINCT ON, AZ-484) Yes NpgsqlException
InsertAsync TileEntity Guid (per-source UPSERT, AZ-484) Yes NpgsqlException
UpdateAsync TileEntity int Yes NpgsqlException
DeleteAsync Guid int Yes NpgsqlException

FindExistingTileAsync was removed by AZ-376 (replaced by direct cell lookups through GetByTileCoordinatesAsync + GetTilesByRegionAsync).

Interface: IRegionRepository

Method Input Output Async Error Types
GetByIdAsync Guid RegionEntity? Yes NpgsqlException
GetByStatusAsync string IEnumerable<RegionEntity> Yes NpgsqlException
InsertAsync RegionEntity Guid Yes NpgsqlException
UpdateAsync RegionEntity int Yes NpgsqlException
DeleteAsync Guid int Yes NpgsqlException

Interface: IRouteRepository

Method Input Output Async Error Types
GetByIdAsync Guid RouteEntity? Yes NpgsqlException
GetRoutePointsAsync Guid routeId IEnumerable<RoutePointEntity> Yes NpgsqlException
InsertRouteAsync RouteEntity Guid Yes NpgsqlException
InsertRoutePointsAsync IEnumerable<RoutePointEntity> void Yes NpgsqlException
UpdateRouteAsync RouteEntity int Yes NpgsqlException
LinkRouteToRegionAsync routeId, regionId, isGeofence, polygonIndex void Yes NpgsqlException
GetRegionIdsByRouteAsync Guid routeId IEnumerable<Guid> Yes NpgsqlException
GetGeofenceRegionIdsByRouteAsync Guid routeId IEnumerable<Guid> Yes NpgsqlException
GetGeofenceRegionsByPolygonAsync Guid routeId Dictionary<int, List<Guid>> Yes NpgsqlException
GetRoutesWithPendingMapsAsync IEnumerable<RouteEntity> Yes NpgsqlException

Class: DatabaseMigrator

Method Input Output Async Error Types
RunMigrations bool No Exception

4. Data Access Patterns

Queries

Query Frequency Hot Path Index Needed
GetByTileCoordinatesAsync (tile lookup) Very High Yes (tile_zoom, tile_x, tile_y)
GetTilesByRegionAsync (spatial) High Yes (latitude, longitude, tile_zoom)
InsertAsync (tile per-source upsert) High Yes Composite unique on (tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid)) (AZ-503: idx_tiles_unique_identity; supersedes the AZ-484 float-based idx_tiles_unique_location_source)
GetByStatusAsync (region polling) Medium No (status)
GetRoutesWithPendingMapsAsync Low No (request_maps, maps_ready)

Storage Estimates

Table Est. Row Count (1yr) Row Size Growth Rate
tiles ~100K1M (depends on usage) ~200B Variable
regions ~10K50K ~150B Proportional to tile requests
routes ~1K5K ~200B Low
route_points ~50K500K ~100B Proportional to routes
route_regions ~10K100K ~50B Proportional to routes

5. Implementation Details

State Management: Stateless — each repository creates a new Npgsql connection per method call. Npgsql handles internal connection pooling.

Key Dependencies:

Library Version Purpose
Dapper 2.1.35 Micro-ORM for SQL queries
Npgsql 9.0.2 PostgreSQL ADO.NET driver
dbup-postgresql 6.0.3 Schema migration runner

Error Handling: Exceptions propagate to callers. No retry logic at the repository level.

7. Caveats & Edge Cases

  • Repository interfaces are defined in this project (not in Common), creating a dependency from Services to DataAccess
  • Column mapping uses SQL aliases (tile_zoom as TileZoom) rather than Dapper attribute mapping
  • TileRepository.InsertAsync uses an integer-only, flight-aware UPSERT pattern (AZ-503; supersedes the AZ-484 5-column float-based UPSERT). Same-source same-flight re-inserts overwrite and refresh captured_at/location_hash/content_sha256; different sources or different flights at the same cell coexist as separate rows. id is intentionally NOT overwritten on conflict so it stays deterministic per AZ-503 AC-2.
  • TileEntity.Source is stored as a plain string (not the TileSource enum) due to Dapper issue #259 — see _docs/LESSONS.md L-001. Conversion happens via SatelliteProvider.Common.Enums.TileSourceConverter
  • AZ-503 deterministic identity: id is Uuidv5(TileNamespace, "{z}/{x}/{y}/{source}/{flight_id or zero-uuid}") and location_hash is Uuidv5(TileNamespace, "{z}/{x}/{y}"). The cross-repo TileNamespace constant lives in SatelliteProvider.Common.Utils.Uuidv5 and MUST match gps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE.
  • The frozen v1.0.0 tile-storage contract (_docs/02_document/contracts/data-access/) is the AZ-484-era spec for read-side selection invariants; the AZ-503 write-side schema change is documented inline in dataaccess_models.md and dataaccess_tile_repository.md. A v2.0.0 contract bump is deferred to AZ-505 (when the POST /api/satellite/tiles/inventory endpoint freezes the new identity surface for external consumers).
  • No soft-delete; DeleteAsync is a hard delete

8. Dependency Graph

Must be implemented after: nothing (parallel with Common) Can be implemented in parallel with: Common Blocks: TileDownloader, RegionProcessing, RouteManagement, WebApi

9. Logging Strategy

Log Level When Example
INFO Migration start/complete Starting database migrations...
ERROR Migration failure Database migration failed

Structured logging via ILogger<T>. Logger injected but rarely used in repositories.