Files
satellite-provider/_docs/02_document/components/02_data_access/description.md
T
Oleksandr Bezdieniezhnykh 51b572108a
ci/woodpecker/push/01-test Pipeline was successful
ci/woodpecker/push/02-build-push Pipeline was successful
[AZ-484] Cycle 1 Steps 12-16: docs, security, perf, deploy report
Captures the post-implementation autodev gates for AZ-484 multi-source
tile storage:

- Step 12 (Test-Spec Sync): added 7 AC rows (AZ-484 AC-1..AC-7) and a
  PT-07 NFR row to traceability-matrix.md; added PT-07 scenario to
  performance-tests.md.
- Step 13 (Update Docs): refreshed data_model.md (tiles columns +
  indexes + selection rule + UPSERT contract + migrations 012/013),
  module-layout.md (Common/Enums section with L-001 guidance,
  DataAccess imports-from now lists 6 sites), 6 module / component
  docs to reflect the new repo signatures, source/captured_at fields,
  and Dapper enum bypass workaround. ripple_log_cycle1.md records
  zero out-of-scope ripple.
- Step 14 (Security Audit): PASS_WITH_WARNINGS - 0 Critical, 0 High,
  5 Medium, 5 Low. AZ-484 itself added zero new findings. Hardening
  items (Postgres default creds, .env in build context, GMaps key
  rotation, ASP.NET Core 8.0.21 -> 8.0.25, rate limiter) recorded
  for separate tickets.
- Step 15 (Performance Test): all PT-01..PT-07 scenarios Unverified
  (non-blocking); PT-07 baseline-comparison harness deferred to a
  leftover for next cycle.
- Step 16 (Deploy): cycle deploy report covering migration safety,
  rollback path, post-deploy verification, security caveats.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 10:03:05 +03:00

5.9 KiB
Raw Blame History

DataAccess (Persistence)

1. High-Level Overview

Purpose: Database persistence layer providing Dapper-based repositories for tiles, regions, routes, and route points, plus DbUp-driven schema migrations.

Architectural Pattern: Repository pattern with raw SQL (Dapper)

Upstream dependencies: None at project level (uses Microsoft.Extensions abstractions from NuGet)

Downstream consumers: TileDownloader (TileRepository), RegionProcessing (RegionRepository), RouteManagement (RouteRepository, RegionRepository), WebApi (TileRepository for ServeTile)

2. Internal Interfaces

Interface: ITileRepository

Method Input Output Async Error Types
GetByIdAsync Guid TileEntity? Yes NpgsqlException
GetByTileCoordinatesAsync zoom, x, y TileEntity? (most-recent across sources, AZ-484) Yes NpgsqlException
GetTilesByRegionAsync lat, lon, sizeM, zoom IEnumerable<TileEntity> (one row per cell via DISTINCT ON, AZ-484) Yes NpgsqlException
InsertAsync TileEntity Guid (per-source UPSERT, AZ-484) Yes NpgsqlException
UpdateAsync TileEntity int Yes NpgsqlException
DeleteAsync Guid int Yes NpgsqlException

FindExistingTileAsync was removed by AZ-376 (replaced by direct cell lookups through GetByTileCoordinatesAsync + GetTilesByRegionAsync).

Interface: IRegionRepository

Method Input Output Async Error Types
GetByIdAsync Guid RegionEntity? Yes NpgsqlException
GetByStatusAsync string IEnumerable<RegionEntity> Yes NpgsqlException
InsertAsync RegionEntity Guid Yes NpgsqlException
UpdateAsync RegionEntity int Yes NpgsqlException
DeleteAsync Guid int Yes NpgsqlException

Interface: IRouteRepository

Method Input Output Async Error Types
GetByIdAsync Guid RouteEntity? Yes NpgsqlException
GetRoutePointsAsync Guid routeId IEnumerable<RoutePointEntity> Yes NpgsqlException
InsertRouteAsync RouteEntity Guid Yes NpgsqlException
InsertRoutePointsAsync IEnumerable<RoutePointEntity> void Yes NpgsqlException
UpdateRouteAsync RouteEntity int Yes NpgsqlException
LinkRouteToRegionAsync routeId, regionId, isGeofence, polygonIndex void Yes NpgsqlException
GetRegionIdsByRouteAsync Guid routeId IEnumerable<Guid> Yes NpgsqlException
GetGeofenceRegionIdsByRouteAsync Guid routeId IEnumerable<Guid> Yes NpgsqlException
GetGeofenceRegionsByPolygonAsync Guid routeId Dictionary<int, List<Guid>> Yes NpgsqlException
GetRoutesWithPendingMapsAsync IEnumerable<RouteEntity> Yes NpgsqlException

Class: DatabaseMigrator

Method Input Output Async Error Types
RunMigrations bool No Exception

4. Data Access Patterns

Queries

Query Frequency Hot Path Index Needed
GetByTileCoordinatesAsync (tile lookup) Very High Yes (tile_zoom, tile_x, tile_y)
GetTilesByRegionAsync (spatial) High Yes (latitude, longitude, tile_zoom)
InsertAsync (tile per-source upsert) High Yes Composite unique on (lat, lon, tile_zoom, tile_size_meters, source) (AZ-484: idx_tiles_unique_location_source)
GetByStatusAsync (region polling) Medium No (status)
GetRoutesWithPendingMapsAsync Low No (request_maps, maps_ready)

Storage Estimates

Table Est. Row Count (1yr) Row Size Growth Rate
tiles ~100K1M (depends on usage) ~200B Variable
regions ~10K50K ~150B Proportional to tile requests
routes ~1K5K ~200B Low
route_points ~50K500K ~100B Proportional to routes
route_regions ~10K100K ~50B Proportional to routes

5. Implementation Details

State Management: Stateless — each repository creates a new Npgsql connection per method call. Npgsql handles internal connection pooling.

Key Dependencies:

Library Version Purpose
Dapper 2.1.35 Micro-ORM for SQL queries
Npgsql 9.0.2 PostgreSQL ADO.NET driver
dbup-postgresql 6.0.3 Schema migration runner

Error Handling: Exceptions propagate to callers. No retry logic at the repository level.

7. Caveats & Edge Cases

  • Repository interfaces are defined in this project (not in Common), creating a dependency from Services to DataAccess
  • Column mapping uses SQL aliases (tile_zoom as TileZoom) rather than Dapper attribute mapping
  • TileRepository.InsertAsync uses a per-source UPSERT pattern (AZ-484); two producers (e.g., google_maps + uav) coexist as separate rows for the same cell, while same-source re-inserts overwrite and refresh captured_at
  • TileEntity.Source is stored as a plain string (not the TileSource enum) due to Dapper issue #259 — see _docs/LESSONS.md L-001. Conversion happens via SatelliteProvider.Common.Enums.TileSourceConverter
  • The frozen v1.0.0 tile-storage contract (_docs/02_document/contracts/data-access/) is the authoritative spec for all tiles invariants enforced here
  • No soft-delete; DeleteAsync is a hard delete

8. Dependency Graph

Must be implemented after: nothing (parallel with Common) Can be implemented in parallel with: Common Blocks: TileDownloader, RegionProcessing, RouteManagement, WebApi

9. Logging Strategy

Log Level When Example
INFO Migration start/complete Starting database migrations...
ERROR Migration failure Database migration failed

Structured logging via ILogger<T>. Logger injected but rarely used in repositories.