[AZ-284] Autodev baseline + testability refactor

Phase A baseline outputs from /autodev (Steps 1-5):
- Problem & solution docs (_docs/00_problem, _docs/01_solution)
- Codebase documentation (_docs/02_document) incl. architecture,
  module-layout, glossary, system-flows, baseline compliance scan
- Test specs (blackbox, performance, resilience, security, resource,
  traceability matrix)
- Test task decomposition (_docs/02_tasks/todo): AZ-285..AZ-290
- Testability refactor (_docs/04_refactoring/01-testability-refactoring):
  - TC-01 Move DownloadedTileInfoV2 + new ExistingTileInfo to Common.DTO
  - TC-02 Replace dead ISatelliteDownloader API with real signatures
  - TC-03 GoogleMapsDownloaderV2 implements ISatelliteDownloader
  - TC-04 TileService depends on ISatelliteDownloader (mockable)
  - TC-05 DI + endpoints use ISatelliteDownloader
- Test runner scripts (scripts/run-tests.sh, run-performance-tests.sh)
- Autodev state pointer (_docs/_autodev_state.md)

Prepares the codebase for AZ-285..AZ-290 unit/integration test work.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-10 04:44:08 +03:00
parent 25a644a9bf
commit b0fffa6d42
68 changed files with 4192 additions and 11 deletions
+88
View File
@@ -0,0 +1,88 @@
# Module: Api/Program.cs
## Purpose
Application entry point. Configures DI container, sets up middleware, defines minimal API endpoints, runs database migrations on startup, and starts background services.
## Public Interface
### API Endpoints
| Method | Route | Handler | Description |
|--------|-------|---------|-------------|
| GET | `/tiles/{z}/{x}/{y}` | `ServeTile` | Slippy map tile server with in-memory caching |
| GET | `/api/satellite/tiles/latlon` | `GetTileByLatLon` | Download single tile by lat/lon/zoom |
| GET | `/api/satellite/tiles/mgrs` | `GetSatelliteTilesByMgrs` | MGRS stub (returns empty) |
| POST | `/api/satellite/upload` | `UploadImage` | Image upload stub (returns `Success: false`) |
| POST | `/api/satellite/request` | `RequestRegion` | Queue region for async tile processing |
| GET | `/api/satellite/region/{id}` | `GetRegionStatus` | Get region processing status |
| POST | `/api/satellite/route` | `CreateRoute` | Create route with intermediate points |
| GET | `/api/satellite/route/{id}` | `GetRoute` | Get route with all points |
### Local Records (defined in Program.cs)
- `GetSatelliteTilesResponse`, `SatelliteTile` — MGRS response stubs
- `UploadImageRequest` — multipart form data request
- `SaveResult` — upload response stub
- `DownloadTileResponse` — tile download response
- `RequestRegionRequest` — region request body
- `ParameterDescriptionFilter` — Swagger operation filter
## Internal Logic
### DI Registration
1. Serilog configured from `appsettings.json`
2. Connection string extracted from `ConnectionStrings:DefaultConnection`
3. Config bindings: `MapConfig`, `StorageConfig`, `ProcessingConfig`
4. Singletons: repositories (`TileRepository`, `RegionRepository`, `RouteRepository`), `GoogleMapsDownloaderV2`, `ITileService`, `IRegionService`, `IRouteService`
5. `IRegionRequestQueue` with configurable capacity
6. Hosted services: `RegionProcessingService`, `RouteProcessingService`
7. CORS policy: `TilesCors` — configured origins from `CorsConfig:AllowedOrigins`, falls back to allow-any
8. JSON options: camelCase, case-insensitive
### Startup
1. Database migration via `DatabaseMigrator.RunMigrations()` — throws on failure
2. Creates tiles and ready directories
3. Swagger enabled in Development mode
4. HTTPS redirection, CORS applied
### ServeTile Handler
1. Checks `IMemoryCache` for tile bytes (1h absolute, 30min sliding expiration)
2. If cache miss: queries `ITileRepository.GetByTileCoordinatesAsync`
3. If no DB record: downloads tile via `GoogleMapsDownloaderV2.DownloadSingleTileAsync`, creates `TileEntity`, inserts
4. Returns image bytes with cache headers (`Cache-Control: public, max-age=86400`)
### GetTileByLatLon Handler
Downloads a tile, persists it, returns metadata as `DownloadTileResponse`.
### RequestRegion Handler
Validates size (10010000m), delegates to `IRegionService.RequestRegionAsync`.
## Dependencies
All project references: Common, DataAccess, Services.
NuGet: `Serilog.AspNetCore`, `Swashbuckle.AspNetCore`, `Microsoft.AspNetCore.OpenApi`, `SixLabors.ImageSharp`, `Newtonsoft.Json`.
## Consumers
- HTTP clients (external)
- Integration tests (via HTTP)
## Data Models
Defines several local request/response records that are not shared with other projects.
## Configuration
All configuration sections are consumed here:
- `ConnectionStrings:DefaultConnection`
- `MapConfig`, `StorageConfig`, `ProcessingConfig`
- `CorsConfig:AllowedOrigins`
- `Serilog` section
## External Integrations
- Google Maps (indirectly via `GoogleMapsDownloaderV2`)
- PostgreSQL (via repositories and DatabaseMigrator)
- File system (`./tiles/`, `./ready/`)
## Security
- CORS configured (permissive by default when no origins specified)
- Swagger only in Development
- HTTPS redirection enabled
- No authentication/authorization implemented
## Tests
Integration tests exercise all endpoints. Unit test project has only a dummy test.
@@ -0,0 +1,63 @@
# Module: Common/Configs
## Purpose
Configuration POCOs that bind to `appsettings.json` sections via `IOptions<T>` pattern.
## Public Interface
### MapConfig
- `Service` (string): map provider name (e.g., "GoogleMaps")
- `ApiKey` (string): API key for the map provider
### StorageConfig
- `TilesDirectory` (string): base path for tile storage (default: `/tiles`)
- `ReadyDirectory` (string): base path for output files (default: `/ready`)
- `GetTileSubdirectoryPath(int zoomLevel, int tileX, int tileY) → string`: computes bucketed subdirectory path (`{tiles}/{zoom}/{xBucket}/{yBucket}`) using integer division by 1000
- `GetTileFilePath(int zoomLevel, int tileX, int tileY, string timestamp) → string`: computes full file path with timestamped filename (`tile_{z}_{x}_{y}_{ts}.jpg`)
### ProcessingConfig
- `MaxConcurrentDownloads` (int, default: 4): semaphore limit for parallel tile downloads
- `MaxConcurrentRegions` (int, default: 3): parallel region processing workers
- `DefaultZoomLevel` (int, default: 20): fallback zoom level
- `QueueCapacity` (int, default: 100): bounded channel capacity for region request queue
- `DelayBetweenRequestsMs` (int, default: 50): throttle delay between Google Maps requests
- `SessionTokenReuseCount` (int, default: 100): tiles per session token before rotation
### DatabaseConfig
- `ConnectionString` (string): DB connection string (unused — connection string is resolved directly from `IConfiguration` in `Program.cs`)
## Internal Logic
`StorageConfig.GetTileSubdirectoryPath` buckets tiles by dividing X/Y coordinates by 1000, preventing filesystem performance degradation from too many files in one directory.
## Dependencies
None (pure POCOs, no internal imports).
## Consumers
- `Program.cs` — binds from config sections via `builder.Services.Configure<T>()`
- `GoogleMapsDownloaderV2` — reads `MapConfig`, `StorageConfig`, `ProcessingConfig` via `IOptions<T>`
- `RegionService` — reads `StorageConfig`
- `RegionProcessingService` — reads `ProcessingConfig`
- `RouteProcessingService` — reads `StorageConfig`
- `RegionRequestQueue` — receives `QueueCapacity` as constructor param
## Data Models
No domain entities; these are configuration DTOs.
## Configuration
These classes **define** the configuration shape consumed by all services.
| Config Class | appsettings Section |
|-------------|-------------------|
| MapConfig | `MapConfig` |
| StorageConfig | `StorageConfig` |
| ProcessingConfig | `ProcessingConfig` |
| DatabaseConfig | (not wired — connection string read directly) |
## External Integrations
None.
## Security
`MapConfig.ApiKey` holds the Google Maps API key. In production, injected via environment variable `MapConfig__ApiKey`.
## Tests
No dedicated tests.
+101
View File
@@ -0,0 +1,101 @@
# Module: Common/DTO
## Purpose
Data transfer objects used across all layers — API requests/responses, inter-service communication, and queue messages.
## Public Interface
### GeoPoint
Geographic coordinate with tolerance-based equality.
- `Lat` (double): latitude, JSON property `"lat"`
- `Lon` (double): longitude, JSON property `"lon"`
- Constructor: `GeoPoint()`, `GeoPoint(double lat, double lon)`
- Equality: two points are equal if both coordinates differ by less than `0.00005` (PRECISION_TOLERANCE)
- Operator overloads: `==`, `!=`
### Direction
Result of a directional calculation between two points.
- `Distance` (double): distance in meters
- `Azimuth` (double): bearing in degrees (0360)
### SatTile
Represents a single map tile with its spatial bounds.
- `X`, `Y` (int): tile coordinates in the slippy map scheme
- `Zoom` (int): zoom level
- `LeftTop`, `BottomRight` (GeoPoint): computed bounding box corners (via `GeoUtils.TileToWorldPos`)
- `Url` (string): download URL
- `FileName → string`: formatted as `{X}.{Y}.{Zoom}.jpg`
### TileMetadata
Metadata about a stored tile (mirrors `TileEntity` but without DB-specific concerns).
- `Id` (Guid), `TileZoom`, `TileX`, `TileY` (int), `Latitude`, `Longitude` (double)
- `TileSizeMeters` (double), `TileSizePixels` (int), `ImageType` (string)
- `MapsVersion` (string?), `Version` (int), `FilePath` (string)
- `CreatedAt`, `UpdatedAt` (DateTime)
### RegionRequest
Queue message for async region processing.
- `Id` (Guid), `Latitude`, `Longitude` (double), `SizeMeters` (double)
- `ZoomLevel` (int), `StitchTiles` (bool)
### RegionStatus
Response DTO for region status queries.
- `Id` (Guid), `Status` (string), `CsvFilePath`, `SummaryFilePath` (string?)
- `TilesDownloaded`, `TilesReused` (int), `CreatedAt`, `UpdatedAt` (DateTime)
### RoutePoint
Input point in a route creation request.
- `Latitude` (double, JSON: `"lat"`), `Longitude` (double, JSON: `"lon"`)
### RoutePointDto
Output point in a route response (includes computed fields).
- `Latitude`, `Longitude` (double), `PointType` (string: "start"/"end"/"action"/"intermediate")
- `SequenceNumber`, `SegmentIndex` (int), `DistanceFromPrevious` (double?)
### CreateRouteRequest
API request body for route creation.
- `Id` (Guid), `Name` (string), `Description` (string?)
- `RegionSizeMeters` (double), `ZoomLevel` (int)
- `Points` (List\<RoutePoint\>), `Geofences` (Geofences?)
- `RequestMaps` (bool), `CreateTilesZip` (bool)
### RouteResponse
API response for route queries.
- All fields from the route entity plus `Points` (List\<RoutePointDto\>)
- `MapsReady` (bool), `TilesZipPath` (string?)
### GeofencePolygon
Axis-aligned bounding box defined by NW and SE corners.
- `NorthWest` (GeoPoint?), `SouthEast` (GeoPoint?)
### Geofences
Container for multiple geofence polygons.
- `Polygons` (List\<GeofencePolygon\>)
## Internal Logic
- `GeoPoint` uses a precision tolerance of `0.00005` degrees (~5.5 meters) for equality comparison.
- `SatTile` eagerly computes its bounding box corners on construction by calling `GeoUtils.TileToWorldPos`.
## Dependencies
- `GeoPoint`, `Direction` — no imports
- `SatTile``SatelliteProvider.Common.Utils.GeoUtils`
- All others — no internal dependencies (or only `System.Text.Json.Serialization`)
## Consumers
- All services, repositories, and API endpoints consume these DTOs
- `RegionRequest` is the message type for `IRegionRequestQueue`
## Data Models
These ARE the data models (DTOs). They map closely to the database entities but are decoupled from the persistence layer.
## Configuration
None consumed directly.
## External Integrations
None.
## Security
None.
## Tests
No dedicated DTO tests.
@@ -0,0 +1,55 @@
# Module: Common/Utils/GeoUtils
## Purpose
Static geographic computation utilities: coordinate conversions, distance calculations (Haversine), bearing computation, point interpolation, and bounding box calculation.
## Public Interface
All methods are static on `GeoUtils`:
- `WorldToTilePos(GeoPoint point, int zoom) → (int x, int y)`: converts lat/lon to slippy map tile coordinates at given zoom
- `TileToWorldPos(int x, int y, int zoom) → GeoPoint`: converts tile coordinates back to lat/lon (NW corner of tile)
- `ToRadians(double degrees) → double`
- `ToDegrees(double radians) → double`
- `DirectionTo(this GeoPoint p1, GeoPoint p2) → Direction` (extension method): Haversine distance + forward azimuth between two points
- `GoDirection(this GeoPoint startPoint, Direction direction) → GeoPoint` (extension method): destination point given start + bearing + distance
- `GetBoundingBox(GeoPoint center, double radiusM) → (minLat, maxLat, minLon, maxLon)`: axis-aligned bounding box around a center point
- `CalculateIntermediatePoints(GeoPoint start, GeoPoint end, double maxSpacingMeters) → List<GeoPoint>`: generates evenly-spaced points along a great-circle path (returns empty if distance ≤ maxSpacing)
- `CalculateDistance(GeoPoint p1, GeoPoint p2) → double`: convenience wrapper around `DirectionTo().Distance`
- `CalculateCenter(GeoPoint northWest, GeoPoint southEast) → GeoPoint`: simple midpoint
- `CalculatePolygonDiagonalDistance(GeoPoint northWest, GeoPoint southEast) → double`: diagonal distance of a bounding box
## Internal Logic
- Earth radius constant: `6378137` meters (WGS-84 semi-major axis)
- Distance calculation uses the Haversine formula
- Bearing uses `atan2` of longitude/latitude components
- `CalculateIntermediatePoints` divides the segment into equal sub-segments, each ≤ `maxSpacingMeters`
- Tile conversion follows the standard Web Mercator / slippy map tile numbering scheme
## Dependencies
- `SatelliteProvider.Common.DTO.GeoPoint`
- `SatelliteProvider.Common.DTO.Direction`
## Consumers
- `GoogleMapsDownloaderV2``WorldToTilePos`, `TileToWorldPos`, `GetBoundingBox`
- `TileService` — indirectly via downloader
- `RegionService``WorldToTilePos` for tile stitching
- `RouteService``CalculateIntermediatePoints`, `CalculateDistance`
- `RouteProcessingService``WorldToTilePos` for map stitching
- `SatTile` constructor — `TileToWorldPos`
- `Program.cs` (ServeTile handler) — `TileToWorldPos`
## Data Models
None.
## Configuration
None.
## External Integrations
None (pure math).
## Security
None.
## Tests
No dedicated unit tests for GeoUtils.
@@ -0,0 +1,56 @@
# Module: Common/Interfaces
## Purpose
Service contracts defining the application's core operations. Implementations live in `SatelliteProvider.Services`.
## Public Interface
### ITileService
- `DownloadAndStoreTilesAsync(double lat, double lon, double sizeMeters, int zoomLevel, CancellationToken) → Task<List<TileMetadata>>`: downloads missing tiles for a region and returns all tile metadata (existing + new)
- `GetTileAsync(Guid id) → Task<TileMetadata?>`: retrieve a single tile by ID
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileMetadata>>`: query tiles within a geographic region
### IRegionService
- `RequestRegionAsync(Guid id, double lat, double lon, double sizeMeters, int zoomLevel, bool stitchTiles) → Task<RegionStatus>`: creates a region record and enqueues for async processing
- `GetRegionStatusAsync(Guid id) → Task<RegionStatus?>`: retrieves current status of a region request
- `ProcessRegionAsync(Guid id, CancellationToken) → Task`: executes tile downloading, CSV/summary generation, optional stitching
### IRouteService
- `CreateRouteAsync(CreateRouteRequest request) → Task<RouteResponse>`: validates input, calculates intermediate points, persists route + points, optionally creates geofence regions
- `GetRouteAsync(Guid id) → Task<RouteResponse?>`: retrieves route with all points
### ISatelliteDownloader
- `GetTiles(GeoPoint geoPoint, double radiusM, int zoomLevel, CancellationToken) → Task`: legacy interface for tile downloading (not directly implemented by `GoogleMapsDownloaderV2`)
### IRegionRequestQueue
- `EnqueueAsync(RegionRequest request, CancellationToken) → ValueTask`: add region request to the bounded queue
- `DequeueAsync(CancellationToken) → ValueTask<RegionRequest?>`: consume next request (blocks until available)
- `Count` (int): current queue depth
## Internal Logic
Pure interface definitions — no logic.
## Dependencies
- All interfaces reference DTOs from `SatelliteProvider.Common.DTO`
## Consumers
- `Program.cs` — DI registration of implementations
- `RegionProcessingService` — consumes `IRegionRequestQueue` and `IRegionService`
- `RouteService` — consumes `IRegionService` (for geofence region creation)
- `RouteProcessingService` — consumes `IRegionService` via service provider scope
- API endpoints — consume `ITileService`, `IRegionService`, `IRouteService`
## Data Models
None defined here.
## Configuration
None.
## External Integrations
None.
## Security
None.
## Tests
No dedicated interface tests.
@@ -0,0 +1,49 @@
# Module: DataAccess/DatabaseMigrator
## Purpose
Runs DbUp-based SQL migrations against PostgreSQL on application startup. Ensures the database schema is up to date before the API begins serving requests.
## Public Interface
### DatabaseMigrator
- Constructor: `DatabaseMigrator(string connectionString, ILogger<DatabaseMigrator>? logger)`
- `RunMigrations() → bool`: creates the database if missing (`EnsureDatabase.For.PostgresqlDatabase`), then runs all embedded SQL scripts matching `.Migrations.` from the DataAccess assembly. Returns `true` on success.
## Internal Logic
- Uses `DbUp.DeployChanges` fluent API targeting PostgreSQL
- Scripts are embedded resources filtered by path containing `.Migrations.`
- Logs to console via DbUp's built-in `LogToConsole()`
- On failure, logs the error and returns `false`
## Dependencies
- NuGet: `dbup-postgresql` (6.0.3)
- `Microsoft.Extensions.Logging`
- Embedded SQL resources from `SatelliteProvider.DataAccess/Migrations/`
## Consumers
- `Program.cs` — instantiated directly (not via DI) and called during startup. If migration fails, the application throws and does not start.
## Migrations (11 scripts)
1. `001_CreateTilesTable.sql`
2. `002_CreateRegionsTable.sql`
3. `003_CreateIndexes.sql`
4. `004_AddVersionColumn.sql`
5. `005_CreateRoutesTables.sql`
6. `006_AddStitchTilesToRegions.sql`
7. `007_AddRouteMapFields.sql`
8. `008_AddGeofenceFlagToRouteRegions.sql`
9. `009_AddGeofencePolygonIndex.sql`
10. `010_AddTilesZipToRoutes.sql`
11. `011_AddTileCoordinates.sql`
## Configuration
Receives connection string directly as constructor parameter.
## External Integrations
PostgreSQL — DDL operations via DbUp.
## Security
None directly, but controls schema evolution.
## Tests
No dedicated tests.
@@ -0,0 +1,66 @@
# Module: DataAccess/Models
## Purpose
Database entity classes that map directly to PostgreSQL tables via Dapper. Property names use PascalCase; column mapping is done with SQL aliases in repository queries.
## Public Interface
### TileEntity
Maps to `tiles` table.
- `Id` (Guid), `TileZoom` (int), `TileX` (int), `TileY` (int)
- `Latitude`, `Longitude` (double), `TileSizeMeters` (double), `TileSizePixels` (int)
- `ImageType` (string), `MapsVersion` (string?), `Version` (int)
- `FilePath` (string), `CreatedAt`, `UpdatedAt` (DateTime)
### RegionEntity
Maps to `regions` table.
- `Id` (Guid), `Latitude`, `Longitude` (double), `SizeMeters` (double)
- `ZoomLevel` (int), `Status` (string: "queued"/"processing"/"completed"/"failed")
- `CsvFilePath`, `SummaryFilePath` (string?)
- `TilesDownloaded`, `TilesReused` (int), `StitchTiles` (bool)
- `CreatedAt`, `UpdatedAt` (DateTime)
### RouteEntity
Maps to `routes` table.
- `Id` (Guid), `Name` (string), `Description` (string?)
- `RegionSizeMeters` (double), `ZoomLevel` (int)
- `TotalDistanceMeters` (double), `TotalPoints` (int)
- `RequestMaps`, `MapsReady`, `CreateTilesZip` (bool)
- `CsvFilePath`, `SummaryFilePath`, `StitchedImagePath`, `TilesZipPath` (string?)
- `CreatedAt`, `UpdatedAt` (DateTime)
### RoutePointEntity
Maps to `route_points` table.
- `Id` (Guid), `RouteId` (Guid), `SequenceNumber` (int)
- `Latitude`, `Longitude` (double), `PointType` (string)
- `SegmentIndex` (int), `DistanceFromPrevious` (double?)
- `CreatedAt` (DateTime)
## Internal Logic
Plain POCOs with no logic.
## Dependencies
None.
## Consumers
- All repository implementations (TileRepository, RegionRepository, RouteRepository)
- `TileService` — creates `TileEntity` instances for persistence
- `RegionService` — creates/updates `RegionEntity`
- `RouteService` — creates `RouteEntity` and `RoutePointEntity`
- `RouteProcessingService` — reads entities from repositories
- `GoogleMapsDownloaderV2.GetTilesWithMetadataAsync` — accepts `IEnumerable<TileEntity>` to check existing tiles
## Data Models
These ARE the data model.
## Configuration
None.
## External Integrations
None.
## Security
None.
## Tests
No dedicated tests.
@@ -0,0 +1,43 @@
# Module: DataAccess/Repositories/RegionRepository
## Purpose
Dapper-based repository for the `regions` table. Tracks region processing requests and their status lifecycle.
## Public Interface
### IRegionRepository (interface)
- `GetByIdAsync(Guid id) → Task<RegionEntity?>`
- `GetByStatusAsync(string status) → Task<IEnumerable<RegionEntity>>`: retrieves all regions with a given status, ordered by creation date ASC
- `InsertAsync(RegionEntity region) → Task<Guid>`
- `UpdateAsync(RegionEntity region) → Task<int>`
- `DeleteAsync(Guid id) → Task<int>`
### RegionRepository (implementation)
Same connection-per-call pattern as TileRepository.
## Internal Logic
Standard CRUD. `GetByStatusAsync` orders by `created_at ASC` to process oldest requests first.
## Dependencies
- NuGet: `Dapper`, `Npgsql`
- `SatelliteProvider.DataAccess.Models.RegionEntity`
- `Microsoft.Extensions.Logging`
## Consumers
- `RegionService` — insert on request, update during processing
- `RouteProcessingService` — reads region records to check status and get CSV paths
## Data Models
Operates on `RegionEntity`.
## Configuration
Connection string via constructor.
## External Integrations
PostgreSQL.
## Security
None.
## Tests
No dedicated tests.
@@ -0,0 +1,51 @@
# Module: DataAccess/Repositories/RouteRepository
## Purpose
Dapper-based repository for the `routes`, `route_points`, and `route_regions` tables. Handles route persistence, point storage, and route-region linking (including geofence metadata).
## Public Interface
### IRouteRepository (interface)
- `GetByIdAsync(Guid id) → Task<RouteEntity?>`
- `GetRoutePointsAsync(Guid routeId) → Task<IEnumerable<RoutePointEntity>>`: ordered by `sequence_number`
- `InsertRouteAsync(RouteEntity route) → Task<Guid>`
- `InsertRoutePointsAsync(IEnumerable<RoutePointEntity> points) → Task`: bulk insert
- `UpdateRouteAsync(RouteEntity route) → Task<int>`
- `DeleteRouteAsync(Guid id) → Task<int>`
- `LinkRouteToRegionAsync(Guid routeId, Guid regionId, bool isGeofence, int? geofencePolygonIndex) → Task`: inserts into `route_regions` with `ON CONFLICT DO NOTHING`
- `GetRegionIdsByRouteAsync(Guid routeId) → Task<IEnumerable<Guid>>`: non-geofence region IDs
- `GetGeofenceRegionIdsByRouteAsync(Guid routeId) → Task<IEnumerable<Guid>>`: geofence-only region IDs
- `GetGeofenceRegionsByPolygonAsync(Guid routeId) → Task<Dictionary<int, List<Guid>>>`: groups geofence regions by polygon index
- `GetRoutesWithPendingMapsAsync() → Task<IEnumerable<RouteEntity>>`: routes where `request_maps = true AND maps_ready = false`
### RouteRepository (implementation)
Same connection-per-call pattern. `InsertRoutePointsAsync` uses Dapper's bulk execute to insert all points in a single round-trip.
## Internal Logic
- `LinkRouteToRegionAsync` uses `ON CONFLICT DO NOTHING` to handle duplicate links gracefully
- `GetGeofenceRegionsByPolygonAsync` groups results into a dictionary keyed by `geofence_polygon_index`
- `GetRoutesWithPendingMapsAsync` drives the `RouteProcessingService` polling loop
## Dependencies
- NuGet: `Dapper`, `Npgsql`
- `SatelliteProvider.DataAccess.Models.RouteEntity`, `RoutePointEntity`
- `Microsoft.Extensions.Logging`
## Consumers
- `RouteService` — insert route, points, link regions
- `RouteProcessingService` — read route state, points, region links; update route after map generation
## Data Models
Operates on `RouteEntity`, `RoutePointEntity`, and the `route_regions` junction table.
## Configuration
Connection string via constructor.
## External Integrations
PostgreSQL.
## Security
None.
## Tests
No dedicated tests.
@@ -0,0 +1,47 @@
# Module: DataAccess/Repositories/TileRepository
## Purpose
Dapper-based repository for the `tiles` table. Handles CRUD operations and spatial queries for satellite tile records.
## Public Interface
### ITileRepository (interface)
- `GetByIdAsync(Guid id) → Task<TileEntity?>`
- `GetByTileCoordinatesAsync(int tileZoom, int tileX, int tileY) → Task<TileEntity?>`: finds tile by slippy map coordinates, returns latest version
- `FindExistingTileAsync(double lat, double lon, double tileSizeMeters, int zoomLevel, int version) → Task<TileEntity?>`: fuzzy coordinate match (tolerance: 0.0001° lat/lon, 1m tile size)
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileEntity>>`: spatial bounding box query with expanded range to cover tile edges
- `InsertAsync(TileEntity tile) → Task<Guid>`: upsert — `ON CONFLICT (latitude, longitude, tile_zoom, tile_size_meters, version) DO UPDATE` file_path, tile_x, tile_y, updated_at
- `UpdateAsync(TileEntity tile) → Task<int>`
- `DeleteAsync(Guid id) → Task<int>`
### TileRepository (implementation)
Constructs a new `NpgsqlConnection` per method call (no connection pooling at the repository level; Npgsql pools connections internally).
## Internal Logic
- `GetTilesByRegionAsync` calculates a bounding box by expanding the requested region by 2 × tile size to ensure edge tiles are included. Uses meters-to-degrees approximation (111,000 m/degree latitude, adjusted for longitude).
- `InsertAsync` uses an upsert pattern to handle duplicate tile downloads gracefully.
- `GetByTileCoordinatesAsync` orders by `version DESC` and takes the latest.
## Dependencies
- NuGet: `Dapper`, `Npgsql`
- `SatelliteProvider.DataAccess.Models.TileEntity`
- `Microsoft.Extensions.Logging`
## Consumers
- `TileService` — all read/write operations
- `Program.cs` (ServeTile, GetTileByLatLon handlers) — `GetByTileCoordinatesAsync`, `InsertAsync`
## Data Models
Operates on `TileEntity`.
## Configuration
Receives connection string via constructor.
## External Integrations
PostgreSQL — SQL queries via Dapper + Npgsql.
## Security
None.
## Tests
No dedicated repository tests.
@@ -0,0 +1,65 @@
# Module: Services/GoogleMapsDownloaderV2
## Purpose
Downloads satellite imagery tiles from Google Maps. Handles session token management, concurrent download throttling, retry logic with exponential backoff, and tile deduplication.
## Public Interface
### GoogleMapsDownloaderV2
- Constructor: `GoogleMapsDownloaderV2(ILogger, IOptions<MapConfig>, IOptions<StorageConfig>, IOptions<ProcessingConfig>, IHttpClientFactory)`
- `DownloadSingleTileAsync(double lat, double lon, int zoomLevel, CancellationToken) → Task<DownloadedTileInfoV2>`: downloads one tile at specified coordinates. Validates zoom level, creates session token, downloads image, saves to disk.
- `GetTilesWithMetadataAsync(GeoPoint center, double radiusM, int zoomLevel, IEnumerable<TileEntity> existingTiles, CancellationToken) → Task<List<DownloadedTileInfoV2>>`: downloads all tiles in a bounding box, skipping those already present in `existingTiles`. Manages session token rotation.
### DownloadedTileInfoV2 (record)
- `X`, `Y` (int), `ZoomLevel` (int), `CenterLatitude`, `CenterLongitude` (double), `FilePath` (string), `TileSizeMeters` (double)
### RateLimitException (exception)
Custom exception thrown when Google Maps returns 429 Too Many Requests and retries are exhausted.
## Internal Logic
- **Allowed zoom levels**: 15, 16, 17, 18, 19 — throws `ArgumentException` for others
- **URL template**: `https://mt{server}.google.com/vt/lyrs=s&x={x}&y={y}&z={z}&token={token}`
- **Session tokens**: obtained via `https://tile.googleapis.com/v1/createSession?key={apiKey}`, rotated every `SessionTokenReuseCount` tiles (default: 100)
- **Concurrency control**: `SemaphoreSlim` limits parallel downloads to `MaxConcurrentDownloads` (default: 4)
- **Deduplication**: `ConcurrentDictionary<string, Task<DownloadedTileInfoV2>>` (`_activeDownloads`) prevents duplicate concurrent downloads of the same tile
- **Retry logic**: exponential backoff (1s base, 30s max, 5 retries) for 429 and 5xx errors. Cancellation and auth errors (401, 403) propagate immediately.
- **Server selection**: `(x + y) % 4` distributes requests across `mt0``mt3`; single-tile downloads always use `mt0`
- **Delay between requests**: configurable via `ProcessingConfig.DelayBetweenRequestsMs`
- **Tile size calculation**: `CalculateTileSizeInMeters` uses Earth circumference × cos(latitude) / (2^zoom × 256)
## Dependencies
- `SatelliteProvider.Common.Configs` — MapConfig, StorageConfig, ProcessingConfig
- `SatelliteProvider.Common.DTO` — GeoPoint
- `SatelliteProvider.Common.Utils` — GeoUtils
- `SatelliteProvider.DataAccess.Models` — TileEntity (for existingTiles parameter)
- NuGet: `Newtonsoft.Json`, `Microsoft.Extensions.Http`, `Microsoft.Extensions.Options`
## Consumers
- `TileService``GetTilesWithMetadataAsync`
- `Program.cs` (ServeTile, GetTileByLatLon) — `DownloadSingleTileAsync`
## Data Models
Produces `DownloadedTileInfoV2` records; accepts `TileEntity` for cache checks.
## Configuration
| Config | Key | Used For |
|--------|-----|----------|
| MapConfig | ApiKey | Session token requests |
| StorageConfig | TilesDirectory | File save paths |
| ProcessingConfig | MaxConcurrentDownloads | SemaphoreSlim capacity |
| ProcessingConfig | DelayBetweenRequestsMs | Throttle delay |
| ProcessingConfig | SessionTokenReuseCount | Token rotation threshold |
## External Integrations
| Integration | Protocol | Details |
|-------------|----------|---------|
| Google Maps Tile API | HTTPS | `mt*.google.com/vt/lyrs=s` for tiles |
| Google Maps Session API | HTTPS | `tile.googleapis.com/v1/createSession` |
| File system | Local FS | Writes JPEG tiles to `StorageConfig.TilesDirectory` |
## Security
- API key transmitted over HTTPS to Google endpoints
- User-Agent spoofs a Chrome browser to match expected Google Maps client
## Tests
No dedicated unit tests (the test file `GoogleMapsDownloaderTests.cs` contains only a dummy test).
@@ -0,0 +1,91 @@
# Module: Services/RegionService + RegionProcessingService + RegionRequestQueue
## Purpose
End-to-end region processing pipeline: API request handling → queue → background worker → tile download → output file generation. Three closely coupled classes form the region processing subsystem.
## Public Interface
### RegionService (implements IRegionService)
- `RequestRegionAsync(...)`: creates `RegionEntity` with status "queued", enqueues a `RegionRequest`, returns `RegionStatus`
- `GetRegionStatusAsync(Guid id)`: reads region record and maps to `RegionStatus`
- `ProcessRegionAsync(Guid id, CancellationToken)`: the main processing pipeline — see Internal Logic
### RegionProcessingService (BackgroundService)
- `ExecuteAsync(CancellationToken)`: spawns `MaxConcurrentRegions` parallel worker tasks, each in an infinite dequeue loop
### RegionRequestQueue (implements IRegionRequestQueue)
- `EnqueueAsync(RegionRequest, CancellationToken)`: writes to a bounded `Channel<RegionRequest>`
- `DequeueAsync(CancellationToken)`: reads from the channel (blocks until available)
- `Count`: current queue depth
## Internal Logic
### RegionService.ProcessRegionAsync
1. Sets region status to "processing"
2. Creates a 5-minute timeout `CancellationTokenSource`
3. Queries existing tiles in the region
4. Calls `TileService.DownloadAndStoreTilesAsync` to fetch missing tiles
5. Counts downloaded vs reused tiles
6. Generates CSV file (`region_{id}_ready.csv`) listing tile coordinates + paths
7. Optionally stitches tiles into a single JPEG image (if `StitchTiles` is true)
8. Generates summary file (`region_{id}_summary.txt`)
9. Updates region status to "completed"
10. On any error: sets status to "failed", generates error summary
### Tile Stitching
Uses ImageSharp to:
1. Compute a tile grid from tile coordinates
2. Create a new image of `(gridWidth × 256) × (gridHeight × 256)` pixels
3. Place each tile image at its grid position
4. Draw a red crosshair at the center coordinates
5. Save as JPEG
### Error Handling
Comprehensive catch blocks for:
- `TaskCanceledException` (timeout vs external cancellation)
- `OperationCanceledException`
- `RateLimitException` (Google rate limiting)
- `HttpRequestException` (with status code)
- Generic `Exception`
Each sets status to "failed" and writes an error summary file.
### RegionProcessingService
- Spawns `MaxConcurrentRegions` worker tasks with staggered startup (100500ms random delay)
- Each worker loops: dequeue → `ProcessRegionAsync` → repeat
- Graceful shutdown on cancellation
### RegionRequestQueue
- Uses `System.Threading.Channels.Channel<T>.CreateBounded` with `BoundedChannelFullMode.Wait`
- Tracks `_totalEnqueued` and `_totalDequeued` counters
## Dependencies
- `ITileService`, `IRegionRepository`, `IRegionRequestQueue`
- `StorageConfig`, `ProcessingConfig`
- `SixLabors.ImageSharp` — tile stitching
- `SatelliteProvider.Common.Utils.GeoUtils` — coordinate conversion for stitching
## Consumers
- `Program.cs` API endpoints — `RequestRegionAsync`, `GetRegionStatusAsync`
- `RouteService``RequestRegionAsync` (for geofence regions)
- `RouteProcessingService``RequestRegionAsync` (for route-point regions)
## Data Models
- Input: `RegionRequest` (queue message)
- Output: `RegionStatus` (API response), CSV files, summary files, stitched images
- Persistence: `RegionEntity`
## Configuration
- `StorageConfig.ReadyDirectory` — output file location
- `ProcessingConfig.MaxConcurrentRegions` — worker count
- `ProcessingConfig.QueueCapacity` — bounded channel size
## External Integrations
- PostgreSQL (via repositories)
- File system (CSV, summary, stitched images in `./ready/`)
- Google Maps (indirectly via TileService → GoogleMapsDownloaderV2)
## Security
None.
## Tests
Integration tests in `RegionTests.cs` cover the request → poll → complete flow.
@@ -0,0 +1,91 @@
# Module: Services/RouteService + RouteProcessingService
## Purpose
Route management and asynchronous map generation. `RouteService` handles route creation with intermediate point interpolation and geofencing. `RouteProcessingService` is a background service that polls for routes needing map generation and produces stitched images, CSVs, summaries, and ZIP archives.
## Public Interface
### RouteService (implements IRouteService)
- `CreateRouteAsync(CreateRouteRequest) → Task<RouteResponse>`: validates input, computes intermediate points, persists route + points, creates geofence regions if specified
- `GetRouteAsync(Guid id) → Task<RouteResponse?>`: retrieves route with all points
### RouteProcessingService (BackgroundService)
- `ExecuteAsync(CancellationToken)`: polls every 5 seconds for routes with `request_maps = true AND maps_ready = false`, then processes each sequentially
## Internal Logic
### RouteService.CreateRouteAsync
1. **Validation**: minimum 2 points, region size 10010000m, name required
2. **Point interpolation**: for each segment between consecutive input points:
- First point typed as "start", last as "end", middle as "action"
- Calls `GeoUtils.CalculateIntermediatePoints(start, end, 200m)` to generate intermediate points
- Each intermediate point gets `PointType = "intermediate"`
- Distance from previous point is computed via `GeoUtils.CalculateDistance`
3. **Persistence**: inserts `RouteEntity` + bulk inserts `RoutePointEntity` via repository
4. **Geofencing** (if `Geofences.Polygons` provided):
- Validates each polygon: non-null corners, non-zero coordinates, valid lat/lon ranges, NW lat > SE lat
- Calls `CreateGeofenceRegionGrid` to divide the polygon bounding box into a grid of region centers
- For each grid point, calls `RegionService.RequestRegionAsync` and links to route as geofence region
5. Returns `RouteResponse` with all computed points
### CreateGeofenceRegionGrid
Divides a bounding box (NW → SE) into a regular grid where each cell is `regionSizeMeters` wide. Uses lat/lon step sizes derived from physical distance calculations. Returns a list of center points.
### RouteProcessingService.ProcessRouteSequentiallyAsync
1. Checks route needs processing (`RequestMaps && !MapsReady`)
2. Loads route points and linked region IDs (both regular and geofence)
3. If no regions linked yet: creates region requests for each route point
4. Checks completion status of all linked regions
5. When enough regions complete: generates consolidated outputs
6. Retries failed regions by creating new region requests
### GenerateRouteMapsAsync
1. Collects all tile data from completed region CSVs, deduplicating by coordinates
2. Generates consolidated route CSV
3. If `RequestMaps`: stitches all tiles into a single image with:
- Geofence polygon borders (yellow rectangles)
- Route point markers (red crosses, 50px arms, 10px thickness)
4. If `CreateTilesZip`: creates ZIP archive of all tile files with directory structure preserved
5. Generates route summary text file
6. Updates route record (`MapsReady = true`, file paths)
7. Cleans up individual region CSV/summary/stitched files
### Tile Stitching (route-level)
- Extracts tile X/Y from filenames (`tile_{z}_{x}_{y}_{ts}.jpg`)
- Creates grid-sized image, places tiles, draws geofence borders and route points
- Background color: black (for missing tiles)
### TileInfo (helper class)
Simple data holder: `Latitude`, `Longitude`, `FilePath`.
## Dependencies
- `IRouteRepository`, `IRegionRepository`, `IRegionService`
- `SatelliteProvider.Common.DTO` — GeoPoint, RoutePointDto, CreateRouteRequest, RouteResponse, GeofencePolygon
- `SatelliteProvider.Common.Utils.GeoUtils`
- `SatelliteProvider.DataAccess.Models` — RouteEntity, RoutePointEntity, RegionEntity
- `SixLabors.ImageSharp` — tile stitching
- `System.IO.Compression` — ZIP creation
- `IServiceProvider` — creates scoped `IRegionService` instances
## Consumers
- `Program.cs` API endpoints — `CreateRouteAsync`, `GetRouteAsync`
## Data Models
- Input: `CreateRouteRequest`, `RoutePoint`, `GeofencePolygon`
- Output: `RouteResponse`, `RoutePointDto`, CSV/summary/stitched/zip files
- Persistence: `RouteEntity`, `RoutePointEntity`, `route_regions` junction
## Configuration
- `StorageConfig.ReadyDirectory` — output directory
- `StorageConfig.TilesDirectory` — used for ZIP relative paths
## External Integrations
- PostgreSQL (via repositories)
- File system (CSV, summary, stitched image, ZIP in `./ready/`)
- Region processing pipeline (for tile downloading)
## Security
None.
## Tests
Integration tests in `BasicRouteTests.cs`, `ComplexRouteTests.cs`, `ExtendedRouteTests.cs`.
@@ -0,0 +1,45 @@
# Module: Services/TileService
## Purpose
Orchestrates tile downloading and persistence. Bridges the downloader (Google Maps) with the tile repository (PostgreSQL), handling cache checks, entity creation, and metadata mapping.
## Public Interface
### TileService (implements ITileService)
- `DownloadAndStoreTilesAsync(double lat, double lon, double sizeMeters, int zoomLevel, CancellationToken) → Task<List<TileMetadata>>`:
1. Queries existing tiles in the region from the repository (filtered to current year's version)
2. Calls `GoogleMapsDownloaderV2.GetTilesWithMetadataAsync` with existing tiles to skip
3. Creates `TileEntity` for each newly downloaded tile and inserts via repository (upsert)
4. Returns combined list of existing + new tile metadata
- `GetTileAsync(Guid id) → Task<TileMetadata?>`: single tile lookup
- `GetTilesByRegionAsync(double lat, double lon, double sizeMeters, int zoomLevel) → Task<IEnumerable<TileMetadata>>`: query tiles in a region
## Internal Logic
- Version is `DateTime.UtcNow.Year` — tiles are considered fresh for the current calendar year
- `MapToMetadata(TileEntity) → TileMetadata`: entity-to-DTO mapping (static helper)
- Tile size hardcoded to 256 pixels, image type "jpg"
- `MapsVersion` formatted as `"downloaded_{date}"`
## Dependencies
- `GoogleMapsDownloaderV2` (concrete class, not interface)
- `ITileRepository`
- `SatelliteProvider.Common.DTO` — GeoPoint, TileMetadata
- `SatelliteProvider.DataAccess.Models` — TileEntity
## Consumers
- `RegionService.ProcessRegionAsync` — downloads and retrieves tiles for a region
## Data Models
Transforms between `TileEntity` (persistence) and `TileMetadata` (DTO).
## Configuration
None directly; relies on `GoogleMapsDownloaderV2`'s configuration.
## External Integrations
Indirect: Google Maps (via downloader), PostgreSQL (via repository).
## Security
None.
## Tests
No dedicated tests.
@@ -0,0 +1,45 @@
# Module: Tests/SatelliteProvider.IntegrationTests
## Purpose
Console application that runs end-to-end integration tests against a live API instance. Designed to run in Docker alongside the API and PostgreSQL containers.
## Public Interface
### Test Classes
- `TileTests` — tile download via lat/lon endpoint
- `RegionTests` — region request → polling → completion flow
- `BasicRouteTests` — route creation with intermediate points
- `ComplexRouteTests` — routes with geofencing
- `ExtendedRouteTests` — routes with `requestMaps: true` and tile ZIP creation
### Supporting Classes
- `Models.cs` — HTTP response DTOs for deserialization
- `RouteTestHelpers.cs` — shared utilities (wait-for-completion polling, geofence polygon builders, test data)
- `Program.cs` — test runner entry point
## Internal Logic
- Makes HTTP calls to the API at `API_URL` environment variable (default: `http://api:8080`)
- Tests are methods called sequentially from `Program.cs` (not xUnit — plain console app)
- Poll-based waiting for async operations (region/route completion)
- Validates response structure, status transitions, file creation
## Dependencies
- No project references (standalone console app)
- Communicates with the API exclusively via HTTP
- NuGet: implicit .NET 8 runtime
## Consumers
- `docker-compose.tests.yml` — runs as a container that depends on the API service
## Configuration
- `API_URL` environment variable (set in docker-compose.tests.yml to `http://api:8080`)
## External Integrations
- HTTP to the SatelliteProvider API
- Reads output files from mounted `./ready/` and `./tiles/` volumes
## Security
None.
## Tests
This IS the integration test suite.
+23
View File
@@ -0,0 +1,23 @@
# Module: Tests/SatelliteProvider.Tests
## Purpose
Unit test project. Currently contains only a single dummy test as a placeholder.
## Public Interface
### DummyTests
- `Dummy_ShouldWork()`: asserts `1 == 1`
## Internal Logic
No meaningful test logic.
## Dependencies
- Project references: `SatelliteProvider.Services`, `SatelliteProvider.Common`
- NuGet: xUnit (2.5.3), Moq (4.20.72), FluentAssertions (8.8.0), coverlet.collector (6.0.0), Microsoft.NET.Test.Sdk (17.8.0), Microsoft.Extensions.* (Configuration, DI, Logging, Options, Http)
- Has `appsettings.json` copied to output (empty config for potential future test setups)
## Consumers
- CI pipeline (`01-test.yml`) runs `dotnet test` against this project
## Tests
This IS the test module. Coverage: effectively zero (only a dummy test).