satellite-provider/_docs/02_document/architecture.md

Satellite Provider — Architecture

Architecture Vision

Satellite Provider is a self-hosted .NET 10 backend service that pre-downloads, caches, and composites Google Maps satellite imagery for offline use. It runs as a single containerized monolith with PostgreSQL, processing requests asynchronously via in-process queues. The dominant pattern is a layered architecture (API → Services → DataAccess → PostgreSQL) with background hosted services for long-running work.

Components & responsibilities (each owns its own .csproj since AZ-309):

• Common (SatelliteProvider.Common) — Shared contracts: DTOs, service interfaces, common exceptions, configuration models, geospatial math
• DataAccess (SatelliteProvider.DataAccess) — PostgreSQL persistence via Dapper + DbUp migrations
• TileDownloader (SatelliteProvider.Services.TileDownloader) — Provider-agnostic tile acquisition via ISatelliteDownloader interface (first implementation: Google Maps) with deduplication, concurrency control, and an in-memory tile-byte cache owned by TileService
• RegionProcessing (SatelliteProvider.Services.RegionProcessing) — Batch tile downloads for geographic areas, stitching, output generation
• RouteManagement (SatelliteProvider.Services.RouteManagement) — Route interpolation, geofenced region generation, consolidated map output

The three Layer-3 service components are compile-time siblings: each only references SatelliteProvider.Common and SatelliteProvider.DataAccess. Cross-component runtime calls flow exclusively through interfaces in SatelliteProvider.Common.Interfaces.

Major data flows:

• Tile acquisition: HTTP request → cache check → Google Maps download → disk + DB persistence
• Region processing: Request queued → background worker calculates tile grid → downloads all tiles → produces CSV/summary/stitched image
• Route expansion: Waypoints → interpolated points every ~200m → geofence filtering → region requests per point → optional ZIP archive

Architectural principles (inferred):

• Single-instance deployment, no horizontal scaling requirements (inferred-from: Channel-based queue, no distributed state)
• Append-by-source-and-flight tile storage (AZ-503; refines AZ-484) — multiple producers (Google Maps, UAV upload from N flights, future SatAR, …) can each persist a row per (tile_zoom, tile_x, tile_y, tile_size_meters) cell. Reads return the most-recent row across sources, ordered by captured_at DESC with deterministic (updated_at DESC, id DESC) tie-breaks. The single-row-per-cell-per-source-per-flight invariant is enforced by idx_tiles_unique_identity (6-column integer-only, COALESCE(flight_id, '00000000-...'::uuid)) introduced in migration 014; this supersedes the AZ-484 float-based idx_tiles_unique_location_source. Identity is deterministic across re-ingests: tiles.id = Uuidv5(TileNamespace, "{z}/{x}/{y}/{source}/{flight_id or zero-uuid}") and tiles.location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}"). The tiles.version column remains vestigial since AZ-357 dropped year-based cache invalidation in favour of cell-level overwrite. (inferred-from: tiles table + AZ-484/AZ-357/AZ-503 migrations + tile-storage contract v1.0.0)
• Cross-repo deterministic tile identity (AZ-503) — the TileNamespace UUID 5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c and the canonical name format are shared with the sibling workspace gps-denied-onboard (components/c6_tile_cache/_uuid.py:TILE_NAMESPACE). Both sides MUST produce byte-identical UUIDv5 output so an onboard-cached tile and a server-cached tile for the same (z, x, y, source, flight_id) are recognized as the same artifact without a round-trip. Changing the namespace constant on either side is a coordinated cross-repo break. (inferred-from: Uuidv5.cs, AZ-503 task spec § Constraints)
• Fire-and-forget async processing with status polling (inferred-from: queue + background service + status endpoint)
• JWT-validated callers only — every HTTP endpoint requires a valid HS256-signed Bearer token, validated locally against a shared JWT_SECRET per the suite-level auth contract (suite/_docs/10_auth.md). Issuer/audience are intentionally not validated yet; signature + lifetime + ≥32-byte key are. Per-endpoint permission claims (e.g. permissions: ["GPS"] on the UAV upload) layer on top of this baseline.
• Strict wire-format validation at the API edge (AZ-795 epic, completed across cycles 7-8) — every public-facing endpoint runs every incoming payload through two collaborating layers BEFORE the handler sees it: (a) JsonSerializerOptions.UnmappedMemberHandling.Disallow + [JsonRequired] on every non-optional DTO axis at the System.Text.Json deserializer; (b) per-endpoint FluentValidation IValidator<T> wired via WithValidation<T>() (JSON bodies) or UavUploadValidationFilter (multipart) or RejectUnknownQueryParamsEndpointFilter + GetTileByLatLonQueryValidator (query params). Both layers produce identically-shaped RFC 7807 ValidationProblemDetails per error-shape.md v1.0.0, so callers see one error contract regardless of which layer fired. The principle is: no payload reaches a handler unless every field is present, every type matches, every range is honored, and no unknown field was silently dropped. This closes the silent-coercion footgun class (e.g. missing id → zero-Guid → untracked region/route; typo ?latitude= → lat=0; misnamed {"Latitude":...} → lat=0) that pre-cycle-7 produced misleading 200-OK responses. Adding a new public endpoint requires either a WithValidation<T>() chain (JSON), a UavUploadValidationFilter-style multipart filter, or an RejectUnknownQueryParamsEndpointFilter + query validator (query string) — there is no other approved path.

Authentication & Authorization (AZ-487):

• Validation library: Microsoft.AspNetCore.Authentication.JwtBearer 10.0.7 (matches Microsoft.AspNetCore.OpenApi 10.0.7; AZ-496 bumped both packages from 8.0.21 → 8.0.25 in cycle 3 to close the cycle-1 D1 + cycle-2 D3 supply-chain findings, then AZ-500 bumped both 8.0.25 → 10.0.7 in cycle 4 as part of the .NET 8 → .NET 10 migration). The TokenValidationParameters shape is unchanged across the JwtBearer 8 → 10 jump — AZ-487/AZ-494 integration tests are the gate and all pass on .NET 10.
• Signing key: read from the JWT_SECRET environment variable (preferred) or the Jwt:Secret configuration key. Startup fails fast if the resolved secret is unset, empty, or shorter than 32 bytes (HMAC-SHA256 minimum per RFC 2104 §3).
• Token contract: ValidateIssuerSigningKey = true, ValidateLifetime = true, RequireSignedTokens = true, RequireExpirationTime = true, ValidateIssuer = true + ValidIssuer = $JWT_ISSUER, ValidateAudience = true + ValidAudience = $JWT_AUDIENCE (AZ-494), ClockSkew = 30s. The 5-minute JwtBearer default is intentionally tightened.
• Authorization model: every endpoint registered in Program.cs is decorated with .RequireAuthorization(). AZ-488 adds permissions-claim policies on top of this baseline (UAV upload requires GPS).
• Test infrastructure: JwtTokenFactory (unit tests) and JwtTestHelpers (integration tests) mint deterministic tokens against the same JWT_SECRET; the integration test runner attaches a default Bearer token to its shared HttpClient so legacy non-auth tests continue to exercise the protected endpoints unchanged.

Planned features (confirmed by user, currently stubs):

• MGRS endpoint — tile access via Military Grid Reference System coordinates

Multi-source tile producers (live as of AZ-488; per-flight evidence isolation as of AZ-503):

• Google Maps — TileService.DownloadAndStoreTilesAsync / DownloadAndStoreSingleTileAsync stamp source='google_maps', flight_id=NULL, and a deterministic UUIDv5 id on every persisted row; tile JPEGs live under {StorageConfig.TilesDirectory}/{zoom}/... per the legacy grandfathered layout. content_sha256 is computed from the on-disk JPEG body.
• UAV — POST /api/satellite/upload (AZ-488; per-flight key extended by AZ-503) accepts a multipart batch of UAV-captured tiles, runs each item through a 5-rule quality gate (UavTileQualityGate), and persists accepted items via ITileRepository.InsertAsync with source='uav', flight_id = metadata.flightId (or NULL for anonymous uploads), and a deterministic UUIDv5 id. UAV JPEGs live under {StorageConfig.TilesDirectory}/uav/{flight_id or 'none'}/{zoom}/{x}/{y}.jpg, so rm -rf ./tiles/uav/{flight_id}/ removes one flight's evidence without touching other flights at overlapping cells. Requires the GPS permission claim on top of the JWT baseline.

The N-source storage contract is authoritative in _docs/02_document/contracts/data-access/tile-storage.md (v2.0.0 — bumped jointly by AZ-503-foundation and AZ-505 in cycle 6 to capture the identity columns, tiles_leaflet_path covering index, and location_hash-keyed leaflet read rule). The UAV upload contract is authoritative in _docs/02_document/contracts/api/uav-tile-upload.md (v1.2.0; AZ-503 added an optional flightId field to per-item metadata in v1.1.0, AZ-810 cycle 8 added the strict metadata-validation section in v1.2.0). The bulk tile-inventory contract is authoritative in _docs/02_document/contracts/api/tile-inventory.md (v2.0.0; AZ-505 v1.0.0, AZ-794+AZ-796 cycle 7 bumped to v2.0.0 with the OSM z/x/y rename + strict validation rules). The four wire-format contracts added in cycle 8 are authoritative for their respective endpoints: _docs/02_document/contracts/api/region-request.md v1.0.0 (POST /api/satellite/request, AZ-808+AZ-812), _docs/02_document/contracts/api/route-creation.md v1.0.0 (POST /api/satellite/route, AZ-809), _docs/02_document/contracts/api/tile-latlon.md v1.0.0 (GET /api/satellite/tiles/latlon, AZ-811), and _docs/02_document/contracts/api/error-shape.md v1.0.0 (the cross-endpoint RFC 7807 ValidationProblemDetails envelope shared by every validating endpoint, AZ-795 cycle 7). Anything that reads or writes tiles MUST follow those contracts rather than re-deriving the rules from prose here.

Drift signals:

• geofence_polygons mentioned in AGENTS.md as a routes table column but does not exist in schema or entity — documentation drift

1. System Context

Problem being solved: A GPS-denied UAV navigation service requires satellite imagery for positioning and route planning without GPS. This service pre-downloads satellite tiles from one or more imagery sources (currently Google Maps; future sources including UAV nadir camera upload and additional providers such as SatAR) for specified regions and routes, stores them alongside each other under a per-source storage key, and serves the most-recent row across sources on access. Tiles are stitched into composite images and packaged for offline use.

System boundaries: The Satellite Provider is a self-contained backend service. It receives HTTP requests (region/route definitions), downloads tiles from Google Maps, stores them on disk and in PostgreSQL, and produces output files (images, CSVs, ZIPs).

External systems:

System	Integration Type	Direction	Purpose
Satellite imagery provider (e.g., Google Maps)	HTTPS (tile download)	Outbound	First implementation of the multi-source tiles storage; provider-agnostic via ISatelliteDownloader. Stamps source='google_maps' on every persisted row.
GPS-Denied Service (UAV)	REST API (multipart)	Inbound	Producer of source='uav' rows via POST /api/satellite/upload (AZ-488). Authenticates with a JWT carrying the GPS permission claim; items pass through the 5-rule quality gate before persistence.
PostgreSQL	TCP (Npgsql)	Both	Tile metadata, region/route state
File System	Local disk	Both	Tile image storage, output artifacts
HTTP Clients	REST API	Inbound	Region/route requests, tile queries

2. Technology Stack

Layer	Technology	Version	Rationale
Language	C#	14.0 (was 12.0 through cycle 3 — AZ-500)	.NET ecosystem, strong typing
Framework	ASP.NET Core (Minimal API)	10.0 (was 8.0 through cycle 3 — AZ-500)	Lightweight HTTP hosting
Database	PostgreSQL	15+	Reliable RDBMS, spatial-friendly
ORM	Dapper	latest	Micro-ORM, raw SQL control
Migrations	DbUp	latest	Simple SQL-file-based schema migrations
Image Processing	SixLabors.ImageSharp	3.1.11	Cross-platform image manipulation
Logging	Serilog	8.0.3	Structured logging with file sinks
Hosting	Docker (docker-compose)	—	Containerized deployment
CI/CD	Woodpecker CI	—	Lightweight self-hosted CI

3. Deployment Model

Environments: Development (docker-compose), Production (Docker)

Infrastructure:

• Docker-based containerized deployment
• PostgreSQL as a separate container
• Shared volumes for tile storage and output artifacts
• No cloud provider dependency (self-hosted capable)

Environment-specific configuration:

Config	Development	Production
Database	localhost:5433 (Docker)	Container network db:5432
Secrets	appsettings.Development.json	Environment variables
Logging	Console + File	File (./logs/)
API URL	http://localhost:5100	http://0.0.0.0:5100

4. Data Model Overview

Core entities:

Entity	Description	Owned By Component
Tile	A single satellite image tile with coordinates and zoom	TileDownloader
Region	A square area request with processing status	RegionProcessing
Route	A named path with geofence polygons	RouteManagement
RoutePoint	An individual point (original or interpolated) on a route	RouteManagement

Key relationships:

• Route → RoutePoint: one-to-many (a route has many sequential points)
• Route → Region: many-to-many via route_regions (each route point generates a region)
• Region → Tile: implicit (a processed region references tiles by coordinate/zoom)

Data flow summary:

• Client → API → Queue → BackgroundService → GoogleMaps → FileSystem + DB: tile acquisition pipeline
• Client → API → RouteService → PointInterpolation → RegionCreation → Queue: route-to-region expansion

5. Integration Points

Internal Communication

From	To	Protocol	Pattern	Notes
WebApi	RegionProcessing	In-process queue (Channel)	Fire-and-forget	Request queued, status polled. Uses IRegionService / IRegionRequestQueue from Common.
WebApi	TileDownloader	ITileService (Common interface)	Request-Response	Single-tile reads (GetOrDownloadTileAsync) and writes (DownloadAndStoreSingleTileAsync) flow through ITileService since AZ-310 / AZ-311. No direct dependency on the concrete GoogleMapsDownloaderV2.
RegionProcessing	TileDownloader	ITileService (Common interface)	Request-Response	Per-tile within region processing. Resolved through DI; no compile-time ProjectReference between RegionProcessing and TileDownloader csprojs.
RouteManagement	RegionProcessing	IRegionService / IRegionRequestQueue (Common interfaces)	Fire-and-forget	Route regions submitted to queue. No compile-time ProjectReference between RouteManagement and RegionProcessing csprojs.
All Services	DataAccess	Direct method call (via repository interfaces)	Repository pattern	Dapper queries

External Integrations

External System	Protocol	Auth	Rate Limits	Failure Mode
Satellite imagery provider (abstracted via ISatelliteDownloader; first implementation: Google Maps)	HTTPS GET	Provider-specific (e.g., session token)	Configured concurrency (MaxConcurrentDownloads)	Retry with backoff, mark region failed

6. Non-Functional Requirements

Requirement	Target	Measurement	Priority
Concurrent Downloads	4 (configurable)	SemaphoreSlim limit	High
Concurrent Regions	20 (configurable)	Processing config	Medium
Queue Capacity	1000 requests	Channel bounded capacity	Medium
Tile Deduplication	100% (no re-download)	DB lookup before fetch	High
Max Zip Size	50 MB	Route zip output	Medium

7. Security Architecture

Authentication: HS256 JWT Bearer tokens (AZ-487 + AZ-494). Signing key from JWT_SECRET env var (≥ 32 bytes, validated at startup). Issuer and audience claims are validated against JWT_ISSUER / JWT_AUDIENCE env vars (AZ-494) — both required, fail-fast at startup if unset. Microsoft.AspNetCore.Authentication.JwtBearer validates signature, lifetime, signing key, issuer, and audience. ClockSkew tightened from JwtBearer default (5 min) to 30 s. Tokens are minted by the centralized Admin API per suite/_docs/10_auth.md; their iss and aud claims MUST match the satellite-provider configured values or validation rejects with 401.

Authorization: Every endpoint requires authentication via .RequireAuthorization(). Permission-claim enforcement is layered on top through the PermissionsRequirement authorization handler, which reads the permissions claim (accepting either repeated string claims OR a single JSON-array string). AZ-488 wires the RequiresGpsPermission policy on POST /api/satellite/upload — callers without GPS receive HTTP 403; other endpoints accept any authenticated principal.

Data protection:

• At rest: No encryption (tiles stored as plain JPEG files)
• In transit: HTTPS for Google Maps calls; API itself runs HTTP behind Kestrel (TLS termination is a deployment-layer concern)
• Secrets management: JWT_SECRET and GOOGLE_MAPS_API_KEY from environment variables / .env (gitignored); .env.example documents the required keys. Production deployments MUST supply both via the host environment, never via the appsettings files.

Audit logging: Serilog writes to file; logs exceptions and processing state transitions. 401/403 responses are emitted by the JwtBearer middleware via the WWW-Authenticate header; no body leakage of internal details.

8. Key Architectural Decisions

ADR-001: Minimal API over Controller-based

Context: Project needed a lightweight HTTP layer for a small set of endpoints.

Decision: Use ASP.NET Core Minimal APIs (no controllers, no MVC).

Consequences: Less ceremony, all routing in Program.cs, but less structure for future growth.

ADR-002: Dapper over Entity Framework

Context: Database access is straightforward CRUD with some spatial queries.

Decision: Use Dapper for raw SQL control and performance, paired with DbUp for schema migrations.

Consequences: Full SQL control, no ORM overhead; trade-off is manual mapping and no change tracking.

ADR-003: In-Process Queue over External Message Broker

Context: Region/route processing needs to be asynchronous but the system is a single service.

Decision: Use System.Threading.Channels as an in-process bounded queue.

Consequences: Simple, no external dependencies; but limited to single-instance deployment — no horizontal scaling of workers.

ADR-004: File-Based Tile Storage

Context: Tiles are immutable JPEG images that need fast random access.

Decision: Store tiles as files in a directory hierarchy with metadata in PostgreSQL. The layout is per-source (and per-flight for UAV since AZ-503) so the bytes for distinct producers / flights at the same cell remain individually addressable on disk:

• Google Maps (legacy, grandfathered): {StorageConfig.TilesDirectory}/{zoom}/{x_bucket}/{y_bucket}/tile_{zoom}_{x}_{y}_{timestamp}.jpg
• UAV anonymous (AZ-488 baseline): {StorageConfig.TilesDirectory}/uav/none/{zoom}/{x}/{y}.jpg
• UAV per-flight (AZ-503): {StorageConfig.TilesDirectory}/uav/{flight_id}/{zoom}/{x}/{y}.jpg

The authoritative source/flight markers are the tiles.source and tiles.flight_id columns; the per-source / per-flight on-disk path matters only for write isolation and bulk-delete granularity.

Consequences: Fast reads, easy backup/migration, producers can run without colliding on bytes, and per-flight rm -rf becomes safe. Requires shared filesystem for multi-instance (not currently needed). No migration of pre-AZ-488 Google Maps files is shipped — the legacy layout stays intact. Pre-AZ-503 UAV files written by the AZ-488 baseline at ./tiles/uav/{z}/{x}/{y}.jpg (no flight segment) are not relocated by the migration; the post-AZ-503 code writes anonymous uploads to ./tiles/uav/none/{z}/{x}/{y}.jpg and the original AZ-488-era files stay where they were. This is acceptable because AZ-488 only landed in cycle 2 and the volume of pre-AZ-503 UAV bytes is small (no production UAV upload traffic yet).

ADR-005: Background Hosted Services for Processing

Context: Region and route processing is long-running and should not block HTTP requests.

Decision: Use IHostedService implementations that consume from the in-process queue.

Consequences: Clean separation of request handling and processing; lifecycle managed by the host.

9. Input Validation (AZ-795)

Every public HTTP endpoint MUST reject malformed or out-of-range payloads with HTTP 400 + RFC 7807 ValidationProblemDetails. The shared infrastructure landed in AZ-795 (cycle 7) is two collaborating layers:

1. Deserializer-level rejection — JsonSerializerOptions.UnmappedMemberHandling.Disallow configured in Program.cs (ConfigureHttpJsonOptions) catches unknown fields, type mismatches, and malformed JSON. The framework wraps the resulting JsonException in BadHttpRequestException; GlobalExceptionHandler extracts the JSON path and emits a structured ValidationProblemDetails body.
2. Business-rule rejection — FluentValidation 12.0.0 validators registered via AddValidatorsFromAssemblyContaining<Program>() and wired through the generic ValidationEndpointFilter<T> (SatelliteProvider.Api/Validators/ValidationEndpointFilter.cs). Endpoints opt in via RouteHandlerBuilder.WithValidation<T>(); the filter calls Results.ValidationProblem(result.ToDictionary()) on failure.

Both layers produce the wire shape documented in _docs/02_document/contracts/api/error-shape.md (v1.0.0).

Validator coverage

Endpoint	Request DTO	Validator	Status	Owning task
POST /api/satellite/tiles/inventory	TileInventoryRequest	InventoryRequestValidator	covered	AZ-796 (cycle 7)
GET /tiles/{z}/{x}/{y}	route params	(route-constraint only — :int covers types; AZ-795 deserializer guards body shape on POST endpoints only)	covered by route-constraint	AZ-487 (cycle 1, JWT gate)
GET /api/satellite/tiles/latlon	query params	(query-binding type checks via [FromQuery]; future AZ-795 child task to add explicit FluentValidation)	partial	future AZ-795 child
POST /api/satellite/upload	UavTileBatchUploadRequest (multipart)	(envelope-level validation in UavTileUploadHandler; future AZ-795 child to formalize as FluentValidation)	partial	future AZ-795 child
POST /api/satellite/request	RequestRegionRequest	(inline SizeMeters range check; future AZ-795 child)	partial	future AZ-795 child
POST /api/satellite/route	CreateRouteRequest	(typed ArgumentException path → 400; future AZ-795 child)	partial	future AZ-795 child
GET /api/satellite/region/{id:guid}	route param	(route-constraint :guid)	covered by route-constraint	—
GET /api/satellite/route/{id:guid}	route param	(route-constraint :guid)	covered by route-constraint	—
GET /api/satellite/tiles/mgrs	(stub)	n/a — returns 501	n/a	AZ-356

The partial rows are tracked under the AZ-795 epic; per-endpoint child tickets to be filed by parent-suite team after enumerating the surface from the OpenAPI spec.