mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-21 15:51:14 +00:00
Oleksandr Bezdieniezhnykh
6b373082c8
Phase C of architecture coupling refactor (epic AZ-309). Closes the
last baseline finding (F5 — DataAccess incorrectly documented as
importing Common) and synchronizes the rest of _docs/02_document/
with the post-split project layout from AZ-312/313/314:
- module-layout.md: per-component sections for the three new csprojs
with explicit ProjectReferences and the no-cross-sibling-reference
invariant the split enforces.
- architecture.md: components and internal-communication tables
updated to show calls flow through Common interfaces.
- architecture_compliance_baseline.md: F1..F5 marked Resolved with
task IDs and commit refs; baseline summary now 0 findings.
- diagrams/components.md, components/03_tile_downloader/description.md,
modules/{common_interfaces,services_tile_service,
services_google_maps_downloader,tests_unit}.md updated for the
split, RateLimitException relocation, and new ITileService methods.
Documentation-only batch — no code, no tests, no build changes.
Epic AZ-309 complete (6 tasks across 3 batches).
Co-authored-by: Cursor <cursoragent@cursor.com>
185 lines
11 KiB
Markdown
185 lines
11 KiB
Markdown
# Satellite Provider — Architecture
|
|
|
|
## Architecture Vision
|
|
|
|
Satellite Provider is a self-hosted .NET 8.0 backend service that pre-downloads, caches, and composites Google Maps satellite imagery for offline use. It runs as a single containerized monolith with PostgreSQL, processing requests asynchronously via in-process queues. The dominant pattern is a layered architecture (API → Services → DataAccess → PostgreSQL) with background hosted services for long-running work.
|
|
|
|
**Components & responsibilities** (each owns its own `.csproj` since AZ-309):
|
|
- **Common** (`SatelliteProvider.Common`) — Shared contracts: DTOs, service interfaces, common exceptions, configuration models, geospatial math
|
|
- **DataAccess** (`SatelliteProvider.DataAccess`) — PostgreSQL persistence via Dapper + DbUp migrations
|
|
- **TileDownloader** (`SatelliteProvider.Services.TileDownloader`) — Provider-agnostic tile acquisition via `ISatelliteDownloader` interface (first implementation: Google Maps) with deduplication, concurrency control, and an in-memory tile-byte cache owned by `TileService`
|
|
- **RegionProcessing** (`SatelliteProvider.Services.RegionProcessing`) — Batch tile downloads for geographic areas, stitching, output generation
|
|
- **RouteManagement** (`SatelliteProvider.Services.RouteManagement`) — Route interpolation, geofenced region generation, consolidated map output
|
|
|
|
The three Layer-3 service components are compile-time siblings: each only references `SatelliteProvider.Common` and `SatelliteProvider.DataAccess`. Cross-component runtime calls flow exclusively through interfaces in `SatelliteProvider.Common.Interfaces`.
|
|
|
|
**Major data flows**:
|
|
- *Tile acquisition*: HTTP request → cache check → Google Maps download → disk + DB persistence
|
|
- *Region processing*: Request queued → background worker calculates tile grid → downloads all tiles → produces CSV/summary/stitched image
|
|
- *Route expansion*: Waypoints → interpolated points every ~200m → geofence filtering → region requests per point → optional ZIP archive
|
|
|
|
**Architectural principles** (inferred):
|
|
- Single-instance deployment, no horizontal scaling requirements (`inferred-from: Channel-based queue, no distributed state`)
|
|
- Immutable tile storage with year-based versioning for cache invalidation (`inferred-from: version column + unique index`)
|
|
- Fire-and-forget async processing with status polling (`inferred-from: queue + background service + status endpoint`)
|
|
- No authentication layer — designed as an internal/trusted network service (`inferred-from: no auth middleware in Program.cs`)
|
|
|
|
**Planned features** (confirmed by user, currently stubs):
|
|
- MGRS endpoint — tile access via Military Grid Reference System coordinates
|
|
- Upload endpoint — UAV nadir camera tile ingestion (Layer 2: orthogonal tiles uploaded post-flight, stored alongside Google Maps Layer 1; most recent layer returned on access)
|
|
|
|
**Drift signals**:
|
|
- `geofence_polygons` mentioned in AGENTS.md as a routes table column but does not exist in schema or entity — documentation drift
|
|
|
|
## 1. System Context
|
|
|
|
**Problem being solved**: A GPS-denied UAV navigation service requires satellite imagery for positioning and route planning without GPS. This service pre-downloads Google Maps satellite tiles (Layer 1) for specified regions and routes, accepts UAV-captured nadir camera imagery uploaded post-flight (Layer 2), and serves the most recent tile layer on access. Tiles are stitched into composite images and packaged for offline use.
|
|
|
|
**System boundaries**: The Satellite Provider is a self-contained backend service. It receives HTTP requests (region/route definitions), downloads tiles from Google Maps, stores them on disk and in PostgreSQL, and produces output files (images, CSVs, ZIPs).
|
|
|
|
**External systems**:
|
|
|
|
| System | Integration Type | Direction | Purpose |
|
|
|--------|-----------------|-----------|---------|
|
|
| Satellite imagery provider (e.g., Google Maps) | HTTPS (tile download) | Outbound | Layer 1 satellite imagery source (provider-agnostic via `ISatelliteDownloader`) |
|
|
| GPS-Denied Service (UAV) | REST API | Inbound | Layer 2 nadir camera tile uploads post-flight |
|
|
| PostgreSQL | TCP (Npgsql) | Both | Tile metadata, region/route state |
|
|
| File System | Local disk | Both | Tile image storage, output artifacts |
|
|
| HTTP Clients | REST API | Inbound | Region/route requests, tile queries |
|
|
|
|
## 2. Technology Stack
|
|
|
|
| Layer | Technology | Version | Rationale |
|
|
|-------|-----------|---------|-----------|
|
|
| Language | C# | 12.0 | .NET ecosystem, strong typing |
|
|
| Framework | ASP.NET Core (Minimal API) | 8.0 | Lightweight HTTP hosting |
|
|
| Database | PostgreSQL | 15+ | Reliable RDBMS, spatial-friendly |
|
|
| ORM | Dapper | latest | Micro-ORM, raw SQL control |
|
|
| Migrations | DbUp | latest | Simple SQL-file-based schema migrations |
|
|
| Image Processing | SixLabors.ImageSharp | 3.1.11 | Cross-platform image manipulation |
|
|
| Logging | Serilog | 8.0.3 | Structured logging with file sinks |
|
|
| Hosting | Docker (docker-compose) | — | Containerized deployment |
|
|
| CI/CD | Woodpecker CI | — | Lightweight self-hosted CI |
|
|
|
|
## 3. Deployment Model
|
|
|
|
**Environments**: Development (docker-compose), Production (Docker)
|
|
|
|
**Infrastructure**:
|
|
- Docker-based containerized deployment
|
|
- PostgreSQL as a separate container
|
|
- Shared volumes for tile storage and output artifacts
|
|
- No cloud provider dependency (self-hosted capable)
|
|
|
|
**Environment-specific configuration**:
|
|
|
|
| Config | Development | Production |
|
|
|--------|-------------|------------|
|
|
| Database | localhost:5432 (Docker) | Container network `db:5432` |
|
|
| Secrets | appsettings.Development.json | Environment variables |
|
|
| Logging | Console + File | File (./logs/) |
|
|
| API URL | http://localhost:5100 | http://0.0.0.0:5100 |
|
|
|
|
## 4. Data Model Overview
|
|
|
|
**Core entities**:
|
|
|
|
| Entity | Description | Owned By Component |
|
|
|--------|-------------|--------------------|
|
|
| Tile | A single satellite image tile with coordinates and zoom | TileDownloader |
|
|
| Region | A square area request with processing status | RegionProcessing |
|
|
| Route | A named path with geofence polygons | RouteManagement |
|
|
| RoutePoint | An individual point (original or interpolated) on a route | RouteManagement |
|
|
|
|
**Key relationships**:
|
|
- Route → RoutePoint: one-to-many (a route has many sequential points)
|
|
- Route → Region: many-to-many via `route_regions` (each route point generates a region)
|
|
- Region → Tile: implicit (a processed region references tiles by coordinate/zoom)
|
|
|
|
**Data flow summary**:
|
|
- Client → API → Queue → BackgroundService → GoogleMaps → FileSystem + DB: tile acquisition pipeline
|
|
- Client → API → RouteService → PointInterpolation → RegionCreation → Queue: route-to-region expansion
|
|
|
|
## 5. Integration Points
|
|
|
|
### Internal Communication
|
|
|
|
| From | To | Protocol | Pattern | Notes |
|
|
|------|----|----------|---------|-------|
|
|
| WebApi | RegionProcessing | In-process queue (Channel) | Fire-and-forget | Request queued, status polled. Uses `IRegionService` / `IRegionRequestQueue` from Common. |
|
|
| WebApi | TileDownloader | `ITileService` (Common interface) | Request-Response | Single-tile reads (`GetOrDownloadTileAsync`) and writes (`DownloadAndStoreSingleTileAsync`) flow through `ITileService` since AZ-310 / AZ-311. No direct dependency on the concrete `GoogleMapsDownloaderV2`. |
|
|
| RegionProcessing | TileDownloader | `ITileService` (Common interface) | Request-Response | Per-tile within region processing. Resolved through DI; no compile-time `ProjectReference` between RegionProcessing and TileDownloader csprojs. |
|
|
| RouteManagement | RegionProcessing | `IRegionService` / `IRegionRequestQueue` (Common interfaces) | Fire-and-forget | Route regions submitted to queue. No compile-time `ProjectReference` between RouteManagement and RegionProcessing csprojs. |
|
|
| All Services | DataAccess | Direct method call (via repository interfaces) | Repository pattern | Dapper queries |
|
|
|
|
### External Integrations
|
|
|
|
| External System | Protocol | Auth | Rate Limits | Failure Mode |
|
|
|----------------|----------|------|-------------|--------------|
|
|
| Satellite imagery provider (abstracted via `ISatelliteDownloader`; first implementation: Google Maps) | HTTPS GET | Provider-specific (e.g., session token) | Configured concurrency (MaxConcurrentDownloads) | Retry with backoff, mark region failed |
|
|
|
|
## 6. Non-Functional Requirements
|
|
|
|
| Requirement | Target | Measurement | Priority |
|
|
|------------|--------|-------------|----------|
|
|
| Concurrent Downloads | 4 (configurable) | SemaphoreSlim limit | High |
|
|
| Concurrent Regions | 20 (configurable) | Processing config | Medium |
|
|
| Queue Capacity | 1000 requests | Channel bounded capacity | Medium |
|
|
| Tile Deduplication | 100% (no re-download) | DB lookup before fetch | High |
|
|
| Max Zip Size | 50 MB | Route zip output | Medium |
|
|
|
|
## 7. Security Architecture
|
|
|
|
**Authentication**: None (internal service, no auth layer)
|
|
|
|
**Authorization**: None (all endpoints are open)
|
|
|
|
**Data protection**:
|
|
- At rest: No encryption (tiles stored as plain JPEG files)
|
|
- In transit: HTTPS for Google Maps calls; API itself on HTTP
|
|
- Secrets management: Google Maps session token in appsettings / env vars
|
|
|
|
**Audit logging**: Serilog writes to file; logs exceptions and processing state transitions
|
|
|
|
## 8. Key Architectural Decisions
|
|
|
|
### ADR-001: Minimal API over Controller-based
|
|
|
|
**Context**: Project needed a lightweight HTTP layer for a small set of endpoints.
|
|
|
|
**Decision**: Use ASP.NET Core Minimal APIs (no controllers, no MVC).
|
|
|
|
**Consequences**: Less ceremony, all routing in `Program.cs`, but less structure for future growth.
|
|
|
|
### ADR-002: Dapper over Entity Framework
|
|
|
|
**Context**: Database access is straightforward CRUD with some spatial queries.
|
|
|
|
**Decision**: Use Dapper for raw SQL control and performance, paired with DbUp for schema migrations.
|
|
|
|
**Consequences**: Full SQL control, no ORM overhead; trade-off is manual mapping and no change tracking.
|
|
|
|
### ADR-003: In-Process Queue over External Message Broker
|
|
|
|
**Context**: Region/route processing needs to be asynchronous but the system is a single service.
|
|
|
|
**Decision**: Use `System.Threading.Channels` as an in-process bounded queue.
|
|
|
|
**Consequences**: Simple, no external dependencies; but limited to single-instance deployment — no horizontal scaling of workers.
|
|
|
|
### ADR-004: File-Based Tile Storage
|
|
|
|
**Context**: Tiles are immutable JPEG images that need fast random access.
|
|
|
|
**Decision**: Store tiles as files in a directory hierarchy (`./tiles/{zoom}/{x}/{y}.jpg`) with metadata in PostgreSQL.
|
|
|
|
**Consequences**: Fast reads, easy backup/migration, but requires shared filesystem for multi-instance (which is not currently needed).
|
|
|
|
### ADR-005: Background Hosted Services for Processing
|
|
|
|
**Context**: Region and route processing is long-running and should not block HTTP requests.
|
|
|
|
**Decision**: Use `IHostedService` implementations that consume from the in-process queue.
|
|
|
|
**Consequences**: Clean separation of request handling and processing; lifecycle managed by the host.
|