azaion/satellite-provider
1
0
Fork 0
mirror of https://github.com/azaion/satellite-provider.git synced 2026-06-21 17:11:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
8c13cd4f302897252d917bef1813d01ec95e5f0a
satellite-provider/_docs/02_document/modules/api_program.md
T
Oleksandr Bezdieniezhnykh bc04ba7f99 [AZ-794] [AZ-795] [AZ-796] Cycle 7 Steps 12-15 sync (test-spec / docs / security / perf) 
Step 12 (Test-Spec Sync): adds BT-27 for the AZ-796 9-rule
validation surface and 12 cycle-7 AC rows + Coverage Summary
update to traceability-matrix.md.

Step 13 (Update Docs): module-layout + module docs for the new
SatelliteProvider.Api/Validators namespace + GlobalExceptionHandler
+ updated TileInventory DTO; tests_unit + tests_integration
document the new InventoryRequestValidatorTests (16 unit tests
covering all 9 rules) + TileInventoryValidationTests (16
integration tests) + ProblemDetailsAssertions support;
glossary entries for Validation Problem Details / FluentValidation
/ Unmapped Member Handling; system-flows F8 (Tile Inventory Bulk
Lookup) expanded with deserializer + validator gates and a 13-row
Validation Surface table; data_parameters § Tile Inventory
documents the v2 input schema + constraints; ripple_log_cycle7
captures the doc-side ripple decisions.

Step 14 (Security Audit): 5-phase audit ran; verdict
PASS_WITH_WARNINGS (3 Low findings — D-AZ795-1 FluentValidation
12.0.0 -> 12.1.1 recommended bump, F-AZ795-1 JsonException.Message
leak in 400 detail, F-AZ795-2 BadHttpRequestException.Message leak).
No Critical / High; auth runs before validation (confirmed in
Program.cs); two NuGet additions (FluentValidation 12.0.0 +
.DependencyInjectionExtensions 12.0.0) both CVE-clean. Per-phase
reports plus consolidated security_report_cycle7.md.

Step 15 (Performance Test): docker compose stack used for perf
run, scripts/run-performance-tests.sh exited 0 with 8/8 scenarios
PASS (second consecutive clean exit-0); added PT-09 cycle-7 smoke
probe (v2 z/x/y schema, 2500-tile all-miss batch) measuring
min=27ms median=44ms p95=73ms max=86ms (13.7x under AZ-505 AC-4
1000ms budget). PT-07/08 improvements traced to the cycle-6 TLS
handshake-overhead identification, not application-side change.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 11:24:27 +03:00

136 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Module: Api/Program.cs
## Purpose
Application entry point. Configures DI container, sets up middleware, defines minimal API endpoints, runs database migrations on startup, and starts background services.
## Public Interface
### API Endpoints
| Method | Route | Handler | Description |
|--------|-------|---------|-------------|
| GET | `/tiles/{z}/{x}/{y}` | `ServeTile` | Slippy map tile server with in-memory caching. AZ-505 rewired the DB lookup to filter on `location_hash` (deterministic UUIDv5) so the read becomes an `Index Only Scan` against `tiles_leaflet_path`; the wire response is byte-identical to pre-AZ-505. |
| GET | `/api/satellite/tiles/latlon` | `GetTileByLatLon` | Download single tile by lat/lon/zoom |
| POST | `/api/satellite/tiles/inventory` | `GetTilesInventory` | Bulk tile-existence/metadata lookup (AZ-505) — body is XOR of `tiles[{z,x,y}]` (Form A) and `locationHashes[uuid]` (Form B), each capped at 5000 entries. Response is one entry per request entry, in input order. AZ-794 (cycle 7) renamed the coord triple from `tileZoom/tileX/tileY` → `z/x/y` (OSM convention); AZ-796 (cycle 7) added strict input validation via `WithValidation<TileInventoryRequest>()` so malformed payloads return RFC 7807 `ValidationProblemDetails` instead of silently coercing to zero. Contracts: `_docs/02_document/contracts/api/tile-inventory.md` v2.0.0 + `_docs/02_document/contracts/api/error-shape.md` v1.0.0. |
| GET | `/api/satellite/tiles/mgrs` | `GetSatelliteTilesByMgrs` | MGRS stub (returns empty) |
| POST | `/api/satellite/upload` | `UploadUavTileBatch` | UAV tile batch upload (AZ-488) — multipart envelope, 5-rule quality gate, per-source UPSERT with `source='uav'`. Requires the `RequiresGpsPermission` policy. |
| POST | `/api/satellite/request` | `RequestRegion` | Queue region for async tile processing |
| GET | `/api/satellite/region/{id}` | `GetRegionStatus` | Get region processing status |
| POST | `/api/satellite/route` | `CreateRoute` | Create route with intermediate points |
| GET | `/api/satellite/route/{id}` | `GetRoute` | Get route with all points |
### Local Records (defined in Program.cs)
- `GetSatelliteTilesResponse`, `SatelliteTile` — MGRS response stubs
- `DownloadTileResponse` — tile download response
- `RequestRegionRequest` — region request body
- `ParameterDescriptionFilter` — Swagger operation filter
### Api/DTOs (AZ-488)
- `UavTileBatchUploadRequest` — multipart envelope with `metadata` (JSON string) and `files` (`IFormFileCollection`)
### Common/DTO (AZ-488)
- `UavTileMetadata`, `UavTileBatchMetadataPayload` — per-item metadata + envelope shape
- `UavTileBatchUploadResponse`, `UavTileUploadResultItem` — per-item response shape
- `UavTileUploadStatus`, `UavTileRejectReasons` — string-constant enumerations exposed in the v1.0.0 contract
### Common/DTO (AZ-505; renamed by AZ-794 in cycle 7)
- `TileInventoryRequest` — XOR body envelope with `Tiles` (Form A) OR `LocationHashes` (Form B)
- `TileCoord` — `{Z, X, Y}` per-entry coord under Form A. Each property is marked `[JsonRequired]` so missing axes surface as `400` at the deserializer layer (System.Text.Json throws, `GlobalExceptionHandler` converts to `ValidationProblemDetails`).
- `TileInventoryResponse` — `{Results: TileInventoryEntry[]}` response shape; ordering matches request
- `TileInventoryEntry` — per-entry response shape (`Z`, `X`, `Y`, `LocationHash`, `Present`, optional `Id`/`CapturedAt`/`Source`/`FlightId`/`ResolutionMPerPx`)
- `TileInventoryLimits.MaxEntriesPerRequest` — hard cap (5000) consumed by `InventoryRequestValidator`
### Api/Validators (AZ-795 + AZ-796, cycle 7)
- `InventoryRequestValidator` — FluentValidation `AbstractValidator<TileInventoryRequest>`. Rules: XOR `tiles`/`locationHashes`, `tiles.Count ≤ MaxEntriesPerRequest`, `locationHashes.Count ≤ MaxEntriesPerRequest`, per-entry `TileCoordValidator`.
- `TileCoordValidator` — per-entry rules: `Z` ∈ [0, 22] (slippy-map range), `X` ∈ [0, 2^Z), `Y` ∈ [0, 2^Z).
- `ValidationEndpointFilter<T>` — generic minimal-API filter that resolves `IValidator<T>` from DI, runs it against the bound argument, and returns `Results.ValidationProblem(result.ToDictionary())` on failure. Wired per-endpoint via `RouteHandlerBuilder.WithValidation<T>()`.
- `GlobalValidatorConfig.ApplyOnce()` — idempotent process-wide FluentValidation configuration. Sets `ValidatorOptions.Global.PropertyNameResolver` so error map keys are camelCase per `error-shape.md` Inv-4. Called from `Program.cs` and from the test assembly's `ValidatorTestModuleInitializer` so both contexts see identical key shapes.
### Api/GlobalExceptionHandler (AZ-795, cycle 7)
- `GlobalExceptionHandler : IExceptionHandler` — registered via `AddExceptionHandler<GlobalExceptionHandler>()` + `AddProblemDetails()`. Intercepts unhandled exceptions and converts `BadHttpRequestException(JsonException)` (unknown-member rejection, missing-required-field, type mismatch) into RFC 7807 `ValidationProblemDetails` matching the FluentValidation output shape (single source of truth — see `error-shape.md` v1.0.0 §"Both paths produce identically-shaped bodies"). 5xx errors pass through with sanitised body + `correlationId` (preserves AZ-353).
## Internal Logic
### DI Registration
1. Serilog configured from `appsettings.json`
2. Connection string extracted from `ConnectionStrings:DefaultConnection`
3. Config bindings: `MapConfig`, `StorageConfig`, `ProcessingConfig`, `UavQualityConfig` (AZ-488)
4. **Request size limits (AZ-488)**: `KestrelServerOptions.Limits.MaxRequestBodySize` and `FormOptions.MultipartBodyLengthLimit` are set to `UavQualityConfig.MaxBatchSize × UavQualityConfig.MaxBytes` (default 100 × 5 MiB = 500 MiB) so an oversized UAV batch is rejected at the framework layer before reaching the handler.
5. Singletons: repositories (`TileRepository`, `RegionRepository`, `RouteRepository`), `GoogleMapsDownloaderV2`, `ITileService`, `IRegionService`, `IRouteService`, `IUavTileQualityGate`, `IUavTileUploadHandler` (AZ-488)
6. `IRegionRequestQueue` with configurable capacity
7. Hosted services: `RegionProcessingService`, `RouteProcessingService`
8. CORS policy: `TilesCors` — configured origins from `CorsConfig:AllowedOrigins`, falls back to allow-any
9. JSON options: camelCase, case-insensitive
10. **JWT authentication (AZ-487 + AZ-494)**: `AddSatelliteJwt(builder.Configuration)` (extension in `SatelliteProvider.Api.Authentication`) registers `JwtBearer` with `TokenValidationParameters` set per the suite auth contract: signature + lifetime + issuer + audience validation, 30 s clock skew, ≥ 32-byte HMAC key. The `iss` value comes from `JWT_ISSUER` env (fallback `Jwt:Issuer` config); the `aud` value comes from `JWT_AUDIENCE` env (fallback `Jwt:Audience` config). All three values (secret, iss, aud) are fail-fast — the API throws `InvalidOperationException` at startup if any is unset or whitespace-only. Production deploys MUST set the env vars with admin-team-confirmed values; `appsettings.json` ships empty so the fail-fast triggers. `appsettings.Development.json` ships clearly-tagged DEV-ONLY values (`DEV-ONLY-iss-admin-azaion-local` / `DEV-ONLY-aud-satellite-provider`) so local dev works out-of-the-box. Followed by `AddAuthorization` with the `RequiresGpsPermission` policy (AZ-488).
11. **Kestrel HTTP/2 (AZ-505)**: `builder.WebHost.ConfigureKestrel(opts => opts.ConfigureEndpointDefaults(lo => lo.Protocols = HttpProtocols.Http1AndHttp2))`. The dev listener is now `https://+:8080` with a self-signed cert (`./certs/api.pfx`, generated idempotently by `scripts/run-tests.sh` and bound via `ASPNETCORE_Kestrel__Certificates__Default__Path` / `__Password` in `docker-compose.yml`). Kestrel needs TLS for HTTP/2 protocol negotiation; ALPN advertises both `h2` and `http/1.1` so HTTP/2-capable clients (browser Leaflet, `HttpClient` with `Version20` + `RequestVersionExact`, httpx `http2=True`) multiplex tile reads on a single TLS connection, and legacy clients fall back to HTTP/1.1. The integration-test container trusts the dev cert via `/usr/local/share/ca-certificates/` + `update-ca-certificates`. AZ-505 AC-5 verifies the multiplex semantics here; production termination is expected at the ingress (Envoy / nginx / ALB) — Kestrel can then drop to HTTP/2 cleartext behind it without changing this code.
12. **ProblemDetails + global exception handler (AZ-795, cycle 7)**: `AddProblemDetails()` + `AddExceptionHandler<GlobalExceptionHandler>()` register the uniform RFC 7807 error pipeline. `app.UseExceptionHandler()` (in the middleware chain) routes unhandled exceptions through `GlobalExceptionHandler`, which converts `BadHttpRequestException(JsonException)` (unknown-member rejection, missing-required-field, JSON type mismatch) into `ValidationProblemDetails` with the same `errors[]` map shape that FluentValidation produces. This is the deserializer-layer half of the strict-validation contract — `error-shape.md` v1.0.0 §"Two collaborating pieces of shared infrastructure".
13. **Strict JSON parsing (AZ-795, cycle 7)**: `ConfigureHttpJsonOptions` sets `PropertyNamingPolicy = CamelCase`, `PropertyNameCaseInsensitive = true`, `UnmappedMemberHandling = Disallow`, and adds `JsonStringEnumConverter` with camelCase naming. `UnmappedMemberHandling.Disallow` is the key strict-parsing knob: any unknown root or nested field is rejected at the deserializer rather than silently dropped. Catches typos (`{"Z":12}` uppercase, `{"tileZoom":...}` post-rename) that no FluentValidation rule can see after deserialization.
14. **FluentValidation registration (AZ-795 + AZ-796, cycle 7)**: `AddValidatorsFromAssemblyContaining<Program>()` auto-registers every `IValidator<T>` in the API assembly (currently `InventoryRequestValidator` + `TileCoordValidator`). `GlobalValidatorConfig.ApplyOnce()` runs the idempotent process-wide config — sets `ValidatorOptions.Global.PropertyNameResolver` so `errors` map keys are camelCase (matches the request body's casing per `error-shape.md` Inv-4). Per-endpoint opt-in via `.WithValidation<TileInventoryRequest>()` on the inventory MapPost — the generic `ValidationEndpointFilter<T>` resolves the validator from DI at request time and returns `Results.ValidationProblem` on failure.
### Startup
1. Database migration via `DatabaseMigrator.RunMigrations()` — throws on failure
2. Creates tiles and ready directories
3. Swagger enabled in Development mode
4. Middleware chain (order matters): `UseExceptionHandler` → `UseHttpsRedirection` → `UseCors("TilesCors")` → `UseAuthentication` → `UseAuthorization` → endpoint mapping.
5. Every `MapGet`/`MapPost` endpoint is decorated with `.RequireAuthorization()`; the framework returns 401 before the handler runs for any anonymous, expired, or invalid-signature request.
### ServeTile Handler
1. Checks `IMemoryCache` for tile bytes (1h absolute, 30min sliding expiration)
2. If cache miss: queries `ITileRepository.GetByTileCoordinatesAsync` — AZ-505 rewired this method to compute `location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}")` and filter by `WHERE location_hash = $1`, hitting `tiles_leaflet_path` as an `Index Only Scan` with `Heap Fetches ≤ 1`. Selection rule is unchanged (most-recent across sources/flights); wire response is byte-identical.
3. If no DB record: downloads tile via `GoogleMapsDownloaderV2.DownloadSingleTileAsync`, creates `TileEntity`, inserts
4. Returns image bytes with cache headers (`Cache-Control: public, max-age=86400`)
### GetTilesInventory Handler (AZ-505 + AZ-796 cycle 7)
1. **Pre-handler validation (cycle 7)**: `ValidationEndpointFilter<TileInventoryRequest>` runs BEFORE the handler. Resolves `InventoryRequestValidator` from DI and asserts XOR `tiles`/`locationHashes`, per-array cap (`TileInventoryLimits.MaxEntriesPerRequest = 5000`), `z` ∈ [0, 22], `x` ∈ [0, 2^z), `y` ∈ [0, 2^z) per entry. Any failure short-circuits with HTTP 400 + `ValidationProblemDetails`. Deserializer-layer failures (missing `z/x/y`, unknown root/nested fields, JSON type mismatch) are caught earlier by System.Text.Json and surfaced as identically-shaped `ValidationProblemDetails` via `GlobalExceptionHandler` (AZ-795).
2. Handler delegates to `ITileService.GetInventoryAsync(request, ct)` — body of the handler is just the service call + `Results.Ok`.
3. Service computes `location_hash` for Form A entries via `Uuidv5.Create(TileNamespace, "{z}/{x}/{y}")`, calls `ITileRepository.GetTilesByLocationHashesAsync(IReadOnlyList<Guid>)`, re-aligns results back to input order.
4. Returns `TileInventoryResponse` with one entry per input — `present=true` entries carry `id` / `capturedAt` / `source` / `flightId` / `resolutionMPerPx`; `present=false` entries carry only `locationHash`.
5. Authenticated by `.RequireAuthorization()` (401 before validation runs for anonymous requests).
### GetTileByLatLon Handler
Downloads a tile, persists it, returns metadata as `DownloadTileResponse`.
### RequestRegion Handler
Validates size (100–10000m), delegates to `IRegionService.RequestRegionAsync`.
### UploadUavTileBatch Handler (AZ-488)
Buffers each `IFormFile` into memory, packages them as `UavUploadFile` records (filename, content-type, bytes), and delegates to `IUavTileUploadHandler.HandleAsync`. Envelope-level errors (mismatched batch, oversized batch, malformed metadata) are surfaced as HTTP 400 ProblemDetails; per-item rejects are returned in the HTTP 200 response payload. The endpoint is protected by `.RequireAuthorization(SatellitePermissions.UavUploadPolicy)` so 401 (no token) and 403 (no `GPS` permission) are returned before the handler runs.
## Dependencies
All project references: Common, DataAccess, Services.
NuGet: `Serilog.AspNetCore` (8.0.3 — fallback retained on .NET 10 per AZ-500 Risk #4: no 10.x line published as of cycle 4; documented in `AGENTS.md`), `Swashbuckle.AspNetCore` (10.1.7 — bumped from 6.6.2 by AZ-500 to land Microsoft.OpenApi 2.x compat required by ASP.NET Core 10), `Microsoft.AspNetCore.OpenApi` (10.0.7 — bumped from 8.0.25 by AZ-500), `Microsoft.AspNetCore.Authentication.JwtBearer` (10.0.7 — added at 8.0.21 by AZ-487, bumped to 8.0.25 by AZ-496, bumped to 10.0.7 by AZ-500), `FluentValidation` + `FluentValidation.DependencyInjectionExtensions` (12.0.0 — added by AZ-795 to back the strict-input-validation epic), `SixLabors.ImageSharp`, `Newtonsoft.Json`.
**Microsoft.OpenApi 2.x refactor note (AZ-500)**: the major bump (1.x → 2.x) drove three internal Swashbuckle-setup edits in this file — `using Microsoft.OpenApi.Models;` → `using Microsoft.OpenApi;`; `AddSecurityRequirement(...)` rewritten to take a `Func<OpenApiDocument, OpenApiSecurityRequirement>` and use `OpenApiSecuritySchemeReference("Bearer")` instead of the removed `OpenApiSecurityScheme.Reference` shape; `MapType<UavTileBatchUploadRequest>` rewritten to use the new `JsonSchemaType` enum and `IDictionary<string, IOpenApiSchema>` properties bag. The Swagger document shape (paths, operations, the Bearer Authorize button, the multipart-batch upload schema) is preserved exactly — `SwaggerDocument_AdvertisesBearerSecurityScheme` and the AZ-353 swagger-ready integration assertions still pass. Eight `ASPDEPR002` deprecation warnings (`WithOpenApi(...)`) remain — they're recorded in `_docs/03_implementation/reviews/batch_01_cycle4_review.md` as a follow-up PBI; the API is still fully functional in .NET 10 (deprecated, not removed).
## Consumers
- HTTP clients (external)
- Integration tests (via HTTP)
## Data Models
Defines several local request/response records that are not shared with other projects.
## Configuration
All configuration sections are consumed here:
- `ConnectionStrings:DefaultConnection`
- `MapConfig`, `StorageConfig`, `ProcessingConfig`
- `UavQuality` (AZ-488) — `MinBytes`, `MaxBytes`, `MaxAgeDays`, `CapturedAtFutureSkewSeconds`, `MinLuminanceVariance`, `MaxBatchSize`, `LuminanceSampleSize`. Drives the 5-rule quality gate AND the per-request body-size limits.
- `CorsConfig:AllowedOrigins`
- `Jwt:Secret` — HMAC-SHA256 signing key for JWT validation (AZ-487). Resolution: `JWT_SECRET` env var (preferred, opaque production secret) → `Jwt:Secret` configuration key (`appsettings.Development.json` placeholder only). Startup fails fast if the resolved value is unset, empty, or shorter than 32 bytes.
- `Jwt:Issuer` — Expected `iss` claim value (AZ-494). Resolution: `JWT_ISSUER` env → `Jwt:Issuer` config. Startup fails fast if unset/empty.
- `Jwt:Audience` — Expected `aud` claim value (AZ-494). Resolution: `JWT_AUDIENCE` env → `Jwt:Audience` config. Startup fails fast if unset/empty.
- `Serilog` section
## External Integrations
- Google Maps (indirectly via `GoogleMapsDownloaderV2`)
- PostgreSQL (via repositories and DatabaseMigrator)
- File system (`./tiles/`, `./ready/`)
## Security
- CORS configured (permissive by default when no origins specified)
- Swagger only in Development; Bearer token "Authorize" button registered via `AddSecurityDefinition`/`AddSecurityRequirement` (AZ-487)
- HTTPS redirection enabled
- JWT bearer authentication (AZ-487) — every endpoint requires a valid HS256-signed token. Anonymous, expired, or signature-tampered requests return 401 before the handler runs.
- Permission-claim policies (AZ-488) — `POST /api/satellite/upload` is wrapped in `.RequireAuthorization(SatellitePermissions.UavUploadPolicy)`. The `PermissionsAuthorizationHandler` reads the `permissions` claim (repeated-string OR JSON-array shape) and returns 403 if `GPS` is not present.
## Tests
Integration tests exercise all endpoints. Unit test project has only a dummy test.
Reference in New Issue View Git Blame Copy Permalink