mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-21 23:51:13 +00:00
Oleksandr Bezdieniezhnykh
8e15e53782
Carries forward new-task research + solution drafts under
_docs/02_task_plans/uav-batch-upload/ that were not included in
the Step 9 task-spec commit (42a3cc7). Also marks the autodev
state as Step 10 in_progress for cycle 2 implementation.
Co-authored-by: Cursor <cursoragent@cursor.com>
96 lines
9.2 KiB
Markdown
96 lines
9.2 KiB
Markdown
# Research Problem — UAV Batch Upload (AZ-485 family)
|
||
|
||
## Context
|
||
|
||
- **Project**: SatelliteProvider — .NET 8.0 ASP.NET Core service that downloads/persists satellite imagery tiles. Currently single-source (Google Maps); AZ-484 just shipped multi-source storage (`source` + `captured_at` columns, 5-col unique index, `tile-storage` v1.0.0 frozen contract).
|
||
- **Cycle**: 2 (started after cycle 1 closed AZ-484 today, 2026-05-11).
|
||
- **Task family** (4 tasks under epic AZ-483, accepted by user):
|
||
- **AZ-485** — UAV batch upload endpoint + per-tile persistence loop (5 SP)
|
||
- **AZ-486** — Quality gate (6 checks + thresholds + reject reasons) (3 SP)
|
||
- **AZ-487** — JWT auth middleware (3 SP)
|
||
- **AZ-488** — Geofence whitelist (2 SP)
|
||
- **Confirmed user requirements** (from new-task Step 1):
|
||
1. Batch upload (multipart with metadata + N image files per request)
|
||
2. Minimum metadata only per tile: `latitude`, `longitude`, `tile_zoom`, `tile_size_meters`, `captured_at` (UTC), image bytes
|
||
3. Quality gate enforces all of: dimensions, byte size, blank/uniform detection, captured_at age, format=JPEG, geofence containment
|
||
4. JWT auth (HS256, signing key in env, required claims: `sub` + valid `exp`)
|
||
5. Sync responses: 200 + per-tile `{ tileId, status, reason? }` array; 4xx for auth/format-of-batch failures
|
||
6. File storage: same `./tiles/{z}/{x}/{y}.jpg` layout as Google Maps tiles
|
||
- **Project constraints to respect**:
|
||
- .NET 8.0 only (no .NET 9 features)
|
||
- Existing libraries: ImageSharp 3.1.11, Npgsql 9.0.2, Dapper 2.1.35, Serilog 8.0.3
|
||
- No auth currently exists in the API (per Security Audit Step 14, 2026-05-11)
|
||
- The "geofence" type in the codebase (`SatelliteProvider.Common.DTO.GeofencePolygon`) is actually a bounding box (NorthWest + SouthEast corners), not an arbitrary polygon. There is no existing point-in-polygon implementation.
|
||
- The frozen `_docs/02_document/contracts/data-access/tile-storage.md` v1.0.0 is the producer-side contract this work must implement (`source='uav'`, `captured_at` per-row, per-source UPSERT)
|
||
- Lessons applied: L-001 (Dapper enum bypass — no new persisted enums planned but the rule applies if any sneak in)
|
||
|
||
## Specific unknowns to investigate
|
||
|
||
### Q1 — JWT auth middleware in ASP.NET Core 8 for an internal service (AZ-487)
|
||
- Recommended pattern for an internal/trusted-network API that needs to validate HS256 tokens issued by an external system?
|
||
- `Microsoft.AspNetCore.Authentication.JwtBearer` (built-in, recommended? version-pinned for ASP.NET Core 8)
|
||
- vs. third-party (e.g., custom middleware, OpenIddict, etc.)
|
||
- Configuration shape — where does the signing key live? appsettings vs env var vs key vault? (Project constraint: this codebase has no secret manager today.)
|
||
- Required claims minimum: just `sub` and unexpired `exp` per user choice. Confirm `JwtBearerOptions.TokenValidationParameters` setup that validates *only* signature + `exp` and tolerates missing `aud`/`iss` (or reject — which is the safer ASP.NET Core 8 default?).
|
||
- Swagger integration: how to surface "Bearer <token>" in the Swagger UI for the upload endpoint without changing the existing public endpoints?
|
||
- 401 response shape — does the global `GlobalExceptionHandler` need to know about it, or does `[Authorize]` short-circuit before the handler?
|
||
- Test strategy — how do current `SatelliteProvider.IntegrationTests` (which call the live API via `HttpClient` per `RouteTestHelpers`) cleanly inject a valid JWT? Pattern for a "test issuer" using the same signing key.
|
||
|
||
### Q2 — Multipart batch upload in ASP.NET Core 8 minimal API (AZ-485)
|
||
- Idiomatic shape for `multipart/form-data` with N files + a JSON metadata document in a minimal API endpoint:
|
||
- `[FromForm]` model binding
|
||
- vs. manual `MultipartReader` for streaming (relevant if batches are large)
|
||
- Request size limits in ASP.NET Core 8 (Kestrel `MaxRequestBodySize` and form options) — defaults and how to override for the upload endpoint only.
|
||
- Memory / streaming trade-off: how big can the per-batch `IFormFile[]` get before we should switch to streaming via `MultipartReader`? Practical batch-size guidance for satellite tiles (~50–500 KB each).
|
||
- Response shape for partial-success batches — common patterns (HTTP 200 with mixed array, HTTP 207 Multi-Status, HTTP 422 with per-item errors).
|
||
- Backpressure / concurrency limit on the per-tile persistence loop — should this be sequential or fanned out? Trade-off vs. Postgres connection pool size (current Npgsql defaults).
|
||
|
||
### Q3 — Image quality heuristics for satellite tiles using ImageSharp 3.1.11 (AZ-486)
|
||
- API surface in **ImageSharp 3.1.11 specifically** for:
|
||
- Reading width/height without full decode (`Image.IdentifyAsync` returns `ImageInfo`)
|
||
- Detecting actual format from magic bytes vs. trusting `Content-Type`
|
||
- Computing a "uniformity / blankness" metric efficiently — luminance stddev across the image, or histogram-based, or a perceptual hash
|
||
- Performance budget: per-tile quality check ideally under ~10ms on a 256×256 JPEG. Confirm which ImageSharp APIs are streaming vs. full-decode.
|
||
- Robustness against adversarial inputs — e.g., a 1-pixel image with a JPEG header. What does ImageSharp 3.1.11 throw, and is there a safe-decode pattern?
|
||
- Threshold-tuning approach — recommended way to tune the blank-detection threshold against a small fixture set of real satellite tiles vs. uniform tiles, without having that fixture set today.
|
||
|
||
### Q4 — Geofence whitelist: bbox vs. polygon (AZ-488)
|
||
- The codebase's existing geofence type is a **bounding box** (`NorthWest`+`SouthEast`). Two options for the AZ-488 whitelist:
|
||
- **Option A**: reuse the existing bbox shape (simple, consistent with current code, but coarser — a UAV tile inside a non-rectangular real-world allowed area can't be expressed)
|
||
- **Option B**: introduce a true polygon (lat/lon list) + ray-casting point-in-polygon, store as JSON in config. Requires writing the ray-cast utility (or pulling in `NetTopologySuite`).
|
||
- Recommendation: which one is the right fit given that the user only said "geofence" without specifying shape and the existing code is bbox? Compare with NetTopologySuite as a third option.
|
||
- Storage: where does the whitelist live? `appsettings.json` array, dedicated config file, dedicated DB table?
|
||
- Performance: typical UAV batch is N tiles vs. M whitelist polygons — is M small enough (e.g., < 10 polygons, < 1000 vertices) that O(N×M) ray-cast per batch is fine?
|
||
|
||
### Q5 — Per-tile persistence loop: file-system + DB consistency (AZ-485)
|
||
- Pattern for "write JPEG to disk, then write DB row" with rollback on either failure:
|
||
- Two-phase: write file with a `.tmp` suffix, INSERT, then rename. On INSERT failure, delete `.tmp`.
|
||
- vs. write DB row first (ON CONFLICT does UPSERT), then write file. On file failure, what happens to the row? (Could leave a row pointing at a missing file path — bad.)
|
||
- Idempotency: if the same (lat, lon, zoom, size, source='uav') tile is uploaded twice, the per-source UPSERT already handles the DB side — but does the file get overwritten cleanly? Atomic file replace pattern.
|
||
- Concurrency: two simultaneous batch uploads for adjacent tiles — any shared resource that needs locking?
|
||
|
||
## Out of scope for this research
|
||
|
||
- Distributed UAV fleet management, queueing, or real-time streaming (these are upstream concerns the user has not raised)
|
||
- Async/202 + status polling response model (user explicitly said sync 200)
|
||
- Per-tile encryption, compression beyond JPEG, or alternate image formats
|
||
- External IdP integration (HS256 with shared secret only)
|
||
- UAV mission/operator/sensor metadata (user explicitly said "not necessary for now")
|
||
|
||
## Acceptance criteria for the research output
|
||
|
||
The research deliverable (`solution_draft01.md`) must:
|
||
|
||
1. Recommend a concrete library + config approach for **Q1 (JWT)**. Include: package name + version, code shape for `builder.Services.AddAuthentication().AddJwtBearer(...)`, key storage recommendation, integration-test pattern.
|
||
2. Recommend a concrete request shape + size-limit approach for **Q2 (multipart)**. Include: `[FromForm]` vs streaming decision criteria, response shape choice with rationale, concurrency-limit recommendation.
|
||
3. Recommend a concrete API call sequence in **ImageSharp 3.1.11** for **Q3 (quality heuristics)**. Include: blank-detection threshold starting point + tuning plan, performance budget validation, error-handling pattern.
|
||
4. Recommend bbox-vs-polygon-vs-NTS for **Q4 (geofence)**. Include: exact-fit verdict per the project's actual operating context (M < 10 polygons, simple ray-cast acceptable, or pull in NTS).
|
||
5. Recommend a file+DB consistency pattern for **Q5**. Include: failure modes covered, rollback shape, idempotency story.
|
||
6. Include a "do not use" / rejected list for each question — alternatives the research considered and rejected with one-line evidence.
|
||
|
||
## Execution mode
|
||
|
||
Mode A — Initial Research. Output class: **Technical-component selection** (recommends specific libraries, ASP.NET Core 8 modes, ImageSharp APIs, optional NTS).
|
||
|
||
Per-mode API capability verification applies to every recommended library. Saved Minimum Viable Examples required for: `Microsoft.AspNetCore.Authentication.JwtBearer` JWT validation, ASP.NET Core 8 minimal-API multipart, ImageSharp 3.1.11 luminance stddev.
|