mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-21 05:41:14 +00:00
chore: cycle 2 step 9 task plan artifacts + step 10 state
Carries forward new-task research + solution drafts under
_docs/02_task_plans/uav-batch-upload/ that were not included in
the Step 9 task-spec commit (42a3cc7). Also marks the autodev
state as Step 10 in_progress for cycle 2 implementation.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,49 @@
|
||||
# Acceptance Criteria Assessment — UAV Batch Upload
|
||||
|
||||
**Date**: 2026-05-11
|
||||
**Mode**: Research Phase 1 (AC & Restrictions Assessment)
|
||||
**Result**: No changes required — user explicitly validated all 5 critical assumptions in the new-task Step 2 BLOCKING gate immediately preceding this research run.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
| Criterion | Our Values | Researched Values | Cost/Timeline Impact | Status |
|
||||
|-----------|------------|-------------------|----------------------|--------|
|
||||
| Per-tile minimum metadata | lat, lon, tile_zoom, tile_size_meters, captured_at, image bytes | Industry-standard for tile uploads (TMS / OGC Tile API) defines x/y/zoom + time + payload as the minimum viable set | None — matches industry norms | Confirmed |
|
||||
| Sync 200 + per-tile mixed array response | 200 OK with `[{tileId, status, reason?}]`; 4xx for batch-level failures only | Common pattern in batch APIs; alternative HTTP 207 Multi-Status (RFC 4918) is more correct semantically but rarely used outside WebDAV ecosystems and adds caller complexity for no real gain | None — sync 200 chosen, no change | Confirmed |
|
||||
| JWT HS256 with `sub` + `exp` only | minimum claims model | NIST SP 800-204A and OAuth 2.0 BCP recommend `iss` + `aud` + `exp` + `sub` for federated; for an internal service with shared HS256 secret, `sub` + `exp` is the documented "trusted-network" minimum (microsoft-jwt-bearer docs) | None | Confirmed |
|
||||
| Quality gate verdict response-only (not persisted) | string in response, no DB column | Avoids triggering L-001 (Dapper enum bypass); reject reasons are inherently transient — re-uploading should re-run the gate | None — saves a DB column | Confirmed |
|
||||
| File storage = same `./tiles/{z}/{x}/{y}.jpg` layout as Google Maps | reuse existing layout | The 5-col unique index `(lat, lon, tile_zoom, tile_size_meters, source)` already keeps UAV and GMaps rows separate at the DB level; the file path can collide BUT the per-source UPSERT will overwrite the file owner deterministically (most-recent wins). Single-file-per-cell layout is acceptable IF we agree that the reader gets the most-recently-uploaded JPEG regardless of source. | Acceptable; flagged as a Risk in the eventual task spec — the user may want UAV files in `./tiles/uav/{z}/{x}/{y}.jpg` to keep both bytes addressable for forensics | **Recommend re-confirm in Step 5 Validate Assumptions** |
|
||||
|
||||
## Restrictions Assessment
|
||||
|
||||
| Restriction | Our Values | Researched Values | Cost/Timeline Impact | Status |
|
||||
|-------------|------------|-------------------|----------------------|--------|
|
||||
| .NET 8.0 only | constraint | LTS until November 2026; all libraries researched (`Microsoft.AspNetCore.Authentication.JwtBearer 8.x`, `SixLabors.ImageSharp 3.1.x`) support .NET 8 | None | Confirmed |
|
||||
| ImageSharp 3.1.11 (existing pin) | constraint | 3.1.11 is current stable in the 3.1 LTS line; APIs for `Image.Identify`, `Image<TPixel>` enumeration are stable since 3.0 | None | Confirmed |
|
||||
| Npgsql 9.0.2, Dapper 2.1.35 | constraint | No version-specific concerns for the per-source UPSERT path already in `TileRepository` | None | Confirmed |
|
||||
| No secret manager today | constraint | Forces JWT signing key into env var or appsettings (not Key Vault). User must accept this trade-off; flagged in Security Audit too | None for this cycle | Confirmed |
|
||||
| Existing `GeofencePolygon` DTO is bbox not polygon | implicit constraint discovered during pre-research grep | Reusing the bbox shape for AZ-488 is consistent with current code; introducing true polygons is a bigger change | Could push AZ-488 from 2 SP to 4-5 SP if true polygons chosen | **Material to AZ-488 sizing — see Q4 in the eventual draft** |
|
||||
|
||||
## Key Findings
|
||||
|
||||
1. **One genuine open assumption to re-confirm** in new-task Step 5: file storage layout. Same-path-as-GMaps lets the latest source win at the file level too (consistent with the DB read rule), but loses bytes-level forensics for the loser. A separate `./tiles/uav/...` path keeps both — at the cost of a divergent storage convention. The user said "yes, as google maps tiles" in Step 1 so the default is the same path; flagging it once for the spec author.
|
||||
|
||||
2. **AZ-488 may be larger than 2 SP** if user wants real polygons. The AZ-485 family research pass should present both options with sizing impact so the user can decide once at spec time, not later during implementation.
|
||||
|
||||
3. **No additional AC needed** — the 5 assumptions cover the deliverable surface for this task family. No regulatory / compliance dimension surfaced (this is an internal API — not a public consumer-facing endpoint subject to data-handling regs).
|
||||
|
||||
4. **No restrictions to relax** — every constraint listed is forced by either project state (existing libs, no auth, no secret manager) or user decision (sync 200, minimum metadata).
|
||||
|
||||
## Sources
|
||||
|
||||
- IETF RFC 4918 (HTTP 207 Multi-Status) — read for the response-shape decision; concluded not applicable
|
||||
- NIST SP 800-204A § 3.4 (Microservice JWT validation) — confirms minimum-claims pattern is acceptable for internal trusted-network services
|
||||
- `Microsoft.AspNetCore.Authentication.JwtBearer` Microsoft Learn page (verified to be the active 8.x line)
|
||||
- `SixLabors.ImageSharp` 3.1 release notes on `Image.Identify` stability
|
||||
- Codebase grep for existing geofence implementation (read locally — no external source)
|
||||
|
||||
## BLOCKING decision
|
||||
|
||||
**Phase 1 closes without re-blocking the user.** All 5 user-confirmed assumptions stand. The single re-confirmation item (file storage path) will surface naturally at new-task Step 5 (Validate Assumptions); the research draft will pre-fill both options with their trade-offs so the user can decide in one click.
|
||||
|
||||
Proceeding to Phase 2 (Problem Research & Solution Draft).
|
||||
@@ -0,0 +1,358 @@
|
||||
# Solution Draft 01 — UAV Batch Upload (AZ-485 family)
|
||||
|
||||
**Date**: 2026-05-11
|
||||
**Mode**: Research Mode A (Initial)
|
||||
**Output Class**: Technical-component selection
|
||||
**Inputs**: `_docs/02_task_plans/uav-batch-upload/problem.md`; `_docs/02_task_plans/uav-batch-upload/00_research/00_ac_assessment.md`
|
||||
**Status**: Draft 01 (no prior drafts to assess)
|
||||
|
||||
## Core conclusion
|
||||
|
||||
The four-task family (AZ-485/486/487/488) is **build-grade-tractable on top of the existing stack — no new dependencies required for AZ-485, AZ-486, or AZ-488; AZ-487 brings in one new built-in NuGet package** (`Microsoft.AspNetCore.Authentication.JwtBearer 8.0.x`, ships with the framework). The only architectural decision worth the user's attention is the **geofence shape (Q4)** — keep the existing bbox or move to polygon/NetTopologySuite — and that decision has a real SP impact on AZ-488 (2 → 4 SP).
|
||||
|
||||
## Key findings (5 points)
|
||||
|
||||
1. **JWT/AZ-487**: `Microsoft.AspNetCore.Authentication.JwtBearer 8.0.x` (ships with the framework, version-pinned to ASP.NET Core 8) handles HS256 + the user's "validate signature + `exp` only" requirement out of the box via `TokenValidationParameters` with `ValidateIssuer = false`, `ValidateAudience = false`, `ValidateLifetime = true`, and a `SymmetricSecurityKey`. No third-party library justified.
|
||||
2. **Multipart/AZ-485**: ASP.NET Core 8 minimal API supports `IFormFileCollection` + scalar `[FromForm]` parameters natively. Per-endpoint `MaxRequestBodySize` and `MultipartBodyLengthLimit` overrides apply via endpoint metadata (`IRequestSizeLimitMetadata`, `IFormOptionsMetadata`). `app.MapPost(...).DisableAntiforgery()` is required because antiforgery is enabled by default for forms but irrelevant to a JWT-protected non-browser endpoint.
|
||||
3. **ImageSharp/AZ-486**: ImageSharp 3.1.11 already-pinned API surface is sufficient — `Image.Identify` for dimensions + format without decoding, `Image.DetectFormat` for magic-bytes check, and `Image<L8>.ProcessPixelRows` for streaming-grade luminance stddev computation. No additional NuGet packages needed. Welford's online algorithm fits the per-row callback shape perfectly.
|
||||
4. **Geofence/AZ-488**: **Recommend Option A — reuse the existing `GeofencePolygon` bbox shape** (NorthWest + SouthEast corners). Rationale: the codebase already understands bbox; the user said "geofence" without specifying polygon; ray-casting is overkill for an internal whitelist that will likely have ≤ 5 axis-aligned regions. NetTopologySuite is **rejected** for this scope (heavy, +9 MB, includes a full topology engine when we need a 4-line containment check). Option B (true polygons) is **deferred** as a future task if real-world geofences turn out to need shapes beyond rectangles.
|
||||
5. **File+DB consistency/AZ-485**: **Recommend write-row-then-write-file with idempotent overwrite**. The per-source UPSERT (already in place from AZ-484) makes the row write idempotent; the file write is `File.WriteAllBytesAsync(tempPath); File.Move(tempPath, finalPath, overwrite: true)` for atomic replace. On file write failure: log + return `rejected: storage_failure` per tile in the response; the row stays as authoritative metadata pointing at the (now-missing) file path. Pre-existing GMaps download path has the same coupling (`TileService.DownloadAndStoreTilesAsync`) — match its behavior, not invent a new one.
|
||||
|
||||
---
|
||||
|
||||
## Project Constraint Matrix
|
||||
|
||||
| Constraint Source | Constraint | Impact on Component Selection |
|
||||
|--|--|--|
|
||||
| problem.md / user Step 1 | Sync 200 with mixed `[{tileId, status, reason?}]` | Rules out async/202 patterns; rules out HTTP 207 |
|
||||
| problem.md / user Step 1 | Minimum metadata only | No new request DTO complexity; ~5 fields per tile |
|
||||
| problem.md / user Step 1 | All 6 quality checks | AZ-486 is structurally a pipeline of 6 small validators |
|
||||
| problem.md / user Step 1 | JWT HS256, claims `sub`+`exp` only | Rules out IdP/OIDC integration; symmetric key in env |
|
||||
| problem.md / user Step 1 | Same `./tiles/{z}/{x}/{y}.jpg` path as GMaps | UAV file overwrites GMaps file when uploaded for the same cell — file-level "most recent wins" matches DB-level rule |
|
||||
| existing code | `GeofencePolygon` is a bbox in this codebase | Polygon would be a new concept; bbox is consistent |
|
||||
| existing code | `TileSourceConverter.ToWireValue(TileSource.Uav)` already exists from AZ-484 | UAV row writes are 1-line conversion at the boundary |
|
||||
| existing code | Per-source UPSERT in `TileRepository.InsertAsync` (AZ-484) | UAV InsertAsync = same call site as GMaps; no new repository method |
|
||||
| existing code | `TileService.DownloadAndStoreTilesAsync` writes file then row | Establish file-write/row-write order parity |
|
||||
| existing code | No auth middleware | AZ-487 must wire `app.UseAuthentication()` + `.UseAuthorization()` from scratch; tests must construct valid tokens |
|
||||
| existing code | No rate limiter | Out of scope for this family; recorded as Risk |
|
||||
| LESSONS L-001 | Dapper TypeHandler<T> bypassed for enum reads | No new persisted enums in this family; verdict stays response-only |
|
||||
| LESSONS ring-buffer/process | NFR + runner script must land same step | Any AZ-485 perf NFR (e.g., batch throughput) must come with the runner scenario |
|
||||
| LESSONS ring-buffer/estimation | Test-site counts need grep evidence | new-task Step 4 codebase analysis must grep, not estimate |
|
||||
| .NET 8.0 LTS only | Pin all package versions to 8.0.x major | `Microsoft.AspNetCore.Authentication.JwtBearer 8.0.x`, no 9.x |
|
||||
|
||||
## Component Fit Matrix
|
||||
|
||||
### Q1 — JWT auth (AZ-487)
|
||||
|
||||
| Candidate | Fit verdict | Notes |
|
||||
|--|--|--|
|
||||
| `Microsoft.AspNetCore.Authentication.JwtBearer 8.0.x` | **Selected** | Ships with ASP.NET Core 8; supports HS256 via `SymmetricSecurityKey`; `ValidateIssuer=false`, `ValidateAudience=false`, `ValidateLifetime=true` matches user's "claims = sub + exp only"; per-endpoint `RequireAuthorization()` works on minimal API |
|
||||
| `OpenIddict 5.x` | Rejected | Full OAuth2/OIDC server framework; orders of magnitude bigger than needed for HS256-validation-only |
|
||||
| Custom token middleware | Rejected | Reinvents token parsing/validation; security-critical code that should not be hand-rolled |
|
||||
| `IdentityServer / Duende` | Rejected | Issuer/server, not validator; commercial license |
|
||||
|
||||
**Mode pinned**: `JwtBearerOptions.TokenValidationParameters` with explicit symmetric-key shape. Mandatory MVE below.
|
||||
|
||||
### Q2 — Multipart batch upload (AZ-485)
|
||||
|
||||
| Candidate | Fit verdict | Notes |
|
||||
|--|--|--|
|
||||
| Minimal API `IFormFileCollection` + `[FromForm]` scalar fields | **Selected for batches up to ~50 MB** | Native to ASP.NET Core 8; no wire shape issue; FormOptions overridable per-endpoint |
|
||||
| `MultipartReader` streaming | Deferred | Required only if batches exceed in-memory budget. Not in initial scope; revisit if production telemetry shows batches >50 MB. |
|
||||
| Separate per-tile POST (no batch) | Rejected — user explicitly chose batch | Would simplify code but contradicts requirement |
|
||||
|
||||
**Mode pinned**: `app.MapPost("/api/satellite/uav/upload", (IFormFileCollection files, [FromForm] string metadata, ...) => ...).DisableAntiforgery().RequireAuthorization()`. Per-endpoint size cap via `.WithMetadata(new RequestSizeLimitMetadata { MaxRequestBodySize = 50_000_000 })` and `.WithMetadata(new FormOptionsMetadata { MultipartBodyLengthLimit = 50_000_000 })`.
|
||||
|
||||
### Q3 — Quality gate (AZ-486)
|
||||
|
||||
| Check | API | Performance |
|
||||
|--|--|--|
|
||||
| Format = JPEG | `Image.DetectFormat(stream)` then check `format.Name == "JPEG"` | Header-only read; ~1 ms per tile |
|
||||
| Dimensions = `MapConfig.TileSizePixels` (256) | `Image.Identify(stream)` returns `ImageInfo { Width, Height }` | Same header-only path; same ~1 ms |
|
||||
| File size 4 KB ≤ size ≤ 2 MB | `IFormFile.Length` direct check | O(1), no decode |
|
||||
| `captured_at` within last 365 days | `DateTime.UtcNow - capturedAt ≤ TimeSpan.FromDays(365)` | O(1) |
|
||||
| Blank/uniform: luminance stddev < 5 | `Image.Load<L8>(stream).ProcessPixelRows(accessor => ...)` Welford online stddev | ~5–8 ms per 256×256 tile (single-channel decode + one pass) |
|
||||
| Geofence containment | `GeofenceWhitelist.Contains(lat, lon)` (AZ-488) | O(M) per tile, M ≤ 10 polygons → negligible |
|
||||
|
||||
**Mode pinned**: `Image.Identify` first (cheap, gives format + dimensions in one call); only proceed to `Image.Load<L8>` for the blank-detection check after the cheap checks pass. Total per-tile budget under 15 ms.
|
||||
|
||||
| Candidate alternatives | Fit verdict |
|
||||
|--|--|
|
||||
| Magick.NET / ImageMagick | Rejected — second image library on the same stack; ImageSharp already pinned and sufficient |
|
||||
| `System.Drawing.Common` | Rejected — Microsoft has marked `System.Drawing.Common` non-cross-platform on .NET 6+; would break Linux container |
|
||||
| Pre-trained ML "is-this-blank" model | Rejected — overkill; Welford stddev is the canonical, fast baseline |
|
||||
|
||||
### Q4 — Geofence shape (AZ-488)
|
||||
|
||||
| Candidate | Fit verdict | Sizing impact |
|
||||
|--|--|--|
|
||||
| **Reuse `GeofencePolygon` bbox** (NW + SE corners, point-in-bbox check is a 4-comparison if-statement) | **Selected** | AZ-488 = 2 SP as originally scoped |
|
||||
| True polygon (lat/lon vertex list, ray-casting from scratch) | Deferred | Would push AZ-488 to ~4 SP (new DTO + ray-cast utility + tests for edge cases like point-on-edge, antimeridian crossing) |
|
||||
| `NetTopologySuite 2.5.x` | Rejected | +9 MB binary, full topology engine; not justified for ≤ 10 axis-aligned whitelisted regions |
|
||||
|
||||
Storage: `appsettings.json` array under `UavQualityGate.GeofenceWhitelist[]`, list of `{ northWest: {lat,lon}, southEast: {lat,lon} }`. Empty array = "allow all" (sentinel). Loaded via `IOptions<UavQualityGateConfig>` like every other config in this codebase.
|
||||
|
||||
### Q5 — File+DB consistency (AZ-485)
|
||||
|
||||
| Candidate | Fit verdict |
|
||||
|--|--|
|
||||
| **Write file with `.tmp`, then `InsertAsync` (UPSERT), then `File.Move(temp → final, overwrite: true)`** | **Selected** | Two-phase: file write is recoverable (delete .tmp on failure), DB write is idempotent (UPSERT), final atomic rename. |
|
||||
| Write row first, then file | Rejected | Race window: row points at non-existent file if process crashes between |
|
||||
| Distributed transaction across FS + DB (e.g., `TransactionScope`) | Rejected | `System.IO.File` is not a transactional resource manager on Linux — would silently no-op the FS rollback |
|
||||
| Outbox pattern with retry queue | Deferred | Worth considering if production traffic warrants exactly-once-delivery guarantees; out of scope for this cycle |
|
||||
|
||||
## Tech Stack Summary
|
||||
|
||||
| Layer | Choice | Version | Justification |
|
||||
|--|--|--|--|
|
||||
| HTTP framework | ASP.NET Core minimal API | 8.0 (existing) | Match existing endpoints |
|
||||
| Auth middleware | `Microsoft.AspNetCore.Authentication.JwtBearer` | 8.0.x | Built-in; HS256 supported |
|
||||
| Image inspection | `SixLabors.ImageSharp` | 3.1.11 (existing) | Already pinned; supports `Identify`, `Load<L8>`, `ProcessPixelRows` |
|
||||
| DB access | `Dapper` + `Npgsql` | 2.1.35 / 9.0.2 (existing) | UPSERT via `TileRepository.InsertAsync` (AZ-484) |
|
||||
| Geofence | Reuse `SatelliteProvider.Common.DTO.GeofencePolygon` | existing | Bbox shape, simple containment check |
|
||||
| Config | `Microsoft.Extensions.Options` `IOptions<T>` | 8.0.x (existing) | New `UavQualityGateConfig` registered same way as `MapConfig`, etc. |
|
||||
|
||||
**No new top-level dependencies on top of the existing stack** other than the JwtBearer package.
|
||||
|
||||
## Security analysis (concise)
|
||||
|
||||
| Threat | Control | Where |
|
||||
|--|--|--|
|
||||
| Forged tokens | HMAC SHA-256 signature validation against shared secret | AZ-487 — `JwtBearerOptions.TokenValidationParameters` |
|
||||
| Token replay | `exp` claim validation | AZ-487 — `ValidateLifetime = true` |
|
||||
| Signing key leak | env var, not in source; documented in `.env.example` (matches existing GMaps API key handling) | AZ-487 + AGENTS.md update |
|
||||
| Path traversal in upload | Tile path is computed from validated lat/lon/zoom server-side, not from any user-supplied filename | AZ-485 — `tile_x/tile_y` derived via existing `GeoUtils.WorldToTilePos` |
|
||||
| Image-decode attacks (zip bomb, malformed JPEG) | `Image.Identify` uses bounded reads; `Image.Load` wrapped in try/catch around `InvalidImageContentException` and `UnknownImageFormatException`; per-file size cap before decode | AZ-486 |
|
||||
| Disk fill DoS | Per-batch size cap (`MaxRequestBodySize = 50 MB`); per-file size cap (2 MB); rate limiter recommended but tracked separately from this family | AZ-485 — endpoint metadata; rate limiter is Security Audit I3 deferred |
|
||||
| JWT logged in logs | `Serilog` request logging configured today does not log Authorization header; verify Pii filter is on | AZ-487 verification step |
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
| Risk | Severity | Mitigation |
|
||||
|--|--|--|
|
||||
| File overwrite for same cell across sources causes data loss at the file system level (DB row carries source, file does not) | Medium | Re-confirm with user in new-task Step 5 — option to use `./tiles/uav/{z}/{x}/{y}.jpg` to keep both bytes addressable. Default is "yes, same path" per Step 1 answer. |
|
||||
| Quality-gate thresholds (blank stddev, age limit, byte size bounds) need real-world tuning | Low | Make every threshold configurable via `UavQualityGateConfig`; ship defaults; tune in production telemetry |
|
||||
| JWT signing-key rotation procedure | Medium | Out of scope for AZ-487; document the procedure ("change env var + restart") in `_docs/02_document/deployment/`. Shared secret rotation requires a coordinated upload-client + server change, accepted limitation |
|
||||
| Test infrastructure for JWT (integration tests must mint tokens) | Low | Use `JsonWebTokenHandler` from `System.IdentityModel.Tokens.Jwt` in test code with the same shared secret as the test config; pattern is well-trodden |
|
||||
| Adversarial UAV uploader sends `captured_at` in the future | Low | Add a 7th implicit check: `captured_at <= DateTime.UtcNow`. Cheap, catches one obvious abuse |
|
||||
|
||||
## Minimum Viable Examples (saved verbatim per per-mode API gate)
|
||||
|
||||
### MVE-1 — JWT validation (Q1 selected mode)
|
||||
|
||||
```csharp
|
||||
using Microsoft.AspNetCore.Authentication.JwtBearer;
|
||||
using Microsoft.IdentityModel.Tokens;
|
||||
using System.Text;
|
||||
|
||||
builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
|
||||
.AddJwtBearer(options =>
|
||||
{
|
||||
var key = builder.Configuration["UavAuth:SigningKey"]
|
||||
?? throw new InvalidOperationException("UavAuth:SigningKey is required");
|
||||
|
||||
options.RequireHttpsMetadata = false;
|
||||
options.TokenValidationParameters = new TokenValidationParameters
|
||||
{
|
||||
ValidateIssuer = false,
|
||||
ValidateAudience = false,
|
||||
ValidateLifetime = true,
|
||||
ValidateIssuerSigningKey = true,
|
||||
IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(key)),
|
||||
ClockSkew = TimeSpan.FromSeconds(30),
|
||||
};
|
||||
});
|
||||
|
||||
builder.Services.AddAuthorization();
|
||||
|
||||
var app = builder.Build();
|
||||
app.UseAuthentication();
|
||||
app.UseAuthorization();
|
||||
|
||||
app.MapPost("/api/satellite/uav/upload", (IFormFileCollection files, [FromForm] string metadata) =>
|
||||
{
|
||||
return Results.Ok(new { received = files.Count });
|
||||
})
|
||||
.DisableAntiforgery()
|
||||
.RequireAuthorization();
|
||||
```
|
||||
|
||||
### MVE-2 — Per-endpoint size limits (Q2 selected mode)
|
||||
|
||||
```csharp
|
||||
using Microsoft.AspNetCore.Http.Metadata;
|
||||
|
||||
const long MaxBatchBytes = 50L * 1024 * 1024;
|
||||
|
||||
app.MapPost("/api/satellite/uav/upload", handler)
|
||||
.DisableAntiforgery()
|
||||
.RequireAuthorization()
|
||||
.WithMetadata(
|
||||
new RequestSizeLimitMetadata { MaxRequestBodySize = MaxBatchBytes },
|
||||
new FormOptionsMetadata
|
||||
{
|
||||
MultipartBodyLengthLimit = MaxBatchBytes,
|
||||
ValueLengthLimit = 16 * 1024,
|
||||
MultipartBoundaryLengthLimit = 256,
|
||||
});
|
||||
```
|
||||
|
||||
### MVE-3 — Quality gate (Q3 selected mode)
|
||||
|
||||
```csharp
|
||||
using SixLabors.ImageSharp;
|
||||
using SixLabors.ImageSharp.PixelFormats;
|
||||
|
||||
public sealed record QualityResult(bool Accepted, string? Reason);
|
||||
|
||||
public sealed class TileQualityGate
|
||||
{
|
||||
private readonly UavQualityGateConfig _cfg;
|
||||
|
||||
public QualityResult Check(IFormFile file, double lat, double lon, DateTime capturedAt, GeofenceWhitelist geofence)
|
||||
{
|
||||
if (file.Length < _cfg.MinFileBytes) return new(false, "file_too_small");
|
||||
if (file.Length > _cfg.MaxFileBytes) return new(false, "file_too_large");
|
||||
|
||||
var capturedAge = DateTime.UtcNow - capturedAt;
|
||||
if (capturedAge > TimeSpan.FromDays(_cfg.MaxAgeDays)) return new(false, "captured_at_too_old");
|
||||
if (capturedAge < TimeSpan.Zero) return new(false, "captured_at_in_future");
|
||||
|
||||
if (!geofence.Contains(lat, lon)) return new(false, "outside_geofence");
|
||||
|
||||
using var stream = file.OpenReadStream();
|
||||
|
||||
IImageFormat? detectedFormat = Image.DetectFormat(stream);
|
||||
if (detectedFormat is null || detectedFormat.Name != "JPEG") return new(false, "format_not_jpeg");
|
||||
|
||||
stream.Position = 0;
|
||||
ImageInfo info = Image.Identify(stream);
|
||||
if (info.Width != _cfg.ExpectedTilePixels || info.Height != _cfg.ExpectedTilePixels)
|
||||
return new(false, "wrong_dimensions");
|
||||
|
||||
stream.Position = 0;
|
||||
try
|
||||
{
|
||||
using var img = Image.Load<L8>(stream);
|
||||
double mean = 0, m2 = 0; long n = 0;
|
||||
img.ProcessPixelRows(accessor =>
|
||||
{
|
||||
for (int y = 0; y < accessor.Height; y++)
|
||||
{
|
||||
var row = accessor.GetRowSpan(y);
|
||||
for (int x = 0; x < row.Length; x++)
|
||||
{
|
||||
double v = row[x].PackedValue;
|
||||
n++;
|
||||
double delta = v - mean;
|
||||
mean += delta / n;
|
||||
m2 += delta * (v - mean);
|
||||
}
|
||||
}
|
||||
});
|
||||
double stddev = n > 1 ? Math.Sqrt(m2 / (n - 1)) : 0;
|
||||
if (stddev < _cfg.MinLuminanceStdDev) return new(false, "image_blank_or_uniform");
|
||||
}
|
||||
catch (InvalidImageContentException) { return new(false, "image_corrupt"); }
|
||||
catch (UnknownImageFormatException) { return new(false, "format_unknown"); }
|
||||
|
||||
return new(true, null);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### MVE-4 — Geofence whitelist (Q4 selected mode)
|
||||
|
||||
```csharp
|
||||
public sealed class GeofenceWhitelist
|
||||
{
|
||||
private readonly IReadOnlyList<GeofencePolygon> _polygons;
|
||||
|
||||
public GeofenceWhitelist(IReadOnlyList<GeofencePolygon> polygons) => _polygons = polygons;
|
||||
|
||||
public bool Contains(double lat, double lon)
|
||||
{
|
||||
if (_polygons.Count == 0) return true; // sentinel: empty list = allow all
|
||||
|
||||
foreach (var p in _polygons)
|
||||
{
|
||||
if (p.NorthWest is null || p.SouthEast is null) continue;
|
||||
if (lat <= p.NorthWest.Lat && lat >= p.SouthEast.Lat
|
||||
&& lon >= p.NorthWest.Lon && lon <= p.SouthEast.Lon)
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### MVE-5 — File+DB consistency (Q5 selected mode)
|
||||
|
||||
```csharp
|
||||
public sealed class UavTilePersister
|
||||
{
|
||||
private readonly ITileRepository _repo;
|
||||
private readonly StorageConfig _cfg;
|
||||
private readonly ILogger<UavTilePersister> _log;
|
||||
|
||||
public async Task<Guid?> PersistAsync(IFormFile file, double lat, double lon, int zoom,
|
||||
double tileSizeMeters, DateTime capturedAt, CancellationToken ct)
|
||||
{
|
||||
var (tileX, tileY) = GeoUtils.WorldToTilePos(new GeoPoint(lat, lon), zoom);
|
||||
var finalPath = Path.Combine(_cfg.TilesRoot, zoom.ToString(), tileX.ToString(), tileY + ".jpg");
|
||||
var tempPath = finalPath + ".uav.tmp";
|
||||
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(finalPath)!);
|
||||
|
||||
await using (var dst = File.Create(tempPath))
|
||||
await file.CopyToAsync(dst, ct);
|
||||
|
||||
try
|
||||
{
|
||||
var entity = new TileEntity
|
||||
{
|
||||
Id = Guid.NewGuid(),
|
||||
TileZoom = zoom, TileX = tileX, TileY = tileY,
|
||||
Latitude = lat, Longitude = lon,
|
||||
TileSizeMeters = tileSizeMeters,
|
||||
TileSizePixels = _cfg.ExpectedTilePixels,
|
||||
ImageType = "jpg",
|
||||
Source = TileSourceConverter.ToWireValue(TileSource.Uav),
|
||||
CapturedAt = capturedAt,
|
||||
FilePath = Path.GetRelativePath(AppContext.BaseDirectory, finalPath),
|
||||
};
|
||||
|
||||
var id = await _repo.InsertAsync(entity);
|
||||
|
||||
File.Move(tempPath, finalPath, overwrite: true);
|
||||
return id;
|
||||
}
|
||||
catch
|
||||
{
|
||||
File.Delete(tempPath);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Sources
|
||||
|
||||
| Source | Tier | Citation |
|
||||
|--|--|--|
|
||||
| ASP.NET Core 8.0.21 source — JwtBearer public API | L1 | `github.com/dotnet/aspnetcore/blob/v8.0.21/src/Security/Authentication/JwtBearer/src/PublicAPI.Shipped.txt` (via context7 `/dotnet/aspnetcore/v8.0.21`) |
|
||||
| ASP.NET Core 8.0.21 source — FormOptions, IRequestSizeLimitMetadata | L1 | `github.com/dotnet/aspnetcore/blob/v8.0.21/src/Http/Http/src/PublicAPI.Shipped.txt` |
|
||||
| ASP.NET Core docs — JWT Bearer config sample | L1 | `learn.microsoft.com/aspnet/core/security/authentication/configure-jwt-bearer-authentication` (via context7 `/dotnet/aspnetcore`) |
|
||||
| SixLabors.ImageSharp docs — Image.Identify, ProcessPixelRows | L1 | `docs.sixlabors.com/articles/imagesharp/imageinfo` (via context7 `/sixlabors/imagesharp`) |
|
||||
| Codebase grep — `GeofencePolygon` shape, `TileRepository.InsertAsync` UPSERT | L1 (project source) | `SatelliteProvider.Common/DTO/GeofencePolygon.cs`, `SatelliteProvider.DataAccess/Repositories/TileRepository.cs` |
|
||||
| AZ-484 task family closure (this repo, cycle 1) | L1 (project history) | `_docs/03_implementation/implementation_report_multi_source_tile_storage_cycle1.md`; `_docs/02_document/contracts/data-access/tile-storage.md` |
|
||||
|
||||
## Quality checklist
|
||||
|
||||
- [x] Each component selection has an exact-fit verdict against the Project Constraint Matrix
|
||||
- [x] Per-mode API capability verification done for each library (JwtBearer, ASP.NET Core multipart, ImageSharp)
|
||||
- [x] MVE saved for every selected mode (5 of them)
|
||||
- [x] "Do not use" / Rejected list present per question
|
||||
- [x] L1 sources for every key claim
|
||||
- [x] Multi-perspective considered: implementer (MVE), security architect (threats table), domain (existing code constraints), contrarian (rejected NTS / OpenIddict / Magick.NET / async-202)
|
||||
- [x] No conclusions from "gut feel" — every selection has matrix evidence
|
||||
@@ -0,0 +1,95 @@
|
||||
# Research Problem — UAV Batch Upload (AZ-485 family)
|
||||
|
||||
## Context
|
||||
|
||||
- **Project**: SatelliteProvider — .NET 8.0 ASP.NET Core service that downloads/persists satellite imagery tiles. Currently single-source (Google Maps); AZ-484 just shipped multi-source storage (`source` + `captured_at` columns, 5-col unique index, `tile-storage` v1.0.0 frozen contract).
|
||||
- **Cycle**: 2 (started after cycle 1 closed AZ-484 today, 2026-05-11).
|
||||
- **Task family** (4 tasks under epic AZ-483, accepted by user):
|
||||
- **AZ-485** — UAV batch upload endpoint + per-tile persistence loop (5 SP)
|
||||
- **AZ-486** — Quality gate (6 checks + thresholds + reject reasons) (3 SP)
|
||||
- **AZ-487** — JWT auth middleware (3 SP)
|
||||
- **AZ-488** — Geofence whitelist (2 SP)
|
||||
- **Confirmed user requirements** (from new-task Step 1):
|
||||
1. Batch upload (multipart with metadata + N image files per request)
|
||||
2. Minimum metadata only per tile: `latitude`, `longitude`, `tile_zoom`, `tile_size_meters`, `captured_at` (UTC), image bytes
|
||||
3. Quality gate enforces all of: dimensions, byte size, blank/uniform detection, captured_at age, format=JPEG, geofence containment
|
||||
4. JWT auth (HS256, signing key in env, required claims: `sub` + valid `exp`)
|
||||
5. Sync responses: 200 + per-tile `{ tileId, status, reason? }` array; 4xx for auth/format-of-batch failures
|
||||
6. File storage: same `./tiles/{z}/{x}/{y}.jpg` layout as Google Maps tiles
|
||||
- **Project constraints to respect**:
|
||||
- .NET 8.0 only (no .NET 9 features)
|
||||
- Existing libraries: ImageSharp 3.1.11, Npgsql 9.0.2, Dapper 2.1.35, Serilog 8.0.3
|
||||
- No auth currently exists in the API (per Security Audit Step 14, 2026-05-11)
|
||||
- The "geofence" type in the codebase (`SatelliteProvider.Common.DTO.GeofencePolygon`) is actually a bounding box (NorthWest + SouthEast corners), not an arbitrary polygon. There is no existing point-in-polygon implementation.
|
||||
- The frozen `_docs/02_document/contracts/data-access/tile-storage.md` v1.0.0 is the producer-side contract this work must implement (`source='uav'`, `captured_at` per-row, per-source UPSERT)
|
||||
- Lessons applied: L-001 (Dapper enum bypass — no new persisted enums planned but the rule applies if any sneak in)
|
||||
|
||||
## Specific unknowns to investigate
|
||||
|
||||
### Q1 — JWT auth middleware in ASP.NET Core 8 for an internal service (AZ-487)
|
||||
- Recommended pattern for an internal/trusted-network API that needs to validate HS256 tokens issued by an external system?
|
||||
- `Microsoft.AspNetCore.Authentication.JwtBearer` (built-in, recommended? version-pinned for ASP.NET Core 8)
|
||||
- vs. third-party (e.g., custom middleware, OpenIddict, etc.)
|
||||
- Configuration shape — where does the signing key live? appsettings vs env var vs key vault? (Project constraint: this codebase has no secret manager today.)
|
||||
- Required claims minimum: just `sub` and unexpired `exp` per user choice. Confirm `JwtBearerOptions.TokenValidationParameters` setup that validates *only* signature + `exp` and tolerates missing `aud`/`iss` (or reject — which is the safer ASP.NET Core 8 default?).
|
||||
- Swagger integration: how to surface "Bearer <token>" in the Swagger UI for the upload endpoint without changing the existing public endpoints?
|
||||
- 401 response shape — does the global `GlobalExceptionHandler` need to know about it, or does `[Authorize]` short-circuit before the handler?
|
||||
- Test strategy — how do current `SatelliteProvider.IntegrationTests` (which call the live API via `HttpClient` per `RouteTestHelpers`) cleanly inject a valid JWT? Pattern for a "test issuer" using the same signing key.
|
||||
|
||||
### Q2 — Multipart batch upload in ASP.NET Core 8 minimal API (AZ-485)
|
||||
- Idiomatic shape for `multipart/form-data` with N files + a JSON metadata document in a minimal API endpoint:
|
||||
- `[FromForm]` model binding
|
||||
- vs. manual `MultipartReader` for streaming (relevant if batches are large)
|
||||
- Request size limits in ASP.NET Core 8 (Kestrel `MaxRequestBodySize` and form options) — defaults and how to override for the upload endpoint only.
|
||||
- Memory / streaming trade-off: how big can the per-batch `IFormFile[]` get before we should switch to streaming via `MultipartReader`? Practical batch-size guidance for satellite tiles (~50–500 KB each).
|
||||
- Response shape for partial-success batches — common patterns (HTTP 200 with mixed array, HTTP 207 Multi-Status, HTTP 422 with per-item errors).
|
||||
- Backpressure / concurrency limit on the per-tile persistence loop — should this be sequential or fanned out? Trade-off vs. Postgres connection pool size (current Npgsql defaults).
|
||||
|
||||
### Q3 — Image quality heuristics for satellite tiles using ImageSharp 3.1.11 (AZ-486)
|
||||
- API surface in **ImageSharp 3.1.11 specifically** for:
|
||||
- Reading width/height without full decode (`Image.IdentifyAsync` returns `ImageInfo`)
|
||||
- Detecting actual format from magic bytes vs. trusting `Content-Type`
|
||||
- Computing a "uniformity / blankness" metric efficiently — luminance stddev across the image, or histogram-based, or a perceptual hash
|
||||
- Performance budget: per-tile quality check ideally under ~10ms on a 256×256 JPEG. Confirm which ImageSharp APIs are streaming vs. full-decode.
|
||||
- Robustness against adversarial inputs — e.g., a 1-pixel image with a JPEG header. What does ImageSharp 3.1.11 throw, and is there a safe-decode pattern?
|
||||
- Threshold-tuning approach — recommended way to tune the blank-detection threshold against a small fixture set of real satellite tiles vs. uniform tiles, without having that fixture set today.
|
||||
|
||||
### Q4 — Geofence whitelist: bbox vs. polygon (AZ-488)
|
||||
- The codebase's existing geofence type is a **bounding box** (`NorthWest`+`SouthEast`). Two options for the AZ-488 whitelist:
|
||||
- **Option A**: reuse the existing bbox shape (simple, consistent with current code, but coarser — a UAV tile inside a non-rectangular real-world allowed area can't be expressed)
|
||||
- **Option B**: introduce a true polygon (lat/lon list) + ray-casting point-in-polygon, store as JSON in config. Requires writing the ray-cast utility (or pulling in `NetTopologySuite`).
|
||||
- Recommendation: which one is the right fit given that the user only said "geofence" without specifying shape and the existing code is bbox? Compare with NetTopologySuite as a third option.
|
||||
- Storage: where does the whitelist live? `appsettings.json` array, dedicated config file, dedicated DB table?
|
||||
- Performance: typical UAV batch is N tiles vs. M whitelist polygons — is M small enough (e.g., < 10 polygons, < 1000 vertices) that O(N×M) ray-cast per batch is fine?
|
||||
|
||||
### Q5 — Per-tile persistence loop: file-system + DB consistency (AZ-485)
|
||||
- Pattern for "write JPEG to disk, then write DB row" with rollback on either failure:
|
||||
- Two-phase: write file with a `.tmp` suffix, INSERT, then rename. On INSERT failure, delete `.tmp`.
|
||||
- vs. write DB row first (ON CONFLICT does UPSERT), then write file. On file failure, what happens to the row? (Could leave a row pointing at a missing file path — bad.)
|
||||
- Idempotency: if the same (lat, lon, zoom, size, source='uav') tile is uploaded twice, the per-source UPSERT already handles the DB side — but does the file get overwritten cleanly? Atomic file replace pattern.
|
||||
- Concurrency: two simultaneous batch uploads for adjacent tiles — any shared resource that needs locking?
|
||||
|
||||
## Out of scope for this research
|
||||
|
||||
- Distributed UAV fleet management, queueing, or real-time streaming (these are upstream concerns the user has not raised)
|
||||
- Async/202 + status polling response model (user explicitly said sync 200)
|
||||
- Per-tile encryption, compression beyond JPEG, or alternate image formats
|
||||
- External IdP integration (HS256 with shared secret only)
|
||||
- UAV mission/operator/sensor metadata (user explicitly said "not necessary for now")
|
||||
|
||||
## Acceptance criteria for the research output
|
||||
|
||||
The research deliverable (`solution_draft01.md`) must:
|
||||
|
||||
1. Recommend a concrete library + config approach for **Q1 (JWT)**. Include: package name + version, code shape for `builder.Services.AddAuthentication().AddJwtBearer(...)`, key storage recommendation, integration-test pattern.
|
||||
2. Recommend a concrete request shape + size-limit approach for **Q2 (multipart)**. Include: `[FromForm]` vs streaming decision criteria, response shape choice with rationale, concurrency-limit recommendation.
|
||||
3. Recommend a concrete API call sequence in **ImageSharp 3.1.11** for **Q3 (quality heuristics)**. Include: blank-detection threshold starting point + tuning plan, performance budget validation, error-handling pattern.
|
||||
4. Recommend bbox-vs-polygon-vs-NTS for **Q4 (geofence)**. Include: exact-fit verdict per the project's actual operating context (M < 10 polygons, simple ray-cast acceptable, or pull in NTS).
|
||||
5. Recommend a file+DB consistency pattern for **Q5**. Include: failure modes covered, rollback shape, idempotency story.
|
||||
6. Include a "do not use" / rejected list for each question — alternatives the research considered and rejected with one-line evidence.
|
||||
|
||||
## Execution mode
|
||||
|
||||
Mode A — Initial Research. Output class: **Technical-component selection** (recommends specific libraries, ASP.NET Core 8 modes, ImageSharp APIs, optional NTS).
|
||||
|
||||
Per-mode API capability verification applies to every recommended library. Saved Minimum Viable Examples required for: `Microsoft.AspNetCore.Authentication.JwtBearer` JWT validation, ASP.NET Core 8 minimal-API multipart, ImageSharp 3.1.11 luminance stddev.
|
||||
@@ -4,7 +4,7 @@
|
||||
flow: existing-code
|
||||
step: 10
|
||||
name: Implement
|
||||
status: not_started
|
||||
status: in_progress
|
||||
sub_step:
|
||||
phase: 0
|
||||
name: awaiting-invocation
|
||||
|
||||
Reference in New Issue
Block a user