mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-21 22:21:15 +00:00
61612044fb
Wrap up cycle 5 verification + documentation: - Steps 10/11 wrap-up reports (implementation_completeness + implementation_report) for the AZ-503-foundation + AZ-504 batch. - Step 12 test-spec sync: AZ-503-foundation/AZ-504 ACs appended; AZ-505 deferred ACs recorded. - Step 13 update-docs: architecture, data-model, glossary, module- layout, uav-tile-upload contract (v1.1.0), DataAccess + Services + Tests module docs synced; new common_uuidv5.md module doc. - Step 14 security audit: PASS_WITH_WARNINGS; 0 new Critical/High; 2 new Low informational (F1 flightId provenance, F2 pgcrypto deploy gap). - Step 15 performance test: PASS_WITH_INFRA_WARNINGS; PT-08 passed twice (AZ-504 fix verified); PT-01/02 failed due to recurring local Docker/colima DNS cold-start (not an app regression). Cycle-3 perf-harness leftover stays OPEN with replay #5 documented. - Autodev state moved to Step 16 (Deploy). Co-authored-by: Cursor <cursoragent@cursor.com>
188 lines
14 KiB
Markdown
188 lines
14 KiB
Markdown
# Contract: uav-tile-upload
|
||
|
||
**Component**: WebApi (`SatelliteProvider.Api`) producing rows via TileDownloader (`SatelliteProvider.Services.TileDownloader`)
|
||
**Producer task**: AZ-488 — `_docs/02_tasks/done/AZ-488_uav_tile_upload.md`
|
||
**Extended by**: AZ-503 — `_docs/02_tasks/done/AZ-503_tile_identity_uuidv5_bulk_list.md` (added optional `flightId` per-item field; per-flight on-disk path; deterministic `tileId`)
|
||
**Consumer tasks**: `gps-denied-onboard`, mission planner UI, any future UAV-equipped client
|
||
**Version**: 1.1.0
|
||
**Status**: frozen
|
||
**Last Updated**: 2026-05-12
|
||
|
||
## Purpose
|
||
|
||
Defines the HTTP contract for the `POST /api/satellite/upload` batch endpoint that ingests UAV-captured satellite tiles, runs each item through a 5-rule quality gate, and persists accepted rows under the `tile-storage` v1.0.0 data contract with `source='uav'`.
|
||
|
||
## Endpoint
|
||
|
||
```
|
||
POST /api/satellite/upload
|
||
Content-Type: multipart/form-data
|
||
Authorization: Bearer <JWT>
|
||
```
|
||
|
||
The request MUST carry a valid JWT (AZ-487) AND the `permissions` claim MUST contain `GPS`. Anonymous requests are rejected with HTTP 401; authenticated requests without `GPS` are rejected with HTTP 403.
|
||
|
||
## Request shape
|
||
|
||
Multipart form fields (case-sensitive part names):
|
||
|
||
| Part | Type | Required | Description |
|
||
|------|------|----------|-------------|
|
||
| `metadata` | JSON string | yes | Document of shape `{ "items": [ <UavTileMetadata>, ... ] }`. See the table below for the per-item schema. |
|
||
| `files` | binary (repeating) | yes | One JPEG file per metadata item. Files are correlated to metadata entries by ordinal index (file index `i` ↔ `metadata.items[i]`). Each part MUST be sent under the form field name `files`. |
|
||
|
||
### `UavTileMetadata` (per item)
|
||
|
||
| Field | Type | Required | Description | Constraints |
|
||
|-------|------|----------|-------------|-------------|
|
||
| `latitude` | number | yes | Geographic latitude of the tile center | WGS-84 decimal degrees |
|
||
| `longitude` | number | yes | Geographic longitude of the tile center | WGS-84 decimal degrees |
|
||
| `tileZoom` | integer | yes | Slippy Map zoom level | Must satisfy the same zoom-level policy as the existing tile pipeline (see `MapConfig.AllowedZoomLevels`) |
|
||
| `tileSizeMeters` | number | yes | Tile size in meters at the captured latitude | Producer-supplied |
|
||
| `capturedAt` | string (ISO-8601 UTC) | yes | Moment of UAV image capture | Must satisfy the captured-at rule (see Quality Gate, Rule 4) |
|
||
| `flightId` | string (UUID) | no | AZ-503: optional flight identifier. When present, two flights uploading the same cell coexist as separate rows; absent uploads share a single anonymous row per cell. Omitting the field is fully backward-compatible with v1.0.0 clients. | RFC 4122 UUID. Backward-compatible default: `null` |
|
||
|
||
Field names are camelCase. Property-name matching is case-insensitive on read.
|
||
|
||
### Constraints
|
||
|
||
- The number of `files` MUST equal the number of `metadata.items` entries (1:1 correlation by ordinal index). Mismatch → HTTP 400.
|
||
- `metadata.items.length` MUST NOT exceed `UavQualityConfig.MaxBatchSize` (default `100`). Oversize → HTTP 400.
|
||
- The total request body size is capped at `MaxBatchSize × MaxBytes` by Kestrel's `MaxRequestBodySize` and the form `MultipartBodyLengthLimit`. Requests beyond this cap are rejected at the framework layer (HTTP 413/400).
|
||
|
||
## Quality Gate (5 rules)
|
||
|
||
Each item is evaluated in this fixed order. The **first** failing rule produces the reported reason; subsequent rules are not evaluated for that item.
|
||
|
||
| # | Rule | Failure condition | Reason code |
|
||
|---|------|-------------------|-------------|
|
||
| 1 | Format | `Content-Type` of the file part is not `image/jpeg` (case-insensitive, allowing trailing parameters) OR the file's first 3 bytes are not `FF D8 FF` | `INVALID_FORMAT` |
|
||
| 2 | Size band | `bytes.length` is outside `[UavQualityConfig.MinBytes, UavQualityConfig.MaxBytes]` (defaults: 5 KiB … 5 MiB) | `SIZE_OUT_OF_BAND` |
|
||
| 3 | Dimensions | Image width OR height ≠ `MapConfig.TileSizePixels` (default 256). Strict equality, no tolerance | `WRONG_DIMENSIONS` |
|
||
| 4a | Captured-at future | `capturedAt > now + UavQualityConfig.CapturedAtFutureSkewSeconds` (default 30s) | `CAPTURED_AT_FUTURE` |
|
||
| 4b | Captured-at age | `capturedAt < now - UavQualityConfig.MaxAgeDays` (default 7 days) | `CAPTURED_AT_TOO_OLD` |
|
||
| 5 | Blank / uniform | Pixel-luminance variance on a downsampled (default 32×32, configurable via `UavQualityConfig.LuminanceSampleSize`) version of the image is below `UavQualityConfig.MinLuminanceVariance` (default 10.0) | `IMAGE_TOO_UNIFORM` |
|
||
|
||
If the file decode itself fails for the variance check, the item is rejected with `INVALID_FORMAT` (the file is not a real JPEG even though the header bytes matched).
|
||
|
||
Storage failures while persisting an accepted item (e.g., disk full, unwritable path) are surfaced as `STORAGE_FAILURE` so the client can retry that specific item without re-uploading the whole batch.
|
||
|
||
### Reject-reason enumeration (closed)
|
||
|
||
| Code | Source rule | When the reason fires |
|
||
|------|-------------|------------------------|
|
||
| `INVALID_FORMAT` | Rule 1 + Rule 5 decode safety net | Wrong content-type, wrong magic bytes, or undecodable bytes |
|
||
| `SIZE_OUT_OF_BAND` | Rule 2 | Byte length outside `[MinBytes, MaxBytes]` |
|
||
| `WRONG_DIMENSIONS` | Rule 3 | Width or height ≠ `MapConfig.TileSizePixels` |
|
||
| `CAPTURED_AT_FUTURE` | Rule 4a | `capturedAt` is more than `CapturedAtFutureSkewSeconds` in the future |
|
||
| `CAPTURED_AT_TOO_OLD` | Rule 4b | `capturedAt` is older than `MaxAgeDays` |
|
||
| `IMAGE_TOO_UNIFORM` | Rule 5 | Luminance variance below `MinLuminanceVariance` |
|
||
| `METADATA_MISSING` | Validation gate | Per-item metadata could not be matched (reserved; not currently surfaced because batch mismatch is reported as an envelope error) |
|
||
| `STORAGE_FAILURE` | Persistence path | The image bytes could not be written to disk or the repository insert raised an IO error |
|
||
|
||
Adding a new code is a **minor** contract version bump per the Versioning Rules below. Removing or renaming a code is **major**.
|
||
|
||
## Response shape
|
||
|
||
### HTTP 200 — per-item results
|
||
|
||
```json
|
||
{
|
||
"items": [
|
||
{ "index": 0, "status": "accepted", "tileId": "11111111-...", "rejectReason": null, "rejectDetails": null },
|
||
{ "index": 1, "status": "rejected", "tileId": null, "rejectReason": "WRONG_DIMENSIONS", "rejectDetails": null }
|
||
]
|
||
}
|
||
```
|
||
|
||
| Field | Type | Description |
|
||
|-------|------|-------------|
|
||
| `items` | array | One result per input item, in the same order as `metadata.items` |
|
||
| `items[].index` | integer | Ordinal index of the item in the request batch |
|
||
| `items[].status` | string | `"accepted"` or `"rejected"` (closed enumeration) |
|
||
| `items[].tileId` | UUID or null | When `accepted`, the persisted `tiles.id` value; null when `rejected` |
|
||
| `items[].rejectReason` | string or null | Reason code (see table above) when `rejected`; null when `accepted` |
|
||
| `items[].rejectDetails` | string or null | Optional human-readable detail; MUST NOT leak server paths, exception types, or internal identifiers |
|
||
|
||
### HTTP 400 — envelope error (RFC 7807 `application/problem+json`)
|
||
|
||
Returned when the request itself is malformed:
|
||
|
||
- `metadata` field absent, empty, or not valid JSON
|
||
- `metadata.items` empty or null
|
||
- `metadata.items.length` ≠ `files.length`
|
||
- `metadata.items.length` > `MaxBatchSize`
|
||
|
||
The 5-rule per-item quality gate never produces a 400; per-item failures always surface in the HTTP 200 response array.
|
||
|
||
### HTTP 401 — missing or invalid JWT (from AZ-487)
|
||
|
||
### HTTP 403 — JWT present but `permissions` claim does not include `GPS`
|
||
|
||
## Persistence semantics
|
||
|
||
- Accepted items are persisted via `ITileRepository.InsertAsync` (the integer-only flight-aware UPSERT path established in AZ-503; supersedes the AZ-484 5-column float-based UPSERT). The `tiles` row carries `source='uav'`, `captured_at` from the request, `flight_id` from the optional `metadata.flightId`, and a deterministic `id = Uuidv5(TileNamespace, "{z}/{x}/{y}/uav/{flight_id or zero-uuid}")`.
|
||
- A UAV upload for a cell that already has a `google_maps` row **coexists** with that row (per `tile-storage.md` Inv-3). The most-recent row across sources wins on read.
|
||
- Two UAV uploads with **different** `flightId` for the same cell coexist as separate rows (one row per flight, sharing `location_hash`). Two UAV uploads with the **same** `flightId` for the same cell UPSERT the existing row, updating `file_path, captured_at, location_hash, content_sha256, updated_at` and overwriting the JPEG bytes on disk. `id` is intentionally NOT regenerated on conflict — re-uploading identical bytes returns the same `tileId` (AZ-503 AC-2).
|
||
- `content_sha256` is computed from the JPEG body on every persisted row.
|
||
|
||
## File-path layout
|
||
|
||
- UAV files (AZ-503): `{StorageConfig.TilesDirectory}/uav/{flightId or 'none'}/{tile_zoom}/{tile_x}/{tile_y}.jpg`
|
||
- Anonymous uploads (`flightId` absent or null) use the literal `none` segment.
|
||
- Per-flight uploads use the full `flightId` UUID as a directory name, so `rm -rf ./tiles/uav/{flightId}/` cleanly removes one flight's evidence without touching other flights or sources.
|
||
- Google Maps files: unchanged from the pre-AZ-488 layout (grandfathered, no migration ships with this contract).
|
||
|
||
`tile_x` and `tile_y` are derived server-side from `(latitude, longitude, tile_zoom)` via `GeoUtils.WorldToTilePos`; the client cannot influence the on-disk path beyond providing valid coordinates and an optional `flightId`.
|
||
|
||
Pre-AZ-503 UAV files written at `./tiles/uav/{z}/{x}/{y}.jpg` (no flight segment) are not relocated. Post-AZ-503 anonymous uploads write to `./tiles/uav/none/{z}/{x}/{y}.jpg`. This split is acceptable because AZ-488 only shipped in cycle 2 and the pre-AZ-503 UAV file count is small.
|
||
|
||
## Concurrency
|
||
|
||
- Per-source UPSERT in the DB is the authoritative serialization point for the same cell.
|
||
- Two concurrent UAV uploads for the exact same `(tile_zoom, tile_x, tile_y)` cell may race on the on-disk bytes; the final file is whichever upload wrote last, and the final DB row is whichever upload UPSERTed last. Per-source `file_path` is identical in this race so the file and row remain self-consistent — there is no orphan reference even under contention.
|
||
|
||
## Invariants
|
||
|
||
- **Inv-1**: Status strings are limited to `"accepted"` and `"rejected"`. New status values require a contract minor bump.
|
||
- **Inv-2**: Reject reasons are drawn from the closed enumeration above. New reasons require a contract minor bump.
|
||
- **Inv-3**: Persisted rows are inserted **only** via `ITileRepository.InsertAsync`. No new write path is introduced by this contract.
|
||
- **Inv-4**: `rejectDetails` MUST NOT contain server-side file paths, .NET exception type names, or internal identifiers. Operators consume these messages directly.
|
||
- **Inv-5**: Item ordering in the response matches item ordering in `metadata.items`.
|
||
|
||
## Non-Goals
|
||
|
||
- **Not covered**: Asynchronous or queued processing — the batch is synchronous. Larger batches than `MaxBatchSize` require a new contract (likely async + status-poll) and a major version bump.
|
||
- **Not covered**: Geofence filtering. UAV uploads outside any operational area still succeed; geofence enforcement is a follow-up PBI.
|
||
- **Not covered**: Per-tile photogrammetry metadata (altitude, focal length, sensor dimensions). Excluded from v1.0.0 by user choice during planning.
|
||
- **Not covered**: Streaming uploads. The endpoint reads each `IFormFile` into memory before validation.
|
||
- **Not covered**: Image storage outside the local filesystem (S3, GCS). Matches the existing Google Maps producer behavior.
|
||
- **Not covered**: Compression or re-encoding of accepted JPEGs. Stored as received.
|
||
|
||
## Versioning Rules
|
||
|
||
- **Patch (1.0.x)**: Documentation clarifications; expanded test cases; tightening of `rejectDetails` content; tuning default thresholds without changing the reject-reason enum.
|
||
- **Minor (1.x.0)**: Adding a new `rejectReason` code; adding optional metadata fields with backward-compatible defaults; relaxing rule thresholds in a backward-compatible way; introducing a new permission (e.g., `SAT`).
|
||
- **Major (2.0.0)**: Changing the request envelope shape; removing or renaming a reject-reason code; changing the persistence path; switching to async/status-poll; changing the `permissions` claim required.
|
||
|
||
Each version bump requires updating the Change Log and notifying every consumer listed in the header.
|
||
|
||
## Test Cases
|
||
|
||
| Case | Inputs | Expected | Reference |
|
||
|------|--------|----------|-----------|
|
||
| happy-1-item | 1× valid 256×256 JPEG, captured_at = now, GPS token | HTTP 200, `accepted`, row with `source='uav'`, file at `./tiles/uav/{z}/{x}/{y}.jpg` | AC-1 |
|
||
| mixed-batch | 3 items: valid, 512×512, non-JPEG bytes | HTTP 200, results = `[accepted, rejected:WRONG_DIMENSIONS, rejected:INVALID_FORMAT]`, 1 new row | AC-2 |
|
||
| multi-source | Pre-seeded `google_maps` row at `T1`; UAV upload for same cell at `T2 > T1` | HTTP 200, both rows persist, subsequent read returns the UAV row | AC-3 |
|
||
| same-source-upsert | UAV row at `T1`; second UAV upload at `T2 > T1` | HTTP 200, exactly one `uav` row remains, refreshed `captured_at` and bytes | AC-4 |
|
||
| no-token | No `Authorization` header | HTTP 401 | AC-5 |
|
||
| no-permission | Token with `permissions=["FL"]` | HTTP 403 | AC-6 |
|
||
| oversized | `metadata.items.length` = `MaxBatchSize + 1` | HTTP 400 envelope error | AC-8 |
|
||
|
||
## Change Log
|
||
|
||
| Version | Date | Change | Author |
|
||
|---------|------|--------|--------|
|
||
| 1.0.0 | 2026-05-11 | Initial contract — batch UAV upload endpoint, 5-rule quality gate, per-source UPSERT, closed reject-reason enum, GPS-permission requirement. Produced by AZ-488. | autodev (cycle 2 step 10) |
|
||
| 1.1.0 | 2026-05-12 | Minor bump for AZ-503: added optional `flightId` per-item metadata field (backward-compatible default `null`); `tileId` in the response is now a deterministic UUIDv5 derived from `(z, x, y, source, flightId)` instead of a random Guid; on-disk path adds a `{flightId or 'none'}` segment for per-flight evidence isolation. No reject-reason changes, no envelope changes, no permission changes. v1.0.0 clients omitting `flightId` keep working unchanged. | autodev (cycle 5 step 13) |
|