mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-22 02:21:14 +00:00
[AZ-503] [AZ-504] Cycle 5 Steps 11-15 sync
Wrap up cycle 5 verification + documentation: - Steps 10/11 wrap-up reports (implementation_completeness + implementation_report) for the AZ-503-foundation + AZ-504 batch. - Step 12 test-spec sync: AZ-503-foundation/AZ-504 ACs appended; AZ-505 deferred ACs recorded. - Step 13 update-docs: architecture, data-model, glossary, module- layout, uav-tile-upload contract (v1.1.0), DataAccess + Services + Tests module docs synced; new common_uuidv5.md module doc. - Step 14 security audit: PASS_WITH_WARNINGS; 0 new Critical/High; 2 new Low informational (F1 flightId provenance, F2 pgcrypto deploy gap). - Step 15 performance test: PASS_WITH_INFRA_WARNINGS; PT-08 passed twice (AZ-504 fix verified); PT-01/02 failed due to recurring local Docker/colima DNS cold-start (not an app regression). Cycle-3 perf-harness leftover stays OPEN with replay #5 documented. - Autodev state moved to Step 16 (Deploy). Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -20,7 +20,8 @@ The three Layer-3 service components are compile-time siblings: each only refere
|
||||
|
||||
**Architectural principles** (inferred):
|
||||
- Single-instance deployment, no horizontal scaling requirements (`inferred-from: Channel-based queue, no distributed state`)
|
||||
- Append-by-source tile storage — multiple producers (Google Maps, UAV upload, future SatAR, …) can each persist a row per `(latitude, longitude, tile_zoom, tile_size_meters)` cell. Reads return the most-recent row across sources, ordered by `captured_at DESC` with deterministic `(updated_at DESC, id DESC)` tie-breaks. The single-row-per-cell-per-source invariant is enforced by the 5-column unique index `idx_tiles_unique_location_source` introduced in migration 013 (AZ-484). The `tiles.version` column is vestigial since AZ-357 dropped year-based cache invalidation in favour of cell-level overwrite. (`inferred-from: tiles table + AZ-484/AZ-357 migrations + tile-storage contract v1.0.0`)
|
||||
- Append-by-source-and-flight tile storage (AZ-503; refines AZ-484) — multiple producers (Google Maps, UAV upload from N flights, future SatAR, …) can each persist a row per `(tile_zoom, tile_x, tile_y, tile_size_meters)` cell. Reads return the most-recent row across sources, ordered by `captured_at DESC` with deterministic `(updated_at DESC, id DESC)` tie-breaks. The single-row-per-cell-per-source-per-flight invariant is enforced by `idx_tiles_unique_identity` (6-column integer-only, `COALESCE(flight_id, '00000000-...'::uuid)`) introduced in migration 014; this supersedes the AZ-484 float-based `idx_tiles_unique_location_source`. Identity is deterministic across re-ingests: `tiles.id = Uuidv5(TileNamespace, "{z}/{x}/{y}/{source}/{flight_id or zero-uuid}")` and `tiles.location_hash = Uuidv5(TileNamespace, "{z}/{x}/{y}")`. The `tiles.version` column remains vestigial since AZ-357 dropped year-based cache invalidation in favour of cell-level overwrite. (`inferred-from: tiles table + AZ-484/AZ-357/AZ-503 migrations + tile-storage contract v1.0.0`)
|
||||
- Cross-repo deterministic tile identity (AZ-503) — the `TileNamespace` UUID `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c` and the canonical name format are shared with the sibling workspace `gps-denied-onboard` (`components/c6_tile_cache/_uuid.py:TILE_NAMESPACE`). Both sides MUST produce byte-identical UUIDv5 output so an onboard-cached tile and a server-cached tile for the same `(z, x, y, source, flight_id)` are recognized as the same artifact without a round-trip. Changing the namespace constant on either side is a coordinated cross-repo break. (`inferred-from: Uuidv5.cs, AZ-503 task spec § Constraints`)
|
||||
- Fire-and-forget async processing with status polling (`inferred-from: queue + background service + status endpoint`)
|
||||
- JWT-validated callers only — every HTTP endpoint requires a valid HS256-signed Bearer token, validated locally against a shared `JWT_SECRET` per the suite-level auth contract (`suite/_docs/10_auth.md`). Issuer/audience are intentionally not validated yet; signature + lifetime + ≥32-byte key are. Per-endpoint permission claims (e.g. `permissions: ["GPS"]` on the UAV upload) layer on top of this baseline.
|
||||
|
||||
@@ -34,11 +35,11 @@ The three Layer-3 service components are compile-time siblings: each only refere
|
||||
**Planned features** (confirmed by user, currently stubs):
|
||||
- MGRS endpoint — tile access via Military Grid Reference System coordinates
|
||||
|
||||
**Multi-source tile producers** (live as of AZ-488):
|
||||
- *Google Maps* — `TileService.DownloadAndStoreTilesAsync` / `DownloadAndStoreSingleTileAsync` stamp `source='google_maps'` on every persisted row; tile JPEGs live under `{StorageConfig.TilesDirectory}/{zoom}/...` per the legacy grandfathered layout.
|
||||
- *UAV* — `POST /api/satellite/upload` (AZ-488) accepts a multipart batch of UAV-captured tiles, runs each item through a 5-rule quality gate (`UavTileQualityGate`), and persists accepted items via `ITileRepository.InsertAsync` with `source='uav'`. UAV JPEGs live under `{StorageConfig.TilesDirectory}/uav/{zoom}/{x}/{y}.jpg`. Requires the `GPS` permission claim on top of the JWT baseline.
|
||||
**Multi-source tile producers** (live as of AZ-488; per-flight evidence isolation as of AZ-503):
|
||||
- *Google Maps* — `TileService.DownloadAndStoreTilesAsync` / `DownloadAndStoreSingleTileAsync` stamp `source='google_maps'`, `flight_id=NULL`, and a deterministic UUIDv5 `id` on every persisted row; tile JPEGs live under `{StorageConfig.TilesDirectory}/{zoom}/...` per the legacy grandfathered layout. `content_sha256` is computed from the on-disk JPEG body.
|
||||
- *UAV* — `POST /api/satellite/upload` (AZ-488; per-flight key extended by AZ-503) accepts a multipart batch of UAV-captured tiles, runs each item through a 5-rule quality gate (`UavTileQualityGate`), and persists accepted items via `ITileRepository.InsertAsync` with `source='uav'`, `flight_id = metadata.flightId` (or NULL for anonymous uploads), and a deterministic UUIDv5 `id`. UAV JPEGs live under `{StorageConfig.TilesDirectory}/uav/{flight_id or 'none'}/{zoom}/{x}/{y}.jpg`, so `rm -rf ./tiles/uav/{flight_id}/` removes one flight's evidence without touching other flights at overlapping cells. Requires the `GPS` permission claim on top of the JWT baseline.
|
||||
|
||||
The N-source storage contract is authoritative in `_docs/02_document/contracts/data-access/tile-storage.md` (v1.0.0). The UAV upload contract is authoritative in `_docs/02_document/contracts/api/uav-tile-upload.md` (v1.0.0). Anything that reads or writes `tiles` MUST follow those contracts rather than re-deriving the rules from prose here.
|
||||
The N-source storage contract is authoritative in `_docs/02_document/contracts/data-access/tile-storage.md` (v1.0.0; v2.0.0 bump tracking the AZ-503 identity columns is deferred to AZ-505 when the new identity surface freezes for external consumers). The UAV upload contract is authoritative in `_docs/02_document/contracts/api/uav-tile-upload.md` (v1.0.0; AZ-503 added an optional `flightId` field to per-item metadata — backward-compatible). Anything that reads or writes `tiles` MUST follow those contracts rather than re-deriving the rules from prose here.
|
||||
|
||||
**Drift signals**:
|
||||
- `geofence_polygons` mentioned in AGENTS.md as a routes table column but does not exist in schema or entity — documentation drift
|
||||
@@ -183,13 +184,14 @@ The N-source storage contract is authoritative in `_docs/02_document/contracts/d
|
||||
|
||||
**Context**: Tiles are immutable JPEG images that need fast random access.
|
||||
|
||||
**Decision**: Store tiles as files in a directory hierarchy with metadata in PostgreSQL. The layout is per-source so the bytes for `google_maps` and `uav` writes for the same cell remain individually addressable on disk:
|
||||
**Decision**: Store tiles as files in a directory hierarchy with metadata in PostgreSQL. The layout is per-source (and per-flight for UAV since AZ-503) so the bytes for distinct producers / flights at the same cell remain individually addressable on disk:
|
||||
- Google Maps (legacy, grandfathered): `{StorageConfig.TilesDirectory}/{zoom}/{x_bucket}/{y_bucket}/tile_{zoom}_{x}_{y}_{timestamp}.jpg`
|
||||
- UAV (AZ-488): `{StorageConfig.TilesDirectory}/uav/{zoom}/{x}/{y}.jpg`
|
||||
- UAV anonymous (AZ-488 baseline): `{StorageConfig.TilesDirectory}/uav/none/{zoom}/{x}/{y}.jpg`
|
||||
- UAV per-flight (AZ-503): `{StorageConfig.TilesDirectory}/uav/{flight_id}/{zoom}/{x}/{y}.jpg`
|
||||
|
||||
The authoritative source marker is the `tiles.source` column; the per-source on-disk path matters only for write isolation between producers.
|
||||
The authoritative source/flight markers are the `tiles.source` and `tiles.flight_id` columns; the per-source / per-flight on-disk path matters only for write isolation and bulk-delete granularity.
|
||||
|
||||
**Consequences**: Fast reads, easy backup/migration, both producers can run without colliding on bytes, but requires shared filesystem for multi-instance (which is not currently needed). No migration of pre-AZ-488 Google Maps files is shipped — the legacy layout stays intact.
|
||||
**Consequences**: Fast reads, easy backup/migration, producers can run without colliding on bytes, and per-flight `rm -rf` becomes safe. Requires shared filesystem for multi-instance (not currently needed). No migration of pre-AZ-488 Google Maps files is shipped — the legacy layout stays intact. Pre-AZ-503 UAV files written by the AZ-488 baseline at `./tiles/uav/{z}/{x}/{y}.jpg` (no flight segment) are not relocated by the migration; the post-AZ-503 code writes anonymous uploads to `./tiles/uav/none/{z}/{x}/{y}.jpg` and the original AZ-488-era files stay where they were. This is acceptable because AZ-488 only landed in cycle 2 and the volume of pre-AZ-503 UAV bytes is small (no production UAV upload traffic yet).
|
||||
|
||||
### ADR-005: Background Hosted Services for Processing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user