azaion/satellite-provider
1
0
Fork 0
mirror of https://github.com/azaion/satellite-provider.git synced 2026-06-21 11:41:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-503] [AZ-504] Cycle 5 Steps 11-15 sync

Browse Source 
Wrap up cycle 5 verification + documentation:
- Steps 10/11 wrap-up reports (implementation_completeness +
  implementation_report) for the AZ-503-foundation + AZ-504 batch.
- Step 12 test-spec sync: AZ-503-foundation/AZ-504 ACs appended;
  AZ-505 deferred ACs recorded.
- Step 13 update-docs: architecture, data-model, glossary, module-
  layout, uav-tile-upload contract (v1.1.0), DataAccess + Services
  + Tests module docs synced; new common_uuidv5.md module doc.
- Step 14 security audit: PASS_WITH_WARNINGS; 0 new Critical/High;
  2 new Low informational (F1 flightId provenance, F2 pgcrypto
  deploy gap).
- Step 15 performance test: PASS_WITH_INFRA_WARNINGS; PT-08
  passed twice (AZ-504 fix verified); PT-01/02 failed due to
  recurring local Docker/colima DNS cold-start (not an app
  regression). Cycle-3 perf-harness leftover stays OPEN with
  replay #5 documented.
- Autodev state moved to Step 16 (Deploy).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-12 18:01:27 +03:00
parent c646aa93e2
commit 61612044fb
27 changed files with 1075 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/03_implementation/implementation_completeness_cycle5_report.md
+121
View File
@@ -0,0 +1,121 @@
# Product Implementation Completeness Gate — Cycle 5
**Cycle**: 5
**Date**: 2026-05-12
**Scope**: AZ-504 (batch 1) + AZ-503-foundation (batch 2)
## Inputs Reviewed
- `_docs/02_tasks/done/AZ-504_perf_script_grep_pipefail_fix.md`
- `_docs/02_tasks/done/AZ-503_tile_identity_uuidv5_bulk_list.md`
- `_docs/02_document/architecture.md`
- `_docs/03_implementation/batch_01_cycle5_report.md`
- `_docs/03_implementation/batch_02_cycle5_report.md`
- Source code under each task's ownership envelope
## Per-Task Classification
### AZ-504 — Perf script: fix `grep | wc -l` pipefail crash
**Verdict**: PASS (with explicit Step 15 deliverables)
Evidence:
- `scripts/run-performance-tests.sh:416-417` — both `accepted` and `rejected` count expressions now wrap `grep -o` in `{ grep -o ... || true; }` so `set -o pipefail` does not kill the pipeline on zero matches.
- AC-1, AC-2 verified by standalone harness (4 cases under `set -e -o pipefail`).
- AC-3, AC-4 are explicitly deferred to autodev Step 15 (Performance Test) by **task spec design** — AC-4's GIVEN clause depends on AC-3 + "the full perf run is green". The deferral is not a gap; it is the natural Step 15 deliverable. Confirmed in `_docs/03_implementation/batch_01_cycle5_report.md` and in the perf-cycle3 leftover entry.
Search for unresolved markers (`placeholder`, `stub`, `TODO`, `NotImplemented`, `fake`, `mock`, `scaffold`, `native bridge`) in the patched file: none in the patched region.
No named runtime dependencies in the task promise.
### AZ-503-foundation — Tile identity → UUIDv5 + integer UPSERT (foundation)
**Verdict**: PASS
Evidence (source code, not tests or reports):
- **`SatelliteProvider.Common/Utils/Uuidv5.cs`** — pure-C# RFC 9562 §5.5 (SHA-1) UUIDv5 implementation, 80 LoC. No third-party dependency. `TileNamespace = 5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c` pinned cross-repo (must match `gps-denied-onboard/components/c6_tile_cache/_uuid.py:TILE_NAMESPACE`).
- **`SatelliteProvider.Common/DTO/UavTileMetadata.cs`** — `FlightId` (Guid?) plumbed through.
- **`SatelliteProvider.DataAccess/Migrations/014_AddTileIdentityColumns.sql`** — applied against the live DB; verified columns `flight_id uuid NULL`, `location_hash uuid NOT NULL`, `content_sha256 bytea NULL`, `legacy_id uuid NULL`; verified `idx_tiles_unique_identity` with `COALESCE(flight_id, '00000000-...'::uuid)`; verified `idx_tiles_location_hash`; verified AZ-484 index `idx_tiles_unique_location_source` dropped.
- **`SatelliteProvider.DataAccess/Models/TileEntity.cs`** — 4 new properties.
- **`SatelliteProvider.DataAccess/Repositories/TileRepository.cs`** — `InsertAsync` UPSERT uses the new integer-only key + `COALESCE(flight_id, ...)`. `id` is intentionally NOT updated on conflict (preserves AC-2 idempotence).
- **`SatelliteProvider.Services.TileDownloader/TileService.cs:146-196`** — `BuildTileEntity` computes deterministic `Id = Uuidv5.Create(TileNamespace, "{z}/{x}/{y}/google_maps/{empty-guid}")` and `LocationHash = Uuidv5.Create(TileNamespace, "{z}/{x}/{y}")`. No `Guid.NewGuid()` call remains. `ContentSha256` computed from the on-disk JPEG body via `SHA256.HashData(stream)`.
- **`SatelliteProvider.Services.TileDownloader/UavTileUploadHandler.cs:144-217`** — `PersistAsync` reads `metadata.FlightId`, computes deterministic `Id` + `LocationHash` + `ContentSha256`, writes file to `./tiles/uav/{flight_id or 'none'}/{z}/{x}/{y}.jpg`. `BuildUavTileFilePath` takes a `Guid? flightId` parameter.
Search for unresolved markers in modified source:
```
$ rg -i 'placeholder|TODO|NotImplemented|scaffold|native bridge' SatelliteProvider.Common/Utils/Uuidv5.cs \
SatelliteProvider.DataAccess/Migrations/014_AddTileIdentityColumns.sql \
SatelliteProvider.DataAccess/Models/TileEntity.cs \
SatelliteProvider.DataAccess/Repositories/TileRepository.cs \
SatelliteProvider.Services.TileDownloader/TileService.cs \
SatelliteProvider.Services.TileDownloader/UavTileUploadHandler.cs
```
→ no matches. (`stub` matches appear only in test-only fixtures from prior cycles; out of this task's scope.)
Named technologies / integrations promised by the task:
- **PostgreSQL `pgcrypto` extension** — migration enables it via `CREATE EXTENSION IF NOT EXISTS pgcrypto` and uses it for the SHA-1-based UUIDv5 backfill. ✓
- **Cross-repo `gps-denied-onboard` UUIDv5 contract** — pinned namespace constant matches by spec (Python parity verified by 10 reference vectors in `Uuidv5Tests`). The onboard repo's matching change is a separate cross-workspace task (`AZ-304` in `gps-denied-onboard`). This is **out of scope for the satellite-provider workspace** by the AZ-503-foundation spec — both sides are coordinated but not co-implemented.
Deferred ACs (5 of 12) are explicitly out of scope per the user-approved split (Option C, /autodev step 10 batch 2):
- AC-5 (inventory endpoint), AC-6 (Leaflet path rewrite), AC-9 (perf SLO), AC-10 (index-only EXPLAIN), AC-12 (HTTP/2) → all moved to AZ-505 with a `Blocks` link.
End-to-end production pipeline check: the `TileTests.RunGetTileByLatLonTest` integration test produced tile id `e228d1aa-25d4-556e-a72d-e0484756e165` — a valid UUIDv5. This confirms the production path (HTTP request → Google Maps download → `TileService.BuildTileEntity` → `TileRepository.InsertAsync` → deterministic id) is connected end-to-end, not just in unit tests.
## Gate Verdict: PASS
Every product task is PASS. No FAIL, no BLOCKED.
- No remediation tasks required.
- Proceed to /implement Step 16 (Final Test Run). Per the existing-code flow, the next autodev step (Step 11 — Run Tests) owns the full-suite gate, so /implement Step 16 hands off to autodev Step 11 rather than re-running the suite.
## Handoff to autodev Step 11 (Run Tests)
The full integration suite was already run twice during /implement Step 6. Results:
- **Run 1** (during batch 2): failed at `UavUploadTests.MultiSourceCoexistence_AZ484_Cycle2` due to my AZ-503 migration making `location_hash` NOT NULL while that seed test inserted a raw row without it. Fix shipped (`UavUploadTests.cs` seed computes location_hash via `Uuidv5.Create`). After fix, the test passed.
- **Run 2** (post-fix): passed JWT (8 ACs), all UAV upload tests (10 ACs including AZ-503 AC-3 + AC-4), and `TileTests.RunGetTileByLatLonTest`. Failed mid-way through `RegionTests.RunRegionProcessingTest_200m_Zoom18` due to intermittent DNS resolution failure on `mt1.google.com` from the Docker test container. This is host-network flakiness, not an AZ-503 regression — same failure mode appeared in Run 1 against `tile.googleapis.com` before the AZ-503 fix was applied.
`MigrationTests` (which sit at the end of the suite, after the flaky Region tests) did not execute via the runner during Run 2 but were verified directly against the running DB: column shape, index shape, deterministic backfill formula match (SQL UUIDv5 of `"18/12345/23456"` = `38b26f49-a966-5121-aaf4-9cc476f57869` — byte-identical to the C# unit test vector), and live row equality on three sampled rows.
Recommendation for autodev Step 11: the user can re-run the full suite or accept the partial evidence above + the verified DB schema. Either way, no AZ-503-related test failures remain.
## Files / Symbols Checked
Production code:
- `SatelliteProvider.Common/Utils/Uuidv5.cs`
- `SatelliteProvider.Common/DTO/UavTileMetadata.cs`
- `SatelliteProvider.DataAccess/Migrations/014_AddTileIdentityColumns.sql`
- `SatelliteProvider.DataAccess/Models/TileEntity.cs`
- `SatelliteProvider.DataAccess/Repositories/TileRepository.cs`
- `SatelliteProvider.Services.TileDownloader/TileService.cs`
- `SatelliteProvider.Services.TileDownloader/UavTileUploadHandler.cs`
- `scripts/run-performance-tests.sh:416-417`
DB schema (live, post-migration):
- `tiles.flight_id` (uuid NULL)
- `tiles.location_hash` (uuid NOT NULL)
- `tiles.content_sha256` (bytea NULL)
- `tiles.legacy_id` (uuid NULL)
- `idx_tiles_unique_identity` (UNIQUE, integer key + `COALESCE(flight_id, ...)`)
- `idx_tiles_location_hash` (non-unique)
- AZ-484 indexes `idx_tiles_unique_location_source` and pre-AZ-484 `idx_tiles_unique_location` dropped.
Tests (existence + AC mapping verified):
- `SatelliteProvider.Tests/Uuidv5Tests.cs` (AC-1 + invariants)
- `SatelliteProvider.Tests/UavTileFilePathTests.cs` (AC-11)
- `SatelliteProvider.Tests/UavTileUploadHandlerTests.cs` (AC-2, AC-3, AC-7, AC-11 unit-level)
- `SatelliteProvider.IntegrationTests/UavUploadTests.cs` (AC-3, AC-4 integration; AZ-488 regression coverage)
- `SatelliteProvider.IntegrationTests/MigrationTests.cs` (3 new AZ-503 assertions + AZ-484 supersession)
## Unresolved Scaffold / Native Placeholders: None
## Named Promised Technologies Not Integrated: None
(`pgcrypto` integrated. Cross-repo Python parity verified by reference vectors but the onboard implementation is owned by a sibling workspace, not satellite-provider.)
## Required Remediation Tasks: None
Cycle 5 is complete from the implementation perspective; AZ-505 carries the deferred scope into a future cycle (already linked in `_docs/02_tasks/_dependencies_table.md`).
_docs/03_implementation/implementation_report_tile_identity_uuidv5_cycle5.md
+107
View File
@@ -0,0 +1,107 @@
# Implementation Report — Cycle 5
**Cycle**: 5
**Date**: 2026-05-12
**Tasks shipped**: AZ-504 (batch 1), AZ-503-foundation (batch 2)
**Verdict**: PASS (Product Implementation Completeness Gate)
## Summary
Cycle 5 delivered the **UUIDv5 tile-identity foundation** plus an **unrelated perf-harness fix** that had been blocking the perf gate since cycle 3.
Original AZ-503 scope was too large; mid-batch, the user-approved Option C split it:
- **AZ-503 (this cycle)** — schema columns, deterministic UUIDv5 ids, integer-only UPSERT, per-flight on-disk paths, content SHA-256 ingestion.
- **AZ-505 (next cycle)** — bulk inventory endpoint, Leaflet covering index, HTTP/2 enablement, perf SLO + EXPLAIN gate.
AZ-505 is created in Jira with a `Blocks` link from AZ-503 and an entry in `_docs/02_tasks/_dependencies_table.md`.
## Batches
| Batch | Tasks | Verdict | Report |
|-------|-------|---------|--------|
| 1 | AZ-504 — perf script `grep | wc -l` pipefail fix | PASS | `batch_01_cycle5_report.md` |
| 2 | AZ-503-foundation — UUIDv5 + integer UPSERT | PASS_WITH_WARNINGS (one Low maintainability finding accepted) | `batch_02_cycle5_report.md` |
Code review accepted PASS_WITH_WARNINGS on batch 2. The single Low-severity finding (legacy-row `content_sha256` left NULLable by migration) is documented in `batch_02_cycle5_report.md`, justified by the volatility of legacy file paths, and bounded by the application-level NOT NULL invariant for new writes.
## Code Changes
### AZ-504 — Perf harness pipefail fix
- `scripts/run-performance-tests.sh:416-417` — wrapped two `grep -o` counters in `{ ... || true; }` so a zero-match doesn't kill the pipeline under `set -o pipefail`.
### AZ-503-foundation — Tile identity
**Schema** (`SatelliteProvider.DataAccess/Migrations/014_AddTileIdentityColumns.sql`):
- Adds `tiles.flight_id uuid NULL`, `tiles.location_hash uuid NOT NULL`, `tiles.content_sha256 bytea NULL`, `tiles.legacy_id uuid NULL`.
- Backfills `location_hash` via a temp PL/pgSQL `pg_temp.uuidv5` function (uses `pgcrypto.digest`) over `{zoom}/{x}/{y}`.
- Drops AZ-484 `idx_tiles_unique_location_source`. Creates `idx_tiles_unique_identity` on `(tile_zoom, tile_x, tile_y, tile_size_meters, source, COALESCE(flight_id, '00000000-0000-0000-0000-000000000000'::uuid))` and `idx_tiles_location_hash`.
**Application code**:
- `SatelliteProvider.Common/Utils/Uuidv5.cs` — RFC 9562 §5.5 SHA-1 UUIDv5, namespace pinned to `5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c` (cross-repo contract with `gps-denied-onboard`).
- `SatelliteProvider.Common/DTO/UavTileMetadata.cs` — `Guid? FlightId` plumbed through.
- `SatelliteProvider.DataAccess/Models/TileEntity.cs` — 4 new properties.
- `SatelliteProvider.DataAccess/Repositories/TileRepository.cs` — UPSERT now uses the integer + flight-id key. `id` is intentionally NOT overwritten on conflict (preserves AC-2 idempotence semantics).
- `SatelliteProvider.Services.TileDownloader/TileService.cs` — `BuildTileEntity` computes deterministic `Id`, `LocationHash`, `ContentSha256` for Google-Maps tiles; `FlightId = null`.
- `SatelliteProvider.Services.TileDownloader/UavTileUploadHandler.cs` — reads `metadata.FlightId`, computes deterministic identity fields, writes file under `./tiles/uav/{flight_id|none}/{z}/{x}/{y}.jpg`.
## Test Changes
- `SatelliteProvider.Tests/Uuidv5Tests.cs` — 10 reference vectors verifying byte-identical parity with Python's `uuid.uuid5` (AC-1).
- `SatelliteProvider.Tests/UavTileFilePathTests.cs` — anonymous-flight `/uav/none/` segment + per-flight directory + cross-flight path-distinctness (AC-11).
- `SatelliteProvider.Tests/UavTileUploadHandlerTests.cs` — two new tests for AC-2/AC-3/AC-7/AC-11 (multi-flight same-cell coexistence; identical upload determinism).
- `SatelliteProvider.IntegrationTests/MigrationTests.cs` — 3 new AZ-503 assertions (columns exist + nullability; new unique index covers integer key + flight id; `location_hash` backfill matches SQL `pg_temp.uuidv5` formula) and renamed the AZ-484 supersession test.
- `SatelliteProvider.IntegrationTests/UavUploadTests.cs` — two new integration tests (`MultiFlightUavRowsCoexist_AZ503_AC3`, `FloatRoundingDoesNotBreakIdempotence_AZ503_AC4`); the pre-existing `MultiSourceCoexistence_AZ484_Cycle2` seeder updated to compute `location_hash` via `Uuidv5.Create` to satisfy the new NOT NULL constraint.
- `SatelliteProvider.IntegrationTests/SatelliteProvider.IntegrationTests.csproj` — added project reference to `SatelliteProvider.Common` so test seeders can call `Uuidv5.Create`.
## AC Coverage
| AC | Status | Test |
|----|--------|------|
| AC-1 UUIDv5 cross-language parity | Covered | `Uuidv5Tests` |
| AC-2 Idempotent re-ingest (same id) | Covered | `UavTileUploadHandlerTests.HandleAsync_IdenticalUpload_*` + `FloatRoundingDoesNotBreakIdempotence_AZ503_AC4` |
| AC-3 Two flights same cell coexist | Covered | `UavTileUploadHandlerTests.HandleAsync_TwoFlightsSameCell_*` + `UavUploadTests.MultiFlightUavRowsCoexist_AZ503_AC3` |
| AC-4 Float rounding does not break idempotence | Covered | `UavUploadTests.FloatRoundingDoesNotBreakIdempotence_AZ503_AC4` |
| AC-5 Inventory endpoint | **Deferred → AZ-505** | n/a |
| AC-6 Leaflet covering index | **Deferred → AZ-505** | n/a |
| AC-7 Deterministic content_sha256 on new writes | Covered | `UavTileUploadHandlerTests.HandleAsync_IdenticalUpload_*` |
| AC-8 Migration columns + indexes | Covered | `MigrationTests.Az503ColumnsExistAndLocationHashIsNotNull` + `Az503NewUniqueIndexCoversIntegerKeyAndFlightId` |
| AC-9 Perf SLO | **Deferred → AZ-505** | n/a |
| AC-10 Index-only EXPLAIN | **Deferred → AZ-505** | n/a |
| AC-11 Per-flight on-disk path | Covered | `UavTileFilePathTests.BuildUavTileFilePath_*` + handler/integration tests |
| AC-12 HTTP/2 | **Deferred → AZ-505** | n/a |
**Foundation half: 7 of 7 in-scope ACs covered.** (5 ACs intentionally deferred via approved scope split.)
AZ-504: AC-1, AC-2 covered by harness; AC-3, AC-4 are explicit Step 15 (Performance Test) deliverables and were not implemented in this batch by task-spec design.
## Completeness Gate
`_docs/03_implementation/implementation_completeness_cycle5_report.md` — **PASS**. Every product task is PASS, no remediation tasks required.
## Handoff to autodev Step 11 (Run Tests)
Per `/implement` Step 16: since the next existing-code flow step is **Run Tests**, the implement skill does **not** run the full suite again. The `test-run` skill owns the full-suite gate to avoid duplicate runs.
Evidence already on file (from `/implement` Step 6 runs during the cycle):
- Run 1 (during batch 2) caught a pre-existing seeder incompatibility with the new NOT NULL `location_hash`; fix shipped in-batch; the test then passed.
- Run 2 (post-fix) green on JWT (8 ACs), all UAV upload tests including AZ-503 AC-3 + AC-4, and `TileTests.RunGetTileByLatLonTest`. Failed mid-suite on `RegionTests.RunRegionProcessing*` due to intermittent host-network DNS failure resolving `mt1.google.com` / `tile.googleapis.com` from the Docker test container — **infrastructure flake, not an AZ-503 regression**.
- `MigrationTests` (which sit after the flaky Region tests in run order) were verified directly against the running DB during Step 15: column shape, index shape, deterministic backfill formula (SQL `pg_temp.uuidv5('18/12345/23456')` = `38b26f49-a966-5121-aaf4-9cc476f57869` byte-identical to the C# `Uuidv5Tests` vector), and equality on three sampled live rows.
Recommendation for `test-run`: re-run the full suite and treat any further `mt1.google.com` / `tile.googleapis.com` DNS failures as host-network flakiness (out of scope for AZ-503). All AZ-503-touched test paths have passed at least once during the cycle.
## Git
- Branch: `dev`
- Auto-push: enabled this session
- Commits pushed (subject lines):
- `[AZ-504] Perf script: guard grep|wc against pipefail on zero matches`
- `[AZ-503] Tile identity: UUIDv5 + integer UPSERT + per-flight paths`
## Open Items
- AZ-505 (To Do, blocked by AZ-503) carries the deferred scope: `POST /api/satellite/tiles/inventory`, Leaflet covering index, HTTP/2, perf SLO + EXPLAIN gate. Scheduled for next cycle.
- Cross-repo: `gps-denied-onboard` needs the matching UUIDv5 + namespace constant (`TILE_NAMESPACE`) wire-up. Tracked separately in that workspace (out of scope for satellite-provider).
- Perf-harness leftover `_docs/_process_leftovers/2026-05-12_perf-cycle3-harness-execution.md` becomes replayable now that AZ-504 lands; `test-run`/`performance-test` will pick it up.