azaion/satellite-provider
1
0
Fork 0
mirror of https://github.com/azaion/satellite-provider.git synced 2026-06-21 23:21:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-484] [AZ-483] Add task spec + tile-storage v1.0.0 contract draft

Browse Source 
Step-9 (new-task) cycle 1 artifacts for the AZ-483 multi-source tile
storage epic. AZ-485 (UAV upload + quality gate) deferred to a future
Step-9 loop and recorded as planned in the dependencies table.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-11 06:05:06 +03:00
parent 08451df027
commit 5ba58b6c8d
4 changed files with 289 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_tasks/_dependencies_table.md
+12
View File
@@ -58,6 +58,13 @@ Roadmap: `_docs/04_refactoring/03-code-quality-refactoring/analysis/refactoring_
| AZ-380 | C27 | Delete CalculatePolygonDiagonalDistance | 4 | — | 1 | Done (In Testing) |
| AZ-372 | C19 | dotnet format + NetAnalyzers + Coverlet | 4 | — | 3 | Done (In Testing) |
### Step 9 cycle 1 — New Task: Multi-source tile storage + UAV upload (AZ-483 epic)
| Task | Title | Depends On | Points | Status |
|------|-------|-----------|--------|--------|
| AZ-484 | Multi-source tile storage schema (source + captured_at) | — | 5 | To Do |
| AZ-485 (planned) | UAV upload endpoint + quality gate | AZ-484, contract `tile-storage.md` v1.0.0 | ~5 | Not yet created (deferred to a future Step 9 loop) |
## Execution Order
### Step 6
@@ -77,11 +84,16 @@ Phase 2 (Correctness): AZ-359 → AZ-357 → AZ-362 (AZ-362 needs AZ-353)
Phase 3 (Structural cleanup): AZ-366 → AZ-377 → AZ-368 → AZ-367 → AZ-369 → AZ-365 → AZ-364 (folds AZ-360) — AZ-377 needs AZ-371
Phase 4 (Typing/config/tooling/polish): AZ-371 → AZ-370 → AZ-373 → AZ-374 → AZ-375 → AZ-376 → AZ-378 → AZ-379 → AZ-380 → AZ-372
### Step 9 cycle 1 (Multi-source tile storage epic AZ-483)
1. AZ-484 — Multi-source tile storage schema (foundational)
2. AZ-485 (planned) — UAV upload endpoint + quality gate (consumer of AZ-484's contract)
## Total Effort
Step 6: 6 tasks, 17 story points
Step 8 (02-coupling-refactoring): 6 tasks, 17 story points
Step 8 (03-code-quality-refactoring): 27 tasks, ~66 story points
Step 9 cycle 1: 1 task created (AZ-484, 5 pts); 1 deferred (AZ-485)
## Coverage Verification
_docs/02_tasks/todo/AZ-484_multi_source_tile_storage.md
+157
View File
@@ -0,0 +1,157 @@
# Multi-source tile storage schema (source + captured_at)
**Task**: AZ-484_multi_source_tile_storage
**Name**: Multi-source tile storage schema
**Description**: Extend `tiles` to allow one row per `(cell, source)`; add `captured_at`; update repository read selection to most-recent-across-sources; backfill existing rows; publish v1.0.0 storage contract; amend Architecture Vision.
**Complexity**: 5 points
**Dependencies**: None (builds on the post-AZ-357 / C06 schema state)
**Component**: DataAccess + Common (new TileSource enum) + TileDownloader (BuildTileEntity)
**Tracker**: AZ-484
**Epic**: AZ-483
## Problem
The `tiles` table currently allows exactly one row per `(latitude, longitude, tile_zoom, tile_size_meters)` (post-AZ-357 / C06). The Architecture Vision in `architecture.md` already commits to a Layer 1 + Layer 2 model where Google Maps and UAV imagery coexist per cell, but the schema cannot represent two producers for the same geographic cell. T1 (this task) closes that gap so that T2 (UAV upload, AZ-485) has a place to write its rows. Without T1, T2 would have to scaffold a temporary single-source path and immediately throw it away.
## Outcome
- `tiles` table accepts multiple rows per `(latitude, longitude, tile_zoom, tile_size_meters)` distinguished by `source` (enum) and stamps `captured_at` (UTC timestamp).
- Repository read paths return the most-recent row per cell across sources, deterministically (`captured_at DESC`, then `updated_at DESC`, then `id DESC`).
- All existing rows are backfilled to `source='google_maps'`, `captured_at=created_at` — zero orphan rows.
- `architecture.md` § Architecture Vision reflects the N-source model rather than only Layer 1 + Layer 2.
- `_docs/02_document/contracts/data-access/tile-storage.md` v1.0.0 published and referenced from this task and from the future T2 task (AZ-485).
- Region/route flows behave identically to today (Google Maps remains the only producer until T2 lands).
- All 200 unit + 5 smoke tests pass.
## Scope
### Included
- Migration `SatelliteProvider.DataAccess/Migrations/013_AddTileSourceAndCapturedAt.sql`:
- Add `source VARCHAR(32) NOT NULL` and `captured_at TIMESTAMP NOT NULL` columns.
- Backfill existing rows: `source='google_maps'`, `captured_at = created_at`.
- Drop existing unique index `idx_tiles_unique_location` (4-column).
- Create new unique index `idx_tiles_unique_location_source` over `(latitude, longitude, tile_zoom, tile_size_meters, source)`.
- Wrap the entire migration in a single transaction.
- New enum `SatelliteProvider.Common.Enums.TileSource { GoogleMaps, Uav }`, string-stored via the existing `EnumStringTypeHandler` pattern (matches AZ-370 status / point-type enums).
- `SatelliteProvider.DataAccess.Models.TileEntity`: add `Source` (TileSource) and `CapturedAt` (DateTime) fields.
- `SatelliteProvider.DataAccess.Repositories.TileRepository`:
- Update `ColumnList` constant to include `source` and `captured_at`.
- Update `GetByIdAsync`, `GetByTileCoordinatesAsync`, `GetTilesByRegionAsync` to apply the most-recent-across-sources selection rule.
- Update `InsertAsync` to UPSERT on the new 5-column key, also setting `captured_at` and (re)setting `source` from the entity.
- Update `UpdateAsync` SET clause to include `source` and `captured_at`.
- `SatelliteProvider.Services.TileDownloader.TileService.BuildTileEntity`: set `Source = TileSource.GoogleMaps` and `CapturedAt = DateTime.UtcNow`.
- Update existing tests that construct `TileEntity` or mock `ITileRepository` to populate the new fields (12+ sites: `TileServiceTests` ~10, `RegionServiceTests` ~3, `RepositoryRefactorTests` ColumnList assertion, `InfrastructureTests` 2 mocks).
- New unit tests:
- Repository read selection: insert two rows for the same cell with different sources and `captured_at`, assert the latest one is returned by both `GetByTileCoordinatesAsync` and `GetTilesByRegionAsync`.
- Repository same-source UPSERT: re-insert a `uav` row, assert single row remains with updated `captured_at` and `file_path`.
- Migration backfill: pre/post row count + per-row `source='google_maps'` + `captured_at = created_at` invariants.
- Documentation:
- `_docs/02_document/architecture.md` § Architecture Vision amendment (Layer 1 + Layer 2 → N sources, append-by-source, latest-across-sources on read).
- `_docs/02_document/glossary.md`: add `Tile Source` and `Captured At` rows; update `Layer 1` / `Layer 2` rows to acknowledge the N-source generalization.
- `_docs/02_document/module-layout.md` § Component: Common Public API: list `SatelliteProvider.Common/Enums/TileSource.cs`.
- Contract file `_docs/02_document/contracts/data-access/tile-storage.md` v1.0.0 (already drafted; flip Status from `draft` to `frozen` upon implementation completion).
### Excluded
- `POST /api/satellite/upload` implementation — handled by T2 (AZ-485).
- Any quality assessment, threshold gating, or rejection logic — T2.
- Multi-revision / season-based / per-source historical retention — out of scope by design (the quality gate, not history retention, is the cache-freshness mechanism).
- Dropping vestigial `version` or `maps_version` columns (per `coderule.mdc` — column drops require explicit confirmation; user has confirmed leaving them).
- Public HTTP response shape changes (no new fields on `DownloadTileResponse` or any other response).
## Acceptance Criteria
**AC-1: Schema accepts source + captured_at**
Given the migration has run
When a tile row is inserted with `source='uav'` for a cell that already has a `source='google_maps'` row
Then both rows are stored and the unique index does not reject the insert.
**AC-2: Read returns most-recent across sources**
Given a cell has two rows: one `source='google_maps' captured_at=T1` and one `source='uav' captured_at=T2 > T1`
When `GetByTileCoordinatesAsync` (or region read) is called for that cell
Then the row with `captured_at=T2` is returned (single row).
**AC-3: Same-source UPSERT**
Given a cell has a row `source='uav' captured_at=T1`
When another insert arrives for `source='uav' captured_at=T2 > T1`
Then exactly one `source='uav'` row remains for that cell, with `captured_at=T2` and `file_path` updated.
**AC-4: Backfill leaves no orphans**
Given the database had N rows in `tiles` before migration
When `013_AddTileSourceAndCapturedAt.sql` runs
Then the table still has N rows, every row has `source='google_maps'`, and every row has `captured_at = created_at`.
**AC-5: Google Maps download path uses new fields**
Given a tile is downloaded via `TileService.DownloadAndStoreSingleTileAsync` or `DownloadAndStoreTilesAsync`
When the row is inspected
Then `source = 'google_maps'` and `captured_at` is the UTC timestamp at download time.
**AC-6: Existing flows unchanged**
Given the post-T1 build
When `scripts/run-tests.sh --smoke` runs
Then all 200 unit + 5 smoke scenarios pass with no functional change to region/route output (CSV row count, stitched-image presence, ZIP size, status transitions all match pre-T1 baseline).
**AC-7: Vision and contract documents updated**
Given T1 is complete
When `architecture.md`, `glossary.md`, `module-layout.md`, and `_docs/02_document/contracts/data-access/tile-storage.md` are inspected
Then the multi-source model is documented in all four, the contract is v1.0.0 with Status `frozen`, and the contract is referenced from this task spec.
## Non-Functional Requirements
**Performance**
- `GetTilesByRegionAsync` 95th-percentile latency must not regress more than 10% vs the pre-T1 baseline. The new 5-column unique index covers the existing read filter, so no regression is expected; the slow-query log threshold remains 500 ms (`TileRepository.SlowQueryThresholdMs`).
**Compatibility**
- No public HTTP response field added or removed.
- Existing `tiles.maps_version` and `tiles.version` columns remain present and nullable; no consumer reads them in v1.0.0 of the storage contract.
**Reliability**
- Migration MUST be transactional. A failure mid-migration MUST leave the table in its pre-migration state — never with a missing unique index or partially backfilled rows.
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|--------------|------------------|
| AC-1 | `TileRepository.InsertAsync` with two distinct `source` values for the same cell | Both rows inserted; `GetTilesByRegionAsync` returns at most one of them per the selection rule. |
| AC-2 | `TileRepository.GetByTileCoordinatesAsync` against a cell with rows of distinct sources | Row with the highest `captured_at` returned; tie-break by `updated_at DESC` then `id DESC` is deterministic across runs. |
| AC-3 | `TileRepository.InsertAsync` re-inserting a `uav` row for the same cell | Exactly one `uav` row remains with the new `captured_at` and `file_path`. |
| AC-5 | `TileService.BuildTileEntity` (via `DownloadAndStoreSingleTileAsync` test) | Resulting entity has `Source = TileSource.GoogleMaps` and `CapturedAt` close to `DateTime.UtcNow`. |
| AC-6 | All existing `TileServiceTests`, `RegionServiceTests`, `RepositoryRefactorTests`, `InfrastructureTests` after mock fixups | Suite remains green; `RepositoryRefactorTests` ColumnList assertion updated to include the new columns. |
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|--------------------------|--------------|-------------------|-----------------|
| AC-4 | Apply migration 013 to a fixture DB seeded with the migration-012 schema and ≥3 sample rows | Run the migration | Pre-count = post-count, every row has `source='google_maps'`, every row has `captured_at = created_at`, the new unique index exists, the old one does not | Reliability |
| AC-6 | Pre-T1 baseline output captured for `BT-01` (single tile), `BT-03` (200m region), `BT-08` (route+ZIP) | Re-run `scripts/run-tests.sh --smoke` after T1 | All 5 smoke scenarios PASS; `./ready/` outputs match the pre-T1 byte-shape (file existence, row counts in CSV, ZIP size within ±1%) | Performance, Compatibility |
## Constraints
- DB column drops require explicit user confirmation (`coderule.mdc`); none performed in this task. The vestigial `version` and `maps_version` columns remain.
- No rename of any existing column.
- Migration must be transactional and must fail loudly on any inconsistency rather than silently corrupt state — per `coderule.mdc` "never suppress errors silently".
- Cross-component calls continue to flow through `ITileRepository` and `ITileService` interfaces in `Common`; no new compile-time `ProjectReference` between Layer-3 sibling components (the AZ-309 invariant).
## Risks & Mitigation
**Risk 1: Migration fails partway against a non-empty production-like DB**
- *Risk*: A failure mid-migration leaves the table without its unique index or with partially backfilled rows.
- *Mitigation*: Wrap the entire migration in `BEGIN; ... COMMIT;`. Verify on a Docker-Compose dev DB with seed data before applying anywhere else. Add a dedicated unit/integration test that runs the migration against a fixture DB and asserts the post-state.
**Risk 2: Read-path selection rule mis-interacts with existing region cache hit logic**
- *Risk*: `TileService.DownloadAndStoreTilesAsync` does an existence check via `GetTilesByRegionAsync` and then re-downloads missing tiles. If the most-recent-per-cell rule changes which row is returned, region processing may double-download or skip cached tiles.
- *Mitigation*: AC-6 explicitly asserts pre/post equivalence. Smoke-test scenario `BT-03` (200m region) exercises the cache-hit path; pre/post tile-row counts must match.
**Risk 3: TileEntity field additions break test mock construction broadly**
- *Risk*: ~12 test sites construct `TileEntity` directly or set up `Mock<ITileRepository>`. A missed site leaves a default-zero `CapturedAt` that may fail ORDER BY tie-breaks non-deterministically.
- *Mitigation*: Pre-implementation audit of all `new TileEntity` and `Mock<ITileRepository>` sites; the implementer must list them in the batch report. The `RepositoryRefactorTests` ColumnList assertion catches incomplete repository updates.
**Risk 4: `EnumStringTypeHandler` registration drift**
- *Risk*: The existing handler may need an extra registration for `TileSource` to round-trip through Dapper. AZ-370 added similar enums; the registration site must be checked.
- *Mitigation*: Verify and extend the handler registration as part of this task (not a separate ticket).
## Contract
This task produces the contract at `_docs/02_document/contracts/data-access/tile-storage.md` (v1.0.0).
Consumers (T2 — UAV upload endpoint AZ-485, future SatAR provider, etc.) MUST read that file — not this task spec — to discover the storage interface.