Commit Graph

14 Commits

Author SHA1 Message Date
Oleksandr Bezdieniezhnykh 865dfdb3b9 [AZ-794] [AZ-795] [AZ-796] Strict input validation + z/x/y rename
ci/woodpecker/push/01-test Pipeline was successful
ci/woodpecker/push/02-build-push Pipeline was successful
AZ-794: rename inventory wire fields tileZoom/tileX/tileY -> z/x/y
to match the slippy-map URL convention. Contract bumped to v2.0.0.

AZ-795: shared validation infrastructure -- FluentValidation +
ValidationEndpointFilter + GlobalValidatorConfig (camelCase paths).
GlobalExceptionHandler now converts JsonException (UnmappedMember +
JsonRequired) into RFC 7807 ValidationProblemDetails. JSON layer
hardened with UnmappedMemberHandling.Disallow + camelCase naming
policy. New error-shape.md contract.

AZ-796: InventoryRequestValidator covers 9 rules (XOR tiles vs
locationHashes, cap 1000, z 0..22, x/y in slippy bounds, hash
length/charset). 16 unit tests + 16 integration tests + a manual
curl probe script.

Adjacent fixes uncovered by the new strict layer:
- IdempotentPostTests RoutePoint payload corrected to lat/lon
  (the DTO has used JsonPropertyName for ages; previously silently
  ignored under PascalCase fallback).
- TileInventoryTests slippy x/y reduced to fit z=18 bounds.
- docker-compose.yml host port for Postgres moved 5432 -> 5433 to
  avoid sibling-project conflict; appsettings.Development + README
  + AGENTS + architecture + containerization docs aligned.

New coderule (suite + repo): API consumer-facing OpenAPI
descriptions must not contain task IDs, contract filenames, or
version-bump history -- internal change tracking belongs in
commits/contract docs/changelogs. Existing offending descriptions
in Program.cs cleaned up.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:02:02 +03:00
Oleksandr Bezdieniezhnykh 909f69cb3a [AZ-505] Tile inventory endpoint + HTTP/2 + Leaflet covering index
Production code:
- POST /api/satellite/tiles/inventory (XOR body, 5000-cap,
  most-recent-per-location_hash select, present/absent shaping).
- Kestrel HttpProtocols.Http1AndHttp2 on every listener (AC-5).
- Migration 015 creates tiles_leaflet_path covering index over
  (location_hash, captured_at DESC, updated_at DESC, id DESC)
  INCLUDE (file_path, source); drops superseded idx_tiles_location_hash.
- TileRepository.GetByTileCoordinatesAsync rewired to filter by
  location_hash (Index Only Scan via tiles_leaflet_path).
- TileRepository.GetTilesByLocationHashesAsync added with Npgsql-
  direct ANY($1::uuid[]) binding (Dapper IEnumerable expansion is
  incompatible with the array form).
- Uuidv5.LocationHashForTile centralises the UUIDv5(TileNamespace,
  "{z}/{x}/{y}") formula — single source of truth for the cross-repo
  invariant (gps-denied-onboard parity).

Contracts:
- New: contracts/api/tile-inventory.md v1.0.0.
- Bumped: contracts/data-access/tile-storage.md to v2.0.0 (joint
  ownership by AZ-503-foundation + AZ-505: schema + covering index +
  GetByTileCoordinatesAsync rewrite).

Tests:
- TileInventoryTests covers AC-1, AC-2 (DB-level), AC-4, AC-6.
- Http2MultiplexingTests covers AC-5 (20 concurrent multiplexed GETs
  over h2c via SocketsHttpHandler + AppContext Http2Unencrypted switch).
- LeafletPathIndexOnlyTests covers AC-3 (EXPLAIN (ANALYZE, BUFFERS)
  asserts Index Only Scan over tiles_leaflet_path with heap_blocks=0).

Docs:
- architecture.md, system-flows.md, data_model.md, module-layout.md,
  glossary.md, modules/api_program.md, modules/dataaccess_tile_repository.md,
  components/02_data_access/description.md all updated to reference the
  v2.0.0 tile-storage contract + new tile-inventory contract + AC-7.

Reports:
- batch_01_cycle6_report.md, batch_01_cycle6_review.md,
  implementation_completeness_cycle6_report.md (PASS),
  implementation_report_tile_inventory_cycle6.md.

Task spec moved todo/ -> done/.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 21:16:37 +03:00
Oleksandr Bezdieniezhnykh c646aa93e2 [AZ-503] Tile identity → UUIDv5 + integer UPSERT (foundation)
ci/woodpecker/push/01-test Pipeline was successful
ci/woodpecker/push/02-build-push Pipeline was successful
Foundation half of original AZ-503 (split during /autodev step 10 batch 2
on user choice; deferred work moved to AZ-505 with a Blocks link).

Adds deterministic tile identity (UUIDv5 over (z, x, y, source, flight_id))
shared cross-repo with gps-denied-onboard via the pinned TileNamespace
5b8d0c2e-7f1a-4d3b-9c5e-1f3a8e7d2b6c, switches the tiles UPSERT key from
floats to integers with per-flight separation, plumbs FlightId through
UavTileMetadata + handler, and writes UAV evidence to per-flight
on-disk directories so two flights at the same (z, x, y) coexist.

- Common: pure-C# RFC 9562 Uuidv5 (no third-party dep) + FlightId DTO
  field; 10 Python-reference unit vectors verify byte parity.
- DataAccess: migration 014 adds flight_id (uuid NULL), location_hash
  (uuid NOT NULL, backfilled via session-scoped pg_temp.uuidv5),
  content_sha256 (bytea NULL), legacy_id (uuid NULL = preserves
  pre-AZ-503 random id one cycle); drops idx_tiles_unique_location_source
  (AZ-484) and adds idx_tiles_unique_identity keyed on
  (tile_zoom, tile_x, tile_y, tile_size_meters, source,
   COALESCE(flight_id, '00000000-...'::uuid)) + idx_tiles_location_hash.
- TileRepository: ColumnList + UPSERT updated; id never updated on
  conflict (preserves AC-2 idempotence). UpdateAsync extended.
- Services: TileService and UavTileUploadHandler compute deterministic
  Id + LocationHash + ContentSha256 before insert; UAV file path
  becomes ./tiles/uav/{flight_id or 'none'}/{z}/{x}/{y}.jpg.
- Tests: Uuidv5Tests (10 reference vectors), UavTileFilePathTests
  (per-flight + anonymous paths), UavTileUploadHandlerTests (AC-2,
  AC-3, AC-7, AC-11 unit-level), UavUploadTests (AC-3 + AC-4
  integration: multi-flight DB coexistence with shared location_hash
  + distinct file_path; float-different lat/lon collapse to 1 row),
  MigrationTests (column shape, idx_tiles_unique_identity supersedes
  AZ-484 index, deterministic backfill).
- IntegrationTests project references Common to reuse Uuidv5 in raw
  SQL seeds.
- AZ-488 MultiSourceCoexistence seed fixed to populate location_hash
  (otherwise migration 014's NOT NULL constraint fails).

ACs covered: AC-1, AC-2, AC-3, AC-4, AC-7, AC-8, AC-11.
ACs deferred to AZ-505: AC-5, AC-6, AC-9, AC-10, AC-12.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 17:07:35 +03:00
Oleksandr Bezdieniezhnykh 813136326f [AZ-500] .NET 8 -> .NET 10 migration
Coordinated cross-cutting bump: 9 csproj TFMs net8.0 -> net10.0;
global.json sdk.version 8.0.0 -> 10.0.0; all Dockerfiles + scripts/
+ .woodpecker on mcr.microsoft.com/dotnet/{sdk,aspnet,runtime}:10.0;
all Microsoft.AspNetCore.* (8.0.25) and Microsoft.Extensions.* (9.0.10)
packages -> 10.0.7. Serilog.AspNetCore retained at 8.0.3 (10.0.0
requires Serilog.Sinks.File >= 7.0.0; out of AZ-500 scope per "no
unrelated package bumps") -- documented in AGENTS.md. Swashbuckle
9.x bumped to 10.1.7 to track Microsoft.OpenApi 2.x; Program.cs +
ParameterDescriptionFilter.cs refactored for the 2.x namespace
(Microsoft.OpenApi), OpenApiSecuritySchemeReference, JsonSchemaType
enum, and IOpenApiSchema dictionary properties. Fixed implicit AC-5
prereq: scripts/run-performance-tests.sh PERF_DLL path bin/Release/
net8.0 -> net10.0. Docs sync: architecture.md + AGENTS.md.

ACs verified: AC-1..AC-4 + AC-7 + AC-8 by grep + build; AC-6 by
./scripts/run-tests.sh --full (271/271 unit tests + full integration
suite green); AC-5 short bootstrap-smoke (PERF_REPEAT_COUNT=2
PERF_UAV_BATCH_SIZE=2) succeeded at the bootstrap step (no exit 3),
PT-01..PT-07 PASS. PT-08 surfaced a pre-existing grep-pipefail bug
in run-performance-tests.sh:417 -- not an SDK problem; recorded as
follow-up in the perf-cycle3 leftover. Code review verdict:
PASS_WITH_WARNINGS (2 Medium deferred per scope discipline:
WithOpenApi ASPDEPR002 deprecation x8, CS8604 nullable in
ParameterDescriptionFilter.cs; both targeted at follow-up PBIs).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 05:28:01 +03:00
Oleksandr Bezdieniezhnykh 1802d32107 [AZ-488] UAV tile batch upload + 5-rule quality gate
Replaces the 501 stub at POST /api/satellite/upload with a multipart
batch endpoint that ingests UAV-captured tiles, runs each item through
a 5-rule quality gate, and persists accepted tiles via the AZ-484
multi-source storage path with source='uav'.

Quality gate (in fixed order, first failure wins): JPEG format
(content-type + magic), size band 5 KiB-5 MiB, exact 256x256
dimensions, captured-at age (no future >30 s skew, no older than
7 days), luminance variance on 32x32 downsample. Closed reject-reason
enumeration in v1.0.0 contract.

Authorization: custom PermissionsRequirement / PermissionsAuthorization
Handler that reads the JWT `permissions` claim (tolerates both
repeated-string and JSON-array shapes). Endpoint protected by
RequiresGpsPermission policy; 401 without token, 403 without GPS perm.

Persistence: file-first to ./tiles/uav/{z}/{x}/{y}.jpg, then
ITileRepository.InsertAsync UPSERT (per-source UPSERT contract from
AZ-484). Per-item failures reported in response without aborting the
batch. Kestrel MaxRequestBodySize and FormOptions limits set to
MaxBatchSize x MaxBytes (default 100 x 5 MiB = 500 MiB).

New frozen contract: _docs/02_document/contracts/api/uav-tile-upload.md
v1.0.0. PT-08 NFR added to performance-tests.md as Deferred (harness
work tracked in PT-07 leftover, per AZ-488 § Risk 4).

Tests: 11 quality-gate unit tests, 5 handler unit tests, 3 file-path
unit tests, 12 permission-handler unit tests, 7 integration tests
(AC-1..AC-6, AC-8). All 253 unit tests + smoke integration suite
green.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 23:50:49 +03:00
Oleksandr Bezdieniezhnykh e9d6db077c [AZ-484] Fix multi-source tile reads: drop Dapper enum handler
Two integration-test failures uncovered after the initial commit:

1) GetTilesByRegionAsync outer ORDER BY referenced 'updated_at' but
   the inner DISTINCT ON subquery aliased it to 'UpdatedAt' (Postgres
   folds to 'updatedat'). DISTINCT ON already guarantees one row per
   (latitude, longitude, ...) so the third tiebreak was unreachable;
   removed it.

2) Dapper 2.1.35 silently bypasses SqlMapper.TypeHandler<T> for enum
   types during read deserialization (Dapper issue #259). The
   TileSourceTypeHandler worked for writes but reads fell through to
   Enum.TryParse, which cannot map 'google_maps' to GoogleMaps.

   Pivoted: TileEntity.Source is now a string (the wire value).
   TileSource enum stays as the public producer surface in
   Common.Enums; TileSourceConverter (Common.Enums) provides
   ToWireValue / FromWireValue / IsValidWireValue at the boundary.
   TileSourceTypeHandler deleted; registration removed from
   DapperEnumTypeHandlers.RegisterAll.

   tile-storage.md Inv-5 amended to document the storage choice.
   _docs/LESSONS.md L-001 records the Dapper bypass for future cycles.

Full suite passes (213 unit + integration suite incl. AZ-484
AC-1..AC-5, security SEC-01..SEC-04, AZ-356/362/357).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 06:44:34 +03:00
Oleksandr Bezdieniezhnykh 687d6bdd5b [AZ-484] Multi-source tile storage: source + captured_at
Add per-source tile rows to support multi-provider imagery (Google
Maps + future UAV). Migration 013 (transactional) introduces
source/captured_at columns, backfills existing rows to
(source='google_maps', captured_at=created_at), and replaces the
4-column unique index with a 5-column index that includes source.

TileRepository:
- ColumnList includes source + captured_at
- GetByTileCoordinatesAsync returns most-recent row across sources
  (ORDER BY captured_at DESC, updated_at DESC, id DESC)
- GetTilesByRegionAsync uses DISTINCT ON to pick the most-recent
  tile per cell, restoring caller-facing row order
- Insert/Update upsert on the new 5-column conflict key

TileSource enum lives in Common.Enums. Snake_case wire format
(google_maps, uav) is enforced by a focused TileSourceTypeHandler
because the generic ToLowerInvariant pattern would emit
"googlemaps", violating contract v1.0.0.

TileService stamps Source=GoogleMaps + CapturedAt=UtcNow on every
new tile. Tile-storage contract is now frozen at v1.0.0.

AC coverage 7/7. New unit + integration tests cover all ACs;
existing 200 unit + 5 smoke tests preserved.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 06:21:59 +03:00
Oleksandr Bezdieniezhnykh 9a53bff92e [AZ-375] [AZ-377] HashSet tile lookup + consolidate Earth constants
ci/woodpecker/push/01-test Pipeline was successful
ci/woodpecker/push/02-build-push Pipeline was successful
Batch 24 of 03-code-quality-refactoring run; closes the run.

AZ-375 (C22): GoogleMapsDownloaderV2.DownloadTilesGridAsync now
builds a HashSet<(int X, int Y, int Z)> once from existingTiles
and tests Contains((x, y, zoomLevel)) per cell. Removes the per-cell
FirstOrDefault tolerance scan and the unused _processingConfig
.LatLonTolerance reference at this site.

AZ-377 (C24): promote Earth + tile-pixel constants to a single
home. GeoUtils now exposes EarthRadiusMeters, EarthEquatorial
CircumferenceMeters, MetersPerDegreeLatitude as public const.
MapConfig adds DefaultTileSizePixels (const) wired as the
TileSizePixels property default. TileRepository and Google
MapsDownloaderV2 read those constants instead of duplicating
the literals 6378137, 40075016.686, 111000.0, and 256.

Tests: +6 new (DownloaderRefactorTests, extended GeoUtils
RefactorTests). 200/200 unit tests pass.

Cumulative K=3 review (batches 22-24): PASS_WITH_WARNINGS,
4 Low findings only — see
_docs/03_implementation/reviews/cumulative_review_22-24.md.

Tooling fix: scripts/run-tests.sh --unit-only path now restores
before testing (was failing on SixLabors resolution in clean
container). Stripped stray BOM from MapConfig.cs to satisfy the
.editorconfig charset gate.

Updates _dependencies_table.md to reflect all 27 03-code-quality-
refactoring tasks done; updates _autodev_state.md to refactor
phase 5 (test-sync).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 05:14:51 +03:00
Oleksandr Bezdieniezhnykh 534ab41b8e [AZ-372] Apply dotnet format whitespace cleanup; archive batch 22
ci/woodpecker/push/01-test Pipeline was successful
ci/woodpecker/push/02-build-push Pipeline was successful
Pure whitespace-only cleanup uncovered by the new format gate from the
previous commit. Verified via `git diff -w --stat`: only 4 files differ
when whitespace is ignored, and those differ only by the BOM byte.

Cleanup kinds applied across 22 source files:
- BOM removal (MapConfig.cs, SatTile.cs, GeoUtils.cs,
  IntegrationTests/Program.cs)
- CRLF -> LF (IntegrationTests/Program.cs)
- Trailing whitespace on blank lines (Common, Api, DataAccess,
  IntegrationTests, Services.RegionProcessing,
  Services.TileDownloader)
- Final newline added (RoutePoint.cs, GeoPoint.cs, others)

After this commit `dotnet format whitespace SatelliteProvider.sln
--verify-no-changes` exits 0; AC-1 is enforceable from `scripts/
run-tests.sh` going forward.

Also lands the batch 22 report, code-review report
(PASS_WITH_WARNINGS, 2 Low findings — both deferred per spec),
dependency-table status update (AZ-372 -> Done (In Testing)), task
archive (todo/ -> done/), and autodev state update.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 04:43:08 +03:00
Oleksandr Bezdieniezhnykh 89b4bfd245 [AZ-374] Refactor C21: named GoogleMapsTiles HttpClient in DI
- Register IHttpClientFactory named client "GoogleMapsTiles" inside
  AddTileDownloader() with User-Agent and 100s timeout (preserves
  HttpClient's implicit default).
- Resolve the same named client from all three CreateClient() call
  sites in GoogleMapsDownloaderV2 (session token, single-tile,
  batch-tile retry lambda) and drop the duplicated per-call
  UserAgent.ParseAdd setup.
- Expose USER_AGENT, the client name, and the timeout as internal
  consts on GoogleMapsDownloaderV2 so the extension and the
  downloader share one source of truth.
- Add AC test that builds the DI container, resolves the named
  client, and asserts both the User-Agent header and the timeout.
- Archive AZ-374 task file: todo/ -> done/.

175 unit + 5 smoke pass.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 04:11:57 +03:00
Oleksandr Bezdieniezhnykh 7c37636fdf [AZ-373] Refactor C20: drop MapsVersion from new writes (option a)
- Stop writing "downloaded_YYYY-MM-DD" into tiles.maps_version: new rows
  bind @MapsVersion to NULL via TileService.BuildTileEntity.
- Retain the tiles.maps_version column (coderule.mdc forbids unprompted
  column drops); pre-existing rows keep their values for forensics.
- Remove MapsVersion property from DownloadTileResponse (API wire shape)
  and TileMetadata (internal DTO); OpenAPI schema regenerates from the
  DTO via Swashbuckle.
- Add 3 AC tests in TileServiceTests covering the captured-entity write
  (AC-1) and the DTO/wire-shape removal (AC-2).
- Update integration-test local DTO + console output; refresh docs in
  common_dtos.md, services_tile_service.md, data_model.md.
- Archive AZ-373 task file: todo/ -> done/.

174 unit + 5 smoke pass.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 04:05:40 +03:00
Oleksandr Bezdieniezhnykh 1dcd089d39 [AZ-371] Refactor C18: magic numbers to ProcessingConfig/MapConfig
Promotes 8 operational levers into config keys with defaults that match
the prior source literals byte-for-byte:
  ProcessingConfig: RegionProcessingTimeoutSeconds (300),
  RouteProcessingPollIntervalSeconds (5),
  MaxRoutePointSpacingMeters (200), LatLonTolerance (0.0001).
  MapConfig: TileSizePixels (256), AllowedZoomLevels ([15..19]),
  RetryBaseDelaySeconds (1), RetryMaxDelaySeconds (30).

Sites updated: RegionService, RouteProcessingService,
RoutePointGraphBuilder, RouteValidator, RouteService 4-arg ctor,
RouteImageRenderer, GoogleMapsDownloaderV2, TileService. Closes LF-2 by
forwarding HttpContext.RequestAborted from GetTileByLatLon into the
downloader. appsettings.json gains the 8 new keys at default values.

Tests: 141 / 141 unit + 5 / 5 smoke green. New ConfigDefaultsTests pins
defaults to original literals; new TileService unit test asserts CT
identity from caller to downloader (AZ-371 AC-3).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 03:30:07 +03:00
Oleksandr Bezdieniezhnykh 581dff206e [AZ-357] Refactor C06: drop tile Version concept; cumulative review batches 7-9
ci/woodpecker/push/01-test Pipeline was successful
ci/woodpecker/push/02-build-push Pipeline was successful
AZ-357 — eliminate year-based tile cache expiry (LF-1):
- Migration 012: drop 5-col unique index, dedupe by (lat,lon,zoom,
  size) keeping max(updated_at), add new 4-col unique index, make
  version column nullable + drop default. Column itself preserved
  per coderule (column drops require explicit confirmation; tracked
  in AZ-373 / C20).
- TileEntity.Version, TileMetadata.Version, DownloadTileResponse.
  Version: int -> int? (HTTP shape preserved; field still in JSON).
- TileService.DownloadAndStoreTilesAsync: drop currentVersion year
  computation and the .Where(t => t.Version == currentVersion)
  cache filter. BuildTileEntity: drop year arg; write Version=null.
- TileRepository: ON CONFLICT now 4-col; lookup queries
  ORDER BY updated_at DESC instead of version DESC.
- Tests: replace inverted BT02b with positive AZ357_AC1
  (prior-year cached tile is reused). Add BuildTileEntity_
  DoesNotPopulateVersion_AZ357 to enforce the no-write contract.
- 69 unit + 5 smoke + 3 stub-contract integration tests pass.

Cumulative code review (batches 7-9, 7 tasks): VERDICT=PASS.
Report at _docs/03_implementation/reviews/batch_09_review.md.
Zero Critical/High/Medium/Low findings. Architecture baseline
remains clean.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 00:20:47 +03:00
Oleksandr Bezdieniezhnykh 8b0ddae075 [AZ-312] [AZ-313] [AZ-314] Split Services into per-component csprojs
Phase B of architecture coupling refactor (epic AZ-309). Replaces
the monolithic SatelliteProvider.Services with three per-component
csprojs to add a compiler-enforced module boundary (resolves F4):

- SatelliteProvider.Services.TileDownloader
- SatelliteProvider.Services.RegionProcessing
- SatelliteProvider.Services.RouteManagement

DI registrations relocated into per-component AddTileDownloader /
AddRegionProcessing / AddRouteManagement extension methods called
from Program.cs. RateLimitException moved to Common/Exceptions/ to
keep the three new csprojs as siblings (no Region->TileDownloader
ProjectReference). Dockerfiles and consumer csprojs (Api, Tests)
rewired to the new project paths. No DI lifetime or hosted-service
order changes.

Build: 0 warn, 0 err. Unit tests: 40/40. Smoke integration: green.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-10 07:15:44 +03:00