azaion/satellite-provider

Fork 0

mirror of https://github.com/azaion/satellite-provider.git synced 2026-06-21 17:51:13 +00:00

Files

T

Oleksandr Bezdieniezhnykh 51b572108a

ci/woodpecker/push/01-test Pipeline was successful

Details

ci/woodpecker/push/02-build-push Pipeline was successful

Details

[AZ-484] Cycle 1 Steps 12-16: docs, security, perf, deploy report

Captures the post-implementation autodev gates for AZ-484 multi-source
tile storage:

- Step 12 (Test-Spec Sync): added 7 AC rows (AZ-484 AC-1..AC-7) and a
  PT-07 NFR row to traceability-matrix.md; added PT-07 scenario to
  performance-tests.md.
- Step 13 (Update Docs): refreshed data_model.md (tiles columns +
  indexes + selection rule + UPSERT contract + migrations 012/013),
  module-layout.md (Common/Enums section with L-001 guidance,
  DataAccess imports-from now lists 6 sites), 6 module / component
  docs to reflect the new repo signatures, source/captured_at fields,
  and Dapper enum bypass workaround. ripple_log_cycle1.md records
  zero out-of-scope ripple.
- Step 14 (Security Audit): PASS_WITH_WARNINGS - 0 Critical, 0 High,
  5 Medium, 5 Low. AZ-484 itself added zero new findings. Hardening
  items (Postgres default creds, .env in build context, GMaps key
  rotation, ASP.NET Core 8.0.21 -> 8.0.25, rate limiter) recorded
  for separate tickets.
- Step 15 (Performance Test): all PT-01..PT-07 scenarios Unverified
  (non-blocking); PT-07 baseline-comparison harness deferred to a
  leftover for next cycle.
- Step 16 (Deploy): cycle deploy report covering migration safety,
  rollback path, post-deploy verification, security caveats.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-05-11 10:03:05 +03:00

4.5 KiB

Raw Blame History

TileDownloader

1. High-Level Overview

Purpose: Acquires satellite imagery tiles from Google Maps, stores them on disk, and persists metadata to the database. Handles session tokens, concurrent downloads, retry logic, and tile deduplication.

Architectural Pattern: Service + Gateway (wraps external API with retry/throttling)

csproj: SatelliteProvider.Services.TileDownloader/SatelliteProvider.Services.TileDownloader.csproj (split out of the monolithic SatelliteProvider.Services project in epic AZ-309)

Upstream dependencies: Common (DTOs, Enums — TileSource + TileSourceConverter since AZ-484, GeoUtils, configs, RateLimitException), DataAccess (TileEntity, ITileRepository)

Downstream consumers: RegionProcessing and WebApi — both via ITileService from Common (no compile-time ProjectReference from any consumer to this project's concrete types).

2. Internal Interfaces

Class: GoogleMapsDownloaderV2

Method	Input	Output	Async	Error Types
`DownloadSingleTileAsync`	lat, lon, zoomLevel, CancellationToken	`DownloadedTileInfoV2`	Yes	ArgumentException, RateLimitException, HttpRequestException
`GetTilesWithMetadataAsync`	center, radiusM, zoom, existingTiles, CancellationToken	`List<DownloadedTileInfoV2>`	Yes	ArgumentException, RateLimitException, HttpRequestException

Service: TileService (implements ITileService)

Method	Input	Output	Async	Error Types
`DownloadAndStoreTilesAsync`	lat, lon, sizeM, zoom, CancellationToken	`List<TileMetadata>`	Yes	propagated from downloader
`GetTileAsync`	Guid	`TileMetadata?`	Yes	NpgsqlException
`GetTilesByRegionAsync`	lat, lon, sizeM, zoom	`IEnumerable<TileMetadata>`	Yes	NpgsqlException
`GetOrDownloadTileAsync` (AZ-310)	z, x, y, CancellationToken	`TileBytes`	Yes	propagated from downloader
`DownloadAndStoreSingleTileAsync` (AZ-311)	lat, lon, zoom, CancellationToken	`TileMetadata`	Yes	propagated from downloader

4. Data Access Patterns

Caching Strategy

Data	Cache Type	TTL	Invalidation
Tile bytes	In-memory (IMemoryCache, owned by TileService since AZ-310)	1h absolute, 30min sliding	None (manual restart)
Tile metadata	Database	Append-by-source per cell (AZ-484); reads return most-recent across sources	Per-source UPSERT keyed on `(latitude, longitude, tile_zoom, tile_size_meters, source)` overwrites the existing same-source row and refreshes `captured_at`
Active downloads	ConcurrentDictionary	Duration of download	Removed on completion

5. Implementation Details

Algorithmic Complexity: Tile grid calculation is O(w×h) where w×h is the number of tiles covering the bounding box.

State Management: _activeDownloads (ConcurrentDictionary) prevents duplicate concurrent downloads. _downloadSemaphore limits parallelism.

Key Dependencies:

Library	Version	Purpose
Newtonsoft.Json	13.0.4	Serialize session creation request body
IHttpClientFactory	built-in	Create HttpClient instances per request

Error Handling:

Exponential backoff retry for 429 (rate limit) and 5xx errors: 1s → 2s → 4s → 8s → 16s, max 30s, 5 retries
Immediate throw for 401/403 (auth errors) and cancellation
RateLimitException thrown after exhausting retries on 429

7. Caveats & Edge Cases

GoogleMapsDownloaderV2 is registered behind ISatelliteDownloader (resolved by AZ-310 cleanup); the previous concrete-type coupling is gone.
User-Agent header spoofs Chrome — could be rejected if Google changes detection
Allowed zoom levels hardcoded to [15,16,17,18,19] — throws for others
Session token rotation threshold (100 tiles) is an educated guess; Google's actual limit is not documented
Static _activeDownloads dictionary means deduplication is process-wide, surviving service scope boundaries

8. Dependency Graph

Must be implemented after: Common, DataAccess Can be implemented in parallel with: nothing (needs both foundations) Blocks: RegionProcessing

9. Logging Strategy

Log Level	When	Example
ERROR	Download failure, session token failure	`Tile download failed. Tile: (X, Y), Status: {StatusCode}`
WARN	Rate limiting retry	`Rate limited (429). Waiting {Delay}s before retry`
INFO	—	(no INFO-level logs in this component)

4.5 KiB Raw Blame History Unescape Escape