Files
Oleksandr Bezdieniezhnykh b15454b9a9 [AZ-777] Phase 1 hotfix (z/x/y) + Phase 2 Derkachi seed + ops
Phase 1 hotfix:
- C11 HttpTileDownloader adapted to satellite-provider v2.0.0
  z/x/y inventory contract (bulk POST keyed by slippy-map coords).
- Unit tests rewritten to exercise the new inventory schema.
- E2E smoke test updated to match the v2.0.0 wire.

Phase 2 (Derkachi seed + smoke-validated on Jetson):
- tests/fixtures/derkachi_c6/{README,bbox.yaml,seed_region.py}
  drives POST /api/satellite/region against satellite-provider
  with Google Maps as the imagery source. Smoke run produced
  4 regions, 175 tiles, inventory 32/32.
- scripts/mint_dev_jwt.py + run-tests-jetson.sh auto-mint and
  export SATELLITE_PROVIDER_API_KEY using JWT_SECRET / JWT_ISSUER
  / JWT_AUDIENCE env vars (no host port mappings; e2e-runner
  reaches SP via internal docker network only).

Spec amendment: AZ-777 todo spec updated to record the
Google Maps imagery source decision and STOP-gate state.

AZ-777 Phase 3+ work is superseded by Epic AZ-835 (see next
commit).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 17:39:21 +03:00

7.5 KiB
Raw Permalink Blame History

Derkachi reference C6 tile catalog — fixture seeding

AZ-777 Phase 2 deliverable. Seeds the parent-suite satellite-provider DB with the satellite tiles the C6 reference catalog needs for the Derkachi replay tests (test_ac3_within_100m_80pct_of_ticks on AC-4 + test_az699_real_flight_validation_emits_verdict_and_report on AC-5).

What this folder contains

File Purpose
bbox.yaml bbox + zoom levels + actual flight extent + imagery source metadata + license attribution + chunking strategy
seed_region.py Python script that submits POST /api/satellite/request to satellite-provider for each (zoom × chunk) and polls until completion
README.md this file

Prerequisites

  1. Running satellite-provider. Typically the Jetson e2e harness via docker-compose.test.jetson.yml (services satellite-provider + satellite-provider-postgres). Verify it's up and healthy:

    ssh jetson-e2e "docker ps --filter name=satellite --format 'table {{.Names}}\t{{.Status}}'"
    
  2. SATELLITE_PROVIDER_URL in .env.test (already covered by the AZ-777 Phase 1 wiring).

  3. SATELLITE_PROVIDER_API_KEY — a valid HS256 JWT signed with the same JWT_SECRET the satellite-provider validates against. Mint with:

    export SATELLITE_PROVIDER_API_KEY="$(python scripts/mint_dev_jwt.py)"
    
  4. Google Maps Platform API key on the satellite-provider side (env GOOGLE_MAPS_API_KEY / config MapConfig__ApiKey). The satellite-provider uses this to actually download the tiles from Google Maps. The seed script does NOT need this key — it only triggers the producer's async download pipeline.

Quick start

# from gps-denied-onboard repo root, with satellite-provider running on Jetson:
export SATELLITE_PROVIDER_API_KEY="$(python scripts/mint_dev_jwt.py)"
python tests/fixtures/derkachi_c6/seed_region.py

Expected runtime: ~5-15 minutes for the full spec bbox (8 region calls × ~30-60s each + inventory verification).

Flags

--bbox-config PATH                  override bbox.yaml location
--env-file PATH                     override .env.test fallback location
--output-summary PATH               write a JSON summary for downstream consumers
--dry-run                           validate config + plan without submitting
--right-sized-flight                use the actual ~1 km^2 flight extent (98% fewer tiles)
--skip-poll                         submit + return; don't wait for terminal status
--skip-inventory-verification       skip the final coverage check

bbox sizing — important

bbox.yaml ships TWO bboxes:

  • bbox: per AZ-777 spec — covers ~11.1 × 7.14 km (~80 km²) of the Derkachi village area. ~4570 tiles z15-z18 (~57 MB). Default seeding target.
  • actual_flight_extent: the real Derkachi flight footprint per data_imu.csv — only ~254 × 457m (~0.12 km²) centered at (50.082, 36.110). ~60 tiles z15-z18 (~1 MB) if seeded right-sized via --right-sized-flight.

The spec bbox is ~300× larger than the actual flight extent. The spec sizing is intentional generality — operators can fly any route within the box without re-seeding. The right-sized mode is appropriate when only the specific Derkachi clip needs coverage (e.g., CI test runs).

Imagery source — IMPORTANT licensing note

AZ-777 was originally specced with CARTO Voyager Basemap (CC-BY-3.0) as the upstream imagery source. The 2026-05-22 black-box probe of the running satellite-provider revealed the actual upstream is Google Maps satellite layer (mt0..mt3.google.com/vt/lyrs=s). The AZ-777 spec was amended to reflect this reality (see Risk 4 in _docs/02_tasks/todo/AZ-777_derkachi_c6_reference_fixture.md).

Operators MUST propagate the attribution string "Imagery © Google" to any end-user-visible context that incorporates tiles seeded by this script. Per bbox.yaml::license.

Dev/research use is approved. Production deploy requires either:

  1. Google Maps Platform licensing review for the offline-cache use case (the C6 reference dataset is a long-lived stored cache, which Google Maps ToS may restrict), OR
  2. A parent-suite ticket to add a true CC-BY satellite imagery provider to satellite-provider (candidates: Esri World Imagery, Mapbox satellite, Sentinel-2 via Copernicus). TBD; not in scope for AZ-777.

Re-seeding (after a satellite-provider DB wipe)

The script is idempotent and safe to re-run:

  • Each invocation generates fresh region UUIDs, so each run creates a new set of region records on the producer side.
  • The producer's tile-storage layer dedups via UPSERT on (zoom, x, y), so tiles already downloaded from Google Maps are NOT re-fetched — they're counted as tilesReused instead.
  • Re-runs are cheap (just the region-tracking overhead) when the DB is warm.

To verify the catalog is populated without re-running the full seed, query inventory directly:

# inside the satellite-provider docker network:
docker run --rm --network gps-denied-onboard_default curlimages/curl:8.10.1 \
  -sk -X POST -H "Authorization: Bearer $SATELLITE_PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tiles":[{"z":18,"x":157497,"y":89000}]}' \
  https://satellite-provider:8080/api/satellite/tiles/inventory

Cost expectations

Per the 2026-05-22 probe baseline (200m @ z18 = 9 tiles, ~13 KB/tile, ~5s end-to-end):

Mode Tile count DB size Wall time (cold) Wall time (warm DB)
Spec bbox (~80 km²) ~4570 ~57 MB ~5-15 min ~30s (reuse)
Right-sized (~1 km²) ~60 ~1 MB ~1-2 min ~10s (reuse)

Google Maps API cost per tile depends on the satellite-provider operator's Maps Platform pricing tier. The seed script does NOT bill — the producer's Google Maps account does.

Failure modes

Exit code Meaning Likely cause
71 config file missing / malformed bbox.yaml corrupted or wrong path
72 required env var missing SATELLITE_PROVIDER_URL or SATELLITE_PROVIDER_API_KEY not set
73 satellite-provider unreachable Service down, wrong URL, or TLS handshake failed (try SATELLITE_PROVIDER_TLS_INSECURE=1)
74 region request rejected HTTP 4xx (auth, validation) or 5xx (producer crash); see stderr for HTTP body
75 one or more regions failed Background processing failed — usually a Google Maps API quota / key issue on the producer side. Check docker logs gps-denied-e2e-satellite-provider
76 inventory verification mismatch < 95% of expected tiles present; re-run to retry, or investigate producer logs

Cross-references

  • AZ-777 spec: _docs/02_tasks/todo/AZ-777_derkachi_c6_reference_fixture.md
  • AZ-777 Phase 1 (the wiring that makes this script callable): completed cycle 3 batch 105
  • AZ-808 (parent-suite): strict validation for region-request endpoint — when this lands, malformed seed_region.py invocations will fail with RFC 7807 ValidationProblemDetails instead of silent zero-coercion; coordinate any consumer-side changes with that release
  • AZ-812 (parent-suite): rename RequestRegionRequest.{Latitude, Longitude}{Lat, Lon} for OSM consistency — when this lands, seed_region.py must be updated to send lat/lon instead of latitude/longitude
  • satellite-provider Region API contract: today informally documented in ../../../../satellite-provider/_docs/02_document/modules/common_dtos.md::RegionRequest + system-flows.md Flow F2; formal region-request.md contract will be published as part of AZ-808