satellite-provider/_docs/02_document/tests/performance-tests.md

Performance Test Scenarios

PT-01: Single Tile Download Latency

Trigger: GET /api/satellite/tiles/latlon (uncached tile) Load: 1 request Expected: Response within 30s (includes Google Maps round-trip) Pass criterion: Response time < 30000ms; HTTP 200

PT-02: Cached Tile Retrieval Latency

Trigger: GET /api/satellite/tiles/latlon (cached tile) Load: 1 request Expected: Response within 500ms (DB lookup + response) Pass criterion: Response time < 500ms; HTTP 200

PT-03: Region Processing Throughput (200m)

Trigger: POST /api/satellite/request with 200m region Load: 1 region Expected: Complete processing within 60s Pass criterion: status="completed" within 60s; tiles downloaded > 0

PT-04: Region Processing Throughput (500m with stitch)

Trigger: POST /api/satellite/request with 500m region + stitch Load: 1 region Expected: Complete processing within 120s (more tiles + stitching) Pass criterion: status="completed" within 120s; stitched image exists

PT-05: Concurrent Region Requests

Trigger: 5 simultaneous POST /api/satellite/request (different coordinates) Load: 5 concurrent requests Expected: All queued immediately; all complete within 5 minutes Pass criterion: All 5 regions reach status="completed"; queue does not reject

PT-06: Route Point Interpolation Speed

Trigger: POST /api/satellite/route with 20 points Load: 1 request Expected: Route created (with interpolation) within 5s Pass criterion: HTTP 200 response within 5000ms; totalPoints > 20

PT-07: GetTilesByRegionAsync Latency Post-AZ-484 (cold + warm distribution)

Status: Implemented (AZ-492). Runner scenario: scripts/run-performance-tests.sh § "PT-07".

Trigger: TileRepository.GetTilesByRegionAsync exercised via POST /api/satellite/request (200m region, zoom 18). The harness issues two passes: a cold pass against N distinct coordinates (each pass populates a fresh cell), then a warm pass that re-requests the SAME coordinates the cold pass just populated. Load: PERF_REPEAT_COUNT requests per pass (default 20) to get a stable distribution. Expected: Warm p95 < cold p95. The new 5-column unique index idx_tiles_unique_location_source covers the same (latitude, longitude, tile_zoom, tile_size_meters) filter columns as the pre-AZ-484 4-column index, so no regression is expected versus the pre-AZ-484 shape. Pass criterion: Warm faster than cold on either p95 or p50 (both reported). AZ-492 AC-2 requires measurable warm < cold without a fixed millisecond threshold; at N=20, p95 is sensitive to single outliers — p50 is the tie-breaker when p95 inverts by noise (<3% on a warm cache). Source: AZ-484 NFR (Performance) — _docs/02_tasks/done/AZ-484_multi_source_tile_storage.md § Non-Functional Requirements; harness landed in AZ-492. Note: For a true pre-AZ-484-vs-post-AZ-484 baseline comparison, capture the cold-pass p95 on the parent commit of the AZ-484 batch and on the current HEAD separately, then compare ratios. The harness provides the measurement primitives; the cross-commit comparison itself is operator-driven (autodev Step 15) rather than baked into the script.

PT-08: UAV Tile Batch Upload Latency

Status: Implemented (AZ-492). Runner scenario: scripts/run-performance-tests.sh § "PT-08".

Trigger: POST /api/satellite/upload exercised via a 256×256 random-noise JPEG generated on-demand by SatelliteProvider.IntegrationTests --gen-uav-fixture (which calls the same JPEG-creation surface as UavUploadTests.CreateValidJpeg). Each batch carries PERF_UAV_BATCH_SIZE items (default 10) at distinct coordinates so the per-source unique index never collides across items. Load: PERF_REPEAT_COUNT batches (default 20) to get a stable distribution. Expected: Per-item quality-gate cost target < 50 ms (Rule 5 dominates — luminance variance after the 32×32 downsample). End-to-end p95 for a 10-item batch < 2 s on the dev hardware (8-core x86 baseline; revise on hardware change). Pass criterion: p95(UploadUavTileBatch[10 items]) ≤ 2000ms. The harness reports batch_p50, batch_p95, and a per_item_proxy_p95 = batch_p95 / batch_size derived value plus accepted/rejected/failed counts. The 2000 ms threshold gates batch p95; per-item gate cost is a derived proxy (precise per-call UavTileQualityGate.Validate timing requires server-side instrumentation that is out of scope for AZ-492 — see _docs/06_metrics/perf_<date>.md for the recorded numbers and follow-up items). Source: AZ-488 NFR (Performance) — _docs/02_tasks/done/AZ-488_uav_tile_upload.md § Non-Functional Requirements; harness landed in AZ-492.

PT-09: Inventory Endpoint Throughput (2500-Tile Batch)

Status: Implemented (AZ-505). Embedded in the integration test TileInventoryTests.PerformanceBudget_AC4 (full-suite only; smoke run prints a documented skip). Not yet promoted to scripts/run-performance-tests.sh because the AZ-505 gate is met inline; promotion is a candidate follow-up if the budget tightens (see Note below).

Trigger: POST /api/satellite/tiles/inventory with a 2500-entry tiles body at zoom 18 against a populated DB. The test seeds 2500 (z, x, y) cells (one google_maps row each) inside a single transaction, runs VACUUM ANALYZE tiles, then issues 20 identical inventory requests serially and records per-call wall-clock latency. Load: 20 calls × 2500-tile batches (single client, sequential — the bottleneck under test is server-side query planning + array binding + per-row hash lookup, not network concurrency). Expected: p95 over the 20 measured calls ≤ 1000 ms. The plan is expected to consume the tiles_leaflet_path covering index on the leading location_hash column (with Index Only Scan for cells where the visibility map is complete, falling back to a bounded heap fetch otherwise — tile-inventory.md v1.0.0 documents this trade-off explicitly). Pass criterion: p95(durations[20]) ≤ 1000ms (samples sorted ascending; p95 = sorted[18] over 20 samples per the test). Cycle 6 measured: min=13ms, median=19ms, p95=66ms, max=117ms — well under budget. Source: AZ-505 AC-4 — _docs/02_tasks/done/AZ-505_tile_inventory_http2_leaflet_index.md § Acceptance Criteria. Resolves the AZ-503 AC-9 perf NFR deferral (budget relaxed from 500 ms to 1000 ms during AZ-505 scoping — see AZ-505 Risk 1 in the task spec). Note (promotion to perf harness): The in-test gate runs against the same Docker compose stack as the rest of the integration suite, so the perf budget is verified on the same hardware as functional tests. If we ever need to tighten the budget (e.g., to 500 ms for production-equivalent hardware) or add cross-commit baseline comparison like PT-07, promote this to scripts/run-performance-tests.sh § PT-09 with a PERF_INVENTORY_BATCH_SIZE env variable controlling the row count and a separate cold/warm distinction.

PT-10: gRPC DeliverRouteTiles Stream Latency

Status: Implemented (AZ-1124). Runner scenario: scripts/run-performance-tests.sh § "PT-10".

Trigger: RouteTileDelivery.DeliverRouteTiles server-streaming RPC over gRPC (TLS + JWT metadata) against a live API. Invoked via SatelliteProvider.IntegrationTests --run-pt10, which reuses GrpcTestHelpers, PerfBootstrap JWT mint, and the standard 2-waypoint / 500 m / zoom-18 fixture. Load: PERF_REPEAT_COUNT sequential cold iterations (default 20) plus one slow-consumer iteration with PERF_PT10_SLOW_MS delay between stream events (default 50 ms). Expected: First Batch event arrives within the cold-path GM budget; full stream completes with DeliveryComplete; slow consumer completes without DeliveryError. Pass criterion: p95(first_batch_ms) ≤ 30000, p95(total_stream_ms) ≤ 120000, and slow-consumer sub-check passes (no fixed ms threshold). Threshold failures exit non-zero; autodev Step 15 may offer override. Source: Cycle 9–11 retro carry-over — gRPC functional coverage (AZ-1074/1075 / BT-32) without streaming NFR evidence; closes the Unverified gRPC perf gap. Note: Cold iterations may include Google Maps download on the first pass over a fresh compose volume; record actuals in _docs/06_metrics/perf_<date>.md. Host-side runs require API_URL=https://localhost:18980 (or perf overlay port) and trust ./certs/api.crt via PROJECT_ROOT (same as REST perf probes).