mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-21 21:11:13 +00:00
Oleksandr Bezdieniezhnykh
080441db5d
Drains all three deferred perf-harness items in one batch: - PT-01..PT-06 now carry Authorization: Bearer minted via the canonical SatelliteProvider.TestSupport.JwtTokenFactory (AZ-491) — no third copy of JWT logic in the shell. - PT-07 implemented as cold + warm dual-pass distribution (N=20 each), reports p50/p95 for both passes and fails if warm p95 >= cold p95. - PT-08 implemented as 20-batch upload distribution with batch p95 gated at the AZ-488 2000 ms target; per-item gate cost reported as derived proxy (batch_p95 / batch_size). New SatelliteProvider.IntegrationTests/PerfBootstrap.cs adds two CLI short-circuit subcommands (--mint-only and --gen-uav-fixture <path>) invoked by the shell so the perf script never inlines the JWT or JPEG-fixture logic. The dispatch sits at the top of Program.cs Main and runs before any HTTP / DB / readiness setup. performance-tests.md PT-07 + PT-08 flip from Deferred to Implemented. traceability-matrix.md PT-07 + PT-08 rows move from recorded to covered (PT-08 partial due to per-item proxy — flagged Low in batch-4 review). _docs/_process_leftovers/2026-05-11_perf-pt07-harness.md deleted; the leftovers directory is now empty. Closes cycle-2 retro Action 2; LESSONS.md [process] rule about Deferred NFRs remains in force as a guardrail. Also includes the previously-uncommitted cumulative review report for cycle-3 batches 01-03 (generated at the end of batch 3 but not staged). Co-authored-by: Cursor <cursoragent@cursor.com>
65 lines
4.7 KiB
Markdown
65 lines
4.7 KiB
Markdown
# Performance Test Scenarios
|
||
|
||
## PT-01: Single Tile Download Latency
|
||
|
||
**Trigger**: GET /api/satellite/tiles/latlon (uncached tile)
|
||
**Load**: 1 request
|
||
**Expected**: Response within 30s (includes Google Maps round-trip)
|
||
**Pass criterion**: Response time < 30000ms; HTTP 200
|
||
|
||
## PT-02: Cached Tile Retrieval Latency
|
||
|
||
**Trigger**: GET /api/satellite/tiles/latlon (cached tile)
|
||
**Load**: 1 request
|
||
**Expected**: Response within 500ms (DB lookup + response)
|
||
**Pass criterion**: Response time < 500ms; HTTP 200
|
||
|
||
## PT-03: Region Processing Throughput (200m)
|
||
|
||
**Trigger**: POST /api/satellite/request with 200m region
|
||
**Load**: 1 region
|
||
**Expected**: Complete processing within 60s
|
||
**Pass criterion**: status="completed" within 60s; tiles downloaded > 0
|
||
|
||
## PT-04: Region Processing Throughput (500m with stitch)
|
||
|
||
**Trigger**: POST /api/satellite/request with 500m region + stitch
|
||
**Load**: 1 region
|
||
**Expected**: Complete processing within 120s (more tiles + stitching)
|
||
**Pass criterion**: status="completed" within 120s; stitched image exists
|
||
|
||
## PT-05: Concurrent Region Requests
|
||
|
||
**Trigger**: 5 simultaneous POST /api/satellite/request (different coordinates)
|
||
**Load**: 5 concurrent requests
|
||
**Expected**: All queued immediately; all complete within 5 minutes
|
||
**Pass criterion**: All 5 regions reach status="completed"; queue does not reject
|
||
|
||
## PT-06: Route Point Interpolation Speed
|
||
|
||
**Trigger**: POST /api/satellite/route with 20 points
|
||
**Load**: 1 request
|
||
**Expected**: Route created (with interpolation) within 5s
|
||
**Pass criterion**: HTTP 200 response within 5000ms; totalPoints > 20
|
||
|
||
## PT-07: GetTilesByRegionAsync Latency Post-AZ-484 (cold + warm distribution)
|
||
|
||
**Status**: **Implemented (AZ-492).** Runner scenario: `scripts/run-performance-tests.sh` § "PT-07".
|
||
|
||
**Trigger**: TileRepository.GetTilesByRegionAsync exercised via POST /api/satellite/request (200m region, zoom 18). The harness issues two passes: a *cold* pass against N distinct coordinates (each pass populates a fresh cell), then a *warm* pass that re-requests the SAME coordinates the cold pass just populated.
|
||
**Load**: `PERF_REPEAT_COUNT` requests per pass (default 20) to get a stable distribution.
|
||
**Expected**: Warm p95 < cold p95. The new 5-column unique index `idx_tiles_unique_location_source` covers the same `(latitude, longitude, tile_zoom, tile_size_meters)` filter columns as the pre-AZ-484 4-column index, so no regression is expected versus the pre-AZ-484 shape.
|
||
**Pass criterion**: warm p95 < cold p95. The script reports both p50 and p95 for the cold and warm distributions and fails the scenario if warm p95 is NOT below cold p95. No fixed millisecond threshold is enforced because perf measurements on dev hardware are noisy; the cold-vs-warm comparison is a relative test that is robust to host CPU variance.
|
||
**Source**: AZ-484 NFR (Performance) — `_docs/02_tasks/done/AZ-484_multi_source_tile_storage.md` § Non-Functional Requirements; harness landed in AZ-492.
|
||
**Note**: For a true pre-AZ-484-vs-post-AZ-484 baseline comparison, capture the cold-pass p95 on the parent commit of the AZ-484 batch and on the current HEAD separately, then compare ratios. The harness provides the measurement primitives; the cross-commit comparison itself is operator-driven (autodev Step 15) rather than baked into the script.
|
||
|
||
## PT-08: UAV Tile Batch Upload Latency
|
||
|
||
**Status**: **Implemented (AZ-492).** Runner scenario: `scripts/run-performance-tests.sh` § "PT-08".
|
||
|
||
**Trigger**: `POST /api/satellite/upload` exercised via a 256×256 random-noise JPEG generated on-demand by `SatelliteProvider.IntegrationTests --gen-uav-fixture` (which calls the same JPEG-creation surface as `UavUploadTests.CreateValidJpeg`). Each batch carries `PERF_UAV_BATCH_SIZE` items (default 10) at distinct coordinates so the per-source unique index never collides across items.
|
||
**Load**: `PERF_REPEAT_COUNT` batches (default 20) to get a stable distribution.
|
||
**Expected**: Per-item quality-gate cost target < 50 ms (Rule 5 dominates — luminance variance after the 32×32 downsample). End-to-end p95 for a 10-item batch < 2 s on the dev hardware (8-core x86 baseline; revise on hardware change).
|
||
**Pass criterion**: `p95(UploadUavTileBatch[10 items]) ≤ 2000ms`. The harness reports `batch_p50`, `batch_p95`, and a `per_item_proxy_p95 = batch_p95 / batch_size` derived value plus accepted/rejected/failed counts. The 2000 ms threshold gates batch p95; per-item gate cost is a derived proxy (precise per-call `UavTileQualityGate.Validate` timing requires server-side instrumentation that is out of scope for AZ-492 — see `_docs/06_metrics/perf_<date>.md` for the recorded numbers and follow-up items).
|
||
**Source**: AZ-488 NFR (Performance) — `_docs/02_tasks/done/AZ-488_uav_tile_upload.md` § Non-Functional Requirements; harness landed in AZ-492.
|