Oleksandr Bezdieniezhnykh
080441db5d
Drains all three deferred perf-harness items in one batch: - PT-01..PT-06 now carry Authorization: Bearer minted via the canonical SatelliteProvider.TestSupport.JwtTokenFactory (AZ-491) — no third copy of JWT logic in the shell. - PT-07 implemented as cold + warm dual-pass distribution (N=20 each), reports p50/p95 for both passes and fails if warm p95 >= cold p95. - PT-08 implemented as 20-batch upload distribution with batch p95 gated at the AZ-488 2000 ms target; per-item gate cost reported as derived proxy (batch_p95 / batch_size). New SatelliteProvider.IntegrationTests/PerfBootstrap.cs adds two CLI short-circuit subcommands (--mint-only and --gen-uav-fixture <path>) invoked by the shell so the perf script never inlines the JWT or JPEG-fixture logic. The dispatch sits at the top of Program.cs Main and runs before any HTTP / DB / readiness setup. performance-tests.md PT-07 + PT-08 flip from Deferred to Implemented. traceability-matrix.md PT-07 + PT-08 rows move from recorded to covered (PT-08 partial due to per-item proxy — flagged Low in batch-4 review). _docs/_process_leftovers/2026-05-11_perf-pt07-harness.md deleted; the leftovers directory is now empty. Closes cycle-2 retro Action 2; LESSONS.md [process] rule about Deferred NFRs remains in force as a guardrail. Also includes the previously-uncommitted cumulative review report for cycle-3 batches 01-03 (generated at the end of batch 3 but not staged). Co-authored-by: Cursor <cursoragent@cursor.com>
4.7 KiB
Performance Test Scenarios
PT-01: Single Tile Download Latency
Trigger: GET /api/satellite/tiles/latlon (uncached tile) Load: 1 request Expected: Response within 30s (includes Google Maps round-trip) Pass criterion: Response time < 30000ms; HTTP 200
PT-02: Cached Tile Retrieval Latency
Trigger: GET /api/satellite/tiles/latlon (cached tile) Load: 1 request Expected: Response within 500ms (DB lookup + response) Pass criterion: Response time < 500ms; HTTP 200
PT-03: Region Processing Throughput (200m)
Trigger: POST /api/satellite/request with 200m region Load: 1 region Expected: Complete processing within 60s Pass criterion: status="completed" within 60s; tiles downloaded > 0
PT-04: Region Processing Throughput (500m with stitch)
Trigger: POST /api/satellite/request with 500m region + stitch Load: 1 region Expected: Complete processing within 120s (more tiles + stitching) Pass criterion: status="completed" within 120s; stitched image exists
PT-05: Concurrent Region Requests
Trigger: 5 simultaneous POST /api/satellite/request (different coordinates) Load: 5 concurrent requests Expected: All queued immediately; all complete within 5 minutes Pass criterion: All 5 regions reach status="completed"; queue does not reject
PT-06: Route Point Interpolation Speed
Trigger: POST /api/satellite/route with 20 points Load: 1 request Expected: Route created (with interpolation) within 5s Pass criterion: HTTP 200 response within 5000ms; totalPoints > 20
PT-07: GetTilesByRegionAsync Latency Post-AZ-484 (cold + warm distribution)
Status: Implemented (AZ-492). Runner scenario: scripts/run-performance-tests.sh § "PT-07".
Trigger: TileRepository.GetTilesByRegionAsync exercised via POST /api/satellite/request (200m region, zoom 18). The harness issues two passes: a cold pass against N distinct coordinates (each pass populates a fresh cell), then a warm pass that re-requests the SAME coordinates the cold pass just populated.
Load: PERF_REPEAT_COUNT requests per pass (default 20) to get a stable distribution.
Expected: Warm p95 < cold p95. The new 5-column unique index idx_tiles_unique_location_source covers the same (latitude, longitude, tile_zoom, tile_size_meters) filter columns as the pre-AZ-484 4-column index, so no regression is expected versus the pre-AZ-484 shape.
Pass criterion: warm p95 < cold p95. The script reports both p50 and p95 for the cold and warm distributions and fails the scenario if warm p95 is NOT below cold p95. No fixed millisecond threshold is enforced because perf measurements on dev hardware are noisy; the cold-vs-warm comparison is a relative test that is robust to host CPU variance.
Source: AZ-484 NFR (Performance) — _docs/02_tasks/done/AZ-484_multi_source_tile_storage.md § Non-Functional Requirements; harness landed in AZ-492.
Note: For a true pre-AZ-484-vs-post-AZ-484 baseline comparison, capture the cold-pass p95 on the parent commit of the AZ-484 batch and on the current HEAD separately, then compare ratios. The harness provides the measurement primitives; the cross-commit comparison itself is operator-driven (autodev Step 15) rather than baked into the script.
PT-08: UAV Tile Batch Upload Latency
Status: Implemented (AZ-492). Runner scenario: scripts/run-performance-tests.sh § "PT-08".
Trigger: POST /api/satellite/upload exercised via a 256×256 random-noise JPEG generated on-demand by SatelliteProvider.IntegrationTests --gen-uav-fixture (which calls the same JPEG-creation surface as UavUploadTests.CreateValidJpeg). Each batch carries PERF_UAV_BATCH_SIZE items (default 10) at distinct coordinates so the per-source unique index never collides across items.
Load: PERF_REPEAT_COUNT batches (default 20) to get a stable distribution.
Expected: Per-item quality-gate cost target < 50 ms (Rule 5 dominates — luminance variance after the 32×32 downsample). End-to-end p95 for a 10-item batch < 2 s on the dev hardware (8-core x86 baseline; revise on hardware change).
Pass criterion: p95(UploadUavTileBatch[10 items]) ≤ 2000ms. The harness reports batch_p50, batch_p95, and a per_item_proxy_p95 = batch_p95 / batch_size derived value plus accepted/rejected/failed counts. The 2000 ms threshold gates batch p95; per-item gate cost is a derived proxy (precise per-call UavTileQualityGate.Validate timing requires server-side instrumentation that is out of scope for AZ-492 — see _docs/06_metrics/perf_<date>.md for the recorded numbers and follow-up items).
Source: AZ-488 NFR (Performance) — _docs/02_tasks/done/AZ-488_uav_tile_upload.md § Non-Functional Requirements; harness landed in AZ-492.