mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-21 14:21:14 +00:00
Oleksandr Bezdieniezhnykh
5d84d2839e
Step 12 (Test-Spec Sync, cycle-update mode): - blackbox-tests.md: append BT-23..BT-26 for AZ-505's new observable behaviors (inventory order/shape; leaflet most-recent via location_hash; HTTP/2 multiplex over TLS+ALPN; request validation). - performance-tests.md: append PT-09 (inventory p95 ≤ 1000ms / 2500 tiles); records cycle-6 measured p95=66ms; documents promotion path to scripts/run-performance-tests.sh if budget ever tightens. - traceability-matrix.md: resolve the 5 AZ-503 deferrals (AC-5/6/9/10/12) by pointing at AZ-505 test names + add 7 AZ-505 AC rows (AC-1..AC-7) + bump totals (90 -> 94 tests, 56/56 -> 63/63 in-scope) + add cycle-6 coverage shape notes (budget relaxation rationale, voting-filter deferral note, TLS+ALPN pivot, NFR propagation). Step 13 (Update Docs, task mode): - common_dtos.md: add 5 new TileInventory DTOs. - common_interfaces.md: add ITileService.GetInventoryAsync. - services_tile_service.md: document TileService.GetInventoryAsync steps + the XOR-validation-in-handler note. - dataaccess_migrator.md: bump migration count 14 -> 15; describe migration 015 (AZ-505 leaflet covering index, lock window, INCLUDE-list trade-off). - system-flows.md: add F7 (Leaflet Tile Serving, AZ-310 + AZ-505 location_hash rewire + TLS+ALPN) and F8 (Tile Inventory Bulk Lookup) with sequence diagrams, validation surface, and AC-4 perf evidence. Update Flow Inventory + Dependencies tables accordingly. - glossary.md: add "Tile Inventory" entry pointing at the v1.0.0 contract. - ripple_log_cycle6.md: new file — exhaustive reverse-dependency analysis confirms zero stale downstream module docs. Advance autodev state from step 11 -> 14 (skipping 12+13 as completed in this commit; auto-chain through Step 14 = Security Audit optional gate). Co-authored-by: Cursor <cursoragent@cursor.com>
76 lines
6.9 KiB
Markdown
76 lines
6.9 KiB
Markdown
# Performance Test Scenarios
|
||
|
||
## PT-01: Single Tile Download Latency
|
||
|
||
**Trigger**: GET /api/satellite/tiles/latlon (uncached tile)
|
||
**Load**: 1 request
|
||
**Expected**: Response within 30s (includes Google Maps round-trip)
|
||
**Pass criterion**: Response time < 30000ms; HTTP 200
|
||
|
||
## PT-02: Cached Tile Retrieval Latency
|
||
|
||
**Trigger**: GET /api/satellite/tiles/latlon (cached tile)
|
||
**Load**: 1 request
|
||
**Expected**: Response within 500ms (DB lookup + response)
|
||
**Pass criterion**: Response time < 500ms; HTTP 200
|
||
|
||
## PT-03: Region Processing Throughput (200m)
|
||
|
||
**Trigger**: POST /api/satellite/request with 200m region
|
||
**Load**: 1 region
|
||
**Expected**: Complete processing within 60s
|
||
**Pass criterion**: status="completed" within 60s; tiles downloaded > 0
|
||
|
||
## PT-04: Region Processing Throughput (500m with stitch)
|
||
|
||
**Trigger**: POST /api/satellite/request with 500m region + stitch
|
||
**Load**: 1 region
|
||
**Expected**: Complete processing within 120s (more tiles + stitching)
|
||
**Pass criterion**: status="completed" within 120s; stitched image exists
|
||
|
||
## PT-05: Concurrent Region Requests
|
||
|
||
**Trigger**: 5 simultaneous POST /api/satellite/request (different coordinates)
|
||
**Load**: 5 concurrent requests
|
||
**Expected**: All queued immediately; all complete within 5 minutes
|
||
**Pass criterion**: All 5 regions reach status="completed"; queue does not reject
|
||
|
||
## PT-06: Route Point Interpolation Speed
|
||
|
||
**Trigger**: POST /api/satellite/route with 20 points
|
||
**Load**: 1 request
|
||
**Expected**: Route created (with interpolation) within 5s
|
||
**Pass criterion**: HTTP 200 response within 5000ms; totalPoints > 20
|
||
|
||
## PT-07: GetTilesByRegionAsync Latency Post-AZ-484 (cold + warm distribution)
|
||
|
||
**Status**: **Implemented (AZ-492).** Runner scenario: `scripts/run-performance-tests.sh` § "PT-07".
|
||
|
||
**Trigger**: TileRepository.GetTilesByRegionAsync exercised via POST /api/satellite/request (200m region, zoom 18). The harness issues two passes: a *cold* pass against N distinct coordinates (each pass populates a fresh cell), then a *warm* pass that re-requests the SAME coordinates the cold pass just populated.
|
||
**Load**: `PERF_REPEAT_COUNT` requests per pass (default 20) to get a stable distribution.
|
||
**Expected**: Warm p95 < cold p95. The new 5-column unique index `idx_tiles_unique_location_source` covers the same `(latitude, longitude, tile_zoom, tile_size_meters)` filter columns as the pre-AZ-484 4-column index, so no regression is expected versus the pre-AZ-484 shape.
|
||
**Pass criterion**: warm p95 < cold p95. The script reports both p50 and p95 for the cold and warm distributions and fails the scenario if warm p95 is NOT below cold p95. No fixed millisecond threshold is enforced because perf measurements on dev hardware are noisy; the cold-vs-warm comparison is a relative test that is robust to host CPU variance.
|
||
**Source**: AZ-484 NFR (Performance) — `_docs/02_tasks/done/AZ-484_multi_source_tile_storage.md` § Non-Functional Requirements; harness landed in AZ-492.
|
||
**Note**: For a true pre-AZ-484-vs-post-AZ-484 baseline comparison, capture the cold-pass p95 on the parent commit of the AZ-484 batch and on the current HEAD separately, then compare ratios. The harness provides the measurement primitives; the cross-commit comparison itself is operator-driven (autodev Step 15) rather than baked into the script.
|
||
|
||
## PT-08: UAV Tile Batch Upload Latency
|
||
|
||
**Status**: **Implemented (AZ-492).** Runner scenario: `scripts/run-performance-tests.sh` § "PT-08".
|
||
|
||
**Trigger**: `POST /api/satellite/upload` exercised via a 256×256 random-noise JPEG generated on-demand by `SatelliteProvider.IntegrationTests --gen-uav-fixture` (which calls the same JPEG-creation surface as `UavUploadTests.CreateValidJpeg`). Each batch carries `PERF_UAV_BATCH_SIZE` items (default 10) at distinct coordinates so the per-source unique index never collides across items.
|
||
**Load**: `PERF_REPEAT_COUNT` batches (default 20) to get a stable distribution.
|
||
**Expected**: Per-item quality-gate cost target < 50 ms (Rule 5 dominates — luminance variance after the 32×32 downsample). End-to-end p95 for a 10-item batch < 2 s on the dev hardware (8-core x86 baseline; revise on hardware change).
|
||
**Pass criterion**: `p95(UploadUavTileBatch[10 items]) ≤ 2000ms`. The harness reports `batch_p50`, `batch_p95`, and a `per_item_proxy_p95 = batch_p95 / batch_size` derived value plus accepted/rejected/failed counts. The 2000 ms threshold gates batch p95; per-item gate cost is a derived proxy (precise per-call `UavTileQualityGate.Validate` timing requires server-side instrumentation that is out of scope for AZ-492 — see `_docs/06_metrics/perf_<date>.md` for the recorded numbers and follow-up items).
|
||
**Source**: AZ-488 NFR (Performance) — `_docs/02_tasks/done/AZ-488_uav_tile_upload.md` § Non-Functional Requirements; harness landed in AZ-492.
|
||
|
||
## PT-09: Inventory Endpoint Throughput (2500-Tile Batch)
|
||
|
||
**Status**: **Implemented (AZ-505).** Embedded in the integration test `TileInventoryTests.PerformanceBudget_AC4` (full-suite only; smoke run prints a documented skip). Not yet promoted to `scripts/run-performance-tests.sh` because the AZ-505 gate is met inline; promotion is a candidate follow-up if the budget tightens (see Note below).
|
||
|
||
**Trigger**: `POST /api/satellite/tiles/inventory` with a 2500-entry `tiles` body at zoom 18 against a populated DB. The test seeds 2500 `(z, x, y)` cells (one `google_maps` row each) inside a single transaction, runs `VACUUM ANALYZE tiles`, then issues 20 identical inventory requests serially and records per-call wall-clock latency.
|
||
**Load**: 20 calls × 2500-tile batches (single client, sequential — the bottleneck under test is server-side query planning + array binding + per-row hash lookup, not network concurrency).
|
||
**Expected**: p95 over the 20 measured calls ≤ 1000 ms. The plan is expected to consume the `tiles_leaflet_path` covering index on the leading `location_hash` column (with `Index Only Scan` for cells where the visibility map is complete, falling back to a bounded heap fetch otherwise — `tile-inventory.md` v1.0.0 documents this trade-off explicitly).
|
||
**Pass criterion**: `p95(durations[20]) ≤ 1000ms` (samples sorted ascending; p95 = `sorted[18]` over 20 samples per the test). Cycle 6 measured: `min=13ms, median=19ms, p95=66ms, max=117ms` — well under budget.
|
||
**Source**: AZ-505 AC-4 — `_docs/02_tasks/done/AZ-505_tile_inventory_http2_leaflet_index.md` § Acceptance Criteria. Resolves the AZ-503 AC-9 perf NFR deferral (budget relaxed from 500 ms to 1000 ms during AZ-505 scoping — see AZ-505 Risk 1 in the task spec).
|
||
**Note (promotion to perf harness)**: The in-test gate runs against the same Docker compose stack as the rest of the integration suite, so the perf budget is verified on the same hardware as functional tests. If we ever need to tighten the budget (e.g., to 500 ms for production-equivalent hardware) or add cross-commit baseline comparison like PT-07, promote this to `scripts/run-performance-tests.sh § PT-09` with a `PERF_INVENTORY_BATCH_SIZE` env variable controlling the row count and a separate cold/warm distinction.
|