mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-27 08:31:13 +00:00
[AZ-1124] Add PT-10 gRPC stream perf scenario
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -73,3 +73,14 @@
|
||||
**Pass criterion**: `p95(durations[20]) ≤ 1000ms` (samples sorted ascending; p95 = `sorted[18]` over 20 samples per the test). Cycle 6 measured: `min=13ms, median=19ms, p95=66ms, max=117ms` — well under budget.
|
||||
**Source**: AZ-505 AC-4 — `_docs/02_tasks/done/AZ-505_tile_inventory_http2_leaflet_index.md` § Acceptance Criteria. Resolves the AZ-503 AC-9 perf NFR deferral (budget relaxed from 500 ms to 1000 ms during AZ-505 scoping — see AZ-505 Risk 1 in the task spec).
|
||||
**Note (promotion to perf harness)**: The in-test gate runs against the same Docker compose stack as the rest of the integration suite, so the perf budget is verified on the same hardware as functional tests. If we ever need to tighten the budget (e.g., to 500 ms for production-equivalent hardware) or add cross-commit baseline comparison like PT-07, promote this to `scripts/run-performance-tests.sh § PT-09` with a `PERF_INVENTORY_BATCH_SIZE` env variable controlling the row count and a separate cold/warm distinction.
|
||||
|
||||
## PT-10: gRPC DeliverRouteTiles Stream Latency
|
||||
|
||||
**Status**: **Implemented (AZ-1124).** Runner scenario: `scripts/run-performance-tests.sh` § "PT-10".
|
||||
|
||||
**Trigger**: `RouteTileDelivery.DeliverRouteTiles` server-streaming RPC over gRPC (TLS + JWT metadata) against a live API. Invoked via `SatelliteProvider.IntegrationTests --run-pt10`, which reuses `GrpcTestHelpers`, `PerfBootstrap` JWT mint, and the standard 2-waypoint / 500 m / zoom-18 fixture.
|
||||
**Load**: `PERF_REPEAT_COUNT` sequential cold iterations (default 20) plus one slow-consumer iteration with `PERF_PT10_SLOW_MS` delay between stream events (default 50 ms).
|
||||
**Expected**: First `Batch` event arrives within the cold-path GM budget; full stream completes with `DeliveryComplete`; slow consumer completes without `DeliveryError`.
|
||||
**Pass criterion**: `p95(first_batch_ms) ≤ 30000`, `p95(total_stream_ms) ≤ 120000`, and slow-consumer sub-check passes (no fixed ms threshold). Threshold failures exit non-zero; autodev Step 15 may offer override.
|
||||
**Source**: Cycle 9–11 retro carry-over — gRPC functional coverage (AZ-1074/1075 / BT-32) without streaming NFR evidence; closes the Unverified gRPC perf gap.
|
||||
**Note**: Cold iterations may include Google Maps download on the first pass over a fresh compose volume; record actuals in `_docs/06_metrics/perf_<date>.md`. Host-side runs require `API_URL=https://localhost:18980` (or perf overlay port) and trust `./certs/api.crt` via `PROJECT_ROOT` (same as REST perf probes).
|
||||
|
||||
@@ -188,6 +188,7 @@
|
||||
| AZ-487 Security — `RequireSignedTokens`, `RequireExpirationTime`, `ClockSkew = 30 s`, secret ≥ 32 bytes, `iss` + `aud` validated (extended by AZ-494) | AZ-487 + AZ-494 task specs § Non-Functional Requirements + Constraints | `AuthenticationServiceCollectionExtensionsTests` (unit) + SEC-05..SEC-09 + AZ-494 AC-1/AC-2 wrong-iss/aud (integration) | ✓ |
|
||||
| AZ-487 Reliability — Fail-fast on missing / short `JWT_SECRET` at startup (extended by AZ-494 to iss + aud) | AZ-487 + AZ-494 task specs § Non-Functional Requirements | SEC-08 (behavioral) + unit `AddSatelliteJwt_ThrowsOnMissingSecret` + `_ThrowsOnMissingIssuer` + `_ThrowsOnMissingAudience` | ✓ |
|
||||
| AZ-488 Performance — Per-item gate cost < 50 ms; p95 batch-of-10 < 2 s | AZ-488 task spec § Non-Functional Requirements | PT-08 (Implemented in AZ-492 — 20-batch distribution, batch p95 gated at 2000 ms; per-item gate cost reported as derived proxy `batch_p95 / batch_size`. True per-call `UavTileQualityGate.Validate` timing requires server-side instrumentation — follow-up). | ✓ (batch p95) / ◐ (per-item proxy only) |
|
||||
| Cycle 9 — gRPC `DeliverRouteTiles` stream latency / backpressure (NFR) | Cycle 9 retro Action #2; AZ-1124 task spec § Non-Functional Requirements | PT-10 (Implemented in AZ-1124 — `scripts/run-performance-tests.sh` § PT-10 + `SatelliteProvider.IntegrationTests --run-pt10`; first_batch + total_stream p50/p95 + slow-consumer smoke) | ✓ |
|
||||
| AZ-488 Reliability — File-first then DB row; per-item failures never fail the batch envelope (except 400/401/403) | AZ-488 task spec § Non-Functional Requirements | BT-14 (mixed-batch shows per-item isolation); `UavTileUploadHandlerTests.*PersistAsync*` (unit); reject reason `STORAGE_FAILURE` defined in contract for the orphan-row recovery path | ✓ |
|
||||
| AZ-488 Compatibility — Replaces 501 stub; coexists with AZ-484 `tile-storage` v1.0.0 contract on the write side | AZ-488 task spec § Non-Functional Requirements + Contract | `StubAndErrorContractTests` updated to drop the stub-501 expectation; BT-15 + BT-16 validate the AZ-484 invariants under live UAV writes | ✓ |
|
||||
| AZ-488 Security — Reject details never leak server internals; integer-only file-path construction | AZ-488 task spec § Non-Functional Requirements + Risk 2 | SEC-11 (blackbox); `UavTileFilePathTests` (unit) | ✓ |
|
||||
@@ -203,7 +204,7 @@
|
||||
|----------|-------------|-------------|---------------------|
|
||||
| Blackbox (positive) | 12 | 19/22 | — |
|
||||
| Blackbox (negative) | 5 | — | — |
|
||||
| Performance | 8 | 4 | 1 |
|
||||
| Performance | 9 | 5 | 1 |
|
||||
| Resilience | 6 | 4 | 3 |
|
||||
| Security | 14 | 9 (AZ-487 AC-1..AC-7, AZ-488 AC-6, leak-hygiene NFR) + 3 (AZ-1113 AC-1..AC-3) | 1 (AZ-487 supersedes "No authentication") |
|
||||
| Resource Limits | 7 | 5 | 4 |
|
||||
@@ -282,6 +283,12 @@
|
||||
| AZ-1123 AC-1 | `containerization.md` documents 5433 conflict + `docker-compose.perf.yml` command | doc-state AC; verified at Step 13 (`deployment/containerization.md` §Compose overlays) | ✓ |
|
||||
| AZ-1123 AC-2 | `environment.md` names perf overlay and links to containerization playbook | doc-state AC; verified at Step 13 (`tests/environment.md`) | ✓ |
|
||||
| AZ-1123 AC-3 | Integration (`docker-compose.tests.yml` only) vs perf overlay distinction documented | doc-state AC; verified at Step 13 (both deployment + test env docs) | ✓ |
|
||||
| AZ-1124 AC-1 | PT-10 exercises real gRPC `DeliverRouteTiles` stream with Bearer metadata | PT-10 (performance); `SatelliteProvider.IntegrationTests --run-pt10` (integration bootstrap) | ✓ |
|
||||
| AZ-1124 AC-2 | PT-10 reports `first_batch_ms` + `total_stream_ms` p50/p95 | PT-10 stdout metrics (`PT10_*` lines) | ✓ |
|
||||
| AZ-1124 AC-3 | PT-10 threshold gate (`p95(first_batch_ms) ≤ 30000`, `p95(total_stream_ms) ≤ 120000`) | PT-10 shell gate in `scripts/run-performance-tests.sh` | ◐ gate at Step 15 |
|
||||
| AZ-1124 AC-4 | PT-10 slow-consumer smoke completes without `DeliveryError` | PT-10 `PT10_SLOW_CONSUMER=PASS` sub-check | ✓ |
|
||||
| AZ-1124 AC-5 | PT-10 documented in `performance-tests.md`; gRPC stream perf no longer Unverified | doc-state AC; verified at Step 13 | ◐ doc-verified at Step 13 |
|
||||
| AZ-1124 AC-6 | PT-10 reuses `PerfBootstrap` / `JwtTokenFactory` / `GrpcTestHelpers` — no third JWT mint in shell | `PerfBootstrapPt10Tests` (unit — static review) | ✓ |
|
||||
|
||||
**Coverage shape notes (Cycle 11 — AZ-1123 perf compose documentation):**
|
||||
- Documentation-only cycle — no new runtime tests, blackbox scenarios, perf thresholds, or security findings. Cycle-update adds traceability rows only; existing Step 11 smoke (450/450) is regression evidence.
|
||||
|
||||
Reference in New Issue
Block a user