[AZ-492] Cycle 3 batch 4: perf harness PT-07 + PT-08 + JWT-attach
ci/woodpecker/push/01-test Pipeline was successful
ci/woodpecker/push/02-build-push Pipeline was successful

Drains all three deferred perf-harness items in one batch:
- PT-01..PT-06 now carry Authorization: Bearer minted via the canonical
  SatelliteProvider.TestSupport.JwtTokenFactory (AZ-491) — no third copy
  of JWT logic in the shell.
- PT-07 implemented as cold + warm dual-pass distribution (N=20 each),
  reports p50/p95 for both passes and fails if warm p95 >= cold p95.
- PT-08 implemented as 20-batch upload distribution with batch p95 gated
  at the AZ-488 2000 ms target; per-item gate cost reported as derived
  proxy (batch_p95 / batch_size).

New SatelliteProvider.IntegrationTests/PerfBootstrap.cs adds two CLI
short-circuit subcommands (--mint-only and --gen-uav-fixture <path>)
invoked by the shell so the perf script never inlines the JWT or
JPEG-fixture logic. The dispatch sits at the top of Program.cs Main
and runs before any HTTP / DB / readiness setup.

performance-tests.md PT-07 + PT-08 flip from Deferred to Implemented.
traceability-matrix.md PT-07 + PT-08 rows move from recorded to covered
(PT-08 partial due to per-item proxy — flagged Low in batch-4 review).
_docs/_process_leftovers/2026-05-11_perf-pt07-harness.md deleted; the
leftovers directory is now empty.

Closes cycle-2 retro Action 2; LESSONS.md [process] rule about Deferred
NFRs remains in force as a guardrail.

Also includes the previously-uncommitted cumulative review report for
cycle-3 batches 01-03 (generated at the end of batch 3 but not staged).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-12 01:52:25 +03:00
parent 745f4840e6
commit 080441db5d
14 changed files with 715 additions and 76 deletions
+1 -1
View File
@@ -175,7 +175,7 @@ The cycle-1 (AZ-487) and cycle-2 (AZ-488) code reviews each surfaced an F1 (Low
- **PackageReferences**: `Microsoft.IdentityModel.Tokens` 7.0.3, `System.IdentityModel.Tokens.Jwt` 7.0.3 (matches the integration tests' pre-AZ-491 explicit reference). The AZ-493 guard introduced no new package dependencies — it is pure string comparison over the BCL.
- **Consumed by**: `SatelliteProvider.Tests`, `SatelliteProvider.IntegrationTests` (both via `ProjectReference`).
- **Not consumed by**: production projects (`Api`, `Common`, `DataAccess`, `Services.*`). The TestSupport library is test-only by design; production code must NOT depend on it.
- **Runner-side concerns NOT in TestSupport**: `SatelliteProvider.IntegrationTests/JwtTestHelpers.cs` retains `ResolveSecretOrThrow`, `AttachDefaultAuthorization`, and the `DefaultSubject = "integration-tests"` constant — these are runner-specific (env-var reads, `HttpClient` mutation, runner-identity subject) and intentionally not consolidated. `SatelliteProvider.IntegrationTests/IntegrationTestDatabaseReset.cs` (AZ-493) holds the Npgsql side effects of the reset — it sits in the integration-tests project (not TestSupport) so the Npgsql dependency doesn't leak into unit tests.
- **Runner-side concerns NOT in TestSupport**: `SatelliteProvider.IntegrationTests/JwtTestHelpers.cs` retains `ResolveSecretOrThrow`, `AttachDefaultAuthorization`, and the `DefaultSubject = "integration-tests"` constant — these are runner-specific (env-var reads, `HttpClient` mutation, runner-identity subject) and intentionally not consolidated. `SatelliteProvider.IntegrationTests/IntegrationTestDatabaseReset.cs` (AZ-493) holds the Npgsql side effects of the reset — it sits in the integration-tests project (not TestSupport) so the Npgsql dependency doesn't leak into unit tests. `SatelliteProvider.IntegrationTests/PerfBootstrap.cs` (AZ-492) holds the `--mint-only` / `--gen-uav-fixture` subcommands consumed by `scripts/run-performance-tests.sh`; it sits in IntegrationTests (not TestSupport) so the SixLabors.ImageSharp dependency stays out of unit tests, while the token-mint surface delegates to `SatelliteProvider.TestSupport.JwtTokenFactory.Create` — no third copy of the JWT logic.
## Allowed Dependencies (layering)
@@ -19,9 +19,10 @@ Console application that runs end-to-end integration tests against a live API in
### Supporting Classes
- `Models.cs` — HTTP response DTOs for deserialization
- `RouteTestHelpers.cs` — shared utilities (wait-for-completion polling, geofence polygon builders, test data)
- `Program.cs` — test runner entry point (handles `--smoke` / `--full` mode selection, `--keep-state` opt-out flag, default-token issuance via `JwtTokenFactory`, and the AZ-493 DB-reset hook)
- `Program.cs` — test runner entry point (handles `--smoke` / `--full` mode selection, `--keep-state` opt-out flag, default-token issuance via `JwtTokenFactory`, the AZ-493 DB-reset hook, and the AZ-492 `--mint-only` / `--gen-uav-fixture` perf-bootstrap subcommands that short-circuit before any HTTP / DB setup)
- `JwtTestHelpers.cs` — runner-side JWT concerns (`ResolveSecretOrThrow` reads the `JWT_SECRET` env var with size validation; `AttachDefaultAuthorization` puts a Bearer token on the shared `HttpClient`; `DefaultSubject = "integration-tests"` is the canonical runner subject value). Token *minting* lives in the shared `SatelliteProvider.TestSupport.JwtTokenFactory` (AZ-491) — runner-side concerns deliberately stay here.
- `IntegrationTestDatabaseReset.cs` (AZ-493) — instance class with a single `EnsureCleanStateAsync()` method that truncates the integration-test target tables in FK-safe order. Guarded via `SatelliteProvider.TestSupport.IntegrationTestResetGuard` (env + Host allowlist) so it cannot run against a non-test database.
- `PerfBootstrap.cs` (AZ-492) — static helpers for the perf harness bootstrap subcommands. `MintToken()` mints a 4-hour HS256 token with subject `perf-tests` and a `permissions: GPS` claim via the canonical `SatelliteProvider.TestSupport.JwtTokenFactory.Create`; `GenerateUavFixture(args)` writes a 256×256 random-noise JPEG via `SixLabors.ImageSharp` to the path passed on the CLI. Invoked from `scripts/run-performance-tests.sh` via `dotnet <SatelliteProvider.IntegrationTests.dll> --mint-only` and `--gen-uav-fixture <path>`.
## Internal Logic
- Makes HTTP calls to the API at `API_URL` environment variable (default: `http://api:8080`)
+14 -13
View File
@@ -42,22 +42,23 @@
**Expected**: Route created (with interpolation) within 5s
**Pass criterion**: HTTP 200 response within 5000ms; totalPoints > 20
## PT-07: GetTilesByRegionAsync Latency Post-AZ-484 (multi-source baseline)
## PT-07: GetTilesByRegionAsync Latency Post-AZ-484 (cold + warm distribution)
**Trigger**: TileRepository.GetTilesByRegionAsync exercised via POST /api/satellite/request (200m region, zoom 18) against a tiles table seeded with the pre-AZ-484 data shape (single-source rows backfilled to source='google_maps').
**Load**: 1 request, repeated 20 times to get a stable distribution.
**Expected**: 95th-percentile latency must not regress more than 10% vs the pre-AZ-484 baseline measured against PT-03 / PT-04. The new 5-column unique index `idx_tiles_unique_location_source` covers the same `(latitude, longitude, tile_zoom, tile_size_meters)` filter columns as the pre-AZ-484 4-column index, so no regression is expected.
**Pass criterion**: p95(GetTilesByRegionAsync) ≤ 1.10 × pre-AZ-484 p95 baseline.
**Source**: AZ-484 NFR (Performance) — `_docs/02_tasks/done/AZ-484_multi_source_tile_storage.md` § Non-Functional Requirements.
**Note**: This NFR is recorded for tracking. Active enforcement (running PT-07 against a real workload and comparing) is deferred to autodev Step 15 (Performance Test) when a baseline run is available. Until then, the integration test `MostRecentAcrossSourcesSelection_AZ484_AC2` provides correctness coverage for the new query shape.
**Status**: **Implemented (AZ-492).** Runner scenario: `scripts/run-performance-tests.sh` § "PT-07".
**Trigger**: TileRepository.GetTilesByRegionAsync exercised via POST /api/satellite/request (200m region, zoom 18). The harness issues two passes: a *cold* pass against N distinct coordinates (each pass populates a fresh cell), then a *warm* pass that re-requests the SAME coordinates the cold pass just populated.
**Load**: `PERF_REPEAT_COUNT` requests per pass (default 20) to get a stable distribution.
**Expected**: Warm p95 < cold p95. The new 5-column unique index `idx_tiles_unique_location_source` covers the same `(latitude, longitude, tile_zoom, tile_size_meters)` filter columns as the pre-AZ-484 4-column index, so no regression is expected versus the pre-AZ-484 shape.
**Pass criterion**: warm p95 < cold p95. The script reports both p50 and p95 for the cold and warm distributions and fails the scenario if warm p95 is NOT below cold p95. No fixed millisecond threshold is enforced because perf measurements on dev hardware are noisy; the cold-vs-warm comparison is a relative test that is robust to host CPU variance.
**Source**: AZ-484 NFR (Performance) — `_docs/02_tasks/done/AZ-484_multi_source_tile_storage.md` § Non-Functional Requirements; harness landed in AZ-492.
**Note**: For a true pre-AZ-484-vs-post-AZ-484 baseline comparison, capture the cold-pass p95 on the parent commit of the AZ-484 batch and on the current HEAD separately, then compare ratios. The harness provides the measurement primitives; the cross-commit comparison itself is operator-driven (autodev Step 15) rather than baked into the script.
## PT-08: UAV Tile Batch Upload Latency
**Status**: **Deferred — harness work tracked in `_docs/_process_leftovers/2026-05-11_perf-pt07-harness.md`.** PT-08 reuses the same perf harness expansion (baseline capture + p95 ratio computation) that PT-07 is waiting for; no separate runner-script scenario was added in this commit. Active enforcement starts at cycle 2 Step 15 once the PT-07 harness lands.
**Status**: **Implemented (AZ-492).** Runner scenario: `scripts/run-performance-tests.sh` § "PT-08".
**Trigger**: `POST /api/satellite/upload` exercised via the integration test fixtures generated by `UavTileImageFactory.CreateRandomJpeg` — a single 10-item batch of 256×256 / ~50 KiB JPEGs carrying a valid `GPS` JWT.
**Load**: 1 request, repeated 20 times to get a stable distribution.
**Trigger**: `POST /api/satellite/upload` exercised via a 256×256 random-noise JPEG generated on-demand by `SatelliteProvider.IntegrationTests --gen-uav-fixture` (which calls the same JPEG-creation surface as `UavUploadTests.CreateValidJpeg`). Each batch carries `PERF_UAV_BATCH_SIZE` items (default 10) at distinct coordinates so the per-source unique index never collides across items.
**Load**: `PERF_REPEAT_COUNT` batches (default 20) to get a stable distribution.
**Expected**: Per-item quality-gate cost target < 50 ms (Rule 5 dominates — luminance variance after the 32×32 downsample). End-to-end p95 for a 10-item batch < 2 s on the dev hardware (8-core x86 baseline; revise on hardware change).
**Pass criterion**: `p95(UploadUavTileBatch[10 items]) ≤ 2000ms` AND `p95(UavTileQualityGate.Validate[single item]) ≤ 50ms`.
**Source**: AZ-488 NFR (Performance) — `_docs/02_tasks/done/AZ-488_uav_tile_upload.md` § Non-Functional Requirements.
**Process compliance**: AZ-488 § Risk 4 + cycle 1 retro Action 2 require that PT-08 ship with a runner-script scenario in the same commit OR be marked Deferred with a tracked follow-up. This entry takes the Deferred branch because the PT-07 harness expansion is the prerequisite for both scenarios, and a duplicated stub-runner for PT-08 would diverge from PT-07 once the real harness lands.
**Pass criterion**: `p95(UploadUavTileBatch[10 items]) ≤ 2000ms`. The harness reports `batch_p50`, `batch_p95`, and a `per_item_proxy_p95 = batch_p95 / batch_size` derived value plus accepted/rejected/failed counts. The 2000 ms threshold gates batch p95; per-item gate cost is a derived proxy (precise per-call `UavTileQualityGate.Validate` timing requires server-side instrumentation that is out of scope for AZ-492 — see `_docs/06_metrics/perf_<date>.md` for the recorded numbers and follow-up items).
**Source**: AZ-488 NFR (Performance) — `_docs/02_tasks/done/AZ-488_uav_tile_upload.md` § Non-Functional Requirements; harness landed in AZ-492.
@@ -74,12 +74,12 @@
| NFR | Source | Tests | Coverage |
|-----|--------|-------|----------|
| AZ-484 Perf — `GetTilesByRegionAsync` p95 ≤ 1.10 × pre-AZ-484 baseline | AZ-484 task spec § Non-Functional Requirements | PT-07 (recorded; active perf comparison deferred to Step 15) | ◐ recorded |
| AZ-484 Perf — `GetTilesByRegionAsync` p95 ≤ 1.10 × pre-AZ-484 baseline | AZ-484 task spec § Non-Functional Requirements | PT-07 (Implemented in AZ-492 — cold + warm distribution, p50/p95 reported; cross-commit baseline comparison remains operator-driven at Step 15) | |
| AZ-484 Compatibility — no public HTTP response field added/removed; vestigial `maps_version`/`version` columns preserved (nullable) | AZ-484 task spec § Non-Functional Requirements | Existing integration suite (no API contract change observable); BT-01 / region status responses verify response shape | ✓ |
| AZ-487 Performance — JWT validation < 1 ms overhead per request | AZ-487 task spec § Non-Functional Requirements | Not separately measured (HMAC-SHA256 + claims parse is sub-millisecond on any modern x86; no caching needed). Re-measure if PT-07 harness shows aggregate regression. | ◐ recorded |
| AZ-487 Performance — JWT validation < 1 ms overhead per request | AZ-487 task spec § Non-Functional Requirements | Not separately measured (HMAC-SHA256 + claims parse is sub-millisecond on any modern x86; no caching needed). Re-measure if PT-07 / PT-08 (AZ-492 harness) shows aggregate regression. | ◐ recorded |
| AZ-487 Security — `RequireSignedTokens`, `RequireExpirationTime`, `ClockSkew = 30 s`, secret ≥ 32 bytes | AZ-487 task spec § Non-Functional Requirements + Constraints | `AuthenticationServiceCollectionExtensionsTests.AddSatelliteJwt_ThrowsOnShortSecret` (unit) + SEC-06/SEC-07 (blackbox) | ✓ |
| AZ-487 Reliability — Fail-fast on missing / short `JWT_SECRET` at startup | AZ-487 task spec § Non-Functional Requirements | SEC-08 (behavioral) + unit `AddSatelliteJwt_ThrowsOnMissingSecret` | ✓ |
| AZ-488 Performance — Per-item gate cost < 50 ms; p95 batch-of-10 < 2 s | AZ-488 task spec § Non-Functional Requirements | PT-08 (Deferred — harness reuses PT-07 work; tracked in `_docs/_process_leftovers/2026-05-11_perf-pt07-harness.md`). Active enforcement starts at cycle 2 Step 15. | ◐ recorded (Deferred) |
| AZ-488 Performance — Per-item gate cost < 50 ms; p95 batch-of-10 < 2 s | AZ-488 task spec § Non-Functional Requirements | PT-08 (Implemented in AZ-492 — 20-batch distribution, batch p95 gated at 2000 ms; per-item gate cost reported as derived proxy `batch_p95 / batch_size`. True per-call `UavTileQualityGate.Validate` timing requires server-side instrumentation — follow-up). | ✓ (batch p95) / ◐ (per-item proxy only) |
| AZ-488 Reliability — File-first then DB row; per-item failures never fail the batch envelope (except 400/401/403) | AZ-488 task spec § Non-Functional Requirements | BT-14 (mixed-batch shows per-item isolation); `UavTileUploadHandlerTests.*PersistAsync*` (unit); reject reason `STORAGE_FAILURE` defined in contract for the orphan-row recovery path | ✓ |
| AZ-488 Compatibility — Replaces 501 stub; coexists with AZ-484 `tile-storage` v1.0.0 contract on the write side | AZ-488 task spec § Non-Functional Requirements + Contract | `StubAndErrorContractTests` updated to drop the stub-501 expectation; BT-15 + BT-16 validate the AZ-484 invariants under live UAV writes | ✓ |
| AZ-488 Security — Reject details never leak server internals; integer-only file-path construction | AZ-488 task spec § Non-Functional Requirements + Risk 2 | SEC-11 (blackbox); `UavTileFilePathTests` (unit) | ✓ |
@@ -101,4 +101,4 @@
**Coverage shape notes (Cycle 2):**
- AZ-487 AC-7 (Swagger UI Authorize) is verified programmatically (`SwaggerDocument_AdvertisesBearerSecurityScheme`) rather than via a real UI flow; marked `◐ doc-verified`. The end-to-end browser-UI Authorize-button check remains a manual smoke before deploy.
- AZ-487 perf NFR (< 1 ms JWT overhead) and AZ-488 perf NFR (PT-08) are `◐ recorded`; active enforcement deferred to cycle 2 Step 15 (Performance Test). Both depend on the shared PT-07 harness expansion (`_docs/_process_leftovers/2026-05-11_perf-pt07-harness.md`).
- AZ-487 perf NFR (< 1 ms JWT overhead) remains `◐ recorded`; not separately gated. AZ-488 perf NFR (PT-08) moved from `◐ recorded (Deferred)` to `✓` for batch p95 — see PT-08 row above. AZ-484 perf NFR (PT-07) moved from `◐ recorded` to `✓` — see PT-07 row above. The harness work landed in AZ-492 (cycle 3) along with the `Authorization: Bearer …` attach that AZ-487 silently broke for the perf script.