mirror of
https://github.com/azaion/satellite-provider.git
synced 2026-06-22 20:01:14 +00:00
Oleksandr Bezdieniezhnykh
cbbb26bd28
Step 15 (Performance Test) — skipped per gate (option B). Recording two deferred items in the existing perf leftover: * PT-07 + PT-08 remain Deferred. Both NFRs depend on the same baseline-capture harness that has not landed; the integration-test fixtures needed for PT-08 already exist (UavUploadTests + UavTileImageFactory), so PT-08 attaches to the same harness as PT-07 when implemented. * scripts/run-performance-tests.sh PT-01..PT-06 currently return 401 against the post-AZ-487 build because they attach no Bearer token. Script must mint an HS256 token from JWT_SECRET at script start before any curl call. Tracked in the leftover so PT-01..PT-06 are runnable again the same cycle PT-07/PT-08 are activated. No code change in this commit — leftover + state advance only. Co-authored-by: Cursor <cursoragent@cursor.com>
48 lines
4.5 KiB
Markdown
48 lines
4.5 KiB
Markdown
# Leftover — PT-07 perf harness + cycle 1 perf run
|
|
|
|
- **Timestamp**: 2026-05-11T07:05:00Z
|
|
- **Origin**: autodev cycle 1, Step 15 (Performance Test)
|
|
- **Blocker class**: non-user-input — work was deferred at the Step 15 user gate; no missing user decision blocks completion.
|
|
|
|
## What was deferred
|
|
|
|
1. **PT-07 implementation** in `scripts/run-performance-tests.sh`: capture pre-change baseline for `GetTilesByRegionAsync` p95 latency, run post-change measurement, compute the ratio, and assert `ratio ≤ 1.10`. PT-07 was recorded as a documentation entry during Step 12 (`_docs/02_document/tests/performance-tests.md` + `traceability-matrix.md` "NFRs → Test Mapping" section) but the runner script does not yet have the corresponding scenario.
|
|
2. **Active perf run** of PT-01..PT-06 against the post-AZ-484 build. The runner exists; it requires `docker-compose up` on the dev host. Not executed this cycle (per the meta-rule "ask before kicking off Docker / long-running perf operations").
|
|
|
|
## Why it is safe to defer
|
|
|
|
- AZ-484 functional correctness validated by 5 dedicated AZ-484 integration tests (Step 11). The DB read paths exercised by PT-07 are the same ones exercised by `MostRecentAcrossSourcesSelection_AZ484_AC2` etc.
|
|
- The post-AZ-484 SQL uses the same `idx_tiles_unique_location_source` index as the contract specifies; structurally there is no new full scan, join, or lock added vs. pre-AZ-484.
|
|
- Cycle 1 perf run is recorded as `Unverified` (not `Fail`) per the test-run perf-mode rules — gate does not block deploy.
|
|
|
|
## Replay actions for next /autodev invocation
|
|
|
|
When the next cycle's autodev runs, before any new tracker write or before re-entering Step 15 in cycle 2:
|
|
|
|
1. Add PT-07 to `scripts/run-performance-tests.sh`:
|
|
- Capture a pre-change baseline by checking out the parent of the AZ-484 commit (`git rev-parse HEAD~N` where N points at the AZ-484 batch), running the existing PT-03/PT-04 region scenarios, and recording the `GetTilesByRegionAsync` timings (the repository already logs slow query warnings at >500 ms — extend that log line to include median/p95 captured per call window).
|
|
- Run the post-change measurement against the current `HEAD`.
|
|
- Compute the p95 ratio and fail when `> 1.10`.
|
|
2. Bring up the docker stack (`docker-compose up --build -d`) and run the full perf script with the user's explicit go-ahead.
|
|
3. Capture results into `_docs/06_metrics/perf_<YYYY-MM-DD>_cycle<N>.md`.
|
|
4. Once results are recorded, delete this leftover file.
|
|
|
|
## AZ-488 follow-on: PT-08 (UAV upload latency)
|
|
|
|
The AZ-488 commit added PT-08 (UAV tile batch upload latency) to `_docs/02_document/tests/performance-tests.md` with Status `Deferred` because it reuses the same harness expansion as PT-07 (baseline capture + p95 ratio). When PT-07's runner-script scenario is implemented in step 1 above, add the PT-08 scenario in the **same commit** — the integration-test fixtures already exist (`SatelliteProvider.IntegrationTests/UavUploadTests` happy-path JWT + `UavTileImageFactory.CreateRandomJpeg`). After PT-08 runs, flip the Status line in `performance-tests.md` from `Deferred` to active. This keeps cycle 1 retro Action 2 satisfied for both NFRs.
|
|
|
|
## AZ-487 follow-on: scripts/run-performance-tests.sh attaches no Bearer token (cycle 2 carry-over)
|
|
|
|
Cycle 2's Step 15 (Performance Test) skip-decision uncovered an additional latent blocker: `scripts/run-performance-tests.sh` calls every `/api/satellite/*` endpoint without an `Authorization` header. Post-AZ-487 every such call returns HTTP 401 — the script is currently broken end-to-end, not just for PT-07/PT-08. Whoever picks up the harness work in step 1 above MUST also:
|
|
|
|
1. Read `JWT_SECRET` from the host env (the same value used by `docker-compose.yml`).
|
|
2. Mint a short-lived HS256 token at the top of the script (mirror `SatelliteProvider.IntegrationTests/JwtTestHelpers.MintValidToken` — small `python3 -c` or `jwt` CLI call; do not commit the dev placeholder secret).
|
|
3. Add the token to every `curl` invocation via `-H "Authorization: Bearer $JWT"`.
|
|
4. Skip-the-test-cleanly when `JWT_SECRET` is unset rather than running and failing on every call.
|
|
|
|
This is purely script work; no production code needs to change. Tracking it here so PT-01..PT-06 are runnable again the same cycle PT-07/PT-08 are activated.
|
|
|
|
## Tracker action (none required this cycle)
|
|
|
|
This leftover does NOT require a Jira ticket on its own — it tracks deferred process work, not user-visible scope. If the perf comparison reveals a regression next cycle, that finding will create a Jira bug; until then there is nothing to file.
|