# Performance Test Report — Cycle 1

**Date**: 2026-05-13
**Cycle**: 1
**Verdict**: **PASS** — all thresholds met, 0% error rate.
**Runner**: k6 v2.0.0 (local) against `docker-compose.test.yml` (Postgres 16-alpine + .NET admin API), seeded with 500 perf users.
**Artifacts**: `scripts/perf-scenarios.js`, `scripts/run-performance-tests.sh`, raw JSON at `e2e/test-results/perf-summary.json`.

## Scenarios run

| ID | Scenario | Threshold | Observed (p95) | Verdict |
|----|----------|-----------|---------------:|---------|
| NFT-PERF-01 | Login (10 VUs, 30s) | p95 < 500ms · err < 1% | **33.4ms · 0%** | Pass (15× headroom) |
| NFT-PERF-04 | User list (10 VUs, 30s, 500 users seeded) | p95 < 1000ms · err < 1% | **152.5ms · 0%** | Pass (6.5× headroom) |

## Scenarios skipped

| ID | Scenario | Reason |
|----|----------|--------|
| NFT-PERF-02 | Encrypted resource download (small) | Endpoint deleted (AZ-183 OTA revert + AZ-197 hardware removal). Pruned from `_docs/02_document/tests/performance-tests.md`. |
| NFT-PERF-03 | Encrypted resource download (large) | Same — the OTA / hardware-bound download path no longer exists. |

## Detailed metrics (full distribution)

### NFT-PERF-01 — Login

| Metric | Value |
|--------|------:|
| Iterations | 2 617 |
| min | 1.3 ms |
| median | 6.3 ms |
| avg | 13.7 ms |
| p90 | 18.6 ms |
| **p95** | **33.4 ms** |
| max | 630.0 ms (single outlier — first request after JIT/connection-pool warmup) |
| Error rate | 0.00% |
| Checks | 2 617 / 2 617 (status 200, token returned) |

### NFT-PERF-04 — User list (500 users)

| Metric | Value |
|--------|------:|
| Iterations | 1 944 |
| min | 3.1 ms |
| median | 12.0 ms |
| avg | 43.8 ms |
| p90 | 86.9 ms |
| **p95** | **152.5 ms** |
| max | 1 974.6 ms (cold-cache outlier) |
| Error rate | 0.00% |
| Checks | 1 944 / 1 944 (status 200, ≥ 500 users returned) |

### Aggregate

| Metric | Value |
|--------|------:|
| Total iterations | 4 561 |
| Total HTTP requests | 4 562 |
| Aggregate throughput | 65.1 req/s |
| Max VUs | 20 (10 per scenario, sequential) |
| Run duration | ~70 s (incl. 5 s gap between scenarios) |

## Threshold table

All four `options.thresholds` entries returned `ok: true`:

```
http_req_duration{scenario:nft_perf_01_login}    p(95)<500   →  ok
http_req_duration{scenario:nft_perf_04_user_list} p(95)<1000  →  ok
http_req_failed{scenario:nft_perf_01_login}      rate<0.01   →  ok (rate=0)
http_req_failed{scenario:nft_perf_04_user_list}  rate<0.01   →  ok (rate=0)
```

## Environment

- Host: macOS 25.4.0 (Apple Silicon)
- Docker Desktop, single host, no resource throttling
- SUT: `system-under-test` container built from repo `Dockerfile`, running on `http://localhost:8080`
- DB: `test-db` (postgres:16-alpine), in-process to the same Docker host
- Seed: functional fixtures from `e2e/db-init/00_run_all.sh` + `99_test_seed.sql`, plus 500 dummy `perf-user-NNNNN@perf.azaion.com` rows inserted by `run-performance-tests.sh` after SUT readiness check
- k6 v2.0.0 (Homebrew bottle, arm64)

## Caveats / coverage gaps

1. **Single-host run** — perf was measured with k6 and the SUT on the same machine, no network RTT, no inter-AZ latency. Production numbers will be higher; the 15×/6.5× headroom should absorb that comfortably for an internal admin API.
2. **No DB warmup phase** — both p99/max values include cold-cache outliers (login max 630ms, user-list max ~2s). The p95 already excludes those, but a future iteration could add a 5–10s warmup ramp.
3. **No realistic load on the user-list filter path** — only the unfiltered `GET /users` is exercised. Adding a `?searchEmail=` variant would catch the case where the LinqToDB `WhereIf` fails to fold into the SQL.
4. **No `/classes` CRUD perf coverage** — cycle 1 added these endpoints (AZ-513) but the perf spec was not extended. Recommend adding a NFT-PERF-05 in the next test-spec sync.
5. **`acceptance_criteria.md` is stale post-cycle-1** — AC-13/14/15/16 (hardware binding) and AC-17–24 (resource management) reference deleted features. Step 12 (Test-Spec Sync) of cycle 1 missed this. Surface in Step 17 retro and clean up in cycle 2.

## Recommendations for next cycle

- **Cycle 2 test-spec sync must prune AC-13..24** and add an AC for `/classes` CRUD.
- **Add NFT-PERF-05** for `POST /classes` and `PATCH /classes/{id}` to cover the new write paths.
- **CI gate**: wire `scripts/run-performance-tests.sh` into the deploy pipeline so threshold breaches block release. Today it is run-on-demand only.

## Verdict logic

PASS — all thresholds met, no failed checks, no errors, no warn-band scenarios. Auto-chain to Step 16 (Deploy).