mirror of
https://github.com/azaion/missions.git
synced 2026-06-21 22:01:07 +00:00
[AZ-581] [AZ-582] [AZ-583] [AZ-584] Sec+Res NFT tests
Batch 3 of test implementation cycle 1 (existing-code Step 6). - AZ-581 AuthClaimsTests: NFT-SEC-01..06+04b (foreign-keypair, byte-flip, 30s skew, iss/aud/perms, multi-value permissions array). - AZ-582 CrossCutting/ErrorRedaction/JwksRotation/StartupConfig/CorsConfig: NFT-SEC-07..13 (alg pin, kid rotation grace window, env fail-fast, CORS Production gate). - AZ-583 CascadeF3/CascadeF4/MigratorRestart: NFT-RES-01..04. CascadeF4 pins current walk-order divergence with carry_forward AC-4.6. - AZ-584 ConfigDbStartup/JwksRotationNoRestart/DefaultVehicleRace: NFT-RES-05..08. NFT-RES-08 pins current behaviour (unique-index closes the race) with carry_forward AC-1.4. Mock contract: SignBody accepts permissions OR permissions_array (mutually exclusive). TokenSigner validates kid_override against published keys so NFT-SEC-11 can assert "mock refuses old kid post-grace". Helpers added: ForeignKeypair (test-only ECDSA P-256), MissionsContainerHelper (docker-run wrapper for startup-time scenarios), DockerLogs. 7 of 22 new tests are Skippable, gated on COMPOSE_RESTART_ENABLED + docker CLI in the e2e-consumer image (explicit skip reason; no silent pass). Build green: test csproj + jwks-mock csproj. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,125 @@
|
||||
# Security Tests — Auth & Claims
|
||||
|
||||
**Task**: AZ-581_test_security_auth_claims
|
||||
**Name**: Security tests — auth & claims (NFT-SEC-01..06 + 04b)
|
||||
**Description**: Implement xUnit blackbox tests for the 7 JWT authn/authz scenarios — missing/invalid header, invalid signature (single-byte flip + foreign-keypair), expired-outside-skew vs inside-30s-skew, wrong `iss`, wrong `aud`, missing `permissions`, wrong/multi-value `permissions` claim (contains-match accepts `["FL","ADMIN"]`).
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-581
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
JWT validation is the only thing standing between the open `e2e-net` and the protected `/vehicles` + `/missions` + `/missions/{id}/waypoints` surface. Six failure modes (no header / bad signature / expired / wrong iss / wrong aud / wrong perm) MUST all produce `401` or `403` deterministically — any drift means an attacker who learns the JWKS public bytes could shape a token that bypasses one rule and rides through. The drift re-verification of 2026-05-14 split AC-5.3 into two checks (`iss` AND `aud`) and tightened the clock skew from .NET's 5-min default to 30s; this task pins both. NFT-SEC-06 specifically asserts the `RequireClaim("permissions","FL")` is contains-match — a multi-permission token `["FL","ADMIN"]` must be accepted, while `"fl"` / `"FLight"` / `"ADMIN"` alone must be rejected.
|
||||
|
||||
## Outcome
|
||||
|
||||
- All seven NFT-SEC-01..06 + 04b scenarios run and pass against the dockerised `missions` service.
|
||||
- Each test produces a CSV row with `Category=Sec`, `Traces=AC-5.x` or `AC-9.x`, `Result=pass`.
|
||||
- NFT-SEC-02 covers BOTH the single-byte-flip case AND the foreign-keypair case (token signed by a separate ECDSA keypair never published in the JWKS).
|
||||
- NFT-SEC-03 verifies the 30s skew BOTH ways — `exp_offset_seconds=-60` rejected, `exp_offset_seconds=-15` accepted.
|
||||
- NFT-SEC-06 verifies multi-permission token acceptance — `permissions: ["FL","ADMIN"]` → `200`.
|
||||
- NFT-SEC-01 asserts no DB side-effect on the `POST /vehicles` 401 path (side-channel count unchanged).
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- NFT-SEC-01 Missing `Authorization` header on `/vehicles` GET/POST, `/missions` GET, `/missions/{any}/waypoints` GET — all `401`, no DB row written on the POST.
|
||||
- NFT-SEC-02 Invalid signature — single-byte-flipped signature segment AND foreign-keypair tokens.
|
||||
- NFT-SEC-03 Expired token — `exp_offset_seconds=-60` → `401`; `exp_offset_seconds=-15` → `200` (inside 30s skew).
|
||||
- NFT-SEC-04 Wrong `iss` — `POST /sign { "iss": "https://attacker.example.com" }` → `401`; default `iss` → `200`.
|
||||
- NFT-SEC-04b Wrong `aud` — `POST /sign { "aud": "wrong-audience" }` → `401`.
|
||||
- NFT-SEC-05 Missing `permissions` claim — `403`.
|
||||
- NFT-SEC-06 Wrong `permissions` value AND multi-permission acceptance — `"fl"`, `"FLight"`, `"ADMIN"` → `403`; `["FL","ADMIN"]` → `200`.
|
||||
|
||||
### Excluded
|
||||
|
||||
- NFT-SEC-07 health-exempt-from-auth lives in Task 15.
|
||||
- NFT-SEC-08 stacktrace-not-leaked overlaps with FT-N-08 in Task 13 (and lives in Task 15 for the security-shaped variant).
|
||||
- NFT-SEC-09 SQL injection guard lives in Task 15.
|
||||
- NFT-SEC-10 alg-pin lives in Task 15.
|
||||
- NFT-SEC-11 unknown-kid rotation lag lives in Task 15.
|
||||
- NFT-SEC-12 missing-env startup throw lives in Task 15.
|
||||
- NFT-SEC-13 CORS Production-gate lives in Task 15.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: NFT-SEC-01 missing header rejects every protected endpoint with 401, no side-effect**
|
||||
Given the running test stack
|
||||
When the consumer issues `GET /vehicles`, `GET /missions`, `GET /missions/{any}/waypoints`, and `POST /vehicles` with a valid body — all without an `Authorization` header
|
||||
Then each response is `401`, AND side-channel `SELECT COUNT(*) FROM vehicles` before and after the `POST` are equal
|
||||
|
||||
**AC-2: NFT-SEC-02 invalid signature rejects two attack shapes**
|
||||
Given a valid signed token `T_good` from `jwks-mock POST /sign`
|
||||
When the consumer flips a single byte in `T_good`'s signature segment producing `T_bad`, and separately mints `T_foreign` signed by an ECDSA keypair never published in the JWKS
|
||||
Then `GET /vehicles` with `T_bad` returns `401` AND `GET /vehicles` with `T_foreign` returns `401`
|
||||
|
||||
**AC-3: NFT-SEC-03 30s clock skew is enforced on both sides**
|
||||
Given the mock with default issuer/audience
|
||||
When the consumer mints two tokens via `POST /sign { exp_offset_seconds: -60 }` and `POST /sign { exp_offset_seconds: -15 }`
|
||||
Then `GET /vehicles` with the −60s token returns `401` AND `GET /vehicles` with the −15s token returns `200`
|
||||
|
||||
**AC-4: NFT-SEC-04 wrong `iss` rejected, matching `iss` accepted**
|
||||
When the consumer mints a token via `POST /sign { iss: "https://attacker.example.com" }` and another via `POST /sign {}` (default iss)
|
||||
Then `GET /vehicles` with the attacker-iss token returns `401` AND with the default-iss token returns `200`
|
||||
|
||||
**AC-5: NFT-SEC-04b wrong `aud` rejected**
|
||||
When the consumer mints a token via `POST /sign { aud: "wrong-audience" }`
|
||||
Then `GET /vehicles` returns `401`
|
||||
|
||||
**AC-6: NFT-SEC-05 missing `permissions` claim rejected with 403**
|
||||
When the consumer mints a token with no `permissions` claim (mock body `{ permissions: "" }` or `{ permissions: null }` per the mock's contract)
|
||||
Then `GET /vehicles` returns `403` (NOT 401 — signature is valid)
|
||||
|
||||
**AC-7: NFT-SEC-06 contains-match policy on `permissions`**
|
||||
When the consumer mints tokens with `permissions` values `"ADMIN"`, `"fl"` (lowercase), `"FLight"`, AND `["FL","ADMIN"]` (multi-value array)
|
||||
Then `GET /vehicles` returns `403` for the first three AND `200` for the multi-value `["FL","ADMIN"]` array (contains-match accepts `"FL"` among the values)
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- NFT-SEC-01..06: ≤ 5s each. The Authorization-header failure paths are cheap (no DB round-trip on the 401/403 short-circuit).
|
||||
|
||||
**Reliability**
|
||||
- NFT-SEC-02 requires an out-of-band ECDSA-keypair helper that lives inside the test project, NOT in `jwks-mock` (the mock must never publish a public key it does not control). The helper generates a P-256 keypair at test-start and signs a token directly using `System.Security.Cryptography.ECDsa` — the public key is never registered with `missions`.
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | running stack | 4 endpoints w/o Authorization | all 401; POST no DB write | AC-5.4 |
|
||||
| AC-2 | `T_good` from mock + foreign keypair | flipped signature; foreign-keypair token | both 401 | AC-5.5 |
|
||||
| AC-3 | mock with default iss/aud | exp_offset −60s vs −15s | 401 / 200 | AC-5.2, AC-5.6 |
|
||||
| AC-4 | mock | iss=attacker vs default | 401 / 200 | AC-5.3, AC-5.11 |
|
||||
| AC-5 | mock | aud=wrong | 401 | AC-5.3, AC-5.12 |
|
||||
| AC-6 | mock | permissions missing | 403 | AC-5.8, AC-9.1 |
|
||||
| AC-7 | mock | permissions=ADMIN/fl/FLight/["FL","ADMIN"] | 403/403/403/200 | AC-9.1, AC-9.2 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080`. Tokens minted via `https://jwks-mock:8443/sign` with parameterised overrides.
|
||||
- NFT-SEC-02 foreign-keypair: a test-only helper inside `Azaion.Missions.E2E.Tests` MAY use `System.Security.Cryptography.ECDsa` directly for the attack-token construction; this is the ONLY in-test signing path allowed — every other test must use the mock.
|
||||
- NFT-SEC-06 multi-permission token requires the mock's `POST /sign` body to accept `permissions` as either a string OR a JSON array; the test-infrastructure ticket (AZ-576) covers this in the mock's contract.
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` per test.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: NFT-SEC-03 flaky due to wall-clock variability**
|
||||
- *Risk*: A −15s offset could fail if Docker time skew between the mock and `missions` is large.
|
||||
- *Mitigation*: Both containers run on the same host clock (no `--init` time isolation); test asserts only at offsets well clear of the 30s boundary (−60s and −15s — 30s and 15s away from the boundary respectively).
|
||||
|
||||
**Risk 2: NFT-SEC-06 multi-permission shape varies between systems**
|
||||
- *Risk*: If the spec for `permissions` claim later changes from "contains-match string" to "exact-array-membership", the multi-value assertion breaks.
|
||||
- *Mitigation*: Test traces explicitly to AC-9.2 and references `Auth/JwtExtensions.cs` policy registration; any change there must update this test in the same commit.
|
||||
|
||||
**Risk 3: Foreign-keypair token validation might pass if the SUT silently trusts any well-formed ECDSA token**
|
||||
- *Risk*: A regression that disables `IssuerSigningKeyResolver` would let the foreign-keypair token through.
|
||||
- *Mitigation*: Mitigated by the structure of AC-2 — both bad-signature shapes (flipped byte AND foreign keypair) must return 401.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface (`http://missions:8080/{vehicles,missions}*`) and acquire signed tokens via `https://jwks-mock:8443/sign` (with the test-only foreign-keypair helper for NFT-SEC-02). Expected outputs are the documented HTTP status codes from `_docs/00_problem/input_data/expected_results/results_report.md` AC-5 rows and AC-9 rows.
|
||||
- Stubs are allowed ONLY for: the external `admin` JWT issuer (`jwks-mock` container).
|
||||
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including `JwtExtensions`, `Program.cs` (auth pipeline registration), the `[Authorize(Policy = "FL")]` filter, or `ErrorHandlingMiddleware`. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
@@ -0,0 +1,140 @@
|
||||
# Security Tests — Alg-pin / Rotation / CORS / No-leak
|
||||
|
||||
**Task**: AZ-582_test_security_alg_rotation_cors
|
||||
**Name**: Security tests — alg-pin, rotation, CORS, no-leak (NFT-SEC-07..13)
|
||||
**Description**: Implement xUnit blackbox tests for the 7 cross-cutting security scenarios — health endpoint anonymous-OK (NFT-SEC-07), 500 redacted body shape (NFT-SEC-08), SQL-injection guard via parameterised queries (NFT-SEC-09), algorithm-pin defends against HS256-confusion and unsigned tokens (NFT-SEC-10), unknown-`kid` rotation lag with old-key grace window (NFT-SEC-11), startup fail-fast on missing required env vars + HTTPS-only JWKS URL (NFT-SEC-12), and CORS Production-gate fail-fast + permissive-default-warning in non-Production (NFT-SEC-13).
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-582
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
Six of these scenarios pin invariants that were broken in earlier code paths and structurally fixed during the 2026-05-14 drift cycle. NFT-SEC-10 (alg-pin) defends against the most common JWKS-public-key-as-HMAC-secret attack. NFT-SEC-11 (kid rotation) verifies that the test-infrastructure JWKS cache shortening (C01) actually shrinks rotation lag inside the 15-minute CI gate. NFT-SEC-12 verifies all four `Infrastructure/ConfigurationResolver.ResolveRequiredOrThrow` calls — `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`. NFT-SEC-13 verifies `CorsConfigurationValidator.EnsureSafeForEnvironment` actually throws on `ASPNETCORE_ENVIRONMENT=Production` with empty allow-list, AND falls back to permissive with a warning log in `Test`/`Development`. Each is a separate failure mode; together they form the "static config and cryptographic posture" surface that nothing else in the suite covers.
|
||||
|
||||
## Outcome
|
||||
|
||||
- All seven NFT-SEC-07..13 scenarios run and pass against the dockerised `missions` service.
|
||||
- Each test produces a CSV row with `Category=Sec`, `Traces=AC-5.x`/`AC-6.x`/`AC-7.x`/`AC-8.x`/`AC-9.x`/`AC-10.x`, `Result=pass`.
|
||||
- NFT-SEC-10 covers BOTH HS256-confusion (mock signs with the public key as HMAC secret) AND `alg: none` (mock emits unsigned JWT) — both must return `401`.
|
||||
- NFT-SEC-11 (rotation lag) completes inside 120s and exercises the three windows: cached-misses-new-kid → 401, cache-refreshed → 200, old-kid-still-valid-during-grace → 200, post-grace-old-kid → mock refuses to sign.
|
||||
- NFT-SEC-12 runs five separate `docker run` invocations (four missing-env + one HTTP-not-HTTPS JWKS URL); each asserts non-zero exit / log line.
|
||||
- NFT-SEC-13 runs five separate `docker run` invocations spanning Production-fail-fast, Production-AllowAny-warning, Production-with-origins, Production-cross-origin-rejection, Test-permissive-warning.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- NFT-SEC-07 Health endpoint anonymous + accepted with expired token (auth pipeline not evaluated).
|
||||
- NFT-SEC-08 500 redacted body — no `stack`/`stackTrace`/`exception`/`inner`/`trace`/file-path/type-name in body; log has the stack info.
|
||||
- NFT-SEC-09 SQL-injection guard — `?name=' OR '1'='1` and `?name=; DROP TABLE vehicles; --` are treated as literal strings.
|
||||
- NFT-SEC-10 Alg-pin — HS256-confusion AND unsigned token both rejected.
|
||||
- NFT-SEC-11 Unknown-kid rotation lag with old-key grace window.
|
||||
- NFT-SEC-12 Missing required env vars (4 vars) + HTTP-JWKS-URL warning path.
|
||||
- NFT-SEC-13 CORS Production-gate fail-fast + AllowAnyOrigin warning + explicit-origin preflight + cross-origin preflight rejection + non-Production permissive-default warning.
|
||||
|
||||
### Excluded
|
||||
|
||||
- The 401/403 auth pipeline (NFT-SEC-01..06 + 04b) lives in Task 14.
|
||||
- The destructive `DROP TABLE` mid-test for the 500 path (FT-N-08) lives in Task 13. NFT-SEC-08 here REUSES the same fixture but adds the response-body redaction assertions.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: NFT-SEC-07 health is anonymous and skips the auth pipeline**
|
||||
When `GET /health` is issued (a) with no `Authorization` header AND (b) with `Authorization: Bearer <expired token>`
|
||||
Then both responses are `200` with body `{ "status": "healthy" }` — proving the auth pipeline does not run for `/health`
|
||||
|
||||
**AC-2: NFT-SEC-08 500 redacted body**
|
||||
Given the same fixture as FT-N-08 (`DROP TABLE vehicles CASCADE`)
|
||||
When `GET /vehicles/{any uuid}` is issued
|
||||
Then response body is EXACTLY `{ "statusCode":500, "message":"Internal server error" }`, contains NO key matching `stack`/`stackTrace`/`exception`/`inner`/`trace`/file-path/exception-type-name
|
||||
And `docker logs missions-sut` contains an `Unhandled exception` line including the exception type or file path of the throw site
|
||||
|
||||
**AC-3: NFT-SEC-09 SQL-injection guard**
|
||||
Given a running stack with `seed_3_vehicles_2_default`
|
||||
When `GET /vehicles?name=' OR '1'='1` (URL-encoded) is issued
|
||||
Then response is `200` with `body.length == 0` (the literal string does not match any `Name`)
|
||||
And when `GET /missions?name=; DROP TABLE vehicles; --` (URL-encoded) is issued
|
||||
Then response is `200` with `body.TotalCount == 0` AND side-channel `SELECT to_regclass('vehicles')` returns a non-null oid (the table still exists)
|
||||
|
||||
**AC-4: NFT-SEC-10 algorithm-pin rejects HS256-confusion and unsigned**
|
||||
When the consumer mints a token via `POST /sign { alg_override: "HS256" }` (mock signs with the JWKS public key as HMAC secret)
|
||||
Then `GET /vehicles` returns `401`
|
||||
And when the consumer mints a token via `POST /sign { alg_override: "none" }` (unsigned JWT)
|
||||
Then `GET /vehicles` returns `401`
|
||||
|
||||
**AC-5: NFT-SEC-11 unknown-kid rotation completes within 120s with grace window honoured**
|
||||
Given `missions` has a warm JWKS cache and `jwks-mock` is configured with `OLD_KEY_GRACE_SECONDS=5`
|
||||
When the consumer issues `POST jwks-mock:8443/rotate-key {}`, immediately mints a token signed with the new kid, and calls `GET /vehicles` BEFORE missions has refreshed
|
||||
Then the first call returns `401` (new kid not yet in cache)
|
||||
And after waiting for the JWKS refresh window (≤ 90s; the mock sets `max-age=60` and missions has `JWT_JWKS_AUTO_REFRESH_INTERVAL_SECONDS=30` per C01), the same token returns `200`
|
||||
And during the 5s grace window, a token still signed with the OLD kid is accepted (`200`)
|
||||
And after the grace window expires, the mock refuses to sign with the old kid (`400`/`410` from `POST /sign`)
|
||||
|
||||
**AC-6: NFT-SEC-12 startup fail-fast on required env vars + HTTPS-only JWKS**
|
||||
When `missions` is launched via separate `docker run` invocations, each missing exactly one of `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL` (4 cases)
|
||||
Then in each case the container exits non-zero within 5s AND its logs contain `InvalidOperationException` mentioning the corresponding variable (or its `Database:Url`/`Jwt:Issuer`/`Jwt:Audience`/`Jwt:JwksUrl` config alias)
|
||||
And when `missions` is launched with `JWT_JWKS_URL=http://jwks-mock:8443/...` (HTTP not HTTPS) and the other three set
|
||||
Then the container STARTS, AND the first protected request fails (`500` body or `401` with `RequireHttps` mention) AND the log contains a line mentioning `HTTPS` / `RequireHttps`
|
||||
|
||||
**AC-7: NFT-SEC-13 CORS Production-gate fail-fast + non-Production warning**
|
||||
When `missions` is launched with `ASPNETCORE_ENVIRONMENT=Production` and no `CorsConfig` env vars
|
||||
Then the container exits non-zero within 5s AND its logs contain `InvalidOperationException` mentioning `CorsConfig`/`AllowedOrigins`/Production
|
||||
And when launched with `ASPNETCORE_ENVIRONMENT=Production` + `CorsConfig__AllowAnyOrigin=true`
|
||||
Then the container starts AND the logs contain a warning that CORS is permissive in Production
|
||||
And when launched with `ASPNETCORE_ENVIRONMENT=Production` + `CorsConfig__AllowedOrigins__0=https://operator.example.com`
|
||||
Then `OPTIONS /vehicles` preflight from `https://operator.example.com` returns `200` with `Access-Control-Allow-Origin: https://operator.example.com`
|
||||
And the same preflight from `https://attacker.example.com` responds without the allow-origin echo
|
||||
And when launched with `ASPNETCORE_ENVIRONMENT=Test` and no `CorsConfig`, the container starts AND the logs contain the documented `PermissiveDefaultWarning`
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- NFT-SEC-07..10: ≤ 5s each.
|
||||
- NFT-SEC-11: ≤ 120s (rotation + cache refresh).
|
||||
- NFT-SEC-12: ≤ 60s (5 docker-run cycles).
|
||||
- NFT-SEC-13: ≤ 90s (5 docker-run cycles + preflight requests).
|
||||
|
||||
**Reliability**
|
||||
- NFT-SEC-11 must run in its own xUnit `[Collection("JwksRotation")]` because rotating the mock affects every subsequent test that already has tokens in flight. After the test, the fixture restores the original key by calling `POST /rotate-key` once more and waits the grace window.
|
||||
- NFT-SEC-12 and NFT-SEC-13 spawn `docker run` from inside the test runner — the runner container must have access to a Docker socket OR the suite-level test orchestrator must run these as separate compose profiles. AZ-576 covers the runner-side Docker access.
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | running stack | `GET /health` no-auth and with expired token | both 200 | AC-7.1, AC-9.4 |
|
||||
| AC-2 | dropped `vehicles` table | `GET /vehicles/{any}` | 500 + body has only `statusCode,message` + log has stacktrace | AC-8.6, AC-10.3 |
|
||||
| AC-3 | `seed_3_vehicles_2_default` | `?name=' OR '1'='1` then `?name=; DROP TABLE…` | 200 + len 0 + table still exists | AC-1.6, AC-2.3 defensive |
|
||||
| AC-4 | mock with alg overrides | HS256-confusion token then unsigned token | both 401 | AC-5.1, AC-5.10 |
|
||||
| AC-5 | warm JWKS cache | `POST /rotate-key` + 3 timing checks | 401 → wait → 200; old-kid grace; post-grace mock refuses | AC-5.7 |
|
||||
| AC-6 | 5 docker-run cases | missing DATABASE_URL/JWT_ISSUER/JWT_AUDIENCE/JWT_JWKS_URL + HTTP-not-HTTPS | 4 fail-fast + 1 start-then-500 | AC-6.1, AC-6.2, E1, E3 |
|
||||
| AC-7 | 5 docker-run cases | Production fail-fast, AllowAnyOrigin warn, explicit-origin allow, cross-origin reject, Test permissive warn | per scenario | AC-6.11, E9 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080` for the cases that run inside the standard compose stack. NFT-SEC-12 and NFT-SEC-13 use `docker run` directly against `azaion/missions:test`.
|
||||
- NFT-SEC-09 second probe (`SELECT to_regclass('vehicles')`) requires side-channel Npgsql access AFTER the SUT response — if the table was dropped, the test was wrong.
|
||||
- NFT-SEC-11 fixture must restore the original key before exit (otherwise every test in subsequent collections fails with `kid` mismatch).
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` per test.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: NFT-SEC-10 false-pass if the mock cannot produce an HS256 token**
|
||||
- *Risk*: If the mock implementation rejects `alg_override="HS256"`, the test never exercises the attack — it gets `400` from the mock and incorrectly thinks `missions` rejected.
|
||||
- *Mitigation*: The test asserts a successful `200 OK` from `jwks-mock POST /sign` BEFORE issuing `GET /vehicles`; mock failure fails Arrange, not Assert.
|
||||
|
||||
**Risk 2: NFT-SEC-11 flake on slow CI**
|
||||
- *Risk*: The 60s `max-age` + 30s `AutoRefresh` + clock variance might push refresh past 120s on a heavily loaded runner.
|
||||
- *Mitigation*: The test polls every 5s for ≤ 120s; if no transition by 120s, fails with a clear "rotation not observed inside the budget" message. The 120s budget already includes margin per `environment.md` § CI gate.
|
||||
|
||||
**Risk 3: NFT-SEC-13 cross-origin preflight assertion misreads CORS header presence**
|
||||
- *Risk*: ASP.NET Core's CORS middleware returns `200` for OPTIONS even when origin is disallowed, just without the allow-origin header. A loose assertion would miss the rejection.
|
||||
- *Mitigation*: Test asserts `Access-Control-Allow-Origin` header EXACTLY: present and matching the allowed origin in the allow case; absent (header == null) in the reject case.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface and verify startup behaviour via `docker run` and `docker logs missions-sut` scrape. Expected outputs are compared against `_docs/00_problem/input_data/expected_results/results_report.md` rows AC-5 (NFT-SEC-10/11), AC-6 (NFT-SEC-12/13), AC-7 (NFT-SEC-07), AC-8 (NFT-SEC-08), AC-9 (NFT-SEC-07), AC-10 (NFT-SEC-08).
|
||||
- Stubs are allowed ONLY for: the external `admin` JWT issuer (`jwks-mock` container).
|
||||
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including `JwtExtensions`, `Program.cs` (config resolution + CORS + auth pipeline), `Infrastructure/ConfigurationResolver`, `Infrastructure/CorsConfigurationValidator`, or `ErrorHandlingMiddleware`. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
@@ -0,0 +1,115 @@
|
||||
# Resilience Tests — Cascade + Migrator
|
||||
|
||||
**Task**: AZ-583_test_resilience_cascade_migrator
|
||||
**Name**: Resilience tests — cascade + migrator (NFT-RES-01..04)
|
||||
**Description**: Implement xUnit blackbox tests for the 4 cascade and migrator resilience scenarios — mission cascade NOT transaction-wrapped (partial deletes survive mid-walk failure; AC-3.3 / ADR-006 carry-forward), waypoint cascade same invariant (AC-4.6), migrator idempotent on container restart (AC-6.6), and the B9 one-shot legacy table drop is destructive on first run + idempotent on subsequent restarts (AC-6.5, AC-10.5).
|
||||
**Complexity**: 3 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-583
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
The cascade tests encode TWO documented carry-forwards — the F3 (mission) and F4 (waypoint) cascades are NOT transaction-wrapped, so when the walk fails mid-way (e.g., `media` table absent), the rows deleted BEFORE the failure stay deleted while the rows deleted AFTER do not. This is documented under ADR-006 and AC-3.3 / AC-3.4 / AC-4.6 / AC-10.2 as deferred work. The tests intentionally pin the current behaviour so a future transaction-wrap change is caught loudly. The migrator tests pin two operational invariants needed for blue-green / restart-during-deploy patterns: NFT-RES-03 verifies a vanilla restart is a no-op, and NFT-RES-04 verifies the post-B9 `DROP TABLE IF EXISTS orthophotos/gps_corrections` block runs once and is idempotent thereafter.
|
||||
|
||||
## Outcome
|
||||
|
||||
- All four NFT-RES-01..04 scenarios run and pass against the dockerised `missions` service.
|
||||
- Each test produces a CSV row with `Category=Res`, `Traces=AC-3.3` / `AC-4.6` / `AC-6.6` / `AC-6.5`, `Result=pass`.
|
||||
- NFT-RES-01 and NFT-RES-02 assert BOTH the partial-state observation (some rows deleted, some not) AND the 500 response shape (envelope keys, no leak) — fail loudly when a future transaction wrap rolls everything back.
|
||||
- NFT-RES-03 asserts no NEW error log lines appear after the restart timestamp (not just "any error", which would conflate pre-existing startup-time warnings).
|
||||
- NFT-RES-04 includes a build-time / source-inspection gate so it only meaningfully runs on a post-B9 build (B9 landed locally 2026-05-15 — verified via `_docs/_process_leftovers/2026-05-14_rename-flights-to-missions.md`).
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- NFT-RES-01 Cascade NOT transaction-wrapped (mission, F3) — `DROP TABLE media CASCADE` before request; `500` response; `map_objects` count `0` (committed); `missions` count `1` (uncommitted).
|
||||
- NFT-RES-02 Cascade NOT transaction-wrapped (waypoint, F4) — same shape against F4 fixture.
|
||||
- NFT-RES-03 Idempotent migrator on restart — `docker compose restart missions`; no NEW error log lines; schema unchanged.
|
||||
- NFT-RES-04 B9 one-shot legacy drop — `seed_legacy_gps_tables` precondition; on first start `orthophotos` + `gps_corrections` are dropped; subsequent restart is no-op.
|
||||
|
||||
### Excluded
|
||||
|
||||
- NFT-RES-05 Required config missing → fail-fast (4 docker-run cases + DB-unreachable) lives in Task 17.
|
||||
- NFT-RES-06 DB does not exist (Npgsql 3D000) lives in Task 17.
|
||||
- NFT-RES-07 JWKS rotation lives in Task 17 (NOTE: also touched by NFT-SEC-11 in Task 15 from a security angle; this resilience variant focuses on the no-restart operational property).
|
||||
- NFT-RES-08 TOCTOU on default-vehicle exclusivity lives in Task 17.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: NFT-RES-01 mission cascade partial-state survives mid-walk failure**
|
||||
Given `fixture_cascade_F3` applied to a running stack
|
||||
When the side-channel executes `DROP TABLE media CASCADE` THEN the consumer issues `DELETE /missions/{mid}` with JWT `FL`
|
||||
Then the response is `500` with envelope `{ statusCode:500, message:"Internal server error" }`
|
||||
And side-channel `SELECT COUNT(*) FROM map_objects WHERE mission_id={mid}` returns `0` (committed before the failure)
|
||||
And side-channel `SELECT COUNT(*) FROM missions WHERE id={mid}` returns `1` (uncommitted after the failure)
|
||||
And `docker logs missions-sut` contains an `Unhandled exception` line mentioning `relation` and `media` within 2s of the request
|
||||
|
||||
**AC-2: NFT-RES-02 waypoint cascade partial-state same invariant**
|
||||
Given `fixture_cascade_F4` applied
|
||||
When the side-channel executes `DROP TABLE media CASCADE` THEN the consumer issues `DELETE /missions/{mid}/waypoints/{wp1}`
|
||||
Then the response is `500`
|
||||
And side-channel `SELECT COUNT(*) FROM detection WHERE annotation_id IN (wp1 chain)` returns `0`
|
||||
And side-channel `SELECT COUNT(*) FROM waypoints WHERE id={wp1}` returns `1`
|
||||
|
||||
**AC-3: NFT-RES-03 migrator is idempotent on restart**
|
||||
Given `missions` has been started once (schema migrated; `seed_empty` state)
|
||||
When `docker compose -f docker-compose.test.yml restart missions` is invoked AND health returns 200 within 30s
|
||||
Then `docker logs missions-sut` since the restart timestamp contains NO new lines matching `(error|Error|exception)`
|
||||
And the side-channel `\d+ vehicles` table description is unchanged from the post-first-start state
|
||||
|
||||
**AC-4: NFT-RES-04 B9 one-shot legacy drop is destructive then idempotent**
|
||||
Given `seed_legacy_gps_tables` (legacy `orthophotos` + `gps_corrections` present), `missions` not yet started for this scenario, AND the build is post-B9 (verified via `to_regclass` or source inspection of `DatabaseMigrator.cs`)
|
||||
When `docker compose up -d missions` is invoked and health returns 200
|
||||
Then side-channel `SELECT to_regclass('orthophotos'), to_regclass('gps_corrections')` returns both NULL (tables dropped)
|
||||
And when `docker compose restart missions` is invoked and health returns 200 again
|
||||
Then side-channel queries still return both NULL, AND `docker logs missions-sut` since the restart contains NO `does not exist` line (the `IF EXISTS` suppresses the no-op error)
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- NFT-RES-01..02: ≤ 10s each (cascade walk + fault injection setup).
|
||||
- NFT-RES-03..04: ≤ 60s each (container restart + health poll).
|
||||
|
||||
**Reliability**
|
||||
- NFT-RES-01 and NFT-RES-02 are destructive (drop `media` table); each runs in its own xUnit `[Collection("ResCascadeF3")]` / `[Collection("ResCascadeF4")]` with `ComposeRestartFixture` teardown (full `down -v && up -d`).
|
||||
- NFT-RES-04 has a build-time gate: the test queries the migrator source (or checks if the legacy tables exist after start) and SKIPS with a recorded reason on pre-B9 builds. Skipped rows appear in the CSV report with `Result=skip` and a clear `ErrorMessage` field.
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | `fixture_cascade_F3` + DROP `media` | `DELETE /missions/{mid}` | 500 + map_objects=0 + missions=1 + log mentions `media` | AC-3.3, AC-10.2 |
|
||||
| AC-2 | `fixture_cascade_F4` + DROP `media` | `DELETE /missions/{mid}/waypoints/{wp1}` | 500 + detection=0 + wp1=1 | AC-4.6, AC-3.3 |
|
||||
| AC-3 | post-first-start `seed_empty` | `docker compose restart missions` | health back in 30s + no new error logs + schema unchanged | AC-6.6, AC-6.4 |
|
||||
| AC-4 | `seed_legacy_gps_tables` + post-B9 build | first start + restart | first drops legacy tables; restart is no-op (no error log) | AC-6.5, AC-10.5 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080` for the cascade requests; side-channel Npgsql for fixture state + post-state assertions.
|
||||
- NFT-RES-01..02 use the same `fixture_cascade_F3.sql` / `fixture_cascade_F4.sql` from Tasks 11/12; do NOT re-author seed SQL.
|
||||
- NFT-RES-03..04 use `docker compose` from inside the runner (Docker-socket-mounted) OR from the suite orchestrator — AZ-576 covers this.
|
||||
- NFT-RES-04 must verify B9 has landed before running; otherwise SKIP with a clear reason (record in CSV).
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` per test.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: NFT-RES-01/02 false-pass when transaction wrap lands**
|
||||
- *Risk*: A future ADR-006 closure wraps the cascade in a transaction; `map_objects` count becomes `> 0` (rolled back) and `missions` count stays `1`. The test would interpret this as a failure of the partial-state invariant — but that failure means the system is BETTER.
|
||||
- *Mitigation*: Both tests include a source-level comment `// CARRY-FORWARD: AC-3.3 / ADR-006 — flip assertions when transaction wrap lands` and `[Trait("carry_forward","ADR-006")]` so a future filter finds them.
|
||||
|
||||
**Risk 2: NFT-RES-03 false-pass when restart-time errors are tolerated**
|
||||
- *Risk*: A simple `docker logs | grep -i error` over the entire log returns the migrator's pre-existing warnings.
|
||||
- *Mitigation*: The test captures `docker logs missions-sut --since=<restart_timestamp>` and greps from THAT slice only.
|
||||
|
||||
**Risk 3: NFT-RES-04 incorrectly runs on a pre-B9 build**
|
||||
- *Risk*: If the build-time gate is silently bypassed, the test asserts dropping the legacy tables — which would never happen, and the test would fail with a misleading message.
|
||||
- *Mitigation*: The gate checks BOTH the migrator source for the `DROP TABLE IF EXISTS orthophotos` line AND verifies the legacy tables are present in the seed BEFORE the SUT starts. If either check fails, the test SKIPS with `Result=skip` and a clear `ErrorMessage`.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface plus container orchestration (`docker compose restart`, `docker compose up -d`) and `docker logs missions-sut` scrape. Side-channel Npgsql for fixture state and post-state assertions. Expected outputs are compared against `_docs/00_problem/input_data/expected_results/results_report.md` rows AC-3 3.3, AC-4 4.6, AC-6 6.4-6.6, AC-10 10.2/10.5.
|
||||
- Stubs are allowed ONLY for: the external `admin` JWT issuer (`jwks-mock` container) and the DB-only stub tables for `media`, `annotations`, `detection`, `map_objects` (seeded via side-channel SQL).
|
||||
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including `MissionService`, `WaypointService`, `MissionsController`, `Database/DatabaseMigrator`, `ErrorHandlingMiddleware`, or `AppDataConnection`. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
@@ -0,0 +1,120 @@
|
||||
# Resilience Tests — Config / DB / JWKS Rotation / TOCTOU Race
|
||||
|
||||
**Task**: AZ-584_test_resilience_config_db_rotation_race
|
||||
**Name**: Resilience tests — config / DB / rotation / race (NFT-RES-05..08)
|
||||
**Description**: Implement xUnit blackbox tests for the 4 resilience scenarios — startup fail-fast on missing required config (6 docker-run cases including the DB-unreachable differentiator), database missing → Npgsql 3D000 process exit, JWKS rotation propagates without `missions` restart, and TOCTOU race on default-vehicle exclusivity (probabilistic, expected to produce `default_count ≥ 2` in at least one iteration).
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-584
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
These four scenarios pin the documented operational and concurrency posture of the service in places nothing else covers. NFT-RES-05 verifies BOTH the new fail-fast resolver path (rows 1–5: missing env vars throw `InvalidOperationException` BEFORE the HTTP server binds) AND the DB-down differentiator (row 6: config resolution succeeds, then Npgsql throws a recognisable connection error). NFT-RES-06 verifies the "database does not exist" case is observably different from "DB host unreachable" — Postgres returns SQLSTATE `3D000` and the container exits non-zero within 30s. NFT-RES-07 is the operational counterpart to NFT-SEC-11 — same JWKS rotation flow, but asserts the no-restart property (`docker inspect StartedAt` unchanged) instead of the kid-cache mechanics. NFT-RES-08 is intentionally probabilistic: it asserts the documented AC-1.4 race window EXISTS by running 100 parallel concurrent INSERTs and verifying that at least one iteration produces `is_default=true count ≥ 2`.
|
||||
|
||||
## Outcome
|
||||
|
||||
- All four NFT-RES-05..08 scenarios run and pass against the dockerised `missions` service.
|
||||
- Each test produces a CSV row with `Category=Res`, `Traces=AC-6.1..2/AC-6.7..8/AC-5.7/AC-1.4`, `Result=pass`.
|
||||
- NFT-RES-05 covers 6 cases — 4 missing-env (rows 1–4), 1 whitespace-only (`JWT_ISSUER=""`), and 1 DB-down-after-config-resolution (row 6 with `Connection refused`).
|
||||
- NFT-RES-06 asserts the Postgres error code `3D000` appears in the container logs and the container exit code is non-zero within 30s.
|
||||
- NFT-RES-07 asserts `docker inspect --format '{{.State.StartedAt}}' missions-sut` returns the SAME value before and after the rotation flow — the service did NOT restart.
|
||||
- NFT-RES-08 records the observed `default_count ≥ 2` iteration count and includes `[Trait("Stability","probabilistic")]` so CI tolerates ≤ 1 failed run per 5. If 0 iterations produce the race, the test FAILS with a clear "race window closed — update AC-1.4 and rewrite this test" message.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- NFT-RES-05 6 docker-run cases (4 missing-env + 1 whitespace + 1 DB-down differentiator).
|
||||
- NFT-RES-06 `DROP DATABASE azaion` → `docker compose up -d missions` → assert non-zero exit + `3D000` in logs.
|
||||
- NFT-RES-07 JWKS rotation flow — `T1` works pre-rotation; `T2` rejected pre-cache-refresh; `T2` accepted post-refresh; `T1` eventually rejected post-grace; `missions` startup timestamp unchanged.
|
||||
- NFT-RES-08 100 parallel `(POST /vehicles { IsDefault:true } || side-channel INSERT (..., is_default=true))` iterations; at least one produces `default_count ≥ 2`.
|
||||
|
||||
### Excluded
|
||||
|
||||
- NFT-SEC-11 (security-shaped variant of JWKS rotation) lives in Task 15.
|
||||
- NFT-SEC-12 (security-shaped variant of startup fail-fast) lives in Task 15. NOTE: NFT-RES-05 and NFT-SEC-12 share 4 of 5 docker-run cases — the test infrastructure (AZ-576) provides a shared `MissionsContainerHelper` so both tasks can reuse the same docker-run primitive without duplicating logic.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: NFT-RES-05 startup fail-fast on missing required config + DB-down differentiator**
|
||||
When `missions` is launched via 6 separate `docker run` invocations:
|
||||
- (1) all 4 required env vars unset
|
||||
- (2) `DATABASE_URL` unset, JWT vars set
|
||||
- (3) `JWT_ISSUER=""` (whitespace-only), others set
|
||||
- (4) `JWT_AUDIENCE` unset, others set
|
||||
- (5) `JWT_JWKS_URL` unset, others set
|
||||
- (6) all 4 vars set correctly, BUT `postgres-test` is stopped before `missions` starts
|
||||
Then rows 1–5 → container exits non-zero within 5s, logs contain `InvalidOperationException`, logs mention the corresponding key (or its config alias)
|
||||
And row 6 → container exits non-zero within 30s, logs contain a Npgsql `Connection refused` line (NOT an `InvalidOperationException` — proving config resolution succeeded BEFORE DB-connect failed)
|
||||
|
||||
**AC-2: NFT-RES-06 database missing → process exits with Npgsql 3D000**
|
||||
Given `postgres-test` running with the `azaion` database NOT yet created (or just dropped via side-channel)
|
||||
When `docker compose -f docker-compose.test.yml up -d missions` is invoked
|
||||
Then the container exits non-zero within 30s AND `docker logs missions-sut` contains at least one line matching `3D000`
|
||||
|
||||
**AC-3: NFT-RES-07 JWKS rotation propagates without missions restart**
|
||||
Given `missions` running with a warm JWKS cache, `jwks-mock` running with `OLD_KEY_GRACE_SECONDS=5` and `Cache-Control: max-age=60`, and Token `T1` minted with the current kid `kid_v1`
|
||||
When `GET /vehicles` is issued with `T1`
|
||||
Then response is `200`
|
||||
And when `POST jwks-mock:8443/rotate-key {}` is invoked, `T2` is minted with `kid_v2`, and `GET /vehicles` is issued with `T2` BEFORE the JWKS cache refresh
|
||||
Then response is `401`
|
||||
And after waiting up to 90s for cache refresh (mock `max-age=60` + service `JWT_JWKS_AUTO_REFRESH_INTERVAL_SECONDS=30`), `GET /vehicles` with the same `T2` returns `200`
|
||||
And `GET /vehicles` with `T1` (still has unexpired lifetime) returns `401` AFTER the grace window expires
|
||||
And `docker inspect --format '{{.State.StartedAt}}' missions-sut` returns the SAME ISO-8601 timestamp before and after the entire rotation flow (the service did NOT restart)
|
||||
|
||||
**AC-4: NFT-RES-08 TOCTOU race produces default_count ≥ 2 in at least one iteration**
|
||||
Given `seed_one_default_vehicle` (default `P1`)
|
||||
When the test runs 100 concurrent iterations, each issuing `POST /vehicles { IsDefault:true }` to the API in parallel with a side-channel `INSERT INTO vehicles (..., is_default=true)`
|
||||
Then after all iterations complete, at least one iteration's post-state shows `SELECT COUNT(*) FROM vehicles WHERE is_default=true ≥ 2`
|
||||
And if 0 iterations produce the race, the test FAILS with `"race window closed — update AC-1.4 carry-forward and rewrite this test"` (this is a structural test failure, not a flake)
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- NFT-RES-05: ≤ 180s (6 docker-run cycles).
|
||||
- NFT-RES-06: ≤ 60s (DROP DATABASE + docker-run + exit poll).
|
||||
- NFT-RES-07: ≤ 180s (JWKS cache refresh window).
|
||||
- NFT-RES-08: ≤ 30s (100 parallel iterations).
|
||||
|
||||
**Reliability**
|
||||
- NFT-RES-07 fixture MUST restore the original key by calling `POST /rotate-key` again at the end AND wait the grace window before yielding control — otherwise every subsequent test runs against an unfamiliar kid.
|
||||
- NFT-RES-08 is probabilistic: `[Trait("Stability","probabilistic")]`. CI tolerates ≤ 1 failed run per 5 — but the structural failure mode ("race never observed in any iteration") still fails the suite. A deterministic-via-advisory-lock follow-up is recorded as a Refactor Backlog item.
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | `missions` not running | 6 docker-run cases | 5 fail-fast (InvalidOperationException) + 1 DB-down (Connection refused) | AC-6.1, AC-6.2, AC-6.7, E3, E4 |
|
||||
| AC-2 | `DROP DATABASE azaion` | `docker compose up -d missions` | exit non-zero in 30s + log has `3D000` | AC-6.8 |
|
||||
| AC-3 | warm JWKS cache + mock with grace=5/max-age=60 | rotate + 3 timing probes | T1→200; T2→401→wait→200; T1→401 post-grace; StartedAt unchanged | AC-5.7 |
|
||||
| AC-4 | `seed_one_default_vehicle` | 100 parallel (POST + side-channel INSERT) | ≥ 1 iteration shows default_count ≥ 2 | AC-1.4 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080` for the runtime cases; `docker run` and `docker compose` for the startup/DB cases.
|
||||
- NFT-RES-05 row 6 (DB-down differentiator) is critical: the test must assert the log is `Connection refused`-shaped, NOT an `InvalidOperationException`. This rules out a regression where the resolver silently accepts an empty DB URL.
|
||||
- NFT-RES-07 must clean up: rotate back to the original key in teardown AND wait `OLD_KEY_GRACE_SECONDS` so subsequent tests do not encounter a stale-kid edge case.
|
||||
- NFT-RES-08 records the per-iteration timing and observed counts to the CSV report's `Traces` field for diagnosis.
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` per test.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: NFT-RES-05 row 6 false-pass when config resolution silently accepts empty `DATABASE_URL`**
|
||||
- *Risk*: A regression that returns an empty default for `DATABASE_URL` would make rows 2/6 indistinguishable — both would log a Npgsql error, but row 2 should log `InvalidOperationException` first.
|
||||
- *Mitigation*: Test asserts row 2 logs the `InvalidOperationException` BEFORE any Npgsql output; row 6 logs Npgsql `Connection refused` directly without `InvalidOperationException`. Failure of either differentiator fails the test.
|
||||
|
||||
**Risk 2: NFT-RES-07 flake on slow CI**
|
||||
- *Risk*: Same as NFT-SEC-11 — slow refresh window.
|
||||
- *Mitigation*: Same — poll every 5s for ≤ 90s; fail clearly if no transition observed in budget.
|
||||
|
||||
**Risk 3: NFT-RES-08 deterministic-pass when race window closes**
|
||||
- *Risk*: If a future TOCTOU fix lands (e.g., adding a `UNIQUE WHERE is_default=true` constraint), the test's "race observed" assertion fails — but the system is BETTER.
|
||||
- *Mitigation*: Test failure message includes `"race window closed — update AC-1.4 carry-forward and rewrite this test"` so a future engineer knows the failure is expected and what to do. The test is gated by `[Trait("carry_forward","AC-1.4")]`.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface plus `docker run`, `docker compose`, `docker inspect`, and `docker logs missions-sut` scrape. Side-channel Npgsql for fixture state, post-state assertions, and concurrent INSERTs. JWKS rotation via `POST https://jwks-mock:8443/rotate-key`. Expected outputs are compared against `_docs/00_problem/input_data/expected_results/results_report.md` rows AC-1 1.4, AC-5 5.7, AC-6 6.1/6.2/6.7/6.8, E3/E4.
|
||||
- Stubs are allowed ONLY for: the external `admin` JWT issuer (`jwks-mock` container).
|
||||
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including `JwtExtensions`, `Program.cs`, `Infrastructure/ConfigurationResolver`, `Database/AppDataConnection`, `Database/DatabaseMigrator`, `Services/VehicleService` (for the TOCTOU race), or `Auth/JwtExtensions`. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
Reference in New Issue
Block a user