azaion/missions
1
0
Fork 0
mirror of https://github.com/azaion/missions.git synced 2026-06-21 16:31:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-576] Add e2e test infrastructure (xUnit + jwks-mock + reporting)
ci/woodpecker/push/build-arm Pipeline failed
Details

Browse Source 
Scaffold the blackbox test project the rest of epic AZ-575 (AZ-577..AZ-586)
will build on. Two new csprojs under tests/, plus the TLS materials and
TRX->CSV reporting hand-off the existing docker-compose.test.yml already
calls for.

JWKS mock (tests/Azaion.Missions.JwksMock/):
- ASP.NET Core minimal API on .NET 10, no NuGet deps; JWS is hand-rolled
  to keep the surface tight and avoid version drift with the SUT
- KeyStore with one in-memory ECDSA P-256 keypair + retired-key grace
  window for NFT-RES-07 / NFT-SEC-11 rotation observability
- Endpoints: GET /.well-known/jwks.json, POST /sign, POST /rotate-key
- Mock-only alg_override / kid_override switches drive NFT-SEC-09/10/11
- TLS keypair committed under tls/; tests/jwks-mock-ca.crt is a copy
  mounted into both missions and e2e-consumer per docker-compose.test.yml

E2E consumer (tests/Azaion.Missions.E2E.Tests/):
- xUnit 2.9.2 + Bogus 35.6.1 + Npgsql 10.0.2 + Xunit.SkippableFact 1.4.13
- TestBase / TokenMinter scaffolding for downstream tasks
- Fixtures/ for DbReset, DbSeed, ComposeRestart, JwksRotate, JwksMockReverse
- Helpers/ for DbAssertions (side-channel), HttpAssertions, FixtureSql
- 8 Tests/<category>/Sanity.cs discovery smoke tests (AC-3)
- Tests/InfrastructureSanity.cs SkippableFacts for AC-1/2/5/6
- Tests/AaaPatternEnforcement.cs greps source files for AC-7
- Tests/Reporting/TrxToCsvPostProcessorTests.cs covers AC-4
- Reporting/TrxToCsvPostProcessor.cs handles VSTest TRX -> environment.md
  CSV; xUnit traits are not propagated by the TRX logger so the converter
  reflects them out of the test DLL via GetCustomAttributesData
- Reporting.Cli/ is a separate console csproj that links the converter
  source files (test project excludes Reporting.Cli/** from compile)
- Dockerfile + entrypoint.sh wire dotnet test -> trx -> csv inside the
  e2e-consumer container the compose file already references

Local verification: 13 pass, 3 skip (with explicit reasons), 0 fail.
End-to-end TRX->CSV manually verified against environment.md header spec.
Docker stack build is handed off to autodev Step 7 (test-run skill).

Reports under _docs/03_implementation/.
AZ-576 task spec moved to _docs/tasks/done/.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-15 06:57:40 +03:00
parent b0c7132889
commit ccd85a09df
48 changed files with 2030 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/tasks/todo/AZ-576_test_infrastructure.md
-227
View File
@@ -1,227 +0,0 @@
# Test Infrastructure
**Task**: AZ-576_test_infrastructure
**Name**: Test Infrastructure (Missions e2e)
**Description**: Scaffold the Blackbox test project — xUnit runner, JWKS mock service, Docker test environment wiring, test data fixtures, reporting. Compose file already exists at repo root and references not-yet-built build contexts; this task fills in those contexts.
**Complexity**: 5 points
**Dependencies**: None (C01 + C02 testability refactor already landed; see `_docs/04_refactoring/01-testability-refactoring/testability_changes_summary.md`)
**Component**: Blackbox Tests
**Tracker**: AZ-576
**Epic**: AZ-575
## Scope
Two artifacts that the existing `docker-compose.test.yml` references but does not yet build, plus the run script the suite expects:
1. `tests/Azaion.Missions.JwksMock/` — minimal HTTPS service holding an ECDSA P-256 keypair in memory, serving JWKS + `POST /sign` + `POST /rotate-key`. Image tag `azaion/jwks-mock:test`.
2. `tests/Azaion.Missions.E2E.Tests/` — xUnit 2.x test project that drives the running `missions` service over HTTP, mints tokens via `https://jwks-mock:8443/sign`, asserts DB side-effects through a side-channel Npgsql connection, and produces `report.csv`.
3. `tests/jwks-mock-ca.crt` — the self-signed CA cert that both `missions` and `e2e-consumer` mount and `update-ca-certificates --fresh` adds to the OS trust bundle (per `docker-entrypoint.sh` from C02).
4. `scripts/run-tests.sh` — wraps `docker compose -f docker-compose.test.yml up --build --abort-on-container-exit e2e-consumer`, collects `report.csv`, then `down -v`.
5. `scripts/run-performance-tests.sh` — same compose stack with the `[Trait("Category","Perf")]` filter and the perf seed.
The `missions` and `postgres-test` services already exist in `docker-compose.test.yml`; the `jwks-mock` and `e2e-consumer` services exist but point at build contexts that this task creates.
## Test Project Folder Layout
```
tests/
├── jwks-mock-ca.crt # self-signed CA (mounted into missions + e2e-consumer)
├── Azaion.Missions.JwksMock/
│ ├── Azaion.Missions.JwksMock.csproj
│ ├── Dockerfile # builds azaion/jwks-mock:test, exposes 8443/tcp
│ ├── Program.cs # ASP.NET Core minimal API
│ ├── Endpoints/
│ │ ├── JwksEndpoint.cs # GET /.well-known/jwks.json
│ │ ├── SignEndpoint.cs # POST /sign
│ │ └── RotateKeyEndpoint.cs # POST /rotate-key
│ ├── Services/
│ │ ├── KeyStore.cs # in-memory ECDSA P-256 keypair + old-key grace window
│ │ └── TokenSigner.cs # ECDSA signing with alg_override/kid_override support
│ └── appsettings.json # JWT_ISSUER, JWT_AUDIENCE, OLD_KEY_GRACE_SECONDS
└── Azaion.Missions.E2E.Tests/
├── Azaion.Missions.E2E.Tests.csproj # xUnit 2.x + Bogus 35.x + Npgsql 10.x
├── Dockerfile # runs `dotnet test --logger trx` + trx→csv post-step
├── TestBase.cs # HttpClient factory, default JWT, shared MissionsBaseUrl
├── TokenMinter.cs # POST jwks-mock:8443/sign with overrides
├── Fixtures/
│ ├── DbResetFixture.cs # IClassFixture<>: TRUNCATE between classes
│ ├── DbSeedFixture.cs # base for the named seed sets in test-data.md
│ ├── ComposeRestartFixture.cs # docker compose down -v && up -d for bootstrap-sensitive tests
│ └── JwksRotateFixture.cs # POST /rotate-key + wait for missions to refresh JWKS cache
├── Helpers/
│ ├── DbAssertions.cs # Npgsql side-channel asserts (row counts, default-vehicle invariants)
│ ├── HttpAssertions.cs # PascalCase shape, error-envelope shape, ordering, pagination
│ └── FixtureSql.cs # loads fixture_cascade_F3.sql / fixture_cascade_F4.sql
├── Tests/
│ ├── Vehicles/ # FT-P-01..06, FT-N-01..03
│ ├── Missions/ # FT-P-07..12, FT-N-04..06
│ ├── Waypoints/ # FT-P-13..15, FT-P-18, FT-N-07
│ ├── Health/ # FT-P-16..17, FT-N-08
│ ├── Security/ # NFT-SEC-01..13, 04b
│ ├── Resilience/ # NFT-RES-01..08
│ ├── ResourceLimits/ # NFT-RES-LIM-01..04
│ └── Performance/ # NFT-PERF-01..04
└── Reporting/
├── TrxToCsvPostProcessor.cs # produces /app/results/report.csv per environment.md § Reporting
└── ResultRow.cs # TestId, TestName, Category, Traces, ExecutionTimeMs, Result, ErrorMessage
```
### Layout Rationale
- **Per-feature test folders** (`Vehicles/`, `Missions/`, etc.) match the natural decomposition surface in `blackbox-tests.md` and let `dotnet test --filter` slice the suite per Jira child ticket.
- **`Fixtures/` separate from `Tests/`** so xUnit `IClassFixture<>` lifetime is explicit (class-scoped DB reset) and not entangled with test cases.
- **`Helpers/` named for the assertion family** (DB, HTTP, FixtureSql) so each test reads as a single `// Arrange` + `// Act` + `// Assert` block per `coderule.mdc`.
- **JwksMock is a SEPARATE csproj**, not nested inside the e2e tests, because the build context is mounted as a service in `docker-compose.test.yml` (`tests/Azaion.Missions.JwksMock/`). Sharing a project would force the e2e runner to ship JWKS code into its container.
- **CA cert lives at `tests/jwks-mock-ca.crt`** rather than inside a project so both consumers (missions + e2e-consumer) mount the same file. The cert is regenerated only when the keypair changes — committed to the repo for deterministic test runs.
## Mock Services
| Mock Service | Replaces | Endpoints | Behavior |
|-------------|----------|-----------|----------|
| `jwks-mock` | `admin` JWT issuer + JWKS endpoint | `GET https://jwks-mock:8443/.well-known/jwks.json`; `POST https://jwks-mock:8443/sign`; `POST https://jwks-mock:8443/rotate-key` | Holds one ECDSA P-256 keypair in memory; serves the public half as JWKS with `Cache-Control: public, max-age=60`; signs ECDSA-SHA256 JWTs on `/sign` honoring optional `iss`/`aud`/`exp_offset_seconds`/`permissions`/`alg_override`/`kid_override`; rotates keypair on `/rotate-key` while retaining the old public key for `OLD_KEY_GRACE_SECONDS` (5s in tests). Private key never leaves the container. |
DB-only stubs (no service running, side-channel SQL inserts only): `annotations`, `detection`, `media`, `map_objects` — see `_docs/02_document/tests/test-data.md` § External Dependency Mocks.
### Mock Control API
`jwks-mock` exposes `POST /sign` and `POST /rotate-key` as its full control surface. The `/sign` body shape is documented in `test-data.md` § "JWKS mock token-minting contract":
```http
POST https://jwks-mock:8443/sign
{
"iss": "https://admin-test.azaion.local", # optional
"aud": "azaion-edge", # optional
"exp_offset_seconds": 3600, # optional; negative for expired
"permissions": "FL", # optional; "" / "ADMIN" / "fl" / "FLight" for claim-mismatch
"alg_override": null, # "HS256" to test alg-confusion (NFT-SEC-10)
"kid_override": null # non-existent kid for unknown-key tests (NFT-SEC-11)
}
```
Response: `{ "token": "<encoded JWT>", "kid": "<key id>" }`.
## Docker Test Environment
### docker-compose.test.yml Structure
| Service | Image / Build | Purpose | Depends On |
|---------|--------------|---------|------------|
| `postgres-test` | `postgres:16-alpine` | Owned test PostgreSQL; `tmpfs:/var/lib/postgresql/data` for `down -v` isolation | — |
| `jwks-mock` | build `tests/Azaion.Missions.JwksMock/` → `azaion/jwks-mock:test` | Mock JWKS issuer | — |
| `missions` | build `.` (repo root `Dockerfile`) → `azaion/missions:test` | System under test | `postgres-test` (healthy), `jwks-mock` (healthy) |
| `e2e-consumer` | build `tests/Azaion.Missions.E2E.Tests/` | xUnit runner; emits `report.csv` to host-mounted `./test-results/` | `missions` (healthy), `jwks-mock` (healthy) |
The compose file is already authored at the repo root. This task does NOT modify it — the file IS the contract; the task fills in the two missing build contexts so the references resolve.
### Networks and Volumes
| Resource | Purpose |
|----------|---------|
| `e2e-net` (bridge) | Isolated test network; no host network access. All four services attach. |
| `tmpfs:/var/lib/postgresql/data` | Ephemeral PG data; recreated per `docker compose down -v`. |
| `./test-results:/app/results` | `e2e-consumer` mounts this for `report.csv` output to the host. |
| `./tests/jwks-mock-ca.crt:/usr/local/share/ca-certificates/jwks-mock-ca.crt:ro` | Mounted into `missions` AND `e2e-consumer` so both trust the mock's HTTPS cert after `update-ca-certificates --fresh` runs in `docker-entrypoint.sh`. |
## Test Runner Configuration
**Framework**: xUnit 2.x
**Plugins**: `Microsoft.NET.Test.Sdk`, `xunit.runner.visualstudio`, `Bogus 35.x` (synthetic data), `Npgsql 10.x` (side-channel only — NO `Azaion.Missions.*` project reference)
**Entry point**: `dotnet test tests/Azaion.Missions.E2E.Tests/Azaion.Missions.E2E.Tests.csproj --logger "trx;LogFileName=results.trx"` followed by `TrxToCsvPostProcessor` converting `results.trx` → `report.csv`
**AAA convention**: every test method has `// Arrange` / `// Act` / `// Assert` comments per `.cursor/rules/coderule.mdc`.
### Fixture Strategy
| Fixture | Scope | Purpose |
|---------|-------|---------|
| `DbResetFixture` | Class (`IClassFixture<>`) | `TRUNCATE TABLE` for all schema tables between classes; cheap reset for read-path tests (AC-1, AC-2, AC-4) |
| `DbSeedFixture<TSeed>` | Class | Applies the named seed sets from `test-data.md` (`seed_empty`, `seed_one_default_vehicle`, `seed_3_vehicles_2_default`, `seed_25_missions`, `fixture_cascade_F3`, `fixture_cascade_F4`, `seed_5_waypoints_unordered`, `seed_legacy_gps_tables`) via Npgsql side-channel |
| `ComposeRestartFixture` | Collection | `docker compose -f docker-compose.test.yml down -v && up -d` between scenarios that assert startup-time behavior (AC-6.3..6.7, AC-5.7) |
| `JwksRotateFixture` | Scenario | `POST jwks-mock:8443/rotate-key` then waits for missions to refresh its JWKS cache (≤ 30s in tests, capped by `JWT_JWKS_AUTO_REFRESH_INTERVAL_SECONDS`) |
| `JwksMockReverseFixture` | Scenario | Boots `missions` outside compose via `docker run` with `ASPNETCORE_ENVIRONMENT=Production` + empty `CorsConfig:AllowedOrigins` to test E9 lock (NFT-SEC-13) |
### xUnit traits
Every test method MUST set `[Trait("Category", "Blackbox" | "Sec" | "Res" | "ResLim" | "Perf")]`. The CSV `Category` column reads from this trait. Traceability IDs go into a second `[Trait("Traces", "AC-1.2,AC-1.4")]` trait, comma-separated.
## Test Data Fixtures
Loaded entirely from `_docs/02_document/tests/test-data.md` § Seed Data Sets. The fixtures bind the named seeds to the AC IDs that consume them:
| Data Set | Source | Format | Used By |
|----------|--------|--------|---------|
| `seed_empty` | `down -v` + `missions` startup migrator | Schema only, no rows | bootstrap, unauth, 404 scenarios |
| `seed_one_default_vehicle` | Side-channel `INSERT INTO vehicles ...` | Inline SQL string | AC-1.2 default-clear, AC-1.3 TOCTOU, AC-1.4 setDefault, AC-2.1 mission-create |
| `seed_3_vehicles_2_default` | Side-channel SQL | Inline | AC-1.5 list, AC-1.6 filter |
| `seed_25_missions` | Side-channel SQL with deterministic UUIDs | Inline | AC-2.3..2.5 pagination + date filter |
| `fixture_cascade_F3` | `_docs/00_problem/input_data/expected_results/fixture_cascade_F3.sql` | SQL file | AC-3.1, 3.3, 3.4, 10.2 |
| `fixture_cascade_F4` | `_docs/00_problem/input_data/expected_results/fixture_cascade_F4.sql` | SQL file | AC-4.5, 4.6 |
| `seed_5_waypoints_unordered` | Side-channel SQL with `order_num [3,1,2,5,4]` | Inline | AC-4.3 unpaginated ordering |
| `seed_legacy_gps_tables` | `CREATE TABLE orthophotos / gps_corrections` + `INSERT` | Inline | AC-3.5 absence, AC-6.5 one-shot drop, AC-10.5 legacy migration |
### Data Isolation
Three tiers, by scenario type (per `test-data.md` § Data Isolation Strategy):
- **Class-scoped DB reset** (`IClassFixture<DbResetFixture>`): for scenarios that share a seed within a class but must not leak across classes. Used for AC-1, AC-2, AC-4 read paths.
- **Scenario-scoped container restart** (`docker compose down -v && up -d`): for scenarios that assert startup-time behavior or migrator side-effects (AC-6.3..6.7, AC-6.11, AC-5.7).
- **No per-test transaction rollback** — the system under test is a separate process; its `DataConnection` is not in the test transaction.
## Test Reporting
**Format**: CSV
**Columns**: `TestId, TestName, Category, Traces, ExecutionTimeMs, Result, ErrorMessage`
**Output path**: `/app/results/report.csv` inside `e2e-consumer`, mounted to `./test-results/report.csv` on the host
**Source**: post-processor reads `results.trx` (xUnit logger output), joins each test's `[Trait("Category",...)]` and `[Trait("Traces",...)]` into the CSV columns. `Result` is `pass` / `fail` / `skip`. `ErrorMessage` is the first line of the failure message (CRs stripped).
## Acceptance Criteria
**AC-1: Test environment starts**
Given the `docker-compose.test.yml` at repo root
When `docker compose -f docker-compose.test.yml up --build` runs
Then `postgres-test`, `jwks-mock`, and `missions` all reach `healthy`, and `e2e-consumer` starts after them
**AC-2: Mock JWKS service responds**
Given the test environment is running
When `GET https://jwks-mock:8443/.well-known/jwks.json` is issued from inside `e2e-net`
Then the response is `200 OK` with a JWKS body containing exactly one ECDSA P-256 public key
And `POST https://jwks-mock:8443/sign` with body `{}` returns a valid ECDSA-SHA256 JWT whose `iss` / `aud` match the mock's env vars
**AC-3: Test runner executes**
Given the test environment is running
When `e2e-consumer` starts and `dotnet test` runs
Then the runner discovers ≥ 1 test in each of the eight test folders (`Vehicles/`, `Missions/`, `Waypoints/`, `Health/`, `Security/`, `Resilience/`, `ResourceLimits/`, `Performance/`)
**AC-4: Test report generated**
Given tests have been executed
When `e2e-consumer` exits
Then `./test-results/report.csv` exists on the host
And the first line is the documented column header `TestId,TestName,Category,Traces,ExecutionTimeMs,Result,ErrorMessage`
And every executed test has exactly one CSV row
**AC-5: CA trust works end-to-end**
Given `tests/jwks-mock-ca.crt` is mounted into both `missions` and `e2e-consumer`
When `docker-entrypoint.sh` runs `update-ca-certificates --fresh` and `missions` issues `GET https://jwks-mock:8443/.well-known/jwks.json` to populate its JWKS cache
Then the TLS handshake succeeds (no `RemoteCertificateNotAvailable` / `RemoteCertificateNameMismatch`)
And the cached JWKS contains the public key the consumer-issued tokens are signed with
**AC-6: JWKS rotation observable inside the 15-minute CI gate**
Given the test compose sets `JWT_JWKS_AUTO_REFRESH_INTERVAL_SECONDS=30` and `JWT_JWKS_REFRESH_INTERVAL_SECONDS=10` (per C01)
When `POST https://jwks-mock:8443/rotate-key` is called
Then within 30s `missions` refreshes its JWKS cache and accepts tokens signed with the new `kid`
And during the 5s `OLD_KEY_GRACE_SECONDS` window tokens signed with the old `kid` are still accepted
**AC-7: AAA pattern enforced**
Given the xUnit test project compiles
When `dotnet build` runs
Then every `[Fact]` / `[Theory]` method in `tests/Azaion.Missions.E2E.Tests/Tests/` contains the literal comment lines `// Arrange` (when setup exists), `// Act`, and `// Assert` in that order — verified by a Roslyn analyzer test or a single integration assertion that greps the source files
## Constraints
- `restrictions.md` SW-01: target framework .NET 10 (matches `Azaion.Missions.csproj`)
- `restrictions.md` HW-01: ARM64 + AMD64 (multi-arch base images on both projects)
- `restrictions.md` ENV-01: HTTPS-only for the JWKS endpoint (HTTP would short-circuit AC-6.12)
- `coderule.mdc`: AAA pattern with `// Arrange` / `// Act` / `// Assert` comments, no narrative comments otherwise
- No project reference from `Azaion.Missions.E2E.Tests` → `Azaion.Missions.csproj` (consumer must remain blackbox; assertions only via HTTP and Npgsql side-channel)
- Side-channel DB access limited to fixture seeding + post-call assertions; marked with `[Trait("db_access","seed-or-assert-only")]` where used
- Token signing happens ONLY inside `jwks-mock`; the consumer never imports a JWT signing library
- `report.csv` lives in `./test-results/` (host-mounted); this directory MUST be in `.gitignore`