azaion/missions
1
0
Fork 0
mirror of https://github.com/azaion/missions.git synced 2026-06-21 09:21:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-585] [AZ-586] ResLim+Perf NFT tests; close test cycle 1

Browse Source 
Batch 4 of test implementation cycle 1 (existing-code Step 6, final batch).

- AZ-585 SteadyStateLoadTests + ColdStartRssTests: NFT-RES-LIM-01..04.
  SteadyStateLoadFixture runs one 5-min sustained-load window and samples
  RSS (docker stats), Npgsql conns (pg_stat_activity), and FDs
  (/proc/1/fd) every 5s; three test methods assert independently. All
  SkippableFact-gated on docker primitives.
- AZ-586 PerformanceTests: NFT-PERF-01..04. Sequential single-client,
  5 warm-ups + N measured calls, P50+P95 via LatencyPercentiles, recorded
  to PERF_RESULTS_FILE. Tagged Category=Perf so default gate excludes them.

Infrastructure:
- entrypoint.sh now applies --filter "${TEST_FILTER:-Category!=Perf}"
  per AZ-586 (default CI gate excludes performance).
- MetricCsvRecorder: idempotent CSV appender keyed on env var, used by
  both Perf and ResLim categories.

Step 6 (Implement Tests) is complete. Final report at
_docs/03_implementation/implementation_report_tests.md handoffs the
full-suite gate to test-run/SKILL.md (Step 7).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-15 09:11:53 +03:00
parent 26126e6216
commit 001e80fe96
14 changed files with 1181 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/03_implementation/implementation_report_tests.md
+119
View File
@@ -0,0 +1,119 @@
# Test Implementation Final Report
**Run**: existing-code Step 6 (Implement Tests)
**Date**: 2026-05-15
**Cycle**: 1
**Verdict**: HANDOFF — full-suite gate owned by `.cursor/skills/test-run/SKILL.md` (Step 7)
## Scope
11 test tasks decomposed by `/decompose-tests` and tracked under epic **AZ-575**:
| Task | Description | SP | Batch |
|---------|----------------------------------------------------------|----|-------|
| AZ-576 | Test infrastructure (compose, csproj, mocks, helpers) | 5 | 1 |
| AZ-577 | Vehicles positive (FT-P-01..06) | 5 | 2 |
| AZ-578 | Missions positive (FT-P-07..12) | 5 | 2 |
| AZ-579 | Waypoints + health positive (FT-P-13..18) | 5 | 2 |
| AZ-580 | Validation + authz negative (FT-N-01..08) | 3 | 2 |
| AZ-581 | Security auth/claims (NFT-SEC-01..06+04b) | 5 | 3 |
| AZ-582 | Security alg/rotation/CORS (NFT-SEC-07..13) | 5 | 3 |
| AZ-583 | Resilience cascade + migrator (NFT-RES-01..04) | 3 | 3 |
| AZ-584 | Resilience config/DB/rotation/race (NFT-RES-05..08) | 5 | 3 |
| AZ-585 | Resource limits (NFT-RES-LIM-01..04) | 3 | 4 |
| AZ-586 | Performance (NFT-PERF-01..04) | 3 | 4 |
| **Total** | | **47** | |
## Results
| Batch | Tasks | SP | Verdict | Carry-forwards |
|-------|----------------------------------|----|----------------------|----------------|
| 1 | AZ-576 | 5 | PASS_WITH_WARNINGS | 0 |
| 2 | AZ-577..AZ-580 | 18 | PASS_WITH_WARNINGS | 3 |
| 3 | AZ-581..AZ-584 | 18 | PASS_WITH_WARNINGS | 3 |
| 4 | AZ-585, AZ-586 | 6 | PASS_WITH_WARNINGS | 0 |
**Cumulative reviews**: 1 (`cumulative_review_batches_01-03_cycle1_report.md`, PASS_WITH_WARNINGS, 4 Low findings).
## AC Test Coverage
| Source | ACs | Tests | Coverage |
|--------|-----|-------|----------|
| FT-P (functional positive) | 18 | 18 | 18/18 |
| FT-N (negative) | 8 | 8 | 8/8 |
| NFT-SEC (security) | 14 | 22 | 14/14 (some scenarios → multiple `Theory` rows) |
| NFT-RES (resilience) | 8 | 12 | 8/8 |
| NFT-RES-LIM (resource lim) | 4 | 4 | 4/4 |
| NFT-PERF (performance) | 4 | 4 | 4/4 |
| **Total** | **56** | **68** | **56/56** |
Every AC has at least one trace via `[Trait("Traces", "AC-X.Y")]`; structural carry-forwards (6 total) are pinned with `[Trait("carry_forward", "...")]` so `dotnet test --filter "carry_forward~..."` surfaces them as a set when the underlying spec/code reconciliation lands.
## Spec-vs-Code Carry-forwards (6 total)
| Site | Spec says | Code says | Carry-forward tag |
|-----------------------------------------|----------------------------------------------|------------------------------------------------------------|-------------------------------|
| FT-P-03 `Vehicles/PositiveTests.cs` | `POST /vehicles/{id}/setDefault` → 200 + body| `[HttpPatch("{id:guid}/default")]` → 204 NoContent | `AC-1.4/route-shape` |
| FT-P-14/15 `Waypoints/PositiveTests.cs` | Nested `GeoPoint:{Lat,Lon,Mgrs}` | LinqToDB entity flat `Lat`/`Lon`/`Mgrs` | `flat-waypoint-shape` |
| FT-N-07 `Waypoints/NegativeTests.cs` | Missing parent → 404 + problem envelope | `GetWaypoints` returns `[]` | `AC-4.2/missing-parent-soft` |
| NFT-RES-01 `Resilience/CascadeF3Tests.cs` | Mid-walk cascade is transactional | `MissionService.DeleteMission` is non-transactional | `ADR-006` |
| NFT-RES-02 `Resilience/CascadeF4Tests.cs` | Waypoint cascade leaves detection=0/waypoint=1 partial state | `WaypointService.DeleteWaypoint` queries `media` BEFORE any deletion — aborts at step 1 with nothing deleted | `AC-4.6/walk-order` |
| NFT-RES-08 `Resilience/DefaultVehicleRaceTests.cs` | TOCTOU race observable | `ux_vehicles_one_default` partial unique index closes the race | `AC-1.4/index-closes-race` |
These carry-forwards flip the moment the spec or the code is reconciled; the tests fail loudly at that point — intentional.
## Code Review Summary
- **0 Critical / 0 High / 0 Medium** across all four batches.
- **4 Low findings** captured in cumulative review (3 follow-up + 1 baseline-carried) — see `_docs/03_implementation/cumulative_review_batches_01-03_cycle1_report.md`.
- Auto-fix rounds across the cycle: batch 2 (89× xUnit1030 warnings), batch 3 (3× missing-using errors), batch 4 (1× TokenMinter parameter-less ctor). All auto-fix-eligible per the Auto-Fix Gate matrix; no escalations.
## Files Added (high level)
- **Helpers** (10): `ApiDtos`, `DbAssertions`, `DockerLogs`, `FixtureSql`, `ForeignKeypair`, `HttpAssertions`, `LatencyPercentiles`, `MetricCsvRecorder`, `MissionsContainerHelper` — plus the existing `TestEnvironment`.
- **Fixtures** (9): `CascadeF3Fixture`, `CascadeF4Fixture`, `ComposeRestartFixture`, `DbResetFixture`, `JwksMockReverseFixture` (spec-only stub), `JwksRotateFixture`, `PostgresStopStartFixture`, `Seeds`, `StubSchema`, `SteadyStateLoadFixture`.
- **Test classes** (24): grouped under `Tests/{Vehicles,Missions,Waypoints,Health,Errors,Security,Resilience,Performance,ResourceLimits,Reporting}/` per the AZ-576 layout.
- **Infrastructure**: `docker-compose.test.yml` extensions (fixtures volume), `entrypoint.sh` Category-filter, `Reporting/TrxToCsvPostProcessor.cs` (from batch 1).
- **JWKS mock**: extended `SignBody` (permissions_array) + `TokenSigner` (kid_override validation) — required by NFT-SEC-06 and NFT-SEC-11.
## SkippableFact / SkippableTheory inventory
| Test | Skip predicate | Reason when skipped |
|------------------------------------------------------------|------------------------------------------------------------------|----------------------|
| `Tests/Health/HealthTests.NFT_P_17` (FT-P-17) | `COMPOSE_RESTART_ENABLED=1` | postgres-test stop/start |
| `Tests/Errors/Error500Tests.NFT_N_08` | `COMPOSE_RESTART_ENABLED=1` | drops vehicles table |
| `Tests/Security/ErrorRedactionTests.NFT_SEC_08` | `COMPOSE_RESTART_ENABLED=1` | drops vehicles table |
| `Tests/Security/StartupConfigTests.NFT_SEC_12` (theory + HTTP-JWKS) | `MissionsContainerHelper.Enabled` | docker run primitives |
| `Tests/Security/CorsConfigTests.NFT_SEC_13` (4 scenarios) | `MissionsContainerHelper.Enabled` | docker run primitives |
| `Tests/Resilience/CascadeF3Tests.NFT_RES_01` | `COMPOSE_RESTART_ENABLED=1` | drops media table |
| `Tests/Resilience/CascadeF4Tests.NFT_RES_02` | `COMPOSE_RESTART_ENABLED=1` | drops media table |
| `Tests/Resilience/MigratorRestartTests.NFT_RES_03/04` | `ComposeRestartFixture.Enabled` | docker compose restart |
| `Tests/Resilience/ConfigDbStartupTests.*` (8 methods) | `MissionsContainerHelper.Enabled` | docker run primitives |
| `Tests/Resilience/JwksRotationNoRestartTests.NFT_RES_07` | `MissionsContainerHelper.Enabled` (for StartedAt read) | docker inspect |
| `Tests/ResourceLimits/SteadyStateLoadTests.*` (3 methods) | `SteadyStateLoadFixture.SkipReason` (set on missing docker) | docker stats / docker exec |
| `Tests/ResourceLimits/ColdStartRssTests.NFT_RES_LIM_04` | `COMPOSE_RESTART_ENABLED=1` + `MissionsContainerHelper.Enabled` | docker compose stop/start |
Every Skippable test surfaces an explicit reason; none silent-pass.
## Handoff to Step 7 (Run Tests)
This report is a **HANDOFF** — the full-suite gate is owned by `.cursor/skills/test-run/SKILL.md`. That skill is responsible for:
1. Building the docker compose stack (`docker compose -f docker-compose.test.yml --profile test build`).
2. Running the e2e-consumer (`docker compose ... up --abort-on-container-exit --exit-code-from e2e-consumer e2e-consumer postgres-test missions jwks-mock`).
3. Inspecting `test-results/report.csv` + the Skippable test reasons.
4. Surfacing any blocking failure to the user via the test-run-skill's BLOCKING-gate protocol.
5. Optionally enabling the Docker-CLI Skippable subset via a one-time consumer-image upgrade (`docker-cli` install + socket bind) before the next cycle.
The performance suite is intentionally NOT part of the default gate — it runs via `scripts/run-performance-tests.sh` only.
## Outstanding follow-ups (NOT blocking Step 7)
1. **Docker-CLI inside e2e-consumer image** — needed to activate the 12 Skippable methods. Recommend a separate ticket sized 3 SP (Dockerfile add of `docker-cli` package + `docker-compose.test.yml` `/var/run/docker.sock` mount). Validates run-perf script's `/app/` → `/src/` path bug at the same time.
2. **Test/source compilation separation** — `Azaion.Missions.csproj` Sdk.Web globs pull `tests/**/*.cs`. Recommend `<Compile Remove="tests/**" />` or moving to a `.sln`. Pre-existing project layout drift.
3. **AC-1.4 carry-forward decision** — see NFT-RES-08 carry-forward. The product team should decide whether the partial unique index OR an application-level guard is the canonical solution; today the test pins the index behaviour.
4. **AC-4.6 walk-order decision** — see NFT-RES-02 carry-forward. The waypoint cascade walks dependency tables in a different order than the spec implied; the team should reconcile spec and code.
## Sign-off
Cycle 1 test implementation complete. 4 batches, 11 tasks, 47 SP. All ACs traced; no blocking findings; tracker tickets transitioned to **In Testing**. Autodev advances to Step 7 (Run Tests).