mirror of
https://github.com/azaion/missions.git
synced 2026-06-22 06:21:07 +00:00
[AZ-577] [AZ-578] [AZ-579] [AZ-580] Implement E2E test batch 2
Adds 26 blackbox tests (FT-P-01..18, FT-N-01..08) covering full AC
matrices for Vehicles/Missions/Waypoints/Health/Errors. Three
spec-vs-code carry-forwards documented in batch_02_report.md and
pinned with [Trait("carry_forward", ...)].
Shared scaffolding: ApiDtos.cs, AssertProblemEnvelopeAsync helper,
Seeds.cs, StubSchema.cs, CascadeF3/F4 fixtures, PostgresStopStart
fixture (gated by COMPOSE_RESTART_ENABLED). Removes the 4 placeholder
Sanity.cs files (now superseded). docker-compose.test.yml gains the
expected_results volume mount + FIXTURE_SQL_DIR for the consumer.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -1,114 +0,0 @@
|
||||
# Vehicles Positive Flow Tests
|
||||
|
||||
**Task**: AZ-577_test_vehicles_positive
|
||||
**Name**: Vehicles positive tests (FT-P-01..06)
|
||||
**Description**: Implement xUnit blackbox tests for the 6 happy-path Vehicle CRUD scenarios — create non-default, create default (demotes prior), setDefault, list (no-pagination + Name ASC), filter (case-INSENSITIVE name + exact isDefault), delete with no references.
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-577
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
The `/vehicles` surface implements two non-obvious invariants that documentation alone cannot guarantee: (1) creating a default vehicle clears any prior default in the same logical step, and (2) the list filter is case-INSENSITIVE on `name` (the docs said case-sensitive until 2026-05-14 — drift now corrected, but only an executable test can pin the actual code path). Without these tests, a future refactor of `VehicleService` could silently re-introduce two default rows or a case-sensitive filter and break consumers (`autopilot` reads the default vehicle on boot).
|
||||
|
||||
## Outcome
|
||||
|
||||
- All six FT-P-01..06 scenarios run against the dockerised `missions` service via HTTP + Npgsql side-channel and pass.
|
||||
- Each test produces a CSV row with `Category=Blackbox`, `Traces=AC-1.x`, `Result=pass`, and an `ExecutionTimeMs` under the documented `Max execution time` (5s for create paths, 2s for read/delete).
|
||||
- The list test asserts both shape (`array` not `PaginatedResponse`) and ordering (`Name ASC`).
|
||||
- The filter test asserts case-INSENSITIVE matching for two casings (`BR` and `br`).
|
||||
- The default-clear invariant is verified via DB count (`is_default=true` count == 1 after every default-creating action).
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- FT-P-01 Create non-default — `POST /vehicles` body shape + PascalCase response + DB row count.
|
||||
- FT-P-02 Create default demotes prior default — `seed_one_default_vehicle` precondition; assert exactly one default after.
|
||||
- FT-P-03 setDefault promotes existing vehicle — `POST /vehicles/{id}/setDefault`; assert clear-then-set via side-channel.
|
||||
- FT-P-04 List unpaginated + Name ASC — assert body is JSON array (not `{Items,Page,…}`), assert length and ordering.
|
||||
- FT-P-05 Filter `name=BR&isDefault=true` then `name=br&…` — assert case-INSENSITIVE substring match against `seed_3_vehicles_2_default`.
|
||||
- FT-P-06 Delete with no references — `204` + DB count 0.
|
||||
|
||||
### Excluded
|
||||
|
||||
- FT-N-03 "delete vehicle in use returns 409" lives in Task 13 (negative tests).
|
||||
- Validation-of-input scenarios (empty `Name`, negative `BatteryCapacity`, unknown `Type` int) are carry-forwards documented in `test-data.md` § Data Validation Rules; they are NOT tested here because the spec marks them as "accepted today" — they belong to the Refactor Backlog, not this task.
|
||||
- TOCTOU race on default-vehicle exclusivity (NFT-RES-08) lives in Task 17.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: FT-P-01 returns 201 with PascalCase body**
|
||||
Given `seed_empty` and a JWT with `permissions=FL`
|
||||
When `POST /vehicles` is issued with the documented body
|
||||
Then response is `201 Created`, body parses as `Vehicle` with PascalCase keys, `Id` parses as UUID, side-channel `SELECT COUNT(*) FROM vehicles WHERE id=<returned>` returns 1
|
||||
|
||||
**AC-2: FT-P-02 demotes prior default**
|
||||
Given `seed_one_default_vehicle` (prior row `P1.is_default=true`)
|
||||
When `POST /vehicles { …, IsDefault:true }` is issued
|
||||
Then response is `201`, side-channel shows new row `is_default=true`, row `P1.is_default=false`, and `SELECT COUNT(*) WHERE is_default=true` == 1
|
||||
|
||||
**AC-3: FT-P-03 setDefault clears prior**
|
||||
Given `seed_one_default_vehicle` plus a non-default row `P2`
|
||||
When `POST /vehicles/{P2}/setDefault { IsDefault:true }` is issued
|
||||
Then response is `200` with `Id==P2, IsDefault==true`, and side-channel shows `P2.is_default=true`, `P1.is_default=false`, count==1
|
||||
|
||||
**AC-4: FT-P-04 list is unpaginated and ordered**
|
||||
Given `seed_3_vehicles_2_default` containing `BR-01, BR-02, MQ-9` in any insert order
|
||||
When `GET /vehicles` is issued
|
||||
Then response is `200`, body parses as a JSON array (NOT an object with `Items`), `body.length == 3`, and `[v.Name for v in body] == ["BR-01","BR-02","MQ-9"]`
|
||||
|
||||
**AC-5: FT-P-05 filter is case-INSENSITIVE**
|
||||
Given `seed_3_vehicles_2_default`
|
||||
When `GET /vehicles?name=BR&isDefault=true` AND `GET /vehicles?name=br&isDefault=true` are issued
|
||||
Then both responses are `200` with `body.length == 1` and `body[0].Name == "BR-01"`
|
||||
|
||||
**AC-6: FT-P-06 delete is 204 + row gone**
|
||||
Given one vehicle row with no missions referencing it
|
||||
When `DELETE /vehicles/{id}` is issued
|
||||
Then response is `204 No Content` with empty body, and side-channel shows `count == 0` for that id
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- Each test must complete inside the documented `Max execution time` from `blackbox-tests.md` (5s for FT-P-01..03, 5s for FT-P-07-style writes, 2s for FT-P-04..06). The xUnit `[Trait("max_ms", "5000")]` or per-test `Timeout` must reflect this.
|
||||
|
||||
**Reliability**
|
||||
- Tests share a `[Collection("Vehicles")]` xUnit collection and use `IClassFixture<DbResetFixture>` to TRUNCATE between scenarios. No state must leak between FT-P-01 and FT-P-04.
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | `seed_empty`, JWT permissions=FL | `POST /vehicles` non-default body | `201` + PascalCase `Vehicle` + DB count 1 | — |
|
||||
| AC-2 | `seed_one_default_vehicle` (P1) | `POST /vehicles { IsDefault:true }` | `201` + DB shows count==1 default after | AC-1.2 invariant |
|
||||
| AC-3 | `seed_one_default_vehicle` + extra P2 | `POST /vehicles/{P2}/setDefault` | `200` + DB count==1 default; P1 cleared | AC-1.2 / AC-1.4 |
|
||||
| AC-4 | `seed_3_vehicles_2_default` (`BR-01,BR-02,MQ-9`) | `GET /vehicles` shape + order | `200` + array + Name ASC | AC-1.5 |
|
||||
| AC-5 | `seed_3_vehicles_2_default` | `GET /vehicles?name=BR…` + `?name=br…` | `200` + len 1 + `BR-01` for both casings | AC-1.6 |
|
||||
| AC-6 | One row, zero missions | `DELETE /vehicles/{id}` | `204` + DB count 0 | AC-1.10 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080` (no project reference to `Azaion.Missions.csproj`).
|
||||
- Bearer token minted via `https://jwks-mock:8443/sign` with `permissions=FL`.
|
||||
- DB assertions through the Npgsql side-channel only; marked `[Trait("db_access","seed-or-assert-only")]`.
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` comments per `coderule.mdc`.
|
||||
- PascalCase JSON contract (`PropertyNamingPolicy = null`) is part of the SUT contract; the test must NOT silently accept camelCase.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: Tests depend on side-channel SQL that drifts from the SUT migrator**
|
||||
- *Risk*: If the migrator changes the `vehicles` column set, hand-rolled `INSERT` in the seed fixture breaks.
|
||||
- *Mitigation*: Seed fixtures use the schema produced by the SUT's own startup migrator — `docker compose up` runs first, then the fixture inserts into the already-migrated tables.
|
||||
|
||||
**Risk 2: Ordering test (AC-4) is flaky if insert order accidentally matches alphabetic order**
|
||||
- *Risk*: A non-deterministic seed insert could mask a missing `OrderBy`.
|
||||
- *Mitigation*: Seed fixture inserts rows in `[MQ-9, BR-02, BR-01]` order (reverse alphabetic) so the test fails if the SUT omits the `OrderBy(a => a.Name)`.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface (`http://missions:8080/vehicles*`) plus the documented DB side-channel for fixture seeding and post-call assertions; expected outputs are compared against `_docs/00_problem/input_data/expected_results/results_report.md` rows AC-1 1.1, 1.2, 1.4, 1.5, 1.6, 1.10.
|
||||
- Stubs are allowed ONLY for the external `admin` JWT issuer (the `jwks-mock` container per `tests/Azaion.Missions.JwksMock/`).
|
||||
- Stubs, fakes, monkeypatches, deterministic fallbacks, or direct imports are NOT allowed for any internal product module — including `VehicleService`, `VehiclesController`, `AppDataConnection`, `DatabaseMigrator`, `JwtExtensions`, or `ErrorHandlingMiddleware`. If any of these is not implemented (e.g., the SUT image hasn't been built), the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
@@ -1,121 +0,0 @@
|
||||
# Missions Positive Flow Tests
|
||||
|
||||
**Task**: AZ-578_test_missions_positive
|
||||
**Name**: Missions positive tests (FT-P-07..12)
|
||||
**Description**: Implement xUnit blackbox tests for the 6 happy-path Mission scenarios — create with default CreatedDate, paginated list (PageSize=20, CreatedDate DESC, case-INSENSITIVE name filter), page 2, date-range filter, partial update preserving null fields, and full cascade delete across map_objects/detection/annotations/media/waypoints/missions.
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-578
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
The `/missions` surface is the project's most consequential read+write path. Three behaviours are easy to silently break: (1) the default `CreatedDate = UtcNow` when the body omits it (AC-2.1), (2) `PaginatedResponse<Mission>` envelope with `Page,PageSize,TotalCount,Items` PascalCase keys + `CreatedDate DESC` ordering (AC-2.3), and (3) the cascade delete walking every dependency table including DB-only stub tables `map_objects`, `detection`, `annotations`, `media` (AC-3.1). The cascade is **not** transaction-wrapped (NFT-RES-01 in Task 16 pins that invariant); the positive scenario here verifies the happy-path walk completes.
|
||||
|
||||
## Outcome
|
||||
|
||||
- All six FT-P-07..12 scenarios run against the dockerised `missions` service and pass.
|
||||
- Each test produces a CSV row with `Category=Blackbox`, `Traces=AC-2.x` or `AC-3.1`, `Result=pass`, within the documented `Max execution time` (5s for create, 2s for list/update, 10s for cascade delete).
|
||||
- The pagination test asserts both the envelope shape (`Items, TotalCount, Page, PageSize` PascalCase) AND `CreatedDate` DESC ordering across all 20 items.
|
||||
- The cascade test compares per-table delete counts against `_docs/00_problem/input_data/expected_results/cascade_F3_walk.json` via `json_diff`.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- FT-P-07 Mission create with default CreatedDate — assert `|body.CreatedDate - t0| ≤ 5s`.
|
||||
- FT-P-08 Mission list default page — envelope shape, `Page==1`, `PageSize==20`, `TotalCount==25`, `Items.length==20`, `CreatedDate` DESC ordering, plus case-INSENSITIVE `?name=re` filter.
|
||||
- FT-P-09 Mission list page 2 — `Page==2`, `Items.length==5`, UUID-set disjoint from page 1.
|
||||
- FT-P-10 Mission list date range — `?fromDate=&toDate=` inclusivity (January 2026 returns 5 of 25).
|
||||
- FT-P-11 Mission partial update — `PUT /missions/{id}` with `VehicleId:null` preserves prior `VehicleId`.
|
||||
- FT-P-12 Mission cascade delete (F3) — `DELETE /missions/{id}` walks every dependency table; per-table counts compared against `cascade_F3_walk.json`.
|
||||
|
||||
### Excluded
|
||||
|
||||
- FT-N-04 "create mission with non-existent VehicleId returns 400" lives in Task 13.
|
||||
- FT-N-05 "GET mission 404" lives in Task 13.
|
||||
- FT-N-06 "cascade delete short-circuits on missing mission (no DELETE issued against dependency tables)" lives in Task 13.
|
||||
- Cascade NOT-transaction-wrapped invariant (NFT-RES-01) lives in Task 16.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: FT-P-07 mission create defaults CreatedDate to UtcNow**
|
||||
Given `seed_one_default_vehicle` and a JWT with `permissions=FL`
|
||||
When the consumer captures `t0 = UtcNow` then issues `POST /missions { Name:"Recon-01", VehicleId:<id>, CreatedDate:null }`
|
||||
Then response is `201`, `body.CreatedDate` parses as UTC, and `abs(body.CreatedDate - t0) ≤ 5s`
|
||||
|
||||
**AC-2: FT-P-08 list returns PaginatedResponse with DESC ordering and case-INSENSITIVE name filter**
|
||||
Given `seed_25_missions` (5 January, 20 February 2026, mix of `Recon-*` names)
|
||||
When `GET /missions` is issued
|
||||
Then response is `200` with `Page==1, PageSize==20, TotalCount==25, Items.length==20`, all PascalCase keys, AND for every `i ∈ [0..18]` `Items[i].CreatedDate >= Items[i+1].CreatedDate` (strictly DESC ordering)
|
||||
And when `GET /missions?name=re` (lowercase) is issued, `body.TotalCount > 0` (case-INSENSITIVE substring match against `Recon-*`)
|
||||
|
||||
**AC-3: FT-P-09 page 2 returns the remaining 5 items, disjoint from page 1**
|
||||
Given `seed_25_missions`
|
||||
When `GET /missions?page=2&pageSize=20` is issued
|
||||
Then response is `200`, `Page==2`, `Items.length==5`, AND the set of `Items[*].Id` is disjoint from the page-1 response
|
||||
|
||||
**AC-4: FT-P-10 date range filter is inclusive of bounds**
|
||||
Given `seed_25_missions` (5 in January 2026, 20 in February 2026)
|
||||
When `GET /missions?fromDate=2026-01-01T00:00:00Z&toDate=2026-01-31T23:59:59Z` is issued
|
||||
Then response is `200`, `TotalCount==5`, and every `Items[i].CreatedDate` is within January 2026 UTC
|
||||
|
||||
**AC-5: FT-P-11 partial update preserves null fields**
|
||||
Given one mission row with known `Name="Original"` and `VehicleId=V1`
|
||||
When `PUT /missions/{id} { Name:"Renamed", VehicleId:null }` is issued
|
||||
Then response is `200`, `body.Name == "Renamed"`, AND `body.VehicleId == V1` (preserved)
|
||||
|
||||
**AC-6: FT-P-12 cascade delete walks every dependency table**
|
||||
Given `fixture_cascade_F3` applied (one mission with 2 waypoints → 2 media → 2 annotations → 2 detection rows + 3 map_objects)
|
||||
When `DELETE /missions/{mid}` is issued
|
||||
Then response is `204`, AND side-channel `SELECT COUNT(*)` returns 0 for `map_objects`, `detection`, `annotations`, `media`, `waypoints`, `missions` rows in the seeded chain
|
||||
And the per-table counts after deletion match `_docs/00_problem/input_data/expected_results/cascade_F3_walk.json` via deep JSON diff
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- FT-P-07: ≤ 5s. FT-P-08..11: ≤ 2s each. FT-P-12: ≤ 10s (cascade through 5 tables).
|
||||
|
||||
**Reliability**
|
||||
- FT-P-12 must use `IClassFixture<DbResetFixture>` that recreates `fixture_cascade_F3` fresh per scenario (the fixture is destructive). FT-P-08..10 share `seed_25_missions` across the same class.
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | `seed_one_default_vehicle` | `POST /missions { CreatedDate:null }` | `201` + `\|body.CreatedDate - t0\| ≤ 5s` | AC-2.1 |
|
||||
| AC-2 | `seed_25_missions` | `GET /missions` then `GET /missions?name=re` | `200` + envelope + DESC + case-INSENSITIVE match | AC-2.3, AC-8.7 |
|
||||
| AC-3 | `seed_25_missions` | `GET /missions?page=2&pageSize=20` | `200` + `Page=2` + len 5 + disjoint UUIDs | AC-2.3 |
|
||||
| AC-4 | `seed_25_missions` | `GET /missions?fromDate=…&toDate=…` (January window) | `200` + `TotalCount=5` + all in window | AC-2.3 |
|
||||
| AC-5 | One row with `Name=Original, VehicleId=V1` | `PUT /missions/{id} { Name:"Renamed", VehicleId:null }` | `200` + Name updated + VehicleId preserved | AC-2.5 |
|
||||
| AC-6 | `fixture_cascade_F3` | `DELETE /missions/{mid}` | `204` + DB counts 0 across 6 tables + `cascade_F3_walk.json` match | AC-3.1 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080/missions*` (no project reference to `Azaion.Missions.csproj`).
|
||||
- Bearer token minted via `https://jwks-mock:8443/sign` with `permissions=FL`.
|
||||
- FT-P-12 fixture uses the SQL file at `_docs/00_problem/input_data/expected_results/fixture_cascade_F3.sql` (NOT a hand-rolled INSERT — the SQL file is the contract).
|
||||
- Per-table count comparison in FT-P-12 uses `json_diff` against `cascade_F3_walk.json`; if the file is missing, the test must fail (not silently pass).
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` per test.
|
||||
- `seed_25_missions` MUST use deterministic UUIDs and deterministic `CreatedDate` values so the disjoint-set assertion in AC-3 and the date-range assertion in AC-4 are reproducible.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: cascade_F3_walk.json drifts from fixture_cascade_F3.sql**
|
||||
- *Risk*: Updating the seed SQL without updating the walk JSON makes AC-6 silently pass with wrong counts.
|
||||
- *Mitigation*: Both files live under the same `expected_results/` directory; the test loads the walk JSON at runtime and verifies BOTH that pre-delete counts match the walk's `before` values AND post-delete counts match the walk's `after` values. A drift fails the "before" assertion first.
|
||||
|
||||
**Risk 2: AC-2 ordering assertion is flaky if seed CreatedDate values collide**
|
||||
- *Risk*: Two missions with identical `CreatedDate` produce a tie-breaker-dependent order; the DESC assertion would be deterministic only if the comparator is stable.
|
||||
- *Mitigation*: `seed_25_missions` SQL assigns distinct `CreatedDate` values spaced ≥ 1 second apart; any future seed change must preserve this invariant.
|
||||
|
||||
**Risk 3: cascade test pollutes neighbour scenarios**
|
||||
- *Risk*: F3 fixture deletes rows across 6 tables; if FT-P-12 runs in the same xUnit class as a read-path test, that test sees an empty DB.
|
||||
- *Mitigation*: FT-P-12 lives in its own xUnit `[Collection("CascadeF3")]` and uses `IClassFixture<DbResetFixture>` to reset between every scenario in the class.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface (`http://missions:8080/missions*`) plus the documented DB side-channel for fixture seeding and post-call assertions. Expected outputs are compared against `_docs/00_problem/input_data/expected_results/results_report.md` rows AC-2 2.1, 2.3, 2.4, 2.5, 2.7 and AC-3 row 3.1, and against the machine-readable file `_docs/00_problem/input_data/expected_results/cascade_F3_walk.json` for the cascade walk.
|
||||
- Stubs are allowed ONLY for: the external `admin` JWT issuer (`jwks-mock` container) and the DB-only stub tables for `media`, `annotations`, `detection`, `map_objects` (seeded via side-channel SQL because the owning services are out of scope per `environment.md`).
|
||||
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including `MissionService`, `MissionsController`, `WaypointService`, `AppDataConnection`, `DatabaseMigrator`, `JwtExtensions`, or `ErrorHandlingMiddleware`. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
@@ -1,120 +0,0 @@
|
||||
# Waypoints + Health Positive Flow Tests
|
||||
|
||||
**Task**: AZ-579_test_waypoints_health_positive
|
||||
**Name**: Waypoints + Health positive tests (FT-P-13..18)
|
||||
**Description**: Implement xUnit blackbox tests for the 6 happy-path Waypoint + Health scenarios — waypoint list ordered by OrderNum ASC, waypoint create echoes geo fields (no auto-conversion), waypoint update is full overwrite, health 200 anonymous, health 200 with Postgres stopped (no DB ping), and waypoint cascade delete scoped to one waypoint (sibling chain intact).
|
||||
**Complexity**: 5 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-579
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
Waypoints carry two non-obvious behaviors: (1) the list endpoint orders by `OrderNum` ASC regardless of insert order (AC-4.3), and (2) `PUT /missions/{id}/waypoints/{wpId}` is a FULL overwrite even though the DTO looks "partial" (non-nullable enums + numerics) — passing `Height:0` overwrites the previous `Height:120` (AC-4.4). The waypoint cascade delete (AC-4.5) is the tighter sibling of the mission cascade — it must remove the target waypoint's chain (`media → annotations → detection`) without touching a sibling waypoint's chain. The health endpoint (AC-7.1, AC-7.2) is the suite's probe contract: it MUST return 200 anonymously AND MUST NOT ping the database, because the suite reverse proxy uses `/health` to decide whether to route traffic — a DB outage must not depool a healthy process.
|
||||
|
||||
## Outcome
|
||||
|
||||
- All six FT-P-13..18 scenarios run against the dockerised `missions` service and pass.
|
||||
- Each test produces a CSV row with `Category=Blackbox`, `Traces=AC-4.x` or `AC-7.x`, `Result=pass`, within the documented `Max execution time` (2s for FT-P-13..16, 5s for FT-P-17 to allow PG stop, 10s for FT-P-18 cascade).
|
||||
- The list test asserts both shape (JSON array) and ordering (`[1,2,3,4,5]` ASC from a `[3,1,2,5,4]` insert order).
|
||||
- The update test asserts the FULL overwrite by passing `Height:0` and checking the new value is 0 (not the preserved 120).
|
||||
- The "PG stopped" health test asserts the process answers `200` even with `postgres-test` stopped — proving the probe does not ping the DB.
|
||||
- The cascade test (F4) asserts target-waypoint chain deleted AND sibling-waypoint chain preserved, with per-table counts compared against `cascade_F4_walk.json`.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- FT-P-13 Waypoint list ordered by `OrderNum` ASC — `seed_5_waypoints_unordered` inserts in `[3,1,2,5,4]` order.
|
||||
- FT-P-14 Waypoint create echoes `GeoPoint` fields (no auto lat/lon ↔ MGRS conversion today — preserves the documented divergence from spec).
|
||||
- FT-P-15 Waypoint update is full overwrite — `Height:0` overwrites `Height:120`, `OrderNum` changes, `GeoPoint:null` clears.
|
||||
- FT-P-16 Health 200 anonymous — no `Authorization` header, exact JSON `{ "status": "healthy" }`.
|
||||
- FT-P-17 Health 200 with PG stopped — proves process-liveness only, no DB ping.
|
||||
- FT-P-18 Waypoint cascade delete (F4) — `DELETE /missions/{mid}/waypoints/{wp1}`; per-table counts on `wp1` chain go to 0; sibling `wp2` chain intact.
|
||||
|
||||
### Excluded
|
||||
|
||||
- FT-N-07 "waypoint operation against missing mission returns 404" lives in Task 13.
|
||||
- Waypoint nested existence check (single composite-FK predicate per `state.json` drift entry) is implementation detail; the blackbox test only asserts the observable 404 in FT-N-07.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: FT-P-13 waypoint list is ordered by OrderNum ASC**
|
||||
Given `seed_5_waypoints_unordered` under one mission, with `order_num` values `[3,1,2,5,4]` inserted in that order
|
||||
When `GET /missions/{id}/waypoints` is issued with a valid JWT
|
||||
Then response is `200`, body parses as JSON array, `body.length == 5`, AND `[w.OrderNum for w in body] == [1,2,3,4,5]`
|
||||
|
||||
**AC-2: FT-P-14 waypoint create echoes geo fields, no MGRS conversion**
|
||||
Given one mission row
|
||||
When `POST /missions/{id}/waypoints { GeoPoint:{Lat:50.45, Lon:30.52, Mgrs:null}, WaypointSource:0, WaypointObjective:0, OrderNum:1, Height:120 }` is issued
|
||||
Then response is `201`, `body.GeoPoint.Lat == 50.45`, `body.GeoPoint.Lon == 30.52`, AND `body.GeoPoint.Mgrs == null` (NO auto-conversion)
|
||||
|
||||
**AC-3: FT-P-15 waypoint update is full overwrite**
|
||||
Given one waypoint with `Height=120, OrderNum=1, GeoPoint=(Lat:50.45, …)`
|
||||
When `PUT /missions/{id}/waypoints/{wpId} { GeoPoint:null, WaypointSource:1, WaypointObjective:1, OrderNum:2, Height:0 }` is issued
|
||||
Then response is `200`, `body.Height == 0` (overwritten from 120), `body.OrderNum == 2`, AND `body.GeoPoint == null`
|
||||
|
||||
**AC-4: FT-P-16 health is 200 anonymous**
|
||||
Given a running `missions` container
|
||||
When `GET /health` is issued with NO `Authorization` header
|
||||
Then response is `200`, body is exactly `{ "status": "healthy" }` with case-sensitive key
|
||||
|
||||
**AC-5: FT-P-17 health is 200 with PG stopped**
|
||||
Given `missions` is running AND `docker compose stop postgres-test` has succeeded
|
||||
When `GET /health` is issued
|
||||
Then response is `200`, body is exactly `{ "status": "healthy" }` — proving the probe does NOT ping the DB
|
||||
|
||||
**AC-6: FT-P-18 waypoint cascade scope is one waypoint**
|
||||
Given `fixture_cascade_F4` (target waypoint `wp1` with chain `media → annotations → detection`; sibling waypoint `wp2` with its own chain)
|
||||
When `DELETE /missions/{mid}/waypoints/{wp1}` is issued
|
||||
Then response is `204`, AND side-channel `SELECT COUNT(*)` returns 0 for the `wp1` chain rows in `detection`, `annotations`, `media`, AND for `wp1` itself in `waypoints`
|
||||
And side-channel returns `1` for `wp2` in `waypoints` AND `> 0` for the `wp2` chain rows in `media, annotations, detection`
|
||||
And the per-table counts after deletion match `_docs/00_problem/input_data/expected_results/cascade_F4_walk.json` via deep JSON diff
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- FT-P-13..16: ≤ 2s each. FT-P-17: ≤ 5s (allow PG stop time). FT-P-18: ≤ 10s (cascade through 4 tables).
|
||||
|
||||
**Reliability**
|
||||
- FT-P-17 must restore `postgres-test` to `Up` before exiting (try/finally with `docker compose start postgres-test` in the fixture teardown) — otherwise subsequent tests fail with `ConnectionRefused`.
|
||||
- FT-P-18 uses `IClassFixture<DbResetFixture>` with the F4 fixture recreated per scenario.
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | `seed_5_waypoints_unordered` ([3,1,2,5,4]) | `GET /missions/{id}/waypoints` | `200` + array + OrderNum ASC | AC-4.3 |
|
||||
| AC-2 | One mission row | `POST /missions/{id}/waypoints { GeoPoint:{Lat,Lon,Mgrs:null} }` | `201` + GeoPoint echoed + Mgrs null (no conversion) | AC-4 (data_parameters § 2.3) |
|
||||
| AC-3 | One waypoint Height=120 | `PUT … { Height:0, GeoPoint:null }` | `200` + Height=0 + GeoPoint=null (full overwrite) | AC-4.4 |
|
||||
| AC-4 | Running container | `GET /health` no auth | `200` + exact `{"status":"healthy"}` | AC-7.1 |
|
||||
| AC-5 | PG stopped | `GET /health` | `200` + exact `{"status":"healthy"}` | AC-7.2, AC-7.3 |
|
||||
| AC-6 | `fixture_cascade_F4` | `DELETE /missions/{mid}/waypoints/{wp1}` | `204` + wp1 chain 0 + wp2 chain intact + `cascade_F4_walk.json` match | AC-4.5 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080`; bearer token via `https://jwks-mock:8443/sign` with `permissions=FL` (for waypoint endpoints); FT-P-16 and FT-P-17 explicitly send no `Authorization` header.
|
||||
- FT-P-17 uses `ComposeRestartFixture`-style helper that runs `docker compose -f docker-compose.test.yml stop postgres-test` then `docker compose -f docker-compose.test.yml start postgres-test` in teardown.
|
||||
- FT-P-18 fixture uses `_docs/00_problem/input_data/expected_results/fixture_cascade_F4.sql` (NOT a hand-rolled INSERT).
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` per test.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: FT-P-15 silently passes if SUT exposes a "partial" update path**
|
||||
- *Risk*: If a future refactor adds a JSON-merge update mode, sending `Height:0` might be interpreted as "leave Height unchanged" rather than overwrite.
|
||||
- *Mitigation*: The test ALSO sets `GeoPoint:null` and asserts the value is null after — proving the path is full-overwrite, not patch.
|
||||
|
||||
**Risk 2: FT-P-17 PG-stop leaks to other tests**
|
||||
- *Risk*: If the test fails before teardown, subsequent tests run against a dead DB.
|
||||
- *Mitigation*: The fixture uses `try/finally`; the teardown waits for `postgres-test` to reach `healthy` (poll `pg_isready`) before yielding control back to xUnit.
|
||||
|
||||
**Risk 3: FT-P-18 sibling-intact assertion gives false-pass if F4 fixture is empty**
|
||||
- *Risk*: If `fixture_cascade_F4.sql` failed to insert `wp2`'s chain, the post-delete assertion `wp2 chain > 0` fails trivially — but with a misleading message.
|
||||
- *Mitigation*: The test asserts pre-delete counts FIRST (`wp1` chain > 0 AND `wp2` chain > 0); fixture failure is caught in the Arrange phase, not the Assert phase.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface (`http://missions:8080/missions/{id}/waypoints*` and `http://missions:8080/health`) plus the documented DB side-channel for fixture seeding and post-call assertions. Expected outputs are compared against `_docs/00_problem/input_data/expected_results/results_report.md` rows AC-4 4.2, 4.3, 4.4, 4.5 and AC-7 rows 7.1, 7.2, and against the machine-readable file `_docs/00_problem/input_data/expected_results/cascade_F4_walk.json`.
|
||||
- Stubs are allowed ONLY for: the external `admin` JWT issuer (`jwks-mock` container) and the DB-only stub tables for `media`, `annotations`, `detection` (seeded via side-channel SQL).
|
||||
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including `WaypointService`, `MissionsController` (health route), `AppDataConnection`, or `Program.cs`'s health middleware. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
@@ -1,134 +0,0 @@
|
||||
# Validation + 404 + Authz Negative Tests
|
||||
|
||||
**Task**: AZ-580_test_validation_authz_negative
|
||||
**Name**: Functional negative tests (FT-N-01..08)
|
||||
**Description**: Implement xUnit blackbox tests for the 8 negative scenarios — case-insensitive filter no-match, 404 for missing GET vehicle/mission/waypoint-parent, 409 for delete-vehicle-in-use, 400 for create-mission-with-bogus-VehicleId (carry-forward divergence), cascade short-circuit on missing mission (no dependency DELETEs issued), and the generic 500 redacted-body + stacktrace-in-log contract.
|
||||
**Complexity**: 3 points
|
||||
**Dependencies**: AZ-576_test_infrastructure
|
||||
**Component**: Blackbox Tests
|
||||
**Tracker**: AZ-580
|
||||
**Epic**: AZ-575
|
||||
|
||||
## Problem
|
||||
|
||||
The negative-path contract is what protects clients from undefined behaviour: every documented failure must produce a predictable status code + `{ statusCode, message }` envelope, and no failure mode may silently mutate state. Three behaviors are especially load-bearing: (1) `DELETE /missions/{missing}` must 404 *before* any dependency-table DELETE issues — otherwise a typo'd UUID could remove rows from `map_objects` belonging to a different mission (AC-3.2); (2) `DELETE /vehicles/{used}` must 409 and leave the row in place (AC-1.8); (3) the generic 500 must redact internals — `Internal server error` body, full stack only in container logs (AC-8.6, AC-10.3).
|
||||
|
||||
## Outcome
|
||||
|
||||
- All eight FT-N-01..08 scenarios run against the dockerised `missions` service and pass.
|
||||
- Each test produces a CSV row with `Category=Blackbox` (negative subset; `Traces=AC-1.6, AC-1.7, AC-1.8, AC-2.2, AC-2.4, AC-3.2, AC-4.2, AC-8.6, AC-10.3`), `Result=pass`.
|
||||
- The 500 test asserts BOTH that the body is exactly `{ "statusCode":500, "message":"Internal server error" }` AND that the container log emitted an `"Unhandled exception"` line within 2s.
|
||||
- FT-N-06 asserts via `pg_stat_statements` (or post-request log scrape) that NO `DELETE FROM map_objects/waypoints/media/annotations/detection` SQL ran during the 404 request — the existence check short-circuits before the cascade.
|
||||
- FT-N-04 explicitly pins the documented spec-divergence (returns 400 today, spec wants 404); test must include a comment marking it as a carry-forward to revisit when the divergence is closed.
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- FT-N-01 Vehicle name filter no-match — `?name=ZZ` and `?name=zz` against `seed_3_vehicles_2_default` both return `body.length == 0`.
|
||||
- FT-N-02 GET vehicle 404 — random UUID returns `{ statusCode:404, message:… }`.
|
||||
- FT-N-03 Delete vehicle in use 409 — row not deleted afterwards.
|
||||
- FT-N-04 Create mission with bogus VehicleId returns 400 today (CARRY-FORWARD comment).
|
||||
- FT-N-05 GET mission 404 — envelope shape.
|
||||
- FT-N-06 Cascade short-circuit — 404 + zero DELETE SQL issued.
|
||||
- FT-N-07 Waypoint operation against missing mission — 404.
|
||||
- FT-N-08 Generic 500 — redacted body + stacktrace in log.
|
||||
|
||||
### Excluded
|
||||
|
||||
- 401 / 403 auth-failure paths (NFT-SEC-01..06) live in Task 14.
|
||||
- 400/422 spec-divergence carry-forwards that are NOT executable today (input validation for empty `Name`, negative `BatteryCapacity`, unknown `Type` int) are documented as Refactor Backlog items in `tests/blackbox-tests.md` and are NOT in scope here.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
**AC-1: FT-N-01 vehicle filter no-match returns empty array for both casings**
|
||||
Given `seed_3_vehicles_2_default` (`BR-01, BR-02, MQ-9`)
|
||||
When `GET /vehicles?name=ZZ` then `GET /vehicles?name=zz` are issued
|
||||
Then both responses are `200` with `body.length == 0`
|
||||
|
||||
**AC-2: FT-N-02 GET vehicle 404 returns the standard envelope**
|
||||
Given any DB state and a valid JWT
|
||||
When `GET /vehicles/{random uuid}` is issued
|
||||
Then response is `404` with body parsing to JSON object having EXACTLY the keys `statusCode` and `message`, and `statusCode == 404`
|
||||
|
||||
**AC-3: FT-N-03 delete in-use vehicle returns 409 and leaves row**
|
||||
Given one vehicle and ≥ 1 mission referencing it
|
||||
When `DELETE /vehicles/{id}` is issued
|
||||
Then response is `409` with envelope `{ statusCode:409, message:<non-empty> }`, and side-channel `SELECT COUNT(*) FROM vehicles WHERE id={id}` returns `1`
|
||||
|
||||
**AC-4: FT-N-04 create mission with bogus VehicleId returns 400 today (carry-forward)**
|
||||
Given `seed_empty`
|
||||
When `POST /missions { Name:"x", VehicleId:<random uuid>, CreatedDate:null }` is issued
|
||||
Then response is `400` with envelope (carry-forward: spec wants 404; the test must include a `// CARRY-FORWARD: expected to flip to 404 when AC-2.2 divergence is closed` comment)
|
||||
And side-channel `SELECT COUNT(*) FROM missions` returns `0`
|
||||
|
||||
**AC-5: FT-N-05 GET mission 404 returns the standard envelope**
|
||||
Given any DB state and a valid JWT
|
||||
When `GET /missions/{random uuid}` is issued
|
||||
Then response is `404` with envelope `{ statusCode:404, message:<non-empty> }`
|
||||
|
||||
**AC-6: FT-N-06 cascade short-circuit issues zero dependency-table DELETEs**
|
||||
Given `fixture_cascade_F3` (seeded chain rooted at `mid`) and a `postgres-test` started with `log_statement=all`
|
||||
When `DELETE /missions/{mid'}` (random UUID, not `mid`) is issued
|
||||
Then response is `404`, side-channel `SELECT COUNT(*) FROM map_objects` is unchanged, AND the `postgres-test` log (or `pg_stat_statements`) shows NO `DELETE FROM map_objects/waypoints/media/annotations/detection` SQL emitted by the request connection
|
||||
|
||||
**AC-7: FT-N-07 waypoint operation against missing mission returns 404**
|
||||
Given any DB state and a valid JWT
|
||||
When `GET /missions/{random uuid}/waypoints` is issued
|
||||
Then response is `404` with envelope `{ statusCode:404, message:<non-empty> }`
|
||||
|
||||
**AC-8: FT-N-08 generic 500 redacts body, stacktrace lands in log**
|
||||
Given side-channel has executed `DROP TABLE vehicles CASCADE`
|
||||
When `GET /vehicles/{any uuid}` is issued with JWT `FL`
|
||||
Then response is `500` with body EXACTLY `{ "statusCode":500, "message":"Internal server error" }`
|
||||
And `docker logs missions-sut` contains an `"Unhandled exception"` line emitted ≤ 2s after the request timestamp, containing the exception type name (`PostgresException` or similar)
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
**Performance**
|
||||
- FT-N-01..05, FT-N-07: ≤ 2s each. FT-N-06: ≤ 5s. FT-N-08: ≤ 5s (allow log scrape).
|
||||
|
||||
**Reliability**
|
||||
- FT-N-06 requires `postgres-test` to be started with `log_statement=all` (`command: ["postgres", "-c", "log_statement=all"]` overlay in `docker-compose.test.yml`, OR `ALTER SYSTEM SET` via side-channel in the fixture). The test must FAIL if logging is not enabled — not silently pass.
|
||||
- FT-N-08 is destructive (drops the `vehicles` table). It MUST run in its own xUnit `[Collection("ErrorEnvelope500")]` with `ComposeRestartFixture` teardown (full `down -v && up -d`).
|
||||
|
||||
## Blackbox Tests
|
||||
|
||||
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|
||||
|--------|------------------------|-------------|-------------------|----------------|
|
||||
| AC-1 | `seed_3_vehicles_2_default` | `?name=ZZ` then `?name=zz` | `200` + `body.length == 0` for both | AC-1.6 |
|
||||
| AC-2 | any | `GET /vehicles/{random}` | `404` + envelope | AC-1.7, AC-8.2 |
|
||||
| AC-3 | Vehicle + mission referencing it | `DELETE /vehicles/{id}` | `409` + row preserved | AC-1.8, AC-8.5 |
|
||||
| AC-4 | `seed_empty` | `POST /missions { VehicleId:<random> }` | `400` (today) + no row written + carry-forward comment | AC-2.2 |
|
||||
| AC-5 | any | `GET /missions/{random}` | `404` + envelope | AC-2.4, AC-8.2 |
|
||||
| AC-6 | `fixture_cascade_F3` + PG logging on | `DELETE /missions/{random}` | `404` + zero dependency-table DELETE SQL | AC-3.2 |
|
||||
| AC-7 | any | `GET /missions/{random}/waypoints` | `404` + envelope | AC-4.2 |
|
||||
| AC-8 | side-channel DROPped vehicles | `GET /vehicles/{any}` | `500` + redacted body + stacktrace logged within 2s | AC-8.6, AC-10.3 |
|
||||
|
||||
## Constraints
|
||||
|
||||
- HTTP only against `http://missions:8080`; bearer token via `https://jwks-mock:8443/sign` with `permissions=FL`.
|
||||
- FT-N-06 requires Postgres logging mode `log_statement=all`; the fixture must verify (via `SHOW log_statement`) that logging is on BEFORE running the test — fail in Arrange if not.
|
||||
- FT-N-08 fixture teardown must restart the compose stack (`down -v && up -d`); subsequent tests would otherwise hit a missing table.
|
||||
- AAA pattern with `// Arrange` / `// Act` / `// Assert` per test.
|
||||
- Carry-forward comments (FT-N-04) are required so future spec-vs-code work knows where to update.
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
**Risk 1: FT-N-06 false-pass when PG logging is off**
|
||||
- *Risk*: If `postgres-test` runs without `log_statement=all`, the "no DELETE issued" assertion trivially passes — the log is empty.
|
||||
- *Mitigation*: Arrange phase runs `SHOW log_statement` via side-channel and fails fast if the result is not `"all"`. The compose overlay setting this MUST be loaded.
|
||||
|
||||
**Risk 2: FT-N-08 leaves the SUT in a broken state**
|
||||
- *Risk*: After `DROP TABLE vehicles CASCADE`, every subsequent test against `/vehicles` returns 500 until the migrator re-creates the table on next startup.
|
||||
- *Mitigation*: Fixture runs `docker compose -f docker-compose.test.yml down -v && up -d` in teardown; subsequent tests wait for `missions` to reach `healthy`.
|
||||
|
||||
**Risk 3: FT-N-04 expectation flips silently when spec divergence closes**
|
||||
- *Risk*: When the spec-aligned 404 lands, this test will fail with a status mismatch — and the test author needs context to know it's intentional.
|
||||
- *Mitigation*: The test includes a `// CARRY-FORWARD: AC-2.2 — expected to flip to 404 when bogus-VehicleId divergence is closed` source-level comment AND `[Trait("carry_forward", "AC-2.2")]` so a future filter can find it.
|
||||
|
||||
## System Under Test Boundary
|
||||
|
||||
- Tests drive the product through the public HTTP surface (`http://missions:8080/{vehicles,missions}*`) plus the documented DB side-channel for fixture seeding, post-call assertions, and (for FT-N-06) reading `pg_stat_statements` / Postgres log lines, and (for FT-N-08) reading `docker logs missions-sut`. Expected outputs are compared against `_docs/00_problem/input_data/expected_results/results_report.md` rows AC-1 1.7, 1.8, 1.9; AC-2 2.2, 2.6; AC-3 3.2; AC-4 4.1; AC-8 8.7; AC-10 10.1.
|
||||
- Stubs are allowed ONLY for: the external `admin` JWT issuer (`jwks-mock` container) and the DB-only stub tables for `media`, `annotations`, `detection`, `map_objects` (seeded via side-channel SQL).
|
||||
- Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including `VehicleService`, `MissionService`, `WaypointService`, the controllers, `ErrorHandlingMiddleware`, `AppDataConnection`, `DatabaseMigrator`, or `JwtExtensions`. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.
|
||||
Reference in New Issue
Block a user