Adds 26 blackbox tests (FT-P-01..18, FT-N-01..08) covering full AC
matrices for Vehicles/Missions/Waypoints/Health/Errors. Three
spec-vs-code carry-forwards documented in batch_02_report.md and
pinned with [Trait("carry_forward", ...)].
Shared scaffolding: ApiDtos.cs, AssertProblemEnvelopeAsync helper,
Seeds.cs, StubSchema.cs, CascadeF3/F4 fixtures, PostgresStopStart
fixture (gated by COMPOSE_RESTART_ENABLED). Removes the 4 placeholder
Sanity.cs files (now superseded). docker-compose.test.yml gains the
expected_results volume mount + FIXTURE_SQL_DIR for the consumer.
Co-authored-by: Cursor <cursoragent@cursor.com>
10 KiB
Validation + 404 + Authz Negative Tests
Task: AZ-580_test_validation_authz_negative Name: Functional negative tests (FT-N-01..08) Description: Implement xUnit blackbox tests for the 8 negative scenarios — case-insensitive filter no-match, 404 for missing GET vehicle/mission/waypoint-parent, 409 for delete-vehicle-in-use, 400 for create-mission-with-bogus-VehicleId (carry-forward divergence), cascade short-circuit on missing mission (no dependency DELETEs issued), and the generic 500 redacted-body + stacktrace-in-log contract. Complexity: 3 points Dependencies: AZ-576_test_infrastructure Component: Blackbox Tests Tracker: AZ-580 Epic: AZ-575
Problem
The negative-path contract is what protects clients from undefined behaviour: every documented failure must produce a predictable status code + { statusCode, message } envelope, and no failure mode may silently mutate state. Three behaviors are especially load-bearing: (1) DELETE /missions/{missing} must 404 before any dependency-table DELETE issues — otherwise a typo'd UUID could remove rows from map_objects belonging to a different mission (AC-3.2); (2) DELETE /vehicles/{used} must 409 and leave the row in place (AC-1.8); (3) the generic 500 must redact internals — Internal server error body, full stack only in container logs (AC-8.6, AC-10.3).
Outcome
- All eight FT-N-01..08 scenarios run against the dockerised
missionsservice and pass. - Each test produces a CSV row with
Category=Blackbox(negative subset;Traces=AC-1.6, AC-1.7, AC-1.8, AC-2.2, AC-2.4, AC-3.2, AC-4.2, AC-8.6, AC-10.3),Result=pass. - The 500 test asserts BOTH that the body is exactly
{ "statusCode":500, "message":"Internal server error" }AND that the container log emitted an"Unhandled exception"line within 2s. - FT-N-06 asserts via
pg_stat_statements(or post-request log scrape) that NODELETE FROM map_objects/waypoints/media/annotations/detectionSQL ran during the 404 request — the existence check short-circuits before the cascade. - FT-N-04 explicitly pins the documented spec-divergence (returns 400 today, spec wants 404); test must include a comment marking it as a carry-forward to revisit when the divergence is closed.
Scope
Included
- FT-N-01 Vehicle name filter no-match —
?name=ZZand?name=zzagainstseed_3_vehicles_2_defaultboth returnbody.length == 0. - FT-N-02 GET vehicle 404 — random UUID returns
{ statusCode:404, message:… }. - FT-N-03 Delete vehicle in use 409 — row not deleted afterwards.
- FT-N-04 Create mission with bogus VehicleId returns 400 today (CARRY-FORWARD comment).
- FT-N-05 GET mission 404 — envelope shape.
- FT-N-06 Cascade short-circuit — 404 + zero DELETE SQL issued.
- FT-N-07 Waypoint operation against missing mission — 404.
- FT-N-08 Generic 500 — redacted body + stacktrace in log.
Excluded
- 401 / 403 auth-failure paths (NFT-SEC-01..06) live in Task 14.
- 400/422 spec-divergence carry-forwards that are NOT executable today (input validation for empty
Name, negativeBatteryCapacity, unknownTypeint) are documented as Refactor Backlog items intests/blackbox-tests.mdand are NOT in scope here.
Acceptance Criteria
AC-1: FT-N-01 vehicle filter no-match returns empty array for both casings
Given seed_3_vehicles_2_default (BR-01, BR-02, MQ-9)
When GET /vehicles?name=ZZ then GET /vehicles?name=zz are issued
Then both responses are 200 with body.length == 0
AC-2: FT-N-02 GET vehicle 404 returns the standard envelope
Given any DB state and a valid JWT
When GET /vehicles/{random uuid} is issued
Then response is 404 with body parsing to JSON object having EXACTLY the keys statusCode and message, and statusCode == 404
AC-3: FT-N-03 delete in-use vehicle returns 409 and leaves row
Given one vehicle and ≥ 1 mission referencing it
When DELETE /vehicles/{id} is issued
Then response is 409 with envelope { statusCode:409, message:<non-empty> }, and side-channel SELECT COUNT(*) FROM vehicles WHERE id={id} returns 1
AC-4: FT-N-04 create mission with bogus VehicleId returns 400 today (carry-forward)
Given seed_empty
When POST /missions { Name:"x", VehicleId:<random uuid>, CreatedDate:null } is issued
Then response is 400 with envelope (carry-forward: spec wants 404; the test must include a // CARRY-FORWARD: expected to flip to 404 when AC-2.2 divergence is closed comment)
And side-channel SELECT COUNT(*) FROM missions returns 0
AC-5: FT-N-05 GET mission 404 returns the standard envelope
Given any DB state and a valid JWT
When GET /missions/{random uuid} is issued
Then response is 404 with envelope { statusCode:404, message:<non-empty> }
AC-6: FT-N-06 cascade short-circuit issues zero dependency-table DELETEs
Given fixture_cascade_F3 (seeded chain rooted at mid) and a postgres-test started with log_statement=all
When DELETE /missions/{mid'} (random UUID, not mid) is issued
Then response is 404, side-channel SELECT COUNT(*) FROM map_objects is unchanged, AND the postgres-test log (or pg_stat_statements) shows NO DELETE FROM map_objects/waypoints/media/annotations/detection SQL emitted by the request connection
AC-7: FT-N-07 waypoint operation against missing mission returns 404
Given any DB state and a valid JWT
When GET /missions/{random uuid}/waypoints is issued
Then response is 404 with envelope { statusCode:404, message:<non-empty> }
AC-8: FT-N-08 generic 500 redacts body, stacktrace lands in log
Given side-channel has executed DROP TABLE vehicles CASCADE
When GET /vehicles/{any uuid} is issued with JWT FL
Then response is 500 with body EXACTLY { "statusCode":500, "message":"Internal server error" }
And docker logs missions-sut contains an "Unhandled exception" line emitted ≤ 2s after the request timestamp, containing the exception type name (PostgresException or similar)
Non-Functional Requirements
Performance
- FT-N-01..05, FT-N-07: ≤ 2s each. FT-N-06: ≤ 5s. FT-N-08: ≤ 5s (allow log scrape).
Reliability
- FT-N-06 requires
postgres-testto be started withlog_statement=all(command: ["postgres", "-c", "log_statement=all"]overlay indocker-compose.test.yml, ORALTER SYSTEM SETvia side-channel in the fixture). The test must FAIL if logging is not enabled — not silently pass. - FT-N-08 is destructive (drops the
vehiclestable). It MUST run in its own xUnit[Collection("ErrorEnvelope500")]withComposeRestartFixtureteardown (fulldown -v && up -d).
Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|---|---|---|---|---|
| AC-1 | seed_3_vehicles_2_default |
?name=ZZ then ?name=zz |
200 + body.length == 0 for both |
AC-1.6 |
| AC-2 | any | GET /vehicles/{random} |
404 + envelope |
AC-1.7, AC-8.2 |
| AC-3 | Vehicle + mission referencing it | DELETE /vehicles/{id} |
409 + row preserved |
AC-1.8, AC-8.5 |
| AC-4 | seed_empty |
POST /missions { VehicleId:<random> } |
400 (today) + no row written + carry-forward comment |
AC-2.2 |
| AC-5 | any | GET /missions/{random} |
404 + envelope |
AC-2.4, AC-8.2 |
| AC-6 | fixture_cascade_F3 + PG logging on |
DELETE /missions/{random} |
404 + zero dependency-table DELETE SQL |
AC-3.2 |
| AC-7 | any | GET /missions/{random}/waypoints |
404 + envelope |
AC-4.2 |
| AC-8 | side-channel DROPped vehicles | GET /vehicles/{any} |
500 + redacted body + stacktrace logged within 2s |
AC-8.6, AC-10.3 |
Constraints
- HTTP only against
http://missions:8080; bearer token viahttps://jwks-mock:8443/signwithpermissions=FL. - FT-N-06 requires Postgres logging mode
log_statement=all; the fixture must verify (viaSHOW log_statement) that logging is on BEFORE running the test — fail in Arrange if not. - FT-N-08 fixture teardown must restart the compose stack (
down -v && up -d); subsequent tests would otherwise hit a missing table. - AAA pattern with
// Arrange/// Act/// Assertper test. - Carry-forward comments (FT-N-04) are required so future spec-vs-code work knows where to update.
Risks & Mitigation
Risk 1: FT-N-06 false-pass when PG logging is off
- Risk: If
postgres-testruns withoutlog_statement=all, the "no DELETE issued" assertion trivially passes — the log is empty. - Mitigation: Arrange phase runs
SHOW log_statementvia side-channel and fails fast if the result is not"all". The compose overlay setting this MUST be loaded.
Risk 2: FT-N-08 leaves the SUT in a broken state
- Risk: After
DROP TABLE vehicles CASCADE, every subsequent test against/vehiclesreturns 500 until the migrator re-creates the table on next startup. - Mitigation: Fixture runs
docker compose -f docker-compose.test.yml down -v && up -din teardown; subsequent tests wait formissionsto reachhealthy.
Risk 3: FT-N-04 expectation flips silently when spec divergence closes
- Risk: When the spec-aligned 404 lands, this test will fail with a status mismatch — and the test author needs context to know it's intentional.
- Mitigation: The test includes a
// CARRY-FORWARD: AC-2.2 — expected to flip to 404 when bogus-VehicleId divergence is closedsource-level comment AND[Trait("carry_forward", "AC-2.2")]so a future filter can find it.
System Under Test Boundary
- Tests drive the product through the public HTTP surface (
http://missions:8080/{vehicles,missions}*) plus the documented DB side-channel for fixture seeding, post-call assertions, and (for FT-N-06) readingpg_stat_statements/ Postgres log lines, and (for FT-N-08) readingdocker logs missions-sut. Expected outputs are compared against_docs/00_problem/input_data/expected_results/results_report.mdrows AC-1 1.7, 1.8, 1.9; AC-2 2.2, 2.6; AC-3 3.2; AC-4 4.1; AC-8 8.7; AC-10 10.1. - Stubs are allowed ONLY for: the external
adminJWT issuer (jwks-mockcontainer) and the DB-only stub tables formedia,annotations,detection,map_objects(seeded via side-channel SQL). - Stubs, fakes, deterministic fallbacks, monkeypatches, or direct imports are NOT allowed for any internal product module — including
VehicleService,MissionService,WaypointService, the controllers,ErrorHandlingMiddleware,AppDataConnection,DatabaseMigrator, orJwtExtensions. If any of these is not implemented, the test MUST fail/block as missing product implementation — it must not pass by replacing the module with a test stub.