docs+src: complete Steps 1-3 outcomes + auth re-sync baseline

This commit captures everything produced during autodev existing-code
Steps 1 (Document), 2 (Architecture Baseline Scan), and 3 (Test Spec),
together with the targeted auth + CORS re-sync triggered on 2026-05-14
when codebase drift was detected at Step 4 entry. None of this work was
previously committed.

Step 1 (Document) — 50+ _docs/02_document/ files: problem, solution,
architecture, system flows, glossary, module-layout, per-component
specs (01..06), modules, deployment, diagrams, data model, FINAL
report, verification log, discovery.

Step 2 (Architecture Baseline) — architecture_compliance_baseline.md.
Verdict PASS_WITH_WARNINGS (0 Critical, 0 High, 1 Medium, 2 Low). No
High/Critical findings; auto-chained to Step 3 per existing-code flow.

Step 3 (Test Spec) — _docs/02_document/tests/* (67 scenarios across
blackbox, security, resilience, resource-limit, performance), plus
e2e/docker-compose.test.yml, e2e/seed/run.sh, scripts/run-tests.sh,
scripts/run-performance-tests.sh. Coverage 88% over the active scope
(40 of 45 items covered, 6 RB-deferred, 5 documented-as-uncovered).

Targeted auth + CORS re-sync — replaces the deleted in-house token
issuer with a JWKS-verifier model. AuthController and TokenService
removed; JwtExtensions switched from HS256 symmetric to ES256 over
admin's JWKS. ConfigurationResolver and CorsConfigurationValidator
added under src/Infrastructure/. ADR-002 and ADR-006 retired; SEC-01,
SEC-02, SEC-03 marked Closed. One new testability risk recorded in
architecture.md Open Risks Section 6 (JWKS HTTPS gating).

Source changes:
- src/Auth/JwtExtensions.cs (modified) — ES256, JWKS, alg pinning
- src/Program.cs (modified) — DI wiring for ConfigurationResolver
  and CorsConfigurationValidator
- src/Controllers/AuthController.cs (deleted) — no in-service issuance
- src/Services/TokenService.cs (deleted) — same
- src/Infrastructure/ConfigurationResolver.cs (new)
- src/Infrastructure/CorsConfigurationValidator.cs (new)
- .env.example (new) — required env var documentation
- .gitignore (updated)

Cross-repo coordination: _docs/cross-repo/flights_h1_h2_h3_change_spec
captures the change-spec for downstream services that consumed the now
deleted /auth endpoints.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-14 20:19:05 +03:00
parent 08eadc1158
commit 03f879206e
66 changed files with 6006 additions and 133 deletions
+73
View File
@@ -0,0 +1,73 @@
# Azaion.Annotations — Acceptance criteria (retrospective)
> Every criterion has a measurable value and a code/config evidence pointer. **No automated test suite exists in the repo today** (`_docs/02_document/00_discovery.md`), so the criteria below are derived from validation rules, configuration limits, and explicit code branches — they are the contract a future test suite (autodev existing-code Step 3 + Step 6) must encode. Criteria that depend on a Refactor Backlog item landing first are flagged with `[after RB-XX]`.
## Functional — annotation lifecycle (component `01 annotations-rest`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-01 | `POST /annotations` with `image_bytes + detections` for the same payload returns the same `id` on every call. | `id` is `XxHash3.Hash128` (32 hex chars) of the sampled `image_bytes` window — byte-stable. | `Services/AnnotationService.cs` `GenerateAnnotationId(...)` (post RB-04 — currently `XxHash64`). |
| AC-F-02 | A repeat `POST /annotations` for an existing id is a no-op write (no duplicate row, no duplicate file). | DB reads return existing row before insert; file write is `WriteAllBytesAsync` overwriting same bytes. | `Services/AnnotationService.cs`. |
| AC-F-03 | `POST /annotations` writes a YOLO-format label file at `images_dir/<id>.txt` containing one line per detection: `<class_id> <cx> <cy> <w> <h>`. | Exact format with space-separated floats, normalised 0..1, line-per-detection. | `Services/AnnotationService.cs` (label-file write site). |
| AC-F-04 | `POST /annotations` returns HTTP 200 with the persisted entity (id, status, detections). | Response shape mirrors `AnnotationDto`. | `Controllers/AnnotationsController.cs`. |
| AC-F-05 | `[after RB-01]` Every successful `POST/PUT/PATCH/DELETE /annotations/*` emits exactly one SSE event AND inserts exactly one `annotations_queue_records` row, with the correct `QueueOperation` enum. | Created=10, Updated=20, Deleted=40. | `_docs/02_document/architecture.md` ADR-009; `Services/QueueOperation.cs`. |
| AC-F-06 | `[after RB-01]` `DELETE /annotations/{id}` flips the row to `Status=Deleted (40)`, relocates `images_dir/<id>.{jpg,txt}` to `deleted_dir/`, and emits the `Deleted` lifecycle event. | Row count unchanged; files moved; status transitions per `AnnotationStatus`. | `_docs/02_document/architecture.md` ADR-009 + glossary "Soft-delete". |
| AC-F-07 | `[after RB-01 + RB-08]` Soft-deleted rows do not appear in `GET /annotations` or `GET /dataset` results. | Filter `WHERE Status <> 40` enforced at every read path. | RB-01, RB-08 in `_docs/02_document/architecture.md`. |
| AC-F-08 | `[after RB-02]` There is no `silent_detection` column, field, DTO property, or branch in code. | Schema diff + grep produces zero matches. | RB-02. |
## Functional — realtime + sync (component `02 annotations-realtime-sync`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-10 | A connected SSE client receives the lifecycle event for a successful `POST /annotations` within 1 second of the response. | <1s P99 in single-instance, single-pod local run. | `Services/AnnotationEventService.cs`. |
| AC-F-11 | A subscriber that joins **after** the event has been published does not receive it (channel is fire-and-forget). | No backfill replay in `Channel<>`. | ADR-001. |
| AC-F-12 | `FailsafeProducer` consumes a row from `annotations_queue_records` and publishes a MessagePack-gzip frame to the `azaion-annotations` stream within the configured drain interval. | Drain loop interval is the configured cadence; row deletion happens after stream confirm. | `Services/FailsafeProducer.cs`. |
| AC-F-13 | `[after RB-09]` Every wire message carries `(annotation_id, operation, date_time)` so a downstream consumer can dedupe re-deliveries. | Three fields present on the wire schema. | `_docs/02_document/architecture.md` ADR-013. |
## Functional — media (component `03 media`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-20 | `POST /media` accepts a multipart upload, persists the file to the configured directory, and returns the persisted `MediaDto`. | HTTP 200 + JSON body. | `Controllers/MediaController.cs`, `Services/MediaService.cs`. |
| AC-F-21 | `POST /media/batch` accepts N files in one request, writes N rows + N files, and returns N persisted DTOs. | N inputs → N outputs, atomic per file. | same. |
## Functional — dataset (component `04 dataset`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-30 | `GET /dataset` honors filter parameters (mission id, status, class). | Returned rows match filter conditions. | `Controllers/DatasetController.cs`, `Services/DatasetService.cs`. |
| AC-F-31 | `POST /dataset/status/bulk` flips status on N rows in a single SQL statement. | One UPDATE WHERE id IN (…). | `Services/DatasetService.cs`. |
## Functional — settings & metadata (component `05 settings-metadata`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-40 | `PUT /settings/directories` persists changes and triggers `pathResolver.Reset()` so subsequent path lookups reflect the new values. | Verified — `Services/SettingsService.cs:71, 85`. | `Services/SettingsService.cs`. |
| AC-F-41 | `GET /classes` returns the 19 seeded detection classes (ids 018: `ArmorVehicle, Truck, Vehicle, Artillery, Shadow, Trenches, MilitaryMan, TyreTracks, AdditionArmoredTank, Smoke, Plane, Moto, CamouflageNet, CamouflageBranches, Roof, Building, Caponier, Ammo, Protect.Struct`). | 19 rows; ids stable from `DatabaseMigrator`. | `Database/DatabaseMigrator.cs:101-121`. |
| AC-F-42 | `[after RB-06]` `[ADM]` write endpoints exist for `/classes`; the in-memory cache invalidates on write via `Reset()`. | Cache hit ratio observable; cache miss on each write. | RB-06. |
## Functional — auth & platform (component `06 platform`)
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-F-50 | A request bearing an ES256 access token issued by admin (`iss = JWT_ISSUER`, `aud = JWT_AUDIENCE`, signature verifies against the JWKS at `JWT_JWKS_URL`, `exp` in the future) reaches the controller. Tokens that fail issuer / audience / signature / lifetime validation, or whose `alg` is not `ES256`, return HTTP 401. | `JwtBearerHandler` defaults + `AddJwtAuth` parameters. | `Auth/JwtExtensions.cs`. |
| AC-F-51 | Annotations does not host any token-issuance or token-refresh endpoint. Long-running callers refresh against admin's `POST /token/refresh` and pass the resulting access token to annotations. | No `[AllowAnonymous]` route except `/health`; `AuthController` removed. | `Program.cs`, suite admin docs. |
| AC-F-52 | Endpoints under policy `ANN` reject callers without that role with HTTP 403. Endpoints under `DATASET` reject non-DATASET callers with HTTP 403. Endpoints under `ADM` reject non-ADM with HTTP 403. | `Authorization` middleware. | `Auth/JwtExtensions.cs`. |
| AC-F-53 | All errors are returned in the `{ error: { code, message, …details } }` envelope. | Single envelope shape across all controllers. | `Middleware/ErrorHandlingMiddleware.cs`, `_docs/02_document/common-helpers/01_http-error-envelope.md`. |
| AC-F-54 | `GET /health` returns HTTP 200 within 5 seconds of process start (Dockerfile `HEALTHCHECK`). | 200 OK on `/health`. | `Dockerfile`, `Program.cs`. |
## Non-functional
| ID | Criterion | Measurable value | Evidence |
|----|-----------|------------------|----------|
| AC-N-01 | Container boot to `/health` 200 ≤ Docker `HEALTHCHECK` interval/timeout configured in the suite-level orchestrator. | Per `Dockerfile` HEALTHCHECK directive (consult orchestrator config for actual values). | `Dockerfile`. |
| AC-N-02 | `DatabaseMigrator.MigrateAsync()` is idempotent — second boot against the same DB makes no schema changes. | `IF NOT EXISTS` / `ON CONFLICT DO NOTHING` everywhere. | `Database/DatabaseMigrator.cs`. |
| AC-N-03 | `FailsafeProducer` keeps `annotations_queue_records` depth bounded under steady-state lifecycle traffic. | Queue depth metric (to be exposed during Step 14 Observability work). | `Services/FailsafeProducer.cs`. |
| AC-N-04 | The service emits zero unhandled exceptions to clients — every uncaught exception is mapped via `ErrorHandlingMiddleware` into the error envelope. | Middleware terminal handler. | `Middleware/ErrorHandlingMiddleware.cs`. |
| AC-N-05 | Single SSE connection survives ≥ 30 minutes idle with bounded memory (channel is unbounded; growth must come from real traffic, not heartbeats). | Heap stable across 30-minute idle window. | `Services/AnnotationEventService.cs`. |
## Gaps acknowledged
- No measurable latency / throughput targets (P50, P95, P99) are stated anywhere in code. Need to be set during Step 15 (Performance Test).
- No security audit findings yet (Step 14). Items like JWT issuer validation, CORS tightening, and Swagger gating are planned, not yet acceptance criteria.
- No backup / RPO / RTO contract for `images_dir` and `deleted_dir` — the storage layer is treated as durable by assumption.
@@ -0,0 +1,92 @@
# Azaion.Annotations — Input data parameters
> Inventory of every external input the service accepts, with shape, evidence, and validation behavior. Sources: REST DTOs, multipart form fields, env vars, database seed contract.
## REST API inputs
### `POST /annotations`
| Field | Type | Required | Constraint | Evidence |
|-------|------|----------|-----------|----------|
| `image_bytes` | `byte[]` (base64-encoded JSON or multipart) | yes | none enforced; sampled by `XxHash3.Hash128` (per RB-04) for id derivation | `Services/AnnotationService.cs` `GenerateAnnotationId(...)` |
| `mission_id` | `Guid` (post RB-07; today `flight_id`) | yes | foreign-key style, but no FK enforced in schema today | `Entities/AnnotationEntity.cs`, `_docs/02_document/glossary.md` |
| `media_type` | `MediaType` enum | yes | `Image=10` or `Video=20`; integer wire format | `Models/Wire/MediaType.cs` |
| `detections` | `Detection[]` | yes (≥0) | each detection: class id, normalised cx/cy/w/h, confidence | `Models/Dto/DetectionDto.cs` |
| `metadata` | optional payload | no | passes through to row | `AnnotationDto.cs` |
### `PUT /annotations/{id}` and `PATCH /annotations/{id}/status`
Same DTO shape on the body; `id` from path. Status transition values come from the `AnnotationStatus` wire enum: `Pending=10, Accepted=20, Rejected=30, Deleted=40`.
### `DELETE /annotations/{id}`
Path param only. Soft-deletes the row (sets status to `Deleted=40`) and relocates files to `deleted_dir` (per RB-01 + glossary "Soft-delete").
### `GET /annotations` and `/dataset`
Query string filters: `mission_id`, `status`, `class_id`, paging (`offset`, `limit`). Validation is implicit through `[FromQuery]` model binding — no explicit validators visible at controller level.
### `POST /media` and `POST /media/batch`
Multipart form: `IFormFile` / `IFormFileCollection`, `mission_id`, `media_type`. No format whitelist visible at controller layer (verify in Step 14).
### `GET /media/{id}/file`, `GET /media/{id}/thumbnail`
Path param only; returns binary stream.
### Auth endpoints
Annotations no longer hosts `POST /auth/login`, `POST /auth/refresh`, or `POST /auth/register`. Token issuance and refresh are owned by the **admin** service. The only auth-related input on the annotations surface is the `Authorization: Bearer <token>` HTTP header on every non-`/health` request, validated by `JwtBearerHandler` against admin's JWKS:
| Header | Required | Notes |
|--------|----------|-------|
| `Authorization` | yes (everywhere except `/health`) | `Bearer <ES256 JWT>` issued by admin; `iss` / `aud` / `exp` / signature all validated; `alg` pinned to `ES256` |
### `/settings/*`
Each controller binds JSON DTOs from `Models/Dto/*` mirroring the `system_settings`, `directory_settings`, `camera_settings`, `user_settings` shapes in `Database/DatabaseMigrator.cs`.
## Database seed inputs (boot-time)
`DatabaseMigrator` issues `ON CONFLICT DO NOTHING` inserts on:
| Table | Seeded rows |
|-------|-------------|
| `directory_settings` | one row with default paths |
| `system_settings` | one row (today still includes `silent_detection`; removal tracked by RB-02) |
| `detection_classes` | 19 rows (ids 018): `ArmorVehicle, Truck, Vehicle, Artillery, Shadow, Trenches, MilitaryMan, TyreTracks, AdditionArmoredTank, Smoke, Plane, Moto, CamouflageNet, CamouflageBranches, Roof, Building, Caponier, Ammo, Protect.Struct` (`Smoke` and `Plane` share color `#000080` — pre-existing data quirk, fixed by RB-06) |
(There is no `users` table in this service — identity is owned by the admin service.)
Detection class catalog becomes admin-CRUD after RB-06.
## Environment variables (process inputs)
| Name | Required | Default | Purpose |
|------|----------|---------|---------|
| `DATABASE_URL` | yes | none — fail-fast (`ConfigurationResolver`) | Postgres connection; URI form auto-converted to Linq2DB form |
| `JWT_ISSUER` | yes | none — fail-fast | Expected `iss` claim (admin's issuer) |
| `JWT_AUDIENCE` | yes | none — fail-fast | Expected `aud` claim (this service) |
| `JWT_JWKS_URL` | yes | none — fail-fast; HTTPS required | Admin's JWKS endpoint for ES256 key resolution |
| `CorsConfig:AllowedOrigins` | yes (prod, unless `AllowAnyOrigin=true`) | empty | Configured origins for the default CORS policy |
| `CorsConfig:AllowAnyOrigin` | optional | `false` | Explicit opt-in to permissive CORS (validator blocks empty allow-list in `Production` unless this is set) |
| `RABBITMQ_HOST` | optional | `127.0.0.1` | stream broker host |
| `RABBITMQ_STREAM_PORT` | optional | `5552` | stream listener port |
| `RABBITMQ_PRODUCER_USER` | optional | `azaion_producer` | stream auth user |
| `RABBITMQ_PRODUCER_PASS` | optional | `producer_pass` | stream auth pass |
| `AZAION_REVISION` | optional | `unknown` | image build stamp; logged at boot |
| `ASPNETCORE_URLS` | optional | `http://+:8080` | bind address |
| `ASPNETCORE_ENVIRONMENT` | optional | `Production` | bound to ASP.NET host |
## Stream consumer wire format
Outbound (this service is producer-only):
- Stream name: `azaion-annotations`
- Body: gzip(MessagePack(`AnnotationStreamMessage`))
- Schema fields (post RB-09): `annotation_id`, `operation` (`QueueOperation`), `date_time`, payload — see `_docs/02_document/components/02_annotations-realtime-sync/description.md` and ADR-013.
## Cross-references
- Wire enum table: `_docs/02_document/modules/wire-enums.md`
- ER diagram: `_docs/02_document/data_model.md`
- Common error envelope: `_docs/02_document/common-helpers/01_http-error-envelope.md`
@@ -0,0 +1,173 @@
# Expected Results — Azaion.Annotations
Maps every input data item the test corpus exercises against `Azaion.Annotations` to its quantifiable expected result. Tests use this mapping to compare actual system output against known-correct answers.
This contract is **annotations-service-shape**, not detections-service-shape. The same binary fixtures are reused (see `../fixtures.md`), but the expected outputs here describe annotation lifecycle behavior — content-addressed ids, persisted DTOs, label-file writes, SSE delivery, outbox + stream — not bounding-box inference.
## Result Format Legend
| Result Type | When to Use | Example |
|-------------|-------------|---------|
| Exact value | Output must match precisely | `status_code: 200`, `detection_count: 3` |
| Tolerance range | Numeric output with acceptable variance | `latency: 800ms ± 200ms` |
| Threshold | Output must exceed or stay below a limit | `latency ≤ 1000ms` |
| Pattern match | Output must match a string/regex pattern | `id =~ /^[0-9a-f]{32}$/` |
| File reference | Complex output compared against a reference file | `match expected_results/F1_001_response.json` |
| Schema match | Output structure must conform to a schema | `body matches AnnotationDto` |
| Set/count | Output must contain specific items or counts | `detections.length == 3` |
## Comparison Methods
| Method | Description | Tolerance Syntax |
|--------|-------------|-----------------|
| `exact` | Actual == Expected | N/A |
| `numeric_tolerance` | abs(actual - expected) ≤ tolerance | `± <value>` or `± <percent>%` |
| `threshold_min` | actual ≥ threshold | `≥ <value>` |
| `threshold_max` | actual ≤ threshold | `≤ <value>` |
| `regex` | actual matches regex pattern | regex string |
| `substring` | actual contains substring | substring |
| `json_diff` | structural comparison against reference JSON | diff tolerance per field |
| `schema_match` | actual conforms to a JSON schema | N/A |
| `file_exists` | a file at a computed path exists on disk | N/A |
| `file_content` | a file's contents match expected (line-by-line) | exact / regex |
## Global invariants
These hold for every successful response from the service unless explicitly negated by the row's own expected result.
| Invariant | Comparison | Notes |
|-----------|------------|-------|
| Response Content-Type is `application/json` for non-binary endpoints | exact | except `/health`, image/thumbnail file routes, and SSE (`text/event-stream`) |
| Error responses follow the suite envelope `{ error: { code, message, …details } }` | schema_match | `_docs/02_document/common-helpers/01_http-error-envelope.md` |
| `id` fields in annotation responses are 32 lowercase hex chars | regex `^[0-9a-f]{32}$` | derived from `XxHash3.Hash128` (post RB-04) over sampled image bytes |
| Tokens passed by callers are ES256 JWTs issued by admin (3 base64url segments) | regex `^[\w-]+\.[\w-]+\.[\w-]+$` | annotations does not issue tokens; this is the shape it accepts |
| For `[after RB-XX]` rows: skip until the listed Refactor Backlog item lands | — | Phase 3 validation removes them otherwise |
## Input → Expected Result Mapping
### Group F1 — Annotation create (`POST /annotations`)
Each row uses one binary fixture from `fixtures.md` plus a synthetic `detections[]` payload from `requests/F1_<NNN>_request.json`. The class_num values come from the seeded `detection_classes` (ids 018).
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F1-001 | `image_small` + `requests/F1_001_request.json` (1 detection: class_num=10 Plane, normalized bbox) | Single small frame, single detection | HTTP 200; body matches `AnnotationDto`; `body.detections.length == 1`; `body.id =~ /^[0-9a-f]{32}$/` | exact (status), schema_match (body), regex (id) | N/A | `expected_results/F1_001_response.json` |
| F1-002 | Same as F1-001 (re-POST with identical payload) | Idempotency check | HTTP 200; `body.id == <id from F1-001>`; no duplicate row written (verifiable via `GET /annotations/{id}` returning a single row) | exact | N/A | N/A |
| F1-003 | `image_empty_scene` + `requests/F1_003_request.json` (0 detections) | Frame with no detections | HTTP 200; `body.detections.length == 0`; YOLO label file `<images_dir>/<id>.txt` exists with 0 lines | exact (count), file_exists, file_content | N/A | N/A |
| F1-004 | `image_dense01` + `requests/F1_004_request.json` (5 detections: mixed class_nums 0,1,2,9,10) | Dense scene, multiple classes | HTTP 200; `body.detections.length == 5`; YOLO label file has 5 lines, each `<class_num> <cx> <cy> <w> <h>` with normalized floats | exact (count), file_content (regex per line) | N/A | `expected_results/F1_004_response.json` |
| F1-005 | `image_large` + `requests/F1_005_request.json` (3 detections) | Large payload (~7 MB) | HTTP 200; same shape as F1-001; latency `≤ 5000ms` (single-instance dev DB, no concurrent load) | exact, threshold_max | latency ± 1000ms | N/A |
| F1-006 | `video_short01` (mediaType=Video) + `requests/F1_006_request.json` (1 detection at videoTime=00:00:02.000) | Video frame annotation | HTTP 200; `body.id =~ /^[0-9a-f]{32}$/`; `body.videoTime == "00:00:02"` | exact, regex | N/A | N/A |
| F1-007 | `video_short01` + `video_short02` content-distinct + same detections payload | Distinct image bytes → distinct ids | `body_F1-007_a.id != body_F1-007_b.id` | exact (inequality) | N/A | N/A |
### Group F1-N — Annotation create negative cases
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F1-N-001 | `requests/F1_N_001_request.json` (no `image` bytes) | Missing image bytes | HTTP 400 or 422; error envelope present; `error.code` not empty | exact, schema_match | N/A | N/A |
| F1-N-002 | `requests/F1_N_002_request.json` (image bytes present but `mediaType` missing) | Missing required field | HTTP 400 or 422; error envelope | exact, schema_match | N/A | N/A |
| F1-N-003 | `image_small` + valid payload + JWT with policy `DATASET` only | Caller missing ANN policy | HTTP 403; error envelope `error.code` ∈ {`forbidden`, `policy_denied`} | exact, set_contains | N/A | N/A |
| F1-N-004 | `image_small` + valid payload + no `Authorization` header | Unauthenticated | HTTP 401; error envelope | exact | N/A | N/A |
| F1-N-005 | `image_small` + payload with `detections[0].centerX = 1.5` (out of 0..1 range) | Invalid bbox value | HTTP 200 today (no validator) → flag as documented gap; OR HTTP 400/422 if validation lands per SEC-05 | exact (today: 200) | N/A | N/A |
### Group F2 — Annotation listing & detail (`GET /annotations`, `/annotations/{id}`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F2-001 | `GET /annotations?limit=10` after F1-001..F1-004 succeeded | Paginated list | HTTP 200; `body.length == 4`; each item matches `AnnotationListItem` schema | exact (count), schema_match | N/A | N/A |
| F2-002 | `GET /annotations/{id from F1-001}` | Detail of an existing annotation | HTTP 200; `body.id == <id>`; `body.detections.length == 1` | exact | N/A | `expected_results/F1_001_response.json` (same file as F1-001) |
| F2-003 | `GET /annotations/00000000000000000000000000000000` | Nonexistent id | HTTP 404; error envelope; `error.code` matches `/not.?found/i` | exact, regex | N/A | N/A |
| F2-004 | `GET /annotations?missionId=<unknown-guid>` | Filter by mission with no annotations | HTTP 200; `body.length == 0` | exact | N/A | N/A |
### Group F3 — Realtime SSE (`GET /annotations/events`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F3-001 | Subscriber connects to `/annotations/events`, then F1-001 fires | SSE delivery for new annotation | Subscriber receives one event with `data` parsing as `AnnotationEventDto`; `event.operation == "Created"`; `event.annotationId == <id from F1-001>`; latency `≤ 1000ms` | schema_match, exact, threshold_max | latency ± 200ms | N/A |
| F3-002 | F1-001 fires, then subscriber connects | No backfill expected | Subscriber receives 0 events for the historical id within 5s window | exact (count) | N/A | N/A |
| F3-003 | Subscriber connects without `Authorization` header | Unauthenticated SSE | HTTP 401 on the SSE connection establishment | exact | N/A | N/A |
| F3-004 `[after RB-01]` | Subscriber connects, then `PUT /annotations/{id}` updates fields | Lifecycle observability for Update | Subscriber receives event with `event.operation == "Updated"`, payload reflecting the update | exact, schema_match | N/A | N/A |
| F3-005 `[after RB-01]` | Subscriber connects, then `DELETE /annotations/{id}` | Lifecycle observability for Delete (soft-delete) | Subscriber receives event with `event.operation == "Deleted"`; row status flips to `Deleted (40)`; image+label files relocate to `deleted_dir` | exact, file_exists | N/A | N/A |
### Group F4 — Outbox + Stream (`FailsafeProducer`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F4-001 | F1-001 succeeds | Outbox row inserted | After F1-001 returns 200, exactly one new row exists in `annotations_queue_records` with `annotation_id == <id>`, `operation == 10` (Created) | exact | N/A | N/A |
| F4-002 | After F4-001, wait for one drain cycle | Drainer publishes to RabbitMQ stream | Within `drain_interval + 2s`, the row is deleted AND a message lands on stream `azaion-annotations` | exact, threshold_max | N/A | N/A |
| F4-003 | Inspect the published stream message | Message wire format | gzip-decoded MessagePack body deserializes into the documented schema (`annotationId`, `operation`, `dateTime`, payload) | schema_match | N/A | `expected_results/F4_003_stream_message.json` |
| F4-004 `[after RB-09]` | Two F1-001 invocations with the same image bytes | Stream dedupe contract | Stream messages carry `(annotationId, operation, dateTime)`; a downstream consumer can collapse duplicates by that triple | exact, schema_match | N/A | N/A |
| F4-005 | RabbitMQ unreachable, then F1-001 fires | Drainer survives broker outage | Row stays in `annotations_queue_records` (does not get deleted); `FailsafeProducer` does not crash; queue depth grows; HTTP 200 still returned to the original caller | exact, schema_match | N/A | N/A |
### Group F5 — Media upload (`POST /media`, `POST /media/batch`)
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F5-001 | multipart `POST /media` with `image_small`, `mediaType=Image`, `waypointId=<guid>` | Single media upload | HTTP 200; body matches `MediaListItem`; file exists at `<media_dir>/<media_id>` (extension preserved) | exact, schema_match, file_exists | N/A | N/A |
| F5-002 | multipart `POST /media/batch` with 3 files (`image_small`, `image_dense01`, `image_dense02`) + same waypointId | Batch upload | HTTP 200; `body.length == 3`; 3 distinct `mediaId` values; 3 files on disk | exact, file_exists | N/A | N/A |
| F5-003 | `POST /media` with no `waypointId` | Missing required field | HTTP 400 or 422; error envelope | exact, schema_match | N/A | N/A |
| F5-004 | `POST /media` with caller missing ANN policy | AuthZ check | HTTP 403; error envelope | exact | N/A | N/A |
### Group F6 — Auth verification (Bearer token validation)
Annotations does not host login / refresh / register — those are owned by admin and out-of-scope for this test corpus. The annotations e2e harness runs against an in-stack **mock JWKS issuer** that mints ES256 tokens with the configured `JWT_ISSUER` / `JWT_AUDIENCE`; runtime tokens are minted on demand by the runner (`fixtures/auth/mock_issuer.py` or equivalent) using a private key whose public half lives in the JWKS the service fetches at boot. See `_docs/02_document/tests/test-data.md` and `_docs/02_document/tests/security-tests.md`.
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F6-001 | Any authenticated route called with a freshly minted ES256 token (correct iss / aud / exp) | Happy-path verification | HTTP 200 | exact | N/A | N/A |
| F6-002 | Same route with `iss` mismatched against `JWT_ISSUER` | Issuer rejection | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-003 | Same route with `aud` mismatched against `JWT_AUDIENCE` | Audience rejection | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-004 | Same route with `exp` 1 minute in the past | Expired token | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-005 | Same route with `alg=HS256` and admin's public ES256 key reused as the HMAC key | Algorithm-confusion attack | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
| F6-006 | Same route with no `Authorization` header | Anonymous rejection (except `/health`) | HTTP 401; error envelope | exact, schema_match | N/A | N/A |
### Group F7 — Settings & metadata
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F7-001 | `GET /classes` after fresh boot | Detection class catalog | HTTP 200; `body.length == 19`; ids `[0..18]` present; entry where `id == 9` has `name == "Smoke"`; entry where `id == 10` has `name == "Plane"` | exact, set_contains | N/A | `expected_results/F7_001_classes.json` |
| F7-002 | `GET /settings/system` | System settings read | HTTP 200; `body` matches `SystemSettings` shape; `silent_detection` field present today (removed post RB-02) | exact, schema_match | N/A | N/A |
| F7-003 | `PUT /settings/directories` with new `imagesDir` value | PathResolver invariant | HTTP 200; subsequent `GET /settings/directories` returns the new value; `pathResolver.Reset()` invariant — the next `POST /annotations` writes to the new path | exact, file_exists (under new path) | N/A | N/A |
| F7-004 | `PUT /settings/directories` with caller missing ADM policy | AuthZ check | HTTP 403; error envelope | exact | N/A | N/A |
| F7-005 `[after RB-06]` | `POST /classes` (admin CRUD) with caller having ADM policy | New class added | HTTP 200; `GET /classes` returns 20 rows; cache invalidated | exact | N/A | N/A |
### Group F8 — Dataset
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F8-001 | `GET /dataset?status=10` after F1-001..F1-004 | Filter by status `Pending` | HTTP 200; all returned items have `status == 10`; `body.length` matches the count of Pending rows in DB | exact | N/A | N/A |
| F8-002 | `GET /dataset?classNum=10` | Filter by class `Plane` | HTTP 200; every returned item's annotation has at least one detection with `class_num == 10` | exact | N/A | N/A |
| F8-003 | `GET /dataset/class-distribution` | Class distribution | HTTP 200; `body` is an array; each entry has `classNum`, `label`, `color`, `count`; sum of counts equals total detection count | exact, schema_match | N/A | N/A |
| F8-004 | `POST /dataset/status/bulk` with `{ annotationIds: [<id1>, <id2>], status: 20 }` | Bulk status update | HTTP 200; both annotations have `status == 20` after the call (atomic SQL `UPDATE … WHERE id IN (…)`) | exact | N/A | N/A |
| F8-005 `[after RB-08]` | F8-004 path | Lifecycle event emission | Each updated annotation emits a `Updated` SSE event AND inserts an outbox row | exact, schema_match | N/A | N/A |
### Group F9 — Health & boot
| # | Input | Input Description | Expected Result | Comparison | Tolerance | Reference File |
|---|-------|-------------------|-----------------|------------|-----------|---------------|
| F9-001 | `GET /health` | Health check | HTTP 200; latency `≤ 200ms` | exact, threshold_max | N/A | N/A |
| F9-002 | Container fresh boot, run migrator twice | Migrator idempotence | Second boot makes 0 schema changes (no new tables, no new columns); 0 errors | exact (DDL diff) | N/A | N/A |
## Coverage summary
- **Functional positive**: 26 rows (F1-001..007, F2-001..004, F3-001..005, F4-001..005, F5-001..002, F6-001/004/007, F7-001..003, F8-001..004, F9-001..002).
- **Functional negative**: 12 rows (F1-N-001..005, F2-003, F3-003, F5-003..004, F6-002/003/005/006, F7-004).
- **`[after RB-XX]` rows** (skipped until the backlog item lands): F3-004, F3-005, F4-004, F7-005, F8-005, plus the post-RB-04 hash invariant in F1-001 — 6 deferred rows.
Total: **44 rows**; **38 active today**, **6 deferred behind backlog items**.
## Reference files (to author next)
The rows above reference these reference files in `expected_results/`. They will be authored as part of this skill's Phase 1 input-data analysis if the runner needs them; complex JSON bodies are best captured here once we run F1-001 against a real DB once and capture the response. For the initial spec, the regex/schema_match patterns above are sufficient.
| File | Purpose |
|------|---------|
| `F1_001_response.json` | Reference `AnnotationDto` body for `image_small` + 1 detection |
| `F1_004_response.json` | Reference body for dense scene (5 detections) |
| `F4_003_stream_message.json` | Reference MessagePack-decoded stream payload |
| `F7_001_classes.json` | Reference class catalog (19 rows, ids 018) |
## Open data gaps (raised during this draft)
- **Performance baselines**: `F1-005` and `F9-001` use single-instance latency thresholds (5000ms / 200ms) inferred from the codebase, NOT a contracted SLA. If suite-level perf targets exist, they override these.
- **`F1-N-005` invalid bbox value**: today the service silently accepts out-of-range `centerX`. Documented in `security_approach.md` SEC-05; needs a decision on whether the test should target the current (lenient) or future (validated) behavior.
- **F4-005 outage simulation**: depends on the test harness being able to restart RabbitMQ between cases — operational concern for the runner script (Phase 4).
+41
View File
@@ -0,0 +1,41 @@
# Test Fixtures
Binary fixtures (frame images + videos) live in the **sibling `detections` repo** under `azaion/suite/detections/_docs/00_problem/input_data/`. We do not duplicate them in this repo — the suite layout already collocates the two services and the test runners (Step 4 onward) resolve fixtures via the relative path below.
## Canonical fixture root
```
$SUITE_ROOT/detections/_docs/00_problem/input_data/
```
Where `$SUITE_ROOT` is the parent directory containing both `annotations/` and `detections/`. Test runner scripts (Phase 4) compute this from the running script's location: `dirname "$0"/../../../detections/_docs/00_problem/input_data/`.
## Image fixtures
| Local id | Source path (relative to suite root) | Dimensions | Size | Notes |
|----------|--------------------------------------|------------|------|-------|
| `image_small` | `detections/_docs/00_problem/input_data/image_small.jpg` | 1280 × 720 | ~1.5 MB | Primary single-frame test |
| `image_dense01` | `detections/_docs/00_problem/input_data/image_dense01.jpg` | n/a (~230 KB) | small | Many-detections test |
| `image_dense02` | `detections/_docs/00_problem/input_data/image_dense02.jpg` | n/a (~2.8 MB) | medium | Many-detections + larger payload |
| `image_different_types` | `detections/_docs/00_problem/input_data/image_different_types.jpg` | 900 × 1600 | ~150 KB | Multi-class detection input |
| `image_empty_scene` | `detections/_docs/00_problem/input_data/image_empty_scene.jpg` | 1920 × 1080 | ~2 MB | Zero-detection input |
| `image_large` | `detections/_docs/00_problem/input_data/image_large.JPG` | 6252 × 4168 | ~7 MB | Large payload boundary |
## Video fixtures
| Local id | Source path | Size | Notes |
|----------|-------------|------|-------|
| `video_short01` | `detections/_docs/00_problem/input_data/video_short01.mp4` | ~150 MB | Video annotation flow |
| `video_short02` | `detections/_docs/00_problem/input_data/video_short02.mp4` | ~150 MB | Distinct-bytes second input — for content-addressed-id divergence |
## Synthetic request payloads
Synthetic JSON request bodies (annotation create / update / dataset query / settings update / auth login) live under `_docs/00_problem/input_data/requests/`. They reference image fixtures by `local_id` from the table above; the runner inlines the binary at request time.
## Why path reference, not copy
- The video binaries are ~150 MB each; committing them would bloat this repo.
- Both services live under the same suite, so the relative path is stable.
- The detections team owns the source-of-truth fixtures (frames, videos). The annotations test corpus consumes them with its own contract layer (`expected_results/results_report.md`) — we do not redefine the inputs, only the contract.
If the layout ever diverges (annotations and detections move into different parent directories), `fixtures.md` is the one place to update the path resolution.
+55
View File
@@ -0,0 +1,55 @@
# Azaion.Annotations — Problem statement (retrospective)
> Reverse-engineered from `_docs/02_document/architecture.md`, `system-flows.md`, the per-component specs, and `suite/_docs/01_annotations.md`. Not copied from a real PRD — this is a retrospective synthesis.
## What this system is
`Azaion.Annotations` is the **annotation lifecycle service** of the AZAION suite. It is the single owner of the `annotations` table, the YOLO-format label files on disk, the lifecycle event stream (RabbitMQ + SSE), and the dataset exploration surface that downstream tooling — annotator UIs, the AI training pipeline, the admin sync worker — relies on.
It is a single .NET 10 service backed by PostgreSQL and a content-addressed filesystem cache, packaged as an ARM64 Docker image, deployed by the suite's branch-driven Woodpecker pipeline.
## Problem it solves
The suite's surveillance / detection pipeline produces a continuous stream of detected objects in video frames. Three independent consumers need that same data shaped differently:
1. **Annotator UI** — humans need to review, correct, accept, or reject each detection in near-real-time, frame-by-frame, with the underlying image visible. They need every change another annotator makes to surface immediately on their screen — no refresh button.
2. **AI training pipeline** — needs the *finalized* annotations + image bytes as a durable, replayable feed so it can build training datasets at any cadence.
3. **Suite-level admin worker** — needs an audit-grade record of every state change (who, when, what) for cross-service synchronisation.
Without a dedicated lifecycle service, these consumers would each poll the detection pipeline directly, which (a) doesn't expose lifecycle semantics — only "the model said this", not "a human accepted it", (b) has no notion of soft delete, status transitions, or human authorship, and (c) cannot deliver realtime updates to UIs and durable replay to batch consumers from the same source of truth.
`Azaion.Annotations` solves that three-way mismatch by being **the one place** where annotation state lives, where state transitions are emitted, and where both push (SSE for humans) and durable-pull (RabbitMQ Stream for machines) consumers attach.
## Users (consumer roles)
| Consumer | How they reach the system | What they need |
|----------|---------------------------|----------------|
| Annotator UI (human-facing web app) | REST + SSE, JWT policy `ANN` | List + detail of annotations, mutations, real-time fan-out of every other annotator's edits |
| Dataset Explorer UI | REST under `/dataset`, JWT policy `DATASET` | Filterable read of the current annotation corpus + bulk-status writes |
| Detections service (upstream pipeline) | REST `POST /annotations`, JWT policy `ANN`; long-running tokens are refreshed against admin's `POST /token/refresh` (annotations is verifier-only) | Push raw detections + the original frame image; receive the assigned annotation id |
| AI training pipeline | RabbitMQ Stream consumer (`azaion-annotations`) | Durable, replayable lifecycle events with the full payload |
| Admin sync worker | RabbitMQ Stream consumer (`azaion-annotations`) | Same stream, different consumer offset; cross-service event correlation |
| Suite admin (humans) | REST `[ADM]` endpoints (planned for `/classes` per RB-06; service-account registration is owned by the admin service, not annotations) | Manage detection class catalog, register service accounts (against admin) |
## How it works at a high level
A detection arrives as `POST /annotations` with the original frame as `image_bytes` and a list of YOLO detections in normalised coordinates. The service:
1. **Content-addresses** the image — sampled hashing produces a stable 32-char hex id; identical re-uploads collapse to the same row.
2. **Persists** the image, optionally a media row, the annotation row, and the detection rows in a transactional unit (subject to the agreed Refactor Backlog item RB-03 — today FS + DB + outbox are not yet wrapped together).
3. **Writes** a YOLO `.txt` label file next to the image so the AI training pipeline can ingest the data with no transformation step.
4. **Publishes** an SSE event so every Annotator UI viewing that mission gets the new annotation immediately.
5. **Enqueues** a row in the transactional outbox `annotations_queue_records`. The in-process `FailsafeProducer` drains that outbox into the RabbitMQ stream as a MessagePack-gzip frame, so the AI pipeline and admin worker get a durable copy.
Mutations (Update / UpdateStatus / Delete) follow the same shape — but only after Refactor Backlog item RB-01 lands (today they are silent on both SSE and outbox; that is a known gap, not a design choice). Deletes are *soft*: status flips to `Deleted (40)`, files relocate to a `deleted_dir`, the row stays.
The service also serves the dataset exploration surface (`/dataset/*`), the media upload pipeline (`/media/*`), and the system-metadata catalog (`/settings/*`, `/classes`).
## Cross-references
- Suite-level integration narrative: `suite/_docs/01_annotations.md`
- Architecture vision + 13 ADRs: `_docs/02_document/architecture.md`
- 8 verified system flows F1F8: `_docs/02_document/system-flows.md`
- Component-level specs: `_docs/02_document/components/*/description.md`
- Glossary (canonical terminology): `_docs/02_document/glossary.md`
- README: none in repo (gap noted in `_docs/02_document/00_discovery.md`).
+53
View File
@@ -0,0 +1,53 @@
# Azaion.Annotations — Restrictions
> Only constraints **evidenced in code, configs, or Dockerfiles** are listed. Inferred-but-unverified items are flagged.
## Hardware
| ID | Restriction | Evidence |
|----|-------------|----------|
| HW-01 | Service binary is built for ARM64 only — no AMD64 image is produced. | `.woodpecker/build-arm.yml` (`platforms: linux/arm64`); `Dockerfile` `--arch=$BUILDARCH` driven by `BUILDPLATFORM=linux/arm64`. |
| HW-02 | Local writable filesystem is required at `images_dir` / `videos_dir` / (planned) `deleted_dir`. | `Services/AnnotationService.cs` (`File.WriteAllBytesAsync`), `Services/PathResolver.cs`, `directory_settings` table. |
| HW-03 | Memory pressure scales with the largest single image read into memory by `FailsafeProducer` (re-reads the image to put bytes on the wire). | `Services/FailsafeProducer.cs:138` neighborhood. |
## Software
| ID | Restriction | Evidence |
|----|-------------|----------|
| SW-01 | .NET 10 SDK and runtime — no fallback. | `Dockerfile` `mcr.microsoft.com/dotnet/sdk:10.0`, `aspnet:10.0`. |
| SW-02 | PostgreSQL backend; migrator emits `IF NOT EXISTS`, `ON CONFLICT`, `CREATE TYPE` — Postgres 13+ semantics expected. | `Database/DatabaseMigrator.cs`. |
| SW-03 | RabbitMQ broker with the **streams plugin** enabled — service uses `RabbitMQ.Stream.Client`, not classic queues. | `Services/FailsafeProducer.cs`. |
| SW-04 | Linq2DB ORM, MessagePack with the contractless resolver, gzip wire format. | `Services/FailsafeProducer.cs`. |
| SW-05 | JWT verification is **ES256 over admin's JWKS** (`JWT_JWKS_URL`); `ValidAlgorithms` is pinned to `EcdsaSha256`. Annotations is verifier-only — admin is the sole token issuer for the suite. JWKS retrieval requires HTTPS. | `Auth/JwtExtensions.cs`. |
## Environment
| ID | Restriction | Evidence |
|----|-------------|----------|
| ENV-01 | Required env vars (fail-fast at startup via `ConfigurationResolver`): `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`. Optional with defaults: `RABBITMQ_HOST`, `RABBITMQ_STREAM_PORT`, `RABBITMQ_PRODUCER_USER`, `RABBITMQ_PRODUCER_PASS`. | `Program.cs`, `Infrastructure/ConfigurationResolver.cs`, `Services/FailsafeProducer.cs`. |
| ENV-02 | Service listens on port `8080` HTTP, no TLS terminator inside the image. | `Dockerfile` `EXPOSE 8080`, `ASPNETCORE_URLS=http://+:8080`. |
| ENV-03 | Build stamps `AZAION_REVISION` from CI; `Program.cs` echoes it on startup. | `Dockerfile` `ARG AZAION_REVISION`, `Program.cs`. |
| ENV-04 | Image tag scheme is branch-driven: `${BRANCH}-arm`. No semver tags. | `.woodpecker/build-arm.yml`. |
| ENV-05 | Swagger UI is mounted unconditionally — present in production builds (ADR-005). | `Program.cs`. |
| ENV-06 | CORS is config-driven (`CorsConfig:AllowedOrigins` + opt-in `CorsConfig:AllowAnyOrigin`); `CorsConfigurationValidator.EnsureSafeForEnvironment` refuses to start in `Production` when the allow-list is empty and `AllowAnyOrigin` is not set. ADR-006 retired. | `Program.cs`, `Infrastructure/CorsConfigurationValidator.cs`. |
| ENV-07 | Boot-time `DatabaseMigrator.MigrateAsync()` runs on startup — no separate migration step in the deploy pipeline (ADR-007). | `Program.cs`, `Database/DatabaseMigrator.cs`. |
## Operational
| ID | Restriction | Evidence |
|----|-------------|----------|
| OP-01 | SSE state is per-instance — no broker fan-out — so horizontal scaling is bounded today. | `Services/AnnotationEventService.cs` (in-process `Channel<>`). |
| OP-02 | Outbox drainer has no row-leasing — running multiple instances will double-publish until RB-09 deduplication contract is in place. | `Services/FailsafeProducer.cs`. |
| OP-03 | No automated test suite in repo; CI does build-and-push only. | `_docs/02_document/00_discovery.md`, `.woodpecker/build-arm.yml`. |
| OP-04 | No lint or formatter step in CI. | `.woodpecker/build-arm.yml`. |
| OP-05 | Dockerfile `HEALTHCHECK` calls `/health`; HTTP 200 expected by orchestrator. | `Dockerfile`. |
| OP-06 | The service must be the only writer of `annotations_queue_records` — the table is treated as a private outbox. | `Services/AnnotationService.cs`, `Services/FailsafeProducer.cs`. |
| OP-07 | DB connection string format is the Java/Hikari `jdbc:postgresql://…` style; `Helpers/PostgreSqlConnectionStringHelper` parses it. | `Helpers/PostgreSqlConnectionStringHelper.cs`. |
## Cross-cutting (suite-level, evidence in `suite/_docs/01_annotations.md`)
| ID | Restriction |
|----|-------------|
| SUITE-01 | The shared JWT secret family is cross-service; revoking it invalidates every service token. |
| SUITE-02 | Wire enums for `AnnotationStatus`, `MediaType`, `QueueOperation` are duplicated across services and must move in lock-step (or a single contract has to be published). |
| SUITE-03 | Stream consumers (admin worker, AI training) commit offsets independently — Annotations does not own retention semantics. |
+86
View File
@@ -0,0 +1,86 @@
# Azaion.Annotations — Security approach (retrospective)
> Inventory of the **security mechanisms actually present in code today** + the gaps that the autodev existing-code Step 14 (Security Audit) will close. Evidence anchored to source files. ADR references point to `_docs/02_document/architecture.md`.
## Authentication
- **Mechanism**: JWT Bearer with **ES256 asymmetric signing**, verified against admin's JWKS endpoint (`JWT_JWKS_URL`, default `https://admin.azaion.com/.well-known/jwks.json`).
- **Token validator code**: `src/Auth/JwtExtensions.cs` (verifier only — annotations does not mint tokens).
- **Validation parameters**: `ValidateIssuer = true`, `ValidateAudience = true`, `ValidateLifetime = true`, `ValidateIssuerSigningKey = true`, `ValidAlgorithms = [SecurityAlgorithms.EcdsaSha256]`, `RequireSignedTokens = true`, `RequireExpirationTime = true`, `ClockSkew = 30s`.
- **Anonymous endpoints**: only `GET /health` — every other endpoint requires authentication. The legacy `POST /auth/refresh` was removed; callers refresh against admin's `POST /token/refresh`.
- **Token storage**: stateless. Refresh tokens live in admin's DB (revocation is enforced by admin); annotations does not persist any token material.
## Authorization
- **Policies declared**: `ANN`, `DATASET`, `ADM` (in `src/Auth/JwtExtensions.cs`).
- **Policy → controller mapping** (verified):
- `ANN`: `AnnotationsController`, `MediaController` (annotation lifecycle + media upload — what humans on the annotation UI need).
- `DATASET`: `DatasetController` (dataset exploration).
- `ADM`: planned `[ADM]` writes on `/classes` (RB-06) and mutating routes on `/settings/*`.
- Mixed `[Authorize]` (any authenticated): the read endpoints under `/settings/*`.
- **Per-action overrides**: writes inside `/settings/*` typically require `ADM`; reads accept any authenticated user.
### Known weakness
- No row-level / tenancy authorization is implemented. A user with policy `ANN` can read/mutate any annotation regardless of mission ownership. This is acceptable for the current single-tenant deployment but must be documented before any multi-tenant rollout.
## Secrets handling
- **Source**: env vars resolved at `Program.cs` boot via `ConfigurationResolver.ResolveRequiredOrThrow` (env var → `IConfiguration` → throw if missing).
- **Required in any environment** (no fallback — service refuses to start without them): `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`.
- **Soft defaults**: `RABBITMQ_*` keep their development defaults; operators MUST still override for production.
- **No secret manager integration** in code today — secrets land via container env (suite-level orchestrator's responsibility).
- `JWT_SECRET` was removed: annotations holds no HMAC material and no longer issues tokens.
## Transport
- Inside the container: HTTP only (`ASPNETCORE_URLS=http://+:8080`).
- TLS termination: outside the container (suite-level reverse proxy / orchestrator).
- No HSTS or HTTPS-redirect middleware in `Program.cs`.
## CORS
- Configuration: config-driven allow-list via `CorsConfig:AllowedOrigins` (`Program.cs` + `Infrastructure/CorsConfigurationValidator.cs`).
- `CorsConfigurationValidator.EnsureSafeForEnvironment` refuses to start in `Production` when the allow-list is empty and `CorsConfig:AllowAnyOrigin` is not explicitly set; a `LogWarning` is emitted in lower environments when running with the permissive fallback so the drift is visible in logs.
## Input validation & sanitization
- **REST DTO binding**: ASP.NET model binding only — no `FluentValidation` or custom validator visible in code.
- **File upload validation**: no MIME / extension whitelist visible at the `MediaController` layer. Step 14 should confirm whether the upstream pipeline restricts upload format or whether the service itself needs to.
- **SQL**: all access via Linq2DB / parameterised queries — no raw string concatenation found.
- **YOLO label file write**: trusts caller-provided detections (class id, coordinates) — clamping / range checks would be a Step 14 candidate.
## Rate limiting / DoS
- **None present** — there is no rate-limiting middleware (`AddRateLimiter`, `UseRateLimiter`) registered in `Program.cs`.
- **Implicit limits**: SSE channel is unbounded; outbox table is unbounded; RabbitMQ stream is bounded by retention config (suite-level).
- Step 14 candidate: per-IP rate limit on `POST /annotations` and `POST /media`, since they accept image bytes and write to disk.
## Auditing & logging
- Console logger configured in `Program.cs` (default ASP.NET Core logging).
- Authentication failures: rely on `JwtBearer` middleware default 401s — not explicitly logged with extra detail.
- No audit log of mutations is written today; the `annotations_queue_records` outbox + downstream stream IS the de-facto audit trail (post RB-01 + RB-09).
## Observability boundary
- `/health` is the only pre-auth endpoint that reveals state. It returns 200/non-200 only — no version or DB info — so it is safe to expose to load balancers without auth.
- Swagger UI is mounted in all environments (ADR-005). It exposes the full controller surface but no secrets. Step 14 should consider gating it behind `ADM` or environment-conditional registration.
## Error response surface
- All errors returned via `Middleware/ErrorHandlingMiddleware` use the suite-standard envelope (`_docs/02_document/common-helpers/01_http-error-envelope.md`).
- Stack traces are not echoed back to the client in prod (verify the `IsDevelopment()` branch in the middleware during Step 14).
## Summary table — known security gaps to address in Step 14
| ID | Area | Gap | Status |
|----|------|-----|--------|
| SEC-01 | Auth | JWT issuer/audience not validated | **Closed**`ValidateIssuer`/`ValidateAudience` enforced; `JWT_ISSUER` and `JWT_AUDIENCE` are required env vars. |
| SEC-02 | Secrets | Dev fallback for `JWT_SECRET` in source | **Closed**`JWT_SECRET` removed; remaining required vars fail fast on startup via `ConfigurationResolver`. |
| SEC-03 | CORS | `AllowAnyOrigin` default | **Closed** — config-driven allow-list; `CorsConfigurationValidator` blocks empty list in `Production`. |
| SEC-04 | Surface | Swagger UI exposed in prod | Open — gate behind `ADM` or `Development` only. |
| SEC-05 | Upload | No MIME / extension whitelist on `/media` | Open — validate at controller before disk write. |
| SEC-06 | DoS | No rate limiting on hot write endpoints | Open — per-IP / per-user limiter on `POST /annotations`, `POST /media`. |
| SEC-07 | Tenancy | No row-level authorization | Open — document constraint; add mission-scoped check before multi-tenant rollout. |
| SEC-08 | Audit | No structured audit log | Open — use post-RB-01 lifecycle stream as the audit substrate; add structured fields. |