mirror of
https://github.com/azaion/annotations.git
synced 2026-06-21 10:51:07 +00:00
docs+src: complete Steps 1-3 outcomes + auth re-sync baseline
This commit captures everything produced during autodev existing-code Steps 1 (Document), 2 (Architecture Baseline Scan), and 3 (Test Spec), together with the targeted auth + CORS re-sync triggered on 2026-05-14 when codebase drift was detected at Step 4 entry. None of this work was previously committed. Step 1 (Document) — 50+ _docs/02_document/ files: problem, solution, architecture, system flows, glossary, module-layout, per-component specs (01..06), modules, deployment, diagrams, data model, FINAL report, verification log, discovery. Step 2 (Architecture Baseline) — architecture_compliance_baseline.md. Verdict PASS_WITH_WARNINGS (0 Critical, 0 High, 1 Medium, 2 Low). No High/Critical findings; auto-chained to Step 3 per existing-code flow. Step 3 (Test Spec) — _docs/02_document/tests/* (67 scenarios across blackbox, security, resilience, resource-limit, performance), plus e2e/docker-compose.test.yml, e2e/seed/run.sh, scripts/run-tests.sh, scripts/run-performance-tests.sh. Coverage 88% over the active scope (40 of 45 items covered, 6 RB-deferred, 5 documented-as-uncovered). Targeted auth + CORS re-sync — replaces the deleted in-house token issuer with a JWKS-verifier model. AuthController and TokenService removed; JwtExtensions switched from HS256 symmetric to ES256 over admin's JWKS. ConfigurationResolver and CorsConfigurationValidator added under src/Infrastructure/. ADR-002 and ADR-006 retired; SEC-01, SEC-02, SEC-03 marked Closed. One new testability risk recorded in architecture.md Open Risks Section 6 (JWKS HTTPS gating). Source changes: - src/Auth/JwtExtensions.cs (modified) — ES256, JWKS, alg pinning - src/Program.cs (modified) — DI wiring for ConfigurationResolver and CorsConfigurationValidator - src/Controllers/AuthController.cs (deleted) — no in-service issuance - src/Services/TokenService.cs (deleted) — same - src/Infrastructure/ConfigurationResolver.cs (new) - src/Infrastructure/CorsConfigurationValidator.cs (new) - .env.example (new) — required env var documentation - .gitignore (updated) Cross-repo coordination: _docs/cross-repo/flights_h1_h2_h3_change_spec captures the change-spec for downstream services that consumed the now deleted /auth endpoints. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,19 @@
|
||||
# Module documentation index
|
||||
|
||||
Modules follow **`suite/_docs/01_annotations.md`**: annotations vs media, SSE, auth/JWT refresh, DB, RabbitMQ sync, plus **dataset** (DATASET) and **settings / detection classes** as implemented in this repo.
|
||||
|
||||
| Order | File | Scope |
|
||||
|------:|------|--------|
|
||||
| 1 | [wire-enums.md](./wire-enums.md) | `src/Enums/*` |
|
||||
| 2 | [database-layer.md](./database-layer.md) | `src/Database/*` |
|
||||
| 3 | [common-infrastructure.md](./common-infrastructure.md) | `PathResolver`, `ErrorHandlingMiddleware`, shared small types |
|
||||
| 4 | [auth-identity.md](./auth-identity.md) | `JwtExtensions` (JWKS verifier), `ConfigurationResolver`, `CorsConfigurationValidator` |
|
||||
| 5 | [media-service.md](./media-service.md) | `MediaService`, `MediaController`, media DTOs |
|
||||
| 6 | [annotations-service.md](./annotations-service.md) | `AnnotationService`, `AnnotationsController` (REST + files) |
|
||||
| 7 | [dataset-service.md](./dataset-service.md) | `DatasetService`, `DatasetController`, dataset DTOs |
|
||||
| 8 | [settings-metadata-service.md](./settings-metadata-service.md) | `SettingsService`, `SettingsController`, `ClassesController`, settings DTOs |
|
||||
| 9 | [sse-realtime.md](./sse-realtime.md) | `AnnotationEventService`, SSE endpoint |
|
||||
| 10 | [rabbitmq-stream-sync.md](./rabbitmq-stream-sync.md) | `FailsafeProducer`, `RabbitMqConfig`, `QueueMessages` |
|
||||
| 11 | [composition-program.md](./composition-program.md) | `Program.cs` |
|
||||
|
||||
`src/DTOs/` types are described in the module that exposes them on the wire.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Module: Annotations service
|
||||
|
||||
## Purpose
|
||||
|
||||
Core **annotation CRUD**, listing, static image/thumbnail delivery, and coordination with **media** and **files on disk**. Maps to **`01_annotations.md` §1–6** (not SSE — see `sse-realtime.md`).
|
||||
|
||||
## Code
|
||||
|
||||
- `AnnotationService` — create/update/status/delete/query/get one; uses `PathResolver`, hashing, label/thumbnail generation, queue handoff to failsafe path as implemented.
|
||||
- `AnnotationsController` — `[Route("annotations")]`, `[Authorize(Policy = "ANN")]` except where noted.
|
||||
- REST: `POST`, `PUT/{id}`, `PATCH/{id}/status`, `DELETE/{id}`, `GET`, `GET/{id}`.
|
||||
- Files: `GET/{id}/thumbnail`, `GET/{id}/image`.
|
||||
- **SSE** `GET/events` documented in `sse-realtime.md` (same controller type).
|
||||
|
||||
## DTOs (this module)
|
||||
|
||||
- `CreateAnnotationRequest`, `UpdateAnnotationRequest`, `UpdateStatusRequest`, `GetAnnotationsQuery`, `AnnotationListItem`, `DetectionDto` (annotation payloads).
|
||||
|
||||
## Dependencies
|
||||
|
||||
Database, `PathResolver`, optional integration with queue/SSE services.
|
||||
|
||||
## Suite vs code (maintain in suite or code)
|
||||
|
||||
- **UserId:** suite pseudo-code shows `UserId` on create; **implementation** uses JWT subject (`AnnotationsController`).
|
||||
- **GET filter:** suite `missionId` vs code `FlightId` + filter behavior — track as open alignment.
|
||||
|
||||
## Suite doc
|
||||
|
||||
§1–6; annotation identity at top of `01_annotations.md`.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Module: Auth & identity
|
||||
|
||||
## Purpose
|
||||
|
||||
JWT validation for API policies. Tokens are minted exclusively by the admin service (ES256-signed); annotations is a **verifier only**.
|
||||
|
||||
## Components
|
||||
|
||||
### `JwtExtensions` (`Auth/JwtExtensions.cs`)
|
||||
|
||||
- `AddJwtAuth(IConfiguration)` — pulls `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL` via `ConfigurationResolver.ResolveRequiredOrThrow` (fail-fast at startup if any is missing).
|
||||
- `TokenValidationParameters` mirrors admin's verifier contract:
|
||||
- `ValidateIssuer = true` / `ValidateAudience = true` / `ValidateLifetime = true`.
|
||||
- `ValidAlgorithms = [SecurityAlgorithms.EcdsaSha256]` — pinned so an HS256-forgery using the public key as the HMAC secret cannot pass.
|
||||
- `RequireSignedTokens = true`, `RequireExpirationTime = true`.
|
||||
- `ClockSkew = 30s`.
|
||||
- Signing keys are fetched from admin's `/.well-known/jwks.json` via a `ConfigurationManager<JsonWebKeySet>` backed by a minimal `IConfigurationRetriever<JsonWebKeySet>` (admin exposes JWKS but not the full OIDC discovery document). The manager honours admin's `Cache-Control: public, max-age=3600` and refreshes on the default schedule. During key rotation both `kid`s are present in JWKS so in-flight tokens still verify.
|
||||
- Policies: `ANN`, `DATASET`, `ADM` — each requires claim `permissions` with that code (matches suite "Required permission: ANN" and Dataset Explorer `DATASET`).
|
||||
|
||||
## Dependencies
|
||||
|
||||
Configuration (all required, no defaults):
|
||||
|
||||
- `JWT_ISSUER` (alt `Jwt:Issuer`) — must match admin's `JwtConfig:Issuer`.
|
||||
- `JWT_AUDIENCE` (alt `Jwt:Audience`) — must match admin's `JwtConfig:Audience`.
|
||||
- `JWT_JWKS_URL` (alt `Jwt:JwksUrl`) — `https://admin.azaion.com/.well-known/jwks.json` in production.
|
||||
|
||||
## Consumers
|
||||
|
||||
All `[Authorize]` controllers.
|
||||
|
||||
## Removed in this cycle
|
||||
|
||||
- `Services/TokenService.cs` (HS256 minting of access tokens from refresh tokens) — deleted; refresh is now the admin service's responsibility (`POST /token/refresh`).
|
||||
- `Controllers/AuthController.cs` and the `POST /auth/refresh` endpoint — deleted along with `TokenService`. Detections (and any other client) must call admin's refresh endpoint and pass the returned access token to annotations.
|
||||
- `JWT_SECRET` env var — no longer read.
|
||||
|
||||
## Suite doc
|
||||
|
||||
`01_annotations.md` §Annotation Sync (verifier role); suite `10_auth.md` for full auth story (admin = issuer, satellite-provider / annotations / flights / ui = verifiers).
|
||||
@@ -0,0 +1,38 @@
|
||||
# Module: Common infrastructure
|
||||
|
||||
## Purpose
|
||||
|
||||
Cross-cutting **filesystem layout** (annotation images, labels, thumbnails, results), **global error JSON**, and trivial shared API types.
|
||||
|
||||
## Components
|
||||
|
||||
### `PathResolver` (`Services/PathResolver.cs`)
|
||||
|
||||
- Lazy-loads paths from `directory_settings` via `AppDataConnection`.
|
||||
- Methods: `GetImagePath`, `GetLabelPath`, `GetThumbnailPath`, `GetResultPath`, `GetMediaDir` — paths under configured dirs with `{annotationId}.jpg` / `.txt` patterns.
|
||||
- `Reset()` clears cache after directory settings change.
|
||||
|
||||
### `ErrorHandlingMiddleware` (`Middleware/`)
|
||||
|
||||
Maps exceptions to `{ statusCode, message }` JSON (400/404/409/500). Aligns HTTP outcomes with `01_annotations.md` status tables.
|
||||
|
||||
### Shared DTOs (`DTOs/`)
|
||||
|
||||
- `PaginatedResponse<T>` — list + `totalCount` / `page` / `pageSize` (annotations list, media list, dataset).
|
||||
- `ErrorResponse` — available for explicit error contracts where used.
|
||||
|
||||
### `GlobalUsings.cs`
|
||||
|
||||
Project-wide usings only.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `PathResolver` → `AppDataConnection`, `DirectorySettings` entity.
|
||||
|
||||
## Consumers
|
||||
|
||||
All services and controllers that touch disk or return paged lists.
|
||||
|
||||
## Suite doc
|
||||
|
||||
File cleanup on DELETE annotation (`GetImgPath` / label / thumb) in `01_annotations.md` §4.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Module: Composition (`Program.cs`)
|
||||
|
||||
## Purpose
|
||||
|
||||
Single **composition root**: configuration, PostgreSQL `AppDataConnection`, service registrations, **JWT**, **CORS**, Swagger, **migrator** on startup, **middleware** order, `MapControllers`, `/health`, `WebApplication.Run`.
|
||||
|
||||
## Notable wiring
|
||||
|
||||
- `DATABASE_URL` (required, no fallback — startup fails fast via `ConfigurationResolver.ResolveRequiredOrThrow`) → Npgsql connection string helper.
|
||||
- `JWT_ISSUER` / `JWT_AUDIENCE` / `JWT_JWKS_URL` for `AddJwtAuth` (all required; resolved by `ConfigurationResolver`). The validator pulls public ES256 keys from admin's JWKS endpoint; this service no longer holds an HMAC secret.
|
||||
- `CorsConfig:AllowedOrigins` / `CorsConfig:AllowAnyOrigin` for the default CORS policy; `CorsConfigurationValidator` refuses to start with a permissive policy in `Production`.
|
||||
- `RabbitMqConfig` from env + `AddHostedService<FailsafeProducer>()`.
|
||||
- Scoped services: `AnnotationService`, `MediaService`, `DatasetService`, `SettingsService`, `PathResolver`; singletons: `AnnotationEventService`, `RabbitMqConfig`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
All modules; documented last after slices are understood.
|
||||
|
||||
## Suite doc
|
||||
|
||||
Operational/env story complements `01_annotations.md` deployment sections in suite architecture docs.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Module: Database layer (`src/Database`)
|
||||
|
||||
## Purpose
|
||||
|
||||
PostgreSQL schema and Linq2DB mapping for annotations, media, detections, queue buffer, settings, and `detection_classes`. Underpins every HTTP module in `01_annotations.md`.
|
||||
|
||||
## Public interface
|
||||
|
||||
- `AppDataConnection` — `ITable<>` for all mapped entities.
|
||||
- `DatabaseMigrator.Migrate` — embedded SQL: `CREATE TABLE IF NOT EXISTS` / `ALTER … IF NOT EXISTS`, seed detection classes.
|
||||
|
||||
## Entities (summary)
|
||||
|
||||
- `Annotation`, `Media`, `Detection` — core annotation + YOLO row model (`time` stored as BIGINT ticks).
|
||||
- `AnnotationsQueueRecord` — failsafe outbox (`operation`, `annotation_ids`).
|
||||
- `SystemSettings` — includes `GenerateAnnotatedImage`, `SilentDetection` (suite §Annotated Image / Silent Detection).
|
||||
- `DirectorySettings` — `/data/...` roots consumed by `PathResolver`.
|
||||
- `DetectionClass`, `UserSettings`, `CameraSettings`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Wire enums on columns.
|
||||
|
||||
## Consumers
|
||||
|
||||
All services and `ClassesController`.
|
||||
|
||||
## Suite doc
|
||||
|
||||
Annotation identity and ER-level behavior; cross-check `00_database_schema.md` in suite when entities evolve.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Module: Dataset service
|
||||
|
||||
## Purpose
|
||||
|
||||
**Dataset Explorer** backend: paginated grid, detail, status updates, bulk status — **`[Authorize(Policy = "DATASET")]`** per suite note on PATCH status (`01_annotations.md` §3 points to `09_dataset_explorer.md`).
|
||||
|
||||
## Code
|
||||
|
||||
- `DatasetService` — queries tuned for dataset views; may reuse annotation entities.
|
||||
- `DatasetController` — `[Route("dataset")]`.
|
||||
|
||||
## DTOs (this module)
|
||||
|
||||
- `GetDatasetQuery`, `DatasetItem`, `ClassDistributionItem`, `BulkStatusRequest` — and shared `UpdateStatusRequest` where used for PATCH.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Database, same status enums as annotator.
|
||||
|
||||
## Suite doc
|
||||
|
||||
Primary behavioral spec: `suite/_docs/09_dataset_explorer.md`; permission cross-ref in `01_annotations.md` §3.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Module: Media service
|
||||
|
||||
## Purpose
|
||||
|
||||
HTTP surface and domain logic for **§7–10** in `01_annotations.md`: create media, batch upload, list, delete, and download raw media file.
|
||||
|
||||
## Code
|
||||
|
||||
- `MediaService` — persistence + disk writes, batch from `IFormFileCollection`.
|
||||
- `MediaController` — `[Route("media")]`, `[Authorize(Policy = "ANN")]`.
|
||||
- `POST /media`, `POST /media/batch` (form: `waypointId` + files), `GET /media`, `GET /media/{id}/file`, delete route as implemented.
|
||||
|
||||
## DTOs (this module)
|
||||
|
||||
- `CreateMediaRequest`, `GetMediaQuery`, `MediaListItem` — plus any media-specific shapes used only here.
|
||||
|
||||
## Dependencies
|
||||
|
||||
`AppDataConnection`, `PathResolver` (media dir), JWT user id from claims.
|
||||
|
||||
## Consumers
|
||||
|
||||
Annotator UI / React upload flows described in suite §Media Browsing.
|
||||
|
||||
## Suite doc
|
||||
|
||||
`01_annotations.md` §7–10; accepted formats table in same doc.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Module: RabbitMQ stream sync (failsafe)
|
||||
|
||||
## Purpose
|
||||
|
||||
**Annotation Sync** outbox and **RabbitMQ Stream** producer — `01_annotations.md` §Annotation Sync, Failsafe Queue, RabbitMQ Stream.
|
||||
|
||||
## Code
|
||||
|
||||
- `RabbitMqConfig` + `FailsafeProducer` (`Services/FailsafeProducer.cs`) — `BackgroundService`; builds `StreamSystem`, drains `annotations_queue_records`, serializes **MessagePack** payloads (`AnnotationQueueMessage`, `AnnotationBulkQueueMessage` in `DTOs/QueueMessages.cs`), gzip as implemented.
|
||||
- Entity `AnnotationsQueueRecord` — see `database-layer.md`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
`AppDataConnection`, `PathResolver` (for image bytes on create), env-driven `RABBITMQ_*` from `Program`.
|
||||
|
||||
## Consumers (downstream, external)
|
||||
|
||||
Admin `AnnotationSyncWorker`, AI Training consumer — described in suite doc.
|
||||
|
||||
## Suite doc
|
||||
|
||||
Full sync topology and stream semantics in `01_annotations.md`; keep MessagePack key layout stable.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Module: Settings & metadata
|
||||
|
||||
## Purpose
|
||||
|
||||
**System / directory / camera / user** settings and **detection class** list for UI color maps (`01_annotations.md` §11–12, “GET /classes” narrative).
|
||||
|
||||
## Code
|
||||
|
||||
- `SettingsService` + `SettingsController` — `[Route("settings")]`, mixed `[Authorize]` and `ADM` for writes.
|
||||
- System, directories, camera, user settings endpoints (see controller for full list).
|
||||
- `ClassesController` — `[Route("classes")]`, `GET` all `detection_classes` via Linq2DB (thin read-through).
|
||||
|
||||
## DTOs (this module)
|
||||
|
||||
- `UpdateSystemSettingsRequest`, `UpdateDirectoriesRequest`, `UpdateCameraSettingsRequest`, `UpdateUserSettingsRequest`, etc.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Database entities `SystemSettings`, `DirectorySettings`, `CameraSettings`, `UserSettings`, `DetectionClass`.
|
||||
|
||||
## Suite doc
|
||||
|
||||
`01_annotations.md` camera §11–12; directory defaults align with `PathResolver` / migrator defaults.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Module: SSE (real-time)
|
||||
|
||||
## Purpose
|
||||
|
||||
**Server-Sent Events** for annotation activity — `01_annotations.md` §“GET /annotations/events (SSE)” and `AnnotationEvent` shape.
|
||||
|
||||
## Code
|
||||
|
||||
- `AnnotationEventService` — unbounded `Channel<AnnotationEventDto>`; `PublishAsync` / `Reader` for subscribers.
|
||||
- `AnnotationsController.Events` — sets `text/event-stream`, subscribes readers, pushes JSON events (implementation detail in source).
|
||||
|
||||
## DTOs
|
||||
|
||||
- `AnnotationEventDto` — ids, `Status`, `Source`, `Detections`, `CreatedDate`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
`AnnotationService` (or controller) calls `PublishAsync` after mutations.
|
||||
|
||||
## Suite doc
|
||||
|
||||
SSE section + `DetectionEvent` vs `AnnotationEvent` distinction (detection progress may be separate pipeline).
|
||||
@@ -0,0 +1,29 @@
|
||||
# Module: Wire enums (`src/Enums`)
|
||||
|
||||
## Purpose
|
||||
|
||||
Integer-backed enums for JSON and MessagePack. **`01_annotations.md`** states all listed enums serialize as **numbers**, not names.
|
||||
|
||||
## Types
|
||||
|
||||
| Enum | File | Suite |
|
||||
|------|------|-------|
|
||||
| `AnnotationSource` | `AnnotationSource.cs` | Suite table (AI=0, Manual=1) |
|
||||
| `AnnotationStatus` | `AnnotationStatus.cs` | Created=10, Edited=20, etc. |
|
||||
| `MediaStatus` | `MediaStatus.cs` | SSE / media lifecycle |
|
||||
| `MediaType` | `MediaType.cs` | Image vs video |
|
||||
| `AffiliationEnum` | `AffiliationEnum.cs` | Detection payload |
|
||||
| `CombatReadiness` | `CombatReadiness.cs` | Detection payload |
|
||||
| `QueueOperation` | `QueueOperation.cs` | Failsafe / bulk queue |
|
||||
|
||||
## Dependencies
|
||||
|
||||
None (leaf).
|
||||
|
||||
## Consumers
|
||||
|
||||
Entities, DTOs, `FailsafeProducer`, services.
|
||||
|
||||
## Suite doc
|
||||
|
||||
Keep enum **numeric** contracts in sync with `01_annotations.md` and consuming UIs.
|
||||
Reference in New Issue
Block a user