# Azaion.Annotations — Documentation Report

## Executive Summary

Reverse-engineered the `Azaion.Annotations` codebase bottom-up — 11 module docs → 6 component specs → 1 system architecture + 8 verified flows + ER diagram + deployment + glossary, then synthesized a retrospective `solution.md` and a 5-file problem extraction. Verification surfaced 8 behavioral discrepancies between code and the suite-level `01_annotations.md` narrative; all 8 were resolved with stakeholder decisions, captured as 13 ADRs and 9 Refactor Backlog items (RB-01..RB-09) inside `architecture.md`.

## Problem Statement

`Azaion.Annotations` is the suite's annotation lifecycle service. It is the single owner of the `annotations` table, the YOLO label files on disk, and the lifecycle event stream. Three independent consumers (Annotator UI, AI training pipeline, admin sync worker) need the same data shaped differently — push for humans (SSE), durable-pull for machines (RabbitMQ Stream) — and this service is the one place that reconciles those needs. See `_docs/00_problem/problem.md`.

## Architecture Overview

Single .NET 10 ASP.NET Core service, single PostgreSQL state-of-record, content-addressed filesystem cache, in-process SSE channel + transactional outbox drained to RabbitMQ Stream by a hosted background service. JWT bearer with three policies (`ANN`, `DATASET`, `ADM`). Idempotent boot-time DDL migrator removes a separate migration deploy step.

13 ADRs captured the choices; 9 Refactor Backlog items capture the agreed-upon next moves. Full detail: `_docs/02_document/architecture.md`.

**Technology stack**: .NET 10 + ASP.NET Core + Linq2DB + Npgsql + JwtBearer + RabbitMQ.Stream.Client + MessagePack + xxHash3 (per RB-04) on PostgreSQL 13+.

**Deployment**: ARM64 multi-arch Docker image; branch-driven Woodpecker CI emits `${BRANCH}-arm` tags; orchestrator-managed at the suite level.

## Component Summary

| # | Component | Purpose | Dependencies | Epic |
|---|-----------|---------|--------------|------|
| 01 | annotations-rest | Annotation CRUD + image/thumbnail file routes; YOLO label write; lifecycle producer | 02, 06 | TBD (Phase B) |
| 02 | annotations-realtime-sync | In-process SSE channel + transactional outbox + `FailsafeProducer` to RabbitMQ Stream | 06 | TBD (Phase B) |
| 03 | media | Multipart media upload (single + batch), file download, soft delete | 06 | TBD (Phase B) |
| 04 | dataset | Dataset exploration: filters, class distribution, bulk status writes | 06 (today couples 01 — RB-08) | TBD (Phase B) |
| 05 | settings-metadata | System / directory / camera / user settings + detection class catalog | 06 | TBD (Phase B) |
| 06 | platform | Composition root, JWT, error envelope, path resolver, DB migrator | — | TBD (Phase B) |

**Implementation order** (logical layer dependency, not "to-build" — the codebase already exists):
1. `06 platform` is the foundation; every other component imports from it.
2. `02 annotations-realtime-sync` is the lifecycle substrate `01` and (post RB-01) `04` feed into.
3. `01 annotations-rest`, `03 media`, `05 settings-metadata` sit on top of `06` directly.
4. `04 dataset` reads the storage `01` writes; today via direct DB coupling, post RB-08 via `AnnotationService`.

**Refactor sequencing** is what Phase B will plan (Steps 8 onward); the Refactor Backlog already orders the items by impact:

1. RB-01 (lifecycle observability across mutations) — unblocks RB-09 stream contract and most Step 14 audit work.
2. RB-03 (transactional outbox wrapper) — required before RB-01 is testable.
3. RB-04 (xxHash3.Hash128) — small, isolated, can run parallel.
4. RB-02 (drop `silent_detection`) — small cleanup, after RB-01.
5. RB-08 (decouple `04 dataset` writes) — unblocks soft-delete read filtering.
6. RB-07 (`Flight*` → `Mission*` rename) — high-touch, needs coordination with suite consumers.
7. RB-06 (admin-managed detection classes) — feature, can run parallel.
8. RB-05 (replace `catch { }` in `FailsafeProducer.cs:138`) — trivial, anytime.
9. RB-09 (stream dedupe contract `(annotation_id, operation, date_time)`) — depends on RB-01.

## System Flows

| Flow | Description | Key Components |
|------|-------------|---------------|
| F1 | Annotation create — content-address image, persist, write label, fan-out (SSE + outbox) | 01, 02, 06 |
| F2 | Annotation listing / detail | 01, 06 |
| F3 | Real-time SSE subscription per mission | 01, 02, 06 |
| F4 | Failsafe outbox drain (background loop) → RabbitMQ Stream | 02, 06 |
| F5 | Media upload (single + batch) | 03, 06 |
| F6 | Auth: login + refresh token rotation | 06 |
| F7 | Directory settings change → `pathResolver.Reset()` invariant | 05, 06 |
| F8 | Dataset bulk status update | 04, 06 |

Full sequence diagrams: `_docs/02_document/system-flows.md` and `_docs/02_document/diagrams/flows/`.

## Risk Summary

Risks here are the operational / behavioral risks captured during verification + Step 14 candidates from `security_approach.md`. They live in `architecture.md` (Risks + Refactor Backlog) and will be mirrored into a formal risk register during Phase B Step 12.

| Level | Count | Key Risks |
|-------|-------|-----------|
| Critical | 0 | — |
| High | 3 | (1) silent mutation paths break downstream consumers (RB-01); (2) outbox not transactional with FS+DB write (RB-03); (3) outbox has no row-leasing → multi-instance double-publish (OP-02 / blocked by RB-09 contract). (Former SEC-01 — JWT issuer/audience not validated — closed by the auth refactor.) |
| Medium | 3 | xxHash64 collision tolerance (RB-04); `silent_detection` ambiguity (RB-02); Swagger in prod (SEC-04); `04 dataset` direct-DB coupling (RB-08). (Former SEC-03 — CORS wide-open — closed by `CorsConfigurationValidator`.) |
| Low | 6 | `Flight` vs `Mission` naming drift (RB-07); empty `catch{}` (RB-05); `detection_classes` not admin-CRUD (RB-06); upload MIME whitelist (SEC-05); rate limiting (SEC-06); audit log substrate (SEC-08). |

**Iterations completed**: 1 verification pass with stakeholder review.
**All Critical/High risks mitigated**: No — High items are tracked as RB-01, RB-03, and OP-02 (multi-instance constraint, time-boxed by current single-instance deployment). SEC-01 / SEC-02 / SEC-03 (the original auth + CORS gaps) were closed by the auth + CORS refactor between Steps 1 and 4. Remaining mitigations are scheduled, not executed.

## Test Coverage

The repo currently has **zero automated tests** (`_docs/02_document/00_discovery.md`) and CI runs only build-and-push (`.woodpecker/build-arm.yml`). Test coverage is planned, not measured.

| Component | Integration | Performance | Security | Acceptance | AC Coverage |
|-----------|-------------|-------------|----------|------------|-------------|
| 01 annotations-rest | 0 / TBD (Step 3) | 0 / TBD (Step 15) | 0 / TBD (Step 14) | 0 / 8 ACs | 0 / 8 |
| 02 realtime-sync | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 4 ACs | 0 / 4 |
| 03 media | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 2 ACs | 0 / 2 |
| 04 dataset | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 2 ACs | 0 / 2 |
| 05 settings-metadata | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 3 ACs | 0 / 3 |
| 06 platform | 0 / TBD | 0 / TBD | 0 / TBD | 0 / 5 ACs | 0 / 5 |

**Overall acceptance criteria coverage**: 0 / 24 functional ACs + 0 / 5 non-functional ACs (0%).

The autodev existing-code Phase A Steps 3 (Test Spec) and 6 (Implement Tests) own filling this matrix.

## Epic Roadmap

The current invocation completed **Phase A Step 1 (Document)** of the autodev existing-code flow. No tracker epics have been opened yet. The work that follows in Phase A produces the test surface and the security/perf baselines:

| Order | Phase A Step | Output | Effort | Dependencies |
|-------|--------------|--------|--------|-------------|
| 1 | Step 2 — Documentation Quality Audit | gap log | S | this report |
| 2 | Step 3 — Test Spec | per-component `tests.md` (functional + integration shape) | M | step 2 |
| 3 | Step 4 — Risk Mitigations | `risk_mitigations.md` | S | step 2 |
| 4 | Step 5 — Solution Extraction (already done — see `_docs/01_solution/solution.md`) | — | — | — |
| 5 | Step 6 — Implement Tests | actual test project + green CI step | L | step 3 |
| 6 | Step 7 — Test Audit | coverage report against AC matrix | S | step 6 |

Phase B (Feature Cycle) then runs per feature/refactor. The 9 Refactor Backlog items become the first batch of Phase B epics; sizing per the user's Jira complexity rules will be 2–5 points each, except RB-07 (rename across DTOs/controllers/consumers — likely 5).

**Total estimated effort**: not committed. Phase A Steps 2–7 are scoped against this report; Phase B sizes per epic.

## Key Decisions Made

These are the 13 ADRs from `architecture.md`. Eight of them came from the verification stakeholder review.

| # | Decision | Rationale | Alternatives rejected |
|---|----------|-----------|----------------------|
| ADR-001 | In-process SSE channel for UI fan-out, separate transactional outbox for durable consumers | Sub-ms UI latency without standing up a broker for the inner loop | Single broker for both (UI latency hit); Postgres LISTEN/NOTIFY for UI (delivery semantics insufficient) |
| ADR-002 (RETIRED) | Originally: symmetric HS256 JWT, no issuer/audience validation. Now: ES256 verifier-only over admin's JWKS, with `iss` / `aud` / `exp` / `alg` all enforced. | Identity is centralised in admin; annotations holds no signing material | The original symmetric scheme it replaced |
| ADR-003 | Linq2DB + idempotent SQL DDL migrator (no EF, no DbUp/FluentMigrator) | Lighter dependency surface; one less deploy step (ADR-007) | EF Core migrations (heavier); FluentMigrator (separate runner) |
| ADR-004 | Annotation id = `XxHash3.Hash128` over a sampled image-bytes window | 128-bit space tolerates the suite's annotation volume; sampled keeps large-frame ingest cheap | Full SHA-256 (CPU); xxHash64 (collision space too small — RB-04 was the upgrade) |
| ADR-005 | Swagger UI mounted unconditionally | Internal-only deployment; aids debugging | Gating by env (deferred to SEC-04) |
| ADR-006 (RETIRED) | Originally: CORS `AllowAny*`. Now: config-driven allow-list (`CorsConfig:AllowedOrigins` + opt-in `AllowAnyOrigin`) gated by `CorsConfigurationValidator` per environment. | Production cannot start without an explicit origin policy | The original wide-open default |
| ADR-007 | DDL applied at boot, not in CI | Single deploy step; matches container-immutable model | Separate migration job (deploy complexity) |
| ADR-008 | Business-transaction wrapper (transactional outbox) for annotation lifecycle | Atomicity across DB + outbox; FS write tolerated as best-effort with cleanup | DTC across FS + DB + RabbitMQ (heavyweight, not portable) |
| ADR-009 | Every mutation path emits SSE + enqueues outbox row | One observability contract for humans + machines | SSE-only (durability gap for AI/admin worker); outbox-only (UI latency) |
| ADR-010 | Remove `silent_detection` flag | Behavior is contradictory once ADR-009 holds | Keep flag and gate on it (forces every consumer to interpret it) |
| ADR-011 | Detection class catalog becomes admin-managed (CRUD + cache) | Catalog evolves with deployments; migrator-only is a deploy-time-only escape hatch | Static catalog (RB-06 supersedes) |
| ADR-012 | Canonical term is `Mission`; `Flight*` symbols renamed | Single suite-level vocabulary | Keep `Flight` in this service (drift cost grows over time) |
| ADR-013 | On-the-wire dedupe key: `(annotation_id, operation, date_time)` | Lets every downstream consumer dedupe re-deliveries safely | Per-consumer offset trust (fragile under outbox replay) |

## Open Questions

All 6 verification-pass questions were resolved during stakeholder review. Genuinely-open follow-ups now:

| # | Question | Impact | Assigned To |
|---|----------|--------|-------------|
| 1 | Are P50/P95/P99 latency / throughput targets contracted anywhere in the suite? | Bounds NFR ACs (`AC-N-*`) and Step 15 perf-test shape. | Suite ops / product |
| 2 | What is the upload format whitelist `/media` should enforce? | Bounds SEC-05 fix scope. | Detections-pipeline owner |
| 3 | RPO/RTO contract for `images_dir` and `deleted_dir`? | Bounds the soft-delete restore story (post RB-01). | Suite ops |
| 4 | Stream retention window for `azaion-annotations`? | Bounds the consumer replay window the AI pipeline depends on. | Suite ops |
| 5 | Is multi-tenancy on the roadmap within the doc horizon? | Decides whether SEC-07 is a Step 14 must-fix or a deferred gap. | Product |

## Artifact Index

| File | Description |
|------|-------------|
| `_docs/02_document/architecture.md` | Architecture vision, 13 ADRs, refactor backlog, NFRs |
| `_docs/02_document/system-flows.md` | F1–F8 verified flow narratives |
| `_docs/02_document/data_model.md` | ERD + per-table contract reproduced from `DatabaseMigrator.cs` |
| `_docs/02_document/glossary.md` | 36 canonical terms (suite + project + code-level) |
| `_docs/02_document/module-layout.md` | Module → component mapping |
| `_docs/02_document/04_verification_log.md` | Verification pass corrections + stakeholder resolutions |
| `_docs/02_document/components/01_annotations-rest/description.md` | Component 01 spec |
| `_docs/02_document/components/02_annotations-realtime-sync/description.md` | Component 02 spec |
| `_docs/02_document/components/03_media/description.md` | Component 03 spec |
| `_docs/02_document/components/04_dataset/description.md` | Component 04 spec |
| `_docs/02_document/components/05_settings-metadata/description.md` | Component 05 spec |
| `_docs/02_document/components/06_platform/description.md` | Component 06 spec |
| `_docs/02_document/modules/*.md` | 11 module-level deep-dives |
| `_docs/02_document/diagrams/components.md` | Component diagram (Mermaid) |
| `_docs/02_document/diagrams/flows/flow_annotation_create.md` | F1 sequence (verified) |
| `_docs/02_document/diagrams/flows/flow_sse_subscription.md` | F3 sequence |
| `_docs/02_document/diagrams/flows/flow_failsafe_drain.md` | F4 sequence |
| `_docs/02_document/deployment/containerization.md` | Dockerfile-derived deployment notes |
| `_docs/02_document/deployment/ci_cd_pipeline.md` | Woodpecker pipeline-derived notes |
| `_docs/02_document/deployment/environment_strategy.md` | Env-var contract + ASPNETCORE_ENVIRONMENT use |
| `_docs/02_document/deployment/observability.md` | Logging + `/health` + outbox depth gap |
| `_docs/02_document/common-helpers/01_http-error-envelope.md` | Suite error envelope contract |
| `_docs/01_solution/solution.md` | Retrospective per-component solution table |
| `_docs/00_problem/problem.md` | Retrospective problem statement |
| `_docs/00_problem/restrictions.md` | HW / SW / ENV / OP / suite-level restrictions |
| `_docs/00_problem/acceptance_criteria.md` | 24 functional + 5 non-functional ACs |
| `_docs/00_problem/input_data/data_parameters.md` | REST DTOs + env vars + seed data + wire format |
| `_docs/00_problem/security_approach.md` | Auth/AuthZ/secrets posture + 8 SEC-XX gaps |

## Cross-references

- Suite-level integration narrative: `suite/_docs/01_annotations.md`
- Repo-config (monorepo discovery): `_docs/_repo-config.yaml`
- Autodev state: `_docs/_autodev_state.md`
- Document skill internal state: `_docs/02_document/state.json`