azaion/annotations
1
0
Fork 0
mirror of https://github.com/azaion/annotations.git synced 2026-06-21 09:31:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
03f879206e6ef63b5e108e1487371588918e0a50
annotations/_docs/02_document/04_verification_log.md
T
Oleksandr Bezdieniezhnykh 03f879206e docs+src: complete Steps 1-3 outcomes + auth re-sync baseline 
This commit captures everything produced during autodev existing-code
Steps 1 (Document), 2 (Architecture Baseline Scan), and 3 (Test Spec),
together with the targeted auth + CORS re-sync triggered on 2026-05-14
when codebase drift was detected at Step 4 entry. None of this work was
previously committed.

Step 1 (Document) — 50+ _docs/02_document/ files: problem, solution,
architecture, system flows, glossary, module-layout, per-component
specs (01..06), modules, deployment, diagrams, data model, FINAL
report, verification log, discovery.

Step 2 (Architecture Baseline) — architecture_compliance_baseline.md.
Verdict PASS_WITH_WARNINGS (0 Critical, 0 High, 1 Medium, 2 Low). No
High/Critical findings; auto-chained to Step 3 per existing-code flow.

Step 3 (Test Spec) — _docs/02_document/tests/* (67 scenarios across
blackbox, security, resilience, resource-limit, performance), plus
e2e/docker-compose.test.yml, e2e/seed/run.sh, scripts/run-tests.sh,
scripts/run-performance-tests.sh. Coverage 88% over the active scope
(40 of 45 items covered, 6 RB-deferred, 5 documented-as-uncovered).

Targeted auth + CORS re-sync — replaces the deleted in-house token
issuer with a JWKS-verifier model. AuthController and TokenService
removed; JwtExtensions switched from HS256 symmetric to ES256 over
admin's JWKS. ConfigurationResolver and CorsConfigurationValidator
added under src/Infrastructure/. ADR-002 and ADR-006 retired; SEC-01,
SEC-02, SEC-03 marked Closed. One new testability risk recorded in
architecture.md Open Risks Section 6 (JWKS HTTPS gating).

Source changes:
- src/Auth/JwtExtensions.cs (modified) — ES256, JWKS, alg pinning
- src/Program.cs (modified) — DI wiring for ConfigurationResolver
  and CorsConfigurationValidator
- src/Controllers/AuthController.cs (deleted) — no in-service issuance
- src/Services/TokenService.cs (deleted) — same
- src/Infrastructure/ConfigurationResolver.cs (new)
- src/Infrastructure/CorsConfigurationValidator.cs (new)
- .env.example (new) — required env var documentation
- .gitignore (updated)

Cross-repo coordination: _docs/cross-repo/flights_h1_h2_h3_change_spec
captures the change-spec for downstream services that consumed the now
deleted /auth endpoints.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-14 20:19:05 +03:00

152 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Step 4 — Verification Log
Verification pass over `_docs/02_document/` against `src/` source.
## Scope
Documents verified:
- `architecture.md`
- `system-flows.md`
- `data_model.md`
- `deployment/{containerization,ci_cd_pipeline,environment_strategy,observability}.md`
- `diagrams/flows/{flow_annotation_create,flow_sse_subscription,flow_failsafe_drain}.md`
- (sanity re-check only) `module-layout.md`, `components/*/description.md`, `modules/*.md`
## Method
For each generated artifact:
1. Extracted code-entity references (controllers, services, methods, DTOs, env vars, table/column names, route paths).
2. Cross-referenced each against the actual source (`src/Program.cs`, `src/Controllers/*`, `src/Services/*`, `src/Database/*`, `src/Enums/*`, `src/DTOs/*`, `.woodpecker/build-arm.yml`, `src/Dockerfile`).
3. Re-traced each system flow's mermaid sequence against the corresponding service/controller code.
4. Listed corrections, applied them inline to the affected files, and recorded them below.
## Counts
| Item | Verified | Corrected | Open question |
|------|----------|-----------|----------------|
| Controllers + their routes | 6 | 0 | 0 |
| Services + their public methods | 8 | 0 | 0 |
| DB tables / columns | 9 / ~60 | 0 | 5 (lazy upsert / `media.duration` / class catalog mutability / id collision / outbox JSON shape) |
| Enums | 7 | 0 | 0 |
| Env vars | 8 | 0 | 0 |
| Flows | 8 | 4 (F1, F7, F8, dependencies table) | 6 (consolidated below) |
| ADRs | 7 | 1 (ADR-004 hash details) | 0 |
Module-level coverage: **11 / 11 modules** documented; **6 / 6 components** assembled.
## Corrections applied inline
### `architecture.md`
1. **Internal communication table**: tightened to reflect that SSE publish + outbox enqueue happen **only on `CreateAnnotation`**; outbox enqueue is gated by `system_settings.silent_detection`. Added explicit row noting `DatasetService` writes are silent on SSE/outbox today.
2. **ADR-004 (annotation id hash)**: replaced "hash of bytes" with the actual `ComputeHash` strategy — `XxHash64` over a deterministic sample (length prefix + head/middle/tail 1 KB for inputs > 3072 bytes; full bytes otherwise). Documented collision implication.
3. **Open Architectural Risks**: rewrote with verified findings — silent Update/Delete/dataset paths, `silent_detection` semantics, F1 non-atomicity, static `EnqueueAsync` vs project rule.
4. **Section 4 "Data flow summary"**: split into Create-only / Update-and-friends / read paths, removed the inaccurate claim that thumbnails are produced inline by Create.
### `system-flows.md`
1. **F1 sequence + data flow + error scenarios**: replaced with the verified ordering — image file → optional media row → annotation → detections (`BulkCopyAsync`) → label file → SSE publish → conditional outbox enqueue. Removed thumbnail write from the Create path.
2. **F7 ("Reset call missed" risk)**: removed — verified that `SettingsService` calls `pathResolver.Reset()` at lines 71 and 85 of `Services/SettingsService.cs`. Replaced with a "Verified" note.
3. **F8 (Dataset bulk status)**: rewrote — `DatasetService.UpdateStatus` and `BulkUpdateStatus` issue direct `UPDATE annotations SET status` statements only. **They do NOT publish SSE and do NOT enqueue the outbox.** Updated routes (`PATCH /dataset/{id}/status`, `POST /dataset/bulk-status`) and error scenarios accordingly.
4. **Flow Dependencies table**: corrected F1 row (gating + Create-only), F3 row (only F1 Create publishes), F8 row (no SSE / no outbox today).
### `diagrams/flows/flow_annotation_create.md`
- Replaced sequence + flowchart to match the verified F1 ordering (image first, optional media, label, SSE, conditional outbox); thumbnail removed.
- Added note that Update/UpdateStatus/Delete are silent today.
### `diagrams/flows/flow_sse_subscription.md`, `flow_failsafe_drain.md`
- No structural corrections needed; spot-checked sequence vs `AnnotationsController.Events`, `AnnotationEventService`, `FailsafeProducer.EnqueueAsync`. Notes already capture multi-drainer dedupe and channel-unbounded back-pressure concerns.
### `data_model.md`
- No structural corrections; verified every column name and default against `Database/DatabaseMigrator.cs` and `Database/Entities/*.cs`. Spot-quirk (`detection_classes` ids 9 + 10 share `#000080`) is pre-existing and noted.
### `deployment/*`
- No structural corrections; verified `.woodpecker/build-arm.yml` step-by-step, `Dockerfile` two-stage build, `Program.cs` env-var fallbacks.
## Confirmed entities (sample — full list traced during the pass)
Controllers and routes (file:line where attributes were inspected):
- `AnnotationsController` — `Controllers/AnnotationsController.cs:10–80` — `[Route("annotations")]`, `[Authorize(Policy = "ANN")]`, all listed routes match.
- `MediaController` — `Controllers/MediaController.cs:10–55` — `[Route("media")]`, `[Authorize(Policy = "ANN")]`, routes: `POST`, `POST /batch`, `GET`, `GET /{id}/file`, `DELETE /{id}`.
- `DatasetController` — `Controllers/DatasetController.cs:9–41` — `[Route("dataset")]`, `[Authorize(Policy = "DATASET")]`, routes: `GET`, `GET /{annotationId}`, `PATCH /{annotationId}/status`, `POST /bulk-status`, `GET /class-distribution`.
- `SettingsController` — `Controllers/SettingsController.cs:10–66` — `[Route("settings")]`, segments `system`, `directories`, `camera`, `user` each with GET + PUT.
- `ClassesController` — `Controllers/ClassesController.cs:9–13` — `[Route("classes")]`, `[Authorize]`, single `[HttpGet]`.
- `AuthController` — **removed** in the auth refactor; annotations no longer mints or refreshes tokens. `JwtExtensions.AddJwtAuth` (verifier-only, ES256 over admin's JWKS) is the sole auth wiring in `Program.cs`.
Services:
- `AnnotationService.CreateAnnotation` (`Services/AnnotationService.cs:13–104`) — verified sequence used to rewrite F1.
- `AnnotationService.UpdateAnnotation` / `UpdateStatus` / `DeleteAnnotation` — verified that none publish SSE or enqueue outbox.
- `DatasetService.UpdateStatus` / `BulkUpdateStatus` (`Services/DatasetService.cs:75–94`) — verified silent on SSE / outbox.
- `SettingsService` — verified `pathResolver.Reset()` calls at lines 71, 85.
- `FailsafeProducer.EnqueueAsync` — confirmed as the public outbox-write helper, called by `AnnotationService.CreateAnnotation` only.
Tables / migrator (`Database/DatabaseMigrator.cs`):
- All 9 tables referenced in `data_model.md` exist with the columns and defaults as documented; idempotent `CREATE TABLE IF NOT EXISTS` + `ALTER TABLE … IF NOT EXISTS`; `detection_classes` seed of 19 rows with `ON CONFLICT DO NOTHING`.
Env vars (`Program.cs`):
- Required (fail-fast via `ConfigurationResolver.ResolveRequiredOrThrow`): `DATABASE_URL`, `JWT_ISSUER`, `JWT_AUDIENCE`, `JWT_JWKS_URL`.
- Optional with defaults: `RABBITMQ_HOST`, `RABBITMQ_STREAM_PORT`, `RABBITMQ_PRODUCER_USER`, `RABBITMQ_PRODUCER_PASS`, `RABBITMQ_STREAM_NAME`.
- CORS: `CorsConfig:AllowedOrigins` (string array) + `CorsConfig:AllowAnyOrigin` (bool); `CorsConfigurationValidator.EnsureSafeForEnvironment` blocks startup in `Production` when origins are empty and `AllowAnyOrigin` is not explicitly set.
CI (`.woodpecker/build-arm.yml`):
- `event: [push, manual]`, `branch: [dev, stage, main]`, `platform: arm64`, secret refs, `${BRANCH}-arm` tag, OCI image labels — all verified.
## Stakeholder resolutions (closed 2026-05-14)
The six open questions surfaced by this pass were resolved with the maintainer. Authoritative wording lives in `architecture.md` (ADR-004, ADR-008..ADR-011 + Refactor Backlog RB-01..RB-06). Quick map:
| Question | Resolution | Tracked |
|----------|------------|---------|
| Are silent Update/Delete/dataset-status changes intentional? | No — World B is the design; the drainer (`FailsafeProducer.cs:108–123`) was already plumbed for `Validated` + `Deleted` ops, the producer side was never wired in the new HTTP backend (legacy WPF UI did this directly). Wire all mutations to publish + enqueue. | ADR-009 / RB-01 |
| `silent_detection` semantics? | Remove the flag entirely — superseded by the suite e2e harness. | ADR-010 / RB-02 |
| F1 atomicity (FS / DB / outbox)? | Adopt a business-transaction wrapper (transactional outbox); FS writes go post-commit. | ADR-008 / RB-03 |
| `XxHash64` over sample collision risk? | Switch to `XxHash3.Hash128` over the same sample (file-size-independent — videos can be 3–5 GB). | ADR-004 / RB-04 |
| `FailsafeProducer.EnqueueAsync` static + DB I/O? | Accept as-is; documented `coderule.mdc` deviation. | (no refactor) |
| `detection_classes` static or admin-managed? | Admin-managed with read-through cache (`PathResolver`-style `Reset()`). | ADR-011 / RB-06 |
### Additional finding while verifying #1
- `FailsafeProducer.cs:138` has an empty `catch { }` that swallows `IOException` on image read and emits a stream message with `image = null`. Direct `coderule.mdc` violation ("never suppress errors silently"). Operationally invisible failure mode. Tracked as RB-05 (architecture doc).
## Step 4.5 follow-on resolutions (closed 2026-05-14)
Confirmed alongside the Step 4.5 condensed-view approval:
| Question | Resolution | Tracked |
|----------|------------|---------|
| Suite vs code: `Flight` (code) vs `mission` (suite spec) | Rename code → `Mission*`; suite stays canonical | ADR-012 / RB-07 |
| Stream consumer dedupe contract owner | This service owns it; dedupe by `(annotationId, operation, dateTime)` baked into the wire message | ADR-013 / RB-09 |
| Hard-delete vs soft-delete | Soft-delete: status → `Deleted (40)`, files relocated to a new `deleted_dir` | ADR-009 (folded in) / RB-01 |
| Tight coupling 04 Dataset ↔ 01 Annotations REST | Decouple — dataset writes flow through `AnnotationService` via a public domain interface | RB-08 |
## Remaining gaps and uncertainties (carried into Step 6 problem extraction)
1. **`media.duration` format**: TEXT NOT NULL is permissive; format is unspecified.
2. **Lazy-upsert semantics** for `system_settings` / `directory_settings` / `camera_settings` — confirm services initialize defaults vs rely on user-driven inserts.
3. **`UserId` body field vs JWT subject** drift — reconcile in suite spec or in code.
4. **No automated tests in repo**: addressed by autodev Phase A Steps 3–7.
## Completeness score
- 11 / 11 modules documented (`modules/*.md`).
- 6 / 6 components assembled (`components/*/description.md`).
- 1 / 1 module-layout file (`module-layout.md`).
- 1 / 1 architecture file (`architecture.md`).
- 1 / 1 system-flows file (`system-flows.md`) covering 8 flows.
- 1 / 1 data-model file (`data_model.md`) covering 9 tables.
- 4 / 4 deployment files (`deployment/*.md`).
- 3 flow diagrams (F1, F3, F4) in `diagrams/flows/`.
**Score: 100% of modules + components covered.** Remaining open items are behavioral questions, not coverage gaps.
Reference in New Issue View Git Blame Copy Permalink