azaion/annotations
1
0
Fork 0
mirror of https://github.com/azaion/annotations.git synced 2026-06-21 09:41:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs+src: complete Steps 1-3 outcomes + auth re-sync baseline

Browse Source 
This commit captures everything produced during autodev existing-code
Steps 1 (Document), 2 (Architecture Baseline Scan), and 3 (Test Spec),
together with the targeted auth + CORS re-sync triggered on 2026-05-14
when codebase drift was detected at Step 4 entry. None of this work was
previously committed.

Step 1 (Document) — 50+ _docs/02_document/ files: problem, solution,
architecture, system flows, glossary, module-layout, per-component
specs (01..06), modules, deployment, diagrams, data model, FINAL
report, verification log, discovery.

Step 2 (Architecture Baseline) — architecture_compliance_baseline.md.
Verdict PASS_WITH_WARNINGS (0 Critical, 0 High, 1 Medium, 2 Low). No
High/Critical findings; auto-chained to Step 3 per existing-code flow.

Step 3 (Test Spec) — _docs/02_document/tests/* (67 scenarios across
blackbox, security, resilience, resource-limit, performance), plus
e2e/docker-compose.test.yml, e2e/seed/run.sh, scripts/run-tests.sh,
scripts/run-performance-tests.sh. Coverage 88% over the active scope
(40 of 45 items covered, 6 RB-deferred, 5 documented-as-uncovered).

Targeted auth + CORS re-sync — replaces the deleted in-house token
issuer with a JWKS-verifier model. AuthController and TokenService
removed; JwtExtensions switched from HS256 symmetric to ES256 over
admin's JWKS. ConfigurationResolver and CorsConfigurationValidator
added under src/Infrastructure/. ADR-002 and ADR-006 retired; SEC-01,
SEC-02, SEC-03 marked Closed. One new testability risk recorded in
architecture.md Open Risks Section 6 (JWKS HTTPS gating).

Source changes:
- src/Auth/JwtExtensions.cs (modified) — ES256, JWKS, alg pinning
- src/Program.cs (modified) — DI wiring for ConfigurationResolver
  and CorsConfigurationValidator
- src/Controllers/AuthController.cs (deleted) — no in-service issuance
- src/Services/TokenService.cs (deleted) — same
- src/Infrastructure/ConfigurationResolver.cs (new)
- src/Infrastructure/CorsConfigurationValidator.cs (new)
- .env.example (new) — required env var documentation
- .gitignore (updated)

Cross-repo coordination: _docs/cross-repo/flights_h1_h2_h3_change_spec
captures the change-spec for downstream services that consumed the now
deleted /auth endpoints.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-14 20:19:05 +03:00
parent 08eadc1158
commit 03f879206e
66 changed files with 6006 additions and 133 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_document/tests/environment.md
+218
View File
@@ -0,0 +1,218 @@
# Test Environment
## Overview
**System under test**: `Azaion.Annotations` HTTP API on port 8080 (REST + SSE) plus its RabbitMQ Stream producer (`azaion-annotations` stream).
**Consumer app purpose**: A standalone test runner that exercises the system through its public HTTP / SSE / Stream interfaces only — no in-process imports, no direct DB queries against the system's main DB, no shared filesystem.
## Docker Environment
### Services
| Service | Image / Build | Purpose | Ports |
|---------|--------------|---------|-------|
| `annotations` | Built from `src/Dockerfile` (ARM64) with `AZAION_REVISION=test-<sha>` | System under test | `8080:8080` |
| `postgres` | `postgres:13` | DB for the system under test | `5432:5432` (private to test net) |
| `rabbitmq` | `rabbitmq:3.13-management` with the **streams plugin** enabled | Stream broker the SUT publishes to | `5552:5552` (stream listener), `15672:15672` (mgmt UI, optional) |
| `e2e-runner` | Built from `tests/Azaion.Annotations.E2E/Dockerfile` | Black-box test runner (xUnit + HttpClient + RabbitMQ.Stream.Client consumer); also holds the ES256 private key used to mint per-test bearer tokens | — |
| `e2e-issuer` | `python:3.12-alpine` running `tests/harness/mock_issuer.py` (≈40 lines, serves a static JWKS over HTTP) | Mock JWKS endpoint stand-in for admin's real issuer; publishes the public ES256 key the SUT validates against | `8080` (on `e2e-net`; not exposed to host) |
| `dataseed` | One-shot job: `psql` only | Boot-time seed of any required reference data (no users — annotations has no `users` table) | — |
The fixture binaries (frame images, videos) are mounted from `../detections/_docs/00_problem/input_data/` (suite-relative path, see `_docs/00_problem/input_data/fixtures.md`) into both the `annotations` service (read-only, for direct file ingestion paths) and the `e2e-runner` (read-only, for upload-as-multipart paths).
### Networks
| Network | Services | Purpose |
|---------|----------|---------|
| `e2e-net` | `annotations`, `postgres`, `rabbitmq`, `e2e-issuer`, `e2e-runner`, `dataseed` | Isolated bridge network — services reach each other by container hostname |
### Volumes
| Volume | Mounted to | Purpose |
|--------|-----------|---------|
| `annotations-images` | `annotations:/data/images` | `images_dir` — content-addressed image bytes + YOLO label files |
| `annotations-videos` | `annotations:/data/videos` | `videos_dir` |
| `annotations-deleted` | `annotations:/data/deleted` | `deleted_dir` (post RB-01 soft-delete relocation) |
| `pg-data` | `postgres:/var/lib/postgresql/data` | DB durability across container restart (resilience scenarios) |
| `fixtures-ro` (bind) | `annotations:/fixtures:ro`, `e2e-runner:/fixtures:ro` | Reuse of detections corpus binaries |
### docker-compose structure
```yaml
services:
postgres:
image: postgres:13
environment:
POSTGRES_DB: annotations
POSTGRES_USER: annotations
POSTGRES_PASSWORD: annotations
volumes:
- pg-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U annotations"]
rabbitmq:
image: rabbitmq:3.13-management
environment:
RABBITMQ_DEFAULT_USER: annotations
RABBITMQ_DEFAULT_PASS: annotations
RABBITMQ_PLUGINS: rabbitmq_stream rabbitmq_management
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "ping"]
e2e-issuer:
image: python:3.12-alpine
command: ["python", "/harness/mock_issuer.py"]
volumes:
- ../tests/harness:/harness:ro
- jwt-keys:/keys
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/.well-known/jwks.json"]
annotations:
build:
context: ../src
environment:
ASPNETCORE_ENVIRONMENT: E2ETest
DATABASE_URL: postgresql://annotations:annotations@postgres:5432/annotations
JWT_ISSUER: https://e2e-issuer.test
JWT_AUDIENCE: annotations-e2e
JWT_JWKS_URL: http://e2e-issuer:8080/.well-known/jwks.json
CorsConfig__AllowedOrigins__0: http://e2e-runner.test
RABBITMQ_HOST: rabbitmq
RABBITMQ_STREAM_PORT: 5552
RABBITMQ_PRODUCER_USER: annotations
RABBITMQ_PRODUCER_PASS: annotations
AZAION_REVISION: test-${GIT_SHA:-local}
volumes:
- annotations-images:/data/images
- annotations-videos:/data/videos
- annotations-deleted:/data/deleted
- ../../detections/_docs/00_problem/input_data:/fixtures:ro
depends_on:
postgres:
condition: service_healthy
rabbitmq:
condition: service_healthy
e2e-issuer:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
dataseed:
image: postgres:13
depends_on:
annotations:
condition: service_healthy
entrypoint: ["/bin/sh", "/seed/run.sh"]
volumes:
- ./seed:/seed:ro
e2e-runner:
build:
context: ../tests/Azaion.Annotations.E2E
depends_on:
dataseed:
condition: service_completed_successfully
environment:
ANNOTATIONS_BASE_URL: http://annotations:8080
JWT_ISSUER: https://e2e-issuer.test
JWT_AUDIENCE: annotations-e2e
RABBITMQ_HOST: rabbitmq
RABBITMQ_STREAM_PORT: 5552
RABBITMQ_USER: annotations
RABBITMQ_PASS: annotations
FIXTURES_DIR: /fixtures
volumes:
- ../../detections/_docs/00_problem/input_data:/fixtures:ro
- jwt-keys:/keys:ro
volumes:
pg-data: {}
annotations-images: {}
annotations-videos: {}
annotations-deleted: {}
jwt-keys: {}
networks:
default:
name: e2e-net
```
## Consumer Application
**Tech stack**: .NET 10 + xUnit (matches the SUT runtime to avoid a second toolchain in CI). Uses `HttpClient` for REST, raw `HttpClient` with `text/event-stream` for SSE, and `RabbitMQ.Stream.Client` for stream-consumer scenarios.
**Entry point**: `dotnet test --logger "console;verbosity=normal" --logger "trx" --results-directory /results`
### Communication with system under test
| Interface | Protocol | Endpoint / Topic | Authentication |
|-----------|----------|-----------------|----------------|
| Annotations REST | HTTP/1.1 JSON | `http://annotations:8080/annotations/*`, `/media/*`, `/dataset/*`, `/settings/*`, `/classes`, `/health` | `Authorization: Bearer <jwt>` (ES256 JWT minted on demand by the runner using the in-stack mock-issuer key) |
| Annotations SSE | HTTP/1.1 `text/event-stream` | `http://annotations:8080/annotations/events?missionId=<guid>` | Same ES256 bearer token |
| Mock JWKS | HTTP/1.1 JSON | `http://e2e-issuer:8080/.well-known/jwks.json` | None (test-net only) |
| RabbitMQ Stream | AMQP 1.0 / streams (port 5552) | Stream `azaion-annotations` | Username + password env vars; consumer offset starts at `next` for fresh test runs |
| Postgres (test-only, read-only assertions on DB state) | direct (out-of-band) | `postgresql://postgres:5432/annotations` | DB user; **only the test runner uses this and only for blackbox-allowed assertions** (e.g., F4-001 verifying the outbox row was inserted). Tests that need DB introspection are clearly marked. |
### What the consumer does NOT have access to
- No in-process import of the `Azaion.Annotations` assembly.
- No direct write to the SUT's `annotations`, `media`, `detection`, `annotations_queue_records` tables (DB read access only, for outbox-state assertions documented in `test-data.md`). Annotations has no `users` table.
- No shared memory or filesystem with the SUT (volumes are mounted read-only).
- No mocking of internal services (`AnnotationService`, `FailsafeProducer`, etc.) — all interactions go through the public surface.
## CI/CD Integration
**When to run**: on every push to `dev` and on every PR; nightly full run including the long-running performance + resilience scenarios.
**Pipeline stage**: after Woodpecker `build` step; new step `test-e2e` invoking `docker compose -f e2e/docker-compose.test.yml up --abort-on-container-exit --exit-code-from e2e-runner` (or, equivalently, `scripts/run-tests.sh`).
**Gate behavior**: any failed scenario blocks the merge; nightly perf failures emit a warning but do not block a green PR.
**Timeout**: 30 min for the standard suite (functional + smoke perf); 2 hours for the nightly full perf + resilience suite.
## Reporting
**Format**: CSV (xUnit's `trx` output is converted by the runner into a flat CSV).
**Columns**: `test_id`, `test_name`, `category`, `traces_to`, `execution_time_ms`, `result`, `error_message`.
**Output path**: `e2e-results/report.csv` inside the `e2e-runner` container, mounted out to `./e2e-results/report.csv` on the host.
In addition, raw xUnit `.trx` is preserved at `e2e-results/results.trx` for human inspection / IDE integration.
## Dependencies on the existing stack
This environment intentionally **does not** re-use the suite's running development DB or RabbitMQ — it stands up its own. The only suite-level dependency is the read-only mount of `detections/_docs/00_problem/input_data/` for fixtures.
## Test Execution
**Decision**: Docker only.
**Rationale** (from Hardware-Dependency Assessment, run between test-spec Phase 3 and Phase 4):
- **Documentation scan** — `restrictions.md` lists HW-01 (ARM64-only image), HW-02 (writable filesystem dirs), HW-03 (memory pressure on `FailsafeProducer`). None of these are accelerator / sensor / OS-feature dependencies; they are generic infrastructure constraints satisfiable in any Linux container.
- **Code scan** — zero hits across `src/` for CUDA, TensorRT, CoreML, OpenCL, Vulkan, TPU, V4L2, GPIO, `cv2.VideoCapture`, `sys.platform`-style branches, or `platform.machine()` checks. The Dockerfile's `TARGETARCH` branch (line 5) is a buildplatform-aware Node toolchain selector, not a runtime hardware gate — the running binary uses managed .NET 10 with no native acceleration paths.
- **Dependency files** — `Azaion.Annotations.csproj` references only managed NuGet packages (Linq2DB, Npgsql, JwtBearer, RabbitMQ.Stream.Client, MessagePack, Swashbuckle, System.IO.Hashing). No native-binding libraries, no hardware-specific packages.
**Classification**: not hardware-dependent. Docker is the preferred default and the only chosen mode.
### Docker mode — execution instructions
Run from the suite root (parent of `annotations/` and `detections/`) so the fixture bind-mount path resolves:
```bash
# From the annotations repo root:
./scripts/run-tests.sh # functional + smoke perf
./scripts/run-performance-tests.sh # full perf scenarios
# Equivalent without the wrapper:
docker compose -f e2e/docker-compose.test.yml up \
--abort-on-container-exit \
--exit-code-from e2e-runner
```
Results land at `e2e/e2e-results/report.csv` (host path), and at `test-results/` for any JUnit/CTRX outputs. The exit code of `e2e-runner` becomes the suite's exit code; CI uses it as the gate.
### Why not local mode
The xUnit test runner CAN execute against a SUT bound to `localhost:8080` if a developer wants to iterate inside the IDE. That path is not the supported test environment for CI; it is a developer convenience. Phase 4 produces only the Docker runner script.
### CI image arch
The Docker test stack runs on the same ARM64 hosts the Woodpecker pipeline already targets (HW-01). If a future CI runner family is x86_64-only, the same docker-compose works because every service in `e2e-net` is multi-arch (`postgres:13`, `rabbitmq:3.13-management`, the SUT itself if rebuilt with `--platform linux/amd64`).