annotations/_docs/02_document/tests/environment.md

Test Environment

Overview

System under test: Azaion.Annotations HTTP API on port 8080 (REST + SSE) plus its RabbitMQ Stream producer (azaion-annotations stream). Consumer app purpose: A standalone test runner that exercises the system through its public HTTP / SSE / Stream interfaces only — no in-process imports, no direct DB queries against the system's main DB, no shared filesystem.

Docker Environment

Services

Service	Image / Build	Purpose	Ports
annotations	Built from src/Dockerfile (ARM64) with AZAION_REVISION=test-<sha>	System under test	8080:8080
postgres	postgres:13	DB for the system under test	5432:5432 (private to test net)
rabbitmq	rabbitmq:3.13-management with the streams plugin enabled	Stream broker the SUT publishes to	5552:5552 (stream listener), 15672:15672 (mgmt UI, optional)
e2e-runner	Built from tests/Azaion.Annotations.E2E/Dockerfile	Black-box test runner (xUnit + HttpClient + RabbitMQ.Stream.Client consumer); also holds the ES256 private key used to mint per-test bearer tokens	—
e2e-issuer	python:3.12-alpine running tests/harness/mock_issuer.py (≈40 lines, serves a static JWKS over HTTP)	Mock JWKS endpoint stand-in for admin's real issuer; publishes the public ES256 key the SUT validates against	8080 (on e2e-net; not exposed to host)
dataseed	One-shot job: psql only	Boot-time seed of any required reference data (no users — annotations has no users table)	—

The fixture binaries (frame images, videos) are mounted from ../detections/_docs/00_problem/input_data/ (suite-relative path, see _docs/00_problem/input_data/fixtures.md) into both the annotations service (read-only, for direct file ingestion paths) and the e2e-runner (read-only, for upload-as-multipart paths).

Networks

Network	Services	Purpose
e2e-net	annotations, postgres, rabbitmq, e2e-issuer, e2e-runner, dataseed	Isolated bridge network — services reach each other by container hostname

Volumes

Volume	Mounted to	Purpose
annotations-images	annotations:/data/images	images_dir — content-addressed image bytes + YOLO label files
annotations-videos	annotations:/data/videos	videos_dir
annotations-deleted	annotations:/data/deleted	deleted_dir (post RB-01 soft-delete relocation)
pg-data	postgres:/var/lib/postgresql/data	DB durability across container restart (resilience scenarios)
fixtures-ro (bind)	annotations:/fixtures:ro, e2e-runner:/fixtures:ro	Reuse of detections corpus binaries

docker-compose structure

services:
  postgres:
    image: postgres:13
    environment:
      POSTGRES_DB: annotations
      POSTGRES_USER: annotations
      POSTGRES_PASSWORD: annotations
    volumes:
      - pg-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U annotations"]

  rabbitmq:
    image: rabbitmq:3.13-management
    environment:
      RABBITMQ_DEFAULT_USER: annotations
      RABBITMQ_DEFAULT_PASS: annotations
      RABBITMQ_PLUGINS: rabbitmq_stream rabbitmq_management
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "ping"]

  e2e-issuer:
    image: python:3.12-alpine
    command: ["python", "/harness/mock_issuer.py"]
    volumes:
      - ../tests/harness:/harness:ro
      - jwt-keys:/keys
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost:8080/.well-known/jwks.json"]

  annotations:
    build:
      context: ../src
    environment:
      ASPNETCORE_ENVIRONMENT: E2ETest
      DATABASE_URL: postgresql://annotations:annotations@postgres:5432/annotations
      JWT_ISSUER: https://e2e-issuer.test
      JWT_AUDIENCE: annotations-e2e
      JWT_JWKS_URL: http://e2e-issuer:8080/.well-known/jwks.json
      CorsConfig__AllowedOrigins__0: http://e2e-runner.test
      RABBITMQ_HOST: rabbitmq
      RABBITMQ_STREAM_PORT: 5552
      RABBITMQ_PRODUCER_USER: annotations
      RABBITMQ_PRODUCER_PASS: annotations
      AZAION_REVISION: test-${GIT_SHA:-local}
    volumes:
      - annotations-images:/data/images
      - annotations-videos:/data/videos
      - annotations-deleted:/data/deleted
      - ../../detections/_docs/00_problem/input_data:/fixtures:ro
    depends_on:
      postgres:
        condition: service_healthy
      rabbitmq:
        condition: service_healthy
      e2e-issuer:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]

  dataseed:
    image: postgres:13
    depends_on:
      annotations:
        condition: service_healthy
    entrypoint: ["/bin/sh", "/seed/run.sh"]
    volumes:
      - ./seed:/seed:ro

  e2e-runner:
    build:
      context: ../tests/Azaion.Annotations.E2E
    depends_on:
      dataseed:
        condition: service_completed_successfully
    environment:
      ANNOTATIONS_BASE_URL: http://annotations:8080
      JWT_ISSUER: https://e2e-issuer.test
      JWT_AUDIENCE: annotations-e2e
      RABBITMQ_HOST: rabbitmq
      RABBITMQ_STREAM_PORT: 5552
      RABBITMQ_USER: annotations
      RABBITMQ_PASS: annotations
      FIXTURES_DIR: /fixtures
    volumes:
      - ../../detections/_docs/00_problem/input_data:/fixtures:ro
      - jwt-keys:/keys:ro

volumes:
  pg-data: {}
  annotations-images: {}
  annotations-videos: {}
  annotations-deleted: {}
  jwt-keys: {}

networks:
  default:
    name: e2e-net

Consumer Application

Tech stack: .NET 10 + xUnit (matches the SUT runtime to avoid a second toolchain in CI). Uses HttpClient for REST, raw HttpClient with text/event-stream for SSE, and RabbitMQ.Stream.Client for stream-consumer scenarios. Entry point: dotnet test --logger "console;verbosity=normal" --logger "trx" --results-directory /results

Communication with system under test

Interface	Protocol	Endpoint / Topic	Authentication
Annotations REST	HTTP/1.1 JSON	http://annotations:8080/annotations/*, /media/*, /dataset/*, /settings/*, /classes, /health	Authorization: Bearer <jwt> (ES256 JWT minted on demand by the runner using the in-stack mock-issuer key)
Annotations SSE	HTTP/1.1 text/event-stream	http://annotations:8080/annotations/events?missionId=<guid>	Same ES256 bearer token
Mock JWKS	HTTP/1.1 JSON	http://e2e-issuer:8080/.well-known/jwks.json	None (test-net only)
RabbitMQ Stream	AMQP 1.0 / streams (port 5552)	Stream azaion-annotations	Username + password env vars; consumer offset starts at next for fresh test runs
Postgres (test-only, read-only assertions on DB state)	direct (out-of-band)	postgresql://postgres:5432/annotations	DB user; only the test runner uses this and only for blackbox-allowed assertions (e.g., F4-001 verifying the outbox row was inserted). Tests that need DB introspection are clearly marked.

What the consumer does NOT have access to

• No in-process import of the Azaion.Annotations assembly.
• No direct write to the SUT's annotations, media, detection, annotations_queue_records tables (DB read access only, for outbox-state assertions documented in test-data.md). Annotations has no users table.
• No shared memory or filesystem with the SUT (volumes are mounted read-only).
• No mocking of internal services (AnnotationService, FailsafeProducer, etc.) — all interactions go through the public surface.

CI/CD Integration

When to run: on every push to dev and on every PR; nightly full run including the long-running performance + resilience scenarios. Pipeline stage: after Woodpecker build step; new step test-e2e invoking docker compose -f e2e/docker-compose.test.yml up --abort-on-container-exit --exit-code-from e2e-runner (or, equivalently, scripts/run-tests.sh). Gate behavior: any failed scenario blocks the merge; nightly perf failures emit a warning but do not block a green PR. Timeout: 30 min for the standard suite (functional + smoke perf); 2 hours for the nightly full perf + resilience suite.

Reporting

Format: CSV (xUnit's trx output is converted by the runner into a flat CSV). Columns: test_id, test_name, category, traces_to, execution_time_ms, result, error_message. Output path: e2e-results/report.csv inside the e2e-runner container, mounted out to ./e2e-results/report.csv on the host.

In addition, raw xUnit .trx is preserved at e2e-results/results.trx for human inspection / IDE integration.

Dependencies on the existing stack

This environment intentionally does not re-use the suite's running development DB or RabbitMQ — it stands up its own. The only suite-level dependency is the read-only mount of detections/_docs/00_problem/input_data/ for fixtures.

Test Execution

Decision: Docker only.

Rationale (from Hardware-Dependency Assessment, run between test-spec Phase 3 and Phase 4):

• Documentation scan — restrictions.md lists HW-01 (ARM64-only image), HW-02 (writable filesystem dirs), HW-03 (memory pressure on FailsafeProducer). None of these are accelerator / sensor / OS-feature dependencies; they are generic infrastructure constraints satisfiable in any Linux container.
• Code scan — zero hits across src/ for CUDA, TensorRT, CoreML, OpenCL, Vulkan, TPU, V4L2, GPIO, cv2.VideoCapture, sys.platform-style branches, or platform.machine() checks. The Dockerfile's TARGETARCH branch (line 5) is a buildplatform-aware Node toolchain selector, not a runtime hardware gate — the running binary uses managed .NET 10 with no native acceleration paths.
• Dependency files — Azaion.Annotations.csproj references only managed NuGet packages (Linq2DB, Npgsql, JwtBearer, RabbitMQ.Stream.Client, MessagePack, Swashbuckle, System.IO.Hashing). No native-binding libraries, no hardware-specific packages.

Classification: not hardware-dependent. Docker is the preferred default and the only chosen mode.

Docker mode — execution instructions

Run from the suite root (parent of annotations/ and detections/) so the fixture bind-mount path resolves:

# From the annotations repo root:
./scripts/run-tests.sh           # functional + smoke perf
./scripts/run-performance-tests.sh   # full perf scenarios

# Equivalent without the wrapper:
docker compose -f e2e/docker-compose.test.yml up \
  --abort-on-container-exit \
  --exit-code-from e2e-runner

Results land at e2e/e2e-results/report.csv (host path), and at test-results/ for any JUnit/CTRX outputs. The exit code of e2e-runner becomes the suite's exit code; CI uses it as the gate.

Why not local mode

The xUnit test runner CAN execute against a SUT bound to localhost:8080 if a developer wants to iterate inside the IDE. That path is not the supported test environment for CI; it is a developer convenience. Phase 4 produces only the Docker runner script.

CI image arch

The Docker test stack runs on the same ARM64 hosts the Woodpecker pipeline already targets (HW-01). If a future CI runner family is x86_64-only, the same docker-compose works because every service in e2e-net is multi-arch (postgres:13, rabbitmq:3.13-management, the SUT itself if rebuilt with --platform linux/amd64).