Replace the WPF desktop application (Azaion.Suite, Azaion.Annotator, Azaion.Common, Azaion.Inference, Azaion.Loader, Azaion.LoaderUI, Azaion.Dataset, Azaion.Test) with a standalone .NET Web API in src/. Made-with: Cursor
4.4 KiB
E2E Black-Box Test Infrastructure Template
Describes a separate consumer application that tests the main system as a black box.
Save as PLANS_DIR/<topic>/e2e_test_infrastructure.md.
# E2E Test Infrastructure
## Overview
**System under test**: [main system name and entry points — API URLs, message queues, etc.]
**Consumer app purpose**: Standalone application that exercises the main system through its public interfaces, validating end-to-end use cases without access to internals.
## Docker Environment
### Services
| Service | Image / Build | Purpose | Ports |
|---------|--------------|---------|-------|
| system-under-test | [main app image or build context] | The main system being tested | [ports] |
| test-db | [postgres/mysql/etc.] | Database for the main system | [ports] |
| e2e-consumer | [build context for consumer app] | Black-box test runner | — |
| [dependency] | [image] | [purpose — cache, queue, etc.] | [ports] |
### Networks
| Network | Services | Purpose |
|---------|----------|---------|
| e2e-net | all | Isolated test network |
### Volumes
| Volume | Mounted to | Purpose |
|--------|-----------|---------|
| [name] | [service:path] | [test data, DB persistence, etc.] |
### docker-compose structure
```yaml
# Outline only — not runnable code
services:
system-under-test:
# main system
test-db:
# database
e2e-consumer:
# consumer test app
depends_on:
- system-under-test
Consumer Application
Tech stack: [language, framework, test runner] Entry point: [how it starts — e.g., pytest, jest, custom runner]
Communication with system under test
| Interface | Protocol | Endpoint / Topic | Authentication |
|---|---|---|---|
| [API name] | [HTTP/gRPC/AMQP/etc.] | [URL or topic] | [method] |
What the consumer does NOT have access to
- No direct database access to the main system
- No internal module imports
- No shared memory or file system with the main system
E2E Test Scenarios
Acceptance Criteria Traceability
| AC ID | Acceptance Criterion | E2E Test IDs | Coverage |
|---|---|---|---|
| AC-01 | [criterion] | E2E-01 | Covered |
| AC-02 | [criterion] | E2E-02, E2E-03 | Covered |
| AC-03 | [criterion] | — | NOT COVERED — [reason] |
E2E-01: [Scenario Name]
Summary: [One sentence: what end-to-end use case this validates]
Traces to: AC-01
Preconditions:
- [System state required before test]
Steps:
| Step | Consumer Action | Expected System Response |
|---|---|---|
| 1 | [call / send] | [response / event] |
| 2 | [call / send] | [response / event] |
Max execution time: [e.g., 10s]
E2E-02: [Scenario Name]
(repeat structure)
Test Data Management
Seed data:
| Data Set | Description | How Loaded | Cleanup |
|---|---|---|---|
| [name] | [what it contains] | [SQL script / API call / fixture file] | [how removed after test] |
Isolation strategy: [e.g., each test run gets a fresh DB via container restart, or transactions are rolled back, or namespaced data]
External dependencies: [any external APIs that need mocking or sandbox environments]
CI/CD Integration
When to run: [e.g., on PR merge to dev, nightly, before production deploy] Pipeline stage: [where in the CI pipeline this fits] Gate behavior: [block merge / warning only / manual approval] Timeout: [max total suite duration before considered failed]
Reporting
Format: CSV Columns: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message (if FAIL) Output path: [where the CSV is written — e.g., ./e2e-results/report.csv]
---
## Guidance Notes
- Every E2E test MUST trace to at least one acceptance criterion. If it doesn't, question whether it's needed.
- The consumer app must treat the main system as a true black box — no internal imports, no direct DB queries against the main system's database.
- Keep the number of E2E tests focused on critical use cases. Exhaustive testing belongs in per-component tests (Step 4).
- Docker environment should be self-contained — `docker compose up` must be sufficient to run the full suite.
- If the main system requires external services (payment gateways, third-party APIs), define mock/stub services in the Docker environment.