mirror of
https://github.com/azaion/annotations.git
synced 2026-04-22 22:46:30 +00:00
Oleksandr Bezdieniezhnykh
9e7dc290db
Replace the WPF desktop application (Azaion.Suite, Azaion.Annotator, Azaion.Common, Azaion.Inference, Azaion.Loader, Azaion.LoaderUI, Azaion.Dataset, Azaion.Test) with a standalone .NET Web API in src/. Made-with: Cursor
142 lines
4.4 KiB
Markdown
142 lines
4.4 KiB
Markdown
# E2E Black-Box Test Infrastructure Template
|
|
|
|
Describes a separate consumer application that tests the main system as a black box.
|
|
Save as `PLANS_DIR/<topic>/e2e_test_infrastructure.md`.
|
|
|
|
---
|
|
|
|
```markdown
|
|
# E2E Test Infrastructure
|
|
|
|
## Overview
|
|
|
|
**System under test**: [main system name and entry points — API URLs, message queues, etc.]
|
|
**Consumer app purpose**: Standalone application that exercises the main system through its public interfaces, validating end-to-end use cases without access to internals.
|
|
|
|
## Docker Environment
|
|
|
|
### Services
|
|
|
|
| Service | Image / Build | Purpose | Ports |
|
|
|---------|--------------|---------|-------|
|
|
| system-under-test | [main app image or build context] | The main system being tested | [ports] |
|
|
| test-db | [postgres/mysql/etc.] | Database for the main system | [ports] |
|
|
| e2e-consumer | [build context for consumer app] | Black-box test runner | — |
|
|
| [dependency] | [image] | [purpose — cache, queue, etc.] | [ports] |
|
|
|
|
### Networks
|
|
|
|
| Network | Services | Purpose |
|
|
|---------|----------|---------|
|
|
| e2e-net | all | Isolated test network |
|
|
|
|
### Volumes
|
|
|
|
| Volume | Mounted to | Purpose |
|
|
|--------|-----------|---------|
|
|
| [name] | [service:path] | [test data, DB persistence, etc.] |
|
|
|
|
### docker-compose structure
|
|
|
|
```yaml
|
|
# Outline only — not runnable code
|
|
services:
|
|
system-under-test:
|
|
# main system
|
|
test-db:
|
|
# database
|
|
e2e-consumer:
|
|
# consumer test app
|
|
depends_on:
|
|
- system-under-test
|
|
```
|
|
|
|
## Consumer Application
|
|
|
|
**Tech stack**: [language, framework, test runner]
|
|
**Entry point**: [how it starts — e.g., pytest, jest, custom runner]
|
|
|
|
### Communication with system under test
|
|
|
|
| Interface | Protocol | Endpoint / Topic | Authentication |
|
|
|-----------|----------|-----------------|----------------|
|
|
| [API name] | [HTTP/gRPC/AMQP/etc.] | [URL or topic] | [method] |
|
|
|
|
### What the consumer does NOT have access to
|
|
|
|
- No direct database access to the main system
|
|
- No internal module imports
|
|
- No shared memory or file system with the main system
|
|
|
|
## E2E Test Scenarios
|
|
|
|
### Acceptance Criteria Traceability
|
|
|
|
| AC ID | Acceptance Criterion | E2E Test IDs | Coverage |
|
|
|-------|---------------------|-------------|----------|
|
|
| AC-01 | [criterion] | E2E-01 | Covered |
|
|
| AC-02 | [criterion] | E2E-02, E2E-03 | Covered |
|
|
| AC-03 | [criterion] | — | NOT COVERED — [reason] |
|
|
|
|
### E2E-01: [Scenario Name]
|
|
|
|
**Summary**: [One sentence: what end-to-end use case this validates]
|
|
|
|
**Traces to**: AC-01
|
|
|
|
**Preconditions**:
|
|
- [System state required before test]
|
|
|
|
**Steps**:
|
|
|
|
| Step | Consumer Action | Expected System Response |
|
|
|------|----------------|------------------------|
|
|
| 1 | [call / send] | [response / event] |
|
|
| 2 | [call / send] | [response / event] |
|
|
|
|
**Max execution time**: [e.g., 10s]
|
|
|
|
---
|
|
|
|
### E2E-02: [Scenario Name]
|
|
|
|
(repeat structure)
|
|
|
|
---
|
|
|
|
## Test Data Management
|
|
|
|
**Seed data**:
|
|
|
|
| Data Set | Description | How Loaded | Cleanup |
|
|
|----------|-------------|-----------|---------|
|
|
| [name] | [what it contains] | [SQL script / API call / fixture file] | [how removed after test] |
|
|
|
|
**Isolation strategy**: [e.g., each test run gets a fresh DB via container restart, or transactions are rolled back, or namespaced data]
|
|
|
|
**External dependencies**: [any external APIs that need mocking or sandbox environments]
|
|
|
|
## CI/CD Integration
|
|
|
|
**When to run**: [e.g., on PR merge to dev, nightly, before production deploy]
|
|
**Pipeline stage**: [where in the CI pipeline this fits]
|
|
**Gate behavior**: [block merge / warning only / manual approval]
|
|
**Timeout**: [max total suite duration before considered failed]
|
|
|
|
## Reporting
|
|
|
|
**Format**: CSV
|
|
**Columns**: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message (if FAIL)
|
|
**Output path**: [where the CSV is written — e.g., ./e2e-results/report.csv]
|
|
```
|
|
|
|
---
|
|
|
|
## Guidance Notes
|
|
|
|
- Every E2E test MUST trace to at least one acceptance criterion. If it doesn't, question whether it's needed.
|
|
- The consumer app must treat the main system as a true black box — no internal imports, no direct DB queries against the main system's database.
|
|
- Keep the number of E2E tests focused on critical use cases. Exhaustive testing belongs in per-component tests (Step 4).
|
|
- Docker environment should be self-contained — `docker compose up` must be sufficient to run the full suite.
|
|
- If the main system requires external services (payment gateways, third-party APIs), define mock/stub services in the Docker environment.
|