azaion/admin
1
0
Fork 0
mirror of https://github.com/azaion/admin.git synced 2026-04-22 08:16:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-189] [AZ-190] [AZ-191] [AZ-192] [AZ-193] [AZ-194] [AZ-195] Add e2e blackbox test suite

Browse Source 
Made-with: Cursor
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-04-16 06:25:36 +03:00
parent 1b38e888e1
commit d320d6dd59
98 changed files with 6883 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_tasks/_dependencies_table.md
+15
View File
@@ -0,0 +1,15 @@
# Dependencies Table
**Date**: 2026-04-16
**Total Tasks**: 7
**Total Complexity Points**: 29
| Task | Name | Complexity | Dependencies | Epic |
|------|------|-----------|-------------|------|
| AZ-189 | test_infrastructure | 5 | None | AZ-188 |
| AZ-190 | auth_tests | 3 | AZ-189 | AZ-188 |
| AZ-191 | user_mgmt_tests | 5 | AZ-189, AZ-190 | AZ-188 |
| AZ-192 | hardware_tests | 3 | AZ-189, AZ-190 | AZ-188 |
| AZ-193 | resource_tests | 5 | AZ-189, AZ-190, AZ-192 | AZ-188 |
| AZ-194 | security_tests | 3 | AZ-189, AZ-190 | AZ-188 |
| AZ-195 | resilience_perf_tests | 5 | AZ-189, AZ-190 | AZ-188 |
_docs/02_tasks/done/AZ-189_test_infrastructure.md
+104
View File
@@ -0,0 +1,104 @@
# Test Infrastructure
**Task**: AZ-189_test_infrastructure
**Name**: Test Infrastructure
**Description**: Scaffold the blackbox test project — test runner, Docker test environment, seed data fixtures, reporting
**Complexity**: 5 points
**Dependencies**: None
**Component**: Blackbox Tests
**Tracker**: AZ-189
**Epic**: AZ-188
## Test Project Folder Layout
```
e2e/
├── Azaion.E2E/
│ ├── Azaion.E2E.csproj
│ ├── Helpers/
│ │ ├── ApiClient.cs
│ │ └── TestFixture.cs
│ ├── Tests/
│ │ ├── AuthTests.cs
│ │ ├── UserManagementTests.cs
│ │ ├── HardwareBindingTests.cs
│ │ ├── ResourceTests.cs
│ │ ├── SecurityTests.cs
│ │ └── ResilienceTests.cs
│ └── appsettings.test.json
├── docker-compose.test.yml
└── README.md
```
### Layout Rationale
xUnit project with shared test fixture for API client and JWT token management. Tests grouped by domain area matching the blackbox test spec categories.
## Docker Test Environment
### docker-compose.test.yml Structure
| Service | Image / Build | Purpose | Depends On |
|---------|--------------|---------|------------|
| test-db | postgres:16-alpine | PostgreSQL with schema init | — |
| system-under-test | Build from Dockerfile | Azaion Admin API | test-db |
| e2e-consumer | Build from e2e/ | xUnit test runner | system-under-test |
### Networks and Volumes
- `e2e-net`: Isolated test network
- `db-init`: SQL scripts mounted to test-db for schema initialization
- `test-resources`: Shared volume for resource file upload/download tests
## Test Runner Configuration
**Framework**: xUnit 2.9.2 with FluentAssertions 6.12.2
**HTTP Client**: System.Net.Http.HttpClient
**Entry point**: `dotnet test` in the e2e-consumer container
### Fixture Strategy
| Fixture | Scope | Purpose |
|---------|-------|---------|
| ApiTestFixture | Collection | Shared HttpClient, admin JWT token, base URL configuration |
| UserFixture | Test | Creates/deletes test users per test method |
| ResourceFixture | Test | Uploads/cleans test resource files per test method |
## Test Data Fixtures
| Data Set | Source | Format | Used By |
|----------|--------|--------|---------|
| seed-users | SQL init scripts (env/db/) | PostgreSQL rows | All tests |
| test-files | Generated at test start | Binary/text files | Resource tests |
### Data Isolation
Fresh Docker Compose environment per test run. Test users created during tests are cleaned up via DELETE API. Resource files cleaned via ClearFolder endpoint.
## Test Reporting
**Format**: CSV via xUnit test logger
**Columns**: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message
**Output path**: `./e2e-results/report.csv`
## Acceptance Criteria
**AC-1: Test environment starts**
Given the docker-compose.test.yml
When `docker compose -f docker-compose.test.yml up` is executed
Then all services start and the system-under-test responds at port 8080
**AC-2: Database initialized**
Given the test environment is running
When the e2e-consumer connects to the API
Then seed users (admin@azaion.com, uploader@azaion.com) exist
**AC-3: Test runner executes**
Given the test environment is running
When the e2e-consumer starts
Then the xUnit test runner discovers and executes test files
**AC-4: Test report generated**
Given tests have been executed
When the test run completes
Then a CSV report file exists at the configured output path
_docs/02_tasks/done/AZ-190_auth_tests.md
+62
View File
@@ -0,0 +1,62 @@
# Authentication Blackbox Tests
**Task**: AZ-190_auth_tests
**Name**: Auth Blackbox Tests
**Description**: Implement blackbox tests for login, JWT validation, and authentication error handling
**Complexity**: 3 points
**Dependencies**: AZ-189_test_infrastructure
**Component**: Blackbox Tests
**Tracker**: AZ-190
**Epic**: AZ-188
## Problem
The login and JWT authentication flows have no automated test coverage. Regressions in credential validation or token generation would go undetected.
## Outcome
- Login with valid credentials returns a JWT token (FT-P-01)
- JWT token contains correct issuer, audience, and lifetime claims (FT-P-03)
- Login with unknown email returns error code 10 (FT-N-01)
- Login with wrong password returns error code 30 (FT-N-02)
## Scope
### Included
- Login endpoint positive and negative scenarios
- JWT token structure and claims validation
### Excluded
- Token refresh (not implemented)
- Rate limiting on login (not implemented)
## Acceptance Criteria
**AC-1: Successful login**
Given a seed user exists
When POST /login is called with valid credentials
Then HTTP 200 is returned with a non-empty JWT token
**AC-2: JWT claims**
Given a valid JWT token from login
When the token payload is decoded
Then iss = "AzaionApi", aud = "Annotators/OrangePi/Admins", exp ≈ iat + 4 hours (± 60s)
**AC-3: Unknown email**
Given no user with the specified email exists
When POST /login is called
Then HTTP 409 is returned with ExceptionEnum code 10
**AC-4: Wrong password**
Given a user exists with a different password
When POST /login is called with wrong password
Then HTTP 409 is returned with ExceptionEnum code 30
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|------------------------|-------------|-------------------|----------------|
| AC-1 | Seed admin user | POST /login valid | HTTP 200, token present | — |
| AC-2 | JWT from AC-1 | Decode token claims | iss, aud, exp correct | — |
| AC-3 | No matching user | POST /login unknown email | HTTP 409, code 10 | — |
| AC-4 | Seed user, wrong password | POST /login wrong pass | HTTP 409, code 30 | — |
_docs/02_tasks/done/AZ-191_user_mgmt_tests.md
+102
View File
@@ -0,0 +1,102 @@
# User Management Blackbox Tests
**Task**: AZ-191_user_mgmt_tests
**Name**: User Management Blackbox Tests
**Description**: Implement blackbox tests for registration, CRUD operations, role changes, enable/disable
**Complexity**: 5 points
**Dependencies**: AZ-189_test_infrastructure, AZ-190_auth_tests
**Component**: Blackbox Tests
**Tracker**: AZ-191
**Epic**: AZ-188
## Problem
User management operations (registration, listing, role changes, deletion) have no automated test coverage.
## Outcome
- Registration with valid data succeeds (FT-P-02)
- User list returns seed users (FT-P-06)
- User filter by email works (FT-P-07)
- Role change succeeds (FT-P-11)
- Account disable succeeds (FT-P-12)
- User deletion succeeds (FT-P-13)
- Registration validation rejects invalid input (FT-N-03, FT-N-04, FT-N-07, FT-N-08)
- Non-admin cannot manage users (tested in security tests)
## Scope
### Included
- Registration positive and negative scenarios
- User CRUD operations (list, filter, role change, enable/disable, delete)
- FluentValidation error cases
### Excluded
- Non-admin access (covered by security tests AZ-194)
## Acceptance Criteria
**AC-1: Registration**
Given caller is ApiAdmin
When POST /users is called with valid email (>= 8 chars, valid format), password (>= 8 chars), and role
Then HTTP 200 is returned
**AC-2: List users**
Given seed users exist
When GET /users is called with ApiAdmin JWT
Then HTTP 200 with JSON array containing >= 1 user
**AC-3: Filter users**
Given seed users exist
When GET /users?email=admin is called
Then all returned emails contain "admin"
**AC-4: Change role**
Given a test user exists
When PUT /users/role is called with new role
Then HTTP 200
**AC-5: Disable user**
Given a test user exists
When PUT /users/enable with isEnabled=false
Then HTTP 200
**AC-6: Delete user**
Given a test user exists
When DELETE /users?email=user
Then HTTP 200
**AC-7: Short email rejected**
Given caller is ApiAdmin
When POST /users with email < 8 chars
Then HTTP 400
**AC-8: Invalid email format rejected**
Given caller is ApiAdmin
When POST /users with invalid email format
Then HTTP 400
**AC-9: Short password rejected**
Given caller is ApiAdmin
When POST /users with password < 8 chars
Then HTTP 400
**AC-10: Duplicate email rejected**
Given user with email already exists
When POST /users with same email
Then HTTP 409 with code 20
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|------------------------|-------------|-------------------|----------------|
| AC-1 | ApiAdmin JWT | POST /users valid | HTTP 200 | — |
| AC-2 | Seed data | GET /users | HTTP 200, array >= 1 | — |
| AC-3 | Seed data | GET /users?email=admin | Filtered results | — |
| AC-4 | Test user | PUT /users/role | HTTP 200 | — |
| AC-5 | Test user | PUT /users/enable false | HTTP 200 | — |
| AC-6 | Test user | DELETE /users | HTTP 200 | — |
| AC-7 | ApiAdmin JWT | POST /users short email | HTTP 400 | — |
| AC-8 | ApiAdmin JWT | POST /users bad format | HTTP 400 | — |
| AC-9 | ApiAdmin JWT | POST /users short pass | HTTP 400 | — |
| AC-10 | Existing user | POST /users duplicate | HTTP 409, code 20 | — |
_docs/02_tasks/done/AZ-192_hardware_tests.md
+54
View File
@@ -0,0 +1,54 @@
# Hardware Binding Blackbox Tests
**Task**: AZ-192_hardware_tests
**Name**: Hardware Binding Blackbox Tests
**Description**: Implement blackbox tests for hardware fingerprint binding, validation, and mismatch
**Complexity**: 3 points
**Dependencies**: AZ-189_test_infrastructure, AZ-190_auth_tests
**Component**: Blackbox Tests
**Tracker**: AZ-192
**Epic**: AZ-188
## Problem
Hardware binding is a critical security feature with no automated tests. A regression could allow unauthorized devices to access resources.
## Outcome
- First hardware check stores the fingerprint (FT-P-04)
- Same hardware passes on subsequent checks (FT-P-05)
- Different hardware triggers mismatch error (FT-N-06)
## Scope
### Included
- Hardware check endpoint (POST /resources/check)
- First-time binding, repeat validation, mismatch
### Excluded
- Admin hardware reset (covered in user management tests)
## Acceptance Criteria
**AC-1: First hardware binding**
Given a user with no hardware bound
When POST /resources/check is called with a hardware string
Then HTTP 200 with body true
**AC-2: Repeat hardware check**
Given a user with hardware already bound
When POST /resources/check is called with the same hardware
Then HTTP 200 with body true
**AC-3: Hardware mismatch**
Given a user with hardware bound to fingerprint A
When POST /resources/check is called with fingerprint B
Then HTTP 409 with ExceptionEnum code 40
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|------------------------|-------------|-------------------|----------------|
| AC-1 | New user, no hardware | POST /resources/check first time | HTTP 200, true | — |
| AC-2 | User with hardware A | POST /resources/check same hw | HTTP 200, true | — |
| AC-3 | User with hardware A | POST /resources/check different hw | HTTP 409, code 40 | — |
_docs/02_tasks/done/AZ-193_resource_tests.md
+71
View File
@@ -0,0 +1,71 @@
# Resource Distribution Blackbox Tests
**Task**: AZ-193_resource_tests
**Name**: Resource Blackbox Tests
**Description**: Implement blackbox tests for upload, encrypted download, and encrypt-decrypt round-trip verification
**Complexity**: 5 points
**Dependencies**: AZ-189_test_infrastructure, AZ-190_auth_tests, AZ-192_hardware_tests
**Component**: Blackbox Tests
**Tracker**: AZ-193
**Epic**: AZ-188
## Problem
The encrypted resource distribution flow is the most complex and security-critical feature, with no automated tests.
## Outcome
- File upload succeeds (FT-P-08)
- Encrypted download returns valid ciphertext (FT-P-09)
- Decrypt with same key derivation produces original content (FT-P-10)
- Upload with no file returns error (FT-N-05)
- Unauthenticated download rejected (tested in security tests)
## Scope
### Included
- Resource upload (POST /resources/{folder})
- Encrypted resource download (POST /resources/get)
- Encryption round-trip verification (client-side decryption)
- Empty upload error handling
### Excluded
- Installer download (simple stream, low risk)
- ClearFolder endpoint (utility)
## Acceptance Criteria
**AC-1: File upload**
Given caller is authenticated
When POST /resources/testfolder with multipart file
Then HTTP 200
**AC-2: Encrypted download**
Given a file is uploaded and user has bound hardware
When POST /resources/get with valid credentials
Then HTTP 200 with application/octet-stream content
**AC-3: Encryption round-trip**
Given a known file is uploaded
When the encrypted download is decrypted with the same key derivation (email + password + hwHash via SHA-384)
Then decrypted content byte-equals the original file
**AC-4: Empty upload rejected**
Given caller is authenticated
When POST /resources/testfolder with no file
Then HTTP 409 with ExceptionEnum code 70
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|------------------------|-------------|-------------------|----------------|
| AC-1 | Auth user | POST /resources/testfolder multipart | HTTP 200 | — |
| AC-2 | Uploaded file, bound hw | POST /resources/get | HTTP 200, binary | — |
| AC-3 | Known file, known creds | Download + decrypt | Byte equality | — |
| AC-4 | Auth user | POST /resources/testfolder no file | HTTP 409, code 70 | — |
## Risks & Mitigation
**Risk 1: Encryption key derivation mismatch**
- *Risk*: Test client must replicate the exact key derivation algorithm (SHA-384 with specific salt format)
- *Mitigation*: Reference Security.GetApiEncryptionKey implementation for exact salt template
_docs/02_tasks/done/AZ-194_security_tests.md
+80
View File
@@ -0,0 +1,80 @@
# Security Blackbox Tests
**Task**: AZ-194_security_tests
**Name**: Security Blackbox Tests
**Description**: Implement security tests: unauthenticated access, non-admin access, password exposure, expired JWT, encryption uniqueness, disabled user
**Complexity**: 3 points
**Dependencies**: AZ-189_test_infrastructure, AZ-190_auth_tests
**Component**: Blackbox Tests
**Tracker**: AZ-194
**Epic**: AZ-188
## Problem
Authorization boundaries and security properties have no automated verification. A misconfigured endpoint could expose data or allow unauthorized access.
## Outcome
- All protected endpoints reject unauthenticated requests (NFT-SEC-01)
- Non-admin users cannot access admin endpoints (NFT-SEC-02)
- Password hashes are not exposed in API responses (NFT-SEC-03)
- Expired JWT tokens are rejected (NFT-SEC-04)
- Different users get different encrypted content (NFT-SEC-05)
- Disabled users cannot login (NFT-SEC-06)
## Scope
### Included
- Authentication boundary tests (all protected endpoints)
- Authorization boundary tests (admin-only endpoints)
- Data exposure tests (password hash)
- Token expiration tests
- Encryption uniqueness verification
- Disabled account access
### Excluded
- CORS testing (browser-enforced)
- SQL injection (covered by ORM parameterization)
## Acceptance Criteria
**AC-1: Unauthenticated access blocked**
Given no JWT token
When any protected endpoint is called
Then HTTP 401
**AC-2: Non-admin blocked from admin endpoints**
Given Operator-role JWT
When admin endpoints (POST /users, PUT /users/role, DELETE /users) are called
Then HTTP 403
**AC-3: No password in response**
Given ApiAdmin JWT
When GET /users is called
Then no user object contains passwordHash or password field
**AC-4: Expired token rejected**
Given a JWT with exp in the past
When any protected endpoint is called
Then HTTP 401
**AC-5: Per-user encryption**
Given two users with different credentials and hardware
When both download the same resource
Then encrypted outputs differ
**AC-6: Disabled user blocked**
Given a disabled user account
When POST /login is called
Then login fails (HTTP 409 or 403)
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|------------------------|-------------|-------------------|----------------|
| AC-1 | No JWT | 6 protected endpoints | All HTTP 401 | — |
| AC-2 | Operator JWT | 4 admin endpoints | All HTTP 403 | — |
| AC-3 | Admin JWT | GET /users response | No password fields | — |
| AC-4 | Expired JWT | GET /users | HTTP 401 | — |
| AC-5 | Two users, same file | Download both | Different ciphertext | — |
| AC-6 | Disabled user | POST /login | Rejected | — |
_docs/02_tasks/done/AZ-195_resilience_perf_tests.md
+87
View File
@@ -0,0 +1,87 @@
# Resilience & Performance Tests
**Task**: AZ-195_resilience_perf_tests
**Name**: Resilience & Performance Tests
**Description**: Implement resilience tests (DB recovery, malformed JWT, concurrency) and performance/resource limit tests
**Complexity**: 5 points
**Dependencies**: AZ-189_test_infrastructure, AZ-190_auth_tests
**Component**: Blackbox Tests
**Tracker**: AZ-195
**Epic**: AZ-188
## Problem
The system's behavior under failure conditions, load, and resource limits is untested.
## Outcome
- API survives database outage and recovers (NFT-RES-01)
- Malformed JWT tokens don't crash the system (NFT-RES-02)
- Concurrent hardware binding doesn't corrupt data (NFT-RES-03)
- Login latency stays under threshold (NFT-PERF-01)
- Resource download latency acceptable (NFT-PERF-02, NFT-PERF-03)
- Max file size boundary enforced (NFT-RES-LIM-01, NFT-RES-LIM-02)
- Memory stays bounded during encryption (NFT-RES-LIM-03)
## Scope
### Included
- Database loss/recovery resilience test
- Malformed JWT handling
- Concurrent hardware binding race condition
- Login endpoint performance under load
- Resource download performance (small + large files)
- Upload file size boundary (200 MB / 201 MB)
- Memory usage during large file encryption
### Excluded
- User list performance under high user count (NFT-PERF-04 — deferred, requires bulk seed data)
## Acceptance Criteria
**AC-1: DB recovery**
Given the API is running
When the database container is stopped and restarted
Then the API returns errors during outage but recovers within 10s of DB restoration
**AC-2: Malformed JWT**
Given various invalid Authorization headers
When protected endpoints are called
Then all return HTTP 401, system remains operational
**AC-3: Concurrent hardware**
Given a user with no hardware
When two concurrent hardware check requests arrive
Then no data corruption; subsequent requests behave consistently
**AC-4: Login latency**
Given 10 concurrent login requests
When 100 total requests complete
Then p95 latency < 500ms
**AC-5: Max upload accepted**
Given a 200 MB file
When POST /resources/testfolder
Then HTTP 200
**AC-6: Over-max upload rejected**
Given a 201 MB file
When POST /resources/testfolder
Then HTTP 413
## Blackbox Tests
| AC Ref | Initial Data/Conditions | What to Test | Expected Behavior | NFR References |
|--------|------------------------|-------------|-------------------|----------------|
| AC-1 | Running system | Stop/restart DB | Error then recovery | — |
| AC-2 | Various bad tokens | Protected endpoints | HTTP 401, no crash | — |
| AC-3 | Unbound user | Concurrent /resources/check | Consistent state | — |
| AC-4 | 10 concurrent users | 100 login requests | p95 < 500ms | — |
| AC-5 | 200 MB file | Upload | HTTP 200 | — |
| AC-6 | 201 MB file | Upload | HTTP 413 | — |
## Risks & Mitigation
**Risk 1: Docker container manipulation in tests**
- *Risk*: Tests need to stop/start the DB container programmatically
- *Mitigation*: Use Docker SDK or shell commands from test fixture to control containers