mirror of
https://github.com/azaion/admin.git
synced 2026-04-22 10:06:33 +00:00
[AZ-189] [AZ-190] [AZ-191] [AZ-192] [AZ-193] [AZ-194] [AZ-195] Add e2e blackbox test suite
Made-with: Cursor
This commit is contained in:
Oleksandr Bezdieniezhnykh
@@ -0,0 +1,475 @@
|
||||
# Blackbox Tests
|
||||
|
||||
## Positive Scenarios
|
||||
|
||||
### FT-P-01: Successful Login
|
||||
|
||||
**Summary**: User with valid credentials receives a JWT token.
|
||||
**Traces to**: AC-1
|
||||
**Category**: Authentication
|
||||
|
||||
**Preconditions**:
|
||||
- Seed user `admin@azaion.com` exists in database
|
||||
|
||||
**Input data**: Valid email/password for seed admin user
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /login with valid email and password | HTTP 200, body contains non-empty `token` string |
|
||||
|
||||
**Expected outcome**: HTTP 200 with JWT token in response body
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-02: Successful User Registration
|
||||
|
||||
**Summary**: ApiAdmin creates a new user account.
|
||||
**Traces to**: AC-5, AC-6, AC-7
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated as ApiAdmin
|
||||
|
||||
**Input data**: `{"email":"newuser@test.com","password":"validpwd1","role":"Operator"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | Login as admin to get JWT | HTTP 200, JWT token |
|
||||
| 2 | POST /users with valid registration data and ApiAdmin JWT | HTTP 200 |
|
||||
|
||||
**Expected outcome**: HTTP 200, user created
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-03: JWT Token Structure Validation
|
||||
|
||||
**Summary**: JWT token contains correct issuer, audience, and lifetime claims.
|
||||
**Traces to**: AC-4
|
||||
**Category**: Authentication
|
||||
|
||||
**Preconditions**:
|
||||
- Valid login completed
|
||||
|
||||
**Input data**: JWT token from login response
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | Login to get JWT | HTTP 200, JWT token |
|
||||
| 2 | Decode JWT payload (Base64) | Claims contain `iss`, `aud`, `exp` |
|
||||
| 3 | Validate `iss` == "AzaionApi" | Match |
|
||||
| 4 | Validate `aud` == "Annotators/OrangePi/Admins" | Match |
|
||||
| 5 | Validate `exp` - `iat` ≈ 14400s (4 hours) | Within ± 60s |
|
||||
|
||||
**Expected outcome**: All JWT claims match expected values
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-04: First Hardware Check Stores Fingerprint
|
||||
|
||||
**Summary**: On first hardware check, the fingerprint is stored for the user.
|
||||
**Traces to**: AC-10
|
||||
**Category**: Hardware Binding
|
||||
|
||||
**Preconditions**:
|
||||
- User exists with no hardware bound
|
||||
|
||||
**Input data**: `{"hardware":"test-hw-fingerprint-001"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | Register new user, login to get JWT | HTTP 200 |
|
||||
| 2 | POST /resources/check with hardware string | HTTP 200, body `true` |
|
||||
|
||||
**Expected outcome**: HTTP 200, hardware stored
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-05: Subsequent Hardware Check Matches
|
||||
|
||||
**Summary**: Same hardware fingerprint passes validation on subsequent calls.
|
||||
**Traces to**: AC-11
|
||||
**Category**: Hardware Binding
|
||||
|
||||
**Preconditions**:
|
||||
- User with hardware already bound (from FT-P-04)
|
||||
|
||||
**Input data**: Same hardware string as initial binding
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /resources/check with same hardware | HTTP 200, body `true` |
|
||||
|
||||
**Expected outcome**: HTTP 200
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-06: List All Users
|
||||
|
||||
**Summary**: ApiAdmin retrieves the user list.
|
||||
**Traces to**: AC-9
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated as ApiAdmin
|
||||
|
||||
**Input data**: GET /users with ApiAdmin JWT
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | GET /users with ApiAdmin JWT | HTTP 200, JSON array with >= 1 user |
|
||||
|
||||
**Expected outcome**: HTTP 200, array containing at least seed users
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-07: Filter Users by Email
|
||||
|
||||
**Summary**: ApiAdmin filters users by email substring.
|
||||
**Traces to**: AC-9
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated as ApiAdmin, seed users exist
|
||||
|
||||
**Input data**: GET /users?email=admin
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | GET /users?email=admin with ApiAdmin JWT | HTTP 200, all returned emails contain "admin" |
|
||||
|
||||
**Expected outcome**: HTTP 200, filtered list
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-08: Upload Resource File
|
||||
|
||||
**Summary**: Authenticated user uploads a file to a resource folder.
|
||||
**Traces to**: AC-13
|
||||
**Category**: Resource Distribution
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated
|
||||
|
||||
**Input data**: Multipart form upload with 1 KB text file
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /resources/testfolder with multipart file | HTTP 200 |
|
||||
|
||||
**Expected outcome**: HTTP 200, file stored
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-09: Download Encrypted Resource
|
||||
|
||||
**Summary**: Authenticated user downloads an encrypted resource file.
|
||||
**Traces to**: AC-14, AC-18
|
||||
**Category**: Resource Distribution
|
||||
|
||||
**Preconditions**:
|
||||
- User authenticated, hardware bound, resource file uploaded
|
||||
|
||||
**Input data**: `{"password":"validpwd1","hardware":"test-hw-001","fileName":"test.txt"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /resources/get with credentials | HTTP 200, Content-Type: application/octet-stream, non-empty body |
|
||||
|
||||
**Expected outcome**: HTTP 200 with encrypted binary content
|
||||
**Max execution time**: 10s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-10: Encryption Round-Trip Verification
|
||||
|
||||
**Summary**: Downloaded encrypted resource decrypts to original file content.
|
||||
**Traces to**: AC-15, AC-19
|
||||
**Category**: Resource Distribution
|
||||
|
||||
**Preconditions**:
|
||||
- Known file uploaded, user credentials known
|
||||
|
||||
**Input data**: Original file content, user email, password, hardware hash
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | Upload known file | HTTP 200 |
|
||||
| 2 | Download encrypted file via API | HTTP 200, encrypted bytes |
|
||||
| 3 | Derive AES key from email + password + hwHash | Key bytes |
|
||||
| 4 | Decrypt downloaded content with derived key | Decrypted bytes |
|
||||
| 5 | Compare decrypted bytes with original | Byte-level equality |
|
||||
|
||||
**Expected outcome**: Decrypted content matches original file exactly
|
||||
**Max execution time**: 10s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-11: Change User Role
|
||||
|
||||
**Summary**: ApiAdmin changes a user's role.
|
||||
**Traces to**: AC-9
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Target user exists, caller is ApiAdmin
|
||||
|
||||
**Input data**: `{"email":"testuser@test.com","role":"Admin"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | PUT /users/role with ApiAdmin JWT | HTTP 200 |
|
||||
|
||||
**Expected outcome**: HTTP 200, role updated
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-12: Disable User Account
|
||||
|
||||
**Summary**: ApiAdmin disables a user account.
|
||||
**Traces to**: AC-9
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Target user exists, caller is ApiAdmin
|
||||
|
||||
**Input data**: `{"email":"testuser@test.com","isEnabled":false}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | PUT /users/enable with ApiAdmin JWT | HTTP 200 |
|
||||
|
||||
**Expected outcome**: HTTP 200, account disabled
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-P-13: Delete User
|
||||
|
||||
**Summary**: ApiAdmin deletes a user account.
|
||||
**Traces to**: AC-9
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Target user exists, caller is ApiAdmin
|
||||
|
||||
**Input data**: DELETE /users?email=testuser@test.com
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | DELETE /users?email=testuser@test.com with ApiAdmin JWT | HTTP 200 |
|
||||
|
||||
**Expected outcome**: HTTP 200, user deleted
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
## Negative Scenarios
|
||||
|
||||
### FT-N-01: Login with Unknown Email
|
||||
|
||||
**Summary**: Login attempt with non-existent email returns appropriate error.
|
||||
**Traces to**: AC-2
|
||||
**Category**: Authentication
|
||||
|
||||
**Preconditions**:
|
||||
- Email does not exist in database
|
||||
|
||||
**Input data**: `{"email":"nonexistent@test.com","password":"anypass1"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /login with unknown email | HTTP 409, ExceptionEnum code 10 (NoEmailFound) |
|
||||
|
||||
**Expected outcome**: HTTP 409 with error code 10
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-N-02: Login with Wrong Password
|
||||
|
||||
**Summary**: Login attempt with correct email but wrong password returns error.
|
||||
**Traces to**: AC-3
|
||||
**Category**: Authentication
|
||||
|
||||
**Preconditions**:
|
||||
- User exists in database
|
||||
|
||||
**Input data**: `{"email":"admin@azaion.com","password":"wrongpassword123"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /login with wrong password | HTTP 409, ExceptionEnum code 30 (WrongPassword) |
|
||||
|
||||
**Expected outcome**: HTTP 409 with error code 30
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-N-03: Register with Short Email
|
||||
|
||||
**Summary**: Registration with email shorter than 8 characters is rejected.
|
||||
**Traces to**: AC-5
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated as ApiAdmin
|
||||
|
||||
**Input data**: `{"email":"short","password":"validpwd1","role":"Operator"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /users with short email | HTTP 400, validation error |
|
||||
|
||||
**Expected outcome**: HTTP 400 with email length validation error
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-N-04: Register with Invalid Email Format
|
||||
|
||||
**Summary**: Registration with invalid email format (>= 8 chars but not email) is rejected.
|
||||
**Traces to**: AC-6
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated as ApiAdmin
|
||||
|
||||
**Input data**: `{"email":"notanemail","password":"validpwd1","role":"Operator"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /users with invalid email format | HTTP 400, validation error |
|
||||
|
||||
**Expected outcome**: HTTP 400 with email format validation error
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-N-05: Upload Empty File
|
||||
|
||||
**Summary**: Upload request with no file attached returns error.
|
||||
**Traces to**: AC-16
|
||||
**Category**: Resource Distribution
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated
|
||||
|
||||
**Input data**: POST /resources/testfolder with no file
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /resources/testfolder with empty request | HTTP 409, ExceptionEnum code 70 (NoFileProvided) |
|
||||
|
||||
**Expected outcome**: HTTP 409 with error code 70
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-N-06: Hardware Mismatch
|
||||
|
||||
**Summary**: Hardware check with different fingerprint after binding returns error.
|
||||
**Traces to**: AC-12
|
||||
**Category**: Hardware Binding
|
||||
|
||||
**Preconditions**:
|
||||
- User has hardware already bound to a different fingerprint
|
||||
|
||||
**Input data**: `{"hardware":"different-hardware-xyz"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /resources/check with different hardware | HTTP 409, ExceptionEnum code 40 (HardwareIdMismatch) |
|
||||
|
||||
**Expected outcome**: HTTP 409 with error code 40
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-N-07: Register Duplicate Email
|
||||
|
||||
**Summary**: Registration with already-existing email returns error.
|
||||
**Traces to**: AC-8
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- User with target email already exists
|
||||
|
||||
**Input data**: `{"email":"admin@azaion.com","password":"validpwd1","role":"Operator"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /users with existing email | HTTP 409, ExceptionEnum code 20 (EmailExists) |
|
||||
|
||||
**Expected outcome**: HTTP 409 with error code 20
|
||||
**Max execution time**: 5s
|
||||
|
||||
---
|
||||
|
||||
### FT-N-08: Register with Short Password
|
||||
|
||||
**Summary**: Registration with password shorter than 8 characters is rejected.
|
||||
**Traces to**: AC-7
|
||||
**Category**: User Management
|
||||
|
||||
**Preconditions**:
|
||||
- Caller authenticated as ApiAdmin
|
||||
|
||||
**Input data**: `{"email":"newuser@test.com","password":"short","role":"Operator"}`
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected System Response |
|
||||
|------|----------------|------------------------|
|
||||
| 1 | POST /users with short password | HTTP 400, validation error |
|
||||
|
||||
**Expected outcome**: HTTP 400 with password length validation error
|
||||
**Max execution time**: 5s
|
||||
@@ -0,0 +1,103 @@
|
||||
# Test Environment
|
||||
|
||||
## Overview
|
||||
|
||||
**System under test**: Azaion Admin API — ASP.NET Core Minimal API at `http://localhost:8080`
|
||||
**Consumer app purpose**: Standalone xUnit test project exercising the API through HTTP requests, validating black-box use cases without access to internals.
|
||||
|
||||
## Docker Environment
|
||||
|
||||
### Services
|
||||
|
||||
| Service | Image / Build | Purpose | Ports |
|
||||
|---------|--------------|---------|-------|
|
||||
| system-under-test | Build from `Dockerfile` | Azaion Admin API | 8080:8080 |
|
||||
| test-db | `postgres:16-alpine` | PostgreSQL database | 4312:5432 |
|
||||
| e2e-consumer | Build from `docker.test/Dockerfile` | Black-box test runner (xUnit) | — |
|
||||
|
||||
### Networks
|
||||
|
||||
| Network | Services | Purpose |
|
||||
|---------|----------|---------|
|
||||
| e2e-net | all | Isolated test network |
|
||||
|
||||
### Volumes
|
||||
|
||||
| Volume | Mounted to | Purpose |
|
||||
|--------|-----------|---------|
|
||||
| test-resources | system-under-test:/app/Content | Resource files for upload/download tests |
|
||||
| db-init | test-db:/docker-entrypoint-initdb.d | Schema DDL + seed data |
|
||||
|
||||
### docker-compose structure
|
||||
|
||||
```yaml
|
||||
services:
|
||||
test-db:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_DB: azaion
|
||||
POSTGRES_USER: azaion_superadmin
|
||||
POSTGRES_PASSWORD: test_password
|
||||
volumes:
|
||||
- ./env/db/01_permissions.sql:/docker-entrypoint-initdb.d/01.sql
|
||||
- ./env/db/02_structure.sql:/docker-entrypoint-initdb.d/02.sql
|
||||
- ./env/db/03_add_timestamp_columns.sql:/docker-entrypoint-initdb.d/03.sql
|
||||
networks:
|
||||
- e2e-net
|
||||
|
||||
system-under-test:
|
||||
build: .
|
||||
environment:
|
||||
ASPNETCORE_ConnectionStrings__AzaionDb: "Host=test-db;Port=5432;Database=azaion;Username=azaion_reader;Password=test_password"
|
||||
ASPNETCORE_ConnectionStrings__AzaionDbAdmin: "Host=test-db;Port=5432;Database=azaion;Username=azaion_admin;Password=test_password"
|
||||
ASPNETCORE_JwtConfig__Secret: "test-jwt-secret-key-at-least-32-chars-long"
|
||||
ASPNETCORE_ENVIRONMENT: Development
|
||||
depends_on:
|
||||
- test-db
|
||||
networks:
|
||||
- e2e-net
|
||||
|
||||
e2e-consumer:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: docker.test/Dockerfile
|
||||
environment:
|
||||
API_BASE_URL: "http://system-under-test:8080"
|
||||
depends_on:
|
||||
- system-under-test
|
||||
networks:
|
||||
- e2e-net
|
||||
|
||||
networks:
|
||||
e2e-net:
|
||||
```
|
||||
|
||||
## Consumer Application
|
||||
|
||||
**Tech stack**: C# / .NET 10.0, xUnit 2.9.2, FluentAssertions 6.12.2, HttpClient
|
||||
**Entry point**: `dotnet test` in the e2e consumer project
|
||||
|
||||
### Communication with system under test
|
||||
|
||||
| Interface | Protocol | Endpoint | Authentication |
|
||||
|-----------|----------|----------|----------------|
|
||||
| REST API | HTTP | `http://system-under-test:8080/*` | JWT Bearer token |
|
||||
|
||||
### What the consumer does NOT have access to
|
||||
|
||||
- No direct database access to the test-db (queries go through the API only)
|
||||
- No internal module imports from Azaion.Services or Azaion.Common
|
||||
- No shared filesystem with the system-under-test (except via API upload/download)
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
**When to run**: On every push to `dev`, `main`, and `stage` branches
|
||||
**Pipeline stage**: After build, before deploy
|
||||
**Gate behavior**: Block merge on failure
|
||||
**Timeout**: 5 minutes
|
||||
|
||||
## Reporting
|
||||
|
||||
**Format**: CSV
|
||||
**Columns**: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message (if FAIL)
|
||||
**Output path**: `./e2e-results/report.csv`
|
||||
@@ -0,0 +1,83 @@
|
||||
# Performance Tests
|
||||
|
||||
### NFT-PERF-01: Login Endpoint Latency
|
||||
|
||||
**Summary**: Login endpoint responds within acceptable latency under normal load.
|
||||
**Traces to**: AC-1
|
||||
**Metric**: Response time (p95)
|
||||
|
||||
**Preconditions**:
|
||||
- System running with seed data
|
||||
- 10 concurrent users
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Measurement |
|
||||
|------|----------------|-------------|
|
||||
| 1 | Send 100 login requests (10 concurrent) | Measure p50, p95, p99 response times |
|
||||
|
||||
**Pass criteria**: p95 latency < 500ms
|
||||
**Duration**: 30 seconds
|
||||
|
||||
---
|
||||
|
||||
### NFT-PERF-02: Resource Download Latency (Small File)
|
||||
|
||||
**Summary**: Encrypted resource download for a small file (1 KB) completes quickly.
|
||||
**Traces to**: AC-14
|
||||
**Metric**: Response time including encryption
|
||||
|
||||
**Preconditions**:
|
||||
- 1 KB test file uploaded
|
||||
- User authenticated with bound hardware
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Measurement |
|
||||
|------|----------------|-------------|
|
||||
| 1 | Send 50 encrypted download requests (5 concurrent) | Measure p50, p95 response times |
|
||||
|
||||
**Pass criteria**: p95 latency < 1000ms
|
||||
**Duration**: 30 seconds
|
||||
|
||||
---
|
||||
|
||||
### NFT-PERF-03: Resource Download Latency (Large File)
|
||||
|
||||
**Summary**: Encrypted resource download for a larger file (50 MB) completes within limits.
|
||||
**Traces to**: AC-13, AC-14
|
||||
**Metric**: Response time including encryption + transfer
|
||||
|
||||
**Preconditions**:
|
||||
- 50 MB test file uploaded
|
||||
- User authenticated with bound hardware
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Measurement |
|
||||
|------|----------------|-------------|
|
||||
| 1 | Send 5 sequential encrypted download requests | Measure p50, p95 response times |
|
||||
|
||||
**Pass criteria**: p95 latency < 30000ms (30s)
|
||||
**Duration**: 3 minutes
|
||||
|
||||
---
|
||||
|
||||
### NFT-PERF-04: User List Endpoint Under Load
|
||||
|
||||
**Summary**: User list endpoint responds within limits when DB has many users.
|
||||
**Traces to**: AC-9
|
||||
**Metric**: Response time
|
||||
|
||||
**Preconditions**:
|
||||
- 500 users in database
|
||||
- Caller is ApiAdmin
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Measurement |
|
||||
|------|----------------|-------------|
|
||||
| 1 | Send 50 GET /users requests (10 concurrent) | Measure p50, p95 response times |
|
||||
|
||||
**Pass criteria**: p95 latency < 1000ms
|
||||
**Duration**: 30 seconds
|
||||
@@ -0,0 +1,71 @@
|
||||
# Resilience Tests
|
||||
|
||||
### NFT-RES-01: Database Connection Loss Recovery
|
||||
|
||||
**Summary**: API returns appropriate errors when database is unavailable, and recovers when it comes back.
|
||||
**Traces to**: AC-1, AC-9
|
||||
|
||||
**Preconditions**:
|
||||
- System running normally with database connected
|
||||
|
||||
**Fault injection**:
|
||||
- Stop the PostgreSQL container
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Behavior |
|
||||
|------|--------|------------------|
|
||||
| 1 | Stop test-db container | Database unavailable |
|
||||
| 2 | Send POST /login request | HTTP 500 (database error, not crash) |
|
||||
| 3 | Verify API process is still running | Process alive, accepting connections |
|
||||
| 4 | Restart test-db container | Database available |
|
||||
| 5 | Wait 5 seconds for connection recovery | — |
|
||||
| 6 | Send POST /login request | HTTP 200 or HTTP 409 (normal behavior) |
|
||||
|
||||
**Pass criteria**: API does not crash on DB loss; recovers within 10s of DB restoration
|
||||
|
||||
---
|
||||
|
||||
### NFT-RES-02: Invalid JWT Token Handling
|
||||
|
||||
**Summary**: API rejects malformed JWT tokens gracefully without crashing.
|
||||
**Traces to**: AC-18
|
||||
|
||||
**Preconditions**:
|
||||
- System running normally
|
||||
|
||||
**Fault injection**:
|
||||
- Send requests with malformed Authorization headers
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Behavior |
|
||||
|------|--------|------------------|
|
||||
| 1 | Send GET /users with `Authorization: Bearer invalid-token` | HTTP 401 |
|
||||
| 2 | Send GET /users with `Authorization: Bearer ` (empty) | HTTP 401 |
|
||||
| 3 | Send GET /users with `Authorization: NotBearer token` | HTTP 401 |
|
||||
| 4 | Send normal login request | HTTP 200 (system unaffected) |
|
||||
|
||||
**Pass criteria**: All malformed tokens return HTTP 401; system remains operational
|
||||
|
||||
---
|
||||
|
||||
### NFT-RES-03: Concurrent Hardware Binding Attempt
|
||||
|
||||
**Summary**: Two simultaneous hardware check requests for the same user do not corrupt data.
|
||||
**Traces to**: AC-10, AC-11
|
||||
|
||||
**Preconditions**:
|
||||
- User with no hardware bound
|
||||
|
||||
**Fault injection**:
|
||||
- Race condition: two concurrent POST /resources/check with same hardware
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Action | Expected Behavior |
|
||||
|------|--------|------------------|
|
||||
| 1 | Send two POST /resources/check simultaneously with same hardware | Both return HTTP 200 or one returns 200 and other returns 409 |
|
||||
| 2 | Send a third POST /resources/check with same hardware | HTTP 200 (consistent state) |
|
||||
|
||||
**Pass criteria**: No database corruption; subsequent requests behave consistently
|
||||
@@ -0,0 +1,69 @@
|
||||
# Resource Limit Tests
|
||||
|
||||
### NFT-RES-LIM-01: Maximum Upload File Size (200 MB)
|
||||
|
||||
**Summary**: System accepts file uploads at the configured maximum size (200 MB).
|
||||
**Traces to**: AC-13
|
||||
|
||||
**Preconditions**:
|
||||
- System running with default Kestrel config (MaxRequestBodySize = 200 MB)
|
||||
- Caller authenticated
|
||||
|
||||
**Monitoring**:
|
||||
- API container memory usage
|
||||
- Response status code
|
||||
|
||||
**Duration**: Single request
|
||||
**Pass criteria**: HTTP 200 for 200 MB file; API memory stays below 1 GB during upload
|
||||
|
||||
---
|
||||
|
||||
### NFT-RES-LIM-02: Over-Maximum Upload File Size (201 MB)
|
||||
|
||||
**Summary**: System rejects file uploads exceeding the configured maximum size.
|
||||
**Traces to**: AC-13
|
||||
|
||||
**Preconditions**:
|
||||
- System running with default Kestrel config
|
||||
- Caller authenticated
|
||||
|
||||
**Monitoring**:
|
||||
- Response status code
|
||||
|
||||
**Duration**: Single request
|
||||
**Pass criteria**: HTTP 413 (Request Entity Too Large) for 201 MB file
|
||||
|
||||
---
|
||||
|
||||
### NFT-RES-LIM-03: Memory Usage During Large File Encryption
|
||||
|
||||
**Summary**: Memory usage during encrypted resource download stays within acceptable bounds.
|
||||
**Traces to**: AC-14
|
||||
|
||||
**Preconditions**:
|
||||
- 100 MB test file uploaded
|
||||
- User authenticated with bound hardware
|
||||
|
||||
**Monitoring**:
|
||||
- API container memory usage (docker stats)
|
||||
- Response time
|
||||
|
||||
**Duration**: Single download request
|
||||
**Pass criteria**: API container memory peak < 500 MB; request completes within 60s
|
||||
|
||||
---
|
||||
|
||||
### NFT-RES-LIM-04: Concurrent User Connections
|
||||
|
||||
**Summary**: System handles multiple simultaneous authenticated requests without errors.
|
||||
**Traces to**: AC-1, AC-18
|
||||
|
||||
**Preconditions**:
|
||||
- 20 unique users in database
|
||||
|
||||
**Monitoring**:
|
||||
- Response status codes
|
||||
- Error rate
|
||||
|
||||
**Duration**: 60 seconds
|
||||
**Pass criteria**: 20 concurrent login requests complete with 0% error rate (all HTTP 200)
|
||||
@@ -0,0 +1,104 @@
|
||||
# Security Tests
|
||||
|
||||
### NFT-SEC-01: Unauthenticated Access to Protected Endpoints
|
||||
|
||||
**Summary**: All protected endpoints reject requests without JWT token.
|
||||
**Traces to**: AC-18
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected Response |
|
||||
|------|----------------|------------------|
|
||||
| 1 | GET /users (no JWT) | HTTP 401 |
|
||||
| 2 | POST /resources/get (no JWT) | HTTP 401 |
|
||||
| 3 | POST /resources/check (no JWT) | HTTP 401 |
|
||||
| 4 | GET /resources/get-installer (no JWT) | HTTP 401 |
|
||||
| 5 | PUT /users/role (no JWT) | HTTP 401 |
|
||||
| 6 | DELETE /users (no JWT) | HTTP 401 |
|
||||
|
||||
**Pass criteria**: All endpoints return HTTP 401 for unauthenticated requests
|
||||
|
||||
---
|
||||
|
||||
### NFT-SEC-02: Non-Admin Access to Admin Endpoints
|
||||
|
||||
**Summary**: Non-ApiAdmin users cannot access admin-only endpoints.
|
||||
**Traces to**: AC-9
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected Response |
|
||||
|------|----------------|------------------|
|
||||
| 1 | Login as Operator role user | HTTP 200, JWT token |
|
||||
| 2 | POST /users (register) with Operator JWT | HTTP 403 |
|
||||
| 3 | PUT /users/role with Operator JWT | HTTP 403 |
|
||||
| 4 | PUT /users/enable with Operator JWT | HTTP 403 |
|
||||
| 5 | DELETE /users with Operator JWT | HTTP 403 |
|
||||
|
||||
**Pass criteria**: All admin endpoints return HTTP 403 for non-admin users
|
||||
|
||||
---
|
||||
|
||||
### NFT-SEC-03: Password Not Returned in User List
|
||||
|
||||
**Summary**: User list endpoint does not expose password hashes.
|
||||
**Traces to**: AC-17
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected Response |
|
||||
|------|----------------|------------------|
|
||||
| 1 | GET /users with ApiAdmin JWT | HTTP 200, JSON array |
|
||||
| 2 | Inspect each user object in response | No `passwordHash` or `password` field present |
|
||||
|
||||
**Pass criteria**: Password hash is never included in API responses
|
||||
|
||||
---
|
||||
|
||||
### NFT-SEC-04: Expired JWT Token Rejection
|
||||
|
||||
**Summary**: Expired JWT tokens are rejected.
|
||||
**Traces to**: AC-4, AC-18
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected Response |
|
||||
|------|----------------|------------------|
|
||||
| 1 | Craft a JWT with `exp` set to past timestamp (same signing key) | Token string |
|
||||
| 2 | GET /users with expired JWT | HTTP 401 |
|
||||
|
||||
**Pass criteria**: Expired token returns HTTP 401
|
||||
|
||||
---
|
||||
|
||||
### NFT-SEC-05: Encryption Key Uniqueness
|
||||
|
||||
**Summary**: Different users produce different encryption keys for the same resource.
|
||||
**Traces to**: AC-19
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected Response |
|
||||
|------|----------------|------------------|
|
||||
| 1 | Upload test file | HTTP 200 |
|
||||
| 2 | Download encrypted file as User A | Encrypted bytes A |
|
||||
| 3 | Download same file as User B (different credentials + hardware) | Encrypted bytes B |
|
||||
| 4 | Compare encrypted bytes A and B | Different |
|
||||
|
||||
**Pass criteria**: Encrypted outputs differ between users
|
||||
|
||||
---
|
||||
|
||||
### NFT-SEC-06: Disabled User Cannot Login
|
||||
|
||||
**Summary**: A disabled user account cannot authenticate.
|
||||
**Traces to**: AC-9
|
||||
|
||||
**Steps**:
|
||||
|
||||
| Step | Consumer Action | Expected Response |
|
||||
|------|----------------|------------------|
|
||||
| 1 | Register user, disable via PUT /users/enable | HTTP 200 |
|
||||
| 2 | Attempt POST /login with disabled user credentials | HTTP 409 or HTTP 403 |
|
||||
|
||||
**Pass criteria**: Disabled user cannot obtain a JWT token
|
||||
@@ -0,0 +1,62 @@
|
||||
# Test Data Management
|
||||
|
||||
## Seed Data Sets
|
||||
|
||||
| Data Set | Description | Used by Tests | How Loaded | Cleanup |
|
||||
|----------|-------------|---------------|-----------|---------|
|
||||
| seed-users | Default admin + uploader users from DDL | All tests requiring auth | SQL init scripts in docker-entrypoint-initdb.d | Fresh container per test run |
|
||||
| test-resource-file | Small text file (1 KB) for upload/download tests | FT-P-08, FT-P-09, FT-P-10, FT-N-05 | Created by test via upload API | Deleted via ClearFolder API |
|
||||
| large-resource-file | 200 MB test file for boundary testing | NFT-RES-LIM-01, NFT-RES-LIM-02 | Generated at test start | Deleted after test |
|
||||
|
||||
## Data Isolation Strategy
|
||||
|
||||
Each test run starts with a fresh Docker Compose environment. The database is initialized from scratch using the SQL scripts in `env/db/`. Tests that create users clean them up via the DELETE API. Tests that upload files clean up via the ClearFolder endpoint.
|
||||
|
||||
## Input Data Mapping
|
||||
|
||||
| Input Data File | Source Location | Description | Covers Scenarios |
|
||||
|-----------------|----------------|-------------|-----------------|
|
||||
| `data_parameters.md` | `_docs/00_problem/input_data/` | DB schema, API request types, config | Reference for all tests |
|
||||
| `results_report.md` | `_docs/00_problem/input_data/expected_results/` | Input→expected result mapping | All tests |
|
||||
|
||||
## Expected Results Mapping
|
||||
|
||||
| Test Scenario ID | Input Data | Expected Result | Comparison Method | Tolerance | Expected Result Source |
|
||||
|-----------------|------------|-----------------|-------------------|-----------|----------------------|
|
||||
| FT-P-01 | Valid login request | HTTP 200, JWT token in body | exact (status), substring (token) | N/A | results_report.md #1 |
|
||||
| FT-P-02 | Valid registration request | HTTP 200 | exact | N/A | results_report.md #5 |
|
||||
| FT-P-03 | JWT token decode | iss, aud, exp claims | exact, numeric_tolerance | exp: ± 60s | results_report.md #4 |
|
||||
| FT-P-04 | First hardware check | HTTP 200, true | exact | N/A | results_report.md #11 |
|
||||
| FT-P-05 | Same hardware second check | HTTP 200, true | exact | N/A | results_report.md #12 |
|
||||
| FT-P-06 | List users with admin JWT | HTTP 200, JSON array >= 1 | exact, threshold_min | N/A | results_report.md #19 |
|
||||
| FT-P-07 | Filter users by email | HTTP 200, emails contain substring | exact, substring | N/A | results_report.md #20 |
|
||||
| FT-P-08 | Upload resource file | HTTP 200 | exact | N/A | results_report.md #17 |
|
||||
| FT-P-09 | Download encrypted resource | HTTP 200, octet-stream, non-empty | exact, threshold_min | N/A | results_report.md #14 |
|
||||
| FT-P-10 | Decrypt downloaded resource | Byte-equals original | exact | N/A | results_report.md #15 |
|
||||
| FT-P-11 | Change user role | HTTP 200 | exact | N/A | results_report.md #21 |
|
||||
| FT-P-12 | Disable user | HTTP 200 | exact | N/A | results_report.md #22 |
|
||||
| FT-P-13 | Delete user | HTTP 200 | exact | N/A | results_report.md #23 |
|
||||
| FT-N-01 | Login unknown email | HTTP 409, code 10 | exact | N/A | results_report.md #2 |
|
||||
| FT-N-02 | Login wrong password | HTTP 409, code 30 | exact | N/A | results_report.md #3 |
|
||||
| FT-N-03 | Register short email | HTTP 400 | exact | N/A | results_report.md #6 |
|
||||
| FT-N-04 | Register invalid email | HTTP 400 | exact | N/A | results_report.md #7 |
|
||||
| FT-N-05 | Upload empty file | HTTP 409, code 70 | exact | N/A | results_report.md #18 |
|
||||
| FT-N-06 | Hardware mismatch | HTTP 409, code 40 | exact | N/A | results_report.md #13 |
|
||||
| FT-N-07 | Register duplicate email | HTTP 409, code 20 | exact | N/A | results_report.md #9 |
|
||||
| FT-N-08 | Register short password | HTTP 400 | exact | N/A | results_report.md #8 |
|
||||
|
||||
## External Dependency Mocks
|
||||
|
||||
| External Service | Mock/Stub | How Provided | Behavior |
|
||||
|-----------------|-----------|-------------|----------|
|
||||
| PostgreSQL | Real instance | Docker container (postgres:16-alpine) | Production-equivalent behavior |
|
||||
| Filesystem | Docker volume | Mounted volume on system-under-test | Real file I/O |
|
||||
|
||||
## Data Validation Rules
|
||||
|
||||
| Data Type | Validation | Invalid Examples | Expected System Behavior |
|
||||
|-----------|-----------|-----------------|------------------------|
|
||||
| Email | >= 8 chars, valid format | `"short"`, `"notanemail"` | HTTP 400 validation error |
|
||||
| Password | >= 8 chars | `"short"` | HTTP 400 validation error |
|
||||
| Hardware string | Not empty | `""` | HTTP 400 validation error |
|
||||
| File upload | Non-null, <= 200 MB | null, 201 MB file | HTTP 409 (code 70), HTTP 413 |
|
||||
@@ -0,0 +1,54 @@
|
||||
# Traceability Matrix
|
||||
|
||||
## Acceptance Criteria Coverage
|
||||
|
||||
| AC ID | Acceptance Criterion | Test IDs | Coverage |
|
||||
|-------|---------------------|----------|----------|
|
||||
| AC-1 | Valid login returns JWT | FT-P-01, NFT-PERF-01, NFT-RES-01, NFT-RES-LIM-04 | Covered |
|
||||
| AC-2 | Unknown email returns code 10 | FT-N-01 | Covered |
|
||||
| AC-3 | Wrong password returns code 30 | FT-N-02 | Covered |
|
||||
| AC-4 | JWT lifetime 4 hours | FT-P-03, NFT-SEC-04 | Covered |
|
||||
| AC-5 | Email min 8 chars | FT-P-02, FT-N-03 | Covered |
|
||||
| AC-6 | Email format validation | FT-P-02, FT-N-04 | Covered |
|
||||
| AC-7 | Password min 8 chars | FT-P-02, FT-N-08 | Covered |
|
||||
| AC-8 | Duplicate email returns code 20 | FT-N-07 | Covered |
|
||||
| AC-9 | Only ApiAdmin can manage users | FT-P-06, FT-P-07, FT-P-11, FT-P-12, FT-P-13, NFT-SEC-02, NFT-SEC-06 | Covered |
|
||||
| AC-10 | First hardware check stores | FT-P-04, NFT-RES-03 | Covered |
|
||||
| AC-11 | Subsequent hardware check validates | FT-P-05, NFT-RES-03 | Covered |
|
||||
| AC-12 | Hardware mismatch returns code 40 | FT-N-06 | Covered |
|
||||
| AC-13 | Max upload 200 MB | FT-P-08, NFT-RES-LIM-01, NFT-RES-LIM-02, NFT-PERF-03 | Covered |
|
||||
| AC-14 | AES-256-CBC encryption | FT-P-09, FT-P-10, NFT-PERF-02, NFT-PERF-03, NFT-RES-LIM-03 | Covered |
|
||||
| AC-15 | Encrypt-decrypt round-trip | FT-P-10 | Covered |
|
||||
| AC-16 | Empty file upload returns code 70 | FT-N-05 | Covered |
|
||||
| AC-17 | SHA-384 password hashing | NFT-SEC-03 | Covered |
|
||||
| AC-18 | All non-login endpoints require auth | FT-P-09, NFT-SEC-01, NFT-RES-02, NFT-RES-LIM-04 | Covered |
|
||||
| AC-19 | Encryption key derived from email+password+hw | FT-P-10, NFT-SEC-05 | Covered |
|
||||
|
||||
## Restrictions Coverage
|
||||
|
||||
| Restriction ID | Restriction | Test IDs | Coverage |
|
||||
|---------------|-------------|----------|----------|
|
||||
| RESTRICT-SW-01 | .NET 10.0 runtime | All tests (implicit — Docker build uses .NET 10.0) | Covered |
|
||||
| RESTRICT-SW-02 | PostgreSQL database | All DB tests (implicit — docker-compose uses PostgreSQL) | Covered |
|
||||
| RESTRICT-SW-03 | Max request body 200 MB | NFT-RES-LIM-01, NFT-RES-LIM-02 | Covered |
|
||||
| RESTRICT-SW-04 | JWT HMAC-SHA256 signing | FT-P-03, NFT-SEC-04 | Covered |
|
||||
| RESTRICT-HW-01 | ARM64 target architecture | — | NOT COVERED — CI builds ARM64; tests run on dev x64 host |
|
||||
| RESTRICT-ENV-01 | Secrets via env vars | All tests (implicit — docker-compose passes env vars) | Covered |
|
||||
| RESTRICT-ENV-02 | CORS admin.azaion.com | — | NOT COVERED — CORS is browser-enforced, not testable at API level |
|
||||
| RESTRICT-OP-01 | Serilog logging | — | NOT COVERED — log output verification not in scope |
|
||||
|
||||
## Coverage Summary
|
||||
|
||||
| Category | Total Items | Covered | Not Covered | Coverage % |
|
||||
|----------|-----------|---------|-------------|-----------|
|
||||
| Acceptance Criteria | 19 | 19 | 0 | 100% |
|
||||
| Restrictions | 8 | 5 | 3 | 63% |
|
||||
| **Total** | **27** | **24** | **3** | **89%** |
|
||||
|
||||
## Uncovered Items Analysis
|
||||
|
||||
| Item | Reason Not Covered | Risk | Mitigation |
|
||||
|------|-------------------|------|-----------|
|
||||
| RESTRICT-HW-01 (ARM64) | Tests run on x64 dev/CI host; cross-architecture testing requires ARM hardware | Low — .NET runtime handles arch differences; no arch-specific code in application | CI builds ARM64 image; manual smoke test on target device |
|
||||
| RESTRICT-ENV-02 (CORS) | CORS is enforced by browsers, not by server-to-server HTTP calls | Low — CORS policy is declarative in Program.cs | Visual inspection of CORS configuration in code |
|
||||
| RESTRICT-OP-01 (Logging) | Log output format/content verification adds complexity without proportional value | Low — Serilog configuration is declarative | Code review of Serilog setup |
|
||||
Reference in New Issue
Block a user