Refactor testing framework to replace integration tests with blackbox tests across various skills and documentation. Update related workflows, templates, and task specifications to align with the new blackbox testing approach. Remove obsolete integration test files and enhance clarity in task management and reporting structures.

This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-03-24 03:38:36 +02:00
parent ae3ad50b9e
commit e609586c7c
49 changed files with 2222 additions and 872 deletions
@@ -0,0 +1,87 @@
# Resource Limit Tests
### NFT-RES-LIM-01: ThreadPoolExecutor worker limit (2 concurrent)
**Summary**: Verify that no more than 2 inference operations run simultaneously.
**Traces to**: RESTRICT-HW-3
**Preconditions**:
- Engine is initialized
**Monitoring**:
- Track concurrent request timings
**Steps**:
| Step | Consumer Action | Expected Behavior |
|------|----------------|------------------|
| 1 | Send 4 concurrent `POST /detect` requests | — |
| 2 | Measure response arrival times | First 2 complete roughly together; next 2 complete after |
**Duration**: ~60s
**Pass criteria**: Clear evidence of 2-at-a-time processing (second batch starts after first completes). All 4 requests eventually succeed.
---
### NFT-RES-LIM-02: SSE queue depth limit (100 events)
**Summary**: Verify that the SSE queue per client does not exceed 100 events.
**Traces to**: AC-API-4
**Preconditions**:
- Engine is initialized
**Monitoring**:
- SSE event count
**Steps**:
| Step | Consumer Action | Expected Behavior |
|------|----------------|------------------|
| 1 | Open SSE connection but do not read (stall client) | — |
| 2 | Trigger async detection that produces > 100 events | — |
| 3 | After processing completes, drain the SSE queue | ≤ 100 events received |
**Duration**: ~120s
**Pass criteria**: No more than 100 events buffered. No OOM or connection errors from queue growth.
---
### NFT-RES-LIM-03: Max 300 detections per frame
**Summary**: Verify that the system returns at most 300 detections per frame (model output limit).
**Traces to**: RESTRICT-SW-6
**Preconditions**:
- Engine is initialized
- Image with dense scene expected to produce many detections
**Monitoring**:
- Detection count per response
**Duration**: ~30s
**Pass criteria**: No response contains more than 300 detections. Dense images hit the cap without errors.
---
### NFT-RES-LIM-04: Log file rotation and retention
**Summary**: Verify that log files rotate daily and are retained for 30 days.
**Traces to**: AC-LOG-1, AC-LOG-2
**Preconditions**:
- Detections service running with Logs/ volume mounted for inspection
**Monitoring**:
- Log file creation, naming, and count
**Steps**:
| Step | Consumer Action | Expected Behavior |
|------|----------------|------------------|
| 1 | Make several detection requests | Logs written to `Logs/log_inference_YYYYMMDD.txt` |
| 2 | Verify log file name matches current date | File name contains today's date |
| 3 | Verify log content format | Contains INFO/DEBUG/WARNING entries with timestamps |
**Duration**: ~10s
**Pass criteria**: Log file exists with correct date-based naming. Content includes structured log entries.