mirror of
https://github.com/azaion/detections.git
synced 2026-04-22 22:26:33 +00:00
Oleksandr Bezdieniezhnykh
95 lines
3.1 KiB
Markdown
95 lines
3.1 KiB
Markdown
# Resilience Tests
|
|
|
|
### NFT-RES-01: Loader service outage after engine initialization
|
|
|
|
**Summary**: Verify that detections continue working when the Loader service goes down after the engine is already loaded.
|
|
**Traces to**: RESTRICT-ENV-1
|
|
|
|
**Preconditions**:
|
|
- Engine is initialized (model already downloaded)
|
|
|
|
**Fault injection**:
|
|
- Stop mock-loader service
|
|
|
|
**Steps**:
|
|
|
|
| Step | Action | Expected Behavior |
|
|
|------|--------|------------------|
|
|
| 1 | Stop mock-loader | — |
|
|
| 2 | `POST /detect` with small-image | 200 OK — detection succeeds (engine already in memory) |
|
|
| 3 | `GET /health` | `aiAvailability` remains "Enabled" |
|
|
|
|
**Pass criteria**: Detection continues to work. Health status remains stable. No errors from loader unavailability.
|
|
|
|
---
|
|
|
|
### NFT-RES-02: Annotations service outage during async detection
|
|
|
|
**Summary**: Verify that async detection completes and delivers SSE events even when Annotations service is down.
|
|
**Traces to**: RESTRICT-ENV-2
|
|
|
|
**Preconditions**:
|
|
- Engine is initialized
|
|
- SSE client connected
|
|
|
|
**Fault injection**:
|
|
- Stop mock-annotations mid-processing
|
|
|
|
**Steps**:
|
|
|
|
| Step | Action | Expected Behavior |
|
|
|------|--------|------------------|
|
|
| 1 | Start async detection: `POST /detect/test-media-res01` | `{"status": "started"}` |
|
|
| 2 | After first few SSE events, stop mock-annotations | — |
|
|
| 3 | Continue listening to SSE | Events continue arriving. Annotations POST failures are silently caught |
|
|
| 4 | Wait for completion | Final `AIProcessed` event received |
|
|
|
|
**Pass criteria**: Detection pipeline completes fully. SSE delivery is unaffected. No crash or 500 errors.
|
|
|
|
---
|
|
|
|
### NFT-RES-03: Engine initialization retry after transient loader failure
|
|
|
|
**Summary**: Verify that if model download fails on first attempt, a subsequent detection request retries initialization.
|
|
**Traces to**: AC-EL-2
|
|
|
|
**Preconditions**:
|
|
- Fresh service (engine not initialized)
|
|
|
|
**Fault injection**:
|
|
- Mock-loader returns 503 on first model request, then recovers
|
|
|
|
**Steps**:
|
|
|
|
| Step | Action | Expected Behavior |
|
|
|------|--------|------------------|
|
|
| 1 | Configure mock-loader to fail first request | — |
|
|
| 2 | `POST /detect` with small-image | Error (503 or 422) |
|
|
| 3 | Configure mock-loader to succeed | — |
|
|
| 4 | `POST /detect` with small-image | 200 OK — engine initializes on retry |
|
|
|
|
**Pass criteria**: Second detection succeeds after loader recovers. System does not permanently lock into error state.
|
|
|
|
---
|
|
|
|
### NFT-RES-04: Service restart with in-memory state loss
|
|
|
|
**Summary**: Verify that after a service restart, all in-memory state (_active_detections, _event_queues) is cleanly reset.
|
|
**Traces to**: RESTRICT-OP-5, RESTRICT-OP-6
|
|
|
|
**Preconditions**:
|
|
- Previous detection may have been in progress
|
|
|
|
**Fault injection**:
|
|
- Restart detections container
|
|
|
|
**Steps**:
|
|
|
|
| Step | Action | Expected Behavior |
|
|
|------|--------|------------------|
|
|
| 1 | Restart detections container | — |
|
|
| 2 | `GET /health` | Returns `aiAvailability: "None"` (fresh start) |
|
|
| 3 | `POST /detect/any-media-id` | Accepted (no stale _active_detections blocking it) |
|
|
|
|
**Pass criteria**: No stale state from previous session. All endpoints functional after restart.
|