# Test Environment

## Overview

**System under test**: Azaion.Detections — FastAPI HTTP service exposing `POST /detect`, `POST /detect/{media_id}`, `GET /detect/stream`, `GET /health`
**Consumer app purpose**: Standalone test runner that exercises the detection service through its public HTTP/SSE interfaces, validating black-box use cases without access to internals.

## Test Execution

**Decision**: local

### Hardware Dependencies Found

| # | Category | Indicator | File | Detail |
|---|----------|-----------|------|--------|
| 1 | Apple CoreML | `import coremltools`, `ct.models.MLModel`, `ct.ComputeUnit.ALL` | `engines/coreml_engine.pyx:15-22` | CoreML engine requires macOS + Apple Silicon for Neural Engine / GPU inference |
| 2 | Apple platform gate | `sys.platform == "darwin"`, `platform.machine() == "arm64"` | `engines/__init__.py:29` | Engine auto-selection branches on macOS arm64 |
| 3 | NVIDIA GPU / CUDA | `import pynvml`, `nvmlDeviceGetCudaComputeCapability` | `engines/__init__.py:7-14` | TensorRT engine requires NVIDIA GPU with compute capability ≥ 6.1 |
| 4 | TensorRT | `from engines.tensorrt_engine import TensorRTEngine` | `engines/__init__.py:43` | TensorRT runtime only available on Linux with NVIDIA drivers |
| 5 | Cython compilation | `setup.py build_ext --inplace` | `Dockerfile:8`, `setup.py` | Cython modules must be compiled natively for the host architecture |

### Rationale

The detection service uses a polymorphic engine factory (`engines/__init__.py:EngineClass`) that auto-selects the inference backend at startup:

1. **TensorRT** — if NVIDIA GPU with CUDA compute ≥ 6.1 is detected
2. **CoreML** — if macOS arm64 with `coremltools` installed
3. **ONNX** — CPU-only fallback

Running in Docker on macOS means a Linux VM where neither CoreML nor TensorRT is available. The service falls back to the ONNX CPU engine — which is a valid but different code path. To test the actual engine used in the target deployment environment, tests must run locally.

### Local Execution Instructions

#### Prerequisites

- macOS arm64 (Apple Silicon) with `coremltools` installed, OR Linux with NVIDIA GPU + TensorRT for the GPU path
- Python 3.11 with Cython modules compiled (`python setup.py build_ext --inplace`)
- `flask` and `gunicorn` installed (for mock services)
- `pytest`, `requests`, `sseclient-py`, `pytest-csv`, `pytest-timeout` installed (for test runner)

#### 1. Start mock services

Open two terminal tabs and start the mocks. The mock-loader serves ONNX/CoreML model files from `e2e/fixtures/models/` and the mock-annotations accepts detection result posts.

```bash
# Terminal 1 — mock-loader (port 8080)
cd e2e/mocks/loader
MODELS_ROOT=../../fixtures gunicorn -b 0.0.0.0:8080 -w 1 --timeout 120 app:app

# Terminal 2 — mock-annotations (port 8081)
cd e2e/mocks/annotations
gunicorn -b 0.0.0.0:8081 -w 1 --timeout 120 app:app
```

#### 2. Start the detection service

```bash
# Terminal 3 — detections (port 8080 by default, override to 8000 to avoid mock conflict)
LOADER_URL=http://localhost:8080 \
ANNOTATIONS_URL=http://localhost:8081 \
python -m uvicorn main:app --host 0.0.0.0 --port 8000
```

#### 3. Run tests

```bash
# Terminal 4 — test runner
cd e2e
BASE_URL=http://localhost:8000 \
MOCK_LOADER_URL=http://localhost:8080 \
MOCK_ANNOTATIONS_URL=http://localhost:8081 \
MEDIA_DIR=./fixtures \
pytest tests/ -v --csv=./results/report.csv --timeout=300
```

#### Environment Variables

| Variable | Default | Purpose |
|----------|---------|---------|
| `BASE_URL` | `http://detections:8080` | Detection service URL (override to `http://localhost:8000` for local) |
| `MOCK_LOADER_URL` | `http://mock-loader:8080` | Mock loader URL (override to `http://localhost:8080` for local) |
| `MOCK_ANNOTATIONS_URL` | `http://mock-annotations:8081` | Mock annotations URL (override to `http://localhost:8081` for local) |
| `MEDIA_DIR` | `/media` | Path to test media fixtures (override to `./fixtures` for local) |
| `LOADER_URL` | `http://loader:8080` | Loader URL for detections service |
| `ANNOTATIONS_URL` | `http://annotations:8080` | Annotations URL for detections service |

## Services

| Service | Role | Local Port | Startup Command |
|---------|------|------------|-----------------|
| detections | System under test | 8000 | `uvicorn main:app --port 8000` |
| mock-loader | Serves model files, accepts engine uploads | 8080 | `gunicorn -b 0.0.0.0:8080 app:app` |
| mock-annotations | Accepts detection results, provides token refresh | 8081 | `gunicorn -b 0.0.0.0:8081 app:app` |

## Consumer Application

**Tech stack**: Python 3, pytest, requests, sseclient-py
**Entry point**: `pytest tests/ -v --csv=./results/report.csv`

### Communication with system under test

| Interface | Protocol | Endpoint | Authentication |
|-----------|----------|----------|----------------|
| Health check | HTTP GET | `{BASE_URL}/health` | None |
| Single image detect | HTTP POST (multipart) | `{BASE_URL}/detect` | None |
| Media detect | HTTP POST (JSON) | `{BASE_URL}/detect/{media_id}` | Bearer JWT + x-refresh-token headers |
| SSE stream | HTTP GET (SSE) | `{BASE_URL}/detect/stream` | None |

### What the consumer does NOT have access to

- No direct import of Cython modules (inference, annotation, engines)
- No direct access to the detections service filesystem or Logs/ directory
- No shared memory with the detections process
- No direct calls to mock-loader or mock-annotations (except for test setup/teardown verification)

## CI/CD Integration

**When to run**: On PR merge to dev, nightly scheduled run
**Pipeline stage**: After unit tests, before deployment
**Gate behavior**: Block merge if any functional test fails; non-functional failures are warnings
**Timeout**: 15 minutes for CPU profile, 30 minutes for GPU profile
**Runner requirement**: macOS arm64 self-hosted runner (for CoreML path) or Linux with NVIDIA GPU (for TensorRT path)

## Reporting

**Format**: CSV
**Columns**: Test ID, Test Name, Execution Time (ms), Result (PASS/FAIL/SKIP), Error Message (if FAIL)
**Output path**: `e2e/results/report.csv`