mirror of
https://github.com/azaion/ai-training.git
synced 2026-04-22 10:56:36 +00:00
Refactor constants management to use Pydantic BaseModel for configuration
- Replaced module-level path variables in constants.py with a structured Pydantic Config class. - Updated all relevant modules (train.py, augmentation.py, exports.py, dataset-visualiser.py, manual_run.py) to access paths through the new config structure. - Fixed bugs related to image processing and model saving. - Enhanced test infrastructure to accommodate the new configuration approach. This refactor improves code maintainability and clarity by centralizing configuration management.
This commit is contained in:
@@ -0,0 +1,71 @@
|
||||
# Test Environment
|
||||
|
||||
## Runtime Requirements
|
||||
|
||||
| Requirement | Specification |
|
||||
|-------------|--------------|
|
||||
| Python | 3.10+ |
|
||||
| OS | Linux or macOS (POSIX filesystem paths) |
|
||||
| GPU | Optional — ONNX inference falls back to CPUExecutionProvider |
|
||||
| Disk | Temp directory for fixture data (~500MB for augmentation output) |
|
||||
| Network | Not required (all tests are offline) |
|
||||
|
||||
## Execution Modes
|
||||
|
||||
Tests MUST be runnable in two ways:
|
||||
|
||||
### 1. Local (no Docker) — primary mode
|
||||
Run directly on the host machine. Required for macOS development where Docker has GPU/performance limitations.
|
||||
|
||||
```bash
|
||||
scripts/run-tests-local.sh
|
||||
```
|
||||
|
||||
### 2. Docker — CI/portable mode
|
||||
Run inside a container for reproducible CI environments (Linux-based CI runners).
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.test.yml up --build --abort-on-container-exit
|
||||
```
|
||||
|
||||
Both modes run the same pytest suite; the only difference is the runtime environment.
|
||||
|
||||
## Dependencies
|
||||
|
||||
All test dependencies are a subset of the production `requirements.txt` plus pytest:
|
||||
|
||||
| Package | Purpose |
|
||||
|---------|---------|
|
||||
| pytest | Test runner |
|
||||
| albumentations | Augmentation tests |
|
||||
| opencv-python-headless | Image I/O (headless — no GUI) |
|
||||
| numpy | Array operations |
|
||||
| onnxruntime | ONNX inference (CPU fallback) |
|
||||
| cryptography | Encryption tests |
|
||||
| msgpack | Annotation queue message tests |
|
||||
| PyYAML | Config/YAML generation tests |
|
||||
|
||||
## Fixture Data
|
||||
|
||||
| Fixture | Location | Size |
|
||||
|---------|----------|------|
|
||||
| 100 annotated images | `_docs/00_problem/input_data/dataset/images/` | ~50MB |
|
||||
| 100 YOLO labels | `_docs/00_problem/input_data/dataset/labels/` | ~10KB |
|
||||
| ONNX model | `_docs/00_problem/input_data/azaion.onnx` | 81MB |
|
||||
| Class definitions | `classes.json` (project root) | 2KB |
|
||||
|
||||
## Test Isolation
|
||||
|
||||
- Each test creates a temporary directory (via `tmp_path` pytest fixture) for filesystem operations
|
||||
- No tests modify the actual `/azaion/` directory structure
|
||||
- No tests require running external services (RabbitMQ, Azaion API, S3 CDN)
|
||||
- Constants paths are patched/overridden to point to temp directories during tests
|
||||
|
||||
## Excluded (Require External Services)
|
||||
|
||||
| Component | Service Required | Reason for Exclusion |
|
||||
|-----------|-----------------|---------------------|
|
||||
| API upload/download | Azaion REST API | No mock server; real API has auth |
|
||||
| CDN upload/download | S3-compatible CDN | No mock S3; real CDN has credentials |
|
||||
| Queue consumption | RabbitMQ Streams | No mock broker; rstream requires live connection |
|
||||
| TensorRT inference | NVIDIA GPU + TensorRT | Hardware-specific; cannot run in CI without GPU |
|
||||
Reference in New Issue
Block a user