mirror of
https://github.com/azaion/ai-training.git
synced 2026-04-22 11:16:35 +00:00
Refactor constants management to use Pydantic BaseModel for configuration
- Replaced module-level path variables in constants.py with a structured Pydantic Config class. - Updated all relevant modules (train.py, augmentation.py, exports.py, dataset-visualiser.py, manual_run.py) to access paths through the new config structure. - Fixed bugs related to image processing and model saving. - Enhanced test infrastructure to accommodate the new configuration approach. This refactor improves code maintainability and clarity by centralizing configuration management.
This commit is contained in:
@@ -0,0 +1,99 @@
|
||||
# Component Relationship Diagram
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph "Core Infrastructure"
|
||||
core[01 Core<br/>constants, utils]
|
||||
end
|
||||
|
||||
subgraph "Security & Hardware"
|
||||
sec[02 Security<br/>security, hardware_service]
|
||||
end
|
||||
|
||||
subgraph "API & CDN Client"
|
||||
api[03 API & CDN<br/>api_client, cdn_manager]
|
||||
end
|
||||
|
||||
subgraph "Data Models"
|
||||
dto[04 Data Models<br/>dto/annotationClass, dto/imageLabel]
|
||||
end
|
||||
|
||||
subgraph "Data Pipeline"
|
||||
data[05 Data Pipeline<br/>augmentation, convert-annotations,<br/>dataset-visualiser]
|
||||
end
|
||||
|
||||
subgraph "Training Pipeline"
|
||||
train[06 Training<br/>train, exports, manual_run]
|
||||
end
|
||||
|
||||
subgraph "Inference Engine"
|
||||
infer[07 Inference<br/>inference/*, start_inference]
|
||||
end
|
||||
|
||||
subgraph "Annotation Queue Service"
|
||||
queue[08 Annotation Queue<br/>annotation-queue/*]
|
||||
end
|
||||
|
||||
core --> api
|
||||
core --> data
|
||||
core --> train
|
||||
core --> infer
|
||||
|
||||
sec --> api
|
||||
sec --> train
|
||||
sec --> infer
|
||||
|
||||
api --> train
|
||||
api --> infer
|
||||
|
||||
dto --> data
|
||||
dto --> train
|
||||
|
||||
data -.->|augmented images<br/>on filesystem| train
|
||||
|
||||
queue -.->|annotation files<br/>on filesystem| data
|
||||
|
||||
style core fill:#e8f5e9
|
||||
style sec fill:#fff3e0
|
||||
style api fill:#e3f2fd
|
||||
style dto fill:#f3e5f5
|
||||
style data fill:#fce4ec
|
||||
style train fill:#e0f2f1
|
||||
style infer fill:#f9fbe7
|
||||
style queue fill:#efebe9
|
||||
```
|
||||
|
||||
## Component Summary
|
||||
|
||||
| # | Component | Modules | Purpose |
|
||||
|---|-----------|---------|---------|
|
||||
| 01 | Core Infrastructure | constants, utils | Shared paths, config keys, helper classes |
|
||||
| 02 | Security & Hardware | security, hardware_service | AES encryption, key derivation, hardware fingerprinting |
|
||||
| 03 | API & CDN Client | api_client, cdn_manager | REST API + S3 CDN communication, split-resource pattern |
|
||||
| 04 | Data Models | dto/annotationClass, dto/imageLabel | Annotation classes, image+label container |
|
||||
| 05 | Data Pipeline | augmentation, convert-annotations, dataset-visualiser | Data prep: augmentation, format conversion, visualization |
|
||||
| 06 | Training Pipeline | train, exports, manual_run | YOLO training, model export, encrypted upload |
|
||||
| 07 | Inference Engine | inference/dto, onnx_engine, tensorrt_engine, inference, start_inference | Real-time video object detection |
|
||||
| 08 | Annotation Queue | annotation_queue_dto, annotation_queue_handler | Async annotation event consumer service |
|
||||
|
||||
## Module Coverage Verification
|
||||
All 21 source modules are covered by exactly one component:
|
||||
- 01: constants, utils (2)
|
||||
- 02: security, hardware_service (2)
|
||||
- 03: api_client, cdn_manager (2)
|
||||
- 04: dto/annotationClass, dto/imageLabel (2)
|
||||
- 05: augmentation, convert-annotations, dataset-visualiser (3)
|
||||
- 06: train, exports, manual_run (3)
|
||||
- 07: inference/dto, inference/onnx_engine, inference/tensorrt_engine, inference/inference, start_inference (5)
|
||||
- 08: annotation-queue/annotation_queue_dto, annotation-queue/annotation_queue_handler (2)
|
||||
- **Total: 21 modules covered**
|
||||
|
||||
## Inter-Component Communication
|
||||
| From | To | Mechanism |
|
||||
|------|----|-----------|
|
||||
| Annotation Queue → Data Pipeline | Filesystem | Queue writes images/labels → augmentation reads them |
|
||||
| Data Pipeline → Training | Filesystem | Augmented images in `/azaion/data-processed/` → dataset formation |
|
||||
| Training → API & CDN | API calls | Encrypted model upload (split big/small) |
|
||||
| Inference → API & CDN | API calls | Encrypted model download (reassemble big/small) |
|
||||
| API & CDN → Security | Function calls | Encryption/decryption for transit protection |
|
||||
| API & CDN → Core | Import | Path constants, config file references |
|
||||
Reference in New Issue
Block a user