mirror of
https://github.com/azaion/ai-training.git
synced 2026-04-22 22:16:35 +00:00
Oleksandr Bezdieniezhnykh
142c6c4de8
- Replaced module-level path variables in constants.py with a structured Pydantic Config class. - Updated all relevant modules (train.py, augmentation.py, exports.py, dataset-visualiser.py, manual_run.py) to access paths through the new config structure. - Fixed bugs related to image processing and model saving. - Enhanced test infrastructure to accommodate the new configuration approach. This refactor improves code maintainability and clarity by centralizing configuration management.
100 lines
3.6 KiB
Markdown
100 lines
3.6 KiB
Markdown
# Component Relationship Diagram
|
|
|
|
```mermaid
|
|
graph TD
|
|
subgraph "Core Infrastructure"
|
|
core[01 Core<br/>constants, utils]
|
|
end
|
|
|
|
subgraph "Security & Hardware"
|
|
sec[02 Security<br/>security, hardware_service]
|
|
end
|
|
|
|
subgraph "API & CDN Client"
|
|
api[03 API & CDN<br/>api_client, cdn_manager]
|
|
end
|
|
|
|
subgraph "Data Models"
|
|
dto[04 Data Models<br/>dto/annotationClass, dto/imageLabel]
|
|
end
|
|
|
|
subgraph "Data Pipeline"
|
|
data[05 Data Pipeline<br/>augmentation, convert-annotations,<br/>dataset-visualiser]
|
|
end
|
|
|
|
subgraph "Training Pipeline"
|
|
train[06 Training<br/>train, exports, manual_run]
|
|
end
|
|
|
|
subgraph "Inference Engine"
|
|
infer[07 Inference<br/>inference/*, start_inference]
|
|
end
|
|
|
|
subgraph "Annotation Queue Service"
|
|
queue[08 Annotation Queue<br/>annotation-queue/*]
|
|
end
|
|
|
|
core --> api
|
|
core --> data
|
|
core --> train
|
|
core --> infer
|
|
|
|
sec --> api
|
|
sec --> train
|
|
sec --> infer
|
|
|
|
api --> train
|
|
api --> infer
|
|
|
|
dto --> data
|
|
dto --> train
|
|
|
|
data -.->|augmented images<br/>on filesystem| train
|
|
|
|
queue -.->|annotation files<br/>on filesystem| data
|
|
|
|
style core fill:#e8f5e9
|
|
style sec fill:#fff3e0
|
|
style api fill:#e3f2fd
|
|
style dto fill:#f3e5f5
|
|
style data fill:#fce4ec
|
|
style train fill:#e0f2f1
|
|
style infer fill:#f9fbe7
|
|
style queue fill:#efebe9
|
|
```
|
|
|
|
## Component Summary
|
|
|
|
| # | Component | Modules | Purpose |
|
|
|---|-----------|---------|---------|
|
|
| 01 | Core Infrastructure | constants, utils | Shared paths, config keys, helper classes |
|
|
| 02 | Security & Hardware | security, hardware_service | AES encryption, key derivation, hardware fingerprinting |
|
|
| 03 | API & CDN Client | api_client, cdn_manager | REST API + S3 CDN communication, split-resource pattern |
|
|
| 04 | Data Models | dto/annotationClass, dto/imageLabel | Annotation classes, image+label container |
|
|
| 05 | Data Pipeline | augmentation, convert-annotations, dataset-visualiser | Data prep: augmentation, format conversion, visualization |
|
|
| 06 | Training Pipeline | train, exports, manual_run | YOLO training, model export, encrypted upload |
|
|
| 07 | Inference Engine | inference/dto, onnx_engine, tensorrt_engine, inference, start_inference | Real-time video object detection |
|
|
| 08 | Annotation Queue | annotation_queue_dto, annotation_queue_handler | Async annotation event consumer service |
|
|
|
|
## Module Coverage Verification
|
|
All 21 source modules are covered by exactly one component:
|
|
- 01: constants, utils (2)
|
|
- 02: security, hardware_service (2)
|
|
- 03: api_client, cdn_manager (2)
|
|
- 04: dto/annotationClass, dto/imageLabel (2)
|
|
- 05: augmentation, convert-annotations, dataset-visualiser (3)
|
|
- 06: train, exports, manual_run (3)
|
|
- 07: inference/dto, inference/onnx_engine, inference/tensorrt_engine, inference/inference, start_inference (5)
|
|
- 08: annotation-queue/annotation_queue_dto, annotation-queue/annotation_queue_handler (2)
|
|
- **Total: 21 modules covered**
|
|
|
|
## Inter-Component Communication
|
|
| From | To | Mechanism |
|
|
|------|----|-----------|
|
|
| Annotation Queue → Data Pipeline | Filesystem | Queue writes images/labels → augmentation reads them |
|
|
| Data Pipeline → Training | Filesystem | Augmented images in `/azaion/data-processed/` → dataset formation |
|
|
| Training → API & CDN | API calls | Encrypted model upload (split big/small) |
|
|
| Inference → API & CDN | API calls | Encrypted model download (reassemble big/small) |
|
|
| API & CDN → Security | Function calls | Encryption/decryption for transit protection |
|
|
| API & CDN → Core | Import | Path constants, config file references |
|