mirror of https://github.com/azaion/ai-training.git synced 2026-04-23 04:16:35 +00:00

Files

T

Oleksandr Bezdieniezhnykh 142c6c4de8 Refactor constants management to use Pydantic BaseModel for configuration

- Replaced module-level path variables in constants.py with a structured Pydantic Config class.
- Updated all relevant modules (train.py, augmentation.py, exports.py, dataset-visualiser.py, manual_run.py) to access paths through the new config structure.
- Fixed bugs related to image processing and model saving.
- Enhanced test infrastructure to accommodate the new configuration approach.

This refactor improves code maintainability and clarity by centralizing configuration management.

2026-03-27 18:18:30 +02:00

1.9 KiB

Raw Blame History

Module: cdn_manager

Purpose

Manages file upload and download to/from an S3-compatible CDN (MinIO/similar) using separate credentials for upload and download operations.

Public Interface

CDNCredentials

Field	Type	Description
`host`	str	CDN endpoint URL
`downloader_access_key`	str	S3 access key for downloads
`downloader_access_secret`	str	S3 secret for downloads
`uploader_access_key`	str	S3 access key for uploads
`uploader_access_secret`	str	S3 secret for uploads

CDNManager

Method	Signature	Returns	Description
`__init__`	`(credentials: CDNCredentials)`	—	Creates two boto3 S3 clients (download + upload)
`upload`	`(bucket: str, filename: str, file_bytes: bytearray) -> bool`	True on success	Uploads bytes to S3 bucket
`download`	`(bucket: str, filename: str) -> bool`	True on success	Downloads file from S3 to current directory

Internal Logic

Maintains two separate boto3 S3 clients with different credentials (read vs write separation)
Upload uses upload_fileobj with in-memory BytesIO wrapper
Download uses download_file (saves directly to disk with same filename)
Both methods catch all exceptions, print error, return bool

Dependencies

boto3 (external) — S3 client
io, sys, yaml, os (stdlib) — Note: sys, yaml, os are imported but unused

Consumers

api_client, exports, train, start_inference

Data Models

CDNCredentials — plain data class holding S3 access credentials.

Configuration

Credentials loaded from cdn.yaml by callers (not by this module directly).

External Integrations

S3-compatible object storage (configured via CDNCredentials.host)

Security

Separate read/write credentials enforce least-privilege access
Credentials passed in at construction time, not hardcoded here

Tests

None.

1.9 KiB Raw Blame History