mirror of https://github.com/azaion/ai-training.git synced 2026-04-22 21:46:35 +00:00

Files

T

Oleksandr Bezdieniezhnykh 142c6c4de8 Refactor constants management to use Pydantic BaseModel for configuration

- Replaced module-level path variables in constants.py with a structured Pydantic Config class.
- Updated all relevant modules (train.py, augmentation.py, exports.py, dataset-visualiser.py, manual_run.py) to access paths through the new config structure.
- Fixed bugs related to image processing and model saving.
- Enhanced test infrastructure to accommodate the new configuration approach.

This refactor improves code maintainability and clarity by centralizing configuration management.

2026-03-27 18:18:30 +02:00

3.1 KiB

Raw Blame History

Module: api_client

Purpose

HTTP client for the Azaion backend API. Handles authentication, file upload/download with encryption, and split-resource management (big/small model parts).

Public Interface

ApiCredentials

Field	Type	Description
`url`	str	API base URL
`email`	str	Login email
`password`	str	Login password

ApiClient

Method	Signature	Returns	Description
`__init__`	`()`	—	Reads `config.yaml` for API creds, reads `cdn.yaml` via `load_bytes`, initializes CDNManager
`login`	`()`	—	POST `/login` → stores JWT token
`upload_file`	`(filename: str, file_bytes: bytearray, folder: str)`	—	Uploads file to API resource endpoint
`load_bytes`	`(filename: str, folder: str) -> bytes`	Decrypted bytes	Downloads encrypted resource from API, decrypts with hardware-bound key
`load_big_small_resource`	`(resource_name: str, folder: str, key: str) -> bytes`	Decrypted bytes	Reassembles a split resource: big part from local disk + small part from API, decrypts combined
`upload_big_small_resource`	`(resource: bytes, resource_name: str, folder: str, key: str)`	—	Encrypts resource, splits into big (CDN) + small (API), uploads both

Internal Logic

Authentication: JWT-based. Auto-login on first request, re-login on 401/403.
load_bytes: Sends hardware fingerprint in request payload. Server returns encrypted bytes. Client decrypts using key derived from credentials + hardware hash.
Split resource pattern: Large files (models) are split into two parts:
- *.small — first N bytes (min of SMALL_SIZE_KB * 1024 or 20% of encrypted size) — stored on API server
- *.big — remainder — stored on CDN (S3)
- This split ensures the model cannot be reconstructed from either storage alone.
CDN initialization: On construction, cdn.yaml is loaded via load_bytes (from API, encrypted), then used to initialize CDNManager.

Dependencies

constants — config file paths, size thresholds, model folder name
cdn_manager — CDNCredentials, CDNManager for S3 operations
hardware_service — get_hardware_info() for hardware fingerprint
security — encryption/decryption, key derivation
requests (external) — HTTP client
yaml (external) — config parsing
io, json, os (stdlib)

Consumers

exports, train, start_inference

Data Models

ApiCredentials — API connection credentials.

Configuration

config.yaml — API URL, email, password
cdn.yaml — CDN credentials (loaded encrypted from API at init time)

External Integrations

Azaion REST API (POST /login, POST /resources/{folder}, POST /resources/get/{folder})
S3-compatible CDN via CDNManager

Security

JWT token-based authentication with auto-refresh on 401/403
Hardware-bound encryption for downloaded resources
Split model storage prevents single-point compromise
Credentials read from config.yaml (hardcoded in file — security concern)

Tests

None.

3.1 KiB Raw Blame History