mirror of
https://github.com/azaion/ai-training.git
synced 2026-04-22 21:56:36 +00:00
Oleksandr Bezdieniezhnykh
142c6c4de8
- Replaced module-level path variables in constants.py with a structured Pydantic Config class. - Updated all relevant modules (train.py, augmentation.py, exports.py, dataset-visualiser.py, manual_run.py) to access paths through the new config structure. - Fixed bugs related to image processing and model saving. - Enhanced test infrastructure to accommodate the new configuration approach. This refactor improves code maintainability and clarity by centralizing configuration management.
65 lines
3.1 KiB
Markdown
65 lines
3.1 KiB
Markdown
# Module: api_client
|
|
|
|
## Purpose
|
|
HTTP client for the Azaion backend API. Handles authentication, file upload/download with encryption, and split-resource management (big/small model parts).
|
|
|
|
## Public Interface
|
|
|
|
### ApiCredentials
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `url` | str | API base URL |
|
|
| `email` | str | Login email |
|
|
| `password` | str | Login password |
|
|
|
|
### ApiClient
|
|
| Method | Signature | Returns | Description |
|
|
|--------|-----------|---------|-------------|
|
|
| `__init__` | `()` | — | Reads `config.yaml` for API creds, reads `cdn.yaml` via `load_bytes`, initializes CDNManager |
|
|
| `login` | `()` | — | POST `/login` → stores JWT token |
|
|
| `upload_file` | `(filename: str, file_bytes: bytearray, folder: str)` | — | Uploads file to API resource endpoint |
|
|
| `load_bytes` | `(filename: str, folder: str) -> bytes` | Decrypted bytes | Downloads encrypted resource from API, decrypts with hardware-bound key |
|
|
| `load_big_small_resource` | `(resource_name: str, folder: str, key: str) -> bytes` | Decrypted bytes | Reassembles a split resource: big part from local disk + small part from API, decrypts combined |
|
|
| `upload_big_small_resource` | `(resource: bytes, resource_name: str, folder: str, key: str)` | — | Encrypts resource, splits into big (CDN) + small (API), uploads both |
|
|
|
|
## Internal Logic
|
|
- **Authentication**: JWT-based. Auto-login on first request, re-login on 401/403.
|
|
- **load_bytes**: Sends hardware fingerprint in request payload. Server returns encrypted bytes. Client decrypts using key derived from credentials + hardware hash.
|
|
- **Split resource pattern**: Large files (models) are split into two parts:
|
|
- `*.small` — first N bytes (min of `SMALL_SIZE_KB * 1024` or 20% of encrypted size) — stored on API server
|
|
- `*.big` — remainder — stored on CDN (S3)
|
|
- This split ensures the model cannot be reconstructed from either storage alone.
|
|
- **CDN initialization**: On construction, `cdn.yaml` is loaded via `load_bytes` (from API, encrypted), then used to initialize `CDNManager`.
|
|
|
|
## Dependencies
|
|
- `constants` — config file paths, size thresholds, model folder name
|
|
- `cdn_manager` — CDNCredentials, CDNManager for S3 operations
|
|
- `hardware_service` — `get_hardware_info()` for hardware fingerprint
|
|
- `security` — encryption/decryption, key derivation
|
|
- `requests` (external) — HTTP client
|
|
- `yaml` (external) — config parsing
|
|
- `io`, `json`, `os` (stdlib)
|
|
|
|
## Consumers
|
|
exports, train, start_inference
|
|
|
|
## Data Models
|
|
`ApiCredentials` — API connection credentials.
|
|
|
|
## Configuration
|
|
- `config.yaml` — API URL, email, password
|
|
- `cdn.yaml` — CDN credentials (loaded encrypted from API at init time)
|
|
|
|
## External Integrations
|
|
- Azaion REST API (`POST /login`, `POST /resources/{folder}`, `POST /resources/get/{folder}`)
|
|
- S3-compatible CDN via CDNManager
|
|
|
|
## Security
|
|
- JWT token-based authentication with auto-refresh on 401/403
|
|
- Hardware-bound encryption for downloaded resources
|
|
- Split model storage prevents single-point compromise
|
|
- Credentials read from `config.yaml` (hardcoded in file — security concern)
|
|
|
|
## Tests
|
|
None.
|