mirror of
https://github.com/azaion/loader.git
synced 2026-04-22 08:56:32 +00:00
[AZ-185][AZ-186] Batch 2
Made-with: Cursor
This commit is contained in:
@@ -0,0 +1,53 @@
|
||||
# Publish artifact script (AZ-186)
|
||||
|
||||
Training services and CI/CD call `scripts/publish_artifact.py` after producing an artifact (for example a `.trt` model or a Docker image tarball). The script gzip-compresses the file, encrypts it with a random 32-byte AES-256 key (AES-CBC with PKCS7, IV prefixed), uploads the ciphertext to S3, and registers metadata with the admin API.
|
||||
|
||||
## CLI
|
||||
|
||||
```text
|
||||
python scripts/publish_artifact.py \
|
||||
--file /path/to/artifact \
|
||||
--resource-name my_model \
|
||||
--dev-stage dev \
|
||||
--architecture arm64 \
|
||||
--version 2026-04-15
|
||||
```
|
||||
|
||||
Object key: `{dev_stage}/{resource_name}-{architecture}-{version}.enc`
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Variable | Required | Purpose |
|
||||
|----------|----------|---------|
|
||||
| `S3_ENDPOINT` | yes | S3-compatible endpoint URL |
|
||||
| `S3_ACCESS_KEY` | yes | Upload credentials |
|
||||
| `S3_SECRET_KEY` | yes | Upload credentials |
|
||||
| `S3_BUCKET` | yes | Target bucket |
|
||||
| `ADMIN_API_URL` | yes | Admin API base URL (no trailing path for publish) |
|
||||
| `ADMIN_API_TOKEN` | yes | Bearer token for the publish request |
|
||||
| `CDN_PUBLIC_BASE_URL` | no | If set, `cdn_url` in the registration payload is `{CDN_PUBLIC_BASE_URL}/{object_key}`; otherwise it defaults to `{S3_ENDPOINT}/{S3_BUCKET}/{object_key}` |
|
||||
| `ADMIN_API_PUBLISH_PATH` | no | Defaults to `internal/resources/publish`; POST is sent to `{ADMIN_API_URL}/{ADMIN_API_PUBLISH_PATH}` |
|
||||
|
||||
## Admin API contract
|
||||
|
||||
`POST {ADMIN_API_URL}/internal/resources/publish` (unless overridden) with JSON body:
|
||||
|
||||
- `resource_name`, `dev_stage`, `architecture`, `version` (strings)
|
||||
- `cdn_url` (string)
|
||||
- `sha256` (lowercase hex of the uploaded ciphertext file, including the 16-byte IV)
|
||||
- `encryption_key` (64-character hex encoding of the raw 32-byte AES key)
|
||||
- `size_bytes` (integer size of the uploaded ciphertext file)
|
||||
|
||||
The loader expects the same `encryption_key` and `sha256` semantics as returned by fleet `POST /get-update` (hex key, hash of the ciphertext object).
|
||||
|
||||
## Dependencies
|
||||
|
||||
Use the same major versions as the loader: `boto3`, `cryptography`, `requests` (see `requirements.txt`). A minimal install for a training host is:
|
||||
|
||||
```text
|
||||
pip install boto3==1.40.9 cryptography==44.0.2 requests==2.32.4
|
||||
```
|
||||
|
||||
## Woodpecker
|
||||
|
||||
Pipeline `.woodpecker/build-arm.yml` saves the built image to `loader-image.tar` and runs this script in a follow-up step. Configure the environment variables above as Woodpecker secrets for that step.
|
||||
Reference in New Issue
Block a user