azaion/loader
1
0
Fork 0
mirror of https://github.com/azaion/loader.git synced 2026-04-22 09:16:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

[AZ-185][AZ-186] Batch 2

Browse Source 
Made-with: Cursor
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-04-15 07:32:37 +03:00
parent d244799f02
commit 9a0248af72
18 changed files with 1857 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_docs/02_tasks/done/AZ-184_resumable_download_manager.md
+79
View File
@@ -0,0 +1,79 @@
# Resumable Download Manager
**Task**: AZ-184_resumable_download_manager
**Name**: Resumable Download Manager
**Description**: Implement a resumable HTTP download manager for the loader that handles intermittent Starlink connectivity
**Complexity**: 3 points
**Dependencies**: None
**Component**: Loader
**Tracker**: AZ-184
**Epic**: AZ-181
## Problem
Jetsons on UAVs have intermittent Starlink connectivity. Downloads of large artifacts (AI models ~500MB, Docker images ~1GB) must survive connection drops and resume from where they left off.
## Outcome
- Downloads resume from the last byte received after connectivity loss
- Completed downloads are verified with SHA-256 before use
- Downloaded artifacts are decrypted with per-artifact AES-256 keys
- State persists across loader restarts
## Scope
### Included
- Resumable HTTP downloads using Range headers (S3 supports natively)
- JSON state file on disk tracking: url, expected_sha256, expected_size, bytes_downloaded, temp_file_path
- SHA-256 verification of completed downloads
- AES-256 decryption of downloaded artifacts using per-artifact key from /get-update response
- Retry with exponential backoff (1min, 5min, 15min, 1hr, max 4hr)
- State machine: pending -> downloading -> paused -> verifying -> decrypting -> complete / failed
### Excluded
- Update check logic (AZ-185)
- Applying updates (AZ-185)
- CDN upload (AZ-186)
## Acceptance Criteria
**AC-1: Resume after connection drop**
Given a download is 60% complete and connectivity is lost
When connectivity returns
Then download resumes from byte offset (60% of file), not from scratch
**AC-2: SHA-256 mismatch triggers re-download**
Given a completed download with corrupted data
When SHA-256 verification fails
Then the partial file is deleted and download restarts from scratch
**AC-3: Decryption produces correct output**
Given a completed and verified download
When decrypted with the per-artifact AES-256 key
Then the output matches the original unencrypted artifact
**AC-4: State survives restart**
Given a download is 40% complete and the loader container restarts
When the loader starts again
Then the download resumes from 40%, not from scratch
**AC-5: Exponential backoff on repeated failures**
Given multiple consecutive connection failures
When retrying
Then wait times follow exponential backoff pattern
## Unit Tests
| AC Ref | What to Test | Required Outcome |
|--------|-------------|-----------------|
| AC-1 | Mock HTTP server drops connection mid-transfer | Resume with Range header from correct offset |
| AC-2 | Corrupt downloaded file | SHA-256 check fails, file deleted, retry flag set |
| AC-3 | Encrypt test file, download, decrypt | Round-trip matches original |
| AC-4 | Write state file, reload | State correctly restored |
| AC-5 | Track retry intervals | Backoff pattern matches spec |
## Constraints
- Must work inside Docker container
- S3-compatible CDN (current CDNManager already uses boto3)
- State file location must be on a volume that persists across container restarts