Oleksandr Bezdieniezhnykh
3.0 KiB
Resumable Download Manager
Task: AZ-184_resumable_download_manager Name: Resumable Download Manager Description: Implement a resumable HTTP download manager for the loader that handles intermittent Starlink connectivity Complexity: 3 points Dependencies: None Component: Loader Tracker: AZ-184 Epic: AZ-181
Problem
Jetsons on UAVs have intermittent Starlink connectivity. Downloads of large artifacts (AI models ~500MB, Docker images ~1GB) must survive connection drops and resume from where they left off.
Outcome
- Downloads resume from the last byte received after connectivity loss
- Completed downloads are verified with SHA-256 before use
- Downloaded artifacts are decrypted with per-artifact AES-256 keys
- State persists across loader restarts
Scope
Included
- Resumable HTTP downloads using Range headers (S3 supports natively)
- JSON state file on disk tracking: url, expected_sha256, expected_size, bytes_downloaded, temp_file_path
- SHA-256 verification of completed downloads
- AES-256 decryption of downloaded artifacts using per-artifact key from /get-update response
- Retry with exponential backoff (1min, 5min, 15min, 1hr, max 4hr)
- State machine: pending -> downloading -> paused -> verifying -> decrypting -> complete / failed
Excluded
- Update check logic (AZ-185)
- Applying updates (AZ-185)
- CDN upload (AZ-186)
Acceptance Criteria
AC-1: Resume after connection drop Given a download is 60% complete and connectivity is lost When connectivity returns Then download resumes from byte offset (60% of file), not from scratch
AC-2: SHA-256 mismatch triggers re-download Given a completed download with corrupted data When SHA-256 verification fails Then the partial file is deleted and download restarts from scratch
AC-3: Decryption produces correct output Given a completed and verified download When decrypted with the per-artifact AES-256 key Then the output matches the original unencrypted artifact
AC-4: State survives restart Given a download is 40% complete and the loader container restarts When the loader starts again Then the download resumes from 40%, not from scratch
AC-5: Exponential backoff on repeated failures Given multiple consecutive connection failures When retrying Then wait times follow exponential backoff pattern
Unit Tests
| AC Ref | What to Test | Required Outcome |
|---|---|---|
| AC-1 | Mock HTTP server drops connection mid-transfer | Resume with Range header from correct offset |
| AC-2 | Corrupt downloaded file | SHA-256 check fails, file deleted, retry flag set |
| AC-3 | Encrypt test file, download, decrypt | Round-trip matches original |
| AC-4 | Write state file, reload | State correctly restored |
| AC-5 | Track retry intervals | Backoff pattern matches spec |
Constraints
- Must work inside Docker container
- S3-compatible CDN (current CDNManager already uses boto3)
- State file location must be on a volume that persists across container restarts