mirror of
https://github.com/azaion/gps-denied-onboard.git
synced 2026-06-21 22:31:13 +00:00
Oleksandr Bezdieniezhnykh
6599d828d2
Three blackbox-harness tasks landed together — all depend only on
AZ-406 and unblock the FT-* / NFT-* scenario tasks scheduled for
batches 69+.
AZ-407 — Static fixture builders (3pt):
* tile-cache-builder/{builder.py, Dockerfile, build.sh} produces a
deterministic tile-cache-fixture Docker volume from
_docs/00_problem/input_data/. Reproducibility primitives: sorted
iteration, frozen PIL JPEG settings, FAISS HNSW32 built single-
threaded with seeded stub descriptors.
* age-injector/{age_injector.py, inject.sh} clones the volume and
shifts capture_date by N×30.44 days; tile JPEG bytes preserved
bit-identical. Emits synth-age-7mo + synth-age-13mo volumes.
* cold-boot/cold_boot_fixture.json: frozen FC pose snapshot at
Derkachi sector centre, schema v1.
* secrets/mavlink-test-passkey.txt: 64-hex with required
`# TEST ONLY` header line per AC-5. Passkey-equality test now
compares the secret line after stripping the header.
* security/cve-2025-53644.jpg: synthetic 158-byte malformed JPEG
(truncated SOS marker). OpenCV 4.11.x rejects gracefully with
imdecode → None. AZ-439 will sharpen for ASan instrumentation.
* Top-level Makefile with `make fixtures` / `make fixtures-*` /
`make e2e-tier1*` / `make unit-tests` targets.
AZ-444 — Tier-2 Jetson harness wrapper (5pt):
* run-tier2.sh rewritten as orchestrator. Detects local
(aarch64 + TIER2_HOST=localhost) vs remote (ssh into TIER2_HOST).
New flags: -k/--selector, --build-kind production|asan,
--reflash (gated behind TIER2_REFLASH_ACK=1 two-key gate),
--dry-run.
* tier2-on-jetson.sh (new) — on-device delegate. Verifies
gps-denied-onboard{,-asan}.service health; restarts with 5s
tolerance; spawns tegrastats + jtop parallel samplers; tails
ASan unit's journal in asan mode; drives docker compose with
TIER=tier2-jetson; forwards SELECTOR to pytest -k.
* docker/run-tier1.sh (new) — selector-parity sibling.
* AC-1 (selector parity) and AC-6 (reflash gating) unit-tested via
--dry-run output assertions. AC-2/AC-3/AC-4/AC-5 are hardware-
loop ACs verified by the Tier-2 runtime smoke (no Jetson in the
unit-test layer).
AZ-445 — CSV reporter + evidence bundler refinements (2pt):
* reporting/nfr_recorder.py (new) — pytest plugin. Provides the
`nfr_recorder` fixture with record_metric(name, value, ac_id)
and partial(ac_id, reason). At session end emits:
- per-nfr/<scenario_id>.json (AC-1)
- traceability-status.json with every AC ID parsed from
traceability-matrix.md, classified Covered/PARTIAL/NOT
COVERED with source scenario IDs (AC-2)
- regression-baseline.json with all numeric metrics (AC-3)
* csv_reporter.py extended — `_outcome_to_result` consults the
aggregator; rows flip PASS → PARTIAL when an AC was marked
PARTIAL by nfr_recorder (AC-4). Graceful fallback when
aggregator isn't registered (unit-test contexts).
* conftest.py registers nfr_recorder in pytest_plugins.
* New --traceability-matrix CLI flag seeds the NOT COVERED rows.
Build / config:
* pyproject.toml dev extras: added Pillow>=10.4,<13.0 for the
tile-cache-builder unit test (broad enough to keep torchvision's
Pillow 12 pin happy; the production builder runs inside its own
Docker image with its own pin).
* Updated test_directory_layout.py to cover 10 new files + replaced
the byte-equal passkey assertion with the header-stripping
variant.
Test results:
* 157 focused tests pass (was 97 in batch 67; +60 new across this
batch). No regressions.
Module-layout / spec drift:
* AZ-407 spec text says `tests/fixtures/...`; module-layout
blackbox_tests entry (commit d7a17a8) authoritatively places the
harness under `e2e/`. Implementation followed the layout entry.
* AZ-444 spec mentions `e2e/tier2/run-tier2.sh`; AZ-406 placed it
at `e2e/jetson/run-tier2.sh`. Kept at `e2e/jetson/` for
consistency.
* Cold-boot README ownership: corrected from AZ-419 to AZ-407 per
AZ-419's own Dependencies field.
Specs archived to _docs/02_tasks/done/. Jira tickets transitioned to
In Testing on commit.
Co-authored-by: Cursor <cursoragent@cursor.com>
81 lines
3.2 KiB
Markdown
81 lines
3.2 KiB
Markdown
# tile-cache-builder (AZ-407)
|
|
|
|
Builds the `tile-cache-fixture` Docker volume from the 60 still-image
|
|
satellite references in `_docs/00_problem/input_data/` plus the
|
|
Derkachi route bbox.
|
|
|
|
## Output schema
|
|
|
|
```
|
|
tile-cache-fixture/
|
|
tiles/<zoom>/<x>/<y>.jpg # tile JPEG body
|
|
tiles/<zoom>/<x>/<y>.json # per-tile sidecar (mirrors `tiles` row)
|
|
manifest.csv # sorted manifest (9 columns)
|
|
descriptors.index # FAISS HNSW32 index (omitted if faiss not available)
|
|
```
|
|
|
|
Manifest columns (per `_docs/00_problem/restrictions.md` § Satellite
|
|
Imagery + `_docs/02_document/data_model.md` § 2.1):
|
|
|
|
| Column | Type | Notes |
|
|
|--------|------|-------|
|
|
| `zoom_level` | int | Slippy/XYZ zoom |
|
|
| `tile_x`, `tile_y` | int | Tile coords at the zoom |
|
|
| `capture_date` | ISO-8601 date | Default `2025-11-01` (frozen so freshness gate treats as fresh) |
|
|
| `source` | enum | `googlemaps` for real paired tiles, `stub` for D-PROJ-3 fallback |
|
|
| `m_per_px` | float | `0.5` (≥ the AC-8.1 floor) |
|
|
| `jpeg_path` | str | Relative path to the JPEG body |
|
|
| `content_hash` | hex | SHA-256 of the JPEG bytes |
|
|
| `provenance` | str | `paired_gmaps:AD000NNN`, `STUB`, or `STUB_BBOX:derkachi:lat,lon,lat,lon` |
|
|
|
|
## Reproducibility (AC-1)
|
|
|
|
Two consecutive invocations from the same input produce a bit-identical
|
|
output tree:
|
|
|
|
* Input files iterated in lexicographic order
|
|
* PIL JPEG encoded with `quality=85, optimize=False, progressive=False, subsampling=2`
|
|
* Manifest rows sorted by `(zoom_level, tile_x, tile_y)` before CSV
|
|
serialisation
|
|
* FAISS index built single-threaded with `omp_set_num_threads(1)` and
|
|
SHA-derived stub descriptors
|
|
|
|
## Provenance (AC-7)
|
|
|
|
| Item | Source | License |
|
|
|------|--------|---------|
|
|
| Real tile bodies | `_docs/00_problem/input_data/AD*_gmaps.png` (2 paired references) | Project test fixture; safe to redistribute under this repo's license |
|
|
| Stub tile bodies | Generated from `_stub_jpeg_bytes(seed)` (PIL solid-fill) | Fully synthetic; no third-party data |
|
|
| Derkachi bbox tile | Synthetic placeholder until D-PROJ-3 lands | Fully synthetic |
|
|
| FAISS index | SHA-derived stub vectors (not real VPR descriptors) | Fully synthetic |
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
# Production (Docker volume):
|
|
e2e/fixtures/tile-cache-builder/build.sh
|
|
|
|
# Local mode (used by AZ-407 unit test):
|
|
e2e/fixtures/tile-cache-builder/build.sh --local /tmp/tile-cache-out
|
|
```
|
|
|
|
The unit test `e2e/_unit_tests/fixtures/test_tile_cache_builder.py`
|
|
verifies AC-1 / AC-2 / AC-7 by invoking `builder.py` twice against a
|
|
`tmp_path` and asserting the output is byte-identical.
|
|
|
|
## Notes on D-PROJ-3
|
|
|
|
When D-PROJ-3 supplies the production tile-corpus for the Derkachi
|
|
sector, the stub tiles produced here (any row with `provenance = STUB`)
|
|
should be replaced by real Suite Sat Service tiles for those
|
|
footprints. The builder will then no longer fall back to
|
|
`_stub_jpeg_bytes` — every still that lacks a paired `_gmaps.png`
|
|
will draw from the real corpus instead.
|
|
|
|
## Owned by
|
|
|
|
AZ-407 (this task). The FAISS-stub descriptor format will not be used
|
|
in production; the production VPR pipeline (C2) emits real DINOv2
|
|
descriptors. The stub format is sufficient for AZ-407's reproducibility
|
|
and schema contracts only.
|