[AZ-407] [AZ-444] [AZ-445] Batch 68: fixtures, Tier-2 harness, NFR reporter

Three blackbox-harness tasks landed together — all depend only on AZ-406 and unblock the FT-* / NFT-* scenario tasks scheduled for batches 69+. AZ-407 — Static fixture builders (3pt): * tile-cache-builder/{builder.py, Dockerfile, build.sh} produces a deterministic tile-cache-fixture Docker volume from _docs/00_problem/input_data/. Reproducibility primitives: sorted iteration, frozen PIL JPEG settings, FAISS HNSW32 built single- threaded with seeded stub descriptors. * age-injector/{age_injector.py, inject.sh} clones the volume and shifts capture_date by N×30.44 days; tile JPEG bytes preserved bit-identical. Emits synth-age-7mo + synth-age-13mo volumes. * cold-boot/cold_boot_fixture.json: frozen FC pose snapshot at Derkachi sector centre, schema v1. * secrets/mavlink-test-passkey.txt: 64-hex with required `# TEST ONLY` header line per AC-5. Passkey-equality test now compares the secret line after stripping the header. * security/cve-2025-53644.jpg: synthetic 158-byte malformed JPEG (truncated SOS marker). OpenCV 4.11.x rejects gracefully with imdecode → None. AZ-439 will sharpen for ASan instrumentation. * Top-level Makefile with `make fixtures` / `make fixtures-*` / `make e2e-tier1*` / `make unit-tests` targets. AZ-444 — Tier-2 Jetson harness wrapper (5pt): * run-tier2.sh rewritten as orchestrator. Detects local (aarch64 + TIER2_HOST=localhost) vs remote (ssh into TIER2_HOST). New flags: -k/--selector, --build-kind production|asan, --reflash (gated behind TIER2_REFLASH_ACK=1 two-key gate), --dry-run. * tier2-on-jetson.sh (new) — on-device delegate. Verifies gps-denied-onboard{,-asan}.service health; restarts with 5s tolerance; spawns tegrastats + jtop parallel samplers; tails ASan unit's journal in asan mode; drives docker compose with TIER=tier2-jetson; forwards SELECTOR to pytest -k. * docker/run-tier1.sh (new) — selector-parity sibling. * AC-1 (selector parity) and AC-6 (reflash gating) unit-tested via --dry-run output assertions. AC-2/AC-3/AC-4/AC-5 are hardware- loop ACs verified by the Tier-2 runtime smoke (no Jetson in the unit-test layer). AZ-445 — CSV reporter + evidence bundler refinements (2pt): * reporting/nfr_recorder.py (new) — pytest plugin. Provides the `nfr_recorder` fixture with record_metric(name, value, ac_id) and partial(ac_id, reason). At session end emits: - per-nfr/<scenario_id>.json (AC-1) - traceability-status.json with every AC ID parsed from traceability-matrix.md, classified Covered/PARTIAL/NOT COVERED with source scenario IDs (AC-2) - regression-baseline.json with all numeric metrics (AC-3) * csv_reporter.py extended — `_outcome_to_result` consults the aggregator; rows flip PASS → PARTIAL when an AC was marked PARTIAL by nfr_recorder (AC-4). Graceful fallback when aggregator isn't registered (unit-test contexts). * conftest.py registers nfr_recorder in pytest_plugins. * New --traceability-matrix CLI flag seeds the NOT COVERED rows. Build / config: * pyproject.toml dev extras: added Pillow>=10.4,<13.0 for the tile-cache-builder unit test (broad enough to keep torchvision's Pillow 12 pin happy; the production builder runs inside its own Docker image with its own pin). * Updated test_directory_layout.py to cover 10 new files + replaced the byte-equal passkey assertion with the header-stripping variant. Test results: * 157 focused tests pass (was 97 in batch 67; +60 new across this batch). No regressions. Module-layout / spec drift: * AZ-407 spec text says `tests/fixtures/...`; module-layout blackbox_tests entry (commit d7a17a8) authoritatively places the harness under `e2e/`. Implementation followed the layout entry. * AZ-444 spec mentions `e2e/tier2/run-tier2.sh`; AZ-406 placed it at `e2e/jetson/run-tier2.sh`. Kept at `e2e/jetson/` for consistency. * Cold-boot README ownership: corrected from AZ-419 to AZ-407 per AZ-419's own Dependencies field. Specs archived to _docs/02_tasks/done/. Jira tickets transitioned to In Testing on commit. Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-23 02:51:13 +00:00 · 2026-05-16 17:18:01 +03:00
parent e9e6e32097
commit 6599d828d2
35 changed files with 3716 additions and 147 deletions
@@ -1,7 +1,50 @@
-# age-injector
+# age-injector (AZ-407)

-Mutates `tile-cache-fixture` manifest dates → `synth-age-tile-set` for
-FT-N-05 / FT-N-06 (stale-tile rejection on freshness violation).
+Clones a `tile-cache-fixture` tree and mutates ONLY the manifest's
+`capture_date` field (and the per-tile sidecar JSON's matching field)
+to age every entry by a target number of months.

-Delivered by **AZ-407** (Static fixture builders). AZ-406 commits to the
-directory location + name only.
+## Output volumes
+
+| Volume | Age shift | Triggers |
+|--------|-----------|----------|
+| `synth-age-7mo`  | now - 7 mo  | > AC-8.2 active-conflict threshold (6 mo) — FT-N-05 |
+| `synth-age-13mo` | now - 13 mo | > AC-8.2 rear threshold (12 mo) — FT-N-06 |
+
+## Reproducibility
+
+* Tile JPEG bodies are copied bit-identical (`shutil.copytree`).
+* Manifest CSV row order is preserved from the source manifest (the
+  builder already sorts rows by `(zoom, x, y)`).
+* The shifted date is `now - age_months × 30.44 days`, rounded — the
+  AC-3 tolerance is `± 1 day`, well within the 30.44-day floor.
+* The descriptors.index (if present in the source) is copied
+  bit-identical.
+
+## Provenance
+
+The injector itself is fully synthetic. The aged volumes are derivative
+works of `tile-cache-fixture` (same license — see
+`e2e/fixtures/tile-cache-builder/README.md` § Provenance).
+
+## Usage
+
+```bash
+# Production (Docker volumes):
+e2e/fixtures/age-injector/inject.sh
+
+# Local mode (used by AZ-407 unit test):
+e2e/fixtures/age-injector/inject.sh --local /tmp/src /tmp/out-7mo /tmp/out-13mo
+```
+
+The unit test `e2e/_unit_tests/fixtures/test_age_injector.py` verifies
+AC-3 by:
+
+1. Building a small tile-cache fixture from a synthetic 4-still input
+2. Running the injector with `--age-months=7` and `--age-months=13`
+3. Asserting the manifest `capture_date` shifts ±1 day from `now - N*30.44 days`
+4. Asserting every tile JPEG body byte-equals the source
+
+## Owned by
+
+AZ-407 (this task).
@@ -0,0 +1,177 @@
+"""Age-injector for the tile-cache fixture.
+
+Clones a ``tile-cache-fixture`` tree and mutates ONLY the manifest's
+``capture_date`` column (and the per-tile sidecar JSON's matching field).
+Tile JPEG bodies are copied bit-identical.
+
+AC-3 (AZ-407): given target=7mo, every row's ``capture_date`` becomes
+``now - 7 mo`` ± 1 day, exceeding the AC-8.2 active-conflict 6-month
+threshold. Given target=13mo, every row's ``capture_date`` becomes
+``now - 13 mo`` ± 1 day, exceeding the rear 12-month threshold.
+
+Used by FT-N-05 / FT-N-06 (stale-tile rejection on freshness violation).
+
+Public-boundary discipline: this module does NOT import any
+``src/gps_denied_onboard`` symbol. The freshness contract lives in
+``_docs/00_problem/restrictions.md`` § Satellite Imagery (AC-8.2).
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import datetime as _dt
+import json
+import logging
+import shutil
+import sys
+from pathlib import Path
+
+logger = logging.getLogger(__name__)
+
+# 30.44 days/month average — gives `now - N*30 days ± 1 day`, which the
+# AC's "±1 day" tolerance accepts.
+_DAYS_PER_MONTH = 30.44
+
+_MANIFEST_HEADERS = (
+    "zoom_level",
+    "tile_x",
+    "tile_y",
+    "capture_date",
+    "source",
+    "m_per_px",
+    "jpeg_path",
+    "content_hash",
+    "provenance",
+)
+
+
+def _shifted_date(now: _dt.date, age_months: int) -> str:
+    delta_days = int(round(age_months * _DAYS_PER_MONTH))
+    return (now - _dt.timedelta(days=delta_days)).isoformat()
+
+
+def inject(
+    source_dir: Path,
+    output_dir: Path,
+    age_months: int,
+    now: _dt.date | None = None,
+) -> dict:
+    """Clone ``source_dir`` into ``output_dir`` and mutate dates.
+
+    Returns a summary dict:
+        {"row_count": int, "shifted_date": "YYYY-MM-DD", "source_dir": str}
+    """
+
+    if age_months <= 0:
+        raise ValueError(f"age_months must be positive; got {age_months}")
+    if now is None:
+        now = _dt.datetime.now(tz=_dt.timezone.utc).date()
+
+    if output_dir.exists():
+        shutil.rmtree(output_dir)
+    output_dir.mkdir(parents=True)
+
+    # Phase 1: clone the tile tree. Pixels copy bit-identical.
+    src_tiles = source_dir / "tiles"
+    if not src_tiles.is_dir():
+        raise FileNotFoundError(
+            f"{source_dir} does not look like a tile-cache fixture "
+            "(no `tiles/` subdir)"
+        )
+    shutil.copytree(src_tiles, output_dir / "tiles")
+
+    shifted = _shifted_date(now, age_months)
+
+    # Phase 2: mutate per-tile sidecar JSON files.
+    sidecar_count = 0
+    for sidecar in sorted((output_dir / "tiles").rglob("*.json")):
+        data = json.loads(sidecar.read_text())
+        data["capture_date"] = shifted
+        sidecar.write_text(
+            json.dumps(data, sort_keys=True, separators=(",", ":")) + "\n"
+        )
+        sidecar_count += 1
+
+    # Phase 3: re-emit manifest.csv with shifted dates. Row order is
+    # preserved (the source manifest is already sorted by builder.py).
+    src_manifest = source_dir / "manifest.csv"
+    if not src_manifest.is_file():
+        raise FileNotFoundError(f"missing manifest.csv at {src_manifest}")
+    with src_manifest.open() as fp:
+        reader = csv.DictReader(fp)
+        if tuple(reader.fieldnames or ()) != _MANIFEST_HEADERS:
+            raise ValueError(
+                f"unexpected manifest schema: {reader.fieldnames} "
+                f"(expected {list(_MANIFEST_HEADERS)})"
+            )
+        rows = list(reader)
+
+    out_manifest = output_dir / "manifest.csv"
+    with out_manifest.open("w", newline="") as fp:
+        writer = csv.writer(fp, lineterminator="\n")
+        writer.writerow(_MANIFEST_HEADERS)
+        for r in rows:
+            writer.writerow(
+                [
+                    r["zoom_level"],
+                    r["tile_x"],
+                    r["tile_y"],
+                    shifted,
+                    r["source"],
+                    r["m_per_px"],
+                    r["jpeg_path"],
+                    r["content_hash"],
+                    r["provenance"],
+                ]
+            )
+
+    # Phase 4: passthrough the descriptors.index if present (FAISS file
+    # is independent of capture_date; copy bit-identical).
+    src_index = source_dir / "descriptors.index"
+    if src_index.is_file():
+        shutil.copyfile(src_index, output_dir / "descriptors.index")
+
+    return {
+        "row_count": len(rows),
+        "sidecar_count": sidecar_count,
+        "shifted_date": shifted,
+        "source_dir": str(source_dir),
+    }
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description="Age-inject the tile-cache fixture")
+    parser.add_argument(
+        "--source-dir",
+        type=Path,
+        required=True,
+        help="Path to the source tile-cache-fixture tree",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=Path,
+        required=True,
+        help="Path to the aged output tree",
+    )
+    parser.add_argument(
+        "--age-months",
+        type=int,
+        required=True,
+        help="Shift capture_date by this many months into the past",
+    )
+    args = parser.parse_args(argv)
+
+    logging.basicConfig(
+        level=logging.INFO,
+        format="%(asctime)s %(levelname)s %(name)s %(message)s",
+    )
+
+    summary = inject(args.source_dir, args.output_dir, args.age_months)
+    json.dump(summary, sys.stdout, sort_keys=True, indent=2)
+    sys.stdout.write("\n")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+# Clone the tile-cache fixture and emit `synth-age-7mo` + `synth-age-13mo`
+# Docker volumes (or local directories in ``--local`` mode).
+#
+# AC-3: dates shifted by 7 mo / 13 mo ±1 day; tile pixel content
+# bit-identical to the source.
+#
+# Env vars:
+#   TILE_CACHE_VOLUME_NAME  Source volume (default: tile-cache-fixture)
+#   AGE_7MO_VOLUME_NAME     Output volume for 7mo (default: synth-age-7mo)
+#   AGE_13MO_VOLUME_NAME    Output volume for 13mo (default: synth-age-13mo)
+#
+# Usage:
+#   inject.sh                                  # Docker mode
+#   inject.sh --local /src /out-7mo /out-13mo  # local mode (unit test path)
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+SOURCE_VOL="${TILE_CACHE_VOLUME_NAME:-tile-cache-fixture}"
+OUT_7MO_VOL="${AGE_7MO_VOLUME_NAME:-synth-age-7mo}"
+OUT_13MO_VOL="${AGE_13MO_VOLUME_NAME:-synth-age-13mo}"
+
+if [[ "${1:-}" == "--local" ]]; then
+    if [[ -z "${2:-}" || -z "${3:-}" || -z "${4:-}" ]]; then
+        echo "ERROR: --local requires <src_dir> <out_7mo_dir> <out_13mo_dir>" >&2
+        exit 2
+    fi
+    python3 "${SCRIPT_DIR}/age_injector.py" \
+        --source-dir "$2" --output-dir "$3" --age-months 7
+    python3 "${SCRIPT_DIR}/age_injector.py" \
+        --source-dir "$2" --output-dir "$4" --age-months 13
+    exit 0
+fi
+
+# Docker mode: reuse the tile-cache-builder image (it already has
+# Python + Pillow + numpy; the injector script is mounted in).
+IMAGE_TAG="azaion-tile-cache-builder:local"
+
+for spec in "${OUT_7MO_VOL}:7" "${OUT_13MO_VOL}:13"; do
+    target_vol="${spec%%:*}"
+    months="${spec##*:}"
+
+    docker volume rm "${target_vol}" >/dev/null 2>&1 || true
+    docker volume create "${target_vol}" >/dev/null
+
+    docker run --rm \
+        -v "${SCRIPT_DIR}:/opt/injector:ro" \
+        -v "${SOURCE_VOL}:/source:ro" \
+        -v "${target_vol}:/output" \
+        --entrypoint python3 \
+        "${IMAGE_TAG}" \
+        /opt/injector/age_injector.py \
+        --source-dir /source \
+        --output-dir /output \
+        --age-months "${months}"
+
+    echo "synth-age volume '${target_vol}' built (age=${months}mo)"
+done