Files
gps-denied-onboard/e2e/fixtures/age-injector/age_injector.py
T
Oleksandr Bezdieniezhnykh 6599d828d2 [AZ-407] [AZ-444] [AZ-445] Batch 68: fixtures, Tier-2 harness, NFR reporter
Three blackbox-harness tasks landed together — all depend only on
AZ-406 and unblock the FT-* / NFT-* scenario tasks scheduled for
batches 69+.

AZ-407 — Static fixture builders (3pt):
  * tile-cache-builder/{builder.py, Dockerfile, build.sh} produces a
    deterministic tile-cache-fixture Docker volume from
    _docs/00_problem/input_data/. Reproducibility primitives: sorted
    iteration, frozen PIL JPEG settings, FAISS HNSW32 built single-
    threaded with seeded stub descriptors.
  * age-injector/{age_injector.py, inject.sh} clones the volume and
    shifts capture_date by N×30.44 days; tile JPEG bytes preserved
    bit-identical. Emits synth-age-7mo + synth-age-13mo volumes.
  * cold-boot/cold_boot_fixture.json: frozen FC pose snapshot at
    Derkachi sector centre, schema v1.
  * secrets/mavlink-test-passkey.txt: 64-hex with required
    `# TEST ONLY` header line per AC-5. Passkey-equality test now
    compares the secret line after stripping the header.
  * security/cve-2025-53644.jpg: synthetic 158-byte malformed JPEG
    (truncated SOS marker). OpenCV 4.11.x rejects gracefully with
    imdecode → None. AZ-439 will sharpen for ASan instrumentation.
  * Top-level Makefile with `make fixtures` / `make fixtures-*` /
    `make e2e-tier1*` / `make unit-tests` targets.

AZ-444 — Tier-2 Jetson harness wrapper (5pt):
  * run-tier2.sh rewritten as orchestrator. Detects local
    (aarch64 + TIER2_HOST=localhost) vs remote (ssh into TIER2_HOST).
    New flags: -k/--selector, --build-kind production|asan,
    --reflash (gated behind TIER2_REFLASH_ACK=1 two-key gate),
    --dry-run.
  * tier2-on-jetson.sh (new) — on-device delegate. Verifies
    gps-denied-onboard{,-asan}.service health; restarts with 5s
    tolerance; spawns tegrastats + jtop parallel samplers; tails
    ASan unit's journal in asan mode; drives docker compose with
    TIER=tier2-jetson; forwards SELECTOR to pytest -k.
  * docker/run-tier1.sh (new) — selector-parity sibling.
  * AC-1 (selector parity) and AC-6 (reflash gating) unit-tested via
    --dry-run output assertions. AC-2/AC-3/AC-4/AC-5 are hardware-
    loop ACs verified by the Tier-2 runtime smoke (no Jetson in the
    unit-test layer).

AZ-445 — CSV reporter + evidence bundler refinements (2pt):
  * reporting/nfr_recorder.py (new) — pytest plugin. Provides the
    `nfr_recorder` fixture with record_metric(name, value, ac_id)
    and partial(ac_id, reason). At session end emits:
      - per-nfr/<scenario_id>.json (AC-1)
      - traceability-status.json with every AC ID parsed from
        traceability-matrix.md, classified Covered/PARTIAL/NOT
        COVERED with source scenario IDs (AC-2)
      - regression-baseline.json with all numeric metrics (AC-3)
  * csv_reporter.py extended — `_outcome_to_result` consults the
    aggregator; rows flip PASS → PARTIAL when an AC was marked
    PARTIAL by nfr_recorder (AC-4). Graceful fallback when
    aggregator isn't registered (unit-test contexts).
  * conftest.py registers nfr_recorder in pytest_plugins.
  * New --traceability-matrix CLI flag seeds the NOT COVERED rows.

Build / config:
  * pyproject.toml dev extras: added Pillow>=10.4,<13.0 for the
    tile-cache-builder unit test (broad enough to keep torchvision's
    Pillow 12 pin happy; the production builder runs inside its own
    Docker image with its own pin).
  * Updated test_directory_layout.py to cover 10 new files + replaced
    the byte-equal passkey assertion with the header-stripping
    variant.

Test results:
  * 157 focused tests pass (was 97 in batch 67; +60 new across this
    batch). No regressions.

Module-layout / spec drift:
  * AZ-407 spec text says `tests/fixtures/...`; module-layout
    blackbox_tests entry (commit d7a17a8) authoritatively places the
    harness under `e2e/`. Implementation followed the layout entry.
  * AZ-444 spec mentions `e2e/tier2/run-tier2.sh`; AZ-406 placed it
    at `e2e/jetson/run-tier2.sh`. Kept at `e2e/jetson/` for
    consistency.
  * Cold-boot README ownership: corrected from AZ-419 to AZ-407 per
    AZ-419's own Dependencies field.

Specs archived to _docs/02_tasks/done/. Jira tickets transitioned to
In Testing on commit.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-16 17:18:01 +03:00

178 lines
5.3 KiB
Python

"""Age-injector for the tile-cache fixture.
Clones a ``tile-cache-fixture`` tree and mutates ONLY the manifest's
``capture_date`` column (and the per-tile sidecar JSON's matching field).
Tile JPEG bodies are copied bit-identical.
AC-3 (AZ-407): given target=7mo, every row's ``capture_date`` becomes
``now - 7 mo`` ± 1 day, exceeding the AC-8.2 active-conflict 6-month
threshold. Given target=13mo, every row's ``capture_date`` becomes
``now - 13 mo`` ± 1 day, exceeding the rear 12-month threshold.
Used by FT-N-05 / FT-N-06 (stale-tile rejection on freshness violation).
Public-boundary discipline: this module does NOT import any
``src/gps_denied_onboard`` symbol. The freshness contract lives in
``_docs/00_problem/restrictions.md`` § Satellite Imagery (AC-8.2).
"""
from __future__ import annotations
import argparse
import csv
import datetime as _dt
import json
import logging
import shutil
import sys
from pathlib import Path
logger = logging.getLogger(__name__)
# 30.44 days/month average — gives `now - N*30 days ± 1 day`, which the
# AC's "±1 day" tolerance accepts.
_DAYS_PER_MONTH = 30.44
_MANIFEST_HEADERS = (
"zoom_level",
"tile_x",
"tile_y",
"capture_date",
"source",
"m_per_px",
"jpeg_path",
"content_hash",
"provenance",
)
def _shifted_date(now: _dt.date, age_months: int) -> str:
delta_days = int(round(age_months * _DAYS_PER_MONTH))
return (now - _dt.timedelta(days=delta_days)).isoformat()
def inject(
source_dir: Path,
output_dir: Path,
age_months: int,
now: _dt.date | None = None,
) -> dict:
"""Clone ``source_dir`` into ``output_dir`` and mutate dates.
Returns a summary dict:
{"row_count": int, "shifted_date": "YYYY-MM-DD", "source_dir": str}
"""
if age_months <= 0:
raise ValueError(f"age_months must be positive; got {age_months}")
if now is None:
now = _dt.datetime.now(tz=_dt.timezone.utc).date()
if output_dir.exists():
shutil.rmtree(output_dir)
output_dir.mkdir(parents=True)
# Phase 1: clone the tile tree. Pixels copy bit-identical.
src_tiles = source_dir / "tiles"
if not src_tiles.is_dir():
raise FileNotFoundError(
f"{source_dir} does not look like a tile-cache fixture "
"(no `tiles/` subdir)"
)
shutil.copytree(src_tiles, output_dir / "tiles")
shifted = _shifted_date(now, age_months)
# Phase 2: mutate per-tile sidecar JSON files.
sidecar_count = 0
for sidecar in sorted((output_dir / "tiles").rglob("*.json")):
data = json.loads(sidecar.read_text())
data["capture_date"] = shifted
sidecar.write_text(
json.dumps(data, sort_keys=True, separators=(",", ":")) + "\n"
)
sidecar_count += 1
# Phase 3: re-emit manifest.csv with shifted dates. Row order is
# preserved (the source manifest is already sorted by builder.py).
src_manifest = source_dir / "manifest.csv"
if not src_manifest.is_file():
raise FileNotFoundError(f"missing manifest.csv at {src_manifest}")
with src_manifest.open() as fp:
reader = csv.DictReader(fp)
if tuple(reader.fieldnames or ()) != _MANIFEST_HEADERS:
raise ValueError(
f"unexpected manifest schema: {reader.fieldnames} "
f"(expected {list(_MANIFEST_HEADERS)})"
)
rows = list(reader)
out_manifest = output_dir / "manifest.csv"
with out_manifest.open("w", newline="") as fp:
writer = csv.writer(fp, lineterminator="\n")
writer.writerow(_MANIFEST_HEADERS)
for r in rows:
writer.writerow(
[
r["zoom_level"],
r["tile_x"],
r["tile_y"],
shifted,
r["source"],
r["m_per_px"],
r["jpeg_path"],
r["content_hash"],
r["provenance"],
]
)
# Phase 4: passthrough the descriptors.index if present (FAISS file
# is independent of capture_date; copy bit-identical).
src_index = source_dir / "descriptors.index"
if src_index.is_file():
shutil.copyfile(src_index, output_dir / "descriptors.index")
return {
"row_count": len(rows),
"sidecar_count": sidecar_count,
"shifted_date": shifted,
"source_dir": str(source_dir),
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Age-inject the tile-cache fixture")
parser.add_argument(
"--source-dir",
type=Path,
required=True,
help="Path to the source tile-cache-fixture tree",
)
parser.add_argument(
"--output-dir",
type=Path,
required=True,
help="Path to the aged output tree",
)
parser.add_argument(
"--age-months",
type=int,
required=True,
help="Shift capture_date by this many months into the past",
)
args = parser.parse_args(argv)
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s %(message)s",
)
summary = inject(args.source_dir, args.output_dir, args.age_months)
json.dump(summary, sys.stdout, sort_keys=True, indent=2)
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())