[AZ-964] FAISS index bootstrap for AZ-839 fixture + build flag

AZ-964 SHIPPED — AZ-840 orchestrator test moves past FAISS gate.

Changes:
* tests/e2e/replay/_faiss_seed.py — extracts the empty HNSW32
  seeding logic from scripts/mk_test_faiss_fixture.py into a
  reusable test-infra module: seed_empty_faiss_index(root_dir,
  *, descriptor_dim=512, backbone_label="ultra_vpr") -> Path.
* scripts/mk_test_faiss_fixture.py rewritten as a thin CLI shim
  importing the same helper. compose `tile-init` contract is
  preserved.
* tests/e2e/replay/conftest.py::_build_operator_pre_flight_cache
  now calls seed_empty_faiss_index(cache_root) immediately before
  build_descriptor_index(config), so the factory's _load() finds
  a valid .index + .sha256 + .meta.json triplet at the fixture's
  override root_dir. populate_c6_from_route later in the fixture
  rebuilds the real index once route tiles are downloaded.
* docker-compose.test.jetson.yml: BUILD_PYTORCH_FP16_RUNTIME: "ON"
  added to e2e-runner.environment. Scope creep documented honestly
  in the spec — Tier-2 surfaced this third config gap on the same
  fixture chain while validating AZ-964 (RuntimeNotAvailableError:
  ... the flag is OFF). One-line wiring; the dustynv/l4t-pytorch
  base image bakes the Tegra-tuned PyTorch wheel and
  pytorch_fp16_runtime.py exists, so flag flip is sufficient.

Tier-2 verdict (4F / 48P / 3S / 1XF / 1XP in 86.07s, 0 errors —
was 2 errors before this commit): AZ-840 orchestrator test moves
from ERROR at FAISS gate to SKIP at empty-backbones gate — exactly
the AZ-965 gate AZ-964 AC-3 promised. test_operator_pre_flight_
integration SKIPs cleanly too. The 4 derkachi_1min ESKF-divergence
FAILs are constant across all three runs today (AZ-963 path,
independent of orchestrator chain).

Three Tier-2 runs today on the orchestrator chain:
  i.   pre-AZ-962: SKIP at env-var gate
  ii.  post-AZ-962: ERROR at FAISS gate
  iii. post-AZ-964: SKIP at backbones gate (AZ-965)

Cycle-4 e2e gate still NOT GREEN. Orchestrator chain remaining =
AZ-965 (NetVLAD backbone provisioning); 60s smoke chain remaining
= AZ-963 (ESKF divergence). OKVIS2 deferral directive unchanged.

Pre-existing yamllint false positive on docker-compose.test.jetson
.yml:185 (sibling `volumes:` keys flagged as duplicates without
respecting parent-key scope) — PyYAML parses cleanly with no
duplicates and docker-compose accepts the file at runtime.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Oleksandr Bezdieniezhnykh
2026-05-29 17:02:49 +03:00
parent 763d8b21ad
commit 288aae881d
7 changed files with 144 additions and 44 deletions
+22 -41
View File
@@ -2,7 +2,9 @@
"""Create a minimal valid FAISS HNSW32 + IndexIDMap2 fixture for the test harness.
Used by the `tile-init` init service in docker-compose.test.jetson.yml.
Writes three files to /var/lib/gps-denied/tiles/:
Writes three files to /var/lib/gps-denied/tiles/ via the shared
`tests.e2e.replay._faiss_seed.seed_empty_faiss_index` helper (AZ-964):
descriptor.index — empty HNSW32 dim=512 binary
descriptor.index.sha256 — sha256 sidecar (matches FaissDescriptorIndex._load)
descriptor.index.meta.json — metadata (descriptor_dim, hnsw_params.metric, ...)
@@ -12,50 +14,29 @@ Running this twice is idempotent (overwrites the previous fixture).
from __future__ import annotations
import hashlib
import json
from datetime import datetime, timezone
import sys
from pathlib import Path
import faiss # type: ignore[import-untyped]
# Make the repo root importable so `tests.e2e.replay._faiss_seed` resolves
# when this script runs in the `tile-init` compose service (which mounts
# the repo at /opt/project but doesn't add it to PYTHONPATH).
_REPO_ROOT = Path(__file__).resolve().parent.parent
if str(_REPO_ROOT) not in sys.path:
sys.path.insert(0, str(_REPO_ROOT))
DESCRIPTOR_DIM = 512
HNSW_M = 32
from tests.e2e.replay._faiss_seed import seed_empty_faiss_index # noqa: E402
root = Path("/var/lib/gps-denied/tiles")
root.mkdir(parents=True, exist_ok=True)
inner = faiss.IndexHNSWFlat(DESCRIPTOR_DIM, HNSW_M, faiss.METRIC_INNER_PRODUCT)
index = faiss.IndexIDMap2(inner)
def main() -> int:
idx_path = seed_empty_faiss_index(Path("/var/lib/gps-denied/tiles"))
sha256_path = idx_path.parent / (idx_path.name + ".sha256")
sha256 = sha256_path.read_text(encoding="ascii").strip()
print(
f"[tile-init] OK: empty HNSW32 index at {idx_path} "
f"sha256={sha256[:16]}..."
)
return 0
idx_path = root / "descriptor.index"
faiss.write_index(index, str(idx_path))
idx_bytes = idx_path.read_bytes()
sha256 = hashlib.sha256(idx_bytes).hexdigest()
(idx_path.parent / (idx_path.name + ".sha256")).write_text(sha256, encoding="ascii")
meta = {
"descriptor_dim": DESCRIPTOR_DIM,
"n_vectors": 0,
"backbone_label": "ultra_vpr",
"backbone_sha256_hex": "0" * 64,
"built_at": datetime.now(timezone.utc).isoformat(),
"hnsw_params": {
"m": HNSW_M,
"ef_construction": 40,
"ef_search": 16,
"metric": "INNER_PRODUCT",
},
"sidecar_sha256_hex": sha256,
"file_path": str(idx_path),
"id_mapping": [],
}
(idx_path.parent / (idx_path.name + ".meta.json")).write_text(
json.dumps(meta, sort_keys=True, indent=2), encoding="utf-8"
)
print(
f"[tile-init] OK: empty HNSW32 dim={DESCRIPTOR_DIM} index "
f"at {idx_path} sha256={sha256[:16]}..."
)
if __name__ == "__main__":
sys.exit(main())