# Contract: DescriptorIndex Protocol

**Component**: c6_tile_cache
**Producer task**: AZ-303 — `_docs/02_tasks/todo/AZ-303_c6_storage_interfaces.md`
**Consumer tasks**:
- AZ-TBD-c6-faiss-descriptor-index (implements: FAISS HNSW)
- TBD at decompose time: E-C2 (AZ-255 — sole runtime consumer; per-frame top-K=10 retrieval), E-C10 (AZ-252 — F1 pre-flight index build via `rebuild_from_descriptors`)
**Version**: 1.0.0
**Status**: draft
**Last Updated**: 2026-05-10

## Purpose

Defines the typed boundary to the per-flight descriptor retrieval index. C2 VPR queries the index per frame at 3 Hz (top-K=10) to nominate candidate tiles for C2.5 ReRanker. The concrete impl is FAISS HNSW (`FaissDescriptorIndex`), but consumers depend only on this Protocol so a future swap (e.g., ScaNN, custom index) does not ripple. C10 CacheProvisioner (`AZ-252`) is the F1 pre-flight write-side caller — it builds the `.index` file once per provisioning; in flight the index is read-only mmap.

## Shape

### Protocol surface

`typing.Protocol` (PEP 544) with `runtime_checkable=True`. All methods are sync; the index is held in memory-mapped form.

| Method | Signature | Throws / Errors | Blocking? |
|--------|-----------|-----------------|-----------|
| `search_topk` | `(query: np.ndarray, k: int) -> list[tuple[TileId, float]]` | `IndexUnavailableError` | sync (HNSW; ≤ 5 ms p95 warm; first call ≤ 1 s cold for mmap page-in) |
| `descriptor_dim` | `() -> int` | — | sync; constant-time |
| `mmap_handle` | `() -> Path` | `IndexUnavailableError` | sync; returns the `.index` file path (consumers needing custom mmap-aware tooling — e.g., operator post-flight inspection — call this) |
| `rebuild_from_descriptors` | `(descriptors: np.ndarray, tile_ids: list[TileId], hnsw_params: HnswParams) -> None` | `IndexBuildError`, `TileFsError` | sync (offline; minutes for a full-area corpus). Atomic file replacement via the AZ-280 sidecar pattern. |
| `index_metadata` | `() -> IndexMetadata` | `IndexUnavailableError` | sync; reads the sidecar metadata block |

### DTOs

```python
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Optional


@dataclass(frozen=True)
class HnswParams:
    """HNSW build hyperparameters. See description.md § 5; defaults from the
    FAISS team's HNSW32+M=32 / efConstruction=200 / efSearch=64 baseline."""
    m: int = 32                    # # of connections per node
    ef_construction: int = 200     # build-time candidate list size
    ef_search: int = 64            # query-time candidate list size
    metric: str = "L2"             # "L2" | "INNER_PRODUCT"


@dataclass(frozen=True)
class IndexMetadata:
    descriptor_dim: int                         # dimension of the indexed vectors
    n_vectors: int                              # number of indexed tiles
    backbone_label: str                         # producer backbone — e.g. "ultra_vpr_v0"
    backbone_sha256_hex: str                    # producer backbone weights hash (D-C10-3 chain)
    built_at: datetime                          # ISO 8601 UTC
    hnsw_params: HnswParams
    sidecar_sha256_hex: str                     # canonical content hash of the .index file
    file_path: Path                             # absolute path to the .index file
```

### Numpy contract

- `query`: shape `(descriptor_dim,)`, dtype `float32`, C-contiguous. The Protocol does NOT auto-pad batches; per-frame is a single query (C2's per-frame call site).
- `descriptors`: shape `(N, descriptor_dim)`, dtype `float32`, C-contiguous; `N == len(tile_ids)`. The Protocol does NOT validate shape mismatch — the impl raises `IndexBuildError` on dtype/shape violation.

### Errors

```
TileCacheError (shared with TileStore / TileMetadataStore)
└── IndexUnavailableError      # mmap handle invalid, file missing, or sidecar mismatched

IndexBuildError                # raised only by rebuild_from_descriptors; NOT in the read-side envelope
```

`IndexBuildError` is intentionally NOT a subclass of `TileCacheError` — the build path is offline, lives in C10's pre-flight provisioning, and has different fault semantics than the runtime-read path. C2 (the only runtime consumer) catches `IndexUnavailableError`; C10 catches `IndexBuildError`.

## Invariants

- **I-1 (immutable in flight):** once an `.index` file is opened via the impl's loader, the file's content MUST NOT change for the lifetime of the impl instance. F1 pre-flight is the only legal write path; a mid-flight rebuild is forbidden (the impl raises `IndexUnavailableError` if it detects a content-hash mismatch on a periodic sidecar re-check — out-of-band tampering signal).
- **I-2 (top-K is best-effort):** `search_topk(query, k=K)` MAY return fewer than K results when the corpus has fewer than K vectors. Consumers (C2) tolerate fewer-than-K results.
- **I-3 (descriptor-dim is fixed at build):** `descriptor_dim()` returns the value baked into the `.index` file at build time; if a consumer's query vector dimension does not match, the impl raises `IndexUnavailableError` (NOT a separate `DimensionMismatchError` — keeps the read-side envelope to a single error type).
- **I-4 (no GPU resident memory):** the impl MUST hold the index in CPU mmap'd memory only. FAISS GPU index variants are explicitly excluded — the F3 hot path's GPU is reserved for `c7_inference` engines (per NFT-LIM-01 / D-CROSS-LATENCY-1).
- **I-5 (atomic rebuild):** `rebuild_from_descriptors` MUST write to a temporary path, sync to disk, atomically rename to the target path, write the sidecar `.sha256`, and only then return. A crash mid-rebuild leaves the prior index intact.
- **I-6 (sidecar coherence):** `mmap_handle()` returns a path whose `.sha256` sidecar matches the file's actual content hash; if the sidecar is missing or mismatched, `IndexUnavailableError` is raised on the FIRST `search_topk` of the flight (not lazily on the read that hits the corrupted region). C10's pre-flight gate is the canonical place this is validated; this Protocol carries the runtime-side check as defence-in-depth.
- **I-7 (frozen DTOs):** `HnswParams`, `IndexMetadata` are `@dataclass(frozen=True)`.
- **I-8 (single-thread search):** `search_topk` is NOT re-entrant; the F3 hot path is single-threaded per the description.md assumption. Future multi-threaded callers MUST use a per-thread impl instance (out of scope this cycle).

## Non-Goals

- **Not covered: tile pixel I/O.** That's `TileStore`.
- **Not covered: tile metadata bbox queries.** That's `TileMetadataStore`.
- **Not covered: incremental updates / online learning.** F1 pre-flight is full-rebuild only. Future task if needed.
- **Not covered: GPU FAISS variants.** I-4 forbids them this cycle.
- **Not covered: cross-flight index sharing.** Each flight provisions its own per-area `.index`; cross-flight is a parent-suite concern (D-PROJ-2).
- **Not covered: descriptor compression / PQ quantisation.** HNSW32 raw float32 is the only supported variant this cycle. Future task if AC-8.3 (10 GB cap) becomes binding.
- **Not covered: backbone retraining.** This Protocol is consumer-facing; the producer side (C10's compile of an UltraVPR engine) lives in E-C7 / E-C10.

## Versioning Rules

Same rules as `tile_store.md` § Versioning Rules. Note that `IndexMetadata.backbone_sha256_hex` ties this contract's lifecycle to the C7 engine cache (AZ-298 / AZ-301 / AZ-281): a backbone weights bump invalidates every prior `.index` AND requires a coordinated update — recorded as a major version of THIS contract only when the field's shape changes; backbone-weight refreshes within the existing schema are non-breaking content updates handled by C10.

## Test Cases

| Case | Input | Expected | Notes |
|------|-------|----------|-------|
| protocol-conformance-full | A class implementing all 5 methods | `isinstance(impl, DescriptorIndex) == True` | Producer AC-1 |
| protocol-conformance-partial | A class missing `index_metadata` | `isinstance == False` | CI drift gate |
| search-topk-warm | Query vector of correct dim against a 10k-vector index, OS page cache warm | Returns `[(tile_id, distance), ...]` length ≤ k; p95 ≤ 5 ms | I-2 / Consumer C2-PT-01 |
| search-topk-fewer-than-k | k=20 against a 10-vector index | Returns 10 results, ordered by distance ascending | I-2 |
| search-topk-dim-mismatch | Query vector of wrong dim | `IndexUnavailableError` | I-3 |
| search-topk-corrupted-sidecar | Index file present, sidecar missing | First `search_topk` raises `IndexUnavailableError`; subsequent calls also raise (no silent recovery) | I-6 |
| descriptor-dim | After a rebuild with `descriptors.shape == (N, 512)` | `descriptor_dim() == 512` | I-3 |
| rebuild-atomic-on-crash | Simulated `os._exit` mid-rebuild | The original `.index` file is intact and still loadable; partial temp file is cleaned up at next start | I-5 |
| rebuild-sidecar-content-hash | Successful rebuild | `.sha256` sidecar matches `sha256(.index)` | I-6 / AZ-280 contract |
| index-metadata | After rebuild | Returns `IndexMetadata` with matching `descriptor_dim`, `n_vectors`, `built_at` (within 1 s of call), `hnsw_params` (mirrors input), `sidecar_sha256_hex` (matches sidecar content) | I-7 |
| frozen-dto-mutation | `HnswParams(m=32, ...).m = 64` | `FrozenInstanceError` | I-7 |

## Change Log

| Version | Date | Change | Author |
|---------|------|--------|--------|
| 1.0.0 | 2026-05-10 | Initial contract — 5-method Protocol + HNSW params DTO + IndexMetadata sidecar shape + immutable-in-flight + atomic-rebuild invariants. | autodev (decompose Step 2 of AZ-250 / E-C6) |
