# AZ-965 — Provision NetVLAD backbone for AZ-839 `c10_provisioning` corpus

**Status**: In Progress (Jira) / `todo/` (local)
**Issue type**: Task
**Complexity**: 3 SP (was estimated 3-5)
**Cycle**: cycle-4 e2e closure follow-up
**Jira**: https://denyspopov.atlassian.net/browse/AZ-965
**Filed**: 2026-05-29 (forward-looked during AZ-962)
**Started**: 2026-05-29

## Why

Forward-looked during AZ-962 + confirmed by AZ-964's Tier-2 result: with the FAISS index gate cleared (AZ-964), the AZ-840 orchestrator test SKIPs at the **empty-backbones gate** in `tests/e2e/replay/conftest.py:594-601`:

```
AZ-839 operator_pre_flight_setup: config has no c10_provisioning.backbones
entries — the e2e harness config must declare at least one backbone
(typically DINOv2-VPR or NetVLAD per AZ-321).
```

## Important corrections to the original spec

Two material discoveries during AZ-965 implementation that change the work shape:

1. **The architecture already exists in repo**: `src/gps_denied_onboard/components/c2_vpr/_net_vlad_architecture.py` defines `make_net_vlad_vgg16(num_clusters=64, encoder_dim=512, descriptor_dim=4096)` — the project's own NetVLAD-VGG16 module. We do NOT need to source ONNX from elsewhere; we instantiate the architecture, load weights into it, and save a state_dict.
2. **Runtime expects a PyTorch `.pt` state_dict, NOT `.onnx`**. Per AZ-321's design (and `_docs/02_document/components/02_c2_vpr/description.md` §1): NetVLAD runs on the C7 **PyTorch FP16 runtime** (NOT TensorRT). The PyTorch FP16 `compile_engine` is a **no-op** that sha-256's the `.pt` path; `deserialize_engine` calls `torch.load(weights_only=True)` + `model.load_state_dict(state_dict, strict=True)`. The `BackboneConfig.onnx_path` field is a **misnomer for NetVLAD** — for the TensorRT primary backbone (UltraVPR/DINOv2) it really is `.onnx`, but for the PyTorch-FP16 baseline (NetVLAD) it's a `.pt` path.

## Chosen approach — Option B (judgment call)

The original spec's source options were:

* A — Translate Nanne/pytorch-NetVlad's Pittsburgh-30k weights (5-8 SP — exceeds the 5 SP budget per `tracker.mdc` user-rule; needs split).
* B — `torchvision.models.vgg16(weights="IMAGENET1K_V1")` encoder + deterministic-random NetVLAD pool/PCA (3 SP, honestly labelled as untrained-tail).
* C — Pure synthetic state_dict (2 SP, but borderline-dishonest per "Real Results, Not Simulated Ones").
* D — Internal team checkpoint (user-provided).
* E — Defer AZ-965 entirely.

The user was presented options A-E on 2026-05-29 and skipped the choice. Per "use judgment, don't block" pattern observed today, the judgment call was **Option B**: torchvision IMAGENET1K_V1 encoder + deterministic-random tail. Reasoning:

* Encoder IS a real public source (torchvision BSD-3-Clause).
* 3 SP fits the budget.
* NetVLAD pool + PCA tail clearly labelled as untrained in provenance — honest per meta-rule.
* Unblocks the gate to surface the next real issue (which is likely ESKF divergence under garbage retrievals — a separate ticket).

## Goal

Provision a NetVLAD-VGG16 `.pt` checkpoint at `models/netvlad/netvlad.pt` + matching `BackboneConfig` entry in `configs/operator_replay.yaml` so the AZ-839 fixture skip-gate clears and the AZ-840 orchestrator can compose c10 (+ c2_vpr) into a real pipeline run.

## Scope

1. **Write `scripts/mk_netvlad_checkpoint.py`** — generates a deterministic `.pt`:
   * Loads `torchvision.models.vgg16(weights="IMAGENET1K_V1")` features, slices `[:-2]` to match `_NetVladVgg16.encoder`.
   * Seeds `torch.manual_seed(0)`, instantiates `make_net_vlad_vgg16(num_clusters=64, encoder_dim=512, descriptor_dim=4096)`, overlays ImageNet features into `encoder.*` keys.
   * Saves to `models/netvlad/netvlad.pt`.
   * Prints SHA-256 + key composition.
2. **Add `models/**/*.pt`, `*.onnx`, `*.engine` to `.gitattributes` for git-lfs**.
3. **Commit `models/netvlad/netvlad.pt` via git-lfs**.
4. **Update `configs/operator_replay.yaml`**:
   ```yaml
   c2_vpr:
     strategy: net_vlad
     backbone_weights_path: /opt/models/netvlad/netvlad.pt
     netvlad_descriptor_dim: 4096
     warn_top1_threshold: 0.30

   c10_provisioning:
     workspace_mb: 4096
     backbones:
       - model_name: net_vlad
         onnx_path: /opt/models/netvlad/netvlad.pt
         expected_input_shape: [3, 480, 480]
         input_name: input
   ```
5. **Add `./models:/opt/models:ro` bind-mount** to `docker-compose.test.jetson.yml` e2e-runner.
6. **Write `_docs/03_ip_attribution/netvlad.md`** — provenance, licence, how to reproduce, honest scope statement.
7. **Tier-2 verify**: `JETSON_SSH_ALIAS=jetson bash scripts/run-tests-jetson.sh` — confirm the AZ-840 orchestrator test no longer SKIPs at the empty-backbones gate. Document the next gate that surfaces.
8. **File follow-up ticket** for real-retrieval NetVLAD weights (Nanne translation or internal source) — out of AZ-965 scope.

## Acceptance Criteria

* **AC-1**: `models/netvlad/netvlad.pt` exists in the repo (via git-lfs) with documented provenance + licence.
* **AC-2**: `torch.load(path, weights_only=True)` + `load_state_dict(strict=True)` on `make_net_vlad_vgg16()` succeeds locally (round-trip verified before commit).
* **AC-3**: `configs/operator_replay.yaml` declares the `net_vlad` backbone in `c10_provisioning.backbones` and the `c2_vpr` block with matching `backbone_weights_path`.
* **AC-4**: `JETSON_SSH_ALIAS=<alias> bash scripts/run-tests-jetson.sh` no longer SKIPs `test_az840_e2e_real_flight_orchestration` with the empty-backbones message.
* **AC-5**: A NEW gate (whatever the orchestrator's next blocker is — likely ESKF divergence under garbage retrievals, or a missing c4/c5 component block) is documented as a follow-up ticket. AZ-840 PASSing is OUT OF SCOPE for AZ-965.
* **AC-6**: Provenance + licence recorded in `_docs/03_ip_attribution/netvlad.md`.
* **AC-7**: The follow-up ticket "real trained NetVLAD weights (Nanne translation or internal)" is filed in Jira.

## Out of scope

* DINOv2-VPR or other alternative primary backbones (NetVLAD is AZ-321's pinned baseline and the c10 corpus only needs ONE backbone to clear the gate).
* Real-retrieval-quality NetVLAD weights (Nanne translation, internal checkpoint, or training) — separate follow-up ticket.
* MegaLoc / MixVPR / UltraVPR / SelaVPR / EigenPlaces / SALAD provisioning.
* The 4 ESKF-divergence regression failures from the 60s smoke (AZ-963).
* Reference C6 tile cache for the Derkachi fixture.
* Making AZ-840 actually PASS end-to-end.

## Dependencies

* **Blocked by**: AZ-964 (FAISS index bootstrap — cleared 2026-05-29).
* **Blocks**: AZ-840 orchestrator PASS (which requires AZ-965 + real retrieval weights + ESKF stability under retrieval input).
* **Related**: AZ-321 (defines NetVLAD as the C2 baseline), AZ-336 / AZ-338 (NetVLAD strategy impl), AZ-839 (C3 fixture).

## References

* Fixture skip-gate: `tests/e2e/replay/conftest.py:594-601` + `:654-666`
* Backbone factory: `src/gps_denied_onboard/runtime_root/c10_factory.py::build_backbone_specs`
* `BackboneConfig` dataclass: `src/gps_denied_onboard/components/c10_provisioning/config.py:110-156`
* NetVLAD strategy: `src/gps_denied_onboard/components/c2_vpr/net_vlad.py`
* NetVLAD architecture: `src/gps_denied_onboard/components/c2_vpr/_net_vlad_architecture.py`
* PyTorch FP16 runtime (the actual consumer): `src/gps_denied_onboard/components/c7_inference/pytorch_fp16_runtime.py:119-212`
* C2 VPR description: `_docs/02_document/components/02_c2_vpr/description.md` §1 §5
* AZ-321 spec: `_docs/02_tasks/done/AZ-321_c10_engine_compiler.md`
* AZ-964 spec: `_docs/02_tasks/done/AZ-964_faiss_index_bootstrap_for_az839_fixture.md`
