# Component Fit Matrix — C7: On-Jetson inference runtime

> Mode A Phase 2 — engine Step 7.5 (Component Applicability Gate, structured per-component candidate-selection table). Status vocabulary in [`00_summary.md`](00_summary.md). Backing fact cards: [`../02_fact_cards/C7_inference_runtime.md`](../02_fact_cards/C7_inference_runtime.md). Backing sources: [`../01_source_registry/C7_inference_runtime.md`](../01_source_registry/C7_inference_runtime.md).
>
> Index: [`00_summary.md`](00_summary.md). Sibling components: [C1 VIO](C1_vio.md), [C2 VPR](C2_vpr.md), [C3 Matchers](C3_matchers.md), [C4 Pose](C4_pose_estimation.md), [C5 State estimator](C5_state_estimator.md), [C6 Tile cache](C6_tile_cache_spatial_index.md), [C8 FC adapter](C8_fc_adapter.md), [C10 Pre-flight provisioning](C10_preflight_provisioning.md). C9 dropped per 2026-05-08 restructure. Cross-component gates: [`99_cross_component_gates.md`](99_cross_component_gates.md).

---

## C7 — On-Jetson inference runtime

**Status**: **CLOSED at 3/N (batch 1 closed 2026-05-08)** at documentary level. **TensorRT native (Cand 1) RECOMMENDED PRIMARY** per the user-pinned scope (`c7_breadth=B`, `c7_quantization=A`, `c7_overkill_options=A` locked via `/autodev` AskQuestion 2026-05-08). ONNX Runtime + TensorRT EP (Cand 2) RECOMMENDED ALTERNATE for cross-architecture-portability axis. Pure PyTorch FP16 (Cand 3) RECOMMENDED MANDATORY SIMPLE-BASELINE per engine Component Option Breadth rule. C7 is a **cross-cutting integration row** — its candidates are runtime backends, not domain components, and they are gated by C2 VPR (D-C2-5 DINOv2 ViT-export) + C3 matcher (D-C3-2 LightGlue runtime) + C1 VIO (if learned-frontend selected) precision policies.

**Pinned mode** (per-frame inference contract for C2 VPR backbone + C3 matcher + optional learned VIO frontend on the project's Jetson Orin Nano Super):

- inputs: `(camera_frame: numpy.ndarray of shape (H, W, 3) and dtype uint8 from C0 nav-camera @ 3 fps; preprocessing pipeline = resize-to-model-input + normalize + transfer-to-CUDA)` for VPR + `(query_descriptors: torch.Tensor / TRT tensor of shape (N_kp, d_desc); reference_descriptors: same shape)` for matcher + `(query_keypoints, query_descriptors, reference_keypoints, reference_descriptors)` for LightGlue/DISK/XFeat-class matchers
- outputs: `{vpr_descriptor: numpy.ndarray of shape (d_vpr,) at 3 Hz from C2 to C6 cache lookup; matcher_correspondences: list of inlier (kp_q, kp_r, match_score) at 3 Hz from C3 to C4 PnP solver}` on `Jetson Orin Nano Super (8 GB shared, JetPack 6.2 Ubuntu 22.04 base, Super Mode enabled, TensorRT 10.3 + CUDA 12.6 + cuDNN 9.3)` at flight time
- runtime: TensorRT 10.3 (primary) + onnxruntime-gpu 1.23+ via Jetson AI Lab (alternate) + PyTorch 2.5+ via Jetson AI Lab (mandatory simple-baseline) on Jetson Orin Nano Super in Super Mode (per Source #104 — 70% AI TOPS increase + 50% memory bandwidth boost vs base mode)

**Locked-in research-time defaults** (carried forward from C1 + C2 + C3 + C4 + C5 + C6 — Fact #41 + user-pinned scope from `/autodev` AskQuestion 2026-05-08):
- D-C1-1 = (c) **keep both license tracks open** through Plan; final license decision deferred to post-Jetson-MVE.
- D-C1-2 = (b) **defer Jetson Orin Nano Super hardware MVE to a dedicated bring-up phase** between research and Plan; research closes with documentary ranking + per-candidate `Verify` gates.
- **C7-specific user-pinned scope** (from `/autodev` AskQuestion 2026-05-08):
  - **`c7_breadth = B`** — top-2 documentary leads only (TensorRT native primary + ONNX Runtime+TRT EP interop alternate); pure PyTorch FP16 mandatory simple-baseline; no separate evaluation of NVIDIA Triton / DeepStream / CUDA-Python custom kernels.
  - **`c7_quantization = A`** — INT8 primary + FP16 fallback per candidate; INT8-only candidates marked Experimental until calibration data exists.
  - **`c7_overkill_options = A`** — Triton / DeepStream / CUDA-Python custom kernels noted-and-rejected in one sentence; no separate rows.

**Interactions with prior C-row closures**:
- **C1 VIO row**: if a learned-frontend candidate is selected (currently OKVIS2 + VINS-Mono are pure-classical; DROID-SLAM + DPVO are pruned for memory; pure-VO baseline is classical), C7 hosts the learned-frontend's network. Currently no learned-VIO candidate is in the active Selected pool, so C7 hosts only C2 + C3 networks. **Implication for C7**: smaller per-frame model footprint than a pipeline that includes a learned VIO frontend; eases AC-4.2 memory budget.
- **C2 VPR row**: C7 hosts the VPR backbone — MixVPR (CNN-ResNet50 + MLP-Mixer), SelaVPR (DINOv2-L two-stage, ViT-class), NetVLAD (CNN-VGG16), EigenPlaces (CNN-ResNet50). **D-C2-5 DINOv2 ViT-export to TensorRT FP16/INT8 path on Jetson Orin Nano Super** is the primary gate this row inherits — closed by Cand 1 + Cand 2 if FP16 path verified; INT8 path for ViT models is deferred to D-C7-6.
- **C3 matcher row**: C7 hosts the matcher — DISK+LightGlue (recommended primary), ALIKED+LightGlue (secondary), XFeat (alternate-modern-lead), SP+LightGlue (mandatory simple-baseline). **D-C3-2 LightGlue inference runtime path** rolls into this row's Cand 1 / Cand 2 selection — both candidates verified-compatible with LightGlue ONNX export per Source #103 (Fabio Sim's `fabio-sim/LightGlue-ONNX` is the canonical export pathway). XFeat sparse / semi-dense / +LighterGlue per D-C3-6 also export to ONNX → both Cand 1 and Cand 2 host them.
- **C4 pose-estimation row**: C7 does NOT host C4 candidates (OpenCV / OpenGV / GTSAM are CPU-only or CPU-+-Eigen; no inference network). **No coupling**.
- **C5 state-estimator row**: C7 does NOT host C5 candidates (Manual ESKF + GTSAM iSAM2 are CPU-only). **No coupling**.
- **C6 tile-cache row**: C7 + C6 share the 8 GB Jetson memory budget at flight time; C6 Cand 1 (mirror-suite-pattern) consumes ~700 MB-1.5 GB combined; C7 candidates each consume ~1-2 GB peak combined for VPR-engine + matcher-engine + executor context. Total system memory at flight: ~1.7-3.5 GB out of 8 GB shared budget — well within AC-4.2.
- **C10 pre-flight cache provisioning row (when opened)**: C10 is the canonical home for the TensorRT engine-build pipeline (per D-C7-7 = build-on-deployed-Jetson at pre-flight). D-C7-1 calibration-dataset-strategy is closed at C7 batch 1 (per the 2026-05-08 C9 / SQ7 restructure); the calibration corpus is assembled at pre-flight by C10 from the fixture-file pin Test Spec (Step 5) supplies (real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles, per C7 batch 1's locked strategy).
- **Cross-suite consistency**: parent-suite `satellite-provider` is .NET 8.0 + Postgres + Dapper — no on-Jetson inference runtime equivalent exists in the suite to mirror. C7 is greenfield-on-Jetson per project; no cross-suite cascade decision (unlike D-C6-7).

| # | Candidate | License | Per-mode verification | Status | Lead reason / disqualifier | Sub-matrix cite |
|---|---|---|---|---|---|---|
| 1 | **TensorRT native primary** — JetPack 6.2 bundled TensorRT 10.3 SDK + `IInt8EntropyCalibrator2` for INT8 calibration + `BuilderFlag.FP16` + `BuilderFlag.INT8` mixed-precision build flow + `IExecutionContext.execute_v2` per-frame inference; engines built directly on Jetson Orin Nano Super (SM 87 Ampere class) via JetPack-bundled `trtexec` or Python `tensorrt` API; no external pip dependency | **Clean throughout** — TensorRT (NVIDIA proprietary EULA but **canonical inference SDK with explicit deployment grant on NVIDIA hardware** = compatible with project's hardware-pinned Jetson Orin Nano Super) + JetPack 6.2 (NVIDIA proprietary EULA) + CUDA 12.6 (NVIDIA proprietary EULA) + cuDNN 9.3 (NVIDIA proprietary EULA); **no external open-source license-track interaction** — the entire stack is NVIDIA's; eligible on every D-C1-1 license-posture choice (a / b / c) since no GPL/BSD-classification applies | ✅ Mode enumeration via Source #99 NVIDIA TensorRT 10.x official documentation context7 lookup at `/websites/nvidia_deeplearning_tensorrt` (9371 code snippets, Source Reputation High, Benchmark Score 75.25) — **all four query modes verified**: INT8 calibrator hierarchy (`IInt8Calibrator` / `IInt8EntropyCalibrator2` / `IInt8MinMaxCalibrator`), Python builder INT8 enable pattern (`config.set_flag(trt.BuilderFlag.INT8)` + `config.int8_calibrator = Int8_calibrator`), mixed-precision flag pattern (`BuilderFlag.FP16` + `BuilderFlag.INT8` combined), per-execution-context inference path (`IExecutionContext.execute_v2`); ✅ runnable example via canonical `trtexec` CLI + Python `tensorrt` builder/runtime; ✅ **NATIVE hardware-aware tactic search** at engine build for SM 87 specifically (selects the fastest kernel implementation per layer); ✅ **NATIVE INT8/FP16 mixed-precision auto-selection** via calibrator-driven sensitivity-based fallback to FP16 for layers that fail INT8 calibration tolerance; ⚠️ **engines are hardware-specific** — must be built directly on the Jetson target (Source #105 #2 + #3); ⚠️ **build-time memory pressure on 8 GB shared budget can segfault during tactic profiling** (Source #105 #4) — mitigated via `config.max_workspace_size = 1 GB` per D-C7-8; ⚠️ **`pip install tensorrt` not supported on Jetson Tegra** (Source #105 #1) — canonical install is JetPack-bundled, no mitigation needed since project is JetPack-based; ⚠️ **feature-matching INT8 quantization-sensitivity** (Source #103 — LightGlue FP8 caused "match counts dropped sometimes hard"; INT8 structurally similar) — mitigated via D-C7-6 per-model-family precision policy (matchers FP16-only, VPR INT8+FP16 fallback) | **Mandatory primary + RECOMMENDED PRIMARY** (engine Component Option Breadth rule role — structurally analogous to OpenCV `solvePnPRansac`'s role in C4 row + NetVLAD's role in C2 row + Manual ESKF's role in C5 row + Cand 1 mirror-suite-pattern's role in C6 row, but shifted to NVIDIA's canonical inference SDK rather than to a permissive-license open-source library because **the C7 row is hardware-pinned to NVIDIA Jetson and TensorRT is the canonical NVIDIA inference SDK**); deployment-ready under every D-C1-1 license-posture choice (NVIDIA proprietary stack does not interact with the open-source license-track decision); **EMPIRICALLY-CONFIRMED** Jetson Orin Nano Super viability via Source #102 Ultralytics April 2026 YOLO26n benchmarks (FP32 7.53 ms / FP16 4.57 ms / INT8 3.80 ms); **EXACT match to project hardware** (TensorRT 10.3 ships bundled in JetPack 6.2 for Jetson Orin Nano Super in Super Mode); final ranking confirmed at Jetson MVE phase per D-C1-2 + D-C7-1 + D-C7-6 | **THREE CONVERGING POSITIVE structural advantages**: (i) **NATIVE NVIDIA STACK — fastest possible inference path** — TensorRT directly maps ONNX graph operations to fused CUDA kernels with hardware-aware scheduling on Ampere SM 87. Per Source #102 benchmarks, TensorRT FP16 is **1.65× faster than TensorRT FP32** and **~2× faster than pure-PyTorch FP16** at the YOLO26n class workload; this gap widens with larger models and is roughly preserved across CNN architectures. The project's tight AC-4.1 400 ms p95 budget combined with C3 matcher K=10 pairs/frame demands the fastest available runtime; (ii) **MIXED-PRECISION PER-LAYER AUTO-SELECTION at INT8 build time** — sensitivity-based fallback to FP16 for layers that fail INT8 calibration tolerance (configured via `config.set_flag(trt.BuilderFlag.OBEY_PRECISION_CONSTRAINTS)` + per-layer `setOutputType` + `setPrecision`). This auto-mitigates Source #103's concern about feature-matching networks suffering INT8 degradation: TensorRT's calibrator can keep the matcher's attention layers at FP16 while quantizing convolutional preprocessing — addressing the matcher-INT8-sensitivity caveat at the runtime layer; (iii) **JETPACK-BUNDLED — ZERO INSTALL FRICTION** — TensorRT 10.3 ships pre-installed with JetPack 6.2 (Source #105 + Source #104); no external pip dependency, no version-mismatch failure modes, no community wheel index dependency. Plan-phase Jetson MVE per D-C1-2 verifies the bundled TensorRT runs at the documented latency on the deployed Jetson. **THREE NEGATIVE-BUT-MITIGABLE structural findings**: (iv) **engines are hardware-specific** — must be rebuilt on the Jetson target per Source #105 #2 + #3 (mitigated: engine build is part of pre-flight cache provisioning per D-C7-7; engines persisted via `IRuntime.deserializeCudaEngine` from disk; build cost ~30-300 sec amortized across flights); (v) **INT8 calibration requires representative dataset** ~500-1,500 input samples covering deployment distribution (mitigated: D-C7-1 closed at C7 batch 1 with strategy = real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles, per the 2026-05-08 C9 / SQ7 restructure; specific fixture-file pin delegated to Test Spec (greenfield Step 5); candidate corpora carried forward to Test Spec: AerialVL S03 + AerialExtreMatch + project's own Mavic + Derkachi flight footage); (vi) **build-time memory pressure on 8 GB shared budget can segfault during tactic profiling** per Source #105 #4 (mitigated: cap `config.max_workspace_size` at 1 GB per D-C7-8; build during pre-flight when no other workloads are active). **TWO MINOR CAVEATS**: (vii) **no per-mode pip install path** — requires JetPack-bundled TensorRT (mitigated: project's deployment is JetPack-based by hardware constraint; no alternative install path is needed); (viii) **feature-matching INT8 quantization-sensitivity** (Source #103: FP8 LightGlue match-count drops; INT8 structurally similar) — mitigated via D-C7-6 per-model-family precision policy. **AC-NEW-4 covariance honesty handling**: N/A (runtime is passive; covariance is C4/C5 responsibility). **AC-NEW-7 cache-poisoning safety**: N/A (runtime does not write to the tile cache) | Fact #94 in [`../02_fact_cards/C7_inference_runtime.md`](../02_fact_cards/C7_inference_runtime.md) (per-mode entry + per-numbered-Restriction × per-numbered-AC sub-matrix block) |
| 2 | **ONNX Runtime + TensorRT EP interop alternate** — onnxruntime-gpu 1.23+ via Jetson AI Lab community wheel index `pypi.jetson-ai-lab.io/jp6/cu126` + `TensorrtExecutionProvider` config dict (`trt_fp16_enable=True`, `trt_int8_enable=True`, `trt_max_workspace_size=1073741824`, `trt_engine_cache_enable=True`, `trt_engine_cache_path='/var/cache/onboard/trt-engines'`) + `CUDAExecutionProvider` + `CPUExecutionProvider` automatic subgraph fallback ladder + per-subgraph engine cache integration via `trt_engine_cache_path` | **Clean throughout** — ONNX Runtime (MIT) + Jetson AI Lab community wheels (build-system MIT) + JetPack-bundled TensorRT 10.3 (NVIDIA proprietary, deployment grant on NVIDIA hardware) + CUDA 12.6 + cuDNN 9.3 (NVIDIA proprietary EULA); eligible on every D-C1-1 license-posture choice (MIT throughout the open-source layer; NVIDIA proprietary at the Jetson hardware layer is project-locked) | ✅ Mode enumeration via Source #100 Microsoft ONNX Runtime official documentation context7 lookup at `/microsoft/onnxruntime` v1.25.0 (1462 code snippets, Source Reputation High, **Benchmark Score 82.23 — highest of the three C7 candidate context7 lookups**) — **all four query modes verified**: provider enumeration (`ort.get_available_providers()`), TensorRT EP config pattern (`('TensorrtExecutionProvider', tensorrt_options)`), CUDA EP fallback (`('CUDAExecutionProvider', cuda_options)`), CPU EP final fallback (`'CPUExecutionProvider'`); ✅ runnable example via canonical Microsoft `InferenceSession` Python API + per-subgraph engine cache integration; ✅ **NATIVE provider-cascade subgraph fallback** — ORT TRT EP attempts to optimize each subgraph via TensorRT; falls back to CUDA EP for unsupported ops; falls back to CPU EP if neither GPU EP applies — subgraph fallback is automatic and per-op transparent (operators that TRT does not support silently route through CUDA EP without runtime error); ✅ **NATIVE engine-cache integration** via `trt_engine_cache_enable=True` + `trt_engine_cache_path` — per-subgraph engines are auto-built on first run and persisted; subsequent runs warm-start in <1 sec; ⚠️ **standard `pip install onnxruntime-gpu` does NOT work on Jetson Tegra** per Source #100 Issue #20503 — Microsoft does not publish prebuilt aarch64 wheels with CUDA/TensorRT EPs (mitigated: canonical install via Jetson AI Lab community wheel index per D-C7-3 = mirror-to-project-artifact-registry); ⚠️ **NumPy 2.x incompatibility** with onnxruntime-gpu v1.23.0 JetPack 6 wheels per Source #100 Issue #27562 (mitigated: pin `numpy<2.0.0` in project requirements per D-C7-4 until upstream rebuild); ⚠️ **slight runtime overhead vs TensorRT-native** ~1-3 ms per `session.run()` call for input marshaling and provider dispatch (mitigated at 3 Hz: ~3-9 ms/sec absolute time, well within AC-4.1 budget); ⚠️ **first-run subgraph build cost** 10-300 sec per model silent at runtime if `trt_engine_cache_path` doesn't exist (mitigated: pre-flight provisioning script builds the cache by running synthetic warm-up batch through each model; runtime startup warm-loads in <1 sec) | **Modern-competitive-lead-cross-architecture-portability** (engine Component Option Breadth rule role — addresses the "model-format-agnostic" axis the user did NOT explicitly name in the c7_breadth=B scope but which is structurally important for the C2+C3 model export path; ORT serves as the de-facto bridge between PyTorch model checkpoints and TensorRT-optimized engines via canonical ONNX export); deployment-ready under every D-C1-1 license-posture choice (MIT throughout the open-source layer); CONTINGENT on Jetson AI Lab wheel availability (D-C7-3) + numpy<2 pin (D-C7-4) + first-run build cost amortization (D-C7-7); final ranking deferred to Jetson MVE phase per D-C1-2 + D-C7-3 | **FOUR CONVERGING POSITIVE structural advantages over Cand 1**: (i) **MODEL-FORMAT-AGNOSTIC** — ONNX is the de-facto interchange format; PyTorch / TensorFlow / JAX / scikit-learn / vendor models all export to ONNX with high fidelity. Avoids the per-framework export friction of pure TensorRT (which historically requires specific UFF/Caffe parsers OR ONNX-then-TRT-builder); critical for the project's heterogeneous model sources (DISK + LightGlue from cvg/cvlab-epfl + EigenPlaces from gmberton + LightGlue-ONNX from fabio-sim per Source #103); (ii) **SUBGRAPH FALLBACK to CUDA EP / CPU EP for unsupported ops** — robust to model-architecture additions that TensorRT does not yet support natively (rare custom attention variants, specialized aggregations); TensorRT-native fails to build the engine in these cases; ORT TRT EP gracefully degrades to CUDA EP. Particularly relevant for transformer-based models like LightGlue with non-standard attention variants per Source #103 LightGlue+FlashAttentionV2 plugin reference; (iii) **ENGINE-CACHE INTEGRATION** — per-subgraph engines are auto-built on first run and persisted; subsequent runs warm-start in <1 sec. Eliminates the explicit `trtexec` build step from the deployment workflow vs Cand 1's manual build pipeline; (iv) **CROSS-ARCHITECTURE PORTABILITY OF SOURCE CODE** — the same Python inference script runs on the dev machine (CUDA EP only) and on the Jetson (TensorRT EP + CUDA EP); no Jetson-specific code paths required. **FOUR NEGATIVE-BUT-MITIGABLE structural findings**: (v) **community-wheel-index dependency** (Jetson AI Lab) — adds an external dependency NOT officially supported by Microsoft per Source #100 Issue #20503 (mitigated: mirror to project-controlled artifact registry per D-C7-3 = (c)); (vi) **NumPy <2.0.0 pin** for onnxruntime-gpu v1.23.0 JetPack 6 wheels per Source #100 Issue #27562 (mitigated: track Issue #27562 status at Plan phase per D-C7-4); (vii) **slight runtime overhead vs TensorRT-native** ~1-3 ms per `session.run()` call (mitigated: at 3 Hz absolute overhead is ~3-9 ms/sec, negligible vs AC-4.1 400 ms budget); (viii) **first-run subgraph build cost** 10-300 sec per model (mitigated: pre-flight cache pre-population per D-C7-7). **TWO HARD CAVEATS**: (ix) **less direct control over TRT build flags** vs Cand 1 — ORT TRT EP exposes a curated subset of flags (`trt_fp16_enable`, `trt_int8_enable`, `trt_max_workspace_size`); fine-grained per-layer precision policy (e.g., `setPrecision` overrides per node) requires the explicit TensorRT API. **The curated subset covers the C7 user-pinned scope `c7_quantization=A`** — per-model-family precision policy is captured in D-C7-6 + handled via `trt_int8_enable` per-engine flag toggling; but if Plan-phase Jetson MVE reveals layer-level precision overrides are needed (e.g., for LightGlue's attention layers specifically), Cand 1 (TensorRT-native) would be elevated; (x) **NOT empirically-verified at project's exact stack** — Source #100 documents the API but does not confirm Jetson Orin Nano Super + JetPack 6.2 + TensorRT 10.3 + onnxruntime-gpu 1.23+ + project models combination at runtime. **COMPARATIVE-IMPROVEMENT-VS-CAND-1 VERDICT** (per user's "significant-improvement-only" bar from session-start clarification): Cand 2's improvements (model-format-agnosticism, subgraph-fallback, engine-cache integration, cross-arch portability) are real BUT **the project's pinned scope already pins ONNX as the export format** (DISK+LightGlue, EigenPlaces, MixVPR, NetVLAD, SelaVPR all have ONNX exports per Source #103 + Source #73 fabio-sim/LightGlue-ONNX); Cand 2's "model-format-agnostic" advantage does not exercise additional models the project doesn't already use. **Cand 2 is the recommended ALTERNATE for the cross-architecture-portability axis** if the project's deployment model evolves to include models that lack TensorRT-native parsers OR if dev-machine + Jetson code-parity becomes important. **Cand 2 promotion criteria**: re-evaluate IF (a) project deployment expands to include models without TensorRT-native parsers OR (b) Jetson MVE phase reveals Cand 1's manual `trtexec` build pipeline introduces material friction in the C10 pre-flight provisioning workflow OR (c) cross-architecture portability between dev machine and Jetson becomes a binding constraint | Fact #95 in [`../02_fact_cards/C7_inference_runtime.md`](../02_fact_cards/C7_inference_runtime.md) (per-mode entry + per-numbered-Restriction × per-numbered-AC sub-matrix block + comparative-improvement-vs-Cand-1 analysis) |
| 3 | **Pure PyTorch FP16 mandatory simple-baseline** — PyTorch 2.5 / 2.9 ARM64 wheel via Jetson AI Lab + `torch.amp.autocast(device_type='cuda', dtype=torch.float16)` per-op precision auto-selection OR `model.half()` eager-mode FP16 weight conversion + `torch.compile(model, backend='inductor')` graph-mode optimization (optional) + `torch.no_grad()` inference context | **Clean throughout** — PyTorch (BSD-3-Clause) + Jetson AI Lab community wheels (build-system MIT) + CUDA 12.6 + cuDNN 9.3 (NVIDIA proprietary EULA, deployment grant on NVIDIA hardware); eligible on every D-C1-1 license-posture choice (BSD-3-Clause is permissive throughout the open-source layer; PyTorch's BSD-3-Clause is commonly compatible with both BSD/permissive-only-track and GPL-3.0-or-later track) | ✅ Mode enumeration via Source #101 PyTorch official documentation context7 lookup at `/pytorch/pytorch` v2.5.1/v2.8.0/v2.9.1/v2.11.0 (4866 code snippets, Source Reputation High, Benchmark Score 76.69) — **all three query modes verified**: AMP context manager (`torch.amp.autocast(device_type, dtype, enabled, cache_enabled)`), eager-mode half-precision (`model.half()`), graph-mode optimization (`torch.compile(model, backend='inductor')`); ✅ runnable example via canonical PyTorch tutorials; ✅ **NATIVE per-op precision auto-selection at FP16** via autocast (matmul / conv / linear at FP16; layer-norm / softmax / accumulators stay FP32 for numerical stability); ✅ **NATIVE eager-mode debugging** — full PyTorch model visibility (set breakpoints, inspect intermediate tensors, swap modules at runtime); critical for the **mandatory simple-baseline** role: when a TensorRT-built engine produces unexpected output, the pure-PyTorch baseline is the reference for accuracy parity verification; ⚠️ **standard `pip install torch` does NOT include CUDA support on Jetson** — must use NVIDIA-published or Jetson AI Lab community wheels per Source #101 (mitigated: project deployment uses Jetson AI Lab wheels per D-C7-5 = PyTorch 2.5 + torchvision 0.20 stable combination); ⚠️ **dependency issues** (`libcudss.so.0`, `libnvdla_runtime.so` missing on PyTorch 2.9 cu129 wheel under JetPack 6.2 cu126) — version-mismatch sensitive (mitigated: prefer cu126 variant for JetPack 6.2 alignment); ⚠️ **~1.5-2× slower than TensorRT FP16** at the same workload per Source #102 extrapolation (DISQUALIFIES from primary path; restricts to mandatory simple-baseline + dev-machine reference + emergency-fallback role); ⚠️ **no INT8 path applicable** — pure PyTorch INT8 requires explicit quantization (e.g., `torch.quantization.quantize_dynamic`) which does NOT use TensorRT INT8 calibrators; out-of-scope for `c7_quantization=A` user-pinned scope which targets TensorRT INT8 calibration | **Mandatory simple-baseline** (engine Component Option Breadth rule role — required by the engine's "always have a runnable simple fallback" principle; analogous to NetVLAD's mandatory-simple-baseline role in C2 row + SuperGlue+SuperPoint's role in C3 row + Manual ESKF Solà 2017's role in C5 row + Cand 1 mirror-suite-pattern's role in C6 row but scoped to **runtime-as-baseline** rather than algorithm-as-baseline; pure PyTorch FP16 provides the dev-machine-+-Jetson same-source-code-path that ORT does not, plus eliminates the engine-build step entirely — the simplest possible deployment path); **DISQUALIFIED from primary path** by AC-4.1 latency budget per ~1.5-2× pure-PyTorch-FP16 vs TensorRT-FP16 penalty per Source #102 extrapolation; eligible **as fallback ONLY** when TensorRT engine build fails on the deployed Jetson (e.g., due to memory-pressure-during-tactic-profiling per Source #105 #4 even after D-C7-8 mitigation, or due to a TensorRT-unsupported op that ORT TRT EP also can't route through CUDA EP within budget) | **FOUR CONVERGING POSITIVE structural advantages — for the simple-baseline role**: (i) **ZERO EXPORT FRICTION** — model is loaded directly from PyTorch checkpoint via `torch.load(path)`; no ONNX export, no TensorRT engine build, no engine cache. Fastest path from "model trained" to "model running on Jetson"; critical for the simple-baseline + emergency-fallback role; (ii) **TRIVIAL DEBUGGING** — full PyTorch eager-mode visibility (set breakpoints, inspect intermediate tensors, swap modules at runtime). Critical for the **mandatory simple-baseline** role: when a TensorRT-built engine produces unexpected output, the pure-PyTorch baseline is the reference for accuracy parity verification; (iii) **PRODUCTION-MATURE FRAMEWORK** — PyTorch is the de-facto research and deployment ML framework with daily-active maintenance; Jetson AI Lab wheels track upstream PyTorch releases at ~1-3 month lag; (iv) **NO INT8 CALIBRATION REQUIRED** — FP16 baseline path is calibration-free; runs as soon as the checkpoint is loaded; the **fallback path** when INT8 calibration data is unavailable (D-C7-1 not yet resolved). **TWO NEGATIVE-BUT-MITIGABLE structural findings**: (v) **~1.5-2× slower than TensorRT FP16** at the same workload per Source #102 extrapolation; material for the project's tight AC-4.1 budget — **DISQUALIFIES pure PyTorch FP16 from the primary path**, restricts it to the simple-baseline role + dev-machine reference role + emergency-fallback role if TensorRT engine build fails on the deployed Jetson; (vi) **Jetson AI Lab wheel dependency** — same community-wheel-index concern as Cand 2 (mitigated: pre-flight wheel mirror + project-controlled artifact registry per D-C7-5 = PyTorch 2.5 + torchvision 0.20). **TWO MINOR CAVEATS**: (vii) **`torch.compile` cold-start cost** ~10-60 sec for graph-mode optimization (mitigated: only applies if `torch.compile` is enabled; eager-mode FP16 has zero cold-start cost); (viii) **No per-layer precision auto-selection at INT8** — pure PyTorch INT8 path requires explicit quantization which does NOT use TensorRT INT8 calibrators; **OUT-OF-SCOPE for c7_quantization=A user-pinned scope** which targets TensorRT INT8 calibration via Cand 1 + Cand 2 only. Pure PyTorch is **FP16-only baseline** for this project. **AC-NEW-4 covariance honesty handling**: N/A. **AC-NEW-7 cache-poisoning safety**: N/A | Fact #96 in [`../02_fact_cards/C7_inference_runtime.md`](../02_fact_cards/C7_inference_runtime.md) (per-mode entry + per-numbered-Restriction × per-numbered-AC sub-matrix block) |

### C7 — Plan-phase deliverables raised by candidate closures

1. **D-C7-1 (Cand-1) — calibration-dataset-strategy CLOSED IN C7 batch 1 (2026-05-08, per the C9 / SQ7 restructure user choice A in `../00_question_decomposition.md`)**. **Closure**: strategy = real UAV nadir flight footage at ~1 km AGL over season-matched satellite tiles as the calibration corpus distribution (matches the Project Constraint Matrix's "Inputs available" pinning + provides realistic noise/illumination/season distribution that the deployed system will see). Specific fixture-file pin (AerialVL S03 vs project's Mavic + Derkachi flight clips vs other corpora) is fixture-class and DELEGATED to Test Spec (greenfield Step 5). Synthetic-tile augmentation via random homography is the documented low-data fallback, only invoked if real flight footage is insufficient for Recall@K-target calibration. ~500–1,500 representative samples per the C7 batch 1 INT8 build constraint. **No Plan-phase Choose block remains.** **Cross-component coupling moves from C7↔C9 to C7↔Test Spec for fixture-file pinning.**

2. **D-C7-2 NEW (Cand-1) — TensorRT mixed-precision flag matrix per model family** (single FP16-only flag for entire pipeline / INT8+FP16 for VPR backbones + FP16-only for matchers + FP16-only for VIO frontends [hybrid per family per D-C7-6] / per-layer precision overrides via `setPrecision`). **Recommendation**: D-C7-2 = (b) ladder per D-C7-6 per-model-family precision policy.

3. **D-C7-3 NEW (Cand-2) — ORT-Jetson-wheel-index-pin choice** (`pypi.jetson-ai-lab.io/jp6/cu126` for JetPack 6.2 / `pypi.jetson-ai-lab.io/jp6/cu129` for JetPack 6.x with newer CUDA / mirror the wheel index to a project-controlled artifact registry for offline-deployment robustness). **Recommendation**: D-C7-3 = (c) mirror to project artifact registry (~50 MB per wheel set; pre-flight provisioning step) + cu126 variant for JetPack 6.2 alignment.

4. **D-C7-4 NEW (Cand-2) — numpy-version-pin choice** (`numpy<2.0.0` per Source #100 Issue #27562 / wait for upstream onnxruntime-gpu rebuild against numpy>=2 / pin to a specific onnxruntime-gpu version known to work with numpy<2). **Recommendation**: D-C7-4 = (a) `numpy<2.0.0` until upstream rebuild; track Issue #27562 status at Plan phase.

5. **D-C7-5 NEW (Cand-3) — PyTorch-Jetson-wheel-pin choice** (PyTorch 2.5 + torchvision 0.20 stable / PyTorch 2.9 + torchvision latest / track Jetson AI Lab cadence). **Recommendation**: D-C7-5 = (a) PyTorch 2.5 + torchvision 0.20 for the project's first deployment (most-stable combination per NVIDIA Developer Forum); revisit at Plan phase based on Jetson MVE results.

6. **D-C7-6 NEW (CROSS-COMPONENT — affects C2 + C3 + C1 + C7) — INT8-vs-FP16-per-model-family-precision-policy** (single INT8 across all model families with sensitivity-fallback / per-family precision policy: VPR INT8+FP16 fallback, matchers FP16-only, VIO frontends FP16-only / FP16 across all model families until calibration data validates per-family INT8). **Recommendation**: D-C7-6 = (b) per-family policy per the table in `../02_fact_cards/C7_inference_runtime.md` "Cross-cutting model-family precision policy" section. **Strongest cross-component lever in the C2+C3+C7 design space — D-C7-6 = (b) operationalizes the matcher-INT8-quantization-sensitivity finding from Source #103 + the VPR-CNN-INT8-tolerability finding from Source #102.**

7. **D-C7-7 NEW (Cand-1, CROSS-COMPONENT with C10) — engine-build-on-Jetson-vs-prebuilt-engine-shipping strategy** (build engines at pre-flight on the deployed Jetson / build engines on a known-good "reference Jetson" then ship the same `.engine` files to all production Jetsons / both — primary path build-on-target with reference-Jetson-built engines as a fallback if pre-flight build fails). **Recommendation**: D-C7-7 = (c) primary build-on-deployed-Jetson during pre-flight (handles SM-version drift + future TensorRT minor version updates); fallback prebuilt engines for emergency provisioning. **Cross-component coupling with C10 pre-flight cache provisioning row.**

8. **D-C7-8 NEW (Cand-1) — `config.max_workspace_size` cap** (1 GB safe default / 2 GB for richer kernel-fusion search / 3 GB for fastest-possible engine but high segfault risk on 8 GB budget). **Recommendation**: D-C7-8 = (a) 1 GB safe default; raise to 2 GB only if Plan-phase Jetson MVE shows engine quality is materially worse at 1 GB.

9. **D-C7-9 NEW (Cand-1) — TensorRT version pin within JetPack lifecycle** (pin to JetPack 6.2's bundled TensorRT 10.3 / track JetPack 6.x minor releases / lock the exact JetPack point release for cross-deployment reproducibility). **Recommendation**: D-C7-9 = (a) lock to JetPack 6.2 + TensorRT 10.3 for the project's first deployment; revisit at Plan phase per JetPack release cadence.

---

## C7 — Cross-row dependencies + working summary

- **C7 row hosts the C2 VPR backbone + C3 matcher** at flight time on the Jetson Orin Nano Super; precision policy per D-C7-6 (VPR INT8+FP16 fallback, matchers FP16-only). Source #103 evidence on FP8 LightGlue match-count drops drives the matcher-FP16-only ruling.
- **C7 row inherits D-C2-5 DINOv2 ViT-export gate** from C2 row — Cand 1 + Cand 2 verified-compatible at FP16; INT8 path for ViT-class VPR backbones (SelaVPR, conditional AnyLoc/BoQ/DINOv2-VLAD) deferred to Jetson MVE per D-C7-6.
- **C7 row inherits D-C3-2 LightGlue-inference-runtime gate** from C3 row — Cand 1 (TensorRT-native) + Cand 2 (ORT TRT EP) both compatible with LightGlue ONNX export per Source #103 (canonical fabio-sim/LightGlue-ONNX path); pure PyTorch FP16 (Cand 3) is the simple-baseline.
- **C7 row does NOT host C4/C5 candidates** (CPU-only or CPU-+-Eigen libraries; no inference network); no coupling.
- **C7 row shares the 8 GB Jetson memory budget** with C6 tile-cache (Cand 1 mirror-suite-pattern: ~700 MB-1.5 GB) + system services (~500 MB-1 GB); C7 candidates each consume ~1-2 GB peak combined for VPR-engine + matcher-engine + executor context; total ~1.7-3.5 GB out of 8 GB shared budget — well within AC-4.2.
- **C7 row depends on C10 pre-flight cache provisioning** for the engine-build pipeline per D-C7-7 = (c) build-on-deployed-Jetson at pre-flight + D-C7-1 = (a) calibration-dataset-strategy via AerialVL S03 + AerialExtreMatch + project Mavic flight footage; **strongest C7+C10 cross-component coupling**.
- **C7 row interacts with parent-suite consistency**: parent-suite `satellite-provider` is .NET 8.0 + Postgres + Dapper — no on-Jetson inference runtime equivalent exists. C7 is greenfield-on-Jetson; no cross-suite cascade decision (unlike D-C6-7).

**Closure summary**: C7 batch 1 closed at 3/N on 2026-05-08. **Cand 1 (TensorRT native primary) is RECOMMENDED PRIMARY**; **Cand 2 (ONNX Runtime + TensorRT EP) is RECOMMENDED ALTERNATE** for cross-architecture-portability axis; **Cand 3 (pure PyTorch FP16) is RECOMMENDED MANDATORY SIMPLE-BASELINE** per engine Component Option Breadth rule + dev-machine reference parity + emergency-fallback role only. Nine Plan-phase Choose blocks raised (D-C7-1 through D-C7-9). Subsequent C7 candidates (NVIDIA Triton, NVIDIA DeepStream, CUDA-Python custom kernels) are noted-and-rejected per the user-pinned `c7_overkill_options=A` scope: Triton + DeepStream are server / video-pipeline class with deployment footprints (~500 MB-2 GB) that exceed the project's embedded budget without delivering proportional benefit; CUDA-Python custom kernels would require ~2-4 weeks of CUDA engineering per model with marginal speedup over TensorRT's hardware-aware tactic search. Further candidate evaluation only if Plan-phase Jetson MVE reveals TensorRT-native + ORT TRT EP do not satisfy AC-4.1 latency budget — at which point CUDA-Python custom kernels for the matcher's inner loop become a NEW candidate (separate session).
