# Reasoning Chain

## Dimension 1: Inference Latency

### Fact Confirmation
ONNX Runtime CUDA EP on Jetson Orin Nano is 7-8x slower than TRT standalone with default settings (Fact #1). Even with the workaround (Fact #2), ONNX RT adds wrapper overhead. ONNX RT TRT-EP claims "performance parity" (Fact #5), but real benchmarks show up to 3x gaps on specific models (Fact #6).

### Reference Comparison
Native TRT uses kernel auto-tuning, layer fusion, and mixed-precision natively — no framework wrapper. Our models (LiteSAM, XFeat) are CNN+transformer architectures where TRT's fusion optimizations are most impactful. LiteSAM's reparameterized MobileOne backbone (Fact #17) is particularly well-suited for TRT fusion.

### Conclusion
Native TRT Engine provides the lowest possible inference latency on Jetson Orin Nano Super. ONNX Runtime adds measurable overhead, ranging from negligible to 3x depending on model architecture and configuration. For our latency-critical pipeline (400ms total budget, satellite matching target ≤200ms), every millisecond matters.

### Confidence
✅ High — supported by multiple sources, confirmed NVIDIA optimization pipeline.

---

## Dimension 2: Memory Consumption

### Fact Confirmation
ONNX RT TRT-EP keeps ~420-440MB during execution vs native TRT at ~130-140MB (Fact #3). This is ~280-300MB extra PER MODEL. On our 8GB shared memory Jetson, OS+runtime takes ~1.5GB, cuVSLAM ~200-500MB, tiles ~200MB (Fact #19).

### Reference Comparison
If we run both LiteSAM and XFeat via ONNX RT TRT-EP: ~560-600MB extra memory overhead. Via native TRT: this overhead drops to near zero.

With native TRT:
- LiteSAM engine: ~50-80MB
- XFeat engine: ~30-50MB
With ONNX RT TRT-EP:
- LiteSAM: ~50-80MB + ~280MB overhead = ~330-360MB
- XFeat: ~30-50MB + ~280MB overhead = ~310-330MB

### Conclusion
Native TRT saves ~280-300MB per model vs ONNX RT TRT-EP. On our 8GB shared memory device, this is 3.5-3.75% of total memory PER MODEL. With two models, that's ~7% of total memory saved — meaningful when memory pressure from cuVSLAM map growth is a known risk.

### Confidence
✅ High — confirmed by MSFT developer with detailed explanation of mechanism.

---

## Dimension 3: Deployment Workflow

### Fact Confirmation
Native TRT requires: PyTorch → ONNX → trtexec → .engine file. Engine must be built ON the target Jetson device (Fact #9). Engine is tied to specific GPU model and TRT version. TRT engine build on 8GB Jetson can OOM for large models (Fact #16).

### Reference Comparison
ONNX Runtime auto-builds TRT engine from ONNX at first run (or caches). Simpler developer experience but first-run latency spike. Torch-TensorRT (Fact #12) offers AOT compilation as middle ground.

Our models are small (LiteSAM 6.31M params, XFeat even smaller). Engine build OOM is unlikely for our model sizes. Build once before flight, ship .engine files.

### Conclusion
Native TRT requires an explicit offline build step (trtexec on Jetson), but this is a one-time cost per model version. For our use case (pre-flight preparation already includes satellite tile download), adding a TRT engine build to the preparation workflow is trivial. The deployment complexity is acceptable.

### Confidence
✅ High — well-documented workflow, our model sizes are small enough.

---

## Dimension 4: Operator Coverage / Fallback

### Fact Confirmation
Native TRT fails if a model contains unsupported operators. ONNX RT TRT-EP auto-falls back to CUDA/CPU for unsupported ops (Fact #5). This is TRT-EP's primary value proposition.

### Reference Comparison
LiteSAM (MobileOne + TAIFormer + MinGRU) and XFeat use standard operations: Conv2d, attention, GRU, ReLU, etc. These are all well-supported by TensorRT 10.3. MobileOne's reparameterized form is pure Conv2d+BN — trivially supported. TAIFormer attention uses standard softmax/matmul — supported in TRT 10. MinGRU is a simplified GRU — may need verification.

Risk: If any op in LiteSAM is unsupported by TRT, the entire export fails. Mitigation: verify with polygraphy before deployment. If an op fails, refactor or use Torch-TensorRT which can handle mixed TRT/PyTorch execution.

### Conclusion
For our specific models, operator coverage risk is LOW. Standard CNN+transformer ops are well-supported in TRT 10.3. ONNX RT's fallback benefit is insurance we're unlikely to need. MinGRU in LiteSAM should be verified, but standard GRU ops are TRT-supported.

### Confidence
⚠️ Medium — high confidence for MobileOne+TAIFormer, medium for MinGRU (needs verification on TRT 10.3).

---

## Dimension 5: API / Integration Effort

### Fact Confirmation
Native TRT Python API (Fact #11): manual buffer allocation with PyCUDA, CUDA stream management, tensor setup via engine.get_tensor_name(). ONNX Runtime: simple InferenceSession with .run().

### Reference Comparison
TRT Python API requires ~30-50 lines of boilerplate per model (engine load, buffer allocation, inference loop). ONNX Runtime requires ~5-10 lines. However, this is write-once code, encapsulated in a wrapper class.

Our pipeline already uses CUDA streams for cuVSLAM pipelining (Stream A for VO, Stream B for satellite matching). Adding TRT inference to Stream B is natural — just pass stream_handle to context.enqueue_v3().

### Conclusion
Slightly more code with native TRT, but it's boilerplate that gets written once and wrapped. The CUDA stream integration actually BENEFITS from native TRT — direct stream control enables better pipelining with cuVSLAM.

### Confidence
✅ High — well-documented API, straightforward integration.

---

## Dimension 6: Hardware Utilization

### Fact Confirmation
ONNX RT CUDA EP does NOT use tensor cores on Jetson Orin Nano by default (Fact #1). Native TRT uses tensor cores, layer fusion, kernel auto-tuning automatically. Jetson Orin Nano Super has 16 tensor cores at 1020 MHz (Fact #7). No DLA available (Fact #8).

### Reference Comparison
Since there's no DLA to offload to, GPU is our only accelerator. Maximizing GPU utilization is critical. Native TRT squeezes every ounce from the 16 tensor cores. ONNX RT has a known bug preventing this on our exact hardware.

### Conclusion
Native TRT is the only way to guarantee full hardware utilization on Jetson Orin Nano Super. ONNX RT's tensor core issue (even if workaround exists) introduces fragility. Since we have no DLA, wasting GPU tensor cores is unacceptable.

### Confidence
✅ High — hardware limitation is confirmed, no alternative accelerator.

---

## Dimension 7: Cross-Platform Portability

### Fact Confirmation
ONNX Runtime runs on NVIDIA, AMD, Intel, CPU. TRT engines are NVIDIA-specific and even GPU-model-specific (Fact #9).

### Reference Comparison
Our system deploys ONLY on Jetson Orin Nano Super. The companion computer is fixed hardware. There is no requirement or plan to run on non-NVIDIA hardware. Cross-platform portability has zero value for this project.

### Conclusion
ONNX Runtime's primary value proposition (portability) is irrelevant for our deployment. We trade unused portability for maximum performance and minimum memory usage.

### Confidence
✅ High — deployment target is fixed hardware.
