# Comparison Framework

## Selected Framework Type
Decision Support

## Selected Dimensions
1. Inference latency
2. Memory consumption
3. Deployment workflow complexity
4. Operator coverage / fallback
5. API / integration effort
6. Hardware utilization (tensor cores)
7. Maintenance / ecosystem
8. Cross-platform portability

## Comparison: Native TRT Engine vs ONNX Runtime (TRT-EP and CUDA EP)

| Dimension | Native TRT Engine | ONNX Runtime TRT-EP | ONNX Runtime CUDA EP | Factual Basis |
|-----------|-------------------|---------------------|----------------------|---------------|
| Inference latency | Optimal — uses TRT kernels directly, hardware-tuned | Near-parity with native TRT (same kernels), but up to 3x slower on some models due to wrapper overhead | 7-8x slower on Orin Nano with default settings (tensor core issue) | Fact #1, #5, #6 |
| Memory consumption | ~130-140MB after engine load (releases serialized blob) | ~420-440MB during execution (keeps serialized engine) | Standard CUDA memory + framework overhead | Fact #3, #4 |
| Memory delta per model | Baseline | +280-300MB vs native TRT | Higher than TRT-EP | Fact #3, #19 |
| Deployment workflow | PyTorch → ONNX → trtexec → .engine (must build ON target device) | PyTorch → ONNX → pass to ONNX Runtime session (auto-builds TRT engine) | PyTorch → ONNX → pass to ONNX Runtime session | Fact #9, #10 |
| Operator coverage | Only TRT-supported ops. Unsupported ops = build failure | Auto-fallback to CUDA/CPU for unsupported ops | All ONNX ops supported via CUDA/cuDNN | Fact #5 |
| API complexity | Lower-level: manual buffer allocation, CUDA streams, tensor management | Higher-level: InferenceSession, automatic I/O | Highest-level: same ONNX Runtime API | Fact #11 |
| Hardware utilization | Full: tensor cores, layer fusion, kernel auto-tuning, mixed precision | Full TRT kernels for supported ops, CUDA fallback for rest | Broken on Orin Nano with default settings (no tensor cores) | Fact #1, #2 |
| Maintenance | Engine must be rebuilt per TRT version and per GPU model | ONNX model is portable, engine rebuilt automatically | ONNX model is portable | Fact #9 |
| Cross-platform | NVIDIA-only, hardware-specific engine files | Multi-platform ONNX model, TRT-EP only on NVIDIA | Multi-platform (NVIDIA, AMD, Intel, CPU) | Fact #9 |
| Relevance to our project | ✅ Best — we deploy only on Jetson Orin Nano Super | ❌ Cross-platform benefit wasted — we're NVIDIA-only | ❌ Performance issue on our target hardware | Fact #7, #8 |

## Per-Model Applicability

| Model | Can Convert to TRT? | Recommended Path | Notes |
|-------|---------------------|------------------|-------|
| cuVSLAM | NO | N/A — already CUDA native | Closed-source NVIDIA library, already optimized |
| LiteSAM | YES | PyTorch → reparameterize MobileOne → ONNX → trtexec --fp16 | INT8 safe for MobileOne backbone only, NOT TAIFormer |
| XFeat | YES | PyTorch → ONNX → trtexec --fp16 (or use XFeatTensorRT C++) | XFeatTensorRT repo already exists |
| ESKF | N/A | N/A — mathematical filter, not a neural network | Python/C++ NumPy |
