Implement the production-default InferenceRuntime strategy on JetPack
6.2 + TensorRT 10.3 (per D-C7-9). The runtime owns the full TRT
lifecycle: compile_engine via the Polygraphy + trtexec + IBuilderConfig
hybrid (FP16 / INT8 / Mixed precision), deserialize_engine with
EngineGate-first ordering and a pre-allocation GPU memory budget gate,
infer via H2D -> enqueueV3 -> D2H -> stream sync on the owned CUDA
stream, idempotent release_engine, and an injected
ThermalStatePublisher delegation for thermal_state.
INT8 calibration cache trust (D-C10-6, AC-2/3/4) is enforced by a
.calib_cache.sha256 file-integrity sidecar (AZ-280) plus a new
.calib_cache.dataset_sha256 sidecar that records the dataset content
hash at compile time; reuse only when both agree, rebuild silently on
dataset hash mismatch, raise CalibrationCacheError on corrupt sidecar
(never silently overwritten).
GPU memory budget (NFT-LIM-01, default 4 GiB) is checked BEFORE any
TRT call beyond the gate (AC-6); a pre-allocation refusal raises
OutOfMemoryError and leaves the resident state unchanged.
TensorRT 10.3 / Polygraphy / PyCUDA are lazy-imported inside the
methods that need them so the module loads cleanly on Tier-0 hosts.
A standalone CLI entry (python -m
gps_denied_onboard.components.c7_inference.tensorrt_runtime compile
<onnx> <build_config.json>) is wired for C10 CacheProvisioner
(AZ-321) to invoke pre-flight without holding a runtime instance.
C7InferenceConfig gains gpu_memory_budget_bytes (default 4 GiB) and
trtexec_timeout_s (default 600 s, Risk 4 mitigation), both validated
in __post_init__.
Tests: 26 active + 6 Tier-2-gated skips; AC-1 / AC-3 / AC-4 / AC-5
/ AC-6 / AC-7 / AC-10 + NFR-reliability fully covered on Tier-1
via fake CUDA / TRT modules; AC-2 / AC-8 / AC-9 / NFR-perf-deserialize
placeholders skip with prerequisite reason and live in the AZ-298
Tier-2 microbench harness. Code review verdict
PASS_WITH_WARNINGS (1 Medium hot-path hoist fix auto-applied).
Co-authored-by: Cursor <cursoragent@cursor.com>
Freezes the c7_inference Public API per
_docs/02_document/contracts/c7_inference/inference_runtime_protocol.md
v1.0.0:
- InferenceRuntime Protocol (6 methods: compile_engine,
deserialize_engine, infer, release_engine, thermal_state,
current_runtime_label) in components/c7_inference/interface.py.
- DTOs (PrecisionMode enum, OptimizationProfile, BuildConfig,
EngineCacheEntry, EngineHandle opaque marker) in _types/inference.py
— placed at the L1 types layer so C10 can re-export EngineCacheEntry
without crossing the components.* boundary (AZ-270 AC-6).
- ThermalState DTO expanded in _types/thermal.py from the AZ-355
forward-declared stub to the AZ-297 contract shape (cpu/gpu temp,
thermal_throttle_active, measured_clock_mhz, measured_at_ns,
is_telemetry_available). Invariant I-6: when telemetry is
unavailable, throttle is False.
- Error family rooted at c7_inference.errors.RuntimeError (9 subtypes:
EngineBuildError, EngineDeserializeError, EngineHashMismatchError,
EngineSchemaMismatchError, EngineSidecarMissingError,
CalibrationCacheError, InferenceError, OutOfMemoryError,
TelemetryUnavailableError). RuntimeNotAvailableError stays in
runtime_root/errors.py — composition-time, outside the family.
- C7InferenceConfig per-component config block (runtime label,
thermal_poll_hz, engine_cache_dir) with constructor-time validation
rejecting unknown runtime labels.
- Composition-root factory build_inference_runtime in
runtime_root/inference_factory.py with three BUILD_* gates
(BUILD_TENSORRT_RUNTIME, BUILD_ONNX_TRT_EP_RUNTIME,
BUILD_PYTORCH_FP16_RUNTIME). Concrete strategy modules are imported
lazily via __import__ AFTER the flag check, so a Tier-0 build with
the flag OFF MUST NOT load the strategy module (AC-5 / I-5;
verifiable via sys.modules).
- 37 conformance tests cover all 8 ACs + NFR-perf-factory
(p99 build under 200 ms × 1000 calls) + NFR-reliability-error-family.
AC-8 introspects the contract file's Shape table and asserts method
parity against the runtime Protocol; also asserts all 9 error
subtypes are documented.
Retired the AZ-263 scaffolding EngineCacheEntry from _types/manifests.py
(replaced by the AZ-297 canonical shape in _types/inference.py); updated
the LightGlue-flavoured EngineHandle Protocol docstring in
_types/manifests.py to rationalize its intentional dual existence
with the C7 opaque EngineHandle (same name, different consumer-side
cut, mirroring the C4/C5 ISam2GraphHandle pattern).
Stale ThermalState.throttle docstring references in c4_pose/config.py,
c4_pose/interface.py, and _types/pose.py updated to
thermal_throttle_active.
Full unit-test sweep: 843 passed, 2 pre-existing environment skips
(cmake, actionlint).
Co-authored-by: Cursor <cursoragent@cursor.com>
Oleksandr Bezdieniezhnykh