High-performance deep learning inference SDK for optimizing trained models on NVIDIA GPUs.
TensorRT is NVIDIA's software development kit for high-throughput, low-latency deep learning inference. The runtime compiles trained models from PyTorch, TensorFlow, ONNX, and JAX into optimized engines targeting NVIDIA Hopper, Blackwell, RTX, and Jetson GPUs through quantization, kernel auto-tuning, and graph fusion.
TensorRT-LLM extends the runtime to large language model serving with in-flight batching, paged attention, and speculative decoding, while TensorRT for RTX targets Windows PCs. The SDK is consumed by Triton Inference Server, NVIDIA NIM microservices, and direct integrations across enterprise AI deployments.
| Attribute | TensorRT | AI Model Hub | Amazon Bedrock | Gemini Enterprise Agent Platform | Lucebox Hub | ROCm | |
|---|---|---|---|---|---|---|---|
| Provider | NVIDIAnvidia.com24.1B raised · Public |  |  |  |  |  | |
| Founded | 1993 | 2016 | 2006 | 1998 | 2025 | 1969 | |
| Sells To | Consumers, Enterprise | — | Small Business | Consumers, Enterprise | Developers | Consumers, Enterprise | |
| Pricing Model | Licensing, Product Sales, Subscription | Usage-based | Recurring | Advertising, Recurring, Transactional | Hardware Sales | Software, Transactional | |
| Ownership | Public | Private, Venture Capital | Public | Public | Privately Held | Public |
Provider NVIDIAnvidia.com24.1B raised · Public
Founded 1993
Sells To Consumers, Enterprise
Pricing Model Licensing, Product Sales, Subscription
Ownership Public
Provider
Founded 2016
Sells To —
Pricing Model Usage-based
Ownership Private, Venture Capital
Provider
Founded 2006
Sells To Small Business
Pricing Model Recurring
Ownership Public
Provider
Founded 1998
Sells To Consumers, Enterprise
Pricing Model Advertising, Recurring, Transactional
Ownership Public
Provider
Founded 2025
Sells To Developers
Pricing Model Hardware Sales
Ownership Privately Held
MLPerf
mlcommons.org
