TensorRT is NVIDIA's software development kit for high-throughput, low-latency deep learning inference. The runtime compiles trained models from PyTorch, TensorFlow, ONNX, and JAX into optimized engines targeting NVIDIA Hopper, Blackwell, RTX, and Jetson GPUs through quantization, kernel auto-tuning, and graph fusion.

TensorRT-LLM extends the runtime to large language model serving with in-flight batching, paged attention, and speculative decoding, while TensorRT for RTX targets Windows PCs. The SDK is consumed by Triton Inference Server, NVIDIA NIM microservices, and direct integrations across enterprise AI deployments.