NVIDIA Turing GPU and Jetson Xavier
Measuring Data Center and Edge AI Inference Performance
Ranked 1st in 5 categories in MLPerf benchmark tests


NVIDIA announced on the 7th that it took first place in all five new benchmark tests measuring AI inference workload performance in data centers and network edges.

In MLPerf Inference 0.5, the industry's first independent AI inference benchmark test, NVIDIA utilized NVIDIA Turing GPUs for data centers and NVIDIA Jetson Xavier SoCs for edge computing.

MLPerf's five inference benchmarks, applying various form factors and four inference scenarios, include existing AI applications such as image classification, object detection, and transformation.

NVIDIA took first place in all five benchmarks across data center scenarios, including server and offline, and Turing GPUs achieved the highest performance per processor among commercial processors. The results of this MLPerf v0.5 inference can be found on www.mlperf.org on the 6th in entries Inf-0.5-15, Inf-0.5-16, Inf-0.5-19, Inf-0.5-21, Inf-0.5-22, Inf-0.5-23, and Inf-0.5-27. Performance per processor is calculated by dividing the primary metric of total performance by the number of reported accelerators.

Jetson Xavier recorded the highest performance among commercially available edge and mobile SoCs in edge-centric single and multi-stream scenarios. These MLPerf v0.5 inference results can also be found at www.mlperf.org under entries Inf-0.5-24, Inf-0.5-28, and Inf-0.5-29.

“AI is currently at a turning point as it moves from the research phase to the stage of being deployed at scale for real-world applications,” said Ian Buck, Vice President and General Manager of Accelerated Computing at NVIDIA. "AI reasoning is a critical challenge for computing," he said.

They added, “NVIDIA is helping to seamlessly deploy increasingly complex AI models in data centers by combining industry-leading programmable accelerators, the CUDA-X AI algorithm suite, and expertise in AI computing.”

NVIDIA was the only company to submit results for all five MLPerf benchmarks, demonstrating the programmability and performance of its computing platform across various AI workloads.

All of NVIDIA's MLPerf results were achieved based on TensorRT 6.

TensorRT 6 is high-performance deep learning inference software that enables easy deployment and optimization of AI applications in production environments, from data centers to the edge. New TensorRT optimizations are also available as open source in the GitHub repository.

NVIDIA also expanded its inference platform by unveiling the Jetson Xavier NX.

Jetson Xavier NX is a small, powerful AI supercomputer for robots and embedded computing devices at the edge, and is a low-power version of the Xavier SoC used in the MLPerf Inference 0.5 benchmark.