As generative AI shakes up industries and markets, NVIDIA, which leads the AI server market, has introduced a next-generation AI support platform.

NVIDIA, a leader in AI computing technology, today announced the next-generation NVIDIA GH200 Grace Hopper platform for accelerated computing and generative AI.

The GH200 Grace Hopper platform is based on the new Grace Hopper superchip, the world’s first with HBM3e processors. It is designed to handle the world’s most complex generative AI workloads, including large-scale language models, recommender systems, and vector databases, and will be available in a variety of configurations.
> Delivering up to 3.5x more memory capacity and 3x higher bandwidth than current-generation products, the dual configuration consists of a single server with 144 Arm Neoverse cores, 8 petaflops of AI performance, and 282GB of the latest HBM3e memory technology.

“To meet the growing demands of generative AI, data centers need accelerated computing platforms with specialized requirements,” said Jensen Huang, founder and CEO of NVIDIA. “Our new GH200 Grace Hopper superchip platform improves throughput with unprecedented memory and bandwidth, and unifies performance by connecting GPUs without compromising performance.”

The Grace Hopper superchip used in the new platform can be connected to additional superchips using NVIDIA NVLink, allowing them to work together to deploy the massive models used in generative AI. This high-speed technology allows the GPU to have full access to CPU memory, providing a total of 1.2TB of fast memory in a dual configuration.

HBM3e memory transfers data 50% faster than existing HBM3 and provides a total bandwidth of 10TB per second. This allows the new platform to run models 3.5X larger than previous versions, while improving performance with 3X faster memory bandwidth.

Leading manufacturers are already demonstrating systems based on the previously announced Grace Hopper superchip. The next-generation Grace Hopper superchip platform with HBM3e is fully compatible with the NVIDIA MGX server specification unveiled at Computex earlier this year to expand adoption of the technology. Any system manufacturer can add Grace Hopper to over 100 server variants through MGX.

According to NVIDIA, major system manufacturers are expected to have systems based on the GH200 Grace Hopper platform available in the second quarter of 2024.