NVIDIA recently unveiled its new GB200 Grace Blackwell superchip at NVIDIA GTC 2024. With the introduction of the Blackwell GPU, NVIDIA's first chiplet architecture GPU, the market was overwhelmed by the emergence of an AI superchip with unparalleled performance and efficiency.

In the AI server market between data centers and hyperscalers, which is virtually an energy war, Nvidia has dropped a powerful bomb.

■ Blackwell GPU, Black Hole in Server Market Due to Power-to-Effect Ratio

The NVIDIA Blackwell GPU is a chiplet-style Blackwell GPU 200 (B200) that packages two single chips, each with 104 billion transistors. It is equipped with eight HBM3e, the fifth-generation high-bandwidth memory.

The B200 GPU delivers 5x the AI compute performance in FP4 and 2.5x in FP8 compared to the previous generation H100 Hopper GPU. “Compared to Pascal, which had 19 teraflops (TFLOPS) eight years ago, Blackwell has achieved a 1,000x performance improvement,” said Jensen Huang, CEO of Nvidia.

In addition, NVIDIA has introduced the GB200 Grace Blackwell Superchip, the highest-performance AI superchip ever, by connecting two Blackwell GPUs and one Grace CPU with NVLink chip-to-chip interconnect technology as an AI server solution.

The performance of this superchip alone is impressive, but NVIDIA's GB200 NVL72, which connects 36 of these superchips into a single rack, boasts unprecedented power-to-performance ratio, delivering 25x the energy efficiency of the HGX H100 platform and 30x faster real-time LLM inference performance for a trillion-parameter language model.

■ Future AI Factory Data Center, AI Power Efficiency = Profit and Carbon Neutrality Directly Linked

As the transition to AI in the industrial sector is in full swing, demand for powerful AI computing server GPUs centered around data centers has skyrocketed. The high power consumption of GPUs has passed on a high cost burden to companies, just like the cost of AI chips, and power efficiency has emerged as an important industry topic.

“In the future, data centers will be viewed as AI factories, and their goal will be to generate revenue,” said CEO Jensen Huang. Jensen Huang's statement is intended to emphasize that the goal of data centers is to create intelligence, but on the other hand, data center operating companies cannot help but hope for lower power consumption efficiency from AI platforms with the same performance.

In that sense, the NVIDIA GB200 NVL72 contributes to power efficiency with △LLM inference 30 times faster, △LLM training 4 times faster, △energy efficiency 25 times faster, and △data processing 18 times faster than the previous generation H100.

At GTC 2024, CEO Jensen Huang boasted about the power efficiency of the NVIDIA GB200 NVL72. It takes about 3 to 5 months at 25,000 amps to train a 1.8 trillion parameter GPT model. Doing this job on Hopper H100 would require about 8,000 GPUs and consume about 15 MW for 90 days.

On the other hand, Blackwell can complete this task with just 2,000 GPUs in the same period. The power requirement for 2,000 GPUs is only 4MW. If we can adopt fewer GPUs and work with 25% of the power, we can achieve space efficiency as well as power efficiency. This is also an important consideration for cooling systems.

Recently, the data center industry has been introducing water cooling methods to improve the shortcomings of air-cooled systems, focusing on power usage efficiency (PUE) and reducing operating costs. Liquid immersion cooling technology is also being developed and applied to some extent, but water cooling alone is currently providing sufficient efficiency. The GB200 NVL72 also adopts liquid cooling to improve efficiency.

The International Energy Agency (IEA) forecasted in its 2024 Electricity Report that data centers worldwide will use 460 terawatt-hours (TWh) of electricity in 2022, equivalent to 2% of global electricity demand, and that consumption will increase to 620-1,050 TWh in 2026.

As artificial intelligence spreads, power consumption in data centers is rapidly increasing, requiring the introduction of efficient, advanced cooling systems and continued innovation in power-efficient AI chip solutions.

■ Apple CEO Tim Cook visits China, “AI is the key to carbon neutrality”

On the 24th, local time, Apple CEO Tim Cook attended a climate change discussion held at the China Development Forum and emphasized that artificial intelligence is a key technology that can significantly reduce carbon emissions.

According to major foreign media outlets such as Bloomberg, CEO Tim Cook said, “AI can help companies track their carbon footprints, such as calculating an individual’s carbon emissions, and identify recyclable/reusable materials and provide recycling strategies to companies.”

Apple is pouring significant investment and resources into AI development. It is reported that the 'Special Projects Group' that has been researching Apple's electric car is in the process of disbanding, and that the main research personnel of the relevant department of Apple will be transferred to the AI department.

In addition, the company is taking proactive steps to recruit AI experts by offering a base salary of 400 million won. Although the specific size of the investment was not disclosed, Apple announced at the beginning of the year that it was investing a large amount of capital in AI development, and it is expected that Apple will make a major announcement in the second half of this year to signal a new beginning in the field of generative AI.

Regarding these circumstances, CEO Tim Cook acknowledged at a climate change forum in China that OpenAI had fallen behind in the AI race, but he also reiterated that large-scale investments and resources are being made in AI development to drive more innovation.