The amount of data required by devices such as cars that function with artificial intelligence technology and software is increasing. As the market needs to respond to the increasing amount of data and to have the technology to utilize this data are increasing, the memory speed that supports the processor must be supported in order to satisfy not only the data processing speed but also the amount of data that can be processed.

“Fundamentally, a CPU fetches information from memory, processes it and updates it, and the amount of information a CPU can process is limited by the size of the ‘pipe’ through which it brings data,” said Ugonna Echeruo, principal engineer in the Design Engineering Group at Intel and general architect for Intel Xeon CPU Max series (code-named Sapphire Rapids HBM). “The larger the pipe, the more information the CPU can process, and the more work it can do.”

Echeluo focuses on enabling a new category of solutions for workloads that are limited by memory bandwidth. For now, the answer to limited data is to use high-bandwidth memory (HBM).

HBM solutions can also meet customer demands by enabling CPUs to process more data. Echeluo explained that while the 4th generation Intel Xeon Scalable processors can handle large workloads on their own, HBM is designed specifically for workloads where performance is limited by memory bandwidth or both memory bandwidth and compute limitations.

Customers such as government labs, federal agencies, and universities are the reason for the existence of Max-series CPUs today. “They are running applications that require a lot of memory bandwidth, but are limited by the bandwidth of existing products,” Echeluo said, adding that these customers are asking Intel to increase the bandwidth to meet their needs.

For example, in a research lab that performs high-performance computing that processes massive amounts of data, researchers have the issue of having to utilize many compute nodes to achieve a solution in a typical setup without HBM. The Max series CPUs improve performance and memory bandwidth due to HBM, and enable researchers to perform the same task with fewer resources without changing code. This increases the overall productivity and energy efficiency of the lab.

If we assume that the CPU is an internal combustion engine of a car, the performance of the car is limited by the amount of air that is mixed with the fuel in the combustion chamber. Therefore, turbo and superchargers are added to inject more air. In the case of the CPU, this corresponds to high-bandwidth memory. This allows more air to be forced in, and the top speed also increases.

The CPU's 'pipes' move more air into what was like a combustion chamber over time. As the pipe gets wider, the interface throughput between memory and CPU increases. Wider pipes allow more powerful CPUs to process more data, and customers are happier.

Echeluo explained that the physical distance between the CPU and HBM is the key to success. HBM is soldered to the circuit board close to the processor, allowing it to quickly and conveniently retrieve the necessary information. This proximity allows for power savings.

However, simply attaching HBM to a CPU package is not a simple task.

“The teams had to go through a lot of testing and validation after the design of the 4th generation Intel Xeon processor was finalized,” said Echeluo, who said all the teams involved had to go through a lot of challenges. “They had to look at each IP in the product to make sure there was no conflict with HBM, and that they could utilize the widest possible bandwidth.”

“We also had to find a way to make the changes needed for HBM to operate successfully without impacting standard product schedules and delivery,” he said.

The Max series CPU's advantage is power efficiency. Not only can the proximity of the CPU and HBM save power, but users can also reduce the system and capacity they generally need without HBM. The era of plugging in multiple RAMs has passed, and now the era of HBM has arrived.

Intel was the first to add HBM to x86 processors. Echeluo introduced this as “Intel’s key advantage,” saying that the key is “to make HBM more accessible and user-friendly to customers by leveraging the software stack.”

From a user-friendly perspective, unlike GPUs with HBM, the Max series CPUs do not require a lot of manual code changes, saving time and effort. “Customers prefer to have less work to do themselves,” Echeluo added. “This is also the direction Intel is going.”

“Xeon Max will power server infrastructure for basic science, medicine, and cloud infrastructure research being conducted by corporations and national laboratories,” he said.

Meanwhile, Intel is gearing up to expand its next-generation market share at the ISC High Performance '23 conference, showcasing its upcoming high-memory bandwidth products codenamed Granite Rapids and future Intel Xeon processors with Multiplexer Fault Rank (MCR).