“AI semiconductor technology centered on DRAM is undergoing complex and sophisticated evolution toward the three keywords of high capacity, high bandwidth, and low power.”

Samsung Electronics Master Son Kyo-min gave a presentation titled 'DRAM Solutions in the AI Era' at the '10th Artificial Intelligence Semiconductor Forum' held at the InterContinental Parnas in Samseong-dong, Seoul on the 13th.

Master Son Kyo-min mentioned that with the advent of the AI era, the core axes of the semiconductor industry are being rapidly reorganized, and in particular, the technological evolution of memory semiconductors that support AI computational performance, especially DRAM, is attracting attention as an element directly related to the performance and power efficiency of AI systems.

DRAM has been developed over a long period of time through high integration and fine processing.

On the other hand, recently, the center of gravity is shifting towards responding to the large-capacity models and high-speed data processing demands unique to AI.

Master Son Kyo-min said that the conditions for DRAM required in AI are simple: high capacity, high bandwidth, and low power, but the technical requirements for realizing them are high. The task, he said, was by no means simple.

First, existing cell scaling strategies are gradually reaching their limits.

Since around 2020, the rate of cell area reduction has slowed, making the introduction of new cell structures and process technologies essential.

Representative examples include technologies that dig deep to secure transistor channel length and designs for high-efficiency capacitor structures, which are representative attempts to overcome the physical limitations of DRAM cells.

Additionally, improvements to high-speed sensing circuits and sense amplifiers are also being made in parallel. Technologies capable of detecting minute voltage changes on the order of tens of millivolts (mV) are becoming increasingly important in highly integrated cell environments.

In addition, new design techniques such as the 'Per row activation counter' to counter the row hammer phenomenon are introduced, ensuring stability and reliability.

Research is also active on how to vertically arrange the structure of DRAM cells.

The attempt to overcome the 4F² cell area limitation by introducing a cell structure based on VCT (Vertical Channel Transistor) is considered a key task for implementing next-generation high-density DRAM.

Additionally, HBM (High Bandwidth Memory), which vertically stacks DRAM dies at the package level, is a representative method of implementing high bandwidth.

HBM alleviates data transfer bottlenecks between GPUs and DRAM by utilizing stacking and bonding technology based on Through Silicon Via (TSV).

In HBM4, a method is proposed to expand bandwidth by doubling the number of I/Os, and heat dissipation and signal transmission characteristics are improved through new packaging technologies such as Hybrid Copper Bonding.

This makes it possible to create a memory structure that smoothly supports parallel operations of AI servers.

Another area of interest is processing-in-memory (PIM), which performs computations within memory. The goal is to reduce power consumption for data movement and maximize bandwidth within memory.

Although PIM is being actively researched, commercialization is still limited and remains a challenge. Various architectural experiments are also being conducted in parallel, including in-memory computing (IMC) and CXL memory modules.

Efforts to improve power efficiency are also being made in various ways.

In addition to basic power consumption reduction through voltage scaling, Power Gating and Dynamic Body Bias technologies are applied to reduce standby power.

In the DDR series, power management is being enhanced through the integration of internal power management circuits (PMICs), and the LPDDR series is evolving into new packaging solutions (such as LP-CAMM) based on small form factors and low voltage characteristics.

With the proliferation of AI servers and data centers, cooling efficiency and heat control technology are also important topics.

Various methods such as air cooling, water cooling, and immersion cooling are being introduced, and as the power demand of AI data centers rapidly increases, securing power infrastructure is emerging as a key operational requirement.

This means that heat generation suppression and power consumption minimization must be considered simultaneously during memory semiconductor design and system integration.

Master Son Kyo-min said, “AI semiconductor technology centered on DRAM is undergoing complex and sophisticated evolution toward three keywords: high capacity, high bandwidth, and low power.” He continued, “This change is not simply a task for memory companies; it is impossible without organic collaboration across processors, packaging, and system architecture. “Innovation in memory technology in the AI era depends on building an integrated ecosystem that goes beyond individual technologies,” he said.