Addressing Data Bottlenecks at Rack and Pod Levels Beyond HBM

As the competitive focus in AI semiconductors shifts from individual chip performance to system-wide optimization, SK Hynix has presented a CPO (Co-Packaged Optics) roadmap, a core technology for next-generation AI infrastructure.

SK Hynix recently announced that it has published a paper in the international academic journal Nature Electronics co-authored with global research teams, detailing the development direction of CPO, an optical interconnect technology for AI and high-performance computing (HPC).

The research was conducted with Hong Seung-hoon, team leader of SK Hynix's AI Infrastructure division, and Professor Lee Kyu-sang from the University of Virginia as corresponding authors, with participating research teams from the University of Illinois Urbana-Champaign (UIUC), MIT, Nanyang Technological University (NTU), and Yonsei University.

The paper analyzes that as generative AI has proliferated, massive AI clusters connecting thousands of GPUs and HBMs have emerged, making data movement between systems a new performance-limiting factor.

While computational performance improves rapidly, the increase in system-to-system data transmission bandwidth is relatively constrained, intensifying the so-called "Bandwidth Wall" phenomenon.

Existing copper-based electrical interconnects face issues of increased signal loss and power consumption with greater distances. To address this, the research team proposed CPO technology, which integrates optical transceivers in the same package as the processor.

The paper also proposes an "Optics-Centric Architecture" concept that would expand optical connectivity technology to memory interfaces in the long term.

By directly connecting processors and memory through optical interposers, multiple AI accelerators can share large-scale memory resources, which can help expand AI model scale and improve data movement efficiency.

The research team has set key technical targets including over 100Tb/s bandwidth per node, energy consumption below 1pJ/bit, and inter-chip latency below 10ns for implementing next-generation AI infrastructure.

SK Hynix explained that the significance of this research lies in presenting the direction of system-level AI infrastructure development that integrates memory, packaging, and optical interconnect technologies, going beyond individual memory technologies such as HBM.