The Electronics and Telecommunications Research Institute (ETRI) has developed a technology that fundamentally solves the GPU memory shortage problem, which has been considered the biggest challenge in the process of large-scale artificial intelligence (AI) learning.

ETRI announced on the 8th that it has developed a new memory technology, 'OmniXtend', that solves the memory limitations and data bottlenecks of GPUs, which are considered the biggest problems in large-scale AI learning.

OmniExtend, developed by ETRI, is a technology that utilizes standard Ethernet to configure the memory of multiple devices into a shared memory pool.

This allows for flexible expansion of the memory required for AI learning as needed, and allows for the creation of large-scale memory environments without replacing existing equipment.

ETRI announced that in an experiment applying OmniExtend, the LLM inference environment that had performance degradation due to memory shortage recovered performance by up to two times.

This means that processing performance similar to that of existing systems with sufficient memory can be achieved.

In addition, the existing PCIe-based expansion structure had limitations in the connection distance and expandability between devices, but OmniExtend is an Ethernet switch.It provides a highly scalable structure that can combine physically separated devices into a single memory pool.

This is why it is evaluated as an infrastructure suitable for ultra-large-scale AI data centers.

The ETRI research team developed key element technologies such as △FPGA-based memory expansion node and △Ethernet-based memory transfer engine and verified the stable operation of the system.

In a real demonstration, multiple devices successfully accessed each other's memory in real time over a network.

Additionally, OmniXtend received international attention when it was unveiled consecutively at the RISC-V Summit Europe and North America events held in Paris, France and Santa Clara, USA last year.

ETRI leads the interconnect working group of the CHIPS Alliance under the Linux Foundation and is also contributing to the standardization of open source-based memory expansion.

ETRI plans to promote technology transfer to data center hardware and software companies in the future and begin commercialization in earnest.

In particular, the goal is to create tangible industrial results in the next-generation AI infrastructure market by applying it to various equipment such as AI learning and inference servers, memory expansion devices, and network switches.

In addition, we plan to expand the technology to large-capacity memory networks for high-reliability embedded systems such as vehicles and ships, and pursue follow-up research to enhance the memory sharing structure between heterogeneous accelerators such as NPUs, GPUs, and CPUs.

ETRI's Kim Kang-ho, head of the High-Performance Computing Research Center, said, "We will expand research on NPU-centered memory interconnect technology and strengthen international cooperation so that it can be applied to the next-generation systems of global AI and semiconductor companies."

This study was conducted by the Ministry of Science and ICT and the Institute of Information and Communications Technology Planning and Evaluation (IITP). This work was supported by the project titled ‘Research on Memory-Centric Next-Generation Computing System Architecture.’