“Current AI platforms cannot satisfy everything. They are not open ecosystems and are locked-in to specific companies.”

Intel unveiled its AI strategy, which aims for open systems across AI, and unveiled new customers and partners in Vision 2024. To this end, it announced the next-generation Gaudi 3 AI accelerator, emphasizing its comprehensive enterprise AI strategy and performance comparable to the NVIDIA H200.

On the 11th, Intel held a domestic press conference to brief the main contents of Intel Vision 2024 and announced the collaboration between Naver and Intel. In particular, it directly attacked the vulnerabilities of the AI ecosystem centered on NVIDIA and emphasized the strengths of Intel's open ecosystem vision.

First, Intel Vision 2024 mainly introduced the following: △ A scalable system strategy that covers all aspects of AI within the enterprise through an open ecosystem approach, △ AI deployment and success stories of enterprise customers, △ An open ecosystem approach to advance enterprise AI, △ Unveiling of the Intel Gaudi 3 AI Accelerator, △ Edge platform for AI workloads, and Ethernet-based networking connectivity products.

In particular, Naver Center Director Ha Jung-woo appeared in the keynote with Pet Gelsinger and announced that they would collaborate with Intel on establishing a joint research center to expand the generative AI ecosystem and on performance evaluation for the introduction of Gaudi2.

■ Naver, Gaudi 2 Evaluation... Leading the Strengthening of Domestic AI SW Ecosystem

Naver Cloud's Dr. Lee Dong-soo is evaluating Gaudi2 for Naver's main business centered around Large Language Models (LLM) and is seeking to expand the ecosystem with domestic universities and startups through it.

Dr. Lee said that since the recent advent of ChatGPT, LLM has been used a lot in the AI field, and engineers are doing a lot of optimization at a lower level, such as assembly (machine language), rather than dealing with NVIDIA's CUDA. These optimizations are difficult to program due to memory management issues and require a lot of hardware knowledge, making LLM software development a minority domain.

On the one hand, CUDA is very difficult to program, and the code changes every time the chip changes, so it has to be optimized again. Also, if a small number of developers create a CUDA-based LLM, the majority of the rest will use it without understanding CUDA, so Dr. Lee questioned whether the open source will have an impact.

The entities leading LLM SW are universities and startups, which is why Naver is building an ecosystem with domestic universities and startups. On the other hand, domestic research labs and startups have difficulty using AI servers such as GPUs, and large corporations need LLM SW development capabilities, so the interests are aligned.

The recently popular vLLM was created by UC Berkeley and is known as an open source project. It is also reported that Hugging Face and Together AI in the US are directly developing the SW required for LLM.

Naver said that it is in the final stages of coordinating collaborations with domestic startups and leading universities, and that it plans to make an announcement as soon as coordination is finalized. Intel and Naver are taking steps to strengthen the AI SW open source community by utilizing Gaudi2, which is currently under evaluation.

■ Next-generation Gaudi 3, equivalent to NVIDIA H200...Strong cost-effectiveness

Naver highly praised the power-to-performance ratio of Intel Gaudi 2. Gaudi 3 is also a product that follows in the footsteps of the previous generation, emphasizing higher performance and power-to-performance ratio, and announced its launch in the second half of the year.

Intel Gaudi 3 is an AI accelerator with 50% higher inference performance and 40% higher power efficiency than Nvidia H100. This product features an open ecosystem and open standard interconnectors that are not locked into a specific vendor.

Gaudi3 has the following specifications: △5nm process-based △64 tensor processor cores △8 MME (Matrix Multiplication Engine) capable of performing 64,000 parallel operations △128 GB HBMe2 memory capacity △3.7 TB memory bandwidth △96 MB SRAM △24 200 Gb Ethernet ports △PyTorch framework integration △PCIe.

According to data released by Intel, Gaudi3 showed inference performance that was comparable or slightly inferior to that of NVIDIA H200 in the Rama-7B and 70B segments, and showed inference performance that was superior in the Falcon 180B segment.

Compared to H100, the Lama2 model and GPT-3 model showed an average 50% higher inference throughput, and the Lama and Falcon models showed an average 40% better performance.

“We aim to ship 100 million AI accelerators by 2025,” said Seungjoo Na, Intel’s vice president. “The Gaudi 3 accelerator will be the foundation for Falcon Shore, Intel’s next-generation graphics card (GPU) for AI and HPC.”