Nota has implemented technology that enhances Large Language Model (LLM) inference efficiency by utilizing both GPUs and NPUs in AI PCs. This aligns with the trend in AI PC performance competition expanding beyond individual chip specifications to system-level computational optimization.

Nota, an AI lightweighting and optimization technology company, announced on June 4 that it has implemented heterogeneous computing-based LLM inference optimization technology on an Intel Lunar Lake-based AI PC.

The core of this technology lies in analyzing the LLM execution process by dividing it into input processing and answer generation phases. Nota applied a separated inference method that assigns input processing operations to the GPU and answer generation operations to the NPU.

Heterogeneous computing is a method of utilizing computing units with different characteristics, such as CPUs, GPUs, and NPUs, by dividing them according to the nature of the task. In AI PCs, not only the performance of a single processor but also how the various computing units within the device are combined affects actual inference performance and power efficiency.

According to Nota, the application of split inference resulted in a reduction of energy consumption per token by approximately 32% and an improvement in generation throughput of approximately 12% compared to a single GPU execution method. Compared to a single NPU execution method, the first response latency was reduced by approximately 89%.

These results are significant in that they do not simply use GPUs and NPUs in parallel, but rather arrange computing units according to the characteristics of each stage of the LLM inference process. This demonstrates that even within the same AI PC, response speed and power efficiency can vary depending on how hardware resources are utilized.

In the global PC industry, the axis of competition for AI PCs is shifting beyond the inclusion of an NPU to system designs that combine CPUs, GPUs, and NPUs. Major semiconductor companies, such as Nvidia and Intel, have recently been successively presenting PC computing platforms and AI acceleration strategies aimed at executing AI agents.

Chae Myeong-su, CEO of Nota, stated, “In the era of AI PCs, simply running AI models on a device is not enough; the actual AI experience is determined by the optimization capability to combine various computing devices, such as GPUs and NPUs, according to the model’s characteristics.” He added, “Nota will enhance the efficiency of on-device AI execution in the AI PC era by combining model lightweighting, runtime optimization, and hardware optimization technologies.”