The AI craze that started with ChatGPT has led to a rapid development in model size and performance, but has also caused AI hardware inflation by requiring massive computing resources and memory capacity. In particular, high-performance GPUs and HBM are gaining attention as essential elements of AI models, and Nvidia and SK Hynix are seeking high corporate values.

However, the emergence of DeepSec is causing a crack in the existing landscape. As the emergence of new transformer models that can reduce the dependence on HBM in the AI hardware market accelerates, it seems necessary to respond to the rapid flow of future innovation.

Recently, on the 7th, researchers at the University of Maryland published a paper titled 'Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach', presenting an innovative AI model structure.

The existing transformer model required a huge amount of memory capacity by going through multiple layers and creating caches for each layer. Additionally, when generating a token, it was necessary to synchronize all cache information of previous tokens, so the communication overhead between GPUs was also very large.

In particular, the Chain of Thought (COT) method shows the AI model's reasoning process step by step, showing the AI model's reasoning process as text, just like the process of solving a math problem. This can increase the accuracy of answers, but it also requires more layers, so it can be said that the high-performance memory requirement is higher.

■ Potential space 'rethinking' AI, lightweight inference innovation draws attention

The reason why the new transformer methodology of the University of Maryland research team, recently released, has attracted attention is also because of the limitations of existing models. The lightweight and optimization of AI models that can reduce high-performance memory requirements are suggesting new possibilities for AI inference.

This study introduced a new paradigm called latent space inference to improve the memory inefficiency of existing AI models. The research team designed a new model consisting of three blocks: Prelude, Recurrent Block, and Coda.

Prelude transforms the input sentence into a latent space, and Recurrent Block performs 'recall' on the latent space by repeatedly using blocks with the same parameters. Finally, Coda derives the final result based on the 'recall' information.

While existing AI models created caches for each layer, the new model maximizes memory efficiency by reusing only fixed-size cache slots in Recurrent Blocks. The core of the research is that it significantly reduces memory usage by improving model performance through iterative inference.

Inference in latent space has several advantages over existing AI models. First, it can improve performance through iterative inference without explicit language generation such as Chain-of-Thought (CoT).

In particular, the research team showed that a 3.5 billion parameter model achieved performance at the level of 50 billion parameters. This suggests the possibility that even a small model can achieve performance at the level of a large model. It is also expected to reduce the cost of AI model development.

■ An efficient PIM alternative to expensive HBM?

Although DeepSeek has emerged as one of the alternatives for improving the efficiency of AI models, hardware that supports AI is still needed. With the development of GPT models, data processing has become memory-intensive, and communication and bandwidth between processing and memory are still emerging as key challenges.

PIM can be more energy efficient than HBM-based chips and help alleviate bottlenecks by performing calculations in the memory itself, reducing the data movement distance. A bigger advantage is its applicability in on-device AI.

If optimized lightweight AI inference models such as latent space inference models and DeepSeek are efficiently operated on-device, it is expected that the development of next-generation on-device AI chips such as PIM Semiconductor will produce synergy effects.

At the 2025 ICT Industry Outlook Conference, SK Hynix Fellow Lim Eui-cheol said in his presentation on “Memory and Computing Combination Semiconductor Technology for AI,” “As the LLM input/output tokens get longer, the proportion of attention layer processing increases,” and “As the tokens get larger, they transition to a memory-intensive nature.”

Since both PIM semiconductors, which effectively perform GEMV (matrix × vector) operations at low power, and latent space inference models aim for memory efficiency, attention is being paid to whether the combination of the two technologies can produce synergy effects.