NetApp has partnered with Intel, TerraTek, and SK Networks Service to launch the 'NetApp AIPod Mini with Intel' in the Korean market, taking an active step to enable in-house departments or regulated industries to rapidly build AI inference infrastructure through CPU-based inference without relying on AI data centers equipped with high-performance GPUs.

At a launch press conference held on the 29th, NetApp stated that the 'NetApp AIPod Mini with Intel' is an “integrated reference system that lowers barriers to cost and complexity, enabling enterprises to rapidly adopt AI inference without excessive infrastructure investment.”

NetApp AIPod Mini is a 'packaged AI inference infrastructure' that combines an Intel Xeon 6-based x86 server, a network switch, and NetApp all-flash storage (based on ONTAP) into a single unit, and runs the Intel-led OPEA (Open Platform for Enterprise AI) software stack on top.

Based on RAG (Search Augmentation Generative) or LLM workflowsThe core objective is to enable AI to leverage the business's data assets, helping generative AI find 'contextual answers from internal data.'

While existing AI data centers were designed around large-scale GPU clusters, resulting in high initial costs and operational difficulties, the AIPod Mini highlights its CPU-based optimization for "inference workloads achievable without GPUs."


Jo Min-sung, Senior Vice President at Intel Korea, divided model sizes into three ranges and emphasized, “Small Language Models (SLMs) at the 7B to 8B level can be executed in a production environment using only a CPU, without the assistance of a GPU.”

Instead of the stereotype that 'AI equals GPU,' it effectively presents the option of CPU inference tailored to the nature of the workload.

Intel AMX (matrix operation acceleration) and MRDIMM for enhanced memory bandwidth, as well as software optimization stacks such as oneAPI and OpenVINO, which are included in Xeon 6, were cited as the basis for the performance.

Executive Director Jo Min-sung explained that AMX is included at the core level to accelerate matrix multiplication operations, and that expanded memory bandwidth plays a crucial role in AI performance.

Another differentiating factor is the 'speed of adoption'.


Kim Ki-seok, a senior executive at NetApp, pointed out that in an enterprise environment, a RAG project typically takes several months, ranging from data collection and environment configuration to model selection, integration of in-house data, and tuning.

On the other hand, it is explained that the AIPod Mini was designed to reduce 'unnecessary technical complexity' and shorten deployment time through a proven reference design (server, switch, and storage) and a packaged design.

The target repeatedly emphasized by NetApp and Intel is not the 'enterprise AI data centers' of large corporations, but rather departmental and branch-level organizations that need to rapidly automate specific tasks.

Kim Ki-seok, Senior Vice President at NetApp, pointed out that when building AI inference at the department or branch level, “the enterprise AI infrastructure is too large and unnecessarily complex, causing a mismatch with requirements,” and presented AIPod Mini as a runtime AI solution to resolve this.

Representative examples of specific applications include contract and documentation work in the legal sector, inventory and personalization in retail, and predictive maintenance in manufacturing.

In particular, the demand for "handling data and AI together locally (on-premises)," such as in industrial and public sectors where it is difficult to upload data to external clouds or internet environments are limited, was cited as a key customer group.

Kim Ki-seok, Senior Vice President at NetApp, stated that it was designed for use in specific departments or areas where the internet environment is constrained by legal and industrial regulations, and Jo Min-sung, Senior Vice President at Intel, also mentioned that the public sector [selected] on-premises private instances.He explained the potential for utilization by citing examples.

Terratec, participating as a implementation partner, put forward a “form that can be quickly implemented even without data scientists or an R&D organization.”


Kang Yu-jin, a senior engineer at Terratec, explained that “system construction within a few days” is possible by running OPEA open source on top of a verified infrastructure structure.

The basic configuration consists of storage, switches, a management server, and an application server, and the application server is configured based on Intel Xeon 6.

As an example of a demo environment, a configuration of 344 cores and 1TB of memory based on 2 application servers (2 sockets each) was mentioned.

In terms of operations, it was explained that administrator/user accounts can be separated through a dashboard, and features such as chat Q&A screens, document summaries, and audio Q&A can be optionally included when downloading and distributing the open source.

It was also highlighted that prompt templates, retriever/reorder parameters, guardrails, and LLM response parameters (e.g., temperature, token limit, etc.) can be adjusted in the UI.

One of the key messages of the AIPod Mini is the security and governance of 'local AI'.

Executive Director Kim Ki-seok [in an on-premise environment]They stated that it processes data and enables the application of security and governance across the entire AI workflow through ONTAP's access control, versioning, and traceability capabilities.

In addition, sensitive data protection was emphasized by mentioning ONTAP's robust ACLs, metadata-based governance, and certifications such as FIPS 140-2/140-3.

In other words, it directly targets the demand from companies seeking to improve business accuracy based on RAG while ensuring that internal data does not leak out.

Terratec presented the perceived performance of CPU inference in terms of 'TTFT (Time to First Token Output)' and concurrent user scalability.

Senior Manager Kang Yu-jin explained that benchmark results showed that TTFT and token throughput did not change significantly at the level of 30 to 50 concurrent users, and presented the number of recommended users in the current configuration.

The intention is that if the number of users increases, the application server can be expanded to respond.

NetApp also mentioned an expansion direction, stating, “If the number of target users increases to 500, 1,000, or 1,500, you can achieve that level of performance by adding servers and deploying them.”

The market outlook presented by the three companies in common at the event is 'inferencing-centered growth'.

Executive Director Kim Ki-seok assessed that AI projects are shifting from a focus on model training to inference, operation, and deployment, and that there is a growing trend toward running on CPUs in optimized environments, moving away from an exclusive reliance on high-performance GPUs.

The OPEA ecosystem was also cited as a catalyst for product diffusion.

Executive Director Jo Min-sung stated that OPEA is an open-source community project designed to facilitate the adoption of enterprise AI, and the necessary pieThey explained that they provide the pre-modules like 'Lego blocks,' allowing customers to customize them or use the end-to-end stack immediately.

NetApp pointed out that “while generative AI is gaining attention as a general-purpose solution, it often fails to achieve expected results due to issues with prompt interpretation accuracy,” and also conveyed the message that AIPod Mini helps turn unique data into business results.

In terms of supply, it is scheduled to be supplied domestically through SK Networks Service and Terratec as of April 29, and implementation and operation support services tailored to customers' AI use cases will also be provided.