"We will lead the democratization of edge AI with low-power, high-efficiency embedded AI platforms"

Edge AI: Fast response speed, reduced power consumption and costs, improved security

STM32N6 implements high-performance vision and audio AI in MCU environment

[Editor's note] Recently, the center of AI is shifting from cloud to edge due to fast response speed, reduced power consumption and costs, and improved security. Additionally, in embedded AI development, model optimization tailored to limited memory environments is becoming increasingly important. In this context, STMicroelectronics is actively working to reduce costs and power consumption by implementing high-performance vision and audio AI in MCU environments through the Neural-ART accelerator in STM32N6. Furthermore, through technical support and development ecosystems, the company is helping customers' AI ideas reach mass production. Against this backdrop, we met with Matteo Maravita, director of ST APAC AI Competence Center (AICC), to discuss the direction of edge AI development and ST's strategy.

I am Matteo Maravita, leading the Asia-Pacific AI Competence Center (AICC), and I currently work at ST's Hong Kong office.

The team I lead is responsible for technical support and demand creation in the Asia region related to AI projects based on ST products and ecosystems, with team members in Japan, Taiwan, and China.

The core role of the AICC is to support customers working with embedded AI.

We particularly focus on supporting customers who are new to embedded AI.

Our main role is to concretize customers' ideas first as PoC (proof of concept) projects, and then support them through to the mass production stage.

■ What is the biggest reason for AI's shift from cloud to edge

There are several reasons. These include the fact that there is no need to maintain continuous internet connectivity, fast response speed, reduced power consumption and cost savings as data transmission to the cloud decreases, and improved security (data remains local).

■ What technical balance is needed to improve AI performance in embedded systems with limited power and memory

The point that developers need to examine most carefully is memory usage.

Many AI developers are accustomed to working on Linux machines where they can use "nearly unlimited memory".

However, when transitioning to embedded systems, developers must pay much more attention to model size so that the AI model can operate within the memory capacity of the chosen microcontroller.

In most cases, this is not a significant constraint.

Looking at embedded AI use cases, they generally do not require complex AI models (with the exception of cases like LLMs).

■ I'm curious how ST's Neural-ART accelerator in STM32N6 can transform the existing MCU-based AI development approach

The STM32N6, equipped with ST's proprietary NPU (Neural Processing Unit), Neural-ART Accelerator™, can run advanced computer vision and audio AI models at high speed in a microcontroller environment (typically 30 frames per second for the latest YOLO object detection models).

This is a level of capability that was only possible with FPGAs or high-end application processors just a few years ago. Previously, similar processing required Linux-based processors, external DRAM, and separate power design, but with STM32N6, you can implement lower cost, reduced power consumption, and faster boot times based on MCUs.

You can see actual application examples using STM32N6 and ST edge AI solutions at ST Edge AI Suite.

■ When implementing generative AI at the edge, are small language models sufficient alone? If a hybrid structure combining cloud's large models and edge models is necessary, how should model partitioning and data processing criteria be set

We already have project examples with several customers where a hybrid approach has been applied.

Small edge AI models run on our microcontrollers, and when more advanced LLM capabilities are needed, we connect to the cloud.

A representative example is interaction with LLM models and AI agents through voice commands.

Using a front-end edge AI model in the microcontroller, we remove ambient noise, detect voice activity, convert speech to text, and maintain very low power consumption (always-on capability) and fast response speed.

Once the text data is ready, it is sent to the LLM model in the cloud, during which the amount of data transmitted is minimized.

■ As generative AI evolves into agentic AI and physical AI, new risks will also grow. How should hallucination, security, reliability, and real-time safety issues be validated and controlled at the embedded device level, and what role is ST preparing for this

All the concerns raised above are realistic issues, and application developers must also develop a new "technical culture" on how to mitigate and address these risks.

The simplest way to address these issues is to apply guardrails to AI model outputs at the upper application level, go through multiple layers of validation, and combine AI model outputs with existing algorithms.

Over the past few years, we have accumulated significant experience through various use cases from many customers.

Based on this experience, we provide specialized technical support through local ST engineers not only for AI tools and models but across the entire application to customers who are new to embedded AI.

■ Finally, if you have something to say to our readers

Embedded AI has driven revolutionary change over the past few years.

While the number of customers adopting ST's solutions is increasing, we have also witnessed new AI model architectures such as Transformers being adopted very rapidly.

Watching new technologies enter the market and be applied to actual products for end users is both challenging and fascinating for us.

ST aims to be more than a chip supplier—we want to be a trusted technology partner that grows with our customers for a long time in the edge AI era.

Based on our specialized expertise built over 10 years in the edge AI field, we will be with our customers throughout their journey from stable product supply and user-friendly development tools to developing new AI use case packages and providing expert technical support to bring their ideas to mass production.