“In order to incorporate AI into robots with limited memory and computational power, three elements must be organically interlocked: a lightweight AI model, a system software stack, and AI hardware.”

At the '2025 Embedded AI Trend Forum' hosted by the Embedded Software and Systems Industry Association (Chairman Lee Chang-yeol, hereinafter referred to as KESSIA) in the main auditorium of the Korea Conference Center on the 16th, Kwon Jin-se, a researcher at the On-Device AI Research Center of the Electronics and Telecommunications Research Institute (ETRI), presented on 'On-Device AI Optimization Technology as Seen through the Case of the Quadruped Robot Guide Dog' and presented an actual case of physical AI technology that combines robots and on-device AI technology.

Researcher Kwon Jin-se said, “ETRI’s On-Device AI Research Center began full-scale technology research in February based on government policies centered on AI semiconductors and devices,” adding, “The focus of the research is on building an on-device environment where AI models perform inference on the robot itself without relying on the cloud. “This is an essential element in environments that support the disabled, where robots must operate reliably even in places where external network access is difficult,” he said.

He continued, “To realize on-device AI, three elements must be organically interlocked: lightweight AI model, system software stack, and AI hardware.” He added, “In particular, it is a technical challenge to optimize large models such as LLM and VLM for devices while maintaining their performance.”

To this end, the ETRI research team attempted to reduce the model size through lightweight techniques such as quantization and pruning, while optimizing the compiler and runtime so that it can operate efficiently even on robot platforms with limited memory and computational performance.

For example, a model that was inferred in 1 second on a GPU-based server took 16 seconds on the Robot Jason Orin AGX platform, resulting in a performance gap. However, by changing the inference engine to Lamar CPP and adjusting the system software, the processing time was reduced to 6 seconds.

Researcher Kwon Jin-se said, “There are approximately 45,000 visually impaired people in Korea, but there are only 80 guide dogs, and they require training costs of 100 million won per year and a training period of two years. To overcome this reality, ETRI launched a robot guide dog development project, and as part of this, a robot equipped with on-device AI was unveiled for the first time,” he said.

He continued, “This robot is equipped with multiple AI functions such as speech recognition (STT), text-to-speech (TTS), and vision-based scene recognition for natural conversation with the visually impaired, and analyzes complex street environments in real time and explains them through voice.” He added, “The robot automatically speaks explanations that help users visualize their surroundings, such as “There is a person on the left, there is a complete walkway on the right.”

Researcher Kwon Jin-se also said that the quadruped robot's power limitations and processing capabilities were also problematic, saying that 90 watts of the 130-watt battery capacity was consumed by the motor, and the range of AI functions that could be used was extremely limited.

When multiple models are mounted on a robot at the same time, it can only operate for about 16 minutes, so balancing battery efficiency and processing speed has become a key challenge.

To address this, ETRI also conducted research on Q-form-based performance prediction models, quantization automation tools, and multi-inference engine surveys.

It contributed to the development of robot technology by suggesting optimal selection criteria through performance benchmark analysis of various models and hardware.

As a result of these efforts, ETRI presented an on-device-based robot demo at its own conference last June.

While existing robots had difficulty operating due to network issues, only the on-device robot operated stably, drawing the attention of the audience.

In the future, more precise technologies will be developed, such as replacing LiDAR with a 360-degree camera and MPU-based depth estimation model, and expanding the MP system through multi-party collaboration. We plan to add environmental awareness technology.

Researcher Kwon Jin-se said, “ETRI’s robot guidance system is a representative example of inclusive technology development for the socially disadvantaged, beyond simple technological implementation,” and added, “We will cooperate with various domestic companies to develop on-device AI technology and strive for technological advancement.”