The inclusion of various AI functions in edge devices is heating up the industry as a next-generation development task. Currently, there is a severe shortage of AI-related experts, developers, and engineers in the field, making it difficult for manufacturing and hardware companies to achieve time-to-market and take the lead in the market.

Accordingly, AI model lightweight and optimization solutions and platforms that provide convenience and completeness in on-device AI implementation are emerging. There is a rapidly increasing demand for solutions that optimize AI models without compromising performance on a variety of chips, even low-spec devices.

We met with Manager Seokyoung Cho of Nota Co., Ltd. (hereinafter referred to as Nota), which has recently been attracting attention from the industry including Samsung, LG, Naver, and Kakao for its on-device AI optimization solution platform 'NetsPresso', and talked about related technologies, industry trends, and prospects.

■ Introduction to Nota

Nota is an edge/on-device AI technology company established in 2015, specializing in providing AI solutions optimized for devices. Nota has attracted attention as an IT startup that has attracted investment from major domestic companies such as Samsung, LG, Kakao, and Naver D2SF.

Currently, we are collaborating through official partnerships with global semiconductor companies such as NVIDIA, Arm, Intel, Qualcomm, Renesas, and STMicroelectronics.

Nota's core technology, the 'Netpresso' platform, supports AI models to be optimized for various devices and run on their own. This can significantly improve edge computing efficiency and performance.

■ The number of companies distributing AI lightweight/optimized solutions is increasing. Why is this field promising, and what are the industry trends?

The emergence of generative AI has been a major turning point for the AI industry. The speed of AI model development is accelerating, and the size of models is also increasing rapidly. While hardware performance has doubled every two years, following Moore's Law, the size of models based on the Transformer architecture has increased 410-fold over the same period.

As privacy and security issues have recently emerged, the importance of on-device AI has increased. This trend strengthens the need for AI lightweight and optimization technology.

Leading AI companies such as MS, Apple, and Nvidia are making large-scale investments in AI optimization and lightweight technology. Nvidia has acquired companies with AI technology capabilities in recent years, and AMD has also recently acquired 3-4 AI companies to compete with Nvidia, striving to lead the AI era.

■ What is the source of Nota’s expertise and competitiveness?

CTO Taeho Kim, who founded Nota, and CEO Myungsoo Chae, who joined later, both graduated from the KAIST Artificial Intelligence Lab, and Nota’s technological capabilities also come from there. Because it is based at KAIST, it has the advantage of easy access to high-level human resources, and it covers any shortcomings through research collaborations with external professors.

Currently, it is creating more than 90 jobs, most of which are researchers. 60~70% are in charge of research and development (R&D). It is also actively conducting internal seminars by inviting external experts, and is focusing on strengthening technological capabilities by publishing papers through joint research. In the process of these activities, there are cases where companies make collaboration proposals first after reading the papers.

Currently, AI researchers are in short supply across the industry, and experts in AI lightweight/optimization technology are even more rare. In particular, only a very small number of experts have both AI and hardware understanding. Although there are companies that simply lighten/optimize AI models, Nota has industry competitiveness through its expertise in covering all SW-AI-HW.

■ Why model optimization is necessary for on-device AI

The development of a general AI solution goes through the following steps: model learning, conversion, distribution, and performance verification, and repeats until a satisfactory result is achieved. This process can consume a lot of resources such as manpower, cost, and time, and this process becomes even more complicated in edge AI development.

Most AI models are trained on GPUs and inferred on CPU and GPU-based servers, adopting products such as Intel and Nvidia. While general-purpose computing platforms offer high development convenience through rich SW stacks, edge devices often use non-general HW specialized for AI inference, such as NPUs.

Edge devices cannot run large-scale AI models due to their limited computing power and memory. Therefore, model weight reduction and optimization are essential to run AI on low-spec edge devices.

For developers who are not familiar with HW in on-device AI development, converting GPU-based learning models into a form that can run on the device can be a challenge. As a result, on-device AI development requires a wide talent pool of AI researchers, engineers, and hardware experts, and Nota has sufficient strengths in this area.

■ What solution does Nota Netpresso offer?

Nota Netpresso is a platform solution that effectively reduces the human resources, costs, and time required for these complex development processes.

Netpresso consists of the following modules: △Trainer △Compressor △Converter △Benchmarker. Through this, it supports the entire process from model learning to compactness/optimization, automatic conversion and distribution suitable for the target device. Even developers without specific device experience can easily develop edge/on-device AI.

In addition, Nota has a 'device farm' consisting of actual devices, so users can obtain accurate performance measurement results from actual devices rather than simulations through Netpresso. This allows for accurate prediction of performance when applying AI models to the field, and significantly shortens the repetitive performance verification process from project planning.
■ Is it possible for developers/manufacturers with limited AI capabilities to relatively easily load AI models onto hardware through Netpresso?

Yes, that's right. As you asked, Nota goes beyond simple model development or lightweighting to provide customized optimization for the actual chipset and HW environment used. This is based on the rich experience in handling various HWs and the know-how accumulated during the actual solution development and field delivery process.

Based on this expertise and original technology, Netpresso boasts high convenience and completeness. The main features of Netpresso include △modular structure △user-friendly interface △various distribution options.

Netpresso is composed of various modules, allowing it to provide customized functions according to the user's needs. Each module is fully interconnected, yet can be used independently, providing high flexibility.

The user-friendly interface is useful for both non-developers and developers, and the intuitive graphical user interface (GUI) allows even non-developers to easily optimize AI models. It also provides a command-line interface (CLI) and a Python-based API for developers.

Netpresso supports both SaaS and on-premise forms, allowing organizations where data security is important to utilize Netpresso in accordance with their internal security policies.

■ Is on-device AI implementation possible even on low-power, low-spec chips such as MCUs?

Nota also achieved remarkable results in the MCU (Microcontroller Unit) field. △We are building partnerships with leading global companies such as Arm, Renesas, STMicroelectronics, and NXP.

We have full support for the various MCU product lines from these companies, and most notably, Arm and Renesas are Nota customers who are integrating Netpresso with their development tools.

In addition, considering that most MCU companies design their products based on Arm’s IP, Nota’s deep know-how in Arm IP is a very important asset. Based on this, Nota can provide broad support for numerous MCU products, which is expected to greatly expand the scope of AI technology application in various industries.

Nota is contributing to accelerating the practical industrial application of AI technology by providing AI-optimized technologies that cover a wide range of hardware platforms, from advanced deep learning models to low-power MCUs.

(Continued in Part 2...)