Is it possible to implement real-time image recognition on microcontroller (MCU)-class hardware without expensive GPUs or SBCs? Starting from this question, the 'ESP32-S3 + YOLO Nano-based Object and Face Recognition Webinar' will be held online on February 19.

The ESP32-S3 is an SoC with built-in Wi-Fi and BLE, characterized by its ability to accelerate neural network computations and signal processing workloads through support for vector instructions. Espressif supports this acceleration with libraries such as ESP-DSP and ESP-NN, and the ESP-DL framework, which can run deep learning models.

Based on these hardware characteristics, this webinar covers the process of operating object and face recognition on the board by lightweighting a YOLO Nano-class model—an ultra-lightweight object detection model—to suit an MCU environment. The aim is to break the premise that "image recognition equals GPU" and demonstrate the feasibility of implementing low-power, low-cost Edge vision through hands-on practice.

In MCU vision, 'constraints' are more important than 'model performance'

When attempting MCU-based vision in the field, the biggest obstacles are computational load and memory. YOLO Nano aims to be a “Highly Compact” detection network designed with these constraints in mind, and has presented the possibility of deployment in embedded and mobile scenarios.

However, lightweight models exhibit distinct trade-offs between accuracy, frame rate, and power. Therefore, rather than simply introducing the models, the webinar will cover the process of adapting them to operate in an MCU environment using optimization techniques such as quantization, as well as practical decision points regarding what compromises (speed, accuracy, heat, and power) must be made depending on the project objective.

Attempts at “YOLO-level detection on ESP32-S3” are already on the rise.

Recently, there have also been reports of attempts to run lightweight YOLO-based models on the ESP32-S3. For instance, Hackster.io features a project running YOLOv5n on the ESP32-S3, demonstrating an approach to adapting the model to the MCU. Additionally, although it is an external tutorial, an example of implementing real-time person counting using YOLOX-Nano on an ESP32-S3-based HMI device has been released.

Espressif is also steadily expanding its focus on object detection and image recognition. ESP-WHO is known as a framework that provides image processing examples such as face detection and recognition, and with the recent release of a lightweight detection project based on ESP-DL, it appears that more projects will be undertaken in the future.

It was announced that this technical webinar, scheduled for February 29, will be available on the webinar page of e4ds.com, and that all participants will be provided with webinar lecture materials and introductory materials on the ESP32 development environment.