Introduction to Xilinx Solutions Applying FPGA-Based Machine Learning

In the automobile manufacturing process, robots perform assembly tasks. They verify that parts are correct and determine which process to send them to. This is possible because these robots are all camera-based. The same applies to defect inspection on semiconductor wafers. Machines demonstrate faster speeds and lower error rates than human visual inspection. This is also why previous vision systems were primarily used in manufacturing processes.

Machine vision, which combines machine learning with vision systems, is expanding this market. As a subsystem of embedded vision systems in which visual processing hardware and software are integrated into a single unit, it is used in various fields such as automotive, electronics, semiconductors, panels, food, transportation, surveillance, security, pharmaceuticals, and military. Although there are differences in technological development depending on the field of application, it is generally developing rapidly and its scope is expanding.


Currently emerging medical image processing devices, Advanced Driver Assistance Systems (ADAS), autonomous vehicles, security image processing systems, drones, and ProAV systems are included in embedded vision systems.

Xilinx recently announced technology that expands vision-based machine learning applications. Jung Woong, a manager at Xilinx, stated, “The scope of various vision-based systems is expanding. The market for utilizing machine vision will broaden from industry to daily life, including autonomous drones that detect and avoid obstacles, augmented reality, autonomous vehicles, automated surveillance, and automated medical diagnosis.”

Because Xilinx's embedded vision solutions are based on Programmable Logic Devices (FPGAs), they enable large-scale parallel processing, which has the advantage of allowing the recognition and analysis of high-resolution, high-frame-rate images.

Of course, the platform can be reused. Xilinx’s ZYNQ product family, which basically implements ARM cores and programmable logic on a single chip, allows for the creation of products optimized for performance or resources to support various applications. The number of IO pins and processors can be adjusted according to the logic size, so the design used in other devices can be used as is.

In addition, sensor fusion is possible by adding sensors of the same or different types. Since IP is provided by partners belonging to the Xilinx Alliance program, it supports various interface standards used in the industry, making system implementation easy.

In addition, it provides performance per watt with lower power requirements to achieve the same performance as an implementation method for embedded vision systems. This is one of the key reasons for using FPGAs from a machine learning perspective. The security and safety provided by Zynq devices are a major advantage, as machine vision systems require a high level of security.

"As vision-related applications expand rapidly, processor solutions that simply process vision have limitations. To create products integrated with machine vision, we must establish an application building framework," said General Manager Jeong. Introducing reVISION, he expressed his desire to share accurate information about embedded vision, adding, "We have established a system capable of comprehensively supporting everything from device hardware to the software-level stack."

Accordingly, the Embedded Vision Series seminar will be held online starting March 28. The first topic examines trends in the embedded vision market, technical factors in machine vision, and challenges faced by developers, and introduces Xilinx's solutions and devices. Further details can be found via the eeWebinar.