MATLAB/Simulink, providing tools for developing autonomous robots
Streamlining the development process for safe robotic systems
Beyond robot operation, support for factory and business monitoring


Robots, which were commonly seen in manufacturing sites, are spreading to other industries and society as a whole. In particular, with the development of 'Robotics and Autonomous Systems (RAS)' technology, today's robots are capable of higher-level tasks that previous robots could not do.

However, due to the lack of specialized knowledge and limited construction time, the difficulties of engineers who have to apply the technology to the actual environment are increasing. On the 18th, MathWorks held an online seminar titled 'MATLAB and Simulink for Autonomous Robot System Development: From Ideas to Reality' and presented a solution to this.

Fred Noto, Robotics Industry Manager at MathWorks, gave a presentation titled “Robotics, Autonomous System Development Trends and MathWorks Solutions” in which he looked at the current state of industrial robot technology and forecasted future trends in autonomous robot systems. And he introduced examples of companies that built such systems.

◇ MathWorks, Building an Ecosystem for Autonomous Robot System Development

Recently, the introduction of industrial robots equipped with autonomous technology is accelerating in various industrial fields such as smart factories, logistics centers, and construction sites. As the autonomous technology of industrial robots is advanced in many industries, the complexity of autonomous robots is also deepening. The process for developing a safe robotic system is becoming more complex and time-consuming.

In particular, many engineers are complaining about difficulties in modeling and simulating complex mechanical and physical characteristics such as robot motors, joints, and bodies; designing autonomous algorithms including perception, motion planning, and control; and testing robotics apps and deploying hardware.

“MATLAB and Simulink from MathWorks provide the ecosystem necessary for autonomous robot development,” said Noto Manager. “We support customers’ development and introduction of autonomous systems with robotics AI solutions and technology services that integrate modeling, simulation, testing, and deployment.”

MATLAB and Simulink support all stages of the autonomous robotic system workflow (▲model-based design ▲modeling and simulation of multi-domain systems ▲development of autonomous algorithms such as perception, motion planning, and control ▲design and optimization of advanced control functions ▲virtual environment-based integrated system testing and ROS integration ▲automatic code generation for hardware deployment).

First, in the model-based design stage, MathWorks supports physical robot modeling through ‘Simspace’ and improving modeling speed through ‘Robotics System Toolbox’.

Autonomous algorithm development The steps include object recognition through 'Deep Learning Toolbox', image classification and point cloud data processing through 'Computer Vision Toolbox' and 'Lidar Toolbox', advanced path planning and collision check and detection-based motion planning algorithm development through 'Robotics System Toolbox' and 'Navigation Toolbox', and task order scheduling, collision avoidance, and reinforcement learning humanoid development through 'Stateflow', 'MPC Toolbox (Model Predictive Control Toolbox)', and 'Reinforcement Learning Toolbox'.

In the test and deployment phase, the 'Gazebo Simulator' is connected with the MATLAB 'ROS Toolbox' interface, and co-simulation is performed and verified by connecting with the Robotics System Toolbox, and then C/C++ or VHDL code for FPGA and CUDA code for GPU are automatically generated and deployed to hardware.

◇ Industrial robotics that pursues autonomy beyond automation

The Noto manager introduced three trends in industrial robotics. The first is ‘activation of the autonomous system ecosystem.’ The 'automation' of automated systems is changing to 'autonomous' with a high degree of freedom.

Existing industrial robots are automated systems that perform pre-programmed tasks 24 hours a day. They have the advantage of being able to increase efficiency through long-term repetition of simple tasks in a fixed work environment, but they do not have the function to recognize the work environment, so they operate with a safety frame to prevent accidents and damage.

Ricoh built a multi-component nonlinear double-motor actuator model, a key component of a robot drive, in half a day using MathWorks Model-Based Design. This was possible thanks to the use of 3D CAD data in SimScape tools and verification of hardware operation through simulation.

3T developed a robot emergency braking system using MathWorks Model-Based Design. It automatically generated vHDL code using MathWorks' HDL Coder and implemented the control algorithm in FPGA using fixed-point arithmetic. It also reduced cleanroom time from weeks to days and reduced bug resolution time from months to one day.

The diversification and advancement of autonomous technologies are leading to the industrial application of robots that perform tasks appropriate to the situation without human intervention. These include advanced algorithms that determine the performance of tasks on their own, and 'collaborative robots' that support interaction with humans. Collaborative robots have been developing and adopting rapidly in recent years due to their agility to quickly adapt to changing work environments.

Kyocera developed a grasping and motion planning application for collaborative robots using MATLAB, Simulink, other MathWorks toolboxes, and MathWorks consulting services. They built a physical model of the robot arm with Simscape Multibody and trained it to perform autonomous movements through modeling and simulation.

“Autonomous systems will play a large role in future manufacturing,” said Noto Manager. “The role of highly adaptable and more complex mobile manipulators or advanced autonomous robots such as humanoids in complex environments will become more important.”

He predicted that “autonomous system robots will not only perform a wide range of tasks autonomously, but will also have intelligent functions that allow them to collaborate with humans or other machine systems and communicate data for optimization and goal achievement.”

The German Aerospace Research Center (DLR) has developed a humanoid robot, ‘Agile Justin’, that demonstrates the evolution of autonomous technology. It is equipped with visual technology based on stereo vision cameras, tactile technology based on special skin sensing, object detection, autonomous driving, and path planning optimization technology. The robot can recognize the entire workspace with space and object recognition technology, perform tasks such as connecting pipes, and even play catch with humans.

MATLAB and Simulink were used to develop real-time control system algorithms, image correction, and path planning algorithms for Agile Justin, a 53-degree-of-freedom humanoid robot. Stateflow supported the design of the action execution sequence, such as phage. The Agile Justin development team shortened the time required to execute complex functions with model-based design, and eliminated manual coding errors with automatic code generation, allowing them to focus on complex system design.

◇ Robot autonomy, positive impact on factory and business operations

The second industrial robotics trend that Noto Manager mentioned is ‘Optimizing the operation of industrial robots in smart factories.’ Smart factories are operated in an environment that integrates cloud, OT, and field autonomous systems based on ultra-high-speed communication, and data is collected by sensors attached to various robots and machines belonging to the autonomous system and used for operational optimization.

The collected smart factory data is utilized for process management, quality improvement, and predictive maintenance through AI-based analysis models, and is also used for factory monitoring by OT managers. Manufacturing companies that operate multiple factories around the world will be able to collect more data and increase the potential for efficiency gains through connectivity between multiple smart factories.

Mondi developed a factory operation monitoring and predictive maintenance software with MATLAB. The software, which runs 24 hours a day, 365 days a year, predicts and reports factory equipment failures based on machine learning. Mondi has achieved annual cost savings of 50,000 euros. Krones developed a digital twin of an automated package handling robot with Simulink, which it uses for design optimization, defect testing, and predictive maintenance.

The third industrial robotics trend is ‘Accelerating the integration of IT/OT and 5G technologies to improve business operations.’ Shanghai Electric developed an investment return calculation algorithm for a distributed energy planning and design platform using MATLAB, and deployed it to its operational IT system using MATLAB Server.

Shell India has successfully implemented a real-time operational optimization application (IT) for its entire petrochemical plant (OT) using MATLAB. Additionally, Lekha Wireless is developing and applying 5G NR technology that provides data transmission performance 10 times faster than LTE using MATLAB and '5G Toolbox'.

In addition to the presentation by the Noto manager, the seminar concluded with a demonstration and insight on autonomous software development methods using MATLAB and Simulink, including: △development of a robot arm hardware platform model, △robot arm trajectory planning, △robot arm controller design for trajectory tracking, △distribution of robot control algorithms, and integration with ROS.

“In addition to the cases described, MATLAB and Simulink can be used for autonomous software development in various fields,” said the Noto manager. “MathWorks provides support for software development that fits the business of a company, not just programs.”