NVIDIA (CEO Jensen Huang), a leader in artificial intelligence (AI) computing technology, has announced a new Robot Operating System (ROS) and is moving to optimize the performance of robotic applications.

NVIDIA announced on the 27th that it has unveiled the new Isaac ROS to provide performance-aware technologies to the ROS developer community at ROS World 2021, the Robot Operating System (ROS) developer conference.

This will help accelerate product development, improve performance, and simplify the integration of cutting-edge computer vision and AI/machine learning capabilities into ROS-based robotics applications.

Isaac ROS GEM provides a package covering image processing and computer vision, including DNN-based algorithms highly optimized for NVIDIA GPUs and Jetson lineup.

Autonomous machines move through a variety of environments, so they need to keep track of where they are. Visual Odometry solves this problem by estimating the relative position of the camera to its initial position. The Isaac ROS GEM for stereo visual odometry provides powerful capabilities to ROS developers.

This provides the highest accuracy for real-time (>60fps @720p) stereo camera visual odometry solutions. Published results based on the widely used KITTI database can be found here. This GPU-accelerated package is not only highly accurate, but also runs quite fast. In fact, it can run HD resolutions in real time on a Jetson Xavier AGX.

DNN Inference GEM is a set of ROS2 packages that allow developers to use NVIDIA’s many inference models available on NGC or provide their own DNNs. Further tuning of pre-trained models or optimization of developers’ own custom models can be done using the NVIDIA TAO toolkit.

After optimization, these packages are deployed by NVIDIA’s inference servers, TensorRT or Triton. The best inference performance is achieved on nodes that utilize TensorRT, NVIDIA’s high-performance inference software development kit (SDK). If the desired DNN model is not supported by TensorRT, you should deploy the model using Triton. GEM includes native support for U-Net and DOPE. The U-Net package, based on TensorRT, can be used to generate semantic segmentation masks from images. And the DOPE package can be used for 3D pose estimation for all detected objects.

This tool is the fastest way to integrate high-performance AI inference into your ROS applications.

Isaac Sim, which will be officially released in November 2021, is the most developer-friendly version to date. Improved simulations enable faster development with a variety of improvements across UI, performance, and usable components. Additionally, improved ROS bridges and more ROS samples enhance the developer experience for ROS developers.

Autonomous robots require large and diverse datasets to train the numerous AI models that run the perception stack. In real-world scenarios, obtaining all of this training data is expensive and potentially dangerous in corner cases. The new synthetic data workflow via Isaac Sim is designed to address safety and quality concerns for autonomous robots and celebrate high-quality datasets.

Developers building datasets can control the probabilistic distribution of scenes, scenes themselves, lighting, and synthetic sensors. Developers also have fine-grained control to help ensure that important corner cases are included in the dataset. The workflow supports versioning and debugging information for dataset replication for audit and safety purposes.

GTC, running November 8-11, will feature tracks for robotics developers and sessions with prominent speakers, including a presentation by OpenRobotics CEO Brian Gerkey. GTC will discuss NVIDIA Jetson, Isaac OS, Isaac Sim, and Isaac GYM.