Release of AI Inference Software Supporting Real-Time Conversation
Published2019.12.19 09:52Support for applications such as voice agents, chatbots, and recommendation engines
With the release of AI inference software that supports real-time conversation, NVIDIA is expected to improve inference latency, which has been a stumbling block in human-AI interaction.
▲ NVIDIA has unveiled the latest version of TensorRT, inference software that supports conversational AI. <Photo = NVIDIA>
NVIDIA has unveiled NVIDIA TensorRT 7, the latest version of AI inference software that will be used by developers worldwide to deliver conversational AI applications.
NVIDIA TensorRT 7 is NVIDIA's 7th generation inference software development kit that facilitates interaction between humans and AI, supporting the real-time implementation of applications such as voice agents, chatbots, and recommendation engines.
In addition, it provides the latest deep learning compiler designed to automatically optimize and accelerate transformer-based Recurrent Neural Networks (RNNs) required for implementing AI voice applications.
This enables conversational AI components to run 10 times faster than before when executed on the CPU, while reducing latency to below the 300-millisecond threshold required for real-time interaction..
According to U.S. market research firm Juniper Research, digital voice assistants are estimated to be used on approximately 3.25 billion devices worldwide, and this number is expected to reach 8 billion by 2023, exceeding the total global population.
NVIDIA Founder and CEO Jensen Huang stated during his keynote speech at GTC China, “The world we live in today is a new AI era where machines can understand human language in real time,” adding, “TensorRT 7 provides developers around the world with the tools to build and deploy fast, smart conversational AI services that help facilitate more natural interactions between humans and AI.”
Meanwhile, TensorRT 7 supports the extension of AI models used for predicting time-series sequence data scenarios that utilize recurrent loop structures known as RNNs. In addition to AI voice networks, RNNs are also used for purposes such as planning arrival times for vehicles or satellites, predicting situations to be entered into electronic medical records, predicting financial assets, and detecting fraud.
관련뉴스
NVIDIA Ranks 1st in 5 AI Inference Benchmark Categories
Nvidia announced on the 7th that it achieved first place in all five new benchmark tests measuring AI inference workload performance in data centers and network edge. In MLPerf Inference 0.5, an independent AI inference benchmark test, Nvidia utilized Nvidia Turing GPU for data centers and Nvidia Jetson Xavier SoC for edge computing. MLPerf's five inference benchmarks, which apply various form factors and four inference scenarios, include existing AI applications such as image classification, object detection, and transformation. Nvidia achieved first place in all five benchmarks under data center scenarios including server and offline, with Turing GPU achieving the highest performance per processor among commercial processors.
2019.11.07by 이수민 기자
사진 1장으로 얼굴 영상 만드는 AI 기술 개발됐다
영상 기술 기업 하이퍼커네트가 AI를 활용해 사진 한 장만으로 움직이는 영상을 만드는 안면재현 기술을 개발했다. 재현하는 얼굴에 변형이 일어나던 한계를 극복한 마리오네트는 딥러닝 모델이 얼굴에서 필요한 부분의 정보만 집중하는 구조를 도입했다.
2019.12.06by 최인영 기자
NVIDIA and Didi Chuxing Develop AI Solution for Autonomous Vehicles
Nvidia is collaborating with China's Didi Chuxing to develop autonomous driving and cloud computing solutions. The partnership will utilize Nvidia GPUs for machine learning algorithm training and leverage Nvidia Drive for Level 4 autonomous driving inference. Through this collaboration, both companies aim to enhance environmental perception around vehicles and improve safety in autonomous vehicle development.
2019.12.19by 최인영 기자
ST Microelectronics and NUS Establish Joint Research Center 'HELIX' for Edge AI
ST Microelectronics and the National University of Singapore have established the 'HELIX' joint research center aimed at implementing embedded AI in edge environments. Over a four-year period, the center will conduct research across the entire technology stack from algorithms to silicon, focusing on memory-centric architecture leveraging ST's 18nm FD-SOI technology and embedded PCM. The core objective is to implement intelligence directly in physical devices such as robots and drones while minimizing energy consumption.
2026.08.28by 배종인 기자
ams, 디지털 포토닉스로 휴머노이드 로봇 감각·소통 기능 지원
ams OSRAM이 디지털 포토닉스 기술을 활용해 휴머노이드 로봇의 감각·소통 기능을 강화하는 솔루션을 공개했다. dToF 센서, 광학식 힘 센서, 구조광 이미터 등으로 주변 환경 인식과 정밀 조작을 구현하며, EVIYOS·ALIYOS 플랫폼으로 LED 기반 비언어 소통을 지원한다. 시장조사에 따르면 휴머노이드 로봇 시장은 2025년 6억 달러에서 2035년 510억 달러로 성장할 전망이며, 로봇의 감각 능력 향상이 핵심 성장 동인으로 작용할 것으로 예상된다.
2026.08.27by 배종인 기자
