Advancement of AI Dubbing Technology Targeting Global Distribution of K-Content
Eleven Labs has set out to target the content localization market by unveiling a next-generation AI dubbing model that preserves the original speaker's emotions and delivery style. The company's strategy is to support the expansion of overseas distribution for K-content by reducing the issue of emotional loss that often occurs during video and audio translation and increasing the efficiency of multilingual production.
Eleven Labs announced on the 16th that it has unveiled a new AI dubbing model, 'Dubbing v2'.
The core of this model is to analyze the emotion, intonation, speech speed, and breathing of the original audio and reflect them in the translated audio. It focuses on supporting content creators, media companies, and marketing organizations to produce multilingual content with less cost and time.
Existing AI dubbing had limitations in that the speaker's emotions or tone were diluted during the process of converting speech to text, translating it, and then regenerating the speech.
Dubbing v2 is designed to achieve similar expressiveness in other languages by learning the performance of the original audio itself. It also provides features that reduce the burden of post-production by adjusting translated sentences to the speech patterns of each language and automatically aligning speech timing.
Eleven Labs explained that this model supports over 90 languages and can maintain the speaker's tone and characteristics without separate settings through an automatic voice cloning function. This is considered an element that can be utilized in the production of videos, podcasts, educational content, and corporate promotional materials targeting the global market.
In particular, within the Korean content industry, there is a growing demand for the overseas expansion of intellectual property (IP) based on dramas, films, webtoons, and web novels, as well as games, animations, and creator content. The industry anticipates that AI dubbing technology can improve the cost structure of the content export process by complementing existing localization processes centered on voice actor recording, translation, and editing.
Verification of the related technology has also been conducted in the domestic broadcasting sector. SBS announced that it utilized Eleven Labs' voice AI technology to recreate the voices of real-life individuals during the production of the documentary *Time of the Monster*, confirming its applicability in actual broadcast production environments.
AI voice technology is rapidly advancing amidst recent trends of content production automation and expanding global distribution. As competition intensifies to move beyond simple translation to preserve emotions and speaker characteristics, the industry predicts that the adoption of AI-based localization will further expand among content companies, broadcasters, game developers, and brand firms.