Deepdub unveils first-ever AI dubbing technology with precise control of characters’ accents

Accent Control will be first used in collaboration with MHz Choice to dub hit international shows for streaming to North American audiences

Deepdub, the leading AI-based audiovisual dubbing and language localization company, announced today the release of its first-of-its-kind Accent Control technology. The proprietary generative AI tech allows the accents of characters to be retained or modified when dubbing content into another language, enabling a new level of authenticity for AI-dubbed material.

Many of the most memorable movie and TV personas, like Shrek, Jack Sparrow and Hannibal Lecter, have unique accents that are inseparable from their characters. In traditional dubbing processes, dubbing directors must decide whether the localized performance should preserve the original accents to maintain authenticity or if they should be adapted in some way to make the content more suited to the cultural nuances of the target audience. AI dubbing, while lowering the barriers to localization and making content more accessible, has until now not given content creators control over how accents are reflected in dubbed content.

Deepdub’s Accent Control technology leverages custom generative AI models to manipulate and control the accents of characters in dubbed content, offering an unprecedented level of authenticity and flexibility. By enabling content creators to either retain the original accents or adapt them to better fit the dubbed material and the target audience, Deepdub ensures that the emotional and cultural nuances of performances are preserved, enhancing the overall viewing experience.

Marketing Technology News: Ask-AI secures $11M for Generative AI assistant that brings siloed data into enterprise workflows

“Audiences crave genuine experiences and our Accent Control technology marks a significant milestone in achieving that,” said Ofir Krakowski, CEO and co-founder of Deepdub. “It reflects our commitment to breaking down language barriers while respecting and preserving the cultural essence of content. This innovation not only enhances the viewing experience but also underscores our leadership in AI-driven localization solutions.”

MHz Choice, a streaming service that brings international television to North American audiences, will be the first to leverage Accent Control. “Our past collaboration with Deepdub has enabled us to dub numerous popular shows from around the world into English while retaining the original emotional expressivity and performance,” said Lance Schwulst, EVP of Content Strategy at MHz Choice. “Deepdub’s new accent control technology is taking this to the next level. We’re excited to leverage this to bring content to audiences that is unparalleled in its engaging and immersive viewer experience.”

Accent Control is powered by Deepdub’s emotional text-to-speech (eTTS) 2.0 model, a multimodal Large Language Model that supports over 130 languages and dialects. Deepdub’s research team developed the massively multilingual model to generate emotionally expressive speech and discovered emergent capabilities that enabled the manipulation of various speech characteristics including accents. These capabilities were developed into Accent Control, which enables precise control to apply and adapt accents across all 130+ languages and dialects. The company is currently working on expanding these capabilities to support regional accents, enabling micro-localization.

The technology is accessible through the Deepdub GO platform, a virtual AI dubbing studio designed for post-production editors, where users can intuitively apply and adjust accents using a simple interface. It is also available through a white glove dubbing service offered by Deepdub’s in-house team of post-production and localization professionals.

Marketing Technology News: MarTech Interview with Karen Ng, SVP of Product and Partnerships at HubSpot