ElevenLabs, a voice AI research and deployment company, has released its most ambitious product yet. The American startup has successfully created AI Dubbing, a tool for translating speech into more than 20 different languages, including long-form content.
According to ElevenLabs, AI Dubbing maintains the original emotion, tonality, and other subtle elements of the voice and simultaneously translates it into a different language within minutes.
ElevenLabs CEO Mati Staniszewski said, “We have tested and iterated this feature in collaboration with hundreds of content creators to dub their content and make it more accessible to wider audiences.” He further added, “We see huge potential for independent creative such as those creating video content and podcast all the way through to film and TV studios.”
Read More: Meta Releases the SpeechMatrix Dataset for Speech-to-Speech Translation
The tool is also easy to use; a user needs to create a new project, pick the source and target languages, and upload the content file. AI Dubbing then automatically determines the number of distinct voices in the content and starts working. Once finished, a user can download the file and use it.
Notably, the final translation of the speech happens through a speech synthesis tool called Eleven Multilingual v2, a software responsible for translating speeches in various languages.
We are living in a world where everything connects to everything else. If we were to achieve singularity, whether it will happen through “language” or “translation of language” remains a matter of introspection. In a world where only a handful of individuals speak English, an AI tool that “translates” validates diversity and authenticity.
