2023-07-17 15:38:41
Tel Aviv Morgan Freeman speaks accent-free Spanish on screen. The deep, distinctive voice of the Hollywood actor then sounds as true to the original as if he actually mastered the language. But nothing there: The Tel Aviv start-up Deepdub takes over the work of dubbing actors and transforms the original voices into any number of other languages with the help of artificial intelligence (AI).
With this, Deepdub wants to revolutionize conventional dubbing, which is mainly done by professional speakers in Germany. Freeman was a first test run, but the list of stars and moderators in the company’s database is now much longer.
Deepdub is a family affair. Together with his brother Nir, Ofir Krakowski worked on the development of the Deepdub platform for two years. Their goal was to apply artificial intelligence and deep learning technologies to the benefit of movie fans. “We wanted to bring the entertainment industry to a global audience,” says Ofir Krakowski. The brothers served as young adults in the Israeli army’s tech units.
After several discussions with leading film industry executives, they realized that their technology could be in high demand. Investors seem to agree.
>> Read more here: Why those USA overtake Europeans in AI investments
In total, the company run by the two brothers has now raised $26 million. The response from the film and television industry to the platform has been “overwhelming,” say the founders. Word of the success has also gotten around in the family: her brother Oz now supports her in marketing.
How does Deepdub work?
The young company creates a speech synthesis that should not differ from the original in terms of tone and linguistic colour. She achieves this with a digital platform that learns and studies the actor’s original voice through contributions, internalizing and adopting his style and characteristic qualities when speaking.
Deepdub team
The Israeli company has already raised $26 million in funding.
(Foto: Deepdub)
Then the platform converts the original sound into the desired language after digital processing. It combines artificial intelligence and deep learning technology. In addition, so-called natural language processing is used, which includes techniques for machine processing of natural language.
Who are the competitors?
The number of providers is growing. The Milan start-up Voiseed, for example, has developed a virtual language machine that enables authors of audio content to produce voices in another language. Compared to Deepdub, however, Voiseed is still very small.
Another competitor is Seattle-based Wellsaid Labs, whose technology produces voice clones. Also in the field is the Indian start-up Dubverse.ai, which has cloned the voice of Prime Minister Narendra Modi and lets him speak in several national languages.
So far, however, only Deepdub has received the Trusted Partner Network (TPN) seal of approval. This review process was created in 2018 by the Association of US Film Production Companies to “establish security benchmarks and a security assessment to prevent content leakage and piracy”.
More start-up checks:
The start-up itself points to another competitive advantage: with films, you have to support a much wider range of emotions, different characters and situations. “How,” asks Ofir Krakowski, “do you dub the voice of a person who is speaking while eating, for example? Our technology even takes that into account – yelling, whispering, whatever.”
Compared to traditional dubbing, Deepdub scores with lower prices. “This tool gives independent producers a chance to compete and even beat the big studios when it comes to reaching larger audiences,” said Ohad Ashkenazi, chief executive of New York-based production company Mila Media.
What’s next for Deepdub’s AI?
The offer is not yet fully automated. A feature film project still requires a lot of “manual work”, say the founders Ofir and Nir Krakowski. Such a translation process therefore currently takes up to eight weeks. The brothers aim to shorten the whole process to two weeks. By the end of the year, Deepdub wants to double the number of available languages to 60.
More: Elon Musk launches new AI company
First publication: 07/15/2023, 10:51 am.
#takes #work #voice #actors