Dubbing a video used to take a translator, a voice actor and an editor. Here AI does all three, and you check the translation before it's voiced.
Upload a video with speech, up to 5 minutes and 100 MB. ElevenLabs Scribe transcribes it and detects the source language on its own, tells up to four speakers apart and works out whether each one has a male or female voice. A language model then translates every line to fit the time it had in the original, so it reads like something a person would actually say rather than a word-for-word copy. You get the script as a list of lines: edit any of them, and if a line runs long, the editor warns you that the speech will speed up there.
Next, pick a voice for each speaker. “Own voice” clones the speaker from the video itself, so they speak the new language in their own voice. Or choose a voice from the ElevenLabs library. Demucs separates the speech from the music and background noise, so the new voiceover sits on top of the original soundtrack, with the speech level evened out across the video.
Two extras are optional. Lip sync (Sync Lipsync 2) reshapes mouth movements to match the new speech, for videos up to 2 minutes with faces clearly in frame. Burned-in subtitles put the translation on screen, and an SRT file is always available to download.
You can translate into 18 languages: English, Spanish, French, German, Italian, Portuguese, Polish, Turkish, Dutch, Russian, Ukrainian, Japanese, Korean, Chinese, Arabic, Hindi, Indonesian and Vietnamese. Ukrainian, Indonesian and Vietnamese use library voices for now. You can only clone your own voice or the voice of someone who has agreed to it, and you confirm that before dubbing.