Voice cloning API for translated video

Keep every speaker's own voice when your videos speak other languages. The API clones each voice from the source video automatically, or uses a custom voice clone you created once, and returns the dubbed video with the cloned voices, voice tracks and subtitles.
Instant voice clone
Custom voice clones
REST + Python SDK
clone, assign, re-dub
OAuth 2.0
# 1. create the dub - each speaker gets an Instant Voice Clone by default
curl -X POST https://api.rask.ai/v2/projects \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"video_id": "b8f7835d-e8c9-4ff4-8e34-5c336d818319", "dst_lang": "es-mx"}'

# 2. list the voices available in this project
curl https://api.rask.ai/v2/projects/$ID/voices \
  -H "Authorization: Bearer $TOKEN"
#  "label": "Voice Clone", "gender": "auto"         instant clone
#  "label": "Maria Lopez", "sample_src": "https://..."  your custom clone

# 3. assign your custom clone - include every speaker in the map
curl -X PATCH https://api.rask.ai/v2/projects/$ID \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"voice": {"SPEAKER_00": "<custom_voice_id>", "SPEAKER_01": "<voice_id>"}}'
GET /v2/projects/{project_id}
polling, 10s
# 4. re-dub with the new voices, then poll until merging_done
curl -X POST https://api.rask.ai/v2/projects/$ID/generate \
  -H "Authorization: Bearer $TOKEN"
while :; do
  curl -s https://api.rask.ai/v2/projects/$ID -H "Authorization: Bearer $TOKEN" | jq .status
  sleep 10
done
Used by global teams
Capabilities

Everything the voice cloning API does

Clone each speaker from the source video, or use a custom voice clone you created once, in every translated version and in multiple languages. Choose AI voice presets for languages without voice cloning. Add optional lip-sync, billed separately.

Instant Voice Clone

Every speaker gets a clone of their own voice from the source video, with no audio samples to record. It is assigned by default in supported target languages.

Custom voice clones

Create a reusable voice in the Rask app from clean audio samples, or save a speaker’s voice from an existing project to your Voice Clone Library. With the speaker’s consent, reuse that voice across API projects in supported target languages.

One voice per speaker

Speakers are separated automatically and each one keeps their own cloned voice, so interviews, panels and podcasts stay easy to follow.

Lip-sync

Match mouth movement to the cloned voice as an optional task on any finished project, billed separately. Face detection runs first, so you apply it only where it will work.

Background audio preserved

Music, room tone and effects are separated from speech and mixed back under the cloned voice, so the result keeps the sound of the original production.

AI voice presets

For languages without voice cloning, or when a stock voice fits better, choose an AI voice preset with a set gender for any speaker.

Voice identity across languages

The clone speaks Spanish, French or Japanese with the voice of the original speaker, so the presenter still sounds like the presenter to global audiences.

Swap a voice, re-dub one speaker

Assign a different voice to a speaker and regenerate. Only the segments that changed are billed, so nobody needs to re-record anything.

Scale without limits

No rate limits, no concurrency quota, no per-key throttle. Dub your full catalogue with cloned voices in parallel, at any volume.
See it in action

Hear one speaker in five languages

The original voices in English, then the cloned voices in French, German, Portuguese and Spanish. Each speaker keeps their own voice, and the background audio stays intact.
EN flag
En
FR flag
Fr
DE flag
De
PT flag
Pt
ES flag
Es
Workflow

Three steps from source video to cloned voices

Creating the project starts the dub with Instant Voice Clones. Use the /generate endpoint after you change voices or edit the script.
1

Upload your video

Send a file as multipart form data, or post a link from YouTube, Google Drive, S3, Vimeo or any direct URL. The speech in this video is the voice sample for each Instant Voice Clone.

POST /api/library/v1/media/link
2

Create the project

Pass the media id as video_id and a target language. Each detected speaker receives an Instant Voice Clone in supported languages, and the dub starts immediately.

POST /v2/projects
3

Assign voices and re-dub

List voices for the project, assign a custom clone or a preset to any speaker, then call /generate and download the files when the dub is complete.

PATCH /v2/projects/{project_id}
Processing time varies by project. When you change a speaker's voice, only that speaker's segments are re-dubbed and billed.
Built around you

Need a voice outside the catalogue? Talk to our team

Voice cloning for a language that is not supported yet, a larger custom voice library, your own delivery method: when a voice project needs something outside the public catalogue, our team reviews your requirements and the available options.

Languages and voices

Need a language or voice outside the public catalogue? Talk to our team about your requirements and the available options.

Delivery and integration

Webhooks, push into your own queue, delivery straight to your S3, custom endpoints. Tell us how your pipeline expects to be fed.

Volume and throughput

Batch sizes, target languages per action and processing priority are all contract terms, not fixed product limits.

Security and hosting

Data residency, SSO and SAML, retention rules and security review support are handled as part of enterprise onboarding.
Use cases

Who uses a voice cloning API

Platforms with video inside

LMS and customer-education platforms, creator and video platforms and their mobile apps, webinar and virtual event tools, video hosting and workflow SaaS, digital adoption platforms, live shopping. Let creators and instructors keep their own voice across supported voice-cloning languages. Use AI voice presets for other target languages, and make multilingual dubbing a feature of your platform.

Localization and production agencies

Deliver custom voiceovers in the original presenter's voice for every client video. Keep a custom voice clone for each recurring presenter, with consent on file, and reuse it in every project through the API without asking anyone to re-record.

In-house continuous video production

International retail and e-commerce, global franchises, media publishers, corporate universities. Trainers, executives and product experts record once, then use their cloned voice for supported target languages and AI voice presets for other market versions of training videos, product updates and internal messages.
Customer stories

Real teams, measurable results

How companies localize video with Rask, in their own numbers.
arrow
arrow
Code

Voice cloning API code examples

One complete example in each language: authenticate, upload, dub with Instant Voice Clones, assign a custom clone to a speaker, re-dub and download.
# pip install git+ssh://[email protected]/braskai/rask-sdk.git@main
import asyncio
from rask_sdk import clients, enums, schemas

async def wait(client, job):
    while job.status is not enums.ProjectStatus.MERGING_DONE:
        await asyncio.sleep(10)
        job = await client.get_project(project_id=job.id)
    return job

async def main():
    client = clients.RaskSDKClient(client_id=CLIENT_ID, client_secret=CLIENT_SECRET)

    video = await client.create_media_link(data=schemas.MediaCreateLink(link="https://youtu.be/..."))
    while video.status is not enums.MediaStatus.READY:
        await asyncio.sleep(5)
        video = await client.get_media(media_id=video.id)

    # first dub: every speaker gets an Instant Voice Clone
    job = await wait(client, await client.create_project(
        data=schemas.ProjectCreate(video_id=video.id, dst_lang="es-mx")))

    # assign a custom clone to SPEAKER_00, keep the other speakers as they are
    voices = await client.get_project_voices(project_id=job.id)
    custom = next(v for v in voices if v.label == "Maria Lopez")
    voice_map = {**job.voice, "SPEAKER_00": custom.id}
    await client.patch_project(project_id=job.id, data=schemas.ProjectPatch(voice=voice_map))

    job = await wait(client, await client.generate_project(project_id=job.id))
    print(job.translated_video, job.voiceover)

asyncio.run(main())
github.com/braskai/rask-sdk
async, typed models
import time, requests

AUTH = "https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token"
API  = "https://api.rask.ai"
token = requests.post(AUTH, auth=(CLIENT_ID, CLIENT_SECRET),
                      data={"grant_type": "client_credentials"}).json()["access_token"]
h = {"Authorization": f"Bearer {token}"}

def wait(pid):
    while (p := requests.get(f"{API}/v2/projects/{pid}", headers=h).json())["status"] != "merging_done":
        time.sleep(10)
    return p

video = requests.post(f"{API}/api/library/v1/media/link", headers=h, json={"link": "https://youtu.be/..."}).json()
while video["status"] != "ready":
    time.sleep(5)
    video = requests.get(f"{API}/api/library/v1/media/{video['id']}", headers=h).json()

pid = requests.post(f"{API}/v2/projects", headers=h,
                    json={"video_id": video["id"], "dst_lang": "es-mx"}).json()["id"]
project = wait(pid)

voices = requests.get(f"{API}/v2/projects/{pid}/voices", headers=h).json()
custom = next(v for v in voices if v["label"] == "Maria Lopez")
voice_map = {**project["voice"], "SPEAKER_00": custom["id"]}   # every speaker stays in the map

requests.patch(f"{API}/v2/projects/{pid}", headers=h, json={"voice": voice_map})
requests.post(f"{API}/v2/projects/{pid}/generate", headers=h)
print(wait(pid)["translated_video"])
requests + polling
no SDK required
// no dependencies, Node 18+ global fetch
const API = 'https://api.rask.ai';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const auth = await fetch('https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token', {
  method: 'POST',
  headers: { Authorization: 'Basic ' + btoa(`${CLIENT_ID}:${CLIENT_SECRET}`) },
  body: new URLSearchParams({ grant_type: 'client_credentials' }),
}).then((r) => r.json());
const h = { Authorization: `Bearer ${auth.access_token}`, 'Content-Type': 'application/json' };
const call = (method, path, body) => fetch(API + path, { method, headers: h, body: body && JSON.stringify(body) }).then((r) => r.json());
const wait = async (id) => { let p; while ((p = await call('GET', `/v2/projects/${id}`)).status !== 'merging_done') await sleep(10000); return p; };

let video = await call('POST', '/api/library/v1/media/link', { link: 'https://youtu.be/...' });
while (video.status !== 'ready') { await sleep(5000); video = await call('GET', `/api/library/v1/media/${video.id}`); }

const { id } = await call('POST', '/v2/projects', { video_id: video.id, dst_lang: 'es-mx' });
const project = await wait(id);

const voices = await call('GET', `/v2/projects/${id}/voices`);
const custom = voices.find((v) => v.label === 'Maria Lopez');
await call('PATCH', `/v2/projects/${id}`, { voice: { ...project.voice, SPEAKER_00: custom.id } });
await call('POST', `/v2/projects/${id}/generate`);

console.log((await wait(id)).translated_video);
fetch, Node 18+
plain HTTP
# 1. token (valid 1 hour - cache and reuse it)
TOKEN=$(curl -s -X POST https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token \
  -u "$CLIENT_ID:$CLIENT_SECRET" -d "grant_type=client_credentials" | jq -r .access_token)

# 2. create the dub - Instant Voice Clones are assigned by default
ID=$(curl -s -X POST https://api.rask.ai/v2/projects -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" -d "{\"video_id\": \"$VIDEO_ID\", \"dst_lang\": \"es-mx\"}" | jq -r .id)

# 3. after merging_done: read the current voice map and the voice options
curl -s https://api.rask.ai/v2/projects/$ID -H "Authorization: Bearer $TOKEN" | jq .voice
curl -s https://api.rask.ai/v2/projects/$ID/voices -H "Authorization: Bearer $TOKEN" | jq '.[] | {id, label, gender}'

# 4. assign voices for every speaker, then re-dub
curl -s -X PATCH https://api.rask.ai/v2/projects/$ID -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" -d '{"voice": {"SPEAKER_00": "<custom_voice_id>", "SPEAKER_01": "<voice_id>"}}'
curl -s -X POST https://api.rask.ai/v2/projects/$ID/generate -H "Authorization: Bearer $TOKEN"
PATCH /v2/projects/{id}
OAuth 2.0 client credentials
<?php
use GuzzleHttp\Client;

$http = new Client(['base_uri' => 'https://api.rask.ai']);
$json = fn($r) => json_decode($r->getBody(), true);
$auth = $json((new Client)->post('https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token',
    ['auth' => [$clientId, $clientSecret], 'form_params' => ['grant_type' => 'client_credentials']]));
$h = ['Authorization' => 'Bearer ' . $auth['access_token']];

$wait = function ($id) use ($http, $h, $json) {
    do { sleep(10); $p = $json($http->get("/v2/projects/$id", ['headers' => $h])); }
    while ($p['status'] !== 'merging_done');
    return $p;
};

$id = $json($http->post('/v2/projects', ['headers' => $h,
    'json' => ['video_id' => $videoId, 'dst_lang' => 'es-mx']]))['id'];
$project = $wait($id);

$voices = $json($http->get("/v2/projects/$id/voices", ['headers' => $h]));
$custom = current(array_filter($voices, fn($v) => $v['label'] === 'Maria Lopez'));
$map = array_merge($project['voice'], ['SPEAKER_00' => $custom['id']]);

$http->patch("/v2/projects/$id", ['headers' => $h, 'json' => ['voice' => $map]]);
$http->post("/v2/projects/$id/generate", ['headers' => $h]);
echo $wait($id)['translated_video'];
Guzzle 7
plain HTTP
Python is the official SDK. The Node, PHP and cURL tabs are plain HTTP examples against the same endpoints.
Progress

How a voice cloning project reports progress

A project moves through a documented state machine on the first dub and again after every re-dub, so your users see real progress each time you change a voice. You only need to act on three groups.

Success, stop here

merging_done
Dubbing is complete. Fetch the project once more to retrieve the download links available for your project.

In progress, keep polling

created
uploading
transcription_started
separate_background_started
determine_speakers_started
translation_started
voiceover_started
merging_started
Plus the matching _done states. These map cleanly onto a staged progress UI.

Failure, stop and report

failed
upload_failed
transcription_failed
translation_failed
voiceover_failed
merging_failed
no_audio
no_words
forbidden_link
no_audio, no_words and forbidden_link are specific enough to show directly to your own user.
Observed run, 3:47 source video, en → es-mx
GET /v2/projects/{project_id}
   0s  created
  11s  transcription_started
  87s  translation_started
  99s  voiceover_started
 154s  merging_started
 165s  merging_done

Or skip polling entirely

Point us at a webhook endpoint and every state transition arrives as a POST, so your queue reacts to each change the moment it happens. Webhook delivery is part of enterprise onboarding.
Read the docs
preserveAspectRatio="xMidYMid meet" aria-hidden="true" role="img">
Output

Cloned voice, dubbed video and subtitles

Download the dubbed video with the cloned voices and translated subtitles in SRT and VTT. On eligible plans, you can also download the cloned voice track on its own as WAV, or mixed with the original background audio.
Field
Format
What it is
translated_video
MP4
The dubbed video, ready to publish
voiceover
WAV
Translated speech in the cloned voice only, for your own mix
translated_audio
WAV
Translated speech mixed with the original background audio
translation_srt_path
.srt
Subtitles in the target language
translation_vtt_path
.vtt
Same subtitles as WebVTT, for web players
original_video
MP4
Your source file as stored, for reference
Control

Change a voice, then re-dub only that speaker

Instant clones suit most speakers. When one voice needs a different choice, assign a custom clone or a preset to that speaker and regenerate. Only the changed segments are billed.

Segment-level editing

Segment-level editing
Every project exposes its transcript as segments with speaker labels, so moving a line to another speaker or changing a voice takes one request.
Fetch the transcript and its segments
Patch the source text and we re-translate it
Patch the translated text directly to keep your own wording
Adjust timestamps, add segments, delete segments
Reassign the voice attached to any speaker

Glossaries for terminology

Glossaries for terminology
Attach a glossary so brand names, product names and technical terms translate the same way in every language, whichever voice speaks them.

A glossary covers one language pair and applies across all regional variants of that target language, so one Spanish glossary serves es-mx and es-es alike. The project records glossary_version, so later edits never silently change finished work.
Input

Supported file formats

Upload any of seven video and four audio formats, or send a URL. The speech in the file is the reference for Instant Voice Clone, so clear audio gives the best results.

Video

7 formats
.mp4
.mov
.mkv
.avi
.webm
.m4v
.3gp

Audio

4 formats
.mp3
.wav
.flac
.m4a

Metadata before you spend

The media endpoint returns duration, file size, video codec, frame rate, resolution, audio codec, sample rate and channel layout. Validate a file and estimate cost in advance, because dubbing with cloned voices is billed by the minute.
Security and compliance

Voice data your legal team can approve

Brask Inc holds SOC 2 Type II and operates under GDPR, and publishes live control status in its Trust Center. Custom voice clones are created with the speaker's consent, and for data security your audio is never used to train general models.

Certifications

SOC 2 Type II
GDPR
CAI and C2PA member

Encryption

TLS 1.2 in transit
AES-256 at rest
SSE-S3 managed keys
Key rotation at least every 12 months

Your content

Closed AI environment
Never used to train general models
Full export, no lock-in

Access

SSO and SAML
Role-based access control
OAuth 2.0 client credentials

Enterprise

Data residency options
Custom SLAs
Dedicated account manager

Voice rights

Synthetic voices with usage rights
Consent-based voice cloning
AI safety controls, human in the loop
Comparison

Rask API vs a voice pipeline you build yourself

A text to speech cloning API voices the text you send. Cloned voices in translated video also need transcription, translation, speaker separation, timing and mixing. Here is the same result, assembled two ways.
What you need
Rask API
Assembled from parts
Clone each speaker from the source video
Instant Voice Clone, by default
Collect and upload samples for every speaker
Reusable custom voices
Custom voice clones by voice_id
A voice library you build and maintain
Transcription, translation, synthesis, mixing
One API
Three to four vendors and the code between them
Speaker detection and one voice per speaker
Included
A diarization vendor plus your own voice-assignment logic
Background music and effects preserved
Included
A source-separation step you build and tune
Languages without voice cloning
AI voice presets
Another text to speech vendor
Lip-sync
Optional, billed separately
A separate vendor
Changing one speaker's voice
Re-dub that speaker, billed for changed segments
Re-run the pipeline for the whole file
Rate limits
None
Whatever your strictest vendor enforces
A voice or language outside the catalogue
Discussed with our team
Depends on each vendor
FAQ

Voice cloning API: common questions

Can't find what you're looking for? Contact sales.

Is there an API for voice cloning?

How does voice cloning work with the API?

What is the difference between Instant Voice Clone and Custom Voice Clone?

Can I create a Custom Voice Clone through the API?

What audio do I need for a Custom Voice Clone?

Which languages support voice cloning?

Can I use the voice cloning API as text to speech?

How do I change a speaker's voice after the dub?

How many custom voices can I save?

Is consent required to clone a voice?

Does the cloned voice keep the speaker's emotion and accent?

Can I use cloned voices for commercial use?

How much does voice cloning cost?

Start dubbing in one afternoon

Make your first request today, and talk to us when you need volume terms or a language nobody else has.