Lip sync API for dubbed video

Match mouth movements to dubbed speech across 135+ language and regional options. Dub your video through the API, run lip-sync on the result and download a video where the speaker on screen says the new words. A face check runs first. Lip-sync is billed separately from dubbing for each full-video generation, including regeneration after edits.
Free preview minute
Multiple faces on screen
REST + Python SDK
check, preview, sync
OAuth 2.0
# 1. on a dubbed project: confirm the video has a face (runs once, usually automatically)
curl https://api.rask.ai/v2/projects/$ID/lipsync \
  -H "Authorization: Bearer $TOKEN"
#  "check_face_task_status": "done", "video_has_face": true

# 2. preview one free minute of lip-sync
curl -X PUT https://api.rask.ai/v2/projects/$ID/lipsync \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"is_multiple_speakers": true, "is_free_lipsync": true}'

# 3. lip-sync the full video - billed separately from dubbing
curl -X PUT https://api.rask.ai/v2/projects/$ID/lipsync \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"is_multiple_speakers": true, "is_free_lipsync": false}'
GET /v2/projects/{project_id}/lipsync
polling, 15s
# Poll until lipsync_task_status is done, then download lipsync_result_path
while :; do
  curl -s https://api.rask.ai/v2/projects/$ID/lipsync -H "Authorization: Bearer $TOKEN" \
    | jq '{lipsync_task_status, lipsync_task_progress, tasks_in_lipsync_queue}'
  sleep 15
done
Used by global teams
Capabilities

Everything the lip sync API does

Add lip-sync to your dubbed videos. Check for a visible face, use your account’s free preview minute if available, and follow progress through the API. Full-video lip-sync is billed separately from dubbing each time you generate it.

Lip-sync on translated video

Match mouth movements to the dubbed speech in a video your project has translated, across the supported catalogue of 135+ language and regional options.

Automatic face check

Before lip-sync, a check face task confirms the video has at least one face. It runs once per project, usually on its own, so you know the result before you spend minutes.

Several faces on screen

Set is_multiple_speakers to true for interviews, panels and multi speaker dialogue. When only one face is on screen at a time, set it to false to speed up processing.

One free preview minute

Every account gets a free lip-sync minute. Run it with is_free_lipsync on your own footage and review the result before you sync the full video.

Separate billing for each generation

Each full-video lip-sync generation is charged separately from dubbing. After script or voice edits, re-dub the project and generate a new lip-synced version with a new lip-sync charge.

Queue position and progress

The lip-sync info endpoint returns the number of tasks ahead of yours and progress from 0 to 100, so your users see real progress.

Voices that match the face

Lip-sync works on top of whichever voice the project uses for a speaker: their cloned voice or an AI voice preset.

Edit, re-dub, re-sync

Change text, timing or voices, re-dub the project, then run lip-sync again on the new version. The face check does not need to run again.

Scale without limits

No rate limits, no concurrency quota, no per-key throttle. Send lip-sync tasks for your full catalogue, and the queue position tells you where each one stands.
See it in action

From dubbed speech to matching lip movements

The same clip dubbed into French, German, Portuguese and Spanish. On a dubbed project like this one, lip-sync adjusts the position of the lips to the new speech.
EN flag
En
FR flag
Fr
DE flag
De
PT flag
Pt
ES flag
Es
Workflow

Three steps from dubbed video to lip-synced video

Lip-sync runs on a project that is already dubbed. Use the /generate endpoint to re-dub after edits, then sync the lips again.
1

Dub the video

Upload the input video or send a public URL, create a project with a target language and wait for merging_done. Lip-sync needs the translated_video of this project.

POST /v2/projects
2

Check the face

The check face task confirms at least one face in the video. It usually starts automatically. If it has not started, run it yourself once for the project.

PUT /v2/projects/{id}/check_face
3

Run lip-sync and download

Run the lip-sync task, follow lipsync_task_progress from 0 to 100, then download the video from lipsync_result_path.

PUT /v2/projects/{id}/lipsync
Processing time varies with video length and the lip-sync queue. The lip-sync info endpoint shows how many tasks are ahead of yours.
Built around you

Need lip-sync for a special case? Talk to our team

Higher throughput, processing priority, your own delivery method: when a lip-sync integration needs something outside the public catalogue, our team reviews your requirements and the available options.

Languages and voices

Need a language or voice outside the public catalogue? Talk to our team about your requirements and the available options.

Delivery and integration

Webhooks, push into your own queue, delivery straight to your S3, custom endpoints. Tell us how your pipeline expects to be fed.

Volume and throughput

Batch sizes, target languages per action and processing priority are all contract terms, not fixed product limits.

Security and hosting

Data residency, SSO and SAML, retention rules and security review support are handled as part of enterprise onboarding.
Use cases

Who uses a lip sync API

Platforms with video inside

LMS and customer-education platforms, creator and video platforms, webinar and virtual event tools, video hosting and workflow SaaS, digital adoption platforms, live shopping. Add lip-synced dubbing to the videos your users and creators already host, so translated presenters look natural on screen and lip-sync becomes a premium feature of your product.

Localization and production agencies

Offer lip-sync as a separate deliverable on top of dubbing. Use your account’s free preview minute, if available, to evaluate the result. Include full-video lip-sync generation and any further generations after review in your client quote.

In-house continuous video production

International retail and e-commerce, global franchises, media publishers, corporate universities. For marketing, training and executive videos where a face fills the screen, lip-sync makes the translated speech match the speaker on camera in every market.
Customer stories

Real teams, measurable results

How companies localize video with Rask, in their own numbers.
arrow
arrow
Code

Lip sync API code examples

One complete example in each language, on a project that is already dubbed: check the face, run lip-sync, follow progress and download the result.
# pip install git+ssh://[email protected]/braskai/rask-sdk.git@main
import asyncio
from rask_sdk import clients, enums, schemas

PROJECT_ID = "..."   # a project with status merging_done

async def main():
    client = clients.RaskSDKClient(client_id=CLIENT_ID, client_secret=CLIENT_SECRET)

    info = await client.get_lipsync_info(project_id=PROJECT_ID)
    if info.check_face_task_status is not enums.LipsyncStatus.DONE:
        await client.run_check_face_task(project_id=PROJECT_ID)
        while (info := await client.get_lipsync_info(project_id=PROJECT_ID)).check_face_task_status is not enums.LipsyncStatus.DONE:
            await asyncio.sleep(10)
    if not info.video_has_face:
        raise SystemExit("No face found - lip-sync is not available for this video")

    await client.run_lipsync_task(project_id=PROJECT_ID,
        data=schemas.LipsyncTaskData(is_multiple_speakers=True, is_free_lipsync=False))

    while (info := await client.get_lipsync_info(project_id=PROJECT_ID)).lipsync_task_status is not enums.LipsyncStatus.DONE:
        print(info.lipsync_task_progress, info.tasks_in_lipsync_queue)
        await asyncio.sleep(15)

    print(info.lipsync_result_path)

asyncio.run(main())
github.com/braskai/rask-sdk
async, typed models
import time, requests

AUTH = "https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token"
URL  = f"https://api.rask.ai/v2/projects/{PROJECT_ID}"   # a dubbed project

token = requests.post(AUTH, auth=(CLIENT_ID, CLIENT_SECRET),
                      data={"grant_type": "client_credentials"}).json()["access_token"]
h = {"Authorization": f"Bearer {token}"}

info = requests.get(f"{URL}/lipsync", headers=h).json()
if info["check_face_task_status"] != "done":
    requests.put(f"{URL}/check_face", headers=h)
    while (info := requests.get(f"{URL}/lipsync", headers=h).json())["check_face_task_status"] != "done":
        time.sleep(10)
assert info["video_has_face"], "No face found"

requests.put(f"{URL}/lipsync", headers=h,
             json={"is_multiple_speakers": True, "is_free_lipsync": False})

while (info := requests.get(f"{URL}/lipsync", headers=h).json())["lipsync_task_status"] not in ("done", "failed"):
    time.sleep(15)

print(info["lipsync_task_status"], info["lipsync_result_path"])
requests + polling
no SDK required
// no dependencies, Node 18+ global fetch - PROJECT_ID is a dubbed project
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const auth = await fetch('https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token', {
  method: 'POST',
  headers: { Authorization: 'Basic ' + btoa(`${CLIENT_ID}:${CLIENT_SECRET}`) },
  body: new URLSearchParams({ grant_type: 'client_credentials' }),
}).then((r) => r.json());
const h = { Authorization: `Bearer ${auth.access_token}`, 'Content-Type': 'application/json' };
const url = `https://api.rask.ai/v2/projects/${PROJECT_ID}`;
const info = () => fetch(`${url}/lipsync`, { headers: h }).then((r) => r.json());

let i = await info();
if (i.check_face_task_status !== 'done') {
  await fetch(`${url}/check_face`, { method: 'PUT', headers: h });
  while ((i = await info()).check_face_task_status !== 'done') await sleep(10000);
}
if (!i.video_has_face) throw new Error('No face found');

await fetch(`${url}/lipsync`, {
  method: 'PUT', headers: h,
  body: JSON.stringify({ is_multiple_speakers: true, is_free_lipsync: false }),
});

while (!['done', 'failed'].includes((i = await info()).lipsync_task_status)) {
  console.log(i.lipsync_task_progress, i.tasks_in_lipsync_queue);
  await sleep(15000);
}
console.log(i.lipsync_result_path);
fetch, Node 18+
plain HTTP
# 1. token (valid 1 hour - cache and reuse it)
TOKEN=$(curl -s -X POST https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token \
  -u "$CLIENT_ID:$CLIENT_SECRET" -d "grant_type=client_credentials" | jq -r .access_token)

# 2. face check on the dubbed project - start it only if it has not run yet
curl -s https://api.rask.ai/v2/projects/$ID/lipsync -H "Authorization: Bearer $TOKEN" \
  | jq '{check_face_task_status, video_has_face}'
curl -s -X PUT https://api.rask.ai/v2/projects/$ID/check_face -H "Authorization: Bearer $TOKEN"

# 3. run lip-sync (set is_free_lipsync to true for the free preview minute)
curl -s -X PUT https://api.rask.ai/v2/projects/$ID/lipsync \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"is_multiple_speakers": true, "is_free_lipsync": false}'

# 4. follow progress, then download the result
curl -s https://api.rask.ai/v2/projects/$ID/lipsync -H "Authorization: Bearer $TOKEN" \
  | jq '{lipsync_task_status, lipsync_task_progress, lipsync_result_path}'
PUT /v2/projects/{id}/lipsync
OAuth 2.0 client credentials
<?php
use GuzzleHttp\Client;

$http = new Client(['base_uri' => 'https://api.rask.ai']);
$json = fn($r) => json_decode($r->getBody(), true);
$auth = $json((new Client)->post('https://rask-prod.auth.us-east-2.amazoncognito.com/oauth2/token',
    ['auth' => [$clientId, $clientSecret], 'form_params' => ['grant_type' => 'client_credentials']]));
$h = ['Authorization' => 'Bearer ' . $auth['access_token']];
$info = fn() => $json($http->get("/v2/projects/$projectId/lipsync", ['headers' => $h]));

$i = $info();
if ($i['check_face_task_status'] !== 'done') {
    $http->put("/v2/projects/$projectId/check_face", ['headers' => $h]);
    while (($i = $info())['check_face_task_status'] !== 'done') sleep(10);
}
if (!$i['video_has_face']) exit('No face found');

$http->put("/v2/projects/$projectId/lipsync", ['headers' => $h,
    'json' => ['is_multiple_speakers' => true, 'is_free_lipsync' => false]]);

while (!in_array(($i = $info())['lipsync_task_status'], ['done', 'failed'])) sleep(15);
echo $i['lipsync_result_path'];
Guzzle 7
plain HTTP
Python is the official SDK. The Node, PHP and cURL tabs are plain HTTP examples against the same endpoints.
Progress

How a lip-sync task reports progress

The lip-sync info endpoint reports the face check, the task status, the queue position and progress from 0 to 100. You only need to act on three groups.

Success, stop here

lipsync_task_status: done
lipsync_task_progress: 100
Lip-sync is complete. The download link is in lipsync_result_path.

In progress, keep polling

check_face_task_status: started
lipsync_task_status: started
lipsync_task_progress: 0-99
tasks_in_lipsync_queue
While progress is null, the task is queued: tasks_in_lipsync_queue shows how many tasks are ahead.

Failure, stop and report

lipsync_task_status: failed
check_face_task_status: failed
video_has_face: false
A video without a visible face cannot be lip-synced. For a failed task, contact support with your project_id.
Lip-sync task, as GET /lipsync reports it
GET /v2/projects/{project_id}/lipsync
check face     check_face_task_status  done     video_has_face  true
queued         lipsync_task_status     started  tasks_in_lipsync_queue  2
processing     lipsync_task_progress   0 → 50 → 100
complete       lipsync_task_status     done
               lipsync_result_path     https://...

Or skip polling entirely

Point us at a webhook endpoint and every state transition arrives as a POST, so your queue reacts to each change the moment it happens. Webhook delivery is part of enterprise onboarding.
Read the docs
preserveAspectRatio="xMidYMid meet" aria-hidden="true" role="img">
Output

Lip-synced video, dub and subtitles

Download the generated video from lipsync_result_path. The project keeps the dubbed video without lip-sync and the translated subtitles in SRT and VTT, and on eligible plans the translated voice as separate audio tracks.
Field
Format
What it is
lipsync_result_path
MP4
The lip-synced video, ready to publish
voiceover
WAV
Translated speech only, for your own mix
translated_audio
WAV
Translated speech mixed with the original background audio
translation_srt_path
.srt
Subtitles in the target language
translation_vtt_path
.vtt
Same subtitles as WebVTT, for web players
translated_video
MP4
The dubbed video without lip-sync, for comparison
Control

Edit the dub, then generate an updated lip-synced version

After you change text, timing or voices, re-dub the changed segments and generate lip-sync for the updated video. Re-dubbing and each new full-video lip-sync generation are billed separately.

Segment-level editing

Segment-level editing
Edit individual transcript segments and re-dub the changed lines. To update the lip movements, generate a new lip-sync for the full video; this is billed separately.
Fetch the transcript and its segments
Patch the source text and we re-translate it
Patch the translated text directly to keep your own wording
Adjust timestamps, add segments, delete segments
Reassign the voice attached to any speaker

Glossaries for terminology

Glossaries for terminology
Attach a glossary so brand names and technical terms are spoken the same way in every dub, and lip-sync follows that same wording.

A glossary covers one language pair and applies across all regional variants of that target language, so one Spanish glossary serves es-mx and es-es alike. The project records glossary_version, so later edits never silently change finished work.
Input

Supported file formats

Upload any of seven video formats, or send a public URL. Audio-only files have no face on screen, so lip-sync applies to video formats only.

Video

7 formats
.mp4
.mov
.mkv
.avi
.webm
.m4v
.3gp

Audio

4 formats
.mp3
.wav
.flac
.m4a

Metadata before you spend

The media endpoint returns duration, file size, video codec, frame rate, resolution, audio codec, sample rate and channel layout. Validate a file and estimate cost in advance, because dubbing and lip-sync are each billed by the minute of video length.
Security and compliance

Faces and voices, handled securely

Brask Inc holds SOC 2 Type II and operates under GDPR, and publishes live control status in its Trust Center. Videos with faces and voices are processed in a closed AI environment.

Certifications

SOC 2 Type II
GDPR
CAI and C2PA member

Encryption

TLS 1.2 in transit
AES-256 at rest
SSE-S3 managed keys
Key rotation at least every 12 months

Your content

Closed AI environment
Never used to train general models
Full export, no lock-in

Access

SSO and SAML
Role-based access control
OAuth 2.0 client credentials

Enterprise

Data residency options
Custom SLAs
Dedicated account manager

Voice rights

Synthetic voices with usage rights
Consent-based voice cloning
AI safety controls, human in the loop
Comparison

Rask API vs a lip-sync pipeline you build yourself

A standalone lip-sync API needs a finished audio track in the target language. Rask produces the dubbed track and syncs the lips in one project. Here is the same result, assembled two ways.
What you need
Rask API
Assembled from parts
A translated audio track to sync
Created by the same project
Transcription, translation and voice vendors first
Face check before you pay
Automatic, once per project
Your own face detection step
Several faces on screen
is_multiple_speakers: true
Per-face tracking you build and tune
Preview before the full run
One free minute per account
Pay for every test run
Re-sync after a script edit
A new full-video lip-sync generation, billed separately
Pay again for the whole file
Background music and effects
Preserved under the new voice
A source-separation step you build
Subtitles in .srt and .vtt
Included
Generated from your own timing data
Queue visibility
Queue position and progress 0-100
Whatever your vendor exposes
Rate limits
None
Whatever your strictest vendor enforces
Billing
Per full-video lip-sync generation
Per run, per vendor
FAQ

Lip-sync API: common questions

Can't find what you're looking for? Contact sales.

Is there a lip-sync API?

Can I lip-sync a video without dubbing it first?

Can I lip-sync my own audio track?

Does the API generate talking avatars from an image or a prompt?

What does a video need for lip-sync?

How is lip-sync billed?

Is there a free lip-sync preview?

Does lip-sync work with several speakers?

How do I track lip-sync progress?

How long does lip-sync take?

What happens after I edit the dub?

What if lip-sync fails?

How do I get API access and an API key?

Start dubbing in one afternoon

Make your first request today, and talk to us when you need volume terms or a language nobody else has.