Running a model
sync.Video to videosync-3
Full-scene lip synchronization with global face understanding and obstruction handling
One of Auray's catalogue models: the same key, the same credits, the same job lifecycle as every other address.
Its request shape is published below. Whether it runs right now is GET /v1/catalogue's word, with no key needed: its addresses say connected for an address that runs, and a submit to any other is refused before anything is charged — described answers 501 model_not_connected, paused 503 model_disabled.
Category video-to-video · Scope video:write (or external:write on an older key) · Price Credits per call: 93. A request that sends inputs.video or inputs.audio or settings.segments.[].audio is refused before any charge: it has no price on this model. Quoted when you submit, at the most the request can cost; the charge settles to what the job cost and never above the quote. The submit's answer carries the quote as credits_quoted, and credits_charged says what has been taken.
#Input
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
inputs | object | yes | — | |
inputs.video | string | yes | — | Video input (an http(s) URL; upload a local file first and send its read URL). A URL. |
inputs.audio | string | no | — | Audio input (an http(s) URL; upload a local file first and send its read URL). A URL. |
settings | object | no | — | |
settings.temperature | number | no | 0.5 | Expressiveness of lip sync and facial movements. Documented range: 0 to 1. |
settings.activeSpeakerDetection | object | no | — | |
settings.activeSpeakerDetection.autoDetect | boolean | no | — | Automatically detect and target the active speaker. |
settings.activeSpeakerDetection.boundingBoxes | number[][] | no | — | Per-frame face bounding boxes across the clip. Each box is [x1, y1, x2, y2]. |
settings.activeSpeakerDetection.coordinates | number[] | no | — | [x, y] point on the target speaker's face in the selected frame. |
settings.activeSpeakerDetection.frameNumber | integer | no | — | Frame index corresponding to the provided face coordinates. Documented range: 0 or more. |
settings.emotion | string | no | — | Emotional tone for performance re-animation. One of "happy", "sad", "angry", "disgusted", "surprised", "neutral". |
settings.mode | string | no | — | Controls the region of facial animation applied during lip sync. One of "lips", "face", "head". |
settings.occlusionDetection | boolean | no | — | Enable occlusion handling for obstructed faces. |
settings.segments | object[] | no | — | |
settings.segments[].audio | string | no | — | Audio input (an http(s) URL; upload a local file first and send its read URL). A URL. |
settings.segments[].audioEndTime | number | no | — | End time in seconds within the source audio file. |
settings.segments[].audioStartTime | number | no | 0 | Start time in seconds within the source audio file. Documented range: 0 or more. |
settings.segments[].endTime | number | no | — | End time in seconds for the segment. Must be greater than startTime and within the source video's duration. |
settings.segments[].startTime | number | no | — | Start time in seconds for the segment. Documented range: 0 or more. |
settings.syncMode | string | no | "cut_off" | Synchronization strategy when audio and video durations don't match. One of "bounce", "cut_off", "silence", "remap". |
outputType | string | no | "URL" | Video output type. One of "URL". |
outputFormat | string | no | "MP4" | Specifies the file format of the generated output. The available values depend on the task type and the specific model's capabilities. - `JPG`: Best for photorealistic images with smaller file sizes (no transparency). - `PNG`: Lossless compression, supports high quality and transparency (alpha channel). - `WEBP`: Modern format providing superior compression and transparency support. - `MP4`: Widely supported video container (H.264), recommended for general use. - `WEBM`: Optimized for web delivery. - `MOV`: QuickTime format, common in professional workflows (Apple ecosystem). - `GIF`: Animated image format (no audio), suitable for short loops or previews. - `MP3`: Compressed audio, smaller file size. - `WAV`: Uncompressed, high-quality audio. - `FLAC`: Lossless compression. - `OGG`: Open-source compressed audio format (Vorbis codec). - `SVG`: Scalable Vector Graphics. - `TIFF`: High-quality output supporting layers. - `GLB`: Binary glTF format for 3D models, including geometry, textures, and PBR materials. > [!NOTE] > **Transparency**: If you are using features like background removal or LayerDiffuse that require transparency, you must select a format that supports an alpha channel (e.g., `PNG`, `WEBP`, `TIFF`). `JPG` does not support transparency. One of "MP4", "WEBM", "MOV". |
outputQuality | integer | no | 95 | Compression quality of the output. Higher values preserve quality but increase file size. Documented range: 20 to 99. |
safety | object | no | — | |
safety.checkContent | boolean | no | — | Enable or disable content safety checking. Increases total generation time. Always on: a false is sent as true. |
safety.mode | string | no | "fast" | Safety checking mode for video generation. Only fast or full is sent here: any other value is sent as fast. One of "fast", "full". |
idempotency_key | string | no | — | Your own name for this request. Send the same one twice and the second call is answered with the first request rather than charged again. Idempotency-Key: as a header is accepted as an alias — that is the spelling a retrying proxy replays, and the one to use when running a model on auray.run, where a request that outlives its connection is answered 504 and recovered by replay. Omit it and the platform mints one. 1 to 64 characters. |
#Rules
Across fields, as the model's documentation states them. Each is judged before anything is charged — warned with a Deprecation header until enforcement starts — and listed as JSON Schema under x-auray-rules in the openapi.json.
- Do not send
settings.activeSpeakerDetection.boundingBoxestogether withsettings.activeSpeakerDetection.coordinates. - Send exactly one of
inputs.audioorsettings.segments. - Send
settings.activeSpeakerDetection.frameNumberwheneversettings.activeSpeakerDetection.coordinatesis sent. Sendsettings.activeSpeakerDetection.coordinateswheneversettings.activeSpeakerDetection.frameNumberis sent.settings.segmentsmust hold at least 1 item. In every item ofsettings.segments:settings.segments[].startTime,settings.segments[].endTimeandsettings.segments[].audioare required.
#Example
curl -X POST https://queue.auray.run/auray-serverless/sync/3-0 \
-H "Authorization: Bearer $AURAY_API_KEY" \
-H "content-type: application/json" \
-d '{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}'import { createClient } from "@auray/client";
const auray = createClient(); // reads AURAY_API_KEY
const result = await auray.subscribe("auray-serverless/sync/3-0", {
input: {
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
},
onQueueUpdate: (update) => console.log(update.status),
});
console.dir(result, { depth: null });from auray_client import Client
auray = Client() # reads AURAY_API_KEY
result = auray.subscribe(
"auray-serverless/sync/3-0",
input={
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
},
on_queue_update=lambda update: print(update["status"]),
)
print(result){
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "run_model",
"arguments": {
"address": "auray-serverless/sync/3-0",
"input": {
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}
}
}
}const res = await fetch("https://queue.auray.run/auray-serverless/sync/3-0", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.AURAY_API_KEY}`,
"content-type": "application/json",
},
body: JSON.stringify({
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}),
});
console.log(await res.json());import os, requests
r = requests.post(
"https://queue.auray.run/auray-serverless/sync/3-0",
headers={"Authorization": f"Bearer {os.environ['AURAY_API_KEY']}"},
json={
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
},
)
r.raise_for_status()
print(r.json())package main
import (
"bytes"
"fmt"
"io"
"net/http"
"os"
)
func main() {
body := []byte(`{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}`)
req, err := http.NewRequest("POST", "https://queue.auray.run/auray-serverless/sync/3-0", bytes.NewReader(body))
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("AURAY_API_KEY"))
req.Header.Set("Content-Type", "application/json")
res, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
out, _ := io.ReadAll(res.Body)
fmt.Println(res.Status, string(out))
}<?php
$ch = curl_init("https://queue.auray.run/auray-serverless/sync/3-0");
curl_setopt_array($ch, [
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
"Authorization: Bearer " . getenv("AURAY_API_KEY"),
"Content-Type: application/json",
],
CURLOPT_POSTFIELDS => json_encode([
"inputs" => [
"video" => "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio" => "https://v3b.auray.media/examples/samples/electronic-loop.mp3",
],
], JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE),
]);
$response = curl_exec($ch);
echo curl_getinfo($ch, CURLINFO_HTTP_CODE), " ", $response, PHP_EOL;require "net/http"
require "json"
uri = URI("https://queue.auray.run/auray-serverless/sync/3-0")
req = Net::HTTP::Post.new(uri)
req["Authorization"] = "Bearer #{ENV.fetch("AURAY_API_KEY")}"
req["Content-Type"] = "application/json"
req.body = {
"inputs" => {
"video" => "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio" => "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}.to_json
res = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) { |http| http.request(req) }
puts res.code, res.bodyimport java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
public class Main {
public static void main(String[] args) throws Exception {
String body = """
{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}""";
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://queue.auray.run/auray-serverless/sync/3-0"))
.header("Authorization", "Bearer " + System.getenv("AURAY_API_KEY"))
.header("Content-Type", "application/json")
.method("POST", HttpRequest.BodyPublishers.ofString(body))
.build();
HttpResponse<String> response = HttpClient.newHttpClient()
.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.statusCode() + " " + response.body());
}
}using System.Net.Http.Headers;
using System.Text;
using var client = new HttpClient();
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", Environment.GetEnvironmentVariable("AURAY_API_KEY"));
using var request = new HttpRequestMessage(HttpMethod.Post, "https://queue.auray.run/auray-serverless/sync/3-0");
var body = """
{
"inputs": {
"video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
"audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
}
}
""";
request.Content = new StringContent(body, Encoding.UTF8, "application/json");
using var response = await client.SendAsync(request);
Console.WriteLine($"{(int)response.StatusCode} {await response.Content.ReadAsStringAsync()}");The same body on https://auray.run runs synchronously and answers the result — unless it usually takes longer than a run may hold its connection, which is refused 400 model_not_synchronous before anything is charged. Options — ?webhook=, ?wait=, priority and timeout headers — are on Submit a request.
#Machine contracts
- openapi.json — this page as an OpenAPI 3.1 document, public.
- llms.txt — the same, written for an agent.