Docs

Running a model

sync.Video to video

sync-3

Full-scene lip synchronization with global face understanding and obstruction handling

POSTauray-serverless/sync/3-0
openapi.jsonllms.txt

One of Auray's catalogue models: the same key, the same credits, the same job lifecycle as every other address.

Its request shape is published below. Whether it runs right now is GET /v1/catalogue's word, with no key needed: its addresses say connected for an address that runs, and a submit to any other is refused before anything is charged — described answers 501 model_not_connected, paused 503 model_disabled.

Category video-to-video · Scope video:write (or external:write on an older key) · Price Credits per call: 93. A request that sends inputs.video or inputs.audio or settings.segments.[].audio is refused before any charge: it has no price on this model. Quoted when you submit, at the most the request can cost; the charge settles to what the job cost and never above the quote. The submit's answer carries the quote as credits_quoted, and credits_charged says what has been taken.

#Input

FieldTypeRequiredDefaultDescription
inputsobjectyes—
inputs.videostringyes—Video input (an http(s) URL; upload a local file first and send its read URL). A URL.
inputs.audiostringno—Audio input (an http(s) URL; upload a local file first and send its read URL). A URL.
settingsobjectno—
settings.temperaturenumberno0.5Expressiveness of lip sync and facial movements. Documented range: 0 to 1.
settings.activeSpeakerDetectionobjectno—
settings.activeSpeakerDetection.autoDetectbooleanno—Automatically detect and target the active speaker.
settings.activeSpeakerDetection.boundingBoxesnumber[][]no—Per-frame face bounding boxes across the clip. Each box is [x1, y1, x2, y2].
settings.activeSpeakerDetection.coordinatesnumber[]no—[x, y] point on the target speaker's face in the selected frame.
settings.activeSpeakerDetection.frameNumberintegerno—Frame index corresponding to the provided face coordinates. Documented range: 0 or more.
settings.emotionstringno—Emotional tone for performance re-animation. One of "happy", "sad", "angry", "disgusted", "surprised", "neutral".
settings.modestringno—Controls the region of facial animation applied during lip sync. One of "lips", "face", "head".
settings.occlusionDetectionbooleanno—Enable occlusion handling for obstructed faces.
settings.segmentsobject[]no—
settings.segments[].audiostringno—Audio input (an http(s) URL; upload a local file first and send its read URL). A URL.
settings.segments[].audioEndTimenumberno—End time in seconds within the source audio file.
settings.segments[].audioStartTimenumberno0Start time in seconds within the source audio file. Documented range: 0 or more.
settings.segments[].endTimenumberno—End time in seconds for the segment. Must be greater than startTime and within the source video's duration.
settings.segments[].startTimenumberno—Start time in seconds for the segment. Documented range: 0 or more.
settings.syncModestringno"cut_off"Synchronization strategy when audio and video durations don't match. One of "bounce", "cut_off", "silence", "remap".
outputTypestringno"URL"Video output type. One of "URL".
outputFormatstringno"MP4"Specifies the file format of the generated output. The available values depend on the task type and the specific model's capabilities. - `JPG`: Best for photorealistic images with smaller file sizes (no transparency). - `PNG`: Lossless compression, supports high quality and transparency (alpha channel). - `WEBP`: Modern format providing superior compression and transparency support. - `MP4`: Widely supported video container (H.264), recommended for general use. - `WEBM`: Optimized for web delivery. - `MOV`: QuickTime format, common in professional workflows (Apple ecosystem). - `GIF`: Animated image format (no audio), suitable for short loops or previews. - `MP3`: Compressed audio, smaller file size. - `WAV`: Uncompressed, high-quality audio. - `FLAC`: Lossless compression. - `OGG`: Open-source compressed audio format (Vorbis codec). - `SVG`: Scalable Vector Graphics. - `TIFF`: High-quality output supporting layers. - `GLB`: Binary glTF format for 3D models, including geometry, textures, and PBR materials. > [!NOTE] > **Transparency**: If you are using features like background removal or LayerDiffuse that require transparency, you must select a format that supports an alpha channel (e.g., `PNG`, `WEBP`, `TIFF`). `JPG` does not support transparency. One of "MP4", "WEBM", "MOV".
outputQualityintegerno95Compression quality of the output. Higher values preserve quality but increase file size. Documented range: 20 to 99.
safetyobjectno—
safety.checkContentbooleanno—Enable or disable content safety checking. Increases total generation time. Always on: a false is sent as true.
safety.modestringno"fast"Safety checking mode for video generation. Only fast or full is sent here: any other value is sent as fast. One of "fast", "full".
idempotency_keystringno—Your own name for this request. Send the same one twice and the second call is answered with the first request rather than charged again. Idempotency-Key: as a header is accepted as an alias — that is the spelling a retrying proxy replays, and the one to use when running a model on auray.run, where a request that outlives its connection is answered 504 and recovered by replay. Omit it and the platform mints one. 1 to 64 characters.

#Rules

Across fields, as the model's documentation states them. Each is judged before anything is charged — warned with a Deprecation header until enforcement starts — and listed as JSON Schema under x-auray-rules in the openapi.json.

  • Do not send settings.activeSpeakerDetection.boundingBoxes together with settings.activeSpeakerDetection.coordinates.
  • Send exactly one of inputs.audio or settings.segments.
  • Send settings.activeSpeakerDetection.frameNumber whenever settings.activeSpeakerDetection.coordinates is sent. Send settings.activeSpeakerDetection.coordinates whenever settings.activeSpeakerDetection.frameNumber is sent. settings.segments must hold at least 1 item. In every item of settings.segments: settings.segments[].startTime, settings.segments[].endTime and settings.segments[].audio are required.

#Example

curl -X POST https://queue.auray.run/auray-serverless/sync/3-0 \
  -H "Authorization: Bearer $AURAY_API_KEY" \
  -H "content-type: application/json" \
  -d '{
  "inputs": {
    "video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
    "audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
  }
}'

The same body on https://auray.run runs synchronously and answers the result — unless it usually takes longer than a run may hold its connection, which is refused 400 model_not_synchronous before anything is charged. Options — ?webhook=, ?wait=, priority and timeout headers — are on Submit a request.

#Machine contracts

  • openapi.json — this page as an OpenAPI 3.1 document, public.
  • llms.txt — the same, written for an agent.

Start typing to search every page of the documentation.

Up arrow Down arrow to moveEnter to openEscape to close