Docs

Running a model

sync.Video to video

sync. lipsync-2-pro

High-resolution diffusion-based lip-sync editing up to 4K

POSTauray-serverless/sync/lipsync-2-pro-1
openapi.jsonllms.txt

One of Auray's catalogue models: the same key, the same credits, the same job lifecycle as every other address.

Its request shape is published below. Whether it runs right now is GET /v1/catalogue's word, with no key needed: its addresses say connected for an address that runs, and a submit to any other is refused before anything is charged — described answers 501 model_not_connected, paused 503 model_disabled.

Category video-to-video · Scope video:write (or external:write on an older key) · Price Credits per call: 58. A request that sends inputs.video or inputs.audio or providerSettings.sync.segments.[].audio is refused before any charge: it has no price on this model. Quoted when you submit, at the most the request can cost; the charge settles to what the job cost and never above the quote. The submit's answer carries the quote as credits_quoted, and credits_charged says what has been taken.

#Input

FieldTypeRequiredDefaultDescription
inputsobjectyes—
inputs.videostringyes—Video input (an http(s) URL; upload a local file first and send its read URL). A URL.
inputs.audiostringyes—Single audio source applied to the whole video, or a list of identified sources that segments reference by id. A URL.
providerSettingsobjectno—
providerSettings.syncobjectno—
providerSettings.sync.activeSpeakerDetectionbooleanno—Automatically target the speaking face in a multi-person clip.
providerSettings.sync.occlusionDetectionEnabledbooleanno—Enable occlusion handling for obstructed faces.
providerSettings.sync.segmentsobject[]no—
providerSettings.sync.segments[].startTimenumberno—Start time in seconds for the segment. Documented range: 0 or more.
providerSettings.sync.segments[].endTimenumberno—End time in seconds for the segment. Must be greater than startTime.
providerSettings.sync.segments[].refstringno—Identifier of the inputs.audio entry this segment uses. Documented range: 1 or more characters.
providerSettings.sync.segments[].audiostringno—Audio source URL for this segment. A URL.
providerSettings.sync.segments[].audioStartTimenumberno0Start time in seconds within the source audio file. Documented range: 0 or more.
providerSettings.sync.segments[].audioEndTimenumberno—End time in seconds within the source audio file. Documented range: 0 or more.
providerSettings.sync.syncModestringno"bounce"Synchronization strategy when audio and video durations don't match. One of "bounce", "loop", "cut_off", "silence", "remap".
providerSettings.sync.temperaturenumberno0.5Expressiveness of lip sync and facial movements. Documented range: 0 to 1.
outputTypestringno"URL"Video output type. One of "URL".
outputFormatstringno"MP4"Specifies the file format of the generated output. The available values depend on the task type and the specific model's capabilities. - `JPG`: Best for photorealistic images with smaller file sizes (no transparency). - `PNG`: Lossless compression, supports high quality and transparency (alpha channel). - `WEBP`: Modern format providing superior compression and transparency support. - `MP4`: Widely supported video container (H.264), recommended for general use. - `WEBM`: Optimized for web delivery. - `MOV`: QuickTime format, common in professional workflows (Apple ecosystem). - `GIF`: Animated image format (no audio), suitable for short loops or previews. - `MP3`: Compressed audio, smaller file size. - `WAV`: Uncompressed, high-quality audio. - `FLAC`: Lossless compression. - `OGG`: Open-source compressed audio format (Vorbis codec). - `SVG`: Scalable Vector Graphics. - `TIFF`: High-quality output supporting layers. - `GLB`: Binary glTF format for 3D models, including geometry, textures, and PBR materials. > [!NOTE] > **Transparency**: If you are using features like background removal or LayerDiffuse that require transparency, you must select a format that supports an alpha channel (e.g., `PNG`, `WEBP`, `TIFF`). `JPG` does not support transparency. One of "MP4", "WEBM", "MOV".
outputQualityintegerno95Compression quality of the output. Higher values preserve quality but increase file size. Documented range: 20 to 99.
safetyobjectno—
safety.checkContentbooleanno—Enable or disable content safety checking. Increases total generation time. Always on: a false is sent as true.
safety.modestringno"fast"Safety checking mode for video generation. Only fast or full is sent here: any other value is sent as fast. One of "fast", "full".
idempotency_keystringno—Your own name for this request. Send the same one twice and the second call is answered with the first request rather than charged again. Idempotency-Key: as a header is accepted as an alias — that is the spelling a retrying proxy replays, and the one to use when running a model on auray.run, where a request that outlives its connection is answered 504 and recovered by replay. Omit it and the platform mints one. 1 to 64 characters.

#Rules

Across fields, as the model's documentation states them. Each is judged before anything is charged — warned with a Deprecation header until enforcement starts — and listed as JSON Schema under x-auray-rules in the openapi.json.

  • providerSettings.sync.segments must hold at least 1 item. In every item of providerSettings.sync.segments: providerSettings.sync.segments[].startTime, providerSettings.sync.segments[].endTime and providerSettings.sync.segments[].ref are required. In every item of providerSettings.sync.segments: send providerSettings.sync.segments[].audioEndTime whenever providerSettings.sync.segments[].audioStartTime is sent. In every item of providerSettings.sync.segments: send providerSettings.sync.segments[].audioStartTime whenever providerSettings.sync.segments[].audioEndTime is sent.

#Example

curl -X POST https://queue.auray.run/auray-serverless/sync/lipsync-2-pro-1 \
  -H "Authorization: Bearer $AURAY_API_KEY" \
  -H "content-type: application/json" \
  -d '{
  "inputs": {
    "video": "https://v3b.auray.media/examples/samples/coffee-pour.mp4",
    "audio": "https://v3b.auray.media/examples/samples/electronic-loop.mp3"
  }
}'

The same body on https://auray.run runs synchronously and answers the result — unless it usually takes longer than a run may hold its connection, which is refused 400 model_not_synchronous before anything is charged. Options — ?webhook=, ?wait=, priority and timeout headers — are on Submit a request.

#Machine contracts

  • openapi.json — this page as an OpenAPI 3.1 document, public.
  • llms.txt — the same, written for an agent.

Start typing to search every page of the documentation.

Up arrow Down arrow to moveEnter to openEscape to close