OpenAI-compatible

The API answers the OpenAI chat completions protocol at its own base URL — and, over the same door, the Responses API and the legacy completions some frameworks call by default. Any client that speaks them — the official openai packages for Python and JavaScript, and the frameworks built on them: the AI SDK's @ai-sdk/openai, the OpenAI Agents SDK, LangChain, LlamaIndex's OpenAILike — works by changing two settings: the base URL, and the key. One framework needs a third for two things: the AI SDK's default provider needs store: false on a conversation of more than one turn, and in a tool loop with a custom tool (openai.tools.customTool) — its loops of function tools run without it.

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.auray.ai/openai/v1", api_key=os.environ["AURAY_API_KEY"])

reply = client.chat.completions.create(
    model="auray-serverless/model/openai-gpt-5-mini",
    messages=[{"role": "user", "content": "Name a brass instrument."}],
)
print(reply.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.auray.ai/openai/v1", apiKey: process.env.AURAY_API_KEY });

const reply = await client.chat.completions.create({
  model: "glm-5-3-flash",
  messages: [{ role: "user", content: "Name a brass instrument." }],
});
console.log(reply.choices[0].message.content);

GET /openai/v1/models lists every model this surface takes, with what each one can do under capabilities. Requests and answers are OpenAI's shapes, not the ones in the API reference; errors use OpenAI's nested error object, with our own code in error.code and our error catalogue explaining it. Inside error, beside OpenAI's fields, retry and charged say whether resending can help and whether anything was taken — the same advice every other refusal on this API carries — and the code's own fields ride with them: required and accepted on a scope refusal, required and balance on an empty wallet, ceiling and spent on a key ceiling, plan on a plan gate. Every answer, refusals included, carries an X-Request-Id header — quote it when you write to us.

#The key, and the plan

The key is the whole string Account → API Keys showed you once, when you created it: it starts with auray_sk_ and is 71 characters. The list in Account names each key by its ID — the twelve characters after the prefix — and the ID is not the key: sent as one, with or without the prefix, it is refused 401 invalid_api_key with a sentence that says so. A key cannot be shown again; if you only have its ID, create a new one.

Completions need a plan whose keys may run them. On a plan where they may not, GET /openai/v1/models still answers, because it only reads, and every POST is refused 403 plan_has_no_api_generate before the model is looked at. Every POST here spends the plan's interactive bucket, and GET /v1/plans shows it closed by either of these:

• limits.api.may_generate is false
• limits.api.interactive.burst is 0

#Two kinds of model

Auray's chat models — glm-5-3-flash, deepseek-v4-1-flash and kimi-k3. They need a key with text:write (or the older chat:write), stream token by token, and are priced by the token exactly as /v1/chat prices them. They take tools.

The catalogue's language models are named by their address, auray-serverless/model/openai-gpt-5-mini; the slug is accepted too, and the address is what comes back as model. They need text:write too (or the older external:write), and each call is charged exactly as a request to that address on the queue host would be. The model runs to completion before the answer is sent, so stream: true delivers the whole answer as one burst of events rather than token by token. While it runs, a stream that is not answered within the first seconds is answered 200 at once and sent an SSE comment line (: keepalive) whenever nothing was written for a while, which every client skips, so no proxy between you and us drops the connection as idle; a failure that comes after that is OpenAI's in-stream error chunk, followed by data: [DONE], rather than an HTTP status. They do not take tools.

A model name with -thinking-<level> and then -fast after it — what an editor such as Cursor sends when a thinking level or Fast is picked — is read as the model it extends. The level (none, minimal, low, medium, high, xhigh or max) becomes the request's reasoning_effort unless the body names one; -fast changes nothing, as none of these models has a faster tier.

#What our own models take

• system and developer messages are sent as the model's own system messages, where they stand in the conversation.
• Text as a string or as a list of text parts. An image part is refused, as these models read text.
• reasoning_effort: none, minimal or low (our lightest rung), medium, high, xhigh or max (our deepest) — every word OpenAI's clients and editors send — and our own light, balanced and deep. Any other word is 400 unknown_effort, whose message lists these. Reasoning comes back as reasoning_content beside content. Send it back on the assistant message while a tool loop is still answering the same question: it reaches the model for the exchange being answered, and is dropped from older exchanges.
• max_tokens or max_completion_tokens, which can only lower the model's ceiling for your plan — GET /openai/v1/models prints it per model as max_output_tokens. That ceiling is your plan's, or, where the plan's is more than the model can write before a turn's time runs out, the model's own: one turn may run 800 seconds, in which glm-5-3-flash and kimi-k3 write 44,032 tokens at their measured pace. Each must be a whole number of at least 1; anything else is 400 bad_value, naming the spelling you sent. finish_reason is length when it cut the answer. Reasoning counts against the same ceiling, and against the time one turn may run: when either runs out before a word of answer or a tool call, the turn is refunded and answered 400 output_limit_reached with x-should-retry: false (a stream ends in that error chunk, after the reasoning it sent) — send it again with a lower reasoning_effort. A completion that is not streamed has usually answered 200 by the time a model thinks that long (see below), so it reads the same error object as the body under that 200, with no choices: check for error before reading choices.
• response_format json_object and json_schema — what the AI SDK's generateObject and LangChain's with_structured_output send — are answered through a function of their own: the schema (any object for json_object) is offered as one tool the request forces by name, and the call's arguments come back as content — alone, without any text the model wrote around the call — streamed as delta.content, with finish_reason: "stop". The schema's root must be an object, and it meets the same bounds as a schema tool_choice forces (below). strict is read and not enforced: the model writes the arguments, and your parser checks them. A model that answers with JSON as text instead of the call is answered with that JSON (a Markdown code fence taken off); one that answers with neither, or calls the function with arguments that are not a JSON object, is refunded — but for the reasoning it wrote first, which you read, and the prompt that reasoning read, at their token price with no per-turn base — and answered 422 response_format_unmet (a stream ends in that error chunk). A stream sends the arguments as they come, so arguments that only their end shows are no object — one never closed — have reached you by then: that stream still ends in the error chunk, and is charged for what it sent. The error's charged is refunded when nothing was kept and stands when something was. Text the model wrote around the call is not sent and not charged. A format sent beside tools, or a type other than text, json_object or json_schema, is 400 response_format_unsupported; a schema past the bounds below is 400 bad_value.
• temperature, top_p, stop, seed, the penalties and n are accepted and not applied: the models run at their own settings, and the answer names the ones you sent in X-Auray-Ignored-Params.
• Every completion carries X-Auray-Job-Id — the turn, readable at GET /v1/jobs/{id} — and one that is not streamed carries X-Auray-Credits-Charged, what the turn cost once it settled. A refusal made before a turn existed names no job.

#How long a request can be

No single message has a length limit here — an attached file inside the user message is fine. The whole request body is capped at one mebibyte. The conversation is held to your plan's history budget, counted in bytes, not tokens: each message as the JSON it is sent as, in UTF-8, plus 64, and 128 for the request. The system prompt is always sent and never counted. The newest exchange — a user message and everything after it — counts first and is always sent whole, even alone over the budget; an agent loop is one exchange, its task and every call and result since, so it is never trimmed. Older exchanges are kept, newest first, while the total fits, and the rest are left out a whole exchange at a time, so a tool result never loses the call it answers. The budget is far smaller than a model's context_window, and a long newest exchange leaves that much less of it for the history before it: GET /models prints yours as history_budget_bytes on each of our three models. An answer whose history was trimmed says how many earlier messages were left out in X-Auray-History-Trimmed (dropped=<n>, shortened=0); if your client needs them, summarise them into the newest exchange. Before the model runs, the request is reserved at a bound on every byte sent plus the output ceiling, and settled down to what the model reported; a wallet or key ceiling that cannot cover the reserve is refused before anything runs.

#Tools on our own models

tools takes function tools, written either way OpenAI's two APIs write them — {"type": "function", "function": {"name", "description", "parameters"}} or the flat {"type": "function", "name", "description", "parameters"} — and tool_choice takes auto, none, required or one tool by name. A call comes back the way OpenAI returns one:

• without streaming, as message.tool_calls, with content: null when the model said nothing else, and finish_reason: "tool_calls";
• streaming, as delta.tool_calls: the first piece of each call carries its index (from 0, one per call, in the order the calls began), its id, its type and its name, and the pieces after it carry its arguments. The last choice chunk says finish_reason: "tool_calls", then comes the usage chunk and data: [DONE].

Answer a call with a tool message carrying the call's tool_call_id, straight after the assistant message that made it (other tool results may sit between them), and send the whole conversation again — the assistant message with its tool_calls, content null or not, then the results. The last message may be a tool result. Send each id back exactly as you received it: some are spelt call_au_… because the model's own id was not in OpenAI's shape, and that spelling is how the model gets its own back.

A custom tool — OpenAI's tool whose input is free text, sometimes with a grammar — is offered to the model as a function of one string, input, with the grammar in its description; a call to it comes back as {"type": "custom", "custom": {"name", "input"}}, the shape you declared. The model writes to the grammar rather than being held to it, so your tool is the judge of its input. When streaming, a custom call streams as OpenAI streams one: its first piece carries custom.name and the start of custom.input, and the pieces after it carry more of custom.input.

A tool of a hosted type (web_search, file_search, …) is not offered here and is refused with 400 tools_unsupported, as are the legacy functions and function_call fields. A definition that cannot be sent as written — a name outside 1 to 64 letters, digits, _ or -, two tools with one name, parameters that is not an object — or a tool_choice naming a tool the request does not offer is 400 bad_value. parallel_tool_calls is accepted and not applied: the model may call several tools in one answer. strict is accepted and not passed on: the model reads each schema and writes its call freely, and your tool checks what it receives.

tool_choice: "required", or one tool by name, is not enforced by these models' servers — measured, two of the three answered in text instead — so the answer is checked here. Text the model writes before the call is held back and sent with the call when it comes, and a call to any tool but the one named is never sent at all — so a framework that runs each call as it arrives never runs one the request did not ask for — nor charged, beside the named call or without it. A model that writes about a page of text instead is stopped there, and a turn that ends without the call — in text, or with only calls to other tools — is refunded and answered 422 tool_choice_unmet (a stream ends in that error chunk, and none of the text is sent). The refund is all of the turn when the model wrote no reasoning first. Otherwise it is all of the turn but that reasoning, which a stream has already sent and the job keeps, and the prompt it read — charged at their token price, with no per-turn base: the reasoning is the model's reading of your prompt, and every other turn that delivers part of an answer pays for its prompt too. The error's charged is refunded when nothing was kept and stands when something was, and X-Auray-Credits-Charged on an answer that is not streamed, or the job's credits_charged, says how much. Send it again with a new idempotency key, or with a prompt that asks for the tool.

The choice is still sent to the model, which may compile the schemas it chooses from into a grammar other requests share. Those schemas — every tool's for required, the named tool's otherwise — are refused 400 bad_value when one is over 16,384 bytes, all of them together over 65,536, nested deeper than 32 levels of JSON, uses pattern or patternProperties, or sets minItems, maxItems, minLength, maxLength, minProperties, maxProperties, minContains or maxContains above 1,024. With tool_choice: "auto" nothing is compiled and none of this applies.

If the model's server refuses the tools as sent — before answering, or inside the stream before a word of answer — the request is 400 tools_rejected and nothing is charged. Streamed, that arrives as OpenAI's in-stream error chunk, followed by data: [DONE].

Tool calls are billed as the output tokens they are, and the tool list as prompt tokens on every request that carries it.

#The Responses API and legacy completions

POST /openai/v1/responses and POST /openai/v1/completions are the same door in OpenAI's two other text protocols: each request is translated into a chat completion, run and charged exactly as one — on either kind of model, with everything above — and answered in the protocol it came in. The AI SDK's createOpenAI(…)(model) and the OpenAI Agents SDK call the first by default, and LlamaIndex's OpenAILike the second unless it is given is_chat_model=True; neither needs a setting changed.

On /responses:

• instructions is the system prompt and input the conversation — a string, or message, function_call, function_call_output (and custom tool call) items, with reasoning items riding along as the model's own reasoning. tools and tool_choice are taken as above, in the flat spelling Responses writes function tools in; max_output_tokens is max_tokens, reasoning.effort is reasoning_effort, and text.format is response_format.
• The answer is a response whose output holds a reasoning item, a message and each function_call, and whose usage is input_tokens and output_tokens; a limit that cut it makes it incomplete, with incomplete_details.reason. Streamed, it is the Responses events, from response.created to response.completed (or response.incomplete, or an error event and response.failed), each numbered.
• Nothing is stored, so what names a stored thing is refused 400 bad_value before anything is priced: previous_response_id, conversation, a stored prompt and background. An item_reference in input that names our own reasoning, or the text a round wrote beside its tool call, is left out — the AI SDK sends both by default, and the round's call and its output, sent in full, are what the model reads — and one that names an answer is refused, since the next question would lose what it follows. So is one that names a custom tool's call, which the AI SDK sends for a tool made with openai.tools.customTool: that reference is the call itself, its name and input nowhere else in the request. Send the whole conversation instead: with the AI SDK, set providerOptions: { openai: { store: false } } on a conversation of more than one turn, and on a tool loop with a custom tool, so it sends its earlier answers and its calls rather than references to them. A loop of function tools inside one generateText or streamText call needs no setting.

On /completions, the one prompt is the user's message, and the answer is a text_completion with the answer's text, its finish_reason and its usage — streamed, text_completion chunks and data: [DONE]. Reasoning has no place in that protocol and is not sent; the job keeps it (GET /v1/jobs/{id}). More than one prompt, a prompt of token ids, suffix, echo, best_of above 1 and logprobs change what an answer means and are refused 400 bad_value.

#What a catalogue model takes

• system and developer messages become the model's system prompt; image_url parts in a user message reach the models that take images. The URL must be an http(s) one: an inline data: image — how OpenAI's clients send a local file — is refused 400 bad_value before anything is charged. Host the image, or upload it with POST /v1/uploads and send the read URL it answers.
• reasoning_effort maps onto the model's own levels, listed as capabilities.reasoning_effort; a word the model lacks is mapped to its nearest level. Reasoning comes back as reasoning_content, beside content, on the models that return it apart — and on a model that writes it into its answer inside a leading <think>…</think> block, which is taken out of content and returned there instead. An unclosed <think> is left in content as written. GET /v1/jobs/{id} returns the job's own text, exactly as the model wrote it.
• response_format takes json_object and json_schema where capabilities.response_format lists them. A schema is strict only when you set strict: true, as in OpenAI's own API.
• tools, functions, a tool_choice naming a tool, an assistant message with tool_calls and a tool result are refused with 400 tools_unsupported rather than dropped, and the message names our own models, which take them.

#What a call costs

A catalogue language model is priced by the tokens a call can use. When the request arrives it is quoted at its prompt — the messages, the tool schemas, each image and each document — and at the whole answer it may write: the max_tokens you send, up to the model's published maximum, or, when you send none, the default its address page prints — the model's own where it is smaller than 8,192 tokens, and 8,192 otherwise. That quote is held from your credits, and when the model finishes the charge comes down to what it actually used; it never goes above the quote. The model is sent the same max_tokens, so an answer longer than that ends with finish_reason: "length" — send a larger max_tokens for a longer answer, and it is reserved for. A request that sends a document is quoted at the model's whole context window — a document's length is not known until the model reads it, and the window is the most it can read — and settled at what it read. The rate per million tokens, and every number the quote counts by, is each model's x-auray-price in its openapi.json and the price line on its address page. A prompt that writes or reads a cache is counted whole in usage.prompt_tokens, the part read from the cache in prompt_tokens_details.cached_tokens.

#Waiting, retries and idempotency

A catalogue completion holds a connection open while the model runs, and your account has a small number of those. When every one is in use the next completion is refused with 429 wait_capacity and a Retry-After header, before anything is charged — the official clients retry it on their own.

A completion that outlives its connection answers 504 run_timed_out after about 280 seconds. It is still running and is charged once, so the error carries charged: true, the idempotency_key it ran under and a status_url to read it at — and the response says x-should-retry: true with Retry-After: 1, because a resend of the same request is handed this same job rather than a second one: it waits on the job again from the top of a new connection and answers the completion when it finishes, with no second charge. The official clients' two retries follow a long completion that way for three connections. A dispatch that could not be confirmed, and a finished job whose result could not be read, say x-should-retry: false. To make a deliberate retry a replay of the same job at any time, send an Idempotency-Key header — or idempotency_key in the body, which the Python client sends with extra_body. A stream already answered 200 (above) carries the same timeout as its in-stream error chunk, code: "run_timed_out" with the same fields, and no header: the clients raise it rather than retry it, so send the same request again yourself to be handed the same job.

Not every framework reads x-should-retry: the Vercel AI SDK, LangChain JS and LlamaIndex resend a 5xx several times on their own, and send no key. So a catalogue completion that names no key is given one: it is derived from the API key that sent it, the model and the request body, and holds for 10 minutes. An identical request sent again within that time — the framework's resend — is handed the same job: it waits on it again while it runs, and answers it once it has finished, instead of starting and charging a second one. A job that failed, and was refunded, is not handed back: the next identical request is a fresh attempt. The key used is in the error's idempotency_key. A key you send always wins, so to ask the same question twice on purpose inside that window and get two answers, send a different Idempotency-Key with each.

On our own chat models a long answer keeps its connection alive: a stream carries an SSE comment line whenever nothing was written for a while — the model may reason for minutes before its first word — and a completion that is not streamed and not ready within the first seconds answers 200 at once, its body beginning with spaces every JSON parser skips. Once committed it cannot change its status, so a refusal or a failure that comes later is OpenAI's error object — error with its message, type and code, and no choices — under that 200, and the cost header is not sent (GET /v1/usage has it). Closing the connection stops the answer; it is charged for what was produced, and an answer that produced nothing — no text, no reasoning, no tool call — and reported no usage is refunded in full.

The same key works on our own chat models and on embeddings: a resend while the first request is still being answered is 409 in_flight, and one after it has settled is 409 key_settled — never a second charge. On our chat models the key names the turn (GET /v1/jobs/chat_<key> reads it), so a turn refused after it was charged — model_busy, chat_upstream, chat_start_failed, each with its charge given back — has used its key: the error says retry: "new_key" and, because you named the key, x-should-retry: false, so the official clients stop there instead of resending into key_settled. Send the turn again under a new key. On embeddings the key names no stored result, so a key whose batch came back whole is not used up: see Embeddings. On embeddings, a body idempotency_key outside 1 to 64 letters, digits, ., - or _ is 400 invalid_idempotency_key, as it always was. Anywhere else a key outside that set is not refused — these routes ignored the header, and chat completions the body field too, before they read them, and a client that already sent one keeps working: it is folded into a key inside the set (its SHA-256), so the same key is still the same request.

An empty wallet, or a key this request would take past its monthly credit ceiling, is insufficient_quota on a 429 whichever model answered it, also marked x-should-retry: false: nothing changes by retrying within the minute. The code says which of the two it was — insufficient_credits or key_credit_ceiling. So is a model we have switched off, 503 vendor_disabled with our reason in message: it stays off until we switch it back on, so pick another model rather than waiting on this one.

#Embeddings

POST /openai/v1/embeddings answers OpenAI's embeddings protocol, so client.embeddings.create works unchanged:

vectors = client.embeddings.create(model="qwen3-embedding-8b", input=["a brass trumpet", "a tuba"])
print(len(vectors.data[0].embedding))

• The model is qwen3-embedding-8b (its Hugging Face id, Qwen/Qwen3-Embedding-8B, is accepted too), and every vector has 4096 dimensions.
• Input is a string or an array of up to 2048 strings, so the AI SDK's embedMany and LangChain send their default batches unchanged. Each string has a window of 40,960 tokens of its own, counted by the model — a batch of many short strings is fine, and a string past the window is 400 input_too_long, with nothing charged. Pre-tokenised input (arrays of integers) is refused with 400 input_invalid; LangChain Python's OpenAIEmbeddings sends it by default, so construct it with check_embedding_ctx_length=False.
• The format is OpenAI's: encoding_format "float" (the default) or "base64" — the little-endian float32 bytes of each vector, which is what the official Node client asks for — and dimensions from 32 to 4096 returns the head of each vector, renormalised to unit length.
• The key needs embeddings:write, a scope of its own — a key minted to chat cannot run embeddings. The plan must include language models (403 plan_has_no_chat otherwise).
• The price is by the token, reserved before the call against a bound on every byte sent and settled down to the tokens the model counted. The answer's credits_charged field and its X-Auray-Credits-Charged header both say what it cost. A wallet that cannot cover the reserve is insufficient_quota on 429, with required, balance and affordable_tokens — how many tokens the balance would cover — inside error.
• The first request after a quiet spell can take up to 3 minutes; requests within 30 minutes of a previous call answer at once. Give your client a longer timeout than that. Past it the answer is 503 embeddings_warming, with Retry-After and retry_after in the error, and nothing charged — send the identical request again after that interval, your idempotency key included; the official clients do it on their own.
• Idempotency works as above: in_flight while the same key's batch is being embedded, key_settled once it has been served. Vectors are not stored, so a served key cannot be answered again — send a new one. A batch that was refused with its whole charge given back — embeddings_warming, embeddings_upstream — does not use up its key: the identical resend is a fresh attempt, charged only if it is served, and a key is tried that way up to 8 times. Past that, or when a promotion paid for the call, the error says retry: "new_key" with x-should-retry: false: send the batch under a new key.

#What is kept

A catalogue completion is a job like any other. Its messages and its answer are stored with the job and deleted after 3 days, as every job's request and results are, and the job appears among your runs in the desktop. The token counts the model reported are kept after the words are gone, for usage.

On our own chat models each request is stored as a conversation of its own — the newest user message (none on a request that ends in a tool result, whose question was stored by the request that asked it) and the answer, with why it ended — archived when it is created, so it never appears in the desktop's Messages app — Playground's Your runs lists it, marked API — and deleted on the same 3-day rule. Tool calls and tool results are not stored: your client holds the conversation and sends it again with every request.