Chat Completions

POST http://127.0.0.1:8080/v1/chat/completions. Any API key works unless you set a token.

curl

curl http://127.0.0.1:8080/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "pooled",
  "messages": [
    {"role": "system", "content": "Answer in one sentence."},
    {"role": "user", "content": "What is a KV cache?"}
  ],
  "stream": true,
  "stream_options": {"include_usage": true}
}'

Python

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="pooled")
stream = client.chat.completions.create(
    model="pooled",
    messages=[{"role": "user", "content": "What is a KV cache?"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Parameters

Parameter	Behavior
messages	system, developer, user, assistant and tool messages. System and developer messages before the first other message are the system prompt. A later one joins the user turn just before it, or, after an assistant turn, the next user turn.
model	Any string. The room serves its one model.
max_completion_tokens, max_tokens	Default 16384. Above 65536 is capped, not refused. Always capped by the context left. max_completion_tokens wins if both are set.
temperature	0 to 2. 0 is greedy. Absent: the room’s own setting (0.8 if you set only top_k).
top_k	1 or more; above 64 is capped to 64. Default 40 when you set temperature.
stop	Up to 4 strings, 64 characters each. Applies to the answer text, not to tool calls.
stream, stream_options.include_usage	Server-sent events. include_usage adds a usage chunk before [DONE].
reasoning_effort	low, medium, high, xhigh, max turn thinking on. none or minimal turn it off. Absent: off.
chat_template_kwargs.enable_thinking	Turns thinking on or off when reasoning_effort is absent.
tools, tool_choice, parallel_tool_calls	See Tool calling.
response_format	json_object or json_schema. See Structured output.
top_p, seed, store, metadata, user, service_tier, prompt_cache_key	Accepted and ignored.

Thinking

Thinking is off unless you ask for it, on every API. With thinking on, the model reasons before it answers. The reasoning comes back as message.reasoning_content (streamed as delta.reasoning_content), separate from content.

Only Qwen3.8 uses the effort level. The other models think the same way at any level.

When you send an assistant message back with reasoning_content, the room gets that reasoning as part of the turn. If it matches what the room wrote, the room reuses its caches.

Response

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "pooled/qwen3.6-35b-moe",
  "choices": [{
    "index": 0,
    "message": { "role": "assistant", "content": "A KV cache stores ...", "refusal": null },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 31, "completion_tokens": 42, "total_tokens": 73
  }
}

finish_reason is one of:

• stop: the model finished, or hit a stop string.
• tool_calls: the answer ends with tool calls.
• length: the answer reached max_tokens, or the room’s context is full.

When the room reused its caches, usage.prompt_tokens_details.cached_tokens counts the prompt tokens it already held from an earlier request. usage.completion_tokens_details.reasoning_tokens counts thinking tokens when thinking was on.

Streaming

The stream is standard Chat Completions chunks: a role chunk, then reasoning, content and tool-call chunks as the room writes them, the finish chunk, the usage chunk if asked, and data: [DONE].

While a request waits or the room reads a long prompt, the bridge sends an SSE comment after 10 s of silence (: keep-alive or : queued, 2 ahead).

If something fails after the stream started, you get an error chunk and no [DONE].

Not supported

Images and files become a short note, [image omitted: this model reads text only], so a pasted screenshot doesn’t break a session. These are a 400:

• custom tools, and the deprecated functions, function_call and role: "function"
• audio input
• n greater than 1
• a non-zero presence_penalty or frequency_penalty, a non-empty logit_bias
• logprobs, top_logprobs
• prediction, web_search_options
• an assistant message last (prefill)

Errors use OpenAI’s shape, {"error": {"message", "type", "param", "code"}}. See status codes.