What each device does

Every device in a room has one of three jobs: model host, worker or guest. This page says what each does and how to see it in the room.

Role	Who	What it does
Model host	One device, picked when the model starts (how)	Owns the conversation. Tokenizes, embeds, runs its own layers, runs the output head, samples each token and drafts the next ones. Runs the Code agent and the question queue.
Worker	Devices dealt a layer range	Runs its range of layers on its own GPU and passes the result to the next device.
Guest	Devices with no layers	Chats and asks. That covers devices without WebGPU, phones left out of the split, devices For speed didn’t need, and pooled serve API clients.

Anyone in the room can ask. The room answers one question at a time. Up to 10 questions wait in the host’s queue, at most two per device.

One token

host      embed the last token → hidden state
   ↓ a few KB over WebRTC (about 10 KB on the 27B, 4 KB on the MoE)
laptop    layers 1–22           ─┐
desktop   layers 23–43           ├─ each device runs its layers on its own GPU
phone     layers 44–64          ─┘
   ↓ back to the host
host      final norm → output head → sample → next token (and draft the ones after it)

One trip round the room is a lap. A lap costs every device’s compute plus one network hop per device, which is why For speed uses as few devices as it can. The prompt goes round in pipelined batches of 16 tokens, and speculative decoding checks several drafted tokens per lap, so the text is the same as plain decoding. Both are explained in Architecture.

Two things the host does for speed:

• One GPU submit per token. The output head, the next token’s embedding and the host’s own layers run as one command buffer with one readback.
• Phone GPU wake. At the start of each lap, the host tells every phone that holds layers that a frame is on its way. The phone keeps its GPU busy until it arrives, for at most 50 ms, so the GPU hasn’t clocked down. ?wake=0 turns this off.

Seeing who does what

Device chips. The header shows a chip per device: a status dot, an icon, the name and the memory it lends. Click one to open its card. API clients show as “API client · tool” with “no layers” and aren’t counted as devices.

The device screen. The black Serve API button opens a dark page whose left half shows what this device is doing:

Title	Meaning
Standing by	No model is running yet.
Loading 40%	Downloading and loading its layers.
Serving	Holds layers, for example “Layers 1–20 · 2.4 GB · Qwen3.6 35B MoE”.
Not holding layers	The model runs on the other devices. This one can still ask.
Room over / Host reconnecting	The host left, or its tab closed and the room is waiting a minute for it.

While it works, a hop dot flashes each time a pass reaches this device, next to its tok/s, the tokens it handled and the milliseconds per hop. It also warns when the tab is in the background, the screen can sleep or the battery is low. See Serve API.