Self-hosted audio protocol | Dial — Communication Stack for AI Agents

In Self-Hosted audio mode (config.type: "audio" on the Self-Hosted configuration), Dial connects the raw call audio to a WebSocket server you host. Where the LLM protocol sends you transcripts and asks for text turns — with Dial running speech-to-text and text-to-speech — the audio protocol hands you the sound itself, both directions. Your server brings the entire voice stack: a speech-to-speech model (OpenAI Realtime, Gemini Live, …) or your own STT → LLM → TTS chain.

These pages are the complete contract — you can build a conformant server from them alone. The SDKs ship a ready-made server helper (signature verification, frame codec, keepalive) so you start from handlers, not from a WebSocket loop.

Connection

Dial opens one WebSocket per call to your configured URL with the call id appended to the path:

wss://<your-host>/<your-path>/<call_id>

call_id is Dial’s own call identifier — stable for the life of the call, and the only call identifier you ever receive. If the connection drops mid-call, Dial reconnects to the same path and sends a fresh call_connected with reconnect: true (see Reconnection).

All frames are JSON text, discriminated by a type field, snake_case throughout. Audio rides inside media frames as base64.

Authentication

Identical to the LLM protocol: every connection carries

X-Dial-Signature: t=<unix_seconds>,v1=<hex_signature>

where hex_signature is HMAC-SHA256(secret, "<t>.<call_id>") as a hex digest, with the signing secret from your Self-Hosted configuration. Verify on connect with a constant-time comparison and reject stale timestamps. The SDK helpers do this for you.

Audio formats

The audio encoding in each direction is set on your Self-Hosted configuration and echoed in every call_connected frame — there is no in-band renegotiation. Both directions support the same five formats:

Format	Encoding	Sample rate	Notes
`mulaw_8000`	G.711 μ-law, 8-bit	8 kHz	Default. Native telephony; zero-transcode into most realtime voice APIs.
`alaw_8000`	G.711 A-law, 8-bit	8 kHz
`l16_8000`	Linear PCM, 16-bit signed little-endian	8 kHz
`l16_16000`	Linear PCM, 16-bit signed little-endian	16 kHz	e.g. Gemini Live input.
`l16_24000`	Linear PCM, 16-bit signed little-endian	24 kHz	e.g. OpenAI Realtime / Gemini Live PCM output.

All audio is mono. Dial sends caller audio in your inbound format in ~20 ms frames; you send agent audio in your outbound format, any chunking (Dial buffers and paces playback).

Message types

From Dial → your server:

call_connected

First frame on every connection — call metadata, formats, reconnect flag.

media

Caller audio, base64, ~20 ms cadence.

dtmf

A keypad digit the caller pressed.

mark

Echo of your checkpoint — playback reached it.

duration_warning

~30s before the max-duration cut — wrap up.

call_ended

The call is over; the socket closes next.

From your server → Dial:

media

Agent audio to play to the caller.

clear

Flush queued playback — barge-in.

mark

Set a playback checkpoint.

end_call

Hang up gracefully.

Both ways: ping_pong — the keepalive: Dial pings every ~2 s; echo it back. Miss enough pongs and Dial treats the link as dead and reconnects.

Reconnection

If the socket drops mid-call, the call itself stays up: Dial discards caller audio while disconnected (never replays it — the conversation is live), and re-dials your endpoint with backoff for up to ~15 seconds. The new connection starts with call_connected carrying reconnect: true — restore your per-call state keyed by call_id and resume streaming. If reconnection fails past the budget, the call ends and your Call record reflects it. After you send end_call, reconnect attempts for that call are refused.