How OpenAI Realtime actually works over WebRTC — an operator’s primer

If you’ve tried to wire up OpenAI’s Realtime API in 2025 and hit an empty 400 on the SDP exchange, you’re not alone. The GA release in mid-2025 changed the endpoint shape from the beta docs everyone learned. Here’s the current mental model.

The three-step handshake 1. **Mint an ephemeral token** on your server. Endpoint: `POST /v1/realtime/client_secrets` with a JSON body wrapping a `session` object (model, voice, instructions, tools, transcription, VAD). Response: `{ value: "ek_...", expires_at }`. Never expose your real API key to the browser. 2. **Exchange SDP over WebRTC**. Browser creates `RTCPeerConnection`, adds mic tracks, creates offer. POST the offer SDP as `multipart/form-data` (field name `sdp`) to `/v1/realtime/calls` — no query parameters, no JSON. Model is inferred from the ephemeral token. 3. **Open the data channel**. Once ICE gathering completes and the peer connection is up, a data channel named `oai-events` streams JSON events both ways: user transcripts, agent transcripts (streaming deltas), tool calls, response completion.

Event names to know - `conversation.item.input_audio_transcription.completed` — the user’s finished utterance transcribed by Whisper. - `response.output_audio_transcript.delta` — tokens of the agent’s spoken reply, streaming. - `response.output_audio_transcript.done` — end of the current spoken turn. - `response.function_call_arguments.done` — the model wants you to run a tool. - `error` — log these, but they usually surface a session config issue you can fix in the mint payload.

Tool-use loop When you get `response.function_call_arguments.done { call_id, name, arguments }`, execute the function locally (calculator, end-call) or via a server round-trip (search, email, calendar). Then send two events back on the data channel: `conversation.item.create` with type `function_call_output` and the call_id + JSON result, then `response.create` to nudge the model to continue speaking.

Common footguns - Sending `?model=...` on the SDP POST — returns empty 400. Model belongs in the ephemeral token only. - Using `application/sdp` content-type — the GA endpoint expects multipart. Use FormData. - Skipping the ICE-gathering-complete wait — causes intermittent 400s on flaky networks. - Setting `output_modalities: ["audio", "text"]` — rejected. Use `["audio"]`; transcripts still stream through data channel events.

Why WebRTC over WebSocket? Jitter buffering, opus codec negotiation, native ICE traversal, and — crucially — built-in echo cancellation from the browser. If you’re on the server-to-server side (Twilio Media Streams → OpenAI), you’ll want the WebSocket transport instead; the events are the same.

Want a working reference? Fork the Callora demo — all this handshake is open-source in our client.