# Voice notes

> Receiving a person's voice notes with their transcript, fetching the audio, and answering with a voice note — your own audio or one nmbr speaks for you.
>
> Markdown mirror of https://nmbr.ai/developers/docs/voice/ — part of the nmbr Agent API docs (https://nmbr.ai/developers/docs/). Spec: https://nmbr.ai/developers/openapi.yaml · Site index: https://nmbr.ai/llms.txt

## Voice in

A person's voice note arrives as a normal `message.received` with `message.type: "voice"`, `message.audio: { data, duration }` and — because nmbr transcribes every voice note in the background, after the event was queued — **no transcript yet**. The words follow as their own event:

```json
{ "type": "message.transcript", "payload": { "conversationId": "…", "messageId": "…", "from": { … }, "transcript": "call me back about the invoice" } }
```

Treat the pair as one message: keep the `messageId`, act when `message.transcript` lands (usually within seconds). Both are masked without the person's `messages:read` — `transcript` is `null` with `accessDenied`, so your agent knows a voice note exists that it may not read.

`audio.data` is an `/objects/…` path only nmbr's apps can open. To run your own speech model, fetch the file with `GET /agent/v1/messages/:messageId/audio` (the SDK's `agent.getMessageAudio(id)` returns the `Response`); it streams the audio with its content type and supports `Range`. The other side's notes need their `messages:read`; your own need nothing.

## Voice out

`POST /messages` with `type: "voice"` in one of two ways:

**Your own audio.** `audioData` is a base64 data URL — mp3, m4a/aac, webm, ogg/opus or wav, at most 10 MB — plus `audioDuration` in seconds. nmbr stores the file and the bubble plays it like any voice note; the row never keeps an inline data URL. Add `transcript` so the words show under the note and travel with it: nmbr does not transcribe agent audio. A path you got from nmbr (`/objects/…`, e.g. forwarding a note) is accepted as is. Anything else is `400 invalid_audio`.

**Let nmbr speak.** `tts: { text, voice? }` synthesizes the audio server-side (OpenAI `tts-1`; voices `alloy`, `echo`, `fable`, `onyx`, `nova` — the default — and `shimmer`; up to 4096 characters), stores the mp3, sets the transcript to the text and, when you leave `content` out, uses the text as the caption too. The duration is estimated from the text so the bubble shows a length before playback.

```bash
curl -s -X POST https://nmbr.ai/api/agent/v1/messages -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"conversationId":"…","type":"voice","tts":{"text":"Your build finished. Two tests failed — want the log?","voice":"onyx"}}'
```

```ts
await agent.speak({ conversationId }, "Your build finished. Two tests failed — want the log?", { voice: "onyx" });
```

`503 tts_unavailable` means speech isn't configured on this deployment (send your own audio); `502 tts_failed` means the synthesis call failed — retry or fall back to text. A voice note counts as a message for limits and ceilings like any other.

## Answering a voice note with a voice note

The loop this phase was built for: on `message.transcript`, think, then `agent.speak({ conversationId }, answer)`. If the answer is long, stream the text with the [stream](/developers/docs/streaming/) first and follow with the spoken version — or skip the audio when the transcript came back `null`, and say so.