> For AI agents: the documentation index is at https://platform.detectif.ai/llms.txt. Every page is also available as Markdown by adding `.md` to its path.

# Stream protocol

Connect a live call over the WebSocket: audio formats, the messages you send, the events you receive, limits, credits and close codes.

One WebSocket per call. Stream the callee's audio for the whole call, and receive the first answer and every change.

```
wss://api.noesis.detectif.ai/v1/detect/stream
```

Open the socket as soon as the call is answered and stream the callee's audio in real time. Keep it open for the whole call: a transfer to a person can happen minutes in. Authenticate with the `Authorization` header on the upgrade request, or with a stream token in `?token=`; see [Authentication](https://platform.detectif.ai/docs/noesis/authentication/).

## Query parameters

| Parameter | Values | Default |
|---|---|---|
| `encoding` | `mulaw`, `alaw` or `pcm_s16le` | `mulaw` |
| `sample_rate` | `8000`, `16000`, `22050`, `24000`, `44100` or `48000` | `8000` |
| `channels` | `1`: the callee only. `2`: the callee and your agent on separate channels | `1` |
| `remote_channel` | The callee's channel when `channels=2`: `0` or `1` | `0` |
| `language` | Optional hint, `en` or `hi`. Leave it out for automatic (English, Hindi, Hinglish) | automatic |
| `call_id` | Your reference for the call, up to 200 characters, returned in every event | none |
| `transcripts` | `true` to also receive what the callee says, for debugging | `false` |
| `record` | `false` to not record this call. See [Data and recording](https://platform.detectif.ai/docs/noesis/data-and-recording/) | `true` |
| `token` | A stream token, for clients that cannot set the `Authorization` header | none |

`callback_url` and other webhook settings are not available; a socket that sets them is refused with `unsupported_parameter`. Answers arrive on the socket, and later through [sessions](https://platform.detectif.ai/docs/noesis/sessions-and-usage/).

## Audio

| Audio | Accepted |
|---|---|
| Encoding | G.711 μ-law (`mulaw`), G.711 A-law (`alaw`), or 16-bit linear PCM, little-endian (`pcm_s16le`) |
| Sample rate | 8, 16, 22.05, 24, 44.1 or 48 kHz |
| Channels | Mono (the callee only), or stereo with your agent on the other channel |
| Framing | Binary WebSocket frames, or Twilio and Plivo media-stream JSON |

Compressed audio such as Opus, MP3, AAC or G.729 is not accepted: decode it to PCM first. Most telephony platforms already give you μ-law or PCM. Set `encoding` and `sample_rate` to what you send; the audio is not converted.

## Messages you send

| Message | When |
|---|---|
| Binary audio | The callee's audio, 20 to 100 ms per frame, in real time from the moment the call is answered |
| `{"event": "media", ...}` | Twilio or Plivo media-stream JSON, unchanged. The `inbound` track is the callee; an `outbound` track is your agent. A `start` message sets `call_id` from the call's ID; `stop` ends the call |
| `{"type": "agent.speech", "state": "started"}` and `{"type": "agent.speech", "state": "ended"}` | Optional: when your agent starts and stops talking, if its audio is not on a second channel or track. It helps Noesis recognise a person sooner |
| `{"type": "end"}` | On hang-up. You receive the `summary` and the socket closes |
| `{"type": "ping"}` | Keep-alive, answered with `{"type": "pong"}` |

## Events you receive

```jsonc
// once, when the socket opens
{"type": "ready", "session_id": "9f2c4e7a-...", "call_id": "your-id", "t": 0.0,
 "audio": {"encoding": "mulaw", "sample_rate": 8000, "channels": 1}}

// the first answer, then one event per change
{"type": "classification", "seq": 2, "t": 13.0,
 "label": "hold", "category": "machine", "confidence": 0.8,
 "previous_label": "ivr", "change": "transition",
 "session_id": "9f2c4e7a-...", "call_id": "your-id"}

// after {"type": "end"} (or a Twilio or Plivo "stop")
{"type": "summary", "label": "human", "category": "human", "timeline": [ ... ]}
{"type": "usage", "billed_seconds": 27}
```

| Field | Meaning |
|---|---|
| `session_id` | Noesis's ID for the call. Keep it to look the call up later in [sessions](https://platform.detectif.ai/docs/noesis/sessions-and-usage/) |
| `call_id` | Your reference, from the URL or the Twilio or Plivo `start` message |
| `seq` | Increases with every event. Act on the highest you have seen |
| `t` | Seconds since the socket opened |
| `label`, `category` | The answer. See [Answers](https://platform.detectif.ai/docs/noesis/#answers) |
| `confidence` | 0 to 1 |
| `change` | `initial`: the first answer. `transition`: person and machine swapped, or a transfer to hold. `refinement`: a more specific label, such as `machine` to `voicemail` |
| `billed_seconds` | In the `usage` event at the end: the seconds of call audio billed for this call |
| Other fields | Events may carry more fields than these. Ignore the ones you do not use |

With `transcripts=true` you also receive `{"type": "transcript", ...}` events with what the callee said.

### A typical transfer

| Time | Label | On the line |
|---|---|---|
| 3.2 s | `ivr` | "For credit cards, press one." |
| 13.0 s | `hold` | "Please hold while we connect you to a customer associate." |
| 23.4 s | `human` | "Hi, this is Priya. How can I help you?" |

## Ending a call

Send `{"type": "end"}` when the call ends (a Twilio or Plivo `stop` does the same). You receive the `summary` and a `usage` event, then the socket closes with code `1000`.

If you close the socket without `end`, the call still ends and its result is kept in the [session](https://platform.detectif.ai/docs/noesis/sessions-and-usage/), but the `summary` is not sent to you.

## Limits

| Limit | Value |
|---|---|
| Binary audio frame | Up to 192,000 bytes; 20 to 100 ms of audio is best |
| JSON message | Up to 384,000 characters |
| One call | Up to 2 hours |
| Silence on the socket | The socket closes after 15 minutes without any message |
| Calls at the same time | Set by your plan. A call over the limit is refused with `429` and close code `4429` |

A frame or message that is too large gets an `error` event with status `413`; the socket stays open.

## Credits

Live calls use credits for the seconds of audio Noesis receives. A call that sends less than a second of audio is not billed. Your balance is on the dashboard Overview page.

If your balance runs out during a call, you receive `{"type": "error", "code": "insufficient_credits", ...}` and the socket closes with code `4402`. The audio already received is billed. Carry on with the call without detection, and add credits before the next one.

## Close codes

| Close code | HTTP status in the `error` event | Meaning | What to do |
|---|---|---|---|
| `1000` | | The call ended normally after `end`, or the socket was silent for 15 minutes, or the call passed 2 hours (`408`) | Open one socket per call |
| `1003` | `400` | A parameter is invalid or not available, such as `callback_url`; the message names it | Fix the URL |
| `1013` | `503` | Busy | Carry on with the call without detection |
| `4401` | `401` | The key or stream token is missing, invalid, expired or already used | Check the key, or create a new token |
| `4402` | `402` | The credit balance ran out (`insufficient_credits`) | Add credits in the dashboard |
| `4403` | `403` | The key is not allowed to use Noesis, or the email address is not verified | Use a key with `noesis:detect` |
| `4429` | `429` | Too many calls at once, or new calls too fast | Wait for a call to end |

Before closing, Noesis sends an `error` event with the details. See [Errors](https://platform.detectif.ai/docs/noesis/errors/).