Noesis / Integrations

FreeSWITCH

Fork a FreeSWITCH channel's audio to a WebSocket relay with an audio-fork module, and forward it to Noesis.

FreeSWITCH can fork a channel's audio to a WebSocket with an audio-fork module, such as mod_audio_fork. The module sends 16-bit PCM as binary frames, after an optional JSON metadata message. Run a relay that forwards the frames to Noesis.

1. Fork the callee's audio to your relay

Start the fork on the channel of the person you called, for example from fs_cli or the event socket, once the call is answered:

fs_cli
uuid_audio_fork <uuid> start wss://relay.example.com/freeswitch mono 16k {"call_id":"<uuid>"}

mono sends that channel's audio only, and 16k sets 16 kHz. Check your module's documentation for its exact options; other audio-fork modules work the same way.

2. Forward the audio to Noesis

The relay uses noesis_relay.py from the relay helper:

Python
# freeswitch_relay.py: FreeSWITCH audio fork -> your server -> Noesis
import asyncio, json

import websockets

from noesis_relay import NoesisCall

async def on_answer(event):
    if event["type"] == "classification":
        print(event["call_id"], event["label"])     # your own logic

async def handle(fork):
    call = None
    try:
        async for message in fork:
            if call is None:                        # metadata, if the fork sends it first
                meta = json.loads(message) if isinstance(message, str) else {}
                call = NoesisCall(on_answer, encoding="pcm_s16le", sample_rate=16000,
                                  call_id=meta.get("call_id"))
                await call.open()
            if isinstance(message, bytes):
                await call.send(message)            # 16-bit PCM
    except websockets.ConnectionClosed:
        pass
    if call is not None:
        await call.close()

async def main():
    async with websockets.serve(handle, "0.0.0.0", 8080):
        await asyncio.Future()                      # run until stopped

asyncio.run(main())

For an 8k fork, use sample_rate=8000. To not record calls, create each call with record="false" as well.