Realtime client

Ultravox is a multimodal LLM that understands text and human speech without separate ASR. It converts audio to LLM’s high-dimensional space, responding faster than systems with separate ASR and LLM components.

You can integrate Ultravox WebSocket API client in your Voximplant application via the Ultravox VoxEngine module. It provides you a client interface to communicate with Ultravox models in real time and produce both audio and text transcriptions.

1. Require the Modules.Ultravox module in your VoxEngine scenario.
2. Create a Ultravox.WebSocketAPIClient instance (via the Ultravox.createWebSocketAPIClient method) and provide your parameters.
3. Send media between Call and Ultravox.WebSocketAPIClient (via the *.sendMedia or VoxEngine.sendMediaBetween methods).
4. Listen to the Ultravox.WebSocketAPIClient events (via the Ultravox.WebSocketAPIEvents event list) and implement you application business logic.

Here is the complete scenario example for your reference:

Q: The AI voice on the phone is choppy and high-pitched. What do I do?
A: Set the inputSampleRate property on the Ultravox side to 16000, this should fix the issue.