Go to CONFIGURATION > Advanced Settings > LLM Streaming.
Turn on the toggle to enable LLM streaming.

Note: LLM Streaming is disabled by default.

Benefits of LLM Streaming

Reduced Latency: Real-time streaming of rephrased responses significantly decreases the time taken for the user to receive the modified content.
Improved User Experience: Faster delivery of rephrased content creates a more natural and interactive conversation flow.
Simplified Process: Streamlining the rephrasing process reduces complexity and potential points of failure.

LLM Streaming for Text-to-Speech (TTS) Providers

LLM streaming is enabled for the following TTS providers:

PlayHT
ElevenLabs
Deepgram

Note: If a GenAI node’s response contains special characters, such as punctuation marks, symbols, or brackets, the system might fail to play the prompt during a call, resulting in silence for the caller.

On this Page

LLM Streaming

Supervisors can enable real-time streaming of LLM responses, reducing latency and enhancing the user experience, particularly for voice interactions.

Steps to enable LLM Streaming:

Go to CONFIGURATION > Advanced Settings > LLM Streaming.
Turn on the toggle to enable LLM streaming.

Note: LLM Streaming is disabled by default.

Benefits of LLM Streaming

Reduced Latency: Real-time streaming of rephrased responses significantly decreases the time taken for the user to receive the modified content.
Improved User Experience: Faster delivery of rephrased content creates a more natural and interactive conversation flow.
Simplified Process: Streamlining the rephrasing process reduces complexity and potential points of failure.

LLM Streaming for Text-to-Speech (TTS) Providers

LLM streaming is enabled for the following TTS providers:

PlayHT
ElevenLabs
Deepgram

Note: If a GenAI node’s response contains special characters, such as punctuation marks, symbols, or brackets, the system might fail to play the prompt during a call, resulting in silence for the caller.