Set up Realtime LLM Voice channel
Realtime LLM Voice is a voice channel that handles live voice conversations through a single provider-native speech session. It supports Realtime LLM models (such as OpenAI Realtime and Gemini Live) through the Kore.ai Voice Gateway (KoreVG) or a Bring Your Own SIP connection. Unlike the STT → LLM → TTS pipeline that Pipeline Voice uses, a provider-native realtime model owns the live audio conversation. KoreVG handles telephony, connection routing, bounded session state, traces, and the call lifecycle.Create a Realtime LLM Voice channel
To create a Realtime LLM Voice channel, go to Deployments > Channels > Voice > Realtime LLM Voice > New connection and complete the following steps in order.Set up the connection
Create the connection
Configure the Realtime LLM Voice channel
Once the channel is created, open it from the Realtime LLM Voice connections list. The connection has three tabs: Settings, Speech, and Testing. The Settings tab holds the voice provider and call routing. The Speech tab holds realtime orchestration, the S2S provider, and advanced voice behavior. The Testing tab holds the shareable softphone link.Choose a call routing option (Kore.ai Voice Gateway)
Configure the phone number (DID only)
- Set Provider to Twilio.
- Toggle Inbound calls (routes calls from this number into the voice agent).
- Toggle Outbound calls (allows this number for campaign dial‑out).
- Select a Country, Number Type, and optional Area Code.
- Choose Search Available Numbers to list matching numbers.
- Confirm the number is bound to this connection before activation.
- After assignment, the number appears in the Phone Number field.
- Select Remove to unbind the number from the connection.
Configure SIP settings (Bring Your Own SIP only)
Set SIP header trust and save the connection settings
Configure Realtime LLM Voice settings
-
Realtime orchestration: Select how runtime coordinates the conversation flow and business tool execution for realtime voice interactions. Realtime orchestration determines whether the realtime model or runtime controls conversation progression and business actions, such as bookings, orders, and payments.
- Changes to the orchestration mode apply only to new realtime sessions. Calls already in progress continue using the mode that was active when the session started.
- Existing channel configurations without a configured orchestration mode continue to use Legacy behavior.
-
Response wording owner: Select who generates the wording of responses for realtime voice conversations. This setting is available only when the Realtime orchestration mode is set to Runtime Owned - canonical turn.
- Runtime text (recommended): Runtime generates the response text. This option provides consistent response generation.
- Provider-grounded voice: Uses the realtime voice provider to generate spoken responses while runtime continues to control conversation flow and business actions. This option is available only for Gemini Live. For unsupported or unsafe conversation turns, it automatically falls back to runtime-generated text responses.
-
Semantic convergence: Semantic Convergence determines how the platform coordinates with the realtime voice provider during a voice interaction. Controls whether realtime voice turns use the coordinated utterance runtime. This setting is available only when the realtime orchestration config is set to Legacy mode. The Effective at runtime line below the setting shows the mode currently in effect.
-
S2S Provider: The realtime speech-to-speech model the channel uses. Select an approved realtime provider and model, and match its audio format, locale, voice, and tool-calling capabilities to the agent. Select one of the following: Amazon Nova Sonic (S2S), Azure OpenAI Realtime, Grok Realtime (S2S), Google Gemini Live, or OpenAI Realtime. Each option shows which credentials it uses:
-
Realtime model connection: Shown for every S2S provider except Amazon Nova Sonic, which authenticates via Voice Service credentials instead. Selects the project-pinned realtime model this provider uses. It shows
Use project realtime modeluntil a workspace admin adds one. The Source line below the dropdown shows where the model comes from, and the deployment uses this exact model connection. To add one:- Configure the model under Voice Services, then add it to the model catalog as an LLM provider.
- In the project, go to Settings > Models and add the model you configured at the workspace level. See Models.
-
Provider-specific settings: After you select an S2S provider, a configuration section for that provider appears. Expand the provider below to see its settings.
Amazon Nova Sonic
Under Amazon Nova Sonic Configuration, set the following fields.Pending tool fillersTurn on Pending tool fillers to let Nova speak a brief hold phrase when a project tool is still running. Fast tools finish without a filler.Coordinator turn fillerTurn on Coordinator turn filler to let Nova speak a short hold phrase during convergence-enabled coordinator turns while Runtime decides and executes the turn.Advanced voice behaviourThe Guarded flow delivery settings are channel defaults for guarded realtime flows only. They apply when Realtime orchestration is set to Runtime guarded, and your values are kept when it’s set to another mode. An agent or step can override each value in its ABL. Changes apply from the next call.Also configure the Advanced voice behaviour settings that apply to all providers, described after the provider list.Azure OpenAI Realtime
Under Azure OpenAI Realtime Configuration, set the following fields.Turn detectionAlso configure the Advanced voice behaviour settings that apply to all providers, described after the provider list.Grok Realtime
Under Grok Realtime Configuration, set the following fields.Also configure the Advanced voice behaviour settings that apply to all providers, described after the provider list.Google Gemini Live
Under Gemini Live Configuration, set the following fields.Tool response schedulingRealtime turn fillerSeparate from project Voice Fillers. Applies only to Google or Vertex realtime S2S calls while semantic convergence is set to Enforce.Filler delivery safeguardsShared by pending-tool and realtime turn fillers.Voice activity detectionControls when Gemini starts listening to a caller turn and how long it waits before closing that turn.Caller transcriptionSilent-turn recoveryAdvanced voice behaviourAlso configure the Advanced voice behaviour settings that apply to all providers, described after the provider list.OpenAI Realtime
Under OpenAI Realtime Configuration, set the following fields.Turn detectionTune how OpenAI detects when speech starts and when the caller has finished speaking.Noise reductionThe provider filters the caller’s audio before it detects speech and before the model hears it, so background sound starts fewer false turns.Caller transcriptionHow the caller’s speech is written down. This text is stored as the caller’s turn and is what the agent checks answers against.Noise reduction removes noise, not other voices. Background talkers and echo of the agent still reach the model. This setting is separate from the gateway’s Noise isolation setting under Advanced voice behaviour.Realtime turn fillerApplies only to OpenAI Realtime calls while semantic convergence is set to Enforce. Off by default.Never put a caller’s own details in Words to expect. The transcriber would then hear them from anyone.Advanced voice behaviourAlso configure the Advanced voice behaviour settings that apply to all providers, described after the provider list. - Parallel caller transcription: For supported Kore.ai Voice Gateway (KoreVG) realtime channels, you can use a separate speech-to-text (STT) provider to transcribe caller audio while the realtime speech-to-speech (S2S) provider handles the conversation. The S2S provider continues to supply the agent’s response transcript.
- Availability: Off by default. A platform administrator must enable the Realtime LLM parallel STT feature for your tenant. When enabled, use the Caller transcription card on the Speech tab to select a configured STT provider instance and the caller’s language.
- Supported S2S providers: OpenAI Realtime, Azure OpenAI Realtime, Grok Realtime (S2S), and Google Gemini Live. Amazon Nova Sonic (S2S) isn’t supported.
- Transcripts: Final transcripts from the STT provider are used for caller messages in the session transcript and conversation history, with one message per caller turn. Interim caller transcripts aren’t sent. Agent transcripts still come from the S2S provider. If the STT provider doesn’t return a final transcript within 3 seconds, the system falls back to the S2S caller transcript.
-
Troubleshooting: The
voice_stttrace can show both the STT transcript (transcript) and the S2S transcript (s2sTranscript) for the same caller turn. They’re two values for one turn, not separate messages. - Billing: Caller audio goes to both the STT provider and the S2S provider, so review usage and billing for both.
-
Advanced voice behaviour (all providers): Optional behaviours for this channel. Each one is off, empty, or set to the earlier platform behaviour until you change it.
-
Voice transfer and survey capabilities: A read-only panel that shows how many of the transfer and survey capabilities Runtime evaluates as available for the concrete voice execution boundary (the execution surface is KoreVG realtime S2S), for example 11 of 11 capabilities available. It lists the following capabilities, each marked Available or not.
Save and confirm provider readiness
Test and verify the channel
Open the Testing tab to verify the channel before going live.-
Shareable softphone link: Generate a time‑limited link that lets anyone call the channel from a phone browser through the real gateway and view a read‑only transcript.
- Links expire after 7 days and can be revoked at any time.
- Active links shows links currently in use, or No active links for this channel.
- Viewing active links requires read access; generating or revoking links requires edit permission.
-
Soft Phone dialer: From the Settings tab, select Soft Phone > Launch dialer to call the agent directly from your browser.
- No DID purchase or telephony charges are required.
Preview with HTTPS voice
Run an isolated PSTN test call
Measure latency budget
Test barge‑in and rapid turns
Exercise failure scenarios
- Protect credentials: Treat realtime provider and carrier credentials as administrator‑managed secrets. Don’t copy API keys into connection identifiers, browsers, screenshots, or agent prompts.
- Minimize overhead: Avoid unnecessary transcoding, buffering, proxy hops, and oversized tool responses. Set explicit connection and turn timeouts, and verify heartbeat behavior through every proxy.
- Validate capacity: Check provider concurrency, KoreVG session limits, phone capacity, and backpressure before load tests. Capacity rejection must fail predictably, not degrade active calls.
- Define fallback behavior: Specify caller‑visible handling for provider unavailability, timeout, safety rejection, tool failure, and transfer failure. Include rules for retry, reprompt, mode switch, or call termination.
- Trace sessions: Capture call/session IDs, provider events, tool calls, turn latency, interruptions, errors, and termination reasons. Scrub audio, transcripts, and credential data from logs.
Manage an existing Realtime LLM Voice channel
Open a channel from Deployments > Channels > Voice > Realtime LLM Voice and select a connection. The connection header shows the display name, its status (for example, Active), and when it was last updated. The connection has three tabs.Troubleshoot Realtime LLM Voice
- Calls to the number fail: an Active phone-routed connection without a provisioned number can’t receive production PSTN calls. Assign or buy a number, and confirm it’s bound to this connection.
- The channel can’t start a session: resolve missing provider credentials and model configuration, then save the connection and test again.
- Duplicate tool calls or stale audio after an interruption: repeat the barge-in test from Verify and secure the Realtime LLM Voice channel and check the traces for the affected call and session IDs. For the full validation checklist, click Setup instructions on the Realtime LLM Voice channel page.
Set up Pipeline Voice channel
Pipeline Voice is a voice channel that handles calls through separate speech-to-text (STT), LLM, and text-to-speech (TTS) stages, connected through Kore.ai Voice Gateway (KoreVG). Unlike Realtime LLM Voice, which uses a single speech-to-speech model, this pipeline lets you choose and configure the speech recognition (ASR) and speech synthesis (TTS) providers independently. KoreVG captures the caller’s speech through STT, sends the transcript to the deployed agent, and streams or plays the agent’s response through TTS. Each stage has its own provider, latency, failure, and observability boundary.Create a Pipeline Voice channel
To create a Pipeline Voice channel, go to Deployments > Channels > Voice > Pipeline Voice > New connection and complete the following steps in order.Set up the connection
Create the connection
- Settings holds the voice provider, call routing, and advanced voice behavior.
- Speech holds speech recognition, speech synthesis, and conversation controls.
- Deployment holds the environment binding.
- Testing holds the shareable softphone link.
Choose a voice provider
Configure call routing (Kore.ai Voice Gateway)
Configure the phone number (DID only)
Configure SIP settings (Bring Your Own SIP only)
+ to outbound numbers sent to the carrier. It’s on by default, which international (full-E.164) numbers require. Turn it off only for local or national-format numbers, since the leading + otherwise misroutes them.Configure advanced voice behavior
en-US), then select Add language.
Configure speech synthesis
Under Speech Synthesis, set the following fields. The available fields vary by provider.
- Use the provider’s required language format. For example, ElevenLabs expects an ISO 639-1 code such as
en, noten-US. - After you add a language, the platform creates a dedicated TTS configuration section for that language. Configure the provider, voice, model, and any provider-specific settings independently for each language.
- Available fields vary by provider. Some providers expose additional settings such as TTS model, output format, speed, temperature, and voice tuning controls.
Test and verify the channel
Open the Testing tab to test the channel before you go live. Under Shareable softphone link, select Generate link to create a link that lets anyone with the link call this channel from a phone browser through the real phone gateway and see a live, read-only transcript. Links expire after 7 days, and you can revoke them at any time. Active links lists the links currently in use, or shows No active links for this channel. You can view active links with read access. Generating or revoking links needs permission to edit the channel. You can also use Soft Phone > Launch dialer on the Settings tab to call the agent from your browser with no DID purchase or telephony charges. Verify and secure the installation Before you go live, validate every phase of the voice pipeline.Run the HTTPS preview
Place a live phone call
Measure STT, LLM, TTS, and full-turn latency
Exercise silence and interruption
Test independent phase failures
- Treat STT, TTS, carrier, and gateway credentials as administrator-managed secrets. Keep provider keys out of connection identifiers, browsers, agent prompts, and screenshots.
- Set explicit phase and turn latency budgets. Avoid unnecessary transcoding and buffering, and keep prompts, tool output, and synthesis chunks small for fast first audio.
- KoreVG processes each session sequentially through a bounded queue. Load-test rapid turns and concurrent calls to confirm that backpressure delays or rejects calls predictably.
- Define what the caller hears for low-confidence or empty STT, model or tool failure, TTS rejection, no-input exhaustion, transfer failure, and gateway disconnect.
- Capture scrubbed
voice_stt,voice_tts, andvoice_turntraces. Include call and session IDs, phase latency, interruptions, errors, provider names, cache state, and termination reasons.
Manage an existing Pipeline Voice channel
Open a channel from Deployments > Channels > Voice > Pipeline Voice. The connection has four tabs.Troubleshooting
- Needs phone provisioning: the connection has no phone number. Assign or buy a number under Call routing on the Settings tab before you route phone calls. A phone-routed connection can’t be activated until it has a provisioned number.
- Voice gateway synchronization is incomplete: the platform saved your configuration but couldn’t sync it to the voice gateway. Click Save Changes again after the gateway recovers.
- Save is rejected on a Bring Your Own SIP connection: Outbound calls is on but no SIP termination URI is set. Add a SIP termination IP address or domain name, or turn off Outbound calls.
- Speech doesn’t work or the preview fails: confirm that both the STT and TTS providers have credentials configured, then save the connection before you preview or activate it.
- No voice for a language: use the provider’s required language format. ElevenLabs expects an ISO 639-1 code such as
en, noten-US. - Caller is cut off mid-sentence: raise Patience under Voice gateway turn detection, or increase Speech Timeout (ms) and Transcript Grouping Timeout (ms). For the full validation checklist, click Setup instructions on the Pipeline Voice channel page.
Pipeline vs. realtime voice
Pipeline voice (speech-to-text, agent, text-to-speech) and Realtime LLM Voice (speech-to-speech) build the agent’s turn differently underneath the same ABL. Knowing which one a channel uses matters when authoring instructions, reasoning about history after a handoff, or sizing tool results.Authoring voice behavior in ABL
The Studio settings on this page configure a channel. The agent’s own spoken behavior is authored in ABL:- Spoken-style rules and grounding guards —
EXECUTION.voice.instructionsandEXECUTION.voice.grounding_guards, which replace the platform’s default voice formatting rules and can refuse ungrounded tool arguments or figures. See Voice instructions and grounding guards. - Realtime delivery tone —
EXECUTION.voice.coordinator_policy(tone, pace, brevity, interruption handling, accessibility read-backs). See Voice coordinator policy. - Per-response voice overrides — the
VOICE:block on aRESPOND(SSML, plain text, provider, voice, speed) plus the step- and response-scopedBARGE_IN,STREAMING, andEND_OF_SPEECH_TURN_TYPEoverrides. See Voice configuration and Scoped barge-in. - Listening and turn-taking —
CONVERSATION: listening:(barge-in, no-input retries, caller backchannel, on-hold phrases, barge-in continuation merging). See Voice turn-taking. - Low-latency defaults — new projects and voice channels get the
customer_service_voice_low_latency_v1profile automatically: tighter compaction, the fast model tier for every operation, reasoning off, and inline gather on. The Studio side of this profile — model routing, compaction budgets, and filler tuning — is covered in Voice low-latency profile.