Orchestration Overview

• Interprets user intent
• Selects the appropriate agent(s)
• Coordinates multi-agent workflows
• Resolves conflicts between outputs
• Delivers unified responses

​

Orchestration Patterns

• Single Agent: One agent handles all requests. Best for focused, well-defined domains.
• Supervisor: Central orchestrator coordinates multiple specialized agents. Best for complex, parallelizable tasks.
• Adaptive Network: Agents dynamically hand off to each other. Best for sequential, multi-domain workflows.

​

Pattern Comparison

​

How Orchestration Works

​

Choosing the Right Pattern

• Your app has one primary capability
• Tasks don’t require coordination between specialists
• You want minimal orchestration overhead
• Response latency is critical

• Tasks can be broken into independent subtasks
• You need parallel execution for speed
• Multiple specialists should contribute to responses
• You want centralized control and conflict resolution

• Tasks flow naturally between domains
• You need dynamic routing based on context
• Agents should autonomously decide when to hand off
• Sequential expertise is required

​

Orchestrator Responsibilities

​

Task Decomposition

User: "I need to cancel my order and get a refund"

Decomposition:
├── Subtask 1: Look up order details (Order Agent)
├── Subtask 2: Process cancellation (Order Agent)
└── Subtask 3: Initiate refund (Billing Agent)

​

Agent Delegation

Request: "What's my order status and can I upgrade my shipping?"

Delegation:
├── Order Agent: Retrieve order status
└── Shipping Agent: Process shipping upgrade

​

Conflict Resolution

Conflict:
├── Agent A: "Item is in stock"
└── Agent B: "Item ships in 2 weeks"

Resolution: Check inventory system → Provide accurate status

​

Context Management

Context:
├── User ID: 12345
├── Session ID: abcde
└── Previous Turns:
    ├── Turn 1: User provides order number
    ├── Turn 2: Agent A uses order number
    └── Turn 3: Agent B receives context, doesn't ask again

​

Orchestrator Configuration

​

Default AI Model

​

Voice-to-Voice Interactions

​

Voice AI Model Settings

​

Behavioral Instructions

​

Response Processor

• The artifact needs to be assembled from multiple tool outputs or session variables rather than a single tool response.
• The payload structure depends on business logic better handled centrally at the application level.
• No tool is needed — the processor can produce artifacts independently based on input context alone.

• Reorder elements to control which artifact renders first.
• Filter artifacts based on channel, user segment, or business logic.
• Transform or enrich payload data before delivery to the client.
• Merge multiple tool outputs into a single consolidated artifact.
• Add metadata, wrapper keys, or channel-specific formatting.

​

Adding a Response Processor

1. Input — The original user input that triggered this agent run.
2. Output — The agent’s response generated by the agent.
3. Artifacts — Array of tool outputs and structured data returned during the run by different tools.

• Access input, output or artifacts in the script using $ prefix.
• Access environment variables using env keyword : env.<variable-name>.
• Access content variables as: content.<variable-name>.
• Access memory stores in the script using these methods.

// Extracts account balance amount from the textual response and only sends back the value, instead of complete textual response.

console.log("[PostProcessor] Input received:", $input); 
console.log("[PostProcessor] Output received:", $output);
console.log("[PostProcessor] Artifacts:", $artifacts);
console.log(env.name) //access environment variable
console.log(content.new) //access content variable

const originalOutput = $output;
let modifiedOutput = originalOutput;

const outputData = $output;

let finalOutput;

// Extract balance amount(number) from text (e.g., "Your balance is 7236")
const match = outputData.match(/\d+/);

if (match) {
    finalOutput = {
      balance: Number(match[0])
    };
} else {
    finalOutput = {
      message: outputData
    };
}

return {
  output: finalOutput
};

​

Single Agent Configuration

1. Agent Prompt — The core instructions that define the agent’s role and behavior.
2. Behavioral Instructions — Guidelines that control tone, constraints, and response style.
3. Tools Assigned to the Agent — Tool definitions available for the agent to invoke.
4. Events Enabled in the Application — Event-related context.

​

Supervisor Configuration

• Orchestrator Prompt — A set of instructions for the supervisor of the app. This includes instructions and requirements that guide the orchestrator’s decision-making process.
• Orchestration Prompt for Voice-to-Voice Interactions — This prompt serves as instructions for the supervisor in case of voice interactions.

​

Adaptive Network Configuration

• Initial Agent — Select the agent that serves as the first point of contact for each task. This agent receives the user’s request, processes the initial requirements, and begins task execution.