Evaluation Forms provide a flexible framework for evaluating Human Agent and AI Agent interactions. They support configurable assignments, evaluation metrics, scoring models, duration thresholds, and version-controlled updates. QA Managers can create evaluation forms for contact center queues or Automation AI Experience Flows based on the selected assignment type.
Evaluation Forms support two assignment types:
- Human Agent: Evaluate human agent interactions routed through a contact center queue. The Assignments step presents Queues for selection.
- AI Agent: Evaluate bot interactions handled by an Automation AI flow. The Assignments step presents Experience Flows for selection.
Set the assignment type during form creation and don’t change it later. Assign each evaluation form to either Queues (Human Agent) or Experience Flows (AI Agent), not both. AI Agent forms don’t support dispute workflows or speech metrics.
Key Features
| Capability | Description |
|---|
| AI Agent Evaluation Forms | Create evaluation forms for AI Agent conversations handled through Automation AI Experience Flows. |
| Experience Flow Assignment | Assign evaluation forms to one or more Experience Flows instead of contact center queues. |
| AI Agent Metric Support | Configure AI-compatible metrics, including By Question (GenAI-Based), By Question (Deterministic), and By AI Agent metrics. |
| Flow-Based Assignment Rules | Support one active evaluation form per Experience Flow and channel combination. Queue and Experience Flow assignments are mutually exclusive. |
| Channel & Language Support | Configure evaluation forms independently for supported channels and languages. |
| Flexible Scoring | Configure percentage-based or points-based scoring, pass scores, metric weights, and response logic. |
| Fatal & Negative Scoring | Define fatal error metrics and negative scoring based on evaluation requirements. |
| Duration Thresholds | Specify the minimum interaction duration required for evaluation eligibility. |
| Cross-Segment Transcript Context | Allow By Question metrics to evaluate transcript context across AI Agent and Human Agent segments in handoff conversations. |
| Versioned Forms | Maintain version history and an audit trail of evaluation form configurations and assignments. |
| Default AI Agent Evaluation Form | Automatically creates and assigns a default AI Agent evaluation form with predefined quality metrics to all Experience Flows. |
Defines the overall configuration of an evaluation form.
| Section | Human Agent | AI Agent |
|---|
| General Settings | Configure form details, language, channel, scoring, thresholds, and assignment type. | Configure the same settings for AI Agent evaluation forms. |
| Assignments | Assign evaluation forms to queues with Contact Direction and Conversation Source. | Assign evaluation forms to Experience Flows. |
| Evaluation Metrics | Configure all supported evaluation metrics. | Configure AI Agent-supported metrics only. |
| Dispute Configuration | Configure dispute and acknowledgement workflows. | Not supported. |
Navigate to Quality AI > Configure > Evaluation Forms.
.
The Evaluation Forms page displays the following list:
| Field | Description |
|---|
| Name | Displays the evaluation form name. |
| Description | Displays the evaluation form description. |
| Assigned In | Displays the assigned Queue (Human Agent) or Experience Flow (AI Agent). |
| Channel | Displays the assigned channel (Voice or Chat). |
| Pass Score | Displays the minimum passing score. |
| Status | Indicates whether the evaluation form is active or inactive. |
| Created By | Displays the user who created the evaluation form. |
| Search | Searches evaluation forms by name. |
To create AI Agent evaluation forms, also enable the Automation AI Conversation Source.
Evaluation Processing
For conversations that transfer between an AI Agent and a Human Agent, By Question metrics can evaluate transcript context across both conversation segments. This provides additional context for AI-based evaluation while each conversation segment continues to use its own assigned Evaluation Form.
This behavior applies only to supported By Question metrics.
Evaluation Form creation consists of three or four steps depending on the selected assignment type.
-
Select the Evaluation Forms tab.
-
Select + New Evaluation Forms.
-
Enter a Name for the new form.
-
Enter a Description (Optional).
-
Select more than one Language if needed.
-
Select a Handled By agent to assign a queue name: Human Agent (Human Agent forms) or AI Agent (Experience Flow name).
-
Select a Channel type:
- Chat: Display only chat metrics (excluding speech and voice-specific Playbook metrics).
- Voice: Display all applicable voice metrics, including speech and Playbook metrics.
-
Select a Scoring Type: Percentage or Points.
-
(Optional) Set the minimum interaction duration required for evaluation by entering values in MIN and SEC.
-
Set a minimum Pass Score required for agents.
-
Select Next.
Assign at least one queue before proceeding to the next step, and make sure the selected direction determines which interaction types the system evaluates for the queue.
Human Agent Assignments
-
Search for and select one or more Queues.
-
For each selected queue, configure the Contact Direction (Inbound, Outbound, or Both) and Conversation Source (such as Contact Center AI (CCAI), AgentAI, or Quality AI Express).
-
Review the selected queues.
-
Select Next.
If an evaluation form includes both CCAI or AgentAI queues and Quality AI Express queues, By Playbook and By Dialog evaluation metrics are unavailable.
AI Agent Assignments
-
Search for and select one or more Experience Flows.
-
Review the selected Experience Flows.
-
Select Next.
Experience Flows are retrieved from the configured Automation AI environment.
Experience Flow Assignment Rules
| Rule | Description |
|---|
| One Form Per Flow Per Channel | Each Experience Flow supports only one active evaluation form per channel. If a form exists for the same flow and channel, the system displays a validation error. |
| Mutually Exclusive Assignment | You can assign an evaluation form either to queues or to Experience Flows, but not both. |
| Flow Source | Experience Flows available for selection come from the Automation AI configuration in the same workspace. |
Queue Assignment Rules
| Assignment Type | Rule |
|---|
| Human Agent | Assign each evaluation form to a unique Queue + Contact Direction combination. You can’t assign the same combination more than once within a form. |
| AI Agent | Assign each Experience Flow to only one active evaluation form per channel. You can’t combine Queue and Experience Flow assignments in the same evaluation form. |
Experience Flows available for selection come from the Automation AI configuration in the same workspace.
| Human Agent Metrics | AI Agent Metrics |
|---|
| 1. Search for and select one or more Evaluation Metrics. | 1. Search for and select one or more Evaluation Metrics. |
| 2. Review the Total Positive Weightage and Total Negative Weightage. | 2. Select the Edit icon to configure the metric Weightage or Points, Response Outcomes, Correct Response, and Fatal Error, if applicable for compliance-critical metrics. |
| 3. Select the Edit icon to configure the metric. | 3. Review the Total Positive Weightage and Total Negative Weightage. |
| 4. Configure the Weightage or Points, Response Outcomes, and Correct Response. | 4. Reorder or delete metrics, if required. |
| 5. Reorder or delete metrics, if required. | 5. Select Create to save the evaluation form. |
| 6. Select Next. | — |
Supported Metric Types
| Human Agent | AI Agent |
|---|
| Supports all available evaluation metric types, including By Question, By Playbook, By Speech, By Hold, By Transfer, By Value, Manual, and other Quality AI metrics. | Supports all available evaluation metric types, including By Question (GenAI-Based), By Question (Deterministic), and By AI Agent metrics. |
| Manual metrics are available only with Points scoring. | AI Agent evaluation forms don’t support By Playbook Adherence, By Hold, By Value, By Transfer, or By Speech (Dead Air, Cross-talk, Speaking Rate). These metric types don’t display in the Evaluation Metrics list. |
Metric Card Configuration
Trigger Scoring Disabled
When Trigger Scoring is off, the metric card shows a Weightage field for score contribution and a Fatal Error toggle that fails the evaluation if the metric isn’t met.
Trigger Scoring Enabled
When Trigger Scoring is on, the metric card expands to outcome-level scoring with Yes or No rows, each having a Weightage field, plus a Correct Response option and a Fatal Error toggle for non-adherent outcomes, with negative scoring set at the outcome level.
Outcome Configuration
Defines Yes or No outcomes with positive, zero, or negative weights, where matching responses get positive weight and non-matching responses get zero or negative weight if configured.
Configure negative scoring only at the outcome level.
- Turn on Dispute Resolution Assignment.
- Select one routing option:
- Same Auditor: Routes disputes assigned back to the original evaluator.
- Different Auditor: Routes disputes assigned to another QA from the selected auditor list.
- If you select Different Auditor, search and select one or more auditors.
- Turn on Allow Multi-Round Re-Disputes, if required.
- Select the Maximum Number of Re-Disputes rounds for agents after re-evaluation.
- Select Create.
If Dispute Resolution Assignment is turned off, agents can’t acknowledge or dispute completed evaluations.
Metric Rules
- The combined positive weightage must equal 100% for percentage-based scoring.
- Configure the expected response and score for every metric.
- Human Agent forms support all available metric types, and Fatal Error settings apply only to supported metrics.
- AI Agent forms support only AI Agent metrics used during Experience Flow evaluations.
Evaluation Processing
Evaluation Behavior
During evaluation, the system selects the applicable form based on queue, channel, and contact direction. Conversations without matching assignments are skipped. Manual metrics are supported only in points-based scoring and excluded from automated scorecards.
During evaluation, the system selects the applicable Evaluation Form using a hierarchy of Queue > Channel > Contact Direction, ensuring the selected form matches the interaction’s configured queue, channel, and contact direction.
An interaction is eligible for evaluation when it has a valid queue association and an active evaluation form mapped to that queue.
Conversations linked to evaluation-enabled queues remain eligible for evaluation even if the handling agent is not directly assigned to the queue, provided they match the configured channel and contact direction.
Evaluation eligibility is based on the conversation’s mapped queue and assigned Evaluation Form, not on agent-to-queue membership in agent or workforce management systems.
The system evaluates contact duration before scoring.
| Contact Duration | Status Assigned | Result |
|---|
| Meets or exceeds threshold | — | Evaluated normally |
| Falls below threshold | Below Threshold | Excluded from scoring and quality metrics |
| Duration unresolved | Duration unavailable | Excluded from evaluation |
Threshold updates apply only to future interactions. A contact may qualify for one evaluation form but not another.
Scoring
Scoring Logic
- Pass: Final score ≥ Pass Score threshold
- Fail: Final score < Pass Score threshold
The system calculates the final score using positive and negative metric weights.
Kore Evaluation Score calculates the weighted impact of met and not-met metrics, subtracts penalties, divides by total positive weights, and multiplies the result by 100.
(∑(Wyi)∑(Myi×Wyi)−∑(Mni×Wni))×100
where,
- Myi, Wyi = Adhered metrics and positive points.
- Mni, Wni = Non-adhered metrics and negative points.
Percentage vs. Points Comparison
Quality AI supports two scoring methods.
| Feature | Percentage-Based | Points-Based |
|---|
| Best For | Fewer than 20 metrics | 20 or more metrics |
| Total Weight | Must equal 100% | No fixed maximum |
| Scalability | Limited | High |
| Weight Per Metric | Decreases as metrics increase (for example, 40 metrics ≈ 2.5% each) | Assign any point value based on importance |
| Weight Precision | May require fractional values (for example, 2.5%) | Uses whole-number allocation (for example, 50 points for critical metrics) |
| Negative Scoring | Managed within 100% structure | Negative points allowed; cannot exceed total positive points |
| Final Evaluation Score | Direct percentage (0–100) | Normalized percentage (0–100) |
Weight Assignment Rules
| Configuration | Percentage-Based | Points-Based |
|---|
| Correct Response = Yes | Positive % for Yes; zero or negative % for No | Positive points for Yes; zero or negative points for No |
| Correct Response = No | Positive % for No; zero or negative % for Yes | Positive points for No; zero or negative points for Yes |
| Validation | Total positive weight must equal 100%; negative weight allowed within the 100% structure | No upper limit on total positive points; total negative points ≤ total positive points |
| Manual Evaluation Metrics | Not supported | Supported |
Fatal Error Behavior
Fatal error configuration remains the same for both scoring types. When a fatal metric fails, the system sets the final score to 0 and marks the interaction as failed, regardless of other metric scores.
Select Edit from the Evaluation Forms list.
You can modify: General Settings, Queue or Experience Flow assignments, Evaluation Metrics, and Dispute Allocation (Human Agent only).
Select Update to save changes.
Select Delete from the Evaluation Forms list.
Before deleting:
- Remove all Queue or Experience Flow assignments.
- Resolve dependent metric or attribute references.
If the form is in use, the system displays a warning before deletion.
Select Update to save changes.
Versioning
Every update creates a new version of the Evaluation Form. Existing evaluations continue to use the version that was active when the evaluation started, while future evaluations use the latest published version.
The system also maintains a history of assignment and configuration changes.
Troubleshooting
Switching Scoring Type
Changing the scoring type clears all existing metric weights and requires reconfiguration; percentage totals must equal 100% and points-based values must meet validation rules.
This error occurs when a new language is added to a form but one or more associated metrics don’t support it. The system blocks the update until all metrics support the selected language. For example, adding Hindi to a form with metrics that support only English and Dutch triggers this error.
To resolve it, review each metric’s language configuration, update metrics to support the new language, and then add the language to the form once all metrics are compatible.
Metric-Level Language Limitation
This warning occurs when a metric is added or updated that doesn’t support a language configured on the form.
Resolve it by configuring the required language on the metric or selecting a metric that supports all form languages.
Language Selection Behavior
Multi-language selection applies an AND condition, so only By-Question metrics available in all selected languages are displayed. For example, if you select English and Dutch, the system shows only metrics available in both languages.
Channel Mode Change
Switching between Voice and Chat triggers a warning and removes speech-based metrics. Update remaining metrics and adjust weights before saving changes.
No Dispute Workflow
AI Agent evaluation forms don’t include Dispute Allocation. Agents can’t acknowledge or dispute AI Agent evaluations.
Speech Metric Limitations
Speech-based metrics aren’t available for AI Agent evaluation forms because Experience Flow evaluations analyze bot interactions rather than voice quality.
Human Agent Considerations
Human Agent evaluation forms:
- Support all evaluation metric types.
- Support Dispute Allocation.
- Support Queue assignments with Contact Direction and Conversation Source.
- Support Manual Evaluation metrics only with Points-based scoring.
- Support speech-specific metrics for Voice channel evaluations.
