AI-Assisted Manual Audit

AI-Assisted Manual Audit is where supervisors and QA teams evaluate voice and chat interactions using a hybrid approach that combines AI-generated insights with manual scoring in a single workspace. When Agent Accept & Dispute is active, agents use the screen as a workspace to review evaluation outcomes, acknowledge metric scores, and submit disputes. Supervisors then re-evaluate each dispute metric by metric. The workspace brings together transcripts, interaction metadata, timeline navigation, and AI insights such as summaries, topics, sentiment, resolution status, and metric-level evaluations. Evaluators review AI-generated recommendations, validate or override scores, add comments, and submit audit results to support coaching and compliance processes. The framework supports inbound and outbound interactions, applies duration-based scoring, and uses structured QA forms to drive consistent evaluations. Key Capabilities:

---

​

Prerequisites

Before using AI-Assisted Manual Audit, confirm the following:

• Auto QA Permission: Access to manage metric types in Quality AI General Settings.
• QA Access: Permission to perform self-assignment and auditing.
• Role-Based Access: Appropriate permissions assigned based on your organizational role.
• GenAI Settings: Enable sentiment, emotions, and topic modeling features as required.
• Metric Settings: Enable Speech, Playbook, Hold Etiquette, and related metrics when needed.
• Agent Dispute: Enable Agent Accept & Dispute in Quality AI > Settings > Quality AI General Settings to activate the metric-level dispute and acknowledgment workflow.

---

​

Access AI-Assisted Manual Audit

Navigate to Quality AI > Analyze > Conversation Mining > Interactions > AI-Assisted Manual Audit. You can open interactions from:

• Conversation Mining: View all conversations within your assigned queues.
• Allocations: View all evaluated interactions along with their current review and dispute status.
• Allocations > Disputes: View disputed interactions routed to you for re-evaluation.
• Review > Pending Review, Disputes, or Resolved (agent view): View evaluations pending a response or active disputes.

---

​

Audit Workflow

1. Select an interaction.
2. Review transcript, timeline, and AI insights.
3. Select Assign to Me if the interaction isn’t assigned.
4. Complete all required metrics.
5. Add comments if needed.
6. Select Mark for Coaching to flag the agent for coaching review.
7. Select Submit to complete the audit. If you configure disputes in the evaluation form, the system shows the completed evaluation to the agent for acknowledgment or dispute.

---

​

Audit Screen Header

The header identifies the conversation, current audit stage, and available workflow actions.

---

​

Audit Review Tabs

The system displays Audit Review statuses based on the selected Allocations tab and audit workflow stage, helping you track review progress from Assigned through Pending Supervisor Review, Resolved, and Closed.

​

Audit Screen Tabs

The Audit Screen provides role-based access to evaluation, transcript review, audit history, and dispute management.

---

​

Audit Tab

The Audit tab combines transcript review, AI analysis, scoring, dispute handling, and coaching workflows.

---

​

Audit Progress

The audit progress bar at the top of the metrics panel tracks evaluation completion across all scoring sources. Its status varies by Allocation tab and audit workflow stage.

---

​

Mark for Coaching

The Audit screen includes a Mark for Coaching action that enables QA users to create coaching recommendations directly from an audit. Only users with the required QA permissions can access this action.

​

Individual Agent Coaching

Select Individual Agent to create a coaching recommendation for the agent associated with the audited interaction.

​

Group of Agents Coaching

Select Group of Agents to create a coaching recommendation for a predefined coaching group.

​

Add New Group

Select + Add New Group to create a coaching group.

​

Coaching Recommendation Creation

When a QA user saves a coaching recommendation:

• The system creates a coaching recommendation based on the selected coaching type.
• The system associates the recommendation with the audited interaction and selected coaching areas.
• The system assigns the recommendation to the selected coaching group.
• The coaching recommendation becomes available for subsequent coaching activities and tracking.

---

​

Supervisor View — Review Panel

The review panel on the right displays evaluation metrics, reviewer responses, comments, and dispute history. Supervisors can review disputed metrics, update responses, and resolve disputes.

​

Comments & Dispute

The Comments & Dispute section displays the conversation history associated with a metric.

​

Dispute Resolution

When a metric dispute occurs, the system helps supervisors to review the dispute history and take one of the following actions

​

Agent View — Review Panel

The system enables agents to review evaluation results and respond to metric-level evaluations when dispute functionality is active.

​

Audit Evaluation

​

AI Overview

Displays conversation insights through AI-powered widgets, helping supervisors evaluate key metrics without reading full transcripts. This displays high-level evaluation details and scoring:

​

Score Summary

Displays key behavioral and linguistic metrics: If analysis is unavailable, these values display as NA.

​

Topics and Intents

​

Resolution and Sentiment

​

Generated Topics

• Expands topic discovery beyond configured taxonomy.
• Provides topic-level sentiment insights.
• Enhances visibility into conversation patterns.

---

​

Transcript and Timeline

The Transcript provides a unified, time-synchronized view of the interaction across chat and voice.

• Speaker-separated conversation (Agent and Customer).
• Timestamped utterances.
• Keyword highlighting and navigation.
• Inline and clickable comments.
• Audio playback with synchronized transcript (voice only).

​

Sentiment Analysis

Shows the overall sentiment of the customer and agent across three phases of the call.

​

Sentiment Ratio

Displays the distribution of sentiment across the interaction as a percentage breakdown (Positive, Neutral, Negative). If no data is available, it shows No Sentiment Ratio Found.

​

Sentiment Patterns

Resolution-Aware Scoring: prioritizes final sentiment and produces a weighted interaction score. Scores use a 1-10 scale (5 = Neutral, 7 = Positive) and produce a final classification of Positive, Neutral, or Negative.

​

Emotion Analysis

Tracks emotional signals for both agent and customer across the timeline.

• Agent Emotions: empathy, patience, happiness, frustration, confusion.
• Customer Emotions: satisfaction, anger, confusion, churn risk, escalation.

Rank emotions by duration percentage from highest to lowest and display the top three emotions for each participant using timeline-based visualization and emoticon indicators.

​

Emotion Analysis

Displays No Emotion Found if unavailable.

---

​

Conversation Insights

---

​

Agent Speech Insights

---

​

Transcript and Timeline

Displays the full interaction with:

• Speaker-separated messages
• Timestamps
• Keyword highlights
• Audio playback (voice only)
• Clickable navigation markers

---

​

By Question (Audit)

The By Question section evaluates interactions using configured audit questions through AI-generated analysis and manual validation.

​

Evaluation Components

​

AI Justification

The AI Justification section uses LLM-generated explanations to clarify Auto QA decisions in By Question metrics. It explains Adhered, Not Adhered, and Not Applicable outcomes and improves transparency by showing why a metric passes, fails, or isn’t evaluated. What AI Justification displays:

​

Dynamic By Question – Not Applicable Justification

For Dynamic By Question metrics, when the system doesn’t detect the trigger intent, it generates an LLM-based justification explaining the absence of relevant conversational evidence and why the intent doesn’t qualify for evaluation. This helps users refine and optimize trigger prompts during design time.

​

Timestamp Generation Rule

• Timestamps included: When a specific agent message or event causes a violation (for example, rude or non-compliant behavior).
• Timestamps not included: For omission scenarios where expected behavior is missing and no specific message or event caused the outcome.

Timestamp Examples

• Valid (Violation-Based): Professionalism metric → Agent used rude language → timestamps displayed for relevant messages.
• Invalid (Omission-Based): Greeting metric → Agent doesn’t greet → no timestamps displayed.

​

Adherence Filter Status

Filter and sort compliance metrics by adherence status:

​

Audit Navigation and Keyword-based Conversation Analysis

Enables quick filtering and navigation by keyword or QA question, with auto-scrolling of transcripts and relevance-based matching. Keyword filters applied on the Conversation Mining page carry over to the Audit screen.

​

Omissions

Highlights cases where the agent doesn’t follow configured compliance requirements, such as playbook steps or dialog tasks. This includes omitted playbook steps for playbook metrics and omitted dialog tasks for dialog metrics. The system displays only when it has configured relevant metrics for the interaction.

​

Violations

Highlights speech metric violations that occurred during the call (for example, Cross Talk, Dead Air, or Speaking Rate Violation). Each violation includes a timestamp so you can navigate directly to that point in the recording.

​

By Playbook

Enables evaluators to assess adherence to configured playbook metrics. Displays for each playbook metric:

• Configured minimum adherence.
• Observed adherence within the interaction.
• Missing steps not completed during the interaction.
• Expected vs. observed steps in a dropdown format.
  To audit Speech and Playbook metrics, enable Audit Speech Metrics and Audit Playbook Metrics under Settings. If not enabled, these metrics show in view-only mode.

Adherence Scoring Logic

​

By Value

Tracks value-related metrics during interaction evaluations. Leverages GenAI to analyze agent behavior beyond predefined scripts. Agent Adherence fields:

• Source System Value - Value obtained from the source system.
• Agent Mentioned Value - Value mentioned by the agent during the conversation.
• AI Justification - Explanation of the AI’s adherence decision.
• GenAI-based adherence - Combines business rule validation with tolerance range analysis.
• Custom script adherence - Includes the agent-mentioned value and business rule justification.

​

By AI Agent

Delivers advanced sentiment analysis through GenAI, enabling Post-Interaction Sentiment Analytics and Key Emotion Moments. Integrates with GenAI Copilot to leverage LLMs for detailed post-interaction insights. Key capabilities:

• Real-time AI-driven analysis.
• Sentiment and emotion detection.
• Topic modeling and intent recognition.
• Predictive analytics.

AI Justification fields (for GenAI-based evaluations):

• Clear reasoning for the AI’s Yes/No outcome (adhered/not adhered/not applicable) with observation time.
• Evidence of trigger presence or absence for dynamic adherence types.
• Specific agent behaviors that influenced the metric outcome.
• Timestamps for all relevant conversation segments.

Adherence Filter Status Filter and sort compliance questions by adherence status:

---

​

Conversation Insights

Provides AI-generated overviews of customer interactions without requiring a full transcript review.

​

Agent Speech Insights

Displays agent-specific performance metrics.

---

​

Comments

Displays feedback submitted by auditors during the evaluation process. Comments appear inline within the Transcript and in the Comments tab. The system displays the Commenter details according to privacy settings (for example, Hide Auditor Details). Supports message-level and metric-level feedback with direct navigation to relevant points in the transcript.

​

Adding a Message-Level Comment

1. Select Assign to Me to begin auditing the conversation.
2. Hover over any message in the Transcript section — a Comment icon displays.
3. Select the Comment icon and enter a comment Name and Comment text (both required).
4. Add or delete your comment before sending.
5. Select Send to publish the comment.

When submitted, the Inline comments box shows:

• Inline in the Transcript, linked to the corresponding message.
• In the Comments tab with the comment title, text, and commenter details (based on privacy settings).

​

Comment Types

​

Click-Through Navigation

All users — including agents without QA permissions — can select a comment to navigate to the related message. The system centers the commented message in the Transcript window, enabling agents to review feedback from supervisors and QA auditors.

---

​

Near-Miss Scenarios

Near-miss evaluations flag responses that resemble but don’t meet adherence standards. Applicable only in Deterministic Adherence mode. How it works:

1. The system compares agent responses to predefined similarity thresholds.
2. The system flags near-miss cases for auditor review.
3. When you Select the View, the system marks the evaluation as Yes (highlighted in green) and highlights the relevant customer response.

---

​

Self-Assignment for Audit

QA users (auditors or supervisors) can self-assign unclaimed interactions for auditing. The system excludes interactions that it has audited, completed, or assigned to another user. To self-assign an interaction:

1. Navigate to the Conversation Mining page.
2. Select an interaction that isn’t audited or assigned.
3. Select Assign to Me. A success message confirms the assignment.
4. The system marks the interaction as Self-Assigned on the Audit Allocations page and becomes unavailable for reassignment.

---

​

Audit Submission

The system displays the Submit option when any interactions assigned to you through Audit Allocations. Before submitting:

1. If By Question, By Value, or By AI Agent metrics are present, select appropriate responses for all required audit questions.
2. Ensure the adherence percentage totals 100%.
3. Select Submit.

After submission:

• The system marks the interaction as Self-Assigned on the Audit Allocations page.
• The audited interaction is unavailable for reassignment.
• You can’t re-audit a completed and submitted interaction.

Agent access to scored interactions setting controlled by the Agent Access to Scored Interactions: Hide Auditor Details for Agent:

• On - The system anonymize the auditor details in the audit screen.
• Off - Auditor details are visible.

---

​

Search

Provides keyword search across the entire transcript to locate specific topics, compliance issues, customer concerns, or training opportunities.

---

​

Conversation Details Tab

Provides contextual and audit-related information for the selected interaction, including direction and custom fields, helping supervisors review the interaction context before or after evaluation. Conversation Details:

• Start Time, Termination Time, End Time, Agent Name, Queue, Channel, Contact Direction, Customer Phone, CSAT, Disposition, Evaluation Form, and Language.

Audit Details:

• Auditor Name, Audited Date, Audit Score, and Kore Evaluation Score.

Identifiers (each includes a copy icon): You can also use Assign to Me on this tab to assign the interaction to yourself for audit.

​

Custom Fields

The Custom Fields section displays business-specific metadata ingested with this conversation from Express File or Agent AI integrations. Each field shows as a header-value pair. If no custom fields are available, the section displays No custom fields found.

​

Right Metadata Panel

The right panel displays the following additional interaction context:

• Start Time: Timestamp when the conversation started (for example, 10 April 2026, 7:00:00 AM).
• Termination Time: Timestamp when the system terminated the conversation.
• End Time: Timestamp when the conversation ended.
• Agent: Name of the agent who handled the interaction.
• Queue: Queue assigned to the interaction (for example, AWSPublicQueue).
• Channel & Direction: Channel type and contact direction (for example, Voice — Inbound).
• Customer Phone: Customer’s phone number, if available.
• CSAT: Customer satisfaction score, if collected.
• Disposition: Disposition applied to the conversation.
• Evaluation Form: Name of the evaluation form applied (for example, New Points Based).
• Language: Language of the conversation (for example, English).

---

​

Audit Logs Tab

This tab provides AI processing details for an interaction. It displays feature execution results, token usage, execution time, and payload information to help users review AI activity and investigate processing issues. Information is organized into process summaries and detailed execution records.

​

Process Status

This section displays the execution status of AI features used during interaction analysis. The system groups features into categories and displays aggregated request and response token counts for each category within the processed interaction. Feature Status Fields

​

GenAI Execution Records

Displays detailed execution records for individual GenAI features. Where, you can filter records by Gen AI Feature and Status. Each record displays as an expandable card with a summary of the AI execution. Execution Details

​

View Payload

Select the View Payload eye icon to open detailed execution information for a GenAI feature. The payload window displays execution metadata along with request and response payloads. Execution Metadata This displays the summary information about the selected execution. Payload Actions The payload viewer provides tools for inspecting payload content. Payload content varies based on the selected AI feature, model response, and processing results. Execution Status This indicates whether a feature completed successfully or failed. When a failure occurs, the system records the reason when available. Select Assign to Me to assign a log entry to yourself for audit review. The system records who assigned it and when, and displays the assigned user in the header and audit history. Payload Visibility: View Request and Response payloads with options to expand/collapse, format or compact, copy to clipboard, or open in full-screen mode for debugging.

​

Audit Trail Tab

The Audit Trail tab provides a history of audit activities, including evaluations, disputes, re-evaluations, assignments, and status updates. It also displays audit information such as the audited agent, audit date and time, and review status. This can include one or more allocation records based on the audit history. Each allocation displays assignment summaries and evaluation details. Dispute and re-evaluation records display when applicable.

​

Allocation Summary

Displays an overview of an audit assignment. Evaluation Outcome Count The system groups metric results into the following categories. Evaluation Details Displays evaluation results for each metric. Evaluation Outcomes

​

Disputed By Details

This section shows disputes raised for evaluation results. It appears when an auditor disputes one or more metrics.

​

Re-evaluation

Displays when a dispute triggers a re-evaluation. The section includes the auditor who assigned the re-evaluation, the outcome summary, updated evaluation score, re-evaluated metrics, updated outcomes, and auditor comments.

​

System Assignment

Displays when workflow automation creates an audit assignment. The Allocation Assigned By field displays System for automated assignments.

---