AI-Assisted Manual Audit

Documentation Index

Fetch the complete documentation index at: https://koreai.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

AI-Assisted Manual Audit enables supervisors and QA teams to evaluate voice and chat interactions using a hybrid approach that combines AI-generated insights with manual scoring. The audit workspace provides a unified view of transcripts, interaction metadata, timeline navigation, and AI insights such as summaries, topics, sentiment, resolution status, and metric cards. Evaluators can review AI outputs, validate or override scores, add comments, and submit final audit results for coaching and compliance. The framework supports inbound and outbound evaluation flows, duration-aware scoring (including exclusions for short interactions), and structured QA forms for consistent assessment. Key Capabilities:

---

​

Prerequisites

Before using AI-Assisted Manual Audit, confirm the following:

• Auto QA Permission: Access to manage metric types in Quality AI General Settings.
• QA Access: Permission to perform self-assignment and auditing.
• Role-Based Access: Appropriate permissions assigned based on your organizational role.
• GenAI Settings: Enable sentiment, emotions, and topic modeling features as required.
• Metric Settings: Enable Speech, Playbook, Hold Etiquette, and related metrics when needed.

---

​

Access AI-Assisted Manual Audit

Navigate to Quality AI > Analyze > Conversation Mining > Interactions > AI-Assisted Manual Audit. You can open interactions from:

• Conversation Mining: View all conversations within your assigned queues.
• Allocations: View all interactions assigned to you for evaluation.

---

​

Audit Workflow

1. Select an interaction.
2. Review transcript, timeline, and AI insights.
3. Select Assign to Me if the interaction is unassigned.
4. Complete all required metrics.
5. Add comments if needed.
6. Select Submit to finalize the audit.

---

​

Audit Screen Overview

---

​

Audit Tab

The Audit tab is the primary workspace for manual review and AI-assisted evaluation. Comments (Bottom Panel): Enables feedback management at both message and metric levels, with direct navigation to relevant points in the transcript.

---

​

Audit Evaluation

​

AI Overview

Displays conversation insights through AI-powered widgets, helping supervisors evaluate key metrics without reading full transcripts. This displays high-level evaluation details and scoring:

​

Score Summary

Displays key behavioral and linguistic metrics: If analysis is unavailable, these values display as NA.

​

Topics and Intents

​

Resolution and Sentiment

​

Generated Topics

• Expands topic discovery beyond configured taxonomy.
• Provides topic-level sentiment insights.
• Enhances visibility into conversation patterns.

---

​

Transcript and Timeline

The Transcript provides a unified, time-synchronized view of the interaction across chat and voice.

• Speaker-separated conversation (Agent and Customer).
• Timestamped utterances.
• Keyword highlighting and navigation.
• Inline and clickable comments.
• Audio playback with synchronized transcript (voice only).

​

Sentiment Analysis

Shows the overall sentiment of the customer and agent across three phases of the call.

​

Sentiment Ratio

Displays the distribution of sentiment across the interaction as a percentage breakdown (Positive, Neutral, Negative). If no data is available, it shows No Sentiment Ratio Found.

​

Sentiment Patterns

Resolution-Aware Scoring: prioritizes final sentiment and produces a weighted interaction score. Scores use a 1-10 scale (5 = Neutral, 7 = Positive) and produce a final classification of Positive, Neutral, or Negative.

​

Emotion Analysis

Tracks emotional signals for both agent and customer across the timeline.

• Agent Emotions: empathy, patience, happiness, frustration, confusion.
• Customer Emotions: satisfaction, anger, confusion, churn risk, escalation.

Emotions are ranked by duration percentage from highest to lowest. The top three emotions for each participant are displayed with timeline-based visualization and emoticon indicators.

​

Emotions

Displays No Emotion Found if unavailable.

---

​

Conversation Insights

---

​

Agent Speech Insights

---

​

Transcript and Timeline

Displays the full interaction with:

• Speaker-separated messages
• Timestamps
• Keyword highlights
• Audio playback (voice only)
• Clickable navigation markers

---

​

By Question (Audit)

Evaluates agent performance against predefined audit questions using AI detection and manual validation.

​

AI Justification

The AI Justification section uses LLM-generated explanations to clarify Auto QA decisions in By Question metrics. It explains Adhered, Not Adhered, and Not Applicable outcomes and improves transparency by showing why a metric passes, fails, or isn’t evaluated. What AI Justification displays:

​

Dynamic By Question – Not Applicable Justification

For Dynamic By Question metrics, when the system doesn’t detect the trigger intent, it generates an LLM-based justification explaining the absence of relevant conversational evidence and why the intent doesn’t qualify for evaluation. This helps users refine and optimize trigger prompts during design time.

​

Timestamp Generation Rule

• Timestamps are included: When a specific agent message or event causes a violation (for example, rude or non-compliant behavior).
• Timestamps are not included: For omission scenarios where expected behavior is missing and no specific message or event caused the outcome.

Timestamp Examples

• Valid (Violation-Based): Professionalism metric → Agent used rude language → timestamps displayed for relevant messages.
• Invalid (Omission-Based): Greeting metric → Agent doesn’t greet → no timestamps displayed.

​

Adherence Filter Status

Filter and sort compliance metrics by adherence status:

​

Keyword-Based Conversation Analysis

Keyword filters applied on the Conversation Mining page carry over to the Audit screen. The transcript view shows the full conversation with keyword highlighting.

​

Omissions

Highlights cases where the agent doesn’t follow configured compliance requirements, such as playbook steps or dialog tasks. This includes omitted playbook steps for playbook metrics and omitted dialog tasks for dialog metrics. The system displays only when relevant metrics are configured for the interaction.

​

Violations

Highlights speech metric violations that occurred during the call (for example, Cross Talk, Dead Air, or Speaking Rate Violation). Each violation includes a timestamp so you can navigate directly to that point in the recording.

​

By Playbook

Enables evaluators to assess adherence to configured playbook metrics. Displays for each playbook metric:

• Configured minimum adherence.
• Observed adherence within the interaction.
• Missing steps not completed during the interaction.
• Expected vs. observed steps in a dropdown format.
  To audit Speech and Playbook metrics, enable Audit Speech Metrics and Audit Playbook Metrics under Settings. If not enabled, these metrics show in view-only mode.

Adherence Scoring Logic

​

By Value

Tracks value-related metrics during interaction evaluations. Leverages GenAI to analyze agent behavior beyond predefined scripts. Agent Adherence fields:

• Source System Value - Value obtained from the source system.
• Agent Mentioned Value - Value mentioned by the agent during the conversation.
• AI Justification - Explanation of the AI’s adherence decision.
• GenAI-based adherence - Combines business rule validation with tolerance range analysis.
• Custom script adherence - Includes the agent-mentioned value and business rule justification.

​

By AI Agent

Delivers advanced sentiment analysis through GenAI, enabling Post-Interaction Sentiment Analytics and Key Emotion Moments. Integrates with GenAI Copilot to leverage LLMs for detailed post-interaction insights. Key capabilities:

• Real-time AI-driven analysis.
• Sentiment and emotion detection.
• Topic modeling and intent recognition.
• Predictive analytics.

AI Justification fields (for GenAI-based evaluations):

• Clear reasoning for the AI’s Yes/No outcome (adhered/not adhered/not applicable) with observation time.
• Evidence of trigger presence or absence for dynamic adherence types.
• Specific agent behaviors that influenced the metric outcome.
• Timestamps for all relevant conversation segments.

Adherence Filter Status Filter and sort compliance questions by adherence status:

---

​

Conversation Insights

Provides AI-generated overviews of customer interactions without requiring a full transcript review.

​

Agent Speech Insights

Displays agent-specific performance metrics.

---

​

Comments

Displays all feedback submitted by auditors during the evaluation process. Comments show both inline in the Transcript and in the Comments tab. Commenter details are displayed based on privacy settings (for example, Hide Auditor Details).

​

Adding a Message-Level Comment

1. Select Assign to Me to begin auditing the conversation.
2. Hover over any message in the Transcript section — a Comment icon displays.
3. Select the Comment icon and enter a comment Name and Comment text (both required).
4. Add or delete your comment before sending.
5. Select Send to publish the comment.

When submitted, the Inline comments box shows:

• Inline in the Transcript, linked to the corresponding message.
• In the Comments tab with the comment title, text, and commenter details (based on privacy settings).

​

Comment Types

​

Click-Through Navigation

All users — including agents without QA permissions — can select a comment to navigate to the related message. The system centers the commented message in the Transcript window, enabling agents to review feedback from supervisors and QA auditors.

---

​

Near-Miss Scenarios

Near-miss evaluations flag responses that closely resemble, but do not fully meet, adherence standards. Applicable only in Deterministic Adherence mode. How it works:

1. The system compares agent responses against predefined similarity thresholds.
2. Near-miss cases are flagged for auditor review.
3. When you Select the View button, the evaluation is marked Yes (highlighted in green) and the relevant customer response is highlighted.

---

​

Self-Assignment for Audit

QA users (auditors or supervisors) can self-assign unclaimed interactions for auditing. Interactions that are already audited, completed, or assigned to another user are not eligible. To self-assign an interaction:

1. Navigate to the Conversation Mining page.
2. Select an interaction that is not audited or assigned.
3. Select Assign to Me. A success message confirms the assignment.
4. The interaction is marked as Self-Assigned on the Audit Allocations page and becomes unavailable for reassignment.

---

​

Audit Submission

The Submit button is enabled only when the interaction is assigned to you through Audit Allocations. Before submitting:

1. If By Question, By Value, or By AI Agent metrics are present, select appropriate responses for all required audit questions.
2. Ensure the adherence percentage totals 100%.
3. Select Submit.

After submission:

• The interaction is marked as Self-Assigned on the Audit Allocations page.
• The audited interaction is unavailable for reassignment.
• A completed and submitted interaction cannot be re-audited.

Agent access to scored interactions is controlled by the Agent Access to Scored Interactions setting: Hide Auditor Details for Agent:

• On - Auditor details are anonymized in the audit screen.
• Off - Auditor details are visible.

---

​

Search

Provides keyword search across the entire transcript to locate specific topics, compliance issues, customer concerns, or training opportunities.

---

​

Conversation Details Tab

Provides contextual and audit-related information for the selected interaction, including direction and custom fields, helping supervisors review the interaction context before or after evaluation. Conversation Details:

• Start Time, Termination Time, End Time, Agent Name, Queue, Channel, Contact Direction, Customer Phone, CSAT, Disposition, Evaluation Form, and Language.

Audit Details:

• Auditor Name, Audited Date, Audit Score, and Kore Evaluation Score.

Identifiers (each includes a copy icon): You can also use Assign to Me on this tab to assign the interaction to yourself for audit.

​

Custom Fields

The Custom Fields section displays business-specific metadata ingested with this conversation from Express File or Agent AI integrations. Each field shows as a header-value pair. If no custom fields are available, the section displays No custom fields found.

​

Right Metadata Panel

The right panel displays the following additional interaction context:

• Start Time: Timestamp when the conversation started (for example, 10 April 2026, 7:00:00 AM).
• Termination Time: Timestamp when the system terminated the conversation.
• End Time: Timestamp when the conversation ended.
• Agent: Name of the agent who handled the interaction.
• Queue: Queue assigned to the interaction (for example, AWSPublicQueue).
• Channel & Direction: Channel type and contact direction (for example, Voice — Inbound).
• Customer Phone: Customer’s phone number, if available.
• CSAT: Customer satisfaction score, if collected.
• Disposition: Disposition applied to the conversation.
• Evaluation Form: Name of the evaluation form applied (for example, New Points Based).
• Language: Language of the conversation (for example, English).

---

​

Audit Logs Tab

Provides a complete audit trail of the interaction evaluation process by recording system and user actions, executing GenAI-based metric evaluations, tracking status changes, and displaying audit progress for transparency and compliance. Log entries capture: Each execution log entry includes:

• Date and Time of execution.
• GenAI Feature Name (for example, By Hold Adherence).
• Language.
• Model Name (for example, GPT-4o).
• Integration Type (System or Custom).
• Prompt Name and Type (Default or Custom).
• Request Token Count, Response Token Count, and Response Duration.
• Execution Status (Success or Failure).

Payload Visibility: View Request and Response payloads with options to expand/collapse, format or compact, copy to clipboard, or open in full-screen mode for debugging. Select Assign to Me to assign a log entry to yourself for audit. The system records who assigned it and when, and displays the assigned user in the header and audit history.

---