> ## Documentation Index > Fetch the complete documentation index at: https://koreai.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Quality insights This document covers agent evaluation in Analytics and Insights. * **Quality Monitor** reports quality by evaluation dimension across the project. * **Agent Performance** reports the same measures agent by agent. The two pages answer different questions. Quality Monitor tells you which aspect of quality slipped. Agent Performance tells you which agent caused it. Both pages populate only after evaluation pipelines process conversations. A measure with no evaluations shows a dash rather than a zero. ## Quality monitor **Quality Monitor** aggregates the output of the evaluation pipelines into one view, scored by dimension. Use it to decide where quality work is worth doing, rather than to judge how a single conversation went. **Navigation**: **Project** -> **Insights** -> **Quality Monitor** | Item | Detail | | - | - | | **Answers** | Where is quality breaking down, and in which conversations? | | **Counts** | All evaluated sessions, including eval and synthetic sessions. | | **Scale** | Every score shows from 0 to 1, as a percentage. Above 0.7 is healthy, above 0.5 is warning, and anything lower is critical. | | **Tabs** | None. The page is one long view. | ### Page filters Environment, channel, and date range controls at the top of the page scope everything below them. Filter by environment before you read the scores. Evaluations of test traffic and evaluations of production traffic answer different questions, and an average of both tells you neither. ![Quality Monitor](https://mintcdn.com/koreai/lXC-Go3bScjrntQx/agent-platform/images/analytics-insights/quality-monitor.png?fit=max\&auto=format\&n=lXC-Go3bScjrntQx\&q=85\&s=8919c21e7a378a4377e63de32eabe414) ### Quality health A banner across the top gives three figures: the number of conversations the pipelines evaluated, a weighted overall score that combines the dimensions, and a count of dimensions in each status. The evaluated count is the largest count across the five pipelines. The issue flag count adds up the flagged sessions across all dimensions. The three statuses are **Critical** (red), **Warning** (amber), and **Healthy** (green). Read the status counts first. They tell you how many dimensions need attention without your having to interpret each card. The banner turns amber or red as soon as any one dimension does. A dimension with no data also counts toward the total, so a warning count can reflect missing coverage as readily as poor performance. ### Dimension cards Each card reports one evaluation dimension for the period. Higher values are better on every dimension, so a low score always signals a problem rather than a threshold you need to interpret. | Dimension | Description | | - | - | | **Overall Quality** | Combines the scores of the dimensions that hold data, as a weighted average: quality 30%, hallucination 25%, guardrails 20%, knowledge gaps 15%, and context 10%. When a dimension has no data, the weights rebalance over the rest. Track it for direction over time. It runs too coarse to act on directly, which is the job of the remaining cards. | | **Faithfulness Score** | Measures how well responses stay grounded in the sources the agent has available. The card averages the hallucination pipeline's score. A session with a score below 0.5 is flagged. A low score means the agent produces claims it can't support, the failure mode that costs the most trust when customers notice it. | | **Knowledge Coverage** | Measures how well the knowledge the agent retrieves supports its answers. Each session scores half retrieval precision plus half citation rate. The gaps figure counts sessions where the judge detected missing knowledge. This dimension scores your content rather than your agent, so a low score tells you to write documentation instead of changing prompts. | | **Safety Score** | Measures how accurately guardrails let good content through and stop bad content. Each session scores 1 minus the larger of its false-positive and false-negative scores. A violation is either score above 0.5, or a detected bypass. It moves only when guardrails exist, so a persistent dash usually means you have no guardrails in place rather than that the platform checked nothing. | | **Context Preservation** | Measures how well the agent carries context across turns and handoffs. The card averages the context score. A session is flagged when its score is below 0.6 or when the agent repeated a response. It degrades first as conversations grow longer, and it usually explains complaints about customers repeating information. | > [!NOTE] > > The **Safety Score** on this page differs from the **Safety Score** on [Agent performance](#agent-performance). This page averages the guardrail score. Agent Performance reports the guardrail pass rate. The two values won't match. A dash means the pipeline behind that dimension produced no value for the period. It doesn't mean the dimension scored zero. ### Quality trend The trend chart plots each of the five dimensions as its own daily series, on a 0 to 1 axis. The quality series divides the 1 to 5 judge score by 5 to fit the axis. Use it for correlation. Dimensions rarely move alone, and a fall in one alongside a rise in another usually points to a single underlying change, such as new traffic the knowledge base doesn't yet cover. ### Dimension details Below the chart, each dimension appears with its score and a **Critical**, **Warning**, or **Healthy** badge. Expand a dimension to see the component measures that produced its score. The expansion separates a dimension that one weak measure pulls down from a dimension that runs weak throughout, and the distinction matters because the first needs a targeted fix and the second usually doesn't. Dimensions with no evaluations in the period carry a no-data badge in place of a status, so you can tell an unmeasured dimension from a failing one. Each card shows the dimension score from 0 to 1, the number of flagged sessions, and a status: **Healthy** above 0.7, **Warning** above 0.5, otherwise **Critical**. | Dimension | Sub-scores | | - | - | | **Quality Evaluation** | One bar per rubric dimension the judge scored, each on a 1 to 5 scale divided by 5. The four defaults come first: helpfulness, accuracy, professionalism, and instruction following. Custom rubric dimensions follow. The lowest bar is the first thing to improve. | | **Hallucination Detection** | **faithfulness score** (0 to 1, below 0.5 flags the session), **consistency index** (how consistent the agent's statements were within a conversation, 0 to 1), and **contradiction detected** (sessions where the agent contradicted itself or its sources). Read contradictions first. | | **Knowledge Gap** | **retrieval precision** (share of retrieved knowledge relevant to the question), **citation rate** (share of answers backed by a cited source), and **gap detected** (sessions where the agent lacked the knowledge to answer). Low precision points at search. Low citation rate points at how answers use sources. | | **Guardrail Analysis** | **false positive score** (how often guardrails blocked content that was fine), **false negative score** (how often guardrails let through content they should have stopped), and **bypass detected** (sessions with a guardrail bypass, each one a violation). Lower is better for both scores. | | **Context Preservation** | **context score** (0 to 1, below 0.6 flags the session), **duplication detected** (sessions where the agent repeated a response), and **handoff count** (handoffs between agents in evaluated sessions). Many duplicates mean the agent is looping. | ![Quality Evaluation expanded](https://mintcdn.com/koreai/lXC-Go3bScjrntQx/agent-platform/images/analytics-insights/quality-monitor-dimension-quality.png?fit=max\&auto=format\&n=lXC-Go3bScjrntQx\&q=85\&s=97c68a4567819f368087e0619db5c79b) ![Hallucination, Knowledge Gap, and Guardrail sub-scores](https://mintcdn.com/koreai/lXC-Go3bScjrntQx/agent-platform/images/analytics-insights/quality-monitor-dimension-subscores.png?fit=max\&auto=format\&n=lXC-Go3bScjrntQx\&q=85\&s=9f564c1b679435dd2f2c22fcdee04297) ### Flagged conversations The **Flagged Conversations** list shows every conversation that any dimension flagged, worst first. A red stripe marks a critical session. | Column | Description | | - | - | | **Health Score** | The session's worst score among the dimensions that flagged it, from 0 to 1. | | **Flag Reasons** | The dimensions that flagged the session, plus any written reasons from the judge. Several reasons on one conversation usually point to one root cause. | | **Rubric Scores** | The judge's raw rubric scores, 1 to 5, when quality evaluation ran for the session. | Two filters narrow the list. | Filter | Options | | - | - | | **Reason** | Show only one flag reason, such as hallucination. | | **Score** | **Critical** (below 0.5), **Warning** (0.5 to 0.7), or **Healthy** (above 0.7). | ![Flagged Conversations](https://mintcdn.com/koreai/lXC-Go3bScjrntQx/agent-platform/images/analytics-insights/quality-monitor-flagged.png?fit=max\&auto=format\&n=lXC-Go3bScjrntQx\&q=85\&s=53bd3b5be3662489217daf1e8971dd99) ## Agent performance **Agent Performance** reports the evaluation measures per agent. Open it once Quality Monitor tells you a dimension slipped. It earns its place in multi-agent projects, where one weak agent can pull down a project average that otherwise looks acceptable. **Navigation**: **Project** -> **Insights** -> **Agent Performance** | Item | Detail | | - | - | | **Answers** | Which agent pulls quality down? | | **Counts** | All evaluated sessions, including eval and synthetic sessions. | | **Compare** | **Compare previous period** adds arrows and changes against the previous period of the same length. | | **Tabs** | None. | ### Page filters Environment, channel, and date range controls scope the page. Select **Compare previous period** to add a comparison with the period immediately before the one you selected. KPI cards gain trend arrows, table cells gain plus or minus change values, and the trend chart adds the previous period's line. The comparison turns the scores from absolute values into a direction of travel. Use it before you draw conclusions from a single period, because evaluation scores move considerably on low volume. ![Agent Performance](https://mintcdn.com/koreai/lXC-Go3bScjrntQx/agent-platform/images/analytics-insights/agent-performance.png?fit=max\&auto=format\&n=lXC-Go3bScjrntQx\&q=85\&s=3cc96f8722c5e9e0275f8bb36db31726) ### Agent health A banner reports how many agents the pipelines evaluated, how many conversations that covered, and how many agents fall into each status: **Critical** (red), **Warning** (amber), and **Healthy** (green). Each agent takes the worst status among its metrics. The conversation total adds up each agent's count, so a session that the pipelines evaluated under two agents counts twice. That is why this total can differ from other pages. It tells you whether you face a broad problem or a single agent, before you read any individual score. ### Metric cards Five cards aggregate the measures across all agents in scope. | Metric | Description | | - | - | | **Quality** | Averages the quality score across evaluated conversations on a 1 to 5 scale. Critical at or below the quality pipeline's flag threshold (2.5 by default), and warning up to 20% above it. The table below reports the same measure per agent, so compare the two to see whether the average describes your agents evenly or conceals a spread. | | **Hallucination Rate** | Divides the conversations flagged for hallucination by evaluated conversations, × 100. Warning from 5%, critical from 10%. Lower is better here, unlike the other cards, so read this card separately rather than scanning the row for low numbers. | | **Knowledge Gaps** | Counts the conversations where the agent lacked the knowledge to answer, with the rate below. Warning above 3 conversations, critical above 7. It reports a count rather than a rate, so weigh it alongside conversation volume before you treat a rise as a regression. Fix gaps with knowledge content, not prompts. | | **Safety Score** | Reports the guardrail pass rate: 100% minus the share of conversations that guardrails flagged. Warning below 90%, critical below 75%. This calculation differs from the Safety Score on [Quality monitor](#quality-monitor). | | **Context Score** | Multiplies the 0 to 1 context score by 5, so it shows on a 0 to 5 scale. Quality Monitor shows the same score on a 0 to 1 scale. | ### Agent table The table repeats these measures for each agent, alongside the agent's conversation count and an overall status. The status is the worst of the agent's individual metric statuses, and appears as a **Critical**, **Warning**, or **Healthy** badge. The conversation count covers sessions that any of the five quality pipelines evaluated for the agent, by session start date. It shows evaluated volume, not all traffic. With **Compare previous period** on, each cell shows its change against the previous period. Read the table with the conversation count in view. An agent that the pipeline evaluated on a handful of conversations can show the worst scores on the page without being the problem worth fixing. To work through the table: * Search by name to find a specific agent. * Sort on status to bring the agents that need attention to the top. * Switch between **Critical**, **Warning**, and **All** using the status chips to handle the failing agents first. ![Agent table](https://mintcdn.com/koreai/lXC-Go3bScjrntQx/agent-platform/images/analytics-insights/agent-performance-table.png?fit=max\&auto=format\&n=lXC-Go3bScjrntQx\&q=85\&s=8bdfdcc6ee9f7dd2cd817e6c9f5c5bd9) ### Quality trend The trend chart plots daily average quality on the left axis and flagged conversations per day on the right axis. With **Compare previous period** on, a third line shows the previous period's quality, matched day by day. Read the two series together. Quality that falls while flags stay flat indicates a gradual drift in scoring. Quality that falls as flags rise points to specific conversations going wrong, which you can open and read. ![Agent Performance Quality Trend](https://mintcdn.com/koreai/lXC-Go3bScjrntQx/agent-platform/images/analytics-insights/agent-performance-trend.png?fit=max\&auto=format\&n=lXC-Go3bScjrntQx\&q=85\&s=6625f7047d3e2b3b3f8bf8844d74ab28) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.