Usage insights

This document covers the two pages that answer how the program performs and what it costs.

At a Glance reports conversation outcomes, quality, sentiment, and return on investment for a period.
Billing and Usage reports what the project consumed and which part of that consumption bills.

Both pages report on conversations that evaluation pipelines have already processed. Until a pipeline runs, the metric cards show dashes rather than zeros.

At a Glance

At a Glance aggregates outcome, quality, sentiment, and cost metrics into a single view that needs no configuration. Use it for a daily check or a leadership review, and to decide which of the deeper pages to open next. Navigation: Project > Insights > At a Glance

Item	Detail
Answers	Is the program healthy, and does it pay for itself?
Counts	Production conversations only.
Colour bands	Healthy, watch, and critical bands come from Settings -> Insights Config. You can change them per project.
Comparison	The page shows no comparison with the previous period.
Tabs	Overview, Trends, ROI, Conversations

Page filters

Three controls at the top of the page scope everything below them: an environment filter, a channel filter, and a date range. The date range offers 7, 30, or 90 days, or a custom range of up to 90 days. The filters are shared across the Insights pages, so a range you set here carries over to the next page. Narrow the environment filter when production and non-production traffic would otherwise mix. Test conversations distort containment and cost figures more than any other input. When you change a filter, a Reset filter control appears with the number of filters you applied. Check it before you conclude that a period was quiet, since a narrowed view looks much the same as a slow week.

Metric cards

Six cards summarize the selected period. Each card shows a headline value and a supporting figure that supplies its denominator. Read the two together, because a containment rate means one thing across a thousand conversations and something else across one.

Metric	Description
Conversations	Counts total conversations in the period and shows how many of them pipelines evaluated. The evaluated count forms the sample that produces every other card on this page, so a wide gap between the two figures means the rest of the view describes only part of your traffic.
Containment Rate	Shows the share of evaluated conversations the agent resolved without handing off to a person. Most programs treat this as their headline automation measure. The card takes a warning treatment when the rate falls below the expected threshold.
Quality Score	Averages the scores the quality evaluation assigns to the agent’s responses. Use it as the counterweight to containment. Containment alone can rise because the agent stopped offering escalation, and quality tells you whether the agent handled the conversations it kept.
Avg Sentiment	Averages sentiment across evaluated conversations. Quality judges what the agent produces, and sentiment reflects how the customer responds to it. The two measures can diverge, and the gap between them usually carries the signal.
Period Savings	Models the cost you avoid when the agent contains a conversation instead of a person handling it. The figure rests on the assumptions you set in ROI cost settings, so treat it as directional and revisit it when your staffing costs change.
Escalation Rate	Shows the share of evaluated conversations the agent handed to a person. This rate isn’t the inverse of containment. Conversations the agent handled only in part and didn’t escalate count toward neither rate, so both can read low over the same period.

A card shows a dash when the pipeline behind it produces no value for the period. A dash means the measure has no data, not that it scored zero, and it usually indicates that the relevant pipeline isn’t running. The following table shows how the platform calculates each card and the default colour bands.

Metric	How it’s calculated	Default colour bands
Conversations	Counts finished production sessions whose end time falls in the period. A supervisor’s handoffs to child agents count as one conversation. Live sessions don’t count. The evaluated figure counts the conversations that have a result from at least one pipeline.	None
Containment Rate	Each finished conversation gets one outcome from the Quality Evaluation pipeline. Escalated means the session raised an escalation. Abandoned means the customer timed out or left. Every other conversation stayed with the AI (contained), and an LLM judge labels it resolved, partial, or unresolved. The card divides contained resolved by all conversations with an outcome.	Healthy at 70% or more. Watch from 50%. Critical below 50%.
Quality Score	Averages the latest overall quality score of each conversation, on a 1 to 5 scale. The overall score weights helpfulness and accuracy at 1.0, and professionalism and instruction following at 0.8. Needs the Quality Evaluation pipeline.	Healthy at 3.5 or more. Critical below 2.5.
Avg Sentiment	Comes from the Sentiment Analysis pipeline, which must be on. An LLM scores every message from -1 to +1. Each conversation takes the mean of its messages, and the card averages the conversations. Agent messages are included, not only customer messages.	Healthy at 0.50 or more. Watch from 0. Critical below 0.
Period Savings	Multiplies contained conversations by the human cost per conversation, then subtracts the actual LLM cost of those conversations.	Green above $0. Red below $0. You can’t change this rule.
Escalation Rate	Divides the conversations that raised an escalation event by all conversations with an outcome. Needs the Quality Evaluation pipeline to assign outcomes.	Healthy up to 15%. Watch up to 25%. Critical above 25%.

Each total is a subset of the one before it. Conversations counts every finished conversation. The evaluated figure counts conversations with a result from at least one pipeline. The outcome total, shown under Outcome Distribution, counts conversations that the Quality Evaluation pipeline gave an outcome. A conversation that only the sentiment or intent pipeline analyzed counts as evaluated but has no outcome yet.

Containment Rate isn’t the same as the Resolution Rate on Customer Insights. Containment asks whether the AI resolved the conversation without a person. Resolution asks whether the customer’s main intent was resolved, whatever happened along the way.

Overview tab

The Overview tab explains the headline numbers. Conversation Volume & Containment Rate plots volume and containment on one timeline. The chart separates the two ways a containment figure moves: the agent resolves a different share of conversations, or the volume changes underneath a stable share. The chart counts volume by the date each session ended, and containment by the date each session started. Days with no data plot as zero, so the current partial day falls near zero. Outcome Distribution divides the evaluated conversations by how they ended. Each category appears as a segment with its count, its share of the total, and a caption that describes the outcome. A footer states how many conversations have an outcome. That total is the denominator of both Containment Rate and Escalation Rate.

Outcome	Description
Contained resolved	The agent resolved the request with no human help. This count is the numerator of Containment Rate.
Contained partial	The agent handled the conversation but didn’t carry it to a full resolution. These conversations count toward neither the containment rate nor the escalation rate, so both rates can read low over the same period.
Escalated to human	The agent handed the conversation to a person. The platform sets this outcome when the session raises an escalation event.
Contained unresolved	The agent kept the conversation, but the judge couldn’t confirm that it resolved the request.

The partial and unresolved categories are the reason to read this section rather than the cards alone. Containment and escalation together imply a binary outcome, and these categories hold the conversations that fit neither.

What’s Driving the Metrics attributes the period to individual agents, so you can locate the cause of a project-level movement.

Column	Description
Volume	Counts the evaluated conversations that belong to this agent. When several pipelines ran, the column takes the largest single-pipeline count, so no conversation counts twice. This is evaluated volume, not total traffic. Sort on this column to find the agents that carry enough weight to move the project figures, since a poor score across a handful of conversations rarely explains an aggregate shift.
Confidence	Shows the average intent classification confidence for this agent’s conversations, from 0 to 100%. Green above 85%, plain from 50%, amber from 25%, red below 25%. Low confidence points to unclear requests or a weak intent list.
Quality	Shows this agent’s own quality score on a 1 to 5 scale, so you can see whether the project average describes your agents evenly or conceals a spread.
Sentiment	Averages sentiment across this agent’s conversations, from -1 to +1.
Trend	Shows the agent’s daily quality as a sparkline. It appears once the project has at least two days of data. Green at 3.5 or more, amber from 2.0, red below 2.0.

Trends tab

The Trends tab answers questions about direction rather than current state. Three charts cover the period. Multi-Metric Trends plots containment, escalation, and quality on one timeline. Reading the three together is what makes the tab useful, because a containment rise that coincides with a quality fall is a different event from one that doesn’t. Quality uses a 1 to 5 scale but shares the 0 to 100 axis with the two rates, so its line looks flat near the bottom. Point to the line to read its real value.

Sentiment & Quality Over Time plots average quality and average sentiment. Point to any date to read its values. Both series share one axis, so small sentiment changes look smaller than they are.

Daily Outcome Distribution shows the outcome mix day by day. Use it to tell whether a change in the period totals comes from a shift in how conversations end or simply from more of them. The chart shows three series: Resolved, Escalated, and Other. Other groups partial, unresolved, abandoned, and unclassified outcomes. A growing Other band means more conversations end without a clean resolution.

ROI tab

The ROI tab converts contained volume into money and staffing. Every figure except cost per conversation rests on the assumptions in cost settings, so agree those with your finance team before you circulate anything from this tab.

Metric	Description
Period Savings	Shows the modeled human cost you avoided, net of what the AI spent. The platform multiplies contained conversations by the human cost per conversation, then subtracts the LLM cost of those conversations. Green at $0 or more.
Period ROI	Expresses that saving as a return on your AI spend: period savings divided by the LLM cost of contained conversations, × 100. The card shows 0 when contained conversations have no cost. Green above 100%, amber above 0%. For example, 5,000% means savings are about 50 times that AI spend. The card takes a warning treatment when the period produces no positive return, which is normal early in a deployment while volume stays low and the assumptions stay fixed.
FTE Equivalent	Restates contained volume as human staffing capacity: contained conversations × average handle time, divided by 9,600 minutes (160 hours, one working month). The capacity value underneath is that FTE × the monthly cost of one person. The divisor is always one month, so a 7-day range understates the figure and a 90-day range overstates it. Use it in headcount discussions, where an FTE figure lands better than a savings total.
Cost / Conversation	Divides total LLM cost in the period by Conversations. The platform estimates each LLM call’s cost from its input and output tokens and its model’s price. A session’s cost is the sum of its calls. A call whose model has no known price carries no cost. These figures are estimates, not provider invoices. When the platform records no real cost, it uses the estimated AI cost from cost settings. Green while it stays below the human cost per conversation. The cost total includes non-production traffic, but the conversation count doesn’t. This card reports real spend rather than your assumptions, so quote it when someone disputes the assumptions.

AI Cost vs Estimated Human Cost plots both costs across the period and states the per-conversation human cost it applies, so anyone who reads the chart can see the assumption behind the comparison. The platform attributes actual model costs to contained conversations.

Series	How it’s calculated
Est. Human Cost	Resolved conversations that day × human cost per conversation.
AI Cost	Cost per conversation × that day’s conversations.

The gap between the two series is the daily saving.

Conversations tab

The Conversations tab lists the individual conversations behind the metrics, 25 per page. Select a conversation to open its session record, which holds the full trace, cost, and message history. Use this tab when a metric looks wrong and you need to read the conversations that produced it. Use the quick filter to narrow the list.

Filter	Shows
All	Every conversation.
Flagged	Conversations with a quality score below 2.5.
Low Quality	Conversations with a quality score below 3.0.
High Quality	Conversations with a quality score above 4.0.

Column	Description
Date	When the conversation took place.
Agent	The agent that handled it.
Status	OK or Flagged, from the quality evaluation. A conversation is flagged when its quality score falls below the flag threshold, 2.5 by default.
Quality Score	Overall quality, 1 to 5. Green at 3.5 or more, plain from 2.5, red below 2.5.
Helpfulness	The judge’s helpfulness score, 1 to 5. One input to the quality score.
Accuracy	The judge’s accuracy score, 1 to 5. One input to the quality score.

Open flagged conversations first. They show what the agent got wrong.

ROI cost settings

Select Cost Settings on the ROI tab to set the assumptions that produce the ROI figures, then select Save. Nothing here affects what the platform bills you. These values only determine the baseline that the platform compares your actual spend with.

Field	Description
Human Cost per Conversation ($)	Sets the loaded cost of one conversation that a person handles. The platform credits every containment with avoiding this amount, so this field drives the savings figures and the human cost line on the ROI chart more than any other input.
Human FTE Monthly Cost ($)	Sets the fully loaded monthly cost of one full-time agent. The platform uses it to convert avoided volume into the FTE Equivalent figure.
Avg Human Handle Time (minutes)	Sets the staffed time that one conversation consumes. With the FTE cost, it determines how much contained volume adds up to one person’s capacity.
Estimated AI Cost per Conversation ($)	Sets the fallback cost the platform applies when it can’t attribute actual model cost to a conversation. Without this value, unattributed conversations appear to cost nothing and overstate your savings.

The default values are $12 human cost per conversation, $4,500 monthly FTE cost, 6 minutes average handle time, and $0.05 estimated AI cost per conversation.

Billing and usage

Billing and Usage shows what the project consumed, how much of that consumption bills, and the policy that decides the difference. Finance and operations teams use it to reconcile spend and to explain gaps between activity and charges. Navigation: Project -> Insights -> Billing & Usage

Item	Detail
Answers	What will this project be billed for?
Counts	Billable sessions only. Test, eval, synthetic, and excluded-channel sessions are left out, so totals sit below the Insights totals by design.
Source	Published billing report rows when they exist. Otherwise, a current estimate.
Date range	Its own picker: 7, 30, or 90 days. It doesn’t follow the Insights filters.

Data currency

The page reads published billing report rows rather than live traffic, and it states when the platform last published those rows. Published rows are the billing record. When no published rows exist for the period, the page shows a current estimate. Check the timestamp before you reconcile a period, because recent activity appears only after the next materialization. A date range control scopes the page.

Usage and billing units

Metric	Description
Included Sessions	Counts the sessions that meet the billing policy and therefore contribute to charges. This number falls below your total session count whenever exclusions apply, and the page itemizes the difference further down.
Billing Units	Counts the chargeable units the platform derives from included sessions using the billing interval and the duration basis. Each session earns ceil(duration ÷ billing interval) base units. LLM, tool, and voice-minute add-ons are added on top. This number maps to your invoice.
LLM Calls	Counts model invocations in the period. This card can show substantial activity while billing units sit at zero, because calls inside excluded sessions still run. Treat a wide gap between the two as a prompt to check your exclusions.
Interactive Turns	Counts user turns across the period, after billing exclusions. The interaction threshold in the billing policy uses turn count to decide whether a session carries enough interaction to bill.

Daily Billing Units plots billing units and included sessions across the period, so you can trace a spike in charges to either more sessions or longer ones. Channel Breakdown reports sessions, units, and model calls per channel. It shows which channel drives your spend, and it exposes an excluded channel as activity that produced no units. For example, web_debug is the Studio test chat. Its LLM calls are counted, but its sessions are excluded from billing. Voice minutes reports voice consumption separately, because voice meters in minutes rather than in the unit model that governs the rest of the page.

Excluded sessions

This section lists the sessions the latest materialization didn’t bill, grouped by the reason that excluded each one. Start here when session volume and billed volume disagree. The reason names the policy rule that removed the sessions, which turns an unexplained gap into a policy question.

Reason	Meaning
`excluded_channel`	The channel is on the excluded channels list in the billing policy.
`below_interaction_threshold`	The customer sent fewer messages than the interaction threshold.

Test, eval, and synthetic sessions are also left out of billing.

Usage by billing scope

This section groups sessions by billing scope. When one agent delegates to another, the parent and its children roll up into a single scope, so the platform counts a multi-agent conversation once rather than once per agent. Read this section when your agent architecture makes the session count look higher than the work the customer experienced.

Resolved billing policy

This section shows the billing policy in effect for the tenant and the level that sets each value. The source label matters most: it tells you whether a value comes from your plan or from this project, and therefore whether changing it needs your account team or your own settings.

Setting	Description
Billing interval (minutes)	Sets the block of session time that one billing unit covers. It determines how session duration rounds into units, so a long interval and short sessions together decide how much of what you pay for goes unused.
Duration basis	Selects which measure of session duration the platform meters: engaged time (while the conversation is active) or wall-clock time. It decides whether idle time inside a session becomes chargeable.
Scope consolidation	Controls whether the roll-up of delegated agent sessions applies to billing or only to reporting. Report only shows the grouping without changing the bill. It determines whether multi-agent fan-out costs more than a single-agent conversation.
Excluded channels	Lists the channels whose sessions never bill. Debug and test channels usually sit here, which explains how development traffic generates model calls without generating charges.
Interaction threshold (min user messages)	Sets the minimum number of user messages a session needs before it bills. It keeps abandoned and no-touch sessions out of your charges.
Source	Shows where each value comes from, such as the plan or a tenant override.

Get started

Build

Test and evaluate

Deploy

Analytics and insights

References

At a Glance

Page filters

Metric cards

Overview tab

Trends tab

ROI tab

Conversations tab

ROI cost settings

Billing and usage

Data currency

Usage and billing units

Excluded sessions

Usage by billing scope

Resolved billing policy

​At a Glance

​Page filters

​Metric cards

​Overview tab

​Trends tab

​ROI tab

​Conversations tab

​ROI cost settings

​Billing and usage

​Data currency

​Usage and billing units

​Excluded sessions

​Usage by billing scope

​Resolved billing policy

At a Glance

Page filters

Metric cards

Overview tab

Trends tab

ROI tab

Conversations tab

ROI cost settings

Billing and usage

Data currency

Usage and billing units

Excluded sessions

Usage by billing scope

Resolved billing policy