Your Voice Agent Passed Every Test and Still Fails Callers: Measuring Task Success, Containment and Escalation (2026)

1. Three layers of voice agent evaluation

Evaluation frameworks published in 2026 split voice agent quality into roughly three layers. They use different names, but the split is consistent (LangChain, Hamming):

Engineering teams usually start with execution because it is easy to automate and easy to own. Execution metrics are necessary, but they don't tell you whether callers are being served. An agent can transcribe perfectly, answer in 600 milliseconds and still send a caller round a loop three times before they hang up. If you have already worked on latency and recognition of names and numbers, the outcome layer is the next place to look.

2. Why containment is a trap metric

Containment is usually defined as the share of calls that finish without a transfer to a human. It became the headline metric because it maps directly to cost: each contained call is a call no agent had to take.

The flaw is what containment leaves out. All of these calls count as contained:

• A caller who got the answer they needed and hung up satisfied.
• A caller who said "agent" four times, wasn't transferred and hung up.
• A caller who gave up after a confirmation loop and will call back tomorrow.
• A caller who hung up and opened a chat or email ticket instead.

Only the first is a success. Any team rewarded on containment alone will drift toward the second and third, usually without anyone intending it, because making a transfer harder to reach raises the number. Making escape harder raises the score.

The fix is to stop reporting containment alone. Report resolved containment instead: calls that ended without a transfer and where the task was completed, with no repeat contact from the same caller on the same issue within a set window (seven days is a common choice). That one change tends to cut a headline number considerably, which is the reason to make it.

3. Seven outcome metrics and their guard metrics

Any single metric can be gamed. A guard metric is a second number that gets worse when someone games the first one. Report them together.

Escalation quality is the most neglected item in that table. A transfer isn't automatically a failure: some calls should go to a human straight away. A transfer is only a failure when the caller has to repeat their name, account and problem to the person who picks up. Measure the share of transfers where a summary reached the human agent and the caller didn't have to restate anything. That is a direct, measurable cost of a bad handoff.

4. What "good" looks like in 2026

Published benchmarks vary by use case, and vendors that sell testing tools publish many of them, so treat these as orientation rather than targets. One 2026 production reference puts good performance at task completion above 85%, first-call resolution above 75% and containment above 70% (Hamming).

Two points matter more than the numbers themselves:

• Benchmarks differ by call type. A booking or status-check line should beat these figures. A billing-dispute line probably shouldn't contain 70% of calls, and if it does, check the guard metrics first.
• Your own trend matters more than an industry figure. A drop in resolved containment after a prompt or model change tells you more than how you compare with someone else's line.

Industry write-ups also suggest that about 80% of evaluation can be automated: latency, WER and task completion on simulated calls. The rest needs human review: tone, edge cases and checking that the automated grader agrees with people (Hamming). Budget for that remaining 20%.

5. Where the voice shows up in outcome data

Teams often treat text-to-speech as a solved component, judged once at selection by a listening test. In production it affects the outcome metrics in specific, measurable ways:

None of these show up in a WER or latency dashboard. They show up as a change in reprompt rate, abandonment or task success on a particular turn. That is why you should log outcomes per turn, not only per call. Per-call logging tells you that a call failed. Per-turn logging tells you which line of audio it failed on.

6. A test plan you can run this month

1. Pick your five highest-volume intents. Ignore the long tail until these five are measured properly.
2. Write a "done" definition for each intent that can be checked downstream. For example: "Booking exists in the calendar with the right date, time and name." "The agent said it booked it" doesn't count.
3. Build a simulated call set for each intent: the happy path, two edge cases, and two difficult callers (accent, background noise, interruptions, someone who changes their mind).
4. Run it before every prompt, model or voice change and treat any drop in task success as a failed build. Execution checks such as pronunciation and latency can sit in the same CI run.
5. In production, report each outcome with its guard metric from the table in section 3, every week, split by intent.
6. Each week, have a human listen to 20 random calls that the system marked as successful. If the reviewer disagrees with the grader on more than a few, fix the grader before trusting the dashboard.
7. Each week, also review 10 abandoned calls, using per-turn logs to find the line where the caller left.

This doesn't need a new tool. A spreadsheet and a listening rota will get you most of the way. The discipline matters more than the platform.

7. A/B testing prompts and voice

Once outcomes are logged per turn, you can test prompt and voice changes against task success rather than taste. For example:

• Test the confirmation line at two speaking rates, and measure the reprompt rate on that turn.
• Test the greeting at two lengths, and measure barge-in and hang-up rates in the first ten seconds.
• Test two voices on the same script, and measure task success and post-call rating. Run this only once the prompts themselves are stable.

Pre-rendering fixed lines helps twice. It removes TTS latency from those turns (see the latency budget), and it removes TTS cost (see the cost-per-minute breakdown). It also makes A/B tests clean, because variant A is the same audio file every time. You change one variable and measure one turn.

Run a test long enough to collect a few hundred calls per variant on the turn you're measuring before you trust a difference. A low-volume line will need weeks for that.

8. FAQ

One 2026 benchmark puts good containment above 70%, but the figure means little on its own. Report resolved containment instead: no transfer, task completed and no repeat contact within about seven days. Report it next to the rate of callers who asked for a human and weren't transferred.

Containment only means the call didn't reach a human. Task success means the caller's goal was actually achieved, ideally verified downstream: the booking exists, or the payment posted. A call can be contained without being successful.

No. Some calls should go to a person. A transfer counts as a failure when it should have been avoidable, or when the human doesn't receive the context and the caller has to repeat everything. Measure escalation quality as well as escalation rate.

Industry estimates put it at about 80%: latency, WER and task completion on simulated calls. Tone, edge cases and checking the automated grader against human judgement still need people listening.

Yes, on specific turns. Numbers read too fast, prompts that are too long and unclear pronunciation all show up as reprompts, confirmation loops and hang-ups. Log outcomes per turn to see it.