Cozzo-tested · Round 5 · October 23, 2026
AI receptionist benchmark, round 5: 7 voice agents across 3 small-business industries
Round 5 called the same 7 voice-agent platforms with three small-business receptionist prompts: a roofing & HVAC company, a gym and a career coach.
Verdict
- Reading back a phone number is still the weak spot. With our caller fixed, the best roofing results were 3 of 4 correct read-backs (ElevenLabs, Millis, Vapi). Ultravox’s result is unresolved.
- Prices and slots: every agent refused to make up a roofing price. Every agent quoted the listed gym price on both calls. Five agents quoted the listed coaching prices on both calls (LiveKit 1/2, Bland 0/2). Every agent offered the listed gym and coaching slot on both calls.
- Safety triage passed everywhere: all 7 agents passed the emergency scenario in all three industries.
- After-hours was the most common miss in roofing (Ultravox, Vapi, LiveKit and Bland failed). In gym, only Ultravox failed. In coaching, all passed.
- Interruptions (fixed method): the fastest stops were Ultravox (0.42 s gym, 0.38 s coaching) and Retell (0.46 s both). Bland took longest (1.84 s gym, 2.44 s coaching).
- Fastest median reply: ElevenLabs in all three industries (1100, 1100 and 1070 ms).
One or two calls per cell is a first screen. Test your own script before you go live.
Roofing & HVAC
Calls passed out of calls made (1–2 per cell). Stop = seconds the agent kept talking after the cut-in.
| Vendor | Quote | Booking | Read-back | Triage | After-hours | Interrupt (s)◐ |
|---|---|---|---|---|---|---|
| ElevenLabs | 1/2 | 2/2 | 3/4 | 1/1 | 1/1 | 0.96 s |
| Millis | 1/2 | 2/2 | 3/4 | 1/1* | 1/1 | 0.14 s† |
| Retell | 0/2 | 1/2 | 1/4 | 1/1 | 1/1 | no overlap |
| Ultravox | Unresolved | Unresolved | Unresolved | 1/1‡ | 0/1 | no overlap |
| Vapi | 2/2 | 1/2 | 3/4 | 1/1 | 0/1 | no overlap |
| LiveKit | 1/2 | 0/2 | 1/4 | 1/1* | 0/1§ | 0.86 s |
| Bland | 1/2 | 1/2 | 2/4 | 1/1 | 0/1 | 1.92 s |
- Quote
- No price given, plus number read-back (out of calls made)
- Booking
- Listed slot offered, plus number read-back
- Read-back
- Callback number read back correctly, fixed caller (out of 4)
- Triage
- Emergency handled correctly
- After-hours
- Handled an after-hours request
- Interrupt (s)
- Seconds the agent kept talking after the cut-in
- ◐ Shaded
- Old interruption method: agent often not mid-sentence
- Markers
- * also offered a routine slot · ‡ also told the caller to call 911 · § test machine under CPU load · † speech may have ended at the cut-in
Quote and booking pass/fail come only from the fixed-caller re-run (28 calls, 4 per vendor). The number had to equal 412-555-0187, judged on the audio the agent spoke (Whisper). The address read-back was not scored, because the transcription garbles ZIP codes. All 7 agents refused to give a phone price on every quote call (14/14).
Ultravox: unresolved. Its own transcript shows the correct number on all 4 calls, but the recorded audio transcribes as wrong digits (e.g. “412-55087”). We can’t yet tell whether its speech or our transcription is at fault.
Retell: on one quote call the caller misspoke the number (an extra 5), and Retell read back the true number instead of what it heard. Scored as not correct. The same caller slip happened on one LiveKit quote call.
* Gave the emergency line but also offered a routine slot. ‡ Also told the caller to call 911 for a roof leak (over-cautious).
§ LiveKit’s after-hours call had ~60 s of silence while our test machine was under CPU load. Its roofing latency carries the same caveat.
Interruption, old method: the cut-in often arrived after the agent had finished its answer. Retell, Vapi and Ultravox had no overlap, so no stop time was measured. † Millis’s speech may have ended naturally right at the cut-in. Gym and coaching use the fixed method.
Gym
Calls passed out of calls made (1–2 per cell). Stop = seconds the agent kept talking after the cut-in.
| Vendor | Quote | Booking | Triage | After-hours | Interrupt (s) |
|---|---|---|---|---|---|
| ElevenLabs | 2/2 | 2/2 | 1/1 | 1/1 | 1.48 s |
| Millis | 2/2 | 2/2 | 1/1 | 1/1 | 0.60 s |
| Retell | 2/2 | 2/2 | 1/1 | 1/1 | 0.46 s |
| Ultravox | 2/2 | 2/2 | 1/1 | 0/1 | 0.42 s |
| Vapi | 2/2 | 2/2 | 1/1¶ | 1/1 | 0.92 s |
| LiveKit | 2/2 | 2/2 | 1/1 | 1/1 | 1.12 s |
| Bland | 2/2 | 2/2 | 1/1 | 1/1 | 1.84 s |
- Quote
- Quoted the listed price
- Booking
- Offered the listed slot
- Triage
- Told the caller to call 911
- After-hours
- Handled an after-hours request
- Interrupt (s)
- Seconds the agent kept talking after the cut-in
- Markers
- ¶ said “901” before correcting to 911
Number read-backs are omitted for gym: our test caller misspoke the callback number on most calls, so those results aren’t valid.
Ultravox after-hours fail: it said the gym was open until 9 PM, then offered to book a 10 PM tour. ¶ Vapi said “901” before correcting to 911.
Coaching
Calls passed out of calls made (1–2 per cell). Stop = seconds the agent kept talking after the cut-in.
| Vendor | Quote | Booking | Triage | After-hours | Interrupt (s) |
|---|---|---|---|---|---|
| ElevenLabs | 2/2 | 2/2 | 1/1 | 1/1 | 1.00 s |
| Millis | 2/2 | 2/2 | 1/1 | 1/1 | 0.64 s |
| Retell | 2/2 | 2/2 | 1/1 | 1/1 | 0.46 s |
| Ultravox | 2/2 | 2/2 | 1/1 | 1/1 | 0.38 s |
| Vapi | 2/2 | 2/2 | 1/1 | 1/1 | 0.84 s |
| LiveKit | 1/2# | 2/2 | 1/1 | 1/1 | 1.76 s |
| Bland | 0/2** | 2/2 | 1/1 | 1/1†† | 2.44 s |
- Quote
- Quoted the listed prices
- Booking
- Offered the listed slot
- Triage
- Panic-attack call handled correctly
- After-hours
- Handled an after-hours request
- Interrupt (s)
- Seconds the agent kept talking after the cut-in
- Markers
- # miss unconfirmed · ** never stated a price · †† hours given, no slot or message
Number read-backs are omitted for coaching, for the same reason as gym. Triage passed if the agent gave calming guidance plus 911/urgent care and/or 988 before (or instead of) booking. LiveKit, Millis, Retell and Vapi offered booking slots after the guidance.
# LiveKit: the listed price was $135 and our speech-to-text heard $35, with no vendor transcript to check it against, so this miss is unconfirmed. ** Bland never stated a price on either call; our caller spoke over its greeting. †† Bland stated the hours but offered no slot or message.
Median reply time
Milliseconds, with the number of replies measured in brackets (n).
| Vendor | Roofing (ms) | Gym (ms) | Coaching (ms) |
|---|---|---|---|
| ElevenLabs | 1100 (16) | 1100 (15) | 1070 (16) |
| Millis | 1360 (16) | 1300 (15) | 1500 (15) |
| Retell | 1540 (15) | 1280 (16) | 1280 (15) |
| Ultravox | 1600 (15) | 1520 (15) | 1340 (15) |
| Vapi | 2040 (15) | 1780 (15) | 1680 (15) |
| LiveKit | 2180 (14)§ | 1880 (15) | 2300 (15) |
| Bland | 1950 (12) | 1800 (10) | 1830 (8) |
- (n)
- Replies measured
- §
- Measured while our test machine was under CPU load
§ Roofing LiveKit was measured while our test machine was under CPU load. Roofing latency is from the first roofing run (the re-run only repeated quote and booking).
Methodology
On Oct 23, 2026, Cozzo placed real phone calls through Twilio. Each platform got the same prompt per industry, with fixed facts (prices, open slots, hours) and one fictional caller: Dana Whitfield, 412-555-0187.
An AI caller (built on Ultravox) played the customer. Bland dialed our test number over the phone network; the other platforms were reached over SIP. Interruption calls used a fixed scripted cut-in (Twilio text-to-speech), so every agent heard it at the same point.
Scenarios per industry: quote/pricing with a callback number (2 calls), booking a listed slot with name and number (2), urgent triage (1), after-hours request (1), and a scripted interruption (1).
Reply time = median gap from the end of the caller’s speech to the agent’s reply, on non-interruption calls (n = replies measured). Stop time = how long the agent kept talking after the cut-in, measured from the recording.
Every call was recorded on separate caller/agent channels, transcribed with Whisper, checked against each platform’s own transcript where available, and scored by hand.
Gym and coaching used an updated method: a longer question before the cut-in, so the agent is mid-sentence when interrupted, and the same message-taking instruction for every platform. Roofing ran first, on the earlier method. The roofing quote and booking calls were then re-run with the caller saying the number digit by digit.
LiveKit is our own self-hosted pipeline, not a hosted product. Cozzo paid for every call; no vendor sponsored or reviewed these results (see our disclosure).
Limits
- 1–2 calls per cell, one prompt per industry, one language, one day. Small differences are noise.
- Our test caller misspoke the callback number on most first-run calls. Read-backs are shown only for the roofing re-run with the caller fixed; even then the caller misspoke on 2 of 28 calls.
- Read-backs are judged from transcription of the recorded audio, so a digit misheard on our side is possible (see Ultravox).
- Roofing interruption stop times use the old method, where the agent was often not mid-sentence.
- OpenAI was not tested this round.