AI receptionist benchmark, round 6: a dental office, 9 voice agents, 3 calls per test

¹ Retell read the number back as “412-550-87” and as “4-pole-555-0187”; the second could be our speech-to-text garbling it. ² Vapi said ER/911, then offered cleaning slots (1 of 3 calls).

³ OpenAI quoted “$899” instead of $189 on one call. Its read-backs were correct on all 6 calls where they were tested.

⁴ LiveKit read back “412-5587”, “412-555-White-7” and “412-555-087” on the quote calls, and “412-55-187” on one booking call. It drops digits, the same kind of fault as Pipecat’s.

⁵ Pipecat is our own build, scored after the digit fix (see the Pipecat section). On one quote call the agent trailed off at “your number…” and the caller hung up; on one booking call it went silent after the number and the call hit Twilio’s 120 s limit with no confirmation.

⁶ Bland has 2 calls per test instead of 3, so its cells aren’t directly comparable. It also gave a garbled insurance answer (“out of network for Friday”) on its interruption call.

⁷ Millis: one booking call never reached the number (the agent greeted after the caller’s opening line, asked who the caller was, and the recording ends about 14 s in), one quote call read the number back as “412-555-50187”, and on one after-hours call the agent never answered the Saturday question before the caller hung up. These are scored as Millis misses but may partly be caller-versus-greeting timing.

OpenAI’s dental results are from a dental-only prompt. In the first run, on a combined four-business prompt, it failed the interruption test by asking which business the caller meant. That failure went away with the dental-only prompt (see the OpenAI section). Talk-overs count each time the agent started speaking over the caller.

Pass line = the slowest stop we measured in Round 5 (2.44 s, Bland, coaching). It is set by our own slowest result, so it ranks the agents against each other, not against an outside standard. Every agent that stopped within it also answered the Friday question correctly (8 a.m. to 1 p.m., closed in the afternoon). Pipecat’s replies after the interruption were slow on two calls (4.5 s and 4.7 s).

Not supported on our setup: Vapi, Retell, ElevenLabs, Millis, LiveKit and OpenAI.

Our script starts each callback (about 47 s after the message call ends), so this measures callback quality, not whether the vendor calls back on its own. Bland skipped the number on one callback. Ultravox’s number was scored from its own transcript; Bland’s callbacks from our speech-to-text only.

Change: on the combined prompt, OpenAI’s one dental interruption call failed because it asked which business the caller meant. With a dental-only prompt, all 3 interruption calls passed (stops of 0.10, 0.16 and 0.06 s) and every call had 0 talk-overs. The coaching and gym runs above are still on the combined prompt.

Gym after-hours passed. On the gym interruption call, our speech-to-text heard the Basic plan quoted as “20” (the listed price is $29), so we count it as a likely miss.

In the first run, 4 of 5 of our Pipecat bot’s number read-backs had missing digits. The cause was our text-to-speech voice (ElevenLabs flash v2.5) dropping digits when it read “412-555-0187”. The text the model wrote was always correct. The fix was to digit pronunciation: a text filter that spells the digits out. This is a fault in our own build, not in Pipecat or a vendor. In an offline test, 2 of 4 raw samples were fully correct and 3 of 4 were correct with the fix.

After the fix, no read-back dropped a digit: 10 of 12 read-backs were correct, and the other 2 had no read-back at all (the dental calls in footnote 5). Dental is only 3 calls, so treat it as a small sample. Median reply 1800 ms (29 turns) on dental. Its cost and speed include LiveKit Cloud, not a direct phone path.