Cozzo
Find yours

Cozzo-tested · Round 4 · S1 · October 22, 2026

Callback number read-back: AI receptionist test

A receptionist that books the appointment but writes down the wrong phone number has lost you the patient. So we checked the boring part: did the agent read the caller’s name and callback number back correctly?

What we found

  • No agent passed all 3 booking calls. Ultravox and our self-hosted LiveKit pipeline did best at 2 of 3.
  • Vapi, ElevenLabs and Bland passed 1 of 3. Millis, Retell and OpenAI passed none (one Retell read-back was too garbled to score).
  • When the caller asked for a callback in S3, Ultravox, OpenAI and Bland took the message but read the number back wrong at least once.

If your agent takes callbacks, have it repeat the number in small groups (412, 555, 0187) and ask the caller to confirm. Then test it with real calls before you go live.

Calls passed, by agent

S1: Book a new patient (calls passed out of 3)
  1. Ultravox
  2. LiveKit
  3. ElevenLabs
  4. Vapi
  5. Bland AI
  6. Millis AI
  7. Retell AI
  8. OpenAI Realtime

What the agents actually said

The caller’s number was 412-555-0187. Examples of wrong read-backs from our Round 4 calls:

  • Millis AI

    • Read back 480-255-50187

  • Ultravox

    • Read back 412-555-4187
    • Read back 421-255-0187 (S3 callback)

  • ElevenLabs

    • Read back 421-1255-0187

  • OpenAI Realtime

    • Read back 415-555-0187
    • Read back 412-550-187
    • Read back 412-555-0877

  • LiveKit

    • Read back 421-255-0087

  • Retell AI

    • “412555, Aurora, wait 7” (scored unclear)

  • Bland AI

    • Didn’t read the number back on one call; its own transcript logged a 402 number

Related: S3, callback for an unknown insurer (calls passed out of 3)

  1. ElevenLabs
  2. Ultravox
  3. Millis AI
  4. Retell AI
  5. OpenAI Realtime
  6. Bland AI
  7. Vapi
  8. LiveKit

† Callback taken, but the number was read back wrong at least once. Vapi said it can’t arrange callbacks; our LiveKit pipeline said it can’t take messages.

Show full table
AgentS1 passedNote
ElevenLabs Agents1/3
Ultravox2/3
Millis AI0/3
Retell AI0/31 unclear
OpenAI Realtime API0/3
Vapi1/3
Bland AI1/3
LiveKit Agents (self-hosted)2/3

The test (S1: Book a new patient)

Pass = books a listed slot, offers no unlisted slot, and reads back both the caller’s name and callback number (412-555-0187) correctly. One wrong digit is a fail.

The pass/fail rubric was fixed before the scored calls. Each agent got the same dental-office prompt with three open slots, three accepted insurers and Saturday hours of 9 AM to 1 PM.

Limits

  • 3 calls per scenario per vendor, one dental-office prompt, English only, one evening (Oct 22, 2026). Small differences are noise.
  • Read-backs were scored from our AI caller’s speech recognition plus a Whisper transcript of the recording, so a digit misheard on our side is possible. Every call was checked by hand.
  • Bland’s scores combine two sessions. Its S4 score mixes 2 scripted cut-ins (both over the limit) with 1 earlier call using our AI caller’s own interruption (passed). Bland dialed in over the phone network; the others were reached over SIP.
  • LiveKit is our own self-hosted pipeline (gpt-4o-mini), not a hosted product. OpenAI ran gpt-realtime-2.1 through OpenAI’s SIP endpoint.
  • Each hosted agent ran on its default model and voice. Results show what you get before tuning.

Cozzo paid for every call. No vendor sponsored, reviewed or approved these results (see our disclosure).

FAQ

Can AI receptionists read back a phone number correctly?

Not reliably yet. In our test no agent read the callback number 412-555-0187 back correctly on all 3 booking calls. The best, Ultravox and our LiveKit pipeline, managed 2 of 3.

Which AI receptionist was best at taking a callback number?

Ultravox and LiveKit passed 2 of 3 booking calls. Vapi, ElevenLabs and Bland passed 1 of 3, and Millis, Retell and OpenAI passed none. On the separate callback-message test (S3), Vapi and LiveKit wouldn’t take a callback at all.

How did Cozzo score the read-back?

A pass meant the agent booked a listed slot, offered no unlisted slot, and read back both the caller’s name and callback number correctly. One wrong digit was a fail. We scored from our caller’s transcript plus a Whisper transcript of the recording and checked each call by hand.

Keep exploring

AI receptionist benchmark

All 5 scenarios and the full scorecard.

All voice AI benchmarks

Every round, plus head-to-head results.

Interruption handling

A scripted caller cut in on 8 voice agents.

Emergency calls

We called 8 AI receptionists about an urgent dental problem, 3 times each.

Reply speed

Median reply time on real receptionist calls: ElevenLabs was fastest of 8 agents at 1.06 s and LiveKit slowest at 2.00 s.

Best AI receptionist for small business

8 receptionist products compared on price and features.

Agents tested: ElevenLabs Agents, Ultravox, Millis AI, Retell AI, OpenAI Realtime API, Vapi, Bland AI, LiveKit Agents (self-hosted).