PolyAI CTO Reveals Why Voice AI Still Falls Short of Human-Like Conversations
By admin | Oct 11, 2026 | 3 min read
The idea that voice will become the next major interface has gained significant traction, with investors channeling billions of dollars into voice AI startups. These companies span everything from model developers to enterprise customer service platforms, and from meeting note-taking tools to AI-driven dictation services. It seems like every week brings a new model or tool that promises to sound and converse like a human. But the reality may be more complicated than the hype suggests.
Shawn Wen, CTO of enterprise voice AI platform PolyAI, believes that despite the arrival of full-duplex models—systems capable of speaking while simultaneously listening—voice AI still hasn't had its "ChatGPT moment." Speaking on stage at the HumanX conference last month, he explained, "We have reached the milestone of developing full-duplex models. The next challenge is to make reasoning very fast, so that the models can fetch answers quickly and the conversation feels natural." He also emphasized that AI agents handling customer service shouldn't sound robotic and need to give callers enough confidence that their problems can actually be solved. "I think the next stage will be slightly different because once the voice is good enough, like, and the customer is willing to engage with them for the first two or three turns, they start to build confidence, and over time, they will feel like I probably don't have to talk to a human if the agent can solve my problem," he said.
Alex Gay, CMO of meeting notetaker Otter, offered a different perspective, noting that speaker identification, intent capture, and combining that with organizational knowledge represent critical steps toward enabling automation. The company is also developing digital twins that could represent people in meetings. For this technology to work, Gay said it's essential that the output voice conveys the same emotional expressions as talking to a real human in a meeting. "If you think about the meetings that you're in right now, the best conversations that you have are where you can have debate, and strategic discussions, and when you feel like there's a relationship that underpins it. If you aren't able to have that with an avatar, then it's just a q and a chatbot," Gay noted.
**Voice AI's understanding and transparency**
Even as voice AI models have advanced, AI assistants frequently fail to understand users, or a meeting notetaker may produce the wrong transcript or summary. Wen pointed out that ASR (Automatic Speech Recognition) models often miss important keywords, which creates problems for capturing the full context. Gay from Otter concurred, adding that the company continues working to improve transcription. He also identified language as an area where voice models need improvement. "For Otter, you know, transcription was never the end point. It was just the layer that we could start to drive some of the productivity gains on the back of. But if your original transcription didn't have the accuracy that you needed, all the follow-up actions that you have become flawed. And the minute that starts to take action, that is wrong. You lose trust in the platform. It is critical for us to continue to improve that ASR model because all of the downstream impacts are significant," he said.
Transparency is another concern raised by these new voice tools. Products should inform customers when they are being recorded or when they are speaking with AI. Otter said it aims to build trust among meeting participants, so even for meetings where the bot isn't present, the company wants to explore methods like notifying everyone in the chat that the meeting is being recorded. PolyAI's Wen also stressed the importance of making it clear to people on enterprise calls that they are talking to an AI.
Comments
Please log in to leave a comment.
No comments yet. Be the first to comment!