Twilio, the customer engagement platform that drives real-time, personalized experiences for today’s leading brands, announced an integration in collaboration with OpenAI to bring the company’s new Realtime API to the Twilio platform. The integration of streaming speech-to-speech (S2S) capabilities – part of the Realtime API – will enable over 300,000 Twilio customers and more than 10 million developers to build powerful conversational AI virtual agents leveraging OpenAI’s flagship multilingual and multimodal GPT-4o model.

“Businesses can use this to create voice experiences that feel more human and can reduce operational costs and drive higher customer satisfaction.”

The new integration builds on existing OpenAI and Twilio product integrations announced last year to bring the power of LLMs to the customer engagement platform.

“Integrating OpenAI’s Realtime API with Twilio’s platform enables businesses to offer more natural, real-time AI voice interactions at scale,” said Inbal Shani, Chief Product Officer, Twilio Communications. “Businesses can use this to create voice experiences that feel more human and can reduce operational costs and drive higher customer satisfaction.”

Marketing Technology News: MarTech Interview with Laurel Rossi, CRO and CMO @ Infillion

Speech-to-speech is an emerging technology that allows for voice conversations by AI virtual agents to feel more like real human dialogue. OpenAI’s Realtime API reduces latency and factors in key components like conversation pacing, interruption handling, tone, and balance between speaking and listening – all critical user experience elements that are essential for the right customer experience.

“The Realtime API’s speech-to-speech capabilities are designed to address strong customer demand for conversational AI solutions,” said Olivier Godement, Head of Product, API at OpenAI. “We’re thrilled to collaborate with Twilio to deliver a world class developer experience for building and deploying conversational AI agents.”

The technology is especially relevant for customer service and sales, delivering both operational efficiency and exceptional customer outcomes. Speech-to-speech is also set to support social impact at scale, empowering nonprofit and public sector organizations to deploy novel use cases like voice translation in real time between constituents and staff members who speak different languages.

Marketing Technology News: Social Listening And It’s Importance To Your Overall Marketing Content Strategy

Businesses will be able to connect these capabilities to Twilio’s customer engagement platform, enabling them to build conversational AI virtual agents into workflows like they would any other voice interaction. Previously, developers would be required to stitch together multiple vendors and solutions to create and deploy these agents.

Twilio’s native integration with OpenAI’s Realtime API with speech-to-speech capabilities makes it possible to build, deploy and serve customers with virtual agents on a single platform. Using Twilio’s scalable voice APIs and software, developers can use advanced features to record calls, view performance and analytics, and extract insights with AI operators. Those calls with virtual agents then become data that can be applied to improve operational efficiency and enable personalization at scale.

Twilio is committed to helping protect customers from new and emerging challenges with this technology such as deep fakes, voice based prompt injections and other emerging threats. As our understanding of these new risks evolves and solutions to these challenges are introduced Twilio is also committed to developing deeper integrations of this capability into our platform – including an upcoming integration with Twilio Alpha’s AI Assistants.