[AINews] GPT-Realtime-2, -Translate, and -Whisper: new SOTA realtime voice APIs

TL;DR

Prime Big Deal Days · Oct 6–7Offer from Amazon

Get the latest gadgets delivered free — and shop member deals

• Fast, free delivery on millions of items
• Access to Prime Big Deal Days deals on October 6–7
• Prime Video, Amazon Music and more included

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

OpenAI has released three new streaming audio models—GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper—offering enhanced real-time voice, translation, and transcription. These models support longer conversations, tool use, and higher reasoning, marking a significant step forward for voice AI.

OpenAI has unveiled GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper, its most advanced real-time voice and speech APIs to date, now accessible via the Realtime API. The new models aim to bring GPT-5-level reasoning to live voice interactions, enabling more natural, responsive, and capable voice agents.

The GPT-Realtime-2 model is positioned as a highly intelligent speech-to-speech system supporting tool use, interruption recovery, and longer conversations, with a context window reportedly expanded to 128K tokens. It is designed for production voice agents requiring complex reasoning and multi-turn dialogue, with independent benchmarks showing top performance in speech reasoning and instruction retention.

Complementing it, GPT-Realtime-Translate provides streaming translation from over 70 input languages into 13 output languages, facilitating real-time multilingual communication. GPT-Realtime-Whisper offers low-latency transcription and captioning, supporting continuous speech understanding for applications like meeting notes and live captioning. All three models are now available in the Realtime API, with OpenAI indicating that ChatGPT voice features are still being upgraded, with a rollout expected soon.

Why It Matters

This development represents a major leap in voice AI capabilities, enabling more natural, context-aware, and interactive voice interactions in real time. It could transform customer service, accessibility, and enterprise communication by providing more sophisticated, human-like voice agents capable of reasoning and multi-language support.

The models’ ability to handle longer conversations and call multiple tools simultaneously could reduce the need for human intervention and increase automation efficiency. As voice interfaces become more capable, they may finally achieve broader adoption beyond niche applications, impacting how users interact with AI daily.

Background

OpenAI’s previous streaming audio models, launched three months ago, offered limited reasoning and context capabilities. The new models significantly expand on this, with a reported 128K context window—four times larger than the prior 32K limit—allowing for more sustained and complex interactions. Industry benchmarks from Scale AI and independent analysts show these models outperform earlier versions in speech reasoning, instruction retention, and real-time responsiveness.

This release follows a broader trend of integrating voice capabilities into AI systems, driven by user demand for more natural, conversational interfaces. OpenAI’s announcement also aligns with ongoing industry efforts to improve multilingual and multimodal AI applications.

“Users are increasingly turning to voice with AI when they need to communicate complex context, and our new models are designed to meet that demand with GPT-5-class reasoning in real time.”

— Sam Altman, OpenAI CEO

“The new models support longer context, tool use, interruption recovery, and more controllable tone, making them suitable for production-level voice agents.”

— OpenAI Developer Blog

What Remains Unclear

It is still unclear when the ChatGPT voice upgrade will be fully rolled out, as OpenAI indicated ongoing development. The precise technical details of the 128K context window and its practical performance in diverse real-world scenarios remain under evaluation. Additionally, the long-term impact on user adoption and interface design is yet to be seen.

What’s Next

OpenAI is expected to gradually roll out ChatGPT voice enhancements aligned with these models, possibly within the next few weeks. Developers and enterprise users will likely begin integrating these APIs into live applications, with ongoing updates to improve stability, features, and multi-language support. Industry analysts anticipate further benchmarks and real-world testing to validate the models’ capabilities.

Key Questions

What are the main capabilities of GPT-Realtime-2?

GPT-Realtime-2 is a speech-to-speech model supporting complex reasoning, tool use, longer conversations (up to 128K tokens), interruption recovery, and tone control, making it suitable for advanced voice agents.

How does GPT-Realtime-Translate improve multilingual communication?

It provides streaming translation from over 70 input languages into 13 output languages, enabling real-time multilingual conversations and applications.

When will ChatGPT voice features be upgraded?

OpenAI has not provided a specific date but indicated that the voice upgrade is in progress and will be announced soon.

What are the benchmarks indicating the models’ performance?

Independent benchmarks report GPT-Realtime-2 achieving 96.6% on speech reasoning and instruction retention of 70.8%, with top scores on various speech and conversational benchmarks.