PolyAI Releases Dialog-RSN-1: An Audio-Native Dialog Model
PolyAI introduces Dialog-RSN-1, an audio-native dialog model fusing turn-taking, speech recognition, function calling, and response generation into a single system.
Stock photo for illustration only, not from the actual event
- PolyAI launched Dialog-RSN-1, an audio-native conversational model.
- Currently available only to existing customers and new customers via early access requests; no open weights or public API yet.
- Achieved top scores on the internal Dialog-Eval benchmark.
- Focused primarily on the English language for this initial release.
PolyAI has officially announced the release of Dialog-RSN-1, an audio-native dialog model designed to reason directly over raw audio. The model integrates turn-taking, speech recognition, function calling, and response generation into a unified workflow. At present, there are no open weights or public APIs available; existing customers can enable the feature immediately, while new customers are required to request early access.
Current voice architectures typically rely on two dominant paradigms, each with distinct compromises. A cascaded stack transmits only the best guess from the ASR to the LLM, discarding tone, hesitation, and recognition uncertainty before the LLM processes the input. Conversely, speech-to-speech models like GPT Realtime and Gemini Live retain audio features but bake the voice directly into the model, restricting pronunciation control and locking up a dedicated GPU for continuous full-duplex calls.
Stock photo for illustration only, not from the actual event
The architectural trade-off in conversational AI usually involves balancing low-latency turn-taking with deep linguistic comprehension. PolyAI's approach—utilizing an audio-aware input model while decoupling the generation phase to a separate TTS system—optimizes computational efficiency and gives developers granular control over the conversational flow, addressing longstanding latency and naturalness hurdles in voice agents.
Dialog-RSN-1 operates as audio-aware exclusively on the input side. A single model reasons over raw audio and hands off text generation to a separate, promptable text-to-speech system. Instead of continuous streaming, the system is probed on demand using a high-recall Voice Activity Detector (VAD) coupled with specific timers, ensuring that the initial token of a reply dictates whether the agent initiates speech.
To construct the model, PolyAI post-trained open-weight multimodal architectures using supervised and reinforcement fine-tuning on proprietary in-house data, evaluating base models including Gemma, GPT-OSS, Qwen, and Mistral. Engineering efforts targeted sub-300ms latency on A100 GPUs, incorporating attention cache prefilling and parallelized transcription that runs concurrently with response generation.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment