Open-source Audio Interaction model listens and reacts every 0.4 seconds

Additional Context

The release of Audio-Interaction coincides with a surge in real-time multilingual capabilities from major players. In June 2026, Google launched Gemini 3.5 Live Translate, which moved away from rigid turn-by-turn processing to a continuous stream model supporting over 70 languages across Google Meet and mobile apps. This transition reflect a broader industry push toward 'agentic' voice behavior, where models do not just respond to prompts but interpret emotional pacing and environmental context in real time. Per Android Headlines (June 2026), these advancements are increasingly designed to run across diverse hardware rather than remaining locked in proprietary ecosystems. At the same time, the competition for low-latency voice infrastructure is intensifying. OpenAI's GPT-Realtime-2, generally available as of May 2026, has already introduced reasoning-capable voice agents that use 'preambles' like 'One moment' to mask backend tool calls, effectively hiding processing gaps from the user (per NotebookCheck). Meanwhile, the open-source community is countering with hyper-specialized models like Rednote’s dots.tts, a 2B-parameter continuous pipeline, and Microsoft’s MAI-Voice-2, which enables zero-shot voice cloning across 17 languages (per Substack, June 2026). Benchmarks such as MMAU (Massive Multi-task Audio Understanding) have become the primary battleground for these models. While proprietary leaders like Amazon’s Nova 2 Omni hold top scores for expert-level reasoning, the gap is narrowing. Audio-Interaction’s ability to compete with 7B-parameter models while remaining open-source suggests that the next generation of voice-first AI will prioritize efficiency and proactive intervention over pure model size.