Skip to content

Voice & audio

AI speech and audio: speech synthesis, real-time conversation, music generation and audio understanding.

125 picksRelated topicsMultimodalAI videoProduct updates

Latest picks

81–100 of 125

May 12Tuesday

AI HOT (Curated Pool)

Thinking Machines Releases Native Multimodal Interaction Model for Real-Time Human-AI Collaboration

Thinking Machines released an interaction model that natively receives audio, video, and text input, processes foreground interaction at 200-millisecond intervals, and uses a background reasoning model for long-horizon planning and tool calls.

Why it matters: Thinking Machines splits real-time multimodal work into a fast 200ms front end and a slower planning back end, an architecture pattern worth studying.

The Verge · AI

Here’s What Mira Murati’s AI Company Is Up To

Thinking Machines announced work on “interaction models” that continuously take in audio, video, and text and respond or act in real time; the post does not disclose model size, release timing, pricing, or the final product format.

Why it matters: Thinking Machines says it is building an interactive model that takes in audio, video and text at once and responds in real time, instead of waiting for the user to finish typing.

May 8Friday

Synced · WeChat

OpenAI launches official CLI for terminal-based model access

OpenAI released the open-source openai-cli, letting developers call Responses, cloud tools, image generation and editing, speech transcription, and TTS from a single terminal command.

Why it matters: A terminal command replaces SDK integration for chat, image generation and speech, which lowers the setup cost for quick tests.

Latent Space

[AINews] GPT-Realtime-2, Translate, and Whisper: new SOTA realtime voice APIs

OpenAI released GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper in the Realtime API, with GPT-Realtime-2 expanding context from 32K to 128K and scoring 96.6% on Artificial Analysis Big Bench Audio.

Why it matters: Three realtime voice models ship on the Realtime API with context expanded from 32K to 128K, giving developers a clear read on the interface's limits.

QbitAI · WeChat

OpenAI releases three realtime voice models for reasoning, translation, and transcription

OpenAI launched GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper as API models, covering 128K-context voice reasoning, streaming translation from more than 70 input languages into 13 output languages, and realtime transcription priced at $0.017 per minute.

Why it matters: OpenAI shipped three realtime voice models covering listen-while-reasoning, streaming translation across 70-plus languages, and transcription at $0.017 per minute, resetting both the reasoning ceiling and the cost floor for voice apps.

AI HOT (Curated Pool)

Apple's First AI Wearable: Camera-Equipped AirPods Enter DVT Stage

Apple’s camera-equipped AirPods have entered DVT, with launch possible in September. Each earbud uses a low-res camera for visual Q&A with the upgraded Siri. The post cites Google Gemini support and a data-upload indicator light.

Why it matters: Apple's camera-equipped AirPods have reached design validation testing, with the shape and features largely settled; the camera lets Siri identify objects in front of the wearer and answer questions about them.

AI HOT (Curated Pool)

OpenAI launches official openai-cli for terminal API calls

OpenAI open-sourced openai-cli for direct API calls from the terminal. The Apache 2.0 tool installs via Homebrew or Go and covers Responses API, structured output, image editing, transcription, and key config. The key detail is Agent workflows using cloud tools like web search and code interpreter.

Why it matters: OpenAI open-sourced openai-cli, letting developers call the API from the terminal and chain cloud tools like web search and the code interpreter, skipping the SDK code they used to write.

The Verge · AI

Apple’s AirPods with cameras for AI are reportedly close to production

Mark Gurman says Apple’s camera-equipped AirPods are in DVT, one step before PVT. Testers are using prototypes; the cameras capture low-resolution visual input, not photos or video, for Siri queries like ingredient prompts.

Why it matters: Apple's camera-equipped AirPods have reached design validation testing, with the camera capturing only low-resolution images to help Siri understand visual questions.

May 7Thursday

OpenAI News

Advancing Voice Intelligence with New Models in the API

OpenAI introduced new realtime voice models in its API for voice intelligence. The RSS snippet says they reason, translate, and transcribe speech; the post does not disclose counts, pricing, or limits.

Why it matters: OpenAI ships three realtime voice models in the API, with GPT-Realtime-2 calling multiple tools while it reasons and GPT-Realtime-Translate handling 70-plus input languages into 13 output languages.

May 5Tuesday

r/LocalLLaMA

vibevoice.cpp: Microsoft VibeVoice ported to ggml/C++ with no Python at inference

LocalAI released vibevoice.cpp, a ggml/C++ port of Microsoft VibeVoice for CPU, CUDA, Metal, and Vulkan inference. TTS uses a 30s reference clip for 24kHz cloned speech; ASR uses a 7B model with diarized JSON and was tested on 17min audio. The key constraint is memory: 17min CPU Q8_0 peaks near 26GB, with no streaming output yet.

Why it matters: LocalAI rewrote Microsoft VibeVoice's speech synthesis and long-audio transcription as a pure C++ version that runs on CPU, CUDA, Metal and Vulkan, so local deployment no longer needs a Python environment.

May 4Monday

r/LocalLLaMA

Gemma 4 E2B runs well on an 8GB Android phone, powering a private voice notes app

A Reddit user ran Gemma 4 E2B locally on an 8GB OnePlus CE 5 and built a private voice notes app. Whisper Small 244MB transcribes, Gemma 4 E2B 2.4GB splits and tags, and a 10-15s note takes 12-15s end to end. Search uses query expansion, FTS lanes, RRF, and optional Gemma top-K reranking with a 15s fallback.

Why it matters: A Reddit user fit a 2.4GB Gemma 4 E2B and Whisper Small into an 8GB phone for a fully offline voice notes app.

May 2Saturday

Hacker News front page

Spotify Adds 'Verified' Badges to Distinguish Human Artists from AI

Spotify added 'Verified' badges for human artists to distinguish them from AI, per the title. The RSS snippet does not disclose the verification process, rollout scope, timing, or review criteria.

Why it matters: Spotify now marks real artists with a green check, using linked social accounts, listener data and tour history to separate people from AI acts.

May 1Friday

QbitAI · WeChat

He Used AI to Run a Music Festival About Not Doing a PhD

Bilibili creator Huntunpi Qiezong made 42 AI-generated “Don’t Do a PhD” songs, passing 50 million views. One track took over 100 generations, racing Suno, MiniMax Music, HeartMuLa, and ACE-Step. The key signal is the human curation cost in AI music workflows.

Why it matters: A Bilibili creator's 42 AI-generated songs against PhD study hit 50 million views, showing that human curation, not generation, is the real cost.

Bloomberg Technology

The Audio Industry Is Grappling with the Rise of ‘Podslop’

Podcast Index says 39% of new podcasts over nine days were likely AI-generated. The title cites industry concern over “podslop”; the post does not disclose detection methods, sample size, or platform split.

Why it matters: Podcast Index scraped nine days of new podcasts and found 39% may be AI-generated, giving a number for how audio supply is shifting.

Apr 30Thursday

Google DeepMind

Google DeepMind announces AI co-clinician medical research program

Google DeepMind announced an AI co-clinician research program, exploring how AI agents can assist patient care under a doctor's clinical supervision. In a blinded evaluation of 98 real primary care queries, the system made no critical errors in 97 cases, and doctors preferred its answers over mainstream evidence synthesis tools. On 140 consultation skills, it matched or beat primary care physicians on 68, but expert physicians were still better overall at spotting red flags and key physical exams.

Why it matters: Google DeepMind details its AI co-clinician research program: in a blinded review of 98 real primary care queries, 97 had no critical errors, and doctors preferred its answers over mainstream evidence synthesis tools.

r/LocalLLaMA

Building a fully local PDF-to-audiobook workflow with Kokoro 82M, Qwen and llama.cpp

Reddit user purellmagents shared a local PDF-to-audiobook workflow using Kokoro 82M, Qwen 3.5 0.8B/2B, and llama.cpp. The Tauri 2.0 app runs on an M1 Mac, reads 15 initial sentences, then prepares the next 15. The hard parts are PDF-text alignment, code snippets, tables, and first-generation latency.

Why it matters: A Reddit user built a fully local PDF-to-audiobook pipeline on an M1 Mac with Kokoro 82M, Qwen and llama.cpp, reading 15 sentences while preparing the next 15 to cut waiting time.

Apr 29Wednesday

Xinzhiyuan · WeChat

Google Translate Turns 20 as Pichai Highlights Four AI Generations

Google Translate turned 20 on April 28, and Pichai said it now has 1B monthly users. The post traces four AI phases: SMT, GNMT, PaLM 2, and Gemini 2.5 Flash Native Audio, including 110 languages added in 2024. The key shift is native speech-to-speech translation that preserves intonation, pacing, and pitch.

Why it matters: Twenty years of Google Translate, traced by Sundar Pichai from statistical machine translation to Gemini 2.5 Flash native audio translation that handles sound directly to keep tone and rhythm.

X · @dotey

Microsoft VibeVoice-ASR tested on Mac for a one-hour podcast

Simon Willison ran 4-bit VibeVoice-ASR on an M5 Max MacBook Pro and transcribed a one-hour podcast in 8m45s. The 9B MIT-licensed model supports 60-minute audio, 50+ languages, and structured speaker output. Memory is the constraint: prefill peaked at 61.5GB, making 32GB laptops impractical.

Why it matters: Simon Willison ran Microsoft's open-source VibeVoice-ASR on an M5 Max with 128GB of memory: 8 minutes 45 seconds to transcribe a one-hour podcast, with prefill memory peaking at 61.5GB.

Apr 28Tuesday

QbitAI · WeChat

ModelBest Releases MiniCPM-o 4.5 Technical Report for Consumer-GPU Deployment

ModelBest, OpenBMB, Tsinghua THUNLP and THUMAI released the MiniCPM-o 4.5 technical report, covering a roughly 9B-parameter model. It supports video, audio and text streams; a 12GB RTX 5070 runs full-duplex mode at RTF 0.4. The key mechanism is Omni-Flow: a unified timeline with time-division multiplexing, without external VAD.

Why it matters: ModelBest with Tsinghua and others published the MiniCPM-o 4.5 technical report, a roughly 9B-parameter model using Omni-Flow to time-share audio, video and text on one timeline, running full-duplex on a 12GB consumer GPU.

QbitAI · WeChat

Xiaomi open-sources MiMo-V2.5 series; Pro builds a macOS-like desktop in 4 hours

Xiaomi open-sourced MiMo-V2.5 weights, covering Pro Agent, multimodal base, TTS, and ASR models. MiMo-V2.5-Pro built a 54-app macOS-like desktop in 4 hours without human takeover; it scored 233/233 on SysY with 672 tool calls in 4.3 hours. Key details for practitioners are the 1M context, 100T-token program, and free Agent-framework access.

Why it matters: Xiaomi open-sourced the MiMo-V2.5 series weights, with the Pro version calling 672 tools over 4.3 hours without human help to run a macOS-like desktop with 54 apps, showing open-source agent execution on long-horizon tasks.