Skip to content

#语音

4 today

May 12Tuesday

Latent Space

Thinking Machines' Native Interaction Models: TML-Interaction-Small 276B-A12B Advances Realtime Voice

Thinking Machines released TML-Interaction-Small, a 276B-parameter MoE model with 12B active parameters, and the post says it advances realtime voice through 200ms time-aligned microturns, encoder-free early fusion for audio and images under 200ms, and benchmark wins over GPT-Realtime-2 and Gemini 3.1-Flash.

Why it matters: Thinking Machines released TML-Interaction-Small, a 276B-parameter MoE with 12B active, building realtime voice interaction into the model instead of bolting on separate speech recognition and synthesis modules.

AI HOT (Curated Pool)

Thinking Machines Releases Native Multimodal Interaction Model for Real-Time Human-AI Collaboration

Thinking Machines released an interaction model that natively receives audio, video, and text input, processes foreground interaction at 200-millisecond intervals, and uses a background reasoning model for long-horizon planning and tool calls.

Why it matters: Thinking Machines splits real-time multimodal work into a fast 200ms front end and a slower planning back end, an architecture pattern worth studying.

The Verge · AI

Here’s What Mira Murati’s AI Company Is Up To

Thinking Machines announced work on “interaction models” that continuously take in audio, video, and text and respond or act in real time; the post does not disclose model size, release timing, pricing, or the final product format.

Why it matters: Thinking Machines says it is building an interactive model that takes in audio, video and text at once and responds in real time, instead of waiting for the user to finish typing.

May 8Friday

Synced · WeChat

OpenAI launches official CLI for terminal-based model access

OpenAI released the open-source openai-cli, letting developers call Responses, cloud tools, image generation and editing, speech transcription, and TTS from a single terminal command.

Why it matters: A terminal command replaces SDK integration for chat, image generation and speech, which lowers the setup cost for quick tests.

Latent Space

[AINews] GPT-Realtime-2, Translate, and Whisper: new SOTA realtime voice APIs

OpenAI released GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper in the Realtime API, with GPT-Realtime-2 expanding context from 32K to 128K and scoring 96.6% on Artificial Analysis Big Bench Audio.

Why it matters: Three realtime voice models ship on the Realtime API with context expanded from 32K to 128K, giving developers a clear read on the interface's limits.

QbitAI · WeChat

OpenAI releases three realtime voice models for reasoning, translation, and transcription

OpenAI launched GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper as API models, covering 128K-context voice reasoning, streaming translation from more than 70 input languages into 13 output languages, and realtime transcription priced at $0.017 per minute.

Why it matters: OpenAI shipped three realtime voice models covering listen-while-reasoning, streaming translation across 70-plus languages, and transcription at $0.017 per minute, resetting both the reasoning ceiling and the cost floor for voice apps.

AI HOT (Curated Pool)

Apple's First AI Wearable: Camera-Equipped AirPods Enter DVT Stage

Apple’s camera-equipped AirPods have entered DVT, with launch possible in September. Each earbud uses a low-res camera for visual Q&A with the upgraded Siri. The post cites Google Gemini support and a data-upload indicator light.

Why it matters: Apple's camera-equipped AirPods have reached design validation testing, with the shape and features largely settled; the camera lets Siri identify objects in front of the wearer and answer questions about them.

AI HOT (Curated Pool)

OpenAI launches official openai-cli for terminal API calls

OpenAI open-sourced openai-cli for direct API calls from the terminal. The Apache 2.0 tool installs via Homebrew or Go and covers Responses API, structured output, image editing, transcription, and key config. The key detail is Agent workflows using cloud tools like web search and code interpreter.

Why it matters: OpenAI open-sourced openai-cli, letting developers call the API from the terminal and chain cloud tools like web search and the code interpreter, skipping the SDK code they used to write.

The Verge · AI

Apple’s AirPods with cameras for AI are reportedly close to production

Mark Gurman says Apple’s camera-equipped AirPods are in DVT, one step before PVT. Testers are using prototypes; the cameras capture low-resolution visual input, not photos or video, for Siri queries like ingredient prompts.

Why it matters: Apple's camera-equipped AirPods have reached design validation testing, with the camera capturing only low-resolution images to help Siri understand visual questions.

May 7Thursday

OpenAI News

Advancing Voice Intelligence with New Models in the API

OpenAI introduced new realtime voice models in its API for voice intelligence. The RSS snippet says they reason, translate, and transcribe speech; the post does not disclose counts, pricing, or limits.

Why it matters: OpenAI ships three realtime voice models in the API, with GPT-Realtime-2 calling multiple tools while it reasons and GPT-Realtime-Translate handling 70-plus input languages into 13 output languages.

May 5Tuesday

r/LocalLLaMA

vibevoice.cpp: Microsoft VibeVoice ported to ggml/C++ with no Python at inference

LocalAI released vibevoice.cpp, a ggml/C++ port of Microsoft VibeVoice for CPU, CUDA, Metal, and Vulkan inference. TTS uses a 30s reference clip for 24kHz cloned speech; ASR uses a 7B model with diarized JSON and was tested on 17min audio. The key constraint is memory: 17min CPU Q8_0 peaks near 26GB, with no streaming output yet.

Why it matters: LocalAI rewrote Microsoft VibeVoice's speech synthesis and long-audio transcription as a pure C++ version that runs on CPU, CUDA, Metal and Vulkan, so local deployment no longer needs a Python environment.

May 4Monday

r/LocalLLaMA

Gemma 4 E2B runs well on an 8GB Android phone, powering a private voice notes app

A Reddit user ran Gemma 4 E2B locally on an 8GB OnePlus CE 5 and built a private voice notes app. Whisper Small 244MB transcribes, Gemma 4 E2B 2.4GB splits and tags, and a 10-15s note takes 12-15s end to end. Search uses query expansion, FTS lanes, RRF, and optional Gemma top-K reranking with a 15s fallback.

Why it matters: A Reddit user fit a 2.4GB Gemma 4 E2B and Whisper Small into an 8GB phone for a fully offline voice notes app.

May 2Saturday

Hacker News front page

Spotify Adds 'Verified' Badges to Distinguish Human Artists from AI

Spotify added 'Verified' badges for human artists to distinguish them from AI, per the title. The RSS snippet does not disclose the verification process, rollout scope, timing, or review criteria.

Why it matters: Spotify now marks real artists with a green check, using linked social accounts, listener data and tour history to separate people from AI acts.

May 1Friday

QbitAI · WeChat

He Used AI to Run a Music Festival About Not Doing a PhD

Bilibili creator Huntunpi Qiezong made 42 AI-generated “Don’t Do a PhD” songs, passing 50 million views. One track took over 100 generations, racing Suno, MiniMax Music, HeartMuLa, and ACE-Step. The key signal is the human curation cost in AI music workflows.

Why it matters: A Bilibili creator's 42 AI-generated songs against PhD study hit 50 million views, showing that human curation, not generation, is the real cost.

Bloomberg Technology

The Audio Industry Is Grappling with the Rise of ‘Podslop’

Podcast Index says 39% of new podcasts over nine days were likely AI-generated. The title cites industry concern over “podslop”; the post does not disclose detection methods, sample size, or platform split.

Why it matters: Podcast Index scraped nine days of new podcasts and found 39% may be AI-generated, giving a number for how audio supply is shifting.

Apr 30Thursday

Google DeepMind

Google DeepMind announces AI co-clinician medical research program

Google DeepMind announced an AI co-clinician research program, exploring how AI agents can assist patient care under a doctor's clinical supervision. In a blinded evaluation of 98 real primary care queries, the system made no critical errors in 97 cases, and doctors preferred its answers over mainstream evidence synthesis tools. On 140 consultation skills, it matched or beat primary care physicians on 68, but expert physicians were still better overall at spotting red flags and key physical exams.

Why it matters: Google DeepMind details its AI co-clinician research program: in a blinded review of 98 real primary care queries, 97 had no critical errors, and doctors preferred its answers over mainstream evidence synthesis tools.

r/LocalLLaMA

Building a fully local PDF-to-audiobook workflow with Kokoro 82M, Qwen and llama.cpp

Reddit user purellmagents shared a local PDF-to-audiobook workflow using Kokoro 82M, Qwen 3.5 0.8B/2B, and llama.cpp. The Tauri 2.0 app runs on an M1 Mac, reads 15 initial sentences, then prepares the next 15. The hard parts are PDF-text alignment, code snippets, tables, and first-generation latency.

Why it matters: A Reddit user built a fully local PDF-to-audiobook pipeline on an M1 Mac with Kokoro 82M, Qwen and llama.cpp, reading 15 sentences while preparing the next 15 to cut waiting time.

Apr 29Wednesday

Xinzhiyuan · WeChat

Google Translate Turns 20 as Pichai Highlights Four AI Generations

Google Translate turned 20 on April 28, and Pichai said it now has 1B monthly users. The post traces four AI phases: SMT, GNMT, PaLM 2, and Gemini 2.5 Flash Native Audio, including 110 languages added in 2024. The key shift is native speech-to-speech translation that preserves intonation, pacing, and pitch.

Why it matters: Twenty years of Google Translate, traced by Sundar Pichai from statistical machine translation to Gemini 2.5 Flash native audio translation that handles sound directly to keep tone and rhythm.

X · @dotey

Microsoft VibeVoice-ASR tested on Mac for a one-hour podcast

Simon Willison ran 4-bit VibeVoice-ASR on an M5 Max MacBook Pro and transcribed a one-hour podcast in 8m45s. The 9B MIT-licensed model supports 60-minute audio, 50+ languages, and structured speaker output. Memory is the constraint: prefill peaked at 61.5GB, making 32GB laptops impractical.

Why it matters: Simon Willison ran Microsoft's open-source VibeVoice-ASR on an M5 Max with 128GB of memory: 8 minutes 45 seconds to transcribe a one-hour podcast, with prefill memory peaking at 61.5GB.

Apr 28Tuesday

QbitAI · WeChat

ModelBest Releases MiniCPM-o 4.5 Technical Report for Consumer-GPU Deployment

ModelBest, OpenBMB, Tsinghua THUNLP and THUMAI released the MiniCPM-o 4.5 technical report, covering a roughly 9B-parameter model. It supports video, audio and text streams; a 12GB RTX 5070 runs full-duplex mode at RTF 0.4. The key mechanism is Omni-Flow: a unified timeline with time-division multiplexing, without external VAD.

Why it matters: ModelBest with Tsinghua and others published the MiniCPM-o 4.5 technical report, a roughly 9B-parameter model using Omni-Flow to time-share audio, video and text on one timeline, running full-duplex on a 12GB consumer GPU.

QbitAI · WeChat

Xiaomi open-sources MiMo-V2.5 series; Pro builds a macOS-like desktop in 4 hours

Xiaomi open-sourced MiMo-V2.5 weights, covering Pro Agent, multimodal base, TTS, and ASR models. MiMo-V2.5-Pro built a 54-app macOS-like desktop in 4 hours without human takeover; it scored 233/233 on SysY with 672 tool calls in 4.3 hours. Key details for practitioners are the 1M context, 100T-token program, and free Agent-framework access.

Why it matters: Xiaomi open-sourced the MiMo-V2.5 series weights, with the Pro version calling 672 tools over 4.3 hours without human help to run a macOS-like desktop with 54 apps, showing open-source agent execution on long-horizon tasks.

Apr 25Saturday

Hacker News front page

Google Flow Music

Google Flow Music launched a web creation entry with six sections: songs, playlists, Spaces, videos, projects, and Turntable. The page says Producer creates full songs with Lyria 3, and AI music videos use Veo. Pricing, regions, model specs, and rights terms are not disclosed.

Why it matters: Google launched the web version of Flow Music, putting Lyria 3 song generation and Veo music videos behind one entry point, with the conversational Producer tool making full tracks with vocals.

Apr 24Friday

Hacker News front page

Refuse to let your doctor record you

Emily M. Bender and Decca Muldowney give 9 reasons to refuse AI medical scribes. The tools record visits and draft chart notes, raising privacy, consent, automation-bias, and speech-recognition disparity risks. The key concern is clinics converting saved time into more visits.

Why it matters: Linguist Emily M. Bender and journalist Decca Muldowney give nine reasons patients should refuse AI scribes in the exam room, noting recordings and transcripts go to third-party firms.

Apr 23Thursday

The Verge · AI

Google Meet will take AI notes for in-person meetings too

Google expanded Gemini notetaking to in-person meetings and added support for Zoom and Microsoft Teams. The post confirms summaries and transcripts; in-person support had previously been limited to Android alpha users. Google also says it works for impromptu meetings outside meeting rooms, which matters because the recorder is no longer confined to native Meet calls.

Why it matters: Google extends Gemini's meeting-notes capability from online video to in-person meetings and adds Zoom and Teams support, widening where meeting-notes tools can be used.

Apr 20Monday

Hacker News front page

Deezer says 44% of songs uploaded to its platform daily are AI-generated

Deezer says 44% of songs uploaded to its platform each day are AI-generated, with the headline disclosing the 44% share. The RSS snippet does not disclose the measurement period, detection method, sample size, or any enforcement policy.

Why it matters: Deezer reports 44% of new songs uploaded daily are AI-generated, yet those tracks draw only 1% to 3% of plays, and 85% of them are flagged as streaming fraud.

Apr 17Friday

X · @dotey

browser-use open-sources video-use, a Claude Code skill that turns raw camera footage into edited videos

browser-use released video-use, a Claude Code skill that turns raw footage into a final.mp4 automatically. It converts footage into ElevenLabs word-level timestamp transcripts, shrinking one asset to about 12KB; the post says feeding frames directly would cost about 45 million tokens. The key detail is the structured editing pipeline: the model mostly reads text, uses timeline images only at uncertain cuts, and runs up to 3 self-check repair passes after rendering.

Why it matters: browser-use open-sourced video-use as a Claude Code skill that takes raw footage and handles filler-word removal, color grading, subtitles and animation overlays to produce a finished cut.

Apr 16Thursday

Google DeepMind

Google DeepMind releases Gemini 3.1 Flash TTS

Google DeepMind released Gemini 3.1 Flash TTS, a text-to-speech model built around controllability and expressiveness. It is in preview on the Gemini API, Google AI Studio, Vertex AI and Google Vids.

Why it matters: Gemini 3.1 Flash TTS opens in preview across the Gemini API, AI Studio, Vertex AI and Google Vids, covering 70+ languages with audio tags for style and pacing.

Apr 14Tuesday

最佳拍档 (BestPartners)

Global GPU shortage worsens: H100 rental prices rose nearly 40% in five months

SemiAnalysis says Nvidia H100 one-year rental pricing rose from $1.70 to $2.35 per GPU-hour between Oct 2025 and Mar 2026, up nearly 40% in five months. The post attributes this to Anthropic-driven demand, multi-agent and media generation workloads, and memory cost spikes, with LPDDR5 and DDR5 contract prices up about 4x and 5x year over year; much new capacity is already prebooked. The key variable is the supply gap, not Blackwell refreshes alone.

Why it matters: SemiAnalysis reports H100 one-year rental prices rose from $1.70 to $2.35 per hour in five months, with spot clusters of 64 GPUs now hard to rent, putting compute tightness into concrete numbers.

Apr 8Wednesday

QbitAI · WeChat

Xiaomi unveils two AI audio frameworks: Any2Speech and Midasheng-audio-generate

Xiaomi's large-model application team introduced Xiaomi Any2Speech and Midasheng-audio-generate. Any2Speech generates up to about 10 minutes per inference, while the other model turns one text prompt into mixed audio with speech, music, and ambient sound. The post names GST labeling, dual-path planning with dimension dropout, Flow Matching, and five-field structured labels; benchmark scores, training scale, and commercial terms are not disclosed.

Why it matters: Xiaomi's team published two speech frameworks, one generating roughly 10 minutes of audio per inference, the other mixing voice, music and ambient sound from text.

QbitAI · WeChat

Free open-source 2B Chinese speech model reproduces Mangzhuang Ren with high-speed tonguetwisters

ModelBest, OpenBMB, and Tsinghua University released VoxCPM 2, a 2B open speech model that supports 9 Chinese dialects, 30 foreign languages, and 48kHz audio. The post says generation often finishes within 1 second, recommends reference audio of at least 5 seconds, and supports denoising, LoRA, and full fine-tuning; the key detail is its tokenizer-free diffusion autoregressive continuous representation design.

Why it matters: ModelBest and Tsinghua's OpenBMB released a 2B-parameter speech model that clones a voice from a 5-second reference clip.

Apr 3Friday

X · @OpenAI

ChatGPT is now available in CarPlay

OpenAI is rolling out ChatGPT in CarPlay to iPhone users on iOS 26.4+ where CarPlay is supported. The post confirms voice mode is available in-car, but does not disclose regions, vehicle coverage, or feature limits. The key shift is distribution into the driving interface, not a new model launch.

Why it matters: OpenAI put ChatGPT voice mode into CarPlay, so iPhone users on iOS 26.4 or later can talk to it in the car; the point is the entry point, not a new model.

Mar 24Tuesday

Mistral AI

Mistral AI releases Voxtral TTS, a 4B-parameter speech model

Mistral AI released Voxtral TTS, its first text-to-speech model. It has 4B parameters and supports nine languages: English, French, German, Spanish, Dutch, Portuguese, Italian, Hindi and Arabic. It handles emotional expression and zero-shot cross-lingual voice adaptation.

Why it matters: A 4B open TTS model covering 9 languages, with a head-to-head against ElevenLabs as a reference point.

Feb 14Saturday

MIT Technology Review · AI

ALS stole this musician’s voice. AI let him sing again.

Patrick Darling, 32, returned to the stage on February 11 in London after two years without singing, using an AI voice clone rebuilt from old recordings. The post says speech cloning typically needs about 10 minutes of clean audio; his singing clone was built from noisy phone clips and kitchen recordings, then refined with Eleven Music over about six weeks. The practical signal is access, not sentiment: ElevenLabs offers the tools free to people who lost their voices to ALS and similar conditions, but the post does not disclose model details.

Why it matters: A 32-year-old musician with ALS sang with his band again using an AI voice clone built from noisy phone video and kitchen recordings.

Feb 9Monday

36Kr (direct RSS)

Voice Ask is live: why is Xiaohongshu pushing search-by-question?

Xiaohongshu fully launched Voice Ask on Jan. 27, letting users long-press to speak on the search page and get structured answers distilled from in-app user experience posts. The post says it can handle 3-minute spoken queries, foreign languages, and dialects, but does not disclose the model, ASR stack, latency, or accuracy. The real shift is from 3-4 character keyword search to longer spoken questions, widening search intent capture and scenario coverage.

Why it matters: Xiaohongshu rolled out voice Q&A to all users, and answers to long-pressed voice questions on the search page are limited to real user experience on the platform rather than general encyclopedic results.

36Kr (direct RSS)

Former Baichuan co-founder Jiao Ke bets on AI audio to build AI hosts

Jiao Ke said Laifu Radio now has 15 Chinese AI hosts and 2 English ones, and raised over $10 million across two rounds by H2 2025. He said users average about 30 minutes per day, AI can prepare timely audio in under an hour, and the team treats DTU plus long-memory infra as the key moat. The real bet is not an AI podcast tool but interactive AI hosts that remember user preferences; the post also says it is working with some automakers on in-car personalized AI radio.

Why it matters: After leaving Baichuan, Jiao Ke founded Laifu Radio with 17 AI hosts in different styles that recommend content from listening history and allow interruptions, betting that voice interaction's long context lets AI remember user preferences.

Feb 5Thursday

Mistral AI

Mistral releases Voxtral Transcribe 2 speech-to-text model family

Mistral released Voxtral Transcribe 2, a family of two speech-to-text models: Voxtral Mini Transcribe V2 for batch transcription and Voxtral Realtime for live use.

Why it matters: An Apache 2.0 realtime transcription model with latency configurable under 200ms and support for 13 languages.

Sep 30, 2025Tuesday

OpenAI News

Sora 2 System Card

OpenAI published the Sora 2 System Card on September 30, 2025, and said the video-audio generation model will launch first via limited invites on sora.com and a standalone iOS app. The post confirms no video uploads and no image uploads with photorealistic people at launch; API timing, pricing, and benchmark scores are not disclosed.

Why it matters: The Sora 2 system card details gains in physics, realism and audio sync, plus the invite-only launch and the initial ban on uploading real people.

OpenAI News

Sora 2 is here

OpenAI released Sora 2 on September 30, 2025 and launched a social iOS app called Sora built on the model. The post says it generates video with synced dialogue and sound effects, and its “characters” feature uses a one-time video and audio recording to verify identity and insert a real person’s likeness; pricing, generation limits, and rollout regions are not disclosed. The key shift is from model demo to a consumer app with a feed, teen limits, and parental controls.

Why it matters: OpenAI shipped Sora 2 and a matching iOS social app the same day, with synced dialogue and sound, and verified users can put their own likeness and voice into scenes.

Aug 28, 2025Thursday

OpenAI News

Introducing gpt-realtime and Realtime API updates for production voice agents

OpenAI released the speech-to-speech model gpt-realtime and made the Realtime API generally available, adding remote MCP server support, image input, and SIP phone calling. The post reports 82.8% on Big Bench Audio versus 65.6% for the December 2024 model, and 30.5% on the audio MultiChallenge benchmark versus 20.6%. The key change is that tool access and phone connectivity now ship in the same production API.

Why it matters: Speech in, speech out without a text step is the change that matters for anyone building voice agents on the Realtime API.

Jul 17, 2025Thursday

Mistral AI

Mistral adds Deep Research, voice mode and more to Le Chat

Mistral rolled out a batch of new Le Chat features: a preview Deep Research mode, a voice mode powered by the new Voxtral speech model, a multilingual thinking mode backed by the Magistral reasoning model, Projects for organizing conversations, and advanced image editing built with Black Forest Labs.

Why it matters: Le Chat ships five features at once, including Deep Research, voice, multilingual reasoning, Projects and image editing, with Deep Research driven by a tool-augmented agent that cites sources.