Skip to content

All news

16 today

May 5Tuesday

r/LocalLLaMA

vibevoice.cpp: Microsoft VibeVoice ported to ggml/C++ with no Python at inference

LocalAI released vibevoice.cpp, a ggml/C++ port of Microsoft VibeVoice for CPU, CUDA, Metal, and Vulkan inference. TTS uses a 30s reference clip for 24kHz cloned speech; ASR uses a 7B model with diarized JSON and was tested on 17min audio. The key constraint is memory: 17min CPU Q8_0 peaks near 26GB, with no streaming output yet.

Why it matters: LocalAI rewrote Microsoft VibeVoice's speech synthesis and long-audio transcription as a pure C++ version that runs on CPU, CUDA, Metal and Vulkan, so local deployment no longer needs a Python environment.

QbitAI · WeChat

Doubao Tests Paid Subscriptions, With Top Tier at 500 Yuan per Month

Doubao listed three App Store subscription tiers at 68, 200, and 500 yuan per month, while keeping a free basic version. QbitAI says the paywall is not live, and ByteDance has only confirmed full details will come through official channels. Doubao app DAU passed 140 million in April, and model calls exceeded 120 trillion tokens per day by March 2026.

Why it matters: Doubao lists three paid monthly tiers on the App Store while keeping a free basic version, a sign ByteDance is testing subscription revenue for its chatbot.

r/LocalLLaMA

MTPLX: 2.24x Faster TPS Native MTP Inference Engine for Apple Silicon

MTPLX raises Qwen3.6-27B on a MacBook Pro M5 Max from 28 to 63 tok/s. The test used 4-bit MLX, temperature 0.6, top_p 0.95, top_k 20, with D3 as the best depth. The key detail is native MTP heads: no external drafter and no second-model memory.

Why it matters: MTPLX on a MacBook Pro M5 Max raised Qwen3.6-27B generation speed from 28 to 63 tokens per second, a 2.24x gain from built-in MTP heads with no separate drafter model to load.

OpenAI News

New Ways to Buy ChatGPT Ads

OpenAI expanded ChatGPT ad buying with a beta self-serve Ads Manager, CPC bidding, and enhanced measurement tools. The post says ads protect privacy and keep chats separate; it does not disclose pricing, rollout scope, or timing.

Why it matters: OpenAI is testing a self-serve ChatGPT ad console in the US, letting advertisers sign up, fund accounts and manage campaigns, with cost-per-click added alongside CPM bidding.

r/LocalLLaMA

FastDMS: 6.4X KV-cache compression running faster than vLLM BF16/FP8

FastDMS released an MIT implementation that cuts KV memory to 1/5–1/8 of vLLM BF16 at 8K context. A Llama-3.2-1B replication reports PPL 9.200 with 6.4x compression; Qwen3-8B c=1 drops KV from 1.406 GiB to 0.184 GiB. The key detail is physical reclamation of evicted slots, not just nominal KV-byte reduction.

Why it matters: FastDMS open-sourced an MIT-licensed KV cache compression implementation that cuts Llama-3.2-1B's cache to one-fifth to one-eighth at 8K context, with concrete perplexity and memory figures.

May 4Monday

r/LocalLLaMA

Deep research report with Hermes Agent and qwen3.6-35b-a3b Q6_K

A Reddit user used Hermes Agent and qwen3.6-35b-a3b Q6_K to produce a 21-page research report. The run took 6 loops and over 5 hours on an RTX 4060, at about 28 tokens/s. The repo includes prompts, scripts, intermediate artifacts, and the final report.

Why it matters: A Reddit user ran Hermes Agent with a quantized qwen3.6-35b on one RTX 4060 through six loops to produce a 21-page report, with prompts and scripts in the repo.

QbitAI · WeChat

DeepSeek-TUI, a “DeepSeek Claude Code,” reaches 2.3k GitHub stars

DeepSeek-TUI reached 2.3k GitHub stars; the Rust project is MIT-licensed. It targets DeepSeek V4 with a 1M-token context, RLM up to 16 V4 Flash subtasks, MCP, Shell, Git, and three control modes. Watch cache misses: uncached tokens cost 10x cached tokens.

Why it matters: DeepSeek-TUI is a Rust terminal coding assistant wired to DeepSeek V4 with 1M-token context, MIT-licensed, for developers studying command-line AI coding tools.

r/LocalLLaMA

Gemma 4 E2B runs well on an 8GB Android phone, powering a private voice notes app

A Reddit user ran Gemma 4 E2B locally on an 8GB OnePlus CE 5 and built a private voice notes app. Whisper Small 244MB transcribes, Gemma 4 E2B 2.4GB splits and tags, and a 10-15s note takes 12-15s end to end. Search uses query expansion, FTS lanes, RRF, and optional Gemma top-K reranking with a 15s fallback.

Why it matters: A Reddit user fit a 2.4GB Gemma 4 E2B and Whisper Small into an 8GB phone for a fully offline voice notes app.

r/LocalLLaMA

Could PC x64 Instruction Extensions Relieve Hardware Shortage?

Intel and AMD unveiled ACE, an x86 extension claiming 1,024 multiplications per clock. It uses 2D tile registers and outer-product algorithms, versus 64 multiplications for AVX. No ACE hardware is released; power, framework support, and shipping timelines are not disclosed.

Why it matters: Intel and AMD's ACE extensions use 2D tile registers and outer-product math to lift multiplies per clock from AVX's 64 to 1024, but no supporting hardware has shipped.

May 3Sunday

r/LocalLLaMA

Upskill: skill registry your agent consults before it starts, with 10k+ indexed skills

Autoloops released Upskill, an open-source skill registry with 10k+ indexed skills for agents. Search combines Postgres full-text search, 1024-dim embeddings, and reranking by stars, installs, and feedback. LLM adversarial review blocked hundreds of skills at index time.

Why it matters: Autoloops open-sourced the Upskill registry, indexing over 10,000 skills an agent can search before starting a task, ranked by full-text, vector match and install feedback.

QbitAI · WeChat

GS-Playground Embodied AI Simulation Framework Open-Sourced with High-Throughput 3DGS Rendering

Tsinghua AIR DISCOVER Lab and partners open-sourced GS-Playground, accepted by RSS 2026. On an RTX 4090, it reports 10,000 FPS at 640×480 and 2,048 parallel scenes; a 50-humanoid benchmark reaches 1,015 FPS. The key point is coupling batch 3DGS rendering with parallel physics.

Why it matters: A single RTX 4090 hitting 10,000 fps while simulating 2,048 scenes at once sets a new bar for robot-learning environments.

r/LocalLLaMA

Built a C++17 transformer from scratch with 0.83M params and CPU training

Reddit user Suspicious_Gap1121 released Quadtrix.cpp, a C++17 GPT-style model with 0.83M parameters. It uses 4 layers, 4 heads, 200d width, and a 128-character context; one CPU core trained on 31.4M characters for 76.2 minutes to 1.6371 nats val loss. The key detail is handwritten backprop for LayerNorm, attention, Q/K/V, dropout, and AdamW without PyTorch, BLAS, or autograd.

Why it matters: A from-scratch C++17 transformer with hand-written backprop shows what a small GPT actually costs to train without PyTorch or BLAS.

May 2Saturday

r/LocalLLaMA

I built Semvec: A constant-cost semantic memory for LLMs, looking for testers

A developer released Semvec, replacing unbounded chat history with fixed-size semantic state. Its 48-turn benchmark claims about 76% token reduction, with identical input footprint at turn 10 and 10,000. It supports OpenAI-compatible LLMs, MCP, Claude Code, Cursor, and multi-agent shared state.

Why it matters: Swapping chat history for fixed-size semantic vectors cuts token use about 76% and keeps input cost flat as a conversation grows.

Hacker News front page

Show HN: Filling PDF Forms with AI Using Client-Side Tool Calling

SimplePDF released a Copilot demo that fills PDF forms via client-side tool calling; SimplePDF has 200k+ monthly users. PDFs stay in the browser, with parsing, rendering, and field detection local. The demo uses a DeepSeek V4 Flash proxy by default, with BYOK, cloud, or LM Studio options.

Why it matters: SimplePDF's Copilot runs PDF parsing, rendering and field detection in the browser, so files never leave the machine.

r/LocalLLaMA

Qwen3.6-27B hits 72 tok/s on RTX 3090 with native vLLM on Windows

Reddit user One_Slip1455 released a native Windows vLLM launcher for Qwen3.6-27B, reaching 72 tok/s on an RTX 3090. It reports 64.5 tok/s at ~25k tokens, 53.4 tok/s at 127k ctx on one GPU, and 160k ctx with PP=2 on 2×3090. The key detail is no WSL or Docker, an OpenAI-compatible endpoint, and an INT4 quant path.

Why it matters: A native Windows vLLM launcher runs Qwen3.6-27B on one RTX 3090 without WSL or Docker, hitting 72 tok/s at short context.

Hacker News front page

Spotify Adds 'Verified' Badges to Distinguish Human Artists from AI

Spotify added 'Verified' badges for human artists to distinguish them from AI, per the title. The RSS snippet does not disclose the verification process, rollout scope, timing, or review criteria.

Why it matters: Spotify now marks real artists with a green check, using linked social accounts, listener data and tour history to separate people from AI acts.

May 1Friday

r/LocalLLaMA

PFlash: 10x prefill speedup over llama.cpp at 128K on an RTX 3090

PFlash cuts Qwen3.6-27B Q4_K_M 128K TTFT to 24.8s on an RTX 3090, versus 248.4s cold for llama.cpp. It uses a Qwen3-0.6B drafter to score token importance, keeps 5% of spans, and runs C++/CUDA without Python, Triton, or PyTorch. The quality caveat is clear: only NIAH single-needle passes from 32K to 128K; RULER and multi-needle results are not disclosed.

Why it matters: PFlash uses Qwen3-0.6B to pick the 5% of chunks that matter, cutting first-token latency on a 128K-token RTX 3090 run from 248.4s to 24.8s.

The Verge · AI

Microsoft wants lawyers to trust its new AI agent in Word documents

Microsoft launched Legal Agent in Word for legal teams, focused on tasks such as contract review. It follows legal workflows, reviews clauses against a playbook, and handles tracked changes; the post does not disclose pricing or rollout scope.

Why it matters: Microsoft's Legal Agent for Word reviews contracts clause by clause against a preset rulebook and handles tracked changes, built by the Robin AI team it acquired.

r/LocalLLaMA

16x Spark Cluster Build Update

Reddit user Kurcide finished a 16-node DGX Spark cluster, with all nodes hitting line rate on the fabric. Each node uses one QSFP56 link to an FS N8510, showing 100–111 Gbps per rail and about 200 Gbps aggregate. The key angle is unified memory: 8 nodes served 434GB GLM-5.1-NVFP4, with DeepSeek and Kimi tests next.

Why it matters: A hands-on build showing how 16 DGX Sparks link over QSFP56 at line rate and run a 434GB model across 8 nodes, useful for anyone planning local multi-node inference.

Xinzhiyuan · WeChat

Developer Builds WorldX, an AI World Generator, During a 10-Day Wedding Leave

An independent developer built WorldX in 10 days, generating a full AI world from one sentence in about 5 minutes. The system uses a 6-step map pipeline, about 30k–180k tokens per world, Tick loops, layered memory, and two-axis emotion. The key mechanism is overlay labeling plus color-difference localization for deterministic coordinates.

Why it matters: A developer built WorldX during a 10-day marriage leave; one sentence generates a full AI world in about 5 minutes, and the post details a 6-step map pipeline, token ranges and a deterministic coordinate scheme.

Xinzhiyuan · WeChat

OpenAI upgrades Codex to control Macs and run cross-app tasks

OpenAI upgraded Codex with Slack, Google Workspace, and Microsoft 365 integrations. Mike Russell tested Codex on a Mac across Adobe Audition, Photoshop, and Firefly, finishing in about 8 minutes with an 85–90 score. The key shift is OS-level computer control, not code completion.

Why it matters: OpenAI wired Codex into Slack, Google Workspace and Microsoft 365, letting the assistant act across apps on a Mac, shifting the focus from writing code to doing everyday office tasks.

Latent Space

[AINews] Agents for Everything Else: Codex for Knowledge Work, Claude for Creative Work

OpenAI expanded Codex to non-coding work, with CUA reported 42% faster. The update connects Microsoft, Google, and Salesforce, covering docs, slides, spreadsheets, research, and planning. The key signal is GUI-agent productization, not one benchmark score.

Why it matters: OpenAI pushed Codex beyond code into documents, slides and spreadsheets with Microsoft, Google and Salesforce integrations, while Anthropic added Blender and Adobe creative apps to Claude.

Financial Times · Technology

Huawei’s AI chip sales surge as Nvidia stalls in China

Huawei received large AI processor orders from Chinese tech companies as Nvidia stalls in China. The post does not disclose order value, chip models, or delivery timing. The key issue is China’s domestic compute substitution path, not one sales headline.

Why it matters: Huawei landed large AI processor orders from Chinese tech companies while Nvidia is blocked in China by export controls, showing the shift in the domestic compute supply chain.

Hacker News front page

Show HN: Pu.sh – a full coding-agent harness in 400 lines of shell

Pu.sh ships a coding-agent harness in about 400 lines of shell, using only sh, curl, and awk. It supports Anthropic and OpenAI, 7 tools, REPL, auto-compaction, checkpoint/resume, pipe mode, and 90 no-API tests. It excludes TUI, streaming, images, OAuth, and Windows.

Why it matters: Pu.sh builds a full coding-agent harness in about 400 lines of shell, covering tool calls, context compaction and resumable runs, a minimal readable reference for anyone who wants the core agent loop without heavy dependencies.

NVIDIA Blog

Nemotron Labs: What OpenClaw Agents Mean for Every Organization

NVIDIA says OpenClaw reached 250,000 GitHub stars by March 2026, passing React within 60 days. OpenClaw is Peter Steinberger’s self-hosted persistent agent; NVIDIA introduced NemoClaw with OpenShell sandboxing and Nemotron models. The key issue is governance: the post claims reasoning AI raised token use 100x, and autonomous agents add another 1,000x.

Why it matters: OpenClaw's self-hosted agents hit 250,000 GitHub stars in 60 days, faster than React, prompting NVIDIA to weigh token costs and safety limits for always-on agents.

TechCrunch · AI

After Dissing Anthropic for Limiting Mythos, OpenAI Restricts Access to Cyber, Too

OpenAI will first roll out GPT-5.5 Cyber only to “critical cyber defenders.” The RSS snippet does not disclose eligibility rules, pricing, or launch timing. The access-tiering model is the key detail for practitioners.

Why it matters: OpenAI gave GPT-5.5 Cyber first to critical cyber defenders while ordinary users wait, after Sam Altman criticized Anthropic for limiting Mythos.

TechCrunch · AI

Google’s Gemini AI assistant is hitting the road in millions of vehicles

Google is bringing its Gemini AI assistant to millions of vehicles. The RSS text says it brings more advanced conversational AI into driving. The post does not disclose models, timing, feature scope, or pricing.

Why it matters: Gemini replacing Google Assistant in car dashboards puts a new model at the voice entry point for millions of drivers.

TechCrunch · AI

Stripe introduces Link, a digital wallet autonomous AI agents can use

Stripe introduced Link, a digital wallet for cards, banks, subscriptions, and AI-agent spending. The post cites approval flows, but does not disclose fees, limits, or merchant coverage. Watch the authorization boundary for agent payments.

Why it matters: Stripe's Link lets users authorize AI agents to pay under an approval flow, which is the part worth studying.

The Verge · AI

Meta is running get-rich-quick ads for its AI tools

The Verge says Meta-owned Manus ran quick-money ads for AI tools after a $2B acquisition. The pitch targets local firms with no or bad websites. Manus also paid creators for Instagram, YouTube, and TikTok promotion; some TikTok accounts were removed after inquiry.

Why it matters: An investigation shows how Meta's own ad platform carries get-rich-quick pitches for the AI site builder it acquired.

Apr 30Thursday

MIT Technology Review · AI

Goodfire releases Silico, a mechanistic interpretability tool for debugging LLMs

Goodfire released Silico, letting engineers inspect and adjust LLM parameters during training. It maps neurons and pathways; one Qwen 3 neuron triggered trolley-problem-style outputs. Pricing is case-by-case, and the post does not disclose rates.

Why it matters: Silico lets engineers inspect and adjust model parameters during training and map neurons and circuits.

Ben's Bites

Building Gets Easier

Ben’s Bites lists agent tooling updates from Cloudflare, Stripe, Cursor SDK and others, with over 10 product leads. Cloudflare lets agents create accounts, buy domains, get API tokens and deploy; Stripe adds Agentic Commerce Suite, Link CLI and agent-ready Treasury accounts. The key shift is external permissions becoming agent-readable interfaces.

Why it matters: Cloudflare turned account setup, domain purchase, tokens and deploys into interfaces AI agents can call directly, while Stripe gives agents payment ability, shifting permission boundaries from people to agents.

Bloomberg Technology

Samsung’s Chip Profit Soars 48-Fold Due to AI Spending Spree

Samsung Electronics’ chip unit posted a 48-fold profit jump in the March quarter, driven by AI data-center orders. The RSS snippet says profit hit a record and beat expectations, but the post does not disclose profit value, memory type, or customers.

Why it matters: Samsung's chip division posted a 48-fold year-on-year profit jump, driven directly by AI data center memory buying that is tightening supply.

TechCrunch · AI

Microsoft says it has over 20M paid Copilot users, and they really are using it

Microsoft says Copilot has over 20M paid users, with engagement growing. The post does not disclose active usage, retention, ARPU, or the counting method.

Why it matters: Microsoft disclosed 20 million paid M365 Copilot seats, a rare public adoption figure, but gave no activity, retention or methodology detail.

Bloomberg Technology

Meta Shares Plunge as AI Investments Raise Spending Outlook

Meta raised its 2026 capex outlook to $125B–$145B, and its shares fell after the update. CFO Susan Li cited higher component prices and extra data center costs. The key issue is AI model ROI timing, not one trading day.

Why it matters: Meta raised its 2026 capex guidance to $125B to $145B, which the CFO attributed to higher component prices and data center spending, and the stock fell as investors questioned the payoff timeline.

The Verge · AI

Google Search queries hit an all-time high last quarter

Sundar Pichai said Google Search queries hit an all-time high in Q1 2026, with Search revenue up 19%. He cited AI experiences and Gemini App growth; paid subscriptions topped 350 million, but the post does not disclose query volume.

Why it matters: Alphabet told its earnings call that Google Search queries hit an all-time high, with search ad revenue up 19% year over year, and tied the growth to AI Overviews and the Gemini App.

r/LocalLLaMA

Building a fully local PDF-to-audiobook workflow with Kokoro 82M, Qwen and llama.cpp

Reddit user purellmagents shared a local PDF-to-audiobook workflow using Kokoro 82M, Qwen 3.5 0.8B/2B, and llama.cpp. The Tauri 2.0 app runs on an M1 Mac, reads 15 initial sentences, then prepares the next 15. The hard parts are PDF-text alignment, code snippets, tables, and first-generation latency.

Why it matters: A Reddit user built a fully local PDF-to-audiobook pipeline on an M1 Mac with Kokoro 82M, Qwen and llama.cpp, reading 15 sentences while preparing the next 15 to cut waiting time.

Bloomberg Technology

Meta’s Need for Gas Power Boosts Entergy Spending by $14 Billion

Entergy raised its four-year capital plan by nearly one-third to $57 billion, mainly for Meta’s Louisiana data center. The work covers gas-fired plants; the post discloses a $14 billion increase, not plant capacity or timing.

Why it matters: Meta's data center power demand in Louisiana pushed Entergy to raise its four-year capital plan by nearly a third to $57 billion, including $14 billion for 10 new gas plants.

X · @dotey

Inside Hermes Agent's Memory System and How It Avoids OpenClaw's Pitfalls

Hermes Agent splits memory into 4 layers: prompt files, SQLite session search, skills, and optional Honcho. MEMORY.md is capped at 2,200 chars, USER.md at 1,375; writes apply after a new session or compression. The key design is cache-first: keep system prompts stable and retrieve long-tail history via tools.

Why it matters: Hermes Agent's memory stack has four layers: prompt memory, SQLite session history, skill management and Honcho user modeling, and the piece breaks down how its memory design choices differ from OpenClaw's.

Apr 29Wednesday

r/LocalLLaMA

Mistral Medium 3.5 Launched

Mistral launched Medium 3.5, according to the title. The RSS snippet says it has open weights and a modified MIT license requiring paid licensing for commercial use; the post does not disclose parameter count, benchmarks, or pricing.

Why it matters: Mistral shipped Medium 3.5 with open weights under a modified MIT license: free for non-commercial use, paid for commercial, and that license is the key to how open the release really is.

The Verge · AI

ChatGPT Downloads Are Slowing and May Affect OpenAI's IPO

Sensor Tower says ChatGPT uninstalls rose 132% year over year in April as users left or tried rivals. After OpenAI’s February Pentagon deal, last month’s uninstall rate rose 413%; MAU growth fell from 168% in January to 78% in April.

Why it matters: Sensor Tower data shows ChatGPT uninstalls up 132% year over year in April and monthly active user growth down to 78% from 168% in January, with slowing downloads weighing on OpenAI's IPO prospects.