Skip to content

All news

1 today

May 12Tuesday

AI HOT (Curated Pool)

How Open Model Ecosystems Compound

China’s open AI model community forms a self-reinforcing loop, with domestic open model downloads rising by more than 200% quarter over quarter.

Why it matters: The author argues trial-and-error compute can be 80% of frontier model costs, and Chinese labs share what failed in technical reports, so an open ecosystem compounds cheaper future R&D.

Synced · WeChat

ByteDance Open-Sources DreamLite for Offline Mobile Image Generation and Editing

ByteDance open-sourced DreamLite, a 0.39B-parameter unified diffusion model that generates or edits a 1024×1024 image on an iPhone 17 Pro in about 3 seconds, using 4-step DMD2 distillation and on-device offline inference without cloud dependency.

Why it matters: ByteDance open-sourced DreamLite, a 390M-parameter unified diffusion model that generates or edits a 1024×1024 image locally on an iPhone 17 Pro in about 3 seconds, using DMD2 distillation to cut inference to 4 steps.

AI HOT (Curated Pool)

Thinking Machines Releases Native Multimodal Interaction Model for Real-Time Human-AI Collaboration

Thinking Machines released an interaction model that natively receives audio, video, and text input, processes foreground interaction at 200-millisecond intervals, and uses a background reasoning model for long-horizon planning and tool calls.

Why it matters: Thinking Machines splits real-time multimodal work into a fast 200ms front end and a slower planning back end, an architecture pattern worth studying.

r/LocalLLaMA

I catalogued every way local models break JSON output and built a repair library across 288 model calls

Reddit user kexxty ran 288 structured-output calls through OpenRouter models, including Llama 3, Mistral, Command R, DeepSeek, and Qwen, and found similar JSON failure categories across local and API-only models. The MIT-licensed Python library outputguard validates against JSON Schema, applies 15 ordered repair strategies, includes 2,001 tests, and has no LLM provider dependency.

Why it matters: Reddit user kexxty ran 288 structured output tests across Llama 3, Mistral, DeepSeek and others, categorized how local models break JSON, and open-sourced a repair library, outputguard.

May 11Monday

AI HOT (Curated Pool)

Fields Medalist Tests ChatGPT 5.5 Pro: Paper-Level Result in 17 Minutes

Timothy Gowers tested ChatGPT 5.5 Pro and said it independently solved an open additive number theory problem in 17 minutes with only a simple prompt, producing PhD thesis-level work; he warned that this pace threatens mathematics research training, while Terence Tao said human value lies in digesting and deeply understanding proofs.

Why it matters: Fields medalist Timothy Gowers gave only a simple prompt, and ChatGPT 5.5 Pro solved an open additive number theory problem on its own in 17 minutes, with a construction that is optimal in theory.

AI HOT (Curated Pool)

Pareto Code Reorders Model Selection Using Market Demand

OpenRouter says Pareto Code observes the Pareto frontier using real market demand; DeepSeek V4 Pro ranks first, followed by GPT 5.4 Mini and Gemini 3.1 Pro, while the post does not disclose the scoring formula or evaluation sample size.

Why it matters: OpenRouter launched the Pareto Code ranking, which measures model value for money from real user calls instead of benchmark scores, with DeepSeek V4 Pro first and GPT 5.4 Mini and Gemini 3.1 Pro close behind.

AI HOT (Curated Pool)

AntLingAGI Releases Trillion-Parameter Ring-2.6-1T Model

AntLingAGI released Ring-2.6-1T, a trillion-parameter thinking model available for free on OpenRouter until May 15, with adjustable thinking intensity, agent-oriented multi-step execution, tool calling, and tasks covering math logic and scientific research.

Why it matters: AntLingAGI released Ring-2.6-1T, a trillion-parameter model with manually adjustable thinking effort, letting users trade answer quality against token cost and latency, with multi-step execution and tool calls tuned for agent work.

Xinzhiyuan · WeChat

The Second Half of Agent Evaluation: Why a Live Benchmark Is Needed

Claw-Eval-Live evaluates 13 frontier models on 105 tasks, and the top model stays below a 70% pass rate, while HR tasks average only 6.8% pass rate.

Why it matters: Claw-Eval-Live ran 105 tasks across 13 frontier models; the best pass rate stayed under 70% and HR tasks averaged 6.8%, showing static benchmarks miss how agents actually perform in deployment.

Xinzhiyuan · WeChat

Claude Mythos Hits 50% Success on 16-Hour Tasks in METR Time Horizons

Claude Mythos Preview reached a 50% success rate on METR Time Horizons tasks that take humans 16 hours, while only 5 of 228 tasks exceeded the 16-hour range, so the article says METR lacks enough samples to quantify longer-horizon performance.

Why it matters: On METR's Time Horizons test, Claude Mythos preview hit 50% on tasks humans take 16 hours, but only 5 of 228 tasks run that long.

AI HOT (Curated Pool)

Local models handle half of daily tasks and respond faster than cloud models

A five-week experiment tested about 1,400 daily work tasks, where local 35B models such as Qwen 3.6 35B handled about 50% and averaged 2.8-second responses, 2.1 times faster than Claude Opus 4.5, while the cloud model still led complex reasoning by about 20%.

Why it matters: A five-week test of a local 35B model across about 1,400 daily requests completed half the tasks at 2.8 seconds average, 2.1x faster than cloud Claude Opus 4.5 at 5.8 seconds.

r/LocalLLaMA

MTP benchmark results: task type determines speculative inference speedups or slowdowns

A Reddit LocalLLaMA user ran 300+ tests on Qwen 3.6 27B MTP quants, finding coding draft acceptance at 79-89% and F16 coding speed up 171%, while Q4_K_M creative writing slowed down 9%.

Why it matters: More than 300 runs on a quantized Qwen 3.6 27B show speculative decoding's speedup depends on the task, not the setup.

May 9Saturday

AI HOT (Curated Pool)

Redis founder uses a C inference engine to run a large model on a personal computer

Antirez open-sourced ds4, a native inference engine for DeepSeek V4 Flash that uses a few thousand lines of C to run a 1M-context model on a 128GB MacBook Pro at a reported 27 tok/s.

Why it matters: Antirez's ds4 runs DeepSeek V4 Flash's million-token context on a 128GB MacBook Pro in a few thousand lines of C, with 2-bit MoE quantization and KV cache spilled to SSD.

r/LocalLLaMA

80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP

Reddit user janvitos ran Qwen3.6-35B-A3B-MTP-GGUF with a llama.cpp MTP PR on an RTX 4070 Super. The posted benchmark shows 69.2-81.9 tok/s, 0.694-0.947 draft acceptance, 131072 context, and a -fitt 1536 setting that reserves 1536 MB for the draft model and KV cache.

Why it matters: Reddit user janvitos runs Qwen3.6 35B A3B on a 12GB RTX 4070 Super and posts measured settings for 69.2 to 81.9 tok/s at 128K context.

AI HOT (Curated Pool)

Baidu releases ERNIE 5.1 with compressed parameters and training cost

Baidu released ERNIE 5.1 with total parameters reduced to about one third of the original scale, active parameters to about one half, and pretraining cost to about 6% of same-scale models; the model is available on the ERNIE platform and Baidu AI Studio.

Why it matters: Baidu's ERNIE 5.1 cuts total parameters to about a third and pretraining cost to 6% of comparable models, with head-to-head numbers against DeepSeek-V4-Pro and Gemini 3.1 Pro.

AI HOT (Curated Pool)

ERNIE 5.1 Released With Pretraining Cost at 6% of Comparable Models

Baidu released ERNIE 5.1, saying it builds on ERNIE 5.0 pretraining and improves search, reasoning, knowledge QA, creative writing, and agent capabilities, with pretraining cost at about 6% of comparable models.

Why it matters: Baidu built ERNIE 5.1 on 5.0, upgrading search, reasoning, knowledge Q&A, creative writing and agent abilities, at 6% of the pretraining cost of comparable models.

Xinzhiyuan · WeChat

CUHK Open-Sources ArbiterOS Agent Governance Kernel With 92.95% High-Risk Interception

CUHK CURE Lab open-sourced ArbiterOS, an agent runtime governance kernel that intercepts, parses, governs, and observes actions before execution, raising high-risk step interception on OpenClaw tasks from 6.17% to 92.95%.

Why it matters: CUHK's CURE lab open-sourced ArbiterOS, a runtime governance layer between model output and system execution, lifting the block rate on high-risk OpenClaw steps from 6.17% to 92.95%.

Synced · WeChat

StarVLA Open-Sources a Unified VLA Framework from HKUST and the Community

HKUST and the open-source community released StarVLA, a unified Vision-Language-Action framework that integrates backbones, action heads, training strategies, and evaluation interfaces; the repository has 2.2k GitHub stars and supports benchmarks including LIBERO, SimplerEnv, RoboTwin 2.0, RoboCasa-GR1, and BEHAVIOR-1K.

Why it matters: HKUST and the community open-sourced StarVLA, making VLA backbones, action heads and training interfaces pluggable instead of rebuilt per paper.

AI HOT (Curated Pool)

Claude Mythos Evaluation Shows 16-Hour Risk Horizon

METR evaluated an early Claude Mythos Preview build during a limited March 2026 window and estimated its 50% time horizon at at least 16 hours, with a 95% confidence interval of 8.5 to 55 hours.

Why it matters: METR's risk assessment of an early Claude Mythos preview measured a 16-hour lower bound on tasks the model can finish alone.

May 8Friday

r/LocalLLaMA

Gemma 4 26B Hits 600 Tok/s on One RTX 5090

chain-77 benchmarked Gemma 4 26B with vLLM 0.19.2rc1, and DFlash raised output throughput on one RTX 5090 from 228 tok/s to 578 tok/s under 256 input tokens, 1024 output tokens, concurrency 1, and num_speculative_tokens=13.

Why it matters: chain-77 ran Gemma 4 26B on a single RTX 5090 with vLLM 0.19.2rc1, and DFlash speculative decoding lifted output throughput from 228 to 578 tokens per second, with reproducible concurrency and draft-token settings.

r/LocalLLaMA

11.67% ARC-AGI-2 Local Eval on a Single 4090: The TOPAS Recursive Architecture

Doug_Bitterbot says TOPAS scored 11.67% on ARC-AGI-2 using one RTX 4090 after about 14 days of training. The 100M-parameter checkpoint hit 36% locally, but recursive TTT caused null outputs on nearly half of Kaggle puzzles. The key detail is time management: the author expects 20% after threshold tuning and 3-5 more weeks of training.

Why it matters: About 14 days of training on one RTX 4090 got the 100M-parameter TOPAS to 11.67% on ARC-AGI-2, with local runs reaching 36%, while test-time training recursion left nearly half the Kaggle tasks with empty output.

May 7Thursday

AI HOT (Curated Pool)

SenseNova-U1 Open-Sources 8-Step Distilled LoRA, Speeds Diffusion Inference by 11x

SenseNova-U1 open-sourced an 8-step distilled LoRA that cuts diffusion generation from 100 steps to 8. GPU inference time drops from 23 seconds to 2 seconds, with ComfyUI workflows for text-to-image, image editing, and interleaved generation. The key signal is distillation for latency, not parameter scale.

Why it matters: An open 8-step LoRA cuts diffusion sampling from 100 steps to 8, dropping GPU inference from 23 seconds to 2, with a ComfyUI workflow included.

OpenAI News

Advancing Voice Intelligence with New Models in the API

OpenAI introduced new realtime voice models in its API for voice intelligence. The RSS snippet says they reason, translate, and transcribe speech; the post does not disclose counts, pricing, or limits.

Why it matters: OpenAI ships three realtime voice models in the API, with GPT-Realtime-2 calling multiple tools while it reasons and GPT-Realtime-Translate handling 70-plus input languages into 13 output languages.

Synced · WeChat

Claude, GPT and Gemini score 0% completion on ProgramBench

ProgramBench tested Claude Opus 4.7, GPT-5.4 and Gemini 3.1 Pro, with 0% full completion on rebuilding software projects. It gives only executables and usage docs, removes source/tests, and grades behavioral equivalence via agent-driven fuzzing. The key signal is system-level engineering, not function-level code generation.

Why it matters: The author of SWE-Bench launched ProgramBench, a new benchmark that asks models to rebuild complete software projects from scratch using only executables and docs; Claude, GPT and Gemini all scored zero.

r/LocalLLaMA

Exaggerated PCI-E Bandwidth Concerns?

Reddit user ziphnor tested 2x RTX 5060 Ti 16GB with vLLM TP=2 and 32k-context prefill. PCIe peaked at 3–4 GB/s, about 40–50% of a PCIe 4.0 x4 link. Prefill reached ~840–850, 1500, and 1600–1700 t/s; the post does not disclose decode bandwidth.

Why it matters: User ziphnor ran vLLM tensor parallelism on two RTX 5060 Ti cards and measured PCIe peak bandwidth of only 3-4 GB/s during 32k-context prefill, 40-50% of the PCIe 4.0 x4 link.

r/LocalLLaMA

Analysis of 922 Agentic Task Traces Finds DeepSeek v4’s Cost Edge in Caching

A Reddit user analyzed 922 agentic task traces and reported $0.01 per task for DeepSeek v4 Flash versus $1.52 for Opus 4.7. Both used about 960K tokens per task, but DeepSeek showed a 97% cache hit rate versus 87%, with a 0.02 cache read/write price ratio versus 0.08. The key issue is caching, not headline pricing.

Why it matters: A Reddit user analyzed 922 agent task traces and compared per-task cost for DeepSeek v4 Flash and Opus 4.7, finding similar token use and the gap coming from cache hit rate and cache read-write pricing.

May 6Wednesday

r/LocalLLaMA

Qwen3.6 27B NVFP4 + MTP on a Single RTX 5090: 200k Context in vLLM

A Reddit user ran Qwen3.6 27B NVFP4 on one RTX 5090 32GB and validated 200k context in vLLM. The setup used fp8_e4m3 KV cache, FlashInfer, and MTP with 3 speculative tokens; a 10-run 200k pass completed with 73.6 tok/s mean generation and 70.2s TTFT. The key constraint is 32GB VRAM: logs showed 8.3GiB KV cache and about 30478MiB total GPU use.

Why it matters: A Reddit user ran Qwen3.6 27B NVFP4 on a single RTX 5090 in vLLM, verified 200k context, and posted the fp8_e4m3 KV cache and MTP speculative decoding config.

r/LocalLLaMA

An Open Benchmark for Testing RAG on Realistic Company-Internal Data

EnterpriseRAG-Bench released a 500k-document corpus for testing RAG on company-internal data. It simulates Redwood Inference across 9 sources and includes 500 questions over 10 retrieval failure modes. Baselines show BM25 beats vector search overall, while agentic/bash retrieval has the best completeness at higher cost and latency.

Why it matters: EnterpriseRAG-Bench tests internal-company RAG with 500k documents and 500 questions across 9 data sources and 10 retrieval failure modes; BM25 beat vector retrieval overall.

Synced · WeChat

Alibaba open-sources PromptEcho for T2I rewards using frozen VLMs

Alibaba open-sourced PromptEcho, which uses one frozen Qwen3-VL-32B forward pass to score T2I training rewards. It computes token-level cross-entropy for the original prompt under teacher forcing, then uses the negative value as a continuous reward. In 5,000 poster tests, text accuracy rose from 68% to 75%.

Why it matters: Alibaba open-sources PromptEcho, which uses a frozen Qwen3-VL-32B to generate continuous reward signals without training a separate scorer, lifting text accuracy from 68% to 75% on 5,000 posters.

Synced · WeChat

Two Chinese open-source projects turn Mac into a private AI workstation

Mininglamp open-sourced Cider and Mano-P 1.0 for Apple Silicon local inference and GUI agents. Cider speeds Qwen3-VL-2B prefill by 57%–61% on M5 Pro; Mano-P 1.0-72B scores 58.2% on OSWorld. The key constraint is W8A8 memory: on 16GB devices accuracy falls from 58.0% to 54.0%, so 32GB+ is recommended.

Why it matters: Minglue open-sources Cider and Mano-P 1.0, with Cider cutting Qwen3-VL-2B first-token latency about 60% on M5 Pro and the 72B Mano-P scoring 58.2 on OSWorld.

May 5Tuesday

r/LocalLLaMA

ProgramBench: Can We Really Rebuild Huge Binaries from Scratch?

ProgramBench released 200 tasks for agents rebuilding programs from target executables and usage files. The team spent about $50k generating 6M lines of black-box behavioral tests, with no internet or decompilation. GitHub, Hugging Face, and Docker images are open-sourced, with pip-based evaluation available.

Why it matters: ProgramBench gives agents 200 tasks to rebuild programs from nothing but a binary and its docs, with no network or decompiling, and the results show agents mostly cannot do it.

r/LocalLLaMA

SenseNova-U1-8B-MoT open-source multimodal architecture draws LocalLLaMA discussion

SenseNova open-sourced SenseNova-U1-8B-MoT, an 8B native multimodal understanding and image-generation model. Its Hugging Face text says NEO-Unify removes VE and VAE, supports interleaved image-text generation, and high-density rendering; the post does not disclose test scores. The key question is whether the monolithic design yields reproducible gains.

Why it matters: SenseTime's open 8B multimodal model drops the vision encoder and VAE, putting image understanding and generation in one model, a different trade-off in multimodal architecture.

r/LocalLLaMA

Heretic 1.3 Released: Reproducible Models, Integrated Benchmarks, Lower Peak VRAM

Heretic 1.3 adds reproducible runs, integrated benchmarks, lower peak VRAM, and broader model support. The project claims 20,000 GitHub stars and 13 million model downloads. Reproduce directories capture PyTorch, GPU, driver, and accelerator details; benchmarks use lm-evaluation-harness for MMLU, EQ-Bench, GSM8K, and HellaSwag. The post names Qwen3.5 and Gemma 4 support, but does not disclose VRAM reduction figures.

Why it matters: Heretic 1.3 records the PyTorch, GPU and driver environment so others can reproduce results, and adds built-in benchmarking with lower peak VRAM use.

r/LocalLLaMA

Prompt injection benchmark: delimiter and strict prompt took Gemma 4 from 21% to 100% defense rate

A Reddit user posted a prompt-injection benchmark covering 15 models, 7 attack types, and 6,100+ cases. The setup wraps untrusted documents in long random delimiters; Gemma 4 E4B rose from 21.6% to 100% defense. The key detail is the reproducible metric: blocked/(blocked+failed).

Why it matters: A Reddit user ran over 6,100 prompt injection tests across 15 models and 7 attack styles, and a long random delimiter plus strict instructions lifted Gemma 4 E4B's defense rate from 21.6% to 100%.

r/LocalLLaMA

DeepSeek V4 Pro matches GPT-5.2 on FoodTruck Bench, 10 weeks later and about 17x cheaper

DeepSeek V4 Pro ranked No. 4 on FoodTruck Bench. The 30-day agentic benchmark uses 34 tools, persistent memory, and daily reflection; its median is within 3% of GPT-5.2 at about 17x lower workload cost. Xiaomi MiMo v2.5 Pro also ranked No. 6, with 5/5 survival, 1,019% median ROI, and $2.41 per run.

Why it matters: DeepSeek V4 Pro placed fourth in the 30-day FoodTruck Bench agent evaluation, with a median score within 3% of GPT-5.2 and a per-turn cost about one seventeenth of it.

r/LocalLLaMA

Benching Local Qwen as a Codex Validator, Co-agent, and Challenger

robert896r1 tested Qwen3.6 27B GGUF beside Codex as a coding validator and released a reproducible eval suite. The runs covered Bartowski, Unsloth, 65k/128k context, and q8/f16 KV cache; three 128k profiles tied for best, with no measured q8 KV accuracy loss in this suite. The useful signal is the sidecar eval: missed directives, overbuilding, UI judgment, and long-context misses, not a universal leaderboard.

Why it matters: The author runs a GGUF-quantized Qwen3.6 27B beside Codex as a code validator and published a reproducible benchmark covering two quantization sources, 65k and 128k context, and q8 and f16 KV cache precision.

May 4Monday

r/LocalLLaMA

M3 Ultra + DGX Spark = M5 Ultra-lite?

A Reddit user benchmarked DGX Spark against M3 Ultra in llama.cpp at pp16384, with Spark 1.4× to 3.4× faster across 4 models. Qwen 27B hit 778 t/s vs 340 t/s, while Mistral 128B hit 241 t/s vs 72 t/s. The concrete tuning note is mmap=0: loading fell from minutes to about 20 seconds.

Why it matters: A Reddit user benchmarked DGX Spark against M3 Ultra at pp16384 context in llama.cpp, with Spark 1.4x to 3.4x faster on Qwen 27B and Mistral 128B.

r/LocalLLaMA

Mistral Medium 3.5 128B and Qwen 3.5 122B A10B on 4x RTX 3080 20GB

A Reddit user benchmarked Mistral Medium 3.5 128B and Qwen 3.5 122B A10B on 4x RTX 3080 20GB. llama.cpp tensor split raised Mistral tg128 from 10.37 to 21.59 t/s, but Qwen MoE fell from 60.08 to 53.49 t/s. vLLM served Qwen GPTQ-Int4 at 187.04 tok/s; the key signal is MoE sensitivity to parallel strategy.

Why it matters: A Reddit user ran two large models on 4 RTX 3080 20GB cards, and Mistral Medium 3.5 128B hit 21.59 tokens per second with tensor split, up from 10.37.

r/LocalLLaMA

450M On-Board VLM Wildfire Detection Pipeline with Sentinel-2 and LFM2.5-VL

PauLabartaBajo shared a wildfire detection PoC using 450M LFM2.5-VL on Sentinel-2 imagery. It pairs RGB and SWIR tiles, simulates orbit with SimSat, and covers 22 fire-prone sites. The key constraint is bandwidth: on-board inference downlinks only a JSON risk profile.

Why it matters: The author deployed a 450M vision-language model on the satellite itself for wildfire inference and downlinks only JSON risk assessments, sidestepping bandwidth limits on multispectral imagery.

May 3Sunday

r/LocalLLaMA

Local LLM Benchmark for Backend Generation via Function Calling: GLM vs Qwen vs DeepSeek

AutoBe posted a controlled backend-generation benchmark and says qwen3.5-35b-a3b matches gpt-5.4 on DB/API design. One shopping-mall run uses 200–300M tokens, costing $1,000–$1,500 per model at GPT 5.5 pricing. The key caveat is n=4 projects and self-scoring harness bias.

Why it matters: AutoBe tests database and API design through function calling, and qwen3.5-35b-a3b matches gpt-5.4 on 4 projects, with one mall project burning 200 to 300 million tokens.

r/LocalLLaMA

LLM proxy that lets Claude Code talk to any model

DataNebula released open-source rosetta-llm, letting Claude Code call multiple providers through one gateway. It translates Anthropic Messages, OpenAI Chat, and OpenAI Responses, and round-trips encrypted reasoning via the signature field. The key detail is thinking-block fidelity for multi-turn agent prompt-cache hits.

Why it matters: DataNebula's open-source rosetta-llm acts as a translation gateway so Claude Code can call models from other vendors, one route to mixing models.