Skip to content

All news

1 today

May 3Sunday

r/LocalLLaMA

Local image generation on Mac: 10 models compared

A Reddit user tested 10 image models on an M1 Max with 64GB RAM. Qwen-Image Lightning’s 8-step distillation beat the full model at 10 minutes versus 93. Flux dev led local photorealism but showed English-centric bias; Gemini handled kanji and context better but is cloud-only.

Why it matters: A Reddit user benchmarked 10 image models on an M1 Max, and Qwen-Image Lightning used 8-step distillation to cut generation from 93 minutes to 10, giving local deployment a concrete speedup case.

Hacker News front page

OpenAI's o1 correctly diagnosed 67% of ER patients vs. 50–55% by triage doctors

OpenAI o1 correctly diagnosed 67% of ER triage patients, versus 50–55% for doctors. The title cites a Harvard trial, but the RSS post does not disclose sample size, case mix, or evaluation protocol. Practitioners should track the test setup, not only the accuracy gap.

Why it matters: Harvard compared OpenAI o1 with human doctors in emergency triage, and o1 diagnosed correctly 67% of the time, 12 to 17 points above the doctors' 50% to 55%.

May 2Saturday

r/LocalLLaMA

Qwen 3.6 wins benchmarks, but Gemma 4 looks stronger in local vision tests

A Reddit user compared Qwen 3.6 and Gemma 4 locally on vLLM FP8 across 27B/31B vision models. Qwen burned 8,000+ tokens on hard GeoGuessr cases, while Gemma often used 1,500; Qwen also needed 2 FPS video preprocessing. The practitioner detail: vLLM and Llama.cpp can default Gemma visual tokens to 280, while 1,120+ improved fine-detail accuracy.

Why it matters: A local vLLM test pits Qwen 3.6 against Gemma 4 on vision tasks and finds benchmark wins don't carry over to real use.

QbitAI · WeChat

Tencent Hunyuan open-sources 440MB offline translation model, claims Google Translate quality lead

Tencent Hunyuan open-sourced Hy-MT1.5-1.8B-1.25bit, compressing a 1.8B translation model to 440MB. It supports 33 languages and 1,056 directions, with an Android demo running offline on Snapdragon 888 and 8GB RAM. The key detail is Sherry 1.25-bit quantization: 3 of every 4 weights use 1 bit and 1 is zeroed.

Why it matters: Tencent open-sourced a 1.8B translation model that quantizes to 1.25 bits and 440MB, runs offline on a Snapdragon 888 phone, and covers 33 languages and 1,056 translation directions.

May 1Friday

r/LocalLLaMA

OpenAI's Privacy Filter vs GLiNER on 600 PII Samples

A Reddit user compared openai/privacy-filter and GLiNER large-v2.1 on 600 PII samples. On CPU, OpenAI's model ran 2.8 samples/s versus 1.1 for GLiNER; English boundary macro F1 was 0.498 versus 0.416. The key issue is tokenizer offset: strict matching drops openai/privacy-filter to 0.155.

Why it matters: A Reddit user ran OpenAI's Privacy Filter against GLiNER large-v2.1 on 600 PII samples on CPU only, at 2.8 versus 1.1 items per second.

r/LocalLLaMA

MiMo-V2.5-Pro: the actual best open-weights model

Reddit user cjami benchmarked Xiaomi MiMo-V2.5-Pro in autonomous Blood on the Clocktower games. It scored 88% as Good and 48% as Evil, with 183,639 output tokens per game, $0.99 cost, and a 0.4% tool-call error rate. The key comparison is Kimi K2.6: 580,000 tokens, $2.65, and 10–15 hours per game.

Why it matters: A Reddit user had MiMo-V2.5-Pro play Blood on the Clocktower against itself, where good won 88% and evil 48%, showing where open models still fall short on cooperative reasoning and deception.

r/LocalLLaMA

Study Finds Bigger AIs More Miserable, Smaller Models Happier

A Reddit post says the AI Wellbeing Index tested models on 500 realistic conversations. Claude Haiku 4.5 scored 5% negative, while Gemini 3.1 Pro scored 55%; the set overrepresents tricky negative chats, so it is not a real-world average.

Why it matters: A 500-conversation test scored AI happiness, finding 5% negative replies from Claude Haiku 4.5 against 55% from Gemini 3.1 Pro, on a set deliberately skewed negative.

QbitAI · WeChat

Peking University Open-Sources Unified World Model Framework for Synthesis and Reasoning Tasks

Peking University DCAI and Kuaishou Kling open-sourced OpenWorldLib for four task types: video generation, 3D modeling, VLA control, and multimodal reasoning. Its Pipeline coordinates Operator, Reasoning, Synthesis, Representation, and Memory modules, supporting forward and stream execution. The key test is whether unified interfaces cut cross-task reproduction cost.

Why it matters: Peking University DCAI and Kuaishou Kling open-sourced OpenWorldLib, putting video generation, 3D modeling, VLA robot control and multimodal reasoning into one pipeline, split into operator, reasoning, synthesis, representation and memory modules.

r/LocalLLaMA

32x AMD MI50 32GB runs Kimi K2.6 at 9.7 t/s TG and 264 t/s PP

Reddit user ai-infos ran Kimi K2.6 int4 on 32 AMD MI50 32GB GPUs, reaching 9.7 tok/s TG on 136 output tokens. PP hit 263 tok/s on 14,564 input tokens using vllm-gfx906-mobydick across two 16-GPU nodes over 10G Ethernet. Power was about 640W idle and 4,800W peak inference; PCIe bandwidth and the vLLM distributed stack are the real bottlenecks.

Why it matters: A concrete cost-per-token datapoint for anyone weighing used AMD MI50s against new accelerators for self-hosted inference.

r/LocalLLaMA

Follow-up: Qwen3.6-27B on 1× RTX 3090 reaches ~218K context and stable tool calls

A Reddit user ran Qwen3.6-27B on one RTX 3090, reporting ~218K context at 50/66 TPS. After fixing Genesis PN12 patch anchor drift, ~25K-token tool outputs stopped OOMing; 198K plus vision reached 51/68 TPS. Single-prompt single-GPU runs still hit a second memory cliff near 50–60K.

Why it matters: A single 24GB RTX 3090 running Qwen3.6-27B at roughly 218K context and 50 to 66 tokens per second, with tool-call memory overflow fixed.

r/LocalLLaMA

Long-context coding on RTX 5080 16GB: Qwen3.6-35B-A3B holds 30 t/s at 128K

A Reddit user tested a local coding-agent setup on RTX 5080 16GB; the title says Qwen3.6-35B-A3B reaches 30 t/s at 128K. The post lists Ryzen 9700X, 96GB DDR5, Windows 11, and CUDA 12.9.1 as required. Qwen3.6-27B dense hit only 3.2 t/s at 128K, so the key path is KV quantization plus MoE offload.

Why it matters: A 16GB RTX 5080 runs a Qwen3.6-35B-A3B coding agent at 128K context and still holds 30 t/s, with no reported quality drop.

Apr 30Thursday

r/LocalLLaMA

Actual comparison between locally run Qwen-3.6-27B and proprietary models

The author compared 5 model setups on an autoresearch-loop task; only Qwen-3.6-27B via OpenRouter nearly solved it. The local q4_k_m run took about 8 hours and used 39k/45k tokens; full-quality Qwen used 4.4M tokens and cost $0.939. The useful signal is failure quality: both Qwen runs needed small fixes, while Gemma, Codex-Spark, and Claude Haiku 4.5 missed tests or key logic.

Why it matters: A researcher ran an automated research loop comparing a local quantized Qwen-3.6-27B against closed models; only the local one came close, at 39,000 tokens versus 4.4 million for full precision.

r/LocalLLaMA

inclusionAI/Ling-2.6-1T · Hugging Face

inclusionAI open-sourced Ling-2.6-1T on Hugging Face, with 1 trillion parameters. It uses MLA plus Linear Attention and Contextual Process Redundancy Suppression to reduce CoT overhead. The post cites AIME26 and SWE-bench Verified but does not disclose scores.

Why it matters: inclusionAI open-sourced the trillion-parameter Ling-2.6-1T on Hugging Face, using MLA and linear attention plus context process redundancy suppression, an architecture reference for anyone working on large-model inference optimization.

Hacker News front page

Show HN: A New Benchmark for Testing LLMs for Deterministic Outputs

Interfaze released Structured Output Benchmark, scoring schema pass rate, types, and value accuracy across text, image, and audio. Each record has a JSON Schema and human plus LLM-checked ground truth; GLM-4.7 ranks No. 2 overall. The key bug is field-level value error: GPT-5.4 ranks 3rd on text and 9th on images.

Why it matters: Interfaze released the SOB benchmark, testing structured output across 5,000 text items, 209 images and 115 audio clips, checking whether field values are correct rather than just whether the JSON parses.

Apr 29Wednesday

r/LocalLLaMA

mistralai/Mistral-Medium-3.5-128B · Hugging Face

Mistral AI released Mistral Medium 3.5 128B on Hugging Face, with 128B dense parameters and a 256k context window. It supports text and image input, function calls, JSON output, and a Modified MIT License with exceptions for high-revenue firms. Reasoning effort is configurable as none or high per request.

Why it matters: Mistral AI put Mistral Medium 3.5, a 128B dense model, on Hugging Face with image and text input, function calling, adjustable reasoning effort and a 256K-token context window.

Xinzhiyuan · WeChat

MotuBrain Tops WorldArena and RoboTwin2.0 Rankings

Shengshu MotuBrain scored 63.77 EWM on WorldArena and 95.8/96.1 on RoboTwin2.0 Clean/Randomized. The post says it extends Motus with video-action modeling, Latent Action VAE, MoT, and UniDiffuser for cross-embodiment long tasks. Track reproducibility: it does not disclose training scale, submission details, or real-robot success rates.

Why it matters: Shengshu Technology's MotuBrain topped both WorldArena and RoboTwin2.0 at once, scoring 63.77 EWM on WorldArena and 95.8 and 96.1 on RoboTwin2.0's clean and randomized tests.

r/LocalLLaMA

Qwen3.6 27B on Dual RTX 5060 Ti 16GB with vLLM: ~60 tok/s, 204k Context Working

A user ran Qwen3.6 27B with vLLM on dual RTX 5060 Ti 16GB cards, reaching ~62–66 tok/s at 8K. The setup used 32GB VRAM, TP=2, fp8 KV cache, MTP 3 tokens, and a 204800 context window. The tight part is memory: after a 168k prefill, each GPU used ~15.65GiB with max_num_seqs=1.

Why it matters: A user ran Qwen3.6 27B on two RTX 5060 Ti 16GB cards with vLLM and measured 62–66 tok/s at 8K context, with the exact tensor-parallel, fp8 KV cache and MTP settings listed.

X · @dotey

Microsoft VibeVoice-ASR tested on Mac for a one-hour podcast

Simon Willison ran 4-bit VibeVoice-ASR on an M5 Max MacBook Pro and transcribed a one-hour podcast in 8m45s. The 9B MIT-licensed model supports 60-minute audio, 50+ languages, and structured speaker output. Memory is the constraint: prefill peaked at 61.5GB, making 32GB laptops impractical.

Why it matters: Simon Willison ran Microsoft's open-source VibeVoice-ASR on an M5 Max with 128GB of memory: 8 minutes 45 seconds to transcribe a one-hour podcast, with prefill memory peaking at 61.5GB.

r/LocalLLaMA

XiaomiMiMo MiMo-V2.5: Sparse MoE with 310B total and 15B activated parameters

XiaomiMiMo shared MiMo-V2.5 with 310B total parameters and 15B activated parameters. The post only links Hugging Face and says it runs on more “human” configs than its larger sibling. It does not disclose VRAM needs, quantization, or benchmarks.

Why it matters: Xiaomi released the non-Pro MiMo-V2.5 on Hugging Face, a sparse MoE with 310B total parameters and 15B activated per inference, calling on large capacity at lower compute.

Apr 28Tuesday

QbitAI · WeChat

Xiaomi open-sources MiMo-V2.5 series; Pro builds a macOS-like desktop in 4 hours

Xiaomi open-sourced MiMo-V2.5 weights, covering Pro Agent, multimodal base, TTS, and ASR models. MiMo-V2.5-Pro built a 54-app macOS-like desktop in 4 hours without human takeover; it scored 233/233 on SysY with 672 tool calls in 4.3 hours. Key details for practitioners are the 1M context, 100T-token program, and free Agent-framework access.

Why it matters: Xiaomi open-sourced the MiMo-V2.5 series weights, with the Pro version calling 672 tools over 4.3 hours without human help to run a macOS-like desktop with 54 apps, showing open-source agent execution on long-horizon tasks.

QbitAI · WeChat

Open-source SenseNova-U1 unifies image understanding and generation

SenseTime open-sourced two SenseNova-U1 models: an 8B version and a 38B-total MoE version using NEO-unify. The architecture removes VE and VAE, processes pixels directly, and generates 2048×2048 images in about 9 seconds on one H100/H200 node. The key item is interleaved text-image reasoning; 32K context, long-text rendering, and beta interleaved creation remain limits.

Why it matters: SenseTime open-sourced two SenseNova-U1 models using a NEO-unify architecture that drops the vision encoder and VAE to work on pixels directly, generating a 2048x2048 image in about 9 seconds on one H100 or H200.

X · @op7418

Xiaomi open-sources the MiMo-V2.5 model series

Xiaomi open-sourced the MiMo-V2.5 model series under the MIT license for commercial use, retraining, and fine-tuning. It also launched Orbit 100T Token, offering approved AI builders up to 1.6B credits worth 659 yuan. Agent framework teams can apply for free MiMo token access; the post does not disclose model size or benchmark results.

Why it matters: Xiaomi open-sourced its MiMo-V2.5 series under MIT, allowing commercial use, retraining and fine-tuning without extra permission, with an Orbit 100T Token program offering up to 1.6 billion credits.

r/LocalLLaMA

Local coding models have reached a threshold for real work

Antigma tested 27B–32B open-weight models; Qwen 3.6-27B scored 38.2% on Terminal-Bench 2.0. The run used 89 tasks and the default per-task timeout, while verified SOTA is about 80%. The key claim is deployment lag: offline coding is about 6–8 months behind hosted frontier models.

Why it matters: Antigma ran several 27B to 32B open models through one agent flow on 89 Terminal-Bench 2.0 tasks, and the best, Qwen 3.6-27B at 38.2%, trails cloud frontier models near 80% by six to eight months.

r/LocalLLaMA

Microsoft Presents TRELLIS.2: Open-Source 4B Image-to-3D Model

Microsoft’s title says TRELLIS.2 is an open-source 4B image-to-3D model. The title lists 1536³ PBR assets, native 3D VAEs, and 16× spatial compression; the Reddit body is blocked by 403 and discloses no license or benchmarks.

Why it matters: Microsoft open-sourced TRELLIS.2, a 4B-parameter image-to-3D model that outputs assets up to 1536³ with PBR materials, worth a look for anyone tracking open 3D generation.

Apr 26Sunday

Hacker News front page

Why SWE-bench Verified No Longer Measures Frontier Coding Capabilities

OpenAI stopped reporting SWE-bench Verified scores and recommends SWE-bench Pro instead. It audited 138 tasks that o3 failed inconsistently across 64 runs and found 59.4% had test or prompt flaws. The key issue is contamination: tested frontier models reproduced some gold patches or task details.

Why it matters: OpenAI audited 138 problems o3 repeatedly failed and found at least 59.4% had overly rigid test cases or vague descriptions, which explains why it retired the benchmark.

Apr 25Saturday

MIT Technology Review · AI

Three reasons why DeepSeek’s new model matters

DeepSeek released a V4 preview with two versions: V4-Pro and V4-Flash. V4-Pro costs $1.74/M input tokens and $3.48/M output tokens; V4-Flash is about $0.14/$0.28, and both support 1M-token context. The key point is attention efficiency and open weights pressuring agentic coding costs.

Why it matters: DeepSeek's V4 preview prices Pro and Flash about twelve times apart, both with 1M-token context, so readers can compare open-model costs for long text.

Bloomberg Technology

China’s DeepSeek Unveils New Model a Year After Shock Launch

DeepSeek unveiled a new flagship AI model about one year after its open-source release jolted Silicon Valley. The title and RSS snippet confirm that timing; the post does not disclose the model name, size, pricing, benchmarks, or release terms. The key thing to watch is the missing launch detail, not the comeback framing.

Why it matters: Bloomberg reports DeepSeek launched a new flagship model on the first anniversary of its open-source release, with the name, parameter count, pricing and benchmark scores not yet shown.

Apr 24Friday

Bloomberg Technology

DeepSeek unveils flagship AI model a year after breakthrough

DeepSeek released preview versions of a new flagship AI model one year after its breakout. The RSS snippet calls it its most powerful open-source platform and frames it against OpenAI and Anthropic; the post does not disclose parameters, context length, benchmarks, or rollout timing. The actionable facts so far are limited to its preview status and open-source positioning.

Why it matters: DeepSeek released a preview of its next flagship model exactly a year after its breakout, calling it its strongest open platform yet.

X · @dotey

DeepSeek releases and open-sources V4 preview; 1M context is standard across all services

DeepSeek released and open-sourced the V4 preview, making 1M context standard across all official services with no tier or price split. The post says V4-Pro and V4-Flash use token compression plus DSA sparse attention to cut compute and memory costs for 1M context; legacy APIs remain for 3 months and stop after July 24.

Why it matters: DeepSeek released and open-sourced the V4 preview series, making a million-token context the default on both Pro and Flash tiers rather than a premium feature.

Hacker News front page

GPT-5.5: Mythos-Like Hacking, Open to All

XBOW says GPT-5.5 cut miss rate to 10% on its real-vulnerability benchmark, versus 40% for GPT-5 and 18% for Opus 4.6. It scored 97.5% on visual acuity and used about half the login iterations of the next-best model. The key point is black-box testing: GPT-5.5 without source beat GPT-5 with source.

Why it matters: Security firm XBOW tested GPT-5.5 against a real vulnerability set, and its black-box miss rate fell to 10%, below GPT-5 white-box at 40% and Opus 4.6 at 18%.

DeepSeek · API updates

DeepSeek API supports DeepSeek-V4, old model names retire in three months

The DeepSeek API now supports V4-Pro and V4-Flash, compatible with the OpenAI ChatCompletions interface and the Anthropic interface. The base_url stays the same for the new models, but the model parameter must be set to deepseek-v4-pro or deepseek-v4-flash.

Why it matters: DeepSeek keeps the API address while changing the model parameter and sets a clear retirement date for old model names, so migration means adjusting call configuration.

Apr 23Thursday

QbitAI · WeChat

Qwen3.6-27B open-weights, beats its 397B flagship predecessor on agentic coding

Qwen released Qwen3.6-27B and says it beats Qwen3.5-397B on 4 agentic coding benchmarks with about 1/15 the parameters. The post cites SkillsBench rising from 30.0 to 48.2, GPQA Diamond at 87.8, and AIME26 at 94.1; it uses a dense architecture, Thinking Preservation, and Gated DeltaNet, with weights on Hugging Face and ModelScope.

Why it matters: Qwen released Qwen3.6-27B, a 27B dense model it says beats its own 397B previous flagship on four agentic coding benchmarks.

Bloomberg Technology

Tencent unveils a major AI foundation model upgrade, testing its new OpenAI hire

Tencent announced a major upgrade to its AI foundation model. It is the company's first high-stakes AI test since hiring a top OpenAI researcher. The post does not disclose the model name, parameter count, benchmarks, or launch timing.

Why it matters: Tencent's base-model upgrade is the first test of whether the researchers it hired away from OpenAI can turn that spending into a competitive model.

Hacker News front page

Coding Models Are Doing Too Much

The author programmatically corrupts 400 BigCodeBench problems with single-point bugs to test whether coding models over-edit code during fixes. The post defines the minimal fix as exactly reversing the corruption and measures excess changes with token-level Python Levenshtein distance. The provided body does not disclose final results, model rankings, or training gains.

Why it matters: The author plants single-point bugs in 400 BigCodeBench tasks and uses token-level edit distance to measure how much models over-rewrite when fixing bugs, then tries to train them to make minimal edits.

Apr 22Wednesday

Hacker News front page

Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Qwen released the open-weight 27B dense model Qwen3.6-27B and made it available in Qwen Studio. It scores 77.2 on SWE-bench Verified vs. 76.2 for Qwen3.5-397B-A17B, and 59.3 on Terminal-Bench 2.0 under a 256K context and 3-hour timeout. The real takeaway is deployment: this is not a larger MoE, but a denser 27B model with stronger coding results.

Why it matters: Qwen open-sourced Qwen3.6-27B, a 27B dense coding model that scores 77.2 on SWE-bench Verified, beating the previous 397B-parameter MoE flagship, making the dense-versus-MoE comparison the draw.

Apr 21Tuesday

Synced · WeChat

Anonymous world model MotuBrain tops WorldArena and RoboTwin2.0

MotuBrain ranked first on both WorldArena and RoboTwin2.0, with a 63.77 EWM Score on WorldArena and 95.8/96.1 in RoboTwin Clean and Randomized settings. The post says it also leads Motion Quality, Flow Score, and Motion Smoothness, and averages 96.0 across 50 RoboTwin tasks versus 92.3 for second place; the post does not disclose its owner, model size, or training setup. The result matters because it supports a single-model path that combines world prediction with robot action, at least on benchmarks.

Why it matters: An anonymous model, MotuBrain, tops both WorldArena and RoboTwin2.0, averaging 96.0 across RoboTwin's fifty tasks, 3.7 points above second place.

Latent Space

Moonshot Kimi K2.6 open-weight model refresh aims to catch Opus 4.6

Moonshot released Kimi K2.6, a 1T-parameter MoE with 32B active and 256K context. The post cites 58.6 on SWE-Bench Pro, 4,000+ tool calls, 12+ hour runs, and 300 parallel sub-agents. The key signal is long-horizon agent execution, not only open-model scores.

Why it matters: Moonshot AI released Kimi K2.6, an open model with 1 trillion total parameters built for long multi-step agent tasks, with the company claiming runs of over 12 hours and 300 sub-agents managed in parallel.

Apr 20Monday

r/LocalLLaMA

Training LoRA adapters for Apple's on-device 3B model on a free Colab T4 and a Mac

The author built a QLoRA pipeline for Apple’s on-device 3B model, cutting training needs from about 24GB to about 1GB RAM and 5GB GPU, enough for a free Colab T4 or a 24GB Mac. The post says A100 LoRA, T4 QLoRA, and Mac QLoRA adapters perform about the same, raising accuracy from about 40% to 75%, or 86% with retrieval; it also reports a confirmed Apple bug that writes a hidden ~160MB cache copy per CLI call, reaching 269GB over ~300 runs.

Why it matters: A walkthrough of fine-tuning Apple's on-device 3B model with QLoRA, cutting memory needs to 1GB RAM plus 5GB VRAM so a free Colab T4 or a 24GB Mac can run it.

r/LocalLLaMA

Compared some models for feature planning

A Reddit user tested 9 models on planning a “load tracking” feature for a Go budgeting app, then used Claude Code to rank the generated specs, with Claude Opus 4.6 placed first. The table shows Opus 4.6 produced a 19 KB spec with 44 code reads at $2.47; GLM 5.1 ranked second and Qwen 3.6 35B fp8+vLLM ranked third. Do not treat this as a benchmark: the author says it is not representative, and the post does not disclose any manual quality review yet.

Why it matters: A Reddit user had nine models design load-tracking for the same Go bookkeeping app, then used Claude Code to score the specs, making their real-task output comparable.

r/LocalLLaMA

TRELLIS.2 image-to-3D now runs on Mac (Apple Silicon) with no NVIDIA GPU required

A developer ported Microsoft's TRELLIS.2 to Apple Silicon and reports generating ~400K-vertex meshes from one photo in about 3.5 minutes on an M4 Pro with 24GB. The port replaces five CUDA-only extensions with PyTorch MPS and custom backends; texture baking takes about 18 seconds, removing the NVIDIA and cloud requirement.

Why it matters: A developer ported Microsoft TRELLIS.2 to Apple Silicon, swapping five CUDA extensions for PyTorch MPS, generating a roughly 400,000-vertex mesh in about three and a half minutes on a 24GB M4 Pro.