Skip to content

Alibaba's Qwen family: open releases and iterations, from flagship models to small on-device ones.

Latest picks

141–160 of 215

May 14Thursday

Synced · WeChat

ACL 2026: Alibaba DAMO I²B-LPO Improves RLVR Exploration

Alibaba DAMO Academy introduced I²B-LPO, an RLVR post-training framework that branches rollouts at high-entropy nodes and filters them with an information-bottleneck self-reward, reporting up to 5.3% accuracy gains and 7.4% semantic-diversity gains on math benchmarks using Qwen2.5-7B and Qwen3-14B.

Why it matters: Alibaba DAMO Academy's I²B-LPO branches sampling where a model hesitates on a problem and uses an information bottleneck self-reward to pick out different solutions, easing the convergence of reasoning paths in RL fine-tuning.

AI HOT (Curated Pool)

UnslothAI Releases Qwen3.6 MTP GGUF Models With Over 1.4x Faster Inference

Daniel Han released experimental Qwen3.6 MTP GGUF models, with the 27B model reaching 140 tokens/s on one GPU and the 35B-A3B version reaching 220 tokens/s, using two draft tokens for speculative decoding.

Why it matters: Unsloth founder Daniel Han released an experimental Qwen3.6 MTP GGUF that drafts two tokens per step, trading accuracy for over 1.4x faster inference.

r/LocalLLaMA

2x RTX 3090 setup for local Qwen 3.6 27B inference

A Reddit user ran Qwen 3.6 27B on a dual RTX 3090 Ubuntu setup, reporting 48GB VRAM, a 262k context window, no NVLink, about 4000 pp/s prompt processing, and 113 tk/s generation.

Why it matters: A Reddit user runs Qwen 3.6 27B on two RTX 3090s in Ubuntu, holding 260K context in 48GB of VRAM at 113 tokens per second.

r/LocalLLaMA

24+ tok/s from ~30B MoE models on an old GTX 1080

User mdda ran Qwen 3.6 35B-A3B on an i7-6700, GTX 1080, and 32GB RAM machine at about 24 tok/s with 128k context; the setup uses llama.cpp MoE offloading plus TurboQuant/RotorQuant KV cache quantization, with PCIe 3.0 x16 saturated and GPU utilization at about 40–50%.

Why it matters: A user runs roughly 30B-parameter MoE models on an i7-6700 with a GTX 1080 at about 24 tokens per second and 128k context, with the exact setup and numbers.

May 13Wednesday

Hacker News front page

Show HN: Needle Distills Gemini Tool Calling into a 26M Model

Cactus open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, with MIT-licensed weights released on Hugging Face.

Why it matters: Cactus distilled Gemini's tool calling into a 26M-parameter model that runs locally on consumer devices at 6,000 tokens per second prefill and 1,200 per second generation.

r/LocalLLaMA

Needle: We Distilled Gemini Tool Calling Into a 26M Model

Cactus Compute open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, using an attention-and-gating architecture with no MLPs.

Why it matters: Cactus Compute distilled Gemini's tool calling into a 26M-parameter model and reports 6,000 tokens per second prefill on consumer devices, a lightweight option for on-device tool use.

May 12Tuesday

r/LocalLLaMA

Local LLM Autocomplete and Agentic Coding on a Single 16GB GPU + 64GB RAM

Reddit user grumd runs Qwen2.5-Coder-7B Q6 for autocomplete and Qwen3.6-35B-A3B Q8 for agentic coding on one RTX 5080 with RAM offloading; the post reports about 145k context, 56GB RAM used with other apps open, and Qwen3.6-35B-A3B speed of tg128 at 35.29 tokens/s.

Why it matters: One user runs both a code-completion model and an agentic coding model on a single 16GB GPU with 64GB RAM, and lists the models, quantization and context lengths.

r/LocalLLaMA

Prompt caching for RL training: 7.5x speedup on long-prompt, short-response workloads

The author proposes prompt caching for RL training. On Qwen3.5-4B, it reports a 7.5x speedup with 16k-token prompts and 64-token outputs, and the G=8 example with 1000-token prompts and 100-token responses reduces 8800 processed tokens to 1800 unique tokens.

Why it matters: The author moved prompt caching from inference into RL training, reusing repeated token computation for long-prompt short-response tasks, reaching a 7.5x training speedup on Qwen3.5-4B with 16k prompts and 64-token responses.

May 11Monday

r/LocalLLaMA

ExLlamaV3 Major Updates

ExLlamaV3 added DFlash in v0.0.31, raising Coding throughput from 59.21 t/s to 177.67 t/s; v0.0.32 optimized five models, with Trinity-Nano gaining 72.4% on 6000 Pro², while v0.0.33 adds DFlash model quantization plus bug fixes and efficiency work.

Why it matters: ExLlamaV3 v0.0.31 adds DFlash, lifting coding throughput from 59.21 to 177.67 tokens per second, and v0.0.32 tunes five models, with Trinity-Nano up 72.4% on the 6000 Pro².

Synced · WeChat

ICML 2026: PRISM Brings Efficient Test-Time Scaling to dLLMs

PRISM raises LLaDA-8B-Instruct on GSM8K from 67.58% to 85.30% by combining hierarchical trajectory search, partial remasking, and self-verified feedback, reducing dLLM test-time scaling cost from O(NT) toward O(N+KT) under a final candidate width K.

Why it matters: An ICML 2026 paper proposes PRISM, a three-layer inference-time search using parallel generation, local rollback rewriting and self-scored selection, raising LLaDA-8B-Instruct accuracy on GSM8K from 67.58% to 85.30%.

AI HOT (Curated Pool)

Qwen-Image-2.0 Technical Report

Qwen-Image-2.0 uses a Qwen3-VL condition encoder and multimodal diffusion transformer for image generation and precise editing, with instruction inputs up to 1K tokens and reported gains in multilingual text rendering, layout quality, and human-rated generation and editing tasks.

Why it matters: Qwen-Image-2.0 wires Qwen3-VL in as a conditioning encoder for a multimodal diffusion model, merging generation and precise editing with prompts up to 1,000 tokens.

AI HOT (Curated Pool)

Local models handle half of daily tasks and respond faster than cloud models

A five-week experiment tested about 1,400 daily work tasks, where local 35B models such as Qwen 3.6 35B handled about 50% and averaged 2.8-second responses, 2.1 times faster than Claude Opus 4.5, while the cloud model still led complex reasoning by about 20%.

Why it matters: A five-week test of a local 35B model across about 1,400 daily requests completed half the tasks at 2.8 seconds average, 2.1x faster than cloud Claude Opus 4.5 at 5.8 seconds.

r/LocalLLaMA

MTP benchmark results: task type determines speculative inference speedups or slowdowns

A Reddit LocalLLaMA user ran 300+ tests on Qwen 3.6 27B MTP quants, finding coding draft acceptance at 79-89% and F16 coding speed up 171%, while Q4_K_M creative writing slowed down 9%.

Why it matters: More than 300 runs on a quantized Qwen 3.6 27B show speculative decoding's speedup depends on the task, not the setup.

AI HOT (Curated Pool)

MachinaCheck: Multi-agent CNC manufacturability analysis system built on AMD MI300X

MachinaCheck runs Qwen 2.5 7B locally on AMD MI300X to analyze STEP files for CNC manufacturability, reducing drawing review for quote analysis from 30–60 minutes to 30 seconds while using 192GB HBM3 to keep customer design data on-premises.

Why it matters: MachinaCheck splits CNC drawing review into geometry parsing and agent roles, running a local Qwen 2.5 7B on an AMD MI300X so design data never leaves the factory.

May 10Sunday

Xinzhiyuan · WeChat

Next-ToBE Targets Short-Sighted Next-Token Prediction in LLMs at ICLR 2026

East China Normal University and Fudan University researchers proposed Next-ToBE, a training objective that keeps standard autoregressive inference while adding a soft target over future-token windows, and the article reports the method ranked best in 35 of 36 experiments across Qwen2.5-Math-1.5B, Qwen2.5-Math-7B, and Llama3.1-8B-Instruct.

Why it matters: ECNU and Fudan researchers propose Next-ToBE, which trains against soft targets from the next few words while still generating token by token, winning 35 of 36 tests.

r/LocalLLaMA

BeeLlama.cpp: DFlash and TurboQuant with reasoning and vision support

Anbeeld released BeeLlama.cpp, a llama.cpp fork that runs Qwen 3.6 27B Q5 with 200k context and vision on a single RTX 3090 or 4090; the title claims 2–3x faster than baseline and a 135 tps peak.

Why it matters: Anbeeld's llama.cpp fork BeeLlama.cpp adds DFlash memory-saving attention and TurboQuant quantization, letting Qwen 3.6 27B run 200k context on a single 3090 at 2 to 3 times baseline speed.

May 9Saturday

r/LocalLLaMA

80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP

Reddit user janvitos ran Qwen3.6-35B-A3B-MTP-GGUF with a llama.cpp MTP PR on an RTX 4070 Super. The posted benchmark shows 69.2-81.9 tok/s, 0.694-0.947 draft acceptance, 131072 context, and a -fitt 1536 setting that reserves 1536 MB for the draft model and KV cache.

Why it matters: Reddit user janvitos runs Qwen3.6 35B A3B on a 12GB RTX 4070 Super and posts measured settings for 69.2 to 81.9 tok/s at 128K context.

QbitAI · WeChat

Qwen AI Glasses S1 Adds Spatial 3D Display, Proactive Reminders, and Daily AI Features

Qwen AI Glasses S1 added spatial 3D display and proactive services, with ride-hailing, instant shopping, and photo-based homework help scheduled for this month; Wellsenn XR says Qwen AI Glasses hold 53% of China’s online AI glasses sales since March 8.

Why it matters: Qwen's AI glasses add spatial 3D display and proactive reminders, and Wellsenn XR data puts them at 53% of China's online AI glasses sales.

r/LocalLLaMA

MTP + TurboQuant Running: Qwen3.6-27B Hits 80+ t/s on a Single RTX 4090

indrasmirror ran Qwen3.6-27B-Heretic-v2 on a single RTX 4090 with 262K context, TBQ4_0 KV cache, and MTP draft 3, improving throughput from about 43 t/s to 80-87 t/s with roughly 73% MTP draft acceptance.

Why it matters: indrasmirror pushed Qwen3.6-27B to 262K context on one RTX 4090, using MTP speculative decoding to lift generation from about 43 t/s to 80-87 t/s at a 73% draft acceptance rate.

May 8Friday

Xinzhiyuan · WeChat

Token-Level Length Control: 3B Model Beats GPT 5.4 and Claude

UC Santa Barbara and Apple researchers introduced LenVM, which models remaining generation length as a token-level value function; Qwen2.5-3B with a 1.5B LenVM scored 62.6 on LIFEBench length control, above GPT-5.4 at 37.4 and Claude-Opus-4-6 at 35.5.

Why it matters: Modeling remaining generation length as a token-level value function lets a 3B model beat GPT-5.4 and Claude on length control.