Skip to content

Alibaba's Qwen family: open releases and iterations, from flagship models to small on-device ones.

Latest picks

181–200 of 215

May 2Saturday

r/LocalLLaMA

Qwen 3.6 wins benchmarks, but Gemma 4 looks stronger in local vision tests

A Reddit user compared Qwen 3.6 and Gemma 4 locally on vLLM FP8 across 27B/31B vision models. Qwen burned 8,000+ tokens on hard GeoGuessr cases, while Gemma often used 1,500; Qwen also needed 2 FPS video preprocessing. The practitioner detail: vLLM and Llama.cpp can default Gemma visual tokens to 280, while 1,120+ improved fine-detail accuracy.

Why it matters: A local vLLM test pits Qwen 3.6 against Gemma 4 on vision tasks and finds benchmark wins don't carry over to real use.

r/LocalLLaMA

Qwen3.6-27B hits 72 tok/s on RTX 3090 with native vLLM on Windows

Reddit user One_Slip1455 released a native Windows vLLM launcher for Qwen3.6-27B, reaching 72 tok/s on an RTX 3090. It reports 64.5 tok/s at ~25k tokens, 53.4 tok/s at 127k ctx on one GPU, and 160k ctx with PP=2 on 2×3090. The key detail is no WSL or Docker, an OpenAI-compatible endpoint, and an INT4 quant path.

Why it matters: A native Windows vLLM launcher runs Qwen3.6-27B on one RTX 3090 without WSL or Docker, hitting 72 tok/s at short context.

May 1Friday

r/LocalLLaMA

PFlash: 10x prefill speedup over llama.cpp at 128K on an RTX 3090

PFlash cuts Qwen3.6-27B Q4_K_M 128K TTFT to 24.8s on an RTX 3090, versus 248.4s cold for llama.cpp. It uses a Qwen3-0.6B drafter to score token importance, keeps 5% of spans, and runs C++/CUDA without Python, Triton, or PyTorch. The quality caveat is clear: only NIAH single-needle passes from 32K to 128K; RULER and multi-needle results are not disclosed.

Why it matters: PFlash uses Qwen3-0.6B to pick the 5% of chunks that matter, cutting first-token latency on a 128K-token RTX 3090 run from 248.4s to 24.8s.

r/LocalLLaMA

Follow-up: Qwen3.6-27B on 1× RTX 3090 reaches ~218K context and stable tool calls

A Reddit user ran Qwen3.6-27B on one RTX 3090, reporting ~218K context at 50/66 TPS. After fixing Genesis PN12 patch anchor drift, ~25K-token tool outputs stopped OOMing; 198K plus vision reached 51/68 TPS. Single-prompt single-GPU runs still hit a second memory cliff near 50–60K.

Why it matters: A single 24GB RTX 3090 running Qwen3.6-27B at roughly 218K context and 50 to 66 tokens per second, with tool-call memory overflow fixed.

r/LocalLLaMA

Long-context coding on RTX 5080 16GB: Qwen3.6-35B-A3B holds 30 t/s at 128K

A Reddit user tested a local coding-agent setup on RTX 5080 16GB; the title says Qwen3.6-35B-A3B reaches 30 t/s at 128K. The post lists Ryzen 9700X, 96GB DDR5, Windows 11, and CUDA 12.9.1 as required. Qwen3.6-27B dense hit only 3.2 t/s at 128K, so the key path is KV quantization plus MoE offload.

Why it matters: A 16GB RTX 5080 runs a Qwen3.6-35B-A3B coding agent at 128K context and still holds 30 t/s, with no reported quality drop.

Apr 30Thursday

MIT Technology Review · AI

Goodfire releases Silico, a mechanistic interpretability tool for debugging LLMs

Goodfire released Silico, letting engineers inspect and adjust LLM parameters during training. It maps neurons and pathways; one Qwen 3 neuron triggered trolley-problem-style outputs. Pricing is case-by-case, and the post does not disclose rates.

Why it matters: Silico lets engineers inspect and adjust model parameters during training and map neurons and circuits.

r/LocalLLaMA

Actual comparison between locally run Qwen-3.6-27B and proprietary models

The author compared 5 model setups on an autoresearch-loop task; only Qwen-3.6-27B via OpenRouter nearly solved it. The local q4_k_m run took about 8 hours and used 39k/45k tokens; full-quality Qwen used 4.4M tokens and cost $0.939. The useful signal is failure quality: both Qwen runs needed small fixes, while Gemma, Codex-Spark, and Claude Haiku 4.5 missed tests or key logic.

Why it matters: A researcher ran an automated research loop comparing a local quantized Qwen-3.6-27B against closed models; only the local one came close, at 39,000 tokens versus 4.4 million for full precision.

r/LocalLLaMA

Notes on what actually breaks when you run a coding agent on small local models

A Reddit user tested small local and free-tier cloud models for weeks on multi-file coding tasks. Sub-7B structured output was unreliable; failures included markdown fences, wrong-file edits, and read/write misclassification, with post-processing and validation as fixes.

Why it matters: After weeks of testing sub-7B local models on multi-file coding tasks, the author logs the three places structured output breaks most and the fixes, from post-processing to JSON schema validation.

r/LocalLLaMA

Qwen-Scope: Official Sparse Autoencoders (SAEs) for Qwen 3.5 models

Qwen Team released Qwen-Scope, SAEs for Qwen 3.5 models from 2B to 35B MoE. It maps residual-stream features across all layers, including Feature #6159 for Chinese activation. The key point is feature-level debugging and steering; the license discourages removing safety filters.

Why it matters: Qwen open-sourced Qwen-Scope, sparse autoencoders spanning 2B to 35B MoE models with full residual streams, splitting tangled activations into separate features people can inspect and debug.

QbitAI · WeChat

NUS and collaborators propose ViF to curb visual hallucination snowballing in multi-agent systems

NUS LV-Lab and collaborators proposed ViF, accepted to ICLR 2026. Across 8 benchmarks, 4 MAS structures, and 10 VLMs, it reports 2.4%–3.8% average gains. ViF replaces text-only passing with visual relay tokens and layered attention redistribution, cutting HS by over 30% on average and nearly 40% in ring topology.

Why it matters: NUS and other teams propose ViF, packing visual information into relay tokens inside multi-agent dialogue so models need no changes to curb hallucination buildup from text-only handoffs.

r/LocalLLaMA

Building a fully local PDF-to-audiobook workflow with Kokoro 82M, Qwen and llama.cpp

Reddit user purellmagents shared a local PDF-to-audiobook workflow using Kokoro 82M, Qwen 3.5 0.8B/2B, and llama.cpp. The Tauri 2.0 app runs on an M1 Mac, reads 15 initial sentences, then prepares the next 15. The hard parts are PDF-text alignment, code snippets, tables, and first-generation latency.

Why it matters: A Reddit user built a fully local PDF-to-audiobook pipeline on an M1 Mac with Kokoro 82M, Qwen and llama.cpp, reading 15 sentences while preparing the next 15 to cut waiting time.

Apr 29Wednesday

r/LocalLLaMA

Qwen3.6 27B on Dual RTX 5060 Ti 16GB with vLLM: ~60 tok/s, 204k Context Working

A user ran Qwen3.6 27B with vLLM on dual RTX 5060 Ti 16GB cards, reaching ~62–66 tok/s at 8K. The setup used 32GB VRAM, TP=2, fp8 KV cache, MTP 3 tokens, and a 204800 context window. The tight part is memory: after a 168k prefill, each GPU used ~15.65GiB with max_num_seqs=1.

Why it matters: A user ran Qwen3.6 27B on two RTX 5060 Ti 16GB cards with vLLM and measured 62–66 tok/s at 8K context, with the exact tensor-parallel, fp8 KV cache and MTP settings listed.

Apr 28Tuesday

Synced · WeChat

Open-source medical video understanding system uAI-NEXUS-MedVLM released

United Imaging Intelligence released uAI-NEXUS-MedVLM for medical video understanding, with a CVPR 2026 paper. MedVidBench has 532k video-instruction pairs across 8 medical sources and 8 tasks. Qwen2.5-VL-7B SFT reached 89.4% CVS accuracy; GPT-5.4 scored 16.4%.

Why it matters: United Imaging Intelligence open-sourced uAI-NEXUS-MedVLM, a large model for ultrasound, endoscopy and other medical video, plus MedVidBench, an evaluation set of 532,000 video instruction pairs.

r/LocalLLaMA

Local coding models have reached a threshold for real work

Antigma tested 27B–32B open-weight models; Qwen 3.6-27B scored 38.2% on Terminal-Bench 2.0. The run used 89 tasks and the default per-task timeout, while verified SOTA is about 80%. The key claim is deployment lag: offline coding is about 6–8 months behind hosted frontier models.

Why it matters: Antigma ran several 27B to 32B open models through one agent flow on 89 Terminal-Bench 2.0 tasks, and the best, Qwen 3.6-27B at 38.2%, trails cloud frontier models near 80% by six to eight months.

Apr 27Monday

Hacker News front page

Running Local LLMs Offline on a Ten-Hour Flight

Dmitri Lerko ran Gemma 4 31B and Qwen 4.6 36B locally during a 10-hour flight with no Wi‑Fi. The MacBook Pro M5 Max had 128GB unified memory and a 40-core GPU; sustained load used about 1% battery per minute, and performance degraded past 100k tokens. The sharp finding is instrumentation: an iPhone cable delivered 60W, while a MacBook cable delivered 94W under the same load.

Why it matters: The author ran two local models offline on a maxed-out MacBook Pro M5 Max during a ten-hour flight, and logged how 60W versus 94W charging changed inference.

Synced · WeChat

Apple paper asks: What do your logits know?

Apple researchers posted an arXiv paper testing whether VLM top-k logits leak image details. Using CLEVR, MSCOCO, and probes, 30–80 logits recover noise, target traits, and some background attributes. The key risk is gray-box APIs exposing top-k log probabilities.

Why it matters: Apple's experiments on CLEVR and MSCOCO found that the top 30 to 80 logits alone can recover image noise, object features and some background attributes, giving quantifiable evidence of output-interface risk in vision-language models.

Apr 24Friday

Synced · WeChat

Remember more, answer faster, use less: HERMES speeds real-time streaming video understanding by 10x

Fudan University, Shanghai Academy of AI for Science, and NUS proposed HERMES, a training-free framework that turns KV cache into hierarchical memory for streaming video understanding and cuts TTFT by up to 10x. The post lists three mechanisms: hierarchical cache management, cross-layer memory smoothing, and position re-indexing; it reports 68% fewer video tokens with comparable or better results, and Qwen2.5-VL-7B on StreamingBench rising from 73.31% to 79.44%. What matters for practitioners: it answers without external retrieval, with TTFT around 27/29/28 ms at 16/64/256 frames.

Why it matters: Fudan University and other teams propose HERMES, a training-free framework that organizes the KV cache into three memory layers to speed up streaming video understanding, cutting first-token latency by up to 10x.

Apr 23Thursday

QbitAI · WeChat

Qwen3.6-27B open-weights, beats its 397B flagship predecessor on agentic coding

Qwen released Qwen3.6-27B and says it beats Qwen3.5-397B on 4 agentic coding benchmarks with about 1/15 the parameters. The post cites SkillsBench rising from 30.0 to 48.2, GPQA Diamond at 87.8, and AIME26 at 94.1; it uses a dense architecture, Thinking Preservation, and Gated DeltaNet, with weights on Hugging Face and ModelScope.

Why it matters: Qwen released Qwen3.6-27B, a 27B dense model it says beats its own 397B previous flagship on four agentic coding benchmarks.

Bloomberg Technology

Alibaba Adds China Eastern Flight Booking to Flagship Qwen App

Alibaba added China Eastern flight booking to the Qwen app, letting users book flights directly; the snippet says this is the first time its agentic AI tech has opened to a major commercial partner. The RSS snippet does not disclose launch regions, fare classes, payment flow, or revenue terms. The real signal is Qwen moving from chat entry to transaction flow, not just another assistant feature.

Why it matters: Alibaba is opening its Qwen agent to a major commercial partner, letting users book China Eastern flights without leaving the app.

Apr 22Wednesday

Hacker News front page

Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Qwen released the open-weight 27B dense model Qwen3.6-27B and made it available in Qwen Studio. It scores 77.2 on SWE-bench Verified vs. 76.2 for Qwen3.5-397B-A17B, and 59.3 on Terminal-Bench 2.0 under a 256K context and 3-hour timeout. The real takeaway is deployment: this is not a larger MoE, but a denser 27B model with stronger coding results.

Why it matters: Qwen open-sourced Qwen3.6-27B, a 27B dense coding model that scores 77.2 on SWE-bench Verified, beating the previous 397B-parameter MoE flagship, making the dense-versus-MoE comparison the draw.