Skip to content

Alibaba's Qwen family: open releases and iterations, from flagship models to small on-device ones.

Latest picks

101–120 of 215

May 28Thursday

r/LocalLLaMA

Inferencing at 10.33 t/s on Qwen 3.5 35B on a $300 laptop

A Reddit user ran Qwen 3.5 35B Q4_K_S on a $300 Lenovo Ideapad Slim 3i and reported 10.33 t/s inference using ik_llama.cpp with two pinned CPU cores, MTP speculative decoding, 64 batch size, and Q8_0 KV cache.

Why it matters: A Reddit user ran the Q4_K_S quant of Qwen 3.5 35B on a $300 Lenovo Ideapad Slim 3i using CPU only, reaching 10.33 tokens per second.

May 27Wednesday

AI HOT (Curated Pool)

Reachy Mini enables fully local voice interaction

Reachy Mini implements local voice interaction through the speech-to-speech library, using a cascaded pipeline with a Realtime API-compatible WebSocket interface and default components including Silero VAD, Parakeet-TDT, and Qwen3-TTS.

Why it matters: Hugging Face built a fully local voice conversation pipeline for Reachy Mini, with each stage from Silero VAD to Parakeet-TDT to Qwen3-TTS documented, giving a complete path for anyone reproducing this on edge devices.

AI HOT (Curated Pool)

Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL

Hugging Face merged TRL PR 5417 for delta weight sync, sending only changed weights as sparse safetensors via a Hugging Face Bucket; on Qwen3-0.6B, the per-step payload falls from 1.2GB to 20–35MB.

Why it matters: Hugging Face implemented incremental weight sync in TRL, sending only the weights that changed between optimizer steps, cutting the per-step payload for Qwen3-0.6B from 1.2GB to 20-35MB.

May 26Tuesday

Alibaba Technology · WeChat

Nearly 9x training speedup: residual streams in DiT are becoming a convergence bottleneck

Nanjing University LAMDA and Alibaba Intelligent Engine proposed DAR, a timestep-aware cross-layer routing method that replaces fixed residual accumulation in DiT; on ImageNet 256x256, it reduced SiT-XL/2 FID from 9.67 to 7.56 and reached baseline convergence quality with 8.75x fewer training iterations.

Why it matters: Nanjing University's LAMDA lab and Alibaba replace DiT's fixed residual accumulation with per-timestep cross-layer routing, reaching equal convergence on ImageNet in about one-ninth the training steps.

AI HOT (Curated Pool)

Qwen3.7-Max Becomes the World’s No. 2 AI Coding Model

Qwen3.7-Max scored 1541 on Code Arena and ranked behind Claude; the post says it can run 35-hour tasks and perform more than 1,000 tool calls.

Why it matters: Qwen3.7-Max scores 1541 on the Code Arena coding benchmark for second place worldwide behind Claude, with the company also claiming 35 hours of continuous operation and over 1,000 tool calls in a single run.

AI HOT (Curated Pool)

ModelBest open-sources MiniCPM5-1B, topping sub-2B models on AA-Index

ModelBest open-sourced MiniCPM5-1B, a 1B-parameter edge language model that beats all sub-2B models on AA-Index, uses a 0.5GB weight file after INT4 quantization, and runs on phones and browsers.

Why it matters: ModelBest, Tsinghua and OpenBMB open-sourced the 1B MiniCPM5-1B, which tops the AA-Index among sub-2B models and runs on phones and browsers at 0.5GB after INT4 quantization.

r/LocalLLaMA

Update on a 12×32GB SXM V100 Cluster for Local Legal Drafting

A lawyer runs a local legal-drafting pipeline across 16 GPUs, with Qwen3.5-122B-A10B reaching about 50 tok/s on four V100s, while a verifier blocks ungrounded citations, dates, and Bates numbers before any final document is used.

Why it matters: A lawyer built a local cluster from 12 V100s with 32GB of VRAM each, running Qwen3.5-122B on 4 cards at roughly 50 tokens/sec to draft legal documents, showing a workable local setup for professional use.

May 25Monday

r/LocalLLaMA

NuExtract3 released: open-weight 4B VLM for Markdown, OCR and structured extraction

Numind released NuExtract3, a 4B open-weight VLM based on Qwen3.5-4B under Apache-2.0, supporting image and text to Markdown, OCR, and JSON-template extraction, with self-hosting from 4GB VRAM and weights in Safetensors, GGUF, and MLX formats.

Why it matters: Numind open-sourced NuExtract3, a 4B vision model that turns image text into Markdown, does OCR and pulls structured data into JSON templates, running locally on as little as 4GB of VRAM.

May 24Sunday

r/LocalLLaMA

BitCPM-CANN: Native 1.58-Bit Large Language Model Training on Ascend NPU

OpenBMB released BitCPM-CANN, a 1.58-bit QAT training stack on Ascend NPU with 0.5B, 1B, 3B, and 8B models trained from scratch, where the 1B to 8B variants retain 95.7%–97.2% of full-precision MiniCPM4 performance across 11 benchmarks.

Why it matters: OpenBMB trained four 1.58-bit ternary models from 0.5B to 8B from scratch on Huawei Ascend NPUs, with the 8B retaining 97% of full-precision performance and compression done during training.

r/LocalLLaMA

Using llama.cpp native tools for web RAG inside llama-server WebUI

A Reddit user describes using llama.cpp native tools for web RAG inside llama-server WebUI with a 7-step setup: enable get_datetime and exec_shell_command, then run wget through firejail, a separate Linux user, and an Alpine OCI VM sandbox.

Why it matters: A Reddit user laid out the exact steps for calling llama.cpp's native tools from inside llama-server's built-in web UI to do web retrieval, a reusable path for giving local models live data.

Synced · WeChat

ICML 2026: First Parallel Thinking Framework for Vision-Language Models

Visual Para-Thinker introduces a parallel thinking framework for vision-language models, using Pa-Attention and LPRoPE to isolate four visual reasoning paths and training on 163,000 question-answer pairs.

Why it matters: Visual Para-Thinker, accepted at ICML 2026, splits visual reasoning into four parallel paths with Pa-Attention and LPRoPE, trained on 163,000 question-answer pairs.

r/LocalLLaMA

It's OK to Quantize the KV Cache; Model Quant Matters More in Qwen3.6 27B KLD Tests

Reddit user hopbel tested Qwen3.6 27B with approximate KLD on wikitext-2 at 16k context, using Q5_K_M as the proxy baseline; Q5_K_S weights with q4_0 KV cache scored 0.016304, while Q4_K_XL with f16 KV cache scored 0.026067, so weight quant tier dominated KV-cache quant in this setup.

Why it matters: One Reddit user's KLD tests on Qwen3.6 27B give first-hand numbers on whether to quantize the KV cache or the weights for local deployment.

May 23Saturday

r/LocalLLaMA

club-rdna16: Practical 16GB AMD/Radeon local LLM testing repo

club-rdna16 publishes a practical 16GB Radeon local LLM testing repo, with an RX 6900 XT running llama.cpp on ROCm/HIP and Qwen3.6 35B-A3B reaching a stable 131k context using q8 KV cache.

Why it matters: A public test repo shows Qwen3.6 35B-A3B holding 131k context with q8 KV cache on an RX 6900 XT, giving 16GB AMD card owners a measured config to copy.

r/LocalLLaMA

How small can the orchestration model in an agent be? Separating it from code generation

HomoAgens1 runs a local ReAct orchestration loop on Qwen3.6-35B-A3B, with about 3B active parameters, a 12GB GPU, 30 expert offload, and 40 tokens/s prompt generation; smaller dense models fail first on tool-call discipline, inventing arguments or repeating bad calls, while reasoning is not identified as the first break point.

Why it matters: HomoAgens1 splits agent orchestration from code generation and runs a local ReAct loop on a roughly 3B-active-parameter MoE at 40 tokens per second on 12GB of VRAM.

r/LocalLLaMA

Experts first llama.cpp

comanderxv published a llama.cpp fork that caches MoE experts in 12GB VRAM; on an RTX 2060 with Qwen3.6-35B-A3B, throughput rose from 19/22 tk/s to 26 tk/s at about a 62% expert-cache hit rate.

Why it matters: comanderxv opened an experts first branch for llama.cpp that tunes MoE memory scheduling, lifting a 12GB RTX 2060 from 19/22 tk/s to 26 tk/s.

May 22Friday

AI HOT (Curated Pool)

Alibaba Qianwen App, PC, and Web Add Qwen3.7-Max

Alibaba added Qwen3.7-Max to the Qianwen app, PC client, and web client, with free access after updating the app to version 6.9.7 or later, and the official test reports a 35-hour autonomous kernel optimization run with more than 1,000 tool calls.

Why it matters: Alibaba put Qwen3.7-Max into the Qwen app, PC and web clients for free, saying it writes code and runs office workflows, and it once ran a 35-hour kernel optimization with over 1,000 tool calls.

May 21Thursday

Xinzhiyuan · WeChat

USTC Papers Study Lifelong Learning for LMMs via Multimodal Knowledge Injection

USTC researchers released MMEVOKE and KORE: MMEVOKE contains 9,422 samples across 159 subcategories, while KORE uses knowledge-tree augmentation and null-space constrained fine-tuning to reduce catastrophic forgetting during multimodal knowledge injection.

Why it matters: A USTC team's MMEVOKE benchmark and KORE fine-tuning method offer reusable evaluation and training ideas for forgetting in continual learning of multimodal models.

May 20Wednesday

AI HOT (Curated Pool)

Qwen3.7: Agent Frontier

Qwen Studio released Qwen3.7 with chatbots, image and video understanding, and image generation. It also covers document processing, web search integration, tool calling, and artifact generation. The RSS snippet frames it as an agent-focused model, but the post does not disclose context length. It also omits benchmark scores, pricing, API limits, release schedule, and reproducible evaluation conditions.

Why it matters: Qwen launched Qwen3.7-Max, a closed model for agent work, demoing 35 hours of continuous operation and over a thousand tool calls for kernel optimization.

r/LocalLLaMA

Nemotron-Labs-Diffusion from NVIDIA

NVIDIA released the Nemotron-Labs-Diffusion 3B, 8B, and 14B dense model family with AR decoding, diffusion parallel decoding, and self-speculation; the 8B model reaches 850 tok/s on GB200 at concurrency 1, compared with 253 tok/s for AR and 360 tok/s for Eagle3.

Why it matters: NVIDIA's Nemotron-Labs-Diffusion ships three dense models from 3B to 14B that decode both autoregressively and by diffusion, plus a self-speculation speedup.

r/LocalLLaMA

KV cache quantization benchmarks: TurboQuant is overrated, q5 deserves attention, q8 may waste VRAM

Anbeeld benchmarked KV cache quantization for Qwen 3.6 27B on one RTX 3090 at 64k and 128k context, reporting q4_0 tail KLD 32% worse than q5_0 and turbo4 running 17% slower than q4_0 with little memory saving.

Why it matters: Anbeeld benchmarks Qwen 3.6 27B KV cache quantization on an RTX 3090, with numbers like q4_0's tail KLD 32% worse than q5_0.