Skip to content

Alibaba's Qwen family: open releases and iterations, from flagship models to small on-device ones.

Latest picks

121–140 of 215

May 20Wednesday

r/LocalLLaMA

Floor for local meeting summarization on a 6GB GPU: Qwen3.5 0.8B works in 57s, Granite 4 350M hallucinates

The author tested VoiceFlow 1.6.0 on an RTX 3060 Laptop 6GB, where Qwen3.5 0.8B summarized a 4-minute meeting in 57 seconds with 16K context, while Granite 4 350M returned summaries in 0.6-2.8 seconds but fabricated Binance and Star Trek content.

Why it matters: A user tested two small models on a 6GB RTX 3060 laptop: Qwen3.5 0.8B summarized a 4-minute meeting in 57 seconds, while the faster Granite 4 350M made things up.

May 19Tuesday

r/LocalLLaMA

Sapient Intelligence releases HRM-Text 1B: 40B tokens, ~$1k pretrain

Sapient Intelligence released HRM-Text 1B, a 1B-parameter model trained from scratch on 16 GPUs for 1.9 days with 40B tokens and a reported ~$1,000 budget; its self-reported chart shows MATH 56.2 and DROP 82.2, while independent evaluation remains pending.

Why it matters: A 1B model pretrained on 16 GPUs in 1.9 days for about $1,000, beating Llama3.2 3B on MATH and DROP, shows one low-cost path for small models.

QbitAI · WeChat

JD and CAS IIE Publish Three Papers Defining Self-Taught RLVR

JD and CAS IIE released three Self-Taught RLVR papers covering RLSD, NPO, and CoPD; RLSD reports that 200 training steps on Qwen3-VL-8B-Instruct exceed GRPO at 400 steps across 8 benchmarks.

Why it matters: JD and the Institute of Information Engineering at CAS published three papers on Self-Taught RLVR, a route where the model generates its own training signal during reinforcement learning, skipping human labels and external reward models.

AI HOT (Curated Pool)

Qwen3.7 Preview lands on Arena; Alibaba rises to fifth in vision ranking

Alibaba says Qwen3.7-Plus-Preview has landed on Arena and that Alibaba now ranks fifth in vision; the post does not disclose benchmark scores, the number of competing models, or a release timeline for the Qwen3.7 series.

Why it matters: Alibaba put Qwen3.7-Plus-Preview on an arena and claims fifth place on a vision leaderboard, with no scores or launch date.

AI HOT (Curated Pool)

Qwen 3.7 Preview

The title identifies Qwen 3.7 Preview, but the post body is empty; it does not disclose parameter size, context window, release timing, access method, pricing, model card details, or benchmark results.

Why it matters: Qwen teased a Qwen 3.7 preview with no parameter count, context window, access method or benchmark scores, so the version name is all there is to note for now.

r/LocalLLaMA

llama.cpp MTP support landed: Qwen3.6 27B reaches 2.44× on Strix Halo

llama.cpp merged MTP speculative decoding in PR #22673; Qwen3.6 27B Q8_0 rose from 7.4 to 18.1 tok/s on Strix Halo, while a dual RTX 3090 Q8_0 setup rose from 25.7 to 55.9 tok/s.

Why it matters: After llama.cpp merged MTP speculative decoding, Qwen3.6 27B measured 7.4 tok/s to 18.1 tok/s on Strix Halo, giving local inference a concrete speedup reference.

May 18Monday

r/LocalLLaMA

Qwen 3.6 27B on 24GB VRAM: backend comparisons, quant choice, and settings

The author tested Qwen 3.6 27B on an RTX 3090 24GB and kept ik_llama.cpp with Qwen3.6-27B-MTP-IQ4_KS.gguf; at 156k context with q8_0 KV and MTP, a ~5.9k-token prompt plus 1024-token output reached about 1261 tok/s prefill and 72.9 tok/s decode, while vLLM lacked a clean single-card long-context run.

Why it matters: The author benchmarked several backends and quantization schemes for Qwen 3.6 27B on a single 24GB RTX 3090, settling on ik_llama.cpp with IQ4_KS, as a single-card comparison for local deployment.

r/LocalLLaMA

I trained TIME: short context-triggered thinking on Qwen instead of overthinking

An independent author trained TIME with QLoRA on Qwen3 4B/8B/14B/32B to trigger short mid-response reasoning when context changes; the post says datasets, notebooks, scripts, curriculum, and TIMEBench are public, with 24GB VRAM enough for training up to 14B.

Why it matters: The author trained TIME on the full Qwen3 range from 4B to 32B with QLoRA, triggering short mid-reasoning only when context changes, and released the dataset, scripts and TIMEBench.

r/LocalLLaMA

LLMs on Android: Snapdragon 8 Elite MoE Experience

A Reddit user tested MoE LLMs on an Honor Magic 7 Pro with Snapdragon 8 Elite and 24GB RAM; under Q4 quantization, LFM2-24b-a2b reached about 24 tokens/s while Gemma reached about 11 tokens/s, and CPU inference was still faster than NPU or GPU in the reported setup.

Why it matters: A Reddit user's real measurements of MoE models on a Snapdragon 8 Elite phone, where CPU inference beat the NPU and LFM2-24b-a2b ran near 24 tok/s at Q4.

May 17Sunday

r/LocalLLaMA

MiroThinker-1.7 Open-Weight Deep Research Agent Based on Qwen3 MoE

MiroMindAI released the MiroThinker-1.7-deepresearch and mini APIs, with the mini version using 30B total parameters and 3B active parameters, weights on HuggingFace, and context management based on sliding window K=5 plus episode restarts.

Why it matters: MiroMindAI released weights for its deep research agent MiroThinker-1.7 on HuggingFace, with a mini version at 30B total and 3B active parameters to lower the bar for local runs.

r/LocalLLaMA

85 GPU-hours comparing 5 abliteration methods on Qwen3.6-27B

Abliterlitics compared five Qwen3.6-27B abliteration variants against the base model using 85 GPU-hours of benchmarks, HarmBench, KL divergence, and weight forensics; Huihui had the smallest benchmark deltas, Heretic had the lowest KL divergence, and all five variants reached near-complete safety removal.

Why it matters: 85 GPU-hours on Qwen3.6-27B comparing five abliteration methods, with benchmarks, HarmBench safety evals, KL divergence and weight forensics all reported together.

r/LocalLLaMA

Same Models Tested Across Strix Halo, RTX 3090, and RTX 5070

C_Coffie published 55 local inference benchmark runs across Strix Halo, RTX 3090, RTX 5070, five backends, and 0.35B to 35B-A3B models; RTX 5070 beats RTX 3090 on models fitting 12GiB, while RTX 3090 leads in the 14–31B band that exceeds 12GiB but fits 24GiB.

Why it matters: A user ran 55 local inference tests across three GPUs and five backends, and reports an RTX 5070 decoding faster than an RTX 3090 within 12GB of VRAM.

May 16Saturday

r/LocalLLaMA

Qwen3.6-35B-A3B and 9B land on the public Terminal-Bench 2.0 leaderboard

little-coder × Qwen3.6-35B-A3B scored 24.6% ±3.2 on Terminal-Bench 2.0, above Gemini 2.5 Pro on Gemini CLI at 19.6% and Qwen3-Coder-480B on Terminus 2 at 23.9%.

Why it matters: Qwen3.6 in 35B-A3B and 9B sizes lands on the public Terminal-Bench 2.0 leaderboard, giving small models a checkable public score on hard terminal-agent tests.

QbitAI · WeChat

Zhejiang University and Microsoft use 3,000 text prompts to improve video 3D consistency with World-R1

Zhejiang University and Microsoft introduced World-R1, training Wan 2.1 with about 3,000 text-only prompts, Flow-GRPO, and a four-part reward; the 1.3B version improves PSNR over the baseline by 10.23 dB.

Why it matters: World-R1 from Zhejiang University and Microsoft aligns Wan 2.1 video generation using about 3,000 text-only prompts and no 3D annotations, a cheap route for teams without labeled data.

r/LocalLLaMA

Orthrus-Qwen3-8B: Up to 7.8× tokens/forward on Qwen3-8B with frozen backbone

Orthrus-Qwen3-8B adds a trainable diffusion attention head to a frozen Qwen3-8B backbone, reaches up to 7.8× tokens per forward and about 6× wall-clock speedup on MATH-500, while training 16% of parameters with under 1B tokens over 24 hours on 8×H200 GPUs.

Why it matters: The team froze the Qwen3-8B backbone and trained only a diffusion attention head, reaching up to 7.8x token throughput per forward pass on MATH-500 and about 6x faster wall-clock time, with the output distribution unchanged.

May 15Friday

r/LocalLLaMA

Used over a million tokens in three sessions to test Qwen 3.6 35B MTP

A Reddit user tested Qwen3.6-35B-A3B MTP across three million-token-scale sessions, using 300k context and KV Q8_0, and reported about 1.5x the tok/sec of earlier tests.

Why it matters: A Reddit user ran over a million tokens across three sessions to test the MTP build of Qwen3.6-35B-A3B at 300k context and KV Q8_0 quantization, measuring roughly 1.5x the generation speed of the earlier version.

r/LocalLLaMA

I Let a Small Model Train on Its Own Mistakes; It Reached 80% on HumanEval and Beat GPT-3.5 on Math

The author fine-tuned Qwen 2.5 7B base on self-mined mistake-correction pairs, raising HumanEval from 25/164 to 112/164; Qwen 2.5 14B used 100 pairs and a 95-minute H100 run costing $3.50.

Why it matters: The author had Qwen 2.5 7B write code and run tests on its own, turned the errors and fixes into training data, and fine-tuning lifted HumanEval pass rate from about 15% to about 68%.

r/LocalLLaMA

The RTX 5000 PRO 48GB arrived and is better than expected

A Reddit user built a $5,600 RTX 5000 PRO 48GB PC and ran Qwen3.6-27B-FP8 with full-precision cache; they report up to 80 tok/s in TG, about 50–60 tok/s on very large prompts, 4,400 tok/s in prompt processing, and 200k tokens fitting in BF16 KV cache.

Why it matters: A builder with no prior experience spent $5,600 on a single RTX 5000 PRO 48GB machine, hitting 80 tok/s on a 27B model with 200k context at full precision.

May 14Thursday

r/LocalLLaMA

Automated AI researcher running locally with llama.cpp

Hugging Face’s ml-intern added local-model support through llama.cpp and ollama; the post says Qwen3.6-35B-A3B can orchestrate CPU/GPU sandboxes and Hub jobs to run an end-to-end SFT workflow.

Why it matters: A community setup wires HuggingFace's ml-intern to llama.cpp and ollama, running a full SFT loop on local Qwen3.6-35B-A3B, a path others can copy.

r/LocalLLaMA

Open-source one-prompt-to-cinematic-reel pipeline on one GPU with FLUX.2 and Wan2.2-I2V

The developer open-sourced StudioMI300, an 8-stage sequential pipeline that turns one English sentence into a 720p MP4 on a single AMD Instinct MI300X, cutting end-to-end time from 25.9 minutes to 10.4 minutes per clip.

Why it matters: StudioMI300 runs the whole 8-step path from one prompt to a 720p short film on a single MI300X, with a vision critic and auto-retry step holding output quality in check.