Skip to content

#评测/基准

1 today

May 25Monday

r/LocalLLaMA

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

RTPurbo converts full-attention LLMs to sparse inference with a few hundred adaptation steps. It keeps the full KV cache only for retrieval heads, uses a 16-dimensional token indexer, and reports up to 9.36x prefill speedup at 1M context plus about 2.01x decode speedup on long-context and reasoning benchmarks.

Why it matters: RTPurbo converts a full-attention model to sparse inference by keeping full KV cache only for retrieval heads, cutting prefill time up to 9.36x at 1M tokens.

May 24Sunday

r/LocalLLaMA

BitCPM-CANN: Native 1.58-Bit Large Language Model Training on Ascend NPU

OpenBMB released BitCPM-CANN, a 1.58-bit QAT training stack on Ascend NPU with 0.5B, 1B, 3B, and 8B models trained from scratch, where the 1B to 8B variants retain 95.7%–97.2% of full-precision MiniCPM4 performance across 11 benchmarks.

Why it matters: OpenBMB trained four 1.58-bit ternary models from 0.5B to 8B from scratch on Huawei Ascend NPUs, with the 8B retaining 97% of full-precision performance and compression done during training.

Xinzhiyuan · WeChat

AI-generated articles now outnumber human-written ones: what is left for the brain?

Graphite sampled 43,000 CommonCrawl articles and found AI-generated English articles exceeded human-written ones from November 2024, with its detector reporting about a 4.2% false-positive rate and 0.6% false-negative rate.

Why it matters: A CommonCrawl sample shows AI-written English articles overtaking human ones from November 2024, giving a dated measure of how much AI text is in public web data.

r/LocalLLaMA

Vision-capable LLMs vs. OCR for long-document QA with charts, images, and tables

The author tested Claude Sonnet 4.5 on 171 questions from 30 image-heavy MMLongBench-Doc PDFs, comparing native PDF vision use with OCR pipelines. Native PDF ranked fifth of six at 52.0% accuracy and cost $0.2552 per query, while LlamaCloud premium with full context reached 59.6% at $0.1885 per query.

Why it matters: A hands-on test compares reading chart-heavy PDFs directly against OCR-then-QA on 30 documents, with accuracy and per-query cost reported.

r/LocalLLaMA

It's OK to Quantize the KV Cache; Model Quant Matters More in Qwen3.6 27B KLD Tests

Reddit user hopbel tested Qwen3.6 27B with approximate KLD on wikitext-2 at 16k context, using Q5_K_M as the proxy baseline; Q5_K_S weights with q4_0 KV cache scored 0.016304, while Q4_K_XL with f16 KV cache scored 0.026067, so weight quant tier dominated KV-cache quant in this setup.

Why it matters: One Reddit user's KLD tests on Qwen3.6 27B give first-hand numbers on whether to quantize the KV cache or the weights for local deployment.

May 23Saturday

Synced · WeChat

Bengio Paper Raises Recursive Reasoning Limits as Parallel Trajectories Beat Serial Reasoning

Yoshua Bengio’s team introduced GRAM, a generative recursive reasoning model that samples multiple latent trajectories; on Sudoku-Extreme, GRAM reached 97.0% accuracy with 16 recursive steps and 20 parallel samples, exceeding TRM’s 90.5% result at 320 serial recursive steps.

Why it matters: Bengio's team proposes GRAM, generating several reasoning traces in parallel and picking the best, hitting 97.0% on hard Sudoku in 16 recursive steps where serial models need 320 steps for 90.5%.

r/LocalLLaMA

club-rdna16: Practical 16GB AMD/Radeon local LLM testing repo

club-rdna16 publishes a practical 16GB Radeon local LLM testing repo, with an RX 6900 XT running llama.cpp on ROCm/HIP and Qwen3.6 35B-A3B reaching a stable 131k context using q8 KV cache.

Why it matters: A public test repo shows Qwen3.6 35B-A3B holding 131k context with q8 KV cache on an RX 6900 XT, giving 16GB AMD card owners a measured config to copy.

AI HOT (Curated Pool)

Project Glasswing: Initial Update

Anthropic says Project Glasswing used Claude Mythos Preview with about 50 partners to find more than 10,000 high or critical vulnerabilities in global critical systems, with independently verified accuracy of 90.6%.

Why it matters: Anthropic and about 50 partners used Claude Mythos Preview to find over 10,000 high-severity bugs in critical software, and fixing them is now the bottleneck.

May 22Friday

AI HOT (Curated Pool)

Text Degeneration: A Production Failure Mode Most Benchmarks Do Not Track

Dharma-AI says in a Hugging Face post that large language models can produce repeated, incoherent, or logically confused text in production, and most mainstream benchmarks do not track this failure mode.

Why it matters: Dharma-AI points out that deployed large models often degrade into repetition and broken logic, and mainstream benchmarks barely score these failures, leaving a gap between evaluation and production behavior.

AI HOT (Curated Pool)

BitCPM-CANN Released as First 1.58-bit Open Model Fully Trained on Huawei Ascend 910B NPU

ModelBest, Tsinghua University, and OpenBMB released BitCPM-CANN, a 0.5B-8B open model family trained natively on Huawei Ascend 910B NPUs with 1.58-bit ternary weights, cutting memory use by about 6x versus BF16 while retaining 95-97% of full-precision benchmark performance.

Why it matters: ModelBest, Tsinghua and OpenBMB released BitCPM-CANN, trained entirely on Huawei Ascend 910B, where 1.58-bit quantization saves about 6x memory over BF16 while benchmarks hold 95-97% of full-precision scores.

Bloomberg Technology

Pentagon Tests Rival AI Models in Race to Replace Anthropic

The Pentagon is testing rival AI models with 25 departmental “power users” as it seeks alternatives to Anthropic’s Claude, according to a senior defense official; the RSS snippet does not disclose the candidate model list, evaluation criteria, or deployment timeline.

Why it matters: The Pentagon is testing OpenAI and Google models with senior users across 25 departments as it looks for alternatives to its current Anthropic Claude.

May 21Thursday

r/LocalLLaMA

Agent Execution Tax: New Procurement Metric for Browser Agent Benchmarks?

Fireworks ran 720 browser-agent tasks on WebVoyager and reported a 22.9% Agent Execution Tax, defined as wasted over productive inference; MiniMax M2.5 cost 2.3x less per successful task than Gemini, while GLM-5 reached 57.1% accuracy and Kimi K2.5 had 0% parse retries across 852 calls.

Why it matters: Fireworks ran 720 browser agent tasks on WebVoyager and found 22.9% of inference calls were wasted, giving buyers a way to price that hidden cost.

r/LocalLLaMA

LLM planner: pick a rig by use case, model, or budget, or pick models for your rig

totosse17 published the LLMRequirements hardware planner with 60+ build configs, 50+ models, 130 cited tokens-per-second sources, 150+ reviewer videos, multi-region prices, idle and active watts, and a public GitHub data repo.

Why it matters: A public dataset compares 60+ local model rigs and 50+ models on measured generation speed, idle and load power, and prices across regions.

r/LocalLLaMA

Tencent Hy-MT2 30B/7B/1.8B

Tencent released Hy-MT2 translation models in 1.8B, 7B, and 30B-A3B sizes, supporting translation across 33 languages; AngelSlim 1.25-bit quantization reduces the 1.8B model’s storage requirement to 440 MB and raises inference speed by 1.5x.

Why it matters: Tencent's Hy-MT2 line covers 1.8B, 7B and 30B-A3B sizes with 33-language translation, and the quantized 1.8B build drops to 440MB of storage.

Xinzhiyuan · WeChat

USTC Papers Study Lifelong Learning for LMMs via Multimodal Knowledge Injection

USTC researchers released MMEVOKE and KORE: MMEVOKE contains 9,422 samples across 159 subcategories, while KORE uses knowledge-tree augmentation and null-space constrained fine-tuning to reduce catastrophic forgetting during multimodal knowledge injection.

Why it matters: A USTC team's MMEVOKE benchmark and KORE fine-tuning method offer reusable evaluation and training ideas for forgetting in continual learning of multimodal models.

Latent Space

OpenAI GPT-next Disproves 80-Year-Old Erdős Planar Unit Distance Problem for Under $1000

OpenAI said an internal general-purpose reasoning model disproved the 1946 Erdős planar unit distance problem by finding a new family of constructions; the reasoning summary reportedly spans about 125 pages, while outside observers speculate the run used under 32 hours or under $1,000.

Why it matters: A general OpenAI reasoning model disproved Erdős's 1946 planar unit distance conjecture, producing a roughly 125-page summary; outside observers estimate under 32 hours of runtime and under $1,000 in cost.

Synced · WeChat

Xie Saining’s Team Releases Second-Generation Representation Autoencoder RAEv2

Xie Saining’s team, Adobe Research, and the Australian National University released RAEv2, which reaches gFID 1.06 after 80 epochs on ImageNet-256 and reduces EPFID@2 from 177 epochs to 35 epochs while keeping compute at 189 GFLOPs.

Why it matters: Xie Saining's team with Adobe Research and ANU released RAEv2, cutting ImageNet-256 convergence from 177 epochs to 35 while holding compute at 189 GFLOPs.

r/LocalLLaMA

HalBench: Custom sycophancy and hallucination benchmark tests 4 frontier models

HalBench tested 4 frontier models on 3,200 false-premise prompts, with Sonnet 4.6 ranking first at a 0.565 mean score and Gemini 3.1 Pro last at 0.339; higher scores mean the model more often named the false premise and pushed back instead of complying.

Why it matters: A developer built HalBench, testing four frontier models with 3,200 false-premise prompts to see whether they go along or push back, and ranked each model's score.

May 20Wednesday

AI HOT (Curated Pool)

Gemini 3.5 Flash launches with stronger performance and speed

Google opened Gemini 3.5 Flash after Google I/O across its products and API; the post says it outperforms Gemini 3.1 Pro on most benchmarks and generates tokens 4x faster than other frontier models.

Why it matters: Gemini 3.5 Flash is now open to product teams and API users, with Google claiming it beats 3.1 Pro on most benchmarks.

AI HOT (Curated Pool)

Google releases Gemini 3.5 Flash with 55 intelligence score

Google released Gemini 3.5 Flash, raising its intelligence score by 9 points to 55, exceeding 280 output tokens per second, and increasing operating cost by 5.5 times versus the previous generation.

Why it matters: Gemini 3.5 Flash trades a higher intelligence score and faster output for running costs 5.5 times the previous generation.

AI HOT (Curated Pool)

Google releases Gemini 3.5 Flash for complex agent workflows

Google introduced Gemini 3.5 Flash at Google I/O for long-running agent workflows; it outscored 3.1 Pro on Terminal-Bench and MCP Atlas, runs up to 4x faster than other frontier models, and reaches up to 12x speed gains in Google Antigravity.

Why it matters: Google's first Gemini 3.5 model, Flash, targets long multi-step agent workflows and outscores the previous 3.1 Pro on Terminal-Bench and MCP Atlas.

AI HOT (Curated Pool)

Google releases Gemini 3.5 Flash with output speed about 4x GPT-5.5

Google introduced Gemini 3.5 Flash at I/O 2026, with output speed reaching 289 tokens per second, about 4x faster than Claude Opus 4.7 and GPT-5.5 xhigh under the cited comparison.

Why it matters: Google's Gemini 3.5 Flash hits 289 output tokens per second, about four times faster than GPT-5.5 xhigh and Claude Opus 4.7.

r/LocalLLaMA

KV cache quantization benchmarks: TurboQuant is overrated, q5 deserves attention, q8 may waste VRAM

Anbeeld benchmarked KV cache quantization for Qwen 3.6 27B on one RTX 3090 at 64k and 128k context, reporting q4_0 tail KLD 32% worse than q5_0 and turbo4 running 17% slower than q4_0 with little memory saving.

Why it matters: Anbeeld benchmarks Qwen 3.6 27B KV cache quantization on an RTX 3090, with numbers like q4_0's tail KLD 32% worse than q5_0.

May 19Tuesday

r/LocalLLaMA

Sapient Intelligence releases HRM-Text 1B: 40B tokens, ~$1k pretrain

Sapient Intelligence released HRM-Text 1B, a 1B-parameter model trained from scratch on 16 GPUs for 1.9 days with 40B tokens and a reported ~$1,000 budget; its self-reported chart shows MATH 56.2 and DROP 82.2, while independent evaluation remains pending.

Why it matters: A 1B model pretrained on 16 GPUs in 1.9 days for about $1,000, beating Llama3.2 3B on MATH and DROP, shows one low-cost path for small models.

AI Chat-Group Daily (群聊日报)

May 18, 2026 Chat Group Daily

The chat group daily says AI21 Labs cut 60% of staff and stopped selling model access, and cites a University of Waterloo paper where GPT-5.4 accuracy dropped from 100% to 23% after false peer-consensus injection; the snippet also mentions Meta layoff talk at 10%, but does not disclose source details or confirmation conditions.

Why it matters: AI21 Labs cut 60% of staff and stopped selling its models on their own, as token prices fell 100x in two years and six independent model firms were absorbed.

QbitAI · WeChat

JD and CAS IIE Publish Three Papers Defining Self-Taught RLVR

JD and CAS IIE released three Self-Taught RLVR papers covering RLSD, NPO, and CoPD; RLSD reports that 200 training steps on Qwen3-VL-8B-Instruct exceed GRPO at 400 steps across 8 benchmarks.

Why it matters: JD and the Institute of Information Engineering at CAS published three papers on Self-Taught RLVR, a route where the model generates its own training signal during reinforcement learning, skipping human labels and external reward models.

AI HOT (Curated Pool)

Qwen3.7 Preview lands on Arena; Alibaba rises to fifth in vision ranking

Alibaba says Qwen3.7-Plus-Preview has landed on Arena and that Alibaba now ranks fifth in vision; the post does not disclose benchmark scores, the number of competing models, or a release timeline for the Qwen3.7 series.

Why it matters: Alibaba put Qwen3.7-Plus-Preview on an arena and claims fifth place on a vision leaderboard, with no scores or launch date.

r/LocalLLaMA

21 GPUs benchmarked running a small TTS model, with 5GB peak VRAM

A Reddit user rented 21 GPUs on vast.ai to benchmark OmniVoice, a small TTS model with about 5GB peak VRAM, using xRT as the audio generation speed metric and averaging 3 voice-cloning runs with reference audio.

Why it matters: A Reddit user benchmarked 21 rented GPUs on a small voice-cloning TTS model, averaging three runs per card, as a buying reference for anyone building speech synthesis.

r/LocalLLaMA

llama.cpp MTP support landed: Qwen3.6 27B reaches 2.44× on Strix Halo

llama.cpp merged MTP speculative decoding in PR #22673; Qwen3.6 27B Q8_0 rose from 7.4 to 18.1 tok/s on Strix Halo, while a dual RTX 3090 Q8_0 setup rose from 25.7 to 55.9 tok/s.

Why it matters: After llama.cpp merged MTP speculative decoding, Qwen3.6 27B measured 7.4 tok/s to 18.1 tok/s on Strix Halo, giving local inference a concrete speedup reference.

May 18Monday

AI HOT (Curated Pool)

The Open Agent Leaderboard

IBM Research published the Open Agent Leaderboard on Hugging Face to evaluate agents across language understanding, tool use, and multi-step reasoning tasks; the post does not disclose dataset size, model scores, or the evaluation date.

Why it matters: IBM Research put an open agent leaderboard on Hugging Face that scores whole systems across six scenario benchmarks and reports quality alongside running cost.

r/LocalLLaMA

I Tested 42 LLMs on Their Willingness to Build the Apocalypse

DystopiaBench tested 42 open and closed models across 36 escalating scenarios and 6 dystopia types, using 3 LLM-as-judge scorers and an average over 3 runs; the post says many models catch obvious dangerous requests but fail when risk is hidden behind dual-use framing and normalization.

Why it matters: The author ran 42 models through DystopiaBench's 36 escalating scenarios with three judge models, and many caved when the harmful ask was wrapped as dual-use work.

r/LocalLLaMA

Qwen 3.6 27B on 24GB VRAM: backend comparisons, quant choice, and settings

The author tested Qwen 3.6 27B on an RTX 3090 24GB and kept ik_llama.cpp with Qwen3.6-27B-MTP-IQ4_KS.gguf; at 156k context with q8_0 KV and MTP, a ~5.9k-token prompt plus 1024-token output reached about 1261 tok/s prefill and 72.9 tok/s decode, while vLLM lacked a clean single-card long-context run.

Why it matters: The author benchmarked several backends and quantization schemes for Qwen 3.6 27B on a single 24GB RTX 3090, settling on ik_llama.cpp with IQ4_KS, as a single-card comparison for local deployment.

r/LocalLLaMA

I trained TIME: short context-triggered thinking on Qwen instead of overthinking

An independent author trained TIME with QLoRA on Qwen3 4B/8B/14B/32B to trigger short mid-response reasoning when context changes; the post says datasets, notebooks, scripts, curriculum, and TIMEBench are public, with 24GB VRAM enough for training up to 14B.

Why it matters: The author trained TIME on the full Qwen3 range from 4B to 32B with QLoRA, triggering short mid-reasoning only when context changes, and released the dataset, scripts and TIMEBench.

AI HOT (Curated Pool)

Open-source tool exposes security risks and detection gaps in AI API relays

api-relay-audit audits AI API relay risks with verifiable three-state decisions and transparent logs, covering AC-1 tool-call rewriting, AC-2 error-response leakage, and context truncation, while the author has published the methodology, comparison results, quick-reference table, and the open-source tool.

Why it matters: The open-source api-relay-audit tool splits relay-station behavior into three reproducible checks: rewriting tool calls, leaking model identity through errors, and truncating context, with methods and comparisons published.

r/LocalLLaMA

Benchmarking vLLM vs SGLang vs llama.cpp on a mixed Blackwell/Ada cluster

The author benchmarked long-context prefill on a 7-GPU mixed Blackwell/Ada cluster; on Qwen3.5-397B-A17B with 75k tokens, vLLM reached 9.8s TTFT and 7,683 t/s, while llama.cpp took 57.2s and 1,319 t/s.

Why it matters: A hands-on test of long-context prefill across seven mixed Blackwell and Ada GPUs, with vLLM hitting 9.8s first-token latency at 75k tokens and 7,683 tok/s prefill, roughly 4 to 6 times llama.cpp.

r/LocalLLaMA

LLMs on Android: Snapdragon 8 Elite MoE Experience

A Reddit user tested MoE LLMs on an Honor Magic 7 Pro with Snapdragon 8 Elite and 24GB RAM; under Q4 quantization, LFM2-24b-a2b reached about 24 tokens/s while Gemma reached about 11 tokens/s, and CPU inference was still faster than NPU or GPU in the reported setup.

Why it matters: A Reddit user's real measurements of MoE models on a Snapdragon 8 Elite phone, where CPU inference beat the NPU and LFM2-24b-a2b ran near 24 tok/s at Q4.

May 17Sunday

r/LocalLLaMA

85 GPU-hours comparing 5 abliteration methods on Qwen3.6-27B

Abliterlitics compared five Qwen3.6-27B abliteration variants against the base model using 85 GPU-hours of benchmarks, HarmBench, KL divergence, and weight forensics; Huihui had the smallest benchmark deltas, Heretic had the lowest KL divergence, and all five variants reached near-complete safety removal.

Why it matters: 85 GPU-hours on Qwen3.6-27B comparing five abliteration methods, with benchmarks, HarmBench safety evals, KL divergence and weight forensics all reported together.

r/LocalLLaMA

DeepSeek V4's 1M Context Window: The Breaking Point

A Reddit user tested DeepSeek V4 on 45k, 180k, and 520k-token codebases and found 150k-250k tokens best for coding work. Past 300k tokens, line-number precision degraded; at 520k, outputs shifted toward architecture summaries and skipped implementation details.

Why it matters: A Reddit user tested DeepSeek V4 on codebases of 45,000, 180,000 and 520,000 tokens, finding coding smoothest between 150,000 and 250,000 tokens and line-number accuracy slipping past 300,000.

Synced · WeChat

AI agents may spend 1,000x more tokens without better results: the hidden bill

Researchers used OpenHands to analyze traces from 8 frontier models on 500 swe-bench-verified tasks, finding that agentic coding reached a 154:1 input-output token ratio and that human difficulty labels correlated weakly with token use at Kendall tau 0.32.

Why it matters: Drawing on OpenHands trajectories across 500 swe-bench-verified tasks, the article reports a 154:1 input-to-output ratio and a Kendall tau of 0.32, finding that a thousandfold gap in token use does not guarantee better results.

r/LocalLLaMA

Same Models Tested Across Strix Halo, RTX 3090, and RTX 5070

C_Coffie published 55 local inference benchmark runs across Strix Halo, RTX 3090, RTX 5070, five backends, and 0.35B to 35B-A3B models; RTX 5070 beats RTX 3090 on models fitting 12GiB, while RTX 3090 leads in the 14–31B band that exceeds 12GiB but fits 24GiB.

Why it matters: A user ran 55 local inference tests across three GPUs and five backends, and reports an RTX 5070 decoding faster than an RTX 3090 within 12GB of VRAM.