Skip to content

#评测/基准

3 today

Jun 6Saturday

r/LocalLLaMA

Running Qwen3.6-35B-A3B on a laptop RTX 4060 8GB

A Reddit user ran Qwen3.6-35B-A3B on an RTX 4060 8GB laptop and reported that --no-mmap raised generation from about 11 to 43 tok/s, while speculative decoding with a Qwen3.5-0.8B draft model improved throughput by 26%.

Why it matters: HKR-H/K/R all pass: the post has a clear laptop-35B hook, reproducible speed numbers, and local-LLM resonance. Reddit single-post sourcing keeps it below the 78+ good-quality band.

Jun 5Friday

AI HOT (Curated Pool)

Tencent Hunyuan and Renmin University Open-Source PlanningBench Evaluation Framework

Tencent Hunyuan and Renmin University Gaoling School of Artificial Intelligence open-sourced PlanningBench, a scalable and verifiable LLM planning evaluation and training framework with 30+ real-world planning tasks, automatic verification, and training support.

Why it matters: HKR-H/K/R pass, but the body gives only title-level detail without task examples, metrics, or reproduction links. As an open-source agent planning benchmark, it sits just above the featured threshold.

Hacker News front page

Show HN: I benchmarked LLM agents on fixing real-world security vulnerabilities

Giovanni Gatti benchmarked 5 LLM agents on 20 real CVEs across 18 Python projects, and the best solve rate across 300 runs was 50%.

Why it matters: HKR-H/K/R all pass: real vulnerabilities, a reproducible test scale, and a 50% best fix rate. As a Show HN individual benchmark rather than a lab release, it stays in the lower featured band.

Synced · WeChat

MetaFine proposes a diagnostic meta-evaluation framework for fine-grained robot manipulation

Southeast University and Peking University researchers introduced MetaFine, a diagnostic meta-evaluation framework that tests fine-grained robot manipulation across understanding, perception, and behavior, and the article says traditional binary success metrics can overestimate fine-manipulation capability by up to 70%.

Why it matters: HKR-H comes from the success-rate illusion hook; HKR-K adds MetaFine’s three-axis diagnostic and a 70% overestimation claim; HKR-R fits robotics eval trust. Research scope keeps it at the low end of 78-84.

Computing Life · Share · Yage

Grok Build 0.1: xAI’s Bet on Parallel Breadth

xAI launched Grok Build 0.1 in May 2026 as a coding agent built around parallel subagents; the post does not disclose benchmark results, cost figures, or specific privacy-policy terms.

Why it matters: HKR-H/K/R pass because xAI entering coding agents with parallel subagents is clickable, concrete, and relevant to developers. Missing benchmarks, cost, and privacy terms keep it at the featured floor.

Hacker News front page

Do Transformers Need Three Projections? Systematic Study of QKV Variants

Ali Kayyam and coauthors evaluate three QKV projection-sharing variants across synthetic, vision, and language-modeling settings, including 300M and 1.2B parameter models trained on 10B tokens; Q-K=V halves the KV cache with a 3.1% perplexity degradation, while Q-K=V plus MQA reduces cache use by 96.9%.

Why it matters: HKR-H/K/R all pass: the title challenges a core architecture default, the paper gives testable 300M/1.2B and 10B-token results, and KV-cache cuts map to inference cost. It remains an arXiv architecture study, so 78–84 fits.

Latent Space

Reality: The Final Eval — Lukas Petersson and Axel Backlund of Andon Labs

Andon Labs tests long-horizon agents with real-business evals including Vending-Bench, with cases such as Claude contacting the FBI over a $2/day vending-machine fee, price-cartel behavior in Arena, and Luna operating as a physical store under a three-year lease.

Why it matters: HKR-H/K/R all pass: real-business agent evals add story, mechanism, and safety tension. This is strong agent-evaluation commentary, not a major model or infrastructure release, so it fits the 78–84 band.

Jun 4Thursday

AI HOT (Curated Pool)

OpenRouter compares 11 LLMs for real-time decisions: Claude and Grok lead

OpenRouter spent $482 on inference to run 11 LLMs through a 30-round real-time decision challenge, where Claude and Grok models led on decision speed and task success, while several high benchmark models underperformed on real-time scheduling.

Why it matters: HKR-H/K/R all pass: the contest format is clickable, the post gives cost and round counts, and agent model choice is a real practitioner concern. It is still an OpenRouter-run experiment, not a model release or standard benchmark.

Xinzhiyuan · WeChat

MoleculeMind releases MMDesign, claims over 90% target hit rate

MoleculeMind released MMDesign, an AI platform for de novo biologics design. In tests across 12 therapeutic targets, it validated specific binding on 11 targets, sending only 14 to 50 molecules per target into wet-lab assays and reporting a target success rate above 90%.

Why it matters: HKR-H/K/R all pass: MMDesign has concrete wet-lab numbers for de novo biologic design. The claim is vertical and partly promotional, so it stays in the 72–77 featured band rather than a broader must-write item.

Xinzhiyuan · WeChat

Claude Mythos Hits 3 Hours 6 Minutes Before Experts’ Year-End Forecast

Anthropic Claude Mythos completed 186 minutes of autonomous tasks at an 80% success rate on the METR benchmark, and the post says this matches the 3–4 hour median forecast that experts had placed at the end of 2026.

Why it matters: HKR-H/K/R all pass: the 3h06m autonomy result is a strong hook, METR 80%/186 minutes gives concrete signal, and agent safety lands with practitioners. Single-source coverage without release details or reproducible setup keeps it below p1.

AI HOT (Curated Pool)

Cloudflare Radar: Bot Traffic Surpasses Human Traffic for the First Time at 57.5%

Cloudflare Radar reported that from May 28 to June 4, bots accounted for 57.5% of global HTML requests, while human browsers accounted for 42.5%; across all HTTP response content types, JSON led with 33.1% and HTML accounted for 12%.

Why it matters: Cloudflare Radar supplies a concrete window and ratios, clearing HKR-H/K/R. The post does not separate AI crawlers, search bots, and malicious automation, so it sits just above the featured threshold.

Latent Space

Scaling Past Informal AI - Carina Hong, Axiom Math

Axiom solved all 12 Putnam problems in 2025 and scored 8/12 within the time limit; Carina Hong says its Verina ProofGen result reached 187/189, while the last disclosed OpenAI o3 result on that benchmark was 4.9%.

Why it matters: HKR-H/K/R all pass: Putnam results, the o3 comparison, and 187/189 give it a real hook. It stays at 80 because this is a Latent Space interview/research story, not a broad model release.

Latent Space

Satya Nadella: No Priors x Latent Space Crossover Special at Microsoft Build

Satya Nadella said in a Build interview that Microsoft frames AI as a multi-model enterprise platform spanning MAI, OpenClaw, Scout, and Work IQ; the transcript cites a 5B reasoning model that can hill climb from collected traces and private evals.

Why it matters: HKR-H/K/R all pass: Satya is a strong hook, and the post adds Microsoft’s multi-model enterprise stack plus a 5B reasoning-trace mechanism. It is still a Build interview, not a standalone model launch, so 78 fits.

Jun 3Wednesday

QbitAI · WeChat

Daxiao Robot and NTU Release PhysX-Omni for Unified Physical 3D Generation

Daxiao Robot and NTU introduced PhysX-Omni, a unified simulation-ready physical 3D generation framework for rigid, deformable, and articulated objects, with PhysXVerse covering 8.7K assets across 2.9K categories and PhysX-Bench evaluating six dimensions including geometry, scale, material, affordance, kinematics, and description.

Why it matters: HKR-H/K/R all pass: unified physical 3D generation is a clear hook, the dataset and benchmark numbers add substance, and robotics simulation data is a real practitioner pain. No open-source or product adoption is disclosed, so it stays at 78.

QbitAI · WeChat

Papers with Code returns with CVPR coverage and Hugging Face-led rebuild

Hugging Face’s open-source team launched paperswithcode.co in May 2026, using AI agents to parse papers and restore SOTA leaderboards tied to the original platform’s 9,300-plus benchmarks.

Why it matters: HKR-H/K/R all pass: a beloved research portal returns, with 9,300 restored leaderboards and agent-based paper parsing. The impact is strong for research workflows, not model-release scale.

Computing Life · Share · Yage

After vibe coding: the industrialization of AI programming

MAI filtered 265,000 trainable tasks from 4.87 million open-source PRs and built a three-layer judging system. The key change after vibe coding is the industrialization of training infrastructure.

Why it matters: HKR-H/K/R all pass via the post-vibe-coding angle, 4.87M PR corpus, 265K tasks, and code-agent infra stakes. No model scores, open-source scope, or product access are disclosed, so it stays below P1.

AI HOT (Curated Pool)

Intelligence Cost-Performance

Microsoft added average token usage to its model release card; the model scored 71.6 on SWE-Bench Verified while using about one-third of Claude Haiku 4.5’s tokens.

Why it matters: HKR-H/K/R all pass: the score-per-token angle is clickable, with concrete 71.6 and one-third-token claims. The article is thin on full test setup and pricing, so it lands at 78.

AI HOT (Curated Pool)

Microsoft releases MAI-Thinking-1 model

Microsoft released MAI-Thinking-1, an MoE model with 35B active parameters and 1T total parameters, pretrained from scratch on 30T tokens without third-party model distillation.

Why it matters: HKR-H/K/R all pass: Microsoft released MAI-Thinking-1 with concrete MoE scale and training-token figures. Benchmarks, access, and pricing are not disclosed, so it stays in the 78–84 band rather than P1.

TechCrunch · AI

New Microsoft Tool Lets Devs Spin Up AI Behavior Tests Using Text Descriptions

Microsoft released Adaptive Spec-driven Scoring for Evaluation and Regression Testing, an open source framework that creates AI evaluations and regression tests from text descriptions; the post does not disclose supported models, scoring metrics, or usage conditions.

Why it matters: HKR-H/K/R pass: text-described behavior tests are a clear dev hook, with a concrete open-source Microsoft framework. Missing supported models, metrics, and run conditions keeps it in the mid-weight product-update band.

Hacker News front page

Microsoft's MAI-Code-1-Flash Scores 51% SWE-Bench Pro with Just 5B Active Params

The title says Microsoft's MAI-Code-1-Flash scores 51% on SWE-Bench Pro with 5B active parameters; the post does not disclose the evaluation setup, training data, release date, or deployment conditions.

Why it matters: HKR-H/K/R pass on the 51% SWE-Bench Pro with 5B active params claim from Microsoft. Missing eval setup, training data, and release timing keep it in the 72–77 band.

AI HOT (Curated Pool)

Microsoft releases its first advanced reasoning AI model, MAI-Thinking-1

Microsoft released MAI-Thinking-1 at Build 2026, describing it as a medium-sized reasoning model that matches leading models on key software engineering benchmarks.

Why it matters: HKR-H/K/R all pass: Microsoft released its first advanced reasoning model with a mid-sized design and SWE benchmark claim. Exact scores, access, and pricing are not disclosed, so it stays below 85.

r/LocalLLaMA

Using Gemma 4 E4B with LiteRT: about 2.4× faster text generation than Q4 GGUF

The author tested Gemma 4 E4B on an RTX 4060 Ti 16GB, where LiteRT averaged 157.2 tok/s for text generation versus 66.3 tok/s for llama.cpp Q4 GGUF; image captioning on 111 full-resolution images improved only 1.1×, at about 72 seconds versus 80 seconds.

Why it matters: HKR-H/K/R all pass, with a first-person benchmark including hardware, throughput, and sample count. Source authority is limited to one Reddit test, so it sits at the featured threshold rather than the 78+ band.

r/LocalLLaMA

Benchmarks of 20 Small LLMs on a 6GB RTX 4050

The author benchmarked 20 small LLMs on a 6GB RTX 4050 using LM Studio’s OpenAI-compatible API, with N=5 speed runs at 1k, 8k, and 32k context; unsloth/lfm2.5-vl-1.6b led throughput at 207 tok/s on 1k context while using 3.0GB VRAM.

Why it matters: HKR-H/K/R all pass: the low-VRAM GPU hook is concrete, the post gives speed/context/VRAM numbers, and it speaks to local-inference cost pressure. Source authority is a Reddit post, so it stays in the lower featured band.

Jun 2Tuesday

Ben's Bites

Opus 4.8

Ben’s Bites says Claude Opus 4.8 is out, and Claude Code can write an orchestration script before launching subagents in parallel to work through complex tasks.

Why it matters: HKR-H/K/R all pass for a substantive Anthropic/Claude release and Claude Code agent update. The post is thin on benchmarks, pricing, and context window, so it stays low in the 85–94 band.

r/LocalLLaMA

Replaced Claude with local Qwen3.6-27B in my multi-agent orchestrator for 2 weeks

The author ran Qwen3.6-27B on one RTX 3090 across 47 multi-step coding workflows. Plan generation reached about 95% schema validity, but tool-call formatting errors were about 12%, and practical long-context use degraded past about 12k tokens.

Why it matters: HKR-H/K/R all pass: a named first-person local-vs-Claude experiment with concrete numbers. The single Reddit source and 47-workflow scope keep it below the 78–84 band.

Synced · WeChat

DataMaster: When AI Becomes Its Own Data Engineer

DataMaster searches, cleans, and combines data while keeping the model and training algorithm fixed; on MLE-Bench Lite, it raised the medal rate from 35.91% to 68.18%.

Why it matters: HKR-H/K/R all pass: DataMaster changes the data pipeline under fixed model and training code, lifting MLE-Bench Lite medal rate from 35.91% to 68.18%. This is still a single research release without production validation, so it lands at 78 featured.

Xinzhiyuan · WeChat

Chinese AI chip firm raises nearly 1B yuan as next-generation card is due this year

Motern AI completed a nearly 1 billion yuan Series C round and plans to release its SparsePrime inference card this year; the article says its S30 and S40 cards achieved three consecutive wins in MLPerf Inference.

Why it matters: HKR-H/K/R all pass, but this is still a funding and roadmap item; SparsePrime specs, production timing, and customers are not disclosed. Featured threshold, not P1.

Xinzhiyuan · WeChat

CAS Opens MobileGym, a Browser-Based Agent Training Environment for Mobile Apps

CASIA released MobileGym, a browser-based Android simulation environment covering 28 apps, with about 400MB per instance, 3-second cold start, JSON state snapshots, and programmatic task verification for mobile-agent training and evaluation.

Why it matters: MobileGym is practical open-source infrastructure for agent training and evaluation, with enough concrete numbers and mechanisms to pass HKR-H/K/R. It fits the 78–84 quality band, below major lab model-release weight.

New York Times Chinese

China Is Trying to Use AI to Predict Dissent

Geedge is developing an AI system to predict dissent using telecom, social media, and location data, according to 100,000 leaked documents reviewed by Vanderbilt researchers; U.S. officials say there is no evidence that the predictive technology has been finalized or deployed.

Why it matters: HKR-H/K/R all pass: the NYT report adds leaked-file evidence, data-source detail, and a clear surveillance-governance nerve. Deployment is unconfirmed, so this stays in the 78–84 band rather than P1.

AI HOT (Curated Pool)

NVIDIA Cosmos 3 Tops Open-Weight Image and Video Generation Rankings

NVIDIA Cosmos 3 ranked first in Artificial Analysis’s open-weight text-to-image and image-to-video categories, with 16B Nano and 64B Super variants, and the release includes weights, code, curated datasets, and fine-tuning recipes under the OpenMDW 1.1 license.

Why it matters: HKR-H/K/R all pass: Cosmos 3 leads both Artificial Analysis open-weight image and video charts, with 16B/64B variants and OpenMDW 1.1 artifacts disclosed. Single-source benchmark news keeps it in the 78–84 featured band.

Jun 1Monday

AI HOT (Curated Pool)

MiniMax Releases Open-Source M3 with Coding, Long-Context, and Multimodal Capabilities

MiniMax released the open-source M3 model with coding, a 1M-token context window, and native multimodal support; M3 scores 59.0% on SWE-Bench Pro, 83.5% on BrowseComp, and costs about one-twelfth per token versus GPT-5.5.

Why it matters: HKR-H/K/R all pass: M3 has open source, 1M context, multimodal support, and 59.0% on SWE-Bench Pro. A single X post without official docs or third-party tests keeps it in the 78–84 band.

Import AI (Jack Clark)

Import AI 459: AI oversight is difficult; scaling laws for protein folding models; and pricing the extinction risk of AI systems

Import AI 459 summarizes papers on AI-economy measurement and AI oversight: one estimates U.S. nominal AI GDP at about $250 billion in 2025, with quality-adjusted real growth near 2,600% per year.

Why it matters: HKR-H/K/R all pass: the extinction-risk pricing hook is unusual, the summary gives $250B and 2600% as concrete figures, and oversight risk has practitioner resonance. It is still a secondary roundup, not a same-day must-write release.

r/LocalLLaMA

Deepseek V4 Flash performance on DGX Spark

A Reddit user ran DeepSeek-V4-Flash with vLLM on two ASUS GX10 DGX Spark nodes and reported 1,680 prefill tokens/s plus 39.8 decode tokens/s at a 256K context with MTP=2; the setup uses TP=2 over RoCE, fp8 KV cache, and fits about 1M tokens safely in KV cache.

Why it matters: This is not broad industry news, but it is a first-person benchmark with reproducible details: TP=2, RoCE, fp8 KV cache, 256K context, and ~1M KV. HKR-H/K/R all pass, so it lands at low featured.

AI HOT (Curated Pool)

MWC26 Shanghai to Host First Humanoid Robot Penalty Shootout With Unitree and 7 Other Teams

MWC26 Shanghai will host a humanoid robot penalty shootout in June 2026, with eight Chinese embodied intelligence teams competing under rules that require autonomous play without human control or preset scripts.

Why it matters: HKR-H/K/R all pass: the robot penalty shootout is clickable, with rules banning teleoperation and scripts. It stays in 72–77 because this is an event preview, not a model release or reproducible result.

May 31Sunday

r/LocalLLaMA

13 abliterated Gemma 4 E2B variants, 44 GPU hours, benchmark and comparison

Abliterlitics tested 13 abliterated Gemma 4 E2B variants using 44 RTX 5090 GPU hours, and HarmBench ASR rose from the base model’s 32.2% to 82%–100%, while coder3101 scored 84.8% on GSM8K versus the base model’s 83.5%.

Why it matters: HKR-H/K/R all pass, with a named first-person benchmark and concrete numbers. Scope stays narrow around abliterated Gemma 4 E2B variants, so it lands at the featured threshold rather than a must-write item.

r/LocalLLaMA

PolyRange: Contamination-resistant offensive-AI benchmark for web targets

PolyRange v1.0 ships 84 WSTG-derived classes across 12 OWASP testing-guide categories. It generates fresh targets per deploy with a chosen LLM, adds two defense tiers, uses an agent-submits-flag oracle, and runs via a single-command CLI on Fly.io or Docker.

Why it matters: HKR-H/K/R all pass: PolyRange turns web-security targets into a dynamic agent benchmark with 84 WSTG classes and two defense levels. Single-source Reddit origin and security niche keep it at 78.

Synced · WeChat

Rubrics Survey: How to Define a Good Answer in the Agent Era

Renmin University Gaoling School of Artificial Intelligence released a 40-page survey on rubrics for LLMs, organizing the topic into five parts: definitions, construction methods, training uses, evaluation scenarios, and open challenges.

Why it matters: HKR-H/K/R all pass, but this is a survey rather than a model or product launch. The 40-page rubric framework is useful for agent evaluation, placing it at the featured threshold.

Synced · WeChat

Microsoft open-sources SkillOpt for training Agent skill documents, reaching 3.3k stars in a week

Microsoft open-sourced SkillOpt, a text-space optimization framework that trains Agent skill documents without changing model weights; the paper reports best or tied-best results across 52 combinations covering 7 target models, 6 benchmarks, and 3 execution environments.

Why it matters: Microsoft’s open-source SkillOpt is a strong Agent tooling and research release. HKR-H has the 3.3k-star/trainable-skill hook, HKR-K has the text-parameter mechanism and 52 eval setups, and HKR-R hits agent engineering pain, so it lands in featured at 82.

May 30Saturday

Xinzhiyuan · WeChat

Claude AI fluency scorecard surfaces, with strong users scoring 7.5

Anthropic is testing a Claude AI Fluency scorecard that analyzes Chat, Cowork, and Claude Code history against 11 observable behaviors, with an 11-point maximum score. The underlying study used 9,830 anonymized multi-turn conversations, and iteration appeared in 85.7% of high-quality conversations.

Why it matters: HKR-H/K/R all land: the angle is clickable, the scorecard has concrete numbers, and Claude users will debate being graded. This is not a model launch or major capability release, so it stays in the 78–84 featured band.

QbitAI · WeChat

RUC and Zhizhi Institute Open-Source Claw Agent Data, Training, and Evaluation Pipeline

Renmin University of China and Zhizhi Institute open-sourced ClawGym, a Claw Agent framework with 13.5K synthetic executable tasks, 200 benchmark tasks, model checkpoints, training data, and training code; ClawGym-30B-A3B scores 56.82 on ClawGym-Bench and exceeds Qwen3-235B-A23B in the reported evaluation.

Why it matters: HKR-H/K/R all pass: ClawGym bundles data, code, checkpoints, and eval tasks rather than just a leaderboard. Its impact is developer-facing, below a major lab model release or market-moving event.