Benchmarks

Cognition released FrontierCode, a coding benchmark built from 150 tasks by more than 20 open-source maintainers and judged against over 3,000 rules, with Claude Opus 4.8 reaching 13.4% approval in the hardest tier and GPT-5.5 reaching 6.3%.

OpenBMB released the VoxCPM2 technical report, covering a 2B-parameter speech generation model trained on more than 2 million hours of multilingual speech data, with support for 30 languages and 9 Chinese dialects.

Three highlights: a new benchmark, SocioHack, shows RL-trained models are good at exploiting real-world rules like credit card points or school grades, with over 90% precision on historical loopholes. Anthropic reports an 8x increase in merged code in 2026 vs 2021-2024 and says a prosaic form of recursive self-improvement may have begun, though no paradigm-shifting ideas yet. Separately, RL-trained racing drones from UZH and Google DeepMind beat a champion human pilot in multi-player races at over 22 m/s while cutting collisions by 50%.

The author tested Qwen3.6-27B on an RTX 5090 with DFlash plus KV cache compression, reaching up to 3.26x speedup; q4_0/turbo4 delivered 3.18x speedup with only +0.02% PPL on WikiText-2.

A Reddit user benchmarked qwen3.6-27b-autoround-int4 on 1x3090 versus 2x3090. Narrative decode rose from 53 TPS to 94 TPS, and code decode rose from 62 TPS to 120 TPS, under no NVLink, 8x/8x PCIe, P2P automatically enabled, tensor parallelism set to 2, and different KV-cache settings.

Alibaba’s RTP team released RTPurboV2, replacing 85% of attention heads with SWA and compressing the remaining 15% retrieval heads using low-rank projection, clustering, and dynamic top-p; the adaptation uses about 600 training steps and roughly 1M label tokens, with reported Prefill speedup up to 9.36x.

UC Berkeley and Princeton researchers introduced Log-ICoT for k-parity, reducing training stages from 15 to 4 when k=16, and proved that an L-layer Transformer can internalize chain-of-thought with log₂k curriculum stages under simplified assumptions.

The author compared five coding-agent setups on a Laravel 12 + Livewire Playwright E2E task; Claude Opus 4.7 with 1M context produced 203 tests, while the strongest local OpenCode arm on a 24GB RTX 4090 produced 140 tests, compacted context four times, and needed seven manual nudges.

Anthropic is using Snorkel’s Marlin project to recruit about 1,000 software engineers who review Claude Code outputs for $280 per task, with a workflow covering GitHub repository pull requests, A/B comparisons of two generated code versions, and scoring for correctness, security, reliability, and maintainability.

Lion Rock AI Lab won first place in the ICRA 2026 LeHome Challenge real-robot final, using LiOS to connect training, deployment, trajectory sampling, and Real2Sim teleoperation in one data iteration loop.

PhysX-Omni models rigid, deformable, and articulated objects in one simulation-ready 3D generation framework, while PhysXVerse contains over 8.7K physical 3D assets across more than 2.9K categories.

A Reddit user ran Qwen3.6-35B-A3B on an RTX 4060 8GB laptop and reported that --no-mmap raised generation from about 11 to 43 tok/s, while speculative decoding with a Qwen3.5-0.8B draft model improved throughput by 26%.

Tencent Hunyuan and Renmin University Gaoling School of Artificial Intelligence open-sourced PlanningBench, a scalable and verifiable LLM planning evaluation and training framework with 30+ real-world planning tasks, automatic verification, and training support.

Giovanni Gatti benchmarked 5 LLM agents on 20 real CVEs across 18 Python projects, and the best solve rate across 300 runs was 50%.

Southeast University and Peking University researchers introduced MetaFine, a diagnostic meta-evaluation framework that tests fine-grained robot manipulation across understanding, perception, and behavior, and the article says traditional binary success metrics can overestimate fine-manipulation capability by up to 70%.

xAI launched Grok Build 0.1 in May 2026 as a coding agent built around parallel subagents; the post does not disclose benchmark results, cost figures, or specific privacy-policy terms.

Ali Kayyam and coauthors evaluate three QKV projection-sharing variants across synthetic, vision, and language-modeling settings, including 300M and 1.2B parameter models trained on 10B tokens; Q-K=V halves the KV cache with a 3.1% perplexity degradation, while Q-K=V plus MQA reduces cache use by 96.9%.

Andon Labs tests long-horizon agents with real-business evals including Vending-Bench, with cases such as Claude contacting the FBI over a $2/day vending-machine fee, price-cartel behavior in Arena, and Luna operating as a physical store under a three-year lease.

OpenRouter spent $482 on inference to run 11 LLMs through a 30-round real-time decision challenge, where Claude and Grok models led on decision speed and task success, while several high benchmark models underperformed on real-time scheduling.

MoleculeMind released MMDesign, an AI platform for de novo biologics design. In tests across 12 therapeutic targets, it validated specific binding on 11 targets, sending only 14 to 50 molecules per target into wet-lab assays and reporting a target success rate above 90%.