Skip to content
Trending storyDeveloping

AI coding agents finish fewer than one in ten tasks in simulated workplace test

1 report1 sourceupdated 4 hours ago

What happened

AI digest

The Decoder reported on November 22 that a new experiment put AI coding agents to work inside a simulated software company, complete with colleagues, a project board and chat logs. Even Claude Opus 5.5 and GPT-6 Astra fully completed fewer than one in ten tasks. The report notes that coding agents already score near the top on isolated tests and standard benchmarks, with SWE-bench and similar benchmarks close to saturation, while Anthropic says Claude writes over 80% of its production code. The experiment shows the gap between benchmark scores and task completion in a simulated workplace.

Written by AI from the coverage · updated 3 hours ago

Coverage

Follow the reports to see the story from different sides.

Nov 22
  1. The Decoder
    AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace

    一项新实验让 AI 编码智能体在模拟软件公司中工作,连 Claude Opus 5.5 和 GPT-6 Astra 完整完成的任务也不足十分之一。模拟环境包含同事、项目看板和聊天记录,而编码智能体在标准基准上的成绩已接近满分。Anthropic 称 Claude 编写了公司超过 80% 的生产代码,SWE-bench 等标准基准也已接近饱和。

Heat over time

Heat now 9·Comparable peak 10(Nov 23 00:00)·Comparable change over 24 hours –

02.557.510Nov 2300:00Nov 2301:00Nov 2301:00Nov 2302:00

The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.