AI coding agents finish fewer than one in ten tasks in simulated workplace test
What happened
The Decoder reported on November 22 that a new experiment put AI coding agents to work inside a simulated software company, complete with colleagues, a project board and chat logs. Even Claude Opus 5.5 and GPT-6 Astra fully completed fewer than one in ten tasks. The report notes that coding agents already score near the top on isolated tests and standard benchmarks, with SWE-bench and similar benchmarks close to saturation, while Anthropic says Claude writes over 80% of its production code. The experiment shows the gap between benchmark scores and task completion in a simulated workplace.
Written by AI from the coverage · updated 3 hours ago
Coverage
Follow the reports to see the story from different sides.
- The DecoderAI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace
一项新实验让 AI 编码智能体在模拟软件公司中工作,连 Claude Opus 5.5 和 GPT-6 Astra 完整完成的任务也不足十分之一。模拟环境包含同事、项目看板和聊天记录,而编码智能体在标准基准上的成绩已接近满分。Anthropic 称 Claude 编写了公司超过 80% 的生产代码,SWE-bench 等标准基准也已接近饱和。
Heat over time
Heat now 9·Comparable peak 10(Nov 23 00:00)·Comparable change over 24 hours –
The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.