METR updates time-horizon benchmark with open-ended tasks
What happened
On November 21, MIT Technology Review reported that METR updated its AI task time-horizon report, adding open-ended tasks that require judgment. The earlier evaluation showed the human time needed for tasks frontier models complete successfully doubles every four months, with some capabilities still growing exponentially. The new tests show models do worse on open-ended tasks than on coding tasks. No AI-designed evaluation met METR's standard for research use, and models also scored below humans on philosophical argument. Task completion times in the new report are mostly estimates, and open-ended tasks are few, so the results cannot pin down how fast AI automates R&D.
Written by AI from the coverage · updated 2 days ago
Coverage
Follow the reports to see the story from different sides.
- MIT Technology Review · AIThe most misunderstood graph in AI just got a major update
METR 更新任务时间跨度报告,发现部分 AI 能力继续指数增长,但模型在需要判断的开放式任务上逊于编码任务。原有评估显示,前沿模型能成功完成的任务所对应的人类用时每 4 个月翻倍;新增测试中,没有模型设计的 AI 评估达到 METR 研究可用标准,哲学论证评分也落后于人类。新版报告的任务完成时间多为估计,开放式任务数量有限,难以据此推断 AI 研发自动化的进展速度。
Heat over time
Heat now 3·Comparable peak 10(Nov 21 04:00)·Comparable change over 24 hours -50%
The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.