Skip to content
Synced · WeChat

AI agents may spend 1,000x more tokens without better results: the hidden bill

花了1000倍的token,效果可能却没有更好:AI Agent的“隐性账单”长什么样

Researchers used OpenHands to analyze traces from 8 frontier models on 500 swe-bench-verified tasks, finding that agentic coding reached a 154:1 input-output token ratio and that human difficulty labels correlated weakly with token use at Kendall tau 0.32.

Why it matters: Drawing on OpenHands trajectories across 500 swe-bench-verified tasks, the article reports a 154:1 input-to-output ratio and a Kendall tau of 0.32, finding that a thousandfold gap in token use does not guarantee better results.

Read the original ↗Export Markdown