AI agents may spend 1,000x more tokens without better results: the hidden bill
Researchers used OpenHands to analyze traces from 8 frontier models on 500 swe-bench-verified tasks, finding that agentic coding reached a 154:1 input-output token ratio and that human difficulty labels correlated weakly with token use at Kendall tau 0.32.
Why it matters: Drawing on OpenHands trajectories across 500 swe-bench-verified tasks, the article reports a 154:1 input-to-output ratio and a Kendall tau of 0.32, finding that a thousandfold gap in token use does not guarantee better results.