Skip to content
Hacker News front page

Coding agent harnesses can 2× your cost with no real accuracy gain

HarnessTax: How Much Does the Harness Matter for Coding Agents?

UC Berkeley and Arena researchers tested 7 models across 3 harnesses—Claude Code, Codex CLI, and Pi—on 30 tasks each from SWE-bench Lite and Terminal-Bench 2.0, with 3 repetitions per pair. Harness choice barely moves success rates (±2–5%), but cost can vary up to 5×. Claude Fable 5 hits 97.8% in Claude Code at $1.33, and 96.7% in Pi at $0.67. Pi, a minimal open-source harness with just read, write, edit, and bash, reaches the Pareto frontier on both benchmarks. The post doesn't spell out the exact open-source licenses for Pi and Codex CLI, and doesn't link the full pricing sheet.

Why it matters: A controlled test across 7 models and 3 coding agent harnesses shows the harness barely moves task success while cost for the same model can vary up to 5x.

Read the original ↗Export Markdown