Same model, 9 harnesses: cost per pass varies 17× in FrontierHarness Eval

Show HN: FrontierHarness Eval – 9 harness, same model, cost per pass varies 17x

Runta benchmarked 12 harness configs on Kimi K3 with identical cold-start environments across 360 runs. Codex led at 66.7% pass rate and $3.47 per task; Exo Harness was cheapest at $1.05 with 53.3% pass rate; Claude Code hit 63.3% but cost $18.34 per task. Cache hit rate doesn't equal savings—Claude Code had the lowest cache hit rate at 67.8% yet the highest cost per successful task at $0.288. The post doesn't disclose which specific software engineering tasks were used or their difficulty distribution.

Why it matters: 360 cold-start trials, same Kimi K3 model, 12 harness configs, 17x cost spread — the cleanest coding-agent benchmark I've seen. Claude Code at $18.34/task with 63.3% pass rate vs Codex at 66.7%/$3.47 is a sharp contrast. Not scoring higher because it's a single blog post with ...