Benchmarking the performance and cost of agent harness engineering
What happened
On November 9, Computing Life reported on controlled experiments by Surge AI and FireFlash on coding agents. Raising a single model's thinking depth, adding test and goal-check gates, and reworking the terminal pipeline all improved results. Surge AI's Compound Engineering cost 8x the baseline and moved the score only from 31.1 to 32.9. Claude Opus 5.5 with higher thinking settings scored 42.4 at 3.5x baseline cost.
Written by AI from the coverage · updated 2 days ago
Coverage
Follow the reports to see the story from different sides.
- Computing Life · Share · YagePickAgent harness techniques tested: which framework designs raise scores and which just raise cost
Controlled experiments from Surge AI and FireFlash show coding agents can improve through deeper single-model reasoning, test and goal-check gates, and a reworked terminal pipeline. In Surge AI's run, Compound Engineering lifted cost to 8x baseline while the score moved only from 31.1 to 32.9. Claude Opus 5.5 with higher thinking settings reached 42.4 at 3.5x baseline cost.
Heat over time
Heat now 3·Comparable peak 10（Nov 10 02:00）·Comparable change over 24 hours -50%
The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.