DoGBench releases a benchmark and evaluation for user document generation

On October 2, Hacker News featured DoGBench, a benchmark for how well AI agents generate user documentation. It uses real open-source project tasks. The paper evaluation covers seven systems and tops out at a combined score of 47.3. A separate leaderboard adds three cloud agents, with a top score of 54.8. The two highs come from different evaluation scopes, not from the same systems improving over time.

Written by AI from the coverage · updated 37 minutes ago

Not enough continuous observations to draw a trend yet.