Skip to content
Hacker News front page

When code is correct but sloppy: measuring LLM-generated bloat

If coding is solved, what now?: Measuring the sloppiness of code

Sebastian at Earendil applied SlopCodeBench metrics to measure AI-generated code bloat. Agent code averaged 0.33 verbosity vs. 0.15 for human repos, and 0.68 erosion vs. 0.31. In multi-round, context-cleared iterations, even SOTA models hit 0% strict pass rate—bad decisions compound. The simplest effective metric is LOC change, but it breaks under Goodhart's law. The post does not spell out which directions he plans to explore next.

Why it matters: Earendil's post quantifies AI code bloat with two novel metrics—verbosity and erosion—using their SlopCodeBench. Concrete data, fresh angle. Downside: it's a single blog post, not peer-reviewed, and the benchmark isn't open-sourced, so reproducibility is unclear. But the topic...

Read the original ↗Export Markdown