Parsimony open-source benchmark measures net new code
What happened
Parsimony, an open-source benchmark for how much code an AI model adds, was reported on Hacker News on October 14. It counts syntactic elements rather than lines, taking additions minus deletions. It parses the Python files touched by published SWE-bench Verified patches, excludes tests, docs and generated files, and reuses the published pass results instead of rerunning tests. Models are ranked by average net code added on successful tasks. The report notes code volume does not indicate readability or design quality.
Written by AI from the coverage · updated 1 hour ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pageShow HN: Parsimony – an open-source benchmark for how much code AI models add
Parsimony 是衡量 AI 模型新增代码量的开源基准，按语法元素统计新增量减去删除量，不按代码行数计数。它解析 SWE-bench Verified 已发布补丁涉及的 Python 文件，排除测试、文档和生成文件，沿用已发布的通过结果而不重跑测试。模型按成功任务的平均净增量从低到高排序，但代码体量不代表可读性或设计质量。
Heat over time
Not enough continuous observations to draw a trend yet.