Skip to content
Hacker News front page

Coding Models Are Doing Too Much

The author programmatically corrupts 400 BigCodeBench problems with single-point bugs to test whether coding models over-edit code during fixes. The post defines the minimal fix as exactly reversing the corruption and measures excess changes with token-level Python Levenshtein distance. The provided body does not disclose final results, model rankings, or training gains.

Why it matters: The author plants single-point bugs in 400 BigCodeBench tasks and uses token-level edit distance to measure how much models over-rewrite when fixing bugs, then tries to train them to make minimal edits.

Read the original ↗Export Markdown