Skip to content
X · @Yuchenj_UW

GLM-5.1 beat Opus 4.6, GPT-5.4, and Gemini 3.1 Pro on SWE-Bench Pro

Wow, GLM-5.1 beat Opus 4.6, GPT-5.4, and Gemini 3.1 Pro on SWE-Bench Pro (58.4 vs 57.3 / 57.7 / 54.2) as an open-weight…

GLM-5.1 scored 58.4 on SWE-Bench Pro, ahead of Opus 4.6 at 57.3, GPT-5.4 at 57.7, and Gemini 3.1 Pro at 54.2. The post also says it is an MIT-licensed open-weight model; the post does not disclose eval setup, cost, or whether all models were tested under identical conditions. Watch reproducibility, not a single leaderboard snapshot.

Why it matters: GLM-5.1 scores 58.4 on SWE-Bench Pro with open weights, above Opus 4.6, GPT-5.4 and Gemini 3.1 Pro, which the poster reads as roughly six months of open-closed gap left.

Read the original ↗Export Markdown