AGI Ranker fixed its ceiling logic—every model dropped 6–15 points
Show HN: I audited my AI leaderboard scale – every score dropped 6-15 points
AGI Ranker aggregates 10 public benchmarks into a single 0–100 score, with 100 marking the AGI threshold. A v2.0.0 audit found that nearly all benchmark ceilings were incorrectly labeled as best-human; only GPQA Diamond has a published human study (0.81). The other nine ceilings are just the benchmark maximum, not human parity. Fixing this dropped every model by 6–15 points. The score is now a mixed scale, not a pure human comparison. They also removed Artificial Analysis's Intelligence Index after its v4.1 redesign caused double-counting with existing benchmarks. The leaderboard relies on published data, doesn't run evals itself, and weights sources by a four-tier credibility system—self-reports get discounted.
Why it matters: An audit of the AGI Ranker leaderboard found only GPQA Diamond has human baseline data, so treating perfect scores as human top performance inflated every model by 6 to 15 points.