Skip to content
MIT Technology Review · AI

AI benchmarks are broken. Here’s what we need instead.

The author proposes HAIC benchmarks that evaluate AI over longer periods inside teams and workflows, not on isolated tasks alone. The post lists four shifts and cites a UK hospital study from 2021–2024 plus an 18-month humanitarian case; the key signal is coordination, error detectability, and downstream effects, not a 98% accuracy headline.

Why it matters: The author reviews AI deployments across UK, US and Asian industries since 2022, argues benchmark scores have drifted from real workflows, and proposes HAIC, an evaluation built around team collaboration and long-horizon tasks.

Read the original ↗Export Markdown