Introducing HealthBench

OpenAI introduced HealthBench, a health AI benchmark built with 262 physicians from 60 countries and 5,000 realistic medical conversations. It includes 48,562 physician-written rubric criteria, with GPT-4.1 grading whether each criterion is met across multi-turn, multilingual, clinician and consumer scenarios. The key point for practitioners is the rubric design is physician-grounded, but the scorer is still a model rather than full human review.

Why it matters: Strong HKR-K from concrete benchmark design and released artifacts: 5,000 dialogs, 262 physicians across 60 countries, 48,562 rubrics, paper and code. HKR-H comes from the doctor-written eval design, and HKR-R from the health-safety and model-as-judge debate, so this is featured,