Evaluating fairness in ChatGPT

OpenAI analyzed millions of ChatGPT requests to test whether user names trigger harmful stereotypes, finding an overall rate of about 0.1%. The study used GPT-4o as a privacy-preserving evaluator; its gender-related judgments matched human raters over 90% of the time, while race and ethnicity agreement was lower. The key signal is model drift across versions: GPT-3.5 Turbo showed the highest task-level bias.

Why it matters: OpenAI provides a rare production-scale fairness audit with concrete rates, evaluator agreement, and a model-comparison result, so HKR-K is strong and HKR-R clears on trust and safety. This is a substantive research release, not a model launch or major product shift, so it lands