Skip to content
OpenAI News

Evaluating fairness in ChatGPT

OpenAI analyzed millions of ChatGPT requests to test whether user names trigger harmful stereotypes, finding an overall rate of about 0.1%. The study used GPT-4o as a privacy-preserving evaluator; its gender-related judgments matched human raters over 90% of the time, while race and ethnicity agreement was lower. The key signal is model drift across versions: GPT-3.5 Turbo showed the highest task-level bias.

Why it matters: OpenAI audits ChatGPT fairness across millions of real conversations, finding answer quality holds across name-implied gender and race, with about 0.1% harmful stereotypes.

Read the original ↗Export Markdown