WPS benchmark finds AI models overlook women, peace and security issues
What happened
On November 13, Hacker News' front page covered results from Our Secure Future's Women, Peace and Security (WPS) benchmark. Run through Weval, it scored 30 scenarios against 11 criteria. The frontier models tested knew WPS principles but routinely dropped the gender dimension when prompts were sparse. Average performance was 0.82 with prompting and 0.16 in sparse settings. Knowing the principles does not mean models raise them unprompted.
Written by AI from the coverage · updated 4 days ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pageAI Models Know What Women, Peace and Security Is. They Just Don't Bring It Up
Our Secure Future(OSF)的 WPS 基准发现,受测前沿模型了解女性、和平与安全原则,却在缺少明确提示的情境中普遍遗漏性别维度。基准通过 Weval 评估 30 个场景、11 项标准,前沿模型群体在带提示情境中的平均表现为 0.82,在稀疏情境中降至 0.16。
Heat over time
Heat now 3·Comparable peak 10(Nov 13 03:00)·Comparable change over 24 hours -51%
The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.