Skip to content
Trending storyPast story

WPS benchmark finds AI models overlook women, peace and security issues

1 report1 sourceupdated 4 days ago

What happened

AI digest

On November 13, Hacker News' front page covered results from Our Secure Future's Women, Peace and Security (WPS) benchmark. Run through Weval, it scored 30 scenarios against 11 criteria. The frontier models tested knew WPS principles but routinely dropped the gender dimension when prompts were sparse. Average performance was 0.82 with prompting and 0.16 in sparse settings. Knowing the principles does not mean models raise them unprompted.

Written by AI from the coverage · updated 4 days ago

Coverage

Follow the reports to see the story from different sides.

Nov 13
  1. Hacker News front page
    AI Models Know What Women, Peace and Security Is. They Just Don't Bring It Up

    Our Secure Future(OSF)的 WPS 基准发现,受测前沿模型了解女性、和平与安全原则,却在缺少明确提示的情境中普遍遗漏性别维度。基准通过 Weval 评估 30 个场景、11 项标准,前沿模型群体在带提示情境中的平均表现为 0.82,在稀疏情境中降至 0.16。

Heat over time

Heat now 3·Comparable peak 10(Nov 13 03:00)·Comparable change over 24 hours -51%

02.557.510Nov 1303:00Nov 1318:00Nov 1410:00Nov 1501:00

The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.