Improving Model Safety Behavior with Rule-Based Rewards
OpenAI said on July 24, 2024 it uses Rule-Based Rewards in the RLHF pipeline to reduce repeated human feedback for safety alignment. The post defines three response types—hard refusal, soft refusal, and comply—and says the method has been part of OpenAI’s safety stack since GPT-4, including GPT-4o mini. The key point is maintainability when policies change; the post excerpt does not disclose quantitative gains.
Why it matters: Replacing part of human feedback with rule-based scores means safety policy changes become a rule edit instead of a fresh labeling round, which changes the cost of iterating on alignment.