Safety & alignment · page 30

Jul 17, 2024Wednesday

OpenAI NewsJul 17, 18:00

Prover-Verifier Games improve legibility of language model outputs

OpenAI trained GPT-4-family prover-verifier games so stronger models write solutions weaker models can verify; under time-limited human review, correctness-only optimization led to nearly 2x more evaluation errors. The post says the large and small models differ by about 3 orders of magnitude in pretraining compute, and checkability training recovers about half the performance gain of correctness-only optimization; the full experimental numbers are not fully disclosed in the provided text.

Why it matters: This is a substantive OpenAI research release with HKR-H/K/R all present: novel setup, clear mechanism, and strong relevance to scalable oversight. The excerpt confirms the method and the human-evaluation effect, but not the full experimental tables, so it fits the 78–84 band, نه

Take

OpenAI’s useful admission here is that optimizing for correct answers can make reasoning hostile to review. In their GPT-4-family prover setup, correctness-only optimization made time-limited human evaluators commit nearly twice as many errors. Training the prover to write chains a weaker verifier can check recovered about half the performance gain from accuracy-only tuning.

I buy the direction because legibility becomes a training target, not a post-hoc “please explain” wrapper. The setup also has the right asymmetry: the large and small models differ by roughly three orders of magnitude in pretraining compute, which resembles strong systems writing for weaker auditors. The caveat is ugly: the provided article cuts off before the full experimental tables, so that “half the gain” claim is not yet enough for product decisions.

May 28, 2024Tuesday

OpenAI NewsMay 28, 11:00

OpenAI Board Forms Safety and Security Committee

OpenAI's board formed a Safety and Security Committee; that action is the only confirmed fact so far. The source provides only a title, and the post does not disclose members, authority, reporting lines, or timing. Watch governance power, not the committee name.

Why it matters: This is an official board-level OpenAI governance move with HKR-H and HKR-R. It stays in the low featured band because HKR-K is weak: the post confirms the committee exists, but gives no members, remit, reporting line, or effective date.

Take

OpenAI is moving safety into a board process, but the independence story is thin. The committee is led by Bret Taylor, Adam D’Angelo, Nicole Seligman, and Sam Altman, and its first job is to deliver recommendations in 90 days. The same post says OpenAI has begun training its next frontier model, so the timing looks like pre-release governance scaffolding.

The technical bench is real: Jakub Pachocki, Lilian Weng, John Schulman, Aleksander Madry, and Matt Knight are named. The weak word is “recommendations.” The post does not disclose veto power, authority to pause training, external audit rights, or a reporting path outside OpenAI management. After the 2023 board fight, a committee title does not answer the only hard governance question: who can tell the product and compute machine to stop.

Latest picks

Jul 17, 2024Wednesday

Prover-Verifier Games improve legibility of language model outputs

May 28, 2024Tuesday

OpenAI Board Forms Safety and Security Committee