Synced · WeChatMay 9, 2026, 10:59 UTC+8

OpenAI's Jiayi Weng: Is the Next AI Training Paradigm Beyond Gradients?

OpenAI翁家翌：梯度之外，下一个AI训练范式有着落了？

OpenAI researcher Jiayi Weng proposes Heuristic Learning: codex gpt-5.4 reached a perfect 864 score on Breakout and generated 342 search trajectories across Atari 57, with updates applied to code, tests, replays, and memory rather than neural-network weights.

Why it matters: HKR-H/K/R all pass: an OpenAI researcher proposes Heuristic Learning with concrete hooks like Breakout 864 and 342 Atari 57 trajectories. This is strong research/commentary signal, not an official model or product release, so it stays in the 78–84 band.

Read the original ↗Export Markdown

Take

Heuristic Learning’s sharp edge is not a comeback for hand-written rules; it moves the learned object from weights into testable software. Breakout went from 387 to the 864 ceiling, and Atari 57 produced 342 coding-agent search trajectories. The edited artifacts were policy code, tests, replays, and memory, so the loop feeds on reproducible failures rather than gradients.

I don’t buy the “next training paradigm” framing yet. Ant at 6000+ and HalfCheetah at a 11836.7 five-run mean show that agents can maintain low-to-mid coupling control systems. Montezuma exposes the wall: one unattended run got 400 points through 86 open-loop macro actions. Long-horizon exploration and perception still belong to neural nets. This smells closer to SWE-agent or AlphaEvolve wired into RL environments than a replacement for Deep RL.

Source: Synced · WeChat · mp.weixin.qq.com
Published: May 9, 2026, 10:59 UTC+8 · May 9, 2026