Explorative Modeling: A Third Pretraining Axis That Also Enables End-to-End Generation

Explorative modeling: Train on the best of K guesses

Alexi Gladstone introduces Explorative Modeling (XM): generate K candidates per step, train only on the best. This adds a third pretraining axis beyond data and parameters. More exploration monotonically improves image, video, and language models, with gains growing at scale—7%→36% with more data, 13%→23% with more parameters. XM achieves 6.2× sample efficiency, 4.1× FLOP efficiency, and 47% better parameter efficiency. As an end-to-end generator, XM matches diffusion on control tasks using up to 256× less inference compute. The post does not disclose specific model names or training costs.

Why it matters: Proposes Explorative Modeling as a third pretraining axis with cross-modal experiments and concrete efficiency numbers. Has code and project page, not just theory. Discounted because the author is an individual researcher, not a known lab, and the post is self-reported without...