An LLM trained only on K–5 curriculum hits a hard capability ceiling

What happens when an LLM never sees material beyond fifth grade?

Researchers from MPI-IS, ELLIS Institute Tübingen, and ETH Zürich trained three model sizes (up to 5B) from scratch on an 88B-token corpus filtered to the US K–5 curriculum, with matched unfiltered controls. Scaling, GRPO post-training, and in-context learning all amplified in-scope performance but barely moved out-of-scope results. The paper argues the pretraining filter sets the effective capability ceiling, and post-training elicits rather than teaches new skills. A live 5B chat model and checkpoints are available.

Why it matters: Clean experimental design with a counterintuitive punch—scaling up and adding RL don't break the pretraining data boundary. Direct signal for anyone working on pretraining or data mixing. Not scored higher because it's a research paper, not a product launch, and the audience s...