Skip to content
Hacker News front page

An LLM trained only on K–5 curriculum hits a hard capability ceiling

What happens when an LLM never sees material beyond fifth grade?

Researchers from MPI-IS, ELLIS Institute Tübingen, and ETH Zürich trained three model sizes (up to 5B) from scratch on an 88B-token corpus filtered to the US K–5 curriculum, with matched unfiltered controls. Scaling, GRPO post-training, and in-context learning all amplified in-scope performance but barely moved out-of-scope results. The paper argues the pretraining filter sets the effective capability ceiling, and post-training elicits rather than teaches new skills. A live 5B chat model and checkpoints are available.

Why it matters: The paper trains three model sizes from scratch on US K-5 material only, with controls, to test whether scale, RL or in-context learning breaks that data ceiling.

Read the original ↗Export Markdown