Skip to content
Hacker News front page

Pantograph pretrains on internet video to build a goal-conditioned Minecraft agent

A General Goal-Conditioned Minecraft Model

Pantograph released Pan, a 4B-parameter Minecraft model that fights mobs, explores, and builds structures. It pretrains on internet-scale video using hindsight relabeling—later frames become the goal for earlier frames, so no hand-labeled rewards are needed. At inference, you give it a goal image and it acts, even in unseen environments. The post doesn't disclose training data volume, hardware, or wall-clock time.

Why it matters: Pantograph relabels later video frames as goals after the fact, so a 40B-parameter Pan learns goal-directed behavior from unlabeled video and acts in Minecraft from a single screenshot.

Read the original ↗Export Markdown