Pantograph pretrains on internet video to build a goal-conditioned Minecraft agent
A General Goal-Conditioned Minecraft Model
Pantograph released Pan, a 4B-parameter Minecraft model that fights mobs, explores, and builds structures. It pretrains on internet-scale video using hindsight relabeling—later frames become the goal for earlier frames, so no hand-labeled rewards are needed. At inference, you give it a goal image and it acts, even in unseen environments. The post doesn't disclose training data volume, hardware, or wall-clock time.
Why it matters: Pantograph relabels later video frames as goals after the fact, so a 40B-parameter Pan learns goal-directed behavior from unlabeled video and acts in Minecraft from a single screenshot.