The data black hole at the center of AI

AI中心的数据黑洞

Dwarkesh Patel argues that recent AI gains come from more and better data, not better sample efficiency. RL is essentially synthetic data generation powered by compute. Each skill demands huge volumes of human expert trajectories—writing examples, rubrics, and chain-of-thought—visible in Mercor's hyper-specific job listings. Epoch reports open models trail closed ones by only 4 months; Patel says that's because data is the real driver and can be distilled from public APIs, while training tricks and architecture tweaks can't. He contrasts human and AI sample efficiency: a human sees ~200M tokens in a lifetime, frontier models train on tens to hundreds of trillions—a million-fold gap. A person learns to teleoperate a robot in hours; self-driving models need 3–4 orders of magnitude more data than a teen learning to drive.

Why it matters: Patel reframes RL as 'compute-intensive answer-fishing to generate training data,' with concrete anchors like Mercor labeling specs and Epoch's 4-month open-source lag. Not scored higher because it's an opinion essay rather than a product launch or paper, and some evidence rel...