Reiner Pope: The Math Behind How LLMs Are Trained and Served
Reiner Pope – The math behind how LLMs are trained and served
Dwarkesh interviewed Reiner Pope in a 1-session blackboard lecture on LLM training and serving. The post lists 7 timestamps on batch size, MoE rack layout, pipeline parallelism, KV cache, and API pricing. The key mechanism is cost: without batching, serving economics can be 1,000x worse.
Why it matters: MatX CEO Reiner Pope breaks down the cost structure of large models from training to serving with formulas and chalk, showing batching affects serving economics by a factor of 1000.