Skip to content
Dwarkesh Patel podcast

Reiner Pope: The Math Behind How LLMs Are Trained and Served

Reiner Pope – The math behind how LLMs are trained and served

Dwarkesh interviewed Reiner Pope in a 1-session blackboard lecture on LLM training and serving. The post lists 7 timestamps on batch size, MoE rack layout, pipeline parallelism, KV cache, and API pricing. The key mechanism is cost: without batching, serving economics can be 1,000x worse.

Why it matters: MatX CEO Reiner Pope breaks down the cost structure of large models from training to serving with formulas and chalk, showing batching affects serving economics by a factor of 1000.

Read the original ↗Export Markdown