samir-vllm tutorial builds an LLM inference engine from scratch in C++17 and CUDA
What happened
A tutorial that teaches LLM inference from first principles appeared on the Hacker News front page on November 9. It builds a small inference engine from scratch in C++17 and CUDA, with more than 6 hours of video and 85 pages of handwritten notes. It already covers the full forward pass for Llama 3.2 1B Instruct: BF16 weight loading, RMSNorm, RoPE, GQA, SwiGLU MLP, KV cache and continuous batching. Context length is fixed at 2,048 tokens and the KV pool defaults to 128 MiB.
Written by AI from the coverage · updated 57 minutes ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pageHours of YouTube and 85 handtyped pages of notes on LLM Inference – Start here
一套从第一性原理讲解 LLM 推理的教程发布，用 C++17 和 CUDA 从零构建小型推理引擎，配套 6 小时以上视频和 85 页手写笔记。项目已支持 Llama 3.2 1B Instruct 的完整前向流程，包括 BF16 权重加载、RMSNorm、RoPE、GQA、SwiGLU MLP、KV cache 与连续批处理，上下文固定 2,048 token，KV 池默认 128 MiB。
Heat over time
Not enough continuous observations to draw a trend yet.