Build a Production Inference Engine from Scratch

The engines that serve large language models are treated as black boxes. This book opens one. You build a real inference engine a layer at a time: the forward pass, the KV cache, the kernels, the sampler, until a model you loaded yourself generates text on your own machine.

Every listing on these pages is pulled from a real, working repository at a pinned commit, and every command's output was captured from actually running it. Nothing here is typed from memory.