Inference at Scale
Video Attention in production: KV cache, MQA, GQA and MLA
Paper Efficient Memory Management for Large Language Model Serving with PagedAttention
Included with a plan
This board is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
Lessons that use this board
Inference at Scale. 11 · INFERENCE AT SCALE — WHY AN LLM IS FAST, AND CHEAP
one fact explains almost everything about serving an LLM. see it, and batching, quantization, KV-paging and speculative deco
Saved in this browser only.