YOUR COURSE

AI engineering and inference

Work through each chapter, practice the ideas, and finish its challenge.

— XPLoading progress…

9 chapters · 54 lessons · 0 completed

E1Inference basics0/6 lessons complete

Trace a generation request through prefill and decode and distinguish latency, throughput and hardware bottlenecks.

E2The KV cache0/6 lessons complete

Calculate KV-cache memory and explain how layers, KV heads, context length and concurrent sequences affect it.

E3Paged attention0/6 lessons complete

Allocate logical token blocks to physical KV-cache blocks and quantify internal waste and shared-prefix behavior.

E4Scheduling and continuous batching0/6 lessons complete

Trace static and iteration-level schedules and explain chunked prefill, preemption and latency tradeoffs.

E5Prefix caching0/6 lessons complete

Identify reusable prompt prefixes, calculate saved prefill work and distinguish token savings from hit rates.

E6Quantization0/6 lessons complete

Estimate weight memory at different precisions and explain quantization error, scales and calibration.

E7Speculative decoding0/6 lessons complete

Trace draft-and-verify decoding, explain distribution-preserving acceptance, and calculate a simplified speedup.

E8Serving at scale0/6 lessons complete

Compare kernel, parallelism and disaggregated-serving choices using memory, communication and latency constraints.

E9Cost and operations0/6 lessons complete

Calculate cost per token, estimate stable concurrency and define operational metrics tied to user-facing objectives.