START HERE
HOW TO WORK THROUGH THIS
Eleven levels, roughly three months at a few hours a week. Levels 00 to 03 are the foundation — do not rush them.
The one idea that organizes everything
Generating a token requires reading every model weight from memory. On modern hardware, moving those bytes takes far longer than the arithmetic performed on them. So decoding a single sequence leaves the GPU almost entirely idle.
Nearly every technique in this course is an answer to one question: how do we do more useful work per byte moved from memory? Batching, quantization, GQA, speculative decoding, PagedAttention — different attacks on the same problem. Keep this in view and the field stops looking like a pile of tricks.
How each level is built
Every level has the same seven parts, and they are colour-coded so you can navigate without reading:
| marker | part | what to do with it |
|---|---|---|
| Pinky (pink) | The big idea | Read first. Intuition before mechanism. |
| Pellets (peach) | Concepts | The substance. Read in order. |
| Inky (cyan) | Math by hand | Do not skip. Work it on paper before opening the solution. |
| Clyde (orange) | Code lab | Small, runnable, CPU-friendly. Type it, do not read it. |
| The key (yellow) | Papers | Each with a reading frame: what to extract, what to skim. |
| Blinky (red) | Pitfalls | The specific things people get wrong. |
| Power pellet | Checkpoint | Answer out loud before revealing. Honest self-assessment. |
The ghost roles are not arbitrary — each one matches how that ghost actually behaves in the arcade game. Pinky targets four tiles ahead of you, so they are the idea out in front. Inky's target is computed from Blinky's position, so they are the derived arithmetic. Blinky chases you directly, so they are the misconception coming for you.
The method
1. Concept pass. Read the big idea and the concepts straight through. Do not stop to look things up; get the shape first.
2. Math by hand. Work every math lab on paper. This is the non-negotiable part. The arithmetic is where the understanding actually lives, and reading a worked solution feels like learning without being learning. Get a number, then check it.
3. Code lab. Type the code. Run it. Change something and predict what happens before you re-run. The TODO items at the bottom of each lab are the real exercise.
4. Papers. Use the reading frame — most of these papers have a payload of two or three sections and a lot of context you can skim. Read those sections properly.
5. Checkpoint. Answer each question out loud, in full sentences, before revealing the answer. If you cannot, go back. Marking a level cleared is for you, not for anyone else, so be honest about it.
Pacing
| levels | why |
|---|---|
| 00–03 | The foundation. Take your time. Everything later assumes you can trace a forward pass and compute a KV cache without notes. |
| 04 | The analytical core. If you only truly master one level, make it this one — the rest of the course is applied roofline reasoning. |
| 05–07 | Systems and kernels. Faster going, because you now have the framework. |
| 08–10 | The frontier. More reading, less arithmetic. |
At a few hours a week this is roughly three months. There is no benefit to going faster; the checkpoints are honest gates and the material compounds.
What you need
- Python and NumPy. PyTorch for a few labs.
- No GPU required. Every code lab runs on CPU; a couple are more interesting on a free Colab GPU and say so.
- A pen and paper. Genuinely — the math labs do not work typed.
Progress
Marking a level cleared stores it in your browser's local storage. Nothing is sent anywhere, there is no account, and clearing your browser data resets it. The score is cosmetic: 2,200 points a level, 24,200 for all eleven, which is the Pac-Man perfect-game score.
Arrow keys move between levels.
A note on currency
The foundational papers here are stable and will not go out of date. The systems material, hardware numbers and frontier discussion will. Where a claim matters to a decision you are making, verify it — particularly hardware specifications, which change between silicon revisions, and anything about which serving engine currently does what.
The habit worth building is not trusting this site. It is being able to re-derive its numbers yourself, at which point you no longer need it.