The Foundations series, in reading order
33 articles explaining the core ideas of data science and AI from scratch. Each article stands alone, but they were written to build on each other, and the publication dates follow the source material rather than the reading order — so this page is the order that makes sense.
Pick the track that matches what you are stuck on. Within a track, read top to bottom.
Before any of the maths
Three habits that decide whether the rest of it helps you. No prerequisites, and worth reading even if you skip every other track.
- Part 1 The job is mostly cleaning data, and no one warns you What the work actually consists of, versus what the tutorials show.
- Part 2 Always build the dumb baseline first The number every later model has to beat before anyone celebrates.
- Part 3 Start with the simplest model that could possibly work Why the simplest model that could work is the one to try first.
Reasoning under uncertainty
Start here if the statistics never quite clicked. Nothing below assumes anything above it except plain arithmetic.
- Part 4 The Monty Hall problem, and why switching wins Why conditioning on how information arrived changes the answer.
- Part 5 Bayes theorem from first principles The same move, generalised. Builds on Monty Hall.
- Part 6 The central limit theorem, from first principles Why the normal distribution turns up everywhere, and what it does not promise.
- Part 7 P-values: what they say and what they do not What the number means. Easier after the central limit theorem.
- Part 8 Monte Carlo methods: answering hard questions by rolling dice When simulating beats solving.
- Part 9 Entropy, or how to measure surprise Information as surprise — the source of the loss functions used later.
- Part 10 Markov chains, from states to stationary distributions Processes with a state and no memory.
Reading data honestly
What a number can and cannot support. No maths beyond the first track.
- Part 11 Correlation is not causation, but The three explanations for any correlation.
- Part 12 Simpson's paradox from first principles When the aggregate and every subgroup disagree.
- Part 13 Automation and jobs: picking the right unit of analysis Picking the right unit of analysis for a contested question.
Models that learn from tables
The workhorses. Read the bias-variance piece early — everything after it refers back.
- Part 14 Overfitting and the bias-variance tradeoff The one idea the rest of machine learning is organised around.
- Part 15 Logistic regression from first principles The simplest honest classifier. Uses entropy from track one.
- Part 16 Decision trees and random forests, from the split up Splitting, and why averaging many trees works.
- Part 17 Gradient boosting, built one mistake at a time Fitting the errors of the last model. Assumes trees.
- Part 18 K-means clustering, and what it is really doing Structure without labels.
- Part 19 PCA explained from first principles Fewer dimensions, most of the variance.
Neural networks
Strictly sequential — each one assumes the one before it.
- Part 20 What a neural network actually is Weights, layers, activations. Start here.
- Part 21 Gradient descent from first principles How the weights get set.
- Part 22 Backpropagation explained plainly How the gradient is computed cheaply.
- Part 23 Convolution, and how computers learned to see The architecture that made vision work.
- Part 24 What word embeddings really are Turning words into geometry.
Generative models and LLMs
Assumes the neural networks track, particularly embeddings.
- Part 25 Transformers explained from scratch The architecture behind every current language model.
- Part 26 Attention is the whole trick The mechanism inside it. Read the transformer piece first.
- Part 27 What a language model is actually doing What next-token prediction produces.
- Part 28 How a model gets trained, from pretraining to RLHF Base model to assistant, stage by stage.
- Part 29 Why prompting works at all Why the wording changes the output.
- Part 30 Why language models make things up The failure mode, and what actually reduces it.
- Part 31 How diffusion models make images Generation by removing noise.
- Part 32 GANs, and why deepfakes work The older generative idea, and why images stopped being evidence.
- Part 33 Reinforcement learning, and what AlphaGo taught Learning from reward instead of labels.