Articles
76 articles, newest first — or search the archive if you know what you are after.
A note on the years below: each article in the Foundations series is dated to the explainer video that prompted it, one week after that video was published. Those dates are the source material’s, not this publication’s — which is why the index reaches back further than the site does.
Years
2026 46 articles
- Incremental tables that do not drift Full rebuilds stop being affordable, incremental builds start being wrong. How to pick the incremental key, handle late data, and catch drift before a stakeholder does.
- Your model started dying the day you deployed it Data drift, concept drift, and delayed labels: a minimal monitoring setup that catches decay before the business does, without buying a platform.
- When you cannot randomise users Interference, marketplaces and features that leak between groups. Cluster and geo randomisation, switchback tests, and the observational fallbacks with their assumptions stated.
- Cohort retention in SQL, without the spreadsheet How to build a retention table that survives contact with real data: cohort definition, a period spine, and the bug that makes retention look like it is improving.
- MAE, RMSE, or MAPE: what your regression metric secretly optimises Each error metric implies a different "best" prediction — median, mean, or something biased low. Choose by the cost of being wrong, not by habit.
- Sizing an experiment before you run it Power analysis without the ritual: choosing the smallest effect worth detecting, what drives the sample size, and why an underpowered test is worse than no test.
- Outliers: diagnose before you delete An outlier is an error, a whale, or a discovery — three different treatments. Robust statistics, sensible detection, and the customers you should not trim away.
- Mutual information for feature selection, and where it misleads It catches relationships correlation misses, including ones you do not want. What it measures, why estimating it is hard, and why a high score can mean a leak.
- From notebook to production without a rewrite Notebooks are where analysis is born, not where it should live. A low-drama path: extract functions, add tests for data logic, parameterise, and schedule a script.
- DBSCAN, and when density beats centroids k-means needs k and assumes round clusters of similar size. Density-based clustering needs neither, and it can say that a point belongs to nothing at all.
- Cross-entropy, log loss and perplexity are the same idea Three names from three fields for one quantity: how surprised your model was. Where each shows up, how to read the units, and what a good value looks like.
- SQL or pandas? Push the heavy lifting to the database A practical rule for splitting work between the warehouse and your laptop: aggregate where the data lives, iterate where your tools live.
- Schema checks: make your pipeline refuse bad data Upstream will change without telling you. Explicit schema and expectation checks at pipeline boundaries turn silent corruption into loud, early failures.
- Partial dependence plots and what they quietly assume PDPs average over the rest of the data, which means they can describe combinations that never occur. ICE curves, correlated features, and how to read both without overclaiming.
- Reproducibility: the checklist nobody follows until it burns them Seeds, pinned environments, data snapshots, and one-command reruns — the practical minimum for results you can regenerate in six months.
- Are your clusters real? k-means always returns clusters, including on noise. How to tell structure from partition: stability, internal indices, and the check most people skip.
- Communicating results: the analysis is not the deliverable The decision is the deliverable. How to structure findings for people who will never read your notebook — answer first, uncertainty in plain language, units they own.
- Feature importance is not explanation Impurity importance, permutation importance, and SHAP each answer a narrower question than people think — and none of them answers "what causes what".
- Seasonality, holidays and the calendar features that earn their place Most time series move on a calendar, not a trend. Which calendar features actually help, why one-hot day-of-week is usually wrong, and the holidays that break every model.
- Class imbalance: move the threshold before you reach for SMOTE Imbalanced classes are usually a thresholding and metrics problem, not a data problem. What actually helps, what mostly does not, how to pick a threshold from the cost of each mistake, and in which order to try things.
- TF-IDF still works, and it should be your first NLP model Before reaching for a transformer: a TF-IDF plus linear classifier baseline trains in seconds, explains itself, and is embarrassingly hard to beat on many text tasks.
- Always build the dumb baseline first Majority class, yesterday-equals-today, one rule of thumb: an hour spent on a deliberately dumb baseline is the best ROI in a data project.
- Backtesting a forecast honestly Rolling-origin evaluation, the baselines a forecast must beat, and the quiet ways a backtest ends up scoring information the model would never have had.
- Your model says 90%. Should anyone believe it? Probability calibration: why good classifiers output bad probabilities, how to see it with a reliability curve, and when Platt scaling or isotonic regression fixes it.
- Reading SHAP values correctly SHAP explains one prediction against a baseline, not the world. What the numbers mean, what the plots hide, and the readings that get a model into trouble.
- Pandas cleaning patterns that scale past the toy example Method chaining, assign, vectorized string ops, coercion audits, and validation asserts — a working style for cleaning code you can still read in six months.
- Bandits are the reinforcement learning you will actually use Full RL rarely survives contact with a business problem. Multi-armed bandits do: the same explore-exploit trade, without the state, the simulator or the reward engineering.
- Encoding categorical variables without shooting yourself One-hot, ordinal, target encoding, and the high-cardinality problem — which encoding to use when, and the leakage trap inside target encoding.
- Gradient boosting is still the default for tabular data Why LightGBM/XGBoost keep beating neural networks on tables, the five hyperparameters that matter, and the standard mistakes — including early stopping done wrong.
- Missing data is information, not an inconvenience Why dropna() is a modelling decision in disguise, the three mechanisms of missingness, and imputation patterns that do not bury the signal.
- Time series forecasting: the five classic self-deceptions Random splits, ignored baselines, leaking features, log-space metrics, and one-step myopia — the mistakes that make forecasts look better than they are.
- Embeddings, explained for people who ship things What embedding vectors actually are, what they are good for besides chatbots, and the operational details — normalisation, chunking, drift — that tutorials skip.
- Your A/B test is probably lying to you Peeking, underpowered tests, and twenty metrics with one winner: the three failure modes that produce most false experiment wins, and the working defenses.
- The bootstrap: uncertainty for people who hate formulas Resample your data with replacement, recompute the statistic, read the spread — honest confidence intervals for medians, ratios, AUCs, and anything else.
- Cross-validation without fooling yourself K-fold is the easy part. Grouped data, temporal data, and the quiet ways a validation scheme overstates how good your model really is.
- Accuracy is almost never the metric you want On imbalanced problems, a model that does nothing scores 99%. Precision, recall, PR curves, and how to pick a metric that matches the cost of being wrong.
- Data leakage is why your model looks too good If your validation score seems miraculous, the most likely explanation is that the answer leaked into the features. The common leaks, two worked through end to end, and how to audit for them.
- The four SQL window functions that do 90% of the work ROW_NUMBER, LAG, SUM OVER, and moving averages — with the patterns for deduplication, sessionisation, and running totals you will reuse for years, plus FILTER, QUALIFY, ROWS versus RANGE, and what a window actually costs.
- Simpson's paradox is hiding in your dashboard Aggregates can reverse when you split by segment — a mix-shift illusion that routinely misleads metric reviews. How to spot it and which number to trust.
- Feature engineering that survives contact with production A feature that boosts your offline metrics but cannot be computed at prediction time is worse than useless. How to build features that keep working after deployment.
- When it is not a data problem Some requests do not need a model, a dashboard, or you. Recognising them early is the highest-leverage thing a data person does, and the least rewarded.
- Start with the simplest model that could possibly work Linear regression is not beneath you. Reaching for deep learning first is usually a mistake — here is the discipline that replaces it, and when to escalate.
- Choosing the unit of analysis Per user, per session, per order, per day. The choice quietly decides what your numbers mean, and getting it wrong invalidates the analysis before it starts.
- How to sample data without lying to yourself head(1000) is not a sample. Random, stratified, and entity-level sampling in SQL and pandas — and the bias traps in each.
- The job is mostly cleaning data, and no one warns you Courses spend 5% of the time on the 80% of the work. What data cleaning actually involves, why it is the real skill, and how to get good at it deliberately.
- Turning a business request into a question you can answer Stakeholders ask for dashboards and models. Underneath is a decision someone needs to make. How to find it before you build the wrong thing.
2025 2 articles
- Markov chains, from states to stationary distributions What a Markov chain is, why the memoryless assumption is less crazy than it sounds, how a chain settles into a long-run distribution, and where you meet one in real work.
- How a model gets trained, from pretraining to RLHF The three stages that turn raw text into a chat assistant, what each one actually changes about the model, and why the base model and the assistant are different things.
2024 3 articles
- What a language model is actually doing A first-principles look at large language models: next-token prediction, why that simple goal produces apparent reasoning, and what temperature and sampling really control.
- Attention is the whole trick Queries, keys and values in plain language: how a word picks up meaning from its neighbours, and why one matrix multiplication solved the long-range dependency problem.
- Transformers explained from scratch Tokens, embeddings and a stack of identical blocks — how a transformer turns next-word prediction into a training objective, and why it trains in parallel where RNNs could not.
2023 4 articles
- Why prompting works at all Prompting is conditioning, not magic words. Understanding what the prompt does to the probability distribution explains which techniques help and which are folklore.
- Entropy, or how to measure surprise Information as surprise, why bits fall out of a logarithm, and the straight line from there to the cross-entropy loss you train with and the splits a decision tree picks.
- Why language models make things up Hallucination is not a bug in the usual sense. It follows from what the model is trained to do, and only some of the popular fixes address the actual cause.
- The central limit theorem, from first principles Why sums of many small independent effects turn into a bell curve, what the theorem promises about sample means, where it fails, and why "n > 30" is folklore.
2022 2 articles
- Convolution, and how computers learned to see What a convolution actually does to an image, why sliding a tiny grid of numbers finds edges, and why learning those numbers instead of designing them changed computer vision.
- How diffusion models make images The idea behind AI image generators, explained plainly: teach a model to remove noise from a picture, and running it in reverse from pure static gives you a picture.
2020 2 articles
- Monte Carlo methods: answering hard questions by rolling dice When the maths is too hard, sample instead. What Monte Carlo is from first principles, why error falls as one over root n, and where it earns its keep in real work.
- Reinforcement learning, and what AlphaGo taught Learning from consequences instead of answers: agent, reward, and the credit assignment problem — plus an honest account of why reinforcement learning is rare in normal data work.
2019 3 articles
- Bayes theorem from first principles Bayes as areas and counts rather than a formula: the medical-test example worked properly, why base rates dominate, and where the idea earns its keep in practice.
- What word embeddings really are Why representing words as IDs fails, how counting the company a word keeps turns language into geometry, and what the famous king-minus-man analogy does and does not prove.
- Gradient boosting, built one mistake at a time How boosting works from scratch: fit a model, fit the next one to what the first got wrong, then repeat. Why that is not bagging, and what the learning rate really buys.
2018 6 articles
- Overfitting and the bias-variance tradeoff What overfitting really is, what bias and variance each mean in concrete terms, how the tradeoff shows up in model choice, and where the classic curve stops holding.
- K-means clustering, and what it is really doing The assign-and-update loop from scratch, what k-means actually minimises, why it only finds round clusters of similar size, and how to choose k without fooling yourself.
- GANs, and why deepfakes work How training two networks against each other produces convincing fake images, why that setup is famously unstable, and what it sensibly changes about trusting a video.
- PCA explained from first principles What a principal component actually is, why scaling comes first, how to read a scree plot, and the two things people wrongly expect PCA to do for them.
- Logistic regression from first principles Why a straight line cannot predict probabilities, what the log-odds link fixes, how to read a coefficient as an odds ratio, and why the loss is cross-entropy.
- Decision trees and random forests, from the split up How a tree decides which question to ask, why a single deep tree memorises its training data, and why averaging hundreds of decorrelated trees fixes it.
2017 4 articles
- Backpropagation explained plainly How a network works out which weight was to blame for a wrong answer, why the chain rule is really credit assignment, and why backprop is bookkeeping rather than magic.
- Simpson's paradox from first principles Why an average over groups can point the opposite way to every group inside it, what the Berkeley admissions case actually showed, and how to decide which number to act on.
- Gradient descent from first principles The cost surface, why the gradient points uphill, what the learning rate really controls, and why local minima matter far less in practice than everyone expects.
- What a neural network actually is Layers, weights, biases and activations built up from scratch on the handwritten-digit problem, and why "learning" means nothing more exotic than adjusting a pile of numbers.