Most models of change are hard because the future depends on everything that came before. A customer’s next action depends on what they did last week, last month, and the day they signed up. Write that down honestly and you get a model with more parameters than data.
A Markov chain makes one brutal simplification: the future depends on the present only. Where you go next is decided by where you are now, not by how you got there. Everything the history could tell you must already be packed into the current state. That is the whole idea. The rest is bookkeeping.
The simplification sounds too strong to be useful. It usually is not, because you get to choose what “state” means. If last week matters, put last week into the state. The assumption is not a claim about the world; it is a demand on your state definition.
States and transitions
A chain needs two things. A finite list of states the system can be in, and, for each state, the probabilities of where it goes next. Nothing else.
Take a subscription product with four states: trial, active, lapsed, churned. Watch a month of user-months and count how often each move happens. Divide each row by its total and you have transition probabilities:
| from \ to | trial | active | lapsed | churned |
|---|---|---|---|---|
| trial | 0.30 | 0.45 | 0.15 | 0.10 |
| active | 0.00 | 0.88 | 0.09 | 0.03 |
| lapsed | 0.00 | 0.25 | 0.55 | 0.20 |
| churned | 0.00 | 0.00 | 0.00 | 1.00 |
Every row sums to one, because the system has to be somewhere next month. Read row three: a lapsed user has a 25% chance of coming back, a 55% chance of staying lapsed, a 20% chance of leaving for good. Churned is an absorbing state — once in, never out. Real products often have a small return rate from churned, and then it is not absorbing any more.
That table is the model. It is also a picture: draw the states as circles and the probabilities as labelled arrows, and you can follow a user’s possible lives with your finger. Here are the last three rows drawn that way — trial is left out only to keep the picture readable.
The three teal arrows are everything that can happen to an active user in one month, so they have to add to 1. Churned has one arrow, pointing at itself, and that arrow is worth 1 — nobody leaves.
Multiplying the matrix is running time forward
Write the current mix of users as a row vector, say 5% trial, 70% active, 20% lapsed, 5% churned. Multiply it by the matrix and you get next month’s mix. Multiply again and you get the month after. Matrix powers are the forecast.
import numpy as np
P = np.array([[0.30, 0.45, 0.15, 0.10], [0.00, 0.88, 0.09, 0.03], [0.00, 0.25, 0.55, 0.20], [0.00, 0.00, 0.00, 1.00]])
state = np.array([0.05, 0.70, 0.20, 0.05])for _ in range(12): state = state @ Pprint(state.round(3)) # the mix twelve months outTwo lines of linear algebra replace a simulation. That is most of why Markov chains earn their keep: questions that sound like they need agent-by-agent simulation collapse into matrix arithmetic. How long until a lapsed user churns? What fraction of a cohort is still active after a year? Same matrix, different question.
The long run
Keep multiplying and something interesting happens. In many chains the mix stops changing. Feed it in, get the same thing out. That fixed mix is the stationary distribution: the shape the system settles into and holds, whatever it started from.
Two conditions make this work. The chain must be able to reach every state from every other state, eventually, and it must not be trapped in a rigid cycle (state A on even steps, state B on odd steps, forever). Given those, there is exactly one stationary distribution and the chain converges to it. This is the result that makes Markov chains more than a toy.
The chain in the table above does not have an interesting stationary distribution: churned is absorbing, so in the long run everyone is churned, and the honest questions are about how fast rather than where. That is the normal case in retention work. Notice it before you quote a long-run number that just says “everybody leaves eventually”.
Where the stationary distribution is the point: PageRank. Model a web surfer who clicks a random link on the current page, with a small chance of jumping to a random page instead. The stationary distribution of that chain is the importance score of every page. Google’s original ranking was a Markov chain’s long-run behaviour, computed by repeated multiplication on a very large, very sparse matrix.
Where the memoryless assumption breaks
The failure is always the same shape: the state is too thin, and history is still leaking in.
Text is the classic example. A chain over single words produces grammatical fragments and total nonsense across a sentence, because a word is not enough state to carry a thought. Widen the state to pairs or triples of words and the output improves and the parameter count explodes. That trade — richer state, exponentially more transitions to estimate — is what eventually pushed language modelling toward architectures that learn their own compressed state.
The same trap appears in business chains. If a user’s churn risk depends on their tenure, a four-state chain will be wrong, and it will be wrong in a way the transition table cannot show you. The fix is to split states by tenure band, and the price is data: every state needs enough observed transitions to estimate its row. Estimating a row from eleven observed moves is not a model, it is a rumour.
Also check the assumption you did not notice you made: that the matrix itself is constant over time. Pricing changes, seasonality and a redesigned onboarding flow all move the probabilities. A chain fitted on last year’s transitions is a forecast with a stale foundation — and it will fail quietly, since it keeps producing plausible numbers.
What to do with one
Fit the transition matrix from your own event log, look at it, and argue about it with someone who knows the product. Half the value arrives before any forecast: a transition table is a compact, checkable statement of how the system moves, and people spot wrong rows fast. Then use it for the questions matrix powers answer cheaply — expected time to absorption, cohort mix at month twelve, the value of nudging one specific transition by two points.
Veritasium covers the same ground, with a longer historical run-up, in The Strange Math That Predicts (Almost) Anything.
Two limits are worth stating plainly before anyone builds one. The transition table only earns its keep if the states are ones the business already argues about; invent states to make the maths tidy and the argument that follows is about the model, not the product. And every matrix-power answer assumes the chain keeps behaving as it did during the window it was fitted on — which is the exact assumption a product team spends its quarter trying to break.