A large language model does one thing. Given some text, it produces a
probability for every possible next chunk of text. No second mechanism sits
behind it. “The capital of France is” comes in, and out comes a table:
Paris 0.91, a 0.02, located 0.01, and so on across a vocabulary of
maybe a hundred thousand pieces. Something picks one. The chosen piece is
glued onto the input, and the model runs again.
Drawn to scale, that table is one spike and a very long flat tail:
The top bar is to scale. Everything under it is magnified twenty times, or you would not see it at all. The seven numbers add up to 1. Greedy sampling always takes the top bar; raising the temperature is what gives the flattened rows a real chance. Everything a chat assistant appears to do — answer questions, write code, argue, apologise — is that loop repeated a few hundred times. There is no separate reasoning module, no fact database being consulted, no plan drawn up before the first word. The model that writes a five-paragraph essay is choosing a next token, then another, and it has no more idea how the essay ends than the reader does when the first word appears.
That description sounds deflating, and it is often used to deflate. But it raises a genuine puzzle rather than settling one. Predicting the next word in arbitrary human text is not a small task. It happens to require most of what we mean by understanding.
Why a boring objective produces interesting behaviour
Consider what it takes to predict well on real text.
To finish “the murderer turned out to be the” in a detective novel, a predictor needs to have tracked who was in the room. To finish “9 × 7 = ” it needs arithmetic. To finish a Python traceback it needs to know how Python works. To finish “the patient’s sodium was 118, so the team” it needs some medicine. None of these were taught as separate skills. They are all just next-token prediction on a corpus that contains novels, textbooks, code, and clinical notes.
This is the key move to internalise: the training objective is narrow, but the data is not. Compression forces structure. A model with a fixed number of parameters cannot memorise the internet, so the cheapest way to lower its prediction error is to learn the regularities that generate the text — grammar, arithmetic, causality, the fact that objects persist. Those regularities are what we later call capabilities.
It also explains the shape of the failures. The model learns whatever helps prediction, including patterns we would rather it not learn. Text on the internet is full of confident wrong answers, so confident wrongness is a well-modelled style. Text is full of stereotypes, so stereotypes get modelled. Nothing in the objective distinguishes “true” from “typical”.
Tokens, not words
Models do not see words or letters. Text is chopped into tokens: common words
are one token, rare words split into pieces, and a space is usually part of
the token that follows it. “unbelievable” might be un, bel, iev,
able.
This is a boring implementation detail with unboring consequences. Counting the letters in a word is hard for a model that never sees letters. Arithmetic on long numbers is awkward because digits group unevenly into tokens. Rhyming and wordplay are harder than the model’s fluency suggests they should be. When a very capable model fails at something a child does easily, tokenisation is a good first suspect.
Temperature and sampling: what the knob does
After the model produces its probability table, something has to choose. That choice is not part of the model.
- Greedy picks the highest-probability token every time. Deterministic, and prone to loops and flat prose.
- Temperature rescales the probabilities before sampling. Below 1.0 it sharpens them, so likely tokens get likelier. Above 1.0 it flattens them, so unlikely tokens get a real chance. At 0 it collapses to greedy.
- Top-p (nucleus) keeps the smallest set of tokens whose probabilities sum to p, then samples within it. This cuts the long tail of nonsense while leaving genuine choice intact.
Two practical points follow. First, “creative” and “accurate” are not two modes of the model; they are two settings of the sampler on identical probabilities. Turning temperature down does not make the model know more, it makes it commit harder to what it already ranked first. Second, sampling is why the same prompt gives different answers, which means any evaluation on a single generation is measuring noise as much as quality. Run it several times, the same way you would with any other stochastic estimate — the reasoning is the same as in bootstrap confidence intervals.
”It doesn’t know anything” — right and wrong
The claim is usually made to end the conversation. It deserves splitting.
Right: there is no fact table inside. The model cannot look anything up. It has no persistent memory between conversations unless a system puts one there. It has no way to check a claim against the world, and no internal signal that separates “I predicted this from strong evidence” from “I predicted this because it sounded like the kind of thing that follows”. This is exactly why it invents citations with the same fluency it reports real ones.
Wrong: “just predicting text” is doing enormous work in that sentence. A system that reliably predicts the next token of a correct proof has, in any useful sense, learned something about proofs. Weights that encode the grammar of French, the syntax of SQL, and the structure of an apology are knowledge in every sense except the one where knowledge means a row in a database. Dismissing the whole thing as autocomplete is as unhelpful as calling it a mind.
The useful position is in between and is boring to say out loud: it is a very good model of how text tends to continue, which correlates strongly with truth in domains where the training text was mostly correct, and not at all in domains where it was not. That single sentence predicts most of what one observes in practice.
Large Language Models explained briefly by 3Blue1Brown is worth twenty minutes for the animation of what actually moves through the network.
Hold on to the loop. Text in, probabilities out, sample, repeat. Almost every surprising behaviour — the fluent nonsense, the sensitivity to phrasing, the inability to say “no idea” — becomes predictable once that loop is the thing being pictured instead of a mind at a keyboard.