The image generators that appeared this year are built on an idea that sounds like a joke when you first hear it. You do not teach the model to draw. You teach it to clean up. Then you hand it a screen of television static and ask it to clean that, and what comes back is a picture that never existed.
No part of that description is a simplification. Everything else is engineering. The rest of this post is why it works, because the plain statement is unbelievable until you see where the training signal comes from.
Start by ruining a picture
Take a photograph. Add a small amount of random noise to every pixel — not enough to matter, the image still looks fine. Do it again. And again. After a few hundred rounds, the photograph is gone and you are looking at pure static, with no trace of what it used to be.
This is the forward process, and the important thing about it is that it requires no intelligence whatsoever. It is a loop with a random number generator. You can run it on any image you have, stop at any point along the way, and you get something valuable: a pair. On one side, a partly ruined image. On the other, the exact noise you added to ruin it, because you are the one who added it.
That pairing is the trick that makes the whole field work. Supervised learning needs labelled examples, and labels are normally the expensive part. Here they are free and effectively infinite: every image, at every noise level, with every random draw, is another example whose correct answer you already know.
Learning to undo one step
Now train a network on that pile of pairs. Input: a noisy image, plus a number saying how far along the ruining process it is. Output: a prediction of the noise that was added. Compare the prediction to the noise you actually added, take the error, adjust the weights. Ordinary supervised learning, no adversary, no reinforcement, nothing exotic. The detail that made this competitive was asking for the noise rather than the clean image: Ho, Jain and Abbeel’s 2020 paper Denoising Diffusion Probabilistic Models set out that training target, and every generator built since works this way.
Note what the model is forced to learn to do well at this. To separate noise from signal, it has to know what signal looks like: that skies are smooth, that faces have two eyes above a nose, that brick walls repeat, that shadows fall consistently. Without that, it could not tell which speckles are noise and which are texture. The chore is denoising; the knowledge it demands is a model of what real images are like.
Why cleaning is the same as creating
Here is the move. The trained model can take a noisy image and give you a slightly less noisy one. So start from an image that is entirely noise, with no photograph hidden underneath, and run the model anyway.
The model does not know there is nothing there. It does what it always does: it looks at the static, decides which parts look like noise, removes them, and leaves behind whatever it judged to be signal. That leftover is not a picture yet — it is a faint suggestion of structure. Feed it back in. The model cleans again, and the suggestion firms up. Repeat a few dozen times and the suggestion becomes an image. The two chains, side by side, are the whole method.
Every arrow in the top strip is a random number generator. Every arrow in the bottom strip is the same trained network, run again on its own output. The model was only ever taught to do one arrow.
It is the same thing you do when you see a face in a cloud, run in a loop, with each pass committing a little harder to the interpretation. And it explains the one property that puzzles people: why the same model produces endless different outputs. The starting static is random. A different sheet of static contains different accidental structure, so the model commits to a different interpretation, and you get a different picture. The randomness is the input, not a setting.
Steering it with words
An unguided model of this kind makes plausible images of nothing in particular. To get a picture of what you asked for, the text has to enter the loop.
The text is first turned into a vector — the same kind of representation described in what word embeddings really are, except trained so that a caption and its matching photograph land in nearby places. That vector is then fed to the denoiser at every step, alongside the noisy image. The model is no longer answering “what noise is in this?” but “what noise is in this, given that the result should be a picture of an astronaut on a horse?”
Because it was trained on captioned images with the caption sometimes present and sometimes withheld, it can be run both ways at generation time — once with the prompt, once without — and the difference between the two answers points in the direction the prompt is pulling. Exaggerate that difference and the prompt gets obeyed harder. That knob, usually exposed as guidance, is the main control the user has. Turn it up and the image matches the words more literally but the colours go garish and the variety collapses. Turn it down and you get prettier, freer images that ignore half of what you asked for. There is no correct setting, only a trade.
The most accessible non-technical explanation of the whole pipeline is AI art, explained by Vox, which covers the text-and-image pairing well.
What the mechanism predicts about the failures
Understanding the loop tells you in advance where these systems fall over, and the failures are consistent across DALL·E 2, Imagen, and everything else built this way.
- Counting. “Five apples” is not a local property of any patch of pixels. The denoiser works from local plausibility, so it produces apple-ish regions that each look right and total four or seven.
- Text inside images. Letters are shapes to this model, not symbols. It reproduces the texture of writing without the constraint that the shapes must spell something.
- Relations between objects. “A red cube on top of a blue sphere” often comes back with the colours swapped. The conditioning vector carries the concepts more reliably than which attribute attaches to which object.
- Anatomy at the edges. Hands and teeth are highly variable in training photographs and highly constrained in reality — the exact combination this approach handles worst.
None of these are bugs to be patched. They are what you get from a system whose whole objective is local plausibility, trained on whatever images were scraped, with no representation of quantity, symbol, or physics.
What stays open is how much of that list is permanent. Counting and text inside images have both improved with scale and with stronger text encoders, and neither improved because anyone gave the model a concept of quantity or of letters. So it is genuinely unclear whether local plausibility, pushed far enough on enough data, ends up doing the same job as understanding the request, or whether the remaining failures are the ones that never go. Nothing in the mechanism above answers that.