Generating a convincing image runs into a problem that has nothing to do with computing power. You cannot write down what you want. A loss function needs a formula, and there is no formula for “looks like a real photograph”. An image can be numerically close to a photo and obviously fake, or numerically far from it and completely convincing.
Worse, the obvious choice makes things actively bad. Train a network to minimise squared error against target images and, wherever the answer is uncertain, the safest prediction is the average of all plausible answers. Averaging plausible images gives you blur. That is why a decade of image generation produced smudges: the loss function was rewarding cowardice.
The generative adversarial network fixes this by refusing to write the loss function at all, and training a second network to be it — the idea Goodfellow and colleagues put forward in Generative Adversarial Nets in 2014.
Two networks with opposite jobs
The setup has two parts.
The generator takes a vector of random numbers and turns it into an image. At the start it produces noise, because its weights are random.
The discriminator takes an image and outputs one number: the probability that this image came from the real dataset rather than from the generator. It is an ordinary binary classifier, trained on a mix of real photographs and the generator’s current output.
They are trained in alternation. The discriminator gets better at spotting fakes. Then the generator is updated to make the discriminator’s job harder — its gradient comes straight through the discriminator, which is differentiable, so the generator learns exactly which direction to move its pixels to raise the “real” score.
The usual framing is a forger and a detective, and it is a good one as long as you keep hold of the detail that makes it work: the forger gets to see inside the detective’s head. It is not guessing what was wrong with the last note. It is told, precisely, which changes would have raised the detective’s confidence.
The whole arrangement is one loop:
The discriminator is trained to make the verdict right. The generator is trained, through the same differentiable discriminator, to make it wrong. Neither network is given a definition of “looks real” — the loop supplies one, and it moves every round.
Why the competition produces realism
The interesting property is that the standard moves. Early on, the discriminator wins on obvious cues — colour statistics are wrong, everything is too smooth. The generator fixes those, and now they are worthless as evidence. The discriminator has to find subtler tells: implausible edges, textures that do not repeat correctly, two eyes that do not match. The generator fixes those too.
Each round, the only remaining way to tell real from fake is a finer detail than the round before. The loss function is not fixed and hand-written; it is learned, adaptive, and always aimed at whatever the generator is currently worst at. That is the actual contribution of the idea. It also explains the sharpness: there is no averaging pressure anywhere in this setup, since a blurry image is trivially spotted as fake.
The theoretical end state is a generator producing images indistinguishable from real ones and a discriminator reduced to guessing — 50% accuracy. It is one of the few places where a classifier at chance level means the project succeeded, which is a reminder of the point made in accuracy is the wrong metric: the number only means something once you say what it is measuring.
Why they are miserable to train
Anyone who has actually trained one will tell you the elegance comes with a bill.
There is no progress bar. Normally the loss goes down and you know things are working. Here both losses wander. A falling generator loss might mean better images, or a weaker discriminator. The honest check is to look at samples with your own eyes, which does not scale and does not automate.
Mode collapse. The generator’s only goal is to fool the discriminator, and producing one extremely convincing image every time achieves that. Variety is not required. So generators routinely collapse onto a narrow slice of the data — every face the same age, every digit a 7 — and the loss looks fine while the output is useless. Fixing it has produced a long line of patches to the loss and to how the discriminator sees its examples.
The balance is fragile. If the discriminator gets too good too fast it rejects everything with confidence, the generator’s gradient vanishes, and learning stops. Too weak and it gives no useful signal. Much of the craft is keeping two networks improving at similar rates, which is why results here are unusually sensitive to hyperparameters and unusually annoying to reproduce.
What this has to do with face swaps
A point worth being precise about, since the two get conflated. The face-swap videos circulating since late last year are not straightforwardly a GAN product. The common tools train a pair of autoencoders — one shared encoder that compresses a face to a compact code, plus a separate decoder for each person. Feed person A’s face through the shared encoder and person B’s decoder, and you get B’s face wearing A’s expression and pose. Adversarial training is increasingly bolted on to sharpen the output, but the core is compression and reconstruction, plus a great deal of unglamorous work on alignment, colour matching, and blending the result back into the frame.
It works because faces are a narrow, well-studied domain with enormous amounts of footage available for anyone public. Narrow domain plus lots of data is exactly where these methods are strongest.
Trusting images, without the panic
The temptation is to declare the end of photographic evidence. That is overstated in one direction and understated in another.
Overstated, because images stopped being self-authenticating long before neural networks. Retouching, staging, and misleading captions have been available for a century. Serious verification has always rested on provenance — who recorded it, on what device, when — not on staring at the pixels. That practice does not change. It just stops being optional.
Understated, because what changed is cost. Convincing manipulation used to need a skilled person and days of work, which limited it to targets worth the effort. It now needs a consumer graphics card and public footage. Volume, not peak quality, is the shift.
Two consequences follow. Automated detection is an arms race by construction: any published detector is a discriminator, and a discriminator is precisely what this training procedure is designed to defeat. And the mere existence of good fakes gives anyone caught on camera a ready dismissal — the damage from plausible deniability may outrun the damage from any individual fake. Deepfakes - Real Consequences by ColdFusion covers that social side without hysterics.
The technique itself is neutral and useful — the same machinery does image-to-image translation, super-resolution, and synthetic training data where real examples are scarce. Understanding how it works is the part that helps. A method whose entire objective is to defeat a detector was never going to be reliably caught by a detector, and knowing that saves you from waiting for a tool that is not coming.