AI bias is systematic, patterned unfairness in a model's outputs — where the tool consistently produces worse or skewed results for a particular group, language, or viewpoint, not because of a random glitch but because of how it was built.
That distinction matters: a random error is noise, while bias is a signal that runs in one direction every time. If a résumé screener downgrades women's CVs, or an image generator renders "CEO" as a man almost every time, that is not bad luck. It is the model faithfully reproducing a pattern it learned.
The mechanism is straightforward once you see it. AI models learn statistical patterns from training data — huge collections of text, images, or records — and from human feedback that ranks some outputs as better than others. If the data is skewed, unrepresentative, or carries historical discrimination, the model absorbs those patterns and reproduces them at scale.
A hiring model trained mostly on résumés from one demographic learns that demographic as the "normal" shape of a successful candidate. The bias is not written into the code by a malicious engineer; it is inherited from the world the data came from. The same logic applies to labelling: if human annotators mark certain accents or dialects as "unprofessional," the model learns that judgment as fact.
A well-documented case makes this concrete. In 2018, Reuters reported that Amazon scrapped an experimental recruiting tool after it was found to penalise résumés that included the word "women's" — as in "women's chess club captain." The system had been trained on a decade of mostly male applicants, so it learned that male-coded language correlated with success.
Nobody programmed that rule. It emerged from the data. A second, widely discussed example comes from early image-generation tools, which produced far more male than female images for prompts like "doctor" and far more female images for "nurse" — again reflecting the distribution of the training images rather than any explicit instruction.
These cases are useful because they show bias is not exotic. It shows up in ordinary tools doing ordinary tasks.
The limits of fixing this are real and worth stating plainly. Bias is hard to measure because you need to know which groups to compare and what "fair" even means — equal outcomes, equal error rates, or something else? Those definitions can conflict.
Debiasing one dimension often degrades performance elsewhere: force an image model to output 50/50 gender splits and you may distort the underlying distribution, which can hurt accuracy for legitimate use cases. And when people say "the model is biased," they often mean the data or the deployment context is — the model is a mirror, and the mirror is only as fair as what it was shown.
Practical mitigation looks like auditing training data for representation, testing outputs across groups before launch, and keeping a human in the loop for high-stakes decisions like hiring or lending. It is ongoing work, not a one-time patch.
If you want to understand the broader pattern of why AI tools produce wrong or skewed outputs, the same underlying mechanics — pattern learning from imperfect data — explain both bias and hallucination. For a deeper look at how models can confidently state falsehoods, see What is an AI hallucination and why do AI tools make things up?.
And if you are wondering whether your own data is being used to train models, What does it mean when an AI tool trains on your data, and should you worry about it? covers that side of the equation.