Discrimination can happen without discriminatory intent. In South Africa, AI trained on historical data carries the weight of apartheid and structural inequality forward — often invisibly. This module shows you how bias enters, how South African law sees it, and how to audit any tool for it.
Module 3 builds directly on Module 2. Finish all four phases of Module 2 first — then this module unlocks automatically.
The big idea
Algorithmic bias is rarely a glitch, and rarely malice. It is what happens when a system learns from an unequal past and then applies those patterns at scale — fast, cheaply, and wearing a mask of objectivity.
In South Africa, “the past” carries the weight of apartheid spatial planning and structural inequality. This phase gives you the vocabulary — disparate impact and proxy discrimination — and the South African law that governs both.
Forget the word “bias” for a second. The thing we actually care about is systematically worse outcomes for a particular group — produced by an AI system, because of the data it learned from or how it was built.
Algorithmic bias is when an AI system gives consistently different — and unfair — results to people based on race, gender, age, disability, language, or where they live, even when no one intended it to.
Two things follow from that definition, and both matter:
The law — and good judgement — recognises two very different ways a decision can be unfair. Telling them apart is the single most useful skill in this module.
Disparate treatment
Treating someone differently because of a protected attribute.
Overt and usually obvious — e.g. “reject applicants from group X.” Rare in AI, because no one codes it openly.
Disparate impact
A neutral-looking rule that lands harder on a protected group.
Invisible and common — the rule never mentions the group, but the outcomes fall unequally. This is the AI problem.
Example
A lender’s AI never asks an applicant’s race. But it rejects applicants from certain townships far more often. No disparate treatment — clear disparate impact. The harm is identical; only the mechanism is hidden.
“Just remove race and gender from the data and the AI can’t discriminate.” It sounds right. It is wrong — because of proxies.
Postal code → race · First language → ethnicity · School attended → class · Name → gender / ethnicity · Phone & device → income
In South Africa, location is the most powerful proxy of all. Apartheid engineered where people live by race — so a model that uses suburb or postal code can discriminate by race almost perfectly, without ever seeing the word “race.”
Bias is not one thing in one place. It can enter at several points — you met some of these as “failure modes” in Module 2. Here they are through an equity lens:
The past was unequal, so data drawn from it is unequal. The model learns “this is normal.”
Some groups are under-represented in the data (few rural, informal, or women-owned records), so the model’s patterns for them are thin and unreliable.
Someone had to decide what counted as a “good hire” or “high performer.” If those past human judgements were biased, the AI inherits that definition.
Proxies (Lesson 3) smuggle protected attributes back in; feedback loops (Module 2) make a biased prediction come true and then cite itself as proof.
This is where South Africa is different from a generic “AI ethics” course. Unfair discrimination — including the indirect, no-intent kind — is already unlawful here. AI does not get an exemption.
The line that matters: in South African law, “we didn’t mean to” is not a defence. Indirect discrimination is unlawful even with no intent.
Why is AI bias often more dangerous than human bias? Because of how much we trust a number that came from a computer.
People defer to a machine’s output more than to a colleague’s opinion — even when the machine is wrong. A biased recommendation gets a free pass because “the system said so.”
Human bias is one person, sometimes. AI bias is scaled, consistent, instant — and wears a mask of neutrality.
“It’s just maths, it can’t be racist” is the trap. The maths is real; the data and the definitions behind it carry human history.
You cannot fix what you cannot see. The practical way to check any tool is to compare outcomes across groups — the test you’ll run in Phase 4.
AI inherits the inequality in its data and hands it back wearing the mask of objectivity. Detecting and correcting that is a human responsibility — and in South Africa, a legal one.
1. In plain language, what is algorithmic bias?
2. An AI loan tool never uses race, yet rejects applicants from certain townships far more often. This is best described as:
3. A team removes the “race” field from their data and says the model is now fair. Why might it still discriminate?
4. In the South African context, why is location such a powerful proxy?
5. Under South African law (the Constitution and PEPUDA), unfair discrimination…
6. POPIA section 71 gives a person affected by a significant, fully automated decision the right to:
7. Why is AI bias often more dangerous than an individual’s bias?
8. What is the most practical first step to check whether a tool has a disparate impact?
A metro police department buys an AI system to decide where to send patrols. The vendor’s pitch:
The model learns from the department’s records: past arrests & reported crime · location / suburb · time of day · prior call-outs. It never uses race.
Three months later
In the oversight meeting
“We removed race from the model. How can it possibly be profiling by race?”
— The project lead
Click each factor once to mark it a likely bias source · click again to mark not the cause · a third click clears it. Then reveal the answers.
“We removed race” misses the point
Three mechanisms from Phase 1 are running at once.
Proxy discrimination (suburb stands in for race), a feedback loop (patrols manufacture the arrests that justify more patrols), and missing data (unrecorded crime in wealthy areas). The result is textbook disparate impact — profiling by race with the race field deleted.
Under the Constitution and PEPUDA this is indirect discrimination — unlawful regardless of intent. Accountability sits with the department and officials who deployed it, not the algorithm. “The system flagged it” is not a defence.
Every model carries hidden assumptions — decisions about what to measure, what counts as “success,” and whose history to learn from. The policing tool assumed “where we arrested before = where crime is.” That one assumption did the damage. Now turn the lens on your own world.
Assumptions hide in ordinary choices…
Each is a value judgement dressed as a data field.
Pick one AI system used (or being considered) in your sector — CV screening, credit or insurance scoring, fraud flags, a chatbot, tenant or customer scoring, predictive scheduling. Run the disparate-impact test on it.
AI Equity Audit worksheet
Fill in what you can — the gaps you can't fill are findings in themselves.
| Question | Your answer |
|---|---|
| Which AI tool are you auditing? | |
| What decision does it drive? | |
| Which protected groups could be affected? | |
| What proxies are in its data? | |
| Disparate-impact check: do outcomes differ by group? | |
| Who is harmed if it is wrong? | |
| What legal exposure applies? | |
| What human oversight exists? |
Apply the equity lens
You can now define algorithmic bias in plain language, tell disparate treatment from disparate impact, spot proxy discrimination, name the South African law that applies — and run an equity audit on a real tool in your sector.
Your equity-audit worksheet and answers have been recorded and will be reviewed as part of Cohort 1 assessment.