Simpson's paradox
Treatment A wins for every group of patients, yet loses overall.
Edward Simpson, 1951; earlier Karl Pearson and Udny Yule
Two treatments, one trap
In 1986 doctors in London compared two treatments for kidney stones. Treatment A was open surgery. Treatment B was a newer, less invasive keyhole method.
Looking at all 700 patients, the newer method seemed better: 83 per cent of B's patients were treated successfully, against 78 per cent for A. Case closed?
Then they split the patients by the size of their stones. For small stones, A succeeded 93 per cent of the time and B 87 per cent. For large stones, A succeeded 73 per cent of the time and B 69 per cent. A was better for small stones and better for large stones.
So A was better for every kind of patient, but worse for all patients put together. How can that be?
The puzzle
A trend that appears in every group can reverse when the groups are combined.
See it for yourself
What's really going on
The hidden factor is which patients got which treatment. Doctors gave the tough cases, large stones, mostly to surgery (A): 263 large-stone patients against only 80 for B. Large stones are harder to treat whatever you do, so A's overall score was dragged down by its difficult caseload, not by being a worse treatment.
Simpson's paradox turns up in medicine, sport, exam results and university admissions. A famous 1973 study of Berkeley admissions seemed to show bias against women overall, but department by department women were admitted at similar or higher rates: they had applied more to the most competitive departments. The lesson is to ask what lurking variable might be mixing the groups.
Think about it
Two cricketers: one has a better batting average than the other in both the first and the second half of a season, but a worse average for the whole season. Can you invent numbers that make this happen?