Number Mistakes — Statistical & Probability Errors
The Big Total Tells A Different Story Than The Smaller Groups
Simpson's Paradox
In Plain English
Simpson's Paradox happens when a big total tells one story, but the story changes when you split the data into smaller groups that matter. The overall numbers are not fake, but they can hide how the groups were mixed together. Maybe one option looks worse overall even though it performs better inside each subgroup. This bug matters because people often trust the headline number and stop there. The fix is to ask whether important groups are being blended in a way that hides the real pattern.
Featured Example
Class average confusion
One teaching method looks worse across the whole school, but inside both beginner and advanced classes it actually performs better.
What This Sounds Like in Classrooms
- A schoolwide average hides the fact that different class levels faced very different conditions.
- One program looks weaker overall until the results are split by starting skill level.
- A headline number hides the subgroup story.
What This Sounds Like in Business
- One team seems less productive overall, but inside each project type it actually performs better.
- A hiring funnel looks unfair or effective until the data is split by role, region, or applicant mix.
- The total makes a policy look bad while the subgroup view changes the story.
What This Sounds Like in Real Life
- A health result looks clear overall, but age groups tell a more careful story.
- A neighborhood total hides the fact that very different blocks have very different patterns.
- One app feature seems popular overall until usage is split by user type.
Examples from Literature or Fiction
Detective and courtroom stories
A broad summary points one way, but the case changes when the details are sorted into the right groups.
The overall picture hides the structure underneath.
School or war novels with class or rank divisions
A general claim about performance or fairness shifts once results are broken apart by role.
Mixed groups hide the real comparison.
Social dramas about status and opportunity
Big averages hide the fact that different groups faced different chances from the start.
The total can tell the wrong lesson when important groups are blended together.
Why People Fall for It
Summary numbers feel clean and powerful. People like the big total because it is easier to read than a more careful grouped analysis.
How to Spot It
- The headline number is doing all the work.
- Important subgroup differences are missing.
- The pattern changes when the data is split more carefully.
- The summary feels simple, but the situation is clearly mixed.
What to say instead
- What happens if we break this into meaningful groups?
- Are different populations getting blended together here?
- Does the overall number hide the real comparison?
- Big totals are useful, but they are not always the full story.
Common Confusion
People mix this up with:
Compare Nearby Ideas
Quick Comparison
Base Rate Neglect vs Availability Heuristic
Base Rate Neglect ignores the big background numbers, while Availability Heuristic overweights whatever example comes to mind most easily.
Mini Practice
Question: A schoolwide average says one method did worse, but inside both beginner and advanced classes it did better. What is the bug?
Answer: Simpson's Paradox.
The overall number hid the subgroup pattern and gave a misleading summary.
Remember This
The big total can hide the real story inside the groups.
Related Brain Bugs
Base Rate Neglect
Ignoring The Big Background Numbers
Number Mistakes
A test flags a rare condition, and someone assumes the condition is now very likely without looking at how rare it is in the first place.
Learn this bugCorrelation vs. Causation
Together Does Not Mean Caused
Number Mistakes
Ice cream sales rise when beach rescues rise. That does not mean ice cream causes the rescues. Hot weather drives both.
Learn this bugTexas Sharpshooter Fallacy
Pick The Pattern After The Fact
Number Mistakes
A team looks at many product metrics, finds the two that moved together, and presents that pair as if the trend had been clear from the s...
Learn this bug