February 20, 2012
Stats crimes – we need your help
What do you think are the biggest media/public misunderstandings around statistics? We know that some statistical concepts can be quite hard to understand (and a bit of a challenge to teach); we’d like to compile a list of the top stats misunderstandings so we can accurately focus some media education projects we are planning ….
Some examples that have already been raised:
- Misunderstanding correlation and causality: All too often causality will be assigned where a study has merely shown a link between two variables.
- Abuse/misuse of the term “potentially fatal”: While many activities/diseases could possibly result in death, the odds should be considered in the context of a developed country with reasonable health-care.
- How to know when something is statistically significant and when not.
- How to know when you are looking at “junk” statistics …
Please share your ideas below …
Atakohu Middleton is an Auckland journalist with a keen interest in the way the media uses/abuses data. She happens to be married to a statistician. See all posts by Atakohu Middleton »
I dont know how to name such abuse of statistics. The most famous media worker who abuses statistics a lot is John Campbell. He walked to a pub or else and asked several people there and made conclusion.
13 years ago
1. Lack of context. “only 5% growth” (rest of world average 2% growth)
2. Lack of scale. 200% growth OMG! When starting with tiny base.
3. Lack of trend. ‘highest death total in NZ ever’ (because there are more people…..)
4. Lack of history. Milk costs $5 a litre!!! (which if you adjust for inflation…..hasn’t changed all that much…)
13 years ago
Love it Eric!
13 years ago
Those things would have to be right up there, along with bogus polls that get reported on by the media from time to time that have been well covered here.
I’ve got reservations about ipredict too, but I’m only an undergraduate statistics student, maybe I need to advance a bit more before I understand prediction markets.
13 years ago
The one that really get’s to me (I’ve even written about it myself) is leading with relative risks of some treatment/environmental factor without giving us an idea of the absolute risk.
The other one relates to political polls. It might be too technical for what you’re trying to do but every time I hear a journalist say a minor party is “within the margin of error” I cringe.
13 years ago
Wholeheartedly agree with David and Eric.
lesser things I would add would be issues being reported as having overwhelmingly support when say, 60% of a poll/vote agrees but only 40% of the potential base has responded. – I would have loved to see the last election results reported as National 33%, Did not vote 31%, Labour 18.5%, Greens 7% etc…
It would also be nice to see biases in poll’s reported, i,e, this poll only called landlines, missing out x% of people who have no land line,
13 years ago
Cherry picking data or papers that support your side of the story.
13 years ago
“Cherry picking data or papers that support your side of the story.”
You mean marketing.
13 years ago
Forgive my non-statistics way of explanation, or even being plain wrong!
Reckon a big misunderstanding is the independence of events. eg, friends have 4 girls, they reckon the odds would be much better if they tried again for a boy, but the way I understand odds, it’s still a 50-50 chance of a girl/boy. Now if you calc’d odds at the start of them breeding it would only be a 6% chance of having 4 girls, or 3% of 5 girls, which is where the confusion comes from, you get the picture.
But this is huge, from red/black runs on roulette to flipping coins etc, I’m sure you get the picture!
13 years ago
Besides your lead suggestions and Eric’s (well done!):
* Lack of source (a recent study suggests… which study? so we can make our own minds).
* Badly defined population (does it apply to me or it is only relevant to residents of small mining towns that limp every second month?).
* Significance fishing (did they try 500 variables and one of them happened to be ‘significant’?)
* Infographic abuse: designers going on a rampage, comparing circles based on diameter and not area.
13 years ago
Cost of X or benefit of X studies. They’re almost universally awful. The figures are not usefully comparable to anything else in the real world, and cannot usefully inform policy, but are politically useful in lobbying for policy changes.
13 years ago
Reporting as significant patterns found in noise.
Eg “deaths from shark attacks up 40% this year”
13 years ago
Correlation does not equal causation.
And the stats are only as good as the experimental methods used.
13 years ago
Margin of error in polls. Results quoted that show no understanding of the concept. Suggesting a result is notable but it is not!Then polls are compared where they have different margins of error.
13 years ago
An event is not interesting just because it is rare. Anyone can shuffle a deck of cards and produce a sequence that has a 1 in 52 factorial chance of occuring. Big deal!
13 years ago