Ideas that didn’t pan out
One way medical statisticians are trained into skepticism over their careers is seeing all the exciting ideas from excited scientists and clinicians that don’t turn out to work. Looking at old hypotheses is a good way to start. This graph is from a 1986 paper in the journal Medical Hypotheses, and the authors are suggesting pork consumption is important in multiple sclerosis, because there’s a strong correlation between rates of multiple sclerosis and pork consumption across countries:
This wasn’t a completely silly idea, but it was never anything but suggestive, for two main reasons. First, it’s just a correlation. Second, it’s not even a correlation at the level of individual people — the graph is just as strong support for the idea that having neighbours who eat pork causes multiple sclerosis. Still, dietary correlations across countries have been useful in research.
If you wanted to push this idea today, as a Twitter account claiming to be from a US medical practice did, you’d want to look carefully at the graph rather than just repeating the correlation. There are some countries missing, and other countries that might have changed over the past three decades.
In particular, the graph does not have data for Korea, Taiwan, or China. These have high per-capita pork consumption, and very low rates of multiple sclerosis — and that’s even more true of Hong Kong, and specifically of Chinese people in Hong Kong. In the other direction, the hypothesis would imply very low levels of multiple sclerosis among US and European Jews. I don’t have data there, but in people born in Israel the rate of multiple sclerosis is moderate among those of Ashkenazi heritage and low in others, which would also mess up the correlations.
You might also notice that the journal is (or was) a little non-standard, or as it said “intended as a forum for unconventional ideas without the traditional filter of scientific peer review”.
Most of this information doesn’t even need a university’s access to scientific journals — it’s just out on the web. It’s a nice example of how an interesting and apparently strong correlation can break down completely with a bit more data.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
I notice the title of the original paper: “Multiple sclerosis, latitude and dietary fat: Is pork the missing link?” also follows Betteridge’s law of headlines. Or is that being churlish? :-)
10 years ago
It’s the same principle in scientific titles as newspaper headlines: you want to say it, but not to be blamed for the evidence not being up to scratch.
Nowadays this title style is discouraged in medicine journals.
10 years ago