Where do the numbers come from?
“What someone doesn’t want you to publish is journalism; all else is publicity.” Paul Fussell
Duncan Hedderley, in a comment on the ‘Overgeneralising again’ post, suggested that the problem with those articles was really advertising masquerading as research, rather than imprecise statistical reporting.
In fact, most of the stories we comment on are like that. It can be interesting to see if you can find any statistics in the typical news broadcast or newspaper that aren’t just publicity or commentary. That is, the story contains a number which is an estimate or summary for a well-defined population, and either
- if the story is based on a press release, at least one of the numbers does not come from the press release or from interviewing the people who produced the press release; or,
- the story contains an informed comment on the number, where the comment comes from someone independent of the source of the number.
There’s plenty of non-statistical information of this sort in newspapers — asking questions and looking for stories is what journalists are for — but statistical examples are a bit thin on the ground. If we look at Stuff.co.nz, there’s a story about same-sex civil unions that qualifies, and in the NZ Herald online, there was a report on a recent drug-use study that has a comment from the NZ Drug Foundation.
Note that this isn’t a very high bar: it would have been nice, in a story commenting on the prevalence of cannabis use, to give some idea of trends over time or to say something about the proportion of people who support cannabis decriminalisation or legalisation. This issue is also not the same as quality of reporting: the Stuff story on the same drug-use survey only had comments from the study authors, but the authors give some useful information on the study difficulties, and mentions that the survey is part of a special series in leading medical journal The Lancet on addiction and drug policy.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »