Overgeneralising again.
The NZ Herald online has had two stories in two days on a survey by Southern Cross Health Society. The survey reported cancer as the number one health fear of Kiwis. That would be the minority of Kiwis with private health insurance. Or, though the actual survey population isn’t stated anywhere, probably the smaller minority of Kiwis who are members of Southern Cross.
I don’t know whether the cancer fear finding generalises to the whole population, and neither do they, but I’m certain the screening results they report do not. They say more than 80% of men aged 55-64, and 93% of men over 65 had a prostate cancer screening test within the past year. Figures based on a national survey published in 2010 by the University of Otago, show that about 44% of men 60+ and 17% of men 40-60 had a PSA test in the the previous year (which includes diagnostic and post-treatment as well as screening tests). Only 64% of men over 60 had ever had a PSA test of any sort and only 32% had ever had a screening PSA test as part of primary care. It’s hardly surprising that people with private health insurance get more screening, though it’s still not completely clear to anyone except perhaps Paul Holmes whether the extra PSA screening is doing them any good.
The statistical message is simple: surveys only measure what they measure, not what you would like them to have measured. Get over it.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
Isn’t this just an example of survey-as-advertising (advertorial?) While there are some teaching points to be made about it, the aim of the survey is to generate a press release which the media can easily turn into content, which keep your pet issue (say, health worries and tests you might want to go private for) in public awareness?
13 years ago
On one hand, yes, it’s definitely an example of survey as advertising, and it would be better if these just weren’t treated as news.
On the other hand, reporting advertorial surveys accurately would automatically make most of them uninteresting as news and wouldn’t require any judgement as to their motive, so statistical literacy is a win-win strategy here.
And a few advertorial surveys do arguably have real content (the Durex sex survey, for example) and might be eligible as news even though their motive is purely advertising. You could argue that CensusAtSchools falls into that category, too.
13 years ago
[…] Hedderley, in a comment on the ‘Overgeneralising again’ post, suggested that the problem with those articles was really advertising masquerading as research, […]
13 years ago