Bird of the Century
For years, Bird of the Year has been the only self-selected (‘bogus’, ‘straw-poll’, ‘unscientific’) survey endorsed by StatsChat. The unique feature of Bird of the Year as a bogus poll is that no-one pretends it is anything else. The pūteketeke won Bird of the Century fair and square, due to John Oliver’s marketing efforts, and no-one seriously thinks this says anything real about the relative popularity of New Zealand birds.
The key takeaway from Bird of the Century is this is what bogus polls are like. All of them. When a bogus poll agrees with population opinion, it’s just an accident. When someone claims to have information from a survey result, it’s always good to ask whether it’s the type of survey that’s more accurate than just pulling a number out of your arse, or not.
A couple of weeks ago, there was a widely–reported claim made by a spokesperson for AA Insurance that 53% of people in New Zealand wanted a ban on domestic use of fireworks. None of the media outlets asked (or reported asking) anything about how he got that number. When a news report says something that’s attributed to an anonymous source like that, you want to know who is vouching for the credibility of the source.
I happened to see a post on social media by someone who had been in a survey that could have been the one quoted, which was run by my2cents. I don’t know how good their surveys are, but they at least qualify as trying to get the right answer. If that survey was actually the one reported by AA Insurance, it would be good to know.
In some contexts, such as election polling or policy decisions, you might want to know more about the methods used and the reputation of the pollsters. Even in simple news reporting, though, it’s important to ask if this is the sort of survey that gives you information or the sort of survey that just gives you grebes.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
It wasn’t just a survey, though. It also asked for a donation. Hopefully they made some money…without annoying too many people.
12 months ago
I had to google “grebes.” Nice one.
12 months ago