Not how polling works
The Herald interactive for election results looks really impressive. The headline infographic for the latest poll, not so much. The graph is designed to display changes between two polls, for which the margin of error is 1.4 times higher than in a single poll: the margin of error for National goes beyond the edge of the graph.
The lead for the story is worse
The Kim Dotcom-inspired event in Auckland’s Town Hall that was supposed to end John Key’s career gave the National Party an immediate bounce in support this week, according to polling for the last Herald DigiPoll survey.
Since both the Dotcom and Greenwald/Snowden Moments of Truth happened in the middle of polling, they’ve split the results into before/after Tuesday. That is, rather than showing an average of polls, or even a single poll, or even a change from a single poll, they are headlining the change between the first and second halves of a single poll!
The observed “bounce” was 1.3%. The quoted margin of error at the bottom of the story is 3.5%, from a poll of 775 people. The actual margin of error for a change between the first and second halves of the poll is about 7%.
Only in the Internet Party’s wildest dreams could this split-half comparison have told us anything reliable. It would need the statistical equivalent of the CSI magic video-zoom enhance button to work.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
It’s probably even worse because the second half of the survey gets different people to the first half of the survey – the hard to contact people are more likely to be in the second half.
10 years ago
Probably, but I wasn’t sure enough of how the polling is actually done to say that. There could be some sort of blocking by time.
10 years ago
morning Thomas. I don’t understand your last paragraph. It isn’t Internet Mana making this claim, it is the Herald. Did you mean Only in the Heralds wildest dreams?
10 years ago
No, I mean that the sort of change that the Herald would be able to detect is what Dotcom had been foreshadowing but didn’t deliver.
10 years ago
I emailed the reporter along similar lines after yesterday’s teaser story for this. Doesn’t the margin of error only apply to major percentages as well. Once you get down to the Act/Conservative etc levels, doesn’t it go out the window? She had headlined a drop of 0.5% from 3.8 to 3.3!
10 years ago
Yes, the margin is smaller for minor parties, but not that small. I posted a table last month.
As usual, margins for change are about 1.4 times larger.
10 years ago