Election poll accuracy

1News had a new electoral opinion poll out this week, headlined Poll: National, ACT maintain wafer-thin advantage. The report gave National and ACT 61 seats out of 120. However, if you put the reported percentages and assumptions into the Electoral Commission’s seat allocation calculator, as various people did, you would get a different result, with National and ACT holding 60 seats, and a potential Labour/Green/Te Pāti Māori alliance also holding 60.  That’s strange, but you wouldn’t really expect 1News to get this wrong (unless you were the sort of social media commenter who immediately assumed it was a conspiracy rather than an error).

The first thing to check in a situation like this is the full report from the polling company. The percentages match what 1News reported, and if you scroll down to the bottom, there’s an corresponding seat allocation that matches the one reported.  At this point a simple error looks even less likely.  So what did happen?

Looking at either the 1News report or the Verian report, we see that percentages were rounded to the nearest percentage point, or to the nearest tenth of a percentage point below 4.85%. So, can we construct a set of numbers that would round to the the same reported percentages and match the reported seat allocation? Yes, easily. We need National and Labour to have been rounded down and the Greens to have been rounded up.  So there’s no evidence of a mistake, and rounding is easily the most plausible explanation.

On the other hand, one might hope 1News or Verian would notice that the headline figures are very close to even, and consider how sensitive the results might be to rounding.

On the other other hand, though, what this shows is that a single poll is never going to be enough to support a claim of “maintain wafer-thin advantage”.  The left-right split can easily be off by five seats, and that’s a big difference; if the advantage is wafer-thin, it’s way below the ability of the polling system to measure.  You can do somewhat better by combining polls and estimating biases of different pollers, as the Herald’s poll of polls is doing, using a fairly sophisticated model.  They have been fairly consistent in predicting that (assuming no major events that change things) Labour/Green are unlikely to get a majority without Te Pāti Māori, but that National/ACT have reasonable odds of doing so.

Even then, you can’t really do ‘wafer-thin’ comparisons.