Winston First?

An ongoing theme of StatsChat is that single political polls aren’t a great source of information, and that you need to combine them. A case in point: this piece at Stuff describing a new Horizon poll.  The headline is Winston Peters returns to kingmaker position in new political poll, and the poll has NZ First on 6.75%.  My second-favourite NZ poll aggregator, Wikipedia, shows other recent polls, where the public results from Curia, Roy Morgan, and Kantar were 2.1%, 1%, and 3% and a leaked result from Talbot Mills was 4%.  It’s possible that this shows a real and massive jump over the past couple of weeks. Stranger things do happen in politics — but not much stranger and not all that often. It’s quite likely that it’s just some sort of blip and doesn’t mean much.

Stuff does add “The poll had a margin of error of 3.2%, meaning NZ First’s crossing the 5% threshold was within the margin of error,”  but that’s the wrong caveat.   The 3.2% margin of error is more strictly called the ‘maximum margin of error’, because it’s the margin of error for proportions near 50%, which is larger than at, say, 5%.  I’ve written before about calculating the corresponding margin of error for minor parties.

In this case, under the pure mathematical sampling approximations used to get 3.2%, a 95% uncertainty interval for NZ First’s true support would go from 5.2% to 8.5%. If we only worried about sampling error, NZ First would be fairly clearly above the 5% threshold.  The problem is that the mathematical sampling error  is typically an underestimate of total survey error — and when you get a very surprising result, it’s sensible to consider that you might possibly be out on the fringes of the total survey error.  Or not. We will find out soon.