Sampling error calculations

In the Stat of the Week competition, Nick Iversen points to a poll with a margin of error given as 3.9% from a sample of 1004, where the simple formula based on sample size would give 3.1%, and writes

The US poll uses 3.9%. I wonder if they are accounting for sampling bias as well as sampling error. Sounds to me like the US pollsters are doing a more sophisticated measure of error than NZ ones.

Following the link to the pollers methodology, we find that weighting is responsible.  The poll answers are reweighted to known population totals (eg, from the census) for age, sex,  and race/ethnicity, and also reweighted based on phone numbers per household.  They say

“Once weights are final, the effective, or weighted, sample size – rather than the actual
sample size – is used to compute the survey’s margin of sampling error.”

So, the 3.9% maximum margin of error doesn’t account for sampling bias in any direct way (which would be hard to impossible), but it corrects for the reweighting that is done to reduce sampling bias.  The more variable the resulting weights, the lower the effective sample size and the higher the maximum margin of error.

There’s some discussion on Simon Jackman’s blog (Australian political scientist now at Stanford) saying that serious US polling organisations usually do adjust the margin of error because of the reweighting, but that Australian ones don’t seem to. It looks like NZ ones don’t either.