Census time

I just got my census form, so it must be about time to write about the NZ Census.  As I wrote when the West Island had theirs, it’s a good occasion to think about what the census is good for.  You might think that the success of surveys means that the census is no longer necessary, but as landline phones steadily become a less important part of people’s lives, the census (or some substitute) is actually increasingly vital to calibrate surveys.  In fact, part of my research is on the most effective ways of using this sort of information.

The primary problem with surveys is non-response — you can’t get hold of people, or you do catch them and they tell you to get far away and let them have dinner. Good survey organisations have ways to entice people into responding, but they also rely heavily on reweighting: if your survey under-represents families with young children, you can increase the weight given to those you did find, and reduce the bias.

This reweighting technique isn’t perfect, but it really does work.  The world’s largest telephone survey, the US Behavioral Risk Factor Surveillance System, used not to call cellphones.  It now does, providing an opportunity to compare the real cellphone results with the attempts to reweight.  Here are results from Michigan and Utah comparing a basic reweighting approach with a more sophisticated one, for landlines only and for landlines and cellphones. The improved reweighting approach (raking) made a big difference for the landline-only sample, moving it much closer to the landline+cellphone sample. So, reweighting really works.

Reweighting needs good data for the population, so every well-conducted survey from marketing research to opinion polls to the unemployment rate depends on the census, or some substitute.  In Scandinavian countries, the substitute is large administrative databases and record linkage.  We don’t have these, they’d be expensive to set up, and generally in English-speaking countries people don’t want them.  If you don’t have that sort of database, you need either a complete census or a mandatory survey of a random sample of people.

The United States uses both approaches: every ten years they have a complete census as required by the US Constitution, and in between they have the mandatory-response American Community Survey, which samples about 1% of the population each year.  In the US, the American Community Survey is both cheaper and more accurate than adding an extra census at five-year intervals.  In NZ it’s not clear — because of the smaller and more urban population, a sufficiently large survey might not be much less expensive than a five-yearly census.

What we want to avoid is the Canadian approach, where they decided to put nearly all the questions in a new, voluntary survey. The head of Statistics Canada resigned, and while he was forbidden by law to reveal the advice he had given to the government, he could say

I want to take this opportunity to comment on a technical statistical issue which has become the subject of media discussion. This relates to the question of whether a voluntary survey can become a substitute for a mandatory census.

It cannot.

 

[ps: the National Business Review has a story with quotes from me]