Survey respondents are lying, not ignorant
At least, that’s the conclusion of a new paper from the National Bureau of Economic Research.
It’s a common observation that some survey responses, if taken seriously, imply many partisans are dumber than a sack of hammers. My favorite example is the 32% of respondents who said the Gulf of Mexico oil well explosion made them more likely to support off-shore oil drilling.
As Dylan Matthews writes in the Washington Post, though, the research suggests people do know better. Ordinarily they give the approved politically-correct answer for their party
In the control group, the authors find what Bartels, Nyhan and Reifler found: There are big partisan gaps in the accuracy of responses. …. For example, Republicans were likelier than Democrats to correctly state that U.S. casualties in Iraq fell from 2007 to 2008, and Democrats were likelier than Republicans to correctly state that unemployment and inflation rose under Bush’s presidency.
But in an experimental group where correct answers increased your chance of winning a prize, the accuracy improved markedly:
Take unemployment: Without any money involved, Democrats’ estimates of the change in unemployment under Bush were about 0.9 points higher than Republicans’ estimates. But when correct answers were rewarded, that gap shrank to 0.4 points. When correct answers and “don’t knows” were rewarded, it shrank to 0.2 points.
This is probably good news for journalism and for democracy. It’s not such good news for statisticians.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
I don’t think they lie. I think they want to get the survey over and done with as quickly as possible so they grab an answer off the top of their head witht the underlying bias that Bush=good/Bush=bad depending on party affiliation. When there is a prize at stake they are willing to invest more time in getting the right answer by going through their memories more rigorously.
12 years ago
That’s possible. I don’t think the experiment reported time taken.
12 years ago
It just goes to prove something the PR industry has been making good use of for over a decade – Bribery Works.
12 years ago
Remarkably small amounts of bribery, though. The expected return was about US$1/correct answer, quite a bit less than the per-person cost of telephone sampling.
12 years ago