Who you gonna call? part 2
Nate Silver, at the fivethirtyeight blog at the New York Times, writes
On Saturday, a survey came out showing Mitt Romney with a large, 21-point lead in South Carolina. The poll is something of an outlier relative to other recent polls of the state, all of which show Mr. Romney ahead, but by margins ranging from 2 to 9 points.
The poll, conducted by Ipsos for Reuters, has already attracted more than 200 citations in the mainstream media. Most of these articles, however, neglected to mention a key detail: in a break with Ipsos’ typical methodology, the survey was conducted online….
He goes on to give a good description of the problems with online polling and how the results match up to other techniques in election polls, where there is good evidence of comparability. These online polls aren’t the `unscientific’ (aka ‘bogus’ web page surveys) we’ve complained about before, they are from polling companies who are at least trying to look accurate.
What Nate Silver doesn’t discuss further is the very large media coverage received by the anomalous poll. If you want election nerds to take you seriously it helps to get the same results as the other polls, but if you want to be newsworthy, it’s better to get very different results. And since Mr Romney is highly likely to win the presidential nomination, an error that overestimates his popularity will be forgotten in the long run.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »