If you see a fork in the road
From Pew Research, a bizarre error
A few Reddit users shared screenshots from a variety of surveys, where questions that should have offered answer options of “yes” and “no” instead offered the choices “forks” and “no.”
In summary: web browsers may offer you the option to translate web pages automatically from languages you don’t read to languages you do read. That is, they have the option to silently rewrite web pages before showing them to you. They aren’t 100% reliable either in guessing which web pages are actually in a foreign language and need translating or in translating those pages to, as it might be, English.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
Comments
Add a comment
First time commenting? Please use your real first name and surname and read the Comment Policy.
“because we went through the survey dozens or hundreds of times before we saw the issue replicated, we had reason to believe that it was seen by a tiny minority of those who took our survey”
I smell a sampling bias here. Pew Research employees’ browser settings might not be representative of the general public’s
3 days ago
It doesn’t matter, though: they aren’t trying to estimate any population quantity here, they are trying to capture an instance of the fork-up for analysis. Doing your forensic analysis at the crime scene isn’t sampling bias.
2 days ago
A similarish thing happened recently in an online trivia competition: a question that was written entirely in Spanish was auto-translated by some browsers into English, making it exceedingly easy for an unknowable number of players. In the end it was awarded as correct for everyone.
2 days ago