Reports about young women and binge-drinking: a caution

Local media have been proclaiming that younger women are binge-drinking themselves into oblivion, (examples here and here),  many of these stories leaning on Canadian journalist and recovering alcoholic Ann Dowsett Johnston’s book Drink: The Intimate Relationship Between Women and Alcohol.  She says the percentage of college students who binge drink, using the measure of consuming five or more drinks in one sitting, was nearly 45% in 2011, and repeated this in a recent Wall Street Journal item called The New Face of Risky Drinking is Female.

However, there is an editor’s note at the bottom of her piece (that probably should be at the top), pointing out that Johnston got her compelling  figure by combining the results  of two separate studies carried out in different years.

The  non-profit, non-partisan Statistical Assessment Service (STATS) in the US points out that a basic tenet of Statistics 101 is that one should never directly compare survey results that come from different populations or that were provided from surveys taken with different methods, even if the surveys are angling for the same kind of data.  Surveys can return different results for a variety of reasons, from the time of year in which they are administered, to the population from which participants are chosen, and even to how the questions are asked. In this case, it says, both surveys  have systematic biases that come from their survey methods, which in turn makes direct comparisons problematic. See its useful analysis of the situation here.