Measuring accuracy
A new scientific test is able to detect which 14-year-olds will become binge drinkers by the time they hit 16.
A study published in the journal Nature describes how scientists have developed a system that weighs up a range of risk factors and predicts – with about 70 per cent accuracy – which teens will become heavy drinkers.
That’s true, but the definition of accuracy is doing quite a bit of work here.
We don’t have figures for 16 year olds, but according to the Ministry of Health about 20% of 15-17 year olds have ‘hazardous drinking patterns.’ That means I can predict with 80% accuracy without even needing to weigh up a range of risk factors — I just need to predict “No” each time. Parents, teachers, or people working with youth could probably do better than my 80% accuracy.
The researchers found that their test correctly classified 73% of the non-binge-drinker and 67% of the binge drinkers, which means it would get 72% of people classified correctly. That’s rather worse than my trivial “the kids are ok” predictor. In order to be any use, the new test, which combines brain imaging and detailed interviews, needs to be set to a higher threshold, so it predicts fewer drinkers. The researchers could have done this, but they didn’t.
Also, in order to be any use, the test needs to identify a group who will selectively benefit from some feasible intervention, and there needs to be funding to supply both this intervention, and the cost of doing long interviews and fMRI brain imaging on large groups of teenagers. And that needs to be the best way to spend the money.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
It’s even more misleading in The Press. The article says
“Scientists have developed a test that can *accurately* [emph. mine] predict which teenagers will become binge drinkers by the age of 16.”
The next sentence has “… was found to be 70 per cent accurate at predicting …”
I presume people believe that none of their children are going to be labelled like this.
10 years ago