Drug driving: dodgy numbers in a good cause?
More than a year ago, ESR scientists produced a report on drugs and alcohol found in blood samples taken after fatal crashes. Now, the Drug Foundation is launching a publicity campaign using the data. Their website says “Nearly half of drivers killed on New Zealand roads are impaired by alcohol, other drugs, or both.” But that’s not what the ESR report found. [Edited to add: the Drug Foundation is launching a campaign, but the TV campaign isn’t from them, it’s from NZTA]
The ESR report defined someone as impaired by alcohol if they had blood alcohol greater than 0.03%, and said they tested positive for other drugs if the other drugs were detectable. If you look at the report in more detail, although 351/1046 drivers had detectable alcohol in their blood, only 191/1046 had more than 0.08%. At 0.03% blood alcohol concentration there may well be some impairment of driving, and near 0.08% there’s quite a lot, but we can’t attribute all those crashes to alcohol impairment rather than inexperience, fatigue, bad luck, or stupidity. At least the blood alcohol concentrations are directly relevant to impairment. An assay for other drugs can be positive long after the actual effect wears off. For example, a single use of cannabis will show up in a blood test for 2-3 days, and regular use for up to a week. In fact, the summary of the ESR report specifically warns “Furthermore, it is important to acknowledge that the presence of drugs and alcohol in the study samples does not necessarily infer significant impairment.” Regular pot smokers who are scrupulously careful not to drive while high would still show up as affected by drugs in the ESR report. In fact, the Drug Foundation makes this distinction when they talk about random roadside drug testing, pointing out the advantages of a test of actual impairment over a test of any presence of a drug.
The Drug Foundation also did a survey of community attitudes to driving while on drugs (also more than a year ago), and it is interesting how many people think that stimulants and cannabis don’t impair their driving. However, if you look at the survey, it turns out that it was an online poll, and “Respondents were recruited to the online survey via an advertising and awareness campaign that aimed to stimulate interest and participation in the study.” Not surprisingly, younger people were over-represented “The mean age of respondents was 38.1 years”, as were people from Auckland and Wellington. Maori, Pasifika, and Asians were all under-represented. 36% of respondents had used cannabis in the past year, more than twice the proportion in the Kiwi population as a whole. No attempt was made to standardise to the whole NZ population, which is the fundamental step in serious attempts at accurate online polling. [If we could use the data as a teaching example, I’d be happy to do this for them and report whether it makes any difference to the conclusions]
And while it’s just irritating that news websites don’t link to primary sources, it is much less excusable that the Drug Foundation page referencing the two studies doesn’t provide links so you can easily read them. The study reports are much more carefully written and open about the limitations of the research than any of the press releases or front-line website material.[The NZTA referencing is substantially less helpful]
For all I know, the conclusions may be broadly correct. I wouldn’t be at all surprised if many drug users do believe silly things about their level of impairment. Before the decades of advertising and enforcement, a lot of people believed silly things about the safety of drunk driving. And the new TV ads are clever, even if they aren’t as good as the ‘ghost chips’ ad. But the numbers used to advertise the campaign don’t mean what the people providing the money say they mean. That’s not ok when it’s politicians or multinational companies, and it’s still not ok when the campaigners have good intentions. [Edited to add: I think this last sentence still stands, but should be directed at least equally at the NZTA].
[Update: Media links: TVNZ, 3 News, Stuff, NZ Herald, Radio NZ]
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
I really hope that we someday get the police to release stats on their breath checks. I want to know what proportion of the driving population registers different BAC concentrations for those who AREN’T in accidents.
If we don’t know the base rates, with time of day adjustments, we have no clue whether folks with BAC 0.05-0.079 are proportionately or disproportionately represented in crash statistics.
13 years ago
And this isn’t the first time NZ Drug’s been a bit off. Here’s my critique of their take on the J-curve in alcohol. In short, the early lit confounded never-drinkers with former drinkers, so health benefits of moderate drinking could have been an artefact of sick quitters being pooled with abstainers. So NZ Drug, and others in NZ, jumped on this as reason to dismiss the J-Curve. Problem is that, for at least the last 6 years, there have been plenty of studies that are far more careful in separating former drinkers from never-drinkers. Rimm & Moats is great on cardiovascular effects; Castelnuovo & Donati have a very nice meta-study on overall mortality. And while correcting for the the sick quitter effect mildly attenuates results, it hardly eliminates them.
And yet, here’s NZ Drug’s current statement.
http://www.drugfoundation.org.nz/alcohol/health-benefits
If I’d have to guess, the NZ pub health guys are terrified of conveying the idea that there can be health benefits from drinking 1-2 drinks per day for fear of encouraging the folks who are having 7-8 – the noble lie…
13 years ago
The Drug Foundation seems to be assuming that P(alcohol, drugs | crashing) is equal to P(crashing | alcohol, drugs) for the purposes of running their campaign. Accident or data manipulation?
13 years ago
Luis,
It’s not that bad. The implicit argument is that P(alcohol, drug| crashing) is higher than any reasonable estimate of P(alcohol, drug). Since crashing is rare, P(alcohol, drug| crashing)/P(alcohol, drug) would be close to the relative risk.
There is still a confounding problem, eg by age and time of day, but at least if you leave out the lower levels of blood alcohol, I still think the P(alcohol, drug|crashing) is higher than I would expect for P(alcohol, drug|young, night time).
13 years ago
@Thomas: Agreed. But we need the increase over base rate to start doing the cost-benefit analysis. Because there are real costs involved in lowering the legal limit from .08 to .05.
Here’s one research design proposal, if the data existed…
13 years ago
It would be a good start just to collect the data to make the relative-risk estimates. That at least would be relatively uncontroversial.
The cost data is going to be much harder: as you say, you want to capture consumer surplus from drinking, but you might also want to capture the benefit from knowing there are fewer drunk drivers out there, and so on [well, you might not, but I but some people would]. Also, at the higher end of the drinking levels you run into the problem that consumption of addictive substances is one area where neither expressed nor revealed preference need be a good guide to utility.
13 years ago
I’d be happy to count psychic benefits to non-drinkers on the road from knowing there are fewer drunks out there if we also counted psychic costs to light drinkers suddenly worried about whether they’re over the line when they’re actually reasonably under. Neither of these can really be measured.
The first order effects would be the road toll cost reductions and the loss in consumer surplus among moderate drinkers (I’d expect your very reasonable concerns about heavy drinkers would be more relevant to the >0.08 cohort than to the folks in the 0.05-0.08 area). I don’t know which way things would run ex ante.
13 years ago
There’s an Australian population-based case-control study using a random sample of drivers. There’s potential for non-response bias, of course, but probably less at the lower levels of drinking.
13 years ago
Downside of that one is the light drinking range runs 51-100 when we’d need the 51-80 range to tell us anything about benefits of reducing from 80 to 50. And I’d trust compulsory police random stops, already being conducted, as data source over voluntary self-report.
13 years ago
Yes, it’s not perfect, but it’s better data than I have seen before.
13 years ago
Hi – thanks for the post on our drug drive stuff. A quick point of clarification -the campaign launching this weekend is by NZTA; we’ve earlier launched a more targeted drug driving project http://www.drivinghigh.org.nz.
Regarding your post of our online survey done in 2009 – we’ll do a response when we’re back from summer holidays.
Cheers
13 years ago
My apologies for not distinguishing the two organizations.
I think all the numbers and arguments I’m quoting do come from the Drug Foundation, although the post was certainly motivated by the publicity push on the NZTA ads, and so was the irritation level.
13 years ago
I think this is worth developing into a teaching experience at NCEA Level 2/curriculum level 7..anyone want to join forces to develop it?
13 years ago