May 22, 2023

Sniffing out Covid accuracy?

By

This research, reported at Newshub, is pretty good: dogs are being evaluated for detecting Covid in reasonably large numbers of people at schools, and the results are actually published in a respectable peer-reviewed journal.  The numbers are still getting oversold a bit:

The study said [the dogs] were 83 percent accurate at identifying COVID-19-positive students, and 90 percent on the mark at picking out virus-negative students.

There are two issues here. First, 90% accuracy on virus-negative students is not at all good.  The current prevalence of Covid in NZ is maybe 1%, and the idea of dog-based-testing is for the subset of people who are possibly infected but apparently healthy, which will cut the numbers further.  Most of the people who test positive will be false positives, who will then need to be isolated for further testing.   Antigen tests have an accuracy in uninfected people (specificity) more like 99%.

Achieving 83% accuracy for identifying actual infections sounds good, and I’ve seen social media comments that it’s better than the antigen tests. It isn’t really: as the story says, the accuracy figure measures how well the dogs agree with the antigen tests.  83% accuracy means that the dogs are missing 17% of the infections that the antigen tests pick up.

If your school was doing regular testing of all students, this study suggests that you could reduce the amount of testing by using dogs to screen people first and only testing the kids who the dogs select. The accuracy would be lower than testing everyone, but it might be cheaper.   The study doesn’t really argue that dogs would be good on their own.

The story goes on to say

Dogs have also been using their powerful noses to detect other diseases such as cancer with shockingly high accuracy. 

Again, this is true (at least in a sense). Research studies, mostly small, have reported results of this sort for years. The fact that we aren’t currently using dogs to screen for any of these diseases suggests either that there are practical barriers to implementation, or that it doesn’t actually work all that well in practice.

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »

Comments

  • avatar
    Steve Curtis

    Dont forget another factor
    ‘after two months of training on COVID-19 scent samples in the laboratory..’
    or you could run a nasal swab rapid test.

    I know someone who visits a Bupa rest home every week and they had the antigen testing for all visitors from when they became available. Bupa stopped the testing after 4 months or so as they only had a few positives over the thousands of tests taken over all their locations.
    I dont think that highly trained dogs would replace a short test thats more accurate and are ready day or night 7 days a week.

    1 year ago