Some numbers on testing

At the moment, NZ policy is to test everyone with suitable symptoms consistent with COVID-19, and potential contacts of cases, but not to go around randomly bothering healthy people for community surveillance.  Looking at some numbers explains why that’s a good strategy, and also gives a way to think about what it takes for other surveillance strategies to be useful.

The first question is how well you do just by testing symptomatic people, when we know some people never develop symptoms, and other people only develop symptoms after passing on the virus.  Professor Nick Wilson and various co-workers^* studied this problem back in May.  They did a lot of computer simulations of what would happen if you introduced one COVID case to ‘an island nation’ where the coronavirus had been eliminated but there was still widespread testing.  Under the assumption that about 40% of cases ended up getting tested, they found that an outbreak had a fifty-fifty chance of being detected when there were only six active cases, but that a reasonable worst case was 50-100 active cases at the time of detection.

You can disagree with the particular assumptions being made (they did this way back in May, the coronavirus equivalent of the Sony Walkman era) but it’s a reasonable ballpark guide.   The idea is that you maybe don’t routinely test absolutely everyone with a cold, but you do test everyone with some (new or worsening) respiratory symptom plus shortness of breath, or fever, or loss of sense of smell, or various other combinations.  We’ve gotten lucky: flu-like illnesses are much less common so far this year than in a usual year, so right now we don’t need to test as many people as they modelled.

So, taking the reasonable worst case, suppose at some point there are 50-100 people out there in NZ with coronavirus and we’ve been unlucky enough that none of them got tested (or a few got tested and the tests were false negatives).  How many random people would we need to test to pick up this outbreak?

Fifty people in NZ is one person in 100,000, so we’d need to test about 100,000 people to have a chance of finding a case.  A simple statistical rule of thumb says that testing about 300,000 people would make us pretty sure to find a case.  Outbreaks grow fast;  if there’s an outbreak with 50 people this week, it had maybe  15 people last week. To get a worthwhile improvement in detecting even the worst outbreaks we’d need to test 100k-300k healthy people each week. That isn’t happening.  Random community testing could be useful if we knew where to look. If we had one suspected case in a town of 10,000 people it might be worth just testing as many people as we could, to try to  get ahead of the contact-tracing process. But if you don’t know where to look there isn’t much point in looking.

Sewage testing is another promising possibility for picking up outbreaks, but these numbers show that it’s not going to be easy.  The testing has to be reliable enough that we’d be prepared to take some fairly major and expensive actions based on finding the virus, but sensitive enough to pick up just 50 or so cases.  It currently isn’t clear whether or not that’s possible, but ESR have the expertise (and some funding) needed to work on the question.  Reliable wastewater testing would be very helpful in the situation we’re now in, where there’s a  suggestion of transmission in NZ but not good evidence — but unreliable wastewater testing would just make things worse.

The take-home message is that we’re probably going to find the next outbreak by testing someone with symptoms. That person might very well have no known contact with international travellers.  If you might be that person, you should call Healthline to ask about getting tested.

^* Statisticians will recognise Matt Parry; any Kiwi who hasn’t been hiding in a cave on Mars with their fingers in their ears should recognise Ayesha Verrall and Michael Baker.