How could we get a vaccine by November?

There are about forty COVID vaccine candidates now, with several heading to Phase 3 trials.   The  CDC has suggested there could be a vaccine before the US presidential election. How could that happen? There’s one obvious answer, sadly, but how else could it happen?

The FDA has said it would expect that a COVID-19 vaccine would prevent disease or decrease its severity in at least 50% of people who are vaccinated.  In the current circumstances, if your vaccine was that effective it would be false economy to run a trial that was too small — you’d want to be very confident of a positive result if your vaccine met the mark.

You might design a trial that ran until you had seen, say, 150 infections or symptomatic cases or whatever. If you ended up with a split of 50 vs 100, you’d estimate a 50% reduction with the vaccine, and the uncertainty interval (‘margin of error’) in would stretch to a 30% reduction, which correspond to the FDA’s guidance.   Only 150 cases perhaps doesn’t sound like much, but remember that at a very high infection rate of 1% you’d need 10,000 people in the control group and the same number in the treatment group. That’s a big trial.

It’s possible, though, that your vaccine is much better than just a 50% reduction in cases. If it actually prevents 90% of cases, you could have run a much smaller trial, at lower cost, finished earlier, and saved more lives.  On the other hand, if you ran the smaller trial and the effectiveness is only 50%, you are in big trouble — you won’t get approved, but you’ll find it harder to run another trial.

So, what we do is to set up a big trial that can detect a 50% benefit, and peek at the data part of the way through.  If the vaccine turns out to be 90% effective, you’ll be able to stop earlier than planned and declare victory.  You’ll still have recruited more people than if you were betting everything on 90% effectiveness, but you won’t be betting everything on 90% effectiveness.

Stopping early requires strong evidence.  It requires strong evidence because it’s easy to get fooled by randomness in early trends. It requires strong evidence because early stopping means less data on safety. And it  requires strong evidence because people are going to suspect political interference.  One popular guideline for designing early-stopping rules, to give us something concrete to look at, says that you take margin you’d need to be convinced at the end of the trial, and use that margin earlier on.  In our hypothetical example, we had 50 cases difference as the success criterion at the end of the trial, so you’d look for 50 cases difference at earlier timepoints as well.  That’s a much bigger relative difference early on:  if you only had 100 cases, a difference of 50 is 25 vs 75; with only 60 cases, it’s 5 vs 55.

My understanding is that a 90%-effective vaccine would be surprising, but not inconceivable and a 70%-effective vaccine isn’t at all unlikely, so interim analysis and the potential for early stopping do make sense.  Political interference doesn’t help, though: even more than most drugs, a vaccine only counts as effective if lots of people are willing to take it, and a trial that doesn’t engender that willingness is a failure no matter the statistical results.