Causation and fair comparisons

This is a version of a graph I saw on Twitter, posted by something who I think was trolling. The purple arrows are the lockdown decisions; the curve is the number of active COVID cases in New Zealand.  As you can see both lockdowns have been followed by a clear increase in the number of active cases, and no such increase has occurred any time when we haven’t imposed a lockdown. So, lockdowns cause COVID? Yeah nah.

Some of you are probably gearing up to say “correlation isn’t causation”; yes, well done. But that’s not the issue here. The relationship between number of active cases and lockdown is not a coincidence. There is a direct causal relationship. It just goes the other way: outbreaks cause lockdowns.

If we’re trying to estimate the effect of the California fires or the potential Gulf of Mexico hurricanes on COVID cases, it does make sense to compare infections shortly after and shortly before the event.  It obviously doesn’t for lockdown, but what (apart from “I know it when I see it”) is the distinction?

Economists would say the lockdown is endogenous (it’s coming from inside the epidemic). Epidemiologists, who have a more detailed taxonomy of bias, would talk about confounding by indication. People who take blood pressure drugs tend to have higher blood pressure than those who don’t; someone with  a headache is more likely to have taken paracetamol than someone without a headache. Interventions look bad precisely because you use them when they’re needed.  My bedroom tends to be warmer when the air conditioning is on (in summer) than when the heat pump is on (in winter).

We need a fair comparison to what actually happens after lockdown, and it isn’t business as usual.  This is where a model is useful.  We know roughly what happens to COVID case numbers with no intervention, because we have a fairly good mathematical model for how the disease spreads.  With no intervention, the number of new cases wouldn’t peak early and decline; it would keep going up.  With alternative, milder, interventions we’d need models on both sides of the comparison.  We have some data to validate the models, including genome sequencing to confirm which people were really infected as part of the same cluster, but the model does a lot of the work.

So, yes, we really can conclude that a New Zealand-style lockdown has worked.  This doesn’t mean it would work everywhere — just having the government say “lockdown” doesn’t do anything unless people cooperate — but the comparison to what we’d expect without it is evidence to say it worked here.

You get the same sort of problems in estimating the cost of lockdowns.  The cost compared to business as usual is relatively easy to estimate. That’s even a fair comparison for some policy questions: if we’re evaluating how much money it’s worth spending on infection control at the border, it’s a useful benchmark to know that a two-week Auckland lockdown won’t leave you much change out of a billion dollars.

But if you have an outbreak already and you want to estimate the cost of a lockdown compared to no lockdown, you can’t do a fair comparison to business as usual.  The economy will suffer during a prolonged outbreak: people will be reluctant to eat out or go to movies or rugby; jobs will be lost; less money will be available for spending.  Even before you add in the economic value of health, just the economic value of the economy will be down.  If you want to talk about the economic cost of the lockdown vs just letting the coronavirus run free, you need to do that comparison.  You can’t just compare to business as usual, any more than you can compare to business as usual and decide that lockdowns cause outbreaks.