Are we number two on COVID?

If you’ve been paying attention to the news you might have read that Forbes ranked NZ second in the world for COVID safety. If you chose more careful news sources, that you might instead have read that Forbes published a story about COVID safety ranking constructed by a company called Deep Knowledge Group.  It’s their second try; they published the first round in June.  And, of course, you might remember stories from the Before Times about New Zealand’s  poor ranking on the Global Health Security Index.

So, which of these indexes are right and which ones are wrong? It turns out that’s the wrong question, just as it is about rankings  of the most liveable city. These rankings aren’t trying to be predictive in a way that makes ‘right’ and ‘wrong’ objectively assessable.  What happens is that a group of people gathers together a whole lot of measurements.  These measurements range from ‘good’ to ‘bad’ on some scale. Points get awarded for how close you are to ‘good’ on each measurement and then added up. The experts then look at the rankings they got out, and probably adjust the points a bit to  make them look more plausible — there’s nothing wrong with this; the experts’ judgment is where there’s potentially value added by having this index.

There is a technical problem with reducing things to a sum of numbers in this way — and I say that as someone who likes numbers. We might all agree that, say, willingness to follow mask recommendations is positive, and that having a policy of mask recommendations when appropriate is positive. But these don’t add up: you need both of them.   Being more willing to follow government advice to wear masks does no good if the government is not giving that advice; having the government give the advice does no good if it isn’t followed.  You can’t represent A AND B as a weighted sum.  Weighted sums are a very limited subset of ways of combining ordered scales.

It’s clear how much opinion goes into the index construction when you see that the Global Health Security Index and the new COVID index share only two countries in their top ten: Australia and South Korea.  The United States and the UK top the GHS Index; Germany and NZ top the Deep one.  The real question is whether the indexes are useful, and they probably are — or, more precisely, whether the reports surrounding the indexes are useful, where the experts talk about the ways countries differ and their strengths and weaknesses.

If you wanted to think of the indexes as predictive, you’d want to look at the June edition and see how it’s fared.  Most of the top ten have done ok, but Israel, at number 3,  has had a bad few months, and now has 27,000 active cases in a country with less than twice the population of New Zealand.  They have been demoted to number 11: four places ahead of Taiwan, 21 ahead of Vietnam, and 83 ahead of Mongolia.

The index and its surrounding report contain useful information; for example they identify (admittedly with 2020 hindsight) the importance of political will to act and social acceptance of restrictions.  The ranking of NZ as the second-safest country is only meaningful if you don’t interpret ‘safest’ in some crudely reductive way as referring to the likely number of cases or hospitalisations or deaths.