Posts from February 2022 (24)

Team Ratings for Week 1

A couple of comments for the start of this competition.

I have in the past have had reasonable success rating new teams at -10, and have done that this time. The Waratahs were a total disaster last year and rated -11. I don’t believe they will be worse than the new teams so did adjust their rating to -9. Even this may be viewing them a little negatively. Overall, be aware this is a new competition in a number of ways, and initially I may not achieve very good results.

I have predicted a result for the opening game involving Moana Pasifika, which seems unlikely to go ahead.

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Predictions for Week 1

Here are the predictions for Week 1. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 17

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 94 matches played, 51 of which were correctly predicted, a success rate of 54.3%.
Here are the predictions for last week’s games.

Predictions for Round 17

Here are the predictions for Round 17. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

For the past couple of years I’ve been arguing against Covid testing for people who don’t have symptoms and aren’t at high risk of exposure: they’ll have only a minute chance of testing positive, so we won’t learn anything, and we have better uses for the testing resources.  The only country that’s been doing systematic surveillance of Covid has been the UK, where the background prevalence has been, let’s say, somewhat higher than it had been here.

New Zealand is now getting a substantial Covid outbreak.  We’ll be over 1000 new cases some day soon, and it will start to matter for hospital planning purposes whether we’re detecting 20% of infections or 10% or 1% — because hospital numbers follow infection numbers with a long enough lag that the information is useful.

We’ve got two possible approaches to estimating the population Covid burden. One is wastewater testing, the other is random sampling.  Both approaches will keep working no matter how high the Covid prevalence is and no matter what fraction of infections are diagnosed and reported.  Sampling is more expensive, but has the advantage that it actually counts people rather than counting viruses and extrapolating to people.  Using both would probably help balance their pros and cons.

Sampling doesn’t have to be ‘simple random sampling’. If we know there’s more Covid in Auckland than in Oamaru, we can sample at a higher rate in Auckland and a lower rate in Oamaru.  We can also do adaptive sampling, where you take more samples in places where you find a hotspot.  Statistical ecologists trying to count plant and animal populations have studied this sort of problem quite a lot over the years — and statistical ecology is, fortunately, an area where NZ has expertise. But even simple random sampling would work, and would give us an estimate of infections and symptomatic cases across the country, and help plan the short to medium term response.

• Good discussion of the overinterpretation of opinion polls from Mediawatch. Hayden Donnell jokingly says “That is, of course, except for Mediawatch, which is the only truly objective outlet in town” — but like StatsChat, Mediawatch has the luxury of not commenting on stories where we don’t have anything to say.
• In contrast to well-conducted opinion polls, Twitter polls are completely valueless as a way of collecting and summarising popular opinion. This means that while they’re fine for entertainment (yay  @nickbakernz) and collecting reckons from your friends, it’s probably not a good idea to rage-retweet batshit political polls.  Let them get 37:0 in favour of banning arithmetic or whatever, rather than 37:1000 against.
• A summary of where the various non-profit Covid vaccines have got to, from Hilda Bastian
• One of the repeated themes of this blog is that you need to measure the right things if you’re going to base decisions on them.  The “Drug Harm Index” may not qualify here because it’s not clear decisions are made based on it, but it’s still worth looking at whether it measures harm the right way.  As Russell Brown points out, the index would say “that cannabis is New Zealand’s most harmful drug – accounting for $626 million in “community harm” every year. Would you be surprised if I told you more than a third of that was lost GST?”
• According to the MoH vaccination data, the vaccine roll-out for kids is going well on average, with 43% having had their first shot, but the differences by ethnicity are about the same as they were for adults. At the start of the Delta outbreak in August  (according to Hannah Martin at Stuff)  just over 40% of Aucklanders had had a first dose, 33% of Pacific people and 28% of Māori. That’s almost creepily close to the current situation with 5-11 year olds across the country now — the percentage for Māori being slightly lower this time.  Equity being a priority doesn’t seem to have had much impact.
  
• Interesting post from Pew Research on writing survey questions: in particular, ‘agree:disagree’ questions give you more ‘agree’ results than forced choice “pineapple or pepperoni” questions on the same issues.
• In New Zealand there are some issues with denominators for vaccination rates — the population that’s used undercounts minority groups.  This seems to be much worse in the UK: from Paul Mainwood on Twitter
  

Team Ratings for Postponed Games

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 111 matches played, 82 of which were correctly predicted, a success rate of 73.9%.
Here are the predictions for last week’s games.

Predictions for Postponed Games

Here are the predictions for Postponed Games. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Week 14

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 67 matches played, 46 of which were correctly predicted, a success rate of 68.7%.
Here are the predictions for last week’s games.

Predictions for Week 14

Here are the predictions for Week 14. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 16

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 88 matches played, 46 of which were correctly predicted, a success rate of 52.3%.
Here are the predictions for last week’s games.

Predictions for Round 16

Here are the predictions for Round 16. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 4

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 9 matches played, 7 of which were correctly predicted, a success rate of 77.8%.
Here are the predictions for last week’s games.

Predictions for Round 4

Here are the predictions for Round 4. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

The Herald and the Spinoff both commented on the Covid testing results yesterday. The Spinoff had a quick paragraph

While the tally of new cases is down, the test positivity rate is up. Yesterday’s report saw 21,471 tests and 243 positive cases – a one in 88 result; today it was 16,873 tests and 208 new cases: a one in 81 result.

and the Herald had a detailed story with quotes from experts

Experts believe Covid fatigue and a perception that Omicron is less of a threat than Delta are to blame for low testing numbers at the start of the community outbreak.

There were 100,000 fewer tests administered in the week following Omicron community transmission than the week following Delta transmission, Ministry of Health data shows.

They’re both right, but the Ministry of Health is not giving out the most helpful numbers or comparisons to understand how much it’s really a problem.

There are three basic reasons for testing: regular surveillance for people in certain high-risk jobs, testing of contacts, and testing of people with symptoms.  The number of surveillance tests is pretty much uninformative — it’s just a policy choice — but the proportion of positive tests is a strong signal.  The number of tests done for (not yet symptomatic) close contacts tells us about the effectiveness of contact tracing and about the number of cases in recent days (which we knew), but it doesn’t tell  us much else, and the positivity rate will mostly depend on who we define as close contacts rather than on anything about the epidemic.  The number of tests prompted by symptoms actually is an indicator of willingness to test, and the test positivity rate is an indicator of Covid prevalence, but only up to a point.

There’s another external factor confusing the interpretation of changes in symptomatic testing: the seasonal changes in the rate of other illnesses.  When Delta appeared, testing was higher than when Omicron appeared.  That could be partly because people (wrongly) thought Omicron didn’t matter, or (wrongly) thought it couldn’t be controlled, or (perhaps correctly) worried that their employers would be less supportive of being absent, or thought the public health system didn’t care as much or something.  It will also be partly because fewer people have colds in December than in August.

As a result of much collective mahi and good luck, most of the people getting tested because of symptoms actually have some other viral upper-respiratory illness, not Covid.  At times of year when there is more not-actually-Covid illness, testing rates should be higher. August is winter and kids had been at school and daycare; it’s the peak season for not-actually-Covid. December, with school out and after a long lockdown to suppress various other viruses, is low season for not-actually-Covid. Fewer tests in December is not a surprise.

Not only will more colds mean more testing, they will also mean a lower test positivity rate — at the extreme if there were no other illnesses, everyone with symptoms would have Covid. The two key testing statistics, counts and positivity rate, are hard to interpret in comparisons between now and August.

It would help some if the Ministry of Health reported test numbers and results by reason for testing: contacts, symptoms, regular surveillance. It would help to compare symptomatic testing rates with independent estimates of the background rate of symptoms (eg from Flutracker).  But it’s always going to be hard to interpret differences over long periods of time — differences over a few weeks are easier to interpret, preferably averaged over more than one day of reporting to reduce random noise.

None of this is to disagree with the call for people with symptoms to get tested.  We know not everyone with symptoms is tested; it’s probably been a minority throughout the pandemic. Getting the rate up would help flatten the wave of Omicron along with masks and vaccines and everything else.

Now we’re at Waitangi weekend we can confirm that New Zealand modellers and epidemiologists, none of whom expected 50,000 cases per day at this point, were correct.  Unfortunately, the Herald has

Questioned on earlier figures that up to 50,000 new cases would be emerging by Waitangi Day – and 80,000 a day a few weeks later – Hipkins described the calculations as useful, saying it was better to have some modelling than none.

Further down, the Herald piece admits that these figures didn’t come from the New Zealand modellers that the Minister is paying and being advised by, but from IHME in Seattle. It’s worse than that, though. The only place I saw tens of thousands of cases as a description of the modelling by the IHME in Seattle was in a Herald headline.

All the other reporting of it that I saw at least said “infections”, even if they weren’t clear enough that this wasn’t remotely the same as cases. 

As you can see, the IHME model prediction for reported cases today, Sunday 6 February, was actually 332 (or 202 with good mask use), even though the projection for infections by tomorrow was nearly 50,000.

The uncertainty interval for that projected 332 went from 85 to nearly 800, so the actual figure was well inside the predicted range.

You might think that this sort of accuracy still isn’t very good. Projecting the timing of the epidemic is hard — think of the exponential-spread cartoon from Toby Morris and Siouxsie Wiles

Especially early on in an outbreak, individual choices and luck can make a big difference to how fast the outbreak spreads.  Eventually it will be overall patterns of vaccination and masking and distancing and isolation that matter for the overall outbreak size. The models will be more accurate as the outbreak gets bigger and less random, and they will likely be more accurate about total outbreak size than about timing.

I’m not a fan of the IHME models — they have notoriously been overly optimistic in the medium to long term in the US — but Michael Baker and the Otago group think they’re reasonable, and you should arguably listen to them rather than me on this topic.  We’ll find out soon. Whatever you think of them in general, though, the modellers certainly didn’t predict 50,000 cases by today, and shouldn’t be criticised for failing to predict something that didn’t happen.