Posts from August 2020 (28)

Team Ratings for Round 17

I got caught out a bit this round with games being played earlier in the week and then again on the weekend. As usual my predictions are what they would have been before the games were played, the predictions are produced by running my algorithm over the past data and information about home grounds.

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 96 matches played, 61 of which were correctly predicted, a success rate of 63.5%.
Here are the predictions for last week’s games.

Predictions for Round 17

Here are the predictions for Round 17. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Auckland is going back to level 2 (God willing and the creek don’t rise) on Monday.  Together with my local circle of health and stats nerds, I’m viewing this with some concern. It hasn’t been very long since we had a case show up with no previously known contact to the cluster. Like, Tuesday.  It’s quite possible there are still a few other people in the cluster who haven’t been found yet.

The concern is not that it will actually be dangerous on Monday to be going to work or going shopping. The number of undetected cases will be small; your chance of getting infected on Monday is tiny.  The problem is, as long as there are undetected, infectious cases, your chance of getting infected on Tuesday is very slightly higher. And slightly higher again on Wednesday, and so on in exponential increase. Eventually, it may get dangerous, and  stopping it then is much more costly in health and freedom and money.  In Victoria today they are talking  about the psychological boost of getting a day with less than 100 new cases. If we’re careful and lucky, the current testing and tracing will be enough to stop this cluster exploding; if we’re not, maybe not.

One challenge in COVID risk communication is that the risk is longer-term and social, not immediate and individual. It’s important that as many  people as possible take precautions against spreading the virus: distancing, masks, getting tested if you have symptoms, working from home if that’s feasible, avoiding places with poor ventilation. But it’s not so much important for your or your family’s immediate safety: the risk is currently very low. If someone jogs past you at close range without a mask or stands a bit too close in  a supermarket line, you don’t need to panic — you almost certainly won’t catch the coronavirus. On the other hand, the more people behave that way, the more chance that they outbreak will slowly get out of control.  The appropriate level of care is a lot higher than the appropriate level of fear.

Ideally, everyone would be as careful as if the virus was everywhere, but nowhere near as scared as if it was everywhere. That’s a very difficult balance, and a reason for the ‘be kind’  message when it doesn’t quite work out.

Q: Did you see this study saying vaping makes you five times more likely to get COVID?

A: Yes, but it’s not in the news, so it doesn’t count for StatsChat

Q: Newshub covered it.

A: Ok. Not entirely convinced

Q: They did a survey and they did lots of reweighting, the way you like. And said exactly what questions they asked.

A: Yes…

Q: So it’s not dodgy like the paper about heartburn drugs

A:  No, not like that.

Q: What’s your problem, then

A: The first problem is the proportion of people with COVID tests. No, actually the first problem is that COVID is so rare that this isn’t a reliable way to estimate proportions, and the second problem is that getting a test depended on a lot of other factors back then.  The third problem is the proportion of people with COVID tests

Q: Which is?

A:  Over 5% of people 13-17 and over 10% of people 21-24. By May 14, when the total cumulative number of tests in the whole US was only about 5% of the population — and you’d expect lower testing rates in younger people.

Q: Where are those numbers in the paper?

A: It’s a combination of the user and non-user columns in Table 1, using the proportions in the Supplementary Material. Which, again, the authors should get credit for providing.  What they call “COVID-related symptoms” are also very high: 14% of non-vapers and 26% of vapers reported having the symptoms right at the time they were surveyed.

Q: You’d think we would know if vaping increased these symptoms that much, separately from COVID. But if they oversampled people at high risk of COVID, it should at least be comparable across their survey

A: They did separate surveys for users and non-users of e-cigarettes, so that’s not actually obvious.

Q: But weighting?

A: Yes, but that doesn’t help as much with matching the surveys to each other, especially as they don’t have separate census totals for vapers and non-vapers.  In particular, in mid-May COVID was concentrated in relatively small areas of the US, and it would have been more valuable to make sure the locations matched up.

Q: But we know that smokers are at higher risk of catching the coronavirus, so this just confirms that.

A: Surprisingly, no.  Smokers don’t seem to be at higher risk of getting infection — and I guarantee that it’s not because no-one tried to show they were.  They may be at higher risk of getting seriously sick if they are infected, but even that’s not as clear as you’d expect.

Q: So should we believe this?

A: It’s not as simple as that.  This study does provide some evidence, but not as strong evidence as the researchers think. It certainly isn’t strong enough evidence to change policy on the regulation of e-cigarettes; whatever  you believed about that before seeing this study, you should believe about the same afterwards. And you probably do — it’s not a topic where people are noted for changing their minds.

Team Ratings for Round 9

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 16 matches played, 13 of which were correctly predicted, a success rate of 81.2%.
Here are the predictions for last week’s games.

Predictions for Round 9

Here are the predictions for Round 9. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 16

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 90 matches played, 58 of which were correctly predicted, a success rate of 64.4%.
Here are the predictions for last week’s games.

Predictions for Round 16

Here are the predictions for Round 16. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 15

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 95 matches played, 72 of which were correctly predicted, a success rate of 75.8%.
Here are the predictions for last week’s games.

Predictions for Round 15

Here are the predictions for Round 15. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 16

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 120 matches played, 83 of which were correctly predicted, a success rate of 69.2%.
Here are the predictions for last week’s games.

Predictions for Round 16

Here are the predictions for Round 16. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Staged debate, as a format, has problems for actual discussion. So I wasn’t expecting the Stuff for/against on the COVID card to be more informative than the various discussions I’ve seen on Twitter.

This, from Ian Taylor was more than I’d anticipated, though

The Government has budgeted $210m for the 2023 census. That’s $210m to get a piece of paper to every New Zealander.

If you were a nasty suspicious person you might think I’ve quoted this out of context, and cut out all the things apart from the mail-out that go into making the value of the census over a billion dollars (PDF) — developing a sampling frame for dwellings, the hardware and software computer systems, data entry and validation,  monitoring of response rates, employing people to go door-to-door to catch up on non-response, the post-census enumeration survey, estimation of under-coverage, imputation of missing data, and so on.

You might think I’d left those out. But I didn’t. Describing the next Census as $210m to get a piece of paper to every New Zealander is like describing the COVID card as $100m to get a lanyard to every New Zealander. It leaves out all the stuff that makes it work.

And it’s not as if there aren’t other, more relevant, comparisons to make. A better comparison for the $100m cost of the COVID card would be the cost a of a few days more for Auckland at Level 3. If the COVID card could save us a week at level 3 in total, over the next couple of years, and there isn’t another solution that would be better or cheaper, then it easily makes sense.

I’m basically in favour of Bluetooth proximity measures as an adjunct to tracing in the current situation of mostly-successful elimination. I don’t think they come anywhere close to allowing us to relax the isolation/quarantine process, as some people had suggested earlier.

For the COVID card in particular I’d like to see some evidence about realistic fractions of people carrying the thing a year from now, and about how many false-positive ‘close contacts’ it generates. This information might exist, but it hasn’t been pushed by proponents. I’m not convinced by the opposing argument that it will take months to roll out:  the optimistic estimates for mass vaccination are probably eighteen months away; we’ve got plenty of time to improve. I think there are questions about cost and reliability and acceptability of COVID card relative to other Bluetooth and non-Bluetooth options, but I’ll leave them to the engineers and designers (preferably people who won’t simply dismiss any reluctance to wear the thing as ‘fashion’).  

I’ve had two people already complain to me on Twitter about the Stickybeak polling at The Spinoff. I’m a lot less negative than they are.

To start with, I think the really important distinction in surveys is between those that are actually trying to get the right answer, and those that aren’t.  Stickybeak are on the “are trying” side.

There’s also a distinction between studies that are really only making an effort to get internally-valid comparisons and those that are trying to match the population.  Internally-valid comparisons can still be useful: if you have a big self-selected internet sample you won’t learn much about what proportion of people take drugs, but you might be able learn how the proportion of cannabis users trying to cut down compares with the proportion of nicotine users trying to cut down, or whether people who smoke weed and drink beer do both at once or on separate days, or other useful things.

Stickybeak are clearly trying to get nationally representative estimates (at least for their overall political polling): they talk about reweighting to match census data by gender, age, and region, and their claimed secret sauce is chatbots to raise response rates for online surveys.

Now, just because you’re trying to get the right answer doesn’t mean you will. There are plenty of people who try to predict Lotto results or earthquakes, too.  And there, it’s too soon to say.  We know that online panels can give good answers: YouGov has done well with this technique, where their respondents are not necessarily representative, but they have a lot of information about them.   We’re also pretty sure that pure random sampling for political opinion doesn’t work any more; response rates are so low that either quota sampling or weighting is needed to make the sample look at all like the population.

So what do I think?  I would have hoped to see more variables used to reweight (ethnicity, and finer-scale geography), with total sample size larger, not smaller, than the traditional polls.  I’d also like to see a better uncertainty description. The Spinoff is quoting

For a random sample of this size and after accounting for weighting the maximum sampling error (using 95% confidence) is approximately ±4%.

The accounting for weighting is not always done by NZ pollsters, so that’s good to see, but ‘For a random sample of this size’ seems a bit evasive.  Either they’re claiming 4% is a good summary of the (maximum) sampling error for their results, in which case they should say so, or they aren’t, in which case they should stop hinting that it is.    Still, we know that the 3.1% error claimed by traditional pollsters is an underestimate, and they largely get a pass on it.

If you want to know whether to trust their results, I can’t tell you. Stickybeak are new enough that we don’t really know how accurate they are.

This is a version of a graph I saw on Twitter, posted by something who I think was trolling. The purple arrows are the lockdown decisions; the curve is the number of active COVID cases in New Zealand.  As you can see both lockdowns have been followed by a clear increase in the number of active cases, and no such increase has occurred any time when we haven’t imposed a lockdown. So, lockdowns cause COVID? Yeah nah.

Some of you are probably gearing up to say “correlation isn’t causation”; yes, well done. But that’s not the issue here. The relationship between number of active cases and lockdown is not a coincidence. There is a direct causal relationship. It just goes the other way: outbreaks cause lockdowns.

If we’re trying to estimate the effect of the California fires or the potential Gulf of Mexico hurricanes on COVID cases, it does make sense to compare infections shortly after and shortly before the event.  It obviously doesn’t for lockdown, but what (apart from “I know it when I see it”) is the distinction?

Economists would say the lockdown is endogenous (it’s coming from inside the epidemic). Epidemiologists, who have a more detailed taxonomy of bias, would talk about confounding by indication. People who take blood pressure drugs tend to have higher blood pressure than those who don’t; someone with  a headache is more likely to have taken paracetamol than someone without a headache. Interventions look bad precisely because you use them when they’re needed.  My bedroom tends to be warmer when the air conditioning is on (in summer) than when the heat pump is on (in winter).

We need a fair comparison to what actually happens after lockdown, and it isn’t business as usual.  This is where a model is useful.  We know roughly what happens to COVID case numbers with no intervention, because we have a fairly good mathematical model for how the disease spreads.  With no intervention, the number of new cases wouldn’t peak early and decline; it would keep going up.  With alternative, milder, interventions we’d need models on both sides of the comparison.  We have some data to validate the models, including genome sequencing to confirm which people were really infected as part of the same cluster, but the model does a lot of the work.

So, yes, we really can conclude that a New Zealand-style lockdown has worked.  This doesn’t mean it would work everywhere — just having the government say “lockdown” doesn’t do anything unless people cooperate — but the comparison to what we’d expect without it is evidence to say it worked here.

You get the same sort of problems in estimating the cost of lockdowns.  The cost compared to business as usual is relatively easy to estimate. That’s even a fair comparison for some policy questions: if we’re evaluating how much money it’s worth spending on infection control at the border, it’s a useful benchmark to know that a two-week Auckland lockdown won’t leave you much change out of a billion dollars.

But if you have an outbreak already and you want to estimate the cost of a lockdown compared to no lockdown, you can’t do a fair comparison to business as usual.  The economy will suffer during a prolonged outbreak: people will be reluctant to eat out or go to movies or rugby; jobs will be lost; less money will be available for spending.  Even before you add in the economic value of health, just the economic value of the economy will be down.  If you want to talk about the economic cost of the lockdown vs just letting the coronavirus run free, you need to do that comparison.  You can’t just compare to business as usual, any more than you can compare to business as usual and decide that lockdowns cause outbreaks.