Posts written by Thomas Lumley (2569)

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

July 4, 2021

Maths

Q: Did you see that learning maths can affect your brain?

A: Well, yes. There wouldn’t be much point otherwise

Q: No, biochemically affect it

A: Yes, that’s how learning works

Q: But the researchers “can actually guess with a very good accuracy whether someone is continuing to study maths or not just based on the concentration of this chemical in their brain.”

A: How good is “very good”?

Q: I thought I was the one who asked those questions.

A:

Q: How good is “very good”?

A: The research paper doesn’t say

Q: Can you get their data?

A: <downloading and analysis noises>

A: Ok, so in their data you can get about 66% accuracy, 55 correct out of 83, using this brain chemical

Q: And just by guessing?

A: 56% (46 out of 83)

Q: Are these changes in the brain good? I mean, apart from learning maths being good for learning maths?

A: That seems to be assumed, but they don’t explain why

Q: And what about subjects other than maths?

A: The Herald piece says the differences they saw with maths don’t happen with any other subject, but the research paper doesn’t say they did the comparisons — in fact, it more or less denies that they did, because it talks about how many comparisons they did just in terms of two brain regions and two chemicals, not in terms of other subjects studied.

Q: But in any case, learning some other subject might still cause different changes in the brain

A: I think we can guarantee that it does, yes

June 19, 2021

The Olympics condom story

Every two years, there’s a news story about how many condoms are being handed out at the Olympics.  I last wrote about it four years ago, when Rio planned to give away 450,000 condoms in the Olympic Village, compared to a mere 150,000 in London, and 90,000 in Sydney (initially 70,000, but they ran out).  The number was 110,000 in Pyeongyang, and the Guardian reports that it’s 160,000 for Tokyo.

It’s pretty clear that advertising for safe sex is a big part of these numbers. Which is a bit tricky this time around — the Tokyo Olympics has a serious problem with the risk of communicable disease, but it’s not one that condoms will fix. This year, according to the Guardian, the condoms are just intended as souvenirs, not as a way around social distancing rules.

June 14, 2021

Controlling emissions

There are two basic ways a government can put downwards pressure on carbon emissions and let consumers find the best ways to adapt. One is to charge a fixed tax per kg of CO2 equivalent, the other is to fix a total cap on emissions and let consumers bid for the rights to use it.  There’s a sense in which these are equivalent: if the price in a cap-and-trade system ends up being $X per kg, the emissions will be the same as if the government charged $X per kg and didn’t have a cap. A sufficiently flexible and adaptive version of either one could match the other.  In reality they aren’t quite the same because governments want a simple and relatively predictable price or cap.

We’ve got a cap (more or less). One of the non-intuitive aspects of having a cap rather than a fixed price is that parallel efforts to reduce carbon emission don’t work the way you’d expect them to. If I replace my gas stove with an electric one, my kitchen will emit less carbon (modulo the impacts of making the new equipment).  If everyone did it, everyone’s kitchen would emit less carbon (again, ignoring the impacts of making the new equipment).  What would happen to NZ’s total carbon emissions? Nothing. We have a cap.  Less of the cap would go on carbon coupons for burning natural gas; more of it would be available for cars or trucks or coal-fired power stations.  The impact of our kitchen-renovation decisions would be cheaper emissions rights for other polluters, not lower emissions.

In principle, I could keep buying emissions rights for the natural gas I wasn’t using. That would turn my lower emissions into reductions for NZ as a whole. Or, the government could monitor the sales of induction cooktops and withdraw emissions rights to compensate (or, more realistically, track kitchen conversions through some sort of subsidy).  But if nothing happens to the total ETS carbon budget, nothing will happen to total emissions. A big enough change in demand could change emissions — if cars were suddenly banned, the government might not be able to sell all its ETS coupons — but a modest change won’t.

When the government says that new subsidies for low-emissions cars will reduce carbon emissions by some large number, there’s a gap in the explanation.  Having more low-emissions cars will lower carbon emissions by cars, but unless the government withdraws the corresponding emissions rights from the carbon budget, it won’t reduce carbon emissions in total. The reduction will go to lowering carbon costs for other polluters.

This, in itself, doesn’t mean the policy is bad; it just means the policy needs to be evaluated in some other way.  Maybe subsidising electric cars will lower the cost of future emissions reductions. Maybe it will improve the political feasibility of reducing the total emissions budget. Maybe there’s some other big benefit that I haven’t seen. But it is a problem that the policy is being sold on emissions reductions and that there doesn’t seem to be media or political reaction asking exactly where these reductions are coming from given the ETS cap.

New Alzheimer’s drug

The US Food and Drug Administration has approved a new treatment, aducanumab for Alzheimer’s Disease. The Chief Executive of Alzheimers NZ was quoted by  TVNZ

“What I’d say is, cautious optimism,” Alzheimer’s New Zealand chief executive Catherine Hall told 1 NEWS when asked about her reaction to the drug’s approval overseas.

She said people living with dementia often ride a rollercoaster of emotion when it comes to new drugs being announced.

“They get told there’s a brand new cure and then very quickly afterwards their hopes are dashed. It’s really important to recognise this is early days and there’s still a lot of data to be collected.”

It’s a very sensible attitude, in the abstract: if the drug turns out to be effective it could be valuable, but it’s too early to know if that will be the case. What’s surprising is that this is the situation we’re in after the drug has been approved, and when its manufacturer is planning to charge US$56000/year for it.

The drug (or, technically, the ‘biologic’ since it’s an antibody) has been through a lot of ups and downs in its clinical trial history.  There were two main trials that were supposed to show it was effective. They failed. A re-analysis of one of them suggested that it might actually work, at least for some patients. Normally, this would be the cue to do a confirmatory trial to see if it does actually help an identifiable group of people. And the FDA did mandate this trial — but they will let the manufacturer sell and promote the medication for nine years while the trial goes on.  Given that the the market for aducanumab is conservatively estimated at tens of millions of dollars per day, and there’s only a possible downside to getting trial results, the trial is unlikely to end a day sooner than it has to; it’s not unheard of for these post-approval trials to just never recruit enough participants and drag on longer than ‘allowed’.

The FDA takes external expert advice on drug approvals. In this case, there were 11 people on the panel. Exactly none of them thought there was good enough evidence for approval; one was uncertain, ten were against. Three of the panel members have since resigned. It’s not unprecedented for the FDA to disagree with the panel when the panel vote is split, but it’s pretty bloody unusual for them to disagree with a unanimous panel.  It’s notable that the FDA approval does not say they think there’s evidence drug improves memory or cognition or ability to live independently or anything like that — the FDA thinks it reduces the amount of amyloid plaque in patients’ brains and hopes this will translate to (currently unobserved) improvements in how they live.

TVNZ goes on to say

It’s only suitable for those with mild Alzheimer’s when the damage to brain function is still limited.

This is what the trials would say, under the optimistic interpretation that they say it’s suitable for anyone. The FDA, however, says it is an amyloid beta-directed antibody indicated for the treatment of Alzheimer’s disease. No restriction. Since most people who’d want the drug are over 65, it will predominantly be funded by Medicare, the US government health program. Medicare tends to pay for anything the FDA approves, but they might not have the option this time. Presumably in NZ we won’t get Pharmac subsidy for aducanumab at least until it’s shown to work; an incautiously  optimistic date would be 2030.

Something like the aducanumab approval is what many of us were afraid of for Covid vaccines — an product with weak evidence of modest effectiveness being given the green light because demand was high: “something must be done; this is something; therefore we must do it”.  We were lucky with Covid that the temptation didn’t arise for the FDA: the vaccines it has been asked to approve are more effective than anyone could have hoped.  Aducanumab won’t be that sort of disaster, but it will put a lot of pressure on a US health cover system for older people where the government is expected to pay but not to count the cost, let alone balance it against effectiveness.

June 4, 2021

How not to track vaccine attitudes

One way not to track vaccine attitudes, which I’ve already covered briefly, is to ask people if some piece of news, such as the rare blood clots with the AZ and J&J vaccine, has made them less likely to get vaccinated.  People who always supported vaccination will say “no”; people who always didn’t will say “yes”.  It’s like if you ask people whether Judith Collins’s speech in Christchurch yesterday made them less likely to vote National. National supporters will say “no”; Labour and Green supporters will  say “yes”, and that’s regardless of whether she actually was in Christchurch or gave a speech.

The Sydney Morning Herald has another way not to do it (via David Hood)

The story says

The exclusive survey, an initiative of The Sydney Morning Herald and The Age with research company Resolve Strategic, found vaccine doubts were stronger than in Ipsos polls carried out in February and September, before an official ruling in April about blood clots linked to AstraZeneca doses for people under 50.

It’s possible that’s true, but we’re not given any reason to believe it. We aren’t told what the figures were in February and September, and the new figures exclude the 15% of Australians who are the most likely to get at least one shot of the vaccine, because they already have.

Also, the numbers in the bar chart add up to 64%, with no explanation of where the missing 36% are. Perhaps the 15% already vaccinated are part of that, and the statement about excluding them is untrue. Perhaps 36% said they didn’t know and that wasn’t thought to be worth mentioning. Perhaps they were eaten by drop bears. We can’t tell.

Attitudes to vaccination actually matter, and it’s plausibly a topic where knowing what other people believe will affect your beliefs, so it’s worth doing better than this.

May 21, 2021

Briefly

  • 92% of people think they’re better than average at complying with Covid-19 rules (UK study).  It could be that people who comply with Covid rules are also more likely to respond to surveys, but it also wouldn’t be surprising if most people just think they’re better than average.
  • A (flashing) map of lighthouses
  • “Did 4 per cent of Americans really drink bleach last year?” — a good piece at the NZ$ Herald, from the Harvard Business Review. Spoiler: no, probably not: it’s the sort of claim that surveys are bad at supporting.
  • “Every recorded battle in history” on a map (click to embiggen).

    “recorded” means recorded by Wikipedia. The European bias is pretty clear, but note that it even misses battles in the Waikato and Taranaki wars where British soldiers were awarded the Victoria Cross, so the Wikipedia completeness bias is more complicated than just ‘European soldiers’.

But they’re all good dogs

Q: Did you see dogs are better than lateral flow tests for detecting Covid?

A: No

Q: The Guardian says: Dogs can better detect Covid in humans than lateral flow tests, finds study. With 97% accuracy!!

A: That’s the detection rate in people with Covid. The detection rate of Covid in people without Covid is 9%, which is a lot higher than you’d like.

Q: Where did they find the people?

A: It was people turning up to a testing centre in Paris, though since 109 out of 335 had Covid, it can’t really have been representative. The test positivity rate in France as a whole is only 4.5% at the moment and peaked at 15.9%

Q: Is 335 people enough?

A: Potentially, though the study initially planned to get 2000 people

Q: The story says lateral flow tests correctly identify on average 72% of people infected with the virus who have symptoms, and 58% who do not. That sounds really bad. Why does anyone use them?

A:  There’s a lot of variability between tests: some of them are better.  Also, they have much, much lower false positive rates than the dogs — around half a percent.  Since there’s a tradeoff between conservative (giving false negatives) and being sensitive (giving false positives), you can’t just compare the sensitivity of two tests that have a ten-fold difference in false positive rate.

Q: Still dogs would be quicker, and you could just use the real test in people the dogs picked out

A: That has potential, but dogs don’t scale all that well. You need to train and test each dog; they can’t just be mass-produced, boxed up, and mailed around the country.  And dogs aren’t that much quicker — this isn’t walk-past sniffing like the beagles looking for smuggled food at Auckland airport; you need to stick a pad in your armpit for a couple of minutes.

Q: How much of the spin is from the research paper and how much is coming from the newspaper.

A: The researchers are reported in the French source: “Ces résultats confirment scientifiquement la capacité des chiens à détecter une signature olfactive de la Covid-19“, souligne l’AP-HP, précisant que cette étude, pas encore publiée dans une revue médicale, est “la première de ce type réalisée au niveau international“.

Q: J’ai pas de clue que that means

A:  “These results scientifically confirm the ability of dogs to detect an olfactory signature of Covid-19 “, emphasise [the hospital], specifying that this study, not yet published in a medical journal, is ” the first of this type carried out in international level “.

Q: So it’s not just the Guardian

A: No.

May 7, 2021

Mind if we call you ‘Bruce’?

From news.com.au (via @LewSOS and @Economissive on Twitter) “The names of Australians most likely to win the lotto have been revealed, with the top three taking home more than a quarter of the prizes last year.”

What they actually have for ‘names’ is first initials. Apparently, more than a quarter of first-division prizes last year were won by people whose names started with “J”, “A”, or “D”.  Of course, people whose names start with these letters are not any more likely to win lotto if they buy tickets. Either more people in this category bought tickets than average (in which case it would be truer to say they are more likely to lose Lotto), or the distribution of initials is pretty much the same as for the country as a whole.

The story does go on to say that name and age can’t affect your chance of winning, but not to explain why, given that, it’s news.

Anyway, since Rob Hyndman and the stats group at Monash have put together a database of frequencies of Australian names, we can see how representative the winners are.  Here are the proportion of Oz babies born each year (up to current 18-year-olds) whose names begin with “J”, “A”, or “D”. As you can see, it’s “more than a quarter” almost every year where we have data.

Since you’re a StatsChat reader, you can probably think of reasons there might be a difference between Lotto name frequencies and baby name frequencies.  The baby names don’t include immigrants and do include emigrants.  There might be ethnic differences in propensity to play Lotto that happen to be correlated with first initial. There might be quite large chance differences because the lottery folks only looked at first-division winners, a very small (but random) sample of Lotto players. But it doesn’t look like we need to go there.

May 3, 2021

Briefly

  • Mediawatch took on bogus polls and interviewed me.
  • If you ask people whether the recent stories about blood clotting affect their views on Covid vaccines they will say ‘yes’. But if you asked people, before and after, what their views actually are, there’s a very slight change to be more pro-vaccine.
  • A study of 9806 blood donors found 8 had antibodies to the Covid virus.  Extrapolated crudely to all of NZ that’s about 4000 undiagnosed cases. I’m not giving a statistical uncertainty interval because the non-sampling uncertainty is going to be larger — blood donors tend to be younger and so more likely to have been asymptomatic/weakly symptomatic and thus not tested, but might also be less likely to have been infected.  Still, the number is in the sort of range you’d expect.
April 30, 2021

Physical punishment of children – reporting

There’s a new research paper out from the Christchurch Health and Development Study, which recruited a group of people when they were born, in mid-1977, and has been following them ever since.  Those of the participants who were parents have been asked about physical discipline of their children on four occasions: when they were 25, 30, 35 and 40.  Obviously, over time, the number who are parents has increased (from about 150 to over 600), and the children have (on average) gotten older — when the parents were 25, most of the kids would have been pre-school; the group now includes a few very young children but many who are teenagers.

The good feature of birth cohorts like the Christchurch study is that you get to see the same people throughout the course of their lives; the bad feature is that at any given time everyone is exactly the same age.  In statistician jargon, age is completely confounded with period: you are completely unable to distinguish effects of ageing from time trends. When you see that the proportion of parents reporting hitting their kids has gone down from 77% to 42% over the 15 years, you can’t tell, at all, whether this is an effect of these specific parents getting older and more experienced or an effect of parents in general being less likely to hit their kids.  It’s hard (though not impossible) even to tell if it’s an effect of the kids being older.

As you’d hope, the research paper, in the NZ Medical Journal (paywalled) is very clear on this

…explanations include: increasing maturity of the parenting sample over time (less reactive, more experienced, older parents); a cultural shift towards the unacceptability of violence towards children over the period of the study; and the law change in 2007, which prohibited physical punishment and violence towards children. Given the nature of its design, it is not possible for the current study to distinguish between these explanations.However, it does not seem unreasonable to conjecture that all three processes are likely to have played a role.

And indeed it doesn’t seem unreasonable, as long as you recognise that the not seeming unreasonable isn’t a conclusion from the data and relies entirely on external plausibility.  The researchers do conclude that there’s still a lot of physical punishment going on, and that efforts are needed to stop it; the former is well-supported by the data and the latter is a policy response, not a scientific conclusion. That’s all good.

So let’s look at the reporting (some of this may have changed before or after I read it, of course)

  • Radio NZ: Number of parents smacking children drops by half in 15 years. No caveat about the study design meaning this conclusion is basically unsupported. Gets the journal name wrong.
  • 1 News: More than 40% of parents still use physical discipline years after law change, latest data shows. The story is better than the headline, and the Children’s Commissioner is quoted as saying “It’s representative of one cohort born in 1977, one group in one year in one generation, but there has been a discernible drop over the years.” I’d be happier if it was clearer from the beginning that this doesn’t claim to be representative of NZ in general over time.
  • NZ Herald. Parents’ physical punishment of children decreasing, but still common – report. Slightly better headline; much clearer in the story. “…the rate of physical punishment against children was higher when parents were younger, and then decreased with age… because of the way the study was designed, it couldn’t pinpoint how much the rates reduced because of the law change.”
  • Otago Daily Times. Parents still smacking, study finds. Good. “The authors warned that their method of studying a cohort of people over time meant they could not gauge what the attitude of new parents in 2021 might be to physical punishment. However, the research did suggest rates of smacking or hitting children were high enough to be a public health concern.”
  • Stuff. Physical punishment of children still ‘fairly common’, despite anti-smacking law change – study.  There’s no caveat about the study design, and the story says “New research, published in the New Zealand Medical Journal on Friday, examined how the prevalence of child physical punishment changed in the 15-year period between 2002 and 2017 – before and after the legislation came into force.”, which isn’t true. And that’s not a link to the research paper.
  • Newshub. Who’s most likely to use physical discipline against their kids revealed. The headline’s a bit dodgy given the non-representative group of parents, but the caveats are good “Because the study followed a cohort of parents who aged 15 years over the course of the study, “it is unclear what rates of physical punishment of children would be in studies of contemporary young parents”.