Posts written by Thomas Lumley (2563)

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

March 31, 2025

Bogus polls are bogus even for the good guys

Nature magazine (yes, that one) has a headline 75% of US scientists who answered Nature poll consider leaving.

This is a more honest phrasing that one usually gets in bogus poll headlines, but it’s still bogus.  The ‘poll’ was

Responses were solicited earlier this month on the journal’s website, on social media and in the Nature Briefing e-mail newsletter. Roughly 1,650 people completed the survey.

The other problem with the poll is that ’emigrational ideation’, as one might call it, is a very low bar. Lots of people consider doing things. Many fewer actually do them. People having been claiming they are considering leaving the US for a long time, and we don’t know from the ‘poll’ how the current numbers compare to the usual state of affairs. Remember this cartoon, from 2006:

In fact, I think it’s quite likely that there are more scientists than before actively looking into leaving the US.  Anecdotally, we are getting more enquiries here and more US applicants to advertised positions.  You might think this sort of anecdata is a pretty weak basis for conclusions . You’d be right, but it’s still better than a bogus poll.

March 26, 2025

Ebikes for brains

Q: Did you see ebikes are good for your brain? If you’re old.

A: How old?

Q: Over 50

A: That’s not… Ahem. That’s a very inclusive definition of  ‘old’, isn’t it?

Q: Does it work?

A: Who knows? It might.

Q: Does this research show it works?

A: What would we want to check?

Q: Mice or people?

A: Good. People, not mice.

Q: Random allocation of people or just correlations?

A: Yes, that’s right.  This one is … somewhere in between.

Q: What’s in between randomly allocating people and just seeing what they choose? You flip a coin, but with your eyes closed?

A:  Well…

Participants were pseudo-randomly assigned to one of three groups: pedal cycling, e-bike or non-cycling control groups. Priority was given to filling the cycling spots then controls were recruited to match sample characteristics. The control group were recruited after the experimental groups had started to be run so that we could match age and gender in the control group with those participants in the experimental groups. The control group were aware that they would not be cycling during the trial and those in the experimental group were all re-engaging with cycling.

Q: So, comparing people who signed up to do cycling with people who signed up to avoid cycling and other exercise.

A: Yes, but in the context of cycling research, not just random people asked to not do any extra exercise.

Q: And did the cycling group have better brains?

A: They measured a whole lot of things. Eight different multi-question assessments. Some of those improved and some didn’t

Q: Is that what they expected?

A: Not really, no. They expected improvements in a lot of the things that didn’t go up

In line with our predictions, we found trends for improvement in executive function in the Stroop task and letter updating task in both cycling groups compared to baseline and the non-cycling controls. We also found improvement in speed of processing for go trials in the Stop-It signal task only for e-bike participants during the intervention. Measures of memory and spatial functioning did not show an effect of cycling. Furthermore, we found increases in self-reported mental health on the SF-36 health survey for only the e-bike cycling group. Despite strong evidence from previous studies for an increase in well-being after exercise and the impacts of the outdoor environment on this aspect of mental health, we did not find increases on the PWB, SL or PANAS questionnaires.

Q: Could it just be the study was too small to give reliable conclusions?

A: Well, if that’s the explanation, then the study is too small to support press releases and media attention.

Q: So e-bikes are bad?

A: E-bikes are a good form of low-carbon transport, especially in hilly cities with nice climates, and let you get some, but not too much, exercise.   I don’t think they need special effects on the brain to be popular.

Bogus Tesla Polls Are Bogus

Via the US website Electrek, which does news about electric vehicles and adjacent subjects, a story about bogus polls.

A bogus online clicky poll in Germany got 100,000 clicks and found that 94% of the time that a button was clicked, it corresponded to “absolutely not willing to buy a Tesla”.  Electrek calls this “100,000 Germans”, based on very little evidence.

The poll kept running.  By the time it had 470,000 clicks, only 29% of the clicks corresponded to “absolutely not willing to buy a Tesla”.  Quite a lot of these clicks — 253,000 — came from just two IP addresses in the US. A bit of maths shows that the clicks from the two US addresses made up about three-quarters of the pro-Tesla clicks.

The magazine, T-Online, was forced to the shocking conclusion that its meaningless customer-engagement exercise had been manipulated by someone else’s meaningless customer-engagement exercise.

March 25, 2025

If you see a fork in the road

From Pew Research, a bizarre error

A few Reddit users shared screenshots from a variety of surveys, where questions that should have offered answer options of “yes” and “no” instead offered the choices “forks” and “no.”

In summary: web browsers may offer you the option to translate web pages automatically from languages you don’t read to languages you do read.  That is, they have the option to silently rewrite web pages before showing them to you. They aren’t 100% reliable either in guessing which web pages are actually in a foreign language and need translating or in translating those pages to, as it might be, English.

March 23, 2025

Briefly

Screen time

In the Herald this week

New Zealanders are spending more time online than ever, with 50% of respondents spending four or more hours of their leisure time on the internet each day, according to a study commissioned by InternetNZ.

But that doesn’t necessarily mean doom-scrolling TikTok.

The result is partly because of the changing way that more traditional media is delivered.

Watching a couple of hours of TVNZ+ on the smart TV in your lounge or listening to Spotify during your commute counts towards your total.

That’s a nice start — giving the headline result and then contextualising it.  I was hoping then to see whether streaming more traditional media did in fact account for the increase. No more mention.

If we look at the report itself, there’s a good description of the survey: a sample of 1001 people, sampled and weighted to be representative of the subset of the NZ population who are online.  What does the report say about the increase?

Half of New Zealanders* (50%) spend four or more hours a day on the internet for personal
use.
This is a slight, but not statistically significant, rise from 2023 findings.

In 2023 the number was 46%, and a four percentage point change is relatively unimpressive compared to the sampling uncertainty.  The report doesn’t estimate how long we spend doing each activity on the internet, but the proportion of people who use the internet for streaming is about the same this year as last year or maybe slightly higher (42% vs 39%)

The report is generally interesting and useful, with the usual caveat that it’s self-reported — people may not be entirely accurate when you ask them how much time they spend online or how much they know about AI or whatever.

March 18, 2025

Good news graph

The Washington Post writes about cervical cancer: the vaccine works.  This isn’t new news: we know the vaccine stops infection with cancer-causing strains of the human papillomavirus (that’s why it was approved). We know the vaccine stops cervical cancer: the first reliable data came out a few years ago.  Now the first cohort of vaccinated girls is old enough that we’re seeing the reduction in cervical cancer deaths.

Using data from the National Center for Health Statistics, researchers looked at cervical cancer deaths in three-year blocks of time. Between 1992 and 2021, there were 398 cervical cancer deaths reported among women younger than 25. During the period of 1992-1994 to 2013-2015, mortality from cervical cancer gradually declined 3.7 percent each year. The period of 2013-2015 to 2019-2021 saw an even greater drop to 15.2 percent annually, according to the study.

The number of deaths decreased from 55 in 1992-1994 to 35 in 2013-2015 to 13 in 2019-2021.

“Assuming that the trend from 1992-1994 to 2013-2015 would have continued, an estimated 26 additional cervical cancer deaths would have been expected to occur between 2016 to 2021, based on projected mortality rates,” the authors wrote.

That description seems like a hard way to present this graph from the article in JAMA (perhaps copyright is the problem?)

The squares are what happened in reality. The orange line shows the downward trend we were seeing before the vaccine, due to better testing and treatment.  As you can see, the last two squares, post-vaccine, are dramatically below the orange line. Below is good.

March 14, 2025

Denominators

Earlier this week, Wikipedia’s front-page “Did you know…” section this week referenced the year that Vatican City had the world’s highest murder rate. In 1998, a double homicide in the Swiss Guards put the murder rate over 200 per 100,000 inhabitants, more than twice the rate in any of the Caribbean countries that usually head the list and roughly infinity times the usual rate in recent years when it has been a separate country. That’s a real rate, with the right denominator, it’s just that the year to year variability makes rates a relatively unhelpful summary.

Vatican City has a denominator problem for minor crimes; in 1992 there were 392 ‘civil offences’ and 608 ‘penal offences’, working out to a bit less than 100% and a bit more than 100% of the number in resident population.  The problem is that the tiny state has many millions of  visitors per year and they, rather than the resident population, are both the perpetrators and victims of the minor crimes.  The resident population isn’t the right denominator.

A less dramatic version of this problem showed up when looking at rates of violent crime in parts of central Wellington some years back.

March 12, 2025

More Panadol scare headlines

The New York Post is saying

Attention-deficit/hyperactivity disorder is a common behavioral condition that affects approximately 7 million US children, including about a million kids diagnosed since 2016.

The reason for the recent rise in diagnoses is under debate — a new study in the journal Nature Mental Health suggests the culprit could be in your medicine cabinet.

They’re talking about paracetamol (or as Americans call it, acetaminophen).  There’s a new study that looks at a fairly small group of US mothers and kids and finds weak evidence of a correlation between ADHD diagnosis in the kid and use of paracetamol by the mother.   You might remember this topic from previous StatsChat episodes.  There was  originally a Danish study that was surprised to stumble across a correlation. A New Zealand study checking up on the correlation also found it.

A Spanish study resulted in the NZ Herald (from the Daily Mail) scare headline One paracetamol in pregnancy could raise risk of autism. This was especially egregious since not only did the study say nothing about ‘one paracetamol’ and wasn’t really about autism, it actually found lower rates of the symptoms in kids whose mothers took paracetamol. The study argued that while the rates were lower in those kids, they should have been even lower based on other risk factors.   At this point it was all plausible and maybe a bit concerning but not reason for major change in medical practice — after all, mothers aren’t taking Panadol for fun.

In 2023, a combination of most of the published studies estimated a slight increase in risk, about 25%.  On the other hand, A very big Swedish study, using data from everyone born in Sweden from 1995 to 2019, then found no suggestion of a correlation.

So what’s the point of this new study? Well, one of the issues in interpreting these correlation studies is that many of them didn’t know for sure who actually took paracetamol.  For normal people this would be a big issue — lots of medications include paracetamol, and you might easily miss some, or just forget. Pregnant women, though, tend not to just casually take cough syrup or whatever.  Even so, if you happened to have data lying around from blood samples which told you who had taken paracetamol, you might be interested in seeing what the correlation was like.   So they did. And the correlation was broadly similar to what the small, early studies had seen: kids whose mothers blood samples showed paracetamol were about twice as likely to have ADHD.  That’s nice to know, even if it’s not a big change in the evidence.

We still have the conflict between ‘no sign of a correlation’ in the big Swedish study and ‘about 25% higher’ in the combined smaller studies.  It’s possible that paracetamol has an effect and that the Swedish study missed it because it didn’t measure paracetamol use as accurately. It’s also possible that the reasons for taking paracetamol (eg illness, fever) are what causes ADHD. Or it could all be some sort of bias and the Swedish study could be correct. It’s hard to tell.  Ask your doctor, etc.

The new study, of course, does not suggest paracetamol is responsible for recent trends in ADHD diagnosis: that claim is down to the New York Post, and is pretty clearly wrong. Here’s the trend:

Paracetamol became popular as a relatively safe, over-the-counter treatment a long time ago now. It might have been response for an ADHD trend in, say,  the 1980s, but not a trend in the 21st century.

Briefly

  • The New York Times has an interactive with graphs showing how everything changed when Covid started.  It’s also an explanation of why it’s hard to estimate the effects of specific actions on Covid: everything changed at once
  • Visualising how different languages represent animal noises: the spellings can look very different, but the underlying phonetics are more similar
  • “The answer to the how-many-significant-digits problem is the same as the answer to the what-to-graph problem: The click-through solution“. Or in other words, you can have tables without stupid numbers of digits and let people who want detail click to see it
  • The website of the US Centers for Disease Control as it existed on Jan 6 has been copied to RestoredCDC.org, hosted in Europe.  This won’t help with ongoing data collection, but it does make the past data from the CDC more reliably available.