Posts from April 2013 (67)

A new research paper with the cheeky title “Power failure: why small sample size undermines the reliability of neuroscience” has come out in a neuroscience journal. The basic idea isn’t novel, but it’s one of these statistical points that makes your life more difficult (if more productive) when you understand it.  Small research studies, as everyone knows, are less likely to detect differences between groups.  What is less widely appreciated is that even if a small study sees a difference between groups, it’s more likely not to be real.

The ‘power’ of a statistical test is the probability that you will detect a difference if there really is a difference of the size you are looking for.  If the power is 90%, say, then you are pretty sure to see a difference if there is one, and based on standard statistical techniques, pretty sure not to see a difference if there isn’t one. Either way, the results are informative.

Often you can’t afford to do a study with 90% power given the current funding system. If you do a study with low power, and the difference you are looking for really is there, you still have to be pretty lucky to see it — the data have to, by chance, be more favorable to your hypothesis than they should be.   But if you’re relying on the  data being more favorable to your hypothesis than they should be, you can see a difference even if there isn’t one there.

Combine this with publication bias: if you find what you are looking for, you get enthusiastic and send it off to high-impact research journals.  If you don’t see anything, you won’t be as enthusiastic, and the results might well not be published.  After all, who is going to want to look at a study that couldn’t have found anything, and didn’t.  The result is that we get lots of exciting neuroscience news, often with very pretty pictures, that isn’t true.

The same is true for nutrition: I have a student doing a Honours project looking at replicability (in a large survey database) of the sort of nutrition and health stories that make it to the local papers. So far, as you’d expect, the associations are a lot weaker when you look in a separate data set.

Clinical trials went through this problem a while ago, and while they often have lower power than one would ideally like, there’s at least no way you’re going to run a clinical trial in the modern world without explicitly working out the power.

Other people’s reactions

• Ed Yong
• Dorothy Bishop
• BPS research digest

I don’t usually bother with general nutrition stories that don’t contain any direct reference to research, but the Herald story about berries was irresistible. There are lots of biologically active compounds in berries, and many of them have been shown to have interesting properties in test-tubes or mice. As you know by now,  this sort of interesting biochemistry is important because it occasionally translates to genuine health benefits, so you should be asking what the human clinical research shows.

If you go to the Cochrane Library (which is free to everyone in New Zealand), and look for clinical research in humans involving blueberries or cranberries you don’t find much. The only topic with enough information to draw any sort of conclusion is on cranberry juice to prevent urinary tract infections. Which it basically doesn’t. The plain-language summary says

JustSpeak is

a non-partisan network of young people speaking to, and speaking up for a new generation of thinkers who want change in our criminal justice system.

I’m linking because they have a good visualisation of the recently-released police crime statistics, comparing the proportion of apprehensions leading to prosecution among Maori and Pakeha youth. The back-to-back bar charts take advantage of the brain’s ability to detect lack of symmetry.

I probably would have left out the homicide category, which has too few to compare, and it would be interesting to see if small gaps between the categories help.

The real problem is in interpretation.  It’s hard to say what you’d expect just from economic differences and differences in where people live, without any differences in how they are treated by police. A higher proportion of prosecutions could mean the police are using their discretion to prosecute more Maori youth, but a lower proportion of prosecutions could just as easily have been interpreted as harassment of innocent Maori youth.

There’s an article in Nature News about one of the most interesting findings from large-scale genomic studies.  People with mutations in a gene called PCSK9 have low levels of cholesterol, and since the protein produced by that gene is active outside cells, it is relatively easy to target.  Also, people with mutations in both of their copies of the PCSK9 gene seem to be healthy, so it looks as though it should be a safe target for treatment

Synthetic antibodies against PCSK9 reduced LDL (bad) cholesterol by 73% in initial trials in a small group of patients, which is huge. There’s a huge trial going on at the moment to see if this translates to a reduction in heart attacks, strokes, etc.  It could still easily fail — several other drugs giving big cholesterol improvements have turned out not to prevent heart disease — but it is very promising.

Team Ratings for Round 6

Here are the team ratings prior to Round 6, along with the ratings at the start of the season. I have created a brief description of the method I use for predicting rugby games. Go to my Department home page to see this.

Performance So Far

So far there have been 40 matches played, 26 of which were correctly predicted, a success rate of 65%.

Here are the predictions for last week’s games.

Predictions for Round 6

Here are the predictions for Round 6. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 9

This year the predictions have been slightly changed with the help of a student, Joshua Dale. The home ground advantage now is different when both teams are from the same country to when the teams are from different countries. The basic method is described on my Department home page.

Here are the team ratings prior to Round 9, along with the ratings at the start of the season.

Performance So Far

So far there have been 48 matches played, 34 of which were correctly predicted, a success rate of 70.8%.

Here are the predictions for last week’s games.

Predictions for Round 9

Here are the predictions for Round 9. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Britain is currently having a measles epidemic.

Measles has been a preventable illness for decades, but the vaccination rate dropped after the widely-publicized and bogus claims of a link to autism. The measles epidemic is especially severe in southwest Wales, in the circulation area of a paper that was especially anti-vaccination.

There are many examples where the media just reported uncertainty within the scientific community. This was not one of them.

From the Herald (actually from the Independent)

The rise in the number of overweight children in Western countries may be as much to do with their genes as their diet and exercise levels,

In fact, just about the only completely uncontroversial fact about the increase in obesity is that it is entirely due to environmental changes of some sort. There’s disagreement on precisely which environmental changes, and on the likely public health impact, but not on the general principle. The reason is very simple: the genes of this generation’s children came from their parents, with almost no changes. There simply hasn’t been enough time for genetic differences to contribute.

The research paper is looking for mechanisms: they aren’t studying a representative population sample, but a group of 1500 children with severe obesity.  Most of the genetic variants they found are very rare, and did not show up in the previous huge genetic study of weight and height that combined genetic data on quarter of a million people. That means these genetic variants can’t contribute much even to explaining or predicting which children are overweight. The last sentence of the research paper is illustrative of the point of the research

 As we observed a significant enrichment for CNVs that deleted GPCRs, which are key targets for drug development, these findings may have potential therapeutic implications.

In English: there were lots of mutations that affected one particular type of cell signalling molecule, one that we’re pretty good at turning on and off with drugs. This could be useful.

Keith Ng’s annual NZ Budget visualization seems to be up. Go play.

You might also like last years’ one.  And possibly even the 2011 radioactive space donut.

Congratulations to Nick Iversen for his humorous nomination of “Sex basically doubles your life span” winning this week’s Stat of the Week competition:

“The anti-ageing benefits [of frequent sexual activity] are amazing. It basically doubles your life span once you get to your 60s and 70s.”

The article obviously get correlation confused with causation but that’s not why I’m posting it here.

The reason for the post is that the statement about life span is so obviously blatantly wrong that I don’t see how any intelligent person could make it.

The average life span of people in their 60s and 70s is (funnily enough) at least 60 or 70. Doubling this means that sex extends your lifespan to 120 or 140.

OK so she really means “remaining lifespan.” For those in their 60s remaining lifespan is about 20 to 25 years. So frequent sex doubles that to 40 to 50 years.

Just doesn’t sound at all reasonable to me. If sex makes that much difference to life expectancy it far outweighs any contributions due to diet, exercise etc and would have been noticed centuries ago.