Stats Chat

Q: Did you see that smart people like chocolate?

A: Isn’t it nice when studies confirm your prejudices? But that’s not what they’re claiming

Q: The headline says “Chocolate intake associated with better cognitive function”, doesn’t it?

A: Yes, but the story starts “Eating chocolate improves brain function, regardless of what else you’re scoffing, a study has found.” 

Q: That’s even better! How much did people’s brain function improve?

A: They don’t know

Q: It was mice?

A: No, it was people, but they only measured cognitive function once, so they couldn’t see any improvements.

Q: Ok, how much better was cognitive function in the people given chocolate?

A: No-one was given chocolate; these were free-range participants in a longitudinal study

Q: So at least they measured chocolate intake over time and then looked at cognitive function later on?

A: Sadly, no.

Q: But the story says it was a rigorous study.

A: Yes. Yes it does.

Q: And I suppose the alleged effects of chocolate are really small, right?

A: If the cognitive function score was scaled like IQ, it would be about 3 IQ points.

Q: That’s not tiny

A: Indeed. Especially when you consider that they didn’t measure how much chocolate people ate or what type of chocolate or how much it had of the flavonols they think are responsible. They just divided people into eating chocolate less than once/week, once/week, and more than once/week.

Q: Aren’t flavonols in other things, too?

A: Yes: tea, red wine, a bunch of other favourites for this sort of story

Q: So if it really was flavonols, the true effect must be huge

A: Yes. So it probably isn’t.

Q: What does the research paper say about the cause-effect relationship?

A: “This precludes any conclusions regarding a causal relationship between chocolate intake and cognition from being drawn.”

Q: Ok. That makes sense.

A: On the other hand, they say “It is evident that nutrients in foods exert differential effects on the brain. As has been repeatedly demonstrated, isolating these nutrients and foods enables the formation of dietary interventions to optimise neuropsychological health.”

Q: “Repeatedly demonstrated” that you can give people diets or supplements to make their brains better? Isn’t that the sort of claim where “Name three” is the appropriate response?

A: Pretty much.

Team Ratings for Round 1

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Predictions for Round 1

Here are the predictions for Round 1. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Two graphs to help people on Twitter who are arguing about home ownership trends in Auckland vs rest of NZ or in generational differences.

Both are percentages of home ownership based on the census question “Do you own or partly own your home?”, with data from the last three censuses.

First, comparisons between Auckland and the Rest of NZ by age, over time. Blue is Auckland, pink is RONZ

Second, trends over 12 years, by age, for three census years. Blue is 2001, pink is 2006, green is 2013.

Data from the nzdotstat table “Tenure holder by age group and sex, for the census usually resident population count aged 15 years and over, 2001, 2006 and 2013 Censuses (RC, TA, AU)”

Update: And one more. Here the lines connect roughly the same group of people (birth cohort) over time (only approximately because the planned 2011 census didn’t happen until 2013).

• “Places“:  Interactive maps of place name distribution in the US. For example “Lake” — with high density in the “Land of Lakes” but also in some less-expected places
• “Spreadsheets, the original analytics dashboard’, from Simply Statistics, about the origin of spreadsheets and what they were good for.

• Cats can see the ‘rotating snake’ optical illusion: video evidence from Rasmus Bååth

• As we’ve mentioned before, most people think teenagers have more risky behaviours now than in the Good Old Days. Most people are wrong. This time, from Vox.

• From Kieran Healy, the network of shared institutional affiliations for the 1000+ authors of the LIGO gravitational waves paper (click to embiggen). That is, many scientists have some sort of connection with more than one university; the graph shows how these link up the LIGO researchers.
  
• Come on, major political parties. Barchart axes start at zero unless you want to look like Fox News. There are reasons for this. If you don’t want to start the axis at zero use some other sort of chart.
  

David Seymour, on the Herald website

 Auckland is already denser than New York, and most American and Australian cities.  The 1.6 million people in Manhattan may live cheek-by-jowl, but not the other 20 million inhabiting the wider urban area.

An intelligent politician wouldn’t say something as apparently bizarre as this first sentence if it wasn’t true, so of course it is. The question is going to be true in what sense?

Based on the population figure, Mr Seymour is talking about the New York Metropolitan Statistical Area, aka, New York Urban Area,  which has a population of 20.1 million and a population density of 724/km²[*]. The Auckland Urban Area has a population of 1.45 million and a density of 2,600/km², and, yes, 2600 is larger than 724. However, as the scenic photos in the Wikipedia page for the New York Metropolitan Area suggest, that might not be a fair comparison.

In fact, it’s true almost by definition that the New York metropolitan area has a lower density than urban Auckland

Urban areas in the United States are defined by the U.S. Census Bureau as contiguous census block groups with a population density of at least 1,000/sq mi (390/km2) with any census block groups around this core having a density of at least 500/sq mi (190/km²).  [Wikipedia, or see full legal definition]

That is, the metropolitan area is defined as the area around New York City all the way out until the local population density is below 190/km². It’s a sensible statistical unit — the US Census Bureau wasn’t trying to make a political point about urban infill when they defined it — but it’s not the same sort of unit as Stats New Zealand’s definition of urban or metro Auckland.

So, what other comparisons could we do? We could compare the New York Metropolitan Area to the Auckland Supercity, whose population density of 320/km² is less than half as high. That might be unfair in the other direction — the Supercity is designed with the future expansion of Auckland in mind, while the US definitions are only intended for a ten-year period between censuses.

We can’t quite do the perfect comparison of redrawing Auckland Urban Area by the US rules, because NZ Area Units are bigger than US Census Block Groups, and NZ meshblocks are smaller, but someone with more time than me could try.

We could compare the Auckland urban area to genuinely urban parts of  the New York metro:  Mr Seymour mentioned Manhattan (density 27,673/km², three times that of the Auckland CBD, nine times that of the Epsom electorate) but the other four boroughs of New York City all have higher density than urban Auckland. Two of them (the Bronx, and Brooklyn) have higher density than the Auckland CBD, Queens (8237/km²) is closer in density to the Auckland CBD than to the rest of Auckland, and even Staten Island is denser than urban Auckland as a whole. In the metropolitan area but across the river from New York City proper we have Hudson County (density 5,241/km²) and Newark (density about 4500/km²). The whole of Long Island, part of the New York metropolitan area, but also known for places like Fire Island and the Hamptons, has population density 2,151/km², not far below urban Auckland.

And finally, an alternative way to do this whole comparison, which is much less sensitive to where the lines are drawn, is to look at population-weighted densities. That is, for the average person in a city, how dense is the population near them? For the whole New York metropolitan area the population-weighted density is 12000/km² (or 120/hectare). For Auckland it is 43/hectare. In other words, while people near the edges of the New York metro area have a lot of space, most New Yorkers don’t. The average person in the broad New York metropolitan area sees three times the local population density of the average Aucklander.

Update: * Mr Seymour tells me he was referring the the definition of metropolitan areas from Demographia, which trims some of the low-density parts of the Census Bureau definition of New York to give a population density of 1800, and agrees well with the StatsNZ definition of urban Auckland.  So, while the issue about the difficult in defining things comparably is still an issue, it is less his fault than I had assumed.

The Herald says “Police Minister Judith Collins has released figures to show crushing boy racers’ cars has worked“.  The data are more consistent with the political interpretation than is usual for claims about crashes, but not as strong as the Minister would like us to think.

Here’s a graph of the data (supplied to the Herald by the Minister’s office), showing crashes, injuries, and deaths where the police reported ‘racing’ as a cause:

It’s fairly clear that something changed. Based purely on the graph you’d say the downwards trend started after 2007; 2009 isn’t unreasonable, but it fits the data a bit less well. This an example of a graph being much more useful than a table.

The next thing to check is other crashes — the road toll has been down in recent years, so this could just have been a general improvement. It’s not; the evidence for a change is a little weaker when considering racing deaths or injuries as a proportion of all fatal or injury crashes, but it’s still there.

In principle there could have been changes in reporting, but it’s hard to see how a government crackdown would make police less likely to report ‘racing’ involvement in a crash.

Finally, there’s publication bias.  The reporter, Nicholas Jones, didn’t notice that Ms Collins was back and decide to pull figures on car crushing; the Minister decided to release the figures. She wouldn’t have done that if they didn’t look favourable. It’s hard to tell how much to discount the evidence for that, but a discount is needed.

Overall, the data are definitely consistent with a deterrent effect of car crushing, but the evidence isn’t all that strong — the best fit to the data suggests things changed earlier than 2009, and looking at the numbers was the Minister’s idea, not the reporter’s.

Updates:

• in addition to the useful comments, I’ve been pointed to Dog & Lemon where Clive Matthew-Wilson says there is reason to believe the ‘boy racer’ thing was already going away on its own.  If so, that would fit the trend starting earlier than the legislation.
• If you check the crash numbers against the Road Crash Statistics system they don’t match.  I think that’s because Table 26 of the Road Crash Statistics only includes crashes causing injury or death — that’s explicit in the 2012 spreadsheet, and I think it’s still true.

From One News, tweeted by various people:

The y-axis label is wrong: this has nothing to do with change, it’s percent support.

The x-axis label is maximally unhelpful: we can guess that the most recent poll is in February, but what are the earlier data? You might think the vertical divisions are months, but the story says the previous poll was in October.

Also, given that the measurement error is large compared to the expected changes, using a line graph without points indicating the observations is misleading.

Overall, the graph doesn’t add to the numbers on the right, which is a bit of a waste.

One of the early motivating applications for linear mixed models was agricultural field studies looking at animal breeding or plant breeding. These are statistical models that combine differences between groups of observations with correlations between similar observations in order to get better comparisons.

John Oliver’s “Last Week Tonight” argues that these models shouldn’t be used to evaluate teachers , because they have been useful in animal breeding (with suitable video footage of a bull mounting a cow).  It’s really annoying when someone bases a reasonable conclusion on totally bogus arguments.

As the American Statistical Association has said on value-added models for teaching (PDF), the basic idea makes some sense, but there are a lot of details you have to get right for the results to be useful. That doesn’t mean rejecting the whole idea of considering the different ways in which classes can be different, or giving up on averages over relevant groups. On the other hand, the mere fact that someone calls something a “value-added model” doesn’t mean it tells you some deep truth.

It would be a real sign of progress if we could discuss whether a model adequately captures educational difficulties due to deprivation and family advantage without automatically rejecting it because it also applies to cows, or without automatically accepting it because it has the words “value-added.”

But it probably wouldn’t be as funny.

2016, NZ Herald, “A new report claims the world is heading for a chocolate deficit” (increased demand, no increase in supply)

There’s not much detail in the story, and I’m not going to provide any more because the report costs £1,700.00 (+VAT if applicable) — so remember, anything you read about it is just marketing.  However, there are other useful forms of context.

2013: Daily Mirror, “Chocolate could run out by 2020”

2012: NZ Herald, “Shortage will be costly for chocaholics”

2010: Discovery Channel, “Chocolate Supply Threatened by Cocoa Crisis”

2010: Independent, “Chocolate will be worth its weight in gold in 2020”

2008, CNN,”I think that in 20 years chocolate will be like caviar,”

2007:  MSN Money, “World chocolate shortage ahead”

2006: Financial Post, “Possible chocolate shortage ahead”

2002, Discover, “Endangered chocolate”

1998, New York Times, “Chocoholics take note: beloved bean in peril” (predicting a shortfall in “as little as 5-10 years”)

It could be that, like bananas, chocolate really always is in peril, or it could be that falling global inequality will make it much more expensive, or it could be that it’s just a good story.

From Stuff

“According to a new study titled, Music Makes it Home, couples who listen to music together saw a huge spike in their sex lives.”

This is a genuine experimental study, but it’s for marketing. Neither the design nor the reporting are done they way they would be if the aim was to find things out.

In addition to a survey of 30,000 people, which just tells you about opinions, Stuff says Sonos did an experiment with 30 families:

Each family was given a Sonos sound system and Apple Music subscription and monitored for two weeks. In the first week, families were supposed to go about their lives as usual. But in the second week, they were to listen to the music.

Sonos says

The first week,participants were instructed not to listen to music out loud. The second week,participants were encouraged to listen to music out loud as much as they wanted.

That’s a big difference.

The reporting, both from Sonos and from Stuff, mixes results from the 30,000-person survey in with the experiment results.  For example, the headline statistic in the Stuff story, 67% more sex, is from the survey, even though the phrasing “saw a huge spike in their sex lives” makes it sound like a change seen in the experiment. The experimental study found 37% more ‘active time in the bedroom’.

Overall, the differences seen in the experimental study still look pretty impressive, but there are two further points to consider.  First, the participants knew exactly what was going on and why, and had been given lots of expensive electronics.  It’s not unreasonable to think this might bias the results.

Second, we don’t have complete results, just the summaries that Sonos has provided — it wouldn’t be surprising if they had highlighted the best bits. In fact, the neuroscientist involved with the study admits in the story that negative results probably wouldn’t have been published.