Posts from October 2012 (66)

A standard criticism of science, especially in areas you don’t agree with, is to quote the ‘obvious’ findings that scientists have wasted time and money checking.

Quite often, though, the obvious doesn’t happen.  Stuff has a story about a randomized trial of intensive weight-loss propaganda in people with Type II diabetes: the trial is being stopped early, after ten years, because it isn’t working.  People in the intervention group are losing weight, and improving their control of blood sugar, but they are still getting heart attacks and strokes at the same rate as the control group.

An earlier trial had given a lot of hope by showing that quite small increases in exercise and reductions in weight could stop pre-diabetic levels of insulin resistance progressing to full-blown diabetes.

The current trial is led by Professor Rena Wing, of Brown University, who is hardly a heretic on the general weight-loss message.  Her research focuses on ways to maintain weight loss, and when the trial started she said

“Getting people to lose weight is talked about as the best treatment for both preventing diabetes in people who are overweight and for treating individuals who already have the disease,” Wing said. “If the long-term data from the study support these approaches, it will make it easier to go to patients, insurance companies and public-policy makers and say, ‘Yes, weight loss works over the long run.’”

We don’t have the full details of results, which are still be analysed. However, the trial succeeded in getting people to about the same amount of weight as in the earlier diabetes prevention trial, and to keep the weight off.  There were also improvement in glucose control, in sleep apnea, and in general quality of life, which might be sufficient motivation for weight loss, but not in heart disease and stroke.   This was a large, well-funded, well-designed study, and I would have bet on the results being the other way.

There won’t be any shortage of people who don’t want to believe the results, and it will be interesting to see if any of them can come up with some useful explanation for why the other differences between treatment and control groups don’t translate to reductions in heart disease or stroke.  [There will also be plenty who are happy to just say the results are wrong without needing anything as difficult as an explanation]

Unless there is something the researchers have completely overlooked, this trial adds to the long list of things that improve blood test results but don’t provide the expected health benefits in Type II diabetes.

An analysis reported in JAMA, and more accessibly by the LA Times shows that really dramatic beneficial effects of new treatments, even in good-quality clinical trials, are usually over-estimates.

They looked at 85,000 treatments for which there were multiple trials and binary (yes/no) outcomes measured.  In the 8000 treatments where the first trial showed a very large benefit, 90% of the time the subsequent trials showed a smaller benefit.

Also, when  large benefits were seen in well-replicated studies these were not typically for life or death issues: the researchers say

These include clinical benefits such as control of nocturnal enuresis with alarms in children, or symptomatic improvement with 5-aminosalicylic acid in ulcerative colitis; mostly mild or modest harms such as burning with capsaicin or local tenderness with the influenza vaccine; laboratory-determined response such as induction of hepatitis B surface antigen with hepatitis B vaccination; and control of acute pain with analgesics such as etoricoxib or diclofenac. As shown, all of these effects corresponded to very large absolute risk differences.

These findings aren’t actually surprising: if a treatment does very well in its first clinical trial it was probably lucky as well as effective.  Very large benefits in survival are very rare, so if you think you’re seeing one, you’re probably wrong .  It’s still useful for someone to put together all the data confirming what we expect, and it’s useful to remember when you see a headline about miracle cures.

The Herald has a headline “Young Kiwis among most savvy web users”, which would be justified if the phrase ‘most savvy’ was omitted.

The story comes from AVG, who sell internet security software, and so want you to be nervous online.  They have been conducting a set of surveys starting with kids and are now on to 18-25 year olds.  All we’re told about the survey methodology (by AVG, not by the Herald) is that the surveys were conducted by Research Now, so it’s not completely bogus but might be more or less representative or, um, ‘targeted’.

However, we do get told that there were only 4400 respondents in 11 countries.  Even with a completely random sample of 400 in each country, that gives a margin of error for comparing two countries of about 7 percentage points.   You can see a set of infographics from the survey here, and it’s pretty clear that a lot of the differences are compatible with random sampling error.

There certainly isn’t a general tendency for Kiwis to be more ‘web savvy’: while NZ has one of the lowest rates of admitting to accessing banned sites while at work, we have one of the highest rates of allowing colleagues to see everything that friends can see on Facebook.  The proportion for other questions, such as ‘friend’ing your boss, was intermediate.

A new paper in the (open-access) journal PLoS Medicine looks at ‘spin’ in reporting of scientific findings in the mass media, in press releases, and in the scientific papers underlying the press releases.

They found that the scope or importance of scientific findings were exaggerated in about half the media stories, almost always based on what was in the press release.  That’s not at all surprising.  The interesting part is that ‘spin’ in the press release often followed ‘spin’ in the scientific paper itself.  That is, it’s not just that the scientists cooperate with the university press office, they even exaggerate when writing for other scientists.

This doesn’t exonerate the journalists in cases of science puffery.  In sports, we expect the media to detect and ridicule any attempt to treat an ordinary All Blacks recruit as the second coming of Colin Meads.  They should also be able to tell the difference between a modestly interesting test-tube experiment and a ground-breaking clinical trial.

But, unlike last time,  it’s supposed to be.

The graph, from Nate Silver’s fivethirtyeight.com, shows the probability of each State (and its Electoral College votes) going to Obama or Romney.  The colour scale makes states near 50% stand out, which is a feature, not a bug. States near 50% are the important ones.

Colorado is dramatically visible because it’s smallest estimated margin that the Republicans have.  It’s also dramatic because it’s surrounded by non-marginal states, which is appropriate.

The only way in which Colorado is inappropriately highly visible is that it’s sparsely populated and so occupies more of the map than its importance to the election result would justify.  That’s a really hard problem to solve with data-based maps: sparsely populated areas do get too much attention, and none of the ways of mitigating this really work.

Researchers frequently want to find out how people behave in a typical day or week.  It turns out, though, that it’s often better to ask them what they did this week, even though this week may be far from typical.

It seems that a ‘typical’ day or week is actually a sort of Platonic ideal rather than a mean or median: in a ‘typical’ week, perhaps you exercise every day, but this particular week you missed a couple of days.  In a ‘typical’ week you eat vegetables every evening, but this particular week there was a social event at work and you didn’t.  In a typical week, you work five days, but this week there was a holiday on Monday.

The graph below shows the distribution of working hours reported for ‘usual’ weeks in two US government surveys and for “last week” in a third survey

People who work a lot tend to report more hours for a ‘usual’ week than last week. People who work less than half time tend to report fewer hours for a ‘usual’ week than last week.

The Washington Post article that provided the graph says that people who claim to work long hours are “lying”, but it’s more complicated than that.  Presumably these are people who ‘typically’ work long hours but reasonably often have to leave work ‘early’ to handle some part of the rest of their lives.  Conversely, the people at the low end of the distribution may have a regular part-time job that provides their ‘usual’ hours of work, but fairly often have over-time or additional jobs so that the average week has more work than a ‘usual’ week.   They aren’t lying, they just aren’t answering the question you thought you wanted to ask.

Earthquake forecasting (for everyone except Ken Ring) is difficult, and the accuracy of forecasts (for everyone, including Ken Ring) is low.  Even for relatively predictable faults such as NZ’s Alpine Fault, the margin of error is in decades.

The Italian government has just taken steps to make earthquake forecasting even less accurate.  Six scientists and a government official who failed to forecast that a set of small tremors near Aquila were forerunners of a big, deadly, quake have been sentenced to six years in prison for manslaughter. (Herald, Stuff).  They are still planning an appeal (and the Stuff headline is  wrong) but the courts do not appear to be friendly to the concept of uncertainty.

A story in Stuff about the benefits of ginger is to be commended for providing actual links to their supporting evidence for some of the claims (assuming you want more evidence than the approval of Confucius).

Unfortunately, if you provide links, there’s always the risk that people will follow them:

rich in antioxidants.  The linked paper describes chemical measurements of the antioxidant effects of ginger.  The abstract doesn’t support “rich” — the chemical analysis was of the antioxidant strength, not the concentrations of antioxidants and, at least in the abstract, didn’t compare to anything else (the journal, unusually, isn’t one that UoA library has access to).

combats nausea: This one appears to actually be true — it’s a combination of six randomised trials, and found ginger was better than placebo.  The researchers did note that publication bias was a concern and said the data are insufficient to draw firm conclusions.

natural pain relief.  The link here says that the result comes from the US National Library of Medicine.  That’s only true to the extent that it’s stored on their virtual shelves, like everything else published in biology and medicine. The study (by some Iranian scientists publishing in the Journal of Alternative and Complementary Medicine) compared ginger to two medications for period pain and didn’t find a statistically significant difference.  The researchers concluded that ginger was as effective, but their data don’t actually support this conclusion: you can’t conclude equivalence just from a lack of statistical signifiance.  If you look at the data in their Table 2 (which you can’t, since it’s not open-access), you can compute a 95% confidence interval for the difference in proportion of women who reported that the treatment helped: ibuprofen could have been 23 percentage points better than ginger, which is hardly a convincing demonstration of equivalence.

There’s also a link to some (then) unpublished research from the University of Sydney showing that chemicals in ginger inhibit the inflammation-related enzymes COX-1 and COX-2.  This link is from 2001 — I noticed how old it was because the researcher was talking enthusiastically about selectively inhibiting COX-2. As Vioxx did.  He said that he planned to do a study in actual patients. Nothing seems to have come of this study in the past decade: either it wasn’t done or it has succumbed to publication bias.

Natural arthritis relief.From the conclusion section of the linked abstract “Due to a paucity of well-conducted trials, evidence of the efficacy of Z. officinale to treat pain remains insufficient. However, the available data provide tentative support for the anti-inflammatory role of Z. officinale constituents,”

Stress reducer: The first link is to the Daily Mail. Enough said.  The second link  is introduced as “Ginger extract showed “significant antidepressant activity” in a study that was published in the International Research Journal of Pharmacy.”  A study in rats, if you follow the link.

 Anti-inflammatory: In test-tubes, ginger extracts inhibit some things related to inflammation. The abstract of the linked study concludes “Identification of the molecular targets of individual ginger constituents provides an opportunity to optimize and standardize ginger products with respect to their effects on specific biomarkers of inflammation. Such preparations will be useful for studies in experimental animals and humans.” In other words, we don’t know whether this translates to benefits in mice, let alone in people.

Antibiotic: This was the one that provoked me to write this post.  The story says “Ginger was more effective than antibiotic drugs in fighting two bacterial staph infections”.  The research says that high concentrations of ginger extract inhibited bacteria growing in a dish in lab more than low doses of antibiotics. No “infections” were involved in the research.

Common colds: The story says “Ginger contains almost a dozen anti-viral compounds and scientists have identified several that can fight the most common cold virus, the rhinoviruses.” The linked research doesn’t mention rhinoviruses, or any other kind of virus. It’s a lab study of four types of  bacteria.

Aids digestion: specifically, stimulates production of stomach acid and speeds emptying of the stomach.  The stomach-emptying is apparently true. No link is given for the stomach acid increase, but a Google search finds lots of web sites telling you how ginger can reduce stomach acid and help with gastric reflux.

Fights diabetes:  The story says “Ginger can help to manage blood sugar levels in long-term diabetic patients”.  The research says “one fraction of the extract was the most effective in reproducing the increase in glucose uptake by the whole extract in muscle cells grown in culture.” and “It is hoped that these promising results for managing blood glucose levels can be examined further in human clinical trials,”  So, again, this is lab bench research, not involving actual diabetic patients.

Boosts circulation: Ginger extracts inhibit blood clotting and platelet aggregation in blood samples in test-tubes.

So, we have one passing grade on nausea, and a partial pass on aiding digestion.  Two of the links provided absolutely no support for the claims, and the rest were mostly test-tube or rat research that might in the future lead to human research that might support the claims.

The AA is shocked (shocked!) to find that traffic and parking fines in Auckland add up to a lot of money.  The Herald did a good job on basic arithmetic, in converting traffic-fine totals of $36 million over 12 months and $20 million over eight months into the much less dramatic $3 million/month and $2.54 million per month.

One further piece of arithmetic would be to divide the $3 million per month by the 1 million registered vehicles in Auckland (table 36).  Is $3/month a surprising average?

For comparison, the Dominion Post reported total fines of “more than $12 million” in Wellington for the 2008-2009 financial year, on 283000 registered vehicles, giving a per-vehicle average of $3.5/month (before the GST increase).

Perhaps, as the AA’s Simon Lambourne believes, this indicates not enough effort put into education of drivers. Perhaps the idea of fining  people who don’t pay for parking  is “not being realistic about the importance of the car to mobility in Auckland.”  But the country’s primary motoring organisation can’t really get away with pretending surprise.

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

• Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday October 26 2012.
• Statistics can be bad, exemplary or fascinating.
• The statistic must be in the NZ media during the period of October 20 – 26 2012 inclusive.
• Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)