Posts filed under Evidence (90)

Stuff has a story with the headline “More kids committing sexual abuse” and the lead

Easy access to increasingly hardcore pornography and the sexualisation of childhood are being blamed for a rise in the number of children sexually abusing each other.

However, if you actually read the story, the only real information is on number of prosecutions, and

 Police say the jump in prosecutions was due to better knowledge and increased reporting of sexual abuse, rather than a rise in incidents.

No other evidence or expert opinion is given to support the claim of a real increase in abuse, which seems to be entirely made up.

And while on the subject of factchecking, Netsafe chief technology officer Sean Lyons is quoted as saying

“You type ‘kiwi chicks’ into Google and the images that come back won’t be small feathered birds.”

I tried this. On the first page of results, eight links and the only image displayed were for small feathered birds, one was to a museum project “‘Kiwi Chicks: New Zealand Girl History/Ngā Kōhine Kiwi: He Hitori Taitamāhine o Aotearoa’”, and one was to a page at TheRockFM, which might offend some people but is not anything shocking for kids.

I got basically the same page using lmgtfy.com in a browser not logged into any Google services, so it isn’t just my pure and innocent search history that’s biasing the results.

Update: here’s the first screenfull of results from a Google image search for ‘kiwi chicks’

From the inimitable Dan Davies, a post on how often you’d expect all the front-page photos in major UK newspapers to be of white people

So a while ago on Twitter, I saw this storify by @KateDaddie, talking about ethnic minority representation in the British media, in the context of this article by Joseph Harker in the British Journalism Review. As I am a notorious stats pedant and practically compulsive mansplainer, my initial reaction was to fire up the Pedantoscope and start nitpicking. On the face of it, it is not difficult to think up Devastating Critiques[1] of the idea of counting “#AllWhiteFrontPages” as an indicator of more or less anything. But if I’ve learned one thing from a working life dealing with numbers (and from reading all those Nassim Taleb and Anthony Stafford Beer books), it’s that the central limit theorem will not be denied, and that simple, robust metrics with a broad-brush correlation to the thing you’re trying to measure are usually better management tools than fragile customised metrics which look like they might in principle be better.

The New York Times has reported on a study in which observers sat discreetly in bathrooms and observed whether people “properly” washed their hands (I reckon it would be quite hard to sit discreetly in a bathroom unless you’re in a cubicle). Anyway, the description of the study gave careful attention to the stats: 10.3% of women and over 15 percent of men didn’t wash at all. Of those who did wash, 22.8% did not use soap. And only 5.8% washed for more than 15 seconds.

The lead author said, “Forty-eight million people a year get sick from contaminated food, and the (American) Centre for Communicable Diseases says 50% would not have gotten sick if people had washed their hands properly. Do as your mum said: Wash your hands.”

Surely there’s some basic confusion over percentages here: 50% of those who got sick wouldn’t have if everyone had washed their hands properly, but we have no idea what percentage of those who don’t wash actually get sick.

As a matter of fact, there is no indication that these particular non-handwashers have anything to do at all with the fact that people eat contaminated food. Does it matter what bathroom activity was being carried out? Whether you use toilet paper or your foot to flush? Whether you work in food services? Whether you subsequently wash your hands before eating dinner?

Though mum may have had good advice, this sort of scare-mongering about food-borne illnesses resulting from not washing one’s hands may actually distract us from the real concerns over germs.

• Read the full analysis by Rebecca Goldin, here. She is Director of Research for STATS, an American non-profit, non-partisan service that  helps journalists think quantitatively through providing education, workshops and direct assistance with data analysis.

If you have read media stories recently that say ‘yes’, you’d better read this article from the Genetic Literacy Project …

While looking into mobile internet options for a trip to Europe, I saw an ad for one of those products that’s supposed to stop dangerous mobile-phone radiation — as usual, it probably wouldn’t work even if dangerous mobile-phone radiation existed.

The company (which is in NZ), says

Cellguard® uses Frequency Infused Technology (FIT) which works to enhance the Bio energy function of the body.

With enhanced Bio energy function your body is better able to maintain an optimum state of wellbeing and significantly reduce the impact of the considered effects of mobile phone use.

which I think qualifies as “not even false”.  They also sell a product that is supposed to improve the acid/alkaline balance in your body — if you drink it, or rub on your skin or sprayed it up your  nose.

a modified liquid silica that is high in oxygen and is highly alkaline to help offset our acidic lifestyles. Alka Vita has a high pH of around 14.3 and is non corrosive ..

The ‘high in oxygen’ doesn’t sound plausible, but who knows? On the other hand, if it has a pH of 14.3 and is non-corrosive, they clearly don’t mean what chemists mean by ‘pH’.  14.3 is more alkaline than drain cleaner, and 60 times more alkaline than the NZ legal limit for dishwasher detergent.

Fortunately, the legal disclaimer page says

The information provided on this website is not intended as professional advice, but as guidelines for convenience only, upon the condition that you, by receiving or reading the material contained on this website, agree not to act in reliance upon it without first satisfying yourself by independent inquiry or advice as to the suitability, appropriateness, relevance, nature, fitness or purpose, likely side effects or long term effects, accuracy, reliability or otherwise of that material, having regard (without limitation) to your physical state, and your general fitness or medical condition.

Stuff has a headline “Overseas advice: ‘Don’t test party pills on animals’” and lead:

New Zealand will fly in the face of international animal welfare conventions if it allows legal highs to be tested on animals.

However, if you actually read the story, you find that this is not just misleading, but clearly untrue.

They mention responses from three countries. In Canada

… unable to find “any information suggesting” there was a ban on using animals in recreational drug or alcohol research testing in Canada.

In France and the EU

But France recently banned the use of animal testing in the cosmetic industry and policymakers are under pressure to extend that to the legal high, alcohol and tobacco industry.

The European Union has also introduced a ban on the sale of any cosmetic or personal care product which was tested on animals.

and only in the UK was the response consistent with the story

Judy MacArthur Clark, head of the animals in science regulation unit at UK’s Home Office, replied that any proposal to test for the safety of recreational drugs would be “rejected”.

You might also ask how these countries test legal highs for safety, if they don’t use animal testing.  The answer, of course, is that they don’t test legal highs for safety. Not one of these countries has legislation or procedures for approving and regulating new recreational psychoactive substances, but they all do require animal testing for new medications, two facts that Stuff might have considered mentioning.  The reference to LD50 lethal toxicity testing is especially misleading, since the same paper claimed credit in January for the fact that LD50 tests had been ruled out by the Minister.

The ‘legal highs’ regulations are new. No other country has anything similar.  New Zealand can’t base its testing regimes on what other countries are doing, because other countries aren’t doing it.  It would be useful to have some informed public discussion on how testing should be done, since it does involve the sort of tradeoffs where, in a democracy, the public should have input. This sort of story doesn’t help. Quite the reverse, in fact.

[Update: Here are three research papers involving animal (mouse and rat) testing of synthetic cannabis compounds done primarily for regulatory or forensic purposes, in Australia, the US, and Russia]

The Herald has a completely over-the-top presentation of what might be an important issue. The headline is “Hospital choice key to kids’ survival”, and the story starts off

Where ambulances take badly injured children first seems to affect their chances, paediatric surgeons say.

Starship children’s hospital surgeons have found that sending badly injured children to the wrong hospital may be contributing to a child death rate from injuries that is twice the rate of Australia’s.

The data:

Six (7 per cent) of the 88 children who went first to Middlemore died, but so did one (8 per cent) of the 12 who went directly to Starship.

That is, to the extent the data tell us anything, the evidence is against the headline.  Of course, the uncertainties are huge: a 95% confidence interval for the relative odds of dying after being sent to Middlemore goes from a 40-fold decrease to a 12-fold increase.  There’s basically no information in the survival data.

So, how much of the two-fold higher rate of death in NZ compared to Australia could reasonably be explained by suboptimal hospital choice? One of the surgeons involved in the study says

… overseas research showed that a good trauma protocol system could cut the death rate for injured adults by 20 to 30 per cent, but there was no good data for children.

That is, hardly any of the difference between NZ and Australia — especially as this specific hospital-choice issue only applies to one sector of one city in New Zealand, with less than 10% of the national population.

On the other hand, we see

The head of Starship’s emergency department, Dr Mike Shepherd, said the major factors contributing to New Zealand’s high fatal injury rate for children lay outside the hospital system in policies such as driver blood-alcohol limits, graduated driver licensing, and laws requiring children’s booster seats and swimming pool fences.

That sounds plausible, but if it’s the whole story you would expect high levels of non-fatal as well as fatal injuries. The overall rate of hospitalisations for injuries in children 0-14 years is almost identical in NZ (1395 per 100 000 per year, p29) and Australia (‘about’ 1500 per 100 000 per year, page v).

A long piece in the Washington Post: by Ezra Klein, recommended by Atul Gawande

Brenner puts it more vividly. “There is a bias in medicine against talking to people and for cutting, scanning and chopping into them. If this was a pill or or a machine with these results it would be front-page news in the Wall Street Journal. If we could get these results for your grandmother, you’d say, ‘Of course I want that.’ But then you’d say, what are the risks? Does she need to have chemotherapy? Does she need to be put in a scanner? Is it a surgery? And you’d say, no, you just have to have a nurse come visit her every week.”

• BBC news report on the Amanda Knox case and misuse of statistics: judge did not believe that repeating a test could increase its accuracy (via Mark Wilson)
• A tool for finding stories based on light makeover of press releases. Unfortunately, it often doesn’t detect recycling of stories from sites such as Medical Daily and Science Daily, which do enough rewriting to mask the sources.
• New York Times opinion piece on evidence and science reporting — yes, distinguish experimental and observational studies, but also distinguish small exploratory studies from larger confirmatory ones. (via @brettkeller)
• Using anecdotes rather than data to convince patients. On one hand, speaking to people in language they understand is good; on the other hand, you can use anecdotes to support anything.

I don’t usually bother with general nutrition stories that don’t contain any direct reference to research, but the Herald story about berries was irresistible. There are lots of biologically active compounds in berries, and many of them have been shown to have interesting properties in test-tubes or mice. As you know by now,  this sort of interesting biochemistry is important because it occasionally translates to genuine health benefits, so you should be asking what the human clinical research shows.

If you go to the Cochrane Library (which is free to everyone in New Zealand), and look for clinical research in humans involving blueberries or cranberries you don’t find much. The only topic with enough information to draw any sort of conclusion is on cranberry juice to prevent urinary tract infections. Which it basically doesn’t. The plain-language summary says