February 27, 2014

They couldn’t hit an elephant at this dist…

James Russell sent me a link to this story from a Canadian paper (originally from the Daily Telegraph).  The Herald has it too, with a very slightly less naff picture.  The research (open access) is good; the story is reasonably informative, but seriously credulous

Blood samples from over 17,000 generally healthy people were screened for 100 biomarkers, and those people monitored over five years.

In that time, 684 died from illnesses including cancer and cardiovascular disease. They all had similar levels of four biomarkers: albumin; alpha-1-acid glycoprotein; citrate, and a similar size of very-low-density lipoprotein particles.

Compare the last sentence to this graph from the research paper. The vertical axis is a combined score on the four biomarkers. The red dots are the people who died. As you can see, they didn’t all have similar values.

journal.pmed.1001606.g004

 

The research is impressive not because the prediction is very accurate, but because its less appalling inaccurate than usual.  Using standard risk factors (age, sex, cholesterol, smoking, diabetes, cancer) if you picked a random person who died and one who didn’t die from their cohort there’s an 80% chance the one with the worse risk factors was the one who died.  Adding the ‘death test’ measurements increases the probability to 83%.  Asking an experienced nurse to guess would probably be more accurate (and cheaper), but is hard to automate.

Despite the impression from the headline and lead, if you’re asked to predict whether someone will live another year, based on this sort of information, the safe bet is “yes”. Even among the 1% of people with the very worst values of the ‘death test’ biomarkers, 80% lived for more than a year and half were still alive at the end of the five year study.

Interestingly, the two republished versions lack the last paragraphs of the original Telegraph story, which talk about whether the test is useful

“If the findings are replicated then this test is surely something we will see becoming widespread,” added Prof Perola.

“But at moment there is ethical question. Would someone want to know their risk of dying if there is nothing we can do about it?”

Dr Kettunen added: “Next we aim to study whether some kind of connecting factor between these biomarkers can be identified.

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »