Headline inflation

The breakthrough of the decade doesn’t happen most years, and the breakthrough of the year doesn’t happen most weeks, but you still need to put out a health news section.  If you do it by hyping whatever turns up, your headlines end up not having a lot of information value.

So, today, “Blood test for ovarian cancer ‘100% accurate‘” in the Herald is grade inflation.  The researchers at Georgia Tech have some impressive findings, but their test still hasn’t been evaluated on anyone other than the 95 women whose cancer status was known in advance and whose blood was used to develop the test. As the research paper says

…because the disease is in low prevalence in the general population (~0.1% in USA), a screening test must attain a positive predictive value (PPV) of >10%, with a specificity ≥99.6% and a sensitivity ≥75% to be of clinical relevance in the general population

That is, they want the test to give no more than 4 false positives per 1000 healthy women. So far, they’ve only looked at 49 healthy women.

The story is better than the headline on how significant this is, with an independent expert.

Dr Simon Newman, of Target Ovarian Cancer, said: “It is exciting preliminary research. It’s crucial to diagnose ovarian cancer promptly, as up to 90 per cent of women would live for five or more years if diagnosed at the earliest stage.

“However, this highly promising discovery needs significant further development and validation in large clinical trials before we know if it is suitable for screening the general population and works as well as predicted.“

Even that’s exaggerated. We just don’t know what the survival would be with early diagnosis. At the moment, you have to be very fortunate to have your ovarian cancer detected at the earliest stage, and these tumours might be very non-representative.  We’ve seen real but smaller-than-expected benefits from screening in other cancers.

There are worse problems with the story than a bit of exaggeration, though. It gets the scientific idea completely wrong, saying:

But when Georgia Institute of Technology researchers looked at the blood of 46 women in the early stages of the disease and that of 49 healthy women, the cancerous samples contained different levels of 16 proteins compared with the healthy ones.

The innovative step in this research was to not use proteins. As the press release says

“People have been looking at proteins for diagnosis of ovarian cancer for a couple of decades, and the results have not been very impressive,”

Instead, the researchers looked at ‘metabolites’, smaller molecules produced by cell processes. Their hypothesis was that tumours might have varying genetic changes and varying proteins, but if they ended up as cancer they would have some cellular processes in common.