Predicting dementia

The Herald has a story about a potential blood test for dementia, which gives the opportunity to talk about an important statistical issue. The research seems to be good, and the results are plausible, though they need to be confirmed in a separate, larger sample before they can really be believed. Also, the predictions so far are just for mild cognitive impairment, not actual dementia. But it’s the description of the accuracy of the test that might be misleading.

The test had 90% sensitivity — 90% of who developed cognitive impairment tested positive. It had 90% specificity — 90% of those who who did not develop cognitive impairment tested negative.  That’s what is described in the story as 90% accuracy.  What a user would care about is the positive predictive value: if you test positive, how likely are you to get cognitive impairment?

In the study 451 people started out cognitively normal; 28 of these developed impairment, the other 423 did not. The test would be correctly positive for about 25 of the 28, and correctly negative for about 381 of the 423. So, of the 25+42=67 who test positive, less than 40% will develop impairment. That’s reasonable for a  diagnostic test but a bit low for a screening test in healthy people.

Where the test is more immediately relevant is in designing clinical trials. So far, attempts to affect Alzheimer’s Disease progression have failed, though there are some modestly effective symptomatic treatments. It’s possible that the treatments are doing the right thing but that clinical illness is too late, so there’s a lot of interest in testing treatments very early in the process. A test like the new one could be very useful