Not a good look
Clinical trials involve experimenting on humans, and so you want them to involve the minimum number of people and use the information as efficiently as possible. Part of that is committing in advance to what you expect the benefits of a treatment to be (the ‘primary endpoint’). If you got to look at the data first, and search for a favorable difference between the treated and untreated people you’d need a lot more evidence to convince people. That’s why clinical trial registration is important.
Derek Lowe, at In the Pipeline, has an unfortunate example of from biotech company studying stem cells in heart disease. Last year, the registration information at ClinicalTrials.gov said
To determine safety and the effect of intracoronary infusion of AMR-001 on myocardial perfusion (RTSS), measured by gated SPECT MPI at baseline and six months in subjects post-STEMI
and
primary endpoint includes safety of bone marrow procurement (measured by adverse events) and AMR-001 cell infusion (including incidence of re-stenosis and stent thrombosis in addition to other adverse events) as well as efficacy measured by quantitative by gated SPECT MPI specifically looking at resting total severity score)
On November 17 this year it changed
To determine safety and efficacy of intracoronary infusion of NBS10.
and
The primary endpoint includes the occurrence of AE’s, SAE’s and Major Adverse Cardiac Events (MACE) and the assessment of myocardial perfusion measured by quantitative gated SPECT MPI specifically looking at resting total severity score.
From the company’s press release
- A statistically significant mortality benefit (p<0.05) in patients treated with NBS10 (also known as AMR-001) as compared to the placebo group; there were no deaths in the treatment group.
- A statistically significant dose-dependent reduction in SAEs (p<0.05).
- Observation of a dose-dependent numerical decrease in MACE. MACE occurred in 14% of control subjects, in 17% of subjects of who received less than 14 million CD34 cells, in 10% of subjects who received greater than 14 million CD34 cells, and in 7% of subjects who received greater than 20 million CD34 cells.
- No meaningful difference in perfusion, as evidenced by SPECT imaging, between the treatment and the control group from baseline to 6 months in resting total severity score (RTSS) suggesting this may not be a future suitable tool to assess NBS10, which is consistent with U.S. Food and Drug Administration (FDA) guidance that mortality and MACE are the appropriate approvable endpoints to determine efficacy of a cellular therapy for cardiac disease as opposed to imaging endpoints.
That is, the trial failed to show a change where there were looking for one, but found evidence for a reduction in other things — apparently after they knew the results.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
The change log seems to indicate that AMR-001 and NBS10 are the same thing, but that doesn’t affect the point you’re making: http://clinicaltrials.gov/archive/NCT01495364/2014_11_17/changes
I’m quite surprised this is allowed at all. It seems to me that it rather undermines the purpose of registering a clinical trial in the first place, which I understood to be both ensuring that the results will be made known and reducing researcher degrees of freedom.
10 years ago
Sometimes you do need to make changes, eg in response to information from other studies, or because something turns out not to be feasible, so the registry has to allow changes and provide a history. You’d hope journals and the FDA would look at when the changes were made in a case like this.
10 years ago