Cherry-Picking Trial Results

Providers, patients and policy-makers utilize the results of randomized clinical trials as important inputs into decision-making.  They typically learn of those results through publication in medical and scientific journals.  Questions have been raised about whether what ends up being published is selectively positive and studies suggest the answer is yes.  The selection bias occurs both across all trials, where those trials with positive outcomes tend to published more often than those with negative outcomes, and within a specific trial, where those outcomes with positive results may be more frequently discussed than those outcomes that did not reach a positive result.  An analysis in the Journal of the American Medical Association indicates that even though steps have been taken to try to reduce the occurrence of selective reporting, it still may happen with some frequency.  (JAMA Article)

One method of ensuring appropriate peer review of reports of trials submitted for publication has been to require registration of the trials.  The JAMA study looked at research reports in selected medical journals, examined the registration status of the trials reported, including the quality of the registration information, and compared the outcomes reported with those registered in the trial.  The study found that it appears that often the primary outcome discussed in the publication differs from that listed as the primary outcome in the registration and that this usually occurs when the outcome discussed in the publication has a positive result.  The commentary in this analysis suggests that peer reviewers and editors need to ensure that readers are getting the full picture.  Given the importance of published research, both medical journals and regulators should take steps to ensure that trials are registered with fully accurate information before the trial starts and that all results, positive and negative, are published.