London School of Hygiene & Tropical Medicine

In an analysis published in the BMJ, medical journal editors including Professor Ian Roberts at the London School of Hygiene & Tropical Medicine state that the growing number of sub-standard clinical trials undermine systematic reviews and may render the findings invalid.

Currently, health professionals and patients depend on results from systematic reviews of relevant clinical trials to know whether treatments are safe and effective. These reviews evaluate data from all clinical trials.

Professor Roberts commented that "publishing clinical trials can help doctors rise through the ranks of the medical research hierarchy. Whilst most play the publication game fairly, some cheat."

 "Most Journal editors and systematic reviewers take clinical trial reports at face value with little or no effort to confirm whether a particular trial even took place." This impacts the quality of reviews: in a recent example, the conclusions of a review of starch infusions in critically ill patients changed substantially after excluding seven entirely fabricated trials.

The authors say the process of investigating and excluding such studies is time consuming, meaning that reviews that advise current health policy are often years out of date. Co-author Emma Sydenham said: "Doctors' career interests are putting the health of millions of patients around the world in jeopardy."

An additional problem highlighted by the authors is that publication is biased towards positive results: a successful clinical trial is more likely to be published than those showing no benefit or harm, and any side-effects are often neglected in the final report.

The authors believe that clinical trial information is of critical importance, as it forms the basis of life-changing medical decisions. They call for change in the system, and argue that systematic reviews should be based on the actual data from compulsory registers of clinical trials. These registers contain all the data recorded in a specific trial, including the study design, every procedure undertaken, and the basic results. Prof Roberts and colleagues urge that "showing all the information collected reduces problems associated with publishing only some of the findings - such as the most exciting data."

Podcast: Ian Roberts - Methodological gloss won't fix a rubbish evidence base

Publication:

• Ian Roberts, Katharine Ker, Phil Edwards, Deirdre Beecher, Daniella Manno and Emma Syndenham: The knowledge system underpinning healthcare is not fit for purpose and must change. BMJ Analysis DOI: 10.1136/bmj.h2463

Related links:

• Prof Roberts writes for The Conversation: Retraction of scientific papers for fraud or bias is just the tip of the iceberg