More Than 1 Percent of Clinical Trial Reports Appear Flawed

A new screening tool flags dozens of papers with potential errors.

By Jef Akst | June 5, 2017

FLICKR, ESOAnalyzing 5,087 randomized controlled trials published between January 2000 and December 2015, John Carlisle, a consultant anesthetist at Torbay Hospital in the U.K., has found around 80 papers that raised cause for concern, with more than half very likely to contain errors in key data. Some of these papers have already been retracted, but many stand in the scientific literature.

While some of these errors may be the result of “misinterpretation, statistical error, or plain simple mistakes,” Carlisle says in a press release emailed to The Scientist, it’s likely that some of the research in question “may have been deliberately falsified.” He published his results today (June 5) in Anaesthesia.

“This study is about working to correct the scientific record,” Andrew Klein, the editor-in-chief of the journal, says in the press release. “The new Carlisle screening tool has been developed here in the U.K. and it is now clear that it should be used by medical publications around the world. The integrity of medical science demands that we do everything we can to ensure complete accuracy in the publication of research.”

See “The Zombie Literature

Carlisle’s screening tool looks at the distributions of means for the variables recorded, scanning for statistical inaccuracies. “Essentially, Carlisle’s method identi?es papers in which the baseline characteristics (e.g. age, weight) exhibit either too narrow or too wide a distribution than expected by chance, resulting in an excess of p values close to either one or zero,” University of Sydney anesthesiologists John Loadsman and Tim McCulloch wrote in an accompanying editorial. Further investigation into the studies in question will be required to determine the severity and source of the errors.

The flagged studies came from eight journals, including the New England Journal of Medicine, Anaesthesia, and JAMA. As Klein tells Retraction Watch, all the journals responded “swiftly and positively” to Carlisle, who reached out with the findings of his study.

The tool is freely available online for other researchers and journals to use.

Add a Comment

Avatar of: You



Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Popular Now

  1. Two University of Rochester Professors Resign in Protest
  2. Dartmouth Professor Investigated for Sexual Misconduct Retires
  3. Theranos Leaders Indicted For Fraud
    The Nutshell Theranos Leaders Indicted For Fraud

    Federal prosecutors filed criminal charges that allege the company’s promise to revolutionize blood testing swindled investors out of hundreds of millions of dollars and put patients in danger.

  4. Koko the Signing Gorilla Dies at 46