More Than 1 Percent of Clinical Trial Reports Appear Flawed

A new screening tool flags dozens of papers with potential errors.

By | June 5, 2017

FLICKR, ESOAnalyzing 5,087 randomized controlled trials published between January 2000 and December 2015, John Carlisle, a consultant anesthetist at Torbay Hospital in the U.K., has found around 80 papers that raised cause for concern, with more than half very likely to contain errors in key data. Some of these papers have already been retracted, but many stand in the scientific literature.

While some of these errors may be the result of “misinterpretation, statistical error, or plain simple mistakes,” Carlisle says in a press release emailed to The Scientist, it’s likely that some of the research in question “may have been deliberately falsified.” He published his results today (June 5) in Anaesthesia.

“This study is about working to correct the scientific record,” Andrew Klein, the editor-in-chief of the journal, says in the press release. “The new Carlisle screening tool has been developed here in the U.K. and it is now clear that it should be used by medical publications around the world. The integrity of medical science demands that we do everything we can to ensure complete accuracy in the publication of research.”

See “The Zombie Literature

Carlisle’s screening tool looks at the distributions of means for the variables recorded, scanning for statistical inaccuracies. “Essentially, Carlisle’s method identi?es papers in which the baseline characteristics (e.g. age, weight) exhibit either too narrow or too wide a distribution than expected by chance, resulting in an excess of p values close to either one or zero,” University of Sydney anesthesiologists John Loadsman and Tim McCulloch wrote in an accompanying editorial. Further investigation into the studies in question will be required to determine the severity and source of the errors.

The flagged studies came from eight journals, including the New England Journal of Medicine, Anaesthesia, and JAMA. As Klein tells Retraction Watch, all the journals responded “swiftly and positively” to Carlisle, who reached out with the findings of his study.

The tool is freely available online for other researchers and journals to use.

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Popular Now

  1. Running on Empty
    Features Running on Empty

    Regularly taking breaks from eating—for hours or days—can trigger changes both expected, such as in metabolic dynamics and inflammation, and surprising, as in immune system function and cancer progression.

  2. Athletes’ Microbiomes Differ from Nonathletes
  3. Mutation Linked to Longer Life Span in Men
  4. Gut Feeling
    Daily News Gut Feeling

    Sensory cells of the mouse intestine let the brain know if certain compounds are present by speaking directly to gut neurons via serotonin.

AAAS