FLICKR, ESOAnalyzing 5,087 randomized controlled trials published between January 2000 and December 2015, John Carlisle, a consultant anesthetist at Torbay Hospital in the U.K., has found around 80 papers that raised cause for concern, with more than half very likely to contain errors in key data. Some of these papers have already been retracted, but many stand in the scientific literature.
While some of these errors may be the result of “misinterpretation, statistical error, or plain simple mistakes,” Carlisle says in a press release emailed to The Scientist, it’s likely that some of the research in question “may have been deliberately falsified.” He published his results today (June 5) in Anaesthesia.
“This study is about working to correct the scientific record,” Andrew Klein, the editor-in-chief of the journal, says in the press release. “The new Carlisle screening tool has been developed here in the U.K. and it...
See “The Zombie Literature”
Carlisle’s screening tool looks at the distributions of means for the variables recorded, scanning for statistical inaccuracies. “Essentially, Carlisle’s method identi?es papers in which the baseline characteristics (e.g. age, weight) exhibit either too narrow or too wide a distribution than expected by chance, resulting in an excess of p values close to either one or zero,” University of Sydney anesthesiologists John Loadsman and Tim McCulloch wrote in an accompanying editorial. Further investigation into the studies in question will be required to determine the severity and source of the errors.
The flagged studies came from eight journals, including the New England Journal of Medicine, Anaesthesia, and JAMA. As Klein tells Retraction Watch, all the journals responded “swiftly and positively” to Carlisle, who reached out with the findings of his study.
The tool is freely available online for other researchers and journals to use.
Interested in reading more?
The Scientist ARCHIVES
Become a Member of
