Menu

Opinion: We Must Demand Evidence of Peer Review

Peer review varies in quality and thoroughness. Making it publicly available could improve it.

May 21, 2018
Nikolai Slavov

ISTOCK, TEMMUZCANHave you read a paper and thought: “How could peer reviews support the publication of such a paper?” I have. More than once. Other times, I have read fascinating papers outside of my field and wondered what the concerns of the experts who peer reviewed the study were. What important caveats am I missing?

Sometimes, I am lucky and find the answers to such questions: A few publications, including those from EMBO Press and eLife, publish the peer reviews alongside the papers. Reading such peer reviews has provided an additional dimension of appreciating and understanding the experiments and the findings, especially when I am not very familiar with the topic. But for most other journals I cannot access the peer reviews that supported a paper’s publication because most journals hide them. 

The exit from this very sorry state is for journals to publish the anonymized peer reviews.

Scientific rigor demands that claims be substantiated by evidence. If I claim that gene A regulates gene B and provide no evidence, my claim will be dismissed. It must be dismissed. Yet, if a journal claims to conduct peer review and provides no evidence of it, the claim is rarely dismissed.

How do we know that a journal conducts peer review? For most journals, the evidence is limited to our anecdotal experiences with the manuscripts that we review ourselves or that we and our friends have submitted. For me this evidence is mixed. I know of manuscripts that have been thoughtfully reviewed and manuscripts that have undergone very expedited peer review or no peer review at all before appearing in the most prestigious journals. This anecdotal evidence is rather weak. If you ask me to substantiate it, I have to refer you to a friend who may or may not be willing to tell you that his or her paper was barely peer reviewed. It is a huge problem that the evidence for such a centrally important process is hidden from public view.        

The evidence for the quality of peer review and editorial oversight is even weaker. How can we evaluate the rigor of peer review at a journal that provides no public evidence that peer review happens? We cannot. The only scientifically justified conclusion is that we must doubt the existence and quality of peer review for any journal that does not publish the editorial and peer-review discussions that support its publishing activity.

The exit from this very sorry state is for journals to publish the anonymized peer reviews. Some journals do this, but they are the exception. The majority of the leading journals provide no evidence whatsoever of their peer review—or the lack of it. Yet peer review is currently the most important function of journals, and I believe that peer-review has much to contribute to the scientific discourse.

I can understand strong arguments for and against signing peer reviews. These arguments form a complex discussion without a simple solution. This discussion must not spill over and complicate the simpler question of whether anonymized peer reviews should be published. I believe they must be published. I see no justification for any journal to hide the anonymized peer reviews.

Nikolai Slavov is an assistant professor of bioengineering at Northeastern University. He is an academic editor at PeerJ.

January 2019

Cannabis on Board

Research suggests ill effects of cannabinoids in the womb

Marketplace

Sponsored Product Updates

FORMULATRIX® digital PCR technology to be acquired by QIAGEN
FORMULATRIX® digital PCR technology to be acquired by QIAGEN
FORMULATRIX has announced that their digital PCR assets, including the CONSTELLATION® series of instruments, is being acquired by QIAGEN N.V. (NYSE: QGEN, Frankfurt Stock Exchange: QIA) for up to $260 million ($125 million upfront payment and $135 million of milestones).  QIAGEN has announced plans for a global launch in 2020 of a new series of digital PCR platforms that utilize the advanced dPCR technology developed by FORMULATRIX combined with QIAGEN’s expertise in assay development and automation.
Application of CRISPR/Cas to the Generation of Genetically Engineered Mice
Application of CRISPR/Cas to the Generation of Genetically Engineered Mice
With this application note from Taconic, learn about the power that the CRISPR/Cas system has to revolutionize the field of custom mouse model generation!
Translational Models of Obesity, Dysmetabolism, Diabetes, and Complications
Translational Models of Obesity, Dysmetabolism, Diabetes, and Complications
This webinar, from Crown Bioscience, presents a unique continuum of translational dysmetabolic platforms that more closely mimic human disease. Learn about using next-generation rodent and spontaneously diabetic non-human primate models to accurately model human-relevant disease progression and complications related to obesity and diabetes here!
BiochemAR: an augmented reality app for easy visualization of virtual 3D molecular models
BiochemAR: an augmented reality app for easy visualization of virtual 3D molecular models
Have you played Pokemon Go? Then you've used Augmented Reality (AR) technology! AR technology holds substantial promise and potential for providing a low-cost, easy to use digital platform for the manipulation of virtual 3D objects, including 3D models of biological macromolecules.