WIKIMEDIA, W. FORBES-LEITHWe often look back on the past through rose-tinted glasses. We admit such might be the case in this instance, as we remember an age now so long-forgotten that it seems like scientific antiquity. In this bygone era (ending around 1997), core scientific principles and cordiality seemed to rule side-by-side. There was little talk of “supplemental data.” You could count the biomedical journals relevant to your field with both your hands. Most papers could be read and understood in less than an hour. Behind the scenes, the art of peer review was passed quietly from advisor to advisee, much as early Christian teachings were passed from coryphaeus to acolyte.
But times have changed: the Internet now rules, biomedical journals have proliferated, access is increasingly open, and big science—complete with ever-expanding datasets—is the new normal. Some of these trends have simplified the review of scientific manuscripts. For example,...
Still, peer review remains mostly an art passed from advisor to advisee. The advisor may be increasingly distracted by the chase for grant funding while the advisee may be unsure of a future in science. But the importance of high-quality peer review has never been greater.
With many of our colleagues, we have noted a decline in the standards for manuscript review. Increasingly, the dispositions and recommendations in reviews are not aligned at all, and sometimes they are disconcertingly orthogonal. We posit that the growing incongruence of reviews reflects, in part, increased variance in the parameters and standards being applied by different reviewers. Problems associated with manuscript review have been ascribed to the escalating conceptual and quantitative complexity in today’s science, the proliferation of journals, and the emergence of full-time, professional editors. Problems associated with peer review attributable to reviewers themselves are less frequently highlighted, but have also received some attention. We are not calling for an ecumenical council on peer review. But, given the changing times, it is appropriate that the review process itself receives more scrutiny and, perhaps, codification.
No system founded on subjective evaluations conducted by a spectrum of individuals can be perfect. Nevertheless, what follows is meant to be a short catechism on the art of reviewing manuscripts, as passed down by our mentors, and as we have attempted to transmit to individuals in our own laboratories. We describe a style of peer review that has worked best for us, both as authors and reviewers, while acknowledging that some of these recommendations may not be appropriate for all situations.
The Golden Rule
Be civil and polite in all your dealings with authors, other reviewers, editors, and so on, even if it is never reciprocated.
As a publishing scientist, you will note that most reviewers break at least a few of the rules that follow. Sometimes that is OK—as reviewers often fail to note, there is more than one way to skin a cat. As an author you will at times feel frustrated by reviews that come across as unnecessarily harsh, nitpicky, or flat-out wrong. Despite the temptation, as a reviewer, never take your frustrations out on others. We call it the “scientific community” for a reason. There is always a chance that you will be rewarded in the long run.
The Cardinal Rule
If you had to publish your review, would you be comfortable doing so? What if you had to sign it? If the answer to either question is no, start over. (That said, do not make editorial decisions in the written comments to the authors. The decision on suitability is the editors’, not yours. Your task is to provide a balanced assessment of the work in question.)
The Seven Deadly Sins of sub-par reviews
- Laundry lists of things the reviewer would have liked to see, but have little bearing on the conclusions.
- Itemizations of styles or approaches the reviewer would have used if they were the author.
- Direct statements of suitability for publication in Journal X (leave that to the editor).
- Vague criticism without specifics as to what, exactly, is being recommended. Specific points are important—especially if the manuscript is rejected.
- Unclear recommendations, with little sense of priority (what must be done, what would be nice to have but is not required, and what is just a matter of curiosity).
- Haphazard, grammatically poor writing. This suggests that the reviewer hasn’t bothered to put in much effort.
- Belligerent or dismissive language. This suggests a hidden agenda. (Back to The Golden Rule: do not abuse the single-blind peer review system in order to exact revenge or waylay a competitor.)
Vow silence
The information you read is confidential. Don’t mention it in public forums. The consequences to the authors are dire if someone you inform uses the information to gain a competitive advantage in their research. Obviously, don’t use the findings to further your own work (once published, however, they are fair game). Never contact the authors directly.
Be timely
Unless otherwise stated, provide a review within three weeks of receiving a manuscript. This old standard has been eroded in recent years, but nevertheless you should try to stick to this deadline if possible.
Be thorough
Read the manuscript thoroughly. Conduct any necessary background research. Remember that you have someone’s fate in your hands, so it is not OK to skip over something without attempting to understand it completely. Even if the paper is terrible and in your view has no hope of acceptance, it is your professional duty to develop a complete and constructive review.
Be honest
If there is a technique employed that is beyond your area of expertise, do the best you can, and state to the editor (or in some cases, in your review) that although outside your area, the data look convincing (or if not, explain why). The editor will know to rely more on the other reviewers for this specific item. If the editor has done his or her job correctly, at least one of the other reviewers will have the needed expertise.
Testify
Most manuscript reviews cover about a page or two. Begin writing by briefly summarizing the state of the field and the intended contribution of the study. Outline any major deficits, but refrain from indicating if you think they preclude publication. Keep in mind that most journals employ copy editors, so unless the language completely obstructs understanding, don’t bother criticizing the English. Go on to itemize any additional defects in the manuscript. Don’t just criticize: saying that X is a weakness is not the same as saying the authors should address weakness X by providing additional supporting data. Be clear and provide no loopholes. Keep in mind that you are not an author. No one should care how you would have done things differently in a perfect world. If you think it helpful, provide additional suggestions as minor comments—the editor will understand that the authors are not bound to them.
Judgment Day
Make a decision as to the suitability of the manuscript for the specific journal in question, keeping in mind their expectations. Is it acceptable in its current state? Would a reasonable number of experiments performed in a reasonable amount of time make it so, or not? Answering these questions will allow you to recommend acceptance, rejection, or major/minor revision.
If the journal allows separate comments to the editor, here is the place to state that in your opinion they should accept and publish the paper as quickly as possible, or that the manuscript falls far below what would be expected for Journal X, or that Y must absolutely be completed to make the manuscript publishable, or that if Z is done you are willing to have it accepted without seeing it again. Good comments here can make the editor’s job easier. The availability of separate comments to the editor does not mean that you should provide only positive comments in the written review and reserve the negative ones for the editor. This approach can result in a rejected manuscript being returned to the authors with glowing reviewer comments.
Resurrection
A second review is not the same as an initial review. There is rarely any good reason why you should not be able to turn it around in a few days—you are already familiar with the manuscript. Add no new issues—doing so would be the equivalent of tripping someone in a race during the home stretch. Determine whether the authors have adequately addressed your criticisms (and those of the other reviewers, if there was something you missed in the initial review that you think is vital). In some cases, data added to a revised manuscript may raise new questions or concerns, but ask yourself if they really matter before bringing them up in your review. Be willing to give a little if the authors have made reasonable accommodation. Make a decision: up or down. Relay it to the editor.
Congratulations. You’ve now been baptized, confirmed, and anointed a professional manuscript reviewer.
Matthew A. Mulvey and Dean Tantin are purely secular Professors of Pathology at the University of Utah, School of Medicine. The authors would like to thank Michael Carey and Janet Shaw for their helpful comments.
Interested in reading more?
The Scientist ARCHIVES
Become a Member of
