FLICKR, NICMCPHEEThe best peer reviewers are the top experts in their fields—so goes the conventional wisdom. But the results of a study published last month (October 21) in PLOS ONE suggest that the harshest critiques often come from those reviewers who know their fields best, and that seniority and inherent bias can skew how some top experts evaluate manuscripts submitted to journals.
“These results highlight the need to explore how reviewers utilize their expertise,” Stephen Gallo of the American Institute of Biological Sciences in Reston, Virginia, and coauthors wrote in their study.
Peer reviewers are known to sometimes disagree on the merits of a given study. One recent analysis of biomedical funding applications found that reviewers with higher expertise detect more weaknesses than non-expert reviewers, suggesting that expertise might sway decision-making.
In the present study, Gallo and colleagues conducted a retrospective analysis of evaluations submitted by reviewers who had self-reported their levels of expertise and were aware of the demographics and training of the submitting authors. “We were then able to measure the effects of subject matter expertise on scoring in a scenario that may be governed ...
