PNAS review policy... by numbers

Some in the research community grouse about how members of the National Academy of Sciences (NAS) can hand pick reviewers and essentially fast track the publication of their papers or papers written by select non-academy members in the high-impact __Proceedings of the National Academy of Sciences of the United States of America__ (__PNAS__). But a citation analysis linkurl:published;http://www.plosone.org/article/info:doi/10.1371/journal.pone.0008092?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+plosone/PLoSONE+(PLoS+ONE+Alerts:+New+Articles)&utm_content=Google+Reader yesterday (1st December) in the __Public Library of Science ONE__ (__PLoS ONE__) suggests that the practice may be doing just what it's meant to do -- facilitate the publication of highly innovative research that might not make it into such a visible journal otherwise.

Image: Wikimedia Commons

"The alternative publication tracks that __PNAS__ provides seem to do a good job in giving NAS members more autonomy and letting them publish really groundbreaking, highly-cited, high-impact work while letting some lower quality work get in," Harvard evolutionary biologist linkurl:David Rand,;http://www.people.fas.harvard.edu/~drand/ lead author on the __PLoS ONE__ paper, told __The Scientist__. "You lower the average quality, but you also pick up the really great stuff. That's a pretty good model." Research papers appear in __PNAS__ via one of three tracks -- manuscripts can be submitted directly to the journal, with the __PNAS__ editorial board overseeing a traditional peer review process, NAS members can submit their own papers after procuring at least two reviews from referees of their choosing, or NAS members can submit manuscripts on behalf of non-members, marshalling the paper through peer review themselves and submitting reviews along with the manuscript. These different tracks result in the publication of "Direct submissions," "Contributed," or "Communicated" papers, respectively. Rand and his coauthor, Harvard evolutionary biologist linkurl:Thomas Pfeiffer;http://www.people.fas.harvard.edu/~pfeiffer/ (neither of whom are NAS members), reviewed citation data for more than 2600 __PNAS__ papers published between June 2004 and April 2005. They found that overall, "Contributed" papers -- those written by NAS members -- tend to be cited less than "Direct submissions" by researchers who do not belong to the academy and whose papers go through standard, blinded peer review. But (and this is an interesting but) though "Direct submissions" tend to be cited more on average, they are less likely to be "truly exceptional papers," Rand and Pfeiffer write. They compared the 10% most cited "Contributed" papers to the 10% most cited "Direct submissions," and found that the most cited of the former receive significantly more citations than the most cited of the latter. "Contributed" papers seem to be less influential on average, they reason, but the best of them tend to have more of an impact on their fields than do the best papers from non-NAS members. "Communicated" papers, the researchers found, received roughly equivalent levels of citation as did papers submitted directly to the journal. But again, the top 10% of "Communicated" papers were cited more than the top cited directly submitted papers. Rand and Pfieffer chalk up this average citation discrepancy to the different ways in which papers arriving at __PNAS__'s doorstep via the three submission tracks are scrutinized by reviewers. Contributed papers are reviewed by referees selected by the author, and this, Rand and Pfeiffer write, may "soften the challenges of the peer review process" for NAS member authors who may be writing out of their area of expertise. This may result in the publication of lower quality "Contributed" papers relative to "Direct submissions," they suggest, but it may also help NAS member-written papers that challenge scientific norms or tweak long-held axioms get into the pages of __PNAS__ faster. "The benefit of facilitating publication of extremely high-impact Contributed papers could be argued to out-weigh the potential cost of allowing more low quality papers to also be published," they write. One factor that Rand and Pfeiffer did not account for, however, is whether or not the __PNAS__ papers they analyzed were press released by the journal. "The effect of press releases, and popular press coverage more generally, on citation counts is an open question which deserves further study," they write. Recently, __PNAS__ linkurl:announced;http://www.the-scientist.com/blog/display/55970/ that the journal will be doing away with the option for NAS members to communicate papers for non-members starting July 1, 2010. Announcement of this decision coincidentally occurred as __PNAS__ was fielding linkurl:criticism;http://www.the-scientist.com/blog/display/56035/ for publishing a controversial -- and "Communicated" -- paper suggesting that butterflies are the evolutionary result of a long-ago mating between worm-like and winged ancestors.
**__Related stories:__***linkurl:PNAS butterfly flap heats up;http://www.the-scientist.com/blog/display/56035/
[1st October 2009]*linkurl:PNAS scraps special submission;http://www.the-scientist.com/blog/display/55970/
[10th September 2009]*linkurl:Publishing in PNAS;http://www.the-scientist.com/article/display/18209/
[14th September 1998]