NIH nearing the end of peer review review
NIH will hold a final working group meeting tomorrow (October 25) to discuss how to linkurl:amend peer review.;http://enhancing-peer-review.nih.gov/index.html The agency linkurl:kicked off;http://www.the-scientist.com/blog/display/53276/ its review of peer review this summer with the aim of "optimizing its efficiency and effectiveness"
NIH will hold a final working group meeting tomorrow (October 25) to discuss how to linkurl:amend peer review.;http://enhancing-peer-review.nih.gov/index.html The agency linkurl:kicked off;http://www.the-scientist.com/blog/display/53276/ its review of peer review this summer with the aim of "optimizing its efficiency and effectiveness" - a process many researchers have agreed is needed. The plan is to present results of the series of meetings at the end of this year, and to propose recommendations for change in January.
linkurl:David Kaplan,;http://path-www.path.cwru.edu/information6.php?info_id=30 a researcher at Case Western who has been very outspoken about peer review issues (he linkurl:wrote;http://www.the-scientist.com/article/display/15501/ two linkurl:opinions;http://www.the-scientist.com/article/display/15701/ for The Scientist on what's wrong with the current system) told me yesterday that he doesn't have much hope that any real change will come out of this review process. "Periodically, they review the process and find, 'Gee, we're not doing so well in identifying innovative stuff,'" he said. But the committee reviewing the process consists of prominent researchers such as Keith Yamamoto and Bruce Alberts from UCSF and Alan Leshner of AAAS - scientists who have excelled in the current system and thus are unlikely to see problems with it. "This is not a committee that's going to do that," he said.
This limitation is similarly evident in peer review itself, stacking the deck against innovation, he said: "The choice [about which grants get funded] are being made by a very few people" - again, those that are most established. He thinks that more reviewers should assess applications, and that this group should include reviewers outside that sphere, such as postdocs, assistant professors, or researchers with fewer publications. Even efforts like the one announced this summer, to fund linkurl:"wild and crazy";http://www.the-scientist.com/blog/display/53456/ projects fall short, he said. "The irony is, these are people they would have funded anyway."
Kaplan is quick to note that he doesn't believe that the current system should be thrown away, but widening the body of reviewers is just part of the fix that's needed. Some of his past suggestions have included cutting grant proposals down to a page or two, and getting rid of the false precision of the 41-point scale used to assess proposals. He has also linkurl:suggested;http://www.the-scientist.com/pubmed/17267383 a statistical measure for judging innovation, which could be built into the review process, and an overhall of statistical techniques the NIH uses in grant review, a proposal he presented to the Peer Review Advisory Committee last year.