Tired of waiting for the National Institutes of Health to approve your grant proposal? If so, you're not alone: First-time applicants now wait an average of nine months to learn if their grant proposals have been approved, and as long as two years if they must revise and resubmit them. But help is on the way, say officials. The NIH is evaluating its peer review system to improve the process and make it more efficient, and is scheduling a pilot program to start in February to shorten the review cycle for new applicants. The pilot, which will be rolled out in about 20 NIH study sections, will shorten many of the steps in the application review process and also let first-time investigators submit revised applications four months sooner. If successful, the process may be expanded to all NIH grant applicants.
The topic of peer review "usually comes up within 30...
TIME FOR NEW CRITERIA?
The PRAC, an 18-member panel whose functions grew out of two earlier NIH advisory committees, has met three times this year to evaluate and recommend improvements to the peer review process. Suggestions being considered include: changing the scoring system as well as individual review criteria; including standard deviations in scoring to convey consensus among reviewers; adding a "degree of difficulty" as another criterion; and inverting the scale so 5.0 is highest and 1.0 is lowest. "There is no perfect system," says Norka Ruiz Bravo, deputy NIH director for extramural research. "I've always felt there was information missing that we could add to inform the program staff and the applicants. Now we're taking a step back to look."
The peer review process has changed relatively little in the 55 years since officials began to rate applications. The last assessment of the process occurred ten years ago when NIH Director Harold Varmus adopted the five criteria used today: significance, approach, innovation, investigator, and environment.
No more paper
In addition to a pilot program, the National Institutes of Health is moving wholly to the digital age in an attempt to streamline its grant-review process. As of October 1, it no longer mails hard copies of notification letters to principal investigators, and on February 1, it will stop sending those to individual fellow applicants. Instead, these documents will be available online. It's all part of the agency's effort to comply with the Congressional mandate to convert to electronic systems. (Find out more at
Currently, 84% of NIH's $28.8 billion budget is spent on research grants, contracts, and awards outside the agency. Research grants are the largest category of outside expenditures, totaling $19.6 billion or about 86 percent of NIH's competing and noncompeting awards given in fiscal 2004, the most recent year available. Research and development contracts were the next largest category, at $2.4 billion or about 10% of all awards given that year. Because NIH's budget has grown over the years, numbers for previous years are smaller but the percentages are roughly the same.
The peer review system has struggled to keep up with the expansion of NIH's budget and scope. Today, the time from grant submission to notification takes nine months, and only 20% to 25% of all first-time applicants receive funding. Those who do not may need to spend an additional four to five months revising their applications and then wait another nine months for their applications to be reviewed. "It seems to be a no-brainer that we have to shorten the cycle," says Antonio Scarpa, director of NIH's Center for Scientific Review (CSR), which receives grant applications and assigns peer reviewers to assess their scientific merit.
In fiscal 2005, CSR received about 78,000 grant applications, of which about 52,000 were evaluated by 15,000 peer reviewers – a 10% increase over the 47,400 applications reviewed in fiscal 2004. Over the past two decades the average age of first-time grantees has increased from 37 to 42 years for PhDs and from 39 to 44 years for MD and MD/PhD degree holders. While the total number of research grants has increased over time, the percentage of R01 grants to new individual investigators has remained steady at only about 6%.
SYSTEM IS 'AN ANACHRONISM'
Some contend, however, that peer review needs more radical changes than those under consideration. "The system was designed for a completely different era. It's an anachronism," says David Kaplan, a pathology professor at the Case Western University School of Medicine in Cleveland who has served on a number of NIH study groups. "Each application is 25 pages, single-spaced, and there may be 50 grants to review. It's an impossible task," he says.
Typically, each R01 grant application is assigned to two or three scientists who do written reviews and report their assessments to the 20 or more members of a study section. After a general discussion each member votes, giving the application a score from 1.0 (highest) to 5.0 (lowest) based on overall impact. These scores are averaged and converted to percentile rankings against applications reviewed in previous rounds.
"It's totally relying on two or three people," Kaplan wrote in the September 12 issue of
To help peer reviewers deal with the workload, Scarpa says PRAC members have discussed shortening the application. He is cautious about eliminating the written review, however, because "sloppy reviewers" could provide a score without providing supporting information. "Whether the application should be shorter, we can discuss. But in my personal view I think providing a critique is indispensable."
The PRAC's next meeting is scheduled for Jan. 23, 2006. No deadline has been set to present its recommendations to NIH leadership.
Interested in reading more?
Become a Member of
