Panel Weighs Overhaul Of NSF's Grant System

An agency report suggests merit review revisions that would simplify proposals, extend grant duration, and give staff more autonomy WASHINGTON--A report of an in-house panel that examined peer review at the National Science Foundation proposes drastic remedies for what it says is an overburdened and inefficient system of selecting and awarding grants. Its recommendations--longer-term grants, more autonomy for program officers and less reliance on outside reviewers, fewer categories of grants, and shorter and simpler proposals--would have to be approved by Walter Massey, President Bush's nominee to be NSF director, assuming he is confirmed by the United States Senate. In the meantime, some early reaction from the scientific community has been positive. For Tulane University physicist John Perdew, the panel's suggestions would mean good riddance to what he believes is now a lottery among highly qualified proposals. For population biologist Robert Wyatt of the University of Georgia, they would mean a welcome reliance on a scientist's track record in reviewing and awarding grants. And for ecologist Garth Redfield, they offer the hope of improved working conditions for himself and his fellow program managers at NSF. The report of the Merit Review Task Force, completed this past summer and then sent out to all grantees for comment, so far has attracted little attention. Yet the study has the potential of changing the way that NSF passes out money--nearly $2 billion this year--for scientific and educational research and training. WASHINGTON--The National Science Foundation isn't the only federal research agency that's taking a fresh look at its peer review system. The National Institutes of Health is also trying to refine a granting process created during an era of plenty that works much less efficiently when the demand for its funds greatly exceeds the supply. For scientists with NIH grants, recommended changes are likely to give them a better idea of their long-term funding at the expense of greater uncertainty over the size of their initial award. Unlike the NSF exercise, the impetus for NIH's draft report on spending policies, released last month and debated vigorously at a public hearing December 17, did not come from within. Rather, the NIH report, entitled "A Plan for Managing the Costs of Biomedical Research," is a response to suggestions from the appropriations committees in both the House of Representatives and the Senate about how NIH could improve its financial management system (The Scientist, Sept. 4, 1990, page 1). With respect to the peer review system, those suggestions include: ending the practice of cutting grant budgets by as much as 15 percent after a project had been funded (a process known euphemistically as "downward negotiations"); shortening the length of grants from the current 4.3 years to four years; holding down the increased size of continuing grants to the rate of a special price index tied to the cost of biomedical research; cutting back on so-called indirect costs that can add up to 80 percent to the overall cost of a research grant; and eliminating the category of "approved but not funded" grants. For some scientists, the most immediate change may be in the way that NIH study sections judge their applications. At present, says John Diggs, NIH deputy director for extramural research, only proposals that "are totally devoid of scientific merit" are rejected; that criterion allows 95 percent or more of all proposals to pass muster. Unfortunately for principal investigators, NIH doesn't have nearly enough money to fund all those ideas. So about two-thirds of those proposals wind up in a bureaucratic limbo known as "approved but not funded." That classification confused Congress and enraged scientists. So NIH has decided to affix the word "approved" only to those ideas that win funding, says Diggs. That would put NIH in line with other federal agencies that calculate a simple success rate for applicants by dividing the total number of awards by the total number of applications. At the same time, says Diggs, NIH will also try to relieve some of the enormous pressure on the system caused by the large number of excellent proposals that are not funded, a problem that Congress identified in its appropriations language. To do so, Diggs says, "we will be instructing our reviewers to identify those proposals that have no significant scientific merit." Such proposals, according to the draft report, would "not be recommended for further consideration." Study sections now reject fewer than 5 percent of NIH proposals. Diggs predicts that percentage will increase under the new system because "significant merit" is a higher threshold of quality, he says, than the current standard. The rest of the proposals will be given priority scores and percentile rankings, as is now the case, and will be sent to the advisory council at the institute most likely to be interested in funding the work. For scientists who succeed, the good news about the funding of their grant is often tempered by bad news about its size. For several years NIH has imposed unilateral and across-the-board reductions in awards--in 1990 those cuts averaged nearly 15 percent--to conform with Congress's wish to fund as many grants as possible. This year, for example, Congress has urged NIH to fund 6,000 new and competing grants, compared with 4,600 in 1990, and provided $800 million in new money to do so (The Scientist, Dec. 10, 1990, page 6). NIH officials predict they will come close to meeting that target, with an expected 5,785 new and competing grants. Under the new plan, Diggs says, NIH will replace such arbitrary decisions on funding with an examination of each grantee's requested level of support. The new method, he says, will allow each institute to meet its program priorities and preserve its autonomy. But he acknowledges that budget considerations will still shape what NIH can do. And he says that the results may not be any better. "Congress doesn't want us to go across-the-board," says Diggs. "So, instead, we will ask review groups to give us their best estimate of the cost of the research, and then have program staff and each council look case-by-case. "There's no way we can fund 6,000 new grants with our 1991 appropriations," he adds. "Furthermore, if we can't use across-the- board cuts, then we're going to have to find another way to accomplish the same goal. And that means that, if some grants are cut by only 5 percent, then others may have to be cut by 20 or 25 percent." But there is good news as well, says Diggs. NIH has begun to notify current grantees that their out-year funding will remain stable, rising by 4 percent annually to account for inflation. This is a change, he says, from past years, when NIH officials sometimes imposed arbitrary reductions on the size of grants after the agency came up short in its annual appropriations request. "We will be telling people...that we will not being making further reductions in their grants," says Diggs. "We feel, based on our current data, that we can support current grantees at their present level plus inflation." That annual adjustment, however, may be lower than the rate recorded by a special price index linked to the cost of biomedical research (BRDPI), which tends to rise faster than the overall consumer price index (CPI). "It may be closer to the CPI than to the BRDPI," he says, "but we're going to try to keep it at 4 percent." The congressional demand to manage its research accounts better is forcing NIH to look for ways to stretch its research dollars. One such way, of course, would be to support cheaper research over more costly science. Diggs suggests that time may have already arrived. "There's no question that we'll continue to fund the proposals at the top of the pile," he says. "But at the margins, when you have two proposals in the same area with similar scores, the one with the lower total costs may get the nod. It's simply a reflection of what Congress is asking us to do." --J.M. The report points to an increasing workload for program officers, as well as the demands of administering a proliferation of new programs. The result is rising pressure on a system that has changed little since it was created 40 years ago. "The world has changed and we don't seem to have the answers," says William Harris, cochairman of the task force and director of NSF's office of science and technology research centers. "Each year you hope that the next year will be better, but next year never comes." At the root of most of the problems the report addresses is money. An ideal situation for Harris and the scientists who rely on NSF for support would be dependable annual increases in all parts of the agency's budget. Such growth would ease the mounting competition for grants, which leads to the phenomenon of hundreds of scientists spending thousands of hours to apply for a program in which only a handful of awards will be made. "If NSF had gotten enough to double its budget [an often-repeated goal of Reagan- and Bush-era administrators], then there wouldn't be a problem," says Perdew, who has received NSF support for a dozen years for his work on the density functional theory of atoms, molecules, and solids. "NSF has gotten increases, but they've gone into programs other than those that support individual investigators." For Perdew and other scientists, NSF has become a less dependable source of funding over the past several years. With only a one-in-four chance of getting money, applicants must spend more time on the process itself--writing proposals, revising them, and shopping around for other funding agencies--if they want to maintain an active research career. "It's a sweepstakes, really," says Wyatt, who has had NSF grants for the past 10 years to study plant populations. "If you happen to be one of the lucky few, you get hundreds of thousands of dollars [over the life of a grant]. But otherwise you get nothing, and there's nothing to tide you over until the next round." For program managers, the increased number of proposals and revised applications--as many as 30 percent of investigators try again, according to the report--are equally discouraging. The added volume translates into less time spent on each application. One unfortunate consequence, says NSF's Redfield, is that the hottest areas of science sometimes get short shrift because of the large number of applications that they generate. A program officer's workload varies among disciplines, according to the report, ranging from fewer than 60 proposals per year in many areas of geology to more than 100 proposals annually in much of the biological and behavioral sciences directorate. Program officers also must deal with the burgeoning array of funding programs that NSF offers: more than a dozen new ones established in the 1980s alone, each with its own goals, procedures, and deadlines. Combined with curbs on travel, instituted over the past few years to save money, the result is that program officers have less and less time to keep tabs on the latest developments affecting the field, and, for some of them, to maintain ties with their own research groups. "The system is near collapse," says Redfield, who left NSF's ecology program last month after eight years to become division director for environmental sciences for the South Florida Water Management District. "For several years the attitude around here [NSF] has been that, as long as we get the paperwork out the door, everything's okay. And all the long-term planning, liaisons with the community, and other work that needs to be done to keep the system healthy gets postponed, a victim of the daily grind." The proposed changes in the grant awards system are meant to make the process both more predictable and more efficient. Specifically, the report urges stretching the term of a standard research grant from two to three years. In addition, the program officer would have the option of renewing such a grant, after careful oversight of the existing grant, without reconvening a formal external review panel. The result, for many scientists, would be a total of six years of continuous support. That would more than double the time that many investigators now have to focus on their research before submitting another grant proposal. NSF would also need fewer review panels, meaning that working scientists would need to spend less time judging the proposals of their colleagues. "I'm delighted by the idea that established scientists will have a reasonable likelihood of getting six years between writing proposals," says John Kice, dean of the faculty of natural sciences, mathematics, and engineering at the University of Denver and an organic chemist with 30 years of NSF support. "And I don't think that it will be abused by a program officer; I think that they can be counted on to pick the right people." William Ausich, a paleontologist at Ohio State University with a history of NSF support, says the task force proposals will help make him a better scientist. "In order to do science, I need two things: time and money," he says. "And once I've gotten funding, my productivity is directly tied to the amount of time that I spend at the bench. So anything that NSF can do to cut down on the bureaucracy needed to administer the grants system will help me." To bring about such changes, the report suggests, NSF must change the merit review system from a process by which scientists pitch ideas to a group of their peers to one that relies more heavily on an applicant's past research. The result, according to those who have studied the report, will be greater success for those who are most deserving, fewer review panels, and less time spent by staff to manage the review process. "Compared to Europe, Canada, and Australia, our system is not good," says Wyatt. "We don't put enough weight on past accomplishments and the track record of the scientist. The current system favors those who invest the time to write a slick proposal, and penalizes those who spend their time in the lab and with their students." Scientists say that the new approach would also improve the review process. "It's easier to judge what's been done than to weigh what somebody plans to do," says Perdew. "After all, it's possible to write a good proposal for something that's not possible." Such a system is not perfect, however. Scientists who support the report's recommendations acknowledge that the revisions could lead to a more conservative system that places too much emphasis on the past and on the reputations of those applying for money. But, they say, such a risk is worth taking if the reward is funding the best science. "I know that relying on one's track record is elitist," says Michael Freeling, a plant geneticist at the University of California, Berkeley, and director of a $2 million plant developmental biology center that is funded in part by NSF. "And I know that it would be nice to have everybody start out equally. But relying on track record makes success more difficult for someone who must overcome a lousy record, and that's how it should be. "Besides, track record is a lot of things--a mixture of intuition, enthusiasm, and resistance to burnout, among others. Maybe it's a little conservative, but it also represents a much-needed emphasis on quality." While all these scientists like the idea of longer grants and fewer reviews, none of them wants to jeopardize the primary role of peer review in making funding decisions. "It's essential that we defend the peer review system," says Freeling. "Whatever problems there are with peer review, all the alternatives are much worse. These recommendations are simply saying that we should reduce the paperwork and the number of times that peers engage in reviewing scientific proposals. Just because something is good, it doesn't follow that more is better." NSF's Harris agrees, and downplays the magnitude of the proposed changes: "We think there's a need to simplify, for the benefit of the reviewer as well as the foundation. And one way to do that is to look at a scientist's body of published work, his institution, his students, and everything that will allow him to carry out the proposal. We're just trying to apply common-sense principles to the process." One of those principles, says Harris, is strict adherence to a 10-page limit for proposals. A related change would be to delay submission of a detailed budget until after the program officer has recommended that the proposal be funded. A third principle mentioned in the report would be routine processing of grant continuations once the program officer has evidence of satisfactory progress. The change would eliminate redundant procedures at NSF and unnecessary accounting requirements at the grantees' institutions. While approving of the importance being placed on past achievements in making grant decisions, every scientist who commented on the report believes that NSF also must set aside enough money to support the next generation of scientists. The task force report addresses this issue by proposing an alternative type of award for young investigators. Called starter research grants, these awards would go to freshly minted Ph.D.'s or those who have never received federal support. The report also suggests that existing programs for these less-experienced scientists, such as the Presidential Young Investigator (PYI) award, be folded into such a category and that similar criteria be used to select the winners of each grant.

Panel Weighs Overhaul Of NSF's Grant System

Interested in reading more?

Become a Member of