Review of Grant Applications

One of the few things that most Americans still believe is that we have the best biomedical research system in the world and that the National Institutes of Health is its indispensable heart. NIH has a dual function: to support basic research in relevant sciences and to support research designed to apply scientific data equitably to improve the health of all United States residents. These two functions are fundamentally different. Basic biomedical research can be, and always should be, as rigorous, specific, and precise in its observations and reports as any of the hard sciences (classical physics, chemistry, and biology). The education of scientists who carry out such research is long and arduous. Only people who are able and willing to submit to the discipline demanded by the specialty they choose to study can hope to make some meaningful contributions by their research. The second function, the application of the findings of basic medical scientists to the health care and disease prevention of all U.S. citizens, demands a more egalitarian approach. Sociological, demographic, and even political factors must be considered. As a result, such studies often involve many investigators with diverse backgrounds, and the results of these studies are generally not as clear-cut as those of the basic scientists. Moreover, most of these results cannot be applied universally, as basic science results can be. It is therefore not surprising that there has been tension between these functions from the day that NIH was established and that applied science, which requires the collaboration of many diverse people with their support staff and specialized instruments, has generally received the lion's share of research funds distributed by NIH. When NIH was established, shortly after World War II, many physicians were trained not only in medicine but also in basic sciences, and some even did research in both areas. For this reason, the funding mechanisms established did not distinguish between these two types of research. Virtually all extramural research is funded the same way. Each NIH institute has several advisory councils. These councils discuss broad issues and long-term policy, and therefore rely on another organization, the Division of Research Grants (DRG), to decide whether an applicant should be funded and how much money should be awarded. Applications for research funds from NIH first go to DRG. DRG has created a number of study sections to review grant applications. The study sections never distinguish between the two types of applications they receive; that is, basic and applied science. Rather, each study section (SS) covers a broad discipline, such as biochemistry or pharmacology or cardiovascular physiology. Each SS is headed by a scientific review administrator (SRA), who is appointed by DRG to this position. The SRA must find 10 to 20 established investigators for the SS who are willing to review proposals for a period of four years. The reviewers meet in Washington, D.C., three or four times each year and rank the grant applications received and decide which ones merit funding. The deliberations of SSs depend greatly on an in-depth review of each application by qualified scientific reviewers (usually two) who are members of the SS. They receive the proposals a month or so prior to the SS meeting. Initially, most members of SS were academics engaged in teaching or clinical work as well as research, and they knew most of the other investigators in their fields. They were replaced when their terms expired by people whom they recommended, who were just as qualified. But then people were added to each SS primarily to represent every race, gender, and geographic region of the country. This diluted the scientific expertise on the SS. Then, as now, when the SSs meet, the peer reviewers present their evaluation of each proposal and their recommendations; then all members of the SS vote on a score, and the final score on which funding depends is the result of this vote. Since most members of the SS have not studied the proposal in detail, they are usually influenced in their vote by the reviewers' recommendation. However, each SS member has one vote, and since they have all received the entire application--that is, not only the actual proposal but also details of the applicant's personal history, the proposed budget, and the extent of support that the applicant is receiving from his institution--the vote of some members, and therefore the final score, will be influenced by personal factors rather than the scientific merit of a proposal. In spite of this problem, the system worked quite well because of two built-in safeguards. One is that the review process has two stages. The final decision as to whether a proposal will be funded or rejected is made by the appropriate council. Councils contain scientific experts as well as other citizens, and applicants whose proposals were rejected by a study section were able to appeal to council for a reevaluation of their proposals. The second safeguard was that applicants whose proposal was not recommended for funding after the initial review were encouraged to respond to the SS critique and resubmit their proposal. These resubmitted proposals were then often funded. When in 1957 the Russians launched Sputnik, we suddenly realized that the U.S. was not the only technologically advanced society in the world and that research in basic science is essential before technological advances occur. We decided to fund researchers engaged in basic research adequately in order to maintain our leadership. Congress began appropriating large sums of money for this work, and for the next 12 years, most basic scientists, able and willing to engage in biomedical research, were able to do so. Unfortunately, however, as more and more money became available to NIH, more specialty groups--for example, institutes and research centers--were created, so that by 1970, when about 80 percent of biomedical research in this country depended on NIH funds, there was no longer enough money available to fund all meritorious applicants for investigator-initiated proposals (R01s). Scientists received letters from NIH informing them that their proposals for research had been approved by the reviewers but would not be granted because of "insufficient funds." There was an outcry from the scientific community, and NIH responded by announcing a series of meetings throughout the country at which scientists could present their problems with the funding mechanism while the administrators investigated how the review process and funding system could be improved. Unfortunately, the "improvements," developed by NIH and announced with great fanfare in 1975, were merely cosmetic. The number of trained investigators continued to grow, and the degree of specialization increased so that it became harder to identify appropriate reviewers for all innovative proposals. The number of institutes and specialized centers also continued to increase, and these received an ever-increasing fraction of the total NIH budget for extramural research at the expense of R01s. Consequently, the number of applicants requesting council review of their rejected proposals increased enormously, and the scientific experts on many councils rebelled. Applicants were actively discouraged from appealing a negative evaluation by a SS and, by now, whether a proposal is funded or not depends almost entirely on the score between 1 and 5 (1 is best, 5 is worst). The fraction of R01s funded continued to decline, so that by 1994 practically none with scores higher than 2.2 were getting funded. By now, NIH is fulfilling only one of its functions, that of funding applied research. In response to the crisis in funding investigator- initiated proposals, NIH administrators are introducing a system of triage (L. Reif-Lehrer, The Scientist, Nov. 28, 1994, page 1). Their aim is to decrease the number of applications that SSs review to one-half of those submitted and thus save the time of SS members. Under the new system, SS chairmen are asked to have their peer reviewers identify proposals that are "non-competitive under present conditions" and not submit those proposals to the entire SS at all. As Jane Koretz, a professor of biology and biophysics at Rensselaer Polytechnic Institute, pointed out in the aforementioned article, this would eliminate an assessment of scientific merit as the major criterion for funding because no appeal of the SS reviewers' decision will be possible. This is a particularly serious problem for basic science proposals because they are more likely to contain new ideas whose merits may not be apparent to a primary reviewer. In the same article, Lawrence Grossman, Distinguished Service Professor of Biochemistry at Johns Hopkins School of Public Health, who has worked as a member and chairman of a study section, expressed alarm because funding decisions are based more and more on assessment of anticipated results rather than on proven productivity. Anticipated results may be acceptable criteria for applied research but not for basic science. However, on the forms now used for applications, the instructions for the abstract start as follows: "State the application's broad long-term objectives and specific aims, making reference to the health relatedness of the project." This is meaningless for basic science proposals because it ignores the fact that almost all major advances in the basic sciences are the result of serendipity. Scientists are motivated primarily by curiosity and the desire to find out a little more about how the world around them works. Their training is designed to teach them to ask appropriate questions of the limited models they study. These questions that can be answered in a reasonable length of time usually lead to more questions. Any comments about long-term aims and health relatedness of the proposed research are irrelevant and should not affect peer reviews of basic science proposals. I submit that the problem is serious and that the review process must be reformed. The system of triage does nothing to address the major problem in the biomedical sciences, which is that young people are no longer being attracted to a career in basic science. It takes some 15 years of highly specialized education to produce a scientist, and we cannot afford to turn off the trained young people who are willing to commit their lives to the profession. Yet now young people consider it virtually impossible to obtain financial support for a career in basic science and, realizing that their proposals may not even receive full review, will be the last straw for many of them, who will opt out of the profession. To stop the hemorrhage of scientific talent and prevent the U.S. from slipping to Third World status in medical research, I suggest that the review procedures for basic research be separated from those for applied research. Each NIH council should designate a specific budget for basic research and publish the amount of money available for this. A higher fraction must be allocated by councils whose mission is the pursuit of a basic science than by those councils whose mission is the management of a disease or the collection of measurements that can be used to establish national norms. To maximize the number of people who can be supported, new applicants should be encouraged to request small amounts of money. A cap of $100,000 per annum might be considered. In order to ensure each basic science proposal of an in-depth review by true peers of the applicant, I suggest a system that worked well at the National Science Foundation. Three reviewers should be selected by NIH personnel: two from among experts familiar to them and one from a list of experts submitted by the applicant. The reviewers would examine the proposals in depth at their own institutions and assign a score (or state that the proposal needs more work). The reviewers would not need to come to Washington to attend study section meetings, and any costs to NIH in coordinating the work of the reviewers would be more than offset by the savings in travel costs for the scientists. The fact that peer review would not require travel to Washington would also make a much larger pool of reviewers available than that tapped now by NIH. A conference phone call before the final decision is made may be necessary, and if the three reviewers disagree significantly the NIH coordinator might mediate at such a conference call. Should the reviewers' judgment be that the proposal is so weak that it does not even merit a score this, together with the reason for the rejection, should be communicated to the applicant by the NIH coordinator together with suggestions about how the proposal could be improved for resubmission. It may take some time to make a major structural change such as the one I have called for, and we have a crisis now. In the November 28 article in The Scientist, Susan Leeman, a professor of pharmacology at Boston University School of Medicine, suggested that scientists be invited to write brief "intention" proposals, that those be subjected to triage, and only those that stand a chance of being funded be asked to submit a full application. Such a procedure would at least show extramural scientists that NIH values their time and respects them. Edith Rosenberg is an associate professor of physiology and biophysics at Howard University College of Medicine. (The Scientist, Vol:9, #16, pg.10 , August 21, 1995) (Copyright © The Scientist, Inc.)

Review of Grant Applications

Interested in reading more?

Become a Member of