Funding Research With NIH Grants: A Losing Battle In A Flawed System

Flawed System Author: Arthur E. Sowers Scientists who do research in academic environments must pay for this research from grant money obtained through a highly competitive process that starts with the writing of a grant proposal. Although this process can end successfully if a grant proposal is funded, it more often than not ends with a proposal being rejected. It is widely known that over the last 15 years the average chance that a proposal will be funded has declined from about 50 percent to about 10 percent or less today. Hence, to get one proposal funded, the average scientist must be constantly writing to produce and submit about 10 proposals. The two-fold question that arises from this situation is: How did this process become so difficult, and can anything be done about it? In the search for answers, one area to concentrate on is how this process works in the case of National Institutes of Health grants because NIH is the granting agency with the largest amount of money for funding the biomedical sciences. Also, unlike grants from other funding agencies, NIH grants can be much larger and active for a much longer period. Grant proposals are read and judged by people on NIH study sections. Being invited to be on a study section gives one the opportunity to see the process that arrives at a priority score for a proposal -- the most important factor that controls the funding decision -- and see examples of good and bad proposals. Seeing how other study section members respond to this can also be somewhat helpful in preparing one's own proposal to "play into" this process. Over the last decade I have been on many study sections. What actually happens during this grant-evaluation process seems to have two components. First, it is a fishing expedition to find identifiable defects, discover directions that are or can be perceived to be unproductive, or notice in the proposal any shortcomings in details or an absence of preliminary data. The more of these, the worse the priority score. Also, if a proposal does not predict convincingly all outcomes, and have plans for dealing with all of them, then it will get a lower priority score. Hence, a proposal needs to be essentially perfect. Second, the grant-evaluation process determines if some kind of admiration develops for the proposal. Admiration can develop if the reviewers and readers can find in the proposal what I might call "high recognition" factors. The presence of molecular biology; a major disease entity (for example, HIV or cancer); or some other idea, paradigm, or activity (such as the Human Genome Project) that is easy to recognize as a hot topic, a meritorious topic, or a subject area receiving attention by many investigators will help the proposal get a better priority score. Of course, the writing must be masterful, and preliminary data must be present. I cannot help feeling that sometimes reviewers decide that they "like" or "dislike," subjectively, a proposal and then go about wording their critiques after the fact, to justify a fundable or nonfundable priority score. Proposals will not compete as well in this evaluation process if it is not possible to know or predict all outcomes before the work is carried out. If little is known to begin with about some area of science, then it will be almost impossible for an author to fill up in great detail all 25 pages of a proposal text. The NIH process for determining which proposals to fund is good because it is fairly open and because a grant-proposal author gets a written critique that gives some explanation for the funding decision on his or her proposal. It is also good because it involves peer review by readers with some of the best credentials and most relevant qualifications to judge a research proposal, and it can lead to the formation of a consensus about any given proposal. Lastly, really poor proposals are easy to identify using this system. The NIH grant-evaluation process also has a number of weaknesses. NIH publishes a book, "NIH Advisory Committees," listing the study section topics, missions, and names of the members of each. The topics and member names, but not the mission, for each study section can also be found on the Internet (gopher to GOPHER.NIH.GOV; at first menu prompt, select "GRANTS," at next menu prompt, select "DIV RES GRANTS"). The mission statements show that the expertise of the various study sections is compartmentalized into sub- areas that are concentrated only on certain themes or programs. If a proposal deals with a topic that can be perceived to be on the edges or outside of these areas, the priority score will be worse. This can very effectively work against new ideas and interdisciplinary proposals. A grant proposal must contain preliminary data from work already done, a hypothesis or set of hypotheses, and a description of work to be done in the future. Usually in the "goals and specific aims" section, the author can point out that the work in the proposal is directed at the next logical and reasonable, if not obvious, extension of the preliminary studies. But what can the reviewers do? They can concentrate on complaining that there are not enough preliminary data or that the data are not of the right kind. This can come about despite the fact that the author probably knows more about the topic than the reviewers. One senior faculty member (not at my institution) told me how he lost his NIH grant, despite productivity in publications. He learned after substantial discussion with the executive secretary of that study section that the only methodological approach that would be found acceptable would have to involve molecular biology. After he learned molecular biology and used this approach in his proposals, he got funded. Where did this leave him and science? Now, instead of learning more about the function of a gene product, which is still unknown, or its interactions with other cellular elements, he is extensively mapping and characterizing a set of genes. I cannot help seeing this situation as an example of intellectual micromanagement. Another common story from NIH proposal authors is that a proposal rejected by one study section gets funded when it is submitted to another study section. This challenges the validity of the process by which the "best" proposals are identified and suggests that each study section has its own peculiar mindset. If a proposal does not fit into that mindset, it is dead on arrival. Despite efforts to prevent real or apparent conflicts of interest, the fact that some members of a study section in some way "know" the particular proposal author -- either in a positive or a negative light -- can also bias the priority score. This means that the real merit of a proposal may not be the central criterion that determines if it gets funded. While the burden of proof is correctly on the author of a proposal, it is another serious shortcoming that reviewers do not have to demonstrate a significant grasp of the material in a proposal or its significance. In my own experience, and from stories from others, the critique may say that "X" was not addressed, yet "X" actually was discussed on page "Z" of a proposal. In one of my proposals, I used one less page of space than was permitted and a significant amount of space in the critique was devoted to my failing to use all of the space permitted. While an author may spend three to six months writing a proposal, there is no way to determine how much effort a reviewer puts into reading it. I know that the discussions of a proposal at a study-section meeting rarely last more than about 10 to 20 minutes. During the rare times that discussion between study section members became as lengthy as 20 to 30 minutes, I had the feeling that it was because of ideological differences among the members, or their differing perception of a given proposal, rather than an attempt to better understand the proposal. In comparison, I have been in journal clubs that went further into the merits and demerits and details of a published paper than I have seen in study sections. It is unfortunate and unfair that there is no appeal process for a poorly carried out review of a proposal and no clear process to ensure that a given reviewer does a good job of reviewing. A proposal author has the privilege of revising and resubmitting a proposal to address a critique. Sometimes this leads to a fundable priority score. However, the whole review cycle takes the better part of a year, and the proposal may go to different or new members of the study section, who can bring up entirely new criticisms without giving credit for addressing the old criticisms. While most proposal authors have learned to live with the aforementioned discouraging weaknesses, the current system has another, very destructive component: It is a fact that if a grant fails to get competitively renewed, the funding is completely cut off. This can sometimes happen just months or weeks after the notification. This breaks up the research group (all people supported by that grant usually lose their jobs) and sometimes damages irreversibly the careers of the key people involved. It is a waste of all the efforts put in by the involved people, and an unjust fate for hard work and a lifetime commitment. Scientists have adapted to the decrease in award rates by writing and submitting ever more proposals. This has understandably caused study sections to respond by giving the best priority scores only to those proposals with the greatest detail, rationality, and utter perfection in their ability to predict results and outcomes. This is causing proposal authors to spend less time on real science and more time on the preparation of proposals for the sake of their "presentability." Proposals containing high amounts of detail and experiments with highly predictable outcomes can be more easily constructed from research subject areas in which there is already a high population of scientists or much is already known. This causes the proposal -- evaluation process to have the effect of selecting "for" these areas and "against" subject areas with low populations of scientists and low amounts of knowledge. This is clearly destructive to the diversity of science. It contributes to the generation of high activity in popular areas at the expense of orphan subject areas, and almost ensures that many important discoveries will not be made because they cannot be predicted. An unreasonably large element of chance is now present in the competition for funding. This adds to much stress in careers that require grant funding. If reform does not take place, then ever larger proportions of scientists will become at risk of seeing their careers end, suddenly and permanently, through no fault of their own. It is ironic that such a rational proposal-evaluation process may lead to the irrational possibility that a fraction of all good scientists must continually deal with a random and increasing probability of losing their careers. This is contrary to the ethic that says hard work will lead to subsequent reward. Arthur E. Sowers is a research professor in the department of pathology at the University of Maryland at Baltimore School of Medicine. His E-mail address is arthures@access.digex.net. (The Scientist, Vol:9, #20, pg.12 , October 16, 1995) (Copyright © The Scientist, Inc.)

Funding Research With NIH Grants: A Losing Battle In A Flawed System

Interested in reading more?

Become a Member of