Is the NIH budget saturated?

Editor's Note: This piece from our November issue ran online ahead of print to spark discussion. What's your hypothesis for why the doubling of the NIH budget doesn't seem to have doubled productivity? Please post a comment by clicking here. Complaints by scientists about the flat NIH budget have grown louder in recent years. For scientists to effectively lobby Congress for increased funding, however, we need to show that increased funding increases productivity. Given this need, I decided to examine scientific productivity as a function of the budget. Since the NIH budget doubled from $15 billion to $26.4 billion from 1999 to 2003 (Figure 1), I reasoned that there should have been a corresponding jump in productivity. The test was the simplest measure of productivity: the number of publications. (continued below)

Here's what I found: The number of biomedical publications from US labs did in fact increase from 1999-2004. However, so did the number of publications from labs outside the US where the research budget did not double. Figure 2 shows a parametric plot of the number of scientific papers indexed in the ISI Web of Science database by the keyword "biology" that were published each year from US labs and non-US labs. There is no upward jump that you would expect to see with a sudden increase in productivity. (continued below)

Perhaps the lack of correlation between funding and the publication rate was an artifact of choosing the wrong keyword. So I repeated the search, this time using the keyword "neuron" instead of "biology." Again, as shown in Figure 3, increasing funding had no effect on the publication rate. The key words "RNA," "DNA," and "disease" produced similar results.(continued below)

The lack of correlation between the publication rate and the budget is further supported by the fact that after 2003, when the budget flattened, the publication rate did not decrease. The analysis is not insensitive to changing publication patterns; during the same period of time there was an enormous increase in publications originating from China. For example, using the keyword "neuron," from 1996 to 1999 publications from US labs increased 2.9% annually, while those from China increased 42.7%. From 1999 to 2005 publications from US labs increased 3.6% annually, while those from China increased 38.9%. The trend to publish more articles seems to reflect the activity of journal publishers rather than scientists. The number of indexed journals increased from 4500 to 6500 between 1994 and 2006.Still, I reasoned, perhaps publication rate is not a good index of productivity, and the papers are twice as good as they were in 1996. That is hard to judge, but analysis of the literature suggest otherwise (Ref 1, Ref 2).An NIH institute director suggested that the absence of correlation may reflect a significant latent period between funding and stimulated productivity. Given this data, however, that latency would have to be greater than ten years, and we are talking about a net excess influx of more than $50 billion since 1996. Presumably, the NIH does not expect a ten-year latency between the granting of funds and the delivery of new data, since they award grants for only 3-5 years.The best data on US research quality should be available from the NIH, where review panels judge the scientific quality of grant applications more than 50,000 times a year. The panel's grades of research quality are averaged to a precision of at least three significant figures; a person may be denied funding if their grant quality is judged to be 0.1% below the current funding levels. The ability of a review panel to decide whether a person's ideas are not worth a grant signifies that they are a good judge of quality. I encourage the NIH to analyze the available data and see if current research is twice as good as older research.The lack of correlation between funding and publication rate does not mean the NIH is funding bad science. It isn't. It is funding some great science, but it is not spending its budget efficiently. According to the director of NIH, doubling of the NIH budget led to a paradoxical decrease in the grant funding rate (Ref. 3). What happened to all the extra money that flowed into the NIH? Was it used for unproductive clinical trials (Ref. 4, Ref. 5)? Was it absorbed by inflation? Wherever the funds went, they left no clear scientific record.Frederick Sachs is a professor of physiology and biophysics at the University of Buffalo.References: 1. E.N. Brown and S. Ramaswamy, "Quality of protein crystal structures," Acta Crystallogr. D. Biol. Crystallogr, 63:941-950, 2007. http://www.the-scientist.com/pubmed/177045622. J.P. Ioannidis, "Why most published research findings are false," PLoS Medicine, 2:696-701, 2005. http://www.the-scientist.com/pubmed/160607223. E.A. Zerhouni, "Research funding - NIH in the post-doubling era: Realities and strategies," Science, 314:1088-1090, 2006. http://www.the-scientist.com/pubmed/171105574. A.R. Marks, "Rescuing the NIH before it is too late," J Clin Invest, 116:844, 2006. http://www.the-scientist.com/pubmed/165859505. A.R. Marks, "Rescuing the NIH: the response," J Clin Invest, 116:1460-1461, 2006. http://www.the-scientist.com/pubmed/16585950