Advertisement
Survey Series
Institutions
For Advertisers
Volume 23 |
Issue 11 |
Page 25
Bookmark & Share
Close Window
NIH Continues to Support the Best Science through R01sA response to accusations that the agency is biased against senior scientists
The September issue of The Scientist included an opinion piece called “NIH R01s: No Longer the Best Science” by Dr. Les Costello.1 In that article, Dr. Costello expressed concerns about NIH policies2 related to new investigators, suggesting that they offer an unfair advantage for new applicants over established investigators. Because of these concerns, we felt the need to provide some context around issues associated with new investigators and why we believe the flow of new talent is essential for the maintenance of a productive scientific enterprise. When Dr. Costello received his first traditional NIH research grant (R01) in 1963, success rates were near 58 percent3 and 35 percent of the competing R01s went to first-time recipients.4 Data published by the NIH and available on the NIH New Investigator Web site5 shows that since that time, the proportion of awards going to previously unfunded investigators has declined and the average age at entry has increased substantially.6 In 1977, the average age of new investigators was nearly 37, success rates had decreased to 28 percent, and the proportion of R01s going to new investigators had decreased to 33 percent. These data indicate that getting that first independent grant award was more difficult and required longer periods of training. At that time, the NIH was sufficiently concerned that it designed the New Investigator Research Award (NIRA) to facilitate the transition to independence. The NIRA was replaced by the First Independent Research Support and Transition Award and a series of other interventions described on the NIH New Investigator Web site.5 In spite of these interventions, by 2006, less than 24 percent of the recipients of competing R01s were new investigators, success rates were below 21 percent, and the average age at first award of an R01 had increased to more than 42. In response to increasing concerns, the NIH attempted to stabilize the flow of new investigators close to the average rates of entry observed between 2001 and 2005. Download Flash player to listen to this audio clip.
Deputy Editor Alison McCook moderates a debate about policies at the National Institutes of Health designed to increase success rates on grant applications submitted by young investigators. Discussing the issue are columnist and NIH-funded biologist Steven Wiley and Richard Gallagher, editor of a recent opinion from Les Costello, who argued that current policies discriminate against senior scientists.
In 2009, the NIH introduced a variant of this policy to accommodate better the differences in application assignments to the NIH Institutes and Centers (ICs). Under this new policy, ICs were encouraged to maintain approximately equal success rates on applications describing new research projects (type 1) from either new or established investigators. This approach was designed to level the playing field for new and experienced investigators on type 1 applications. Success rates on competing continuation applications (type 2) were not affected and continued to provide stability for successful projects led by established investigators. The new policy included two additional features. The first involved clustering applications from new investigators during peer review to facilitate the identification of the most meritorious applications from researchers at all stages in their careers. The second involved identifying Early Stage Investigators (ESIs) or those New Investigators within ten years of their terminal research degree or the completion of their medical residency. The ESI elements of the policy provide incentives to encourage the reduction of the excessively long periods of training required to enter faculty ranks and apply for NIH research support. This policy is age-independent and is based on the duration of time since completing medical residency or the terminal research degree. In addition, breaks in training associated with family care, illness, or extended periods of clinical training can justify an extension of the ESI period.7 We find no evidence that scientists who enter the pool of NIH-funded researchers today are less accomplished than those who took similar positions in 1963. Reduced success rates and the observed elongation of the training period suggest that new entrants may be better prepared to contribute than in the past. The NIH will continue to monitor the replenishment of the nation’s scientific leadership and we welcome input from the research community that we serve. Walter T. Schaffer, Ph.D., is Senior Advisor for Extramural Research at the NIH. Sally J. Rockey, Ph.D., is Deputy Director for Extramural Research at the NIH. Editor's note: When originally posted, this story contained the incorrect subhed. This has been fixed. Advertisement
Rate this article
Return to Top comment: Missing information at NIH REPORTER Database by Rafaela Canete-Soler [Comment posted 2009-11-16 12:05:28] Dear NIH, This is to report that there is missing information at your Reporter Database. I just realized that there are several contributions that are not registered in the history of the following grants: 5R01NS015722-20 through 26: Neurofilament expression and motorneuron disease 1R01NS037552-A1through 8: Mechanisms of motorneuron degeneration in transgenic mice. The missing publications are: 1. Canete-Soler et al J Biol Chem 273:12650-4 (1998) 2. Canete-Soler et al. J Biol Chem 273:12655-61 (1998) 3. Nie et al J Neurosci 22:7662-70 (2002) 4. Ge et al J Biol Chem 277:42701-5 (2002). In addition, there are no records on Patent: ?Factors Relating to Motor Neuron Degeneration? in neither grant. This patent was filed in 1999. You can retrieve the information at the University of Pennsylvania. Penn Reference ID: L1915. Even though this information can be easily found at Google, I believe it is appropriate to report it as research performed with public funds. Thank you. Return to Top comment: World News Reporting by anonymous poster [Comment posted 2009-11-11 06:29:06] Good evening ladies and gentlemen, This is George Moustache reporting for World News, sitting tonight for Larry Slender. You are in the situation, Sorry, you are in the Revolution Room. Once more, we have Paris today at the center of a stunning call for action and order. Thousands and thousands of people have concentrated at Champs Elysees Avenue following the melancholic whining of two individuals. We have Heather Hart at the phone right there. George: Hi Heather, can you hear me?, Heather, are you there ? Heather: Hi George. Yes. I hear you but the crowd here is too noisy. What I can tell you so far is that people keep coming at the cry of two individuals. I was told that they have a New Orleans?s kind of accent. It appears as if they have been identified as: Junior Investigate, 45 yrs-old, big complexion and Early Seeker Insisting, 52 years old, 6 feet tall and extremely slender. I just talked to a French reporter and it seems that Carla Bruni is very distressed trying to get uphold of President Sarkosy, who apparently is in an undisclosed location. George: Heather, Heather, what are those individuals doing ?. Heather: They are smoking and sort of whining-singing. Let?s listen. The sound reminds me of ?April in Paris?. Listen, listen, Best R01s, Great R21s, Awesome P30s plus P50ssssssss, Beautiful RCs, Amazing supplements, Consolidating Resource I never had an R01, Never touched R21s I wonder what that might mean Wish I got one soon, I will be fired?. I had an R01, Have to renew it soon, I wonder what might happen next I wish it happened soon I will be fired Whom can I run to What is WIH doing to my science?? Heather: George, did you hear them ??. Hey, hey, hey, Carla Bruni is at the Presidential Balcony. She looks distressed?.. She went back inside ?. She is coming out again with the guitar.. Let?s listen??. Tu ne veux pas travailler, Tu ne veux pas produir, Tu ne veux pas publiquer, Incompetent ! Heather: George, my French is not that great but hear next to me there is a student from Wisconsin, who is fluent. Hey Paul, could you please tell us what is she singing. Paul: Well, she is distressed at the two individuals because they are not good enough. You don?t want to work You don?t want to produce You don?t want to publish, You?re incompetent George: Listen Heather. I will turn off the connection now. The Boston Globe is just reporting that there has been an individual, in the vicinity of Cambridge, with a prophetical look, soft-spoken, very elegant. He appears to have captivated Harvard student?s souls, their families and the entire neigborhood. This is a transcript of what he is reading: ? Blessed are those who believe in WYH because the kingdom of wealth will be theirs? (In: Having a vision, Benjamin Dean, Pennsylvania Medicine). Ladies and gentlemen, our time is over for tonight. We will keep you updated as more news unfold. Good night and good luck Return to Top comment: sour by anonymous poster [Comment posted 2009-11-10 15:13:08] I can only say that I'm extremely disappointed with how the system has been running. Indeed, years ago (70's) the funding level was so high that if you just submitted two grants, you were likely to get one.
I had a great project, at Berkeley, and submitted my grant 3 times. I'm a first time R01 investigator (or, I was). I was rejected all three times. I still don't think the reviewers even understood what I was proposing! Pathetic. And to think that stale projects are continuing to obtain funding is particularly troublesome. And now, since I'm out of the academic realm, and I see how much money is out there - to think that I was busting my behind for a measly $250K/yr, and a very average salary? WHAT was I thinking? I've been in labs for quite a while (as I'm sure most of you have too). We all know what goes on. Some good experiments, some not so good ones - but lots of times the same old stuff goes on and on and on and on. It's too bad... NIH has lost another young scientist. Sure, I will continue to contribute in my own (private start-up) kind of way... but I really would have loved to continue on in academia. I guess my point is that if you don't give young investigators a chance - especially those that persist and really try hard (and therefore showing a real commitment to something)... I think the system will continue to become even more top-heavy, and bright, young talent will be lost in the university of the future. (of course this is a generalization, but... ) I encourage giving younger people grants - and give them the benefit of the doubt sometimes too! It's too late for me, but what about the next person? Return to Top comment: Fair or unfair? by anonymous poster [Comment posted 2009-11-10 13:47:48] Many of us jr faculty appreciate NIH's new direction by supporting new investigators. We also understand Dr. Costello's concerns for many established (usually, but not always, senior) investigators. I see two sides of the coin, but the system is not what we had in 1963. We all wish we could have the success rate that Dr. Costello and his generation enjoyed then. But we are in a stretched NIH system now. Giving young investigators somewhat favored treatment will benefit the sciences in the long run in this country. I respect what Dr. Costello and his generation did for science, but I hope he will be willing to take it easy and enjoy supporting jr faculty around him, even if this means that he may not renew his 30th (I guessed this number) RO1! Return to Top comment: The first competitive renewal is the most difficult task by anonymous poster [Comment posted 2009-11-10 11:53:51] The historical success rate for the first time competitive renewal is around 20-25%. It is a monumental test for the investigator. Once you succeed, the success rate for subsequent competitive renewal is much higher.
The chance of a new investigator R01 being renewed right now is extremely low right now. The elimination of A2 will end up hurting the people who are trying to renew their first grant more. On the other hand, the success rate of a new application is always higher than the first competitive renewal. One of my friend build her career by always applying for new grants :-). Return to Top comment: Re: How many new investigators renew their grants? by Rafaela Canete-Soler [Comment posted 2009-11-10 06:48:04] How many new investigators renew their grants? We might have an answer to this question when NIH fills the Reporter with all the numbers, in dollar amount, given to any PI from 1976-present. Maybe then we can better evaluate correlations between numbers of dollars and scientific ideas advancing any field of biomedical research. We will be able to evaluate, in any given area of research, who were the new investigators that managed to productively remain in the system with meaningful contributions. We will also be able to assess motivations for "movement of researchers" from one area to another area of research etc. It will be illuminating for the educated public, as well as, for the scientific community, NIH administrators and policy makers. Based then on historical factual data, the scientific community, the public, NIH administrators and our political representatives might have an educated input on how, why and for what purposes, funding policies ought to be reevaluated, implemented and subjected to periodical revisions. What would be the goals?. Optimization and rationalization of resources to meet urgent scientific and public health needs and a provision for immediate and long-term scientific future in the USA. As somebody has pointed out here at The Scientist, the USA as a country, has lost its long-standing leadership. Are we, as individual scientists or group of researchers or as a country, going to get depressed and pessimistic because we lost our number 1 place in the ranking ?. NO, absolutely not. We know the rules of competition. We have been practicing them for all our history. If we have forgotten them, let?s ask Martina Navratilova, the William Sisters, Steffi Graff, Monica Sellers, Andre Agassi, Roger Federer, Rafael Nadal, and a long list of true champions, more or equally famous or even not remembered any longer. They went and are going through periods of winning, losing, falling, getting up and so on. Has this anything to do with failure ?. NO, in my view. It's all about DYNAMISM in the DIVERSITY OF TALENTS AND OPPORTUNITIES. That is our history. Return to Top comment: How many new investigators renew their grants? by anonymous poster [Comment posted 2009-11-09 17:35:02] At a recent seminar I heard a statistic that the average number of total years an investigator is funded from the NIH is about 7. This would indicate that most new investigators do not get their 1st grant renewed. Is this indeed the case? if most 1st RO1s are not renewed then money is indeed wasted. How then do we encourage new investigators? Return to Top comment: correction to typo in earlier post by anonymous poster [Comment posted 2009-11-09 14:18:46] I must correct a typographical error in my earlier post. The following sentence makes no sense because the last two words should have been "percentile scores," not "priority scores": "First of all, the RO1's are not ranked against each other by "priority score" but rather they are ranked against each other by 'priority scores.'" Please read this to mean "rather they are ranked against each other by PERCENTILE SCORES" which is what I intended to write. Thank you. Return to Top comment: Significance of funding rates not clear, but system is failing by john salerno [Comment posted 2009-11-09 12:11:46] I am sympathetic to offering some help to new investigators, if only as a counter to the network of personal connections which is the 800 pound gorilla in peer review. The percentage of R01s going to new investigators has fallen from 33% to 24%; this is cited as a cause for concern, but would be more meaningful if data were provided about the fraction of the research establishment made up of new investigators. There are many more senior investigators now than in 1963, which might in itself cause the fraction of new investigator grants to fall. I am much more worried about the increased average age of the first R01, which I attribute to the reduction to absurdity of our flat-earth academic model, in which each PI trains numerous potential successors, all of whom are expected to work in temporary jobs, acting as high-end migrant workers. This can't go on forever. Return to Top comment: NHBLI paylines for A1 and A2 revisions by Louis Curacao [Comment posted 2009-11-07 08:16:01] The paylines are for NHBLI revised grants are now posted with A2s being at an almost impossibly low 7.0.
This policy is not just capricious it is cruel. Return to Top comment: capricious process indeed by anonymous poster [Comment posted 2009-11-04 15:47:52] Louis Curacao is indeed on-target with his observation that the NIH "peer review process" is capricious.
Yet, he apparently clings to what I view as an absurd notion, that the "priority score" is a "clear indication" of the rank-order quality of the science in the applications. First of all, the RO1's are not ranked against each other by "priority score" but rather they are ranked against each other by "priority scores." These are not the same metrics! Priority scores are voted by study section members who examine relatively small groups of applications(compared to the total pool of RO1s under review in that round of review) at a time; these small groups of applications are then ranked in priority score order and assigned the corresponding "percentile" rankings. It is this "percentile" score (i.e., the ranking within the group) that becomes the magic number by which the whole universe of RO1 applications are then ranked against each other. It is thus possible that the 15th percentile in one study section is achieved by an application that received a priority score of 150, whereas the same 15th percentile in a different study section might be achieved by an application that received a priority score of 200. According to the "proces," the "quality" of the two proposals is identical because they are both at the 15th percentile, regardless of the underlying disparity in priority scores. Obviously, there's something weird there. Perhaps one study section was more lenient with overall scoring than the other; OR, perhaps one study section had a larger proportion of truly outstanding applications in its pool than the other. How can you tell? That's one way in which the process is indeed capricious. Another way in which the process is capricious has to do with the reasons why a reviewer might like one application better than another. Does the reviewer like risky proposals, or not? Does the reviewer prefer certain paradigms or hypotheses over others? Or certain methodological approaches or model organisms over others? Does the reviewer carry a subconscious (or even conscious) bias for or against a particular research scientist (and yes, sometimes scientists are pals with, or enemies with, one another)? How can we interpolate the inherent reviewers' biases (that reviewers necessarily bring to their reviews and scores) with "objective" measures of quality? And if some scientist/reviewers favor one paradigm or one hypothesis or one methodological approach or model organism over others, WHO is able to tell us -- objectively -- which scientist/reviewers are "right" and which are "wrong" (in advance of actually carrying out the work to find out)? Clearly this is a legitimately complicated intellectual mess. But it is exactly this kind of intellectual mess that leads some potentially "high quality" applications to get significantly lower percentile scores than others. How can the metrics reported out of study sections possibly be the best way to judge which applications are truly of the highest quality and worthy of funding? In other words, is it even reasonable to hope to "objectively" measure quality of a forward-looking scientific grant application, given the human nature of the reviewers and the caveats inherent in the bureaucracy of the review process? I for one think that letting the "capricious" peer reviewers have 100% complete control which grant applications are chosen for funding is a recipe for disaster. The US is already falling behind Europe and Asia in terms of the quality of biomedical research we produce, especially at the cutting edge of new knowledge and new paradigms. Our system for predicting which applications will produce "the best science" is clearly not working. It is therefore imperative that NIH administratively tweak the process so as to try to ensure that fresh new ideas, new blood so-to-speak, get supported. As far as I'm concerned, NIH does not seem to be willing to go far enough toward changing the stagnant culture of peer review and grant application selection. Return to Top comment: Administrative meddling with peer review by Louis Curacao [Comment posted 2009-11-04 07:14:29] The angst over new and early stage scientists indicates a broader anxiety among established NIH investigators over what is seen as administrative meddling adding to an already capricious peer review process.
Another deeply worrying example is the favor currently been shown/signaled by CSR to new submissions over 1st A1 and second (soon to be phased out) A2 revisions of RO-1 grants. That revised grants judged at peer review to be of high priority may be not be funded over new grants receiving lower priority scores is a clear indication that the best science may not necessarily supported by the NIH. Return to Top comment: Maybe by Rafaela Canete-Soler [Comment posted 2009-11-03 05:10:46] NIH continues to support the best science through R01s Maybe. What appears to be much more evident is that the R01, as investigator-initiated research, has been the one mechanism providing best returns. I don?t know much about the Program Projects Review Process. It occurs to me that the returns could be improved with two levels of initial review: 1. Evaluating individual components of the Program Project as if they were R01s for significance, innovation, advancing the field etc 2. Evaluating the Program Project as a whole, by a Board of independent reviewers from step 1, for their scientific significance and impact in public health. If that is the case, my apologies. |