Advertisement

NIH R01s: No Longer the Best Science

By Les Costello NIH R01s: No Longer the Best Science Funding preferences penalize senior investigators, lower the quality of science. © DAN PAGE For 60 years, the US National Institutes of Health R01 research grant mechanism has aimed at funding the highest-quality science to address the important contemporary issues. That began to change in 2008, after NIH Director Elias Zerhouni issued the goal to “Fund the best science, by the best scienti

By | September 1, 2009

NIH R01s: No Longer the Best Science

Funding preferences penalize senior investigators, lower the quality of science.

© DAN PAGE

For 60 years, the US National Institutes of Health R01 research grant mechanism has aimed at funding the highest-quality science to address the important contemporary issues.

That began to change in 2008, after NIH Director Elias Zerhouni issued the goal to “Fund the best science, by the best scientists…” 1 As a result, new guidelines, requirements, and considerations have been introduced. Unfortunately, these changes are antithetical and counterproductive to achieving Dr. Zerhouni’s stated goal.

The new process has transformed the R01 mechanism into a channel that provides funds for the training and development of new and early-stage investigators (ESIs).

New directives require that “NIH will support New Investigator R01 awards at success rates comparable to those for established investigators submitting new R01 applications.” 2 Even recipients of several already-existing NIH training and development grant programs qualify as ESIs for up to 10 years of such funding.Recently NIH required that “...applicants eligible for consideration as first-time R01 investigators...will be paid using an extended payline of the 22.0 percentile; compared to 16.0 percentile for others.” 3 These directives mean that funds for the best science by the best scientists will be diverted to fund lesser-quality research.

Moreover, the NIH policy introduces and justifies a form of age discrimination, which guarantees that grant proposals from senior investigators and longtime-funded investigators will be denied based on age, not on scientific merit. This policy will introduce, exacerbate, and even justify covert and overt discriminatory tendencies of reviewers, when it is essential to suppress such influences from a scientifically credible and objective peer review process.

As a senior investigator with grant funding for 48 years, and a past reviewer on several NIH and other agencies’ grant review panels, I vehemently object to this policy. I do so as an obligation to defend a 60-year history of advancements in science and medicine, which was based exclusively on funding the best science. As young investigators, I and my colleagues successfully competed with established researchers based on merit, without preferential treatment. NIH states that in recent years young investigators have not competed successfully with established investigators, hence the need to downgrade the quality of science funded through the R01 research grant mechanism. 2 But this neither recognizes nor addresses the cause(s) of the problem.

A major factor is that contemporary biomedical training programs fail to train young investigators to be scientists. They are trained to be myopic super-technologists, predominantly in areas of molecular biology and molecular technology. They lack the broad holistic background and capacity to integrate molecular events with cellular through organ-systems physiological and pathophysiological principles and relationships.

So, we have a striking contradiction. On the one hand, NIH identifies the critical importance of, and need for, “the most accomplished, broad-thinking, and creative scientists to serve on NIH study sections.” 1 On the other hand, those “most accomplished, broad-thinking, and creative scientists” are penalized in the grant review process because of their experience and success. Until this problem is addressed, the number of broadly trained and knowledgeable biomedical scientists will continue to decline, as will the quality of biomedical research. Then there will be no need for NIH to impose special considerations for young investigators—there will be no high-quality science and scientists to compete against.

Les Costello is Professor of Physiology and Endocrinology at the University of Maryland, Baltimore, Md.

Advertisement

Comments

September 1, 2009

\nI feel sympathetic with the author. My main take home message from this article is that SCIENCE IS SCIENCE and is age-independent and, most of the time, experience- dependent. \n\nExperienced investigators are not only excellent practitioners of the scientific method but also knowledgeable of other?s work in their field and, I suspect, have developed an special ability to apply their knowledge to emerging concepts and novel ideas. They are also skillful in communicating scientific advances (papers) and have charisma when it comes to making the scientific process a live process. I have come across few of them in my life experience. \n\nNothing justifies discrimination. Obviously, when it comes to science, no discrimination should be allowed. What I would emphasize, though, on the author?s article is: discrimination for no one, opportunities for all investigators. This is very easy to say but very difficult, though not impossible, to implement. Why I think is difficult?. Because resources are finite and, unfortunately, as the community grows larger and older, the resources are becoming more and more restrictive.\n\nOne practical question I have is: given the increasing finitude of available resources, what would be desired opportunities for excellent senior investigators who have been productively engaged and funded for 30-35 years ?. Perhaps, academic and research institutions could come up with creative models, supported with federal funds, to use the excellence of these investigators for the benefit of science and younger scientists. I don?t mean at all a ?club to keep senior researchers happy?. No, not at all. What I mean is ?societies of investigators, senior and non seniors?, where to breathe an atmosphere of inter and multidisciplinarity, with hands on the scientific method from diverse experimental settings, where scientific communication is the ?abecedario? (spelling) to teach and learn (for publication and research proposal writing) and where science is both challenge and fun. Younger generations can also learn how to pass the scientific torch productively and meaningfully.\n\nPerhaps this is a too idealistic vision but is doable, in my view. One platform is that of systems biology of which we all are, some more than others, newcomers. We have in our hands the human genome and, in front of us, a number of biological problems, as well as diseases to explain, and hopefully, to correct.\n\nMany things, pros and cons are certainly left unsaid. I am sure that some of you will say them much better.\n\nThank you\n
Avatar of: anonymous poster

anonymous poster

Posts: 5

September 1, 2009

When I heard about this I knew the old guard would kick up a fuss.\n\nThe best part is "we survived the process - so can you!" This is the same logic that keeps medical residents up for 24 hours at a time when this has been proven to be dangerous to patients health.\n\nThe truth is, when a grant review committee receives the same exact grant application from a new investigator with 15 papers, and an ancient one with 200 papers, they give it to the old guy due to his "reputation" whether it is deserved or not.\n\nThe argument that young scientists are technologists with no broad reach is ridiculous - no doubt you started the same way sir, and only accumulated your integrated knowledge over a lifetime.\n\nI left basic academic research for one reason - people like you running the field. With a payline of 10%, do you really expect a young investigator to try and build a lab and raise a family and teach? The system is now set up so that only the old established labs survive, and the few craziest young investigators willing to sleep in their labs while writing grant applications furiously have a slim chance of making it through.\n\nYou just don't realize that you are losing a whole generation of good scientists due to the lack of funding for young PIs and a ridiculous 5-6 year post-doc system that leaves you with no IRA and a slim chance of finding a decent job.\n\nThese changes MIGHT help, but they won't be enough - most of the smartest people I know realized this was essentially a lottery and left science ages ago. The whole system is broken, and people like you are going to keep it broken while you write papers on the backs of grad students and post-docs that should be earning twice what they are, and should have a reasonably predictable future but don't.
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 1, 2009

Dr. Costello makes the argument that today's early-stage investigators should be considered through the lens of the environment when he began 48 years ago.\n\nBut times have changed drastically since then. Success rates and much lower today than they were in his time, and the age of PIs is much older. For example, more than 50% of PIs were below age 40 in 1980 (probably even lower than in Dr. Costello's time); that number is less than 20% and declining today.\n\nI suspect Dr. Costello's experience would be quite different if he was just starting his career in 2009 instead of 1961.
Avatar of: anonymous poster

anonymous poster

Posts: 6

September 1, 2009

The exact opposite arguments also exist. The best science and the best scientist will be always awarded, no matter how old they are and how established they are. It is the review process that needs to be updated.
Avatar of: anonymous poster

anonymous poster

Posts: 10

September 1, 2009

As a chartered study section member, I have the first hand knowledge of the current reviewing process. It is a little disingenuous to claim that the grant ranked at 16%tile is better science than a grant ranked at 22%tile. Essentially, all grants in the top 25% are great scientifically. The reputation of an established investigator often carries a lot of weights and resulted more favourable scoring than a new investigator could get for a grant with equal scientific merits. Therefore, it seems to be very inaccurate to claim that the quality of science will suffer by giving new investigators a small break.
Avatar of: anonymous poster

anonymous poster

Posts: 16

September 1, 2009

A quick scan through NIH report (http://projectreporter.nih.gov/) shows that Dr. Costello was last funded by NIH in 2005.\n\nSo, Dr. Costello has not submitted a competitive application to NIH in over 4 years. Perhaps, it is Dr. Costello who is no longer doing the best science? I can't see how he can place his lack of success on the backs of New Investigators. \n\nI wonder if any of his recent submissions would have been funded without the differential paylines for NIs/ESIs?
Avatar of: anonymous poster

anonymous poster

Posts: 1

September 1, 2009

It is unfortunate, and certainly unscientific, that several comments of an ad hominem nature have been insinuated into this discussion. Please address the merits of the argument; refrain from speculating about the merits of the author.
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 1, 2009

As another senior investigator in a seemingly similar situation I suggest there is considerable validity to the views of the author. After 15 years of continuous NIH support it is indeed frustrating to receive a percentile score better than any of my previously successful applications only to go unfunded, while the range is extended perhaps 2-fold for "new" investigators. Although in part this reflects the increase in competition that characterizes our current situation that doesn't mean it is a correct solution to the problem.
Avatar of: anonymous poster

anonymous poster

Posts: 4

September 1, 2009

I agree whole-heartedly with most comments on this article. Dr. Costello is failing miserably at comparing apples to apples. NIH paylines when he was applying for grants were dramatically different than they are today. From 1975 to 1979 NIH paylines were at 48% (238!) and from 1980-84 were at 38% (the so called down years). No young investigator today will ever compete for such high paylines. (http://books.nap.edu/openbook.php?record_id=9928&page=14)\n\nAdditionally writing grants is an art, that is why some refer to it as the art of grant writing. I will disagree completely that new age scientists are not creative or broad thinkers, they just have not mastered the art of selling their science or writing the grant in the most lucent and compelling manner. I actually feel most young scientists are more creative thinkers, write riskier grant proposals that are often too ambitious, and therefore get poor scores because they are not proposing baby-step science that the NIH prefers to fund.
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 1, 2009

A hearty "hear-hear" to the post below entitled "Talk about defending the status quo".\n\nI too have left basic research, because as that poster says, it is at best a lottery, and at worse a good old boy network, plain and simple.\n\nI'm sympathetic to Dr. Costello but let's be honest: he has far more funding options available to him than the first-year faculty member. Corporate sponsorships, named professorships, private funds, etc. \n\nAnd no, I didn't leave basic research science because I was poorly trained and couldn't hack it - it was quite the opposite. I left for another profession where those skills were more respected and appreciated.\n\nScience will continue to sputter in this country until peer review for both publications and grants is dramatically overhauled - starting with anonymous submissions. If I don't get to know who my reviewers are, they shouldn't get to know who I am. Either have it double-blind or zero-blind.\n\nAll in all though, the complaints of Dr. Costello mean the system is doing what it is designed to do. If more young PIs are getting funded and the field is growing some roots and a future instead of being a baby-boomer top heavy farce, I'm all for it.
Avatar of: anonymous poster

anonymous poster

Posts: 6

September 1, 2009

I am very disappointed with this commentary. Dr Costello appears to be using this journal as an outlet for his subjective grievances which are not supported by scientific fact. They are an opinion and as such are open to challenge as many have done here and are to be considered with scepticism. \n\nThe NIH system is creaking as a funding mechanism for new and established investigators. The system works but it is not without it's flaws. I have to agree with many of my colleagues who have supported the peer-review model of the study section and I support those that have rallied to the defense of the beleagured junior (read investigator under the age of 45) investigator. \n\nThe science foundation in the United States is suffering from grant funding erosion, a bottle kneck of investigators competing for limited funding and a consequential exodus of talent from this country to other, arguably more competitve nations. This sad fact was argubaly not helped by an unsustainable inward investment of Federal Tax dollars to the NIH research programs pool. The ballooning number of investigators that succeeded in attracting R01s and other grants under the Clinton administration years recruited post-doctoral fellows who are now the next wave of new investigators. Those that recruited are now the established pool of equally concerned senior or established investigators. The unsustainable doubling of the NIH budget created a Catch 22 for the new wave of junior investigators and established investigators alike. Now that the NIH pool of funds has essentially capped off, both junior and senior investigators are crying foul. Additionally, I have to say that junior investigators that are failing to attract adequate funding are losing their positions, the ecomonic situation is forcing redundancies and non renewal of contracts either because the academic clock is ticking past their recruitment start-up "sell by date" of 3 years or because medical schools can no longer support them on soft money in start-up recruitment packages.\n\nAs a result, junior investigators have two rather unpleasant, stressful and unfortunate choices: 1) Failure to attract an R01 within 3 years of recruitment will result in them losing their job - fact; 2)In weighing up the options of the ruthless funding environment of academic research in the Unites Staes, junior investigators have the option to exit academic science altogether and cross over to industry or government related jobs.\n\nGiven the above facts, I was dismayed to read this commentary by Dr Costello and come across as a grievance agenda and are not at all written in an objective or sympathetic manner. This is likely the worst time in history for a young investigator to try and survive the Darwinian selection processes of grant review, manuscript peer-review as well as the political landscapes and biases of the subjective peer-review system. \n\nThe NIH policy to review somewhat differently an application from an established investigator as compared with a junior investigator is wholly appropriate. It is a system designed to attract the best science from the best young investigators and not to lose this talent from the research pool. \n\nYoung investigators may have the progressive and insightful ideas that established investigators have but they certainly do not have the large bodies of preliminary and pilot data, established research teams, established infrastructure, collaboratrive arrangements and so forth. The junior investigator is truly up against it and thoroughly deserves the slightly more favorable conditions for critically analysing their R01 grants.\n\nTo deprive them of this seeks only to damage the future academic and scientific longevity of the United States research infrastructure. Senior investigators, have in my opinion, and without doubt all the advantages. One can not surely deprive the junior investigator of any special status for selfish or other self-serving agendas that represent perhaps a grievance by an established investigator.
Avatar of: anonymous poster

anonymous poster

Posts: 16

September 1, 2009

Stick to the facts,\n It is hard to argue with the merits of his arguments when he provides 0 evidence in support. He states that ESIs have a slightly easier time getting funded therefore some of their science is not the best. As others in the comments noted, the difference in quality of science between a 10 percentile and a 20 percentile is minuscule. Both represent scores of less than 180 for most study sections. Solidly in the excellent range. \n\nThe author provides no objective data that supports his claims. If he thinks he would have gotten funded in the absence of the various ESI/NI programs show us his numbers. \n\nI think it is relevant that he is writing this from the perspective of senior scientist whose grants are no longer competitive. His comments malign all ESIs/NIs who got an R01 over the past few years.\n\nFinally, to put the blame on NIH and its programs for his troubles getting funded without providing a single piece of supporting info sounds like sour grapes and whining to me. If he is going to whine he should complain to Congress for a flat budget and all of his colleagues who trained more Post-docs and grad students than the system can support.
Avatar of: anonymous poster

anonymous poster

Posts: 7

September 1, 2009

It seems that basic science is no longer funded, whereas medical-associated research is, even though their hypothesis can be quite a reach. As long as you are studying something that MAY be involved in disease, you can probably get funding.\n\nAfter spending 5 yrs in graduate school, while watching many labs at my institution fail to renew decade old R01 grants, I am leaving science for good. It's not worth my life competing with scientists who try to link minor data to bogus hypotheses just to ensure funding. Didn't James Watson suggest that basic science is worth funding? Is the NIH even listening to anyone, or just listening to themselves?
Avatar of: Martha Stokely

Martha Stokely

Posts: 8

September 1, 2009

I believe the author missed understanding that NIH was simply trying to compensate for subconscious bias, and was not trying to actually favor early stage investigators. Subconscious bias has always favored established male investigators from the "best" schools, and while some of those aspects have genuine merit, they are often given more weight than they deserve (because they are easier to assess than scientific merit). NIH's peer review system might be better served if a proposal and its PI were scored separately (in isolation from each other) and those scores later combined. This could control any undue influence of subconscious bias without resorting to differential paylines (which will always engender hostility).
Avatar of: Stephen Dolle

Stephen Dolle

Posts: 16

September 1, 2009

I can't speak to the very detailed points posed by this author and commentors. But - if you think your funding status is tough as an established scientist, try getting it as a patient-inventor. It's impossible! In 1997, in response to world-wide diagnostic challenges of hydrocephalus and CNS shunts, I created a home monitoring system. But my technology sits on a shelf so that all kinds of less important stuff can be funded!\n\nSo I and others wait for a "famous" person or scientist with small aspirations to take up an interest in hydrocephalus. The failures of innovation in some of our health and disease areas is a national disgrace, no doubt a result of special interests NIH funding. Add in the popularity of projects with big $ commercial aspirations, and universities who encourage this practice to generate royalties, and you've got a grants system in need of major repair.
Avatar of: anonymous poster

anonymous poster

Posts: 24

September 2, 2009

It's ironic to find established med school PIs blaming new investigators for the difficulty in getting steady research support. This is like blaming low income buyers of cheap starter houses for the mortgage collapse. Culprits include translational research, big science 'omics' proliferation, overfunded mega labs, and greeedy overexpansion by med schools. Poor support for science by some admins didn't help either. BTW, I am old, currently funded and quite productive. Dr. Costello could be great & still lose his funds. Looking at publications/citations is a better way of assessing this than whether he won the funding lottery recently.

September 2, 2009

\nHello anonymous,\n\nHave you seen the recent news here at The Scientist on a retracted nature genetics paper ?. \n\n
Avatar of: les costello

les costello

Posts: 2

September 2, 2009

\nI am delighted that my opinion piece created such responses; favorable and unfavorable. I do feel compelled to respond briefly to some of the criticisms. I had hoped that the motives for this piece would not decline to the level of personal attacks. I wrote this piece to challenge a process; not to justify any personal dissatisfaction based on supposed lack of success in obtaining grants. With my colleague, we currently have two active grants. The comments of some seemed to be bent on discovering a ?sour grapes? motive. I can assure all, that my life-time record of scientific achievement is most satisfying and gratifying for me; and would not be dampened by any current or future failures in obtaining NIH grants; though I still compete. About thirty grants, encompassing forty-eight years of continued funding, and about one hundred and thirty budget years is satisfying enough for me.\nSetting that aside, the issue of unique difficulty of contemporary times for young investigators is invoked by several respondents. The fact is that the NIH justifications were based on data of five to fifteen years ago. The issue that I raise is not that assistance in the development of young investigators is inappropriate. Quite the contrary; and there exists more programs than ever to provide such assistance. The fact is that, by NIH own data (1997 Report of the NIH Working Group on New Investigators) ?the R23, R29, R03, or K type development awards have had little or no impact in enhancing the success rate of young investigators.? This is the major concern that some apparently wish to ignore. Such conditions cause me to have concern and objection for the use of the RO1 mechanism to compensate for this problem.\nNo matter how one wishes to ?slice it? or to ?spin it? or to ?justify it? or to ?divert attention from it?, the new requirements are discriminatory; and I do not support such discrimination! It has no legitimate role in the RO1 review process. It is a fact that it will result in some of the best science being unfunded to support lesser quality science; and that should not be an outcome of the RO1 grant support mechanism. Justifying it on the basis that it will affect only a small number of grants is obnoxious. It is incompatible with the stated R01 goal ?to fund the best science by the best scientists.? It violates CSR?s own requirement ?...to see that NIH grant applications receive fair reviews -- free from inappropriate influences -- so NIH can fund the most promising research.? \n
Avatar of: anonymous poster

anonymous poster

Posts: 24

September 2, 2009

Yes I did. It shows that when a rare instance of fraud or a really serious buy honest screwup surfaces it causes a big scandal; this often ruins the careers of the people involved and cuts off serious citations. A bigger problem is the stealing of original ideas by the over ambitious and under creative. \n What of it? Publications and citations is really the only effective metric of productivity. It isn't really controversial to say that if you didn't write a paper you weren't very productive, and if nobody cites your paper it didn't have much impact. Do you think that there are dozens of unpublished/uncited but highly productive people in science? Who or what did their work influence? I'd like to see productivity linked to funding, but in fact the linkage is weak; see, for instance, the case of Dr. Costello who looks pretty good on the production end. \n People like Dr. Costello should have some funding, but a better way to provide it might be to give a few fewer R01s to giant labs, to open up the review process to new ideas (it is stultifyingly conservative), and to place less of the available money in the hand of great leaders who make decisions based on magazine articles. I'm NOT in agreement that we should eat our young, even if somebody makes a Modest Proposal in amusing fashion.
Avatar of: anonymous poster

anonymous poster

Posts: 24

September 2, 2009

Dr. Costello, do you seriously think that big science 'omics' projects and translational research are compatible with funding the 'best science'? It is pretty clear that major med schools and other players have succeeded in gaming the system over the last 20-30 years, and the new kids on the block are fed up with some reason. It was ridiculous for people to attack you based on your record of achievement, but you should realize that junior investigator support is pretty minor compared to the gross abuses of the system and at least has a point. NIH has lost much of its ability to fund discovery science, and it's ludicrous to pretend that we know enough biology to concentrate on disease.
Avatar of: anonymous poster

anonymous poster

Posts: 85

September 2, 2009

Dr. Costello's response fails to acknowledge the key -- and in my opinion, thoroughly valid -- comments of the various thoughtful posters who disagreed with his thesis. \n\nSpecifically, it seems to me that he fails to acknowledge that: 1, the current NIH peer review system contains a built-in bias toward established "well known" senior investigators, and AGAINST truly novel ideas (frequently proposed by creative but naive beginning investigators); 2, there is absolutely no real difference between a 15th%ile and a 25th%ile score in terms of the reviewers' determination of the "quality" of the research proposed (this is a fact that is quite well understood in the DDER's office at NIH); and 3, if we don't make room for the younger scientists, then when the older scientists retire or die off there will be nobody left.\n\nYes, I have taken some liberties in my paraphrasing of the arguments of others who have posted here. But I am sure that the posters will recognize their intentions in my words.\n\nThere are other issues that others have not raised. For example, several posters seem to agree with Dr. Costello that US science, in particular NIH-funded biomedical science, is the best in the world. Frankly, I don't think that is true any more. Rather, I think that we are still coasting on our laurels of yesteryear. My observation is that the real action, in terms of new ideas and breakthroughs, ESPECIALLY in the areas of very basic biology, is taking place outside the US (Europe and Asia especially). And without breakthroughs in basic understanding, we have no platform on which to build medical progress. So I have become convinced that, for perhaps the past ten years or so, the US has lost its leadership position. There are lots and lots of reasons for this, but I think that the "peer review" mentality, which generally favors incremental progress in "under the lamplight" areas of research, is certainly one of them. \n\nI think that Dr. Costello's perspective, which is highly "conservative" in its essence, is rather typical of the NIH study section reviewer. This is probably the reason why NIH top brass keeps trying to change the paradigms under which study sections operate, and probably also why NIH keeps failing in this regard. \n\nPS -- I do agree with those who expressed the opinion that, all too often, contemporary graduate students and postdocs are trained to "do" research rather than to think in creative and analytic ways. This has been going on for so long now that the products of such scientific maleducation are training the next generations of scientists to be just like themselves and sitting on study sections in judgment of grant proposals. This, too, is probably one of the contributing reasons why US science is losing its punch.

September 2, 2009

\n\n\nHello Dr Costello,\n\nThe problem in the article is that you make some assertions/generalizations on new investigators that, in my view, are not based on evidence and too much of huge statements to accept. For example, and this is only one example\n\n ?They lack the broad holistic background and capacity to integrate molecular events with cellular through organ-systems physiological and pathophysiological principles and relationships?.\n\nI absolutely disagree with this and is hurting to hear it from an investigator who has been funded for so many years. Sorry.\n
Avatar of: Greg Czyryca

Greg Czyryca

Posts: 4

September 2, 2009

Dear Dr. Costello:\n\nSadly, you are wrong by equating the score with the quality of science (and this equating is the cornerstone of your argumentation). As long as there are criteria like "Investigators" or "Environment", the score DOES NOT represent the scientific quality, and the review process discriminates against younger investigators. Perhaps the outright affirmative steps toward promoting new investigators is indeed a bad solution, but relying on the score only is even worse. This dilemma has a simple solution: the review process should be made as blind as possible. The review panels should evaluate shortened, anonymous proposals focused on the scientific merits ONLY. \n\nBest regards,\nGreg Czyryca
Avatar of: anonymous poster

anonymous poster

Posts: 16

September 2, 2009

I can't help but wonder how much of this discussion is tied to the huge expansion of universities, medical schools, and non-profit org's in recent years (not to mention hikes in tuition). Such over-expansion has been fueled by Wall Street non-profit moneys looking to avoid taxes, but still wanting the influence of the moneys - as Anonymous below raises.\n\nReply to:\nNot the best science, but don't blame new investigators\nby anonymous poster\n\n[Comment posted 2009-09-02 12:15:45]\nIt's ironic to find established med school PIs blaming new investigators for the difficulty in getting steady research support. This is like blaming low income buyers of cheap starter houses for the mortgage collapse. Culprits include translational research, big science 'omics' proliferation, overfunded mega labs, and greeedy overexpansion by med schools. Poor support for science by some admins didn't help either. BTW, I am old, currently funded and quite productive. Dr. Costello could be great & still lose his funds. Looking at publications/citations is a better way of assessing this than whether he won the funding lottery recently.

September 2, 2009

\n\n?Publications and citations is really the only effective metric of productivity. It isn't really controversial to say that if you didn't write a paper you weren't very productive, and if nobody cites your paper it didn't have much impact?\n\n\nHello anonymous,\n\nI would agree with the statement if I knew how you define productivity and impact. I am sure that your conviction comes from your experience in your field of research. \n\nIn my field, productivity is not always in direct relationship with quality but with quantity and resources (positioning in editorial boards, multiple connections, funding capacity that is also helped by positioning in review committees).\n\nIn my field, citations/impact are not always in direct relationship with creativity and advancement of the field but in direct relationship with quantity and influential positioning. \n\nThanks\n
Avatar of: null null

null null

Posts: 2

September 2, 2009

Dear Prof. Costello,\n\nI am likewise delighted by the impact of your opinion article; people were sending me the link all day with outraged comments. I am happy to see in the comments here, as well as from my email box, that many scientists are able to see right through your complaints. I think you would do well to review the initial paragraph of anon @ 13:35 today (9/2) and think seriously about the points being made. \n\nWhat you seem to be overlooking in your assertion that you know the nature of "scientific quality" is that 1) yours is only one of several valid and as yet unfalsified hypotheses, 2) we frequently only appreciate the true value of a scientific finding or approach decades later and 3) most so-called objective measures of quality suffer severely from the circular logic of those that are most successful and respected defining "high quality" as that which they conduct. \n\nLet us return to your cry against "discrimination". The NI/ESI designation did not invent this. This effort (as well as the prior R29, etc) is designed to overcome an existing, demonstrable discrimination against young investigators. Objective grant review data (scores, funding %, etc) over past decades testify clearly to this. \n\nThe only way we can credit your assertions are to believe that the younger investigators of today (who are 38+, btw) are poorer scientists than those of more advanced age are now, or when they received their appointments at age 28 or so three to four decades ago. I contest this. \n\nFrom the perspective of my narrow subfield and a 4yr grant review stint recently completed this seems utter nonsense. As others have pointed out in this thread, the differences between the 12%ile and 25%ile application at present time is negligible...it comes down to essentially subjective preference. Which leaves a big whopping window for subtle and not-subtle review bias. IME, backed by the overall CSR/NIH review numbers, there is an overall slant to give slightly better scores to the older and more established investigators. This slight push is easily enough to differentiate essentially identical applications by 5 or 10 percentile points. \n\nThe bias at the point of primary review is also enhanced by program decisions about which grants to fund out of review order. POs are not dissimilar to reviewers in their bias for investigators they know and who have lengthy records of performance...features that are not correlated perfectly with "the best science", I feel confident you will admit? Perhaps the ESI initiative will shake up this bias as well. Again, it is a measure to redress an existing bias, not a novel introduction of "discrimination" as you put it. \n\nI close by congratulating you on your willingness to publish what many of your age cohort peers are saying privately. It is a contribution to the enterprise of biomedical science if such ideas, as misguided as many think they are, are discussed openly instead of whispered around the water cooler and asserted uncontested at department meetings. I would encourage you, however, to engage your demonstrated scientific acumen in your analysis of this matter. Listen to the alternative hypotheses being advanced, actually look at the data being linked, openly consider that you might be wrong. \n\nIf each of us do this, perhaps we can come to a better understanding of which changes to the NIH funding system are likely to be positive or negative. \n\nMike Taffe
Avatar of: anonymous poster

anonymous poster

Posts: 125

September 2, 2009

Students are mostly, if not totally, valued as the cheap laborers to do the grueling and tedious experiments, especially those that require odd schedules, to basically end up as glorified lab technicians with PhD's in the biomedical science education. Obviously, they are given little, if any, real opportunity or time to develop as broader scientists that Les Costello claims are lacking. Naturally, in such circumstance, who would have a greater advantage of winning the NIH RO1 funding? Why is a preferential treatment given to the disadvantaged unfair, but fair to the already established and privileged? Dr. Costello, you and your fellow senior researchers who agree with you, ought to be ashamed of yourself!
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 2, 2009

Part of the reason than R23s, R29s, and the like have not been successful is that they are not R01s and have a smaller budget to match.\n\nOthers have done a good job arguing the folly of trying to split hairs in the worthiness of priority scores in the top 25 perentile.
Avatar of: anonymous poster

anonymous poster

Posts: 28

September 3, 2009

Since 2004, I, as a young investigator, have submitted >15 R01 (two R21). Two of them are scored once but none of them are finally funded, because of short of the fund to pursue "picky" comments from outsider reviewers (difficult to understand this kind of comments). \n\n"Unfortunately", with the supprot from small grants, I published at least two papers each year in average. Somce papers got > 10 citation within one year of publicaton (at least 10 of IF???). I comaped to someones with one or two R01, quality and quantity are not behind them. However, I am still suffering, because more money is needed to address reviewer's outsiding comments.\n\nMany peers encouraging me to spend more time on connections. How come? \n
Avatar of: anonymous poster

anonymous poster

Posts: 107

September 3, 2009

From the distant perspective of someone who is no longer competing for public research money: no way can you eliminate bias from the grant review process, even with blind reviews. Evaluating research ideas on their merits does not take place in a vacuum. It always seemed to me that this process took place at research conferences, where a consensus about priorities would emerge, which participants would then bring to their study sections, almost as if it were an unwritten request for proposals. If you were good at chatting up people at conferences, you could get your ideas on this unwritten agenda and get funded. Obviously, big names tended to get more attention and consequently had a leg up in this process.\n\nIf you can't eliminate bias, your only alternative is to manage it -- either passively by accepting it, or actively by counteracting it, as in current NIH policy. Either way is unfair, so fairness is off the table as a criterion. I guess you end up deciding what's healthiest for the profession in the long run. Giving a leg up to the next generation of researchers seems not unreasonable.

September 3, 2009

\n\n\nResponse to\n\n[Comment posted 2009-09-03 00:48:33]\nSince 2004, I, as a young investigator, have submitted >15 R01 (two R21). Two of them are scored once but none of them are finally funded, because of short of the fund to pursue "picky" comments from outsider reviewers (difficult to understand this kind of comments). \n\n"Unfortunately", with the supprot from small grants, I published at least two papers each year in average. Somce papers got > 10 citation within one year of publicaton (at least 10 of IF???). I comaped to someones with one or two R01, quality and quantity are not behind them. However, I am still suffering, because more money is needed to address reviewer's outsiding comments. \n\nMany peers encouraging me to spend more time on connections. How come?\n\n\nDear anonymous,\n\nYour numbers (>>15R01s +2R21s) speak tons of your drive for science. You?re equaling the numbers of grants that some established investigators are awarded in a similar period of time. So, don?t get discouraged. Time and opportunities might be on your side.\n\nWhat are the connections that I would look for, if I were in your shoes?\n\n\n1. Talk to true pioneers in your specific field and closely related ones. They know a lot about scientific challenges and adversity episodes. True science never comes easy. Send them your last two summary statements and ask them for candid input. Expose yourself totally so that they are also aware of your weaknesses. Don?t be afraid because true pioneers will offer you ways to improve your weak points; will never ask you for information to benefit personally from you or hurt you professionally. Listen and reflect on their suggestions.\n\n2. Talk to the authority in your Department (Chair, Vice Chair for Research etc). Explain your situation. They might come up with useful ideas to tackle your grant situation and maybe can find generous donors to help you temporarily with your experiment finances. Trust them. Trust elicits trust.\n\n3. Rewrite or write your proposal taking into account the advice from your pioneer peers, particularly in those pieces in which you and them see you as to improve. Since proposals are now much shorter, you might want to go back to those peers and see whether they will have time to read your final version. They most likely will try to find the time.\n\n4. If you?re unsuccessful again, reflect on your new SS and write down your scientific view of the critique, include potential biases and do not underestimate your own biases. Write them down also.\n\n5. Send your scientific view of the critique to your Program Director, listen to his/her view. If you are convinced that your SS does not reflect a rigorous and fair review, tell him/her that you?re sending a copy to your SRO and the Director of the CSR. They all are your scientific representatives and need to know what the constituents are going through. Wait for an answer, very likely and hopefully they will do it in a reasonable period of time. If they don?t, write back. If their answers do not convince you, send a respectful rebuttal.\n\n6. At this point, and after 17-20 attempts, if you are unsuccessful and you have a family to raise and serious commitments to fulfill, you need to consider if your drive for science can be realized through alternatives scientific avenues, rather than the academic one. Start looking for those alternatives in industry (not very good prospects now), in government agencies. Look at the bright side of life. Your experience has been tough but very valuable. You know now the good and not as good sides of academic science. Without looking for it, the circumstances have made you an effective agent for change. Science needs that very badly. Go for it.\n\nPS. Always try to not act at the back of anyone. If you take a decision to leave academic science, visit your political representatives in both sides and explain them your situation and why you?re leaving academia. Science is of national interest to the USA, should be political party-independent. You and them can help making the system better for future generations.\n\nWarning: I cannot guarantee success but I believe that this is the Connectome that science deserves.\n\nThank you and good luck.\n
Avatar of: anonymous poster

anonymous poster

Posts: 4

September 3, 2009

Dear Dr. Canete-Soler:\n\nYour advice to the young investigator is excellent, perhaps with one exception: it is largely useless to seek help from the NIH PO and SRA. They are not the impartial representatives of the applicants - above all, they represent the institution. \n\nAdmitting the fact of bias/incompetence on the part of the reviewers may result in litigation, and for this reason the NIH staff will do their ABSOLUTELY BEST to sweep the issue under the carpet, hence their usual advice is "amend according to the Reviewers' comments and resubmit".\n\nI do not have a better advice to give though. Perhaps people/institutions should start litigating against the NIH to force the Institutes to implement some element of responsibility on the part of the reviewers.
Avatar of: anonymous poster

anonymous poster

Posts: 23

September 3, 2009

Yes, some jr faculty may not be doing the best science YET as those sr colleagues. However, there is one biological law we all have to follow: the old ones, including myself, will retire soon or worse die. Often, the best science in most sr faculty's labs are conducted by very young students and postdocs, not by the PI him/herself. Without supporting young investigators today, the best science of tomorrow will just not come freely. So, please keep this long-term goal in mind when voicing your immediate concerns about federal funding.

September 3, 2009

\n\nDear anonymous,\n\nThank you so much for your outstanding input. My view has always been that the Program Director, who represents the specific Institute, was supposed to be the sponsor applicant and, therefore, the applicant ?lawyer?. In practical terms, you are absolutely right. I have always received that recomendation: review the grant and resubmit.\n\nI absolutely agree and support your suggestion to look for effective mechanisms ?to force the Institutes to implement some element of responsibility on the part of the reviewers?.\n\nThanks again. I do appreciate it.\n\nRafaela\n
Avatar of: anonymous poster

anonymous poster

Posts: 4

September 3, 2009

Rafaela,\n\nThe lack of support from the officers varies from institute to institute. IMO the absolutely worst is the NIAID - even if you appeal, the Council will ignore the specific issues raised in your appeal and issue a template-based rejection letter.\n\nI see two solutions for this problem:\n\nSolution 1. The culture of permissiveness toward substandard reviews encourages reviewers to write more and more ridiculous critiques. Eventually, even without the NIH admitting the faults during the review process, successful litigation may become possible, based on the summary statements alone (my own collection of rubbish is approaching this critical mass). At some point I will hire a lawyer, offer a 50% bounty...\n\nSolution 2. Some young investigators still become members of the review panels. We can do a lot of mischief there. Especially, we can DEMAND discussing EVERY application, which was unjustly trashed by the assigned reviewers. Force these %#!@(%& to re-book their flights, at the very least! :)\n\nRegards,\nAnonymous

September 3, 2009

\nHi anonymous,\n\nYou sound like a true pioneer. I am all for your proposed solutions.\n\nI would also like to hear from the rest of people. Please, speak up your minds.\n\nThanks
Avatar of: anonymous poster

anonymous poster

Posts: 6

September 3, 2009

\nI wrote a response to this controversial article under the title "Grievance Agenda of Dr Costello".\n\nI am following-up. First, I applaud Dr Costello for his rebuttal. This is in keeping with sound scientific reasoning and argument. However, I still feel he has serious grievances that are misplaced and flawed in principle. Second, there is an issue that has concerned me deeply as a new investigator (not necessarily a young investigator, though that definition tends to age like a cellared wine such that a young investigator is <45 years these days). It is true to say the NIH grantsmanship is a game. It is played by senior investigators extremely well who know the game like a seasoned quarterback. The rookie young/new/early stage investigator is like a draft pick quarterpack who is still learning the game.\n\nSenior investigators know full well that the R01 application procedure and review is a farce. Any R01 can be trashed for any subjective reasons the peer-reviewer wishes - be it point-scoring, suppressing new ideas, or deliberate destruction of someone;s career so that their's may flourish in the absence of any competition (particularly where there is a difference in scientific reasoning or dogma). \n\nLet me explain. An applicant, sets out a series of specific aims (hypothesis driven) that are set up to succeed - the applicant is enouraged to even guess the outcome of such experiments described in each specific aim under the heading of Anticipated Results and Potential Pitfalls. This is where the senior investigator has a distinct advantage. A new investigator is largely coming to terms with his/her research program and pilot data - to contemplate guessing what the outcome may be is akin to alchemy and black-magic. To guess what may be expected from hypothesis-driven experiments is extremely difficult and a potential hotbed for critique and grant rejection. A Catch-22 and the reviewers know it.\n\nHowever, there is a much bigger issue at stake. Many R01s that I have been involved in as a junior investigator, or post-doc rarely complete the aims described therein. There must be penalties for the lead PI for failing to complete an R01 according to the specific aims described. \n\nAlso, what is said in the grant as planned experiments are rarely if ever carried out in reality. What is done in the lab on having that R01 funded are elements of that R01 together with a revised structured thinking of the PI to conduct experiments that were not even defined in the original grant application. The R01 is in essence out of date from the time it is submitted ot the time the check is in the post. This has to change.\n\nMoreover, on every R01 that I have seen funded in labs where I have worked, side-projects have resulted. Projects that were not specifically funded by a particular R01 were sprouting like new seedlings as a result fo this fresh input of funds. There has to be greater scrutiny and monitoring of R01 funded research. Clearly, much of this can be concealed in carefully crafted R01 grant applications - another advantage of the senior and seasoned PI weho has learned how to do this in competitive renewals. Otherwise I think it encourages malpractice and misappropriation of funds for other projects not strictly supported as intended in the parent R01.\n\nThe current model of peer-review of all R01 grants allows junior investigators to be assessed in direct competition with senior investigators. The paylines do little to help the new investigator except perhaps increase the chances of that R01 being approved for final funding. I think greater emphasis needs to be placed on completion of R01s by senior and junior investigators precisely as set out in their original R01. It is inappropriate to use any funded R01 as a means to flesh out other undiscloseded projects not described in a parent R01 application. This is not a separate issue, but part of the same issue partitioning a greater advantage to the more senior investigator. \n
Avatar of: anonymous poster

anonymous poster

Posts: 1

September 3, 2009

Dr. Costello: I read and re-read your piece and arrived at the same conclusion, that you are misguided by cognitive distortions. First, it was troubling to read that you equated being a young investigator with diminished science. I guess according to you the reverse is true, that all Senior Scientist produce science of high quality. Are you serious? Do you really believe this? It is reflective of illogical thinking. I understand that you have been well funded through out your career some 22.million, so I fail to see how you have suffered in any way. It is a fact, that senior investigators have strategically monopolized study sections, and competing renewals resulting in very little funding for up and comming scientist. It is a professional insult to suggest the NIH is discriminating against senior investigators, you have got to be kidding me. What is sad is that you really believe this. New Investigators are held to a higher standard, for the preliminary work and publications. I have seen seasoned investigators get away with methodological issues that a New Investigator would never get past a study section. You decry the personal attacks and yet, that is exactly what you did by suggesting New Investigators produce poor science. I am disappointed by your commentary and the strategic institutionalized self promotion that so many like you in your generation have enjoyed, and because the NIH in its wisdom wants to level the funding field, all you have to offer is "funding new investigators diminishes science. Your words speak for themselves.
Avatar of: anonymous poster

anonymous poster

Posts: 3

September 3, 2009

In addition to making very broad generalizations regarding the training of students in biomedical graduate programs, the author also fails to acknowledge or mention how a "good old boys" network might operate in and accordingly bias funding of RO1s.\n\nI'm surprised that this article was published by The Scientist and even more surprised that a scientist would admit to writing it.\n\nFemale ESI in California\n\n
Avatar of: anonymous poster

anonymous poster

Posts: 3

September 3, 2009

Dr. Costello cites some very impressive numbers for his funding history. I'd be interested in learning how this has translated into published articles?? \n\nUntil we make use of quantitative data it seems a waste of time to try and argue about the quality of science being funded by anyone.
Avatar of: anonymous poster

anonymous poster

Posts: 23

September 3, 2009

Dr. Costello is deadly wrong on this issue\nby anonymous poster\n\nI find myself in a complete agreement with the person who wrote the 'troubling mind' comment. Also see this earlier comment. \n___________________________________\n\n[Comment posted 2009-09-03 12:04:27]\nYes, some jr faculty may not be doing the best science YET as those sr colleagues. However, there is one biological law we all have to follow: the old ones, including myself, will retire soon or worse die. Often, the best science in most sr faculty's labs are conducted by very young students and postdocs, not by the PI him/herself. Without supporting young investigators today, the best science of tomorrow will just not come freely. So, please keep this long-term goal in mind when voicing your immediate concerns about federal funding.
Avatar of: anonymous poster

anonymous poster

Posts: 23

September 3, 2009

I was about ready to quit this page but could not help write this response to Dr. Costello's reply (see below).\n\nThe key word he used here is 'discrimination'. I do see your point if you are denied funding for a score slightly below a young investigator's proposal. However, as many others have pointed out, the play field is level for jr scientists. You ought to acknowledge this as a form of discrimination. \n\nPlus, the judgement of the best sciences done by the best scientists are more or less arbitrary if you are not one of the top top 5% scientists in the nation. Often young scientists are discriminated against in this 'judgement'. You ought to acknowledge this as a form of discrimination. \n\nFinally, you need to make it clear that your science may be among the 'best', but is it all your talents and efforts? Unless you still work each day in the lab and you gather all your data, most sr colleagues rely on highly talented and motivated jrs (grads or postdocs) to do the lab work. It is nothing wrong with this. But would you want to give the credit to these truly 'best' scientists instead of yourself? Hence, You ought to acknowledge this as a form of discrimination. \n\n\nCopied from Dr. Costello's reply:\n\nNo matter how one wishes to ?slice it? or to ?spin it? or to ?justify it? or to ?divert attention from it?, the new requirements are discriminatory; and I do not support such discrimination! It has no legitimate role in the RO1 review process. It is a fact that it will result in some of the best science being unfunded to support lesser quality science; and that should not be an outcome of the RO1 grant support mechanism. Justifying it on the basis that it will affect only a small number of grants is obnoxious. It is incompatible with the stated R01 goal ?to fund the best science by the best scientists.? It violates CSR?s own requirement ?...to see that NIH grant applications receive fair reviews -- free from inappropriate influences -- so NIH can fund the most promising research.?

September 3, 2009

Dear anonymous,\n\nI?ve read the voices that have followed your suggestions on litigation as a tool to address effectively the serious problems that many of us have faced. Hopefully, people will continue to express their views. I have no experience on litigation and should be (I feel) the very last resort to use when there is breach of civil or scientific rights and responsibilities. Litigation is also lengthy, with heavy cost of psychological and economic resources. \n\nRigorous and fair review are both a right and responsibility for applicants and reviewers and NIH should be their ultimate and proven guarantor. The present status of science would require, in my view, an effort in creativity and common work to solve the problems. I hope that Dr Collins is reading the sentiments of the community. This, in my view, is a wake-up call for NIH Institutes to work seriously and unselfishly on the problem. \n\nThe investigator-initiated R01 needs to be vigorously reassessed and enhanced. \n\nThanks.\n\nR\n
Avatar of: anonymous poster

anonymous poster

Posts: 1

September 4, 2009

I would like to comment on "contemporary biomedical training programs fail to train young investigators to be scientists. They are trained to be myopic super-technologists". Dr Costello makes it sounds like a generality and I really don't agree with that. In that matter, I would like to remind Dr Costello that the main training young scientists receive is from their mentor (during a PhD or a postdoc) that is from senior, well established investigators. So if what you say is true (and I personally believe you're wrong), who is at fault for this bad training?
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 4, 2009

This commentary only perpetuates the fear of many of my junior colleagues that they do not receive fair NIH reviews from senior investigators who are reviewing their grants. Certainly, junior investigators do not have the stature, resources, or expertise to compete with senior scientists. My worry and experience is that the science is not always critiqued, but it is the researcher's idea that is critiqued. If the idea is not well-liked, regardless of the rigor of the science, it is devalued with an unethusiastic score.
Avatar of: anonymous poster

anonymous poster

Posts: 10

September 4, 2009

The current review system nay not be perfect. However, I would argue it works overall. Many people, including myself over the years, have complained about bad reviews. Serving on the study section changed my perspective. Let's be honest, an unscored grant means that three study section members agree that a grant is not competitive. Grant with huge differences in preliminary scores are often actually discussed in the meeting. Any study section member can pull a grant off the unscored list and have it discussed at the meeting. With a great SO, a study section can do a great job.\nI am writing this because some of the posters here made some highly inacurate and misleading statements about the NIH reviewing process. A study section meeting is not a bunch of good old boys meeting in a dark room to sabotage someone's career :-). \nFinally, I would encourage all junior faculties to volunteer to serve on the study section. Instead of complaining about the system, try to work in the system to make it better.
Avatar of: anonymous poster

anonymous poster

Posts: 18

September 4, 2009

I think this piece was wonderful for stimulating some excellent discussion (and a few not so insightful). Lost in this is what I truly call the "big" issue. I am a very senior investigator who at the height of my grant productivity had five funded Federal projects. I gave up two of them because I felt I was losing contact with my work. The other three RO1s were actually of modest size, but gave me a decent lab for many years -- a lab whose research I felt truly involved with. The real problem I see if that empires were allowed to be built. That the empires attract bright young talent is true, but it is as much because that is where the money and prestige are as that it is the best way to help train the next generation of scientists or allow them to be independent. I wonder how much research misconduct is missed because PIs are too detached from the nitty-gritty aspects of their research. For all of its merits, I think the NIH Roadmap has had a big downside in encouraging this. Costello's airing of his thoughts is useful in that it sparks discussion of the whole enterprise.
Avatar of: anonymous poster

anonymous poster

Posts: 20

September 4, 2009

You say:\n---------------\nFinally, I would encourage all junior faculties to volunteer to serve on the study section. Instead of complaining about the system, try to work in the system to make it better. \n---------------\n\nBut then who would these people have to complain about why their grant is rejected? It seems, from reading these comments, that it is much easier to blame a cabal of senior investigators when you don't get your grant than it is to take a good look in the mirror and perhaps see a scientist whose ideas are second rate. \n\nThere are a lot decrying the commentary, but its idea is straightforward and correct. Young instigators do not have to get as high a score as more senior investigators in order to receive a grant. That is, by any measure, discriminatory. Now you may say that discrimination is justified, or that there are other discriminatory practises in the process and therefore it does not hurt (and may help) to add another. But to those who say it isn't discriminatory, you are obviously wrong.\n\nFrom a young investigator.

September 4, 2009

\n\n\nDear anonymous,\n\nI agree that it should not be a game. Except that that is the advice that some established investigators haven given new investigators:"You have to play the grant game".\n\nI would agree with your statement "an unscored grant means that three study section members agree that a grant is not competitive". The question is how those three members came to agree with that assessment. I also agree that a Study Section is not to sabotage anyone's career but it should also be said that the opinion of the primary reviewer, for various reasons, was much more frequently than not the final word.\n\nI absolutely agree with you that junior investigators should volunteer to serve on study sections. Perhaps with the airing of the problems, once they have served one time, they will be allowed and encouraged to continue to serve.\n\nThanks
Avatar of: anonymous poster

anonymous poster

Posts: 4

September 4, 2009

""an unscored grant means that three study section members agree that a grant is not competitive"\n\nWhat should we do if this "agreement" is backed by a summary statement full of nonsense? Not scientific differences, but just nonsense, self contradictions, or ignorance of facts which should be known to undergraduate students? Should I still consider such critiques valid, because "three reviewers agreed"?\n
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 4, 2009

I worry that this type of dialogue creates divisive feelings between junior and senior faculty when we should be working toward a common goal: how can we improve the NIH review system so that it works well for all investigators at all levels to promote the best science....

September 5, 2009

\nDear anonymous,\n\nThanks for bringing the issue of divisiveness as a wise alert. I think that the issue could also be a trap leading to inertia. The Spanish say goes: A río revuelto, ganancia de pescadores and translates: There is good fishing in troubled waters.\n\nI agree with the posters who have commended Professor Costello for his openness and audacity to present his personal view and experience on issues of great importance to the community. I am sure that he knew he was going to get a response with multiple faces and yet he took the lead and did it. In my view, that?s part of what science is all about: exposing oneself to the scrutiny of other views and, in the process, advance the field.\n\nWe all assume good faith and willingness to avoid stagnation when promoting community discussion and we all know that the greatest breakthroughs in the history of sciences have come after periods of quiescence and ?apparent consensus?.\n\nMoments of heat in the arguments could be also interpreted as an opportunity for nearing personal interests to the interest of common good. \n\nJunior and senior investigators: if we care about science, we all have an obligation to support each other. \n\nThanks again for the alert.
Avatar of: tom hennessy

tom hennessy

Posts: 2

September 5, 2009

It seems maybe someone has noticed the HIGH INCIDENCE of those researchers who merely apply for grants BECAUSE those grants can **guarantee** a five to ten year research study.\nTHEY .. in which .. are TOO now guaranteed a cozy little life.\nAs opposed to those researchers who live on bread and crumbs FOR their research.\n

September 6, 2009

\nHello Tom,\n\n\nNobody speaking in good faith is suggesting to penalize senior investigators for the sake of penalizing. Senior and junior researchers are expressing their views, which is what **those researchers who merely apply for grants BECAUSE those grants can **guarantee** a five to ten year research study ** do not want to happen. They are upset because there is discussion and freedom of speech.\n\nThere are excellent senior investigators who have been penalized because some of their highly funded established senior colleagues and cooperative highly paid administrators, with many privileges other than leading a cozy life, are being given the funds that creative seniors and juniors deserve.\n\nThere is nothing wrong with eating **bread and crumbs** if one chooses to do so. By the way, I eat bread and butter every morning because I like it even though I have the option to eat much more than that.\n\nThank you\n\n\n
Avatar of: TS Raman

TS Raman

Posts: 31

September 6, 2009

My comment is specifically on Professor Costello's opinion, "A major factor is that contemporary biomedical training programs fail to train young investigators to be scientists. They are trained to be myopic super-technologists, predominantly in areas of molecular biology and molecular technology...[etc.]" \n\nI agree, and add the following quote/paraphrase (from my scrapbook): "The DNA Disease: ... some of the gene products derived from recombinant DNA experiments commanded a high market value. the gene which was enhanced was that for venality and the unanticipated gene product was money. The disease proved to be quite contagious ... With great eagerness, everyone wanted a piece of the action. ... those affected [by the disease] simply will not tell their friends and neighbours what they are doing or what they intend to do. ... Concurrently a change was noted in the dissertation problems assigned to graduate students. Whereas these formerly always contained an element of new knowledge, now the stress is more towards a new product and preferably a marketable one. The student may not be assured of an excellent graduate education, but he would be guaranteed a well-paid job at the end of the road. ... Science gives way to technology, research to development, and publication to patent. Lip service, of course, is still paid to basic research, but clearly the product is the thing. ... Having allowed the venality gene to escape, we cannot foresee how the matter will ever be put right again. [DeWitt Stetten Jr., Department of Health and Human Services, NIH, Bethesda, Maryland, USA. Correspondence in Nature (27 May, 1982) 297: 260]" \n\nI have repeatedly seen manifestation of this phenomenon. Just two instances out of numerous: (1) everyone in a batch of 5 graduate students taking their "qualifying exams" could give "erudite" lectures on, say, RNA polymerases of E.coli, but none could tell me what "logarithm" was; and (2) All the bright new scientists including several of my colleagues, who insisted on calling themselves "molecular biologists" and not just ordinary biochemists, who were using polyacrylamide gel electrophoresis as a tool were polymerising the gel by exposing the columns to UV light, although the gel contained ammonium persulphate, not riboflavin as in Davis's original method. \nAs Erwin Chargaff said: "[ERWIN CHARGAFF, talking of his meeting with WATSON and CRICK; New Scientist, (17 August 1978): 484] “I was faced with a novelty: enormous ambition and aggressiveness, coupled with an almost complete ignorance, and a contempt for chemistry, that most ideal of exact sciences ─ a contempt that was later to have a nefarious influence on the development of 'molecular biology'." \nThe "nefarious influence" is very much in evidence.
Avatar of: anonymous poster

anonymous poster

Posts: 1

September 8, 2009

I must second many of the comments of younger scientists to your ridiculous proposition - basically that older scientists are smarter than younger ones, and that explains the need for "affirmative action".\n\nWhat nonsense! - Having a prior R-01 is a HUGE, HUGE advantage when it comes to preparing a grant application. The R-01 payline mechanism was developed precisely because the R-29 mechanism was NOT effective in getting new investigators going independently (max grant was MUCH smaller than R-01). Would Dr. Costello be somehow happier if the payline mechanism fed into an equivalent size grant with some name other than R-01? By any name, it MUST start getting even a fraction of the money that older scientists get into the hands of younger investigators. \n\nThis complaint about "age discrimination" must mean that at last NIH is proposing a mechanism with some teeth - we'll see if it stands...
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 24, 2009

As a program official at the NIH, I had to shake my head at Dr. Costello's article.\n\nI weigh in today in light of a recent article on the new-PI advantage in the NY Times (9/22), and because of some previous comments here.\n\nFrom where I sit, having audited grant reviews in many study sections for years, I absolutely concur with three main criticisms in previous comments.\n\nFirst, new PIs and big-name established PIs are given vastly different benefit of the doubt whenever there is any question about experimental approach--- namely when a methods detail is not including in a research plan. A "sin of omission" in a methods detal voiced by a section member on a methods detail can easily ding a score down from elite to really good. Conversely, I saw a competitive renewal application that used only 16 pages to describe TWO DOZEN new experiments get an airtight, slam-dunk score, after a couple reviewers agreed that "Yeah, there's not much detail there, but this is Dr.X, so we know they'll get done." Just try that young Asst. Professor Softmoney!\n\nI fear for new PIs once R01 research plans go down to 12 pages for the Jan 2010 due dates and beyond....\n\nSecond, the difference in scientific merit between the different degrees of excellence is slight and completely subjective. At least at my institute, I cannot think of any new-PI applications that did not have a score that still indicated significant scientific merit (and only trivial flaws).\n\nThird, I also believe that the funding climate is VASTLY more difficult for a newish investigator at today's paylines! At the paylines of today, we in program are forced to make Solomon-like decisions between excellent and compelling applications in our recommendations up the chain.\n\nMoreover, I take great issue with a previous comment about the ostensible futility of interacting with a program official at the institute to which one is applying for funding-- under the rationale that we're biased and have an agenda for the institute. Not only do I try to help PIs hone applications and consider study sections strategically, I also try to get applications honed with an eye toward what my Institute and its leadership would find compelling. Remember, as representatives of the NIH, we're ultimately the customer, and the PI is providing the goods. It only makes sense to ask the customer what he/she is looking for.\n\nNew-PIs: The time to consult with a program official is BEFORE you submit, not just after your application is scored (or not) and you wonder what it means.\n\nFinally, the complaining about how appeals are not taken seriously is also off-target. Appealing a review is a SERIOUS matter, because it essentially is an accusation that NIH review officials failed in their duty. Not only is that a charge not to be made lightly, PIs fail to understand that differences of scientific opinion does not necessarily constitute a bias that compromises the integrity of a review. Historically, appeals fare poorly because they are often seen as the sour grapes they are.\n\nIn sum, yes, by axiom experienced PIs with their slightly higher scores will lose out over a new PI occasionally in the zero-sum contest between them. In that, Dr. Costello and other critics of the NIH policy are absolutely correct. To say that substandard science is being funded instead is a gross exaggeration.\n\nIronically, the investigators most at risk now are mid-career PIs, who neither get the "Dr. Big" reputation aura in review, nor the New Investigator love.
Avatar of: anonymous poster

anonymous poster

Posts: 2

September 24, 2009

I meant to say that in my neck of the woods, there have been no new-investigator applications SERIOUSLY CONSIDERED FOR FUNDING that did not have very good scores in their own right....
Avatar of: anonymous poster

anonymous poster

Posts: 4

September 25, 2009

Again I will reiterate that I would have loved to write and compete for grants when Dr. Costello starting applying for grants - approximately 50% of grants were funded at that time!!\n\nhttp://books.nap.edu/openbook.php?record_id=9928&page=14\n\nImagine that - but I am sure he will stick to the tried and true - when I did it I had to walk 12 miles through the snow barefoot - style of recollection.\n\nDr. Gregory Petsko has recently written a commentary that is much more accurate regarding the plight of the young investigator. Dr. Costello would do himself some good to read this article.\n\nhttp://genomebiology.com/2009/10/7/109\n\n\n\n\n
Avatar of: anonymous poster

anonymous poster

Posts: 1

October 6, 2009

Costello says that he "does not support discrimination." Surely he does. Grants are discriminated based on quality of ideas, track records of investigators, feasibility, and so on. So clearly, discrimination can be good. The question is whether discriminating new investigators to give them a slight bump is good or bad type of discrimination. Most think that this is a good discrimination and actually benefits science greatly. Currently, the odds are very much against the new investigator, unlike when Costello was a new investigator himself (when the payline was much better). The age when the first award is received has been steadily climbing. IIRC, from 35 or so it has climbed to 42. Because of the need for extensive preliminary data and track record, the deck is stacked agist the young investigator. Bright people generally do not want to be postdocs into their 40s, and hence leave science. (And the argument that "young people are losing because they are not smart or well-trained anymore, unlike when Costello was young" does not hold water under any objective analysis). NIH recognized the risk of losing a generation of excellent scientists due to this trend, and hence is attempting to redress the problem by giving a slight advantage to such people.\n\nOne can argue that by funding a proposal with a score of 142 instead of one with a score of 140, you are no longer doing the best science. But everyone knows that such small differences are very subjective and not very meaningful. The grant with the score of 142 is still excellent and meritorious and is money well spent. And if it prevents a bright scientist from leaving the field and serving science for decades instead, all the better. The argument that you are "no longer doing the best science" because it is based on only 99% meritocracy and not 100% meritocracy becomes weak when you look at the quality of grants being funded.\n\nThis is somewhat analogous to giving blacks a slight preference in admission to a university. Although this takes you away from 100% meritocracy and you are discriminating based on race, it is actually good for the society and a good ideas given how the odds were against blacks for hundreds of years. Believe it or not, some discrimination CAN be good.
Avatar of: anonymous poster

anonymous poster

Posts: 1

November 9, 2009

An interesting comment thread has started in response to the rebuttal of this article in a later issue of the "The Scientist" vol 23/11 by Schaffer and Rocky.\n\nAt issue is further administrative actions decreasing input by peer review in judgments as to what constitutes the "best science". \n\nThe issue appears to be the intended and unintended consequences of decreases in the paylines for A1 and A2 revisions relative to new A0 submissions that were put in place by the October 09 NHLBI council.
Avatar of: anonymous poster

anonymous poster

Posts: 5

January 6, 2010

After reading the article and the comments from the readers, I can say that I am happier than ever with my decision of leaving academic biomedical research. The article and the comments reassure me that I took the right decision. Wasting most of my time fighting for the improbable and remote? Not anymore?thanks?\n\nThe NIH grant system is completely broken, but nobody seems to realize that it is broken at its core, discussing issues like the one in the article is futile. There are more serious issues, for example, the NIH score system is useless, nobody sees this? The current scoring works when you have a normal distribution of scores, but now all the scores are packed at the left of the distribution, with a very steep slope, so minimal changes have tremendous impact. Is someone analyzing the statistics of this? And the author still claims that couple of points in the score can differentiate the best from the bad science? Does he knows statistics? What a joke! And these are just a few examples?.\n
Avatar of: anonymous poster

anonymous poster

Posts: 1

January 6, 2010

I agree completely. As an RO1 researcher for 30 years, publishing over 300 articles and serving on study sections, I would only add that the SRAs need to expend tremendous energy on who they pick to serve on study sections. I have seen many more cliques and buddy systems, and lack of broad understanding by primary and secondary reviewers in recent years. Some reviewers just don't read the grants they are assigned well at all, and they should not be invited back. I am 60 years old and still have 2 NIH RO1 grants, but I worry that I can't compete in the potential train wreck that will happen this year when the stimulus money ends. Between the preference for younger faculty grants and funding pet areas for HUGE clinical centers with second rate research such as the SCORE and others, I worry about basic science, from whence so much translational research comes. It's like we are moving toward Sesame Street science--only fund that which promises clinical treatment very soon. There needs to be a balance. There are a lot of baby boomers like myself who do not yet want to go to pasture, and we have the broadness that I believe is still very much needed to review grants. Balance!

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
Advertisement
Mettler Toledo
BD Biosciences
BD Biosciences