Messages posted by: eveTS132260

I think this is completely a non-issue. It has absolutely nothing to do with the new format favoring one group of scientists over another on the basis of age or experience, but rather it has everything to do with resistance to change. Can anyone name ONE CHANGE in the peer review system that NIH has instituted that has not resulted in howls of protest from the scientific community? Review criteria, study section organization, funding flexibility with respect to percentile scores, emphasis on novelty of ideas as a criterion, etc. etc. etc. -- everything elicits howls. It's not the nature or substance of the change per se, it's just the fact of the change, that creates anxiety.

Let's give it a try. NSF has been using ten-page applications (with one inch margins and minimum font sizes) for decades. Maybe twelve pages with minimal margins will work for NIH. Just give it a try, folks, and stop coming up with creative yet fallacious "reasons" for not doing it.

Strange article. Why didn't the writer/poster contact the scientists who are familiar with those "Bush" lines and ASK THEM ?

If we attempt to define "high risk high reward" science, in terms any less broad than the NSB definition, we will immediately constrain it and render it no longer "HRHR." So I would argue strongly against the premise that the NSB definition is overly broad. The breadth of the definition is to my mind the important key to the hard job of recognizing risky, potentially transformative research. After all, THAT'S THE POINT, REALLY -- we won't know for 100% sure what's paradigm-shattering, or what is so novel or of such unexpected significance that it will make a profound change in how we think or what we do until AFTER THE FACT -- after the idea is explored, after the experiments are done, after the work has withstood rigorous. testing Trying to limit the definition a priori will only serve to limit the downstream potential possibilities. How can you constrain serendipity??

As for young investigators -- that's a chin-rubbing notion. On the one hand, young investigators are intrinsically risky not necessarily because of their ideas but because of their lack of track record. In other words, the risk is not in the project but in the investigator himself or herself. Literally, we don't know until after an investigator has been working independently for ten years or so whether or not he or she is any good. Is that a reason for not providing support? I think not. I think that's a reason for supporting lots of these young investigators, to ensure a pipeline of creative and productive scientists for the future. In other words, supporting promising young investigators, giving them the opportunity to "play in the lab" (I mean that literally, and I consider it a good thing to do) and gain the experience in both depth and breadth that they will need in order to make the great contributions of the future, is the only way to get great contributions in the future. That's the reward. The hard part is figuring out which "promising" young scientists will pan out in the long run -- and there's the risk. Yes, some of the promising young investigators we support will drop out of the system eventually for one reason or another. Nobody can predict with 100% accuracy which ones will and which ones won't hang in there in the long run. That's why we need to support as many as we can, to ensure the pipeline.

The problem with "high risk high reward" projects (including young investigators), which is the thing that bothers everyone, is that you can't predict outcomes (by definition -- if you can predict the outcome, it ain't High Risk). Therefore, conservative souls tend to shy away from such things instinctively. But we need to support risky science, in order to reap potential rewards, and so it is necessary to do the hard thing -- take the risks, acknowledging up front that not every "high risk" project, and not every promising young investigator, can possibly pan out in the end, but recognizing that even if one in five, or even if one in ten, DO pan out, then the investment in all of them will have been worth it.

Amen.

Perhaps what we need is a clear distinction between "immediate outcomes" research and "new discovery" research. By "new discovery" I don't mean running a gazillion assays through a machine to find a chemical that behaves in a specific way, but rather new concepts or new ways of doing things or thinking about things. Sometimes such "new discovery" research takes longer than you might think, and the discovery process takes you down paths that twist and turn in unexpected ways, and the paths may even branch, or seem to branch, and you don't have the resources to go down both paths at once so you flip a coin and choose, never knowing where that other path might have led. If this sounds poetical, or existential, well, yes, it is. But think of where we would be today, in terms of scientific knowledge, if scientists hadn't taken such existential, or poetical, or off-the-beaten-path risks. We wouldn't know very much, would we, if all that scientists ever did was work within the confines of existing knowledge -- clearly, the sun and moon both revolve around the earth (we can see that with our own eyes, right?). Yet, even when we thought the sun moved around the earth, we were able to use the information we had at the time to calculate the seasonal changes, the time of day, etc. etc. -- practical, useful knowledge based on filling in obvious knowledge gaps. Nothing wrong with being able to plan a planting season. And you don't need to know about black holes and distant galaxies to do that. That's important -- which is why incremental research is important for the short term. But we do need to know about distant galaxies and black holes (well, I think we do -- some may argue that point, I guess), which is why "meaningful" and even "frivolous" research is important and should not be shut out of the process by establishing "rules" that eliminate such projects from consideration for support.

To Christopher: I don't know what you mean by "frivolous research," nor do I understand how anyone can do "meaningful research" in the manner you describe. Some might even question, if a project can be KNOWN to be completed exactly as described in a proposal in a given amount of time then why bother doing it? (or worse, one might even suspect that the investigator has actually already done the work). Such "guaranteed outcome" research could not possibly stretch the envelope, or uncover significantly new knowledge or understanding. What you are describing is what some call "grind and find" work, providing some incremental progress and filling-in some highly specific knowledge gap. Don't misunderstand -- this kind of work is extremely important for applied research and for translational research. But the OTHER kind of research -- the kind you deplore as "frivolous" -- provides the breakthroughs that lay new ground, open new ideas, new ways of thinking about things, and is the root of real progress for the long term as opposed to incremental progress for the short and intermediate terms. That, to my mind, is "meaningful."

Fred -- thank you for your comments. I agree 100%.

BTW, even at NIH, the "career program officers" are, for the most part, themselves reputable published scientists with expertise in the research area for which they are responsible. This was not necessarily the case 20 or 30 years ago, but it definitely is the case nowadays. And yes, they can and do see a bigger "picture" than can be seen by most members of study sections or individual PIs. AND -- they don't act capriciously or in a vacuum; any "stretch" that they want to make has to be justified in writing and approved by higher-ups.

Again, thank you for your comments.

Beverly -- I certainly respect discretion. Thank you. But I do wish to add that a few anecdotes do not a pattern prove. Even a few genuinely bad actors do not prove that all actors are bad, or even the vast majority. And misinterpretation always remains a possibility whenever human beings are involved (heck, everything remains a possibility whenever humans are involved).

Beverly -- I certainly would not equate "getting to know your Program Director" with "cronyism." At NSF, would-be PIs are strongly encouraged to contact their Program Directors -- generally by telephone, but also in person. I don't know if it's still recommended on the NSF website but it certainly used to be, and before the website era, it used to be encouraged in official brochures. Sometimes Universities send delegations of would-be PIs to the DC area, often in the company of a university outreach official, to meet with program officers at NSF, NIH, and whatever other agencies might be appropriate for the science involved. I often met with such groups of scientists (usually Assistant Profs but not always), sometimes by myself and sometimes in a group-grope with other Program Officers, depending on the circumstances, and I think it was beneficial to the scientists to learn what the agency was looking for and what kinds of criteria the agency used to evaluate proposed projects during the decision-making process. Of course, the scientists would do a show-and-tell of their work as part of the meeting, and sometimes program officers would offer advice and suggestions. WE WOULD DO THAT FOR ANYONE WHO ASKED, so long as we were not asked directly to critique a draft proposal (if we did, we would have to recuse ourselves from the review and decision-making process because the proposal would in that case obviously be in part our own work). The scientists would leave with their impressions of the program directors they met and of the agency as a whole, and the program directors would be left with their own impressions of the scientists -- both positive and negative, of course. This getting-to-know-you process was always considered a desirable (albeit timeconsuming!!!) part of the outreach process. Similarly, several Federal agencies routinely rent exhibit booth space and/or arrange for alternative modes of interaction between program officers and scientists at major scientific meetings -- the purpose being not only to peddle brochures and answer questions, but also to put "a human face" on the cast of characters. All of these outreach activities do not amount to "cronyism." And keep in mind that a brown-nosing scientist would be equally likely to make a good impression as a bad one.
In the case of currently-funded PIs, it was critical for program officers to be in touch in order to learn what the PI is doing, and if the PI is doing something really exciting, to be able to use this information within the agency for purposes of programmatic progress reviews and, if it's really really exciting work, to brag about it in reports to OMB and Congress as a rationale for requesting more funds.
And as for "wining and dining" -- the implication here being bribery -- wow, uh-uh, no-no-no, that's a bigtime Federal no-no with dire consequences for any Federal employee (whether permanent or temporary) who is caught accepting "wining and dining" or any other legally inappropriate gifts.

I realize that you genuinely believe the worst-case scenario you described, and I don't know exactly how to dissuade you of your belief, but I genuinely believe that you have misinterpreted something along the way.

Let's be clear about those "Paylines" -- they're based on numerical scores that are in term based on the personal opinions of one, two or at best three study-section members. Neither the numerical scores nor the "paylines" represent an objective assessment of either the quality or the importance of the research. The issue of "cronyism" raised by some of the commentors in this forum applies far more often to study section members than it does to NIH staff (or NSF staff! -- disclosure: I used to be an NSF program officer myself, and I deeply resent the insinuation that either I or any of the wonderful, dedicated colleagues that I worked with while I was there would award grants based on "cronyism;" in fact, conflict of interest is taken very seriously at NSF at all levels). The problem with scores and paylines is that study section members are notoriously and documentably averse to new ideas, ideas that conflict with their own beliefs or with the dogma in their fields, and to newcomers. As a result, the applications that get the best scores tend to be the ones that propose experiments that will incrementally fill existing knowledge gaps in currently-hot areas of research that are well-explored by other scientists. Yes, it is important to incrementally fill existing knowledge gaps. However, it is also very very important to push the envelope of knowledge in new directions, possibly even potentially paradigm-shattering directions, or in directions that will open up not only new "facts" but also new ways about thinking about the problem and new ways of thinking about previously-uncovered facts. Some may argue that it is even MORE important to push the envelope in this way than to systematically and incrementally fill in gaps. Not all scientists have the creative ability to push the envelope of science, and I would even venture to say that most scientists do not even appreciate the envelope-pushing work until after the "new" has become the "new hot thing" and everybody else is jumping on the new bandwagon. But how can envelope-pushing get to the point of "bandwagon" if the work isn't being supported? And THAT, in my humble opinion, is why it is imperative that wise grant program officers look carefully not just at the numerical scores but also the texts of the reviews, and use their unbiased judgment to select some (20% may not be nearly enough) projects that may (or may not -- that's the definition of "risk") lead to real breakthroughs instead of just filling in knowledge gaps. That's what program officers try to do at DARPA, and to a lesser extent (unfortunately) at NSF, and to a much lesser extent (even more unfortunately) at NIH. I am pleased to learn that program officers at NIH are "reaching" more and "covering their asses (via priority scores)" less than they used to. They should do so more.

From where I've been sitting, it's been my observation that for several decades now, too much NIH funding has gone to projects with "premature relevance" -- i.e., projects that leap from a tiny undeveloped new idea or discovery directly to disease related research. Most of such projects have been wastes of precious resources of time, labor and money. We need MORE fundamental information, not less, before plunging into relevant and intelligently-designed translational research that holds the possibility to cure or prevent or ameliorate diseases.

Weisbach simply doesn't understand science. Or biomedicine. (and the two are related but definitely not exactly the same thing).