Advertisement

Q&A: Overhaul the Funding System

Can science step in to find the best ways of allocating money for research?

By | September 28, 2011

FLICKR, THOMAS HAWK

It's a perennial complaint among scientists: grant writing takes up too much time. “I think that there's pretty much wide agreement that scientists have to spend a lot of time to write proposals, to review proposals, to write progress reports, final reports, and do lots of things that are not necessarily contributing to science,” said John Ioannidis, a professor at Stanford University's School of Medicine.

But it isn't just time that's spent unwisely, but billions and billions of dollars that could be allocated in smarter ways, Ioannidis wrote in a comment in today's Nature. The Scientist spoke with Ioannidis about his ideas to fix science funding in the United States.

The Scientist: What are the current problems with the way science is funded in America?

John Ioannidis:I think that the way that the funding system works, people have to promise something that is exaggerated. They have to compete against others who are promising something that would be spectacular, and therefore they need to make a huge claim that often they may not be able to deliver, just because of the odds of science.

Or they have to promise something that would be highly predictable. It's something that they may have already done or that they know what the answer is going to be...So in a sense I think the current system is almost destroying the potential for innovative ideas and in many cases it even fosters mediocrity.

TS: What's the best way to improve upon the current funding paradigm?

JI: What I advocated in that comment was we need to have some studies to directly test, ideally in a randomized fashion, whether one strategy performs better than another. One could think about pilot projects where you have consenting scientists who say I'm willing to be randomized to funding scheme A versus funding scheme B. A could be lottery allocation, for example, and B could be merit-based. If we run these types of studies, in a few years we can see what these scientists have done in the short term. Maybe in the longer term we can also see whether their research really made a difference. I'm in favor of experimenting, because we're spending billions and billions of dollars and we're doing that with no good evidence really.

John Ioannidis, Stanford University School of Medicine
John Ioannidis, Stanford University School of Medicine
JOHN IOANNIDIS

TS: In your proposal for funding scientists according to merit, you mention independent indices to measure a proposal's worth. What are some examples?

JI: I think that publications and citations should not be undervalued. They could play a very important role in that assessment. But maybe combining indices and paying attention to quality aspects other than quantity could be informative. One could think of merging indices that exclude self-citation or take into account the impact of papers rather than the amount of papers and adjust for co-authorship. There are indices to do that, and I think they can be objective if you combine them.

On top of this we have the opportunity to build additional information into the profile of an investigator based on scientific citizenship practices. Things like sharing of data or protocols, and how often these data and protocols are utilized by other researchers, how much do they contribute to other aspects of improving science.

I think it's an opportunity: if we really feel that some practices are essential for science, and we want to improve these practices, then tying them to the funding mechanism is really the prime way to make them happen.

TS: In this kind of scheme is seems like weight might be given to more established scientists. What are ways to help younger faculty who don't have such a track record?

JI: I would think that this is actually a problem of the current system, rather than any proposal to create something new. The average age for an investigator to get his first RO1 currently in the states is about 40 or 41 years old. People are going through 15 years of being active researchers, and they still don't have independence. So I think that a system that is based on merit could really get younger scientists on board much faster than the current system.

Moreover, for example, if you use citation indices, you can always adjust for the number of years that someone has been active. And there's some evidence that the track record of some individuals could be identified from fairly early in their careers.

I would also think that for young investigators it's probably okay to move closer to the possibility of an egalitarian sharing, so give some opportunity to lots of people early on for a few years and see what they can do. They will build a track record within a few years, and then you can start having some impact measures or citizenship practice measures.

TS: How do we balance the interests of funding sure-bet science or translational research versus very risky or basic science?

JI: This is not an easy question. I think people have a different perception of what would be the appropriate allocation between these two strategies. It has to be a common consensus in the scientific community of what we really want to do. Currently I think innovative research is undervalued and not given enough opportunities to succe.

J.P.A. Ioannidis, “Fund people not projects,” Nature, 477:529-31, 2011.

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: Greymattermom

Anonymous

September 29, 2011

It's unfortunate that all of the money has to come from the federal government. A position should come with a research budget, perhaps allocated by some measure of productivity, and additional funding for larger projects could come from NIH.  We could get back to judging our colleagues by the quality of their work rather than the size of their grant portfolio.

Avatar of: A reader

Anonymous

September 29, 2011

Given that about as many Americans now believe in UFO's as in evolution it's hard to have much confidence in the current system of funding science in this country. An alternative strategy would be to award more, smaller grants to greater numbers of PI's.  The current approach centralizes power and influence in science, making it difficult to sustain the diverse points of view that would promote a lively competition among ideas.  Such a change also would tend to increase the numbers of educational institutions that can support research programs that would presumably encourage the institutions to hire more science faculty.
 

Avatar of: Rlanning

Anonymous

September 29, 2011

take out the middleman (NIH). instead of for-profit science/healthcare companies sending tax dollars to DC to be filtered thru the reverse sewage system of politics, their taxes go to a non-government science foundation where funding is at the discretion of scientist/healthcare professionals. people's tax money that oppose certain initiatives would not be an issue that blocks research. 

Avatar of: Dnldedmnd

Anonymous

September 29, 2011

Rlanning,
It is a good idea, but you have to crawl through the political sewer before you can build the UTOPIA.  You need the cash and influence to first build the infrastructure. That means turning scientists into a political group, it does not have to be democrat or republican. 
Look at banks, they fund both republicans and democrats. When their business falls apart because of their own greed, corruption and stupidity, they don't collapse, they get a trillion dollars, with little opposition from either party. We live in a moneytocracy, you spend the money you got a voice. Money controls the issues that congress hears, and votes on. If the public average Joe vote on its own really made a difference, it would be illegal. 
Get the scientists together, undergrads, grads, doctors, PI's, and even industry - it benefits directly from public funding of basic research, and get them to ante up. Include in a research funding bill perks for biotech companies - such as tax breaks if they give their "public research dollars" back to universities to fund projects. Oh yes, you need to make the public research money available to them to for sociallly necessary research.
I could even write the bill, we all could.  We just need someone to ram it through the sewer.
- Don

Avatar of: oldscientist

Anonymous

September 29, 2011

the major problem in US science is not the method of allocation- it the ratio of $$/would be investigator.  However the money is split up there are either too few $$ or too many investigators.  We have trained too many people and have created an infrastructure in medical schools around the country that are fed exclusively by NIH. It is arguable (but not by me) that this selection by ability to get funded is fine, and that science should be like Russian literature- train a many people as wish to be trained and let most of them find an unrelated job when finished. 

The long training periods occur because that limits the number of would be PIs and helps the $$/would be PI ratio in the short term.

I personally think that study sections scould save a lot of effort by simply rating grants inthree catagories- absolutely must fund (no more than one/ss member/round) fundable (I would guess about 50% of the applications) and not fundable.  The NIH could X could fund the must fund grants (a couple per study section per meeting) and then randomly fund among the fundable proposals.  Never going to happen because it is politically indefensable but would probably have the same results in terms of research output as now.

Avatar of: Edgoodwin

Anonymous

September 29, 2011

The NIH has grown from nothing to $30 billion in the last 70 years and should be totally de-funded. The system is prone to gaming anad conflict of interest. It unfairly
awards most grants to the most connected researchers and universities who are more interested in advancing their careers than providing the public with usefull information. Then when they get caught in the act they fight the NIH determintions as in the recent OHRP determinations against Hebrew Rehab (Boston), University oof Maryland, and University of Washington (St Louis) (please see OHRP website). They like the grant money, but don't like to follow the rules attached to the grant.---Ed Goodwin  

Avatar of: Dnldedmnd

Anonymous

September 29, 2011

I remember an article back in about 1991 claiming that the aids virus did not intially infect hosts through T cells. The article contradicted the dogma of the day, and little else was said of the issue (my guess is just about any study contradicting the holy grails of the NIH will lose funding), until about 1999, when another lab made the discovery of the CxCr4 receptor on a type of dendritic cell that allowed the virus entry.
Is the bureacracy of the grant writing schemes a way to screen applicants, or is it a way to arbitrarily deny funding and foster favoritism? It would be very easy to give more research dollars to those who have theories that will confirm the dogma of the top researchers reviewing the grants, and deny those who do not agree, and really use the arbitrariness of the proposals to stifle competing theories - or worse, use the system to give more dollars to a group of "friends" of the grant scrutinizers. I am not suggesting that very large NIH grants are given to NIH country club members. There is likely no good solution to fix this system.

The only problem with the author's egalitarian solution, is that there is then less dollars are available for the older tried and true researchers, and more dollars being risked on the younger, unproven researchers. But, if the US government increased its budget for science research by an additional $50 billion, that would take care of the problem, then there would be enought money for everyone. Remeber the 90s when research dollars began to flow and all of the great discoveries and inventions that resulted from that investment?

Scientists need to organize, and form a lobby to get the money. You don't see Colin Powell going on tv and saying that the next drug resistant strain of bacteria could cause a mushroom cloud of destruction (which is more likely than finding wmds in Iraq).  Haliburton did not pay him to do that.  However, if you go to DC and spend enough cash, you can get people to listen. We could hire some congressman to say that we are facing a bacterial 911, put fear into the public, and then the rest of the congressman would cave for fear that the frightened voting public who just saw the news on Fox would give them the axe if they do not pass the bill. The reform that is needed is more money, and a political body to get the money from the government. Science needs to be in the face of the public everyday, and the public needs to fear what will happen if our reserach cannot keep up with things like microbial antibiotic resistance.  Let the public know that politicians that favor research budget cuts in favor of continuing oil and bank subsidies and the funding of wars are going to wipe us all off the map.

Avatar of: Platycryptus

Anonymous

September 29, 2011

A good starting point for discussion here.  Personally, much of the research that I am interested in takes place outside of the government-controlled network.  People with funding can spend a lot of public money on travel, expensive equipment, overhead administration, etc.  Those of us outside of the network have the advantage that we can study what we want to study, and don't have to hype it up (no conflict of interest).  Our disadvantage is that we generally don't have access to the kind of equipment that the networked people can buy with the funds that they obtain, and thus some areas of experimental science are beyond our scope.  Our corresponding advantage is that we are willing to use simpler, less costly methods that are overlooked by the networked scientists.  In some areas, this can be better.

Avatar of: Slove

Anonymous

September 29, 2011

While it is important to support new investigators our current system totally eliminates the ability to seduce experienced clinicians into research.  Translational research used to be from the bedside to the bench and then back to the bedside.  Now it is just from the bench to the bedside.  As all the baby boomer doctors near retirement it would be interesting to pair them with young scientists and have the clinical acumen joined with the modern molecular methods.  Or to give grants for retired clinicians to join a lab.  Don't leave us out!

Avatar of: Ebarbieri

Anonymous

September 29, 2011

Not sure if someone else noticed this, but how do you propose to invent a new funding system that would give a fair change to tenure-track junior faculty at tenure & promotion, when in the current university system nobody knows how to truly factor the teaching and service components in research, teaching, service? No research funding == no tenure at most institutions.

Avatar of: coffeebreak

Anonymous

September 29, 2011

I don't have a design for a perfect system.  I suspect that the best strategy is to make sure that public funds are widely dispersed through multiple agencies using multiple different funding strategies, and with a significant overlap of scope.  Natural selection of a sort ought to find one or more useful ways to get things done.  Perhaps we need a mechanism to prevent over-centralization. When a monolithic system fails or becomes corrupted, the entire enterprise is lost.   

Avatar of: Kay

Anonymous

September 29, 2011

Some thoughts about John Ioannidis’ suggested changes in
funding:

 

“…consenting scientists who say I’m willing to be randomized
to funding scheme A versus funding scheme B.â€쳌

Isn’t that the situation that we already in? Grants that may
rank highly (and maybe even get funded) by one funding organization are being unscored
at a review panel of another organization. The peer review process itself is subjective
and random. So, we really don’t need additional funding schemes.   

 

“One could think of merging indices that exclude
self-citation or take into account the impact of papers rather than the amount
of papersâ€쳌

One has to ask whether the Impact Factor (IF) of a paper
based on the citations is always an indicator for quality. The average citation
rate of a particular article may not correspond with the average IF of a
journal. I have read an article in the past (I wish I had saved it) that argues
that the high IF of some journal hinges on very few highly cited papers. If
that is correct, we overvalue many individual contributions that appear in a
particular 'high-end' journal. That would mean that a reviewer has to go to the ISI
database and get the real citation rate of the individual papers from a PI. On
the other hand, this is still no clear indication for the correctness of the
actual research data presented in a highly cited article. For example, I know
of an article in Cell that is still being cited despite the fact that it was
retracted in 1998. So, I would be very cautious with such a suggestion for an
overhaul of our funding system.   

 

“Things like sharing of data or protocols, and how often
these data and protocols are utilized by other researchers,…â€쳌

That would be great, but the reality is that anyone who
utilizes research tools from a particular PI then has a conflict of interest
and is being removed from a peer review study section. The downside of this is
that mostly people who are not familiar with a PI’s contributions (and their
significance to the field) conduct the peer review of an application.  

 

 “Moreover, for
example, if you use citation indices, you can always adjust for the number of
years that someone has been active...â€쳌

That becomes very complicated now. A postdoc from a famous
PI who has a first-author paper in a high-end journal and who then assumes an
independent position may have initially better cards for getting funding
according to this scheme. This is already often the case for K awards, for
example. Hence, we already “give some opportunity to lots of people early on
for a few years and see what they can doâ€쳌 as requested in this article. There
should be no mistake, the name of the postdoc’s mentor carries a lot of weight
here. If you then would only count the papers AFTER a postdoc becomes independent
(specifically those without the former mentor’s name on it), the situation
might be quite different. I see a number of these younger PIs who publish
nothing at all over a number of years while they gather more and more data for
another (potential) article in a top journal, which frequently never happens.
By then, their career is going downhill. So again, the use of citation
indices is not a sole indicator for productivity. I still feel that 100+
citations of an article that appeared in a journal with a medium or even low IF
means much more in terms of impact than the same amount (or twice that) of citations of a
similar article published in Nature or Science. In summary, if someone uses citation
indices then they also need to be adjusted by the average IF of a journal
(i.e., a penalty for a paper that appears in Science, for example, and is not cited at least
30-50 times a year, or whatever the current IF is). Face it, who is going to do
all that adjusting etc.? Nobody, and therefore I think these particular suggestions to overhaul our funding
system are dead on arrival.

 

Here are two suggestions that are quite different and could be
implemented without an overhaul of the entire system to alleviate some of the
problems that we face:

 

1) In order to help younger PIs to establish an independent track
record, the funding pool for K-type awards needs to be increased. There has to
be a better transition period where someone can show that he/she can
successfully run a lab. Without funding, universities tend to not hire new
faculty (I am not even speaking about tenure, which often means little to
nothing at some universities).

 

2) There has to be a sliding scale for funding of R21 and R01
grants based on merit. The current situation that 10% (or less) grants get
funded is problematic. There are often innovative grants within the 12-25 %
tile that receive absolutely nothing, and these projects will not move forward
without at least some support for a limited time. This is particularly
important for grants that (really) challenge exiting paradigms (peer review
tends to negatively select those out of the funding range; peer review still is
the “modern day inquisitionâ€쳌). As it is right now, the specific aims of an
excellent grant cannot be used in another application to NIH, and it is a waste
of time for PIs to come up with entirely new ideas each time, and it is counterproductive
that PIs are unable to pursue their research interests that are formulated in a
grant that barely missed the funding line.  

I wish the NIH and NCI directors would take notice of these suggestions.

Avatar of: Sine Nomine

Anonymous

September 29, 2011

I agree with oldscientist, "the problem is not the method of allocation is the ratio of $$/would be investigators'.  When funding levels are sufficient to fund 25-30% of the applications (all of the must fund and some of the fundable grants), then our present NIH system works (and works fairly well).  When budgetary constraints hold funding "level" for years on end so that only 10-12% of grants are funded (some of the must fund and none of the fundable), important research is lost (consider the great idea that was not well-argued versus the so-so idea that was promoted effectively--proposing good science is not the same as good grantsmanship). 

The real problem then is how to get NIH back to a 25-30% success rate.  There are two solutions.  Reduce the number of scientists--not a great idea if we want the US to be the world-leader in research.  Increase funding to NIH--but how to do this in the present economy?   I think the answer is fairly simple (execution is harder--and ensuring that costs aren't just funneled back to the general public is also a concern).  Who benefits financially from the scientific knowledge that we generate?  1) The general public (their tax dollars help support us already), 2) The healthcare establishment (mostly tax-exempt--would it hurt to place a small tax there to further research?  E.g. a $1 excise tax on health insurance premiums or a similarly small percentage on hosptial "earnings"--money that is normally carried over to the next year.  3) Drug/healthcare suppliers/Biotech companies--would it hurt to ask the companies that benefit from sales related to scientific knowledge to help support the pipeline that makes much of their profits possible?

If the scientific community is important to the US, the US will support it.  If it is not, the US will lose its place in the global research community and scientists can go to work for commercial interests or find other jobs.  Universities can't support the research of their faculty (tuition is already too high most places), the taxpayer alone may even be insufficient in the present economy, we should look to those who profit from the work.

Avatar of:

Posts: 0

September 29, 2011

It's unfortunate that all of the money has to come from the federal government. A position should come with a research budget, perhaps allocated by some measure of productivity, and additional funding for larger projects could come from NIH.  We could get back to judging our colleagues by the quality of their work rather than the size of their grant portfolio.

Avatar of:

Posts: 0

September 29, 2011

Given that about as many Americans now believe in UFO's as in evolution it's hard to have much confidence in the current system of funding science in this country. An alternative strategy would be to award more, smaller grants to greater numbers of PI's.  The current approach centralizes power and influence in science, making it difficult to sustain the diverse points of view that would promote a lively competition among ideas.  Such a change also would tend to increase the numbers of educational institutions that can support research programs that would presumably encourage the institutions to hire more science faculty.
 

Avatar of:

Posts: 0

September 29, 2011

take out the middleman (NIH). instead of for-profit science/healthcare companies sending tax dollars to DC to be filtered thru the reverse sewage system of politics, their taxes go to a non-government science foundation where funding is at the discretion of scientist/healthcare professionals. people's tax money that oppose certain initiatives would not be an issue that blocks research. 

Avatar of:

Posts: 0

September 29, 2011

Rlanning,
It is a good idea, but you have to crawl through the political sewer before you can build the UTOPIA.  You need the cash and influence to first build the infrastructure. That means turning scientists into a political group, it does not have to be democrat or republican. 
Look at banks, they fund both republicans and democrats. When their business falls apart because of their own greed, corruption and stupidity, they don't collapse, they get a trillion dollars, with little opposition from either party. We live in a moneytocracy, you spend the money you got a voice. Money controls the issues that congress hears, and votes on. If the public average Joe vote on its own really made a difference, it would be illegal. 
Get the scientists together, undergrads, grads, doctors, PI's, and even industry - it benefits directly from public funding of basic research, and get them to ante up. Include in a research funding bill perks for biotech companies - such as tax breaks if they give their "public research dollars" back to universities to fund projects. Oh yes, you need to make the public research money available to them to for sociallly necessary research.
I could even write the bill, we all could.  We just need someone to ram it through the sewer.
- Don

Avatar of:

Posts: 0

September 29, 2011

the major problem in US science is not the method of allocation- it the ratio of $$/would be investigator.  However the money is split up there are either too few $$ or too many investigators.  We have trained too many people and have created an infrastructure in medical schools around the country that are fed exclusively by NIH. It is arguable (but not by me) that this selection by ability to get funded is fine, and that science should be like Russian literature- train a many people as wish to be trained and let most of them find an unrelated job when finished. 

The long training periods occur because that limits the number of would be PIs and helps the $$/would be PI ratio in the short term.

I personally think that study sections scould save a lot of effort by simply rating grants inthree catagories- absolutely must fund (no more than one/ss member/round) fundable (I would guess about 50% of the applications) and not fundable.  The NIH could X could fund the must fund grants (a couple per study section per meeting) and then randomly fund among the fundable proposals.  Never going to happen because it is politically indefensable but would probably have the same results in terms of research output as now.

Avatar of:

Posts: 0

September 29, 2011

The NIH has grown from nothing to $30 billion in the last 70 years and should be totally de-funded. The system is prone to gaming anad conflict of interest. It unfairly
awards most grants to the most connected researchers and universities who are more interested in advancing their careers than providing the public with usefull information. Then when they get caught in the act they fight the NIH determintions as in the recent OHRP determinations against Hebrew Rehab (Boston), University oof Maryland, and University of Washington (St Louis) (please see OHRP website). They like the grant money, but don't like to follow the rules attached to the grant.---Ed Goodwin  

Avatar of:

Posts: 0

September 29, 2011

I remember an article back in about 1991 claiming that the aids virus did not intially infect hosts through T cells. The article contradicted the dogma of the day, and little else was said of the issue (my guess is just about any study contradicting the holy grails of the NIH will lose funding), until about 1999, when another lab made the discovery of the CxCr4 receptor on a type of dendritic cell that allowed the virus entry.
Is the bureacracy of the grant writing schemes a way to screen applicants, or is it a way to arbitrarily deny funding and foster favoritism? It would be very easy to give more research dollars to those who have theories that will confirm the dogma of the top researchers reviewing the grants, and deny those who do not agree, and really use the arbitrariness of the proposals to stifle competing theories - or worse, use the system to give more dollars to a group of "friends" of the grant scrutinizers. I am not suggesting that very large NIH grants are given to NIH country club members. There is likely no good solution to fix this system.

The only problem with the author's egalitarian solution, is that there is then less dollars are available for the older tried and true researchers, and more dollars being risked on the younger, unproven researchers. But, if the US government increased its budget for science research by an additional $50 billion, that would take care of the problem, then there would be enought money for everyone. Remeber the 90s when research dollars began to flow and all of the great discoveries and inventions that resulted from that investment?

Scientists need to organize, and form a lobby to get the money. You don't see Colin Powell going on tv and saying that the next drug resistant strain of bacteria could cause a mushroom cloud of destruction (which is more likely than finding wmds in Iraq).  Haliburton did not pay him to do that.  However, if you go to DC and spend enough cash, you can get people to listen. We could hire some congressman to say that we are facing a bacterial 911, put fear into the public, and then the rest of the congressman would cave for fear that the frightened voting public who just saw the news on Fox would give them the axe if they do not pass the bill. The reform that is needed is more money, and a political body to get the money from the government. Science needs to be in the face of the public everyday, and the public needs to fear what will happen if our reserach cannot keep up with things like microbial antibiotic resistance.  Let the public know that politicians that favor research budget cuts in favor of continuing oil and bank subsidies and the funding of wars are going to wipe us all off the map.

Avatar of:

Posts: 0

September 29, 2011

A good starting point for discussion here.  Personally, much of the research that I am interested in takes place outside of the government-controlled network.  People with funding can spend a lot of public money on travel, expensive equipment, overhead administration, etc.  Those of us outside of the network have the advantage that we can study what we want to study, and don't have to hype it up (no conflict of interest).  Our disadvantage is that we generally don't have access to the kind of equipment that the networked people can buy with the funds that they obtain, and thus some areas of experimental science are beyond our scope.  Our corresponding advantage is that we are willing to use simpler, less costly methods that are overlooked by the networked scientists.  In some areas, this can be better.

Avatar of:

Posts: 0

September 29, 2011

While it is important to support new investigators our current system totally eliminates the ability to seduce experienced clinicians into research.  Translational research used to be from the bedside to the bench and then back to the bedside.  Now it is just from the bench to the bedside.  As all the baby boomer doctors near retirement it would be interesting to pair them with young scientists and have the clinical acumen joined with the modern molecular methods.  Or to give grants for retired clinicians to join a lab.  Don't leave us out!

Avatar of:

Posts: 0

September 29, 2011

Not sure if someone else noticed this, but how do you propose to invent a new funding system that would give a fair change to tenure-track junior faculty at tenure & promotion, when in the current university system nobody knows how to truly factor the teaching and service components in research, teaching, service? No research funding == no tenure at most institutions.

Avatar of:

Posts: 0

September 29, 2011

I don't have a design for a perfect system.  I suspect that the best strategy is to make sure that public funds are widely dispersed through multiple agencies using multiple different funding strategies, and with a significant overlap of scope.  Natural selection of a sort ought to find one or more useful ways to get things done.  Perhaps we need a mechanism to prevent over-centralization. When a monolithic system fails or becomes corrupted, the entire enterprise is lost.   

Avatar of:

Posts: 0

September 29, 2011

Some thoughts about John Ioannidis’ suggested changes in
funding:

 

“…consenting scientists who say I’m willing to be randomized
to funding scheme A versus funding scheme B.â€쳌

Isn’t that the situation that we already in? Grants that may
rank highly (and maybe even get funded) by one funding organization are being unscored
at a review panel of another organization. The peer review process itself is subjective
and random. So, we really don’t need additional funding schemes.   

 

“One could think of merging indices that exclude
self-citation or take into account the impact of papers rather than the amount
of papersâ€쳌

One has to ask whether the Impact Factor (IF) of a paper
based on the citations is always an indicator for quality. The average citation
rate of a particular article may not correspond with the average IF of a
journal. I have read an article in the past (I wish I had saved it) that argues
that the high IF of some journal hinges on very few highly cited papers. If
that is correct, we overvalue many individual contributions that appear in a
particular 'high-end' journal. That would mean that a reviewer has to go to the ISI
database and get the real citation rate of the individual papers from a PI. On
the other hand, this is still no clear indication for the correctness of the
actual research data presented in a highly cited article. For example, I know
of an article in Cell that is still being cited despite the fact that it was
retracted in 1998. So, I would be very cautious with such a suggestion for an
overhaul of our funding system.   

 

“Things like sharing of data or protocols, and how often
these data and protocols are utilized by other researchers,…â€쳌

That would be great, but the reality is that anyone who
utilizes research tools from a particular PI then has a conflict of interest
and is being removed from a peer review study section. The downside of this is
that mostly people who are not familiar with a PI’s contributions (and their
significance to the field) conduct the peer review of an application.  

 

 “Moreover, for
example, if you use citation indices, you can always adjust for the number of
years that someone has been active...â€쳌

That becomes very complicated now. A postdoc from a famous
PI who has a first-author paper in a high-end journal and who then assumes an
independent position may have initially better cards for getting funding
according to this scheme. This is already often the case for K awards, for
example. Hence, we already “give some opportunity to lots of people early on
for a few years and see what they can doâ€쳌 as requested in this article. There
should be no mistake, the name of the postdoc’s mentor carries a lot of weight
here. If you then would only count the papers AFTER a postdoc becomes independent
(specifically those without the former mentor’s name on it), the situation
might be quite different. I see a number of these younger PIs who publish
nothing at all over a number of years while they gather more and more data for
another (potential) article in a top journal, which frequently never happens.
By then, their career is going downhill. So again, the use of citation
indices is not a sole indicator for productivity. I still feel that 100+
citations of an article that appeared in a journal with a medium or even low IF
means much more in terms of impact than the same amount (or twice that) of citations of a
similar article published in Nature or Science. In summary, if someone uses citation
indices then they also need to be adjusted by the average IF of a journal
(i.e., a penalty for a paper that appears in Science, for example, and is not cited at least
30-50 times a year, or whatever the current IF is). Face it, who is going to do
all that adjusting etc.? Nobody, and therefore I think these particular suggestions to overhaul our funding
system are dead on arrival.

 

Here are two suggestions that are quite different and could be
implemented without an overhaul of the entire system to alleviate some of the
problems that we face:

 

1) In order to help younger PIs to establish an independent track
record, the funding pool for K-type awards needs to be increased. There has to
be a better transition period where someone can show that he/she can
successfully run a lab. Without funding, universities tend to not hire new
faculty (I am not even speaking about tenure, which often means little to
nothing at some universities).

 

2) There has to be a sliding scale for funding of R21 and R01
grants based on merit. The current situation that 10% (or less) grants get
funded is problematic. There are often innovative grants within the 12-25 %
tile that receive absolutely nothing, and these projects will not move forward
without at least some support for a limited time. This is particularly
important for grants that (really) challenge exiting paradigms (peer review
tends to negatively select those out of the funding range; peer review still is
the “modern day inquisitionâ€쳌). As it is right now, the specific aims of an
excellent grant cannot be used in another application to NIH, and it is a waste
of time for PIs to come up with entirely new ideas each time, and it is counterproductive
that PIs are unable to pursue their research interests that are formulated in a
grant that barely missed the funding line.  

I wish the NIH and NCI directors would take notice of these suggestions.

Avatar of:

Posts: 0

September 29, 2011

I agree with oldscientist, "the problem is not the method of allocation is the ratio of $$/would be investigators'.  When funding levels are sufficient to fund 25-30% of the applications (all of the must fund and some of the fundable grants), then our present NIH system works (and works fairly well).  When budgetary constraints hold funding "level" for years on end so that only 10-12% of grants are funded (some of the must fund and none of the fundable), important research is lost (consider the great idea that was not well-argued versus the so-so idea that was promoted effectively--proposing good science is not the same as good grantsmanship). 

The real problem then is how to get NIH back to a 25-30% success rate.  There are two solutions.  Reduce the number of scientists--not a great idea if we want the US to be the world-leader in research.  Increase funding to NIH--but how to do this in the present economy?   I think the answer is fairly simple (execution is harder--and ensuring that costs aren't just funneled back to the general public is also a concern).  Who benefits financially from the scientific knowledge that we generate?  1) The general public (their tax dollars help support us already), 2) The healthcare establishment (mostly tax-exempt--would it hurt to place a small tax there to further research?  E.g. a $1 excise tax on health insurance premiums or a similarly small percentage on hosptial "earnings"--money that is normally carried over to the next year.  3) Drug/healthcare suppliers/Biotech companies--would it hurt to ask the companies that benefit from sales related to scientific knowledge to help support the pipeline that makes much of their profits possible?

If the scientific community is important to the US, the US will support it.  If it is not, the US will lose its place in the global research community and scientists can go to work for commercial interests or find other jobs.  Universities can't support the research of their faculty (tuition is already too high most places), the taxpayer alone may even be insufficient in the present economy, we should look to those who profit from the work.

Avatar of:

Posts: 0

September 29, 2011

It's unfortunate that all of the money has to come from the federal government. A position should come with a research budget, perhaps allocated by some measure of productivity, and additional funding for larger projects could come from NIH.  We could get back to judging our colleagues by the quality of their work rather than the size of their grant portfolio.

Avatar of:

Posts: 0

September 29, 2011

Given that about as many Americans now believe in UFO's as in evolution it's hard to have much confidence in the current system of funding science in this country. An alternative strategy would be to award more, smaller grants to greater numbers of PI's.  The current approach centralizes power and influence in science, making it difficult to sustain the diverse points of view that would promote a lively competition among ideas.  Such a change also would tend to increase the numbers of educational institutions that can support research programs that would presumably encourage the institutions to hire more science faculty.
 

Avatar of:

Posts: 0

September 29, 2011

take out the middleman (NIH). instead of for-profit science/healthcare companies sending tax dollars to DC to be filtered thru the reverse sewage system of politics, their taxes go to a non-government science foundation where funding is at the discretion of scientist/healthcare professionals. people's tax money that oppose certain initiatives would not be an issue that blocks research. 

Avatar of:

Posts: 0

September 29, 2011

Rlanning,
It is a good idea, but you have to crawl through the political sewer before you can build the UTOPIA.  You need the cash and influence to first build the infrastructure. That means turning scientists into a political group, it does not have to be democrat or republican. 
Look at banks, they fund both republicans and democrats. When their business falls apart because of their own greed, corruption and stupidity, they don't collapse, they get a trillion dollars, with little opposition from either party. We live in a moneytocracy, you spend the money you got a voice. Money controls the issues that congress hears, and votes on. If the public average Joe vote on its own really made a difference, it would be illegal. 
Get the scientists together, undergrads, grads, doctors, PI's, and even industry - it benefits directly from public funding of basic research, and get them to ante up. Include in a research funding bill perks for biotech companies - such as tax breaks if they give their "public research dollars" back to universities to fund projects. Oh yes, you need to make the public research money available to them to for sociallly necessary research.
I could even write the bill, we all could.  We just need someone to ram it through the sewer.
- Don

Avatar of:

Posts: 0

September 29, 2011

the major problem in US science is not the method of allocation- it the ratio of $$/would be investigator.  However the money is split up there are either too few $$ or too many investigators.  We have trained too many people and have created an infrastructure in medical schools around the country that are fed exclusively by NIH. It is arguable (but not by me) that this selection by ability to get funded is fine, and that science should be like Russian literature- train a many people as wish to be trained and let most of them find an unrelated job when finished. 

The long training periods occur because that limits the number of would be PIs and helps the $$/would be PI ratio in the short term.

I personally think that study sections scould save a lot of effort by simply rating grants inthree catagories- absolutely must fund (no more than one/ss member/round) fundable (I would guess about 50% of the applications) and not fundable.  The NIH could X could fund the must fund grants (a couple per study section per meeting) and then randomly fund among the fundable proposals.  Never going to happen because it is politically indefensable but would probably have the same results in terms of research output as now.

Avatar of:

Posts: 0

September 29, 2011

The NIH has grown from nothing to $30 billion in the last 70 years and should be totally de-funded. The system is prone to gaming anad conflict of interest. It unfairly
awards most grants to the most connected researchers and universities who are more interested in advancing their careers than providing the public with usefull information. Then when they get caught in the act they fight the NIH determintions as in the recent OHRP determinations against Hebrew Rehab (Boston), University oof Maryland, and University of Washington (St Louis) (please see OHRP website). They like the grant money, but don't like to follow the rules attached to the grant.---Ed Goodwin  

Avatar of:

Posts: 0

September 29, 2011

I remember an article back in about 1991 claiming that the aids virus did not intially infect hosts through T cells. The article contradicted the dogma of the day, and little else was said of the issue (my guess is just about any study contradicting the holy grails of the NIH will lose funding), until about 1999, when another lab made the discovery of the CxCr4 receptor on a type of dendritic cell that allowed the virus entry.
Is the bureacracy of the grant writing schemes a way to screen applicants, or is it a way to arbitrarily deny funding and foster favoritism? It would be very easy to give more research dollars to those who have theories that will confirm the dogma of the top researchers reviewing the grants, and deny those who do not agree, and really use the arbitrariness of the proposals to stifle competing theories - or worse, use the system to give more dollars to a group of "friends" of the grant scrutinizers. I am not suggesting that very large NIH grants are given to NIH country club members. There is likely no good solution to fix this system.

The only problem with the author's egalitarian solution, is that there is then less dollars are available for the older tried and true researchers, and more dollars being risked on the younger, unproven researchers. But, if the US government increased its budget for science research by an additional $50 billion, that would take care of the problem, then there would be enought money for everyone. Remeber the 90s when research dollars began to flow and all of the great discoveries and inventions that resulted from that investment?

Scientists need to organize, and form a lobby to get the money. You don't see Colin Powell going on tv and saying that the next drug resistant strain of bacteria could cause a mushroom cloud of destruction (which is more likely than finding wmds in Iraq).  Haliburton did not pay him to do that.  However, if you go to DC and spend enough cash, you can get people to listen. We could hire some congressman to say that we are facing a bacterial 911, put fear into the public, and then the rest of the congressman would cave for fear that the frightened voting public who just saw the news on Fox would give them the axe if they do not pass the bill. The reform that is needed is more money, and a political body to get the money from the government. Science needs to be in the face of the public everyday, and the public needs to fear what will happen if our reserach cannot keep up with things like microbial antibiotic resistance.  Let the public know that politicians that favor research budget cuts in favor of continuing oil and bank subsidies and the funding of wars are going to wipe us all off the map.

Avatar of:

Posts: 0

September 29, 2011

A good starting point for discussion here.  Personally, much of the research that I am interested in takes place outside of the government-controlled network.  People with funding can spend a lot of public money on travel, expensive equipment, overhead administration, etc.  Those of us outside of the network have the advantage that we can study what we want to study, and don't have to hype it up (no conflict of interest).  Our disadvantage is that we generally don't have access to the kind of equipment that the networked people can buy with the funds that they obtain, and thus some areas of experimental science are beyond our scope.  Our corresponding advantage is that we are willing to use simpler, less costly methods that are overlooked by the networked scientists.  In some areas, this can be better.

Avatar of:

Posts: 0

September 29, 2011

While it is important to support new investigators our current system totally eliminates the ability to seduce experienced clinicians into research.  Translational research used to be from the bedside to the bench and then back to the bedside.  Now it is just from the bench to the bedside.  As all the baby boomer doctors near retirement it would be interesting to pair them with young scientists and have the clinical acumen joined with the modern molecular methods.  Or to give grants for retired clinicians to join a lab.  Don't leave us out!

Avatar of:

Posts: 0

September 29, 2011

Not sure if someone else noticed this, but how do you propose to invent a new funding system that would give a fair change to tenure-track junior faculty at tenure & promotion, when in the current university system nobody knows how to truly factor the teaching and service components in research, teaching, service? No research funding == no tenure at most institutions.

Avatar of:

Posts: 0

September 29, 2011

I don't have a design for a perfect system.  I suspect that the best strategy is to make sure that public funds are widely dispersed through multiple agencies using multiple different funding strategies, and with a significant overlap of scope.  Natural selection of a sort ought to find one or more useful ways to get things done.  Perhaps we need a mechanism to prevent over-centralization. When a monolithic system fails or becomes corrupted, the entire enterprise is lost.   

Avatar of:

Posts: 0

September 29, 2011

Some thoughts about John Ioannidis’ suggested changes in
funding:

 

“…consenting scientists who say I’m willing to be randomized
to funding scheme A versus funding scheme B.â€쳌

Isn’t that the situation that we already in? Grants that may
rank highly (and maybe even get funded) by one funding organization are being unscored
at a review panel of another organization. The peer review process itself is subjective
and random. So, we really don’t need additional funding schemes.   

 

“One could think of merging indices that exclude
self-citation or take into account the impact of papers rather than the amount
of papersâ€쳌

One has to ask whether the Impact Factor (IF) of a paper
based on the citations is always an indicator for quality. The average citation
rate of a particular article may not correspond with the average IF of a
journal. I have read an article in the past (I wish I had saved it) that argues
that the high IF of some journal hinges on very few highly cited papers. If
that is correct, we overvalue many individual contributions that appear in a
particular 'high-end' journal. That would mean that a reviewer has to go to the ISI
database and get the real citation rate of the individual papers from a PI. On
the other hand, this is still no clear indication for the correctness of the
actual research data presented in a highly cited article. For example, I know
of an article in Cell that is still being cited despite the fact that it was
retracted in 1998. So, I would be very cautious with such a suggestion for an
overhaul of our funding system.   

 

“Things like sharing of data or protocols, and how often
these data and protocols are utilized by other researchers,…â€쳌

That would be great, but the reality is that anyone who
utilizes research tools from a particular PI then has a conflict of interest
and is being removed from a peer review study section. The downside of this is
that mostly people who are not familiar with a PI’s contributions (and their
significance to the field) conduct the peer review of an application.  

 

 “Moreover, for
example, if you use citation indices, you can always adjust for the number of
years that someone has been active...â€쳌

That becomes very complicated now. A postdoc from a famous
PI who has a first-author paper in a high-end journal and who then assumes an
independent position may have initially better cards for getting funding
according to this scheme. This is already often the case for K awards, for
example. Hence, we already “give some opportunity to lots of people early on
for a few years and see what they can doâ€쳌 as requested in this article. There
should be no mistake, the name of the postdoc’s mentor carries a lot of weight
here. If you then would only count the papers AFTER a postdoc becomes independent
(specifically those without the former mentor’s name on it), the situation
might be quite different. I see a number of these younger PIs who publish
nothing at all over a number of years while they gather more and more data for
another (potential) article in a top journal, which frequently never happens.
By then, their career is going downhill. So again, the use of citation
indices is not a sole indicator for productivity. I still feel that 100+
citations of an article that appeared in a journal with a medium or even low IF
means much more in terms of impact than the same amount (or twice that) of citations of a
similar article published in Nature or Science. In summary, if someone uses citation
indices then they also need to be adjusted by the average IF of a journal
(i.e., a penalty for a paper that appears in Science, for example, and is not cited at least
30-50 times a year, or whatever the current IF is). Face it, who is going to do
all that adjusting etc.? Nobody, and therefore I think these particular suggestions to overhaul our funding
system are dead on arrival.

 

Here are two suggestions that are quite different and could be
implemented without an overhaul of the entire system to alleviate some of the
problems that we face:

 

1) In order to help younger PIs to establish an independent track
record, the funding pool for K-type awards needs to be increased. There has to
be a better transition period where someone can show that he/she can
successfully run a lab. Without funding, universities tend to not hire new
faculty (I am not even speaking about tenure, which often means little to
nothing at some universities).

 

2) There has to be a sliding scale for funding of R21 and R01
grants based on merit. The current situation that 10% (or less) grants get
funded is problematic. There are often innovative grants within the 12-25 %
tile that receive absolutely nothing, and these projects will not move forward
without at least some support for a limited time. This is particularly
important for grants that (really) challenge exiting paradigms (peer review
tends to negatively select those out of the funding range; peer review still is
the “modern day inquisitionâ€쳌). As it is right now, the specific aims of an
excellent grant cannot be used in another application to NIH, and it is a waste
of time for PIs to come up with entirely new ideas each time, and it is counterproductive
that PIs are unable to pursue their research interests that are formulated in a
grant that barely missed the funding line.  

I wish the NIH and NCI directors would take notice of these suggestions.

Avatar of:

Posts: 0

September 29, 2011

I agree with oldscientist, "the problem is not the method of allocation is the ratio of $$/would be investigators'.  When funding levels are sufficient to fund 25-30% of the applications (all of the must fund and some of the fundable grants), then our present NIH system works (and works fairly well).  When budgetary constraints hold funding "level" for years on end so that only 10-12% of grants are funded (some of the must fund and none of the fundable), important research is lost (consider the great idea that was not well-argued versus the so-so idea that was promoted effectively--proposing good science is not the same as good grantsmanship). 

The real problem then is how to get NIH back to a 25-30% success rate.  There are two solutions.  Reduce the number of scientists--not a great idea if we want the US to be the world-leader in research.  Increase funding to NIH--but how to do this in the present economy?   I think the answer is fairly simple (execution is harder--and ensuring that costs aren't just funneled back to the general public is also a concern).  Who benefits financially from the scientific knowledge that we generate?  1) The general public (their tax dollars help support us already), 2) The healthcare establishment (mostly tax-exempt--would it hurt to place a small tax there to further research?  E.g. a $1 excise tax on health insurance premiums or a similarly small percentage on hosptial "earnings"--money that is normally carried over to the next year.  3) Drug/healthcare suppliers/Biotech companies--would it hurt to ask the companies that benefit from sales related to scientific knowledge to help support the pipeline that makes much of their profits possible?

If the scientific community is important to the US, the US will support it.  If it is not, the US will lose its place in the global research community and scientists can go to work for commercial interests or find other jobs.  Universities can't support the research of their faculty (tuition is already too high most places), the taxpayer alone may even be insufficient in the present economy, we should look to those who profit from the work.

Avatar of:

Posts: 0

September 30, 2011

Realization of scientific technological progress, is minimization of risk of insurance obligations of the state.

Avatar of:

Posts: 0

September 30, 2011

The fact that becoming an independent researcher takes upwards of 15years is just too long for me. Furthermore, being in research for almost a decade (undergrad, graduate, 1 yr postdoc), I have noticed that funding favors medical researchers. I'm not talking about PhDs who study medicine, rather, MDs who apply for grants. They seem to get a more favorable score/review simply bc they are clinicians or are related to the medical field, regardless of the importance of the work. What about basic science? Basic science is the foundation of scientific research today! It also seems that researchers these days don't care about science, but as another commenter said, use it to further their careers.

We don't train that many Phds; it's a misconception. We are allowing too many POORLY trained FOREIGN Phds to compete for research positions (just bc they can be hired on the cheap). What this does is reduce the amount of spots for citizens, lowers wages, stifles interaction/discussion in the lab and department, and create an atmosphere of 'Us' versus 'Them.'

I did undergrad research at a great institution, grad program at a top notch university, but I am appalled and disgusted at what I'm seeing as a postdoc.

I am leaving science after this year, and will discourage undergraduates and family from entering graduate school or scientific research. The system is broken, no one seems to care, and it isn't going to change any time soon.

Avatar of:

Posts: 0

September 30, 2011

Realization of scientific technological progress, is minimization of risk of insurance obligations of the state.

Avatar of:

Posts: 0

September 30, 2011

The fact that becoming an independent researcher takes upwards of 15years is just too long for me. Furthermore, being in research for almost a decade (undergrad, graduate, 1 yr postdoc), I have noticed that funding favors medical researchers. I'm not talking about PhDs who study medicine, rather, MDs who apply for grants. They seem to get a more favorable score/review simply bc they are clinicians or are related to the medical field, regardless of the importance of the work. What about basic science? Basic science is the foundation of scientific research today! It also seems that researchers these days don't care about science, but as another commenter said, use it to further their careers.

We don't train that many Phds; it's a misconception. We are allowing too many POORLY trained FOREIGN Phds to compete for research positions (just bc they can be hired on the cheap). What this does is reduce the amount of spots for citizens, lowers wages, stifles interaction/discussion in the lab and department, and create an atmosphere of 'Us' versus 'Them.'

I did undergrad research at a great institution, grad program at a top notch university, but I am appalled and disgusted at what I'm seeing as a postdoc.

I am leaving science after this year, and will discourage undergraduates and family from entering graduate school or scientific research. The system is broken, no one seems to care, and it isn't going to change any time soon.

Avatar of: Viktor

Anonymous

September 30, 2011

Realization of scientific technological progress, is minimization of risk of insurance obligations of the state.

Avatar of: Sick of Science

Anonymous

September 30, 2011

The fact that becoming an independent researcher takes upwards of 15years is just too long for me. Furthermore, being in research for almost a decade (undergrad, graduate, 1 yr postdoc), I have noticed that funding favors medical researchers. I'm not talking about PhDs who study medicine, rather, MDs who apply for grants. They seem to get a more favorable score/review simply bc they are clinicians or are related to the medical field, regardless of the importance of the work. What about basic science? Basic science is the foundation of scientific research today! It also seems that researchers these days don't care about science, but as another commenter said, use it to further their careers.

We don't train that many Phds; it's a misconception. We are allowing too many POORLY trained FOREIGN Phds to compete for research positions (just bc they can be hired on the cheap). What this does is reduce the amount of spots for citizens, lowers wages, stifles interaction/discussion in the lab and department, and create an atmosphere of 'Us' versus 'Them.'

I did undergrad research at a great institution, grad program at a top notch university, but I am appalled and disgusted at what I'm seeing as a postdoc.

I am leaving science after this year, and will discourage undergraduates and family from entering graduate school or scientific research. The system is broken, no one seems to care, and it isn't going to change any time soon.

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement
Ingenuity
Ingenuity

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
Hamamatsu
Hamamatsu
Advertisement
NeuroScientistNews
NeuroScientistNews
Life Technologies