Radical Proposal For Reorganizing Research Support: Lotteries, Prizes

It's hard to escape the sense of concern in the scientific community that something has gone awry with the mechanisms by which public funds are allocated for research. Before putting forth several proposals for change in the current arrangement, I feel it is important to point out certain realities--moral hazards, perhaps--that must be taken into consideration by anyone presuming to tamper with the status quo: * A robust government demands answers for the investment of public wealth, whereas a healthy scientific research effort will measure success by generating some answers and a lot more questions. * The public must be able to ascertain what it gets for its investment more swiftly than most research projects require for their completion; and elected representatives of the public naturally want to show results for their supportive votes. * The resulting tension between scientists and policymakers cannot be resolved: Public representatives in legislatures want tangible results, which are the product of technology rather than the direct product of science; public servants in the science bureaus try to balance competing demands and avoid mistakes. Given the hazards, my suggested correctives are delivered in the hope that while they will not quickly resolve past negative consequences of public endowments, they might diminish their continued generation. They are geared to support advances in a number of areas: to encourage, with appropriate critique, some existing experiments by agencies; to recommend the privatization of some funding to reduce the centralizing and risk-avoiding tendencies of federal agencies; and to advocate the reorganization of federal granting arrangements for basic research. Three specific programs require consideration: prizes for research results achieved, rather than awards for proposals of research to be done; a lottery to encourage new research; and a research-loan program. Experiments One current intergovernmental experiment in decentralized science funding is at work in Florida, and it gives reason for guarded optimism. In 1984 the National Academy of Sciences addressed the problems of "red tape" in agency funding of science and recommended a two-year experiment in federal science resource packaging at the state level, called the Florida Demonstration Project (Daniel Charles, "Pilot Program Cuts Red Tape for Federal Grants," Science, 235:966, 1987). This exemplary project allows university researchers to combine grants from different sources into a single research program, and it relies more on university management of routine tasks, including authority to change line items in the budget and to grant program extensions of up to one year. In addition to improving productivity, the project seeks to reduce the administrative work of researchers--a major complaint of those who have held federal grants--and to cut indirect costs. Many arguments recently have been put forth for placing a variety of public services in the private sector--public transportation, incarceration of prisoners, refuse removal, postal services, and so forth. These proposals reflect the efficiencies expected from private competition for government contracts. But most such proposals do not ask whether the service could withstand a market test, and while I am not suggesting that the management of federal scientific research funding be turned over to private organizations, it does seem that other experiments are in order. Scientific societies could be encouraged to undertake demonstration projects in science funding to diversify and decentralize the decision-making process. For example, the 105,000-member scientific honor society, Sigma Xi--whose mission includes the encouragement of research across disciplines--has 500 chapters and clubs that provide a uniquely decentralized information network through which it encourages an active grants-in-aid program for research and distributes small sums to promising young scientists. The society, working in conjunction with the National Academy of Sciences, awards prizes on behalf of not only the academy, but also some private organizations, such as the Bank of Delaware. In executing these functions the society uses the voluntary efforts of individual academic and nonacademic scientists across the country, representing more than 170 disciplines. It would be possible for the National Science Foundation to subcontract to the society such programs as the Presidential Young Investigator Awards. Or, the administration of a single program could go to several organizations, to develop a comparative test of performance (with appropriate rewards to the most competitive). Other honor societies or scientific societies--such as Phi Beta Kappa or the American Association for the Advancement of Science--could emulate the Sigma Xi model or create alternatives. Another example of privatization involves contractor services. At present, as much as 15 percent of the budget for agencies that fund scientific work is spent through direct purchase of services from consultants. Many of these purchases are for equipment and maintenance, and some are for data collection and analysis--such as that upon which much science policy analysis relies--as well as for a host of other functions ostensibly in support of the agency's management. Contractors often hold indefinite quantity contracts (IQCs) or basic ordering agreements (BOAs) with the agency. These contracts ordinarily result from responses to requests for proposals that an agency solicits; they are awarded competitively after being judged by panels or boards internal to the organization. The winning firm is then essentially on "standby" for projects the agency develops. These projects are purchases of applied research, pure and simple, without either the benefit or the burden of peer review. This is an example of privatized research and service, but the direction of BOAs remains in agency control. BOAs highlight the question of applied research, that is, whether their derivative products would be purchased by actual consumers in the absence of public funding. Their results would be expected to reflect or support the agency's preferences, since a contractor who consistently delivers findings contrary to an agency's interests may diminish its chances for contract renewal or for a successful subsequent bid. But such consulting firms have developed considerable expertise in many fields and in the management of research. They might be likely institutions to emerge as scientific research managers if a prize and lottery system were adopted. Prize System Virtually all federal funding of R&D reflects a proposal's promise to carry out a research program. But sometimes the proposer already knows the research results. Peers, sitting together either as panels or as dispersed individual evaluators, review NSF and NIH proposals, for example. They select which project to support on the most secure information available: the researcher's reputation and often that of his or her institution. Many think that the weight of these reputational elements often is greater than that of the proposal's merit. Indeed, a widespread belief prevails that proposal merit is less important than personal acquaintance with the funding agents. This problem begs a consideration of Gordon Tullock's proposal of a prize system to reward accomplishments rather than promises (The Organization of Inquiry, pages 205-208, Duke University Press, 1966). In my view, research accomplishments should become prize candidates no earlier than two years after the first public exposition of the results, thereby giving time for true peer examination and debate of their relative merits, not to mention checking of the results to reduce public support of fraudulent research. The award's size would reflect cost recovery for research plus a bonus stipend. This procedure would place the accounting burden on the researcher or the researcher's agents, who could be advised in advance of allowable budget categories. Uncertainty over the funding would lead to more cost-effective management of resources. Firms or university corporations might back many researchers. But lack of such backing would not exclude the independent researcher from competition. The prize competition would be open to anyone, without restriction, and upon nomination or application. Juries, which the awarding agen- chaos could draw from a national pool, would evaluate project results and determine winners. Selection of jurors might be based on a random, stratified sample of people with differential degrees of specialized knowledge of the subject under review. All jurors would remain unknown to each other until the final list of jurors had been chosen for any group of related research papers. Removing bureau agents from specific allocations of research funds avoids current reliance on hand-picked referees who may reflect disciplinary orthodoxies and may rely too much on reputational criteria over clear merit. Lotteries A lottery would overcome the chief objection to a prize system: that new researchers lack a history of successful research that might gain them external support from firms, foundations, patrons, or institutions during their beginning years. Hence, they might never be able to conduct any research. A lottery allocation assures that bureau officials and legislators cannot collude to respond to interest-group demands of disciplinary or institutional consortia. Indeed, a lottery can be unabashedly "fair," with the right to enter in pursuit of public scientific research funds constrained by a publication of criteria whose merits scientists, citizens, and public officials can openly debate. After a body of entrants had all met the established criteria, a drawing would be held--perhaps an annual event--with winners deter- mined by chance. The drawing could be run by, say, a bonded accounting firm. Two lotteries seem appropriate: one for individual researchers and the other for institutions. The individual researchers' lottery awards would be open to anyone meeting qualifying criteria, regardless of background, upon presentation of a statement of research intent. The awards would be limited in size, but a researcher could increase the probability of winning by soliciting matching funds from private sources. If, for example, the researcher has found support for one-half of a project's expected costs, then his or her number of "chances" might be increased by some other factor. This would help make the scientist more consumer-sensitive, a value that can be further supported by the choice of jurors. A jury--half drawn from the scheduled pool of scientists described previously in the discussion of prizes, and half being literate citizens--would evaluate awards. This constitution of juries seeks to thrust upon scientists the burden of explaining why they are doing scientific research as well as how they have done it. This process is aimed to correct a system that sometimes supports researchers who are merely, and often mindlessly, executing someone s research program. Explanation of research in lay terms as well as in professional language can help to reduce the growing public ignorance of science, thereby promoting its greater acceptance. One might argue that scientists would dominate lay members of such a jury, but that problem exists by design under the current system. The jury can grade the results of lottery-supported research. The grade then will adjust any of the researcher's subsequent lottery entries, to increase or to decrease the factor by which his or her chances are multiplied. Research reports judged to be fraudulent would result in permanent exclusion from the individual-researcher lottery. But high-scoring research also could be a good candidate for prize nomination in the competition described earlier, and from which the researcher would be eligible to receive the bonus (but not the research-cost recovery). The institutional lottery awards might be larger. But the process of institutional- and individual-award evaluation would be identical. Institutions could gather researchers who had proved their worth in lottery results, or who had won in the prize competition. Indeed, they would try to hire successful scientists, who would bring with them lottery "grades" and who would best complement an institution's existing research staff. Institutions could sponsor individual scientists or combine their grades to enhance the chances that institutional proposals would win the lottery. Institutions that hosted prize-winning research also would receive "institutional grades," which would increase their chances of winning subsequent lotteries. This arrangement would shift current institutional and individual emphasis from securing awards to a greater concern for research results. It would also introduce a self-organizing property characteristic of open market systems, which promotes human capital accumulation and fosters productivity. The dual-lottery proposal generally outlined here enjoys several advantages: * It avoids many of the negative consequences of bureau-directed control of scientific inquiry, especially the avoidance of unfamiliar ideas and risky proposals. * It would enhance public science literacy by requiring scientists to be more cognizant of lay evaluation of the process and content of research. * It would promote responsibility among scientists to those who pay for their research, thereby eroding the view held by many scientists that public support is an "entitlement." * It would place an emphasis on creativity rather than credentials. * It would strengthen the negotiating positions of successful scientists with potential employers. * It would stimulate new kinds of research institutions, to compete with universities and consulting groups. * It would shift the outlook of researchers to results rather than to the securing of awards. * It would suppress whatever perverse institutional forces currently promote "undue concentration" of awards. * It would increase the rate of introduction of new ideas. * It would provide an experiment, to test the hypothesis that the existing science bureau granting mechanisms are not the best approach to allocations. Research Loans Some scientists with high research ambitions and confidence in their programs, who nevertheless have failed in the lottery or in securing independent support, could be candidates for limited, federally secured research loans, not unlike student loans. After a time, the research could be judged in the same pool as the lottery-supported research, and reviewers would determine how much of the loan must be repaid, including the possibility of a negative premium to discourage flippant applications. Research results would be eligible for the prize awards described earlier. Such loan programs would be available to anyone with a coherent proposal and some collateral. Existing financial institutions could administer the program. Those who believe that fundamental scientific endeavor can and should be managed for a national purpose probably will consider as fanciful the contentions and propositions suggested here. Others--who view public support for basic research as an investment in a culture that has yielded beneficial, if unpredictable, results--may see a new horizon for science in the kind of self-organizing arrangement identified here. Jack Sommer heads the Political Economy Research Institute in McLean, Va. The ideas expressed in this article were presented in expanded form at the AAAS Annual Meeting this past February.

Radical Proposal For Reorganizing Research Support: Lotteries, Prizes

Interested in reading more?

Become a Member of