How Do We Measure What We Get When We `Buy' Research?

Editor's Note: Today's academic scientists are being asked more frequently than in the past to defend the value of their research. And the pursuit and acquisition of knowledge, once accepted at face value as positive objectives within a democratic society, are no longer good enough, by themselves, to justify scientific investigations. Politicians and the public now demand to know what they are getting for their money, and the most satisfying answers tend to be couched in terms of economic payoff to the citizenry as well as the creation of a more skilled labor force. For a long time, nobody was able to quantify the impact that academic research was having on the industrial sector. Enter Edwin Mansfield, a professor of economics and director of the center for economics and technology at the University of Pennsylvania. In the mid-1980s, Mansfield decided to tackle the problem. Within a few years, he and his graduate students had come up with a way to measure the extent to which academic research leads to the development of new products and processes that benefit both the commercial developers and the consumers of those products and processes. Mansfield's calculations--in particular, his finding that the social rate of return of such research was a robust 28 percent in seven selected industries--were seized upon by officials in the Bush administration as a way to demonstrate the value of the United States' investment in basic research. The president's budget request for fiscal year 1992, for example, contains an explanation of the concept and trumpets Mansfield's "number." The data are still crude, Mansfield stresses, and there is much more work still to be done. But the desire by policymakers to know what they are buying when they fund academic research is certain to increase as fiscal pressures grow greater. To that end, The Scientist asked Mansfield to describe his recent work. The economist also agreed to discuss with news editor Jeffrey Mervis some of the issues raised by his work. For more than four years I have been carrying out a study of the extent to which new products and new processes in various industries have been based on recent academic research. The results, which constitute some of the first systematic evidence on this score for a broad segment of United States industry, indicate that about 10 percent of the new products and processes commercialized in 1975-85 by firms in seven industries--information processing, electronics, chemicals, instruments, drugs, metals, and oil--could not have been developed (without substantial delay) in the absence of academic research carried out within 15 years of the introduction of the new product or process. This study, financed by the National Science Foundation, indicated that the contribution of academic research to industrial innovation has been considerable, particularly in industries such as drugs, instruments, and information processing. Of course, this does not mean that other influences--like industrial research, plants and equipment, labor, and management--have not been important as well. But whereas the contribution of these other factors generally is taken for granted, the role of academic research sometimes has been regarded as far more questionable. The percentage of new products and processes based on recent academic research seems to be highest in the drug industry and lowest in the oil industry. Regarding only those academic-research-based products that were first commercialized in 1982-85 in the seven industries, the 1985 sales of these products totaled about $41 billion, or 5 percent of the industries' total sales. The cost reductions realized through implementation of that same period's new processes totaled about $18 billion in 1985, or about 3 percent of total costs in these industries. These figures are based on a random sample of 76 major U.S. firms that, in 1985, accounted for about one-third of these seven industries' sales. A firm's percentage of new products based on academic research seems to be directly related to the extent of its R&D investment. Firms with substantial R&D investment appear more likely than others to produce products or implement processes based on recent academic research because they tend to follow such research more closely. Many of the new products and processes whose development was dependent on academic research were not invented at colleges or universities. Instead, academic research provided new theoretical and empirical findings and new types of instrumentation that were essential for the development of the new product or process. One concept, commonly used in economics, that can be applied to help measure the social benefits from academic research (via its contributions to new products and processes) is the idea of the social rate of return. The social benefits are extra profits to the innovator from the innovation plus the benefits to consumers from being able to buy the product at a lower price. Rough calculations suggest that this social rate of return has been substantial, perhaps 20 percent or higher, even if the central role of academic research in the educational process is ignored. While the problems in determining how much of the social returns are attributable to academic research, rather than industrial R&D, are obvious, crude estimates suggest that the rate of return from academic research remains substantial, even if seemingly generous assumptions are made concerning the social rate of return from industrial research and other industrial innovation costs. However, I must stress that these estimates are only crude beginnings in our attempt to measure the economic effects of academic research. About 60 major firms in the aforementioned industries have also identified the academic researchers whom they regard as having contributed most significantly to their new products and processes. While the highest-rated schools tended to have on their faculties more of the cited researchers than lesser-rated schools, the difference in this regard between the highest-rated and lowest-rated colleges and universities often was smaller than might have been expected. Proximity appears to play an important role in explaining why research at a particular university found application at a particular firm. The characteristics of these highly cited academic researchers are also worth noting. Practically all of these scientists were supported, at least in part, by federal funds. Most of them also received some support from industry, although its contribution was much less than the government's. On average, their research was carried out in small or medium-sized projects, with annual research budgets of less than $250,000. There has been an enormous amount of interest, both in the U.S. and in other countries, in the results of this study. Over the next few years, I intend to take up a variety of related questions regarding the economic effects of academic research. Q What motivated you to investigate the concept of social rate of return? A This is an obvious subject of study for economists, because they use this concept routinely. It doesn't mean that they measure it routinely, because it isn't easy. But it's a way of looking at things that they are taught to utilize. Q Why hasn't this type of work been done earlier? A One reason is that it's enormously difficult. I think that most people have felt, and 10 years ago I would have agreed, that it was just too difficult to make any progress. I didn't set out to estimate the social returns, but as I got more and more data, I could see where some crude estimates could be made, given certain assumptions. And as we got more information, we could make the assumptions more realistic. Q What made it possible to do this type of analysis? A The key problem with academic research, no matter where it's done, is that the results are utilized in a great many places. They show up in the economic sphere perhaps two or three steps removed. There's also a long time span between when basic research takes place and when the economic effects are felt. So the conventional wisdom was that it was impossible to make anything out of this. But I think you can begin by being very conservative and just looking at some of the effects. . . . Even if you just look at one part of the picture, and only within a few industries, the remarkable thing is that you still come up with a considerable payoff to society. And this certainly isn't a measure of the complete payoff of academic research, as I keep saying. Q You make it appear to be a very straightforward process. A Well, the difference between something being impossible and something being straightforward is very small. It's a case where somebody gets an idea, you get lucky, and it seems to work. Q What do your colleagues in the field think about all this? A Most economists these days are far more interested in mathematical modeling. Besides, the concept of social rate of return is as old as the hills. However, this topic, the economics of technological change, is a relatively hot one in economics, and the measurement issues are regarded as very important. Q Your findings have been quoted extensively by such top policymakers as the president's science adviser and the director of NSF. Do you think they've done so accurately? A Well, I don't know. These people are in an arena where, if they were to go through all the caveats that I do, the public would go to sleep. I assume that most of the people they're talking to are smart enough to realize that these figures have lots of problems. They can even call me--I've had a lot of people call, and I've told them what I've told you. Q The words "social rate of return" connote something that makes a contribution to society. But the classical economic definition doesn't draw any distinction between a smaller cordless telephone and a cure for cancer. Isn't it important to weigh the relative importance of various products and processes? A The implicit weighting is the market mechanism itself. But it depends in part on how much you invest. If you only invest a dollar and you get back a significantly better cordless telephone, then it may be that the social rate of return is higher than if you invest billions of dollars in a cure for cancer and only get a lousy, partial cure. It's an inherently difficult problem. It's so hard for economists to measure the value of life and the value of pain-free existence. They make attempts, but it's a very tough area. And it's hard to believe that we'll ever have any more than very crude results here. Q Will it ever be possible to apply this approach to help businesses identify particular types of research with the greatest potential rate of return, or to help government agencies decide how to spend their research dollars? A I don't know. Firms aren't interested in the social rate of return--they're interested in the private rate of return. If our theories of capitalism are correct, you want firms to maximize profits. They might find it of interest, but not of much use. Government agencies, on the other hand, could make use of it to make guesses about the future. But it's very difficult to forecast what will come out of a research project. Q What areas do you plan to explore next? A There are a number of things I'd like to do. There is some unpublished material that is just as useful as the data on social rate of return. I've collected information from firms on which academic research they consider to have been most important to them in their product development. And I've been looking at the characteristics of the universities where they work, and of the projects, and who supports them. We've never had data like this before, and we're beginning to see who these people are. And it's not what you might have guessed. They are not necessarily the leading lights in academia . . . . We are trying to see how these links work, between academic research and industrial innovation. It's eminently doable, and it does not take huge sums of money.

How Do We Measure What We Get When We `Buy' Research?

Interested in reading more?

Become a Member of