Evaluation of Scientific Productivity

As the amount of scientific research proliferates and science becomes more costly to produce, funding agencies around the world are increasingly interested in objectively assessing the quality of academic research. Several governments with centralized academic funding mechanisms (e.g., the United Kingdom and Australia) have already implemented research evaluation systems and distribute at least a portion of research funding on the basis of quality assessments. The National Science Foundation is also being pressured by Congress to better assess the results of investments in research. Given this reality, it is important for scientists to understand the strengths and weaknesses of research quality assessment methodologies and to contribute to the ongoing international debate about their appropriate use.

Of course, scientists have always been in the business of evaluating research. Although peer review has a long history, many studies suggest that it is, at best, extremely imperfect.¹ To any researcher who has had...