The key issue in research evaluation is the combination of the scientific process being such a complex phenomenon, and our constant desire to quantify its outcomes. This has led to the use of multiple indicators, such as counts of publications, counts of citations, and peer-review. The vast majority of the problems discussed by Phelan, Perry, and other writers are primarily due to the issues inherent in measuring complex phenomena with an array of indicators. Products from research include a wide variety of outcomes, from "nuggets" of knowledge to an array of ideas, concepts, methods, techniques, and explanations. Downstream outcomes include patents, prototypes, and design criteria. Certain metrics (such as bibliometrics) can only capture a portion of these outcomes. Moreover, these outcomes (or products) are differentially prioritized by performers and sponsors and create inherent conflicts.
Given that only a fraction of research is documented and captured by bibliometric measures, some, if ...
