Long-Term Vs. Short-Term Journal Impact: Does It Matter?

Date: February 2, 1998 Chart 1 Chart 2 The first published report on journal impact factors was included in E. Garfield, I.H. Sher, "New factors in the evaluation of scientific literature through citation indexing," American Documentation, 14[3]:195-201, July 1963. The late Irving H. Sher, who then was director of R&D at the Philadelphia-based Institute for Scientific Information (ISI), and I created the impact factor to help evaluate and select journals for Current Contents. The current impact factor is determined by counting citations in the current year's publications to papers published in the previous two years and dividing by the number of papers published in the same period. Editors often have complained that this measure, which records average influence in the first and second years after publication, is biased against journals in slow-moving fields. They have argued that measurement of long-term impact would show such journals in a better light. The original reason for creating the impact factor was to make sure that Current Contents covered the most significant journals. Thus, a current impact calculation, based on the two preceding years of publications, served us well enough. Later, ISI started to produce its Journal Citation Reports (JCR) as a byproduct of the Science Citation Index. Publishing long-term journal impact data was not considered a high priority, but the data were there for those persistent enough to combine the input from consecutive annual JCR volumes. Recently, ISI's Journal Performance Indicators database became available. [For information, contact David Pendlebury at ISI; (215) 386-0100, Ext. 1411.] The 1995 edition, which contains publication and citation data on ISI-indexed journals for each year from 1981 to 1995, helped us examine short- and long-term changes in journal citation rates. We used papers published in 1981-1982 and in 1989-1990 as the source groups of cited articles, and used the database to compile 15-year and seven-year cumulative impact data, from 1981-1995 and 1989-1995, respectively. The study was limited to journals that published more than 200 articles in 1981-1982 and eliminated all review journals regardless of size. The table that follows includes the 100 journals with the highest cumulative impact based on 15 years of data. The first group of columns shows the number of articles published in 1981-1982, the total cumulated citations over 15 years, the impact measured as citations per article, and the impact rank. This is followed by the ranking for each journal when the then-current impact factors were published in 1983. The second group of columns shows the same data for the 1989-1990 articles, with the rank based on seven-year citation data and the then-current impact factor measured in 1991. With few exceptions, the top journals in terms of citations and productivity retain prominent rankings. The top 10-Cell, New England Journal of Medicine, Journal of Experimental Medicine, Journal of Cell Biology, Proceedings of the National Academy of Sciences, Archives of General Psychiatry, Journal of Clinical Investigation, Nature, Journal of Molecular Biology, and Science-are of the highest impact when measured over two-, seven-, or 15-year periods. Significant changes between current and cumulative impact rankings indeed do occur. Archives of General Psychiatry, Molecular and Cell Biology, EMBO Journal, and Circulation Research move up by 12 or more positions when one looks at long-term impact. Even more dramatic shifts occur for the Journal of Lipid Research, Journal of Histochemistry and Cytochemistry, and several physiology journals, including the American Journal of Physiology, Journal of General Physiology, Journal of Neurophysiology, and Journal of Physiology (London). On the other hand, significant downward changes in the rankings occur for such journals as Endocrinology, Kidney International, Journal of Virology, and almost all letters journals. These changes can be attributed to a variety of factors. For letters journals, one can assume that the authors went on to publish other work that superseded their earlier short reports. On the other hand, some journals may have improved long-term ranks owing to cumulative effects of a few highly cited "Citation Classics." More than one third of the citations to articles published in 1981-82 in Journal of Histochemistry and Cytochemistry were to a single article by S.M. Hsu et al. (29:577-80, 1981). My report entitled "The Significant Scientific Literature Appears In A Small Core Of Journals" (The Scientist, Sept. 2, 1996, page 13) listed the 50 journals that were most cited in absolute terms in 1994 and the 50 that published the largest numbers of articles. Many of these journals do not appear in the new lists ranked by long-term cumulative impact. These titles include the Journal of Geophysical Research, Physical Review B, Journal of Chemical Physics, Brain Research, and Biochimica et Biophysica Acta. Few would dispute the significance of these large journals in their respective fields, but further study is required to fully understand these data. The new data reported here show dramatic changes in impact rankings. How would these data affect journal selection based on current impact? Since meaningful comparisons can be made only within subject categories, the key question is whether these data affect rankings within a field such as physiology. Cross-disciplinary comparisons may not take into account the innate character of physiological research, in which advances may not be absorbed as rapidly as in other fields. Will journal rankings within categories differ significantly using a long-term impact? Or are current data good predictors of future rankings within the field? It is impossible without an article-by-article audit of each journal to make absolute comparisons. For example, more than 20 percent of the articles in Cell are reviews. This inflates its already high impact and ranking. Most other journals do not include this proportion of review articles. The New England Journal of Medicine does publish a large number of review articles, but most leading biomedical research journals do not. The data reported here are subject to human error, since they are a derivative of a large database. It is impossible to equate all publishing units involved, but I believe that the results reported will generally support independent peer-review judgments of the importance of these journals in contemporary life sciences. Every reasonable effort has been made to ensure accuracy, but the original data sources should be consulted to validate the results. Table: Chart 1 Chart 2

Long-Term Vs. Short-Term Journal Impact: Does It Matter?

Interested in reading more?

Become a Member of