I Had a Dream ... about Uncitedness

My first paper proposing the creation of the Science Citation Index® (Science, 122(3159): 108-111, 1955) began with a quotation from P. Thomasson and J.C. Stanley: "The uncritical citation of disputed data by a writer, whether it be deliberate or not, is a serious matter. Of course, knowingly propagandizing unsubstantiated claims is particularly abhorrent, but just as many naïve students may be swayed by unfounded assertions presented by a writer who is unaware of the criticisms. Buried in scholarly journals, critical notes are increasingly likely to be overlooked with the passage of time, while the studies to which they pertain, having been reported more widely, are apt to be rediscovered." When the Science Citation Index (SCI®) was finally launched in the sixties, I dreamt that, one day, scholars would use it to routinely avoid the unwitting perpetuation of errors. Today I can still only hope that the World Wide Web and access to the SCI on Web of Science® will make that dream come true. But, in the meantime, I can only groan when I see errors perpetuated year after year. One particular case concerns a news story by David P. Hamilton that appeared in Science seven years ago ( Science, 250:1331-2, 1990 and Science, 251:25, 1991). Hamilton zealously criticized scholarship in the sciences and social sciences, but especially in the arts and humanities, using citation frequency data. He concluded, without qualifications, that huge percentages of the scholarly literature were never cited. His misguided reports on uncitedness have unduly influenced many scholars and policy makers ever since. His claims continue to be cited even though David Pendlebury subsequently published a factually correct rebuttal in a letter in Science (251:1410-1411, 1991), which was published with several others under the title "Science, Citation, and Funding." Pertinent extracts of Pendlebury's letter follow: "Hamilton's two articles about the percentage of journal literature that remains uncited within five years of publication require comment and further explanation. The figures reported by Hamilton--47.4 percent uncited for the sciences, 74.7 percent for the social sciences, and 98.0 percent for the arts and humanities--are indeed correct. However, as Maxine Singer was quoted as saying in Hamilton's article, it is necessary to know what is in the numbers before interpreting them. "These statistics represent every type of article that appears in journals indexed by the Institute for Scientific Information in its [Citation Indexes]. The journals which ISI indexes contain not only articles, reviews, and notes, but meeting abstracts, editorials, obituaries, letters like this one, and other marginalia, which one might expect to be largely uncited. In 1984, the year of the data quoted by Hamilton, about 27 percent of the items indexed in the Science Citation Index were such marginalia. The comparable figures for the Social Sciences Citation Index and the Arts and Humanities Citation Index were at 48 percent and 69 percent, respectively. "If one analyzes the data more narrowly and examines the extent of uncited articles alone (this information was not yet available when Hamilton wrote his articles), the figures shrink, some more than others: 22.4 percent of 1984 science articles remained uncited by the end of 1988, as did 48.0 percent of social sciences articles and 93.1 percent of articles in arts and humanities journals. ... "The figures originally quoted by Hamilton seem to have been interpreted by many readers as some sort of measure of the health of U.S. science. The numbers, however, reflect a lack of citation of papers by authors the world over, not only those by U.S. researchers. ... "If one restricts the analysis even further and examines the extent of uncited articles by U.S. authors alone, the numbers are even less "worrisome." Only 14.7 percent of 1984 science articles by U.S. authors were left uncited by the end of 1988. We estimate the share of uncited 1984 articles by non-U.S. scientists to be about 28 percent. ... "A certain level of 'uncitedness' in journal literature is probably more an expression of the process of knowledge creation and dissemination than any sort of measure of performance. A trend toward more or less 'uncitedness,' however, might be meaningful. For the 1980s, we see no such trend in the scientific literature: the numbers are essentially flat, both for the United States alone and for the world. ... "We hope this information clarifies the record and will end further misunderstanding or politicization of these statistics." As Pendlebury indicates, and as we all know, journals such as Science, Nature, and New England Journal of Medicine publish many high-impact research articles, but they also publish many other editorial items, such as letters, obituaries, and book reviews. All of these are indexed in the Science Citation Index. When one quantifies the output and utility of journals, research and review articles must be differentiated from the rest. When ISI calculates journal impact factors, only "substantive" items are included in the counts. So when one describes uncitedness, one must specify which editorial categories are included in the calculation. In Pendlebury's data, uncitedness is defined in terms of ISI's journal coverage. Since most authors sooner or later mention their own work in a review or as part of an ongoing series, it is indeed remarkable how many papers are never cited. Due to the cumulative character of science and scholarship, a great deal of the literature is cited but once, simply because supercedence is a fundamental characteristic of the literature. To get an idea of how often "onesies" occur, consider this: In a study by SCI for the years 1945-88, almost 56 percent of all types of publications (papers, books, etc.) were cited just once. However, that is inflated, because many references include typographical errors or spelling or pagination variations that defy easy unification. A small group of journals account for more than 90 percent of significant research. The overwhelming majority of articles published in the 200 journals with highest cumulative impact (The Scientist, 12[3]:11-12, Feb. 2, 1998 and pages 12-13 of this issue) are cited within a few years of publication, and after five years, uncitedness is almost nonexistent.

I Had a Dream ... about Uncitedness

Interested in reading more?

Become a Member of