Scientists Take Aim at Impact Factor

A declaration asks the scientific community to put less weight on the metric, widely used to evaluate journals’ prestige.

May 20, 2013
Kate Yandell

FLICKR, ELVERT BARNESA declaration asking that the research community pay less attention to impact factor now has 484 signatures, 87 of them from institutions and 397 from individual members of the scientific community. Impact factor is calculated by Thomson Reuters based on academic journals’ citation rates, as a measure of prestige. The document, called the San Francisco Declaration on Research Assessment (DORA), asks that scientists, funding bodies, and others stop using the number as “a surrogate measure of the quality of individual research articles.”

DORA, set into motion at the Annual Meeting of The American Society for Cell Biology (ACSB) in December and spearheaded by the society, argues that, within journals, most citations are likely to come from relatively few papers and so the aggregate impact factors do not reflect an individual paper’s merit. Impact factors also vary by field and don’t differentiate between review papers and original research. And the emphasis encourages editors to enact policies to drive up their impact factors, sometimes artificially, the declaration said.

The declaration also asks that measures of research impact be transparently calculated and openly available for anyone to use. Thomson Reuters currently has not explained its full methodology and its data require permission to use.

“We, the scientific community, are to blame—we created this mess, this perception that if you don’t publish in CellNature, or Science, you won’t get a job,” Stefano Bertuzzi, executive director of the ACSB, told Nature. “The time is right for the scientific community to take control of this issue.”

Many organizations involved in scientific publishing have signed the declaration, including the Public Library of Science (PLOS), EMBO, eLife, and the American Association for the Advancement of Science (AAAS), the publisher of Science. Bruce Alberts, editor-in-chief of Science, wrote in an editorial that “Impact Factor mania makes no sense.” He argued that it discourages journals from publishing research from fields that tend to get cited less, inundates journals like Science with inappropriate submissions, and discourages risky and innovative research.

Nature did not sign the declaration. Editor-in-Chief Philip Campbell told a reporter from his organization’s Nature News Blog that “the draft statement contained many specific elements, some of which were too sweeping for me or my colleagues to sign up to.”

Thomson Reuters said in a statement that it didn’t see the declaration as a blanket condemnation of impact factors themselves, but rather how they are used. “Thomson Reuters continues to encourage publishers, researchers, and funders to consider the correct use of the many metrics available, including the Journal Impact Factor and data from the Web of Science, when performing research assessments.”