New Journal Ratings Questioned

A new system of ranking scientific journals irks some metrics experts.

By | October 5, 2011

WIKIMEDIA, VMENKOV

A novel scheme for rating the relative impact of scientific journals, unveiled this week by post-publication peer review outfit Faculty of 1000 (F1000), is being questioned by scholarly publication experts. The rankings, which place Nature on top of the list for biology journals and the New England Journal of Medicine atop the medicine heap, were built using scores awarded to papers published in 2010 in thousands of journals by F1000's 10,000-stong "faculty" of researchers and clinicians.

While the upper echelons of F1000's rankings, which include Cell, Science, PNAS, and Lancet, more or less correspond with rankings of the same journals based on their impact factors—a metric calculated via an algorithm that takes citation frequency, among other factors, into account—the list contains some surprises further down in the pack, the validity of which is in question. For example, Nature reported that one lesser-known journal "seems to owe its surprisingly high ranking to a series of very positive evaluations of its articles by its own editor."

F1000 editor Richard Grant (no relation to author) told Nature that the journal editor should have reported such a conflict, and that journal in question was yanked from the ratings while F1000 investigated the matter.

Critics are also decrying the system's dependence on scores awarded to papers by working researchers, who may carry their own biases into the process. "The scores may tell us as much about the composition of the F1000 faculty as they do about the relative quality of various journals," University of Washington, Seattle, biologist and F1000 faculty member Carl Bergstrom told Nature. Bergstrom publishes a rival ranking method called the Eigenfactor.

Full Disclosure—F1000 is owned by the same company, Science Navigation Group, as The Scientist.

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: Phil_Davis

Phil_Davis

Posts: 10

October 6, 2011

As I argue in a more complete post (http://wp.me/pcvbl-5IX) the real value of F1000 is not what
the aggregate data can tell us about individual journals, but in what
experts can tell us about individual articles.

Avatar of:

Posts: 0

October 6, 2011

As I argue in a more complete post (http://wp.me/pcvbl-5IX) the real value of F1000 is not what
the aggregate data can tell us about individual journals, but in what
experts can tell us about individual articles.

Avatar of:

Posts: 0

October 6, 2011

As I argue in a more complete post (http://wp.me/pcvbl-5IX) the real value of F1000 is not what
the aggregate data can tell us about individual journals, but in what
experts can tell us about individual articles.

Popular Now

  1. How Plants Evolved Different Ways to Make Caffeine
  2. Thomson Reuters Predicts Nobelists
    The Nutshell Thomson Reuters Predicts Nobelists

    According to citation statistics, researchers behind programmed cell death pathways and CRISPR/Cas9 are among those in line for Nobel Prizes this year.

  3. Monsanto Buys Rights to CRISPR
    The Nutshell Monsanto Buys Rights to CRISPR

    The US agribusiness secures a global, nonexclusive licensing agreement from the Broad Institute to use the gene-editing technology for agricultural applications.

  4. Reviewing Results-Free Manuscripts
    The Nutshell Reviewing Results-Free Manuscripts

    An open-access journal is trialing a peer-review process in which reviewers do not have access to the results or discussion sections of submitted papers.

RayBiotech