Impact Forecasts Cloudy

Current models do a poor job of predicting the impact individual scientists will have, according to a study.

By | October 31, 2013

FLICKR, DANIEL WEHNERWhen a hiring committee or a grant study section evaluates a candidate, it must look at past achievements as a predictor of future success. In addition to other metrics, these groups are increasingly using the h-index, which is a score calculated based on one’s publication and citation records. But researchers from Aalto University School of Science in Finland and the IMT Institute for Advanced Studies in Italy have shown that models based on the h-index are flawed, and that they do a poor job of predicting the impact that a scientist will have in the future. Their work was published this week (October 29) in Scientific Reports.

The researchers used publication and citation data from 476 physicists, 236 cell biologists, and 50 mathematicians and also calculated those researchers’ h-indices. By comparing these three metrics, they showed that for each, “the correlation between past and future is similar,” the researchers wrote. “Thus our analysis suggests that all these measures are equally good (or equally bad) in predicting future impact,” they concluded. The researchers also showed that “predictive power” of the h-index did not predict the impact of early career researchers well. Because the h-index is cumulative, if it was high early on in a scientist’s career, it was also likely to be high later. Though there may be a correlation between early and later success, the authors wrote that it “will require a highly sensitive and powerful approach” to discover it.

“Based on our results, the predictability of current models for real application in recruitment decisions is questionable,” coauthor Santo Fortunato said in a press release. “Efforts to model future impact need to be aimed more directly at applications in the career advancement decision process.”

Add a Comment

Avatar of: You



Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Popular Now

  1. Monsanto Buys Rights to CRISPR
    The Nutshell Monsanto Buys Rights to CRISPR

    The US agribusiness secures a global, nonexclusive licensing agreement from the Broad Institute to use the gene-editing technology for agricultural applications.

  2. How Plants Evolved Different Ways to Make Caffeine
  3. Thomson Reuters Predicts Nobelists
    The Nutshell Thomson Reuters Predicts Nobelists

    According to citation statistics, researchers behind programmed cell death pathways and CRISPR/Cas9 are among those in line for Nobel Prizes this year.

  4. ESP on Trial
    Foundations ESP on Trial

    In the 1930s, parapsychologist Joseph Banks Rhine aimed to use scientific methods to confirm the existence of extrasensory perception, but faced criticisms of dubious analyses and irreproducible results.