Can Publication Records Predict Future PIs?

While having papers in Nature and Science can certainly help, it turns out that high-profile publications are not the only factors that determine whether an early-career scientist will one day lead her own academic lab, the team found. Rather, it’s the total number of publications, the impact factors of the journals in which they’re published, and whether each paper meets or exceeds the average number of citations for a given manuscript in that journal that seem to matter most. In other words, quantity and quality count. Overall, the researchers noted, higher h-indices—metrics that attempt to quantify the productivity and impact of a scientist’s publications—are predictive of a greater chance of academic career success, lending support to a concept first proposed in 2012 by Rehabilitation Institute of Chicago’s Daniel Acuna and his colleagues in Nature.

“However, both the scientist’s gender and the rank of their university are also of importance, suggesting that non-publication features play a statistically significant role in the academic hiring process,” Carey and his colleagues wrote in their paper. The researchers found that, given the same publication record and all else being equal, male authors are more likely to become PIs than their female counterparts. Their model controls for both gender and institution rank.

Randall Ribaudo, CEO and cofounder of the career training firm SciPhD.com, spent five years as a PI at the National Cancer Institute’s Laboratory of Immune Cell Biology in Bethesda, Maryland, before moving on to work in industry. According to PIPredictor, he currently has a 59 percent chance of becoming a PI, Ribaudo told The Scientist. He questioned whether the team’s tool could account for factors such as the academic job market, which has changed considerably during the last few decades. “Over the past 20 years, hiring for tenure-track positions has gone down a lot,” he wrote in an e-mail. “If [the authors] are using longitudinal data and not considering that downward trend over time, the better statistical likelihoods in earlier years could be giving artificially high numbers.”

Paula Stephan, who studies the scientific workforce at Georgia State University, agreed. “The times are changing, and with that, the underlying probability of becoming a PI,” she wrote in an e-mail to The Scientist.

Stephan added that while the model is “based on a well-constructed bibliometric database,” publication records alone cannot account for “factors that reflect the scientist’s ability to produce the type of research that may be funded, such as innovativeness [and] creativity.”

Even so, in an increasingly competitive job market, it could be beneficial for a young scientist to get a feel for where her publication record stands. “The tool provides one benchmark that early career scientists can use to see their relative position based on publication metrics and reputation of university,” said Stephan. “I suspect that many young scientists already have a good idea of where they are, and some a more accurate idea than this [tool] can provide within their narrowly defined research world.”

D. van Dijk et al., “Publication metrics and success on the academic job market,” Current Biology, 24(11), 2014.