WIKIMEDIA, PIETRO RICCIARDIThe Italian government is attempting to safeguard the quality of the nation’s science by evaluating researchers and institutions, and by rewarding those that meet certain criteria with opportunities for promotion and increased funding. But the move has angered many Italian scientists, who argue that the evaluations rely on flawed metrics that don’t accurately reflect research quality, and will therefore not reward the best scientists.
In an effort to root out nepotism in academia, the National Agency for the Evaluation of Universities and Research Institutes (ANVUR) implemented the new review scheme in 2011. Evaluators use three criteria: the number of papers published in the last 10 years, the number of citations accrued, and the “h-index” of those publications—a measure of both output and impact. Only applications that score above the national median on 2 of the 3 criteria can get the stamp of approval required to apply for higher positions. And only the top-ranking universities can expect to receive extra funding.
But many scientists argue that the criteria are too crude to identify the true leaders. Some groups—including the Mathematics Union and the Association of Psychologists—have launched official protests. “It's like judging the bottles in a wine contest by the labels only without tasting their content,” Alberto Baccini, a professor of political economics at the University of Siena, told ScienceInsider.
Critics say ANVUR methods for gathering bibliometric data are unreliable, and that even if they are calculated correctly, the data aren’t always a measure of research quality. Some lab leaders are listed as co-authors on dozens of papers for which they did very little work, for example, but ANVUR’s system will reward them with high scores. On the other hand, younger but more creative researcher may struggle to make the grade.
The ANVUR president Stefano Fantoni told ScienceInsider that the criteria will only be used as a guide, and that committees can still give the green light to researchers who fail to meet the minimum requirements. But the rules are not clearly stated, critics hold.