ABOVE: © ISTOCK.COM, KENANOLGUN
Ideally, scientific publishing is an objective endeavor in which manuscripts are held to high standards of review to ensure accuracy and guard against conflicts of interest that could compromise a study’s trustworthiness. Yet, as Retraction Watch and occasionally other outlets document, it’s not uncommon for poor-quality, or sometimes fraudulent or nonsensical, papers to gain the imprimatur of publication in a peer-reviewed journal.
See “The Top Retractions of 2020”
A study published today (November 23) in PLOS Biology points to potential favoritism that could be present within specific journals’ editorial procedures, allowing less-than-stellar papers through. Clinical pharmacologist Clara Locher and a team of researchers from the University of Rennes in France examined nearly 5 million papers published between 2015 and 2019 in 5,468 journals and found that while a majority of journals carried publications distributed across a large number of authors, five percent of journals had a single, highly prolific author that was responsible for at least 11 percent of the published articles in the journal. Furthermore, in a random sample of this subset of journals, the highly published author was a member of the editorial board 61 percent of the time and their papers were accepted in a median time of three weeks after submission, a much speedier rate than the typical 100+ days reported in a Nature article about studies published in Nature and PLOS ONE (no assessments of paper quality were performed). The authors of the new survey argue in their paper that more transparency is needed around journals’ editorial practices.
See “Revealing Peer Review Identities Could Introduce Bias: Study”
In an email interview with The Scientist, Locher discussed her survey and what the findings suggest about certain biomedical research journals.
The Scientist: What inspired you and your team to investigate favoritism in research publications?
Clara Locher: We started exploring favoritism in research publications following up the hydroxychloroquine saga. . . . A common thread among the first articles supporting the use of hydroxychloroquine in COVID-19 was that these articles were all published in journals where at least one author was part of the editorial board or, indeed, editors-in-chief. Furthermore, the time between submission and acceptance was unusually short while articles were below general research standards. Altogether, these elements cast doubt on the quality of the editorial process.
See “Journal Publisher Concerned over Hydroxychloroquine Study”
TS: Can you walk me through your process of assessing favoritism in publications? How did you decide what to assess regarding authorship data?
CL: Among articles supporting the use of hydroxychloroquine, a so-called meta-analysis was published in New Microbes and New Infections while the scope of this journal really does not match with therapeutic issues. So, we took a closer look at this journal and found that 35 percent of all articles were published by at least one author on the current editorial board. A value that is not expected!
Looking at other authors of the meta-analysis, we found that Didier Raoult has signed [authored] 235 of the 728 articles published in New Microbes and New Infections, making him the most prolific author of this journal. Didier Raoult is not part of the editorial board, but as the director of the Institut Hospitalo-Universitaire Méditerranée Infection, six members of the editorial board ([including] the editor-in-chief and the deputy editor-in-chief) report to him. Coincidently, Dorothy Bishop reported a similar analysis in her blogpost on the field of psychology. These convergent analyses led us to consider the ‘percentage of papers by the most prolific author’ as a potential warning to identify journals that are suspected of dubious editorial practice.
TS: Building off of that, what is the Gini index and what does it tell you about authorship in journals?
CL: The first warning that we identified, the ‘percentage of papers by the most prolific author,’ focuses on a single author and is sensitive to the number of annual publications of the journal. This is why Alexandre Scanff suggested completing it with the Gini index, a statistical measure widely used in econometrics to [evaluate] the level of inequality in the distribution of income. In our study, ‘income’ corresponds to the number of articles signed by authors in a given journal. The advantage of this measure is that it allows identifying journals where a group of authors monopolizes authorship.
TS: Did any of your results surprise you?
CL: Yes and no! The impression is that we might have cases, for different reasons, of favoritism in the editorial decision-making, and it isn’t new. What is new and surprising is that in a subset of journals, indexed in the NLM [National Library of Medicine] catalogue, a few authors, often members of the editorial board, were responsible for a disproportionate number of publications.
TS: What are some limitations of this study?
CL: The main limitation of our study is that these quantitative metrics are not sufficient to assert that there is biased editorial decision-making. These metrics must be seen as warnings which should lead to a more detailed analysis of the journal: qualitative analysis of papers published in this journal, and inspection of the place of prolific authors in the editorial board. This careful inspection of the journal should allow to eliminate false positives represented by active editors and/or professional journalists.
On the other hand, these quantitative metrics might only point the tip of the iceberg by identifying only the extreme cases. This is particularly the case for the ‘percentage of papers by the most prolific author’: the more the number of articles published by a given journal increases, the more difficult it is for an author to sign 10 percent or more of them.
TS: Why does editorial bias and potential nepotism in research journals harm the research community, and in particular the biomedical research community?
CL: As long as researchers will be rewarded according to productivity metrics, favoritism in journals’ editorial procedures could be viewed as unethical. In fact, these journals could be used to increase productivity-based metrics such as number of publications, number of citations, with a positive effect on decisions about promotion, tenure and grant funding. Furthermore, editorial board members might use their position to publish articles that do not reach the required quality standard for publication. In the case of this biomedical research, that may have negative consequences on evidence-based medicine, as we have all witnessed, during the case of hydroxychloroquine and COVID-19.
TS: What are future steps you want to take from this research?
CL: Our survey provides information about the broad scene of what we call ‘nepotistic journals.’ The next step is therefore to describe this phenomenon [in a fine-grained way], notably by studying the quality and the integrity of publications by editors in their own journal.
TS: What do you hope readers and research publications and their editorial boards will take away from your findings?
CL: We hope that both readers and editorial boards will become aware of the need to enhance trust in editorial practices. For that reason, journals need to be transparent about their editorial and peer review practices.
Editor’s note: This interview has been edited for brevity.