A few years ago, Achal Agrawal, a data science researcher-turned research integrity expert, noticed a paper in his Google Scholar profile that he had not authored. The paper had been incorrectly attributed to him because it was written by someone with the same name.
While Agrawal manually corrected the error, he noted that everybody might not have a similar response. “Some people wouldn't mind someone else's paper being listed in [their profile] because it increases their h-index,” said Agrawal.
Achal Agrawal is a data science researcher-turned research integrity expert from India.
Achal Agrawal
To him, this highlighted a concern about the search engine. “It can easily be gamed,” said Agrawal. The more he thought about the platform, the more he recognized other potential issues. “I realized that [in] Google Scholar, you can get [a] lot of nice things, but you cannot find out the history of a person. You don't have nice analytics,” he said.
This motivated him to develop a platform that would provide deeper insight into a researcher’s publication footprint. The beta version of the tool, called Author Trends, launched a couple of weeks ago. It offers information about researchers’ normalized impact metrics and potential research integrity indicators, including retractions and published work in delisted journals.
“[This could be a] useful starting point for background verification for funding agencies, universities, and future collaborators,” said Agrawal.
Author Trends Offers a Multi-Source Integrity Screen
Author Trends draws on data from various sources, including OpenAlex, a platform that catalogs scholarly works from all over the globe. Using this information, the tool reports citations per paper, which are further weighted based on the number of authors in each article.
To include research integrity information for each researcher, Agrawal designed Author Trends to combine data from the Problematic Paper Screener, an automated tool that combs through hundreds of millions of papers every week to identify and flag problematic articles.1 The platform also scans the Retraction Watch database to indicate retractions and Scopus to identify any published work in journals delisted from the citation database due to suspicious activity.
“The idea is a good one. Pulling together a lot of information from different sources…is really helpful,” said Jodi Schneider, an information scientist at the University of Wisconsin-Madison, who was not involved in developing Author Trends. However, she noted that the immediate implications of the tool were not clear to her because people could still look into different sources to know more about a researcher.
She also cautioned that retractions, which help researchers correct the scientific record, need not always be red flags. “Having a small number of retractions is not necessarily problematic,” said Schneider. “There are Nobel Prize winners who have…intentionally retracted papers because they found problems.”
Agrawal agreed. “We also give a disclaimer that one or two cases of this kind of thing is not a red flag, but if you see a pattern, then you need to investigate more.”
Sifting Through Collaboration Networks
Author Trends also offers insight into researchers’ collaboration history, including the number of coauthors for their papers and the number of distinct coauthors per year. While a large number of coauthors once in a while could indicate a researcher’s work as part of a larger consortium, frequently publishing papers with hundreds of other researchers could potentially indicate paper mill activity, Agrawal noted.
He also explained that frequent new coauthors could also signal possible questionable research practices, “because in paper mills, whenever you buy a paper from some source, you don't choose your coauthors because someone else will be buying some other position.” And while it is common for researchers to form collaborations, it is unlikely that they will end up with different collaborators on each paper they publish, he said.
However, Schneider was not entirely convinced. While she agreed that a researcher consistently having new coauthors could potentially signal fraudulent practices, “over a long career somebody might work in different fields.”
Agrawal agreed that there are nuances. “You have to look at it more carefully. It's not very black and white,” he said. “But at least it gives you a starting point to see if there's potentially a problem, and then you can try to find out the exact [issue].”
Agrawal noted that the tool is not without limitations. Some of the data it relies on could be inaccurate, but he believes this will improve over time. He has also received feedback about some practical issues with the platform, such as merged profiles of two authors with the same names, but he noted there is a way to tackle this problem on the website and on the parent OpenAlex database containing this information. Going forward, he hopes that he and his team are able to include some additional features and that people all over the world use the tool.
- Cabanac G, et al. The “Problematic Paper Screener” automatically selects suspect publications for post-publication (re)assessment. arXiv. 2022
