The Lancet has announced changes to its editorial policies following the controversial publication and retraction earlier this year of a paper on COVID-19 patients treated with the antimalarial drug hydroxychloroquine. The changes, described in an comment entitled “Learning from a retraction” last Thursday (September 17), include alterations to peer review and other paper acceptance procedures, and have prompted mixed responses from the scientific and science publishing community.
“The proposed changes are a welcome addition, and I am glad to see Lancet implementing them,” Mario Malički, co–editor-in-chief of the journal Research Integrity and Peer Review, writes in an email to The Scientist. “However, they do not answer the questions at the core of the peer review process. How is it that the Lancet’s editorial team and reviewers (including a statistical reviewer) missed what the research community saw when the paper came out, [and] how do the new proposed changes ensure that Lancet or other top journals indeed consult the best reviewers for the studies submitted to them?”
The Lancet attracted criticism in May for publishing the hydroxychloroquine paper, which was purportedly based on a huge database of electronic medical data managed by Surigsphere Corporation, a tiny Illinois-based company owned by study coauthor and vascular surgeon Sapan Desai. The paper reported a link between treatment with hydroxychloroquine and increased mortality in hospitalized COVID-19 patients and had fast and far-reaching effects on clinical research and global health policy, but fell apart under scrutiny from scientists and journalists within days of its publication.
That paper and another study based on the same database published in The New England Journal of Medicine were retracted on June 4, after independent auditors said that they’d been unable to access and verify the company’s data. Surgisphere itself disappeared a couple weeks later amid accusations of scientific fraud, with its website going offline and Desai ceasing to respond to The Scientist’s requests for comment.
Speaking to The Scientist in the wake of the scandal this July, The Lancet’s editor-in-chief, Richard Horton, said that the journal would review its policies and aim to implement changes that would strike a balance “between learning the lessons but also not over-responding, because you don’t want to impose another layer of bureaucracy on science that actually makes it more difficult either to do science or to publish science.”
A process which reflects the capabilities of modern, on-line journals would provide a record of post-publication review and would recognize the work of external reviewers.—Matthew Spinelli, University of California, San Francisco
Among the changes announced last week are alterations to author declaration forms. “All authors will be asked to sign the author statements form to confirm they had full access to the data . . . and accept responsibility for submitting the Article for publication,” the comment reads. In the hydroxychloroquine study, only one author, Amit Patel—a physician and Desai’s relative—signed such a statement.
Authors will be required to include a data-sharing statement, too, detailing what data will be shared and when. While this was already a requirement for clinical trials, the changes make this obligatory for all research papers, and “editors will take data-sharing statements into account when making editorial decisions,” the comment says.
The journal will also introduce “additional peer-review requirements,” including ensuring that “at least one peer reviewer is knowledgeable about the details of the dataset being reported”—for example, by recruiting a data scientist for studies with very large datasets—and explicitly asking reviewers if they have concerns regarding ethics or research integrity.
The changes are “a great first step” toward improving transparency and author responsibility in the publication process, says Daniel Kulp, the vice-chair and chair-elect of the nonprofit Committee on Publication Ethics (COPE), which develops guidance for publishers. “It’s very important for [journals] to have not only a policy for correcting the literature,” he says, “but a process by which they can investigate how things happen and if there are ways of reducing the possibility of repeating errors. I’m completely in favor of that.”
Kulp adds that while journals can’t guarantee against the publication of bad research, The Lancet’s move should help push scientific publishing toward greater openness. He says he hopes that the journal will reflect on the effects of the changes a year or so from now “to see if it’s accomplished what they wanted—and if not, then tweak it again in whatever direction they need to. These things should be dynamic.”
James Watson, a senior scientist at the Mahidol Oxford Tropical Medicine Research Unit (MORU) in Thailand who in May organized an open letter listing concerns about the hydroxychloroquine study, tells The Scientist in an email that the new data-sharing statements in particular are an important change, “although Lancet are pretty late to the game,” as many other publishers already require such statements.
He adds that “making it clear that the data sharing statement will be used by the editors to evaluate a new submission is a great incentive for people to think hard about data sharing and find clever ways of doing it.” MORU’s own study of hydroxychloroquine as COVID-19 prevention in healthcare workers was suspended following The Lancet paper’s publication, although it has since been reinstated.
Like Malički, Watson and other researchers note that the changes don’t address larger, more general issues highlighted by Surgisphere’s papers, including journals’ reluctance to push authors to share data and code for published studies, and an overall lack of transparency in how papers are reviewed before and after publication.
University of California, San Francisco (UCSF), infectious diseases researcher Matthew Spinelli, for example, tells The Scientist in an email that he is particularly disappointed by “the lack of acknowledgement of post-publication review,” without which, he adds, “this fraud would never have been detected.”
Spinelli and UCSF biostatistician David Glidden were among many scientists to write letters to The Lancet with concerns about the paper within days of its publication. Despite accepting several of those letters, The Lancet dropped plans to publish them after retracting the original paper, with Lancet senior editor Josefine Gibson writing to authors that, “in the light of the retraction, the Editor in Chief and I no longer feel that we can offer a platform for these questions and discussion.”
Spinelli and Glidden tell The Scientist that journals should put more emphasis on the conversation that takes place after a paper is published. “A process which reflects the capabilities of modern, on-line journals would provide a record of post-publication review and would recognize the work of external reviewers,” Spinelli writes. “This process would be more accountable, fairer, and more transparent.”
The New England Journal of Medicine wrote in a statement to The Scientist in July that it would also be reflecting on its editorial procedures in light of its own Surgisphere retraction, but has so far declined to provide detail about specific changes.