FLICKR, MINISTERIO TIC COLOMBIAWhen its anonymous founders launched PubPeer in 2012, they intended to create a space for open, forthright discussions of the scientific literature. But they were surprised at the volume of posts—from other anonymous users, as well as those posting under their names—pointing out seemingly significant issues with published papers. “The great majority of comments point out some kind of problem,” PubPeer moderators wrote in an e-mail to The Scientist. “And a large proportion of those seem potentially serious to us—e.g. duplication of gel bands that would rarely occur by accident.”
Some papers that have been criticized on PubPeer have since been corrected or even retracted. But in other cases, critiques remain on the site for many months without any apparent resolution.
For instance, Paul Brookes, a mitochondrial researcher at the University of Rochester Medical School in New York, this February commented on a 2011...
The PLOS Biology commenter said there appeared to be “some errors” in Western blot figures, including similar-looking control bands meant to represent different conditions. The paper has been cited 66 times, according to the journal’s publisher, the Public Library of Science (PLOS). A day after Brookes posted on PubPeer in February, PLOS user “caricilli”—presumably the paper's first author Andrea Caricilli, who completed the work as a graduate student at the State University of Campinas in Brazil—responded to the commenter’s note, explaining that the journal and authors were looking into the issues. Caricilli did not respond to The Scientist’s request for comment.
Brookes most recently posted to PubPeer in late July about the apparent lack of resolution of the issue. “Stuff is just taking way too long to be corrected,” Brookes recently told The Scientist.
“PLOS is actively working toward a resolution, which will be made public when reached,” wrote PLOS’s public relations manager, David Knutson, in an e-mail. “However, this process does take time and is confidential.”
Corresponding author Mário Saad of the State University of Campinas told The Scientist that he and his colleagues have spotted errors in their figures and submitted a correction to the journal around March. “We recognize that there are unintentional mistakes in figures of that paper that we attribute to a not very rigid process of editing the manuscript by the authors,” Saad wrote in an e-mail to The Scientist. “These mistakes do not change one line of our manuscript, which was reproduced many times in our lab, and similar results have been reproduced over and over.”
Of course, in some cases, comments made on papers go unnoticed by the journals that published them.
In a separate thread posted to PubPeer in February, an anonymous commenter pointed out that two sets of data in a June 2011 Nature Protocols paper appeared surprisingly similar. The lead author was RIKEN’s Haruko Obokata, whose work on stimulus-triggered acquisition of pluripotency (STAP) stem cells was retracted earlier this year.
However, it was not until earlier this month that Nature Protocols noticed the comment after University of Tokyo seismologist Robert Geller sent the journal’s editors an e-mail pointing out the apparent irregularities after friends had alerted him to the PubPeer comment.
“Thank you for the note and for flagging up the issue, which we were not aware of and are now looking into,” Nature Protocols responded to Geller on Twitter last week (August 18).
In an e-mail to The Scientist, Veronique Kiermer, executive editor at Nature Publishing Group, explained that while it is not Nature’s policy to monitor PubPeer, “we investigate all serious allegations that are brought to our attention, including anonymous reports.”
Editors at Cell and PNAS also told The Scientist that they do not have policies in place to monitor comments made on PubPeer. “We have found the concerns in PubPeer to be mixed in terms of accuracy,” wrote Daniel Salsbury, deputy executive editor at PNAS.
“Some of the editors may monitor PubPeer,” wrote Science’s executive editor, Monica Bradford. “However, we have not set up a routine, consistent process for monitoring the site.”
Knutson of PLOS noted that his organization does “keep an eye on PubPeer, but we would prefer that readers alert the editors directly about any concerns they may have about PLOS publications.”
Currently, authors of papers discussed on PubPeer are automatically sent e-mail notifications. PubPeer’s organizers said that some journals have enquired about adding a feature to notify them when papers published in their pages are discussed. The organizers plan to add automated notifications for journals as a site feature soon. “However, our experience so far has been that journals would rather ignore PubPeer, because it represents unwanted work and negative publicity for them,” they said.
“The rise of sites like PubPeer is giving people a chance to point out problems without having to put their life/career on the line,” Geller wrote in an e-mail to The Scientist. “But there needs to be a better mechanism for getting the journals and the authors’ institutions to pay attention.”
Interested in reading more?
The Scientist ARCHIVES
Become a Member of
