Supplemental or detrimental?

Last November, Journal of Neuroscience abolished supplemental materials -- the extra figures and tables that appear online, but not in the printed version of a paper. Editor-in-chief linkurl:John Maunsell;http://f1000.com/thefaculty/member/305132154309970 argued linkurl:in an editorial;http://www.jneurosci.org/cgi/content/full/30/32/10599 that the escalating amount of supplemental materials had begun to devalue the peer review process.

Image: Wikimedia commons, Autopilot

The decision highlights a tension between the need for rigorous peer review of scientific research and the desire to provide as much data as possible to the scientific community."More data, in and of itself, is always a good thing -- if there aren't adverse effects," said Maunsell, who is also a neuroscientist at Harvard University. But peer review was becoming less effective because many reviewers failed to evaluate the supplemental materials, which the journal wasn't even required to provide, he explained. "We were taking a hit on peer review for something that wasn't formally our responsibility."Usually embraced as a benefit of publishing online, supplemental materials allow researchers to show additional data in the form of figures, tables, video or audio clips. Short format journals like Nature or Science rely heavily on supplemental materials to include data that don't fit into the severe space limitations of their articles. But Journal of Neuroscience doesn't limit the number of figures in a paper, and still supplemental materials have grown over the last ten years to rival the amount of material in the main body of a paper. The ease of getting this material online led to what Maunsell refers to as a "supplemental arms race" in which reviewers started asking for more data, and authors responded by including more data to immunize themselves against these demands.But the reviewers weren't keeping up, Maunsell said. While some said they had evaluated the supplemental materials, others said they hadn't, and still others didn't say. "We can't afford to let peer review get devalued quietly without attention to it," Maunsell said.Seeking to clarify what had been reviewed, and concerned by the amount of time reviewers spent on supplemental materials compared to the main paper, they decided to do away with supplemental materials altogether. Instead, authors can publish one URL that refers to supplemental materials on a Web site maintained by the authors.Some have decried this decision as a step backwards for data transparency, which advocates making more data available -- not less. "The benefits outweigh the risks," said Iain Hrynaszkiewicz, a journal publisher at open-access publisher BioMed Central, who wrote a defense of supplemental materials when Journal of Neuroscience announced its decision.But more and more journals are beginning to disagree. In March 2010, Neuroscience instituted a similar policy to do away with supplemental material, except movies and audio clips. And in October 2009, Cell Press introduced a policy that allowed only one supplemental figure per figure in the main body of a paper, which must directly support a point made in that figure. "It had become a limitless bag of stuff," said linkurl:Emilie Marcus,;http://www.elsevier.com/wps/find/newsroom.newsroom/bio_emiliemarcus editor-in-chief of Cell Press journals.The publisher did not consider abolishing supplementary materials altogether because they have a diverse readership, with different levels of interest in a study's details, Marcus explained, but it was necessary to rein it in. "I do think there are different solutions for different journals," Marcus said. "Scientific communities and journals have probably not given enough thought to what to do with this capacity for supplemental materials. That needs to evolve."Hrynaszkiewicz agreed that the hodgepodge of content currently found in supplemental materials is a far cry from a standardized format for sharing data, but argued that it can act as a stopgap until something better becomes available."When there aren't any other options in certain fields, then publishing data online with the journal seems to be a good interim measure," he said.But both Marcus and Maunsell say that their supplemental material policies have been well received. And Maunsell hasn't yet detected a change in number or length of the manuscripts submitted."There was a large contingent of authors and readers who felt that this was stuff they didn't need," he said.
**__Related stories:__***linkurl:Publish or post?;http://www.the-scientist.com/blog/display/57613/
[9th August 2010]*linkurl:I Hate Your Paper;http://www.the-scientist.com/2010/8/1/36/1/
[August 2010]*linkurl:Peer review trickery?;http://www.the-scientist.com/blog/display/57130/
[2nd February 2010]*linkurl:New site pits 'published' vs. 'posted';http://www.the-scientist.com/news/display/53294/
[19th June 2007]