From extending lifespan to bolstering the immune system, the drug’s effects are only just beginning to be understood.
How to navigate the choppy waters of scientific publication
March 1, 2015|
© ISTOCK.COM/WRAGGIn July 2012, while attending the 12th International Coral Reef Symposium in Cairns, Australia, Emily Darling received a heart-wrenching e-mail: Ecology Letters had rejected her paper on coral life histories. She was shocked. The paper had already gone through one very positive round of reviews, and Darling, then in the final months of her PhD program at Simon Fraser University outside Vancouver, Canada, had resubmitted it with the requested changes. Just the day before, a senior colleague at a cocktail hour had revealed to her that he was one of the reviewers and that he had recommended that the journal publish her paper. Another reviewer had clearly disagreed.
“It felt like the worst place in the world that I ever could have got the rejection,” says Darling, who received the e-mail in a conference room full of her peers. “I’m looking around at this amphitheater of people and saying, ‘Who did it?’ It sort of personalized the rejection experience.”
But Darling’s advisor, Isabelle Côté, wasn’t about to take the rejection without a fight. The pair tracked down the paper’s editor, who was also at the conference, and learned that the negative reviewer had been added in the second round because two first-round reviewers had been unavailable. Darling and Côté asked the editor if he would consider a letter rebutting the negative reviewer’s criticisms, then tracked down the positive reviewer Darling had met, who agreed to send a letter vouching for the paper. A couple of months later, after the authors addressed some legitimate critiques from the positive reviewer and submitted a third draft, the paper was published (Ecol Lett, 15:1378-86, 2012).
Coping with publication rejection is a key skill for any academic. It can require emotional fortitude, an ability to profit from criticism, and, sometimes, negotiating skills. Many papers in the scientific literature were initially rejected, often sparking vigorous scientific discussions before their eventual publication, whether in the same journal or a different one.
Most papers can be much improved by more work on them. When we are rejected, we are forced to do that.—Vincent Calcagno,
French Institute for Agricultural Research
“It does really take until you are a postdoc to start to be able to reflect more broadly on rejection and see the forest for the trees,” says Darling, who is now a David H. Smith Conservation Research Fellow at the University of North Carolina at Chapel Hill. “You’ve done it enough by then that it’s not a knife in the back every time a rejection comes in.”
Here, The Scientist gathers researchers’ strategies for navigating the peer-review process.
One of the first decisions researchers have to make when preparing to publish a manuscript is where to submit. “I think it’s important to know the type of paper that you have and who would be potentially interested in it,” says cognitive neuroscientist Jane Wang, who is currently a postdoc at Northwestern University’s Feinberg School of Medicine. “You don’t want to waste your time going for journals that are not even going to take a [serious] look at it.”
Jason McDermott, a computational biologist at the Pacific Northwest National Laboratory in Washington State, learned that lesson firsthand as a postdoc, when he and his mentor were convinced his paper was worthy of a top-tier journal. In the end, the paper was never published. “It got rejected [at least five] times and then it went off to die,” says McDermott, who suspects it would have been published had they aimed their sights at a lower-impact publication.
For the most part, however, scientists have a good sense of where their papers are most likely to be accepted. Of more than 80,000 papers published in 923 biology journals from 2006 to 2008, 75 percent were published in the first journal they were submitted to, according to a 2012 study (Science, 338:1065-69). Lead author Vincent Calcagno conceived the work while he was dealing with a string of rejections of a paper from his PhD work at the University of Montpellier in France. Calcagno, now an evolutionary biologist and ecologist at the French Institute for Agricultural Research, says he was pleased to see that so many scientists reported acceptance by the first journal to which they submitted their research. “It made me more confident in the process . . . because rejection was not the rule.”
For early-career scientists who are not yet sure where to submit, asking senior colleagues for suggestions is a good way to start. It may also be possible to gauge a journal’s interest by providing a short summary of the paper before going to the trouble of a full submission. Some journals have formal presubmission processes. Nature, for instance, says that editors usually respond to presubmissions in two working days. In other cases, it’s possible to more informally ask editors about their interest in a paper. Additionally, a couple of new companies now offer to do the work for you, sending out papers to independent reviewers before they are ever submitted to a journal. (See “A Helping Hand.”)
Once you’ve selected the journal you’re going to submit to, it’s time to format the paper’s figures and text to meet the journal’s requirements. All journals have a set of submission standards. Check guidelines on length, structure, and technical term definitions, and read through past articles from the journal to get a sense of style and tone.
Once you submit, it becomes a waiting game. In some cases, the journal editors will decide not to send the paper out for review. This decision should be made within a few weeks. If your manuscript passes this first evaluation, the peer-review process can take anywhere from weeks to months. Many journals, including the Cell Press research journals, PNAS, and Scientific Reports, say that they try to respond to submissions within three weeks to a month. Others set more expansive limits or don’t specify response times. The Journal of Ecology, for instance, promises to either respond to submissions within three months or explain why the process is delayed.
Such wait times may be an important factor to consider in deciding where to submit. Earlier this year, Santiago Salinas of the University of California, Santa Cruz, and Stephan Munch of the National Oceanic and Atmospheric Administration published a mathematical strategy to weigh citation numbers, acceptance rates, and average review times of 61 ecology journals (PLOS ONE, doi:10.1371/journal.pone.0115451, 2015). One could choose to maximize citations over a given time period, or to minimize number of resubmissions or total review time.
If the journal’s long-awaited response is the dreaded rejection letter, take a deep breath: read through it to get a sense of its major points, then step away. “The emotion is high when you get a negative decision,” says Lakshmi Goyal, editor of the journal Cell Host and Microbe and an editor at Cell Press since 2001. “[Later], go back and . . . look rationally at what is being asked and how it can be addressed.”
While some reviews can be snide or dismissive, others offer well-reasoned advice. The trick for getting the most out of reviews, Wang says, is to assume the best of the reviewer. “I never go at it with the mind-set that they are against me,” she says. “I always try to assume they are trying to help me.”
Nikolaus Grigorieff, a structural biologist at the Howard Hughes Medical Institute’s Janelia Research Campus in Ashburn, Virginia, points out that, like any type of written communication, manuscript reviews often do a poor job of portraying tone. A simple list of a paper’s shortcomings can come across as extremely harsh, when the reviewer may have simply been in a rush. And even if a review is overtly scathing, it’s still important to look beyond the nasty comments, says Grigorieff, who hosts an online repository of rejected papers, mainly from his own lab, and their reviews. “Sometimes reviewers are very disrespectful. You have to ignore that part and just extract the information that is there.”
Indeed, while rejection is frustrating, it can ultimately benefit a paper. Calcagno’s Science paper analyzing rejection showed that manuscripts that had been rejected from one or more journals before ultimately finding a home tended to be cited more than papers that sailed directly into the pages of that same journal. “Most papers can be much improved by more work on them,” Calcagno says. “When we are rejected, we are forced to do that.”
Also key to getting the most out of a review is to not approach comments too literally, Cell Press’s Goyal advises. If a reviewer proposes an unrealistic new experiment, the author should try to understand why the reviewer is asking for the experiment and then try to find another way to answer the question. It is a mistake to dismiss criticisms outright, even if a reviewer is making inappropriate suggestions, Goyal says.
Darling agrees that taking time with reviewers’ comments is worth it. “The worst thing you can ever do in my opinion is just resubmit the same paper to another journal without addressing the comments at all,” she says. Editors’ comments should also be taken seriously, she adds, as they often have years of experience in both science and publishing.
She recalls that the first paper she ever submitted was a very long manuscript attempting to analyze 20 years’ worth of data on changes in coral ecosystems in Kenya. The editors of Ecology sent the paper out for review but then rejected it, telling her that it was far too long and that some of the paper’s conclusions were unsupported. Darling went back and revised, ultimately publishing part of the work in Conservation Letters (3:122-30, 2010) and another bit a few years later in Global Change Biology (19:1930-40, 2013). “That rejection laid out the path of my PhD,” she says. “It became a much stronger body of work because of that rejection.”
Of course, reviewers can make mistakes. In some cases, reviewers simply missed a key point that was not well explained. “Oftentimes papers are rejected because of a misunderstanding,” says Nikolai Slavov, a postdoc in quantitative cell biology at Harvard University. In such cases, it may be warranted to fight the rejection.
“You can’t fight them all; that’s not the point of peer review,” says Darling. But sometimes “it’s worth standing up for yourself.”
Researchers should also recognize that no one journal is the key to career success. Slavov keeps a page on his blog of seminal papers that were initially rejected. In 1937, Nature rejected a paper from Hans Krebs on the now-eponymous Krebs cycle, which was published a couple months later in Enzymologia (4:148-56, 1937). (See “Nature Rejects Krebs’s Paper,” The Scientist, March 2010.) Lynn Margulis’s paper on endosymbiont theory was rejected about 15 times before landing in the Journal of Theoretical Biology in 1967 (14:225-74). And both Nature and Science rejected Kary Mullis’s paper on the polymerase chain reaction, which was ultimately published in Methods in Enzymology in 1987 (155:335-50).
More recently, David Botstein, now at Princeton University, Michael Eisen, now at the University of California, Berkeley, and their colleagues had their paper on cluster analysis, a widely used method for interpreting microarray data, rejected by Science. Botstein recalls phoning the editor to unsuccessfully appeal the rejection. “The only thing I remember telling her was that it was my thought that this would someday be a citation classic, and in this case I was right,” Botstein says. The paper, which was eventually published in PNAS (95:14863-68, 1998), now has nearly 15,000 citations, according to Google Scholar.
“The important thing is that you get it published,” Botstein advises. “If it’s good and it’s new and it’s true and important, people will find it.”
In recent years, a new strategy for getting papers accepted has sprung up: the pre-peer review process. New companies now offer to send out papers for review by independent scientists before they have been submitted to a journal. Such early feedback can help researchers target the most appropriate journals, and the pre-reviews can be submitted alongside the manuscript to help expedite the journal review process.
Axios Review, which currently focuses on evolution and ecology papers, asks authors to send in each manuscript with a ranked list of four target journals where they would like to see it published. Reviewers selected by the company comment not only on the substance of the paper, but also on its suitability for each target journal. Taking these comments into account, an editor at the company decides where Axios will be willing to refer the paper. If the authors agree, Axios will then send a presubmission inquiry, including the reviews, to the journal. If the journal editors decide that the paper is of interest to them, the author may then submit it through normal channels. If the editors are not interested, Axios will refer the paper to another target journal.
Journals including Molecular Ecology, Evolutionary Biology, and Ecology and Evolution have published papers reviewed by Axios. Ecology Letters even mentions its willingness to consider Axios referrals on its author guidelines page. If Axios successfully places a paper at a target journal, the authors must pay $250. If the process does not result in publication, the authors owe nothing. Axios reviewers and editors, who are unpaid volunteers, are given discounts if they wish to put their own papers through the Axios review process.
For those outside the evolution and ecology fields, a second company, called Rubriq, also sends out papers for review and placement recommendations. Rubriq broadly reviews papers in the biomedical field and is now even expanding beyond the life sciences. Its service costs $650 and also includes a plagiarism check. Its reviewers are paid $100 for their efforts and fill out formalized reports for each paper, giving scores on a 10-point scale for each of three categories: novelty and interest, quality of research, and quality of presentation. Authors can revise their papers based on their Rubriq report and attach the report to their ultimate submissions.
The stated goal of these companies is to reduce time wasted on submitting to inappropriate journals. They also hope that some journals speed the decision to publish by referencing pre-reviews and sending the paper out to fewer or no additional reviewers.
March 18, 2015
I only ask what would have happened if they had not by chance been at a meeting with all the players of this drama present. That editor did not do his/her job in the second round by not asking the only reviewer of both rounds his/her opinion of the rejection of the new referee. This is also a reason I never referee a manuscript if I don't believe that I will be able to look at the revised manuscript
March 18, 2015
Coping with rejection of manuscripts is one of the more challenging aspects of doing peer-reviewed science. You learn to expect criticism, hopefully the kind that can be rebutted. My mentor, Mary Ellen Avery, discovered the cause of Hyaline Membrane Disease, which is due to lung surfactant deficiency at birth. It's best known as the disease that the Kennedy baby died of in 1962. The discovery that HMD was due to a treatable developmental deficiency disease is considered one of the milestones of 20th Century Medicine. Yet Avery had a hard time getting the first paper on the subject published. She often recounted that story to give her trainees heart. That annecdote came in handy 20 years later when I was in the throes of trying to get a paper published describing a new method for measuring lung surfactant for clinical assay purposes. The scientist who had published the standard method at the time was reviewing the paper in journal after journal to block its publication. One of my research Fellows pointed out that the typewriter key for the letter 'i' was punching holes in the paper, so we knew it was the same reviewer shooting the paper down......Avery suggested I send the paper to 'a local journal, The New Engand Journal of Medicine', which published it as their lead article in September, 1979. Perseverance paid off, eventually. DON'T TAKE NO FOR AN ANSWER.
March 18, 2015
Editors should be able to substantially reduce the rate of unjustified rejections by requiring reviewers to substantiate their criticism or ignoring unsubstantiated criticism altogether. It is very hard and very unlikely to reject a great paper with substantiated, level-headed criticism. Unfortunately, many reviews contain vague statements and factually incorrect claims. No scientist should ever be in a position to be embarrassed to show the reviews to his/her students who had done all the work. Yet, that happens .... surprisingly often, especially to outsiders with results that genuinely alter our conceptual understanding.
March 18, 2015
Yes, is true that a researcher needs to deal with frustration in order to get something published. Yes, you need to keep focused to profit from critics. But, Do you really have to negotiate your way to acceptation? I don´t think so. Maybe that's what is wrong with scientific publications these days. Apparently, you DO NOT NEED high quality results as much you NEED connections and negotiating skills. What would happen if you don't know the editors? What happen if you do not coincide with them in a meeting? Is it ethical to openly discuss a manuscript/project under review? Or anticipate the acceptation without the Editor's decision? Can I go an write an author to tell him that I am reviewing his paper? From my point of view this little story tell's a tale of what is and should not be in science. It is unacceptable that a lot of hard and high-quality work is judged from a "Who are you" instead a "How good is your work" point of view. Nevertheles, I've seen it happen.
March 18, 2015
Professor George Klein said, publishing is a Monte-Carlo game, one journal rejects your ms, send it to the next.. till it's published. Michael Lerman, Ph.D., M.D.
March 21, 2015
Rejections may be fair or not, but you always get (kind of) an explanation about why your manuscript was not accepted. The real frustration comes when a paper is returned by the Editor with no clues and a cut-and-paste comment. I know by direct experience how hard is to have a referee reading a manuscript -- but too many journals are filtering manuscripts at the editorial level, often within one hour (yes, less than 60 mins!) from submission (do they read at least the abstract?). This really gives no chance of publication to young or not renowned scientists. When peer-review is systematically replaced by editiorial selection, non-mainstream research and fresh views always pay a big price. And good science thereof.
March 30, 2015
Stephan, don't be too quick to rush to judgement regarding the actions of the EIC. While I agree bringing in a third reviewer is problematic, at times it is essential. This is especially true when several weak reviews give an author a false perception of the perceived worth of their work. On the other hand, every EIC is under pressure to perform and move manuscripts through as fast as possible. This includes sometimes taking short cuts, like not waiting for a previous reviwer's schedule to free up to give them an opportunity to review the manuscript. I rarely like to take this route with an additional reviewer for an R1 or R2, but sometimes it is needed.
I think the best advice is in the middle of the article and advice my late advisor gave me. On my first peer-reviewed manusript, I was pretty hot about what was said in the review. Boss said put it away in the drawer for a week and then take a look at it with a clear mind. Worked like a charm and this is advice I give our students when teaching these concepts in class.
One thing that is important, get back up on the horse, it'll work out.
April 6, 2015
I have had one paper that was rejected by both reviewers. However, the editor disagreed with the reviewers and sent the paper for review by two different reviewers who both recommended that the paper published. He was an example of an editor who read the papers before sending out rejection letters.
While reading this interesting article, I was reminded of another famous rejection. I verified the accuracy before writing this comment. Two papers submitted by Arthur Kornberg on DNA polymerase to the Journal of Biological Chemistry were rejected. After exchange of correspondence with the editor he decided to withdraw the papers. A month later, John Edsall became the new editor of the Journal of Biological Chemistry and published the papers. (See Kornberg’s Autobiography “For the love of Enzymes”, Page 158).
June 18, 2015
"the negative reviewer had been added in the second round because two first-round reviewers had been unavailable"
Does anybody believe such a limp excuse?
Or the editor really thinks that people are so stupid to buy this?
This is a standard practice to shut away a paper with two positive reviews, as at present the author is not allowed to respond to the comments from the second review, even when the reviewer appears for the first time.
Such practice should be made illegal, as it breaches the fundamental right of the author to respond and COPE should include this in its Minimum Mandatory Standards for editors.