Do Preprints Belong in Grant Applications?

As the NIH questions the place of preprints in grant applications, the U.K.’s MRC encourages scientists to submit non-peer–reviewed works for consideration.

By | January 6, 2017

NATIONAL CANCER INSTITUTEUpdate (March 27): The NIH on Friday (March 24) announced that “interim research products,” including preprints, “can be cited anywhere other research products are cited” in agency grant applications and reports. In addition to instructing users how to cite preprints and other non-peer reviewed publications assigned digital object identifiers (DOIs), the NIH also issued guidance for selecting where to post such reports. “NIH has never restricted the materials that can be cited in the reference section of a research plan,” the agency noted in its announcement. –The Scientist staff 

The UK’s Medical Research Council (MRC) this week (January 3) announced that it will accept manuscripts published on preprint servers prior to peer review in biomedical research grant applications. “This is not a major policy change. What we’re doing now is making it clear that applicants can submit preprints, and that we actively encourage it,” said Tony Peatfield, director of corporate affairs at MRC.

“There are certain potential advantages to preprints,” Peatfield continued. One is, “you can get your research findings out in the public domain more quickly. This may be of particular benefit to younger researchers.” Preprints, he said, enable scientists “to show their achievements, up to that time.”

Meanwhile, in the U.S., stakeholders continue to debate whether preprints belong in biomedical research grant applications. In an attempt to settle the question, the National Institutes of Health (NIH) seeks public input regarding the inclusion of preprints in NIH grant applications.

The Federation of American Societies for Experimental Biology (FASEB), an organization that represents 30 member societies including around 125,000 biomedical researchers, responded unequivocally. “We do not support the inclusion of preprints or interim research products in NIH grant applications and reports at this time,” FASEB wrote in response to the agency’s request for comments last month.

Yvette Seger, director of science policy at FASEB, said she had held several conference calls with the coalition’s members and found a surprising consensus. “We’re very confident that this does reflect the views of the community,” Seger said of her organization’s letter to the agency. “There was no clear position in support” for the inclusion of preprints in NIH grant applications, she said.

Without a clear definition of what quality of work constitutes a “preprint,” Seger told The Scientist, the NIH cannot hope to instruct grant reviewers in how much weight to place on non-peer-reviewed works, or guide applicants on which works to submit. She also raised the concern that, by accepting preprints in grant applications, the agency could negatively affect research rigor and reproducibility.

“Preprints are an opportunity for members of the community to engage in discussion about the merits of research,” Seger said. “But this is a document that reflects only the initial findings and hasn’t had to respond to peer review.”

Howard Garrison, director of public affairs at FASEB, added that lengthy preprints could be used to circumvent page limits in grant applications, potentially giving an unfair advantage to researchers who submit work that has not yet been peer reviewed. “If somebody is using preprints to extend the amount of information they are able to put into their proposal, that raises concerns about how much weight the reviewer should be giving to this additional information,” he said.

While FASEB does not support the inclusion of preprints in grant applications, the organization acknowledged that preprints play an important role in science publishing, especially for junior researchers. “It’s a tough funding environment for everyone, and early-career investigators may be more likely to demonstrate progress using preprint servers,” Seger said. “We wanted to be sure to highlight that. While we do have several concerns about using these documents in grant applications, we do recognize that it’s a good way for junior investigators to show that they’ve been working.”

Paul Ginsparg, creator of the world’s largest preprint server, arXiv, agreed that preprints are “useful as an indicator of most recent activity.” But in response to the question of whether NIH should allow preprints in grant applications, Ginsparg was less ambivalent. “Why is this even a question?” he asked. “Reviewers are free to ignore anything they don’t find relevant.”

In an October 2016 commentary for The EMBO Journal, Ginsparg noted that preprints are already an integral part of applications for grants, jobs, and prizes within mathematics and the physical sciences. “Preprints are certainly used as evidence of recent productivity in applicant CVs for jobs and grants,” he wrote. “A reviewer for a grant application in a field that heavily uses arXiv would likely be surprised to see no recent relevant work posted there.”

See “Q&A: 1 Million Preprints and Counting

He added that mathematician Grigori Perelman was awarded the Fields Medal in 2006 and the Millennium Prize in 2010 for a mathematical proof that only appeared on arXiv. “It’s difficult to imagine why results accepted and used by an entire expert community would be disqualified for consideration because they didn’t pass through a less-stringent journal filter,” Ginsparg wrote in his commentary.

Peatfield said that the MRC has released a working definition of preprints for grant application purposes. The agency will only allow preprints with a permanent identifier—such as a digital object identifier, or DOI—and only if it comprises a complete scientific manuscript that has been uploaded by the authors, themselves, to a recognized preprint repository.

Although by no means a substitute for peer review, Peatfield noted that many prominent preprint servers enforce editorial guidelines that weed out spurious submissions.

As for concerns that grant reviewers might be confused by preprints, or weigh them inappropriately, Peatfield argued that reviewers should focus on the quality of the research. Well-trained reviewers, he said, ought to know how to evaluate non-peer-reviewed works. “What we’re trying to promote is that reviewers don’t just look at the journal where the research is published, but, having identified a piece of research, will look at it and judge whether or not it’s good,” he told The Scientist. “If they can do that, they can do it with a preprint as much as they can with a fully peer-reviewed paper.”

The submission period for public comments on NIH’s request for information ended on November 29. The agency has not released information about when it will reach a decision, and has not yet responded to The Scientist’s request for comment.

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: Josh N.

Josh N.

Posts: 1

January 6, 2017

I think that we are in a very interesting time right now.  With Sci-Hub acting as the research equivalent of Napster I expect to see many preprints and author-controlled services to gain popularity and prominence.

We've just launched a new service for researchers to write their documents online, which we believe will be an integral push towards a more service-oriented angle in research communication (ie here's the tools, go forth and publish!).

https://www.youtube.com/watch?v=Qa1ObxI_dqU

Avatar of: oldGuy2

oldGuy2

Posts: 1

January 9, 2017

As I understand the history: arxiv grew out of the old physics and math preprint server as Los Alamos National Lab.  That server used to live at XXX.lanl.gov and the logo for the website was a skull and crossbones.    The clear message is: this stuff is not safe for consumption.    Back in the 90's the lag from accepted paper to publication on dead trees was 18 months and many journals didn't do their own "ahead of publication" releases so it filled an important role.  Including time for peer review and revisions, it could be two and a half years from submission to publication -- that's half the time to do a PhD.  

Arxiv is all fancy and legitimate looking.  Maybe that's  actually a dis-service to it's role as repository for things on the bleeding edge.   Is it more important to get ideas out that are new or is it more important to preserve the status quo? 

As usual, medicine is late the show.  This is partly because peoples' lives are at stake and partly because FDA is terrified of the unknown. 

 

Popular Now

  1. Publishers’ Legal Action Advances Against Sci-Hub
  2. Decoding the Tripping Brain
  3. Metabolomics Data Under Scrutiny
    Daily News Metabolomics Data Under Scrutiny

    Out of 25,000 features originally detected by metabolic profiling of E. coli, fewer than 1,000 represent unique metabolites, a study finds.

  4. Do Microbes Trigger Alzheimer’s Disease?
AAAS