Do Preprints Belong in Grant Applications?

“There are certain potential advantages to preprints,” Peatfield continued. One is, “you can get your research findings out in the public domain more quickly. This may be of particular benefit to younger researchers.” Preprints, he said, enable scientists “to show their achievements, up to that time.”

Meanwhile, in the U.S., stakeholders continue to debate whether preprints belong in biomedical research grant applications. In an attempt to settle the question, the National Institutes of Health (NIH) seeks public input regarding the inclusion of preprints in NIH grant applications.

The Federation of American Societies for Experimental Biology (FASEB), an organization that represents 30 member societies including around 125,000 biomedical researchers, responded unequivocally. “We do not support the inclusion of preprints or interim research products in NIH grant applications and reports at this time,” FASEB wrote in response to the agency’s request for comments last month.

Yvette Seger, director of science policy at FASEB, said she had held several conference calls with the coalition’s members and found a surprising consensus. “We’re very confident that this does reflect the views of the community,” Seger said of her organization’s letter to the agency. “There was no clear position in support” for the inclusion of preprints in NIH grant applications, she said.

Without a clear definition of what quality of work constitutes a “preprint,” Seger told The Scientist, the NIH cannot hope to instruct grant reviewers in how much weight to place on non-peer-reviewed works, or guide applicants on which works to submit. She also raised the concern that, by accepting preprints in grant applications, the agency could negatively affect research rigor and reproducibility.

“Preprints are an opportunity for members of the community to engage in discussion about the merits of research,” Seger said. “But this is a document that reflects only the initial findings and hasn’t had to respond to peer review.”

Howard Garrison, director of public affairs at FASEB, added that lengthy preprints could be used to circumvent page limits in grant applications, potentially giving an unfair advantage to researchers who submit work that has not yet been peer reviewed. “If somebody is using preprints to extend the amount of information they are able to put into their proposal, that raises concerns about how much weight the reviewer should be giving to this additional information,” he said.

While FASEB does not support the inclusion of preprints in grant applications, the organization acknowledged that preprints play an important role in science publishing, especially for junior researchers. “It’s a tough funding environment for everyone, and early-career investigators may be more likely to demonstrate progress using preprint servers,” Seger said. “We wanted to be sure to highlight that. While we do have several concerns about using these documents in grant applications, we do recognize that it’s a good way for junior investigators to show that they’ve been working.”

Paul Ginsparg, creator of the world’s largest preprint server, arXiv, agreed that preprints are “useful as an indicator of most recent activity.” But in response to the question of whether NIH should allow preprints in grant applications, Ginsparg was less ambivalent. “Why is this even a question?” he asked. “Reviewers are free to ignore anything they don’t find relevant.”

In an October 2016 commentary for The EMBO Journal, Ginsparg noted that preprints are already an integral part of applications for grants, jobs, and prizes within mathematics and the physical sciences. “Preprints are certainly used as evidence of recent productivity in applicant CVs for jobs and grants,” he wrote. “A reviewer for a grant application in a field that heavily uses arXiv would likely be surprised to see no recent relevant work posted there.”

See “Q&A: 1 Million Preprints and Counting”

He added that mathematician Grigori Perelman was awarded the Fields Medal in 2006 and the Millennium Prize in 2010 for a mathematical proof that only appeared on arXiv. “It’s difficult to imagine why results accepted and used by an entire expert community would be disqualified for consideration because they didn’t pass through a less-stringent journal filter,” Ginsparg wrote in his commentary.

Peatfield said that the MRC has released a working definition of preprints for grant application purposes. The agency will only allow preprints with a permanent identifier—such as a digital object identifier, or DOI—and only if it comprises a complete scientific manuscript that has been uploaded by the authors, themselves, to a recognized preprint repository.

Although by no means a substitute for peer review, Peatfield noted that many prominent preprint servers enforce editorial guidelines that weed out spurious submissions.

As for concerns that grant reviewers might be confused by preprints, or weigh them inappropriately, Peatfield argued that reviewers should focus on the quality of the research. Well-trained reviewers, he said, ought to know how to evaluate non-peer-reviewed works. “What we’re trying to promote is that reviewers don’t just look at the journal where the research is published, but, having identified a piece of research, will look at it and judge whether or not it’s good,” he told The Scientist. “If they can do that, they can do it with a preprint as much as they can with a fully peer-reviewed paper.”

The submission period for public comments on NIH’s request for information ended on November 29. The agency has not released information about when it will reach a decision, and has not yet responded to The Scientist’s request for comment.