Breakthroughs from the Second Tier
Peer review isn’t perfect— meet 5 high-impact papers that should have ended up in bigger journals.
Often the exalted scientific and medical journals sitting atop the impact factor pyramid are considered the only publications that offer legitimate breakthroughs in basic and clinical research. But some of the most important findings have been published in considerably less prestigious titles.
Take the paper describing BLAST—the software that revolutionized bioinformatics by making it easier to search for homologous sequences. This manuscript has, not surprisingly, accumulated nearly 30,000 citations since it was published in 1990. What may be surprising, however, was the fact that this paper was published in a journal with a current impact factor of 3.9 (J Mol Biol, 215:403–10, 1990). In contrast, Nature enjoys an impact factor more than 8 times higher (34.5), and Science (29.7) is not far...
One of the most commonly voiced criticisms of traditional peer review is that it discourages truly innovative ideas, rejecting field-changing papers while publishing ideas that fall into a status quo and the “hot” fields of the day—think RNAi, etc. Another is that it is nearly impossible to immediately spot the importance of a paper—to truly evaluate a paper, one needs months, if not years, to see the impact it has on its field.
In the following pages, we present some papers that suggest these two criticisms are correct, at least in part. These studies were published in lower-profile journals (all with current impact factors of 6 or below), suggesting they should have had less of an impact. But these papers eventually accumulated at least 1,000 citations. Many were rejected from higher-tier journals. All changed their fields forever.
P.A. Zuk et al., “Multilineage cells from human adipose tissue: Implications for cell-based therapies,” Tissue Eng, 7: 211–28, 2001. Times cited: 1,175
P.A. Zuk et al., “Human adipose tissue is a source of multipotent stem cells,” Mol Biol Cell, 13: 4279–95, 2002. Times cited: 1,010
Findings: Fat contains pluripotent stem cells.
Impact: More than 25 clinical trials have occurred or are in progress that use fat-derived stem cells to treat a wide range of indications, including diabetes and cardiovascular disease.
Not long after Patricia Zuk began as a postdoc in August 1999 in Marc Hedrick’s lab at the University of California, Los Angeles School of Medicine, “Marc tossed this manuscript on my desk and said ‘Fix this,’” she recalls. The manuscript had to do with a population of cells in adipose (fat) tissue, and based on the work he had conducted with his colleagues at the University of Pittsburgh, Hedrick suspected they could be multipotent stem cells. But clearly the evidence that had raised his suspicion wasn’t strong enough to warrant publication—the manuscript had been rejected.
The problem, Zuk explains, was that “people just auto-assumed the cells were just committed preadipocytes”—if you put the cells in culture, you would get fat. But, Zuk adds, many diseases cause fat and other tissues to calcify—in other words, turn into bone—suggesting fat contains at least some cells capable of differentiating into other tissues.
So Zuk, along with another recently hired postdoc, Min Zhu, took on the responsibility of more fully characterizing these cells. Using fat samples obtained from liposuction procedures, the researchers ran a series of molecular and biochemical tests—expression assays, immunofluorescence experiments, flow cytometry tests, and more—and concluded that the cells were indeed multipotent and capable of differentiating into a variety of different lineages, including fat, bone, muscle, cartilage, and even neural tissues.
“Fat is definitely an underappreciated tissue source,” Zuk says. In addition to being far easier to obtain than bone marrow—the most commonly used source of mesenchymal stem cells—there’s usually a lot more of it to spare, she adds. “There is no tissue in the human body that is as expendable as adipose tissue.” Indeed, the tissue has proven to be a useful clinical application since the publication of these papers, with more than 25 clinical trials completed or in progress for a wide range of indications, including diabetes and cardiovascular disease.
Once the lab members felt they had a strong enough case, they sent their manuscript out to Cell, but it was immediately rejected. “They obviously didn’t think it was high impact enough because they didn’t even send it out for review,” Zuk says. They tried again at Tissue Engineering, and this time they were successful. A follow-up paper was published the following year in Molecular Biology of the Cell.
“There was just something about the papers that really caught everybody’s attention,” says Bruce Bunnell of Tulane University School of Medicine, who has cited the two papers 16 times in his own work. “Those two papers have become [the go-to papers] in the field of adipose stem cell research if you need to reference the basic biology.”
“We were lucky enough to be the seminal paper on this topic,” Zuk says. “When I think hematopoietic [stem cells], I think of McCulloch and Till; when I think of the mesenchymal stem cell population, it’s Friedman, 1969.” Now, she muses, “I guess I’ll be permanently associated with fat.”
S.P. Gygi et al. “Correlation between protein and mRNA abundance in yeast,” Mol Cell Biol, 19:1720–30, 1999. Times Cited: 1,607
Finding: It is not possible to infer protein levels in a cell by measuring RNA transcripts, an easier technique than quantifying proteins directly. This affirmed why the field of proteomics should exist.
Impact: The search term “proteomics” retrieves more than 24,000 papers on PubMed.
For 2 years, researchers at the University of Washington toiled away, running gel after gel to isolate, label, and count the proteins in yeast. The idea of a “proteome” was still a new one in the late 1990s, but throughout the decade, researchers had developed better and better techniques to measure the amount and type of proteins in a cell.
Two years earlier, in 1997, a team at the Johns Hopkins University School of Medicine had published the yeast transcriptome—the set of genes expressed from the yeast genome (Cell, 88:243–51, 1997). Ruedi Aebersold, then a biologist at the University of Washington, realized he finally had all the tools to answer a pressing question: Do RNA transcripts directly correlate with protein levels in a cell?
If so, it would be good news to the research community, since technologies to measure RNA transcripts have always been more advanced and easier to use than those to quantify and identify proteins, more biochemically complex molecules. But to use mRNA levels to predict protein levels, researchers had to assume there was a direct correlation between the two. Unfortunately, they had long suspected that wasn’t the case, recognizing that there are a host of post-transcriptional events that control protein translation and degradation rates, skewing the ratio of the two sets.
To set the suspicion to rest, Aebersold and postdoc Steven Gygi spent years using high-resolution, two-dimensional gel electrophoresis to separate proteins in yeast cells, then excised and identified them using mass spectrometry and database searching. Finally, they compared their data with the mRNA levels of the 1997 paper, and found a very poor correlation between protein and mRNA levels. For genes with equal mRNA levels, protein levels varied by more than 20-fold. For proteins with equal abundance, mRNA levels varied by as much as 30-fold. “I personally wasn’t surprised,” says Aebersold, “but as is always the case, we wanted to show it with data.”
The team submitted the paper to Molecular and Cellular Biology only, since half the data had already been published and it was such a new field. It went right in, recalls first author Steve Gygi. The results were published in March of 1999. Aebersold received nice comments from colleagues, he recalls, who were pleased to finally have data confirming their belief that there was no strong correlation between the two data sets. With one paper, they singlehandedly illustrated why the field of proteomics needed to exist.
“This is a very important paper,” says Matthias Selbach, a proteomics researcher at the Max Delbrück Center for Molecular Medicine in Berlin. “Many people from the proteomics field like to cite this work. They all want to show that protein levels have nothing to do with RNA levels, so then they cite this paper to make [the point] and to justify why they are doing proteomics rather than transcriptomics,” says Selbach. A range of subsequent studies analyzed the same correlation in other cell types, always with similar conclusions.
“It helped my career a lot,” says Aebersold, who went on to continue working in the field of proteomics and cofounded the Institute for Systems Biology in Seattle in 2000. Today he is a professor at the Institute of Molecular Systems Biology at ETH Zurich in Switzerland. “Proteomics was a fringe field for a long time,” says Aebersold. “Compared to 10 years ago, there’s been pretty amazing progress.”
JE Meredith et al., “The extracellular matrix as a cell survival factor,” Mol Biol Cell, 4:953–61, 1993. Times cited: 1100
Finding: The extracellular matrix prevents programmed cell death, sparking a new field that another paper termed “anoikis.”
Impact: Using “anoikis” as a search keyword retrieves more than 700 articles on PubMed.
Most field-changing observations are completely serendipitous. The discovery that a cell’s external environment is essential to its survival—as in, without it, the cell undergoes apoptosis—was no different. Martin Schwartz, then at Scripps Research Institute in La Jolla, Calif., left endothelial cells he was studying in suspension overnight. When he realized his mistake, the cells were dead, and they had formed prototypical “bleb” shapes—the hallmark of programmed cell death, a trendy concept at the time.
Schwartz hypothesized that the extracellular matrix (ECM) contained elements that protected cells from programmed cell death, and he planned a couple of straightforward experiments to confirm it. He had his new postdoc, Jere Meredith, repeat his “accident” and confirm the cells were undergoing apoptosis. Meredith, now at Bristol-Myers Squibb, remembers it as one of the most satisfying experiments of his career. “Everything just seemed to work,” recalls Meredith. “It was kind of neat because science never seems to work that way.”
And like many other science stories, another lab was unknowingly pursuing the same project—across the street from Schwartz’s, in fact. “We could see each other’s buildings,” says Steven Frisch, then at the La Jolla Cancer Research Foundation, now called the Sanford-Burnham Medical Research Institute. “And neither of us knew what the other was doing.” He had also discovered the role of the ECM in preventing apoptosis accidentally, while doing experiments on the E1A adenovirus gene, which he says can convert human tumor cells into normal epithelial cells. When plating tumor cells into soft agar, which prevents cells from forming an ECM, he saw that all cells with an inserted E1A gene disappeared. Frisch reasoned that the cells with the E1A gene were converted to a normal epithelial phenotype, making them dependent on the ECM for survival. But the cancerous cells did not depend on the ECM for survival, enabling them to spread around the body, or metastasize. Additional experiments supported his hypothesis.
Recognizing the importance of his finding and what it could mean for cancer research, Frisch sent his manuscript to “some of the very top-tier journals,” he says. “It kept getting rejected.” Meanwhile, Schwartz, now at the University of Virginia, says he did not know of Frisch’s work, but had heard about a similar project by another scientist at a meeting, and knew he needed to publish fast. Plus, the finding was relatively “simple,” he says, with no mechanism, and higher-impact journals “don’t publish ‘simple’ papers.” They sent their manuscript to only one journal—Molecular Biology of the Cell, then a new publication, but with a reputation for reviewing papers quickly. It was accepted with just small changes, and published a few months after submission. Meanwhile, Frisch’s paper was trapped in the review process, and didn’t appear until 5 months later, in the Journal of Cell Biology (124:619–26, 1994), complete with a citation of Schwartz’s paper. “I didn’t know about Steve Frisch’s work until it was published,” Schwartz says. Both papers showed that the lack of the ECM induces apoptosis, although Frisch gave it a name—“anoikis,” or “homelessnes” in ancient Greek. (He originally named it “homelessness,” but the journal nixed it. After calling a Greek restaurant and ancient Greek scholar, he settled on “anoikis.”)
The findings had obvious implications for cancer research, but additional research has also shown that anoikis is critical in various phases of embryogenesis, and may even play a role in neurodegenerative disease.
Frisch says he “obviously” wishes he hadn’t tried to publish his paper in a top-tier journal, so he could have been the first to describe anoikis. But his publication has accumulated more than 1800 citations, and when the field caught on to the importance of anoikis, he was asked to write reviews and speak about his work at more conferences than he could attend. “For a while, our lab was pretty well known,” says Frisch, now at West Virginia University School of Medicine. To this day, when the journals that rejected it asked him to write articles about anoikis, he reminds them that they rejected the first paper to describe it.
A. Carr et al. “A syndrome of peripheral lipodystrophy, hyperlipidaemia and insulin resistance in patients receiving HIV protease inhibitors,” AIDS, 12:F51–58, 1998. Times Cited: 1,234
Finding: HIV-positive people receiving protease inhibitors develop a syndrome of acquired lipodystrophy.
Impact: The terms “HIV” and “lipodystrophy” bring up more than 1900 papers on PubMed.
In the mid 1990s, science and medicine began gaining ground on the recently characterized HIV virus, and patients were taking new medications that had a dramatic impact on the disease. But with these new treatments came unforeseen side effects.
Andrew Carr, of the University of New South Wales, was hearing complaints from his HIV patients who were receiving new drugs called protease inhibitors that prevented viral replication. Carr says that many came to him to ask why their limbs seemed to have shriveled, the veins on their arms and legs bulging. They were living longer than HIV patients in the previous decade, but also showing off-the-charts cholesterol measurements, and complaining of gaunt faces and protruding bellies.
Carr consulted Donald Chisolm, an endocrinologist at Sydney’s Garvan Institute of Medical Research. Chisolm told Carr it sounded like lipodystrophy, a generic medical term for the abnormal distribution of lipids around the body, a disorder that is normally either inherited or, more rarely, acquired.
“That’s when we decided to systematically survey patients,” remembers Carr. With the help of Chisolm and others, Carr conducted a cross-sectional study of nearly 150 HIV-positive patients, some of whom were taking protease inhibitors. The team submitted their findings—that protease inhibitors were causing a syndrome of acquired lipodystrophy that included high blood lipids, reduced body fat (especially in the extremities), a migration of fat to the abdomen and other areas, and insulin resistance—to The Lancet in the latter half of 1997.
Reviewers at the premier medical journal were unimpressed. “[The paper] got those sort of surprised reviews you get when something completely new comes up and you’re not sure whether you should believe it or not,” says Carr.
—Andrew Carr
Early the next year, Carr and his colleagues took their findings to a large HIV meeting in the United States, and presented a poster positioned next to posters from authors of two recent Lancet papers showing an enlargement of the fat pad on the upper back and expanding abdominal fat in HIV patients. He says that conference attendees were lined up at the three posters 10 people deep, all day long. When Carr and his coauthors returned home from the conference, they submitted their paper to the journal AIDS, which had a fast-track publication process. The paper was accepted, and since its publication, it’s been cited more than 1200 times. Dozens of researchers around the world now devote their careers to studying the ancillary health problems that accompany HIV treatments.
“The whole complexion of HIV changed from a horrible, catastrophic, fatal disease to suddenly have to start worrying about metabolic problems,” says University of California, San Francisco, endocrinologist Morris Schambelan, who authored one of those early Lancet papers on lipid problems in HIV patients. “I think they did a service to the field by throwing that paper out there and letting us chew on it.”
“[The paper] played a major role in getting people to focus on the abnormal loss of fat that was occurring in HIV,” agrees Carl Grunfeld, University of California, San Francisco professor of medicine. “It was the paper that everyone took notice of and started the field” of studying lipid abnormalities in HIV patients.
But Grunfeld and Schambelan fault the authors for laying all of the blame on protease inhibitors, when the study subjects were being treated with a panoply of HIV drugs. “We now understand that the [symptoms] had different causes,” Grunfeld says. Carr agrees. “Attributing [the syndrome] to protease inhibitors might have been wrong.”
This misattribution, Carr says, led some HIV patients to alter their treatment regimens, which could have led to increased HIV transmission in some populations. “I suppose part of me feels guilty that I contributed a little bit to patients stopping their pills.”
S. Sakaguchi et al., “Immunological self-tolerance maintained by activated T cells expressing IL-2 receptor a-chains (CD25). Breakdown of a single mechanism of self-tolerance causes various autoimmune diseases,” J Immunol, 155:1151–64, 1995. Times cited: 2,069
Finding: A distinct cell population (Tregs) keeps the immune system in check.
Impact: The term “Tregs” brings up nearly 1500 papers on PubMed.
For over a decade, immunologists had poured their resources into searching for a cell that suppresses the immune system, only to have their hopes dashed. By the late 1980s, most had given up hope.
The thought that such a powerful suppressor cell could exist stemmed from the late 1970s, when researchers found evidence the body had a fail-safe mechanism to clamp down an immune reaction before it became too aggressive, speculating that it might prevent autoimmune diseases that killed normal as well as infected tissue. Researchers followed one lead after another to find the population of cells that was responsible. After a decade of work, with the best hypotheses proven false, most immunologists abandoned the field. Publication rates for suppressor immune cells dropped from some 1,300 per year in the 1980s to around 150 in the 1990s (Semin Immunol, 16; 69–71, 2004). “The suppressor field had its heyday and failed,” says Ethan Shevach from the National Institute of Allergy and Infectious Diseases, who himself abandoned suppressor cell research. “The field was dead.”
So when researchers led by Shimon Sakaguchi, then at Tokyo Metropolitan Institute for Gerontology in Japan, found a distinct population of immune suppressive cells, publishing their results proved challenging.
Their work demonstrated that a small subset of T cells studded by the surface molecule called CD25 kept the immune system in check. When Sakaguchi used an anti-CD25 antibody to deplete this population of immune cells, mice immediately developed a spate of autoimmune disease such as gastritis, arthritis, and adrenalitis. And when the researchers reinserted cells with CD25, the inflammation directed at healthy tissue was quashed.
With its clear clinical implications, Sakaguchi and colleagues submitted the paper to the Journal of Experimental Biology. They were rejected. “It was a bit of disappointment for us,” says Sakaguchi, now at Kyoto University. They decided to resubmit to the Journal of Immunology, where it was published.
“I was one of the biggest nonbelievers” in this suppressive cell, says Shevach. He read the paper critically as soon as the issue hit his desk and noticed that the antibody Sakaguchi used to deplete CD25-expressing cells came from his own lab. Sakaguchi had used the antibody correctly, Shevach says, and everything just “clicked” in his mind. Right away, Shevach set off to replicate Sakaguchi’s findings. “When we confirmed his studies, I certainly underwent a bit of a ‘religious’ transformation,” says Shevach, becoming a sort of “cheerleader” for the Tregs, which was the new name given to these suppressive cells.
What Sakaguchi’s paper had done that none of the earlier research had accomplished was to identify a reliable surface marker that enabled scientists to isolate and study the cell population in a controlled manner. “Immunologists are attracted by cell surface markers,” says Shevach.
With an identifiable subset available, researchers flocked back to the field. In the years that followed, T regulatory cells were also studied for their role in transplantation medicine (by suppressing the immune reaction that normally rejects transplanted tissue) and cancer biology (by suppressing the cells that could potentially attack and kill tumors). But researchers are still scratching their heads about the best methods for manipulating Tregs. “In the future, we must think more seriously about clinical application,” says Sakaguchi.
Have a comment? Email us at mail@the-scientist.com