SXC.HU, MANKAEEvery practicing scientist knows how difficult it can be to make an experiment work in the lab. Especially frustrating is not being able to reproduce already published experiments. You read a paper, closely follow its materials and methods, buy all of the reagents, run across your department to secure all of the equipment, begin your experiment, and . . . it doesn’t work. At some point, after trying time and time again, stuck alone at midnight in the lab and pulling your hair out, you raise your eyes to the skies and ask: Why doesn’t it work? The paper was published in a prestigious scientific journal by a famous professor. I’ve done everything as written and still it doesn’t work. Why?
Nearly every researcher is well aware of this reproducibility problem, especially in the biological sciences. Accepting this situation as the status quo, however, just got a bit harder after the release of two studies by pharma giants Amgen and Bayer. The companies assigned groups of scientists to repeat experiments published by academic researchers in well-known scientific journals. The results were astonishing: only 10 to 30 percent of these studies were found to be readily reproducible.
Reproducibility is the foundation of science. It is the standard by which scientific claims are evaluated. So when published scientific findings can’t be validated by others, they can’t be taken at face value. In the current climate, battered by budget cuts and government shutdowns, you could ask: Who has time to care? But let’s be practical. The US government gives nearly $31 billion every year in science funding. Hypothetically, with a 10 percent to 30 percent reproducibility rate, that means 70 percent to 90 percent of this money—$22 billion to $28 billion—is being wasted.
Some have called for research articles to include a more detailed materials and methods sections, even though a number of journals have been pursuing this option for decades and the situation hasn’t improved. Another suggestion, from the National Institutes of Health, proposes random audits of published studies. I only can imagine how much money, time, and people it would take to do such audits.
At the Journal of Visualized Experiments (JoVE), our approach to addressing the reproducibility problem is simple: use video to publish science. In other words, convert text-only research papers to videos that show how to do experiments step by step.
While video is a relatively new format for scientific publication, scientists are already finding value. At JoVE, we often ask researchers about the benefits of our publishing format. Following are three who shared their stories.
Theresa Casey and her colleagues, who were studying circadian clock genes in the department of animal sciences at Purdue University, needed to learn how to dissect a specific area of a mouse brain called the suprachiasmatic nucleus. While searching for methods papers, Casey discovered a JoVE video article authored by a research group led by Gabriella Schmitz Lundkvist from the Karolinska Institute in Sweden. The video showed Casey how to perform the technique, saving her group about two months of time.
In another instance, Baylor College of Medicine researchers used JoVE to learn experimental techniques established many years ago so they could conduct neuroscience experiments on Drosophila. Nikolaos Giagtzoglou, a postdoc, needed to learn three Drosophila techniques: giant fiber recording and dissection, dissection of ovaries, and backfilling of motor neurons with calcium indicators. “I really had no starting point to learn these techniques,” said Giagtzoglou. So he used a video article featuring Gregory Macleod from the University of Texas at San Antonio that demonstrated loading calcium indicators into Drosophila nerve terminals. Guided by the video articles, Giagtzoglou and his colleagues were able to reduce the time it took to learn a technique from weeks to days. According to their estimations, it saved thousands of dollars that would otherwise be tied up in staff time and upkeep of the fruit flies.
Meanwhile, Jeanette Moore, a researcher in the Kelly Drew’s lab at the University of Alaska at Fairbanks, used JoVE in another way: to improve training for new students in the lab. While text-based materials and teacher-led instruction are common tools for training new students, they aren’t always enough, she said. “Often, no one is trained in the techniques needed. In that case, we would spend up to six months looking at text references to successfully complete an experiment. Instead of trying to learn a technique from the literature for six months, new researchers can master the method in a few days,” Moore said.
While JoVE is currently the only peer-reviewed journal devoted to video publication of research methods, other scientific publishers have begun to introduce multimedia as well. The New England Journal of Medicine has published a number of videos on clinical procedures, and Cell has launched a project to include multimedia to create what it calls the “article of the future.” You can also find a collection of short videos on the cellular and molecular processes on the Journal of Cell Biology website.
Video-based science publication is catching on. In a world where failure to replicate afflicts more than half of all life-science experiments published in scientific journals, many scientists view video papers as a recipe for success.
Moshe Pritsker is the CEO and co-founder of JoVE, a peer reviewed, PubMed-indexed journal that publishes experimental techniques for the biological, medical, chemical, and physical sciences in video format in an effort to increase reproducibility and productivity of scientific research.