Menu

Preclinical Cancer Studies Not as Reproducible as Thought

Researchers overestimate the reliability of findings from animal studies that are part of the Reproducibility Project: Cancer Biology.

Jun 30, 2017
Jef Akst

Squamous cell carcinomaFLICKR, ED UTHMANResearchers working at the Reproducibility Project: Cancer Biology, a collaboration between the Center for Open Science (COS) and Science Exchange, were unable to replicate the results from six mouse experiments published in top-tier medical journals. But when 196 scientists who were unaware of this were asked to predict what would happen if the experiments were repeated, they predicted a 75-percent probability that the results would be similarly significant and a 50-percent probability that the effect size would be the same, according to a study published in PLOS Biology yesterday (June 29).

“What is surprising here is that researchers are not very accurate, actually they are less accurate than chance, at predicting whether a study will replicate,” Benjamin Neel, director of New York University’s Perlmutter Cancer Center who was not involved in the research, tells Reuters.

The scientists surveyed for this study included both early-career researchers and more-established investigators, and the team found that the experienced scientists tended to be more accurate in their predictions. This suggests that training could help researchers better interpret published findings.

While the study highlights the long-recognized reproducibility problem in science, it comes on the heels of some promising news in this area: Reproducibility Project researchers were recently able to replicate the findings of two highly cited leukemia studies.

See “Cancer Studies Seem Replicable

Of course, not all of the work from the Reproducibility Project has yielded such positive results. In fact, the first five replication studies completed by project investigators, published in January, were all largely unsuccessful, yielding different results than the original publications had reported.

See “Replication complications”                          

“Probably the biggest reason the studies don’t hold up is because the sample size is too small. For example, if only 5 to 10 mice are used, a 50-animal study might not yield the same result,” Neel tells Reuters. “I think all preclinical results should be validated by an independent laboratory before they are used as the basis for clinical trials.”

See “The rules of replication

January 2019

Cannabis on Board

Research suggests ill effects of cannabinoids in the womb

Marketplace

Sponsored Product Updates

FORMULATRIX® digital PCR technology to be acquired by QIAGEN
FORMULATRIX® digital PCR technology to be acquired by QIAGEN
FORMULATRIX has announced that their digital PCR assets, including the CONSTELLATION® series of instruments, is being acquired by QIAGEN N.V. (NYSE: QGEN, Frankfurt Stock Exchange: QIA) for up to $260 million ($125 million upfront payment and $135 million of milestones).  QIAGEN has announced plans for a global launch in 2020 of a new series of digital PCR platforms that utilize the advanced dPCR technology developed by FORMULATRIX combined with QIAGEN’s expertise in assay development and automation.
Application of CRISPR/Cas to the Generation of Genetically Engineered Mice
Application of CRISPR/Cas to the Generation of Genetically Engineered Mice
With this application note from Taconic, learn about the power that the CRISPR/Cas system has to revolutionize the field of custom mouse model generation!
Translational Models of Obesity, Dysmetabolism, Diabetes, and Complications
Translational Models of Obesity, Dysmetabolism, Diabetes, and Complications
This webinar, from Crown Bioscience, presents a unique continuum of translational dysmetabolic platforms that more closely mimic human disease. Learn about using next-generation rodent and spontaneously diabetic non-human primate models to accurately model human-relevant disease progression and complications related to obesity and diabetes here!
BiochemAR: an augmented reality app for easy visualization of virtual 3D molecular models
BiochemAR: an augmented reality app for easy visualization of virtual 3D molecular models
Have you played Pokemon Go? Then you've used Augmented Reality (AR) technology! AR technology holds substantial promise and potential for providing a low-cost, easy to use digital platform for the manipulation of virtual 3D objects, including 3D models of biological macromolecules.