Advertisement
Roche
Roche

Lost in Translation

Failure to translate preclinical research to humans may be due in part to biased reporting.

By | July 16, 2013

WIKIMEDIA, RAYSONHOThere is excessive reporting of positive results in papers that describe animal testing of potential therapies, just like the publishing bias seen in clinical research, according to a paper published today (July 16) in PLOS Biology. As a result, many potential therapies move forward into human trials when they probably should not.

“It’s really important [work] in that it gives another explanation for why treatments that appear to work in animals don’t work in humans,” said David Torgerson, director of the York Trials Unit at the University of York in the U.K., who was not involved in the study. “I’ve personally always thought that animal models are potentially not as good as people might assume, but actually that view could be completely wrong, according to this paper.”

Indeed, “many people have argued that maybe there are problems with animal studies—that they cannot capture human physiology and pathophysiology,” said John Ioannidis, a professor of medicine at Stanford University in California, who led the research. “I have believed all along that animal studies should be perfectly fine, if the model is ok. It should be a very decent step toward screening interventions,” he said. Ioannidis was instead worried that an inherent bias toward publishing positive results and suppressing negative or neutral results might create a misleading impression about the effectiveness of interventions, making them destined to fail in human trials.

“We know publication bias happens a lot in clinical trials,” said Torgerson, “so I was surprised at myself for being surprised at the results, because of course, if it happens in human research, why wouldn’t it happen in animal research?”

Ioannidis confirmed his suspicions about publication bias by performing a statistical meta-analysis of thousands of reported animal tests for various neurological interventions—a total of 4,445 reported tests of 160 different drugs and other treatments for conditions that included Alzheimer’s disease, Parkinson’s disease, brain ischemia, and more.

The analysis compared the number of expected significant results—calculated from the results of the largest and most precise individual studies—with the number of observed significant results present in the literature. “We saw that it was very common to have more significant results in the literature compared with what would be expected,” Ioannidis said, “which is a strong signal that that literature is enriched in statistical significance.”

There are two main reasons why this would happen, said Ioannidis. One, as mentioned, is the suppression of negative results. The second is selective reporting of only the statistical analyses of data that provide a significant score. “Practically any data set, if it is tortured enough, will confess, and you will get a statistically significant result,” said Ioannidis. Not surprisingly, such post hoc massaging of data is scorned in the scientific community.

Although the present study focused on animal testing of neurological interventions, a publication bias “almost certainly” applies in other areas of preclinical research, said Bart van der Worp, a neurologist at the Brain Center Rudolf Magnus Institute in Utrecht, The Netherlands, who also was not involved in the study.

So, what can be done to avoid it? “One possibility is to develop registries for all animal studies,” said van der Worp. “Then, if you are working in a specific field at least you’ll know some studies are going on, or have been performed, and may not have been published yet.”

Such registries exist for human clinical trials, so should not be too difficult to implement, he said. Furthermore, he suggested that a forum where investigators can deposit neutral or negative results in the form of articles, should be established to ensure that such findings are in the public arena and not hidden. This should help prevent other researchers from pursuing fruitless avenues of research, he said, “which is a waste of animals and a waste of research money.” Not to mention a risk to people enrolled in potentially pointless trials.

 

K.K. Tsilidis et al., “Evaluation of excess significance bias in animal studies of neurological diseases,” PLOS Biology, 11: e1001609, 2013.

Advertisement

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: EvMedDr

EvMedDr

Posts: 7

July 17, 2013

Beyond the usually-cited problems with the translation of animal experimentation to clinical application, the FDA approval process is flawed because it doesn't require  mimicking the disease process in the animal testing, so you're nominally treating an otherwise healthy organism, which is a false premise. Moreover, the current biomedical paradigm is foiled because it does not take into account the evolutionary principles that have led to a maladaptive condition. In the post-genomic era we shouldn't be tethered to pathophysiology as dichotomous health and disease- they are a continuum that must be understood and exploited in the treatment of disease.

Short of a revoutionary change in our perspective on biology and medicine, publishing negative results in a repository is a great idea.

Avatar of: VM Miller

VM Miller

Posts: 1

July 17, 2013

There is a lack of reporting of the sex and hormonal status of the experimental material, especially in experiments using cultured cells, and a bias to using only male animals in pre-clinical studies.  Results from these experiments are then moved to testing in men and women. Furthermore, clinical trial data is not reported by sex and fewer women than men participate in trials.  It is a false assumption that results from male cells and male animals will apply equally to females.  I challenge Drs. Ioannidis, Torgerson, Van der Worp and Tsilidis to examine the papers they reviewed to see if the sex of the experimental material is identified.  In the era of  personalized medicine, it is critical to consider sex as the important biological variable that it is and design experiments accordingly.

Visit www.ossdweb.org and  http://genderedinnovations.stanford.edu for more information.

July 17, 2013

This article was indeed very informative, and I agree in general with most of the reasons. But, is it really fair to say that these models serve the same role for all clinical indications? To be clear in my comment, I think animal model of infection is still quite useful and is often credited to be a good predictor of efficacy for Phase II human clinical trail. Is it not true?

Avatar of: Paul Stein

Paul Stein

Posts: 119

July 18, 2013

Forget bias for a second.  One needs to understand that much "preclinical" research is an intellectual, not a medical, effort, and because of that, translation, as a goal, is automatically in a far second place.  For the rest, there are lots of issues regarding animal models.  First and foremost is the horrid choice of species.  The focussed use of micro-animals, particularly mice, make applicability to humans almost impossible.  Ramping up therapies has been so onerous as to be neglected by all but the exceedingly brave.  Secondly, many of the disease "models" are used only because they are simple, reliable, and reproducible, and are, in effect, oftentimes, pretty poor at being even close to clinical relevancy.  Again, the models were developed as an intellectual means-to-an-end, based on limited, focussed hypotheses, so thinking that direct, rapid  translation is even possible is almost nonsensical.   

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
Advertisement
Life Technologies