Was a Drop in CRISPR Firms’ Stock Warranted?

A study of off-target effects that sparked fear among investors of genome-editing companies receives methodological criticisms.

Jun 7, 2017
Claire Asher

ISTOCK, VCHALUpdate (June 15): Yesterday (June 14) Nature Methods posted an editorial note on the paper in question: “Readers are alerted that the conclusions of this paper are subject to criticisms that are being considered by editors. A further editorial response will follow the resolution of these issues.” —Jef Akst

Last week, a study made headlines worldwide with its claim that the CRISPR-Cas9 genome-editing technique is more error prone than expected. In response, some investors chose to sell their shares in CRISPR-based biotech firms and stock values dipped by up to 15 percent for some companies.

But do the findings in the latest CRISPR report justify all this panic?

Scientists and companies developing CRISPR-based treatments say no, pointing to flaws in the evidence. Furthermore, the need to screen for off-target mutations and optimize techniques long before they are presented to regulatory bodies is something CRISPR insiders are well aware of, says Ritu Baral, a biotechnology stock analyst from the investment firm Cowen. “All of these things are happening behind the scenes, but non-specialists and retail investors may not be aware.”

The analysis of off-target effects, published in Nature Methods May 30, stems from a 2016 study by Stephen Tsang at Columbia University and colleagues. They successfully used the CRISPR-Cas9 technique to repair a genetic mutation in mice embryos that causes blindness. After the study was complete, co-author Vinit Mahajan, an ophthalmologist and scientist at Stanford University, saw an opportunity to investigate how error prone the Cas9 enzyme is.

Scientists have always acknowledged that CRISPR is imperfect. Computer algorithms have been developed to identify sites in the genome that share similarities with the RNA guide sequence, which might become accidental targets for the Cas9 enzyme. So far, studies have suggested that the rate of off-target mutations for CRISPR techniques is generally low, but Mahajan explains, “we wanted to do our due diligence to see if the CRISPR-Cas9 might accidentally be hitting other sites.”

Mahajan and his colleagues sequenced the genomes of stored tissue samples from their 2016 study and compared them to published sequence data for mice. Mutations in the tissue sample sequences that weren’t present in public sequence databases were identified as off-target mutations caused by the CRISPR intervention.

The results were a surprise to the team—the two CRISPR-treated mice had 10 times more mutations than expected. Even more concerning, none of these mutations were in the locations that computer algorithms had predicted as off-site targets of Cas9.

“The whole-genome sequences seemed to pick up mutations in other areas that weren’t predicted,” Mahajan tells The Scientist. This suggests that experiments on cells in culture may not be good predictors of how CRISPR proteins behave when injected into whole organisms.

Mahajan says predictive algorithms may miss mutations by searching for potential off-site targets only in the regions of the genome that code for proteins. Non-coding regions are “important for regulating genes and regulating the cells,” but he admits we aren’t yet able to predict how mutations there might affect whole organisms. Despite their new mutations, the mice in his study seemed quite normal, he says.

“This is a very small, focused study, looking at a single gene in mice,” Mahajan cautions. “We need to do more research to understand how off-targeting works and how it might appear in different situations.”

Gaétan Burgio, a geneticist at the Australian National University in Canberra, says he isn’t surprised by the finding. “What is unusual is the magnitude of the effect,” he says, but he suggests there are alternative explanations for the result. “Many of these off-target mutations are unlikely to be due to CRISPR,” Burgio tells The Scientist.

A major flaw in the study, continues Burgio, was the use of published sequence data rather than control mice—animals kept in the same laboratory conditions but not injected with the Cas9 protein. “The mice you have in your facility have drifted genetically from those that have been sequenced.” Because of this, Burgio says, the study isn’t able to “tease out which variants are due to the natural genetic variation and which ones are CRISPR-related.”

In support of this claim, critics have noted that the two mice shared many mutations that were homozygous, meaning that the same variant was present on both copies of the chromosome. This suggests that they may have been inherited from a shared ancestor and present in the mice prior to the CRISPR treatment. With CRISPR-Cas9, “it is extremely rare, in my experience, to create two point mutations that are exactly the same” in two different mice, says Burgio.

Mahajan admits, “I don’t think we can be 100 percent certain that they’re all caused by CRISPR. There are a number of caveats in the system and how the experiment was done.”

Critics also claim that the paper includes significant inaccuracies, such as incorrectly identifying off-target sites, which may have affected the validity of the results. Mahajan didn’t offer a detailed counter-critique, but describes the criticisms as “factually wrong.”

Nevertheless, this small, preliminary study has been enough to grab headlines and frighten investors, with several biotechnology companies that use the CRISPR gene-editing system seeing falls of up to 15 percent in their share prices. This kind of reaction is rare, says Baral. “But when you do have incredibly high-profile fields it’s definitely not unheard of for this to happen.”

In response to the study, Intellia Therapeutics states in an email to The Scientist that the “study is flawed,” and, “the authors have failed to consider alternative and simpler explanations of the alleged off-targets.” They emphasize their ongoing work to “find therapeutically relevant guides with no detectable off-target editing” for developing CRISPR therapeutics.

Editas Medicine echoes this, writing in an email that they “rigorously assess the specificity of our molecules,” and have “demonstrated that we can make CRISPR molecules with no detectable off-target effects.”

Baral says she doesn’t believe these results are cause for major concern. “I think that where the science is going, it can advance to meet that extremely high level of safety scrutiny. . . . What it does suggest is that the time that these companies spend in pre-clinical development for their therapies is completely justified.”

Mahajan also remains optimistic. “I’m still very excited about CRISPR,” he says, but adds that the research community needs to address questions about the rate of off-target mutations in real tissues, and the effect of these mutations on living organisms. If off-target mutations prove to be as common as his data suggest, the risk may still be worthwhile for patients suffering from debilitating or terminal illnesses that have no alternative treatment, he says.

K.A. Schaefer et al., “Unexpected mutations after CRISPR–Cas9 editing in vivo,” Nature Methods, 14:547–48, 2017.