A Flawed Study Shows How Little We Understand Crispr’s Effects


Biotech has been betting big on Crispr, the gene-editing technique that promises to snip away some of humanity’s worst diseases. But last May, a small case study suggested the much-hyped technology might actually be quite dangerous—and pop went the Crispr bubble, briefly tanking shares of Crispr companies like Editas Medicine, Intellia Therapeutics, and Crispr Therapeutics.

It was an overreaction, as so many market spikes and dips turn out to be; the company CEOs were quick to push back, and scientists and journalists soon pointed out flaws in the paper that misconstrued cause and effect. After nearly a year of continued criticisms, and failures to reproduce the results, the authors conceded that the skeptics might be right. Last week, the journal that published the paper, Nature Methods, finally retracted it.

But that doesn’t mean that Crispr has been given the all-clear. The now-retracted paper claimed that Crispr caused nearly 2,000 unexpected mutations—ten times previously observed off-target rates—in two mice that it cured of blindness. But just as that evidence should never have been enough to squash Crispr’s clinical potential, neither does retracting said paper prove it. If anything, the kerfuffle just proves how young the field is. There’s so little published data about the potential for Crispr’s unintended cuts that this single, flawed paper had an outsized effect on perception of the field. Can Crispr be safely used in humans? Retraction or not, it’s still an open question.

Not that scientists haven’t been trying to answer it. But developing trustworthy methods to detect off-target mutations has proven challenging. So far, they’ve come with about a half-dozen approaches; the fastest and most economical is something called targeted sequencing, where researchers look for rogue snips in areas an algorithm suspects they might show up. But that method won’t detect mutations that pop up in utterly unexpected places.

To do that, you can upgrade to whole exome or whole genome sequencing—but those come with huge price tags. Sequencing costs might be going down, but they’re still the most expensive part of these assays, running up to $10,000 depending on the number of samples and level of resolution. A cheaper option would be to biochemically label double-stranded breaks, so you can see all the places Crispr snaps the DNA in two. But this method has its own limitations-—it doesn’t pick up earlier breaks that have already been repaired.

Researchers employing any of these assays will also need to team up with bioinformatics experts to analyze the resulting data—no off-the-shelf software can do the job. One new startup called Beacon Genomics is offering a fee-for-service model to run custom code on tagged double-stranded break data, but it and other commercialized tools are only just coming online. The gold standard for judging Crispr’s messiness doesn’t yet exist.

That has left the leading Crispr companies to develop their own. According to an Intellia spokesperson, the gene-editing firm is working with the US Food and Drug Administration and the National Institutes of Standards and Technology to help set such specifications. “We have developed a comprehensive workflow that uses a combination of both genome-wide assays and targeted sequencing to look for potential off-target mutations,” the company wrote in an email. “We always include appropriate controls in our experiments, such as sequencing of cells that are not edited.”

Editas similarly employs both targeted and genome-wide approaches. Additionally, its scientists have developed a new way to evaluate Crispr edits called Uni-Directional Targeted Sequencing which can identify and quantify structural changes beyond cuts, deletions, and insertions. “We believe UDiTaS not only will help us continue to apply the most rigorous, unbiased approach to analyzing the specificity of Crispr medicines in the field, but also help enable the broader field of genome editing through important standard setting,” says Vic Myer, chief technology officer at Editas.

Crispr Therapeutics declined discussing its off-target evaluation tools for this article. But the company, which could begin collecting safety data from their first clinical trials as soon as later this year, may be in the best position to amass the kinds of evidence that Crispr will ultimately need to pass regulatory muster and make its way to patients. If it does come up with an industry-leading standard, now that’s something that should have a real effect on its valuation.

More on Gene Editing

Read More On This At Science Latest