Do Devices that Monitor or Zap the Brain Live Up to Their Claims?

They are not first to raise the issue. In the last few years, scientists and ethicists have voiced concerns about poor oversight of these products and the paucity of data on their efficacy. In fact, some researchers say that the jury’s still out on whether EEG- and tDCS-based techniques can actually provide the benefits the promise.

“I don’t think these companies are trying to be deceitful—I think they are well-intentioned,” says Robert Thibault, a postdoc at the University of Bristol. “But they’re trying to sell a product, and like anyone, they can succumb to biases when trying to achieve their goal.”

This has led some in the neurotech industry to push for better practices for marketing these gadgets and has prompted academics to search for more robust ways to evaluate their effects on users.

It’s particularly important for companies to innovate responsibly in the brain space, Illes says. “If these devices don’t work, we definitely want people [with neurologic or mental health disorders] to remain on their standard medical care . . . and if in fact these devices do work as claimed, they are necessarily making a brain change,” she explains, “So in both cases, the ethical implications are huge.”

Unsubstantiated, unapproved claims

To assess the evidence behind the claims about direct-to-consumer (DTC) wearable brain technologies, Illes and her colleagues first scoured the internet to find companies that sold these products and identified 41 devices.

After taking a look through their webpages, the team discovered that there were only eight products with at least one peer-reviewed publication directly connected to their claims. Of the remaining devices, 29 cited general scientific research or concepts that were not explicitly linked to their products, and the remainder pointed to other resources, such as user testimonials, in-house research, and irrelevant scholarly papers. “Many of the claims were unsubstantiated,” Illes says.

In the US, devices that claim to benefit patients require approval from the Food and Drug Administration (FDA) before entering the market. Illes’s team reports that eight devices claimed to improve medical conditions, such as post-traumatic stress disorder, chronic pain, and insomnia, without being FDA-approved. On top of that, the researchers found several other products that advertised general “wellness” claims that a customer could interpret as potential medical benefits. One company, for example, stated that its device targeted a nerve pathway that is implicated in a number of diseases. These devices are “kind of skirting under the radar of [regulators like] the FDA,” Illes tells The Scientist. “And I think that’s problematic.”

Other researchers have identified similar issues with DTC products. In a study published last year in the Journal of Cognitive Enhancement, Thibault and Anna Wexler, a medical ethicist at the University of Pennsylvania, assessed the claims made by 18 companies selling EEG devices for wellness purposes, such as improving focus, meditation, and sleep.

The duo found that only a few of the companies publicly shared research about their wares—and if they did, it was difficult to rule out the placebo effect. For example, some did not include a control group or a double-blind design. Another issue was that, in the majority of cases, it was unclear whether the consumer EEG product was able to record brainwaves as well as a research-grade device.

Some companies will cite proprietary reasons for not sharing this information, Wexler says. “In the corporate world. . . they’re not really interested in publishing in scientific journals, so it’s hard to tell what’s working and what’s not.”

See “The Next Generation of Noninvasive Brain Stimulation”

Inconclusive science

The science that supports the potential benefits of these consumer EEG tools has also turned up less than impressive results, according to Thibault and Wexler. In their 2018 paper, they say there’s a lack evidence to prove that neurofeedback—a technique most of these devices are based on, in which individuals monitor and try to control their brainwaves to alter behavior—even works. “If you look through the academic literature, you’ll find no evidence that neurofeedback with EEG changes behavior through more than just placebo effect,” Thibault says.

Neurofeedback using EEG also depends on the ability to map a brain signal onto a specific mental state, Thibault explains. Though scientists have been trying to do this for the last 60 years, “we’re not at a point where we can say this brain activity that’s happening right now means you’re being attentive and we can improve it and make you more attentive.” But people are starting to use sophisticated methods such as machine learning to better identify different brain states, he adds. “I think there’s potentially a good future in this data-driven approach.”

Graeme Moffat, the chief scientist and vice president of regulatory affairs at Toronto-based InteraXon, a company that makes consumer EEG devices, agrees that there are open questions about the utility of neurofeedback. (His company sells a headband-like gadget for “technology-enhanced meditation” that costs around $200.) “There is good reason to believe neurofeedback is effective for some things and that you can operantly condition the EEG signal,” Moffat says. “But what those things are is much more circumscribed than some of the claims that are out there.”

Some scientists say there are similar limitations for tDCS. “The jury is still out,” says Simon Hanslmayr, a neuroscientist at the University of Birmingham. “We don’t have unequivocal scientific evidence that this brain stimulation technique [can] change cognitive processes in the healthy, let alone alleviate any medical problems.” Hanslmayr adds that while there are peer-reviewed articles that support the potential benefits of tDCS, there are also papers in the scholarly literature that argue the opposite.

See “Doubts Raised About Brain Stimulation to Reduce Food Cravings”                                                      

In one 2017 meta-analysis, for instance, scientists reported that there was “minimal to no evidence” that tDCS could influence cognitive processes such as working memory in healthy humans, because the statistical power of the published studies was too low to provide meaningful conclusions. Earlier this year, a paper in Royal Society Open Science reported that tDCS didn’t reduce food cravings, challenging prior findings. And while a growing body of research suggests that tDCS may be an effective treatment for depression, some studies have found mixed results.

Daniel Mansson, a clinical psychologist and the CEO of Flow, a company that recently launched a £399 tDCS device—for people with depression—in the UK, says that while he agrees that there is uncertainty in many areas of tDCS research, the evidence for its anti-depressive effects is robust. Mansson points to a recent meta-analysis in the BMJ that found tDCS to be an efficacious treatment for depression, as well as a randomized clinical trial published in the New England Journal of Medicine in 2017 that found treatment with tDCS was superior to placebo (although not more effective than an antidepressant drug, escitalopram.) 

“The idea that a patient would be able to use a device like this at home for treatment for some of these brain diseases like seizures or depression is very exciting,” says Michael Fox, a neurologist at Harvard Medical School. “It just needs to be done in a responsible and thoughtful way.”

A call for validation

In 2016, Fox was part of a group of clinicians and scientists who published an open letter in the Annals of Neurology in response to the rising do-it-yourself (DIY) brain stimulation movement, outlining the potential risks of using tDCS devices. “We published that letter to try and inform people that there are things that aren’t necessarily made clear [about these products] in academic publications that you should be aware of if you think about doing this yourself,” he tells The Scientist. “As scientists, we know how to read and interpret these studies in a certain way that’s not always apparent to the lay community.” (Fox has shared intellectual property with Neuroelectrics, a company that sells EEG and tDCS devices to scientists and medical professionals and conducts clinical trials with some of these products.)

InteraXon helped establish the Center for Responsible Brainwave Technologies to advance conversations about best practices, including “clarity and honesty” about claims of benefits.

Some of the issues raised in that paper included the possibility of unintended alterations to brain functions and the variability of tDCS outcomes. While consistent effects are seen when data are averaged across a group of people, they tend to be variable and unpredictable in any given individual, according to Fox. This means that “we don’t yet know whether this brain stimulation technology is having a very tiny effect and the variability is just noise in the data, or whether it’s having a fairly big effect in only some individuals.”

According to Hanslmayr, there’s another, very basic problem with tDCS: scientists don’t yet have a way to test if the devices are having a meaningful effect on the brain. With transcranial magnetic stimulation (TMS), a more-powerful noninvasive brain stimulation method, researchers can verify their devices by seeing if zapping a person’s motor cortex initiates a muscle twitch in their hands. There’s no such check for tDCS, however, since it doesn’t generate immediately observable behavioral outputs.

See “Study Raises Questions About Brain Stimulation Boosting Memory”

See “Noninvasive Brain Stimulation Turns Back Clock on Memory”

But such a test for tDCS devices might one day exist. Hanslmayr is currently part of an effort called the TACS Challenge, a consortium of around 50 scientists who are putting their heads together to figure out how to make this happen. “That’s in the very early stages,” he says. “But we’ll hopefully start data collection at the end of this year.”

Developing ethical codes

Some members of the neurotech industry have made efforts to act responsibility—and encourage others to do so as well. “There are companies that are really stepping up to the plate and being role models for others,” says Illes. She points to InteraXon as an example: The company helped establish the Center for Responsible Brainwave Technologies (CeReB), which was developed to advance conversations about best practices for consumer EEG devices, including “clarity and honesty” about claims of their benefits.

“The problem is you sort of get tarred by this brush of, you’re a consumer neurotechnology [company] and look at all these problem children out there,” InteraXon’s Moffat says. “The better companies in the space will collaborate extensively with academic researchers and established public institutions in ways that don't have conflicts of interest.” (There have been several studies conducted with Interaxon’s product, Muse, at various universities. This includes work that has provided evidence for the potential cognitive and emotional benefits of technology-mediated meditation, as well as research in other domains, such as the effect of neurofeedback on learning.)

Illes is optimistic. Although her latest study revealed widespread issues among currently available DTC neurodevices, she says she thinks the industry is trying to do better. “I don’t believe that there is a company doing neurotech that isn’t at least talking about ethics right now,” she adds. “And that’s a big step forward.”