How Augmented Reality is Advancing Brain and Mental Health Treatment

Imagine transforming any room into a behavioral maze that neuroscientists can use to study how the human brain processes rewards. What about bringing therapy directly into a patient’s home with guided instruction and realistic props? Once a distant dream, augmented reality (AR) is now pushing neuroscience research into simulations to explore and treat real-world disorders.

Understanding how the brain responds to stimuli and drives behavior is important for unraveling the foundation of neurological and psychological disorders. Although scientists can model many of these processes in animals, studying them in humans presents a greater challenge. AR technology bridges this gap, providing researchers with a powerful tool to explore brain function and develop targeted interventions.

A Moving Body Behaves Differently

Travis Baker, a cognitive neuroscientist at Rutgers University, studies how alterations in goal-oriented behavior manifest as psychological disorders, including substance abuse disorder. He explained that, while researchers typically study these processes in animals using mazes, when the studies are translated to humans, they are converted into button-pressing tasks.

Although these tests enable researchers to measure decision-making phenomena with the help of an electroencephalogram (EEG), it is unclear how well these oversimplified designs replicate complex tasks in the real-world. “It's a kind of impoverished way to actually look at cognition,” Baker said. Researchers previously demonstrated that movement influenced cognitive performance, which led Baker to explore ways to make cognitive neuroscience research more representative of real-world situations.¹

Travis Baker (left, wearing a blue shirt) and Jaleesa Stringfellow (right, wearing a black shirt) smile at the camera while wearing AR headsets. — Travis Baker and Jaleesa Stringfellow use AR to study goal-oriented behavior.
Travis Baker

“A very important aspect of navigation is actually movement and immersing in your environment,” Baker said. “[Rats will] walk down a T-maze, and they'll make a decision, so you can record brain waves that are very specific to navigation. But with humans, you can't if you're just sitting still and you're just using 2D environments.”

Previously, Baker and his team used virtual reality (VR) technology to study brain activity in humans as they navigated a simple maze.² While they demonstrated that movement influenced brain activity in participants' making goal-oriented decisions, VR systems induce motion sickness in some participants. Additionally, participants cannot see themselves, creating a disembodied sensation, and a fully-simulated environment restricts their range of movement.

Baker and his team turned to AR to overcome these limitations. Unlike VR, AR overlays simulated content onto a participant’s real-world field of view. “Using augmented reality is great when you think about wanting to look at a stimulus or wanting to look at a certain type of behavior in a more naturalistic setting,” said Jaleesa Stringfellow a cognitive neuroscience graduate student in Baker’s group. Additionally, AR gives researchers the ability to study variables that they could not investigate with traditional button-pressing models, like velocity and the amount of time that participants look at a cue.

Traditionally, EEG measurements require participants to be as still as possible, even to the point of not speaking, to collect accurate readings, limiting its use in studies with mobile participants. “We kind of threw that all out the window so we can get a more naturalistic setting where people are moving around, they're talking and we're recording,” Stringfellow said. To do this, the team incorporated mobile EEG systems, which have developed in parallel with VR and AR technology. “The data is a little bit noisier,” Baker said about recording mobile participants’ brain electrical activity. But, “We get the same data [to traditional EEG], which is really very cool.”

Recently, Stringfellow demonstrated the feasibility of this AR system for measuring goal-oriented behavior.³ Participants wearing AR goggles had the option to move either to the left or the right toward one of two identical visual cues. Once they came within range of the cue, the system revealed either a reward of five cents or no reward. The team used a mobile EEG that allowed them to simultaneously measure neural activity in response to reward outcomes and track a participant's behavior as they navigated the virtual maze.

Although they observed brain activity comparable to previous screen-based tasks, they noticed differences in how participants behaved during the trials. Whereas participants in a button-pressing experiment were more likely to repeat their decision following a reward, Baker’s team saw that participants in AR and VR studies were equally likely to repeat their decision or try the other direction, which the researchers attributed to the differences in cognition that arise during navigation. “When people are behaving in a more naturalistic environment,” Baker explained, “it’s closer to the actual cognition that we are using in our day-to-day life.”

Although the research is still in its infancy, Stringfellow said that the team looks forward to using AR to study increasingly realistic responses. “It just really broadens or expands our research beyond the laboratory, where we could actually go into the real world and get laboratory quality neuroimaging data,” Stringfellow said.

Baker explained that by mapping the neural pathways involved in both normal and disordered goal-orientation, researchers could develop interventions targeting these altered circuits. However, for some psychiatric and neurological conditions, AR could serve as a treatment itself.

Facing Fears with Simulations

Photograph of Arash Javanbakht standing on the sidewalk by red brick townhomes. He is bald and is wearing a dark shirt and gray dress jacket. — Arash Javanbakht sees patients who have anxiety disorders and PTSD. He developed an AR system to deliver exposure therapy for these individuals.
Beau Rosario

When it comes to surviving threatening situations, most people think about the flight or fight response. “Biologically speaking, fear is one of our most fundamental aspects,” said Arash Javanbakht, a psychiatrist and neuroscientist at Wayne State University. He explained that the fear response is vital to ensure that an organism stays alive and remains safe. However, this process can go awry, leading to psychiatric disorders such as anxiety and post-traumatic stress disorder (PTSD). “These are conditions that, for [a] variety of reasons, people have been sensitized to some neutral situation that they're avoiding,” he said.

In response to a potential threat, the amygdala, the brain's rapid response center, sends signals to the hippocampus and prefrontal cortex, which pull contextual information from the environment to determine whether the cue poses a danger and initiate a bodily response.^4-6 While this system is critical for survival, in individuals with fear- and anxiety-related disorders, the amygdala becomes overreactive while signaling from the hippocampus and prefrontal cortex is weakened. Although individuals suffering from anxiety are aware that their fears may be irrational in the absence of obvious danger, they are unable to overcome the overwhelming emotions.

While medication can alleviate symptoms, it does not address the fear responses to specific triggers. Instead, psychiatrists like Javanbakht recommend exposure therapy to help patients confront and overcome these fears. Similar to the idea of exposing the immune system to an allergen, like peanuts, in small quantities to retrain overreactive responses, psychological exposure therapy aims to retrain the brain to not react to specific triggers, gradually reducing fear and anxiety responses. “The new memory forms that now, within this context, you’re safe,” Javanbakht said.

However, because of limitations in the ability to effectively introduce individuals’ specific triggers in a clinical setting, this therapy is often underutilized.

Javanbakht first heard about AR in a TEDTalk that Meron Gribetz gave on the technology and immediately saw its potential to help his patients. However, for it to work, Javabakht knew that AR simulations would need to induce the same fear response as the real phobia cue.

Collaborating with the AR company CrossComm, Javanbakht designed a pilot system to test people’s reaction to AR-generated spiders to try to treat arachnophobia.⁷ “What amazed us was that this terrifies people,” he said.

Not only did the simulated spiders induce fear responses comparable to real insects, after just one session, most patients exhibited significant improvements in their phobia. In an ongoing clinical trial, Javanbakht and his collaborators are using their system to treat patients who have a fear of dogs.

Unlike a specific phobia, generalized anxiety like in PTSD is harder to address. “[For] people with PTSD, their challenge is that the brain is in fight and flight mode—it doesn't leave. So, they're afraid of being around people, they don't go to the grocery store, they don't go to a restaurant, they don't go shopping, they don't go to parties,” Javanbakht said.

Bringing these fears to life requires the simulation of humans and speech, which is much more difficult. So, to make this happen, Javanbakht collaborated with product design studio 2XPI and incorporated artificial intelligence to create realistic avatars that participants can engage in complete conversations. Additionally, with his new system, he can introduce features into the participant’s field of view so that they experience walking in a specific location. The system currently includes multiple locations and more than 100 characters that a physician can select to set the scene. “Basically, I turn the therapist to a movie director,” he said.

In a preliminary study, Javanbakht modified the AR technology to treat PTSD in first responders.⁸ Similar to his trials with spiders and dogs, Javanbakht said that the AR-simulated avatars and environments induce fear responses in his patients. “It's not pleasant to see the negative emotions come up, but this is what we want in therapy, because then we can address them and habituate to those emotions,” Javanbakht explained.

Photo of a user’s view wearing an AR headset depicting a grocery store. Within the participant’s real-world view of a carpeted room with white walls, there are simulations of grocery shelves holding beverages and chips, with simulated avatars of shoppers and employees. — Javabakht and his collaborators developed scenes depicting real-life locations, like grocery stores, for participants to practice exposure therapy in realistic settings.
Arash Javanbakht

Javanbakht and his colleagues will begin recruiting participants for a formal clinical trial beginning in the summer of 2025, but he said that the early feedback he received from national experts in PTSD treatments about his AR optimization study is promising. When one of his patients started the therapy, they were only able to handle a simulation with two individuals in a room. “It was very hard for her,” Javanbakht recalled. However, gradually, the patient gained confidence, eventually working through more scenarios with more people until she could face one that resembled the source of her trauma.

“[Medication and talk therapy] had helped with their nightmares, with the flashback, with hyperarousal,” Javanbakht said. “But the avoidance was still there. They still weren't doing anything. They still weren't going out, and this treatment helped with those avoidances.”

AR-Guided Therapy for Better Mobility

Beyond behavioral interventions, researchers are looking to AR as a tool to advance clinical interventions for neurodegenerative diseases like Parkinson’s disease (PD). PD results in a decline in both cognitive and motor functions, making tasks like walking down busy streets difficult.⁹ This disease also impairs mobility and increases an individual’s chance of falling.

The most effective therapy for treating these symptoms is dual-task training, where a therapist guides a patient to complete a motor and cognitive task simultaneously.¹⁰ “The problem is, it doesn’t get used in clinical practice,” said Jay Alberts, a biomedical engineer at Cleveland Clinic.

Developing effective tasks for specific patients and monitoring all aspects of a person’s movement is difficult. These shortcomings encouraged Alberts to develop a technology that could help clinicians more easily provide dual-task therapy to their patients.

He and his team created a digital avatar within an AR platform that gives patients instructions to complete simultaneous motor and cognitive tasks and records how effectively they complete the directives. In two Phase II clinical trials, patients who used the AR system had comparable improvements in multiple walking parameters compared to patients who received traditional dual-task therapy.^11,12

Seeing the positive response to the technology for therapy, Alberts turned his attention to the early stages of PD: detection. Five to seven years before their PD diagnosis, individuals experience declines in their ability to complete routine tasks important to independent living like shopping and driving.¹³ However, current assessments rely on either self-reported surveys or subjective scoring metrics.

To improve early PD detection, Alberts and his team developed a VR assessment that led participants through a virtual shopping experience with the help of an omnidirectional treadmill. As they traversed the simulated grocery store, the researchers measured the participants’ speed and their time spent walking, stopping, looking at the virtual shopping list, and taking turns from aisle to aisle. When compared to traditional dual-task clinical measuring, the researchers showed that their technology detected discrepancies in patients with PD overlooked by the current clinical assessment.¹⁴

Alberts and his team are currently exploring an AR assessment to record information about how an individual moves while completing a dual-task to identify early signs of PD and track its progression. “It’s really an opportunity with these [activities] to provide a very objective, quantitative assessment of cognitive and motor function simultaneously,” Alberts said.

In the future, Alberts and his team intend to integrate their AR system with patients receiving deep brain stimulation intervention for PD, aiming to measure how that therapy affects patients’ motor and cognitive functions.

As these devices continue to decrease in cost, Alberts sees an opportunity for patients to be able to remotely complete neurological assessments or therapy tasks. “I’m from a small town in Iowa,” Albert noted. “If you have Parkinson's in my town, you have to drive 75 miles for a movement disorders neurologist. And so what happens is people just don't go.”

If AR equipment could be sent to patients' homes, then the results from these remote sessions could be sent to specialists for review of an individual’s progress, closing this gap in medical care. “I’m excited about this technology in the sense that it really is enabling and empowering for not a huge cost.”

Disclosure of Conflicts of Interest: Arash Javanbakht is the inventor on a patent for the development of ARET. The patent is owned by Wayne State University.

Disclosure of Conflicts of Interest: Jay Alberts is an inventor of the dual-task augmented reality system discussed in this story. This technology has been licensed to STROLL, a digital therapeutic software company, who pays royalties to the Cleveland Clinic. Alberts is also on the scientific advisory board for Curate Health and Ceraxis Health.

References