Mapping the Neural Circuit of Social Avoidance

Oxytocin released in a region of the hypothalamus helps mice remember who to avoid.

Black and white portrait by Mariella Bodemeier Loayza Careaga, PhD
| 4 min read
A black mouse hides from a white mouse inside a paper tube.

Researchers studied the brains of mice that had lost a fight and uncovered a key role for oxytocin in regulating social avoidance learning.

© iStock, Georgejason

Register for free to listen to this article
Listen with Speechify
0:00
4:00
Share

Like humans, mice live in complex social groups in which learning to distinguish between who to approach and who to avoid is key. Scientists knew that after losing a fight, defeated mice quickly learn to shy away from their victorious opponents, but the neural mechanisms that drive these behavioral changes were not fully understood.1

Recently, researchers led by Dayu Lin, a neuroscientist at New York University, characterized a hypothalamic brain circuit that regulates social avoidance in mice and uncovered the essential role of oxytocin in facilitating neural plasticity after a single defeat.2 These findings, published in Nature, support the idea that the brain’s oxytocin modulates more than just prosocial behaviors in animals.

“They discovered a very different set of oxytocin neurons and oxytocin-receptor-expressing neurons that have never really been studied before,” said Brian Trainor, a behavioral neuroscientist at the University of California, Davis, who was not involved in the study.

When Lin started this research almost five years ago, she did not plan to study the brain circuits mediating social avoidance. “It was actually a project related to maternal aggression,” she explained. At the time, Lin wanted to investigate whether oxytocin released in the ventrolateral part of the ventromedial hypothalamus (VMHvl), a brain region implicated in aggressive behavior and social fear, mediated the increase in female aggression during lactation, a critical behavioral change that enables a mother to protect her offspring from threats.3-5

After data from a year of experiments negated that hypothesis, the researchers explored new ideas. While brainstorming with her team, Lin remembered an experiment she completed as a postdoctoral researcher a few years earlier. One day, out of curiosity, Lin looked at the expression of c-fos, a marker of neuronal activity, in the VMHvl of animals that had lost a fight. The c-fos profile in those brains resembled the oxytocin receptor patterns in the VMHvl of mouse mothers. This similarity intrigued Lin, who hypothesized that these oxytocin-receptor-expressing (OXTR) neurons could have a role in defeat-induced behaviors.

To test this idea, Lin and her team exposed male and female mice to an aggressive same-sex conspecific for 10 minutes and assessed the animals’ behavior and brain activity before and after they lost a fight. The researchers focused on the anterior part of the VMHvl (aVMHvl), as previous findings from Lin’s team indicated that cells in this region become active in mice defending themselves against aggressors.6

Fiber photometry recording of the OXTR neurons in the aVMHvl revealed increased cell responses when the researchers exposed the defeated animals to their bullies confined inside a perforated cup. ”Defeated animals spend most of [their] time away from the aggressor, in comparison to before the defeat, when they spend most of the time close to the aggressor,” Lin explained.

A mouse brain slice and brain cells expressing oxytocin receptors shown in green.
Oxytocin-receptor-expressing cells (green) in the anterior portion of VMHvl play a key role mediating social avoidance in mice.
Takuya Osakada

To assess the functional relevance of these cells to defeat-induced behaviors, the team optogenetically activated aVMHvl OXTR neurons in mice that had never experienced a fight. Stimulation of these cells in naïve animals induced strong shunning from a restrained animal, suggesting that increased activity of these neurons drives social avoidance. In contrast, optogenetic inhibition of these cells in defeated animals increased the time they spent investigating their confined bullies, indicating that these neurons are necessary to induce avoidance in mice that lost a fight.

Since these cells expressed oxytocin receptors, the team next sought the source of oxytocin for the aVMHvl neurons. By mapping the expression of c-fos and oxytocin after defeat, the researchers found that oxytocin-producing cells from different areas of the hypothalamus were engaged, with the highest percentage of them belonging to a region called the retrochiasmatic supraoptic nucleus (SOR).

Additional experiments looking at the SOR oxytocin-producing cells revealed that these neurons fired when the mice lost a fight, but not when the defeated animals later encountered their bullies. These cells increased their activity when the animals experienced different painful stimuli, such as being poked or pinched with a tweezer, suggesting that SOR oxytocin-producing cells convey sensory information about discomforting aspects of the fight. “I've never seen anyone publish anything about those cells,” Trainor said. “They really discovered something that was sitting there in plain sight.”

Electrophysiological recordings in brain slices further showed that stimulation of SOR oxytocin-producing cells led to membrane depolarization of aVMHvl neurons. Addition of an oxytocin receptor agonist to the brain slices increased the likelihood of aVMHvl cells firing in response to an excitatory input, suggesting that the neuropeptide strengthens synaptic transmission within the aVMHvl OXTR cells.

Moving forward, Lin plans to continue studying this circuit to uncover other neuronal populations and brain regions that may contribute to social avoidance learning. She also wants to examine this brain circuit in more naturalistic settings, such as during the establishment of hierarchy within a group of mice.

According to Trainor, the study findings reinforce the idea that oxytocin’s role in the brain goes beyond modulating prosocial behaviors, and that the neuropeptide acts as an enhancer of social experiences, making good experiences better and bad experiences worse. “It fills in this gap of oxytocin having these social avoidance types of effects,” Trainor said. “A lot of people have a hard time grasping that. Hopefully, this work will change some minds in the field.”

  1. Qi CC, et al. Interaction of basolateral amygdala, ventral hippocampus and medial prefrontal cortex regulates the consolidation and extinction of social fear. Behav Brain Funct. 2018;14(1):7.
  2. Osakada T, et al. A dedicated hypothalamic oxytocin circuit controls aversive social learning. Nature. 2024;626(7998):347-356.
  3. Lin D, et al. Functional identification of an aggression locus in the mouse hypothalamus. Nature. 2011;470(7333):221-226.
  4. Silva BA, et al. Independent hypothalamic circuits for social and predator fear. Nat Neurosci. 2013;16(12):1731-1733.
  5. Georgescu T, et al. Prolactin-mediated restraint of maternal aggression in lactation. Proc Natl Acad Sci U S A. 2022;119(6):e2116972119.
  6. Wang L, et al. Hypothalamic control of conspecific self-defense. Cell Rep. 2019;26(7):1747-1758.e5.

Keywords

Meet the Author

  • Black and white portrait by Mariella Bodemeier Loayza Careaga, PhD

    Mariella Bodemeier Loayza Careaga, PhD

    Mariella is an assistant editor at The Scientist. She has a background in neuroscience, and her work has appeared in Drug Discovery News and Massive Science.
Share
You might also be interested in...
Loading Next Article...
You might also be interested in...
Loading Next Article...
3D illustration of a gold lipid nanoparticle with pink nucleic acid inside of it. Purple and teal spikes stick out from the lipid bilayer representing polyethylene glycol.
February 2025, Issue 1

A Nanoparticle Delivery System for Gene Therapy

A reimagined lipid vehicle for nucleic acids could overcome the limitations of current vectors.

View this Issue
Considerations for Cell-Based Assays in Immuno-Oncology Research

Considerations for Cell-Based Assays in Immuno-Oncology Research

Lonza
An illustration of animal and tree silhouettes.

From Water Bears to Grizzly Bears: Unusual Animal Models

Taconic Biosciences
Sex Differences in Neurological Research

Sex Differences in Neurological Research

bit.bio logo
New Frontiers in Vaccine Development

New Frontiers in Vaccine Development

Sino

Products

Tecan Logo

Tecan introduces Veya: bringing digital, scalable automation to labs worldwide

Inventia Life Science

Inventia Life Science Launches RASTRUM™ Allegro to Revolutionize High-Throughput 3D Cell Culture for Drug Discovery and Disease Research

An illustration of differently shaped viruses.

Detecting Novel Viruses Using a Comprehensive Enrichment Panel

Twist Bio 
Zymo Research

Zymo Research Launches Microbiome Grant to Support Innovation in Microbial Sciences