Autistic individuals of show reduced frequency y of social indications and a tendency to engage in isolating and repetitive behaviors; the condition has also been linked to dysfunction in the amygdala. Researchers pinpointed antagonistic neuron populations in the mouse amygdala that control whether the animal participates in social behaviors or repetitive grooming.

The discovery could help researchers understand the neural circuit dysfunction that are behind autism in humans, the California Institute of Technology reported.

"We know that there is some hierarchy of behaviors, and they interact with each other because the animal can't exhibit both social and asocial behaviors at the same time. In this study, we wanted to figure out how the brain does that," said postdoctoral scholar Weizhe Hong in the laboratory of David J. Anderson, the Seymour Benzer Professor of Biology at Caltech.

The team discovered two "intermingled but distinct" neuron populations in the amygdala; one population promotes social behaviors while the other controls repetitive self-grooming. In order to study the relationship between these two cell types the researchers employed a technique called optogenetics, in which alters neurons so they express light-sensitive proteins from microbial organisms. When a light is shined on these modified neurons researchers can control the activity of the cells as well as their associated behaviors.

The behavior that was elicited depended on the intensity of the light signal. When high-intensity light was used the mice became aggressive in the presence of an intruder mouse, when high-intensity light was used the mice became more docile and no longer attacked. When the neurons associated with asocial behaviors were switched on the mice exhibited self-grooming behaviors and ignored intruders; they were able to halt this behavior through optogenetic activation of the social neurons.

The two groups of neurons were found to interfere with each other's functions; the activiation of social neurons stopped self-grooming behavior while the activation of self-grooming neurons stopped social behavior.

"In autism there is a decrease in social interactions, and there is often an increase in repetitive, sometimes asocial or self-oriented, behaviors" Anderson said.  "Here, by stimulating a particular set of neurons, we are both inhibiting social interactions and promoting these perseverative, persistent behaviors."

In the distant future the method could be used to modify human behavior.

"All of this is very far away, but if you found the right population of neurons, it might be possible to override the genetic component of a behavioral disorder like autism, by just changing the activity of the circuits -- tipping the balance of the see-saw in the other direction," Anderson said. 

The findings were published Sept. 11 in the journal Cell.