When we learn a up-to-date skill, the brain must decide, cell by cell, what to change. Fresh research from MIT suggests it can do this with surprising precision by sending targeted feedback to individual neurons so that each can adjust its activity in the right direction.
The discovery reflects a key idea of current artificial intelligence. Many AI systems learn by comparing their output with target values, calculating an “error” signal, and using it to fine-tune connections in the network. A long-standing question has been whether the brain also uses this type of personalized feedback. In an open-access study published in the journal February 25 issue of the magazine MIT researchers present evidence that yes.
Research team led by Mark Harnettresearcher at the McGovern Institute for Brain Research and associate professor in MIT’s Department of Brain and Cognitive Sciences, discovered these instructive signals in mice by training animals to control the activity of specific neurons using a brain-computer interface (BCI). The researchers say their approach could be used to further explore the relationship between artificial neural networks and real brains in a way that is expected to both improve understanding of biological learning and enable better brain-inspired artificial intelligence.
The changing brain
Our brains are constantly changing as we interact with the world, modifying their circuits as we learn and adapt. “We know a lot from 50 years of research that there are many ways to change the strength of the connections between neurons,” Harnett says. “What’s really missing in the field is a way to understand how these changes are organized to actually deliver effective learning.”
Certain actions—and the neural connections that enable them—are enhanced by the release of neuromodulators such as dopamine or norepinephrine in the brain. However, these signals are sent to huge groups of neurons, without distinguishing between individual cells’ contributions to failure or success. “Reinforcement learning with neuromodulators works, but it is inefficient because all neurons and all synapses essentially receive only one signal,” Harnett says.
Machine learning uses an alternative and extremely effective way of learning from errors. Using a method called backpropagation, artificial neural networks calculate the error signal and operate it to adjust individual connections. They do this over and over again, learning from experience how to fine-tune their networks for success. “It works really well and is very computationally efficient,” Harnett says.
It seemed likely that brains might operate similar error signals to learn. Neuroscientists, however, were skeptical that brains would have enough precision to send tailored signals to individual neurons because of the limitations imposed by using living cells and circuits rather than software and equations. The main problem in testing this idea was finding the signals that provide the neurons with personalized instructions, called vector instruction signals. The challenge, explains Valerio Francioni, the paper’s first author and a former postdoc in Harnett’s lab, is that scientists don’t know how individual neurons contribute to specific behaviors.
“If I recorded your brain activity while you were learning to play the piano,” Francioni explains, “I would learn that there is a correlation between changes in your brain and learning to play the piano. But if you asked me to make you a better pianist by manipulating your brain activity, I wouldn’t be able to do that because we don’t know how the activity of individual neurons translates into that final performance.”
Without knowing which neurons require more activity and which need to be tamed, it is impossible to look for signals that guide these changes.
Understanding the functions of neurons
To get around this problem, Harnett’s team developed a brain-computer interface task that is designed to directly link neural activity to the reward effect—like piano keys are directly linked to the activity of individual neurons. To be successful, some neurons had to boost their activity, while others had to decrease their activity.
They created the BCI to directly link the activity of these neurons – just eight to 10 of the millions of neurons in the mouse brain – to a visual readout, providing the mice with sensory feedback on their activity. Success was accompanied by the delivery of a sweet reward.
“Now, if you ask me, ‘How does a mouse get more rewards?’ Which neuron do you need to activate and which should you inhibit? I know exactly what the answer to that question is,” says Francioni, whose work was supported by a Y. Eva Tan Fellowship from the Yang Tan Collective at MIT.
The researchers didn’t know the exact function of the individual neurons they connected to the BCI, but the cells were active enough that the mice received occasional rewards whenever the signals were correct. Within a week, the mice learned to turn on the appropriate neurons, leaving the other set of neurons inactive, thus earning more rewards.
Francioni monitored target neurons daily during the learning process, using a powerful microscope to visualize fluorescent indicators of neural activity. He focused on the branching dendrites of neurons, where it had long been suspected that the appropriate feedback signals arrived. At the same time, he tracked activity in the stem cell bodies of these neurons. The team used this data to examine the relationship between the signals received by a neuron’s dendrites and its activity, and how these changes changed when the mice were rewarded for activating the appropriate neurons or when they failed to complete the task.
Vectorized neural signals
They concluded that the two groups of neurons whose activity was controlled by BCI in opposite ways also received opposing error signals on their dendrites when the mice learned. Some were told to increase their activity while performing the task, while others were told to decrease their activity. Moreover, when the team manipulated the dendrites to inhibit these instructive signals, the mice failed to learn the task. “This is the first biological evidence that has been vectorized [neuron-specific] signal-based learning takes place in the cerebral cortex,” says Harnett.
The discovery of vectorized signals in the brain – and the team’s ability to find them – should encourage more exchanges between neuroscientists and machine learning researchers, says postdoc Vincent Tang. “This provides additional incentive for the machine learning community to continue developing models and generating new hypotheses in this direction,” he says. “Then we can go back and test them.”
The researchers say they are as excited about applying their approach to future experiments as they are about their current discovery.
“Machine learning offers a hearty, mathematically applicable way to truly study learning. The fact that we can now translate at least some of this directly to the brain is hugely significant,” says Francioni.
Harnett says this approach opens up new opportunities to explore possible similarities between the brain and machine learning. “Now we can learn how the cerebral cortex learns? How do other areas of the brain learn? How similar or how different is it to this particular algorithm? Can we learn how to build better, more brain-inspired models based on what we learn from biology?” says. “It really feels like a big new beginning.”