How the brain learns from mistakes

How the brain learns from mistakes

Learning in the cerebellum -- a part of the vertebrate brain located at the back of the skull that directs and regulates movements -- is guided through a process of trial and error. For example, when learning to shoot a basketball, people usually miss many times before getting one shot through the hoop. As the arm moves, the cerebellum makes predictions about the consequences of the action. When the prediction does not match reality -- that is, the ball misses the hoop -- the cerebellum receives feedback from the eyes and the arm to learn from the error, fine-tuning factors such as aim, force and release to make a basket. This trial-to-trial learning from error produces gradual improvements in performance.

However, research efforts have fallen short of decoding how the cells in the cerebellum make predictions and how they learn from their mistakes, according to the senior author.

To decode the signal, the team began with the principal neurons in the cerebellum, called Purkinje cells. Purkinje cells communicate through two types of electrical signals: simple spikes, which reflect information regarding the prediction that the cells are making, and complex spikes, which reflect information that is sent back to the Purkinje cells, informing them of the error in their prediction.

"You can think of the simple spikes as the 'student' that makes a prediction and the complex spikes as the 'teacher' that provides feedback," says the senior author.

In a past study, the research team found that the Purkinje cells organize into small groups of about 50 and together make predictions, sending their output of all the members simultaneously. The neurons that make up these groups share a critical feature: They all receive the same error signal. Therefore, the fundamental computational unit in the cerebellum was not a single Purkinje cell, but a small group that learns together from a common mistake. While earlier work had tried to decipher the simple spikes of individual Purkinje cells, these results suggested that the key to understanding the cerebellum was to organize the Purkinje cells into groups and then count the sum total of simple spikes being produced by the members of the group. By doing so, the team discovered that the group generated simple spikes that precisely predicted the motion of a monkey's eyes as it gazed around a screen.

Having found a way to decipher the language with which the cerebellum made predictions, the next step was to understand how the cerebellum learns from its prediction errors. To investigate this question, the researchers measured cerebellar activity in seven rhesus monkeys fitted with electrodes and trained them to follow a small target with their eyes. The researchers replicated a "mistake" by moving the target more quickly than the monkeys could accurately predict, causing them to frequently miss the target.

They found that simple spikes measured through the electrodes correlated with the intensity of movement -- more simple spikes meant that the monkeys needed to move their eyes more quickly to catch the target. According to the researchers, the Purkinje cells produced an average of 50-70 spikes per second.

The researchers found that the complex spikes did not convey error in the same manner. Regardless of how far off the monkeys' eyes were from the target, the number of complex spikes produced in the cerebellum stayed the same. Instead, they found that the direction of the error affected the probability of generating a complex spike, whereas the magnitude of the error affected the timing of the complex spike. Therefore, the language of prediction was in rate of simple spikes, whereas the language of prediction error was in timing of the complex spike.

For example, the team found that when the monkeys missed the target by 3 degrees of distance, a complex spike appeared after 100 milliseconds, and if the monkeys missed by 8 degrees, a complex spike appeared in 120 milliseconds.

"In short, the 'language' of the teacher is time, while the language of the student is rate," says the senior author, and explains that current AI machines in the world rely on the same basic structure and are composed of many layers of neuronlike units that together make predictions. Layers of such units make calculations, passing results to the next layer until the process reaches the "output" layer that makes the final prediction. When an AI machine's prediction is wrong, it experiences an error and, with current AI networks, sends that error back to all units.

"These machines are inspired by the way biological networks learn, but what is interesting is that the architecture of the cerebellum is somewhat different than what has been designed in artificial systems," says the senior author. Instead of all the cells receiving complex spikes after the error, in the cerebellum the Purkinje cells organize into small groups, each specializing in responding to specific errors. The Purkinje cells appear to be organized based on a preference for error in only a small part of the task space.

One of the advantages of the cerebellum's architecture, says the senior author, is that it protects memories. When you experience an error to the right, it causes learning by engaging Purkinje cells that register that error. If on the next trial you experience an error to the left, the new error engages a new group of Purkinje cells. As a result, new errors do not erase the memory, but result in two independent memories.

Damage to the cerebellum can profoundly impair the ability to make movements, resulting in symptoms that together are called ataxia. A puzzle that has remained unsolved is how the cerebellum translates its predictions into motor commands that guide movements.

This final result may make it possible to refine diagnosis of cerebellar disease by measuring learning from error and allow scientists to connect motor learning impairment with the loss of specific groups of Purkinje cells.