Hippocampal pattern separation supports reinforcement learning

Avid hikers know to be cautious of plants with leaves made up of three leaflets if they are red in the spring or fall. Parents worldwide know the precarious relationship between proximity to bedtime and roughhousing with their children.

How do hikers know to link the color of a leaf with the season to determine if it is poison ivy? How do parents know to link the time of day with a child's excitement level to determine the success of a bedtime routine? Just like Pavlov's dogs salivated when a bell rang, people learn to recognize poisonous plants or prevent tears of exhaustion in a toddler by forming associations among details in their surroundings and what happens.

Researchers analyzed patterns of brain activity in humans and discovered a previously unknown role for the hippocampus, a brain area important to memory, in forming associations during learning. The study will be published in Nature Communications.

When people learn, they build associations among features - like the time of day and mood of a toddler - to predict an outcome, like whether or not roughhousing will end in tears.

"These associations have a big influence on behavior," said the first author on the paper. "But in the real world, objects or events are defined by more than one feature or combinations of features, and we wanted to understand how the brain builds associations over similar configurations of features."

To unpack how the brain might handle the problem of building associations over such complex, real-world information, the research team focused on a small brain structure that looks like a snail shell.

The hippocampus is approximately one-third the size of a gumball, and this small brain structure is crucial for memory formation. Without it, people cannot form new memories about facts or events, like what day it is or the names of coworkers. During memory formation, the hippocampus represents individual details of an event, like where you parked your car, to be as different as possible from each other.

"One challenge with memory is that it is hard to distinguish similar experiences. So, if you use the same parking garage at work every day, you have to remember which floor and space to go to at the end of the day," said senior author on the paper. "The problem is that it is easy to confuse where you parked on different days. The hippocampus is critical for remembering the combination of where and when."

How the hippocampus forms memories provided a mechanism for how complex combinations of features could be represented in the brain, but whether the small, coiled structure actually contributed to how people learn about the world was an open question.

To identify how the hippocampus might contribute to how people form associations in the real world, the researchers designed a learning task that required participants to use combinations of features to predict whether an outcome would happen. A series of stimulus images, like a single face or a face paired with a building, appeared sequentially on a screen. The participants then had to predict whether a target image would appear after the stimulus images. The goal for the task was to respond as quickly as possible to any target image that showed up.

It was only the combinations of two stimulus pictures, like a face paired with a townhouse, that could be used to predict when a target would appear. The individual stimuli, like the face image, were not useful for predicting by themselves.

As the participants worked through the task, the research team used functional magnetic resonance imaging (fMRI) to measure brain activity from the hippocampus and other brain structures known to be involved in learning. The team then examined the patterns of activity during the task and noticed something interesting about the activity in the hippocampus. It was the only brain structure that represented the stimulus images bound together, which is important because success on the task required forming associations about combinations to accurately respond to the target images.

"We found the hippocampus uniquely represented bound features: faces and houses were distinct from a face and house mingled together," the senior author said.

When the research team looked at how the patterns of activity in the hippocampus were related to other brain areas, they found hippocampal activity was tightly correlated with activity in the striatum. Nestled beneath the cortex, the striatum consists of three separate structures - the caudate, putamen and nucleus accumbens - and plays an important role in learning what predicts desirable outcomes.

"The hippocampus formed bound associations of multiple features that supported learning in the striatum about the configuration of multiple features in the environment," the senior author said.

How the striatum handles information during learning is well known, but where all the information comes from is an open question. The author added that this study starts to chip away at that question by showing the hippocampus provided information about combinations of features to the striatum and that information was used to learn how to succeed at the task.

Until recently, the brain was thought to have separate learning systems, but the findings suggest the hippocampal memory system and striatal reinforcement learning system are interrelated.

"It is really important to think of the brain as an interconnected structure, with different parts that work together to produce our impressive mental feats. Neuroscience and psychology have done a good job at understanding how individual parts work. It is exciting to start trying to figure out how they begin to work together," the first author said. "We suspect that understanding both normal and diseased mental functions requires figuring out how all of the parts work together -- or not."

https://www.nature.com/articles/s41467-019-08998-1