For nearly a decade, a team of researchers at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) has been trying to discover why some images persist in people’s minds while many others fade. To this end, they set out to map the spatiotemporal brain dynamics associated with visual image recognition. Now, for the first time, scientists have used the combined benefits of magnetoencephalography (MEG), which records the duration of brain activity, and functional magnetic resonance imaging (fMRI), which identifies lively brain areas, to precisely determine when and where brain processes are occurring. a memorable image.
Their study in open access, published this month in , used 78 pairs of images that fit the same concept but differed in terms of memorability – one was very memorable, the other was easily forgotten. These photos were shown to 15 people and depicted scenes of skateboarding, animals in different environments, everyday objects such as cups and chairs, natural landscapes such as forests and beaches, urban scenes of streets and buildings, and faces with different expressions. They found that a more distributed network of brain regions than previously thought is actively involved in the encoding and storage processes that underlie memory.
“People tend to remember some images better than others, even if they are conceptually similar, such as different scenes of a person riding a skateboard,” says Benjamin Lahner, an MIT doctoral candidate in electrical engineering and computer science, CSAIL collaborator and first author study books. “We identified a brain signature associated with visual memory that appears approximately 300 milliseconds after viewing an image and involves areas in the ventral occipital cortex and temporal cortex that process information such as color perception and object recognition. “This signature indicates that highly memorable images evoke stronger and longer-lasting brain responses, particularly in areas such as the early visual cortex that we previously underestimated in memory processing.”
While highly memorable images maintain a higher and longer-lasting response for about half a second, the response to less memorable images decreases rapidly. This insight, Lahner elaborated, could redefine our understanding of how memories are formed and maintained. The team anticipates that this research will have potential for future clinical applications, particularly in the early diagnosis and treatment of memory-related disorders.
The MEG/fMRI fusion method, developed in the laboratory of CSAIL senior scientist Aude Oliva, skillfully captures the spatial and temporal dynamics of the brain, overcoming conventional limitations resulting from spatial or temporal specificity. She was helped with the fusion method by a friend in machine learning who allowed her to better examine and compare brain activity when looking at different images. They created a “representational matrix” that resembles a detailed graph showing similar neural responses in different areas of the brain. This chart helped them identify patterns that determine where and when the brain processes what we see.
Selecting conceptually similar pairs of images with high and low recall scores was a key element in obtaining knowledge about recall. Lahner explained the process of aggregating behavioral data to assign recall metrics to images, in which he created a diverse set of high and low recall images with balanced representation across visual categories.
Despite the progress made, the team notes several limitations. Although this work can identify brain regions that show significant effects on memory, it cannot elucidate the function of these regions in terms of contributing to better memory encoding/retrieval.
“Understanding the neural basis of memory opens up exciting opportunities for clinical advances, particularly in the early diagnosis and treatment of memory-related disorders,” Oliva says. “The specific brain signatures we have identified for memory may lead to early biomarkers of Alzheimer’s disease and other dementias. This research paves the way for novel intervention strategies that are precisely tuned to an individual’s neural profile, potentially changing the therapeutic landscape for memory disorders and significantly improving patient outcomes.”
“These findings are exciting because they give us insight into what happens in the brain between seeing something and storing it in memory,” says Wilma Bainbridge, an assistant professor of psychology at the University of Chicago, who was not involved in the study. “Scientists pick up a cortical signal that reflects what to remember and what to forget early on.”
Lahner and Oliva, who is also director of strategic industry engagement at the MIT Schwarzman College of Computing, director of the MIT-IBM Watson AI Lab and principal investigator of CSAIL, join Western University assistant professor Yalda Mohsenzadeh and York University researcher Caitlin Mullin on the paper. The team is supported by a Joint Facilities Grant from the National Institutes of Health, and its work was funded by a Vannevar Bush Faculty Fellowship under an Office of Naval Research grant, a National Science Foundation award, a University Multidisciplinary Research Initiative award under an Army Research Office grant, and an EECS grant MathWorks Fellowship. Their article appears in .