Towards semantic fMRI neurofeedback: navigating among mental states using real-time representational similarity analysis

AbstractObjectiveReal-time functional magnetic resonance imaging neurofeedback (rt-fMRI-NF) is a non-invasive MRI procedure allowing examined participants to learn to self-regulate brain activity by performing mental tasks. A novel two-step rt-fMRI-NF procedure is proposed whereby the feedback display is updated in real-time based on high level (semantic) representations of experimental stimuli via real-time representational similarity analysis of multi-voxel patterns of brain activity.ApproachIn a localizer session, the stimuli become associated with anchored points on a two-dimensional representational space where distances approximate between-pattern (dis)similarities. In the NF session, participants modulate their brain response, displayed as a movable point, to engage in a specific neural representation. The developed method pipeline is verified in a proof-of-concept rt-fMRI-NF study at 7 Tesla using imagery of concrete objects. The dependence on noise is more systematically assessed on artificial fMRI data with similar (simulated) spatio-temporal structure and variable (injected) signal and noise. A series of brain activity patterns from the ventral visual cortex is evaluated via on-line and off-line analyses and the performances of the method are reported under different noise conditions.Main resultsThe participant in the proof-of-concept study exhibited robust activation patterns in the localizer session and managed to control the neural representation of a stimulus towards the selected target, in the NF session. The offline analyses validated the rt-fMRI-NF results, showing that the rapid convergence to the target representation is noise-dependent.SignificanceOur proof-of-concept study demonstrates the potential of semantic NF designs where the participant navigates among different mental states. Compared to traditional NF designs (e.g. using a thermometer display to set the level of the neural signal), the proposed approach provides content-specific feedback to the participant and extra degrees of freedom to the experimenter enabling real-time control of the neural activity towards a target brain state without suggesting a specific mental strategy to the subject.

Download Full-text

Potential cortical and computational biases in representational similarity analysis

10.32470/ccn.2018.1127-0 ◽

2018 ◽

Author(s):

Daniel Leeds ◽

David Shutov

Keyword(s):

Similarity Analysis ◽

Representational Similarity Analysis

Download Full-text

Stable neural representations of mental states across target people and stimulus modalities

10.31234/osf.io/7axzj ◽

2020 ◽

Author(s):

Miriam E. Weaverdyck ◽

Mark Allen Thornton ◽

Diana Tamir

Keyword(s):

Mental State ◽

Mental States ◽

Brain Regions ◽

The Self ◽

Similarity Analysis ◽

Multiple Modalities ◽

Neural Representations ◽

Conceptual Core ◽

Stimulus Modalities ◽

Huge Variety

Each individual experiences mental states in their own idiosyncratic way, yet perceivers are able to accurately understand a huge variety of states across unique individuals. How do they accomplish this feat? Do people think about their own anger in the same ways as another person’s? Is reading about someone’s anxiety the same as seeing it? Here, we test the hypothesis that a common conceptual core unites mental state representations across contexts. Across three studies, participants judged the mental states of multiple targets, including a generic other, the self, a socially close other, and a socially distant other. Participants viewed mental state stimuli in multiple modalities, including written scenarios and images. Using representational similarity analysis, we found that brain regions associated with social cognition expressed stable neural representations of mental states across both targets and modalities. This suggests that people use stable models of mental states across different people and contexts.

Download Full-text

Should you trust your RSA result? A Bayesian method for reducing bias in neural representational similarity analysis.

Journal of Vision ◽

10.1167/17.10.571 ◽

2017 ◽

Vol 17 (10) ◽

pp. 571

Author(s):

Ming Bo Cai ◽

Nicolas Schuck ◽

Michael Anderson ◽

Jonathan Pillow ◽

Yael Niv

Keyword(s):

Bayesian Method ◽

Similarity Analysis ◽

Representational Similarity Analysis

Download Full-text

Neural evidence for the prediction of animacy features during language comprehension: Evidence from MEG and EEG Representational Similarity Analysis

10.1101/709394 ◽

2019 ◽

Author(s):

Lin Wang ◽

Edward Wlotko ◽

Edward Alexander ◽

Lotte Schoot ◽

Minjae Kim ◽

...

Keyword(s):

Spatial Patterns ◽

Language Comprehension ◽

Neural Activity ◽

Semantic Processing ◽

Sentence Comprehension ◽

Brain Activity ◽

Coarse Grained ◽

Similarity Analysis ◽

Semantic Features ◽

Representational Similarity Analysis

AbstractIt has been proposed that people can generate probabilistic predictions at multiple levels of representation during language comprehension. We used Magnetoencephalography (MEG) and Electroencephalography (EEG), in combination with Representational Similarity Analysis (RSA), to seek neural evidence for the prediction of animacy features. In two studies, MEG and EEG activity was measured as human participants (both sexes) read three-sentence scenarios. Verbs in the final sentences constrained for either animate or inanimate semantic features of upcoming nouns, and the broader discourse context constrained for either a specific noun or for multiple nouns belonging to the same animacy category. We quantified the similarity between spatial patterns of brain activity following the verbs until just before the presentation of the nouns. The MEG and EEG datasets revealed converging evidence that the similarity between spatial patterns of neural activity following animate constraining verbs was greater than following inanimate constraining verbs. This effect could not be explained by lexical-semantic processing of the verbs themselves. We therefore suggest that it reflected the inherent difference in the semantic similarity structure of the predicted animate and inanimate nouns. Moreover, the effect was present regardless of whether a specific word could be predicted, providing strong evidence for the prediction of coarse-grained semantic features that goes beyond the prediction of individual words.Significance statementLanguage inputs unfold very quickly during real-time communication. By predicting ahead we can give our brains a “head-start”, so that language comprehension is faster and more efficient. While most contexts do not constrain strongly for a specific word, they do allow us to predict some upcoming information. For example, following the context, “they cautioned the…”, we can predict that the next word will be animate rather than inanimate (we can caution a person, but not an object). Here we used EEG and MEG techniques to show that the brain is able to use these contextual constraints to predict the animacy of upcoming words during sentence comprehension, and that these predictions are associated with specific spatial patterns of neural activity.

Download Full-text