scholarly journals Traces of Meaning Itself: Encoding distributional word vectors in brain activity

2019 ◽  
Author(s):  
Jona Sassenhagen ◽  
Christian J. Fiebach
Keyword(s):  
Language Processing ◽  
Word Processing ◽  
Time Window ◽  
Brain Activity ◽  
Word Meaning ◽  
Human Mind ◽  
Semantic Knowledge ◽  
Brain Potentials ◽  
Word Onset ◽  
Vector Representations

AbstractHow is semantic information stored in the human mind and brain? Some philosophers and cognitive scientists argue for vectorial representations of concepts, where the meaning of a word is represented as its position in a high-dimensional neural state space. At the intersection of natural language processing and artificial intelligence, a class of very successful distributional word vector models has developed that can account for classic EEG findings of language, i.e., the ease vs. difficulty of integrating a word with its sentence context. However, models of semantics have to account not only for context-based word processing, but should also describe how word meaning is represented. Here, we investigate whether distributional vector representations of word meaning can model brain activity induced by words presented without context. Using EEG activity (event-related brain potentials) collected while participants in two experiments (English, German) read isolated words, we encode and decode word vectors taken from the family of prediction-based word2vec algorithms. We find that, first, the position of a word in vector space allows the prediction of the pattern of corresponding neural activity over time, in particular during a time window of 300 to 500 ms after word onset. Second, distributional models perform better than a human-created taxonomic baseline model (WordNet), and this holds for several distinct vector-based models. Third, multiple latent semantic dimensions of word meaning can be decoded from brain activity. Combined, these results suggest that empiricist, prediction-based vectorial representations of meaning are a viable candidate for the representational architecture of human semantic knowledge.

scholarly journals Traces of Meaning Itself: Encoding Distributional Word Vectors in Brain Activity

2020 ◽  
Vol 1 (1) ◽  
pp. 54-76 ◽  
Author(s):  
Jona Sassenhagen ◽  
Christian J. Fiebach
Keyword(s):  
Language Processing ◽  
Word Processing ◽  
Time Window ◽  
Brain Activity ◽  
Word Meaning ◽  
Human Mind ◽  
Semantic Knowledge ◽  
Brain Potentials ◽  
Word Onset ◽  
Vector Representations

How is semantic information stored in the human mind and brain? Some philosophers and cognitive scientists argue for vectorial representations of concepts, where the meaning of a word is represented as its position in a high-dimensional neural state space. At the intersection of natural language processing and artificial intelligence, a class of very successful distributional word vector models has developed that can account for classic EEG findings of language, that is, the ease versus difficulty of integrating a word with its sentence context. However, models of semantics have to account not only for context-based word processing, but should also describe how word meaning is represented. Here, we investigate whether distributional vector representations of word meaning can model brain activity induced by words presented without context. Using EEG activity (event-related brain potentials) collected while participants in two experiments (English and German) read isolated words, we encoded and decoded word vectors taken from the family of prediction-based Word2vec algorithms. We found that, first, the position of a word in vector space allows the prediction of the pattern of corresponding neural activity over time, in particular during a time window of 300 to 500 ms after word onset. Second, distributional models perform better than a human-created taxonomic baseline model (WordNet), and this holds for several distinct vector-based models. Third, multiple latent semantic dimensions of word meaning can be decoded from brain activity. Combined, these results suggest that empiricist, prediction-based vectorial representations of meaning are a viable candidate for the representational architecture of human semantic knowledge.

scholarly journals Semantic and syntactic composition of minimal adjective-noun phrases in Dutch: an MEG study

2020 ◽  
Author(s):  
Arnold Kochari ◽  
Ashley Lewis ◽  
Jan-Mathijs Schoffelen ◽  
Herbert Schriefers
Keyword(s):  
Language Processing ◽  
Word Processing ◽  
Time Window ◽  
Noun Phrases ◽  
Cortical Region ◽  
Composition Effect ◽  
Multiple Regions ◽  
Processing Activity ◽  
Confirmatory Analysis ◽  
The Brain

AbstractThe possibility to combine smaller units of meaning (e.g., words) to create new and more complex meanings (e.g., phrases and sentences) is a fundamental feature of human language. In the present project, we investigated how the brain supports the semantic and syntactic composition of two-word adjective-noun phrases in Dutch, using magnetoencephalography (MEG). The present investigation followed up on previous studies reporting a composition effect in the left anterior temporal lobe (LATL) when comparing neural activity at nouns combined with adjectives, as opposed to nouns in a non-compositional context. The first aim of the present study was to investigate whether this effect, as well as its modulation by noun specificity and adjective class, can also be observed in Dutch. A second aim was to investigate to what extent these effects may be driven by syntactic composition rather than primarily by semantic composition as was previously proposed. To this end, a novel condition was administered in which participants saw nouns combined with pseudowords lacking meaning but agreeing with the nouns in terms of grammatical gender, as real adjectives would. We failed to observe a composition effect or its modulation in both a confirmatory analysis (focused on the cortical region and time-window where it has previously been reported) and in exploratory analyses (where we tested multiple regions and an extended potential time-window of the effect). A syntactically driven composition effect was also not observed in our data. We do, however, successfully observe an independent, previously reported effect on single word processing in our data, confirming that our MEG data processing pipeline does meaningfully capture language processing activity by the brain. The failure to observe the composition effect in LATL is surprising given that it has been previously reported in multiple studies. Reviewing all previous studies investigating this effect, we propose that materials and a task involving imagery might be necessary for this effect to be observed. In addition, we identified substantial variability in the regions of interest analysed in previous studies, which warrants additional checks of robustness of the effect. Further research should identify limits and conditions under which this effect can be observed. The failure to observe specifically a syntactic composition effect in such minimal phrases is less surprising given that it has not been previously reported in MEG data.

scholarly journals Looking for effects of qualia on event-related brain potentials of close others in search for a cause of the similarity of qualia assumed across individuals

F1000Research ◽  
2018 ◽  
Vol 3 ◽  
pp. 316
Author(s):  
Sheila Bouten ◽  
Hugo Pantecouteau ◽  
J. Bruno Debruille
Keyword(s):  
Time Window ◽  
Brain Activity ◽  
Conscious Experience ◽  
Event Related Potentials ◽  
The Other ◽  
Brain Potentials ◽  
Frontal Electrode ◽  
Related Potentials ◽  
The Individual ◽  
The Brain

Qualia, the individual instances of subjective conscious experience, are private events. However, in everyday life, we assume qualia of others and their perceptual worlds, to be similar to ours. One way this similarity is possible is if qualia of others somehow contribute to the production of qualia by our own brain and vice versa. To test this hypothesis, we focused on the mean voltages of event-related potentials (ERPs) in the time-window of the P600 component, whose amplitude correlates positively with conscious awareness. These ERPs were elicited by images of the international affective picture system in 16 pairs of friends, siblings or couples going side by side through hyperscanning without having to interact. Each of the 32 members of these 16 pairs faced one half of the screen and could not see what the other member was presented with on the other half. One stimulus occurred on each half simultaneously. The sameness of these stimulus pairs was manipulated as well as the participants’ belief in that sameness by telling subjects’ pairs that they were going to be presented with the same stimuli in two blocks and with different ones in the two others. ERPs were more positive at all electrode subsets for stimulus pairs that were inconsistent with the belief than for those that were consistent. In the N400 time window, at frontal electrode sites, ERPs were again more positive for inconsistent than for consistent stimuli. As participants had no way to see the stimulus their partner was presented with and thus no way to detect inconsistence, these data might reveal an impact of the qualia of a person on the brain activity of another. Such impact could provide a research avenue when trying to explain the similarity of qualia across individuals.

scholarly journals Ereigniskorrelierte Potenziale (EKPs)

2012 ◽  
Vol 40 (1) ◽  
pp. 68-96
Author(s):  
Heiner Drenhaus ◽  
Peter beim Graben
Keyword(s):  
Language Processing ◽  
Speech Processing ◽  
Brain Activity ◽  
Average Technique ◽  
Laboratory Equipment ◽  
Brain Potentials ◽  
Technical Requirements ◽  
Ereigniskorrelierte Potenziale ◽  
Classical Average ◽  
Processing Steps

AbstractIn this article we give a short introduction to the online method of event-related (brain) potentials (ERPs) and their importance for our understanding of language structure and grammar. This methodology places high demands on (technical) requirements for laboratory equipment as well as on the skills of the investigator. However, the high costs are relatively balanced compared to the advantages of this experimental method. By using ERPs, it becomes possible to monitor the electrophysiological brain activity associated with speech processing in real time (millisecond by millisecond) and to draw conclusions on human language processing and the human parser.First, we present briefly how this method works and how ERPs can be classified (Section 1 and 2). In the following, we show that the ERP method can be used to study the processing of e. g. semantic, pragmatic and syntactic information (Section 3). Crucial for our discussion will be the interpretation of the so-called ERP components and their connection and importance for psycholinguistics and theoretical linguistics. In our presentation, we emphasize, that the electrophysiological brain activity in relation to specific (e. g. linguistic) stimuli can be used to identify distinct processes, which give a deeper insight into the different processing steps of language. At the end of this article (Section 4), we present some results from ERP studies of German negative-polar elements. Additionally, we highlight the advantage and benefits of an alternative method to analyze ERP data compared to the more ‘classical’ average technique.

Dissociation of Brain Activity Related to Syntactic and Semantic Aspects of Language

1993 ◽  
Vol 5 (3) ◽  
pp. 335-344 ◽  
Author(s):  
Thomas F. Münte ◽  
Hans-Jochen Heinze ◽  
George R Mangun
Keyword(s):  
Language Processing ◽  
Early Stage ◽  
Brain Activity ◽  
Brain Potentials ◽  
Semantic Condition ◽  
Distributional Information ◽  
Preceding Word ◽  
N400 Component ◽  
Young Subjects ◽  
Theoretical Issues

In psycholinguistic research, there has been considerable interest in understanding the interactions of difFerent types of linguistic information during language processing. For example, does syntactic information interact with semantic or pragmatic information at an early stage of language processing, or only at later stages in order to resolve ambiguities of language? Developing reliable measures of language processes such as syntax and semantics is important to address many of these theoretical issues in psycholinguistics. In the present study, event-related brain potentials (ERPs) were recorded from healthy young subjects while they read pairs of words presented one word at a time. The ERPs for the second word of each pair were compared as a function of whether the preceding word was or was not (1) semantically related (i.e., synonyms; “semantic condition”) or (2) grammatically correct (“syntactic condition”). In the semantic condition the ERPs obtained to words preceded by nonsemantically related words elicited an N400 component that was maximal over centroparietal scalp regions. In contrast, in the syntactic condition the ERPs obtained to words preceded by grammatically incorrect articles or pronouns yielded a negativity with a later onset, and a frontopolar, left hemisphere scalp maximum. This replicates our previous findings of a syntactic negativity in a word pair design that was performed in the German language. Further, the present data provide scalp distributional information, which suggests that the syntactic negativity represents brain processes that are dissociable from the centroparietal N400 component. Thus, these findings provide strong evidence for a separate negative polarity ERP component that indexes syntactic aspects of language processing.

scholarly journals Predicting hand movements with distributional semantics: evidence from mouse-tracking

2021 ◽  
Author(s):  
Daniele Gatti ◽  
Marco Marelli ◽  
Luca Rinaldi
Keyword(s):  
Decision Making ◽  
Time Window ◽  
Word Meaning ◽  
Semantic Relatedness ◽  
Semantic Knowledge ◽  
Response Conflict ◽  
Hand Movements ◽  
Distributional Semantics ◽  
Mouse Tracking ◽  
Decision Making Processes

Despite mouse-tracking has been taken as a real-time window on different aspects of human decision-making processes, whether purely semantic information affects response conflict at the level of motor output is still unknown. Here, we investigated the possible effects of semantic knowledge on hand movements by predicting participants’ performance in a mouse-tracking task through distributional semantics, a usage-based modelling approach to meaning. Participants were shown word pairs and were required to perform a two-alternative forced choice task by moving the computer mouse and selecting either the more abstract or the more concrete word, depending on the experimental condition. Results showed that mouse trajectories were affected by the semantic relatedness of the word-pairs as indexed by the distributional semantic model, despite this dimension was not relevant for the task at hand. In particular, mouse trajectories reflected the response conflict and its temporal evolution, with larger deviation and movement uncertainty for increasing word semantic relatedness. These findings testify the effects of word meaning on decision-making processes as indexed by a mouse-tracking conflict. They also document a strict interplay between sensory-motor processes and the structure of human semantic memory.

Skipping Breakfast Affects the Early Steps of Cognitive Processing

2019 ◽  
Vol 33 (2) ◽  
pp. 109-118
Author(s):  
Andrés Antonio González-Garrido ◽  
Jacobo José Brofman-Epelbaum ◽  
Fabiola Reveca Gómez-Velázquez ◽  
Sebastián Agustín Balart-Sánchez ◽  
Julieta Ramos-Loyo
Keyword(s):  
Working Memory ◽  
Cognitive Processing ◽  
Task Difficulty ◽  
Brain Activity ◽  
Reaction Times ◽  
Brain Potentials ◽  
Brain Functioning ◽  
Attentional Processes ◽  
Electrical Brain Activity ◽  
Skipping Breakfast

Abstract. It has been generally accepted that skipping breakfast adversely affects cognition, mainly disturbing the attentional processes. However, the effects of short-term fasting upon brain functioning are still unclear. We aimed to evaluate the effect of skipping breakfast on cognitive processing by studying the electrical brain activity of young healthy individuals while performing several working memory tasks. Accordingly, the behavioral results and event-related brain potentials (ERPs) of 20 healthy university students (10 males) were obtained and compared through analysis of variances (ANOVAs), during the performance of three n-back working memory (WM) tasks in two morning sessions on both normal (after breakfast) and 12-hour fasting conditions. Significantly fewer correct responses were achieved during fasting, mainly affecting the higher WM load task. In addition, there were prolonged reaction times with increased task difficulty, regardless of breakfast intake. ERP showed a significant voltage decrement for N200 and P300 during fasting, while the amplitude of P200 notably increased. The results suggest skipping breakfast disturbs earlier cognitive processing steps, particularly attention allocation, early decoding in working memory, and stimulus evaluation, and this effect increases with task difficulty.

Mol2vec: Unsupervised Machine Learning Approach with Chemical Intuition

2017 ◽  
Author(s):  
Sabrina Jaeger ◽  
Simone Fulle ◽  
Samo Turk
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Supervised Machine Learning ◽  
Learning Approach ◽  
Learning Approaches ◽  
Unsupervised Machine Learning ◽  
Feature Representations ◽  
Machine Learning Approach ◽  
The Individual ◽  
Vector Representations

Inspired by natural language processing techniques we here introduce Mol2vec which is an unsupervised machine learning approach to learn vector representations of molecular substructures. Similarly, to the Word2vec models where vectors of closely related words are in close proximity in the vector space, Mol2vec learns vector representations of molecular substructures that are pointing in similar directions for chemically related substructures. Compounds can finally be encoded as vectors by summing up vectors of the individual substructures and, for instance, feed into supervised machine learning approaches to predict compound properties. The underlying substructure vector embeddings are obtained by training an unsupervised machine learning approach on a so-called corpus of compounds that consists of all available chemical matter. The resulting Mol2vec model is pre-trained once, yields dense vector representations and overcomes drawbacks of common compound feature representations such as sparseness and bit collisions. The prediction capabilities are demonstrated on several compound property and bioactivity data sets and compared with results obtained for Morgan fingerprints as reference compound representation. Mol2vec can be easily combined with ProtVec, which employs the same Word2vec concept on protein sequences, resulting in a proteochemometric approach that is alignment independent and can be thus also easily used for proteins with low sequence similarities.

The Opposition of Surprisal and Semantic Similarity in the Prediction of Language Processing: Evidence from Eye-tracking Data

2020 ◽  
Author(s):  
Kun Sun
Keyword(s):  
Eye Tracking ◽  
Semantic Similarity ◽  
Cognitive Processing ◽  
Language Processing ◽  
Language Comprehension ◽  
Word Processing ◽  
Reading Time ◽  
Computational Models ◽  
Tracking Data ◽  
Dynamic Approach

Expectations or predictions about upcoming content play an important role during language comprehension and processing. One important aspect of recent studies of language comprehension and processing concerns the estimation of the upcoming words in a sentence or discourse. Many studies have used eye-tracking data to explore computational and cognitive models for contextual word predictions and word processing. Eye-tracking data has previously been widely explored with a view to investigating the factors that influence word prediction. However, these studies are problematic on several levels, including the stimuli, corpora, statistical tools they applied. Although various computational models have been proposed for simulating contextual word predictions, past studies usually preferred to use a single computational model. The disadvantage of this is that it often cannot give an adequate account of cognitive processing in language comprehension. To avoid these problems, this study draws upon a massive natural and coherent discourse as stimuli in collecting the data on reading time. This study trains two state-of-art computational models (surprisal and semantic (dis)similarity from word vectors by linear discriminative learning (LDL)), measuring knowledge of both the syntagmatic and paradigmatic structure of language. We develop a `dynamic approach' to compute semantic (dis)similarity. It is the first time that these two computational models have been merged. Models are evaluated using advanced statistical methods. Meanwhile, in order to test the efficiency of our approach, one recently developed cosine method of computing semantic (dis)similarity based on word vectors data adopted is used to compare with our `dynamic' approach. The two computational and fixed-effect statistical models can be used to cross-verify the findings, thus ensuring that the result is reliable. All results support that surprisal and semantic similarity are opposed in the prediction of the reading time of words although both can make good predictions. Additionally, our `dynamic' approach performs better than the popular cosine method. The findings of this study are therefore of significance with regard to acquiring a better understanding how humans process words in a real-world context and how they make predictions in language cognition and processing.

scholarly journals A starring role for inference in the neurocognition of visual narratives

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Neil Cohn
Keyword(s):  
Language Processing ◽  
Mental Model ◽  
Brain Potentials ◽  
Visual Narratives ◽  
Inferential Processing

AbstractResearch in verbal and visual narratives has often emphasized backward-looking inferences, where absent information is subsequently inferred. However, comics use conventions like star-shaped “action stars” where a reader knows events are undepicted at that moment, rather than omitted entirely. We contrasted the event-related brain potentials (ERPs) to visual narratives depicting an explicit event, an action star, or a “noise” panel of scrambled lines. Both action stars and noise panels evoked large N400s compared to explicit-events (300–500 ms), but action stars and noise panels then differed in their later effects (500–900 ms). Action stars elicited sustained negativities and P600s, which could indicate further interpretive processes and integration of meaning into a mental model, while noise panels evoked late frontal positivities possibly indexing that they were improbable narrative units. Nevertheless, panels following action stars and noise panels both evoked late sustained negativities, implying further inferential processing. Inference in visual narratives thus uses cascading mechanisms resembling those in language processing that differ based on the inferential techniques.

Sign in / Sign up

Export Citation Format

Share Document