scholarly journals Context-Based Facilitation of Semantic Access Follows Both Logarithmic and Linear Functions of Stimulus Probability

2021 ◽  
Author(s):  
Jakub M. Szewczyk ◽  
Kara D. Federmeier
Keyword(s):  
Language Model ◽  
Linear Functions ◽  
Semantic Features ◽  
Composite Function ◽  
Related Information ◽  
Semantic Access ◽  
Preceding Context ◽  
Incoming Stimulus ◽  
Word Predictability ◽  
Processing Effort

Stimuli are easier to process when the preceding context (e.g., a sentence, in the case of a word) makes them predictable. However, it remains unclear whether context-based facilitation arises due to predictive preactivation of a limited set of relatively probable upcoming stimuli (with facilitation then linearly related to probability) or, instead, arises because the system maintains and updates a probability distribution across all items, as posited by accounts (e.g., surprisal theory) assuming a logarithmic function between predictability and processing effort. To adjudicate between these accounts, we measured the N400 component, an index of semantic access, evoked by sentence-final words of varying probability, including unpredictable words, which are never generated in human production norms. Word predictability was measured using both cloze probabilities and a state-of-the-art machine learning language model (GPT-2). We reanalyzed five datasets (n=138) to first demonstrate and then replicate that context-based facilitation on the N400 is graded and dissociates even among words with cloze probabilities at or near 0, as a function of very small differences in model-estimated predictability. Furthermore, we established that the relationship between word predictability and context-based facilitation on the N400 is neither purely linear nor purely logarithmic but instead combines both functions. We argue that such a composite function reveals properties of the mapping between words and semantic features and how feature- and word- related information is activated during on-line processing. Overall, the results provide powerful evidence for the role of internal models in shaping how the brain apprehends incoming stimulus information.

scholarly journals MRE: A Military Relation Extraction Model Based on BiGRU and Multi-Head Attention

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1742
Author(s):  
Yiwei Lu ◽  
Ruopeng Yang ◽  
Xuping Jiang ◽  
Dan Zhou ◽  
Changshen Yin ◽  
...  
Keyword(s):  
Comprehensive Evaluation ◽  
Language Model ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Semantic Features ◽  
Chinese Word Segmentation ◽  
Conceptual Foundation ◽  
Attention Model ◽  
Operational Information ◽  
Military Relations

A great deal of operational information exists in the form of text. Therefore, extracting operational information from unstructured military text is of great significance for assisting command decision making and operations. Military relation extraction is one of the main tasks of military information extraction, which aims at identifying the relation between two named entities from unstructured military texts. However, the traditional methods of extracting military relations cannot easily resolve problems such as inadequate manual features and inaccurate Chinese word segmentation in military fields, failing to make full use of symmetrical entity relations in military texts. With our approach, based on the pre-trained language model, we present a Chinese military relation extraction method, which combines the bi-directional gate recurrent unit (BiGRU) and multi-head attention mechanism (MHATT). More specifically, the conceptual foundation of our method lies in constructing an embedding layer and combining word embedding with position embedding, based on the pre-trained language model; the output vectors of BiGRU neural networks are symmetrically spliced to learn the semantic features of context, and they fuse the multi-head attention mechanism to improve the ability of expressing semantic information. On the military text corpus that we have built, we conduct extensive experiments. We demonstrate the superiority of our method over the traditional non-attention model, attention model, and improved attention model, and the comprehensive evaluation value F1-score of the model is improved by about 4%.

Postmodifying verb-second clauses in spoken German

2012 ◽  
Vol 19 (2) ◽  
pp. 235-264
Author(s):  
Regina Weinert
Keyword(s):  
Relative Clauses ◽  
Language Model ◽  
Noun Phrases ◽  
Corpus Analysis ◽  
Direct Result ◽  
Semantic Features ◽  
Verb Second ◽  
Discourse Function ◽  
Form Part ◽  
Clause Combining

This usage-based and corpus-based study examines the use of verb-second clauses as restrictive postmodifiers of noun phrases in spoken German (ich kenn leute die haben immer pech ‘I know people they are always unlucky’) in relation to verb-final relative clauses. Previous accounts largely work with de-contextualised and constructed data and stop short of accounting for the discourse function of verb-second postmodifying structures. The ratio of verb-final relative clauses to postmodifying verb-second clauses does not indicate a shift towards main clause syntax. Rather, the verb-second clauses form part of a set of existential or presentational and ascriptive copular constructions which serve to highlight properties of entities and/or introduce discourse topics. Relative clauses can be used for such functions, but this is not as common. The syntactic and semantic features associated with postmodifying verb-second clauses can be seen as a direct result of their discourse function, which only a corpus analysis could reveal. The paper also comments on the wider related aspects of verb position, clause combining and pronoun use in spoken German from the perspective of a usage-based language model.

scholarly journals Bert-Enhanced Text Graph Neural Network for Classification

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1536
Author(s):  
Yiping Yang ◽  
Xiaohui Cui
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Structural Information ◽  
Text Processing ◽  
Language Model ◽  
Research Direction ◽  
Semantic Features ◽  
Structure Information ◽  
Single Text ◽  
Graph Neural Networks

Text classification is a fundamental research direction, aims to assign tags to text units. Recently, graph neural networks (GNN) have exhibited some excellent properties in textual information processing. Furthermore, the pre-trained language model also realized promising effects in many tasks. However, many text processing methods cannot model a single text unit’s structure or ignore the semantic features. To solve these problems and comprehensively utilize the text’s structure information and semantic information, we propose a Bert-Enhanced text Graph Neural Network model (BEGNN). For each text, we construct a text graph separately according to the co-occurrence relationship of words and use GNN to extract text features. Moreover, we employ Bert to extract semantic features. The former part can take into account the structural information, and the latter can focus on modeling the semantic information. Finally, we interact and aggregate these two features of different granularity to get a more effective representation. Experiments on standard datasets demonstrate the effectiveness of BEGNN.

Absence of Low-Level Visual Difference Between Canonical and Noncanonical Finger-Numeral Configurations

2010 ◽  
Vol 57 (3) ◽  
pp. 202-207 ◽  
Author(s):  
Samuel Di Luca ◽  
Mauro Pesenti
Keyword(s):  
Processing Speed ◽  
Visual Detection ◽  
Semantic Features ◽  
Low Level ◽  
Perceptual Features ◽  
Perceptual Differences ◽  
Semantic Access ◽  
High Level ◽  
Visual Difference

Canonical finger numeral configurations are named faster than less familiar finger configurations and activate a semantic place-coding representation as symbolic stimuli. However, this does not exclude categorically the possibility that mere visuo-perceptual differences between canonical and noncanonical finger configurations may induce differences in processing speed. This study capitalizes on the fact that, in typical visual-detection tasks, participants focus on low-level visuo-perceptual features to detect a target among distractors sharing the same high-level semantic features, producing the so-called pop-out effect. Participants had to decide whether a canonical finger configuration was present among a set of distractors expressing the same numerosity in a noncanonical way. The results showed that the time needed to detect the presence of the target grew linearly with the number of distractors. This indicates that the canonical target enjoyed no perceptual saliency among the noncanonical configurations (i.e., no pop-out effect) excluding visuo-perceptual differences as the source of the better identification of and semantic access of canonical configurations.

scholarly journals Research on Automatic Question Answering of Generative Knowledge Graph Based on Pointer Network

Information ◽  
2021 ◽  
Vol 12 (3) ◽  
pp. 136
Author(s):  
Shuang Liu ◽  
Nannan Tan ◽  
Yaqian Ge ◽  
Niko Lukač
Keyword(s):  
Knowledge Base ◽  
Word Frequency ◽  
Language Processing ◽  
Question Answering ◽  
Language Model ◽  
Word List ◽  
Superior Performance ◽  
Knowledge Graph ◽  
Semantic Features ◽  
Question Answering Systems

Question-answering systems based on knowledge graphs are extremely challenging tasks in the field of natural language processing. Most of the existing Chinese Knowledge Base Question Answering(KBQA) can only return the knowledge stored in the knowledge base by extractive methods. Nevertheless, this processing does not conform to the reading habits and cannot solve the Out-of-vocabulary(OOV) problem. In this paper, a new generative question answering method based on knowledge graph is proposed, including three parts of knowledge vocabulary construction, data pre-processing, and answer generation. In the word list construction, BiLSTM-CRF is used to identify the entity in the source text, finding the triples contained in the entity, counting the word frequency, and constructing it. In the part of data pre-processing, a pre-trained language model BERT combining word frequency semantic features is adopted to obtain word vectors. In the answer generation part, one combination of a vocabulary constructed by the knowledge graph and a pointer generator network(PGN) is proposed to point to the corresponding entity for generating answer. The experimental results show that the proposed method can achieve superior performance on WebQA datasets than other methods.

scholarly journals Bidirectional Recurrent Neural Network Language Model: Cross Entropy Churn Metrics for Defect Prediction Modelling

2019 ◽  
Vol 8 (6) ◽  
pp. 2792-2800
Keyword(s):  
Neural Network ◽  
Prediction Models ◽  
Dynamic Error ◽  
Language Model ◽  
Source Code ◽  
Cross Entropy ◽  
Machine Language ◽  
Defect Prediction ◽  
Semantic Features

Software Defect Prediction (SDP) plays an active area in many research domain of Software Quality of Assurance (SQA). Many existing research studies are based on software traditional metric sets and defect prediction models are built in machine language to detect the bug for limited source code line. Inspired by the above existing system. In this paper, defect prediction is focused on predicting defects in source code. The aim of this dissertation is to enhance the quality of the software for precise prediction of defects. So, that it helps the developer to find the bug and fix the issue, to make better use of a resource which reduces the test effort, minimize the cost and improve the quality of software. A new approach is introduced to improve the prediction performance of Bidirectional RNNLM in Deep Neural Network. To build the defect prediction model a defect learner framework is proposed and first it need to build a Neural Language Model. Using this Language Model it helps to learn to deep semantic features in source code and it train & test the model. Based on language model it combined with software traditional metric sets to measure the code and find the defect. The probability of language model and metric set Cross-Entropy with Abstract Syntax Tree (CE-AST) metric is used to evaluate the defect proneness and set as a metric label. For classification the metric label K-NN classifier is used. BPTT algorithm for learning RNN will provide additional improvement, it improves the predictions performance to find the dynamic error.

scholarly journals Using Morphological Data in Language Modeling for Serbian Large Vocabulary Speech Recognition

2019 ◽  
Vol 2019 ◽  
pp. 1-8 ◽  
Author(s):  
Edvin Pakoci ◽  
Branislav Popović ◽  
Darko Pekar
Keyword(s):  
Speech Recognition ◽  
Language Model ◽  
Recognition System ◽  
Language Modeling ◽  
Error Rates ◽  
Language Models ◽  
Morphological Data ◽  
Semantic Features ◽  
Automatic Speech Recognition System ◽  
Large Vocabulary

Serbian is in a group of highly inflective and morphologically rich languages that use a lot of different word suffixes to express different grammatical, syntactic, or semantic features. This kind of behaviour usually produces a lot of recognition errors, especially in large vocabulary systems—even when, due to good acoustical matching, the correct lemma is predicted by the automatic speech recognition system, often a wrong word ending occurs, which is nevertheless counted as an error. This effect is larger for contexts not present in the language model training corpus. In this manuscript, an approach which takes into account different morphological categories of words for language modeling is examined, and the benefits in terms of word error rates and perplexities are presented. These categories include word type, word case, grammatical number, and gender, and they were all assigned to words in the system vocabulary, where applicable. These additional word features helped to produce significant improvements in relation to the baseline system, both for n-gram-based and neural network-based language models. The proposed system can help overcome a lot of tedious errors in a large vocabulary system, for example, for dictation, both for Serbian and for other languages with similar characteristics.

Lexical Comprehension in Young Children With Developmental Delays

1994 ◽  
Vol 3 (1) ◽  
pp. 79-88 ◽  
Author(s):  
Theresa A. Kouri
Keyword(s):  
Young Children ◽  
Developmental Delays ◽  
Normal Development ◽  
Clinical Applications ◽  
Group Status ◽  
Novel Object ◽  
Related Information ◽  
Comprehension Skills ◽  
Lexical Items ◽  
Lexical Concepts

Lexical comprehension skills were examined in 20 young children (aged 28–45 months) with developmental delays (DD) and 20 children (aged 19–34 months) with normal development (ND). Each was assigned to either a story-like script condition or a simple ostensive labeling condition in which the names of three novel object and action items were presented over two experimental sessions. During the experimental sessions, receptive knowledge of the lexical items was assessed through a series of target and generalization probes. Results indicated that all children, irrespective of group status, acquired more lexical concepts in the ostensive labeling condition than in the story narrative condition. Overall, both groups acquired more object than action words, although subjects with ND comprehended more action words than subjects with DD. More target than generalization items were also comprehended by both groups. It is concluded that young children’s comprehension of new lexical concepts is facilitated more by a context in which simple ostensive labels accompany the presentation of specific objects and actions than one in which objects and actions are surrounded by thematic and event-related information. Various clinical applications focusing on the lexical training of young children with DD are discussed.

Cognitive Processing of Miscommunication in Interactive Listening: An Evaluation of Listener Indecision and Cognitive Effort

2021 ◽  
pp. 1-17
Author(s):  
Jennifer M. Roche ◽  
Arkady Zgonnikov ◽  
Laura M. Morett
Keyword(s):  
Decision Making ◽  
Cognitive Processing ◽  
Language Processing ◽  
Cognitive Effort ◽  
Mouse Tracking ◽  
Computer Mouse ◽  
Decision Making Processes ◽  
Computer Mouse Tracking ◽  
Listening Task ◽  
Processing Effort

Purpose The purpose of the current study was to evaluate the social and cognitive underpinnings of miscommunication during an interactive listening task. Method An eye and computer mouse–tracking visual-world paradigm was used to investigate how a listener's cognitive effort (local and global) and decision-making processes were affected by a speaker's use of ambiguity that led to a miscommunication. Results Experiments 1 and 2 found that an environmental cue that made a miscommunication more or less salient impacted listener language processing effort (eye-tracking). Experiment 2 also indicated that listeners may develop different processing heuristics dependent upon the speaker's use of ambiguity that led to a miscommunication, exerting a significant impact on cognition and decision making. We also found that perspective-taking effort and decision-making complexity metrics (computer mouse tracking) predict language processing effort, indicating that instances of miscommunication produced cognitive consequences of indecision, thinking, and cognitive pull. Conclusion Together, these results indicate that listeners behave both reciprocally and adaptively when miscommunications occur, but the way they respond is largely dependent upon the type of ambiguity and how often it is produced by the speaker.

Sign in / Sign up

Export Citation Format

Share Document