scholarly journals Incremental Sense Weight Training for In-Depth Interpretation of Contextualized Word Embeddings (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13823-13824
Author(s):  
Xinyi Jiang ◽  
Zhengzhe Yang ◽  
Jinho D. Choi
Keyword(s):  
Online Algorithm ◽  
Weight Training ◽  
Word Sense Disambiguation ◽  
Word Sense ◽  
Word Embeddings ◽  
Sense Disambiguation ◽  
The One

We present a novel online algorithm that learns the essence of each dimension in word embeddings. We first mask dimensions determined unessential by our algorithm, apply the masked word embeddings to a word sense disambiguation task (WSD), and compare its performance against the one achieved by the original embeddings. Our results show that the masked word embeddings do not hurt the performance and can improve it by 3%.

scholarly journals AutoExtend: Combining Word Embeddings with Semantic Resources

2017 ◽  
Vol 43 (3) ◽  
pp. 593-617 ◽  
Author(s):  
Sascha Rothe ◽  
Hinrich Schütze
Keyword(s):  
Semantic Information ◽  
State Of The Art ◽  
Word Sense Disambiguation ◽  
Input Word ◽  
Word Sense ◽  
Word Embeddings ◽  
Training Corpus ◽  
Context Similarity ◽  
Sense Disambiguation ◽  
Semantic Resources

We present AutoExtend, a system that combines word embeddings with semantic resources by learning embeddings for non-word objects like synsets and entities and learning word embeddings that incorporate the semantic information from the resource. The method is based on encoding and decoding the word embeddings and is flexible in that it can take any word embeddings as input and does not need an additional training corpus. The obtained embeddings live in the same vector space as the input word embeddings. A sparse tensor formalization guarantees efficiency and parallelizability. We use WordNet, GermaNet, and Freebase as semantic resources. AutoExtend achieves state-of-the-art performance on Word-in-Context Similarity and Word Sense Disambiguation tasks.

scholarly journals Domain Adaptation using Word Embeddings for Word Sense Disambiguation

2018 ◽  
Vol 25 (4) ◽  
pp. 463-480
Author(s):  
Kanako Komiya ◽  
Minoru Sasaki ◽  
Hiroyuki Shinnou ◽  
Manabu Okumura
Keyword(s):  
Domain Adaptation ◽  
Word Sense Disambiguation ◽  
Word Sense ◽  
Word Embeddings ◽  
Sense Disambiguation

scholarly journals Supervised Learning and Knowledge-Based Approaches Applied to Biomedical Word Sense Disambiguation

2017 ◽  
Vol 14 (4) ◽  
Author(s):  
Rui Antunes ◽  
Sérgio Matos
Keyword(s):  
Word Sense Disambiguation ◽  
Word Embedding ◽  
Biomedical Text Mining ◽  
Bag Of Words ◽  
Word Sense ◽  
Word Embeddings ◽  
Global Features ◽  
Knowledge Based ◽  
Sense Disambiguation ◽  
Averaging Functions

AbstractWord sense disambiguation (WSD) is an important step in biomedical text mining, which is responsible for assigning an unequivocal concept to an ambiguous term, improving the accuracy of biomedical information extraction systems. In this work we followed supervised and knowledge-based disambiguation approaches, with the best results obtained by supervised means. In the supervised method we used bag-of-words as local features, and word embeddings as global features. In the knowledge-based method we combined word embeddings, concept textual definitions extracted from the UMLS database, and concept association values calculated from the MeSH co-occurrence counts from MEDLINE articles. Also, in the knowledge-based method, we tested different word embedding averaging functions to calculate the surrounding context vectors, with the goal to give more importance to closest words of the ambiguous term. The MSH WSD dataset, the most common dataset used for evaluating biomedical concept disambiguation, was used to evaluate our methods. We obtained a top accuracy of 95.6 % by supervised means, while the best knowledge-based accuracy was 87.4 %. Our results show that word embedding models improved the disambiguation accuracy, proving to be a powerful resource in the WSD task.

scholarly journals Cognitive Metaphor and Discourse: Research Methods and Paradigms

2022 ◽  
pp. 108-121
Author(s):  
Oleg Kalinin
Keyword(s):  
Word Sense Disambiguation ◽  
The Other ◽  
Computer Assisted ◽  
Word Sense ◽  
Research Papers ◽  
Cognitive Metaphor ◽  
Sense Disambiguation ◽  
Points Of View ◽  
Basic Approaches ◽  
The One

The article dwells on a modern cognitive and discourse study of metaphors. Taking the advantage of the analysis and fusion of information in foreign and domestic papers, the researcher delves into their classification from the ontological, axiological and epistemological points of view. The ontological level breaks down into two basic approaches, namely metaphorical nature of discourse and discursive nature of metaphors. The former analyses metaphors to fathom characteristics of discourse, while the other provides for the study of metaphorical features in the context of discursive communication. The axiological aspect covers critical and descriptive studies and the epistemological angle comprises quantitive and qualitative methods in metaphorical studies. Other issues covered in the paper incorporate a thorough review of methods for identification of metaphors to include computer-assisted solutions (Word Sense Disambiguation, Categorisation, Metaphor Clusters) and numerical analysis of the metaphorical nature of discourse – descriptor analysis, metaphor power index, cluster analysis, and complex metaphor power analysis. On the one hand, the conceptualization of research papers boils down to major features of the discursive approach to metaphors and on the other, multiple studies of metaphors in the context of discourse pave the way for a discursive trend in cognitive metaphorology.

Sign in / Sign up

Export Citation Format

Share Document