scholarly journals Mapping Distributional Semantics to Property Norms with Deep Neural Networks

2019 ◽  
Vol 3 (2) ◽  
pp. 30 ◽  
Author(s):  
Dandan Li ◽  
Douglas Summers-Stay
Keyword(s):  
Language Processing ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Feature Space ◽  
Semantic Space ◽  
Distributional Semantics ◽  
Word Embeddings ◽  
Different Types ◽  
Property Space ◽  
Psychology Literature

Word embeddings have been very successful in many natural language processing tasks, but they characterize the meaning of a word/concept by uninterpretable “context signatures”. Such a representation can render results obtained using embeddings difficult to interpret. Neighboring word vectors may have similar meanings, but in what way are they similar? That similarity may represent a synonymy, metonymy, or even antonymy relation. In the cognitive psychology literature, in contrast, concepts are frequently represented by their relations with properties. These properties are produced by test subjects when asked to describe important features of concepts. As such, they form a natural, intuitive feature space. In this work, we present a neural-network-based method for mapping a distributional semantic space onto a human-built property space automatically. We evaluate our method on word embeddings learned with different types of contexts, and report state-of-the-art performances on the widely used McRae semantic feature production norms.

scholarly journals Word-embeddings Italian semantic spaces: A semantic model for psycholinguistic research

Psihologija ◽  
2017 ◽  
Vol 50 (4) ◽  
pp. 503-520 ◽  
Author(s):  
Marco Marelli
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Semantic Space ◽  
Distributional Semantics ◽  
Semantic Model ◽  
Word Embeddings ◽  
Psycholinguistic Research ◽  
Graphic Interface ◽  
Test Sets ◽  
Semantic Spaces

Distributional semantics has been for long a source of successful models in psycholinguistics, permitting to obtain semantic estimates for a large number of words in an automatic and fast way. However, resources in this respect remain scarce or limitedly accessible for languages different from English. The present paper describes WEISS (Word-Embeddings Italian Semantic Space), a distributional semantic model based on Italian. WEISS includes models of semantic representations that are trained adopting state-of-the-art word-embeddings methods, applying neural networks to induce distributed representations for lexical meanings. The resource is evaluated against two test sets, demonstrating that WEISS obtains a better performance with respect to a baseline encoding word associations. Moreover, an extensive qualitative analysis of the WEISS output provides examples of the model potentialities in capturing several semantic phenomena. Two variants of WEISS are released and made easily accessible via web through the SNAUT graphic interface.

Learning emotional word embeddings for sentiment analysis

2021 ◽  
pp. 1-13
Author(s):  
Qingtian Zeng ◽  
Xishi Zhao ◽  
Xiaohui Hu ◽  
Hua Duan ◽  
Zhongying Zhao ◽  
...  
Keyword(s):  
Sentiment Analysis ◽  
Language Processing ◽  
State Of The Art ◽  
Research Problem ◽  
Emotional Word ◽  
Classification Model ◽  
Data Sets ◽  
Word Embeddings ◽  
Real World Data ◽  
Text Documents

Word embeddings have been successfully applied in many natural language processing tasks due to its their effectiveness. However, the state-of-the-art algorithms for learning word representations from large amounts of text documents ignore emotional information, which is a significant research problem that must be addressed. To solve the above problem, we propose an emotional word embedding (EWE) model for sentiment analysis in this paper. This method first applies pre-trained word vectors to represent document features using two different linear weighting methods. Then, the resulting document vectors are input to a classification model and used to train a text sentiment classifier, which is based on a neural network. In this way, the emotional polarity of the text is propagated into the word vectors. The experimental results on three kinds of real-world data sets demonstrate that the proposed EWE model achieves superior performances on text sentiment prediction, text similarity calculation, and word emotional expression tasks compared to other state-of-the-art models.

scholarly journals Word Embeddings as Metric Recovery in Semantic Spaces

2016 ◽  
Vol 4 ◽  
pp. 273-286 ◽  
Author(s):  
Tatsunori B. Hashimoto ◽  
David Alvarez-Melis ◽  
Tommi S. Jaakkola
Keyword(s):  
Random Walks ◽  
Manifold Learning ◽  
State Of The Art ◽  
Inductive Reasoning ◽  
Semantic Space ◽  
Word Embedding ◽  
Word Embeddings ◽  
Recovery Algorithm ◽  
Series Completion ◽  
Semantic Spaces

Continuous word representations have been remarkably useful across NLP tasks but remain poorly understood. We ground word embeddings in semantic spaces studied in the cognitive-psychometric literature, taking these spaces as the primary objects to recover. To this end, we relate log co-occurrences of words in large corpora to semantic similarity assessments and show that co-occurrences are indeed consistent with an Euclidean semantic space hypothesis. Framing word embedding as metric recovery of a semantic space unifies existing word embedding algorithms, ties them to manifold learning, and demonstrates that existing algorithms are consistent metric recovery methods given co-occurrence counts from random walks. Furthermore, we propose a simple, principled, direct metric recovery algorithm that performs on par with the state-of-the-art word embedding and manifold learning methods. Finally, we complement recent focus on analogies by constructing two new inductive reasoning datasets—series completion and classification—and demonstrate that word embeddings can be used to solve them as well.

scholarly journals Title-Guided Encoding for Keyphrase Generation

2019 ◽  
Vol 33 ◽  
pp. 6268-6275 ◽  
Author(s):  
Wang Chen ◽  
Yifan Gao ◽  
Jiani Zhang ◽  
Irwin King ◽  
Michael R. Lyu
Keyword(s):  
Language Processing ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Relevant Information ◽  
The State ◽  
Main Body ◽  
Generation Task ◽  
Leading Role ◽  
Very High

Keyphrase generation (KG) aims to generate a set of keyphrases given a document, which is a fundamental task in natural language processing (NLP). Most previous methods solve this problem in an extractive manner, while recently, several attempts are made under the generative setting using deep neural networks. However, the state-of-the-art generative methods simply treat the document title and the document main body equally, ignoring the leading role of the title to the overall document. To solve this problem, we introduce a new model called Title-Guided Network (TG-Net) for automatic keyphrase generation task based on the encoderdecoder architecture with two new features: (i) the title is additionally employed as a query-like input, and (ii) a titleguided encoder gathers the relevant information from the title to each word in the document. Experiments on a range of KG datasets demonstrate that our model outperforms the state-of-the-art models with a large margin, especially for documents with either very low or very high title length ratios.

scholarly journals The Image Annotation Refinement in Embedding Feature Space based on Mutual Information

2022 ◽  
Vol 16 ◽  
pp. 191-201
Author(s):  
Wei Li ◽  
Haiyu Song ◽  
Hongda Zhang ◽  
Houjie Li ◽  
Pengjie Wang
Keyword(s):  
Mutual Information ◽  
Image Annotation ◽  
State Of The Art ◽  
Feature Space ◽  
Semantic Space ◽  
Visual Features ◽  
Novel Approach ◽  
Proposed Model ◽  
Information Method ◽  
Available Information

The ever-increasing size of images has made automatic image annotation one of the most important tasks in the fields of machine learning and computer vision. Despite continuous efforts in inventing new annotation algorithms and new models, results of the state-of-the-art image annotation methods are often unsatisfactory. In this paper, to further improve annotation refinement performance, a novel approach based on weighted mutual information to automatically refine the original annotations of images is proposed. Unlike the traditional refinement model using only visual feature, the proposed model use semantic embedding to properly map labels and visual features to a meaningful semantic space. To accurately measure the relevance between the particular image and its original annotations, the proposed model utilize all available information including image-to-image, label-to-label and image-to-label. Experimental results conducted on three typical datasets show not only the validity of the refinement, but also the superiority of the proposed algorithm over existing ones. The improvement largely benefits from our proposed mutual information method and utilizing all available information.

scholarly journals Measuring the Evolution of a Scientific Field through Citation Frames

2018 ◽  
Vol 6 ◽  
pp. 391-406 ◽  
Author(s):  
David Jurgens ◽  
Srijan Kumar ◽  
Raine Hoover ◽  
Dan McFarland ◽  
Dan Jurafsky
Keyword(s):  
Natural Language Processing ◽  
Citation Analysis ◽  
Language Processing ◽  
State Of The Art ◽  
Citation Count ◽  
Scientific Field ◽  
Discourse Structure ◽  
Small Scale ◽  
Behavioral Study ◽  
Different Types

Citations have long been used to characterize the state of a scientific field and to identify influential works. However, writers use citations for different purposes, and this varied purpose influences uptake by future scholars. Unfortunately, our understanding of how scholars use and frame citations has been limited to small-scale manual citation analysis of individual papers. We perform the largest behavioral study of citations to date, analyzing how scientific works frame their contributions through different types of citations and how this framing affects the field as a whole. We introduce a new dataset of nearly 2,000 citations annotated for their function, and use it to develop a state-of-the-art classifier and label the papers of an entire field: Natural Language Processing. We then show how differences in framing affect scientific uptake and reveal the evolution of the publication venues and the field as a whole. We demonstrate that authors are sensitive to discourse structure and publication venue when citing, and that how a paper frames its work through citations is predictive of the citation count it will receive. Finally, we use changes in citation framing to show that the field of NLP is undergoing a significant increase in consensus.

scholarly journals VivesDebate: A New Annotated Multilingual Corpus of Argumentation in a Debate Tournament

2021 ◽  
Vol 11 (15) ◽  
pp. 7160
Author(s):  
Ramon Ruiz-Dolz ◽  
Montserrat Nofre ◽  
Mariona Taulé ◽  
Stella Heras ◽  
Ana García-Fornes
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Argument Analysis ◽  
Argument Evaluation ◽  
Learning Techniques ◽  
Multilingual Corpus ◽  
Computational Argumentation

The application of the latest Natural Language Processing breakthroughs in computational argumentation has shown promising results, which have raised the interest in this area of research. However, the available corpora with argumentative annotations are often limited to a very specific purpose or are not of adequate size to take advantage of state-of-the-art deep learning techniques (e.g., deep neural networks). In this paper, we present VivesDebate, a large, richly annotated and versatile professional debate corpus for computational argumentation research. The corpus has been created from 29 transcripts of a debate tournament in Catalan and has been machine-translated into Spanish and English. The annotation contains argumentative propositions, argumentative relations, debate interactions and professional evaluations of the arguments and argumentation. The presented corpus can be useful for research on a heterogeneous set of computational argumentation underlying tasks such as Argument Mining, Argument Analysis, Argument Evaluation or Argument Generation, among others. All this makes VivesDebate a valuable resource for computational argumentation research within the context of massive corpora aimed at Natural Language Processing tasks.

scholarly journals A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations

2020 ◽  
Vol 34 (05) ◽  
pp. 9434-9441
Author(s):  
Zekun Yang ◽  
Juan Feng
Keyword(s):  
Gender Bias ◽  
Language Processing ◽  
State Of The Art ◽  
Word Embedding ◽  
Coreference Resolution ◽  
Word Embeddings ◽  
Inference Method ◽  
Sentence Level ◽  
Statistical Dependency ◽  
And Gender

Word embedding has become essential for natural language processing as it boosts empirical performances of various tasks. However, recent research discovers that gender bias is incorporated in neural word embeddings, and downstream tasks that rely on these biased word vectors also produce gender-biased results. While some word-embedding gender-debiasing methods have been developed, these methods mainly focus on reducing gender bias associated with gender direction and fail to reduce the gender bias presented in word embedding relations. In this paper, we design a causal and simple approach for mitigating gender bias in word vector relation by utilizing the statistical dependency between gender-definition word embeddings and gender-biased word embeddings. Our method attains state-of-the-art results on gender-debiasing tasks, lexical- and sentence-level evaluation tasks, and downstream coreference resolution tasks.

Counterfactual Autoencoder for Unsupervised Semantic Learning

2018 ◽  
Vol 9 (4) ◽  
pp. 1-20 ◽  
Author(s):  
Saad Sadiq ◽  
Mei-Ling Shyu ◽  
Daniel J. Feaster
Keyword(s):  
Language Processing ◽  
Speech Processing ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Potential Outcomes ◽  
Model Data ◽  
Semantic Learning ◽  
Novel Approach ◽  
Training Samples ◽  
Latent Space

Deep Neural Networks (DNNs) are best known for being the state-of-the-art in artificial intelligence (AI) applications including natural language processing (NLP), speech processing, computer vision, etc. In spite of all recent achievements of deep learning, it has yet to achieve semantic learning required to reason about the data. This lack of reasoning is partially imputed to the boorish memorization of patterns and curves from millions of training samples and ignoring the spatiotemporal relationships. The proposed framework puts forward a novel approach based on variational autoencoders (VAEs) by using the potential outcomes model and developing the counterfactual autoencoders. The proposed framework transforms any sort of multimedia input distributions to a meaningful latent space while giving more control over how the latent space is created. This allows us to model data that is better suited to answer inference-based queries, which is very valuable in reasoning-based AI applications.

scholarly journals Deep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts

2017 ◽  
Vol 24 (4) ◽  
pp. 813-821 ◽  
Author(s):  
Anne Cocos ◽  
Alexander G Fiks ◽  
Aaron J Masino
Keyword(s):  
Neural Network ◽  
Social Media ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
State Of The Art ◽  
Detection Performance ◽  
Word Embeddings

Abstract Objective Social media is an important pharmacovigilance data source for adverse drug reaction (ADR) identification. Human review of social media data is infeasible due to data quantity, thus natural language processing techniques are necessary. Social media includes informal vocabulary and irregular grammar, which challenge natural language processing methods. Our objective is to develop a scalable, deep-learning approach that exceeds state-of-the-art ADR detection performance in social media. Materials and Methods We developed a recurrent neural network (RNN) model that labels words in an input sequence with ADR membership tags. The only input features are word-embedding vectors, which can be formed through task-independent pretraining or during ADR detection training. Results Our best-performing RNN model used pretrained word embeddings created from a large, non–domain-specific Twitter dataset. It achieved an approximate match F-measure of 0.755 for ADR identification on the dataset, compared to 0.631 for a baseline lexicon system and 0.65 for the state-of-the-art conditional random field model. Feature analysis indicated that semantic information in pretrained word embeddings boosted sensitivity and, combined with contextual awareness captured in the RNN, precision. Discussion Our model required no task-specific feature engineering, suggesting generalizability to additional sequence-labeling tasks. Learning curve analysis showed that our model reached optimal performance with fewer training examples than the other models. Conclusions ADR detection performance in social media is significantly improved by using a contextually aware model and word embeddings formed from large, unlabeled datasets. The approach reduces manual data-labeling requirements and is scalable to large social media datasets.

Sign in / Sign up

Export Citation Format

Share Document