SE4SA: a deep syntactical contextualized text representation learning approach for sentiment analysis

2021 ◽  
pp. 1-20
Author(s):  
Tham Vo
Keyword(s):  
Sentiment Analysis ◽  
Long Range ◽  
Representation Learning ◽  
Textual Representations ◽  
Embedding Technique ◽  
Analysis Task ◽  
Latent Features ◽  
Benchmark Datasets ◽  
Multi Level ◽  
The Given

Recently, many pre-trained text embedding models have been applied to effectively extract latent features from texts and achieve remarkable performance in various downstream tasks of sentiment analysis domain. However, these pre-trained text embedding models also encounter limitations related to the capability preserving the syntactical structure as well as the global long-range dependent relationships of words. Thus, they might fail to recognize the relevant syntactical features of words as valuable evidences for analyzing sentiment aspects. To overcome these limitations, we proposed a novel deep semantic contextual embedding technique for sentiment analysis, called as: SE4SA. Our proposed SE4SA is a multi-level text embedding model which enables to jointly exploit the long-range syntactical and sequential representations of texts. Then, these achieved rich semantic textual representations can support to have a better understanding on the sentiment aspects of the given text corpus, thereby resulting the better performance on sentiment analysis task. Extensive experiments in several benchmark datasets demonstrate the effectiveness or our proposed SE4SA model in comparing with recent state-of-the-art model.

scholarly journals Attention-based Multi-level Feature Fusion for Named Entity Recognition

2020 ◽  
Author(s):  
Zhiwei Yang ◽  
Hechang Chen ◽  
Jiawei Zhang ◽  
Jing Ma ◽  
Yi Chang
Keyword(s):  
Language Processing ◽  
Feature Fusion ◽  
Named Entity Recognition ◽  
Representation Learning ◽  
Entity Recognition ◽  
Named Entity ◽  
Local Character ◽  
Word Level ◽  
Benchmark Datasets ◽  
Multi Level

Named entity recognition (NER) is a fundamental task in the natural language processing (NLP) area. Recently, representation learning methods (e.g., character embedding and word embedding) have achieved promising recognition results. However, existing models only consider partial features derived from words or characters while failing to integrate semantic and syntactic information (e.g., capitalization, inter-word relations, keywords, lexical phrases, etc.) from multi-level perspectives. Intuitively, multi-level features can be helpful when recognizing named entities from complex sentences. In this study, we propose a novel framework called attention-based multi-level feature fusion (AMFF), which is used to capture the multi-level features from different perspectives to improve NER. Our model consists of four components to respectively capture the local character-level, global character-level, local word-level, and global word-level features, which are then fed into a BiLSTM-CRF network for the final sequence labeling. Extensive experimental results on four benchmark datasets show that our proposed model outperforms a set of state-of-the-art baselines.

Dialogue History Matters! Personalized Response Selection in Multi-Turn Retrieval-Based Chatbots

2021 ◽  
Vol 39 (4) ◽  
pp. 1-25
Author(s):  
Juntao Li ◽  
Chang Liu ◽  
Chongyang Tao ◽  
Zhangming Chan ◽  
Dongyan Zhao ◽  
...  
Keyword(s):  
Real World ◽  
Response Selection ◽  
Representation Learning ◽  
Large Datasets ◽  
Experimental Results ◽  
User Identification ◽  
Real Place ◽  
Real World Applications ◽  
Multi Level ◽  
The Given

Existing multi-turn context-response matching methods mainly concentrate on obtaining multi-level and multi-dimension representations and better interactions between context utterances and response. However, in real-place conversation scenarios, whether a response candidate is suitable not only counts on the given dialogue context but also other backgrounds, e.g., wording habits, user-specific dialogue history content. To fill the gap between these up-to-date methods and the real-world applications, we incorporate user-specific dialogue history into the response selection and propose a personalized hybrid matching network (PHMN). Our contributions are two-fold: (1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information; (2) we perform hybrid representation learning on context-response utterances and explicitly incorporate a customized attention mechanism to extract vital information from context-response interactions so as to improve the accuracy of matching. We evaluate our model on two large datasets with user identification, i.e., personalized Ubuntu dialogue Corpus (P-Ubuntu) and personalized Weibo dataset (P-Weibo). Experimental results confirm that our method significantly outperforms several strong models by combining personalized attention, wording behaviors, and hybrid representation learning.

Unsupervised Subspace Extraction via Deep Kernelized Clustering

2021 ◽  
Vol 16 (1) ◽  
pp. 1-15
Author(s):  
Gyoung S. Na ◽  
Hyunju Chang
Keyword(s):  
Feature Extraction ◽  
Side Information ◽  
Feature Space ◽  
Extraction Process ◽  
Extraction Methods ◽  
Feature Extraction Method ◽  
Latent Features ◽  
Benchmark Datasets ◽  
The Given ◽  
Unsupervised Feature Extraction

Feature extraction has been widely studied to find informative latent features and reduce the dimensionality of data. In particular, due to the difficulty in obtaining labeled data, unsupervised feature extraction has received much attention in data mining. However, widely used unsupervised feature extraction methods require side information about data or rigid assumptions on the latent feature space. Furthermore, most feature extraction methods require predefined dimensionality of the latent feature space,which should be manually tuned as a hyperparameter. In this article, we propose a new unsupervised feature extraction method called Unsupervised Subspace Extractor ( USE ), which does not require any side information and rigid assumptions on data. Furthermore, USE can find a subspace generated by a nonlinear combination of the input feature and automatically determine the optimal dimensionality of the subspace for the given nonlinear combination. The feature extraction process of USE is well justified mathematically, and we also empirically demonstrate the effectiveness of USE for several benchmark datasets.

scholarly journals Efficient Graph Collaborative Filtering via Contrastive Learning

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4666
Author(s):  
Zhiqiang Pan ◽  
Honghui Chen
Keyword(s):  
Collaborative Filtering ◽  
State Of The Art ◽  
Representation Learning ◽  
First Order ◽  
Satisfactory Performance ◽  
Potential Applications ◽  
Benchmark Datasets ◽  
Bayesian Personalized Ranking ◽  
Graph Neural Networks ◽  
Training Efficiency

Collaborative filtering (CF) aims to make recommendations for users by detecting user’s preference from the historical user–item interactions. Existing graph neural networks (GNN) based methods achieve satisfactory performance by exploiting the high-order connectivity between users and items, however they suffer from the poor training efficiency problem and easily introduce bias for information propagation. Moreover, the widely applied Bayesian personalized ranking (BPR) loss is insufficient to provide supervision signals for training due to the extremely sparse observed interactions. To deal with the above issues, we propose the Efficient Graph Collaborative Filtering (EGCF) method. Specifically, EGCF adopts merely one-layer graph convolution to model the collaborative signal for users and items from the first-order neighbors in the user–item interactions. Moreover, we introduce contrastive learning to enhance the representation learning of users and items by deriving the self-supervisions, which is jointly trained with the supervised learning. Extensive experiments are conducted on two benchmark datasets, i.e., Yelp2018 and Amazon-book, and the experimental results demonstrate that EGCF can achieve the state-of-the-art performance in terms of Recall and normalized discounted cumulative gain (NDCG), especially on ranking the target items at right positions. In addition, EGCF shows obvious advantages in the training efficiency compared with the competitive baselines, making it practicable for potential applications.

Markov processes associated with critical Galton-Watson processes with application to extinction probabilities

1976 ◽  
Vol 8 (2) ◽  
pp. 278-295 ◽  
Author(s):  
Michael Sze
Keyword(s):  
Continuous Time ◽  
Branching Process ◽  
Continuous Process ◽  
Regularly Varying Functions ◽  
Original Process ◽  
Additional Information ◽  
Embedding Technique ◽  
Extinction Probabilities ◽  
Watson Process ◽  
The Given

As an alternative to the embedding technique of T. E. Harris, S. Karlin and J. McGregor, we show that given a critical Galton–Watson process satisfying some mild assumptions, we can always construct a continuous-time Markov branching process having the same asymptotic behaviour as the given process. Thus, via the associated continuous process, additional information about the original process is obtained. We apply this technique to the study of extinction probabilities of a critical Galton–Watson process, and provide estimates for the extinction probabilities by regularly varying functions.

scholarly journals Domain-specific Evaluation Dataset Generator for Multilingual Text Analysis

2019 ◽  
pp. 140-147
Author(s):  
Emrah Inan ◽  
Vahab Mostafapour ◽  
Fatif Tekbacak
Keyword(s):  
Text Analysis ◽  
General Purpose ◽  
Entity Linking ◽  
Named Entity ◽  
Domain Specific ◽  
Benchmark Datasets ◽  
Concise Information ◽  
Multilingual Text ◽  
The Given ◽  
Specific Evaluation

Web enables to retrieve concise information about specific entities including people, organizations, movies and their features. Additionally, large amount of Web resources generally lies on a unstructured form and it tackles to find critical information for specific entities. Text analysis approaches such as Named Entity Recognizer and Entity Linking aim to identify entities and link them to relevant entities in the given knowledge base. To evaluate these approaches, there are a vast amount of general purpose benchmark datasets. However, it is difficult to evaluate domain-specific approaches due to lack of evaluation datasets for specific domains. This study presents WeDGeM that is a multilingual evaluation set generator for specific domains exploiting Wikipedia category pages and DBpedia hierarchy. Also, Wikipedia disambiguation pages are used to adjust the ambiguity level of the generated texts. Based on this generated test data, a use case for well-known Entity Linking systems supporting Turkish texts are evaluated in the movie domain.

scholarly journals VLSP SHARED TASK: SENTIMENT ANALYSIS

2019 ◽  
Vol 34 (4) ◽  
pp. 295-310 ◽  
Author(s):  
Huyen T M Nguyen ◽  
Hung V Nguyen ◽  
Quyen T Ngo ◽  
Luong X Vu ◽  
Vu Mai Tran ◽  
...  
Keyword(s):  
Sentiment Analysis ◽  
Language Processing ◽  
Objective Evaluation ◽  
Shared Task ◽  
Electronic Products ◽  
Performance Quality ◽  
Benchmark Datasets ◽  
Active Research ◽  
Evaluation Measurement

Sentiment analysis is a natural language processing (NLP) task of identifying orextracting the sentiment content of a text unit. This task has become an active research topic since the early 2000s. During the two last editions of the VLSP workshop series, the shared task on Sentiment Analysis (SA) for Vietnamese has been organized in order to provide an objective evaluation measurement about the performance (quality) of sentiment analysis tools, and encouragethe development of Vietnamese sentiment analysis systems, as well as to provide benchmark datasets for this task. The rst campaign in 2016 only focused on the sentiment polarity classication, with a dataset containing reviews of electronic products. The second campaign in 2018 addressed the problem of Aspect Based Sentiment Analysis (ABSA) for Vietnamese, by providing two datasets containing reviews in restaurant and hotel domains. These data are accessible for research purpose via the VLSP website vlsp.org.vn/resources. This paper describes the built datasets as well as the evaluation results of the systems participating to these campaigns.

scholarly journals Multi-Interactive Memory Network for Aspect Based Multimodal Sentiment Analysis

2019 ◽  
Vol 33 ◽  
pp. 371-378 ◽  
Author(s):  
Nan Xu ◽  
Wenji Mao ◽  
Guandan Chen
Keyword(s):  
Sentiment Analysis ◽  
Visual Information ◽  
Specific Aspect ◽  
Text And Image ◽  
Research Attention ◽  
Multimodal Data ◽  
Proposed Model ◽  
Memory Network ◽  
Multimodal Sentiment Analysis ◽  
The Given

As a fundamental task of sentiment analysis, aspect-level sentiment analysis aims to identify the sentiment polarity of a specific aspect in the context. Previous work on aspect-level sentiment analysis is text-based. With the prevalence of multimodal user-generated content (e.g. text and image) on the Internet, multimodal sentiment analysis has attracted increasing research attention in recent years. In the context of aspect-level sentiment analysis, multimodal data are often more important than text-only data, and have various correlations including impacts that aspect brings to text and image as well as the interactions associated with text and image. However, there has not been any related work carried out so far at the intersection of aspect-level and multimodal sentiment analysis. To fill this gap, we are among the first to put forward the new task, aspect based multimodal sentiment analysis, and propose a novel Multi-Interactive Memory Network (MIMN) model for this task. Our model includes two interactive memory networks to supervise the textual and visual information with the given aspect, and learns not only the interactive influences between cross-modality data but also the self influences in single-modality data. We provide a new publicly available multimodal aspect-level sentiment dataset to evaluate our model, and the experimental results demonstrate the effectiveness of our proposed model for this new task.

scholarly journals A Unified Model for Opinion Target Extraction and Target Sentiment Prediction

2019 ◽  
Vol 33 ◽  
pp. 6714-6721 ◽  
Author(s):  
Xin Li ◽  
Lidong Bing ◽  
Piji Li ◽  
Wai Lam
Keyword(s):  
Neural Networks ◽  
Sentiment Analysis ◽  
Recurrent Neural Networks ◽  
Unified Model ◽  
Primary Target ◽  
Target Extraction ◽  
Final Output ◽  
Benchmark Datasets ◽  
Task Dependency ◽  
Current Word

Target-based sentiment analysis involves opinion target extraction and target sentiment classification. However, most of the existing works usually studied one of these two sub-tasks alone, which hinders their practical use. This paper aims to solve the complete task of target-based sentiment analysis in an end-to-end fashion, and presents a novel unified model which applies a unified tagging scheme. Our framework involves two stacked recurrent neural networks: The upper one predicts the unified tags to produce the final output results of the primary target-based sentiment analysis; The lower one performs an auxiliary target boundary prediction aiming at guiding the upper network to improve the performance of the primary task. To explore the inter-task dependency, we propose to explicitly model the constrained transitions from target boundaries to target sentiment polarities. We also propose to maintain the sentiment consistency within an opinion target via a gate mechanism which models the relation between the features for the current word and the previous word. We conduct extensive experiments on three benchmark datasets and our framework achieves consistently superior results.

Sign in / Sign up

Export Citation Format

Share Document