Multi-Domain Sentiment Classification Based on Domain-Aware Embedding and Attention

Attention Mechanism ◽

Classification Model ◽

Word Embeddings ◽

Classification Models ◽

Domain Specific ◽

Proposed Model ◽

Public Datasets ◽

Form Domain

Sentiment classification is a fundamental task in NLP. However, as revealed by many researches, sentiment classification models are highly domain-dependent. It is worth investigating to leverage data from different domains to improve the classification performance in each domain. In this work, we propose a novel completely-shared multi-domain neural sentiment classification model to learn domain-aware word embeddings and make use of domain-aware attention mechanism. Our model first utilizes BiLSTM for domain classification and extracts domain-specific features for words, which are then combined with general word embeddings to form domain-aware word embeddings. Domain-aware word embeddings are fed into another BiLSTM to extract sentence features. The domain-aware attention mechanism is used for selecting significant features, by using the domain-aware sentence representation as the query vector. Evaluation results on public datasets with 16 different domains demonstrate the efficacy of our proposed model. Further experiments show the generalization ability and the transferability of our model.

A Novel Model for Imbalanced Data Classification

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.6145 ◽

2020 ◽

Vol 34 (04) ◽

pp. 6680-6687

Author(s):

Jian Yin ◽

Chunjing Gan ◽

Kaiqi Zhao ◽

Xuan Lin ◽

Zhe Quan ◽

...

Keyword(s):

Imbalanced Data ◽

Data Classification ◽

Classification Model ◽

Proposed Model ◽

Imbalanced Data Classification ◽

Public Datasets ◽

Distribution Cost ◽

Novel Model ◽

Learning Data

Recently, imbalanced data classification has received much attention due to its wide applications. In the literature, existing researches have attempted to improve the classification performance by considering various factors such as the imbalanced distribution, cost-sensitive learning, data space improvement, and ensemble learning. Nevertheless, most of the existing methods focus on only part of these main aspects/factors. In this work, we propose a novel imbalanced data classification model that considers all these main aspects. To evaluate the performance of our proposed model, we have conducted experiments based on 14 public datasets. The results show that our model outperforms the state-of-the-art methods in terms of recall, G-mean, F-measure and AUC.

Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification

Applied Sciences ◽

10.3390/app9183717 ◽

2019 ◽

Vol 9 (18) ◽

pp. 3717 ◽

Cited By ~ 3

Author(s):

Wenkuan Li ◽

Dongyuan Li ◽

Hongxia Yin ◽

Lindong Zhang ◽

Zhenfang Zhu ◽

...

Keyword(s):

Deep Learning ◽

Large Scale ◽

Attention Mechanism ◽

Classification Model ◽

Linguistic Knowledge ◽

Great Success ◽

Text Representation ◽

Attention Network ◽

Sentiment Lexicon

Text representation learning is an important but challenging issue for various natural language processing tasks. Recently, deep learning-based representation models have achieved great success for sentiment classification. However, these existing models focus on more semantic information rather than sentiment linguistic knowledge, which provides rich sentiment information and plays a key role in sentiment analysis. In this paper, we propose a lexicon-enhanced attention network (LAN) based on text representation to improve the performance of sentiment classification. Specifically, we first propose a lexicon-enhanced attention mechanism by combining the sentiment lexicon with an attention mechanism to incorporate sentiment linguistic knowledge into deep learning methods. Second, we introduce a multi-head attention mechanism in the deep neural network to interactively capture the contextual information from different representation subspaces at different positions. Furthermore, we stack a LAN model to build a hierarchical sentiment classification model for large-scale text. Extensive experiments are conducted to evaluate the effectiveness of the proposed models on four popular real-world sentiment classification datasets at both the sentence level and the document level. The experimental results demonstrate that our proposed models can achieve comparable or better performance than the state-of-the-art methods.

ECG Heartbeat Classification Based on an Improved ResNet-18 Model

Computational and Mathematical Methods in Medicine ◽

10.1155/2021/6649970 ◽

2021 ◽

Vol 2021 ◽

pp. 1-13

Author(s):

Enbiao Jing ◽

Haiyang Zhang ◽

ZhiGang Li ◽

Yazhi Liu ◽

Zhanlin Ji ◽

...

Keyword(s):

State Of The Art ◽

Classification Models ◽

Heartbeat Classification ◽

Ecg Signals ◽

Residual Structure ◽

Proposed Model ◽

Model Training ◽

Electrocardiogram Ecg

Based on a convolutional neural network (CNN) approach, this article proposes an improved ResNet-18 model for heartbeat classification of electrocardiogram (ECG) signals through appropriate model training and parameter adjustment. Due to the unique residual structure of the model, the utilized CNN layered structure can be deepened in order to achieve better classification performance. The results of applying the proposed model to the MIT-BIH arrhythmia database demonstrate that the model achieves higher accuracy (96.50%) compared to other state-of-the-art classification models, while specifically for the ventricular ectopic heartbeat class, its sensitivity is 93.83% and the precision is 97.44%.

Fine-Grained Image Classification for Crop Disease Based on Attention Mechanism

Frontiers in Plant Science ◽

10.3389/fpls.2020.600854 ◽

2020 ◽

Vol 11 ◽

Author(s):

Guofeng Yang ◽

Yong He ◽

Yong Yang ◽

Beibei Xu

Keyword(s):

Image Classification ◽

Classification Accuracy ◽

Attention Mechanism ◽

Classification Model ◽

Classification Models ◽

Fine Grained ◽

Complex Scenes ◽

Crop Disease ◽

Visual Disturbances

Fine-grained image classification is a challenging task because of the difficulty in identifying discriminant features, it is not easy to find the subtle features that fully represent the object. In the fine-grained classification of crop disease, visual disturbances such as light, fog, overlap, and jitter are frequently encountered. To explore the influence of the features of crop leaf images on the classification results, a classification model should focus on the more discriminative regions of the image while improving the classification accuracy of the model in complex scenes. This paper proposes a novel attention mechanism that effectively utilizes the informative regions of an image, and describes the use of transfer learning to quickly construct several fine-grained image classification models of crop disease based on this attention mechanism. This study uses 58,200 crop leaf images as a dataset, including 14 different crops and 37 different categories of healthy/diseased crops. Among them, different diseases of the same crop have strong similarities. The NASNetLarge fine-grained classification model based on the proposed attention mechanism achieves the best classification effect, with an F1 score of up to 93.05%. The results show that the proposed attention mechanism effectively improves the fine-grained classification of crop disease images.

Aspect-Level Sentiment Analysis Based on Position Features Using Multilevel Interactive Bidirectional GRU and Attention Mechanism

Discrete Dynamics in Nature and Society ◽

10.1155/2020/5824873 ◽

2020 ◽

Vol 2020 ◽

pp. 1-13

Author(s):

Xiaodi Wang ◽

Xiaoliang Chen ◽

Mingwei Tang ◽

Tian Yang ◽

Zhen Wang

Keyword(s):

Neural Network ◽

Sentiment Analysis ◽

Network Models ◽

Attention Mechanism ◽

Classification Model ◽

Neural Network Models ◽

Position Information ◽

Gated Recurrent Unit ◽

Relative Position Information

The aim of aspect-level sentiment analysis is to identify the sentiment polarity of a given target term in sentences. Existing neural network models provide a useful account of how to judge the polarity. However, context relative position information for the target terms is adversely ignored under the limitation of training datasets. Considering position features between words into the models can improve the accuracy of sentiment classification. Hence, this study proposes an improved classification model by combining multilevel interactive bidirectional Gated Recurrent Unit (GRU), attention mechanisms, and position features (MI-biGRU). Firstly, the position features of words in a sentence are initialized to enrich word embedding. Secondly, the approach extracts the features of target terms and context by using a well-constructed multilevel interactive bidirectional neural network. Thirdly, an attention mechanism is introduced so that the model can pay greater attention to those words that are important for sentiment analysis. Finally, four classic sentiment classification datasets are used to deal with aspect-level tasks. Experimental results indicate that there is a correlation between the multilevel interactive attention network and the position features. MI-biGRU can obviously improve the performance of classification.

A Sentiment Classification Model Using Group Characteristics of Writing Style Features

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s021800141756016x ◽

2017 ◽

Vol 31 (12) ◽

pp. 1756016 ◽

Cited By ~ 3

Author(s):

Huan Zhao ◽

Xixiang Zhang ◽

Keqin Li

Keyword(s):

Classification Accuracy ◽

Classification Model ◽

Writing Style ◽

Data Set ◽

Proposed Model ◽

Polarity Classification ◽

And Gender ◽

Authorship Identification ◽

Group Characteristics

Sentiment analysis is becoming increasingly important mainly because of the growth of web comments. Sentiment polarity classification is a popular process in this field. Writing style features, such as lexical and word-based features, are often used in the authorship identification and gender classification of online messages. However, writing style features were only used in feature selection for sentiment classification. This research presents an exploratory study of the group characteristics of writing style features on the Internet Movie Database (IMDb) movie sentiment data set. Furthermore, this study utilizes the specific group characteristics of writing style in improving the performance of sentiment classification. We determine the optimum clustering number of user reviews based on writing style features distribution. According to the classification model trained on a training subset with specific writing style clustering tags, we determine that the model trained on the data set of a specific writing style group has an optimal effect on the classification accuracy, which is better than the model trained on the entire data set in a particular positive or negative polarity. Through the polarity characteristics of specific writing style groups, we propose a general model in improving the performance of the existing classification approach. Results of the experiments on sentiment classification using the IMDb data set demonstrate that the proposed model improves the performance in terms of classification accuracy.

Application of an emotional classification model in e-commerce text based on an improved transformer model

PLoS ONE ◽

10.1371/journal.pone.0247984 ◽

2021 ◽

Vol 16 (3) ◽

pp. e0247984

Author(s):

Xuyang Wang ◽

Yixuan Tong

Keyword(s):

Neural Network ◽

Deep Learning ◽

Text Classification ◽

Rapid Development ◽

Attention Mechanism ◽

Mobile Internet ◽

Classification Model ◽

Test Model ◽

Transformer Model

With the rapid development of the mobile internet, people are becoming more dependent on the internet to express their comments on products or stores; meanwhile, text sentiment classification of these comments has become a research hotspot. In existing methods, it is fairly popular to apply a deep learning method to the text classification task. Aiming at solving information loss, weak context and other problems, this paper makes an improvement based on the transformer model to reduce the difficulty of model training and training time cost and achieve higher overall model recall and accuracy in text sentiment classification. The transformer model replaces the traditional convolutional neural network (CNN) and the recurrent neural network (RNN) and is fully based on the attention mechanism; therefore, the transformer model effectively improves the training speed and reduces training difficulty. This paper selects e-commerce reviews as research objects and applies deep learning theory. First, the text is preprocessed by word vectorization. Then the IN standardized method and the GELUs activation function are applied based on the original model to analyze the emotional tendencies of online users towards stores or products. The experimental results show that our method improves by 9.71%, 6.05%, 5.58% and 5.12% in terms of recall and approaches the peak level of the F1 value in the test model by comparing BiLSTM, Naive Bayesian Model, the serial BiLSTM_CNN model and BiLSTM with an attention mechanism model. Therefore, this finding proves that our method can be used to improve the text sentiment classification accuracy and effectively apply the method to text classification.

Compound Multiscale Weak Dense Network with Hybrid Attention for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13163305 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3305

Author(s):

Zixian Ge ◽

Guo Cao ◽

Hao Shi ◽

Youqiang Zhang ◽

Xuesong Li ◽

...

Keyword(s):

Feature Extraction ◽

Hyperspectral Image ◽

Spectral Feature ◽

Research Direction ◽

Performance Comparison ◽

Attention Mechanism ◽

Semantic Features ◽

Public Datasets ◽

High Level

Recently, hyperspectral image (HSI) classification has become a popular research direction in remote sensing. The emergence of convolutional neural networks (CNNs) has greatly promoted the development of this field and demonstrated excellent classification performance. However, due to the particularity of HSIs, redundant information and limited samples pose huge challenges for extracting strong discriminative features. In addition, addressing how to fully mine the internal correlation of the data or features based on the existing model is also crucial in improving classification performance. To overcome the above limitations, this work presents a strong feature extraction neural network with an attention mechanism. Firstly, the original HSI is weighted by means of the hybrid spectral–spatial attention mechanism. Then, the data are input into a spectral feature extraction branch and a spatial feature extraction branch, composed of multiscale feature extraction modules and weak dense feature extraction modules, to extract high-level semantic features. These two features are compressed and fused using the global average pooling and concat approaches. Finally, the classification results are obtained by using two fully connected layers and one Softmax layer. A performance comparison shows the enhanced classification performance of the proposed model compared to the current state of the art on three public datasets.

Improving Document-Level Sentiment Classification Using Importance of Sentences

Entropy ◽

10.3390/e22121336 ◽

2020 ◽

Vol 22 (12) ◽

pp. 1336

Author(s):

Gihyeon Choi ◽

Shinhyeok Oh ◽

Harksoo Kim

Keyword(s):

Sentiment Analysis ◽

Deep Neural Networks ◽

Classification Model ◽

Classification Task ◽

Analysis Model ◽

Proposed Model ◽

Previous State ◽

Restaurant Reviews ◽

Document Level

Previous researchers have considered sentiment analysis as a document classification task, in which input documents are classified into predefined sentiment classes. Although there are sentences in a document that support important evidences for sentiment analysis and sentences that do not, they have treated the document as a bag of sentences. In other words, they have not considered the importance of each sentence in the document. To effectively determine polarity of a document, each sentence in the document should be dealt with different degrees of importance. To address this problem, we propose a document-level sentence classification model based on deep neural networks, in which the importance degrees of sentences in documents are automatically determined through gate mechanisms. To verify our new sentiment analysis model, we conducted experiments using the sentiment datasets in the four different domains such as movie reviews, hotel reviews, restaurant reviews, and music reviews. In the experiments, the proposed model outperformed previous state-of-the-art models that do not consider importance differences of sentences in a document. The experimental results show that the importance of sentences should be considered in a document-level sentiment classification task.

Leveraging Pre-Trained Contextualized Word Embeddings to Enhance Sentiment Classification of Drug Reviews

Revue d intelligence artificielle ◽

10.18280/ria.350405 ◽

2021 ◽

Vol 35 (4) ◽

pp. 307-314

Author(s):

Redouane Karsi ◽

Mounia Zaim ◽

Jamila El Alami

Keyword(s):

Sentiment Analysis ◽

Small Sample ◽

Training Data ◽

Svm Classifier ◽

Word Embeddings ◽

Classification Problems ◽

Lexical Resources ◽

Word Representation

Traditionally, pharmacovigilance data are collected during clinical trials on a small sample of patients and are therefore insufficient to adequately assess drugs. Nowadays, consumers use online drug forums to share their opinions and experiences about medication. These feedbacks, which are widely available on the web, are automatically analyzed to extract relevant information for decision-making. Currently, sentiment analysis methods are being put forward to leverage consumers' opinions and produce useful drug monitoring indicators. However, these methods' effectiveness depends on the quality of word representation, which presents a real challenge because the information contained in user reviews is noisy and very subjective. Over time, several sentiment classification problems use machine learning methods based on the traditional bag of words model, sometimes enhanced with lexical resources. In recent years, word embedding models have significantly improved classification performance due to their ability to capture words' syntactic and semantic properties. Unfortunately, these latter models are weak in sentiment classification tasks because they are unable to encode sentiment information in the word representation. Indeed, two words with opposite polarities can have close word embeddings as they appear together in the same context. To overcome this drawback, some studies have proposed refining pre-trained word embeddings with lexical resources or learning word embeddings using training data. However, these models depend on external resources and are complex to implement. This work proposes a deep contextual word embeddings model called ELMo that inherently captures the sentiment information by providing separate vectors for words with opposite polarities. Different variants of our proposed model are compared with a benchmark of pre-trained word embeddings models using SVM classifier trained on Drug Review Dataset. Experimental results show that ELMo embeddings improve classification performance in sentiment analysis tasks on the pharmaceutical domain.