scholarly journals Sentiment Classification Algorithm Based on the Cascade of BERT Model and Adaptive Sentiment Dictionary

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Ruixue Duan ◽  
Zhuofan Huang ◽  
Yangsen Zhang ◽  
Xiulei Liu ◽  
Yue Dang
Keyword(s):  
Deep Learning ◽  
Mobile Network ◽  
Sentiment Classification ◽  
Language Models ◽  
Classification Model ◽  
Data Sets ◽  
Probability Ratio ◽  
Sentiment Dictionary ◽  
Accuracy Increase ◽  
Rule Method

The mobile social network contains a large amount of information in a form of commentary. Effective analysis of the sentiment in the comments would help improve the recommendations in the mobile network. With the development of well-performing pretrained language models, the performance of sentiment classification task based on deep learning has seen new breakthroughs in the past decade. However, deep learning models suffer from poor interpretability, making it difficult to integrate sentiment knowledge into the model. This paper proposes a sentiment classification model based on the cascade of the BERT model and the adaptive sentiment dictionary. First, the pretrained BERT model is used to fine-tune with the training corpus, and the probability of sentiment classification in different categories is obtained through the softmax layer. Next, to allow a more effective comparison between the probabilities for the two classes, a nonlinearity is introduced in a form of positive-negative probability ratio, using the rule method based on sentiment dictionary to deal with the probability ratio below the threshold. This method of cascading the pretrained model and the semantic rules of the sentiment dictionary allows to utilize the advantages of both models. Different sized Chnsenticorp data sets are used to train the proposed model. Experimental results show that the Dict-BERT model is better than the BERT-only model, especially when the training set is relatively small. The improvement is obvious with the accuracy increase of 0.8%.

scholarly journals Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification

2019 ◽  
Vol 9 (18) ◽  
pp. 3717 ◽  
Author(s):  
Wenkuan Li ◽  
Dongyuan Li ◽  
Hongxia Yin ◽  
Lindong Zhang ◽  
Zhenfang Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Attention Mechanism ◽  
Sentiment Classification ◽  
Classification Model ◽  
Linguistic Knowledge ◽  
Great Success ◽  
Text Representation ◽  
Attention Network ◽  
Sentiment Lexicon

Text representation learning is an important but challenging issue for various natural language processing tasks. Recently, deep learning-based representation models have achieved great success for sentiment classification. However, these existing models focus on more semantic information rather than sentiment linguistic knowledge, which provides rich sentiment information and plays a key role in sentiment analysis. In this paper, we propose a lexicon-enhanced attention network (LAN) based on text representation to improve the performance of sentiment classification. Specifically, we first propose a lexicon-enhanced attention mechanism by combining the sentiment lexicon with an attention mechanism to incorporate sentiment linguistic knowledge into deep learning methods. Second, we introduce a multi-head attention mechanism in the deep neural network to interactively capture the contextual information from different representation subspaces at different positions. Furthermore, we stack a LAN model to build a hierarchical sentiment classification model for large-scale text. Extensive experiments are conducted to evaluate the effectiveness of the proposed models on four popular real-world sentiment classification datasets at both the sentence level and the document level. The experimental results demonstrate that our proposed models can achieve comparable or better performance than the state-of-the-art methods.

Negative emotion diffusion and intervention countermeasures of social networks based on deep learning

2020 ◽  
Vol 39 (4) ◽  
pp. 4935-4945
Author(s):  
Qiuyun Cheng ◽  
Yun Ke ◽  
Ahmed Abdelmouty
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Negative Emotion ◽  
Sentiment Classification ◽  
Classification Model ◽  
Learning Models ◽  
Data Set ◽  
Position Information ◽  
Processing Technologies ◽  
And Training

Aiming at the limitation of using only word features in traditional deep learning sentiment classification, this paper combines topic features with deep learning models to build a topic-fused deep learning sentiment classification model. The model can fuse topic features to obtain high-quality high-level text features. Experiments show that in binary sentiment classification, the highest classification accuracy of the model can reach more than 90%, which is higher than that of commonly used deep learning models. This paper focuses on the combination of deep neural networks and emerging text processing technologies, and improves and perfects them from two aspects of model architecture and training methods, and designs an efficient deep network sentiment analysis model. A CNN (Convolutional Neural Network) model based on polymorphism is proposed. The model constructs the CNN input matrix by combining the word vector information of the text, the emotion information of the words, and the position information of the words, and adjusts the importance of different feature information in the training process by means of weight control. The multi-objective sample data set is used to verify the effectiveness of the proposed model in the sentiment analysis task of related objects from the classification effect and training performance.

scholarly journals Deep Relation Network for Hyperspectral Image Few-Shot Classification

2020 ◽  
Vol 12 (6) ◽  
pp. 923 ◽  
Author(s):  
Kuiliang Gao ◽  
Bing Liu ◽  
Xuchu Yu ◽  
Jinchun Qin ◽  
Pengqiang Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Hyperspectral Image ◽  
Hyperspectral Images ◽  
Classification Model ◽  
Support Vector ◽  
Data Sets ◽  
Learning Models ◽  
Shot Classification ◽  
Learning Module ◽  
Relation Learning

Deep learning has achieved great success in hyperspectral image classification. However, when processing new hyperspectral images, the existing deep learning models must be retrained from scratch with sufficient samples, which is inefficient and undesirable in practical tasks. This paper aims to explore how to accurately classify new hyperspectral images with only a few labeled samples, i.e., the hyperspectral images few-shot classification. Specifically, we design a new deep classification model based on relational network and train it with the idea of meta-learning. Firstly, the feature learning module and the relation learning module of the model can make full use of the spatial–spectral information in hyperspectral images and carry out relation learning by comparing the similarity between samples. Secondly, the task-based learning strategy can enable the model to continuously enhance its ability to learn how to learn with a large number of tasks randomly generated from different data sets. Benefitting from the above two points, the proposed method has excellent generalization ability and can obtain satisfactory classification results with only a few labeled samples. In order to verify the performance of the proposed method, experiments were carried out on three public data sets. The results indicate that the proposed method can achieve better classification results than the traditional semisupervised support vector machine and semisupervised deep learning models.

scholarly journals Application of an emotional classification model in e-commerce text based on an improved transformer model

PLoS ONE ◽  
2021 ◽  
Vol 16 (3) ◽  
pp. e0247984
Author(s):  
Xuyang Wang ◽  
Yixuan Tong
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Text Classification ◽  
Rapid Development ◽  
Attention Mechanism ◽  
Mobile Internet ◽  
Sentiment Classification ◽  
Classification Model ◽  
Test Model ◽  
Transformer Model

With the rapid development of the mobile internet, people are becoming more dependent on the internet to express their comments on products or stores; meanwhile, text sentiment classification of these comments has become a research hotspot. In existing methods, it is fairly popular to apply a deep learning method to the text classification task. Aiming at solving information loss, weak context and other problems, this paper makes an improvement based on the transformer model to reduce the difficulty of model training and training time cost and achieve higher overall model recall and accuracy in text sentiment classification. The transformer model replaces the traditional convolutional neural network (CNN) and the recurrent neural network (RNN) and is fully based on the attention mechanism; therefore, the transformer model effectively improves the training speed and reduces training difficulty. This paper selects e-commerce reviews as research objects and applies deep learning theory. First, the text is preprocessed by word vectorization. Then the IN standardized method and the GELUs activation function are applied based on the original model to analyze the emotional tendencies of online users towards stores or products. The experimental results show that our method improves by 9.71%, 6.05%, 5.58% and 5.12% in terms of recall and approaches the peak level of the F1 value in the test model by comparing BiLSTM, Naive Bayesian Model, the serial BiLSTM_CNN model and BiLSTM with an attention mechanism model. Therefore, this finding proves that our method can be used to improve the text sentiment classification accuracy and effectively apply the method to text classification.

scholarly journals SupportNet: a novel incremental learning framework through deep learning and support data

2018 ◽  
Author(s):  
Yu Li ◽  
Zhongxiao Li ◽  
Lizhong Ding ◽  
Yuhui Hu ◽  
Wei Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Incremental Learning ◽  
Learning Model ◽  
Biological Data ◽  
Classification Model ◽  
Support Vector ◽  
Data Sets ◽  
Essential Information ◽  
New Information ◽  
Deep Learning Model

ABSTRACTMotivationIn most biological data sets, the amount of data is regularly growing and the number of classes is continuously increasing. To deal with the new data from the new classes, one approach is to train a classification model, e.g., a deep learning model, from scratch based on both old and new data. This approach is highly computationally costly and the extracted features are likely very different from the ones extracted by the model trained on the old data alone, which leads to poor model robustness. Another approach is to fine tune the trained model from the old data on the new data. However, this approach often does not have the ability to learn new knowledge without forgetting the previously learned knowledge, which is known as the catastrophic forgetting problem. To our knowledge, this problem has not been studied in the field of bioinformatics despite its existence in many bioinformatic problems.ResultsHere we propose a novel method, SupportNet, to solve the catastrophic forgetting problem efficiently and effectively. SupportNet combines the strength of deep learning and support vector machine (SVM), where SVM is used to identify the support data from the old data, which are fed to the deep learning model together with the new data for further training so that the model can review the essential information of the old data when learning the new information. Two powerful consolidation regularizers are applied to ensure the robustness of the learned model. Comprehensive experiments on various tasks, including enzyme function prediction, subcellular structure classification and breast tumor classification, show that SupportNet drastically outperforms the state-of-the-art incremental learning methods and reaches similar performance as the deep learning model trained from scratch on both old and new data.AvailabilityOur program is accessible at: https://github.com/lykaust15/SupportNet.

scholarly journals Detection of Sentiment Analysis in Social Media using Deep Learning

2020 ◽  
Vol 9 (1) ◽  
pp. 636-643
Keyword(s):  
Social Media ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Opinion Mining ◽  
Activation Function ◽  
Sentiment Classification ◽  
Network Layers ◽  
Sentiment Dictionary ◽  
Lstm Network ◽  
The Military

With the rapid climb of web page from social media, such studies as online opinion mining or sentiment analysis of text have started receiving attention from government, industry, and academic sectors. In recent years, sentiment analysis has not only emerged under knowledge fusion within the big data era, but has also become a well-liked research topic within the area of AI and machine learning. This study used the Military life PTT board of Taiwan’s largest online forum because the source of its experimental data. the aim of this study was to construct a sentiment analysis framework and processes for social media so as to propose a self-developed military sentiment dictionary for improving sentiment classification and analyze the performance of various deep learning models with various parameter calibration combinations. The experimental results show that the accuracy and F1-measure of the model that mixes existing sentiment dictionaries and therefore the self-developed military sentiment dictionary are better than the results from using existing sentiment dictionaries only. Furthermore, the prediction model trained using the activation function, Tanh, and when the amount of Bi-LSTM network layers is 2, the accuracy and F1-measure have a good better performance for sentiment classification.

Student sentiment classification model based on GRU neural network and TF-IDF algorithm

2020 ◽  
pp. 1-11
Author(s):  
Hailong Yu ◽  
Yannan Ji ◽  
Qinglin Li
Keyword(s):  
Neural Network ◽  
Sentiment Classification ◽  
Classification Model ◽  
Research Model ◽  
Student Group ◽  
Comprehensive Performance ◽  
Sentiment Dictionary ◽  
Neural Network Algorithm ◽  
Sentence Matching

Due to the diversity of text expressions, the text sentiment classification algorithm based on semantic understanding is difficult to establish a perfect sentiment dictionary and sentence matching template, which leads to strong limitations of the algorithm. In particular, it has certain difficulties in the classification of student sentiments. Based on this, this paper analyzes the student sentiment classification model by neural network algorithm and uses the student group as an example to explore the application of neural network model in sentiment classification. Moreover, the regularization method is added to the loss function of LSTM so that the output at any time is related to the output at the previous time. In addition, the sentimental drift distribution of sentimental words on each sentimental label is added to the regularizer, and the sentimental information is merged with the two-way LSTM to allow the model to choose forward or reverse. Finally, in order to verify the research model, the performance of the model proposed in this paper is studied through experimental research. The research shows that the model proposed in this paper has better comprehensive performance than the traditional model and can meet the actual needs of students’ sentiment classification.

A study of Turkish emotion classification with pretrained language models

2021 ◽  
pp. 016555152098550
Author(s):  
Alaettin Uçan ◽  
Murat Dörterler ◽  
Ebru Akçapınar Sezer
Keyword(s):  
Machine Learning ◽  
Language Model ◽  
Experimental Studies ◽  
Classification Performance ◽  
Research Field ◽  
Language Models ◽  
Classification Model ◽  
Data Sets ◽  
Emotion Classification ◽  
Model Approach

Emotion classification is a research field that aims to detect the emotions in a text using machine learning methods. In traditional machine learning (TML) methods, feature engineering processes cause the loss of some meaningful information, and classification performance is negatively affected. In addition, the success of modelling using deep learning (DL) approaches depends on the sample size. More samples are needed for Turkish due to the unique characteristics of the language. However, emotion classification data sets in Turkish are quite limited. In this study, the pretrained language model approach was used to create a stronger emotion classification model for Turkish. Well-known pretrained language models were fine-tuned for this purpose. The performances of these fine-tuned models for Turkish emotion classification were comprehensively compared with the performances of TML and DL methods in experimental studies. The proposed approach provides state-of-the-art performance for Turkish emotion classification.

scholarly journals Text classification models for the automatic detection of nonmedical prescription medication use from social media

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Mohammed Ali Al-Garadi ◽  
Yuan-Chi Yang ◽  
Haitao Cai ◽  
Yucheng Ruan ◽  
Karen O’Connor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Deep Learning ◽  
Text Classification ◽  
Prescription Medication ◽  
The United States ◽  
Language Models ◽  
Classification Model ◽  
Future Research ◽  
Classification Models

Abstract Background Prescription medication (PM) misuse/abuse has emerged as a national crisis in the United States, and social media has been suggested as a potential resource for performing active monitoring. However, automating a social media-based monitoring system is challenging—requiring advanced natural language processing (NLP) and machine learning methods. In this paper, we describe the development and evaluation of automatic text classification models for detecting self-reports of PM abuse from Twitter. Methods We experimented with state-of-the-art bi-directional transformer-based language models, which utilize tweet-level representations that enable transfer learning (e.g., BERT, RoBERTa, XLNet, AlBERT, and DistilBERT), proposed fusion-based approaches, and compared the developed models with several traditional machine learning, including deep learning, approaches. Using a public dataset, we evaluated the performances of the classifiers on their abilities to classify the non-majority “abuse/misuse” class. Results Our proposed fusion-based model performs significantly better than the best traditional model (F1-score [95% CI]: 0.67 [0.64–0.69] vs. 0.45 [0.42–0.48]). We illustrate, via experimentation using varying training set sizes, that the transformer-based models are more stable and require less annotated data compared to the other models. The significant improvements achieved by our best-performing classification model over past approaches makes it suitable for automated continuous monitoring of nonmedical PM use from Twitter. Conclusions BERT, BERT-like and fusion-based models outperform traditional machine learning and deep learning models, achieving substantial improvements over many years of past research on the topic of prescription medication misuse/abuse classification from social media, which had been shown to be a complex task due to the unique ways in which information about nonmedical use is presented. Several challenges associated with the lack of context and the nature of social media language need to be overcome to further improve BERT and BERT-like models. These experimental driven challenges are represented as potential future research directions.

Sign in / Sign up

Export Citation Format

Share Document