scholarly journals Attention-Based CNN and Bi-LSTM Model Based on TF-IDF and GloVe Word Embedding for Sentiment Analysis

2021 ◽  
Vol 11 (23) ◽  
pp. 11255
Author(s):  
Marjan Kamyab ◽  
Guohua Liu ◽  
Michael Adjeisah
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Feature Weighting ◽  
Word Embedding ◽  
Max Pooling ◽  
Document Frequency ◽  
High Level ◽  
Computational Resources

Sentiment analysis (SA) detects people’s opinions from text engaging natural language processing (NLP) techniques. Recent research has shown that deep learning models, i.e., Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Transformer-based provide promising results for recognizing sentiment. Nonetheless, CNN has the advantage of extracting high-level features by using convolutional and max-pooling layers; it cannot efficiently learn a sequence of correlations. At the same time, Bidirectional RNN uses two RNN directions to improve extracting long-term dependencies. However, it cannot extract local features in parallel, and Transformer-based like Bidirectional Encoder Representations from Transformers (BERT) are the computational resources needed to fine-tune, facing an overfitting problem on small datasets. This paper proposes a novel attention-based model that utilizes CNNs with LSTM (named ACL-SA). First, it applies a preprocessor to enhance the data quality and employ term frequency-inverse document frequency (TF-IDF) feature weighting and pre-trained Glove word embedding approaches to extract meaningful information from textual data. In addition, it utilizes CNN’s max-pooling to extract contextual features and reduce feature dimensionality. Moreover, it uses an integrated bidirectional LSTM to capture long-term dependencies. Furthermore, it applies the attention mechanism at the CNN’s output layer to emphasize each word’s attention level. To avoid overfitting, the Guasiannoise and GuasianDroupout are adopted as regularization. The model’s robustness is evaluated on four English standard datasets, i.e., Sentiment140, US-airline, Sentiment140-MV, SA4A with various performance matrices, and compared efficiency with existing baseline models and approaches. The experiment results show that the proposed method significantly outperforms the state-of-the-art models.

An Improved Cross-Domain Sentiment Analysis Based on a Semi-Supervised Convolutional Neural Network

2022 ◽  
pp. 155-170
Author(s):  
Lap-Kei Lee ◽  
Kwok Tai Chui ◽  
Jingjing Wang ◽  
Yin-Chun Fung ◽  
Zhanhui Tan
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Research Work ◽  
Latent Semantic Indexing ◽  
Future Research ◽  
Training Time ◽  
Cross Domain ◽  
Document Frequency

The dependence on Internet in our daily life is ever-growing, which provides opportunity to discover valuable and subjective information using advanced techniques such as natural language processing and artificial intelligence. In this chapter, the research focus is a convolutional neural network for three-class (positive, neutral, and negative) cross-domain sentiment analysis. The model is enhanced in two-fold. First, a similarity label method facilitates the management between the source and target domains to generate more labelled data. Second, term frequency-inverse document frequency (TF-IDF) and latent semantic indexing (LSI) are employed to compute the similarity between source and target domains. Performance evaluation is conducted using three datasets, beauty reviews, toys reviews, and phone reviews. The proposed method enhances the accuracy by 4.3-7.6% and reduces the training time by 50%. The limitations of the research work have been discussed, which serve as the rationales of future research directions.

scholarly journals A data-driven neural network architecture for sentiment analysis

2019 ◽  
Vol 53 (1) ◽  
pp. 2-19 ◽  
Author(s):  
Erion Çano ◽  
Maurizio Morisio
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Network Architecture ◽  
Network Models ◽  
Data Sets ◽  
Feature Maps ◽  
Neural Network Architecture ◽  
Neural Network Models ◽  
Content Type ◽  
Max Pooling

Purpose The fabulous results of convolution neural networks in image-related tasks attracted attention of text mining, sentiment analysis and other text analysis researchers. It is, however, difficult to find enough data for feeding such networks, optimize their parameters, and make the right design choices when constructing network architectures. The purpose of this paper is to present the creation steps of two big data sets of song emotions. The authors also explore usage of convolution and max-pooling neural layers on song lyrics, product and movie review text data sets. Three variants of a simple and flexible neural network architecture are also compared. Design/methodology/approach The intention was to spot any important patterns that can serve as guidelines for parameter optimization of similar models. The authors also wanted to identify architecture design choices which lead to high performing sentiment analysis models. To this end, the authors conducted a series of experiments with neural architectures of various configurations. Findings The results indicate that parallel convolutions of filter lengths up to 3 are usually enough for capturing relevant text features. Also, max-pooling region size should be adapted to the length of text documents for producing the best feature maps. Originality/value Top results the authors got are obtained with feature maps of lengths 6–18. An improvement on future neural network models for sentiment analysis could be generating sentiment polarity prediction of documents using aggregation of predictions on smaller excerpt of the entire text.

scholarly journals Unmasking the conversation on masks: Natural language processing for topical sentiment analysis of COVID-19 Twitter discourse

2020 ◽  
Author(s):  
Abraham Sanders ◽  
Rachael White ◽  
Lauren Severson ◽  
Rufeng Ma ◽  
Richard McQueen ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Public Attitudes ◽  
Online Activity ◽  
Health Crisis ◽  
Public Response ◽  
Health Community ◽  
High Level

In this exploratory study, we scrutinize a database of over 1 million tweets collected across the first five months of 2020 to draw conclusions about public attitudes towards the preventative measure of mask usage during the COVID-19 pandemic. In recent months, a body of literature has emerged to suggest the robustness of trends in online activity as proxies for the epidemiological and sociological impact of COVID-19. We employ natural language processing, clustering and sentiment analysis techniques to organize tweets relating to mask-wearing into high-level themes, then relay narratives for individual clusters through automatic text summarization. We find that topic clustering and visualization based on mask-related Twitter data offers revealing insights into societal perceptions of COVID-19 and techniques for its prevention. We observe that the volume and polarity of mask related tweets has greatly increased. Importantly, the analysis pipeline presented can be leveraged by the health community for the assessment of public response to health interventions in the ongoing global health crisis.

Multi-neural network-based sentiment analysis of food reviews based on character and word embeddings

2020 ◽  
pp. 002072092092849
Author(s):  
Yong Li ◽  
Qingyu Jin ◽  
Min Zuo ◽  
Haisheng Li ◽  
Xiaojun Yang ◽  
...  
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Chinese Character ◽  
Semantic Features ◽  
Word Embeddings ◽  
Emotional Information ◽  
Related Sequence ◽  
Active Research ◽  
Multi Neural Network

Sentiment analysis becomes one of the most active research hotspots in the field of natural language processing tasks in recent years. However, the inability to fully and effectively use emotional information is a problem in present deep learning models. A single Chinese character has different meanings in different words, and the character embeddings are combined with the word embeddings to extract more precise meaning information. In this paper, a single Chinese character and word are used as input units to train. Based on BLSTM, the attention mechanism based on vocabulary semantics in food field is introduced to realize distance-related sequence semantic feature extraction. CNN is used to realize semantic sentiment classification of sequence semantic features. Therefore, a model based on multi-neural network for sentiment information extraction and analysis is proposed. Experiments show that the model has excellent characteristics in sentiment analysis and obtains high accuracy and F value.

scholarly journals Real-time event detection using recurrent neural network in social sensors

2019 ◽  
Vol 15 (6) ◽  
pp. 155014771985649 ◽  
Author(s):  
Van Quan Nguyen ◽  
Tien Nguyen Anh ◽  
Hyung-Jeong Yang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
Event Detection ◽  
Word Embedding ◽  
Series Data ◽  
Accuracy Score ◽  
Data Set ◽  
Size Limitation

We proposed an approach for temporal event detection using deep learning and multi-embedding on a set of text data from social media. First, a convolutional neural network augmented with multiple word-embedding architectures is used as a text classifier for the pre-processing of the input textual data. Second, an event detection model using a recurrent neural network is employed to learn time series data features by extracting temporal information. Recently, convolutional neural networks have been used in natural language processing problems and have obtained excellent results as performing on available embedding vector. In this article, word-embedding features at the embedding layer are combined and fed to convolutional neural network. The proposed method shows no size limitation, supplementation of more embeddings than standard multichannel based approaches, and obtained similar performance (accuracy score) on some benchmark data sets, especially in an imbalanced data set. For event detection, a long short-term memory network is used as a predictor that learns higher level temporal features so as to predict future values. An error distribution estimation model is built to calculate the anomaly score of observation. Events are detected using a window-based method on the anomaly scores.

scholarly journals Deep learning model for metagenome fragment classification using spaced k-mers feature extraction

2020 ◽  
Vol 8 (3) ◽  
pp. 234-238
Author(s):  
Nur Choiriyati ◽  
Yandra Arkeman ◽  
Wisnu Ananta Kusuma
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Deep Learning ◽  
Language Processing ◽  
Computational Time ◽  
Genus Level ◽  
Computational Resources ◽  
Learning Architectures ◽  
Deep Learning Model

An open challenge in bioinformatics is the analysis of the sequenced metagenomes from the various environments. Several studies demonstrated bacteria classification at the genus level using k-mers as feature extraction where the highest value of k gives better accuracy but it is costly in terms of computational resources and computational time. Spaced k-mers method was used to extract the feature of the sequence using 111 1111 10001 where 1 was a match and 0 was the condition that could be a match or did not match. Currently, deep learning provides the best solutions to many problems in image recognition, speech recognition, and natural language processing. In this research, two different deep learning architectures, namely Deep Neural Network (DNN) and Convolutional Neural Network (CNN), trained to approach the taxonomic classification of metagenome data and spaced k-mers method for feature extraction. The result showed the DNN classifier reached 90.89 % and the CNN classifier reached 88.89 % accuracy at the genus level taxonomy.

scholarly journals IMPLEMENTASI PROBABILISTIC NEURAL NETWORK DAN WORD EMBEDDING UNTUK ANALISIS SENTIMEN VAKSIN SINOVAC

2021 ◽  
Vol 3 (2) ◽  
pp. 233-242
Author(s):  
Abdul Rahman Wahid Rapsanjani ◽  
Erfian Junianto
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Probabilistic Neural Network ◽  
Word Embedding

Penelitian ini bertujuan melakukan implementasi Probabilistic neural network dan Word Embedding dalam kasus sentiment analysis tentang tanggapan masyarakat tentang pemberian vaksin sinovac yangg diunggah di Twitter dan 3 class:positif, negative dan netral. Metode yang dipilih adalah metode klasifikasi Probabilistic Neural Network. Sebelum melakukan klasifikasi, praprocessing pada penelitian ini meliputi tokenizasi, normalisasi, menghilangkan emoticon, Convert Negasi, Stemming, Stopword Removal serta Word embedding. dataset yang digunakan berjumlah 1177 dataset dengan pembagiannya yaitu 560 dataset positif, 355 dataset negative dan 262 dataset netral. Program dirancang menggunakan Bahasa pemrograman python dengan beberapa library seperti keras, tensorflow dan pandas. Akurasi yang didapatkan pada pelatihan menggunakan Probabilistic  Neural Network sebesar 91%. Hasil pengujian adalah penelitian ini mampu melakukan sentiment analysis dengan kesalahan sebesar 9%.

scholarly journals Sentiment Analysis Method based on Piecewise Convolutional Neural Network and Generative Adversarial Network

2019 ◽  
Vol 14 (1) ◽  
pp. 7-20 ◽  
Author(s):  
Changshun Du ◽  
Lei Huang
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Convolution Neural Network ◽  
Training Data ◽  
Sentence Structure ◽  
Analysis Model ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Text Sentiment Analysis

Text sentiment analysis is one of the most important tasks in the field of public opinion monitoring, service evaluation and satisfaction analysis under network environments. Compared with the traditional Natural Language Processing analysis tools, convolution neural networks can automatically learn useful features from sentences and improve the performance of the affective analysis model. However, the original convolution neural network model ignores sentence structure information which is very important for text sentiment analysis. In this paper, we add piece-wise pooling to the convolution neural network, which allows the model to obtain the sentence structure. And the main features of different sentences are extracted to analyze the emotional tendencies of the text. At the same time, the user’s feedback involves many different fields, and there is less labeled data. In order to alleviate the sparsity of the data, this paper also uses the generative adversarial network to make common feature extractions, so that the model can obtain the common features associated with emotions in different fields, and improves the model’s Generalization ability with less training data. Experiments on different datasets demonstrate the effectiveness of this method.

scholarly journals Analysis of comparative performance of deep learning models for sentiment analysis

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Mirza Murtaza
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
Neural Nets ◽  
Test Accuracy ◽  
Comparative Performance ◽  
Text Filtering ◽  
Percent Accuracy

Abstract Sentiment analysis of text can be performed using machine learning and natural language processing methods. However, there is no single tool or method that is effective in all cases. The objective of this research project is to determine the effectiveness of neural network-based architecture to perform sentiment analysis of customer comments and reviews, such as the ones on Amazon site. A typical sentiment analysis process involves text preparation (of acquired content), sentiment detection, sentiment classification and analysis of results. In this research, the objective is to a) identify the best approach for text preparation in a given application (text filtering approach to remove errors in data), and, most importantly, b) what is the best machine learning (feed forward neural nets, convolutional neural nets, Long Short-Term Memory networks) approach that provides best classification accuracy. In this research, a set of three thousand two hundred reviews of food related products were used to train and experiment with a neural network-based sentiment analysis system. The neural network implementation of six different models provided close to one-hundred percent accuracy of test data, and a decent test accuracy in mid-80%. The results of the research would be useful to businesses in evaluating customer preferences for products or services.  

Sign in / Sign up

Export Citation Format

Share Document