scholarly journals Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism

2020 ◽  
Vol 10 (17) ◽  
pp. 5841 ◽  
Author(s):  
Beakcheol Jang ◽  
Myeonghwi Kim ◽  
Gaspard Harerimana ◽  
Sang-ug Kang ◽  
Jong Wook Kim
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Text Classification ◽  
Short Term Memory ◽  
User Behavior ◽  
Hybrid Approach ◽  
Attention Mechanism

There is a need to extract meaningful information from big data, classify it into different categories, and predict end-user behavior or emotions. Large amounts of data are generated from various sources such as social media and websites. Text classification is a representative research topic in the field of natural-language processing that categorizes unstructured text data into meaningful categorical classes. The long short-term memory (LSTM) model and the convolutional neural network for sentence classification produce accurate results and have been recently used in various natural-language processing (NLP) tasks. Convolutional neural network (CNN) models use convolutional layers and maximum pooling or max-overtime pooling layers to extract higher-level features, while LSTM models can capture long-term dependencies between word sequences hence are better used for text classification. However, even with the hybrid approach that leverages the powers of these two deep-learning models, the number of features to remember for classification remains huge, hence hindering the training process. In this study, we propose an attention-based Bi-LSTM+CNN hybrid model that capitalize on the advantages of LSTM and CNN with an additional attention mechanism. We trained the model using the Internet Movie Database (IMDB) movie review data to evaluate the performance of the proposed model, and the test results showed that the proposed hybrid attention Bi-LSTM+CNN model produces more accurate classification results, as well as higher recall and F1 scores, than individual multi-layer perceptron (MLP), CNN or LSTM models as well as the hybrid models.

scholarly journals Malicious URL Detection Algorithm Based on Multi Neural Network Series

CONVERTER ◽  
2021 ◽  
pp. 579-590
Author(s):  
Weirong Xiu
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
Detection Algorithm ◽  
Attention Mechanism ◽  
Global Features ◽  
Multi Neural Network

Convolutional neural network based on attention mechanism and a bidirectional independent recurrent neural network tandem joint algorithm (CATIR) are proposed. In natural language processing related technologies, word vector features are extracted based on URLs, and the extracted URL information features and host information features are merged. The proposed CATIR algorithm uses CNN (Convolutional Neural Network) to obtain the deep local features in the data, uses the Attention mechanism to adjust the weights, and uses IndRNN (Independent Recurrent Neural Network) to obtain the global features in the data. The experimental results shows that the CATIR algorithm has significantly improved the accuracy of malicious URL detection based on traditional algorithms to 96.9%.

scholarly journals Towards Accurate Deceptive Opinions Detection Based on Word Order-Preserving CNN

2018 ◽  
Vol 2018 ◽  
pp. 1-9 ◽  
Author(s):  
Siyuan Zhao ◽  
Zhiwei Xu ◽  
Limin Liu ◽  
Mengjie Guo ◽  
Jing Yun
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Word Order ◽  
Text Analysis ◽  
Important Application ◽  
Detection Mechanism ◽  
Short Text

Convolutional neural network (CNN) has revolutionized the field of natural language processing, which is considerably efficient at semantics analysis that underlies difficult natural language processing problems in a variety of domains. The deceptive opinion detection is an important application of the existing CNN models. The detection mechanism based on CNN models has better self-adaptability and can effectively identify all kinds of deceptive opinions. Online opinions are quite short, varying in their types and content. In order to effectively identify deceptive opinions, we need to comprehensively study the characteristics of deceptive opinions and explore novel characteristics besides the textual semantics and emotional polarity that have been widely used in text analysis. In this paper, we optimize the convolutional neural network model by embedding the word order characteristics in its convolution layer and pooling layer, which makes convolutional neural network more suitable for short text classification and deceptive opinions detection. The TensorFlow-based experiments demonstrate that the proposed detection mechanism achieves more accurate deceptive opinion detection results.

High accuracy offering attention mechanisms based deep learning approach using CNN/bi-LSTM for sentiment analysis

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Venkateswara Rao Kota ◽  
Shyamala Devi Munisamy
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Attention Mechanism ◽  
Supervised Machine Learning ◽  
Method Performance ◽  
Content Type

PurposeNeural network (NN)-based deep learning (DL) approach is considered for sentiment analysis (SA) by incorporating convolutional neural network (CNN), bi-directional long short-term memory (Bi-LSTM) and attention methods. Unlike the conventional supervised machine learning natural language processing algorithms, the authors have used unsupervised deep learning algorithms.Design/methodology/approachThe method presented for sentiment analysis is designed using CNN, Bi-LSTM and the attention mechanism. Word2vec word embedding is used for natural language processing (NLP). The discussed approach is designed for sentence-level SA which consists of one embedding layer, two convolutional layers with max-pooling, one LSTM layer and two fully connected (FC) layers. Overall the system training time is 30 min.FindingsThe method performance is analyzed using metrics like precision, recall, F1 score, and accuracy. CNN is helped to reduce the complexity and Bi-LSTM is helped to process the long sequence input text.Originality/valueThe attention mechanism is adopted to decide the significance of every hidden state and give a weighted sum of all the features fed as input.

scholarly journals Sentiment Analysis of Multilingual Tweets based on Natural Language Processing (NLP)

2021 ◽  
Vol 10 (4) ◽  
pp. 0-0
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
Research Work ◽  
The Social ◽  
Machine Learning Approach ◽  
Simple Neural Network

Multilingual Sentiment analysis plays an important role in a country like India with many languages as the style of expression varies in different languages. The Indian people speak in total 22 different languages and with the help of Google Indic keyboard people can express their sentiments i.e reviews about anything in the social media in their native language from individual smart phones. It has been found that machine learning approach has overcome the limitations of other approaches. In this paper, a detailed study has been carried out based on Natural Language Processing (NLP) using Simple Neural Network (SNN) ,Convolutional Neural Network(CNN), and Long Short Term Memory (LSTM)Neural Network followed by another amalgamated model adding a CNN layer on top of the LSTM without worrying about versatility of multilingualism. Around 4000 samples of reviews in English, Hindi and in Bengali languages are considered to generate outputs for the above models and analyzed. The experimental results on these realistic reviews are found to be effective for further research work.

scholarly journals Sentiment Analysis of Multilingual Tweets Based on Natural Language Processing (NLP)

2021 ◽  
Vol 10 (4) ◽  
pp. 1-12
Author(s):  
Abhijit Bera ◽  
Mrinal Kanti Ghose ◽  
Dibyendu Kumar Pal
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
Research Work ◽  
The Social ◽  
Machine Learning Approach ◽  
Simple Neural Network

Multilingual Sentiment analysis plays an important role in a country like India with many languages as the style of expression varies in different languages. The Indian people speak in total 22 different languages and with the help of Google Indic keyboard people can express their sentiments i.e reviews about anything in the social media in their native language from individual smart phones. It has been found that machine learning approach has overcome the limitations of other approaches. In this paper, a detailed study has been carried out based on Natural Language Processing (NLP) using Simple Neural Network (SNN) ,Convolutional Neural Network(CNN), and Long Short Term Memory (LSTM)Neural Network followed by another amalgamated model adding a CNN layer on top of the LSTM without worrying about versatility of multilingualism. Around 4000 samples of reviews in English, Hindi and in Bengali languages are considered to generate outputs for the above models and analyzed. The experimental results on these realistic reviews are found to be effective for further research work.

Speech-Act Classification Using Convolutional Neural Network and Word Embedding

2018 ◽  
Vol 27 (06) ◽  
pp. 1850026
Author(s):  
Kyoungman Bae ◽  
Youngjoong Ko
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Speech Acts ◽  
Speech Act ◽  
Distributed Representation ◽  
Learning Techniques

The application of deep learning techniques in natural language processing tasks has been increased in recent years. Many studies have used the deep learning techniques to obtain a distributed representation of features. In particular, the convolutional neural network (CNN) with the distributed representation have subsequently been shown to be effective for the natural language processing tasks. This paper presents how to apply the CNN to speech-act classification. Then we analyze the experimental results on two issues, how to solve two problems about sparse speech-acts in train data and out of vocabulary, and how to utilize the advantages of CNN in the speech-act classification. As a result, we obtain the significant improved performances when CNN is applied to the speech-act classification.

Sign in / Sign up

Export Citation Format

Share Document