Evaluation of Impact of Neural Networks in Text Classification

One of the most trending and major areas of research in Natural Language Processing (NLP) is the classification of text data. This necessarily means that the category that the text belongs to is determined by the content of the text. Various algorithms such as Recurrent Neural Network along with its variation which is Long Short-Term Memory, Hierarchical Attention Networks and also Convolutional Neural Network have been used to analyse how the context of the text can be determined from the text data which in available in terms of datasets. These algorithms each have a special characteristic of their own. While Recurrent Neural Network maintains the structural sequence of the contexts, the Convolutional Neural Network manages to obtain the n-gram feature and the Hierarchical Attention Network manages the hierarchy of the documents or data. The above said algorithms have been implemented on the British Broadcasting Corporation News datasets. Various parameters such as recall, precision, accuracy etc. have been considered along with standards such as F1-score, confusion matrix etc. to deduce the impact.

Download Full-text

Real-time event detection using recurrent neural network in social sensors

International Journal of Distributed Sensor Networks ◽

10.1177/1550147719856492 ◽

2019 ◽

Vol 15 (6) ◽

pp. 155014771985649 ◽

Cited By ~ 2

Author(s):

Van Quan Nguyen ◽

Tien Nguyen Anh ◽

Hyung-Jeong Yang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Language Processing ◽

Recurrent Neural Network ◽

Event Detection ◽

Word Embedding ◽

Series Data ◽

Accuracy Score ◽

Data Set ◽

Size Limitation

We proposed an approach for temporal event detection using deep learning and multi-embedding on a set of text data from social media. First, a convolutional neural network augmented with multiple word-embedding architectures is used as a text classifier for the pre-processing of the input textual data. Second, an event detection model using a recurrent neural network is employed to learn time series data features by extracting temporal information. Recently, convolutional neural networks have been used in natural language processing problems and have obtained excellent results as performing on available embedding vector. In this article, word-embedding features at the embedding layer are combined and fed to convolutional neural network. The proposed method shows no size limitation, supplementation of more embeddings than standard multichannel based approaches, and obtained similar performance (accuracy score) on some benchmark data sets, especially in an imbalanced data set. For event detection, a long short-term memory network is used as a predictor that learns higher level temporal features so as to predict future values. An error distribution estimation model is built to calculate the anomaly score of observation. Events are detected using a window-based method on the anomaly scores.

Download Full-text

Classification of Analyzed Text in Speech Recognition Using RNN-LSTM in Comparison with Convolutional Neural Network to Improve Precision for Identification of Keywords

Revista Gestão Inovação e Tecnologias ◽

10.47059/revistageintec.v11i2.1739 ◽

2021 ◽

Vol 11 (2) ◽

pp. 1097-1108

Author(s):

Bathaloori Reddy Prasad

Keyword(s):

Neural Network ◽

Speech Recognition ◽

Convolutional Neural Network ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Statistical Significance ◽

Language Translation ◽

Accuracy And Precision ◽

Long Short Term Memory

Aim: Text classification is a method to classify the features from language translation in speech recognition from English to Telugu using a recurrent neural network- long short term memory (RNN-LSTM) comparison with convolutional neural network (CNN). Materials and Methods: Accuracy and precision are performed with dataset alexa and english-telugu of size 8166 sentences. Classification of language translation is performed by the recurrent neural network where a number of the samples (N=62) and convolutional neural network were a number of samples (N=62) techniques, the algorithm RNN implies speech recognition that can be compared with convolutional is the second technique. Results and Discussion: RNN-LSTM from the dataset speech recognition, feature Telugu_id produce accuracy 93% and precision 68.04% which can be comparatively higher than CNN accuracy 66.11%, precision 61.90%. It shows a statistical significance as 0.007 from Independent Sample T-test. Conclusion: The RNN-LSTM performs better in finding accuracy and precision when compared to CNN.

Download Full-text

A Hybrid Deep Learning Model to Predict the Impact of COVID-19 on Mental Health form Social Media Big Data

10.20944/preprints202106.0654.v1 ◽

2021 ◽

Author(s):

Tapotosh Ghosh ◽

Md. Hasan Al Banna ◽

Md. Jaber Al Nahian ◽

Kazi Abu Taher ◽

M Shamim Kaiser ◽

...

Keyword(s):

Neural Network ◽

Mental Health ◽

Social Media ◽

Big Data ◽

Language Processing ◽

Short Term Memory ◽

Daily Basis ◽

Significant Rise ◽

Special Focus ◽

The Impact

The novel coronavirus disease (COVID-19) pandemic is provoking a prevalent consequence on mental health because of less interaction among people, economic collapse, negativity, fear of losing jobs, and death of the near and dear ones. To express their mental state, people often are using social media as one of the preferred means. Due to reduced outdoor activities, people are spending more time on social media than usual and expressing their emotion of anxiety, fear, and depression. On a daily basis, about 2.5 quintillion bytes of data are generated on social media, analyzing this big data can become an excellent means to evaluate the effect of COVID-19 on mental health. In this work, we have analyzed data from Twitter microblog (tweets) to find out the effect of COVID-19 on peoples mental health with a special focus on depression. We propose a novel pipeline, based on recurrent neural network (in the form of long-short term memory or LSTM) and convolutional neural network, capable of identifying depressive tweets with an accuracy of 99.42%. Preprocessed using various natural language processing techniques, the aim was to find out depressive emotion from these tweets. Analyzing over 571 thousand tweets posted between October 2019 and May 2020 by 482 users, a significant rise in depressing tweets was observed between February and May of 2020, which indicates as an impact of the long ongoing COVID-19 pandemic situation.

Download Full-text

Application of deep learning models for aspect based sentiment analysis.

Bulletin of Bryansk state technical university ◽

10.12737/22917 ◽

2015 ◽

Vol 2015 (3) ◽

pp. 117-126

Author(s):

Дмитрий Будыльский ◽

Dmitriy Budylskiy ◽

Александр Подвесовский ◽

Aleksandr Podvesovskiy

Keyword(s):

Neural Network ◽

Deep Learning ◽

Natural Language Processing ◽

Convolutional Neural Network ◽

Language Processing ◽

Recurrent Neural Network ◽

Russian Text ◽

Learning Models ◽

Actual Problem ◽

Good Efficiency

This paper describes actual problem of sentiment based aspect analysis and four deep learning models: convolutional neural network, recurrent neural network, GRU and LSTM networks. We evaluated these models on Russian text dataset from SentiRuEval-2015. Results show good efficiency and high potential for further natural language processing applications.

Download Full-text

Development of an Early Alert System for an Additional Wave of COVID-19 Cases Using a Recurrent Neural Network with Long Short-Term Memory

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18147376 ◽

2021 ◽

Vol 18 (14) ◽

pp. 7376

Author(s):

Finn Stevenson ◽

Kentaro Hayasi ◽

Nicola Luigi Bragazzi ◽

Jude Dzevela Kong ◽

Ali Asgary ◽

...

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Prediction Models ◽

Short Term Memory ◽

Alert System ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

The Impact ◽

Early Alert

The impact of the still ongoing “Coronavirus Disease 2019” (COVID-19) pandemic has been and is still vast, affecting not only global human health and stretching healthcare facilities, but also profoundly disrupting societal and economic systems worldwide. The nature of the way the virus spreads causes cases to come in further recurring waves. This is due a complex array of biological, societal and environmental factors, including the novel nature of the emerging pathogen. Other parameters explaining the epidemic trend consisting of recurring waves are logistic–organizational challenges in the implementation of the vaccine roll-out, scarcity of doses and human resources, seasonality, meteorological drivers, and community heterogeneity, as well as cycles of strengthening and easing/lifting of the mitigation interventions. Therefore, it is crucial to be able to have an early alert system to identify when another wave of cases is about to occur. The availability of a variety of newly developed indicators allows for the exploration of multi-feature prediction models for case data. Ten indicators were selected as features for our prediction model. The model chosen is a Recurrent Neural Network with Long Short-Term Memory. This paper documents the development of an early alert/detection system that functions by predicting future daily confirmed cases based on a series of features that include mobility and stringency indices, and epidemiological parameters. The model is trained on the intermittent period in between the first and the second wave, in all of the South African provinces.

Download Full-text

Klasifikasi Teks Hadis Bukhari Terjemahan Indonesia Menggunakan Recurrent Convolutional Neural Network (CRNN)

Jurnal Teknologi Informasi dan Ilmu Komputer ◽

10.25126/jtiik.2021853750 ◽

2021 ◽

Vol 8 (5) ◽

pp. 907

Author(s):

Muhammad Yuslan Abu Bakar ◽

Adiwijaya Adiwijaya

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Recurrent Neural Network ◽

Recurrent Neural Networks ◽

Spatial Information ◽

Contextual Information ◽

Computing Time ◽

Sequential Data ◽

Text Data

Hadis merupakan sumber hukum dan pedoman kedua bagi umat Islam setelah Al-Qur’an dan banyak sekali hadis yang telah diriwayatkan oleh para ahli hadis selama ini. Penelitian ini membangun sebuah sistem yang dapat melakukan klasifikasi teks hadis Bukhari terjemahan berbahasa Indonesia. Topik ini diangkat untuk memenuhi kebutuhan umat Islam dalam mengetahui apa saja informasi mengenai anjuran dan larangan yang terdapat dalam suatu hadis. Klasifikasi teks memiliki tantangannya tersendiri terkait dengan jumlah fitur yang sangat banyak (dimensi sangat besar) sehingga waktu komputasi menjadi besar dan mengakibatkan sulitnya mendapatkan hasil yang optimal. Pada penelitian ini, digunakan salah satu metode hibrid dalam dunia deep learning dengan menggabungkan Convolutional Neural Network dan Recurrent Neural Network, yaitu Convolutional Recurrent Neural Network (CRNN). Convolutional Neural Network dipilih sebagai metode seleksi dan reduksi data dikarenakan dapat menangkap informasi spasial yang saling berhubungan dan berkorelasi. Sementara Recurrent Neural Network digunakan sebagai metode klasifikasi dengan mengusung kemampuan utamanya yaitu dapat menangkap informasi kontekstual yang sangat panjang khususnya pada data sekuens seperti data teks dengan mengandalkan ‘memori’ yang dimilikinya. Hasil penelitian menyajikan beberapa hasil klasifikasi menggunakan deep learning, dimana hasil akurasi terbaik diberikan oleh Convolutional Recurrent Neural Network (CRNN), yakni sebesar 80.79%. Abstract Hadith is a source of law and guidance for Muslims after the Qur'an and many hadith have been narrated by hadith experts so far. This research builds a system that can classify Bukhari hadith in Indonesian translations. This topic was raised to meet the needs of Muslims in knowing what information about the suggestions and prohibitions that exist in a hadith. Text classification has its own challenges related to several features whose dimensions are very large so that it increases computing time and causes difficulties in getting optimal results. This research uses a hybrid method in deep learning by combining a Convolutional Neural Network and a Recurrent Neural Network, namely Convolutional Recurrent Neural Network (CRNN). Convolutional Neural Network was chosen as a method of selecting and reducing data that can be determined as spatial information that is interrelated and correlated. While Recurrent Neural Networks are used as a classification method by carrying out capabilities that can be used as very long contextual information specifically on sequential data such as text data by relying on the ‘memory’ it has. This research presents several classification results using deep learning, where the best accuracy results are given by the Convolutional Recurrent Neural Network (CRNN), which is equal to 80.79%.

Download Full-text

Indonesian news classification using convolutional neural network

Indonesian Journal of Electrical Engineering and Computer Science ◽

10.11591/ijeecs.v19.i2.pp1000-1009 ◽

2020 ◽

Vol 19 (2) ◽

pp. 1000

Author(s):

Muhammad Ali Ramdhani ◽

Dian Sa’adillah Maylawati ◽

Teddy Mantoro

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Language Processing ◽

Research Area ◽

Training Data ◽

Text Data ◽

Testing Data ◽

Headline News ◽

Language Characteristics ◽

Area Data

Every language has unique characteristics, structures, and grammar. Thus, different styles will have different processes and result in processed in Natural Language Processing (NLP) research area. In the current NLP research area, Data Mining (DM) or Machine Learning (ML) technique is popular, especially for Deep Learning (DL) method. This research aims to classify text data in the Indonesian language using Convolutional Neural Network (CNN) as one of the DL algorithms. The CNN algorithm used modified following the Indonesian language characteristics. Thereby, in the text pre-processing phase, stopword removal and stemming are particularly suitable for the Indonesian language. The experiment conducted using 472 Indonesian News text data from various sources with four categories: ‘hiburan’ (entertainment), ‘olahraga’ (sport), ‘tajuk utama’ (headline news), and ‘teknologi’ (technology). Based on the experiment and evaluation using 377 training data and 95 testing data, producing five models with ten epoch for each model, CNN has the best percentage of accuracy around 90,74% and loss value around 29,05% for 300 hidden layers in classifying the Indonesian News data.

Download Full-text

Sentimental Classification of News Headlines using Recurrent Neural Network

International Journal of Innovative Technology and Exploring Engineering - Special Issue ◽

10.35940/ijitee.f3573.049620 ◽

2020 ◽

Vol 9 (6) ◽

pp. 207-210

Keyword(s):

Neural Network ◽

Language Processing ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Attention Mechanism ◽

Machine Learning Techniques ◽

Learning Techniques ◽

Negative Comments ◽

News Headlines

Sentiment analysis combines the natural language processing task and analysis of the text that attempts to predict the sentiment of the text in terms of positive and negative comments. Nowadays, the tremendous volume of news originated via different webpages, and it is feasible to determine the opinion of particular news. This work tries to judge completely various machine learning techniques to classify the view of the news headlines. In this project, propose the appliance of Recurrent Neural Network with Long Short Term Memory Unit(LSTM), focus on seeking out similar news headlines, and predict the opinion of news headlines from numerous sources. The main objective is to classify the sentiment of news headlines from various sources using a recurrent neural network. Interestingly, the proposed attention mechanism performs better than the more complex attention mechanism on a held-out set of articles.

Download Full-text

Malicious URL Detection Algorithm Based on Multi Neural Network Series

CONVERTER ◽

10.17762/converter.209 ◽

2021 ◽

pp. 579-590

Author(s):

Weirong Xiu

Keyword(s):

Neural Network ◽

Natural Language Processing ◽

Natural Language ◽

Convolutional Neural Network ◽

Language Processing ◽

Recurrent Neural Network ◽

Detection Algorithm ◽

Attention Mechanism ◽

Global Features ◽

Multi Neural Network

Convolutional neural network based on attention mechanism and a bidirectional independent recurrent neural network tandem joint algorithm (CATIR) are proposed. In natural language processing related technologies, word vector features are extracted based on URLs, and the extracted URL information features and host information features are merged. The proposed CATIR algorithm uses CNN (Convolutional Neural Network) to obtain the deep local features in the data, uses the Attention mechanism to adjust the weights, and uses IndRNN (Independent Recurrent Neural Network) to obtain the global features in the data. The experimental results shows that the CATIR algorithm has significantly improved the accuracy of malicious URL detection based on traditional algorithms to 96.9%.

Download Full-text

Neural Machine Translation Using Sequence Modeling

International Journal for Research in Applied Science and Engineering Technology ◽

10.22214/ijraset.2021.37687 ◽

2021 ◽

Vol 9 (8) ◽

pp. 2153-2169

Author(s):

N Revathi

Keyword(s):

Neural Network ◽

Machine Translation ◽

Language Processing ◽

Recurrent Neural Network ◽

Deep Neural Network ◽

Short Term Memory ◽

Short Term ◽

Neural Machine Translation ◽

Term Memory ◽

Sequence Modeling

Abstract: Language is a main mode of communication, and translation is a critical tool for understanding information in a foreign language. Without the help of human translators, machine translation allows users to absorb unfamiliar linguistic material. The main goal of this project is to create a practical language translation from English to Hindi. Given its relevance and potential in the English-Hindi translation, machine translation is an efficient way to turn content into a new language without employing people. Among all available translation machines, Neural Machine Translation (NMT) is one of the most efficient ways. So, in this case, we're employing Sequence to Sequence Modeling, which includes the Recurrent Neural Network (RNN), Long and Short Term Memory (LSTM), and Encoder-Decoder methods. Deep Neural Network (DNN) comprehension and principles of deep learning, i.e. machine translation, are disclosed in the field of Natural Language Processing (NLP). In machine reclining techniques, DNN plays a crucial role. Keywords: Sequence to Sequence, Encoder-Decoder, Recurrent Neural Network, Long & Short term Memory, Deep Neural Network.

Download Full-text