Deep Learning based Semantic Similarity Detection using Text Data

Similarity detection in the text is the main task for a number of Natural Language Processing (NLP) applications. As textual data is comparatively large in quantity and huge in volume than the numeric data, therefore measuring textual similarity is one of the important problems. Most of the similarity detection algorithms are based upon word to word matching, sentence/paragraph matching, and matching of the whole document. In this research, a novel approach is proposed using deep learning models, combining Long Short Term Memory network (LSTM) with Convolutional Neural Network (CNN) for measuring semantics similarity between two questions. The proposed model takes sentence pairs as input to measure the similarity between them. The model is tested on publicly available Quora’s dataset. The model in comparison to the existing techniques gave 87.50 % accuracy which is better than the previous approaches.

Download Full-text

A Hybrid Deep Learning Model for Predicting Protein Hydroxylation Sites

International Journal of Molecular Sciences ◽

10.3390/ijms19092817 ◽

2018 ◽

Vol 19 (9) ◽

pp. 2817 ◽

Cited By ~ 9

Author(s):

Haixia Long ◽

Bo Liao ◽

Xingyu Xu ◽

Jialiang Yang

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Learning Model ◽

New Drugs ◽

Post Translational Modifications ◽

Novel Approach ◽

Benchmark Datasets ◽

Memory Network ◽

Scoring Matrix ◽

Deep Learning Model

Protein hydroxylation is one type of post-translational modifications (PTMs) playing critical roles in human diseases. It is known that protein sequence contains many uncharacterized residues of proline and lysine. The question that needs to be answered is: which residue can be hydroxylated, and which one cannot. The answer will not only help understand the mechanism of hydroxylation but can also benefit the development of new drugs. In this paper, we proposed a novel approach for predicting hydroxylation using a hybrid deep learning model integrating the convolutional neural network (CNN) and long short-term memory network (LSTM). We employed a pseudo amino acid composition (PseAAC) method to construct valid benchmark datasets based on a sliding window strategy and used the position-specific scoring matrix (PSSM) to represent samples as inputs to the deep learning model. In addition, we compared our method with popular predictors including CNN, iHyd-PseAAC, and iHyd-PseCp. The results for 5-fold cross-validations all demonstrated that our method significantly outperforms the other methods in prediction accuracy.

Download Full-text

Arabic Sentiment Analysis (ASA) Using Deep Learning Approach

Journal of Engineering ◽

10.31026/j.eng.2020.06.07 ◽

2020 ◽

Vol 26 (6) ◽

pp. 85-93

Author(s):

Abdulhakeem Qusay Al-Bayati ◽

Ahmed S. Al-Araji ◽

Saman Hameed Ameen

Keyword(s):

Deep Learning ◽

Sentiment Analysis ◽

Language Processing ◽

Web Sites ◽

Short Term Memory ◽

Morphological Structure ◽

Arabic Language ◽

Feature Representation ◽

Main Task ◽

Arabic Sentiment Analysis

Sentiment analysis is one of the major fields in natural language processing whose main task is to extract sentiments, opinions, attitudes, and emotions from a subjective text. And for its importance in decision making and in people's trust with reviews on web sites, there are many academic researches to address sentiment analysis problems. Deep Learning (DL) is a powerful Machine Learning (ML) technique that has emerged with its ability of feature representation and differentiating data, leading to state-of-the-art prediction results. In recent years, DL has been widely used in sentiment analysis, however, there is scarce in its implementation in the Arabic language field. Most of the previous researches address other languages like English. The proposed model tackles Arabic Sentiment Analysis (ASA) by using a DL approach. ASA is a challenging field where Arabic language has a rich morphological structure more than other languages. In this work, Long Short-Term Memory (LSTM) as a deep neural network has been used for training the model combined with word embedding as a first hidden layer for features extracting. The results show an accuracy of about 82% is achievable using DL method.

Download Full-text

A Tweet Sentiment Classification Approach Using a Hybrid Stacked Ensemble Technique

Information ◽

10.3390/info12090374 ◽

2021 ◽

Vol 12 (9) ◽

pp. 374

Author(s):

Babacar Gaye ◽

Dezheng Zhang ◽

Aziguli Wulamu

Keyword(s):

Machine Learning ◽

Logistic Regression ◽

Deep Learning ◽

Sentiment Analysis ◽

Language Processing ◽

Short Term Memory ◽

State Of The Art ◽

Accuracy Score ◽

Learning Models ◽

Proposed Model

With the extensive availability of social media platforms, Twitter has become a significant tool for the acquisition of peoples’ views, opinions, attitudes, and emotions towards certain entities. Within this frame of reference, sentiment analysis of tweets has become one of the most fascinating research areas in the field of natural language processing. A variety of techniques have been devised for sentiment analysis, but there is still room for improvement where the accuracy and efficacy of the system are concerned. This study proposes a novel approach that exploits the advantages of the lexical dictionary, machine learning, and deep learning classifiers. We classified the tweets based on the sentiments extracted by TextBlob using a stacked ensemble of three long short-term memory (LSTM) as base classifiers and logistic regression (LR) as a meta classifier. The proposed model proved to be effective and time-saving since it does not require feature extraction, as LSTM extracts features without any human intervention. We also compared our proposed approach with conventional machine learning models such as logistic regression, AdaBoost, and random forest. We also included state-of-the-art deep learning models in comparison with the proposed model. Experiments were conducted on the sentiment140 dataset and were evaluated in terms of accuracy, precision, recall, and F1 Score. Empirical results showed that our proposed approach manifested state-of-the-art results by achieving an accuracy score of 99%.

Download Full-text

Deep-Kcr: accurate detection of lysine crotonylation sites using deep learning method

Briefings in Bioinformatics ◽

10.1093/bib/bbaa255 ◽

2020 ◽

Author(s):

Hao Lv ◽

Fu-Ying Dao ◽

Zheng-Xing Guan ◽

Hui Yang ◽

Yan-Wen Li ◽

...

Keyword(s):

Deep Learning ◽

Large Scale ◽

Short Term Memory ◽

Information Gain ◽

Independent Set ◽

Cost Effective ◽

Cellular Regulation ◽

Proposed Model ◽

Experimental Approaches ◽

Memory Network

Abstract As a newly discovered protein posttranslational modification, histone lysine crotonylation (Kcr) involved in cellular regulation and human diseases. Various proteomics technologies have been developed to detect Kcr sites. However, experimental approaches for identifying Kcr sites are often time-consuming and labor-intensive, which is difficult to widely popularize in large-scale species. Computational approaches are cost-effective and can be used in a high-throughput manner to generate relatively precise identification. In this study, we develop a deep learning-based method termed as Deep-Kcr for Kcr sites prediction by combining sequence-based features, physicochemical property-based features and numerical space-derived information with information gain feature selection. We investigate the performances of convolutional neural network (CNN) and five commonly used classifiers (long short-term memory network, random forest, LogitBoost, naive Bayes and logistic regression) using 10-fold cross-validation and independent set test. Results show that CNN could always display the best performance with high computational efficiency on large dataset. We also compare the Deep-Kcr with other existing tools to demonstrate the excellent predictive power and robustness of our method. Based on the proposed model, a webserver called Deep-Kcr was established and is freely accessible at http://lin-group.cn/server/Deep-Kcr.

Download Full-text

Malicious Text Identification: Deep Learning from Public Comments and Emails

Information ◽

10.3390/info11060312 ◽

2020 ◽

Vol 11 (6) ◽

pp. 312 ◽

Cited By ~ 1

Author(s):

Asma Baccouche ◽

Sadaf Ahmed ◽

Daniel Sierra-Sosa ◽

Adel Elmaghraby

Keyword(s):

Social Media ◽

Deep Learning ◽

Language Processing ◽

Short Term Memory ◽

Good Alternative ◽

Classification Problems ◽

Short Term ◽

Independent Dataset ◽

Proposed Model ◽

Long Short Term Memory

Identifying internet spam has been a challenging problem for decades. Several solutions have succeeded to detect spam comments in social media or fraudulent emails. However, an adequate strategy for filtering messages is difficult to achieve, as these messages resemble real communications. From the Natural Language Processing (NLP) perspective, Deep Learning models are a good alternative for classifying text after being preprocessed. In particular, Long Short-Term Memory (LSTM) networks are one of the models that perform well for the binary and multi-label text classification problems. In this paper, an approach merging two different data sources, one intended for Spam in social media posts and the other for Fraud classification in emails, is presented. We designed a multi-label LSTM model and trained it on the joint datasets including text with common bigrams, extracted from each independent dataset. The experiment results show that our proposed model is capable of identifying malicious text regardless of the source. The LSTM model trained with the merged dataset outperforms the models trained independently on each dataset.

Download Full-text

Deep learning based algorithm (ConvLSTM) for Copy Move Forgery Detection

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-201192 ◽

2021 ◽

Vol 40 (3) ◽

pp. 4385-4405

Author(s):

Mohamed A. Elaskily ◽

Monagi H. Alkinani ◽

Ahmed Sedik ◽

Mohamed M. Dessouky

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Image Features ◽

Testing Time ◽

Forgery Detection ◽

Detection Algorithms ◽

Proposed Model ◽

Important Challenge ◽

Copy Move Forgery Detection ◽

And Coping

Protecting information from manipulation is important challenge in current days. Digital images are one of the most popular information representation. Images could be used in several fields such as military, social media, security purposes, intelligence fields, evidences in courts, and newspapers. Digital image forgeries mean adding unusual patterns to the original images that cause a heterogeneity manner in form of image properties. Copy move forgery is one of the hardest types of image forgeries to be detected. It is happened by duplicating part or section of the image then adding again in the image itself but in another location. Forgery detection algorithms are used in image security when the original content is not available. This paper illustrates a new approach for Copy Move Forgery Detection (CMFD) built basically on deep learning. The proposed model is depending on applying (Convolution Neural Network) CNN in addition to Convolutional Long Short-Term Memory (CovLSTM) networks. This method extracts image features by a sequence number of Convolutions (CNVs) layers, ConvLSTM layers, and pooling layers then matching features and detecting copy move forgery. This model had been applied to four aboveboard available databases: MICC-F220, MICC-F2000, MICC-F600, and SATs-130. Moreover, datasets have been combined to build new datasets for all purposes of generalization testing and coping with an over-fitting problem. In addition, the results of applying ConvLSTM model only have been added to show the differences in performance between using hybrid ConvLSTM and CNN compared with using CNN only. The proposed algorithm, when using number of epoch’s equal 100, gives high accuracy reached to 100% for some datasets with lowest Testing Time (TT) time nearly 1 second for some datasets when compared with the different previous algorithms.

Download Full-text

A Hybrid CNN-LSTM: A Deep Learning Approach for Consumer Sentiment Analysis Using Qualitative User-Generated Contents

ACM Transactions on Asian and Low-Resource Language Information Processing ◽

10.1145/3457206 ◽

2021 ◽

Vol 20 (5) ◽

pp. 1-15

Author(s):

Praphula Kumar Jain ◽

Vijayalakshmi Saravanan ◽

Rajendra Pamula

Keyword(s):

Deep Learning ◽

Sentiment Analysis ◽

Language Processing ◽

Short Term Memory ◽

Online Reviews ◽

Web Content ◽

Proposed Model ◽

Social Media Platforms ◽

Information And Communication ◽

Better Than

With the fastest growth of information and communication technology (ICT), the availability of web content on social media platforms is increasing day by day. Sentiment analysis from online reviews drawing researchers’ attention from various organizations such as academics, government, and private industries. Sentiment analysis has been a hot research topic in Machine Learning (ML) and Natural Language Processing (NLP). Currently, Deep Learning (DL) techniques are implemented in sentiment analysis to get excellent results. This study proposed a hybrid convolutional neural network-long short-term memory (CNN-LSTM) model for sentiment analysis. Our proposed model is being applied with dropout, max pooling, and batch normalization to get results. Experimental analysis carried out on Airlinequality and Twitter airline sentiment datasets. We employed the Keras word embedding approach, which converts texts into vectors of numeric values, where similar words have small vector distances between them. We calculated various parameters, such as accuracy, precision, recall, and F1-measure, to measure the model’s performance. These parameters for the proposed model are better than the classical ML models in sentiment analysis. Our results analysis demonstrates that the proposed model outperforms with 91.3% accuracy in sentiment analysis.

Download Full-text

A COMBINED DEEP LEARNING MODEL FOR PERSIAN SENTIMENT ANALYSIS

IIUM Engineering Journal ◽

10.31436/iiumej.v20i1.1036 ◽

2019 ◽

Vol 20 (1) ◽

pp. 129-139 ◽

Cited By ~ 2

Author(s):

Zahra Bokaee Nezhad ◽

Mohammad Ali Deihimi

Keyword(s):

Deep Learning ◽

Natural Language Processing ◽

Sentiment Analysis ◽

Language Processing ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Proposed Model ◽

Long Short Term Memory ◽

Deep Learning Model

With increasing members in social media sites today, people tend to share their views about everything online. It is a convenient way to convey their messages to end users on a specific subject. Sentiment Analysis is a subfield of Natural Language Processing (NLP) that refers to the identification of users’ opinions toward specific topics. It is used in several fields such as marketing, customer services, etc. However, limited works have been done on Persian Sentiment Analysis. On the other hand, deep learning has recently become popular because of its successful role in several Natural Language Processing tasks. The objective of this paper is to propose a novel hybrid deep learning architecture for Persian Sentiment Analysis. According to the proposed model, local features are extracted by Convolutional Neural Networks (CNN) and long-term dependencies are learned by Long Short Term Memory (LSTM). Therefore, the model can harness both CNN's and LSTM's abilities. Furthermore, Word2vec is used for word representation as an unsupervised learning step. To the best of our knowledge, this is the first attempt where a hybrid deep learning model is used for Persian Sentiment Analysis. We evaluate the model on a Persian dataset that is introduced in this study. The experimental results show the effectiveness of the proposed model with an accuracy of 85%. ABSTRAK: Hari ini dengan ahli yang semakin meningkat di laman media sosial, orang cenderung untuk berkongsi pandangan mereka tentang segala-galanya dalam talian. Ini adalah cara mudah untuk menyampaikan mesej mereka kepada pengguna akhir mengenai subjek tertentu. Analisis Sentimen adalah subfield Pemprosesan Bahasa Semula Jadi yang merujuk kepada pengenalan pendapat pengguna ke arah topik tertentu. Ia digunakan dalam beberapa bidang seperti pemasaran, perkhidmatan pelanggan, dan sebagainya. Walau bagaimanapun, kerja-kerja terhad telah dilakukan ke atas Analisis Sentimen Parsi. Sebaliknya, pembelajaran mendalam baru menjadi popular kerana peranannya yang berjaya dalam beberapa tugas Pemprosesan Bahasa Asli (NLP). Objektif makalah ini adalah mencadangkan senibina pembelajaran hibrid yang baru dalam Analisis Sentimen Parsi. Menurut model yang dicadangkan, ciri-ciri tempatan ditangkap oleh Rangkaian Neural Convolutional (CNN) dan ketergantungan jangka panjang dipelajari oleh Long Short Term Memory (LSTM). Oleh itu, model boleh memanfaatkan kebolehan CNN dan LSTM. Selain itu, Word2vec digunakan untuk perwakilan perkataan sebagai langkah pembelajaran tanpa pengawasan. Untuk pengetahuan yang terbaik, ini adalah percubaan pertama di mana model pembelajaran mendalam hibrid digunakan untuk Analisis Sentimen Persia. Kami menilai model pada dataset Persia yang memperkenalkan dalam kajian ini. Keputusan eksperimen menunjukkan keberkesanan model yang dicadangkan dengan ketepatan 85%.

Download Full-text

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

International Journal of Sensors Wireless Communications and Control ◽

10.2174/2210327908666180727123657 ◽

2019 ◽

Vol 9 (1) ◽

pp. 16-31

Author(s):

Kyungkoo Jun

Keyword(s):

Fourier Transform ◽

Deep Learning ◽

Short Term Memory ◽

Window Size ◽

Sensor Data ◽

Data Sets ◽

Data Set ◽

Proposed Model ◽

Testing Data ◽

Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

Download Full-text

Adaptive particle swarm optimization algorithm based long short-term memory networks for sentiment analysis

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-201644 ◽

2021 ◽

pp. 1-17

Author(s):

J. Shobana ◽

M. Murali

Keyword(s):

Particle Swarm Optimization ◽

Sentiment Analysis ◽

Language Processing ◽

Short Term Memory ◽

Contextual Information ◽

Particle Swarm ◽

Pso Algorithm ◽

Swarm Optimization ◽

Adaptive Particle Swarm Optimization ◽

Proposed Model

Text Sentiment analysis is the process of predicting whether a segment of text has opinionated or objective content and analyzing the polarity of the text’s sentiment. Understanding the needs and behavior of the target customer plays a vital role in the success of the business so the sentiment analysis process would help the marketer to improve the quality of the product as well as a shopper to buy the correct product. Due to its automatic learning capability, deep learning is the current research interest in Natural language processing. Skip-gram architecture is used in the proposed model for better extraction of the semantic relationships as well as contextual information of words. However, the main contribution of this work is Adaptive Particle Swarm Optimization (APSO) algorithm based LSTM for sentiment analysis. LSTM is used in the proposed model for understanding complex patterns in textual data. To improve the performance of the LSTM, weight parameters are enhanced by presenting the Adaptive PSO algorithm. Opposition based learning (OBL) method combined with PSO algorithm becomes the Adaptive Particle Swarm Optimization (APSO) classifier which assists LSTM in selecting optimal weight for the environment in less number of iterations. So APSO - LSTM ‘s ability in adjusting the attributes such as optimal weights and learning rates combined with the good hyper parameter choices leads to improved accuracy and reduces losses. Extensive experiments were conducted on four datasets proved that our proposed APSO-LSTM model secured higher accuracy over the classical methods such as traditional LSTM, ANN, and SVM. According to simulation results, the proposed model is outperforming other existing models.

Download Full-text