CL-ACP: a parallel combination of CNN and LSTM anticancer peptide recognition model

Abstract Background Anticancer peptides are defence substances with innate immune functions that can selectively act on cancer cells without harming normal cells and many studies have been conducted to identify anticancer peptides. In this paper, we introduce the anticancer peptide secondary structures as additional features and propose an effective computational model, CL-ACP, that uses a combined network and attention mechanism to predict anticancer peptides. Results The CL-ACP model uses secondary structures and original sequences of anticancer peptides to construct the feature space. The long short-term memory and convolutional neural network are used to extract the contextual dependence and local correlations of the feature space. Furthermore, a multi-head self-attention mechanism is used to strengthen the anticancer peptide sequences. Finally, three categories of feature information are classified by cascading. CL-ACP was validated using two types of datasets, anticancer peptide datasets and antimicrobial peptide datasets, on which it achieved good results compared to previous methods. CL-ACP achieved the highest AUC values of 0.935 and 0.972 on the anticancer peptide and antimicrobial peptide datasets, respectively. Conclusions CL-ACP can effectively recognize antimicrobial peptides, especially anticancer peptides, and the parallel combined neural network structure of CL-ACP does not require complex feature design and high time cost. It is suitable for application as a useful tool in antimicrobial peptide design.

Download Full-text

Daily runoff forecasting based on data-augmented neural network model

Journal of Hydroinformatics ◽

10.2166/hydro.2020.017 ◽

2020 ◽

Vol 22 (4) ◽

pp. 900-915 ◽

Cited By ~ 2

Author(s):

Xiao-ying Bi ◽

Bo Li ◽

Wen-long Lu ◽

Xin-zhi Zhou

Keyword(s):

Neural Network ◽

Data Augmentation ◽

Short Term Memory ◽

Interpolation Method ◽

Linear Interpolation ◽

Attention Mechanism ◽

Ar Model ◽

Support Vector ◽

Runoff Forecasting ◽

Comparison Results

Abstract Accurate daily runoff prediction plays an important role in the management and utilization of water resources. In order to improve the accuracy of prediction, this paper proposes a deep neural network (CAGANet) composed of a convolutional layer, an attention mechanism, a gated recurrent unit (GRU) neural network, and an autoregressive (AR) model. Given that the daily runoff sequence is abrupt and unstable, it is difficult for a single model and combined model to obtain high-precision daily runoff predictions directly. Therefore, this paper uses a linear interpolation method to enhance the stability of hydrological data and apply the augmented data to the CAGANet model, the support vector machine (SVM) model, the long short-term memory (LSTM) neural network and the attention-mechanism-based LSTM model (AM-LSTM). The comparison results show that among the four models based on data augmentation, the CAGANet model proposed in this paper has the best prediction accuracy. Its Nash–Sutcliffe efficiency can reach 0.993. Therefore, the CAGANet model based on data augmentation is a feasible daily runoff forecasting scheme.

Download Full-text

An Improved Model for Analyzing Textual Sentiment Based on a Deep Neural Network Using Multi-Head Attention Mechanism

Applied System Innovation ◽

10.3390/asi4040085 ◽

2021 ◽

Vol 4 (4) ◽

pp. 85

Author(s):

Hashem Saleh Sharaf Al-deen ◽

Zhiwen Zeng ◽

Raeed Al-sabri ◽

Arash Hekmat

Keyword(s):

Neural Network ◽

Sentiment Analysis ◽

Deep Neural Network ◽

Short Term Memory ◽

Attention Mechanism ◽

Short Term ◽

Arbitrary Length ◽

Term Memory ◽

Long Short Term Memory ◽

Improved Model

Due to the increasing growth of social media content on websites such as Twitter and Facebook, analyzing textual sentiment has become a challenging task. Therefore, many studies have focused on textual sentiment analysis. Recently, deep learning models, such as convolutional neural networks and long short-term memory, have achieved promising performance in sentiment analysis. These models have proven their ability to cope with the arbitrary length of sequences. However, when they are used in the feature extraction layer, the feature distance is highly dimensional, the text data are sparse, and they assign equal importance to various features. To address these issues, we propose a hybrid model that combines a deep neural network with a multi-head attention mechanism (DNN–MHAT). In the DNN–MHAT model, we first design an improved deep neural network to capture the text's actual context and extract the local features of position invariants by combining recurrent bidirectional long short-term memory units (Bi-LSTM) with a convolutional neural network (CNN). Second, we present a multi-head attention mechanism to capture the words in the text that are significantly related to long space and encoding dependencies, which adds a different focus to the information outputted from the hidden layers of BiLSTM. Finally, a global average pooling is applied for transforming the vector into a high-level sentiment representation to avoid model overfitting, and a sigmoid classifier is applied to carry out the sentiment polarity classification of texts. The DNN–MHAT model is tested on four reviews and two Twitter datasets. The results of the experiments illustrate the effectiveness of the DNN–MHAT model, which achieved excellent performance compared to the state-of-the-art baseline methods based on short tweets and long reviews.

Download Full-text

Student-Performulator: Student Academic Performance Using Hybrid Deep Neural Network

Sustainability ◽

10.3390/su13179775 ◽

2021 ◽

Vol 13 (17) ◽

pp. 9775

Author(s):

Bashir Khan Yousafzai ◽

Sher Afzal ◽

Taj Rahman ◽

Inayat Khan ◽

Inam Ullah ◽

...

Keyword(s):

Neural Network ◽

Data Mining ◽

Student Performance ◽

Performance Prediction ◽

Deep Neural Network ◽

Short Term Memory ◽

Educational Data Mining ◽

Attention Mechanism ◽

Feature Representation ◽

Superior Performance

Educational data generated through various platforms such as e-learning, e-admission systems, and automated result management systems can be effectively processed through educational data mining techniques in order to gather highly useful insights into students’ performance. The prediction of student performance from historical academic data is a highly desirable application of educational data mining. In this regard, there is an urgent need to develop an automated technique for student performance prediction. Existing studies on student performance prediction primarily focus on utilizing the conventional feature representation schemes, where extracted features are fed to a classifier. In recent years, deep learning has enabled researchers to automatically extract high-level features from raw data. Such advanced feature representation schemes enable superior performance in challenging tasks. In this work, we examine the deep neural network model, namely, the attention-based Bidirectional Long Short-Term Memory (BiLSTM) network to efficiently predict student performance (grades) from historical data. In this article, we have used the most advanced BiLSTM combined with an attention mechanism model by analyzing existing research problems, which are based on advanced feature classification and prediction. This work is really vital for academicians, universities, and government departments to early predict the performance. The superior sequence learning capabilities of BiLSTM combined with attention mechanism yield superior performance compared to the existing state-of-the-art. The proposed method has achieved a prediction accuracy of 90.16%.

Download Full-text

Sentimental Classification of News Headlines using Recurrent Neural Network

International Journal of Innovative Technology and Exploring Engineering - Special Issue ◽

10.35940/ijitee.f3573.049620 ◽

2020 ◽

Vol 9 (6) ◽

pp. 207-210

Keyword(s):

Neural Network ◽

Language Processing ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Attention Mechanism ◽

Machine Learning Techniques ◽

Learning Techniques ◽

Negative Comments ◽

News Headlines

Sentiment analysis combines the natural language processing task and analysis of the text that attempts to predict the sentiment of the text in terms of positive and negative comments. Nowadays, the tremendous volume of news originated via different webpages, and it is feasible to determine the opinion of particular news. This work tries to judge completely various machine learning techniques to classify the view of the news headlines. In this project, propose the appliance of Recurrent Neural Network with Long Short Term Memory Unit(LSTM), focus on seeking out similar news headlines, and predict the opinion of news headlines from numerous sources. The main objective is to classify the sentiment of news headlines from various sources using a recurrent neural network. Interestingly, the proposed attention mechanism performs better than the more complex attention mechanism on a held-out set of articles.

Download Full-text

Siamese CNN-BiLSTM Architecture for 3D Shape Representation Learning

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/93 ◽

2018 ◽

Cited By ~ 11

Author(s):

Guoxian Dai ◽

Jin Xie ◽

Yi Fang

Keyword(s):

Neural Network ◽

Loss Function ◽

Short Term Memory ◽

Shape Representation ◽

Feature Space ◽

Representation Learning ◽

3D Shape ◽

Aggregate Information ◽

3D Shapes ◽

2D Images

Learning a 3D shape representation from a collection of its rendered 2D images has been extensively studied. However, existing view-based techniques have not yet fully exploited the information among all the views of projections. In this paper, by employing recurrent neural network to efficiently capture features across different views, we propose a siamese CNN-BiLSTM network for 3D shape representation learning. The proposed method minimizes a discriminative loss function to learn a deep nonlinear transformation, mapping 3D shapes from the original space into a nonlinear feature space. In the transformed space, the distance of 3D shapes with the same label is minimized, otherwise the distance is maximized to a large margin. Specifically, the 3D shapes are first projected into a group of 2D images from different views. Then convolutional neural network (CNN) is adopted to extract features from different view images, followed by a bidirectional long short-term memory (LSTM) to aggregate information across different views. Finally, we construct the whole CNN-BiLSTM network into a siamese structure with contrastive loss function. Our proposed method is evaluated on two benchmarks, ModelNet40 and SHREC 2014, demonstrating superiority over the state-of-the-art methods.

Download Full-text

An End-to-End Model Based on Multiple Neural Networks with Data Augmentation for Keyword Spotting

International Journal of Asian Language Processing ◽

10.1142/s271755452050006x ◽

2020 ◽

pp. 2050006

Author(s):

Shuzhou Chai ◽

Wei-Qiang Zhang ◽

Changsheng Lv ◽

Zhenye Yang

Keyword(s):

Neural Network ◽

Data Augmentation ◽

Short Term Memory ◽

Attention Mechanism ◽

Keyword Spotting ◽

Data Set ◽

Positive Rate ◽

Memory Network ◽

Hidden Layer ◽

High Level

In this paper, we propose a network for small footprint keyword spotting. It includes four parts, data augmentation, Time-Delay Neural Network (TDNN) and Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and attention mechanism. Data augmentation is Google SpecAugment with time warping and frequency mask and time mask to the spectrum on clean data set and noisy data set. TDNN and CNN model the spectrogram features from the time and space dimensions. RNN-type networks include RNN, Long Short-Term Memory network (LSTM), Gated Recurrent Unit (GRU), Bidirectional Long Short-Term Memory network (BiLSTM), and Bidirectional Gated Recurrent Unit (BiGRU). The RNN extracts hidden layer features and transforms them into high-level representations. The attention mechanism is selected to generate different weights and multiplied by the high-level representation generated by the RNN to obtain a fixed-length vector. Finally, we use a linear transformation and softmax function to generate scores. We also explored the size of attention mechanism, two attention mechanisms, rectified linear unit and hidden layer of RNN. Our model has achieved a true positive rate of 99.81% at a 5% false positive rate.

Download Full-text

Gated Hierarchical LSTMs for Target-Based Sentiment Analysis

International Journal of Software Engineering and Knowledge Engineering ◽

10.1142/s0218194018400259 ◽

2018 ◽

Vol 28 (11n12) ◽

pp. 1719-1737

Author(s):

Hao Wang ◽

Xiaofang Zhang ◽

Bin Liang ◽

Qian Zhou ◽

Baowen Xu

Keyword(s):

Neural Network ◽

Sentiment Analysis ◽

Short Term Memory ◽

Network Models ◽

Neural Model ◽

Attention Mechanism ◽

Support Vector ◽

Neural Network Models ◽

Long Distance ◽

Sentence Level

In the field of target-based sentiment analysis, the deep neural model combining attention mechanism is a remarkable success. In current research, it is commonly seen that attention mechanism is combined with Long Short-Term Memory (LSTM) networks. However, such neural network-based architectures generally rely on complex computation and only focus on single target. In this paper, we propose a gated hierarchical LSTM (GH-LSTMs) model which combines regional LSTM and sentence-level LSTM via a gated operation for the task of target-based sentiment analysis. This approach can distinguish different polarities of sentiment of different targets in the same sentence through a regional LSTM. Furthermore, it is able to concentrate on the long-distance dependency of target in the whole sentence via a sentence-level LSTM. The final results of our experiments on multi-domain datasets of two languages from SemEval 2016 indicate that our approach yields better performance than Support Vector Machine (SVM) and several typical neural network models. A case study of some typical examples also makes a supplement to this conclusion.

Download Full-text

Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism

Applied Sciences ◽

10.3390/app10175841 ◽

2020 ◽

Vol 10 (17) ◽

pp. 5841 ◽

Cited By ~ 4

Author(s):

Beakcheol Jang ◽

Myeonghwi Kim ◽

Gaspard Harerimana ◽

Sang-ug Kang ◽

Jong Wook Kim

Keyword(s):

Neural Network ◽

Natural Language Processing ◽

Natural Language ◽

Convolutional Neural Network ◽

Language Processing ◽

Text Classification ◽

Short Term Memory ◽

User Behavior ◽

Hybrid Approach ◽

Attention Mechanism

There is a need to extract meaningful information from big data, classify it into different categories, and predict end-user behavior or emotions. Large amounts of data are generated from various sources such as social media and websites. Text classification is a representative research topic in the field of natural-language processing that categorizes unstructured text data into meaningful categorical classes. The long short-term memory (LSTM) model and the convolutional neural network for sentence classification produce accurate results and have been recently used in various natural-language processing (NLP) tasks. Convolutional neural network (CNN) models use convolutional layers and maximum pooling or max-overtime pooling layers to extract higher-level features, while LSTM models can capture long-term dependencies between word sequences hence are better used for text classification. However, even with the hybrid approach that leverages the powers of these two deep-learning models, the number of features to remember for classification remains huge, hence hindering the training process. In this study, we propose an attention-based Bi-LSTM+CNN hybrid model that capitalize on the advantages of LSTM and CNN with an additional attention mechanism. We trained the model using the Internet Movie Database (IMDB) movie review data to evaluate the performance of the proposed model, and the test results showed that the proposed hybrid attention Bi-LSTM+CNN model produces more accurate classification results, as well as higher recall and F1 scores, than individual multi-layer perceptron (MLP), CNN or LSTM models as well as the hybrid models.

Download Full-text

Deep Learning Enhanced Solar Energy Forecasting with AI-Driven IoT

Wireless Communications and Mobile Computing ◽

10.1155/2021/9249387 ◽

2021 ◽

Vol 2021 ◽

pp. 1-11

Author(s):

Hangxia Zhou ◽

Qian Liu ◽

Ke Yan ◽

Yang Du

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Short Term Memory ◽

Energy Generation ◽

Attention Mechanism ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

Short-term photovoltaic (PV) energy generation forecasting models are important, stabilizing the power integration between the PV and the smart grid for artificial intelligence- (AI-) driven internet of things (IoT) modeling of smart cities. With the recent development of AI and IoT technologies, it is possible for deep learning techniques to achieve more accurate energy generation forecasting results for the PV systems. Difficulties exist for the traditional PV energy generation forecasting method considering external feature variables, such as the seasonality. In this study, we propose a hybrid deep learning method that combines the clustering techniques, convolutional neural network (CNN), long short-term memory (LSTM), and attention mechanism with the wireless sensor network to overcome the existing difficulties of the PV energy generation forecasting problem. The overall proposed method is divided into three stages, namely, clustering, training, and forecasting. In the clustering stage, correlation analysis and self-organizing mapping are employed to select the highest relevant factors in historical data. In the training stage, a convolutional neural network, long short-term memory neural network, and attention mechanism are combined to construct a hybrid deep learning model to perform the forecasting task. In the testing stage, the most appropriate training model is selected based on the month of the testing data. The experimental results showed significantly higher prediction accuracy rates for all time intervals compared to existing methods, including traditional artificial neural networks, long short-term memory neural networks, and an algorithm combining long short-term memory neural network and attention mechanism.

Download Full-text