Identifying protein subcellular localisation in scientific literature using bidirectional deep recurrent neural network

AbstractThe increased diversity and scale of published biological data has to led to a growing appreciation for the applications of machine learning and statistical methodologies to gain new insights. Key to achieving this aim is solving the Relationship Extraction problem which specifies the semantic interaction between two or more biological entities in a published study. Here, we employed two deep neural network natural language processing (NLP) methods, namely: the continuous bag of words (CBOW), and the bi-directional long short-term memory (bi-LSTM). These methods were employed to predict relations between entities that describe protein subcellular localisation in plants. We applied our system to 1700 published Arabidopsis protein subcellular studies from the SUBA manually curated dataset. The system combines pre-processing of full-text articles in a machine-readable format with relevant sentence extraction for downstream NLP analysis. Using the SUBA corpus, the neural network classifier predicted interactions between protein name, subcellular localisation and experimental methodology with an average precision, recall rate, accuracy and F1 scores of 95.1%, 82.8%, 89.3% and 88.4% respectively (n = 30). Comparable scoring metrics were obtained using the CropPAL database as an independent testing dataset that stores protein subcellular localisation in crop species, demonstrating wide applicability of prediction model. We provide a framework for extracting protein functional features from unstructured text in the literature with high accuracy, improving data dissemination and unlocking the potential of big data text analytics for generating new hypotheses.

Download Full-text

Identifying protein subcellular localisation in scientific literature using bidirectional deep recurrent neural network

10.1101/2020.09.09.290577 ◽

2020 ◽

Author(s):

Rakesh David ◽

Rhys-Joshua D. Menezes ◽

Jan De Klerk ◽

Ian R. Castleden ◽

Cornelia M. Hooper ◽

...

Keyword(s):

Neural Network ◽

Language Processing ◽

Recurrent Neural Network ◽

Experimental Methodology ◽

Subcellular Localisation ◽

Crop Species ◽

Accuracy Measure ◽

Deep Recurrent Neural Network ◽

Functional Features ◽

Biological Entities

AbstractWith the advent of increased diversity and scale of molecular data, there has been a growing appreciation for the applications of machine learning and statistical methodologies to gain new biological insights. An important step in achieving this aim is the Relation Extraction process which specifies if an interaction exists between two or more biological entities in a published study. Here, we employed natural-language processing (CBOW) and deep Recurrent Neural Network (bi-directional LSTM) to predict relations between biological entities that describe protein subcellular localisation in plants. We applied our system to 1700 published Arabidopsis protein subcellular studies from the SUBA manually curated dataset. The system was able to extract relevant text and the classifier predicted interactions between protein name, subcellular localisation and experimental methodology. It obtained a final precision, recall rate, accuracy and F1 scores of 0.951, 0.828, 0.893 and 0.884 respectively. The classifier was subsequently tested on a similar problem in crop species (CropPAL) and demonstrated a comparable accuracy measure (0.897). Consequently, our approach can be used to extract protein functional features from unstructured text in the literature with high accuracy. The developed system will improve dissemination or protein functional data to the scientific community and unlock the potential of big data text analytics for generating new hypotheses from diverse datasets.

Download Full-text

Multi-Transformer: A New Neural Network-Based Architecture for Forecasting S&P Volatility

Mathematics ◽

10.3390/math9151794 ◽

2021 ◽

Vol 9 (15) ◽

pp. 1794

Author(s):

Eduardo Ramos-Pérez ◽

Pablo J. Alonso-González ◽

José Javier Núñez-Velázquez

Keyword(s):

Neural Network ◽

Language Processing ◽

Short Term Memory ◽

Risk Measures ◽

Hybrid Models ◽

Stock Volatility ◽

Management Actions ◽

Equity Risk ◽

Hedging Strategies ◽

Volatility Models

Events such as the Financial Crisis of 2007–2008 or the COVID-19 pandemic caused significant losses to banks and insurance entities. They also demonstrated the importance of using accurate equity risk models and having a risk management function able to implement effective hedging strategies. Stock volatility forecasts play a key role in the estimation of equity risk and, thus, in the management actions carried out by financial institutions. Therefore, this paper has the aim of proposing more accurate stock volatility models based on novel machine and deep learning techniques. This paper introduces a neural network-based architecture, called Multi-Transformer. Multi-Transformer is a variant of Transformer models, which have already been successfully applied in the field of natural language processing. Indeed, this paper also adapts traditional Transformer layers in order to be used in volatility forecasting models. The empirical results obtained in this paper suggest that the hybrid models based on Multi-Transformer and Transformer layers are more accurate and, hence, they lead to more appropriate risk measures than other autoregressive algorithms or hybrid models based on feed forward layers or long short term memory cells.

Download Full-text

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189593 ◽

2021 ◽

pp. 1-10

Author(s):

Hye-Jeong Song ◽

Tak-Sung Heo ◽

Jong-Dae Kim ◽

Chan-Young Park ◽

Yu-Seop Kim

Keyword(s):

Neural Network ◽

Language Processing ◽

Short Term Memory ◽

Parallel Structure ◽

Short Term ◽

Similarity Estimation ◽

Accurate Judgment ◽

Proposed Model ◽

Sentence Similarity ◽

Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

Download Full-text

Part-of-Speech Tagging via Deep Neural Networks for Northern-Ethiopic Languages

Information Technology And Control ◽

10.5755/j01.itc.49.4.26808 ◽

2020 ◽

Vol 49 (4) ◽

pp. 482-494

Author(s):

Jurgita Kapočiūtė-Dzikienė ◽

Senait Gebremichael Tesfagergish

Keyword(s):

Neural Network ◽

Neural Networks ◽

Language Processing ◽

Deep Neural Networks ◽

Short Term Memory ◽

Parameter Tuning ◽

Feed Forward Neural Network ◽

Pos Tagging ◽

Part Of Speech ◽

Pos Tagger

Deep Neural Networks (DNNs) have proven to be especially successful in the area of Natural Language Processing (NLP) and Part-Of-Speech (POS) tagging—which is the process of mapping words to their corresponding POS labels depending on the context. Despite recent development of language technologies, low-resourced languages (such as an East African Tigrinya language), have received too little attention. We investigate the effectiveness of Deep Learning (DL) solutions for the low-resourced Tigrinya language of the Northern-Ethiopic branch. We have selected Tigrinya as the testbed example and have tested state-of-the-art DL approaches seeking to build the most accurate POS tagger. We have evaluated DNN classifiers (Feed Forward Neural Network – FFNN, Long Short-Term Memory method – LSTM, Bidirectional LSTM, and Convolutional Neural Network – CNN) on a top of neural word2vec word embeddings with a small training corpus known as Nagaoka Tigrinya Corpus. To determine the best DNN classifier type, its architecture and hyper-parameter set both manual and automatic hyper-parameter tuning has been performed. BiLSTM method was proved to be the most suitable for our solving task: it achieved the highest accuracy equal to 92% that is 65% above the random baseline.

Download Full-text

A Deep Recurrent Neural Network for Non-Intrusive Load Monitoring Based on Multi-Feature Input Space and Post-Processing

Energies ◽

10.3390/en13092195 ◽

2020 ◽

Vol 13 (9) ◽

pp. 2195

Author(s):

Hasan Rafiq ◽

Xiaohan Shi ◽

Hengxu Zhang ◽

Huimin Li ◽

Manesh Kumar Ochani

Keyword(s):

Neural Network ◽

Power Consumption ◽

Real Time ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Estimation Accuracy ◽

Post Processing ◽

Input Space ◽

Deep Recurrent Neural Network ◽

Load Monitoring

Non-intrusive load monitoring (NILM) is a process of estimating operational states and power consumption of individual appliances, which if implemented in real-time, can provide actionable feedback in terms of energy usage and personalized recommendations to consumers. Intelligent disaggregation algorithms such as deep neural networks can fulfill this objective if they possess high estimation accuracy and lowest generalization error. In order to achieve these two goals, this paper presents a disaggregation algorithm based on a deep recurrent neural network using multi-feature input space and post-processing. First, the mutual information method was used to select electrical parameters that had the most influence on the power consumption of each target appliance. Second, selected steady-state parameters based multi-feature input space (MFS) was used to train the 4-layered bidirectional long short-term memory (LSTM) model for each target appliance. Finally, a post-processing technique was used at the disaggregation stage to eliminate irrelevant predicted sequences, enhancing the classification and estimation accuracy of the algorithm. A comprehensive evaluation was conducted on 1-Hz sampled UKDALE and ECO datasets in a noised scenario with seen and unseen test cases. Performance evaluation showed that the MFS-LSTM algorithm is computationally efficient, scalable, and possesses better estimation accuracy in a noised scenario, and generalized to unseen loads as compared to benchmark algorithms. Presented results proved that the proposed algorithm fulfills practical application requirements and can be deployed in real-time.

Download Full-text

Chinese Text Classification Model Based on Deep Learning

Future Internet ◽

10.3390/fi10110113 ◽

2018 ◽

Vol 10 (11) ◽

pp. 113 ◽

Cited By ~ 17

Author(s):

Yue Li ◽

Xutao Wang ◽

Pengjian Xu

Keyword(s):

Neural Network ◽

Deep Learning ◽

Language Processing ◽

Chinese Text ◽

Text Classification ◽

Short Term Memory ◽

Classification Model ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

Text classification is of importance in natural language processing, as the massive text information containing huge amounts of value needs to be classified into different categories for further use. In order to better classify text, our paper tries to build a deep learning model which achieves better classification results in Chinese text than those of other researchers’ models. After comparing different methods, long short-term memory (LSTM) and convolutional neural network (CNN) methods were selected as deep learning methods to classify Chinese text. LSTM is a special kind of recurrent neural network (RNN), which is capable of processing serialized information through its recurrent structure. By contrast, CNN has shown its ability to extract features from visual imagery. Therefore, two layers of LSTM and one layer of CNN were integrated to our new model: the BLSTM-C model (BLSTM stands for bi-directional long short-term memory while C stands for CNN.) LSTM was responsible for obtaining a sequence output based on past and future contexts, which was then input to the convolutional layer for extracting features. In our experiments, the proposed BLSTM-C model was evaluated in several ways. In the results, the model exhibited remarkable performance in text classification, especially in Chinese texts.

Download Full-text

Innovative Deep Neural Network Modeling for Fine-Grained Chinese Entity Recognition

Electronics ◽

10.3390/electronics9061001 ◽

2020 ◽

Vol 9 (6) ◽

pp. 1001 ◽

Cited By ~ 1

Author(s):

Jingang Liu ◽

Chunhe Xia ◽

Haihua Yan ◽

Wenjing Xu

Keyword(s):

Neural Network ◽

Language Processing ◽

Short Term Memory ◽

Named Entity Recognition ◽

Training Model ◽

Entity Recognition ◽

Coarse Grained ◽

Neural Network Modeling ◽

Fine Grained ◽

Named Entity

Named entity recognition (NER) is a basic but crucial task in the field of natural language processing (NLP) and big data analysis. The recognition of named entities based on Chinese is more complicated and difficult than English, which makes the task of NER in Chinese more challenging. In particular, fine-grained named entity recognition is more challenging than traditional named entity recognition tasks, mainly because fine-grained tasks have higher requirements for the ability of automatic feature extraction and information representation of deep neural models. In this paper, we propose an innovative neural network model named En2BiLSTM-CRF to improve the effect of fine-grained Chinese entity recognition tasks. This proposed model including the initial encoding layer, the enhanced encoding layer, and the decoding layer combines the advantages of pre-training model encoding, dual bidirectional long short-term memory (BiLSTM) networks, and a residual connection mechanism. Hence, it can encode information multiple times and extract contextual features hierarchically. We conducted sufficient experiments on two representative datasets using multiple important metrics and compared them with other advanced baselines. We present promising results showing that our proposed En2BiLSTM-CRF has better performance as well as better generalization ability in both fine-grained and coarse-grained Chinese entity recognition tasks.

Download Full-text

Development of a daily PM<sub>10</sub> and PM<sub>2.5</sub> prediction system using a deep long short-term memory neural network model

Atmospheric Chemistry and Physics ◽

10.5194/acp-19-12935-2019 ◽

2019 ◽

Vol 19 (20) ◽

pp. 12935-12951 ◽

Cited By ~ 5

Author(s):

Hyun S. Kim ◽

Inyoung Park ◽

Chul H. Song ◽

Kyunghwa Lee ◽

Jae W. Yun ◽

...

Keyword(s):

Neural Network ◽

Short Term Memory ◽

Transport Model ◽

Initial Conditions ◽

Current System ◽

Prediction System ◽

Short Term ◽

Term Memory ◽

Deep Recurrent Neural Network ◽

Long Short Term Memory

Abstract. A deep recurrent neural network system based on a long short-term memory (LSTM) model was developed for daily PM10 and PM2.5 predictions in South Korea. The structural and learnable parameters of the newly developed system were optimized from iterative model training. Independent variables were obtained from ground-based observations over 2.3 years. The performance of the particulate matter (PM) prediction LSTM was then evaluated by comparisons with ground PM observations and with the PM concentrations predicted from two sets of 3-D chemistry-transport model (CTM) simulations (with and without data assimilation for initial conditions). The comparisons showed, in general, better performance with the LSTM than with the 3-D CTM simulations. For example, in terms of IOAs (index of agreements), the PM prediction IOAs were enhanced from 0.36–0.78 with the 3-D CTM simulations to 0.62–0.79 with the LSTM-based model. The deep LSTM-based PM prediction system developed at observation sites is expected to be further integrated with 3-D CTM-based prediction systems in the future. In addition to this, further possible applications of the deep LSTM-based system are discussed, together with some limitations of the current system.

Download Full-text

Estimation of Bladder Pressure and Volume from the Neural Activity of Lumbosacral Dorsal Horn Using a Long-Short-Term-Memory-based Deep Neural Network

Scientific Reports ◽

10.1038/s41598-019-54144-8 ◽

2019 ◽

Vol 9 (1) ◽

Author(s):

Milad Jabbari ◽

Abbas Erfanian

Keyword(s):

Neural Network ◽

Neural Activity ◽

Short Term Memory ◽

Field Potential ◽

Bladder Function ◽

Bladder Pressure ◽

Short Term ◽

Term Memory ◽

Deep Recurrent Neural Network ◽

Long Short Term Memory

AbstractIn this paper, we propose a deep recurrent neural network (DRNN) for the estimation of bladder pressure and volume from neural activity recorded directly from spinal cord gray matter neurons. The model was based on the Long Short-Term Memory (LSTM) architecture, which has emerged as a general and effective model for capturing long-term temporal dependencies with good generalization performance. In this way, training the network with the data recorded from one rat could lead to estimating the bladder status of different rats. We combined modeling of spiking and local field potential (LFP) activity into a unified framework to estimate the pressure and volume of the bladder. Moreover, we investigated the effect of two-electrode recording on decoding performance. The results show that the two-electrode recordings significantly improve the decoding performance compared to single-electrode recordings. The proposed framework could estimate bladder pressure and volume with an average normalized root-mean-squared (NRMS) error of 14.9 ± 4.8% and 19.7 ± 4.7% and a correlation coefficient (CC) of 83.2 ± 3.2% and 74.2 ± 6.2%, respectively. This work represents a promising approach to the real-time estimation of bladder pressure/volume in the closed-loop control of bladder function using functional electrical stimulation.

Download Full-text

A Robust False Spam Review Detection Using Deep Long Short-Term Memory (LSTM) Based Recurrent Neural Network

Journal of Computational and Theoretical Nanoscience ◽

10.1166/jctn.2020.9198 ◽

2020 ◽

Vol 17 (8) ◽

pp. 3421-3426

Author(s):

D. Deva Hema ◽

J. Tharun ◽

G. Arun Dev ◽

N. Sateesh

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Online Reviews ◽

Physical Contact ◽

Proposed Model ◽

Deep Recurrent Neural Network ◽

Bayes Algorithm ◽

Fake Reviews

Our day-to-day activity is highly influenced by development of Internet. One of the rapid growing area in Internet is E-commerce. People are eager to buy products from online sites like Amazon, embay, Flipkart etc. Customers can write reviews about the products purchased online. The purchasing of good through online has been increasing exponentially since last few years. As there is no physical contact with goods before purchasing through online, people totally rely on reviews about the product before purchasing it. Hence review plays an important role in deciding the quality of the product. There are many customers who give online reviews about the product after using it. Hence the quality of the product is decided by the reviews of the customers. Thus, detection of fake reviews has become one of the important task. The proposed system will help in finding such fake reviews about the product, so that the fake reviews can be eliminated. Therefore, the purchasing of the products will be totally based on the genuine reviews. The proposed system uses Deep Recurrent Neural Network (DRNN) to predict the fake reviews and the performance of the proposed method has compared with Naïve Bayes Algorithm. The proposed model shows good accuracy and can handle huge amount of data over the existing system.

Download Full-text