Summary Generation of Dengue Outbreaks from ProMED-mail Database using a Linguistic Pattern-infused Dual-channel BiLSTM (Preprint)

2021 ◽  
Author(s):  
Yung-Chun Chang ◽  
Yu-Wen Chiu ◽  
Ting-Wu Chuang
Keyword(s):  
Public Health ◽  
Neural Network ◽  
Language Processing ◽  
Environmental Changes ◽  
Short Term Memory ◽  
Dual Channel ◽  
Unstructured Text ◽  
Syntactic Features ◽  
Dengue Outbreaks ◽  
Text Content

BACKGROUND Globalization and environmental changes have increased the emergence and re-emergence of infectious diseases worldwide. The collaboration of regional infectious disease surveillance systems is critical but difficult to achieve because of the different transparency levels of health information sharing systems among countries. ProMED-mail is the most comprehensive expert-curated platform that provides rich outbreak information among humans, animals, and plants from different countries. However, owing to unstructured text content in reports, it is difficult to analyze them for further applications. Therefore, we have devised an idea to develop an automatic summary of the alerting articles from ProMED-mail. In this research, we propose a text summarization method that uses natural language processing to extract important sentences automatically from alert articles in ProMED emails to generate summaries of dengue outbreaks in Southeast Asia. Our method, can be used to capture crucial information quickly and make decisions for epidemic surveillance. OBJECTIVE To generate automatic summaries of unstructured text content from reports. METHODS Our materials come from the ProMED-mail website, spanning a period from 1994 to 2019. The collected data were annotated by professionals to establish a unique Taiwan dengue corpus through, which achieved almost perfect agreement (90% Cohen’s Kappa statistic). To generate a ProMED-mail summary, we developed a dual-channel bidirectional long-short term memory with an attention mechanism that infuses latent syntactic features to identify crucial sentences from the alerting articles. RESULTS Our method is superior to many well-known machine learning and neural network approaches in identifying important sentences, achieving a macro average F1-score of 93%. Moreover, the method can successfully extract key information about dengue fever outbreaks in ProMED-mail, and help researchers or public health practitioners to capture important summaries quickly. Besides verifying the model, we also recruited five professional experts and five students from related fields to carry out a satisfaction survey on the generated summary. The results showed that 83.6% of the summaries received high satisfaction ratings. CONCLUSIONS The proposed approach successfully fuses latent syntactic features into a deep neural network to analyze syntactic, semantic, and content information in the text. It then exploits the derived information to identify the crucial sentences in ProMED-mail. The experimental results show that the proposed method is effective and outperforms the comparisons. In addition, our method demonstrated the potential for summary generation from ProMED-mail. When a new alerting article arrives, public health decision makers can identify the outbreak information in a lengthy article quickly and deliver immediate responses to disease control and prevention. CLINICALTRIAL NA

scholarly journals Identifying protein subcellular localisation in scientific literature using bidirectional deep recurrent neural network

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Rakesh David ◽  
Rhys-Joshua D. Menezes ◽  
Jan De Klerk ◽  
Ian R. Castleden ◽  
Cornelia M. Hooper ◽  
...  
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Data Dissemination ◽  
Short Term Memory ◽  
Biological Data ◽  
Experimental Methodology ◽  
Subcellular Localisation ◽  
Crop Species ◽  
Deep Recurrent Neural Network ◽  
Functional Features

AbstractThe increased diversity and scale of published biological data has to led to a growing appreciation for the applications of machine learning and statistical methodologies to gain new insights. Key to achieving this aim is solving the Relationship Extraction problem which specifies the semantic interaction between two or more biological entities in a published study. Here, we employed two deep neural network natural language processing (NLP) methods, namely: the continuous bag of words (CBOW), and the bi-directional long short-term memory (bi-LSTM). These methods were employed to predict relations between entities that describe protein subcellular localisation in plants. We applied our system to 1700 published Arabidopsis protein subcellular studies from the SUBA manually curated dataset. The system combines pre-processing of full-text articles in a machine-readable format with relevant sentence extraction for downstream NLP analysis. Using the SUBA corpus, the neural network classifier predicted interactions between protein name, subcellular localisation and experimental methodology with an average precision, recall rate, accuracy and F1 scores of 95.1%, 82.8%, 89.3% and 88.4% respectively (n = 30). Comparable scoring metrics were obtained using the CropPAL database as an independent testing dataset that stores protein subcellular localisation in crop species, demonstrating wide applicability of prediction model. We provide a framework for extracting protein functional features from unstructured text in the literature with high accuracy, improving data dissemination and unlocking the potential of big data text analytics for generating new hypotheses.

scholarly journals Multi-Transformer: A New Neural Network-Based Architecture for Forecasting S&P Volatility

Mathematics ◽  
2021 ◽  
Vol 9 (15) ◽  
pp. 1794
Author(s):  
Eduardo Ramos-Pérez ◽  
Pablo J. Alonso-González ◽  
José Javier Núñez-Velázquez
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Short Term Memory ◽  
Risk Measures ◽  
Hybrid Models ◽  
Stock Volatility ◽  
Management Actions ◽  
Equity Risk ◽  
Hedging Strategies ◽  
Volatility Models

Events such as the Financial Crisis of 2007–2008 or the COVID-19 pandemic caused significant losses to banks and insurance entities. They also demonstrated the importance of using accurate equity risk models and having a risk management function able to implement effective hedging strategies. Stock volatility forecasts play a key role in the estimation of equity risk and, thus, in the management actions carried out by financial institutions. Therefore, this paper has the aim of proposing more accurate stock volatility models based on novel machine and deep learning techniques. This paper introduces a neural network-based architecture, called Multi-Transformer. Multi-Transformer is a variant of Transformer models, which have already been successfully applied in the field of natural language processing. Indeed, this paper also adapts traditional Transformer layers in order to be used in volatility forecasting models. The empirical results obtained in this paper suggest that the hybrid models based on Multi-Transformer and Transformer layers are more accurate and, hence, they lead to more appropriate risk measures than other autoregressive algorithms or hybrid models based on feed forward layers or long short term memory cells.

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

2021 ◽  
pp. 1-10
Author(s):  
Hye-Jeong Song ◽  
Tak-Sung Heo ◽  
Jong-Dae Kim ◽  
Chan-Young Park ◽  
Yu-Seop Kim
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Short Term Memory ◽  
Parallel Structure ◽  
Short Term ◽  
Similarity Estimation ◽  
Accurate Judgment ◽  
Proposed Model ◽  
Sentence Similarity ◽  
Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

scholarly journals Part-of-Speech Tagging via Deep Neural Networks for Northern-Ethiopic Languages

2020 ◽  
Vol 49 (4) ◽  
pp. 482-494
Author(s):  
Jurgita Kapočiūtė-Dzikienė ◽  
Senait Gebremichael Tesfagergish
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Parameter Tuning ◽  
Feed Forward Neural Network ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Pos Tagger

Deep Neural Networks (DNNs) have proven to be especially successful in the area of Natural Language Processing (NLP) and Part-Of-Speech (POS) tagging—which is the process of mapping words to their corresponding POS labels depending on the context. Despite recent development of language technologies, low-resourced languages (such as an East African Tigrinya language), have received too little attention. We investigate the effectiveness of Deep Learning (DL) solutions for the low-resourced Tigrinya language of the Northern-Ethiopic branch. We have selected Tigrinya as the testbed example and have tested state-of-the-art DL approaches seeking to build the most accurate POS tagger. We have evaluated DNN classifiers (Feed Forward Neural Network – FFNN, Long Short-Term Memory method – LSTM, Bidirectional LSTM, and Convolutional Neural Network – CNN) on a top of neural word2vec word embeddings with a small training corpus known as Nagaoka Tigrinya Corpus. To determine the best DNN classifier type, its architecture and hyper-parameter set both manual and automatic hyper-parameter tuning has been performed. BiLSTM method was proved to be the most suitable for our solving task: it achieved the highest accuracy equal to 92% that is 65% above the random baseline.

scholarly journals Chinese Text Classification Model Based on Deep Learning

2018 ◽  
Vol 10 (11) ◽  
pp. 113 ◽  
Author(s):  
Yue Li ◽  
Xutao Wang ◽  
Pengjian Xu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Language Processing ◽  
Chinese Text ◽  
Text Classification ◽  
Short Term Memory ◽  
Classification Model ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Text classification is of importance in natural language processing, as the massive text information containing huge amounts of value needs to be classified into different categories for further use. In order to better classify text, our paper tries to build a deep learning model which achieves better classification results in Chinese text than those of other researchers’ models. After comparing different methods, long short-term memory (LSTM) and convolutional neural network (CNN) methods were selected as deep learning methods to classify Chinese text. LSTM is a special kind of recurrent neural network (RNN), which is capable of processing serialized information through its recurrent structure. By contrast, CNN has shown its ability to extract features from visual imagery. Therefore, two layers of LSTM and one layer of CNN were integrated to our new model: the BLSTM-C model (BLSTM stands for bi-directional long short-term memory while C stands for CNN.) LSTM was responsible for obtaining a sequence output based on past and future contexts, which was then input to the convolutional layer for extracting features. In our experiments, the proposed BLSTM-C model was evaluated in several ways. In the results, the model exhibited remarkable performance in text classification, especially in Chinese texts.

scholarly journals Innovative Deep Neural Network Modeling for Fine-Grained Chinese Entity Recognition

Electronics ◽  
2020 ◽  
Vol 9 (6) ◽  
pp. 1001 ◽  
Author(s):  
Jingang Liu ◽  
Chunhe Xia ◽  
Haihua Yan ◽  
Wenjing Xu
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Short Term Memory ◽  
Named Entity Recognition ◽  
Training Model ◽  
Entity Recognition ◽  
Coarse Grained ◽  
Neural Network Modeling ◽  
Fine Grained ◽  
Named Entity

Named entity recognition (NER) is a basic but crucial task in the field of natural language processing (NLP) and big data analysis. The recognition of named entities based on Chinese is more complicated and difficult than English, which makes the task of NER in Chinese more challenging. In particular, fine-grained named entity recognition is more challenging than traditional named entity recognition tasks, mainly because fine-grained tasks have higher requirements for the ability of automatic feature extraction and information representation of deep neural models. In this paper, we propose an innovative neural network model named En2BiLSTM-CRF to improve the effect of fine-grained Chinese entity recognition tasks. This proposed model including the initial encoding layer, the enhanced encoding layer, and the decoding layer combines the advantages of pre-training model encoding, dual bidirectional long short-term memory (BiLSTM) networks, and a residual connection mechanism. Hence, it can encode information multiple times and extract contextual features hierarchically. We conducted sufficient experiments on two representative datasets using multiple important metrics and compared them with other advanced baselines. We present promising results showing that our proposed En2BiLSTM-CRF has better performance as well as better generalization ability in both fine-grained and coarse-grained Chinese entity recognition tasks.

scholarly journals Developing a dengue forecast model using Long Short Term Memory neural networks method

2019 ◽  
Author(s):  
Jiucheng Xu ◽  
Keqiang Xu ◽  
Zhichao Li ◽  
Taotian Tu ◽  
Lei Xu ◽  
...  
Keyword(s):  
Public Health ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Forecast Model ◽  
Short Term ◽  
Term Memory ◽  
Chinese Cities ◽  
Dengue Outbreaks ◽  
Long Short Term Memory

AbstractBackgroundDengue Fever (DF) is a tropical mosquito-borne disease that threatens public health and causes enormous economic burdens worldwide. In China, DF expanded from coastal region to inner land, and the incidence sharply increased in the last few years. In this study, we conduct the analysis of dengue using the Long Short Term Memory (LSTM) recurrent neural networks. This is an artificial intelligence technology, to develop a precise dengue forecast model.Methodology/Principal FindingsThe model is developed from monthly dengue cases and local meteorological data of 2005–2018 among top 20 Chinese cities with a record of the highest dengue incidence. The first 13 year data were used to construct the LSTM and to predict the dengue outbreaks in 2018. The results are compared with the estimated dengue cases of other previously published models. Model performance and prediction accuracy were assessed using Root Mean Square Error (RMSE). With the LSTM method, the prediction measurements of average RMSE drop by 54.79% and 34.76% as compared with the Susceptible Infected Recovered (SIR) model and Zero Inflated Generalized Additive Model (ZIGAM). Our results showed that if only local data were used to develop forecast models, the LSTM neural networks would fail to capture the transmission characteristics of dengue virus in areas with fewer dengue cases. Contrarily, transfer learning (TL) can improve the accuracy of prediction of the LSTM neural network model in areas with fewer dengue incidences.Conclusion and significanceThe LSTM model is beneficial in predicting dengue incidence as compared with other previously published forecasting models. The findings provide a more precise forecast dengue model, which can help the local government and health-related departments respond early to dengue epidemics.Author summaryIn China, DF is a public health concern that poses a great economic burden on local governments. However, the incidence has sharply increased in recent years with growth in the sub-regions. With this issue, it will be challenging to develop an accurate and timely dengue forecast model. LSTM recurrent neural networks, deep learning methods and virus propagation rules by learning from observational data offer more advantages in predicting the prevalence of infectious disease dynamics than the traditional statistical model. The 2005–2017 data of the top 20 Chinese cities with the highest dengue incidence were used to construct the LSTM model, advantageous in predicting dengue in most cities. Moreover, the model helped to predict the dengue outbreaks in 2018 and used to compare the estimated dengue cases with the RMSE results of other previously published models. A thorough search of the literature shows that this is the first established dengue forecast model using the LSTM method, which is effective in predicting the trend of dengue dynamics.

scholarly journals A Hybrid Deep Learning Model to Predict the Impact of COVID-19 on Mental Health form Social Media Big Data

2021 ◽  
Author(s):  
Tapotosh Ghosh ◽  
Md. Hasan Al Banna ◽  
Md. Jaber Al Nahian ◽  
Kazi Abu Taher ◽  
M Shamim Kaiser ◽  
...  
Keyword(s):  
Neural Network ◽  
Mental Health ◽  
Social Media ◽  
Big Data ◽  
Language Processing ◽  
Short Term Memory ◽  
Daily Basis ◽  
Significant Rise ◽  
Special Focus ◽  
The Impact

The novel coronavirus disease (COVID-19) pandemic is provoking a prevalent consequence on mental health because of less interaction among people, economic collapse, negativity, fear of losing jobs, and death of the near and dear ones. To express their mental state, people often are using social media as one of the preferred means. Due to reduced outdoor activities, people are spending more time on social media than usual and expressing their emotion of anxiety, fear, and depression. On a daily basis, about 2.5 quintillion bytes of data are generated on social media, analyzing this big data can become an excellent means to evaluate the effect of COVID-19 on mental health. In this work, we have analyzed data from Twitter microblog (tweets) to find out the effect of COVID-19 on peoples mental health with a special focus on depression. We propose a novel pipeline, based on recurrent neural network (in the form of long-short term memory or LSTM) and convolutional neural network, capable of identifying depressive tweets with an accuracy of 99.42%. Preprocessed using various natural language processing techniques, the aim was to find out depressive emotion from these tweets. Analyzing over 571 thousand tweets posted between October 2019 and May 2020 by 482 users, a significant rise in depressing tweets was observed between February and May of 2020, which indicates as an impact of the long ongoing COVID-19 pandemic situation.

scholarly journals Analysis of comparative performance of deep learning models for sentiment analysis

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Mirza Murtaza
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
Neural Nets ◽  
Test Accuracy ◽  
Comparative Performance ◽  
Text Filtering ◽  
Percent Accuracy

Abstract Sentiment analysis of text can be performed using machine learning and natural language processing methods. However, there is no single tool or method that is effective in all cases. The objective of this research project is to determine the effectiveness of neural network-based architecture to perform sentiment analysis of customer comments and reviews, such as the ones on Amazon site. A typical sentiment analysis process involves text preparation (of acquired content), sentiment detection, sentiment classification and analysis of results. In this research, the objective is to a) identify the best approach for text preparation in a given application (text filtering approach to remove errors in data), and, most importantly, b) what is the best machine learning (feed forward neural nets, convolutional neural nets, Long Short-Term Memory networks) approach that provides best classification accuracy. In this research, a set of three thousand two hundred reviews of food related products were used to train and experiment with a neural network-based sentiment analysis system. The neural network implementation of six different models provided close to one-hundred percent accuracy of test data, and a decent test accuracy in mid-80%. The results of the research would be useful to businesses in evaluating customer preferences for products or services.  

scholarly journals Forex exchange rate forecasting using deep recurrent neural networks

2020 ◽  
Vol 2 (1-2) ◽  
pp. 69-96 ◽  
Author(s):  
Alexander Jakob Dautel ◽  
Wolfgang Karl Härdle ◽  
Stefan Lessmann ◽  
Hsin-Vonn Seow
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Exchange Rate ◽  
Language Processing ◽  
Short Term Memory ◽  
Recurrent Network ◽  
Network Architectures ◽  
Exchange Rate Forecasting ◽  
Trading Model ◽  
Gated Recurrent Units

Abstract Deep learning has substantially advanced the state of the art in computer vision, natural language processing, and other fields. The paper examines the potential of deep learning for exchange rate forecasting. We systematically compare long short-term memory networks and gated recurrent units to traditional recurrent network architectures as well as feedforward networks in terms of their directional forecasting accuracy and the profitability of trading model predictions. Empirical results indicate the suitability of deep networks for exchange rate forecasting in general but also evidence the difficulty of implementing and tuning corresponding architectures. Especially with regard to trading profit, a simpler neural network may perform as well as if not better than a more complex deep neural network.

Sign in / Sign up

Export Citation Format

Share Document