Personalized Predictive Models for Identifying Clinical Deterioration Using LSTM in Emergency Departments

Early detection of deterioration at hospitals could be beneficial in terms of reducing mortality and morbidity rates and costs. In this paper, we present a model based on Long Short-Term Memory (LSTM) neural network used in deep learning to predict the illness severity of patients in advance. Hence, by predicting health severity, this model can be used to identify deteriorating patients. Our proposed model utilizes continuous monitored vital signs, including heart rate, respiratory rate, oxygen saturation, and blood pressure automatically collected from patients during hospitalization. In this study, a short-time prediction using a sliding window approach is applied. The performance of the proposed model was compared with the Multi-Layer Perceptron (MLP) neural network, a feedforward class of neural network, based on R2 score and Root Mean Square Error (RMSE) metrics. The results showed that the LSTM has a better performance and could predict the illness severity of patients more accurately.

Download Full-text

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189593 ◽

2021 ◽

pp. 1-10

Author(s):

Hye-Jeong Song ◽

Tak-Sung Heo ◽

Jong-Dae Kim ◽

Chan-Young Park ◽

Yu-Seop Kim

Keyword(s):

Neural Network ◽

Language Processing ◽

Short Term Memory ◽

Parallel Structure ◽

Short Term ◽

Similarity Estimation ◽

Accurate Judgment ◽

Proposed Model ◽

Sentence Similarity ◽

Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

Download Full-text

Production Forecasting with the Interwell Interference by Integrating Graph Convolutional and Long Short-Term Memory Neural Network

SPE Reservoir Evaluation & Engineering ◽

10.2118/208596-pa ◽

2021 ◽

pp. 1-17

Author(s):

Enda Du ◽

Yuetian Liu ◽

Ziyan Cheng ◽

Liang Xue ◽

Jing Ma ◽

...

Keyword(s):

Neural Network ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Production Forecasting ◽

Temporal Correlations ◽

Proposed Model ◽

The Mean ◽

Long Short Term Memory ◽

The Impact

Summary Accurate production forecasting is an essential task and accompanies the entire process of reservoir development. With the limitation of prediction principles and processes, the traditional approaches are difficult to make rapid predictions. With the development of artificial intelligence, the data-driven model provides an alternative approach for production forecasting. To fully take the impact of interwell interference on production into account, this paper proposes a deep learning-based hybrid model (GCN-LSTM), where graph convolutional network (GCN) is used to capture complicated spatial patterns between each well, and long short-term memory (LSTM) neural network is adopted to extract intricate temporal correlations from historical production data. To implement the proposed model more efficiently, two data preprocessing procedures are performed: Outliers in the data set are removed by using a box plot visualization, and measurement noise is reduced by a wavelet transform. The robustness and applicability of the proposed model are evaluated in two scenarios of different data types with the root mean square error (RMSE), the mean absolute error (MAE), and the mean absolute percentage error (MAPE). The results show that the proposed model can effectively capture spatial and temporal correlations to make a rapid and accurate oil production forecast.

Download Full-text

Bimodal Emotion Recognition Model for Minnan Songs

Information ◽

10.3390/info11030145 ◽

2020 ◽

Vol 11 (3) ◽

pp. 145 ◽

Cited By ~ 1

Author(s):

Zhenglong Xiang ◽

Xialei Dong ◽

Yuanxiang Li ◽

Fei Yu ◽

Xing Xu ◽

...

Keyword(s):

Neural Network ◽

Emotion Recognition ◽

Short Term Memory ◽

Music Appreciation ◽

Research Papers ◽

Audio Features ◽

Analysis Theory ◽

Proposed Model ◽

Song Lyrics ◽

Long Short Term Memory

Most of the existing research papers study the emotion recognition of Minnan songs from the perspectives of music analysis theory and music appreciation. However, these investigations do not explore any possibility of carrying out an automatic emotion recognition of Minnan songs. In this paper, we propose a model that consists of four main modules to classify the emotion of Minnan songs by using the bimodal data—song lyrics and audio. In the proposed model, an attention-based Long Short-Term Memory (LSTM) neural network is applied to extract lyrical features, and a Convolutional Neural Network (CNN) is used to extract the audio features from the spectrum. Then, two kinds of extracted features are concatenated by multimodal compact bilinear pooling, and finally, the concatenated features are input to the classifying module to determine the song emotion. We designed three experiment groups to investigate the classifying performance of combinations of the four main parts, the comparisons of proposed model with the current approaches and the influence of a few key parameters on the performance of emotion recognition. The results show that the proposed model exhibits better performance over all other experimental groups. The accuracy, precision and recall of the proposed model exceed 0.80 in a combination of appropriate parameters.

Download Full-text

Trajectory-Based Air-Writing Recognition Using Deep Neural Network and Depth Sensor

Sensors ◽

10.3390/s20020376 ◽

2020 ◽

Vol 20 (2) ◽

pp. 376 ◽

Cited By ~ 6

Author(s):

Md. Shahinur Alam ◽

Ki-Chul Kwon ◽

Md. Ashraful Alam ◽

Mohammed Y. Abbass ◽

Shariar Md Imtiaz ◽

...

Keyword(s):

Neural Network ◽

Nearest Neighbor ◽

Short Term Memory ◽

Three Dimensional ◽

Recognition System ◽

Writing Systems ◽

Depth Sensor ◽

Handheld Device ◽

Proposed Model ◽

Long Short Term Memory

Trajectory-based writing system refers to writing a linguistic character or word in free space by moving a finger, marker, or handheld device. It is widely applicable where traditional pen-up and pen-down writing systems are troublesome. Due to the simple writing style, it has a great advantage over the gesture-based system. However, it is a challenging task because of the non-uniform characters and different writing styles. In this research, we developed an air-writing recognition system using three-dimensional (3D) trajectories collected by a depth camera that tracks the fingertip. For better feature selection, the nearest neighbor and root point translation was used to normalize the trajectory. We employed the long short-term memory (LSTM) and a convolutional neural network (CNN) as a recognizer. The model was tested and verified by the self-collected dataset. To evaluate the robustness of our model, we also employed the 6D motion gesture (6DMG) alphanumeric character dataset and achieved 99.32% accuracy which is the highest to date. Hence, it verifies that the proposed model is invariant for digits and characters. Moreover, we publish a dataset containing 21,000 digits; which solves the lack of dataset in the current research.

Download Full-text

Attention-Based Long Short-Term Memory Recurrent Neural Network for Capacity Degradation of Lithium-Ion Batteries

Batteries ◽

10.3390/batteries7040066 ◽

2021 ◽

Vol 7 (4) ◽

pp. 66

Author(s):

Tadele Mamo ◽

Fu-Kwun Wang

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Prediction Accuracy ◽

Short Term Memory ◽

Lithium Ion ◽

Short Term ◽

Term Memory ◽

Proposed Model ◽

Capacity Degradation ◽

Long Short Term Memory

Monitoring cycle life can provide a prediction of the remaining battery life. To improve the prediction accuracy of lithium-ion battery capacity degradation, we propose a hybrid long short-term memory recurrent neural network model with an attention mechanism. The hyper-parameters of the proposed model are also optimized by a differential evolution algorithm. Using public battery datasets, the proposed model is compared to some published models, and it gives better prediction performance in terms of mean absolute percentage error and root mean square error. In addition, the proposed model can achieve higher prediction accuracy of battery end of life.

Download Full-text

A Deep Neural Network Model for Short-Term Load Forecast Based on Long Short-Term Memory Network and Convolutional Neural Network

Energies ◽

10.3390/en11123493 ◽

2018 ◽

Vol 11 (12) ◽

pp. 3493 ◽

Cited By ~ 60

Author(s):

Chujie Tian ◽

Jian Ma ◽

Chunhong Zhang ◽

Panpan Zhan

Keyword(s):

Neural Network ◽

Real World ◽

Deep Neural Network ◽

Short Term Memory ◽

Electrical Load ◽

Short Term ◽

Term Memory ◽

Load Forecast ◽

Proposed Model ◽

Long Short Term Memory

Accurate electrical load forecasting is of great significance to help power companies in better scheduling and efficient management. Since high levels of uncertainties exist in the load time series, it is a challenging task to make accurate short-term load forecast (STLF). In recent years, deep learning approaches provide better performance to predict electrical load in real world cases. The convolutional neural network (CNN) can extract the local trend and capture the same pattern, and the long short-term memory (LSTM) is proposed to learn the relationship in time steps. In this paper, a new deep neural network framework that integrates the hidden feature of the CNN model and the LSTM model is proposed to improve the forecasting accuracy. The proposed model was tested in a real-world case, and detailed experiments were conducted to validate its practicality and stability. The forecasting performance of the proposed model was compared with the LSTM model and the CNN model. The Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE) were used as the evaluation indexes. The experimental results demonstrate that the proposed model can achieve better and stable performance in STLF.

Download Full-text

Temperature Prediction Using the Missing Data Refinement Model Based on a Long Short-Term Memory Neural Network

Atmosphere ◽

10.3390/atmos10110718 ◽

2019 ◽

Vol 10 (11) ◽

pp. 718 ◽

Cited By ~ 2

Author(s):

Park ◽

Kim ◽

Lee ◽

Kim ◽

Song ◽

...

Keyword(s):

Neural Network ◽

Missing Data ◽

Prediction Model ◽

Short Term Memory ◽

Weather Data ◽

Short Term ◽

Temperature Prediction ◽

Term Memory ◽

Proposed Model ◽

Long Short Term Memory

In this paper, we propose a new temperature prediction model based on deep learning by using real observed weather data. To this end, a huge amount of model training data is needed, but these data should not be defective. However, there is a limitation in collecting weather data since it is not possible to measure data that have been missed. Thus, the collected data are apt to be incomplete, with random or extended gaps. Therefore, the proposed temperature prediction model is used to refine missing data in order to restore missed weather data. In addition, since temperature is seasonal, the proposed model utilizes a long short-term memory (LSTM) neural network, which is a kind of recurrent neural network known to be suitable for time-series data modeling. Furthermore, different configurations of LSTMs are investigated so that the proposed LSTM-based model can reflect the time-series traits of the temperature data. In particular, when a part of the data is detected as missing, it is restored by using the proposed model’s refinement function. After all the missing data are refined, the LSTM-based model is retrained using the refined data. Finally, the proposed LSTM-based temperature prediction model can predict the temperature through three time steps: 6, 12, and 24 h. Furthermore, the model is extended to predict 7 and 14 day future temperatures. The performance of the proposed model is measured by its root-mean-squared error (RMSE) and compared with the RMSEs of a feedforward deep neural network, a conventional LSTM neural network without any refinement function, and a mathematical model currently used by the meteorological office in Korea. Consequently, it is shown that the proposed LSTM-based model employing LSTM-refinement achieves the lowest RMSEs for 6, 12, and 24 h temperature prediction as well as for 7 and 14 day temperature prediction, compared to other DNN-based and LSTM-based models with either no refinement or linear interpolation. Moreover, the prediction accuracy of the proposed model is higher than that of the Unified Model (UM) Local Data Assimilation and Prediction System (LDAPS) for 24 h temperature predictions.

Download Full-text

A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering

Computational Intelligence and Neuroscience ◽

10.1155/2019/9543490 ◽

2019 ◽

Vol 2019 ◽

pp. 1-12 ◽

Cited By ~ 4

Author(s):

Linqin Cai ◽

Sitong Zhou ◽

Xun Yan ◽

Rongdi Yuan

Keyword(s):

Neural Network ◽

Deep Learning ◽

Euclidean Distance ◽

Question Answering ◽

Short Term Memory ◽

Short Term ◽

Proposed Model ◽

Questions And Answers ◽

Textual Representation ◽

Long Short Term Memory

Deep learning is the crucial technology in intelligent question answering research tasks. Nowadays, extensive studies on question answering have been conducted by adopting the methods of deep learning. The challenge is that it not only requires an effective semantic understanding model to generate a textual representation but also needs the consideration of semantic interaction between questions and answers simultaneously. In this paper, we propose a stacked Bidirectional Long Short-Term Memory (BiLSTM) neural network based on the coattention mechanism to extract the interaction between questions and answers, combining cosine similarity and Euclidean distance to score the question and answer sentences. Experiments are tested and evaluated on publicly available Text REtrieval Conference (TREC) 8-13 dataset and Wiki-QA dataset. Experimental results confirm that the proposed model is efficient and particularly it achieves a higher mean average precision (MAR) of 0.7613 and mean reciprocal rank (MRR) of 0.8401 on the TREC dataset.

Download Full-text

Entity-Extraction Using Hybrid Deep-Learning Approach for Hindi text

International Journal of Cognitive Informatics and Natural Intelligence ◽

10.4018/ijcini.20210701.oa1 ◽

2021 ◽

Vol 15 (3) ◽

pp. 1-11

Author(s):

Richa Sharma ◽

Sudha Morwal ◽

Basant Agarwal

Keyword(s):

Neural Network ◽

Deep Learning ◽

Short Term Memory ◽

Conditional Random Field ◽

Research Work ◽

Entity Recognition ◽

Short Term ◽

Term Memory ◽

Proposed Model ◽

Long Short Term Memory

This article presents a neural network-based approach to develop named entity recognition for Hindi text. In this paper, the authors propose a deep learning architecture based on convolutional neural network (CNN) and bi-directional long short-term memory (Bi-LSTM) neural network. Skip-gram approach of word2vec model is used in the proposed model to generate word vectors. In this research work, several deep learning models have been developed and evaluated as baseline systems such as recurrent neural network (RNN), long short-term memory (LSTM), Bi-LSTM. Furthermore, these baseline systems are promoted to a proposed model with the integration of CNN and conditional random field (CRF) layers. After a comparative analysis of results, it is verified that the performance of the proposed model (i.e., Bi-LSTM-CNN-CRF) is impressive. The proposed system achieves 61% precision, 56% recall, and 58% F-measure.

Download Full-text

SACNN: Self-attentive Convolutional Neural Network Model for Natural Language Inference

ACM Transactions on Asian and Low-Resource Language Information Processing ◽

10.1145/3426884 ◽

2021 ◽

Vol 20 (3) ◽

pp. 1-16

Author(s):

Waris Quamer ◽

Praphula Kumar Jain ◽

Arpit Rai ◽

Vijayalakshmi Saravanan ◽

Rajendra Pamula ◽

...

Keyword(s):

Neural Network ◽

Natural Language ◽

Short Term Memory ◽

Local Context ◽

Short Term ◽

Proposed Model ◽

Long Short Term Memory ◽

Complex Relationships ◽

Gated Recurrent Units

Inference has been central problem for understanding and reasoning in artificial intelligence. Especially, Natural Language Inference is an interesting problem that has attracted the attention of many researchers. Natural language inference intends to predict whether a hypothesis sentence can be inferred from the premise sentence. Most prior works rely on a simplistic association between the premise and hypothesis sentence pairs, which is not sufficient for learning complex relationships between them. The strategy also fails to exploit local context information fully. Long Short Term Memory (LSTM) or gated recurrent units networks (GRU) are not effective in modeling long-term dependencies, and their schemes are far more complex as compared to Convolutional Neural Networks (CNN). To address this problem of long-term dependency, and to involve context for modeling better representation of a sentence, in this article, a general Self-Attentive Convolution Neural Network (SACNN) is presented for natural language inference and sentence pair modeling tasks. The proposed model uses CNNs to integrate mutual interactions between sentences, and each sentence with their counterparts is taken into consideration for the formulation of their representation. Moreover, the self-attention mechanism helps fully exploit the context semantics and long-term dependencies within a sentence. Experimental results proved that SACNN was able to outperform strong baselines and achieved an accuracy of 89.7% on the stanford natural language inference (SNLI) dataset.

Download Full-text