scholarly journals Processing Method of Missing Data in Dam Safety Monitoring

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Wei Wei ◽  
Chongshi Gu ◽  
Xiao Fu
Keyword(s):  
Missing Data ◽  
Short Term Memory ◽  
Safety Monitoring ◽  
Processing Method ◽  
Dam Safety ◽  
Short Term ◽  
Dam Safety Monitoring ◽  
Long Short Term Memory ◽  
Data Missing ◽  
Dam Monitoring

A large amount of data obtained by dam safety monitoring provides the basis to evaluate the dam operation state. Due to the interference caused by equipment failure and human error, it is common or even inevitable to suffer the loss of measurement data. Most of the traditional data processing methods for dam monitoring ignore the actual correlation between different measurement points, which brings difficulties to the objective diagnosis of dam safety and even leads to misdiagnosis. Therefore, it is necessary to conduct further study on how to process the missing data in dam safety monitoring. In this study, a data processing method based on partial distance combining fuzzy C-means with long short-term memory (PDS-FCM-LSTM) was proposed to deal with the data missing from dam monitoring. Based on the fuzzy clustering performed for the measurement points of the same category deployed on the dam, the membership degree of each measurement point to cluster center was described by using the fuzzy C-means clustering algorithm based on partial distance (PDS-FCM), so as to determine the clustering results and preprocess the missing data of corresponding measurement points. Then, the bidirectional long short-term memory (LSTM) network was applied to explore the pattern of changes of measurement values under identical clustering conditions, thus processing the data missing from monitoring effectively.

A Study of Missing Data Processing Method Based on Long Short Term Memory Neural Network for Electronic Health Record

2020 ◽  
Vol 10 (7) ◽  
pp. 1769-1775
Author(s):  
Xuanchi Chen ◽  
Xiangwei Zheng ◽  
Yang Yang
Keyword(s):  
Neural Network ◽  
Missing Data ◽  
Data Processing ◽  
Short Term Memory ◽  
Processing Method ◽  
Health Record ◽  
Short Term ◽  
Term Memory ◽  
Data Processing Method ◽  
Long Short Term Memory

Data-driven healthcare is considered as a promising technology of health care reform and Electronic Health Record (EHR) is an important vehicle. However, EHR are characterized by high dimensionality, temporality, sparsity and so on and it is hard for traditional deep learning algorithms to directly use sparse EHR data. In this paper, we first select the medical information mart for intensive care (MIMIC-III) database to detect the test data after patient admission within 48 hours. Then we use Long Short Term Memory Neural Network (LSTM) to learn the characteristic change model of existing data and apply the learned model to generate the missing values. Finally, the performance of the missing data processing method is verified by the prediction results of the classification model on patient mortality. Experimental results demonstrate that LSTM is an effective method for filling in missing data and the filled data based on LSTM is superior to the data filled by Linear Regression (L), K Nearest Neighbor (KNN) and Forward Padding (F) in predicting patient death outcomes.

scholarly journals Temporal Belief Memory: Imputing Missing Data during RNN Training

2018 ◽  
Author(s):  
Yeo Jin Kim ◽  
Min Chi
Keyword(s):  
Septic Shock ◽  
Missing Data ◽  
Real Time ◽  
Short Term Memory ◽  
Early Prediction ◽  
Short Term ◽  
Time Intervals ◽  
Domain Experts ◽  
Long Short Term Memory ◽  
Temporal Sequences

We propose a bio-inspired approach named Temporal Belief Memory (TBM) for handling missing data with recurrent neural networks (RNNs). When modeling irregularly observed temporal sequences, conventional RNNs generally ignore the real-time intervals between consecutive observations. TBM is a missing value imputation method that considers the time continuity and captures latent missing patterns based on irregular real time intervals of the inputs. We evaluate our TBM approach with real-world electronic health records (EHRs) consisting of 52,919 visits and 4,224,567 events on a task of early prediction of septic shock. We compare TBM against multiple baselines including both domain experts' rules and the state-of-the-art missing data handling approach using both RNN and long-short term memory. The experimental results show that TBM outperforms all the competitive baseline approaches for the septic shock early prediction task. 

The Application of a New Compromise Thresholding Method in the Safety-Monitoring of Dam Displacement

2014 ◽  
Vol 584-586 ◽  
pp. 2113-2116
Author(s):  
Gaotang Cai
Keyword(s):  
Prediction Accuracy ◽  
Safety Monitoring ◽  
Horizontal Displacement ◽  
Monitoring Data ◽  
Dam Safety ◽  
Important Work ◽  
Thresholding Method ◽  
Dam Safety Monitoring ◽  
Displacement Based ◽  
Dam Monitoring

How to identify and remove the noise of the dam monitoring data is an important work of dam safety-monitoring. The article makes use of a new compromise thresholding method to denoise the the dam horizontal displacement. Based on the de-noising data, the article uses the PSO-SVM to overfit and predict the dam horizontal displacement and compare the results with multiple regression.The result shows that the combination of the new compromise thresholding method and PSO-SVM has a better prediction accuracy.

A New Method of Dam Crack Safe-Monitoring Based on the Markov Chain

2014 ◽  
Vol 638-640 ◽  
pp. 722-725
Author(s):  
Ai Xiang Shi ◽  
Kai Zhu
Keyword(s):  
Markov Chain ◽  
Crack Width ◽  
Safety Monitoring ◽  
Regression Method ◽  
Dam Safety ◽  
Important Work ◽  
Correction Model ◽  
Dam Safety Monitoring ◽  
Multiple Regression Method ◽  
Dam Monitoring

How to identify and remove the noise of the dam monitoring data is an important work of dam safety-monitoring. The article makes use of a new compromise thres-holding method to de-noise the dam horizontal dam crack width. Afterwards, the article uses the Markov correction model to predict the crack width of the dam body based on the de-nosing data. In order to identify the effectiveness of the model, the article makes a comparison between the results of the above method with that of the multiple regression method. And the results show that the method of the article is of effective.

scholarly journals Temperature Prediction Using the Missing Data Refinement Model Based on a Long Short-Term Memory Neural Network

Atmosphere ◽  
2019 ◽  
Vol 10 (11) ◽  
pp. 718 ◽  
Author(s):  
Park ◽  
Kim ◽  
Lee ◽  
Kim ◽  
Song ◽  
...  
Keyword(s):  
Neural Network ◽  
Missing Data ◽  
Prediction Model ◽  
Short Term Memory ◽  
Weather Data ◽  
Short Term ◽  
Temperature Prediction ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory

In this paper, we propose a new temperature prediction model based on deep learning by using real observed weather data. To this end, a huge amount of model training data is needed, but these data should not be defective. However, there is a limitation in collecting weather data since it is not possible to measure data that have been missed. Thus, the collected data are apt to be incomplete, with random or extended gaps. Therefore, the proposed temperature prediction model is used to refine missing data in order to restore missed weather data. In addition, since temperature is seasonal, the proposed model utilizes a long short-term memory (LSTM) neural network, which is a kind of recurrent neural network known to be suitable for time-series data modeling. Furthermore, different configurations of LSTMs are investigated so that the proposed LSTM-based model can reflect the time-series traits of the temperature data. In particular, when a part of the data is detected as missing, it is restored by using the proposed model’s refinement function. After all the missing data are refined, the LSTM-based model is retrained using the refined data. Finally, the proposed LSTM-based temperature prediction model can predict the temperature through three time steps: 6, 12, and 24 h. Furthermore, the model is extended to predict 7 and 14 day future temperatures. The performance of the proposed model is measured by its root-mean-squared error (RMSE) and compared with the RMSEs of a feedforward deep neural network, a conventional LSTM neural network without any refinement function, and a mathematical model currently used by the meteorological office in Korea. Consequently, it is shown that the proposed LSTM-based model employing LSTM-refinement achieves the lowest RMSEs for 6, 12, and 24 h temperature prediction as well as for 7 and 14 day temperature prediction, compared to other DNN-based and LSTM-based models with either no refinement or linear interpolation. Moreover, the prediction accuracy of the proposed model is higher than that of the Unified Model (UM) Local Data Assimilation and Prediction System (LDAPS) for 24 h temperature predictions.

Sign in / Sign up

Export Citation Format

Share Document