scholarly journals Remaining Useful Life Prediction Using Temporal Convolution with Attention

AI ◽  
2021 ◽  
Vol 2 (1) ◽  
pp. 48-70
Author(s):  
Wei Ming Tan ◽  
T. Hui Teo

Prognostic techniques attempt to predict the Remaining Useful Life (RUL) of a subsystem or a component. Such techniques often use sensor data which are periodically measured and recorded into a time series data set. Such multivariate data sets form complex and non-linear inter-dependencies through recorded time steps and between sensors. Many current existing algorithms for prognostic purposes starts to explore Deep Neural Network (DNN) and its effectiveness in the field. Although Deep Learning (DL) techniques outperform the traditional prognostic algorithms, the networks are generally complex to deploy or train. This paper proposes a Multi-variable Time Series (MTS) focused approach to prognostics that implements a lightweight Convolutional Neural Network (CNN) with attention mechanism. The convolution filters work to extract the abstract temporal patterns from the multiple time series, while the attention mechanisms review the information across the time axis and select the relevant information. The results suggest that the proposed method not only produces a superior accuracy of RUL estimation but it also trains many folds faster than the reported works. The superiority of deploying the network is also demonstrated on a lightweight hardware platform by not just being much compact, but also more efficient for the resource restricted environment.

2021 ◽  
Vol 13 (1) ◽  
Author(s):  
Mahbubul Alam ◽  
Laleh Jalali ◽  
Mahbubul Alam ◽  
Ahmed Farahat ◽  
Chetan Gupta

Abstract—Prognostics aims to predict the degradation of equipment by estimating their remaining useful life (RUL) and/or the failure probability within a specific time horizon. The high demand of equipment prognostics in the industry have propelled researchers to develop robust and efficient prognostics techniques. Among data driven techniques for prognostics, machine learning and deep learning (DL) based techniques, particularly Recurrent Neural Networks (RNNs) have gained significant attention due to their ability of effectively representing the degradation progress by employing dynamic temporal behaviors. RNNs are well known for handling sequential data, especially continuous time series sequential data where the data follows certain pattern. Such data is usually obtained from sensors attached to the equipment. However, in many scenarios sensor data is not readily available and often very tedious to acquire. Conversely, event data is more common and can easily be obtained from the error logs saved by the equipment and transmitted to a backend for further processing. Nevertheless, performing prognostics using event data is substantially more difficult than that of the sensor data due to the unique nature of event data. Though event data is sequential, it differs from other seminal sequential data such as time series and natural language in the following manner, i) unlike time series data, events may appear at any time, i.e., the appearance of events lacks periodicity; ii) unlike natural languages, event data do not follow any specific linguistic rule. Additionally, there may be a significant variability in the event types appearing within the same sequence.  Therefore, this paper proposes an RUL estimation framework to effectively handle the intricate and novel event data. The proposed framework takes discrete events generated by an equipment (e.g., type, time, etc.) as input, and generates for each new event an estimate of the remaining operating cycles in the life of a given component. To evaluate the efficacy of our proposed method, we conduct extensive experiments using benchmark datasets such as the CMAPSS data after converting the time-series data in these datasets to sequential event data. The event data conversion is carried out by careful exploration and application of appropriate transformation techniques to the time series. To the best of our knowledge this is the first time such event-based RUL estimation problem is introduced to the community. Furthermore, we propose several deep learning and machine learning based solution for the event-based RUL estimation problem. Our results suggest that the deep learning models, 1D-CNN, LSTM, and multi-head attention show similar RMSE, MAE and Score performance. Foreseeably, the XGBoost model achieve lower performance compared to the deep learning models since the XGBoost model fails to capture ordering information from the sequence of events. 


Processes ◽  
2021 ◽  
Vol 9 (7) ◽  
pp. 1115
Author(s):  
Gilseung Ahn ◽  
Hyungseok Yun ◽  
Sun Hur ◽  
Si-Yeong Lim

Accurate predictions of remaining useful life (RUL) of equipment using machine learning (ML) or deep learning (DL) models that collect data until the equipment fails are crucial for maintenance scheduling. Because the data are unavailable until the equipment fails, collecting sufficient data to train a model without overfitting can be challenging. Here, we propose a method of generating time-series data for RUL models to resolve the problems posed by insufficient data. The proposed method converts every training time series into a sequence of alphabetical strings by symbolic aggregate approximation and identifies occurrence patterns in the converted sequences. The method then generates a new sequence and inversely transforms it to a new time series. Experiments with various RUL prediction datasets and ML/DL models verified that the proposed data-generation model can help avoid overfitting in RUL prediction model.


2019 ◽  
Author(s):  
Srishti Mishra ◽  
Zohair Shafi ◽  
Santanu Pathak

Data driven decision making is becoming increasingly an important aspect for successful business execution. More and more organizations are moving towards taking informed decisions based on the data that they are generating. Most of this data are in temporal format - time series data. Effective analysis across time series data sets, in an efficient and quick manner is a challenge. The most interesting and valuable part of such analysis is to generate insights on correlation and causation across multiple time series data sets. This paper looks at methods that can be used to analyze such data sets and gain useful insights from it, primarily in the form of correlation and causation analysis. This paper focuses on two methods for doing so, Two Sample Test with Dynamic Time Warping and Hierarchical Clustering and looks at how the results returned from both can be used to gain a better understanding of the data. Moreover, the methods used are meant to work with any data set, regardless of the subject domain and idiosyncrasies of the data set, primarily, a data agnostic approach.


Author(s):  
Baoquan Wang ◽  
Tonghai Jiang ◽  
Xi Zhou ◽  
Bo Ma ◽  
Fan Zhao ◽  
...  

For abnormal detection of time series data, the supervised anomaly detection methods require labeled data. While the range of outlier factors used by the existing semi-supervised methods varies with data, model and time, the threshold for determining abnormality is difficult to obtain, in addition, the computational cost of the way to calculate outlier factors from other data points in the data set is also very large. These make such methods difficult to practically apply. This paper proposes a framework named LSTM-VE which uses clustering combined with visualization method to roughly label normal data, and then uses the normal data to train long short-term memory (LSTM) neural network for semi-supervised anomaly detection. The variance error (VE) of the normal data category classification probability sequence is used as outlier factor. The framework enables anomaly detection based on deep learning to be practically applied and using VE avoids the shortcomings of existing outlier factors and gains a better performance. In addition, the framework is easy to expand because the LSTM neural network can be replaced with other classification models. Experiments on the labeled and real unlabeled data sets prove that the framework is better than replicator neural networks with reconstruction error (RNN-RS) and has good scalability as well as practicability.


Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7109
Author(s):  
Chengying Zhao ◽  
Xianzhen Huang ◽  
Yuxiong Li ◽  
Muhammad Yousaf Iqbal

In recent years, prognostic and health management (PHM) has played an important role in industrial engineering. Efficient remaining useful life (RUL) prediction can ensure the development of maintenance strategies and reduce industrial losses. Recently, data-driven based deep learning RUL prediction methods have attracted more attention. The convolution neural network (CNN) is a kind of deep neural network widely used in RUL prediction. It shows great potential for application in RUL prediction. A CNN is used to extract the features of time-series data according to the spatial feature method. This way of processing features without considering the time dimension will affect the prediction accuracy of the model. On the contrary, the commonly used long short-term memory (LSTM) network considers the timing of the data. However, compared with CNN, it lacks spatial data extraction capabilities. This paper proposes a double-channel hybrid prediction model based on the CNN and a bidirectional LSTM network to avoid those drawbacks. The sliding time window is used for data preprocessing, and an improved piece-wise linear function is used for model validating. The prediction model is evaluated using the C-MAPSS dataset provided by NASA. The predicted results show the proposed prediction model to have a better prediction performance compared with other state-of-the-art models.


Author(s):  
Pradeep Lall ◽  
Tony Thomas ◽  
Ken Blecker

Abstract This study focuses on the feature vector identification and Remaining Useful Life (RUL) estimation of SAC305 solder alloy PCB's of two different configurations during varying conditions of temperature and vibration. The feature vectors are identified using the strain signals acquired from four symmetrical locations of the PCB at regular intervals during vibration. Two different types of experiments are employed to characterize the PCB's dynamic changes with varying temperature and acceleration levels. The strain signals acquired during each of these experiments are compared based on both time and frequency domain characteristics. Different statistical and frequency-based techniques were used to identify the strain signal variations with changes in the environment and loading conditions. The feature vectors in predicting failure at a constant working temperature and load were identified, and as an extension to this work, the effectiveness of the feature vectors during varying conditions of temperature and acceleration levels are investigated. The remaining Useful Life of the packages was estimated using a deep learning approach based on Long Short Term Memory (LSTM) network. This technique can identify the underlying patterns in multivariate time series data that can predict the packages' life. The autocorrelation function's residuals were used as the multivariate time series data in conjunction with the LSTM deep learning technique to forecast the packages' life at different varying temperatures and acceleration levels during vibration.


2021 ◽  
Author(s):  
Yong-Keun Park ◽  
Min-Kyung Kim ◽  
Jumyung Um

Abstract The research on predictive maintenance of rotating machines, the most important element in manufacturing facilities, has been very active. The widespread availability of smart factory solutions has led to improved data collection from machines and processes and is able to provide key information. For our purpose, the collected information enables the maintenance system to predict the remaining useful life using deep learning models. The introduction of multi-layer perceptron of signal processing originating from bearings, in time series data, has been discussed in many publications. However, estimating accuracy for the remaining useful life is determined by the selection of the feature domain and the concatenation network model. Herein, we introduce a convolutional Autoencoder based on multi-domain ensemble learning in order to include various feature domains and a concatenation network operated by latent space into a single neural network. The performance of the proposed model is evaluated by using a simple health indicator and a PRONOSTIA dataset and compared with a simple concatenation model, 2-stage Autoencoder, and a recurrent neural network.


2020 ◽  
Vol 3 (1) ◽  
pp. 23-30
Author(s):  
Nurfia Oktaviani Syamsiah ◽  
Indah Purwandani

Time series data is interesting research material for many people. Not a few models have been produced, but very optimal accuracy has not been obtained. Neural network is one that is widely used because of its ability to understand non-linear relationships between data. This study will combine a neural network with exponential smoothing to produce higher accuracy. Exponential smoothing is one of the best linear methods is used for data set transformation and thereafter the new data set will be used in training and testing the Neural Network model. The resulting model will be evaluated using the standard error measure Root Mean Square Error (RMSE). Each model was compared with its RMSE value and then performed a T-Test. The proposed ES-NN model proved to have better predictive results than using only one method.


Sign in / Sign up

Export Citation Format

Share Document