Outlier Detection of Water Quality Data Using Ensemble Empirical Mode Decomposition

Objectives : This study was conducted to propose a new methodology for efficiently identifying and removing various outliers that occur in data collected through automated water quality monitoring systems. In the present study, water temperature data were collected from domestic G_water supply system, and the performance of the proposed methodology was tested for water temperature data collected from domestic G_water supply system.Methods : We applied the following analytical procedure to identify outliers in the water quality data: First, a normality test was performed on the collected data. If normality condition was satisfied, the Z-score was used. However, if the normality condition was not satisfied, outliers were identified using the quartile, and the limitations of the existing methodology were analyzed. Second, we decomposed the intrinsic mode function using empirical mode decomposition and ensemble empirical mode decomposition for the collected data, and then considered the occurrence of modal mixing. Finally, a group of intrinsic mode functions was selected using statistical characteristics to identify outliers. In addition, the performance of the method was verified after removing and interpolating outliers using regression analysis and Cook’s distance.Results and Discussion : In the case of water temperature data, as normality condition was not satisfied, outlier identification was carried out by applying the modified quartile method. It was confirmed that outliers distributed within the seasonal component could not be identified at all. In the case of empirical mode decomposition, modal mixing occurred because of the effect of outliers. However, in the case of the ensemble empirical mode decomposition, modal mixing was resolved and the distinct seasonal components were decomposed as intrinsic mode functions. The intrinsic mode functions were synthesized, which showed statistical correlation with the raw water temperature data. As a result of developing a regression model using the synthesized intrinsic mode functions and raw water temperature data and performing outlier search based on Cook’s distances, we concluded that various outliers distributed within the seasonal component could be effectively identified.Conclusions : Considering that satisfactory results could be derived from statistical analysis of the data collected from the automated water quality monitoring system, it can be concluded that outlier identification procedures are essential. However, in the case of the conventional univariate outlier search method, it is apparent that the outlier search performance is significantly poor for data with strong inherent variability, and the interpolation method for the searched outlier cannot be performed. Conversely, the outlier identification method based on ensemble empirical mode decomposition and regression analysis proposed in this study shows excellent discrimination performance for outliers distributed in data with strong inherent variability. Moreover, this method has the advantage of reducing the analyst’s dependence on subjective judgment by presenting statistical cutoff criteria. An additional advantage of the method is that data can be interpolated after removing outliers using intrinsic mode functions. Therefore, the outlier search and interpolation method proposed in this study is expected to have greater applicability as a more effective analysis tool compared to the existing univariate outlier search method.

Download Full-text

Ensemble Empirical Mode Decomposition Based Deep Learning Model for Short-Term Wind Power Forecasting

10.21741/9781644901731-8 ◽

2022 ◽

Author(s):

J.M. González-Sopeña

Keyword(s):

Deep Learning ◽

Wind Power ◽

Empirical Mode Decomposition ◽

Time Series Data ◽

Ensemble Empirical Mode Decomposition ◽

Series Data ◽

Intrinsic Mode Functions ◽

Wind Power Forecasting ◽

Mode Decomposition ◽

Power Forecasting

Abstract. In the last few years, wind power forecasting has established itself as an essential tool in the energy industry due to the increase of wind power penetration in the electric grid. This paper presents a wind power forecasting method based on ensemble empirical mode decomposition (EEMD) and deep learning. EEMD is employed to decompose wind power time series data into several intrinsic mode functions and a residual component. Afterwards, every intrinsic mode function is trained by means of a CNN-LSTM architecture. Finally, wind power forecast is obtained by adding the prediction of every component. Compared to the benchmark model, the proposed approach provides more accurate predictions for several time horizons. Furthermore, prediction intervals are modelled using quantile regression.

Download Full-text

Spatiotemporal Trend and Variability of Precipitation in Taiwan Based on Multi-dimensional Ensemble Empirical Mode Decomposition (MEEMD)

10.5194/egusphere-egu21-10609 ◽

2021 ◽

Author(s):

Chun-Hsiang Tang ◽

Christina W. Tsai

Keyword(s):

Time Series ◽

Empirical Mode Decomposition ◽

Ensemble Empirical Mode Decomposition ◽

Intrinsic Mode Functions ◽

Short Term ◽

Time Frequency ◽

Spatial Locality ◽

Temperature And Precipitation ◽

Mode Decomposition ◽

Climatic Scenarios

AbstractMost of the time series in nature are nonlinear and nonstationary affected by climate change particularly. It is inevitable that Taiwan has also experienced frequent drought events in recent years. However, drought events are natural disasters with no clear warnings and their influences are cumulative. The difficulty of detecting and analyzing the drought phenomenon remains. To deal with the above-mentioned problem, Multi-dimensional Ensemble Empirical Mode Decomposition (MEEMD) is introduced to analyze the temperature and rainfall data from 1975~2018 in this study, which is a powerful method developed for the time-frequency analysis of nonlinear, nonstationary time series. This method can not only analyze the spatial locality and temporal locality of signals but also decompose the multiple-dimensional time series into several Intrinsic Mode Functions (IMFs). By the set of IMFs, the meaningful instantaneous frequency and the trend of the signals can be observed. Considering stochastic and deterministic influences, to enhance the accuracy this study also reconstruct IMFs into two components, stochastic and deterministic, by the coefficient of auto-correlation.In this study, the influences of temperature and precipitation on the drought events will be discussed. Furthermore, to decrease the significant impact of drought events, this study also attempts to forecast the occurrences of drought events in the short-term via the Artificial Neural Network technique. And, based on the CMIP5 model, this study also investigates the trend and variability of drought events and warming in different climatic scenarios.&#160;Keywords: Multi-dimensional Ensemble Empirical Mode Decomposition (MEEMD), Intrinsic Mode Function(IMF), Drought

Download Full-text

Gearbox Fault Diagnosis Using Complementary Ensemble Empirical Mode Decomposition and Permutation Entropy

Shock and Vibration ◽

10.1155/2016/3891429 ◽

2016 ◽

Vol 2016 ◽

pp. 1-8 ◽

Cited By ~ 14

Author(s):

Liye Zhao ◽

Wei Yu ◽

Ruqiang Yan

Keyword(s):

Fault Diagnosis ◽

Empirical Mode Decomposition ◽

Correlation Coefficients ◽

Ensemble Empirical Mode Decomposition ◽

Permutation Entropy ◽

Support Vector ◽

Svm Classifier ◽

Intrinsic Mode Functions ◽

Vibration Signals ◽

Mode Decomposition

This paper presents an improved gearbox fault diagnosis approach by integrating complementary ensemble empirical mode decomposition (CEEMD) with permutation entropy (PE). The presented approach identifies faults appearing in a gearbox system based on PE values calculated from selected intrinsic mode functions (IMFs) of vibration signals decomposed by CEEMD. Specifically, CEEMD is first used to decompose vibration signals characterizing various defect severities into a series of IMFs. Then, filtered vibration signals are obtained from appropriate selection of IMFs, and correlation coefficients between the filtered signal and each IMF are used as the basis for useful IMFs selection. Subsequently, PE values of those selected IMFs are utilized as input features to a support vector machine (SVM) classifier for characterizing the defect severity of a gearbox. Case study conducted on a gearbox system indicates the effectiveness of the proposed approach for identifying the gearbox faults.

Download Full-text

Adaptive Ultrasound Tissue Harmonic Imaging Based on an Improved Ensemble Empirical Mode Decomposition Algorithm

Ultrasonic Imaging ◽

10.1177/0161734619900147 ◽

2020 ◽

Vol 42 (2) ◽

pp. 57-73

Author(s):

Suya Han ◽

Yufeng Zhang ◽

Keyan Wu ◽

Bingbing He ◽

Kexin Zhang ◽

...

Keyword(s):

Empirical Mode Decomposition ◽

Second Harmonic ◽

Ensemble Empirical Mode Decomposition ◽

Harmonic Imaging ◽

Motion Artifacts ◽

Intrinsic Mode Functions ◽

Tissue Harmonic Imaging ◽

Mode Decomposition ◽

Adaptive Noise ◽

Harmonic Components

Complete and accurate separation of harmonic components from the ultrasonic radio frequency (RF) echo signals is essential to improve the quality of harmonic imaging. There are limitations in the existing two commonly used separation methods, that is, the subjectivity for the high-pass filtering (S_HPF) method and motion artifacts for the pulse inversion (S_PI) method. A novel separation method called S_CEEMDAN, based on the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) algorithm, is proposed to adaptively separate the second harmonic components for ultrasound tissue harmonic imaging. First, the ensemble size of the CEEMDAN algorithm is calculated adaptively according to the standard deviation of the added white noise. A set of intrinsic mode functions (IMFs) is then obtained by the CEEMDAN algorithm from the ultrasonic RF echo signals. According to the IMF spectra, the IMFs that contain both fundamental and harmonic components are further decomposed. The separation process is performed until all the obtained IMFs have been divided into either fundamental or harmonic categories. Finally, the fundamental and harmonic RF echo signals are obtained from the accumulations of signals from these two categories, respectively. In simulation experiments based on CREANUIS, the S_CEEMDAN-based results are similar to the S_HPF-based results, but better than the S_PI-based results. For the dynamic carotid artery measurements, the contrasts, contrast-to-noise ratios (CNRs), and tissue-to-clutter ratios (TCRs) of the harmonic images based on the S_CEEMDAN are averagely increased by 31.43% and 50.82%, 18.96% and 10.83%, as well as 34.23% and 44.18%, respectively, compared with those based on the S_HPF and S_PI methods. In conclusion, the S_CEEMDAN method provides improved harmonic images owing to its good adaptivity and lower motion artifacts, and is thus a potential alternative to the current methods for ultrasonic harmonic imaging.

Download Full-text

A fault diagnosis method of rolling element bearings based on CEEMDAN

Proceedings of the Institution of Mechanical Engineers Part C Journal of Mechanical Engineering Science ◽

10.1177/0954406215624126 ◽

2015 ◽

Vol 231 (10) ◽

pp. 1804-1815 ◽

Cited By ~ 22

Author(s):

Yaguo Lei ◽

Zongyao Liu ◽

Julien Ouazri ◽

Jing Lin

Keyword(s):

Fault Diagnosis ◽

Empirical Mode Decomposition ◽

Ensemble Empirical Mode Decomposition ◽

Rolling Element Bearings ◽

Intrinsic Mode Functions ◽

Residual Noise ◽

Mode Decomposition ◽

Rolling Element ◽

Noisy Signals ◽

Diagnosis Method

Ensemble empirical mode decomposition (EEMD) represents a valuable aid in empirical mode decomposition (EMD) and has been widely used in fault diagnosis of rolling element bearings. However, the intrinsic mode functions (IMFs) generated by EEMD often contain residual noise. In addition, adding different white Gaussian noise to the signal to be analyzed probably produces a different number of IMFs, and different number of IMFs makes difficult the averaging. To alleviate these two drawbacks, complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) was previously presented. Utilizing the advantages of CEEMDAN in extracting weak characteristics from noisy signals, a new fault diagnosis method of rolling element bearings based on CEEMDAN is proposed. With this method, a particular noise is added at each stage and after each IMF extraction, a unique residue is computed. In this way, this method solves the problem of the final averaging and obtains IMFs with less noise. A simulated signal is used to illustrate the effectiveness of the proposed method, and the decomposition results show that the method obtains more accurate IMFs than the EEMD. To further demonstrate the proposed method, it is applied to fault diagnosis of locomotive rolling element bearings. The diagnosis results prove that the method based on CEEMDAN may reveal the fault characteristic information of rolling element bearings better.

Download Full-text

Applications of variational mode decomposition in seismic time-frequency analysis

Geophysics ◽

10.1190/geo2015-0489.1 ◽

2016 ◽

Vol 81 (5) ◽

pp. V365-V378 ◽

Cited By ~ 74

Author(s):

Wei Liu ◽

Siyuan Cao ◽

Yangkang Chen

Keyword(s):

Empirical Mode Decomposition ◽

Frequency Analysis ◽

Synthetic Data ◽

Ensemble Empirical Mode Decomposition ◽

Variational Mode Decomposition ◽

Intrinsic Mode Functions ◽

Decomposition Approach ◽

Time Frequency Analysis ◽

Time Frequency ◽

Mode Decomposition

We have introduced a novel time-frequency decomposition approach for analyzing seismic data. This method is inspired by the newly developed variational mode decomposition (VMD). The principle of VMD is to look for an ensemble of modes with their respective center frequencies, such that the modes collectively reproduce the input signal and each mode is smooth after demodulation into baseband. The advantage of VMD is that there is no residual noise in the modes and it can further decrease redundant modes compared with the complete ensemble empirical mode decomposition (CEEMD) and improved CEEMD (ICEEMD). Moreover, VMD is an adaptive signal decomposition technique, which can nonrecursively decompose a multicomponent signal into several quasi-orthogonal intrinsic mode functions. This new tool, in contrast to empirical mode decomposition (EMD) and its variations, such as EEMD, CEEMD, and ICEEMD, is based on a solid mathematical foundation and can obtain a time-frequency representation that is less sensitive to noise. Two tests on synthetic data showed the effectiveness of our VMD-based time-frequency analysis method. Application on field data showed the potential of the proposed approach in highlighting geologic characteristics and stratigraphic information effectively. All the performances of the VMD-based approach were compared with those from the CEEMD- and ICEEMD-based approaches.

Download Full-text

Trends analysis of precipitation data over the tropical South-West Indian Ocean (SWIO) basin using the Ensemble Empirical Mode Decomposition (EEMD) method

MAUSAM ◽

10.54302/mausam.v67i2.1341 ◽

2021 ◽

Vol 67 (2) ◽

pp. 423-430

Author(s):

K. BOODHOO ◽

M. R. LOLLCHUND ◽

A. F. DILMAHAMOD

Keyword(s):

Time Series ◽

Indian Ocean ◽

Empirical Mode Decomposition ◽

Ensemble Empirical Mode Decomposition ◽

Intrinsic Mode Functions ◽

South West Indian Ocean ◽

South West ◽

Mode Decomposition ◽

West Indian Ocean ◽

Eemd Method

In this paper, we propose the use of the Ensemble Empirical Mode Decomposition (EEMD) method in the analysis of trends in climate data. As compared to existing traditional methods, EEMD is simple, fast and reliable. It works by decomposing the time-series data into intrinsic mode functions until a residual component is obtained which represents the trend in the data. The dataset considered consists of satellite precipitation estimates (SPE) obtained from the Tropical Rainfall Measuring Mission (TRMM) for the tropical South-West Indian Ocean (SWIO) basin recorded during the periods January 1998 to December 2013. The SWIO basin spans from the latitudes 5° S to 35° S and the longitudes 30° E to 70° E and comprises of part of the east coast of Africa and some small island developing states (SIDS) such as Comoros, Madagascar, Mauritius and Reunion Island. The EEMD analysis is carried out for summer, winter and yearly time series of the SPE data. The results from the study are presented in terms of intrinsic mode functions (IMFs) and the trends. The analysis reveals that in summer, there is a tendency to have an increase in the amount of rainfall, whereas in winter, from 1998 to 2004 there has been an initial increase of 0.0022 mm/hr/year and from there onwards till 2013 a decrease of 0.00052 mm/hr/year was noted.

Download Full-text

Denoising the ECG Signal Using Ensemble Empirical Mode Decomposition

Engineering, Technology & Applied Science Research ◽

10.48084/etasr.4302 ◽

2021 ◽

Vol 11 (5) ◽

pp. 7536-7541

Author(s):

W. Mohguen ◽

S. Bouguezel

Keyword(s):

Empirical Mode Decomposition ◽

Additive White Gaussian Noise ◽

Ensemble Empirical Mode Decomposition ◽

Ecg Signal ◽

Mean Square ◽

Intrinsic Mode Functions ◽

Denoising Method ◽

Mode Decomposition ◽

Mean Square Difference ◽

Thresholding Function

In this paper, a novel electrocardiogram (ECG) denoising method based on the Ensemble Empirical Mode Decomposition (EEMD) is proposed by introducing a modified customized thresholding function. The basic principle of this method is to decompose the noisy ECG signal into a series of Intrinsic Mode Functions (IMFs) using the EEMD algorithm. Moreover, a modified customized thresholding function was adopted for reducing the noise from the ECG signal and preserve the QRS complexes. The denoised signal was reconstructed using all thresholded IMFs. Real ECG signals having different Additive White Gaussian Noise (AWGN) levels were employed from the MIT-BIH database to evaluate the performance of the proposed method. For this purpose, output SNR (SNRout), Mean Square Error (MSE), and Percentage Root mean square Difference (PRD) parameters were used at different input SNRs (SNRin). The simulation results showed that the proposed method provided significant improvements over existing denoising methods.

Download Full-text

Fault Diagnosis of Rolling Bearing based on an Improved Denoising Technique using Complete Ensemble Empirical Mode Decomposition and Adaptive Thresholding Method

10.21203/rs.3.rs-159429/v1 ◽

2021 ◽

Author(s):

Prashant Kumar Sahu ◽

Rajiv Nandan Rai

Keyword(s):

Empirical Mode Decomposition ◽

Signal Reconstruction ◽

Low Frequency ◽

Rolling Bearing ◽

Ensemble Empirical Mode Decomposition ◽

Adaptive Thresholding ◽

Intrinsic Mode Functions ◽

Vibration Signals ◽

Mode Decomposition ◽

Envelope Spectrum

Abstract The vibration signals for rotating machines are generally polluted by excessive noise and can lose the fault information at the early development phase. In this paper, an improved denoising technique is proposed for early faults diagnosis of rolling bearing based on the complete ensemble empirical mode decomposition (CEEMD) and adaptive thresholding (ATD) method. Firstly, the bearing vibration signals are decomposed into a set of various intrinsic mode functions (IMFs) using CEEMD algorithm. The IMFs grouping and selection are formed based upon the correlation coefficient value. The noise-predominant IMFs are subjected to adaptive thresholding for denoising and then added to the low-frequency IMFs for signal reconstruction. The effectiveness of the proposed method denoised signals are measured based on kurtosis value and the envelope spectrum analysis. The presented method results on experimental datasets illustrate that the proposed approach is an effective denoising technique for early fault detection in the rolling bearing.

Download Full-text

Noise Suppression of Microseismic Signals via Adaptive Variational Mode Decomposition and Akaike Information Criterion

Applied Sciences ◽

10.3390/app10113790 ◽

2020 ◽

Vol 10 (11) ◽

pp. 3790 ◽

Cited By ~ 1

Author(s):

Jinyong Zhang ◽

Linlu Dong ◽

Nuwen Xu

Keyword(s):

Empirical Mode Decomposition ◽

Akaike Information Criterion ◽

Noise Suppression ◽

Information Criterion ◽

Ensemble Empirical Mode Decomposition ◽

Frequency Noise ◽

Variational Mode Decomposition ◽

Intrinsic Mode Functions ◽

Denoising Method ◽

Mode Decomposition

Microseismic (MS) signals recorded by sensors are often mixed with various noise, which produce some interference to the further analysis of the collected data. One problem of many existing noise suppression methods is to deal with noisy signals in a unified strategy, which results in low-frequency noise in the non-microseismic section remaining. Based on this, we have developed a novel MS denoising method combining variational mode decomposition (VMD) and Akaike information criterion (AIC). The method first applied VMD to decompose a signal into several limited-bandwidth intrinsic mode functions and adaptively determined the effective components by the difference of correlation coefficient. After reconstructing, the improved AIC method was used to determine the location of the valuable waveform, and the residual fluctuations in other positions were further removed. A synthetic wavelet signal and some synthetic MS signals with different signal-to-noise ratios (SNRs) were used to test its denoising effect with ensemble empirical mode decomposition (EEMD), complete ensemble empirical mode decomposition (CEEMD), and the VMD method. The experimental results depicted that the SNRs of the proposed method were obviously larger than that of other methods, and the waveform and spectrum became cleaner based on VMD. The processing results of the MS signal of Shuangjiangkou Hydropower Station also illustrated its good denoising ability and robust performance to signals with different characteristics.

Download Full-text