scholarly journals Research on the Fastest Detection Method for Weak Trends under Noise Interference

Entropy ◽  
2021 ◽  
Vol 23 (8) ◽  
pp. 1093
Author(s):  
Guang Li ◽  
Jing Liang ◽  
Caitong Yue
Keyword(s):  
Anomaly Detection ◽  
Data Streams ◽  
Concept Drift ◽  
Detection Algorithm ◽  
Detection Accuracy ◽  
Industrial Data ◽  
Oil Drilling ◽  
Noise Interference ◽  
Detection Score ◽  
Weak Trend

Trend anomaly detection is the practice of comparing and analyzing current and historical data trends to detect real-time abnormalities in online industrial data-streams. It has the advantages of tracking a concept drift automatically and predicting trend changes in the shortest time, making it important both for algorithmic research and industry. However, industrial data streams contain considerable noise that interferes with detecting weak anomalies. In this paper, the fastest detection algorithm “sliding nesting” is adopted. It is based on calculating the data weight in each window by applying variable weights, while maintaining the method of trend-effective integration accumulation. The new algorithm changes the traditional calculation method of the trend anomaly detection score, which calculates the score in a short window. This algorithm, SNWFD–DS, can detect weak trend abnormalities in the presence of noise interference. Compared with other methods, it has significant advantages. An on-site oil drilling data test shows that this method can significantly reduce delays compared with other methods and can improve the detection accuracy of weak trend anomalies under noise interference.

Study for Detection Algorithm of QRS Complex in ECG Signal

2013 ◽  
Vol 765-767 ◽  
pp. 2105-2108
Author(s):  
Xu Wen Li ◽  
Bi Wei Zhang ◽  
Qiang Wu
Keyword(s):  
Wavelet Transform ◽  
Detection Algorithm ◽  
Automatic Analysis ◽  
Detection Accuracy ◽  
Ecg Signal ◽  
Good Precision ◽  
Qrs Complex ◽  
Noise Interference ◽  
Wavelet Transform Modulus Maxima ◽  
Modulus Maxima

In ECG signals accurate detection to the position of QRS complex is a key to automatic analysis and diagnosis system. And its premise is that effectively remove all kinds of noise interference in ECG signal. Here, a method of detecting QRS based on EMD and wavelet transform was presented which is aim to improve the anti-noise performance of the detection algorithm. It is combined EMD with the theory of singularity detecting based on wavelet transform modulus maxima method. It has the high detection accuracy and good precision that can give an effective way to the automatic analysis for ECG signal.

scholarly journals A Novel Active Anomaly Discovery Method and its Applications in Additive Manufacturing

2021 ◽  
Author(s):  
Bo Shen ◽  
Zhenyu Kong
Keyword(s):  
Additive Manufacturing ◽  
Anomaly Detection ◽  
State Of The Art ◽  
Detection Algorithm ◽  
The State ◽  
Detection Accuracy ◽  
Detection Algorithms ◽  
Detection Analysis ◽  
Unsupervised Anomaly Detection ◽  
Discovery Method

Anomaly detection aims to identify the true anomalies from a given set of data instances. Unsupervised anomaly detection algorithms are applied to an unlabeled dataset by producing a ranked list based on anomaly scores. Unfortunately, due to the inherent limitations, many of the top-ranked instances by unsupervised algorithms are not anomalies or not interesting from an application perspective, which leads to high false-positive rates. Active anomaly discovery (AAD) is proposed to overcome this deficiency, which sequentially selects instances to get the labeling information and incorporate it into the anomaly detection algorithm to improve the detection accuracy iteratively. However, labeling is often costly. Therefore, the way to balance detection accuracy and labeling cost is essential. Along this line, this paper proposes a novel AAD method to achieve the goal. Our approach is based on the state-of-the-art unsupervised anomaly detection algorithm, namely, Isolation Forest, to extract features. Thereafter, the sparsity of the extracted features is utilized to improve its anomaly detection performance. To enforce the sparsity of the features and subsequent improvement of the detection analysis, a new algorithm based on online gradient descent, namely, Sparse Approximated Linear Anomaly Discovery (SALAD), is proposed with its theoretical Regret analysis. Extensive experiments on both open-source and additive manufacturing datasets demonstrate that the proposed algorithm significantly outperforms the state-of-the-art algorithms for anomaly detection.

scholarly journals A Hyperspectral Anomaly Detection Algorithm Based on Morphological Profile and Attribute Filter with Band Selection and Automatic Determination of Maximum Area

2020 ◽  
Vol 12 (20) ◽  
pp. 3387
Author(s):  
Ferdi Andika ◽  
Mia Rizkinia ◽  
Masahiro Okuda
Keyword(s):  
Anomaly Detection ◽  
Computing Time ◽  
Area Under The Curve ◽  
Detection Algorithm ◽  
High Spectral Resolution ◽  
Band Selection ◽  
Detection Accuracy ◽  
Maximum Area ◽  
Detection Algorithms ◽  
Rich Information

Anomaly detection is one of the most challenging topics in hyperspectral imaging due to the high spectral resolution of the images and the lack of spatial and spectral information about the anomaly. In this paper, a novel hyperspectral anomaly detection method called morphological profile and attribute filter (MPAF) algorithm is proposed. Aiming to increase the detection accuracy and reduce computing time, it consists of three steps. First, select a band containing rich information for anomaly detection using a novel band selection algorithm based on entropy and histogram counts. Second, remove the background of the selected band with morphological profile. Third, filter the false anomalous pixels with attribute filter. A novel algorithm is also proposed in this paper to define the maximum area of anomalous objects. Experiments were run on real hyperspectral datasets to evaluate the performance, and analysis was also conducted to verify the contribution of each step of MPAF. The results show that the performance of MPAF yields competitive results in terms of average area under the curve (AUC) for receiver operating characteristic (ROC), precision-recall, and computing time, i.e., 0.9916, 0.7055, and 0.25 s, respectively. Compared with four other anomaly detection algorithms, MPAF yielded the highest average AUC for ROC and precision-recall in eight out of thirteen and nine out of thirteen datasets, respectively. Further analysis also proved that each step of MPAF has its effectiveness in the detection performance.

scholarly journals An Anomaly Detection Algorithm of Cloud Platform Based on Self-Organizing Maps

2016 ◽  
Vol 2016 ◽  
pp. 1-9 ◽  
Author(s):  
Jun Liu ◽  
Shuyu Chen ◽  
Zhen Zhou ◽  
Tianshu Wu
Keyword(s):  
Anomaly Detection ◽  
Virtual Machine ◽  
Large Scale ◽  
Virtual Machines ◽  
Detection Algorithm ◽  
Detection Accuracy ◽  
Cloud Platform ◽  
Self Organizing Maps ◽  
Detection Speed ◽  
Self Organizing

Virtual machines (VM) on a Cloud platform can be influenced by a variety of factors which can lead to decreased performance and downtime, affecting the reliability of the Cloud platform. Traditional anomaly detection algorithms and strategies for Cloud platforms have some flaws in their accuracy of detection, detection speed, and adaptability. In this paper, a dynamic and adaptive anomaly detection algorithm based on Self-Organizing Maps (SOM) for virtual machines is proposed. A unified modeling method based on SOM to detect the machine performance within the detection region is presented, which avoids the cost of modeling a single virtual machine and enhances the detection speed and reliability of large-scale virtual machines in Cloud platform. The important parameters that affect the modeling speed are optimized in the SOM process to significantly improve the accuracy of the SOM modeling and therefore the anomaly detection accuracy of the virtual machine.

scholarly journals A Novel Active Anomaly Discovery Method and its Applications in Additive Manufacturing

2021 ◽  
Author(s):  
Bo Shen ◽  
Zhenyu Kong
Keyword(s):  
Additive Manufacturing ◽  
Anomaly Detection ◽  
State Of The Art ◽  
Detection Algorithm ◽  
The State ◽  
Detection Accuracy ◽  
Detection Algorithms ◽  
Detection Analysis ◽  
Unsupervised Anomaly Detection ◽  
Discovery Method

Anomaly detection aims to identify the true anomalies from a given set of data instances. Unsupervised anomaly detection algorithms are applied to an unlabeled dataset by producing a ranked list based on anomaly scores. Unfortunately, due to the inherent limitations, many of the top-ranked instances by unsupervised algorithms are not anomalies or not interesting from an application perspective, which leads to high false-positive rates. Active anomaly discovery (AAD) is proposed to overcome this deficiency, which sequentially selects instances to get the labeling information and incorporate it into the anomaly detection algorithm to improve the detection accuracy iteratively. However, labeling is often costly. Therefore, the way to balance detection accuracy and labeling cost is essential. Along this line, this paper proposes a novel AAD method to achieve the goal. Our approach is based on the state-of-the-art unsupervised anomaly detection algorithm, namely, Isolation Forest, to extract features. Thereafter, the sparsity of the extracted features is utilized to improve its anomaly detection performance. To enforce the sparsity of the features and subsequent improvement of the detection analysis, a new algorithm based on online gradient descent, namely, Sparse Approximated Linear Anomaly Discovery (SALAD), is proposed with its theoretical Regret analysis. Extensive experiments on both open-source and additive manufacturing datasets demonstrate that the proposed algorithm significantly outperforms the state-of-the-art algorithms for anomaly detection.

scholarly journals Anomalies Detection Using Isolation in Concept-Drifting Data Streams

Computers ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 13
Author(s):  
Maurras Ulbricht Togbe ◽  
Yousra Chabchoub ◽  
Aliou Boly ◽  
Mariam Barry ◽  
Raja Chiky ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Half Space ◽  
Data Streams ◽  
Detection Efficiency ◽  
Concept Drift ◽  
Streaming Data ◽  
Detection Methods ◽  
Data Sets ◽  
Stream Data ◽  
Isolation Forest

Detecting anomalies in streaming data is an important issue for many application domains, such as cybersecurity, natural disasters, or bank frauds. Different approaches have been designed in order to detect anomalies: statistics-based, isolation-based, clustering-based, etc. In this paper, we present a structured survey of the existing anomaly detection methods for data streams with a deep view on Isolation Forest (iForest). We first provide an implementation of Isolation Forest Anomalies detection in Stream Data (IForestASD), a variant of iForest for data streams. This implementation is built on top of scikit-multiflow (River), which is an open source machine learning framework for data streams containing a single anomaly detection algorithm in data streams, called Streaming half-space trees. We performed experiments on different real and well known data sets in order to compare the performance of our implementation of IForestASD and half-space trees. Moreover, we extended the IForestASD algorithm to handle drifting data by proposing three algorithms that involve two main well known drift detection methods: ADWIN and KSWIN. ADWIN is an adaptive sliding window algorithm for detecting change in a data stream. KSWIN is a more recent method and it refers to the Kolmogorov–Smirnov Windowing method for concept drift detection. More precisely, we extended KSWIN to be able to deal with n-dimensional data streams. We validated and compared all of the proposed methods on both real and synthetic data sets. In particular, we evaluated the F1-score, the execution time, and the memory consumption. The experiments show that our extensions have lower resource consumption than the original version of IForestASD with a similar or better detection efficiency.

scholarly journals A Novel Drift Detection Algorithm Based on Features’ Importance Analysis in a Data Streams Environment

2020 ◽  
Vol 10 (4) ◽  
pp. 287-298
Author(s):  
Piotr Duda ◽  
Krzysztof Przybyszewski ◽  
Lipo Wang
Keyword(s):  
Random Forest ◽  
Data Streams ◽  
Data Stream ◽  
Concept Drift ◽  
Ensemble Methods ◽  
Real Data ◽  
Relevant Information ◽  
Detection Algorithm ◽  
Important Indicator ◽  
Features Importance

AbstractThe training set consists of many features that influence the classifier in different degrees. Choosing the most important features and rejecting those that do not carry relevant information is of great importance to the operating of the learned model. In the case of data streams, the importance of the features may additionally change over time. Such changes affect the performance of the classifier but can also be an important indicator of occurring concept-drift. In this work, we propose a new algorithm for data streams classification, called Random Forest with Features Importance (RFFI), which uses the measure of features importance as a drift detector. The RFFT algorithm implements solutions inspired by the Random Forest algorithm to the data stream scenarios. The proposed algorithm combines the ability of ensemble methods for handling slow changes in a data stream with a new method for detecting concept drift occurrence. The work contains an experimental analysis of the proposed algorithm, carried out on synthetic and real data.

Sign in / Sign up

Export Citation Format

Share Document