scholarly journals Research on Outlier Detection Algorithm for Evaluation of Battery System Safety

2014 ◽  
Vol 6 ◽  
pp. 830402 ◽  
Author(s):  
Changhao Piao ◽  
Zhi Huang ◽  
Ling Su ◽  
Sheng Lu
Keyword(s):  
Outlier Detection ◽  
Data Stream ◽  
Concept Drift ◽  
Detection Algorithm ◽  
High Dimensional ◽  
Small Scale ◽  
Data Sets ◽  
Angle Distribution ◽  
System Safety ◽  
Battery System

Battery system is the key part of the electric vehicle. To realize outlier detection in the running process of battery system effectively, a new high-dimensional data stream outlier detection algorithm (DSOD) based on angle distribution is proposed. First, in order to improve the algorithm stability in high-dimensional space, the method of angle distribution-based outlier detection algorithm is employed. Second, to reduce the computational complexity, a small-scale calculation set of data stream is established, which is composed of normal set and border set. For the purpose of solving the problem of concept drift, an update mechanism for the normal set and border set is developed in this paper. By this way, these hidden abnormal points will be rapidly detected. The experimental results on real data sets and battery system simulation data sets demonstrate that DSOD is more efficient than Simple variance of angles (Simple VOA) and angle-based outlier detection (ABOD) and is very suitable for the evaluation of battery system safety.

A Data Stream Outlier Detection Algorithm Based on Reverse K Nearest Neighbors

2011 ◽  
Vol 225-226 ◽  
pp. 1032-1035 ◽  
Author(s):  
Zhong Ping Zhang ◽  
Yong Xin Liang
Keyword(s):  
Outlier Detection ◽  
Data Stream ◽  
Concept Drift ◽  
Real Data ◽  
Nearest Neighbors ◽  
Detection Algorithm ◽  
Data Sets ◽  
K Nearest Neighbors ◽  
Query Manager ◽  
Current Window

This paper proposes a new data stream outlier detection algorithm SODRNN based on reverse nearest neighbors. We deal with the sliding window model, where outlier queries are performed in order to detect anomalies in the current window. The update of insertion or deletion only needs one scan of the current window, which improves efficiency. The capability of queries at arbitrary time on the whole current window is achieved by Query Manager Procedure, which can capture the phenomenon of concept drift of data stream in time. Results of experiments conducted on both synthetic and real data sets show that SODRNN algorithm is both effective and efficient.

scholarly journals Outlier Detection Algorithm Basing on Similarity Measurement Relation

2012 ◽  
Vol 6-7 ◽  
pp. 621-624
Author(s):  
Hong Bin Fang
Keyword(s):  
Outlier Detection ◽  
Credit Card ◽  
High Dimensional Data ◽  
Detection Algorithm ◽  
Experimental Result ◽  
Similarity Measurement ◽  
High Dimensional ◽  
Data Set ◽  
Network Intrusion ◽  
Metric Function

Outlier detection is an important field of data mining, which is widely used in credit card fraud detection, network intrusion detection ,etc. A kind of high dimensional data similarity metric function and the concept of class density are given in the paper, basing on the combination of hierarchical clustering and similarity, as well as outlier detection algorithm about similarity measurement is presented after the redefinition of high dimension density outliers is put. The algorithm has some value for outliers detection of high dimensional data set in view of experimental result.

scholarly journals Detecting Outliers in High Dimensional Data Sets using Z-Score Methodology

2019 ◽  
Vol 9 (1) ◽  
pp. 48-53
Keyword(s):  
Outlier Detection ◽  
Credit Card ◽  
High Dimensional Data ◽  
Research Area ◽  
High Dimensional ◽  
Data Sets ◽  
Z Score ◽  
Wide Range ◽  
Efficiency And Effectiveness ◽  
Projected Methods

Outlier detection is an interesting research area in machine learning. With the recently emergent tools and varied applications, the attention of outlier recognition is growing significantly. Recently, a significant number of outlier detection approaches have been observed and effectively applied in a wide range of fields, comprising medical health, credit card fraud and intrusion detection. They can be utilized for conservative data analysis. However, Outlier recognition aims to discover sequence in data that do not conform to estimated performance. In this paper, we presented a statistical approach called Z-score method for outlier recognition in high-dimensional data. Z-scores is a novel method for deciding distant data based on data positions on charts. The projected method is computationally fast and robust to outliers’ recognition. A comparative Analysis with extant methods is implemented with high dimensional datasets. Exploratory outcomes determines an enhanced accomplishment, efficiency and effectiveness of our projected methods.

An Efficient Method to Detect Outliers in High Dimensional Data

2019 ◽  
Vol 16 (9) ◽  
pp. 3938-3944
Author(s):  
Atul Garg ◽  
Kamaljeet Kaur
Keyword(s):  
Outlier Detection ◽  
High Dimensional Data ◽  
Detection Algorithm ◽  
High Dimensional ◽  
The Novel ◽  
Linear Discriminant ◽  
Detection Techniques ◽  
Detection Algorithms ◽  
Large Sets ◽  
Detection Of Outliers

In this era, detection of outliers or anomalies from high dimensional data is really a great challenge. Normal data is distinguished from data containing anomalies using Outlier detection techniques which classifies new data as normal or abnormal. Different Outlier Detection algorithms are proposed by many researchers for high dimensional data and each algorithm has its own benefits and limitations. In the literature the researchers proposed different algorithms. For this work few algorithms such as Dice-Coefficient Index (DCI), Mapreduce Function and Linear Discriminant Analysis Algorithm (LDA) are considered. Mapreduce function is used to overcome the problem of large datasets. LDA is basically used in the reduction of the data dimensionality. In the present work a novel Hybrid Outlier Detection Algorithm (HbODA) is proposed for efficiently detection of outliers in high dimensional data. The important parameters efficiency, accuracy, computation cost, precision, recall etc. are focused for analyzing the performance of the novel hybrid algorithm. Experimental results on real large sets show that the proposed algorithm is better in detecting outliers than other traditional methods.

scholarly journals Designing a Streaming Algorithm for Outlier Detection in Data Mining—An Incrementa Approach

Sensors ◽  
2020 ◽  
Vol 20 (5) ◽  
pp. 1261 ◽  
Author(s):  
Kangqing Yu ◽  
Wei Shi ◽  
Nicola Santoro
Keyword(s):  
Real Time ◽  
Outlier Detection ◽  
Complexity Analysis ◽  
Concept Drift ◽  
Area Under The Curve ◽  
Sliding Window ◽  
Detection Algorithm ◽  
Streaming Data ◽  
Processing Unit ◽  
Time Data

To design an algorithm for detecting outliers over streaming data has become an important task in many common applications, arising in areas such as fraud detections, network analysis, environment monitoring and so forth. Due to the fact that real-time data may arrive in the form of streams rather than batches, properties such as concept drift, temporal context, transiency, and uncertainty need to be considered. In addition, data processing needs to be incremental with limited memory resource, and scalable. These facts create big challenges for existing outlier detection algorithms in terms of their accuracies when they are implemented in an incremental fashion, especially in the streaming environment. To address these problems, we first propose C_KDE_WR, which uses sliding window and kernel function to process the streaming data online, and reports its results demonstrating high throughput on handling real-time streaming data, implemented in a CUDA framework on Graphics Processing Unit (GPU). We also present another algorithm, C_LOF, based on a very popular and effective outlier detection algorithm called Local Outlier Factor (LOF) which unfortunately works only on batched data. Using a novel incremental approach that compensates the drawback of high complexity in LOF, we show how to implement it in a streaming context and to obtain results in a timely manner. Like C_KDE_WR, C_LOF also employs sliding-window and statistical-summary to help making decision based on the data in the current window. It also addresses all those challenges of streaming data as addressed in C_KDE_WR. In addition, we report the comparative evaluation on the accuracy of C_KDE_WR with the state-of-the-art SOD_GPU using Precision, Recall and F-score metrics. Furthermore, a t-test is also performed to demonstrate the significance of the improvement. We further report the testing results of C_LOF on different parameter settings and drew ROC and PR curve with their area under the curve (AUC) and Average Precision (AP) values calculated respectively. Experimental results show that C_LOF can overcome the masquerading problem, which often exists in outlier detection on streaming data. We provide complexity analysis and report experiment results on the accuracy of both C_KDE_WR and C_LOF algorithms in order to evaluate their effectiveness as well as their efficiencies.

scholarly journals Mining moving object data

2012 ◽  
Vol 2 (3) ◽  
Author(s):  
Jaroslav Zendulka ◽  
Martin Pešek
Keyword(s):  
Outlier Detection ◽  
Real World ◽  
Moving Objects ◽  
Detection Algorithm ◽  
Moving Object ◽  
Location Based Services ◽  
Data Sets ◽  
Real World Data ◽  
World Data ◽  
Trajectory Outlier Detection

AbstractCurrently many devices provide information about moving objects and location-based services that accumulate a huge volume of moving object data, including trajectories. This paper deals with two useful analysis tasks — mining moving object patterns and trajectory outlier detection. We also present our experience with the TOP-EYE trajectory outlier detection algorithm, which we applied to two real-world data sets.

Sign in / Sign up

Export Citation Format

Share Document