Anomaly Detection Algorithm Based on CFSFDP

2020 ◽  
Vol 24 (4) ◽  
pp. 453-460
Author(s):  
Weiwu Ren ◽  
Jianfei Zhang ◽  
Xiaoqiang Di ◽  
Yinan Lu ◽  
Bochen Zhang ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Real Time ◽  
Data Storage ◽  
Clustering Algorithm ◽  
Detection Algorithm ◽  
Original Algorithm ◽  
Redundant Data ◽  
Data Points ◽  
Behavior Profiles ◽  
Improved Algorithm

Clustering by fast search and find of density peak (CFSFDP) is a simple and crisp density-clustering algorithm. The original algorithm is not suitable for direct application to anomaly detection. Its clustering results have a high level of redundant density information. If used directly as behavior profiles, the computation and storage costs of anomaly detection are high. Therefore, an improved algorithm based on CFSFDP is proposed for anomaly detection. The improved algorithm uses a few data points and their radius to support behavior profiles, and deletes the redundant data points without supporting profiles. This method not only reduces the large amount of data storage and distance calculation in the process of generating profiles, but also reduces the search space of profiles in the detection process. Numerous experiments show that the improved algorithm generates profiles faster than density-based spatial clustering of application with noise (DBSCAN), and has better profile precision than adaptive real-time anomaly detection with incremental clustering (ADWICE). The improved algorithm inherits the arbitrary shape clusters of CFSFDP, and improves the storage and computation performance. Compared with DBSCAN and ADWICE, the improved anomaly-detection algorithm based on CFSFDP has more balanced detection precision and real-time performance.

scholarly journals Multivariate-Time-Series-Driven Real-time Anomaly Detection Based on Bayesian Network

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3367 ◽  
Author(s):  
Nan Ding ◽  
Huanbo Gao ◽  
Hongyu Bu ◽  
Haoxuan Ma ◽  
Huaiwei Si
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Bayesian Network ◽  
Real Time ◽  
Multivariate Time Series ◽  
Research Direction ◽  
Detection Algorithm ◽  
The Real ◽  
Hierarchical Temporal Memory ◽  
Sensing Time

Anomaly detection is an important research direction, which takes the real-time information system from different sensors and conditional information sources into consideration. Based on this, we can detect possible anomalies expected of the devices and components. One of the challenges is anomaly detection in multivariate-sensing time-series in this paper. Based on this situation, we propose RADM, a real-time anomaly detection algorithm based on Hierarchical Temporal Memory (HTM) and Bayesian Network (BN). First of all, we use HTM model to evaluate the real-time anomalies of each univariate-sensing time-series. Secondly, a model of anomalous state detection in multivariate-sensing time-series based on Naive Bayesian is designed to analyze the validity of the above time-series. Lastly, considering the real-time monitoring cases of the system states of terminal nodes in Cloud Platform, the effectiveness of the methodology is demonstrated using a simulated example. Extensive simulation results show that using RADM in multivariate-sensing time-series is able to detect more abnormal, and thus can remarkably improve the performance of real-time anomaly detection.

Dynamically Real-time Anomaly Detection Algorithm with Immune Negative Selection

2013 ◽  
Vol 7 (3) ◽  
pp. 1157-1163 ◽  
Author(s):  
Lingxi Peng ◽  
Wenbin Chen ◽  
Dongqing Xie ◽  
Ying Gao ◽  
Chunlin Liang
Keyword(s):  
Anomaly Detection ◽  
Real Time ◽  
Negative Selection ◽  
Detection Algorithm

Hierarchy Topic Detection and Hot Topic Identification

2014 ◽  
Vol 701-702 ◽  
pp. 180-186
Author(s):  
Xue Mei Zhou ◽  
Shan Ying Cheng
Keyword(s):  
Public Opinion ◽  
Real Time ◽  
Hierarchical Structure ◽  
Clustering Algorithm ◽  
Detection Algorithm ◽  
Topic Detection ◽  
Topic Identification ◽  
Evaluation Indexes ◽  
Detection Algorithms ◽  
The Hierarchical Structure

Due to the problem that the existing topic detection algorithms can not satisfy accuracy,real time and topic hierarchical clustering at the same time, this article builds a hierarchy topic detection algorithm based on improved single pass clustering algorithm. In addition, using public opinion evaluation indexes to analyze topic temperature,the method proposed in this paper can detect hot topics accurately and timely while showing the hierarchical structure of the topic .

scholarly journals Multi-Floor Indoor Pedestrian Dead Reckoning with a Backtracking Particle Filter and Viterbi-Based Floor Number Detection

Sensors ◽  
2021 ◽  
Vol 21 (13) ◽  
pp. 4565
Author(s):  
Cedric De Cock ◽  
Wout Joseph ◽  
Luc Martens ◽  
Jens Trogh ◽  
David Plets
Keyword(s):  
Particle Filter ◽  
Real Time ◽  
Clustering Algorithm ◽  
Dead Reckoning ◽  
Detection Algorithm ◽  
Measurement Unit ◽  
Batch Mode ◽  
Access Points ◽  
Median Position ◽  
Pedestrian Dead Reckoning

We present a smartphone-based indoor localisation system, able to track pedestrians over multiple floors. The system uses Pedestrian Dead Reckoning (PDR), which exploits data from the smartphone’s inertial measurement unit to estimate the trajectory. The PDR output is matched to a scaled floor plan and fused with model-based WiFi received signal strength fingerprinting by a Backtracking Particle Filter (BPF). We proposed a new Viterbi-based floor detection algorithm, which fuses data from the smartphone’s accelerometer, barometer and WiFi RSS measurements to detect stairs and elevator usage and to estimate the correct floor number. We also proposed a clustering algorithm on top of the BPF to solve multimodality, a known problem with particle filters. The proposed system relies on only a few pre-existing access points, whereas most systems assume or require the presence of a dedicated localisation infrastructure. In most public buildings and offices, access points are often available at smaller densities than used for localisation. Our system was extensively tested in a real office environment with seven 41 m × 27 m floors, each of which had two WiFi access points. Our system was evaluated in real-time and batch mode, since the system was able to correct past states. The clustering algorithm reduced the median position error by 17% in real-time and 13% in batch mode, while the floor detection algorithm achieved a 99.1% and 99.7% floor number accuracy in real-time and batch mode, respectively.

scholarly journals Highway Event Detection Algorithm Based on Improved Fast Peak Clustering

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Lili Pei ◽  
Zhaoyun Sun ◽  
Yuxi Han ◽  
Wei Li ◽  
Huaixin Zhao
Keyword(s):  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Original Data ◽  
Detection Algorithm ◽  
Massive Data ◽  
Highway Traffic ◽  
Traffic Condition ◽  
Original Algorithm ◽  
Event Mining ◽  
Complex Relationships

Aiming at the mining of traffic events based on large amounts of highway data, this paper proposes an improved fast peak clustering algorithm to process highway toll data. The highway toll data are first analyzed, and a data cleaning method based on the sum of similar coefficients is proposed to process the original data. Next, to avoid the shortcomings of the excessive subjectivity of the original algorithm, an improved fast peak clustering algorithm is proposed. Finally, the improved algorithm is applied to highway traffic condition analysis and abnormal event mining to obtain more accurate and intuitive clustering results. Compared with two classical algorithms, namely, the k-means and density-based spatial clustering of applications with noise (DBSCAN) algorithms, as well as the unimproved original fast peak clustering algorithm, the proposed algorithm is faster and more accurate and can reveal the complex relationships among massive data more efficiently. During the process of reforming the toll system, the algorithm can automatically and more efficiently analyze massive toll data and detect abnormal events, thereby providing a theoretical basis and data support for the operation monitoring and maintenance of highways.

scholarly journals Dimensional Reduction of Data for Anomaly Detection and Speed Performance using PCA and DBSCAN

2019 ◽  
Vol 9 (1S2) ◽  
pp. 39-41
Keyword(s):  
Anomaly Detection ◽  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Test Results ◽  
High Complexity ◽  
Medical Sciences ◽  
Data Set ◽  
Network Intrusion ◽  
Speed Performance ◽  
Data Points

Anomaly detection is the major problem facing by many of industries. It includes network intrusion and medical sciences. Several fields like Astronomy and research also facing difficulties in finding effective anomaly detection. They have included several techniques to solve such problems. Clustering is the technique which has been employed by many of the researchers. The most commonly used algorithm to perform clustering is DBSCAN. It is well known clustering algorithm used in data mining and Machine learning. It is referred as Density based spatial clustering of application with noise. Because of its high complexity in computation, it must be decreased in terms of dimensionality of data points. PCA is a method used then to reduce dimensionality and produced a new data set which is again undergo DBSCAN. Here by the nature of the test results was precise there by such a methodology can be adjusted. The mix of PCA and DBSCAN was acutely confirmed and resultant examination shows that a speedup of 25% was improved while the quality was 80% diminishing the dimensionality of informational index of half.

Sign in / Sign up

Export Citation Format

Share Document