Realistic Synchrophasor Data Generation for Anomaly Detection and Event Classification

2015 ◽

pp. 403-425

Author(s):

Ji Zhang

Keyword(s):

Anomaly Detection ◽

Data Streams ◽

Training Data ◽

Detection Methods ◽

Network Data ◽

Data Generation ◽

Research Attention ◽

Network Connection ◽

Dimensional Network ◽

Anomaly Classification

A great deal of research attention has been paid to data mining on data streams in recent years. In this chapter, the authors carry out a case study of anomaly detection in large and high-dimensional network connection data streams using Stream Projected Outlier deTector (SPOT) that is proposed in Zhang et al. (2009) to detect anomalies from data streams using subspace analysis. SPOT is deployed on 1999 KDD CUP anomaly detection application. Innovative approaches for training data generation, anomaly classification, false positive reduction, and adoptive detection subspace generation are proposed in this chapter as well. Experimental results demonstrate that SPOT is effective and efficient in detecting anomalies from network data streams and outperforms existing anomaly detection methods.

Download Full-text

Realistic Data Generation for Anomaly Detection in Industrial Settings Using Simulations

Computer Security - Lecture Notes in Computer Science ◽

10.1007/978-3-030-12786-2_8 ◽

2019 ◽

pp. 119-134

Author(s):

Peter Schneider ◽

Alexander Giehl

Keyword(s):

Anomaly Detection ◽

Data Generation ◽

Industrial Settings ◽

Realistic Data

Download Full-text

A Subspace-Based Analysis Method for Anomaly Detection in Large and High-Dimensional Network Connection Data Streams

Privacy, Intrusion Detection and Response ◽

10.4018/978-1-60960-836-1.ch008 ◽

2011 ◽

pp. 193-219

Author(s):

Ji Zhang

Keyword(s):

Anomaly Detection ◽

Data Streams ◽

Training Data ◽

Detection Methods ◽

High Dimensional ◽

Data Generation ◽

Research Attention ◽

Network Connection ◽

Dimensional Network ◽

Anomaly Classification

A great deal of research attention has been paid to data mining on data streams in recent years. In this chapter, the authors carry out a case study of anomaly detection in large and high-dimensional network connection data streams using Stream Projected Outlier deTector (SPOT) that is proposed in (Zhang et al. 2009) to detect anomalies from data streams using subspace analysis. SPOT is deployed on the 1999 KDD CUP anomaly detection application. Innovative approaches for training data generation, anomaly classification, and false positive reduction are proposed in this chapter as well. Experimental results demonstrate that SPOT is effective and efficient in detecting anomalies from network data streams and outperforms existing anomaly detection methods.

Download Full-text

CLOUD-BASED SOLUTION FOR NATIONWIDE POWER LINE MAPPING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w13-119-2019 ◽

2019 ◽

Vol XLII-2/W13 ◽

pp. 119-126 ◽

Cited By ~ 1

Author(s):

I. Toschi ◽

D. Morabito ◽

E. Grilli ◽

F. Remondino ◽

C. Carlevaro ◽

...

Keyword(s):

Anomaly Detection ◽

Point Clouds ◽

Power Line ◽

Data Generation ◽

Reconstruction Algorithms ◽

Web Based ◽

Promising Alternative ◽

3D Data ◽

Dense Point ◽

Point Cloud Classification

<p><strong>Abstract.</strong> Automatic tools for power line mapping and monitoring are increasingly required by modern societies. Since traditional methods, like ground-based onsite inspections, are very labour- and time-intensive, the use of Geomatics techniques is becoming the most promising solution. However, there is a need for an all-in-one solution that allows the entire 3D mapping pipeline in a nationwide data context. The aim of this paper is to introduce a novel cloud-based solution for nationwide power line mapping. The innovative aspects of the system are threefold. First, to exploit image-based 3D reconstruction algorithms to derive dense point clouds over power line corridors, thus demonstrating the potential of photogrammetry as a promising alternative to costly LiDAR surveys. Second, to supply an all-in-one web-based pipeline that automatically manages all steps of power line mapping, from 3D data generation to clearance anomaly detection. Finally, to exploit cloud-computing technology, to handle massive input data. First tests show promising results for (i) 3D image-based reconstruction, (ii) point cloud classification and (iii) anomaly detection.</p>

Download Full-text

A Subspace-Based Analysis Method for Anomaly Detection in Large and High-Dimensional Network Connection Data Streams

Data Mining ◽

10.4018/978-1-4666-2455-9.ch026 ◽

2013 ◽

pp. 530-549

Author(s):

Ji Zhang

Keyword(s):

Anomaly Detection ◽

Data Streams ◽

Training Data ◽

Detection Methods ◽

High Dimensional ◽

Data Generation ◽

Research Attention ◽

Network Connection ◽

Dimensional Network ◽

Anomaly Classification

A great deal of research attention has been paid to data mining on data streams in recent years. In this chapter, the authors carry out a case study of anomaly detection in large and high-dimensional network connection data streams using Stream Projected Outlier deTector (SPOT) that is proposed in (Zhang et al. 2009) to detect anomalies from data streams using subspace analysis. SPOT is deployed on the 1999 KDD CUP anomaly detection application. Innovative approaches for training data generation, anomaly classification, and false positive reduction are proposed in this chapter as well. Experimental results demonstrate that SPOT is effective and efficient in detecting anomalies from network data streams and outperforms existing anomaly detection methods.

Download Full-text

An Efficient Anomaly Detection Based on Optimal Deep Belief Network in Big Data

International Journal of Engineering and Advanced Technology - Regular Issue ◽

10.35940/ijeat.f9178.109119 ◽

2019 ◽

Vol 9 (1) ◽

pp. 708-716

Keyword(s):

Machine Learning ◽

Big Data ◽

Anomaly Detection ◽

Computer Network ◽

Large Datasets ◽

Support Vector ◽

The Internet ◽

Data Sets ◽

Data Generation ◽

Detection Techniques

Nowadays, the internet and network service user’s counts are increasing and the data generation speed also very high. Then again, we see greater security dangers on the internet, enterprise network, websites and the network. Anomaly has been known as one of the effective cyber threats over the internet which increasing exponentially and thus overcomes the commonly used approaches for anomaly detection and classification. Anomaly detection is used in big data analytics to recognize the unexpected behaviour. The most commonly used characteristics in network environment are size and dimensionality, which are big datasets and also impose problems in recognizing useful patterns, For example, to identify the network traffic anomalies from the large datasets. Due to the enormous increase of computer network based facilities it is a challenge to perform fast and efficient anomaly detection. The anomaly recognition in big data sets is more useful to discover fraud and abnormal action. Here, we mainly focus on the problems regarding anomaly detection, so we introduce a novel machine learning based anomaly detection technique. Machine learning approach is used to enhance the anomaly detection speed which is very much useful to detect the anomaly from the large datasets. We evaluate the proposed framework by performing experiments with larger data sets and compare to several existing techniques such as fuzzy, SVM (Support Vector Machine) and PSO (Particle swarm optimization). It has shown 98% percentage of accuracy and the false rate of 0.002 % on proposed classifier. The experimental results illuminate that better performance than existing anomaly detection techniques in big data environment.

Download Full-text