Study on Anomaly Detection Methods in Hyperspectral Image

2014 ◽  
Vol 631-632 ◽  
pp. 631-635
Author(s):  
Yi Ting Wang ◽  
Shi Qi Huang ◽  
Hong Xia Wang ◽  
Dai Zhi Liu
Keyword(s):  
Anomaly Detection ◽  
Target Detection ◽  
Hyperspectral Image ◽  
Detection Methods ◽  
Imaging Data ◽  
Detection Algorithms ◽  
Sensing Technology ◽  
Anomaly Detector ◽  
And Performance ◽  
Spectral Diagnosis

Hyperspectral remote sensing technology can be used to make a correct spectral diagnosis on substances. So it is widely used in the field of target detection and recognition. However, it is very difficult to gather accurate prior information for target detect since the spectral uncertainty of objects is pervasive in existence. An anomaly detector can enable one to detect targets whose signatures are spectrally distinct from their surroundings with no prior knowledge. It becomes a focus in the field of target detection. Therefore, we study four anomaly detection algorithms and conclude with empirical results that use hyperspectral imaging data to illustrate the operation and performance of various detectors.

scholarly journals Systematic Review of Anomaly Detection in Hyperspectral Remote Sensing Applications

2021 ◽  
Vol 11 (11) ◽  
pp. 4878
Author(s):  
Ivan Racetin ◽  
Andrija Krtalić
Keyword(s):  
Remote Sensing ◽  
Image Processing ◽  
Anomaly Detection ◽  
Target Detection ◽  
Hyperspectral Image ◽  
Hyperspectral Images ◽  
Detection Methods ◽  
Hyperspectral Image Processing ◽  
Sensing Applications ◽  
Remote Sensing Applications

Hyperspectral sensors are passive instruments that record reflected electromagnetic radiation in tens or hundreds of narrow and consecutive spectral bands. In the last two decades, the availability of hyperspectral data has sharply increased, propelling the development of a plethora of hyperspectral classification and target detection algorithms. Anomaly detection methods in hyperspectral images refer to a class of target detection methods that do not require any a-priori knowledge about a hyperspectral scene or target spectrum. They are unsupervised learning techniques that automatically discover rare features on hyperspectral images. This review paper is organized into two parts: part A provides a bibliographic analysis of hyperspectral image processing for anomaly detection in remote sensing applications. Development of the subject field is discussed, and key authors and journals are highlighted. In part B an overview of the topic is presented, starting from the mathematical framework for anomaly detection. The anomaly detection methods were generally categorized as techniques that implement structured or unstructured background models and then organized into appropriate sub-categories. Specific anomaly detection methods are presented with corresponding detection statistics, and their properties are discussed. This paper represents the first review regarding hyperspectral image processing for anomaly detection in remote sensing applications.

scholarly journals A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3627 ◽  
Author(s):  
Yi Zhang ◽  
Zebin Wu ◽  
Jin Sun ◽  
Yan Zhang ◽  
Yaoqin Zhu ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Sparse Representation ◽  
Parallel Algorithm ◽  
Hyperspectral Image ◽  
Hyperspectral Images ◽  
Low Rank ◽  
Detection Methods ◽  
Matrix Computations ◽  
Hyperspectral Image Processing ◽  
Mapreduce Model

Anomaly detection aims to separate anomalous pixels from the background, and has become an important application of remotely sensed hyperspectral image processing. Anomaly detection methods based on low-rank and sparse representation (LRASR) can accurately detect anomalous pixels. However, with the significant volume increase of hyperspectral image repositories, such techniques consume a significant amount of time (mainly due to the massive amount of matrix computations involved). In this paper, we propose a novel distributed parallel algorithm (DPA) by redesigning key operators of LRASR in terms of MapReduce model to accelerate LRASR on cloud computing architectures. Independent computation operators are explored and executed in parallel on Spark. Specifically, we reconstitute the hyperspectral images in an appropriate format for efficient DPA processing, design the optimized storage strategy, and develop a pre-merge mechanism to reduce data transmission. Besides, a repartitioning policy is also proposed to improve DPA’s efficiency. Our experimental results demonstrate that the newly developed DPA achieves very high speedups when accelerating LRASR, in addition to maintaining similar accuracies. Moreover, our proposed DPA is shown to be scalable with the number of computing nodes and capable of processing big hyperspectral images involving massive amounts of data.

scholarly journals Multivariate Anomaly Detection for Earth Observations: A Comparison of Algorithms and Feature Extraction Techniques

2016 ◽  
Author(s):  
Milan Flach ◽  
Fabian Gans ◽  
Alexander Brenning ◽  
Joachim Denzler ◽  
Markus Reichstein ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Anomaly Detection ◽  
Data Streams ◽  
Multivariate Data ◽  
Detection Methods ◽  
Earth System ◽  
Earth System Science ◽  
System Science ◽  
Detection Algorithms ◽  
Earth Observations

Abstract. Today, many processes at the Earth's surface are constantly monitored by multiple data streams. These observations have become central to advance our understanding of e.g. vegetation dynamics in response to climate or land use change. Another set of important applications is monitoring effects of climatic extreme events, other disturbances such as fires, or abrupt land transitions. One important methodological question is how to reliably detect anomalies in an automated and generic way within multivariate data streams, which typically vary seasonally and are interconnected across variables. Although many algorithms have been proposed for detecting anomalies in multivariate data, only few have been investigated in the context of Earth system science applications. In this study, we systematically combine and compare feature extraction and anomaly detection algorithms for detecting anomalous events. Our aim is to identify suitable workflows for automatically detecting anomalous patterns in multivariate Earth system data streams. We rely on artificial data that mimic typical properties and anomalies in multivariate spatiotemporal Earth observations. This artificial experiment is needed as there is no 'gold standard' for the identification of anomalies in real Earth observations. Our results show that a well chosen feature extraction step (e.g. subtracting seasonal cycles, or dimensionality reduction) is more important than the choice of a particular anomaly detection algorithm. Nevertheless, we identify 3 detection algorithms (k-nearest neighbours mean distance, kernel density estimation, a recurrence approach) and their combinations (ensembles) that outperform other multivariate approaches as well as univariate extreme event detection methods. Our results therefore provide an effective workflow to automatically detect anomalies in Earth system science data.

scholarly journals Multirapid Serial Visual Presentation Framework for EEG-Based Target Detection

2017 ◽  
Vol 2017 ◽  
pp. 1-12 ◽  
Author(s):  
Zhimin Lin ◽  
Ying Zeng ◽  
Hui Gao ◽  
Li Tong ◽  
Chi Zhang ◽  
...  
Keyword(s):  
Image Retrieval ◽  
Target Detection ◽  
Visual Presentation ◽  
Research Direction ◽  
Detection Methods ◽  
Detection Accuracy ◽  
Single Trial ◽  
Target Image ◽  
Detection Algorithms ◽  
New Research

Target image detection based on a rapid serial visual presentation (RSVP) paradigm is a typical brain-computer interface system with various applications, such as image retrieval. In an RSVP paradigm, a P300 component is detected to determine target images. This strategy requires high-precision single-trial P300 detection methods. However, the performance of single-trial detection methods is relatively lower than that of multitrial P300 detection methods. Image retrieval based on multitrial P300 is a new research direction. In this paper, we propose a triple-RSVP paradigm with three images being presented simultaneously and a target image appearing three times. Thus, multitrial P300 classification methods can be used to improve detection accuracy. In this study, these mechanisms were extended and validated, and the characteristics of the multi-RSVP framework were further explored. Two different P300 detection algorithms were also utilized in multi-RSVP to demonstrate that the scheme is universally applicable. Results revealed that the detection accuracy of the multi-RSVP paradigm was higher than that of the standard RSVP paradigm. The results validate the effectiveness of the proposed method, and this method can provide a whole new idea in the field of EEG-based target detection.

scholarly journals Hyperspectral Anomaly Detection: Comparative Evaluation in Scenes with Diverse Complexity

2012 ◽  
Vol 2012 ◽  
pp. 1-16 ◽  
Author(s):  
Dirk Borghys ◽  
Ingebjørg Kåsen ◽  
Véronique Achard ◽  
Christiaan Perneel
Keyword(s):  
Anomaly Detection ◽  
Hyperspectral Data ◽  
Data Cube ◽  
Detection Methods ◽  
Multivariate Normal ◽  
Normal Probability ◽  
Local Methods ◽  
Anomaly Detector ◽  
The Difference ◽  
Rx Detector

Anomaly detection (AD) in hyperspectral data has received a lot of attention for various applications. The aim of anomaly detection is to detect pixels in the hyperspectral data cube whose spectra differ significantly from the background spectra. Many anomaly detectors have been proposed in the literature. They differ in the way the background is characterized and in the method used for determining the difference between the current pixel and the background. The most well-known anomaly detector is the RX detector that calculates the Mahalanobis distance between the pixel under test (PUT) and the background. Global RX characterizes the background of the complete scene by a single multivariate normal probability density function. In many cases, this model is not appropriate for describing the background. For that reason a variety of other anomaly detection methods have been developed. This paper examines three classes of anomaly detectors: subspace methods, local methods, and segmentation-based methods. Representative examples of each class are chosen and applied on a set of hyperspectral data with diverse complexity. The results are evaluated and compared.

scholarly journals Evaluation of Unsupervised Anomaly Detection Methods in Sentiment Mining

2019 ◽  
Vol 8 (9) ◽  
pp. 1080-1085
Keyword(s):  
Anomaly Detection ◽  
Computation Time ◽  
Vital Role ◽  
Detection Methods ◽  
Research Focus ◽  
Comparative Performance ◽  
Detection Techniques ◽  
Detection Algorithms ◽  
Sentiment Mining ◽  
Unsupervised Anomaly Detection

Anomaly detection has vital role in data preprocessing and also in the mining of outstanding points for marketing, network sensors, fraud detection, intrusion detection, stock market analysis. Recent studies have been found to concentrate more on outlier detection for real time datasets. Anomaly detection study is at present focuses on the expansion of innovative machine learning methods and on enhancing the computation time. Sentiment mining is the process to discover how people feel about a particular topic. Though many anomaly detection techniques have been proposed, it is also notable that the research focus lacks a comparative performance evaluation in sentiment mining datasets. In this study, three popular unsupervised anomaly detection algorithms such as density based, statistical based and cluster based anomaly detection methods are evaluated on movie review sentiment mining dataset. This paper will set a base for anomaly detection methods in sentiment mining research. The results show that density based (LOF) anomaly detection method suits best for the movie review sentiment dataset.

scholarly journals Hyperspectral Target Detection via Adaptive Information—Theoretic Metric Learning with Local Constraints

2018 ◽  
Vol 10 (9) ◽  
pp. 1415 ◽  
Author(s):  
Yanni Dong ◽  
Bo Du ◽  
Liangpei Zhang ◽  
Xiangyun Hu
Keyword(s):  
Target Detection ◽  
State Of The Art ◽  
Metric Learning ◽  
High Spectral Resolution ◽  
Detection Methods ◽  
Significant Information ◽  
Local Constraints ◽  
Information Theoretic ◽  
Detection Algorithms ◽  
Before And After

By using the high spectral resolution, hyperspectral images (HSIs) provide significant information for target detection, which is of great interest in HSI processing. However, most classical target detection methods may only perform well based on certain assumptions. Simultaneously, using limited numbers of target samples and preserving the discriminative information is also a challenging problem in hyperspectral target detection. To overcome these shortcomings, this paper proposes a novel adaptive information-theoretic metric learning with local constraints (ITML-ALC) for hyperspectral target detection. The proposed method firstly uses the information-theoretic metric learning (ITML) method as the objective function for learning a Mahalanobis distance to separate similar and dissimilar point-pairs without certain assumptions, needing fewer adjusted parameters. Then, adaptively local constraints are applied to shrink the distances between samples of similar pairs and expand the distances between samples of dissimilar pairs. Finally, target detection decision can be made by considering both the threshold and the changes between the distances before and after metric learning. Experimental results demonstrate that the proposed method can obviously separate target samples from background ones and outperform both the state-of-the-art target detection algorithms and the other classical metric learning methods.

An Insider Threat Detection Method Based on Business Process Mining

2017 ◽  
Vol 13 (2) ◽  
pp. 83-98 ◽  
Author(s):  
Taiming Zhu ◽  
Yuanbo Guo ◽  
Ankang Ju ◽  
Jun Ma ◽  
Xuan Wang
Keyword(s):  
Business Process ◽  
Process Mining ◽  
Detection System ◽  
Detection Methods ◽  
Insider Threat ◽  
Threat Detection ◽  
Detection Algorithms ◽  
Business Cases ◽  
Business Process Mining ◽  
And Performance

Current intrusion detection systems are mostly for detecting external attacks, but the “Prism Door” and other similar events indicate that internal staff may bring greater harm to organizations in information security. Traditional insider threat detection methods only consider the audit records of personal behavior and failed to combine it with business activities, which may miss the insider threat happened during a business process. The authors consider operators' behavior and correctness and performance of the business activities, propose a business process mining based insider threat detection system. The system firstly establishes the normal profiles of business activities and the operators by mining the business log, and then detects specific anomalies by comparing the content of real-time log with the corresponding normal profile in order to find out the insiders and the threats they have brought. The relating anomalies are defined and the corresponding detection algorithms are presented. The authors have performed experimentation using the ProM framework and Java programming, with five synthetic business cases, and found that the system can effectively identify anomalies of both operators and business activities that may be indicative of potential insider threat.

scholarly journals A Hybrid Semi-Supervised Anomaly Detection Model for High-Dimensional Data

2017 ◽  
Vol 2017 ◽  
pp. 1-9 ◽  
Author(s):  
Hongchao Song ◽  
Zhuqing Jiang ◽  
Aidong Men ◽  
Bo Yang
Keyword(s):  
Anomaly Detection ◽  
Nearest Neighbor ◽  
Dimensional Space ◽  
High Dimensional Data ◽  
Real Life ◽  
Detection Methods ◽  
High Dimensional ◽  
Detection Accuracy ◽  
Detection Model ◽  
Anomaly Detector

Anomaly detection, which aims to identify observations that deviate from a nominal sample, is a challenging task for high-dimensional data. Traditional distance-based anomaly detection methods compute the neighborhood distance between each observation and suffer from the curse of dimensionality in high-dimensional space; for example, the distances between any pair of samples are similar and each sample may perform like an outlier. In this paper, we propose a hybrid semi-supervised anomaly detection model for high-dimensional data that consists of two parts: a deep autoencoder (DAE) and an ensemble k-nearest neighbor graphs- (K-NNG-) based anomaly detector. Benefiting from the ability of nonlinear mapping, the DAE is first trained to learn the intrinsic features of a high-dimensional dataset to represent the high-dimensional data in a more compact subspace. Several nonparametric KNN-based anomaly detectors are then built from different subsets that are randomly sampled from the whole dataset. The final prediction is made by all the anomaly detectors. The performance of the proposed method is evaluated on several real-life datasets, and the results confirm that the proposed hybrid model improves the detection accuracy and reduces the computational complexity.

Sign in / Sign up

Export Citation Format

Share Document