NOVEL HYPERSPECTRAL ANOMALY DETECTION METHODS BASED ON UNSUPERVISED NEAREST REGULARIZED SUBSPACE

Anomaly detection has been of great interest in hyperspectral imagery analysis. Most conventional anomaly detectors merely take advantage of spectral and spatial information within neighboring pixels. In this paper, two methods of Unsupervised Nearest Regularized Subspace-based with Outlier Removal Anomaly Detector (UNRSORAD) and Local Summation UNRSORAD (LSUNRSORAD) are proposed, which are based on the concept that each pixel in background can be approximately represented by its spatial neighborhoods, while anomalies cannot. Using a dual window, an approximation of each testing pixel is a representation of surrounding data via a linear combination. The existence of outliers in the dual window will affect detection accuracy. Proposed detectors remove outlier pixels that are significantly different from majority of pixels. In order to make full use of various local spatial distributions information with the neighboring pixels of the pixels under test, we take the local summation dual-window sliding strategy. The residual image is constituted by subtracting the predicted background from the original hyperspectral imagery, and anomalies can be detected in the residual image. Experimental results show that the proposed methods have greatly improved the detection accuracy compared with other traditional detection method.

Download Full-text

Anomaly Detection for Hyperspectral Imagery Based on the Regularized Subspace Method and Collaborative Representation

Remote Sensing ◽

10.3390/rs11111318 ◽

2019 ◽

Vol 11 (11) ◽

pp. 1318 ◽

Cited By ~ 5

Author(s):

Kun Tan ◽

Zengfu Hou ◽

Fuyu Wu ◽

Qian Du ◽

Yu Chen

Keyword(s):

Anomaly Detection ◽

Spatial Information ◽

Collaborative Representation ◽

Detection Accuracy ◽

Subspace Method ◽

Outlier Removal ◽

Computational Speed ◽

Anomaly Detector ◽

Computationally Expensive ◽

Inverse Distance

Most of the conventional anomaly detectors only take advantage of the spectral information and do not consider the spatial information within neighboring pixels. Recently, the spectral-spatial based local summation anomaly detection (LSAD) algorithm has achieved excellent detection performances. In order to obtain various local spatial distributions with the neighboring pixels of the pixels under test, the LSAD algorithm exploits a multiple-window sliding filter, which can be computationally expensive and time-consuming. In this paper, to address these issues, two modified LSAD-based methods are proposed. The first method, called local summation unsupervised nearest regularized subspace with an outlier removal anomaly detector (LSUNRSORAD), is based on the concept that each pixel in the background can be approximately represented by its spatial neighborhood. The second method, called local summation anomaly detection based on collaborative representation and inverse distance weight (LSAD-CR-IDW), uses the surrounding pixels collected inside the outer window, while outside the inner window, to linearly represent the test pixel and introduces collaborative representation and inverse distance weight to further improve the computational speed and detection precision, respectively. The proposed methods were applied to a synthetic dataset and three real datasets. The experimental results show that the proposed methods have a better detection accuracy and computational speed when compared with the LSAD algorithm and others.

Download Full-text

A Hybrid Semi-Supervised Anomaly Detection Model for High-Dimensional Data

Computational Intelligence and Neuroscience ◽

10.1155/2017/8501683 ◽

2017 ◽

Vol 2017 ◽

pp. 1-9 ◽

Cited By ~ 14

Author(s):

Hongchao Song ◽

Zhuqing Jiang ◽

Aidong Men ◽

Bo Yang

Keyword(s):

Anomaly Detection ◽

Nearest Neighbor ◽

Dimensional Space ◽

High Dimensional Data ◽

Real Life ◽

Detection Methods ◽

High Dimensional ◽

Detection Accuracy ◽

Detection Model ◽

Anomaly Detector

Anomaly detection, which aims to identify observations that deviate from a nominal sample, is a challenging task for high-dimensional data. Traditional distance-based anomaly detection methods compute the neighborhood distance between each observation and suffer from the curse of dimensionality in high-dimensional space; for example, the distances between any pair of samples are similar and each sample may perform like an outlier. In this paper, we propose a hybrid semi-supervised anomaly detection model for high-dimensional data that consists of two parts: a deep autoencoder (DAE) and an ensemble k-nearest neighbor graphs- (K-NNG-) based anomaly detector. Benefiting from the ability of nonlinear mapping, the DAE is first trained to learn the intrinsic features of a high-dimensional dataset to represent the high-dimensional data in a more compact subspace. Several nonparametric KNN-based anomaly detectors are then built from different subsets that are randomly sampled from the whole dataset. The final prediction is made by all the anomaly detectors. The performance of the proposed method is evaluated on several real-life datasets, and the results confirm that the proposed hybrid model improves the detection accuracy and reduces the computational complexity.

Download Full-text

OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection

Sensors ◽

10.3390/s21144805 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4805

Author(s):

Saad Abbasi ◽

Mahmoud Famouri ◽

Mohammad Javad Shafiee ◽

Alexander Wong

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Detection Methods ◽

Detection Accuracy ◽

Network Architectures ◽

Design Exploration ◽

Convolutional Autoencoder ◽

Acoustic Anomaly ◽

Human Operators ◽

Computational Resources

Human operators often diagnose industrial machinery via anomalous sounds. Given the new advances in the field of machine learning, automated acoustic anomaly detection can lead to reliable maintenance of machinery. However, deep learning-driven anomaly detection methods often require an extensive amount of computational resources prohibiting their deployment in factories. Here we explore a machine-driven design exploration strategy to create OutlierNets, a family of highly compact deep convolutional autoencoder network architectures featuring as few as 686 parameters, model sizes as small as 2.7 KB, and as low as 2.8 million FLOPs, with a detection accuracy matching or exceeding published architectures with as many as 4 million parameters. The architectures are deployed on an Intel Core i5 as well as a ARM Cortex A72 to assess performance on hardware that is likely to be used in industry. Experimental results on the model’s latency show that the OutlierNet architectures can achieve as much as 30x lower latency than published networks.

Download Full-text

Edge Computing for Data Anomaly Detection of Multi-Sensors in Underground Mining

Electronics ◽

10.3390/electronics10030302 ◽

2021 ◽

Vol 10 (3) ◽

pp. 302

Author(s):

Chunde Liu ◽

Xianli Su ◽

Chuanwen Li

Keyword(s):

Energy Consumption ◽

Anomaly Detection ◽

Underground Mining ◽

Heterogeneous Data ◽

Edge Computing ◽

Sensor Nodes ◽

Detection Methods ◽

Detection Accuracy ◽

Clustering Methods ◽

Safety Warning

There is a growing interest in safety warning of underground mining due to the huge threat being faced by those working in underground mining. Data acquisition of sensors based on Internet of Things (IoT) is currently the main method, but the data anomaly detection and analysis of multi-sensors is a challenging task: firstly, the data that are collected by different sensors of underground mining are heterogeneous; secondly, real-time is required for the data anomaly detection of safety warning. Currently, there are many anomaly detection methods, such as traditional clustering methods K-means and C-means. Meanwhile, Artificial Intelligence (AI) is widely used in data analysis and prediction. However, K-means and C-means cannot directly process heterogeneous data, and AI algorithms require equipment with high computing and storage capabilities. IoT equipment of underground mining cannot perform complex calculation due to the limitation of energy consumption. Therefore, many existing methods cannot be directly used for IoT applications in underground mining. In this paper, a multi-sensors data anomaly detection method based on edge computing is proposed. Firstly, an edge computing model is designed, and according to the computing capabilities of different types of devices, anomaly detection tasks are migrated to different edge devices, which solve the problem of insufficient computing capabilities of the devices. Secondly, according to the requirements of different anomaly detection tasks, edge anomaly detection algorithms for sensor nodes and sink nodes are designed respectively. Lastly, an experimental platform is built for performance comparison analysis, and the experimental results show that the proposed algorithm has better performance in anomaly detection accuracy, delay, and energy consumption.

Download Full-text

Hyperspectral Anomaly Detection: Comparative Evaluation in Scenes with Diverse Complexity

Journal of Electrical and Computer Engineering ◽

10.1155/2012/162106 ◽

2012 ◽

Vol 2012 ◽

pp. 1-16 ◽

Cited By ~ 17

Author(s):

Dirk Borghys ◽

Ingebjørg Kåsen ◽

Véronique Achard ◽

Christiaan Perneel

Keyword(s):

Anomaly Detection ◽

Hyperspectral Data ◽

Data Cube ◽

Detection Methods ◽

Multivariate Normal ◽

Normal Probability ◽

Local Methods ◽

Anomaly Detector ◽

The Difference ◽

Rx Detector

Anomaly detection (AD) in hyperspectral data has received a lot of attention for various applications. The aim of anomaly detection is to detect pixels in the hyperspectral data cube whose spectra differ significantly from the background spectra. Many anomaly detectors have been proposed in the literature. They differ in the way the background is characterized and in the method used for determining the difference between the current pixel and the background. The most well-known anomaly detector is the RX detector that calculates the Mahalanobis distance between the pixel under test (PUT) and the background. Global RX characterizes the background of the complete scene by a single multivariate normal probability density function. In many cases, this model is not appropriate for describing the background. For that reason a variety of other anomaly detection methods have been developed. This paper examines three classes of anomaly detectors: subspace methods, local methods, and segmentation-based methods. Representative examples of each class are chosen and applied on a set of hyperspectral data with diverse complexity. The results are evaluated and compared.

Download Full-text

Study on Anomaly Detection Methods in Hyperspectral Image

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.631-632.631 ◽

2014 ◽

Vol 631-632 ◽

pp. 631-635

Author(s):

Yi Ting Wang ◽

Shi Qi Huang ◽

Hong Xia Wang ◽

Dai Zhi Liu

Keyword(s):

Anomaly Detection ◽

Target Detection ◽

Hyperspectral Image ◽

Detection Methods ◽

Imaging Data ◽

Detection Algorithms ◽

Sensing Technology ◽

Anomaly Detector ◽

And Performance ◽

Spectral Diagnosis

Hyperspectral remote sensing technology can be used to make a correct spectral diagnosis on substances. So it is widely used in the field of target detection and recognition. However, it is very difficult to gather accurate prior information for target detect since the spectral uncertainty of objects is pervasive in existence. An anomaly detector can enable one to detect targets whose signatures are spectrally distinct from their surroundings with no prior knowledge. It becomes a focus in the field of target detection. Therefore, we study four anomaly detection algorithms and conclude with empirical results that use hyperspectral imaging data to illustrate the operation and performance of various detectors.

Download Full-text

HYPERSPECTRAL ANOMALY DETECTION IN URBAN SCENARIOS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xli-b7-111-2016 ◽

2016 ◽

Vol XLI-B7 ◽

pp. 111-116 ◽

Cited By ~ 1

Author(s):

J. G. Rejas Ayuga ◽

R. Martínez Marín ◽

M. Marchamalo Sacristán ◽

J. Bonatti ◽

J. C. Ojeda

Keyword(s):

Anomaly Detection ◽

Near Infrared ◽

Hyperspectral Data ◽

Detection Methods ◽

San Jose ◽

First Results ◽

Anomaly Detector ◽

A New Technique ◽

Spectral Ranges ◽

Shortwave Infrared

We have studied the spectral features of reflectance and emissivity in the pattern recognition of urban materials in several single hyperspectral scenes through a comparative analysis of anomaly detection methods and their relationship with city surfaces with the aim to improve information extraction processes. Spectral ranges of the visible-near infrared (VNIR), shortwave infrared (SWIR) and thermal infrared (TIR) from hyperspectral data cubes of AHS sensor and HyMAP and MASTER of two cities, Alcalá de Henares (Spain) and San José (Costa Rica) respectively, have been used. <br><br> In this research it is assumed no prior knowledge of the targets, thus, the pixels are automatically separated according to their spectral information, significantly differentiated with respect to a background, either globally for the full scene, or locally by image segmentation. Several experiments on urban scenarios and semi-urban have been designed, analyzing the behaviour of the standard RX anomaly detector and different methods based on subspace, image projection and segmentation-based anomaly detection methods. A new technique for anomaly detection in hyperspectral data called DATB (Detector of Anomalies from Thermal Background) based on dimensionality reduction by projecting targets with unknown spectral signatures to a background calculated from thermal spectrum wavelengths is presented. First results and their consequences in non-supervised classification and extraction information processes are discussed.

Download Full-text

Hyperspectral Anomaly Detection Based on Separability-Aware Sample Cascade

Remote Sensing ◽

10.3390/rs11212537 ◽

2019 ◽

Vol 11 (21) ◽

pp. 2537 ◽

Cited By ~ 2

Author(s):

Dandan Ma ◽

Yuan Yuan ◽

Qi Wang

Keyword(s):

Anomaly Detection ◽

Large Scale ◽

Hyperspectral Image ◽

Superior Performance ◽

Detection Methods ◽

Background Information ◽

Detection Accuracy ◽

Layer By Layer ◽

Edge Preserving ◽

Image Pixels

A hyperspectral image usually covers a large scale of ground scene, which contains various materials with different spectral properties. When directly exploring the background information using all the image pixels, complex spectral interactions and inter-/intra-difference of different samples will significantly reduce the accuracy of background evaluation and further affect the detection performance. To address this problem, this paper proposes a novel hyperspectral anomaly detection method based on separability-aware sample cascade model. Through identifying separability of hyperspectral pixels, background samples are sifted out layer-by-layer according to their separable degrees from anomalies, which can ensure the accuracy and distinctiveness of background representation. First, as spatial structure is beneficial for recognizing target, a new spectral–spatial feature extraction technique is used in this work based on the PCA technique and edge-preserving filtering. Second, depending on different separability computed by sparse representation, samples are separated into different sets which can effectively and completely reflect various characteristics of background across all the cascade layers. Meanwhile, some potential abnormal targets are removed at each selection step to avoid their effects on subsequent layers. Finally, comprehensively taking different good properties of all the separability-aware layers into consideration, a simple multilayer anomaly detection strategy is adopted to obtain the final detection map. Extensive experimental results on five real-world hyperspectral images demonstrate our method’s superior performance. Compared with seven representative anomaly detection methods, our method improves the average detection accuracy with great advantages.

Download Full-text

FrHPI: A Discriminative Patch-Image Model for Hyperspectral Anomaly Detection

Mathematical Problems in Engineering ◽

10.1155/2021/6854954 ◽

2021 ◽

Vol 2021 ◽

pp. 1-12

Author(s):

Hao Li ◽

Ganghui Fan ◽

Shan Zeng ◽

Zhen Kang

Keyword(s):

Anomaly Detection ◽

Spatial Information ◽

Fractional Fourier Transform ◽

Hyperspectral Image ◽

Sparse Matrix ◽

Noise Removal ◽

Low Rank ◽

Image Model ◽

Anomaly Detector ◽

Patch Image

Anomaly detection is now a significantly important part of hyperspectral image analysis to detect targets in an unsupervised manner. Traditional hyperspectral anomaly detectors fail to consider spatial information, which is vital in hyperspectral anomaly detection. Moreover, they usually take the raw data without feature extraction as input, limiting the detection performance. We propose a new anomaly detector based on the fractional Fourier transform (FrFT) and a modified patch-image model called the hyperspectral patch-image (HPI) model to tackle these two problems. By combining them, the proposed anomaly detector is named fractional hyperspectral patch-image (FrHPI) detector. Under the assumption that the target patch-image is a sparse matrix while the background patch-image is a low-rank matrix, we first formulate a matrix by sliding a rectangle window on the first three principal components (PCs) of HSI. The matrix can be decomposed into three parts representing the background, targets, and noise with the well-known low-rank and sparse matrix decomposition (LRaSMD). Then, distinctive features are extracted via FrFT, a transformation which is desirable for noise removal. Background atoms are selected to construct the covariance matrix. Finally, anomalies are picked up with Mahalanobis distance. Extensive experiments are conducted to verify the proposed FrHPI detector’s superiority in hyperspectral anomaly detection compared with other state-of-the-art detectors.

Download Full-text