scholarly journals Multi-task learning based Encoder-Decoder: A comprehensive detection and diagnosis system for multi-sensor data

2021 ◽  
Vol 13 (5) ◽  
pp. 168781402110131
Author(s):  
Junfeng Wu ◽  
Li Yao ◽  
Bin Liu ◽  
Zheyuan Ding ◽  
Lei Zhang
Keyword(s):  
Anomaly Detection ◽  
Event Detection ◽  
Large Scale ◽  
Multivariate Time Series ◽  
Sensor Data ◽  
Unified Framework ◽  
Diagnosis System ◽  
Learning Framework ◽  
Task Learning ◽  
Detection And Diagnosis

As more and more sensor data have been collected, automated detection, and diagnosis systems are urgently needed to lessen the increasing monitoring burden and reduce the risk of system faults. A plethora of researches have been done on anomaly detection, event detection, anomaly diagnosis respectively. However, none of current approaches can explore all these respects in one unified framework. In this work, a Multi-Task Learning based Encoder-Decoder (MTLED) which can simultaneously detect anomalies, diagnose anomalies, and detect events is proposed. In MTLED, feature matrix is introduced so that features are extracted for each time point and point-wise anomaly detection can be realized in an end-to-end way. Anomaly diagnosis and event detection share the same feature matrix with anomaly detection in the multi-task learning framework and also provide important information for system monitoring. To train such a comprehensive detection and diagnosis system, a large-scale multivariate time series dataset which contains anomalies of multiple types is generated with simulation tools. Extensive experiments on the synthetic dataset verify the effectiveness of MTLED and its multi-task learning framework, and the evaluation on a real-world dataset demonstrates that MTLED can be used in other application scenarios through transfer learning.

scholarly journals AURORA: A Unified fRamework fOR Anomaly detection on multivariate time series

2021 ◽  
Author(s):  
Lin Zhang ◽  
Wenyu Zhang ◽  
Maxwell J. McNeil ◽  
Nachuan Chengwang ◽  
David S. Matteson ◽  
...  
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Multivariate Time Series ◽  
Aurora A ◽  
Unified Framework

Distributed Monitoring and Intelligent Fault Diagnosis System for Large-Scale Electro-Hydraulic Devices

2015 ◽  
Vol 779 ◽  
pp. 163-168
Author(s):  
Gui You Hao ◽  
Jie Cheng ◽  
Zong Gui Zheng ◽  
Zhi Gang Yu ◽  
Gang Wu
Keyword(s):  
Fault Diagnosis ◽  
Information Fusion ◽  
Large Scale ◽  
Sensor Data ◽  
Distributed Monitoring ◽  
Diagnosis System ◽  
Fusion Methods ◽  
Level Information ◽  
Fault Diagnosis System ◽  
Hydraulic Devices

In this paper, a distributed monitoring and intelligent diagnosis system is designed for large-scale electro-hydraulic devices, based on the utilization of information fusion technology. As a result of using multi-sensor data level, feature level and decision-making level information fusion methods, both precision and real-time performance of condition monitoring and fault diagnosis have been greatly improved, which leads to an advance in the accuracy of monitoring and diagnosis.

Urban Perception: Sensing Cities via a Deep Interactive Multi-task Learning Framework

2021 ◽  
Vol 17 (1s) ◽  
pp. 1-20
Author(s):  
Weili Guan ◽  
Zhaozheng Chen ◽  
Fuli Feng ◽  
Weifeng Liu ◽  
Liqiang Nie
Keyword(s):  
Large Scale ◽  
Pairwise Comparisons ◽  
Specific Information ◽  
Social Scientists ◽  
Learning Framework ◽  
Task Learning ◽  
Highly Correlated ◽  
Image Pairs ◽  
The Given

Social scientists have shown evidence that visual perceptions of urban attributes, such as safe, wealthy, and beautiful perspectives of the given cities, are highly correlated to the residents’ behaviors and quality of life. Despite their significance, measuring visual perceptions of urban attributes is challenging due to the following facts: (1) Visual perceptions are subjectively contradistinctive rather than absolute. (2) Perception comparisons between image pairs are usually conducted region by region, and highly related to the specific urban attributes. And (3) the urban attributes have both the shared and specific information. To address these problems, in this article, we present a Deep inteRActive Multi-task leArning scheme, DRAMA for short. DRAMA comparatively quantifies the perceptions of urban attributes by jointly integrating the pairwise comparisons, regional interactions, and urban attribute correlations within a unified deep scheme. In DRAMA, each urban attribute is treated as a task, whereby the task-sharing and the task-specific information is fully explored. By conducting extensive experiments over a public large-scale benchmark dataset, it is demonstrated that our proposed DRAMA scheme outperforms several state-of-the-art baselines. Meanwhile, we applied the pairwise comparisons of our DRAMA model to further quantify the urban attributes and hence rank cities with respect to the given urban attributes. As a byproduct, we have released the codes and parameter settings to facilitate other researches.

scholarly journals Anomaly Detection and Diagnosis In Manufacturing Systems

2019 ◽  
Vol 11 (1) ◽  
Author(s):  
Kalyani Zope ◽  
Kuldeep Singh ◽  
Sri Harsha Nistala ◽  
Arghya Basak ◽  
Pradeep Rathore ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Anomaly Detection ◽  
Manufacturing Systems ◽  
Predictive Maintenance ◽  
Sensor Data ◽  
Statistical Techniques ◽  
Industrial Systems ◽  
Learning Techniques ◽  
Detection And Diagnosis

Multivariate sensor data collected from manufacturing and process industries represents actual operational behavior and can be used for predictive maintenance of the plants. Anomaly detection and diagnosis, that forms an integral part of predictive maintenance, in industrial systems is however challenging due to their complex behavior, interactions among sensors, corrective actions of control systems and variability in anomalous behavior. While several statistical techniques for anomaly detection have been in use for a long time, these are not particularly suited for temporal (or contextual) anomalies that are characteristic of multivariate time series sensor data. On the other hand, several machine learning and deep learning techniques for anomaly detection gained significant interest in the recent years. Further, anomaly diagnosis that involves localization of the faults did not receive much attention. In this work, we compare the anomaly detection and diagnosis capabilities, in semi-supervised mode, of several statistical, machine learning and deep learning techniques on two systems viz. the interacting quadruple tank system and the continuous stirred tank reactor (CSTR) system both of which are representative of the complexity of large industrial systems. The techniques studied include principal component analysis (PCA), Mahalanobis distance (MD), one-class support vector machine (OCSVM), isolation forest, elliptic envelope, dense auto-encoder and long short term memory auto-encoder (LSTM AE). The study revealed that MD and LSTM-AE have the highest anomaly detection capability, followed closely by PCA and OCSVM. The above techniques also exhibited good diagnosis capability. The study indicates that statistical techniques in spite of their simplicity could be as powerful as machine learning and deep learning techniques, and may be considered for anomaly detection and diagnosis in manufacturing systems.

scholarly journals On the Usage of Generative Models for Network Anomaly Detection in Multivariate Time-Series

2021 ◽  
Vol 48 (4) ◽  
pp. 49-52
Author(s):  
Gastón García González ◽  
Pedro Casas ◽  
Alicia Fernández ◽  
Gabriel Gómez
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Communication Networks ◽  
Multivariate Time Series ◽  
Data Communication ◽  
Generative Models ◽  
Sensor Data ◽  
Underlying Distribution ◽  
Novel Approach ◽  
Network Anomaly Detection

Despite the many attempts and approaches for anomaly de- tection explored over the years, the automatic detection of rare events in data communication networks remains a com- plex problem. In this paper we introduce Net-GAN, a novel approach to network anomaly detection in time-series, us- ing recurrent neural networks (RNNs) and generative ad- versarial networks (GAN). Different from the state of the art, which traditionally focuses on univariate measurements, Net-GAN detects anomalies in multivariate time-series, ex- ploiting temporal dependencies through RNNs. Net-GAN discovers the underlying distribution of the baseline, multi- variate data, without making any assumptions on its nature, offering a powerful approach to detect anomalies in com- plex, difficult to model network monitoring data. We further exploit the concepts behind generative models to conceive Net-VAE, a complementary approach to Net-GAN for net- work anomaly detection, based on variational auto-encoders (VAE). We evaluate Net-GAN and Net-VAE in different monitoring scenarios, including anomaly detection in IoT sensor data, and intrusion detection in network measure- ments. Generative models represent a promising approach for network anomaly detection, especially when considering the complexity and ever-growing number of time-series to monitor in operational networks.

scholarly journals A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data

2019 ◽  
Vol 33 ◽  
pp. 1409-1416 ◽  
Author(s):  
Chuxu Zhang ◽  
Dongjin Song ◽  
Yuncong Chen ◽  
Xinyang Feng ◽  
Cristian Lumezanu ◽  
...  
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Power Plants ◽  
Time Series Data ◽  
Multivariate Time Series ◽  
Series Data ◽  
Feature Maps ◽  
Multi Scale ◽  
Unsupervised Anomaly Detection ◽  
Detection And Diagnosis

Nowadays, multivariate time series data are increasingly collected in various real world systems, e.g., power plants, wearable devices, etc. Anomaly detection and diagnosis in multivariate time series refer to identifying abnormal status in certain time steps and pinpointing the root causes. Building such a system, however, is challenging since it not only requires to capture the temporal dependency in each time series, but also need encode the inter-correlations between different pairs of time series. In addition, the system should be robust to noise and provide operators with different levels of anomaly scores based upon the severity of different incidents. Despite the fact that a number of unsupervised anomaly detection algorithms have been developed, few of them can jointly address these challenges. In this paper, we propose a Multi-Scale Convolutional Recurrent Encoder-Decoder (MSCRED), to perform anomaly detection and diagnosis in multivariate time series data. Specifically, MSCRED first constructs multi-scale (resolution) signature matrices to characterize multiple levels of the system statuses in different time steps. Subsequently, given the signature matrices, a convolutional encoder is employed to encode the inter-sensor (time series) correlations and an attention based Convolutional Long-Short Term Memory (ConvLSTM) network is developed to capture the temporal patterns. Finally, based upon the feature maps which encode the inter-sensor correlations and temporal information, a convolutional decoder is used to reconstruct the input signature matrices and the residual signature matrices are further utilized to detect and diagnose anomalies. Extensive empirical studies based on a synthetic dataset and a real power plant dataset demonstrate that MSCRED can outperform state-ofthe-art baseline methods.

Scalable Anomaly Detection for Large-Scale Heterogeneous Data in Cloud Using Optimal Elliptic Curve Cryptography and Gaussian Kernel Fuzzy C-Means Clustering

2019 ◽  
Vol 29 (05) ◽  
pp. 2050074 ◽  
Author(s):  
P. Santhosh Kumar ◽  
Latha Parthiban
Keyword(s):  
Anomaly Detection ◽  
Elliptic Curve ◽  
Elliptic Curve Cryptography ◽  
Large Scale ◽  
Privacy Preserving ◽  
Sensor Data ◽  
Gaussian Kernel ◽  
Cloud Platform ◽  
Privacy And Security ◽  
Cloud Data

In most systems, a smart functionality is enabled through an essential vital service such as detecting anomalies from complex, large-scale and dynamic data. However, ensuring the privacy and security for the cloud data is the most crucial and challenging task in the present world. Moreover, it is important to safeguard the security of sensitive data and its privacy from unauthorized parties who are trying to access the data. Therefore, to accomplish this task, several encryption, decryption and key generation mechanisms were introduced in the existing works for privacy preserving in cloud platform. But, there still remain open issues such as increased communication overhead, reduced security and increased time consumption. Also, these existing works followed the symmetric key cryptographic mechanism for privacy preservation of data; hence, a single secret key is shared by several users for accessing the original data. Due to this fact, a high security risk arises and it allows unauthorized parties to access the data. Thus, this work introduces a cloud-based privacy preserving model for offering a scalable and reliable anomaly detection service for sensor data through holding the benefits of cloud resources. Also, this paper aims to impose a newly developed Elliptic Curve Cryptography-based Collective Decision Optimization (ECDO) approach over the proposed framework for improving the privacy and security of the data. Furthermore, to perform the data clustering computation we used the Gaussian kernel fuzzy [Formula: see text]-means clustering (GKFCM) algorithm within the cloud platform, especially for data partitioning and to classify the anomalies. Thus, the computational difficulties are limited by adopting this suitable privacy preserving model which collaborates a private server and a set of public servers through a cloud data center. Moreover, on encrypted data the granular anomaly detection operations are performed by the virtual nodes executed over public servers. Experimental validation was performed on four datasets resulting from Intel Labs publicly available sensor data. The experimental outcomes demonstrate the ability of the proposed framework in providing high anomaly detection accuracy without any degradation in data privacy.

Sign in / Sign up

Export Citation Format

Share Document