Anomaly Detection Methods for IIoT Networks

2018 ◽  
Author(s):  
Luying Zhou ◽  
Huaqun Guo
Keyword(s):  
Anomaly Detection ◽  
Detection Methods

Un-Supervised approach to backorder prediction using deep autoencoder

2019 ◽  
Vol 12 ◽  
Author(s):  
Gunjan Saraogi ◽  
Deepa Gupta ◽  
Lavanya Sharma ◽  
Ajay Rana
Keyword(s):  
Anomaly Detection ◽  
Predictive Model ◽  
Inventory Management ◽  
Supervised Classification ◽  
Operating Characteristic ◽  
Imbalanced Data ◽  
Bullwhip Effect ◽  
Detection Methods ◽  
Good Representation ◽  
Operational Management

Background: Backorders are an accepted abnormality affecting accumulation alternation and logistics, sales, chump service, and manufacturing, which generally leads to low sales and low chump satisfaction. A predictive archetypal can analyse which articles are best acceptable to acquaintance backorders giving the alignment advice and time to adjust, thereby demography accomplishes to aerate their profit. Objective: To address the issue of predicting backorders, this paper has proposed an un-supervised approach to backorder prediction using Deep Autoencoder. Method: In this paper, artificial intelligence paradigms are researched in order to introduce a predictive model for the present unbalanced data issues, where the number of products going on backorder is rare. Result: Un-supervised anomaly detection using deep auto encoders has shown better Area under the Receiver Operating Characteristic and precision-recall curves than supervised classification techniques employed with resampling techniques for imbalanced data problems. Conclusion: We demonstrated that Un-supervised anomaly detection methods specifically deep auto-encoders can be used to learn a good representation of the data. The method can be used as predictive model for inventory management and help to reduce bullwhip effect, raise customer satisfaction as well as improve operational management in the organization. This technology is expected to create the sentient supply chain of the future – able to feel, perceive and react to situations at an extraordinarily granular level

scholarly journals OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4805
Author(s):  
Saad Abbasi ◽  
Mahmoud Famouri ◽  
Mohammad Javad Shafiee ◽  
Alexander Wong
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Detection Methods ◽  
Detection Accuracy ◽  
Network Architectures ◽  
Design Exploration ◽  
Convolutional Autoencoder ◽  
Acoustic Anomaly ◽  
Human Operators ◽  
Computational Resources

Human operators often diagnose industrial machinery via anomalous sounds. Given the new advances in the field of machine learning, automated acoustic anomaly detection can lead to reliable maintenance of machinery. However, deep learning-driven anomaly detection methods often require an extensive amount of computational resources prohibiting their deployment in factories. Here we explore a machine-driven design exploration strategy to create OutlierNets, a family of highly compact deep convolutional autoencoder network architectures featuring as few as 686 parameters, model sizes as small as 2.7 KB, and as low as 2.8 million FLOPs, with a detection accuracy matching or exceeding published architectures with as many as 4 million parameters. The architectures are deployed on an Intel Core i5 as well as a ARM Cortex A72 to assess performance on hardware that is likely to be used in industry. Experimental results on the model’s latency show that the OutlierNet architectures can achieve as much as 30x lower latency than published networks.

scholarly journals Edge Computing for Data Anomaly Detection of Multi-Sensors in Underground Mining

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 302
Author(s):  
Chunde Liu ◽  
Xianli Su ◽  
Chuanwen Li
Keyword(s):  
Energy Consumption ◽  
Anomaly Detection ◽  
Underground Mining ◽  
Heterogeneous Data ◽  
Edge Computing ◽  
Sensor Nodes ◽  
Detection Methods ◽  
Detection Accuracy ◽  
Clustering Methods ◽  
Safety Warning

There is a growing interest in safety warning of underground mining due to the huge threat being faced by those working in underground mining. Data acquisition of sensors based on Internet of Things (IoT) is currently the main method, but the data anomaly detection and analysis of multi-sensors is a challenging task: firstly, the data that are collected by different sensors of underground mining are heterogeneous; secondly, real-time is required for the data anomaly detection of safety warning. Currently, there are many anomaly detection methods, such as traditional clustering methods K-means and C-means. Meanwhile, Artificial Intelligence (AI) is widely used in data analysis and prediction. However, K-means and C-means cannot directly process heterogeneous data, and AI algorithms require equipment with high computing and storage capabilities. IoT equipment of underground mining cannot perform complex calculation due to the limitation of energy consumption. Therefore, many existing methods cannot be directly used for IoT applications in underground mining. In this paper, a multi-sensors data anomaly detection method based on edge computing is proposed. Firstly, an edge computing model is designed, and according to the computing capabilities of different types of devices, anomaly detection tasks are migrated to different edge devices, which solve the problem of insufficient computing capabilities of the devices. Secondly, according to the requirements of different anomaly detection tasks, edge anomaly detection algorithms for sensor nodes and sink nodes are designed respectively. Lastly, an experimental platform is built for performance comparison analysis, and the experimental results show that the proposed algorithm has better performance in anomaly detection accuracy, delay, and energy consumption.

scholarly journals Indirectly Supervised Anomaly Detection of Clinically Meaningful Health Events from Smart Home Data

2021 ◽  
Vol 12 (2) ◽  
pp. 1-18
Author(s):  
Jessamyn Dahmen ◽  
Diane J. Cook
Keyword(s):  
Anomaly Detection ◽  
Time Series Data ◽  
Bayesian Optimization ◽  
Detection Methods ◽  
Series Data ◽  
Detection Techniques ◽  
Health Events ◽  
Warm Start ◽  
Health Related ◽  
Unsupervised Algorithms

Anomaly detection techniques can extract a wealth of information about unusual events. Unfortunately, these methods yield an abundance of findings that are not of interest, obscuring relevant anomalies. In this work, we improve upon traditional anomaly detection methods by introducing Isudra, an Indirectly Supervised Detector of Relevant Anomalies from time series data. Isudra employs Bayesian optimization to select time scales, features, base detector algorithms, and algorithm hyperparameters that increase true positive and decrease false positive detection. This optimization is driven by a small amount of example anomalies, driving an indirectly supervised approach to anomaly detection. Additionally, we enhance the approach by introducing a warm-start method that reduces optimization time between similar problems. We validate the feasibility of Isudra to detect clinically relevant behavior anomalies from over 2M sensor readings collected in five smart homes, reflecting 26 health events. Results indicate that indirectly supervised anomaly detection outperforms both supervised and unsupervised algorithms at detecting instances of health-related anomalies such as falls, nocturia, depression, and weakness.

scholarly journals A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3627 ◽  
Author(s):  
Yi Zhang ◽  
Zebin Wu ◽  
Jin Sun ◽  
Yan Zhang ◽  
Yaoqin Zhu ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Sparse Representation ◽  
Parallel Algorithm ◽  
Hyperspectral Image ◽  
Hyperspectral Images ◽  
Low Rank ◽  
Detection Methods ◽  
Matrix Computations ◽  
Hyperspectral Image Processing ◽  
Mapreduce Model

Anomaly detection aims to separate anomalous pixels from the background, and has become an important application of remotely sensed hyperspectral image processing. Anomaly detection methods based on low-rank and sparse representation (LRASR) can accurately detect anomalous pixels. However, with the significant volume increase of hyperspectral image repositories, such techniques consume a significant amount of time (mainly due to the massive amount of matrix computations involved). In this paper, we propose a novel distributed parallel algorithm (DPA) by redesigning key operators of LRASR in terms of MapReduce model to accelerate LRASR on cloud computing architectures. Independent computation operators are explored and executed in parallel on Spark. Specifically, we reconstitute the hyperspectral images in an appropriate format for efficient DPA processing, design the optimized storage strategy, and develop a pre-merge mechanism to reduce data transmission. Besides, a repartitioning policy is also proposed to improve DPA’s efficiency. Our experimental results demonstrate that the newly developed DPA achieves very high speedups when accelerating LRASR, in addition to maintaining similar accuracies. Moreover, our proposed DPA is shown to be scalable with the number of computing nodes and capable of processing big hyperspectral images involving massive amounts of data.

scholarly journals Multivariate Anomaly Detection for Earth Observations: A Comparison of Algorithms and Feature Extraction Techniques

2016 ◽  
Author(s):  
Milan Flach ◽  
Fabian Gans ◽  
Alexander Brenning ◽  
Joachim Denzler ◽  
Markus Reichstein ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Anomaly Detection ◽  
Data Streams ◽  
Multivariate Data ◽  
Detection Methods ◽  
Earth System ◽  
Earth System Science ◽  
System Science ◽  
Detection Algorithms ◽  
Earth Observations

Abstract. Today, many processes at the Earth's surface are constantly monitored by multiple data streams. These observations have become central to advance our understanding of e.g. vegetation dynamics in response to climate or land use change. Another set of important applications is monitoring effects of climatic extreme events, other disturbances such as fires, or abrupt land transitions. One important methodological question is how to reliably detect anomalies in an automated and generic way within multivariate data streams, which typically vary seasonally and are interconnected across variables. Although many algorithms have been proposed for detecting anomalies in multivariate data, only few have been investigated in the context of Earth system science applications. In this study, we systematically combine and compare feature extraction and anomaly detection algorithms for detecting anomalous events. Our aim is to identify suitable workflows for automatically detecting anomalous patterns in multivariate Earth system data streams. We rely on artificial data that mimic typical properties and anomalies in multivariate spatiotemporal Earth observations. This artificial experiment is needed as there is no 'gold standard' for the identification of anomalies in real Earth observations. Our results show that a well chosen feature extraction step (e.g. subtracting seasonal cycles, or dimensionality reduction) is more important than the choice of a particular anomaly detection algorithm. Nevertheless, we identify 3 detection algorithms (k-nearest neighbours mean distance, kernel density estimation, a recurrence approach) and their combinations (ensembles) that outperform other multivariate approaches as well as univariate extreme event detection methods. Our results therefore provide an effective workflow to automatically detect anomalies in Earth system science data.

Detecting the Onset of Machine Failure Using Anomaly Detection Methods

2019 ◽  
pp. 3-12 ◽  
Author(s):  
Mohammad Riazi ◽  
Osmar Zaiane ◽  
Tomoharu Takeuchi ◽  
Anthony Maltais ◽  
Johannes Günther ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Detection Methods ◽  
Machine Failure

scholarly journals ConAnomaly: Content-Based Anomaly Detection for System Logs

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6125
Author(s):  
Dan Lv ◽  
Nurbol Luktarhan ◽  
Yiyong Chen
Keyword(s):  
Anomaly Detection ◽  
Semantic Information ◽  
Short Term Memory ◽  
Weighted Average ◽  
Detection Methods ◽  
Detection Model ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
System Logs ◽  
System Maintenance

Enterprise systems typically produce a large number of logs to record runtime states and important events. Log anomaly detection is efficient for business management and system maintenance. Most existing log-based anomaly detection methods use log parser to get log event indexes or event templates and then utilize machine learning methods to detect anomalies. However, these methods cannot handle unknown log types and do not take advantage of the log semantic information. In this article, we propose ConAnomaly, a log-based anomaly detection model composed of a log sequence encoder (log2vec) and multi-layer Long Short Term Memory Network (LSTM). We designed log2vec based on the Word2vec model, which first vectorized the words in the log content, then deleted the invalid words through part of speech tagging, and finally obtained the sequence vector by the weighted average method. In this way, ConAnomaly not only captures semantic information in the log but also leverages log sequential relationships. We evaluate our proposed approach on two log datasets. Our experimental results show that ConAnomaly has good stability and can deal with unseen log types to a certain extent, and it provides better performance than most log-based anomaly detection methods.

scholarly journals OUTLIER DETECTION METHOD USE FOR THE NETWORK FLOW ANOMALY DETECTION / IŠSKIRČIŲ RADIMO METODŲ TAIKYMAS ANOMALIJOMS KOMPIUTERIŲ TINKLO PAKETŲ SRAUTUOSE APTIKTI

2016 ◽  
Vol 8 (3) ◽  
pp. 327-333 ◽  
Author(s):  
Rimas Ciplinskas ◽  
Nerijus Paulauskas
Keyword(s):  
Anomaly Detection ◽  
Network Flow ◽  
Detection Method ◽  
Attack Detection ◽  
Detection Methods ◽  
Cyber Attack ◽  
Normal Network ◽  
New Type ◽  
Flow Detection ◽  
F Measure

New and existing methods of cyber-attack detection are constantly being developed and improved because there is a great number of attacks and the demand to protect from them. In prac-tice, current methods of attack detection operates like antivirus programs, i. e. known attacks signatures are created and attacks are detected by using them. These methods have a drawback – they cannot detect new attacks. As a solution, anomaly detection methods are used. They allow to detect deviations from normal network behaviour that may show a new type of attack. This article introduces a new method that allows to detect network flow anomalies by using local outlier factor algorithm. Accom-plished research allowed to identify groups of features which showed the best results of anomaly flow detection according the highest values of precision, recall and F-measure. Kibernetinių atakų gausa ir įvairovė bei siekis nuo jų apsisaugoti verčia nuolat kurti naujus ir tobulinti jau esamus atakų aptikimo metodus. Kaip rodo praktika, dabartiniai atakų atpažinimo metodai iš esmės veikia pagal antivirusinių programų principą, t.y. sudaromi žinomų atakų šablonai, kuriais remiantis yra aptinkamos atakos, tačiau pagrindinis tokių metodų trūkumas – negalėjimas aptikti naujų, dar nežinomų atakų. Šiai problemai spręsti yra pasitelkiami anomalijų aptikimo metodai, kurie leidžia aptikti nukrypimus nuo normalios tinklo būsenos. Straipsnyje yra pateiktas naujas metodas, leidžiantis aptikti kompiuterių tinklo paketų srauto anomalijas taikant lokalių išskirčių faktorių algoritmą. Atliktas tyrimas leido surasti požymių grupes, kurias taikant anomalūs tinklo srautai yra atpažįstami geriausiai, t. y. pasiekiamos didžiausios tikslumo, atkuriamumo ir F-mato reikšmės.

Sign in / Sign up

Export Citation Format

Share Document