An Intelligent Outlier Detection Method With One Class Support Tucker Machine and Genetic Algorithm Toward Big Sensor Data in Internet of Things

2019 ◽  
Vol 66 (6) ◽  
pp. 4672-4683 ◽  
Author(s):  
Xiaowu Deng ◽  
Peng Jiang ◽  
Xiaoning Peng ◽  
Chunqiao Mi
Keyword(s):  
Genetic Algorithm ◽  
Internet Of Things ◽  
Outlier Detection ◽  
Detection Method ◽  
Sensor Data

scholarly journals AdaGUM: An Adaptive Graph Updating Model-Based Anomaly Detection Method for Edge Computing Environment

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Xiang Yu ◽  
Chun Shan ◽  
Jilong Bian ◽  
Xianfei Yang ◽  
Ying Chen ◽  
...  
Keyword(s):  
Internet Of Things ◽  
Anomaly Detection ◽  
Real Time ◽  
Detection Method ◽  
Edge Computing ◽  
Sensor Data ◽  
Computing Environment ◽  
The Real ◽  
Edge Node ◽  
Model Based

With the rapid development of Internet of Things (IoT), massive sensor data are being generated by the sensors deployed everywhere at an unprecedented rate. As the number of Internet of Things devices is estimated to grow to 25 billion by 2021, when facing the explicit or implicit anomalies in the real-time sensor data collected from Internet of Things devices, it is necessary to develop an effective and efficient anomaly detection method for IoT devices. Recent advances in the edge computing have significant impacts on the solution of anomaly detection in IoT. In this study, an adaptive graph updating model is first presented, based on which a novel anomaly detection method for edge computing environment is then proposed. At the cloud center, the unknown patterns are classified by a deep leaning model, based on the classification results, the feature graphs are updated periodically, and the classification results are constantly transmitted to each edge node where a cache is employed to keep the newly emerging anomalies or normal patterns temporarily until the edge node receives a newly updated feature graph. Finally, a series of comparison experiments are conducted to demonstrate the effectiveness of the proposed anomaly detection method for edge computing. And the results show that the proposed method can detect the anomalies in the real-time sensor data efficiently and accurately. More than that, the proposed method performs well when there exist newly emerging patterns, no matter they are anomalous or normal.

scholarly journals Outlier Detection for Sensor Systems (ODSS): A MATLAB Macro for Evaluating Microphone Sensor Data Quality

Sensors ◽  
2017 ◽  
Vol 17 (10) ◽  
pp. 2329 ◽  
Author(s):  
Robert Vasta ◽  
Ian Crandell ◽  
Anthony Millican ◽  
Leanna House ◽  
Eric Smith
Keyword(s):  
Data Quality ◽  
Outlier Detection ◽  
Sensor Data ◽  
Sensor Systems

Using Machine Learning for Dependable Outlier Detection in Environmental Monitoring Systems

2021 ◽  
Vol 5 (3) ◽  
pp. 1-30
Author(s):  
Gonçalo Jesus ◽  
António Casimiro ◽  
Anabela Oliveira
Keyword(s):  
Machine Learning ◽  
Environmental Monitoring ◽  
Data Quality ◽  
Outlier Detection ◽  
Prediction Models ◽  
Sensor Data ◽  
Natural Phenomenon ◽  
Monitoring Systems ◽  
Data Errors ◽  
Redundant Data

Sensor platforms used in environmental monitoring applications are often subject to harsh environmental conditions while monitoring complex phenomena. Therefore, designing dependable monitoring systems is challenging given the external disturbances affecting sensor measurements. Even the apparently simple task of outlier detection in sensor data becomes a hard problem, amplified by the difficulty in distinguishing true data errors due to sensor faults from deviations due to natural phenomenon, which look like data errors. Existing solutions for runtime outlier detection typically assume that the physical processes can be accurately modeled, or that outliers consist in large deviations that are easily detected and filtered by appropriate thresholds. Other solutions assume that it is possible to deploy multiple sensors providing redundant data to support voting-based techniques. In this article, we propose a new methodology for dependable runtime detection of outliers in environmental monitoring systems, aiming to increase data quality by treating them. We propose the use of machine learning techniques to model each sensor behavior, exploiting the existence of correlated data provided by other related sensors. Using these models, along with knowledge of processed past measurements, it is possible to obtain accurate estimations of the observed environment parameters and build failure detectors that use these estimations. When a failure is detected, these estimations also allow one to correct the erroneous measurements and hence improve the overall data quality. Our methodology not only allows one to distinguish truly abnormal measurements from deviations due to complex natural phenomena, but also allows the quantification of each measurement quality, which is relevant from a dependability perspective. We apply the methodology to real datasets from a complex aquatic monitoring system, measuring temperature and salinity parameters, through which we illustrate the process for building the machine learning prediction models using a technique based on Artificial Neural Networks, denoted ANNODE ( ANN Outlier Detection ). From this application, we also observe the effectiveness of our ANNODE approach for accurate outlier detection in harsh environments. Then we validate these positive results by comparing ANNODE with state-of-the-art solutions for outlier detection. The results show that ANNODE improves existing solutions regarding accuracy of outlier detection.

scholarly journals Outlier Detection Transilience-Probabilistic Model for Wind Tunnels Based on Sensor Data

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2532
Author(s):  
Encarna Quesada ◽  
Juan J. Cuadrado-Gallego ◽  
Miguel Ángel Patricio ◽  
Luis Usero
Keyword(s):  
Anomaly Detection ◽  
Outlier Detection ◽  
Research Work ◽  
Industrial Sector ◽  
Sensor Data ◽  
Correct Selection ◽  
Wind Tunnels ◽  
Data Set ◽  
First Case ◽  
Anomaly Analysis

Anomaly Detection research is focused on the development and application of methods that allow for the identification of data that are different enough—compared with the rest of the data set that is being analyzed—and considered anomalies (or, as they are more commonly called, outliers). These values mainly originate from two sources: they may be errors introduced during the collection or handling of the data, or they can be correct, but very different from the rest of the values. It is essential to correctly identify each type as, in the first case, they must be removed from the data set but, in the second case, they must be carefully analyzed and taken into account. The correct selection and use of the model to be applied to a specific problem is fundamental for the success of the anomaly detection study and, in many cases, the use of only one model cannot provide sufficient results, which can be only reached by using a mixture model resulting from the integration of existing and/or ad hoc-developed models. This is the kind of model that is developed and applied to solve the problem presented in this paper. This study deals with the definition and application of an anomaly detection model that combines statistical models and a new method defined by the authors, the Local Transilience Outlier Identification Method, in order to improve the identification of outliers in the sensor-obtained values of variables that affect the operations of wind tunnels. The correct detection of outliers for the variables involved in wind tunnel operations is very important for the industrial ventilation systems industry, especially for vertical wind tunnels, which are used as training facilities for indoor skydiving, as the incorrect performance of such devices may put human lives at risk. In consequence, the use of the presented model for outlier detection may have a high impact in this industrial sector. In this research work, a proof-of-concept is carried out using data from a real installation, in order to test the proposed anomaly analysis method and its application to control the correct performance of wind tunnels.

scholarly journals Developing an On-Demand Cloud-Based Sensing-as-a-Service System for Internet of Things

2016 ◽  
Vol 2016 ◽  
pp. 1-17 ◽  
Author(s):  
Mihui Kim ◽  
Mihir Asthana ◽  
Siddhartha Bhargava ◽  
Kartik Krishnan Iyyer ◽  
Rohan Tangadpalliwar ◽  
...  
Keyword(s):  
Sensor Networks ◽  
Internet Of Things ◽  
Service System ◽  
Value Added ◽  
Computing System ◽  
Sensor Data ◽  
Dynamic Provisioning ◽  
On Demand ◽  
Cloud Computing System ◽  
Iot Devices

The increasing number of Internet of Things (IoT) devices with various sensors has resulted in a focus on Cloud-based sensing-as-a-service (CSaaS) as a new value-added service, for example, providing temperature-sensing data via a cloud computing system. However, the industry encounters various challenges in the dynamic provisioning of on-demand CSaaS on diverse sensor networks. We require a system that will provide users with standardized access to various sensor networks and a level of abstraction that hides the underlying complexity. In this study, we aim to develop a cloud-based solution to address the challenges mentioned earlier. Our solution, SenseCloud, includes asensor virtualizationmechanism that interfaces with diverse sensor networks, amultitenancymechanism that grants multiple users access to virtualized sensor networks while sharing the same underlying infrastructure, and adynamic provisioningmechanism to allow the users to leverage the vast pool of resources on demand and on a pay-per-use basis. We implement a prototype of SenseCloud by using real sensors and verify the feasibility of our system and its performance. SenseCloud bridges the gap between sensor providers and sensor data consumers who wish to utilize sensor data.

Sign in / Sign up

Export Citation Format

Share Document