scholarly journals Outlier Detection for Sensor Systems (ODSS): A MATLAB Macro for Evaluating Microphone Sensor Data Quality

Sensors ◽  
2017 ◽  
Vol 17 (10) ◽  
pp. 2329 ◽  
Author(s):  
Robert Vasta ◽  
Ian Crandell ◽  
Anthony Millican ◽  
Leanna House ◽  
Eric Smith
Keyword(s):  
Data Quality ◽  
Outlier Detection ◽  
Sensor Data ◽  
Sensor Systems

Using Machine Learning for Dependable Outlier Detection in Environmental Monitoring Systems

2021 ◽  
Vol 5 (3) ◽  
pp. 1-30
Author(s):  
Gonçalo Jesus ◽  
António Casimiro ◽  
Anabela Oliveira
Keyword(s):  
Machine Learning ◽  
Environmental Monitoring ◽  
Data Quality ◽  
Outlier Detection ◽  
Prediction Models ◽  
Sensor Data ◽  
Natural Phenomenon ◽  
Monitoring Systems ◽  
Data Errors ◽  
Redundant Data

Sensor platforms used in environmental monitoring applications are often subject to harsh environmental conditions while monitoring complex phenomena. Therefore, designing dependable monitoring systems is challenging given the external disturbances affecting sensor measurements. Even the apparently simple task of outlier detection in sensor data becomes a hard problem, amplified by the difficulty in distinguishing true data errors due to sensor faults from deviations due to natural phenomenon, which look like data errors. Existing solutions for runtime outlier detection typically assume that the physical processes can be accurately modeled, or that outliers consist in large deviations that are easily detected and filtered by appropriate thresholds. Other solutions assume that it is possible to deploy multiple sensors providing redundant data to support voting-based techniques. In this article, we propose a new methodology for dependable runtime detection of outliers in environmental monitoring systems, aiming to increase data quality by treating them. We propose the use of machine learning techniques to model each sensor behavior, exploiting the existence of correlated data provided by other related sensors. Using these models, along with knowledge of processed past measurements, it is possible to obtain accurate estimations of the observed environment parameters and build failure detectors that use these estimations. When a failure is detected, these estimations also allow one to correct the erroneous measurements and hence improve the overall data quality. Our methodology not only allows one to distinguish truly abnormal measurements from deviations due to complex natural phenomena, but also allows the quantification of each measurement quality, which is relevant from a dependability perspective. We apply the methodology to real datasets from a complex aquatic monitoring system, measuring temperature and salinity parameters, through which we illustrate the process for building the machine learning prediction models using a technique based on Artificial Neural Networks, denoted ANNODE ( ANN Outlier Detection ). From this application, we also observe the effectiveness of our ANNODE approach for accurate outlier detection in harsh environments. Then we validate these positive results by comparing ANNODE with state-of-the-art solutions for outlier detection. The results show that ANNODE improves existing solutions regarding accuracy of outlier detection.

scholarly journals Outlier Detection Transilience-Probabilistic Model for Wind Tunnels Based on Sensor Data

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2532
Author(s):  
Encarna Quesada ◽  
Juan J. Cuadrado-Gallego ◽  
Miguel Ángel Patricio ◽  
Luis Usero
Keyword(s):  
Anomaly Detection ◽  
Outlier Detection ◽  
Research Work ◽  
Industrial Sector ◽  
Sensor Data ◽  
Correct Selection ◽  
Wind Tunnels ◽  
Data Set ◽  
First Case ◽  
Anomaly Analysis

Anomaly Detection research is focused on the development and application of methods that allow for the identification of data that are different enough—compared with the rest of the data set that is being analyzed—and considered anomalies (or, as they are more commonly called, outliers). These values mainly originate from two sources: they may be errors introduced during the collection or handling of the data, or they can be correct, but very different from the rest of the values. It is essential to correctly identify each type as, in the first case, they must be removed from the data set but, in the second case, they must be carefully analyzed and taken into account. The correct selection and use of the model to be applied to a specific problem is fundamental for the success of the anomaly detection study and, in many cases, the use of only one model cannot provide sufficient results, which can be only reached by using a mixture model resulting from the integration of existing and/or ad hoc-developed models. This is the kind of model that is developed and applied to solve the problem presented in this paper. This study deals with the definition and application of an anomaly detection model that combines statistical models and a new method defined by the authors, the Local Transilience Outlier Identification Method, in order to improve the identification of outliers in the sensor-obtained values of variables that affect the operations of wind tunnels. The correct detection of outliers for the variables involved in wind tunnel operations is very important for the industrial ventilation systems industry, especially for vertical wind tunnels, which are used as training facilities for indoor skydiving, as the incorrect performance of such devices may put human lives at risk. In consequence, the use of the presented model for outlier detection may have a high impact in this industrial sector. In this research work, a proof-of-concept is carried out using data from a real installation, in order to test the proposed anomaly analysis method and its application to control the correct performance of wind tunnels.

scholarly journals A Hybrid Approach: Dynamic Diagnostic Rules for Sensor Systems in Industry 4.0 Generated by Online Hyperparameter Tuned Random Forest

2020 ◽  
Author(s):  
Ahlam Mallak ◽  
Madjid Fathi
Keyword(s):  
Random Forest ◽  
Random Search ◽  
Hybrid Approach ◽  
Fault Detection And Diagnosis ◽  
Data Driven ◽  
Sensor Data ◽  
Sensor Systems ◽  
Hydraulic Test ◽  
Model Based ◽  
Detection And Diagnosis

In this work, A hybrid component Fault Detection and Diagnosis (FDD) approach for industrial sensor systems is established and analyzed, to provide a hybrid schema that combines the advantages and eliminates the drawbacks of both model-based and data-driven methods of diagnosis. Moreover, spotting the light on a new utilization of Random Forest (RF) together with model-based diagnosis, beyond its ordinary data-driven application. RF is trained and hyperparameter tuned using 3-fold cross-validation over a random grid of parameters using random search, to finally generate diagnostic graphs as the dynamic, data-driven part of this system. Followed by translating those graphs into model-based rules in the form of if-else statements, SQL queries or semantic queries such as SPARQL, in order to feed the dynamic rules into a structured model essential for further diagnosis. The RF hyperparameters are consistently updated online using the newly generated sensor data, in order to maintain the dynamicity and accuracy of the generated graphs and rules thereafter. The architecture of the proposed method is demonstrated in a comprehensive manner, as well as the dynamic rules extraction phase is applied using a case study on condition monitoring of a hydraulic test rig using time series multivariate sensor readings.

scholarly journals Outlier Detection Using Improved Support Vector Data Description in Wireless Sensor Networks

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4712
Author(s):  
Pei Shi ◽  
Guanghui Li ◽  
Yongming Yuan ◽  
Liang Kuang
Keyword(s):  
Wireless Sensor Networks ◽  
Sensor Networks ◽  
Relative Density ◽  
Outlier Detection ◽  
Sensor Data ◽  
Wireless Sensor ◽  
Support Vector ◽  
Support Vector Data Description ◽  
Vector Data ◽  
Data Description

Wireless sensor networks (WSNs) are susceptible to faults in sensor data. Outlier detection is crucial for ensuring the quality of data analysis in WSNs. This paper proposes a novel improved support vector data description method (ID-SVDD) to effectively detect outliers of sensor data. ID-SVDD utilizes the density distribution of data to compensate SVDD. The Parzen-window algorithm is applied to calculate the relative density for each data point in a data set. Meanwhile, we use Mahalanobis distance (MD) to improve the Gaussian function in Parzen-window density estimation. Through combining new relative density weight with SVDD, this approach can efficiently map the data points from sparse space to high-density space. In order to assess the outlier detection performance, the ID-SVDD algorithm was implemented on several datasets. The experimental results demonstrated that ID-SVDD achieved high performance, and could be applied in real water quality monitoring.

scholarly journals Outlier Detection in Wireless Sensor Networks Using Model Selection-Based Support Vector Data Descriptions

Sensors ◽  
2018 ◽  
Vol 18 (12) ◽  
pp. 4328 ◽  
Author(s):  
Zhan Huan ◽  
Chang Wei ◽  
Guang-Hui Li
Keyword(s):  
Wireless Sensor Networks ◽  
Sensor Networks ◽  
Model Selection ◽  
Outlier Detection ◽  
Sensor Data ◽  
Wireless Sensor ◽  
Quality Data ◽  
Support Vector ◽  
Support Vector Data Description ◽  
Vector Data

Wireless sensor networks (WSNs) are often deployed in harsh and unattended environments, which may cause the generation of abnormal or low quality data. The inaccurate and unreliable sensor data may increase generation of false alarms and erroneous decisions, so it’s very important to detect outliers in sensor data efficiently and accurately to ensure sound scientific decision-making. In this paper, an outlier detection algorithm (TSVDD) using model selection-based support vector data description (SVDD) is proposed. Firstly, the Toeplitz matrix random feature mapping is used to reduce the time and space complexity of outlier detection. Secondly, a novel model selection strategy is realized to keep the algorithm stable under the low feature dimensions, this strategy can select a relatively optimal decision model and avoid both under-fitting and overfitting phenomena. The simulation results on SensorScope and IBRL datasets demonstrate that, TSVDD achieves higher accuracy and lower time complexity for outlier detection in WSNs compared with existing methods.

A Distributed Storage System Using a Mobile Ad hoc Network

2019 ◽  
Vol 10 (3) ◽  
pp. 76-89
Author(s):  
Arhantika Nathaniel ◽  
Angelic Goyal ◽  
Parmeet Kaur
Keyword(s):  
Data Quality ◽  
Mobile Ad Hoc Network ◽  
Indoor Localization ◽  
Distributed Storage ◽  
Storage System ◽  
Ambient Assisted Living ◽  
Quality Criteria ◽  
Sensor Data ◽  
Evaluation Procedure ◽  
Quality Model

The Ambient Assisted Living (AAL) domain aims to support the daily life activities of elders, patients with chronic conditions, and disabled people. Several AAL platforms have been developed over the last two decades. Hence, there is a need to identify Quality Criteria (QC) and make it well defined in order to achieve the AAL system purposes. To be able to convince all stakeholders including both technologies and end users of AAL systems, high quality must be guaranteed. The goal of this article is to obtain a set of data quality characteristics that would be applicable to AAL system, and have its performance evaluated using sensor data. To this end, this work uses the ISO/IEC 25012 and ISO/IEC 25010 standards to extract the most relevant criteria that are apt for AAL systems. As a result, an evaluation approach on an indoor localization platform was made, and an evaluation procedure has been established. This is done by first generating a hierarchical data quality model, and have it evaluated using the metrics, based on the sensor data and the concept of fuzzy logic.

Sign in / Sign up

Export Citation Format

Share Document