Ecological Sensor Data Quality Assessed Using Observational Data and Combined Uncertainties

2019 ◽  
Author(s):  
Guy Litt ◽  
Janae Csavina ◽  
Joshua Roberti ◽  
Jesse Vance
Keyword(s):  
Data Quality ◽  
Observational Data ◽  
Sensor Data

scholarly journals Outlier Detection for Sensor Systems (ODSS): A MATLAB Macro for Evaluating Microphone Sensor Data Quality

Sensors ◽  
2017 ◽  
Vol 17 (10) ◽  
pp. 2329 ◽  
Author(s):  
Robert Vasta ◽  
Ian Crandell ◽  
Anthony Millican ◽  
Leanna House ◽  
Eric Smith
Keyword(s):  
Data Quality ◽  
Outlier Detection ◽  
Sensor Data ◽  
Sensor Systems

Using Machine Learning for Dependable Outlier Detection in Environmental Monitoring Systems

2021 ◽  
Vol 5 (3) ◽  
pp. 1-30
Author(s):  
Gonçalo Jesus ◽  
António Casimiro ◽  
Anabela Oliveira
Keyword(s):  
Machine Learning ◽  
Environmental Monitoring ◽  
Data Quality ◽  
Outlier Detection ◽  
Prediction Models ◽  
Sensor Data ◽  
Natural Phenomenon ◽  
Monitoring Systems ◽  
Data Errors ◽  
Redundant Data

Sensor platforms used in environmental monitoring applications are often subject to harsh environmental conditions while monitoring complex phenomena. Therefore, designing dependable monitoring systems is challenging given the external disturbances affecting sensor measurements. Even the apparently simple task of outlier detection in sensor data becomes a hard problem, amplified by the difficulty in distinguishing true data errors due to sensor faults from deviations due to natural phenomenon, which look like data errors. Existing solutions for runtime outlier detection typically assume that the physical processes can be accurately modeled, or that outliers consist in large deviations that are easily detected and filtered by appropriate thresholds. Other solutions assume that it is possible to deploy multiple sensors providing redundant data to support voting-based techniques. In this article, we propose a new methodology for dependable runtime detection of outliers in environmental monitoring systems, aiming to increase data quality by treating them. We propose the use of machine learning techniques to model each sensor behavior, exploiting the existence of correlated data provided by other related sensors. Using these models, along with knowledge of processed past measurements, it is possible to obtain accurate estimations of the observed environment parameters and build failure detectors that use these estimations. When a failure is detected, these estimations also allow one to correct the erroneous measurements and hence improve the overall data quality. Our methodology not only allows one to distinguish truly abnormal measurements from deviations due to complex natural phenomena, but also allows the quantification of each measurement quality, which is relevant from a dependability perspective. We apply the methodology to real datasets from a complex aquatic monitoring system, measuring temperature and salinity parameters, through which we illustrate the process for building the machine learning prediction models using a technique based on Artificial Neural Networks, denoted ANNODE ( ANN Outlier Detection ). From this application, we also observe the effectiveness of our ANNODE approach for accurate outlier detection in harsh environments. Then we validate these positive results by comparing ANNODE with state-of-the-art solutions for outlier detection. The results show that ANNODE improves existing solutions regarding accuracy of outlier detection.

A Distributed Storage System Using a Mobile Ad hoc Network

2019 ◽  
Vol 10 (3) ◽  
pp. 76-89
Author(s):  
Arhantika Nathaniel ◽  
Angelic Goyal ◽  
Parmeet Kaur
Keyword(s):  
Data Quality ◽  
Mobile Ad Hoc Network ◽  
Indoor Localization ◽  
Distributed Storage ◽  
Storage System ◽  
Ambient Assisted Living ◽  
Quality Criteria ◽  
Sensor Data ◽  
Evaluation Procedure ◽  
Quality Model

The Ambient Assisted Living (AAL) domain aims to support the daily life activities of elders, patients with chronic conditions, and disabled people. Several AAL platforms have been developed over the last two decades. Hence, there is a need to identify Quality Criteria (QC) and make it well defined in order to achieve the AAL system purposes. To be able to convince all stakeholders including both technologies and end users of AAL systems, high quality must be guaranteed. The goal of this article is to obtain a set of data quality characteristics that would be applicable to AAL system, and have its performance evaluated using sensor data. To this end, this work uses the ISO/IEC 25012 and ISO/IEC 25010 standards to extract the most relevant criteria that are apt for AAL systems. As a result, an evaluation approach on an indoor localization platform was made, and an evaluation procedure has been established. This is done by first generating a hierarchical data quality model, and have it evaluated using the metrics, based on the sensor data and the concept of fuzzy logic.

scholarly journals Automated quality control methods for sensor data: a novel observatory approach

2012 ◽  
Vol 9 (12) ◽  
pp. 18175-18210
Author(s):  
J. R. Taylor ◽  
H. L. Loescher
Keyword(s):  
Quality Control ◽  
Data Quality ◽  
Control Measures ◽  
Sensor Data ◽  
Data Quality Control ◽  
Test Parameter ◽  
Novel Approach ◽  
Level Data ◽  
Data Driven Approach ◽  
Automated Quality Control

Abstract. National and international networks and observatories of terrestrial-based sensors are emerging rapidly. As such, there is demand for a standardized approach to data quality control, as well as interoperability of data among sensor networks. The National Ecological Observatory Network (NEON) has begun constructing their first terrestrial observing sites with 60 locations expected to be distributed across the US by 2017. This will result in over 14 000 automated sensors recording more than > 100 Tb of data per year. These data are then used to create other datasets and subsequent "higher-level" data products. In anticipation of this challenge, an overall data quality assurance plan has been developed and the first suite of data quality control measures defined. This data-driven approach focuses on automated methods for defining a suite of plausibility test parameter thresholds. Specifically, these plausibility tests scrutinize data range, persistence, and stochasticity on each measurement type by employing a suite of binary checks. The statistical basis for each of these tests is developed and the methods for calculating test parameter thresholds are explored here. While these tests have been used elsewhere, we apply them in a novel approach by calculating their relevant test parameter thresholds. Finally, implementing automated quality control is demonstrated with preliminary data from a NEON prototype site.

Improving sensor data quality with predictive models

2021 ◽  
Author(s):  
Juliane Regina de Oliveira ◽  
Eduardo R. de Lima ◽  
Larissa M. de Almeida ◽  
Lucas Wanner
Keyword(s):  
Data Quality ◽  
Predictive Models ◽  
Sensor Data

scholarly journals Data Quality and Reliability Assessment of Wearable EMG and IMU Sensor for Construction Activity Recognition

Sensors ◽  
2020 ◽  
Vol 20 (18) ◽  
pp. 5264
Author(s):  
Srikanth Sagar Bangaru ◽  
Chao Wang ◽  
Fereydoun Aghazadeh
Keyword(s):  
Data Quality ◽  
Activity Recognition ◽  
Sensor Data ◽  
Measurement Unit ◽  
Activity Classification ◽  
Workforce Shortage ◽  
Safety And Health ◽  
Construction Activity ◽  
Sensor Models ◽  
Quality And Reliability

The workforce shortage is one of the significant problems in the construction industry. To overcome the challenges due to workforce shortage, various researchers have proposed wearable sensor-based systems in the area of construction safety and health. Although sensors provide rich and detailed information, not all sensors can be used for construction applications. This study evaluates the data quality and reliability of forearm electromyography (EMG) and inertial measurement unit (IMU) of armband sensors for construction activity classification. To achieve the proposed objective, the forearm EMG and IMU data collected from eight participants while performing construction activities such as screwing, wrenching, lifting, and carrying on two different days were used to analyze the data quality and reliability for activity recognition through seven different experiments. The results of these experiments show that the armband sensor data quality is comparable to the conventional EMG and IMU sensors with excellent relative and absolute reliability between trials for all the five activities. The activity classification results were highly reliable, with minimal change in classification accuracies for both the days. Moreover, the results conclude that the combined EMG and IMU models classify activities with higher accuracies compared to individual sensor models.

scholarly journals Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4410 ◽  
Author(s):  
Seunghwan Jeong ◽  
Gwangpyo Yoo ◽  
Minjong Yoo ◽  
Ikjun Yeom ◽  
Honguk Woo
Keyword(s):  
Reinforcement Learning ◽  
Data Quality ◽  
Data Management ◽  
Autonomous Systems ◽  
Sensor Data ◽  
Data Uncertainty ◽  
Seamless Integration ◽  
Digital Twin ◽  
Physical Attributes ◽  
Sensor Data Management

Hyperconnectivity via modern Internet of Things (IoT) technologies has recently driven us to envision “digital twin”, in which physical attributes are all embedded, and their latest updates are synchronized on digital spaces in a timely fashion. From the point of view of cyberphysical system (CPS) architectures, the goals of digital twin include providing common programming abstraction on the same level of databases, thereby facilitating seamless integration of real-world physical objects and digital assets at several different system layers. However, the inherent limitations of sampling and observing physical attributes often pose issues related to data uncertainty in practice. In this paper, we propose a learning-based data management scheme where the implementation is layered between sensors attached to physical attributes and domain-specific applications, thereby mitigating the data uncertainty between them. To do so, we present a sensor data management framework, namely D2WIN, which adopts reinforcement learning (RL) techniques to manage the data quality for CPS applications and autonomous systems. To deal with the scale issue incurred by many physical attributes and sensor streams when adopting RL, we propose an action embedding strategy that exploits their distance-based similarity in the physical space coordination. We introduce two embedding methods, i.e., a user-defined function and a generative model, for different conditions. Through experiments, we demonstrate that the D2WIN framework with the action embedding outperforms several known heuristics in terms of achievable data quality under certain resource restrictions. We also test the framework with an autonomous driving simulator, clearly showing its benefit. For example, with only 30% of updates selectively applied by the learned policy, the driving agent maintains its performance about 96.2%, as compared to the ideal condition with full updates.

Sign in / Sign up

Export Citation Format

Share Document