Managing Sensor Data Uncertainty

2013 ◽  
Vol 4 (1) ◽  
pp. 35-54 ◽  
Author(s):  
Claudia C. Gutiérrez Rodríguez ◽  
Sylvie Servigne
Keyword(s):  
Environmental Monitoring ◽  
Data Quality ◽  
Real World ◽  
Sensor Data ◽  
Data Uncertainty ◽  
Monitoring Systems ◽  
Quality Information ◽  
Technological Improvement ◽  
Monitoring Applications

With an increasingly technological improvement, sensors infrastructure actually supports many current and promising environmental applications. Environmental Monitoring Systems built on such sensors removes geographical, temporal and other restraints while increasing both the coverage and the quality of real world understanding. However, a main issue for such applications is the uncertainty of data coming from sensors, which may impact experts’ decisions. In this paper, the authors address this problem with an approach dedicated to provide environmental monitoring applications and users with data quality information.

Using Machine Learning for Dependable Outlier Detection in Environmental Monitoring Systems

2021 ◽  
Vol 5 (3) ◽  
pp. 1-30
Author(s):  
Gonçalo Jesus ◽  
António Casimiro ◽  
Anabela Oliveira
Keyword(s):  
Machine Learning ◽  
Environmental Monitoring ◽  
Data Quality ◽  
Outlier Detection ◽  
Prediction Models ◽  
Sensor Data ◽  
Natural Phenomenon ◽  
Monitoring Systems ◽  
Data Errors ◽  
Redundant Data

Sensor platforms used in environmental monitoring applications are often subject to harsh environmental conditions while monitoring complex phenomena. Therefore, designing dependable monitoring systems is challenging given the external disturbances affecting sensor measurements. Even the apparently simple task of outlier detection in sensor data becomes a hard problem, amplified by the difficulty in distinguishing true data errors due to sensor faults from deviations due to natural phenomenon, which look like data errors. Existing solutions for runtime outlier detection typically assume that the physical processes can be accurately modeled, or that outliers consist in large deviations that are easily detected and filtered by appropriate thresholds. Other solutions assume that it is possible to deploy multiple sensors providing redundant data to support voting-based techniques. In this article, we propose a new methodology for dependable runtime detection of outliers in environmental monitoring systems, aiming to increase data quality by treating them. We propose the use of machine learning techniques to model each sensor behavior, exploiting the existence of correlated data provided by other related sensors. Using these models, along with knowledge of processed past measurements, it is possible to obtain accurate estimations of the observed environment parameters and build failure detectors that use these estimations. When a failure is detected, these estimations also allow one to correct the erroneous measurements and hence improve the overall data quality. Our methodology not only allows one to distinguish truly abnormal measurements from deviations due to complex natural phenomena, but also allows the quantification of each measurement quality, which is relevant from a dependability perspective. We apply the methodology to real datasets from a complex aquatic monitoring system, measuring temperature and salinity parameters, through which we illustrate the process for building the machine learning prediction models using a technique based on Artificial Neural Networks, denoted ANNODE ( ANN Outlier Detection ). From this application, we also observe the effectiveness of our ANNODE approach for accurate outlier detection in harsh environments. Then we validate these positive results by comparing ANNODE with state-of-the-art solutions for outlier detection. The results show that ANNODE improves existing solutions regarding accuracy of outlier detection.

scholarly journals MULTI SENSOR AND PLATFORMS SETUPS FOR VARIOUS AIRBORNE APPLICATIONS

2016 ◽  
Vol XLI-B1 ◽  
pp. 211-215
Author(s):  
G. Kemper ◽  
R. Vasel
Keyword(s):  
High Resolution ◽  
Environmental Monitoring ◽  
Monitoring Systems ◽  
Sensor Systems ◽  
Monitoring Applications ◽  
Pros And Cons

To combine various sensors to get a system for specific use became popular within the last 10 years. Metric mid format cameras meanwhile reach the 100 MPix and entered the mapping market to compete with the big format sensors. Beside that also other sensors as SLR Cameras provide high resolution and enter the aerial surveying market for orthophoto production or monitoring applications. Flexibility, purchase-costs, size and weight are common aspects to design multi-sensor systems. Some sensors are useful for mapping while others are part of environmental monitoring systems. Beside classical surveying aircrafts also UL Airplanes, Para/Trikes or UAVs make use of multi sensor systems. Many of them are customer specific while other already are frequently used in the market. This paper aims to show some setup, their application, what are the results and what are the pros and cons of them are.

scholarly journals Adaptive error compensation for primary me-ters of environmental monitoring systems

2020 ◽  
pp. 157-162
Author(s):  
I.A. Skatkov ◽  
◽  
D.V. Moiseev ◽  
Keyword(s):  
Environmental Monitoring ◽  
Environmental Parameters ◽  
Critical Energy ◽  
Monitoring Systems ◽  
Main Task ◽  
Chemical Production ◽  
Separate Model ◽  
Adaptive Parameters ◽  
Adaptive Correction

The paper focuses on the construction of separate model representations in the study of reliability of data obtained from environmental monitoring systems. Modern systems for monitoring environmental parameters are complex complexes of different technical and software tools. Special requirements to the quality of monitoring systems arise when they are used as subsystems in systems of responsible application, monitoring the state of critical energy facilities, mining, transport, and chemical production processes. One of the main reasons for reducing the reliability of such data is degradation processes in the primary meter, which are often installed in aggressive environments with an unsatisfactory set of parameters in the places where they are located. As a result of degradation effects, there is a significant decrease in the accuracy of primary meters, up to complete failures. An approach is described that allows increasing the duration of continuous operation of the system for monitoring environmental parameters. The basis of this approach is adaptive correction of primary meter readings in the event of a decrease in their accuracy due to degradation failures. It is noted that in order to identify interdependencies in such a system, it is necessary to conduct a simulation of the adaptation process in the system for monitoring environmental parameters, and the task is to create such a model. The structure and equations of such a system are proposed, and the task of creating a simulation model of the system is set. Thus, the main task of the proposed approach is to extend the intervals between repairs of environmental monitoring systems. This is achieved by extending the service life of primary meters in the event of their degradation failures. This resource extension is achieved by creating an additional feedback channel with adaptive parameters in the system.

scholarly journals Increasing Trust in Real-World Evidence Through Evaluation of Observational Data Quality

2021 ◽  
Author(s):  
Clair Blacketer ◽  
Frank J Defalco ◽  
Patrick B Ryan ◽  
Peter R Rijnbeek
Keyword(s):  
Data Quality ◽  
Real World ◽  
R Package ◽  
Observational Research ◽  
Real World Data ◽  
Quality Reporting ◽  
Healthcare Data ◽  
Real World Evidence ◽  
Quality Checks

Advances in standardization of observational healthcare data have enabled methodological breakthroughs, rapid global collaboration, and generation of real-world evidence to improve patient outcomes. Standardizations in data structure, such as use of Common Data Models (CDM), need to be coupled with standardized approaches for data quality assessment. To ensure confidence in real-world evidence generated from the analysis of real-world data, one must first have confidence in the data itself. The Data Quality Dashboard is an open-source R package that reports potential quality issues in an OMOP CDM instance through the systematic execution and summarization of over 3,300 configurable data quality checks. We describe the implementation of check types across a data quality framework of conformance, completeness, plausibility, with both verification and validation. We illustrate how data quality checks, paired with decision thresholds, can be configured to customize data quality reporting across a range of observational health data sources. We discuss how data quality reporting can become part of the overall real-world evidence generation and dissemination process to promote transparency and build confidence in the resulting output. Transparently communicating how well CDM standardized databases adhere to a set of quality measures adds a crucial piece that is currently missing from observational research. Assessing and improving the quality of our data will inherently improve the quality of the evidence we generate.

scholarly journals Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4410 ◽  
Author(s):  
Seunghwan Jeong ◽  
Gwangpyo Yoo ◽  
Minjong Yoo ◽  
Ikjun Yeom ◽  
Honguk Woo
Keyword(s):  
Reinforcement Learning ◽  
Data Quality ◽  
Data Management ◽  
Autonomous Systems ◽  
Sensor Data ◽  
Data Uncertainty ◽  
Seamless Integration ◽  
Digital Twin ◽  
Physical Attributes ◽  
Sensor Data Management

Hyperconnectivity via modern Internet of Things (IoT) technologies has recently driven us to envision “digital twin”, in which physical attributes are all embedded, and their latest updates are synchronized on digital spaces in a timely fashion. From the point of view of cyberphysical system (CPS) architectures, the goals of digital twin include providing common programming abstraction on the same level of databases, thereby facilitating seamless integration of real-world physical objects and digital assets at several different system layers. However, the inherent limitations of sampling and observing physical attributes often pose issues related to data uncertainty in practice. In this paper, we propose a learning-based data management scheme where the implementation is layered between sensors attached to physical attributes and domain-specific applications, thereby mitigating the data uncertainty between them. To do so, we present a sensor data management framework, namely D2WIN, which adopts reinforcement learning (RL) techniques to manage the data quality for CPS applications and autonomous systems. To deal with the scale issue incurred by many physical attributes and sensor streams when adopting RL, we propose an action embedding strategy that exploits their distance-based similarity in the physical space coordination. We introduce two embedding methods, i.e., a user-defined function and a generative model, for different conditions. Through experiments, we demonstrate that the D2WIN framework with the action embedding outperforms several known heuristics in terms of achievable data quality under certain resource restrictions. We also test the framework with an autonomous driving simulator, clearly showing its benefit. For example, with only 30% of updates selectively applied by the learned policy, the driving agent maintains its performance about 96.2%, as compared to the ideal condition with full updates.

Data Quality of PAH Determinations in Environmental Monitoring

2004 ◽  
Vol 94 (9-10) ◽  
pp. 691-698 ◽  
Author(s):  
Elisabetta Borsella ◽  
Patrizia di Filippo ◽  
Carmela Riccardi ◽  
Sergio Spicaglia ◽  
Angelo Cecinato
Keyword(s):  
Environmental Monitoring ◽  
Data Quality

scholarly journals Developing a fuzzy logic based system for monitoring and early detection of residential fire based on thermistor sensors

2015 ◽  
Vol 12 (1) ◽  
pp. 63-89 ◽  
Author(s):  
Mirjana Maksimovic ◽  
Vladimir Vujovic ◽  
Branko Perisic ◽  
Vladimir Milosevic
Keyword(s):  
Fuzzy Logic ◽  
Real World ◽  
Network Performance ◽  
Performance Monitoring ◽  
Sensor Nodes ◽  
Sensor Data ◽  
Critical Event ◽  
The Internet ◽  
Network Performance Monitoring

The recent proliferation of global networking has an enormous impact on the cooperation of smart elements, of arbitrary kind and purpose that can be located anywhere and interact with each other according to the predefined protocol. Furthermore, these elements have to be intelligently orchestrated in order to support distributed sensing and/or monitoring/control of real world phenomena. That is why the Internet of Things (IoT) concept raises like a new, promising paradigm for Future Internet development. Considering that Wireless Sensor Networks (WSNs) are envisioned as integral part of arbitrary IoTs, and the potentially huge number of cooperating IoTs that are usually used in the real world phenomena monitoring and management, the reliability of individual sensor nodes and the overall network performance monitoring and improvement are definitely challenging issues. One of the most interesting real world phenomena that can be monitored by WSN is indoor or outdoor fire. The incorporation of soft computing technologies, like fuzzy logic, in sensor nodes has to be investigated in order to gain the manageable network performance monitoring/control and the maximal extension of components life cycle. Many aspects, such as routes, channel access, locating, energy efficiency, coverage, network capacity, data aggregation and Quality of Services (QoS) have been explored extensively. In this article two fuzzy logic approaches, with temporal characteristics, are proposed for monitoring and determining confidence of fire in order to optimize and reduce the number of rules that have to be checked to make the correct decisions. We assume that this reduction may lower sensor activities without relevant impact on quality of operation and extend battery life directly contributing the efficiency, robustness and cost effectiveness of sensing network. In order to get a real time verification of proposed approaches a prototype sensor web node, based on Representational State Transfer (RESTful) services, is created as an infrastructure that supports fast critical event signaling and remote access to sensor data via the Internet.

scholarly journals Serverless GEO Labels for the Semantic Sensor Web

2020 ◽  
Author(s):  
Anika Graupner ◽  
Daniel Nüst
Keyword(s):  
Real World ◽  
Large Scale ◽  
Sensor Data ◽  
Quality Information ◽  
World Semantic ◽  
Semantic Web Technologies ◽  
Web Technologies ◽  
Sensor Web ◽  
Semantic Sensor Web ◽  
Cloud Infrastructures

As the amount of sensor data made available online increases, it becomes more difficult for users to identify useful datasets. Semantic web technologies improve discovery with meaningful ontologies, but the decision of suitability remains with the users. The GEO label provides a visual summary of the standardised metadata to aid users in this process. This work presents novel rules for deriving the information for the GEO label's multiple facets, such as user feedback or quality information, based on the Semantic Sensor Network Ontology and related ontologies. It enhances an existing implementation of the GEO label API to generate labels for resources of the Semantic Sensor Web. The prototype is deployed to serverless cloud infrastructures. We find that serverless GEO label generation is capable of handling two evaluation scenarios for concurrent users and burst generation. More real-world semantic sensor descriptions and an integration into large scale discovery platforms are needed to develop the presented solutions further.

scholarly journals Perspectives on Citizen Science Data Quality

2021 ◽  
Vol 3 ◽  
Author(s):  
Robert R. Downs ◽  
Hampapuram K. Ramapriyan ◽  
Ge Peng ◽  
Yaxing Wei
Keyword(s):  
Quality Control ◽  
Data Quality ◽  
Quality Assessment ◽  
Citizen Science ◽  
Data Reuse ◽  
Quality Information ◽  
Science Data ◽  
Conducting Research

Information about data quality helps potential data users to determine whether and how data can be used and enables the analysis and interpretation of such data. Providing data quality information improves opportunities for data reuse by increasing the trustworthiness of the data. Recognizing the need for improving the quality of citizen science data, we describe quality assessment and quality control (QA/QC) issues for these data and offer perspectives on aspects of improving or ensuring citizen science data quality and for conducting research on related issues.

Sign in / Sign up

Export Citation Format

Share Document