Real time data quality control applied on an IOT sensor water quality network

2020 ◽  
Author(s):  
Matthias Maeyens ◽  
Brianna Pagán ◽  
Piet Seuntjens ◽  
Bino Maiheu ◽  
Nele Desmet ◽  
...  
Keyword(s):  
Water Quality ◽  
Quality Control ◽  
Surface Water ◽  
Data Quality ◽  
Real Time ◽  
Water Quality Monitoring ◽  
Data Quality Control ◽  
Time Data ◽  
Data Set ◽  
Real Time Data

<p>In recent years, extend periods of drought have been affecting the water quality and availability in  the Flanders region in Belgium. Especially the coastal region experienced an increased salinization of ground and surface water. The Flemish government therefore decided to invest in a dense IoT water quality monitoring network aiming to deploy 2500 water quality sensors  primarily in surface water but also in ground water and sewers. The goal of this "Internet of Water" project is to establish an operational state of the art monitoring and prediction system in support of future water policy in Flanders. </p><p>Since Flanders is a relatively small region (13,522 km²), placing this many sensors will result in one of the most dense surface water quality sensor networks in the world. Each sensor will continuously measure several indicators of water quality and transmit the data wirelessly. This allows us to continuously monitor the water quality and build a big enough data set to be able to use a more data driven approach to predicting changes  in water quality. However, as with any sensor system, the quality of the data can vary in time due to problems with the sensors, incorrect calibration or unforeseen issues. Real-time data quality control is crucial to prevent unsound decisions due to faulty data.</p><p>This contribution will give a general overview of the network and it’s specifications, but mainly focus on the implementation of the data stream as well as methods that are implemented to guarantee good data quality. More specifically the architecture and setup of a real-time data quality control system is described. Which will add quality control flags to measurements.  This system is  integrated with the NGSI API introduced by FIWARE, which forces us to make specific design decisions to acommodate to the NGSI API.</p>

scholarly journals Development and Implementation of Water Quality Assessment Monitoring (WQAM) System using the Internet of Things (IoT) in Water Environment

2021 ◽  
Vol 2 (2) ◽  
Author(s):  
Muhammad Farhan Johan ◽  
◽  
Samihah Abdullah ◽  
Nor Shahanim Mohamad Hadis ◽  
Saodah Omar ◽  
...  
Keyword(s):  
Water Quality ◽  
Internet Of Things ◽  
Quality Assessment ◽  
Real Time ◽  
Technology Implementation ◽  
Water Quality Assessment ◽  
Water Quality Monitoring ◽  
Time Data ◽  
Cloud Server ◽  
Real Time Data

Water quality monitoring (WQM) system is widely being explored as it is needed to prevent the problem of water contamination worldwide. Nowadays, there are various studies on WQM system that are being integrated with Internet of Things (IoT) concept for Wireless Sensor Network (WSN) technology implementation to get real time data measurement. Traditional ways of collecting the data are more time consuming and they lack real time changes in the quality of water. This paper presents the development and implementation of Water Quality Assessment and Monitoring (WQAM) system. The system development used WiFi enabled microcontroller to connect with the IoT environment and store the data in the IoT cloud server. The microcontroller used is Arduino UNO that interacts with three types of sensor probes which are pH, turbidity and temperature probe. All the data measurements is transferred using a WiFi module which is ESP8266. The IoT cloud used to utilize the data frame is ThingSpeak. This system was implemented on Bandar Perda Lake and Derhaka River in Pulau Pinang with two systems implemented at each location. The sensors were placed on the water surface for more accurate measurements. This system continuously measures the readings of pH, turbidity dan temperature on the lake/river for every 1 hour. Twenty readings were taken for every 1 hour within the first 20 minutes with 1 minute interval and the readings were stored in the IoT cloud server. The readings are accessible via ThingSpeak GUI. In conclusion, this system would benefit the authorities to take advantage of using the WQAM system with the aid of the IoT that is less time consuming, less cost and more reliable in real time data reading.

Fault detection in a real-time monitoring network for water quality in the lagoon of Venice (Italy)

2004 ◽  
Vol 50 (11) ◽  
pp. 51-58 ◽  
Author(s):  
S. Ciavatta ◽  
R. Pastres ◽  
Z. Lin ◽  
M.B. Beck ◽  
C. Badetti ◽  
...  
Keyword(s):  
Water Quality ◽  
Quality Control ◽  
Data Quality ◽  
Real Time ◽  
Monitoring Network ◽  
Recursive Estimation ◽  
Data Quality Control ◽  
Real Time Monitoring ◽  
Lagoon Of Venice ◽  
On Line

In the context of monitoring water quality in natural ecosystems in real time, on-line data quality control is a very important issue for effective system surveillance and for optimizing maintenance of the monitoring network. This paper presents some applications of recursive state-parameter estimation algorithms to real-time detection of signal drift in high-frequency observations. Two continuous-discrete recursive estimation schemes, namely the Extended Kalman Filter and the Recursive Prediction Error algorithm, were applied to assuring the quality of the dissolved oxygen (DO) time series, as obtained from the Lagoon of Venice (Italy) during August 2002, through the real-time monitoring network of the Magistrato alle Acque (the Venice Water Authority). Results demonstrate the effectiveness of the methodology in early detection of a probable drift in the DO signal. Comparison of these results with those obtained from the application of a related recursive scheme (a Dynamic Linear Regression procedure) suggests the strong benefits of approaching the problem of on-line data quality control with several (not merely a single) independent such estimation methods.

Students Monitoring Coastal and Inland Waters With the Basic Observation Buoy (BOB)

2012 ◽  
Vol 46 (2) ◽  
pp. 56-64 ◽  
Author(s):  
Lisa G. Adams ◽  
Douglas R. Levin ◽  
Lundie Spence
Keyword(s):  
Water Quality ◽  
New England ◽  
Real Time ◽  
Scientific Discovery ◽  
Water Quality Monitoring ◽  
Inland Waters ◽  
Time Data ◽  
Link Type ◽  
Ocean Science ◽  
Real Time Data

AbstractThe Basic Observation Buoy (BOB), a student-designed and built monitoring device, provides opportunities for scientific discovery through monitoring and data collection. BOB is a scaled-down buoy fitted with sensors that monitor aquatic and atmospheric parameters in protected coastal and inland waters. Water quality parameters, like temperature, pH, dissolved oxygen, and conductivity, are measured continuously to provide real-time data. Depending on the sensor type, information can be used for education or research purposes. The BOB program was initiated in the Chesapeake Bay region by Levin and expanded to the southeast U.S. coast in 2008, via a series of three workshops funded by the Southeast Coastal Ocean Observing Regional Association (SECOORA) and the Centers for Ocean Science Education Excellence-Southeast (COSEE SE) and led by Spence (Spence et al., 2009) and Adams (Adams et al., 2010). These workshops assisted with the introduction and establishment of new programs in New England and the Great Lakes. BOB is a tool that provides science, technology, engineering, and mathematics (STEM) exploration in real-world situations for middle school to university students. Expected benefits include engagement of students in the scientific process, water quality monitoring experiences, analysis and sharing of real-time data, and the opportunity to contribute their findings to other coastal monitoring databases. Students can upload their data to an online portal supported by SECOORA and housed at the University of North Carolina-Wilmington (<ext-link ext-link-type="uri" href="http://cormp2.das.uncw.edu/dev/">http://cormp2.das.uncw.edu/dev/</ext-link>) or the National Geographic Fieldscope™ program (<ext-link ext-link-type="uri" href="http://chesapeake.fieldscope.org/">http://chesapeake.fieldscope.org/</ext-link>).

Intelligent Data Quality Control of Real-time Rig Data

2013 ◽  
Author(s):  
Arghad Arnaout ◽  
Philipp Zoellner ◽  
Gerhard Thonhauser ◽  
Neil Johnstone
Keyword(s):  
Quality Control ◽  
Data Quality ◽  
Real Time ◽  
Data Quality Control

scholarly journals System-Aware Smart Network Management for Nano-Enriched Water Quality Monitoring

2016 ◽  
Vol 2016 ◽  
pp. 1-13 ◽  
Author(s):  
B. Mokhtar ◽  
M. Azab ◽  
N. Shehata ◽  
M. Rizk
Keyword(s):  
Water Quality ◽  
Network Management ◽  
Real Time ◽  
Learning Algorithm ◽  
Rule Learning ◽  
Water Quality Monitoring ◽  
Quality Monitoring ◽  
System Control ◽  
Test Bed ◽  
Time Data

This paper presents a comprehensive water quality monitoring system that employs a smart network management, nano-enriched sensing framework, and intelligent and efficient data analysis and forwarding protocols for smart and system-aware decision making. The presented system comprises two main subsystems, a data sensing and forwarding subsystem (DSFS), and Operation Management Subsystem (OMS). The OMS operates based on real-time learned patterns and rules of system operations projected from the DSFS to manage the entire network of sensors. The main tasks of OMS are to enable real-time data visualization, managed system control, and secure system operation. The DSFS employs a Hybrid Intelligence (HI) scheme which is proposed through integrating an association rule learning algorithm withfuzzylogic and weighted decision trees. The DSFS operation is based on profiling and registering raw data readings, generated from a set of optical nanosensors, as profiles of attribute-value pairs. As a case study, we evaluate our implemented test bed via simulation scenarios in a water quality monitoring framework. The monitoring processes are simulated based on measuring the percentage of dissolved oxygen and potential hydrogen (PH) in fresh water. Simulation results show the efficiency of the proposed HI-based methodology at learning different water quality classes.

Pycheron: A Python-Based Seismic Waveform Data Quality Control Software Package

2021 ◽  
Author(s):  
Katherine Anderson Aur ◽  
Jessica Bobeck ◽  
Anthony Alberti ◽  
Phillip Kay
Keyword(s):  
Quality Control ◽  
Data Processing ◽  
Data Quality ◽  
Monitoring Network ◽  
Downstream Processing ◽  
Data Quality Control ◽  
Data Set ◽  
Waveform Data ◽  
Seismic Waveform ◽  
Wide Range

Abstract Supplementing an existing high-quality seismic monitoring network with openly available station data could improve coverage and decrease magnitudes of completeness; however, this can present challenges when varying levels of data quality exist. Without discerning the quality of openly available data, using it poses significant data management, analysis, and interpretation issues. Incorporating additional stations without properly identifying and mitigating data quality problems can degrade overall monitoring capability. If openly available stations are to be used routinely, a robust, automated data quality assessment for a wide range of quality control (QC) issues is essential. To meet this need, we developed Pycheron, a Python-based library for QC of seismic waveform data. Pycheron was initially based on the Incorporated Research Institutions for Seismology’s Modular Utility for STAtistical kNowledge Gathering but has been expanded to include more functionality. Pycheron can be implemented at the beginning of a data processing pipeline or can process stand-alone data sets. Its objectives are to (1) identify specific QC issues; (2) automatically assess data quality and instrumentation health; (3) serve as a basic service that all data processing builds on by alerting downstream processing algorithms to any quality degradation; and (4) improve our ability to process orders of magnitudes more data through performance optimizations. This article provides an overview of Pycheron, its features, basic workflow, and an example application using a synthetic QC data set.

Sign in / Sign up

Export Citation Format

Share Document