scholarly journals Smartphone Pressure Collection and Bias Correction Using Machine Learning

2018 ◽  
Vol 35 (3) ◽  
pp. 523-540 ◽  
Author(s):  
Conor McNicholas ◽  
Clifford F. Mass
Keyword(s):  
Machine Learning ◽  
Quality Control ◽  
Bias Correction ◽  
Control Strategies ◽  
Weather Prediction ◽  
Sensor Data ◽  
Observation Error ◽  
Test Bed ◽  
Machine Learning Approach ◽  
Complex Relationships

AbstractOver half a billion smartphones worldwide are now capable of measuring atmospheric pressure, providing a pressure network of unprecedented density and coverage. This paper describes novel approaches for the collection, quality control, and bias correction of such smartphone pressures. An Android app was developed and distributed to several thousand users, serving as a test bed for onboard pressure collection and quality-control strategies. New methods of pressure collection were evaluated, with a focus on reducing and quantifying sources of observation error and uncertainty. Using a machine learning approach, complex relationships between pressure bias and ancillary sensor data were used to predict and correct future pressure biases over a 4-week period from 10 November to 5 December 2016. This approach, in combination with simple quality-control checks, produced an 82% reduction in the average smartphone pressure bias, substantially improving the quality of smartphone pressures and facilitating their use in numerical weather prediction.

P 120 A machine learning approach to detecting sleep and sleep disorders in acceleration sensor data

2017 ◽  
Vol 128 (10) ◽  
pp. e388
Author(s):  
R. Leenings ◽  
C. Glatz ◽  
A. Heidbreder ◽  
M. Boentert ◽  
G. Pipa ◽  
...  
Keyword(s):  
Machine Learning ◽  
Sleep Disorders ◽  
Sensor Data ◽  
Learning Approach ◽  
Acceleration Sensor ◽  
Machine Learning Approach

scholarly journals Outlier detection in WSN by entropy based machine learning approach

2020 ◽  
Vol 20 (3) ◽  
pp. 1435
Author(s):  
Manmohan Singh Yadav ◽  
Shish Ahamad
Keyword(s):  
Machine Learning ◽  
Outlier Detection ◽  
Sensor Data ◽  
Learning Approach ◽  
Random Subspace ◽  
Environmental Disasters ◽  
Machine Learning Approach ◽  
The World ◽  
Prediction Capability

<p>Environmental disasters like flooding, earthquake etc. causes catastrophic effects all over the world. WSN based techniques have become popular in susceptibility modelling of such disaster due to their greater strength and efficiency in the prediction of such threats. This paper demonstrates the machine learning-based approach to predict outlier in sensor data with bagging, boosting, random subspace, SVM and KNN based frameworks for outlier prediction using a WSN data. First of all database is pre processed with 14 sensor motes with presence of outlier due to intrusion. Subsequently segmented database is created from sensor pairs. Finally, the data entropy is calculated and used as a feature to determine the presence of outlier used different approach. Results show that the KNN model has the highest prediction capability for outlier assessment.</p>

scholarly journals Influence of synchronization within a sensor network on machine learning results

2021 ◽  
Vol 10 (2) ◽  
pp. 233-245
Author(s):  
Tanja Dorst ◽  
Yannick Robin ◽  
Sascha Eichstädt ◽  
Andreas Schütze ◽  
Tizian Schneider
Keyword(s):  
Machine Learning ◽  
Feature Extraction ◽  
Data Augmentation ◽  
Time Synchronization ◽  
Fundamental Problem ◽  
Training Data ◽  
Sensor Data ◽  
Test Bed ◽  
Data Set ◽  
Linear Discriminant

Abstract. Process sensor data allow for not only the control of industrial processes but also an assessment of plant conditions to detect fault conditions and wear by using sensor fusion and machine learning (ML). A fundamental problem is the data quality, which is limited, inter alia, by time synchronization problems. To examine the influence of time synchronization within a distributed sensor system on the prediction performance, a test bed for end-of-line tests, lifetime prediction, and condition monitoring of electromechanical cylinders is considered. The test bed drives the cylinder in a periodic cycle at maximum load, a 1 s period at constant drive speed is used to predict the remaining useful lifetime (RUL). The various sensors for vibration, force, etc. integrated into the test bed are sampled at rates between 10 kHz and 1 MHz. The sensor data are used to train a classification ML model to predict the RUL with a resolution of 1 % based on feature extraction, feature selection, and linear discriminant analysis (LDA) projection. In this contribution, artificial time shifts of up to 50 ms between individual sensors' cycles are introduced, and their influence on the performance of the RUL prediction is investigated. While the ML model achieves good results if no time shifts are introduced, we observed that applying the model trained with unmodified data only to data sets with time shifts results in very poor performance of the RUL prediction even for small time shifts of 0.1 ms. To achieve an acceptable performance also for time-shifted data and thus achieve a more robust model for application, different approaches were investigated. One approach is based on a modified feature extraction approach excluding the phase values after Fourier transformation; a second is based on extending the training data set by including artificially time-shifted data. This latter approach is thus similar to data augmentation used to improve training of neural networks.

scholarly journals All-Sky Microwave Radiance Assimilation in NCEP’s GSI Analysis System

2016 ◽  
Vol 144 (12) ◽  
pp. 4709-4735 ◽  
Author(s):  
Yanqiu Zhu ◽  
Emily Liu ◽  
Rahul Mahajan ◽  
Catherine Thomas ◽  
David Groff ◽  
...  
Keyword(s):  
Quality Control ◽  
Bias Correction ◽  
Microwave Radiometer ◽  
Cloud Water ◽  
Development Effort ◽  
Observation Error ◽  
Background Error ◽  
Radiance Assimilation ◽  
Analysis System ◽  
Sky Conditions

Abstract The capability of all-sky microwave radiance assimilation in the Gridpoint Statistical Interpolation (GSI) analysis system has been developed at the National Centers for Environmental Prediction (NCEP). This development effort required the adaptation of quality control, observation error assignment, bias correction, and background error covariance to all-sky conditions within the ensemble–variational (EnVar) framework. The assimilation of cloudy radiances from the Advanced Microwave Sounding Unit-A (AMSU-A) microwave radiometer for ocean fields of view (FOVs) is the primary emphasis of this study. In the original operational hybrid 3D EnVar Global Forecast System (GFS), the clear-sky approach for radiance data assimilation is applied. Changes to data thinning and quality control have allowed all-sky satellite radiances to be assimilated in the GSI. Along with the symmetric observation error assignment, additional situation-dependent observation error inflation is employed for all-sky conditions. Moreover, in addition to the current radiance bias correction, a new bias correction strategy has been applied to all-sky radiances. In this work, the static background error variance and the ensemble spread of cloud water are examined, and the levels of cloud variability from the ensemble forecast in single- and dual-resolution configurations are discussed. Overall, the all-sky approach provides more realistic simulated brightness temperatures and cloud water analysis increments, and improves analysis off the west coasts of the continents by reducing a known bias in stratus. An approximate 10% increase in the use of AMSU-A channels 1–5 and a 12% increase for channel 15 are also observed. The all-sky AMSU-A radiance assimilation became operational in the 4D EnVar GFS system upgrade of 12 May 2016.

A machine learning approach to the observation operator for satellite radiance data assimilation

2021 ◽  
Author(s):  
Jianyu Liang ◽  
Koji Terasaki ◽  
Takemasa Miyoshi
Keyword(s):  
Machine Learning ◽  
Data Assimilation ◽  
Bias Correction ◽  
Reference System ◽  
Weather Prediction ◽  
Learning Model ◽  
Correction Procedure ◽  
Observation Operator ◽  
Machine Learning Model ◽  
Physically Based

&lt;p&gt;The &amp;#8216;observation operator&amp;#8217; is essential in data assimilation (DA) to derive the model equivalent of the observations from the model variables. For satellite radiance observations, it is usually based on complex radiative transfer model (RTM) with a bias correction procedure. Therefore, it usually takes time to start using new satellite data after launching the satellites. Here we take advantage of the recent fast development of machine learning (ML) which is good at finding the complex relationships within data. ML can potentially be used as the &amp;#8216;observation operator&amp;#8217; to reveal the relationships between the model variables and the observations without knowing their physical relationships. In this study, we test with the numerical weather prediction system composed of the Nonhydrostatic Icosahedral Atmospheric Model (NICAM) and the Local Ensemble Transform Kalman Filter (LETKF). We focus on the satellite microwave brightness temperature (BT) from the Advanced Microwave Sounding Unit-A (AMSU-A). Conventional observations and AMSU-A data were assimilated every 6 hours. The reference DA system employed the observation operator based on the RTTOV and an online bias correction method.&lt;/p&gt;&lt;p&gt;We used this reference system to generate 1-month data to train the machine learning model. Since the reference system includes running a physically-based RTM, we implicitly used the information from RTM for training the ML model in this study, although in our future research we will explore methods without the use of RTM. The machine learning model is artificial neural networks with 5 fully connected layers. The input of the ML model includes the NICAM model variables and predictors for bias correction, and the output of the ML model is the corresponding satellite BT in 3 channels from 5 satellites. Next, we ran the DA cycle for the same month the following year to test the performance of the ML model. Two experiments were conducted. The control experiment (CTRL) was performed with the reference system. In the test experiment (TEST), the ML model was used as the observation operator and there is no separate bias correction procedure since the training includes biased differences between the model and observation. The results showed no significant bias of the simulated BT by the ML model. Using the ECMWF global atmospheric reanalysis (ERA-interim) as a benchmark to evaluate the analysis accuracy, the global-mean RMSE, bias, and ensemble spread for temperature in TEST are 2% higher, 4% higher, and 1% lower respectively than those in CTRL. The result is encouraging since our ML can emulate the RTM. The limitation of our study is that we rely on the physically-based RTM in the reference DA system, which is used for training the ML model. This is the first result and still preliminary. We are currently considering other methods to train the ML model without using the RTM at all.&lt;/p&gt;

On the potential and challenges of using machine-learning for automated quality control of environmental sensor data

2020 ◽  
Author(s):  
Lennart Schmidt ◽  
Hannes Mollenhauer ◽  
Corinna Rebmann ◽  
David Schäfer ◽  
Antje Claussnitzer ◽  
...  
Keyword(s):  
Machine Learning ◽  
Quality Control ◽  
Ground Truth ◽  
Sensor Data ◽  
Small Scale ◽  
Ground Truth Data ◽  
Starting Point ◽  
Environmental Sensor ◽  
Spatio Temporal ◽  
Automated Quality Control

&lt;p&gt;With more and more data being gathered from environmental sensor networks, the importance of automated quality-control (QC) routines to provide usable data in near-real time is becoming increasingly apparent. Machine-learning (ML) algorithms exhibit a high potential to this respect as they are able to exploit the spatio-temporal relation of multiple sensors to identify anomalies while allowing for non-linear functional relations in the data. In this study, we evaluate the potential of ML for automated QC on two spatio-temporal datasets at different spatial scales: One is a dataset of atmospheric variables at 53 stations across Northern Germany. The second dataset contains timeseries of soil moisture and temperature at 40 sensors at a small-scale measurement plot.&lt;/p&gt;&lt;p&gt;Furthermore, we investigate strategies to tackle three challenges that are commonly present when applying ML for QC: 1) As sensors might drop out, the ML models have to be designed to be robust against missing values in the input data. We address this by comparing different data imputation methods, coupled with a binary representation of whether a value is missing or not. 2) Quality flags that mark erroneous data points to serve as ground truth for model training might not be available. And 3) There is no guarantee that the system under study is stationary, which might render the outputs of a trained model useless in the future. To address 2) and 3), we frame the problem both as a supervised and unsupervised learning problem. Here, the use of unsupervised ML-models can be beneficial as they do not require ground truth data and can thus be retrained more easily should the system be subject to significant changes. In this presentation, we discuss the performance, advantages and drawbacks of the proposed strategies to tackle the aforementioned challenges. Thus, we provide a starting point for researchers in the largely untouched field of ML application for automated quality control of environmental sensor data.&lt;/p&gt;

scholarly journals Wearable Sensor Data to Track Subject-Specific Movement Patterns Related to Clinical Outcomes Using a Machine Learning Approach

Sensors ◽  
2018 ◽  
Vol 18 (9) ◽  
pp. 2828 ◽  
Author(s):  
Dylan Kobsar ◽  
Reed Ferber
Keyword(s):  
Machine Learning ◽  
Clinical Outcomes ◽  
Movement Patterns ◽  
Movement Pattern ◽  
Human Movement ◽  
Sensor Data ◽  
Learning Approach ◽  
Wearable Sensor ◽  
Machine Learning Approach ◽  
Subject Specific

Wearable sensors can provide detailed information on human movement but the clinical impact of this information remains limited. We propose a machine learning approach, using wearable sensor data, to identify subject-specific changes in gait patterns related to improvements in clinical outcomes. Eight patients with knee osteoarthritis (OA) completed two gait trials before and one following an exercise intervention. Wearable sensor data (e.g., 3-dimensional (3D) linear accelerations) were collected from a sensor located near the lower back, lateral thigh and lateral shank during level treadmill walking at a preferred speed. Wearable sensor data from the 2 pre-intervention gait trials were used to define each individual’s typical movement pattern using a one-class support vector machine (OCSVM). The percentage of strides defined as outliers, based on the pre-intervention gait data and the OCSVM, were used to define the overall change in an individual’s movement pattern. The correlation between the change in movement patterns following the intervention (i.e., percentage of outliers) and improvement in self-reported clinical outcomes (e.g., pain and function) was assessed using a Spearman rank correlation. The number of outliers observed post-intervention exhibited a large association (ρ = 0.78) with improvements in self-reported clinical outcomes. These findings demonstrate a proof-of-concept and a novel methodological approach for integrating machine learning and wearable sensor data. This approach provides an objective and evidence-informed way to understand clinically important changes in human movement patterns in response to exercise therapy.

Sign in / Sign up

Export Citation Format

Share Document