A Review of Missing Sensor Data Imputation Methods

2019 ◽  
Author(s):  
Rahmat Nur Faizin ◽  
Mardhani Riasetiawan ◽  
Ahmad Ashari
Keyword(s):  
Sensor Data ◽  
Data Imputation ◽  
Imputation Methods

scholarly journals Temporal and Spatial Nearest Neighbor Values Based Missing Data Imputation in Wireless Sensor Networks

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1782
Author(s):  
Yulong Deng ◽  
Chong Han ◽  
Jian Guo ◽  
Lijuan Sun
Keyword(s):  
Wireless Sensor Networks ◽  
Sensor Networks ◽  
Missing Data ◽  
Nearest Neighbor ◽  
Sensor Data ◽  
Wireless Sensor ◽  
Data Imputation ◽  
Missing Data Imputation ◽  
Temporal And Spatial ◽  
Two Parameters

Data missing is a common problem in wireless sensor networks. Currently, to ensure the performance of data processing, making imputation for the missing data is the most common method before getting into sensor data analysis. In this paper, the temporal and spatial nearest neighbor values-based missing data imputation (TSNN), a new imputation based on the temporal and spatial nearest neighbor values has been presented. First, four nearest neighbor values have been defined from the perspective of space and time dimensions as well as the geometrical and data distances, which are the bases of the algorithm that help to exploit the correlations among sensor data on the nodes with the regression tool. Next, the algorithm has been elaborated as well as two parameters, the best number of neighbors and spatial–temporal coefficient. Finally, the algorithm has been tested on an indoor and an outdoor wireless sensor network, and the result shows that TSNN is able to improve the accuracy of imputation and increase the number of cases that can be imputed effectively.

scholarly journals A Pattern-Recognition-Based Ensemble Data Imputation Framework for Sensors from Building Energy Systems

Sensors ◽  
2020 ◽  
Vol 20 (20) ◽  
pp. 5947
Author(s):  
Liang Zhang
Keyword(s):  
Pattern Recognition ◽  
Missing Data ◽  
Energy Systems ◽  
Building Energy ◽  
Imputation Method ◽  
Validation Dataset ◽  
Data Imputation ◽  
Validation Data ◽  
Imputation Methods ◽  
Building Energy Systems

Building operation data are important for monitoring, analysis, modeling, and control of building energy systems. However, missing data is one of the major data quality issues, making data imputation techniques become increasingly important. There are two key research gaps for missing sensor data imputation in buildings: the lack of customized and automated imputation methodology, and the difficulty of the validation of data imputation methods. In this paper, a framework is developed to address these two gaps. First, a validation data generation module is developed based on pattern recognition to create a validation dataset to quantify the performance of data imputation methods. Second, a pool of data imputation methods is tested under the validation dataset to find an optimal single imputation method for each sensor, which is termed as an ensemble method. The method can reflect the specific mechanism and randomness of missing data from each sensor. The effectiveness of the framework is demonstrated by 18 sensors from a real campus building. The overall accuracy of data imputation for those sensors improves by 18.2% on average compared with the best single data imputation method.

scholarly journals Multiple imputation of maritime search and rescue data at multiple missing patterns

PLoS ONE ◽  
2021 ◽  
Vol 16 (6) ◽  
pp. e0252129
Author(s):  
Guobo Wang ◽  
Minglu Ma ◽  
Lili Jiang ◽  
Fengyun Chen ◽  
Liansheng Xu
Keyword(s):  
Multiple Imputation ◽  
Expectation Maximization ◽  
Data Augmentation ◽  
Search And Rescue ◽  
Data Sets ◽  
Data Imputation ◽  
Imputation Methods ◽  
High Data ◽  
Data Missing ◽  
Maritime Search And Rescue

Based on the missing situation and actual needs of maritime search and rescue data, multiple imputation methods were used to construct complete data sets under different missing patterns. Probability density curves and overimputation diagnostics were used to explore the effects of multiple imputation. The results showed that the Data Augmentation (DA) algorithm had the characteristics of high operation efficiency and good imputation effect, but the algorithm was not suitable for data imputation when there was a high data missing rate. The EMB algorithm effectively restored the distribution of datasets with different data missing rates, and was less affected by the missing position; the EMB algorithm could obtain a good imputation effect even when there was a high data missing rate. Overimputation diagnostics could not only reflect the data imputation effect, but also show the correlation between different datasets, which was of great importance for deep data mining and imputation effect improvement. The Expectation-Maximization with Bootstrap (EMB) algorithm had a poor estimation effect on extreme data and failed to reflect the dataset’s variability characteristics.

scholarly journals Missing Data Imputation in Internet of Things Gateways

Information ◽  
2021 ◽  
Vol 12 (10) ◽  
pp. 425
Author(s):  
Cinthya M. França ◽  
Rodrigo S. Couto ◽  
Pedro B. Velloso
Keyword(s):  
Neural Network ◽  
Missing Data ◽  
Internet Of Things ◽  
Execution Time ◽  
Regression Models ◽  
Weather Data ◽  
Data Imputation ◽  
Missing Data Imputation ◽  
Imputation Methods ◽  
The Neural Network

In an Internet of Things (IoT) environment, sensors collect and send data to application servers through IoT gateways. However, these data may be missing values due to networking problems or sensor malfunction, which reduces applications’ reliability. This work proposes a mechanism to predict and impute missing data in IoT gateways to achieve greater autonomy at the network edge. These gateways typically have limited computing resources. Therefore, the missing data imputation methods must be simple and provide good results. Thus, this work presents two regression models based on neural networks to impute missing data in IoT gateways. In addition to the prediction quality, we analyzed both the execution time and the amount of memory used. We validated our models using six years of weather data from Rio de Janeiro, varying the missing data percentages. The results show that the neural network regression models perform better than the other imputation methods analyzed, based on the averages and repetition of previous values, for all missing data percentages. In addition, the neural network models present a short execution time and need less than 140 KiB of memory, which allows them to run on IoT gateways.

Sign in / Sign up

Export Citation Format

Share Document