scholarly journals Data Cleansing of Multiple Environmental Monitoring Time Series Using Spatio-Temporal Correlation

2021 ◽  
Author(s):  
Ranier A. A. Moura ◽  
Domingos B. S. Santos ◽  
Daniel G. M. Lira ◽  
José E. B. Maia

Aplicações computacionais baseadas em dados de sensores são uma realidade, mas os dados coletados e transmitidos para as aplicações raramente chegam prontos para o uso devido a perdas e ruídos de vários tipos. Neste trabalho desenvolve-se uma abordagem baseada em correlação espaço temporal para limpeza de dados de múltiplas séries temporais de sensores quanto à ruído, dados ausentes e outliers. O método foi testato em seis conjuntos de dados reais publicamente disponíveis e o seu desempenho foi comparado com um método baseline, com um autoencoder denoising e com outro método publicado. Os resultados mostram que a abordagem proposta é competitiva e requer menos dados de treinamento do que os concorrentes.

2020 ◽  
Vol 10 (5) ◽  
pp. 1876
Author(s):  
Zhongya Fan ◽  
Huiyun Feng ◽  
Jingang Jiang ◽  
Changjin Zhao ◽  
Ni Jiang ◽  
...  

Outliers are often present in large datasets of water quality monitoring time series data. A method of combining the sliding window technique with Dixon detection criterion for the automatic detection of outliers in time series data is limited by the empirical determination of sliding window sizes. The scientific determination of the optimal sliding window size is very meaningful research work. This paper presents a new Monte Carlo Search Method (MCSM) based on random sampling to optimize the size of the sliding window, which fully takes advantage of computers and statistics. The MCSM was applied in a case study to automatic monitoring data of water quality factors in order to test its validity and usefulness. The results of comparing the accuracy and efficiency of the MCSM show that the new method in this paper is scientific and effective. The experimental results show that, at different sample sizes, the average accuracy is between 58.70% and 75.75%, and the average computation time increase is between 17.09% and 45.53%. In the era of big data in environmental monitoring, the proposed new methods can meet the required accuracy of outlier detection and improve the efficiency of calculation.


2008 ◽  
Vol 4 (3) ◽  
pp. 181-192 ◽  
Author(s):  
Giovanni Sparacino ◽  
Andrea Facchinetti ◽  
Alberto Maran ◽  
Claudio Cobelli

2021 ◽  
Vol 436 ◽  
pp. 273-282
Author(s):  
Youmin Yan ◽  
Xixian Guo ◽  
Jin Tang ◽  
Chenglong Li ◽  
Xin Wang

2021 ◽  
Vol 13 (12) ◽  
pp. 2333
Author(s):  
Lilu Zhu ◽  
Xiaolu Su ◽  
Yanfeng Hu ◽  
Xianqing Tai ◽  
Kun Fu

It is extremely important to extract valuable information and achieve efficient integration of remote sensing data. The multi-source and heterogeneous nature of remote sensing data leads to the increasing complexity of these relationships, and means that the processing mode based on data ontology cannot meet requirements any more. On the other hand, the multi-dimensional features of remote sensing data bring more difficulties in data query and analysis, especially for datasets with a lot of noise. Therefore, data quality has become the bottleneck of data value discovery, and a single batch query is not enough to support the optimal combination of global data resources. In this paper, we propose a spatio-temporal local association query algorithm for remote sensing data (STLAQ). Firstly, we design a spatio-temporal data model and a bottom-up spatio-temporal correlation network. Then, we use the method of partition-based clustering and the method of spectral clustering to measure the correlation between spatio-temporal correlation networks. Finally, we construct a spatio-temporal index to provide joint query capabilities. We carry out local association query efficiency experiments to verify the feasibility of STLAQ on multi-scale datasets. The results show that the STLAQ weakens the barriers between remote sensing data, and improves their application value effectively.


Author(s):  
Carlos A. Severiano ◽  
Petrônio de Cândido de Lima e Silva ◽  
Miri Weiss Cohen ◽  
Frederico Gadelha Guimarães

2020 ◽  
Vol 72 (1) ◽  
Author(s):  
Masayuki Kano ◽  
Shin’ichi Miyazaki ◽  
Yoichi Ishikawa ◽  
Kazuro Hirahara

Abstract Postseismic Global Navigation Satellite System (GNSS) time series followed by megathrust earthquakes can be interpreted as a result of afterslip on the plate interface, especially in its early phase. Afterslip is a stress release process accumulated by adjacent coseismic slip and can be considered a recovery process for future events during earthquake cycles. Spatio-temporal evolution of afterslip often triggers subsequent earthquakes through stress perturbation. Therefore, it is important to quantitatively capture the spatio-temporal evolution of afterslip and related postseismic crustal deformation and to predict their future evolution with a physics-based simulation. We developed an adjoint data assimilation method, which directly assimilates GNSS time series into a physics-based model to optimize the frictional parameters that control the slip behavior on the fault. The developed method was validated with synthetic data. Through the optimization of frictional parameters, the spatial distributions of afterslip could roughly (but not in detail) be reproduced if the observation noise was included. The optimization of frictional parameters reproduced not only the postseismic displacements used for the assimilation, but also improved the prediction skill of the following time series. Then, we applied the developed method to the observed GNSS time series for the first 15 days following the 2003 Tokachi-oki earthquake. The frictional parameters in the afterslip regions were optimized to A–B ~ O(10 kPa), A ~ O(100 kPa), and L ~ O(10 mm). A large afterslip is inferred on the shallower side of the coseismic slip area. The optimized frictional parameters quantitatively predicted the postseismic GNSS time series for the following 15 days. These characteristics can also be detected if the simulation variables can be simultaneously optimized. The developed data assimilation method, which can be directly applied to GNSS time series following megathrust earthquakes, is an effective quantitative evaluation method for assessing risks of subsequent earthquakes and for monitoring the recovery process of megathrust earthquakes.


Sign in / Sign up

Export Citation Format

Share Document