Effective aggregation of gappy replicated time series using INLA

2020 ◽  
Author(s):  
Thomas Wutzler ◽  
Mirco Migliavacca ◽  
Kendalynn Morris
Keyword(s):  
Time Series ◽  
Net Ecosystem Exchange ◽  
Data Cube ◽  
Correlation Structure ◽  
Co2 Efflux ◽  
Learning Approaches ◽  
Gap Filling ◽  
Marginal Posterior Distribution ◽  
Fully Bayesian ◽  
Replicated Measurement

<p>Soil CO2 efflux data from automated chambers provide an important constraint for ecosystem and soil respiration. Usually, half-hourly time series of several replicated chambers have to be aggregated to plot-level while gaps in the time series have to be accommodated. Gaps cause jumps and other problems in aggregation of replicated measurement in each half-hour, therefore, lookup tables and machine learning approaches are used to fill gaps beforehand.</p><p>Here, we present an alternative fully Bayesian approach for the combined gap-filling and aggregation based on Integrated Nested Laplace Approximation (INLA). This method integrates all information from every measurement across replicates and across time and therefore efficiently estimates the correlation structure among all observations. It provides the full marginal posterior distribution of the aggregated time series at the plot level across the time span of the time series. We compare several aggregation approaches using four years of data from 16 automatic chambers at the eddy-covariance site in Majadas de Tietar in Spain (ES-LM1, ES-LMa).</p><p>This approach is applicable for other replicated time series as well. We further explore its usage for analysing time-varying effects across treatments and habitats and its usage for gap-filling net ecosystem exchange (NEE) data based on the full correlation structure in a data-cube of time and environmental conditions.</p>

Spectroscopy-Based Mapping with Scanning Microwave Impedance Microscopy

2018 ◽  
Author(s):  
Peter De Wolf ◽  
Zhuangqun Huang ◽  
Bede Pittenger
Keyword(s):  
Single Point ◽  
Electrical Characterization ◽  
Principal Component ◽  
High Sensitivity ◽  
Data Cube ◽  
Nanometer Scale ◽  
Learning Approaches ◽  
Data Set ◽  
3D Data ◽  
Higher Dimensional

Abstract Methods are available to measure conductivity, charge, surface potential, carrier density, piezo-electric and other electrical properties with nanometer scale resolution. One of these methods, scanning microwave impedance microscopy (sMIM), has gained interest due to its capability to measure the full impedance (capacitance and resistive part) with high sensitivity and high spatial resolution. This paper introduces a novel data-cube approach that combines sMIM imaging and sMIM point spectroscopy, producing an integrated and complete 3D data set. This approach replaces the subjective approach of guessing locations of interest (for single point spectroscopy) with a big data approach resulting in higher dimensional data that can be sliced along any axis or plane and is conducive to principal component analysis or other machine learning approaches to data reduction. The data-cube approach is also applicable to other AFM-based electrical characterization modes.

scholarly journals A comparison of methods for smoothing and gap filling time series of remote sensing observations – application to MODIS LAI products

2013 ◽  
Vol 10 (6) ◽  
pp. 4055-4071 ◽  
Author(s):  
S. Kandasamy ◽  
F. Baret ◽  
A. Verger ◽  
P. Neveux ◽  
M. Weiss
Keyword(s):  
Time Series ◽  
Missing Data ◽  
Time Course ◽  
Singular Spectrum Analysis ◽  
Gaussian Function ◽  
Gap Filling ◽  
Area Index ◽  
Filling Time ◽  
Moderate Resolution ◽  
Temporal Profiles

Abstract. Moderate resolution satellite sensors including MODIS (Moderate Resolution Imaging Spectroradiometer) already provide more than 10 yr of observations well suited to describe and understand the dynamics of earth's surface. However, these time series are associated with significant uncertainties and incomplete because of cloud cover. This study compares eight methods designed to improve the continuity by filling gaps and consistency by smoothing the time course. It includes methods exploiting the time series as a whole (iterative caterpillar singular spectrum analysis (ICSSA), empirical mode decomposition (EMD), low pass filtering (LPF) and Whittaker smoother (Whit)) as well as methods working on limited temporal windows of a few weeks to few months (adaptive Savitzky–Golay filter (SGF), temporal smoothing and gap filling (TSGF), and asymmetric Gaussian function (AGF)), in addition to the simple climatological LAI yearly profile (Clim). Methods were applied to the MODIS leaf area index product for the period 2000–2008 and over 25 sites showed a large range of seasonal patterns. Performances were discussed with emphasis on the balance achieved by each method between accuracy and roughness depending on the fraction of missing observations and the length of the gaps. Results demonstrate that the EMD, LPF and AGF methods were failing because of a significant fraction of gaps (more than 20%), while ICSSA, Whit and SGF were always providing estimates for dates with missing data. TSGF (Clim) was able to fill more than 50% of the gaps for sites with more than 60% (80%) fraction of gaps. However, investigation of the accuracy of the reconstructed values shows that it degrades rapidly for sites with more than 20% missing data, particularly for ICSSA, Whit and SGF. In these conditions, TSGF provides the best performances that are significantly better than the simple Clim for gaps shorter than about 100 days. The roughness of the reconstructed temporal profiles shows large differences between the various methods, with a decrease of the roughness with the fraction of missing data, except for ICSSA. TSGF provides the smoothest temporal profiles for sites with a % gap > 30%. Conversely, ICSSA, LPF, Whit, AGF and Clim provide smoother profiles than TSGF for sites with a % gap < 30%. Impact of the accuracy and smoothness of the reconstructed time series were evaluated on the timing of phenological stages. The dates of start, maximum and end of the season are estimated with an accuracy of about 10 days for the sites with a % gap < 10% and increases rapidly with the % gap. TSGF provides more accurate estimates of phenological timing up to a % gap < 60%.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 1.91% to 6.69%. <div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Comparison of Machine Learning Approaches to Improve Diagnosis of Optic Neuropathy Using Photopic Negative Response Measured Using a Handheld Device

2021 ◽  
Vol 8 ◽  
Author(s):  
Tina Diao ◽  
Fareshta Kushzad ◽  
Megh D. Patel ◽  
Megha P. Bindiganavale ◽  
Munam Wasi ◽  
...  
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Optic Neuropathy ◽  
Negative Response ◽  
Photopic Negative Response ◽  
Learning Approaches ◽  
Data Set ◽  
Full Field ◽  
Handheld Device ◽  
Technical Requirements

The photopic negative response of the full-field electroretinogram (ERG) is reduced in optic neuropathies. However, technical requirements for measurement and poor classification performance have limited widespread clinical application. Recent advances in hardware facilitate efficient clinic-based recording of the full-field ERG. Time series classification, a machine learning approach, may improve classification by using the entire ERG waveform as the input. In this study, full-field ERGs were recorded in 217 eyes (109 optic neuropathy and 108 controls) of 155 subjects. User-defined ERG features including photopic negative response were reduced in optic neuropathy eyes (p &lt; 0.0005, generalized estimating equation models accounting for age). However, classification of optic neuropathy based on user-defined features was only fair with receiver operating characteristic area under the curve ranging between 0.62 and 0.68 and F1 score at the optimal cutoff ranging between 0.30 and 0.33. In comparison, machine learning classifiers using a variety of time series analysis approaches had F1 scores of 0.58–0.76 on a test data set. Time series classifications are promising for improving optic neuropathy diagnosis using ERG waveforms. Larger sample sizes will be important to refine the models.

Pre-treating GNSS time series using a recurrent neural network to improve the automated detection of jump discontinuities

2021 ◽  
Author(s):  
Luca Tavasci ◽  
Pasquale Cascarano ◽  
Stefano Gandolfi
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Time Series Analysis ◽  
Ad Hoc ◽  
Short Term Memory ◽  
Learning Approaches ◽  
Jump Detection ◽  
Series Analysis ◽  
Jump Discontinuities ◽  
Gnss Time Series

&lt;p&gt;Ground motion monitoring is one of the main goals in the geoscientist community and at the time it is mainly performed by analyzing time series of data. Our capability of describing the most significant features characterizing the time evolution of a point-position is affected by the presence of undetected discontinuities in the time series. One of the most critical aspects in the automated time series analysis, which is quite necessary since the amount of data is increasing more and more, is still the detection of discontinuities and in particular the definition of their epoch. A number of algorithms have already been developed and proposed to the community in the last years, following different statistical approaches and different hypotheses on the coordinates behavior. In this work, we have chosen to analyze GNSS time series and to use an already published algorithm (STARS) for jump detection as a benchmark to test our approach, consisting of pre-treating the time series to be analyzed using a neural network. In particular, we chose a Long Short Term Memory (LSTM) neural network belonging to the class of the Recurrent Neural Networks (RNNs), ad hoc modified for the GNSS time series analysis. We focused both on the training algorithm and the testing one. The latter has been the object of a parametric test to find out the number of predicted data that mostly emphasize our capability of detecting jump discontinuities. Results will be presented considering several GNSS time series of daily positions. Finally, a discussion on the possible integration of machine learning approaches and classical deterministic approaches will be done.&lt;/p&gt;

scholarly journals Evaluating four gap-filling methods for eddy covariance measurements of evapotranspiration over hilly crop fields

2017 ◽  
Author(s):  
Nissaf Boudhina ◽  
Rim Zitouna-Chebbi ◽  
Insaf Mekki ◽  
Frédéric Jacob ◽  
Nétij Ben Mechlia ◽  
...  
Keyword(s):  
Time Series ◽  
Linear Regression ◽  
Eddy Covariance ◽  
Water Status ◽  
Growth Cycle ◽  
Filling Rate ◽  
Gap Filling ◽  
Crop Fields ◽  
Downslope Winds ◽  
Aerodynamic Properties

Abstract. Estimating evapotranspiration in hilly watersheds is paramount for managing water resources, especially in semi-arid regions. Eddy covariance (EC) technique allows continuous measurements of latent heat flux LE. However, time series of EC measurements often experience large portions of missing data, because of instrumental dysfunctions or quality filtering. Existing gap-filling methods are questionable over hilly crop fields, because of changes in airflow inclination and subsequent aerodynamic properties. We evaluated the performances of different gap-filling methods before and after tailoring to conditions of hilly crop fields. The tailoring consisted of beforehand splitting the LE time series on the basis of upslope and downslope winds. The experiment was setup within an agricultural hilly watershed in northeastern Tunisia. EC measurements were collected throughout the growth cycle of three wheat crops, two of them located in adjacent fields on opposite hillslopes, and the third one located in a flat field. We considered four gap-filling methods: the REddyProc method, the linear regression between LE and net radiation Rn, the multi-linear regression of LE against the other energy fluxes, and the use of evaporative fraction EF. Regardless of method, the splitting of the LE time series did not impact the gap filling rate, and it might improve the accuracies on LE retrievals in some cases. Regardless of method, the obtained accuracies on LE estimates after gap filling were close to instrumental accuracies, and were comparable to those reported in previous studies over flat and mountainous terrains. Overall, REddyProc was the most appropriate method, for both gap filling rate and retrieval accuracy. Thus, it seems possible to conduct gap-filling for LE time series collected over hilly crop fields, provided the LE time series are beforehand split on the basis of upslope / downslope winds. Future works should address consecutive vegetation growth cycles for a larger panel of conditions in terms of climate, vegetation and water status.

Sign in / Sign up

Export Citation Format

Share Document