Data-model comparison of temporal variability in long-term time series of large-scale soil moisture

AbstractSoil moisture is an important variable for land-climate and hydrological interactions. To investigate emergent large-scale, long-term interactions between soil moisture and other key hydro-climatic variables (precipitation, actual evapotranspiration, runoff, temperature), we analyze monthly values and anomalies of these variables in 1378 hydrological catchments across Europe over the period 1980–2010. The study distinguishes results for the main European climate regions, and tests how sensitive or robust they are to the use of three alternative observational and re-analysis datasets. Robustly across the European climates and datasets, monthly soil moisture anomalies correlate well with runoff anomalies, and extreme soil moisture and runoff values also largely co-occur. For precipitation, evapotranspiration, and temperature, anomaly correlation and extreme value co-occurrence with soil moisture are overall lower than for runoff. The runoff results indicate a possible new approach to assessing variability and change of large-scale soil moisture conditions by use of long-term time series of monitored catchment-integrating stream discharges.

Download Full-text

Global soil moisture data derived through machine learning trained with in-situ measurements

Scientific Data ◽

10.1038/s41597-021-00964-1 ◽

2021 ◽

Vol 8 (1) ◽

Author(s):

Sungmin O. ◽

Rene Orth

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

Large Scale ◽

Short Term Memory ◽

Temporal Dynamics ◽

Soil Moisture Data ◽

Wide Range ◽

Global Soil

AbstractWhile soil moisture information is essential for a wide range of hydrologic and climate applications, spatially-continuous soil moisture data is only available from satellite observations or model simulations. Here we present a global, long-term dataset of soil moisture derived through machine learning trained with in-situ measurements, SoMo.ml. We train a Long Short-Term Memory (LSTM) model to extrapolate daily soil moisture dynamics in space and in time, based on in-situ data collected from more than 1,000 stations across the globe. SoMo.ml provides multi-layer soil moisture data (0–10 cm, 10–30 cm, and 30–50 cm) at 0.25° spatial and daily temporal resolution over the period 2000–2019. The performance of the resulting dataset is evaluated through cross validation and inter-comparison with existing soil moisture datasets. SoMo.ml performs especially well in terms of temporal dynamics, making it particularly useful for applications requiring time-varying soil moisture, such as anomaly detection and memory analyses. SoMo.ml complements the existing suite of modelled and satellite-based datasets given its distinct derivation, to support large-scale hydrological, meteorological, and ecological analyses.

Download Full-text

Evaluating Machine Learning and Geostatistical Methods for Spatial Gap-filling of Monthly ESA CCI Soil Moisture in China

Remote Sensing ◽

10.3390/rs13142848 ◽

2021 ◽

Vol 13 (14) ◽

pp. 2848

Author(s):

Hao Sun ◽

Qian Xu

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

Water Resource Management ◽

Large Scale ◽

Gap Filling ◽

Spatial Continuity ◽

Study Results ◽

Data Gaps

Obtaining large-scale, long-term, and spatial continuous soil moisture (SM) data is crucial for climate change, hydrology, and water resource management, etc. ESA CCI SM is such a large-scale and long-term SM (longer than 40 years until now). However, there exist data gaps, especially for the area of China, due to the limitations in remote sensing of SM such as complex topography, human-induced radio frequency interference (RFI), and vegetation disturbances, etc. The data gaps make the CCI SM data cannot achieve spatial continuity, which entails the study of gap-filling methods. In order to develop suitable methods to fill the gaps of CCI SM in the whole area of China, we compared typical Machine Learning (ML) methods, including Random Forest method (RF), Feedforward Neural Network method (FNN), and Generalized Linear Model (GLM) with a geostatistical method, i.e., Ordinary Kriging (OK) in this study. More than 30 years of passive–active combined CCI SM from 1982 to 2018 and other biophysical variables such as Normalized Difference Vegetation Index (NDVI), precipitation, air temperature, Digital Elevation Model (DEM), soil type, and in situ SM from International Soil Moisture Network (ISMN) were utilized in this study. Results indicated that: 1) the data gap of CCI SM is frequent in China, which is found not only in cold seasons and areas but also in warm seasons and areas. The ratio of gap pixel numbers to the whole pixel numbers can be greater than 80%, and its average is around 40%. 2) ML methods can fill the gaps of CCI SM all up. Among the ML methods, RF had the best performance in fitting the relationship between CCI SM and biophysical variables. 3) Over simulated gap areas, RF had a comparable performance with OK, and they outperformed the FNN and GLM methods greatly. 4) Over in situ SM networks, RF achieved better performance than the OK method. 5) We also explored various strategies for gap-filling CCI SM. Results demonstrated that the strategy of constructing a monthly model with one RF for simulating monthly average SM and another RF for simulating monthly SM disturbance achieved the best performance. Such strategy combining with the ML method such as the RF is suggested in this study for filling the gaps of CCI SM in China.

Download Full-text

Machine-Learning for the Prediction of Lost Circulation Events - Time Series Analysis and Model Evaluation

10.2118/204706-ms ◽

2021 ◽

Author(s):

Arturo Magana-Mora ◽

Mohammad AlJubran ◽

Jothibasu Ramasamy ◽

Mohammed AlBassam ◽

Chinthaka Gooneratne ◽

...

Keyword(s):

Machine Learning ◽

Time Series ◽

Real Time ◽

Large Scale ◽

Model Comparison ◽

Drilling Fluid ◽

Sensor Data ◽

False Alarms ◽

Suitable Model ◽

Lost Circulation

Abstract Objective/Scope. Lost circulation events (LCEs) are among the top causes for drilling nonproductive time (NPT). The presence of natural fractures and vugular formations causes loss of drilling fluid circulation. Drilling depleted zones with incorrect mud weights can also lead to drilling induced losses. LCEs can also develop into additional drilling hazards, such as stuck pipe incidents, kicks, and blowouts. An LCE is traditionally diagnosed only when there is a reduction in mud volume in mud pits in the case of moderate losses or reduction of mud column in the annulus in total losses. Using machine learning (ML) for predicting the presence of a loss zone and the estimation of fracture parameters ahead is very beneficial as it can immediately alert the drilling crew in order for them to take the required actions to mitigate or cure LCEs. Methods, Procedures, Process. Although different computational methods have been proposed for the prediction of LCEs, there is a need to further improve the models and reduce the number of false alarms. Robust and generalizable ML models require a sufficiently large amount of data that captures the different parameters and scenarios representing an LCE. For this, we derived a framework that automatically searches through historical data, locates LCEs, and extracts the surface drilling and rheology parameters surrounding such events. Results, Observations, and Conclusions. We derived different ML models utilizing various algorithms and evaluated them using the data-split technique at the level of wells to find the most suitable model for the prediction of an LCE. From the model comparison, random forest classifier achieved the best results and successfully predicted LCEs before they occurred. The developed LCE model is designed to be implemented in the real-time drilling portal as an aid to the drilling engineers and the rig crew to minimize or avoid NPT. Novel/Additive Information. The main contribution of this study is the analysis of real-time surface drilling parameters and sensor data to predict an LCE from a statistically representative number of wells. The large-scale analysis of several wells that appropriately describe the different conditions before an LCE is critical for avoiding model undertraining or lack of model generalization. Finally, we formulated the prediction of LCEs as a time-series problem and considered parameter trends to accurately determine the early signs of LCEs.

Download Full-text

Large-scale features of Pliocene climate: results from the Pliocene Model Intercomparison Project

Climate of the Past Discussions ◽

10.5194/cpd-8-2969-2012 ◽

2012 ◽

Vol 8 (4) ◽

pp. 2969-3013 ◽

Cited By ~ 6

Author(s):

A. M. Haywood ◽

D. J. Hill ◽

A. M. Dolan ◽

B. Otto-Bliesner ◽

F. Bragg ◽

...

Keyword(s):

Data Model ◽

Large Scale ◽

Model Comparison ◽

Numerical Models ◽

Model Intercomparison ◽

System Sensitivity ◽

Sensitivity Tests ◽

Intercomparison Project ◽

Known Unknowns ◽

Shed Light

Abstract. Climate and environments of the mid-Pliocene Warm Period (3.264 to 3.025 Ma) have been extensively studied. Whilst numerical models have shed light on the nature of climate at the time, uncertainties in their predictions have not been systematically examined. The Pliocene Model Intercomparison Project quantifies uncertainties in model outputs through a co-ordinated multi-model and multi-model/data intercomparison. Whilst commonalities in model outputs for the Pliocene are evident, we show substantial variation in the sensitivity of models to the implementation of Pliocene boundary conditions. Models appear able to reproduce many regional changes in temperature reconstructed from geological proxies. However, data/model comparison highlights the potential for models to underestimate polar amplification. To assert this conclusion with greater confidence, limitations in the time-averaged proxy data currently available must be addressed. Sensitivity tests exploring the "known unknowns" in modelling Pliocene climate specifically relevant to the high-latitudes are also essential (e.g. palaeogeography, gateways, orbital forcing and trace gasses). Estimates of longer-term sensitivity to CO2 (also known as Earth System Sensitivity; ESS), suggest that ESS is greater than Climate Sensitivity (CS), and that the ratio of ESS to CS is between 1 and 2, with a best estimate of 1.5.

Download Full-text

Large-scale features of Pliocene climate: results from the Pliocene Model Intercomparison Project

Climate of the Past ◽

10.5194/cp-9-191-2013 ◽

2013 ◽

Vol 9 (1) ◽

pp. 191-209 ◽

Cited By ~ 202

Author(s):

A. M. Haywood ◽

D. J. Hill ◽

A. M. Dolan ◽

B. L. Otto-Bliesner ◽

F. Bragg ◽

...

Keyword(s):

Data Model ◽

Large Scale ◽

Model Comparison ◽

Numerical Models ◽

Model Intercomparison ◽

System Sensitivity ◽

Sensitivity Tests ◽

Intercomparison Project ◽

Known Unknowns ◽

Shed Light

Abstract. Climate and environments of the mid-Pliocene warm period (3.264 to 3.025 Ma) have been extensively studied. Whilst numerical models have shed light on the nature of climate at the time, uncertainties in their predictions have not been systematically examined. The Pliocene Model Intercomparison Project quantifies uncertainties in model outputs through a coordinated multi-model and multi-model/data intercomparison. Whilst commonalities in model outputs for the Pliocene are clearly evident, we show substantial variation in the sensitivity of models to the implementation of Pliocene boundary conditions. Models appear able to reproduce many regional changes in temperature reconstructed from geological proxies. However, data/model comparison highlights that models potentially underestimate polar amplification. To assert this conclusion with greater confidence, limitations in the time-averaged proxy data currently available must be addressed. Furthermore, sensitivity tests exploring the known unknowns in modelling Pliocene climate specifically relevant to the high latitudes are essential (e.g. palaeogeography, gateways, orbital forcing and trace gasses). Estimates of longer-term sensitivity to CO2 (also known as Earth System Sensitivity; ESS), support previous work suggesting that ESS is greater than Climate Sensitivity (CS), and suggest that the ratio of ESS to CS is between 1 and 2, with a "best" estimate of 1.5.

Download Full-text

Long‐term predictability of soil moisture dynamics at the global scale: Persistence versus large‐scale drivers

Geophysical Research Letters ◽

10.1002/2016gl069847 ◽

2016 ◽

Vol 43 (16) ◽

pp. 8554-8562 ◽

Cited By ~ 19

Author(s):

Nadine Nicolai‐Shaw ◽

Lukas Gudmundsson ◽

Martin Hirschi ◽

Sonia I. Seneviratne

Keyword(s):

Soil Moisture ◽

Large Scale ◽

Global Scale ◽

Soil Moisture Dynamics ◽

Moisture Dynamics

Download Full-text

MONITORING OF TIME-SERIES SOIL MOISTURE BASED ON ADVANCED DINSAR

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-v-3-2021-51-2021 ◽

2021 ◽

Vol V-3-2021 ◽

pp. 51-55

Author(s):

C. H. Yang ◽

A. Müterthies

Keyword(s):

Time Series ◽

Soil Moisture ◽

Water Content ◽

Large Scale ◽

Pearson Correlation ◽

Correlation Coefficients ◽

Organic Soil ◽

Underground Water ◽

Multi Temporal ◽

Mean Square Errors

Abstract. Understanding soil moisture is essential for earth and environmental sciences especially in geology, hydrology, and meteorology. Remote sensing techniques are widely applied to large-scale monitoring tasks. Among them, DInSAR using multi-temporal spaceborne SAR images is able to derive surface movement up to mm level over an area. One of the factors inducing the movement is variation of soil moisture. Based on this, a semi-empirical approach can be tailored to retrieve the underground water content. However, the derived movement is often contaminated with other irrelevant noise. Besides, a time-series analysis could not be simply implemented without additional fusion and calibration. In this paper, we propose a novel modelling based on advanced DInSAR to solve these problems. The irrelevant noise will be removed as parts of the modelled elements in the DInSAR processing. A forward model on a scene is built by regressing the measured soil moisture on the DInSAR-derived movement series. We tested our approach using Sentinel-1 images in the grasslands of organic soil within State of Brandenburg, Germany. The Pearson correlation coefficients between the measured soil moistures and the DInSAR-derived movements are up to 0.91. The mean square errors of the predicted soil moistures compared with the measurements reach 3.03 % (volumetric water content) at best. Our study shows a promising new concept to develop a global monitoring of soil moisture in the future.

Download Full-text

Long-term Soil Moisture Time Series Analyses based on Active Microwave Backscatter Measurements

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-7-w3-545-2015 ◽

2015 ◽

Vol XL-7/W3 ◽

pp. 545-550

Author(s):

W. Wagner ◽

C. Reimer ◽

B. Bauer-Marschallinger ◽

M. Enenkel ◽

S. Hahn ◽

...

Keyword(s):

Time Series ◽

Soil Moisture ◽

Microwave Frequencies ◽

Vegetation Modelling ◽

Time Series Analyses ◽

Vegetation Parameters ◽

Microwave Backscatter ◽

Microwave Sensors ◽

Modelling Approach

Active microwave sensors operating at lower microwave frequencies in the range from 1 to 10 GHz provide backscatter measurements that are sensitive to the moisture content of the soil. Thanks to a series of European C-band (5.3 GHz) scatterometers, which were first flown on board of the European Remote Sensing satellites ERS-1 and ERS-2, and later on board of MetOp-A and MetOp -B, we are now in the possession of a long-term soil moisture time series starting in 1991. The creation of globally consistent long-term soil moisture time series is a challenging task. The TU-Wien soil moisture algorithm is adopted to tackle these challenges. In this paper we present two methodologies that were developed to ensure radiometric stability of the European C-band scatterometers. The objective of sensor intra-calibration is to monitor and correct for radiometric instabilities within one scatterometer mission, while sensor inter-calibration aims to remove radiometric differences across several missions. In addition, a novel vegetation modelling approach is presented that enables the estimation of vegetation parameters for each day across several years to account for yearly to longer-term changes in vegetation phenology and land cover.

Download Full-text

On the influence of soil moisture on intra-annual peat soil dynamics as observed from SAR amplitude and phase data

10.5194/egusphere-egu2020-18234 ◽

2020 ◽

Author(s):

Falco Bentvelsen ◽

Floris Heuff ◽

Susan Steele-Dunne ◽

Wolfgang Wagner ◽

Raphael Quast ◽

...

Keyword(s):

Time Series ◽

Soil Moisture ◽

Surface Soil ◽

Peat Soil ◽

Soil Dynamics ◽

Surface Elevation ◽

Driving Mechanism ◽

Surface Soil Moisture ◽

Precipitation And Temperature

Polders in the western Netherlands are often covered by pastures. Around 30 percent of the pastures are situated on peat soils, which are artificially drained. Consequently, the exposure to oxygen leads to a decomposition (oxidation) of the material and desiccation leading to shrinking. This results in a decadal subsidence, up to a few centimeters per year, which causes increasingly severe socio-economic impact. However, this long-term subsidence signal has a high spatial variability due to local soil morphology, and possibly high intra-annual temporal variability which is caused by precipitation and evaporation. The problem is that there are currently no geodetic methods that can reliably measure these soil dynamics over wide areas and with high temporal revisits.Here we show how Sentinel-1 SAR interferometry (InSAR) can potentially be used to estimate the surface displacements, given prior information on precipitation and temperature. We observe intra-annual dynamics of surface elevation which seem to be one order of magnitude stronger than the decadal long-term subsidence. InSAR surface elevation measurements show &#160;discontinuities (hysteresis) in late summer and early autumn due to strong vegetation and changes in temperature and precipitation patterns. As soil moisture variability appears to be the main driving mechanism for the observed surface elevation dynamics, we investigate whether we can use the amplitude of the identical SAR acquisitions to estimate the soil moisture directly, to reduce the dependency on external precipitation and temperature data.The analysis is performed on time series of the European Space Agency&#8217;s Sentinel-1 mission. Subsidence and upheaval are estimated using a novel InSAR algorithm, which was specially designed for peat soil dynamics. The surface elevation dynamics are compared to surface soil moisture estimates from Sentinel-1 amplitude &#160;data. Soil moisture is retrieved from backscatter time series using a first-order radiative transfer model (RT1) developed at TU Wien. This model describes the scattering behaviour of both soil- and vegetation by using linear combinations of idealized scattering distribution functions. Clay Soil swelling and subsidence are likely influenced by soil layers much deeper than those associated with the surface soil moisture estimates. Therefore, the subsidence estimates are also compared to Soil Water Index (SWI) derived from the surface soil moisture product. This is considered an indicator of moisture availability in the top 100 cm. These results show that the same complex SAR data acquisitions can be used simultaneously, but independently, for estimating soil moisture and for estimating surface elevation dynamics. An integrated application is proposed and evaluated for further exploration.

Download Full-text