Augmenting satellite-derived soil moisture with multiple data streams using machine learning

2020 ◽  
Author(s):  
Rene Orth ◽  
Sungmin Oh
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Data Streams ◽  
Large Scale ◽  
Extreme Weather Events ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Hydrological Hazards ◽  
Target Data

<p>Soil moisture plays a key role in land-atmosphere interactions through its influence on the energy and water cycles. Furthermore, its spatiotemporal variations can affect the development and persistence of extreme weather events. Consequently, soil moisture information is required for a wide range of research and applications, such as agricultural monitoring, flood and drought prediction, climate projection, and carbon-cycle modeling. Despite its scientific and societal importance, observations of soil moisture are sparse, in particular across time and at large spatial scales. Only models and satellite retrievals can provide global soil moisture information. While the ability of land surface models to represent the complex land-atmosphere interplay is still limited, satellite-based soil moisture data are a valuable alternative. However, these products suffer from a scaling based on models, and can only capture the top few centimeters of the soil. </p><p>In this study, we aim to augment satellite-based soil moisture data using machine learning. For this purpose we integrate satellite soil moisture with multiple hydro-meteorological data streams to derive global gridded soil moisture using Long Short-Term Memory (LSTM) neural networks. These networks are trained using in-situ soil moisture measurements as target data. With the resulting self-learned relationships, the LSTMs can produce in-situ-like soil moisture globally. We further analyze the implications of using point-scale target data to infer large scale information. The new dataset is derived separately for the surface and the deeper soil, thereby extending beyond the range covered by the satellite-based products. The integration of many data streams and multiple soil moisture observations through a powerful synergistic technique offers the potential to yield high accuracy. This is tested through rigorous cross-validation of the derived dataset. Finally, the planned datasets will permit consistent long-term, large-scale analysis to enhance our understanding of the hydrology-biosphere-climate interplay, to better constrain models and to support hydrological hazards monitoring and climate projections.</p>

scholarly journals Global soil moisture data derived through machine learning trained with in-situ measurements

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Sungmin O. ◽  
Rene Orth
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Large Scale ◽  
Short Term Memory ◽  
Temporal Dynamics ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Global Soil

AbstractWhile soil moisture information is essential for a wide range of hydrologic and climate applications, spatially-continuous soil moisture data is only available from satellite observations or model simulations. Here we present a global, long-term dataset of soil moisture derived through machine learning trained with in-situ measurements, SoMo.ml. We train a Long Short-Term Memory (LSTM) model to extrapolate daily soil moisture dynamics in space and in time, based on in-situ data collected from more than 1,000 stations across the globe. SoMo.ml provides multi-layer soil moisture data (0–10 cm, 10–30 cm, and 30–50 cm) at 0.25° spatial and daily temporal resolution over the period 2000–2019. The performance of the resulting dataset is evaluated through cross validation and inter-comparison with existing soil moisture datasets. SoMo.ml performs especially well in terms of temporal dynamics, making it particularly useful for applications requiring time-varying soil moisture, such as anomaly detection and memory analyses. SoMo.ml complements the existing suite of modelled and satellite-based datasets given its distinct derivation, to support large-scale hydrological, meteorological, and ecological analyses.

scholarly journals Evaluating Machine Learning and Geostatistical Methods for Spatial Gap-filling of Monthly ESA CCI Soil Moisture in China

2021 ◽  
Vol 13 (14) ◽  
pp. 2848
Author(s):  
Hao Sun ◽  
Qian Xu
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Water Resource Management ◽  
Large Scale ◽  
Gap Filling ◽  
Spatial Continuity ◽  
Study Results ◽  
Data Gaps

Obtaining large-scale, long-term, and spatial continuous soil moisture (SM) data is crucial for climate change, hydrology, and water resource management, etc. ESA CCI SM is such a large-scale and long-term SM (longer than 40 years until now). However, there exist data gaps, especially for the area of China, due to the limitations in remote sensing of SM such as complex topography, human-induced radio frequency interference (RFI), and vegetation disturbances, etc. The data gaps make the CCI SM data cannot achieve spatial continuity, which entails the study of gap-filling methods. In order to develop suitable methods to fill the gaps of CCI SM in the whole area of China, we compared typical Machine Learning (ML) methods, including Random Forest method (RF), Feedforward Neural Network method (FNN), and Generalized Linear Model (GLM) with a geostatistical method, i.e., Ordinary Kriging (OK) in this study. More than 30 years of passive–active combined CCI SM from 1982 to 2018 and other biophysical variables such as Normalized Difference Vegetation Index (NDVI), precipitation, air temperature, Digital Elevation Model (DEM), soil type, and in situ SM from International Soil Moisture Network (ISMN) were utilized in this study. Results indicated that: 1) the data gap of CCI SM is frequent in China, which is found not only in cold seasons and areas but also in warm seasons and areas. The ratio of gap pixel numbers to the whole pixel numbers can be greater than 80%, and its average is around 40%. 2) ML methods can fill the gaps of CCI SM all up. Among the ML methods, RF had the best performance in fitting the relationship between CCI SM and biophysical variables. 3) Over simulated gap areas, RF had a comparable performance with OK, and they outperformed the FNN and GLM methods greatly. 4) Over in situ SM networks, RF achieved better performance than the OK method. 5) We also explored various strategies for gap-filling CCI SM. Results demonstrated that the strategy of constructing a monthly model with one RF for simulating monthly average SM and another RF for simulating monthly SM disturbance achieved the best performance. Such strategy combining with the ML method such as the RF is suggested in this study for filling the gaps of CCI SM in China.

scholarly journals Mapping High Spatiotemporal-Resolution Soil Moisture by Upscaling Sparse Ground-Based Observations Using a Bayesian Linear Regression Method for Comparison with Microwave Remotely Sensed Soil Moisture Products

2021 ◽  
Vol 13 (2) ◽  
pp. 228
Author(s):  
Jian Kang ◽  
Rui Jin ◽  
Xin Li ◽  
Yang Zhang
Keyword(s):  
Soil Moisture ◽  
Large Scale ◽  
Spatial Scales ◽  
Microwave Remote Sensing ◽  
In Situ Measurements ◽  
Spatial Density ◽  
Linear Regression Method ◽  
Spatiotemporal Resolution ◽  
Pixel Scale

In recent decades, microwave remote sensing (RS) has been used to measure soil moisture (SM). Long-term and large-scale RS SM datasets derived from various microwave sensors have been used in environmental fields. Understanding the accuracies of RS SM products is essential for their proper applications. However, due to the mismatched spatial scale between the ground-based and RS observations, the truth at the pixel scale may not be accurately represented by ground-based observations, especially when the spatial density of in situ measurements is low. Because ground-based observations are often sparsely distributed, temporal upscaling was adopted to transform a few in situ measurements into SM values at a pixel scale of 1 km by introducing the temperature vegetation dryness index (TVDI) related to SM. The upscaled SM showed high consistency with in situ SM observations and could accurately capture rainfall events. The upscaled SM was considered as the reference data to evaluate RS SM products at different spatial scales. In regard to the validation results, in addition to the correlation coefficient (R) of the Soil Moisture Active Passive (SMAP) SM being slightly lower than that of the Climate Change Initiative (CCI) SM, SMAP had the best performance in terms of the root-mean-square error (RMSE), unbiased RMSE and bias, followed by the CCI. The Soil Moisture and Ocean Salinity (SMOS) products were in worse agreement with the upscaled SM and were inferior to the R value of the X-band SM of the Advanced Microwave Scanning Radiometer 2 (AMSR2). In conclusion, in the study area, the SMAP and CCI SM are more reliable, although both products were underestimated by 0.060 cm3 cm−3 and 0.077 cm3 cm−3, respectively. If the biases are corrected, then the improved SMAP with an RMSE of 0.043 cm3 cm−3 and the CCI with an RMSE of 0.039 cm3 cm−3 will hopefully reach the application requirement for an accuracy with an RMSE less than 0.040 cm3 cm−3.

scholarly journals Use of satellite and modeled soil moisture data for predicting event soil loss at plot scale

2015 ◽  
Vol 19 (9) ◽  
pp. 3845-3856 ◽  
Author(s):  
F. Todisco ◽  
L. Brocca ◽  
L. F. Termite ◽  
W. Wagner
Keyword(s):  
Soil Moisture ◽  
Soil Loss ◽  
Large Scale ◽  
Central Italy ◽  
Remote Sensing Data ◽  
Loss Estimation ◽  
Balance Model ◽  
Estimation Accuracy ◽  
Soil Losses

Abstract. The potential of coupling soil moisture and a Universal Soil Loss Equation-based (USLE-based) model for event soil loss estimation at plot scale is carefully investigated at the Masse area, in central Italy. The derived model, named Soil Moisture for Erosion (SM4E), is applied by considering the unavailability of in situ soil moisture measurements, by using the data predicted by a soil water balance model (SWBM) and derived from satellite sensors, i.e., the Advanced SCATterometer (ASCAT). The soil loss estimation accuracy is validated using in situ measurements in which event observations at plot scale are available for the period 2008–2013. The results showed that including soil moisture observations in the event rainfall–runoff erosivity factor of the USLE enhances the capability of the model to account for variations in event soil losses, the soil moisture being an effective alternative to the estimated runoff, in the prediction of the event soil loss at Masse. The agreement between observed and estimated soil losses (through SM4E) is fairly satisfactory with a determination coefficient (log-scale) equal to ~ 0.35 and a root mean square error (RMSE) of ~ 2.8 Mg ha−1. These results are particularly significant for the operational estimation of soil losses. Indeed, currently, soil moisture is a relatively simple measurement at the field scale and remote sensing data are also widely available on a global scale. Through satellite data, there is the potential of applying the SM4E model for large-scale monitoring and quantification of the soil erosion process.

Coupled Thermo-Hydro-Geochemical Models of Engineered Barrier Systems: The Febex Project

2000 ◽  
Vol 663 ◽  
Author(s):  
J. Samper ◽  
R. Juncosa ◽  
V. Navarro ◽  
J. Delgado ◽  
L. Montenegro ◽  
...  
Keyword(s):  
Large Scale ◽  
Reference Model ◽  
Numerical Models ◽  
Full Scale ◽  
Small Scale ◽  
Model Parameters ◽  
In Situ Test ◽  
Wide Range ◽  
Engineered Barrier

ABSTRACTFEBEX (Full-scale Engineered Barrier EXperiment) is a demonstration and research project dealing with the bentonite engineered barrier designed for sealing and containment of waste in a high level radioactive waste repository (HLWR). It includes two main experiments: an situ full-scale test performed at Grimsel (GTS) and a mock-up test operating since February 1997 at CIEMAT facilities in Madrid (Spain) [1,2,3]. One of the objectives of FEBEX is the development and testing of conceptual and numerical models for the thermal, hydrodynamic, and geochemical (THG) processes expected to take place in engineered clay barriers. A significant improvement in coupled THG modeling of the clay barrier has been achieved both in terms of a better understanding of THG processes and more sophisticated THG computer codes. The ability of these models to reproduce the observed THG patterns in a wide range of THG conditions enhances the confidence in their prediction capabilities. Numerical THG models of heating and hydration experiments performed on small-scale lab cells provide excellent results for temperatures, water inflow and final water content in the cells [3]. Calculated concentrations at the end of the experiments reproduce most of the patterns of measured data. In general, the fit of concentrations of dissolved species is better than that of exchanged cations. These models were later used to simulate the evolution of the large-scale experiments (in situ and mock-up). Some thermo-hydrodynamic hypotheses and bentonite parameters were slightly revised during TH calibration of the mock-up test. The results of the reference model reproduce simultaneously the observed water inflows and bentonite temperatures and relative humidities. Although the model is highly sensitive to one-at-a-time variations in model parameters, the possibility of parameter combinations leading to similar fits cannot be precluded. The TH model of the “in situ” test is based on the same bentonite TH parameters and assumptions as for the “mock-up” test. Granite parameters were slightly modified during the calibration process in order to reproduce the observed thermal and hydrodynamic evolution. The reference model captures properly relative humidities and temperatures in the bentonite [3]. It also reproduces the observed spatial distribution of water pressures and temperatures in the granite. Once calibrated the TH aspects of the model, predictions of the THG evolution of both tests were performed. Data from the dismantling of the in situ test, which is planned for the summer of 2001, will provide a unique opportunity to test and validate current THG models of the EBS.

Exploring trends for the H SAF ASCAT root-zone soil moisture data records

2021 ◽  
Author(s):  
David Fairbairn ◽  
Patricia de Rosnay ◽  
Peter Weston
Keyword(s):  
Soil Moisture ◽  
Root Zone ◽  
Weather Prediction ◽  
Climate Change Impacts ◽  
Soil Moisture Data ◽  
Negative Root ◽  
In Situ Observations ◽  
Prediction Systems ◽  
Water Scarce

<p>Environmental (e.g. floods, droughts) and weather prediction systems rely on an accurate representation of soil moisture (SM). The EUMETSAT H SAF aims to provide high quality satellite-based hydrological products, including SM.<br>ECMWF is producing ASCAT root zone SM for H SAF. The production relies on an Extended Kalman filter to retrieve root zone SM from surface SM satellite data. A 10 km sampling reanalysis product (1992-2020) forced by ERA5 atmospheric fields (H141/H142) is produced for H SAF, which assimilates ERS/SCAT (1992-2006) and ASCAT-A/B/C (2007-2020) derived surface SM. The root-zone SM performance is validated using sparse in situ observations globally and generally demonstrates a positive and consistent correlation over the period. A negative trend in root-zone SM is found during summer and autumn months over much of Europe during the period (1992-2020). This is consistent with expected climate change impacts and is particularly alarming over the water-scarce Mediterranean region. The recent hot and dry summer of 2019 and dry spring of 2020 are well captured by negative root-zone SM anomalies. Plans for the future H SAF data record products will be presented, including the assimilation of high-resolution EPS-SCA-derived soil moisture data.</p>

scholarly journals Spatial variability and its scale dependency of observed and modeled soil moisture over different climate regions

2013 ◽  
Vol 17 (3) ◽  
pp. 1177-1188 ◽  
Author(s):  
B. Li ◽  
M. Rodell
Keyword(s):  
Soil Moisture ◽  
Spatial Variability ◽  
Land Surface ◽  
Large Scale ◽  
Warm Season ◽  
Scale Dependency ◽  
Convex Shape ◽  
Spatial Moments ◽  
Environmental Controls

Abstract. Past studies on soil moisture spatial variability have been mainly conducted at catchment scales where soil moisture is often sampled over a short time period; as a result, the observed soil moisture often exhibited smaller dynamic ranges, which prevented the complete revelation of soil moisture spatial variability as a function of mean soil moisture. In this study, spatial statistics (mean, spatial variability and skewness) of in situ soil moisture, modeled and satellite-retrieved soil moisture obtained in a warm season (198 days) were examined over three large climate regions in the US. The study found that spatial moments of in situ measurements strongly depend on climates, with distinct mean, spatial variability and skewness observed in each climate zone. In addition, an upward convex shape, which was revealed in several smaller scale studies, was observed for the relationship between spatial variability of in situ soil moisture and its spatial mean when statistics from dry, intermediate, and wet climates were combined. This upward convex shape was vaguely or partially observable in modeled and satellite-retrieved soil moisture estimates due to their smaller dynamic ranges. Despite different environmental controls on large-scale soil moisture spatial variability, the correlation between spatial variability and mean soil moisture remained similar to that observed at small scales, which is attributed to the boundedness of soil moisture. From the smaller support (effective area or volume represented by a measurement or estimate) to larger ones, soil moisture spatial variability decreased in each climate region. The scale dependency of spatial variability all followed the power law, but data with large supports showed stronger scale dependency than those with smaller supports. The scale dependency of soil moisture variability also varied with climates, which may be linked to the scale dependency of precipitation spatial variability. Influences of environmental controls on soil moisture spatial variability at large scales are discussed. The results of this study should be useful for diagnosing large scale soil moisture estimates and for improving the estimation of land surface processes.

scholarly journals Using data assimilation to optimize pedotransfer functions using large-scale in-situ soil moisture observations

2020 ◽  
Author(s):  
Elizabeth Cooper ◽  
Eleanor Blyth ◽  
Hollie Cooper ◽  
Rich Ellis ◽  
Ewan Pinnington ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Data Assimilation ◽  
Land Surface ◽  
Large Scale ◽  
Pedotransfer Functions ◽  
Surface Model ◽  
Climate Projections ◽  
Land Surface Models ◽  
Surface Models

Abstract. Soil moisture predictions from land surface models are important in hydrological, ecological and meteorological applications. In recent years the availability of wide-area soil-moisture measurements has increased, but few studies have combined model-based soil moisture predictions with in-situ observations beyond the point scale. Here we show that we can markedly improve soil moisture estimates from the JULES land surface model using field scale observations and data assimilation techniques. Rather than directly updating soil moisture estimates towards observed values, we optimize constants in the underlying pedotransfer functions, which relate soil texture to JULES soil physics parameters. In this way we generate a single set of newly calibrated pedotransfer functions based on observations from a number of UK sites with different soil textures. We demonstrate that calibrating a pedotransfer function in this way can improve the performance of land surface models, leading to the potential for better flood, drought and climate projections.

scholarly journals Large Scale Operational Soil Moisture Mapping from Passive MW Radiometry: SMOS product evaluation in Europe & USA

2019 ◽  
Author(s):  
Khidir Abdalla Kwal Deng ◽  
Salim Lamine ◽  
Andrew Pavlides ◽  
Yansong Bao ◽  
George Petropoulos ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Large Scale ◽  
Water Security ◽  
Product Evaluation ◽  
Practical Applications ◽  
Study Results ◽  
Different Seasons ◽  
Lower Correlation ◽  
Land Types

Earth Observation (EO) allows deriving from a range of sensors, often globally, operational estimates of surface soil moisture (SSM) at range of spatiotemporal resolutions. Yet, an evaluation of the accuracy of those products in a variety of environmental conditions has been often limited. In this study the accuracy of the SMOS SSM global operational product across 2 continents (USA, and Europe) is investigated. SMOS predictions were compared against near concurrent in-situ SSM measurements from the FLUXNET observational network. In total, 7 experimental sites were used to assess the accuracy of SMOS derived soil moisture for 2 complete years of observations (2010 to 2011). The accuracy of the SMOS SSM product is investigated in different seasons for the seasonal cycle as well as different continents and land types. Results showed a generally reasonable agreement between the SMOS product and the in-situ soil moisture measurements in the 0-5 cm soil moisture layer. Root Mean Square Error (RMSE) in most cases was close to 0.1 m3 m-3 (minimum 0.067 m3 m-3). With a few exceptions, Pearson’s correlation coefficient was found up to approx. 55%. Grassland, shrublands and woody savanna land cover types attained a satisfactory agreement between satellite derived and in-situ measurements but needleleaf forests had lower correlation. Better agreement was found for the grassland sites in both continents. Seasonally, summer and autumn underperformed spring and winter. Our study results provide supportive evidence of the potential value of this operational product for meso-scale studies in a range of practical applications, helping to address key challenges present nowadays linked to food and water security.

A regionally explicit, global SWI calibration based on ISMN observations

2021 ◽  
Author(s):  
Manolis G. Grillakis
Keyword(s):  
Soil Moisture ◽  
Soil Water ◽  
Large Scale ◽  
Root Zone ◽  
Soil Depth ◽  
European Space Agency ◽  
Soil Layer ◽  
Added Value ◽  
Water Index ◽  
Soil Moisture Data

<p>Remote sensing has proven to be an irreplaceable tool for monitoring soil moisture. The European Space Agency (ESA), through the Climate Change Initiative (CCI), has provided one of the most substantial contributions in the soil water monitoring, with almost 4 decades of global satellite derived and homogenized soil moisture data for the uppermost soil layer. Yet, due to the inherent limitations of many of the remote sensors, only a limited soil depth can be monitored. To enable the assessment of the deeper soil layer moisture from surface remotely sensed products, the Soil Water Index (SWI) has been established as a convolutive transformation of the surface soil moisture estimation, under the assumption of uniform hydraulic conductivity and the absence of transpiration. The SWI uses a single calibration parameter, the T-value, to modify its response over time.</p><p>Here the Soil Water Index (SWI) is calibrated using ESA CCI soil moisture against in situ observations from the International Soil Moisture Network and then use Artificial Neural Networks (ANNs) to find the best physical soil, climate, and vegetation descriptors at a global scale to regionalize the calibration of the T-value. The calibration is then used to assess a root zone related soil moisture for the period 2001 – 2018.</p><p>The results are compared against the European Centre for Medium-Range Weather Forecasts, ERA5 Land reanalysis soil moisture dataset, showing a good agreement, mainly over mid-latitudes. The results indicate that there is added value to the results of the machine learning calibration, comparing to the uniform T-value. This work contributes to the exploitation of ESA CCI soil moisture data, while the produced data can support large scale soil moisture related studies.</p>

Sign in / Sign up

Export Citation Format

Share Document