scholarly journals Evaluating Machine Learning and Geostatistical Methods for Spatial Gap-filling of Monthly ESA CCI Soil Moisture in China

2021 ◽  
Vol 13 (14) ◽  
pp. 2848
Author(s):  
Hao Sun ◽  
Qian Xu
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Water Resource Management ◽  
Large Scale ◽  
Gap Filling ◽  
Spatial Continuity ◽  
Study Results ◽  
Data Gaps

Obtaining large-scale, long-term, and spatial continuous soil moisture (SM) data is crucial for climate change, hydrology, and water resource management, etc. ESA CCI SM is such a large-scale and long-term SM (longer than 40 years until now). However, there exist data gaps, especially for the area of China, due to the limitations in remote sensing of SM such as complex topography, human-induced radio frequency interference (RFI), and vegetation disturbances, etc. The data gaps make the CCI SM data cannot achieve spatial continuity, which entails the study of gap-filling methods. In order to develop suitable methods to fill the gaps of CCI SM in the whole area of China, we compared typical Machine Learning (ML) methods, including Random Forest method (RF), Feedforward Neural Network method (FNN), and Generalized Linear Model (GLM) with a geostatistical method, i.e., Ordinary Kriging (OK) in this study. More than 30 years of passive–active combined CCI SM from 1982 to 2018 and other biophysical variables such as Normalized Difference Vegetation Index (NDVI), precipitation, air temperature, Digital Elevation Model (DEM), soil type, and in situ SM from International Soil Moisture Network (ISMN) were utilized in this study. Results indicated that: 1) the data gap of CCI SM is frequent in China, which is found not only in cold seasons and areas but also in warm seasons and areas. The ratio of gap pixel numbers to the whole pixel numbers can be greater than 80%, and its average is around 40%. 2) ML methods can fill the gaps of CCI SM all up. Among the ML methods, RF had the best performance in fitting the relationship between CCI SM and biophysical variables. 3) Over simulated gap areas, RF had a comparable performance with OK, and they outperformed the FNN and GLM methods greatly. 4) Over in situ SM networks, RF achieved better performance than the OK method. 5) We also explored various strategies for gap-filling CCI SM. Results demonstrated that the strategy of constructing a monthly model with one RF for simulating monthly average SM and another RF for simulating monthly SM disturbance achieved the best performance. Such strategy combining with the ML method such as the RF is suggested in this study for filling the gaps of CCI SM in China.

scholarly journals Global soil moisture data derived through machine learning trained with in-situ measurements

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Sungmin O. ◽  
Rene Orth
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Large Scale ◽  
Short Term Memory ◽  
Temporal Dynamics ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Global Soil

AbstractWhile soil moisture information is essential for a wide range of hydrologic and climate applications, spatially-continuous soil moisture data is only available from satellite observations or model simulations. Here we present a global, long-term dataset of soil moisture derived through machine learning trained with in-situ measurements, SoMo.ml. We train a Long Short-Term Memory (LSTM) model to extrapolate daily soil moisture dynamics in space and in time, based on in-situ data collected from more than 1,000 stations across the globe. SoMo.ml provides multi-layer soil moisture data (0–10 cm, 10–30 cm, and 30–50 cm) at 0.25° spatial and daily temporal resolution over the period 2000–2019. The performance of the resulting dataset is evaluated through cross validation and inter-comparison with existing soil moisture datasets. SoMo.ml performs especially well in terms of temporal dynamics, making it particularly useful for applications requiring time-varying soil moisture, such as anomaly detection and memory analyses. SoMo.ml complements the existing suite of modelled and satellite-based datasets given its distinct derivation, to support large-scale hydrological, meteorological, and ecological analyses.

scholarly journals Large Scale Operational Soil Moisture Mapping from Passive MW Radiometry: SMOS product evaluation in Europe & USA

2019 ◽  
Author(s):  
Khidir Abdalla Kwal Deng ◽  
Salim Lamine ◽  
Andrew Pavlides ◽  
Yansong Bao ◽  
George Petropoulos ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Large Scale ◽  
Water Security ◽  
Product Evaluation ◽  
Practical Applications ◽  
Study Results ◽  
Different Seasons ◽  
Lower Correlation ◽  
Land Types

Earth Observation (EO) allows deriving from a range of sensors, often globally, operational estimates of surface soil moisture (SSM) at range of spatiotemporal resolutions. Yet, an evaluation of the accuracy of those products in a variety of environmental conditions has been often limited. In this study the accuracy of the SMOS SSM global operational product across 2 continents (USA, and Europe) is investigated. SMOS predictions were compared against near concurrent in-situ SSM measurements from the FLUXNET observational network. In total, 7 experimental sites were used to assess the accuracy of SMOS derived soil moisture for 2 complete years of observations (2010 to 2011). The accuracy of the SMOS SSM product is investigated in different seasons for the seasonal cycle as well as different continents and land types. Results showed a generally reasonable agreement between the SMOS product and the in-situ soil moisture measurements in the 0-5 cm soil moisture layer. Root Mean Square Error (RMSE) in most cases was close to 0.1 m3 m-3 (minimum 0.067 m3 m-3). With a few exceptions, Pearson’s correlation coefficient was found up to approx. 55%. Grassland, shrublands and woody savanna land cover types attained a satisfactory agreement between satellite derived and in-situ measurements but needleleaf forests had lower correlation. Better agreement was found for the grassland sites in both continents. Seasonally, summer and autumn underperformed spring and winter. Our study results provide supportive evidence of the potential value of this operational product for meso-scale studies in a range of practical applications, helping to address key challenges present nowadays linked to food and water security.

Performance of a Physically Based Gap-Filling Technique of in-situ Soil Moisture, in Comparison with Machine Learning

2020 ◽  
Author(s):  
Seulchan Lee ◽  
Hyunho Jeon ◽  
Jongmin Park ◽  
Minha Choi
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Low Voltage ◽  
Time Domain Reflectometry ◽  
Limited Information ◽  
Gap Filling ◽  
Flux Tower ◽  
Physically Based ◽  
Filling Method

<p>As the importance of Soil Moisture (SM) has been recognized in various fields, including agricultural practices, natural hazards, and climate predictions, ground-based SM sensors such as Frequency Domain Reflectometry (FDR), Time Domain Reflectometry (TDR) are being widely used. However, gaps in in-situ SM data are still unavoidable due not only to sensor failure or low voltage supply, but to environmental conditions. Since it is essential to acquire accurate and continuous SM data for its application purpose, the gaps in the data should be handled properly. In this study, we propose a physically based gap-filling method in a mountainous region, in which in-situ SM measurements and flux tower are located. This method is developed only with in-situ SM and precipitation data, by considering variation characteristics of SM: increases rapidly with precipitation and decreases asymptotically afterward. SM data from the past is used to build Look-Up-Tables (LUTs) that contains the amount and speed of increment and decrement of SM, with and without precipitation, respectively. Based on the developed LUTs, the gaps are filled successively from where the gaps started. At the same time, we also introduce a machine learning-based gap-filling framework for the comparison. Ancillary data from the flux tower (e.g. net radiation, relative humidity) was used as input for training, with the same period as in the physically based method. The trained models are then used to fill the gaps. We found that both proposed methods are able to fill the gaps of in-situ SM reasonably, with capabilities to capture the characteristics of SM variation. Results from the comparison indicate that the physically based gap-filling method is very accurate and efficient when there’s limited information, and also suitable to be used for prediction purposes.</p>

Augmenting satellite-derived soil moisture with multiple data streams using machine learning

2020 ◽  
Author(s):  
Rene Orth ◽  
Sungmin Oh
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Data Streams ◽  
Large Scale ◽  
Extreme Weather Events ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Hydrological Hazards ◽  
Target Data

<p>Soil moisture plays a key role in land-atmosphere interactions through its influence on the energy and water cycles. Furthermore, its spatiotemporal variations can affect the development and persistence of extreme weather events. Consequently, soil moisture information is required for a wide range of research and applications, such as agricultural monitoring, flood and drought prediction, climate projection, and carbon-cycle modeling. Despite its scientific and societal importance, observations of soil moisture are sparse, in particular across time and at large spatial scales. Only models and satellite retrievals can provide global soil moisture information. While the ability of land surface models to represent the complex land-atmosphere interplay is still limited, satellite-based soil moisture data are a valuable alternative. However, these products suffer from a scaling based on models, and can only capture the top few centimeters of the soil. </p><p>In this study, we aim to augment satellite-based soil moisture data using machine learning. For this purpose we integrate satellite soil moisture with multiple hydro-meteorological data streams to derive global gridded soil moisture using Long Short-Term Memory (LSTM) neural networks. These networks are trained using in-situ soil moisture measurements as target data. With the resulting self-learned relationships, the LSTMs can produce in-situ-like soil moisture globally. We further analyze the implications of using point-scale target data to infer large scale information. The new dataset is derived separately for the surface and the deeper soil, thereby extending beyond the range covered by the satellite-based products. The integration of many data streams and multiple soil moisture observations through a powerful synergistic technique offers the potential to yield high accuracy. This is tested through rigorous cross-validation of the derived dataset. Finally, the planned datasets will permit consistent long-term, large-scale analysis to enhance our understanding of the hydrology-biosphere-climate interplay, to better constrain models and to support hydrological hazards monitoring and climate projections.</p>

scholarly journals Large scale operational soil moisture mapping from passive MW radiometry: SMOS product evaluation in Europe & USA

2020 ◽  
Author(s):  
Khidir Abdalla Kwal Deng ◽  
George Petropoulos ◽  
Salim LAMINE ◽  
Andrew Pavlides ◽  
Yansong Bao ◽  
...  
Keyword(s):  
Land Use ◽  
Soil Moisture ◽  
Large Scale ◽  
Water Security ◽  
Product Evaluation ◽  
Practical Applications ◽  
Study Results ◽  
Different Seasons ◽  
Lower Correlation

Earth Observation (EO) allows deriving from a range of sensors, often globally, operational estimates of surfacesoil moisture (SSM) at range of spatiotemporal resolutions. Yet, an evaluation of the accuracy of those productsin a variety of environmental conditions has been often limited. In this study, the accuracy of the SMOS SSMglobal operational product across 2 continents (USA, and Europe) and a range of land use/cover types is investigated.SMOS predictions were compared against near concurrent in-situ SSM measurements from theFLUXNET observational network. In total, 7 experimental sites were used to assess the accuracy of SMOS derivedsoil moisture for 2 complete years of observations (2010–2011). The accuracy of the SMOS SSM product isinvestigated in different seasons for the seasonal cycle as well as different continents and land use/cover types.Results showed a generally reasonable agreement between the SMOS product and the in-situ soil moisturemeasurements in the 0–5 cm soil moisture layer. Root Mean Square Error (RMSE) in most cases was close to 0.1m3 m−3 (minimum 0.067m3 m−3). With a few exceptions, Pearson’s correlation coefficient was found up toapprox. 55%. Grassland, shrublands and woody savanna land cover types attained a satisfactory agreementbetween satellite derived and in-situ measurements but needleleaf forests had lower correlation. Better agreementwas found for the grassland sites in both continents. Seasonally, summer and autumn underperformedspring and winter. Our study results provide supportive evidence of the potential value of this operationalproduct for meso-scale studies in a range of practical applications, helping to address key challenges presentnowadays linked to food and water security.

scholarly journals SGD-SM: Generating Seamless Global Daily AMSR2 Soil Moisture Long-term Productions (2013–2019)

2020 ◽  
Author(s):  
Qiang Zhang ◽  
Qiangqiang Yuan ◽  
Jie Li ◽  
Yuan Wang ◽  
Fujun Sun ◽  
...  
Keyword(s):  
Time Series ◽  
Soil Moisture ◽  
Satellite Orbit ◽  
Temporal Consistency ◽  
Spatial Continuity ◽  
Coverage Ratio ◽  
Original Dataset ◽  
Original Time

Abstract. High quality and long-term soil moisture productions are significant for hydrologic monitoring and agricultural management. However, the acquired daily soil moisture productions are incomplete in global land (just about 30 %∼80 % coverage ratio), due to the satellite orbit coverage and the limitations of soil moisture retrieving algorithms. To solve this inevitable problem, we develop a novel 3D spatio-temporal partial convolutional neural network (CNN) for Advanced Microwave Scanning Radiometer 2 (AMSR2) soil moisture productions gap-filling. Through the proposed framework, we generate the seamless global daily (SGD) AMSR2 soil moisture long-term productions from 2013 to 2019. To further validate the effectiveness of these productions, three verification ways are employed as follow: 1) In-situ validation; 2) Time-series validation; And 3) simulated missing regions validation. Results show that the seamless global daily soil moisture productions have reliable cooperativity with the selected in-situ values. The evaluation indexes of the reconstructed (original) dataset are R: 0.683 (0.687), RMSE: 0.099 m3/m3 (0.095 m3/m3), and MAE: 0.081 m3/m3 (0.078 m3/m3), respectively. Temporal consistency of the reconstructed daily soil moisture productions is ensured with the original time-series distribution of valid values. Besides, the spatial continuity of the reconstructed regions is also accorded with the context information (R: 0.963∼0.974, RMSE: 0.065∼0.073 m3/m3, and MAE: 0.044∼0.052 m3/m3). More details of this work are released at https://qzhang95.github.io/Projects/Global-Daily-Seamless-AMSR2/. This dataset can be downloaded at https://zenodo.org/record/3960425 (Zhang et al., 2020. DOI:https://doi.org/10.5281/zenodo.3960425).

scholarly journals Generating seamless global daily AMSR2 soil moisture (SGD-SM) long-term products for the years 2013–2019

2021 ◽  
Vol 13 (3) ◽  
pp. 1385-1401
Author(s):  
Qiang Zhang ◽  
Qiangqiang Yuan ◽  
Jie Li ◽  
Yuan Wang ◽  
Fujun Sun ◽  
...  
Keyword(s):  
Time Series ◽  
Soil Moisture ◽  
Spatial Information ◽  
Mean Squared Error ◽  
Spatial Continuity ◽  
Coverage Ratio ◽  
Original Dataset ◽  
Verification Methods

Abstract. High-quality and long-term soil moisture products are significant for hydrologic monitoring and agricultural management. However, the acquired daily Advanced Microwave Scanning Radiometer 2 (AMSR2) soil moisture products are incomplete in global land (just about 30 %–80 % coverage ratio), due to the satellite orbit coverage and the limitations of soil moisture retrieval algorithms. To solve this inevitable problem, we develop a novel spatio-temporal partial convolutional neural network (CNN) for AMSR2 soil moisture product gap-filling. Through the proposed framework, we generate the seamless daily global (SGD) AMSR2 long-term soil moisture products from 2013 to 2019. To further validate the effectiveness of these products, three verification methods are used as follows: (1) in situ validation, (2) time-series validation, and (3) simulated missing-region validation. Results show that the seamless global daily soil moisture products have reliable cooperativity with the selected in situ values. The evaluation indexes of the reconstructed (original) dataset are a correlation coefficient (R) of 0.685 (0.689), root-mean-squared error (RMSE) of 0.097 (0.093), and mean absolute error (MAE) of 0.079 (0.077). The temporal consistency of the reconstructed daily soil moisture products is ensured with the original time-series distribution of valid values. The spatial continuity of the reconstructed regions is in accordance with the spatial information (R: 0.963–0.974, RMSE: 0.065–0.073, and MAE: 0.044–0.052). This dataset can be downloaded at https://doi.org/10.5281/zenodo.4417458 (Zhang et al., 2021).

scholarly journals Mapping High Spatiotemporal-Resolution Soil Moisture by Upscaling Sparse Ground-Based Observations Using a Bayesian Linear Regression Method for Comparison with Microwave Remotely Sensed Soil Moisture Products

2021 ◽  
Vol 13 (2) ◽  
pp. 228
Author(s):  
Jian Kang ◽  
Rui Jin ◽  
Xin Li ◽  
Yang Zhang
Keyword(s):  
Soil Moisture ◽  
Large Scale ◽  
Spatial Scales ◽  
Microwave Remote Sensing ◽  
In Situ Measurements ◽  
Spatial Density ◽  
Linear Regression Method ◽  
Spatiotemporal Resolution ◽  
Pixel Scale

In recent decades, microwave remote sensing (RS) has been used to measure soil moisture (SM). Long-term and large-scale RS SM datasets derived from various microwave sensors have been used in environmental fields. Understanding the accuracies of RS SM products is essential for their proper applications. However, due to the mismatched spatial scale between the ground-based and RS observations, the truth at the pixel scale may not be accurately represented by ground-based observations, especially when the spatial density of in situ measurements is low. Because ground-based observations are often sparsely distributed, temporal upscaling was adopted to transform a few in situ measurements into SM values at a pixel scale of 1 km by introducing the temperature vegetation dryness index (TVDI) related to SM. The upscaled SM showed high consistency with in situ SM observations and could accurately capture rainfall events. The upscaled SM was considered as the reference data to evaluate RS SM products at different spatial scales. In regard to the validation results, in addition to the correlation coefficient (R) of the Soil Moisture Active Passive (SMAP) SM being slightly lower than that of the Climate Change Initiative (CCI) SM, SMAP had the best performance in terms of the root-mean-square error (RMSE), unbiased RMSE and bias, followed by the CCI. The Soil Moisture and Ocean Salinity (SMOS) products were in worse agreement with the upscaled SM and were inferior to the R value of the X-band SM of the Advanced Microwave Scanning Radiometer 2 (AMSR2). In conclusion, in the study area, the SMAP and CCI SM are more reliable, although both products were underestimated by 0.060 cm3 cm−3 and 0.077 cm3 cm−3, respectively. If the biases are corrected, then the improved SMAP with an RMSE of 0.043 cm3 cm−3 and the CCI with an RMSE of 0.039 cm3 cm−3 will hopefully reach the application requirement for an accuracy with an RMSE less than 0.040 cm3 cm−3.

In Situ Tests on Creep Behavior of Rock Mass with Joint or Shearing Zone in Foundation of Large-Scale Hydroelectric Projects

2004 ◽  
Vol 261-263 ◽  
pp. 1097-1102 ◽  
Author(s):  
Jian Liu ◽  
Xia Ting Feng ◽  
Xiu Li Ding ◽  
Huo Ming Zhou
Keyword(s):  
Rock Mass ◽  
Creep Behavior ◽  
Large Scale ◽  
Time Dependent ◽  
Three Gorges Project ◽  
Three Gorges ◽  
The Three Gorges ◽  
The Three Gorges Project

The time-dependent behavior of rock mass, which is generally governed by joints and shearing zones, is of great significance for engineering design and prediction of long-term deformation and stability. In situ creep test is a more effective method than laboratory test in characterizing the creep behavior of rock mass with joint or shearing zone due to the complexity of field conditions. A series of in situ creep tests on granite with joint at the shiplock area of the Three-Gorges Project and basalt with shearing zone at the right abutment of the Xiluodu Project were performed in this study. Based on the test results, the stress-displacement-time responses of the joints and basalt are analyzed, and their time-dependent constitutive model and model coefficients are given, which is crucial for the design to prevent the creep deformations of rock masses from causing the failure of the operation of the shiplock gate at the Three-Gorges Project and long-term stability of the Xiluodu arc dam.

scholarly journals Use of satellite and modeled soil moisture data for predicting event soil loss at plot scale

2015 ◽  
Vol 19 (9) ◽  
pp. 3845-3856 ◽  
Author(s):  
F. Todisco ◽  
L. Brocca ◽  
L. F. Termite ◽  
W. Wagner
Keyword(s):  
Soil Moisture ◽  
Soil Loss ◽  
Large Scale ◽  
Central Italy ◽  
Remote Sensing Data ◽  
Loss Estimation ◽  
Balance Model ◽  
Estimation Accuracy ◽  
Soil Losses

Abstract. The potential of coupling soil moisture and a Universal Soil Loss Equation-based (USLE-based) model for event soil loss estimation at plot scale is carefully investigated at the Masse area, in central Italy. The derived model, named Soil Moisture for Erosion (SM4E), is applied by considering the unavailability of in situ soil moisture measurements, by using the data predicted by a soil water balance model (SWBM) and derived from satellite sensors, i.e., the Advanced SCATterometer (ASCAT). The soil loss estimation accuracy is validated using in situ measurements in which event observations at plot scale are available for the period 2008–2013. The results showed that including soil moisture observations in the event rainfall–runoff erosivity factor of the USLE enhances the capability of the model to account for variations in event soil losses, the soil moisture being an effective alternative to the estimated runoff, in the prediction of the event soil loss at Masse. The agreement between observed and estimated soil losses (through SM4E) is fairly satisfactory with a determination coefficient (log-scale) equal to ~ 0.35 and a root mean square error (RMSE) of ~ 2.8 Mg ha−1. These results are particularly significant for the operational estimation of soil losses. Indeed, currently, soil moisture is a relatively simple measurement at the field scale and remote sensing data are also widely available on a global scale. Through satellite data, there is the potential of applying the SM4E model for large-scale monitoring and quantification of the soil erosion process.

Sign in / Sign up

Export Citation Format

Share Document