scholarly journals Global soil moisture data derived through machine learning trained with in-situ measurements

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Sungmin O. ◽  
Rene Orth
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Large Scale ◽  
Short Term Memory ◽  
Temporal Dynamics ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Global Soil

AbstractWhile soil moisture information is essential for a wide range of hydrologic and climate applications, spatially-continuous soil moisture data is only available from satellite observations or model simulations. Here we present a global, long-term dataset of soil moisture derived through machine learning trained with in-situ measurements, SoMo.ml. We train a Long Short-Term Memory (LSTM) model to extrapolate daily soil moisture dynamics in space and in time, based on in-situ data collected from more than 1,000 stations across the globe. SoMo.ml provides multi-layer soil moisture data (0–10 cm, 10–30 cm, and 30–50 cm) at 0.25° spatial and daily temporal resolution over the period 2000–2019. The performance of the resulting dataset is evaluated through cross validation and inter-comparison with existing soil moisture datasets. SoMo.ml performs especially well in terms of temporal dynamics, making it particularly useful for applications requiring time-varying soil moisture, such as anomaly detection and memory analyses. SoMo.ml complements the existing suite of modelled and satellite-based datasets given its distinct derivation, to support large-scale hydrological, meteorological, and ecological analyses.

Augmenting satellite-derived soil moisture with multiple data streams using machine learning

2020 ◽  
Author(s):  
Rene Orth ◽  
Sungmin Oh
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Data Streams ◽  
Large Scale ◽  
Extreme Weather Events ◽  
Soil Moisture Data ◽  
Wide Range ◽  
Hydrological Hazards ◽  
Target Data

<p>Soil moisture plays a key role in land-atmosphere interactions through its influence on the energy and water cycles. Furthermore, its spatiotemporal variations can affect the development and persistence of extreme weather events. Consequently, soil moisture information is required for a wide range of research and applications, such as agricultural monitoring, flood and drought prediction, climate projection, and carbon-cycle modeling. Despite its scientific and societal importance, observations of soil moisture are sparse, in particular across time and at large spatial scales. Only models and satellite retrievals can provide global soil moisture information. While the ability of land surface models to represent the complex land-atmosphere interplay is still limited, satellite-based soil moisture data are a valuable alternative. However, these products suffer from a scaling based on models, and can only capture the top few centimeters of the soil. </p><p>In this study, we aim to augment satellite-based soil moisture data using machine learning. For this purpose we integrate satellite soil moisture with multiple hydro-meteorological data streams to derive global gridded soil moisture using Long Short-Term Memory (LSTM) neural networks. These networks are trained using in-situ soil moisture measurements as target data. With the resulting self-learned relationships, the LSTMs can produce in-situ-like soil moisture globally. We further analyze the implications of using point-scale target data to infer large scale information. The new dataset is derived separately for the surface and the deeper soil, thereby extending beyond the range covered by the satellite-based products. The integration of many data streams and multiple soil moisture observations through a powerful synergistic technique offers the potential to yield high accuracy. This is tested through rigorous cross-validation of the derived dataset. Finally, the planned datasets will permit consistent long-term, large-scale analysis to enhance our understanding of the hydrology-biosphere-climate interplay, to better constrain models and to support hydrological hazards monitoring and climate projections.</p>

scholarly journals Evaluating Machine Learning and Geostatistical Methods for Spatial Gap-filling of Monthly ESA CCI Soil Moisture in China

2021 ◽  
Vol 13 (14) ◽  
pp. 2848
Author(s):  
Hao Sun ◽  
Qian Xu
Keyword(s):  
Machine Learning ◽  
Soil Moisture ◽  
Water Resource Management ◽  
Large Scale ◽  
Gap Filling ◽  
Spatial Continuity ◽  
Study Results ◽  
Data Gaps

Obtaining large-scale, long-term, and spatial continuous soil moisture (SM) data is crucial for climate change, hydrology, and water resource management, etc. ESA CCI SM is such a large-scale and long-term SM (longer than 40 years until now). However, there exist data gaps, especially for the area of China, due to the limitations in remote sensing of SM such as complex topography, human-induced radio frequency interference (RFI), and vegetation disturbances, etc. The data gaps make the CCI SM data cannot achieve spatial continuity, which entails the study of gap-filling methods. In order to develop suitable methods to fill the gaps of CCI SM in the whole area of China, we compared typical Machine Learning (ML) methods, including Random Forest method (RF), Feedforward Neural Network method (FNN), and Generalized Linear Model (GLM) with a geostatistical method, i.e., Ordinary Kriging (OK) in this study. More than 30 years of passive–active combined CCI SM from 1982 to 2018 and other biophysical variables such as Normalized Difference Vegetation Index (NDVI), precipitation, air temperature, Digital Elevation Model (DEM), soil type, and in situ SM from International Soil Moisture Network (ISMN) were utilized in this study. Results indicated that: 1) the data gap of CCI SM is frequent in China, which is found not only in cold seasons and areas but also in warm seasons and areas. The ratio of gap pixel numbers to the whole pixel numbers can be greater than 80%, and its average is around 40%. 2) ML methods can fill the gaps of CCI SM all up. Among the ML methods, RF had the best performance in fitting the relationship between CCI SM and biophysical variables. 3) Over simulated gap areas, RF had a comparable performance with OK, and they outperformed the FNN and GLM methods greatly. 4) Over in situ SM networks, RF achieved better performance than the OK method. 5) We also explored various strategies for gap-filling CCI SM. Results demonstrated that the strategy of constructing a monthly model with one RF for simulating monthly average SM and another RF for simulating monthly SM disturbance achieved the best performance. Such strategy combining with the ML method such as the RF is suggested in this study for filling the gaps of CCI SM in China.

scholarly journals The International Soil Moisture Network: a data hosting facility for global in situ soil moisture measurements

2011 ◽  
Vol 8 (1) ◽  
pp. 1609-1663 ◽  
Author(s):  
W. A. Dorigo ◽  
W. Wagner ◽  
R. Hohensinn ◽  
S. Hahn ◽  
C. Paulik ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Land Surface ◽  
Water Cycle ◽  
Soil Moisture Data ◽  
The North ◽  
Time Period ◽  
Different Depths ◽  
Operational Networks

Abstract. In situ measurements of soil moisture are invaluable for calibrating and validating land surface models and satellite-based soil moisture retrievals. In addition, long-term time series of in situ soil moisture measurements themselves can reveal trends in the water cycle related to climate or land cover change. Nevertheless, on a worldwide basis the number of meteorological networks and stations measuring soil moisture, in particular on a continuous basis, is still limited and the data they provide lack standardization of technique and protocol. To overcome many of these limitations, the International Soil Moisture Network (ISMN; http://www.ipf.tuwien.ac.at/insitu) was initiated to serve as a centralized data hosting facility where globally available in situ soil moisture measurements from operational networks and validation campaigns are collected, harmonized, and made available to users. Data collecting networks share their soil moisture datasets with the ISMN on a voluntary and no-cost basis. Incoming soil moisture data are automatically transformed into common volumetric soil moisture units and checked for outliers and implausible values. Apart from soil water measurements from different depths, important metadata and meteorological variables (e.g., precipitation and soil temperature) are stored in the database. These will assist the user in correctly interpreting the soil moisture data. The database is queried through a graphical user interface while output of data selected for download is provided according to common standards for data and metadata. Currently (status January 2011), the ISMN contains data of 16 networks and more than 500 stations located in the North America, Europe, Asia, and Australia. The time period spanned by the entire database runs from 1952 until the present, although most datasets have originated during the last decade. The database is rapidly expanding, which means that both the number of stations and the time period covered by the existing stations are still growing. Hence, it will become an increasingly important resource for validating and improving satellite-derived soil moisture products and studying climate related trends. As the ISMN is animated by the scientific community itself, we invite potential networks to enrich the collection by sharing their in situ soil moisture data.

scholarly journals The International Soil Moisture Network: a data hosting facility for global in situ soil moisture measurements

2011 ◽  
Vol 15 (5) ◽  
pp. 1675-1698 ◽  
Author(s):  
W. A. Dorigo ◽  
W. Wagner ◽  
R. Hohensinn ◽  
S. Hahn ◽  
C. Paulik ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Land Surface ◽  
Water Cycle ◽  
Soil Moisture Data ◽  
Time Period ◽  
Surface Models ◽  
Different Depths ◽  
Operational Networks

Abstract. In situ measurements of soil moisture are invaluable for calibrating and validating land surface models and satellite-based soil moisture retrievals. In addition, long-term time series of in situ soil moisture measurements themselves can reveal trends in the water cycle related to climate or land cover change. Nevertheless, on a worldwide basis the number of meteorological networks and stations measuring soil moisture, in particular on a continuous basis, is still limited and the data they provide lack standardization of technique and protocol. To overcome many of these limitations, the International Soil Moisture Network (ISMN; http://www.ipf.tuwien.ac.at/insitu) was initiated to serve as a centralized data hosting facility where globally available in situ soil moisture measurements from operational networks and validation campaigns are collected, harmonized, and made available to users. Data collecting networks share their soil moisture datasets with the ISMN on a voluntary and no-cost basis. Incoming soil moisture data are automatically transformed into common volumetric soil moisture units and checked for outliers and implausible values. Apart from soil water measurements from different depths, important metadata and meteorological variables (e.g., precipitation and soil temperature) are stored in the database. These will assist the user in correctly interpreting the soil moisture data. The database is queried through a graphical user interface while output of data selected for download is provided according to common standards for data and metadata. Currently (status May 2011), the ISMN contains data of 19 networks and more than 500 stations located in North America, Europe, Asia, and Australia. The time period spanned by the entire database runs from 1952 until the present, although most datasets have originated during the last decade. The database is rapidly expanding, which means that both the number of stations and the time period covered by the existing stations are still growing. Hence, it will become an increasingly important resource for validating and improving satellite-derived soil moisture products and studying climate related trends. As the ISMN is animated by the scientific community itself, we invite potential networks to enrich the collection by sharing their in situ soil moisture data.

scholarly journals Combining satellite observations to develop a daily global soil moisture product for a wide range of applications

2015 ◽  
Vol 12 (11) ◽  
pp. 11549-11589 ◽  
Author(s):  
M. Enenkel ◽  
C. Reimer ◽  
W. Dorigo ◽  
W. Wagner ◽  
I. Pfeil ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Weather Forecasting ◽  
European Space Agency ◽  
Space Agency ◽  
Global Correlation ◽  
Wide Range ◽  
Global Soil ◽  
Agricultural Applications ◽  
The Individual

Abstract. The soil moisture dataset that is generated via the Climate Change Initiative (CCI) of the European Space Agency (ESA) (ESA CCI SM) is a popular research product. It is composed of observations from nine different satellites and aims to exploit the individual strengths of active (radar) and passive (radiometer) sensors, thereby providing surface soil moisture estimates at a spatial resolution of 0.25°. However, the annual updating cycle limits the use of the ESA CCI SM dataset for operational applications. Therefore, this study proposes an adaptation of the ESA CCI processing chain for daily global updates via satellite-derived near real-time (NRT) soil moisture observations. In order to extend the ESA CCI SM dataset from 1978 to present we use NRT observations from the Advanced SCATterometer on-board the MetOp satellites and the Advanced Microwave Scanning Radiometer 2 on-board GCOM-W. Since these NRT observations do not incorporate the latest algorithmic updates, parameter databases, and intercalibration efforts, by nature they offer a lower quality than reprocessed offline datasets. Our findings indicate that, despite issues in arid regions, the new "CCI NRT" dataset shows a good correlation with ESA CCI SM. The average global correlation coefficient between CCI NRT and ESA CCI SM (Pearson's R) is 0.8. An initial validation with 40 in-situ observations in France, Kenya, Senegal and Kenya yields an average R of 0.58 and 0.49 for ESA CCI SM and CCI NRT respectively. In summary, the CCI NRT dataset is getting ready for operational use, supporting applications such as drought and flood monitoring, weather forecasting or agricultural applications.

scholarly journals Mapping High Spatiotemporal-Resolution Soil Moisture by Upscaling Sparse Ground-Based Observations Using a Bayesian Linear Regression Method for Comparison with Microwave Remotely Sensed Soil Moisture Products

2021 ◽  
Vol 13 (2) ◽  
pp. 228
Author(s):  
Jian Kang ◽  
Rui Jin ◽  
Xin Li ◽  
Yang Zhang
Keyword(s):  
Soil Moisture ◽  
Large Scale ◽  
Spatial Scales ◽  
Microwave Remote Sensing ◽  
In Situ Measurements ◽  
Spatial Density ◽  
Linear Regression Method ◽  
Spatiotemporal Resolution ◽  
Pixel Scale

In recent decades, microwave remote sensing (RS) has been used to measure soil moisture (SM). Long-term and large-scale RS SM datasets derived from various microwave sensors have been used in environmental fields. Understanding the accuracies of RS SM products is essential for their proper applications. However, due to the mismatched spatial scale between the ground-based and RS observations, the truth at the pixel scale may not be accurately represented by ground-based observations, especially when the spatial density of in situ measurements is low. Because ground-based observations are often sparsely distributed, temporal upscaling was adopted to transform a few in situ measurements into SM values at a pixel scale of 1 km by introducing the temperature vegetation dryness index (TVDI) related to SM. The upscaled SM showed high consistency with in situ SM observations and could accurately capture rainfall events. The upscaled SM was considered as the reference data to evaluate RS SM products at different spatial scales. In regard to the validation results, in addition to the correlation coefficient (R) of the Soil Moisture Active Passive (SMAP) SM being slightly lower than that of the Climate Change Initiative (CCI) SM, SMAP had the best performance in terms of the root-mean-square error (RMSE), unbiased RMSE and bias, followed by the CCI. The Soil Moisture and Ocean Salinity (SMOS) products were in worse agreement with the upscaled SM and were inferior to the R value of the X-band SM of the Advanced Microwave Scanning Radiometer 2 (AMSR2). In conclusion, in the study area, the SMAP and CCI SM are more reliable, although both products were underestimated by 0.060 cm3 cm−3 and 0.077 cm3 cm−3, respectively. If the biases are corrected, then the improved SMAP with an RMSE of 0.043 cm3 cm−3 and the CCI with an RMSE of 0.039 cm3 cm−3 will hopefully reach the application requirement for an accuracy with an RMSE less than 0.040 cm3 cm−3.

In Situ Tests on Creep Behavior of Rock Mass with Joint or Shearing Zone in Foundation of Large-Scale Hydroelectric Projects

2004 ◽  
Vol 261-263 ◽  
pp. 1097-1102 ◽  
Author(s):  
Jian Liu ◽  
Xia Ting Feng ◽  
Xiu Li Ding ◽  
Huo Ming Zhou
Keyword(s):  
Rock Mass ◽  
Creep Behavior ◽  
Large Scale ◽  
Time Dependent ◽  
Three Gorges Project ◽  
Three Gorges ◽  
The Three Gorges ◽  
The Three Gorges Project

The time-dependent behavior of rock mass, which is generally governed by joints and shearing zones, is of great significance for engineering design and prediction of long-term deformation and stability. In situ creep test is a more effective method than laboratory test in characterizing the creep behavior of rock mass with joint or shearing zone due to the complexity of field conditions. A series of in situ creep tests on granite with joint at the shiplock area of the Three-Gorges Project and basalt with shearing zone at the right abutment of the Xiluodu Project were performed in this study. Based on the test results, the stress-displacement-time responses of the joints and basalt are analyzed, and their time-dependent constitutive model and model coefficients are given, which is crucial for the design to prevent the creep deformations of rock masses from causing the failure of the operation of the shiplock gate at the Three-Gorges Project and long-term stability of the Xiluodu arc dam.

scholarly journals Two Stage Continuous Gesture Recognition Based on Deep Learning

Electronics ◽  
2021 ◽  
Vol 10 (5) ◽  
pp. 534
Author(s):  
Huogen Wang
Keyword(s):  
Gesture Recognition ◽  
Large Scale ◽  
Short Term Memory ◽  
Short Term ◽  
Hand Motion ◽  
Spatiotemporal Features ◽  
Spatiotemporal Information ◽  
Video Frames ◽  
Depth Sequences

The paper proposes an effective continuous gesture recognition method, which includes two modules: segmentation and recognition. In the segmentation module, the video frames are divided into gesture frames and transitional frames by using the information of hand motion and appearance, and continuous gesture sequences are segmented into isolated sequences. In the recognition module, our method exploits the spatiotemporal information embedded in RGB and depth sequences. For the RGB modality, our method adopts Convolutional Long Short-Term Memory Networks to learn long-term spatiotemporal features from short-term spatiotemporal features obtained from a 3D convolutional neural network. For the depth modality, our method converts a sequence into Dynamic Images and Motion Dynamic Images through weighted rank pooling and feed them into Convolutional Neural Networks, respectively. Our method has been evaluated on both ChaLearn LAP Large-scale Continuous Gesture Dataset and Montalbano Gesture Dataset and achieved state-of-the-art performance.

scholarly journals Use of satellite and modeled soil moisture data for predicting event soil loss at plot scale

2015 ◽  
Vol 19 (9) ◽  
pp. 3845-3856 ◽  
Author(s):  
F. Todisco ◽  
L. Brocca ◽  
L. F. Termite ◽  
W. Wagner
Keyword(s):  
Soil Moisture ◽  
Soil Loss ◽  
Large Scale ◽  
Central Italy ◽  
Remote Sensing Data ◽  
Loss Estimation ◽  
Balance Model ◽  
Estimation Accuracy ◽  
Soil Losses

Abstract. The potential of coupling soil moisture and a Universal Soil Loss Equation-based (USLE-based) model for event soil loss estimation at plot scale is carefully investigated at the Masse area, in central Italy. The derived model, named Soil Moisture for Erosion (SM4E), is applied by considering the unavailability of in situ soil moisture measurements, by using the data predicted by a soil water balance model (SWBM) and derived from satellite sensors, i.e., the Advanced SCATterometer (ASCAT). The soil loss estimation accuracy is validated using in situ measurements in which event observations at plot scale are available for the period 2008–2013. The results showed that including soil moisture observations in the event rainfall–runoff erosivity factor of the USLE enhances the capability of the model to account for variations in event soil losses, the soil moisture being an effective alternative to the estimated runoff, in the prediction of the event soil loss at Masse. The agreement between observed and estimated soil losses (through SM4E) is fairly satisfactory with a determination coefficient (log-scale) equal to ~ 0.35 and a root mean square error (RMSE) of ~ 2.8 Mg ha−1. These results are particularly significant for the operational estimation of soil losses. Indeed, currently, soil moisture is a relatively simple measurement at the field scale and remote sensing data are also widely available on a global scale. Through satellite data, there is the potential of applying the SM4E model for large-scale monitoring and quantification of the soil erosion process.

scholarly journals NOAA Satellite Soil Moisture Operational Product System (SMOPS) Version 3.0 Generates Higher Accuracy Blended Satellite Soil Moisture

2020 ◽  
Vol 12 (17) ◽  
pp. 2861
Author(s):  
Jifu Yin ◽  
Xiwu Zhan ◽  
Jicheng Liu
Keyword(s):  
Soil Moisture ◽  
Real Time ◽  
Land Surface ◽  
Vital Role ◽  
Product System ◽  
Global Soil ◽  
One Stop ◽  
Satellite Sensors ◽  
Blended Data

Soil moisture plays a vital role for the understanding of hydrological, meteorological, and climatological land surface processes. To meet the need of real time global soil moisture datasets, a Soil Moisture Operational Product System (SMOPS) has been developed at National Oceanic and Atmospheric Administration to produce a one-stop shop for soil moisture observations from all available satellite sensors. What makes the SMOPS unique is its near real time global blended soil moisture product. Since the first version SMOPS publicly released in 2010, the SMOPS has been updated twice based on the users’ feedbacks through improving retrieval algorithms and including observations from new satellite sensors. The version 3.0 SMOPS has been operationally released since 2017. Significant differences in climatological averages lead to remarkable distinctions in data quality between the newest and the older versions of SMOPS blended soil moisture products. This study reveals that the SMOPS version 3.0 has overwhelming advantages of reduced data uncertainties and increased correlations with respect to the quality controlled in situ measurements. The new version SMOPS also presents more robust agreements with the European Space Agency’s Climate Change Initiative (ESA_CCI) soil moisture datasets. With the higher accuracy, the blended data product from the new version SMOPS is expected to benefit the hydrological, meteorological, and climatological researches, as well as numerical weather, climate, and water prediction operations.

Sign in / Sign up

Export Citation Format

Share Document