Gap Filling of High-Resolution Soil Moisture for SMAP/Sentinel-1: A Two-Layer Machine Learning-Based Framework

Obtaining large-scale, long-term, and spatial continuous soil moisture (SM) data is crucial for climate change, hydrology, and water resource management, etc. ESA CCI SM is such a large-scale and long-term SM (longer than 40 years until now). However, there exist data gaps, especially for the area of China, due to the limitations in remote sensing of SM such as complex topography, human-induced radio frequency interference (RFI), and vegetation disturbances, etc. The data gaps make the CCI SM data cannot achieve spatial continuity, which entails the study of gap-filling methods. In order to develop suitable methods to fill the gaps of CCI SM in the whole area of China, we compared typical Machine Learning (ML) methods, including Random Forest method (RF), Feedforward Neural Network method (FNN), and Generalized Linear Model (GLM) with a geostatistical method, i.e., Ordinary Kriging (OK) in this study. More than 30 years of passive–active combined CCI SM from 1982 to 2018 and other biophysical variables such as Normalized Difference Vegetation Index (NDVI), precipitation, air temperature, Digital Elevation Model (DEM), soil type, and in situ SM from International Soil Moisture Network (ISMN) were utilized in this study. Results indicated that: 1) the data gap of CCI SM is frequent in China, which is found not only in cold seasons and areas but also in warm seasons and areas. The ratio of gap pixel numbers to the whole pixel numbers can be greater than 80%, and its average is around 40%. 2) ML methods can fill the gaps of CCI SM all up. Among the ML methods, RF had the best performance in fitting the relationship between CCI SM and biophysical variables. 3) Over simulated gap areas, RF had a comparable performance with OK, and they outperformed the FNN and GLM methods greatly. 4) Over in situ SM networks, RF achieved better performance than the OK method. 5) We also explored various strategies for gap-filling CCI SM. Results demonstrated that the strategy of constructing a monthly model with one RF for simulating monthly average SM and another RF for simulating monthly SM disturbance achieved the best performance. Such strategy combining with the ML method such as the RF is suggested in this study for filling the gaps of CCI SM in China.

Download Full-text

Generating high-resolution daily soil moisture by using spatial downscaling techniques: a comparison of six machine learning algorithms

Advances in Water Resources ◽

10.1016/j.advwatres.2020.103601 ◽

2020 ◽

Vol 141 ◽

pp. 103601 ◽

Cited By ~ 3

Author(s):

Yangxiaoyue Liu ◽

Wenlong Jing ◽

Qi Wang ◽

Xiaolin Xia

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

High Resolution ◽

Learning Algorithms ◽

Machine Learning Algorithms ◽

Spatial Downscaling

Download Full-text

Spatial gap-filling of SMAP soil moisture pixels over Tibetan Plateau via machine learning versus geostatistics

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing ◽

10.1109/jstars.2021.3112623 ◽

2021 ◽

pp. 1-1

Author(s):

Cheng Tong ◽

Hongquan Wang ◽

Ramata Magagi ◽

Kalifa Goita ◽

Ke Wang

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

Tibetan Plateau ◽

Gap Filling

Download Full-text

Advances in Soil Moisture Retrieval from Multispectral Remote Sensing Using Unmanned Aircraft Systems and Machine Learning Techniques

10.5194/hess-2020-271 ◽

2020 ◽

Author(s):

Samuel N. Araya ◽

Anna Fryjoff-Hung ◽

Andreas Anderson ◽

Joshua H. Viers ◽

Teamrat A. Ghezzehei

Keyword(s):

Machine Learning ◽

Remote Sensing ◽

Soil Moisture ◽

High Resolution ◽

Environmental Modeling ◽

Unmanned Aircraft ◽

Machine Learning Algorithms ◽

Machine Learning Techniques ◽

Boosted Regression Tree ◽

Terrain Attributes

Abstract. We developed machine learning models to retrieve surface soil moisture (0–4 cm) from high resolution multispectral imagery using terrain attributes and local climate covariates. Using a small unmanned aircraft system (UAS) equipped with a multispectral sensor we captured high resolution imagery in part to create a high-resolution digital elevation model (DEM) as well as quantify relative vegetation photosynthetic status. We tested four different machine learning algorithms. The boosted regression tree algorithm gave the best prediction with mean absolute error of 3.8 % volumetric water content. The most important variables for the prediction of soil moisture were precipitation, reflectance in the red wavelengths, potential evapotranspiration, and topographic position indices (TPI). Our results demonstrate that the dynamics of soil water status across heterogeneous terrain may be adequately described and predicted by UAS remote sensing data and machine learning. Our modeling approach and the variable importance and relationships we have assessed in this study should be useful for management and environmental modeling tasks where spatially explicit soil moisture information is important.

Download Full-text

Soil moisture estimation from high-resolution UASs imagery based on machine learning approaches for land cover classification

10.5194/egusphere-egu21-15586 ◽

2021 ◽

Author(s):

Teresa Pizzolla ◽

Silvano Fortunato Dal Sasso ◽

Ruodan Zhuang ◽

Alonso Pizarro ◽

Salvatore Manfreda

Keyword(s):

Machine Learning ◽

Remote Sensing ◽

Soil Moisture ◽

High Resolution ◽

Thermal Inertia ◽

Bare Soil ◽

Unmanned Aerial Systems ◽

Learning Approaches ◽

Land Use Classification ◽

Aerial Systems

Soil moisture (SM) is an essential variable in the earth system as it influences water, energy and, carbon fluxes between the land surface and the atmosphere. The SM spatio-temporal variability requires detailed analyses, high-definition optics and fast computing approaches for near real-time SM estimation at different spatial scales. Remote Sensing-based Unmanned Aerial Systems (UASs) represents the actual solution providing low-cost approaches to meet the requirements of spatial, spectral and temporal resolutions [1; 3; 4]. In this context, a proper land use classification is crucial in order to discriminate the behaviors of vegetation and bare soil in such high-resolution imagery. Therefore, high-resolution UASs-based imagery requires a specific images classification approach also considering the illumination conditions. In this work, the land use classification was carried out using a methodology based on a combined machine learning approaches: k-means clustering algorithm for removing shadow pixels from UASs images and, binary classifier for vegetation filtering. This approach led to identifying the bare soil on which SM estimation was computed using the Apparent Thermal Inertia (ATI) method [2]. The estimated SM values were compared with field measurements obtaining a good correlation (R2 = 0.80). The accuracy of the results shows good reliability of the procedure and allows extending the use of UASs also in unclassified areas and ungauged basins, where the monitoring of the SM is very complex.References[1] Manfreda, S., McCabe, M.F., Miller, P.E., Lucas, R., Pajuelo Madrigal, V., Mallinis, G., Ben Dor, E., Helman, D., Estes, L., Ciraolo, G., et al. On the Use of Unmanned Aerial Systems for Environmental Monitoring, Remote Sensing, 2018, 10, 641.[2] Minacapilli, M., Cammalleri, C., Ciraolo, G., D&#8217;Asaro, F., Iovino, M., and Maltese, A. Thermal Inertia Modeling for Soil Surface Water Content Estimation: A Laboratory Experiment. Soil. Sci. Soc. Amer. J. 2012, vol.76, n.1, pp. 92&#8211;100[3] Paruta, A., P. Nasta, G. Ciraolo, F. Capodici, S. Manfreda, N. Romano, E. Bendor, Y. Zeng, A. Maltese, S. F. Dal Sasso and R. Zhuang, A geostatistical approach to map near-surface soil moisture through hyper-spatial resolution thermal inertia, IEEE Transactions on Geoscience and Remote Sensing, 2020.[4] Petropoulos, G.P., A. Maltese, T. N. Carlson, G. Provenzano, A. Pavlides, G. Ciraolo, D. Hristopulos, F. Capodici, C. Chalkias, G. Dardanelli, S. Manfreda, Exploring the use of UAVs with the simplified &#8220;triangle&#8221; technique for Soil Water Content and Evaporative Fraction retrievals in a Mediterranean setting, International Journal of Remote Sensing, 2020.

Download Full-text

Soil moisture mapping at high resolution by merging SMAP, Sentinel1 and COSMO SkyMed with the support of machine learning

10.1117/12.2601845 ◽

2021 ◽

Author(s):

Emanuele Santi ◽

Fabrizio Baroni ◽

Giacomo Fontanelli ◽

Alessandro Lapini ◽

Simonetta Paloscia ◽

...

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

High Resolution

Download Full-text

Performance of a Physically Based Gap-Filling Technique of in-situ Soil Moisture, in Comparison with Machine Learning

10.5194/egusphere-egu2020-13671 ◽

2020 ◽

Author(s):

Seulchan Lee ◽

Hyunho Jeon ◽

Jongmin Park ◽

Minha Choi

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

Low Voltage ◽

Time Domain Reflectometry ◽

Limited Information ◽

Gap Filling ◽

Flux Tower ◽

Physically Based ◽

Filling Method

As the importance of Soil Moisture (SM) has been recognized in various fields, including agricultural practices, natural hazards, and climate predictions, ground-based SM sensors such as Frequency Domain Reflectometry (FDR), Time Domain Reflectometry (TDR) are being widely used. However, gaps in in-situ SM data are still unavoidable due not only to sensor failure or low voltage supply, but to environmental conditions. Since it is essential to acquire accurate and continuous SM data for its application purpose, the gaps in the data should be handled properly. In this study, we propose a physically based gap-filling method in a mountainous region, in which in-situ SM measurements and flux tower are located. This method is developed only with in-situ SM and precipitation data, by considering variation characteristics of SM: increases rapidly with precipitation and decreases asymptotically afterward. SM data from the past is used to build Look-Up-Tables (LUTs) that contains the amount and speed of increment and decrement of SM, with and without precipitation, respectively. Based on the developed LUTs, the gaps are filled successively from where the gaps started. At the same time, we also introduce a machine learning-based gap-filling framework for the comparison. Ancillary data from the flux tower (e.g. net radiation, relative humidity) was used as input for training, with the same period as in the physically based method. The trained models are then used to fill the gaps. We found that both proposed methods are able to fill the gaps of in-situ SM reasonably, with capabilities to capture the characteristics of SM variation. Results from the comparison indicate that the physically based gap-filling method is very accurate and efficient when there&#8217;s limited information, and also suitable to be used for prediction purposes.

Download Full-text

Use of multiple LIDAR-derived digital terrain indices and machine learning for high-resolution national-scale soil moisture mapping of the Swedish forest landscape

Geoderma ◽

10.1016/j.geoderma.2021.115280 ◽

2021 ◽

Vol 404 ◽

pp. 115280

Author(s):

Anneli M. Ågren ◽

Johannes Larson ◽

Siddhartho Shekhar Paul ◽

Hjalmar Laudon ◽

William Lidberg

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

High Resolution ◽

Forest Landscape ◽

National Scale ◽

Digital Terrain ◽

Terrain Indices

Download Full-text

Using machine learning to generate high-resolution soil wetness maps for planning forest management

10.5194/egusphere-egu21-2578 ◽

2021 ◽

Author(s):

William Lidberg ◽

Johannes Larson ◽

siddhartho Paul ◽

Hjalmar Laudon ◽

Anneli Ågren

Keyword(s):

Machine Learning ◽

Soil Moisture ◽

High Resolution ◽

Field Data ◽

Airborne Lidar ◽

Aerial Photographs ◽

Riparian Zones ◽

Gradient Boosting ◽

Extreme Gradient Boosting ◽

Boreal Landscape

Open peatlands are a recognizable feature in the boreal landscape that are commonly mapped from aerial photographs. However, wet soils also occur on tree covered peatlands and in the riparian zones of forest streams and surrounding lakes. Comparisons between field data and available maps show that only 36 % of wet soils in the boreal landscape are marked on maps, making them difficult to manage. Wet soils have lower bearing capacity than dry soils and are more susceptible to soil disturbance from land-use management with heavy machinery. Topographical modelling of wet area indices has been suggested as a solution to this problem and high-resolution digital elevation models (DEM) derived from airborne LiDAR are becoming accessible in many countries. However, most of these topographical methods relies on the user to define appropriate threshold values in order to define wet areas. Soil textures, topography and climatic differences make any application difficult on a large scale. This complex landscape variability can be captured by utilizing machine learners that uses automated data mining methods to discover patterns in large data sets. By using soil moisture data from 20&#160;000 field plots from the National Forest Inventory of Sweden, we combined information from 24 indices and ancillary environmental features using a machine learning known as extreme gradient boosting. Extreme gradient boosting used the field data to learn how to classify soil moisture and delivered high performance compared to many traditional single algorithm methods. With this method we mapped soil moisture at 2 m spatial resolution across the Swedish forest landscape in five days using a workstation with 32 cores. This new map captured 79 % (kappa 0.69) of all wet soils compared to only 36 % (kappa 0.39) captured by current maps. In addition to capture open wetlands this new map also capture riparian zones and previously unmapped cryptic wetlands underneath the forest canopy. The new maps can, for example, be used to plan hydrologically adapted buffer zones, suggest machine free zones near streams and lakes in order to prevent rutting from forestry machines to reduce sediment, mercury and nutrient loads to downstream streams, lakes and sea.

Download Full-text