scholarly journals Developing the Swiss mid-infrared soil spectral library for local estimation and monitoring

SOIL ◽  
2021 ◽  
Vol 7 (2) ◽  
pp. 525-546
Author(s):  
Philipp Baumann ◽  
Anatol Helfenstein ◽  
Andreas Gubler ◽  
Armin Keller ◽  
Reto Giulio Meuli ◽  
...  
Keyword(s):  
Soil Properties ◽  
Chemical Diversity ◽  
Total Carbon ◽  
Data Driven ◽  
Organic Soils ◽  
Spectral Library ◽  
Soil Monitoring ◽  
Mid Infrared ◽  
Measured Range

Abstract. Information on soils' composition and physical, chemical and biological properties is paramount to elucidate agroecosystem functioning in space and over time. For this purpose, we developed a national Swiss soil spectral library (SSL; n=4374) in the mid-infrared (mid-IR), calibrating 16 properties from legacy measurements on soils from the Swiss Biodiversity Monitoring program (BDM; n=3778; 1094 sites) and the Swiss long-term Soil Monitoring Network (NABO; n=596; 71 sites). General models were trained with the interpretable rule-based learner CUBIST, testing combinations of {5,10,20,50, and 100} ensembles of rules (committees) and {2, 5, 7, and 9} nearest neighbors used for local averaging with repeated 10-fold cross-validation grouped by location. To evaluate the information in spectra to facilitate long-term soil monitoring at a plot level, we conducted 71 model transfers for the NABO sites to induce locally relevant information from the SSL, using the data-driven sample selection method RS-LOCAL. In total, 10 soil properties were estimated with discrimination capacity suitable for screening (R2≥0.72; ratio of performance to interquartile distance (RPIQ) ≥ 2.0), out of which total carbon (C), organic C (OC), total nitrogen (N), pH and clay showed accuracy eligible for accurate diagnostics (R2>0.8; RPIQ ≥ 3.0). CUBIST and the spectra estimated total C accurately with the root mean square error (RMSE) = 8.4 g kg−1 and the RPIQ = 4.3, while the measured range was 1–583 g kg−1 and OC with RMSE = 9.3 g kg−1 and RPIQ = 3.4 (measured range 0–583 g kg−1). Compared to the general statistical learning approach, the local transfer approach – using two respective training samples – on average reduced the RMSE of total C per site fourfold. We found that the selected SSL subsets were highly dissimilar compared to validation samples, in terms of both their spectral input space and the measured values. This suggests that data-driven selection with RS-LOCAL leverages chemical diversity in composition rather than similarity. Our results suggest that mid-IR soil estimates were sufficiently accurate to support many soil applications that require a large volume of input data, such as precision agriculture, soil C accounting and monitoring and digital soil mapping. This SSL can be updated continuously, for example, with samples from deeper profiles and organic soils, so that the measurement of key soil properties becomes even more accurate and efficient in the near future.

scholarly journals Developing the Swiss soil spectral library for local estimation and monitoring

2021 ◽  
Author(s):  
Philipp Baumann ◽  
Anatol Helfenstein ◽  
Andreas Gubler ◽  
Armin Keller ◽  
Reto Giulio Meuli ◽  
...  
Keyword(s):  
Soil Properties ◽  
Chemical Diversity ◽  
Total Carbon ◽  
Data Driven ◽  
Organic Soils ◽  
Spectral Library ◽  
Total N ◽  
Organic C ◽  
Measured Range

Abstract. Information on soils' composition and physical, chemical and biological properties is paramount to elucidate agroecosystem functioning in space and over time. For this purposes we developed a national Swiss soil spectral library (SSL; n = 4374) in the mid-infrared (mid-IR), calibrating 17 properties from legacy measurements on soils from the Swiss biodiversity monitoring program (n = 3778; 1094 sites) and the Swiss long-term monitoring network (n = 596; 71 sites). General models were trained with the interpretable rule-based learner CUBIST, testing combinations of {5, 10, 20, 50, 100} committees of rules and {2, 5, 7, 9} neighbors to localize predictions with repeated by location grouped ten-fold cross-validation. To evaluate the information in spectra to facilitate long-term soil monitoring at a plot-level, we conducted 71 model transfers for the NABO sites to induce locally relevant information from the SSL, using the data-driven sample selection method rs-local. Eleven soil properties were estimated with discrimination capacity suitable for screening (R2 > 0.6), out of which total carbon (C), organic C (OC), total N, organic matter content, pH, and clay showed accuracy eligible for accurate diagnostics (R2 > 0.8). Cubist and the spectra estimated total C accurately with RMSE = 0.84 % while the measured range was 0.1–⁠58.3 %, and OC with RMSE = 1.20 % (measured range 0.0–⁠27.3 %). Compared to general estimates of properties from Cubist, local modeling on average reduced the root mean square error of total C per site fourfold. We found that the selected SSL subsets were highly dissimilar in terms of both their spectral input space and the measured values. This suggests that data-driven selection with RS-LOCAL leverages chemical diversity in composition rather than similarity. Our results suggest that mid-IR soil estimates were sufficiently accurate to support many soil applications that require a large volume of input data, such as precision agriculture, soil C accounting and monitoring, and digital soil mapping. This SSL can be updated continuously, for example with samples from deeper profiles and organic soils, so that the measurement of key soil properties becomes even more accurate and efficient in the near future.

scholarly journals Predicting soil carbon by efficiently using variation in a mid-IR soil spectral library

2021 ◽  
Author(s):  
Anatol Helfenstein ◽  
Philipp Baumann ◽  
Raphael Viscarra Rossel ◽  
Andreas Gubler ◽  
Stefan Oechslin ◽  
...  
Keyword(s):  
Soil Carbon ◽  
Scale Up ◽  
Monitoring Program ◽  
Partial Least Square ◽  
Least Square ◽  
Total Carbon ◽  
Local Algorithm ◽  
Organic Soils ◽  
Infrared Range ◽  
Spectral Library

Abstract. Traditional laboratory methods of acquiring soil information remain important for assessing key soil properties, soil functions and ecosystem services over space and time. Infrared spectroscopic modelling can link and massively scale up these methods for many soil characteristics in a cost-effective and timely manner. In Switzerland, only 10 % to 15 % of agricultural soils have been mapped sufficiently to serve spatial decision support systems, presenting an urgent need for rapid quantitative soil characterization. The current Swiss soil spectral library (SSL; n = 4374) in the mid-infrared range includes soil samples from the Biodiversity Monitoring Program (BDM), arranged in a regularly spaced grid across Switzerland, and temporally-resolved data from the Swiss Soil Monitoring Network (NABO). Given the relatively low representation of organic soils and their organo-mineral diversity in the SSL, we aimed to develop both an efficient calibration sampling scheme and accurate modelling strategy to estimate soil carbon (SC) contents of heterogeneous samples between 0 m to 2 m depth from 26 locations within two drained peatland regions (HAFL dataset; n = 116). The focus was on minimizing the need for new reference analyses by efficiently mining the spectral information of SSL instances and their target-feature representations. We used partial least square regressions (PLSR) together with a 5 times repeated, grouped by location, 10-fold cross validation (CV) to predict SC ranging from 1 % to 52 % in the local HAFL dataset. We compared the validation performance of different calibration schemes involving local models (1), models using the entire SSL spiked with local samples (2) and 15 subsets of local and SSL samples using the RS-LOCAL algorithm (3). Using local and RS-LOCAL calibrations with at least 5 local samples, we achieved similar validation results for predictions of SC up to 52 % (R2 = 0.94–0.96, bias = −0.6–1.5, RMSE = 2.6 % to 3.5 % total carbon). However, calibrations of representative SSL and local samples using RS-LOCAL only required 5 local samples for very accurate models (RMSE = 2.9 % total carbon), while local calibrations required 50 samples for similarly accurate results (RMSE 

scholarly journals Soil properties after 36 years of N fertilization under continuous corn and corn-soybean management

2021 ◽  
Author(s):  
Nakian Kim ◽  
Gevan D. Behnke ◽  
María B. Villamil
Keyword(s):  
Soil Properties ◽  
Crop Rotation ◽  
Crop Yields ◽  
Inorganic Nitrogen ◽  
Block Design ◽  
N Fertilization ◽  
Total Carbon ◽  
Nitrous Oxide Emissions ◽  
Continuous Corn

Abstract. Modern agricultural systems rely on inorganic nitrogen (N) fertilization to enhance crop yields, but its overuse may negatively affect soil properties. Our objective was to investigate the effect of long-term N fertilization on key soil properties under continuous corn [Zea mays L.] (CCC) and both the corn (Cs) and soybean [Glycine max L. Merr.] (Sc) phases of a corn-soybean rotation. Research plots were established in 1981 with treatments arranged as a split-plot design in a randomized complete block design with three replications. The main plot was crop rotation (CCC, Cs, and Sc), and the subplots were N fertilizer rates of 0 kg N ha−1 (N0, controls), and 202 kg N ha−1, and 269 kg N ha−1 (N202, and N269, respectively). After 36 years and within the CCC, the yearly addition of N269 compared to unfertilized controls significantly increased cation exchange capacity (CEC, 65 % higher under N269) and acidified the top 15 cm of the soil (pH 4.8 vs. pH 6.5). Soil organic matter (SOM) and total carbon stocks (TCs) were not affected by treatments, yet water aggregate stability (WAS) decreased by 6.7 % within the soybean phase of the CS rotation compared to CCC. Soil bulk density (BD) decreased with increased fertilization by 5 % from N0 to N269. Although ammonium (NH4+) did not differ by treatments, nitrate (NO3−) increased eight-fold with N269 compared to N0, implying increased nitrification. Soils of unfertilized controls under CCC have over twice the available phosphorus level (P) and 40 % more potassium (K) than the soils of fertilized plots (N202 and N269). On average, corn yields increased 60 % with N fertilization compared to N0. Likewise, under N0, rotated corn yielded 45 % more than CCC; the addition of N (N202 and N269) decreased the crop rotation benefit to 17 %. Our results indicated that due to the increased level of corn residues returned to the soil in fertilized systems, long-term N fertilization improved WAS and BD, yet not SOM, at the cost of significant soil acidification and greater risk of N leaching and increased nitrous oxide emissions.

scholarly journals Estimation of soil properties with mid-infrared soil spectroscopy across yam production landscapes in West Africa

2021 ◽  
Author(s):  
Philipp Baumann ◽  
Juhwan Lee ◽  
Emmanuel Frossard ◽  
Laurie Paule Schönholzer ◽  
Lucien Diby ◽  
...  
Keyword(s):  
West Africa ◽  
Soil Fertility ◽  
Soil Properties ◽  
Regional Scale ◽  
Partial Least Square ◽  
Least Square ◽  
Partial Least Square Regression ◽  
Total Carbon ◽  
Humid Forest ◽  
Mid Infrared

Abstract. Low soil fertility is challenging the sustainable production of staple crops in the yam belt of West Africa. Quantitative soil measures are needed to assess soil fertility decline and to improve crop fertilization management in the region. We developed and tested a mid-infrared (mid-IR) soil spectral library to enable timely and cost-efficient assessments of soil properties. Our collection included 80 soil samples from four landscapes (10 km × 10 km) and 20 fields/landscape across a gradient from humid forest to savanna, and 14 additional samples from one landscape that had been sampled within the Land Health Degradation Framework. We derived partial least square regression models to estimate soil properties with spectra.The models produced accurate cross-validated estimates of total carbon, total nitrogen, total sulfur, total iron, total aluminum, total potassium, total calcium, exchangeable calcium, effective cation exchange capacity, diethylenetriaminepentaacetic acid (DTPA) extractable iron and clay content (R2 > 0.75). The estimates of total zinc, pH, exchangeable magnesium, bioavailable copper and manganese were less predictable (R2 > 0.50). Our results confirm that mid-IR spectroscopy is a reliable and quick method assess the regional-scale variation in most soil properties, especially the ones closely associated with soil organic matter. Although the relatively small mid-IR library shows satisfactory performance, we expect that frequent but small model updates will be needed to adapt the library to the variation of soil quality attributes within individual fields in the regions and their temporal fluctuations.

scholarly journals Is Standardization Necessary for Sharing of a Large Mid-Infrared Soil Spectral Library?

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6729
Author(s):  
Shree R. S. Dangal ◽  
Jonathan Sanderman
Keyword(s):  
Soil Properties ◽  
Predictive Models ◽  
Soil Samples ◽  
Soil Survey ◽  
Least Squares Regression ◽  
Spectral Library ◽  
Calibration Transfer ◽  
Mid Infrared ◽  
Calibration Models ◽  
Spectral Libraries

Recent developments in diffuse reflectance soil spectroscopy have increasingly focused on building and using large soil spectral libraries with the purpose of supporting many activities relevant to monitoring, mapping and managing soil resources. A potential limitation of using a mid-infrared (MIR) spectral library developed by another laboratory is the need to account for inherent differences in the signal strength at each wavelength associated with different instrumental and environmental conditions. Here we apply predictive models built using the USDA National Soil Survey Center–Kellogg Soil Survey Laboratory (NSSC-KSSL) MIR spectral library (n = 56,155) to samples sets of European and US origin scanned on a secondary spectrometer to assess the need for calibration transfer using a piecewise direct standardization (PDS) approach in transforming spectra before predicting carbon cycle relevant soil properties (bulk density, CaCO3, organic carbon, clay and pH). The European soil samples were from the land use/cover area frame statistical survey (LUCAS) database available through the European Soil Data Center (ESDAC), while the US soil samples were from the National Ecological Observatory Network (NEON). Additionally, the performance of the predictive models on PDS transfer spectra was tested against the direct calibration models built using samples scanned on the secondary spectrometer. On independent test sets of European and US origin, PDS improved predictions for most but not all soil properties with memory based learning (MBL) models generally outperforming partial least squares regression and Cubist models. Our study suggests that while good-to-excellent results can be obtained without calibration transfer, for most of the cases presented in this study, PDS was necessary for unbiased predictions. The MBL models also outperformed the direct calibration models for most of the soil properties. For laboratories building new spectroscopy capacity utilizing existing spectral libraries, it appears necessary to develop calibration transfer using PDS or other calibration transfer techniques to obtain the least biased and most precise predictions of different soil properties.

scholarly journals Accurate and Precise Prediction of Soil Properties from a Large Mid-Infrared Spectral Library

Soil Systems ◽  
2019 ◽  
Vol 3 (1) ◽  
pp. 11 ◽  
Author(s):  
Shree Dangal ◽  
Jonathan Sanderman ◽  
Skye Wills ◽  
Leonardo Ramirez-Lopez
Keyword(s):  
Soil Properties ◽  
Diffuse Reflectance Spectroscopy ◽  
Cost Effective ◽  
Machine Learning Algorithms ◽  
Soil Survey ◽  
Spectral Library ◽  
Mid Infrared ◽  
Large N ◽  
Infrared Spectral ◽  
Precise Prediction

Diffuse reflectance spectroscopy (DRS) is emerging as a rapid and cost-effective alternative to routine laboratory analysis for many soil properties. However, it has primarily been applied in project-specific contexts. Here, we provide an assessment of DRS spectroscopy at the scale of the continental United States by utilizing the large (n > 50,000) USDA National Soil Survey Center mid-infrared spectral library and associated soil characterization database. We tested and optimized several advanced statistical approaches for providing routine predictions of numerous soil properties relevant to studying carbon cycling. On independent validation sets, the machine learning algorithms Cubist and memory-based learner (MBL) both outperformed random forest (RF) and partial least squares regressions (PLSR) and produced excellent overall models with a mean R2 of 0.92 (mean ratio of performance to deviation = 6.5) across all 10 soil properties. We found that the use of root-mean-square error (RMSE) was misleading for understanding the actual uncertainty about any particular prediction; therefore, we developed routines to assess the prediction uncertainty for all models except Cubist. The MBL models produced much more precise predictions compared with global PLSR and RF. Finally, we present several techniques that can be used to flag predictions of new samples that may not be reliable because their spectra fall outside of the calibration set.

Development of a Swiss National Soil Spectral Model Library using data-driven modeling

2020 ◽  
Author(s):  
Philipp Baumann ◽  
Anatol Helfenstein ◽  
Andreas Gubler ◽  
Reto Meuli ◽  
Armin Keller ◽  
...  
Keyword(s):  
Soil Properties ◽  
Near Infrared ◽  
Diffuse Reflectance Spectroscopy ◽  
Monitoring Network ◽  
Total Carbon ◽  
Data Driven ◽  
Local Approach ◽  
Organic C ◽  
Soil C ◽  
Over Time

<p>Soil data at different scales are needed for assessments and monitoring of soil functions. Soil diffuse reflectance spectroscopy using visible–Near Infrared and mid-Infrared energies can be used to estimate a range of soil properties, rapidly and inexpensively. However the spectroscopic modeling is challenging because of the large soil diversity and its complex composition. We developed a National Soil Spectral library (SSL) (n = 4339) using samples from (i) the Swiss Soil Monitoring Network (NABO; 7 sampling campaigns at 71 agricultural locations since 1985, n = 592) and (ii) the National Biodiversity Monitoring (BDM) Program (n = 4295, 1094 locations across a 5x5 km grid). The SSL will provide spectroscopic models for estimation of functional soil properties at different scales (e.g. total carbon (C) and nitrogen, organic C, texture, pH and cation exchange capacity). We used a rule-based algorithm, Cubist, for the modelling. The models were tuned across full combinations of {5, 10, 20, 50, 100} committees and {2, 5, 7, 9} neighbors, using 5 times repeated 10-fold cross-validation grouped by location. Further, transfer learning with RS-LOCAL tuning was performed for each of the 71 monitoring sites separately by a hold out approach in order to select optimal instances from the remaining SSL. Total soil C in the reference data ranged from 0.1% to 58.3% C and the best Cubist model had a cross-validated RMSE of 0.82% C. The RS-LOCAL approach (RMSE<sub>mean</sub> = 0.14 %) was on average 2.5 times more accurate for the estimation of C over time at each of the 71 NABO sites compared to the general Cubist approach. Our results suggest that data-driven selection of SSL instances targeted to closely related soils produces less biased estimation of soil properties over time at smaller geographic extents. The general Cubist calibration models are useful when reference analyses in a new study area are scarce. In conclusion, the Swiss SSL models can be used to cost-efficiently estimate a range of soil properties for a diverse applications and purposes in Switzerland.</p>

scholarly journals The central African soil spectral library: a new soil infrared repository and a geographical prediction analysis

SOIL ◽  
2021 ◽  
Vol 7 (2) ◽  
pp. 693-715
Author(s):  
Laura Summerauer ◽  
Philipp Baumann ◽  
Leonardo Ramirez-Lopez ◽  
Matti Barthel ◽  
Marijn Bauters ◽  
...  
Keyword(s):  
Soil Properties ◽  
Information Service ◽  
Spectral Feature ◽  
Central Africa ◽  
Congo Basin ◽  
Total Carbon ◽  
Great Success ◽  
Spectral Library ◽  
Address Data ◽  
African Soil

Abstract. Information on soil properties is crucial for soil preservation, the improvement of food security, and the provision of ecosystem services. In particular, for the African continent, spatially explicit information on soils and their ability to sustain these services is still scarce. To address data gaps, infrared spectroscopy has achieved great success as a cost-effective solution to quantify soil properties in recent decades. Here, we present a mid-infrared soil spectral library (SSL) for central Africa (CSSL) that can predict key soil properties, allowing for future soil estimates with a minimal need for expensive and time-consuming wet chemistry. Currently, our CSSL contains over 1800 soil samples from 10 distinct geoclimatic regions throughout the Congo Basin and along the Albertine Rift. For the analysis, we selected six regions from the CSSL, for which we built predictive models for total carbon (TC) and total nitrogen (TN) using an existing continental SSL (African Soil Information Service, AfSIS SSL; n=1902) that does not include central African soils. Using memory-based learning (MBL), we explored three different strategies at decreasing degrees of geographic extrapolation, using models built with (1) the AfSIS SSL only, (2) AfSIS SSL combined with the five remaining central African regions, and (3) a combination of AfSIS SSL, the remaining five regions, and selected samples from the target region (spiking). For this last strategy we introduce a method for spiking MBL models. We found that when using the AfSIS SSL only to predict the six central African regions, the root mean square error of the predictions (RMSEpred) was between 3.85–8.74 and 0.40–1.66 g kg−1 for TC and TN, respectively. The ratio of performance to the interquartile distance (RPIQpred) ranged between 0.96–3.95 for TC and 0.59–2.86 for TN. While the effect of the second strategy compared to the first strategy was mixed, the third strategy, spiking with samples from the target regions, could clearly reduce the RMSEpred to 3.19–7.32 g kg−1 for TC and 0.24–0.89 g kg−1 for TN. RPIQpred values were increased to ranges of 1.43–5.48 and 1.62–4.45 for TC and TN, respectively. In general, predicted TC and TN for soils of each of the six regions were accurate; the effect of spiking and avoiding geographical extrapolation was noticeably large. We conclude that our CSSL adds valuable soil diversity that can improve predictions for the Congo Basin region compared to using the continental AfSIS SSL alone; thus, analyses of other soils in central Africa will be able to profit from a more diverse spectral feature space. Given these promising results, the library comprises an important tool to facilitate economical soil analyses and predict soil properties in an understudied yet critical region of Africa. Our SSL is openly available for application and for enlargement with more spectral and reference data to further improve soil diagnostic accuracy and cost-effectiveness.

scholarly journals Estimation of soil properties with mid-infrared soil spectroscopy across yam production landscapes in West Africa

SOIL ◽  
2021 ◽  
Vol 7 (2) ◽  
pp. 717-731
Author(s):  
Philipp Baumann ◽  
Juhwan Lee ◽  
Emmanuel Frossard ◽  
Laurie Paule Schönholzer ◽  
Lucien Diby ◽  
...  
Keyword(s):  
West Africa ◽  
Soil Fertility ◽  
Soil Properties ◽  
Clay Content ◽  
Exchange Capacity ◽  
Total Carbon ◽  
Least Squares Regression ◽  
Quick Method ◽  
Humid Forest ◽  
Mid Infrared

Abstract. Low soil fertility is challenging the sustainable production of yam and other staple crops in the yam belt of West Africa. Quantitative soil measures are needed to assess soil fertility decline and to improve crop nutrient supply in the region. We developed and tested a mid-infrared (mid-IR) soil spectral library to enable timely and cost-efficient assessments of soil properties. Our collection included 80 soil samples from four landscapes (10 km × 10 km) and 20 fields per landscape across a gradient from humid forest to savannah and 14 additional samples from one landscape that had been sampled within the Land Health Degradation Framework. We derived partial least squares regression models to spectrally estimate soil properties. The models produced accurate cross-validated estimates of total carbon, total nitrogen, total sulfur, total iron, total aluminum, total potassium, total calcium, exchangeable calcium, effective cation exchange capacity, and diethylenetriaminepentaacetic acid (DTPA)-extractable iron and clay content (R2>0.75). The estimates of total zinc, pH, exchangeable magnesium, bioavailable copper, and manganese were less predictable (R2>0.50). Our results confirm that mid-IR spectroscopy is a reliable and quick method to assess the regional-level variation of most soil properties, especially the ones closely associated with soil organic matter. Although the relatively small mid-IR library shows satisfactory performance, we expect that frequent but small model updates will be needed to adapt the library to the variation of soil quality within individual fields in the regions and their temporal fluctuations.

Sign in / Sign up

Export Citation Format

Share Document