scholarly journals Is Standardization Necessary for Sharing of a Large Mid-Infrared Soil Spectral Library?

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6729
Author(s):  
Shree R. S. Dangal ◽  
Jonathan Sanderman
Keyword(s):  
Soil Properties ◽  
Predictive Models ◽  
Soil Samples ◽  
Soil Survey ◽  
Least Squares Regression ◽  
Spectral Library ◽  
Calibration Transfer ◽  
Mid Infrared ◽  
Calibration Models ◽  
Spectral Libraries

Recent developments in diffuse reflectance soil spectroscopy have increasingly focused on building and using large soil spectral libraries with the purpose of supporting many activities relevant to monitoring, mapping and managing soil resources. A potential limitation of using a mid-infrared (MIR) spectral library developed by another laboratory is the need to account for inherent differences in the signal strength at each wavelength associated with different instrumental and environmental conditions. Here we apply predictive models built using the USDA National Soil Survey Center–Kellogg Soil Survey Laboratory (NSSC-KSSL) MIR spectral library (n = 56,155) to samples sets of European and US origin scanned on a secondary spectrometer to assess the need for calibration transfer using a piecewise direct standardization (PDS) approach in transforming spectra before predicting carbon cycle relevant soil properties (bulk density, CaCO3, organic carbon, clay and pH). The European soil samples were from the land use/cover area frame statistical survey (LUCAS) database available through the European Soil Data Center (ESDAC), while the US soil samples were from the National Ecological Observatory Network (NEON). Additionally, the performance of the predictive models on PDS transfer spectra was tested against the direct calibration models built using samples scanned on the secondary spectrometer. On independent test sets of European and US origin, PDS improved predictions for most but not all soil properties with memory based learning (MBL) models generally outperforming partial least squares regression and Cubist models. Our study suggests that while good-to-excellent results can be obtained without calibration transfer, for most of the cases presented in this study, PDS was necessary for unbiased predictions. The MBL models also outperformed the direct calibration models for most of the soil properties. For laboratories building new spectroscopy capacity utilizing existing spectral libraries, it appears necessary to develop calibration transfer using PDS or other calibration transfer techniques to obtain the least biased and most precise predictions of different soil properties.

Soil exchangeable cations estimation using Vis-NIR spectroscopy in different depths: Effects of multiple calibration models and spiking 

2021 ◽  
Author(s):  
Dongxue Zhao ◽  
Maryem Arshad ◽  
Jie Wang ◽  
John Triantafilis
Keyword(s):  
Near Infrared ◽  
Nir Spectroscopy ◽  
High Rate ◽  
Support Vector ◽  
Least Squares Regression ◽  
Spectral Library ◽  
Calibration Models ◽  
Development Data ◽  
Different Depths ◽  
Spectral Libraries

<p>Due to high rate of nutrient removal by cotton plants, the productive cotton-growing soils of Australia is becoming depleted of exchangeable (exch.) cations. For long-term development, data on exch. calcium (Ca), magnesium (Mg), potassium (K) and sodium (Na) throughout the soil profile is required. However, traditional laboratory analysis is tedious. The visible-near-infrared (Vis-NIR) spectroscopy is an alternative; whereby, spectral libraries are built which couple soil data and Vis-NIR spectra using models. While various models have been used to predict exch. cations, their performance was seldom systematically compared. Moreover, most previous studies have focused on prediction of topsoil (0–0.3 m) exch. cations while the effects of depth on applicability of topsoil spectral libraries are rarely investigated. Our first aim was to determine which model (i.e. partial least squares regression (PLSR), Cubist, random forest (RF), or support vector machine regression (SVMR)) produces the best prediction of topsoil exch. Ca, Mg, K and Na. The second aim was to evaluate if the best topsoil model can be used to predict subsurface (0.3–0.6 m) and subsoil (0.9–1.2 m) cations. The third aim was to explore the effect of spiking on the prediction in subsurface and subsoil. The fourth aim was to see if combining all depths to build a profile spectral library improved prediction. Based on independent validation, PLSR was superior for topsoil exch. cations prediction, while Cubist outperformed PLSR in some cases when spiking was applied, and the profile spectral library was considered. Topsoil PLSR could be applied to predict exch. Ca and Mg in the subsurface and subsoil, while spiking improved prediction. Moreover, a profile spectral library achieved equivalent results with when topsoil samples coupled with spiking were considered. We, therefore, recommended to predict exch. Ca and Mg throughout the profile using topsoil spectral library coupled with spiking approach.</p>

scholarly journals Accurate and Precise Prediction of Soil Properties from a Large Mid-Infrared Spectral Library

Soil Systems ◽  
2019 ◽  
Vol 3 (1) ◽  
pp. 11 ◽  
Author(s):  
Shree Dangal ◽  
Jonathan Sanderman ◽  
Skye Wills ◽  
Leonardo Ramirez-Lopez
Keyword(s):  
Soil Properties ◽  
Diffuse Reflectance Spectroscopy ◽  
Cost Effective ◽  
Machine Learning Algorithms ◽  
Soil Survey ◽  
Spectral Library ◽  
Mid Infrared ◽  
Large N ◽  
Infrared Spectral ◽  
Precise Prediction

Diffuse reflectance spectroscopy (DRS) is emerging as a rapid and cost-effective alternative to routine laboratory analysis for many soil properties. However, it has primarily been applied in project-specific contexts. Here, we provide an assessment of DRS spectroscopy at the scale of the continental United States by utilizing the large (n > 50,000) USDA National Soil Survey Center mid-infrared spectral library and associated soil characterization database. We tested and optimized several advanced statistical approaches for providing routine predictions of numerous soil properties relevant to studying carbon cycling. On independent validation sets, the machine learning algorithms Cubist and memory-based learner (MBL) both outperformed random forest (RF) and partial least squares regressions (PLSR) and produced excellent overall models with a mean R2 of 0.92 (mean ratio of performance to deviation = 6.5) across all 10 soil properties. We found that the use of root-mean-square error (RMSE) was misleading for understanding the actual uncertainty about any particular prediction; therefore, we developed routines to assess the prediction uncertainty for all models except Cubist. The MBL models produced much more precise predictions compared with global PLSR and RF. Finally, we present several techniques that can be used to flag predictions of new samples that may not be reliable because their spectra fall outside of the calibration set.

Deep Learning-based Soil Property Prediction for Remediation of Radioactive Contamination in Agriculture

2021 ◽  
Author(s):  
Franck Albinet ◽  
Gerd Dercon ◽  
Tetsuya Eguchi
Keyword(s):  
Deep Learning ◽  
Soil Properties ◽  
Radioactive Contamination ◽  
The United States ◽  
Soil Survey ◽  
United States Department ◽  
Least Squares Regression ◽  
Transfer Factors ◽  
Wide Range ◽  
Plant Transfer

<p>The Joint IAEA/FAO Division of Nuclear Techniques in Food and Agriculture, through its Soil and Water Management & Crop Nutrition Laboratory (SWMCNL), launched in October 2019, a new Coordinated Research Project (D15019) called “Monitoring and Predicting Radionuclide Uptake and Dynamics for Optimizing Remediation of Radioactive Contamination in Agriculture''. Within this context, the high-throughput characterization of soil properties in general and the estimation of soil-to-plant transfer factors of radionuclides are of critical importance.</p><p>For several decades, soil researchers have been successfully using near and mid-infrared spectroscopy (MIRS) techniques to estimate a wide range of soil physical, chemical and biological properties such as carbon (C), Cation Exchange Capacities (CEC), among others. However, models developed were often limited in scope as only small and region-specific MIR spectra libraries of soils were accessible.</p><p>This situation of data scarcity is changing radically today with the availability of large and growing library of MIR-scanned soil samples maintained by the National Soil Survey Center (NSSC) Kellogg Soil Survey Laboratory (KSSL) from the United States Department of Agriculture (USDA-NRCS) and the Global Soil Laboratory Network (GLOSOLAN) initiative of the Food Agency Organization (FAO). As a result, the unprecedented volume of data now available allows soil science researchers to increasingly shift their focus from traditional modeling techniques such as PLSR (Partial Least Squares Regression) to classes of modeling approaches, such as Ensemble Learning or Deep Learning, that have proven to outperform PLSR on most soil properties prediction in a large data regime.</p><p>As part of our research, the opportunity to train higher capacity models on the KSSL large dataset (all soil taxonomic orders included ~ 50K samples) makes it possible to reach a quality of prediction for exchangeable potassium so far unsurpassed with a Residual Prediction Deviation (RPD) around 3. Potassium is known for its difficulty of being predicted but remains extremely important in the context of remediation of radioactive contamination after a nuclear accident. Potassium can help reduce the uptake of radiocaesium by crops, as it competes with radiocaesium in soil-to-plant transfer.</p><p>To ensure informed decision making, we also guarantee that (i) individual predictions uncertainty is estimated (using Monte Carlo Dropout) and (ii) individual predictions can be interpreted (i.e. how much specific MIRS wavenumber regions contribute to the prediction) using methods such as Shapley Additive exPlanations (SHAP) values.</p><p>SWMCNL is now a member of the GLOSOLAN network, which helps enhance the usability of MIRS for soil monitoring worldwide. SWMCNL is further developing training packages on the use of traditional and advanced mathematical techniques to process MIRS data for predicting soil properties. This training package has been tested in October 2020 with thirteen staff members of the FAO/IAEA Laboratories in Seibersdorf, Austria.</p>

EXPRESS: Comparison of Spectroscopic Techniques for Determining the Peroxide Value of 19 Classes of Naturally Aged, Plant-Based Edible Oils

2020 ◽  
pp. 000370282097470
Author(s):  
Joshua M. Ottaway ◽  
J. Chance Carter ◽  
Kristl L Adams ◽  
Joseph Camancho ◽  
Barry Lavine ◽  
...  
Keyword(s):  
Olive Oil ◽  
Peroxide Value ◽  
Near Infrared ◽  
Edible Oils ◽  
Virgin Olive Oil ◽  
Extra Virgin Olive Oil ◽  
Spectroscopic Techniques ◽  
Least Squares Regression ◽  
Mid Infrared ◽  
Calibration Models

The peroxide value (PV) of edible oils is a measure of the degree of oxidation, which directly relates to the freshness of the oil sample. Several studies previously reported in the literature have paired various spectroscopic techniques with multivariate analyses to rapidly determine PVs using field portable and process instrumentation; those efforts presented ‘best-case’ scenarios with oils from narrowly defined training and test sets. The purpose of this paper is to evaluate the use of near- and mid-infrared absorption and Raman scattering spectroscopies on oil samples from different oil classes, including seasonal and vendor variations, to determine which measurement technique, or combination thereof, is best for predicting PVs. Following PV assays of each oil class using an established titration-based method, global and global-subset calibration models were constructed from spectroscopic data collected on the 19 oil classes used in this study. Spectra from each optical technique were used to create partial least squares regression (PLSR) calibration models to predict the PV of unknown oil samples. A global PV model based on near-infrared (8 mm optical path length – OPL) oil measurements produced the lowest RMSEP (4.9), followed by 24 mm OPL near infrared (5.1), Raman (6.9) and 50 μm OPL mid-infrared (7.3). However, it was determined that the Raman RMSEP resulted from chance correlations. Global PV models based on low-level fusion of the NIR (8 and 24 mm OPL) data and all infrared data produced the same RMSEP of 5.1. Global subset models, based on any of the spectroscopies and olive oil training sets from any class (pure, extra light, extra virgin), all failed to extrapolate to the non-olive oils. However, the near-infrared global subset model built on extra virgin olive oil could extrapolate to test samples from other olive oil classes.

Prediction of soil organic and inorganic carbon concentrations in Tunisian samples by mid-infrared reflectance spectroscopy using a French national library

2020 ◽  
Author(s):  
Tiphaine Chevallier ◽  
Cécile Gomez ◽  
Patricia Moulin ◽  
Imane Bouferra ◽  
Kaouther Hmaidi ◽  
...  
Keyword(s):  
Inorganic Carbon ◽  
Prediction Models ◽  
Reflectance Spectroscopy ◽  
Soil Samples ◽  
Infrared Reflectance ◽  
Least Squares Regression ◽  
Mid Infrared ◽  
Climate Conditions ◽  
Infrared Reflectance Spectroscopy ◽  
Soil Database

<p>Mid-Infrared Reflectance Spectroscopy (MIRS, 4000–400 cm<sup>-1</sup>) is being considered to provide accurate estimations of soil properties, including soil organic carbon (SOC) and soil inorganic carbon (SIC) contents. This has mainly been demonstrated when datasets used to build, validate and test the prediction model originate from the same area A, with similar geopedological conditions. The objective of this study was to analyze how MIRS performed when used to predict SOC and SIC contents, from a calibration database collected over a region A, to predict over a region B, where A and B have no common area and different soil and climate conditions. This study used a French MIRS soil dataset including 2178 soil samples to calibrate SIC and SOC prediction models with partial least squares regression (PLSR), and a Tunisian MIRS soil dataset including 96 soil samples to test them. Our results showed that using the French MIRS soil database i) SOC and SIC of French samples were successfully predicted, ii) SIC of Tunisian samples was also predicted successfully, iii) local calibration significantly improved SOC prediction of Tunisian samples and iv) prediction models seemed more robust for SIC than for SOC. So in future, MIRS might replace, or at least be considered as, a conventional physico-chemical analysis technique, especially when as exhaustive as possible calibration database will become available.</p>

scholarly journals Feasibility of handheld mid-infrared spectroscopy to predict particle size distribution: influence of soil field condition and utilisation of existing spectral libraries

Soil Research ◽  
2020 ◽  
Vol 58 (6) ◽  
pp. 528
Author(s):  
Leslie J. Janik ◽  
José M. Soriano-Disla ◽  
Sean T. Forrester
Keyword(s):  
Particle Size ◽  
Particle Size Distribution ◽  
Size Distribution ◽  
Aggregate Size ◽  
Clay Content ◽  
Coefficient Of Determination ◽  
Least Squares Regression ◽  
Mid Infrared ◽  
Spectral Libraries ◽  
Direct Standardisation

Partial least-squares regression (PLSR), using spectra from a handheld mid-infrared instrument (the ExoScan), was tested for the prediction of particle size distribution. Soils were sampled from agricultural sites in the Eyre Peninsula under field conditions and with varying degrees of soil preparation. Issues relevant to field sampling were identified, such as sample heterogeneity, micro-aggregate size and moisture content. The PLSR models for particle size distribution were derived with the varying degrees of preparation. Cross-validation of clay content in the as-received in situ soils resulted in low accuracy: coefficient of determination (R2) = 0.55 and root mean square error (RMSE) = 7%. This was improved by manual mixing, drying, sieving to < 2 mm and fine grinding, resulting in R2 values of 0.64, 0.75 and 0.81, and RMSE of 6%, 5% and 4% respectively; less improvement resulted for sand, with corresponding R2 values of 0.82, 0.88, 0.91 and 0.89, and RMSE of 10%, 8%, 6% and 7%. Predictions for silt remained poor. Where only archival benchtop calibration models were available, predictions of clay contents for spectra scanned with the handheld ExoScan spectrometer resulted in high error because of spectral intensity mismatch between benchtop and handheld spectra (R2 = 0.72, RMSE = 24.2% and bias = 21%). Pre-processing the benchtop spectra by piecewise direct standardisation resulted in more successful predictions (R2 = 0.73, RMSE = 6.7% and bias = –1.5%), confirming the advantage of piecewise direct standardisation for prediction from archival spectral libraries.

Evaluating near infrared spectroscopy for field prediction of soil properties

Soil Research ◽  
2009 ◽  
Vol 47 (7) ◽  
pp. 664 ◽  
Author(s):  
Budiman Minasny ◽  
Alex B. McBratney ◽  
Leo Pichon ◽  
Wei Sun ◽  
Michael G. Short
Keyword(s):  
Soil Properties ◽  
Near Infrared ◽  
Agricultural Land ◽  
Diffuse Reflectance Spectroscopy ◽  
Soil Samples ◽  
Soil Mapping ◽  
Digital Soil Mapping ◽  
Soil Conditions ◽  
Spectral Library ◽  
Field Samples

This paper demonstrates the application of near infrared diffuse reflectance spectroscopy (NIR-DRS) measurements as part of digital soil mapping. We also investigate whether calibration functions developed from a spectral library can be used for rapid characterisation of soil properties in the field. Soil samples were collected along 24 toposequences in the Pokolbin irrigation district, ~7 km2 of predominantly agricultural land in the Hunter Valley, NSW, Australia. Soil samples at 2 depths: 0–0.10 and 0.40–0.50 m were collected. The soil samples were scanned using NIR under 3 different conditions: field condition, dried unground, and dried ground. A separate spectral library containing soil laboratory measurements was used to develop functions to predict 3 main soil properties from NIR spectra (total C content, clay content, and sum of exchangeable cations). The absorbance spectra were found to be different for the 3 soil conditions. The field spectra appear to have higher absorbance, followed by dried unground samples and then dried ground samples. Although most spectral signatures or peaks were similar for the 3 soil conditions, field samples appear to have higher absorbance, particularly at 1400 nm and 1900 nm. The convex hull of the first 2 principal components of the soil spectra is an easy tool to evaluate the similarity of spectra from a calibration set to an observation. For field prediction, samples need to be calibrated using field samples. Finally, this study shows that NIR-DRS measurement is a useful part of digital soil mapping.

Sign in / Sign up

Export Citation Format

Share Document