A Monte Carlo Study of the Effect of Noise on Wavelength Selection during Computerized Wavelength Searches

1988 ◽  
Vol 42 (8) ◽  
pp. 1427-1440 ◽  
Author(s):  
Howard Mark
Keyword(s):  
Monte Carlo ◽  
Quantitative Analysis ◽  
Spectral Data ◽  
Near Infrared ◽  
Selection Process ◽  
Monte Carlo Study ◽  
Monte Carlo Technique ◽  
Wavelength Selection ◽  
Data Set ◽  
Random Nature

The process of selecting wavelengths for performing quantitative analysis in the near-infrared is notorious for its instability. A Monte Carlo technique was used to investigate the sensitivity of the wavelength selection process to the noise content of the spectra. The random nature of the noise causes the wavelengths to be selected at random; this seems to be sufficient to explain the instability of the selection process. The statistics of the selection process are insensitive to error in the dependent variable, and, within limits, also insensitive to the amount of noise in the spectral data. The statistics are sensitive to the number of samples in the data set and to the nature of the distribution of the noise.

scholarly journals Robust Wavelength Selection Using Filter-Wrapper Method and Input Scaling on Near Infrared Spectral Data

Sensors ◽  
2020 ◽  
Vol 20 (17) ◽  
pp. 5001 ◽  
Author(s):  
Divo Dharma Silalahi ◽  
Habshah Midi ◽  
Jayanthi Arasan ◽  
Mohd Shafie Mustafa ◽  
Jean-Pierre Caliman
Keyword(s):  
Monte Carlo ◽  
Spectral Data ◽  
Near Infrared ◽  
Selection Procedure ◽  
Partial Least Square ◽  
Least Square ◽  
Partial Least Square Regression ◽  
Tolerance Interval ◽  
Wavelength Selection ◽  
Wrapper Method

The extraction of relevant wavelengths from a large dataset of Near Infrared Spectroscopy (NIRS) is a significant challenge in vibrational spectroscopy research. Nonetheless, this process allows the improvement in the chemical interpretability by emphasizing the chemical entities related to the chemical parameters of samples. With the complexity in the dataset, it may be possible that irrelevant wavelengths are still included in the multivariate calibration. This yields the computational process to become unnecessary complex and decreases the accuracy and robustness of the model. In multivariate analysis, Partial Least Square Regression (PLSR) is a method commonly used to build a predictive model from NIR spectral data. However, in the PLSR method and common commercial chemometrics software, there is no standard wavelength selection procedure applied to screen the irrelevant wavelengths. In this study, a new robust wavelength selection procedure called the modified VIP-MCUVE (mod-VIP-MCUVE) using Filter-Wrapper method and input scaling strategy is introduced. The proposed method combines the modified Variable Importance in Projection (VIP) and modified Monte Carlo Uninformative Variable Elimination (MCUVE) to calculate the scale matrix of the input variable. The modified VIP uses the orthogonal components of Partial Least Square (PLS) in investigating the informative variable in the model by applying the amount of variation both in X and y{SSX,SSY}, simultaneously. The modified MCUVE uses a robust reliability coefficient and a robust tolerance interval in the selection procedure. To evaluate the superiority of the proposed method, the classical VIP, MCUVE, and autoscaling procedure in classical PLSR were also included in the evaluation. Using artificial data with Monte Carlo simulation and NIR spectral data of oil palm (Elaeis guineensis Jacq.) fruit mesocarp, the study shows that the proposed method offers advantages to improve model interpretability, to be computationally extensive, and to produce better model accuracy.

Investigation of the particle size-dependent near-infrared spectral features of binary mixture samples in conjunction with Monte Carlo simulation and the influence of particle size on the accuracy of quantitative analysis

The Analyst ◽  
2018 ◽  
Vol 143 (18) ◽  
pp. 4306-4315 ◽  
Author(s):  
Pham K. Duy ◽  
Seulah Chun ◽  
Yoonjeong Lee ◽  
Hoeil Chung
Keyword(s):  
Monte Carlo Simulation ◽  
Particle Size ◽  
Monte Carlo ◽  
Quantitative Analysis ◽  
Near Infrared ◽  
Spectral Variation ◽  
Spectral Features ◽  
Size Dependent ◽  
Infrared Spectral ◽  
Mixture Samples

The origin of particle size-induced near-infrared (NIR) spectral variation, which is fundamental for robust quantitative analysis, was systematically studied in conjunction with Monte Carlo simulation.

New Near Infrared Wavelength Selection Algorithm Based on Monte-Carlo Method

2010 ◽  
Vol 30 (12) ◽  
pp. 3637-3642 ◽  
Author(s):  
洪明坚 Hong Mingjian ◽  
温泉 Wen Quan ◽  
温志渝 Wen Zhiyu
Keyword(s):  
Monte Carlo ◽  
Monte Carlo Method ◽  
Near Infrared ◽  
Selection Algorithm ◽  
Wavelength Selection ◽  
Infrared Wavelength

scholarly journals Near-Infrared Brain Volumetric Imaging Method: A Monte Carlo Study

2012 ◽  
Vol 18 (3) ◽  
pp. 1122-1129 ◽  
Author(s):  
Ching-Cheng Chuang ◽  
Pei-Ning Wang ◽  
Wei-Ta Chen ◽  
Tsuo-Hung Lan ◽  
Chung-Ming Chen ◽  
...  
Keyword(s):  
Monte Carlo ◽  
Near Infrared ◽  
Monte Carlo Study ◽  
Imaging Method ◽  
Volumetric Imaging

An Optimised Six-Wavelength Model for Predicting Kiwifruit Dry Matter

2015 ◽  
Vol 23 (2) ◽  
pp. 103-109 ◽  
Author(s):  
Khairul Anuar Shafie ◽  
Rainer Künnemeyer ◽  
Andrew McGlone ◽  
Sadhana Talele ◽  
Varvara Vetrova
Keyword(s):  
Dry Matter ◽  
Near Infrared ◽  
Random Noise ◽  
Linear Regression Models ◽  
Wavelength Selection ◽  
Data Set ◽  
Noise Component ◽  
Light Emitting ◽  
Partial Least Squares Model ◽  
Infrared Instruments

Optimised wavelength selection is important to the development of new types of inexpensive and portable near infrared instruments that might be used on fruit in orchards. The use of discrete bandwidth devices, such as light-emitting diodes, requires preselection of a small number of discrete wavelengths. In this work, a kiwifruit data set consisting of 834 absorbance spectra and corresponding fruit dry-matter measurements, an important maturity indicator for kiwifruit, has been subjected to an exhaustive wavelength search to build optimal multiple linear regression models of up to seven wavelengths. Using a standard partial least-squares model as a benchmark, a six-wavelength model has been identified as an optimum, predicting kiwifruit dry matter with r2 of 0.88 and root mean square error of prediction ( RMSEP) of 1.22%. The sensitivity of the model to shifts in the key wavelengths was also evaluated, revealing that a 1 nm offset or a 0.25 nm random noise component would be enough to increase the RMSEP by around 0.04% in actual dry matter value or 3% in relative percentage terms.

Modeling with Multiple Correlated Spectral Data Based on Approximating the Nonlinear Spectrum Induced by Scattering

2021 ◽  
pp. 000370282110365
Author(s):  
Yongshun Luo ◽  
Gang Li ◽  
Guosong Shan ◽  
Ling Lin
Keyword(s):  
Quantitative Analysis ◽  
Spectral Data ◽  
Prediction Accuracy ◽  
Data Set ◽  
Analysis Experiment ◽  
Scattering Material ◽  
Modeling Data ◽  
Highly Correlated ◽  
Equal Space ◽  
Strong Scattering

In the spectral quantitative analysis of scattering solution, the improvement of accuracy is seriously restricted by the nonlinearity caused by scattering, and even the measurement will fail due to the influence of scattering. The important reasons are that the modeling variables are greatly affected by nonlinearity, and the information contained in the modeling data cannot represent the scattering characteristics. In this paper, a method is proposed, in which the spectral data of several optical pathlengths with equal space are combined as the modeling data set of a sample. These highly correlated spectral data contain relatively nonlinear information. The addition of the spectral data provides more options for the selection of principal components in modeling with PLS method. By giving lower weight to the corresponding wavelength which is greatly affected by scattering, the model is insensitive to scattering and the prediction accuracy is improved. Through the spectral quantitative analysis experiment on strong scattering material, the prediction accuracy of the model was 61.7% higher than that of the traditional method and was 58.5% higher than that of the variable sorting for normalization method. The feasibility of the method is verified.

Sign in / Sign up

Export Citation Format

Share Document