A Calibration Tutorial for Spectral Data. Part 2. Partial Least Squares Regression Using Matlab and Some Neural Network Results

Part 1 explained multiplicative scatter correction (MSC), the building of a principal component regression (PCR) model and how the test data can be used in prediction. Emphasis was on data pretreatment for linearistion and on spectral/chemical interpretation of the results. Part 2 discusses partial least squares (PLS or PLSR) regression. The data set prepared in Part 1 is also used here. Details on data pretreatment are, therefore, not repeated. Some details of PLS modeling are explained using the calculations of the example. Also, the interpretation of the PLS model gets some attention. Neural network calculation results are included for comparison. Artifical neural networks (ANN) are non-linear, so linearisation is not considered necessary. Latent variable regression methods such as PLS and PCR and ANNs are all successive approximations to the unknown function y = f(x) that forms the basis of all calibration methods. In latent variable regression, the rank of the model determines the degree of approximation. In ANNs, the number of hidden nodes and the number of iterations determine the degree of approximation.

Download Full-text

Partial Least Squares local calibration of a UV–visible spectrometer used for in situ measurements of COD and TSS concentrations in urban drainage systems

Water Science & Technology ◽

10.2166/wst.2008.131 ◽

2008 ◽

Vol 57 (4) ◽

pp. 581-588 ◽

Cited By ~ 32

Author(s):

A. Torres ◽

J.-L. Bertrand-Krajewski

Keyword(s):

Least Squares ◽

Partial Least Squares ◽

Urban Drainage ◽

Local Calibration ◽

Data Set ◽

Drainage Systems ◽

Calibration Methods ◽

Urban Drainage Systems ◽

Uv Visible

Recent UV–visible spectrometers deliver on line and in situ absorbance spectra in wastewater or stormwater transported in urban drainage systems. After calibration with local data sets, spectra can be used to estimate pollutant concentrations. Calibration methods are usually based on PLS (Partial Least Squares) regression. Their most important difficulty lies in the identification of the number of both i) the latent vectors and ii) the independent variables. A method is proposed to identify these variables, based on an exhaustive tests procedure (Jackknife cross validation and matrix of prediction indicator). It was applied to estimate TSS (total suspended solids) or COD (chemical oxygen demand) concentrations at the inlet of a storage-settling tank in a stormwater separate sewer system, and compared to three other calibration methods used either for turbidity meters or UV–visible spectrometers. With the available calibration data set: i) the spectrometer gives results with better prediction quality than the turbidity meter, ii) for the spectrometer, local calibration gives better results than global calibration, iii) the proposed PLS method gives results with a similar order of magnitude in uncertainties as the manufacturer local calibration method, but is more open and transparent for the user. Similar results were obtained for a second data set.

Download Full-text