A Survey of Ridge Regression and Related Techniques for Improvements over Ordinary Least Squares

1978 ◽  
Vol 60 (1) ◽  
pp. 121 ◽  
Author(s):  
Hrishikesh D. Vinod
Keyword(s):  
Least Squares ◽  
Ridge Regression ◽  
Ordinary Least Squares

Prediction of voluntary Intake of grass silage by growing cattle using ridge regression

1989 ◽  
Vol 1989 ◽  
pp. 68-68 ◽  
Author(s):  
A. J. Rook ◽  
M. Gill ◽  
M. S. Dhanoa
Keyword(s):  
Least Squares ◽  
Ridge Regression ◽  
Prediction Models ◽  
Ordinary Least Squares ◽  
Independent Data ◽  
College Of Agriculture ◽  
Grass Silage ◽  
Backward Elimination ◽  
Growing Cattle ◽  
Controlled Experimentation

Due to collinearity among the independent varlates, intake prediction models based on least squares multiple regression are likely to predict poorly with independent data. In addition, the regression coefficients are sensitive to small changes in the estimation data and tend not to reflect causal relationships expected from the results of controlled experimentation. Ridge regression (Hoerl and Kennard, 1970) allows the estimation of new coefficients for the independent variables which overcome these effects of collinearity. In order to assess the usefulness of the method for Intake prediction, ordinary least squares (OLS) models, obtained using backward elimination of variables, and ridge regression models were constructed from the same data and then tested with independent data.Estimation data consisted of results of experiments of IGAP, Hurley and Greenmount College of Agriculture in which growing cattle were individually fed grass silage ad-libitum with or without supplementary feeds. Two subsets of the estimation data were used. Subset A included 395 animals and 36 silages; subset B included 192 animals and 16 silages and was for Hurley data only.

Parameter Estimation in Marketing Models in the Presence of Multicollinearity: An Application of Ridge Regression

1977 ◽  
Vol 14 (4) ◽  
pp. 586-591 ◽  
Author(s):  
Vijay Mahajan ◽  
Arun K. Jain ◽  
Michel Bergier
Keyword(s):  
Parameter Estimation ◽  
Regression Analysis ◽  
Least Squares ◽  
Ridge Regression ◽  
Ordinary Least Squares ◽  
Regression Coefficients ◽  
Estimation Of Parameters ◽  
Marketing Models ◽  
Marketing Data ◽  
Ridge Regression Analysis

In the presence of multicollinearity in data, the estimation of parameters or regression coefficients in marketing models by means of ordinary least squares may give inflated estimates with a high variance and wrong signs. The authors demonstrate the potential usefulness of the ridge regression analysis to handle multicollinearity in marketing data.

Ridge Regression in the Presence of Multicollinearity

1984 ◽  
Vol 54 (2) ◽  
pp. 559-566 ◽  
Author(s):  
Rashmi Garg
Keyword(s):  
Social Sciences ◽  
Least Squares ◽  
Simple Form ◽  
Ridge Regression ◽  
Least Squares Method ◽  
Ordinary Least Squares ◽  
Regression Coefficients ◽  
Predictor Variables ◽  
Regression Problems ◽  
Ordinary Least Squares Method

The ordinary least squares solution is generally applied to multiple regression problems in social sciences. When the intercorrelations among predictor variables are close to one, the estimates of regression coefficients obtained from ordinary least squares are very unstable. This situation is often referred to as near multicollinearity. When there is a problem of near mulricollinearity, the ridge regression provides an alternative to the ordinary least squares method. The ridge estimates are biased but more stable from sample to sample. The purpose of this article is to describe the method of ridge regression in a simple form and to provide examples of its application.

scholarly journals Using Robust Ridge Regression Diagnostic Method to Handle Multicollinearity Caused High Leverage Points

2021 ◽  
Vol 10 (1) ◽  
pp. 326
Author(s):  
Kafi Dano Pati
Keyword(s):  
Regression Model ◽  
Least Squares ◽  
Ridge Regression ◽  
Robust Regression ◽  
Ordinary Least Squares ◽  
Substantial Improvement ◽  
Least Trimmed Squares ◽  
Regression Estimate ◽  
Leverage Points ◽  
Parameter Coefficient

Statistics practitioners have been depending on the ordinary least squares (OLS) method in the linear regression model for generation because of its optimal properties and simplicity of calculation. However, the OLS estimators can be strongly affected by the existence of multicollinearity which is a near linear dependency between two or more independent variables in the regression model. Even though in the presence of multicollinearity the OLS estimate still remained unbiased, they will be inaccurate prediction about the dependent variable with the inflated standard errors of the estimated parameter coefficient of the regression model. It is now evident that the existence of high leverage points which are the outliers in x-direction are the prime factor of collinearity influential observations. In this paper, we proposed some alternative to regression methods for estimating the regression parameter coefficient in the presence of multiple high leverage points which cause the multicollinearity problem. This procedure utilized the ordinary least squares estimates of the parameter as the initial followed by an estimate of the ridge regression. We incorporated the Least Trimmed Squares (LTS) robust regression estimate to down weight the effects of multiple high leverage points which lead to the reduction of the effects of multicollinearity. The result seemed to suggest that the RLTS give a substantial improvement over the Ridge Regression.

scholarly journals Predictive efficiency of ridge regression estimator

2017 ◽  
Vol 27 (2) ◽  
pp. 243-247
Author(s):  
Manoj Tiwari ◽  
Amit Sharma
Keyword(s):  
Least Squares ◽  
Ridge Regression ◽  
Ordinary Least Squares ◽  
Regression Estimator ◽  
Performance Properties ◽  
Regression Estimators ◽  
Ridge Regression Estimator ◽  
Predictive Efficiency

In this article we have considered the problem of prediction within and outside the sample for actual and average values of the study variables in case of ordinary least squares and ridge regression estimators. Finally, the performance properties of the estimators are analyzed.

scholarly journals Scale-Selective Ridge Regression for Multimodel Forecasting

2013 ◽  
Vol 26 (20) ◽  
pp. 7957-7965 ◽  
Author(s):  
Timothy DelSole ◽  
Liwei Jia ◽  
Michael K. Tippett
Keyword(s):  
Least Squares ◽  
Ridge Regression ◽  
Large Scale ◽  
Small Sample Size ◽  
Ordinary Least Squares ◽  
Small Sample ◽  
Spatial Gradients ◽  
Smoothness Constraint ◽  
Grid Points ◽  
The Individual

Abstract This paper proposes a new approach to linearly combining multimodel forecasts, called scale-selective ridge regression, which ensures that the weighting coefficients satisfy certain smoothness constraints. The smoothness constraint reflects the “prior assumption” that seasonally predictable patterns tend to be large scale. In the absence of a smoothness constraint, regression methods typically produce noisy weights and hence noisy predictions. Constraining the weights to be smooth ensures that the multimodel combination is no less smooth than the individual model forecasts. The proposed method is equivalent to minimizing a cost function comprising the familiar mean square error plus a “penalty function” that penalizes weights with large spatial gradients. The method reduces to pointwise ridge regression for a suitable choice of constraint. The method is tested using the Ensemble-Based Predictions of Climate Changes and Their Impacts (ENSEMBLES) hindcast dataset during 1960–2005. The cross-validated skill of the proposed forecast method is shown to be larger than the skill of either ordinary least squares or pointwise ridge regression, although the significance of this difference is difficult to test owing to the small sample size. The model weights derived from the method are much smoother than those obtained from ordinary least squares or pointwise ridge regression. Interestingly, regressions in which the weights are completely independent of space give comparable overall skill. The scale-selective ridge is numerically more intensive than pointwise methods since the solution requires solving equations that couple all grid points together.

An Examination of an Ecological Model of Neighborhood Change, Which Employs a Biased Estimation Technique

1983 ◽  
Vol 15 (7) ◽  
pp. 889-901
Author(s):  
W A Schwab ◽  
M K Miller
Keyword(s):  
Least Squares ◽  
Ridge Regression ◽  
Census Data ◽  
Ecological Model ◽  
Secondary Data ◽  
Single Equation ◽  
Ordinary Least Squares ◽  
Neighborhood Change ◽  
Ecological Research ◽  
Least Squares Estimates

The problems created by multicollinearity when estimating a single equation least squares model are well-known. This problem is particularly troublesome in ecological research because of the heavy reliance of this field on census data and other secondary data sources. This paper explores one solution to the problem of multicollinearity in ecological research—ridge regression. Using census-tract data from Cleveland, Ohio for the years 1960–1970, an ecological model, known as the tipping-point model, of neighborhood change, was evaluated. It was determined that multicollinearity was a problem. To minimize the detrimental effects of multicollinearity and to facilitate structural interpretation, the model was reestimated with the ridge regression technique. The result was a total mean square error that was significantly smaller than the total variance resulting from an ordinary least squares solution. Hence, the estimates of the coefficients produced by the ridge regression were closer, on the average, to the true population parameters than the ordinary least squares estimates. Results of the ordinary least squares and ridge regression are presented and discussed and the utility of ridge regression in ecological research is evaluated.

INVESTIGATING THE IMPACT OF MULTICOLLINEARITY ON LINEAR REGRESSION ESTIMATES.

2021 ◽  
Vol 6 (1) ◽  
pp. 698
Author(s):  
Kunle Bayo Adewoye ◽  
Ayinla Bayo Rafiu ◽  
Titilope Funmilayo Aminu ◽  
Isaac Oluyemi Onikola
Keyword(s):  
Linear Regression ◽  
Sample Size ◽  
Least Squares ◽  
Ridge Regression ◽  
Research Work ◽  
Ordinary Least Squares ◽  
Small Sample ◽  
Elastic Net ◽  
Sample Sizes ◽  
The Impact

Multicollinearity is a case of multiple regression in which the predictor variables are themselves highly correlated. The aim of the study was to investigate the impact of multicollinearity on linear regression estimates. The study was guided by the following specific objectives, (i) to examined the asymptotic properties of estimators and (ii) to compared lasso, ridge, elastic net with ordinary least squares. The study employed Monte-carlo simulation to generate set of highly collinear and induced multicollinearity variables with sample sizes of 25, 50, 100, 150, 200, 250, 1000 as a source of data in this research work and the data was analyzed with lasso, ridge, elastic net and ordinary least squares using statistical package. The study findings revealed that absolute bias of ordinary least squares was consistent at all sample sizes as revealed by past researched on multicollinearity as well while lasso type estimators were fluctuate alternately. Also revealed that, mean square error of ridge regression was outperformed other estimators with minimum variance at small sample size and ordinary least squares was the best at large sample size. The study recommended that ols was asymptotically consistent at a specified sample sizes on this research work and ridge regression was efficient at small and moderate sample size.

Sign in / Sign up

Export Citation Format

Share Document