Bayesian bootstrap adaptive lasso estimators of regression models

Author(s):  
Bohan Li ◽  
Juan Wu
2019 ◽  
Vol 1 (1) ◽  
pp. 359-383 ◽  
Author(s):  
Frank Emmert-Streib ◽  
Matthias Dehmer

Regression models are a form of supervised learning methods that are important for machine learning, statistics, and general data science. Despite the fact that classical ordinary least squares (OLS) regression models have been known for a long time, in recent years there are many new developments that extend this model significantly. Above all, the least absolute shrinkage and selection operator (LASSO) model gained considerable interest. In this paper, we review general regression models with a focus on the LASSO and extensions thereof, including the adaptive LASSO, elastic net, and group LASSO. We discuss the regularization terms responsible for inducing coefficient shrinkage and variable selection leading to improved performance metrics of these regression models. This makes these modern, computational regression models valuable tools for analyzing high-dimensional problems.


Author(s):  
Marta Galvani ◽  
Chiara Bardelli ◽  
Silvia Figini ◽  
Pietro Muliere

Bootstrap resampling techinques, introduced by Efron and Rubin, can be presented in a general Bayesian framework, approximating the statistical distribution of a statistical functional φ(F), where F is a random distribution function. Efron’s and Rubin’s bootstrap procedures can be extended introducing an informative prior through the Proper Bayesian bootstrap. In this paper different bootstrap techniques are used and compared in predictive classification and regression models based on ensemble approaches, i.e. bagging models involving decision trees. Proper Bayesian bootstrap, proposed by Muliere and Secchi, is used to sample the posterior distribution over trees, introducing prior distributions on the covariates and the target variable. The results obtained are compared with respect to other competitive procedures employing different bootstrap techniques. The empirical analysis reports the results obtained on simulated and real data.


Algorithms ◽  
2021 ◽  
Vol 14 (1) ◽  
pp. 11
Author(s):  
Marta Galvani ◽  
Chiara Bardelli ◽  
Silvia Figini ◽  
Pietro Muliere

Bootstrap resampling techniques, introduced by Efron and Rubin, can be presented in a general Bayesian framework, approximating the statistical distribution of a statistical functional ϕ(F), where F is a random distribution function. Efron’s and Rubin’s bootstrap procedures can be extended, introducing an informative prior through the Proper Bayesian bootstrap. In this paper different bootstrap techniques are used and compared in predictive classification and regression models based on ensemble approaches, i.e., bagging models involving decision trees. Proper Bayesian bootstrap, proposed by Muliere and Secchi, is used to sample the posterior distribution over trees, introducing prior distributions on the covariates and the target variable. The results obtained are compared with respect to other competitive procedures employing different bootstrap techniques. The empirical analysis reports the results obtained on simulated and real data.


PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0242730
Author(s):  
Christopher J. Greenwood ◽  
George J. Youssef ◽  
Primrose Letcher ◽  
Jacqui A. Macdonald ◽  
Lauryn J. Hagg ◽  
...  

Background Penalised regression methods are a useful atheoretical approach for both developing predictive models and selecting key indicators within an often substantially larger pool of available indicators. In comparison to traditional methods, penalised regression models improve prediction in new data by shrinking the size of coefficients and retaining those with coefficients greater than zero. However, the performance and selection of indicators depends on the specific algorithm implemented. The purpose of this study was to examine the predictive performance and feature (i.e., indicator) selection capability of common penalised logistic regression methods (LASSO, adaptive LASSO, and elastic-net), compared with traditional logistic regression and forward selection methods. Design Data were drawn from the Australian Temperament Project, a multigenerational longitudinal study established in 1983. The analytic sample consisted of 1,292 (707 women) participants. A total of 102 adolescent psychosocial and contextual indicators were available to predict young adult daily smoking. Findings Penalised logistic regression methods showed small improvements in predictive performance over logistic regression and forward selection. However, no single penalised logistic regression model outperformed the others. Elastic-net models selected more indicators than either LASSO or adaptive LASSO. Additionally, more regularised models included fewer indicators, yet had comparable predictive performance. Forward selection methods dismissed many indicators identified as important in the penalised logistic regression models. Conclusions Although overall predictive accuracy was only marginally better with penalised logistic regression methods, benefits were most clear in their capacity to select a manageable subset of indicators. Preference to competing penalised logistic regression methods may therefore be guided by feature selection capability, and thus interpretative considerations, rather than predictive performance alone.


2020 ◽  
Author(s):  
Haruyo Nakamura ◽  
Floriano Amimo ◽  
Siyan Yi ◽  
Sovannary Tuot ◽  
Tomoya Yoshida ◽  
...  

Abstract BackgroundFinancial protection is a key health system objective and an essential dimension of universal health coverage. However, it is a challenge for low- and middle-income countries, where the general tax revenue is limited, and a majority of the population is engaged in the informal economy. This study developed and validated regression models for Cambodia to predict household consumption, which allows the country to collect insurance contributions according to one’s ability to pay. This strategy would maximize the contribution revenue, optimize the government subsidy, and simultaneously ensure equity in healthcare access.MethodsThis study used nationally representative survey data collected annually between 2010 and 2017, involving 38472 households. We developed four alternative prediction models for annual household consumption: ordinary least squares (OLS) method with manually selected predictors, OLS method with stepwise backward variable selection, mixed-effects linear regression, and elastic net regression, which resulted in an adaptive least absolute shrinkage and selection operator (LASSO) regression. Household-level socioeconomic characteristics were also included as the predictors. Subsequently, we performed out-of-sample cross-validation for each model. Finally, we evaluated the prediction performance of the models using mean absolute error, root mean squared error, and mean absolute percentage error (MAPE). ResultsOverall, we found a linearly positive relationship between observed and predicted household consumptions in all four models. While the prediction performance of the four alternative models did not substantially differ, Stepwise Linear Model showed the best performance with the lowest values in all three statistical measurements, including MAPE of 1.376%. The use of regularization and the mixed effects in the regression was not particularly effective in this environment. The household consumption was better predicted for those with lower consumption, and the predictive performance declined as the consumption level increased. Although the richer household consumptions were likely to be overestimated, the trend was less noticeable in Adaptive LASSO Model.ConclusionsThis study suggests the possibility of predicting household consumption at a reasonable level with the existing survey data. Such a prediction would enable the country to raise the secured health insurance revenue equitably. The prediction model should be tested in real settings and continuously improved.


Risk Analysis ◽  
2014 ◽  
Vol 34 (6) ◽  
pp. 1112-1127 ◽  
Author(s):  
Yanlin Tang ◽  
Liya Xiang ◽  
Zhongyi Zhu

Sign in / Sign up

Export Citation Format

Share Document