ON VARIABLE SELECTION IN LINEAR REGRESSION

2002 ◽  
Vol 18 (4) ◽  
pp. 913-925 ◽  
Author(s):  
Paul Kabaila
Keyword(s):  
Linear Regression ◽  
Variable Selection ◽  
Bayesian Information Criterion ◽  
Selection Criteria ◽  
Large Data ◽  
Information Criterion ◽  
Simple Class ◽  
Linear Regressions ◽  
Data Length ◽  
Better Than

Shibata (1981, Biometrika 68, 45–54) considers data-generating mechanisms belonging to a certain class of linear regressions with errors that are independent and identically normally distributed. He compares the variable selection criteria AIC (Akaike information criterion) and BIC (Bayesian information criterion) using the following type of comparison. For each fixed possible data–generating mechanism, these criteria are compared as the data length increases. The results of this comparison have been interpreted as meaning that, in the context of the data-generating mechanisms considered by Shibata, AIC is better than BIC for large data lengths. Shibata's comparison is pointwise in the space of data–generating mechanisms (as the data length increases). Such comparisons are potentially misleading. We consider a simple class of data-generating mechanisms satisfying Shibata's assumptions and carry out a different type of comparison. For each fixed data length (possibly large) we compare the variable selection criteria for every possible data-generating mechanism in this class. According to this comparison, for this class of data-generating mechanisms no matter how large the data length AIC is not better than BIC.

scholarly journals Using Model Selection Criteria to Choose the Number of Principal Components

2021 ◽  
Vol 20 (3) ◽  
pp. 450-461
Author(s):  
Stanley L. Sclove
Keyword(s):  
Model Selection ◽  
Principal Components ◽  
Bayesian Information Criterion ◽  
Selection Criteria ◽  
Information Criterion ◽  
Information Criteria ◽  
Akaike's Information Criterion ◽  
Model Selection Criteria ◽  
Adequate Number ◽  
Number Of Principal Components

AbstractThe use of information criteria, especially AIC (Akaike’s information criterion) and BIC (Bayesian information criterion), for choosing an adequate number of principal components is illustrated.

Application of statistical model selection criteria to the Stock Synthesis assessment program

2000 ◽  
Vol 57 (9) ◽  
pp. 1784-1793 ◽  
Author(s):  
S Langitoto Helu ◽  
David B Sampson ◽  
Yanshui Yin
Keyword(s):  
Model Selection ◽  
Akaike Information Criterion ◽  
Bayesian Information Criterion ◽  
Selection Criteria ◽  
Information Criterion ◽  
Model Selection Criteria ◽  
Assessment Program ◽  
Correct Model ◽  
Complex Models ◽  
Multiple Data Sets

Statistical modeling involves building sufficiently complex models to represent the system being investigated. Overly complex models lead to imprecise parameter estimates, increase the subjective role of the modeler, and can distort the perceived characteristics of the system under investigation. One approach for controlling the tendency to increased complexity and subjectivity is to use model selection criteria that account for these factors. The effectiveness of two selection criteria was tested in an application with the stock assessment program known as Stock Synthesis. This program, which is often used on the U.S. west coast to assess the status of exploited marine fish stocks, can handle multiple data sets and mimic highly complex population dynamics. The Akaike information criterion and Schwarz's Bayesian information criterion are criteria that satisfy the fundamental principles of model selection: goodness-of-fit, parsimony, and objectivity. Their ability to select the correct model form and produce accurate estimates was evaluated in Monte Carlo experiments with the Stock Synthesis program. In general, the Akaike information criterion and the Bayesian information criterion had similar performance in selecting the correct model, and they produced comparable levels of accuracy in their estimates of ending stock biomass.

scholarly journals Performance comparison of model selection criteria by generated experimental data

2018 ◽  
Vol 16 ◽  
pp. 02006
Author(s):  
Radoslav Mavrevski ◽  
Peter Milanov ◽  
Metodi Traykov ◽  
Nevena Pencheva
Keyword(s):  
Experimental Data ◽  
Model Selection ◽  
Mathematical Models ◽  
Bayesian Information Criterion ◽  
Selection Criteria ◽  
Information Criterion ◽  
Data Fitting ◽  
Performance Comparison ◽  
Artificial Data ◽  
Criteria For Evaluation

In Bioinformatics and other areas the model selection is a process of choosing a model from set of candidate models of different classes which will provide the best balance between goodness of fitting of the data and complexity of the model. There are many criteria for evaluation of mathematical models for data fitting. The main objectives of this study are: (1) to fitting artificial experimental data with different models with increasing complexity; (2) to test whether two known criteria as Akaike’s information criterion (AIC) and Bayesian information criterion (BIC) can correctly identify the model, used to generate the artificial data and (3) to assess and compare empirically the performance of AIC and BIC.

scholarly journals COMPARISON OF LINEAR REGRESSIONS AND NEURAL NETWORKS FOR FORECASTING ELECTRICITY CONSUMPTION

2020 ◽  
Vol 16 (2) ◽  
pp. 135-140
Author(s):  
Tyas Setiyorini ◽  
Frieyadie Frieyadie
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Linear Regression ◽  
Electricity Consumption ◽  
Nonlinear Problems ◽  
Complex Data ◽  
The Neural Network ◽  
The Neural Networks ◽  
Linear Regressions ◽  
Better Than

Electricity has a major role in humans that is very necessary for daily life. Forecasting of electricity consumption can guide the government's strategy for the use and development of energy in the future. But the complex and non-linear electricity consumption dataset is a challenge. Traditional time series models in such as linear regression are unable to solve nonlinear and complex data-related problems. While neural networks can overcome the problems of nonlinear and complex data relationships. This was proven in the experiments in this study. Experiments carried out with linear regressions and neural networks on the electricity consumption dataset A and the electricity consumption dataset B. Then the RMSE results are compared on the linear regressions and neural networks of the two datasets. On the electricity consumption dataset, A obtained by RMSE of 0.032 used the linear regression, and RMSE of 0.015 used the neural network. On the electricity consumption, dataset B obtained by RMSE of 0.488 used the linear regression, and RMSE of 0.466 used the neural network. The use of neural networks shows a smaller RMSE value compared to the use of linear regressions. This shows that neural networks can overcome nonlinear problems in the electricity consumption dataset A and the electricity consumption dataset B. So that the neural networks are afforded to improve performance better than linear regressions.  This study to prove that there is a nonlinear relationship in the electricity consumption dataset used in this study, and compare which performance is better between using linear regression and neural networks.

Bayesian information criterion approximations to Bayes factors for univariate and multivariate logistic regression models

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Katharina Selig ◽  
Pamela Shaw ◽  
Donna Ankerst
Keyword(s):  
Logistic Regression ◽  
Bayesian Information Criterion ◽  
Bayes Factor ◽  
Large Data ◽  
Information Criterion ◽  
Small Samples ◽  
Multivariate Logistic Regression ◽  
Logistic Regression Models ◽  
Nested Models ◽  
Under Construction

AbstractSchwarz’s criterion, also known as the Bayesian Information Criterion or BIC, is commonly used for model selection in logistic regression due to its simple intuitive formula. For tests of nested hypotheses in independent and identically distributed data as well as in Normal linear regression, previous results have motivated use of Schwarz’s criterion by its consistent approximation to the Bayes factor (BF), defined as the ratio of posterior to prior model odds. Furthermore, under construction of an intuitive unit-information prior for the parameters of interest to test for inclusion in the nested models, previous results have shown that Schwarz’s criterion approximates the BF to higher order in the neighborhood of the simpler nested model. This paper extends these results to univariate and multivariate logistic regression, providing approximations to the BF for arbitrary prior distributions and definitions of the unit-information prior corresponding to Schwarz’s approximation. Simulations show accuracies of the approximations for small samples sizes as well as comparisons to conclusions from frequentist testing. We present an application in prostate cancer, the motivating setting for our work, which illustrates the approximation for large data sets in a practical example.

Applying Artificial Neural Networks. I. Estimating Nicotine in Tobacco from near Infrared Data

1995 ◽  
Vol 3 (3) ◽  
pp. 133-142 ◽  
Author(s):  
M. Hana ◽  
W.F. McClure ◽  
T.B. Whitaker ◽  
M. White ◽  
D.R. Bahler
Keyword(s):  
Linear Regression ◽  
Regression Model ◽  
Linear Regression Model ◽  
Near Infrared ◽  
Back Propagation ◽  
Linear Network ◽  
Data Set ◽  
Input Layer ◽  
Propagation Network ◽  
Better Than

Two artificial neural network models were used to estimate the nicotine in tobacco: (i) a back-propagation network and (ii) a linear network. The back-propagation network consisted of an input layer, an output layer and one hidden layer. The linear network consisted of an input layer and an output layer. Both networks used the generalised delta rule for learning. Performances of both networks were compared to the multiple linear regression method MLR of calibration. The nicotine content in tobacco samples was estimated for two different data sets. Data set A contained 110 near infrared (NIR) spectra each consisting of reflected energy at eight wavelengths. Data set B consisted of 200 NIR spectra with each spectrum having 840 spectral data points. The Fast Fourier transformation was applied to data set B in order to compress each spectrum into 13 Fourier coefficients. For data set A, the linear regression model gave better results followed by the back-propagation network which was followed by the linear network. The true performance of the linear regression model was better than the back-propagation and the linear networks by 14.0% and 18.1%, respectively. For data set B, the back-propagation network gave the best result followed by MLR and the linear network. Both the linear network and MLR models gave almost the same results. The true performance of the back-propagation network model was better than the MLR and linear network by 35.14%.

Sign in / Sign up

Export Citation Format

Share Document