scholarly journals Statistical Significance Test for Neural Network Classification

2012 ◽  
Vol 11 (1) ◽  
pp. 64
Author(s):  
Sri Rezeki ◽  
Subanar Subanar ◽  
Suryo Guritno
Keyword(s):  
Neural Networks ◽  
Model Selection ◽  
Cross Validation ◽  
Statistical Significance ◽  
Wald Test ◽  
Significance Test ◽  
Nonparametric Models ◽  
Statistical Significance Test ◽  
Selection Strategies ◽  
Iris Data

Model selection in neural networks can be guided by statistical procedures, such as hypothesis tests, informationcriteria and cross validation. Taking a statistical perspective is especially important for nonparametric models likeneural networks, because the reason for applying them is the lack of knowledge about an adequate functionalform. Many researchers have developed model selection strategies for neural networks which are based onstatistical concepts. In this paper, we focused on the model evaluation by implementing statistical significancetest. We used Wald-test to evaluate the relevance of parameters in the networks for classification problem.Parameters with no significance influence on any of the network outputs have to be removed. In general, theresults show that Wald-test work properly to determine significance of each weight from the selected model. Anempirical study by using Iris data yields all parameters in the network are significance, except bias at the firstoutput neuron.

Model selection for direct marketing: performance criteria and validation methods

2008 ◽  
Vol 26 (3) ◽  
pp. 275-292 ◽  
Author(s):  
Geng Cui ◽  
Man Leung Wong ◽  
Guichang Zhang ◽  
Lin Li
Keyword(s):  
Neural Networks ◽  
Model Selection ◽  
Cross Validation ◽  
Direct Marketing ◽  
Latent Class Models ◽  
Performance Criteria ◽  
Content Type ◽  
Validation Methods ◽  
Practical Implications ◽  
Fold Cross Validation

PurposeThe purpose of this paper is to assess the performance of competing methods and model selection, which are non‐trivial issues given the financial implications. Researchers have adopted various methods including statistical models and machine learning methods such as neural networks to assist decision making in direct marketing. However, due to the different performance criteria and validation techniques currently in practice, comparing different methods is often not straightforward.Design/methodology/approachThis study compares the performance of neural networks with that of classification and regression tree, latent class models and logistic regression using three criteria – simple error rate, area under the receiver operating characteristic curve (AUROC), and cumulative lift – and two validation methods, i.e. bootstrap and stratified k‐fold cross‐validation. Systematic experiments are conducted to compare their performance.FindingsThe results suggest that these methods vary in performance across different criteria and validation methods. Overall, neural networks outperform the others in AUROC value and cumulative lifts, and the stratified ten‐fold cross‐validation produces more accurate results than bootstrap validation.Practical implicationsTo select predictive models to support direct marketing decisions, researchers need to adopt appropriate performance criteria and validation procedures.Originality/valueThe study addresses the key issues in model selection, i.e. performance criteria and validation methods, and conducts systematic analyses to generate the findings and practical implications.

What Happened in Hazelwood: Statistics, Employment Discrimination, and the 80% Rule

1984 ◽  
Vol 9 (1) ◽  
pp. 139-186 ◽  
Author(s):  
Paul Meier ◽  
Jerome Sacks ◽  
Sandy L. Zabell
Keyword(s):  
Employment Discrimination ◽  
Equal Employment Opportunity Commission ◽  
Statistical Significance ◽  
United States Supreme Court ◽  
The United States ◽  
Significance Test ◽  
Black Teachers ◽  
Prima Facie ◽  
Statistical Significance Test ◽  
The Difference

Tests of statistical significance have increasingly been used in employment discrimination cases since the Supreme Court's decision in Hazelwood. In that case, the United States Supreme Court ruled that “in a proper case” statistical evidence can suffice for a prima facie showing of employment discrimination. The Court also discussed the use of a binomial significance test to assess whether the difference between the proportion of black teachers employed by the Hazelwood School District and the proportion of black teachers in the relevant labor market was substantial enough to indicate discrimination. The Equal Employment Opportunity Commission has proposed a somewhat stricter standard for evaluating how substantial a difference must be to constitute evidence of discrimination. Under the so-called 80% rule promulgated by the EEOC, the difference must not only be statistically significant, but the hire rate for the allegedly discriminated group must also be less than 80% of the rate for the favored group. This article argues that a binomial statistical significance test standing alone is unsatisfactory for evaluating allegations of discrimination because many of the assumptions on which such tests are based are inapplicable to employment settings; the 80% rule is a more appropriate standard for evaluating whether a difference in hire rates should be treated as a prima facie showing of discrimination.

Sign in / Sign up

Export Citation Format

Share Document