Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy

In the machine learning literature we can find numerous methods to solve classification problems. We propose two new performance measures to analyze such methods. These measures are defined by using the concept of proportional reduction of classification error with respect to three benchmark classifiers, the random and two intuitive classifiers which are based on how a non-expert person could realize classification simply by applying a frequentist approach. We show that these three simple methods are closely related to different aspects of the entropy of the dataset. Therefore, these measures account somewhat for entropy in the dataset when evaluating the performance of classifiers. This allows us to measure the improvement in the classification results compared to simple methods, and at the same time how entropy affects classification capacity. To illustrate how these new performance measures can be used to analyze classifiers taking into account the entropy of the dataset, we carry out an intensive experiment in which we use the well-known J48 algorithm, and a UCI repository dataset on which we have previously selected a subset of the most relevant attributes. Then we carry out an extensive experiment in which we consider four heuristic classifiers, and 11 datasets.

Reporting Bayesian Results

Because of the different philosophy of Bayesian statistics, where parameters are random variables and data are considered fixed, the analysis and presentation of results will differ from that of frequentist statistics. Most importantly, the probabilities that a parameter is in certain regions of the parameter space are crucial quantities in Bayesian statistics that are not calculable (or considered important) in the frequentist approach that is the basis of much of traditional statistics. In this article, I discuss the implications of these differences for presentation of the results of Bayesian analyses. In doing so, I present more detailed guidelines than are usually provided and explain the rationale for my suggestions.

Improving Graduation Rate Estimates Using Regularly Updating Multi-Level Absorbing Markov Chains

American universities use a procedure based on a rolling six-year graduation rate to calculate statistics regarding their students’ final educational outcomes (graduating or not graduating). As an alternative to the six-year graduation rate method, many studies have applied absorbing Markov chains for estimating graduation rates. In both cases, a frequentist approach is used. For the standard six-year graduation rate method, the frequentist approach corresponds to counting the number of students who finished their program within six years and dividing by the number of students who entered that year. In the case of absorbing Markov chains, the frequentist approach is used to compute the underlying transition matrix, which is then used to estimate the graduation rate. In this paper, we apply a sensitivity analysis to compare the performance of the standard six-year graduation rate method with that of absorbing Markov chains. Through the analysis, we highlight significant limitations with regards to the estimation accuracy of both approaches when applied to small sample sizes or cohorts at a university. Additionally, we note that the Absorbing Markov chain method introduces a significant bias, which leads to an underestimation of the true graduation rate. To overcome both these challenges, we propose and evaluate the use of a regularly updating multi-level absorbing Markov chain (RUML-AMC) in which the transition matrix is updated year to year. We empirically demonstrate that the proposed RUML-AMC approach nearly eliminates estimation bias while reducing the estimation variation by more than 40%, especially for populations with small sample sizes.

A Comparison Between Bayesian and Frequentist Approach in the Analysis of Risk Factors for Female Cardiovascular Disease Patients in Malaysia

Cardiovascular disease (CVD) is the number one killer among women in Malaysia and globally, with over two million deaths each year. In this study, two modelling approaches namely Bayesian approach and frequentist approach were considered to identify associated risk factors in CVD among female patients presenting with ST Elevation Myocardial Infarction (STEMI) and to obtain feasible model to fit the data. Comparisons were made to find the best model. A total of 874 STEMI female patients from 18 participating hospitals across Malaysia in the National Cardiovascular Disease Database-Acute Coronary Syndrome (NCVD-ACS) registry year 2006-2013 were analysed. Univariate and multivariate analysis were performed for both Bayesian and frequentist approaches. Six variables namely smoking, dyslipidaemia, myocardial infarction (MI), renal disease, Killip class and age group were found to be significant at the multivariate analysis. The standard errors obtained from the Bayesian approach were much smaller than the frequentist approach. Also, the model fit using Bayesian approach was much better than the frequentist as the deviance value produced by the Bayesian approach was smaller. The Bayesian analysis provides a better alternative to the frequentist approach in the analysis of the risk factors associated with mortality among female CVD patients.

Bayesian statistics

This chapter describes the Bayesian approach to statistical analysis in contrast to the frequentist approach. It discusses how clinicians often use a Bayesian approach in interpreting clinical findings and forming management plans. It describes how Bayesian methods work including a description of prior and posterior distributions. The chapter outlines the role and choice of prior distributions and how they are combined with the data collected to provide an updated estimate of the unknown quantity being studied. It includes examples of the use of Bayesian methods in medicine, and discusses the pros and cons of the Bayesian approach compared to the frequentist approach. Finally, guidance is given on how to read and interpret Bayesian analyses in the medical literature.