An iterative feature selection procedure for a classification problem based on the method of logical analysis of data

An iterative procedure for selecting features for classifying observations is proposed. The main principles of the proposed iterative procedure are ranking and selection of features according to the frequency of their use when constructing logical patterns based on the method of logical analysis of data. The empirical confirmation of the expediency of this procedure is given.

A Practical Approach to Subset Selection for Multi-objective Optimization via Simulation

We describe a practical two-stage algorithm, BootComp, for multi-objective optimization via simulation. Our algorithm finds a subset of good designs that a decision-maker can compare to identify the one that works best when considering all aspects of the system, including those that cannot be modeled. BootComp is designed to be straightforward to implement by a practitioner with basic statistical knowledge in a simulation package that does not support sequential ranking and selection. These requirements restrict us to a two-stage procedure that works with any distributions of the outputs and allows for the use of common random numbers. Comparisons with sequential ranking and selection methods suggest that it performs well, and we also demonstrate its use analyzing a real simulation aiming to determine the optimal ward configuration for a UK hospital.

Replicated Computational Results (RCR) Report for“A Practical Approach to Subset Selection for Multi-Objective Optimization via Simulation”

In “A Practical Approach to Subset Selection for Multi-Objective Optimization via Simulation,” Currie and Monks propose an algorithm for multi-objective simulation-based optimization. In contrast to sequential ranking and selection schemes, their algorithm follows a two-stage scheme. The approach is evaluated by comparing the results to those obtained using the existing OCBA-m algorithm for synthetic problems and for a hospital ward configuration problem. The authors provide the Python code used in the experiments in the form of Jupyter notebooks. The code successfully reproduced the results shown in the article.

Predicting Risk of Mortality in Pediatric ICU Based on Ensemble Step-Wise Feature Selection

Background. Prediction of mortality risk in intensive care units (ICU) is an important task. Data-driven methods such as scoring systems, machine learning methods, and deep learning methods have been investigated for a long time. However, few data-driven methods are specially developed for pediatric ICU. In this paper, we aim to amend this gap—build a simple yet effective linear machine learning model from a number of hand-crafted features for mortality prediction in pediatric ICU. Methods. We use a recently released publicly available pediatric ICU dataset named pediatric intensive care (PIC) from Children’s Hospital of Zhejiang University School of Medicine in China. Unlike previous sophisticated machine learning methods, we want our method to keep simple that can be easily understood by clinical staffs. Thus, an ensemble step-wise feature ranking and selection method is proposed to select a small subset of effective features from the entire feature set. A logistic regression classifier is built upon selected features for mortality prediction. Results. The final predictive linear model with 11 features achieves a 0.7531 ROC-AUC score on the hold-out test set, which is comparable with a logistic regression classifier using all 397 features (0.7610 ROC-AUC score) and is higher than the existing well known pediatric mortality risk scorer PRISM III (0.6895 ROC-AUC score). Conclusions. Our method improves feature ranking and selection by utilizing an ensemble method while keeping a simple linear form of the predictive model and therefore achieves better generalizability and performance on mortality prediction in pediatric ICU.

Knockout-Tournament Procedures for Large-Scale Ranking and Selection in Parallel Computing Environments

On one hand, large-scale ranking and selection (R&S) problems require a large amount of computation. On the other hand, parallel computing environments that provide a large capacity for computation are becoming prevalent today, and they are accessible by ordinary users. Therefore, solving large-scale R&S problems in parallel computing environments has emerged as an important research topic in recent years. However, directly implementing traditional stagewise procedures and fully sequential procedures in parallel computing environments may encounter problems because either the procedures require too many simulation observations or the procedures’ selection structures induce too many comparisons and too frequent communications among the processors. In this paper, inspired by the knockout-tournament arrangement of tennis Grand Slam tournaments, we develop new R&S procedures to solve large-scale problems in parallel computing environments. We show that no matter whether the variances of the alternatives are known or not, our procedures can theoretically achieve the lowest growth rate on the expected total sample size with respect to the number of alternatives and thus, are optimal in rate. Moreover, common random numbers can be easily adopted in our procedures to further reduce the total sample size. Meanwhile, the comparison time in our procedures is negligible compared with the simulation time, and our procedures barely request for communications among the processors.

Review on ranking and selection: A new perspective

In this paper, we briefly review the development of ranking and selection (R&S) in the past 70 years, especially the theoretical achievements and practical applications in the past 20 years. Different from the frequentist and Bayesian classifications adopted by Kim and Nelson (2006b) and Chick (2006) in their review articles, we categorize existing R&S procedures into fixed-precision and fixed-budget procedures, as in Hunter and Nelson (2017). We show that these two categories of procedures essentially differ in the underlying methodological formulations, i.e., they are built on hypothesis testing and dynamic programming, respectively. In light of this variation, we review in detail some well-known procedures in the literature and show how they fit into these two formulations. In addition, we discuss the use of R&S procedures in solving various practical problems and propose what we think are the important research questions in the field.

Ranking and Selection with Covariates for Personalized Decision Making

We consider a problem of ranking and selection via simulation in the context of personalized decision making, in which the best alternative is not universal, but varies as a function of some observable covariates. The goal of ranking and selection with covariates (R&S-C) is to use simulation samples to obtain a selection policy that specifies the best alternative with a certain statistical guarantee for subsequent individuals upon observing their covariates. A linear model is proposed to capture the relationship between the mean performance of an alternative and the covariates. Under the indifference-zone formulation, we develop two-stage procedures for both homoscedastic and heteroscedastic simulation errors, respectively, and prove their statistical validity in terms of average probability of correct selection. We also generalize the well-known slippage configuration and prove that the generalized slippage configuration is the least favorable configuration for our procedures. Extensive numerical experiments are conducted to investigate the performance of the proposed procedures, the experimental design issue, and the robustness to the linearity assumption. Finally, we demonstrate the usefulness of R&S-C via a case study of selecting the best treatment regimen in the prevention of esophageal cancer. We find that by leveraging disease-related personal information, R&S-C can substantially improve patients’ expected quality-adjusted life years by providing a patient-specific treatment regimen.