Selecting the best stochastic systems for large scale engineering problems

Selecting a subset of the best solutions among large-scale problems is an important area of research. When the alternative solutions are stochastic in nature, then it puts more burden on the problem. The objective of this paper is to select a set that is likely to contain the actual best solutions with high probability. If the selected set contains all the best solutions, then the selection is denoted as correct selection. We are interested in maximizing the probability of this selection; P(CS). In many cases, the available computation budget for simulating the solution set in order to maximize P(CS) is limited. Therefore, instead of distributing these computational efforts equally likely among the alternatives, the optimal computing budget allocation (OCBA) procedure came to put more effort on the solutions that have more impact on the selected set. In this paper, we derive formulas of how to distribute the available budget asymptotically to find the approximation of P(CS). We then present a procedure that uses OCBA with the ordinal optimization (OO) in order to select the set of best solutions. The properties and performance of the proposed procedure are illustrated through a numerical example. Overall results indicate that the procedure is able to select a subset of the best systems with high probability of correct selection using small number of simulation samples under different parameter settings.

Fixed-Confidence, Fixed-Tolerance Guarantees for Ranking-and-Selection Procedures

Ever since the conception of the statistical ranking-and-selection (R8S) problem, a predominant approach has been the indifference-zone (IZ) formulation. Under the IZ formulation, R8S procedures are designed to provide a guarantee on the probability of correct selection (PCS) whenever the performance of the best system exceeds that of the second-best system by a specified amount. We discuss the shortcomings of this guarantee and argue that providing a guarantee on the probability of good selection (PGS)—selecting a system whose performance is within a specified tolerance of the best—is a more justifiable goal. Unfortunately, this form of fixed-confidence, fixed-tolerance guarantee has received far less attention within the simulation community. We present an overview of the PGS guarantee with the aim of reorienting the simulation community toward this goal. We examine numerous techniques for proving the PGS guarantee, including sufficient conditions under which selection and subset-selection procedures that deliver the IZ-inspired PCS guarantee also deliver the PGS guarantee.

Selecting the Best Alternative Based on Its Quantile

A value-at-risk, or quantile, is widely used as an appropriate investment selection measure for risk-conscious decision makers. We present two quantile-based sequential procedures—with and without consideration of equivalency between alternatives—for selecting the best alternative from a set of simulated alternatives. These procedures asymptotically guarantee a user-defined target probability of correct selection within a prespecified indifference zone. Experimental results demonstrate the trade-off between the indifference-zone size and the number of simulation iterations needed to render a correct selection while satisfying a desired probability of correct selection.

Algorithm for Calculating the Initial Sample Size in a Fully Sequential Ranking and Selection Procedure

In some fully sequential ranking and selection procedures, such as the KN procedure and Rinott’s procedure, some initial samples must be taken to estimate the variance. We analyze the impact of the initial sample size (ISS) on the total sample size and propose an algorithm to calculate the ISS in this type of procedure. To better illustrate our approach, we implement this algorithm on the KN procedure and propose the KN-ISS procedure. Comprehensive numerical experiments reveal that this procedure can significantly improve the efficiency compared with the KN procedure and still deliver the desired probability of correct selection.