Abstract
A surrogate model is a common tool to approximate system response at untested points for design space exploration. Adaptive sampling has been studied for improving the accuracy of surrogates iteratively by introducing additional samples (simulations and experiments). New samples are often selected based on the estimated uncertainty in the design space. While some surrogates such as kriging have readily available uncertainty models for their predictions, other surrogates do not. Consequently, there have been studies of using the process of leaving-samples-out (LSO) used in cross-validation tools to estimate prediction uncertainty, such as the universal prediction distribution (UPD). In this paper, an adaptive sampling scheme for general surrogates is proposed based on LSO, similar to cross-validation and interquartile range (IQR). Multiple submodels are first developed from LSO. The uncertainty is then estimated from the IQR of these surrogates at a given point. New samples are added iteratively at the point with maximum IQR for design space exploration. The proposed scheme is illustrated using kriging, radial basis function, and neural network surrogates. The proposed scheme is evaluated using four algebraic test functions. Multiple sets of initial samples were produced to account for randomness. For these test functions, the proposed scheme was found to be more accurate and robust than kriging with its own uncertainty model. The proposed scheme was more accurate than the UPD for three out of the four test functions. For a fixed number of samples, the IQR-based adaptive sampling also proved to be more accurate than all-at-once sampling in most cases even when the estimated uncertainty was only mildly correlated with prediction errors.
Adaptive Sampling with Varying Sampling Cost for Design Space Exploration
