scholarly journals Explaining inference queries with bayesian optimization

2021 ◽  
Vol 14 (11) ◽  
pp. 2576-2585
Author(s):  
Brandon Lockhart ◽  
Jinglin Peng ◽  
Weiyuan Wu ◽  
Jiannan Wang ◽  
Eugene Wu
Keyword(s):  
Global Optimum ◽  
Black Box ◽  
Training Data ◽  
Bayesian Optimization ◽  
Categorical Variables ◽  
Warm Start ◽  
Query Result ◽  
Real World Datasets ◽  
Data Inference ◽  
Sql Query

Obtaining an explanation for an SQL query result can enrich the analysis experience, reveal data errors, and provide deeper insight into the data. Inference query explanation seeks to explain unexpected aggregate query results on inference data; such queries are challenging to explain because an explanation may need to be derived from the source, training, or inference data in an ML pipeline. In this paper, we model an objective function as a black-box function and propose BOExplain, a novel framework for explaining inference queries using Bayesian optimization (BO). An explanation is a predicate defining the input tuples that should be removed so that the query result of interest is significantly affected. BO --- a technique for finding the global optimum of a black-box function --- is used to find the best predicate. We develop two new techniques (individual contribution encoding and warm start) to handle categorical variables. We perform experiments showing that the predicates found by BOExplain have a higher degree of explanation compared to those found by the state-of-the-art query explanation engines. We also show that BOExplain is effective at deriving explanations for inference queries from source and training data on a variety of real-world datasets. BOExplain is open-sourced as a Python package at https://github.com/sfu-db/BOExplain.

scholarly journals Bayesian optimization with partially specified queries

2022 ◽  
Author(s):  
Shogo Hayashi ◽  
Junya Honda ◽  
Hisashi Kashima
Keyword(s):  
Black Box ◽  
Bayesian Optimization ◽  
Unknown Input ◽  
Bandit Problem ◽  
Test Functions ◽  
Significant Cost ◽  
Posterior Sampling ◽  
Regret Bounds ◽  
Input Variables ◽  
Real World Datasets

AbstractBayesian optimization (BO) is an approach to optimizing an expensive-to-evaluate black-box function and sequentially determines the values of input variables to evaluate the function. However, it is expensive and in some cases becomes difficult to specify values for all input variables, for example, in outsourcing scenarios where production of input queries with many input variables involves significant cost. In this paper, we propose a novel Gaussian process bandit problem, BO with partially specified queries (BOPSQ). In BOPSQ, unlike the standard BO setting, a learner specifies only the values of some input variables, and the values of the unspecified input variables are randomly determined according to a known or unknown distribution. We propose two algorithms based on posterior sampling for cases of known and unknown input distributions. We further derive their regret bounds that are sublinear for popular kernels. We demonstrate the effectiveness of the proposed algorithms using test functions and real-world datasets.

An Approach to Bayesian Optimization for Design Feasibility Check on Discontinuous Black-Box Functions

2020 ◽  
Author(s):  
Arpan Biswas ◽  
Christopher Hoyle
Keyword(s):  
Objective Function ◽  
Prior Knowledge ◽  
Design Space ◽  
Black Box ◽  
Training Data ◽  
Bayesian Optimization ◽  
Design Parameters ◽  
Classification Error ◽  
True Solution ◽  
Classification Tool

Abstract The paper presents a novel approach to apply Bayesian Optimization (BO) in predicting an unknown constraint boundary, also representing the discontinuity of an unknown function, for a feasibility check on the design space, thereby representing a classification tool to discern between a feasible and infeasible region. Bayesian optimization is an emerging field of study in the Sequential Design Methods where we learn and update our knowledge from prior evaluated designs, and proceed to the selection of new designs for future evaluation. It has been considered as a low-cost global optimization tool for design problems having expensive black-box objective functions. However, BO is mostly suited to problems with the assumption of a continuous objective function, and does not guarantee true convergence if the objective function has a discontinuity. This is because of the insufficient knowledge of the BO about the nature of the discontinuity of the unknown true function. Therefore, in this paper, we have proposed to predict the discontinuity of the objective function using a BO algorithm which can be considered as the pre-stage before optimizing the same unknown objective function. The proposed approach has been implemented in a thin tube design with the risk of creep-fatigue failure under constant loading of temperature and pressure. The stated risk depends on the location of the designs in terms of safe and unsafe regions, where the discontinuities lie at the transitions between those regions; therefore, the discontinuity has also been treated as an unknown constraint. The paper focuses on developing BO framework with maximizing the reformulated objective function on the same design space to predict the transition regions as a design methodology or classification tool between safe and unsafe designs, where we start with very limited data or no prior knowledge and then iteratively focus on sampling most designs near the transition region through better prior knowledge (training data) and thereby increase the accuracy of prediction to the true boundary while minimizing the number of expensive function evaluations. The converged model has been compared with the true solution for different design parameters and the results provided a classification error rate and function evaluations at an average of < 1% and ∼150, respectively. The results in this paper show some future research directions in extending the application of BO and considered as the proof of concept on large scale problem of complex diffusion bonded hybrid Compact Heat Exchangers.

scholarly journals High-dimensional Bayesian optimization using low-dimensional feature spaces

2020 ◽  
Vol 109 (9-10) ◽  
pp. 1925-1943 ◽  
Author(s):  
Riccardo Moriconi ◽  
Marc Peter Deisenroth ◽  
K. S. Sesh Kumar
Keyword(s):  
Optimization Problem ◽  
Feature Space ◽  
Dimensional Subspace ◽  
Global Optimum ◽  
Black Box ◽  
Fine Tuning ◽  
Bayesian Optimization ◽  
Feature Spaces ◽  
Low Dimensional ◽  
Lower Dimensional

Abstract Bayesian optimization (BO) is a powerful approach for seeking the global optimum of expensive black-box functions and has proven successful for fine tuning hyper-parameters of machine learning models. However, BO is practically limited to optimizing 10–20 parameters. To scale BO to high dimensions, we usually make structural assumptions on the decomposition of the objective and/or exploit the intrinsic lower dimensionality of the problem, e.g. by using linear projections. We could achieve a higher compression rate with nonlinear projections, but learning these nonlinear embeddings typically requires much data. This contradicts the BO objective of a relatively small evaluation budget. To address this challenge, we propose to learn a low-dimensional feature space jointly with (a) the response surface and (b) a reconstruction mapping. Our approach allows for optimization of BO’s acquisition function in the lower-dimensional subspace, which significantly simplifies the optimization problem. We reconstruct the original parameter space from the lower-dimensional subspace for evaluating the black-box function. For meaningful exploration, we solve a constrained optimization problem.

An Approach to Bayesian Optimization in Optimizing Weighted Tchebycheff Multi-Objective Black-Box Functions

2020 ◽  
Author(s):  
Arpan Biswas ◽  
Claudio Fuentes ◽  
Christopher Hoyle
Keyword(s):  
Process Model ◽  
Large Scale ◽  
Low Cost ◽  
Pareto Frontier ◽  
Black Box ◽  
Training Data ◽  
Bayesian Optimization ◽  
Future Research ◽  
Objective Functions ◽  
Multi Objective

Abstract Bayesian optimization (BO) is a low-cost global optimization tool for expensive black-box objective functions, where we learn from prior evaluated designs, update a posterior surrogate Gaussian process model, and select new designs for future evaluation using an acquisition function. This research focuses upon developing a BO model with multiple black-box objective functions. In the standard multi-objective optimization problem, the weighted Tchebycheff method is efficiently used to find both convex and non-convex Pareto frontier. This approach requires knowledge of utopia values before we start optimization. However, in the BO framework, since the functions are expensive to evaluate, it is very expensive to obtain the utopia values as a priori knowledge. Therefore, in this paper, we develop a Multi-Objective Bayesian Optimization (MO-BO) framework where we calibrate with Multiple Linear Regression (MLR) models to estimate the utopia value for each objective as a function of design input variables; the models are updated iteratively with sampled training data from the proposed multi-objective BO. The iteratively estimated mean utopia values are used to formulate the weighted Tchebycheff multi-objective acquisition function. The proposed approach is implemented in optimizing a thin tube design under constant loading of temperature and pressure, with multiple objectives such as minimizing the risk of creep-fatigue failure and design cost along with risk-based and manufacturing constraints. Finally, the model accuracy with and without MLR-based calibration is compared to the true Pareto solutions. The results show potential broader impacts, future research directions for further improving the proposed MO-BO model, and potential extensions to the application of large-scale design problems.

A MO-BAYESIAN OPTIMIZATION APPROACH USING THE WEIGHTED TCHEBYCHEFF METHOD

2021 ◽  
pp. 1-30
Author(s):  
Arpan Biswas ◽  
Claudio Fuentes ◽  
Christopher Hoyle
Keyword(s):  
Process Model ◽  
Low Cost ◽  
Black Box ◽  
Training Data ◽  
Bayesian Optimization ◽  
Optimization Approach ◽  
Objective Functions ◽  
Multi Objective ◽  
Input Variables ◽  
Future Evaluation

Abstract Bayesian optimization (BO) is a low-cost global optimization tool for expensive black-box objective functions, where we learn from prior evaluated designs, update a posterior surrogate Gaussian process model, and select new designs for future evaluation using an acquisition function. This research focuses upon developing a BO model with multiple black-box objective functions. In the standard Multi-Objective optimization (MOO) problem, the weighted Tchebycheff method is efficiently used to find both convex and non-convex Pareto frontiers. This approach requires knowledge of utopia values before we start optimization. However, in the BO framework, since the functions are expensive to evaluate, it is very expensive to obtain the utopia values as a prior knowledge. Therefore, in this paper, we develop a MO-BO framework where we calibrate with multiple linear regression (MLR) models to estimate the utopia value for each objective as a function of design input variables; the models are updated iteratively with sampled training data from the proposed multi-objective BO. This iteratively estimated mean utopia values is used to formulate the weighted Tchebycheff multi-objective acquisition function. The proposed approach is implemented in optimizing thin tube geometries under constant loading of temperature and pressure, with minimizing the risk of creep-fatigue failure and design cost, along with risk-based and manufacturing constraints. Finally, the model accuracy with frequentist, Bayesian and without MLR-based calibration are compared to true Pareto solutions.

scholarly journals Multi-fidelity black-box optimization for time-optimal quadrotor maneuvers

2021 ◽  
pp. 027836492110333
Author(s):  
Gilhyun Ryou ◽  
Ezra Tal ◽  
Sertac Karaman
Keyword(s):  
Real World ◽  
High Speed ◽  
Analytical Approximation ◽  
Black Box ◽  
Bayesian Optimization ◽  
Optimization Framework ◽  
Quadrotor Aircraft ◽  
Time Optimal ◽  
Feasibility Constraints ◽  
Generation Problem

We consider the problem of generating a time-optimal quadrotor trajectory for highly maneuverable vehicles, such as quadrotor aircraft. The problem is challenging because the optimal trajectory is located on the boundary of the set of dynamically feasible trajectories. This boundary is hard to model as it involves limitations of the entire system, including complex aerodynamic and electromechanical phenomena, in agile high-speed flight. In this work, we propose a multi-fidelity Bayesian optimization framework that models the feasibility constraints based on analytical approximation, numerical simulation, and real-world flight experiments. By combining evaluations at different fidelities, trajectory time is optimized while the number of costly flight experiments is kept to a minimum. The algorithm is thoroughly evaluated for the trajectory generation problem in two different scenarios: (1) connecting predetermined waypoints; (2) planning in obstacle-rich environments. For each scenario, we conduct both simulation and real-world flight experiments at speeds up to 11 m/s. Resulting trajectories were found to be significantly faster than those obtained through minimum-snap trajectory planning.

scholarly journals Global optimization based on active preference learning with radial basis functions

2020 ◽  
Author(s):  
Alberto Bemporad ◽  
Dario Piga
Keyword(s):  
Global Optimization ◽  
Objective Function ◽  
Decision Maker ◽  
Optimization Problems ◽  
Global Optimum ◽  
Bayesian Optimization ◽  
Preference Learning ◽  
Global Optimizer ◽  
Radial Basis ◽  
Decision Vector

AbstractThis paper proposes a method for solving optimization problems in which the decision-maker cannot evaluate the objective function, but rather can only express a preference such as “this is better than that” between two candidate decision vectors. The algorithm described in this paper aims at reaching the global optimizer by iteratively proposing the decision maker a new comparison to make, based on actively learning a surrogate of the latent (unknown and perhaps unquantifiable) objective function from past sampled decision vectors and pairwise preferences. A radial-basis function surrogate is fit via linear or quadratic programming, satisfying if possible the preferences expressed by the decision maker on existing samples. The surrogate is used to propose a new sample of the decision vector for comparison with the current best candidate based on two possible criteria: minimize a combination of the surrogate and an inverse weighting distance function to balance between exploitation of the surrogate and exploration of the decision space, or maximize a function related to the probability that the new candidate will be preferred. Compared to active preference learning based on Bayesian optimization, we show that our approach is competitive in that, within the same number of comparisons, it usually approaches the global optimum more closely and is computationally lighter. Applications of the proposed algorithm to solve a set of benchmark global optimization problems, for multi-objective optimization, and for optimal tuning of a cost-sensitive neural network classifier for object recognition from images are described in the paper. MATLAB and a Python implementations of the algorithms described in the paper are available at http://cse.lab.imtlucca.it/~bemporad/glis.

scholarly journals Indirectly Supervised Anomaly Detection of Clinically Meaningful Health Events from Smart Home Data

2021 ◽  
Vol 12 (2) ◽  
pp. 1-18
Author(s):  
Jessamyn Dahmen ◽  
Diane J. Cook
Keyword(s):  
Anomaly Detection ◽  
Time Series Data ◽  
Bayesian Optimization ◽  
Detection Methods ◽  
Series Data ◽  
Detection Techniques ◽  
Health Events ◽  
Warm Start ◽  
Health Related ◽  
Unsupervised Algorithms

Anomaly detection techniques can extract a wealth of information about unusual events. Unfortunately, these methods yield an abundance of findings that are not of interest, obscuring relevant anomalies. In this work, we improve upon traditional anomaly detection methods by introducing Isudra, an Indirectly Supervised Detector of Relevant Anomalies from time series data. Isudra employs Bayesian optimization to select time scales, features, base detector algorithms, and algorithm hyperparameters that increase true positive and decrease false positive detection. This optimization is driven by a small amount of example anomalies, driving an indirectly supervised approach to anomaly detection. Additionally, we enhance the approach by introducing a warm-start method that reduces optimization time between similar problems. We validate the feasibility of Isudra to detect clinically relevant behavior anomalies from over 2M sensor readings collected in five smart homes, reflecting 26 health events. Results indicate that indirectly supervised anomaly detection outperforms both supervised and unsupervised algorithms at detecting instances of health-related anomalies such as falls, nocturia, depression, and weakness.

MetaTP

2021 ◽  
Vol 5 (3) ◽  
pp. 1-28
Author(s):  
Weida Zhong ◽  
Qiuling Suo ◽  
Abhishek Gupta ◽  
Xiaowei Jia ◽  
Chunming Qiao ◽  
...  
Keyword(s):  
Time Series ◽  
Large Scale ◽  
Multivariate Time Series ◽  
Modern Society ◽  
Training Data ◽  
Traffic Prediction ◽  
Temporal Prediction ◽  
Reference Space ◽  
Meta Learning ◽  
Real World Datasets

With the popularity of smartphones, large-scale road sensing data is being collected to perform traffic prediction, which is an important task in modern society. Due to the nature of the roving sensors on smartphones, the collected traffic data which is in the form of multivariate time series, is often temporally sparse and unevenly distributed across regions. Moreover, different regions can have different traffic patterns, which makes it challenging to adapt models learned from regions with sufficient training data to target regions. Given that many regions may have very sparse data, it is also impossible to build individual models for each region separately. In this paper, we propose a meta-learning based framework named MetaTP to overcome these challenges. MetaTP has two key parts, i.e., basic traffic prediction network (base model) and meta-knowledge transfer. In base model, a two-layer interpolation network is employed to map original time series onto uniformly-spaced reference time points, so that temporal prediction can be effectively performed in the reference space. The meta-learning framework is employed to transfer knowledge from source regions with a large amount of data to target regions with a few data examples via fast adaptation, in order to improve model generalizability on target regions. Moreover, we use two memory networks to capture the global patterns of spatial and temporal information across regions. We evaluate the proposed framework on two real-world datasets, and experimental results show the effectiveness of the proposed framework.

scholarly journals Zero-Shot Feature Selection via Transferring Supervised Knowledge

2021 ◽  
Vol 17 (2) ◽  
pp. 1-20
Author(s):  
Zheng Wang ◽  
Qiao Wang ◽  
Tingzhang Zhao ◽  
Chaokun Wang ◽  
Xiaojun Ye
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Real World ◽  
Rapid Growth ◽  
Learning Systems ◽  
Training Data ◽  
Effective Technique ◽  
Supervised Methods ◽  
Real World Datasets

Feature selection, an effective technique for dimensionality reduction, plays an important role in many machine learning systems. Supervised knowledge can significantly improve the performance. However, faced with the rapid growth of newly emerging concepts, existing supervised methods might easily suffer from the scarcity and validity of labeled data for training. In this paper, the authors study the problem of zero-shot feature selection (i.e., building a feature selection model that generalizes well to “unseen” concepts with limited training data of “seen” concepts). Specifically, they adopt class-semantic descriptions (i.e., attributes) as supervision for feature selection, so as to utilize the supervised knowledge transferred from the seen concepts. For more reliable discriminative features, they further propose the center-characteristic loss which encourages the selected features to capture the central characteristics of seen concepts. Extensive experiments conducted on various real-world datasets demonstrate the effectiveness of the method.

Sign in / Sign up

Export Citation Format

Share Document