scholarly journals High-dimensional Bayesian optimization using low-dimensional feature spaces

2020 ◽  
Vol 109 (9-10) ◽  
pp. 1925-1943 ◽  
Author(s):  
Riccardo Moriconi ◽  
Marc Peter Deisenroth ◽  
K. S. Sesh Kumar
Keyword(s):  
Optimization Problem ◽  
Feature Space ◽  
Dimensional Subspace ◽  
Global Optimum ◽  
Black Box ◽  
Fine Tuning ◽  
Bayesian Optimization ◽  
Feature Spaces ◽  
Low Dimensional ◽  
Lower Dimensional

Abstract Bayesian optimization (BO) is a powerful approach for seeking the global optimum of expensive black-box functions and has proven successful for fine tuning hyper-parameters of machine learning models. However, BO is practically limited to optimizing 10–20 parameters. To scale BO to high dimensions, we usually make structural assumptions on the decomposition of the objective and/or exploit the intrinsic lower dimensionality of the problem, e.g. by using linear projections. We could achieve a higher compression rate with nonlinear projections, but learning these nonlinear embeddings typically requires much data. This contradicts the BO objective of a relatively small evaluation budget. To address this challenge, we propose to learn a low-dimensional feature space jointly with (a) the response surface and (b) a reconstruction mapping. Our approach allows for optimization of BO’s acquisition function in the lower-dimensional subspace, which significantly simplifies the optimization problem. We reconstruct the original parameter space from the lower-dimensional subspace for evaluating the black-box function. For meaningful exploration, we solve a constrained optimization problem.

scholarly journals Eigencages: Learning a Latent Space of Porous Cage Molecules

2018 ◽  
Author(s):  
Arni Sturluson ◽  
Melanie T. Huynh ◽  
Arthur H. P. York ◽  
Cory Simon
Keyword(s):  
Dimensional Subspace ◽  
Singular Value ◽  
Adsorption Properties ◽  
3D Image ◽  
Latent Space ◽  
Cage Molecules ◽  
Gas Molecules ◽  
Low Dimensional ◽  
Value Decomposition ◽  
Lower Dimensional

<div>Porous organic cage molecules harbor nano-sized cavities that can selectively adsorb gas molecules, lending them applications in separations and sensing. The geometry of</div><div>the cavity strongly influences their adsorptive selectivity. </div><div><br></div><div>For comparing cages and predicting their adsorption properties, we embed/encode a set of 74 porous organic</div><div>cage molecules into a low-dimensional, latent “cage space” on the basis of their intrinsic porosity. </div><div><br></div><div>We first computationally scan each cage to generate a 3D image of its porosity. Leveraging the singular value decomposition, in an unsupervised manner, we then learn across all cages an approximate, lower-dimensional subspace in which the 3D porosity images lay. The “eigencages” are the set of orthogonal characteristic 3D porosity images that span this lower-dimensional subspace, ordered in terms of importance. A latent representation/encoding of each cage follows from expressing it as a combination of the eigencages. </div><div><br></div><div>We show that the learned encoding captures salient features of the cavities of porous cages and is predictive of properties of the cages that arise from cavity shape.</div>

Git Recognition with Incomplete GEI Based on Random Forests

2014 ◽  
Vol 519-520 ◽  
pp. 661-666
Author(s):  
Qing Zhu ◽  
Jie Zhang
Keyword(s):  
Random Forests ◽  
Gait Recognition ◽  
Feature Space ◽  
High Dimensional ◽  
Random Forest Algorithm ◽  
Recognition Method ◽  
Locality Preserving ◽  
Feature Spaces ◽  
Feature Subspace ◽  
Low Dimensional

Abstract. This paper proposes an incomplete GEI gait recognition method based on Random Forests. There are numerous methods exist for git recognition,but they all lead to high dimensional feature spaces. To address the problem of high dimensional feature space, we propose the use of the Random Forest algorithm to rank features' importance . In order to efficiently search throughout subspaces, we apply a backward feature elimination search strategy.This demonstrate static areas of a GEI also contain useful information.Then, we project the selected feature to a low-dimensional feature subspace via the newly proposed two-dimensional locality preserving projections (2DLPP) method.Asa sequence,we further improve the discriminative power of the extracted features. Experimental results on the CASIA gait database demonstrate the effectiveness of the proposed method.

scholarly journals A real negative selection algorithm with evolutionary preference for anomaly detection

Open Physics ◽  
2017 ◽  
Vol 15 (1) ◽  
pp. 121-134 ◽  
Author(s):  
Tao Yang ◽  
Wen Chen ◽  
Tao Li
Keyword(s):  
Negative Selection ◽  
Feature Space ◽  
Dimensional Subspace ◽  
Experimental Result ◽  
High Dimensional ◽  
Selection Algorithm ◽  
Negative Selection Algorithm ◽  
Preference Theory ◽  
Low Dimensional ◽  
Selection Algorithms

AbstractTraditional real negative selection algorithms (RNSAs) adopt the estimated coverage (c0) as the algorithm termination threshold, and generate detectors randomly. With increasing dimensions, the data samples could reside in the low-dimensional subspace, so that the traditional detectors cannot effectively distinguish these samples. Furthermore, in high-dimensional feature space,c0cannot exactly reflect the detectors set coverage rate for the nonself space, and it could lead the algorithm to be terminated unexpectedly when the number of detectors is insufficient. These shortcomings make the traditional RNSAs to perform poorly in high-dimensional feature space. Based upon “evolutionary preference” theory in immunology, this paper presents a real negative selection algorithm with evolutionary preference (RNSAP). RNSAP utilizes the “unknown nonself space”, “low-dimensional target subspace” and “known nonself feature” as the evolutionary preference to guide the generation of detectors, thus ensuring the detectors can cover the nonself space more effectively. Besides, RNSAP uses redundancy to replacec0as the termination threshold, in this way RNSAP can generate adequate detectors under a proper convergence rate. The theoretical analysis and experimental result demonstrate that, compared to the classical RNSA (V-detector), RNSAP can achieve a higher detection rate, but with less detectors and computing cost.

scholarly journals Eigencages: Learning a Latent Space of Porous Cage Molecules

2018 ◽  
Author(s):  
Arni Sturluson ◽  
Melanie T. Huynh ◽  
Arthur H. P. York ◽  
Cory Simon
Keyword(s):  
Dimensional Subspace ◽  
Singular Value ◽  
Adsorption Properties ◽  
3D Image ◽  
Latent Space ◽  
Cage Molecules ◽  
Gas Molecules ◽  
Low Dimensional ◽  
Value Decomposition ◽  
Lower Dimensional

<div>Porous organic cage molecules harbor nano-sized cavities that can selectively adsorb gas molecules, lending them applications in separations and sensing. The geometry of</div><div>the cavity strongly influences their adsorptive selectivity. </div><div><br></div><div>For comparing cages and predicting their adsorption properties, we embed/encode a set of 74 porous organic</div><div>cage molecules into a low-dimensional, latent “cage space” on the basis of their intrinsic porosity. </div><div><br></div><div>We first computationally scan each cage to generate a 3D image of its porosity. Leveraging the singular value decomposition, in an unsupervised manner, we then learn across all cages an approximate, lower-dimensional subspace in which the 3D porosity images lay. The “eigencages” are the set of orthogonal characteristic 3D porosity images that span this lower-dimensional subspace, ordered in terms of importance. A latent representation/encoding of each cage follows from expressing it as a combination of the eigencages. </div><div><br></div><div>We show that the learned encoding captures salient features of the cavities of porous cages and is predictive of properties of the cages that arise from cavity shape.</div>

scholarly journals Eigencages: Learning a Latent Space of Porous Cage Molecules

2018 ◽  
Author(s):  
Arni Sturluson ◽  
Melanie T. Huynh ◽  
Arthur H. P. York ◽  
Cory Simon
Keyword(s):  
Dimensional Subspace ◽  
Singular Value ◽  
Adsorption Properties ◽  
3D Image ◽  
Latent Space ◽  
Cage Molecules ◽  
Gas Molecules ◽  
Low Dimensional ◽  
Value Decomposition ◽  
Lower Dimensional

<div>Porous organic cage molecules harbor nano-sized cavities that can selectively adsorb gas molecules, lending them applications in separations and sensing. The geometry of the cavity strongly influences adsorptive selectivity.</div><div><br></div><div>For comparing cages and predicting their adsorption properties, we embed/encode the cavities of a set of 74 porous organic cage molecules into a low-dimensional, latent "cage space".</div><div><br></div><div>We first scan the cavity of each cage to generate a 3D image of its porosity. Leveraging the singular value decomposition, in an unsupervised manner, we then learn across all cages an approximate, lower-dimensional subspace in which the 3D cage cavity images lay. The "eigencages" are the set of characteristic 3D cage cavity images that span this lower-dimensional subspace. A latent representation/encoding of each cage then follows from expressing it as a combination of the eigencages.</div><div><br></div><div>We show that the learned encoding captures salient features of the cavities of porous cages and is predictive of properties of the cages that arise from cavity shape.</div>

scholarly journals Eigencages: Learning a Latent Space of Porous Cage Molecules

2018 ◽  
Author(s):  
Arni Sturluson ◽  
Melanie T. Huynh ◽  
Arthur H. P. York ◽  
Cory Simon
Keyword(s):  
Dimensional Subspace ◽  
Singular Value ◽  
Adsorption Properties ◽  
3D Image ◽  
Latent Space ◽  
Cage Molecules ◽  
Gas Molecules ◽  
Low Dimensional ◽  
Value Decomposition ◽  
Lower Dimensional

<div>Porous organic cage molecules harbor nano-sized cavities that can selectively adsorb gas molecules, lending them applications in separations and sensing. The geometry of the cavity strongly influences adsorptive selectivity.</div><div><br></div><div>For comparing cages and predicting their adsorption properties, we embed/encode the cavities of a set of 74 porous organic cage molecules into a low-dimensional, latent "cage space".</div><div><br></div><div>We first scan the cavity of each cage to generate a 3D image of its porosity. Leveraging the singular value decomposition, in an unsupervised manner, we then learn across all cages an approximate, lower-dimensional subspace in which the 3D cage cavity images lay. The "eigencages" are the set of characteristic 3D cage cavity images that span this lower-dimensional subspace. A latent representation/encoding of each cage then follows from expressing it as a combination of the eigencages.</div><div><br></div><div>We show that the learned encoding captures salient features of the cavities of porous cages and is predictive of properties of the cages that arise from cavity shape.</div>

scholarly journals Explaining inference queries with bayesian optimization

2021 ◽  
Vol 14 (11) ◽  
pp. 2576-2585
Author(s):  
Brandon Lockhart ◽  
Jinglin Peng ◽  
Weiyuan Wu ◽  
Jiannan Wang ◽  
Eugene Wu
Keyword(s):  
Global Optimum ◽  
Black Box ◽  
Training Data ◽  
Bayesian Optimization ◽  
Categorical Variables ◽  
Warm Start ◽  
Query Result ◽  
Real World Datasets ◽  
Data Inference ◽  
Sql Query

Obtaining an explanation for an SQL query result can enrich the analysis experience, reveal data errors, and provide deeper insight into the data. Inference query explanation seeks to explain unexpected aggregate query results on inference data; such queries are challenging to explain because an explanation may need to be derived from the source, training, or inference data in an ML pipeline. In this paper, we model an objective function as a black-box function and propose BOExplain, a novel framework for explaining inference queries using Bayesian optimization (BO). An explanation is a predicate defining the input tuples that should be removed so that the query result of interest is significantly affected. BO --- a technique for finding the global optimum of a black-box function --- is used to find the best predicate. We develop two new techniques (individual contribution encoding and warm start) to handle categorical variables. We perform experiments showing that the predicates found by BOExplain have a higher degree of explanation compared to those found by the state-of-the-art query explanation engines. We also show that BOExplain is effective at deriving explanations for inference queries from source and training data on a variety of real-world datasets. BOExplain is open-sourced as a Python package at https://github.com/sfu-db/BOExplain.

scholarly journals Self-Guided Evolution Strategies with Historical Estimated Gradients

2020 ◽  
Author(s):  
Fei-Yu Liu ◽  
Zi-Niu Li ◽  
Chao Qian
Keyword(s):  
State Of The Art ◽  
Dimensional Subspace ◽  
Black Box ◽  
Evolution Strategies ◽  
Superior Performance ◽  
High Dimensional ◽  
Parameter Perturbations ◽  
Low Dimensional ◽  
Dimensional Optimization ◽  
Search Directions

Evolution Strategies (ES) are a class of black-box optimization algorithms and have been widely applied to solve problems, e.g., in reinforcement learning (RL), where the true gradient is unavailable. ES estimate the gradient of an objective function with respect to the parameters by randomly sampling search directions and evaluating parameter perturbations in these directions. However, the gradient estimator of ES tends to have a high variance for high-dimensional optimization, thus requiring a large number of samples and making ES inefficient. In this paper, we propose a new ES algorithm SGES, which utilizes historical estimated gradients to construct a low-dimensional subspace for sampling search directions, and adjusts the importance of this subspace adaptively. We prove that the variance of the gradient estimator of SGES can be much smaller than that of Vanilla ES; meanwhile, its bias can be well bounded. Empirical results on benchmark black-box functions and a set of popular RL tasks exhibit the superior performance of SGES over state-of-the-art ES algorithms.

scholarly journals Zero Shot Learning via Low-rank Embedded Semantic AutoEncoder

2018 ◽  
Author(s):  
Yang Liu ◽  
Quanxue Gao ◽  
Jin Li ◽  
Jungong Han ◽  
Ling Shao
Keyword(s):  
State Of The Art ◽  
Feature Space ◽  
Original Data ◽  
Semantic Space ◽  
Dimensional Subspace ◽  
Low Rank ◽  
Visual Features ◽  
Visual Feature ◽  
Benchmark Datasets ◽  
Low Dimensional

Zero-shot learning (ZSL) has been widely researched and get successful in machine learning. Most existing ZSL methods aim to accurately recognize objects of unseen classes by learning a shared mapping from the feature space to a semantic space. However, such methods did not investigate in-depth whether the mapping can precisely reconstruct the original visual feature. Motivated by the fact that the data have low intrinsic dimensionality e.g. low-dimensional subspace. In this paper, we formulate a novel framework named Low-rank Embedded Semantic AutoEncoder (LESAE) to jointly seek a low-rank mapping to link visual features with their semantic representations. Taking the encoder-decoder paradigm, the encoder part aims to learn a low-rank mapping from the visual feature to the semantic space, while decoder part manages to reconstruct the original data with the learned mapping. In addition, a non-greedy iterative algorithm is adopted to solve our model. Extensive experiments on six benchmark datasets demonstrate its superiority over several state-of-the-art algorithms.

Sign in / Sign up

Export Citation Format

Share Document