scholarly journals Randomized Simplicial Hessian Update

Mathematics ◽  
2021 ◽  
Vol 9 (15) ◽  
pp. 1775
Author(s):  
Árpád Bűrmen ◽  
Tadej Tuma ◽  
Jernej Olenšek
Keyword(s):  
Lower Bound ◽  
Dimensional Subspace ◽  
Expected Improvement ◽  
Closed Form Expression ◽  
Frobenius Norm ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Derivative Information ◽  
Approximate Hessian ◽  
Special Case

Recently, a derivative-free optimization algorithm was proposed that utilizes a minimum Frobenius norm (MFN) Hessian update for estimating the second derivative information, which in turn is used for accelerating the search. The proposed update formula relies only on computed function values and is a closed-form expression for a special case of a more general approach first published by Powell. This paper analyzes the convergence of the proposed update formula under the assumption that the points from Rn where the function value is known are random. The analysis assumes that the N+2 points used by the update formula are obtained by adding N+1 vectors to a central point. The vectors are obtained by transforming a prototype set of N+1 vectors with a random orthogonal matrix from the Haar measure. The prototype set must positively span a N≤n dimensional subspace. Because the update is random by nature we can estimate a lower bound on the expected improvement of the approximate Hessian. This lower bound was derived for a special case of the proposed update by Leventhal and Lewis. We generalize their result and show that the amount of improvement greatly depends on N as well as the choice of the vectors in the prototype set. The obtained result is then used for analyzing the performance of the update based on various commonly used prototype sets. One of the results obtained by this analysis states that a regular n-simplex is a bad choice for a prototype set because it does not guarantee any improvement of the approximate Hessian.

scholarly journals Derivative-free optimization methods

Acta Numerica ◽  
2019 ◽  
Vol 28 ◽  
pp. 287-404 ◽  
Author(s):  
Jeffrey Larson ◽  
Matt Menickelly ◽  
Stefan M. Wild
Keyword(s):  
Optimization Problems ◽  
Linear Optimization ◽  
Optimization Methods ◽  
Black Box ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Recent Developments ◽  
Randomized Methods ◽  
Derivative Information ◽  
Scientific Engineering

In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.

scholarly journals Optimal Wells Placement to Maximize the Field Coverage Using Derivative-Free Optimization

2020 ◽  
Vol 178 ◽  
pp. 65-74
Author(s):  
Ksenia Balabaeva ◽  
Liya Akmadieva ◽  
Sergey Kovalchuk
Keyword(s):  
Derivative Free Optimization ◽  
Derivative Free

A Note on a Lower Bound for the Multiplicative Odds Theorem of Optimal Stopping

2014 ◽  
Vol 51 (03) ◽  
pp. 885-889 ◽  
Author(s):  
Tomomi Matsui ◽  
Katsunori Ano
Keyword(s):  
Lower Bound ◽  
Optimal Stopping ◽  
Secretary Problem ◽  
Bernoulli Trials ◽  
Optimal Stopping Problem ◽  
Maximum Probability ◽  
Optimal Stopping Rule ◽  
Optimal Probability ◽  
Optimal Stopping Theory ◽  
Special Case

In this note we present a bound of the optimal maximum probability for the multiplicative odds theorem of optimal stopping theory. We deal with an optimal stopping problem that maximizes the probability of stopping on any of the last m successes of a sequence of independent Bernoulli trials of length N, where m and N are predetermined integers satisfying 1 ≤ m < N. This problem is an extension of Bruss' (2000) odds problem. In a previous work, Tamaki (2010) derived an optimal stopping rule. We present a lower bound of the optimal probability. Interestingly, our lower bound is attained using a variation of the well-known secretary problem, which is a special case of the odds problem.

Benchmarking and Field-Testing of the Distributed Quasi-Newton Derivative-Free Optimization Method for Field Development Optimization

2021 ◽  
Author(s):  
Faruk Alpak ◽  
Yixuan Wang ◽  
Guohua Gao ◽  
Vivek Jain
Keyword(s):  
Optimization Problems ◽  
Field Testing ◽  
Optimization Method ◽  
Training Data ◽  
Local Optima ◽  
Field Development ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Data Points ◽  
Quasi Newton

Abstract Recently, a novel distributed quasi-Newton (DQN) derivative-free optimization (DFO) method was developed for generic reservoir performance optimization problems including well-location optimization (WLO) and well-control optimization (WCO). DQN is designed to effectively locate multiple local optima of highly nonlinear optimization problems. However, its performance has neither been validated by realistic applications nor compared to other DFO methods. We have integrated DQN into a versatile field-development optimization platform designed specifically for iterative workflows enabled through distributed-parallel flow simulations. DQN is benchmarked against alternative DFO techniques, namely, the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method hybridized with Direct Pattern Search (BFGS-DPS), Mesh Adaptive Direct Search (MADS), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA). DQN is a multi-thread optimization method that distributes an ensemble of optimization tasks among multiple high-performance-computing nodes. Thus, it can locate multiple optima of the objective function in parallel within a single run. Simulation results computed from one DQN optimization thread are shared with others by updating a unified set of training data points composed of responses (implicit variables) of all successful simulation jobs. The sensitivity matrix at the current best solution of each optimization thread is approximated by a linear-interpolation technique using all or a subset of training-data points. The gradient of the objective function is analytically computed using the estimated sensitivities of implicit variables with respect to explicit variables. The Hessian matrix is then updated using the quasi-Newton method. A new search point for each thread is solved from a trust-region subproblem for the next iteration. In contrast, other DFO methods rely on a single-thread optimization paradigm that can only locate a single optimum. To locate multiple optima, one must repeat the same optimization process multiple times starting from different initial guesses for such methods. Moreover, simulation results generated from a single-thread optimization task cannot be shared with other tasks. Benchmarking results are presented for synthetic yet challenging WLO and WCO problems. Finally, DQN method is field-tested on two realistic applications. DQN identifies the global optimum with the least number of simulations and the shortest run time on a synthetic problem with known solution. On other benchmarking problems without a known solution, DQN identified compatible local optima with reasonably smaller numbers of simulations compared to alternative techniques. Field-testing results reinforce the auspicious computational attributes of DQN. Overall, the results indicate that DQN is a novel and effective parallel algorithm for field-scale development optimization problems.

scholarly journals Improved bounds for the availability and unavailability in a fixed time interval for systems of maintained, interdependent components

1980 ◽  
Vol 12 (01) ◽  
pp. 200-221 ◽  
Author(s):  
B. Natvig
Keyword(s):  
Lower Bound ◽  
System Reliability ◽  
Nuclear Reactors ◽  
Random Variables ◽  
Fixed Time ◽  
Time Interval ◽  
Coherent System ◽  
Exact Results ◽  
Fixed Time Interval ◽  
Special Case

In this paper we arrive at a series of bounds for the availability and unavailability in the time interval I = [t A , t B ] ⊂ [0, ∞), for a coherent system of maintained, interdependent components. These generalize the minimal cut lower bound for the availability in [0, t] given in Esary and Proschan (1970) and also most bounds for the reliability at time t given in Bodin (1970) and Barlow and Proschan (1975). In the latter special case also some new improved bounds are given. The bounds arrived at are of great interest when trying to predict the performance process of the system. In particular, Lewis et al. (1978) have revealed the great need for adequate tools to treat the dependence between the random variables of interest when considering the safety of nuclear reactors. Satyanarayana and Prabhakar (1978) give a rapid algorithm for computing exact system reliability at time t. This can also be used in cases where some simpler assumptions on the dependence between the components are made. It seems, however, impossible to extend their approach to obtain exact results for the cases treated in the present paper.

Sign in / Sign up

Export Citation Format

Share Document