A Comprehensive Investigation of Ensembles of Gaussian-Based and Gradient-Based Transformed Reliability Analyses: When and How to Use Them

2016 ◽  
Author(s):  
Po Ting Lin ◽  
Wei-Hao Lu ◽  
Shu-Ping Lin
Keyword(s):  
Design Optimization ◽  
Design Space ◽  
Optimization Problems ◽  
Nonlinear Problems ◽  
Kernel Functions ◽  
Sensitivity Analyses ◽  
Normal Space ◽  
Highly Nonlinear ◽  
Standard Normal ◽  
Gradient Based

In the past few years, researchers have begun to investigate the existence of arbitrary uncertainties in the design optimization problems. Most traditional reliability-based design optimization (RBDO) methods transform the design space to the standard normal space for reliability analysis but may not work well when the random variables are arbitrarily distributed. It is because that the transformation to the standard normal space cannot be determined or the distribution type is unknown. The methods of Ensemble of Gaussian-based Reliability Analyses (EoGRA) and Ensemble of Gradient-based Transformed Reliability Analyses (EGTRA) have been developed to estimate the joint probability density function using the ensemble of kernel functions. EoGRA performs a series of Gaussian-based kernel reliability analyses and merged them together to compute the reliability of the design point. EGTRA transforms the design space to the single-variate design space toward the constraint gradient, where the kernel reliability analyses become much less costly. In this paper, a series of comprehensive investigations were performed to study the similarities and differences between EoGRA and EGTRA. The results showed that EGTRA performs accurate and effective reliability analyses for both linear and nonlinear problems. When the constraints are highly nonlinear, EGTRA may have little problem but still can be effective in terms of starting from deterministic optimal points. On the other hands, the sensitivity analyses of EoGRA may be ineffective when the random distribution is completely inside the feasible space or infeasible space. However, EoGRA can find acceptable design points when starting from deterministic optimal points. Moreover, EoGRA is capable of delivering estimated failure probability of each constraint during the optimization processes, which may be convenient for some applications.

An Effective Approach to Solve Design Optimization Problems With Arbitrarily Distributed Uncertainties in the Original Design Space Using Ensemble of Gaussian Reliability Analyses

2016 ◽  
Vol 138 (7) ◽  
Author(s):  
Po Ting Lin ◽  
Shu-Ping Lin
Keyword(s):  
Design Optimization ◽  
Reliability Analysis ◽  
Reliability Index ◽  
Design Space ◽  
Optimization Problems ◽  
Performance Measure ◽  
Probabilistic Constraints ◽  
Original Design ◽  
Design Point ◽  
Standard Normal

Reliability-based design optimization (RBDO) algorithms have been developed to solve design optimization problems with existence of uncertainties. Traditionally, the original random design space is transformed to the standard normal design space, where the reliability index can be measured in a standardized unit. In the standard normal design space, the modified reliability index approach (MRIA) measured the minimum distance from the design point to the failure region to represent the reliability index; on the other hand, the performance measure approach (PMA) performed inverse reliability analysis to evaluate the target function performance in a distance of reliability index away from the design point. MRIA was able to provide stable and accurate reliability analysis while PMA showed greater efficiency and was widely used in various engineering applications. However, the existing methods cannot properly perform reliability analysis in the standard normal design space if the transformation to the standard normal space does not exist or is difficult to determine. To this end, a new algorithm, ensemble of Gaussian reliability analyses (EoGRA), was developed to estimate the failure probability using Gaussian-based kernel density estimation (KDE) in the original design space. The probabilistic constraints were formulated based on each kernel reliability analysis for the optimization processes. This paper proposed an efficient way to estimate the constraint gradient and linearly approximate the probabilistic constraints with fewer function evaluations (FEs). Some numerical examples with various random distributions are studied to investigate the numerical performances of the proposed method. The results showed that EoGRA is capable of finding correct solutions in some problems that cannot be solved by traditional methods. Furthermore, experiments of image processing with arbitrarily distributed photo pixels are performed. The lighting of image pixels is maximized subject to the acceptable limit. Our implementation showed that the accuracy of the estimation of normal distribution is poor while the proposed method is capable of finding the optimal solution with acceptable accuracy.

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

2021 ◽  
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

2021 ◽  
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

scholarly journals Reliability Analysis via an Optimal Covariance Matrix Adaptation Evolution Strategy: Emphasis on Applications in Civil Engineering

2020 ◽  
Author(s):  
Ali Kaveh ◽  
Seyed Mohammad Javadi ◽  
Roya Mahdipour Moghanni
Keyword(s):  
Reliability Index ◽  
Civil Engineering ◽  
Limit State ◽  
Normal Space ◽  
State Function ◽  
Optimization Approach ◽  
Covariance Matrix Adaptation ◽  
Highly Nonlinear ◽  
Standard Normal ◽  
Reliability Based Optimization

In this paper, a reliability-based optimization approach is applied using a recently proposed CMA-ES with optimal covariance update and storage complexity. Cholesky-CMA-ES gives a significant increase in optimization speed and reduces the runtime complexity of the standard CMA-ES. The reliability index is the shortest distance between the surface of Limit-State Function (LSF) and the origin of the standard normal space. Hence, finding the reliability index can be expressed as a constrained optimization problem. To verify the concept and test the feasibility of this algorithm, several numerical examples consisting of mathematical and highly nonlinear civil engineering problems are investigated. The reliability indexes obtained agree reasonably well with reported values from some existing approximation methods and Monte Carlo simulation.

Reliability-Based Multidisciplinary Design Optimization Using Probabilistic Gradient-Based Transformation Method

2012 ◽  
Vol 135 (2) ◽  
Author(s):  
Po Ting Lin ◽  
Hae Chang Gea
Keyword(s):  
Design Optimization ◽  
Multidisciplinary Design Optimization ◽  
Transformation Method ◽  
Performance Measure ◽  
Probabilistic Constraints ◽  
Multidisciplinary Design ◽  
Sensitivity Analyses ◽  
Chance Constrained Programming ◽  
Complex Design ◽  
Gradient Based

Recently, solving the complex design optimization problems with design uncertainties has become an important but very challenging task in the communities of reliability-based design optimization (RBDO) and multidisciplinary design optimization (MDO). The MDO algorithms decompose the complex design problem into the hierarchical or nonhierarchical optimization structure and distribute the workloads to each discipline (or subproblem) in the decomposed structure. The coordination of the local responses is crucial for the success of finding the optimal design point. The problem complexity increases dramatically when the existence of the design uncertainties is not negligible. The RBDO algorithms perform the reliability analyses to evaluate the probabilities that the random variables violate the constraints. However, the required reliability analyses build up the degree of complexity. In this paper, the gradient-based transformation method (GTM) is utilized to reduce the complexity of the MDO problems by transforming the design space to multiple single-variate monotonic coordinates along the directions of the constraint gradients. The subsystem responses are found using the monotonicity principles (MP) and then coordinated for the new design points based on two general principles. To consider the design uncertainties, the probabilistic gradient-based transformation method (PGTM) is proposed to adapt the first-order probabilistic constraints from three different RBDO algorithms, including the chance constrained programming (CCP), reliability index approach (RIA), and performance measure approach (PMA), to the framework of the GTM. PGTM is efficient because only the sensitivity analyses and the reliability analyses require function evaluations (FE). The optimization processes of monotonicity analyses and the coordination procedures are free of function evaluations. Several mathematical and engineering examples show the PGTM is capable of finding the optimal solutions with desirable reliability levels.

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

Algorithms ◽  
2021 ◽  
Vol 15 (1) ◽  
pp. 6
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient-based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have been shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method, though less commonly used in training neural networks, is known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks, and to briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

A Hybrid Loop Approach Using the Sufficient Descent Condition for Accurate, Robust, and Efficient Reliability-Based Design Optimization

2016 ◽  
Vol 138 (12) ◽  
Author(s):  
Behrooz Keshtegar ◽  
Peng Hao
Keyword(s):  
Design Optimization ◽  
Nonlinear Problems ◽  
Performance Measure ◽  
Reliability Based Design Optimization ◽  
Nonlinear Constraint ◽  
Sufficient Descent Condition ◽  
Reliability Based Design ◽  
Highly Nonlinear ◽  
Sufficient Descent ◽  
Descent Condition

For reliability-based design optimization (RBDO) problems, single loop approaches (SLA) are very efficient but prone to converge to inappropriate point for highly nonlinear constraint functions, and double loop approaches (DLA) are proven to be accurate but require more iterations to achieve stable results. In this paper, an adjusted advanced mean value (AAMV) method is firstly proposed to improve the robustness and efficiency of performance measure approach. The global convergence of the AAMV is guaranteed using sufficient descent condition for the reliability loop in RBDO. Then, a hybrid RBDO method is developed to improve the efficiency of DLA and accuracy of SLA, on the basis of sufficient descent condition and AAMV method, named as hybrid single and double loops (HSD) method. Three nonlinear concave and convex performance functions are used to illustrate the efficiency and robustness of the AAMV method; then the accuracy, robustness, and efficiency of the proposed HSD method are compared to current SLA and DLA through another three benchmark nonlinear RBDO examples. Results show that the AAMV is more robust and efficient than the existing reliability analysis methods. The HSD is more accurate than the SLA for highly nonlinear problems, and also exhibits a better performance than the DLA from the point of view of both robustness and efficiency.

An Investigation of Nonlinearity of Reliability-Based Design Optimization Approaches

2002 ◽  
Author(s):  
Kyung K. Choi ◽  
Byeng D. Youn
Keyword(s):  
Design Optimization ◽  
Reliability Analysis ◽  
Probability Distributions ◽  
Performance Measure ◽  
Probabilistic Constraints ◽  
Random Parameters ◽  
Reliability Based Design Optimization ◽  
Reliability Based Design ◽  
Highly Nonlinear ◽  
Standard Normal

Deterministic optimum designs that are obtained without consideration of uncertainty could lead to unreliable designs, which call for a reliability approach to design optimization, using a Reliability-Based Design Optimization (RBDO) method. A typical RBDO process iteratively carries out a design optimization in an original random space (X-space) and reliability analysis in an independent and standard normal random space (U-space). This process requires numerous nonlinear mapping between X- and U-spaces for a various probability distributions. Therefore, the nonlinearity of RBDO problem will depend on the type of distribution of random parameters, since a transformation between X- and U-spaces introduces additional nonlinearity to reliability-based performance measures evaluated during the RBDO process. Evaluation of probabilistic constraints in RBDO can be carried out in two different ways: the Reliability Index Approach (RIA) and the Performance Measure Approach (PMA). Different reliability analysis approaches employed in RIA and PMA result in different behaviors of nonlinearity of RIA and PMA in the RBDO process. In this paper, it is shown that RIA becomes much more difficult to solve for non-normally distributed random parameters because of highly nonlinear transformations involved. However, PMA is rather independent of probability distributions because of little involvement of the nonlinear transformation.

Sign in / Sign up

Export Citation Format

Share Document