scholarly journals A Flow Perspective on Nonlinear Least-Squares Problems

2020 ◽  
Vol 48 (4) ◽  
pp. 987-1003
Author(s):  
Hans Georg Bock ◽  
Jürgen Gutekunst ◽  
Andreas Potschka ◽  
María Elena Suaréz Garcés
Keyword(s):  
Least Squares ◽  
Newton Method ◽  
Large Scale ◽  
Gradient Flow ◽  
Nonlinear Least Squares ◽  
Bundle Adjustment ◽  
Least Squares Problems ◽  
Flow Equations ◽  
Backward Euler ◽  
Forward Euler

AbstractJust as the damped Newton method for the numerical solution of nonlinear algebraic problems can be interpreted as a forward Euler timestepping on the Newton flow equations, the damped Gauß–Newton method for nonlinear least squares problems is equivalent to forward Euler timestepping on the corresponding Gauß–Newton flow equations. We highlight the advantages of the Gauß–Newton flow and the Gauß–Newton method from a statistical and a numerical perspective in comparison with the Newton method, steepest descent, and the Levenberg–Marquardt method, which are respectively equivalent to Newton flow forward Euler, gradient flow forward Euler, and gradient flow backward Euler. We finally show an unconditional descent property for a generalized Gauß–Newton flow, which is linked to Krylov–Gauß–Newton methods for large-scale nonlinear least squares problems. We provide numerical results for large-scale problems: An academic generalized Rosenbrock function and a real-world bundle adjustment problem from 3D reconstruction based on 2D images.

scholarly journals Cluster Gauss–Newton method

2020 ◽  
Author(s):  
Yasunori Aoki ◽  
Ken Hayami ◽  
Kota Toshimoto ◽  
Yuichi Sugiyama
Keyword(s):  
Least Squares ◽  
Iterative Methods ◽  
Newton Method ◽  
Computational Cost ◽  
Nonlinear Least Squares ◽  
Least Squares Problems ◽  
Pbpk Models ◽  
Estimation Problems ◽  
Computationally Intensive ◽  
Initial Iterate

Abstract Parameter estimation problems of mathematical models can often be formulated as nonlinear least squares problems. Typically these problems are solved numerically using iterative methods. The local minimiser obtained using these iterative methods usually depends on the choice of the initial iterate. Thus, the estimated parameter and subsequent analyses using it depend on the choice of the initial iterate. One way to reduce the analysis bias due to the choice of the initial iterate is to repeat the algorithm from multiple initial iterates (i.e. use a multi-start method). However, the procedure can be computationally intensive and is not always used in practice. To overcome this problem, we propose the Cluster Gauss–Newton (CGN) method, an efficient algorithm for finding multiple approximate minimisers of nonlinear-least squares problems. CGN simultaneously solves the nonlinear least squares problem from multiple initial iterates. Then, CGN iteratively improves the approximations from these initial iterates similarly to the Gauss–Newton method. However, it uses a global linear approximation instead of the Jacobian. The global linear approximations are computed collectively among all the iterates to minimise the computational cost associated with the evaluation of the mathematical model. We use physiologically based pharmacokinetic (PBPK) models used in pharmaceutical drug development to demonstrate its use and show that CGN is computationally more efficient and more robust against local minima compared to the standard Levenberg–Marquardt method, as well as state-of-the art multi-start and derivative-free methods.

Sign in / Sign up

Export Citation Format

Share Document