scholarly journals The Linear and Asymptotically Superlinear Convergence Rates of the Augmented Lagrangian Method with a Practical Relative Error Criterion

2020 ◽  
Vol 37 (04) ◽  
pp. 2040001
Author(s):  
Xin-Yuan Zhao ◽  
Liang Chen
Keyword(s):  
Convergence Rate ◽  
Rate Of Convergence ◽  
Relative Error ◽  
Superlinear Convergence ◽  
Augmented Lagrangian ◽  
Augmented Lagrangian Method ◽  
Penalty Parameter ◽  
Error Criterion ◽  
Linear Rate ◽  
Relative Error Criterion

In this paper, we conduct a convergence rate analysis of the augmented Lagrangian method with a practical relative error criterion designed in Eckstein and Silva [Mathematical Programming, 141, 319–348 (2013)] for convex nonlinear programming problems. We show that under a mild local error bound condition, this method admits locally a Q-linear rate of convergence. More importantly, we show that the modulus of the convergence rate is inversely proportional to the penalty parameter. That is, an asymptotically superlinear convergence is obtained if the penalty parameter used in the algorithm is increasing to infinity, or an arbitrarily Q-linear rate of convergence can be guaranteed if the penalty parameter is fixed but it is sufficiently large. Besides, as a byproduct, the convergence, as well as the convergence rate, of the distance from the primal sequence to the solution set of the problem is obtained.

A practical relative error criterion for augmented Lagrangians

2012 ◽  
Vol 141 (1-2) ◽  
pp. 319-348 ◽  
Author(s):  
Jonathan Eckstein ◽  
Paulo J. S. Silva
Keyword(s):  
Relative Error ◽  
Augmented Lagrangians ◽  
Error Criterion ◽  
Relative Error Criterion

scholarly journals Extended-space full waveform inversion in the time domain with the augmented Lagrangian method

Geophysics ◽  
2021 ◽  
pp. 1-57
Author(s):  
Ali Gholami ◽  
Hossein S. Aghamiry ◽  
Stéphane Operto
Keyword(s):  
Wave Equation ◽  
Augmented Lagrangian ◽  
Augmented Lagrangian Method ◽  
Waveform Inversion ◽  
Normal Equation ◽  
Full Waveform Inversion ◽  
Penalty Parameter ◽  
Time Stepping ◽  
Full Waveform ◽  
The Time Domain

The search space of Full Waveform Inversion (FWI) can be extended via a relaxation of the wave equation to increase the linear regime of the inversion. This wave equation relaxation is implemented by solving jointly (in a least-squares sense) the wave equation weighted by a penalty parameter and the observation equation such that the reconstructed wavefields closely match the data, hence preventing cycle skipping at receivers. Then, the subsurface parameters are updated by minimizing the temporal and spatial source extension generated by the wave-equation relaxation to push back the data-assimilated wavefields toward the physics.This extended formulation of FWI has been efficiently implemented in the frequency domain with the augmented Lagrangian method where the overdetermined systems of the data-assimilated wavefields can be solved separately for each frequency with linear algebra methods and the sensitivity of the optimization to the penalty parameter is mitigated through the action of the Lagrange multipliers.Applying this method in the time domain is however hampered by two main issues: the computation of data-assimilated wavefields with explicit time-stepping schemes and the storage of the Lagrange multipliers capturing the history of the source residuals in the state space.These two issues are solved by recognizing that the source residuals on the right-hand side of the extended wave equation, when formulated in a form suitable for explicit time stepping, are related to the extended data residuals through an adjoint equation.This relationship first allows us to relate the extended data residuals to the reduced data residuals through a normal equation in the data space. Once the extended data residuals have been estimated by solving (exactly or approximately) this normal equation, the data-assimilated wavefields are computed with explicit time stepping schemes by cascading an adjoint and a forward simulation.

The Rate of Convergence of a NLM Based on F–B NCP for Constrained Optimization Problems Without Strict Complementarity

2015 ◽  
Vol 32 (03) ◽  
pp. 1550012 ◽  
Author(s):  
Suxiang He ◽  
Liwei Zhang ◽  
Jie Zhang
Keyword(s):  
Constrained Optimization ◽  
Rate Of Convergence ◽  
Constraint Qualification ◽  
Optimization Problems ◽  
Augmented Lagrangian Method ◽  
Inequality Constraints ◽  
Constrained Optimization Problems ◽  
Strict Complementarity ◽  
Linear Rate ◽  
Nonlinear Lagrangian

It is well-known that the linear rate of convergence can be established for the classical augmented Lagrangian method for constrained optimization problems without strict complementarity. Whether this result is still valid for other nonlinear Lagrangian methods (NLM) is an interesting problem. This paper proposes a nonlinear Lagrangian function based on Fischer–Burmeister (F–B) nonlinear complimentarity problem (NCP) function for constrained optimization problems. The rate of convergence of this NLM is analyzed under the linear independent constraint qualification and the strong second-order sufficient condition without strict complementarity when subproblems are assumed to be solved exactly and inexactly, respectively. Interestingly, it is demonstrated that the Lagrange multipliers associating with inactive inequality constraints at the local minimum point converge to zeros superlinearly. Several illustrative examples are reported to show the behavior of the NLM.

Sign in / Sign up

Export Citation Format

Share Document