A Review of Well-Known Robust Line Search and Trust Region Numerical Optimization Algorithms for Solving Nonlinear Least-Squares Problems

The conditional, unconditional, or the exact maximum likelihood estimation and the least-squares estimation involve minimizing either the conditional or the unconditional residual sum of squares. The maximum likelihood estimation (MLE) approach and the nonlinear least squares (NLS) procedure involve an iterative search technique for obtaining global rather than local optimal estimates. Several authors have presented brief overviews of algorithms for solving NLS problems. Snezana S. Djordjevic (2019) presented a review of some unconstrained optimization methods based on the line search techniques. Mahaboob et al. (2017) proposed a different approach to estimate nonlinear regression models using numerical methods also based on the line search techniques. Mohammad, Waziri, and Santos (2019) have briefly reviewed methods for solving NLS problems, paying special attention to the structured quasi-Newton methods which are the family of the search line techniques. Ya-Xiang Yuan (2011) reviewed some recent results on numerical methods for nonlinear equations and NLS problems based on online searches and trust regions techniques, particularly on Levenberg-Marquardt type methods, quasi-Newton type methods, and trust regions algorithms. The purpose of this paper is to review some online searches and trust region's more well-known robust numerical optimization algorithms and the most used in practice for the estimation of time series models and other nonlinear regression models. The line searches algorithms considered are: Gradient algorithm, Steepest Descent (SD) algorithm, Newton-Raphson (NR) algorithm, Murray’s algorithm, Quasi-Newton (QN) algorithm, Gauss-Newton (GN) algorithm, Fletcher and Powell algorithm (FP), Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. While the only trust-region algorithm considered is the Levenberg-Marquardt (LM) algorithm. We also give some main advantages and disadvantages of these different algorithms.

A New Hybrid CQ Algorithm for the Split Feasibility Problem in Hilbert Spaces and Its Applications to Compressed Sensing

In this paper, we focus on studying the split feasibility problem (SFP), which has many applications in signal processing and image reconstruction. A popular technique is to employ the iterative method which is so called the relaxed CQ algorithm. However, the speed of convergence usually depends on the way of selecting the step size of such algorithms. We aim to suggest a new hybrid CQ algorithm for the SFP by using the self adaptive and the line-search techniques. There is no computation on the inverse and the spectral radius of a matrix. We then prove the weak convergence theorem under mild conditions. Numerical experiments are included to illustrate its performance in compressed sensing. Some comparisons are also given to show the efficiency with other CQ methods in the literature.

An amplitude-preserving deblending approach for simultaneous sources

A goal of simultaneous shooting is to acquire high-quality seismic data more efficiently, while reducing operational costs and improving acquisition efficiency. Effective sampling and deblending techniques are essential to achieve this goal. Inspired by compressive sensing (CS), we have formulated deblending as an analysis-based sparse inversion problem. We solve the inversion problem with an algorithm derived from the classic alternating direction method (ADM), associated with variable splitting and nonmonotone line-search techniques. In our testing, the analysis-based formulation together with nonmonotone ADM algorithm provides improved performance compared with synthesis-based approaches. A major issue for all deblending approaches is how to deal with real-world variations in seismic data caused by static shifts and amplitude imbalances. We evaluate the concept of including static and amplitude corrections obtained from surface-consistent solutions into the deblending formulation. We implement solutions that use a multistage inversion scheme to overcome the practical issues embedded in the field-blended data, such as strong coherent noise, statics, and shot-amplitude variations. The combination of these techniques gives high-fidelity deblending results for marine and land data. We use two field-data examples acquired with simultaneous sources to demonstrate the effectiveness of the proposed approach. Imaging and amplitude variation with offset quantitative analysis are carried out to indicate the amplitude-preserving character of deblended data with this methodology.

Parameter Estimation of Spring-Damping System using Unconstrained Optimization by the Quasi-Newton Methods using Line Search Techniques

Optimization is the basic tools to study the behaviour of many complicated mechanical systems by having the knowledge of differential equations which determine the system. The basis of this paper was to present a method to estimate the parameters such as spring constant and damping coefficient of the spring damped system by unconstrained optimization using derivative methods Such as quasi-newton method by Broyden-Fletcher-Goldfarb-Shanno and davidon-Fletcher-Powell hessian updating method by using backtracking line search methods along with Armijo’s condition.it uses the output error approximation procedure. It shows the convergence of different methods which are used to estimate the parameters and how accurately they are measured.