Novel forward–backward algorithms for optimization and applications to compressive sensing and image inpainting

AbstractThe forward–backward algorithm is a splitting method for solving convex minimization problems of the sum of two objective functions. It has a great attention in optimization due to its broad application to many disciplines, such as image and signal processing, optimal control, regression, and classification problems. In this work, we aim to introduce new forward–backward algorithms for solving both unconstrained and constrained convex minimization problems by using linesearch technique. We discuss the convergence under mild conditions that do not depend on the Lipschitz continuity assumption of the gradient. Finally, we provide some applications to solving compressive sensing and image inpainting problems. Numerical results show that the proposed algorithm is more efficient than some algorithms in the literature. We also discuss the optimal choice of parameters in algorithms via numerical experiments.

Download Full-text

A Novel Forward-Backward Algorithm for Solving Convex Minimization Problem in Hilbert Spaces

Mathematics ◽

10.3390/math8010042 ◽

2020 ◽

Vol 8 (1) ◽

pp. 42

Author(s):

Suthep Suantai ◽

Kunrada Kankam ◽

Prasit Cholamjiak

Keyword(s):

Hilbert Spaces ◽

Minimization Problem ◽

Line Search ◽

Optimization Problem ◽

Splitting Method ◽

Optimal Choice ◽

Convex Minimization ◽

Signal Recovery ◽

Objective Functions ◽

Convex Minimization Problem

In this work, we aim to investigate the convex minimization problem of the sum of two objective functions. This optimization problem includes, in particular, image reconstruction and signal recovery. We then propose a new modified forward-backward splitting method without the assumption of the Lipschitz continuity of the gradient of functions by using the line search procedures. It is shown that the sequence generated by the proposed algorithm weakly converges to minimizers of the sum of two convex functions. We also provide some applications of the proposed method to compressed sensing in the frequency domain. The numerical reports show that our method has a better convergence behavior than other methods in terms of the number of iterations and CPU time. Moreover, the numerical results of the comparative analysis are also discussed to show the optimal choice of parameters in the line search.

Download Full-text

On convergence and complexity analysis of an accelerated forward–backward algorithm with linesearch technique for convex minimization problems and applications to data prediction and classification

Journal of Inequalities and Applications ◽

10.1186/s13660-021-02675-y ◽

2021 ◽

Vol 2021 (1) ◽

Author(s):

Panitarn Sarnmeta ◽

Warunun Inthakon ◽

Dawan Chumpungam ◽

Suthep Suantai

Keyword(s):

Numerical Experiments ◽

Lipschitz Continuity ◽

Complexity Analysis ◽

Lower Semicontinuous ◽

Convex Minimization ◽

Classification Problems ◽

Minimization Problems ◽

Data Prediction ◽

Convex Minimization Problems ◽

Better Than

AbstractIn this work, we introduce a new accelerated algorithm using a linesearch technique for solving convex minimization problems in the form of a summation of two lower semicontinuous convex functions. A weak convergence of the proposed algorithm is given without assuming the Lipschitz continuity on the gradient of the objective function. Moreover, the convexity of this algorithm is also analyzed. Some numerical experiments in machine learning are also discussed, namely regression and classification problems. Furthermore, in our experiments, we evaluate the convergent behavior of this new algorithm, then compare it with various algorithms mentioned in the literature. It is found that our algorithm performs better than the others.

Download Full-text

An accelerated viscosity forward-backward splitting algorithm with the linesearch process for convex minimization problems

Journal of Inequalities and Applications ◽

10.1186/s13660-021-02571-5 ◽

2021 ◽

Vol 2021 (1) ◽

Author(s):

Suthep Suantai ◽

Pachara Jailoka ◽

Adisak Hanjing

Keyword(s):

Lipschitz Constant ◽

Optimization Methods ◽

Convergence Result ◽

Convex Minimization ◽

Signal Recovery ◽

Splitting Algorithm ◽

Minimization Problems ◽

Convex Minimization Problems ◽

Continuity Assumption ◽

Inertial Technique

AbstractIn this paper, we consider and investigate a convex minimization problem of the sum of two convex functions in a Hilbert space. The forward-backward splitting algorithm is one of the popular optimization methods for approximating a minimizer of the function; however, the stepsize of this algorithm depends on the Lipschitz constant of the gradient of the function, which is not an easy work to find in general practice. By using a new modification of the linesearches of Cruz and Nghia [Optim. Methods Softw. 31:1209–1238, 2016] and Kankam et al. [Math. Methods Appl. Sci. 42:1352–1362, 2019] and an inertial technique, we introduce an accelerated viscosity-type algorithm without any Lipschitz continuity assumption on the gradient. A strong convergence result of the proposed algorithm is established under some control conditions. As applications, we apply our algorithm to solving image and signal recovery problems. Numerical experiments show that our method has a higher efficiency than the well-known methods in the literature.

Download Full-text