Adaptive operator extrapolation method

This paper is devoted to the study of nоvel algorithm with Bregman projection for solving variational inequalities in Hilbert space. Proposed algorithm is an adaptive version of the operator extrapolation method, where the used rule for updating the step size does not require knowledge of Lipschitz constants and the calculation of operator values at additional points. An attractive feature of the algorithm is only one computation at the iterative step of the Bregman projection onto the feasible set.

CONVERGENCE OF THE EXTRAPOLATION METHOD FROM THE PAST AND THE OPERATOR EXTRAPOLATION METHOD

One of the popular areas of modern applied nonlinear analysis is the study of variational inequalities. Many important problems of operations research and mathematical physics can be written in the form of variational inequalities. With the advent of generating adversarial neural networks, interest in algorithms for solving variational inequalities arose in the ML-community. This paper is devoted to the study of three new algorithms with Bregman projection for solving variational inequalities in Hilbert space. The first algorithm is the result of a modification of the two-stage Bregman method by low-cost adjusting the step size that without the prior knowledge of the Lipschitz constant of operator. The second algorithm, which we call the operator extrapolation algorithm, is obtained by replacing the Euclidean metric in the Malitsky–Tam method with the Bregman divergence. An attractive feature of the algorithm is only one computation at the iterative step of the Bregman projection onto the feasible set. The third algorithm is an adaptive version of the second, where the used rule for updating the step size does not require knowledge of Lipschitz constants and the calculation of operator values at additional points. For variational inequalities with pseudo-monotone, Lipschitz-continuous, and sequentially weakly continuous operators acting in a Hilbert space, convergence theorems are proved.

Overrelaxed Sinkhorn–Knopp Algorithm for Regularized Optimal Transport

This article describes a set of methods for quickly computing the solution to the regularized optimal transport problem. It generalizes and improves upon the widely used iterative Bregman projections algorithm (or Sinkhorn–Knopp algorithm). We first proposed to rely on regularized nonlinear acceleration schemes. In practice, such approaches lead to fast algorithms, but their global convergence is not ensured. Hence, we next proposed a new algorithm with convergence guarantees. The idea is to overrelax the Bregman projection operators, allowing for faster convergence. We proposed a simple method for establishing global convergence by ensuring the decrease of a Lyapunov function at each step. An adaptive choice of the overrelaxation parameter based on the Lyapunov function was constructed. We also suggested a heuristic to choose a suitable asymptotic overrelaxation parameter, based on a local convergence analysis. Our numerical experiments showed a gain in convergence speed by an order of magnitude in certain regimes.

Two Bregman Projection Methods for Solving Variational Inequality Problems in Hilbert Spaces with Applications to Signal Processing

Studying Bregman distance iterative methods for solving optimization problems has become an important and very interesting topic because of the numerous applications of the Bregman distance techniques. These applications are based on the type of convex functions associated with the Bregman distance. In this paper, two different extragraident methods were proposed for studying pseudomonotone variational inequality problems using Bregman distance in real Hilbert spaces. The first algorithm uses a fixed stepsize which depends on a prior estimate of the Lipschitz constant of the cost operator. The second algorithm uses a self-adaptive stepsize which does not require prior estimate of the Lipschitz constant of the cost operator. Some convergence results were proved for approximating the solutions of pseudomonotone variational inequality problem under standard assumptions. Moreso, some numerical experiments were also given to illustrate the performance of the proposed algorithms using different convex functions such as the Shannon entropy and the Burg entropy. In addition, an application of the result to a signal processing problem is also presented.