scholarly journals A Decentralized Second-Order Method with Exact Linear Convergence Rate for Consensus Optimization

2016 ◽  
Vol 2 (4) ◽  
pp. 507-522 ◽  
Author(s):  
Aryan Mokhtari ◽  
Wei Shi ◽  
Qing Ling ◽  
Alejandro Ribeiro
Keyword(s):  
Convergence Rate ◽  
Linear Convergence ◽  
Second Order ◽  
Order Method ◽  
Linear Convergence Rate ◽  
Consensus Optimization

scholarly journals Convergence rate analysis of an iterative algorithm for solving the multiple-sets split equality problem

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Shijie Sun ◽  
Meiling Feng ◽  
Luoyi Shi
Keyword(s):  
Convergence Rate ◽  
Iterative Algorithm ◽  
Sufficient Conditions ◽  
Linear Convergence ◽  
Regularity Property ◽  
Step Size ◽  
Bounded Linear ◽  
Split Equality Problem ◽  
Linear Convergence Rate ◽  
Equality Problem

Abstract This paper considers an iterative algorithm of solving the multiple-sets split equality problem (MSSEP) whose step size is independent of the norm of the related operators, and investigates its sublinear and linear convergence rate. In particular, we present a notion of bounded Hölder regularity property for the MSSEP, which is a generalization of the well-known concept of bounded linear regularity property, and give several sufficient conditions to ensure it. Then we use this property to conclude the sublinear and linear convergence rate of the algorithm. In the end, some numerical experiments are provided to verify the validity of our consequences.

ESOM: Exact second-order method for consensus optimization

2016 ◽  
Author(s):  
Aryan Mokhtari ◽  
Wei Shi ◽  
Qing Ling
Keyword(s):  
Second Order ◽  
Order Method ◽  
Consensus Optimization

A modified limited memory steepest descent method motivated by an inexact super-linear convergence rate analysis

2020 ◽  
Author(s):  
Ran Gu ◽  
Qiang Du
Keyword(s):  
Convergence Rate ◽  
Linear Convergence ◽  
Descent Method ◽  
Steepest Descent ◽  
Steepest Descent Method ◽  
Step Size ◽  
Linear Convergence Rate ◽  
Limited Memory ◽  
Rate Analysis ◽  
Convergence Rate Analysis

Abstract How to choose the step size of gradient descent method has been a popular subject of research. In this paper we propose a modified limited memory steepest descent method (MLMSD). In each iteration we propose a selection rule to pick a unique step size from a candidate set, which is calculated by Fletcher’s limited memory steepest descent method (LMSD), instead of going through all the step sizes in a sweep, as in Fletcher’s original LMSD algorithm. MLMSD is motivated by an inexact super-linear convergence rate analysis. The R-linear convergence of MLMSD is proved for a strictly convex quadratic minimization problem. Numerical tests are presented to show that our algorithm is efficient and robust.

Linear convergence rate for the MDM algorithm for the Nearest Point Problem

2015 ◽  
Vol 48 (4) ◽  
pp. 1510-1522 ◽  
Author(s):  
Jorge López ◽  
José R. Dorronsoro
Keyword(s):  
Convergence Rate ◽  
Linear Convergence ◽  
Point Problem ◽  
Linear Convergence Rate ◽  
Nearest Point

scholarly journals Stochastic Second-Order Method for Large-Scale Nonconvex Sparse Learning Models

2018 ◽  
Author(s):  
Hongchang Gao ◽  
Heng Huang
Keyword(s):  
Large Scale ◽  
Linear Convergence ◽  
Second Order ◽  
Sparse Learning ◽  
Order Method ◽  
Learning Models ◽  
Main Challenge ◽  
Convex Problem ◽  
Machine Learning Applications ◽  
Nonconvex Constraint

Sparse learning models have shown promising performance in the high dimensional machine learning applications. The main challenge of sparse learning models is how to optimize it efficiently. Most existing methods solve this problem by relaxing it as a convex problem, incurring large estimation bias. Thus, the sparse learning model with nonconvex constraint has attracted much attention due to its better performance. But it is difficult to optimize due to the non-convexity. In this paper, we propose a linearly convergent stochastic second-order method to optimize this nonconvex problem for large-scale datasets. The proposed method incorporates second-order information to improve the convergence speed. Theoretical analysis shows that our proposed method enjoys linear convergence rate and guarantees to converge to the underlying true model parameter. Experimental results have verified the efficiency and correctness of our proposed method.

Sign in / Sign up

Export Citation Format

Share Document