A modified limited memory steepest descent method motivated by an inexact super-linear convergence rate analysis

2020 ◽  
Author(s):  
Ran Gu ◽  
Qiang Du
Keyword(s):  
Convergence Rate ◽  
Linear Convergence ◽  
Descent Method ◽  
Steepest Descent ◽  
Steepest Descent Method ◽  
Step Size ◽  
Linear Convergence Rate ◽  
Limited Memory ◽  
Rate Analysis ◽  
Convergence Rate Analysis

Abstract How to choose the step size of gradient descent method has been a popular subject of research. In this paper we propose a modified limited memory steepest descent method (MLMSD). In each iteration we propose a selection rule to pick a unique step size from a candidate set, which is calculated by Fletcher’s limited memory steepest descent method (LMSD), instead of going through all the step sizes in a sweep, as in Fletcher’s original LMSD algorithm. MLMSD is motivated by an inexact super-linear convergence rate analysis. The R-linear convergence of MLMSD is proved for a strictly convex quadratic minimization problem. Numerical tests are presented to show that our algorithm is efficient and robust.

scholarly journals Convergence rate analysis of an iterative algorithm for solving the multiple-sets split equality problem

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Shijie Sun ◽  
Meiling Feng ◽  
Luoyi Shi
Keyword(s):  
Convergence Rate ◽  
Iterative Algorithm ◽  
Sufficient Conditions ◽  
Linear Convergence ◽  
Regularity Property ◽  
Step Size ◽  
Bounded Linear ◽  
Split Equality Problem ◽  
Linear Convergence Rate ◽  
Equality Problem

Abstract This paper considers an iterative algorithm of solving the multiple-sets split equality problem (MSSEP) whose step size is independent of the norm of the related operators, and investigates its sublinear and linear convergence rate. In particular, we present a notion of bounded Hölder regularity property for the MSSEP, which is a generalization of the well-known concept of bounded linear regularity property, and give several sufficient conditions to ensure it. Then we use this property to conclude the sublinear and linear convergence rate of the algorithm. In the end, some numerical experiments are provided to verify the validity of our consequences.

Self-Interference Cancellation in Full Duplex Communication using Steepest Descent Method

2021 ◽  
Vol 9 (VI) ◽  
pp. 1433-1442
Author(s):  
N. Alivelu Manga
Keyword(s):  
Interference Cancellation ◽  
Descent Method ◽  
Steepest Descent ◽  
Full Duplex ◽  
Steepest Descent Method ◽  
Interference Signal ◽  
Least Mean Square ◽  
Echo Canceller ◽  
Step Size ◽  
Signal Processing Technique

The present-day communication system uses Frequency Division Duplex (FDD) to emulate the benefits of Full Duplex Communication. But it requires more bandwidth as the cost of the spectrum is very high it becomes a major limitation. To overcome this problem implementation of Full Duplex Communication is the best solution. Implementation of full duplex communication is difficult because of a significant problem called self-interference. while transmitting and receiving signals on the same frequency band, receiving signal is interfered with the transmitted signal this phenomenon is called self-interference. The objective of this project is to minimize that self-interference signal from the received signal by using signal processing technique, LMS echo cancellation. Least Mean Square (LMS) echo canceller whose coefficients are updated iteratively is used to cancel the self-interference. An algorithm based on steepest descent method is used to obtain coefficients that change iteratively with varying step size to solve Weiner-Hopfs equation.

scholarly journals Adaptive hybrid steepest descent algorithms involving an inertial extrapolation term for split monotone variational inclusion problems

2021 ◽  
Author(s):  
Zheng Zhou ◽  
Bing Tan ◽  
Songxiao Li
Keyword(s):  
Descent Method ◽  
Steepest Descent ◽  
Steepest Descent Method ◽  
Variational Inclusion ◽  
Iterative Sequence ◽  
Inclusion Problem ◽  
Step Size ◽  
Hybrid Steepest Descent Method ◽  
Variational Inclusion Problem ◽  
Size Criterion

In this paper, we discuss the split monotone variational inclusion problem and propose two new inertial algorithms in infinite-dimensional Hilbert spaces. As well as, the iterative sequence by the proposed algorithms converges strongly to the solution of a certain variational inequality with the help of the hybrid steepest descent method. Furthermore, an adaptive step size criterion is considered in suggested algorithms to avoid the difficulty of calculating the operator norm. Finally, some numerical experiments show that our algorithms are realistic and summarize the known results.

scholarly journals The exact worst-case convergence rate of the gradient method with fixed step lengths for L-smooth functions

2021 ◽  
Author(s):  
Hadi Abbaszadehpeivasti ◽  
Etienne de Klerk ◽  
Moslem Zamani
Keyword(s):  
Convergence Rate ◽  
Gradient Method ◽  
Stationary Point ◽  
Smooth Function ◽  
Step Length ◽  
Descent Method ◽  
Steepest Descent ◽  
Steepest Descent Method ◽  
Smooth Functions ◽  
Worst Case

AbstractIn this paper, we study the convergence rate of the gradient (or steepest descent) method with fixed step lengths for finding a stationary point of an L-smooth function. We establish a new convergence rate, and show that the bound may be exact in some cases, in particular when all step lengths lie in the interval (0, 1/L]. In addition, we derive an optimal step length with respect to the new bound.

scholarly journals A steepest descent algorithm for the optimal control of a cascaded hydropower system

2020 ◽  
Vol 10 (4) ◽  
pp. 4136
Author(s):  
Olalekan Ogunbiyi ◽  
Cornelius T. Thomas ◽  
Oludare Y. Ogundepo ◽  
Isaac O. A. Omeiza ◽  
Jimoh Akanni ◽  
...  
Keyword(s):  
Power Generation ◽  
Descent Method ◽  
Steepest Descent ◽  
Steepest Descent Method ◽  
Hydroelectric Power ◽  
Step Size ◽  
Infinite Dimensional ◽  
Optimal Power ◽  
Hydropower System ◽  
And Control

Optimal power generation along the cascaded Kainji-Jebba hydroelectric power system had been very difficult to achieve. The reservoirs operating heads are being affected by possible variation in impoundments upstream, stochastic factors that are weather-related, availability of the turbo-alternators and power generated at any time. Proposed in this paper, is an algorithm for solving the optimal release of water on the cascaded hydropower system based on steepest descent method. The uniqueness of this work is the conversion of the infinite dimensional control problem to a finite one, the introduction of clever techniques for choosing the steepest descent step size in each iteration and the nonlinear penalty embedded in the procedure. The control algorithm was implemented in an Excel VBA® environment to solve the ormulated Lagrange problem within an accuracy of 0.03%. It is recommended for use in system studies and control design for the optimal power generation in the cascaded hydropower system.

Sign in / Sign up

Export Citation Format

Share Document