INITIAL IMPROVEMENT OF THE HYBRID ACCELERATED GRADIENT DESCENT PROCESS

2018 ◽  
Vol 98 (2) ◽  
pp. 331-338 ◽  
Author(s):  
STEFAN PANIĆ ◽  
MILENA J. PETROVIĆ ◽  
MIROSLAVA MIHAJLOV CAREVIĆ
Keyword(s):  
Gradient Descent ◽  
Line Search ◽  
Initial Step ◽  
Step Length ◽  
Descent Method ◽  
Gradient Descent Method ◽  
Convergence Properties ◽  
Backtracking Line Search ◽  
Initial Improvement ◽  
Accelerated Gradient

We improve the convergence properties of the iterative scheme for solving unconstrained optimisation problems introduced in Petrovic et al. [‘Hybridization of accelerated gradient descent method’, Numer. Algorithms (2017), doi:10.1007/s11075-017-0460-4] by optimising the value of the initial step length parameter in the backtracking line search procedure. We prove the validity of the algorithm and illustrate its advantages by numerical experiments and comparisons.

scholarly journals Coordinate Descent-Based Sparse Nonnegative Matrix Factorization for Robust Cancer-Class Discovery and Microarray Data Analysis

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Melisew Tefera Belachew
Keyword(s):  
Microarray Data ◽  
Matrix Factorization ◽  
Data Science ◽  
Nonnegative Matrix Factorization ◽  
Real Life ◽  
Nonnegative Matrix ◽  
Descent Method ◽  
Coordinate Descent ◽  
Microarray Data Analysis ◽  
Robust Clustering

Determining the number of clusters in high-dimensional real-life datasets and interpreting the final outcome are among the challenging problems in data science. Discovering the number of classes in cancer and microarray data plays a vital role in the treatment and diagnosis of cancers and other related diseases. Nonnegative matrix factorization (NMF) plays a paramount role as an efficient data exploratory tool for extracting basis features inherent in massive data. Some algorithms which are based on incorporating sparsity constraints in the nonconvex NMF optimization problem are applied in the past for analyzing microarray datasets. However, to the best of our knowledge, none of these algorithms use block coordinate descent method which is known for providing closed form solutions. In this paper, we apply an algorithm developed based on columnwise partitioning and rank-one matrix approximation. We test this algorithm on two well-known cancer datasets: leukemia and multiple myeloma. The numerical results indicate that the proposed algorithm performs significantly better than related state-of-the-art methods. In particular, it is shown that this method is capable of robust clustering and discovering larger cancer classes in which the cluster splits are stable.

scholarly journals A Transformation of Accelerated Double Step Size Method for Unconstrained Optimization

2015 ◽  
Vol 2015 ◽  
pp. 1-8 ◽  
Author(s):  
Predrag S. Stanimirović ◽  
Gradimir V. Milovanović ◽  
Milena J. Petrović ◽  
Nataša Z. Kontrec
Keyword(s):  
Gradient Descent ◽  
Linear Convergence ◽  
Step Length ◽  
Descent Method ◽  
Single Step ◽  
Gradient Descent Method ◽  
Step Size ◽  
Double Step ◽  
Substantial Progress ◽  
Accelerated Gradient

A reduction of the originally double step size iteration into the single step length scheme is derived under the proposed condition that relates two step lengths in the accelerated double step size gradient descent scheme. The proposed transformation is numerically tested. Obtained results confirm the substantial progress in comparison with the single step size accelerated gradient descent method defined in a classical way regarding all analyzed characteristics: number of iterations, CPU time, and number of function evaluations. Linear convergence of derived method has been proved.

scholarly journals An Inexact Update Method with Double Parameters for Nonnegative Matrix Factorization

2016 ◽  
Vol 2016 ◽  
pp. 1-6
Author(s):  
Xiangli Li ◽  
Wen Zhang ◽  
Xiaoliang Dong ◽  
Juanjuan Shi
Keyword(s):  
Matrix Factorization ◽  
Gradient Descent ◽  
Nonnegative Matrix Factorization ◽  
Nonnegative Matrix ◽  
Optimal Solution ◽  
Active Set ◽  
Active Set Method ◽  
New Methods ◽  
Two Parameters ◽  
Multiplicative Update

Nonnegative matrix factorization (NMF) has been used as a powerful date representation tool in real world, because the nonnegativity of matrices is usually required. In recent years, many new methods are available to solve NMF in addition to multiplicative update algorithm, such as gradient descent algorithms, the active set method, and alternating nonnegative least squares (ANLS). In this paper, we propose an inexact update method, with two parameters, which can ensure that the objective function is always descent before the optimal solution is found. Experiment results show that the proposed method is effective.

Temporal community detection based on symmetric nonnegative matrix factorization

2017 ◽  
Vol 31 (13) ◽  
pp. 1750102 ◽  
Author(s):  
Pengfei Jiao ◽  
Haodong Lyu ◽  
Xiaoming Li ◽  
Wei Yu ◽  
Wenjun Wang
Keyword(s):  
Community Detection ◽  
Matrix Factorization ◽  
Gradient Descent ◽  
Nonnegative Matrix Factorization ◽  
Nonnegative Matrix ◽  
Essential Property ◽  
Evolving Networks ◽  
Gradient Descent Algorithm ◽  
Symmetric Nonnegative Matrix Factorization ◽  
Internal Evolution

To understand time-evolving networks, researchers should not only concentrate on the community structures, an essential property of complex networks, in each snapshot, but also study the internal evolution of the entire networks. Temporal communities provide insights into such mechanism, i.e., how the communities emerge, expand, shrink, merge, split and decay over time. Based on the symmetric nonnegative matrix factorization (SNMF), we present a dynamic model to detect temporal communities, which not only could find a well community structure in a given snapshot but also demands the results bear some similarity to the partition obtained from the previous snapshot. Moreover, our method can handle the situation that of the number of community changes in the networks. Also, a gradient descent algorithm is proposed to optimize the objective function of the model. Experimental results on both the synthetic and real-world networks indicate that our method outperforms the state-of-art methods for temporal community detection.

Sign in / Sign up

Export Citation Format

Share Document