scholarly journals Approximated least-squares solutions of a generalized Sylvester-transpose matrix equation via gradient-descent iterative algorithm

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Adisorn Kittisopaporn ◽  
Pattrawut Chansangiam
Keyword(s):  
Least Squares ◽  
Iterative Algorithm ◽  
Gradient Descent ◽  
Main Idea ◽  
Full Column Rank ◽  
Minimum Error ◽  
Matrix Coefficients ◽  
Special Cases ◽  
Efficiency And Effectiveness ◽  
Associated Matrix

AbstractThis paper proposes an effective gradient-descent iterative algorithm for solving a generalized Sylvester-transpose equation with rectangular matrix coefficients. The algorithm is applicable for the equation and its interesting special cases when the associated matrix has full column-rank. The main idea of the algorithm is to have a minimum error at each iteration. The algorithm produces a sequence of approximated solutions converging to either the unique solution, or the unique least-squares solution when the problem has no solution. The convergence analysis points out that the algorithm converges fast for a small condition number of the associated matrix. Numerical examples demonstrate the efficiency and effectiveness of the algorithm compared to renowned and recent iterative methods.

scholarly journals An Iterative Algorithm for the Reflexive Solution of the General Coupled Matrix Equations

2013 ◽  
Vol 2013 ◽  
pp. 1-15
Author(s):  
Zhongli Zhou ◽  
Guangxin Huang
Keyword(s):  
Iterative Algorithm ◽  
Matrix Group ◽  
Frobenius Norm ◽  
Matrix Equations ◽  
Matrix Solution ◽  
Initial Matrix ◽  
Special Cases ◽  
Reflexive Matrix ◽  
General Coupled Matrix Equations ◽  
Sylvester Matrix Equations

The general coupled matrix equations (including the generalized coupled Sylvester matrix equations as special cases) have numerous applications in control and system theory. In this paper, an iterative algorithm is constructed to solve the general coupled matrix equations over reflexive matrix solution. When the general coupled matrix equations are consistent over reflexive matrices, the reflexive solution can be determined automatically by the iterative algorithm within finite iterative steps in the absence of round-off errors. The least Frobenius norm reflexive solution of the general coupled matrix equations can be derived when an appropriate initial matrix is chosen. Furthermore, the unique optimal approximation reflexive solution to a given matrix group in Frobenius norm can be derived by finding the least-norm reflexive solution of the corresponding general coupled matrix equations. A numerical example is given to illustrate the effectiveness of the proposed iterative algorithm.

scholarly journals An Iterative Algorithm for the Generalized Reflexive Solutions of the Generalized Coupled Sylvester Matrix Equations

2012 ◽  
Vol 2012 ◽  
pp. 1-28 ◽  
Author(s):  
Feng Yin ◽  
Guang-Xin Huang
Keyword(s):  
Iterative Algorithm ◽  
Special Kind ◽  
Frobenius Norm ◽  
Matrix Equations ◽  
Sylvester Matrix ◽  
Special Cases ◽  
The Matrix ◽  
Solution Pair ◽  
Sylvester Matrix Equations ◽  
Matrix Pair

An iterative algorithm is constructed to solve the generalized coupled Sylvester matrix equations(AXB-CYD,EXF-GYH)=(M,N), which includes Sylvester and Lyapunov matrix equations as special cases, over generalized reflexive matricesXandY. When the matrix equations are consistent, for any initial generalized reflexive matrix pair[X1,Y1], the generalized reflexive solutions can be obtained by the iterative algorithm within finite iterative steps in the absence of round-off errors, and the least Frobenius norm generalized reflexive solutions can be obtained by choosing a special kind of initial matrix pair. The unique optimal approximation generalized reflexive solution pair[X̂,Ŷ]to a given matrix pair[X0,Y0]in Frobenius norm can be derived by finding the least-norm generalized reflexive solution pair[X̃*,Ỹ*]of a new corresponding generalized coupled Sylvester matrix equation pair(AX̃B-CỸD,EX̃F-GỸH)=(M̃,Ñ), whereM̃=M-AX0B+CY0D,Ñ=N-EX0F+GY0H. Several numerical examples are given to show the effectiveness of the presented iterative algorithm.

scholarly journals Estimation and filtering of fractional generalised random fields

2000 ◽  
Vol 69 (3) ◽  
pp. 336-361 ◽  
Author(s):  
J. M. Angulo ◽  
M. D. Ruiz-Medina ◽  
V. V. Anh
Keyword(s):  
Brownian Motion ◽  
Random Field ◽  
Least Squares ◽  
Random Fields ◽  
General Class ◽  
Duality Theory ◽  
Weak Sense ◽  
Linear Estimation ◽  
Special Cases ◽  
Estimation And Filtering

AbstractThis paper considers the estimation and filtering of fractional random fields, of which fractional Brownian motion and fractional Riesz-Bessel motion are important special cases. A least-squares solution to the problem is derived by using the duality theory and covariance factorisation of fractional generalised random fields. The minimum fractional duality order of the information random field leads to the most general class of solutions corresponding to the largest function space where the output random field can be approximated. The second-order properties that define the class of random fields for which the least-squares linear estimation problem is solved in a weak-sense are also investigated in terms of the covariance spectrum of the information random field.

scholarly journals Credibility Approximations for Bayesian Prediction of Second Moments

1985 ◽  
Vol 15 (2) ◽  
pp. 103-121 ◽  
Author(s):  
William S. Jewell ◽  
Rene Schnieper
Keyword(s):  
Least Squares ◽  
Linear Functions ◽  
Quadratic Functions ◽  
Sample Variance ◽  
Credibility Theory ◽  
Sample Mean ◽  
Second Moment ◽  
Second Moments ◽  
Special Cases ◽  
The Mean

AbstractCredibility theory refers to the use of linear least-squares theory to approximate the Bayesian forecast of the mean of a future observation; families are known where the credibility formula is exact Bayesian. Second-moment forecasts are also of interest, for example, in assessing the precision of the mean estimate. For some of these same families, the second-moment forecast is exact in linear and quadratic functions of the sample mean. On the other hand, for the normal distribution with normal-gamma prior on the mean and variance, the exact forecast of the variance is a linear function of the sample variance and the squared deviation of the sample mean from the prior mean. Bühlmann has given a credibility approximation to the variance in terms of the sample mean and sample variance.In this paper, we present a unified approach to estimating both first and second moments of future observations using linear functions of the sample mean and two sample second moments; the resulting least-squares analysis requires the solution of a 3 × 3 linear system, using 11 prior moments from the collective and giving joint predictions of all moments of interest. Previously developed special cases follow immediately. For many analytic models of interest, 3-dimensional joint prediction is significantly better than independent forecasts using the “natural” statistics for each moment when the number of samples is small. However, the expected squared-errors of the forecasts become comparable as the sample size increases.

scholarly journals LSMR: An Iterative Algorithm for Sparse Least-Squares Problems

2011 ◽  
Vol 33 (5) ◽  
pp. 2950-2971 ◽  
Author(s):  
David Chin-Lung Fong ◽  
Michael Saunders
Keyword(s):  
Least Squares ◽  
Iterative Algorithm ◽  
Least Squares Problems

scholarly journals Gradient Descent with Identity Initialization Efficiently Learns Positive-Definite Linear Transformations by Deep Residual Networks

2019 ◽  
Vol 31 (3) ◽  
pp. 477-502 ◽  
Author(s):  
Peter L. Bartlett ◽  
David P. Helmbold ◽  
Philip M. Long
Keyword(s):  
Least Squares ◽  
Gradient Descent ◽  
Singular Values ◽  
Negative Eigenvalue ◽  
Positive Definite ◽  
Quadratic Loss ◽  
Linear Transformations ◽  
Excess Loss ◽  
Initial Hypothesis ◽  
Matrix Formula

We analyze algorithms for approximating a function [Formula: see text] mapping [Formula: see text] to [Formula: see text] using deep linear neural networks, that is, that learn a function [Formula: see text] parameterized by matrices [Formula: see text] and defined by [Formula: see text]. We focus on algorithms that learn through gradient descent on the population quadratic loss in the case that the distribution over the inputs is isotropic. We provide polynomial bounds on the number of iterations for gradient descent to approximate the least-squares matrix [Formula: see text], in the case where the initial hypothesis [Formula: see text] has excess loss bounded by a small enough constant. We also show that gradient descent fails to converge for [Formula: see text] whose distance from the identity is a larger constant, and we show that some forms of regularization toward the identity in each layer do not help. If [Formula: see text] is symmetric positive definite, we show that an algorithm that initializes [Formula: see text] learns an [Formula: see text]-approximation of [Formula: see text] using a number of updates polynomial in [Formula: see text], the condition number of [Formula: see text], and [Formula: see text]. In contrast, we show that if the least-squares matrix [Formula: see text] is symmetric and has a negative eigenvalue, then all members of a class of algorithms that perform gradient descent with identity initialization, and optionally regularize toward the identity in each layer, fail to converge. We analyze an algorithm for the case that [Formula: see text] satisfies [Formula: see text] for all [Formula: see text] but may not be symmetric. This algorithm uses two regularizers: one that maintains the invariant [Formula: see text] for all [Formula: see text] and the other that “balances” [Formula: see text] so that they have the same singular values.

scholarly journals Semi-Supervised Minimum Error Entropy Principle with Distributed Method

Entropy ◽  
2018 ◽  
Vol 20 (12) ◽  
pp. 968
Author(s):  
Baobin Wang ◽  
Ting Hu
Keyword(s):  
Gradient Descent ◽  
Mean Squared Error ◽  
Learning Ability ◽  
Minimum Error ◽  
Additional Information ◽  
Squared Error ◽  
Gradient Descent Algorithm ◽  
Entropy Principle ◽  
Non Gaussian ◽  
Distributed Method

The minimum error entropy principle (MEE) is an alternative of the classical least squares for its robustness to non-Gaussian noise. This paper studies the gradient descent algorithm for MEE with a semi-supervised approach and distributed method, and shows that using the additional information of unlabeled data can enhance the learning ability of the distributed MEE algorithm. Our result proves that the mean squared error of the distributed gradient descent MEE algorithm can be minimax optimal for regression if the number of local machines increases polynomially as the total datasize.

Sign in / Sign up

Export Citation Format

Share Document