scholarly journals Inexact and Stochastic Generalized Conditional Gradient with Augmented Lagrangian and Proximal Step

2021 ◽  
Vol Volume 2 (Original research articles) ◽  
Author(s):  
Antonio Silveti-Falls ◽  
Cesare Molinari ◽  
Jalal Fadili
Keyword(s):  
Convergence Rates ◽  
Lower Semicontinuous ◽  
Almost Sure Convergence ◽  
Rates Of Convergence ◽  
Learning Problems ◽  
Bounded Linear ◽  
Infinite Dimensional ◽  
Dual Variables ◽  
Computationally Intensive ◽  
Semicontinuous Functions

In this paper we propose and analyze inexact and stochastic versions of the CGALP algorithm developed in [25], which we denote ICGALP , that allow for errors in the computation of several important quantities. In particular this allows one to compute some gradients, proximal terms, and/or linear minimization oracles in an inexact fashion that facilitates the practical application of the algorithm to computationally intensive settings, e.g., in high (or possibly infinite) dimensional Hilbert spaces commonly found in machine learning problems. The algorithm is able to solve composite minimization problems involving the sum of three convex proper lower-semicontinuous functions subject to an affine constraint of the form Ax = b for some bounded linear operator A. Only one of the functions in the objective is assumed to be differentiable, the other two are assumed to have an accessible proximal operator and a linear minimization oracle. As main results, we show convergence of the Lagrangian values (so-called convergence in the Bregman sense) and asymptotic feasibility of the affine constraint as well as strong convergence of the sequence of dual variables to a solution of the dual problem, in an almost sure sense. Almost sure convergence rates are given for the Lagrangian values and the feasibility gap for the ergodic primal variables. Rates in expectation are given for the Lagrangian values and the feasibility gap subsequentially in the pointwise sense. Numerical experiments verifying the predicted rates of convergence are shown as well.

Convergence rates in the law of large numbers. II

1968 ◽  
Vol 64 (2) ◽  
pp. 485-488 ◽  
Author(s):  
V. K. Rohatgi
Keyword(s):  
Convergence Rates ◽  
Random Variables ◽  
Almost Sure Convergence ◽  
Random Variable ◽  
Rates Of Convergence ◽  
Independent Random Variables ◽  
Large Numbers ◽  
Present Context ◽  
Image Position ◽  
Uniformly Bounded

Let {Xn: n ≥ 1} be a sequence of independent random variables and write Suppose that the random vairables Xn are uniformly bounded by a random variable X in the sense thatSet qn(x) = Pr(|Xn| > x) and q(x) = Pr(|Xn| > x). If qn ≤ q and E|X|r < ∞ with 0 < r < 2 then we have (see Loève(4), 242)where ak = 0, if 0 < r < 1, and = EXk if 1 ≤ r < 2 and ‘a.s.’ stands for almost sure convergence. the purpose of this paper is to study the rates of convergence ofto zero for arbitrary ε > 0. We shall extend to the present context, results of (3) where the case of identically distributed random variables was treated. The techniques used here are strongly related to those of (3).

scholarly journals Convergence Rates for Quantum Evolution and Entropic Continuity Bounds in Infinite Dimensions

2019 ◽  
Vol 374 (2) ◽  
pp. 823-871 ◽  
Author(s):  
Simon Becker ◽  
Nilanjana Datta
Keyword(s):  
Convergence Rates ◽  
Lipschitz Continuity ◽  
Open Quantum Systems ◽  
High Energy ◽  
Quantum Systems ◽  
Quantum Evolution ◽  
Infinite Dimensions ◽  
Quantum Brownian Motion ◽  
Infinite Dimensional ◽  
Energy Constrained

Abstract By extending the concept of energy-constrained diamond norms, we obtain continuity bounds on the dynamics of both closed and open quantum systems in infinite dimensions, which are stronger than previously known bounds. We extensively discuss applications of our theory to quantum speed limits, attenuator and amplifier channels, the quantum Boltzmann equation, and quantum Brownian motion. Next, we obtain explicit log-Lipschitz continuity bounds for entropies of infinite-dimensional quantum systems, and classical capacities of infinite-dimensional quantum channels under energy-constraints. These bounds are determined by the high energy spectrum of the underlying Hamiltonian and can be evaluated using Weyl’s law.

Sandwich Theorems for Semicontinuous Operators

1992 ◽  
Vol 35 (4) ◽  
pp. 463-474 ◽  
Author(s):  
J. M. Borwein ◽  
M. Théra
Keyword(s):  
Lower Semicontinuous ◽  
Lower Semicontinuous Functions ◽  
Classical Interpolation ◽  
Semicontinuous Functions

AbstractWe provide vector analogues of the classical interpolation theorems for lower semicontinuous functions due to Dowker and to Hahn and Katetov-Tong.

Elements of Spectral Theory for Generalized Derivations II : The Semifredholm Domain

1981 ◽  
Vol 33 (5) ◽  
pp. 1205-1231 ◽  
Author(s):  
Lawrence A. Fialkow
Keyword(s):  
Hilbert Spaces ◽  
Simple Formula ◽  
Decomposition Theorem ◽  
Linear Operators ◽  
Generalized Derivations ◽  
Bounded Linear ◽  
Infinite Dimensional ◽  
Hilbert Space Operators ◽  
Algebraic Invariants ◽  
Fredholm Theorem

Let and denote infinite dimensional Hilbert spaces and let denote the space of all bounded linear operators from to . For A in and B in , let τAB denote the operator on defined by τAB(X) = AX – XB. The purpose of this note is to characterize the semi-Fredholm domain of τAB (Corollary 3.16). Section 3 also contains formulas for ind(τAB – λ). These results depend in part on a decomposition theorem for Hilbert space operators corresponding to certain “singular points” of the semi-Fredholm domain (Theorem 2.2). Section 4 contains a particularly simple formula for ind(τAB – λ) (in terms of spectral and algebraic invariants of A and B) for the case when τAB – λ is Fredholm (Theorem 4.2). This result is used to prove that (τBA) = –ind(τAB) (Corollary 4.3). We also prove that when A and B are bi-quasi-triangular, then the semi-Fredholm domain of τAB contains no points corresponding to nonzero indices.

scholarly journals MDFEM: Multivariate decomposition finite element method for elliptic PDEs with lognormal diffusion coefficients using higher-order QMC and FEM

2021 ◽  
Author(s):  
Dong T.P. Nguyen ◽  
Dirk Nuyens
Keyword(s):  
Finite Element Method ◽  
Finite Element ◽  
Monte Carlo ◽  
Convergence Rates ◽  
Higher Order ◽  
Elliptic Pdes ◽  
Infinite Dimensional ◽  
Quasi Monte Carlo ◽  
Higher Order Convergence ◽  
Element Method

We introduce the \emph{multivariate decomposition finite element method} (MDFEM) for elliptic PDEs with lognormal diffusion coefficients, that is, when the diffusion coefficient has the form $a=\exp(Z)$ where $Z$ is a Gaussian random field defined by an infinite series expansion $Z(\bsy) = \sum_{j \ge 1} y_j \, \phi_j$ with $y_j \sim \calN(0,1)$ and a given sequence of functions $\{\phi_j\}_{j \ge 1}$. We use the MDFEM to approximate the expected value of a linear functional of the solution of the PDE which is an infinite-dimensional integral over the parameter space. The proposed algorithm uses the \emph{multivariate decomposition method} (MDM) to compute the infinite-dimensional integral by a decomposition into finite-dimensional integrals, which we resolve using \emph{quasi-Monte Carlo} (QMC) methods, and for which we use the \emph{finite element method} (FEM) to solve different instances of the PDE.   We develop higher-order quasi-Monte Carlo rules for integration over the finite-di\-men\-si\-onal Euclidean space with respect to the Gaussian distribution by use of a truncation strategy. By linear transformations of interlaced polynomial lattice rules from the unit cube to a multivariate box of the Euclidean space we achieve higher-order convergence rates for functions belonging to a class of \emph{anchored Gaussian Sobolev spaces} while taking into account the truncation error. These cubature rules are then used in the MDFEM algorithm.   Under appropriate conditions, the MDFEM achieves higher-order convergence rates in term of error versus cost, i.e., to achieve an accuracy of $O(\epsilon)$ the computational cost is $O(\epsilon^{-1/\lambda-\dd/\lambda}) = O(\epsilon^{-(p^* + \dd/\tau)/(1-p^*)})$ where $\epsilon^{-1/\lambda}$ and $\epsilon^{-\dd/\lambda}$ are respectively the cost of the quasi-Monte Carlo cubature and the finite element approximations, with $\dd = d \, (1+\ddelta)$ for some $\ddelta \ge 0$ and $d$ the physical dimension, and $0 < p^* \le (2 + \dd/\tau)^{-1}$ is a parameter representing the sparsity of $\{\phi_j\}_{j \ge 1}$.

scholarly journals The Strength of Nesterov's Extrapolation2019

2020 ◽  
Author(s):  
Qing Tao
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Convergence Rates ◽  
Optimization Problems ◽  
Gradient Methods ◽  
Learning Problems ◽  
Smooth Convex ◽  
Simple Modification ◽  
Convex Problems ◽  
Hinge Loss

The extrapolation strategy raised by Nesterov, which can accelerate the convergence rate of gradient descent methods by orders of magnitude when dealing with smooth convex objective, has led to tremendous success in training machine learning tasks. In this paper, we theoretically study its strength in the convergence of individual iterates of general non-smooth convex optimization problems, which we name \textit{individual convergence}. We prove that Nesterov's extrapolation is capable of making the individual convergence of projected gradient methods optimal for general convex problems, which is now a challenging problem in the machine learning community. In light of this consideration, a simple modification of the gradient operation suffices to achieve optimal individual convergence for strongly convex problems, which can be regarded as making an interesting step towards the open question about SGD posed by Shamir \cite{shamir2012open}. Furthermore, the derived algorithms are extended to solve regularized non-smooth learning problems in stochastic settings. {\color{blue}They can serve as an alternative to the most basic SGD especially in coping with machine learning problems, where an individual output is needed to guarantee the regularization structure while keeping an optimal rate of convergence.} Typically, our method is applicable as an efficient tool for solving large-scale $l_1$-regularized hinge-loss learning problems. Several real experiments demonstrate that the derived algorithms not only achieve optimal individual convergence rates but also guarantee better sparsity than the averaged solution.

scholarly journals Asymptotic behaviors with convergence rates of distributions of negative-binomial sums

2020 ◽  
Vol 22 (4) ◽  
pp. 415-421
Author(s):  
Tran Loc Hung ◽  
Phan Tri Kien ◽  
Nguyen Tan Nhut
Keyword(s):  
Limit Theorems ◽  
Negative Binomial ◽  
Convergence Rates ◽  
Random Variables ◽  
Rates Of Convergence ◽  
Weak Limit ◽  
Probability Metric ◽  
Weak Limit Theorems ◽  
Distribution Theorem ◽  
Binomial Sums

The negative-binomial sum is an extension of a geometric sum. It has been arisen from the necessity to resolve practical problems in telecommunications, network analysis, stochastic finance and insurance mathematics, etc. Up to the present, the topics related to negative-binomial sums like asymptotic distributions and rates of convergence have been investigated by many mathematicians. However, in a lot of various situations, the results concerned the rates of convergence for negative-binomial sums are still restrictive. The main purpose of this paper is to establish some weak limit theorems for negative-binomial sums of independent, identically distributed (i.i.d.) random variables via Gnedenko's Transfer Theorem originated by Gnedenko and Fahim (1969). Using Zolotarev's probability metric, the rate of convergence in weak limit theorems for negativebinomial sum are established. The received results are the rates of convergence in weak limit theorem for partial sum of i.i.d random variables related to symmetric stable distribution (Theorem 1), and asymptotic distribution together with the convergence rates for negative-binomial sums of i.i.d. random variables concerning to symmetric Linnik laws and Generalized Linnik distribution (Theorem 2 and Theorem 3). Based on the results of this paper, the analogous results for geometric sums of i.i.d. random variables will be concluded as direct consequences. However, the article has just been solved for the case of 1 <a < 2; it is quite hard to estimate in the case of a 2 (0;1) via the Zolotarev's probability metric. Mathematics Subject Classification 2010: 60G50; 60F05; 60E07.

Sign in / Sign up

Export Citation Format

Share Document