Affine-invariant contracting-point methods for Convex Optimization

In this paper, we develop new affine-invariant algorithms for solving composite convex minimization problems with bounded domain. We present a general framework of Contracting-Point methods, which solve at each iteration an auxiliary subproblem restricting the smooth part of the objective function onto contraction of the initial domain. This framework provides us with a systematic way for developing optimization methods of different order, endowed with the global complexity bounds. We show that using an appropriate affine-invariant smoothness condition, it is possible to implement one iteration of the Contracting-Point method by one step of the pure tensor method of degree $$p \ge 1$$ p ≥ 1 . The resulting global rate of convergence in functional residual is then $${\mathcal {O}}(1 / k^p)$$ O ( 1 / k p ) , where k is the iteration counter. It is important that all constants in our bounds are affine-invariant. For $$p = 1$$ p = 1 , our scheme recovers well-known Frank–Wolfe algorithm, providing it with a new interpretation by a general perspective of tensor methods. Finally, within our framework, we present efficient implementation and total complexity analysis of the inexact second-order scheme $$(p = 2)$$ ( p = 2 ) , called Contracting Newton method. It can be seen as a proper implementation of the trust-region idea. Preliminary numerical results confirm its good practical performance both in the number of iterations, and in computational time.

Online Linear Programming: Dual Convergence, New Algorithms, and Regret Bounds

We study an online linear programming (OLP) problem under a random input model in which the columns of the constraint matrix along with the corresponding coefficients in the objective function are independently and identically drawn from an unknown distribution and revealed sequentially over time. Virtually all existing online algorithms were based on learning the dual optimal solutions/prices of the linear programs (LPs), and their analyses were focused on the aggregate objective value and solving the packing LP, where all coefficients in the constraint matrix and objective are nonnegative. However, two major open questions were as follows. (i) Does the set of LP optimal dual prices learned in the existing algorithms converge to those of the “offline” LP? (ii) Could the results be extended to general LP problems where the coefficients can be either positive or negative? We resolve these two questions by establishing convergence results for the dual prices under moderate regularity conditions for general LP problems. Specifically, we identify an equivalent form of the dual problem that relates the dual LP with a sample average approximation to a stochastic program. Furthermore, we propose a new type of OLP algorithm, action-history-dependent learning algorithm, which improves the previous algorithm performances by taking into account the past input data and the past decisions/actions. We derive an [Formula: see text] regret bound (under a locally strong convexity and smoothness condition) for the proposed algorithm, against the [Formula: see text] bound for typical dual-price learning algorithms, where n is the number of decision variables. Numerical experiments demonstrate the effectiveness of the proposed algorithm and the action-history-dependent design.

Thin homotopy and the holonomy approach to gauge theories

We survey several mathematical developments in the holonomy approach to gauge theory. A cornerstone of this approach is the introduction of group structures on spaces of based loops on a smooth manifold, relying on certain homotopy equivalence relations — such as the so-called thin homotopy — and the resulting interpretation of gauge fields as group homomorphisms to a Lie group G G satisfying a suitable smoothness condition, encoding the holonomy of a gauge orbit of smooth connections on a principal G G -bundle. We also prove several structural results on thin homotopy, and in particular we clarify the difference between thin equivalence and retrace equivalence for piecewise-smooth based loops on a smooth manifold, which are often used interchangeably in the physics literature. We conclude by listing a set of questions on topological and functional analytic aspects of groups of based loops, which we consider to be fundamental to establish a rigorous differential geometric foundation of the holonomy formulation of gauge theory.

A Few Bad Apples Spoil the Barrel: An Anti-Folk Theorem for Anonymous Repeated Games with Incomplete Information

We study anonymous repeated games where players may be “commitment types” who always take the same action. We establish a stark anti-folk theorem: if the distribution of the number of commitment types satisfies a smoothness condition and the game has a “pairwise dominant” action, this action is almost always taken. This implies that cooperation is impossible in the repeated prisoner's dilemma with anonymous random matching. We also bound equilibrium payoffs for general games. Our bound implies that industry profits converge to zero in linear-demand Cournot oligopoly as the number of firms increases. (JEL C72, C73, D83)

A new form of the Saint-Venant equations for variable topography

The solution stability of river models using the one-dimensional (1D) Saint-Venant equations can be easily undermined when source terms in the discrete equations do not satisfy the Lipschitz smoothness condition for partial differential equations. Although instability issues have been previously noted, they are typically treated as model implementation issues rather than as underlying problems associated with the form of the governing equations. This study proposes a new reference slope form of the Saint-Venant equations to ensure smooth slope source terms and eliminate one source of potential numerical oscillations. It is shown that a simple algebraic transformation of channel geometry provides a smooth reference slope while preserving the correct cross-section flow area and the total Piezometric pressure gradient that drives the flow. The reference slope method ensures the slope source term in the governing equations is Lipschitz continuous while maintaining all the underlying complexity of the real-world geometry. The validity of the mathematical concept is demonstrated with the open-source Simulation Program for River Networks (SPRNT) model in a series of artificial test cases and a simulation of a small urban creek. Validation comparisons are made with analytical solutions and the Hydrologic Engineering Center's River Analysis System (HEC-RAS) model. The new method reduces numerical oscillations and instabilities without requiring ad hoc smoothing algorithms.

A Question of Norton–Sullivan in the Analytic Case

In 1996, A. Norton and D. Sullivan asked the following question: If $f:\mathbb{T}^2\rightarrow \mathbb{T}^2$ is a diffeomorphism, $h:\mathbb{T}^2\rightarrow \mathbb{T}^2$ is a continuous map homotopic to the identity, and $h f=T_{\rho } h$, where $\rho \in \mathbb{R}^2$ is a totally irrational vector and $T_{\rho }:\mathbb{T}^2\rightarrow \mathbb{T}^2,\, z\mapsto z+\rho $ is a translation, are there natural geometric conditions (e.g., smoothness) on $f$ that force $h$ to be a homeomorphism? In [ 22], the 1st author and Z. Zhang gave a negative answer to the above question in the $C^{\infty }$ category: in general, not even the infinite smoothness condition can force $h$ to be a homeomorphism. In this article, we give a negative answer in the $C^{\omega }$ category (see also [ 22, Question 3]): we construct a real analytic conservative and minimal totally irrational pseudo-rotation of $\mathbb{T}^2$ that is semi-conjugate to a translation but not conjugate to a translation.

A new form of the Saint–Venant equations for variable topography

The solution stability of river models using the one-dimensional (1D) Saint–Venant equations can be easily undermined when source terms in the discrete equations do not satisfy the Lipschitz smoothness condition for partial differential equations. Although instability issues have been previously noted, they are typically treated as model implementation issues rather than as underlying problems associated with the form of the governing equations. This study proposes a new reference slope form of the Saint–Venant equations to ensure smooth source terms and eliminate potential numerical oscillations. It is shown that a simple algebraic transformation of channel geometry provides a smooth reference slope while preserving the correct cross-sectional flow area and the total Piezometric pressure gradient that drives the flow. The reference slope method ensures the slope source term in the governing equations is Lipschitz-continuous while maintaining all the underlying complexity of the real-world geometry. The validity of the mathematical concept is demonstrated with the open-source SPRNT model in a series of artificial test cases and simulation of a small urban creek. Validation comparisons are made with analytical solutions and the HEC-RAS model. The new method reduces numerical oscillations and instabilities without requiring ad hoc smoothing algorithms.

Optimal learning with Gaussians and correntropy loss

Correntropy-based learning has achieved great success in practice during the last decades. It is originated from information-theoretic learning and provides an alternative to classical least squares method in the presence of non-Gaussian noise. In this paper, we investigate the theoretical properties of learning algorithms generated by Tikhonov regularization schemes associated with Gaussian kernels and correntropy loss. By choosing an appropriate scale parameter of Gaussian kernel, we show the polynomial decay of approximation error under a Sobolev smoothness condition. In addition, we employ a tight upper bound for the uniform covering number of Gaussian RKHS in order to improve the estimate of sample error. Based on these two results, we show that the proposed algorithm using varying Gaussian kernel achieves the minimax rate of convergence (up to a logarithmic factor) without knowing the smoothness level of the regression function.

Implementable tensor methods in unconstrained convex optimization

In this paper we develop new tensor methods for unconstrained convex optimization, which solve at each iteration an auxiliary problem of minimizing convex multivariate polynomial. We analyze the simplest scheme, based on minimization of a regularized local model of the objective function, and its accelerated version obtained in the framework of estimating sequences. Their rates of convergence are compared with the worst-case lower complexity bounds for corresponding problem classes. Finally, for the third-order methods, we suggest an efficient technique for solving the auxiliary problem, which is based on the recently developed relative smoothness condition (Bauschke et al. in Math Oper Res 42:330–348, 2017; Lu et al. in SIOPT 28(1):333–354, 2018). With this elaboration, the third-order methods become implementable and very fast. The rate of convergence in terms of the function value for the accelerated third-order scheme reaches the level $$O\left( {1 \over k^4}\right) $$O1k4, where k is the number of iterations. This is very close to the lower bound of the order $$O\left( {1 \over k^5}\right) $$O1k5, which is also justified in this paper. At the same time, in many important cases the computational cost of one iteration of this method remains on the level typical for the second-order methods.