Smooth Approximation of Lipschitz Maps and Their Subgradients

We derive new representations for the generalised Jacobian of a locally Lipschitz map between finite dimensional real Euclidean spaces as the lower limit (i.e., limit inferior) of the classical derivative of the map where it exists. The new representations lead to significantly shorter proofs for the basic properties of the subgradient and the generalised Jacobian including the chain rule. We establish that a sequence of locally Lipschitz maps between finite dimensional Euclidean spaces converges to a given locally Lipschitz map in the L-topology—that is, the weakest refinement of the sup norm topology on the space of locally Lipschitz maps that makes the generalised Jacobian a continuous functional—if and only if the limit superior of the sequence of directional derivatives of the maps in a given vector direction coincides with the generalised directional derivative of the given map in that direction, with the convergence to the limit superior being uniform for all unit vectors. We then prove our main result that the subspace of Lipschitz C ∞ maps between finite dimensional Euclidean spaces is dense in the space of Lipschitz maps equipped with the L-topology, and, for a given Lipschitz map, we explicitly construct a sequence of Lipschitz C ∞ maps converging to it in the L-topology, allowing global smooth approximation of a Lipschitz map and its differential properties. As an application, we obtain a short proof of the extension of Green’s theorem to interval-valued vector fields. For infinite dimensions, we show that the subgradient of a Lipschitz map on a Banach space is upper continuous, and, for a given real-valued Lipschitz map on a separable Banach space, we construct a sequence of Gateaux differentiable functions that converges to the map in the sup norm topology such that the limit superior of the directional derivatives in any direction coincides with the generalised directional derivative of the Lipschitz map in that direction.

Manifolds of mappings on Cartesian products

Given smooth manifolds $$M_1,\ldots , M_n$$ M 1 , … , M n (which may have a boundary or corners), a smooth manifold N modeled on locally convex spaces and $$\alpha \in ({{\mathbb {N}}}_0\cup \{\infty \})^n$$ α ∈ ( N 0 ∪ { ∞ } ) n , we consider the set $$C^\alpha (M_1\times \cdots \times M_n,N)$$ C α ( M 1 × ⋯ × M n , N ) of all mappings $$f:M_1\times \cdots \times M_n\rightarrow N$$ f : M 1 × ⋯ × M n → N which are $$C^\alpha $$ C α in the sense of Alzaareer. Such mappings admit, simultaneously, continuous iterated directional derivatives of orders $$\le \alpha _j$$ ≤ α j in the jth variable for $$j\in \{1,\ldots , n\}$$ j ∈ { 1 , … , n } , in local charts. We show that $$C^\alpha (M_1\times \cdots \times M_n,N)$$ C α ( M 1 × ⋯ × M n , N ) admits a canonical smooth manifold structure whenever each $$M_j$$ M j is compact and N admits a local addition. The case of non-compact domains is also considered.

Estimates of Generalized Hessians for Optimal Value Functions in Mathematical Programming

We consider the optimal value function of a parametric optimization problem. A large number of publications have been dedicated to the study of continuity and differentiability properties of the function. However, the differentiability aspect of works in the current literature has mostly been limited to first order analysis, with focus on estimates of its directional derivatives and subdifferentials, given that the function is typically nonsmooth. With the progress made in the last two to three decades in major subfields of optimization such as robust, minmax, semi-infinite and bilevel optimization, and their connection to the optimal value function, there is a need for a second order analysis of the generalized differentiability properties of this function. This could enable the development of robust solution algorithms, such as the Newton method. The main goal of this paper is to provide estimates of the generalized Hessian for the optimal value function. Our results are based on two handy tools from parametric optimization, namely the optimal solution and Lagrange multiplier mappings, for which completely detailed estimates of their generalized derivatives are either well-known or can easily be obtained.

Beyond gradients: Noise correlations control Hebbian plasticity to shape credit assignment

Two key problems that span biological and industrial neural network research are how networks can be trained to generalize well and to minimize destructive interference between tasks. Both hinge on credit assignment, the targeting of specific network weights for change. In artificial networks, credit assignment is typically governed by gradient descent. Biological learning is thus often analyzed as a means to approximate gradients. We take the complementary perspective that biological learning rules likely confer advantages when they aren't gradient approximations. Further, we hypothesized that noise correlations, often considered detrimental, could usefully shape this learning. Indeed, we show that noise and three-factor plasticity interact to compute directional derivatives of reward, which can improve generalization, robustness to interference, and multi-task learning. This interaction also provides a method for routing learning quasi-independently of activity and connectivity, and demonstrates how biologically inspired inductive biases can be fruitfully embedded in learning algorithms.

Lateral directional aircraft aerodynamic parameter estimation using adaptive stochastic nonlinear filter

This paper aims to accurately estimate the lateral directional aerodynamic parameters in real time irrespective of the variations in the process and measurement covariance matrices. The proposed algorithm for parameter estimation is based on the integration of adaptive techniques into a stochastic nonlinear filter. The proposed adaptive estimation algorithm is applied to flight test data, and the lateral directional derivatives are estimated in real time. The estimates are compared with those obtained from the Filter Error Method (FEM), an offline parameter estimation method accounting for process noise. The estimation results are observed to be very comparable, and the supremacy of the adaptive filter is illustrated by varying the covariance matrices of both process and measurement noises. The parameters estimated by the adaptive filter are found to converge to their actual values, whereas the estimates of the regular filter are observed to diverge from the actual values when changing the noise covariance matrices. The proposed adaptive algorithm can estimate the lateral directional aerodynamic derivatives more accurately without prior knowledge of either process or measurement noise covariance matrices. Hence, it is of great value in online implementations.

Decoupling on the Wiener Space, Related Besov Spaces, and Applications to BSDEs

We introduce a decoupling method on the Wiener space to define a wide class of anisotropic Besov spaces. The decoupling method is based on a general distributional approach and not restricted to the Wiener space. The class of Besov spaces we introduce contains the traditional isotropic Besov spaces obtained by the real interpolation method, but also new spaces that are designed to investigate backwards stochastic differential equations (BSDEs). As examples we discuss the Besov regularity (in the sense of our spaces) of forward diffusions and local times. It is shown that among our newly introduced Besov spaces there are spaces that characterize quantitative properties of directional derivatives in the Malliavin sense without computing or accessing these Malliavin derivatives explicitly. Regarding BSDEs, we deduce regularity properties of the solution processes from the Besov regularity of the initial data, in particular upper bounds for their L p L_p -variation, where the generator might be of quadratic type and where no structural assumptions, for example in terms of a forward diffusion, are assumed. As an example we treat sub-quadratic BSDEs with unbounded terminal conditions. Among other tools, we use methods from harmonic analysis. As a by-product, we improve the asymptotic behaviour of the multiplicative constant in a generalized Fefferman inequality and verify the optimality of the bound we established.

Fractional Line Integral

This paper proposed a definition of the fractional line integral, generalising the concept of the fractional definite integral. The proposal replicated the properties of the classic definite integral, namely the fundamental theorem of integral calculus. It was based on the concept of the fractional anti-derivative used to generalise the Barrow formula. To define the fractional line integral, the Grünwald–Letnikov and Liouville directional derivatives were introduced and their properties described. The integral was defined for a piecewise linear path first and, from it, for any regular curve.

Fractional Line Integral

This paper proposes a definition of fractional line integral, generalising the concept of fractional definite integral. The proposal replicates the properties of the classic definite integral, namely the fundamental theorem of integral calculus. It is based on the concept of fractional anti-derivative used to generalise the Barrow formula. To define the fractional line integrals the Gr\"unwald-Letnikov and Liouville directional derivatives are introduced and their properties described. The integral is defined first for broken line paths and afterwards to any regular curve