Theoretical Investigation of Generalization Bound for Residual Networks

This paper presents a framework for norm-based capacity control with respect to an lp,q-norm in weight-normalized Residual Neural Networks (ResNets). We first formulate the representation of each residual block. For the regression problem, we analyze the Rademacher Complexity of the ResNets family. We also establish a tighter generalization upper bound for weight-normalized ResNets. in a more general sight. Using the lp,q-norm weight normalization in which 1/p+1/q >=1, we discuss the properties of a width-independent capacity control, which only relies on the depth according to a square root term. Several comparisons suggest that our result is tighter than previous work. Parallel results for Deep Neural Networks (DNN) and Convolutional Neural Networks (CNN) are included by introducing the lp,q-norm weight normalization for DNN and the lp,q-norm kernel normalization for CNN. Numerical experiments also verify that ResNet structures contribute to better generalization properties.

Intersect a quartic to extract its roots

In this note we present a new method for determining the roots of a quartic polynomial, wherein the curve of the given quartic polynomial is intersected by the curve of a quadratic polynomial (which has two unknown coefficients) at its root point; so the root satisfies both the quartic and the quadratic equations. Elimination of the root term from the two equations leads to an expression in the two unknowns of quadratic polynomial. In addition, we introduce another expression in one unknown, which leads to determination of the two unknowns and subsequently the roots of quartic polynomial.

Second Order Duality in Multiobjective Fractional Programming with Square Root Term under Generalized Univex Function

Three approaches of second order mixed type duality are introduced for a nondifferentiable multiobjective fractional programming problem in which the numerator and denominator of objective function contain square root of positive semidefinite quadratic form. Also, the necessary and sufficient conditions of efficient solution for fractional programming are established and a parameterization technique is used to establish duality results under generalized second order ρ-univexity assumption.

Wolfe Type Second Order Nondifferentiable Symmetric Duality in Multiobjective Programming over Cone with Generalized (K, F)-Convexity

A new class of second order (K, F) pseudoconvex function is introduced with example. A pair of Wolfe type second order nondifferentiable symmetric dual programs over arbitrary cones with square root term is formulated. The duality results are established under second order (K, F) pseudoconvexity assumption. Also a Wolfe type second order minimax mixed integer programming problem is formulated and the symmetric duality results are established under second order (K, F) pseudoconvexity assumption.

Mond-Weir type second order multiobjective mixed symmetric duality with square root term under generalized univex function

In this paper, a new class of second order (Φ,ρ)-univex and second order (Φ,ρ)-pseudo univex function are introduced with example. A pair Mond-Weir type second order mixed symmetric duality for multiobjective nondifferentiable programming is formulated and the duality results are established under the mild assumption of second order (∅,ρ) univexity and second order pseudo univexity. Special cases are discussed to show that this study extends some of the known results in related domain..