A New Likelihood Ratio Method for Training Artificial Neural Networks

We investigate a new approach to compute the gradients of artificial neural networks (ANNs), based on the so-called push-out likelihood ratio method. Unlike the widely used backpropagation (BP) method that requires continuity of the loss function and the activation function, our approach bypasses this requirement by injecting artificial noises into the signals passed along the neurons. We show how this approach has a similar computational complexity as BP, and moreover is more advantageous in terms of removing the backward recursion and eliciting transparent formulas. We also formalize the connection between BP, a pivotal technique for training ANNs, and infinitesimal perturbation analysis, a classic path-wise derivative estimation approach, so that both our new proposed methods and BP can be better understood in the context of stochastic gradient estimation. Our approach allows efficient training for ANNs with more flexibility on the loss and activation functions, and shows empirical improvements on the robustness of ANNs under adversarial attacks and corruptions of natural noises. Summary of Contribution: Stochastic gradient estimation has been studied actively in simulation for decades and becomes more important in the era of machine learning and artificial intelligence. The stochastic gradient descent is a standard technique for training the artificial neural networks (ANNs), a pivotal problem in deep learning. The most popular stochastic gradient estimation technique is the backpropagation method. We find that the backpropagation method lies in the family of infinitesimal perturbation analysis, a path-wise gradient estimation technique in simulation. Moreover, we develop a new likelihood ratio-based method, another popular family of gradient estimation technique in simulation, for training more general ANNs, and demonstrate that the new training method can improve the robustness of the ANN.

AN IMPROVEMENT OF MARKOVIAN INTEGRATION BY PARTS FORMULA AND APPLICATION TO SENSITIVITY COMPUTATION

This paper establishes a new version of integration by parts formula of Markov chains for sensitivity computation, under much lower restrictions than the existing researches. Our approach is more fundamental and applicable without using Girsanov theorem or Malliavin calculus as did by past papers. Numerically, we apply this formula to compute sensitivity regarding the transition rate matrix and compare with a recent research by an IPA (infinitesimal perturbation analysis) method and other approaches.

The Limit Point of the Pentagram Map and Infinitesimal Monodromy

The pentagram map takes a planar polygon $P$ to a polygon $P^{\prime }$ whose vertices are the intersection points of the consecutive shortest diagonals of $P$. The orbit of a convex polygon under this map is a sequence of polygons that converges exponentially to a point. Furthermore, as recently proved by Glick, coordinates of that limit point can be computed as an eigenvector of a certain operator associated with the polygon. In the present paper, we show that Glick’s operator can be interpreted as the infinitesimal monodromy of the polygon. Namely, there exists a certain natural infinitesimal perturbation of a polygon, which is again a polygon but in general not closed; what Glick’s operator measures is the extent to which this perturbed polygon does not close up.

Static and stability analysis of partial slip texture multi-lobe journal bearings

This paper presents one-dimensional analysis of modified dynamic Reynolds equation derived for partial slip texture multi-lobe journal bearings. The novelty included in this study is the configuration of partial slip texture region on the bottom bearing lobe surface of a multi-lobe journal bearing under a constant vertical load. The nondimensional pressure and shear stress for steady-state and nondimensional pressure gradients for dynamic coefficients for each lobe with partial slip texture configuration are derived based on narrow groove theory. Linearized stability analysis is evaluated using infinitesimal perturbation method. Results of static and stability characteristics of partial slip texture multi-lobe (two-axial groove, elliptical, three-lobe and offset) journal bearings are presented. Partial slip texture configuration significantly enhances load capacity, coefficient of friction, and stability of two-lobe journal bearing.

Analysis of dynamic characteristics of spiral groove liquid film seal considering cavitation

Purpose The purpose of this paper is to investigate the dynamic characteristics of a spiral groove liquid film seal considering the effect of cavitation. Design/methodology/approach A mathematical model of a spiral groove liquid film seal was established based on the mass-conserving Jakobsson–Floberg–Olsson cavitation boundary condition. The film rupture and film reformation boundaries were assumed to be unchanged under infinitesimal perturbation conditions. Governing equations under steady and perturbed states were solved by the finite element method, and then the dynamic characteristics of the spiral groove liquid film seal were theoretically investigated considering the effect of cavitation. Findings The results indicate that dynamic coefficients considering cavitation are smaller than those neglecting cavitation. The difference value is consistent with the change in cavitation area. The liquid film seal does not suffer axial instability whether considering cavitation, but its angular instability is more likely to occur when cavitation is considered. Originality/value For liquid lubricated non-contacting mechanical seals, the dynamic characteristics considering cavitation are investigated. The results are expected to provide a theoretical basis for improving the design method of liquid film seals.

The Stability of the Aggregate Loss Distribution

In this article we introduce the stability analysis of a compound sum: it consists of computing the standardized variation of the survival function of the sum resulting from an infinitesimal perturbation of the common distribution of the summands. Stability analysis is complementary to the classical sensitivity analysis, which consists of computing the derivative of an important indicator of the model, with respect to a model parameter. We obtain a computational formula for this stability from the saddlepoint approximation. We apply the formula to the compound Poisson insurer loss with gamma individual claim amounts and to the compound geometric loss with Weibull individual claim amounts.