scholarly journals A PERTURBATIVE APPROACH TO NONLINEARITIES IN THE INFORMATION CARRIED BY A TWO LAYER NEURAL NETWORK

2001 ◽  
Vol 15 (03) ◽  
pp. 281-295 ◽  
Author(s):  
E. KORUTCHEVA ◽  
V. DEL PRETE ◽  
J.-P. NADAL
Keyword(s):  
Neural Network ◽  
Perturbation Theory ◽  
Mutual Information ◽  
Nonlinear Term ◽  
Exact Expression ◽  
Neural Systems ◽  
Neuronal Dynamics ◽  
Perturbative Approach ◽  
First Order ◽  
Nonlinear Analogue

We evaluate the mutual information between the input and the output of a two layer network in the case of a noisy and nonlinear analogue channel. In the case where the nonlinearity is small with respect to the variability in the noise, we derive an exact expression for the contribution to the mutual information given by the nonlinear term in first order of perturbation theory. Finally we show how the calculation can be simplified by means of a diagrammatic expansion. Our results suggest that the use of perturbation theories applied to neural systems might give an insight on the contribution of nonlinearities to the information transmission and in general to the neuronal dynamics.

scholarly journals A Geometric Perspective on Information Plane Analysis

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 711
Author(s):  
Mina Basirat ◽  
Bernhard C. Geiger ◽  
Peter M. Roth
Keyword(s):  
Neural Network ◽  
Mutual Information ◽  
Geometric Interpretation ◽  
Neural Network Training ◽  
Neural Network Learning ◽  
Network Learning ◽  
Plane Analysis ◽  
Network Training ◽  
Hidden Layer ◽  
The Impact

Information plane analysis, describing the mutual information between the input and a hidden layer and between a hidden layer and the target over time, has recently been proposed to analyze the training of neural networks. Since the activations of a hidden layer are typically continuous-valued, this mutual information cannot be computed analytically and must thus be estimated, resulting in apparently inconsistent or even contradicting results in the literature. The goal of this paper is to demonstrate how information plane analysis can still be a valuable tool for analyzing neural network training. To this end, we complement the prevailing binning estimator for mutual information with a geometric interpretation. With this geometric interpretation in mind, we evaluate the impact of regularization and interpret phenomena such as underfitting and overfitting. In addition, we investigate neural network learning in the presence of noisy data and noisy labels.

scholarly journals Optimization of Optical Machine Structure by Backpropagation Neural Network Based on Particle Swarm Optimization and Bayesian Regularization Algorithms

Materials ◽  
2021 ◽  
Vol 14 (11) ◽  
pp. 2998
Author(s):  
Xinyong Zhang ◽  
Liwei Sun
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Prediction Model ◽  
Bayesian Regularization ◽  
Backpropagation Neural Network ◽  
Swarm Optimization ◽  
Modal Frequency ◽  
Supporting Structure ◽  
First Order ◽  
Machine Structure

Fit of the highly nonlinear functional relationship between input variables and output response is important and challenging for the optical machine structure optimization design process. The backpropagation neural network method based on particle swarm optimization and Bayesian regularization algorithms (called BMPB) is proposed to solve this problem. A prediction model of the mass and first-order modal frequency of the supporting structure is developed using the supporting structure as an example. The first-order modal frequency is used as the constraint condition to optimize the lightweight design of the supporting structure’s mass. Results show that the prediction model has more than 99% accuracy in predicting the mass and the first-order modal frequency of the supporting structure, and converges quickly in the supporting structure’s mass-optimization process. The supporting structure results demonstrate the advantages of the method proposed in the article in terms of high accuracy and efficiency. The study in this paper provides an effective method for the optimized design of optical machine structures.

scholarly journals Theoretical uncertainties for cosmological first-order phase transitions

2021 ◽  
Vol 2021 (4) ◽  
Author(s):  
Djuna Croon ◽  
Oliver Gould ◽  
Philipp Schicho ◽  
Tuomas V. I. Tenkanen ◽  
Graham White
Keyword(s):  
Phase Transitions ◽  
Standard Model ◽  
Scale Dependence ◽  
Effective Field ◽  
Bubble Nucleation ◽  
Perturbative Approach ◽  
First Order ◽  
The Standard Model ◽  
First Order Phase ◽  
Renormalisation Scale

Abstract We critically examine the magnitude of theoretical uncertainties in perturbative calculations of fist-order phase transitions, using the Standard Model effective field theory as our guide. In the usual daisy-resummed approach, we find large uncertainties due to renormalisation scale dependence, which amount to two to three orders-of-magnitude uncertainty in the peak gravitational wave amplitude, relevant to experiments such as LISA. Alternatively, utilising dimensional reduction in a more sophisticated perturbative approach drastically reduces this scale dependence, pushing it to higher orders. Further, this approach resolves other thorny problems with daisy resummation: it is gauge invariant which is explicitly demonstrated for the Standard Model, and avoids an uncontrolled derivative expansion in the bubble nucleation rate.

scholarly journals An exact expression of positive periodic solution for a first-order singular equation

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Yun Xin ◽  
Xiaoxiao Cui ◽  
Jie Liu
Keyword(s):  
Differential Equation ◽  
Periodic Solution ◽  
Lower Bounds ◽  
Topological Degree ◽  
Order Differential Equation ◽  
Positive Periodic Solution ◽  
Exact Expression ◽  
Upper And Lower Bounds ◽  
Singular Equation ◽  
First Order

Abstract The main purpose of this paper is to obtain an exact expression of the positive periodic solution for a first-order differential equation with attractive and repulsive singularities. Moreover, we prove the existence of at least one positive periodic solution for this equation with an indefinite singularity by applications of topological degree theorem, and give the upper and lower bounds of the positive periodic solution.

scholarly journals Chiral perturbation theory for GR

2020 ◽  
Vol 2020 (9) ◽  
Author(s):  
Kirill Krasnov ◽  
Yuri Shtanov
Keyword(s):  
Perturbation Theory ◽  
Chiral Perturbation ◽  
Perturbative Calculation ◽  
New Formalism ◽  
Yang Mills ◽  
First Order ◽  
Starting Point ◽  
Gauge Fixing ◽  
New Perturbation ◽  
Effective Description

Abstract We describe a new perturbation theory for General Relativity, with the chiral first-order Einstein-Cartan action as the starting point. Our main result is a new gauge-fixing procedure that eliminates the connection-to-connection propagator. All other known first-order formalisms have this propagator non-zero, which significantly increases the combinatorial complexity of any perturbative calculation. In contrast, in the absence of the connection-to-connection propagator, our formalism leads to an effective description in which only the metric (or tetrad) propagates, there are only cubic and quartic vertices, but some vertex legs are special in that they cannot be connected by the propagator. The new formalism is the gravity analog of the well-known and powerful chiral description of Yang-Mills theory.

Neural Network and Perturbation Theory Hybrid Models for Eigenvalue Prediction

1999 ◽  
Vol 132 (1) ◽  
pp. 78-89 ◽  
Author(s):  
Michael G. Lysenko ◽  
Hing-Ip Wong ◽  
G. Ivan Maldonado
Keyword(s):  
Neural Network ◽  
Perturbation Theory ◽  
Hybrid Models

Note on Interference Effects in Two-Step Reaction Processes

1975 ◽  
Vol 53 (23) ◽  
pp. 2590-2592
Author(s):  
J. Cejpek ◽  
J. Dobeš
Keyword(s):  
Perturbation Theory ◽  
Point Of View ◽  
Order Perturbation ◽  
Interference Effects ◽  
Qualitative Explanation ◽  
First Order ◽  
One Step ◽  
Step Transition ◽  
Reaction Processes ◽  
First Order Perturbation

The reaction processes in which a one-step transition is forbidden are analyzed from the point of view of the first order perturbation theory. The interference between two competing two-step reaction paths is found to be always constructive. A qualitative explanation of the experimentally observed reaction intensities is presented.

Sign in / Sign up

Export Citation Format

Share Document