A transport equation approach for deep neural networks with quenched random weights

2021 ◽  
Author(s):  
Elena Agliari ◽  
Linda Albanese ◽  
Francesco Alemanno ◽  
Alberto Fachechi
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Symmetric Case ◽  
Hopfield Network ◽  
Equation Approach ◽  
Replica Symmetry ◽  
Replica Symmetry Breaking ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
Random Weights

Abstract We consider a multi-layer Sherrington-Kirkpatrick spin-glass as a model for deep restricted Boltzmann machines with quenched random weights and solve for its free energy in the thermodynamic limit by means of Guerra's interpolating techniques under the RS and 1RSB ansatz. In particular, we recover the expression already known for the replica-symmetric case. Further, we drop the restriction constraint by introducing intra-layer connections among spins and we show that the resulting system can be mapped into a modular Hopfield network, which is also addressed via the same techniques up to the first step of replica symmetry breaking.

MEMORY RETRIEVAL IN OPTIMAL SUBSPACES

1992 ◽  
Vol 03 (supp01) ◽  
pp. 71-77 ◽  
Author(s):  
G. Boffetta ◽  
R. Monasson ◽  
R. Zecchina
Keyword(s):  
Neural Networks ◽  
Memory Retrieval ◽  
Learning Rule ◽  
Dynamical Process ◽  
Replica Symmetry ◽  
Replica Symmetry Breaking ◽  
Analytical Estimate ◽  
Dynamical Scheme ◽  
Hebb Learning ◽  
Selection Of

A simple dynamical scheme for Attractor Neural Networks with non-monotonic three state effective neurons is discussed. For the unsupervised Hebb learning rule, we give some basic numerical results which are interpreted in terms of a combinatorial task realized by the dynamical process (dynamical selection of optimal subspaces). An analytical estimate of optimal performance is given by resorting to two different simplified versions of the model. We show that replica symmetry breaking is required since the replica symmetric solutions are unstable.

REPLICA SYMMETRY BREAKING IN NEURAL NETWORKS WITH NON-MONOTONE ACTIVATION FUNCTION

1996 ◽  
Vol 07 (01) ◽  
pp. 19-32 ◽  
Author(s):  
A. LAMURA ◽  
C. MARANGI ◽  
G. NARDULLI
Keyword(s):  
Neural Networks ◽  
Symmetry Breaking ◽  
Fixed Points ◽  
Storage Capacity ◽  
Activation Function ◽  
Replica Symmetry ◽  
Replica Symmetry Breaking ◽  
Domains Of Attraction ◽  
One Step ◽  
The Stability

In this paper we analyze replica symmetry breaking in attractor neural networks with non-monotone activation function. We study the non-monotone version of the Edinburgh model, which allows the control of the domains of attraction by the stability parameter K, and we compute, at one step of symmetry breaking, storage capacity and, for the strongly dilute model, the domains of attraction of the stable fixed points.

Annotation of tandem mass spectrometry data using stochastic neural networks in shotgun proteomics

2020 ◽  
Vol 36 (12) ◽  
pp. 3781-3787
Author(s):  
Pavel Sulimov ◽  
Anastasia Voronkova ◽  
Attila Kertész-Farkas
Keyword(s):  
Neural Networks ◽  
Shotgun Proteomics ◽  
Mass Spectrometry Data ◽  
Stochastic Neural Networks ◽  
Restricted Boltzmann Machines ◽  
Score Functions ◽  
Boltzmann Machines ◽  
Discrimination Ability ◽  
False Discovery ◽  
Tandem Mass Spectrometry Data

Abstract Motivation The discrimination ability of score functions to separate correct from incorrect peptide-spectrum-matches in database-searching-based spectrum identification is hindered by many superfluous peaks belonging to unexpected fragmentation ions or by the lacking peaks of anticipated fragmentation ions. Results Here, we present a new method, called BoltzMatch, to learn score functions using a particular stochastic neural networks, called restricted Boltzmann machines, in order to enhance their discrimination ability. BoltzMatch learns chemically explainable patterns among peak pairs in the spectrum data, and it can augment peaks depending on their semantic context or even reconstruct lacking peaks of expected ions during its internal scoring mechanism. As a result, BoltzMatch achieved 50% and 33% more annotations on high- and low-resolution MS2 data than XCorr at a 0.1% false discovery rate in our benchmark; conversely, XCorr yielded the same number of spectrum annotations as BoltzMatch, albeit with 4–6 times more errors. In addition, BoltzMatch alone does yield 14% more annotations than Prosit (which runs with Percolator), and BoltzMatch with Percolator yields 32% more annotations than Prosit at 0.1% FDR level in our benchmark. Availability and implementation BoltzMatch is freely available at: https://github.com/kfattila/BoltzMatch. Contact [email protected] Supporting information Supplementary data are available at Bioinformatics online.

scholarly journals Replica symmetry breaking in neural networks: a few steps toward rigorous results

2020 ◽  
Vol 53 (41) ◽  
pp. 415005 ◽  
Author(s):  
Elena Agliari ◽  
Linda Albanese ◽  
Adriano Barra ◽  
Gabriele Ottaviani
Keyword(s):  
Neural Networks ◽  
Symmetry Breaking ◽  
Replica Symmetry ◽  
Replica Symmetry Breaking ◽  
Rigorous Results

scholarly journals Classifying a type of brain disorder in children: an effective fMRI based deep attempt

2021 ◽  
Vol 22 (1) ◽  
pp. 260
Author(s):  
Abeer M Mahmoud ◽  
Hanen Karamti
Keyword(s):  
Neural Networks ◽  
Children With Autism ◽  
Learning Approaches ◽  
Restricted Boltzmann Machines ◽  
Brain Disorder ◽  
Boltzmann Machines ◽  
Learning Framework ◽  
Straight Line ◽  
Healthy Control ◽  
The Brain

<span>Recent advanced intelligent learning approaches that are based on using neural networks in medical diagnosing increased researcher expectations. In fact, the literature proved a straight-line relation of the exact needs and the achieved results. Accordingly, it encouraged promising directions of applying these approaches toward saving time and efforts. This paper proposes a novel hybrid deep learning framework that is based on the restricted boltzmann machines (RBM) and the contractive autoencoder (CA) to classify the brain disorder and healthy control cases in children less than 12 years. The RBM focuses on obtaining the discriminative set of high guided features in the classification process, while the CA provides the regularization and the robustness of features for optimal objectives. The proposed framework diagnosed children with autism recording accuracy of 91, 14% and proved enhancement compared to literature.</span>

Sign in / Sign up

Export Citation Format

Share Document