A New Learning Algorithm for Mean Field Boltzmann Machines

2002 ◽  
pp. 351-357 ◽  
Author(s):  
Max Welling ◽  
Geoffrey E. Hinton
Keyword(s):  
Learning Algorithm ◽  
Mean Field ◽  
Boltzmann Machines ◽  
New Learning

Learning in Boltzmann Trees

1994 ◽  
Vol 6 (6) ◽  
pp. 1174-1184 ◽  
Author(s):  
Lawrence Saul ◽  
Michael I. Jordan
Keyword(s):  
Supervised Learning ◽  
Gradient Descent ◽  
Learning Algorithm ◽  
Mean Field ◽  
Large Family ◽  
Weight Space ◽  
Hidden Symmetries ◽  
Boltzmann Machines ◽  
Field Annealing

We introduce a large family of Boltzmann machines that can be trained by standard gradient descent. The networks can have one or more layers of hidden units, with tree-like connectivity. We show how to implement the supervised learning algorithm for these Boltzmann machines exactly, without resort to simulated or mean-field annealing. The stochastic averages that yield the gradients in weight space are computed by the technique of decimation. We present results on the problems of N-bit parity and the detection of hidden symmetries.

scholarly journals Efficient Learning in Boltzmann Machines Using Linear Response Theory

1998 ◽  
Vol 10 (5) ◽  
pp. 1137-1156 ◽  
Author(s):  
H. J. Kappen ◽  
F. B. Rodríguez
Keyword(s):  
Computational Complexity ◽  
Learning Process ◽  
Linear Response ◽  
Learning Algorithm ◽  
Mean Field Theory ◽  
Mean Field ◽  
Exact Algorithm ◽  
Linear Response Theory ◽  
Boltzmann Machines ◽  
Efficient Learning

The learning process in Boltzmann machines is computationally very expensive. The computational complexity of the exact algorithm is exponential in the number of neurons. We present a new approximate learning algorithm for Boltzmann machines, based on mean-field theory and the linear response theorem. The computational complexity of the algorithm is cubic in the number of neurons. In the absence of hidden units, we show how the weights can be directly computed from the fixed-point equation of the learning rules. Thus, in this case we do not need to use a gradient descent procedure for the learning process. We show that the solutions of this method are close to the optimal solutions and give a significant improvement when correlations play a significant role. Finally, we apply the method to a pattern completion task and show good performance for networks up to 100 neurons.

Approximate Learning Algorithm in Boltzmann Machines

2009 ◽  
Vol 21 (11) ◽  
pp. 3130-3178 ◽  
Author(s):  
Muneki Yasuda ◽  
Kazuyuki Tanaka
Keyword(s):  
Markov Random Fields ◽  
Belief Propagation ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Mean Field ◽  
Spin Model ◽  
Field Methods ◽  
Approximate Methods ◽  
Boltzmann Machines ◽  
Ising Spin

Boltzmann machines can be regarded as Markov random fields. For binary cases, they are equivalent to the Ising spin model in statistical mechanics. Learning systems in Boltzmann machines are one of the NP-hard problems. Thus, in general we have to use approximate methods to construct practical learning algorithms in this context. In this letter, we propose new and practical learning algorithms for Boltzmann machines by using the belief propagation algorithm and the linear response approximation, which are often referred as advanced mean field methods. Finally, we show the validity of our algorithm using numerical experiments.

Boltzmann machines with clusters of stochastic binary units

2016 ◽  
Vol 07 (02) ◽  
pp. 1650018
Author(s):  
Da Teng ◽  
Zhang Li ◽  
Guanghong Gong ◽  
Liang Han
Keyword(s):  
Gaussian Distribution ◽  
Learning Algorithm ◽  
Hidden Variables ◽  
Recognition Task ◽  
Boltzmann Machine ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
New Variant ◽  
Deep Boltzmann Machine ◽  
New Learning

The original restricted Boltzmann machines (RBMs) are extended by replacing the binary visible and hidden variables with clusters of binary units, and a new learning algorithm for training deep Boltzmann machine of this new variant is proposed. The sum of binary units of each cluster is approximated by a Gaussian distribution. Experiments demonstrate that the proposed Boltzmann machines can achieve good performance in the MNIST handwritten digital recognition task.

A New Learning Algorithm for Optimal Stopping

2008 ◽  
Vol 19 (1) ◽  
pp. 91-113 ◽  
Author(s):  
Vivek S. Borkar ◽  
Jervis Pinto ◽  
Tarun Prabhu
Keyword(s):  
Optimal Stopping ◽  
Learning Algorithm ◽  
New Learning

A new learning algorithm with logarithmic performance index for complex-valued neural networks

2009 ◽  
Vol 72 (16-18) ◽  
pp. 3771-3781 ◽  
Author(s):  
R. Savitha ◽  
S. Suresh ◽  
N. Sundararajan ◽  
P. Saratchandran
Keyword(s):  
Neural Networks ◽  
Performance Index ◽  
Learning Algorithm ◽  
New Learning ◽  
Complex Valued

Kernel-Based Topographic Map Formation by Local Density Modeling

2002 ◽  
Vol 14 (7) ◽  
pp. 1561-1573 ◽  
Author(s):  
Marc M. Van Hulle
Keyword(s):  
Learning Algorithm ◽  
Local Density ◽  
Gaussian Mixture ◽  
Density Model ◽  
Topographic Map ◽  
Mixture Density ◽  
Gaussian Kernels ◽  
New Learning ◽  
Local Input ◽  
Density Modeling

We introduce a new learning algorithm for kernel-based topographic map formation. The algorithm generates a gaussian mixture density model by individually adapting the gaussian kernels' centers and radii to the assumed gaussian local input densities.

A new learning algorithm for a forecasting neuro-fuzzy network

2003 ◽  
Vol 10 (4) ◽  
pp. 399-409 ◽  
Author(s):  
Peter Otto ◽  
Yevgeniy Bodyanskiy ◽  
Vitaliy Kolodyazhniy
Keyword(s):  
Learning Algorithm ◽  
Neuro Fuzzy ◽  
New Learning ◽  
Fuzzy Network
Sign in / Sign up

Export Citation Format

Share Document