scholarly journals Convergence Analysis of Contrastive Divergence Algorithm Based on Gradient Method with Errors

2015 ◽  
Vol 2015 ◽  
pp. 1-9 ◽  
Author(s):  
Xuesi Ma ◽  
Xiaojie Wang
Keyword(s):  
Finite Number ◽  
Gibbs Sampling ◽  
Gradient Method ◽  
Convergence Theorem ◽  
Learning Algorithm ◽  
Restricted Boltzmann Machines ◽  
Convergence Conditions ◽  
Boltzmann Machines ◽  
Contrastive Divergence ◽  
Step Number

Contrastive Divergence has become a common way to train Restricted Boltzmann Machines; however, its convergence has not been made clear yet. This paper studies the convergence of Contrastive Divergence algorithm. We relate Contrastive Divergence algorithm to gradient method with errors and derive convergence conditions of Contrastive Divergence algorithm using the convergence theorem of gradient method with errors. We give specific convergence conditions of Contrastive Divergence learning algorithm for Restricted Boltzmann Machines in which both visible units and hidden units can only take a finite number of values. Two new convergence conditions are obtained by specifying the learning rate. Finally, we give specific conditions that the step number of Gibbs sampling must be satisfied in order to guarantee the Contrastive Divergence algorithm convergence.

Bounding the Bias of Contrastive Divergence Learning

2011 ◽  
Vol 23 (3) ◽  
pp. 664-673 ◽  
Author(s):  
Asja Fischer ◽  
Christian Igel
Keyword(s):  
Gibbs Sampling ◽  
Upper Bound ◽  
Single Variable ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
Maximum Change ◽  
Log Likelihood ◽  
Contrastive Divergence ◽  
The Absolute

Optimization based on k-step contrastive divergence (CD) has become a common way to train restricted Boltzmann machines (RBMs). The k-step CD is a biased estimator of the log-likelihood gradient relying on Gibbs sampling. We derive a new upper bound for this bias. Its magnitude depends on k, the number of variables in the RBM, and the maximum change in energy that can be produced by changing a single variable. The last reflects the dependence on the absolute values of the RBM parameters. The magnitude of the bias is also affected by the distance in variation between the modeled distribution and the starting distribution of the Gibbs chain.

scholarly journals Reinforcement learning using quantum Boltzmann machines

2018 ◽  
Vol 18 (1&2) ◽  
pp. 51-74 ◽  
Author(s):  
Daniel Crawford ◽  
Anna Levit ◽  
Navid Ghadermarzy ◽  
Jaspreet S. Oberoi ◽  
Pooya Ronagh
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Transverse Field ◽  
Boltzmann Machine ◽  
Quantum Annealing ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
Ising Spin ◽  
Learning Tasks ◽  
Deep Boltzmann Machine

We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.

scholarly journals Online Sequential Extreme Learning Machine: A New Training Scheme for Restricted Boltzmann Machines

2020 ◽  
Author(s):  
BERGHOUT Tarek
Keyword(s):  
Extreme Learning Machine ◽  
Recursive Least Squares ◽  
Training Algorithms ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
Training Scheme ◽  
Learning Path ◽  
Contrastive Divergence ◽  
Learning Machine ◽  
Time Accuracy

Abstract: The main contribution of this paper is to introduce a new iterative training algorithm for restricted Boltzmann machines. The proposed learning path is inspired from online sequential extreme learning machine one of extreme learning machine variants which deals with time accumulated sequences of data with fixed or varied sizes. Recursive least squares rules are integrated for weights adaptation to avoid learning rate tuning and local minimum issues. The proposed approach is compared to one of the well known training algorithms for Boltzmann machines named “contrastive divergence”, in term of time, accuracy and algorithmic complexity under the same conditions. Results strongly encourage the new given rules during data reconstruction.

Quickly Generating Representative Samples from an RBM-Derived Process

2011 ◽  
Vol 23 (8) ◽  
pp. 2058-2073 ◽  
Author(s):  
Olivier Breuleux ◽  
Yoshua Bengio ◽  
Pascal Vincent
Keyword(s):  
Markov Chain ◽  
Lower Bound ◽  
Gibbs Sampling ◽  
Learning Algorithm ◽  
Training Data ◽  
Model Parameters ◽  
Sampling Process ◽  
Log Likelihood ◽  
Contrastive Divergence ◽  
Representative Samples

Two recently proposed learning algorithms, herding and fast persistent contrastive divergence (FPCD), share the following interesting characteristic: they exploit changes in the model parameters while sampling in order to escape modes and mix better during the sampling process that is part of the learning algorithm. We justify such approaches as ways to escape modes while keeping approximately the same asymptotic distribution of the Markov chain. In that spirit, we extend FPCD using an idea borrowed from Herding in order to obtain a pure sampling algorithm, which we call the rates-FPCD sampler. Interestingly, this sampler can improve the model as we collect more samples, since it optimizes a lower bound on the log likelihood of the training data. We provide empirical evidence that this new algorithm displays substantially better and more robust mixing than Gibbs sampling.

Sign in / Sign up

Export Citation Format

Share Document