Boltzmann machines with clusters of stochastic binary units

2016 ◽  
Vol 07 (02) ◽  
pp. 1650018
Author(s):  
Da Teng ◽  
Zhang Li ◽  
Guanghong Gong ◽  
Liang Han
Keyword(s):  
Gaussian Distribution ◽  
Learning Algorithm ◽  
Hidden Variables ◽  
Recognition Task ◽  
Boltzmann Machine ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
New Variant ◽  
Deep Boltzmann Machine ◽  
New Learning

The original restricted Boltzmann machines (RBMs) are extended by replacing the binary visible and hidden variables with clusters of binary units, and a new learning algorithm for training deep Boltzmann machine of this new variant is proposed. The sum of binary units of each cluster is approximated by a Gaussian distribution. Experiments demonstrate that the proposed Boltzmann machines can achieve good performance in the MNIST handwritten digital recognition task.

scholarly journals Reinforcement learning using quantum Boltzmann machines

2018 ◽  
Vol 18 (1&2) ◽  
pp. 51-74 ◽  
Author(s):  
Daniel Crawford ◽  
Anna Levit ◽  
Navid Ghadermarzy ◽  
Jaspreet S. Oberoi ◽  
Pooya Ronagh
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Transverse Field ◽  
Boltzmann Machine ◽  
Quantum Annealing ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
Ising Spin ◽  
Learning Tasks ◽  
Deep Boltzmann Machine

We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.

scholarly journals Analyzing joint brand purchases by conditional restricted Boltzmann machines

2021 ◽  
Author(s):  
Harald Hruschka
Keyword(s):  
Hidden Variables ◽  
Restricted Boltzmann Machine ◽  
Boltzmann Machine ◽  
Restricted Boltzmann Machines ◽  
Market Basket ◽  
Boltzmann Machines ◽  
Independent Variables ◽  
Product Categories ◽  
Pseudo Likelihood ◽  
Marketing Variables

AbstractWe introduce the conditional restricted Boltzmann machine as method to analyze brand-level market basket data of individual households. The conditional restricted Boltzmann machine includes marketing variables and household attributes as independent variables. To our knowledge this is the first study comparing the conditional restricted Boltzmann machine to homogeneous and heterogeneous multivariate logit models for brand-level market basket data across several product categories. We explain how to estimate the conditional restricted Boltzmann machine starting from a restricted Boltzmann machine without independent variables. The conditional restricted Boltzmann machine turns out to excel all the other investigated models in terms of log pseudo-likelihood for holdout data. We interpret the selected conditional restricted Boltzmann machine based on coefficients linking purchases to hidden variables, interdependences between brand pairs as well as own and cross effects of marketing variables. The conditional restricted Boltzmann machine indicates pairwise relationships between brands that are more varied than those of the multivariate logit model are. Based on the pairwise interdependences inferred from the restricted Boltzmann machine we determine the competitive structure of brands by means of cluster analysis. Using counterfactual simulations, we investigate what three different models (independent logit, heterogeneous multivariate logit, conditional restricted Boltzmann machine) imply with respect to the retailer’s revenue if each brand is put on display. Finally, we mention possibilities for further research, such as applying the conditional restricted Boltzmann machine to other areas in marketing or retailing.

scholarly journals Dempster–Shafer Fusion Based on a Deep Boltzmann Machine for Blood Pressure Estimation

2018 ◽  
Vol 9 (1) ◽  
pp. 96 ◽  
Author(s):  
Soojeong Lee ◽  
Joon-Hyuk Chang
Keyword(s):  
Blood Pressure ◽  
Middle Layer ◽  
Upper And Lower Bounds ◽  
Boltzmann Machine ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
Estimation Uncertainty ◽  
Deep Boltzmann Machine ◽  
Blood Pressure Estimation ◽  
Pressure Estimate

We propose a technique using Dempster–Shafer fusion based on a deep Boltzmann machine to classify and estimate systolic blood pressure and diastolic blood pressure categories using oscillometric blood pressure measurements. The deep Boltzmann machine is a state-of-the-art technology in which multiple restricted Boltzmann machines are accumulated. Unlike deep belief networks, each unit in the middle layer of the deep Boltzmann machine obtain information up and down to prevent uncertainty at the inference step. Dempster–Shafer fusion can be incorporated to enable combined independent estimation of the observations, and a confidence increase for a given deep Boltzmann machine estimate can be clearly observed. Our work provides an accurate blood pressure estimate, a blood pressure category with upper and lower bounds, and a solution that can reduce estimation uncertainty. This study is one of the first to use deep Boltzmann machine-based Dempster–Shafer fusion to classify and estimate blood pressure.

scholarly journals Boltzmann Machines as Multidimensional Item Response Theory Models

2021 ◽  
Author(s):  
Gunter Maris ◽  
Timo Bechger
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Multidimensional Item Response Theory ◽  
Multidimensional Item Response ◽  
Response Theory ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
New Learning ◽  
Carry Over ◽  
Item Response Theory Models

We show that Boltzmann machines can formally be represented as multidimensional item response theory models. This relationship inspired a new learning principle and new ways to regularize Boltzmann machines to make them more interpretable. The core results carry over to a broader class of models including Gaussian-Bernoulli restricted Boltzmann machines.

scholarly journals Mode-assisted joint training of deep Boltzmann machines

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Haik Manukian ◽  
Massimiliano Di Ventra
Keyword(s):  
Probability Distributions ◽  
Great Success ◽  
Boltzmann Machine ◽  
Data Set ◽  
Boltzmann Machines ◽  
Compact Representations ◽  
Hardware Implementations ◽  
Deep Boltzmann Machine ◽  
Unsupervised Training ◽  
Performance Gains

AbstractThe deep extension of the restricted Boltzmann machine (RBM), known as the deep Boltzmann machine (DBM), is an expressive family of machine learning models which can serve as compact representations of complex probability distributions. However, jointly training DBMs in the unsupervised setting has proven to be a formidable task. A recent technique we have proposed, called mode-assisted training, has shown great success in improving the unsupervised training of RBMs. Here, we show that the performance gains of the mode-assisted training are even more dramatic for DBMs. In fact, DBMs jointly trained with the mode-assisted algorithm can represent the same data set with orders of magnitude lower number of total parameters compared to state-of-the-art training procedures and even with respect to RBMs, provided a fan-in network topology is also introduced. This substantial saving in number of parameters makes this training method very appealing also for hardware implementations.

scholarly journals Convergence Analysis of Contrastive Divergence Algorithm Based on Gradient Method with Errors

2015 ◽  
Vol 2015 ◽  
pp. 1-9 ◽  
Author(s):  
Xuesi Ma ◽  
Xiaojie Wang
Keyword(s):  
Finite Number ◽  
Gibbs Sampling ◽  
Gradient Method ◽  
Convergence Theorem ◽  
Learning Algorithm ◽  
Restricted Boltzmann Machines ◽  
Convergence Conditions ◽  
Boltzmann Machines ◽  
Contrastive Divergence ◽  
Step Number

Contrastive Divergence has become a common way to train Restricted Boltzmann Machines; however, its convergence has not been made clear yet. This paper studies the convergence of Contrastive Divergence algorithm. We relate Contrastive Divergence algorithm to gradient method with errors and derive convergence conditions of Contrastive Divergence algorithm using the convergence theorem of gradient method with errors. We give specific convergence conditions of Contrastive Divergence learning algorithm for Restricted Boltzmann Machines in which both visible units and hidden units can only take a finite number of values. Two new convergence conditions are obtained by specifying the learning rate. Finally, we give specific conditions that the step number of Gibbs sampling must be satisfied in order to guarantee the Contrastive Divergence algorithm convergence.

scholarly journals Restricted Boltzmann Machine representation for the groundstate and excited states of Kitaev Honeycomb model

2021 ◽  
Author(s):  
Mohammadreza Noormandipour ◽  
Youran Sun ◽  
Babak Haghighat
Keyword(s):  
Excited States ◽  
Wave Functions ◽  
Boltzmann Machine ◽  
Restricted Boltzmann Machines ◽  
Conformal Blocks ◽  
Boltzmann Machines ◽  
Exact Ground State ◽  
Small Lattice ◽  
Topological Field ◽  
Machine Representation

Abstract In this work, the capability of restricted Boltzmann machines (RBMs) to find solutions for the Kitaev honeycomb model with periodic boundary conditions is investigated. The measured groundstate (GS) energy of the system is compared and, for small lattice sizes (e.g. 3×3 with 18 spinors), shown to agree with the analytically derived value of the energy up to a deviation of 0.09 %. Moreover, the wave-functions we find have 99.89 % overlap with the exact ground state wave-functions. Furthermore, the possibility of realizing anyons in the RBM is discussed and an algorithm is given to build these anyonic excitations and braid them for possible future applications in quantum computation. Using the correspondence between topological field theories in (2+1)d and 2d CFTs, we propose an identification between our RBM states with the Moore-Read state and conformal blocks of the 2 d Ising model.

scholarly journals Inverse problems for structured datasets using parallel TAP equations and restricted Boltzmann machines

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Aurelien Decelle ◽  
Sungmin Hwang ◽  
Jacopo Rocchi ◽  
Daniele Tantari
Keyword(s):  
Inverse Problems ◽  
Efficient Algorithm ◽  
Mean Field ◽  
Boltzmann Machine ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines ◽  
Teacher Student ◽  
Pseudo Likelihood ◽  
Field Approaches ◽  
Retrieval Phase

AbstractWe propose an efficient algorithm to solve inverse problems in the presence of binary clustered datasets. We consider the paradigmatic Hopfield model in a teacher student scenario, where this situation is found in the retrieval phase. This problem has been widely analyzed through various methods such as mean-field approaches or the pseudo-likelihood optimization. Our approach is based on the estimation of the posterior using the Thouless–Anderson–Palmer (TAP) equations in a parallel updating scheme. Unlike other methods, it allows to retrieve the original patterns of the teacher dataset and thanks to the parallel update it can be applied to large system sizes. We tackle the same problem using a restricted Boltzmann machine (RBM) and discuss analogies and differences between our algorithm and RBM learning.

Sign in / Sign up

Export Citation Format

Share Document