Analyzing joint brand purchases by conditional restricted Boltzmann machines

Boltzmann Machine ◽

Market Basket ◽

Boltzmann Machines ◽

Independent Variables ◽

Product Categories ◽

Pseudo Likelihood ◽

Marketing Variables

AbstractWe introduce the conditional restricted Boltzmann machine as method to analyze brand-level market basket data of individual households. The conditional restricted Boltzmann machine includes marketing variables and household attributes as independent variables. To our knowledge this is the first study comparing the conditional restricted Boltzmann machine to homogeneous and heterogeneous multivariate logit models for brand-level market basket data across several product categories. We explain how to estimate the conditional restricted Boltzmann machine starting from a restricted Boltzmann machine without independent variables. The conditional restricted Boltzmann machine turns out to excel all the other investigated models in terms of log pseudo-likelihood for holdout data. We interpret the selected conditional restricted Boltzmann machine based on coefficients linking purchases to hidden variables, interdependences between brand pairs as well as own and cross effects of marketing variables. The conditional restricted Boltzmann machine indicates pairwise relationships between brands that are more varied than those of the multivariate logit model are. Based on the pairwise interdependences inferred from the restricted Boltzmann machine we determine the competitive structure of brands by means of cluster analysis. Using counterfactual simulations, we investigate what three different models (independent logit, heterogeneous multivariate logit, conditional restricted Boltzmann machine) imply with respect to the retailer’s revenue if each brand is put on display. Finally, we mention possibilities for further research, such as applying the conditional restricted Boltzmann machine to other areas in marketing or retailing.

Interdependences of Products in Market Baskets: Comparing the Conditional Restricted Boltzmann Machine to the Multivariate Logit Model

Review of Marketing Science ◽

10.1515/roms-2020-0074 ◽

2020 ◽

Vol 0 (0) ◽

Author(s):

Harald Hruschka

Keyword(s):

Hidden Variables ◽

Future Research ◽

Data Sets ◽

Boltzmann Machine ◽

Consumer Durables ◽

Product Categories ◽

Simulation Based ◽

Out Of Sample ◽

Average Probability

AbstractWe analyze market baskets of individual households in two consumer durables categories (music, computer related products) by the multivariate logit (MVL) model, its finite mixture extension (FM-MVL) and the conditional restricted Boltzmann machine (CRBM). The CRBM attains a vastly better out-of-sample performance than MVL and FM-MVL models. Based on simulation-based likelihood ratio tests we prefer the CRBM to the FM-MVL model. To interpret hidden variables of conditional Boltzmann machines we look at their average probability differences between purchase and non-purchases of any sub-category across all baskets. To measure interdependences we compute cross effects between sub-categories for the best performing FM-MVL model and CRBM. In both product categories the CRBM indicates more or higher positive cross effects than the FM-MVL model. Finally, we suggest appropriate future research based on larger and more detailed data sets.

Inverse problems for structured datasets using parallel TAP equations and restricted Boltzmann machines

Scientific Reports ◽

10.1038/s41598-021-99353-2 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Aurelien Decelle ◽

Sungmin Hwang ◽

Jacopo Rocchi ◽

Daniele Tantari

Keyword(s):

Inverse Problems ◽

Efficient Algorithm ◽

Mean Field ◽

Boltzmann Machine ◽

Boltzmann Machines ◽

Teacher Student ◽

Pseudo Likelihood ◽

Field Approaches ◽

Retrieval Phase

AbstractWe propose an efficient algorithm to solve inverse problems in the presence of binary clustered datasets. We consider the paradigmatic Hopfield model in a teacher student scenario, where this situation is found in the retrieval phase. This problem has been widely analyzed through various methods such as mean-field approaches or the pseudo-likelihood optimization. Our approach is based on the estimation of the posterior using the Thouless–Anderson–Palmer (TAP) equations in a parallel updating scheme. Unlike other methods, it allows to retrieve the original patterns of the teacher dataset and thanks to the parallel update it can be applied to large system sizes. We tackle the same problem using a restricted Boltzmann machine (RBM) and discuss analogies and differences between our algorithm and RBM learning.

Boltzmann machines with clusters of stochastic binary units

International Journal of Modeling Simulation and Scientific Computing ◽

10.1142/s1793962316500185 ◽

2016 ◽

Vol 07 (02) ◽

pp. 1650018

Author(s):

Da Teng ◽

Zhang Li ◽

Guanghong Gong ◽

Liang Han

Keyword(s):

Gaussian Distribution ◽

Learning Algorithm ◽

Hidden Variables ◽

Recognition Task ◽

Boltzmann Machine ◽

Boltzmann Machines ◽

New Variant ◽

Deep Boltzmann Machine ◽

New Learning

The original restricted Boltzmann machines (RBMs) are extended by replacing the binary visible and hidden variables with clusters of binary units, and a new learning algorithm for training deep Boltzmann machine of this new variant is proposed. The sum of binary units of each cluster is approximated by a Gaussian distribution. Experiments demonstrate that the proposed Boltzmann machines can achieve good performance in the MNIST handwritten digital recognition task.

Proceedings of the Institution of Mechanical Engineers Part C Journal of Mechanical Engineering Science ◽

Single and combined fault diagnosis of reciprocating compressor valves using a hybrid deep belief network

10.1177/0954406217740929 ◽

2017 ◽

Vol 232 (20) ◽

pp. 3767-3780 ◽

Cited By ~ 5

Author(s):

Van Tung Tran ◽

Faisal AlThobiani ◽

Tiedo Tinga ◽

Andrew Ball ◽

Gang Niu

Keyword(s):

Receptive Fields ◽

Deep Belief Network ◽

Fault Classification ◽

Fuzzy Artmap ◽

Training Procedure ◽

Reciprocating Compressor ◽

Boltzmann Machine ◽

Belief Network

In this paper, a hybrid deep belief network is proposed to diagnose single and combined faults of suction and discharge valves in a reciprocating compressor. This hybrid integrates the deep belief network structured by multiple stacked restricted Boltzmann machines for pre-training and simplified fuzzy ARTMAP (SFAM) for fault classification. In the pre-training procedure, an algorithm for selecting local receptive fields is used to group the most similar features into the receptive fields of which top values are the units of each layer, and then restricted Boltzmann machine is applied to these units to construct a network. Unsupervised learning is also carried out for each restricted Boltzmann machine layer in this procedure to compute the network weights and biases. Finally, the network output is fed into SFAM to perform fault classification. In order to diagnose the valve faults, three signal types of vibration, pressure, and current are acquired from a two-stage reciprocating air compressor under different valve conditions such as suction leakages, discharge leakages, spring deterioration, and their combination. These signals are subsequently processed so that the useful fault information from the signals can be revealed; next, statistical features in the time and frequency domains are extracted from the signals and used as the inputs for hybrid deep belief network. Performance of hybrid deep belief network in fault classification is compared with that of the original deep belief network and the deep belief network combined with generalized discriminant analysis, where softmax regression is used as a classifier for the latter two models. The results indicate that hybrid deep belief network is more capable of improving the diagnosis accuracy and is feasible in industrial applications.

Reinforcement learning using quantum Boltzmann machines

Quantum Information and Computation ◽

10.26421/qic18.1-2-3 ◽

2018 ◽

Vol 18 (1&2) ◽

pp. 51-74 ◽

Cited By ~ 2

Author(s):

Daniel Crawford ◽

Anna Levit ◽

Navid Ghadermarzy ◽

Jaspreet S. Oberoi ◽

Pooya Ronagh

Keyword(s):

Reinforcement Learning ◽

Learning Algorithm ◽

Transverse Field ◽

Boltzmann Machine ◽

Quantum Annealing ◽

Boltzmann Machines ◽

Ising Spin ◽

Learning Tasks ◽

Deep Boltzmann Machine

We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.

Restricted Boltzmann Machine with Multivalued Hidden Variables

The Review of Socionetwork Strategies ◽

10.1007/s12626-019-00042-4 ◽

2019 ◽

Vol 13 (2) ◽

pp. 253-266 ◽

Cited By ~ 1

Author(s):

Yuuki Yokoyama ◽

Tomu Katsumata ◽

Muneki Yasuda

Keyword(s):

Hidden Variables ◽

Boltzmann Machine

Entropy, Free Energy, and Work of Restricted Boltzmann Machines

Entropy ◽

10.3390/e22050538 ◽

2020 ◽

Vol 22 (5) ◽

pp. 538

Author(s):

Sangchul Oh ◽

Abdelkader Baggag ◽

Hyunchul Nha

Keyword(s):

Free Energy ◽

Energy Function ◽

Statistical Physics ◽

Boltzmann Distribution ◽

Training Data ◽

Thermodynamic Quantities ◽

Boltzmann Machine ◽

The Difference

A restricted Boltzmann machine is a generative probabilistic graphic network. A probability of finding the network in a certain configuration is given by the Boltzmann distribution. Given training data, its learning is done by optimizing the parameters of the energy function of the network. In this paper, we analyze the training process of the restricted Boltzmann machine in the context of statistical physics. As an illustration, for small size bar-and-stripe patterns, we calculate thermodynamic quantities such as entropy, free energy, and internal energy as a function of the training epoch. We demonstrate the growth of the correlation between the visible and hidden layers via the subadditivity of entropies as the training proceeds. Using the Monte-Carlo simulation of trajectories of the visible and hidden vectors in the configuration space, we also calculate the distribution of the work done on the restricted Boltzmann machine by switching the parameters of the energy function. We discuss the Jarzynski equality which connects the path average of the exponential function of the work and the difference in free energies before and after training.

Dempster–Shafer Fusion Based on a Deep Boltzmann Machine for Blood Pressure Estimation

Applied Sciences ◽

10.3390/app9010096 ◽

2018 ◽

Vol 9 (1) ◽

pp. 96 ◽

Cited By ~ 3

Author(s):

Soojeong Lee ◽

Joon-Hyuk Chang

Keyword(s):

Blood Pressure ◽

Middle Layer ◽

Upper And Lower Bounds ◽

Boltzmann Machine ◽

Boltzmann Machines ◽

Estimation Uncertainty ◽

Deep Boltzmann Machine ◽

Blood Pressure Estimation ◽

Pressure Estimate

We propose a technique using Dempster–Shafer fusion based on a deep Boltzmann machine to classify and estimate systolic blood pressure and diastolic blood pressure categories using oscillometric blood pressure measurements. The deep Boltzmann machine is a state-of-the-art technology in which multiple restricted Boltzmann machines are accumulated. Unlike deep belief networks, each unit in the middle layer of the deep Boltzmann machine obtain information up and down to prevent uncertainty at the inference step. Dempster–Shafer fusion can be incorporated to enable combined independent estimation of the observations, and a confidence increase for a given deep Boltzmann machine estimate can be clearly observed. Our work provides an accurate blood pressure estimate, a blood pressure category with upper and lower bounds, and a solution that can reduce estimation uncertainty. This study is one of the first to use deep Boltzmann machine-based Dempster–Shafer fusion to classify and estimate blood pressure.

A complete restricted Boltzmann machine on an adiabatic quantum computer

International Journal of Quantum Information ◽

10.1142/s0219749921410033 ◽

2021 ◽

pp. 2141003

Author(s):

Lorenzo Rocutto ◽

Enrico Prati

Keyword(s):

Complete Graph ◽

Quantum Computer ◽

Computational Time ◽

Quantum Computers ◽

Boltzmann Machine ◽

Computational Power ◽

Trade Off ◽

Boltzmann Machines ◽

The Neural Network

Boltzmann Machines constitute a paramount class of neural networks for unsupervised learning and recommendation systems. Their bipartite version, called Restricted Boltzmann Machine (RBM), is the most developed because of its satisfactory trade-off between computability on classical computers and computational power. Though the diffusion of RBMs is quite limited as their training remains hard. Recently, a renewed interest has emerged as Adiabatic Quantum Computers (AQCs), which suggest a potential increase of the training speed with respect to conventional hardware. Due to the limited number of connections among the qubits forming the graph of existing hardware, associating one qubit per node of the neural network implies an incomplete graph. Thanks to embedding techniques, we developed a complete graph connecting nodes constituted by virtual qubits. The complete graph outperforms previous implementations based on incomplete graphs. Despite the fact that the learning rate per epoch is still slower with respect to a classical machine, the advantage is expected by the increase of number of nodes which impacts on the classical computational time but not on the quantum hardware based computation.

Restricted Boltzmann Machine representation for the groundstate and excited states of Kitaev Honeycomb model

Machine Learning: Science and Technology ◽

10.1088/2632-2153/ac3ddf ◽

2021 ◽

Author(s):

Mohammadreza Noormandipour ◽

Youran Sun ◽

Babak Haghighat

Keyword(s):

Excited States ◽

Wave Functions ◽

Boltzmann Machine ◽

Conformal Blocks ◽

Boltzmann Machines ◽

Exact Ground State ◽

Small Lattice ◽

Topological Field ◽

Machine Representation

Abstract In this work, the capability of restricted Boltzmann machines (RBMs) to find solutions for the Kitaev honeycomb model with periodic boundary conditions is investigated. The measured groundstate (GS) energy of the system is compared and, for small lattice sizes (e.g. 3×3 with 18 spinors), shown to agree with the analytically derived value of the energy up to a deviation of 0.09 %. Moreover, the wave-functions we find have 99.89 % overlap with the exact ground state wave-functions. Furthermore, the possibility of realizing anyons in the RBM is discussed and an algorithm is given to build these anyonic excitations and braid them for possible future applications in quantum computation. Using the correspondence between topological field theories in (2+1)d and 2d CFTs, we propose an identification between our RBM states with the Moore-Read state and conformal blocks of the 2 d Ising model.