Learning from demonstration using products of experts: Applications to manipulation and task prioritization

Probability distributions are key components of many learning from demonstration (LfD) approaches, with the spaces chosen to represent tasks playing a central role. Although the robot configuration is defined by its joint angles, end-effector poses are often best explained within several task spaces. In many approaches, distributions within relevant task spaces are learned independently and only combined at the control level. This simplification implies several problems that are addressed in this work. We show that the fusion of models in different task spaces can be expressed as products of experts (PoE), where the probabilities of the models are multiplied and renormalized so that it becomes a proper distribution of joint angles. Multiple experiments are presented to show that learning the different models jointly in the PoE framework significantly improves the quality of the final model. The proposed approach particularly stands out when the robot has to learn hierarchical objectives that arise when a task requires the prioritization of several sub-tasks (e.g. in a humanoid robot, keeping balance has a higher priority than reaching for an object). Since training the model jointly usually relies on contrastive divergence, which requires costly approximations that can affect performance, we propose an alternative strategy using variational inference and mixture model approximations. In particular, we show that the proposed approach can be extended to PoE with a nullspace structure (PoENS), where the model is able to recover secondary tasks that are masked by the resolution of tasks of higher-importance.

A Generalization of Spatial Monte Carlo Integration

Spatial Monte Carlo integration (SMCI) is an extension of standard Monte Carlo integration and can approximate expectations on Markov random fields with high accuracy. SMCI was applied to pairwise Boltzmann machine (PBM) learning, achieving superior results over those of some existing methods. The approximation level of SMCI can be altered, and it was proved that a higher-order approximation of SMCI is statistically more accurate than a lower-order approximation. However, SMCI as proposed in previous studies suffers from a limitation that prevents the application of a higher-order method to dense systems. This study makes two contributions. First, a generalization of SMCI (called generalized SMCI (GSMCI)) is proposed, which allows a relaxation of the above-mentioned limitation; moreover, a statistical accuracy bound of GSMCI is proved. Second, a new PBM learning method based on SMCI is proposed, which is obtained by combining SMCI and persistent contrastive divergence. The proposed learning method significantly improves learning accuracy.

Online Sequential Extreme Learning Machine: A New Training Scheme for Restricted Boltzmann Machines

The main contribution of this paper is to introduce a new iterative training algorithm for restricted Boltzmann machines. The proposed learning path is inspired from online sequential extreme learning machine one of extreme learning machine variants which deals with time accumulated sequences of data with fixed or varied sizes. Recursive least squares rules are integrated for weights adaptation to avoid learning rate tuning and local minimum issues. The proposed approach is compared to one of the well known training algorithms for Boltzmann machines named “contrastive divergence”, in term of time, accuracy and algorithmic complexity under the same conditions. Results strongly encourage the new given rules during data reconstruction.

Semisupervised Deep Stacking Network with Adaptive Learning Rate Strategy for Motor Imagery EEG Recognition

Practical motor imagery electroencephalogram (EEG) data-based applications are limited by the waste of unlabeled samples in supervised learning and excessive time consumption in the pretraining period. A semisupervised deep stacking network with an adaptive learning rate strategy (SADSN) is proposed to solve the sample loss caused by supervised learning of EEG data and the extraction of manual features. The SADSN adopts the idea of an adaptive learning rate into a contrastive divergence (CD) algorithm to accelerate its convergence. Prior knowledge is introduced into the intermediary layer of the deep stacking network, and a restricted Boltzmann machine is trained by a semisupervised method in which the adjusting scope of the coefficient in learning rate is determined by performance analysis. Several EEG data sets are carried out to evaluate the performance of the proposed method. The results show that the recognition accuracy of SADSN is advanced with a more significant convergence rate and successfully classifies motor imagery.

Automatic Deep Feature Learning via Patch-Based Deep Belief Network for Vertebrae Segmentation in CT Images

Precise automatic vertebra segmentation in computed tomography (CT) images is important for the quantitative analysis of vertebrae-related diseases but remains a challenging task due to high variation in spinal anatomy among patients. In this paper, we propose a deep learning approach for automatic CT vertebra segmentation named patch-based deep belief networks (PaDBNs). Our proposed PaDBN model automatically selects the features from image patches and then measures the differences between classes and investigates performance. The region of interest (ROI) is obtained from CT images. Unsupervised feature reduction contrastive divergence algorithm is applied for weight initialization, and the weights are optimized by layers in a supervised fine-tuning procedure. The discriminative learning features obtained from the steps above are used as input of a classifier to obtain the likelihood of the vertebrae. Experimental results demonstrate that the proposed PaDBN model can considerably reduce computational cost and produce an excellent performance in vertebra segmentation in terms of accuracy compared with state-of-the-art methods.

Feature-Based Decipherment for Machine Translation

Orthographic similarities across languages provide a strong signal for unsupervised probabilistic transduction (decipherment) for closely related language pairs. The existing decipherment models, however, are not well suited for exploiting these orthographic similarities. We propose a log-linear model with latent variables that incorporates orthographic similarity features. Maximum likelihood training is computationally expensive for the proposed log-linear model. To address this challenge, we perform approximate inference via Markov chain Monte Carlo sampling and contrastive divergence. Our results show that the proposed log-linear model with contrastive divergence outperforms the existing generative decipherment models by exploiting the orthographic features. The model both scales to large vocabularies and preserves accuracy in low- and no-resource contexts.

Synthetic protein alignments by CCMgen quantify noise in residue-residue contact prediction

Compensatory mutations between protein residues that are in physical contact with each other can manifest themselves as statistical couplings between the corresponding columns in a multiple sequence alignment (MSA) of the protein family. Conversely, high coupling coefficients predict residues contacts. Methods for de-novo protein structure prediction based on this approach are becoming increasingly reliable. Their main limitation is the strong systematic and statistical noise in the estimation of coupling coefficients, which has so far limited their application to very large protein families. While most research has focused on boosting contact prediction quality by adding external information, little progress has been made to improve the statistical procedure at the core. In that regard, our lack of understanding of the sources of noise poses a major obstacle. We have developed CCMgen, the first method for simulating protein evolution by providing full control over the generation of realistic synthetic MSAs with pairwise statistical couplings between residue positions. This procedure requires an exact statistical model that reliably reproduces observed alignment statistics. With CCMpredPy we also provide an implementation of persistent contrastive divergence (PCD), a precise inference technique that enables us to learn the required high-quality statistical models. We demonstrate how CCMgen can facilitate the development and testing of contact prediction methods by analysing the systematic noise contributions from phylogeny and entropy. For that purpose we propose a simple entropy correction (EC) strategy which disentangles the correction for both sources of noise. We find that entropy contributes typically roughly twice as much noise as phylogeny.