A Multiple Cause Mixture Model for Unsupervised Learning

1995 ◽  
Vol 7 (1) ◽  
pp. 51-71 ◽  
Author(s):  
Eric Saund
Keyword(s):  
Unsupervised Learning ◽  
Mixture Model ◽  
Binary Data ◽  
Gradient Descent ◽  
Clustering Algorithm ◽  
Learning Algorithm ◽  
Causal Structure ◽  
Data Sets ◽  
Weighted Sum ◽  
Crucial Issue

This paper presents a formulation for unsupervised learning of clusters reflecting multiple causal structure in binary data. Unlike the “hard” k-means clustering algorithm and the “soft” mixture model, each of which assumes that a single hidden event generates each data point, a multiple cause model accounts for observed data by combining assertions from many hidden causes, each of which can pertain to varying degree to any subset of the observable dimensions. We employ an objective function and iterative gradient descent learning algorithm resembling the conventional mixture model. A crucial issue is the mixing function for combining beliefs from different cluster centers in order to generate data predictions whose errors are minimized both during recognition and learning. The mixing function constitutes a prior assumption about underlying structural regularities of the data domain; we demonstrate a weakness inherent to the popular weighted sum followed by sigmoid squashing, and offer alternative forms of the nonlinearity for two types of data domain. Results are presented demonstrating the algorithm's ability successfully to discover coherent multiple causal representations in several experimental data sets.

A new stochastic gradient descent possibilistic clustering algorithm

2021 ◽  
pp. 1-18
Author(s):  
Angeliki Koutsimpela ◽  
Konstantinos D. Koutroumbas
Keyword(s):  
Cost Function ◽  
Gradient Descent ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Real Data ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Data Sets ◽  
Convergence Results ◽  
Possibilistic Clustering

Several well known clustering algorithms have their own online counterparts, in order to deal effectively with the big data issue, as well as with the case where the data become available in a streaming fashion. However, very few of them follow the stochastic gradient descent philosophy, despite the fact that the latter enjoys certain practical advantages (such as the possibility of (a) running faster than their batch processing counterparts and (b) escaping from local minima of the associated cost function), while, in addition, strong theoretical convergence results have been established for it. In this paper a novel stochastic gradient descent possibilistic clustering algorithm, called O- PCM 2 is introduced. The algorithm is presented in detail and it is rigorously proved that the gradient of the associated cost function tends to zero in the L 2 sense, based on general convergence results established for the family of the stochastic gradient descent algorithms. Furthermore, an additional discussion is provided on the nature of the points where the algorithm may converge. Finally, the performance of the proposed algorithm is tested against other related algorithms, on the basis of both synthetic and real data sets.

Mixture Models Based on Neural Network Averaging

2006 ◽  
Vol 18 (1) ◽  
pp. 1-9 ◽  
Author(s):  
Walter W. Focke
Keyword(s):  
Neural Network ◽  
Transfer Function ◽  
Mixture Model ◽  
Network Architecture ◽  
Binary Data ◽  
Data Sets ◽  
Wilson Equation ◽  
Power Mean ◽  
Hidden Layer ◽  
Box Cox Transformation

A modified version of the single hidden-layer perceptron architecture is proposed for modeling mixtures. A particular flexible mixture model is obtained by implementing the Box-Cox transformation as transfer function. In this case, the network response can be expressed in closed form as a weighted power mean. The quadratic Scheffé K-polynomial and the exponential Wilson equation turn out to be special forms of this general mixture model. Advantages of the proposed network architecture are that binary data sets suffice for “training” and that it is readily extended to incorporate additional mixture components while retaining all previously determined weights.

A PROBABILISTIC SELF-ORGANIZING MAP FOR BINARY DATA TOPOGRAPHIC CLUSTERING

2008 ◽  
Vol 07 (04) ◽  
pp. 363-383 ◽  
Author(s):  
MUSTAPHA LEBBAH ◽  
YOUNÈS BENNANI ◽  
NICOLETA ROGOVSCHI
Keyword(s):  
Binary Data ◽  
Learning Algorithm ◽  
Data Sets ◽  
Self Organizing Map ◽  
Data Set ◽  
Binary Coding ◽  
Public Data ◽  
Multivariate Binary Data ◽  
Self Organizing

This paper introduces a probabilistic self-organizing map for topographic clustering, analysis and visualization of multivariate binary data or categorical data using binary coding. We propose a probabilistic formalism dedicated to binary data in which cells are represented by a Bernoulli distribution. Each cell is characterized by a prototype with the same binary coding as used in the data space and the probability of being different from this prototype. The learning algorithm, Bernoulli on self-organizing map, that we propose is an application of the EM standard algorithm. We illustrate the power of this method with six data sets taken from a public data set repository. The results show a good quality of the topological ordering and homogenous clustering.

Stochastic Gradient Descent Based K-Means Algorithm on Large Scale Data Clustering

2014 ◽  
Vol 687-691 ◽  
pp. 1342-1345 ◽  
Author(s):  
Jie Ding ◽  
Li Peng Zhu ◽  
Bin Hu ◽  
Ren Long Hang ◽  
Yu Bao Sun
Keyword(s):  
Gradient Descent ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Distance Matrix ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Data Sets ◽  
Human Beings ◽  
Large Scale Data ◽  
Scale Data

With the rapid advance of data collection and storage technique, it is easy to acquire tens of millions or even billions of data sets. How to explore and exploit the useful or interesting information for human beings from these data sets has become an urgent issue. Traditional k-means clustering algorithm has been widely used in data mining community. First, randomly initialize k clustering centres. Then, all instances are classified into k different classes according to their distances to clustering centres. Lastly, update the clustering centres by the mean of its corresponding constituent instances. This whole process will be iterated until convergence. Obviously, at each iteration, distance matrix from all instances to k clustering centres must be calculated which will cost so much time when encounter large scale data sets. To address this issue, in this paper, we proposed a fast optimization algorithm based on stochastic gradient descent (SGD). At each iteration, randomly choose an instance, search its corresponding clustering centre and then update it immediately. Experimental results show that our proposed method achieves a competitive clustering results with less time cost.

scholarly journals A cluster Analysis for Binary Data Using Genetic Algorithms

2018 ◽  
Vol 7 (4.30) ◽  
pp. 550
Author(s):  
Sabariah Saharan ◽  
Wong Yu Xian ◽  
Roberto Baragona
Keyword(s):  
Binary Data ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Imbalanced Data ◽  
Unbalanced Data ◽  
Data Sets ◽  
Main Subject ◽  
Promising Technique ◽  
Data Standard ◽  
Genetic Clustering

This research was initially driven by the lack of clustering algorithms that focus on binary data. A promising technique to analyze this type of data, namely Genetic Clustering for Unknown K (GCUK) became the main subject in this research. GCUK was applied to cluster four binary data and there is a presence of an imbalanced data in one of the data sets. The results show that GCUK is an efficient and effective clustering algorithm compared to K-means. The other contribution is the capability of GCUK for clustering the unbalanced data. Standard clustering algorithms cannot simply be applied to this type of data sets as it can cause a misclassification results. 

scholarly journals A Novel Maximum Mean Discrepancy-Based Semi-Supervised Learning Algorithm

Mathematics ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 39
Author(s):  
Qihang Huang ◽  
Yulin He ◽  
Zhexue Huang
Keyword(s):  
Statistical Analysis ◽  
Supervised Learning ◽  
Clustering Algorithm ◽  
Learning Algorithm ◽  
Data Sets ◽  
Generalization Capability ◽  
Maximum Mean Discrepancy ◽  
Benchmark Data ◽  
Testing Accuracy ◽  
Multiple Groups

To provide more external knowledge for training self-supervised learning (SSL) algorithms, this paper proposes a maximum mean discrepancy-based SSL (MMD-SSL) algorithm, which trains a well-performing classifier by iteratively refining the classifier using highly confident unlabeled samples. The MMD-SSL algorithm performs three main steps. First, a multilayer perceptron (MLP) is trained based on the labeled samples and is then used to assign labels to unlabeled samples. Second, the unlabeled samples are divided into multiple groups with the k-means clustering algorithm. Third, the maximum mean discrepancy (MMD) criterion is used to measure the distribution consistency between k-means-clustered samples and MLP-classified samples. The samples having a consistent distribution are labeled as highly confident samples and used to retrain the MLP. The MMD-SSL algorithm performs an iterative training until all unlabeled samples are consistently labeled. We conducted extensive experiments on 29 benchmark data sets to validate the rationality and effectiveness of the MMD-SSL algorithm. Experimental results show that the generalization capability of the MLP algorithm can gradually improve with the increase of labeled samples and the statistical analysis demonstrates that the MMD-SSL algorithm can provide better testing accuracy and kappa values than 10 other self-training and co-training SSL algorithms.

scholarly journals Time series clustering in large data sets

2011 ◽  
Vol 59 (2) ◽  
pp. 75-80 ◽  
Author(s):  
Jiří Fejfar ◽  
Jiří Šťastný
Keyword(s):  
Time Series ◽  
Digital Libraries ◽  
Clustering Algorithm ◽  
Learning Algorithm ◽  
Large Data ◽  
Data Sets ◽  
Self Organizing Map ◽  
Time Series Clustering ◽  
Feature Vectors ◽  
Cover Songs

The clustering of time series is a widely researched area. There are many methods for dealing with this task. We are actually using the Self-organizing map (SOM) with the unsupervised learning algorithm for clustering of time series. After the first experiment (Fejfar, Weinlichová, Šťastný, 2009) it seems that the whole concept of the clustering algorithm is correct but that we have to perform time series clustering on much larger dataset to obtain more accurate results and to find the correlation between configured parameters and results more precisely. The second requirement arose in a need for a well-defined evaluation of results. It seems useful to use sound recordings as instances of time series again. There are many recordings to use in digital libraries, many interesting features and patterns can be found in this area. We are searching for recordings with the similar development of information density in this experiment. It can be used for musical form investigation, cover songs detection and many others applications.The objective of the presented paper is to compare clustering results made with different parameters of feature vectors and the SOM itself. We are describing time series in a simplistic way evaluating standard deviations for separated parts of recordings. The resulting feature vectors are clustered with the SOM in batch training mode with different topologies varying from few neurons to large maps.There are other algorithms discussed, usable for finding similarities between time series and finally conclusions for further research are presented. We also present an overview of the related actual literature and projects.

An Unsupervised Learning Algorithm to Compute Fluid Volumes From NMR T1-T2 Logs in Unconventional Reservoirs

2018 ◽  
Vol 59 ◽  
pp. 617-632 ◽  
Author(s):  
Lalitha Venkataramanan ◽  
◽  
Noyan Evirgen ◽  
David F. Allen ◽  
Albina Mutina ◽  
...  
Keyword(s):  
Unsupervised Learning ◽  
Learning Algorithm ◽  
Unconventional Reservoirs

Application of Machine Learning in Animal Disease Analysis and Prediction

2020 ◽  
Vol 15 ◽  
Author(s):  
Shuwen Zhang ◽  
Qiang Su ◽  
Qin Chen
Keyword(s):  
Machine Learning ◽  
Unsupervised Learning ◽  
Supervised Learning ◽  
Clustering Algorithm ◽  
Principal Component ◽  
Support Vector ◽  
Animal Disease ◽  
Human Beings ◽  
Animal Diseases ◽  
Disease Analysis

Abstract: Major animal diseases pose a great threat to animal husbandry and human beings. With the deepening of globalization and the abundance of data resources, the prediction and analysis of animal diseases by using big data are becoming more and more important. The focus of machine learning is to make computers learn how to learn from data and use the learned experience to analyze and predict. Firstly, this paper introduces the animal epidemic situation and machine learning. Then it briefly introduces the application of machine learning in animal disease analysis and prediction. Machine learning is mainly divided into supervised learning and unsupervised learning. Supervised learning includes support vector machines, naive bayes, decision trees, random forests, logistic regression, artificial neural networks, deep learning, and AdaBoost. Unsupervised learning has maximum expectation algorithm, principal component analysis hierarchical clustering algorithm and maxent. Through the discussion of this paper, people have a clearer concept of machine learning and understand its application prospect in animal diseases.

Sign in / Sign up

Export Citation Format

Share Document