Unsupervised training of Bayesian networks for data clustering

2009 ◽  
Vol 465 (2109) ◽  
pp. 2927-2948 ◽  
Author(s):  
Duc Truong Pham ◽  
Gonzalo A. Ruz
Keyword(s):  
Em Algorithm ◽  
Bayesian Network ◽  
Structure Learning ◽  
Clustering Algorithms ◽  
Processing Application ◽  
Wood Board ◽  
Bayesian Network Classifiers ◽  
The Em Algorithm ◽  
Cent Increase ◽  
Unsupervised Training

This paper presents a new approach to the unsupervised training of Bayesian network classifiers. Three models have been analysed: the Chow and Liu (CL) multinets; the tree-augmented naive Bayes; and a new model called the simple Bayesian network classifier, which is more robust in its structure learning. To perform the unsupervised training of these models, the classification maximum likelihood criterion is used. The maximization of this criterion is derived for each model under the classification expectation–maximization (EM) algorithm framework. To test the proposed unsupervised training approach, 10 well-known benchmark datasets have been used to measure their clustering performance. Also, for comparison, the results for the k -means and the EM algorithm, as well as those obtained when the three Bayesian network classifiers are trained in a supervised way, are analysed. A real-world image processing application is also presented, dealing with clustering of wood board images described by 165 attributes. Results show that the proposed learning method, in general, outperforms traditional clustering algorithms and, in the wood board image application, the CL multinets obtained a 12 per cent increase, on average, in clustering accuracy when compared with the k -means method and a 7 per cent increase, on average, when compared with the EM algorithm.

A novel approach to fully representing the diversity in conditional dependencies for learning Bayesian network classifier

2021 ◽  
Vol 25 (1) ◽  
pp. 35-55
Author(s):  
Limin Wang ◽  
Peng Chen ◽  
Shenglei Chen ◽  
Minghui Sun
Keyword(s):  
Bayesian Network ◽  
Conditional Independence ◽  
Structure Learning ◽  
Classification Performance ◽  
Data Sets ◽  
Independence Assumption ◽  
Bayesian Network Classifiers ◽  
Novel Approach ◽  
Conditional Independence Assumption ◽  
Dependence Criterion

Bayesian network classifiers (BNCs) have proved their effectiveness and efficiency in the supervised learning framework. Numerous variations of conditional independence assumption have been proposed to address the issue of NP-hard structure learning of BNC. However, researchers focus on identifying conditional dependence rather than conditional independence, and information-theoretic criteria cannot identify the diversity in conditional (in)dependencies for different instances. In this paper, the maximum correlation criterion and minimum dependence criterion are introduced to sort attributes and identify conditional independencies, respectively. The heuristic search strategy is applied to find possible global solution for achieving the trade-off between significant dependency relationships and independence assumption. Our extensive experimental evaluation on widely used benchmark data sets reveals that the proposed algorithm achieves competitive classification performance compared to state-of-the-art single model learners (e.g., TAN, KDB, KNN and SVM) and ensemble learners (e.g., ATAN and AODE).

Bandit-Based Structure Learning for Bayesian Network Classifiers

2012 ◽  
pp. 349-356 ◽  
Author(s):  
Sepehr Eghbali ◽  
Mohammad Hassan Zokaei Ashtiani ◽  
Majid Nili Ahmadabadi ◽  
Babak Nadjar Araabi
Keyword(s):  
Bayesian Network ◽  
Structure Learning ◽  
Bayesian Network Classifiers

scholarly journals Efficient Heuristics for Structure Learning of k-Dependence Bayesian Classifier

Entropy ◽  
2018 ◽  
Vol 20 (12) ◽  
pp. 897 ◽  
Author(s):  
Yang Liu ◽  
Limin Wang ◽  
Minghui Sun
Keyword(s):  
Bayesian Network ◽  
Structure Learning ◽  
State Of The Art ◽  
Classification Performance ◽  
Bayesian Classifier ◽  
Bayesian Network Classifiers ◽  
Discriminative Model ◽  
Minimal Redundancy ◽  
Structure Complexity ◽  
Maximal Relevance

The rapid growth in data makes the quest for highly scalable learners a popular one. To achieve the trade-off between structure complexity and classification accuracy, the k-dependence Bayesian classifier (KDB) allows to represent different number of interdependencies for different data sizes. In this paper, we proposed two methods to improve the classification performance of KDB. Firstly, we use the minimal-redundancy-maximal-relevance analysis, which sorts the predictive features to identify redundant ones. Then, we propose an improved discriminative model selection to select an optimal sub-model by removing redundant features and arcs in the Bayesian network. Experimental results on 40 UCI datasets demonstrate that these two techniques are complementary and the proposed algorithm achieves competitive classification performance, and less classification time than other state-of-the-art Bayesian network classifiers like tree-augmented naive Bayes and averaged one-dependence estimators.

scholarly journals Structure Learning of Bayesian Network Based on Adaptive Thresholding

Entropy ◽  
2019 ◽  
Vol 21 (7) ◽  
pp. 665
Author(s):  
Yang Zhang ◽  
Limin Wang ◽  
Zhiyi Duan ◽  
Minghui Sun
Keyword(s):  
Bayesian Network ◽  
Structure Learning ◽  
Mean Squared Error ◽  
Structural Complexity ◽  
Classification Performance ◽  
Adaptive Thresholding ◽  
Bayesian Network Classifiers ◽  
Structure Reliability ◽  
Adaptive Thresholds ◽  
Error Bias

Direct dependencies and conditional dependencies in restricted Bayesian network classifiers (BNCs) are two basic kinds of dependencies. Traditional approaches, such as filter and wrapper, have proved to be beneficial to identify non-significant dependencies one by one, whereas the high computational overheads make them inefficient especially for those BNCs with high structural complexity. Study of the distributions of information-theoretic measures provides a feasible approach to identifying non-significant dependencies in batch that may help increase the structure reliability and avoid overfitting. In this paper, we investigate two extensions to the k-dependence Bayesian classifier, MI-based feature selection, and CMI-based dependence selection. These two techniques apply a novel adaptive thresholding method to filter out redundancy and can work jointly. Experimental results on 30 datasets from the UCI machine learning repository demonstrate that adaptive thresholds can help distinguish between dependencies and independencies and the proposed algorithm achieves competitive classification performance compared to several state-of-the-art BNCs in terms of 0–1 loss, root mean squared error, bias, and variance.

scholarly journals Discriminative Structure Learning of Bayesian Network Classifiers from Training Dataset and Testing Instance

Entropy ◽  
2019 ◽  
Vol 21 (5) ◽  
pp. 489 ◽  
Author(s):  
Limin Wang ◽  
Yang Liu ◽  
Musa Mammadov ◽  
Minghui Sun ◽  
Sikai Qi
Keyword(s):  
Bayesian Network ◽  
Learning Strategy ◽  
Structure Learning ◽  
Naive Bayes ◽  
Search Space ◽  
Naïve Bayes ◽  
Bayesian Classifier ◽  
Training Data ◽  
Training Dataset ◽  
Bayesian Network Classifiers

Over recent decades, the rapid growth in data makes ever more urgent the quest for highly scalable Bayesian networks that have better classification performance and expressivity (that is, capacity to respectively describe dependence relationships between attributes in different situations). To reduce the search space of possible attribute orders, k-dependence Bayesian classifier (KDB) simply applies mutual information to sort attributes. This sorting strategy is very efficient but it neglects the conditional dependencies between attributes and is sub-optimal. In this paper, we propose a novel sorting strategy and extend KDB from a single restricted network to unrestricted ensemble networks, i.e., unrestricted Bayesian classifier (UKDB), in terms of Markov blanket analysis and target learning. Target learning is a framework that takes each unlabeled testing instance P as a target and builds a specific Bayesian model Bayesian network classifiers (BNC) P to complement BNC T learned from training data T . UKDB respectively introduced UKDB P and UKDB T to flexibly describe the change in dependence relationships for different testing instances and the robust dependence relationships implicated in training data. They both use UKDB as the base classifier by applying the same learning strategy while modeling different parts of the data space, thus they are complementary in nature. The extensive experimental results on the Wisconsin breast cancer database for case study and other 10 datasets by involving classifiers with different structure complexities, such as Naive Bayes (0-dependence), Tree augmented Naive Bayes (1-dependence) and KDB (arbitrary k-dependence), prove the effectiveness and robustness of the proposed approach.

Sign in / Sign up

Export Citation Format

Share Document