scholarly journals Attribute Selecting in Tree-Augmented Naive Bayes by Cross Validation Risk Minimization

Mathematics ◽  
2021 ◽  
Vol 9 (20) ◽  
pp. 2564
Author(s):  
Shenglei Chen ◽  
Zhonghui Zhang ◽  
Linyuan Liu
Keyword(s):  
Cross Validation ◽  
Naive Bayes ◽  
Model Space ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Superior Performance ◽  
Data Sets ◽  
Risk Minimization ◽  
Local Optima ◽  
Class Variable

As an important improvement to naive Bayes, Tree-Augmented Naive Bayes (TAN) exhibits excellent classification performance and efficiency since it allows that every attribute depends on at most one other attribute in addition to the class variable. However, its performance might be lowered as some attributes might be redundant. In this paper, we propose an attribute Selective Tree-Augmented Naive Bayes (STAN) algorithm which builds a sequence of approximate models each involving only the top certain attributes and searches the model to minimize the cross validation risk. Five different approaches to ranking the attributes have been explored. As the models can be evaluated simultaneously in one pass learning through the data, it is efficient and can avoid local optima in the model space. The extensive experiments on 70 UCI data sets demonstrated that STAN achieves superior performance while maintaining the efficiency and simplicity.

USING INSTANCE CLONING TO IMPROVE NAIVE BAYES FOR RANKING

2008 ◽  
Vol 22 (06) ◽  
pp. 1121-1140 ◽  
Author(s):  
LIANGXIAO JIANG ◽  
DIANHONG WANG ◽  
HARRY ZHANG ◽  
ZHIHUA CAI ◽  
BO HUANG
Keyword(s):  
Naive Bayes ◽  
Scaling Up ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Data Sets ◽  
Lazy Learning ◽  
Real World Data ◽  
Ranking Performance ◽  
Test Instance ◽  
Significant Attention

Improving naive Bayes (simply NB)15,28 for classification has received significant attention. Related work can be broadly divided into two approaches: eager learning and lazy learning.1 Different from eager learning, the key idea for extending naive Bayes using lazy learning is to learn an improved naive Bayes for each test instance. In recent years, several lazy extensions of naive Bayes have been proposed. For example, LBR,30 SNNB,27 and LWNB.8 All these algorithms aim to improve naive Bayes' classification performance. Indeed, they achieve significant improvement in terms of classification, measured by accuracy. In many real-world data mining applications, however, an accurate ranking is more desirable than an accurate classification. Thus a natural question is whether they also achieve significant improvement in terms of ranking, measured by AUC (the area under the ROC curve).2,11,17 Responding to this question, we conduct experiments on the 36 UCI data sets18 selected by Weka12 to investigate their ranking performance and find that they do not significantly improve the ranking performance of naive Bayes. Aiming at scaling up naive Bayes' ranking performance, we present a novel lazy method ICNB (instance cloned naive Bayes) and develop three ICNB algorithms using different instance cloning strategies. We empirically compare them with naive Bayes. The experimental results show that our algorithms achieve significant improvement in terms of AUC. Our research provides a simple but effective method for the applications where an accurate ranking is desirable.

scholarly journals A Pairwise Naïve Bayes Approach to Bayesian Classification

2015 ◽  
Vol 29 (07) ◽  
pp. 1550023 ◽  
Author(s):  
Josephine K. Asafu-Adjei ◽  
Rebecca A. Betensky
Keyword(s):  
Joint Distribution ◽  
Naive Bayes ◽  
Sufficient Conditions ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Estimation Methods ◽  
Data Sets ◽  
Normal Density ◽  
Bayes Classifier ◽  
Empirical Estimation

Despite the relatively high accuracy of the naïve Bayes (NB) classifier, there may be several instances where it is not optimal, i.e. does not have the same classification performance as the Bayes classifier utilizing the joint distribution of the examined attributes. However, the Bayes classifier can be computationally intractable due to its required knowledge of the joint distribution. Therefore, we introduce a "pairwise naïve" Bayes (PNB) classifier that incorporates all pairwise relationships among the examined attributes, but does not require specification of the joint distribution. In this paper, we first describe the necessary and sufficient conditions under which the PNB classifier is optimal. We then discuss sufficient conditions for which the PNB classifier, and not NB, is optimal for normal attributes. Through simulation and actual studies, we evaluate the performance of our proposed classifier relative to the Bayes and NB classifiers, along with the HNB, AODE, LBR and TAN classifiers, using normal density and empirical estimation methods. Our applications show that the PNB classifier using normal density estimation yields the highest accuracy for data sets containing continuous attributes. We conclude that it offers a useful compromise between the Bayes and NB classifiers.

Analysis of the Use of Background Distribution for Naive Bayes Classifiers

2019 ◽  
Vol 28 (2) ◽  
pp. 259-273 ◽  
Author(s):  
Daniel Andrade ◽  
Akihiro Tamura ◽  
Masaaki Tsuchida
Keyword(s):  
Text Classification ◽  
Cross Validation ◽  
Graphical Model ◽  
Naive Bayes ◽  
Parameter Tuning ◽  
Naïve Bayes ◽  
Data Sets ◽  
Bayes Classifier ◽  
Background Distribution ◽  
Standard Text

Abstract The naive Bayes classifier is a popular classifier, as it is easy to train, requires no cross-validation for parameter tuning, and can be easily extended due to its generative model. Moreover, recently it was shown that the word probabilities (background distribution) estimated from large unlabeled corpora could be used to improve the parameter estimation of naive Bayes. However, previous methods do not explicitly allow to control how much the background distribution can influence the estimation of naive Bayes parameters. In contrast, we investigate an extension of the graphical model of naive Bayes such that a word is either generated from a background distribution or from a class-specific word distribution. We theoretically analyze this model and show the connection to Jelinek-Mercer smoothing. Experiments using four standard text classification data sets show that the proposed method can statistically significantly outperform previous methods that use the same background distribution.

scholarly journals Exact Learning Augmented Naive Bayes Classifier

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1703
Author(s):  
Shouta Sugahara ◽  
Maomi Ueno
Keyword(s):  
Naive Bayes ◽  
Marginal Likelihood ◽  
Large Data ◽  
Naïve Bayes ◽  
Superior Performance ◽  
Naive Bayes Classifier ◽  
Bayes Classifier ◽  
Naïve Bayes Classifier ◽  
Exact Learning ◽  
Class Variable

Earlier studies have shown that classification accuracies of Bayesian networks (BNs) obtained by maximizing the conditional log likelihood (CLL) of a class variable, given the feature variables, were higher than those obtained by maximizing the marginal likelihood (ML). However, differences between the performances of the two scores in the earlier studies may be attributed to the fact that they used approximate learning algorithms, not exact ones. This paper compares the classification accuracies of BNs with approximate learning using CLL to those with exact learning using ML. The results demonstrate that the classification accuracies of BNs obtained by maximizing the ML are higher than those obtained by maximizing the CLL for large data. However, the results also demonstrate that the classification accuracies of exact learning BNs using the ML are much worse than those of other methods when the sample size is small and the class variable has numerous parents. To resolve the problem, we propose an exact learning augmented naive Bayes classifier (ANB), which ensures a class variable with no parents. The proposed method is guaranteed to asymptotically estimate the identical class posterior to that of the exactly learned BN. Comparison experiments demonstrated the superior performance of the proposed method.

scholarly journals Structure Extension of Tree-Augmented Naive Bayes

Entropy ◽  
2019 ◽  
Vol 21 (8) ◽  
pp. 721 ◽  
Author(s):  
YuGuang Long ◽  
LiMin Wang ◽  
MingHui Sun
Keyword(s):  
Naive Bayes ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Training Data ◽  
Conditional Probability Distribution ◽  
Independence Assumption ◽  
Bayesian Network Classifiers ◽  
Leibler Divergence ◽  
The Difference ◽  
Structure Extension

Due to the simplicity and competitive classification performance of the naive Bayes (NB), researchers have proposed many approaches to improve NB by weakening its attribute independence assumption. Through the theoretical analysis of Kullback–Leibler divergence, the difference between NB and its variations lies in different orders of conditional mutual information represented by these augmenting edges in the tree-shaped network structure. In this paper, we propose to relax the independence assumption by further generalizing tree-augmented naive Bayes (TAN) from 1-dependence Bayesian network classifiers (BNC) to arbitrary k-dependence. Sub-models of TAN that are built to respectively represent specific conditional dependence relationships may “best match” the conditional probability distribution over the training data. Extensive experimental results reveal that the proposed algorithm achieves bias-variance trade-off and substantially better generalization performance than state-of-the-art classifiers such as logistic regression.

scholarly journals Multiple Naïve Bayes Classifiers Ensemble for Traffic Incident Detection

2014 ◽  
Vol 2014 ◽  
pp. 1-16 ◽  
Author(s):  
Qingchao Liu ◽  
Jian Lu ◽  
Shuyan Chen ◽  
Kangjia Zhao
Keyword(s):  
Decision Tree ◽  
Naive Bayes ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Classifier Ensemble ◽  
Optimal Threshold ◽  
Incident Detection ◽  
Bayes Classifier ◽  
Traffic Incident ◽  
Better Than

This study presents the applicability of the Naïve Bayes classifier ensemble for traffic incident detection. The standard Naive Bayes (NB) has been applied to traffic incident detection and has achieved good results. However, the detection result of the practically implemented NB depends on the choice of the optimal threshold, which is determined mathematically by using Bayesian concepts in the incident-detection process. To avoid the burden of choosing the optimal threshold and tuning the parameters and, furthermore, to improve the limited classification performance of the NB and to enhance the detection performance, we propose an NB classifier ensemble for incident detection. In addition, we also propose to combine the Naïve Bayes and decision tree (NBTree) to detect incidents. In this paper, we discuss extensive experiments that were performed to evaluate the performances of three algorithms: standard NB, NB ensemble, and NBTree. The experimental results indicate that the performances of five rules of the NB classifier ensemble are significantly better than those of standard NB and slightly better than those of NBTree in terms of some indicators. More importantly, the performances of the NB classifier ensemble are very stable.

scholarly journals Sistem Informasi Posyandu Ibu Hamil dengan Penerapan Klasifikasi Resiko Kehamilan Menggunakan Metode Naïve Bayes

2018 ◽  
Vol 6 (1) ◽  
pp. 1
Author(s):  
Qomariyatul Hasanah ◽  
Anang Andrianto ◽  
Muhammad Arief Hidayat
Keyword(s):  
Cross Validation ◽  
Naive Bayes ◽  
Naïve Bayes ◽  
Training Set ◽  
Test Set ◽  
Fold Cross Validation

Sistem informasi posyandu ibu hamil dapat mengelola data kesehatan ibu hamil yang berkaitan dengan faktor resiko kehamilan. Faktor resiko kehamilan berdasarkan ketentuan Kartu Skor Poedji Rochyati (KSPR) digunakan bidan untuk menentukan resiko kehamilan dengan memberikan skor pada masing-masing parameter. KSPR memiliki kelemahan tidak dapat memberikan skor pada parameter yang belum pasti sehingga jika belum diketahui dengan pasti maka dianggap tidak terjadi. Konsep membaca pola data yang diadopsi dari teknik datamining menggunakan metode klasifikasi naive bayes dapat menjadi alternatif untuk kelemahan KSPR tersebut yaitu dengan mengklasifikasikan resiko kehamilan. Metode naïve bayes menghitung probabilitas parameter tertentu berdasarkan data pada periode sebelumnya yang telah ditentukan sebagai data training, berdasarkan hasil perhitungan tersebut dapat diketahui resiko kehamilan secara tepat sesuai parameter yang telah diketahui. Metode naïve bayes dipilih karena memiliki tingkat akurasi yang cukup tinggi daripada metode klasifikasi lainnya. Sistem informasi ini dibangun berbasis website agar dapat diakses secara mudah oleh beberapa posyandu yang berbeda tempat. Sistem dibangun mengadopsi dari model Waterfall. Sistem informasi posyandu ibu hamil dirancang dan dibangun dengan tiga (3) hak akses yaitu admin, bidan dan kader dengan masing-masing fitur yang dapat memudahkan penggunanya. Hasil dari penelitian ini adalah sistem informasi posyandu ibu hamil dengan penerapan klasifikasi resiko kehamilan menggunakan metode naïve bayes, dengan tingkat akurasi ketika menggunakan 17 atribut didapatkan 53.913%, 19 atribut didapatkan 54.348%, , 21 atribut didapatkan 54.783%, dan 22 atribut didapatkan 56.957%. Tingkat akurasi klasifikasi diperoleh menggunakan metode pengujian menggunakan Ten-Fold Cross Validation dimana training set dibagi menjadi 10 kelompok, jika kelompok 1 dijadikan test set maka kelompok 2 hingga 10 menjadi training set. Kata Kunci: Posyandu, Resiko Kehamilan, Waterfall, Datamining, Klasifikasi, Naïve bayes

scholarly journals Estratégias de Construções de Comitês de Classificadores Multirrótulos no Aprendizado Semissupervisionado Multidescrição

2017 ◽  
Vol 24 (2) ◽  
pp. 71
Author(s):  
Wilamis Kleiton Nunes Da Silva ◽  
Araken De Medeiros Santos
Keyword(s):  
Cross Validation ◽  
Naive Bayes ◽  
Naïve Bayes ◽  
Binary Relevance ◽  
Fold Cross Validation

Classificação multirrótulo é um problema de aprendizado supervisionado no qual um objeto pode estar associado a múltiplas classes. Dentre os diferentes métodos de classificação multirrótulo destacam-se os métodos BR (Binary Relevance), LP (Label Powerset) e RAkEL (RAndom k-labELsets). O trabalho realizou um estudo sobre as construções de comitês de classificadores multirrótulos construídos através da aplicação de técnicas de aprendizado semissupervisionado multidescrição. Os comitês de classificadores utilizados nos experimentos foram o Bagging, Boosting e Stacking; como métodos de transformação do problema utilizamos os métodos BR, LP e Rakel; na classificação multirrótulo semissupervisionada multidescrição foi utilizado o Co-Training; foram aplicados cinco diferentes algoritmos como classificadores base: k-NN (k Vizinhos Mais Próximos), J48 (Algoritmo de Indução de Árvores de Decisão), SVM (Máquinas de Vetores Suporte), NB (Naive Bayes) e o JRip (Extended Repeated Incremental Pruning). Todos os experimentos utilizaram a metodologia de validação cruzada com 10 grupos (10-fold Cross-Validation) e o framework MULAN, o qual é implementado utilizando o WEKA. Para os tamanhos dos comitês de classificadores adotamos os valores 3, 5, 7 e 9. Para a análise dos resultados foi utilizado o teste esta- tístico de Wilcoxon. Ao final das análises experimentais, verificou-se que a abordagem semissupervisionado apresentou resultados competitivos em relação ao aprendizado supervisionado, uma vez que as duas abordagens utilizadas apresentaram resultados estatisticamente semelhantes.   

scholarly journals Klasifikasi Berita Hoax Dengan Menggunakan Metode Naive Bayes

2019 ◽  
Vol 1 (1) ◽  
pp. 1
Author(s):  
Hery Mustofa ◽  
Adzhal Arwani Mahfudh
Keyword(s):  
Information Technology ◽  
Cross Validation ◽  
Naive Bayes ◽  
Intelligent System ◽  
Naïve Bayes ◽  
The Internet ◽  
Average Value ◽  
Internet Media ◽  
News Value ◽  
False News

<p><em>Hoaxes contain false news or non-sourced news. Today, hoaxes are very widely spread through internet media. The development of information technology that has so quickly triggered the spread of hoax information through the internet has become uncontrolled. So we need an intelligent system that can classify hoax news content that is spread through internet media. The hoax classification process can be done through the preprocessing stage then weighting the word and classification using naive bayes. Measurements were made using the 10-ford cross validation method. The results obtained from these measurements, it is known that the value of fold 6 has the highest accuracy, which is equal to 85.28% which is classified as relevant documents as much as 307 and irrelevant as much as 53 or an error rate of 14.72%. While the average value based on hoax news and true news value precision 0.896 and recall 0.853</em></p>

Locally-Adaptive Naïve Bayes Framework Design via Density-Based Clustering for Large Scale Datasets

2021 ◽  
pp. 278-292
Author(s):  
Faruk Bulut
Keyword(s):  
Learning Style ◽  
Large Scale ◽  
Naive Bayes ◽  
Empirical Studies ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Test Point ◽  
Framework Design ◽  
Locally Adaptive ◽  
Generalized Framework

In this chapter, local conditional probabilities of a query point are used in classification rather than consulting a generalized framework containing a conditional probability. In the proposed locally adaptive naïve Bayes (LANB) learning style, a certain amount of local instances, which are close the test point, construct an adaptive probability estimation. In the empirical studies of over the 53 benchmark UCI datasets, more accurate classification performance has been obtained. A total 8.2% increase in classification accuracy has been gained with LANB when compared to the conventional naïve Bayes model. The presented LANB method has outperformed according to the statistical paired t-test comparisons: 31 wins, 14 ties, and 8 losses of all UCI sets.

Sign in / Sign up

Export Citation Format

Share Document