scholarly journals Transductive Bounds for the Multi-Class Majority Vote Classifier

2019 ◽  
Vol 33 ◽  
pp. 3566-3573
Author(s):  
Vasilii Feofanov ◽  
Emilie Devijver ◽  
Massih-Reza Amini
Keyword(s):  
Learning Algorithm ◽  
Majority Vote ◽  
Confusion Matrix ◽  
Data Sets ◽  
Bayes Classifier ◽  
Partially Labeled Data ◽  
Margin Distribution ◽  
Training Examples ◽  
Multi Class Classification ◽  
Self Learning

In this paper, we propose a transductive bound over the risk of the majority vote classifier learned with partially labeled data for the multi-class classification. The bound is obtained by considering the class confusion matrix as an error indicator and it involves the margin distribution of the classifier over each class and a bound over the risk of the associated Gibbs classifier. When this latter bound is tight and, the errors of the majority vote classifier per class are concentrated on a low margin zone; we prove that the bound over the Bayes classifier’ risk is tight. As an application, we extend the self-learning algorithm to the multi-class case. The algorithm iteratively assigns pseudo-labels to a subset of unlabeled training examples that have their associated class margin above a threshold obtained from the proposed transductive bound. Empirical results on different data sets show the effectiveness of our approach compared to the same algorithm where the threshold is fixed manually, to the extension of TSVM to multi-class classification and to a graph-based semi-supervised algorithm.

scholarly journals Rademacher Complexity Bounds for a Penalized Multi-class Semi-supervised Algorithm

2018 ◽  
Vol 61 ◽  
pp. 761-786 ◽  
Author(s):  
Yury Maximov ◽  
Massih-Reza Amini ◽  
Zaid Harchaoui
Keyword(s):  
Convergence Rates ◽  
Training Data ◽  
Classification Problems ◽  
Complexity Bounds ◽  
Rademacher Complexity ◽  
Partially Labeled Data ◽  
Margin Distribution ◽  
Training Examples ◽  
Multi Class Classification ◽  
The Stability

We propose Rademacher complexity bounds for multi-class classifiers trained with a two-step semi-supervised model. In the first step, the algorithm partitions the partially labeled data and then identifies dense clusters containing k predominant classes using the labeled training examples such that the proportion of their non-predominant classes is below a fixed threshold stands for clustering consistency. In the second step, a classifier is trained by minimizing a margin empirical loss over the labeled training set and a penalization term measuring the disability of the learner to predict the k predominant classes of the identified clusters. The resulting data-dependent generalization error bound involves the margin distribution of the classifier, the stability of the clustering technique used in the first step and Rademacher complexity terms corresponding to partially labeled training data. Our theoretical result exhibit convergence rates extending those proposed in the literature for the binary case, and experimental results on different multi-class classification problems show empirical evidence that supports the theory.

scholarly journals Rademacher Complexity Bounds for a Penalized Multi-class Semi-supervised Algorithm (Extended Abstract)

2018 ◽  
Author(s):  
Yury Maximov ◽  
Massih-Reza Amini ◽  
Zaid Harchaoui
Keyword(s):  
Convergence Rates ◽  
Training Data ◽  
Classification Problems ◽  
Complexity Bounds ◽  
Rademacher Complexity ◽  
Partially Labeled Data ◽  
Margin Distribution ◽  
Training Examples ◽  
Multi Class Classification ◽  
The Stability

We propose Rademacher complexity bounds for multi-class classifiers trained with a two-step semi-supervised model. In the first step, the algorithm partitions the partially labeled data and then identifies dense clusters containing k predominant classes using the labeled training examples such that the proportion of their non-predominant classes is below a fixed threshold stands for clustering consistency. In the second step, a classifier is trained by minimizing a margin empirical loss over the labeled training set and a penalization term measuring the disability of the learner to predict the k predominant classes of the identified clusters. The resulting data-dependent generalization error bound involves the margin distribution of the classifier, the stability of the clustering technique used in the first step and Rademacher complexity terms corresponding to partially labeled training data. Our theoretical result exhibit convergence rates extending those proposed in the literature for the binary case, and experimental results on different multi-class classification problems show empirical evidence that supports the theory.

FUZZY RULE EXTRACTION FROM SIMPLE EVOLVING CONNECTIONIST SYSTEMS

2004 ◽  
Vol 04 (03) ◽  
pp. 299-308 ◽  
Author(s):  
MICHAEL J. WATTS
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Learning Algorithm ◽  
Fuzzy Rule ◽  
Rule Extraction ◽  
Data Sets ◽  
Neural Network Architecture ◽  
The Neural Network ◽  
Extraction Algorithm ◽  
Training Examples

A method for extracting Zadeh–Mamdani fuzzy rules from a minimalist constructive neural network model is described. The network contains no embedded fuzzy logic elements. The rule extraction algorithm needs no modification of the neural network architecture. No modification of the network learning algorithm is required, nor is it necessary to retain any training examples. The algorithm is illustrated on two well known benchmark data sets and compared with a relevant existing rule extraction algorithm.

scholarly journals Weighted Bagging in Decision Trees: Data Mining

2020 ◽  
Vol 1 (1) ◽  
pp. 1-14
Author(s):  
Yousef Elgimati
Keyword(s):  
Decision Trees ◽  
Learning Algorithm ◽  
Majority Vote ◽  
Random Noise ◽  
Real Data ◽  
Data Sets ◽  
Bootstrap Sample ◽  
Training Set ◽  
Multiple Classifier ◽  
Better Than

The main focus of this paper is on the use of resampling techniques to construct predictive models from data and the goal is to identify the best possible model which can produce better predications. Bagging or Bootstrap aggregating is a general method for improving the performance of given learning algorithm by using a majority vote to combine multiple classifier outputs derived from a single classifier on a bootstrap resample version of a training set. A bootstrap sample is generated by a random sample with replacement from the original training set. Inspired by the idea of bagging, we present an improved method based on a distance function in decision trees, called modified bagging (or weighted Bagging) in this study. The experimental results show that modified bagging is superior to the usual majority vote. These results are confirmed by both real data and artificial data sets with random noise. The Modified bagged classifier performs significantly better than usual bagging on various tree levels for all sample sizes. An interesting observation is that the weighted bagging performs somewhat better than usual bagging with sumps.

scholarly journals ONLINE MODIFICATION OF THE METHOD OF X-MEDIUM ON THE BASIS OF ANSAMBLY OF SELORGANIZED MAP T. KOHONEN

2017 ◽  
pp. 96-107
Author(s):  
Є.В. БОДЯНСЬКИЙ ◽  
А.О. ДЕЙНЕКО ◽  
П.Є. ЖЕРНОВА ◽  
В.О. РЄПІН
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Data Sets ◽  
Self Organizing Maps ◽  
Online Mode ◽  
Wide Range ◽  
Parallel Mode ◽  
Computational Simplicity ◽  
Self Learning

The modified X-means method for clustering in the case when observations are sequentially fed to processing the proposed. This approach’s based on the ensemble of the clustering neural networks, proposed ensemble contains the T. Kohonen’s self-organizing maps. Each of the clustering neural networks consist of different number of neurons, where number of clusters is connected with the quality of there neurons. All ensemble members process information that siquentionally is fed to the system in the parallel mode. The effectiveness of clustering process is determined using Caliński-Harabasz index. The self-learning algorithm uses similarity measure of special type that. The feature of proposed method is absent of the competition step, i.e. neuron-winner is not determined. A number of experiments has been held in order to investigate the proposed system’s properties. Experimental results have proven the fact that the system under consideration could be used to solve a wide range of Data Mining tasks when data sets are processed in an online mode. The proposed ensemble system provides computational simplicity, and data sets are pro-cessed faster due to the possibility of parallel tuning.

scholarly journals The Stock Exchange Prediction using Machine Learning Techniques: A Comprehensive and Systematic Literature Review

2021 ◽  
Vol 14 (2) ◽  
pp. 91-112
Author(s):  
Rico Bayu Wiranata ◽  
Arif Djunaidy
Keyword(s):  
Literature Review ◽  
Learning Algorithm ◽  
Stock Exchange ◽  
Majority Vote ◽  
Estimation Method ◽  
Small Error ◽  
Machine Learning Techniques ◽  
Data Sets ◽  
Stock Prediction ◽  
Data Set

This literature review identifies and analyzes research topic trends, types of data sets, learning algorithm, methods improvements, and frameworks used in stock exchange prediction. A total of 81 studies were investigated, which were published regarding stock predictions in the period January 2015 to June 2020 which took into account the inclusion and exclusion criteria. The literature review methodology is carried out in three major phases: review planning, implementation, and report preparation, in nine steps from defining systematic review requirements to presentation of results. Estimation or regression, clustering, association, classification, and preprocessing analysis of data sets are the five main focuses revealed in the main study of stock prediction research. The classification method gets a share of 35.80% from related studies, the estimation method is 56.79%, data analytics is 4.94%, the rest is clustering and association is 1.23%. Furthermore, the use of the technical indicator data set is 74.07%, the rest are combinations of datasets. To develop a stock prediction model 48 different methods have been applied, 9 of the most widely applied methods were identified. The best method in terms of accuracy and also small error rate such as SVM, DNN, CNN, RNN, LSTM, bagging ensembles such as RF, boosting ensembles such as XGBoost, ensemble majority vote and the meta-learner approach is ensemble Stacking. Several techniques are proposed to improve prediction accuracy by combining several methods, using boosting algorithms, adding feature selection and using parameter and hyper-parameter optimization.

IMPROVING SUPERVISED LEARNING BY ADAPTING THE PROBLEM TO THE LEARNER

2009 ◽  
Vol 19 (01) ◽  
pp. 1-9 ◽  
Author(s):  
JOSHUA MENKE ◽  
TONY MARTINEZ
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Supervised Learning ◽  
Learning Algorithm ◽  
Training Data ◽  
Data Sets ◽  
Proof Of Concept ◽  
Original Function ◽  
Equivalent Function ◽  
Training Examples

While no supervised learning algorithm can do well over all functions, we show that it may be possible to adapt a given function to a given supervised learning algorithm so as to allow the learning algorithm to better classify the original function. Although this seems counterintuitive, adapting the problem to the learner may result in an equivalent function that is "easier" for the algorithm to learn. One method of adapting a problem to the learner is to relabel the targets given in the training data. The following presents two problem adaptation methods, SOL-CTR-E and SOL-CTR-P, variants of Self-Oracle Learning with Confidence-based Target Relabeling (SOL-CTR) as a proof of concept for problem adaptation. The SOL-CTR methods produce "easier" target functions for training artificial neural networks (ANNs). Applying SOL-CTR over 41 data sets consistently results in a statistically significant (p < 0.05) improvement in accuracy over 0/1 targets on data sets containing over 10,000 training examples.

scholarly journals Robust Semi-Supervised Manifold Learning Algorithm for Classification

2018 ◽  
Vol 2018 ◽  
pp. 1-8 ◽  
Author(s):  
Mingxia Chen ◽  
Jing Wang ◽  
Xueqing Li ◽  
Xiaolong Sun
Keyword(s):  
Manifold Learning ◽  
Optimization Model ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Data Sets ◽  
Regularization Term ◽  
Partially Labeled Data ◽  
Label Information ◽  
Low Dimensional ◽  
The Impact

In the recent years, manifold learning methods have been widely used in data classification to tackle the curse of dimensionality problem, since they can discover the potential intrinsic low-dimensional structures of the high-dimensional data. Given partially labeled data, the semi-supervised manifold learning algorithms are proposed to predict the labels of the unlabeled points, taking into account label information. However, these semi-supervised manifold learning algorithms are not robust against noisy points, especially when the labeled data contain noise. In this paper, we propose a framework for robust semi-supervised manifold learning (RSSML) to address this problem. The noisy levels of the labeled points are firstly predicted, and then a regularization term is constructed to reduce the impact of labeled points containing noise. A new robust semi-supervised optimization model is proposed by adding the regularization term to the traditional semi-supervised optimization model. Numerical experiments are given to show the improvement and efficiency of RSSML on noisy data sets.

Operon Prediction Based On an Iterative Self-learning Algorithm*

2011 ◽  
Vol 38 (7) ◽  
pp. 642-651
Author(s):  
Wen-Qi Wu ◽  
Xiao-Bin ZHENG ◽  
Yong-Chu LIU ◽  
Kai TANG ◽  
Huai-Qiu ZHU
Keyword(s):  
Learning Algorithm ◽  
Operon Prediction ◽  
Self Learning

scholarly journals Predictive Modelling of Employee Turnover in Indian IT Industry Using Machine Learning Techniques

2019 ◽  
Vol 23 (1) ◽  
pp. 12-21 ◽  
Author(s):  
Shikha N. Khera ◽  
Divya
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Confusion Matrix ◽  
Predictive Modelling ◽  
Supervised Machine Learning ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
It Industry ◽  
Knowledge Based ◽  
Employee Attrition

Information technology (IT) industry in India has been facing a systemic issue of high attrition in the past few years, resulting in monetary and knowledge-based loses to the companies. The aim of this research is to develop a model to predict employee attrition and provide the organizations opportunities to address any issue and improve retention. Predictive model was developed based on supervised machine learning algorithm, support vector machine (SVM). Archival employee data (consisting of 22 input features) were collected from Human Resource databases of three IT companies in India, including their employment status (response variable) at the time of collection. Accuracy results from the confusion matrix for the SVM model showed that the model has an accuracy of 85 per cent. Also, results show that the model performs better in predicting who will leave the firm as compared to predicting who will not leave the company.

Sign in / Sign up

Export Citation Format

Share Document