scholarly journals A Learning Framework of Nonparallel Hyperplanes Classifier

2015 ◽  
Vol 2015 ◽  
pp. 1-12
Author(s):  
Zhi-Xia Yang ◽  
Yuan-Hai Shao ◽  
Yao-Lin Jiang
Keyword(s):  
Binary Classification ◽  
Computational Cost ◽  
Classification Problem ◽  
Multiclass Classification ◽  
Decision Function ◽  
Support Vector ◽  
Learning Framework ◽  
Hinge Loss ◽  
Benchmark Datasets ◽  
Nonparallel Hyperplanes

A novel learning framework of nonparallel hyperplanes support vector machines (NPSVMs) is proposed for binary classification and multiclass classification. This framework not only includes twin SVM (TWSVM) and its many deformation versions but also extends them into multiclass classification problem when different parameters or loss functions are chosen. Concretely, we discuss the linear and nonlinear cases of the framework, in which we select the hinge loss function as example. Moreover, we also give the primal problems of several extension versions of TWSVM’s deformation versions. It is worth mentioning that, in the decision function, the Euclidean distance is replaced by the absolute value|wTx+b|, which keeps the consistency between the decision function and the optimization problem and reduces the computational cost particularly when the kernel function is introduced. The numerical experiments on several artificial and benchmark datasets indicate that our framework is not only fast but also shows good generalization.

Multiple sub-hyper-spheres support vector machine for multi-class classification

2014 ◽  
Vol 12 (03) ◽  
pp. 1450035 ◽  
Author(s):  
Shuang Liu ◽  
Peng Chen ◽  
Keqiu Li
Keyword(s):  
Support Vector Machine ◽  
Binary Classification ◽  
Decision Rules ◽  
Classification Problem ◽  
Decision Function ◽  
Support Vector ◽  
Similarity Function ◽  
Data Set ◽  
Computation Cost ◽  
Multi Class Classification

Support vector machine (SVM) is originally proposed to solve binary classification problem. Multi-class classification is solved by combining multiple binary classifiers, which leads to high computation cost by introducing many quadratic programming (QP) problems. To decrease computation cost, hyper-sphere SVM is put forward to compute class-specific hyper-sphere for each class. If all resulting hyper-spheres are independent, all training and test samples can be correctly classified. When some of hyper-spheres intersect, new decision rules should be adopted. To solve this problem, a multiple sub-hyper-sphere SVM is put forward in this paper. New algorithm computed hyper-spheres by SMO algorithm for all classes first, and then obtained position relationships between hyper-spheres. If hyper-spheres belong to the intersection set, overlap coefficient is computed based on map of key value index and mother hyper-spheres are partitioned into a series of sub-hyper-spheres. For the new intersecting hyper-spheres, one similarity function or same error sub-hyper-sphere or different error sub-hyper-sphere are used as decision rule. If hyper-spheres belong to the inclusion set, the hyper-sphere with larger radius is partitioned into sub-hyper-spheres. If hyper-spheres belong to the independence set, a decision function is defined for classification. With experimental results compared to other hyper-sphere SVMs, our new proposed algorithm improves the performance of the resulting classifier and decreases computation complexity for decision on both artificial and benchmark data set.

Gaussian Processes for Classification: Mean-Field Algorithms

2000 ◽  
Vol 12 (11) ◽  
pp. 2655-2684 ◽  
Author(s):  
Manfred Opper ◽  
Ole Winther
Keyword(s):  
Support Vector Machines ◽  
Gaussian Processes ◽  
Disordered Systems ◽  
Binary Classification ◽  
Computational Cost ◽  
Mean Field ◽  
Strong Support ◽  
Support Vector ◽  
Vector Machines ◽  
Leave One Out

We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler “naive” mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.

STRUCTURE-EMBEDDED AUC-SVM

2010 ◽  
Vol 24 (05) ◽  
pp. 667-690 ◽  
Author(s):  
YUNYUN WANG ◽  
SONGCAN CHEN ◽  
HUI XUE
Keyword(s):  
Computational Complexity ◽  
Training Sample ◽  
Decision Function ◽  
Information Loss ◽  
Global Structure ◽  
Support Vector ◽  
Structure Information ◽  
Hinge Loss ◽  
Novel Structure ◽  
High Computational Complexity

AUC-SVM directly maximizes the area under the ROC curve (AUC) through minimizing its hinge loss relaxation, and the decision function is determined by those support vector sample pairs playing the same roles as the support vector samples in SVM. Such a learning paradigm generally emphasizes more on the local discriminative information just associated with these support vectors whereas hardly takes the overall view of data into account, thereby it may incur loss of the global distribution information in data favorable for classification. Moreover, due to the high computational complexity of AUC-SVM induced by the large number of training sample pairs quadratic in the number of samples, sampling is usually adopted, incurring a further loss of the distribution information in data. In order to compensate the distribution information loss and simultaneously boost the AUC-SVM performance, in this paper, we develop a novel structure-embedded AUC-SVM (SAUC-SVM for short) through embedding the global structure information in the whole data into AUC-SVM. With such an embedding, the proposed SAUC-SVM incorporates the local discriminative information and global structure information in data into a uniform formulation and consequently guarantees better generalization performance. Comparative experiments on both synthetic and real datasets confirm its effectiveness.

scholarly journals An Ensemble Learning Based Framework for Traditional Chinese Medicine Data Analysis with ICD-10 Labels

2015 ◽  
Vol 2015 ◽  
pp. 1-8
Author(s):  
Gang Zhang ◽  
Yonghui Huang ◽  
Ling Zhong ◽  
Shanxing Ou ◽  
Yi Zhang ◽  
...  
Keyword(s):  
Ensemble Learning ◽  
Learning Strategy ◽  
Computational Cost ◽  
Training Dataset ◽  
Support Vector ◽  
Learning Framework ◽  
Analysis Task ◽  
Icd 10 ◽  
Effective Diagnosis ◽  
Clinical Records

Objective.This study aims to establish a model to analyze clinical experience of TCM veteran doctors. We propose an ensemble learning based framework to analyze clinical records with ICD-10 labels information for effective diagnosis and acupoints recommendation.Methods.We propose an ensemble learning framework for the analysis task. A set of base learners composed of decision tree (DT) and support vector machine (SVM) are trained by bootstrapping the training dataset. The base learners are sorted by accuracy and diversity through nondominated sort (NDS) algorithm and combined through a deep ensemble learning strategy.Results.We evaluate the proposed method with comparison to two currently successful methods on a clinical diagnosis dataset with manually labeled ICD-10 information. ICD-10 label annotation and acupoints recommendation are evaluated for three methods. The proposed method achieves an accuracy rate of 88.2%  ±  2.8% measured by zero-one loss for the first evaluation session and 79.6%  ±  3.6% measured by Hamming loss, which are superior to the other two methods.Conclusion.The proposed ensemble model can effectively model the implied knowledge and experience in historic clinical data records. The computational cost of training a set of base learners is relatively low.

MINIMAL CLASSIFICATION METHOD WITH ERROR-CORRECTING CODES FOR MULTICLASS RECOGNITION

2005 ◽  
Vol 19 (05) ◽  
pp. 663-680 ◽  
Author(s):  
DAYAN MANOHAR SIVALINGAM ◽  
NARENKUMAR PANDIAN ◽  
JEZEKIEL BEN-ARIE
Keyword(s):  
Binary Classification ◽  
Classification Problem ◽  
Error Correcting Codes ◽  
Classification Method ◽  
Support Vector ◽  
Object Image ◽  
Generalization Error ◽  
Image Library ◽  
Binary Classification Problem ◽  
Binary Classifiers

In this work, we develop an efficient technique to transform a multiclass recognition problem into a minimal binary classification problem using the Minimal Classification Method (MCM). The MCM requires only log 2 N classifications whereas the other methods require much more. For the classification, we use Support Vector Machine (SVM) based binary classifiers since they have superior generalization performance. Unlike the prevalent one-versus-one strategy (the bottom-up one-versus-one strategy is called tournament method) that separates only two classes at each classification, the binary classifiers in our method have to separate two groups of multiple classes. As a result, the probability of generalization error increases. This problem is alleviated by utilizing error correcting codes, which results only in a marginal increase in the required number of classifications. However, in comparison to the tournament method, our method requires only 50% of the classifications and still similar performance can be attained. The proposed solution is tested with the Columbia Object Image Library (COIL). We also test the performance under conditions of noise and occlusion.

scholarly journals Predicting Takeover Success Using Machine Learning Techniques

2012 ◽  
Vol 10 (10) ◽  
pp. 547
Author(s):  
Mei Zhang ◽  
Gregory Johnson ◽  
Jia Wang
Keyword(s):  
Machine Learning ◽  
Learning Community ◽  
Binary Classification ◽  
Classification Problem ◽  
Machine Learning Techniques ◽  
Success Prediction ◽  
Support Vector ◽  
Font Size ◽  
Network Support ◽  
Learning Techniques

<span style="font-family: Times New Roman; font-size: small;"> </span><p style="margin: 0in 0.5in 0pt; text-align: justify; mso-pagination: none; mso-layout-grid-align: none;" class="MsoNormal"><span style="color: black; font-size: 10pt; mso-themecolor: text1;"><span style="font-family: Times New Roman;">A takeover success prediction model aims at predicting the probability that a takeover attempt will succeed by using publicly available information at the time of the announcement.<span style="mso-spacerun: yes;"> </span>We perform a thorough study using machine learning techniques to predict takeover success.<span style="mso-spacerun: yes;"> </span>Specifically, we model takeover success prediction as a binary classification problem, which has been widely studied in the machine learning community.<span style="mso-spacerun: yes;"> </span>Motivated by the recent advance in machine learning, we empirically evaluate and analyze many state-of-the-art classifiers, including logistic regression, artificial neural network, support vector machines with different kernels, decision trees, random forest, and Adaboost.<span style="mso-spacerun: yes;"> </span>The experiments validate the effectiveness of applying machine learning in takeover success prediction, and we found that the support vector machine with linear kernel and the Adaboost with stump weak classifiers perform the best for the task.<span style="mso-spacerun: yes;"> </span>The result is consistent with the general observations of these two approaches.</span></span></p><span style="font-family: Times New Roman; font-size: small;"> </span>

scholarly journals Hierarchical Multi Twin Support Vector Machine

2021 ◽  
Vol 130 (2B) ◽  
Author(s):  
Nguyen The Cuong
Keyword(s):  
Support Vector Machine ◽  
Structural Information ◽  
Binary Classification ◽  
Classification Problem ◽  
Data Type ◽  
Twin Support Vector Machine ◽  
Support Vector ◽  
Classification Problems ◽  
Data Simulation ◽  
Better Than

In binary classification problems, two classes normally have different tendencies. More complex, the clusters in each class also tend to be different. Traditional algorithms as Support Vector Machine (SVM) or Twin Support Vector Machine (TWSVM) don't sufficiently exploit structural information with cluster granularity of the data, cause of restricts the capability of simulation of data trends. Structural twin support vector machine (S-TWSVM) sufficiently exploits structural information with cluster granularity of one class for learning a represented hyperplane of that class. This makes S-TWSVM's data simulation capabilities better than TWSVM. However, for the data type that each class consists of clusters of different trends, the capability of simulation of S-TWSVM is restricted. In this paper, we propose a new Hierarchical Multi Twin Support Vector Machine (called HM-TWSVM) for classification problem with each cluster-vs-class strategy. HM-TWSVM overcomes the limitations of S-TWSVM. Experiment results show that HM-TWSVM could describe the tendency of each cluster.

scholarly journals Predicting Anesthetic Infusion Events Using Machine Learning

2021 ◽  
Author(s):  
Naoki Miyaguchi ◽  
Koh Takeuchi ◽  
Hisashi Kashima ◽  
Mizuki Morita ◽  
Hiroshi Morimatsu
Keyword(s):  
Machine Learning ◽  
Flow Rate ◽  
Short Term Memory ◽  
Binary Classification ◽  
Classification Problem ◽  
Clinical Findings ◽  
Support Vector ◽  
Learning Models ◽  
Continuous Administration ◽  
Machine Learning Models

Abstract Recently, research has been conducted to automatically control anesthesia using machine learning, with the aim of alleviating the shortage of anesthesiologists. In this study, we address the problem of predicting decisions made by anesthesiologists during surgery using machine learning; specifically, we formulate a decision making problem by increasing the flow rate at each time point in the continuous administration of analgesic remifentanil as a supervised binary classification problem. The experiments were conducted to evaluate the prediction performance using six machine learning models: logistic regression, support vector machine, random forest, LightGBM, artificial neural network, and long short-term memory (LSTM), using 210 case data collected during actual surgeries. The results demonstrated that when predicting the future increase in flow rate of remifentanil after 1 min, the model using LSTM was able to predict with scores of 0.659 for sensitivity, 0.732 for specificity, and 0.753 for ROC-AUC; this demonstrates the potential to predict the decisions made by anesthesiologists using machine learning. Furthermore, we examined the importance and contribution of the features of each model using shapley additive explanations—a method for interpreting predictions made by machine learning models. The trends indicated by the results were partially consistent with known clinical findings.

scholarly journals An SVM-Based Framework for Long-Term Learning Systems

2019 ◽  
Vol 33 ◽  
pp. 9915-9916
Author(s):  
Diana Benavides-Prado
Keyword(s):  
Binary Classification ◽  
Upper Bounds ◽  
Support Vector ◽  
Transfer Of Knowledge ◽  
Learning Framework ◽  
Vector Machines ◽  
Novel Method ◽  
Classification Tasks ◽  
Retention Of Knowledge

In our research, we study the problem of learning a sequence of supervised tasks. This is a long-standing challenge in machine learning. Our work relies on transfer of knowledge between hypotheses learned with Support Vector Machines. Transfer occurs in two directions: forward and backward. We have proposed to selectively transfer forward support vector coefficients from previous hypotheses as upper-bounds on support vector coefficients to be learned on a target task. We also proposed a novel method for refining existing hypotheses by transferring backward knowledge from a target hypothesis learned recently. We have improved this method through a hypothesis refinement approach that refines whilst encouraging retention of knowledge. Our contribution is represented in a long-term learning framework for binary classification tasks received sequentially one at a time.

Sign in / Sign up

Export Citation Format

Share Document