scholarly journals Robust relative margin support vector machines

2016 ◽  
Vol 11 (2) ◽  
pp. 186-191 ◽  
Author(s):  
Yunyan Song ◽  
Wenxin Zhu ◽  
Yingyuan Xiao ◽  
Ping Zhong
Keyword(s):  
Computational Complexity ◽  
Binary Classification ◽  
Support Vector ◽  
Decision Boundary ◽  
Large Margin ◽  
Hinge Loss ◽  
Shortest Distance ◽  
Vector Machines ◽  
Pinball Loss ◽  
Real World Problems

Recently, a class of classifiers, called relative margin machine, has been developed. Relative margin machine has shown significant improvements over the large margin counterparts on real-world problems. In binary classification, the most widely used loss function is the hinge loss, which results in the hinge loss relative margin machine. The hinge loss relative margin machine is sensitive to outliers. In this article, we proposed to change maximizing the shortest distance used in relative margin machine into maximizing the quantile distance, the pinball loss which is related to quantiles was used in classification. The proposed method is less sensitive to noise, especially the feature noise around the decision boundary. Meanwhile, the computational complexity of the proposed method is similar to that of the relative margin machine.

scholarly journals Robust Support Vector Machines for Classification with Nonconvex and Smooth Losses

2016 ◽  
Vol 28 (6) ◽  
pp. 1217-1247 ◽  
Author(s):  
Yunlong Feng ◽  
Yuning Yang ◽  
Xiaolin Huang ◽  
Siamak Mehrkanoon ◽  
Johan A. K. Suykens
Keyword(s):  
Support Vector Machines ◽  
Robust Statistics ◽  
Real Data ◽  
Constructive Proof ◽  
Support Vector ◽  
Data Sets ◽  
Large Margin ◽  
Hinge Loss ◽  
Vector Machines ◽  
Smooth Classification

This letter addresses the robustness problem when learning a large margin classifier in the presence of label noise. In our study, we achieve this purpose by proposing robustified large margin support vector machines. The robustness of the proposed robust support vector classifiers (RSVC), which is interpreted from a weighted viewpoint in this work, is due to the use of nonconvex classification losses. Besides the robustness, we also show that the proposed RSCV is simultaneously smooth, which again benefits from using smooth classification losses. The idea of proposing RSVC comes from M-estimation in statistics since the proposed robust and smooth classification losses can be taken as one-sided cost functions in robust statistics. Its Fisher consistency property and generalization ability are also investigated. Besides the robustness and smoothness, another nice property of RSVC lies in the fact that its solution can be obtained by solving weighted squared hinge loss–based support vector machine problems iteratively. We further show that in each iteration, it is a quadratic programming problem in its dual space and can be solved by using state-of-the-art methods. We thus propose an iteratively reweighted type algorithm and provide a constructive proof of its convergence to a stationary point. Effectiveness of the proposed classifiers is verified on both artificial and real data sets.

Pinball Loss Twin Support Vector Clustering

2021 ◽  
Vol 17 (2s) ◽  
pp. 1-23
Author(s):  
M. Tanveer ◽  
Tarun Gupta ◽  
Miten Shah ◽  
Keyword(s):  
Loss Function ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Structural Mri ◽  
Twin Support Vector Machine ◽  
Support Vector ◽  
Support Vector Clustering ◽  
Hinge Loss ◽  
Pinball Loss ◽  
Vector Clustering

Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and artificial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noise-corrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model.

Gaussian Processes for Classification: Mean-Field Algorithms

2000 ◽  
Vol 12 (11) ◽  
pp. 2655-2684 ◽  
Author(s):  
Manfred Opper ◽  
Ole Winther
Keyword(s):  
Support Vector Machines ◽  
Gaussian Processes ◽  
Disordered Systems ◽  
Binary Classification ◽  
Computational Cost ◽  
Mean Field ◽  
Strong Support ◽  
Support Vector ◽  
Vector Machines ◽  
Leave One Out

We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler “naive” mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.

scholarly journals Bankruptcy Prediction of Engineering Companies in the EU Using Classification Methods

2018 ◽  
Vol 66 (5) ◽  
pp. 1347-1356 ◽  
Author(s):  
Michaela Staňková ◽  
David Hampel
Keyword(s):  
Logistic Regression ◽  
Support Vector Machines ◽  
Binary Classification ◽  
Classification Tree ◽  
Classification Trees ◽  
Bankruptcy Prediction ◽  
Support Vector ◽  
Type I ◽  
Vector Machines ◽  
The Eu

This article focuses on the problem of binary classification of 902 small- and medium‑sized engineering companies active in the EU, together with additional 51 companies which went bankrupt in 2014. For classification purposes, the basic statistical method of logistic regression has been selected, together with a representative of machine learning (support vector machines and classification trees method) to construct models for bankruptcy prediction. Different settings have been tested for each method. Furthermore, the models were estimated based on complete data and also using identified artificial factors. To evaluate the quality of prediction we observe not only the total accuracy with the type I and II errors but also the area under ROC curve criterion. The results clearly show that increasing distance to bankruptcy decreases the predictive ability of all models. The classification tree method leads us to rather simple models. The best classification results were achieved through logistic regression based on artificial factors. Moreover, this procedure provides good and stable results regardless of other settings. Artificial factors also seem to be a suitable variable for support vector machines models, but classification trees achieved better results using original data.

scholarly journals On the parameter optimization of Support Vector Machines for binary classification

2012 ◽  
Vol 9 (3) ◽  
pp. 33-43 ◽  
Author(s):  
Paulo Gaspar ◽  
Jaime Carbonell ◽  
José Luís Oliveira
Keyword(s):  
Support Vector Machines ◽  
Binary Classification ◽  
Classification Performance ◽  
Biological Data ◽  
Parameters Optimization ◽  
Support Vector ◽  
Minimal Risk ◽  
Class Separation ◽  
Vector Machines ◽  
Analyse Data

Summary Classifying biological data is a common task in the biomedical context. Predicting the class of new, unknown information allows researchers to gain insight and make decisions based on the available data. Also, using classification methods often implies choosing the best parameters to obtain optimal class separation, and the number of parameters might be large in biological datasets.Support Vector Machines provide a well-established and powerful classification method to analyse data and find the minimal-risk separation between different classes. Finding that separation strongly depends on the available feature set and the tuning of hyper-parameters. Techniques for feature selection and SVM parameters optimization are known to improve classification accuracy, and its literature is extensive.In this paper we review the strategies that are used to improve the classification performance of SVMs and perform our own experimentation to study the influence of features and hyper-parameters in the optimization process, using several known kernels.

STRUCTURE-EMBEDDED AUC-SVM

2010 ◽  
Vol 24 (05) ◽  
pp. 667-690 ◽  
Author(s):  
YUNYUN WANG ◽  
SONGCAN CHEN ◽  
HUI XUE
Keyword(s):  
Computational Complexity ◽  
Training Sample ◽  
Decision Function ◽  
Information Loss ◽  
Global Structure ◽  
Support Vector ◽  
Structure Information ◽  
Hinge Loss ◽  
Novel Structure ◽  
High Computational Complexity

AUC-SVM directly maximizes the area under the ROC curve (AUC) through minimizing its hinge loss relaxation, and the decision function is determined by those support vector sample pairs playing the same roles as the support vector samples in SVM. Such a learning paradigm generally emphasizes more on the local discriminative information just associated with these support vectors whereas hardly takes the overall view of data into account, thereby it may incur loss of the global distribution information in data favorable for classification. Moreover, due to the high computational complexity of AUC-SVM induced by the large number of training sample pairs quadratic in the number of samples, sampling is usually adopted, incurring a further loss of the distribution information in data. In order to compensate the distribution information loss and simultaneously boost the AUC-SVM performance, in this paper, we develop a novel structure-embedded AUC-SVM (SAUC-SVM for short) through embedding the global structure information in the whole data into AUC-SVM. With such an embedding, the proposed SAUC-SVM incorporates the local discriminative information and global structure information in data into a uniform formulation and consequently guarantees better generalization performance. Comparative experiments on both synthetic and real datasets confirm its effectiveness.

scholarly journals Learning Complexity vs Communication Complexity

2009 ◽  
Vol 18 (1-2) ◽  
pp. 227-245 ◽  
Author(s):  
NATI LINIAL ◽  
ADI SHRAIBMAN
Keyword(s):  
Communication Complexity ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Polynomial Hierarchy ◽  
Open Problems ◽  
Multiplicative Constant ◽  
Large Margin ◽  
Vector Machines ◽  
Large Margin Classifiers ◽  
Class Of Functions

This paper has two main focal points. We first consider an important class of machine learning algorithms: large margin classifiers, such as Support Vector Machines. The notion of margin complexity quantifies the extent to which a given class of functions can be learned by large margin classifiers. We prove that up to a small multiplicative constant, margin complexity is equal to the inverse of discrepancy. This establishes a strong tie between seemingly very different notions from two distinct areas.In the same way that matrix rigidity is related to rank, we introduce the notion of rigidity of margin complexity. We prove that sign matrices with small margin complexity rigidity are very rare. This leads to the question of proving lower bounds on the rigidity of margin complexity. Quite surprisingly, this question turns out to be closely related to basic open problems in communication complexity, e.g., whether PSPACE can be separated from the polynomial hierarchy in communication complexity.Communication is a key ingredient in many types of learning. This explains the relations between the field of learning theory and that of communication complexity [6, l0, 16, 26]. The results of this paper constitute another link in this rich web of relations. These new results have already been applied toward the solution of several open problems in communication complexity [18, 20, 29].

Sign in / Sign up

Export Citation Format

Share Document