scholarly journals Universal consistency of twin support vector machines

2021 ◽  
Author(s):  
Weixia Xu ◽  
Dingjiang Huang ◽  
Shuigeng Zhou
Keyword(s):  
Support Vector Machines ◽  
Sample Size ◽  
General Framework ◽  
Binary Classification ◽  
Classification Problem ◽  
Covering Number ◽  
Support Vector ◽  
Classification Problems ◽  
Vector Machines ◽  
Universal Consistency

AbstractA classification problem aims at constructing a best classifier with the smallest risk. When the sample size approaches infinity, the learning algorithms for a classification problem are characterized by an asymptotical property, i.e., universal consistency. It plays a crucial role in measuring the construction of classification rules. A universal consistent algorithm ensures that the larger the sample size of the algorithm is, the more accurately the distribution of the samples could be reconstructed. Support vector machines (SVMs) are regarded as one of the most important models in binary classification problems. How to effectively extend SVMs to twin support vector machines (TWSVMs) so as to improve performance of classification has gained increasing interest in many research areas recently. Many variants for TWSVMs have been proposed and used in practice. Thus in this paper, we focus on the universal consistency of TWSVMs in a binary classification setting. We first give a general framework for TWSVM classifiers that unifies most of the variants of TWSVMs for binary classification problems. Based on it, we then investigate the universal consistency of TWSVMs. To do this, we give some useful definitions of risk, Bayes risk and universal consistency for TWSVMs. Theoretical results indicate that universal consistency is valid for various TWSVM classifiers under some certain conditions, including covering number, localized covering number and stability. For applications of our general framework, several variants of TWSVMs are considered.

Support Vector Machines

2011 ◽  
pp. 1518-1523
Author(s):  
Cecilio Angulo ◽  
Luis Gonzalez-Abril
Keyword(s):  
Support Vector Machines ◽  
Text Categorization ◽  
Binary Classification ◽  
Support Vector ◽  
Risk Minimization ◽  
Classification Problems ◽  
Microarray Gene Expression ◽  
Vector Machines ◽  
The Sixties ◽  
Learning Machine

Support Vector Machines -- SVMs -- are learning machines, originally designed for bi-classification problems, implementing the well-known Structural Risk Minimization (SRM) inductive principle to obtain good generalization on a limited number of learning patterns (Vapnik, 1998). The optimization criterion for these machines is maximizing the margin between two classes, i.e. the distance between two parallel hyperplanes that split the vectors of each one of the two classes, since larger is the margin separating classes, smaller is the VC dimension of the learning machine, which theoretically ensures a good generalization performance (Vapnik, 1998), as it has been demonstrated in a number of real applications (Cristianini, 2000). In its formulation is applicable the kernel trick, which improves the capacity of these algorithms, learning not being directly performed in the original space of data but in a new space called feature space; for this reason this algorithm is one of the most representative of the called Kernel Machines (KMs). Main theory was originally developed on the sixties and seventies by V. Vapnik and A. Chervonenkis (Vapnik et al., 1963, Vapnik et al., 1971, Vapnik, 1995, Vapnik, 1998), on the basis of a separable binary classification problem, however generalization in the use of these learning algorithms did not take place until the nineties (Boser et al., 1992). SVMs has been used thoroughly in any kind of learning problems, mainly in classification problems, although also in other problems like regression (Schölkopf et al., 2004) or clustering (Ben-Hur et al., 2001). The fields of Optic Character Recognition (Cortes et al., 1995) and Text Categorization (Sebastiani, 2002) were the most important initial applications where SVMs were used. With the extended application of new kernels, novel applications have taken place in the field of Bioinformatics, concretely many works are related with the classification of data in Genetic Expression (Microarray Gene Expression) (Brown et al., 1997) and detecting structures between proteins and their relationship with the chains of DNA (Jaakkola et al., 2000). Other applications include image identification, voice recognition, prediction in time series, etc. A more extensive list of applications can be found in (Guyon, 2006).

REAL-TIME PEDESTRIAN DETECTION USING SUPPORT VECTOR MACHINES

2003 ◽  
Vol 17 (03) ◽  
pp. 405-416 ◽  
Author(s):  
SEONGHOON KANG ◽  
HYERAN BYUN ◽  
SEONG-WHAN LEE
Keyword(s):  
Support Vector Machines ◽  
Real Time ◽  
Face Detection ◽  
Detection Method ◽  
Binary Classification ◽  
Pedestrian Detection ◽  
Classification Problem ◽  
Guidance System ◽  
Support Vector ◽  
Vector Machines

In this paper, we present a real-time pedestrian detection method in outdoor environments. It is necessary for pedestrian detection to implement obstacle and face detection which are major parts of a walking guidance system for the visually impaired. It detects foreground objects on the ground, discriminates pedestrians from other noninterest objects, and extracts candidate regions for face detection and recognition. For effective real-time pedestrian detection, we have developed a method using stereo-based segmentation and the SVM (Support Vector Machines), which works well particularly in binary classification problem (e.g. object detection). We used vertical edge features extracted from arms, legs and torso. In our experiments, test results on a large number of outdoor scenes demonstrated the effectiveness of the proposed pedestrian detection method.

Gaussian Processes for Classification: Mean-Field Algorithms

2000 ◽  
Vol 12 (11) ◽  
pp. 2655-2684 ◽  
Author(s):  
Manfred Opper ◽  
Ole Winther
Keyword(s):  
Support Vector Machines ◽  
Gaussian Processes ◽  
Disordered Systems ◽  
Binary Classification ◽  
Computational Cost ◽  
Mean Field ◽  
Strong Support ◽  
Support Vector ◽  
Vector Machines ◽  
Leave One Out

We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler “naive” mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.

scholarly journals Bankruptcy Prediction of Engineering Companies in the EU Using Classification Methods

2018 ◽  
Vol 66 (5) ◽  
pp. 1347-1356 ◽  
Author(s):  
Michaela Staňková ◽  
David Hampel
Keyword(s):  
Logistic Regression ◽  
Support Vector Machines ◽  
Binary Classification ◽  
Classification Tree ◽  
Classification Trees ◽  
Bankruptcy Prediction ◽  
Support Vector ◽  
Type I ◽  
Vector Machines ◽  
The Eu

This article focuses on the problem of binary classification of 902 small- and medium‑sized engineering companies active in the EU, together with additional 51 companies which went bankrupt in 2014. For classification purposes, the basic statistical method of logistic regression has been selected, together with a representative of machine learning (support vector machines and classification trees method) to construct models for bankruptcy prediction. Different settings have been tested for each method. Furthermore, the models were estimated based on complete data and also using identified artificial factors. To evaluate the quality of prediction we observe not only the total accuracy with the type I and II errors but also the area under ROC curve criterion. The results clearly show that increasing distance to bankruptcy decreases the predictive ability of all models. The classification tree method leads us to rather simple models. The best classification results were achieved through logistic regression based on artificial factors. Moreover, this procedure provides good and stable results regardless of other settings. Artificial factors also seem to be a suitable variable for support vector machines models, but classification trees achieved better results using original data.

scholarly journals On the parameter optimization of Support Vector Machines for binary classification

2012 ◽  
Vol 9 (3) ◽  
pp. 33-43 ◽  
Author(s):  
Paulo Gaspar ◽  
Jaime Carbonell ◽  
José Luís Oliveira
Keyword(s):  
Support Vector Machines ◽  
Binary Classification ◽  
Classification Performance ◽  
Biological Data ◽  
Parameters Optimization ◽  
Support Vector ◽  
Minimal Risk ◽  
Class Separation ◽  
Vector Machines ◽  
Analyse Data

Summary Classifying biological data is a common task in the biomedical context. Predicting the class of new, unknown information allows researchers to gain insight and make decisions based on the available data. Also, using classification methods often implies choosing the best parameters to obtain optimal class separation, and the number of parameters might be large in biological datasets.Support Vector Machines provide a well-established and powerful classification method to analyse data and find the minimal-risk separation between different classes. Finding that separation strongly depends on the available feature set and the tuning of hyper-parameters. Techniques for feature selection and SVM parameters optimization are known to improve classification accuracy, and its literature is extensive.In this paper we review the strategies that are used to improve the classification performance of SVMs and perform our own experimentation to study the influence of features and hyper-parameters in the optimization process, using several known kernels.

scholarly journals Correlation Kernels for Support Vector Machines Classification with Applications in Cancer Data

2012 ◽  
Vol 2012 ◽  
pp. 1-7 ◽  
Author(s):  
Hao Jiang ◽  
Wai-Ki Ching
Keyword(s):  
Support Vector Machines ◽  
Positive Semidefinite ◽  
Superior Performance ◽  
Support Vector ◽  
Svm Classifier ◽  
Classification Problems ◽  
Correlation Kernel ◽  
Cancer Data ◽  
Tumor Tissues ◽  
Vector Machines

High dimensional bioinformatics data sets provide an excellent and challenging research problem in machine learning area. In particular, DNA microarrays generated gene expression data are of high dimension with significant level of noise. Supervised kernel learning with an SVM classifier was successfully applied in biomedical diagnosis such as discriminating different kinds of tumor tissues. Correlation Kernel has been recently applied to classification problems with Support Vector Machines (SVMs). In this paper, we develop a novel and parsimonious positive semidefinite kernel. The proposed kernel is shown experimentally to have better performance when compared to the usual correlation kernel. In addition, we propose a new kernel based on the correlation matrix incorporating techniques dealing with indefinite kernel. The resulting kernel is shown to be positive semidefinite and it exhibits superior performance to the two kernels mentioned above. We then apply the proposed method to some cancer data in discriminating different tumor tissues, providing information for diagnosis of diseases. Numerical experiments indicate that our method outperforms the existing methods such as the decision tree method and KNN method.

Sign in / Sign up

Export Citation Format

Share Document