SVM Soft Margin Classifiers: Linear Programming versus Quadratic Programming

2005 ◽  
Vol 17 (5) ◽  
pp. 1160-1187 ◽  
Author(s):  
Qiang Wu ◽  
Ding-Xuan Zhou
Keyword(s):  
Linear Programming ◽  
Quadratic Programming ◽  
Optimization Problems ◽  
Large Data ◽  
Misclassification Error ◽  
Support Vector ◽  
Svm Classifier ◽  
Classification Problems ◽  
Noise Condition ◽  
Soft Margin

Support vector machine (SVM) soft margin classifiers are important learning algorithms for classification problems. They can be stated as convex optimization problems and are suitable for a large data setting. Linear programming SVM classifiers are especially efficient for very large size samples. But little is known about their convergence, compared with the well-understood quadratic programming SVM classifier. In this article, we point out the difficulty and provide an error analysis. Our analysis shows that the convergence behavior of the linear programming SVM is almost the same as that of the quadratic programming SVM. This is implemented by setting a stepping-stone between the linear programming SVM and the classical 1-norm soft margin classifier. An upper bound for the misclassification error is presented for general probability distributions. Explicit learning rates are derived for deterministic and weakly separable distributions, and for distributions satisfying some Tsybakov noise condition.

scholarly journals Randomized kernel methods for least-squares support vector machines

2017 ◽  
Vol 28 (02) ◽  
pp. 1750015 ◽  
Author(s):  
M. Andrecut
Keyword(s):  
Least Squares ◽  
Kernel Method ◽  
Large Data ◽  
Large Data Sets ◽  
Support Vector ◽  
Svm Classifier ◽  
Data Sets ◽  
Classification Problems ◽  
Vector Machines ◽  
Multi Class Classification

The least-squares support vector machine (LS-SVM) is a frequently used kernel method for non-linear regression and classification tasks. Here we discuss several approximation algorithms for the LS-SVM classifier. The proposed methods are based on randomized block kernel matrices, and we show that they provide good accuracy and reliable scaling for multi-class classification problems with relatively large data sets. Also, we present several numerical experiments that illustrate the practical applicability of the proposed methods.

Shape-restricted support vector machine (SR-SVM): a SVM classifier taking supplementary shape information of input

2021 ◽  
Vol 40 (1) ◽  
pp. 1481-1494
Author(s):  
Geng Deng ◽  
Yaoguo Xie ◽  
Xindong Wang ◽  
Qiang Fu
Keyword(s):  
Support Vector Machine ◽  
Classification Performance ◽  
Research Literature ◽  
Support Vector ◽  
Svm Classifier ◽  
Classification Problems ◽  
Active Set ◽  
Shape Information ◽  
Convex Optimization Problem ◽  
Shape Restrictions

Many classification problems contain shape information from input features, such as monotonic, convex, and concave. In this research, we propose a new classifier, called Shape-Restricted Support Vector Machine (SR-SVM), which takes the component-wise shape information to enhance classification accuracy. There exists vast research literature on monotonic classification covering monotonic or ordinal shapes. Our proposed classifier extends to handle convex and concave types of features, and combinations of these types. While standard SVM uses linear separating hyperplanes, our novel SR-SVM essentially constructs non-parametric and nonlinear separating planes subject to component-wise shape restrictions. We formulate SR-SVM classifier as a convex optimization problem and solve it using an active-set algorithm. The approach applies basis function expansions on the input and effectively utilizes the standard SVM solver. We illustrate our methodology using simulation and real world examples, and show that SR-SVM improves the classification performance with additional shape information of input.

scholarly journals Correlation Kernels for Support Vector Machines Classification with Applications in Cancer Data

2012 ◽  
Vol 2012 ◽  
pp. 1-7 ◽  
Author(s):  
Hao Jiang ◽  
Wai-Ki Ching
Keyword(s):  
Support Vector Machines ◽  
Positive Semidefinite ◽  
Superior Performance ◽  
Support Vector ◽  
Svm Classifier ◽  
Classification Problems ◽  
Correlation Kernel ◽  
Cancer Data ◽  
Tumor Tissues ◽  
Vector Machines

High dimensional bioinformatics data sets provide an excellent and challenging research problem in machine learning area. In particular, DNA microarrays generated gene expression data are of high dimension with significant level of noise. Supervised kernel learning with an SVM classifier was successfully applied in biomedical diagnosis such as discriminating different kinds of tumor tissues. Correlation Kernel has been recently applied to classification problems with Support Vector Machines (SVMs). In this paper, we develop a novel and parsimonious positive semidefinite kernel. The proposed kernel is shown experimentally to have better performance when compared to the usual correlation kernel. In addition, we propose a new kernel based on the correlation matrix incorporating techniques dealing with indefinite kernel. The resulting kernel is shown to be positive semidefinite and it exhibits superior performance to the two kernels mentioned above. We then apply the proposed method to some cancer data in discriminating different tumor tissues, providing information for diagnosis of diseases. Numerical experiments indicate that our method outperforms the existing methods such as the decision tree method and KNN method.

A Novel Classification Method of Fish Based on Multi-Feature Fusion

2015 ◽  
Vol 713-715 ◽  
pp. 1513-1519 ◽  
Author(s):  
Wei Dong Du ◽  
Bao Wei Chen ◽  
Hai Sen Li ◽  
Chao Xu
Keyword(s):  
Discrete Cosine Transform ◽  
Feature Fusion ◽  
Acoustic Scattering ◽  
Scattering Data ◽  
Support Vector ◽  
Svm Classifier ◽  
Correct Identification ◽  
Classification Problems ◽  
Cosine Transform ◽  
Identification Rate

In order to solve fish classification problems based on acoustic scattering data, temporal centroid (TC) features and discrete cosine transform (DCT) coefficients features used to analyze acoustic scattering characteristics of fish from different aspects are extracted. The extracted features of fish are reduced in dimension and fused, and support vector machine (SVM) classifier is used to classify and identify the fishes. Three kinds of different fishes are selected as research objects in this paper, the correct identification rates are given based on temporal centroid features and discrete cosine transform coefficients features and fused features. The processing results of actual experimental data show that multi-feature fusion method can improve the identification rate at about 5% effectively.

scholarly journals EXTRACTION OF BUILT-UP AREA BY COMBINING TEXTURAL FEATURES AND SPECTRAL INDICES FROM LANDSAT-8 MULTISPECTRAL IMAGE

2018 ◽  
Vol XLII-5 ◽  
pp. 727-733 ◽  
Author(s):  
V. S. Bramhe ◽  
S. K. Ghosh ◽  
P. K. Garg
Keyword(s):  
Correlation Energy ◽  
Feature Space ◽  
Cost Effective ◽  
Misclassification Error ◽  
Support Vector ◽  
Svm Classifier ◽  
Landsat 8 ◽  
Textural Features ◽  
Spectral Indices ◽  
Spectral Channels

<p><strong>Abstract.</strong> Remote sensing techniques provide efficient and cost-effective approach to monitor the expansion of built-up area, in comparison to other traditional approaches. For extracting built-up class, one of the common approaches is to use spectral and spatial features such as, Normalized Difference Built- up index (NDBI), GLCM texture, Gabor filters etc. However, it is observed that classes such as river soil and fallow land usually mix up with built-up class due to their close spectral similarity. Intermixing of classes have been observed in the classified image when using spectral channels. In this paper, an approach has been proposed which uses urban based spectral indices and textural features to extract built-up areas. Three well known spectral indices i.e. NDBI, Built-up Area Extraction Index (BAEI) and Normalized Difference Bareness Index (NDBai) have been used in this work. Along with spectral indices, local spatial dependency of neighborhood regions is captured using eight GLCM based textural feature, such as, Contrast, Correlation, Energy and Homogeneity etc. for each image band. All textural and spectral indices bands are combined and used for extracting built-up areas using Support Vector Machine (SVM) classifier. Results suggest 4.91% increase in overall accuracy when using texture and spectral indices in comparison with 84.38% overall accuracy achieved when using spectral data only. It is observed that built-up class are more separable in the projected spectral-spatial feature space in comparison to spectral channels. Incorporation of textural features with spectral features reduces the misclassification error and provides results with less salt and pepper noise.</p>

scholarly journals SVM-RFE Based Feature Selection and Taguchi Parameters Optimization for Multiclass SVM Classifier

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Mei-Ling Huang ◽  
Yung-Hsiang Hung ◽  
W. M. Lee ◽  
R. K. Li ◽  
Bo-Ru Jiang
Keyword(s):  
Feature Selection ◽  
Classification Accuracy ◽  
Explanatory Power ◽  
Disease Diagnosis ◽  
Parameters Optimization ◽  
Recursive Feature Elimination ◽  
Support Vector ◽  
Svm Classifier ◽  
Classification Problems ◽  
Class Variable

Recently, support vector machine (SVM) has excellent performance on classification and prediction and is widely used on disease diagnosis or medical assistance. However, SVM only functions well on two-group classification problems. This study combines feature selection and SVM recursive feature elimination (SVM-RFE) to investigate the classification accuracy of multiclass problems for Dermatology and Zoo databases. Dermatology dataset contains 33 feature variables, 1 class variable, and 366 testing instances; and the Zoo dataset contains 16 feature variables, 1 class variable, and 101 testing instances. The feature variables in the two datasets were sorted in descending order by explanatory power, and different feature sets were selected by SVM-RFE to explore classification accuracy. Meanwhile, Taguchi method was jointly combined with SVM classifier in order to optimize parametersCandγto increase classification accuracy for multiclass classification. The experimental results show that the classification accuracy can be more than 95% after SVM-RFE feature selection and Taguchi parameter optimization for Dermatology and Zoo databases.

Design of a Power Aware Systolic Array based Support Vector Machine Classifier

2015 ◽  
pp. 96-138
Author(s):  
Bhaswati Mandal ◽  
Manash Pratim Sarma ◽  
Kandarpa Kumar Sarma
Keyword(s):  
Support Vector Machine ◽  
Systolic Array ◽  
Support Vector ◽  
Svm Classifier ◽  
Classification Problems ◽  
Processing Elements ◽  
Array Architecture ◽  
Multiclass Support Vector Machine ◽  
Kernel Operation

This chapter presents a method for generating binary and multiclass Support Vector Machine (SVM) classifier with multiplierless kernel function. This design provides reduced power, area and reduced cost due to the use of multiplierless kernel operation. Binary SVM classifier classifies two groups of linearly or nonlinearly separable data while the multiclass classification provides classification of three nonlinearly separable data. Here, at first SVM classifier is trained for different classification problems and then the extracted training parameters are used in the testing phase of the same. The dataflow from all the processing elements (PEs) are parallely supported by systolic array. This systolic array architecture provides faster processing of the whole system design.

scholarly journals Multiple graph fusion based on Riemannian geometry for motor imagery classification

2022 ◽  
Author(s):  
Xiaofeng Xie ◽  
Xiaokun Zou ◽  
Tianyou Yu ◽  
Rongnian Tang ◽  
Yao Hou ◽  
...  
Keyword(s):  
Riemannian Manifold ◽  
Motor Imagery ◽  
Classification Performance ◽  
Support Vector ◽  
Svm Classifier ◽  
Classification Problems ◽  
Eeg Signals ◽  
Spatial Covariance ◽  
Bilinear Mapping ◽  
Subject Specific

AbstractIn motor imagery-based brain-computer interfaces (BCIs), the spatial covariance features of electroencephalography (EEG) signals that lie on Riemannian manifolds are used to enhance the classification performance of motor imagery BCIs. However, the problem of subject-specific bandpass frequency selection frequently arises in Riemannian manifold-based methods. In this study, we propose a multiple Riemannian graph fusion (MRGF) model to optimize the subject-specific frequency band for a Riemannian manifold. After constructing multiple Riemannian graphs corresponding to multiple bandpass frequency bands, graph embedding based on bilinear mapping and graph fusion based on mutual information were applied to simultaneously extract the spatial and spectral features of the EEG signals from Riemannian graphs. Furthermore, with a support vector machine (SVM) classifier performed on learned features, we obtained an efficient algorithm, which achieves higher classification performance on various datasets, such as BCI competition IIa and in-house BCI datasets. The proposed methods can also be used in other classification problems with sample data in the form of covariance matrices.

HYBRID DECISION TREE ARCHITECTURE UTILIZING LOCAL SVMs FOR EFFICIENT MULTI-LABEL LEARNING

2013 ◽  
Vol 27 (07) ◽  
pp. 1351004 ◽  
Author(s):  
DEJAN GJORGJEVIKJ ◽  
GJORGJI MADJAROV ◽  
SAŠO DŽEROSKI
Keyword(s):  
Decision Tree ◽  
Text Categorization ◽  
Large Scale ◽  
Semantic Annotation ◽  
Predictive Performance ◽  
Tree Architecture ◽  
Support Vector ◽  
Svm Classifier ◽  
Strong Impact ◽  
Classification Problems

Multi-label learning (MLL) problems abound in many areas, including text categorization, protein function classification, and semantic annotation of multimedia. Issues that severely limit the applicability of many current machine learning approaches to MLL are the large-scale problem, which have a strong impact on the computational complexity of learning. These problems are especially pronounced for approaches that transform MLL problems into a set of binary classification problems for which Support Vector Machines (SVMs) are used. On the other hand, the most efficient approaches to MLL, based on decision trees, have clearly lower predictive performance. We propose a hybrid decision tree architecture, where the leaves do not give multi-label predictions directly, but rather utilize local SVM-based classifiers giving multi-label predictions. A binary relevance architecture is employed in the leaves, where a binary SVM classifier is built for each of the labels relevant to that particular leaf. We use a broad range of multi-label datasets with a variety of evaluation measures to evaluate the proposed method against related and state-of-the-art methods, both in terms of predictive performance and time complexity. Our hybrid architecture on almost every large classification problem outperforms the competing approaches in terms of the predictive performance, while its computational efficiency is significantly improved as a result of the integrated decision tree.

Sign in / Sign up

Export Citation Format

Share Document