scholarly journals Two New Decomposition Algorithms for Training Bound-Constrained Support Vector Machines*

2015 ◽  
Vol 40 (1) ◽  
pp. 67-86 ◽  
Author(s):  
Lingfeng Niu ◽  
Ruizhi Zhou ◽  
Xi Zhao ◽  
Yong Shi
Keyword(s):  
Binary Classification ◽  
Gradient Algorithm ◽  
Support Vector ◽  
Data Sets ◽  
Decomposition Algorithms ◽  
Public Data ◽  
Vector Machines ◽  
Working Set ◽  
Derivative Information ◽  
New Algorithms

Abstract Bound-constrained Support Vector Machine(SVM) is one of the stateof- art model for binary classification. The decomposition method is currently one of the major methods for training SVMs, especially when the nonlinear kernel is used. In this paper, we proposed two new decomposition algorithms for training bound-constrained SVMs. Projected gradient algorithm and interior point method are combined together to solve the quadratic subproblem effciently. The main difference between the two algorithms is the way of choosing working set. The first one only uses first order derivative information of the model for simplicity. The second one incorporate part of second order information into the process of working set selection, besides the gradient. Both algorithms are proved to be global convergent in theory. New algorithms is compared with the famous package BSVM. Numerical experiments on several public data sets validate the effciency of the proposed methods.

scholarly journals Multi-choice wavelet thresholding based binary classification method

Methodology ◽  
2020 ◽  
Vol 16 (2) ◽  
pp. 127-146 ◽  
Author(s):  
Seung Hyun Baek ◽  
Alberto Garcia-Diaz ◽  
Yuanshun Dai
Keyword(s):  
Data Mining ◽  
Binary Classification ◽  
High Dimensional ◽  
Support Vector ◽  
Published Data ◽  
Data Sets ◽  
Wavelet Thresholding ◽  
Reliability Of Results ◽  
Vector Machines ◽  
Cognition Model

Data mining is one of the most effective statistical methodologies to investigate a variety of problems in areas including pattern recognition, machine learning, bioinformatics, chemometrics, and statistics. In particular, statistically-sophisticated procedures that emphasize on reliability of results and computational efficiency are required for the analysis of high-dimensional data. Optimization principles can play a significant role in the rationalization and validation of specialized data mining procedures. This paper presents a novel methodology which is Multi-Choice Wavelet Thresholding (MCWT) based three-step methodology consists of three processes: perception (dimension reduction), decision (feature ranking), and cognition (model selection). In these steps three concepts known as wavelet thresholding, support vector machines for classification and information complexity are integrated to evaluate learning models. Three published data sets are used to illustrate the proposed methodology. Additionally, performance comparisons with recent and widely applied methods are shown.

Nondegenerate Piecewise Linear Systems: A Finite Newton Algorithm and Applications in Machine Learning

2012 ◽  
Vol 24 (4) ◽  
pp. 1047-1084 ◽  
Author(s):  
Xiao-Tong Yuan ◽  
Shuicheng Yan
Keyword(s):  
Linear Systems ◽  
Optimization Problems ◽  
Piecewise Linear ◽  
Optimization Methods ◽  
Coefficient Matrix ◽  
Learning Problems ◽  
Support Vector ◽  
Data Sets ◽  
Piecewise Linear Systems ◽  
Vector Machines

We investigate Newton-type optimization methods for solving piecewise linear systems (PLSs) with nondegenerate coefficient matrix. Such systems arise, for example, from the numerical solution of linear complementarity problem, which is useful to model several learning and optimization problems. In this letter, we propose an effective damped Newton method, PLS-DN, to find the exact (up to machine precision) solution of nondegenerate PLSs. PLS-DN exhibits provable semiiterative property, that is, the algorithm converges globally to the exact solution in a finite number of iterations. The rate of convergence is shown to be at least linear before termination. We emphasize the applications of our method in modeling, from a novel perspective of PLSs, some statistical learning problems such as box-constrained least squares, elitist Lasso (Kowalski & Torreesani, 2008 ), and support vector machines (Cortes & Vapnik, 1995 ). Numerical results on synthetic and benchmark data sets are presented to demonstrate the effectiveness and efficiency of PLS-DN on these problems.

Gaussian Processes for Classification: Mean-Field Algorithms

2000 ◽  
Vol 12 (11) ◽  
pp. 2655-2684 ◽  
Author(s):  
Manfred Opper ◽  
Ole Winther
Keyword(s):  
Support Vector Machines ◽  
Gaussian Processes ◽  
Disordered Systems ◽  
Binary Classification ◽  
Computational Cost ◽  
Mean Field ◽  
Strong Support ◽  
Support Vector ◽  
Vector Machines ◽  
Leave One Out

We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler “naive” mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.

scholarly journals Bankruptcy Prediction of Engineering Companies in the EU Using Classification Methods

2018 ◽  
Vol 66 (5) ◽  
pp. 1347-1356 ◽  
Author(s):  
Michaela Staňková ◽  
David Hampel
Keyword(s):  
Logistic Regression ◽  
Support Vector Machines ◽  
Binary Classification ◽  
Classification Tree ◽  
Classification Trees ◽  
Bankruptcy Prediction ◽  
Support Vector ◽  
Type I ◽  
Vector Machines ◽  
The Eu

This article focuses on the problem of binary classification of 902 small- and medium‑sized engineering companies active in the EU, together with additional 51 companies which went bankrupt in 2014. For classification purposes, the basic statistical method of logistic regression has been selected, together with a representative of machine learning (support vector machines and classification trees method) to construct models for bankruptcy prediction. Different settings have been tested for each method. Furthermore, the models were estimated based on complete data and also using identified artificial factors. To evaluate the quality of prediction we observe not only the total accuracy with the type I and II errors but also the area under ROC curve criterion. The results clearly show that increasing distance to bankruptcy decreases the predictive ability of all models. The classification tree method leads us to rather simple models. The best classification results were achieved through logistic regression based on artificial factors. Moreover, this procedure provides good and stable results regardless of other settings. Artificial factors also seem to be a suitable variable for support vector machines models, but classification trees achieved better results using original data.

scholarly journals On the parameter optimization of Support Vector Machines for binary classification

2012 ◽  
Vol 9 (3) ◽  
pp. 33-43 ◽  
Author(s):  
Paulo Gaspar ◽  
Jaime Carbonell ◽  
José Luís Oliveira
Keyword(s):  
Support Vector Machines ◽  
Binary Classification ◽  
Classification Performance ◽  
Biological Data ◽  
Parameters Optimization ◽  
Support Vector ◽  
Minimal Risk ◽  
Class Separation ◽  
Vector Machines ◽  
Analyse Data

Summary Classifying biological data is a common task in the biomedical context. Predicting the class of new, unknown information allows researchers to gain insight and make decisions based on the available data. Also, using classification methods often implies choosing the best parameters to obtain optimal class separation, and the number of parameters might be large in biological datasets.Support Vector Machines provide a well-established and powerful classification method to analyse data and find the minimal-risk separation between different classes. Finding that separation strongly depends on the available feature set and the tuning of hyper-parameters. Techniques for feature selection and SVM parameters optimization are known to improve classification accuracy, and its literature is extensive.In this paper we review the strategies that are used to improve the classification performance of SVMs and perform our own experimentation to study the influence of features and hyper-parameters in the optimization process, using several known kernels.

A Computational Intelligence Approach to Supply Chain Demand Forecasting

2012 ◽  
pp. 1551-1565 ◽  
Author(s):  
Nicholas Ampazis
Keyword(s):  
Supply Chain ◽  
Computational Intelligence ◽  
Demand Forecasting ◽  
Business Environment ◽  
Chain Structure ◽  
Support Vector ◽  
Customer Demand ◽  
Public Data ◽  
Vector Machines ◽  
Supply Chain Structure

Estimating customer demand in a multi-level supply chain structure is crucial for companies seeking to maintain their competitive advantage within an uncertain business environment. This work explores the potential of computational intelligence approaches as forecasting mechanisms for predicting customer demand at the first level of organization of a supply chain where products are presented and sold to customers. The computational intelligence approaches that we utilize are Artificial Neural Networks (ANNs), trained with the OLMAM algorithm (Optimized Levenberg-Marquardt with Adaptive Momentum), and Support Vector Machines (SVMs) for regression. The effectiveness of the proposed approach was evaluated using public data from the Netflix movie rental online DVD store in order to predict the demand for movie rentals during the critical, for sales, Christmas holiday season.

Assessment of Kidney Function Using Dynamic Contrast Enhanced MRI Techniques

2010 ◽  
pp. 214-233
Author(s):  
Melih S. Aslan ◽  
Hossam Abd El Munim ◽  
Aly A. Farag ◽  
Mohamed Abou El-Ghar
Keyword(s):  
Kidney Function ◽  
Least Square ◽  
Renal Diseases ◽  
Support Vector ◽  
Data Sets ◽  
Dce Mri ◽  
Dynamic Contrast Enhanced ◽  
Contrast Enhanced ◽  
Vector Machines ◽  
Kidney Rejection

Graft failure of kidneys after transplantation is most often the consequence of the acute rejection. Hence, early detection of the kidney rejection is important for the treatment of renal diseases. In this chapter, authors introduce a new automatic approach to classify normal kidney function from kidney rejection using dynamic contrast enhanced magnetic resonance imaging (DCE-MRI). The kidney has three regions named the cortex, medulla, and pelvis. In their experiment, they use the medulla region because it has specific responses to DCE-MRI that are helpful to identify kidney rejection. In the authors’ process they segment the kidney using the level sets method. They then employ several classification methods such as the Euclidean distance, Mahalanobis distance, and least square support vector machines (LS-SVM). The authors’preliminary results are very encouraging and reproducibility of the results was achieved for 55 clinical data sets. The classification accuracy, diagnostic sensitivity, and diagnostic specificity are 84%, 75%, and 96%, respectively.

Sign in / Sign up

Export Citation Format

Share Document