scholarly journals Reduced Dilation-Erosion Perceptron for Binary Classification

Mathematics ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 512
Author(s):  
Marcos Eduardo Valle
Keyword(s):  
Mathematical Morphology ◽  
Computing Methodology ◽  
Convex Combination ◽  
Binary Classification ◽  
Feature Space ◽  
Support Vector ◽  
Training Set ◽  
Linear Lattice ◽  
Hinge Loss ◽  
Classification Tasks

Dilation and erosion are two elementary operations from mathematical morphology, a non-linear lattice computing methodology widely used for image processing and analysis. The dilation-erosion perceptron (DEP) is a morphological neural network obtained by a convex combination of a dilation and an erosion followed by the application of a hard-limiter function for binary classification tasks. A DEP classifier can be trained using a convex-concave procedure along with the minimization of the hinge loss function. As a lattice computing model, the DEP classifier assumes the feature and class spaces are partially ordered sets. In many practical situations, however, there is no natural ordering for the feature patterns. Using concepts from multi-valued mathematical morphology, this paper introduces the reduced dilation-erosion (r-DEP) classifier. An r-DEP classifier is obtained by endowing the feature space with an appropriate reduced ordering. Such reduced ordering can be determined using two approaches: one based on an ensemble of support vector classifiers (SVCs) with different kernels and the other based on a bagging of similar SVCs trained using different samples of the training set. Using several binary classification datasets from the OpenML repository, the ensemble and bagging r-DEP classifiers yielded mean higher balanced accuracy scores than the linear, polynomial, and radial basis function (RBF) SVCs as well as their ensemble and a bagging of RBF SVCs.

scholarly journals Properties of Support Vector Machines

1998 ◽  
Vol 10 (4) ◽  
pp. 955-974 ◽  
Author(s):  
Massimiliano Pontil ◽  
Alessandro Verri
Keyword(s):  
Support Vector Machines ◽  
Finite Dimension ◽  
Regularization Parameter ◽  
Feature Space ◽  
Support Vector ◽  
Training Set ◽  
Mathematical Properties ◽  
Support Vectors ◽  
Vector Machines ◽  
Almost All

Support vector machines (SVMs) perform pattern recognition between two point classes by finding a decision surface determined by certain points of the training set, termed support vectors (SV). This surface, which in some feature space of possibly infinite dimension can be regarded as a hyperplane, is obtained from the solution of a problem of quadratic programming that depends on a regularization parameter. In this article, we study some mathematical properties of support vectors and show that the decision surface can be written as the sum of two orthogonal terms, the first depending on only the margin vectors (which are SVs lying on the margin), the second proportional to the regularization parameter. For almost all values of the parameter, this enables us to predict how the decision surface varies for small parameter changes. In the special but important case of feature space of finite dimension m, we also show that there are at most m + 1 margin vectors and observe that m + 1 SVs are usually sufficient to determine the decision surface fully. For relatively small m, this latter result leads to a consistent reduction of the SV number.

scholarly journals An SVM-Based Framework for Long-Term Learning Systems

2019 ◽  
Vol 33 ◽  
pp. 9915-9916
Author(s):  
Diana Benavides-Prado
Keyword(s):  
Binary Classification ◽  
Upper Bounds ◽  
Support Vector ◽  
Transfer Of Knowledge ◽  
Learning Framework ◽  
Vector Machines ◽  
Novel Method ◽  
Classification Tasks ◽  
Retention Of Knowledge

In our research, we study the problem of learning a sequence of supervised tasks. This is a long-standing challenge in machine learning. Our work relies on transfer of knowledge between hypotheses learned with Support Vector Machines. Transfer occurs in two directions: forward and backward. We have proposed to selectively transfer forward support vector coefficients from previous hypotheses as upper-bounds on support vector coefficients to be learned on a target task. We also proposed a novel method for refining existing hypotheses by transferring backward knowledge from a target hypothesis learned recently. We have improved this method through a hypothesis refinement approach that refines whilst encouraging retention of knowledge. Our contribution is represented in a long-term learning framework for binary classification tasks received sequentially one at a time.

scholarly journals SVM-Based Multiple Instance Classification via DC Optimization

Algorithms ◽  
2019 ◽  
Vol 12 (12) ◽  
pp. 249 ◽  
Author(s):  
Annabella Astorino ◽  
Antonio Fuduli ◽  
Giovanni Giallombardo ◽  
Giovanna Miglionico
Keyword(s):  
Binary Classification ◽  
Error Function ◽  
Multiple Instance Learning ◽  
Classification Error ◽  
Support Vector ◽  
Data Sets ◽  
Training Set ◽  
Dc Algorithm ◽  
Difference Of Convex ◽  
Dc Decomposition

A multiple instance learning problem consists of categorizing objects, each represented as a set (bag) of points. Unlike the supervised classification paradigm, where each point of the training set is labeled, the labels are only associated with bags, while the labels of the points inside the bags are unknown. We focus on the binary classification case, where the objective is to discriminate between positive and negative bags using a separating surface. Adopting a support vector machine setting at the training level, the problem of minimizing the classification-error function can be formulated as a nonconvex nonsmooth unconstrained program. We propose a difference-of-convex (DC) decomposition of the nonconvex function, which we face using an appropriate nonsmooth DC algorithm. Some of the numerical results on benchmark data sets are reported.

Comparing Classification Methods for Longitudinal fMRI Studies

2010 ◽  
Vol 22 (11) ◽  
pp. 2729-2762 ◽  
Author(s):  
Tanya Schmah ◽  
Grigori Yourganov ◽  
Richard S. Zemel ◽  
Geoffrey E. Hinton ◽  
Steven L. Small ◽  
...  
Keyword(s):  
Support Vector Machines ◽  
Binary Classification ◽  
Stroke Recovery ◽  
Support Vector ◽  
Linear Quadratic ◽  
Restricted Boltzmann Machines ◽  
K Nearest Neighbors ◽  
Vector Machines ◽  
Out Of Sample ◽  
Classification Tasks

We compare 10 methods of classifying fMRI volumes by applying them to data from a longitudinal study of stroke recovery: adaptive Fisher's linear and quadratic discriminant; gaussian naive Bayes; support vector machines with linear, quadratic, and radial basis function (RBF) kernels; logistic regression; two novel methods based on pairs of restricted Boltzmann machines (RBM); and K-nearest neighbors. All methods were tested on three binary classification tasks, and their out-of-sample classification accuracies are compared. The relative performance of the methods varies considerably across subjects and classification tasks. The best overall performers were adaptive quadratic discriminant, support vector machines with RBF kernels, and generatively trained pairs of RBMs.

scholarly journals THE USE OF MACHINE LEARNING METHODS FOR BINARY CLASSIFICATION OF THE WORKING CONDITION OF BEARINGS USING THE SIGNALS OF VIBRATION ACCELERATION

2021 ◽  
pp. 15-22
Author(s):  
Ruslan Babudzhan ◽  
Konstantyn Isaienkov ◽  
Danilo Krasiy ◽  
Oleksii Vodka ◽  
Ivan Zadorozhny ◽  
...  
Keyword(s):  
Machine Learning ◽  
Binary Classification ◽  
Fractal Dimensions ◽  
Feature Space ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Support Vector ◽  
Data Sets ◽  
Vibration Acceleration ◽  
K Nearest Neighbors

The paper investigates the relationship between vibration acceleration of bearings with their operational state. To determine these dependencies, a testbench was built and 112 experiments were carried out with different bearings: 100 bearings that developed an internal defect during operation and 12bearings without a defect. From the obtained records, a dataset was formed, which was used to build classifiers. Dataset is freely available. A methodfor classifying new and used bearings was proposed, which consists in searching for dependencies and regularities of the signal using descriptive functions: statistical, entropy, fractal dimensions and others. In addition to processing the signal itself, the frequency domain of the bearing operationsignal was also used to complement the feature space. The paper considered the possibility of generalizing the classification for its application on thosesignals that were not obtained in the course of laboratory experiments. An extraneous dataset was found in the public domain. This dataset was used todetermine how accurate a classifier was when it was trained and tested on significantly different signals. Training and validation were carried out usingthe bootstrapping method to eradicate the effect of randomness, given the small amount of training data available. To estimate the quality of theclassifiers, the F1-measure was used as the main metric due to the imbalance of the data sets. The following supervised machine learning methodswere chosen as classifier models: logistic regression, support vector machine, random forest, and K nearest neighbors. The results are presented in theform of plots of density distribution and diagrams.

scholarly journals Exploiting Synthetically Generated Data with Semi-Supervised Learning for Small and Imbalanced Datasets

2019 ◽  
Vol 33 ◽  
pp. 4715-4722
Author(s):  
M. Peréz-Ortiz ◽  
P. Tiňo ◽  
R. Mantiuk ◽  
C. Hervás-Martínez
Keyword(s):  
Supervised Learning ◽  
Data Augmentation ◽  
Convex Combination ◽  
Binary Classification ◽  
Synthetic Data ◽  
Learning Problems ◽  
Support Vector ◽  
Learning Framework ◽  
Main Challenge ◽  
Cluster Assumption

Data augmentation is rapidly gaining attention in machine learning. Synthetic data can be generated by simple transformations or through the data distribution. In the latter case, the main challenge is to estimate the label associated to new synthetic patterns. This paper studies the effect of generating synthetic data by convex combination of patterns and the use of these as unsupervised information in a semi-supervised learning framework with support vector machines, avoiding thus the need to label synthetic examples. We perform experiments on a total of 53 binary classification datasets. Our results show that this type of data over-sampling supports the well-known cluster assumption in semi-supervised learning, showing outstanding results for small high-dimensional datasets and imbalanced learning problems.

scholarly journals Toward Accelerated Training of Parallel Support Vector Machines Based on Voronoi Diagrams

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1605
Author(s):  
Cesar Alfaro ◽  
Javier Gomez ◽  
Javier M. Moguerza ◽  
Javier Castillo ◽  
Jose I. Martinez
Keyword(s):  
Support Vector Machines ◽  
Data Exchange ◽  
Smart Cities ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Training Set ◽  
Vector Machines ◽  
Comparable Performance ◽  
Novel Method ◽  
Classification Tasks

Typical applications of wireless sensor networks (WSN), such as in Industry 4.0 and smart cities, involves acquiring and processing large amounts of data in federated systems. Important challenges arise for machine learning algorithms in this scenario, such as reducing energy consumption and minimizing data exchange between devices in different zones. This paper introduces a novel method for accelerated training of parallel Support Vector Machines (pSVMs), based on ensembles, tailored to these kinds of problems. To achieve this, the training set is split into several Voronoi regions. These regions are small enough to permit faster parallel training of SVMs, reducing computational payload. Results from experiments comparing the proposed method with a single SVM and a standard ensemble of SVMs demonstrate that this approach can provide comparable performance while limiting the number of regions required to solve classification tasks. These advantages facilitate the development of energy-efficient policies in WSN.

scholarly journals A Learning Framework of Nonparallel Hyperplanes Classifier

2015 ◽  
Vol 2015 ◽  
pp. 1-12
Author(s):  
Zhi-Xia Yang ◽  
Yuan-Hai Shao ◽  
Yao-Lin Jiang
Keyword(s):  
Binary Classification ◽  
Computational Cost ◽  
Classification Problem ◽  
Multiclass Classification ◽  
Decision Function ◽  
Support Vector ◽  
Learning Framework ◽  
Hinge Loss ◽  
Benchmark Datasets ◽  
Nonparallel Hyperplanes

A novel learning framework of nonparallel hyperplanes support vector machines (NPSVMs) is proposed for binary classification and multiclass classification. This framework not only includes twin SVM (TWSVM) and its many deformation versions but also extends them into multiclass classification problem when different parameters or loss functions are chosen. Concretely, we discuss the linear and nonlinear cases of the framework, in which we select the hinge loss function as example. Moreover, we also give the primal problems of several extension versions of TWSVM’s deformation versions. It is worth mentioning that, in the decision function, the Euclidean distance is replaced by the absolute value|wTx+b|, which keeps the consistency between the decision function and the optimization problem and reduces the computational cost particularly when the kernel function is introduced. The numerical experiments on several artificial and benchmark datasets indicate that our framework is not only fast but also shows good generalization.

scholarly journals Robust relative margin support vector machines

2016 ◽  
Vol 11 (2) ◽  
pp. 186-191 ◽  
Author(s):  
Yunyan Song ◽  
Wenxin Zhu ◽  
Yingyuan Xiao ◽  
Ping Zhong
Keyword(s):  
Computational Complexity ◽  
Binary Classification ◽  
Support Vector ◽  
Decision Boundary ◽  
Large Margin ◽  
Hinge Loss ◽  
Shortest Distance ◽  
Vector Machines ◽  
Pinball Loss ◽  
Real World Problems

Recently, a class of classifiers, called relative margin machine, has been developed. Relative margin machine has shown significant improvements over the large margin counterparts on real-world problems. In binary classification, the most widely used loss function is the hinge loss, which results in the hinge loss relative margin machine. The hinge loss relative margin machine is sensitive to outliers. In this article, we proposed to change maximizing the shortest distance used in relative margin machine into maximizing the quantile distance, the pinball loss which is related to quantiles was used in classification. The proposed method is less sensitive to noise, especially the feature noise around the decision boundary. Meanwhile, the computational complexity of the proposed method is similar to that of the relative margin machine.

scholarly journals TRAINING SUPPORT VECTOR MACHINES USING FRANK–WOLFE OPTIMIZATION METHODS

2013 ◽  
Vol 27 (03) ◽  
pp. 1360003 ◽  
Author(s):  
EMANUELE FRANDI ◽  
RICARDO ÑANCULEF ◽  
MARIA GRAZIA GASPARO ◽  
STEFANO LODI ◽  
CLAUDIO SARTORI
Keyword(s):  
Large Scale ◽  
Binary Classification ◽  
Feature Space ◽  
Optimization Methods ◽  
Learning Problems ◽  
Support Vector ◽  
Fast Method ◽  
Training Support ◽  
Lower Accuracy ◽  
Vector Machines

Training a support vector machine (SVM) requires the solution of a quadratic programming problem (QP) whose computational complexity becomes prohibitively expensive for large scale datasets. Traditional optimization methods cannot be directly applied in these cases, mainly due to memory restrictions. By adopting a slightly different objective function and under mild conditions on the kernel used within the model, efficient algorithms to train SVMs have been devised under the name of core vector machines (CVMs). This framework exploits the equivalence of the resulting learning problem with the task of building a minimal enclosing ball (MEB) problem in a feature space, where data is implicitly embedded by a kernel function. In this paper, we improve on the CVM approach by proposing two novel methods to build SVMs based on the Frank–Wolfe algorithm, recently revisited as a fast method to approximate the solution of a MEB problem. In contrast to CVMs, our algorithms do not require to compute the solutions of a sequence of increasingly complex QPs and are defined by using only analytic optimization steps. Experiments on a large collection of datasets show that our methods scale better than CVMs in most cases, sometimes at the price of a slightly lower accuracy. As CVMs, the proposed methods can be easily extended to machine learning problems other than binary classification. However, effective classifiers are also obtained using kernels which do not satisfy the condition required by CVMs, and thus our methods can be used for a wider set of problems.

Sign in / Sign up

Export Citation Format

Share Document