Deep support vector neural networks

2020 ◽  
Vol 27 (4) ◽  
pp. 389-402
Author(s):  
David Díaz-Vico ◽  
Jesús Prada ◽  
Adil Omari ◽  
José Dorronsoro
Keyword(s):  
Neural Networks ◽  
Sample Size ◽  
Deep Neural Networks ◽  
Support Vector ◽  
Classification Problems ◽  
Vector Machines ◽  
Training Cost ◽  
Regression Problems ◽  
Classification And Regression ◽  
Deep Support

Kernel based Support Vector Machines, SVM, one of the most popular machine learning models, usually achieve top performances in two-class classification and regression problems. However, their training cost is at least quadratic on sample size, making them thus unsuitable for large sample problems. However, Deep Neural Networks (DNNs), with a cost linear on sample size, are able to solve big data problems relatively easily. In this work we propose to combine the advanced representations that DNNs can achieve in their last hidden layers with the hinge and ϵ insensitive losses that are used in two-class SVM classification and regression. We can thus have much better scalability while achieving performances comparable to those of SVMs. Moreover, we will also show that the resulting Deep SVM models are competitive with standard DNNs in two-class classification problems but have an edge in regression ones.

scholarly journals Beyond the Hype: Deep Neural Networks Outperform Established Methods Using A ChEMBL Bioactivity Benchmark Set

2017 ◽  
Author(s):  
Eelke B. Lenselink ◽  
Niels ten Dijke ◽  
Brandon Bongers ◽  
George Papadatos ◽  
Herman W.T. van Vlijmen ◽  
...  
Keyword(s):  
Neural Networks ◽  
Logistic Regression ◽  
Support Vector Machines ◽  
Random Forests ◽  
Deep Neural Networks ◽  
Direct Result ◽  
Support Vector ◽  
Temporal Validation ◽  
Vector Machines ◽  
The Mean

AbstractThe increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics.In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naive Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution.Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized DNN_PCM).Here, a standardized set to test and evaluate different machine learning algorithms in the context of multitask learning is offered by providing the data and the protocols.

scholarly journals Universal consistency of twin support vector machines

2021 ◽  
Author(s):  
Weixia Xu ◽  
Dingjiang Huang ◽  
Shuigeng Zhou
Keyword(s):  
Support Vector Machines ◽  
Sample Size ◽  
General Framework ◽  
Binary Classification ◽  
Classification Problem ◽  
Covering Number ◽  
Support Vector ◽  
Classification Problems ◽  
Vector Machines ◽  
Universal Consistency

AbstractA classification problem aims at constructing a best classifier with the smallest risk. When the sample size approaches infinity, the learning algorithms for a classification problem are characterized by an asymptotical property, i.e., universal consistency. It plays a crucial role in measuring the construction of classification rules. A universal consistent algorithm ensures that the larger the sample size of the algorithm is, the more accurately the distribution of the samples could be reconstructed. Support vector machines (SVMs) are regarded as one of the most important models in binary classification problems. How to effectively extend SVMs to twin support vector machines (TWSVMs) so as to improve performance of classification has gained increasing interest in many research areas recently. Many variants for TWSVMs have been proposed and used in practice. Thus in this paper, we focus on the universal consistency of TWSVMs in a binary classification setting. We first give a general framework for TWSVM classifiers that unifies most of the variants of TWSVMs for binary classification problems. Based on it, we then investigate the universal consistency of TWSVMs. To do this, we give some useful definitions of risk, Bayes risk and universal consistency for TWSVMs. Theoretical results indicate that universal consistency is valid for various TWSVM classifiers under some certain conditions, including covering number, localized covering number and stability. For applications of our general framework, several variants of TWSVMs are considered.

Sign in / Sign up

Export Citation Format

Share Document