Stochastic gradient descent‐based support vector machines training optimization on Big Data and HPC frameworks

2021 ◽  
Author(s):  
Vibhatha Abeykoon ◽  
Geoffrey Fox ◽  
Minje Kim ◽  
Saliya Ekanayake ◽  
Supun Kamburugamuve ◽  
...  
Keyword(s):  
Big Data ◽  
Support Vector Machines ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Vector Machines

scholarly journals Parallel Implementation on FPGA of Support Vector Machines Using Stochastic Gradient Descent

Electronics ◽  
2019 ◽  
Vol 8 (6) ◽  
pp. 631 ◽  
Author(s):  
Felipe F. Lopes ◽  
João Canas Ferreira ◽  
Marcelo A. C. Fernandes
Keyword(s):  
Support Vector Machines ◽  
Gradient Descent ◽  
Parallel Implementation ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Data Set ◽  
Viable Solution ◽  
Vector Machines ◽  
Field Programmable

Sequential Minimal Optimization (SMO) is the traditional training algorithm for Support Vector Machines (SVMs). However, SMO does not scale well with the size of the training set. For that reason, Stochastic Gradient Descent (SGD) algorithms, which have better scalability, are a better option for massive data mining applications. Furthermore, even with the use of SGD, training times can become extremely large depending on the data set. For this reason, accelerators such as Field-programmable Gate Arrays (FPGAs) are used. This work describes an implementation in hardware, using FPGA, of a fully parallel SVM using Stochastic Gradient Descent. The proposed FPGA implementation of an SVM with SGD presents speedups of more than 10,000× relative to software implementations running on a quad-core processor and up to 319× compared to state-of-the-art FPGA implementations while requiring fewer hardware resources. The results show that the proposed architecture is a viable solution for highly demanding problems such as those present in big data analysis.

scholarly journals Will they repay their debt? Identification of borrowers likely to be charged off

2020 ◽  
Vol 15 (3) ◽  
pp. 393-409
Author(s):  
Raluca Dana Caplescu ◽  
Ana-Maria Panaite ◽  
Daniel Traian Pele ◽  
Vasile Alecsandru Strat
Keyword(s):  
Logistic Regression ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Sampled Data ◽  
K Nearest Neighbors ◽  
Vector Machines ◽  
Income Ratio ◽  
Strong Candidate

AbstractRecent increase in peer-to-peer lending prompted for development of models to separate good and bad clients to mitigate risks both for lenders and for the platforms. The rapidly increasing body of literature provides several comparisons between various models. Among the most frequently employed ones are logistic regression, Support Vector Machines, neural networks and decision tree-based models. Among them, logistic regression has proved to be a strong candidate both because its good performance and due to its high explainability. The present paper aims to compare four pairs of models (for imbalanced and under-sampled data) meant to predict charged off clients by optimizing F1 score. We found that, if the data is balanced, Logistic Regression, both simple and with Stochastic Gradient Descent, outperforms LightGBM and K-Nearest Neighbors in optimizing F1 score. We chose this metric as it provides balance between the interests of the lenders and those of the platform. Loan term, debt-to-income ratio and number of accounts were found to be important positively related predictors of risk of charge off. At the other end of the spectrum, by far the strongest impact on charge off probability is that of the FICO score. The final number of features retained by the two models differs very much, because, although both models use Lasso for feature selection, Stochastic Gradient Descent Logistic Regression uses a stronger regularization. The analysis was performed using Python (numpy, pandas, sklearn and imblearn).

Comparison of SVM, RF and SGD Methods for Determination of Programmer's Performance Classification Model in Social Media Activities

2020 ◽  
Vol 4 (2) ◽  
pp. 329-335
Author(s):  
Rusydi Umar ◽  
Imam Riadi ◽  
Purwono
Keyword(s):  
Social Media ◽  
Gradient Descent ◽  
Classification Model ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Svm Algorithm ◽  
Vector Machines ◽  
Performance Patterns ◽  
A Company

The failure of most startups in Indonesia is caused by team performance that is not solid and competent. Programmers are an integral profession in a startup team. The development of social media can be used as a strategic tool for recruiting the best programmer candidates in a company. This strategic tool is in the form of an automatic classification system of social media posting from prospective programmers. The classification results are expected to be able to predict the performance patterns of each candidate with a predicate of good or bad performance. The classification method with the best accuracy needs to be chosen in order to get an effective strategic tool so that a comparison of several methods is needed. This study compares classification methods including the Support Vector Machines (SVM) algorithm, Random Forest (RF) and Stochastic Gradient Descent (SGD). The classification results show the percentage of accuracy with k = 10 cross validation for the SVM algorithm reaches 81.3%, RF at 74.4%, and SGD at 80.1% so that the SVM method is chosen as a model of programmer performance classification on social media activities.

Exploring Support Vector Machines for Big Data Analyses

2021 ◽  
Author(s):  
Siyang Lu ◽  
Yihong Chen ◽  
Xiaolin Zhu ◽  
Ziyi Wang ◽  
Yangjun Ou ◽  
...  
Keyword(s):  
Big Data ◽  
Support Vector Machines ◽  
Support Vector ◽  
Data Analyses ◽  
Vector Machines

scholarly journals Research on Parallel Support Vector Machine Based on Spark Big Data Platform

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Yao Huimin
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Big Data ◽  
Support Vector Machines ◽  
Cross Validation ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Lambda Architecture ◽  
Vector Machines ◽  
Data Platform

With the development of cloud computing and distributed cluster technology, the concept of big data has been expanded and extended in terms of capacity and value, and machine learning technology has also received unprecedented attention in recent years. Traditional machine learning algorithms cannot solve the problem of effective parallelization, so a parallelization support vector machine based on Spark big data platform is proposed. Firstly, the big data platform is designed with Lambda architecture, which is divided into three layers: Batch Layer, Serving Layer, and Speed Layer. Secondly, in order to improve the training efficiency of support vector machines on large-scale data, when merging two support vector machines, the “special points” other than support vectors are considered, that is, the points where the nonsupport vectors in one subset violate the training results of the other subset, and a cross-validation merging algorithm is proposed. Then, a parallelized support vector machine based on cross-validation is proposed, and the parallelization process of the support vector machine is realized on the Spark platform. Finally, experiments on different datasets verify the effectiveness and stability of the proposed method. Experimental results show that the proposed parallelized support vector machine has outstanding performance in speed-up ratio, training time, and prediction accuracy.

Sign in / Sign up

Export Citation Format

Share Document