Solving large-scale multiclass learning problems via an efficient support vector classifier

2006 ◽  
Vol 17 (4) ◽  
pp. 910-915 ◽  
Author(s):  
Zheng Shuibo ◽  
Tang Houjun ◽  
Han Zhengzhi ◽  
Zhang Haoran
Keyword(s):  
Large Scale ◽  
Learning Problems ◽  
Support Vector ◽  
Support Vector Classifier

scholarly journals Effects of network topology on the performance of consensus and distributed learning of SVMs using ADMM

2021 ◽  
Vol 7 ◽  
pp. e397
Author(s):  
Shirin Tavara ◽  
Alexander Schliep
Keyword(s):  
Network Topology ◽  
Large Scale ◽  
Learning Problems ◽  
Support Vector ◽  
Spectral Gaps ◽  
Distributed Framework ◽  
Alternating Direction ◽  
Communication Time ◽  
Network Topologies ◽  
The Impact

The Alternating Direction Method of Multipliers (ADMM) is a popular and promising distributed framework for solving large-scale machine learning problems. We consider decentralized consensus-based ADMM in which nodes may only communicate with one-hop neighbors. This may cause slow convergence. We investigate the impact of network topology on the performance of an ADMM-based learning of Support Vector Machine using expander, and mean-degree graphs, and additionally some of the common modern network topologies. In particular, we investigate to which degree the expansion property of the network influences the convergence in terms of iterations, training and communication time. We furthermore suggest which topology is preferable. Additionally, we provide an implementation that makes these theoretical advances easily available. The results show that the performance of decentralized ADMM-based learning of SVMs in terms of convergence is improved using graphs with large spectral gaps, higher and homogeneous degrees.

scholarly journals TRAINING SUPPORT VECTOR MACHINES USING FRANK–WOLFE OPTIMIZATION METHODS

2013 ◽  
Vol 27 (03) ◽  
pp. 1360003 ◽  
Author(s):  
EMANUELE FRANDI ◽  
RICARDO ÑANCULEF ◽  
MARIA GRAZIA GASPARO ◽  
STEFANO LODI ◽  
CLAUDIO SARTORI
Keyword(s):  
Large Scale ◽  
Binary Classification ◽  
Feature Space ◽  
Optimization Methods ◽  
Learning Problems ◽  
Support Vector ◽  
Fast Method ◽  
Training Support ◽  
Lower Accuracy ◽  
Vector Machines

Training a support vector machine (SVM) requires the solution of a quadratic programming problem (QP) whose computational complexity becomes prohibitively expensive for large scale datasets. Traditional optimization methods cannot be directly applied in these cases, mainly due to memory restrictions. By adopting a slightly different objective function and under mild conditions on the kernel used within the model, efficient algorithms to train SVMs have been devised under the name of core vector machines (CVMs). This framework exploits the equivalence of the resulting learning problem with the task of building a minimal enclosing ball (MEB) problem in a feature space, where data is implicitly embedded by a kernel function. In this paper, we improve on the CVM approach by proposing two novel methods to build SVMs based on the Frank–Wolfe algorithm, recently revisited as a fast method to approximate the solution of a MEB problem. In contrast to CVMs, our algorithms do not require to compute the solutions of a sequence of increasingly complex QPs and are defined by using only analytic optimization steps. Experiments on a large collection of datasets show that our methods scale better than CVMs in most cases, sometimes at the price of a slightly lower accuracy. As CVMs, the proposed methods can be easily extended to machine learning problems other than binary classification. However, effective classifiers are also obtained using kernels which do not satisfy the condition required by CVMs, and thus our methods can be used for a wider set of problems.

A FAST SVM TRAINING ALGORITHM

2003 ◽  
Vol 17 (03) ◽  
pp. 367-384 ◽  
Author(s):  
JIAN-XIONG DONG ◽  
CHING Y. SUEN ◽  
ADAM KRZYŻAK
Keyword(s):  
Data Mining ◽  
Principal Component Analysis ◽  
Support Vector Machine ◽  
Large Scale ◽  
Principal Component ◽  
Learning Problems ◽  
Support Vector ◽  
Training Algorithm ◽  
Test Set ◽  
Handwritten Digit

A fast support vector machine (SVM) training algorithm is proposed under SVM's decomposition framework by effectively integrating kernel caching, digest and shrinking policies and stopping conditions. Kernel caching plays a key role in reducing the number of kernel evaluations by maximal reusage of cached kernel elements. Extensive experiments have been conducted on a large handwritten digit database MNIST to show that the proposed algorithm is much faster than Keerthi et al.'s improved SMO, about nine times. Combined with principal component analysis, the total training for ten one-against-the-rest classifiers on MNIST took less than an hour. Moreover, the proposed fast algorithm speeds up SVM training without sacrificing the generalization performance. The 0.6% error rate on MNIST test set has been achieved. The promising scalability of the proposed scheme paves a new way to solve more large-scale learning problems in other domains such as data mining.

Research on improving accuracy of MWD based on Support vector Classifier and K-proximity method

2021 ◽  
pp. 1-1
Author(s):  
Hai Yang ◽  
Lizao Zhang ◽  
Tao Luo ◽  
Haibo Liang ◽  
Li Li ◽  
...  
Keyword(s):  
Support Vector ◽  
Support Vector Classifier ◽  
Improving Accuracy

scholarly journals QUBO formulations for training machine learning models

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Prasanna Date ◽  
Davis Arthur ◽  
Lauren Pusey-Nazzaro
Keyword(s):  
Machine Learning ◽  
Linear Regression ◽  
Large Scale ◽  
Support Vector ◽  
Quantum Computers ◽  
Np Hard ◽  
Learning Models ◽  
Moore’S Law ◽  
Moore's Law ◽  
Machine Learning Models

AbstractTraining machine learning models on classical computers is usually a time and compute intensive process. With Moore’s law nearing its inevitable end and an ever-increasing demand for large-scale data analysis using machine learning, we must leverage non-conventional computing paradigms like quantum computing to train machine learning models efficiently. Adiabatic quantum computers can approximately solve NP-hard problems, such as the quadratic unconstrained binary optimization (QUBO), faster than classical computers. Since many machine learning problems are also NP-hard, we believe adiabatic quantum computers might be instrumental in training machine learning models efficiently in the post Moore’s law era. In order to solve problems on adiabatic quantum computers, they must be formulated as QUBO problems, which is very challenging. In this paper, we formulate the training problems of three machine learning models—linear regression, support vector machine (SVM) and balanced k-means clustering—as QUBO problems, making them conducive to be trained on adiabatic quantum computers. We also analyze the computational complexities of our formulations and compare them to corresponding state-of-the-art classical approaches. We show that the time and space complexities of our formulations are better (in case of SVM and balanced k-means clustering) or equivalent (in case of linear regression) to their classical counterparts.

scholarly journals Large-Scale Landslide Displacement Rate Prediction Based on Multi-Factor Support Vector Regression Machine

2021 ◽  
Vol 11 (4) ◽  
pp. 1381
Author(s):  
Xiuzhen Li ◽  
Shengwei Li
Keyword(s):  
Support Vector Regression ◽  
Water Level ◽  
Large Scale ◽  
Displacement Rate ◽  
Support Vector ◽  
Single Factor ◽  
Reservoir Water ◽  
Reservoir Water Level ◽  
Depth Analysis ◽  
Three Factor

Forecasting the development of large-scale landslides is a contentious and complicated issue. In this study, we put forward the use of multi-factor support vector regression machines (SVRMs) for predicting the displacement rate of a large-scale landslide. The relative relationships between the main monitoring factors were analyzed based on the long-term monitoring data of the landslide and the grey correlation analysis theory. We found that the average correlation between landslide displacement and rainfall is 0.894, and the correlation between landslide displacement and reservoir water level is 0.338. Finally, based on an in-depth analysis of the basic characteristics, influencing factors, and development of landslides, three main factors (i.e., the displacement rate, reservoir water level, and rainfall) were selected to build single-factor, two-factor, and three-factor SVRM models. The key parameters of the models were determined using a grid-search method, and the models showed high accuracies. Moreover, the accuracy of the two-factor SVRM model (displacement rate and rainfall) is the highest with the smallest standard error (RMSE) of 0.00614; it is followed by the three-factor and single-factor SVRM models, the latter of which has the lowest prediction accuracy, with the largest RMSE of 0.01644.

scholarly journals Land Cover Classification of Nine Perennial Crops Using Sentinel-1 and -2 Data

2019 ◽  
Vol 12 (1) ◽  
pp. 96 ◽  
Author(s):  
James Brinkhoff ◽  
Justin Vardanega ◽  
Andrew J. Robson
Keyword(s):  
Land Cover ◽  
Large Scale ◽  
Satellite Image ◽  
Resource Planning ◽  
Support Vector ◽  
Perennial Crop ◽  
Perennial Crops ◽  
Object Based ◽  
Rbf Kernel

Land cover mapping of intensive cropping areas facilitates an enhanced regional response to biosecurity threats and to natural disasters such as drought and flooding. Such maps also provide information for natural resource planning and analysis of the temporal and spatial trends in crop distribution and gross production. In this work, 10 meter resolution land cover maps were generated over a 6200 km2 area of the Riverina region in New South Wales (NSW), Australia, with a focus on locating the most important perennial crops in the region. The maps discriminated between 12 classes, including nine perennial crop classes. A satellite image time series (SITS) of freely available Sentinel-1 synthetic aperture radar (SAR) and Sentinel-2 multispectral imagery was used. A segmentation technique grouped spectrally similar adjacent pixels together, to enable object-based image analysis (OBIA). K-means unsupervised clustering was used to filter training points and classify some map areas, which improved supervised classification of the remaining areas. The support vector machine (SVM) supervised classifier with radial basis function (RBF) kernel gave the best results among several algorithms trialled. The accuracies of maps generated using several combinations of the multispectral and radar bands were compared to assess the relative value of each combination. An object-based post classification refinement step was developed, enabling optimization of the tradeoff between producers’ accuracy and users’ accuracy. Accuracy was assessed against randomly sampled segments, and the final map achieved an overall count-based accuracy of 84.8% and area-weighted accuracy of 90.9%. Producers’ accuracies for the perennial crop classes ranged from 78 to 100%, and users’ accuracies ranged from 63 to 100%. This work develops methods to generate detailed and large-scale maps that accurately discriminate between many perennial crops and can be updated frequently.

scholarly journals A Compound Wind Power Forecasting Strategy Based on Clustering, Two-Stage Decomposition, Parameter Optimization, and Optimal Combination of Multiple Machine Learning Approaches

Energies ◽  
2019 ◽  
Vol 12 (18) ◽  
pp. 3586 ◽  
Author(s):  
Sizhou Sun ◽  
Jingqi Fu ◽  
Ang Li
Keyword(s):  
Wind Power ◽  
Large Scale ◽  
Wind Farm ◽  
Search Algorithm ◽  
Interpolation Method ◽  
Least Square ◽  
Support Vector ◽  
Two Stage ◽  
Wind Power Forecasting ◽  
Power Forecasting

Given the large-scale exploitation and utilization of wind power, the problems caused by the high stochastic and random characteristics of wind speed make researchers develop more reliable and precise wind power forecasting (WPF) models. To obtain better predicting accuracy, this study proposes a novel compound WPF strategy by optimal integration of four base forecasting engines. In the forecasting process, density-based spatial clustering of applications with noise (DBSCAN) is firstly employed to identify meaningful information and discard the abnormal wind power data. To eliminate the adverse influence of the missing data on the forecasting accuracy, Lagrange interpolation method is developed to get the corrected values of the missing points. Then, the two-stage decomposition (TSD) method including ensemble empirical mode decomposition (EEMD) and wavelet transform (WT) is utilized to preprocess the wind power data. In the decomposition process, the empirical wind power data are disassembled into different intrinsic mode functions (IMFs) and one residual (Res) by EEMD, and the highest frequent time series IMF1 is further broken into different components by WT. After determination of the input matrix by a partial autocorrelation function (PACF) and normalization into [0, 1], these decomposed components are used as the input variables of all the base forecasting engines, including least square support vector machine (LSSVM), wavelet neural networks (WNN), extreme learning machine (ELM) and autoregressive integrated moving average (ARIMA), to make the multistep WPF. To avoid local optima and improve the forecasting performance, the parameters in LSSVM, ELM, and WNN are tuned by backtracking search algorithm (BSA). On this basis, BSA algorithm is also employed to optimize the weighted coefficients of the individual forecasting results that produced by the four base forecasting engines to generate an ensemble of the forecasts. In the end, case studies for a certain wind farm in China are carried out to assess the proposed forecasting strategy.

Sign in / Sign up

Export Citation Format

Share Document