Minimum Distribution Support Vector Clustering

Support vector clustering (SVC) is a boundary-based algorithm, which has several advantages over other clustering methods, including identifying clusters of arbitrary shapes and numbers. Leveraged by the high generalization ability of the large margin distribution machine (LDM) and the optimal margin distribution clustering (ODMC), we propose a new clustering method: minimum distribution for support vector clustering (MDSVC), for improving the robustness of boundary point recognition, which characterizes the optimal hypersphere by the first-order and second-order statistics and tries to minimize the mean and variance simultaneously. In addition, we further prove, theoretically, that our algorithm can obtain better generalization performance. Some instructive insights for adjusting the number of support vector points are gained. For the optimization problem of MDSVC, we propose a double coordinate descent algorithm for small and medium samples. The experimental results on both artificial and real datasets indicate that our MDSVC has a significant improvement in generalization performance compared to SVC.

Download Full-text

Robust Transductive Support Vector Machine for Multi-View Classification

Journal of Circuits System and Computers ◽

10.1142/s0218126618501852 ◽

2018 ◽

Vol 27 (12) ◽

pp. 1850185 ◽

Cited By ~ 2

Author(s):

Yanchao Li ◽

Yongli Wang ◽

Junlong Zhou ◽

Xiaohui Jiang

Keyword(s):

Support Vector Machine ◽

Support Vector ◽

Learning Methods ◽

Generalization Performance ◽

Single View ◽

Second Order Statistics ◽

Margin Distribution ◽

Small Set ◽

Transductive Support Vector Machine ◽

Specific Difficulty

Semi-Supervised Learning (SSL) aims to improve the performance of models trained with a small set of labeled data and a large collection of unlabeled data. Learning multi-view representations from different perspectives of data has proved to be very effectively for improving generalization performance. However, existing semi-supervised multi-view learning methods tend to ignore the specific difficulty of different unlabeled examples, such as the outliers and noise, leading to error-prone classification. To address this problem, this paper proposes Robust Transductive Support Vector Machine (RTSVM) that introduces the margin distribution into TSVM, which is robust to the outliers and noise. Specifically, the first-order (margin mean) and second-order statistics (margin variance) are regularized into TSVM, which try to achieve strong generalization performance. Then, we impose a global similarity constraint between distinct RTSVMs each trained from one view of the data. Moreover, our algorithm runs with fast convergence by using concave–convex procedure. Finally, we validate our proposed method on a variety of multi-view datasets, and the experimental results demonstrate that our proposed algorithm is effective. By exploring large amount of unlabeled examples and being robust to the outliers and noise among different views, the generalization performance of our method show the superiority to single-view learning and other semi-supervised multi-view learning methods.

Download Full-text

Semi-Supervised Optimal Margin Distribution Machines

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/431 ◽

2018 ◽

Cited By ~ 2

Author(s):

Teng Zhang ◽

Zhi-Hua Zhou

Keyword(s):

Support Vector Machines ◽

Minimax Problem ◽

Support Vector ◽

Data Sets ◽

Second Order Statistics ◽

Novel Approach ◽

Margin Distribution ◽

Vector Machines ◽

Mean And Variance ◽

Minimum Margin

Semi-supervised support vector machines is an extension of standard support vector machines with unlabeled instances, and the goal is to find a label assignment of the unlabeled instances, so that the decision boundary has the maximal \textit{minimum margin} on both the original labeled instances and unlabeled instances. Recent studies, however, disclosed that maximizing the minimum margin does not necessarily lead to better performance, and instead, it is crucial to optimize the \textit{margin distribution}. In this paper, we propose a novel approach SODM (Semi-supervised Optimal margin Distribution Machine), which tries to assign the label to unlabeled instances and achieve optimal margin distribution simultaneously. Specifically, we characterize the margin distribution by the first- and second-order statistics, i.e., the margin mean and variance, and extend a stochastic mirror prox method to solve the resultant minimax problem. Extensive experiments on UCI data sets show that SODM is significantly better than compared methods, which verifies the superiority of optimal margin distribution learning.

Download Full-text

Facial expression recognition algorithm fused ICA and support vector clustering

Journal of Computer Applications ◽

10.3724/sp.j.1087.2011.01605 ◽

2012 ◽

Vol 31 (6) ◽

pp. 1605-1608 ◽

Cited By ~ 1

Author(s):

Shu-ren ZHOU ◽

Xi-ming LIANG

Keyword(s):

Facial Expression ◽

Facial Expression Recognition ◽

Recognition Algorithm ◽

Support Vector ◽

Expression Recognition ◽

Support Vector Clustering ◽

Vector Clustering

Download Full-text

Multi-parameter Radar Signal Sorting Method Based on Fast Support Vector Clustering and Similitude Entropy

JOURNAL OF ELECTRONICS INFORMATION TECHNOLOGY ◽

10.3724/sp.j.1146.2011.00261 ◽

2011 ◽

Vol 33 (11) ◽

pp. 2735-2741 ◽

Cited By ~ 3

Author(s):

Shi-qiang Wang ◽

Deng-fu Zhang ◽

Du-yan Bi ◽

Xiao-ju Yong

Keyword(s):

Radar Signal ◽

Support Vector ◽

Sorting Method ◽

Support Vector Clustering ◽

Vector Clustering

Download Full-text

Sparse Twin Support Vector Clustering using Pinball Loss

IEEE Journal of Biomedical and Health Informatics ◽

10.1109/jbhi.2021.3059910 ◽

2021 ◽

pp. 1-1 ◽

Cited By ~ 1

Author(s):

M. Tanveer ◽

Tarun Gupta ◽

Miten Shah ◽

Bharat Richhariya

Keyword(s):

Support Vector ◽

Support Vector Clustering ◽

Pinball Loss ◽

Vector Clustering

Download Full-text

Pinball Loss Twin Support Vector Clustering

ACM Transactions on Multimedia Computing Communications and Applications ◽

10.1145/3409264 ◽

2021 ◽

Vol 17 (2s) ◽

pp. 1-23

Author(s):

M. Tanveer ◽

Tarun Gupta ◽

Miten Shah ◽

Keyword(s):

Loss Function ◽

Clustering Algorithm ◽

Clustering Algorithms ◽

Structural Mri ◽

Twin Support Vector Machine ◽

Support Vector ◽

Support Vector Clustering ◽

Hinge Loss ◽

Pinball Loss ◽

Vector Clustering

Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and artificial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noise-corrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model.

Download Full-text