Rough margin-based ν-twin support tensor machine in pattern recognition

2021 ◽  
Vol 40 (1) ◽  
pp. 685-702
Author(s):  
Huiru Wang ◽  
Zhijian Zhou
Keyword(s):  
Optimization Problem ◽  
Small Sample Size ◽  
Classification Performance ◽  
Small Sample ◽  
Twin Support Vector Machine ◽  
Support Vector ◽  
Rough Boundary ◽  
Lower Margin ◽  
Rough Theory ◽  
Overfitting Problem

 In Rough margin-based ν-Twin Support Vector Machine (Rν-TSVM) algorithm, the rough theory is introduced. Rν-TSVM gives different penalties to the corresponding misclassified samples according to their positions, so it avoids the overfitting problem to some extent. While the input data is a tensor, Rν-TSVM cannot handle it directly and may not utilize the data information effectively. Therefore, we propose a novel classifier based on tensor data, termed as Rough margin-based ν-Twin Support Tensor Machine (Rν-TSTM). Similar to Rν-TSVM, Rν-TSTM constructs rough lower margin, rough upper margin and rough boundary in tensor space. Rν-TSTM not only retains the superiority of Rν-TSVM, but also has its unique advantages. Firstly, the data topology is retained more efficiently by the direct use of tensor representation. Secondly, it has better classification performance compared to other classification algorithms. Thirdly, it can avoid overfitting problem to a great extent. Lastly, it is more suitable for high dimensional and small sample size problem. To solve the corresponding optimization problem in Rν-TSTM, we adopt the alternating iteration method in which the parameters corresponding to the hyperplanes are estimated by solving a series of Rν-TSVM optimization problem. The efficiency and superiority of the proposed method are demonstrated by computational experiments.

Twin Bounded Support Tensor Machine for Classification

2015 ◽  
Vol 30 (01) ◽  
pp. 1650002 ◽  
Author(s):  
Haifa Shi ◽  
Xinbin Zhao ◽  
Ling Zhen ◽  
Ling Jing
Keyword(s):  
Support Vector Machine ◽  
Optimization Problems ◽  
Small Sample Size ◽  
Computational Cost ◽  
Small Sample ◽  
Twin Support Vector Machine ◽  
Support Vector ◽  
Structure Information ◽  
Bounded Support ◽  
Tensor Data

The traditional vector-based classifiers, such as support vector machine (SVM) and twin support vector machine (TSVM), cannot handle tensor data directly and may not utilize the data informations effectively. In this paper, we propose a novel classifier based on tensor data, called twin bounded support tensor machine (TBSTM) which is an extension of twin bounded support vector machine (TBSVM). Similar to TBSVM, TBSTM gets two hyperplanes and obtains the solution by solving two quadratic programming problems (QPPs). The computational complexity of each QPPs is smaller than that of support tensor machine (STM). TBSTM not only retains the advantage of TBSVM, but also has its unique superior characteristics: (1) it makes full use of the structure information of data; (2) it has acceptable or better classification accuracy compared to STM, TBSVM and SVM; (3) the computational cost is basically less than STM; (4) it can deal with large data that TBSVM is not easy to achieve, especially for small-sample-size (S3) problems; (5) it adopts alternating successive over relaxation iteration (ASOR) method to solve optimization problems which accelerates the pace of training. Finally, we demonstrate the effectiveness and superiority by the experiments based on vector and tensor data.

scholarly journals A New Nearest Centroid Neighbor Classifier Based on K Local Means Using Harmonic Mean Distance

Information ◽  
2018 ◽  
Vol 9 (9) ◽  
pp. 234 ◽  
Author(s):  
Sumet Mehta ◽  
Xiangjun Shen ◽  
Jiangping Gou ◽  
Dejiao Niu
Keyword(s):  
Small Sample Size ◽  
Classification Performance ◽  
Error Rates ◽  
Small Sample ◽  
Harmonic Mean ◽  
Local Means ◽  
Local Mean ◽  
Real World Datasets ◽  
Nearest Neighbour Classifier ◽  
Query Sample

The K-nearest neighbour classifier is very effective and simple non-parametric technique in pattern classification; however, it only considers the distance closeness, but not the geometricalplacement of the k neighbors. Also, its classification performance is highly influenced by the neighborhood size k and existing outliers. In this paper, we propose a new local mean based k-harmonic nearest centroid neighbor (LMKHNCN) classifier in orderto consider both distance-based proximity, as well as spatial distribution of k neighbors. In our method, firstly the k nearest centroid neighbors in each class are found which are used to find k different local mean vectors, and then employed to compute their harmonic mean distance to the query sample. Lastly, the query sample is assigned to the class with minimum harmonic mean distance. The experimental results based on twenty-six real-world datasets shows that the proposed LMKHNCN classifier achieves lower error rates, particularly in small sample-size situations, and that it is less sensitive to parameter k when compared to therelated four KNN-based classifiers.

scholarly journals An Ensemble Classification Method for High-Dimensional Data Using Neighborhood Rough Set

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Jing Zhang ◽  
Guang Lu ◽  
Jiaquan Li ◽  
Chuanwen Li
Keyword(s):  
Feature Selection ◽  
Rough Set ◽  
Small Sample Size ◽  
High Dimensional Data ◽  
Classification Performance ◽  
Small Sample ◽  
Ensemble Classification ◽  
High Dimensional ◽  
Sample Classification ◽  
Neighborhood Rough Set

Mining useful knowledge from high-dimensional data is a hot research topic. Efficient and effective sample classification and feature selection are challenging tasks due to high dimensionality and small sample size of microarray data. Feature selection is necessary in the process of constructing the model to reduce time and space consumption. Therefore, a feature selection model based on prior knowledge and rough set is proposed. Pathway knowledge is used to select feature subsets, and rough set based on intersection neighborhood is then used to select important feature in each subset, since it can select features without redundancy and deals with numerical features directly. In order to improve the diversity among base classifiers and the efficiency of classification, it is necessary to select part of base classifiers. Classifiers are grouped into several clusters by k-means clustering using the proposed combination distance of Kappa-based diversity and accuracy. The base classifier with the best classification performance in each cluster will be selected to generate the final ensemble model. Experimental results on three Arabidopsis thaliana stress response datasets showed that the proposed method achieved better classification performance than existing ensemble models.

scholarly journals A Complete Subspace Analysis of Linear Discriminant Analysis and Its Robust Implementation

2016 ◽  
Vol 2016 ◽  
pp. 1-10
Author(s):  
Zhicheng Lu ◽  
Zhizheng Liang
Keyword(s):  
Discriminant Analysis ◽  
Linear Discriminant Analysis ◽  
Null Space ◽  
Small Sample Size ◽  
Classification Performance ◽  
Optimization Criterion ◽  
Small Sample ◽  
Scatter Matrix ◽  
Linear Discriminant ◽  
Maximum Margin Criterion

Linear discriminant analysis has been widely studied in data mining and pattern recognition. However, when performing the eigen-decomposition on the matrix pair (within-class scatter matrix and between-class scatter matrix) in some cases, one can find that there exist some degenerated eigenvalues, thereby resulting in indistinguishability of information from the eigen-subspace corresponding to some degenerated eigenvalue. In order to address this problem, we revisit linear discriminant analysis in this paper and propose a stable and effective algorithm for linear discriminant analysis in terms of an optimization criterion. By discussing the properties of the optimization criterion, we find that the eigenvectors in some eigen-subspaces may be indistinguishable if the degenerated eigenvalue occurs. Inspired from the idea of the maximum margin criterion (MMC), we embed MMC into the eigen-subspace corresponding to the degenerated eigenvalue to exploit discriminability of the eigenvectors in the eigen-subspace. Since the proposed algorithm can deal with the degenerated case of eigenvalues, it not only handles the small-sample-size problem but also enables us to select projection vectors from the null space of the between-class scatter matrix. Extensive experiments on several face images and microarray data sets are conducted to evaluate the proposed algorithm in terms of the classification performance, and experimental results show that our method has smaller standard deviations than other methods in most cases.

scholarly journals Performance and Characteristics of Wearable Sensor Systems Discriminating and Classifying Older Adults According to Fall Risk: A Systematic Review

Sensors ◽  
2021 ◽  
Vol 21 (17) ◽  
pp. 5863 ◽  
Author(s):  
Annica Kristoffersson ◽  
Jiaying Du ◽  
Maria Ehn
Keyword(s):  
Systematic Review ◽  
Risk Assessment ◽  
Model Validation ◽  
Fall Risk ◽  
Small Sample Size ◽  
Classification Performance ◽  
Small Sample ◽  
Risk Of Bias ◽  
Classification Models ◽  
Fall Risk Assessment

Sensor-based fall risk assessment (SFRA) utilizes wearable sensors for monitoring individuals’ motions in fall risk assessment tasks. Previous SFRA reviews recommend methodological improvements to better support the use of SFRA in clinical practice. This systematic review aimed to investigate the existing evidence of SFRA (discriminative capability, classification performance) and methodological factors (study design, samples, sensor features, and model validation) contributing to the risk of bias. The review was conducted according to recommended guidelines and 33 of 389 screened records were eligible for inclusion. Evidence of SFRA was identified: several sensor features and three classification models differed significantly between groups with different fall risk (mostly fallers/non-fallers). Moreover, classification performance corresponding the AUCs of at least 0.74 and/or accuracies of at least 84% were obtained from sensor features in six studies and from classification models in seven studies. Specificity was at least as high as sensitivity among studies reporting both values. Insufficient use of prospective design, small sample size, low in-sample inclusion of participants with elevated fall risk, high amounts and low degree of consensus in used features, and limited use of recommended model validation methods were identified in the included studies. Hence, future SFRA research should further reduce risk of bias by continuously improving methodology.

scholarly journals Improving the Efficacy of Deep-Learning Models for Heart Beat Detection on Heterogeneous Datasets

2021 ◽  
Vol 8 (12) ◽  
pp. 193
Author(s):  
Andrea Bizzego ◽  
Giulio Gabrieli ◽  
Michelle Jin Yee Neoh ◽  
Gianluca Esposito
Keyword(s):  
Deep Learning ◽  
Small Sample Size ◽  
Classification Performance ◽  
Heart Beat ◽  
Small Sample ◽  
Electrocardiogram Signals ◽  
Heterogeneous Datasets ◽  
Beat Detection ◽  
Physiological Markers ◽  
Bioelectric Signals

Deep learning (DL) has greatly contributed to bioelectric signal processing, in particular to extract physiological markers. However, the efficacy and applicability of the results proposed in the literature is often constrained to the population represented by the data used to train the models. In this study, we investigate the issues related to applying a DL model on heterogeneous datasets. In particular, by focusing on heart beat detection from electrocardiogram signals (ECG), we show that the performance of a model trained on data from healthy subjects decreases when applied to patients with cardiac conditions and to signals collected with different devices. We then evaluate the use of transfer learning (TL) to adapt the model to the different datasets. In particular, we show that the classification performance is improved, even with datasets with a small sample size. These results suggest that a greater effort should be made towards the generalizability of DL models applied on bioelectric signals, in particular, by retrieving more representative datasets.

scholarly journals An investigation on the factors affecting machine learning classifications in gamma-ray astronomy

2020 ◽  
Vol 492 (4) ◽  
pp. 5377-5390 ◽  
Author(s):  
Shengda Luo ◽  
Alex P Leung ◽  
C Y Hui ◽  
K L Li
Keyword(s):  
Machine Learning ◽  
Gamma Ray ◽  
Small Sample Size ◽  
Classification Performance ◽  
Small Sample ◽  
Classification Model ◽  
Machine Learning Techniques ◽  
Large Area ◽  
Actual Performance ◽  
Statistical Fluctuations

ABSTRACT We have investigated a number of factors that can have significant impacts on the classification performance of gamma-ray sources detected by Fermi Large Area Telescope (LAT) with machine learning techniques. We show that a framework of automatic feature selection can construct a simple model with a small set of features that yields better performance over previous results. Secondly, because of the small sample size of the training/test sets of certain classes in gamma-ray, nested re-sampling and cross-validations are suggested for quantifying the statistical fluctuations of the quoted accuracy. We have also constructed a test set by cross-matching the identified active galactic nuclei (AGNs) and the pulsars (PSRs) in the Fermi-LAT 8-yr point source catalogue (4FGL) with those unidentified sources in the previous 3rd Fermi-LAT Source Catalog (3FGL). Using this cross-matched set, we show that some features used for building classification model with the identified source can suffer from the problem of covariate shift, which can be a result of various observational effects. This can possibly hamper the actual performance when one applies such model in classifying unidentified sources. Using our framework, both AGN/PSR and young pulsar (YNG)/millisecond pulsar (MSP) classifiers are automatically updated with the new features and the enlarged training samples in 4FGL catalogue incorporated. Using a two-layer model with these updated classifiers, we have selected 20 promising MSP candidates with confidence scores $\gt 98{{\ \rm per\ cent}}$ from the unidentified sources in 4FGL catalogue that can provide inputs for a multiwavelength identification campaign.

scholarly journals Optimization Performance Comparison of Three Different Group Intelligence Algorithms on a SVM for Hyperspectral Imagery Classification

2019 ◽  
Vol 11 (6) ◽  
pp. 734 ◽  
Author(s):  
Xiufang Zhu ◽  
Nan Li ◽  
Yaozhong Pan
Keyword(s):  
Convergence Rate ◽  
Sample Size ◽  
Parameter Optimization ◽  
Classification Accuracy ◽  
Small Sample Size ◽  
Original Data ◽  
Performance Comparison ◽  
Small Sample ◽  
Support Vector ◽  
Remote Sensing Classification

Group intelligence algorithms have been widely used in support vector machine (SVM) parameter optimization due to their obvious characteristics of strong parallel processing ability, fast optimization, and global optimization. However, few studies have made optimization performance comparisons of different group intelligence algorithms on SVMs, especially in terms of their application to hyperspectral remote sensing classification. In this paper, we compare the optimization performance of three different group intelligence algorithms that were run on a SVM in terms of five aspects by using three hyperspectral images (one each of the Indian Pines, University of Pavia, and Salinas): the stability to parameter settings, convergence rate, feature selection ability, sample size, and classification accuracy. Particle swarm optimization (PSO), genetic algorithms (GAs), and artificial bee colony (ABC) algorithms are the three group intelligence algorithms. Our results showed the influence of these three optimization algorithms on the C-parameter optimization of the SVM was less than their influence on the σ-parameter. The convergence rate, the number of selected features, and the accuracy of the three group intelligence algorithms were statistically significant different at the p = 0.01 level. The GA algorithm could compress more than 70% of the original data and it was the least affected by sample size. GA-SVM had the highest average overall accuracy (91.77%), followed by ABC-SVM (88.73%), and PSO-SVM (86.65%). Especially, in complex scenes (e.g., the Indian Pines image), GA-SVM showed the highest classification accuracy (87.34%, which was 8.23% higher than ABC-SVM and 16.42% higher than PSO-SVM) and the best stability (the standard deviation of its classification accuracy was 0.82%, which was 5.54% lower than ABC-SVM, and 21.63% lower than PSO-SVM). Therefore, when compared with the ABC and PSO algorithms, the GA had more advantages in terms of feature band selection, small sample size classification, and classification accuracy.

Study on the Self-Organize Selective Fusion Support Vector Machine Algorithm

2011 ◽  
Vol 282-283 ◽  
pp. 165-168
Author(s):  
Yong Ming Cai ◽  
Qing Chang
Keyword(s):  
Support Vector Machine ◽  
Classification Performance ◽  
Small Sample ◽  
Support Vector ◽  
Support Vector Machine Algorithm ◽  
Multiple Classifiers ◽  
Memory Overhead ◽  
Selective Fusion ◽  
Statistical Learning Method ◽  
Sample Support

As a major statistical learning method in case of small sample, Support Vector Machine Algorithm (SVM) has some disadvantages in dealing with vast amounts of data, such as the memory overhead and slow training. we use Multi-class Support Vector Machine (MSVM) with Self-Organize Selective Fusion (SOSF) to optimize the multiple classifiers selectively, which can update the classification and self-adjust its classification performance, and eliminate some redundancy and conflicts, achieve the fusion of multiple classifiers selectively, and effectively solve the shortcoming of disturbances by the sub-samples distribution in large sample, and improve the training efficiency and classification efficiency.

The Support Feature Machine: Classification with the Least Number of Features and Application to Neuroimaging Data

2013 ◽  
Vol 25 (6) ◽  
pp. 1548-1584 ◽  
Author(s):  
Sascha Klement ◽  
Silke Anders ◽  
Thomas Martinetz
Keyword(s):  
Feature Selection ◽  
Small Sample Size ◽  
Small Sample ◽  
Biological Data ◽  
Support Vector ◽  
Data Sets ◽  
Universal Method ◽  
Data Set ◽  
Separating Hyperplane ◽  
New Formulation

By minimizing the zero-norm of the separating hyperplane, the support feature machine (SFM) finds the smallest subspace (the least number of features) of a data set such that within this subspace, two classes are linearly separable without error. This way, the dimensionality of the data is more efficiently reduced than with support vector–based feature selection, which can be shown both theoretically and empirically. In this letter, we first provide a new formulation of the previously introduced concept of the SFM. With this new formulation, classification of unbalanced and nonseparable data is straightforward, which allows using the SFM for feature selection and classification in a large variety of different scenarios. To illustrate how the SFM can be used to identify both the smallest subset of discriminative features and the total number of informative features in biological data sets we apply repetitive feature selection based on the SFM to a functional magnetic resonance imaging data set. We suggest that these capabilities qualify the SFM as a universal method for feature selection, especially for high-dimensional small-sample-size data sets that often occur in biological and medical applications.

Sign in / Sign up

Export Citation Format

Share Document