Dimensionality Reduction with Unsupervised Feature Selection and Applying Non-Euclidean Norms for Classification Accuracy

2011 ◽  
pp. 91-109
Author(s):  
Amit Saxena ◽  
John Wang
Keyword(s):  
Classification Accuracy ◽  
Nearest Neighbor ◽  
Fitness Function ◽  
Synthetic Data ◽  
Feature Subset Selection ◽  
Second Phase ◽  
Data Sets ◽  
Feature Subset ◽  
K Nearest Neighbor ◽  
Two Phase

This paper presents a two-phase scheme to select reduced number of features from a dataset using Genetic Algorithm (GA) and testing the classification accuracy (CA) of the dataset with the reduced feature set. In the first phase of the proposed work, an unsupervised approach to select a subset of features is applied. GA is used to select stochastically reduced number of features with Sammon Error as the fitness function. Different subsets of features are obtained. In the second phase, each of the reduced features set is applied to test the CA of the dataset. The CA of a data set is validated using supervised k-nearest neighbor (k-nn) algorithm. The novelty of the proposed scheme is that each reduced feature set obtained in the first phase is investigated for CA using the k-nn classification with different Minkowski metric i.e. non-Euclidean norms instead of conventional Euclidean norm (L2). Final results are presented in the paper with extensive simulations on seven real and one synthetic, data sets. It is revealed from the proposed investigation that taking different norms produces better CA and hence a scope for better feature subset selection.

Dimensionality Reduction with Unsupervised Feature Selection and Applying Non-Euclidean Norms for Classification Accuracy

2010 ◽  
Vol 6 (2) ◽  
pp. 22-40 ◽  
Author(s):  
Amit Saxena ◽  
John Wang
Keyword(s):  
Classification Accuracy ◽  
Nearest Neighbor ◽  
Fitness Function ◽  
Feature Subset Selection ◽  
Second Phase ◽  
Feature Subset ◽  
K Nearest Neighbor ◽  
Two Phase ◽  
Data Set ◽  
Minkowski Metric

This paper presents a two-phase scheme to select reduced number of features from a dataset using Genetic Algorithm (GA) and testing the classification accuracy (CA) of the dataset with the reduced feature set. In the first phase of the proposed work, an unsupervised approach to select a subset of features is applied. GA is used to select stochastically reduced number of features with Sammon Error as the fitness function. Different subsets of features are obtained. In the second phase, each of the reduced features set is applied to test the CA of the dataset. The CA of a data set is validated using supervised k-nearest neighbor (k-nn) algorithm. The novelty of the proposed scheme is that each reduced feature set obtained in the first phase is investigated for CA using the k-nn classification with different Minkowski metric i.e. non-Euclidean norms instead of conventional Euclidean norm (L2). Final results are presented in the paper with extensive simulations on seven real and one synthetic, data sets. It is revealed from the proposed investigation that taking different norms produces better CA and hence a scope for better feature subset selection.

Optimized Feature Subset Selection and Relevance Feedback for Image Retrieval Based on Multiresolution Enhanced Orthogonal Polynomials Model

2015 ◽  
Vol 6 (2) ◽  
pp. 25-40
Author(s):  
S. Sathiya Devi
Keyword(s):  
Image Retrieval ◽  
Orthogonal Polynomials ◽  
Relevance Feedback ◽  
Nearest Neighbor ◽  
Image Features ◽  
Feature Subset Selection ◽  
Support Vector ◽  
Feature Subset ◽  
K Nearest Neighbor ◽  
Multi Objective Genetic Algorithm

In this paper, a simple image retrieval method incorporating relevance feedback based on the multiresolution enhanced orthogonal polynomials model is proposed. In the proposed method, the low level image features such as texture, shape and color are extracted from the reordered orthogonal polynomials model coefficients and linearly combined to form a multifeature set. Then the dimensionality of the multifeature set is reduced by utilizing multi objective Genetic Algorithm (GA) and multiclass binary Support Vector Machine (SVM). The obtained optimized multifeature set is used for image retrieval. In order to improve the retrieval accuracy and to bridge the semantic gap, a correlation based k-Nearest Neighbor (k-NN) method for relevance feedback is also proposed. In this method, an appropriate relevance score is computed for each image in the database based on relevant and non relevant set chosen by the user with correlation based k-NN method. The experiments are carried out with Corel and Caltech database images and the retrieval rates are computed. The proposed method with correlation based k-NN for relevance feedback gives an average retrieval rate of 94.67%.

scholarly journals A Hybrid Feature Selection Method for Improve the Accuracy of Medical Classification Process

2021 ◽  
Vol 11 (1) ◽  
pp. 50-55
Author(s):  
Maria Mohammad Yousef ◽  
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Classification Accuracy ◽  
Fitness Function ◽  
Machine Learning Algorithms ◽  
Feature Subset Selection ◽  
High Dimensionality ◽  
Support Vector ◽  
Feature Subset

Generally, medical dataset classification has become one of the biggest problems in data mining research. Every database has a given number of features but it is observed that some of these features can be redundant and can be harmful as well as disrupt the process of classification and this problem is known as a high dimensionality problem. Dimensionality reduction in data preprocessing is critical for increasing the performance of machine learning algorithms. Besides the contribution of feature subset selection in dimensionality reduction gives a significant improvement in classification accuracy. In this paper, we proposed a new hybrid feature selection approach based on (GA assisted by KNN) to deal with issues of high dimensionality in biomedical data classification. The proposed method first applies the combination between GA and KNN for feature selection to find the optimal subset of features where the classification accuracy of the k-Nearest Neighbor (kNN) method is used as the fitness function for GA. After selecting the best-suggested subset of features, Support Vector Machine (SVM) are used as the classifiers. The proposed method experiments on five medical datasets of the UCI Machine Learning Repository. It is noted that the suggested technique performs admirably on these databases, achieving higher classification accuracy while using fewer features.

Clustering Based on a Novel Density Estimation Method

2013 ◽  
Vol 748 ◽  
pp. 590-594
Author(s):  
Li Liao ◽  
Yong Gang Lu ◽  
Xu Rong Chen
Keyword(s):  
Density Estimation ◽  
Nearest Neighbor ◽  
Mean Shift ◽  
Estimation Method ◽  
Synthetic Data ◽  
Real Data ◽  
Data Sets ◽  
Clustering Methods ◽  
K Nearest Neighbor ◽  
Data Set

We propose a novel density estimation method using both the k-nearest neighbor (KNN) graph and the potential field of the data points to capture the local and global data distribution information respectively. The clustering is performed based on the computed density values. A forest of trees is built using each data point as the tree node. And the clusters are formed according to the trees in the forest. The new clustering method is evaluated by comparing with three popular clustering methods, K-means++, Mean Shift and DBSCAN. Experiments on two synthetic data sets and one real data set show that our approach can effectively improve the clustering results.

scholarly journals A Feature Subset Selection Algorithm Automatic Recommendation Method

2013 ◽  
Vol 47 ◽  
pp. 1-34 ◽  
Author(s):  
G. Wang ◽  
Q. Song ◽  
H. Sun ◽  
X. Zhang ◽  
B. Xu ◽  
...  
Keyword(s):  
Feature Selection ◽  
Subset Selection ◽  
Feature Subset Selection ◽  
Data Sets ◽  
Feature Subset ◽  
Similar Data ◽  
K Nearest Neighbor ◽  
Real World Data ◽  
Feature Selection Problem ◽  
Data Set

Many feature subset selection (FSS) algorithms have been proposed, but not all of them are appropriate for a given feature selection problem. At the same time, so far there is rarely a good way to choose appropriate FSS algorithms for the problem at hand. Thus, FSS algorithm automatic recommendation is very important and practically useful. In this paper, a meta learning based FSS algorithm automatic recommendation method is presented. The proposed method first identifies the data sets that are most similar to the one at hand by the k-nearest neighbor classification algorithm, and the distances among these data sets are calculated based on the commonly-used data set characteristics. Then, it ranks all the candidate FSS algorithms according to their performance on these similar data sets, and chooses the algorithms with best performance as the appropriate ones. The performance of the candidate FSS algorithms is evaluated by a multi-criteria metric that takes into account not only the classification accuracy over the selected features, but also the runtime of feature selection and the number of selected features. The proposed recommendation method is extensively tested on 115 real world data sets with 22 well-known and frequently-used different FSS algorithms for five representative classifiers. The results show the effectiveness of our proposed FSS algorithm recommendation method.

The Influence of Hubness on NN-Descent

2019 ◽  
Vol 28 (06) ◽  
pp. 1960002 ◽  
Author(s):  
Brankica Bratić ◽  
Michael E. Houle ◽  
Vladimir Kurbalija ◽  
Vincent Oria ◽  
Miloš Radovanović
Keyword(s):  
Nearest Neighbor ◽  
Synthetic Data ◽  
Machine Learning Algorithms ◽  
Data Sets ◽  
Accurate Approximation ◽  
K Nearest Neighbor ◽  
Major Drawback ◽  
Neighbor Graph ◽  
Nearest Neighbor Graph ◽  
The Cost

The K-nearest neighbor graph (K-NNG) is a data structure used by many machine-learning algorithms. Naive computation of the K-NNG has quadratic time complexity, which in many cases is not efficient enough, producing the need for fast and accurate approximation algorithms. NN-Descent is one such algorithm that is highly efficient, but has a major drawback in that K-NNG approximations are accurate only on data of low intrinsic dimensionality. This paper represents an experimental analysis of this behavior, and investigates possible solutions. Experimental results show that there is a link between the performance of NN-Descent and the phenomenon of hubness, defined as the tendency of intrinsically high-dimensional data to contain hubs – points with large in-degrees in the K-NNG. First, we explain how the presence of the hubness phenomenon causes bad NN-Descent performance. In light of that, we propose four NN-Descent variants to alleviate the observed negative inuence of hubs. By evaluating the proposed approaches on several real and synthetic data sets, we conclude that our approaches are more accurate, but often at the cost of higher scan rates.

The Classification of Skateboarding Tricks : A Transfer Learning and Machine Learning Approach

Mekatronika ◽  
2020 ◽  
Vol 2 (2) ◽  
pp. 1-12
Author(s):  
Muhammad Nur Aiman Shapiee ◽  
Muhammad Ar Rahim Ibrahim ◽  
Muhammad Amirul Abdullah ◽  
Rabiu Muazu Musa ◽  
Noor Azuan Abu Osman ◽  
...  
Keyword(s):  
Machine Learning ◽  
Classification Accuracy ◽  
Nearest Neighbor ◽  
Olympic Games ◽  
Learning Approach ◽  
K Nearest Neighbor ◽  
Test Dataset ◽  
Machine Learning Approach ◽  
Competitive Games

The skateboarding scene has arrived at new statures, particularly with its first appearance at the now delayed Tokyo Summer Olympic Games. Hence, attributable to the size of the game in such competitive games, progressed creative appraisal approaches have progressively increased due consideration by pertinent partners, particularly with the enthusiasm of a more goal-based assessment. This study purposes for classifying skateboarding tricks, specifically Frontside 180, Kickflip, Ollie, Nollie Front Shove-it, and Pop Shove-it over the integration of image processing, Trasnfer Learning (TL) to feature extraction enhanced with tradisional Machine Learning (ML) classifier. A male skateboarder performed five tricks every sort of trick consistently and the YI Action camera captured the movement by a range of 1.26 m. Then, the image dataset were features built and extricated by means of  three TL models, and afterward in this manner arranged to utilize by k-Nearest Neighbor (k-NN) classifier. The perception via the initial experiments showed, the MobileNet, NASNetMobile, and NASNetLarge coupled with optimized k-NN classifiers attain a classification accuracy (CA) of 95%, 92% and 90%, respectively on the test dataset. Besides, the result evident from the robustness evaluation showed the MobileNet+k-NN pipeline is more robust as it could provide a decent average CA than other pipelines. It would be demonstrated that the suggested study could characterize the skateboard tricks sufficiently and could, over the long haul, uphold judges decided for giving progressively objective-based decision.

scholarly journals Determination of Reactivity Ratios from Binary Copolymerization Using the k-Nearest Neighbor Non-Parametric Regression

Polymers ◽  
2021 ◽  
Vol 13 (21) ◽  
pp. 3811
Author(s):  
Iosif Sorin Fazakas-Anca ◽  
Arina Modrea ◽  
Sorin Vlase
Keyword(s):  
Experimental Data ◽  
Nearest Neighbor ◽  
Optimization Method ◽  
Reactivity Ratios ◽  
Data Sets ◽  
K Nearest Neighbor ◽  
Integration Algorithm ◽  
Data Set ◽  
Parametric Regression ◽  
Non Parametric

This paper proposes a new method for calculating the monomer reactivity ratios for binary copolymerization based on the terminal model. The original optimization method involves a numerical integration algorithm and an optimization algorithm based on k-nearest neighbour non-parametric regression. The calculation method has been tested on simulated and experimental data sets, at low (<10%), medium (10–35%) and high conversions (>40%), yielding reactivity ratios in a good agreement with the usual methods such as intersection, Fineman–Ross, reverse Fineman–Ross, Kelen–Tüdös, extended Kelen–Tüdös and the error in variable method. The experimental data sets used in this comparative analysis are copolymerization of 2-(N-phthalimido) ethyl acrylate with 1-vinyl-2-pyrolidone for low conversion, copolymerization of isoprene with glycidyl methacrylate for medium conversion and copolymerization of N-isopropylacrylamide with N,N-dimethylacrylamide for high conversion. Also, the possibility to estimate experimental errors from a single experimental data set formed by n experimental data is shown.

scholarly journals A novel feature selection algorithm based on damping oscillation theory

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0255307
Author(s):  
Fujun Wang ◽  
Xing Wang
Keyword(s):  
Feature Selection ◽  
Optimization Algorithm ◽  
Euclidean Distance ◽  
Oscillation Theory ◽  
Feature Subset Selection ◽  
Support Vector ◽  
Data Sets ◽  
Feature Subset ◽  
Selection Algorithm ◽  
Filter Model

Feature selection is an important task in big data analysis and information retrieval processing. It reduces the number of features by removing noise, extraneous data. In this paper, one feature subset selection algorithm based on damping oscillation theory and support vector machine classifier is proposed. This algorithm is called the Maximum Kendall coefficient Maximum Euclidean Distance Improved Gray Wolf Optimization algorithm (MKMDIGWO). In MKMDIGWO, first, a filter model based on Kendall coefficient and Euclidean distance is proposed, which is used to measure the correlation and redundancy of the candidate feature subset. Second, the wrapper model is an improved grey wolf optimization algorithm, in which its position update formula has been improved in order to achieve optimal results. Third, the filter model and the wrapper model are dynamically adjusted by the damping oscillation theory to achieve the effect of finding an optimal feature subset. Therefore, MKMDIGWO achieves both the efficiency of the filter model and the high precision of the wrapper model. Experimental results on five UCI public data sets and two microarray data sets have demonstrated the higher classification accuracy of the MKMDIGWO algorithm than that of other four state-of-the-art algorithms. The maximum ACC value of the MKMDIGWO algorithm is at least 0.5% higher than other algorithms on 10 data sets.

scholarly journals MpsLDA-ProSVM: predicting multi-label protein subcellular localization by wMLDAe dimensionality reduction and ProSVM classifier

2020 ◽  
Author(s):  
Qi Zhang ◽  
Shan Li ◽  
Bin Yu ◽  
Yang Li ◽  
Yandan Zhang ◽  
...  
Keyword(s):  
Subcellular Localization ◽  
Nearest Neighbor ◽  
Chemical Information ◽  
Sequence Information ◽  
Feature Subset ◽  
Protein Subcellular Localization ◽  
K Nearest Neighbor ◽  
Entropy Weight ◽  
Linear Discriminant ◽  
Optimal Feature Subset

ABSTRACTProteins play a significant part in life processes such as cell growth, development, and reproduction. Exploring protein subcellular localization (SCL) is a direct way to better understand the function of proteins in cells. Studies have found that more and more proteins belong to multiple subcellular locations, and these proteins are called multi-label proteins. They not only play a key role in cell life activities, but also play an indispensable role in medicine and drug development. This article first presents a new prediction model, MpsLDA-ProSVM, to predict the SCL of multi-label proteins. Firstly, the physical and chemical information, evolution information, sequence information and annotation information of protein sequences are fused. Then, for the first time, use a weighted multi-label linear discriminant analysis framework based on entropy weight form (wMLDAe) to refine and purify features, reduce the difficulty of learning. Finally, input the optimal feature subset into the multi-label learning with label-specific features (LIFT) and multi-label k-nearest neighbor (ML-KNN) algorithms to obtain a synthetic ranking of relevant labels, and then use Prediction and Relevance Ordering based SVM (ProSVM) classifier to predict the SCLs. This method can rank and classify related tags at the same time, which greatly improves the efficiency of the model. Tested by jackknife method, the overall actual accuracy (OAA) on virus, plant, Gram-positive bacteria and Gram-negative bacteria datasets are 98.06%, 98.97%, 99.81% and 98.49%, which are 0.56%-9.16%, 5.37%-30.87%, 3.51%-6.91% and 3.99%-8.59% higher than other advanced methods respectively. The source codes and datasets are available at https://github.com/QUST-AIBBDRC/MpsLDA-ProSVM/.

Sign in / Sign up

Export Citation Format

Share Document