Scalable and Flexible Unsupervised Feature Selection

2019 ◽  
Vol 31 (3) ◽  
pp. 517-537 ◽  
Author(s):  
Haojie Hu ◽  
Rong Wang ◽  
Xiaojun Yang ◽  
Feiping Nie
Keyword(s):  
Feature Selection ◽  
Large Scale ◽  
Graph Laplacian ◽  
Linear Constraint ◽  
Data Matrix ◽  
Data Sets ◽  
Unsupervised Feature Selection ◽  
Manifold Embedding ◽  
Public Data ◽  
Assignment Strategy

Recently, graph-based unsupervised feature selection algorithms (GUFS) have been shown to efficiently handle prevalent high-dimensional unlabeled data. One common drawback associated with existing graph-based approaches is that they tend to be time-consuming and in need of large storage, especially when faced with the increasing size of data. Research has started using anchors to accelerate graph-based learning model for feature selection, while the hard linear constraint between the data matrix and the lower-dimensional representation is usually overstrict in many applications. In this letter, we propose a flexible linearization model with anchor graph and [Formula: see text]-norm regularization, which can deal with large-scale data sets and improve the performance of the existing anchor-based method. In addition, the anchor-based graph Laplacian is constructed to characterize the manifold embedding structure by means of a parameter-free adaptive neighbor assignment strategy. An efficient iterative algorithm is developed to address the optimization problem, and we also prove the convergence of the algorithm. Experiments on several public data sets demonstrate the effectiveness and efficiency of the method we propose.

Unsupervised Feature Selection Based on Spectral Clustering with Maximum Relevancy and Minimum Redundancy Approach

2021 ◽  
Vol 35 (11) ◽  
pp. 2150031
Author(s):  
Bahareh Khozaei ◽  
Mahdi Eftekhari
Keyword(s):  
Feature Selection ◽  
Spectral Clustering ◽  
Information Gain ◽  
State Of The Art ◽  
Nearest Neighbors ◽  
Data Sets ◽  
Unsupervised Feature Selection ◽  
Significant Difference ◽  
Cluster A ◽  
Novel Approaches

In this paper, two novel approaches for unsupervised feature selection are proposed based on the spectral clustering. In the first proposed method, spectral clustering is employed over the features and the center of clusters is selected as well as their nearest-neighbors. These features have a minimum similarity (redundancy) between themselves since they belong to different clusters. Next, samples of data sets are clustered employing spectral clustering so that to the samples of each cluster a specific pseudo-label is assigned. After that according to the obtained pseudo-labels, the information gain of the features is computed that secures the maximum relevancy. Finally, the intersection of the selected features in the two previous steps is determined that simultaneously guarantees both the maximum relevancy and minimum redundancy. Our second proposed approach is very similar to the first one whose only but significant difference with the first method is that it selects one feature from each cluster and sorts all the features in terms of their relevancy. Then, by appending the selected features to a sorted list and ignoring them for the next step, the algorithm continues with the remaining features until all the features to be appended into the sorted list. Both of our proposed methods are compared with state-of-the-art methods and the obtained results confirm the performance of our proposed approaches especially the second one.

scholarly journals Unified Methods for Feature Selection in Large-Scale Genomic Studies with Censored Survival Outcomes

2020 ◽  
Author(s):  
Lauren Spirko-Burns ◽  
Karthik Devarajan
Keyword(s):  
Feature Selection ◽  
Large Scale ◽  
Proportional Hazards ◽  
Genomic Feature ◽  
Data Sets ◽  
Prognostic Impact ◽  
Genomic Features ◽  
Special Cases ◽  
Information Divergence ◽  
Genomic Studies

AbstractOne of the major goals in large-scale genomic studies is to identify genes with a prognostic impact on time-to-event outcomes which provide insight into the disease’s process. With rapid developments in high-throughput genomic technologies in the past two decades, the scientific community is able to monitor the expression levels of tens of thousands of genes and proteins resulting in enormous data sets where the number of genomic features is far greater than the number of subjects. Methods based on univariate Cox regression are often used to select genomic features related to survival outcome; however, the Cox model assumes proportional hazards (PH), which is unlikely to hold for each feature. When applied to genomic features exhibiting some form of non-proportional hazards (NPH), these methods could lead to an under- or over-estimation of the effects. We propose a broad array of marginal screening techniques that aid in feature ranking and selection by accommodating various forms of NPH. First, we develop an approach based on Kullback-Leibler information divergence and the Yang-Prentice model that includes methods for the PH and proportional odds (PO) models as special cases. Next, we propose R2 indices for the PH and PO models that can be interpreted in terms of explained randomness. Lastly, we propose a generalized pseudo-R2 measure that includes PH, PO, crossing hazards and crossing odds models as special cases and can be interpreted as the percentage of separability between subjects experiencing the event and not experiencing the event according to feature expression. We evaluate the performance of our measures using extensive simulation studies and publicly available data sets in cancer genomics. We demonstrate that the proposed methods successfully address the issue of NPH in genomic feature selection and outperform existing methods. The proposed information divergence, R2 and pseudo-R2 measures were implemented in R (www.R-project.org) and code is available upon request.

An Empirical Study on Initializing Centroid in K-Means Clustering for Feature Selection

2021 ◽  
Vol 13 (1) ◽  
pp. 1-16
Author(s):  
Amit Saxena ◽  
John Wang ◽  
Wutiphol Sintunavarat
Keyword(s):  
Data Mining ◽  
Feature Selection ◽  
Empirical Study ◽  
Data Sets ◽  
Data Set ◽  
Unsupervised Feature Selection ◽  
The Impact

One of the main problems in K-means clustering is setting of initial centroids which can cause misclustering of patterns which affects clustering accuracy. Recently, a density and distance-based technique for determining initial centroids has claimed a faster convergence of clusters. Motivated from this key idea, the authors study the impact of initial centroids on clustering accuracy for unsupervised feature selection. Three metrics are used to rank the features of a data set. The centroids of the clusters in the data sets, to be applied in K-means clustering, are initialized randomly as well as by density and distance-based approaches. Extensive experiments are performed on 15 datasets. The main significance of the paper is that the K-means clustering yields higher accuracies in majority of these datasets using proposed density and distance-based approach. As an impact of the paper, with fewer features, a good clustering accuracy can be achieved which can be useful in data mining of data sets with thousands of features.

COMBINING FEATURE SELECTION WITH EXTRACTION: UNSUPERVISED FEATURE SELECTION BASED ON PRINCIPAL COMPONENT ANALYSIS

2009 ◽  
Vol 18 (06) ◽  
pp. 883-904
Author(s):  
YUN LI ◽  
BAO-LIANG LU ◽  
TENG-FEI ZHANG
Keyword(s):  
Feature Selection ◽  
Principal Components ◽  
Nearest Neighbor ◽  
Dimensional Space ◽  
Image Data ◽  
Principal Component ◽  
Data Sets ◽  
K Nearest Neighbor ◽  
Unsupervised Feature Selection ◽  
Linear Feature

Principal components analysis (PCA) is a popular linear feature extractor, and widely used in signal processing, face recognition, etc. However, axes of the lower-dimensional space, i.e., principal components, are a set of new variables carrying no clear physical meanings. Thus we propose unsupervised feature selection algorithms based on eigenvectors analysis to identify critical original features for principal component. The presented algorithms are based on k-nearest neighbor rule to find the predominant row components and eight new measures are proposed to compute the correlation between row components in transformation matrix. Experiments are conducted on benchmark data sets and facial image data sets for gender classification to show their superiorities.

scholarly journals Unsupervised Feature Selection by Pareto Optimization

2019 ◽  
Vol 33 ◽  
pp. 3534-3541 ◽  
Author(s):  
Chao Feng ◽  
Chao Qian ◽  
Ke Tang
Keyword(s):  
Feature Selection ◽  
Reconstruction Error ◽  
Superior Performance ◽  
Data Matrix ◽  
Feature Transformation ◽  
Huge Number ◽  
Unsupervised Feature Selection ◽  
Column Subset Selection ◽  
Approximation Guarantee ◽  
Natural Formulation

Dimensionality reduction is often employed to deal with the data with a huge number of features, which can be generally divided into two categories: feature transformation and feature selection. Due to the interpretability, the efficiency during inference and the abundance of unlabeled data, unsupervised feature selection has attracted much attention. In this paper, we consider its natural formulation, column subset selection (CSS), which is to minimize the reconstruction error of a data matrix by selecting a subset of features. We propose an anytime randomized iterative approach POCSS, which minimizes the reconstruction error and the number of selected features simultaneously. Its approximation guarantee is well bounded. Empirical results exhibit the superior performance of POCSS over the state-of-the-art algorithms.

Sign in / Sign up

Export Citation Format

Share Document