Proportional Hybrid Mechanism for Population Based Feature Selection Algorithm

2017 ◽  
Vol 16 (05) ◽  
pp. 1309-1338 ◽  
Author(s):  
Pin Wang ◽  
Yongming Li ◽  
Bohan Chen ◽  
Xianling Hu ◽  
Jin Yan ◽  
...  
Keyword(s):  
Feature Selection ◽  
High Efficiency ◽  
Search Algorithm ◽  
Population Based ◽  
Time Cost ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Filter Model ◽  
Hybrid Mechanism ◽  
Selection Algorithms

Feature selection is an important research field for pattern classification, data mining, etc. Population-based optimization algorithms (POA) have high parallelism and are widely used as search algorithm for feature selection. Population-based feature selection algorithms (PFSA) involve compromise between precision and time cost. In order to optimize the PFSA, the feature selection models need to be improved. Feature selection algorithms broadly fall into two categories: the filter model and the wrapper model. The filter model is fast but less precise; while the wrapper model is more precise but generally computationally more intensive. In this paper, we proposed a new mechanism — proportional hybrid mechanism (PHM) to combine the advantages of filter and wrapper models. The mechanism can be applied in PFSA to improve their performance. Genetic algorithm (GA) has been applied in many kinds of feature selection problems as search algorithm because of its high efficiency and implicit parallelism. Therefore, GAs are used in this paper. In order to validate the mechanism, seven datasets from university of California Irvine (UCI) database and artificial toy datasets are tested. The experiments are carried out for different GAs, classifiers, and evaluation criteria, the results show that with the introduction of PHM, the GA-based feature selection algorithm can be improved in both time cost and classification accuracy. Moreover, the comparison of GA-based, PSO-based and some other feature selection algorithms demonstrate that the PHM can be used in other population-based feature selection algorithms and obtain satisfying results.

AN EFFICIENT FEATURE SELECTION ALGORITHM FOR COMPUTER-AIDED POLYP DETECTION

2006 ◽  
Vol 15 (06) ◽  
pp. 893-915 ◽  
Author(s):  
JIANG LI ◽  
JIANHUA YAO ◽  
RONALD M. SUMMERS ◽  
NICHOLAS PETRICK ◽  
MICHAEL T. MANRY ◽  
...  
Keyword(s):  
Feature Selection ◽  
Cross Validation ◽  
Search Algorithm ◽  
Piecewise Linear ◽  
Least Square ◽  
Support Vector ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Polyp Detection ◽  
Computer Aided

We present an efficient feature selection algorithm for computer aided detection (CAD) computed tomographic (CT) colonography. The algorithm (1) determines an appropriate piecewise linear network (PLN) model by cross validation, (2) applies the orthonormal least square (OLS) procedure to the PLN model utilizing a Modified Schmidt procedure, and (3) uses a floating search algorithm to select features that minimize the output variance. The undesirable "nesting effect" is prevented by the floating search approach, and the piecewise linear OLS procedure makes this algorithm very computationally efficient because the Modified Schmidt procedure only requires one data pass during the whole searching process. The selected features are compared to those obtained by other methods, through cross validation with support vector machines (SVMs).

A NOVEL FEATURE SELECTION ALGORITHM WITH SUPERVISED MUTUAL INFORMATION FOR CLASSIFICATION

2013 ◽  
Vol 22 (04) ◽  
pp. 1350027
Author(s):  
JAGANATHAN PALANICHAMY ◽  
KUPPUCHAMY RAMASAMY
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Feature Selection ◽  
Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class A ◽  
Selection Algorithms ◽  
The Relationship ◽  
Class Variable

Feature selection is essential in data mining and pattern recognition, especially for database classification. During past years, several feature selection algorithms have been proposed to measure the relevance of various features to each class. A suitable feature selection algorithm normally maximizes the relevancy and minimizes the redundancy of the selected features. The mutual information measure can successfully estimate the dependency of features on the entire sampling space, but it cannot exactly represent the redundancies among features. In this paper, a novel feature selection algorithm is proposed based on maximum relevance and minimum redundancy criterion. The mutual information is used to measure the relevancy of each feature with class variable and calculate the redundancy by utilizing the relationship between candidate features, selected features and class variables. The effectiveness is tested with ten benchmarked datasets available in UCI Machine Learning Repository. The experimental results show better performance when compared with some existing algorithms.

scholarly journals Improved Relief Weight Feature Selection Algorithm Based on Relief and Mutual Information

Information ◽  
2021 ◽  
Vol 12 (6) ◽  
pp. 228
Author(s):  
Hongbin Wang ◽  
Pengming Wang ◽  
Shengchun Deng ◽  
Haoran Li
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
High Efficiency ◽  
Feature Subset ◽  
Evaluation Function ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Relief Algorithm ◽  
Sequential Forward Search ◽  
Computation Speed

As the classic feature selection algorithm, the Relief algorithm has the advantages of simple computation and high efficiency, but the algorithm itself is limited to only dealing with binary classification problems, and the comprehensive distinguishing ability of the feature subsets composed of the former K features selected by the Relief algorithm is often redundant, as the algorithm cannot select the ideal feature subset. When calculating the correlation and redundancy between characteristics by mutual information, the computation speed is slow because of the high computational complexity and the method’s need to calculate the probability density function of the corresponding features. Aiming to solve the above problems, we first improve the weight of the Relief algorithm, so that it can be used to evaluate a set of candidate feature sets. Then we use the improved joint mutual information evaluation function to replace the basic mutual information computation and solve the problem of computation speed and correlation, and redundancy between features. Finally, a compound correlation feature selection algorithm based on Relief and joint mutual information is proposed using the evaluation function and the heuristic sequential forward search strategy. This algorithm can effectively select feature subsets with small redundancy and strong classification characteristics, and has the excellent characteristics of faster calculation speed.

scholarly journals A Feature Selection Algorithm Integrating Maximum Classification Information and Minimum Interaction Feature Dependency Information

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Li Zhang
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Information Gain ◽  
Small Sample ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class Labels ◽  
Minimum Interaction ◽  
Classification Information ◽  
Selection Algorithms

Feature selection is the key step in the analysis of high-dimensional small sample data. The core of feature selection is to analyse and quantify the correlation between features and class labels and the redundancy between features. However, most of the existing feature selection algorithms only consider the classification contribution of individual features and ignore the influence of interfeature redundancy and correlation. Therefore, this paper proposes a feature selection algorithm for nonlinear dynamic conditional relevance (NDCRFS) through the study and analysis of the existing feature selection algorithm ideas and method. Firstly, redundancy and relevance between features and between features and class labels are discriminated by mutual information, conditional mutual information, and interactive mutual information. Secondly, the selected features and candidate features are dynamically weighted utilizing information gain factors. Finally, to evaluate the performance of this feature selection algorithm, NDCRFS was validated against 6 other feature selection algorithms on three classifiers, using 12 different data sets, for variability and classification metrics between the different algorithms. The experimental results show that the NDCRFS method can improve the quality of the feature subsets and obtain better classification results.

Feature Selection Algorithm Using Relative Odds for Data Mining Classification

2020 ◽  
pp. 81-106 ◽  
Author(s):  
Donald Douglas Atsa'am
Keyword(s):  
Feature Selection ◽  
Binary Classification ◽  
Initial Step ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Classification Problems ◽  
Odds Ratios ◽  
Relative Odds ◽  
Importance Ranking ◽  
Selection Algorithms

A filter feature selection algorithm is developed and its performance tested. In the initial step, the algorithm dichotomizes the dataset then separately computes the association between each predictor and the class variable using relative odds (odds ratios). The value of the odds ratios becomes the importance ranking of the corresponding explanatory variable in determining the output. Logistic regression classification is deployed to test the performance of the new algorithm in comparison with three existing feature selection algorithms: the Fisher index, Pearson's correlation, and the varImp function. A number of experimental datasets are employed, and in most cases, the subsets selected by the new algorithm produced models with higher classification accuracy than the subsets suggested by the existing feature selection algorithms. Therefore, the proposed algorithm is a reliable alternative in filter feature selection for binary classification problems.

scholarly journals Feature Selection Combining Information Theory View and Algebraic View in the Neighborhood Decision System

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 704
Author(s):  
Jiucheng Xu ◽  
Kanglin Qu ◽  
Meng Yuan ◽  
Jie Yang
Keyword(s):  
Information Theory ◽  
Feature Selection ◽  
Rough Set ◽  
Rough Set Theory ◽  
Classification Performance ◽  
Joint Entropy ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Selection Algorithms ◽  
Theory View

Feature selection is one of the core contents of rough set theory and application. Since the reduction ability and classification performance of many feature selection algorithms based on rough set theory and its extensions are not ideal, this paper proposes a feature selection algorithm that combines the information theory view and algebraic view in the neighborhood decision system. First, the neighborhood relationship in the neighborhood rough set model is used to retain the classification information of continuous data, to study some uncertainty measures of neighborhood information entropy. Second, to fully reflect the decision ability and classification performance of the neighborhood system, the neighborhood credibility and neighborhood coverage are defined and introduced into the neighborhood joint entropy. Third, a feature selection algorithm based on neighborhood joint entropy is designed, which improves the disadvantage that most feature selection algorithms only consider information theory definition or algebraic definition. Finally, experiments and statistical analyses on nine data sets prove that the algorithm can effectively select the optimal feature subset, and the selection result can maintain or improve the classification performance of the data set.

scholarly journals A Feature Selection Algorithm Performance Metric for Comparative Analysis

Algorithms ◽  
2021 ◽  
Vol 14 (3) ◽  
pp. 100
Author(s):  
Werner Mostert ◽  
Katherine M. Malan ◽  
Andries P. Engelbrecht
Keyword(s):  
Feature Selection ◽  
Comparative Analysis ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Algorithm Selection ◽  
Algorithm Performance ◽  
Performance Space ◽  
Performance Metric ◽  
Real World Datasets ◽  
Selection Algorithms

This study presents a novel performance metric for feature selection algorithms that is unbiased and can be used for comparative analysis across feature selection problems. The baseline fitness improvement (BFI) measure quantifies the potential value gained by applying feature selection. The BFI measure can be used to compare the performance of feature selection algorithms across datasets by measuring the change in classifier performance as a result of feature selection, with respect to the baseline where all features are included. Empirical results are presented to show that there is performance complementarity for a suite of feature selection algorithms on a variety of real world datasets. The BFI measure is a normalised performance metric that can be used to correlate problem characteristics with feature selection algorithm performance, across multiple datasets. This ability paves the way towards describing the performance space of the per-instance algorithm selection problem for feature selection algorithms.

A dynamic recursive feature elimination framework (dRFE) to further refine a set of OMIC biomarkers

2021 ◽  
Author(s):  
Yuanyuan Han ◽  
Lan Huang ◽  
Fengfeng Zhou
Keyword(s):  
Experimental Data ◽  
Feature Selection ◽  
The Other ◽  
Supplementary Information ◽  
Recursive Feature Elimination ◽  
Supplementary Data ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class Labels ◽  
Selection Algorithms

Abstract Motivation A feature selection algorithm may select the subset of features with the best associations with the class labels. The recursive feature elimination (RFE) is a heuristic feature screening framework and has been widely used to select the biological OMIC biomarkers. This study proposed a dynamic recursive feature elimination (dRFE) framework with more flexible feature elimination operations. The proposed dRFE was comprehensively compared with 11 existing feature selection algorithms and five classifiers on the eight difficult transcriptome datasets from a previous study, the ten newly collected transcriptome datasets and the five methylome datasets. Results The experimental data suggested that the regular RFE framework did not perform well, and dRFE outperformed the existing feature selection algorithms in most cases. The dRFE-detected features achieved Acc = 1.0000 for the two methylome datasets GSE53045 and GSE66695. The best prediction accuracies of the dRFE-detected features were 0.9259, 0.9424 and 0.8601 for the other three methylome datasets GSE74845, GSE103186 and GSE80970, respectively. Four transcriptome datasets received Acc = 1.0000 using the dRFE-detected features, and the prediction accuracies for the other six newly collected transcriptome datasets were between 0.6301 and 0.9917. Availability and implementation The experiments in this study are implemented and tested using the programming language Python version 3.7.6. Supplementary information Supplementary data are available at Bioinformatics online.

Sign in / Sign up

Export Citation Format

Share Document