Revisiting Feature Ranking Methods using Information-Centric and Evolutionary Approaches: Survey

2021 ◽  
Vol 11 ◽  
Author(s):  
Rashmi Gandhi ◽  
Udayan Ghose ◽  
Hardeo Kumar Thakur
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Differential Evolution ◽  
Role Models ◽  
Feature Ranking ◽  
Ranking Methods ◽  
Research Directions ◽  
Research Questions ◽  
Ranking Problems ◽  
The Impact

: Feature ranking can have a severe impact on the feature selection problem. Feature ranking methods refer to the structure of features that can accept the designed data and have a positive effect on the quality of features. Moreover, accessing useful features helps in reducing cost and improving performance of a feature ranking algorithm. There are numerous methods for ranking the features are available in literature. The developments of the past 20 years in the domain of knowledge research has been explored and presented in terms of relevance and various known concepts of feature ranking problems. The latest developments are mostly based on the evolutionary approaches which broadly include variations in ranking, mutual information, entropy, mutation, parent selection, genetic algorithm etc. For a variety of algorithms based on differential evolution, it observed that the suitability of the mutation operator is extremely important for feature selection but other operators can be considered. Therefore, the special emphasis of various algorithms is observing and reviewing the algorithms and to find new research directions.: The general approach is to do a rigorous collection of articles first and then obtain the most accurate and relevant data followed by the narrow down of research questions. Research is based on the research questions. These are reviewed in four phases : designing the review, conducting the review, analysis, and then writing the review. Threats to Validity is also considered with research questions. In this paper, many feature ranking methods have been discussed to find further direction in feature ranking and differential evolution. A literature survey is performed on 93 papers to find out the performance in relevance, redundancy, correlation with differential evolution. Discussion is suitable for cascading the direction of differential evolution in integration with information-theoretic, entropy and sparse learning. As differential evolution is multi-objective in nature so it can be incorporated with feature ranking problems. The survey is being conducted on many renowned journals and is verified with their research questions.Conclusions of the survey prove to be essential role models for multiple directions of a research entity. In this paper, a comprehensive view on the current-day understanding of the underlying mechanisms describing the impact of algorithms and review current and future research directions for use of evolutionary computations, mutual information and entropy in the field of feature ranking is complemented by the list of promising research directions. However, there are no strict rules for the pros and cons of alternative algorithms.

PET Imaging of Tau Pathology and Amyloid-β, and MRI for Alzheimer’s Disease Feature Fusion and Multimodal Classification

2021 ◽  
pp. 1-18
Author(s):  
Mehdi Shojaie ◽  
Solale Tabarestani ◽  
Mercedes Cabrerizo ◽  
Steven T. DeKosky ◽  
David E. Vaillancourt ◽  
...  
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Feature Selection ◽  
Mutual Information ◽  
Feature Fusion ◽  
Early Stage ◽  
Model Performance ◽  
Amyloid Β ◽  
Feature Ranking ◽  
Promising Tool

Background: Machine learning is a promising tool for biomarker-based diagnosis of Alzheimer’s disease (AD). Performing multimodal feature selection and studying the interaction between biological and clinical AD can help to improve the performance of the diagnosis models. Objective: This study aims to formulate a feature ranking metric based on the mutual information index to assess the relevance and redundancy of regional biomarkers and improve the AD classification accuracy. Methods: From the Alzheimer’s Disease Neuroimaging Initiative (ADNI), 722 participants with three modalities, including florbetapir-PET, flortaucipir-PET, and MRI, were studied. The multivariate mutual information metric was utilized to capture the redundancy and complementarity of the predictors and develop a feature ranking approach. This was followed by evaluating the capability of single-modal and multimodal biomarkers in predicting the cognitive stage. Results: Although amyloid-β deposition is an earlier event in the disease trajectory, tau PET with feature selection yielded a higher early-stage classification F1-score (65.4%) compared to amyloid-β PET (63.3%) and MRI (63.2%). The SVC multimodal scenario with feature selection improved the F1-score to 70.0% and 71.8% for the early and late-stage, respectively. When age and risk factors were included, the scores improved by 2 to 4%. The Amyloid-Tau-Neurodegeneration [AT(N)] framework helped to interpret the classification results for different biomarker categories. Conclusion: The results underscore the utility of a novel feature selection approach to reduce the dimensionality of multimodal datasets and enhance model performance. The AT(N) biomarker framework can help to explore the misclassified cases by revealing the relationship between neuropathological biomarkers and cognition.

A NEW ENSEMBLE METHOD FOR FEATURE RANKING IN TEXT MINING

2013 ◽  
Vol 22 (03) ◽  
pp. 1350010 ◽  
Author(s):  
SABEREH SADEGHI ◽  
HAMID BEIGY
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Ensemble Methods ◽  
Ease Of Use ◽  
Convergence Time ◽  
Feature Ranking ◽  
Initial Population ◽  
Feature Subset ◽  
Ranking Methods

Dimensionality reduction is a necessary task in data mining when working with high dimensional data. A type of dimensionality reduction is feature selection. Feature selection based on feature ranking has received much attention by researchers. The major reasons are its scalability, ease of use, and fast computation. Feature ranking methods can be divided into different categories and may use different measures for ranking features. Recently, ensemble methods have entered in the field of ranking and achieved more accuracy among others. Accordingly, in this paper a Heterogeneous ensemble based algorithm for feature ranking is proposed. The base ranking methods in this ensemble structure are chosen from different categories like information theoretic, distance based, and statistical methods. The results of the base ranking methods are then fused into a final feature subset by means of genetic algorithm. The diversity of the base methods improves the quality of initial population of the genetic algorithm and thus reducing the convergence time of the genetic algorithm. In most of ranking methods, it's the user's task to determine the threshold for choosing the appropriate subset of features. It is a problem, which may cause the user to try many different values to select a good one. In the proposed algorithm, the difficulty of determining a proper threshold by the user is decreased. The performance of the algorithm is evaluated on four different text datasets and the experimental results show that the proposed method outperforms all other five feature ranking methods used for comparison. One advantage of the proposed method is that it is independent to the classification method used for classification.

scholarly journals Differential evolution for filter feature selection based on information theory and feature ranking

2021 ◽  
Author(s):  
E Hancer ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Differential Evolution ◽  
Nearest Neighbor ◽  
Feature Subset ◽  
Fisher Score ◽  
Multi Objective ◽  
Advantages And Disadvantages ◽  
Class Labels ◽  
Single Objective

© 2017 Elsevier B.V. Feature selection is an essential step in various tasks, where filter feature selection algorithms are increasingly attractive due to their simplicity and fast speed. A common filter is to use mutual information to estimate the relationships between each feature and the class labels (mutual relevancy), and between each pair of features (mutual redundancy). This strategy has gained popularity resulting a variety of criteria based on mutual information. Other well-known strategies are to order each feature based on the nearest neighbor distance as in ReliefF, and based on the between-class variance and the within-class variance as in Fisher Score. However, each strategy comes with its own advantages and disadvantages. This paper proposes a new filter criterion inspired by the concepts of mutual information, ReliefF and Fisher Score. Instead of using mutual redundancy, the proposed criterion tries to choose the highest ranked features determined by ReliefF and Fisher Score while providing the mutual relevance between features and the class labels. Based on the proposed criterion, two new differential evolution (DE) based filter approaches are developed. While the former uses the proposed criterion as a single objective problem in a weighted manner, the latter considers the proposed criterion in a multi-objective design. Moreover, a well known mutual information feature selection approach (MIFS) based on maximum-relevance and minimum-redundancy is also adopted in single-objective and multi-objective DE algorithms for feature selection. The results show that the proposed criterion outperforms MIFS in both single objective and multi-objective DE frameworks. The results also indicate that considering feature selection as a multi-objective problem can generally provide better performance in terms of the feature subset size and the classification accuracy. © This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/

scholarly journals A novel sensitivity-based method for feature selection

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Dayakar L. Naik ◽  
Ravi kiran
Keyword(s):  
Sensitivity Analysis ◽  
Feature Selection ◽  
Feature Selection Method ◽  
Feature Ranking ◽  
Ranking Methods ◽  
Novel Approach ◽  
Machine Learning Model ◽  
Real World Datasets ◽  
Perturbation Parameters ◽  
Feature Selection Approach

AbstractSensitivity analysis is a popular feature selection approach employed to identify the important features in a dataset. In sensitivity analysis, each input feature is perturbed one-at-a-time and the response of the machine learning model is examined to determine the feature's rank. Note that the existing perturbation techniques may lead to inaccurate feature ranking due to their sensitivity to perturbation parameters. This study proposes a novel approach that involves the perturbation of input features using a complex-step. The implementation of complex-step perturbation in the framework of deep neural networks as a feature selection method is provided in this paper, and its efficacy in determining important features for real-world datasets is demonstrated. Furthermore, the filter-based feature selection methods are employed, and the results obtained from the proposed method are compared. While the results obtained for the classification task indicated that the proposed method outperformed other feature ranking methods, in the case of the regression task, it was found to perform more or less similar to that of other feature ranking methods.

scholarly journals Differential evolution for filter feature selection based on information theory and feature ranking

2021 ◽  
Author(s):  
E Hancer ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Differential Evolution ◽  
Nearest Neighbor ◽  
Feature Subset ◽  
Fisher Score ◽  
Multi Objective ◽  
Advantages And Disadvantages ◽  
Class Labels ◽  
Single Objective

© 2017 Elsevier B.V. Feature selection is an essential step in various tasks, where filter feature selection algorithms are increasingly attractive due to their simplicity and fast speed. A common filter is to use mutual information to estimate the relationships between each feature and the class labels (mutual relevancy), and between each pair of features (mutual redundancy). This strategy has gained popularity resulting a variety of criteria based on mutual information. Other well-known strategies are to order each feature based on the nearest neighbor distance as in ReliefF, and based on the between-class variance and the within-class variance as in Fisher Score. However, each strategy comes with its own advantages and disadvantages. This paper proposes a new filter criterion inspired by the concepts of mutual information, ReliefF and Fisher Score. Instead of using mutual redundancy, the proposed criterion tries to choose the highest ranked features determined by ReliefF and Fisher Score while providing the mutual relevance between features and the class labels. Based on the proposed criterion, two new differential evolution (DE) based filter approaches are developed. While the former uses the proposed criterion as a single objective problem in a weighted manner, the latter considers the proposed criterion in a multi-objective design. Moreover, a well known mutual information feature selection approach (MIFS) based on maximum-relevance and minimum-redundancy is also adopted in single-objective and multi-objective DE algorithms for feature selection. The results show that the proposed criterion outperforms MIFS in both single objective and multi-objective DE frameworks. The results also indicate that considering feature selection as a multi-objective problem can generally provide better performance in terms of the feature subset size and the classification accuracy. © This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/

scholarly journals Modelling, Measuring, and Visualising Community Resilience: A Systematic Review

2020 ◽  
Vol 12 (19) ◽  
pp. 7896 ◽  
Author(s):  
Hoang Long Nguyen ◽  
Rajendra Akerkar
Keyword(s):  
Systematic Review ◽  
Community Resilience ◽  
Future Research ◽  
Multiple Perspectives ◽  
Research Directions ◽  
Starting Point ◽  
Future Research Directions ◽  
Research Questions ◽  
The Impact ◽  
Complete Process

The concept of community resilience receives much attention in studies and applications due to its ability to provide preparedness against hazards, to protect our life against risks, and to recover to stable living conditions. Nevertheless, community resilience is complex, contextual, multifaceted, and therefore hard to define, recognise, and operationalise. An essential advantage of having a complete process for community resilience is the capacity to be aware of and respond appropriately in times of adversity. A three-step process constituting of modelling, measurement, and visualisation is crucial to determine components, to assess value, and to represent information of community resilience, respectively. The goal of this review is to offer a general overview of multiple perspectives for modelling, measuring, and visualising community resilience derived from related and emerging studies, projects, and tools. By engaging throughout the entire process, which involves three sequential steps as we mentioned above, communities can discover important components of resilience, optimise available local and natural resources, and mitigate the impact of impairments effectively and efficiently. To this end, we conduct a systematic review of 77 different literature records published from 2000 to 2020, concentrating on five research questions. We believe that researchers, practitioners, and policymakers can utilise this paper as a potential reference and a starting point to surpass current hindrances as well as to sharpen their future research directions.

scholarly journals A Filter Feature Selection Algorithm Based on Mutual Information for Intrusion Detection

2018 ◽  
Vol 8 (9) ◽  
pp. 1535 ◽  
Author(s):  
Fei Zhao ◽  
Jiyong Zhao ◽  
Xinxin Niu ◽  
Shoushan Luo ◽  
Yang Xin
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Intrusion Detection ◽  
Detection Efficiency ◽  
Feature Selection Method ◽  
Selection Algorithm ◽  
Accuracy Rate ◽  
Network Attacks ◽  
The Impact ◽  
The Relationship

For a large number of network attacks, feature selection is used to improve intrusion detection efficiency. A new mutual information algorithm of the redundant penalty between features (RPFMI) algorithm with the ability to select optimal features is proposed in this paper. Three factors are considered in this new algorithm: the redundancy between features, the impact between selected features and classes and the relationship between candidate features and classes. An experiment is conducted using the proposed algorithm for intrusion detection on the KDD Cup 99 intrusion dataset and the Kyoto 2006+ dataset. Compared with other algorithms, the proposed algorithm has a much higher accuracy rate (i.e., 99.772%) on the DOS data and can achieve better performance on remote-to-login (R2L) data and user-to-root (U2R) data. For the Kyoto 2006+ dataset, the proposed algorithm possesses the highest accuracy rate (i.e., 97.749%) among the other algorithms. The experiment results demonstrate that the proposed algorithm is a highly effective feature selection method in the intrusion detection.

scholarly journals Feature Ranking and Differential Evolution for Feature Selection in Brushless DC Motor Fault Diagnosis

Symmetry ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1291
Author(s):  
Chun-Yao Lee ◽  
Chen-Hsu Hung
Keyword(s):  
Feature Selection ◽  
Fault Diagnosis ◽  
Differential Evolution ◽  
Dc Motor ◽  
Brushless Dc Motor ◽  
Identification System ◽  
Feature Ranking ◽  
Discrete Wavelet ◽  
Accuracy Rate ◽  
Brushless Dc

A fault diagnosis system with the ability to recognize many different faults obviously has a certain complexity. Therefore, improving the performance of similar systems has attracted much research interest. This article proposes a system of feature ranking and differential evolution for feature selection in BLDC fault diagnosis. First, this study used the Hilbert–Huang transform (HHT) to extract the features of four different types of brushless DC motor Hall signal. Second, we used feature selection based on a distance discriminant (FSDD) to calculate the feature factors which base on the category separability of features to select the features which have a positive correlation with the types. The features were entered sequentially into the two supervised classifiers: backpropagation neural network (BPNN) and linear discriminant analysis (LDA), and the identification results were then evaluated. The feature input for the classifier was derived from the FSDD, and then we optimized the feature rank using differential evolution (DE). Finally, the results were verified from the BLDC motor’s operating environment simulation with the same features by adding appropriate signal-to-noise ratio magnitudes. The identification system obtained an accuracy rate of 96% when there were 14 features. Additionally, the experimental results show that the proposed system has a robust anti-noise ability, and the accuracy rate is 92.04%, even when 20 dB of white Gaussian noise is added to the signal. Moreover, compared with the systems established from the discrete wavelet transform (DWT) and a variety of classifiers, our proposed system has a higher accuracy with fewer features.

Sign in / Sign up

Export Citation Format

Share Document