A Feature Selection Algorithm Based on Approximate Markov Blanket and Dynamic Mutual Information

2012 ◽  
pp. 226-233
Author(s):  
Xiaodan Wang ◽  
Xu Yao ◽  
Yuxi Zhang ◽  
Lei Lei
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Markov Blanket

A NOVEL FEATURE SELECTION ALGORITHM WITH SUPERVISED MUTUAL INFORMATION FOR CLASSIFICATION

2013 ◽  
Vol 22 (04) ◽  
pp. 1350027
Author(s):  
JAGANATHAN PALANICHAMY ◽  
KUPPUCHAMY RAMASAMY
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Feature Selection ◽  
Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class A ◽  
Selection Algorithms ◽  
The Relationship ◽  
Class Variable

Feature selection is essential in data mining and pattern recognition, especially for database classification. During past years, several feature selection algorithms have been proposed to measure the relevance of various features to each class. A suitable feature selection algorithm normally maximizes the relevancy and minimizes the redundancy of the selected features. The mutual information measure can successfully estimate the dependency of features on the entire sampling space, but it cannot exactly represent the redundancies among features. In this paper, a novel feature selection algorithm is proposed based on maximum relevance and minimum redundancy criterion. The mutual information is used to measure the relevancy of each feature with class variable and calculate the redundancy by utilizing the relationship between candidate features, selected features and class variables. The effectiveness is tested with ten benchmarked datasets available in UCI Machine Learning Repository. The experimental results show better performance when compared with some existing algorithms.

An Improved Feature Selection Algorithm Based on Parzen Window and Conditional Mutual Information

2013 ◽  
Vol 347-350 ◽  
pp. 2614-2619
Author(s):  
Deng Chao He ◽  
Wen Ning Hao ◽  
Gang Chen ◽  
Da Wei Jin
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Probability Density Functions ◽  
Density Functions ◽  
Continuous Variables ◽  
Conditional Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Parzen Window ◽  
Selection For

In this paper, an improved feature selection algorithm by conditional mutual information with Parzen window was proposed, which adopted conditional mutual information as an evaluation criterion of feature selection in order to overcome the deficiency of feature redundant and used Parzen window to estimate the probability density functions and calculate the conditional mutual information of continuous variables, in such a way as to achieve feature selection for continuous data.

scholarly journals Improved Relief Weight Feature Selection Algorithm Based on Relief and Mutual Information

Information ◽  
2021 ◽  
Vol 12 (6) ◽  
pp. 228
Author(s):  
Hongbin Wang ◽  
Pengming Wang ◽  
Shengchun Deng ◽  
Haoran Li
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
High Efficiency ◽  
Feature Subset ◽  
Evaluation Function ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Relief Algorithm ◽  
Sequential Forward Search ◽  
Computation Speed

As the classic feature selection algorithm, the Relief algorithm has the advantages of simple computation and high efficiency, but the algorithm itself is limited to only dealing with binary classification problems, and the comprehensive distinguishing ability of the feature subsets composed of the former K features selected by the Relief algorithm is often redundant, as the algorithm cannot select the ideal feature subset. When calculating the correlation and redundancy between characteristics by mutual information, the computation speed is slow because of the high computational complexity and the method’s need to calculate the probability density function of the corresponding features. Aiming to solve the above problems, we first improve the weight of the Relief algorithm, so that it can be used to evaluate a set of candidate feature sets. Then we use the improved joint mutual information evaluation function to replace the basic mutual information computation and solve the problem of computation speed and correlation, and redundancy between features. Finally, a compound correlation feature selection algorithm based on Relief and joint mutual information is proposed using the evaluation function and the heuristic sequential forward search strategy. This algorithm can effectively select feature subsets with small redundancy and strong classification characteristics, and has the excellent characteristics of faster calculation speed.

scholarly journals A Feature Selection Algorithm Integrating Maximum Classification Information and Minimum Interaction Feature Dependency Information

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Li Zhang
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Information Gain ◽  
Small Sample ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class Labels ◽  
Minimum Interaction ◽  
Classification Information ◽  
Selection Algorithms

Feature selection is the key step in the analysis of high-dimensional small sample data. The core of feature selection is to analyse and quantify the correlation between features and class labels and the redundancy between features. However, most of the existing feature selection algorithms only consider the classification contribution of individual features and ignore the influence of interfeature redundancy and correlation. Therefore, this paper proposes a feature selection algorithm for nonlinear dynamic conditional relevance (NDCRFS) through the study and analysis of the existing feature selection algorithm ideas and method. Firstly, redundancy and relevance between features and between features and class labels are discriminated by mutual information, conditional mutual information, and interactive mutual information. Secondly, the selected features and candidate features are dynamically weighted utilizing information gain factors. Finally, to evaluate the performance of this feature selection algorithm, NDCRFS was validated against 6 other feature selection algorithms on three classifiers, using 12 different data sets, for variability and classification metrics between the different algorithms. The experimental results show that the NDCRFS method can improve the quality of the feature subsets and obtain better classification results.

Sign in / Sign up

Export Citation Format

Share Document