Feature Selection in Data Mining

Data Mining ◽  
2011 ◽  
pp. 80-105 ◽  
Author(s):  
Yong Seong Kim ◽  
W. Nick Street ◽  
Filippo Menczer
Keyword(s):  
Feature Selection ◽  
Unsupervised Learning ◽  
General Framework ◽  
Search Algorithm ◽  
Feature Subset Selection ◽  
Feature Subset ◽  
Em Algorithms ◽  
Global Comparison ◽  
Supervised And Unsupervised Learning ◽  
Local Selection

Feature subset selection is an important problem in knowledge discovery, not only for the insight gained from determining relevant modeling variables, but also for the improved understandability, scalability, and, possibly, accuracy of the resulting models. The purpose of this chapter is to provide a comprehensive analysis of feature selection via evolutionary search in supervised and unsupervised learning. To achieve this purpose, we first discuss a general framework for feature selection based on a new search algorithm, Evolutionary Local Selection Algorithm (ELSA). The search is formulated as a multi-objective optimization problem to examine the trade-off between the complexity of the generated solutions against their quality. ELSA considers multiple objectives efficiently while avoiding computationally expensive global comparison. We combine ELSA with Artificial Neural Networks (ANNs) and Expectation-Maximization (EM) algorithms for feature selection in supervised and unsupervised learning respectively. Further, we provide a new two-level evolutionary algorithm, Meta-Evolutionary Ensembles (MEE), where feature selection is used to promote the diversity among classifiers in the same ensemble.

Intrusion Detection System for Malicious Traffic Using Evolutionary Search Algorithm

2020 ◽  
Vol 13 ◽  
Author(s):  
Samar Al-Saqqa ◽  
Mustafa Al-Fayoumi ◽  
Malik Qasaimeh
Keyword(s):  
Feature Selection ◽  
Intrusion Detection ◽  
Intrusion Detection System ◽  
Search Algorithm ◽  
Detection System ◽  
Feature Subset Selection ◽  
Intrusion Detection Systems ◽  
Feature Subset ◽  
Evolutionary Search ◽  
Detection Systems

Introduction: Intrusion detection systems play a key role in system security by identifying potential attacks and giving appropriate responses. As new attacks are always emerging, intrusion detection systems must adapt to these attacks, and more work is continuously needed to develop and propose new methods and techniques that can improve efficient and effective adaptive intrusion systems. Feature selection is one of the challenging areas that need more work because of its importance and impact on the performance of intrusion detection systems. This paper applies evolutionary search algorithm in feature subset selection for intrusion detection systems. Methods: The evolutionary search algorithm for the feature subset selection is applied and two classifiers are used, Naïve Bayes and decision tree J48, to evaluate system performance before and after features selection. NSL-KDD dataset and its subsets are used in all evaluation experiments. Results: The results show that feature selection using the evolutionary search algorithm enhances the intrusion detection system with respect to detection accuracy and detection of unknown attacks. Furthermore, time performance is achieved by reducing training time, which is reflected positively in overall system performance. Discussion: The evolutionary search applied to select IDS algorithm features can be developed by modifying and enhancing mutation and crossover operators and applying new enhanced techniques in the selection process, which can give better results and enhance the performance of intrusion detection for rare and complicated attacks. Conclusion: The evolutionary search algorithm is applied to find the best subset of features for the intrusion detection system. In conclusion, it is a promising approach to be used as a feature selection method for intrusion detection. The results showed better performance for the intrusion detection system in terms of accuracy and detection rate.

scholarly journals A novel feature selection algorithm based on damping oscillation theory

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0255307
Author(s):  
Fujun Wang ◽  
Xing Wang
Keyword(s):  
Feature Selection ◽  
Optimization Algorithm ◽  
Euclidean Distance ◽  
Oscillation Theory ◽  
Feature Subset Selection ◽  
Support Vector ◽  
Data Sets ◽  
Feature Subset ◽  
Selection Algorithm ◽  
Filter Model

Feature selection is an important task in big data analysis and information retrieval processing. It reduces the number of features by removing noise, extraneous data. In this paper, one feature subset selection algorithm based on damping oscillation theory and support vector machine classifier is proposed. This algorithm is called the Maximum Kendall coefficient Maximum Euclidean Distance Improved Gray Wolf Optimization algorithm (MKMDIGWO). In MKMDIGWO, first, a filter model based on Kendall coefficient and Euclidean distance is proposed, which is used to measure the correlation and redundancy of the candidate feature subset. Second, the wrapper model is an improved grey wolf optimization algorithm, in which its position update formula has been improved in order to achieve optimal results. Third, the filter model and the wrapper model are dynamically adjusted by the damping oscillation theory to achieve the effect of finding an optimal feature subset. Therefore, MKMDIGWO achieves both the efficiency of the filter model and the high precision of the wrapper model. Experimental results on five UCI public data sets and two microarray data sets have demonstrated the higher classification accuracy of the MKMDIGWO algorithm than that of other four state-of-the-art algorithms. The maximum ACC value of the MKMDIGWO algorithm is at least 0.5% higher than other algorithms on 10 data sets.

scholarly journals Feature Selection Optimization for Highlighting Opinions Using Supervised and Unsupervised Learning on Arabic Language

2021 ◽  
Vol 10 (2) ◽  
pp. 636-642
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Text Mining ◽  
Unsupervised Learning ◽  
Arabic Language ◽  
Machine Learning Techniques ◽  
Business Decision ◽  
Supervised And Unsupervised Learning ◽  
Proposed Model ◽  
The Impact

Text mining utilizes machine learning (ML) and natural language processing (NLP) for text implicit knowledge recognition, such knowledge serves many domains as translation, media searching, and business decision making. Opinion mining (OM) is one of the promised text mining fields, which are used for polarity discovering via text and has terminus benefits for business. ML techniques are divided into two approaches: supervised and unsupervised learning, since we herein testified an OM feature selection(FS)using four ML techniques. In this paper, we had implemented number of experiments via four machine learning techniques on the same three Arabic language corpora. This paper aims at increasing the accuracy of opinion highlighting on Arabic language, by using enhanced feature selection approaches. FS proposed model is adopted for enhancing opinion highlighting purpose. The experimental results show the outperformance of the proposed approaches in variant levels of supervisory,i.e. different techniques via distinct data domains. Multiple levels of comparison are carried out and discussed for further understanding of the impact of proposed model on several ML techniques.

scholarly journals Liver Cancer Classification Model Using Hybrid Feature Selection Based on Class-Dependent Technique for the Central Region of Thailand

Information ◽  
2019 ◽  
Vol 10 (6) ◽  
pp. 187
Author(s):  
Rattanawadee Panthong ◽  
Anongnart Srivihok
Keyword(s):  
Feature Selection ◽  
Liver Cancer ◽  
Predictive Model ◽  
Information Gain ◽  
Classification Performance ◽  
Cancer Classification ◽  
Feature Subset Selection ◽  
Classification Model ◽  
Feature Subset ◽  
Cancer Data

Liver cancer data always consist of a large number of multidimensional datasets. A dataset that has huge features and multiple classes may be irrelevant to the pattern classification in machine learning. Hence, feature selection improves the performance of the classification model to achieve maximum classification accuracy. The aims of the present study were to find the best feature subset and to evaluate the classification performance of the predictive model. This paper proposed a hybrid feature selection approach by combining information gain and sequential forward selection based on the class-dependent technique (IGSFS-CD) for the liver cancer classification model. Two different classifiers (decision tree and naïve Bayes) were used to evaluate feature subsets. The liver cancer datasets were obtained from the Cancer Hospital Thailand database. Three ensemble methods (ensemble classifiers, bagging, and AdaBoost) were applied to improve the performance of classification. The IGSFS-CD method provided good accuracy of 78.36% (sensitivity 0.7841 and specificity 0.9159) on LC_dataset-1. In addition, LC_dataset II delivered the best performance with an accuracy of 84.82% (sensitivity 0.8481 and specificity 0.9437). The IGSFS-CD method achieved better classification performance compared to the class-independent method. Furthermore, the best feature subset selection could help reduce the complexity of the predictive model.

A New Hybrid Feature Subset Selection Framework Based on Binary Genetic Algorithm and Information Theory

2019 ◽  
Vol 18 (03) ◽  
pp. 1950020 ◽  
Author(s):  
Alok Kumar Shukla ◽  
Pradeep Singh ◽  
Manu Vardhan
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Classification Accuracy ◽  
B Cell Lymphoma ◽  
Feature Subset Selection ◽  
Classification Model ◽  
Significant Feature ◽  
Support Vector ◽  
Feature Subset ◽  
Binary Genetic Algorithm

The explosion of the high-dimensional dataset in the scientific repository has been encouraging interdisciplinary research on data mining, pattern recognition and bioinformatics. The fundamental problem of the individual Feature Selection (FS) method is extracting informative features for classification model and to seek for the malignant disease at low computational cost. In addition, existing FS approaches overlook the fact that for a given cardinality, there can be several subsets with similar information. This paper introduces a novel hybrid FS algorithm, called Filter-Wrapper Feature Selection (FWFS) for a classification problem and also addresses the limitations of existing methods. In the proposed model, the front-end filter ranking method as Conditional Mutual Information Maximization (CMIM) selects the high ranked feature subset while the succeeding method as Binary Genetic Algorithm (BGA) accelerates the search in identifying the significant feature subsets. One of the merits of the proposed method is that, unlike an exhaustive method, it speeds up the FS procedure without lancing of classification accuracy on reduced dataset when a learning model is applied to the selected subsets of features. The efficacy of the proposed (FWFS) method is examined by Naive Bayes (NB) classifier which works as a fitness function. The effectiveness of the selected feature subset is evaluated using numerous classifiers on five biological datasets and five UCI datasets of a varied dimensionality and number of instances. The experimental results emphasize that the proposed method provides additional support to the significant reduction of the features and outperforms the existing methods. For microarray data-sets, we found the lowest classification accuracy is 61.24% on SRBCT dataset and highest accuracy is 99.32% on Diffuse large B-cell lymphoma (DLBCL). In UCI datasets, the lowest classification accuracy is 40.04% on the Lymphography using k-nearest neighbor (k-NN) and highest classification accuracy is 99.05% on the ionosphere using support vector machine (SVM).

Sign in / Sign up

Export Citation Format

Share Document