scholarly journals A Constrained Feature Selection Approach Based on Feature Clustering and Hypothesis Margin Maximization

2021 ◽  
Vol 2021 ◽  
pp. 1-18
Author(s):  
Samah Hijazi ◽  
Vinh Truong Hoang
Keyword(s):  
Feature Selection ◽  
Classification Accuracy ◽  
Feature Subset ◽  
Feature Clustering ◽  
Discriminative Feature ◽  
Selection Approach ◽  
Benchmark Datasets ◽  
Margin Maximization ◽  
Feature Selection Approach ◽  
The Right

In this paper, we propose a semisupervised feature selection approach that is based on feature clustering and hypothesis margin maximization. The aim is to improve the classification accuracy by choosing the right feature subset and to allow building more interpretable models. Our approach handles the two core aspects of feature selection, i.e., relevance and redundancy, and is divided into three steps. First, the similarity weights between features are represented by a sparse graph where each feature can be reconstructed from the sparse linear combination of the others. Second, features are then hierarchically clustered identifying groups of the most similar ones. Finally, a semisupervised margin-based objective function is optimized to select the most data discriminative feature from within each cluster, hence maximizing relevance while minimizing redundancy among features. Eventually, we empirically validate our proposed approach on multiple well-known UCI benchmark datasets in terms of classification accuracy and representation entropy, where it proved to outperform four other semisupervised and unsupervised methods and competed with two widely used supervised ones.

Performance Analysis of Classifiers on Filter-Based Feature Selection Approaches on Microarray Data

2017 ◽  
pp. 41-70 ◽  
Author(s):  
Arunkumar Chinnaswamy ◽  
Ramakrishnan Srinivasan
Keyword(s):  
Feature Selection ◽  
Microarray Data ◽  
Classification Accuracy ◽  
Information Gain ◽  
Feature Subset ◽  
Classification Problems ◽  
Raw Data ◽  
Correlation Based Feature Selection ◽  
Feature Selection Approach ◽  
Gene Expression Levels

The process of Feature selection in machine learning involves the reduction in the number of features (genes) and similar activities that results in an acceptable level of classification accuracy. This paper discusses the filter based feature selection methods such as Information Gain and Correlation coefficient. After the process of feature selection is performed, the selected genes are subjected to five classification problems such as Naïve Bayes, Bagging, Random Forest, J48 and Decision Stump. The same experiment is performed on the raw data as well. Experimental results show that the filter based approaches reduce the number of gene expression levels effectively and thereby has a reduced feature subset that produces higher classification accuracy compared to the same experiment performed on the raw data. Also Correlation Based Feature Selection uses very fewer genes and produces higher accuracy compared to Information Gain based Feature Selection approach.

scholarly journals SU-CCE: A Novel Feature Selection Approach for Reducing High Dimensionality

2021 ◽  
Author(s):  
A B Pawar ◽  
M A Jawale ◽  
Ravi Kumar Tirandasu ◽  
Saiprasad Potharaju
Keyword(s):  
Feature Selection ◽  
Classification Accuracy ◽  
Feature Space ◽  
Microarray Dataset ◽  
Classification Model ◽  
High Dimensionality ◽  
High Dimensional ◽  
Selection Approach ◽  
Feature Selection Approach ◽  
Careful Investigation

High dimensionality is the serious issue in the preprocessing of data mining. Having large number of features in the dataset leads to several complications for classifying an unknown instance. In a initial dataspace there may be redundant and irrelevant features present, which leads to high memory consumption, and confuse the learning model created with those properties of features. Always it is advisable to select the best features and generate the classification model for better accuracy. In this research, we proposed a novel feature selection approach and Symmetrical uncertainty and Correlation Coefficient (SU-CCE) for reducing the high dimensional feature space and increasing the classification accuracy. The experiment is performed on colon cancer microarray dataset which has 2000 features. The proposed method derived 38 best features from it. To measure the strength of proposed method, top 38 features extracted by 4 traditional filter-based methods are compared with various classifiers. After careful investigation of result, the proposed approach is competing with most of the traditional methods.

A Feature Selection Approach for Network Intrusion Classification

2013 ◽  
Vol 3 (4) ◽  
pp. 51-59 ◽  
Author(s):  
Heba F. Eid ◽  
Mostafa A. Salama ◽  
Aboul Ella Hassanien
Keyword(s):  
Feature Selection ◽  
Classification Accuracy ◽  
Learning Algorithm ◽  
Local Maximum ◽  
Selection Methods ◽  
Hybrid Filter ◽  
Data Set ◽  
Network Intrusion ◽  
Selection Approach ◽  
Feature Selection Approach

Feature selection is a preprocessing step to machine learning, leads to increase the classification accuracy and reduce its complexity. Feature selection methods are classified into two main categories: filter and wrapper. Filter methods evaluate features without involving any learning algorithm, while wrapper methods depend on a learning algorithm for feature evaluation. Variety hybrid Filter and wrapper methods have been proposed in the literature. However, hybrid filter and wrapper approaches suffer from the problem of determining the cut-off point of the ranked features. This leads to decrease the classification accuracy by eliminating important features. In this paper the authors proposed a Hybrid Bi-Layer behavioral-based feature selection approach, which combines filter and wrapper feature selection methods. The proposed approach solves the cut-off point problem for the ranked features. It consists of two layers, at the first layer Information gain is used to rank the features and select a new set of features depending on a global maxima classification accuracy. Then, at the second layer a new subset of features is selected from within the first layer redacted data set by searching for a group of local maximum classification accuracy. To evaluate the proposed approach it is applied on NSL-KDD dataset, where the number of features is reduced from 41 to 34 features at the first layer. Then reduced from 34 to 20 features at the second layer, which leads to improve the classification accuracy to 99.2%.

scholarly journals Hybrid Feature Selection Approach to Improve the Deep Neural Network on New Flow-Based Dataset for NIDS

2021 ◽  
Vol 1 (1) ◽  
pp. 66-83
Author(s):  
Rawaa Ismael Farhan ◽  
Abeer Tariq Maolood ◽  
NidaaFlaih Hassan
Keyword(s):  
Feature Selection ◽  
Detection System ◽  
Machine Learning Algorithms ◽  
Binary Particle Swarm Optimization ◽  
Feature Subset ◽  
Network Intrusion ◽  
Selection Approach ◽  
Long Time ◽  
Feature Selection Approach ◽  
Computational Resources

Network Intrusion Detection System (NIDS) detects normal and malicious behavior by analyzing network traffic, this analysis has the potential to detect novel attacks especially in IoT environments. Deep Learning (DL)has proven its outperformance compared to machine learning algorithms in solving the complex problems of the real-world like NIDS. Although, this approach needs more computational resources and consumes a long time. Feature selection plays a significant role in choosing the best features only that describe the target concept optimally during a classification process. However, when handling a large number of features the selecting such relevant features becomes a difficult task. Therefore, this paper proposes Enhanced BPSO using Binary Particle Swarm Optimization (BPSO) and correlation–based (CFS) classical statistical feature selection approach to solve the problem on BPSO feature selection. The selected feature subset has evaluated on Deep Neural Networks (DNN) classifiers and the new flow-based CSE-CIC-IDS2018 dataset. Experimental results have shown a high accuracy of 95% based on processing time, detection rate, and false alarm rate compared with other benchmark classifiers.

scholarly journals A Probabilistic Derivation of LASSO and L12-Norm Feature Selections

2019 ◽  
Vol 33 ◽  
pp. 4586-4593 ◽  
Author(s):  
Di Ming ◽  
Chris Ding ◽  
Feiping Nie
Keyword(s):  
Feature Selection ◽  
Sparse Coding ◽  
Point Of View ◽  
Probabilistic Framework ◽  
Independent Point ◽  
Joint Sparsity ◽  
Selection Approach ◽  
Benchmark Datasets ◽  
Feature Selection Approach

LASSO and ℓ2,1-norm based feature selection had achieved success in many application areas. In this paper, we first derive LASSO and ℓ1,2-norm feature selection from a probabilistic framework, which provides an independent point of view from the usual sparse coding point of view. From here, we further propose a feature selection approach based on the probability-derived ℓ1,2-norm. We point out some inflexibility in the standard feature selection that the feature selected for all different classes are enforced to be exactly the same using the widely used ℓ2,1-norm, which enforces the joint sparsity across all the data instances. Using the probabilityderived ℓ1,2-norm feature selection, allowing certain flexibility that the selected features do not have to be exactly same for all classes, the resulting features lead to better classification on six benchmark datasets.

scholarly journals Feature Selection Approach based on Firefly Algorithm and Chi-square

2018 ◽  
Vol 8 (4) ◽  
pp. 2338 ◽  
Author(s):  
Emad Mohamed Mashhour ◽  
Enas M. F. El Houby ◽  
Khaled Tawfik Wassif ◽  
Akram I. Salah
Keyword(s):  
Feature Selection ◽  
Discriminant Analysis ◽  
Classification Accuracy ◽  
Firefly Algorithm ◽  
Nearest Neighbor ◽  
K Nearest Neighbor ◽  
Chi Square ◽  
Fitness Functions ◽  
Selection Approach ◽  
Feature Selection Approach

Dimensionality problem is a well-known challenging issue for most classifiers in which datasets have unbalanced number of samples and features. Features may contain unreliable data which may lead the classification process to produce undesirable results. Feature selection approach is considered a solution for this kind of problems. In this paperan enhanced firefly algorithm is proposed to serve as a feature selection solution for reducing dimensionality and picking the most informative features to be used in classification. The main purpose of the proposedmodel is to improve the classification accuracy through using the selected features produced from the model, thus classification errors will decrease. Modeling firefly in this research appears through simulating firefly position by cell chi-square value which is changed after every move, and simulating firefly intensity by calculating a set of different fitness functionsas a weight for each feature. K-nearest neighbor and Discriminant analysis are used as classifiers to test the proposed firefly algorithm in selecting features. Experimental results showed that the proposed enhanced algorithmbased on firefly algorithm with chi-square and different fitness functions can provide better results than others. Results showed that reduction of dataset is useful for gaining higher accuracy in classification.

scholarly journals Particle Swarm Optimisation for Feature Selection in Classification

2021 ◽  
Author(s):  
◽  
Bing Xue
Keyword(s):  
Feature Selection ◽  
Classification Accuracy ◽  
Classification Performance ◽  
Classification Algorithm ◽  
Computational Time ◽  
Classification Algorithms ◽  
Multi Objective ◽  
Selection Approach ◽  
Feature Selection Approach ◽  
Single Objective

<p>Classification problems often have a large number of features, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the classification accuracy. Feature selection is a process of selecting a subset of relevant features, which can decrease the dimensionality, shorten the running time, and/or improve the classification accuracy. There are two types of feature selection approaches, i.e. wrapper and filter approaches. Their main difference is that wrappers use a classification algorithm to evaluate the goodness of the features during the feature selection process while filters are independent of any classification algorithm. Feature selection is a difficult task because of feature interactions and the large search space. Existing feature selection methods suffer from different problems, such as stagnation in local optima and high computational cost. Evolutionary computation (EC) techniques are well-known global search algorithms. Particle swarm optimisation (PSO) is an EC technique that is computationally less expensive and can converge faster than other methods. PSO has been successfully applied to many areas, but its potential for feature selection has not been fully investigated.  The overall goal of this thesis is to investigate and improve the capability of PSO for feature selection to select a smaller number of features and achieve similar or better classification performance than using all features.  This thesis investigates the use of PSO for both wrapper and filter, and for both single objective and multi-objective feature selection, and also investigates the differences between wrappers and filters.  This thesis proposes a new PSO based wrapper, single objective feature selection approach by developing new initialisation and updating mechanisms. The results show that by considering the number of features in the initialisation and updating procedures, the new algorithm can improve the classification performance, reduce the number of features and decrease computational time.  This thesis develops the first PSO based wrapper multi-objective feature selection approach, which aims to maximise the classification accuracy and simultaneously minimise the number of features. The results show that the proposed multi-objective algorithm can obtain more and better feature subsets than single objective algorithms, and outperform other well-known EC based multi-objective feature selection algorithms.  This thesis develops a filter, single objective feature selection approach based on PSO and information theory. Two measures are proposed to evaluate the relevance of the selected features based on each pair of features and a group of features, respectively. The results show that PSO and information based algorithms can successfully address feature selection tasks. The group based method achieves higher classification accuracies, but the pair based method is faster and selects smaller feature subsets.  This thesis proposes the first PSO based multi-objective filter feature selection approach using information based measures. This work is also the first work using other two well-known multi-objective EC algorithms in filter feature selection, which are also used to compare the performance of the PSO based approach. The results show that the PSO based multiobjective filter approach can successfully address feature selection problems, outperform single objective filter algorithms and achieve better classification performance than other multi-objective algorithms.   This thesis investigates the difference between wrapper and filter approaches in terms of the classification performance and computational time, and also examines the generality of wrappers. The results show that wrappers generally achieve better or similar classification performance than filters, but do not always need longer computational time than filters. The results also show that wrappers built with simple classification algorithms can be general to other classification algorithms.</p>

Sign in / Sign up

Export Citation Format

Share Document