A NEW ENSEMBLE METHOD FOR FEATURE RANKING IN TEXT MINING

2013 ◽  
Vol 22 (03) ◽  
pp. 1350010 ◽  
Author(s):  
SABEREH SADEGHI ◽  
HAMID BEIGY
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Ensemble Methods ◽  
Ease Of Use ◽  
Convergence Time ◽  
Feature Ranking ◽  
Initial Population ◽  
Feature Subset ◽  
Ranking Methods

Dimensionality reduction is a necessary task in data mining when working with high dimensional data. A type of dimensionality reduction is feature selection. Feature selection based on feature ranking has received much attention by researchers. The major reasons are its scalability, ease of use, and fast computation. Feature ranking methods can be divided into different categories and may use different measures for ranking features. Recently, ensemble methods have entered in the field of ranking and achieved more accuracy among others. Accordingly, in this paper a Heterogeneous ensemble based algorithm for feature ranking is proposed. The base ranking methods in this ensemble structure are chosen from different categories like information theoretic, distance based, and statistical methods. The results of the base ranking methods are then fused into a final feature subset by means of genetic algorithm. The diversity of the base methods improves the quality of initial population of the genetic algorithm and thus reducing the convergence time of the genetic algorithm. In most of ranking methods, it's the user's task to determine the threshold for choosing the appropriate subset of features. It is a problem, which may cause the user to try many different values to select a good one. In the proposed algorithm, the difficulty of determining a proper threshold by the user is decreased. The performance of the algorithm is evaluated on four different text datasets and the experimental results show that the proposed method outperforms all other five feature ranking methods used for comparison. One advantage of the proposed method is that it is independent to the classification method used for classification.

A New Hybrid Feature Subset Selection Framework Based on Binary Genetic Algorithm and Information Theory

2019 ◽  
Vol 18 (03) ◽  
pp. 1950020 ◽  
Author(s):  
Alok Kumar Shukla ◽  
Pradeep Singh ◽  
Manu Vardhan
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Classification Accuracy ◽  
B Cell Lymphoma ◽  
Feature Subset Selection ◽  
Classification Model ◽  
Significant Feature ◽  
Support Vector ◽  
Feature Subset ◽  
Binary Genetic Algorithm

The explosion of the high-dimensional dataset in the scientific repository has been encouraging interdisciplinary research on data mining, pattern recognition and bioinformatics. The fundamental problem of the individual Feature Selection (FS) method is extracting informative features for classification model and to seek for the malignant disease at low computational cost. In addition, existing FS approaches overlook the fact that for a given cardinality, there can be several subsets with similar information. This paper introduces a novel hybrid FS algorithm, called Filter-Wrapper Feature Selection (FWFS) for a classification problem and also addresses the limitations of existing methods. In the proposed model, the front-end filter ranking method as Conditional Mutual Information Maximization (CMIM) selects the high ranked feature subset while the succeeding method as Binary Genetic Algorithm (BGA) accelerates the search in identifying the significant feature subsets. One of the merits of the proposed method is that, unlike an exhaustive method, it speeds up the FS procedure without lancing of classification accuracy on reduced dataset when a learning model is applied to the selected subsets of features. The efficacy of the proposed (FWFS) method is examined by Naive Bayes (NB) classifier which works as a fitness function. The effectiveness of the selected feature subset is evaluated using numerous classifiers on five biological datasets and five UCI datasets of a varied dimensionality and number of instances. The experimental results emphasize that the proposed method provides additional support to the significant reduction of the features and outperforms the existing methods. For microarray data-sets, we found the lowest classification accuracy is 61.24% on SRBCT dataset and highest accuracy is 99.32% on Diffuse large B-cell lymphoma (DLBCL). In UCI datasets, the lowest classification accuracy is 40.04% on the Lymphography using k-nearest neighbor (k-NN) and highest classification accuracy is 99.05% on the ionosphere using support vector machine (SVM).

scholarly journals Deriving Rules for Forecasting Air Carrier Financial Stress and Insolvency: A Genetic Algorithm Approach

2010 ◽  
Author(s):  
Sergio Davalos ◽  
Richard Gritta ◽  
Bahram Adrangi
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Genetic Algorithm ◽  
Artificial Neural Network ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Local Optima ◽  
Air Carrier ◽  
Genetic Algorithm Approach ◽  
Genetic Algorithm Model

Statistical and artificial intelligence methods have successfully classified organizational solvency, but are limited in terms of generalization, knowledge on how a conclusion was reached, convergence to a local optima, or inconsistent results. Issues such as dimensionality reduction and feature selection can also affect a model's performance. This research explores the use of the genetic algorithm that has the advantages of the artificial neural network but without its limitations. The genetic algorithm model resulted in a set of easy to understand, if-then rules that were used to assess U.S. air carrier solvency with a 94% accuracy.

scholarly journals A Composite Hybrid Feature Selection Learning-Based Optimization of Genetic Algorithm For Breast Cancer Detection

2020 ◽  
Author(s):  
Ahmed Abdullah Farid ◽  
Gamal Selim ◽  
Hatem Khater
Keyword(s):  
Breast Cancer ◽  
Genetic Algorithm ◽  
Feature Selection ◽  
Early Stage ◽  
Fitness Function ◽  
Support Vector ◽  
Initial Population ◽  
Tree Classifier ◽  
Selection Approach ◽  
Feature Selection Approach

Breast cancer is a significant health issue across the world. Breast cancer is the most widely-diagnosed cancer in women; early-stage diagnosis of disease and therapies increase patient safety. This paper proposes a synthetic model set of features focused on the optimization of the genetic algorithm (CHFS-BOGA) to forecast breast cancer. This hybrid feature selection approach combines the advantages of three filter feature selection approaches with an optimize Genetic Algorithm (OGA) to select the best features to improve the performance of the classification process and scalability. We propose OGA by improving the initial population generating and genetic operators using the results of filter approaches as some prior information with using the C4.5 decision tree classifier as a fitness function instead of probability and random selection. The authors collected available updated data from Wisconsin UCI machine learning with a total of 569 rows and 32 columns. The dataset evaluated using an explorer set of weka data mining open-source software for the analysis purpose. The results show that the proposed hybrid feature selection approach significantly outperforms the single filter approaches and principal component analysis (PCA) for optimum feature selection. These characteristics are good indicators for the return prediction. The highest accuracy achieved with the proposed system before (CHFS-BOGA) using the support vector machine (SVM) classifiers was 97.3%. The highest accuracy after (CHFS-BOGA-SVM) was 98.25% on split 70.0% train, remainder test, and 100% on the full training set. Moreover, the receiver operating characteristic (ROC) curve was equal to 1.0. The results showed that the proposed (CHFS-BOGA-SVM) system was able to accurately classify the type of breast tumor, whether malignant or benign.

scholarly journals A Hybrid Feature Selection Method for Improve the Accuracy of Medical Classification Process

2021 ◽  
Vol 11 (1) ◽  
pp. 50-55
Author(s):  
Maria Mohammad Yousef ◽  
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Classification Accuracy ◽  
Fitness Function ◽  
Machine Learning Algorithms ◽  
Feature Subset Selection ◽  
High Dimensionality ◽  
Support Vector ◽  
Feature Subset

Generally, medical dataset classification has become one of the biggest problems in data mining research. Every database has a given number of features but it is observed that some of these features can be redundant and can be harmful as well as disrupt the process of classification and this problem is known as a high dimensionality problem. Dimensionality reduction in data preprocessing is critical for increasing the performance of machine learning algorithms. Besides the contribution of feature subset selection in dimensionality reduction gives a significant improvement in classification accuracy. In this paper, we proposed a new hybrid feature selection approach based on (GA assisted by KNN) to deal with issues of high dimensionality in biomedical data classification. The proposed method first applies the combination between GA and KNN for feature selection to find the optimal subset of features where the classification accuracy of the k-Nearest Neighbor (kNN) method is used as the fitness function for GA. After selecting the best-suggested subset of features, Support Vector Machine (SVM) are used as the classifiers. The proposed method experiments on five medical datasets of the UCI Machine Learning Repository. It is noted that the suggested technique performs admirably on these databases, achieving higher classification accuracy while using fewer features.

A hybrid model with novel feature selection method and enhanced voting method for credit scoring

2021 ◽  
pp. 1-15
Author(s):  
Jianrong Yao ◽  
Zhongyi Wang ◽  
Lu Wang ◽  
Zhebin Zhang ◽  
Hui Jiang ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Hybrid Model ◽  
Credit Scoring ◽  
Feature Selection Method ◽  
Selection Method ◽  
Feature Subset ◽  
Proposed Model ◽  
Prediction Probability ◽  
Voting Method

With the in-depth application of artificial intelligence technology in the financial field, credit scoring models constructed by machine learning algorithms have become mainstream. However, the high-dimensional and complex attribute features of the borrower pose challenges to the predictive competence of the model. This paper proposes a hybrid model with a novel feature selection method and an enhanced voting method for credit scoring. First, a novel feature selection combined method based on a genetic algorithm (FSCM-GA) is proposed, in which different classifiers are used to select features in combination with a genetic algorithm and combine them to generate an optimal feature subset. Furthermore, an enhanced voting method (EVM) is proposed to integrate classifiers, with the aim of improving the classification results in which the prediction probability values are close to the threshold. Finally, the predictive competence of the proposed model was validated on three public datasets and five evaluation metrics (accuracy, AUC, F-score, Log loss and Brier score). The comparative experiment and significance test results confirmed the good performance and robustness of the proposed model.

Effective feature reduction for link prediction in location-based social networks

2018 ◽  
Vol 45 (5) ◽  
pp. 676-690 ◽  
Author(s):  
Ahmet Engin Bayrak ◽  
Faruk Polat
Keyword(s):  
Genetic Algorithm ◽  
Social Networks ◽  
Feature Selection ◽  
Link Prediction ◽  
Feature Selection Method ◽  
Selection Method ◽  
Prediction Performance ◽  
Feature Reduction ◽  
Feature Subset ◽  
Location Based Social Networks

In this study, we investigated feature-based approaches for improving the link prediction performance for location-based social networks (LBSNs) and analysed their performances. We developed new features based on time, common friend detail and place category information of check-in data in order to make use of information in the data which cannot be utilised by the existing features from the literature. We proposed a feature selection method to determine a feature subset that enhances the prediction performance with the removal of redundant features by clustering them. After clustering features, a genetic algorithm is used to determine the ones to select from each cluster. A non-monotonic and feasible feature selection is ensured by the proposed genetic algorithm. Results depict that both new features and the proposed feature selection method improved link prediction performance for LBSNs.

scholarly journals A Novel Feature Selection Measure Partnership-Gain

2019 ◽  
Vol 15 (04) ◽  
pp. 4 ◽  
Author(s):  
Mostafa A. Salama ◽  
Ghada Hassan
Keyword(s):  
Feature Selection ◽  
Classification Accuracy ◽  
Classification Problem ◽  
Feature Ranking ◽  
Feature Subset ◽  
The Novel ◽  
Target Class ◽  
Target Feature ◽  
Mutual Correlation ◽  
Feature Selection Techniques

Multivariate feature selection techniques search for the optimal features subset to reduce the dimensionality and hence the complexity of a classification task. Statistical feature selection techniques measure the mutual correlation between features well as the correlation of each feature to the tar- get feature. However, adding a feature to a feature subset could deteriorate the classification accuracy even though this feature positively correlates to the target class. Although most of existing feature ranking/selection techniques consider the interdependency between features, the nature of interaction be- tween features in relationship to the classification problem is still not well investigated. This study proposes a technique for forward feature selection that calculates the novel measure Partnership-Gain to select a subset of features whose partnership constructively correlates to the target feature classification. Comparative analysis to other well-known techniques shows that the proposed technique has either an enhanced or a comparable classification accuracy on the datasets studied. We present a visualization of the degree and direction of the proposed measure of features’ partnerships for a better understanding of the measure’s nature.

scholarly journals A novel sensitivity-based method for feature selection

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Dayakar L. Naik ◽  
Ravi kiran
Keyword(s):  
Sensitivity Analysis ◽  
Feature Selection ◽  
Feature Selection Method ◽  
Feature Ranking ◽  
Ranking Methods ◽  
Novel Approach ◽  
Machine Learning Model ◽  
Real World Datasets ◽  
Perturbation Parameters ◽  
Feature Selection Approach

AbstractSensitivity analysis is a popular feature selection approach employed to identify the important features in a dataset. In sensitivity analysis, each input feature is perturbed one-at-a-time and the response of the machine learning model is examined to determine the feature's rank. Note that the existing perturbation techniques may lead to inaccurate feature ranking due to their sensitivity to perturbation parameters. This study proposes a novel approach that involves the perturbation of input features using a complex-step. The implementation of complex-step perturbation in the framework of deep neural networks as a feature selection method is provided in this paper, and its efficacy in determining important features for real-world datasets is demonstrated. Furthermore, the filter-based feature selection methods are employed, and the results obtained from the proposed method are compared. While the results obtained for the classification task indicated that the proposed method outperformed other feature ranking methods, in the case of the regression task, it was found to perform more or less similar to that of other feature ranking methods.

Advanced Dimensionality Reduction Method for Big Data

Big Data ◽  
2016 ◽  
pp. 2388-2400 ◽  
Author(s):  
Sufal Das ◽  
Hemanta Kumar Kalita
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Big Data ◽  
Dimensionality Reduction ◽  
Association Rule ◽  
Computation Time ◽  
Distribution Management ◽  
Redundant Data ◽  
Data Mining Algorithms ◽  
Mining Algorithms

The growing glut of data in the worlds of science, business and government create an urgent need for consideration of big data. Big data is a term that describes large volumes of high velocity, complex and variable data that require advanced techniques and technologies to enable the capture, storage, distribution, management, and analysis of the information. Big data challenge is becoming one of the most exciting opportunities for the next years. Data mining algorithms like association rule mining perform an exhaustive search to find all rules satisfying some constraints. it is clear that it is difficult to identify the most effective rule from big data. A novel method for feature selection and extraction has been introduced for big data using genetic algorithm. Dimensionality reduction can be considered a problem of global combinatorial optimization in machine learning, which reduces the number of features, removes irrelevant, noisy and redundant data, to obtain the accuracy and saves the computation time and simplifies the result. A genetic algorithm was developed based approach utilizing a feedback linkage between feature selection and association rule using MapReduce for big data.

Sign in / Sign up

Export Citation Format

Share Document