Feature selection via maximizing global information gain for text classification

2013 ◽  
Vol 54 ◽  
pp. 298-309 ◽  
Author(s):  
Changxing Shang ◽  
Min Li ◽  
Shengzhong Feng ◽  
Qingshan Jiang ◽  
Jianping Fan
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Information Gain ◽  
Global Information

A NEW FEATURE SELECTION METHOD FOR TEXT CLASSIFICATION

2007 ◽  
Vol 21 (02) ◽  
pp. 423-438 ◽  
Author(s):  
GULDEN UCHYIGIT ◽  
KEITH CLARK
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Information Gain ◽  
Feature Selection Method ◽  
Feature Space ◽  
Selection Method ◽  
Computational Time ◽  
Small Subset ◽  
Selection Methods ◽  
New Feature

Text classification is the problem of classifying a set of documents into a pre-defined set of classes. A major problem with text classification problems is the high dimensionality of the feature space. Only a small subset of these words are feature words which can be used in determining a document's class, while the rest adds noise and can make the results unreliable and significantly increase computational time. A common approach in dealing with this problem is feature selection where the number of words in the feature space are significantly reduced. In this paper we present the experiments of a comparative study of feature selection methods used for text classification. Ten feature selection methods were evaluated in this study including the new feature selection method, called the GU metric. The other feature selection methods evaluated in this study are: Chi-Squared (χ2) statistic, NGL coefficient, GSS coefficient, Mutual Information, Information Gain, Odds Ratio, Term Frequency, Fisher Criterion, BSS/WSS coefficient. The experimental evaluations show that the GU metric obtained the best F1 and F2 scores. The experiments were performed on the 20 Newsgroups data sets with the Naive Bayesian Probabilistic Classifier.

scholarly journals OPTIMIZATION OF K-NEAREST NEIGHBOUR TO CATEGORIZE INDONESIAN'S NEWS ARTICLES

2021 ◽  
Vol 10 (01) ◽  
pp. 43-51
Author(s):  
Afdhalul Ihsan ◽  
Ednawati Rainarli
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Information Gain ◽  
Selection Process ◽  
Performance Testing ◽  
Parameter Tuning ◽  
Pso Algorithm ◽  
Optimal Number ◽  
Nearest Neighbour ◽  
Chi Square

Text classification is the process of grouping documents based on similarity in categories. Some of the obstacles in doing text classification are many words appeared in the text, and some words come up with infrequent frequency (sparse words). The way to solve this problem is to conduct the feature selection process. There are several filter-based feature selection methods; some are Chi-Square, Information Gain, Genetic Algorithm, and Particle Swarm Optimization (PSO). Aghdam's research shows that PSO is the best among those methods. This study examined PSO to optimize the k-Nearest Neighbour (k-NN) algorithm's performance in categorizing news articles. k-NN is an algorithm that is simple and easy to implement. If we use the appropriate features, then the k-NN will be a reliable algorithm. PSO algorithm is used to select keywords (term features), and it is continued with classifying the documents using k-NN. The testing process consists of three stages. The stages are tuning the parameter of k-NN, the parameter of PSO, and measuring the testing performance. The parameter tuning process aims to determine the number of neighbours used in k-NN and optimize the PSO particles. Otherwise, the performance testing compares the performance of k-NN with and without using PSO. The optimal number of neighbours is 9, with the number of particles is 50. The testing showed that using the k-NN with PSO and a 50% reduction in terms. The results 20 per cent better accuracy than k-NN without PSO. Although the PSO's process did not always find the optimal conditions, the k-NN method can produce better accuracy. In this way, the k-NN method can work better in grouping news articles, especially in Indonesian language news articles

scholarly journals The Impact of Feature Selection Methods for Classifying Arabic Textual Data

2019 ◽  
Vol 8 (4) ◽  
pp. 1333-1338
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Information Gain ◽  
Feature Space ◽  
Support Vector ◽  
Selection Methods ◽  
K Nearest Neighbors ◽  
Chi Square ◽  
Selection Algorithms ◽  
The Impact

Text classification is a vital process due to the large volume of electronic articles. One of the drawbacks of text classification is the high dimensionality of feature space. Scholars developed several algorithms to choose relevant features from article text such as Chi-square (x2 ), Information Gain (IG), and Correlation (CFS). These algorithms have been investigated widely for English text, while studies for Arabic text are still limited. In this paper, we investigated four well-known algorithms: Support Vector Machines (SVMs), Naïve Bayes (NB), K-Nearest Neighbors (KNN), and Decision Tree against benchmark Arabic textual datasets, called Saudi Press Agency (SPA) to evaluate the impact of feature selection methods. Using the WEKA tool, we have experimented the application of the four mentioned classification algorithms with and without feature selection algorithms. The results provided clear evidence that the three feature selection methods often improves classification accuracy by eliminating irrelevant features.

Indonesian News Classification Using Naïve Bayes and Two-Phase Feature Selection Model

2017 ◽  
Vol 8 (3) ◽  
pp. 610 ◽  
Author(s):  
M. Ali Fauzi ◽  
Agus Zainal Arifin ◽  
Sonny Christiano Gosaria
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Information Gain ◽  
Information Filtering ◽  
Feature Selection Method ◽  
Feature Space ◽  
Online News ◽  
New Method ◽  
Second Phase ◽  
Two Phase

Since the rise of WWW, information available online is growing rapidly. One of the example is Indonesian online news. Therefore, automatic text classification became very important task for information filtering. One of the major issue in text classification is its high dimensionality of feature space. Most of the features are irrelevant, noisy, and redundant, which may decline the accuracy of the system. Hence, feature selection is needed. Maximal Marginal Relevance for Feature Selection (MMR-FS) has been proven to be a good feature selection for text with many redundant features, but it has high computational complexity. In this paper, we propose a two-phased feature selection method. In the first phase, to lower the complexity of MMR-FS we utilize Information Gain first to reduce features. This reduced feature will be selected using MMR-FS in the second phase. The experiment result showed that our new method can reach the best accuracy by 86%. This new method could lower the complexity of MMR-FS but still retain its accuracy.

Enhancing Effectiveness of Dimension Reduction in Text Classification

2016 ◽  
Vol 26 (03) ◽  
pp. 1750008 ◽  
Author(s):  
Seyyed Hossein Seyyedi ◽  
Behrouz Minaei-Bidgoli
Keyword(s):  
Feature Extraction ◽  
Feature Selection ◽  
Dimension Reduction ◽  
Text Classification ◽  
Field Experiments ◽  
Information Gain ◽  
Dimensional Space ◽  
Feature Space ◽  
Spam Detection ◽  
Selection Methods

Nowadays, text is one prevalent forms of data and text classification is a widely used data mining task, which has various application fields. One mass-produced instance of text is email. As a communication medium, despite having a lot of advantages, email suffers from a serious problem. The number of spam emails has steadily increased in the recent years, leading to considerable irritation. Therefore, spam detection has emerged as a separate field of text classification. A primary challenge of text classification, which is more severe in spam detection and impedes the process, is high-dimensionality of feature space. Various dimension reduction methods have been proposed that produce a lower dimensional space compared to the original. These methods are divided mainly into two groups: feature selection and feature extraction. This research deals with dimension reduction in the text classification task and especially performs experiments in the spam detection field. We employ Information Gain (IG) and Chi-square Statistic (CHI) as well-known feature selection methods. Also, we propose a new feature extraction method called Sprinkled Semantic Feature Space (SSFS). Furthermore, this paper presents a new hybrid method called IG_SSFS. In IG_SSFS, we combine the selection and extraction processes to reap the benefits from both. To evaluate the mentioned methods in the spam detection field, experiments are conducted on some well-known email datasets. According to the results, SSFS demonstrated superior effectiveness over the basic selection methods in terms of improving classifiers’ performance, and IG_SSFS further enhanced the performance despite consuming less processing time.

scholarly journals Comparison of feature selection techniques in classifying stroke documents

2019 ◽  
Vol 14 (3) ◽  
pp. 1244
Author(s):  
Nur Syaza Izzati Mohd Rafei ◽  
Rohayanti Hassan ◽  
RD Rohmat Saedudin ◽  
Anis Farihan Mat Raffei ◽  
Zalmiyah Zakaria ◽  
...  
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Information Gain ◽  
Biomedical Literature ◽  
High Dimensionality ◽  
Support Vector ◽  
Pearson’S Correlation ◽  
Pearson's Correlation ◽  
Feature Selection Techniques ◽  
Selection Phase

<span>The amount of digital biomedical literature grows that make most of the researchers facing the difficulties to manage and retrieve the required information from the Internet because this task is very challenging. The application of text classification on biomedical literature is one of the solutions in order to solve problem that have been faced by researchers but managing the high dimensionality of data being a common issue on text classification. Therefore, the aim of this research is to compare the techniques that could be used to select the relevant features for classifying biomedical text abstracts. This research focus on Pearson’s Correlation and Information Gain as feature selection techniques for reducing the high dimensionality of data. Towards this effort, we conduct and evaluate several experiments using 100 abstract of stroke documents that retrieved from PubMed database as datasets. This dataset underwent the text pre-processing that is crucial before proceed to feature selection phase. Features selection phase is involving Information Gain and Pearson Correlation technique. Support Vector Machine classifier is used in order to evaluate and compare the effectiveness of two feature selection techniques. For this dataset, Information Gain has outperformed Pearson’s Correlation by 3.3%. This research tends to extract the meaningful features from a subset of stroke documents that can be used for various application especially in diagnose the stroke disease.</span>

Survey of Feature Selection and Text Classification Methods for Genetic Mutation Classification

2019 ◽  
Vol 7 (4) ◽  
pp. 933-937
Author(s):  
Varun Saproo ◽  
Rujuta Upadhyay ◽  
Manisha Valera
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Genetic Mutation ◽  
Classification Methods

Feature Selection Algorithm for High-dimensional Biomedical Data Using Information Gain and Improved Chemical Reaction Optimization

2021 ◽  
Vol 15 (8) ◽  
pp. 912-926
Author(s):  
Ge Zhang ◽  
Pan Yu ◽  
Jianlin Wang ◽  
Chaokun Yan
Keyword(s):  
Feature Selection ◽  
Chemical Reaction ◽  
Information Gain ◽  
Feature Selection Method ◽  
Search Space ◽  
Neighborhood Search ◽  
Biomedical Data ◽  
Chemical Reaction Optimization ◽  
Search Mechanism ◽  
Reaction Optimization

Background: There have been rapid developments in various bioinformatics technologies, which have led to the accumulation of a large amount of biomedical data. However, these datasets usually involve thousands of features and include much irrelevant or redundant information, which leads to confusion during diagnosis. Feature selection is a solution that consists of finding the optimal subset, which is known to be an NP problem because of the large search space. Objective: For the issue, this paper proposes a hybrid feature selection method based on an improved chemical reaction optimization algorithm (ICRO) and an information gain (IG) approach, which called IGICRO. Methods: IG is adopted to obtain some important features. The neighborhood search mechanism is combined with ICRO to increase the diversity of the population and improve the capacity of local search. Results: Experimental results of eight public available data sets demonstrate that our proposed approach outperforms original CRO and other state-of-the-art approaches.

Analysis of Feature Selection and Ensemble Classifier Methods for Intrusion Detection

2018 ◽  
Vol 7 (1) ◽  
pp. 57-72
Author(s):  
H.P. Vinutha ◽  
Poornima Basavaraju
Keyword(s):  
Feature Selection ◽  
Intrusion Detection ◽  
Detection Rate ◽  
Information Gain ◽  
False Positive Rate ◽  
Ensemble Classifier ◽  
Ensemble Classification ◽  
Chi Square ◽  
Traffic Pattern ◽  
Data Mining Algorithms

Day by day network security is becoming more challenging task. Intrusion detection systems (IDSs) are one of the methods used to monitor the network activities. Data mining algorithms play a major role in the field of IDS. NSL-KDD'99 dataset is used to study the network traffic pattern which helps us to identify possible attacks takes place on the network. The dataset contains 41 attributes and one class attribute categorized as normal, DoS, Probe, R2L and U2R. In proposed methodology, it is necessary to reduce the false positive rate and improve the detection rate by reducing the dimensionality of the dataset, use of all 41 attributes in detection technology is not good practices. Four different feature selection methods like Chi-Square, SU, Gain Ratio and Information Gain feature are used to evaluate the attributes and unimportant features are removed to reduce the dimension of the data. Ensemble classification techniques like Boosting, Bagging, Stacking and Voting are used to observe the detection rate separately with three base algorithms called Decision stump, J48 and Random forest.

Sign in / Sign up

Export Citation Format

Share Document