scholarly journals Twitter User Topic Profiling Using Knowledge Base and Term Frequency – Inverse Document Frequency Feature Selection Method

2018 ◽  
Vol 1 (1) ◽  
Keyword(s):  
Feature Selection ◽  
Knowledge Base ◽  
Feature Selection Method ◽  
Selection Method ◽  
Inverse Document Frequency ◽  
Term Frequency ◽  
Twitter User ◽  
Document Frequency ◽  
Frequency Feature

Integrating Feature and Instance Selection Techniques in Opinion Mining

2020 ◽  
Vol 16 (3) ◽  
pp. 168-182
Author(s):  
Zi-Hung You ◽  
Ya-Han Hu ◽  
Chih-Fong Tsai ◽  
Yen-Ming Kuo
Keyword(s):  
Feature Selection ◽  
Opinion Mining ◽  
Classification Performance ◽  
Problem Instance ◽  
Instance Selection ◽  
Selection Methods ◽  
Inverse Document Frequency ◽  
Term Frequency ◽  
Document Frequency ◽  
Text Features

Opinion mining focuses on extracting polarity information from texts. For textual term representation, different feature selection methods, e.g. term frequency (TF) or term frequency–inverse document frequency (TF–IDF), can yield diverse numbers of text features. In text classification, however, a selected training set may contain noisy documents (or outliers), which can degrade the classification performance. To solve this problem, instance selection can be adopted to filter out unrepresentative training documents. Therefore, this article investigates the opinion mining performance associated with feature and instance selection steps simultaneously. Two combination processes based on performing feature selection and instance selection in different orders, were compared. Specifically, two feature selection methods, namely TF and TF–IDF, and two instance selection methods, namely DROP3 and IB3, were employed for comparison. The experimental results by using three Twitter datasets to develop sentiment classifiers showed that TF–IDF followed by DROP3 performs the best.

scholarly journals Improve the Accuracy of Support Vector Machine Using Chi Square Statistic and Term Frequency Inverse Document Frequency on Movie Review Sentiment Analysis

2019 ◽  
Vol 6 (1) ◽  
pp. 138-149
Author(s):  
Ukhti Ikhsani Larasati ◽  
Much Aziz Muslim ◽  
Riza Arifudin ◽  
Alamsyah Alamsyah
Keyword(s):  
Support Vector Machine ◽  
Feature Selection ◽  
Text Mining ◽  
Sentiment Analysis ◽  
Feature Weighting ◽  
Support Vector ◽  
Chi Square ◽  
Inverse Document Frequency ◽  
Term Frequency ◽  
Document Frequency

Data processing can be done with text mining techniques. To process large text data is required a machine to explore opinions, including positive or negative opinions. Sentiment analysis is a process that applies text mining methods. Sentiment analysis is a process that aims to determine the content of the dataset in the form of text is positive or negative. Support vector machine is one of the classification algorithms that can be used for sentiment analysis. However, support vector machine works less well on the large-sized data. In addition, in the text mining process there are constraints one is number of attributes used. With many attributes it will reduce the performance of the classifier so as to provide a low level of accuracy. The purpose of this research is to increase the support vector machine accuracy with implementation of feature selection and feature weighting. Feature selection will reduce a large number of irrelevant attributes. In this study the feature is selected based on the top value of K = 500. Once selected the relevant attributes are then performed feature weighting to calculate the weight of each attribute selected. The feature selection method used is chi square statistic and feature weighting using Term Frequency Inverse Document Frequency (TFIDF). Result of experiment using Matlab R2017b is integration of support vector machine with chi square statistic and TFIDF that uses 10 fold cross validation gives an increase of accuracy of 11.5% with the following explanation, the accuracy of the support vector machine without applying chi square statistic and TFIDF resulted in an accuracy of 68.7% and the accuracy of the support vector machine by applying chi square statistic and TFIDF resulted in an accuracy of 80.2%.

Term frequency combined hybrid feature selection method for spam filtering

2014 ◽  
Vol 19 (2) ◽  
pp. 369-383 ◽  
Author(s):  
Yuanning Liu ◽  
Youwei Wang ◽  
Lizhou Feng ◽  
Xiaodong Zhu
Keyword(s):  
Feature Selection ◽  
Feature Selection Method ◽  
Selection Method ◽  
Spam Filtering ◽  
Term Frequency

scholarly journals Motor Imagery EEG Decoding Based on New Spatial-Frequency Feature and Hybrid Feature Selection Method

2022 ◽  
Vol 2022 ◽  
pp. 1-12
Author(s):  
Yuan Tang ◽  
Zining Zhao ◽  
Shaorong Zhang ◽  
Zhi Li ◽  
Yun Mo ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Feature Selection ◽  
Spatial Frequency ◽  
Motor Imagery ◽  
Feature Selection Method ◽  
Selection Method ◽  
Support Vector ◽  
Fisher Score ◽  
Feature Extraction And Selection ◽  
Frequency Feature

Feature extraction and selection are important parts of motor imagery electroencephalogram (EEG) decoding and have always been the focus and difficulty of brain-computer interface (BCI) system research. In order to improve the accuracy of EEG decoding and reduce model training time, new feature extraction and selection methods are proposed in this paper. First, a new spatial-frequency feature extraction method is proposed. The original EEG signal is preprocessed, and then the common spatial pattern (CSP) is used for spatial filtering and dimensionality reduction. Finally, the filter bank method is used to decompose the spatially filtered signals into multiple frequency subbands, and the logarithmic band power feature of each frequency subband is extracted. Second, to select the subject-specific spatial-frequency features, a hybrid feature selection method based on the Fisher score and support vector machine (SVM) is proposed. The Fisher score of each feature is calculated, then a series of threshold parameters are set to generate different feature subsets, and finally, SVM and cross-validation are used to select the optimal feature subset. The effectiveness of the proposed method is validated using two sets of publicly available BCI competition data and a set of self-collected data. The total average accuracy of the three data sets achieved by the proposed method is 82.39%, which is 2.99% higher than the CSP method. The experimental results show that the proposed method has a better classification effect than the existing methods, and at the same time, feature extraction and feature selection time also have greater advantages.

scholarly journals Hybrid Feature Selection for Amharic News Document Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Demeke Endalie ◽  
Getamesay Haile
Keyword(s):  
Feature Selection ◽  
Information Gain ◽  
Feature Selection Method ◽  
Selection Method ◽  
Computational Time ◽  
Features Selection ◽  
Chi Square ◽  
Document Frequency ◽  
Automatic Text Classification ◽  
The Right

Today, the amount of Amharic digital documents has grown rapidly. Because of this, automatic text classification is extremely important. Proper selection of features has a crucial role in the accuracy of classification and computational time. When the initial feature set is considerably larger, it is important to pick the right features. In this paper, we present a hybrid feature selection method, called IGCHIDF, which consists of information gain (IG), chi-square (CHI), and document frequency (DF) features’ selection methods. We evaluate the proposed feature selection method on two datasets: dataset 1 containing 9 news categories and dataset 2 containing 13 news categories. Our experimental results showed that the proposed method performs better than other methods on both datasets 1and 2. The IGCHIDF method’s classification accuracy is up to 3.96% higher than the IG method, up to 11.16% higher than CHI, and 7.3% higher than DF on dataset 2, respectively.

Improvement of feature selection method in spam filtering

2009 ◽  
Vol 29 (10) ◽  
pp. 2812-2815
Author(s):  
Yang-zhu LU ◽  
Xin-you ZHANG ◽  
Yu QI
Keyword(s):  
Feature Selection ◽  
Feature Selection Method ◽  
Selection Method ◽  
Spam Filtering
Sign in / Sign up

Export Citation Format

Share Document