Multilingual text categorization using Character N-gram

2008 ◽  
Author(s):  
Makoto Suzuki ◽  
Naohide Yamagishi ◽  
Yi-Ching Tsai ◽  
Shigeichi Hirasawa
Keyword(s):  
Text Categorization ◽  
N Gram ◽  
Multilingual Text

Efficient n-gram construction for text categorization using feature selection techniques

2021 ◽  
Vol 25 (3) ◽  
pp. 509-525
Author(s):  
Maximiliano García ◽  
Sebastián Maldonado ◽  
Carla Vairetti
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Text Categorization ◽  
A Priori ◽  
Predictive Performance ◽  
Online Reviews ◽  
Additional Advantage ◽  
Novel Approach ◽  
N Gram ◽  
Feature Selection Techniques

In this paper, we present a novel approach for n-gram generation in text classification. The a-priori algorithm is adapted to prune word sequences by combining three feature selection techniques. Unlike the traditional two-step approach for text classification in which feature selection is performed after the n-gram construction process, our proposal performs an embedded feature elimination during the application of the a-priori algorithm. The proposed strategy reduces the number of branches to be explored, speeding up the process and making the construction of all the word sequences tractable. Our proposal has the additional advantage of constructing a low-dimensional dataset with only the features that are relevant for classification, that can be used directly without the need for a feature selection step. Experiments on text classification datasets for sentiment analysis demonstrate that our approach yields the best predictive performance when compared with other feature selection approaches, while also facilitating a better understanding of the words and phrases that explain a given task; in our case online reviews and ratings in various domains.

Multi-Agents Machine Learning (MML) System for Plagiarism Detection

2018 ◽  
pp. 103-119
Author(s):  
Hadj Ahmed Bouarara
Keyword(s):  
Machine Learning ◽  
Text Categorization ◽  
Learning System ◽  
Modern World ◽  
Support Vector ◽  
Plagiarism Detection ◽  
Supervised Learning Algorithms ◽  
N Gram ◽  
The Web ◽  
Multi Agents

Day after day the cases of plagiarism increase and become a crucial problem in the modern world caused by the quantity of textual information available in the web. As data mining becomes the foundation for many different domains, one of its chores is a text categorization that can be used in order to resolve the impediment of automatic plagiarism detection. This chapter is devoted to a new approach for combating plagiarism named MML (Multi-agents Machine Learning system) composed of three modules: data preparation and digitalization, using n-gram character or bag of words as methods for the text representation, TF*IDF as weighting to calculate the importance of each term in the corpus in order to transform each document to a vector, and learning and vote phase using three supervised learning algorithms (decision tree c4.5, naïve Bayes, and support vector machine).

scholarly journals Text Categorization of Heart, Lung, and Blood Studies in the Database of Genotypes and Phenotypes (dbGaP) Utilizing n-grams and Metadata Features

2013 ◽  
Vol 6 ◽  
pp. BII.S11987 ◽  
Author(s):  
Mindy K. Ross ◽  
Ko-Wei Lin ◽  
Karen Truong ◽  
Abhishek Kumar ◽  
Mike Conway
Keyword(s):  
Text Categorization ◽  
Document Retrieval ◽  
Classification Performance ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Study Results ◽  
Classifier Performance ◽  
C4.5 Decision Tree ◽  
N Gram ◽  
Effective Use

The database of Genotypes and Phenotypes (dbGaP) allows researchers to understand phenotypic contribution to genetic conditions, generate new hypotheses, confirm previous study results, and identify control populations. However, effective use of the database is hindered by suboptimal study retrieval. Our objective is to evaluate text classification techniques to improve study retrieval in the context of the dbGaP database. We utilized standard machine learning algorithms (naive Bayes, support vector machines, and the C4.5 decision tree) trained on dbGaP study text and incorporated n-gram features and study metadata to identify heart, lung, and blood studies. We used the χ2 feature selection algorithm to identify features that contributed most to classification performance and experimented with dbGaP associated PubMed papers as a proxy for topicality. Classifier performance was favorable in comparison to keyword-based search results. It was determined that text categorization is a useful complement to document retrieval techniques in the dbGaP

Sign in / Sign up

Export Citation Format

Share Document