scholarly journals A semantic approach for text document clustering using frequent itemsets and WordNet

2018 ◽  
Vol 7 (2.18) ◽  
pp. 102
Author(s):  
Harsha Patil ◽  
Ramjeevan Singh Thakur
Keyword(s):  
Clustering Algorithms ◽  
Document Clustering ◽  
Knowledge Bases ◽  
Experimental Result ◽  
Semantic Approach ◽  
Text Document ◽  
Clustering Quality ◽  
Ship Function ◽  
Membership Score ◽  
Specific Cluster

Document Clustering is an unsupervised method for classified documents in clusters on the basis of their similarity. Any document get it place in any specific cluster, on the basis of membership score, which calculated through membership function. But many of the traditional clustering algorithms are generally based on only BOW (Bag of Words), which ignores the semantic similarity between document and Cluster. In this research we consider the semantic association between cluster and text document during the calculation of membership score of any document for any specific cluster. Several researchers are working on semantic aspects of document clustering to develop clustering performance. Many external knowledge bases like WordNet, Wikipedia, Lucene etc. are utilized for this purpose. The proposed approach exploits WordNet to improve cluster member ship function. The experimental result shows that clustering quality improved significantly by using proposed framework of semantic approach. 

An improved ant algorithm with LDA-based representation for text document clustering

2016 ◽  
Vol 43 (2) ◽  
pp. 275-292 ◽  
Author(s):  
Aytug Onan ◽  
Hasan Bulut ◽  
Serdar Korukoglu
Keyword(s):  
Clustering Algorithm ◽  
Latent Dirichlet Allocation ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Clustering Methods ◽  
Initial Value ◽  
Text Document ◽  
Clustering Quality ◽  
Text Features

Document clustering can be applied in document organisation and browsing, document summarisation and classification. The identification of an appropriate representation for textual documents is extremely important for the performance of clustering or classification algorithms. Textual documents suffer from the high dimensionality and irrelevancy of text features. Besides, conventional clustering algorithms suffer from several shortcomings, such as slow convergence and sensitivity to the initial value. To tackle the problems of conventional clustering algorithms, metaheuristic algorithms are frequently applied to clustering. In this paper, an improved ant clustering algorithm is presented, where two novel heuristic methods are proposed to enhance the clustering quality of ant-based clustering. In addition, the latent Dirichlet allocation (LDA) is used to represent textual documents in a compact and efficient way. The clustering quality of the proposed ant clustering algorithm is compared to the conventional clustering algorithms using 25 text benchmarks in terms of F-measure values. The experimental results indicate that the proposed clustering scheme outperforms the compared conventional and metaheuristic clustering methods for textual documents.

scholarly journals Metaheuristics Based Clustering Algorithms on Document Clustering

2019 ◽  
pp. 39-45
Author(s):  
Aytug Onan
Keyword(s):  
Cluster Analysis ◽  
Optimization Problems ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Cuckoo Search ◽  
Text Documents ◽  
Text Document ◽  
Analysis Technique ◽  
Clustering Quality ◽  
Exploratory Data

Cluster analysis is an important exploratory data analysis technique which divides data into groups based on their similarity. Document clustering is the process of employing clustering algorithms on textual data so that text documents can be retrieved, organized, navigated and summarized in an efficient way. Document clustering can be utilized in the organization, summarization and classification of text documents. Metaheuristic algorithms have been successfully utilized to deal with complex optimization problems, including cluster analysis. In this paper, we analyze the clustering quality of five metaheuristic clustering algorithms (namely, particle swarm optimization, genetic algorithm, cuckoo search, firefly algorithm and yarasa algorithm) on fifteen text collections in term of F-measure. In the empirical analysis, two conventional clustering algorithms (K-means and bi-secting k-means) are also considered. The experimental analysis indicates that swarm-based clustering algorithms outperform conventional clustering algorithms on text document clustering.

scholarly journals Optimizing Text Categorization for Indonesian Text Using Clustering Label Technique

2021 ◽  
Vol 12 (3) ◽  
pp. 1483-1491
Author(s):  
Syopiansyah Jaya Putra Et.al
Keyword(s):  
Text Categorization ◽  
Intelligent System ◽  
Text Processing ◽  
Cluster Formation ◽  
Clustering Algorithms ◽  
Experimental Result ◽  
Text Documents ◽  
A Value ◽  
Digital Format ◽  
Clustering Quality

Text Categorization plays an important role for clustering the rapidly growing, yet unstructured, Indonesian text in digital format. Furthermore, it is deemed even more important since access to digital format text has become more necessary and widespread. There are many clustering algorithms used for text categorization. Unfortunately, clustering algorithms for text categorization cannot easily cluster the texts due to imperfect process of stemming and stopword of Indonesian language. This paper presents an intelligent system that categorizes Indonesian text documents into meaningful cluster labels. Label Induction Grouping Algorithm (LINGO) and Bisecting K- means are applied to process it through five phases, namely the pre-processing, frequent phrase extraction, cluster label induction, content discovery and final cluster formation. The experimental result showed that the system could categorize Indonesian text and reach to 93%. Furthermore, clustering quality evaluation indicates that text categorization using LINGO has high Precision and Recall with a value of 0.85 and 1, respectively, compare to Bisecting K-means which has a value of 0.78 and 0.99. Therefore, the result shows that LINGO is suitable for categorizing Indonesian text. The main contribution of this study is to optimize the clustering results by applying and maximizing text processing using Indonesian stemmer and stopword.

Swarm Intelligence in Text Document Clustering

2010 ◽  
pp. 165-180 ◽  
Author(s):  
Xiaohui Cui
Keyword(s):  
Swarm Intelligence ◽  
Clustering Analysis ◽  
Information Society ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
High Quality ◽  
Fish Schools ◽  
Text Document ◽  
Self Organized ◽  
Swarm Algorithms

In this chapter, we introduce three nature inspired swarm intelligence clustering approaches for document clustering analysis. The major challenge of today’s information society is being overwhelmed with information on any topic they are searching for. Fast and high-quality document clustering algorithms play an important role in helping users to effectively navigate, summarize, and organize the overwhelmed information. The swarm intelligence clustering algorithms use stochastic and heuristic principles discovered from observing bird flocks, fish schools, and ant food forage. Compared to the traditional clustering algorithms, the swarm algorithms are usually flexible, robust, decentralized, and self-organized. These characters make the swarm algorithms suitable for solving complex problems, such as document clustering.

DIC-DOC-K-means: Dissimilarity-based Initial Centroid selection for DOCument clustering using K-means for improving the effectiveness of text document clustering

2018 ◽  
Vol 45 (6) ◽  
pp. 818-832 ◽  
Author(s):  
R Lakshmi ◽  
S Baskar
Keyword(s):  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Weighted Average ◽  
Minimum Standard ◽  
Data Set ◽  
Text Document ◽  
Selection For ◽  
Number Of Classes ◽  
Better Than

In this article, a new initial centroid selection for a K-means document clustering algorithm, namely, Dissimilarity-based Initial Centroid selection for DOCument clustering using K-means (DIC-DOC- K-means), to improve the performance of text document clustering is proposed. The first centroid is the document having the minimum standard deviation of its term frequency. Each of the other subsequent centroids is selected based on the dissimilarities of the previously selected centroids. For comparing the performance of the proposed DIC-DOC- K-means algorithm, the results of the K-means, K-means++ and weighted average of terms-based initial centroid selection +  K-means (Weight_Avg_Initials +  K-means) clustering algorithms are considered. The results show that the proposed DIC-DOC- K-means algorithm performs significantly better than the K-means, K-means++ and Weight_Avg_Initials+  K-means clustering algorithms for Reuters-21578 and WebKB with respect to purity, entropy and F-measure for most of the cluster sizes. The cluster sizes used for Reuters-8 are 8, 16, 24 and 32 and those for WebKB are 4, 8, 12 and 16. The results of the proposed DIC-DOC- K-means give a better performance for the number of clusters that are equal to the number of classes in the data set.

scholarly journals Evaluation of text document clustering approach based on particle swarm optimization

2013 ◽  
Vol 3 (2) ◽  
Author(s):  
Stuti Karol ◽  
Veenu Mangat
Keyword(s):  
Data Mining ◽  
Particle Swarm Optimization ◽  
Clustering Algorithms ◽  
Hybrid Approach ◽  
Document Clustering ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Swarm Optimization ◽  
Fuzzy C Means ◽  
Text Document

AbstractClustering, an extremely important technique in Data Mining is an automatic learning technique aimed at grouping a set of objects into subsets or clusters. The goal is to create clusters that are coherent internally, but substantially different from each other. Text Document Clustering refers to the clustering of related text documents into groups based upon their content. It is a fundamental operation used in unsupervised document organization, text data mining, automatic topic extraction, and information retrieval. Fast and high-quality document clustering algorithms play an important role in effectively navigating, summarizing, and organizing information. The documents to be clustered can be web news articles, abstracts of research papers etc. This paper proposes two techniques for efficient document clustering involving the application of soft computing approach as an intelligent hybrid approach PSO algorithm. The proposed approach involves partitioning Fuzzy C-Means algorithm and K-Means algorithm each hybridized with Particle Swarm Optimization (PSO). The performance of these hybrid algorithms has been evaluated against traditional partitioning techniques (K-Means and Fuzzy C Means).

An Improved B-hill Climbing Optimization Technique for Solving the Text Documents Clustering Problem

2020 ◽  
Vol 16 (4) ◽  
pp. 296-306 ◽  
Author(s):  
Laith Mohammad Abualigah ◽  
Essam Said Hanandeh ◽  
Ahamad Tajudin Khader ◽  
Mohammed Abdallh Otair ◽  
Shishir Kumar Shandilya
Keyword(s):  
Optimization Technique ◽  
Document Clustering ◽  
Text Clustering ◽  
Hill Climbing ◽  
Text Documents ◽  
Clustering Problem ◽  
Text Document ◽  
Text Information ◽  
Amount Of Knowledge ◽  
The Hill

Background: Considering the increasing volume of text document information on Internet pages, dealing with such a tremendous amount of knowledge becomes totally complex due to its large size. Text clustering is a common optimization problem used to manage a large amount of text information into a subset of comparable and coherent clusters. Aims: This paper presents a novel local clustering technique, namely, β-hill climbing, to solve the problem of the text document clustering through modeling the β-hill climbing technique for partitioning the similar documents into the same cluster. Methods: The β parameter is the primary innovation in β-hill climbing technique. It has been introduced in order to perform a balance between local and global search. Local search methods are successfully applied to solve the problem of the text document clustering such as; k-medoid and kmean techniques. Results: Experiments were conducted on eight benchmark standard text datasets with different characteristics taken from the Laboratory of Computational Intelligence (LABIC). The results proved that the proposed β-hill climbing achieved better results in comparison with the original hill climbing technique in solving the text clustering problem. Conclusion: The performance of the text clustering is useful by adding the β operator to the hill climbing.

scholarly journals Natural language processing methods for knowledge management—Applying document clustering for fast search and grouping of engineering documents

2021 ◽  
pp. 1063293X2098297
Author(s):  
Ivar Örn Arnarsson ◽  
Otto Frost ◽  
Emil Gustavsson ◽  
Mats Jirstrand ◽  
Johan Malmqvist
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Domain Knowledge ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Unstructured Data ◽  
Free Text ◽  
Engineering Change ◽  
Engineering Documents

Product development companies collect data in form of Engineering Change Requests for logged design issues, tests, and product iterations. These documents are rich in unstructured data (e.g. free text). Previous research affirms that product developers find that current IT systems lack capabilities to accurately retrieve relevant documents with unstructured data. In this research, we demonstrate a method using Natural Language Processing and document clustering algorithms to find structurally or contextually related documents from databases containing Engineering Change Request documents. The aim is to radically decrease the time needed to effectively search for related engineering documents, organize search results, and create labeled clusters from these documents by utilizing Natural Language Processing algorithms. A domain knowledge expert at the case company evaluated the results and confirmed that the algorithms we applied managed to find relevant document clusters given the queries tested.

Sign in / Sign up

Export Citation Format

Share Document