Document Clustering

2018 ◽  
pp. 47-64 ◽  
Author(s):  
Harsha Patil ◽  
R. S. Thakur
Keyword(s):  
Text Mining ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Web Pages ◽  
Digital Form ◽  
Search Query ◽  
Text Documents ◽  
Keen Interest ◽  
Use Of Internet

As we know use of Internet flourishes with its full velocity and in all dimensions. Enormous availability of Text documents in digital form (email, web pages, blog post, news articles, ebooks and other text files) on internet challenges technology to appropriate retrieval of document as a response for any search query. As a result there has been an eruption of interest in people to mine these vast resources and classify them properly. It invigorates researchers and developers to work on numerous approaches of document clustering. Researchers got keen interest in this problem of text mining. The aim of this chapter is to summarised different document clustering algorithms used by researchers.

Document Clustering

2017 ◽  
pp. 264-281
Author(s):  
Harsha Patil ◽  
R. S. Thakur
Keyword(s):  
Text Mining ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Web Pages ◽  
Digital Form ◽  
Search Query ◽  
Text Documents ◽  
Keen Interest ◽  
Use Of Internet

As we know use of Internet flourishes with its full velocity and in all dimensions. Enormous availability of Text documents in digital form (email, web pages, blog post, news articles, ebooks and other text files) on internet challenges technology to appropriate retrieval of document as a response for any search query. As a result there has been an eruption of interest in people to mine these vast resources and classify them properly. It invigorates researchers and developers to work on numerous approaches of document clustering. Researchers got keen interest in this problem of text mining. The aim of this chapter is to summarised different document clustering algorithms used by researchers.

Incorporating Text OLAP in Business Intelligence

2011 ◽  
pp. 77-101 ◽  
Author(s):  
Byung-Kwon Park ◽  
Il-Yeol Song
Keyword(s):  
Information Retrieval ◽  
Text Mining ◽  
Business Intelligence ◽  
Multidimensional Analysis ◽  
Web Pages ◽  
Data Types ◽  
Text Documents ◽  
Text Data ◽  
Platform Architecture ◽  
Unstructured Text

As the amount of data grows very fast inside and outside of an enterprise, it is getting important to seamlessly analyze both data types for total business intelligence. The data can be classified into two categories: structured and unstructured. For getting total business intelligence, it is important to seamlessly analyze both of them. Especially, as most of business data are unstructured text documents, including the Web pages in Internet, we need a Text OLAP solution to perform multidimensional analysis of text documents in the same way as structured relational data. We first survey the representative works selected for demonstrating how the technologies of text mining and information retrieval can be applied for multidimensional analysis of text documents, because they are major technologies handling text data. And then, we survey the representative works selected for demonstrating how we can associate and consolidate both unstructured text documents and structured relation data for obtaining total business intelligence. Finally, we present a future business intelligence platform architecture as well as related research topics. We expect the proposed total heterogeneous business intelligence architecture, which integrates information retrieval, text mining, and information extraction technologies all together, including relational OLAP technologies, would make a better platform toward total business intelligence.

The Method to Analyze Freely Described Data from Questionnaires

2009 ◽  
Vol 13 (3) ◽  
pp. 268-274 ◽  
Author(s):  
Masaomi Kimura ◽  
Keyword(s):  
Text Mining ◽  
Call Center ◽  
Clustering Algorithms ◽  
Web Pages ◽  
Research Papers ◽  
Text Data ◽  
Textual Data ◽  
The Way ◽  
Newspaper Articles

Text mining has been growing; mainly due to the need to extract useful information from vast amounts of textual data. Our target here is text data, a collection of freely described data from questionnaires. Unlike research papers, newspaper articles, call-center logs and web pages, which are usually the targets of text mining analysis, the freely described data contained in the questionnaire responses have specific characteristics, including a small number of short sentences forming individual pieces of data, while the wide variety of content precludes the applications of clustering algorithms used to classify the same. In this paper, we suggest the way to extract the opinions which are delivered by multiple respondents, based on the modification relationships included in each sentence in the freely described data. Certain applications of our method are also presented after the introduction of our approach.

scholarly journals Research of Clustering Algorithms using Enhanced Feature Selection

2019 ◽  
Vol 9 (2) ◽  
pp. 4612-4615
Keyword(s):  
Feature Selection ◽  
Text Mining ◽  
Clustering Algorithms ◽  
Present Situation ◽  
Features Selection ◽  
Entity Extraction ◽  
Text Documents ◽  
Clustering Techniques ◽  
Selection For ◽  
Video And Audio

In Present situation, a huge quantity of data is recorded in variety of forms like text, image, video, and audio and is estimated to enhance in future. The major tasks related to text are entity extraction, information extraction, entity relation modeling, document summarization are performed by using text mining. This paper main focus is on document clustering, a sub task of text mining and to measure the performance of different clustering techniques. In this paper we are using an enhanced features selection for clustering of text documents to prove that it produces better results compared to traditional feature selection.

Hierarchical Document Clustering

2011 ◽  
pp. 970-975 ◽  
Author(s):  
Benjamin C.M. Fung ◽  
Ke Wang ◽  
Martin Ester
Keyword(s):  
Hierarchical Clustering ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
High Volume ◽  
Frequent Itemset ◽  
Parent Child Relationship ◽  
Text Documents ◽  
Clustering Problem ◽  
Child Relationship ◽  
Divisive Hierarchical Clustering

Document clustering is an automatic grouping of text documents into clusters so that documents within a cluster have high similarity in comparison to one another, but are dissimilar to documents in other clusters. Unlike document classification (Wang, Zhou, & He, 2001), no labeled documents are provided in clustering; hence, clustering is also known as unsupervised learning. Hierarchical document clustering organizes clusters into a tree or a hierarchy that facilitates browsing. The parent-child relationship among the nodes in the tree can be viewed as a topic-subtopic relationship in a subject hierarchy such as the Yahoo! directory. This chapter discusses several special challenges in hierarchical document clustering: high dimensionality, high volume of data, ease of browsing, and meaningful cluster labels. State-of-the-art document clustering algorithms are reviewed: the partitioning method (Steinbach, Karypis, & Kumar, 2000), agglomerative and divisive hierarchical clustering (Kaufman & Rousseeuw, 1990), and frequent itemset-based hierarchical clustering (Fung, Wang, & Ester, 2003). The last one, which was developed by the authors, is further elaborated since it has been specially designed to address the hierarchical document clustering problem.

DBSCAN Algorithm for Document Clustering

2019 ◽  
Vol 9 (1) ◽  
pp. 58-66
Author(s):  
Radu G. Creţulescu ◽  
Daniel I. Morariu ◽  
Macarie Breazu ◽  
Daniel Volovici
Keyword(s):  
Clustering Algorithm ◽  
Information Gain ◽  
Spatial Clustering ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Feature Selection Method ◽  
Web Pages ◽  
Dbscan Algorithm ◽  
User Query ◽  
The Web

AbstractDocument clustering is a problem of automatically grouping similar document into categories based on some similarity metrics. Almost all available data, usually on the web, are unclassified so we need powerful clustering algorithms that work with these types of data. All common search engines return a list of pages relevant to the user query. This list needs to be generated fast and as correct as possible. For this type of problems, because the web pages are unclassified, we need powerful clustering algorithms. In this paper we present a clustering algorithm called DBSCAN – Density-Based Spatial Clustering of Applications with Noise – and its limitations on documents (or web pages) clustering. Documents are represented using the “bag-of-words” representation (word occurrence frequency). For this type o representation usually a lot of algorithms fail. In this paper we use Information Gain as feature selection method and evaluate the DBSCAN algorithm by its capacity to integrate in the clusters all the samples from the dataset.

scholarly journals Metaheuristics Based Clustering Algorithms on Document Clustering

2019 ◽  
pp. 39-45
Author(s):  
Aytug Onan
Keyword(s):  
Cluster Analysis ◽  
Optimization Problems ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Cuckoo Search ◽  
Text Documents ◽  
Text Document ◽  
Analysis Technique ◽  
Clustering Quality ◽  
Exploratory Data

Cluster analysis is an important exploratory data analysis technique which divides data into groups based on their similarity. Document clustering is the process of employing clustering algorithms on textual data so that text documents can be retrieved, organized, navigated and summarized in an efficient way. Document clustering can be utilized in the organization, summarization and classification of text documents. Metaheuristic algorithms have been successfully utilized to deal with complex optimization problems, including cluster analysis. In this paper, we analyze the clustering quality of five metaheuristic clustering algorithms (namely, particle swarm optimization, genetic algorithm, cuckoo search, firefly algorithm and yarasa algorithm) on fifteen text collections in term of F-measure. In the empirical analysis, two conventional clustering algorithms (K-means and bi-secting k-means) are also considered. The experimental analysis indicates that swarm-based clustering algorithms outperform conventional clustering algorithms on text document clustering.

An Improved B-hill Climbing Optimization Technique for Solving the Text Documents Clustering Problem

2020 ◽  
Vol 16 (4) ◽  
pp. 296-306 ◽  
Author(s):  
Laith Mohammad Abualigah ◽  
Essam Said Hanandeh ◽  
Ahamad Tajudin Khader ◽  
Mohammed Abdallh Otair ◽  
Shishir Kumar Shandilya
Keyword(s):  
Optimization Technique ◽  
Document Clustering ◽  
Text Clustering ◽  
Hill Climbing ◽  
Text Documents ◽  
Clustering Problem ◽  
Text Document ◽  
Text Information ◽  
Amount Of Knowledge ◽  
The Hill

Background: Considering the increasing volume of text document information on Internet pages, dealing with such a tremendous amount of knowledge becomes totally complex due to its large size. Text clustering is a common optimization problem used to manage a large amount of text information into a subset of comparable and coherent clusters. Aims: This paper presents a novel local clustering technique, namely, β-hill climbing, to solve the problem of the text document clustering through modeling the β-hill climbing technique for partitioning the similar documents into the same cluster. Methods: The β parameter is the primary innovation in β-hill climbing technique. It has been introduced in order to perform a balance between local and global search. Local search methods are successfully applied to solve the problem of the text document clustering such as; k-medoid and kmean techniques. Results: Experiments were conducted on eight benchmark standard text datasets with different characteristics taken from the Laboratory of Computational Intelligence (LABIC). The results proved that the proposed β-hill climbing achieved better results in comparison with the original hill climbing technique in solving the text clustering problem. Conclusion: The performance of the text clustering is useful by adding the β operator to the hill climbing.

scholarly journals Natural language processing methods for knowledge management—Applying document clustering for fast search and grouping of engineering documents

2021 ◽  
pp. 1063293X2098297
Author(s):  
Ivar Örn Arnarsson ◽  
Otto Frost ◽  
Emil Gustavsson ◽  
Mats Jirstrand ◽  
Johan Malmqvist
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Domain Knowledge ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Unstructured Data ◽  
Free Text ◽  
Engineering Change ◽  
Engineering Documents

Product development companies collect data in form of Engineering Change Requests for logged design issues, tests, and product iterations. These documents are rich in unstructured data (e.g. free text). Previous research affirms that product developers find that current IT systems lack capabilities to accurately retrieve relevant documents with unstructured data. In this research, we demonstrate a method using Natural Language Processing and document clustering algorithms to find structurally or contextually related documents from databases containing Engineering Change Request documents. The aim is to radically decrease the time needed to effectively search for related engineering documents, organize search results, and create labeled clusters from these documents by utilizing Natural Language Processing algorithms. A domain knowledge expert at the case company evaluated the results and confirmed that the algorithms we applied managed to find relevant document clusters given the queries tested.

Biomedical Document Clustering Based on Accelerated Symbiotic Organisms Search Algorithm

2021 ◽  
Vol 12 (4) ◽  
pp. 169-185
Author(s):  
Saida Ishak Boushaki ◽  
Omar Bendjeghaba ◽  
Nadjet Kamel
Keyword(s):  
Clustering Algorithm ◽  
Search Algorithm ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Latent Semantic Indexing ◽  
Research Area ◽  
Semantic Indexing ◽  
Local Optima ◽  
Symbiotic Organisms Search ◽  
Symbiotic Organisms

Clustering is an important unsupervised analysis technique for big data mining. It finds its application in several domains including biomedical documents of the MEDLINE database. Document clustering algorithms based on metaheuristics is an active research area. However, these algorithms suffer from the problems of getting trapped in local optima, need many parameters to adjust, and the documents should be indexed by a high dimensionality matrix using the traditional vector space model. In order to overcome these limitations, in this paper a new documents clustering algorithm (ASOS-LSI) with no parameters is proposed. It is based on the recent symbiotic organisms search metaheuristic (SOS) and enhanced by an acceleration technique. Furthermore, the documents are represented by semantic indexing based on the famous latent semantic indexing (LSI). Conducted experiments on well-known biomedical documents datasets show the significant superiority of ASOS-LSI over five famous algorithms in terms of compactness, f-measure, purity, misclassified documents, entropy, and runtime.

Sign in / Sign up

Export Citation Format

Share Document