An ensemble clustering approach for topic discovery using implicit text segmentation

2020 ◽  
pp. 016555152091159
Author(s):  
Muhammad Qasim Memon ◽  
Yu Lu ◽  
Penghe Chen ◽  
Aasma Memon ◽  
Muhammad Salman Pathan ◽  
...  
Keyword(s):  
Clustering Algorithms ◽  
Data Retrieval ◽  
Text Clustering ◽  
Text Segmentation ◽  
Ensemble Clustering ◽  
Reduction Techniques ◽  
Average Improvement ◽  
Text Collections ◽  
Clustering Approach ◽  
Topic Structure

Text segmentation (TS) is the process of dividing multi-topic text collections into cohesive segments using topic boundaries. Similarly, text clustering has been renowned as a major concern when it comes to multi-topic text collections, as they are distinguished by sub-topic structure and their contents are not associated with each other. Existing clustering approaches follow the TS method which relies on word frequencies and may not be suitable to cluster multi-topic text collections. In this work, we propose a new ensemble clustering approach (ECA) is a novel topic-modelling-based clustering approach, which induces the combination of TS and text clustering. We improvised a LDA-onto (LDA-ontology) is a TS-based model, which presents a deterioration of a document into segments (i.e. sub-documents), wherein each sub-document is associated with exactly one sub-topic. We deal with the problem of clustering when it comes to a document that is intrinsically related to various topics and its topical structure is missing. ECA is tested through well-known datasets in order to provide a comprehensive presentation and validation of clustering algorithms using LDA-onto. ECA exhibits the semantic relations of keywords in sub-documents and resultant clusters belong to original documents that they contain. Moreover, present research sheds the light on clustering performances and it indicates that there is no difference over performances (in terms of F-measure) when the number of topics changes. Our findings give above par results in order to analyse the problem of text clustering in a broader spectrum without applying dimension reduction techniques over high sparse data. Specifically, ECA provides an efficient and significant framework than the traditional and segment-based approach, such that achieved results are statistically significant with an average improvement of over 10.2%. For the most part, proposed framework can be evaluated in applications where meaningful data retrieval is useful, such as document summarization, text retrieval, novelty and topic detection.

scholarly journals FIREFLYCLUST: AN AUTOMATED HIERARCHICAL TEXT CLUSTERING APPROACH

2017 ◽  
Vol 79 (5) ◽  
Author(s):  
Athraa Jasim Mohammed ◽  
Yuhanis Yusof ◽  
Husniza Husni
Keyword(s):  
Clustering Algorithms ◽  
Threshold Value ◽  
Text Clustering ◽  
Optimal Number ◽  
Clustering Method ◽  
Text Documents ◽  
Clustering Approach ◽  
Retrieval Engine ◽  
Five Phases ◽  
Optimal Number Of Clusters

Text clustering is one of the text mining tasks that is employed in search engines. Discovering the optimal number of clusters for a dataset or repository is a challenging problem. Various clustering algorithms have been reported in the literature but most of them rely on a pre-defined value of the k clusters. In this study, a variant of Firefly algorithm, termed as FireflyClust, is proposed to automatically cluster text documents in a hierarchical manner. The proposed clustering method operates based on five phases: data pre-processing, clustering, item re-location, cluster selection and cluster refinement. Experiments are undertaken based on different selections of threshold value. Results on the TREC collection named TR11, TR12, TR23 and TR45, showed that the FireflyClust is a better approach than the Bisect K-means, hybrid Bisect K-means and Practical General Stochastic Clustering Method. Such a result would enlighten the directions in developing a better information retrieval engine for this dynamic and fast growing big data era.

A Hard C-Means Clustering Algorithm Incorporating Membership KL Divergence and Local Data Information for Noisy Image Segmentation

2017 ◽  
Vol 32 (04) ◽  
pp. 1850012 ◽  
Author(s):  
R. R. Gharieb ◽  
G. Gendy ◽  
H. Selim
Keyword(s):  
Image Segmentation ◽  
Membership Function ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Cluster Center ◽  
Local Data ◽  
Cluster Membership ◽  
Kl Divergence ◽  
Clustering Approach ◽  
Center Distance

In this paper, the standard hard C-means (HCM) clustering approach to image segmentation is modified by incorporating weighted membership Kullback–Leibler (KL) divergence and local data information into the HCM objective function. The membership KL divergence, used for fuzzification, measures the proximity between each cluster membership function of a pixel and the locally-smoothed value of the membership in the pixel vicinity. The fuzzification weight is a function of the pixel to cluster-centers distances. The used pixel to a cluster-center distance is composed of the original pixel data distance plus a fraction of the distance generated from the locally-smoothed pixel data. It is shown that the obtained membership function of a pixel is proportional to the locally-smoothed membership function of this pixel multiplied by an exponentially distributed function of the minus pixel distance relative to the minimum distance provided by the nearest cluster-center to the pixel. Therefore, since incorporating the locally-smoothed membership and data information in addition to the relative distance, which is more tolerant to additive noise than the absolute distance, the proposed algorithm has a threefold noise-handling process. The presented algorithm, named local data and membership KL divergence based fuzzy C-means (LDMKLFCM), is tested by synthetic and real-world noisy images and its results are compared with those of several FCM-based clustering algorithms.

scholarly journals Canonical PSO Based K-Means Clustering Approach for Real Datasets

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Lopamudra Dey ◽  
Sanjay Chakraborty
Keyword(s):  
Data Mining ◽  
Air Pollution ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Real Life ◽  
Cluster Validity ◽  
Validity Assessment ◽  
Different Types ◽  
Clustering Approach ◽  
Validity Measure

“Clustering” the significance and application of this technique is spread over various fields. Clustering is an unsupervised process in data mining, that is why the proper evaluation of the results and measuring the compactness and separability of the clusters are important issues. The procedure of evaluating the results of a clustering algorithm is known as cluster validity measure. Different types of indexes are used to solve different types of problems and indices selection depends on the kind of available data. This paper first proposes Canonical PSO based K-means clustering algorithm and also analyses some important clustering indices (intercluster, intracluster) and then evaluates the effects of those indices on real-time air pollution database, wholesale customer, wine, and vehicle datasets using typical K-means, Canonical PSO based K-means, simple PSO based K-means, DBSCAN, and Hierarchical clustering algorithms. This paper also describes the nature of the clusters and finally compares the performances of these clustering algorithms according to the validity assessment. It also defines which algorithm will be more desirable among all these algorithms to make proper compact clusters on this particular real life datasets. It actually deals with the behaviour of these clustering algorithms with respect to validation indexes and represents their results of evaluation in terms of mathematical and graphical forms.

Short Text Clustering Algorithms for Weibo Topic Detection

2014 ◽  
Vol 971-973 ◽  
pp. 1747-1751 ◽  
Author(s):  
Lei Zhang ◽  
Hai Qiang Chen ◽  
Wei Jie Li ◽  
Yan Zhao Liu ◽  
Run Pu Wu
Keyword(s):  
Text Analysis ◽  
Semantic Information ◽  
Clustering Algorithms ◽  
Text Clustering ◽  
Massive Data ◽  
Topic Detection ◽  
Clustering Methods ◽  
Short Text ◽  
Short Text Clustering ◽  
Application Requirements

Text clustering is a popular research topic in the field of text mining, and now there are a lot of text clustering methods catering to different application requirements. Currently, Weibo data acquisition is through the API provided by big microblogging platforms. In this essay, we will discuss the algorithm of extracting popular topics posted by Weibo users by text clustering after massive data collection. Due to the fact that traditional text analysis may not be applicable to short texts used in Weibo, text clustering shall be carried out through combining multiple posts into long texts, based on their features (forwards, comments and followers, etc.). Either frequency-based or density-based short text clustering can deliver in most cases. The former is applicable to find hot topics from large Weibo short texts, and the latter is applicable to find abnormal contents. Both the two methods use semantic information to improve the accuracy of clustering. Besides, they improve the performance of clustering through the parallelism.

Energy capable clustering method for extend the duration of IoT based mobile wireless sensor network with remote nodes

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Giji Kiruba ◽  
Benita
Keyword(s):  
Cluster Formation ◽  
Clustering Algorithms ◽  
Cluster Head ◽  
Average Energy ◽  
Energy Performance ◽  
Residual Energy ◽  
Clustering Approach ◽  
Study Methodology ◽  
Mobile Wireless Sensor ◽  
Local Average

Abstract The energy performance of IoT-MWSNs may be augmented by using a suitable clustering technique for integrating IoT sensors. Clustering, on the other hand, requires additional overhead, such as determining the cluster head and cluster formation. Environmental Energy Attentive Clustering with Remote Nodes is a unique environmental energy attentive clustering approach for IoT-MWSNs proposed in this study methodology (E2ACRN). Cluster head (CH) in E2ACRN is entirely determined by weight. The residual energy of each IoT sensor and the local average energy of all IoT sensors in the cluster are used to calculate the weight. Inappropriately planned allocated clustering techniques might result in nodes being too far away from CH. These distant nodes communicate with the sink by using more energy. The ambient average energy, remoteness among IoT sensors, and sink are used to determine whether a distant node transmits its information to a CH in the previous cycle or to sink in order to lengthen lifetime. The simulation results of the current technique revealed that E2ACRN performs better than previous clustering algorithms.

Featureless Data Clustering

2010 ◽  
pp. 141-164 ◽  
Author(s):  
Wilson Wong
Keyword(s):  
Data Clustering ◽  
Dominant Role ◽  
Clustering Algorithms ◽  
Adaptive Clustering ◽  
Feature Based ◽  
Clustering Approach ◽  
Semantic Computation ◽  
The Many ◽  
Time Required ◽  
Existing Data

Feature-based semantic measurements have played a dominant role in conventional data clustering algorithms for many existing applications. However, the applicability of existing data clustering approaches to a wider range of applications is limited due to issues such as complexity involved in semantic computation, long pre-processing time required for feature preparation, and poor extensibility of semantic measurement due to non-incremental feature source. This chapter first summarises the many commonly used clustering algorithms and feature-based semantic measurements, and then highlights the shortcomings to make way for the proposal of an adaptive clustering approach based on featureless semantic measurements. The chapter concludes with experiments demonstrating the performance and wide applicability of the proposed clustering approach.

scholarly journals A Novel Hierarchical Clustering Approach Based on Universal Gravitation

2020 ◽  
Vol 2020 ◽  
pp. 1-15
Author(s):  
Peng Zhang ◽  
Kun She
Keyword(s):  
Hierarchical Clustering ◽  
Clustering Analysis ◽  
Gravitational Force ◽  
Clustering Algorithms ◽  
Influence Coefficient ◽  
Data Sets ◽  
Universal Gravitation ◽  
Real World Data ◽  
Gravitational Influence ◽  
Clustering Approach

The target of the clustering analysis is to group a set of data points into several clusters based on the similarity or distance. The similarity or distance is usually a scalar used in numerous traditional clustering algorithms. Nevertheless, a vector, such as data gravitational force, contains more information than a scalar and can be applied in clustering analysis to promote clustering performance. Therefore, this paper proposes a three-stage hierarchical clustering approach called GHC, which takes advantage of the vector characteristic of data gravitational force inspired by the law of universal gravitation. In the first stage, a sparse gravitational graph is constructed based on the top k data gravitations between each data point and its neighbors in the local region. Then the sparse graph is partitioned into many subgraphs by the gravitational influence coefficient. In the last stage, the satisfactory clustering result is obtained by merging these subgraphs iteratively by using a new linkage criterion. To demonstrate the performance of GHC algorithm, the experiments on synthetic and real-world data sets are conducted, and the results show that the GHC algorithm achieves better performance than the other existing clustering algorithms.

scholarly journals Enriching Documents by Linking Salient Entities and Lexical-Semantic Expansion

2018 ◽  
Vol 29 (1) ◽  
pp. 1109-1121
Author(s):  
Mohsen Pourvali ◽  
Salvatore Orlando
Keyword(s):  
Clustering Algorithms ◽  
Ensemble Clustering ◽  
British Broadcasting Corporation ◽  
Text Documents ◽  
Classical Text ◽  
Text Corpora ◽  
Clustering Quality ◽  
Semantic Expansion ◽  
Document Representations

Abstract This paper explores a multi-strategy technique that aims at enriching text documents for improving clustering quality. We use a combination of entity linking and document summarization in order to determine the identity of the most salient entities mentioned in texts. To effectively enrich documents without introducing noise, we limit ourselves to the text fragments mentioning the salient entities, in turn, belonging to a knowledge base like Wikipedia, while the actual enrichment of text fragments is carried out using WordNet. To feed clustering algorithms, we investigate different document representations obtained using several combinations of document enrichment and feature extraction. This allows us to exploit ensemble clustering, by combining multiple clustering results obtained using different document representations. Our experiments indicate that our novel enriching strategies, combined with ensemble clustering, can improve the quality of classical text clustering when applied to text corpora like The British Broadcasting Corporation (BBC) NEWS.

Sign in / Sign up

Export Citation Format

Share Document