A survey on parallel clustering algorithms for Big Data

2020 ◽  
Author(s):  
Zineb Dafir ◽  
Yasmine Lamari ◽  
Said Chah Slaoui
Keyword(s):  
Big Data ◽  
Clustering Algorithms ◽  
Parallel Clustering

scholarly journals PARSUC: A Parallel Subsampling-Based Method for Clustering Remote Sensing Big Data

Sensors ◽  
2019 ◽  
Vol 19 (15) ◽  
pp. 3438 ◽  
Author(s):  
Xia ◽  
Huang ◽  
Li ◽  
Zhou ◽  
Zhang
Keyword(s):  
Remote Sensing ◽  
Big Data ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Image Data ◽  
Data Partitioning ◽  
Data Mining Technique ◽  
Mining Technique ◽  
Hadoop Platform ◽  
Parallel Clustering

Remote sensing big data (RSBD) is generally characterized by huge volumes, diversity, and high dimensionality. Mining hidden information from RSBD for different applications imposes significant computational challenges. Clustering is an important data mining technique widely used in processing and analyzing remote sensing imagery. However, conventional clustering algorithms are designed for relatively small datasets. When applied to problems with RSBD, they are, in general, too slow or inefficient for practical use. In this paper, we proposed a parallel subsampling-based clustering (PARSUC) method for improving the performance of RSBD clustering in terms of both efficiency and accuracy. PARSUC leverages a novel subsampling-based data partitioning (SubDP) method to realize three-step parallel clustering, effectively solving the notable performance bottleneck of the existing parallel clustering algorithms; that is, they must cope with numerous repeated calculations to get a reasonable result. Furthermore, we propose a centroid filtering algorithm (CFA) to eliminate subsampling errors and to guarantee the accuracy of the clustering results. PARSUC was implemented on a Hadoop platform by using the MapReduce parallel model. Experiments conducted on massive remote sensing imageries with different sizes showed that PARSUC (1) provided much better accuracy than conventional remote sensing clustering algorithms in handling larger image data; (2) achieved notable scalability with increased computing nodes added; and (3) spent much less time than the existing parallel clustering algorithm in handling RSBD.

scholarly journals A Survey of Parallel Clustering Algorithms Based on Spark

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Wen Xiao ◽  
Juan Hu
Keyword(s):  
Machine Learning ◽  
Image Processing ◽  
Big Data ◽  
Information Retrieval ◽  
Social Network ◽  
Clustering Algorithms ◽  
Future Research ◽  
Parallel Design ◽  
Learning Tasks ◽  
Parallel Clustering

Clustering is one of the most important unsupervised machine learning tasks, which is widely used in information retrieval, social network analysis, image processing, and other fields. With the explosive growth of data, the classical clustering algorithms cannot meet the requirements of clustering for big data. Spark is one of the most popular parallel processing platforms for big data, and many researchers have proposed many parallel clustering algorithms based on Spark. In this paper, the existing parallel clustering algorithms based on Spark are classified and summarized, the parallel design framework of each kind of algorithms is discussed, and after comparing different kinds of algorithms, the direction of the future research is discussed.

An Introduction to Clustering Algorithms in Big Data

2021 ◽  
pp. 559-576
Author(s):  
Rajit Nair ◽  
Amit Bhagat
Keyword(s):  
Big Data ◽  
Single Machine ◽  
Data Clustering ◽  
Clustering Algorithms ◽  
Time Limit ◽  
Computation Cost ◽  
Different Types ◽  
Clustering Approach ◽  
Future Path ◽  
Parallel Clustering

In big data, clustering is the process through which analysis is performed. Since the data is big, it is very difficult to perform clustering approach. Big data is mainly termed as petabytes and zeta bytes of data and high computation cost is needed for the implementation of clusters. In this chapter, the authors show how clustering can be performed on big data and what are the different types of clustering approach. The challenge during clustering approach is to find observations within the time limit. The chapter also covers the possible future path for more advanced clustering algorithms. The chapter will cover single machine clustering and multiple machines clustering, which also includes parallel clustering.

Survey on Partition based Clustering Algorithms in Big Data

2017 ◽  
Vol 5 (12) ◽  
pp. 323-325
Author(s):  
E. Mahima Jane ◽  
◽  
◽  
E. George Dharma Prakash Raj
Keyword(s):  
Big Data ◽  
Clustering Algorithms

Big Data Mining Based on Computational Intelligence and Fuzzy Clustering

2015 ◽  
pp. 130-148 ◽  
Author(s):  
Usman Akhtar ◽  
Mehdi Hassan
Keyword(s):  
Big Data ◽  
Computational Intelligence ◽  
Clustering Algorithms ◽  
Heterogeneous Data ◽  
Computational Time ◽  
Statistical Features ◽  
Feature Sets ◽  
Clustering Quality ◽  
The Given ◽  
Different Sources

The availability of a huge amount of heterogeneous data from different sources to the Internet has been termed as the problem of Big Data. Clustering is widely used as a knowledge discovery tool that separate the data into manageable parts. There is a need of clustering algorithms that scale on big databases. In this chapter we have explored various schemes that have been used to tackle the big databases. Statistical features have been extracted and most important and relevant features have been extracted from the given dataset. Reduce and irrelevant features have been eliminated and most important features have been selected by genetic algorithms (GA).Clustering with reduced feature sets requires lower computational time and resources. Experiments have been performed at standard datasets and results indicate that the proposed scheme based clustering offers high clustering accuracy. To check the clustering quality various quality measures have been computed and it has been observed that the proposed methodology results improved significantly. It has been observed that the proposed technique offers high quality clustering.

Big Data Mining Based on Computational Intelligence and Fuzzy Clustering

Web Services ◽  
2019 ◽  
pp. 413-430
Author(s):  
Usman Akhtar ◽  
Mehdi Hassan
Keyword(s):  
Big Data ◽  
Computational Intelligence ◽  
Clustering Algorithms ◽  
Heterogeneous Data ◽  
Computational Time ◽  
Statistical Features ◽  
Feature Sets ◽  
Clustering Quality ◽  
The Given ◽  
Different Sources

The availability of a huge amount of heterogeneous data from different sources to the Internet has been termed as the problem of Big Data. Clustering is widely used as a knowledge discovery tool that separate the data into manageable parts. There is a need of clustering algorithms that scale on big databases. In this chapter we have explored various schemes that have been used to tackle the big databases. Statistical features have been extracted and most important and relevant features have been extracted from the given dataset. Reduce and irrelevant features have been eliminated and most important features have been selected by genetic algorithms (GA). Clustering with reduced feature sets requires lower computational time and resources. Experiments have been performed at standard datasets and results indicate that the proposed scheme based clustering offers high clustering accuracy. To check the clustering quality various quality measures have been computed and it has been observed that the proposed methodology results improved significantly. It has been observed that the proposed technique offers high quality clustering.

Sign in / Sign up

Export Citation Format

Share Document