An Efficient K-Medoids Clustering Algorithm for Large Scale Data

2019 ◽  
pp. 85-108
Author(s):  
Xiaochun Wang ◽  
Xiali Wang ◽  
Don Mitchell Wilkes
Keyword(s):  
Large Scale ◽  
Clustering Algorithm ◽  
Large Scale Data ◽  
Scale Data

A stratified sampling based clustering algorithm for large-scale data

2019 ◽  
Vol 163 ◽  
pp. 416-428 ◽  
Author(s):  
Xingwang Zhao ◽  
Jiye Liang ◽  
Chuangyin Dang
Keyword(s):  
Large Scale ◽  
Clustering Algorithm ◽  
Stratified Sampling ◽  
Large Scale Data ◽  
Scale Data

Using Uncertain DM-Chameleon Clustering Algorithm Based on Machine Learning to Predict Landslide Hazards

2019 ◽  
Vol 31 (2) ◽  
pp. 329-338 ◽  
Author(s):  
Jian Hu ◽  
Haiwan Zhu ◽  
Yimin Mao ◽  
Canlong Zhang ◽  
Tian Liang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Uncertain Data ◽  
Landslide Hazard ◽  
Data Sets ◽  
Large Scale Data ◽  
Landslide Hazards ◽  
Hazard Levels ◽  
Scale Data

Landslide hazard prediction is a difficult, time-consuming process when traditional methods are used. This paper presents a method that uses machine learning to predict landslide hazard levels automatically. Due to difficulties in obtaining and effectively processing rainfall in landslide hazard prediction, and to the existing limitation in dealing with large-scale data sets in the M-chameleon algorithm, a new method based on an uncertain DM-chameleon algorithm (developed M-chameleon) is proposed to assess the landslide susceptibility model. First, this method designs a new two-phase clustering algorithm based on M-chameleon, which effectively processes large-scale data sets. Second, the new E-H distance formula is designed by combining the Euclidean and Hausdorff distances, and this enables the new method to manage uncertain data effectively. The uncertain data model is presented at the same time to effectively quantify triggering factors. Finally, the model for predicting landslide hazards is constructed and verified using the data from the Baota district of the city of Yan’an, China. The experimental results show that the uncertain DM-chameleon algorithm of machine learning can effectively improve the accuracy of landslide prediction and has high feasibility. Furthermore, the relationships between hazard factors and landslide hazard levels can be extracted based on clustering results.

scholarly journals Privacy-preserving constrained spectral clustering algorithm for large-scale data sets

2020 ◽  
Vol 14 (3) ◽  
pp. 321-331 ◽  
Author(s):  
Ji Li ◽  
Jianghong Wei ◽  
Mao Ye ◽  
Wenfen Liu ◽  
Xuexian Hu
Keyword(s):  
Spectral Clustering ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Privacy Preserving ◽  
Data Sets ◽  
Large Scale Data ◽  
Spectral Clustering Algorithm ◽  
Scale Data ◽  
Large Scale Data Sets

Stochastic Gradient Descent Based K-Means Algorithm on Large Scale Data Clustering

2014 ◽  
Vol 687-691 ◽  
pp. 1342-1345 ◽  
Author(s):  
Jie Ding ◽  
Li Peng Zhu ◽  
Bin Hu ◽  
Ren Long Hang ◽  
Yu Bao Sun
Keyword(s):  
Gradient Descent ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Distance Matrix ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Data Sets ◽  
Human Beings ◽  
Large Scale Data ◽  
Scale Data

With the rapid advance of data collection and storage technique, it is easy to acquire tens of millions or even billions of data sets. How to explore and exploit the useful or interesting information for human beings from these data sets has become an urgent issue. Traditional k-means clustering algorithm has been widely used in data mining community. First, randomly initialize k clustering centres. Then, all instances are classified into k different classes according to their distances to clustering centres. Lastly, update the clustering centres by the mean of its corresponding constituent instances. This whole process will be iterated until convergence. Obviously, at each iteration, distance matrix from all instances to k clustering centres must be calculated which will cost so much time when encounter large scale data sets. To address this issue, in this paper, we proposed a fast optimization algorithm based on stochastic gradient descent (SGD). At each iteration, randomly choose an instance, search its corresponding clustering centre and then update it immediately. Experimental results show that our proposed method achieves a competitive clustering results with less time cost.

scholarly journals Parallel Implementation of Improved K-Means Based on a Cloud Platform

2019 ◽  
Vol 48 (4) ◽  
pp. 673-681
Author(s):  
Shufen Zhang ◽  
Zhiyu Liu ◽  
Xuebin Chen ◽  
Changyin Luo
Keyword(s):  
Large Scale ◽  
Clustering Algorithm ◽  
Programming Model ◽  
Parallel Implementation ◽  
Clustering Algorithms ◽  
Data Set ◽  
Large Scale Data ◽  
Sample Density ◽  
Scale Data ◽  
Selection Of

In order to solve the problem of traditional K-Means clustering algorithm in dealing with large-scale data set, a Hadoop K-Means (referred to HKM) clustering algorithm is proposed. Firstly, according to the sample density, the algorithm eliminates the effects of noise points in the data set. Secondly, it optimizes the selection of the initial center point using the thought of the max-min distance. Finally, it uses a MapReduce programming model to realize the parallelization. Experimental results show that the proposed algorithm not only has high accuracy and stability in clustering results, but can also solve the problems of scalability encountered by traditional clustering algorithms in dealing with large scale data.

Sign in / Sign up

Export Citation Format

Share Document