scholarly journals A Comparison of K-Means and Mean Shift Algorithms

2021 ◽  
Author(s):  
Mehak Nigar Shumaila
Keyword(s):  
Cluster Analysis ◽  
Data Analysis ◽  
Time Complexity ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Mean Shift ◽  
Prediction Performance ◽  
Learning Problem ◽  
Cluster A ◽  
Formation Of Groups

Clustering, or otherwise known as cluster analysis, is a learning problem that takes place without any human supervision. This technique has often been utilized, much efficiently, in data analysis, and serves for observing and identifying interesting, useful, or desired patterns in the said data. The clustering technique functions by performing a structured division of the data involved, in similar objects based on the characteristics that it identifies. This process results in the formation of groups, and each group that is formed, is called a cluster. A single said cluster consists of objects from the data, that have similarities among other objects found in the same cluster, and resemble differences when compared to objects identified from the data that now exist in other clusters. The process of clustering is very significant in various aspects of data analysis, as it determines and presents the intrinsic grouping of objects present in the data, based on their attributes, in a batch of unlabeled raw data. A textbook or otherwise said, good criteria, does not exist in this method of cluster analysis. That is because this process is so different and so customizable for every user, that needs it in his/her various and different needs. There is no outright best clustering algorithm, as it massively depends on the user’s scenario and needs. This paper is intended to compare and study two different clustering algorithms. The algorithms under investigation are k-mean and mean shift. These algorithms are compared according to the following factors: time complexity, training, prediction performance and accuracy of the clustering algorithms.

DRSA: a non-hierarchical clustering algorithm using k-NN graph and its application in vegetation classification

2015 ◽  
pp. 125-138 ◽  
Author(s):  
I. V. Goncharenko
Keyword(s):  
Cluster Analysis ◽  
Clustering Algorithm ◽  
Nearest Neighbor ◽  
Clustering Algorithms ◽  
Protein Structures ◽  
Hierarchical Cluster ◽  
Vegetation Classification ◽  
K Nearest Neighbor ◽  
Neighbor Graph ◽  
Nearest Neighbor Graph

In this article we proposed a new method of non-hierarchical cluster analysis using k-nearest-neighbor graph and discussed it with respect to vegetation classification. The method of k-nearest neighbor (k-NN) classification was originally developed in 1951 (Fix, Hodges, 1951). Later a term “k-NN graph” and a few algorithms of k-NN clustering appeared (Cover, Hart, 1967; Brito et al., 1997). In biology k-NN is used in analysis of protein structures and genome sequences. Most of k-NN clustering algorithms build «excessive» graph firstly, so called hypergraph, and then truncate it to subgraphs, just partitioning and coarsening hypergraph. We developed other strategy, the “upward” clustering in forming (assembling consequentially) one cluster after the other. Until today graph-based cluster analysis has not been considered concerning classification of vegetation datasets.

An improved OPTICS clustering algorithm for discovering clusters with uneven densities

2021 ◽  
Vol 25 (6) ◽  
pp. 1453-1471
Author(s):  
Chunhua Tang ◽  
Han Wang ◽  
Zhiwen Wang ◽  
Xiangkun Zeng ◽  
Huaran Yan ◽  
...  
Keyword(s):  
Time Complexity ◽  
Clustering Algorithm ◽  
Nearest Neighbor ◽  
Clustering Algorithms ◽  
Substantial Improvement ◽  
Experimental Results ◽  
High Time ◽  
Parameter Setting ◽  
K Nearest Neighbor ◽  
Density Based Clustering

Most density-based clustering algorithms have the problems of difficult parameter setting, high time complexity, poor noise recognition, and weak clustering for datasets with uneven density. To solve these problems, this paper proposes FOP-OPTICS algorithm (Finding of the Ordering Peaks Based on OPTICS), which is a substantial improvement of OPTICS (Ordering Points To Identify the Clustering Structure). The proposed algorithm finds the demarcation point (DP) from the Augmented Cluster-Ordering generated by OPTICS and uses the reachability-distance of DP as the radius of neighborhood eps of its corresponding cluster. It overcomes the weakness of most algorithms in clustering datasets with uneven densities. By computing the distance of the k-nearest neighbor of each point, it reduces the time complexity of OPTICS; by calculating density-mutation points within the clusters, it can efficiently recognize noise. The experimental results show that FOP-OPTICS has the lowest time complexity, and outperforms other algorithms in parameter setting and noise recognition.

A Data Distribution View of Clustering Algorithms

2011 ◽  
pp. 374-381 ◽  
Author(s):  
Junjie Wu ◽  
Jian Chen ◽  
Hui Xiong
Keyword(s):  
Data Mining ◽  
Cluster Analysis ◽  
Clustering Analysis ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Data Distribution ◽  
Point Of View ◽  
Group Method ◽  
Data Sets ◽  
Distribution Point

Cluster analysis (Jain & Dubes, 1988) provides insight into the data by dividing the objects into groups (clusters), such that objects in a cluster are more similar to each other than objects in other clusters. Cluster analysis has long played an important role in a wide variety of fields, such as psychology, bioinformatics, pattern recognition, information retrieval, machine learning, and data mining. Many clustering algorithms, such as K-means and Unweighted Pair Group Method with Arithmetic Mean (UPGMA), have been wellestablished. A recent research focus on clustering analysis is to understand the strength and weakness of various clustering algorithms with respect to data factors. Indeed, people have identified some data characteristics that may strongly affect clustering analysis including high dimensionality and sparseness, the large size, noise, types of attributes and data sets, and scales of attributes (Tan, Steinbach, & Kumar, 2005). However, further investigation is expected to reveal whether and how the data distributions can have the impact on the performance of clustering algorithms. Along this line, we study clustering algorithms by answering three questions: 1. What are the systematic differences between the distributions of the resultant clusters by different clustering algorithms? 2. How can the distribution of the “true” cluster sizes make impact on the performances of clustering algorithms? 3. How to choose an appropriate clustering algorithm in practice? The answers to these questions can guide us for the better understanding and the use of clustering methods. This is noteworthy, since 1) in theory, people seldom realized that there are strong relationships between the clustering algorithms and the cluster size distributions, and 2) in practice, how to choose an appropriate clustering algorithm is still a challenging task, especially after an algorithm boom in data mining area. This chapter thus tries to fill this void initially. To this end, we carefully select two widely used categories of clustering algorithms, i.e., K-means and Agglomerative Hierarchical Clustering (AHC), as the representative algorithms for illustration. In the chapter, we first show that K-means tends to generate the clusters with a relatively uniform distribution on the cluster sizes. Then we demonstrate that UPGMA, one of the robust AHC methods, acts in an opposite way to K-means; that is, UPGMA tends to generate the clusters with high variation on the cluster sizes. Indeed, the experimental results indicate that the variations of the resultant cluster sizes by K-means and UPGMA, measured by the Coefficient of Variation (CV), are in the specific intervals, say [0.3, 1.0] and [1.0, 2.5] respectively. Finally, we put together K-means and UPGMA for a further comparison, and propose some rules for the better choice of the clustering schemes from the data distribution point of view.

scholarly journals Software implementation of the main cluster analysis tools

2021 ◽  
Vol 10 (47) ◽  
pp. 81-92
Author(s):  
Andrey V. Silin ◽  
Olga N. Grinyuk ◽  
Tatyana A. Lartseva ◽  
Olga V. Aleksashina ◽  
Tatiana S. Sukhova
Keyword(s):  
Cluster Analysis ◽  
Data Analysis ◽  
Nearest Neighbor ◽  
Clustering Algorithms ◽  
Point Of View ◽  
Software Implementation ◽  
Practical Significance ◽  
Data Set ◽  
Main Cluster ◽  
Analysis Tools

This article discusses an approach to creating a complex of programs for the implementation of cluster analysis methods. A number of cluster analysis tools for processing the initial data set and their software implementation are analyzed, as well as the complexity of the application of cluster data analysis. An approach to data is generalized from the point of view of factual material that supplies information for the problem under study and is the basis for discussion, analysis and decision-making. Cluster analysis is a procedure that combines objects or variables into groups based on a given rule. The work provides a grouping of multivariate data using proximity measures such as sample correlation coefficient and its module, cosine of the angle between vectors and Euclidean distance. The authors proposed a method for grouping by centers, by the nearest neighbor and by selected standards. The results can be used by analysts in the process of creating a data analysis structure and will improve the efficiency of clustering algorithms. The practical significance of the results of the application of the developed algorithms is expressed in the software package created by means of the C ++ language in the VS environment.

scholarly journals Design of an Unsupervised Machine Learning-Based Movie Recommender System

2020 ◽  
Author(s):  
Debby Cintia Ganesha Putri ◽  
Jenq-Shiou Leu ◽  
Pavel Seda
Keyword(s):  
Recommender System ◽  
Clustering Algorithm ◽  
System Development ◽  
Clustering Algorithms ◽  
Mean Shift ◽  
Computational Time ◽  
Agglomerative Clustering ◽  
Method Performance ◽  
Cluster Validity Indices ◽  
Validity Indices

This research aims to determine the similarities in groups of people to build a film recommender system for users. Users often have difficulty in finding suitable movies due to the increasing amount of movie information. The recommender system is very useful for helping customers choose a preferred movie with the existing features. In this study, the recommender system development is established by using several algorithms to obtain groupings, such as the K-Means algorithm, birch algorithm, mini-batch K-Means algorithm, mean-shift algorithm, affinity propagation algorithm, agglomerative clustering algorithm, and spectral clustering algorithm. We propose methods optimizing K so that each cluster may not significantly increase variance. We are limited to using groupings based on Genre and, Tags for movies. This research can discover better methods for evaluating clustering algorithms. To verify the quality of the recommender system, we adopted the mean square error (MSE), such as the Dunn Matrix and Cluster Validity Indices, and social network analysis (SNA), such as Degree Centrality, Closeness Centrality, and Betweenness Centrality. We also used Average Similarity, Computational Time, Association Rule with Apriori algorithm, and Clustering Performance Evaluation as evaluation measures to compare method performance of recommender systems using Silhouette Coefficient, Calinski-Harabaz Index, and Davies-Bouldin Index.

scholarly journals Adaptive Initialization Method for K-Means Algorithm

2021 ◽  
Vol 4 ◽  
Author(s):  
Jie Yang ◽  
Yu-Kai Wang ◽  
Xin Yao ◽  
Chin-Teng Lin
Keyword(s):  
Time Complexity ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Real Life ◽  
Superior Performance ◽  
Local Optima ◽  
Initial Cluster ◽  
Higher Dimensional ◽  
Real World Datasets ◽  
Random Method

The K-means algorithm is a widely used clustering algorithm that offers simplicity and efficiency. However, the traditional K-means algorithm uses a random method to determine the initial cluster centers, which make clustering results prone to local optima and then result in worse clustering performance. In this research, we propose an adaptive initialization method for the K-means algorithm (AIMK) which can adapt to the various characteristics in different datasets and obtain better clustering performance with stable results. For larger or higher-dimensional datasets, we even leverage random sampling in AIMK (name as AIMK-RS) to reduce the time complexity. 22 real-world datasets were applied for performance comparisons. The experimental results show AIMK and AIMK-RS outperform the current initialization methods and several well-known clustering algorithms. Specifically, AIMK-RS can significantly reduce the time complexity to O (n). Moreover, we exploit AIMK to initialize K-medoids and spectral clustering, and better performance is also explored. The above results demonstrate superior performance and good scalability by AIMK or AIMK-RS. In the future, we would like to apply AIMK to more partition-based clustering algorithms to solve real-life practical problems.

scholarly journals Comparison of Clustering Algorithms on Air Quality Substances in Peninsular Malaysia

2018 ◽  
Vol 2 (1) ◽  
pp. 36-44
Author(s):  
Sitti Sufiah Atirah Rosly ◽  
Balkiah Moktar ◽  
Muhamad Hasbullah Mohd Razali
Keyword(s):  
Cluster Analysis ◽  
Air Quality ◽  
Expectation Maximization ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Peninsular Malaysia ◽  
Monthly Data ◽  
Knowledge Analysis ◽  
Em Clustering ◽  
Monitoring Stations

Air quality is one of the most popular environmental problems in this globalization era. Air pollution is the poisonous air that comes from car emissions, smog, open burning, chemicals from factories and other particles and gases. This harmful air can give adverse effects to human health and the environment. In order to provide information which areas are better for the residents in Malaysia, cluster analysis is used to determine the areas that can be clustering together based on their a ir quality through several air quality substances. Monthly data from 37 monitoring stations in Peninsular Malaysia from the year 2013 to 2015 were used in this study. K - Means (KM) clustering algorithm, Expectation Maximization (EM) clustering algorithm and Density Based (DB) clustering algorithm have been chosen as the techniques to analyze the cluster analysis by utilizing the Waikato Environment for Knowledge Analysis (WEKA) tools. Results show that K - means clustering algorithm is the best method among ot her algorithms due to its simplicity and time taken to build the model. The output of K - means clustering algorithm shows that it can cluster the area into two clusters, namely as cluster 0 and cluster 1. Clusters 0 consist of 16 monitoring stations and clu ster 1 consists of 36 monitoring stations in Peninsular Malaysia.

scholarly journals Fuzzy clustering and fuzzy c-means partition cluster analysis and validation studies on a subset of citescore dataset

2019 ◽  
Vol 9 (4) ◽  
pp. 2760
Author(s):  
K. Varada Rajkumar ◽  
Adimulam Yesubabu ◽  
K. Subrahmanyam
Keyword(s):  
Cluster Analysis ◽  
Fuzzy Clustering ◽  
Time Complexity ◽  
Clustering Algorithm ◽  
Fuzzy Cluster ◽  
Fuzzy Cluster Analysis ◽  
Fuzzy C Means ◽  
A Value ◽  
Data Points ◽  
Partition Clustering

A hard partition clustering algorithm assigns equally distant points to one of the clusters, where each datum has the probability to appear in simultaneous assignment to further clusters. The fuzzy cluster analysis assigns membership coefficients of data points which are equidistant between two clusters so the information directs have a place toward in excess of one cluster in the meantime. For a subset of CiteScore dataset, fuzzy clustering (fanny) and fuzzy c-means (fcm) algorithms were implemented to study the data points that lie equally distant from each other. Before analysis, clusterability of the dataset was evaluated with Hopkins statistic which resulted in 0.4371, a value &lt; 0.5, indicating that the data is highly clusterable. The optimal clusters were determined using NbClust package, where it is evidenced that 9 various indices proposed 3 cluster solutions as best clusters. Further, appropriate value of fuzziness parameter <em>m</em> was evaluated to determine the distribution of membership values with variation in <em>m</em> from 1 to 2. Coefficient of variation (CV), also known as relative variability was evaluated to study the spread of data. The time complexity of fuzzy clustering (fanny) and fuzzy c-means algorithms were evaluated by keeping data points constant and varying number of clusters.

scholarly journals Data Analysis Using Representation Theory and Clustering Algorithms

2021 ◽  
Vol 19 ◽  
pp. 310-320
Author(s):  
Suboh Alkhushayni ◽  
Taeyoung Choi ◽  
Du’a Alzaleq
Keyword(s):  
Data Analysis ◽  
Random Forest ◽  
Hierarchical Clustering ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Optimal Number ◽  
Categorical Variables ◽  
Common Disease ◽  
Agglomerative Hierarchical Clustering ◽  
Data Set

This work aims to expand the knowledge of the area of data analysis through both persistence homology, as well as representations of directed graphs. To be specific, we looked for how we can analyze homology cluster groups using agglomerative Hierarchical Clustering algorithms and methods. Additionally, the Wine data, which is offered in R studio, was analyzed using various cluster algorithms such as Hierarchical Clustering, K-Means Clustering, and PAM Clustering. The goal of the analysis was to find out which cluster's method is proper for a given numerical data set. By testing the data, we tried to find the agglomerative hierarchical clustering method that will be the optimal clustering algorithm among these three; K-Means, PAM, and Random Forest methods. By comparing each model's accuracy value with cultivar coefficients, we came with a conclusion that K-Means methods are the most helpful when working with numerical variables. On the other hand, PAM clustering and Gower with random forest are the most beneficial approaches when working with categorical variables. All these tests can determine the optimal number of clustering groups, given the data set, and by doing the proper analysis. Using those the project, we can apply our method to several industrial areas such that clinical, business, and others. For example, people can make different groups based on each patient who has a common disease, required therapy, and other things in the clinical society. Additionally, for the business area, people can expect to get several clustered groups based on the marginal profit, marginal cost, or other economic indicators.

Multi-Attribute Utility Theory Based K-Means Clustering Applications

2017 ◽  
Vol 13 (2) ◽  
pp. 1-12 ◽  
Author(s):  
Jungmok Ma
Keyword(s):  
Cluster Analysis ◽  
Utility Theory ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
User Preferences ◽  
Number Of Clusters ◽  
Clustering Problem ◽  
Multi Attribute Utility Theory ◽  
Systematic Framework ◽  
Selection Of

One of major obstacles in the application of the k-means clustering algorithm is the selection of the number of clusters k. The multi-attribute utility theory (MAUT)-based k-means clustering algorithm is proposed to tackle the problem by incorporating user preferences. Using MAUT, the decision maker's value structure for the number of clusters and other attributes can be quantitatively modeled, and it can be used as an objective function of the k-means. A target clustering problem for military targeting process is used to demonstrate the MAUT-based k-means and provide a comparative study. The result shows that the existing clustering algorithms do not necessarily reflect user preferences while the MAUT-based k-means provides a systematic framework of preferences modeling in cluster analysis.

Sign in / Sign up

Export Citation Format

Share Document