scholarly journals A dynamic K-means clustering for data mining

2019 ◽  
Vol 13 (2) ◽  
pp. 521
Author(s):  
Md. Zakir Hossain ◽  
Md.Nasim Akhtar ◽  
R.B. Ahmad ◽  
Mostafijur Rahman
Keyword(s):  
Data Mining ◽  
Clustering Algorithm ◽  
Large Data ◽  
Threshold Value ◽  
Specific Pattern ◽  
Large Data Sets ◽  
Data Sets ◽  
Data Set ◽  
Number Of Clusters ◽  
Data Points

<span>Data mining is the process of finding structure of data from large data sets. With this process, the decision makers can make a particular decision for further development of the real-world problems. Several data clusteringtechniques are used in data mining for finding a specific pattern of data. The K-means method isone of the familiar clustering techniques for clustering large data sets.  The K-means clustering method partitions the data set based on the assumption that the number of clusters are fixed.The main problem of this method is that if the number of clusters is to be chosen small then there is a higher probability of adding dissimilar items into the same group. On the other hand, if the number of clusters is chosen to be high, then there is a higher chance of adding similar items in the different groups. In this paper, we address this issue by proposing a new K-Means clustering algorithm. The proposed method performs data clustering dynamically. The proposed method initially calculates a threshold value as a centroid of K-Means and based on this value the number of clusters are formed. At each iteration of K-Means, if the Euclidian distance between two points is less than or equal to the threshold value, then these two data points will be in the same group. Otherwise, the proposed method will create a new cluster with the dissimilar data point. The results show that the proposed method outperforms the original K-Means method.</span>

Summary of Affinity Propagation

2011 ◽  
Vol 268-270 ◽  
pp. 811-816
Author(s):  
Yong Zhou ◽  
Yan Xing
Keyword(s):  
Clustering Algorithm ◽  
Large Data ◽  
Large Data Sets ◽  
Affinity Propagation ◽  
Damping Factor ◽  
Data Sets ◽  
Similarity Matrix ◽  
Data Points

Affinity Propagation(AP)is a new clustering algorithm, which is based on the similarity matrix between pairs of data points and messages are exchanged between data points until clustering result emerges. It is efficient and fast , and it can solve the clustering on large data sets. But the traditional Affinity Propagation has many limitations, this paper introduces the Affinity Propagation, and analyzes in depth the advantages and limitations of it, focuses on the improvements of the algorithm — improve the similarity matrix, adjust the preference and the damping-factor, combine with other algorithms. Finally, discusses the development of Affinity Propagation.

scholarly journals DETERMINISTIC INITIALIZATION OF THE K-MEANS ALGORITHM USING HIERARCHICAL CLUSTERING

2012 ◽  
Vol 26 (07) ◽  
pp. 1250018 ◽  
Author(s):  
M. EMRE CELEBI ◽  
HASSAN A. KINGRAVI
Keyword(s):  
Clustering Algorithm ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Partitional Clustering ◽  
Highly Sensitive ◽  
Data Points ◽  
Initial Placement ◽  
Random Initialization ◽  
Common Deficiency

K-means is undoubtedly the most widely used partitional clustering algorithm. Unfortunately, due to its gradient descent nature, this algorithm is highly sensitive to the initial placement of the cluster centers. Numerous initialization methods have been proposed to address this problem. Many of these methods, however, have superlinear complexity in the number of data points, making them impractical for large data sets. On the other hand, linear methods are often random and/or order-sensitive, which renders their results unrepeatable. Recently, Su and Dy proposed two highly successful hierarchical initialization methods named Var-Part and PCA-Part that are not only linear, but also deterministic (nonrandom) and order-invariant. In this paper, we propose a discriminant analysis based approach that addresses a common deficiency of these two methods. Experiments on a large and diverse collection of data sets from the UCI machine learning repository demonstrate that Var-Part and PCA-Part are highly competitive with one of the best random initialization methods to date, i.e. k-means++, and that the proposed approach significantly improves the performance of both hierarchical methods.

From data to knowledge mining

2009 ◽  
Vol 23 (4) ◽  
pp. 427-441 ◽  
Author(s):  
Ana Cristina Bicharra Garcia ◽  
Inhauma Ferraz ◽  
Adriana S. Vivacqua
Keyword(s):  
Data Mining ◽  
Association Rules ◽  
Association Rule ◽  
Evaluation Criteria ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Data Mining Technique ◽  
Mining Technique ◽  
Data Points

AbstractMost past approaches to data mining have been based on association rules. However, the simple application of association rules usually only changes the user's problem from dealing with millions of data points to dealing with thousands of rules. Although this may somewhat reduce the scale of the problem, it is not a completely satisfactory solution. This paper presents a new data mining technique, called knowledge cohesion (KC), which takes into account a domain ontology and the user's interest in exploring certain data sets to extract knowledge, in the form of semantic nets, from large data sets. The KC method has been successfully applied to mine causal relations from oil platform accident reports. In a comparison with association rule techniques for the same domain, KC has shown a significant improvement in the extraction of relevant knowledge, using processing complexity and knowledge manageability as the evaluation criteria.

scholarly journals Optimal fast Johnson–Lindenstrauss embeddings for large data sets

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Stefan Bamberger ◽  
Felix Krahmer
Keyword(s):  
Fast Algorithms ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Embedding Dimension ◽  
Data Set ◽  
Optimal Dimension ◽  
Discrete Algorithms ◽  
Fast Multiplication ◽  
Data Points

AbstractJohnson–Lindenstrauss embeddings are widely used to reduce the dimension and thus the processing time of data. To reduce the total complexity, also fast algorithms for applying these embeddings are necessary. To date, such fast algorithms are only available either for a non-optimal embedding dimension or up to a certain threshold on the number of data points. We address a variant of this problem where one aims to simultaneously embed larger subsets of the data set. Our method follows an approach by Nelson et al. (New constructions of RIP matrices with fast multiplication and fewer rows. In: Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1515-1528, 2014): a subsampled Hadamard transform maps points into a space of lower, but not optimal dimension. Subsequently, a random matrix with independent entries projects to an optimal embedding dimension. For subsets whose size scales at least polynomially in the ambient dimension, the complexity of this method comes close to the number of operations just to read the data under mild assumptions on the size of the data set that are considerably less restrictive than in previous works. We also prove a lower bound showing that subsampled Hadamard matrices alone cannot reach an optimal embedding dimension. Hence, the second embedding cannot be omitted.

scholarly journals Spatial Modification in the Parameters of Mountain Image Clustering Algorithm

2019 ◽  
Vol 22 (1) ◽  
pp. 55-58
Author(s):  
Nahla Ibraheem Jabbar
Keyword(s):  
Clustering Algorithm ◽  
Spatial Information ◽  
Clustering Algorithms ◽  
Large Data ◽  
Large Data Sets ◽  
Image Clustering ◽  
Optimum Number ◽  
Data Sets ◽  
Number Of Clusters ◽  
Pixel Value

Our proposed method used to overcome the drawbacks of computing values parameters in the mountain algorithm to image clustering. All existing clustering algorithms are required values of parameters to starting the clustering process such as these algorithms have a big problem in computing parameters. One of the famous clustering is a mountain algorithm that gives expected number of clusters, we presented in this paper a new modification of mountain clustering called Spatial Modification in the Parameters of Mountain Image Clustering Algorithm. This modification in the spatial information of image by taking a window mask for each center pixel value to compute distance between pixel and neighborhood for estimation the values of parameters σ, β that gives a potential optimum number of clusters requiring in image segmentation process. Our experiments show ability the proposed algorithm in image brain segmentation with a quality in the large data sets

Weighted Fuzzy-Possibilistic C-Means Over Large Data Sets

2012 ◽  
Vol 8 (4) ◽  
pp. 82-107 ◽  
Author(s):  
Renxia Wan ◽  
Yuelin Gao ◽  
Caixia Li
Keyword(s):  
Large Data ◽  
Real Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Data Set ◽  
Data Segment ◽  
Possibilistic Clustering ◽  
Data Points ◽  
Weighted Data ◽  
Natural Classes

Up to now, several algorithms for clustering large data sets have been presented. Most clustering approaches for data sets are the crisp ones, which cannot be well suitable to the fuzzy case. In this paper, the authors explore a single pass approach to fuzzy possibilistic clustering over large data set. The basic idea of the proposed approach (weighted fuzzy-possibilistic c-means, WFPCM) is to use a modified possibilistic c-means (PCM) algorithm to cluster the weighted data points and centroids with one data segment as a unit. Experimental results on both synthetic and real data sets show that WFPCM can save significant memory usage when comparing with the fuzzy c-means (FCM) algorithm and the possibilistic c-means (PCM) algorithm. Furthermore, the proposed algorithm is of an excellent immunity to noise and can avoid splitting or merging the exact clusters into some inaccurate clusters, and ensures the integrity and purity of the natural classes.

scholarly journals Data Mining on 5G Technology IOT

2019 ◽  
Vol 8 (05) ◽  
pp. 24655-24660
Author(s):  
Kranthi K Lammatha
Keyword(s):  
Data Mining ◽  
Mobile Networks ◽  
Health Sector ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Telecommunication Sector ◽  
Data Set ◽  
5G Network ◽  
Mining Scheme

Data Mining on 5G Technology IOT Currently, data mining is regarded as one of the essential factors for the next generation of mobile networks. Through research and data analysis, there are expectations that complexity of these networks will be overcome and it will be possible to carry out dynamic management and operation activities. In order to full comprehend the particulars of 5G network, there are certain kind of information that should be gathered by network components in order to be analyzed by a data mining scheme. The recent years have seen a tremendous effort put in the course of designing the 5th Generation of mobile networks (5G). The innovation of 5G mobile networks have been aimed at providing tailor cut solution for different kinds of industries particularly the telecommunication sector, intelligent transportation industries, health sector and even in smart factories. On the other hand, the scientific community has realized that big data solutions can significantly enhance the operation and management of both current and future mobile networks. Usually, data mining is employed in the course of discovering patterns and relationships between different variables particularly in large data sets. Through the use of statistical analysis, machine learning and artificial intelligence are used in the data set in the course of extracting necessary knowledge from the examined data. Data mining is integral in 5G technology because it is through data mining that 5G is considered different particularly through the ease in decision making process that has been offered by the system in order to mitigate some common challenges through a dynamic and proactive mechanism.

RECENT RESULTS IN HIERARCHICAL CLUSTERING: I–THE REDUCIBLE NEIGHBORHOODS CLUSTERING ALGORITHM

1993 ◽  
Vol 07 (03) ◽  
pp. 541-571 ◽  
Author(s):  
MICHEL BRUYNOOGHE
Keyword(s):  
Hierarchical Clustering ◽  
Speech Processing ◽  
Clustering Algorithm ◽  
Large Data ◽  
Original Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Data Set ◽  
Hierarchical Clustering Algorithm ◽  
Better Than

The clustering of large data sets is of great interest in fields such as pattern recognition, numerical taxonomy, image or speech processing. The traditional Ascendant Hierarchical Algorithm (AHC) cannot be run for sets of more than a few thousand elements. The reducible neighborhoods clustering algorithm, which is presented in this paper, has overtaken the limits of the traditional hierarchical clustering algorithm by generating an exact hierarchy on a large data set. The theoretical justification of this algorithm is the so-called Bruynooghe reducibility principle, that lays down the condition under which the exact hierarchy may be constructed locally, by carrying out aggregations in restricted regions of the representation space. As for the Day and Edelsbrunner algorithm, the maximum theoretical time complexity of the reducible neighborhoods clustering algorithm is O(n2 log n), regardless of the chosen clustering strategy. But the reducible neighborhoods clustering algorithm uses the original data table and its practical performances are by far better than Day and Edelsbrunner’s algorithm, thus allowing the hierarchical clustering of large data sets, i.e. composed of more than 10 000 objects.

scholarly journals Generation of geometric interpolations of building types with deep variational autoencoders

2020 ◽  
Vol 6 ◽  
Author(s):  
Jaime de Miguel Rodríguez ◽  
Maria Eugenia Villafañe ◽  
Luka Piškorec ◽  
Fernando Sancho Caparrini
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Large Data ◽  
Learning Model ◽  
Large Data Sets ◽  
Data Sets ◽  
Connectivity Map ◽  
Data Set ◽  
3D Objects ◽  
Machine Learning Model

Abstract This work presents a methodology for the generation of novel 3D objects resembling wireframes of building types. These result from the reconstruction of interpolated locations within the learnt distribution of variational autoencoders (VAEs), a deep generative machine learning model based on neural networks. The data set used features a scheme for geometry representation based on a ‘connectivity map’ that is especially suited to express the wireframe objects that compose it. Additionally, the input samples are generated through ‘parametric augmentation’, a strategy proposed in this study that creates coherent variations among data by enabling a set of parameters to alter representative features on a given building type. In the experiments that are described in this paper, more than 150 k input samples belonging to two building types have been processed during the training of a VAE model. The main contribution of this paper has been to explore parametric augmentation for the generation of large data sets of 3D geometries, showcasing its problems and limitations in the context of neural networks and VAEs. Results show that the generation of interpolated hybrid geometries is a challenging task. Despite the difficulty of the endeavour, promising advances are presented.

Sign in / Sign up

Export Citation Format

Share Document